2024-04-26T21:28:04.072 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T21:28:04.072 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage} --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223 -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/orig.config.yaml 2024-04-26T21:28:04.123 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:28:04.212 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage} email: null first_in_suite: false job_id: '7675223' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only - with standby daemon mds - MDS abort because newly corrupt dentry sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMceLGwR35dAZSSudtYSHL1ybAdwyYRVEjpS+mgdXkrHj1ytWWlf1r5NlLNCdKl6azqDvCeKUHgGsxQ5Rl4Hcdo= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKan2IGcost+zW6VIL4s1cJg9V+X7IhGyedKAosCI9IoPfXg7p3VYVauwE6r40KLfk+kRyVzPAc+e8xX5KLS3D0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T21:28:04.212 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T21:28:04.213 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T21:28:04.213 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T21:28:04.215 INFO:teuthology.task.internal:Checking packages... 2024-04-26T21:28:04.237 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T21:28:04.237 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T21:28:04.237 INFO:teuthology.packaging:ref: None 2024-04-26T21:28:04.237 INFO:teuthology.packaging:tag: None 2024-04-26T21:28:04.238 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T21:28:04.238 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:28:04.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T21:28:04.385 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T21:28:04.386 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T21:28:04.394 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T21:28:04.394 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T21:28:04.445 INFO:teuthology.task.internal:Saving configuration 2024-04-26T21:28:04.468 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T21:28:04.479 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:28:04.503 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:23:11.203167', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMceLGwR35dAZSSudtYSHL1ybAdwyYRVEjpS+mgdXkrHj1ytWWlf1r5NlLNCdKl6azqDvCeKUHgGsxQ5Rl4Hcdo='} 2024-04-26T21:28:04.519 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:23:11.200175', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKan2IGcost+zW6VIL4s1cJg9V+X7IhGyedKAosCI9IoPfXg7p3VYVauwE6r40KLfk+kRyVzPAc+e8xX5KLS3D0='} 2024-04-26T21:28:04.519 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T21:28:04.532 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T21:28:04.532 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T21:28:04.532 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T21:28:04.599 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5f92800160>, signals=[15]) 2024-04-26T21:28:04.599 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T21:28:04.607 INFO:teuthology.task.internal:Opening connections... 2024-04-26T21:28:04.607 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:28:04.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:28:04.689 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:28:04.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:28:04.762 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T21:28:04.769 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-04-26T21:28:04.787 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-04-26T21:28:04.788 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-04-26T21:28:04.846 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-04-26T21:28:04.846 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-04-26T21:28:04.846 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-04-26T21:28:04.846 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:28:04.847 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:28:04.848 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-04-26T21:28:04.865 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-04-26T21:28:04.883 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-04-26T21:28:04.883 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-04-26T21:28:04.941 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:28:04.942 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:28:04.942 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:28:04.942 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:28:04.942 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:28:04.942 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:28:04.942 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-04-26T21:28:04.965 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T21:28:04.974 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T21:28:04.982 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T21:28:04.982 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:28:04.984 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:28:05.001 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T21:28:05.009 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T21:28:05.009 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:28:05.042 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:28:05.112 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T21:28:05.164 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T21:28:05.178 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T21:28:05.178 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-26T21:28:05.178 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-26T21:28:05.179 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:28:05.179 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:28:05.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:28:05.196 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-04-26T21:28:05.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:28:05.197 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-26T21:28:05.254 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:28:05.254 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.14.0-437.el9.x86_64 2024-04-26T21:28:05.254 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2024-04-26T21:28:05.255 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:28:05.255 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-437.el9.x86_64 2024-04-26T21:28:05.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-04-26T21:28:07.541 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T21:28:07.639 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-04-26T21:28:12.317 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-04-26T21:28:14.923 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 19 MB 00:06 2024-04-26T21:28:16.667 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-04-26T21:28:18.168 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-26T21:28:19.132 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T21:28:20.164 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-26T21:28:21.024 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T21:28:22.578 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-26T21:28:25.344 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-26T21:28:27.284 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T21:28:27.284 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T21:28:27.366 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-26T21:28:27.367 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-26T21:28:27.367 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-26T21:28:27.468 DEBUG:teuthology.orchestra.run.smithi040:> echo no | sudo yum reinstall kernel || true 2024-04-26T21:28:27.675 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T21:28:27.991 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 09:28:25 PM UTC. 2024-04-26T21:28:28.121 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-26T21:28:28.129 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:28:28.129 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-26T21:28:28.129 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:28:28.129 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-26T21:28:28.129 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:28:28.129 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:28:28.130 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-26T21:28:28.130 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:28:28.130 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:28:28.130 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-26T21:28:28.130 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-26T21:28:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:Operation aborted. 2024-04-26T21:28:28.189 INFO:teuthology.orchestra.run.smithi040.stdout:Is this ok [y/N]: 2024-04-26T21:28:28.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum reinstall -y kernel || true 2024-04-26T21:28:28.709 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:28:25 PM UTC. 2024-04-26T21:28:28.829 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-26T21:28:28.830 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-26T21:28:28.831 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-26T21:28:29.105 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.6 MB/s | 255 kB 00:00 2024-04-26T21:28:29.105 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:28:29.105 INFO:teuthology.orchestra.run.smithi040.stdout:Total 927 kB/s | 255 kB 00:00 2024-04-26T21:28:29.131 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-26T21:28:29.326 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-26T21:28:29.326 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-26T21:28:29.331 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-26T21:28:29.331 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-26T21:28:29.463 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-26T21:28:29.547 INFO:teuthology.orchestra.run.smithi040.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:28:29.548 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:28:29.564 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T21:28:29.565 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T21:28:29.659 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-04-26T21:28:29.660 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-04-26T21:28:29.660 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-04-26T21:28:29.777 DEBUG:teuthology.orchestra.run.smithi184:> echo no | sudo yum reinstall kernel || true 2024-04-26T21:28:30.279 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:28:27 PM UTC. 2024-04-26T21:28:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:28:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:28:30.421 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-04-26T21:28:30.436 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:28:30.436 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-04-26T21:28:30.436 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:28:30.436 INFO:teuthology.orchestra.run.smithi184.stdout:Reinstalling: 2024-04-26T21:28:30.436 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:28:30.437 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:28:30.437 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-04-26T21:28:30.437 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:28:30.437 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:28:30.437 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 255 k 2024-04-26T21:28:30.437 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 0 2024-04-26T21:28:30.438 INFO:teuthology.orchestra.run.smithi184.stderr:Operation aborted. 2024-04-26T21:28:30.497 INFO:teuthology.orchestra.run.smithi184.stdout:Is this ok [y/N]: 2024-04-26T21:28:30.497 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum reinstall -y kernel || true 2024-04-26T21:28:30.992 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:28:27 PM UTC. 2024-04-26T21:28:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:28:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:28:31.022 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalled: 2024-04-26T21:28:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T21:28:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:28:31.022 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-26T21:28:31.109 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-04-26T21:28:31.110 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:28:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-04-26T21:28:31.110 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:28:31.110 INFO:teuthology.orchestra.run.smithi184.stdout:Reinstalling: 2024-04-26T21:28:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:28:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:28:31.111 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-04-26T21:28:31.111 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:28:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:28:31.111 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 255 k 2024-04-26T21:28:31.111 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 0 2024-04-26T21:28:31.111 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-04-26T21:28:31.203 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T21:28:31.294 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T21:28:31.294 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T21:28:31.294 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T21:28:31.294 DEBUG:teuthology.parallel:result is None 2024-04-26T21:28:31.510 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.1 MB/s | 255 kB 00:00 2024-04-26T21:28:31.511 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:28:31.511 INFO:teuthology.orchestra.run.smithi184.stdout:Total 637 kB/s | 255 kB 00:00 2024-04-26T21:28:31.527 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-04-26T21:28:31.722 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-04-26T21:28:31.722 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-04-26T21:28:31.727 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-04-26T21:28:31.728 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-04-26T21:28:31.864 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-04-26T21:28:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:28:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:28:32.705 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:28:32.706 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:28:33.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:28:33.399 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:28:33.399 INFO:teuthology.orchestra.run.smithi184.stdout:Reinstalled: 2024-04-26T21:28:33.399 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T21:28:33.400 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:28:33.400 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-04-26T21:28:33.575 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T21:28:33.621 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T21:28:33.622 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T21:28:33.622 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T21:28:33.622 DEBUG:teuthology.parallel:result is None 2024-04-26T21:28:33.622 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:28:33.630 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:28:33.630 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:28:33.633 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:28:33.649 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:28:33.656 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:28:33.667 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:28:33.667 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:28:33.691 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:28:33.724 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:28:33.730 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:28:33.731 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:28:33.761 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:28:33.792 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:28:33.794 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:28:33.806 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:28:33.809 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:28:33.811 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:28:33.819 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:28:33.819 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:28:33.853 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:28:33.879 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:28:33.891 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:28:33.891 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:28:33.921 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:28:33.936 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:28:34.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:28:34.076 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:28:34.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:28:34.202 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:28:34.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:28:34.268 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:28:34.310 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:28:34.357 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:28:34.439 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:28:34.478 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:28:34.478 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:28:34.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-04-26T21:28:34.544 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-04-26T21:28:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:28:34.618 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:28:35.102 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:28:35.115 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:28:35.115 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:28:35.132 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:28:35.147 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:28:35.148 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-04-26T21:28:35.225 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-04-26T21:28:35.227 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-04-26T21:28:35.291 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-04-26T21:28:35.293 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:28:35.293 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-04-26T21:28:35.315 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-04-26T21:28:35.316 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-04-26T21:28:35.341 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-04-26T21:28:35.342 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:28:35.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:28:35.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:28:35.382 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:28:35.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:28:35.406 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:28:35.406 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:28:35.447 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:28:35.470 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:28:35.483 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:28:35.498 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:28:35.500 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:32:04.080 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-04-26T21:32:04.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-26T21:32:04.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:32:04.158 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-26T21:32:04.237 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-26T21:32:04.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-04-26T21:32:04.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:32:04.309 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-04-26T21:32:04.344 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-04-26T21:32:04.344 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:32:04.355 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:32:04.356 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:32:04.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:32:04.358 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:32:04.359 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:32:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:32:04.399 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:32:04.412 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:32:04.423 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-04-26T21:32:04.425 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:32:04.434 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-04-26T21:32:04.448 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-04-26T21:32:04.449 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:32:04.458 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-04-26T21:32:04.463 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:32:04.470 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:32:04.483 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:32:04.531 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:32:04.535 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:32:04.535 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-26T21:32:04.535 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:32:04.535 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:32:04.535 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:32:04.535 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:32:04.535 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:32:04.538 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:32:04.538 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-04-26T21:32:04.538 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:32:04.538 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:32:04.539 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:32:04.539 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:32:04.540 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:32:04.548 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:32:04.548 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:32:04.548 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:32:04.548 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:32:04.554 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:32:04.554 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:32:04.555 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:32:04.555 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:32:04.557 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:32:04.557 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:32:04.699 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:32:04.699 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:32:04.785 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:32:04.785 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:32:04.871 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:32:04.871 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:32:04.872 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:32:04.896 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:32:04.897 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:32:04.956 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:32:04.956 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:32:04.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:32:04.961 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:32:04.982 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:32:04.982 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:32:05.033 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:32:05.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:32:05.097 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-04-26T21:32:05.098 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-04-26T21:32:05.122 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:32:05.186 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-04-26T21:32:05.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-04-26T21:32:05.323 INFO:teuthology.orchestra.run.smithi184.stdout:48 files removed 2024-04-26T21:32:05.350 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:32:05.422 INFO:teuthology.orchestra.run.smithi040.stdout:48 files removed 2024-04-26T21:32:05.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:32:06.080 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 177 kB/s | 84 kB 00:00 2024-04-26T21:32:06.081 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 233 kB/s | 84 kB 00:00 2024-04-26T21:32:06.439 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-26T21:32:06.507 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T21:32:06.815 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-26T21:32:06.933 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-26T21:32:07.482 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T21:32:08.249 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.0 MB 00:01 2024-04-26T21:32:09.249 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-26T21:32:10.267 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-26T21:32:13.794 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.2 MB 00:00 2024-04-26T21:32:15.268 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-26T21:32:16.364 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-26T21:32:20.834 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 916 kB/s | 6.2 MB 00:06 2024-04-26T21:32:21.378 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-26T21:32:22.402 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-26T21:32:23.512 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:32:23.512 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:32:23.523 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:32:23.524 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:32:23.524 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:32:23.525 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:32:23.527 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:32:23.529 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:32:23.530 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:32:23.572 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-26T21:32:23.589 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-04-26T21:32:23.598 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:32:23.599 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:32:23.600 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-04-26T21:32:23.601 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:32:23.602 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:32:23.603 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:32:23.604 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:32:23.605 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:32:23.606 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:32:23.607 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:32:23.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:32:23.609 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-04-26T21:32:23.610 INFO:teuthology.orchestra.run.smithi184.stdout:Install 148 Packages 2024-04-26T21:32:23.611 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 4 Packages 2024-04-26T21:32:23.611 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:32:23.611 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 222 M 2024-04-26T21:32:23.611 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-04-26T21:32:24.279 INFO:teuthology.orchestra.run.smithi184.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-04-26T21:32:24.496 INFO:teuthology.orchestra.run.smithi184.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.0 MB/s | 889 kB 00:00 2024-04-26T21:32:24.546 INFO:teuthology.orchestra.run.smithi184.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.8 MB/s | 145 kB 00:00 2024-04-26T21:32:24.805 INFO:teuthology.orchestra.run.smithi184.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 9.1 MB/s | 2.4 MB 00:00 2024-04-26T21:32:25.005 INFO:teuthology.orchestra.run.smithi184.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 7.6 MB/s | 1.5 MB 00:00 2024-04-26T21:32:25.539 INFO:teuthology.orchestra.run.smithi184.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.8 MB/s | 4.7 MB 00:00 2024-04-26T21:32:25.831 INFO:teuthology.orchestra.run.smithi184.stdout:(7/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 12 MB/s | 21 MB 00:01 2024-04-26T21:32:26.651 INFO:teuthology.orchestra.run.smithi184.stdout:(8/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-26T21:32:26.814 INFO:teuthology.orchestra.run.smithi184.stdout:(9/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 11 MB/s | 11 MB 00:00 2024-04-26T21:32:26.839 INFO:teuthology.orchestra.run.smithi184.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 134 kB/s | 25 kB 00:00 2024-04-26T21:32:27.015 INFO:teuthology.orchestra.run.smithi184.stdout:(11/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 11 MB/s | 17 MB 00:01 2024-04-26T21:32:27.040 INFO:teuthology.orchestra.run.smithi184.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.1 MB/s | 225 kB 00:00 2024-04-26T21:32:27.065 INFO:teuthology.orchestra.run.smithi184.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 544 kB/s | 27 kB 00:00 2024-04-26T21:32:27.099 INFO:teuthology.orchestra.run.smithi184.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 541 kB/s | 32 kB 00:00 2024-04-26T21:32:27.166 INFO:teuthology.orchestra.run.smithi184.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 7.3 MB/s | 747 kB 00:00 2024-04-26T21:32:27.191 INFO:teuthology.orchestra.run.smithi184.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.7 MB/s | 163 kB 00:00 2024-04-26T21:32:27.233 INFO:teuthology.orchestra.run.smithi184.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.9 MB/s | 127 kB 00:00 2024-04-26T21:32:27.408 INFO:teuthology.orchestra.run.smithi184.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 2.3 MB/s | 501 kB 00:00 2024-04-26T21:32:27.509 INFO:teuthology.orchestra.run.smithi184.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 451 kB/s | 45 kB 00:00 2024-04-26T21:32:27.609 INFO:teuthology.orchestra.run.smithi184.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.3 MB/s | 135 kB 00:00 2024-04-26T21:32:27.718 INFO:teuthology.orchestra.run.smithi184.stdout:(21/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 5.4 MB 00:00 2024-04-26T21:32:27.743 INFO:teuthology.orchestra.run.smithi184.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.2 MB/s | 161 kB 00:00 2024-04-26T21:32:27.793 INFO:teuthology.orchestra.run.smithi184.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 4.2 MB/s | 321 kB 00:00 2024-04-26T21:32:27.860 INFO:teuthology.orchestra.run.smithi184.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T21:32:27.885 INFO:teuthology.orchestra.run.smithi184.stdout:(25/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.0 MB/s | 298 kB 00:00 2024-04-26T21:32:27.935 INFO:teuthology.orchestra.run.smithi184.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T21:32:28.011 INFO:teuthology.orchestra.run.smithi184.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-26T21:32:28.078 INFO:teuthology.orchestra.run.smithi184.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 515 kB/s | 34 kB 00:00 2024-04-26T21:32:28.162 INFO:teuthology.orchestra.run.smithi184.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.7 MB/s | 142 kB 00:00 2024-04-26T21:32:28.537 INFO:teuthology.orchestra.run.smithi184.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 9.4 MB/s | 3.5 MB 00:00 2024-04-26T21:32:28.820 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-04-26T21:32:28.906 INFO:teuthology.orchestra.run.smithi184.stdout:(31/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-26T21:32:29.031 INFO:teuthology.orchestra.run.smithi184.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.9 MB/s | 249 kB 00:00 2024-04-26T21:32:29.132 INFO:teuthology.orchestra.run.smithi184.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 494 kB/s | 49 kB 00:00 2024-04-26T21:32:29.223 INFO:teuthology.orchestra.run.smithi184.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-26T21:32:29.240 INFO:teuthology.orchestra.run.smithi184.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 152 kB/s | 16 kB 00:00 2024-04-26T21:32:29.299 INFO:teuthology.orchestra.run.smithi184.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.6 MB/s | 273 kB 00:00 2024-04-26T21:32:29.366 INFO:teuthology.orchestra.run.smithi184.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 274 kB/s | 18 kB 00:00 2024-04-26T21:32:29.458 INFO:teuthology.orchestra.run.smithi184.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 3.4 MB/s | 762 kB 00:00 2024-04-26T21:32:29.500 INFO:teuthology.orchestra.run.smithi184.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 600 kB/s | 80 kB 00:00 2024-04-26T21:32:29.951 INFO:teuthology.orchestra.run.smithi184.stdout:(40/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 16 MB/s | 49 MB 00:03 2024-04-26T21:32:29.976 INFO:teuthology.orchestra.run.smithi184.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 411 kB/s | 195 kB 00:00 2024-04-26T21:32:30.109 INFO:teuthology.orchestra.run.smithi184.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-26T21:32:30.135 INFO:teuthology.orchestra.run.smithi184.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-26T21:32:30.168 INFO:teuthology.orchestra.run.smithi184.stdout:(44/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-26T21:32:30.193 INFO:teuthology.orchestra.run.smithi184.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-26T21:32:30.211 INFO:teuthology.orchestra.run.smithi184.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 135 kB 00:00 2024-04-26T21:32:30.236 INFO:teuthology.orchestra.run.smithi184.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-26T21:32:30.261 INFO:teuthology.orchestra.run.smithi184.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-26T21:32:30.945 INFO:teuthology.orchestra.run.smithi184.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 156 kB/s | 107 kB 00:00 2024-04-26T21:32:31.022 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:32:31.022 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:32:31.034 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:32:31.034 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:32:31.035 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:32:31.036 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:32:31.038 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:32:31.040 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:32:31.041 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:32:31.107 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-26T21:32:31.116 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-26T21:32:31.116 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:32:31.117 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:32:31.118 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:32:31.119 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:32:31.120 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:32:31.121 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:32:31.122 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:32:31.123 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:32:31.124 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:32:31.125 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:32:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:32:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:32:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:32:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:32:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:32:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:32:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:32:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:32:31.127 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:32:31.128 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout:Install 148 Packages 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 4 Packages 2024-04-26T21:32:31.129 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:32:31.130 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 222 M 2024-04-26T21:32:31.130 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-26T21:32:31.470 INFO:teuthology.orchestra.run.smithi184.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 41 kB/s | 21 kB 00:00 2024-04-26T21:32:31.512 INFO:teuthology.orchestra.run.smithi184.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 530 kB/s | 689 kB 00:01 2024-04-26T21:32:31.879 INFO:teuthology.orchestra.run.smithi184.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 574 kB/s | 943 kB 00:01 2024-04-26T21:32:32.063 INFO:teuthology.orchestra.run.smithi040.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-26T21:32:32.070 INFO:teuthology.orchestra.run.smithi184.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-04-26T21:32:32.362 INFO:teuthology.orchestra.run.smithi184.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 351 kB/s | 313 kB 00:00 2024-04-26T21:32:32.388 INFO:teuthology.orchestra.run.smithi040.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.7 MB/s | 889 kB 00:00 2024-04-26T21:32:32.464 INFO:teuthology.orchestra.run.smithi040.stdout:(3/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 8.4 MB/s | 5.4 MB 00:00 2024-04-26T21:32:32.489 INFO:teuthology.orchestra.run.smithi040.stdout:(4/152): ceph-immutable-object-cache-19.0.0-330 1.4 MB/s | 145 kB 00:00 2024-04-26T21:32:32.596 INFO:teuthology.orchestra.run.smithi184.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-04-26T21:32:32.681 INFO:teuthology.orchestra.run.smithi040.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 2.4 MB 00:00 2024-04-26T21:32:32.839 INFO:teuthology.orchestra.run.smithi040.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 4.3 MB/s | 1.5 MB 00:00 2024-04-26T21:32:32.888 INFO:teuthology.orchestra.run.smithi184.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-04-26T21:32:33.048 INFO:teuthology.orchestra.run.smithi040.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-26T21:32:33.580 INFO:teuthology.orchestra.run.smithi184.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 167 kB/s | 160 kB 00:00 2024-04-26T21:32:33.663 INFO:teuthology.orchestra.run.smithi184.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 58 kB/s | 45 kB 00:00 2024-04-26T21:32:33.790 INFO:teuthology.orchestra.run.smithi040.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 14 MB/s | 11 MB 00:00 2024-04-26T21:32:33.848 INFO:teuthology.orchestra.run.smithi040.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 432 kB/s | 25 kB 00:00 2024-04-26T21:32:35.022 INFO:teuthology.orchestra.run.smithi184.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 429 kB/s | 582 kB 00:01 2024-04-26T21:32:35.081 INFO:teuthology.orchestra.run.smithi184.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 441 kB/s | 662 kB 00:01 2024-04-26T21:32:35.240 INFO:teuthology.orchestra.run.smithi040.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 6.2 MB/s | 21 MB 00:03 2024-04-26T21:32:35.432 INFO:teuthology.orchestra.run.smithi040.stdout:(11/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 6.5 MB/s | 17 MB 00:02 2024-04-26T21:32:35.457 INFO:teuthology.orchestra.run.smithi040.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.0 MB/s | 225 kB 00:00 2024-04-26T21:32:35.499 INFO:teuthology.orchestra.run.smithi040.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 405 kB/s | 27 kB 00:00 2024-04-26T21:32:35.533 INFO:teuthology.orchestra.run.smithi040.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 420 kB/s | 32 kB 00:00 2024-04-26T21:32:35.564 INFO:teuthology.orchestra.run.smithi184.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 47 kB/s | 25 kB 00:00 2024-04-26T21:32:35.625 INFO:teuthology.orchestra.run.smithi040.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.7 MB/s | 163 kB 00:00 2024-04-26T21:32:35.658 INFO:teuthology.orchestra.run.smithi040.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.6 MB/s | 747 kB 00:00 2024-04-26T21:32:35.673 INFO:teuthology.orchestra.run.smithi184.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-04-26T21:32:35.700 INFO:teuthology.orchestra.run.smithi040.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.6 MB/s | 127 kB 00:00 2024-04-26T21:32:35.751 INFO:teuthology.orchestra.run.smithi040.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 5.3 MB/s | 501 kB 00:00 2024-04-26T21:32:35.826 INFO:teuthology.orchestra.run.smithi040.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 602 kB/s | 45 kB 00:00 2024-04-26T21:32:35.918 INFO:teuthology.orchestra.run.smithi040.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.4 MB/s | 135 kB 00:00 2024-04-26T21:32:36.010 INFO:teuthology.orchestra.run.smithi040.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.7 MB/s | 161 kB 00:00 2024-04-26T21:32:36.127 INFO:teuthology.orchestra.run.smithi040.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 321 kB 00:00 2024-04-26T21:32:36.227 INFO:teuthology.orchestra.run.smithi040.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.9 MB/s | 298 kB 00:00 2024-04-26T21:32:36.302 INFO:teuthology.orchestra.run.smithi040.stdout:(24/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 8.9 MB/s | 5.4 MB 00:00 2024-04-26T21:32:36.306 INFO:teuthology.orchestra.run.smithi184.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 691 kB/s | 3.0 MB 00:04 2024-04-26T21:32:36.328 INFO:teuthology.orchestra.run.smithi040.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 990 kB/s | 99 kB 00:00 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi040.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-26T21:32:36.448 INFO:teuthology.orchestra.run.smithi184.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 331 kB/s | 292 kB 00:00 2024-04-26T21:32:36.462 INFO:teuthology.orchestra.run.smithi040.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-26T21:32:36.473 INFO:teuthology.orchestra.run.smithi184.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 235 kB/s | 188 kB 00:00 2024-04-26T21:32:36.537 INFO:teuthology.orchestra.run.smithi040.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 457 kB/s | 34 kB 00:00 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi040.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.5 MB/s | 142 kB 00:00 2024-04-26T21:32:36.688 INFO:teuthology.orchestra.run.smithi040.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 8.6 MB/s | 3.1 MB 00:00 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi184.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 60 kB/s | 35 kB 00:00 2024-04-26T21:32:37.105 INFO:teuthology.orchestra.run.smithi040.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.4 MB/s | 3.5 MB 00:00 2024-04-26T21:32:37.539 INFO:teuthology.orchestra.run.smithi040.stdout:(32/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 13 MB/s | 49 MB 00:03 2024-04-26T21:32:37.564 INFO:teuthology.orchestra.run.smithi040.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 543 kB/s | 249 kB 00:00 2024-04-26T21:32:37.589 INFO:teuthology.orchestra.run.smithi040.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 988 kB/s | 49 kB 00:00 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi040.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 281 kB/s | 16 kB 00:00 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi040.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 4.6 MB/s | 273 kB 00:00 2024-04-26T21:32:37.748 INFO:teuthology.orchestra.run.smithi040.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-04-26T21:32:37.774 INFO:teuthology.orchestra.run.smithi040.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 145 kB/s | 18 kB 00:00 2024-04-26T21:32:37.807 INFO:teuthology.orchestra.run.smithi040.stdout:(39/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 4.0 MB/s | 762 kB 00:00 2024-04-26T21:32:37.932 INFO:teuthology.orchestra.run.smithi040.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-04-26T21:32:37.999 INFO:teuthology.orchestra.run.smithi040.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-26T21:32:38.049 INFO:teuthology.orchestra.run.smithi040.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-26T21:32:38.083 INFO:teuthology.orchestra.run.smithi040.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-26T21:32:38.108 INFO:teuthology.orchestra.run.smithi040.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T21:32:38.142 INFO:teuthology.orchestra.run.smithi040.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T21:32:38.159 INFO:teuthology.orchestra.run.smithi040.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T21:32:38.205 INFO:teuthology.orchestra.run.smithi040.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.3 MB/s | 215 kB 00:00 2024-04-26T21:32:38.243 INFO:teuthology.orchestra.run.smithi040.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 5.0 MB/s | 1.2 MB 00:00 2024-04-26T21:32:38.385 INFO:teuthology.orchestra.run.smithi040.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 756 kB/s | 107 kB 00:00 2024-04-26T21:32:38.435 INFO:teuthology.orchestra.run.smithi040.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-26T21:32:38.460 INFO:teuthology.orchestra.run.smithi040.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-04-26T21:32:38.502 INFO:teuthology.orchestra.run.smithi040.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-04-26T21:32:38.527 INFO:teuthology.orchestra.run.smithi040.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-04-26T21:32:38.553 INFO:teuthology.orchestra.run.smithi040.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-26T21:32:38.578 INFO:teuthology.orchestra.run.smithi040.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-04-26T21:32:38.603 INFO:teuthology.orchestra.run.smithi040.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-04-26T21:32:38.628 INFO:teuthology.orchestra.run.smithi040.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T21:32:38.654 INFO:teuthology.orchestra.run.smithi040.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-04-26T21:32:38.779 INFO:teuthology.orchestra.run.smithi040.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-26T21:32:38.821 INFO:teuthology.orchestra.run.smithi040.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 609 kB/s | 25 kB 00:00 2024-04-26T21:32:38.872 INFO:teuthology.orchestra.run.smithi040.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-26T21:32:38.897 INFO:teuthology.orchestra.run.smithi040.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-04-26T21:32:38.955 INFO:teuthology.orchestra.run.smithi040.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-26T21:32:38.997 INFO:teuthology.orchestra.run.smithi040.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 844 kB/s | 35 kB 00:00 2024-04-26T21:32:39.123 INFO:teuthology.orchestra.run.smithi040.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 836 kB/s | 188 kB 00:00 2024-04-26T21:32:39.157 INFO:teuthology.orchestra.run.smithi184.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 718 kB/s | 1.6 MB 00:02 2024-04-26T21:32:39.231 INFO:teuthology.orchestra.run.smithi040.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-04-26T21:32:39.582 INFO:teuthology.orchestra.run.smithi040.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-04-26T21:32:39.790 INFO:teuthology.orchestra.run.smithi040.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.0 MB/s | 4.7 MB 00:00 2024-04-26T21:32:39.840 INFO:teuthology.orchestra.run.smithi040.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-04-26T21:32:39.857 INFO:teuthology.orchestra.run.smithi040.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-26T21:32:40.066 INFO:teuthology.orchestra.run.smithi040.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-04-26T21:32:40.116 INFO:teuthology.orchestra.run.smithi040.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-26T21:32:40.650 INFO:teuthology.orchestra.run.smithi040.stdout:(73/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-04-26T21:32:40.757 INFO:teuthology.orchestra.run.smithi184.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 570 kB/s | 911 kB 00:01 2024-04-26T21:32:40.834 INFO:teuthology.orchestra.run.smithi040.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-04-26T21:32:40.950 INFO:teuthology.orchestra.run.smithi040.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-26T21:32:41.026 INFO:teuthology.orchestra.run.smithi040.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.6 MB/s | 6.0 MB 00:00 2024-04-26T21:32:41.051 INFO:teuthology.orchestra.run.smithi040.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-04-26T21:32:41.135 INFO:teuthology.orchestra.run.smithi040.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-26T21:32:41.160 INFO:teuthology.orchestra.run.smithi040.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-04-26T21:32:41.193 INFO:teuthology.orchestra.run.smithi040.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-26T21:32:41.310 INFO:teuthology.orchestra.run.smithi040.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-26T21:32:41.394 INFO:teuthology.orchestra.run.smithi040.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-26T21:32:41.432 INFO:teuthology.orchestra.run.smithi184.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 198 kB/s | 133 kB 00:00 2024-04-26T21:32:41.444 INFO:teuthology.orchestra.run.smithi040.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T21:32:41.503 INFO:teuthology.orchestra.run.smithi040.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-04-26T21:32:41.603 INFO:teuthology.orchestra.run.smithi040.stdout:(85/152): postgresql-server-13.14-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-04-26T21:32:41.628 INFO:teuthology.orchestra.run.smithi040.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-26T21:32:41.687 INFO:teuthology.orchestra.run.smithi040.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-26T21:32:41.724 INFO:teuthology.orchestra.run.smithi184.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 863 kB/s | 4.4 MB 00:05 2024-04-26T21:32:41.737 INFO:teuthology.orchestra.run.smithi040.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T21:32:41.771 INFO:teuthology.orchestra.run.smithi040.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-04-26T21:32:41.805 INFO:teuthology.orchestra.run.smithi040.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 800 kB/s | 54 kB 00:00 2024-04-26T21:32:41.855 INFO:teuthology.orchestra.run.smithi040.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-26T21:32:41.905 INFO:teuthology.orchestra.run.smithi040.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-04-26T21:32:41.989 INFO:teuthology.orchestra.run.smithi040.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-26T21:32:42.039 INFO:teuthology.orchestra.run.smithi040.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 722 kB/s | 36 kB 00:00 2024-04-26T21:32:42.089 INFO:teuthology.orchestra.run.smithi040.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-26T21:32:42.166 INFO:teuthology.orchestra.run.smithi184.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 849 kB/s | 4.7 MB 00:05 2024-04-26T21:32:42.206 INFO:teuthology.orchestra.run.smithi040.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-26T21:32:42.257 INFO:teuthology.orchestra.run.smithi040.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T21:32:42.307 INFO:teuthology.orchestra.run.smithi040.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-26T21:32:42.374 INFO:teuthology.orchestra.run.smithi040.stdout:(99/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-04-26T21:32:42.391 INFO:teuthology.orchestra.run.smithi040.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 761 kB/s | 64 kB 00:00 2024-04-26T21:32:42.416 INFO:teuthology.orchestra.run.smithi040.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-04-26T21:32:42.492 INFO:teuthology.orchestra.run.smithi040.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-04-26T21:32:42.617 INFO:teuthology.orchestra.run.smithi040.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-04-26T21:32:42.775 INFO:teuthology.orchestra.run.smithi184.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 144 kB/s | 87 kB 00:00 2024-04-26T21:32:42.809 INFO:teuthology.orchestra.run.smithi040.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-26T21:32:42.917 INFO:teuthology.orchestra.run.smithi040.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-04-26T21:32:42.942 INFO:teuthology.orchestra.run.smithi040.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 949 kB/s | 309 kB 00:00 2024-04-26T21:32:43.076 INFO:teuthology.orchestra.run.smithi040.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2024-04-26T21:32:43.160 INFO:teuthology.orchestra.run.smithi040.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 584 kB/s | 49 kB 00:00 2024-04-26T21:32:43.235 INFO:teuthology.orchestra.run.smithi040.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-04-26T21:32:43.302 INFO:teuthology.orchestra.run.smithi040.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-26T21:32:43.344 INFO:teuthology.orchestra.run.smithi040.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T21:32:43.366 INFO:teuthology.orchestra.run.smithi184.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 630 kB/s | 1.0 MB 00:01 2024-04-26T21:32:43.503 INFO:teuthology.orchestra.run.smithi040.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-04-26T21:32:43.528 INFO:teuthology.orchestra.run.smithi040.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-26T21:32:43.587 INFO:teuthology.orchestra.run.smithi040.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2024-04-26T21:32:43.662 INFO:teuthology.orchestra.run.smithi040.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-04-26T21:32:43.679 INFO:teuthology.orchestra.run.smithi040.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 214 kB/s | 32 kB 00:00 2024-04-26T21:32:43.771 INFO:teuthology.orchestra.run.smithi040.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-26T21:32:43.798 INFO:teuthology.orchestra.run.smithi040.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.9 MB/s | 358 kB 00:00 2024-04-26T21:32:43.857 INFO:teuthology.orchestra.run.smithi040.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-26T21:32:43.890 INFO:teuthology.orchestra.run.smithi040.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-26T21:32:43.941 INFO:teuthology.orchestra.run.smithi040.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T21:32:44.107 INFO:teuthology.orchestra.run.smithi040.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-04-26T21:32:44.133 INFO:teuthology.orchestra.run.smithi040.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 754 kB/s | 144 kB 00:00 2024-04-26T21:32:44.150 INFO:teuthology.orchestra.run.smithi184.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 312 kB/s | 244 kB 00:00 2024-04-26T21:32:44.208 INFO:teuthology.orchestra.run.smithi040.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-26T21:32:44.234 INFO:teuthology.orchestra.run.smithi040.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-26T21:32:44.309 INFO:teuthology.orchestra.run.smithi040.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-26T21:32:44.334 INFO:teuthology.orchestra.run.smithi040.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-26T21:32:44.384 INFO:teuthology.orchestra.run.smithi040.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-26T21:32:44.468 INFO:teuthology.orchestra.run.smithi040.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 556 kB/s | 46 kB 00:00 2024-04-26T21:32:44.543 INFO:teuthology.orchestra.run.smithi040.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-26T21:32:44.577 INFO:teuthology.orchestra.run.smithi040.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-04-26T21:32:44.619 INFO:teuthology.orchestra.run.smithi040.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-04-26T21:32:44.677 INFO:teuthology.orchestra.run.smithi040.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-26T21:32:44.736 INFO:teuthology.orchestra.run.smithi040.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-26T21:32:44.786 INFO:teuthology.orchestra.run.smithi040.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 829 kB/s | 90 kB 00:00 2024-04-26T21:32:44.820 INFO:teuthology.orchestra.run.smithi040.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-04-26T21:32:44.862 INFO:teuthology.orchestra.run.smithi040.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-26T21:32:44.929 INFO:teuthology.orchestra.run.smithi040.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T21:32:44.933 INFO:teuthology.orchestra.run.smithi184.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 317 kB/s | 248 kB 00:00 2024-04-26T21:32:44.954 INFO:teuthology.orchestra.run.smithi040.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-26T21:32:45.021 INFO:teuthology.orchestra.run.smithi040.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-04-26T21:32:45.046 INFO:teuthology.orchestra.run.smithi040.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-26T21:32:45.146 INFO:teuthology.orchestra.run.smithi040.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-26T21:32:45.213 INFO:teuthology.orchestra.run.smithi040.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 537 kB/s | 90 kB 00:00 2024-04-26T21:32:45.289 INFO:teuthology.orchestra.run.smithi040.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-26T21:32:45.323 INFO:teuthology.orchestra.run.smithi040.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-26T21:32:45.475 INFO:teuthology.orchestra.run.smithi184.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 47 kB/s | 26 kB 00:00 2024-04-26T21:32:45.631 INFO:teuthology.orchestra.run.smithi040.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-04-26T21:32:45.657 INFO:teuthology.orchestra.run.smithi040.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 522 kB/s | 192 kB 00:00 2024-04-26T21:32:45.942 INFO:teuthology.orchestra.run.smithi184.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 41 kB/s | 19 kB 00:00 2024-04-26T21:32:46.040 INFO:teuthology.orchestra.run.smithi040.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 8.3 MB/s | 3.4 MB 00:00 2024-04-26T21:32:46.207 INFO:teuthology.orchestra.run.smithi040.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-26T21:32:46.391 INFO:teuthology.orchestra.run.smithi040.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 1.3 MB 00:00 2024-04-26T21:32:46.583 INFO:teuthology.orchestra.run.smithi040.stdout:(151/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.4 MB/s | 16 MB 00:04 2024-04-26T21:32:46.692 INFO:teuthology.orchestra.run.smithi184.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 230 kB/s | 172 kB 00:00 2024-04-26T21:32:46.766 INFO:teuthology.orchestra.run.smithi040.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 2.4 MB 00:00 2024-04-26T21:32:46.769 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:32:46.769 INFO:teuthology.orchestra.run.smithi040.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-26T21:32:47.301 INFO:teuthology.orchestra.run.smithi184.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 57 kB/s | 35 kB 00:00 2024-04-26T21:32:47.863 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-26T21:32:48.020 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-26T21:32:48.021 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-26T21:32:48.318 INFO:teuthology.orchestra.run.smithi184.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 859 kB/s | 5.8 MB 00:06 2024-04-26T21:32:49.101 INFO:teuthology.orchestra.run.smithi184.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 320 kB/s | 250 kB 00:00 2024-04-26T21:32:49.643 INFO:teuthology.orchestra.run.smithi184.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 889 kB/s | 6.0 MB 00:06 2024-04-26T21:32:49.899 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-26T21:32:49.899 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-26T21:32:49.902 INFO:teuthology.orchestra.run.smithi184.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 278 kB/s | 222 kB 00:00 2024-04-26T21:32:50.310 INFO:teuthology.orchestra.run.smithi184.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 116 kB/s | 77 kB 00:00 2024-04-26T21:32:50.569 INFO:teuthology.orchestra.run.smithi184.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 63 kB/s | 42 kB 00:00 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi184.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 296 kB/s | 269 kB 00:00 2024-04-26T21:32:51.303 INFO:teuthology.orchestra.run.smithi184.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 225 kB/s | 159 kB 00:00 2024-04-26T21:32:51.903 INFO:teuthology.orchestra.run.smithi184.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-04-26T21:32:52.086 INFO:teuthology.orchestra.run.smithi184.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 322 kB/s | 279 kB 00:00 2024-04-26T21:32:52.478 INFO:teuthology.orchestra.run.smithi184.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 93 kB/s | 54 kB 00:00 2024-04-26T21:32:53.020 INFO:teuthology.orchestra.run.smithi184.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 77 kB/s | 42 kB 00:00 2024-04-26T21:32:53.629 INFO:teuthology.orchestra.run.smithi184.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 73 kB/s | 44 kB 00:00 2024-04-26T21:32:53.696 INFO:teuthology.orchestra.run.smithi184.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 816 kB/s | 5.1 MB 00:06 2024-04-26T21:32:54.255 INFO:teuthology.orchestra.run.smithi184.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 64 kB/s | 36 kB 00:00 2024-04-26T21:32:54.463 INFO:teuthology.orchestra.run.smithi184.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 265 kB/s | 221 kB 00:00 2024-04-26T21:32:54.822 INFO:teuthology.orchestra.run.smithi184.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 84 kB/s | 47 kB 00:00 2024-04-26T21:32:55.372 INFO:teuthology.orchestra.run.smithi184.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 345 kB/s | 305 kB 00:00 2024-04-26T21:32:55.389 INFO:teuthology.orchestra.run.smithi184.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 102 kB/s | 58 kB 00:00 2024-04-26T21:32:55.997 INFO:teuthology.orchestra.run.smithi184.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 105 kB/s | 64 kB 00:00 2024-04-26T21:32:56.022 INFO:teuthology.orchestra.run.smithi184.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 144 kB/s | 94 kB 00:00 2024-04-26T21:32:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-26T21:32:56.178 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:32:56.189 INFO:teuthology.orchestra.run.smithi184.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-04-26T21:32:56.226 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:32:56.279 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:32:56.313 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:32:56.338 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:32:56.339 INFO:teuthology.orchestra.run.smithi184.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-26T21:32:56.381 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:32:56.556 INFO:teuthology.orchestra.run.smithi184.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-04-26T21:32:56.582 INFO:teuthology.orchestra.run.smithi184.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-04-26T21:32:56.632 INFO:teuthology.orchestra.run.smithi184.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-04-26T21:32:56.734 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:32:56.740 INFO:teuthology.orchestra.run.smithi184.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-04-26T21:32:56.824 INFO:teuthology.orchestra.run.smithi184.stdout:(107/152): libarrow-9.0.0-11.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-04-26T21:32:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:32:56.849 INFO:teuthology.orchestra.run.smithi184.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-26T21:32:56.875 INFO:teuthology.orchestra.run.smithi184.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 971 kB/s | 49 kB 00:00 2024-04-26T21:32:56.900 INFO:teuthology.orchestra.run.smithi184.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-26T21:32:56.925 INFO:teuthology.orchestra.run.smithi184.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-26T21:32:56.972 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:32:56.976 INFO:teuthology.orchestra.run.smithi184.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-04-26T21:32:56.998 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:32:57.009 INFO:teuthology.orchestra.run.smithi184.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-04-26T21:32:57.043 INFO:teuthology.orchestra.run.smithi184.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 645 kB/s | 43 kB 00:00 2024-04-26T21:32:57.068 INFO:teuthology.orchestra.run.smithi184.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 546 kB/s | 32 kB 00:00 2024-04-26T21:32:57.091 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:32:57.094 INFO:teuthology.orchestra.run.smithi184.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-26T21:32:57.119 INFO:teuthology.orchestra.run.smithi184.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-26T21:32:57.128 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:32:57.152 INFO:teuthology.orchestra.run.smithi184.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-26T21:32:57.178 INFO:teuthology.orchestra.run.smithi184.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-26T21:32:57.203 INFO:teuthology.orchestra.run.smithi184.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 700 kB/s | 35 kB 00:00 2024-04-26T21:32:57.229 INFO:teuthology.orchestra.run.smithi184.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-26T21:32:57.279 INFO:teuthology.orchestra.run.smithi184.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-26T21:32:57.315 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:32:57.321 INFO:teuthology.orchestra.run.smithi184.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-26T21:32:57.338 INFO:teuthology.orchestra.run.smithi184.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-26T21:32:57.363 INFO:teuthology.orchestra.run.smithi184.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-04-26T21:32:57.367 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:32:57.380 INFO:teuthology.orchestra.run.smithi184.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 552 kB/s | 23 kB 00:00 2024-04-26T21:32:57.394 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:32:57.405 INFO:teuthology.orchestra.run.smithi184.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 464 kB/s | 19 kB 00:00 2024-04-26T21:32:57.422 INFO:teuthology.orchestra.run.smithi184.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 469 kB/s | 20 kB 00:00 2024-04-26T21:32:57.465 INFO:teuthology.orchestra.run.smithi184.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-04-26T21:32:57.489 INFO:teuthology.orchestra.run.smithi184.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-04-26T21:32:57.500 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:32:57.514 INFO:teuthology.orchestra.run.smithi184.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-26T21:32:57.528 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:32:57.540 INFO:teuthology.orchestra.run.smithi184.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-26T21:32:57.565 INFO:teuthology.orchestra.run.smithi184.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-26T21:32:57.590 INFO:teuthology.orchestra.run.smithi184.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-26T21:32:57.591 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:32:57.613 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:32:57.615 INFO:teuthology.orchestra.run.smithi184.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-26T21:32:57.635 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:32:57.641 INFO:teuthology.orchestra.run.smithi184.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-26T21:32:57.666 INFO:teuthology.orchestra.run.smithi184.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-04-26T21:32:57.692 INFO:teuthology.orchestra.run.smithi184.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-26T21:32:57.717 INFO:teuthology.orchestra.run.smithi184.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-26T21:32:57.742 INFO:teuthology.orchestra.run.smithi184.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-04-26T21:32:57.768 INFO:teuthology.orchestra.run.smithi184.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-26T21:32:57.781 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:32:57.793 INFO:teuthology.orchestra.run.smithi184.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-26T21:32:57.813 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:32:57.827 INFO:teuthology.orchestra.run.smithi184.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-26T21:32:57.887 INFO:teuthology.orchestra.run.smithi184.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-26T21:32:57.911 INFO:teuthology.orchestra.run.smithi184.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-26T21:32:57.923 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:32:57.936 INFO:teuthology.orchestra.run.smithi184.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-04-26T21:32:57.978 INFO:teuthology.orchestra.run.smithi184.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-26T21:32:58.022 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:32:58.062 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:32:58.097 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:32:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:32:58.222 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:32:58.245 INFO:teuthology.orchestra.run.smithi184.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 11 MB/s | 3.4 MB 00:00 2024-04-26T21:32:58.265 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:32:58.298 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:32:58.325 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:32:58.397 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:32:58.445 INFO:teuthology.orchestra.run.smithi184.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 1.3 MB 00:00 2024-04-26T21:32:58.449 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:32:58.486 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:32:58.531 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:32:58.554 INFO:teuthology.orchestra.run.smithi184.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-26T21:32:58.609 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:32:58.650 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:32:58.670 INFO:teuthology.orchestra.run.smithi184.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-04-26T21:32:58.694 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:32:58.721 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:32:58.797 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:32:58.845 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:32:58.922 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:32:59.078 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:32:59.133 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:32:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:32:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:32:59.822 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:32:59.865 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:32:59.903 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:32:59.945 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:33:00.108 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:33:00.172 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:33:00.229 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:33:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:33:00.341 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:33:00.376 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:33:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:33:00.459 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:33:00.503 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:33:00.545 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:33:00.688 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:33:00.733 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:33:00.763 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:33:00.805 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:33:00.904 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:33:00.938 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:33:00.978 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:33:01.003 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:33:01.028 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:33:01.107 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:33:01.235 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:33:01.489 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:33:01.565 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:33:01.599 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:33:01.690 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:33:01.729 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:33:01.774 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:33:01.809 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:33:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:33:01.886 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:33:01.919 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:33:02.017 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:33:02.067 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:33:02.113 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:33:02.162 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:33:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:33:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:33:03.103 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:33:03.153 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:33:03.187 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:33:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:33:03.346 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:33:03.436 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:33:03.472 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:33:03.655 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:33:03.678 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:33:03.757 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:33:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:33:04.571 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:33:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:33:05.940 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:33:05.968 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:33:06.002 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:33:06.330 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:33:06.395 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:33:06.497 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:33:06.528 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:33:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:33:07.068 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:33:07.101 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:33:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:33:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:33:09.587 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:33:09.615 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:33:10.185 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:33:10.227 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:33:10.311 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:33:10.311 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:33:10.311 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:33:10.311 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:10.345 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:33:10.737 INFO:teuthology.orchestra.run.smithi184.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 892 kB/s | 16 MB 00:18 2024-04-26T21:33:10.741 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:33:10.741 INFO:teuthology.orchestra.run.smithi184.stdout:Total 4.7 MB/s | 222 MB 00:47 2024-04-26T21:33:11.817 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-04-26T21:33:11.958 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-04-26T21:33:11.958 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-04-26T21:33:13.722 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-04-26T21:33:13.722 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-04-26T21:33:19.731 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-04-26T21:33:19.815 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:33:19.861 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:33:19.922 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:33:19.957 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:33:19.991 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:33:20.033 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:33:20.360 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:33:20.476 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:33:20.597 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:33:20.626 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:33:20.717 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:33:20.747 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:33:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:33:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:33:20.972 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:33:21.076 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:33:21.099 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:33:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:33:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:33:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:33:21.391 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:33:21.424 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:33:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:33:21.618 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:33:21.656 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:33:21.691 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:33:21.755 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:33:21.790 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:33:21.834 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:33:21.883 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:33:21.944 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:33:22.130 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:33:22.191 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:33:22.222 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:33:22.257 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:33:22.329 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:33:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-04-26T21:33:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-04-26T21:33:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-04-26T21:33:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-04-26T21:33:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-04-26T21:33:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-04-26T21:33:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-04-26T21:33:22.330 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:22.333 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:33:22.368 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:33:22.411 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:33:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:33:22.500 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:33:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:33:22.518 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:33:22.519 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:22.519 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:33:22.519 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:22.554 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:33:22.631 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:33:22.844 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:33:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:33:23.431 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:33:23.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:33:23.540 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:33:23.575 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:33:23.613 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:33:23.655 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:33:23.814 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:33:23.883 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:33:23.937 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:33:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:33:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:33:24.044 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:33:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:33:24.072 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:33:24.072 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:24.072 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:33:24.072 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:33:24.072 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:33:24.073 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:24.116 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:33:24.152 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:33:24.195 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:33:24.238 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:33:24.303 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:33:24.327 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:33:24.327 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:24.327 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:33:24.327 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:33:24.327 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:33:24.328 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:24.373 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:33:24.427 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:33:24.457 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:33:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:33:24.600 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:33:24.632 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:33:24.663 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:33:24.689 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:33:24.714 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:33:24.788 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:33:24.820 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:33:24.844 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:33:24.844 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:24.844 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:33:24.845 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:33:24.845 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:33:24.845 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:24.915 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:33:24.955 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:33:24.960 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:33:24.983 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:33:25.017 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:33:25.018 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:33:25.056 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:33:25.082 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:33:25.088 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:33:25.117 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:33:25.143 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:33:25.177 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:33:25.218 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:33:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:33:25.280 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:33:25.331 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:33:25.364 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:33:25.408 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:33:25.412 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:33:25.440 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:33:25.486 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:33:25.510 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:33:25.593 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:33:25.649 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:33:25.654 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:33:25.697 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:33:25.737 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:33:25.764 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:33:26.212 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:33:26.238 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:33:26.271 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:33:26.344 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:33:26.360 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:33:26.378 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:33:26.405 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:33:26.438 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:33:26.536 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:33:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:33:26.686 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:33:26.724 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:33:26.900 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:33:26.939 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:33:27.016 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:33:27.477 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:33:27.513 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:33:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:33:27.762 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:33:28.528 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:33:28.555 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:33:28.663 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:33:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:33:28.857 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:33:28.881 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:33:28.882 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:28.882 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:33:28.882 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:33:28.882 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:33:28.882 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:28.933 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:33:28.944 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:33:28.975 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:33:29.046 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:33:29.084 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:33:29.105 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:33:29.112 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:33:29.161 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:33:29.473 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:33:29.509 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:33:29.599 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:33:29.630 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:33:29.666 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:33:30.072 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:33:30.102 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:33:30.126 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:33:30.126 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:30.126 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:33:30.126 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:33:30.126 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:33:30.127 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:30.143 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:33:30.169 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:33:30.170 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:33:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:33:30.192 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:30.193 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:33:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:30.237 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:33:30.241 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:33:30.260 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:33:30.260 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:30.260 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:33:30.261 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:33:30.261 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:33:30.261 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:30.263 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:33:30.579 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:33:30.603 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:33:30.603 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:30.603 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:33:30.603 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:33:30.603 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:33:30.603 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:32.527 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:33:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:33:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:33:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:33:33.218 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:33:33.218 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:33:33.218 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:33:33.219 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:33.248 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:33:35.459 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:33:35.501 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:33:35.542 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:33:35.573 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:33:35.600 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:33:35.800 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:33:35.838 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:33:35.867 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:33:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:33:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:33:36.511 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:33:37.424 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:33:37.504 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:33:37.545 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:33:37.631 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:33:37.666 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:33:37.666 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:33:37.734 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:33:37.767 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:33:37.767 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:33:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:33:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:33:43.730 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:33:43.730 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:33:43.730 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:33:43.730 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:33:43.730 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:33:43.730 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:33:43.730 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:33:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:33:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:33:43.744 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:33:43.744 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:33:43.744 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:33:43.745 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:33:43.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:33:43.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:33:43.748 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:33:43.749 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:33:43.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:33:43.751 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:33:43.752 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:33:43.753 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-04-26T21:33:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:44.860 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:33:44.875 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:33:44.875 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:44.875 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:33:44.875 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:45.869 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:33:45.870 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.871 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:33:45.872 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:33:45.873 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:33:45.874 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:33:45.875 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:33:45.876 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:33:45.877 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:33:45.878 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:33:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.881 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:33:45.881 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:33:45.881 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:33:45.881 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:33:45.881 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:33:45.881 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:33:45.881 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:33:45.881 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:33:45.882 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:33:45.882 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-26T21:33:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:33:46.345 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:33:46.345 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:46.345 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:33:46.345 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:33:46.345 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:33:46.345 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:46.356 DEBUG:teuthology.parallel:result is None 2024-04-26T21:33:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:33:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:33:46.592 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:46.592 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:33:46.592 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:33:46.592 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:33:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:47.029 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:33:47.051 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:33:47.051 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:47.051 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:33:47.051 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:33:47.051 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:33:47.052 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:47.158 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:33:47.207 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:33:47.262 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:33:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:33:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:33:47.573 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:33:47.661 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:33:47.800 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:33:47.838 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:33:47.870 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:33:47.906 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:33:48.307 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:33:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:33:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:33:48.440 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:33:48.478 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:33:49.510 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:33:49.546 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:33:50.504 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:33:50.530 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:33:50.631 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:33:50.801 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:33:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:33:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:33:50.865 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:50.865 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:33:50.865 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:33:50.865 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:33:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:33:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:33:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:33:51.047 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:33:51.968 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:33:52.002 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:33:52.024 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:33:52.024 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:52.025 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:33:52.025 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:33:52.025 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:33:52.025 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:52.069 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:33:52.091 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:33:52.091 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:52.091 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:33:52.091 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:52.136 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:33:52.158 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:33:52.158 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:52.159 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:33:52.159 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:33:52.159 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:33:52.159 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:52.462 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:33:52.484 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:33:52.485 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:33:52.485 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:33:52.485 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:33:52.485 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:33:52.485 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:33:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:33:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:33:57.190 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:33:57.222 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:33:57.249 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:33:57.429 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:33:57.470 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:33:57.501 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:33:58.131 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:33:58.165 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:33:58.185 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:33:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:33:59.103 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:33:59.144 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:33:59.225 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:33:59.257 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:33:59.257 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:33:59.325 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:33:59.358 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:33:59.358 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:33:59.446 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:33:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:34:05.606 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:34:05.606 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:34:05.606 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:34:05.606 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:34:05.606 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:34:05.606 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:34:05.606 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:34:05.606 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:34:05.607 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:34:05.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:34:05.619 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:34:05.619 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:34:05.620 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:34:05.621 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:34:05.622 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:34:05.623 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:34:05.624 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:34:05.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:34:05.626 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:34:05.627 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:34:05.628 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:34:05.629 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:34:05.630 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:34:05.630 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:34:05.630 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:34:05.630 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:34:05.630 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:34:05.630 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:34:05.630 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:34:05.630 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:34:07.656 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:34:07.657 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.658 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:34:07.659 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.660 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:34:07.661 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:34:07.662 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:34:07.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:34:07.664 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:34:07.665 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:34:07.666 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:34:07.667 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:34:07.667 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:34:07.667 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.667 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.667 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:34:07.667 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:34:07.667 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:34:07.668 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:34:07.669 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:34:07.670 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:34:07.670 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:34:07.670 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:34:07.670 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:07.670 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-04-26T21:34:08.037 DEBUG:teuthology.parallel:result is None 2024-04-26T21:34:08.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:34:08.310 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:34:08.337 INFO:teuthology.orchestra.run.smithi040.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:34:08.338 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:34:08.338 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:34:08.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:34:08.481 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:34:08.502 INFO:teuthology.orchestra.run.smithi184.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:34:08.503 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:34:08.503 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:34:08.504 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:34:08.505 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:08.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:34:08.536 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:08.536 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:34:08.568 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:34:08.568 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:08.569 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:34:08.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:34:08.671 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:08.672 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:34:08.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:34:08.759 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:34:08.759 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:08.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:34:08.786 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:34:08.853 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:08.853 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:34:08.876 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:34:08.941 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:34:08.942 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:08.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:34:08.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:34:09.030 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:09.030 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:34:09.054 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:34:09.117 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T21:34:09.214 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T21:34:09.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-04-26T21:34:09.217 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2024-04-26T21:34:09.249 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T21:34:09.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:34:09.292 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:34:09.316 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T21:34:09.316 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:34:09.360 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:34:09.390 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T21:34:09.391 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T21:34:09.391 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T21:34:09.391 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:09.391 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:34:09.437 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:34:09.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:34:09.573 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:09.573 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:34:09.587 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:34:09.783 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:34:09.812 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T21:34:09.812 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T21:34:09.812 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage}', 'email': None, 'first_in_suite': False, 'job_id': '7675223', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMceLGwR35dAZSSudtYSHL1ybAdwyYRVEjpS+mgdXkrHj1ytWWlf1r5NlLNCdKl6azqDvCeKUHgGsxQ5Rl4Hcdo=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKan2IGcost+zW6VIL4s1cJg9V+X7IhGyedKAosCI9IoPfXg7p3VYVauwE6r40KLfk+kRyVzPAc+e8xX5KLS3D0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T21:34:09.813 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:34:09.815 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:34:09.862 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:34:09.870 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:34:09.925 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:09.925 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:34:09.947 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:34:09.947 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-04-26T21:34:10.001 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:34:10.002 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:34:10.002 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T21:34:10.002 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:34:10.002 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:34:10.002 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:33:38.297973094 +0000 2024-04-26T21:34:10.002 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:31:46.081665361 +0000 2024-04-26T21:34:10.002 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:31:46.081665361 +0000 2024-04-26T21:34:10.002 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:31:46.081665361 +0000 2024-04-26T21:34:10.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:34:10.066 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-26T21:34:10.066 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-26T21:34:10.066 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000259651 s, 2.0 MB/s 2024-04-26T21:34:10.067 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:34:10.124 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:33:38.297973094 +0000 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:31:46.423661353 +0000 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:31:46.423661353 +0000 2024-04-26T21:34:10.179 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:31:46.423661353 +0000 2024-04-26T21:34:10.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:34:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-26T21:34:10.242 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-26T21:34:10.242 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000200148 s, 2.6 MB/s 2024-04-26T21:34:10.243 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:34:10.299 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:33:38.297973094 +0000 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:31:46.771657274 +0000 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:31:46.771657274 +0000 2024-04-26T21:34:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:31:46.771657274 +0000 2024-04-26T21:34:10.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:34:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-26T21:34:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-26T21:34:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.00018273 s, 2.8 MB/s 2024-04-26T21:34:10.418 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:34:10.474 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-04-26T21:34:10.528 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:34:10.529 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:34:10.529 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T21:34:10.529 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:34:10.529 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:34:10.529 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:33:38.298973083 +0000 2024-04-26T21:34:10.529 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:31:47.125653125 +0000 2024-04-26T21:34:10.529 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:31:47.125653125 +0000 2024-04-26T21:34:10.529 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:31:47.125653125 +0000 2024-04-26T21:34:10.529 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:34:10.591 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-26T21:34:10.591 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-26T21:34:10.591 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000180371 s, 2.8 MB/s 2024-04-26T21:34:10.592 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:34:10.649 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:10.649 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:10.649 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:34:10.664 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:34:10.664 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-04-26T21:34:10.719 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:34:10.720 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:34:10.720 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T21:34:10.720 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:34:10.720 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:34:10.720 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-04-26 21:33:59.907834942 +0000 2024-04-26T21:34:10.720 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-04-26 21:31:42.136984798 +0000 2024-04-26T21:34:10.720 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-04-26 21:31:42.136984798 +0000 2024-04-26T21:34:10.720 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-04-26 21:31:42.136984798 +0000 2024-04-26T21:34:10.720 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:34:10.782 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-04-26T21:34:10.782 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-04-26T21:34:10.783 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000214858 s, 2.4 MB/s 2024-04-26T21:34:10.784 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:34:10.839 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-04-26 21:33:59.908834922 +0000 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-04-26 21:31:42.478977086 +0000 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-04-26 21:31:42.478977086 +0000 2024-04-26T21:34:10.895 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-04-26 21:31:42.478977086 +0000 2024-04-26T21:34:10.896 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:34:10.957 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-04-26T21:34:10.957 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-04-26T21:34:10.957 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000192569 s, 2.7 MB/s 2024-04-26T21:34:10.958 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:34:11.014 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-04-26 21:33:59.908834922 +0000 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-04-26 21:31:42.810969599 +0000 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-04-26 21:31:42.810969599 +0000 2024-04-26T21:34:11.069 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-04-26 21:31:42.810969599 +0000 2024-04-26T21:34:11.071 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:34:11.132 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-04-26T21:34:11.132 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-04-26T21:34:11.132 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000213552 s, 2.4 MB/s 2024-04-26T21:34:11.133 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:34:11.189 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-04-26T21:34:11.243 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:34:11.243 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:34:11.243 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T21:34:11.243 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:34:11.243 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:34:11.243 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-04-26 21:33:59.908834922 +0000 2024-04-26T21:34:11.243 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-04-26 21:31:43.141962135 +0000 2024-04-26T21:34:11.244 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-04-26 21:31:43.141962135 +0000 2024-04-26T21:34:11.244 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-04-26 21:31:43.141962135 +0000 2024-04-26T21:34:11.244 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:34:11.306 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-04-26T21:34:11.307 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-04-26T21:34:11.307 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000190598 s, 2.7 MB/s 2024-04-26T21:34:11.308 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:34:11.363 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:11.363 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:34:11.363 INFO:tasks.ceph:Generating config... 2024-04-26T21:34:11.364 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T21:34:11.364 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T21:34:11.364 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T21:34:11.364 INFO:tasks.ceph:[client] pid file = 2024-04-26T21:34:11.365 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T21:34:11.365 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T21:34:11.365 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T21:34:11.365 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T21:34:11.366 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T21:34:11.366 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T21:34:11.366 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T21:34:11.366 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T21:34:11.366 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T21:34:11.366 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T21:34:11.366 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T21:34:11.366 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T21:34:11.366 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T21:34:11.366 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T21:34:11.366 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T21:34:11.366 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T21:34:11.366 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T21:34:11.366 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T21:34:11.367 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T21:34:11.367 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:34:11.367 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T21:34:11.367 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T21:34:11.367 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T21:34:11.367 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T21:34:11.367 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T21:34:11.367 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T21:34:11.367 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T21:34:11.425 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T21:34:11.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T21:34:11.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:34:11.498 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.184'), ('mon.c', '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]')] 2024-04-26T21:34:11.499 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T21:34:11.499 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:11.499 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:34:11.555 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.184 --addv c '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:11.641 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:11.641 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid ec445c36-0c6e-44d6-884b-6999dea32f55 2024-04-26T21:34:11.641 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = quincy 2024-04-26T21:34:11.641 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:fsid ec445c36-0c6e-44d6-884b-6999dea32f55 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-04-26T21:34:11.641875+0000 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-04-26T21:34:11.641875+0000 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 17 (quincy) 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.b 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] mon.c 2024-04-26T21:34:11.642 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T21:34:11.643 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:34:11.658 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ec445c36-0c6e-44d6-884b-6999dea32f55... 2024-04-26T21:34:11.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:34:11.701 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:34:11.740 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-04-26T21:34:11.741 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:34:11.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-04-26T21:34:11.743 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on skipped message = False 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = ec445c36-0c6e-44d6-884b-6999dea32f55 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-04-26T21:34:11.744 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug trim objects = true 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.745 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2024-04-26T21:34:11.746 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-04-26T21:34:11.747 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:34:11.748 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon down mkfs grace = 300 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-04-26T21:34:11.749 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = "" 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-04-26T21:34:11.750 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 20 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-04-26T21:34:11.751 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-04-26T21:34:11.758 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2024-04-26T21:34:11.758 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2024-04-26T21:34:11.758 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:34:11.758 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2024-04-26T21:34:11.758 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.758 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:34:11.759 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.760 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.40,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on skipped message = False 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = ec445c36-0c6e-44d6-884b-6999dea32f55 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2024-04-26T21:34:11.761 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug trim objects = true 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.762 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2024-04-26T21:34:11.763 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 20 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 20 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 10 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2024-04-26T21:34:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2024-04-26T21:34:11.765 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.766 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: mon down mkfs grace = 300 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: mon op complaint time = 120 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: client mount timeout = 600 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2024-04-26T21:34:11.767 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = "" 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout:[mds] 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds = 20 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds balancer = 20 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:34:11.768 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug frag = True 2024-04-26T21:34:11.769 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug scatterstat = True 2024-04-26T21:34:11.769 INFO:teuthology.orchestra.run.smithi184.stdout: mds op complaint time = 180 2024-04-26T21:34:11.769 INFO:teuthology.orchestra.run.smithi184.stdout: mds verify scatter = True 2024-04-26T21:34:11.769 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2024-04-26T21:34:11.769 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2024-04-26T21:34:11.769 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2024-04-26T21:34:11.770 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T21:34:11.770 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T21:34:11.847 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T21:34:11.847 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:11.847 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T21:34:11.863 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:11.863 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T21:34:11.918 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:34:11.918 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:11.918 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:34:11.918 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:34:11.991 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:11.991 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:12.046 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:34:12.047 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:12.047 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:34:12.047 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:34:12.079 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:12.079 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:12.134 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T21:34:12.134 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T21:34:12.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:34:12.184 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:34:12.187 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:34:12.243 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:34:12.246 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T21:34:12.246 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:34:12.297 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:34:12.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T21:34:12.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:34:12.413 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:34:12.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T21:34:12.440 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:34:12.489 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:34:12.491 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T21:34:12.514 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:34:12.603 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:34:12.606 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T21:34:12.629 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T21:34:12.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:34:12.669 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T21:34:12.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T21:34:12.762 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T21:34:12.775 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T21:34:12.814 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T21:34:12.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T21:34:12.905 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T21:34:12.917 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T21:34:12.917 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:34:12.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T21:34:12.942 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:12.942 INFO:tasks.ceph:role: osd.0 2024-04-26T21:34:12.942 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:34:12.942 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:34:13.013 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:34:13.508 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-26T21:34:13.510 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-26T21:34:13.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T21:34:13.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T21:34:13.662 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T21:34:13.729 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:13.729 INFO:tasks.ceph:role: osd.1 2024-04-26T21:34:13.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:34:13.729 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:34:13.801 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:34:13.801 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:34:13.801 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:34:13.801 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:34:13.801 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:34:13.802 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:34:13.802 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:34:13.802 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:34:13.802 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:34:13.802 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:34:14.259 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-26T21:34:14.262 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-26T21:34:14.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T21:34:14.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T21:34:14.382 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T21:34:14.450 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:14.450 INFO:tasks.ceph:role: osd.2 2024-04-26T21:34:14.450 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:34:14.450 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:34:14.523 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:34:14.523 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:34:14.523 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:34:14.523 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:34:14.523 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:34:14.523 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:34:14.524 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:34:14.524 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:34:14.524 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:34:14.524 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:34:15.009 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-26T21:34:15.012 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-26T21:34:15.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T21:34:15.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T21:34:15.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T21:34:15.256 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:15.256 INFO:tasks.ceph:role: osd.3 2024-04-26T21:34:15.256 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:34:15.256 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:34:15.330 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:34:15.331 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:34:15.842 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-26T21:34:15.844 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-26T21:34:15.845 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T21:34:15.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T21:34:16.008 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:16.103+0000 7f001eab5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T21:34:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:16.104+0000 7f001eab5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T21:34:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:16.104+0000 7f001eab5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:34:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:16.104+0000 7f001eab5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:34:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:16.104+0000 7f001eab5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:34:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:16.104+0000 7f001eab5640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T21:34:18.954 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T21:34:18.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:19.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:19.074+0000 7f19b863a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T21:34:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:19.074+0000 7f19b863a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T21:34:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:19.074+0000 7f19b863a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:34:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:19.074+0000 7f19b863a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:34:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:19.074+0000 7f19b863a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:34:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:19.074+0000 7f19b863a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T21:34:21.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T21:34:21.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:22.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:22.043+0000 7f7f4c410640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T21:34:22.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:22.043+0000 7f7f4c410640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T21:34:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:22.043+0000 7f7f4c410640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:34:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:22.043+0000 7f7f4c410640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:34:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:22.043+0000 7f7f4c410640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:34:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:22.044+0000 7f7f4c410640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T21:34:24.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T21:34:24.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:25.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:25.018+0000 7fad59d44640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T21:34:25.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:25.018+0000 7fad59d44640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T21:34:25.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:25.018+0000 7fad59d44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:34:25.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:25.018+0000 7fad59d44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:34:25.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:25.018+0000 7fad59d44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:34:25.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:25.019+0000 7fad59d44640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T21:34:27.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T21:34:27.905 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T21:34:27.932 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:27.933 INFO:tasks.ceph:role: osd.4 2024-04-26T21:34:27.933 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:34:27.933 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:34:28.006 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:34:28.007 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:34:28.508 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-04-26T21:34:28.510 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-04-26T21:34:28.510 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T21:34:28.656 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T21:34:28.690 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T21:34:28.758 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:28.758 INFO:tasks.ceph:role: osd.5 2024-04-26T21:34:28.758 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:34:28.758 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:34:28.831 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:34:28.832 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:34:29.281 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-04-26T21:34:29.283 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-04-26T21:34:29.284 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T21:34:29.377 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T21:34:29.407 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T21:34:29.473 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:29.473 INFO:tasks.ceph:role: osd.6 2024-04-26T21:34:29.474 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:34:29.474 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:34:29.546 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:34:29.546 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:34:29.546 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:34:29.546 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:34:29.546 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:34:29.546 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:34:29.546 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:34:29.547 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:34:29.547 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:34:29.547 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:34:30.044 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-04-26T21:34:30.046 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-04-26T21:34:30.046 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T21:34:30.185 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T21:34:30.216 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T21:34:30.284 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:34:30.284 INFO:tasks.ceph:role: osd.7 2024-04-26T21:34:30.284 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:34:30.284 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:34:30.357 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:34:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:34:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:34:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:34:30.358 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:34:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:34:30.358 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:34:30.358 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:34:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:34:30.358 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:34:30.877 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-04-26T21:34:30.879 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-04-26T21:34:30.879 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T21:34:31.004 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T21:34:31.037 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:31.139 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:31.138+0000 7fe287add640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T21:34:31.140 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:31.139+0000 7fe287add640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T21:34:31.140 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:31.139+0000 7fe287add640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:34:31.140 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:31.139+0000 7fe287add640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:34:31.140 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:31.139+0000 7fe287add640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:34:31.140 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:31.139+0000 7fe287add640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T21:34:33.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T21:34:34.019 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:34.109+0000 7f3dd7fc4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T21:34:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:34.109+0000 7f3dd7fc4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T21:34:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:34.109+0000 7f3dd7fc4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:34:34.109 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:34.109+0000 7f3dd7fc4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:34:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:34.109+0000 7f3dd7fc4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:34:34.110 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:34.110+0000 7f3dd7fc4640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T21:34:36.961 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T21:34:36.990 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:37.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:37.080+0000 7f49dca2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T21:34:37.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:37.081+0000 7f49dca2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T21:34:37.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:37.081+0000 7f49dca2c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:34:37.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:37.081+0000 7f49dca2c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:34:37.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:37.081+0000 7f49dca2c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:34:37.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:37.081+0000 7f49dca2c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T21:34:39.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:34:39.817 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:34:39.939 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T21:34:39.965 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:40.051 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:40.051+0000 7f10bada3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T21:34:40.052 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:40.052+0000 7f10bada3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T21:34:40.052 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:40.052+0000 7f10bada3640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:34:40.052 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:40.052+0000 7f10bada3640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:34:40.052 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:40.052+0000 7f10bada3640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:34:40.052 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:40.052+0000 7f10bada3640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T21:34:42.909 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T21:34:42.935 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T21:34:42.935 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:42.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T21:34:42.964 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:42.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T21:34:43.034 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:43.034 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T21:34:43.103 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:43.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T21:34:43.171 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:43.171 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T21:34:43.238 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:43.238 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T21:34:43.304 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:43.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T21:34:43.371 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.371 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T21:34:43.398 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.398 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T21:34:43.464 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.464 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T21:34:43.530 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.530 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T21:34:43.594 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.594 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T21:34:43.659 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.659 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T21:34:43.722 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.722 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T21:34:43.786 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:43.786 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T21:34:43.805 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:43.805 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T21:34:43.864 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.864 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T21:34:43.878 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:43.879 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T21:34:43.933 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T21:34:43.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:34:43.936 DEBUG:teuthology.orchestra.run.smithi184:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:34:43.977 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2024-04-26T21:34:43.977 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmsiwICxAAmKAZ2KS3/dYubutkcGfolA== 2024-04-26T21:34:43.977 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-04-26T21:34:43.977 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmesG+ERAAJS53jYoKUawIOs2zrG5TLg== 2024-04-26T21:34:43.977 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-04-26T21:34:43.977 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmLRmvGBAA6xAcjLDF7DMjfDqiUCexKA== 2024-04-26T21:34:43.977 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDYHSxmRkhFBhAAz1UfvJwhA8bolDiRN2eM+Q== 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDbHSxmKAB/BBAAzW5dtz6eczjkDiTQBjfjuA== 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDeHSxmrienAhAA2JS0l6/Uo9UPc0TeZ1RwRA== 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDhHSxmv40rARAAN8z+beJe/bTERvO2euMr6w== 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmYjmPDhAADvvIHrcacI8kOAdeiJGd+Q== 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmq9cvHRAAiYWrMjfAth8wMbs9YlfWgQ== 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmXxkDJBAAbzzf+LUO3Rr8bdzXb+Gj2Q== 2024-04-26T21:34:43.978 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDnHSxmxQ1XCBAAYptVM8WPir+zIHOjbX0tGg== 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDqHSxmv32IBhAAMROkCLjWTOx3ELdkEiVSmw== 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDtHSxmJdPTBBAAO32qvVLuAzqx1xdajL9CFw== 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDwHSxmcUMYAxAAIvoAo82ApDXERodD+4+cBQ== 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw== 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout:[client.2] 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmi6l6LRAAF4EjChVEB8ReULEsl3YppA== 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmifWYMBAA6Nskvqt3MZSvFCMHjslELw== 2024-04-26T21:34:43.979 INFO:teuthology.orchestra.run.smithi040.stdout:[client.3] 2024-04-26T21:34:43.980 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmhSMFNhAAjCJvc4FknsSUJQ4NjXRV6Q== 2024-04-26T21:34:43.995 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.y] 2024-04-26T21:34:43.995 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmsiwICxAAmKAZ2KS3/dYubutkcGfolA== 2024-04-26T21:34:43.995 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2024-04-26T21:34:43.995 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmesG+ERAAJS53jYoKUawIOs2zrG5TLg== 2024-04-26T21:34:43.995 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2024-04-26T21:34:43.995 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmLRmvGBAA6xAcjLDF7DMjfDqiUCexKA== 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDYHSxmRkhFBhAAz1UfvJwhA8bolDiRN2eM+Q== 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDbHSxmKAB/BBAAzW5dtz6eczjkDiTQBjfjuA== 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDeHSxmrienAhAA2JS0l6/Uo9UPc0TeZ1RwRA== 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDhHSxmv40rARAAN8z+beJe/bTERvO2euMr6w== 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.x] 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmYjmPDhAADvvIHrcacI8kOAdeiJGd+Q== 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmq9cvHRAAiYWrMjfAth8wMbs9YlfWgQ== 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmXxkDJBAAbzzf+LUO3Rr8bdzXb+Gj2Q== 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.4] 2024-04-26T21:34:43.996 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDnHSxmxQ1XCBAAYptVM8WPir+zIHOjbX0tGg== 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.5] 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDqHSxmv32IBhAAMROkCLjWTOx3ELdkEiVSmw== 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.6] 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDtHSxmJdPTBBAAO32qvVLuAzqx1xdajL9CFw== 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.7] 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDwHSxmcUMYAxAAIvoAo82ApDXERodD+4+cBQ== 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:[client.0] 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw== 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:[client.2] 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmi6l6LRAAF4EjChVEB8ReULEsl3YppA== 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmifWYMBAA6Nskvqt3MZSvFCMHjslELw== 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:[client.3] 2024-04-26T21:34:43.997 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDUHSxmhSMFNhAAjCJvc4FknsSUJQ4NjXRV6Q== 2024-04-26T21:34:43.998 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:34:44.024 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:34:44.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:34:44.091 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:34:44.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:34:44.147 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:34:44.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.201 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.252 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.350 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.352 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:34:44.403 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:34:44.451 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:34:44.453 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:34:44.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:34:44.507 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:34:44.558 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.560 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.610 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.658 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.709 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.711 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:34:44.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:34:44.762 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:34:44.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:34:44.815 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:34:44.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:34:44.869 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:34:44.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:34:44.922 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:34:44.973 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T21:34:44.973 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T21:34:44.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:34:45.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T21:34:45.115 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T21:34:45.138 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:34:45.231 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T21:34:45.254 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T21:34:45.317 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:34:45.411 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T21:34:45.434 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:45.436 DEBUG:teuthology.orchestra.run.smithi184:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:34:45.487 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T21:34:45.488 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T21:34:45.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T21:34:45.492 INFO:tasks.ceph.mon.a:Started 2024-04-26T21:34:45.492 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T21:34:45.492 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T21:34:45.530 INFO:tasks.ceph.mon.b:Started 2024-04-26T21:34:45.530 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T21:34:45.530 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T21:34:45.532 INFO:tasks.ceph.mon.c:Started 2024-04-26T21:34:45.532 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T21:34:45.532 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T21:34:45.532 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T21:34:45.534 INFO:tasks.ceph.mgr.y:Started 2024-04-26T21:34:45.534 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T21:34:45.534 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T21:34:45.536 INFO:tasks.ceph.mgr.x:Started 2024-04-26T21:34:45.536 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T21:34:45.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T21:34:45.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.638+0000 7f1a9bdc1640 1 Processor -- start 2024-04-26T21:34:45.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.638+0000 7f1a9bdc1640 1 -- start start 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.639+0000 7f1a9bdc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 0x7f1a94160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.639+0000 7f1a9bdc1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a940b0990 con 0x7f1a9406c950 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.639+0000 7f1a9bdc1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a940b0c70 con 0x7f1a9415d100 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.639+0000 7f1a9bdc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1a940b0f50 con 0x7f1a94070930 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.639+0000 7f1a9935c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1a9406c950 0x7f1a94073160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47464/0 (socket says 172.21.15.40:47464) 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.639+0000 7f1a9935c640 1 -- 172.21.15.40:0/1965900465 learned_addr learned my addr 172.21.15.40:0/1965900465 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.639+0000 7f1a9a35e640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 0x7f1a94160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.640+0000 7f1a9a35e640 1 -- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 msgr2=0x7f1a94160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.640+0000 7f1a9a35e640 1 -- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 msgr2=0x7f1a94160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.640+0000 7f1a9a35e640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 0x7f1a94160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:34:45.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.640+0000 7f1a9a35e640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 0x7f1a94160ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:34:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.798+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3946704485 0 0) 0x7f1a940b0990 con 0x7f1a9406c950 2024-04-26T21:34:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.798+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a80003520 con 0x7f1a9406c950 2024-04-26T21:34:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800006415 0 0) 0x7f1a940b0c70 con 0x7f1a9415d100 2024-04-26T21:34:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a940b0990 con 0x7f1a9415d100 2024-04-26T21:34:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1337375732 0 0) 0x7f1a940b0990 con 0x7f1a9415d100 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 msgr2=0x7f1a94160ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 0x7f1a94160ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 >> v1:172.21.15.40:6789/0 conn(0x7f1a9406c950 legacy=0x7f1a94073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a940b1230 con 0x7f1a9415d100 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1a84001670 con 0x7f1a9415d100 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 >> v1:172.21.15.184:6789/0 conn(0x7f1a9415d100 legacy=0x7f1a9406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a80003b40 0x7f1a80003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a80004800 0x7f1a80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 0x7f1a800051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a940b0f50 con 0x7f1a80004800 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8000bb00 con 0x7f1a80003b40 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8000bd00 con 0x7f1a94070930 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a9935c640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a80004800 0x7f1a80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a9a35e640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 0x7f1a800051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.799+0000 7f1a99b5d640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a80003b40 0x7f1a80003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.800+0000 7f1a9935c640 1 -- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 msgr2=0x7f1a800051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.800+0000 7f1a9935c640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a94070930 0x7f1a800051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.800+0000 7f1a9935c640 1 -- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a80003b40 msgr2=0x7f1a80003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.800+0000 7f1a9935c640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a80003b40 0x7f1a80003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.800+0000 7f1a9935c640 1 -- 172.21.15.40:0/1965900465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a940b0990 con 0x7f1a80004800 2024-04-26T21:34:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.800+0000 7f1a9935c640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a80004800 0x7f1a80004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a900019c0 tx=0x7f1a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ca2fea0af952fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.802+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90001610 con 0x7f1a80004800 2024-04-26T21:34:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.802+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9002e070 con 0x7f1a80004800 2024-04-26T21:34:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.802+0000 7f1a98b5b640 1 -- 172.21.15.40:0/1965900465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a900048a0 con 0x7f1a80004800 2024-04-26T21:34:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.802+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/1965900465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a80004800 msgr2=0x7f1a80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:45.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.802+0000 7f1a9bdc1640 1 --2- 172.21.15.40:0/1965900465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a80004800 0x7f1a80004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a900019c0 tx=0x7f1a80005830 comp rx=0 tx=0).stop 2024-04-26T21:34:45.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.802+0000 7f1a9a35e640 1 -- 172.21.15.40:0/1965900465 reap_dead start 2024-04-26T21:34:45.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.802+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/1965900465 shutdown_connections 2024-04-26T21:34:45.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.802+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/1965900465 >> 172.21.15.40:0/1965900465 conn(0x7f1a940b39b0 msgr2=0x7f1a940b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:45.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.803+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/1965900465 shutdown_connections 2024-04-26T21:34:45.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.803+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/1965900465 wait complete. 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.804+0000 7f1a9bdc1640 1 Processor -- start 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.804+0000 7f1a9bdc1640 1 -- start start 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9bdc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a80004800 0x7f1a94154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9bdc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a941446f0 0x7f1a941451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9bdc1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a940b4aa0 0x7f1a940b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9bdc1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94160d20 con 0x7f1a941446f0 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9bdc1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94160ba0 con 0x7f1a940b4aa0 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9bdc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1a94160ea0 con 0x7f1a80004800 2024-04-26T21:34:45.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a941446f0 0x7f1a941451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a941446f0 0x7f1a941451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47282/0 (socket says 172.21.15.40:47282) 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 -- 172.21.15.40:0/3610384164 learned_addr learned my addr 172.21.15.40:0/3610384164 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9a35e640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a940b4aa0 0x7f1a940b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a99b5d640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a80004800 0x7f1a94154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 -- 172.21.15.40:0/3610384164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a80004800 msgr2=0x7f1a94154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a80004800 0x7f1a94154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 -- 172.21.15.40:0/3610384164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a940b4aa0 msgr2=0x7f1a940b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a940b4aa0 0x7f1a940b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 -- 172.21.15.40:0/3610384164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9415aad0 con 0x7f1a941446f0 2024-04-26T21:34:45.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.805+0000 7f1a9935c640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a941446f0 0x7f1a941451d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a90001ae0 tx=0x7f1a900012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:45.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.806+0000 7f1a8a7fc640 1 -- 172.21.15.40:0/3610384164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90004c70 con 0x7f1a941446f0 2024-04-26T21:34:45.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.806+0000 7f1a8a7fc640 1 -- 172.21.15.40:0/3610384164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a90004e10 con 0x7f1a941446f0 2024-04-26T21:34:45.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.806+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9415adb0 con 0x7f1a941446f0 2024-04-26T21:34:45.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.806+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9415b370 con 0x7f1a941446f0 2024-04-26T21:34:45.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.806+0000 7f1a8a7fc640 1 -- 172.21.15.40:0/3610384164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90007270 con 0x7f1a941446f0 2024-04-26T21:34:45.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.806+0000 7f1a8a7fc640 1 -- 172.21.15.40:0/3610384164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a90045020 con 0x7f1a941446f0 2024-04-26T21:34:45.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.806+0000 7f1a8a7fc640 1 -- 172.21.15.40:0/3610384164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1a90047400 con 0x7f1a941446f0 2024-04-26T21:34:45.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.807+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a940694f0 con 0x7f1a941446f0 2024-04-26T21:34:45.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.809+0000 7f1a8a7fc640 1 -- 172.21.15.40:0/3610384164 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1a90033420 con 0x7f1a941446f0 2024-04-26T21:34:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.877+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1a9406cd70 con 0x7f1a941446f0 2024-04-26T21:34:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.879+0000 7f1a8a7fc640 1 -- 172.21.15.40:0/3610384164 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f1a90033600 con 0x7f1a941446f0 2024-04-26T21:34:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.879+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a941446f0 msgr2=0x7f1a941451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.879+0000 7f1a9bdc1640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a941446f0 0x7f1a941451d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a90001ae0 tx=0x7f1a900012d0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.879+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 shutdown_connections 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.879+0000 7f1a9bdc1640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a940b4aa0 0x7f1a940b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.879+0000 7f1a9bdc1640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a941446f0 0x7f1a941451d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.879+0000 7f1a9bdc1640 1 --2- 172.21.15.40:0/3610384164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a80004800 0x7f1a94154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.879+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 >> 172.21.15.40:0/3610384164 conn(0x7f1a940b39b0 msgr2=0x7f1a941558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.880+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 shutdown_connections 2024-04-26T21:34:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:45.880+0000 7f1a9bdc1640 1 -- 172.21.15.40:0/3610384164 wait complete. 2024-04-26T21:34:45.890 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T21:34:45.890 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T21:34:45.891 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:45.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T21:34:45.915 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:45.915 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T21:34:45.916 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:45.916+0000 7efe9fdbf040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:34:45.922 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:45.922+0000 7efe66a12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:34:45.945 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:45.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T21:34:46.010 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:34:46.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T21:34:46.074 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:46.074 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T21:34:46.079 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:46.078+0000 7efe9fdbf040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:34:46.099 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:46.099 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T21:34:46.109 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:46.109+0000 7efe66a12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:34:46.135 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:46.135+0000 7efe9fdbf040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:34:46.158 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:46.158 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T21:34:46.165 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:46.165+0000 7efe66a12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:34:46.222 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:34:46.222 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T21:34:46.286 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new d7180d7b-0b28-4813-a6e6-11d0a2c38363 0 2024-04-26T21:34:46.391 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:46.390+0000 7efe9fdbf040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:34:46.414 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:46.414+0000 7efe66a12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:34:46.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.422+0000 7fe12ee83640 1 Processor -- start 2024-04-26T21:34:46.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.422+0000 7fe12ee83640 1 -- start start 2024-04-26T21:34:46.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12ee83640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe1280d27d0 0x7fe1280d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12ee83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12806e470 0x7fe12815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12ee83640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe128070c10 con 0x7fe128070790 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12ee83640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe128070ef0 con 0x7fe1280d27d0 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12ee83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1280711d0 con 0x7fe12806e470 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe127fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fe128070790 0x7fe12806dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:42564/0 (socket says 172.21.15.184:42564) 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12d420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12806e470 0x7fe12815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe127fff640 1 -- 172.21.15.184:0/162620255 learned_addr learned my addr 172.21.15.184:0/162620255 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12cc1f640 1 --2- 172.21.15.184:0/162620255 >> v2:172.21.15.40:3300/0 conn(0x7fe1280d27d0 0x7fe1280d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe1277fe640 1 -- 172.21.15.184:0/162620255 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730048925 0 0) 0x7fe128070c10 con 0x7fe128070790 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe1277fe640 1 -- 172.21.15.184:0/162620255 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe118003520 con 0x7fe128070790 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12d420640 1 -- 172.21.15.184:0/162620255 >> v2:172.21.15.40:3300/0 conn(0x7fe1280d27d0 msgr2=0x7fe1280d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12d420640 1 --2- 172.21.15.184:0/162620255 >> v2:172.21.15.40:3300/0 conn(0x7fe1280d27d0 0x7fe1280d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.423+0000 7fe12d420640 1 -- 172.21.15.184:0/162620255 >> v1:172.21.15.184:6789/0 conn(0x7fe128070790 legacy=0x7fe12806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.424+0000 7fe12d420640 1 -- 172.21.15.184:0/162620255 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1280714b0 con 0x7fe12806e470 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.424+0000 7fe12d420640 1 --2- 172.21.15.184:0/162620255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12806e470 0x7fe12815ea70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe120009d90 tx=0x7fe12002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f210fc71aa1ad7be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.424+0000 7fe1277fe640 1 -- 172.21.15.184:0/162620255 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120007740 con 0x7fe12806e470 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.424+0000 7fe1277fe640 1 -- 172.21.15.184:0/162620255 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1200078e0 con 0x7fe12806e470 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.424+0000 7fe1277fe640 1 -- 172.21.15.184:0/162620255 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120007cb0 con 0x7fe12806e470 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.425+0000 7fe12ee83640 1 -- 172.21.15.184:0/162620255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12806e470 msgr2=0x7fe12815ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.425+0000 7fe12ee83640 1 --2- 172.21.15.184:0/162620255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12806e470 0x7fe12815ea70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe120009d90 tx=0x7fe12002a660 comp rx=0 tx=0).stop 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.425+0000 7fe12ee83640 1 -- 172.21.15.184:0/162620255 shutdown_connections 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.425+0000 7fe12ee83640 1 --2- 172.21.15.184:0/162620255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12806e470 0x7fe12815ea70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.425+0000 7fe12ee83640 1 --2- 172.21.15.184:0/162620255 >> v2:172.21.15.40:3300/0 conn(0x7fe1280d27d0 0x7fe1280d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.425+0000 7fe12ee83640 1 -- 172.21.15.184:0/162620255 >> 172.21.15.184:0/162620255 conn(0x7fe1280d2e00 msgr2=0x7fe1280d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.425+0000 7fe12ee83640 1 -- 172.21.15.184:0/162620255 shutdown_connections 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.425+0000 7fe12ee83640 1 -- 172.21.15.184:0/162620255 wait complete. 2024-04-26T21:34:46.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.426+0000 7fe12ee83640 1 Processor -- start 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.426+0000 7fe12ee83640 1 -- start start 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe12ee83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12806e470 0x7fe12815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe12ee83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe128070790 0x7fe128158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe12ee83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1280d27d0 0x7fe1281f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe12ee83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280d06e0 con 0x7fe12806e470 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe12ee83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280d2bb0 con 0x7fe128070790 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe12ee83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe12806d790 con 0x7fe1280d27d0 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe127fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe128070790 0x7fe128158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe12d420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1280d27d0 0x7fe1281f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe127fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe128070790 0x7fe128158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:34026/0 (socket says 172.21.15.184:34026) 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe127fff640 1 -- 172.21.15.184:0/166727231 learned_addr learned my addr 172.21.15.184:0/166727231 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe12cc1f640 1 --2- 172.21.15.184:0/166727231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12806e470 0x7fe12815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe127fff640 1 -- 172.21.15.184:0/166727231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1280d27d0 msgr2=0x7fe1281f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe127fff640 1 --2- 172.21.15.184:0/166727231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1280d27d0 0x7fe1281f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe127fff640 1 -- 172.21.15.184:0/166727231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12806e470 msgr2=0x7fe12815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe127fff640 1 --2- 172.21.15.184:0/166727231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12806e470 0x7fe12815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.427+0000 7fe127fff640 1 -- 172.21.15.184:0/166727231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12815b610 con 0x7fe128070790 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.428+0000 7fe127fff640 1 --2- 172.21.15.184:0/166727231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe128070790 0x7fe128158710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1140010e0 tx=0x7fe11400c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.428+0000 7fe1257fa640 1 -- 172.21.15.184:0/166727231 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114017070 con 0x7fe128070790 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.428+0000 7fe1257fa640 1 -- 172.21.15.184:0/166727231 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe114005d00 con 0x7fe128070790 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.428+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12815b8f0 con 0x7fe128070790 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.428+0000 7fe1257fa640 1 -- 172.21.15.184:0/166727231 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140072b0 con 0x7fe128070790 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.428+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1281f8920 con 0x7fe128070790 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.428+0000 7fe1257fa640 1 -- 172.21.15.184:0/166727231 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe11401a070 con 0x7fe128070790 2024-04-26T21:34:46.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.428+0000 7fe1257fa640 1 -- 172.21.15.184:0/166727231 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fe114012740 con 0x7fe128070790 2024-04-26T21:34:46.434 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.429+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1281f8b20 con 0x7fe128070790 2024-04-26T21:34:46.434 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.431+0000 7fe1257fa640 1 -- 172.21.15.184:0/166727231 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe1281f8b20 con 0x7fe128070790 2024-04-26T21:34:46.445 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:46.445+0000 7efe9fdbf040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:34:46.461 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:46.461+0000 7efe66a12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:34:46.557 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.556+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d7180d7b-0b28-4813-a6e6-11d0a2c38363", "id": 0} v 0) -- 0x7fe1280694f0 con 0x7fe128070790 2024-04-26T21:34:46.561 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.560+0000 7fe1257fa640 1 -- 172.21.15.184:0/166727231 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d7180d7b-0b28-4813-a6e6-11d0a2c38363", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fe114021020 con 0x7fe128070790 2024-04-26T21:34:46.561 INFO:teuthology.orchestra.run.smithi184.stdout:0 2024-04-26T21:34:46.562 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe128070790 msgr2=0x7fe128158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.562 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 --2- 172.21.15.184:0/166727231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe128070790 0x7fe128158710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1140010e0 tx=0x7fe11400c960 comp rx=0 tx=0).stop 2024-04-26T21:34:46.562 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 shutdown_connections 2024-04-26T21:34:46.562 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 --2- 172.21.15.184:0/166727231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1280d27d0 0x7fe1281f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.562 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 --2- 172.21.15.184:0/166727231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe128070790 0x7fe128158710 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.562 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 --2- 172.21.15.184:0/166727231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12806e470 0x7fe12815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.563 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 >> 172.21.15.184:0/166727231 conn(0x7fe1280d2e00 msgr2=0x7fe12806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:46.563 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 shutdown_connections 2024-04-26T21:34:46.563 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.561+0000 7fe12ee83640 1 -- 172.21.15.184:0/166727231 wait complete. 2024-04-26T21:34:46.572 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 5f3271b9-0a80-4ee5-89e7-3b6db512e273 1 2024-04-26T21:34:46.670 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.669+0000 7fc924e6d640 1 Processor -- start 2024-04-26T21:34:46.670 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.669+0000 7fc924e6d640 1 -- start start 2024-04-26T21:34:46.672 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc924e6d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc9200b88a0 0x7fc9200d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.672 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc924e6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.672 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc924e6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9200d04a0 0x7fc9200cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.672 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc924e6d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc9200d2370 con 0x7fc9200d04a0 2024-04-26T21:34:46.672 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc924e6d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc9200b8c80 con 0x7fc9200b88a0 2024-04-26T21:34:46.672 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc924e6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9200cd5b0 con 0x7fc9200b5c10 2024-04-26T21:34:46.672 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91e59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc9200b88a0 0x7fc9200d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.672 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91e59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc9200b88a0 0x7fc9200d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38588/0 (socket says 172.21.15.184:38588) 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91e59c640 1 -- 172.21.15.184:0/1331591895 learned_addr learned my addr 172.21.15.184:0/1331591895 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91ed9d640 1 --2- 172.21.15.184:0/1331591895 >> v2:172.21.15.40:3300/0 conn(0x7fc9200d04a0 0x7fc9200cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91e59c640 1 -- 172.21.15.184:0/1331591895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 msgr2=0x7fc9200cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91e59c640 1 --2- 172.21.15.184:0/1331591895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91e59c640 1 -- 172.21.15.184:0/1331591895 >> v2:172.21.15.40:3300/0 conn(0x7fc9200d04a0 msgr2=0x7fc9200cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91e59c640 1 --2- 172.21.15.184:0/1331591895 >> v2:172.21.15.40:3300/0 conn(0x7fc9200d04a0 0x7fc9200cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91e59c640 1 -- 172.21.15.184:0/1331591895 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9200cd890 con 0x7fc9200b88a0 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91ed9d640 1 --2- 172.21.15.184:0/1331591895 >> v2:172.21.15.40:3300/0 conn(0x7fc9200d04a0 0x7fc9200cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.670+0000 7fc91dd9b640 1 --2- 172.21.15.184:0/1331591895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.671+0000 7fc91e59c640 1 --2- 172.21.15.184:0/1331591895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc9200d19f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc9200736c0 tx=0x7fc91002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607ab1d12c77049f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.671+0000 7fc91d59a640 1 -- 172.21.15.184:0/1331591895 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910037070 con 0x7fc9200b88a0 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.671+0000 7fc91d59a640 1 -- 172.21.15.184:0/1331591895 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910032440 con 0x7fc9200b88a0 2024-04-26T21:34:46.673 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.671+0000 7fc91d59a640 1 -- 172.21.15.184:0/1331591895 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910032810 con 0x7fc9200b88a0 2024-04-26T21:34:46.676 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.671+0000 7fc924e6d640 1 -- 172.21.15.184:0/1331591895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 msgr2=0x7fc9200d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.676 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.671+0000 7fc924e6d640 1 --2- 172.21.15.184:0/1331591895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc9200d19f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc9200736c0 tx=0x7fc91002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.672+0000 7fc924e6d640 1 -- 172.21.15.184:0/1331591895 shutdown_connections 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.672+0000 7fc924e6d640 1 --2- 172.21.15.184:0/1331591895 >> v2:172.21.15.40:3300/0 conn(0x7fc9200d04a0 0x7fc9200cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.672+0000 7fc924e6d640 1 --2- 172.21.15.184:0/1331591895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.672+0000 7fc924e6d640 1 --2- 172.21.15.184:0/1331591895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc9200d19f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.672+0000 7fc924e6d640 1 -- 172.21.15.184:0/1331591895 >> 172.21.15.184:0/1331591895 conn(0x7fc9200d2dc0 msgr2=0x7fc9200d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.672+0000 7fc924e6d640 1 -- 172.21.15.184:0/1331591895 shutdown_connections 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.672+0000 7fc924e6d640 1 -- 172.21.15.184:0/1331591895 wait complete. 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.673+0000 7fc924e6d640 1 Processor -- start 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.673+0000 7fc924e6d640 1 -- start start 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc924e6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc924e6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc92015d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.677 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc924e6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9200d04a0 0x7fc9201f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc924e6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9200d1f30 con 0x7fc9200d04a0 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc924e6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9200cde50 con 0x7fc9200b88a0 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc924e6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9200d20b0 con 0x7fc9200b5c10 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc92015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc92015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38604/0 (socket says 172.21.15.184:38604) 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:47538/0 (socket says 172.21.15.184:47538) 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 -- 172.21.15.184:0/639341602 learned_addr learned my addr 172.21.15.184:0/639341602 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91ed9d640 1 --2- 172.21.15.184:0/639341602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9200d04a0 0x7fc9201f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 -- 172.21.15.184:0/639341602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 msgr2=0x7fc9200cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 --2- 172.21.15.184:0/639341602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 -- 172.21.15.184:0/639341602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9200d04a0 msgr2=0x7fc9201f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.678 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 --2- 172.21.15.184:0/639341602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9200d04a0 0x7fc9201f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 -- 172.21.15.184:0/639341602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc920160220 con 0x7fc9200b88a0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc91dd9b640 1 --2- 172.21.15.184:0/639341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc92015d850 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc90800e8e0 tx=0x7fc90800eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc916ffd640 1 -- 172.21.15.184:0/639341602 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908017070 con 0x7fc9200b88a0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc916ffd640 1 -- 172.21.15.184:0/639341602 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90800c8e0 con 0x7fc9200b88a0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc916ffd640 1 -- 172.21.15.184:0/639341602 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90800cc90 con 0x7fc9200b88a0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.674+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc920160500 con 0x7fc9200b88a0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.675+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc920160ac0 con 0x7fc9200b88a0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.675+0000 7fc916ffd640 1 -- 172.21.15.184:0/639341602 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc90801a070 con 0x7fc9200b88a0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.675+0000 7fc916ffd640 1 -- 172.21.15.184:0/639341602 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fc90801c800 con 0x7fc9200b88a0 2024-04-26T21:34:46.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.675+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9200694f0 con 0x7fc9200b88a0 2024-04-26T21:34:46.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.678+0000 7fc916ffd640 1 -- 172.21.15.184:0/639341602 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc90801ca50 con 0x7fc9200b88a0 2024-04-26T21:34:46.751 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.750+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5f3271b9-0a80-4ee5-89e7-3b6db512e273", "id": 1} v 0) -- 0x7fc9200b5580 con 0x7fc9200b88a0 2024-04-26T21:34:46.755 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.754+0000 7fc916ffd640 1 -- 172.21.15.184:0/639341602 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5f3271b9-0a80-4ee5-89e7-3b6db512e273", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fc90801cc30 con 0x7fc9200b88a0 2024-04-26T21:34:46.755 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:34:46.756 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 msgr2=0x7fc92015d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.756 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 --2- 172.21.15.184:0/639341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc92015d850 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc90800e8e0 tx=0x7fc90800eda0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.756 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 shutdown_connections 2024-04-26T21:34:46.756 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 --2- 172.21.15.184:0/639341602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9200d04a0 0x7fc9201f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.756 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 --2- 172.21.15.184:0/639341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9200b88a0 0x7fc92015d850 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.756 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 --2- 172.21.15.184:0/639341602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9200b5c10 0x7fc9200cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.757 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 >> 172.21.15.184:0/639341602 conn(0x7fc9200d2dc0 msgr2=0x7fc9200703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:46.757 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 shutdown_connections 2024-04-26T21:34:46.757 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.755+0000 7fc924e6d640 1 -- 172.21.15.184:0/639341602 wait complete. 2024-04-26T21:34:46.766 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 2a3d220c-7de8-4f4f-b1d7-929338ce542c 2 2024-04-26T21:34:46.865 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.863+0000 7fb029751640 1 Processor -- start 2024-04-26T21:34:46.865 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.863+0000 7fb029751640 1 -- start start 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.863+0000 7fb029751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02415f3a0 0x7fb0240d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.863+0000 7fb029751640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0240d1770 0x7fb0240d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.863+0000 7fb029751640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0240d25a0 con 0x7fb0240d2130 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb029751640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb0240d0e20 con 0x7fb0240d1770 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb029751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0240b1d70 con 0x7fb02415f3a0 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb022ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02415f3a0 0x7fb0240d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb0237fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb0240d2130 0x7fb0240d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:42534/0 (socket says 172.21.15.184:42534) 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb0237fe640 1 -- 172.21.15.184:0/1141815174 learned_addr learned my addr 172.21.15.184:0/1141815174 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb0227fc640 1 --2- 172.21.15.184:0/1141815174 >> v2:172.21.15.40:3300/0 conn(0x7fb0240d1770 0x7fb0240d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb021ffb640 1 -- 172.21.15.184:0/1141815174 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1722859197 0 0) 0x7fb01c001490 con 0x7fb0240d2130 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb021ffb640 1 -- 172.21.15.184:0/1141815174 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb00c003520 con 0x7fb0240d2130 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb022ffd640 1 -- 172.21.15.184:0/1141815174 >> v2:172.21.15.40:3300/0 conn(0x7fb0240d1770 msgr2=0x7fb0240d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb022ffd640 1 --2- 172.21.15.184:0/1141815174 >> v2:172.21.15.40:3300/0 conn(0x7fb0240d1770 0x7fb0240d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb022ffd640 1 -- 172.21.15.184:0/1141815174 >> v1:172.21.15.184:6789/0 conn(0x7fb0240d2130 legacy=0x7fb0240d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.867 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.864+0000 7fb022ffd640 1 -- 172.21.15.184:0/1141815174 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0240b2050 con 0x7fb02415f3a0 2024-04-26T21:34:46.867 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.865+0000 7fb022ffd640 1 --2- 172.21.15.184:0/1141815174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02415f3a0 0x7fb0240d06f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0140097a0 tx=0x7fb01402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7685642e84d482c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.865+0000 7fb021ffb640 1 -- 172.21.15.184:0/1141815174 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014037070 con 0x7fb02415f3a0 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.865+0000 7fb021ffb640 1 -- 172.21.15.184:0/1141815174 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb014032440 con 0x7fb02415f3a0 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.865+0000 7fb021ffb640 1 -- 172.21.15.184:0/1141815174 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014032810 con 0x7fb02415f3a0 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.866+0000 7fb029751640 1 -- 172.21.15.184:0/1141815174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02415f3a0 msgr2=0x7fb0240d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.866+0000 7fb029751640 1 --2- 172.21.15.184:0/1141815174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02415f3a0 0x7fb0240d06f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0140097a0 tx=0x7fb01402a180 comp rx=0 tx=0).stop 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.866+0000 7fb029751640 1 -- 172.21.15.184:0/1141815174 shutdown_connections 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.866+0000 7fb029751640 1 --2- 172.21.15.184:0/1141815174 >> v2:172.21.15.40:3300/0 conn(0x7fb0240d1770 0x7fb0240d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.866+0000 7fb029751640 1 --2- 172.21.15.184:0/1141815174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02415f3a0 0x7fb0240d06f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.866+0000 7fb029751640 1 -- 172.21.15.184:0/1141815174 >> 172.21.15.184:0/1141815174 conn(0x7fb0240d4f90 msgr2=0x7fb0240d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.866+0000 7fb029751640 1 -- 172.21.15.184:0/1141815174 shutdown_connections 2024-04-26T21:34:46.868 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.866+0000 7fb029751640 1 -- 172.21.15.184:0/1141815174 wait complete. 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.867+0000 7fb029751640 1 Processor -- start 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.867+0000 7fb029751640 1 -- start start 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb029751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0240d1770 0x7fb024159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb029751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0240d2130 0x7fb0241594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb029751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb02415f3a0 0x7fb0240cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb029751640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb02415f900 con 0x7fb02415f3a0 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb029751640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb02415f780 con 0x7fb0240d2130 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb029751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0240b3860 con 0x7fb0240d1770 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb022ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0240d1770 0x7fb024159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb0227fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0240d2130 0x7fb0241594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb022ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0240d1770 0x7fb024159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:47564/0 (socket says 172.21.15.184:47564) 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb022ffd640 1 -- 172.21.15.184:0/318279997 learned_addr learned my addr 172.21.15.184:0/318279997 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb0237fe640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb02415f3a0 0x7fb0240cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb0227fc640 1 -- 172.21.15.184:0/318279997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0240d1770 msgr2=0x7fb024159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.868+0000 7fb0227fc640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0240d1770 0x7fb024159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb0227fc640 1 -- 172.21.15.184:0/318279997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb02415f3a0 msgr2=0x7fb0240cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb0227fc640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb02415f3a0 0x7fb0240cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb0227fc640 1 -- 172.21.15.184:0/318279997 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0240cc4b0 con 0x7fb0240d2130 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb022ffd640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0240d1770 0x7fb024159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb0227fc640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0240d2130 0x7fb0241594e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb01800e850 tx=0x7fb01800ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb00b7fe640 1 -- 172.21.15.184:0/318279997 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018017070 con 0x7fb0240d2130 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb00b7fe640 1 -- 172.21.15.184:0/318279997 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb01800c940 con 0x7fb0240d2130 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0240cc790 con 0x7fb0240d2130 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb00b7fe640 1 -- 172.21.15.184:0/318279997 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01800cd10 con 0x7fb0240d2130 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0240ccd50 con 0x7fb0240d2130 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb00b7fe640 1 -- 172.21.15.184:0/318279997 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb01801a070 con 0x7fb0240d2130 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.869+0000 7fb00b7fe640 1 -- 172.21.15.184:0/318279997 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fb01801c870 con 0x7fb0240d2130 2024-04-26T21:34:46.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.870+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0240694f0 con 0x7fb0240d2130 2024-04-26T21:34:46.873 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.872+0000 7fb00b7fe640 1 -- 172.21.15.184:0/318279997 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb01801c450 con 0x7fb0240d2130 2024-04-26T21:34:46.945 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.945+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a3d220c-7de8-4f4f-b1d7-929338ce542c", "id": 2} v 0) -- 0x7fb0240babb0 con 0x7fb0240d2130 2024-04-26T21:34:46.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.948+0000 7fb00b7fe640 1 -- 172.21.15.184:0/318279997 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a3d220c-7de8-4f4f-b1d7-929338ce542c", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fb01801cb40 con 0x7fb0240d2130 2024-04-26T21:34:46.949 INFO:teuthology.orchestra.run.smithi184.stdout:2 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.948+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0240d2130 msgr2=0x7fb0241594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.948+0000 7fb029751640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0240d2130 0x7fb0241594e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb01800e850 tx=0x7fb01800ed10 comp rx=0 tx=0).stop 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.949+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 shutdown_connections 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.949+0000 7fb029751640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb02415f3a0 0x7fb0240cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.949+0000 7fb029751640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0240d2130 0x7fb0241594e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.949+0000 7fb029751640 1 --2- 172.21.15.184:0/318279997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0240d1770 0x7fb024159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.949+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 >> 172.21.15.184:0/318279997 conn(0x7fb0240d4f90 msgr2=0x7fb0240d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.949+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 shutdown_connections 2024-04-26T21:34:46.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:46.949+0000 7fb029751640 1 -- 172.21.15.184:0/318279997 wait complete. 2024-04-26T21:34:46.960 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new c19af625-ba1e-464d-9733-94998080ed8a 3 2024-04-26T21:34:47.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.056+0000 7fa39a99d640 1 Processor -- start 2024-04-26T21:34:47.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.056+0000 7fa39a99d640 1 -- start start 2024-04-26T21:34:47.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa39a99d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa394071110 0x7fa394071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa39a99d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa394072f50 0x7fa3940733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa39a99d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3940cfbf0 con 0x7fa39415bcc0 2024-04-26T21:34:47.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa39a99d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa3940738e0 con 0x7fa394072f50 2024-04-26T21:34:47.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa39a99d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa3940d0800 con 0x7fa394071110 2024-04-26T21:34:47.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa3937fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa394071110 0x7fa394071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa3937fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa394071110 0x7fa394071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38610/0 (socket says 172.21.15.184:38610) 2024-04-26T21:34:47.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa3937fe640 1 -- 172.21.15.184:0/3527240631 learned_addr learned my addr 172.21.15.184:0/3527240631 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa398f3a640 1 --2- 172.21.15.184:0/3527240631 >> v2:172.21.15.40:3300/0 conn(0x7fa394072f50 0x7fa3940733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa3937fe640 1 -- 172.21.15.184:0/3527240631 >> v2:172.21.15.40:3300/0 conn(0x7fa394072f50 msgr2=0x7fa3940733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa3937fe640 1 --2- 172.21.15.184:0/3527240631 >> v2:172.21.15.40:3300/0 conn(0x7fa394072f50 0x7fa3940733a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa3937fe640 1 -- 172.21.15.184:0/3527240631 >> v1:172.21.15.40:6789/0 conn(0x7fa39415bcc0 legacy=0x7fa3940cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa3937fe640 1 -- 172.21.15.184:0/3527240631 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3940cfed0 con 0x7fa394071110 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.057+0000 7fa398f3a640 1 --2- 172.21.15.184:0/3527240631 >> v2:172.21.15.40:3300/0 conn(0x7fa394072f50 0x7fa3940733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.058+0000 7fa3937fe640 1 --2- 172.21.15.184:0/3527240631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394071110 0x7fa394071570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3940748d0 tx=0x7fa38402c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=653cabd74f7a613c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.058+0000 7fa392ffd640 1 -- 172.21.15.184:0/3527240631 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384036020 con 0x7fa394071110 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.058+0000 7fa392ffd640 1 -- 172.21.15.184:0/3527240631 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa384032640 con 0x7fa394071110 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.058+0000 7fa392ffd640 1 -- 172.21.15.184:0/3527240631 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384032a10 con 0x7fa394071110 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.059+0000 7fa39a99d640 1 -- 172.21.15.184:0/3527240631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394071110 msgr2=0x7fa394071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.059+0000 7fa39a99d640 1 --2- 172.21.15.184:0/3527240631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394071110 0x7fa394071570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3940748d0 tx=0x7fa38402c940 comp rx=0 tx=0).stop 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.059+0000 7fa39a99d640 1 -- 172.21.15.184:0/3527240631 shutdown_connections 2024-04-26T21:34:47.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.059+0000 7fa39a99d640 1 --2- 172.21.15.184:0/3527240631 >> v2:172.21.15.40:3300/0 conn(0x7fa394072f50 0x7fa3940733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.062 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.059+0000 7fa39a99d640 1 --2- 172.21.15.184:0/3527240631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394071110 0x7fa394071570 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.062 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.059+0000 7fa39a99d640 1 -- 172.21.15.184:0/3527240631 >> 172.21.15.184:0/3527240631 conn(0x7fa3940d1cb0 msgr2=0x7fa3940d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.062 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.059+0000 7fa39a99d640 1 -- 172.21.15.184:0/3527240631 shutdown_connections 2024-04-26T21:34:47.062 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.059+0000 7fa39a99d640 1 -- 172.21.15.184:0/3527240631 wait complete. 2024-04-26T21:34:47.064 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.060+0000 7fa39a99d640 1 Processor -- start 2024-04-26T21:34:47.064 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.060+0000 7fa39a99d640 1 -- start start 2024-04-26T21:34:47.064 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa39a99d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa394071110 0x7fa3940c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.064 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa39a99d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394072f50 0x7fa3941f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.064 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa39a99d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa39415bcc0 0x7fa3941f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa39a99d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3940d0490 con 0x7fa394071110 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa39a99d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3940cca10 con 0x7fa394072f50 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa39a99d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa3940cf880 con 0x7fa39415bcc0 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa3937fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394072f50 0x7fa3941f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa398f3a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa39415bcc0 0x7fa3941f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa3937fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394072f50 0x7fa3941f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38620/0 (socket says 172.21.15.184:38620) 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa393fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa394071110 0x7fa3940c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa3937fe640 1 -- 172.21.15.184:0/337004725 learned_addr learned my addr 172.21.15.184:0/337004725 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa398f3a640 1 -- 172.21.15.184:0/337004725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394072f50 msgr2=0x7fa3941f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa398f3a640 1 --2- 172.21.15.184:0/337004725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394072f50 0x7fa3941f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa398f3a640 1 -- 172.21.15.184:0/337004725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa394071110 msgr2=0x7fa3940c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa398f3a640 1 --2- 172.21.15.184:0/337004725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa394071110 0x7fa3940c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa398f3a640 1 -- 172.21.15.184:0/337004725 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3940c5cd0 con 0x7fa39415bcc0 2024-04-26T21:34:47.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa3937fe640 1 --2- 172.21.15.184:0/337004725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394072f50 0x7fa3941f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.061+0000 7fa398f3a640 1 --2- 172.21.15.184:0/337004725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa39415bcc0 0x7fa3941f46e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007ef0 tx=0x7fa38c00d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.062+0000 7fa390ff9640 1 -- 172.21.15.184:0/337004725 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c01a070 con 0x7fa39415bcc0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.062+0000 7fa390ff9640 1 -- 172.21.15.184:0/337004725 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38c003b90 con 0x7fa39415bcc0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.062+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3940c5fb0 con 0x7fa39415bcc0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.062+0000 7fa390ff9640 1 -- 172.21.15.184:0/337004725 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c004030 con 0x7fa39415bcc0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.062+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3940c6570 con 0x7fa39415bcc0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.062+0000 7fa390ff9640 1 -- 172.21.15.184:0/337004725 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa38c01d070 con 0x7fa39415bcc0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.062+0000 7fa390ff9640 1 -- 172.21.15.184:0/337004725 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fa38c01fd30 con 0x7fa39415bcc0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.063+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3940694f0 con 0x7fa39415bcc0 2024-04-26T21:34:47.066 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.065+0000 7fa390ff9640 1 -- 172.21.15.184:0/337004725 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa38c010070 con 0x7fa39415bcc0 2024-04-26T21:34:47.138 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.137+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c19af625-ba1e-464d-9733-94998080ed8a", "id": 3} v 0) -- 0x7fa3940d2830 con 0x7fa39415bcc0 2024-04-26T21:34:47.141 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.140+0000 7fa390ff9640 1 -- 172.21.15.184:0/337004725 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c19af625-ba1e-464d-9733-94998080ed8a", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fa38c01f8a0 con 0x7fa39415bcc0 2024-04-26T21:34:47.141 INFO:teuthology.orchestra.run.smithi184.stdout:3 2024-04-26T21:34:47.142 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa39415bcc0 msgr2=0x7fa3941f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.143 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 --2- 172.21.15.184:0/337004725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa39415bcc0 0x7fa3941f46e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007ef0 tx=0x7fa38c00d4a0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.143 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 shutdown_connections 2024-04-26T21:34:47.143 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 --2- 172.21.15.184:0/337004725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa39415bcc0 0x7fa3941f46e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.143 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 --2- 172.21.15.184:0/337004725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa394072f50 0x7fa3941f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.143 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 --2- 172.21.15.184:0/337004725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa394071110 0x7fa3940c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.143 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 >> 172.21.15.184:0/337004725 conn(0x7fa3940d1cb0 msgr2=0x7fa3940cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.143 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 shutdown_connections 2024-04-26T21:34:47.143 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.141+0000 7fa39a99d640 1 -- 172.21.15.184:0/337004725 wait complete. 2024-04-26T21:34:47.153 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33 4 2024-04-26T21:34:47.200 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:47.200+0000 7efe66a12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:34:47.222 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:47.221+0000 7efe9fdbf040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:34:47.250 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.249+0000 7f1e3ca20640 1 Processor -- start 2024-04-26T21:34:47.251 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.250+0000 7f1e3ca20640 1 -- start start 2024-04-26T21:34:47.252 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.250+0000 7f1e3ca20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b0b70 0x7f1e380b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.252 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.250+0000 7f1e3ca20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e380b4d40 0x7f1e380b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.252 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3ca20640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e380b1490 con 0x7f1e380b6c50 2024-04-26T21:34:47.252 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3ca20640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1e38073bc0 con 0x7f1e380b4d40 2024-04-26T21:34:47.252 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3ca20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1e38072fb0 con 0x7f1e380b0b70 2024-04-26T21:34:47.252 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3659c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b0b70 0x7f1e380b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.252 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3659c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b0b70 0x7f1e380b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:47594/0 (socket says 172.21.15.184:47594) 2024-04-26T21:34:47.252 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3659c640 1 -- 172.21.15.184:0/4198317853 learned_addr learned my addr 172.21.15.184:0/4198317853 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.253 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e35d9b640 1 --2- 172.21.15.184:0/4198317853 >> v2:172.21.15.40:3300/0 conn(0x7f1e380b4d40 0x7f1e380b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.253 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3659c640 1 -- 172.21.15.184:0/4198317853 >> v2:172.21.15.40:3300/0 conn(0x7f1e380b4d40 msgr2=0x7f1e380b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:47.253 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3659c640 1 --2- 172.21.15.184:0/4198317853 >> v2:172.21.15.40:3300/0 conn(0x7f1e380b4d40 0x7f1e380b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.253 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3659c640 1 -- 172.21.15.184:0/4198317853 >> v1:172.21.15.40:6789/0 conn(0x7f1e380b6c50 legacy=0x7f1e3815f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.253 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.251+0000 7f1e3659c640 1 -- 172.21.15.184:0/4198317853 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e38073290 con 0x7f1e380b0b70 2024-04-26T21:34:47.254 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.252+0000 7f1e3659c640 1 --2- 172.21.15.184:0/4198317853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b0b70 0x7f1e380b0f50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009870 tx=0x7f1e2802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c70415789e2f834 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.254 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.252+0000 7f1e3559a640 1 -- 172.21.15.184:0/4198317853 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28037070 con 0x7f1e380b0b70 2024-04-26T21:34:47.254 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.252+0000 7f1e3559a640 1 -- 172.21.15.184:0/4198317853 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e28032440 con 0x7f1e380b0b70 2024-04-26T21:34:47.254 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.252+0000 7f1e3559a640 1 -- 172.21.15.184:0/4198317853 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28032810 con 0x7f1e380b0b70 2024-04-26T21:34:47.254 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.252+0000 7f1e3ca20640 1 -- 172.21.15.184:0/4198317853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b0b70 msgr2=0x7f1e380b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.254 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.252+0000 7f1e3ca20640 1 --2- 172.21.15.184:0/4198317853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b0b70 0x7f1e380b0f50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009870 tx=0x7f1e2802a270 comp rx=0 tx=0).stop 2024-04-26T21:34:47.254 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.253+0000 7f1e3ca20640 1 -- 172.21.15.184:0/4198317853 shutdown_connections 2024-04-26T21:34:47.255 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.253+0000 7f1e3ca20640 1 --2- 172.21.15.184:0/4198317853 >> v2:172.21.15.40:3300/0 conn(0x7f1e380b4d40 0x7f1e380b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.255 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.253+0000 7f1e3ca20640 1 --2- 172.21.15.184:0/4198317853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b0b70 0x7f1e380b0f50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.255 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.253+0000 7f1e3ca20640 1 -- 172.21.15.184:0/4198317853 >> 172.21.15.184:0/4198317853 conn(0x7f1e380b39b0 msgr2=0x7f1e380b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.255 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.253+0000 7f1e3ca20640 1 -- 172.21.15.184:0/4198317853 shutdown_connections 2024-04-26T21:34:47.255 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.253+0000 7f1e3ca20640 1 -- 172.21.15.184:0/4198317853 wait complete. 2024-04-26T21:34:47.257 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.254+0000 7f1e3ca20640 1 Processor -- start 2024-04-26T21:34:47.257 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.254+0000 7f1e3ca20640 1 -- start start 2024-04-26T21:34:47.257 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3ca20640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e380b0b70 0x7f1e38146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3ca20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e380b4d40 0x7f1e38149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3ca20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b6c50 0x7f1e3806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3ca20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e380afda0 con 0x7f1e380b4d40 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3ca20640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38073850 con 0x7f1e380b0b70 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3ca20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1e380aff20 con 0x7f1e380b6c50 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e380b0b70 0x7f1e38146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e36d9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b6c50 0x7f1e3806c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e380b0b70 0x7f1e38146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38634/0 (socket says 172.21.15.184:38634) 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 -- 172.21.15.184:0/2273888100 learned_addr learned my addr 172.21.15.184:0/2273888100 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e35d9b640 1 --2- 172.21.15.184:0/2273888100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e380b4d40 0x7f1e38149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 -- 172.21.15.184:0/2273888100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b6c50 msgr2=0x7f1e3806c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 --2- 172.21.15.184:0/2273888100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b6c50 0x7f1e3806c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 -- 172.21.15.184:0/2273888100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e380b4d40 msgr2=0x7f1e38149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 --2- 172.21.15.184:0/2273888100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e380b4d40 0x7f1e38149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 -- 172.21.15.184:0/2273888100 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e38147150 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3659c640 1 --2- 172.21.15.184:0/2273888100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e380b0b70 0x7f1e38146b70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000c00 tx=0x7f1e280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e26ffd640 1 -- 172.21.15.184:0/2273888100 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28037040 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e26ffd640 1 -- 172.21.15.184:0/2273888100 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e28042070 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e38147430 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e26ffd640 1 -- 172.21.15.184:0/2273888100 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28030880 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.255+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e381479f0 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.256+0000 7f1e26ffd640 1 -- 172.21.15.184:0/2273888100 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e28030a20 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.256+0000 7f1e26ffd640 1 -- 172.21.15.184:0/2273888100 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f1e28047c10 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.257+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e380694f0 con 0x7f1e380b0b70 2024-04-26T21:34:47.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.258+0000 7f1e26ffd640 1 -- 172.21.15.184:0/2273888100 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1e28047460 con 0x7f1e380b0b70 2024-04-26T21:34:47.292 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:47.292+0000 7efe66a12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:34:47.313 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:47.312+0000 7efe9fdbf040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:34:47.332 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.331+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33", "id": 4} v 0) -- 0x7f1e3815b970 con 0x7f1e380b0b70 2024-04-26T21:34:47.337 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:47.337+0000 7efe66a12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:34:47.337 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.336+0000 7f1e26ffd640 1 -- 172.21.15.184:0/2273888100 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f1e28047e10 con 0x7f1e380b0b70 2024-04-26T21:34:47.337 INFO:teuthology.orchestra.run.smithi184.stdout:4 2024-04-26T21:34:47.338 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e380b0b70 msgr2=0x7f1e38146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.338 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 --2- 172.21.15.184:0/2273888100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e380b0b70 0x7f1e38146b70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000c00 tx=0x7f1e280046c0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 shutdown_connections 2024-04-26T21:34:47.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 --2- 172.21.15.184:0/2273888100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e380b6c50 0x7f1e3806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 --2- 172.21.15.184:0/2273888100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e380b4d40 0x7f1e38149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 --2- 172.21.15.184:0/2273888100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e380b0b70 0x7f1e38146b70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 >> 172.21.15.184:0/2273888100 conn(0x7f1e380b39b0 msgr2=0x7f1e380b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 shutdown_connections 2024-04-26T21:34:47.339 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.337+0000 7f1e3ca20640 1 -- 172.21.15.184:0/2273888100 wait complete. 2024-04-26T21:34:47.349 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new afd6e92c-4942-477c-9963-4930f3f9f17b 5 2024-04-26T21:34:47.358 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:47.357+0000 7efe9fdbf040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:34:47.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.445+0000 7ff0fa845640 1 Processor -- start 2024-04-26T21:34:47.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.445+0000 7ff0fa845640 1 -- start start 2024-04-26T21:34:47.452 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.445+0000 7ff0fa845640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b0b70 0x7ff0f40b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.452 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.445+0000 7ff0fa845640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff0f40b4d40 0x7ff0f40b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.452 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.445+0000 7ff0fa845640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0f40b1490 con 0x7ff0f40b6c50 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.445+0000 7ff0fa845640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f4073bc0 con 0x7ff0f40b4d40 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.445+0000 7ff0fa845640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f4072fb0 con 0x7ff0f40b0b70 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b0b70 0x7ff0f40b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b0b70 0x7ff0f40b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:47612/0 (socket says 172.21.15.184:47612) 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f3fff640 1 -- 172.21.15.184:0/3169619251 learned_addr learned my addr 172.21.15.184:0/3169619251 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f37fe640 1 --2- 172.21.15.184:0/3169619251 >> v2:172.21.15.40:3300/0 conn(0x7ff0f40b4d40 0x7ff0f40b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f3fff640 1 -- 172.21.15.184:0/3169619251 >> v2:172.21.15.40:3300/0 conn(0x7ff0f40b4d40 msgr2=0x7ff0f40b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f3fff640 1 --2- 172.21.15.184:0/3169619251 >> v2:172.21.15.40:3300/0 conn(0x7ff0f40b4d40 0x7ff0f40b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f2ffd640 1 -- 172.21.15.184:0/3169619251 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3122694129 0 0) 0x7ff0f40b1490 con 0x7ff0f40b6c50 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f3fff640 1 -- 172.21.15.184:0/3169619251 >> v1:172.21.15.40:6789/0 conn(0x7ff0f40b6c50 legacy=0x7ff0f415f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.446+0000 7ff0f3fff640 1 -- 172.21.15.184:0/3169619251 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f4073290 con 0x7ff0f40b0b70 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.447+0000 7ff0f3fff640 1 --2- 172.21.15.184:0/3169619251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b0b70 0x7ff0f40b0f50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009870 tx=0x7ff0e402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c6bbd2099175e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.447+0000 7ff0f2ffd640 1 -- 172.21.15.184:0/3169619251 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4037070 con 0x7ff0f40b0b70 2024-04-26T21:34:47.453 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.447+0000 7ff0f2ffd640 1 -- 172.21.15.184:0/3169619251 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e4032440 con 0x7ff0f40b0b70 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.447+0000 7ff0f2ffd640 1 -- 172.21.15.184:0/3169619251 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4032810 con 0x7ff0f40b0b70 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.447+0000 7ff0fa845640 1 -- 172.21.15.184:0/3169619251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b0b70 msgr2=0x7ff0f40b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.447+0000 7ff0fa845640 1 --2- 172.21.15.184:0/3169619251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b0b70 0x7ff0f40b0f50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009870 tx=0x7ff0e402a270 comp rx=0 tx=0).stop 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.448+0000 7ff0fa845640 1 -- 172.21.15.184:0/3169619251 shutdown_connections 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.448+0000 7ff0fa845640 1 --2- 172.21.15.184:0/3169619251 >> v2:172.21.15.40:3300/0 conn(0x7ff0f40b4d40 0x7ff0f40b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.448+0000 7ff0fa845640 1 --2- 172.21.15.184:0/3169619251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b0b70 0x7ff0f40b0f50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.448+0000 7ff0fa845640 1 -- 172.21.15.184:0/3169619251 >> 172.21.15.184:0/3169619251 conn(0x7ff0f40b39b0 msgr2=0x7ff0f40b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.448+0000 7ff0fa845640 1 -- 172.21.15.184:0/3169619251 shutdown_connections 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.448+0000 7ff0fa845640 1 -- 172.21.15.184:0/3169619251 wait complete. 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.449+0000 7ff0fa845640 1 Processor -- start 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.449+0000 7ff0fa845640 1 -- start start 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.449+0000 7ff0fa845640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0f40b0b70 0x7ff0f40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0fa845640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b4d40 0x7ff0f41553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0fa845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0f40b6c50 0x7ff0f4157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0fa845640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f40afda0 con 0x7ff0f40b6c50 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0fa845640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4073850 con 0x7ff0f40b0b70 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0fa845640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f40aff20 con 0x7ff0f40b4d40 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0f40b0b70 0x7ff0f40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f37fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b4d40 0x7ff0f41553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0f40b0b70 0x7ff0f40cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38640/0 (socket says 172.21.15.184:38640) 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 -- 172.21.15.184:0/2173662393 learned_addr learned my addr 172.21.15.184:0/2173662393 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f37fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b4d40 0x7ff0f41553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:47626/0 (socket says 172.21.15.184:47626) 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f8de2640 1 --2- 172.21.15.184:0/2173662393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0f40b6c50 0x7ff0f4157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 -- 172.21.15.184:0/2173662393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b4d40 msgr2=0x7ff0f41553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 --2- 172.21.15.184:0/2173662393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b4d40 0x7ff0f41553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 -- 172.21.15.184:0/2173662393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0f40b6c50 msgr2=0x7ff0f4157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 --2- 172.21.15.184:0/2173662393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0f40b6c50 0x7ff0f4157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.455 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 -- 172.21.15.184:0/2173662393 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f4157a00 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f3fff640 1 --2- 172.21.15.184:0/2173662393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0f40b0b70 0x7ff0f40cf340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009870 tx=0x7ff0e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f0ff9640 1 -- 172.21.15.184:0/2173662393 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4041020 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f0ff9640 1 -- 172.21.15.184:0/2173662393 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e4030640 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f41605b0 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0f0ff9640 1 -- 172.21.15.184:0/2173662393 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4037040 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.450+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f4160b70 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.451+0000 7ff0f0ff9640 1 -- 172.21.15.184:0/2173662393 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0e4044020 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.451+0000 7ff0f0ff9640 1 -- 172.21.15.184:0/2173662393 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7ff0e403dba0 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.451+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f41603a0 con 0x7ff0f40b0b70 2024-04-26T21:34:47.456 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.453+0000 7ff0f0ff9640 1 -- 172.21.15.184:0/2173662393 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff0e404b030 con 0x7ff0f40b0b70 2024-04-26T21:34:47.511 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:47.511+0000 7efe66a12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:34:47.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.524+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "afd6e92c-4942-477c-9963-4930f3f9f17b", "id": 5} v 0) -- 0x7ff0f40694f0 con 0x7ff0f40b0b70 2024-04-26T21:34:47.529 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:47.528+0000 7efe9fdbf040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:34:47.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.529+0000 7ff0f0ff9640 1 -- 172.21.15.184:0/2173662393 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "afd6e92c-4942-477c-9963-4930f3f9f17b", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7ff0e403d420 con 0x7ff0f40b0b70 2024-04-26T21:34:47.530 INFO:teuthology.orchestra.run.smithi184.stdout:5 2024-04-26T21:34:47.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0f40b0b70 msgr2=0x7ff0f40cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 --2- 172.21.15.184:0/2173662393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0f40b0b70 0x7ff0f40cf340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009870 tx=0x7ff0e40046c0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 shutdown_connections 2024-04-26T21:34:47.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 --2- 172.21.15.184:0/2173662393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0f40b6c50 0x7ff0f4157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 --2- 172.21.15.184:0/2173662393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f40b4d40 0x7ff0f41553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 --2- 172.21.15.184:0/2173662393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0f40b0b70 0x7ff0f40cf340 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 >> 172.21.15.184:0/2173662393 conn(0x7ff0f40b39b0 msgr2=0x7ff0f40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.532 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 shutdown_connections 2024-04-26T21:34:47.532 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.530+0000 7ff0fa845640 1 -- 172.21.15.184:0/2173662393 wait complete. 2024-04-26T21:34:47.541 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new b2bcbe55-8d3f-4ea4-806b-c5b5016466c1 6 2024-04-26T21:34:47.561 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:47.561+0000 7efe66a12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:34:47.579 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:47.578+0000 7efe9fdbf040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:34:47.616 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:47.616+0000 7efe66a12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:34:47.635 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:47.634+0000 7efe9fdbf040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:34:47.676 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.674+0000 7fa0e0007640 1 Processor -- start 2024-04-26T21:34:47.676 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.675+0000 7fa0e0007640 1 -- start start 2024-04-26T21:34:47.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.675+0000 7fa0e0007640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0d80736c0 0x7fa0d815f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.675+0000 7fa0e0007640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d81605a0 con 0x7fa0d80d1350 2024-04-26T21:34:47.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.675+0000 7fa0e0007640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d815c4b0 con 0x7fa0d815cf90 2024-04-26T21:34:47.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.675+0000 7fa0e0007640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d8160880 con 0x7fa0d80736c0 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0de5a4640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fa0d815cf90 0x7fa0d80c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:42538/0 (socket says 172.21.15.184:42538) 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0de5a4640 1 -- 172.21.15.184:0/1390960129 learned_addr learned my addr 172.21.15.184:0/1390960129 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0dd5a2640 1 --2- 172.21.15.184:0/1390960129 >> v2:172.21.15.40:3300/0 conn(0x7fa0d80736c0 0x7fa0d815f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0dcda1640 1 -- 172.21.15.184:0/1390960129 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937017188 0 0) 0x7fa0d815c4b0 con 0x7fa0d815cf90 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0dcda1640 1 -- 172.21.15.184:0/1390960129 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0c4005730 con 0x7fa0d815cf90 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0dd5a2640 1 -- 172.21.15.184:0/1390960129 >> v1:172.21.15.184:6789/0 conn(0x7fa0d815cf90 legacy=0x7fa0d80c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0dd5a2640 1 -- 172.21.15.184:0/1390960129 >> v1:172.21.15.40:6789/0 conn(0x7fa0d80d1350 legacy=0x7fa0d8072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0dcda1640 1 -- 172.21.15.184:0/1390960129 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1391124558 0 0) 0x7fa0c4005730 con 0x7fa0d815cf90 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.676+0000 7fa0dd5a2640 1 -- 172.21.15.184:0/1390960129 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d815c790 con 0x7fa0d80736c0 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.677+0000 7fa0dd5a2640 1 --2- 172.21.15.184:0/1390960129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d80736c0 0x7fa0d815f900 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8160a00 tx=0x7fa0c802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9f20285f77d86b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.677+0000 7fa0dcda1640 1 -- 172.21.15.184:0/1390960129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8037070 con 0x7fa0d80736c0 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.677+0000 7fa0dcda1640 1 -- 172.21.15.184:0/1390960129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c802ad10 con 0x7fa0d80736c0 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.677+0000 7fa0dcda1640 1 -- 172.21.15.184:0/1390960129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8032660 con 0x7fa0d80736c0 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.678+0000 7fa0e0007640 1 -- 172.21.15.184:0/1390960129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d80736c0 msgr2=0x7fa0d815f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.678+0000 7fa0e0007640 1 --2- 172.21.15.184:0/1390960129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d80736c0 0x7fa0d815f900 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8160a00 tx=0x7fa0c802a080 comp rx=0 tx=0).stop 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.678+0000 7fa0e0007640 1 -- 172.21.15.184:0/1390960129 shutdown_connections 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.678+0000 7fa0e0007640 1 --2- 172.21.15.184:0/1390960129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d80736c0 0x7fa0d815f900 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.678+0000 7fa0e0007640 1 -- 172.21.15.184:0/1390960129 >> 172.21.15.184:0/1390960129 conn(0x7fa0d80d1c90 msgr2=0x7fa0d80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.678+0000 7fa0e0007640 1 -- 172.21.15.184:0/1390960129 shutdown_connections 2024-04-26T21:34:47.681 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.678+0000 7fa0e0007640 1 -- 172.21.15.184:0/1390960129 wait complete. 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.679+0000 7fa0e0007640 1 Processor -- start 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.679+0000 7fa0e0007640 1 -- start start 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.679+0000 7fa0e0007640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d80736c0 0x7fa0d815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0e0007640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d80d1350 0x7fa0d815e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0e0007640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d815cf90 0x7fa0d8141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0e0007640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d80d18b0 con 0x7fa0d815cf90 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0e0007640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d80d1730 con 0x7fa0d80736c0 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0e0007640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d815fe40 con 0x7fa0d80d1350 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d80736c0 0x7fa0d815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d80736c0 0x7fa0d815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38644/0 (socket says 172.21.15.184:38644) 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 -- 172.21.15.184:0/2409114671 learned_addr learned my addr 172.21.15.184:0/2409114671 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0dd5a2640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d80d1350 0x7fa0d815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0de5a4640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d815cf90 0x7fa0d8141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 -- 172.21.15.184:0/2409114671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d80d1350 msgr2=0x7fa0d815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d80d1350 0x7fa0d815e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 -- 172.21.15.184:0/2409114671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d815cf90 msgr2=0x7fa0d8141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d815cf90 0x7fa0d8141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 -- 172.21.15.184:0/2409114671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d80c96b0 con 0x7fa0d80736c0 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ddda3640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d80736c0 0x7fa0d815d4b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c750 tx=0x7fa0d400cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ce7fc640 1 -- 172.21.15.184:0/2409114671 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4017070 con 0x7fa0d80736c0 2024-04-26T21:34:47.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ce7fc640 1 -- 172.21.15.184:0/2409114671 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d4005270 con 0x7fa0d80736c0 2024-04-26T21:34:47.686 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0ce7fc640 1 -- 172.21.15.184:0/2409114671 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400a640 con 0x7fa0d80736c0 2024-04-26T21:34:47.686 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d806fe90 con 0x7fa0d80736c0 2024-04-26T21:34:47.686 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.680+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d8070450 con 0x7fa0d80736c0 2024-04-26T21:34:47.686 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.681+0000 7fa0ce7fc640 1 -- 172.21.15.184:0/2409114671 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa0d401a070 con 0x7fa0d80736c0 2024-04-26T21:34:47.686 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.681+0000 7fa0ce7fc640 1 -- 172.21.15.184:0/2409114671 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fa0d4003b40 con 0x7fa0d80736c0 2024-04-26T21:34:47.686 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.681+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d80694f0 con 0x7fa0d80736c0 2024-04-26T21:34:47.686 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.683+0000 7fa0ce7fc640 1 -- 172.21.15.184:0/2409114671 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa0d4021020 con 0x7fa0d80736c0 2024-04-26T21:34:47.734 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:47.734+0000 7efe66a12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:34:47.751 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:47.750+0000 7efe9fdbf040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:34:47.757 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.756+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b2bcbe55-8d3f-4ea4-806b-c5b5016466c1", "id": 6} v 0) -- 0x7fa0d80c9880 con 0x7fa0d80736c0 2024-04-26T21:34:47.762 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.761+0000 7fa0ce7fc640 1 -- 172.21.15.184:0/2409114671 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b2bcbe55-8d3f-4ea4-806b-c5b5016466c1", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fa0d4003720 con 0x7fa0d80736c0 2024-04-26T21:34:47.762 INFO:teuthology.orchestra.run.smithi184.stdout:6 2024-04-26T21:34:47.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d80736c0 msgr2=0x7fa0d815d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d80736c0 0x7fa0d815d4b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c750 tx=0x7fa0d400cc10 comp rx=0 tx=0).stop 2024-04-26T21:34:47.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 shutdown_connections 2024-04-26T21:34:47.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d815cf90 0x7fa0d8141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d80d1350 0x7fa0d815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 --2- 172.21.15.184:0/2409114671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d80736c0 0x7fa0d815d4b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 >> 172.21.15.184:0/2409114671 conn(0x7fa0d80d1c90 msgr2=0x7fa0d806c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.764 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 shutdown_connections 2024-04-26T21:34:47.764 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.762+0000 7fa0e0007640 1 -- 172.21.15.184:0/2409114671 wait complete. 2024-04-26T21:34:47.773 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new c57e4b4b-5307-4e1e-ac19-9f59168f08ae 7 2024-04-26T21:34:47.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.870+0000 7fd0d5140640 1 Processor -- start 2024-04-26T21:34:47.871 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.870+0000 7fd0d5140640 1 -- start start 2024-04-26T21:34:47.874 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0d5140640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0d015bcc0 0x7fd0d00cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.874 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0d5140640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd0d0071110 0x7fd0d0071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0d5140640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d00cfbf0 con 0x7fd0d0072f50 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0d5140640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d00d0800 con 0x7fd0d015bcc0 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0d5140640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d00cfed0 con 0x7fd0d0071110 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0ce59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd0d0071110 0x7fd0d0071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0ce59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd0d0071110 0x7fd0d0071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38660/0 (socket says 172.21.15.184:38660) 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0ce59c640 1 -- 172.21.15.184:0/362962968 learned_addr learned my addr 172.21.15.184:0/362962968 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0ced9d640 1 --2- 172.21.15.184:0/362962968 >> v2:172.21.15.40:3300/0 conn(0x7fd0d015bcc0 0x7fd0d00cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0ce59c640 1 -- 172.21.15.184:0/362962968 >> v2:172.21.15.40:3300/0 conn(0x7fd0d015bcc0 msgr2=0x7fd0d00cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0ce59c640 1 --2- 172.21.15.184:0/362962968 >> v2:172.21.15.40:3300/0 conn(0x7fd0d015bcc0 0x7fd0d00cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0ce59c640 1 -- 172.21.15.184:0/362962968 >> v1:172.21.15.40:6789/0 conn(0x7fd0d0072f50 legacy=0x7fd0d00d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.871+0000 7fd0ce59c640 1 -- 172.21.15.184:0/362962968 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d00d01b0 con 0x7fd0d0071110 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.872+0000 7fd0ce59c640 1 --2- 172.21.15.184:0/362962968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0071570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00d0980 tx=0x7fd0c002c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ff45801e5bf2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.872+0000 7fd0cdd9b640 1 -- 172.21.15.184:0/362962968 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0036020 con 0x7fd0d0071110 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.872+0000 7fd0cdd9b640 1 -- 172.21.15.184:0/362962968 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c0032820 con 0x7fd0d0071110 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.872+0000 7fd0cdd9b640 1 -- 172.21.15.184:0/362962968 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0032bf0 con 0x7fd0d0071110 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.873+0000 7fd0d5140640 1 -- 172.21.15.184:0/362962968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 msgr2=0x7fd0d0071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.873+0000 7fd0d5140640 1 --2- 172.21.15.184:0/362962968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0071570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00d0980 tx=0x7fd0c002c920 comp rx=0 tx=0).stop 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.873+0000 7fd0d5140640 1 -- 172.21.15.184:0/362962968 shutdown_connections 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.873+0000 7fd0d5140640 1 --2- 172.21.15.184:0/362962968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0071570 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.873+0000 7fd0d5140640 1 --2- 172.21.15.184:0/362962968 >> v2:172.21.15.40:3300/0 conn(0x7fd0d015bcc0 0x7fd0d00cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.873+0000 7fd0d5140640 1 -- 172.21.15.184:0/362962968 >> 172.21.15.184:0/362962968 conn(0x7fd0d00d1cb0 msgr2=0x7fd0d00d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.873+0000 7fd0d5140640 1 -- 172.21.15.184:0/362962968 shutdown_connections 2024-04-26T21:34:47.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.873+0000 7fd0d5140640 1 -- 172.21.15.184:0/362962968 wait complete. 2024-04-26T21:34:47.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.874+0000 7fd0d5140640 1 Processor -- start 2024-04-26T21:34:47.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.874+0000 7fd0d5140640 1 -- start start 2024-04-26T21:34:47.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0d5140640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0d5140640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0d0072f50 0x7fd0d00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0d5140640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d015bcc0 0x7fd0d0158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:47.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0d5140640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d00cc840 con 0x7fd0d0072f50 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0d5140640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d00cf880 con 0x7fd0d0071110 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0d5140640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d00cc9c0 con 0x7fd0d015bcc0 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0ced9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0cf59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d015bcc0 0x7fd0d0158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0ced9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38676/0 (socket says 172.21.15.184:38676) 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0ced9d640 1 -- 172.21.15.184:0/1818647218 learned_addr learned my addr 172.21.15.184:0/1818647218 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0ce59c640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0d0072f50 0x7fd0d00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0cf59e640 1 -- 172.21.15.184:0/1818647218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 msgr2=0x7fd0d0153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0cf59e640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0cf59e640 1 -- 172.21.15.184:0/1818647218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0d0072f50 msgr2=0x7fd0d00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0cf59e640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0d0072f50 0x7fd0d00cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0cf59e640 1 -- 172.21.15.184:0/1818647218 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d015e870 con 0x7fd0d015bcc0 2024-04-26T21:34:47.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0ced9d640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.875+0000 7fd0ce59c640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0d0072f50 0x7fd0d00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.876+0000 7fd0cf59e640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d015bcc0 0x7fd0d0158920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8001e50 tx=0x7fd0c8010420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.876+0000 7fd0b77fe640 1 -- 172.21.15.184:0/1818647218 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800e0d0 con 0x7fd0d015bcc0 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.876+0000 7fd0b77fe640 1 -- 172.21.15.184:0/1818647218 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c800e270 con 0x7fd0d015bcc0 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.876+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d015eb50 con 0x7fd0d015bcc0 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.876+0000 7fd0b77fe640 1 -- 172.21.15.184:0/1818647218 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8010d70 con 0x7fd0d015bcc0 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.876+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d00c8fd0 con 0x7fd0d015bcc0 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.876+0000 7fd0b77fe640 1 -- 172.21.15.184:0/1818647218 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd0c8003d00 con 0x7fd0d015bcc0 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.876+0000 7fd0b77fe640 1 -- 172.21.15.184:0/1818647218 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd0c800cd30 con 0x7fd0d015bcc0 2024-04-26T21:34:47.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.877+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d00694f0 con 0x7fd0d015bcc0 2024-04-26T21:34:47.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.879+0000 7fd0b77fe640 1 -- 172.21.15.184:0/1818647218 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd0c800c6d0 con 0x7fd0d015bcc0 2024-04-26T21:34:47.952 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.951+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c57e4b4b-5307-4e1e-ac19-9f59168f08ae", "id": 7} v 0) -- 0x7fd0d0156890 con 0x7fd0d015bcc0 2024-04-26T21:34:47.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.959+0000 7fd0b77fe640 1 -- 172.21.15.184:0/1818647218 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c57e4b4b-5307-4e1e-ac19-9f59168f08ae", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd0c800c980 con 0x7fd0d015bcc0 2024-04-26T21:34:47.960 INFO:teuthology.orchestra.run.smithi184.stdout:7 2024-04-26T21:34:47.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d015bcc0 msgr2=0x7fd0d0158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:47.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d015bcc0 0x7fd0d0158920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8001e50 tx=0x7fd0c8010420 comp rx=0 tx=0).stop 2024-04-26T21:34:47.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 shutdown_connections 2024-04-26T21:34:47.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d015bcc0 0x7fd0d0158920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0d0072f50 0x7fd0d00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 --2- 172.21.15.184:0/1818647218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0071110 0x7fd0d0153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:47.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 >> 172.21.15.184:0/1818647218 conn(0x7fd0d00d1cb0 msgr2=0x7fd0d0157090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:47.962 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 shutdown_connections 2024-04-26T21:34:47.962 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:34:47.960+0000 7fd0d5140640 1 -- 172.21.15.184:0/1818647218 wait complete. 2024-04-26T21:34:47.972 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T21:34:47.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T21:34:47.974 INFO:tasks.ceph.osd.0:Started 2024-04-26T21:34:47.974 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T21:34:47.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T21:34:47.978 INFO:tasks.ceph.osd.1:Started 2024-04-26T21:34:47.978 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T21:34:47.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T21:34:47.981 INFO:tasks.ceph.osd.2:Started 2024-04-26T21:34:47.981 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T21:34:47.981 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T21:34:47.983 INFO:tasks.ceph.osd.3:Started 2024-04-26T21:34:47.983 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T21:34:47.983 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T21:34:47.986 INFO:tasks.ceph.osd.4:Started 2024-04-26T21:34:47.986 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T21:34:47.986 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T21:34:47.988 INFO:tasks.ceph.osd.5:Started 2024-04-26T21:34:47.989 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T21:34:47.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T21:34:47.991 INFO:tasks.ceph.osd.6:Started 2024-04-26T21:34:47.991 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T21:34:47.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T21:34:47.993 INFO:tasks.ceph.osd.7:Started 2024-04-26T21:34:47.994 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:34:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.116+0000 7ffb9c338640 1 Processor -- start 2024-04-26T21:34:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.116+0000 7ffb9c338640 1 -- start start 2024-04-26T21:34:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb9c338640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffb9415ecc0 0x7ffb9406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb9c338640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9406eb50 con 0x7ffb9406c240 2024-04-26T21:34:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb9c338640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9406ee30 con 0x7ffb9406f910 2024-04-26T21:34:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb9a8d5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffb9406c240 0x7ffb940b4410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47490/0 (socket says 172.21.15.40:47490) 2024-04-26T21:34:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb9c338640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffb940705b0 con 0x7ffb9415ecc0 2024-04-26T21:34:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb9a8d5640 1 -- 172.21.15.40:0/544018840 learned_addr learned my addr 172.21.15.40:0/544018840 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:34:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb9a0d4640 1 --2- 172.21.15.40:0/544018840 >> v2:172.21.15.184:3300/0 conn(0x7ffb9415ecc0 0x7ffb9406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb990d2640 1 -- 172.21.15.40:0/544018840 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995298246 0 0) 0x7ffb9406eb50 con 0x7ffb9406c240 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb990d2640 1 -- 172.21.15.40:0/544018840 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb80003520 con 0x7ffb9406c240 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb990d2640 1 -- 172.21.15.40:0/544018840 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576388729 0 0) 0x7ffb9406ee30 con 0x7ffb9406f910 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.117+0000 7ffb990d2640 1 -- 172.21.15.40:0/544018840 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb9406eb50 con 0x7ffb9406f910 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.118+0000 7ffb9a0d4640 1 -- 172.21.15.40:0/544018840 >> v1:172.21.15.184:6789/0 conn(0x7ffb9406f910 legacy=0x7ffb9415fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.118+0000 7ffb9a0d4640 1 -- 172.21.15.40:0/544018840 >> v1:172.21.15.40:6789/0 conn(0x7ffb9406c240 legacy=0x7ffb940b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.118+0000 7ffb9a0d4640 1 -- 172.21.15.40:0/544018840 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9406f110 con 0x7ffb9415ecc0 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.118+0000 7ffb9a0d4640 1 --2- 172.21.15.40:0/544018840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9415ecc0 0x7ffb9406ff60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb900097a0 tx=0x7ffb9002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89497f00cb3e5002 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb990d2640 1 -- 172.21.15.40:0/544018840 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90037070 con 0x7ffb9415ecc0 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb990d2640 1 -- 172.21.15.40:0/544018840 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb90032440 con 0x7ffb9415ecc0 2024-04-26T21:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb990d2640 1 -- 172.21.15.40:0/544018840 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9002ae60 con 0x7ffb9415ecc0 2024-04-26T21:34:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb9c338640 1 -- 172.21.15.40:0/544018840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9415ecc0 msgr2=0x7ffb9406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb9c338640 1 --2- 172.21.15.40:0/544018840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9415ecc0 0x7ffb9406ff60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb900097a0 tx=0x7ffb9002a110 comp rx=0 tx=0).stop 2024-04-26T21:34:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb9c338640 1 -- 172.21.15.40:0/544018840 shutdown_connections 2024-04-26T21:34:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb9c338640 1 --2- 172.21.15.40:0/544018840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9415ecc0 0x7ffb9406ff60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb9c338640 1 -- 172.21.15.40:0/544018840 >> 172.21.15.40:0/544018840 conn(0x7ffb940b3110 msgr2=0x7ffb940b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb9c338640 1 -- 172.21.15.40:0/544018840 shutdown_connections 2024-04-26T21:34:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.119+0000 7ffb9c338640 1 -- 172.21.15.40:0/544018840 wait complete. 2024-04-26T21:34:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.120+0000 7ffb9c338640 1 Processor -- start 2024-04-26T21:34:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9c338640 1 -- start start 2024-04-26T21:34:48.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9c338640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb9406c240 0x7ffb9415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:48.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9c338640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9406f910 0x7ffb9414c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:48.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9c338640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb9415ecc0 0x7ffb9415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9c338640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9406c810 con 0x7ffb9406c240 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9c338640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9406c690 con 0x7ffb9406f910 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9c338640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9406c990 con 0x7ffb9415ecc0 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9a0d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb9406c240 0x7ffb9415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9a0d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb9406c240 0x7ffb9415bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47322/0 (socket says 172.21.15.40:47322) 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9a0d4640 1 -- 172.21.15.40:0/215482000 learned_addr learned my addr 172.21.15.40:0/215482000 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb998d3640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9406f910 0x7ffb9414c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.121+0000 7ffb9a8d5640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb9415ecc0 0x7ffb9415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb9a0d4640 1 -- 172.21.15.40:0/215482000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb9415ecc0 msgr2=0x7ffb9415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb9a0d4640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb9415ecc0 0x7ffb9415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb9a0d4640 1 -- 172.21.15.40:0/215482000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9406f910 msgr2=0x7ffb9414c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb9a0d4640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9406f910 0x7ffb9414c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb9a0d4640 1 -- 172.21.15.40:0/215482000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9414c970 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb9a0d4640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb9406c240 0x7ffb9415bb50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb90035040 tx=0x7ffb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb8affd640 1 -- 172.21.15.40:0/215482000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90037040 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb8affd640 1 -- 172.21.15.40:0/215482000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb90042070 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9415ccc0 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb8affd640 1 -- 172.21.15.40:0/215482000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb900044e0 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9415b070 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.122+0000 7ffb8affd640 1 -- 172.21.15.40:0/215482000 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb90032830 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.123+0000 7ffb8affd640 1 -- 172.21.15.40:0/215482000 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ffb9003db10 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.123+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb94068fc0 con 0x7ffb9406c240 2024-04-26T21:34:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.126+0000 7ffb8affd640 1 -- 172.21.15.40:0/215482000 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ffb9414c970 con 0x7ffb9406c240 2024-04-26T21:34:48.172 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:48.172+0000 7efe66a12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:34:48.176 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:48.175+0000 7efe9fdbf040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:34:48.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.188+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb940cdc90 con 0x7ffb9406c240 2024-04-26T21:34:48.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.189+0000 7ffb8affd640 1 -- 172.21.15.40:0/215482000 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ffb9003d440 con 0x7ffb9406c240 2024-04-26T21:34:48.189 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:34:48.189 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:34:47.953711+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:34:48.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb9406c240 msgr2=0x7ffb9415bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:48.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb9406c240 0x7ffb9415bb50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb90035040 tx=0x7ffb900046c0 comp rx=0 tx=0).stop 2024-04-26T21:34:48.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 shutdown_connections 2024-04-26T21:34:48.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb9415ecc0 0x7ffb9415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb9406f910 0x7ffb9414c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 --2- 172.21.15.40:0/215482000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb9406c240 0x7ffb9415bb50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 >> 172.21.15.40:0/215482000 conn(0x7ffb940b3110 msgr2=0x7ffb940b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 shutdown_connections 2024-04-26T21:34:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:34:48.190+0000 7ffb9c338640 1 -- 172.21.15.40:0/215482000 wait complete. 2024-04-26T21:34:48.201 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T21:34:48.201 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T21:34:48.433 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:48.432+0000 7efe9fdbf040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:34:48.445 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:48.444+0000 7efe66a12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:34:48.484 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:48.483+0000 7efe9fdbf040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:34:48.499 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:48.499+0000 7efe66a12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:34:48.579 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-04-26T21:34:48.578+0000 7f4dc44e3640 -1 Falling back to public interface 2024-04-26T21:34:48.584 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-26T21:34:48.584+0000 7f53a7e32640 -1 Falling back to public interface 2024-04-26T21:34:48.587 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-04-26T21:34:48.586+0000 7faf34417640 -1 Falling back to public interface 2024-04-26T21:34:48.587 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-04-26T21:34:48.586+0000 7f9d98a18640 -1 Falling back to public interface 2024-04-26T21:34:48.599 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:48.598+0000 7efe9fdbf040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:34:48.600 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-26T21:34:48.601+0000 7f6d39901640 -1 Falling back to public interface 2024-04-26T21:34:48.601 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:34:48.601+0000 7efe5ce11640 -1 Falling back to public interface 2024-04-26T21:34:48.601 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:34:48.601+0000 7efe5b7b1640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6810/0: (98) Address already in use 2024-04-26T21:34:48.604 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-26T21:34:48.604+0000 7efd71321640 -1 Falling back to public interface 2024-04-26T21:34:48.619 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-04-26T21:34:48.618+0000 7f7819dbe640 -1 Falling back to public interface 2024-04-26T21:34:48.623 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:48.623+0000 7efe66a12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:34:48.880 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:48.879+0000 7efe9fdbf040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:34:48.917 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:48.917+0000 7efe66a12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:34:48.928 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:48.927+0000 7efe9fdbf040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:34:48.967 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:48.967+0000 7efe66a12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:34:49.007 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:34:49.085 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:49.083+0000 7efe9fdbf040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:34:49.099 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.099+0000 7fbccfe20640 1 Processor -- start 2024-04-26T21:34:49.099 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.099+0000 7fbccfe20640 1 -- start start 2024-04-26T21:34:49.100 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccfe20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 0x7fbcc80d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:49.100 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccfe20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcc80b2690 0x7fbcc80d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccfe20640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcc806e860 0x7fbcc806ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccfe20640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc80cd410 con 0x7fbcc80b2690 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccfe20640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc80cdf90 con 0x7fbcc806e860 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccfe20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc80cd660 con 0x7fbcc80b4450 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccd3bb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcc80b2690 0x7fbcc80d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccd3bb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcc80b2690 0x7fbcc80d1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55278/0 (socket says 172.21.15.40:55278) 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccd3bb640 1 -- 172.21.15.40:0/809557724 learned_addr learned my addr 172.21.15.40:0/809557724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccdbbc640 1 --2- 172.21.15.40:0/809557724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 0x7fbcc80d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbcce3bd640 1 --2- 172.21.15.40:0/809557724 >> v2:172.21.15.184:3300/0 conn(0x7fbcc806e860 0x7fbcc806ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccd3bb640 1 -- 172.21.15.40:0/809557724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 msgr2=0x7fbcc80d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccd3bb640 1 --2- 172.21.15.40:0/809557724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 0x7fbcc80d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccd3bb640 1 -- 172.21.15.40:0/809557724 >> v2:172.21.15.184:3300/0 conn(0x7fbcc806e860 msgr2=0x7fbcc806ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccd3bb640 1 --2- 172.21.15.40:0/809557724 >> v2:172.21.15.184:3300/0 conn(0x7fbcc806e860 0x7fbcc806ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.101 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.100+0000 7fbccd3bb640 1 -- 172.21.15.40:0/809557724 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc80cd940 con 0x7fbcc80b2690 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.101+0000 7fbccd3bb640 1 --2- 172.21.15.40:0/809557724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80d1630 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8073410 tx=0x7fbcb802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7f82a726b995f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.101+0000 7fbcccbba640 1 -- 172.21.15.40:0/809557724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8037070 con 0x7fbcc80b2690 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.101+0000 7fbcccbba640 1 -- 172.21.15.40:0/809557724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb8032440 con 0x7fbcc80b2690 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.101+0000 7fbcccbba640 1 -- 172.21.15.40:0/809557724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8032810 con 0x7fbcc80b2690 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 -- 172.21.15.40:0/809557724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 msgr2=0x7fbcc80d1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 --2- 172.21.15.40:0/809557724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80d1630 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8073410 tx=0x7fbcb802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 -- 172.21.15.40:0/809557724 shutdown_connections 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 --2- 172.21.15.40:0/809557724 >> v2:172.21.15.184:3300/0 conn(0x7fbcc806e860 0x7fbcc806ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.103 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 --2- 172.21.15.40:0/809557724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80d1630 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.104 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 --2- 172.21.15.40:0/809557724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 0x7fbcc80d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.104 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 -- 172.21.15.40:0/809557724 >> 172.21.15.40:0/809557724 conn(0x7fbcc80d2dc0 msgr2=0x7fbcc80d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:49.104 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 -- 172.21.15.40:0/809557724 shutdown_connections 2024-04-26T21:34:49.104 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.102+0000 7fbccfe20640 1 -- 172.21.15.40:0/809557724 wait complete. 2024-04-26T21:34:49.104 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.103+0000 7fbccfe20640 1 Processor -- start 2024-04-26T21:34:49.104 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.103+0000 7fbccfe20640 1 -- start start 2024-04-26T21:34:49.106 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccfe20640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc806e860 0x7fbcc8154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:49.106 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccfe20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:49.106 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccfe20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 0x7fbcc8161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:49.106 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccfe20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc80ce420 con 0x7fbcc80b2690 2024-04-26T21:34:49.106 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccfe20640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc806f230 con 0x7fbcc806e860 2024-04-26T21:34:49.106 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccfe20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc80ce5a0 con 0x7fbcc80b4450 2024-04-26T21:34:49.106 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:49.106 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55294/0 (socket says 172.21.15.40:55294) 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 -- 172.21.15.40:0/2891067535 learned_addr learned my addr 172.21.15.40:0/2891067535 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccdbbc640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc806e860 0x7fbcc8154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbcce3bd640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 0x7fbcc8161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 -- 172.21.15.40:0/2891067535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 msgr2=0x7fbcc8161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 0x7fbcc8161110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 -- 172.21.15.40:0/2891067535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc806e860 msgr2=0x7fbcc8154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc806e860 0x7fbcc8154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 -- 172.21.15.40:0/2891067535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc81555e0 con 0x7fbcc80b2690 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.104+0000 7fbccd3bb640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80c9e90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80097c0 tx=0x7fbcb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.105+0000 7fbcbe7fc640 1 -- 172.21.15.40:0/2891067535 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8041020 con 0x7fbcc80b2690 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.105+0000 7fbcbe7fc640 1 -- 172.21.15.40:0/2891067535 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb803d620 con 0x7fbcc80b2690 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.105+0000 7fbcbe7fc640 1 -- 172.21.15.40:0/2891067535 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8037040 con 0x7fbcc80b2690 2024-04-26T21:34:49.107 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.105+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc80cc9a0 con 0x7fbcc80b2690 2024-04-26T21:34:49.108 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.105+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc8156ad0 con 0x7fbcc80b2690 2024-04-26T21:34:49.108 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.105+0000 7fbcbe7fc640 1 -- 172.21.15.40:0/2891067535 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbcb8044020 con 0x7fbcc80b2690 2024-04-26T21:34:49.108 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.105+0000 7fbcbe7fc640 1 -- 172.21.15.40:0/2891067535 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fbcb8046c20 con 0x7fbcc80b2690 2024-04-26T21:34:49.108 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.106+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc8076610 con 0x7fbcc80b2690 2024-04-26T21:34:49.109 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.109+0000 7fbcbe7fc640 1 -- 172.21.15.40:0/2891067535 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbcb804b030 con 0x7fbcc80b2690 2024-04-26T21:34:49.135 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:49.135+0000 7efe66a12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:34:49.172 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.172+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcc80d0240 con 0x7fbcc80b2690 2024-04-26T21:34:49.173 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.173+0000 7fbcbe7fc640 1 -- 172.21.15.40:0/2891067535 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fbcb8046400 con 0x7fbcc80b2690 2024-04-26T21:34:49.173 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-26T21:34:49.173 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:34:47.953711+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:34:49.174 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 msgr2=0x7fbcc80c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:49.174 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80c9e90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80097c0 tx=0x7fbcb80046c0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.174 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 shutdown_connections 2024-04-26T21:34:49.174 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc80b4450 0x7fbcc8161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.174 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc80b2690 0x7fbcc80c9e90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.175 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 --2- 172.21.15.40:0/2891067535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc806e860 0x7fbcc8154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:49.175 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 >> 172.21.15.40:0/2891067535 conn(0x7fbcc80d2dc0 msgr2=0x7fbcc806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:49.175 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 shutdown_connections 2024-04-26T21:34:49.175 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:49.174+0000 7fbccfe20640 1 -- 172.21.15.40:0/2891067535 wait complete. 2024-04-26T21:34:49.183 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T21:34:49.222 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:49.221+0000 7efe9fdbf040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:34:49.270 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:49.269+0000 7efe9fdbf040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:34:49.281 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:49.281+0000 7efe66a12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:34:49.332 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:49.333+0000 7efe66a12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:34:49.337 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:49.337+0000 7efe9fdbf040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:34:49.382 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:49.381+0000 7efe9fdbf040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:34:49.406 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:49.406+0000 7efe66a12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:34:49.454 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:49.454+0000 7efe66a12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:34:49.862 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:49.862+0000 7efe9fdbf040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:34:49.957 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:49.957+0000 7efe66a12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:34:49.979 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:49.979+0000 7efe9fdbf040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:34:50.082 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:34:50.082+0000 7efe66a12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:34:50.996 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:50.995+0000 7efe3ecd3640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T21:34:50.996 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:34:50.995+0000 7efe3ecd3640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T21:34:52.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-26T21:34:52.067+0000 7f53a7e32640 -1 osd.1 0 log_to_monitors true 2024-04-26T21:34:52.067 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-04-26T21:34:52.066+0000 7f4dc44e3640 -1 osd.4 0 log_to_monitors true 2024-04-26T21:34:52.068 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-04-26T21:34:52.067+0000 7f9d98a18640 -1 osd.7 0 log_to_monitors true 2024-04-26T21:34:52.097 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-04-26T21:34:52.097+0000 7faf34417640 -1 osd.5 0 log_to_monitors true 2024-04-26T21:34:52.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-26T21:34:52.118+0000 7f6d39901640 -1 osd.0 0 log_to_monitors true 2024-04-26T21:34:52.120 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-04-26T21:34:52.120+0000 7f7819dbe640 -1 osd.6 0 log_to_monitors true 2024-04-26T21:34:52.132 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-26T21:34:52.133+0000 7efd71321640 -1 osd.3 0 log_to_monitors true 2024-04-26T21:34:52.139 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:34:52.139+0000 7efe5ce11640 -1 osd.2 0 log_to_monitors true 2024-04-26T21:34:55.989 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:34:56.043 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-04-26T21:34:56.042+0000 7f9d94380640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:34:56.065 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-04-26T21:34:56.063+0000 7f9d8c1a0640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:34:56.092 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-04-26T21:34:56.091+0000 7f4dc0479640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:34:56.103 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:34:56.101+0000 7efe58798640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:34:56.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-26T21:34:56.105+0000 7f6d35897640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:34:56.112 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-04-26T21:34:56.112+0000 7faf2fe07640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:34:56.115 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-26T21:34:56.115+0000 7efd6d2b7640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T21:34:56.116 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-04-26T21:34:56.116+0000 7f4db8299640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:34:56.117 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:34:56.117+0000 7efe505b8640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:34:56.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-26T21:34:56.123+0000 7f6d2d6b7640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:34:56.125 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-26T21:34:56.125+0000 7f53a3dc8640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:34:56.128 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-04-26T21:34:56.128+0000 7faf27c27640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:34:56.134 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-26T21:34:56.134+0000 7efd650d7640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:34:56.135 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.136+0000 7f0ff9939640 1 Processor -- start 2024-04-26T21:34:56.136 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.136+0000 7f0ff9939640 1 -- start start 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.136+0000 7f0ff9939640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ff4145880 0x7f0ff40cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.136+0000 7f0ff9939640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff40ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.136+0000 7f0ff9939640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ff4070e50 con 0x7f0ff40cebc0 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.136+0000 7f0ff9939640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff406fdb0 con 0x7f0ff4145880 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.136+0000 7f0ff9939640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff4070090 con 0x7f0ff40d00d0 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.136+0000 7f0ff2ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ff4145880 0x7f0ff40cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff2ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ff4145880 0x7f0ff40cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55344/0 (socket says 172.21.15.40:55344) 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff2ffd640 1 -- 172.21.15.40:0/3911733380 learned_addr learned my addr 172.21.15.40:0/3911733380 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff37fe640 1 --1- 172.21.15.40:0/3911733380 >> v1:172.21.15.40:6789/0 conn(0x7f0ff40cebc0 0x7f0ff4072f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50120/0 (socket says 172.21.15.40:50120) 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff27fc640 1 --2- 172.21.15.40:0/3911733380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff40ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:56.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff2ffd640 1 -- 172.21.15.40:0/3911733380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 msgr2=0x7f0ff40ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff2ffd640 1 --2- 172.21.15.40:0/3911733380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff40ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff2ffd640 1 -- 172.21.15.40:0/3911733380 >> v1:172.21.15.40:6789/0 conn(0x7f0ff40cebc0 legacy=0x7f0ff4072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff2ffd640 1 -- 172.21.15.40:0/3911733380 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff4070370 con 0x7f0ff4145880 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff27fc640 1 --2- 172.21.15.40:0/3911733380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff40ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff1ffb640 1 -- 172.21.15.40:0/3911733380 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239714904 0 0) 0x7f0ff4070e50 con 0x7f0ff40cebc0 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.137+0000 7f0ff2ffd640 1 --2- 172.21.15.40:0/3911733380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff4145880 0x7f0ff40cfb90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009870 tx=0x7f0fe402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4d4e70bd7eeb89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff1ffb640 1 -- 172.21.15.40:0/3911733380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4037070 con 0x7f0ff4145880 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff1ffb640 1 -- 172.21.15.40:0/3911733380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe4032440 con 0x7f0ff4145880 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff1ffb640 1 -- 172.21.15.40:0/3911733380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4032810 con 0x7f0ff4145880 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff9939640 1 -- 172.21.15.40:0/3911733380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff4145880 msgr2=0x7f0ff40cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff9939640 1 --2- 172.21.15.40:0/3911733380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff4145880 0x7f0ff40cfb90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009870 tx=0x7f0fe402a270 comp rx=0 tx=0).stop 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff9939640 1 -- 172.21.15.40:0/3911733380 shutdown_connections 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff9939640 1 --2- 172.21.15.40:0/3911733380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff40ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff9939640 1 --2- 172.21.15.40:0/3911733380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff4145880 0x7f0ff40cfb90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.140 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff9939640 1 -- 172.21.15.40:0/3911733380 >> 172.21.15.40:0/3911733380 conn(0x7f0ff40b38c0 msgr2=0x7f0ff40b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:56.141 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff9939640 1 -- 172.21.15.40:0/3911733380 shutdown_connections 2024-04-26T21:34:56.141 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.138+0000 7f0ff9939640 1 -- 172.21.15.40:0/3911733380 wait complete. 2024-04-26T21:34:56.143 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.139+0000 7f0ff9939640 1 Processor -- start 2024-04-26T21:34:56.143 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff9939640 1 -- start start 2024-04-26T21:34:56.143 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff9939640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff40cebc0 0x7f0ff4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:56.143 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff9939640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff41425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:56.143 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff9939640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ff4145880 0x7f0ff4155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:56.143 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff9939640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff406f9d0 con 0x7f0ff40cebc0 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff9939640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff4070930 con 0x7f0ff4145880 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff9939640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff406fb50 con 0x7f0ff40d00d0 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff40cebc0 0x7f0ff4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff40cebc0 0x7f0ff4158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55354/0 (socket says 172.21.15.40:55354) 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff2ffd640 1 -- 172.21.15.40:0/200637817 learned_addr learned my addr 172.21.15.40:0/200637817 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.140+0000 7f0ff37fe640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ff4145880 0x7f0ff4155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff27fc640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff41425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff2ffd640 1 -- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 msgr2=0x7f0ff41425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff2ffd640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff41425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff2ffd640 1 -- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ff4145880 msgr2=0x7f0ff4155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff2ffd640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ff4145880 0x7f0ff4155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff2ffd640 1 -- 172.21.15.40:0/200637817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff41561a0 con 0x7f0ff40cebc0 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff2ffd640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff40cebc0 0x7f0ff4158940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4000c00 tx=0x7f0fe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:56.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0fdbfff640 1 -- 172.21.15.40:0/200637817 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4041020 con 0x7f0ff40cebc0 2024-04-26T21:34:56.145 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0fdbfff640 1 -- 172.21.15.40:0/200637817 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe4030780 con 0x7f0ff40cebc0 2024-04-26T21:34:56.145 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff4156480 con 0x7f0ff40cebc0 2024-04-26T21:34:56.145 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0fdbfff640 1 -- 172.21.15.40:0/200637817 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4037040 con 0x7f0ff40cebc0 2024-04-26T21:34:56.145 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.141+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff4156a40 con 0x7f0ff40cebc0 2024-04-26T21:34:56.145 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.142+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff4076610 con 0x7f0ff40cebc0 2024-04-26T21:34:56.145 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.142+0000 7f0fdbfff640 1 -- 172.21.15.40:0/200637817 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f0fe4044020 con 0x7f0ff40cebc0 2024-04-26T21:34:56.145 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-26T21:34:56.143+0000 7f539bbe8640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:34:56.146 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.143+0000 7f0fdbfff640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fe006c860 0x7f0fe006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:34:56.146 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.143+0000 7f0fdbfff640 1 -- 172.21.15.40:0/200637817 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f0fe40ab380 con 0x7f0ff40cebc0 2024-04-26T21:34:56.146 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.143+0000 7f0ff27fc640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fe006c860 0x7f0fe006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:34:56.146 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.143+0000 7f0ff27fc640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fe006c860 0x7f0fe006ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ff41442f0 tx=0x7f0fe8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:34:56.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.147+0000 7f0fdbfff640 1 -- 172.21.15.40:0/200637817 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0fe404b030 con 0x7f0ff40cebc0 2024-04-26T21:34:56.158 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-04-26T21:34:56.158+0000 7f7815d54640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:34:56.168 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-04-26T21:34:56.168+0000 7f780db74640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:34:56.338 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.338+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ff40b3d40 con 0x7f0ff40cebc0 2024-04-26T21:34:56.339 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.339+0000 7f0fdbfff640 1 -- 172.21.15.40:0/200637817 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f0fe4079fd0 con 0x7f0ff40cebc0 2024-04-26T21:34:56.339 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-26T21:34:56.339 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":12,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:34:53.995195+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:34:56.341 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fe006c860 msgr2=0x7f0fe006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:56.341 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fe006c860 0x7f0fe006ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ff41442f0 tx=0x7f0fe8008040 comp rx=0 tx=0).stop 2024-04-26T21:34:56.341 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff40cebc0 msgr2=0x7f0ff4158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:34:56.341 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff40cebc0 0x7f0ff4158940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4000c00 tx=0x7f0fe40046c0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.341 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 shutdown_connections 2024-04-26T21:34:56.341 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fe006c860 0x7f0fe006ed20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.342 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ff4145880 0x7f0ff4155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.342 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ff40d00d0 0x7f0ff41425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.342 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 --2- 172.21.15.40:0/200637817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ff40cebc0 0x7f0ff4158940 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:34:56.342 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 >> 172.21.15.40:0/200637817 conn(0x7f0ff40b38c0 msgr2=0x7f0ff406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:34:56.342 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 shutdown_connections 2024-04-26T21:34:56.342 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:34:56.341+0000 7f0ff9939640 1 -- 172.21.15.40:0/200637817 wait complete. 2024-04-26T21:34:56.350 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T21:35:03.158 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:03.261 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.260+0000 7f45917e4640 1 Processor -- start 2024-04-26T21:35:03.261 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.260+0000 7f45917e4640 1 -- start start 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f45917e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f458c15f3c0 0x7f458c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f45917e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0d3da0 0x7f458c0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f45917e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c0d0430 con 0x7f458c0cfbc0 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f45917e4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f458c0d0fb0 con 0x7f458c15f3c0 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f45917e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f458c0d0710 con 0x7f458c0d3da0 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f458c15f3c0 0x7f458c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f458c15f3c0 0x7f458c0d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60162/0 (socket says 172.21.15.40:60162) 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f458c0cfbc0 0x7f458c0d3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49798/0 (socket says 172.21.15.40:49798) 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458affd640 1 -- 172.21.15.40:0/2587098593 learned_addr learned my addr 172.21.15.40:0/2587098593 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458b7fe640 1 --2- 172.21.15.40:0/2587098593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0d3da0 0x7f458c0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458affd640 1 -- 172.21.15.40:0/2587098593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0d3da0 msgr2=0x7f458c0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458affd640 1 --2- 172.21.15.40:0/2587098593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0d3da0 0x7f458c0b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.263 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458affd640 1 -- 172.21.15.40:0/2587098593 >> v1:172.21.15.40:6789/0 conn(0x7f458c0cfbc0 legacy=0x7f458c0d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.264 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f458affd640 1 -- 172.21.15.40:0/2587098593 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c0d09f0 con 0x7f458c15f3c0 2024-04-26T21:35:03.264 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.261+0000 7f4589ffb640 1 -- 172.21.15.40:0/2587098593 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164669902 0 0) 0x7f458c0d0430 con 0x7f458c0cfbc0 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.262+0000 7f458affd640 1 --2- 172.21.15.40:0/2587098593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c0d20f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f457c009870 tx=0x7f457c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f43bc4772edc75c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.262+0000 7f4589ffb640 1 -- 172.21.15.40:0/2587098593 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c037070 con 0x7f458c15f3c0 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.262+0000 7f4589ffb640 1 -- 172.21.15.40:0/2587098593 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457c032440 con 0x7f458c15f3c0 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.262+0000 7f4589ffb640 1 -- 172.21.15.40:0/2587098593 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c032810 con 0x7f458c15f3c0 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.263+0000 7f45917e4640 1 -- 172.21.15.40:0/2587098593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 msgr2=0x7f458c0d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.263+0000 7f45917e4640 1 --2- 172.21.15.40:0/2587098593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c0d20f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f457c009870 tx=0x7f457c02a270 comp rx=0 tx=0).stop 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.263+0000 7f45917e4640 1 -- 172.21.15.40:0/2587098593 shutdown_connections 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.263+0000 7f45917e4640 1 --2- 172.21.15.40:0/2587098593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0d3da0 0x7f458c0b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.263+0000 7f45917e4640 1 --2- 172.21.15.40:0/2587098593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c0d20f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.263+0000 7f45917e4640 1 -- 172.21.15.40:0/2587098593 >> 172.21.15.40:0/2587098593 conn(0x7f458c09e360 msgr2=0x7f458c0d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:03.265 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.263+0000 7f45917e4640 1 -- 172.21.15.40:0/2587098593 shutdown_connections 2024-04-26T21:35:03.266 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.263+0000 7f45917e4640 1 -- 172.21.15.40:0/2587098593 wait complete. 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.264+0000 7f45917e4640 1 Processor -- start 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.264+0000 7f45917e4640 1 -- start start 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f45917e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0cfbc0 0x7f458c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f45917e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c0d3da0 0x7f458c0b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f45917e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c1500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f45917e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0d0030 con 0x7f458c15f3c0 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f45917e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f458c15f7a0 con 0x7f458c0d3da0 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f45917e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f458c0d01b0 con 0x7f458c0cfbc0 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c1500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c1500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60168/0 (socket says 172.21.15.40:60168) 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f458b7fe640 1 -- 172.21.15.40:0/3393029114 learned_addr learned my addr 172.21.15.40:0/3393029114 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:03.268 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f458affd640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0cfbc0 0x7f458c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.265+0000 7f458a7fc640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c0d3da0 0x7f458c0b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f458b7fe640 1 -- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0cfbc0 msgr2=0x7f458c15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f458b7fe640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0cfbc0 0x7f458c15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f458b7fe640 1 -- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c0d3da0 msgr2=0x7f458c0b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f458b7fe640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c0d3da0 0x7f458c0b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f458b7fe640 1 -- 172.21.15.40:0/3393029114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c1f5460 con 0x7f458c15f3c0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f458affd640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0cfbc0 0x7f458c15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f458a7fc640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c0d3da0 0x7f458c0b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f458b7fe640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c1500d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4584009fd0 tx=0x7f458400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f45737fe640 1 -- 172.21.15.40:0/3393029114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584014420 con 0x7f458c15f3c0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f45737fe640 1 -- 172.21.15.40:0/3393029114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4584019070 con 0x7f458c15f3c0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f45737fe640 1 -- 172.21.15.40:0/3393029114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584014800 con 0x7f458c15f3c0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c1f5740 con 0x7f458c15f3c0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.266+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1f5d00 con 0x7f458c15f3c0 2024-04-26T21:35:03.269 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.267+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c076610 con 0x7f458c15f3c0 2024-04-26T21:35:03.270 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.268+0000 7f45737fe640 1 -- 172.21.15.40:0/3393029114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45840149a0 con 0x7f458c15f3c0 2024-04-26T21:35:03.270 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.268+0000 7f45737fe640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.270 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.268+0000 7f45737fe640 1 -- 172.21.15.40:0/3393029114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f458408c060 con 0x7f458c15f3c0 2024-04-26T21:35:03.271 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.268+0000 7f458affd640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.271 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.269+0000 7f458affd640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f457c034000 tx=0x7f457c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.274 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.273+0000 7f45737fe640 1 -- 172.21.15.40:0/3393029114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f458405a7e0 con 0x7f458c15f3c0 2024-04-26T21:35:03.462 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.460+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f458c0d5fc0 con 0x7f458c15f3c0 2024-04-26T21:35:03.462 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.461+0000 7f45737fe640 1 -- 172.21.15.40:0/3393029114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7f458405a0e0 con 0x7f458c15f3c0 2024-04-26T21:35:03.463 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-26T21:35:03.463 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":16,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:00.188941+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 msgr2=0x7f457806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f457c034000 tx=0x7f457c02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 msgr2=0x7f458c1500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c1500d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4584009fd0 tx=0x7f458400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 shutdown_connections 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c15f3c0 0x7f458c1500d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c0d3da0 0x7f458c0b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 --2- 172.21.15.40:0/3393029114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c0cfbc0 0x7f458c15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.463+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 >> 172.21.15.40:0/3393029114 conn(0x7f458c09e360 msgr2=0x7f458c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.464+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 shutdown_connections 2024-04-26T21:35:03.465 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:35:03.464+0000 7f45917e4640 1 -- 172.21.15.40:0/3393029114 wait complete. 2024-04-26T21:35:03.474 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T21:35:03.475 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T21:35:03.475 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:35:03.475 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:35:03.477 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:35:03.477 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:35:03.477 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:35:03.479 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:35:03.479 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:35:03.480 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:35:03.482 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:35:03.483 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:35:03.483 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:35:03.485 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:35:03.485 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T21:35:03.485 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:03.553 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:35:03.553 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:35:03.566 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:35:03.567 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:35:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.589+0000 7f9315c1d640 1 Processor -- start 2024-04-26T21:35:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.589+0000 7f9315c1d640 1 -- start start 2024-04-26T21:35:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.590+0000 7f9315c1d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93100d1bf0 0x7f93100d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.590+0000 7f9315c1d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93100cde20 con 0x7f93101608f0 2024-04-26T21:35:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.590+0000 7f9315c1d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93100ce100 con 0x7f93100cffa0 2024-04-26T21:35:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.590+0000 7f9315c1d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f93100cea30 con 0x7f93100d1bf0 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.590+0000 7f930f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f93101608f0 0x7f93100cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49806/0 (socket says 172.21.15.40:49806) 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.590+0000 7f930f7fe640 1 -- 172.21.15.40:0/346378400 learned_addr learned my addr 172.21.15.40:0/346378400 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.590+0000 7f930ffff640 1 --2- 172.21.15.40:0/346378400 >> v2:172.21.15.184:3300/0 conn(0x7f93100d1bf0 0x7f93100d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1475308473 0 0) 0x7f93100cde20 con 0x7f93101608f0 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92f8003520 con 0x7f93101608f0 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 824691708 0 0) 0x7f92f8003520 con 0x7f93101608f0 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 >> v2:172.21.15.184:3300/0 conn(0x7f93100d1bf0 msgr2=0x7f93100d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 --2- 172.21.15.40:0/346378400 >> v2:172.21.15.184:3300/0 conn(0x7f93100d1bf0 0x7f93100d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 >> v1:172.21.15.184:6789/0 conn(0x7f93100cffa0 legacy=0x7f93100b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93100ce3e0 con 0x7f93101608f0 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9300001670 con 0x7f93101608f0 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 >> v1:172.21.15.40:6789/0 conn(0x7f93101608f0 legacy=0x7f93100cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f80039b0 0x7f92f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f8004800 0x7f92f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f80051c0 0x7f92f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93100cea30 con 0x7f92f80039b0 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92f800c060 con 0x7f92f80051c0 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92f800c260 con 0x7f92f8004800 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930f7fe640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f80039b0 0x7f92f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930ffff640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f80051c0 0x7f92f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930effd640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f8004800 0x7f92f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930f7fe640 1 -- 172.21.15.40:0/346378400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f8004800 msgr2=0x7f92f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930f7fe640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f8004800 0x7f92f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930f7fe640 1 -- 172.21.15.40:0/346378400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f80051c0 msgr2=0x7f92f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930f7fe640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f80051c0 0x7f92f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930ffff640 1 -- 172.21.15.40:0/346378400 reap_dead start 2024-04-26T21:35:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.591+0000 7f930f7fe640 1 -- 172.21.15.40:0/346378400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f8003520 con 0x7f92f80039b0 2024-04-26T21:35:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.592+0000 7f930f7fe640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f80039b0 0x7f92f8003dd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f930000c210 tx=0x7f92f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962481cb94fb0ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.592+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300007d30 con 0x7f92f80039b0 2024-04-26T21:35:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.592+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9300005ad0 con 0x7f92f80039b0 2024-04-26T21:35:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.592+0000 7f930e7fc640 1 -- 172.21.15.40:0/346378400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300007070 con 0x7f92f80039b0 2024-04-26T21:35:03.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.593+0000 7f9315c1d640 1 -- 172.21.15.40:0/346378400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f80039b0 msgr2=0x7f92f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.593+0000 7f9315c1d640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f80039b0 0x7f92f8003dd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f930000c210 tx=0x7f92f8005830 comp rx=0 tx=0).stop 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.593+0000 7f9315c1d640 1 -- 172.21.15.40:0/346378400 shutdown_connections 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.593+0000 7f9315c1d640 1 --2- 172.21.15.40:0/346378400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f80039b0 0x7f92f8003dd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.593+0000 7f9315c1d640 1 -- 172.21.15.40:0/346378400 >> 172.21.15.40:0/346378400 conn(0x7f93100d2ec0 msgr2=0x7f93100d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.593+0000 7f9315c1d640 1 -- 172.21.15.40:0/346378400 shutdown_connections 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.593+0000 7f9315c1d640 1 -- 172.21.15.40:0/346378400 wait complete. 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.594+0000 7f9315c1d640 1 Processor -- start 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.594+0000 7f9315c1d640 1 -- start start 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f9315c1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f80039b0 0x7f93100c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f9315c1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100c8e10 0x7f93101e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f9315c1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93100c9290 0x7f93101e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f9315c1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93100cda40 con 0x7f93100c8e10 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f9315c1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93100ceec0 con 0x7f93100c9290 2024-04-26T21:35:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f9315c1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f93100cdbc0 con 0x7f92f80039b0 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100c8e10 0x7f93101e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100c8e10 0x7f93101e7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60230/0 (socket says 172.21.15.40:60230) 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 -- 172.21.15.40:0/3721486516 learned_addr learned my addr 172.21.15.40:0/3721486516 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930ffff640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93100c9290 0x7f93101e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930f7fe640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f80039b0 0x7f93100c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 -- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f80039b0 msgr2=0x7f93100c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f80039b0 0x7f93100c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 -- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93100c9290 msgr2=0x7f93101e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93100c9290 0x7f93101e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 -- 172.21.15.40:0/3721486516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93100c6a70 con 0x7f93100c8e10 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930ffff640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93100c9290 0x7f93101e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f930effd640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100c8e10 0x7f93101e7ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f930400f5f0 tx=0x7f930400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.595+0000 7f92f7fff640 1 -- 172.21.15.40:0/3721486516 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040052e0 con 0x7f93100c8e10 2024-04-26T21:35:03.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.596+0000 7f92f7fff640 1 -- 172.21.15.40:0/3721486516 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9304018070 con 0x7f93100c8e10 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.596+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93100c6d50 con 0x7f93100c8e10 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.596+0000 7f92f7fff640 1 -- 172.21.15.40:0/3721486516 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304003810 con 0x7f93100c8e10 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.596+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93100c7310 con 0x7f93100c8e10 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.597+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93100d29d0 con 0x7f93100c8e10 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.597+0000 7f92f7fff640 1 -- 172.21.15.40:0/3721486516 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93040039b0 con 0x7f93100c8e10 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.597+0000 7f92f7fff640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92fc06c8f0 0x7f92fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.598+0000 7f92f7fff640 1 -- 172.21.15.40:0/3721486516 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f930408b650 con 0x7f93100c8e10 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.598+0000 7f930f7fe640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92fc06c8f0 0x7f92fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.598+0000 7f930f7fe640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92fc06c8f0 0x7f92fc06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9300032000 tx=0x7f930002d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.602+0000 7f92f7fff640 1 -- 172.21.15.40:0/3721486516 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93100d29d0 con 0x7f93100c8e10 2024-04-26T21:35:03.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.789+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93100c6600 con 0x7f93100c8e10 2024-04-26T21:35:03.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.789+0000 7f92f7fff640 1 -- 172.21.15.40:0/3721486516 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7f9304059dd0 con 0x7f93100c8e10 2024-04-26T21:35:03.790 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:03.791 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:00.188941+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92fc06c8f0 msgr2=0x7f92fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92fc06c8f0 0x7f92fc06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9300032000 tx=0x7f930002d000 comp rx=0 tx=0).stop 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100c8e10 msgr2=0x7f93101e7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100c8e10 0x7f93101e7ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f930400f5f0 tx=0x7f930400fab0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 shutdown_connections 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92fc06c8f0 0x7f92fc06edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93100c9290 0x7f93101e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100c8e10 0x7f93101e7ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 --2- 172.21.15.40:0/3721486516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f80039b0 0x7f93100c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 >> 172.21.15.40:0/3721486516 conn(0x7f93100d2ec0 msgr2=0x7f9310071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 shutdown_connections 2024-04-26T21:35:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.791+0000 7f9315c1d640 1 -- 172.21.15.40:0/3721486516 wait complete. 2024-04-26T21:35:03.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:35:03.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.906+0000 7fc44762a640 1 Processor -- start 2024-04-26T21:35:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.906+0000 7fc44762a640 1 -- start start 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc44762a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400d1a00 0x7fc4400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc44762a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc4400d2320 0x7fc4401625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc44762a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4400b40d0 0x7fc440162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc44762a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc440163070 con 0x7fc4400b40d0 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc44762a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc4400d0a10 con 0x7fc4400d2320 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc44762a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4400d0cf0 con 0x7fc4400d1a00 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc445bc7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4400b40d0 0x7fc440162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc445bc7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4400b40d0 0x7fc440162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60232/0 (socket says 172.21.15.40:60232) 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc445bc7640 1 -- 172.21.15.40:0/847806770 learned_addr learned my addr 172.21.15.40:0/847806770 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc444bc5640 1 --2- 172.21.15.40:0/847806770 >> v2:172.21.15.184:3300/0 conn(0x7fc4400d2320 0x7fc4401625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc4453c6640 1 --2- 172.21.15.40:0/847806770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400d1a00 0x7fc4400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.907+0000 7fc445bc7640 1 -- 172.21.15.40:0/847806770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400d1a00 msgr2=0x7fc4400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:03.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.908+0000 7fc445bc7640 1 --2- 172.21.15.40:0/847806770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400d1a00 0x7fc4400d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.908+0000 7fc445bc7640 1 -- 172.21.15.40:0/847806770 >> v2:172.21.15.184:3300/0 conn(0x7fc4400d2320 msgr2=0x7fc4401625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:03.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.908+0000 7fc445bc7640 1 --2- 172.21.15.40:0/847806770 >> v2:172.21.15.184:3300/0 conn(0x7fc4400d2320 0x7fc4401625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.908+0000 7fc445bc7640 1 -- 172.21.15.40:0/847806770 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4400d0fd0 con 0x7fc4400b40d0 2024-04-26T21:35:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.908+0000 7fc445bc7640 1 --2- 172.21.15.40:0/847806770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400b40d0 0x7fc440162b30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc44008d3d0 tx=0x7fc43002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c758c9710ec26aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.908+0000 7fc43ffff640 1 -- 172.21.15.40:0/847806770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430036070 con 0x7fc4400b40d0 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.908+0000 7fc43ffff640 1 -- 172.21.15.40:0/847806770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc430031600 con 0x7fc4400b40d0 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.908+0000 7fc43ffff640 1 -- 172.21.15.40:0/847806770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4300319d0 con 0x7fc4400b40d0 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 -- 172.21.15.40:0/847806770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400b40d0 msgr2=0x7fc440162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 --2- 172.21.15.40:0/847806770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400b40d0 0x7fc440162b30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc44008d3d0 tx=0x7fc43002a610 comp rx=0 tx=0).stop 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 -- 172.21.15.40:0/847806770 shutdown_connections 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 --2- 172.21.15.40:0/847806770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400b40d0 0x7fc440162b30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 --2- 172.21.15.40:0/847806770 >> v2:172.21.15.184:3300/0 conn(0x7fc4400d2320 0x7fc4401625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 --2- 172.21.15.40:0/847806770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400d1a00 0x7fc4400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 -- 172.21.15.40:0/847806770 >> 172.21.15.40:0/847806770 conn(0x7fc4400d4fd0 msgr2=0x7fc4400d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 -- 172.21.15.40:0/847806770 shutdown_connections 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.909+0000 7fc44762a640 1 -- 172.21.15.40:0/847806770 wait complete. 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.910+0000 7fc44762a640 1 Processor -- start 2024-04-26T21:35:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.910+0000 7fc44762a640 1 -- start start 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc44762a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400b40d0 0x7fc4401f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc44762a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400d1a00 0x7fc4400ba630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc44762a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4400d2320 0x7fc4401608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc44762a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4400d3050 con 0x7fc4400d1a00 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc44762a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4400d2ed0 con 0x7fc4400d2320 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc44762a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4400d31d0 con 0x7fc4400b40d0 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc444bc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400d1a00 0x7fc4400ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc444bc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400d1a00 0x7fc4400ba630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60234/0 (socket says 172.21.15.40:60234) 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc444bc5640 1 -- 172.21.15.40:0/2825536955 learned_addr learned my addr 172.21.15.40:0/2825536955 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc445bc7640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4400d2320 0x7fc4401608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc444bc5640 1 -- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400b40d0 msgr2=0x7fc4401f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc4453c6640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400b40d0 0x7fc4401f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc444bc5640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400b40d0 0x7fc4401f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc444bc5640 1 -- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4400d2320 msgr2=0x7fc4401608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc444bc5640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4400d2320 0x7fc4401608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.911+0000 7fc444bc5640 1 -- 172.21.15.40:0/2825536955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4400b7ed0 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.912+0000 7fc444bc5640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400d1a00 0x7fc4400ba630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c7f0 tx=0x7fc43400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.912+0000 7fc43dffb640 1 -- 172.21.15.40:0/2825536955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434015070 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.912+0000 7fc43dffb640 1 -- 172.21.15.40:0/2825536955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc434004220 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.912+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4401f55d0 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.912+0000 7fc43dffb640 1 -- 172.21.15.40:0/2825536955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434003610 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.912+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4401f5b90 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.913+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4400694f0 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.913+0000 7fc43dffb640 1 -- 172.21.15.40:0/2825536955 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4340039c0 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.913+0000 7fc43dffb640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc42c06c8f0 0x7fc42c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.914+0000 7fc43dffb640 1 -- 172.21.15.40:0/2825536955 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc43408ba50 con 0x7fc4400d1a00 2024-04-26T21:35:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.914+0000 7fc4453c6640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc42c06c8f0 0x7fc42c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.914+0000 7fc4453c6640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc42c06c8f0 0x7fc42c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b7e0 tx=0x7fc438008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:03.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:03.918+0000 7fc43dffb640 1 -- 172.21.15.40:0/2825536955 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc43405a260 con 0x7fc4400d1a00 2024-04-26T21:35:04.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.102+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4400cbe60 con 0x7fc4400d1a00 2024-04-26T21:35:04.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.102+0000 7fc43dffb640 1 -- 172.21.15.40:0/2825536955 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fc434059b60 con 0x7fc4400d1a00 2024-04-26T21:35:04.103 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:04.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc42c06c8f0 msgr2=0x7fc42c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:04.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc42c06c8f0 0x7fc42c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b7e0 tx=0x7fc438008040 comp rx=0 tx=0).stop 2024-04-26T21:35:04.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400d1a00 msgr2=0x7fc4400ba630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400d1a00 0x7fc4400ba630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c7f0 tx=0x7fc43400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 shutdown_connections 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc42c06c8f0 0x7fc42c06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4400d2320 0x7fc4401608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4400d1a00 0x7fc4400ba630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 --2- 172.21.15.40:0/2825536955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4400b40d0 0x7fc4401f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 >> 172.21.15.40:0/2825536955 conn(0x7fc4400d4fd0 msgr2=0x7fc4401623c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 shutdown_connections 2024-04-26T21:35:04.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.104+0000 7fc44762a640 1 -- 172.21.15.40:0/2825536955 wait complete. 2024-04-26T21:35:04.116 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:35:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.215+0000 7ffb7d2e1640 1 Processor -- start 2024-04-26T21:35:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.215+0000 7ffb7d2e1640 1 -- start start 2024-04-26T21:35:04.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb7d2e1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb7815f3a0 0x7ffb780d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:04.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb7d2e1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffb780d2300 0x7ffb780b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:04.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb7d2e1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb780d0e20 con 0x7ffb780d1770 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb7d2e1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffb780b1d70 con 0x7ffb780d2300 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb7d2e1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffb780b2050 con 0x7ffb7815f3a0 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb7815f3a0 0x7ffb780d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb777fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffb780d2300 0x7ffb780b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb76ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb7815f3a0 0x7ffb780d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:40548/0 (socket says 172.21.15.40:40548) 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb76ffd640 1 -- 172.21.15.40:0/1264449960 learned_addr learned my addr 172.21.15.40:0/1264449960 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb767fc640 1 --1- 172.21.15.40:0/1264449960 >> v1:172.21.15.184:6789/0 conn(0x7ffb780d1770 0x7ffb780d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:48624/0 (socket says 172.21.15.40:48624) 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb76ffd640 1 -- 172.21.15.40:0/1264449960 >> v2:172.21.15.184:3300/0 conn(0x7ffb780d2300 msgr2=0x7ffb780b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb76ffd640 1 --2- 172.21.15.40:0/1264449960 >> v2:172.21.15.184:3300/0 conn(0x7ffb780d2300 0x7ffb780b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb76ffd640 1 -- 172.21.15.40:0/1264449960 >> v1:172.21.15.184:6789/0 conn(0x7ffb780d1770 legacy=0x7ffb780d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb76ffd640 1 -- 172.21.15.40:0/1264449960 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb780b2330 con 0x7ffb7815f3a0 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.216+0000 7ffb777fe640 1 --2- 172.21.15.40:0/1264449960 >> v2:172.21.15.184:3300/0 conn(0x7ffb780d2300 0x7ffb780b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.217+0000 7ffb76ffd640 1 --2- 172.21.15.40:0/1264449960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb7815f3a0 0x7ffb780d06f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009870 tx=0x7ffb7002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732858146ebdb1cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.217+0000 7ffb75ffb640 1 -- 172.21.15.40:0/1264449960 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70037070 con 0x7ffb7815f3a0 2024-04-26T21:35:04.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.217+0000 7ffb75ffb640 1 -- 172.21.15.40:0/1264449960 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb70032440 con 0x7ffb7815f3a0 2024-04-26T21:35:04.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.217+0000 7ffb75ffb640 1 -- 172.21.15.40:0/1264449960 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70032810 con 0x7ffb7815f3a0 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.218+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1264449960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb7815f3a0 msgr2=0x7ffb780d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.218+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1264449960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb7815f3a0 0x7ffb780d06f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009870 tx=0x7ffb7002a270 comp rx=0 tx=0).stop 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.218+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1264449960 shutdown_connections 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.218+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1264449960 >> v2:172.21.15.184:3300/0 conn(0x7ffb780d2300 0x7ffb780b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.218+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1264449960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb7815f3a0 0x7ffb780d06f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.218+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1264449960 >> 172.21.15.40:0/1264449960 conn(0x7ffb780d4f90 msgr2=0x7ffb780d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.218+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1264449960 shutdown_connections 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.218+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1264449960 wait complete. 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.219+0000 7ffb7d2e1640 1 Processor -- start 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.219+0000 7ffb7d2e1640 1 -- start start 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb7d2e1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb780d1770 0x7ffb78159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb7d2e1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb780d2300 0x7ffb780b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb7d2e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb7815f3a0 0x7ffb780b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb7d2e1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffb780b2f10 con 0x7ffb7815f3a0 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb7d2e1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffb780b2d90 con 0x7ffb780d1770 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb7d2e1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffb780b3090 con 0x7ffb780d2300 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb7815f3a0 0x7ffb780b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb7815f3a0 0x7ffb780b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60236/0 (socket says 172.21.15.40:60236) 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb777fe640 1 -- 172.21.15.40:0/1866686539 learned_addr learned my addr 172.21.15.40:0/1866686539 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb76ffd640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb780d1770 0x7ffb78159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb767fc640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb780d2300 0x7ffb780b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb777fe640 1 -- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb780d2300 msgr2=0x7ffb780b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb777fe640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb780d2300 0x7ffb780b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb777fe640 1 -- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb780d1770 msgr2=0x7ffb78159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb777fe640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb780d1770 0x7ffb78159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.220+0000 7ffb777fe640 1 -- 172.21.15.40:0/1866686539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb780b7cb0 con 0x7ffb7815f3a0 2024-04-26T21:35:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.221+0000 7ffb777fe640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb7815f3a0 0x7ffb780b5c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009eb0 tx=0x7ffb6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.221+0000 7ffb6f7fe640 1 -- 172.21.15.40:0/1866686539 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68018020 con 0x7ffb7815f3a0 2024-04-26T21:35:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.221+0000 7ffb6f7fe640 1 -- 172.21.15.40:0/1866686539 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb68014480 con 0x7ffb7815f3a0 2024-04-26T21:35:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.221+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb780b7f50 con 0x7ffb7815f3a0 2024-04-26T21:35:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.221+0000 7ffb6f7fe640 1 -- 172.21.15.40:0/1866686539 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68014850 con 0x7ffb7815f3a0 2024-04-26T21:35:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.221+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb780b8510 con 0x7ffb7815f3a0 2024-04-26T21:35:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.222+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb780694f0 con 0x7ffb7815f3a0 2024-04-26T21:35:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.222+0000 7ffb6f7fe640 1 -- 172.21.15.40:0/1866686539 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb68005d10 con 0x7ffb7815f3a0 2024-04-26T21:35:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.223+0000 7ffb6f7fe640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb5806c8f0 0x7ffb5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.223+0000 7ffb6f7fe640 1 -- 172.21.15.40:0/1866686539 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ffb6808c3f0 con 0x7ffb7815f3a0 2024-04-26T21:35:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.223+0000 7ffb76ffd640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb5806c8f0 0x7ffb5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.223+0000 7ffb76ffd640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb5806c8f0 0x7ffb5806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffb70034000 tx=0x7ffb7002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:04.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.227+0000 7ffb6f7fe640 1 -- 172.21.15.40:0/1866686539 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb68022030 con 0x7ffb7815f3a0 2024-04-26T21:35:04.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.448+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb780b6860 con 0x7ffb7815f3a0 2024-04-26T21:35:04.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.449+0000 7ffb6f7fe640 1 -- 172.21.15.40:0/1866686539 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7ffb68010040 con 0x7ffb7815f3a0 2024-04-26T21:35:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:04.450 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:35:04.451 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2024-04-26T21:35:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb5806c8f0 msgr2=0x7ffb5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb5806c8f0 0x7ffb5806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffb70034000 tx=0x7ffb7002e000 comp rx=0 tx=0).stop 2024-04-26T21:35:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb7815f3a0 msgr2=0x7ffb780b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb7815f3a0 0x7ffb780b5c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009eb0 tx=0x7ffb6800c490 comp rx=0 tx=0).stop 2024-04-26T21:35:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 shutdown_connections 2024-04-26T21:35:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb5806c8f0 0x7ffb5806edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb7815f3a0 0x7ffb780b5c20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb780d2300 0x7ffb780b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 --2- 172.21.15.40:0/1866686539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb780d1770 0x7ffb78159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:04.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 >> 172.21.15.40:0/1866686539 conn(0x7ffb780d4f90 msgr2=0x7ffb78075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:04.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 shutdown_connections 2024-04-26T21:35:04.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:04.451+0000 7ffb7d2e1640 1 -- 172.21.15.40:0/1866686539 wait complete. 2024-04-26T21:35:06.464 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:35:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.575+0000 7f72e2d66640 1 Processor -- start 2024-04-26T21:35:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.575+0000 7f72e2d66640 1 -- start start 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.576+0000 7f72e2d66640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72dc0d3bc0 0x7f72dc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.576+0000 7f72e2d66640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72dc0b7e50 con 0x7f72dc0b5f20 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.576+0000 7f72e2d66640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72dc0d1800 con 0x7f72dc162880 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.576+0000 7f72e2d66640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f72dc0d0580 con 0x7f72dc0d3bc0 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.576+0000 7f72dbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72dc0d3bc0 0x7f72dc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.576+0000 7f72e1303640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f72dc0b5f20 0x7f72dc0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49808/0 (socket says 172.21.15.40:49808) 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.576+0000 7f72e1303640 1 -- 172.21.15.40:0/1336413091 learned_addr learned my addr 172.21.15.40:0/1336413091 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.577+0000 7f72dbfff640 1 -- 172.21.15.40:0/1336413091 >> v1:172.21.15.184:6789/0 conn(0x7f72dc162880 legacy=0x7f72dc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.577+0000 7f72dbfff640 1 -- 172.21.15.40:0/1336413091 >> v1:172.21.15.40:6789/0 conn(0x7f72dc0b5f20 legacy=0x7f72dc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.577+0000 7f72dbfff640 1 -- 172.21.15.40:0/1336413091 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc0d07d0 con 0x7f72dc0d3bc0 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.577+0000 7f72dbfff640 1 --2- 172.21.15.40:0/1336413091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0d3bc0 0x7f72dc0d2cb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009d90 tx=0x7f72cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ce9c4f25b69b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72db7fe640 1 -- 172.21.15.40:0/1336413091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc007740 con 0x7f72dc0d3bc0 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72db7fe640 1 -- 172.21.15.40:0/1336413091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc0078e0 con 0x7f72dc0d3bc0 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72db7fe640 1 -- 172.21.15.40:0/1336413091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc007cb0 con 0x7f72dc0d3bc0 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72e2d66640 1 -- 172.21.15.40:0/1336413091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0d3bc0 msgr2=0x7f72dc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72e2d66640 1 --2- 172.21.15.40:0/1336413091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0d3bc0 0x7f72dc0d2cb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009d90 tx=0x7f72cc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72e2d66640 1 -- 172.21.15.40:0/1336413091 shutdown_connections 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72e2d66640 1 --2- 172.21.15.40:0/1336413091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0d3bc0 0x7f72dc0d2cb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72e2d66640 1 -- 172.21.15.40:0/1336413091 >> 172.21.15.40:0/1336413091 conn(0x7f72dc0d4f60 msgr2=0x7f72dc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72e2d66640 1 -- 172.21.15.40:0/1336413091 shutdown_connections 2024-04-26T21:35:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.578+0000 7f72e2d66640 1 -- 172.21.15.40:0/1336413091 wait complete. 2024-04-26T21:35:06.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.579+0000 7f72e2d66640 1 Processor -- start 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.579+0000 7f72e2d66640 1 -- start start 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e2d66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0b5f20 0x7f72dc15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e2d66640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d3bc0 0x7f72dc1f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e2d66640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc162880 0x7f72dc15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e2d66640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc162c60 con 0x7f72dc0b5f20 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e2d66640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc0d0d90 con 0x7f72dc0d3bc0 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e2d66640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc162de0 con 0x7f72dc162880 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e0b02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0b5f20 0x7f72dc15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e0b02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0b5f20 0x7f72dc15ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60256/0 (socket says 172.21.15.40:60256) 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.580+0000 7f72e0b02640 1 -- 172.21.15.40:0/977341162 learned_addr learned my addr 172.21.15.40:0/977341162 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e1303640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc162880 0x7f72dc15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72dbfff640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d3bc0 0x7f72dc1f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e0b02640 1 -- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc162880 msgr2=0x7f72dc15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e0b02640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc162880 0x7f72dc15cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e0b02640 1 -- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d3bc0 msgr2=0x7f72dc1f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e0b02640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d3bc0 0x7f72dc1f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e0b02640 1 -- 172.21.15.40:0/977341162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc15d140 con 0x7f72dc0b5f20 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72dbfff640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d3bc0 0x7f72dc1f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e1303640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc162880 0x7f72dc15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e0b02640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0b5f20 0x7f72dc15ae40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72d400b3f0 tx=0x7f72d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72d97fa640 1 -- 172.21.15.40:0/977341162 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4015020 con 0x7f72dc0b5f20 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72d97fa640 1 -- 172.21.15.40:0/977341162 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d4011830 con 0x7f72dc0b5f20 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72dc15b4d0 con 0x7f72dc0b5f20 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72d97fa640 1 -- 172.21.15.40:0/977341162 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4011c20 con 0x7f72dc0b5f20 2024-04-26T21:35:06.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.581+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72dc15ba90 con 0x7f72dc0b5f20 2024-04-26T21:35:06.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.582+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72dc0694f0 con 0x7f72dc0b5f20 2024-04-26T21:35:06.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.583+0000 7f72d97fa640 1 -- 172.21.15.40:0/977341162 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72d4018020 con 0x7f72dc0b5f20 2024-04-26T21:35:06.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.583+0000 7f72d97fa640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.583+0000 7f72d97fa640 1 -- 172.21.15.40:0/977341162 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f72d408b400 con 0x7f72dc0b5f20 2024-04-26T21:35:06.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.583+0000 7f72dbfff640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.584+0000 7f72dbfff640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f72cc034000 tx=0x7f72cc02f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:06.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.588+0000 7f72d97fa640 1 -- 172.21.15.40:0/977341162 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72d401f030 con 0x7f72dc0b5f20 2024-04-26T21:35:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.816+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72dc0d2cb0 con 0x7f72dc0b5f20 2024-04-26T21:35:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.816+0000 7f72d97fa640 1 -- 172.21.15.40:0/977341162 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f72d40172c0 con 0x7f72dc0b5f20 2024-04-26T21:35:06.818 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:06.818 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:35:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 msgr2=0x7f72bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f72cc034000 tx=0x7f72cc02f000 comp rx=0 tx=0).stop 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0b5f20 msgr2=0x7f72dc15ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0b5f20 0x7f72dc15ae40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72d400b3f0 tx=0x7f72d400b8b0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 shutdown_connections 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc162880 0x7f72dc15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d3bc0 0x7f72dc1f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 --2- 172.21.15.40:0/977341162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc0b5f20 0x7f72dc15ae40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.818+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 >> 172.21.15.40:0/977341162 conn(0x7f72dc0d4f60 msgr2=0x7f72dc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.819+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 shutdown_connections 2024-04-26T21:35:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.819+0000 7f72e2d66640 1 -- 172.21.15.40:0/977341162 wait complete. 2024-04-26T21:35:06.830 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:06.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.930+0000 7f70a60ea640 1 Processor -- start 2024-04-26T21:35:06.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.930+0000 7f70a60ea640 1 -- start start 2024-04-26T21:35:06.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f70a60ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70a015f3a0 0x7f70a00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f70a60ea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a00d0e20 con 0x7f70a00d2330 2024-04-26T21:35:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f70a60ea640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a00b1d70 con 0x7f70a00d1770 2024-04-26T21:35:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f70a60ea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f70a00b2050 con 0x7f70a015f3a0 2024-04-26T21:35:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f709f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70a015f3a0 0x7f70a00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f709f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70a015f3a0 0x7f70a00d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60258/0 (socket says 172.21.15.40:60258) 2024-04-26T21:35:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f709ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f70a00d2330 0x7f70a00b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49814/0 (socket says 172.21.15.40:49814) 2024-04-26T21:35:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f709f7fe640 1 -- 172.21.15.40:0/1599443444 learned_addr learned my addr 172.21.15.40:0/1599443444 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f709f7fe640 1 -- 172.21.15.40:0/1599443444 >> v1:172.21.15.184:6789/0 conn(0x7f70a00d1770 legacy=0x7f70a00d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f709f7fe640 1 -- 172.21.15.40:0/1599443444 >> v1:172.21.15.40:6789/0 conn(0x7f70a00d2330 legacy=0x7f70a00b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f709f7fe640 1 -- 172.21.15.40:0/1599443444 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a00b2330 con 0x7f70a015f3a0 2024-04-26T21:35:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.931+0000 7f709effd640 1 -- 172.21.15.40:0/1599443444 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742065671 0 0) 0x7f70a00d0e20 con 0x7f70a00d2330 2024-04-26T21:35:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.932+0000 7f709f7fe640 1 --2- 172.21.15.40:0/1599443444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a00d06f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7090009870 tx=0x7f709002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e6bb7104a114d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.932+0000 7f709effd640 1 -- 172.21.15.40:0/1599443444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090037070 con 0x7f70a015f3a0 2024-04-26T21:35:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.932+0000 7f709effd640 1 -- 172.21.15.40:0/1599443444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7090032440 con 0x7f70a015f3a0 2024-04-26T21:35:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.932+0000 7f709effd640 1 -- 172.21.15.40:0/1599443444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090032810 con 0x7f70a015f3a0 2024-04-26T21:35:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.933+0000 7f70a60ea640 1 -- 172.21.15.40:0/1599443444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 msgr2=0x7f70a00d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.933+0000 7f70a60ea640 1 --2- 172.21.15.40:0/1599443444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a00d06f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7090009870 tx=0x7f709002a2a0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.933+0000 7f70a60ea640 1 -- 172.21.15.40:0/1599443444 shutdown_connections 2024-04-26T21:35:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.933+0000 7f70a60ea640 1 --2- 172.21.15.40:0/1599443444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a00d06f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.933+0000 7f70a60ea640 1 -- 172.21.15.40:0/1599443444 >> 172.21.15.40:0/1599443444 conn(0x7f70a00d4f90 msgr2=0x7f70a00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.933+0000 7f70a60ea640 1 -- 172.21.15.40:0/1599443444 shutdown_connections 2024-04-26T21:35:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.933+0000 7f70a60ea640 1 -- 172.21.15.40:0/1599443444 wait complete. 2024-04-26T21:35:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.934+0000 7f70a60ea640 1 Processor -- start 2024-04-26T21:35:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.934+0000 7f70a60ea640 1 -- start start 2024-04-26T21:35:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f70a60ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70a00d1770 0x7f70a00a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f70a60ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70a00d2330 0x7f70a01f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f70a60ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a01f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f70a60ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70a00b30e0 con 0x7f70a015f3a0 2024-04-26T21:35:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f70a60ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70a00b2f60 con 0x7f70a00d1770 2024-04-26T21:35:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f70a60ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70a00b3260 con 0x7f70a00d2330 2024-04-26T21:35:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a01f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a01f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60262/0 (socket says 172.21.15.40:60262) 2024-04-26T21:35:06.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 -- 172.21.15.40:0/553791108 learned_addr learned my addr 172.21.15.40:0/553791108 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:06.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 -- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70a00d2330 msgr2=0x7f70a01f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70a00d2330 0x7f70a01f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 -- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70a00d1770 msgr2=0x7f70a00a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:06.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709f7fe640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70a00d1770 0x7f70a00a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70a00d1770 0x7f70a00a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:06.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 -- 172.21.15.40:0/553791108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a00a2be0 con 0x7f70a015f3a0 2024-04-26T21:35:06.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709f7fe640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70a00d1770 0x7f70a00a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:06.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709ffff640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a01f5700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f709800c760 tx=0x7f709800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:06.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709cff9640 1 -- 172.21.15.40:0/553791108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709800a640 con 0x7f70a015f3a0 2024-04-26T21:35:06.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709cff9640 1 -- 172.21.15.40:0/553791108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f709800a800 con 0x7f70a015f3a0 2024-04-26T21:35:06.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a00b9820 con 0x7f70a015f3a0 2024-04-26T21:35:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f709cff9640 1 -- 172.21.15.40:0/553791108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709800abf0 con 0x7f70a015f3a0 2024-04-26T21:35:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.935+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a00b9de0 con 0x7f70a015f3a0 2024-04-26T21:35:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.936+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a00b3fb0 con 0x7f70a015f3a0 2024-04-26T21:35:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.937+0000 7f709cff9640 1 -- 172.21.15.40:0/553791108 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7098007070 con 0x7f70a015f3a0 2024-04-26T21:35:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.937+0000 7f709cff9640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.937+0000 7f709cff9640 1 -- 172.21.15.40:0/553791108 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f709808b780 con 0x7f70a015f3a0 2024-04-26T21:35:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.937+0000 7f709f7fe640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:06.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.938+0000 7f709f7fe640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f708006c8f0 0x7f708006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7090034000 tx=0x7f709002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:06.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:06.942+0000 7f709cff9640 1 -- 172.21.15.40:0/553791108 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70a00b3fb0 con 0x7f70a015f3a0 2024-04-26T21:35:07.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.127+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70a00d0330 con 0x7f70a015f3a0 2024-04-26T21:35:07.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.128+0000 7f709cff9640 1 -- 172.21.15.40:0/553791108 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7f70a00d0330 con 0x7f70a015f3a0 2024-04-26T21:35:07.129 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:07.129 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:00.188941+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f708006c8f0 msgr2=0x7f708006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f708006c8f0 0x7f708006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7090034000 tx=0x7f709002e000 comp rx=0 tx=0).stop 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 msgr2=0x7f70a01f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a01f5700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f709800c760 tx=0x7f709800cc20 comp rx=0 tx=0).stop 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 shutdown_connections 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a015f3a0 0x7f70a01f5700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70a00d2330 0x7f70a01f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 --2- 172.21.15.40:0/553791108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70a00d1770 0x7f70a00a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 >> 172.21.15.40:0/553791108 conn(0x7f70a00d4f90 msgr2=0x7f70a00b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 shutdown_connections 2024-04-26T21:35:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.130+0000 7f70a60ea640 1 -- 172.21.15.40:0/553791108 wait complete. 2024-04-26T21:35:07.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:35:07.143 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.245+0000 7f90812cf640 1 Processor -- start 2024-04-26T21:35:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.245+0000 7f90812cf640 1 -- start start 2024-04-26T21:35:07.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f90812cf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f907c0d1a00 0x7f907c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f90812cf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f907c0d2320 0x7f907c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f90812cf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c0d0a10 con 0x7f907c0b40d0 2024-04-26T21:35:07.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f90812cf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f907c0d0cf0 con 0x7f907c0d1a00 2024-04-26T21:35:07.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f90812cf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f907c0d0fd0 con 0x7f907c0d2320 2024-04-26T21:35:07.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f907c0d1a00 0x7f907c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f907c0b40d0 0x7f907c0d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49824/0 (socket says 172.21.15.40:49824) 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f907c0d1a00 0x7f907c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60264/0 (socket says 172.21.15.40:60264) 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907b7fe640 1 -- 172.21.15.40:0/1524085249 learned_addr learned my addr 172.21.15.40:0/1524085249 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f9073fff640 1 --2- 172.21.15.40:0/1524085249 >> v2:172.21.15.184:3300/0 conn(0x7f907c0d2320 0x7f907c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907affd640 1 -- 172.21.15.40:0/1524085249 >> v2:172.21.15.184:3300/0 conn(0x7f907c0d2320 msgr2=0x7f907c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907affd640 1 --2- 172.21.15.40:0/1524085249 >> v2:172.21.15.184:3300/0 conn(0x7f907c0d2320 0x7f907c1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907affd640 1 -- 172.21.15.40:0/1524085249 >> v1:172.21.15.40:6789/0 conn(0x7f907c0b40d0 legacy=0x7f907c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907affd640 1 -- 172.21.15.40:0/1524085249 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c0d40a0 con 0x7f907c0d1a00 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f907a7fc640 1 -- 172.21.15.40:0/1524085249 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745091020 0 0) 0x7f907c0d0a10 con 0x7f907c0b40d0 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.246+0000 7f9073fff640 1 --2- 172.21.15.40:0/1524085249 >> v2:172.21.15.184:3300/0 conn(0x7f907c0d2320 0x7f907c1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.247+0000 7f907affd640 1 --2- 172.21.15.40:0/1524085249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0d1a00 0x7f907c0d1de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f907c08d380 tx=0x7f906c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9b4d895d958b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.247+0000 7f907a7fc640 1 -- 172.21.15.40:0/1524085249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c037070 con 0x7f907c0d1a00 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.247+0000 7f907a7fc640 1 -- 172.21.15.40:0/1524085249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906c02add0 con 0x7f907c0d1a00 2024-04-26T21:35:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.247+0000 7f907a7fc640 1 -- 172.21.15.40:0/1524085249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c032660 con 0x7f907c0d1a00 2024-04-26T21:35:07.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.248+0000 7f90812cf640 1 -- 172.21.15.40:0/1524085249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0d1a00 msgr2=0x7f907c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.248+0000 7f90812cf640 1 --2- 172.21.15.40:0/1524085249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0d1a00 0x7f907c0d1de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f907c08d380 tx=0x7f906c02a080 comp rx=0 tx=0).stop 2024-04-26T21:35:07.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.248+0000 7f90812cf640 1 -- 172.21.15.40:0/1524085249 shutdown_connections 2024-04-26T21:35:07.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.248+0000 7f90812cf640 1 --2- 172.21.15.40:0/1524085249 >> v2:172.21.15.184:3300/0 conn(0x7f907c0d2320 0x7f907c1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.248+0000 7f90812cf640 1 --2- 172.21.15.40:0/1524085249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0d1a00 0x7f907c0d1de0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.248+0000 7f90812cf640 1 -- 172.21.15.40:0/1524085249 >> 172.21.15.40:0/1524085249 conn(0x7f907c0d4fd0 msgr2=0x7f907c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:07.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.248+0000 7f90812cf640 1 -- 172.21.15.40:0/1524085249 shutdown_connections 2024-04-26T21:35:07.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.248+0000 7f90812cf640 1 -- 172.21.15.40:0/1524085249 wait complete. 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.249+0000 7f90812cf640 1 Processor -- start 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.249+0000 7f90812cf640 1 -- start start 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90812cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0b40d0 0x7f907c14f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90812cf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f907c0d1a00 0x7f907c0b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90812cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f907c0d2320 0x7f907c0ba190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90812cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f907c162b30 con 0x7f907c0b40d0 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90812cf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0b5d60 con 0x7f907c0d1a00 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90812cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f907c162cb0 con 0x7f907c0d2320 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0b40d0 0x7f907c14f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0b40d0 0x7f907c14f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60276/0 (socket says 172.21.15.40:60276) 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 -- 172.21.15.40:0/3962785584 learned_addr learned my addr 172.21.15.40:0/3962785584 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f9073fff640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f907c0d1a00 0x7f907c0b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907b7fe640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f907c0d2320 0x7f907c0ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 -- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f907c0d2320 msgr2=0x7f907c0ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f907c0d2320 0x7f907c0ba190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 -- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f907c0d1a00 msgr2=0x7f907c0b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f907c0d1a00 0x7f907c0b9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 -- 172.21.15.40:0/3962785584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c0b8e90 con 0x7f907c0b40d0 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f9073fff640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f907c0d1a00 0x7f907c0b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907b7fe640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f907c0d2320 0x7f907c0ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f907affd640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0b40d0 0x7f907c14f2c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f906c035040 tx=0x7f906c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90737fe640 1 -- 172.21.15.40:0/3962785584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c037040 con 0x7f907c0b40d0 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90737fe640 1 -- 172.21.15.40:0/3962785584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906c03b430 con 0x7f907c0b40d0 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c0b9170 con 0x7f907c0b40d0 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90737fe640 1 -- 172.21.15.40:0/3962785584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c03b820 con 0x7f907c0b40d0 2024-04-26T21:35:07.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.250+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c0b9730 con 0x7f907c0b40d0 2024-04-26T21:35:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.251+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907c0694f0 con 0x7f907c0b40d0 2024-04-26T21:35:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.252+0000 7f90737fe640 1 -- 172.21.15.40:0/3962785584 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f906c044020 con 0x7f907c0b40d0 2024-04-26T21:35:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.252+0000 7f90737fe640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f905c06c8f0 0x7f905c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.252+0000 7f90737fe640 1 -- 172.21.15.40:0/3962785584 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f906c0ab920 con 0x7f907c0b40d0 2024-04-26T21:35:07.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.252+0000 7f9073fff640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f905c06c8f0 0x7f905c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.253+0000 7f9073fff640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f905c06c8f0 0x7f905c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9064006fd0 tx=0x7f9064008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.257+0000 7f90737fe640 1 -- 172.21.15.40:0/3962785584 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f906c07a0a0 con 0x7f907c0b40d0 2024-04-26T21:35:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.440+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f907c0d36e0 con 0x7f907c0b40d0 2024-04-26T21:35:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.441+0000 7f90737fe640 1 -- 172.21.15.40:0/3962785584 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f906c0799a0 con 0x7f907c0b40d0 2024-04-26T21:35:07.442 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.442+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f905c06c8f0 msgr2=0x7f905c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.442+0000 7f90812cf640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f905c06c8f0 0x7f905c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9064006fd0 tx=0x7f9064008040 comp rx=0 tx=0).stop 2024-04-26T21:35:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.442+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0b40d0 msgr2=0x7f907c14f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.442+0000 7f90812cf640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0b40d0 0x7f907c14f2c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f906c035040 tx=0x7f906c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.443+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 shutdown_connections 2024-04-26T21:35:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.443+0000 7f90812cf640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f905c06c8f0 0x7f905c06edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.443+0000 7f90812cf640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f907c0d2320 0x7f907c0ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.443+0000 7f90812cf640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f907c0d1a00 0x7f907c0b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.443+0000 7f90812cf640 1 --2- 172.21.15.40:0/3962785584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c0b40d0 0x7f907c14f2c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.443+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 >> 172.21.15.40:0/3962785584 conn(0x7f907c0d4fd0 msgr2=0x7f907c161380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:07.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.443+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 shutdown_connections 2024-04-26T21:35:07.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.443+0000 7f90812cf640 1 -- 172.21.15.40:0/3962785584 wait complete. 2024-04-26T21:35:07.454 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:35:07.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.553+0000 7f8515083640 1 Processor -- start 2024-04-26T21:35:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.553+0000 7f8515083640 1 -- start start 2024-04-26T21:35:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f8515083640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f85101608f0 0x7f85100d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f8515083640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f85100cffa0 0x7f85100b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f8515083640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85100cde20 con 0x7f85100d0a90 2024-04-26T21:35:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f8515083640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f85100cea30 con 0x7f85100cffa0 2024-04-26T21:35:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f8515083640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f85100ce100 con 0x7f85101608f0 2024-04-26T21:35:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f850f59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f85100cffa0 0x7f85100b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f850f59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f85100cffa0 0x7f85100b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60282/0 (socket says 172.21.15.40:60282) 2024-04-26T21:35:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f850e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f85100d0a90 0x7f85100cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49832/0 (socket says 172.21.15.40:49832) 2024-04-26T21:35:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.554+0000 7f850f59e640 1 -- 172.21.15.40:0/1791205767 learned_addr learned my addr 172.21.15.40:0/1791205767 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850ed9d640 1 --2- 172.21.15.40:0/1791205767 >> v2:172.21.15.184:3300/0 conn(0x7f85101608f0 0x7f85100d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850f59e640 1 -- 172.21.15.40:0/1791205767 >> v2:172.21.15.184:3300/0 conn(0x7f85101608f0 msgr2=0x7f85100d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850f59e640 1 --2- 172.21.15.40:0/1791205767 >> v2:172.21.15.184:3300/0 conn(0x7f85101608f0 0x7f85100d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850f59e640 1 -- 172.21.15.40:0/1791205767 >> v1:172.21.15.40:6789/0 conn(0x7f85100d0a90 legacy=0x7f85100cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850f59e640 1 -- 172.21.15.40:0/1791205767 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85100ce3e0 con 0x7f85100cffa0 2024-04-26T21:35:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850dd9b640 1 -- 172.21.15.40:0/1791205767 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211337940 0 0) 0x7f85100cde20 con 0x7f85100d0a90 2024-04-26T21:35:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850ed9d640 1 --2- 172.21.15.40:0/1791205767 >> v2:172.21.15.184:3300/0 conn(0x7f85101608f0 0x7f85100d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850f59e640 1 --2- 172.21.15.40:0/1791205767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85100cffa0 0x7f85100b2300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f85100731d0 tx=0x7f850002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b247d39eca104d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.555+0000 7f850dd9b640 1 -- 172.21.15.40:0/1791205767 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500036070 con 0x7f85100cffa0 2024-04-26T21:35:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f850dd9b640 1 -- 172.21.15.40:0/1791205767 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500031600 con 0x7f85100cffa0 2024-04-26T21:35:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f850dd9b640 1 -- 172.21.15.40:0/1791205767 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85000319d0 con 0x7f85100cffa0 2024-04-26T21:35:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f8515083640 1 -- 172.21.15.40:0/1791205767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85100cffa0 msgr2=0x7f85100b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f8515083640 1 --2- 172.21.15.40:0/1791205767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85100cffa0 0x7f85100b2300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f85100731d0 tx=0x7f850002a730 comp rx=0 tx=0).stop 2024-04-26T21:35:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f8515083640 1 -- 172.21.15.40:0/1791205767 shutdown_connections 2024-04-26T21:35:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f8515083640 1 --2- 172.21.15.40:0/1791205767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85100cffa0 0x7f85100b2300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f8515083640 1 --2- 172.21.15.40:0/1791205767 >> v2:172.21.15.184:3300/0 conn(0x7f85101608f0 0x7f85100d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f8515083640 1 -- 172.21.15.40:0/1791205767 >> 172.21.15.40:0/1791205767 conn(0x7f85100d2ec0 msgr2=0x7f85100d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f8515083640 1 -- 172.21.15.40:0/1791205767 shutdown_connections 2024-04-26T21:35:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.556+0000 7f8515083640 1 -- 172.21.15.40:0/1791205767 wait complete. 2024-04-26T21:35:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.558+0000 7f8515083640 1 Processor -- start 2024-04-26T21:35:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.558+0000 7f8515083640 1 -- start start 2024-04-26T21:35:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.558+0000 7f8515083640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85100cffa0 0x7f85100cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.558+0000 7f8515083640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85100d0a90 0x7f85101f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.558+0000 7f8515083640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85101608f0 0x7f85100cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.558+0000 7f8515083640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85100ceec0 con 0x7f85101608f0 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.558+0000 7f8515083640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85100d0ef0 con 0x7f85100d0a90 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.558+0000 7f8515083640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85100cda40 con 0x7f85100cffa0 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85101608f0 0x7f85100cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85101608f0 0x7f85100cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60290/0 (socket says 172.21.15.40:60290) 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 -- 172.21.15.40:0/3011629704 learned_addr learned my addr 172.21.15.40:0/3011629704 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850e59c640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85100d0a90 0x7f85101f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850ed9d640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85100cffa0 0x7f85100cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 -- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85100cffa0 msgr2=0x7f85100cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85100cffa0 0x7f85100cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 -- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85100d0a90 msgr2=0x7f85101f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85100d0a90 0x7f85101f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 -- 172.21.15.40:0/3011629704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85101f3910 con 0x7f85101608f0 2024-04-26T21:35:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f850f59e640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85101608f0 0x7f85100cc8e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8500009f40 tx=0x7f8500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f85077fe640 1 -- 172.21.15.40:0/3011629704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500041020 con 0x7f85101608f0 2024-04-26T21:35:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f85077fe640 1 -- 172.21.15.40:0/3011629704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500037040 con 0x7f85101608f0 2024-04-26T21:35:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f85077fe640 1 -- 172.21.15.40:0/3011629704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500036050 con 0x7f85101608f0 2024-04-26T21:35:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85101f3bf0 con 0x7f85101608f0 2024-04-26T21:35:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.559+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85101f44b0 con 0x7f85101608f0 2024-04-26T21:35:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.560+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85100694f0 con 0x7f85101608f0 2024-04-26T21:35:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.561+0000 7f85077fe640 1 -- 172.21.15.40:0/3011629704 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8500007bf0 con 0x7f85101608f0 2024-04-26T21:35:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.561+0000 7f85077fe640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84f006c8f0 0x7f84f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.561+0000 7f85077fe640 1 -- 172.21.15.40:0/3011629704 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f85000ac000 con 0x7f85101608f0 2024-04-26T21:35:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.561+0000 7f850ed9d640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84f006c8f0 0x7f84f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.562+0000 7f850ed9d640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84f006c8f0 0x7f84f006edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f850800a990 tx=0x7f8508008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.565+0000 7f85077fe640 1 -- 172.21.15.40:0/3011629704 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f850004b060 con 0x7f85101608f0 2024-04-26T21:35:07.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.752+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f85101f4680 con 0x7f85101608f0 2024-04-26T21:35:07.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.753+0000 7f85077fe640 1 -- 172.21.15.40:0/3011629704 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f850007a810 con 0x7f85101608f0 2024-04-26T21:35:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:07.754 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:35:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:35:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:35:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:35:07.755 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:35:07.755 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:35:07.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.754+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84f006c8f0 msgr2=0x7f84f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.754+0000 7f8515083640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84f006c8f0 0x7f84f006edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f850800a990 tx=0x7f8508008040 comp rx=0 tx=0).stop 2024-04-26T21:35:07.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.754+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85101608f0 msgr2=0x7f85100cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85101608f0 0x7f85100cc8e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8500009f40 tx=0x7f8500004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:07.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 shutdown_connections 2024-04-26T21:35:07.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84f006c8f0 0x7f84f006edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85101608f0 0x7f85100cc8e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85100d0a90 0x7f85101f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 --2- 172.21.15.40:0/3011629704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85100cffa0 0x7f85100cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 >> 172.21.15.40:0/3011629704 conn(0x7f85100d2ec0 msgr2=0x7f851015af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 shutdown_connections 2024-04-26T21:35:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.755+0000 7f8515083640 1 -- 172.21.15.40:0/3011629704 wait complete. 2024-04-26T21:35:07.766 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:35:07.766 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:35:07.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.870+0000 7f76bd3f0640 1 Processor -- start 2024-04-26T21:35:07.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.870+0000 7f76bd3f0640 1 -- start start 2024-04-26T21:35:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76bd3f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f76b815f3e0 0x7f76b80d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76bd3f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76b80d31e0 0x7f76b80d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76bd3f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b8160930 con 0x7f76b815f7c0 2024-04-26T21:35:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76bd3f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f76b815fd20 con 0x7f76b80d31e0 2024-04-26T21:35:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76bd3f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f76b8160000 con 0x7f76b815f3e0 2024-04-26T21:35:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b67fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76b80d31e0 0x7f76b80d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f76b815f7c0 0x7f76b81620a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49846/0 (socket says 172.21.15.40:49846) 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b77fe640 1 -- 172.21.15.40:0/445988959 learned_addr learned my addr 172.21.15.40:0/445988959 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b6ffd640 1 --2- 172.21.15.40:0/445988959 >> v2:172.21.15.184:3300/0 conn(0x7f76b815f3e0 0x7f76b80d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b67fc640 1 -- 172.21.15.40:0/445988959 >> v2:172.21.15.184:3300/0 conn(0x7f76b815f3e0 msgr2=0x7f76b80d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b67fc640 1 --2- 172.21.15.40:0/445988959 >> v2:172.21.15.184:3300/0 conn(0x7f76b815f3e0 0x7f76b80d2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b67fc640 1 -- 172.21.15.40:0/445988959 >> v1:172.21.15.40:6789/0 conn(0x7f76b815f7c0 legacy=0x7f76b81620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b5ffb640 1 -- 172.21.15.40:0/445988959 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403347098 0 0) 0x7f76b8160930 con 0x7f76b815f7c0 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b67fc640 1 -- 172.21.15.40:0/445988959 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b81602e0 con 0x7f76b80d31e0 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.871+0000 7f76b6ffd640 1 --2- 172.21.15.40:0/445988959 >> v2:172.21.15.184:3300/0 conn(0x7f76b815f3e0 0x7f76b80d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.872+0000 7f76b67fc640 1 --2- 172.21.15.40:0/445988959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b80d31e0 0x7f76b80d3690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76b808d480 tx=0x7f76a002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7de641c73b26b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.872+0000 7f76b5ffb640 1 -- 172.21.15.40:0/445988959 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0037070 con 0x7f76b80d31e0 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.872+0000 7f76b5ffb640 1 -- 172.21.15.40:0/445988959 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0032440 con 0x7f76b80d31e0 2024-04-26T21:35:07.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.872+0000 7f76b5ffb640 1 -- 172.21.15.40:0/445988959 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a002ae60 con 0x7f76b80d31e0 2024-04-26T21:35:07.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.873+0000 7f76bd3f0640 1 -- 172.21.15.40:0/445988959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b80d31e0 msgr2=0x7f76b80d3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.873+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/445988959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b80d31e0 0x7f76b80d3690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76b808d480 tx=0x7f76a002a110 comp rx=0 tx=0).stop 2024-04-26T21:35:07.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.873+0000 7f76bd3f0640 1 -- 172.21.15.40:0/445988959 shutdown_connections 2024-04-26T21:35:07.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.873+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/445988959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b80d31e0 0x7f76b80d3690 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.873+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/445988959 >> v2:172.21.15.184:3300/0 conn(0x7f76b815f3e0 0x7f76b80d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.873+0000 7f76bd3f0640 1 -- 172.21.15.40:0/445988959 >> 172.21.15.40:0/445988959 conn(0x7f76b809e3c0 msgr2=0x7f76b80d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:07.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.873+0000 7f76bd3f0640 1 -- 172.21.15.40:0/445988959 shutdown_connections 2024-04-26T21:35:07.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.873+0000 7f76bd3f0640 1 -- 172.21.15.40:0/445988959 wait complete. 2024-04-26T21:35:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.874+0000 7f76bd3f0640 1 Processor -- start 2024-04-26T21:35:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.874+0000 7f76bd3f0640 1 -- start start 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76bd3f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76b80d31e0 0x7f76b81569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76bd3f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b815f3e0 0x7f76b815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76bd3f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b815f7c0 0x7f76b815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76bd3f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80d3bd0 con 0x7f76b815f3e0 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76bd3f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8160d40 con 0x7f76b815f7c0 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76bd3f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76b80d3d50 con 0x7f76b80d31e0 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b815f3e0 0x7f76b815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b815f3e0 0x7f76b815b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60306/0 (socket says 172.21.15.40:60306) 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 -- 172.21.15.40:0/3804020783 learned_addr learned my addr 172.21.15.40:0/3804020783 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b77fe640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b815f7c0 0x7f76b815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b6ffd640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76b80d31e0 0x7f76b81569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 -- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76b80d31e0 msgr2=0x7f76b81569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76b80d31e0 0x7f76b81569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 -- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b815f7c0 msgr2=0x7f76b815bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b815f7c0 0x7f76b815bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 -- 172.21.15.40:0/3804020783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b80ce620 con 0x7f76b815f3e0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b77fe640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b815f7c0 0x7f76b815bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b6ffd640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76b80d31e0 0x7f76b81569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.875+0000 7f76b67fc640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b815f3e0 0x7f76b815b590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c00 tx=0x7f76a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.876+0000 7f76af7fe640 1 -- 172.21.15.40:0/3804020783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0037040 con 0x7f76b815f3e0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.876+0000 7f76af7fe640 1 -- 172.21.15.40:0/3804020783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0042070 con 0x7f76b815f3e0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.876+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b80ce900 con 0x7f76b815f3e0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.876+0000 7f76af7fe640 1 -- 172.21.15.40:0/3804020783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0030bd0 con 0x7f76b815f3e0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.876+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b80ceec0 con 0x7f76b815f3e0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.877+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b80cf090 con 0x7f76b815f3e0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.877+0000 7f76af7fe640 1 -- 172.21.15.40:0/3804020783 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76a0047460 con 0x7f76b815f3e0 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.878+0000 7f76af7fe640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f769806c920 0x7f769806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.878+0000 7f76af7fe640 1 -- 172.21.15.40:0/3804020783 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f76a00ac430 con 0x7f76b815f3e0 2024-04-26T21:35:07.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.878+0000 7f76b6ffd640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f769806c920 0x7f769806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:07.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.878+0000 7f76b6ffd640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f769806c920 0x7f769806ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006fd0 tx=0x7f76b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:07.882+0000 7f76af7fe640 1 -- 172.21.15.40:0/3804020783 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f76a0032e20 con 0x7f76b815f3e0 2024-04-26T21:35:08.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.067+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f76b80d57e0 con 0x7f76b815f3e0 2024-04-26T21:35:08.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.096+0000 7f76af7fe640 1 -- 172.21.15.40:0/3804020783 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f76a007ac40 con 0x7f76b815f3e0 2024-04-26T21:35:08.097 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f769806c920 msgr2=0x7f769806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f769806c920 0x7f769806ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006fd0 tx=0x7f76b0008040 comp rx=0 tx=0).stop 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b815f3e0 msgr2=0x7f76b815b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b815f3e0 0x7f76b815b590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c00 tx=0x7f76a00046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 shutdown_connections 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f769806c920 0x7f769806ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b815f7c0 0x7f76b815bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b815f3e0 0x7f76b815b590 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 --2- 172.21.15.40:0/3804020783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76b80d31e0 0x7f76b81569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 >> 172.21.15.40:0/3804020783 conn(0x7f76b809e3c0 msgr2=0x7f76b8162090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 shutdown_connections 2024-04-26T21:35:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.098+0000 7f76bd3f0640 1 -- 172.21.15.40:0/3804020783 wait complete. 2024-04-26T21:35:08.114 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T21:35:08.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.220+0000 7f399e87e640 1 Processor -- start 2024-04-26T21:35:08.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.220+0000 7f399e87e640 1 -- start start 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.220+0000 7f399e87e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f39980d3ee0 0x7f39980d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.220+0000 7f399e87e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39980c8bc0 con 0x7f3998162a20 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.220+0000 7f399e87e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39980d09f0 con 0x7f39980c9350 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.220+0000 7f399e87e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f39980d07a0 con 0x7f39980d3ee0 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3997fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3998162a20 0x7f39980d1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49860/0 (socket says 172.21.15.40:49860) 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3997fff640 1 -- 172.21.15.40:0/1800179045 learned_addr learned my addr 172.21.15.40:0/1800179045 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f39977fe640 1 --2- 172.21.15.40:0/1800179045 >> v2:172.21.15.184:3300/0 conn(0x7f39980d3ee0 0x7f39980d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602092664 0 0) 0x7f39980c8bc0 con 0x7f3998162a20 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3980003520 con 0x7f3998162a20 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037686002 0 0) 0x7f39980d09f0 con 0x7f39980c9350 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39980c8bc0 con 0x7f39980c9350 2024-04-26T21:35:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1038563866 0 0) 0x7f3980003520 con 0x7f3998162a20 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 >> v2:172.21.15.184:3300/0 conn(0x7f39980d3ee0 msgr2=0x7f39980d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 --2- 172.21.15.40:0/1800179045 >> v2:172.21.15.184:3300/0 conn(0x7f39980d3ee0 0x7f39980d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 >> v1:172.21.15.184:6789/0 conn(0x7f39980c9350 legacy=0x7f39980c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39980d0cd0 con 0x7f3998162a20 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f3988001670 con 0x7f3998162a20 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 >> v1:172.21.15.40:6789/0 conn(0x7f3998162a20 legacy=0x7f39980d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980003b40 0x7f3980003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3980004800 0x7f3980004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39800051c0 0x7f398000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39980d07a0 con 0x7f3980004800 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f398000c0f0 con 0x7f3980003b40 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f398000c2f0 con 0x7f39800051c0 2024-04-26T21:35:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f39977fe640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3980004800 0x7f3980004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f3997fff640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980003b40 0x7f3980003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.221+0000 7f399ce1b640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39800051c0 0x7f398000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f39977fe640 1 -- 172.21.15.40:0/1800179045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39800051c0 msgr2=0x7f398000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f39977fe640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39800051c0 0x7f398000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f39977fe640 1 -- 172.21.15.40:0/1800179045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980003b40 msgr2=0x7f3980003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f39977fe640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980003b40 0x7f3980003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f39977fe640 1 -- 172.21.15.40:0/1800179045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3980003520 con 0x7f3980004800 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f399ce1b640 1 -- 172.21.15.40:0/1800179045 reap_dead start 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f39977fe640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3980004800 0x7f3980004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f398c009ac0 tx=0x7f3980005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d96cbe5b7ee027d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c036070 con 0x7f3980004800 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398c031400 con 0x7f3980004800 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.222+0000 7f3996ffd640 1 -- 172.21.15.40:0/1800179045 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c009690 con 0x7f3980004800 2024-04-26T21:35:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.223+0000 7f399e87e640 1 -- 172.21.15.40:0/1800179045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3980004800 msgr2=0x7f3980004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.223+0000 7f399e87e640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3980004800 0x7f3980004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f398c009ac0 tx=0x7f3980005830 comp rx=0 tx=0).stop 2024-04-26T21:35:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.223+0000 7f399e87e640 1 -- 172.21.15.40:0/1800179045 shutdown_connections 2024-04-26T21:35:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.223+0000 7f399e87e640 1 --2- 172.21.15.40:0/1800179045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3980004800 0x7f3980004c80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.223+0000 7f399e87e640 1 -- 172.21.15.40:0/1800179045 >> 172.21.15.40:0/1800179045 conn(0x7f39980d4fd0 msgr2=0x7f39980d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.223+0000 7f399e87e640 1 -- 172.21.15.40:0/1800179045 shutdown_connections 2024-04-26T21:35:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.223+0000 7f399e87e640 1 -- 172.21.15.40:0/1800179045 wait complete. 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.224+0000 7f399e87e640 1 Processor -- start 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.224+0000 7f399e87e640 1 -- start start 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399e87e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980004800 0x7f39981f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399e87e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39980b8570 0x7f39980cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399e87e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39981f57e0 0x7f39981f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399e87e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39980cbd90 con 0x7f39981f57e0 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399e87e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39980d0fb0 con 0x7f3980004800 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399e87e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3998162e00 con 0x7f39980b8570 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399ce1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39981f57e0 0x7f39981f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399ce1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39981f57e0 0x7f39981f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60338/0 (socket says 172.21.15.40:60338) 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399ce1b640 1 -- 172.21.15.40:0/1706737499 learned_addr learned my addr 172.21.15.40:0/1706737499 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f39977fe640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39980b8570 0x7f39980cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f3997fff640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980004800 0x7f39981f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.225+0000 7f399ce1b640 1 -- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39980b8570 msgr2=0x7f39980cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f399ce1b640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39980b8570 0x7f39980cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f399ce1b640 1 -- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980004800 msgr2=0x7f39981f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f399ce1b640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980004800 0x7f39981f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f399ce1b640 1 -- 172.21.15.40:0/1706737499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39980b89f0 con 0x7f39981f57e0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f39977fe640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39980b8570 0x7f39980cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f3997fff640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980004800 0x7f39981f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f399ce1b640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39981f57e0 0x7f39981f5c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f399808f280 tx=0x7f399000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f3994ff9640 1 -- 172.21.15.40:0/1706737499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399000ee50 con 0x7f39981f57e0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f3994ff9640 1 -- 172.21.15.40:0/1706737499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3990018070 con 0x7f39981f57e0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39981f8d60 con 0x7f39981f57e0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f3994ff9640 1 -- 172.21.15.40:0/1706737499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990003dc0 con 0x7f39981f57e0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.226+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39981f9320 con 0x7f39981f57e0 2024-04-26T21:35:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.227+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39980694f0 con 0x7f39981f57e0 2024-04-26T21:35:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.227+0000 7f3994ff9640 1 -- 172.21.15.40:0/1706737499 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39900050d0 con 0x7f39981f57e0 2024-04-26T21:35:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.228+0000 7f3994ff9640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.228+0000 7f3994ff9640 1 -- 172.21.15.40:0/1706737499 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f399008b660 con 0x7f39981f57e0 2024-04-26T21:35:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.228+0000 7f3997fff640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.229+0000 7f3997fff640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f398406c8f0 0x7f398406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3988000fd0 tx=0x7f3988009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.233+0000 7f3994ff9640 1 -- 172.21.15.40:0/1706737499 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3990059c70 con 0x7f39981f57e0 2024-04-26T21:35:08.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:08.418+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f3998161d70 con 0x7f39981f57e0 2024-04-26T21:35:09.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.102+0000 7f3994ff9640 1 -- 172.21.15.40:0/1706737499 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f3990059570 con 0x7f39981f57e0 2024-04-26T21:35:09.104 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f398406c8f0 msgr2=0x7f398406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f398406c8f0 0x7f398406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3988000fd0 tx=0x7f3988009040 comp rx=0 tx=0).stop 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39981f57e0 msgr2=0x7f39981f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39981f57e0 0x7f39981f5c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f399808f280 tx=0x7f399000ec00 comp rx=0 tx=0).stop 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 shutdown_connections 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39981f57e0 0x7f39981f5c70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39980b8570 0x7f39980cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 --2- 172.21.15.40:0/1706737499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3980004800 0x7f39981f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 >> 172.21.15.40:0/1706737499 conn(0x7f39980d4fd0 msgr2=0x7f39980d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.106+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 shutdown_connections 2024-04-26T21:35:09.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.107+0000 7f399e87e640 1 -- 172.21.15.40:0/1706737499 wait complete. 2024-04-26T21:35:09.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:35:09.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.242+0000 7f6de2be7640 1 Processor -- start 2024-04-26T21:35:09.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.242+0000 7f6de2be7640 1 -- start start 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6de2be7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc0b40d0 0x7f6ddc1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6de2be7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc0d0a10 con 0x7f6ddc0d1a00 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6de2be7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc0d0cf0 con 0x7f6ddc162b30 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6de2be7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc0d0fd0 con 0x7f6ddc0b40d0 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6de0983640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6ddc0d1a00 0x7f6ddc0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60994/0 (socket says 172.21.15.40:60994) 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6de0983640 1 -- 172.21.15.40:0/25617399 learned_addr learned my addr 172.21.15.40:0/25617399 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddbfff640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc0b40d0 0x7f6ddc1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1227566865 0 0) 0x7f6ddc0d0a10 con 0x7f6ddc0d1a00 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dcc003520 con 0x7f6ddc0d1a00 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2227481081 0 0) 0x7f6dcc003520 con 0x7f6ddc0d1a00 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc0b40d0 msgr2=0x7f6ddc1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:09.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc0b40d0 0x7f6ddc1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 >> v1:172.21.15.184:6789/0 conn(0x7f6ddc162b30 legacy=0x7f6ddc0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc162fa0 con 0x7f6ddc0d1a00 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f6dd0001670 con 0x7f6ddc0d1a00 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 >> v1:172.21.15.40:6789/0 conn(0x7f6ddc0d1a00 legacy=0x7f6ddc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc0b40d0 0x7f6dcc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.243+0000 7f6ddb7fe640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc003f60 0x7f6dcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6ddb7fe640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dcc004d40 0x7f6dcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0d0fd0 con 0x7f6dcc004d40 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc00bb50 con 0x7f6dcc003f60 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc00bd50 con 0x7f6ddc0b40d0 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6de0983640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc0b40d0 0x7f6dcc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6de1184640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dcc004d40 0x7f6dcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6ddbfff640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc003f60 0x7f6dcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:09.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6de1184640 1 -- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc0b40d0 msgr2=0x7f6dcc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:09.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6de1184640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc0b40d0 0x7f6dcc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6de1184640 1 -- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc003f60 msgr2=0x7f6dcc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:09.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6de1184640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc003f60 0x7f6dcc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.244+0000 7f6de1184640 1 -- 172.21.15.40:0/25617399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc003520 con 0x7f6dcc004d40 2024-04-26T21:35:09.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6de1184640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dcc004d40 0x7f6dcc0051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4001e50 tx=0x7f6dcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bc88538c44ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:09.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4001a70 con 0x7f6dcc004d40 2024-04-26T21:35:09.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd4003970 con 0x7f6dcc004d40 2024-04-26T21:35:09.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6ddb7fe640 1 -- 172.21.15.40:0/25617399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4003d60 con 0x7f6dcc004d40 2024-04-26T21:35:09.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6de2be7640 1 -- 172.21.15.40:0/25617399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dcc004d40 msgr2=0x7f6dcc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:09.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6de2be7640 1 --2- 172.21.15.40:0/25617399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dcc004d40 0x7f6dcc0051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4001e50 tx=0x7f6dcc005830 comp rx=0 tx=0).stop 2024-04-26T21:35:09.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6de1184640 1 -- 172.21.15.40:0/25617399 reap_dead start 2024-04-26T21:35:09.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6de2be7640 1 -- 172.21.15.40:0/25617399 shutdown_connections 2024-04-26T21:35:09.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6de2be7640 1 -- 172.21.15.40:0/25617399 >> 172.21.15.40:0/25617399 conn(0x7f6ddc0d4fd0 msgr2=0x7f6ddc0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:09.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6de2be7640 1 -- 172.21.15.40:0/25617399 shutdown_connections 2024-04-26T21:35:09.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.245+0000 7f6de2be7640 1 -- 172.21.15.40:0/25617399 wait complete. 2024-04-26T21:35:09.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.246+0000 7f6de2be7640 1 Processor -- start 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.246+0000 7f6de2be7640 1 -- start start 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de2be7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc004d40 0x7f6ddc1f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de2be7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc155c20 0x7f6ddc155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de2be7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ddc160d10 0x7f6ddc1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de2be7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0d2670 con 0x7f6ddc160d10 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de2be7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0d24f0 con 0x7f6dcc004d40 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de2be7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc1625f0 con 0x7f6ddc155c20 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de1184640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ddc160d10 0x7f6ddc1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de1184640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ddc160d10 0x7f6ddc1611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35470/0 (socket says 172.21.15.40:35470) 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.247+0000 7f6de1184640 1 -- 172.21.15.40:0/3106805858 learned_addr learned my addr 172.21.15.40:0/3106805858 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6ddbfff640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc155c20 0x7f6ddc155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de0983640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc004d40 0x7f6ddc1f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de1184640 1 -- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc155c20 msgr2=0x7f6ddc155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de1184640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc155c20 0x7f6ddc155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de1184640 1 -- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc004d40 msgr2=0x7f6ddc1f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de1184640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc004d40 0x7f6ddc1f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de1184640 1 -- 172.21.15.40:0/3106805858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc156d10 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de1184640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ddc160d10 0x7f6ddc1611a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4008fd0 tx=0x7f6dd40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6dd97fa640 1 -- 172.21.15.40:0/3106805858 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4037040 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6dd97fa640 1 -- 172.21.15.40:0/3106805858 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd4003e50 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc156470 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6dd97fa640 1 -- 172.21.15.40:0/3106805858 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4007db0 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.248+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc156a30 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.249+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc0694f0 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.250+0000 7f6dd97fa640 1 -- 172.21.15.40:0/3106805858 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dd4042020 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.250+0000 7f6dd97fa640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6dc006c8f0 0x7f6dc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.250+0000 7f6dd97fa640 1 -- 172.21.15.40:0/3106805858 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f6dd40aaf20 con 0x7f6ddc160d10 2024-04-26T21:35:09.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.250+0000 7f6de0983640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6dc006c8f0 0x7f6dc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:09.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.251+0000 7f6de0983640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6dc006c8f0 0x7f6dc006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc15fc50 tx=0x7f6dd0000fa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:09.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.255+0000 7f6dd97fa640 1 -- 172.21.15.40:0/3106805858 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6dd4079520 con 0x7f6ddc160d10 2024-04-26T21:35:09.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:09.477+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6ddc15f9a0 con 0x7f6ddc160d10 2024-04-26T21:35:09.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:35:09.897 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:35:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.115+0000 7f6dd97fa640 1 -- 172.21.15.40:0/3106805858 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f6dd4078e20 con 0x7f6ddc160d10 2024-04-26T21:35:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6dc006c8f0 msgr2=0x7f6dc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6dc006c8f0 0x7f6dc006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc15fc50 tx=0x7f6dd0000fa0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ddc160d10 msgr2=0x7f6ddc1611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ddc160d10 0x7f6ddc1611a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4008fd0 tx=0x7f6dd40014a0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 shutdown_connections 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6dc006c8f0 0x7f6dc006edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ddc160d10 0x7f6ddc1611a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ddc155c20 0x7f6ddc155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 --2- 172.21.15.40:0/3106805858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dcc004d40 0x7f6ddc1f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.121+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 >> 172.21.15.40:0/3106805858 conn(0x7f6ddc0d4fd0 msgr2=0x7f6ddc0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:10.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.123+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 shutdown_connections 2024-04-26T21:35:10.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.125+0000 7f6de2be7640 1 -- 172.21.15.40:0/3106805858 wait complete. 2024-04-26T21:35:10.147 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T21:35:10.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T21:35:10.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.252+0000 7ff59f187640 1 Processor -- start 2024-04-26T21:35:10.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.252+0000 7ff59f187640 1 -- start start 2024-04-26T21:35:10.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59f187640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5980d3da0 0x7ff5981620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:10.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59f187640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff598160930 con 0x7ff59815f7c0 2024-04-26T21:35:10.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59f187640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff59815fd20 con 0x7ff59815f3e0 2024-04-26T21:35:10.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59f187640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff598160000 con 0x7ff5980d3da0 2024-04-26T21:35:10.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59d724640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5980d3da0 0x7ff5981620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59d724640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5980d3da0 0x7ff5981620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35476/0 (socket says 172.21.15.40:35476) 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff597fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff59815f7c0 0x7ff5980d3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32770/0 (socket says 172.21.15.40:32770) 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59d724640 1 -- 172.21.15.40:0/3569680050 learned_addr learned my addr 172.21.15.40:0/3569680050 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59d724640 1 -- 172.21.15.40:0/3569680050 >> v1:172.21.15.184:6789/0 conn(0x7ff59815f3e0 legacy=0x7ff5980d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.253+0000 7ff59d724640 1 -- 172.21.15.40:0/3569680050 >> v1:172.21.15.40:6789/0 conn(0x7ff59815f7c0 legacy=0x7ff5980d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.254+0000 7ff59d724640 1 -- 172.21.15.40:0/3569680050 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5981602e0 con 0x7ff5980d3da0 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.254+0000 7ff5977fe640 1 -- 172.21.15.40:0/3569680050 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3738726166 0 0) 0x7ff598160930 con 0x7ff59815f7c0 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.254+0000 7ff59d724640 1 --2- 172.21.15.40:0/3569680050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5980d3da0 0x7ff5981620a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff590009e50 tx=0x7ff59002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be4d63389c399d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.254+0000 7ff5977fe640 1 -- 172.21.15.40:0/3569680050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590036070 con 0x7ff5980d3da0 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.254+0000 7ff5977fe640 1 -- 172.21.15.40:0/3569680050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff590031600 con 0x7ff5980d3da0 2024-04-26T21:35:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.254+0000 7ff5977fe640 1 -- 172.21.15.40:0/3569680050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5900319d0 con 0x7ff5980d3da0 2024-04-26T21:35:10.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.255+0000 7ff59f187640 1 -- 172.21.15.40:0/3569680050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5980d3da0 msgr2=0x7ff5981620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:10.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.255+0000 7ff59f187640 1 --2- 172.21.15.40:0/3569680050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5980d3da0 0x7ff5981620a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff590009e50 tx=0x7ff59002a6d0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.255+0000 7ff59f187640 1 -- 172.21.15.40:0/3569680050 shutdown_connections 2024-04-26T21:35:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.255+0000 7ff59f187640 1 --2- 172.21.15.40:0/3569680050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5980d3da0 0x7ff5981620a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.255+0000 7ff59f187640 1 -- 172.21.15.40:0/3569680050 >> 172.21.15.40:0/3569680050 conn(0x7ff59809e3c0 msgr2=0x7ff5980d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.255+0000 7ff59f187640 1 -- 172.21.15.40:0/3569680050 shutdown_connections 2024-04-26T21:35:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.255+0000 7ff59f187640 1 -- 172.21.15.40:0/3569680050 wait complete. 2024-04-26T21:35:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.256+0000 7ff59f187640 1 Processor -- start 2024-04-26T21:35:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.256+0000 7ff59f187640 1 -- start start 2024-04-26T21:35:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59f187640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5980d3da0 0x7ff5981e8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59f187640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff59815f3e0 0x7ff5981e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59f187640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff59815f7c0 0x7ff5981e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59f187640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5980d1040 con 0x7ff59815f3e0 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59f187640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff598160d40 con 0x7ff5980d3da0 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59f187640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5980d33b0 con 0x7ff59815f7c0 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff597fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff59815f3e0 0x7ff5981e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff597fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff59815f3e0 0x7ff5981e5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35486/0 (socket says 172.21.15.40:35486) 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff597fff640 1 -- 172.21.15.40:0/2247591515 learned_addr learned my addr 172.21.15.40:0/2247591515 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59cf23640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5980d3da0 0x7ff5981e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff597fff640 1 -- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff59815f7c0 msgr2=0x7ff5981e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59d724640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff59815f7c0 0x7ff5981e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff597fff640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff59815f7c0 0x7ff5981e64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff597fff640 1 -- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5980d3da0 msgr2=0x7ff5981e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff597fff640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5980d3da0 0x7ff5981e8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff597fff640 1 -- 172.21.15.40:0/2247591515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5981e6b60 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.257+0000 7ff59cf23640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5980d3da0 0x7ff5981e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.258+0000 7ff597fff640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff59815f3e0 0x7ff5981e5f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00c440 tx=0x7ff58c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.258+0000 7ff5957fa640 1 -- 172.21.15.40:0/2247591515 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c018070 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.258+0000 7ff5957fa640 1 -- 172.21.15.40:0/2247591515 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58c0074a0 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.258+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5981e6e40 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.258+0000 7ff5957fa640 1 -- 172.21.15.40:0/2247591515 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c004a20 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.258+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5981e7400 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.259+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff598069600 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.259+0000 7ff5957fa640 1 -- 172.21.15.40:0/2247591515 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff58c01b070 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.260+0000 7ff5957fa640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff58406c8f0 0x7ff58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.260+0000 7ff5957fa640 1 -- 172.21.15.40:0/2247591515 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7ff58c08ae10 con 0x7ff59815f3e0 2024-04-26T21:35:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.260+0000 7ff59cf23640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff58406c8f0 0x7ff58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:10.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.260+0000 7ff59cf23640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff58406c8f0 0x7ff58406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff588007fd0 tx=0x7ff588009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:10.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.264+0000 7ff5957fa640 1 -- 172.21.15.40:0/2247591515 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff58c08e030 con 0x7ff59815f3e0 2024-04-26T21:35:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:10.457+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff5980d2680 con 0x7ff59815f3e0 2024-04-26T21:35:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.119+0000 7ff5957fa640 1 -- 172.21.15.40:0/2247591515 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7ff58c007650 con 0x7ff59815f3e0 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff58406c8f0 msgr2=0x7ff58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff58406c8f0 0x7ff58406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff588007fd0 tx=0x7ff588009040 comp rx=0 tx=0).stop 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff59815f3e0 msgr2=0x7ff5981e5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff59815f3e0 0x7ff5981e5f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00c440 tx=0x7ff58c00c900 comp rx=0 tx=0).stop 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 shutdown_connections 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff58406c8f0 0x7ff58406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff59815f7c0 0x7ff5981e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff59815f3e0 0x7ff5981e5f80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 --2- 172.21.15.40:0/2247591515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5980d3da0 0x7ff5981e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 >> 172.21.15.40:0/2247591515 conn(0x7ff59809e3c0 msgr2=0x7ff59806e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 shutdown_connections 2024-04-26T21:35:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.122+0000 7ff59f187640 1 -- 172.21.15.40:0/2247591515 wait complete. 2024-04-26T21:35:11.137 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:35:11.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.250+0000 7fd853151640 1 Processor -- start 2024-04-26T21:35:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.250+0000 7fd853151640 1 -- start start 2024-04-26T21:35:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd853151640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd84c0d1880 0x7fd84c0b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd853151640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd84c0d1c60 0x7fd84c0b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd853151640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd84c0d0f30 con 0x7fd84c0d5710 2024-04-26T21:35:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd853151640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd84c0cfb10 con 0x7fd84c0d1c60 2024-04-26T21:35:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd853151640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd84c0d5ef0 con 0x7fd84c0d1880 2024-04-26T21:35:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd8516ee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd84c0d1c60 0x7fd84c0b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd84bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd84c0d5710 0x7fd84c0d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32772/0 (socket says 172.21.15.40:32772) 2024-04-26T21:35:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd84bfff640 1 -- 172.21.15.40:0/1815942999 learned_addr learned my addr 172.21.15.40:0/1815942999 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.251+0000 7fd8516ee640 1 --2- 172.21.15.40:0/1815942999 >> v2:172.21.15.40:3300/0 conn(0x7fd84c0d1c60 0x7fd84c0b7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35488/0 (socket says 172.21.15.40:35488) 2024-04-26T21:35:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.252+0000 7fd8516ee640 1 -- 172.21.15.40:0/1815942999 >> v2:172.21.15.184:3300/0 conn(0x7fd84c0d1880 msgr2=0x7fd84c0b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.252+0000 7fd850eed640 1 --2- 172.21.15.40:0/1815942999 >> v2:172.21.15.184:3300/0 conn(0x7fd84c0d1880 0x7fd84c0b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.252+0000 7fd8516ee640 1 --2- 172.21.15.40:0/1815942999 >> v2:172.21.15.184:3300/0 conn(0x7fd84c0d1880 0x7fd84c0b60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.252+0000 7fd84b7fe640 1 -- 172.21.15.40:0/1815942999 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 976108681 0 0) 0x7fd84c0d0f30 con 0x7fd84c0d5710 2024-04-26T21:35:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.252+0000 7fd8516ee640 1 -- 172.21.15.40:0/1815942999 >> v1:172.21.15.40:6789/0 conn(0x7fd84c0d5710 legacy=0x7fd84c0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.252+0000 7fd8516ee640 1 -- 172.21.15.40:0/1815942999 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c0d5c10 con 0x7fd84c0d1c60 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.252+0000 7fd8516ee640 1 --2- 172.21.15.40:0/1815942999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1c60 0x7fd84c0b7e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0b7b60 tx=0x7fd84402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3112f10ae0d5111 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd84b7fe640 1 -- 172.21.15.40:0/1815942999 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844007740 con 0x7fd84c0d1c60 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd84b7fe640 1 -- 172.21.15.40:0/1815942999 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8440078e0 con 0x7fd84c0d1c60 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd84b7fe640 1 -- 172.21.15.40:0/1815942999 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844007cb0 con 0x7fd84c0d1c60 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd853151640 1 -- 172.21.15.40:0/1815942999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1c60 msgr2=0x7fd84c0b7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd853151640 1 --2- 172.21.15.40:0/1815942999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1c60 0x7fd84c0b7e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0b7b60 tx=0x7fd84402a710 comp rx=0 tx=0).stop 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd853151640 1 -- 172.21.15.40:0/1815942999 shutdown_connections 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd853151640 1 --2- 172.21.15.40:0/1815942999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1c60 0x7fd84c0b7e50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd853151640 1 --2- 172.21.15.40:0/1815942999 >> v2:172.21.15.184:3300/0 conn(0x7fd84c0d1880 0x7fd84c0b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd853151640 1 -- 172.21.15.40:0/1815942999 >> 172.21.15.40:0/1815942999 conn(0x7fd84c09e360 msgr2=0x7fd84c0d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd853151640 1 -- 172.21.15.40:0/1815942999 shutdown_connections 2024-04-26T21:35:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.253+0000 7fd853151640 1 -- 172.21.15.40:0/1815942999 wait complete. 2024-04-26T21:35:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.254+0000 7fd853151640 1 Processor -- start 2024-04-26T21:35:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.254+0000 7fd853151640 1 -- start start 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd853151640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1880 0x7fd84c15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd853151640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84c0d1c60 0x7fd84c0b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd853151640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd84c0d5710 0x7fd84c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd853151640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c0d0300 con 0x7fd84c0d1880 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd853151640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c0d0020 con 0x7fd84c0d1c60 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd853151640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd84c0b6600 con 0x7fd84c0d5710 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd850eed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1880 0x7fd84c15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd850eed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1880 0x7fd84c15e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35504/0 (socket says 172.21.15.40:35504) 2024-04-26T21:35:11.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.255+0000 7fd850eed640 1 -- 172.21.15.40:0/3858412515 learned_addr learned my addr 172.21.15.40:0/3858412515 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd850eed640 1 -- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd84c0d5710 msgr2=0x7fd84c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd850eed640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd84c0d5710 0x7fd84c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd850eed640 1 -- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84c0d1c60 msgr2=0x7fd84c0b9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd84bfff640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84c0d1c60 0x7fd84c0b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd850eed640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84c0d1c60 0x7fd84c0b9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd850eed640 1 -- 172.21.15.40:0/3858412515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c0b7cc0 con 0x7fd84c0d1880 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd850eed640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1880 0x7fd84c15e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00c7f0 tx=0x7fd83c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd8497fa640 1 -- 172.21.15.40:0/3858412515 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c015070 con 0x7fd84c0d1880 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd8497fa640 1 -- 172.21.15.40:0/3858412515 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd83c004240 con 0x7fd84c0d1880 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c0b7fa0 con 0x7fd84c0d1880 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd8497fa640 1 -- 172.21.15.40:0/3858412515 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c007620 con 0x7fd84c0d1880 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.256+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c0b8840 con 0x7fd84c0d1880 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.257+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84c0b8170 con 0x7fd84c0d1880 2024-04-26T21:35:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.257+0000 7fd8497fa640 1 -- 172.21.15.40:0/3858412515 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd83c0079d0 con 0x7fd84c0d1880 2024-04-26T21:35:11.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.258+0000 7fd8497fa640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd83806c8f0 0x7fd83806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:11.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.258+0000 7fd8497fa640 1 -- 172.21.15.40:0/3858412515 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7fd83c08bd80 con 0x7fd84c0d1880 2024-04-26T21:35:11.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.258+0000 7fd84bfff640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd83806c8f0 0x7fd83806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:11.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.258+0000 7fd84bfff640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd83806c8f0 0x7fd83806edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0b96a0 tx=0x7fd840009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.262+0000 7fd8497fa640 1 -- 172.21.15.40:0/3858412515 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd83c05a080 con 0x7fd84c0d1880 2024-04-26T21:35:11.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:11.452+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd84c069590 con 0x7fd84c0d1880 2024-04-26T21:35:13.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.130+0000 7fd8497fa640 1 -- 172.21.15.40:0/3858412515 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7fd83c059980 con 0x7fd84c0d1880 2024-04-26T21:35:13.132 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data_ec' created 2024-04-26T21:35:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd83806c8f0 msgr2=0x7fd83806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd83806c8f0 0x7fd83806edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0b96a0 tx=0x7fd840009040 comp rx=0 tx=0).stop 2024-04-26T21:35:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1880 msgr2=0x7fd84c15e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1880 0x7fd84c15e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00c7f0 tx=0x7fd83c00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 shutdown_connections 2024-04-26T21:35:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd83806c8f0 0x7fd83806edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd84c0d5710 0x7fd84c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84c0d1c60 0x7fd84c0b9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 --2- 172.21.15.40:0/3858412515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd84c0d1880 0x7fd84c15e6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 >> 172.21.15.40:0/3858412515 conn(0x7fd84c09e360 msgr2=0x7fd84c160d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 shutdown_connections 2024-04-26T21:35:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.134+0000 7fd853151640 1 -- 172.21.15.40:0/3858412515 wait complete. 2024-04-26T21:35:13.155 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T21:35:13.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.265+0000 7fed7e6d9640 1 Processor -- start 2024-04-26T21:35:13.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.265+0000 7fed7e6d9640 1 -- start start 2024-04-26T21:35:13.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed7e6d9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed780b5fd0 0x7fed780b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:13.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed7e6d9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed78162d80 con 0x7fed781629a0 2024-04-26T21:35:13.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed7e6d9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed780d39d0 con 0x7fed780d1950 2024-04-26T21:35:13.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed7e6d9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fed780d5f30 con 0x7fed780b5fd0 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed77fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fed781629a0 0x7fed780d5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32776/0 (socket says 172.21.15.40:32776) 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed77fff640 1 -- 172.21.15.40:0/2242768837 learned_addr learned my addr 172.21.15.40:0/2242768837 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed777fe640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed780b5fd0 0x7fed780b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216303061 0 0) 0x7fed78162d80 con 0x7fed781629a0 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed60003520 con 0x7fed781629a0 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 953082949 0 0) 0x7fed60003520 con 0x7fed781629a0 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed780b5fd0 msgr2=0x7fed780b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed780b5fd0 0x7fed780b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 >> v1:172.21.15.184:6789/0 conn(0x7fed780d1950 legacy=0x7fed780d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed780d3cb0 con 0x7fed781629a0 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fed68001670 con 0x7fed781629a0 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 >> v1:172.21.15.40:6789/0 conn(0x7fed781629a0 legacy=0x7fed780d5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:13.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed600039e0 0x7fed60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed780b5fd0 0x7fed60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed60004d40 0x7fed600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed780d5f30 con 0x7fed600039e0 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed6000b9c0 con 0x7fed60004d40 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fed6000bbc0 con 0x7fed780b5fd0 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.266+0000 7fed77fff640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed600039e0 0x7fed60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.267+0000 7fed777fe640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed780b5fd0 0x7fed60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.267+0000 7fed7cc76640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed60004d40 0x7fed600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.267+0000 7fed77fff640 1 -- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed780b5fd0 msgr2=0x7fed60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.267+0000 7fed77fff640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed780b5fd0 0x7fed60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.267+0000 7fed77fff640 1 -- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed60004d40 msgr2=0x7fed600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.267+0000 7fed77fff640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed60004d40 0x7fed600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.267+0000 7fed77fff640 1 -- 172.21.15.40:0/2242768837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed60003520 con 0x7fed600039e0 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.267+0000 7fed77fff640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed600039e0 0x7fed60003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c290 tx=0x7fed60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c553a2dc5687027d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68007d50 con 0x7fed600039e0 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed68005af0 con 0x7fed600039e0 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed76ffd640 1 -- 172.21.15.40:0/2242768837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68007070 con 0x7fed600039e0 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed7e6d9640 1 -- 172.21.15.40:0/2242768837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed600039e0 msgr2=0x7fed60003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed7e6d9640 1 --2- 172.21.15.40:0/2242768837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed600039e0 0x7fed60003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c290 tx=0x7fed60005830 comp rx=0 tx=0).stop 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed7cc76640 1 -- 172.21.15.40:0/2242768837 reap_dead start 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed7e6d9640 1 -- 172.21.15.40:0/2242768837 shutdown_connections 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed7e6d9640 1 -- 172.21.15.40:0/2242768837 >> 172.21.15.40:0/2242768837 conn(0x7fed7809e3e0 msgr2=0x7fed780d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed7e6d9640 1 -- 172.21.15.40:0/2242768837 shutdown_connections 2024-04-26T21:35:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.268+0000 7fed7e6d9640 1 -- 172.21.15.40:0/2242768837 wait complete. 2024-04-26T21:35:13.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.269+0000 7fed7e6d9640 1 Processor -- start 2024-04-26T21:35:13.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7e6d9640 1 -- start start 2024-04-26T21:35:13.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7e6d9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed600039e0 0x7fed7815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7e6d9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed780b7c80 0x7fed780ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7e6d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed780cef30 0x7fed7814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7e6d9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed780d3f90 con 0x7fed780cef30 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7e6d9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed780d36a0 con 0x7fed780b7c80 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7e6d9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fed780b4890 con 0x7fed600039e0 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7cc76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed780cef30 0x7fed7814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7cc76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed780cef30 0x7fed7814c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35526/0 (socket says 172.21.15.40:35526) 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.270+0000 7fed7cc76640 1 -- 172.21.15.40:0/384730960 learned_addr learned my addr 172.21.15.40:0/384730960 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed777fe640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed780b7c80 0x7fed780ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed77fff640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed600039e0 0x7fed7815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed7cc76640 1 -- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed600039e0 msgr2=0x7fed7815d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed7cc76640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed600039e0 0x7fed7815d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed7cc76640 1 -- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed780b7c80 msgr2=0x7fed780ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:13.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed7cc76640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed780b7c80 0x7fed780ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed7cc76640 1 -- 172.21.15.40:0/384730960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed780b8120 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed7cc76640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed780cef30 0x7fed7814c360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed780d49a0 tx=0x7fed7000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed74ff9640 1 -- 172.21.15.40:0/384730960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70019070 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed74ff9640 1 -- 172.21.15.40:0/384730960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed70006d60 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed74ff9640 1 -- 172.21.15.40:0/384730960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70003bb0 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7815c8d0 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.271+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7815b6c0 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.272+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed78069600 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.273+0000 7fed74ff9640 1 -- 172.21.15.40:0/384730960 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed7001c070 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.273+0000 7fed74ff9640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed6406c8f0 0x7fed6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.273+0000 7fed74ff9640 1 -- 172.21.15.40:0/384730960 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fed7008b1d0 con 0x7fed780cef30 2024-04-26T21:35:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.274+0000 7fed77fff640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed6406c8f0 0x7fed6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:13.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.274+0000 7fed77fff640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed6406c8f0 0x7fed6406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed68032000 tx=0x7fed6802d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:13.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.278+0000 7fed74ff9640 1 -- 172.21.15.40:0/384730960 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed7815c8d0 con 0x7fed780cef30 2024-04-26T21:35:13.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:13.462+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fed780d26c0 con 0x7fed780cef30 2024-04-26T21:35:14.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.136+0000 7fed74ff9640 1 -- 172.21.15.40:0/384730960 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fed700592e0 con 0x7fed780cef30 2024-04-26T21:35:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T21:35:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.141+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed6406c8f0 msgr2=0x7fed6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.141+0000 7fed7e6d9640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed6406c8f0 0x7fed6406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed68032000 tx=0x7fed6802d000 comp rx=0 tx=0).stop 2024-04-26T21:35:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.141+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed780cef30 msgr2=0x7fed7814c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.141+0000 7fed7e6d9640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed780cef30 0x7fed7814c360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed780d49a0 tx=0x7fed7000ff70 comp rx=0 tx=0).stop 2024-04-26T21:35:14.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.142+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 shutdown_connections 2024-04-26T21:35:14.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.142+0000 7fed7e6d9640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed6406c8f0 0x7fed6406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:14.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.142+0000 7fed7e6d9640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed780cef30 0x7fed7814c360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:14.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.142+0000 7fed7e6d9640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed780b7c80 0x7fed780ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:14.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.142+0000 7fed7e6d9640 1 --2- 172.21.15.40:0/384730960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed600039e0 0x7fed7815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:14.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.142+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 >> 172.21.15.40:0/384730960 conn(0x7fed7809e3e0 msgr2=0x7fed7806d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:14.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.142+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 shutdown_connections 2024-04-26T21:35:14.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.145+0000 7fed7e6d9640 1 -- 172.21.15.40:0/384730960 wait complete. 2024-04-26T21:35:14.174 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T21:35:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.283+0000 7f3f0e683640 1 Processor -- start 2024-04-26T21:35:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.284+0000 7f3f0e683640 1 -- start start 2024-04-26T21:35:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.284+0000 7f3f0e683640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f0806c240 0x7f3f0806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.284+0000 7f3f0e683640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f0806cb60 con 0x7f3f080ae930 2024-04-26T21:35:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.284+0000 7f3f0e683640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f080c7230 con 0x7f3f080b09a0 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.284+0000 7f3f0e683640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f080c6fe0 con 0x7f3f0806c240 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f0cc20640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3f080ae930 0x7f3f080aedc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32788/0 (socket says 172.21.15.40:32788) 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f0cc20640 1 -- 172.21.15.40:0/1305407919 learned_addr learned my addr 172.21.15.40:0/1305407919 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f07fff640 1 --2- 172.21.15.40:0/1305407919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f0806c240 0x7f3f0806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f06ffd640 1 -- 172.21.15.40:0/1305407919 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1012160219 0 0) 0x7f3f0806cb60 con 0x7f3f080ae930 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f06ffd640 1 -- 172.21.15.40:0/1305407919 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ef4003520 con 0x7f3f080ae930 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f06ffd640 1 -- 172.21.15.40:0/1305407919 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200145292 0 0) 0x7f3f080c7230 con 0x7f3f080b09a0 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f06ffd640 1 -- 172.21.15.40:0/1305407919 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f0806cb60 con 0x7f3f080b09a0 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f07fff640 1 -- 172.21.15.40:0/1305407919 >> v1:172.21.15.184:6789/0 conn(0x7f3f080b09a0 legacy=0x7f3f080b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f07fff640 1 -- 172.21.15.40:0/1305407919 >> v1:172.21.15.40:6789/0 conn(0x7f3f080ae930 legacy=0x7f3f080aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.285+0000 7f3f07fff640 1 -- 172.21.15.40:0/1305407919 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f080c7510 con 0x7f3f0806c240 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.286+0000 7f3f07fff640 1 --2- 172.21.15.40:0/1305407919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f0806c240 0x7f3f0806c620 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00097d0 tx=0x7f3ef002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89b0e113507d01c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.286+0000 7f3f06ffd640 1 -- 172.21.15.40:0/1305407919 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0037070 con 0x7f3f0806c240 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.286+0000 7f3f06ffd640 1 -- 172.21.15.40:0/1305407919 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0032440 con 0x7f3f0806c240 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.286+0000 7f3f06ffd640 1 -- 172.21.15.40:0/1305407919 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00327f0 con 0x7f3f0806c240 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.287+0000 7f3f0e683640 1 -- 172.21.15.40:0/1305407919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f0806c240 msgr2=0x7f3f0806c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.287+0000 7f3f0e683640 1 --2- 172.21.15.40:0/1305407919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f0806c240 0x7f3f0806c620 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00097d0 tx=0x7f3ef002a210 comp rx=0 tx=0).stop 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.287+0000 7f3f0e683640 1 -- 172.21.15.40:0/1305407919 shutdown_connections 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.287+0000 7f3f0e683640 1 --2- 172.21.15.40:0/1305407919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f0806c240 0x7f3f0806c620 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.287+0000 7f3f0e683640 1 -- 172.21.15.40:0/1305407919 >> 172.21.15.40:0/1305407919 conn(0x7f3f080d1c90 msgr2=0x7f3f080d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.287+0000 7f3f0e683640 1 -- 172.21.15.40:0/1305407919 shutdown_connections 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.287+0000 7f3f0e683640 1 -- 172.21.15.40:0/1305407919 wait complete. 2024-04-26T21:35:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.288+0000 7f3f0e683640 1 Processor -- start 2024-04-26T21:35:14.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.288+0000 7f3f0e683640 1 -- start start 2024-04-26T21:35:14.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0e683640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f0806c240 0x7f3f08162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:14.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0e683640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f080ae930 0x7f3f081529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:14.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0e683640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f080b09a0 0x7f3f080b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:14.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0e683640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f080c7db0 con 0x7f3f080b09a0 2024-04-26T21:35:14.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0e683640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f080c7ad0 con 0x7f3f0806c240 2024-04-26T21:35:14.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0e683640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0815f440 con 0x7f3f080ae930 2024-04-26T21:35:14.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0cc20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f080b09a0 0x7f3f080b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0cc20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f080b09a0 0x7f3f080b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35540/0 (socket says 172.21.15.40:35540) 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0cc20640 1 -- 172.21.15.40:0/3813124974 learned_addr learned my addr 172.21.15.40:0/3813124974 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f077fe640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f080ae930 0x7f3f081529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f07fff640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f0806c240 0x7f3f08162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0cc20640 1 -- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f080ae930 msgr2=0x7f3f081529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0cc20640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f080ae930 0x7f3f081529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0cc20640 1 -- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f0806c240 msgr2=0x7f3f08162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0cc20640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f0806c240 0x7f3f08162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.289+0000 7f3f0cc20640 1 -- 172.21.15.40:0/3813124974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f081610b0 con 0x7f3f080b09a0 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.290+0000 7f3f0cc20640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f080b09a0 0x7f3f080b51e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000cba0 tx=0x7f3f0000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.290+0000 7f3f04ff9640 1 -- 172.21.15.40:0/3813124974 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00017070 con 0x7f3f080b09a0 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.290+0000 7f3f04ff9640 1 -- 172.21.15.40:0/3813124974 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f0000ace0 con 0x7f3f080b09a0 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.290+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f08161390 con 0x7f3f080b09a0 2024-04-26T21:35:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.290+0000 7f3f04ff9640 1 -- 172.21.15.40:0/3813124974 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00012620 con 0x7f3f080b09a0 2024-04-26T21:35:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.290+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f08161950 con 0x7f3f080b09a0 2024-04-26T21:35:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.291+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f080d2b30 con 0x7f3f080b09a0 2024-04-26T21:35:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.291+0000 7f3f04ff9640 1 -- 172.21.15.40:0/3813124974 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f0001a070 con 0x7f3f080b09a0 2024-04-26T21:35:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.291+0000 7f3f04ff9640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee806c8f0 0x7f3ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.292+0000 7f3f04ff9640 1 -- 172.21.15.40:0/3813124974 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f3f0008bcf0 con 0x7f3f080b09a0 2024-04-26T21:35:14.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.292+0000 7f3f07fff640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee806c8f0 0x7f3ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:14.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.292+0000 7f3f07fff640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee806c8f0 0x7f3ee806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0034000 tx=0x7f3ef002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.296+0000 7f3f04ff9640 1 -- 172.21.15.40:0/3813124974 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f080d2b30 con 0x7f3f080b09a0 2024-04-26T21:35:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:14.525+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f3f080af930 con 0x7f3f080b09a0 2024-04-26T21:35:14.565 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:35:14.564+0000 7f66caa24640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:35:15.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.138+0000 7f3f04ff9640 1 -- 172.21.15.40:0/3813124974 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7f3f00090070 con 0x7f3f080b09a0 2024-04-26T21:35:15.139 INFO:teuthology.orchestra.run.smithi040.stderr:added data pool 4 to fsmap 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee806c8f0 msgr2=0x7f3ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee806c8f0 0x7f3ee806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0034000 tx=0x7f3ef002e000 comp rx=0 tx=0).stop 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f080b09a0 msgr2=0x7f3f080b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f080b09a0 0x7f3f080b51e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000cba0 tx=0x7f3f0000a430 comp rx=0 tx=0).stop 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 shutdown_connections 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee806c8f0 0x7f3ee806edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f080b09a0 0x7f3f080b51e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f080ae930 0x7f3f081529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 --2- 172.21.15.40:0/3813124974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f0806c240 0x7f3f08162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 >> 172.21.15.40:0/3813124974 conn(0x7f3f080d1c90 msgr2=0x7f3f080cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 shutdown_connections 2024-04-26T21:35:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.141+0000 7f3f0e683640 1 -- 172.21.15.40:0/3813124974 wait complete. 2024-04-26T21:35:15.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:35:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.261+0000 7f4eb489b640 1 Processor -- start 2024-04-26T21:35:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.261+0000 7f4eb489b640 1 -- start start 2024-04-26T21:35:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eb489b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4eb00b5f20 0x7f4eb00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eb489b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00b7e50 con 0x7f4eb00d3bc0 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eb489b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00d1800 con 0x7f4eb0162880 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eb489b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb00d0580 con 0x7f4eb00b5f20 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4ea7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4eb00d3bc0 0x7f4eb00b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32790/0 (socket says 172.21.15.40:32790) 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4ea7fff640 1 -- 172.21.15.40:0/242007320 learned_addr learned my addr 172.21.15.40:0/242007320 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eaed9d640 1 --2- 172.21.15.40:0/242007320 >> v2:172.21.15.184:3300/0 conn(0x7f4eb00b5f20 0x7f4eb00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775885603 0 0) 0x7f4eb00b7e50 con 0x7f4eb00d3bc0 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e9c003520 con 0x7f4eb00d3bc0 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2576364347 0 0) 0x7f4e9c003520 con 0x7f4eb00d3bc0 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 >> v2:172.21.15.184:3300/0 conn(0x7f4eb00b5f20 msgr2=0x7f4eb00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eadd9b640 1 --2- 172.21.15.40:0/242007320 >> v2:172.21.15.184:3300/0 conn(0x7f4eb00b5f20 0x7f4eb00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 >> v1:172.21.15.184:6789/0 conn(0x7f4eb0162880 legacy=0x7f4eb00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb00d07d0 con 0x7f4eb00d3bc0 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.262+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f4e98001670 con 0x7f4eb00d3bc0 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 >> v1:172.21.15.40:6789/0 conn(0x7f4eb00d3bc0 legacy=0x7f4eb00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eadd9b640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c0039b0 0x7f4e9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eadd9b640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e9c0043b0 0x7f4e9c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eadd9b640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c005080 0x7f4e9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb00d0580 con 0x7f4e9c0039b0 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c00bfe0 con 0x7f4e9c0043b0 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9c00c1e0 con 0x7f4e9c005080 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eae59c640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c0039b0 0x7f4e9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4ea7fff640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e9c0043b0 0x7f4e9c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eaed9d640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c005080 0x7f4e9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eae59c640 1 -- 172.21.15.40:0/242007320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c005080 msgr2=0x7f4e9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eae59c640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c005080 0x7f4e9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eae59c640 1 -- 172.21.15.40:0/242007320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e9c0043b0 msgr2=0x7f4e9c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eae59c640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e9c0043b0 0x7f4e9c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eaed9d640 1 -- 172.21.15.40:0/242007320 reap_dead start 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.263+0000 7f4eae59c640 1 -- 172.21.15.40:0/242007320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c003520 con 0x7f4e9c0039b0 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eae59c640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c0039b0 0x7f4e9c003d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0001a30 tx=0x7f4e9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a387990fab3452 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0037070 con 0x7f4e9c0039b0 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea002adf0 con 0x7f4e9c0039b0 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eadd9b640 1 -- 172.21.15.40:0/242007320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0001600 con 0x7f4e9c0039b0 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eb489b640 1 -- 172.21.15.40:0/242007320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c0039b0 msgr2=0x7f4e9c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eb489b640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c0039b0 0x7f4e9c003d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0001a30 tx=0x7f4e9c005830 comp rx=0 tx=0).stop 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eb489b640 1 -- 172.21.15.40:0/242007320 shutdown_connections 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eb489b640 1 --2- 172.21.15.40:0/242007320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c0039b0 0x7f4e9c003d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eb489b640 1 -- 172.21.15.40:0/242007320 >> 172.21.15.40:0/242007320 conn(0x7f4eb00d4f60 msgr2=0x7f4eb00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eb489b640 1 -- 172.21.15.40:0/242007320 shutdown_connections 2024-04-26T21:35:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.264+0000 7f4eb489b640 1 -- 172.21.15.40:0/242007320 wait complete. 2024-04-26T21:35:15.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.266+0000 7f4eb489b640 1 Processor -- start 2024-04-26T21:35:15.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.266+0000 7f4eb489b640 1 -- start start 2024-04-26T21:35:15.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.266+0000 7f4eb489b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c0039b0 0x7f4eb015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eb489b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb015c6b0 0x7f4eb00b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eb489b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb00b3100 0x7f4eb00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eb489b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0162c60 con 0x7f4eb00b3100 2024-04-26T21:35:15.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eb489b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb00d0d90 con 0x7f4eb015c6b0 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eb489b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb0162de0 con 0x7f4e9c0039b0 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb00b3100 0x7f4eb00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb00b3100 0x7f4eb00b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35568/0 (socket says 172.21.15.40:35568) 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 -- 172.21.15.40:0/3153632525 learned_addr learned my addr 172.21.15.40:0/3153632525 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4ea7fff640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb015c6b0 0x7f4eb00b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eae59c640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c0039b0 0x7f4eb015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 -- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c0039b0 msgr2=0x7f4eb015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c0039b0 0x7f4eb015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 -- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb015c6b0 msgr2=0x7f4eb00b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb015c6b0 0x7f4eb00b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 -- 172.21.15.40:0/3153632525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb00cf1c0 con 0x7f4eb00b3100 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eaed9d640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb00b3100 0x7f4eb00b1b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00bafd0 tx=0x7f4ea800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3153632525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800fc30 con 0x7f4eb00b3100 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3153632525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea801d070 con 0x7f4eb00b3100 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb00ce050 con 0x7f4eb00b3100 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3153632525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8015430 con 0x7f4eb00b3100 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.267+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb00cbfa0 con 0x7f4eb00b3100 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.268+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb00694f0 con 0x7f4eb00b3100 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.269+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3153632525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ea80155d0 con 0x7f4eb00b3100 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.269+0000 7f4ea6ffd640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e9006c920 0x7f4e9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.270+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3153632525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4ea808c570 con 0x7f4eb00b3100 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.270+0000 7f4eae59c640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e9006c920 0x7f4e9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.270+0000 7f4eae59c640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e9006c920 0x7f4e9006ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0034000 tx=0x7f4ea0009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.274+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3153632525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ea8010040 con 0x7f4eb00b3100 2024-04-26T21:35:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.496+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eb015c440 con 0x7f4eb00b3100 2024-04-26T21:35:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.497+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3153632525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5180 (secure 0 0 0) 0x7f4ea805a660 con 0x7f4eb00b3100 2024-04-26T21:35:15.498 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:15.499 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:14.527772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:35:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e9006c920 msgr2=0x7f4e9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e9006c920 0x7f4e9006ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0034000 tx=0x7f4ea0009030 comp rx=0 tx=0).stop 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb00b3100 msgr2=0x7f4eb00b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb00b3100 0x7f4eb00b1b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00bafd0 tx=0x7f4ea800f1b0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 shutdown_connections 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e9006c920 0x7f4e9006ede0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb00b3100 0x7f4eb00b1b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb015c6b0 0x7f4eb00b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 --2- 172.21.15.40:0/3153632525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e9c0039b0 0x7f4eb015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 >> 172.21.15.40:0/3153632525 conn(0x7f4eb00d4f60 msgr2=0x7f4eb006d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 shutdown_connections 2024-04-26T21:35:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.499+0000 7f4eb489b640 1 -- 172.21.15.40:0/3153632525 wait complete. 2024-04-26T21:35:15.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.610+0000 7f221d7af640 1 Processor -- start 2024-04-26T21:35:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.610+0000 7f221d7af640 1 -- start start 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f221d7af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f22180d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f221d7af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22180b5f20 0x7f22180d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f221d7af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22180b7e50 con 0x7f22180d39f0 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f221d7af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f22180d0580 con 0x7f22180b5f20 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f221d7af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22180d1800 con 0x7f2218162880 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f220e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22180b5f20 0x7f22180d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f220e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22180b5f20 0x7f22180d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35570/0 (socket says 172.21.15.40:35570) 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f220e7fc640 1 -- 172.21.15.40:0/1108400951 learned_addr learned my addr 172.21.15.40:0/1108400951 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f2216ffd640 1 --2- 172.21.15.40:0/1108400951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f22180d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f220e7fc640 1 -- 172.21.15.40:0/1108400951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 msgr2=0x7f22180d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f220e7fc640 1 --2- 172.21.15.40:0/1108400951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f22180d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f220e7fc640 1 -- 172.21.15.40:0/1108400951 >> v1:172.21.15.184:6789/0 conn(0x7f22180d39f0 legacy=0x7f22180b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.611+0000 7f220e7fc640 1 -- 172.21.15.40:0/1108400951 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22180d07d0 con 0x7f22180b5f20 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.612+0000 7f220e7fc640 1 --2- 172.21.15.40:0/1108400951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180b5f20 0x7f22180d34b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2200009870 tx=0x7f220002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be401125c74e3753 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.612+0000 7f22167fc640 1 -- 172.21.15.40:0/1108400951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200037070 con 0x7f22180b5f20 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.612+0000 7f22167fc640 1 -- 172.21.15.40:0/1108400951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2200032440 con 0x7f22180b5f20 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.612+0000 7f22167fc640 1 -- 172.21.15.40:0/1108400951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200032810 con 0x7f22180b5f20 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.612+0000 7f221d7af640 1 -- 172.21.15.40:0/1108400951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180b5f20 msgr2=0x7f22180d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.612+0000 7f221d7af640 1 --2- 172.21.15.40:0/1108400951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180b5f20 0x7f22180d34b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2200009870 tx=0x7f220002a270 comp rx=0 tx=0).stop 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.613+0000 7f221d7af640 1 -- 172.21.15.40:0/1108400951 shutdown_connections 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.613+0000 7f221d7af640 1 --2- 172.21.15.40:0/1108400951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180b5f20 0x7f22180d34b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.613+0000 7f221d7af640 1 --2- 172.21.15.40:0/1108400951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f22180d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.613+0000 7f221d7af640 1 -- 172.21.15.40:0/1108400951 >> 172.21.15.40:0/1108400951 conn(0x7f22180d4f60 msgr2=0x7f22180d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:15.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.613+0000 7f221d7af640 1 -- 172.21.15.40:0/1108400951 shutdown_connections 2024-04-26T21:35:15.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.613+0000 7f221d7af640 1 -- 172.21.15.40:0/1108400951 wait complete. 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.614+0000 7f221d7af640 1 Processor -- start 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.614+0000 7f221d7af640 1 -- start start 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f221d7af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22180b5f20 0x7f22180cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f221d7af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180d39f0 0x7f221815b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f221d7af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f221806df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f221d7af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2218162c60 con 0x7f22180d39f0 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f221d7af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22180d0d90 con 0x7f22180b5f20 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f221d7af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2218162de0 con 0x7f2218162880 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180d39f0 0x7f221815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180d39f0 0x7f221815b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35582/0 (socket says 172.21.15.40:35582) 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 -- 172.21.15.40:0/2508270147 learned_addr learned my addr 172.21.15.40:0/2508270147 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f2216ffd640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22180b5f20 0x7f22180cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f22177fe640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f221806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 -- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 msgr2=0x7f221806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f221806df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 -- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22180b5f20 msgr2=0x7f22180cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22180b5f20 0x7f22180cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 -- 172.21.15.40:0/2508270147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221815bc90 con 0x7f22180d39f0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f22177fe640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f221806df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220e7fc640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180d39f0 0x7f221815b210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2200000c00 tx=0x7f22000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220ffff640 1 -- 172.21.15.40:0/2508270147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200041020 con 0x7f22180d39f0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220ffff640 1 -- 172.21.15.40:0/2508270147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22000306c0 con 0x7f22180d39f0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221815bf70 con 0x7f22180d39f0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f220ffff640 1 -- 172.21.15.40:0/2508270147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200037040 con 0x7f22180d39f0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.615+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22180cc040 con 0x7f22180d39f0 2024-04-26T21:35:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.616+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22180694f0 con 0x7f22180d39f0 2024-04-26T21:35:15.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.617+0000 7f220ffff640 1 -- 172.21.15.40:0/2508270147 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2200044020 con 0x7f22180d39f0 2024-04-26T21:35:15.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.617+0000 7f220ffff640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f806c8f0 0x7f21f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.617+0000 7f220ffff640 1 -- 172.21.15.40:0/2508270147 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f22000ac5e0 con 0x7f22180d39f0 2024-04-26T21:35:15.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.618+0000 7f2216ffd640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f806c8f0 0x7f21f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.618+0000 7f2216ffd640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f806c8f0 0x7f21f806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2208006fd0 tx=0x7f2208008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.622+0000 7f220ffff640 1 -- 172.21.15.40:0/2508270147 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f220004b030 con 0x7f22180d39f0 2024-04-26T21:35:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.806+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f221815fac0 con 0x7f22180d39f0 2024-04-26T21:35:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.807+0000 7f220ffff640 1 -- 172.21.15.40:0/2508270147 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f220007a6c0 con 0x7f22180d39f0 2024-04-26T21:35:15.809 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:15.809 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f806c8f0 msgr2=0x7f21f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f806c8f0 0x7f21f806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2208006fd0 tx=0x7f2208008040 comp rx=0 tx=0).stop 2024-04-26T21:35:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180d39f0 msgr2=0x7f221815b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180d39f0 0x7f221815b210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2200000c00 tx=0x7f22000046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 shutdown_connections 2024-04-26T21:35:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f806c8f0 0x7f21f806edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2218162880 0x7f221806df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22180d39f0 0x7f221815b210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 --2- 172.21.15.40:0/2508270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22180b5f20 0x7f22180cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.809+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 >> 172.21.15.40:0/2508270147 conn(0x7f22180d4f60 msgr2=0x7f22180ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.810+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 shutdown_connections 2024-04-26T21:35:15.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.810+0000 7f221d7af640 1 -- 172.21.15.40:0/2508270147 wait complete. 2024-04-26T21:35:15.822 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.921+0000 7f90fcd0a640 1 Processor -- start 2024-04-26T21:35:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.921+0000 7f90fcd0a640 1 -- start start 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90fcd0a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90f815b600 0x7f90f8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90fcd0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf0e0 0x7f90f806f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90fcd0a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f8072450 con 0x7f90f80cf5e0 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90fcd0a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f90f80cb9b0 con 0x7f90f815b600 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90fcd0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90f815b9e0 con 0x7f90f80cf0e0 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90f659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90f815b600 0x7f90f8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90f5d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f90f80cf5e0 0x7f90f80ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32792/0 (socket says 172.21.15.40:32792) 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90f5d9b640 1 -- 172.21.15.40:0/615673923 learned_addr learned my addr 172.21.15.40:0/615673923 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90f6d9d640 1 --2- 172.21.15.40:0/615673923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf0e0 0x7f90f806f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90f659c640 1 -- 172.21.15.40:0/615673923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf0e0 msgr2=0x7f90f806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90f659c640 1 --2- 172.21.15.40:0/615673923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf0e0 0x7f90f806f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.922+0000 7f90f659c640 1 -- 172.21.15.40:0/615673923 >> v1:172.21.15.40:6789/0 conn(0x7f90f80cf5e0 legacy=0x7f90f80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.923+0000 7f90f659c640 1 -- 172.21.15.40:0/615673923 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f80d0740 con 0x7f90f815b600 2024-04-26T21:35:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.923+0000 7f90f559a640 1 -- 172.21.15.40:0/615673923 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3889012232 0 0) 0x7f90e0001110 con 0x7f90f80cf5e0 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.923+0000 7f90f659c640 1 --2- 172.21.15.40:0/615673923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8071f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009d90 tx=0x7f90e802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d6eb45b885dbb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.923+0000 7f90f559a640 1 -- 172.21.15.40:0/615673923 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8007740 con 0x7f90f815b600 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.923+0000 7f90f559a640 1 -- 172.21.15.40:0/615673923 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e80078e0 con 0x7f90f815b600 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.923+0000 7f90f559a640 1 -- 172.21.15.40:0/615673923 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8007cb0 con 0x7f90f815b600 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.924+0000 7f90fcd0a640 1 -- 172.21.15.40:0/615673923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 msgr2=0x7f90f8071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.924+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/615673923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8071f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009d90 tx=0x7f90e802a660 comp rx=0 tx=0).stop 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.924+0000 7f90fcd0a640 1 -- 172.21.15.40:0/615673923 shutdown_connections 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.924+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/615673923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf0e0 0x7f90f806f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.924+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/615673923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8071f10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.924+0000 7f90fcd0a640 1 -- 172.21.15.40:0/615673923 >> 172.21.15.40:0/615673923 conn(0x7f90f80d0be0 msgr2=0x7f90f80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.924+0000 7f90fcd0a640 1 -- 172.21.15.40:0/615673923 shutdown_connections 2024-04-26T21:35:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.924+0000 7f90fcd0a640 1 -- 172.21.15.40:0/615673923 wait complete. 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.925+0000 7f90fcd0a640 1 Processor -- start 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.925+0000 7f90fcd0a640 1 -- start start 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90fcd0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90f80cf0e0 0x7f90f80b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90fcd0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf5e0 0x7f90f8154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90fcd0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90fcd0a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90f80d02f0 con 0x7f90f815b600 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90fcd0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90f80cb640 con 0x7f90f80cf0e0 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90fcd0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90f80d0470 con 0x7f90f80cf5e0 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35600/0 (socket says 172.21.15.40:35600) 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f6d9d640 1 -- 172.21.15.40:0/1430777789 learned_addr learned my addr 172.21.15.40:0/1430777789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f659c640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90f80cf0e0 0x7f90f80b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f5d9b640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf5e0 0x7f90f8154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f6d9d640 1 -- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf5e0 msgr2=0x7f90f8154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f6d9d640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf5e0 0x7f90f8154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f6d9d640 1 -- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90f80cf0e0 msgr2=0x7f90f80b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f6d9d640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90f80cf0e0 0x7f90f80b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.926+0000 7f90f6d9d640 1 -- 172.21.15.40:0/1430777789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f80b3ad0 con 0x7f90f815b600 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.927+0000 7f90f6d9d640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8154d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90f000d5a0 tx=0x7f90f000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.927+0000 7f90eeffd640 1 -- 172.21.15.40:0/1430777789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00125f0 con 0x7f90f815b600 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.927+0000 7f90eeffd640 1 -- 172.21.15.40:0/1430777789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f0019070 con 0x7f90f815b600 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.927+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f80b3db0 con 0x7f90f815b600 2024-04-26T21:35:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.927+0000 7f90eeffd640 1 -- 172.21.15.40:0/1430777789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00129d0 con 0x7f90f815b600 2024-04-26T21:35:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.927+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f80b4370 con 0x7f90f815b600 2024-04-26T21:35:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.928+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f80694f0 con 0x7f90f815b600 2024-04-26T21:35:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.928+0000 7f90eeffd640 1 -- 172.21.15.40:0/1430777789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90f0012b70 con 0x7f90f815b600 2024-04-26T21:35:15.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.929+0000 7f90eeffd640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d806c8f0 0x7f90d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:15.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.929+0000 7f90eeffd640 1 -- 172.21.15.40:0/1430777789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f90f008c290 con 0x7f90f815b600 2024-04-26T21:35:15.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.929+0000 7f90f659c640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d806c8f0 0x7f90d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:15.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.930+0000 7f90f659c640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d806c8f0 0x7f90d806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90e8034000 tx=0x7f90e8033040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:15.933+0000 7f90eeffd640 1 -- 172.21.15.40:0/1430777789 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90f005a380 con 0x7f90f815b600 2024-04-26T21:35:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.122+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90f80b4540 con 0x7f90f815b600 2024-04-26T21:35:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.124+0000 7f90eeffd640 1 -- 172.21.15.40:0/1430777789 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f90f0059c80 con 0x7f90f815b600 2024-04-26T21:35:16.125 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:16.125 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d806c8f0 msgr2=0x7f90d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d806c8f0 0x7f90d806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90e8034000 tx=0x7f90e8033040 comp rx=0 tx=0).stop 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 msgr2=0x7f90f8154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8154d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90f000d5a0 tx=0x7f90f000da60 comp rx=0 tx=0).stop 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 shutdown_connections 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d806c8f0 0x7f90d806edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f815b600 0x7f90f8154d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f80cf5e0 0x7f90f8154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 --2- 172.21.15.40:0/1430777789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90f80cf0e0 0x7f90f80b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 >> 172.21.15.40:0/1430777789 conn(0x7f90f80d0be0 msgr2=0x7f90f80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 shutdown_connections 2024-04-26T21:35:16.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.126+0000 7f90fcd0a640 1 -- 172.21.15.40:0/1430777789 wait complete. 2024-04-26T21:35:16.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.252+0000 7f791650a640 1 Processor -- start 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.252+0000 7f791650a640 1 -- start start 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f791650a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f791015f3a0 0x7f79100d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f791650a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79100d1770 0x7f79100d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f791650a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79100d2130 0x7f79100d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f791650a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f79100d25a0 con 0x7f79100d2130 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f791650a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f79100d0e20 con 0x7f791015f3a0 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f791650a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79100b1d70 con 0x7f79100d1770 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f7914aa7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79100d2130 0x7f79100d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f7914aa7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79100d2130 0x7f79100d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35610/0 (socket says 172.21.15.40:35610) 2024-04-26T21:35:16.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f7914aa7640 1 -- 172.21.15.40:0/2755030632 learned_addr learned my addr 172.21.15.40:0/2755030632 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f7914aa7640 1 -- 172.21.15.40:0/2755030632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79100d1770 msgr2=0x7f79100d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f7914aa7640 1 --2- 172.21.15.40:0/2755030632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79100d1770 0x7f79100d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f790ffff640 1 --2- 172.21.15.40:0/2755030632 >> v2:172.21.15.184:3300/0 conn(0x7f791015f3a0 0x7f79100d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f7914aa7640 1 -- 172.21.15.40:0/2755030632 >> v2:172.21.15.184:3300/0 conn(0x7f791015f3a0 msgr2=0x7f79100d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f7914aa7640 1 --2- 172.21.15.40:0/2755030632 >> v2:172.21.15.184:3300/0 conn(0x7f791015f3a0 0x7f79100d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f7914aa7640 1 -- 172.21.15.40:0/2755030632 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79100b2050 con 0x7f79100d2130 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.253+0000 7f790ffff640 1 --2- 172.21.15.40:0/2755030632 >> v2:172.21.15.184:3300/0 conn(0x7f791015f3a0 0x7f79100d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.254+0000 7f7914aa7640 1 --2- 172.21.15.40:0/2755030632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100d2e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7910073b50 tx=0x7f790802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f48c7dfe955b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.254+0000 7f790effd640 1 -- 172.21.15.40:0/2755030632 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908036070 con 0x7f79100d2130 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.254+0000 7f790effd640 1 -- 172.21.15.40:0/2755030632 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7908031600 con 0x7f79100d2130 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.254+0000 7f790effd640 1 -- 172.21.15.40:0/2755030632 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79080319d0 con 0x7f79100d2130 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.254+0000 7f791650a640 1 -- 172.21.15.40:0/2755030632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 msgr2=0x7f79100d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.254+0000 7f791650a640 1 --2- 172.21.15.40:0/2755030632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100d2e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7910073b50 tx=0x7f790802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.255+0000 7f791650a640 1 -- 172.21.15.40:0/2755030632 shutdown_connections 2024-04-26T21:35:16.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.255+0000 7f791650a640 1 --2- 172.21.15.40:0/2755030632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100d2e50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.255+0000 7f791650a640 1 --2- 172.21.15.40:0/2755030632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79100d1770 0x7f79100d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.255+0000 7f791650a640 1 --2- 172.21.15.40:0/2755030632 >> v2:172.21.15.184:3300/0 conn(0x7f791015f3a0 0x7f79100d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.255+0000 7f791650a640 1 -- 172.21.15.40:0/2755030632 >> 172.21.15.40:0/2755030632 conn(0x7f79100d4f90 msgr2=0x7f79100d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:16.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.255+0000 7f791650a640 1 -- 172.21.15.40:0/2755030632 shutdown_connections 2024-04-26T21:35:16.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.255+0000 7f791650a640 1 -- 172.21.15.40:0/2755030632 wait complete. 2024-04-26T21:35:16.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.256+0000 7f791650a640 1 Processor -- start 2024-04-26T21:35:16.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.256+0000 7f791650a640 1 -- start start 2024-04-26T21:35:16.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f791650a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79100d1770 0x7f79100ba040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f791650a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f791650a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791015f3a0 0x7f79101622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f791650a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f791015f900 con 0x7f79100d2130 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f791650a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f791015f780 con 0x7f79100d1770 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f791650a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79100d3390 con 0x7f791015f3a0 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35620/0 (socket says 172.21.15.40:35620) 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 -- 172.21.15.40:0/2180697040 learned_addr learned my addr 172.21.15.40:0/2180697040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790ffff640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79100d1770 0x7f79100ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f7914aa7640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791015f3a0 0x7f79101622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 -- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791015f3a0 msgr2=0x7f79101622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791015f3a0 0x7f79101622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 -- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79100d1770 msgr2=0x7f79100ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79100d1770 0x7f79100ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 -- 172.21.15.40:0/2180697040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79100b83b0 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790f7fe640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100b7b80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f790000b4d0 tx=0x7f790000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790cff9640 1 -- 172.21.15.40:0/2180697040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900014070 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790cff9640 1 -- 172.21.15.40:0/2180697040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7900010870 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79100b8690 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f790cff9640 1 -- 172.21.15.40:0/2180697040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900010c60 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.257+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79100b8c50 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.258+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79100694f0 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.259+0000 7f790cff9640 1 -- 172.21.15.40:0/2180697040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7900017070 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.259+0000 7f790cff9640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78fc06c8f0 0x7f78fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.259+0000 7f790cff9640 1 -- 172.21.15.40:0/2180697040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f790008bc00 con 0x7f79100d2130 2024-04-26T21:35:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.260+0000 7f790ffff640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78fc06c8f0 0x7f78fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.260+0000 7f790ffff640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78fc06c8f0 0x7f78fc06edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f790400a950 tx=0x7f7904008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:16.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.264+0000 7f790cff9640 1 -- 172.21.15.40:0/2180697040 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7900092080 con 0x7f79100d2130 2024-04-26T21:35:16.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.486+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79100b6860 con 0x7f79100d2130 2024-04-26T21:35:16.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.486+0000 7f790cff9640 1 -- 172.21.15.40:0/2180697040 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5180 (secure 0 0 0) 0x7f7900059d80 con 0x7f79100d2130 2024-04-26T21:35:16.488 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:16.488 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:14.527772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:35:16.488 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:35:16.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78fc06c8f0 msgr2=0x7f78fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78fc06c8f0 0x7f78fc06edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f790400a950 tx=0x7f7904008040 comp rx=0 tx=0).stop 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 msgr2=0x7f79100b7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100b7b80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f790000b4d0 tx=0x7f790000b990 comp rx=0 tx=0).stop 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 shutdown_connections 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78fc06c8f0 0x7f78fc06edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791015f3a0 0x7f79101622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79100d2130 0x7f79100b7b80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 --2- 172.21.15.40:0/2180697040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79100d1770 0x7f79100ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 >> 172.21.15.40:0/2180697040 conn(0x7f79100d4f90 msgr2=0x7f79100d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 shutdown_connections 2024-04-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.488+0000 7f791650a640 1 -- 172.21.15.40:0/2180697040 wait complete. 2024-04-26T21:35:16.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:16.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.600+0000 7f2d21059640 1 Processor -- start 2024-04-26T21:35:16.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.600+0000 7f2d21059640 1 -- start start 2024-04-26T21:35:16.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d21059640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d1c0d3bc0 0x7f2d1c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d21059640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c0b7e50 con 0x7f2d1c0b5f20 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d21059640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c0d1800 con 0x7f2d1c162880 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d21059640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c0d0580 con 0x7f2d1c0d3bc0 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1b59e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2d1c0b5f20 0x7f2d1c0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32796/0 (socket says 172.21.15.40:32796) 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1b59e640 1 -- 172.21.15.40:0/3330068945 learned_addr learned my addr 172.21.15.40:0/3330068945 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d13fff640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d1c0d3bc0 0x7f2d1c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196233894 0 0) 0x7f2d1c0b7e50 con 0x7f2d1c0b5f20 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d08003520 con 0x7f2d1c0b5f20 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 871069586 0 0) 0x7f2d08003520 con 0x7f2d1c0b5f20 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d1c0d3bc0 msgr2=0x7f2d1c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d1c0d3bc0 0x7f2d1c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 >> v1:172.21.15.184:6789/0 conn(0x7f2d1c162880 legacy=0x7f2d1c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c0d07d0 con 0x7f2d1c0b5f20 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f2d140019f0 con 0x7f2d1c0b5f20 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 >> v1:172.21.15.40:6789/0 conn(0x7f2d1c0b5f20 legacy=0x7f2d1c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080039e0 0x7f2d08003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d08004390 0x7f2d08004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.601+0000 7f2d1a59c640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d1c0d3bc0 0x7f2d080050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0d0580 con 0x7f2d080039e0 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0800b9c0 con 0x7f2d08004390 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0800bbc0 con 0x7f2d1c0d3bc0 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1ad9d640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080039e0 0x7f2d08003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d13fff640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d08004390 0x7f2d08004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1b59e640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d1c0d3bc0 0x7f2d080050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1ad9d640 1 -- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d1c0d3bc0 msgr2=0x7f2d080050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1ad9d640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d1c0d3bc0 0x7f2d080050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1ad9d640 1 -- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d08004390 msgr2=0x7f2d08004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1ad9d640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d08004390 0x7f2d08004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.602+0000 7f2d1ad9d640 1 -- 172.21.15.40:0/3330068945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d08003520 con 0x7f2d080039e0 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d1ad9d640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080039e0 0x7f2d08003dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001a30 tx=0x7f2d08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc6bc6803de1a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c037070 con 0x7f2d080039e0 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c02adf0 con 0x7f2d080039e0 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d1a59c640 1 -- 172.21.15.40:0/3330068945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c001600 con 0x7f2d080039e0 2024-04-26T21:35:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d21059640 1 -- 172.21.15.40:0/3330068945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080039e0 msgr2=0x7f2d08003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d21059640 1 --2- 172.21.15.40:0/3330068945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080039e0 0x7f2d08003dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001a30 tx=0x7f2d08005830 comp rx=0 tx=0).stop 2024-04-26T21:35:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d1b59e640 1 -- 172.21.15.40:0/3330068945 reap_dead start 2024-04-26T21:35:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d21059640 1 -- 172.21.15.40:0/3330068945 shutdown_connections 2024-04-26T21:35:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d21059640 1 -- 172.21.15.40:0/3330068945 >> 172.21.15.40:0/3330068945 conn(0x7f2d1c0d4f60 msgr2=0x7f2d1c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d21059640 1 -- 172.21.15.40:0/3330068945 shutdown_connections 2024-04-26T21:35:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.603+0000 7f2d21059640 1 -- 172.21.15.40:0/3330068945 wait complete. 2024-04-26T21:35:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d21059640 1 Processor -- start 2024-04-26T21:35:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d21059640 1 -- start start 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d21059640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080039e0 0x7f2d1c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d21059640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c0cbfa0 0x7f2d1c0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d21059640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d1c0b33e0 0x7f2d1c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d21059640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c162c60 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d21059640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0d0d90 con 0x7f2d1c0cbfa0 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d21059640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c162de0 con 0x7f2d080039e0 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.605+0000 7f2d1b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d1c0b33e0 0x7f2d1c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d1c0b33e0 0x7f2d1c0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35634/0 (socket says 172.21.15.40:35634) 2024-04-26T21:35:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1b59e640 1 -- 172.21.15.40:0/109812691 learned_addr learned my addr 172.21.15.40:0/109812691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1ad9d640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080039e0 0x7f2d1c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d13fff640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c0cbfa0 0x7f2d1c0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1b59e640 1 -- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080039e0 msgr2=0x7f2d1c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1b59e640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080039e0 0x7f2d1c153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1b59e640 1 -- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c0cbfa0 msgr2=0x7f2d1c0cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1b59e640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c0cbfa0 0x7f2d1c0cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1b59e640 1 -- 172.21.15.40:0/109812691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c0ccb40 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d13fff640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c0cbfa0 0x7f2d1c0cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d1b59e640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d1c0b33e0 0x7f2d1c0b3890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000f80 tx=0x7f2d14007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d137fe640 1 -- 172.21.15.40:0/109812691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400ace0 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d137fe640 1 -- 172.21.15.40:0/109812691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d14019070 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d137fe640 1 -- 172.21.15.40:0/109812691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14013630 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c0cce20 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.606+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c0cd3e0 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.607+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c0694f0 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.608+0000 7f2d137fe640 1 -- 172.21.15.40:0/109812691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d140137d0 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.608+0000 7f2d137fe640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cfc06c8f0 0x7f2cfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.608+0000 7f2d137fe640 1 -- 172.21.15.40:0/109812691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2d1408c550 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.608+0000 7f2d1ad9d640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cfc06c8f0 0x7f2cfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.609+0000 7f2d1ad9d640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cfc06c8f0 0x7f2cfc06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001a30 tx=0x7f2d0c009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.612+0000 7f2d137fe640 1 -- 172.21.15.40:0/109812691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d1405a640 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.798+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d1c06da10 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.799+0000 7f2d137fe640 1 -- 172.21.15.40:0/109812691 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f2d14059f40 con 0x7f2d1c0b33e0 2024-04-26T21:35:16.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:16.801 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cfc06c8f0 msgr2=0x7f2cfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cfc06c8f0 0x7f2cfc06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001a30 tx=0x7f2d0c009030 comp rx=0 tx=0).stop 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d1c0b33e0 msgr2=0x7f2d1c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d1c0b33e0 0x7f2d1c0b3890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000f80 tx=0x7f2d14007bd0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 shutdown_connections 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cfc06c8f0 0x7f2cfc06edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d1c0b33e0 0x7f2d1c0b3890 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c0cbfa0 0x7f2d1c0cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 --2- 172.21.15.40:0/109812691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080039e0 0x7f2d1c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 >> 172.21.15.40:0/109812691 conn(0x7f2d1c0d4f60 msgr2=0x7f2d1c160660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 shutdown_connections 2024-04-26T21:35:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:16.801+0000 7f2d21059640 1 -- 172.21.15.40:0/109812691 wait complete. 2024-04-26T21:35:17.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:35:09.479293+0000', 'modified': '2024-04-26T21:35:14.527772+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6834/2862109951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 2862109951}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 2862109951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4293, 'qdb_cluster': [4293]} 2024-04-26T21:35:17.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:35:17.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:35:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.323+0000 7fc715209640 1 Processor -- start 2024-04-26T21:35:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.324+0000 7fc715209640 1 -- start start 2024-04-26T21:35:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.324+0000 7fc715209640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc7100d0070 0x7fc7100d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.324+0000 7fc715209640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc71006d560 0x7fc71006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.324+0000 7fc715209640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7100d2420 con 0x7fc71006e0a0 2024-04-26T21:35:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.324+0000 7fc715209640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc71006e510 con 0x7fc71006d560 2024-04-26T21:35:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.324+0000 7fc715209640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc710162d00 con 0x7fc7100d0070 2024-04-26T21:35:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.324+0000 7fc70e59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc71006d560 0x7fc71006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70f59e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc71006e0a0 0x7fc710162f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32802/0 (socket says 172.21.15.40:32802) 2024-04-26T21:35:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70e59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc71006d560 0x7fc71006db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35636/0 (socket says 172.21.15.40:35636) 2024-04-26T21:35:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70f59e640 1 -- 172.21.15.40:0/3603895829 learned_addr learned my addr 172.21.15.40:0/3603895829 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/3603895829 >> v2:172.21.15.184:3300/0 conn(0x7fc7100d0070 0x7fc7100d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70e59c640 1 -- 172.21.15.40:0/3603895829 >> v2:172.21.15.184:3300/0 conn(0x7fc7100d0070 msgr2=0x7fc7100d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70e59c640 1 --2- 172.21.15.40:0/3603895829 >> v2:172.21.15.184:3300/0 conn(0x7fc7100d0070 0x7fc7100d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70e59c640 1 -- 172.21.15.40:0/3603895829 >> v1:172.21.15.40:6789/0 conn(0x7fc71006e0a0 legacy=0x7fc710162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70e59c640 1 -- 172.21.15.40:0/3603895829 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7100cd6b0 con 0x7fc71006d560 2024-04-26T21:35:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.325+0000 7fc70dd9b640 1 -- 172.21.15.40:0/3603895829 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136572360 0 0) 0x7fc7100d2420 con 0x7fc71006e0a0 2024-04-26T21:35:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc70e59c640 1 --2- 172.21.15.40:0/3603895829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006d560 0x7fc71006db60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc7100731d0 tx=0x7fc70402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9961f1a57530d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc70dd9b640 1 -- 172.21.15.40:0/3603895829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704037070 con 0x7fc71006d560 2024-04-26T21:35:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc70dd9b640 1 -- 172.21.15.40:0/3603895829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704032440 con 0x7fc71006d560 2024-04-26T21:35:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc70dd9b640 1 -- 172.21.15.40:0/3603895829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704032810 con 0x7fc71006d560 2024-04-26T21:35:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc715209640 1 -- 172.21.15.40:0/3603895829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006d560 msgr2=0x7fc71006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc715209640 1 --2- 172.21.15.40:0/3603895829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006d560 0x7fc71006db60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc7100731d0 tx=0x7fc70402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc715209640 1 -- 172.21.15.40:0/3603895829 shutdown_connections 2024-04-26T21:35:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc715209640 1 --2- 172.21.15.40:0/3603895829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006d560 0x7fc71006db60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc715209640 1 --2- 172.21.15.40:0/3603895829 >> v2:172.21.15.184:3300/0 conn(0x7fc7100d0070 0x7fc7100d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc715209640 1 -- 172.21.15.40:0/3603895829 >> 172.21.15.40:0/3603895829 conn(0x7fc7100d2ec0 msgr2=0x7fc7100d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc715209640 1 -- 172.21.15.40:0/3603895829 shutdown_connections 2024-04-26T21:35:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.326+0000 7fc715209640 1 -- 172.21.15.40:0/3603895829 wait complete. 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.328+0000 7fc715209640 1 Processor -- start 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.328+0000 7fc715209640 1 -- start start 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.328+0000 7fc715209640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc71006d560 0x7fc71015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.328+0000 7fc715209640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006e0a0 0x7fc7100c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.328+0000 7fc715209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d0070 0x7fc7100ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc715209640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100cdc70 con 0x7fc71006e0a0 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc715209640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100d2d10 con 0x7fc71006d560 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc715209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc7100cf1a0 con 0x7fc7100d0070 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006e0a0 0x7fc7100c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006e0a0 0x7fc7100c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35638/0 (socket says 172.21.15.40:35638) 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 -- 172.21.15.40:0/2886996626 learned_addr learned my addr 172.21.15.40:0/2886996626 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc71006d560 0x7fc71015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70f59e640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d0070 0x7fc7100ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 -- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d0070 msgr2=0x7fc7100ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d0070 0x7fc7100ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 -- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc71006d560 msgr2=0x7fc71015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc71006d560 0x7fc71015d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 -- 172.21.15.40:0/2886996626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71009f710 con 0x7fc71006e0a0 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc70e59c640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006e0a0 0x7fc7100c7330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc704035040 tx=0x7fc7040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc6f77fe640 1 -- 172.21.15.40:0/2886996626 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704037050 con 0x7fc71006e0a0 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc6f77fe640 1 -- 172.21.15.40:0/2886996626 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704038040 con 0x7fc71006e0a0 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc715209640 1 -- 172.21.15.40:0/2886996626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71009f9f0 con 0x7fc71006e0a0 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc6f77fe640 1 -- 172.21.15.40:0/2886996626 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704004470 con 0x7fc71006e0a0 2024-04-26T21:35:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.329+0000 7fc715209640 1 -- 172.21.15.40:0/2886996626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71009ffb0 con 0x7fc71006e0a0 2024-04-26T21:35:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.331+0000 7fc6f77fe640 1 -- 172.21.15.40:0/2886996626 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc704030900 con 0x7fc71006e0a0 2024-04-26T21:35:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.331+0000 7fc6f77fe640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.331+0000 7fc6f77fe640 1 -- 172.21.15.40:0/2886996626 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc7040ac940 con 0x7fc71006e0a0 2024-04-26T21:35:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.331+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.332+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc700006fd0 tx=0x7fc700008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.332+0000 7fc717fff740 1 Processor -- start 2024-04-26T21:35:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.332+0000 7fc717fff740 1 -- start start 2024-04-26T21:35:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc717fff740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f014440 0x55970f014820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc717fff740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f031ce0 0x55970f032230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc717fff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970efea120 0x55970f0303a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc717fff740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55970ef4e740 con 0x55970f014440 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc717fff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55970eeea3a0 con 0x55970efea120 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc717fff740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55970ef4a700 con 0x55970f031ce0 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f014440 0x55970f014820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f014440 0x55970f014820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35654/0 (socket says 172.21.15.40:35654) 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70ed9d640 1 -- 172.21.15.40:0/2527790212 learned_addr learned my addr 172.21.15.40:0/2527790212 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70f59e640 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970efea120 0x55970f0303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70e59c640 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f031ce0 0x55970f032230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70ed9d640 1 -- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f031ce0 msgr2=0x55970f032230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f031ce0 0x55970f032230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70ed9d640 1 -- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970efea120 msgr2=0x55970f0303a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970efea120 0x55970f0303a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70ed9d640 1 -- 172.21.15.40:0/2527790212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55970ef06750 con 0x55970f014440 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.333+0000 7fc70f59e640 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970efea120 0x55970f0303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f014440 0x55970f014820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e840 tx=0x7fc700011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3710554148e2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc6f4ff9640 1 -- 172.21.15.40:0/2527790212 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700025070 con 0x55970f014440 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc6f4ff9640 1 -- 172.21.15.40:0/2527790212 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc700015640 con 0x55970f014440 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc6f4ff9640 1 -- 172.21.15.40:0/2527790212 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70000f040 con 0x55970f014440 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc717fff740 1 -- 172.21.15.40:0/2527790212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f014440 msgr2=0x55970f014820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc717fff740 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f014440 0x55970f014820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e840 tx=0x7fc700011000 comp rx=0 tx=0).stop 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc717fff740 1 -- 172.21.15.40:0/2527790212 shutdown_connections 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc717fff740 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970efea120 0x55970f0303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc717fff740 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f031ce0 0x55970f032230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc717fff740 1 --2- 172.21.15.40:0/2527790212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f014440 0x55970f014820 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.334+0000 7fc717fff740 1 -- 172.21.15.40:0/2527790212 >> 172.21.15.40:0/2527790212 conn(0x55970efe6a50 msgr2=0x55970f02f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.335+0000 7fc717fff740 1 -- 172.21.15.40:0/2527790212 shutdown_connections 2024-04-26T21:35:17.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.335+0000 7fc717fff740 1 -- 172.21.15.40:0/2527790212 wait complete. 2024-04-26T21:35:17.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.335+0000 7fc717fff740 1 -- start start 2024-04-26T21:35:17.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.335+0000 7fc717fff740 1 Processor -- start 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc717fff740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970f00fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc717fff740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f03f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc717fff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 0x55970f027b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc717fff740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55970efeaec0 con 0x55970f031ce0 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc717fff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55970eed5fe0 con 0x55970f015100 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc717fff740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55970efe4980 con 0x55970f014440 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc717fff740 10 client.0 _ll_register_callbacks cb 0x55970eeec150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f03f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f03f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35656/0 (socket says 172.21.15.40:35656) 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70e59c640 1 -- 172.21.15.40:0/591397463 learned_addr learned my addr 172.21.15.40:0/591397463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970f00fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70f59e640 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 0x55970f027b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70e59c640 1 -- 172.21.15.40:0/591397463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 msgr2=0x55970f00fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70e59c640 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970f00fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70e59c640 1 -- 172.21.15.40:0/591397463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 msgr2=0x55970f027b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70e59c640 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 0x55970f027b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.336+0000 7fc70e59c640 1 -- 172.21.15.40:0/591397463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55970ec602d0 con 0x55970f031ce0 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc70e59c640 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f03f8c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc70407a350 tx=0x7fc704032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc6f9d2a15ca93f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc6edffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc6edffb640 1 -- 172.21.15.40:0/591397463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70407e770 con 0x55970f031ce0 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc6edffb640 1 -- 172.21.15.40:0/591397463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc704076a30 con 0x55970f031ce0 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 1 -- 172.21.15.40:0/591397463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc700015640 con 0x55970f031ce0 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc6edffb640 1 -- 172.21.15.40:0/591397463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fc704082fa0 con 0x55970f031ce0 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 10 client.4458 fetch_fsmap learned FSMap version 6 2024-04-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 1 -- 172.21.15.40:0/591397463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55970ed2a6b0 con 0x55970f031ce0 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc6edffb640 1 -- 172.21.15.40:0/591397463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc700015640 con 0x55970f031ce0 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc6edffb640 1 -- 172.21.15.40:0/591397463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7fc70407a620 con 0x55970f031ce0 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 10 client.4458 fetch_fsmap finished waiting for FSMap version 6 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 10 client.4458 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 20 client.4458 populate_metadata read hostname 'smithi040' 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55970f043080 0x55970f043460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 4 client.4458 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc717fff740 1 -- 172.21.15.40:0/591397463 --> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55970ef5c350 con 0x55970f043080 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.337+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55970f043080 0x55970f043460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc6edffb640 10 client.4458 ms_handle_connect on v2:172.21.15.40:6832/2862109951 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55970f043080 0x55970f043460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60834ad495400a2b server_cookie=914658e97dd7908e in_seq=0 out_seq=0 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc6edffb640 1 -- 172.21.15.40:0/591397463 <== mds.0 v2:172.21.15.40:6832/2862109951 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55970ef5c350 con 0x55970f043080 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc6edffb640 10 client.4458 handle_command_reply: tid=0 2024-04-26T21:35:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 client.4458 shutdown 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 -- 172.21.15.40:0/591397463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 msgr2=0x55970f03f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f03f8c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc70407a350 tx=0x7fc704032e40 comp rx=0 tx=0).stop 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 -- 172.21.15.40:0/591397463 shutdown_connections 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55970f043080 0x55970f043460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 0x55970f027b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f03f8c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc6edffb640 0 client.4458 ms_handle_reset on v2:172.21.15.40:6832/2862109951 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 --2- 172.21.15.40:0/591397463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970f00fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.338+0000 7fc717fff740 1 -- 172.21.15.40:0/591397463 >> 172.21.15.40:0/591397463 conn(0x55970efea120 msgr2=0x55970f0327d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.339+0000 7fc717fff740 1 -- 172.21.15.40:0/591397463 shutdown_connections 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.339+0000 7fc717fff740 1 -- 172.21.15.40:0/591397463 wait complete. 2024-04-26T21:35:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.339+0000 7fc717fff740 20 client.4458 trim_cache size 0 max 16384 2024-04-26T21:35:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.357+0000 7fc717fff740 1 Processor -- start 2024-04-26T21:35:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.357+0000 7fc717fff740 1 -- start start 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc717fff740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970efe3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc717fff740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f015100 0x55970f012d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc717fff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f031ce0 0x55970f028f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc717fff740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc710201a70 con 0x55970f015100 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc717fff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc710200dd0 con 0x55970f031ce0 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc717fff740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc710201420 con 0x55970f014440 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f015100 0x55970f012d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f015100 0x55970f012d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35658/0 (socket says 172.21.15.40:35658) 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 -- 172.21.15.40:0/3908065745 learned_addr learned my addr 172.21.15.40:0/3908065745 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 -- 172.21.15.40:0/3908065745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 msgr2=0x55970efe3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 --2- 172.21.15.40:0/3908065745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970efe3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 -- 172.21.15.40:0/3908065745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f031ce0 msgr2=0x55970f028f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70f59e640 1 --2- 172.21.15.40:0/3908065745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f031ce0 0x55970f028f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 --2- 172.21.15.40:0/3908065745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f031ce0 0x55970f028f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 -- 172.21.15.40:0/3908065745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55970f0132a0 con 0x55970f015100 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc70e59c640 1 --2- 172.21.15.40:0/3908065745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f015100 0x55970f012d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc70407a0c0 tx=0x7fc7040ac050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83aa136b971bf22c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc6effff640 1 -- 172.21.15.40:0/3908065745 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70407e770 con 0x55970f015100 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc6effff640 1 -- 172.21.15.40:0/3908065745 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70407a620 con 0x55970f015100 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.358+0000 7fc6effff640 1 -- 172.21.15.40:0/3908065745 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704082b10 con 0x55970f015100 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 -- 172.21.15.40:0/3908065745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f015100 msgr2=0x55970f012d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 --2- 172.21.15.40:0/3908065745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f015100 0x55970f012d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc70407a0c0 tx=0x7fc7040ac050 comp rx=0 tx=0).stop 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 -- 172.21.15.40:0/3908065745 shutdown_connections 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 --2- 172.21.15.40:0/3908065745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f031ce0 0x55970f028f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 --2- 172.21.15.40:0/3908065745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f015100 0x55970f012d60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 --2- 172.21.15.40:0/3908065745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970efe3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 -- 172.21.15.40:0/3908065745 >> 172.21.15.40:0/3908065745 conn(0x55970efea120 msgr2=0x55970f02f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 -- 172.21.15.40:0/3908065745 shutdown_connections 2024-04-26T21:35:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 -- 172.21.15.40:0/3908065745 wait complete. 2024-04-26T21:35:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 -- start start 2024-04-26T21:35:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.359+0000 7fc717fff740 1 Processor -- start 2024-04-26T21:35:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc717fff740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970f0aa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc717fff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 0x55970f016170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc717fff740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f032520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc717fff740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55970f0155e0 con 0x55970f031ce0 2024-04-26T21:35:17.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc717fff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55970f026550 con 0x55970f015100 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc717fff740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55970f026e70 con 0x55970f014440 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc717fff740 10 client.0 _ll_register_callbacks cb 0x7fc7040aaf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc70f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f032520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc70f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f032520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35670/0 (socket says 172.21.15.40:35670) 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc70f59e640 1 -- 172.21.15.40:0/163246848 learned_addr learned my addr 172.21.15.40:0/163246848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc70e59c640 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 0x55970f016170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.360+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970f0aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc70f59e640 1 -- 172.21.15.40:0/163246848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 msgr2=0x55970f0aa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc70f59e640 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970f0aa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc70f59e640 1 -- 172.21.15.40:0/163246848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 msgr2=0x55970f016170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc70f59e640 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 0x55970f016170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc70f59e640 1 -- 172.21.15.40:0/163246848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55970f00f1c0 con 0x55970f031ce0 2024-04-26T21:35:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc70f59e640 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f032520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc7080012e0 tx=0x7fc708001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4209e050a4a5531b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc6edffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc6edffb640 1 -- 172.21.15.40:0/163246848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70801a020 con 0x55970f031ce0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc6edffb640 1 -- 172.21.15.40:0/163246848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc708016660 con 0x55970f031ce0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 1 -- 172.21.15.40:0/163246848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70407a620 con 0x55970f031ce0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc6edffb640 1 -- 172.21.15.40:0/163246848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fc708024050 con 0x55970f031ce0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 10 client.4464 fetch_fsmap learned FSMap version 6 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 1 -- 172.21.15.40:0/163246848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55970f0166b0 con 0x55970f031ce0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc6edffb640 1 -- 172.21.15.40:0/163246848 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70407a620 con 0x55970f031ce0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc6edffb640 1 -- 172.21.15.40:0/163246848 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7fc708027020 con 0x55970f031ce0 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 10 client.4464 fetch_fsmap finished waiting for FSMap version 6 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 10 client.4464 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 20 client.4464 populate_metadata read hostname 'smithi040' 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55970f0309e0 0x55970f0adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 4 client.4464 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.361+0000 7fc717fff740 1 -- 172.21.15.40:0/163246848 --> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] -- command(tid 0: {"prefix": "status"}) -- 0x55970ef561c0 con 0x55970f0309e0 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55970f0309e0 0x55970f0adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc70ed9d640 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55970f0309e0 0x55970f0adb30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbdb9070e8660c5 server_cookie=c790d6514d8cc39b in_seq=0 out_seq=0 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc6edffb640 10 client.4464 ms_handle_connect on v2:172.21.15.40:6832/2862109951 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc6edffb640 1 -- 172.21.15.40:0/163246848 <== mds.0 v2:172.21.15.40:6832/2862109951 1 ==== command_reply(tid 0: 0 ) ==== 8+0+317 (crc 0 0 0) 0x55970ef561c0 con 0x55970f0309e0 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc6edffb640 10 client.4464 handle_command_reply: tid=0 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 client.4464 shutdown 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 -- 172.21.15.40:0/163246848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 msgr2=0x55970f032520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f032520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc7080012e0 tx=0x7fc708001a00 comp rx=0 tx=0).stop 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 -- 172.21.15.40:0/163246848 shutdown_connections 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55970f0309e0 0x55970f0adb30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55970f031ce0 0x55970f032520 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc6edffb640 0 client.4464 ms_handle_reset on v2:172.21.15.40:6832/2862109951 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55970f015100 0x55970f016170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 --2- 172.21.15.40:0/163246848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55970f014440 0x55970f0aa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 -- 172.21.15.40:0/163246848 >> 172.21.15.40:0/163246848 conn(0x55970efea120 msgr2=0x55970f03fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 -- 172.21.15.40:0/163246848 shutdown_connections 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 1 -- 172.21.15.40:0/163246848 wait complete. 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.362+0000 7fc717fff740 20 client.4464 trim_cache size 0 max 16384 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:35:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4293, 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 7.247364804, 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 6, 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 24, 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 23, 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 13.808135169 2024-04-26T21:35:17.371 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 -- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6fc06c8f0 msgr2=0x7fc6fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc700006fd0 tx=0x7fc700008040 comp rx=0 tx=0).stop 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 -- 172.21.15.40:0/2886996626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006e0a0 msgr2=0x7fc7100c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006e0a0 0x7fc7100c7330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc704035040 tx=0x7fc7040046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 -- 172.21.15.40:0/2886996626 shutdown_connections 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d0070 0x7fc7100ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc71006e0a0 0x7fc7100c7330 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 --2- 172.21.15.40:0/2886996626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc71006d560 0x7fc71015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 -- 172.21.15.40:0/2886996626 >> 172.21.15.40:0/2886996626 conn(0x7fc7100d2ec0 msgr2=0x7fc7100701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 -- 172.21.15.40:0/2886996626 shutdown_connections 2024-04-26T21:35:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.363+0000 7fc715209640 1 -- 172.21.15.40:0/2886996626 wait complete. 2024-04-26T21:35:17.374 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:35:17.375 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:35:17.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:35:17.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.473+0000 7efe38571640 1 Processor -- start 2024-04-26T21:35:17.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.473+0000 7efe38571640 1 -- start start 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.474+0000 7efe38571640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b4d40 0x7efe300b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.474+0000 7efe38571640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe300b6c50 0x7efe300aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.474+0000 7efe38571640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe30072fb0 con 0x7efe300b0b70 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.474+0000 7efe38571640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efe30073bc0 con 0x7efe300b6c50 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.474+0000 7efe38571640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe30073290 con 0x7efe300b4d40 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe36b0e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe300b6c50 0x7efe300aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe3630d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efe300b0b70 0x7efe300b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32804/0 (socket says 172.21.15.40:32804) 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe3630d640 1 -- 172.21.15.40:0/4094046225 learned_addr learned my addr 172.21.15.40:0/4094046225 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe35b0c640 1 --2- 172.21.15.40:0/4094046225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b4d40 0x7efe300b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe36b0e640 1 -- 172.21.15.40:0/4094046225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b4d40 msgr2=0x7efe300b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe36b0e640 1 --2- 172.21.15.40:0/4094046225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b4d40 0x7efe300b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe36b0e640 1 -- 172.21.15.40:0/4094046225 >> v1:172.21.15.40:6789/0 conn(0x7efe300b0b70 legacy=0x7efe300b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe36b0e640 1 -- 172.21.15.40:0/4094046225 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30073570 con 0x7efe300b6c50 2024-04-26T21:35:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.475+0000 7efe3530b640 1 -- 172.21.15.40:0/4094046225 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731025251 0 0) 0x7efe30072fb0 con 0x7efe300b0b70 2024-04-26T21:35:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe36b0e640 1 --2- 172.21.15.40:0/4094046225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b6c50 0x7efe300aed90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efe30073d40 tx=0x7efe2802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3174caad8f7a14ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe3530b640 1 -- 172.21.15.40:0/4094046225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2802ae60 con 0x7efe300b6c50 2024-04-26T21:35:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe3530b640 1 -- 172.21.15.40:0/4094046225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe28007740 con 0x7efe300b6c50 2024-04-26T21:35:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe3530b640 1 -- 172.21.15.40:0/4094046225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28007b10 con 0x7efe300b6c50 2024-04-26T21:35:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe38571640 1 -- 172.21.15.40:0/4094046225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b6c50 msgr2=0x7efe300aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe38571640 1 --2- 172.21.15.40:0/4094046225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b6c50 0x7efe300aed90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efe30073d40 tx=0x7efe2802a680 comp rx=0 tx=0).stop 2024-04-26T21:35:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe38571640 1 -- 172.21.15.40:0/4094046225 shutdown_connections 2024-04-26T21:35:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe38571640 1 --2- 172.21.15.40:0/4094046225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b6c50 0x7efe300aed90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe38571640 1 --2- 172.21.15.40:0/4094046225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b4d40 0x7efe300b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe38571640 1 -- 172.21.15.40:0/4094046225 >> 172.21.15.40:0/4094046225 conn(0x7efe300b39b0 msgr2=0x7efe300b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe38571640 1 -- 172.21.15.40:0/4094046225 shutdown_connections 2024-04-26T21:35:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.476+0000 7efe38571640 1 -- 172.21.15.40:0/4094046225 wait complete. 2024-04-26T21:35:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe38571640 1 Processor -- start 2024-04-26T21:35:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe38571640 1 -- start start 2024-04-26T21:35:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe38571640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe300b0b70 0x7efe301561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe38571640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b4d40 0x7efe30155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe38571640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b6c50 0x7efe301e7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe38571640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efe301604e0 con 0x7efe300b4d40 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe38571640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe30160360 con 0x7efe300b0b70 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe38571640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe300af2d0 con 0x7efe300b6c50 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.478+0000 7efe35b0c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b4d40 0x7efe30155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe35b0c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b4d40 0x7efe30155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35680/0 (socket says 172.21.15.40:35680) 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe35b0c640 1 -- 172.21.15.40:0/1311738756 learned_addr learned my addr 172.21.15.40:0/1311738756 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe3630d640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe300b0b70 0x7efe301561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe36b0e640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b6c50 0x7efe301e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe35b0c640 1 -- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b6c50 msgr2=0x7efe301e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe35b0c640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b6c50 0x7efe301e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe35b0c640 1 -- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe300b0b70 msgr2=0x7efe301561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe35b0c640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe300b0b70 0x7efe301561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe35b0c640 1 -- 172.21.15.40:0/1311738756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301463e0 con 0x7efe300b4d40 2024-04-26T21:35:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe35b0c640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b4d40 0x7efe30155270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe2000e9b0 tx=0x7efe2000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe26ffd640 1 -- 172.21.15.40:0/1311738756 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20017070 con 0x7efe300b4d40 2024-04-26T21:35:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe26ffd640 1 -- 172.21.15.40:0/1311738756 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2000c9a0 con 0x7efe300b4d40 2024-04-26T21:35:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe30143540 con 0x7efe300b4d40 2024-04-26T21:35:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe26ffd640 1 -- 172.21.15.40:0/1311738756 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2000cd70 con 0x7efe300b4d40 2024-04-26T21:35:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.479+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe30143b00 con 0x7efe300b4d40 2024-04-26T21:35:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.480+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe300694f0 con 0x7efe300b4d40 2024-04-26T21:35:17.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.481+0000 7efe26ffd640 1 -- 172.21.15.40:0/1311738756 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe2001a070 con 0x7efe300b4d40 2024-04-26T21:35:17.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.481+0000 7efe26ffd640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe1406c8f0 0x7efe1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:17.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.481+0000 7efe26ffd640 1 -- 172.21.15.40:0/1311738756 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efe2008bd50 con 0x7efe300b4d40 2024-04-26T21:35:17.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.481+0000 7efe3630d640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe1406c8f0 0x7efe1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:17.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.482+0000 7efe3630d640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe1406c8f0 0x7efe1406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000f60 tx=0x7efe2c007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.485+0000 7efe26ffd640 1 -- 172.21.15.40:0/1311738756 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe2008f030 con 0x7efe300b4d40 2024-04-26T21:35:17.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.707+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efe300b0600 con 0x7efe300b4d40 2024-04-26T21:35:17.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.708+0000 7efe26ffd640 1 -- 172.21.15.40:0/1311738756 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7efe20059880 con 0x7efe300b4d40 2024-04-26T21:35:17.709 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe1406c8f0 msgr2=0x7efe1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe1406c8f0 0x7efe1406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000f60 tx=0x7efe2c007d50 comp rx=0 tx=0).stop 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b4d40 msgr2=0x7efe30155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b4d40 0x7efe30155270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe2000e9b0 tx=0x7efe2000ee70 comp rx=0 tx=0).stop 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 shutdown_connections 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe1406c8f0 0x7efe1406edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe300b6c50 0x7efe301e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe300b4d40 0x7efe30155270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 --2- 172.21.15.40:0/1311738756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe300b0b70 0x7efe301561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.709+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 >> 172.21.15.40:0/1311738756 conn(0x7efe300b39b0 msgr2=0x7efe300afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:17.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.710+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 shutdown_connections 2024-04-26T21:35:17.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17.710+0000 7efe38571640 1 -- 172.21.15.40:0/1311738756 wait complete. 2024-04-26T21:35:17.721 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:35:17.721 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:35:17.721 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:35:17.721 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T21:35:17.721 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:35:17.721 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:35:17.721 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:35:17.721 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:35:17.722 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T21:35:17.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:17.722 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:35:17.737 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:17.737 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:17.737 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:17.737 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:35:17.738 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42730sec preferred_lft 42730sec 2024-04-26T21:35:17.739 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:35:17.739 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:17.739 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:17.739 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:17.739 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:35:17.739 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:35:17.739 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:17.740 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:17.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:35:17.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-04-26T21:35:17.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-04-26T21:35:17.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:35:17.740 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:17.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:17.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:17.920 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:35:17.944 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:35:17.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:17.946 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:35:18.008 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:35:18.008 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:35:18.008 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:35:18.008 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:35:18.008 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:35:18.008 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:18.008 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:18.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:35:18.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:35:18.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:35:18.009 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:18.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:18.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:18.294 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:35:18.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:18.310 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-04-26T21:35:18.365 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:18.365 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:18.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:35:18.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T21:35:18.365 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:18.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:18.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:18.483 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T21:35:18.483 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:18.483 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:18.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T21:35:18.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:35:18.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T21:35:18.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T21:35:18.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T21:35:18.483 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:18.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:18.608 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:18.608 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:18.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:35:18.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:35:18.608 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:18.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:18.685 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T21:35:18.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:18.685 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:35:18.700 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T21:35:18.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:18.700 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:35:18.755 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-26T21:35:18.820 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:35:18.876 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-26T21:35:18.876 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-26T21:35:18.876 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-26T21:35:18.876 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-26T21:35:18.876 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/selinux 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-26T21:35:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:35:18.878 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:35:18.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:18.879 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-26T21:35:18.932 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:35:18.932 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T21:35:18.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-26T21:35:18.998 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:35:19.053 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/selinux 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:35:19.054 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T21:35:19.055 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T21:35:19.056 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T21:35:19.056 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-26T21:35:19.056 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:35:19.056 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:35:19.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:19.056 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-26T21:35:19.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:2024-04-26T21:35:19.082+0000 7fcf53322480 -1 init, newargv = 0x55652a4aa4b0 newargc=15 2024-04-26T21:35:19.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[42133]: starting ceph client 2024-04-26T21:35:19.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[42133]: starting fuse 2024-04-26T21:35:19.098 INFO:teuthology.orchestra.run.smithi040.stdout:43 2024-04-26T21:35:19.098 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-26T21:35:19.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> import re 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:35:19.099 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> return f 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.admin")) 2024-04-26T21:35:19.100 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:35:19.219 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.admin.42133.asok 2024-04-26T21:35:19.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:19.229 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42133.asok 2024-04-26T21:35:19.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:19.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42133.asok status 2024-04-26T21:35:19.364 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "admin", 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "42133", 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4482, 2024-04-26T21:35:19.365 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4482 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1995408462 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-26T21:35:19.366 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1995408462 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4482 192.168.0.1:0/1995408462", 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/1995408462", 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 6, 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 24, 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2024-04-26T21:35:19.367 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:35:19.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:19.373 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2024-04-26T21:35:19.373 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:35:19.389 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2024-04-26T21:35:19.389 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:35:19.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:19.389 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:35:19.454 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T21:35:19.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:19.561 INFO:teuthology.orchestra.run.smithi040.stdout:# file: . 2024-04-26T21:35:19.561 INFO:teuthology.orchestra.run.smithi040.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T21:35:19.561 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:19.568 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:35:19.568 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:35:19.584 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.724+0000 7f26feb24640 1 Processor -- start 2024-04-26T21:35:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.724+0000 7f26feb24640 1 -- start start 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26feb24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26feb24640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f26f8070ae0 0x7f26f8070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26feb24640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f80d1ce0 con 0x7f26f815d1e0 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26feb24640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80cede0 con 0x7f26f8070ae0 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26feb24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26f8070650 con 0x7f26f80d1380 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26fc8c0640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f26f815d1e0 0x7f26f80d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:60984/0 (socket says 172.21.15.40:60984) 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26fd0c1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f26f8070ae0 0x7f26f8070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.725+0000 7f26fc8c0640 1 -- 172.21.15.40:0/4278255476 learned_addr learned my addr 172.21.15.40:0/4278255476 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.726+0000 7f26f7fff640 1 -- 172.21.15.40:0/4278255476 >> v2:172.21.15.184:3300/0 conn(0x7f26f8070ae0 msgr2=0x7f26f8070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:19.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.726+0000 7f26f7fff640 1 --2- 172.21.15.40:0/4278255476 >> v2:172.21.15.184:3300/0 conn(0x7f26f8070ae0 0x7f26f8070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.726+0000 7f26f7fff640 1 -- 172.21.15.40:0/4278255476 >> v1:172.21.15.184:6789/0 conn(0x7f26f815d1e0 legacy=0x7f26f80d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:19.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.726+0000 7f26f7fff640 1 -- 172.21.15.40:0/4278255476 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f80ce840 con 0x7f26f80d1380 2024-04-26T21:35:19.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.726+0000 7f26fd0c1640 1 --2- 172.21.15.40:0/4278255476 >> v2:172.21.15.184:3300/0 conn(0x7f26f8070ae0 0x7f26f8070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.726+0000 7f26f7fff640 1 --2- 172.21.15.40:0/4278255476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80d17a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0097a0 tx=0x7f26ec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e30eb13da6f56f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26f77fe640 1 -- 172.21.15.40:0/4278255476 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec037070 con 0x7f26f80d1380 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26f77fe640 1 -- 172.21.15.40:0/4278255476 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ec032440 con 0x7f26f80d1380 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26f77fe640 1 -- 172.21.15.40:0/4278255476 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec032810 con 0x7f26f80d1380 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26feb24640 1 -- 172.21.15.40:0/4278255476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 msgr2=0x7f26f80d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26feb24640 1 --2- 172.21.15.40:0/4278255476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80d17a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0097a0 tx=0x7f26ec02a180 comp rx=0 tx=0).stop 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26feb24640 1 -- 172.21.15.40:0/4278255476 shutdown_connections 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26feb24640 1 --2- 172.21.15.40:0/4278255476 >> v2:172.21.15.184:3300/0 conn(0x7f26f8070ae0 0x7f26f8070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26feb24640 1 --2- 172.21.15.40:0/4278255476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80d17a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26feb24640 1 -- 172.21.15.40:0/4278255476 >> 172.21.15.40:0/4278255476 conn(0x7f26f80d2da0 msgr2=0x7f26f80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26feb24640 1 -- 172.21.15.40:0/4278255476 shutdown_connections 2024-04-26T21:35:19.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.727+0000 7f26feb24640 1 -- 172.21.15.40:0/4278255476 wait complete. 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.728+0000 7f26feb24640 1 Processor -- start 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.728+0000 7f26feb24640 1 -- start start 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26feb24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26f8070ae0 0x7f26f8159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26feb24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26feb24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26f815d1e0 0x7f26f80c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26feb24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80d10f0 con 0x7f26f815d1e0 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26feb24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80702e0 con 0x7f26f8070ae0 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26feb24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26f80ce4c0 con 0x7f26f80d1380 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26fd0c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26f815d1e0 0x7f26f80c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26fd0c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26f815d1e0 0x7f26f80c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57506/0 (socket says 172.21.15.40:57506) 2024-04-26T21:35:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.729+0000 7f26fd0c1640 1 -- 172.21.15.40:0/2761578270 learned_addr learned my addr 172.21.15.40:0/2761578270 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26f7fff640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26fc8c0640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26f8070ae0 0x7f26f8159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26fd0c1640 1 -- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 msgr2=0x7f26f80c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26fd0c1640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26fd0c1640 1 -- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26f8070ae0 msgr2=0x7f26f8159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26fd0c1640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26f8070ae0 0x7f26f8159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26fd0c1640 1 -- 172.21.15.40:0/2761578270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f81e8a50 con 0x7f26f815d1e0 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26fc8c0640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26f8070ae0 0x7f26f8159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26fd0c1640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26f815d1e0 0x7f26f80c5b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f26f000efc0 tx=0x7f26f0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26f57fa640 1 -- 172.21.15.40:0/2761578270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0018030 con 0x7f26f815d1e0 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26f57fa640 1 -- 172.21.15.40:0/2761578270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f000cde0 con 0x7f26f815d1e0 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26f57fa640 1 -- 172.21.15.40:0/2761578270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0003bb0 con 0x7f26f815d1e0 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f80c6a70 con 0x7f26f815d1e0 2024-04-26T21:35:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.730+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f80c7030 con 0x7f26f815d1e0 2024-04-26T21:35:19.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.731+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f80694f0 con 0x7f26f815d1e0 2024-04-26T21:35:19.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.732+0000 7f26f57fa640 1 -- 172.21.15.40:0/2761578270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26f001b070 con 0x7f26f815d1e0 2024-04-26T21:35:19.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.732+0000 7f26f57fa640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e406c8f0 0x7f26e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:19.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.732+0000 7f26f57fa640 1 -- 172.21.15.40:0/2761578270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f26f008c8a0 con 0x7f26f815d1e0 2024-04-26T21:35:19.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.732+0000 7f26fc8c0640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e406c8f0 0x7f26e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:19.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.733+0000 7f26fc8c0640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e406c8f0 0x7f26e406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26f80cc770 tx=0x7f26e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:19.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.737+0000 7f26f57fa640 1 -- 172.21.15.40:0/2761578270 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26f0010040 con 0x7f26f815d1e0 2024-04-26T21:35:19.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.921+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26f80b1ba0 con 0x7f26f815d1e0 2024-04-26T21:35:19.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.922+0000 7f26f57fa640 1 -- 172.21.15.40:0/2761578270 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f26f005aa20 con 0x7f26f815d1e0 2024-04-26T21:35:19.923 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:19.923 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e406c8f0 msgr2=0x7f26e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e406c8f0 0x7f26e406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26f80cc770 tx=0x7f26e8009040 comp rx=0 tx=0).stop 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26f815d1e0 msgr2=0x7f26f80c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26f815d1e0 0x7f26f80c5b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f26f000efc0 tx=0x7f26f0009f90 comp rx=0 tx=0).stop 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 shutdown_connections 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e406c8f0 0x7f26e406edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26f815d1e0 0x7f26f80c5b20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26f80d1380 0x7f26f80c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 --2- 172.21.15.40:0/2761578270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26f8070ae0 0x7f26f8159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 >> 172.21.15.40:0/2761578270 conn(0x7f26f80d2da0 msgr2=0x7f26f80cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 shutdown_connections 2024-04-26T21:35:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:19.924+0000 7f26feb24640 1 -- 172.21.15.40:0/2761578270 wait complete. 2024-04-26T21:35:19.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:19.937 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:20.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.040+0000 7f5dfe19e640 1 Processor -- start 2024-04-26T21:35:20.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.040+0000 7f5dfe19e640 1 -- start start 2024-04-26T21:35:20.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.040+0000 7f5dfe19e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5df81628f0 0x7f5df80d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.040+0000 7f5dfe19e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df8162cd0 0x7f5df80d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.040+0000 7f5dfe19e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df80d0890 con 0x7f5df80d1550 2024-04-26T21:35:20.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.040+0000 7f5dfe19e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5df80d1130 con 0x7f5df81628f0 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.040+0000 7f5dfe19e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80d0b70 con 0x7f5df8162cd0 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.041+0000 7f5df77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5df81628f0 0x7f5df80d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.041+0000 7f5df77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5df81628f0 0x7f5df80d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57518/0 (socket says 172.21.15.40:57518) 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.041+0000 7f5df77fe640 1 -- 172.21.15.40:0/1992438357 learned_addr learned my addr 172.21.15.40:0/1992438357 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.041+0000 7f5df6ffd640 1 --2- 172.21.15.40:0/1992438357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df8162cd0 0x7f5df80d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.041+0000 7f5df77fe640 1 -- 172.21.15.40:0/1992438357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df8162cd0 msgr2=0x7f5df80d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.041+0000 7f5df77fe640 1 --2- 172.21.15.40:0/1992438357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df8162cd0 0x7f5df80d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.041+0000 7f5df77fe640 1 -- 172.21.15.40:0/1992438357 >> v1:172.21.15.184:6789/0 conn(0x7f5df80d1550 legacy=0x7f5df80d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.041+0000 7f5df77fe640 1 -- 172.21.15.40:0/1992438357 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80d0e50 con 0x7f5df81628f0 2024-04-26T21:35:20.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5df77fe640 1 --2- 172.21.15.40:0/1992438357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df80d1a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009870 tx=0x7f5de802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277418a6502cbdc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5df67fc640 1 -- 172.21.15.40:0/1992438357 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8037070 con 0x7f5df81628f0 2024-04-26T21:35:20.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5df67fc640 1 -- 172.21.15.40:0/1992438357 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de8032440 con 0x7f5df81628f0 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5df67fc640 1 -- 172.21.15.40:0/1992438357 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8032810 con 0x7f5df81628f0 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5dfe19e640 1 -- 172.21.15.40:0/1992438357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 msgr2=0x7f5df80d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/1992438357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df80d1a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009870 tx=0x7f5de802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5dfe19e640 1 -- 172.21.15.40:0/1992438357 shutdown_connections 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/1992438357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df8162cd0 0x7f5df80d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/1992438357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df80d1a00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5dfe19e640 1 -- 172.21.15.40:0/1992438357 >> 172.21.15.40:0/1992438357 conn(0x7f5df80d4ff0 msgr2=0x7f5df80d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5dfe19e640 1 -- 172.21.15.40:0/1992438357 shutdown_connections 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.042+0000 7f5dfe19e640 1 -- 172.21.15.40:0/1992438357 wait complete. 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.043+0000 7f5dfe19e640 1 Processor -- start 2024-04-26T21:35:20.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5dfe19e640 1 -- start start 2024-04-26T21:35:20.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5dfe19e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80d1550 0x7f5df80cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5dfe19e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df8155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5dfe19e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df8162cd0 0x7f5df80c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5dfe19e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80b5df0 con 0x7f5df81628f0 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5dfe19e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80b5c70 con 0x7f5df8162cd0 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5dfe19e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80b5f70 con 0x7f5df80d1550 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df8155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df8155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57520/0 (socket says 172.21.15.40:57520) 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.044+0000 7f5df6ffd640 1 -- 172.21.15.40:0/2970118266 learned_addr learned my addr 172.21.15.40:0/2970118266 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df7fff640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df8162cd0 0x7f5df80c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df77fe640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80d1550 0x7f5df80cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df6ffd640 1 -- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80d1550 msgr2=0x7f5df80cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df6ffd640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80d1550 0x7f5df80cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df6ffd640 1 -- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df8162cd0 msgr2=0x7f5df80c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df6ffd640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df8162cd0 0x7f5df80c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df6ffd640 1 -- 172.21.15.40:0/2970118266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80ca720 con 0x7f5df81628f0 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df7fff640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df8162cd0 0x7f5df80c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5df6ffd640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df8155400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00d5f0 tx=0x7f5dec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5ddffff640 1 -- 172.21.15.40:0/2970118266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0125f0 con 0x7f5df81628f0 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5ddffff640 1 -- 172.21.15.40:0/2970118266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec019070 con 0x7f5df81628f0 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df80caa00 con 0x7f5df81628f0 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5ddffff640 1 -- 172.21.15.40:0/2970118266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0129d0 con 0x7f5df81628f0 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.045+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df80cafc0 con 0x7f5df81628f0 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.046+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df80694f0 con 0x7f5df81628f0 2024-04-26T21:35:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.047+0000 7f5ddffff640 1 -- 172.21.15.40:0/2970118266 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dec012b70 con 0x7f5df81628f0 2024-04-26T21:35:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.047+0000 7f5ddffff640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5de406c8f0 0x7f5de406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.047+0000 7f5ddffff640 1 -- 172.21.15.40:0/2970118266 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5dec08c260 con 0x7f5df81628f0 2024-04-26T21:35:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.047+0000 7f5df77fe640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5de406c8f0 0x7f5de406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.048+0000 7f5df77fe640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5de406c8f0 0x7f5de406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5de8034000 tx=0x7f5de802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.051+0000 7f5ddffff640 1 -- 172.21.15.40:0/2970118266 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5dec05a350 con 0x7f5df81628f0 2024-04-26T21:35:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.237+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5df80b3b10 con 0x7f5df81628f0 2024-04-26T21:35:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.238+0000 7f5ddffff640 1 -- 172.21.15.40:0/2970118266 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f5dec059c50 con 0x7f5df81628f0 2024-04-26T21:35:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5de406c8f0 msgr2=0x7f5de406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5de406c8f0 0x7f5de406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5de8034000 tx=0x7f5de802e000 comp rx=0 tx=0).stop 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 msgr2=0x7f5df8155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df8155400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00d5f0 tx=0x7f5dec00dab0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 shutdown_connections 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5de406c8f0 0x7f5de406edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df8162cd0 0x7f5df80c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df81628f0 0x7f5df8155400 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 --2- 172.21.15.40:0/2970118266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80d1550 0x7f5df80cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.239+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 >> 172.21.15.40:0/2970118266 conn(0x7f5df80d4ff0 msgr2=0x7f5df80b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.240+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 shutdown_connections 2024-04-26T21:35:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.240+0000 7f5dfe19e640 1 -- 172.21.15.40:0/2970118266 wait complete. 2024-04-26T21:35:20.251 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:20.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.353+0000 7f2aefba4640 1 Processor -- start 2024-04-26T21:35:20.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.354+0000 7f2aefba4640 1 -- start start 2024-04-26T21:35:20.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.354+0000 7f2aefba4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae815d100 0x7f2ae8160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.354+0000 7f2aefba4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2ae8070930 0x7f2ae80aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.354+0000 7f2aefba4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae80b0990 con 0x7f2ae806c240 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.354+0000 7f2aefba4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae80af2d0 con 0x7f2ae8070930 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.354+0000 7f2aefba4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae80b0c70 con 0x7f2ae815d100 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aee141640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2ae806c240 0x7f2ae8073160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45160/0 (socket says 172.21.15.40:45160) 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aee141640 1 -- 172.21.15.40:0/2135410641 learned_addr learned my addr 172.21.15.40:0/2135410641 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aed13f640 1 --2- 172.21.15.40:0/2135410641 >> v2:172.21.15.184:3300/0 conn(0x7f2ae8070930 0x7f2ae80aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aed940640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae815d100 0x7f2ae8160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909837417 0 0) 0x7f2ae80b0990 con 0x7f2ae806c240 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ad4003520 con 0x7f2ae806c240 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 977116588 0 0) 0x7f2ad4003520 con 0x7f2ae806c240 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae815d100 msgr2=0x7f2ae8160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae815d100 0x7f2ae8160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 >> v2:172.21.15.184:3300/0 conn(0x7f2ae8070930 msgr2=0x7f2ae80aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 --2- 172.21.15.40:0/2135410641 >> v2:172.21.15.184:3300/0 conn(0x7f2ae8070930 0x7f2ae80aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae80b0f50 con 0x7f2ae806c240 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f2ae00019f0 con 0x7f2ae806c240 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 >> v1:172.21.15.40:6789/0 conn(0x7f2ae806c240 legacy=0x7f2ae8073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae815d100 0x7f2ad40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad4003f30 0x7f2ad4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4004d40 0x7f2ad40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae80b0c70 con 0x7f2ad4003f30 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad400b9c0 con 0x7f2ad4004d40 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad400bb40 con 0x7f2ae815d100 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aed13f640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad4003f30 0x7f2ad4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aed940640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae815d100 0x7f2ad40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.355+0000 7f2aee141640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4004d40 0x7f2ad40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aed13f640 1 -- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae815d100 msgr2=0x7f2ad40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aed13f640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae815d100 0x7f2ad40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aed13f640 1 -- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4004d40 msgr2=0x7f2ad40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aed13f640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4004d40 0x7f2ad40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aed13f640 1 -- 172.21.15.40:0/2135410641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad4003520 con 0x7f2ad4003f30 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aed13f640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad4003f30 0x7f2ad4004800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009ac0 tx=0x7f2ad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbf5912afe53914 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8036070 con 0x7f2ad4003f30 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad8031400 con 0x7f2ad4003f30 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.356+0000 7f2aec93e640 1 -- 172.21.15.40:0/2135410641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8009690 con 0x7f2ad4003f30 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.357+0000 7f2aefba4640 1 -- 172.21.15.40:0/2135410641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad4003f30 msgr2=0x7f2ad4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.357+0000 7f2aefba4640 1 --2- 172.21.15.40:0/2135410641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad4003f30 0x7f2ad4004800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009ac0 tx=0x7f2ad4005830 comp rx=0 tx=0).stop 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.357+0000 7f2aee141640 1 -- 172.21.15.40:0/2135410641 reap_dead start 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.357+0000 7f2aefba4640 1 -- 172.21.15.40:0/2135410641 shutdown_connections 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.357+0000 7f2aefba4640 1 -- 172.21.15.40:0/2135410641 >> 172.21.15.40:0/2135410641 conn(0x7f2ae80b39b0 msgr2=0x7f2ae80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.357+0000 7f2aefba4640 1 -- 172.21.15.40:0/2135410641 shutdown_connections 2024-04-26T21:35:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.357+0000 7f2aefba4640 1 -- 172.21.15.40:0/2135410641 wait complete. 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.358+0000 7f2aefba4640 1 Processor -- start 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.358+0000 7f2aefba4640 1 -- start start 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aefba4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4003f30 0x7f2ae80b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aefba4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae81503f0 0x7f2ae80b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aefba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8150910 0x7f2ae80b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aefba4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae8160ba0 con 0x7f2ae8150910 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aefba4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae80b1510 con 0x7f2ad4003f30 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aefba4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae8160d20 con 0x7f2ae81503f0 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8150910 0x7f2ae80b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8150910 0x7f2ae80b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57550/0 (socket says 172.21.15.40:57550) 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 -- 172.21.15.40:0/2921663467 learned_addr learned my addr 172.21.15.40:0/2921663467 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aed940640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4003f30 0x7f2ae80b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aed13f640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae81503f0 0x7f2ae80b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 -- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae81503f0 msgr2=0x7f2ae80b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae81503f0 0x7f2ae80b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 -- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4003f30 msgr2=0x7f2ae80b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4003f30 0x7f2ae80b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 -- 172.21.15.40:0/2921663467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae8151e70 con 0x7f2ae8150910 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aed13f640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae81503f0 0x7f2ae80b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.359+0000 7f2aee141640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8150910 0x7f2ae80b30a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0002230 tx=0x7f2ae0007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.360+0000 7f2ade7fc640 1 -- 172.21.15.40:0/2921663467 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0019070 con 0x7f2ae8150910 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.360+0000 7f2ade7fc640 1 -- 172.21.15.40:0/2921663467 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae000ace0 con 0x7f2ae8150910 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.360+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae8152150 con 0x7f2ae8150910 2024-04-26T21:35:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.360+0000 7f2ade7fc640 1 -- 172.21.15.40:0/2921663467 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00135f0 con 0x7f2ae8150910 2024-04-26T21:35:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.360+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae8152710 con 0x7f2ae8150910 2024-04-26T21:35:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.361+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae80694f0 con 0x7f2ae8150910 2024-04-26T21:35:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.361+0000 7f2ade7fc640 1 -- 172.21.15.40:0/2921663467 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ae001c070 con 0x7f2ae8150910 2024-04-26T21:35:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.362+0000 7f2ade7fc640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2acc06c8f0 0x7f2acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.362+0000 7f2ade7fc640 1 -- 172.21.15.40:0/2921663467 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ae008ba50 con 0x7f2ae8150910 2024-04-26T21:35:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.362+0000 7f2aed940640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2acc06c8f0 0x7f2acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.362+0000 7f2aed940640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2acc06c8f0 0x7f2acc06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80b3840 tx=0x7f2ae4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.366+0000 7f2ade7fc640 1 -- 172.21.15.40:0/2921663467 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2ae00594f0 con 0x7f2ae8150910 2024-04-26T21:35:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.553+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2ae806cd70 con 0x7f2ae8150910 2024-04-26T21:35:20.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.554+0000 7f2ade7fc640 1 -- 172.21.15.40:0/2921663467 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f2ae005d7f0 con 0x7f2ae8150910 2024-04-26T21:35:20.555 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2acc06c8f0 msgr2=0x7f2acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2acc06c8f0 0x7f2acc06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80b3840 tx=0x7f2ae4008040 comp rx=0 tx=0).stop 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8150910 msgr2=0x7f2ae80b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8150910 0x7f2ae80b30a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0002230 tx=0x7f2ae0007b20 comp rx=0 tx=0).stop 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 shutdown_connections 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2acc06c8f0 0x7f2acc06edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8150910 0x7f2ae80b30a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ae81503f0 0x7f2ae80b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 --2- 172.21.15.40:0/2921663467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ad4003f30 0x7f2ae80b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 >> 172.21.15.40:0/2921663467 conn(0x7f2ae80b39b0 msgr2=0x7f2ae80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 shutdown_connections 2024-04-26T21:35:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.556+0000 7f2aefba4640 1 -- 172.21.15.40:0/2921663467 wait complete. 2024-04-26T21:35:20.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.670+0000 7f2bc787b640 1 Processor -- start 2024-04-26T21:35:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.670+0000 7f2bc787b640 1 -- start start 2024-04-26T21:35:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc787b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2bc00b40d0 0x7f2bc01625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc787b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 0x7f2bc00b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc787b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc00d0a10 con 0x7f2bc00d1a00 2024-04-26T21:35:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc787b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2bc00d0cf0 con 0x7f2bc00b40d0 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc787b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc00d0fd0 con 0x7f2bc0162b30 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc5617640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2bc00d1a00 0x7f2bc00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45172/0 (socket says 172.21.15.40:45172) 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc5617640 1 -- 172.21.15.40:0/1392441924 learned_addr learned my addr 172.21.15.40:0/1392441924 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc5e18640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 0x7f2bc00b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2534935466 0 0) 0x7f2bc00d0a10 con 0x7f2bc00d1a00 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bac003520 con 0x7f2bc00d1a00 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.671+0000 7f2bc4e16640 1 --2- 172.21.15.40:0/1392441924 >> v2:172.21.15.184:3300/0 conn(0x7f2bc00b40d0 0x7f2bc01625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1178784384 0 0) 0x7f2bac003520 con 0x7f2bc00d1a00 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 msgr2=0x7f2bc00b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 0x7f2bc00b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 >> v2:172.21.15.184:3300/0 conn(0x7f2bc00b40d0 msgr2=0x7f2bc01625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 --2- 172.21.15.40:0/1392441924 >> v2:172.21.15.184:3300/0 conn(0x7f2bc00b40d0 0x7f2bc01625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc0162f80 con 0x7f2bc00d1a00 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f2bbc001670 con 0x7f2bc00d1a00 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 >> v1:172.21.15.40:6789/0 conn(0x7f2bc00d1a00 legacy=0x7f2bc00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 0x7f2bac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac003f30 0x7f2bac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac004d40 0x7f2bac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00d0cf0 con 0x7f2bac004d40 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00d0fd0 con 0x7f2bac003f30 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2bac00ba10 con 0x7f2bc0162b30 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc5617640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 0x7f2bac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc5e18640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac004d40 0x7f2bac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc4e16640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac003f30 0x7f2bac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc5e18640 1 -- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 msgr2=0x7f2bac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc5e18640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 0x7f2bac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc5e18640 1 -- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac003f30 msgr2=0x7f2bac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc5e18640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac003f30 0x7f2bac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc5e18640 1 -- 172.21.15.40:0/1392441924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac003520 con 0x7f2bac004d40 2024-04-26T21:35:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.672+0000 7f2bc5617640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bc0162b30 0x7f2bac0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.673+0000 7f2bc5e18640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac004d40 0x7f2bac0051c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800e170 tx=0x7f2bac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce082b7dbcba91d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.673+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8034020 con 0x7f2bac004d40 2024-04-26T21:35:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.673+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb80305f0 con 0x7f2bac004d40 2024-04-26T21:35:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.673+0000 7f2bb7fff640 1 -- 172.21.15.40:0/1392441924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8009c10 con 0x7f2bac004d40 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.674+0000 7f2bc787b640 1 -- 172.21.15.40:0/1392441924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac004d40 msgr2=0x7f2bac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.674+0000 7f2bc787b640 1 --2- 172.21.15.40:0/1392441924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bac004d40 0x7f2bac0051c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800e170 tx=0x7f2bac005830 comp rx=0 tx=0).stop 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.674+0000 7f2bc5e18640 1 -- 172.21.15.40:0/1392441924 reap_dead start 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.674+0000 7f2bc787b640 1 -- 172.21.15.40:0/1392441924 shutdown_connections 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.674+0000 7f2bc787b640 1 -- 172.21.15.40:0/1392441924 >> 172.21.15.40:0/1392441924 conn(0x7f2bc00d4fd0 msgr2=0x7f2bc00d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.674+0000 7f2bc787b640 1 -- 172.21.15.40:0/1392441924 shutdown_connections 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.674+0000 7f2bc787b640 1 -- 172.21.15.40:0/1392441924 wait complete. 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.675+0000 7f2bc787b640 1 Processor -- start 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.675+0000 7f2bc787b640 1 -- start start 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc787b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac004d40 0x7f2bc0155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc787b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bc0156c10 0x7f2bc00cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc787b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc00ce3f0 0x7f2bc00cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc787b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00d2670 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc787b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00d24f0 con 0x7f2bc0156c10 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc787b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc00b4b00 con 0x7f2bac004d40 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc00ce3f0 0x7f2bc00cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc00ce3f0 0x7f2bc00cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57568/0 (socket says 172.21.15.40:57568) 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 -- 172.21.15.40:0/2385169998 learned_addr learned my addr 172.21.15.40:0/2385169998 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc4e16640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bc0156c10 0x7f2bc00cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5617640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac004d40 0x7f2bc0155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 -- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac004d40 msgr2=0x7f2bc0155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac004d40 0x7f2bc0155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 -- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bc0156c10 msgr2=0x7f2bc00cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bc0156c10 0x7f2bc00cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 -- 172.21.15.40:0/2385169998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc00ceec0 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.676+0000 7f2bc5e18640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc00ce3f0 0x7f2bc00cca40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800e170 tx=0x7f2bb8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.677+0000 7f2bb5ffb640 1 -- 172.21.15.40:0/2385169998 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8040070 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.677+0000 7f2bb5ffb640 1 -- 172.21.15.40:0/2385169998 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb8003990 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.677+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc00ce900 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.677+0000 7f2bb5ffb640 1 -- 172.21.15.40:0/2385169998 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8034040 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.677+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc00cc240 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.678+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc00694f0 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.678+0000 7f2bb5ffb640 1 -- 172.21.15.40:0/2385169998 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bb8037020 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.679+0000 7f2bb5ffb640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ba406c8f0 0x7f2ba406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.679+0000 7f2bb5ffb640 1 -- 172.21.15.40:0/2385169998 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2bb80ac290 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.679+0000 7f2bc5617640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ba406c8f0 0x7f2ba406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.680+0000 7f2bc5617640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ba406c8f0 0x7f2ba406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc000fb0 tx=0x7f2bbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.683+0000 7f2bb5ffb640 1 -- 172.21.15.40:0/2385169998 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2bb807a2e0 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.871+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2bc00b3c60 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.872+0000 7f2bb5ffb640 1 -- 172.21.15.40:0/2385169998 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f2bb8079be0 con 0x7f2bc00ce3f0 2024-04-26T21:35:20.873 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:20.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ba406c8f0 msgr2=0x7f2ba406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ba406c8f0 0x7f2ba406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc000fb0 tx=0x7f2bbc009040 comp rx=0 tx=0).stop 2024-04-26T21:35:20.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc00ce3f0 msgr2=0x7f2bc00cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc00ce3f0 0x7f2bc00cca40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800e170 tx=0x7f2bb8009750 comp rx=0 tx=0).stop 2024-04-26T21:35:20.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 shutdown_connections 2024-04-26T21:35:20.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ba406c8f0 0x7f2ba406edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc00ce3f0 0x7f2bc00cca40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bc0156c10 0x7f2bc00cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 --2- 172.21.15.40:0/2385169998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac004d40 0x7f2bc0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 >> 172.21.15.40:0/2385169998 conn(0x7f2bc00d4fd0 msgr2=0x7f2bc00cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:20.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 shutdown_connections 2024-04-26T21:35:20.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.874+0000 7f2bc787b640 1 -- 172.21.15.40:0/2385169998 wait complete. 2024-04-26T21:35:20.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:20.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.988+0000 7ffb7c7e7640 1 Processor -- start 2024-04-26T21:35:20.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.988+0000 7ffb7c7e7640 1 -- start start 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.989+0000 7ffb7c7e7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffb740b1fd0 0x7ffb740d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.989+0000 7ffb7c7e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740cdb70 0x7ffb740ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.989+0000 7ffb7c7e7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb7406e580 con 0x7ffb7415f770 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.989+0000 7ffb7c7e7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffb7406f3e0 con 0x7ffb740b1fd0 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.989+0000 7ffb7c7e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffb740cd6e0 con 0x7ffb740cdb70 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.989+0000 7ffb7a583640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffb7415f770 0x7ffb740d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45182/0 (socket says 172.21.15.40:45182) 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.989+0000 7ffb7a583640 1 -- 172.21.15.40:0/1462161631 learned_addr learned my addr 172.21.15.40:0/1462161631 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.990+0000 7ffb79d82640 1 --2- 172.21.15.40:0/1462161631 >> v2:172.21.15.184:3300/0 conn(0x7ffb740b1fd0 0x7ffb740d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.990+0000 7ffb7ad84640 1 --2- 172.21.15.40:0/1462161631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740cdb70 0x7ffb740ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.990+0000 7ffb79581640 1 -- 172.21.15.40:0/1462161631 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364548547 0 0) 0x7ffb7406e580 con 0x7ffb7415f770 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.990+0000 7ffb79581640 1 -- 172.21.15.40:0/1462161631 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb5c003520 con 0x7ffb7415f770 2024-04-26T21:35:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.990+0000 7ffb79d82640 1 -- 172.21.15.40:0/1462161631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740cdb70 msgr2=0x7ffb740ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.990+0000 7ffb79d82640 1 --2- 172.21.15.40:0/1462161631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740cdb70 0x7ffb740ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.990+0000 7ffb79d82640 1 -- 172.21.15.40:0/1462161631 >> v1:172.21.15.40:6789/0 conn(0x7ffb7415f770 legacy=0x7ffb740d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.990+0000 7ffb79d82640 1 -- 172.21.15.40:0/1462161631 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7406e860 con 0x7ffb740b1fd0 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb79d82640 1 --2- 172.21.15.40:0/1462161631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb740b1fd0 0x7ffb740d1470 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffb680097a0 tx=0x7ffb6802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b42e7609dbd802e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb79581640 1 -- 172.21.15.40:0/1462161631 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68037070 con 0x7ffb740b1fd0 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb79581640 1 -- 172.21.15.40:0/1462161631 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb68032440 con 0x7ffb740b1fd0 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb79581640 1 -- 172.21.15.40:0/1462161631 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680327f0 con 0x7ffb740b1fd0 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/1462161631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb740b1fd0 msgr2=0x7ffb740d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/1462161631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb740b1fd0 0x7ffb740d1470 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffb680097a0 tx=0x7ffb6802a180 comp rx=0 tx=0).stop 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/1462161631 shutdown_connections 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/1462161631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740cdb70 0x7ffb740ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/1462161631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb740b1fd0 0x7ffb740d1470 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/1462161631 >> 172.21.15.40:0/1462161631 conn(0x7ffb740d2e60 msgr2=0x7ffb740d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/1462161631 shutdown_connections 2024-04-26T21:35:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.991+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/1462161631 wait complete. 2024-04-26T21:35:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb7c7e7640 1 Processor -- start 2024-04-26T21:35:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb7c7e7640 1 -- start start 2024-04-26T21:35:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb7c7e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740b1fd0 0x7ffb740c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb7c7e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb740cdb70 0x7ffb74155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb7c7e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7415f770 0x7ffb740711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb7c7e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffb740d1b30 con 0x7ffb740cdb70 2024-04-26T21:35:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb7c7e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffb740d19b0 con 0x7ffb7415f770 2024-04-26T21:35:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb7c7e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffb740ce540 con 0x7ffb740b1fd0 2024-04-26T21:35:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.993+0000 7ffb79d82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb740cdb70 0x7ffb74155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb79d82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb740cdb70 0x7ffb74155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57572/0 (socket says 172.21.15.40:57572) 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb79d82640 1 -- 172.21.15.40:0/2203649629 learned_addr learned my addr 172.21.15.40:0/2203649629 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb7ad84640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7415f770 0x7ffb740711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb7a583640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740b1fd0 0x7ffb740c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb79d82640 1 -- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740b1fd0 msgr2=0x7ffb740c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb79d82640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740b1fd0 0x7ffb740c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb79d82640 1 -- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7415f770 msgr2=0x7ffb740711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb79d82640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7415f770 0x7ffb740711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb79d82640 1 -- 172.21.15.40:0/2203649629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb740c9fd0 con 0x7ffb740cdb70 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb79d82640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb740cdb70 0x7ffb74155320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb68000c00 tx=0x7ffb680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb72ffd640 1 -- 172.21.15.40:0/2203649629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68041020 con 0x7ffb740cdb70 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb72ffd640 1 -- 172.21.15.40:0/2203649629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb6803d620 con 0x7ffb740cdb70 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb72ffd640 1 -- 172.21.15.40:0/2203649629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68037040 con 0x7ffb740cdb70 2024-04-26T21:35:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb740c8160 con 0x7ffb740cdb70 2024-04-26T21:35:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.994+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb740c8720 con 0x7ffb740cdb70 2024-04-26T21:35:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.995+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb740694f0 con 0x7ffb740cdb70 2024-04-26T21:35:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.996+0000 7ffb72ffd640 1 -- 172.21.15.40:0/2203649629 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb68044020 con 0x7ffb740cdb70 2024-04-26T21:35:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.996+0000 7ffb72ffd640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb6006c8f0 0x7ffb6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.996+0000 7ffb72ffd640 1 -- 172.21.15.40:0/2203649629 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ffb680ac4f0 con 0x7ffb740cdb70 2024-04-26T21:35:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.996+0000 7ffb7a583640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb6006c8f0 0x7ffb6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:20.997+0000 7ffb7a583640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb6006c8f0 0x7ffb6006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005880 tx=0x7ffb6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.001+0000 7ffb72ffd640 1 -- 172.21.15.40:0/2203649629 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb6804b030 con 0x7ffb740cdb70 2024-04-26T21:35:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.188+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ffb7406fe50 con 0x7ffb740cdb70 2024-04-26T21:35:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.188+0000 7ffb72ffd640 1 -- 172.21.15.40:0/2203649629 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7ffb68079f00 con 0x7ffb740cdb70 2024-04-26T21:35:21.189 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:21.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb6006c8f0 msgr2=0x7ffb6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb6006c8f0 0x7ffb6006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005880 tx=0x7ffb6c009040 comp rx=0 tx=0).stop 2024-04-26T21:35:21.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb740cdb70 msgr2=0x7ffb74155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb740cdb70 0x7ffb74155320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb68000c00 tx=0x7ffb680046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 shutdown_connections 2024-04-26T21:35:21.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb6006c8f0 0x7ffb6006edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7415f770 0x7ffb740711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb740cdb70 0x7ffb74155320 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 --2- 172.21.15.40:0/2203649629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb740b1fd0 0x7ffb740c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 >> 172.21.15.40:0/2203649629 conn(0x7ffb740d2e60 msgr2=0x7ffb740b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:21.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 shutdown_connections 2024-04-26T21:35:21.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.190+0000 7ffb7c7e7640 1 -- 172.21.15.40:0/2203649629 wait complete. 2024-04-26T21:35:21.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:35:21.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.303+0000 7fee05189640 1 Processor -- start 2024-04-26T21:35:21.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.303+0000 7fee05189640 1 -- start start 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fee05189640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fee000b3860 0x7fee0016a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fee05189640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee000d0ba0 con 0x7fee0015f3a0 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fee05189640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee000d2590 con 0x7fee000d06f0 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fee05189640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fee000d0e20 con 0x7fee000b3860 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fedfed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fee0015f3a0 0x7fee000d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45188/0 (socket says 172.21.15.40:45188) 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fedfed9d640 1 -- 172.21.15.40:0/963118345 learned_addr learned my addr 172.21.15.40:0/963118345 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fedff59e640 1 --2- 172.21.15.40:0/963118345 >> v2:172.21.15.184:3300/0 conn(0x7fee000b3860 0x7fee0016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4196926983 0 0) 0x7fedf0001490 con 0x7fee0015f3a0 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fede8003520 con 0x7fee0015f3a0 2024-04-26T21:35:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.304+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1861275646 0 0) 0x7fede8003520 con 0x7fee0015f3a0 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 >> v2:172.21.15.184:3300/0 conn(0x7fee000b3860 msgr2=0x7fee0016a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 --2- 172.21.15.40:0/963118345 >> v2:172.21.15.184:3300/0 conn(0x7fee000b3860 0x7fee0016a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 >> v1:172.21.15.184:6789/0 conn(0x7fee000d06f0 legacy=0x7fee000d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee000b1d70 con 0x7fee0015f3a0 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fedf0001bc0 con 0x7fee0015f3a0 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 >> v1:172.21.15.40:6789/0 conn(0x7fee0015f3a0 legacy=0x7fee000d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fede80039b0 0x7fede8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fede8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede80051c0 0x7fede800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee000d0e20 con 0x7fede8004800 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fede800c160 con 0x7fede80051c0 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fede800c360 con 0x7fede80039b0 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfe59c640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fede8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedff59e640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede80051c0 0x7fede800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfed9d640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fede80039b0 0x7fede8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfe59c640 1 -- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fede80039b0 msgr2=0x7fede8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfe59c640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fede80039b0 0x7fede8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfe59c640 1 -- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede80051c0 msgr2=0x7fede800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfe59c640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede80051c0 0x7fede800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfe59c640 1 -- 172.21.15.40:0/963118345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede8003520 con 0x7fede8004800 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedff59e640 1 -- 172.21.15.40:0/963118345 reap_dead start 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.305+0000 7fedfed9d640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fede80039b0 0x7fede8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fedfe59c640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fede8004c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fedf4001750 tx=0x7fede8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b84b3623c285463 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4001370 con 0x7fede8004800 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf40052d0 con 0x7fede8004800 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fedfdd9b640 1 -- 172.21.15.40:0/963118345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf402abc0 con 0x7fede8004800 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fee05189640 1 -- 172.21.15.40:0/963118345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 msgr2=0x7fede8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fee05189640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fede8004c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fedf4001750 tx=0x7fede8005830 comp rx=0 tx=0).stop 2024-04-26T21:35:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fee05189640 1 -- 172.21.15.40:0/963118345 shutdown_connections 2024-04-26T21:35:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fee05189640 1 --2- 172.21.15.40:0/963118345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fede8004c80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fee05189640 1 -- 172.21.15.40:0/963118345 >> 172.21.15.40:0/963118345 conn(0x7fee000d4f90 msgr2=0x7fee000d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fee05189640 1 -- 172.21.15.40:0/963118345 shutdown_connections 2024-04-26T21:35:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.306+0000 7fee05189640 1 -- 172.21.15.40:0/963118345 wait complete. 2024-04-26T21:35:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.308+0000 7fee05189640 1 Processor -- start 2024-04-26T21:35:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.308+0000 7fee05189640 1 -- start start 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fee05189640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fee000b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fee05189640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee000b85c0 0x7fee000c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fee05189640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee000c9fb0 0x7fee0015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fee05189640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee0015f780 con 0x7fede8004800 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fee05189640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee000b2330 con 0x7fee000c9fb0 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fee05189640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee0015f900 con 0x7fee000b85c0 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fee000b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fee000b8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57592/0 (socket says 172.21.15.40:57592) 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 -- 172.21.15.40:0/3880927433 learned_addr learned my addr 172.21.15.40:0/3880927433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedff59e640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee000c9fb0 0x7fee0015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfe59c640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee000b85c0 0x7fee000c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 -- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee000b85c0 msgr2=0x7fee000c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee000b85c0 0x7fee000c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 -- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee000c9fb0 msgr2=0x7fee0015aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee000c9fb0 0x7fee0015aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 -- 172.21.15.40:0/3880927433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee000ca4c0 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fedfed9d640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fee000b8080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fedf0001330 tx=0x7fedf000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.309+0000 7fede77fe640 1 -- 172.21.15.40:0/3880927433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0017020 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.310+0000 7fede77fe640 1 -- 172.21.15.40:0/3880927433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf000ec40 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.310+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee000ca7a0 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.310+0000 7fede77fe640 1 -- 172.21.15.40:0/3880927433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0013420 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.310+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee000cad60 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.311+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee000694f0 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.311+0000 7fede77fe640 1 -- 172.21.15.40:0/3880927433 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedf001a020 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.312+0000 7fede77fe640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fedec06c920 0x7fedec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.312+0000 7fede77fe640 1 -- 172.21.15.40:0/3880927433 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fedf008c0f0 con 0x7fede8004800 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.312+0000 7fedfe59c640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fedec06c920 0x7fedec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.312+0000 7fedfe59c640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fedec06c920 0x7fedec06ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedf4032000 tx=0x7fedf402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.316+0000 7fede77fe640 1 -- 172.21.15.40:0/3880927433 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fedf0021030 con 0x7fede8004800 2024-04-26T21:35:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.512+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fee000b6860 con 0x7fede8004800 2024-04-26T21:35:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.513+0000 7fede77fe640 1 -- 172.21.15.40:0/3880927433 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fee000d0ba0 con 0x7fede8004800 2024-04-26T21:35:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.514+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fedec06c920 msgr2=0x7fedec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.514+0000 7fee05189640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fedec06c920 0x7fedec06ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedf4032000 tx=0x7fedf402e000 comp rx=0 tx=0).stop 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 msgr2=0x7fee000b8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fee000b8080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fedf0001330 tx=0x7fedf000e530 comp rx=0 tx=0).stop 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 shutdown_connections 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fedec06c920 0x7fedec06ede0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee000c9fb0 0x7fee0015aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee000b85c0 0x7fee000c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 --2- 172.21.15.40:0/3880927433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8004800 0x7fee000b8080 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 >> 172.21.15.40:0/3880927433 conn(0x7fee000d4f90 msgr2=0x7fee000b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 shutdown_connections 2024-04-26T21:35:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.515+0000 7fee05189640 1 -- 172.21.15.40:0/3880927433 wait complete. 2024-04-26T21:35:21.526 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:35:21.527 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:35:21.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.681+0000 7f2c1b646640 1 Processor -- start 2024-04-26T21:35:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.681+0000 7f2c1b646640 1 -- start start 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c1b646640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 0x7f2c140d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c1b646640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c140d2300 0x7f2c140b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c1b646640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c140d0e20 con 0x7f2c140d1770 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c1b646640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2c140b1d70 con 0x7f2c140d2300 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c1b646640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c140b2050 con 0x7f2c1415f3a0 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c19be3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c140d2300 0x7f2c140b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c19be3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c140d2300 0x7f2c140b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57604/0 (socket says 172.21.15.40:57604) 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c19be3640 1 -- 172.21.15.40:0/2292325516 learned_addr learned my addr 172.21.15.40:0/2292325516 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c193e2640 1 --2- 172.21.15.40:0/2292325516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 0x7f2c140d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c19be3640 1 -- 172.21.15.40:0/2292325516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 msgr2=0x7f2c140d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c19be3640 1 --2- 172.21.15.40:0/2292325516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 0x7f2c140d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c19be3640 1 -- 172.21.15.40:0/2292325516 >> v1:172.21.15.184:6789/0 conn(0x7f2c140d1770 legacy=0x7f2c140d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.682+0000 7f2c19be3640 1 -- 172.21.15.40:0/2292325516 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c140b2330 con 0x7f2c140d2300 2024-04-26T21:35:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.683+0000 7f2c19be3640 1 --2- 172.21.15.40:0/2292325516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b2850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009e50 tx=0x7f2c0c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73d953a7b2cdb8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.683+0000 7f2c0bfff640 1 -- 172.21.15.40:0/2292325516 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c036070 con 0x7f2c140d2300 2024-04-26T21:35:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.683+0000 7f2c0bfff640 1 -- 172.21.15.40:0/2292325516 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0c031600 con 0x7f2c140d2300 2024-04-26T21:35:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.683+0000 7f2c0bfff640 1 -- 172.21.15.40:0/2292325516 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c0319d0 con 0x7f2c140d2300 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.683+0000 7f2c1b646640 1 -- 172.21.15.40:0/2292325516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 msgr2=0x7f2c140b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.683+0000 7f2c1b646640 1 --2- 172.21.15.40:0/2292325516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b2850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009e50 tx=0x7f2c0c02a710 comp rx=0 tx=0).stop 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.684+0000 7f2c1b646640 1 -- 172.21.15.40:0/2292325516 shutdown_connections 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.684+0000 7f2c1b646640 1 --2- 172.21.15.40:0/2292325516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b2850 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.684+0000 7f2c1b646640 1 --2- 172.21.15.40:0/2292325516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 0x7f2c140d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.684+0000 7f2c1b646640 1 -- 172.21.15.40:0/2292325516 >> 172.21.15.40:0/2292325516 conn(0x7f2c140d4f90 msgr2=0x7f2c140d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.684+0000 7f2c1b646640 1 -- 172.21.15.40:0/2292325516 shutdown_connections 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.684+0000 7f2c1b646640 1 -- 172.21.15.40:0/2292325516 wait complete. 2024-04-26T21:35:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.685+0000 7f2c1b646640 1 Processor -- start 2024-04-26T21:35:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.685+0000 7f2c1b646640 1 -- start start 2024-04-26T21:35:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.685+0000 7f2c1b646640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c140d1770 0x7f2c14159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.685+0000 7f2c1b646640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c1b646640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 0x7f2c140d3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c1b646640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2c140b2f10 con 0x7f2c140d2300 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c1b646640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c140b2d90 con 0x7f2c140d1770 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c1b646640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c140b3090 con 0x7f2c1415f3a0 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57610/0 (socket says 172.21.15.40:57610) 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 -- 172.21.15.40:0/1678080771 learned_addr learned my addr 172.21.15.40:0/1678080771 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c193e2640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c140d1770 0x7f2c14159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c19be3640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 0x7f2c140d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 -- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 msgr2=0x7f2c140d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 0x7f2c140d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 -- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c140d1770 msgr2=0x7f2c14159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c140d1770 0x7f2c14159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 -- 172.21.15.40:0/1678080771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1415ceb0 con 0x7f2c140d2300 2024-04-26T21:35:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c18be1640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b9700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400c400 tx=0x7f2c0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c09ffb640 1 -- 172.21.15.40:0/1678080771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040052d0 con 0x7f2c140d2300 2024-04-26T21:35:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c09ffb640 1 -- 172.21.15.40:0/1678080771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c04005490 con 0x7f2c140d2300 2024-04-26T21:35:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c140c8b60 con 0x7f2c140d2300 2024-04-26T21:35:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.686+0000 7f2c09ffb640 1 -- 172.21.15.40:0/1678080771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04007260 con 0x7f2c140d2300 2024-04-26T21:35:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.687+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c140b8a80 con 0x7f2c140d2300 2024-04-26T21:35:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.687+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c140694f0 con 0x7f2c140d2300 2024-04-26T21:35:21.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.688+0000 7f2c09ffb640 1 -- 172.21.15.40:0/1678080771 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c04007400 con 0x7f2c140d2300 2024-04-26T21:35:21.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.688+0000 7f2c09ffb640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf806c8f0 0x7f2bf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:21.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.688+0000 7f2c09ffb640 1 -- 172.21.15.40:0/1678080771 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2c0408b370 con 0x7f2c140d2300 2024-04-26T21:35:21.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.689+0000 7f2c193e2640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf806c8f0 0x7f2bf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:21.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.689+0000 7f2c193e2640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf806c8f0 0x7f2bf806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c100079a0 tx=0x7f2c10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:21.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.693+0000 7f2c09ffb640 1 -- 172.21.15.40:0/1678080771 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c1415ceb0 con 0x7f2c140d2300 2024-04-26T21:35:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.880+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c140b6860 con 0x7f2c140d2300 2024-04-26T21:35:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.881+0000 7f2c09ffb640 1 -- 172.21.15.40:0/1678080771 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f2c04058e10 con 0x7f2c140d2300 2024-04-26T21:35:21.882 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:21.883 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.883+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf806c8f0 msgr2=0x7f2bf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.883+0000 7f2c1b646640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf806c8f0 0x7f2bf806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c100079a0 tx=0x7f2c10008040 comp rx=0 tx=0).stop 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.883+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 msgr2=0x7f2c140b9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.883+0000 7f2c1b646640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b9700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400c400 tx=0x7f2c0400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.884+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 shutdown_connections 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.884+0000 7f2c1b646640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf806c8f0 0x7f2bf806edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.884+0000 7f2c1b646640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1415f3a0 0x7f2c140d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.884+0000 7f2c1b646640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c140d2300 0x7f2c140b9700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.884+0000 7f2c1b646640 1 --2- 172.21.15.40:0/1678080771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c140d1770 0x7f2c14159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.884+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 >> 172.21.15.40:0/1678080771 conn(0x7f2c140d4f90 msgr2=0x7f2c1406df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.884+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 shutdown_connections 2024-04-26T21:35:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.884+0000 7f2c1b646640 1 -- 172.21.15.40:0/1678080771 wait complete. 2024-04-26T21:35:21.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:21.896 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:22.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.999+0000 7fddb3659640 1 Processor -- start 2024-04-26T21:35:22.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:21.999+0000 7fddb3659640 1 -- start start 2024-04-26T21:35:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.000+0000 7fddb3659640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fddac162cd0 0x7fddac0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.000+0000 7fddb3659640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddac0d0890 con 0x7fddac1628f0 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.000+0000 7fddb3659640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddac0d0b70 con 0x7fddac0d2110 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.000+0000 7fddb3659640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fddac0d1130 con 0x7fddac162cd0 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.000+0000 7fddb13f5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fddac1628f0 0x7fddac0d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45196/0 (socket says 172.21.15.40:45196) 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.000+0000 7fddb13f5640 1 -- 172.21.15.40:0/386164543 learned_addr learned my addr 172.21.15.40:0/386164543 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2567586147 0 0) 0x7fddac0d0890 con 0x7fddac1628f0 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd98003520 con 0x7fddac1628f0 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/386164543 >> v2:172.21.15.184:3300/0 conn(0x7fddac162cd0 0x7fddac0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3157665198 0 0) 0x7fdd98003520 con 0x7fddac1628f0 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 >> v2:172.21.15.184:3300/0 conn(0x7fddac162cd0 msgr2=0x7fddac0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 --2- 172.21.15.40:0/386164543 >> v2:172.21.15.184:3300/0 conn(0x7fddac162cd0 0x7fddac0b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 >> v1:172.21.15.184:6789/0 conn(0x7fddac0d2110 legacy=0x7fddac0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac0d0e50 con 0x7fddac1628f0 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fdda8001670 con 0x7fddac1628f0 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/386164543 >> v2:172.21.15.184:3300/0 conn(0x7fddac162cd0 0x7fddac0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 >> v1:172.21.15.40:6789/0 conn(0x7fddac1628f0 legacy=0x7fddac0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd980039b0 0x7fdd98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd98004800 0x7fdd98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd980052e0 0x7fdd9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fddac0d1130 con 0x7fdd98004800 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9800c1b0 con 0x7fdd980052e0 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9800c3b0 con 0x7fdd980039b0 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd98004800 0x7fdd98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb13f5640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd980039b0 0x7fdd98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb1bf6640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd980052e0 0x7fdd9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb0bf4640 1 -- 172.21.15.40:0/386164543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd980039b0 msgr2=0x7fdd98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd980039b0 0x7fdd98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb0bf4640 1 -- 172.21.15.40:0/386164543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd980052e0 msgr2=0x7fdd9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd980052e0 0x7fdd9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb1bf6640 1 -- 172.21.15.40:0/386164543 reap_dead start 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.001+0000 7fddb0bf4640 1 -- 172.21.15.40:0/386164543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd98003520 con 0x7fdd98004800 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.002+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd98004800 0x7fdd98004c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c006fd0 tx=0x7fdd98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a4c50fde7930f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.002+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00f040 con 0x7fdd98004800 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.002+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c02ac90 con 0x7fdd98004800 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.002+0000 7fdda3fff640 1 -- 172.21.15.40:0/386164543 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c004250 con 0x7fdd98004800 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.003+0000 7fddb3659640 1 -- 172.21.15.40:0/386164543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd98004800 msgr2=0x7fdd98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.003+0000 7fddb3659640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd98004800 0x7fdd98004c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c006fd0 tx=0x7fdd98005830 comp rx=0 tx=0).stop 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.003+0000 7fddb3659640 1 -- 172.21.15.40:0/386164543 shutdown_connections 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.003+0000 7fddb3659640 1 --2- 172.21.15.40:0/386164543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd98004800 0x7fdd98004c80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.003+0000 7fddb3659640 1 -- 172.21.15.40:0/386164543 >> 172.21.15.40:0/386164543 conn(0x7fddac0d4ff0 msgr2=0x7fddac0d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:22.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.003+0000 7fddb3659640 1 -- 172.21.15.40:0/386164543 shutdown_connections 2024-04-26T21:35:22.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.003+0000 7fddb3659640 1 -- 172.21.15.40:0/386164543 wait complete. 2024-04-26T21:35:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.004+0000 7fddb3659640 1 Processor -- start 2024-04-26T21:35:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.004+0000 7fddb3659640 1 -- start start 2024-04-26T21:35:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb3659640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd98004800 0x7fddac0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb3659640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddac15c790 0x7fddac0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb3659640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddac15d3e0 0x7fddac15d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb3659640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fddac0d16d0 con 0x7fddac15c790 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb3659640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddac0d1550 con 0x7fdd98004800 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb3659640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fddac0b3f50 con 0x7fddac15d3e0 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb0bf4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddac15c790 0x7fddac0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb0bf4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddac15c790 0x7fddac0ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57630/0 (socket says 172.21.15.40:57630) 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb0bf4640 1 -- 172.21.15.40:0/2433882517 learned_addr learned my addr 172.21.15.40:0/2433882517 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb13f5640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd98004800 0x7fddac0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb1bf6640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddac15d3e0 0x7fddac15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb0bf4640 1 -- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddac15d3e0 msgr2=0x7fddac15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddac15d3e0 0x7fddac15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb0bf4640 1 -- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd98004800 msgr2=0x7fddac0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd98004800 0x7fddac0c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb0bf4640 1 -- 172.21.15.40:0/2433882517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac0c8710 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.005+0000 7fddb13f5640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd98004800 0x7fddac0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.006+0000 7fddb0bf4640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddac15c790 0x7fddac0ce760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009990 tx=0x7fdd9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.006+0000 7fdda1ffb640 1 -- 172.21.15.40:0/2433882517 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c041070 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.006+0000 7fdda1ffb640 1 -- 172.21.15.40:0/2433882517 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c02aa90 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.006+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddac0c89d0 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.006+0000 7fdda1ffb640 1 -- 172.21.15.40:0/2433882517 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c03d430 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.006+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddac0c8f90 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.007+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddac0694f0 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.007+0000 7fdda1ffb640 1 -- 172.21.15.40:0/2433882517 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd9c044070 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.008+0000 7fdda1ffb640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.008+0000 7fdda1ffb640 1 -- 172.21.15.40:0/2433882517 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdd9c0ac6c0 con 0x7fddac15c790 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.008+0000 7fddb13f5640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.009+0000 7fddb13f5640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd9006c8f0 0x7fdd9006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdda80011e0 tx=0x7fdda8000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.012+0000 7fdda1ffb640 1 -- 172.21.15.40:0/2433882517 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd9c07a0c0 con 0x7fddac15c790 2024-04-26T21:35:22.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.201+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fddac0cf490 con 0x7fddac15c790 2024-04-26T21:35:22.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.202+0000 7fdda1ffb640 1 -- 172.21.15.40:0/2433882517 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fdd9c07e3c0 con 0x7fddac15c790 2024-04-26T21:35:22.203 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:22.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd9006c8f0 msgr2=0x7fdd9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd9006c8f0 0x7fdd9006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdda80011e0 tx=0x7fdda8000ed0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddac15c790 msgr2=0x7fddac0ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddac15c790 0x7fddac0ce760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009990 tx=0x7fdd9c004730 comp rx=0 tx=0).stop 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 shutdown_connections 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd9006c8f0 0x7fdd9006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddac15d3e0 0x7fddac15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddac15c790 0x7fddac0ce760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 --2- 172.21.15.40:0/2433882517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd98004800 0x7fddac0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 >> 172.21.15.40:0/2433882517 conn(0x7fddac0d4ff0 msgr2=0x7fddac0b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 shutdown_connections 2024-04-26T21:35:22.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.204+0000 7fddb3659640 1 -- 172.21.15.40:0/2433882517 wait complete. 2024-04-26T21:35:22.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:22.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.319+0000 7fc5d2712640 1 Processor -- start 2024-04-26T21:35:22.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.319+0000 7fc5d2712640 1 -- start start 2024-04-26T21:35:22.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d2712640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d1a00 0x7fc5cc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d2712640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc5cc0b40d0 0x7fc5cc0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d2712640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5cc0d0a10 con 0x7fc5cc0d2320 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d2712640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc5cc0d0cf0 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d2712640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc0d0fd0 con 0x7fc5cc0d1a00 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d0caf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc5cc0b40d0 0x7fc5cc0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5cb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc5cc0d2320 0x7fc5cc1625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45210/0 (socket says 172.21.15.40:45210) 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5cb7fe640 1 -- 172.21.15.40:0/2715197943 learned_addr learned my addr 172.21.15.40:0/2715197943 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5caffd640 1 -- 172.21.15.40:0/2715197943 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3005929159 0 0) 0x7fc5cc0d0a10 con 0x7fc5cc0d2320 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5cbfff640 1 --2- 172.21.15.40:0/2715197943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d1a00 0x7fc5cc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5caffd640 1 -- 172.21.15.40:0/2715197943 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5b8003520 con 0x7fc5cc0d2320 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d0caf640 1 -- 172.21.15.40:0/2715197943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d1a00 msgr2=0x7fc5cc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d0caf640 1 --2- 172.21.15.40:0/2715197943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d1a00 0x7fc5cc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d0caf640 1 -- 172.21.15.40:0/2715197943 >> v1:172.21.15.40:6789/0 conn(0x7fc5cc0d2320 legacy=0x7fc5cc1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5d0caf640 1 -- 172.21.15.40:0/2715197943 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc0d40a0 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.320+0000 7fc5cbfff640 1 --2- 172.21.15.40:0/2715197943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d1a00 0x7fc5cc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.321+0000 7fc5d0caf640 1 --2- 172.21.15.40:0/2715197943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0b4b00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009d90 tx=0x7fc5c402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df94f9d4dcebac66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.321+0000 7fc5caffd640 1 -- 172.21.15.40:0/2715197943 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4031600 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.321+0000 7fc5caffd640 1 -- 172.21.15.40:0/2715197943 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c40317a0 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.321+0000 7fc5caffd640 1 -- 172.21.15.40:0/2715197943 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4031b70 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.322+0000 7fc5d2712640 1 -- 172.21.15.40:0/2715197943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 msgr2=0x7fc5cc0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.322+0000 7fc5d2712640 1 --2- 172.21.15.40:0/2715197943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0b4b00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009d90 tx=0x7fc5c402a5d0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.322+0000 7fc5d2712640 1 -- 172.21.15.40:0/2715197943 shutdown_connections 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.322+0000 7fc5d2712640 1 --2- 172.21.15.40:0/2715197943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0b4b00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.322+0000 7fc5d2712640 1 --2- 172.21.15.40:0/2715197943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d1a00 0x7fc5cc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.322+0000 7fc5d2712640 1 -- 172.21.15.40:0/2715197943 >> 172.21.15.40:0/2715197943 conn(0x7fc5cc0d4fd0 msgr2=0x7fc5cc0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:22.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.322+0000 7fc5d2712640 1 -- 172.21.15.40:0/2715197943 shutdown_connections 2024-04-26T21:35:22.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.322+0000 7fc5d2712640 1 -- 172.21.15.40:0/2715197943 wait complete. 2024-04-26T21:35:22.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.323+0000 7fc5d2712640 1 Processor -- start 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.323+0000 7fc5d2712640 1 -- start start 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5d2712640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5d2712640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5cc0d1a00 0x7fc5cc0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5d2712640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d2320 0x7fc5cc0b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5d2712640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc0b5040 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5d2712640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc162f50 con 0x7fc5cc0d1a00 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5d2712640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc0b51c0 con 0x7fc5cc0d2320 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57662/0 (socket says 172.21.15.40:57662) 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5cbfff640 1 -- 172.21.15.40:0/3847522201 learned_addr learned my addr 172.21.15.40:0/3847522201 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5cb7fe640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5cc0d1a00 0x7fc5cc0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5d0caf640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d2320 0x7fc5cc0b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5cbfff640 1 -- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d2320 msgr2=0x7fc5cc0b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5cbfff640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d2320 0x7fc5cc0b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5cbfff640 1 -- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5cc0d1a00 msgr2=0x7fc5cc0cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.324+0000 7fc5cbfff640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5cc0d1a00 0x7fc5cc0cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.325+0000 7fc5cbfff640 1 -- 172.21.15.40:0/3847522201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc0b8580 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.325+0000 7fc5cb7fe640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5cc0d1a00 0x7fc5cc0cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.325+0000 7fc5cbfff640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0ce6a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00c820 tx=0x7fc5bc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.325+0000 7fc5c8ff9640 1 -- 172.21.15.40:0/3847522201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc015070 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.325+0000 7fc5c8ff9640 1 -- 172.21.15.40:0/3847522201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc007a30 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.325+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc0b8820 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.325+0000 7fc5c8ff9640 1 -- 172.21.15.40:0/3847522201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc007e00 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.325+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc0cc240 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.326+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc0694f0 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.326+0000 7fc5c8ff9640 1 -- 172.21.15.40:0/3847522201 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5bc018070 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.327+0000 7fc5c8ff9640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5ac06c8f0 0x7fc5ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.327+0000 7fc5c8ff9640 1 -- 172.21.15.40:0/3847522201 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc5bc08be60 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.327+0000 7fc5cb7fe640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5ac06c8f0 0x7fc5ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.327+0000 7fc5cb7fe640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5ac06c8f0 0x7fc5ac06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0b6840 tx=0x7fc5b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.331+0000 7fc5c8ff9640 1 -- 172.21.15.40:0/3847522201 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5bc059f50 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.515+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc5cc0b8d40 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.516+0000 7fc5c8ff9640 1 -- 172.21.15.40:0/3847522201 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fc5bc059850 con 0x7fc5cc0b40d0 2024-04-26T21:35:22.517 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.517+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5ac06c8f0 msgr2=0x7fc5ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.517+0000 7fc5d2712640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5ac06c8f0 0x7fc5ac06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0b6840 tx=0x7fc5b4009040 comp rx=0 tx=0).stop 2024-04-26T21:35:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.517+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 msgr2=0x7fc5cc0ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.517+0000 7fc5d2712640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0ce6a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00c820 tx=0x7fc5bc00cc20 comp rx=0 tx=0).stop 2024-04-26T21:35:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.518+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 shutdown_connections 2024-04-26T21:35:22.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.518+0000 7fc5d2712640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5ac06c8f0 0x7fc5ac06edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.518+0000 7fc5d2712640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5cc0d2320 0x7fc5cc0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.518+0000 7fc5d2712640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5cc0d1a00 0x7fc5cc0cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.518+0000 7fc5d2712640 1 --2- 172.21.15.40:0/3847522201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5cc0b40d0 0x7fc5cc0ce6a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.518+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 >> 172.21.15.40:0/3847522201 conn(0x7fc5cc0d4fd0 msgr2=0x7fc5cc06d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:22.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.518+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 shutdown_connections 2024-04-26T21:35:22.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.518+0000 7fc5d2712640 1 -- 172.21.15.40:0/3847522201 wait complete. 2024-04-26T21:35:22.530 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.632+0000 7f1871f11640 1 Processor -- start 2024-04-26T21:35:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.633+0000 7f1871f11640 1 -- start start 2024-04-26T21:35:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.633+0000 7f1871f11640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f186c0b1fd0 0x7f186c0d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.633+0000 7f1871f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0cdb70 0x7f186c0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.633+0000 7f1871f11640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f186c06e580 con 0x7f186c15f770 2024-04-26T21:35:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.633+0000 7f1871f11640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f186c06f3e0 con 0x7f186c0b1fd0 2024-04-26T21:35:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.633+0000 7f1871f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f186c0cd6e0 con 0x7f186c0cdb70 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.633+0000 7f186affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f186c0b1fd0 0x7f186c0d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.633+0000 7f186b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f186c15f770 0x7f186c0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45216/0 (socket says 172.21.15.40:45216) 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.634+0000 7f186b7fe640 1 -- 172.21.15.40:0/141592355 learned_addr learned my addr 172.21.15.40:0/141592355 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.634+0000 7f186bfff640 1 --2- 172.21.15.40:0/141592355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0cdb70 0x7f186c0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.634+0000 7f186affd640 1 -- 172.21.15.40:0/141592355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0cdb70 msgr2=0x7f186c0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.634+0000 7f186affd640 1 --2- 172.21.15.40:0/141592355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0cdb70 0x7f186c0ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.634+0000 7f186affd640 1 -- 172.21.15.40:0/141592355 >> v1:172.21.15.40:6789/0 conn(0x7f186c15f770 legacy=0x7f186c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.634+0000 7f186affd640 1 -- 172.21.15.40:0/141592355 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c06e860 con 0x7f186c0b1fd0 2024-04-26T21:35:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.634+0000 7f186a7fc640 1 -- 172.21.15.40:0/141592355 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1195810448 0 0) 0x7f186c06e580 con 0x7f186c15f770 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f186affd640 1 --2- 172.21.15.40:0/141592355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c0b1fd0 0x7f186c0d1470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f18600097a0 tx=0x7f186002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf7f10e56097b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f186a7fc640 1 -- 172.21.15.40:0/141592355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860037070 con 0x7f186c0b1fd0 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f186a7fc640 1 -- 172.21.15.40:0/141592355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1860032440 con 0x7f186c0b1fd0 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f186a7fc640 1 -- 172.21.15.40:0/141592355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860032810 con 0x7f186c0b1fd0 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f1871f11640 1 -- 172.21.15.40:0/141592355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c0b1fd0 msgr2=0x7f186c0d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f1871f11640 1 --2- 172.21.15.40:0/141592355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c0b1fd0 0x7f186c0d1470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f18600097a0 tx=0x7f186002a180 comp rx=0 tx=0).stop 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f1871f11640 1 -- 172.21.15.40:0/141592355 shutdown_connections 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f1871f11640 1 --2- 172.21.15.40:0/141592355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0cdb70 0x7f186c0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f1871f11640 1 --2- 172.21.15.40:0/141592355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c0b1fd0 0x7f186c0d1470 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f1871f11640 1 -- 172.21.15.40:0/141592355 >> 172.21.15.40:0/141592355 conn(0x7f186c0d2e60 msgr2=0x7f186c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:22.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f1871f11640 1 -- 172.21.15.40:0/141592355 shutdown_connections 2024-04-26T21:35:22.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.635+0000 7f1871f11640 1 -- 172.21.15.40:0/141592355 wait complete. 2024-04-26T21:35:22.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.636+0000 7f1871f11640 1 Processor -- start 2024-04-26T21:35:22.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f1871f11640 1 -- start start 2024-04-26T21:35:22.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f1871f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0b1fd0 0x7f186c1f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f1871f11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186c0cdb70 0x7f186c155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f1871f11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c15f770 0x7f186c1f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f1871f11640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0d1b30 con 0x7f186c15f770 2024-04-26T21:35:22.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f1871f11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0d19b0 con 0x7f186c0cdb70 2024-04-26T21:35:22.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f1871f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f186c0ce540 con 0x7f186c0b1fd0 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f186bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c15f770 0x7f186c1f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f186bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c15f770 0x7f186c1f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57686/0 (socket says 172.21.15.40:57686) 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.637+0000 7f186bfff640 1 -- 172.21.15.40:0/3765210052 learned_addr learned my addr 172.21.15.40:0/3765210052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f186b7fe640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0b1fd0 0x7f186c1f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f186affd640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186c0cdb70 0x7f186c155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f186bfff640 1 -- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0b1fd0 msgr2=0x7f186c1f93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f186bfff640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0b1fd0 0x7f186c1f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f186bfff640 1 -- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186c0cdb70 msgr2=0x7f186c155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f186bfff640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186c0cdb70 0x7f186c155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f186bfff640 1 -- 172.21.15.40:0/3765210052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c1f5e30 con 0x7f186c15f770 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f186bfff640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c15f770 0x7f186c1f5370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f186400efc0 tx=0x7f186400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f1853fff640 1 -- 172.21.15.40:0/3765210052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864018020 con 0x7f186c15f770 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f1853fff640 1 -- 172.21.15.40:0/3765210052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864014480 con 0x7f186c15f770 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c1f6110 con 0x7f186c15f770 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f1853fff640 1 -- 172.21.15.40:0/3765210052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864014870 con 0x7f186c15f770 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.638+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c1f8e90 con 0x7f186c15f770 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.639+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c0694f0 con 0x7f186c15f770 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.640+0000 7f1853fff640 1 -- 172.21.15.40:0/3765210052 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1864005d10 con 0x7f186c15f770 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.640+0000 7f1853fff640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f185806c8f0 0x7f185806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.640+0000 7f1853fff640 1 -- 172.21.15.40:0/3765210052 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f186408ca20 con 0x7f186c15f770 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.640+0000 7f186b7fe640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f185806c8f0 0x7f185806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.641+0000 7f186b7fe640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f185806c8f0 0x7f185806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f185c000f60 tx=0x7f185c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.644+0000 7f1853fff640 1 -- 172.21.15.40:0/3765210052 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1864022070 con 0x7f186c15f770 2024-04-26T21:35:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.829+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f186c0b8cc0 con 0x7f186c15f770 2024-04-26T21:35:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.830+0000 7f1853fff640 1 -- 172.21.15.40:0/3765210052 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f1864010040 con 0x7f186c15f770 2024-04-26T21:35:22.831 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f185806c8f0 msgr2=0x7f185806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f185806c8f0 0x7f185806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f185c000f60 tx=0x7f185c009040 comp rx=0 tx=0).stop 2024-04-26T21:35:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c15f770 msgr2=0x7f186c1f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c15f770 0x7f186c1f5370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f186400efc0 tx=0x7f186400c490 comp rx=0 tx=0).stop 2024-04-26T21:35:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 shutdown_connections 2024-04-26T21:35:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f185806c8f0 0x7f185806edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f186c15f770 0x7f186c1f5370 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186c0cdb70 0x7f186c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 --2- 172.21.15.40:0/3765210052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f186c0b1fd0 0x7f186c1f93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 >> 172.21.15.40:0/3765210052 conn(0x7f186c0d2e60 msgr2=0x7f186c15ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 shutdown_connections 2024-04-26T21:35:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.832+0000 7f1871f11640 1 -- 172.21.15.40:0/3765210052 wait complete. 2024-04-26T21:35:22.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:22.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.947+0000 7f9332e83640 1 Processor -- start 2024-04-26T21:35:22.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.947+0000 7f9332e83640 1 -- start start 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.947+0000 7f9332e83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.947+0000 7f9332e83640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f932c0d0d60 0x7f932c0d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.947+0000 7f9332e83640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f932c0d1720 0x7f932c0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.947+0000 7f9332e83640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f932c06f3e0 con 0x7f932c0d0d60 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.947+0000 7f9332e83640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f932c0cd6e0 con 0x7f932c0d1720 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.947+0000 7f9332e83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0d1bb0 con 0x7f932c15f770 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9323fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f932c0d0d60 0x7f932c0d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9323fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f932c0d0d60 0x7f932c0d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57700/0 (socket says 172.21.15.40:57700) 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9323fff640 1 -- 172.21.15.40:0/3640937425 learned_addr learned my addr 172.21.15.40:0/3640937425 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9331420640 1 --2- 172.21.15.40:0/3640937425 >> v2:172.21.15.184:3300/0 conn(0x7f932c0d1720 0x7f932c0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9330c1f640 1 --2- 172.21.15.40:0/3640937425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9323fff640 1 -- 172.21.15.40:0/3640937425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 msgr2=0x7f932c0b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9323fff640 1 --2- 172.21.15.40:0/3640937425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9323fff640 1 -- 172.21.15.40:0/3640937425 >> v2:172.21.15.184:3300/0 conn(0x7f932c0d1720 msgr2=0x7f932c0cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9323fff640 1 --2- 172.21.15.40:0/3640937425 >> v2:172.21.15.184:3300/0 conn(0x7f932c0d1720 0x7f932c0cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9323fff640 1 -- 172.21.15.40:0/3640937425 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c06e580 con 0x7f932c0d0d60 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.948+0000 7f9330c1f640 1 --2- 172.21.15.40:0/3640937425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9323fff640 1 --2- 172.21.15.40:0/3640937425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0d11e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098a0 tx=0x7f931c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7231ddaa06d9ce85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f93237fe640 1 -- 172.21.15.40:0/3640937425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c037070 con 0x7f932c0d0d60 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f93237fe640 1 -- 172.21.15.40:0/3640937425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c032440 con 0x7f932c0d0d60 2024-04-26T21:35:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f93237fe640 1 -- 172.21.15.40:0/3640937425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c032810 con 0x7f932c0d0d60 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 -- 172.21.15.40:0/3640937425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 msgr2=0x7f932c0d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 --2- 172.21.15.40:0/3640937425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0d11e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098a0 tx=0x7f931c02a250 comp rx=0 tx=0).stop 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 -- 172.21.15.40:0/3640937425 shutdown_connections 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 --2- 172.21.15.40:0/3640937425 >> v2:172.21.15.184:3300/0 conn(0x7f932c0d1720 0x7f932c0cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 --2- 172.21.15.40:0/3640937425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0d11e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 --2- 172.21.15.40:0/3640937425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 -- 172.21.15.40:0/3640937425 >> 172.21.15.40:0/3640937425 conn(0x7f932c0d2e60 msgr2=0x7f932c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 -- 172.21.15.40:0/3640937425 shutdown_connections 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.949+0000 7f9332e83640 1 -- 172.21.15.40:0/3640937425 wait complete. 2024-04-26T21:35:22.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.950+0000 7f9332e83640 1 Processor -- start 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.950+0000 7f9332e83640 1 -- start start 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9332e83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9332e83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0d1720 0x7f932c0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9332e83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9332e83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0cfc20 con 0x7f932c0d0d60 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9332e83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f932c06eb40 con 0x7f932c0d1720 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9332e83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0cfda0 con 0x7f932c15f770 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9330c1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9330c1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57704/0 (socket says 172.21.15.40:57704) 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.951+0000 7f9330c1f640 1 -- 172.21.15.40:0/2285431661 learned_addr learned my addr 172.21.15.40:0/2285431661 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9331420640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9323fff640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0d1720 0x7f932c0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9330c1f640 1 -- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 msgr2=0x7f932c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9330c1f640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9330c1f640 1 -- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0d1720 msgr2=0x7f932c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9330c1f640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0d1720 0x7f932c0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9330c1f640 1 -- 172.21.15.40:0/2285431661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c1f8580 con 0x7f932c0d0d60 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9323fff640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0d1720 0x7f932c0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9330c1f640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0a05d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f932800c7f0 tx=0x7f932800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f93217fa640 1 -- 172.21.15.40:0/2285431661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328015070 con 0x7f932c0d0d60 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f93217fa640 1 -- 172.21.15.40:0/2285431661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9328007820 con 0x7f932c0d0d60 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f93217fa640 1 -- 172.21.15.40:0/2285431661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328007bd0 con 0x7f932c0d0d60 2024-04-26T21:35:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c1f8820 con 0x7f932c0d0d60 2024-04-26T21:35:22.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.952+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1f8de0 con 0x7f932c0d0d60 2024-04-26T21:35:22.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.953+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c0694f0 con 0x7f932c0d0d60 2024-04-26T21:35:22.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.954+0000 7f93217fa640 1 -- 172.21.15.40:0/2285431661 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9328018070 con 0x7f932c0d0d60 2024-04-26T21:35:22.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.954+0000 7f93217fa640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f931806c8f0 0x7f931806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:22.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.954+0000 7f93217fa640 1 -- 172.21.15.40:0/2285431661 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f932808bfa0 con 0x7f932c0d0d60 2024-04-26T21:35:22.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.954+0000 7f9323fff640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f931806c8f0 0x7f931806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:22.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.955+0000 7f9323fff640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f931806c8f0 0x7f931806edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f931c034000 tx=0x7f931c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:22.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:22.959+0000 7f93217fa640 1 -- 172.21.15.40:0/2285431661 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f932805a090 con 0x7f932c0d0d60 2024-04-26T21:35:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.144+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f932c06fe50 con 0x7f932c0d0d60 2024-04-26T21:35:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.145+0000 7f93217fa640 1 -- 172.21.15.40:0/2285431661 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f9328059990 con 0x7f932c0d0d60 2024-04-26T21:35:23.146 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f931806c8f0 msgr2=0x7f931806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f931806c8f0 0x7f931806edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f931c034000 tx=0x7f931c02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 msgr2=0x7f932c0a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0a05d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f932800c7f0 tx=0x7f932800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 shutdown_connections 2024-04-26T21:35:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f931806c8f0 0x7f931806edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c15f770 0x7f932c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0d1720 0x7f932c0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 --2- 172.21.15.40:0/2285431661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f932c0d0d60 0x7f932c0a05d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 >> 172.21.15.40:0/2285431661 conn(0x7f932c0d2e60 msgr2=0x7f932c15d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 shutdown_connections 2024-04-26T21:35:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.147+0000 7f9332e83640 1 -- 172.21.15.40:0/2285431661 wait complete. 2024-04-26T21:35:23.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:35:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.259+0000 7f26684c8640 1 Processor -- start 2024-04-26T21:35:23.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.259+0000 7f26684c8640 1 -- start start 2024-04-26T21:35:23.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f26684c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f266015f3a0 0x7f26600d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f26684c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d1770 0x7f26600d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f26684c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26600d25a0 con 0x7f26600d2130 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f26684c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f26600d0e20 con 0x7f266015f3a0 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f26684c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26600b1d70 con 0x7f26600d1770 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f2666264640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f266015f3a0 0x7f26600d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f2666a65640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f26600d2130 0x7f26600d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45220/0 (socket says 172.21.15.40:45220) 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f2666a65640 1 -- 172.21.15.40:0/1309748586 learned_addr learned my addr 172.21.15.40:0/1309748586 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.260+0000 7f2665a63640 1 --2- 172.21.15.40:0/1309748586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d1770 0x7f26600d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.261+0000 7f2666264640 1 -- 172.21.15.40:0/1309748586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d1770 msgr2=0x7f26600d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.261+0000 7f2666264640 1 --2- 172.21.15.40:0/1309748586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d1770 0x7f26600d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.261+0000 7f2666264640 1 -- 172.21.15.40:0/1309748586 >> v1:172.21.15.40:6789/0 conn(0x7f26600d2130 legacy=0x7f26600d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.261+0000 7f2666264640 1 -- 172.21.15.40:0/1309748586 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26600b2050 con 0x7f266015f3a0 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.261+0000 7f2666264640 1 --2- 172.21.15.40:0/1309748586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266015f3a0 0x7f26600d06f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26580097a0 tx=0x7f265802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b283b77c00fa4225 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.261+0000 7f2665262640 1 -- 172.21.15.40:0/1309748586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658037070 con 0x7f266015f3a0 2024-04-26T21:35:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.261+0000 7f2665262640 1 -- 172.21.15.40:0/1309748586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658032440 con 0x7f266015f3a0 2024-04-26T21:35:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.261+0000 7f2665262640 1 -- 172.21.15.40:0/1309748586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658032810 con 0x7f266015f3a0 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.262+0000 7f26684c8640 1 -- 172.21.15.40:0/1309748586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266015f3a0 msgr2=0x7f26600d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.262+0000 7f26684c8640 1 --2- 172.21.15.40:0/1309748586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266015f3a0 0x7f26600d06f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26580097a0 tx=0x7f265802a180 comp rx=0 tx=0).stop 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.262+0000 7f26684c8640 1 -- 172.21.15.40:0/1309748586 shutdown_connections 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.262+0000 7f26684c8640 1 --2- 172.21.15.40:0/1309748586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d1770 0x7f26600d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.262+0000 7f26684c8640 1 --2- 172.21.15.40:0/1309748586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266015f3a0 0x7f26600d06f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.262+0000 7f26684c8640 1 -- 172.21.15.40:0/1309748586 >> 172.21.15.40:0/1309748586 conn(0x7f26600d4f90 msgr2=0x7f26600d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.262+0000 7f26684c8640 1 -- 172.21.15.40:0/1309748586 shutdown_connections 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.262+0000 7f26684c8640 1 -- 172.21.15.40:0/1309748586 wait complete. 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.263+0000 7f26684c8640 1 Processor -- start 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.263+0000 7f26684c8640 1 -- start start 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f26684c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26600d1770 0x7f26600a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f26684c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d2130 0x7f266009ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f26684c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f266015f3a0 0x7f26601f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f26684c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f266015f900 con 0x7f26600d1770 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f26684c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f266015f780 con 0x7f266015f3a0 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f26684c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26600b3860 con 0x7f26600d2130 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666264640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26600d1770 0x7f26600a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666264640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26600d1770 0x7f26600a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57714/0 (socket says 172.21.15.40:57714) 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666264640 1 -- 172.21.15.40:0/4262624803 learned_addr learned my addr 172.21.15.40:0/4262624803 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2665a63640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d2130 0x7f266009ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666a65640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f266015f3a0 0x7f26601f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666264640 1 -- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d2130 msgr2=0x7f266009ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666264640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d2130 0x7f266009ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666264640 1 -- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f266015f3a0 msgr2=0x7f26601f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666264640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f266015f3a0 0x7f26601f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666264640 1 -- 172.21.15.40:0/4262624803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2660152ac0 con 0x7f26600d1770 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2665a63640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d2130 0x7f266009ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.264+0000 7f2666a65640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f266015f3a0 0x7f26601f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.265+0000 7f2666264640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26600d1770 0x7f26600a0870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2658035040 tx=0x7f26580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.265+0000 7f265effd640 1 -- 172.21.15.40:0/4262624803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658037040 con 0x7f26600d1770 2024-04-26T21:35:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.265+0000 7f265effd640 1 -- 172.21.15.40:0/4262624803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658030770 con 0x7f26600d1770 2024-04-26T21:35:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.265+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2660152da0 con 0x7f26600d1770 2024-04-26T21:35:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.265+0000 7f265effd640 1 -- 172.21.15.40:0/4262624803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658030b40 con 0x7f26600d1770 2024-04-26T21:35:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.265+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2660153360 con 0x7f26600d1770 2024-04-26T21:35:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.266+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26601537a0 con 0x7f26600d1770 2024-04-26T21:35:23.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.266+0000 7f265effd640 1 -- 172.21.15.40:0/4262624803 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2658044020 con 0x7f26600d1770 2024-04-26T21:35:23.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.267+0000 7f265effd640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264c06c8f0 0x7f264c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:23.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.267+0000 7f265effd640 1 -- 172.21.15.40:0/4262624803 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f26580ac420 con 0x7f26600d1770 2024-04-26T21:35:23.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.267+0000 7f2665a63640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264c06c8f0 0x7f264c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:23.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.267+0000 7f2665a63640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264c06c8f0 0x7f264c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f26601f8af0 tx=0x7f2654008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.271+0000 7f265effd640 1 -- 172.21.15.40:0/4262624803 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2658032e20 con 0x7f26600d1770 2024-04-26T21:35:23.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.466+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f26600694f0 con 0x7f26600d1770 2024-04-26T21:35:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.466+0000 7f265effd640 1 -- 172.21.15.40:0/4262624803 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f265807a480 con 0x7f26600d1770 2024-04-26T21:35:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:35:23.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264c06c8f0 msgr2=0x7f264c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264c06c8f0 0x7f264c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f26601f8af0 tx=0x7f2654008040 comp rx=0 tx=0).stop 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26600d1770 msgr2=0x7f26600a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26600d1770 0x7f26600a0870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2658035040 tx=0x7f26580046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 shutdown_connections 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264c06c8f0 0x7f264c06edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f266015f3a0 0x7f26601f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26600d2130 0x7f266009ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 --2- 172.21.15.40:0/4262624803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26600d1770 0x7f26600a0870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 >> 172.21.15.40:0/4262624803 conn(0x7f26600d4f90 msgr2=0x7f26600b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 shutdown_connections 2024-04-26T21:35:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:23.468+0000 7f26684c8640 1 -- 172.21.15.40:0/4262624803 wait complete. 2024-04-26T21:35:23.480 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2024-04-26T21:35:23.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:23.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:35:23.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[42133]: fuse finished with error 0 and tester_r 0 2024-04-26T21:35:23.504 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:35:29.606 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:35:29.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:29.607 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T21:35:29.645 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.800+0000 7f4a9c15f640 1 Processor -- start 2024-04-26T21:35:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.800+0000 7f4a9c15f640 1 -- start start 2024-04-26T21:35:29.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9c15f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 0x7f4a940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:29.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9c15f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a940d3bc0 0x7f4a940b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9c15f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a940b7e50 con 0x7f4a940b5f20 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9c15f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4a940d0580 con 0x7f4a940d3bc0 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9c15f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4a940d1800 con 0x7f4a94162880 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9a6fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a940d3bc0 0x7f4a940b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9a6fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a940d3bc0 0x7f4a940b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45280/0 (socket says 172.21.15.40:45280) 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9a6fc640 1 -- 172.21.15.40:0/2470636570 learned_addr learned my addr 172.21.15.40:0/2470636570 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a99efb640 1 --2- 172.21.15.40:0/2470636570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 0x7f4a940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9a6fc640 1 -- 172.21.15.40:0/2470636570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 msgr2=0x7f4a940d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9a6fc640 1 --2- 172.21.15.40:0/2470636570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 0x7f4a940d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9a6fc640 1 -- 172.21.15.40:0/2470636570 >> v1:172.21.15.184:6789/0 conn(0x7f4a940b5f20 legacy=0x7f4a940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.801+0000 7f4a9a6fc640 1 -- 172.21.15.40:0/2470636570 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a940d07d0 con 0x7f4a940d3bc0 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.802+0000 7f4a9a6fc640 1 --2- 172.21.15.40:0/2470636570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940d3bc0 0x7f4a940b8300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009e50 tx=0x7f4a8c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6447e82fcbe43b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.802+0000 7f4a98ef9640 1 -- 172.21.15.40:0/2470636570 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c036070 con 0x7f4a940d3bc0 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.802+0000 7f4a98ef9640 1 -- 172.21.15.40:0/2470636570 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c031600 con 0x7f4a940d3bc0 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.802+0000 7f4a98ef9640 1 -- 172.21.15.40:0/2470636570 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0319d0 con 0x7f4a940d3bc0 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.803+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2470636570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940d3bc0 msgr2=0x7f4a940b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.803+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2470636570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940d3bc0 0x7f4a940b8300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009e50 tx=0x7f4a8c02a6c0 comp rx=0 tx=0).stop 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.803+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2470636570 shutdown_connections 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.803+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2470636570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940d3bc0 0x7f4a940b8300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.803+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2470636570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 0x7f4a940d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.803+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2470636570 >> 172.21.15.40:0/2470636570 conn(0x7f4a940d4f60 msgr2=0x7f4a940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.803+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2470636570 shutdown_connections 2024-04-26T21:35:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.803+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2470636570 wait complete. 2024-04-26T21:35:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.804+0000 7f4a9c15f640 1 Processor -- start 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.804+0000 7f4a9c15f640 1 -- start start 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a9c15f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940b5f20 0x7f4a94159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a9c15f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a940d3bc0 0x7f4a940cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a9c15f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 0x7f4a9406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a9c15f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94162c60 con 0x7f4a940b5f20 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a9c15f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a940d0d90 con 0x7f4a940d3bc0 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a9c15f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4a94162de0 con 0x7f4a94162880 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a99efb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940b5f20 0x7f4a94159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a99efb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940b5f20 0x7f4a94159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45282/0 (socket says 172.21.15.40:45282) 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a99efb640 1 -- 172.21.15.40:0/2555555409 learned_addr learned my addr 172.21.15.40:0/2555555409 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a9a6fc640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 0x7f4a9406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a996fa640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a940d3bc0 0x7f4a940cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a99efb640 1 -- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 msgr2=0x7f4a9406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a99efb640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 0x7f4a9406cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a99efb640 1 -- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a940d3bc0 msgr2=0x7f4a940cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a99efb640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a940d3bc0 0x7f4a940cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.805+0000 7f4a99efb640 1 -- 172.21.15.40:0/2555555409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a940ceba0 con 0x7f4a940b5f20 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.806+0000 7f4a99efb640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940b5f20 0x7f4a94159110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000d5f0 tx=0x7f4a9000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.806+0000 7f4a8a7fc640 1 -- 172.21.15.40:0/2555555409 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900125f0 con 0x7f4a940b5f20 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.806+0000 7f4a8a7fc640 1 -- 172.21.15.40:0/2555555409 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a90019070 con 0x7f4a940b5f20 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.806+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a94159650 con 0x7f4a940b5f20 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.806+0000 7f4a8a7fc640 1 -- 172.21.15.40:0/2555555409 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900129d0 con 0x7f4a940b5f20 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.806+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a940ce8c0 con 0x7f4a940b5f20 2024-04-26T21:35:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.807+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a940694f0 con 0x7f4a940b5f20 2024-04-26T21:35:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.807+0000 7f4a8a7fc640 1 -- 172.21.15.40:0/2555555409 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a90012b70 con 0x7f4a940b5f20 2024-04-26T21:35:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.808+0000 7f4a8a7fc640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4a7806c8f0 0x7f4a7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.808+0000 7f4a8a7fc640 1 -- 172.21.15.40:0/2555555409 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4a9008c2a0 con 0x7f4a940b5f20 2024-04-26T21:35:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.808+0000 7f4a996fa640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4a7806c8f0 0x7f4a7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.809+0000 7f4a996fa640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4a7806c8f0 0x7f4a7806edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a940bac50 tx=0x7f4a84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:29.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:29.812+0000 7f4a8a7fc640 1 -- 172.21.15.40:0/2555555409 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a9005a380 con 0x7f4a940b5f20 2024-04-26T21:35:30.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.002+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a940cbaf0 con 0x7f4a940b5f20 2024-04-26T21:35:30.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.003+0000 7f4a8a7fc640 1 -- 172.21.15.40:0/2555555409 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f4a90059c80 con 0x7f4a940b5f20 2024-04-26T21:35:30.004 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:30.004 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4a7806c8f0 msgr2=0x7f4a7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4a7806c8f0 0x7f4a7806edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a940bac50 tx=0x7f4a84009040 comp rx=0 tx=0).stop 2024-04-26T21:35:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940b5f20 msgr2=0x7f4a94159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940b5f20 0x7f4a94159110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000d5f0 tx=0x7f4a9000dab0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 shutdown_connections 2024-04-26T21:35:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4a7806c8f0 0x7f4a7806edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a94162880 0x7f4a9406cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a940d3bc0 0x7f4a940cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 --2- 172.21.15.40:0/2555555409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a940b5f20 0x7f4a94159110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 >> 172.21.15.40:0/2555555409 conn(0x7f4a940d4f60 msgr2=0x7f4a940d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 shutdown_connections 2024-04-26T21:35:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.005+0000 7f4a9c15f640 1 -- 172.21.15.40:0/2555555409 wait complete. 2024-04-26T21:35:30.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:30.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.117+0000 7fe68fcf1640 1 Processor -- start 2024-04-26T21:35:30.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.117+0000 7fe68fcf1640 1 -- start start 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.118+0000 7fe68fcf1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe68815f3a0 0x7fe6880d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.118+0000 7fe68fcf1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe6880d2330 0x7fe6880b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.118+0000 7fe68fcf1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6880d0e20 con 0x7fe6880d1770 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.118+0000 7fe68fcf1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe6880b1d70 con 0x7fe68815f3a0 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.118+0000 7fe68fcf1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe6880b2050 con 0x7fe6880d2330 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.118+0000 7fe68da8d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe68815f3a0 0x7fe6880d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.118+0000 7fe68da8d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe68815f3a0 0x7fe6880d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45294/0 (socket says 172.21.15.40:45294) 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.118+0000 7fe68da8d640 1 -- 172.21.15.40:0/856285169 learned_addr learned my addr 172.21.15.40:0/856285169 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68e28e640 1 --2- 172.21.15.40:0/856285169 >> v2:172.21.15.184:3300/0 conn(0x7fe6880d2330 0x7fe6880b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68da8d640 1 -- 172.21.15.40:0/856285169 >> v2:172.21.15.184:3300/0 conn(0x7fe6880d2330 msgr2=0x7fe6880b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68da8d640 1 --2- 172.21.15.40:0/856285169 >> v2:172.21.15.184:3300/0 conn(0x7fe6880d2330 0x7fe6880b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68da8d640 1 -- 172.21.15.40:0/856285169 >> v1:172.21.15.184:6789/0 conn(0x7fe6880d1770 legacy=0x7fe6880d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68da8d640 1 -- 172.21.15.40:0/856285169 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6880b2330 con 0x7fe68815f3a0 2024-04-26T21:35:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68e28e640 1 --2- 172.21.15.40:0/856285169 >> v2:172.21.15.184:3300/0 conn(0x7fe6880d2330 0x7fe6880b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68da8d640 1 --2- 172.21.15.40:0/856285169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68815f3a0 0x7fe6880d06f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe688073b50 tx=0x7fe68402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb82b9e178ebe365 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68ca8b640 1 -- 172.21.15.40:0/856285169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684036070 con 0x7fe68815f3a0 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68ca8b640 1 -- 172.21.15.40:0/856285169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe684031600 con 0x7fe68815f3a0 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.119+0000 7fe68ca8b640 1 -- 172.21.15.40:0/856285169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840319d0 con 0x7fe68815f3a0 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.120+0000 7fe68fcf1640 1 -- 172.21.15.40:0/856285169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68815f3a0 msgr2=0x7fe6880d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.120+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/856285169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68815f3a0 0x7fe6880d06f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe688073b50 tx=0x7fe68402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.120+0000 7fe68fcf1640 1 -- 172.21.15.40:0/856285169 shutdown_connections 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.120+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/856285169 >> v2:172.21.15.184:3300/0 conn(0x7fe6880d2330 0x7fe6880b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.120+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/856285169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68815f3a0 0x7fe6880d06f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.120+0000 7fe68fcf1640 1 -- 172.21.15.40:0/856285169 >> 172.21.15.40:0/856285169 conn(0x7fe6880d4f90 msgr2=0x7fe6880d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:30.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.120+0000 7fe68fcf1640 1 -- 172.21.15.40:0/856285169 shutdown_connections 2024-04-26T21:35:30.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.120+0000 7fe68fcf1640 1 -- 172.21.15.40:0/856285169 wait complete. 2024-04-26T21:35:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.121+0000 7fe68fcf1640 1 Processor -- start 2024-04-26T21:35:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.121+0000 7fe68fcf1640 1 -- start start 2024-04-26T21:35:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68fcf1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6880d1770 0x7fe6880cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68fcf1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6880d2330 0x7fe68815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68fcf1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68815f3a0 0x7fe688159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68fcf1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe6880b2f10 con 0x7fe6880d2330 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68fcf1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6880b2d90 con 0x7fe68815f3a0 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68fcf1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6880b3090 con 0x7fe6880d1770 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68d28c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6880d2330 0x7fe68815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68d28c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6880d2330 0x7fe68815af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45304/0 (socket says 172.21.15.40:45304) 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68d28c640 1 -- 172.21.15.40:0/2555944254 learned_addr learned my addr 172.21.15.40:0/2555944254 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68e28e640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68815f3a0 0x7fe688159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68da8d640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6880d1770 0x7fe6880cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68d28c640 1 -- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6880d1770 msgr2=0x7fe6880cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68d28c640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6880d1770 0x7fe6880cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68d28c640 1 -- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68815f3a0 msgr2=0x7fe688159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68d28c640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68815f3a0 0x7fe688159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.122+0000 7fe68d28c640 1 -- 172.21.15.40:0/2555944254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68815b7d0 con 0x7fe6880d2330 2024-04-26T21:35:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.123+0000 7fe68d28c640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6880d2330 0x7fe68815af80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe67800c460 tx=0x7fe67800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.123+0000 7fe67e7fc640 1 -- 172.21.15.40:0/2555944254 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678016020 con 0x7fe6880d2330 2024-04-26T21:35:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.123+0000 7fe67e7fc640 1 -- 172.21.15.40:0/2555944254 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678012870 con 0x7fe6880d2330 2024-04-26T21:35:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.123+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68815bab0 con 0x7fe6880d2330 2024-04-26T21:35:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.123+0000 7fe67e7fc640 1 -- 172.21.15.40:0/2555944254 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678012c60 con 0x7fe6880d2330 2024-04-26T21:35:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.123+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68815c070 con 0x7fe6880d2330 2024-04-26T21:35:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.124+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6880694f0 con 0x7fe6880d2330 2024-04-26T21:35:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.124+0000 7fe67e7fc640 1 -- 172.21.15.40:0/2555944254 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe678019020 con 0x7fe6880d2330 2024-04-26T21:35:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.125+0000 7fe67e7fc640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe66c06c8f0 0x7fe66c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.125+0000 7fe67e7fc640 1 -- 172.21.15.40:0/2555944254 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe67808bad0 con 0x7fe6880d2330 2024-04-26T21:35:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.125+0000 7fe68da8d640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe66c06c8f0 0x7fe66c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.125+0000 7fe68da8d640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe66c06c8f0 0x7fe66c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe684033000 tx=0x7fe684009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.129+0000 7fe67e7fc640 1 -- 172.21.15.40:0/2555944254 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe678020030 con 0x7fe6880d2330 2024-04-26T21:35:30.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.320+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6880b6860 con 0x7fe6880d2330 2024-04-26T21:35:30.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.321+0000 7fe67e7fc640 1 -- 172.21.15.40:0/2555944254 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7fe678059560 con 0x7fe6880d2330 2024-04-26T21:35:30.323 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:30.323 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe66c06c8f0 msgr2=0x7fe66c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe66c06c8f0 0x7fe66c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe684033000 tx=0x7fe684009000 comp rx=0 tx=0).stop 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6880d2330 msgr2=0x7fe68815af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6880d2330 0x7fe68815af80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe67800c460 tx=0x7fe67800c920 comp rx=0 tx=0).stop 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 shutdown_connections 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe66c06c8f0 0x7fe66c06edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68815f3a0 0x7fe688159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6880d2330 0x7fe68815af80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 --2- 172.21.15.40:0/2555944254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6880d1770 0x7fe6880cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.323+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 >> 172.21.15.40:0/2555944254 conn(0x7fe6880d4f90 msgr2=0x7fe6880b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:30.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.324+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 shutdown_connections 2024-04-26T21:35:30.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.324+0000 7fe68fcf1640 1 -- 172.21.15.40:0/2555944254 wait complete. 2024-04-26T21:35:30.336 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:35:30.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.438+0000 7f0bf5bf7640 1 Processor -- start 2024-04-26T21:35:30.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.439+0000 7f0bf5bf7640 1 -- start start 2024-04-26T21:35:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.439+0000 7f0bf5bf7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0bf00ceb80 0x7f0bf016a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.439+0000 7f0bf5bf7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf00d1b80 con 0x7f0bf00b1fd0 2024-04-26T21:35:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.439+0000 7f0bf5bf7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf006e580 con 0x7f0bf015f770 2024-04-26T21:35:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.439+0000 7f0bf5bf7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf006f3e0 con 0x7f0bf00ceb80 2024-04-26T21:35:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.439+0000 7f0beeffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0bf00b1fd0 0x7f0bf00d1470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40078/0 (socket says 172.21.15.40:40078) 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.439+0000 7f0beeffd640 1 -- 172.21.15.40:0/1372638603 learned_addr learned my addr 172.21.15.40:0/1372638603 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0beffff640 1 --2- 172.21.15.40:0/1372638603 >> v2:172.21.15.184:3300/0 conn(0x7f0bf00ceb80 0x7f0bf016a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3159962667 0 0) 0x7f0bf00d1b80 con 0x7f0bf00b1fd0 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bd8003520 con 0x7f0bf00b1fd0 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914071277 0 0) 0x7f0bf006e580 con 0x7f0bf015f770 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bf00d1b80 con 0x7f0bf015f770 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3573280608 0 0) 0x7f0bd8003520 con 0x7f0bf00b1fd0 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 >> v2:172.21.15.184:3300/0 conn(0x7f0bf00ceb80 msgr2=0x7f0bf016a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 --2- 172.21.15.40:0/1372638603 >> v2:172.21.15.184:3300/0 conn(0x7f0bf00ceb80 0x7f0bf016a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 >> v1:172.21.15.184:6789/0 conn(0x7f0bf015f770 legacy=0x7f0bf00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf006e860 con 0x7f0bf00b1fd0 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f0be4001670 con 0x7f0bf00b1fd0 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 >> v1:172.21.15.40:6789/0 conn(0x7f0bf00b1fd0 legacy=0x7f0bf00d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd8003b40 0x7f0bd8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bd8004800 0x7f0bd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bd80051c0 0x7f0bd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf006f3e0 con 0x7f0bd80051c0 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd800c040 con 0x7f0bd8003b40 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd800c240 con 0x7f0bd8004800 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0beffff640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bd80051c0 0x7f0bd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0bef7fe640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd8003b40 0x7f0bd8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.440+0000 7f0beeffd640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bd8004800 0x7f0bd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0beffff640 1 -- 172.21.15.40:0/1372638603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bd8004800 msgr2=0x7f0bd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0beffff640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bd8004800 0x7f0bd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0beffff640 1 -- 172.21.15.40:0/1372638603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd8003b40 msgr2=0x7f0bd8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0beffff640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd8003b40 0x7f0bd8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0beffff640 1 -- 172.21.15.40:0/1372638603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd8003520 con 0x7f0bd80051c0 2024-04-26T21:35:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0beffff640 1 -- 172.21.15.40:0/1372638603 reap_dead start 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0beffff640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bd80051c0 0x7f0bd800b9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009f90 tx=0x7f0bd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c760da7a322ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8009c10 con 0x7f0bd80051c0 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be8031440 con 0x7f0bd80051c0 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.441+0000 7f0bee7fc640 1 -- 172.21.15.40:0/1372638603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8031810 con 0x7f0bd80051c0 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.442+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/1372638603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bd80051c0 msgr2=0x7f0bd800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.442+0000 7f0bf5bf7640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bd80051c0 0x7f0bd800b9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009f90 tx=0x7f0bd8005830 comp rx=0 tx=0).stop 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.442+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/1372638603 shutdown_connections 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.442+0000 7f0bf5bf7640 1 --2- 172.21.15.40:0/1372638603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bd80051c0 0x7f0bd800b9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.442+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/1372638603 >> 172.21.15.40:0/1372638603 conn(0x7f0bf00d2e60 msgr2=0x7f0bf00d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.442+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/1372638603 shutdown_connections 2024-04-26T21:35:30.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.442+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/1372638603 wait complete. 2024-04-26T21:35:30.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.443+0000 7f0bf5bf7640 1 Processor -- start 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.443+0000 7f0bf5bf7640 1 -- start start 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0bf5bf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd80051c0 0x7f0bf01f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0bf5bf7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf00c69a0 0x7f0bf015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0bf5bf7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf00c8e30 0x7f0bf00c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0bf5bf7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf00d0d60 con 0x7f0bf00c69a0 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0bf5bf7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf00cd6e0 con 0x7f0bd80051c0 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0bf5bf7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf00d0ee0 con 0x7f0bf00c8e30 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0beeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf00c69a0 0x7f0bf015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0beeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf00c69a0 0x7f0bf015baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45328/0 (socket says 172.21.15.40:45328) 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0beeffd640 1 -- 172.21.15.40:0/737710824 learned_addr learned my addr 172.21.15.40:0/737710824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0bef7fe640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd80051c0 0x7f0bf01f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0beeffd640 1 -- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf00c8e30 msgr2=0x7f0bf00c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0beeffd640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf00c8e30 0x7f0bf00c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0beeffd640 1 -- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd80051c0 msgr2=0x7f0bf01f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0beeffd640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd80051c0 0x7f0bf01f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.444+0000 7f0beeffd640 1 -- 172.21.15.40:0/737710824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf01f6280 con 0x7f0bf00c69a0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.445+0000 7f0beeffd640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf00c69a0 0x7f0bf015baa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0be4001050 tx=0x7f0be4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.445+0000 7f0bd7fff640 1 -- 172.21.15.40:0/737710824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be400f6e0 con 0x7f0bf00c69a0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.445+0000 7f0bd7fff640 1 -- 172.21.15.40:0/737710824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be4018070 con 0x7f0bf00c69a0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.445+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf00c7020 con 0x7f0bf00c69a0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.445+0000 7f0bd7fff640 1 -- 172.21.15.40:0/737710824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be400fb00 con 0x7f0bf00c69a0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.445+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf00c75e0 con 0x7f0bf00c69a0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.446+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf00694f0 con 0x7f0bf00c69a0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.446+0000 7f0bd7fff640 1 -- 172.21.15.40:0/737710824 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0be400fca0 con 0x7f0bf00c69a0 2024-04-26T21:35:30.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.447+0000 7f0bd7fff640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.447+0000 7f0bd7fff640 1 -- 172.21.15.40:0/737710824 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0be408be00 con 0x7f0bf00c69a0 2024-04-26T21:35:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.447+0000 7f0bef7fe640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.447+0000 7f0bef7fe640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0be0000c00 tx=0x7f0be0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:30.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.451+0000 7f0bd7fff640 1 -- 172.21.15.40:0/737710824 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0bf01f6280 con 0x7f0bf00c69a0 2024-04-26T21:35:30.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:30.682+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0bf006fe50 con 0x7f0bf00c69a0 2024-04-26T21:35:31.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.193+0000 7f0bd7fff640 1 -- 172.21.15.40:0/737710824 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f0be4059930 con 0x7f0bf00c69a0 2024-04-26T21:35:31.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bdc06c8f0 msgr2=0x7f0bdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:31.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0be0000c00 tx=0x7f0be0009040 comp rx=0 tx=0).stop 2024-04-26T21:35:31.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf00c69a0 msgr2=0x7f0bf015baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:31.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf00c69a0 0x7f0bf015baa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0be4001050 tx=0x7f0be4004f90 comp rx=0 tx=0).stop 2024-04-26T21:35:31.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 shutdown_connections 2024-04-26T21:35:31.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf00c8e30 0x7f0bf00c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf00c69a0 0x7f0bf015baa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 --2- 172.21.15.40:0/737710824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bd80051c0 0x7f0bf01f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 >> 172.21.15.40:0/737710824 conn(0x7f0bf00d2e60 msgr2=0x7f0bf00700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:31.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 shutdown_connections 2024-04-26T21:35:31.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.196+0000 7f0bf5bf7640 1 -- 172.21.15.40:0/737710824 wait complete. 2024-04-26T21:35:31.210 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-26T21:35:31.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:35:31.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.320+0000 7fb8e15e4640 1 Processor -- start 2024-04-26T21:35:31.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.320+0000 7fb8e15e4640 1 -- start start 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8e15e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8dc072fb0 0x7fb8dc0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8e15e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8dc0b06f0 0x7fb8dc06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8e15e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8dc0b5e30 con 0x7fb8dc073390 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8e15e4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc0b0b80 con 0x7fb8dc072fb0 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8e15e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc0b5c00 con 0x7fb8dc0b06f0 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8daffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8dc072fb0 0x7fb8dc0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8da7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8dc073390 0x7fb8dc06f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40094/0 (socket says 172.21.15.40:40094) 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8daffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8dc072fb0 0x7fb8dc0b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45344/0 (socket says 172.21.15.40:45344) 2024-04-26T21:35:31.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.321+0000 7fb8da7fc640 1 -- 172.21.15.40:0/1007417804 learned_addr learned my addr 172.21.15.40:0/1007417804 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:31.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.322+0000 7fb8db7fe640 1 --2- 172.21.15.40:0/1007417804 >> v2:172.21.15.184:3300/0 conn(0x7fb8dc0b06f0 0x7fb8dc06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:31.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.322+0000 7fb8daffd640 1 -- 172.21.15.40:0/1007417804 >> v2:172.21.15.184:3300/0 conn(0x7fb8dc0b06f0 msgr2=0x7fb8dc06fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:31.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.322+0000 7fb8daffd640 1 --2- 172.21.15.40:0/1007417804 >> v2:172.21.15.184:3300/0 conn(0x7fb8dc0b06f0 0x7fb8dc06fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.322+0000 7fb8d9ffb640 1 -- 172.21.15.40:0/1007417804 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3067385132 0 0) 0x7fb8dc0b5e30 con 0x7fb8dc073390 2024-04-26T21:35:31.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.322+0000 7fb8daffd640 1 -- 172.21.15.40:0/1007417804 >> v1:172.21.15.40:6789/0 conn(0x7fb8dc073390 legacy=0x7fb8dc06f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:31.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.322+0000 7fb8daffd640 1 -- 172.21.15.40:0/1007417804 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc0b6110 con 0x7fb8dc072fb0 2024-04-26T21:35:31.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.322+0000 7fb8daffd640 1 --2- 172.21.15.40:0/1007417804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc072fb0 0x7fb8dc0b56c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc071b40 tx=0x7fb8d402c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50918f6e467a5aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:31.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8d9ffb640 1 -- 172.21.15.40:0/1007417804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4032770 con 0x7fb8dc072fb0 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8d9ffb640 1 -- 172.21.15.40:0/1007417804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d4038070 con 0x7fb8dc072fb0 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8d9ffb640 1 -- 172.21.15.40:0/1007417804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4032b30 con 0x7fb8dc072fb0 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1007417804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc072fb0 msgr2=0x7fb8dc0b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1007417804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc072fb0 0x7fb8dc0b56c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc071b40 tx=0x7fb8d402c870 comp rx=0 tx=0).stop 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1007417804 shutdown_connections 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1007417804 >> v2:172.21.15.184:3300/0 conn(0x7fb8dc0b06f0 0x7fb8dc06fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1007417804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc072fb0 0x7fb8dc0b56c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1007417804 >> 172.21.15.40:0/1007417804 conn(0x7fb8dc0b39b0 msgr2=0x7fb8dc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1007417804 shutdown_connections 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.323+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1007417804 wait complete. 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.324+0000 7fb8e15e4640 1 Processor -- start 2024-04-26T21:35:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8e15e4640 1 -- start start 2024-04-26T21:35:31.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8e15e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc072fb0 0x7fb8dc1f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8e15e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc073390 0x7fb8dc06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8e15e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0b06f0 0x7fb8dc070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8e15e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0737f0 con 0x7fb8dc073390 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8e15e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc06f2b0 con 0x7fb8dc072fb0 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8e15e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc15c160 con 0x7fb8dc0b06f0 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8da7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc073390 0x7fb8dc06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8da7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc073390 0x7fb8dc06fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45360/0 (socket says 172.21.15.40:45360) 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8da7fc640 1 -- 172.21.15.40:0/1858050505 learned_addr learned my addr 172.21.15.40:0/1858050505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8daffd640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc072fb0 0x7fb8dc1f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.325+0000 7fb8db7fe640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0b06f0 0x7fb8dc070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8da7fc640 1 -- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0b06f0 msgr2=0x7fb8dc070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8da7fc640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0b06f0 0x7fb8dc070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8da7fc640 1 -- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc072fb0 msgr2=0x7fb8dc1f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8da7fc640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc072fb0 0x7fb8dc1f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8da7fc640 1 -- 172.21.15.40:0/1858050505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc14b110 con 0x7fb8dc073390 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8da7fc640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc073390 0x7fb8dc06fc50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c500 tx=0x7fb8c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8d37fe640 1 -- 172.21.15.40:0/1858050505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4018070 con 0x7fb8dc073390 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8d37fe640 1 -- 172.21.15.40:0/1858050505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c4007580 con 0x7fb8dc073390 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc14b3f0 con 0x7fb8dc073390 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8d37fe640 1 -- 172.21.15.40:0/1858050505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4004a20 con 0x7fb8dc073390 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.326+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc14b9b0 con 0x7fb8dc073390 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.327+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc14bbb0 con 0x7fb8dc073390 2024-04-26T21:35:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.328+0000 7fb8d37fe640 1 -- 172.21.15.40:0/1858050505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8c401b070 con 0x7fb8dc073390 2024-04-26T21:35:31.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.328+0000 7fb8d37fe640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:31.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.328+0000 7fb8d37fe640 1 -- 172.21.15.40:0/1858050505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb8c408b0b0 con 0x7fb8dc073390 2024-04-26T21:35:31.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.329+0000 7fb8daffd640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:31.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.329+0000 7fb8daffd640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4037000 tx=0x7fb8d4032000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:31.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.333+0000 7fb8d37fe640 1 -- 172.21.15.40:0/1858050505 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8dc14bbb0 con 0x7fb8dc073390 2024-04-26T21:35:31.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:31.564+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb8dc0694f0 con 0x7fb8dc073390 2024-04-26T21:35:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.198+0000 7fb8d37fe640 1 -- 172.21.15.40:0/1858050505 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fb8c4058be0 con 0x7fb8dc073390 2024-04-26T21:35:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8bc06c8f0 msgr2=0x7fb8bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4037000 tx=0x7fb8d4032000 comp rx=0 tx=0).stop 2024-04-26T21:35:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc073390 msgr2=0x7fb8dc06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc073390 0x7fb8dc06fc50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c500 tx=0x7fb8c400c9c0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 shutdown_connections 2024-04-26T21:35:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0b06f0 0x7fb8dc070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc073390 0x7fb8dc06fc50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 --2- 172.21.15.40:0/1858050505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc072fb0 0x7fb8dc1f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 >> 172.21.15.40:0/1858050505 conn(0x7fb8dc0b39b0 msgr2=0x7fb8dc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 shutdown_connections 2024-04-26T21:35:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.201+0000 7fb8e15e4640 1 -- 172.21.15.40:0/1858050505 wait complete. 2024-04-26T21:35:32.215 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:35:32.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:35:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.321+0000 7f670eb4e640 1 Processor -- start 2024-04-26T21:35:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670eb4e640 1 -- start start 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670eb4e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f670815f3a0 0x7f67080d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670eb4e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f67080d1770 0x7f67080d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670eb4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67080d2590 con 0x7f67080d2160 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670eb4e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f67080d0e20 con 0x7f670815f3a0 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670eb4e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f67080b1d70 con 0x7f67080d1770 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670c8ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f670815f3a0 0x7f67080d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670c8ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f670815f3a0 0x7f67080d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45362/0 (socket says 172.21.15.40:45362) 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.322+0000 7f670c8ea640 1 -- 172.21.15.40:0/1217619733 learned_addr learned my addr 172.21.15.40:0/1217619733 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.323+0000 7f6707fff640 1 --2- 172.21.15.40:0/1217619733 >> v2:172.21.15.184:3300/0 conn(0x7f67080d1770 0x7f67080d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.323+0000 7f670c8ea640 1 -- 172.21.15.40:0/1217619733 >> v2:172.21.15.184:3300/0 conn(0x7f67080d1770 msgr2=0x7f67080d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.323+0000 7f670c8ea640 1 --2- 172.21.15.40:0/1217619733 >> v2:172.21.15.184:3300/0 conn(0x7f67080d1770 0x7f67080d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.323+0000 7f670c8ea640 1 -- 172.21.15.40:0/1217619733 >> v1:172.21.15.184:6789/0 conn(0x7f67080d2160 legacy=0x7f67080d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.323+0000 7f670c8ea640 1 -- 172.21.15.40:0/1217619733 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67080b2050 con 0x7f670815f3a0 2024-04-26T21:35:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.323+0000 7f670c8ea640 1 --2- 172.21.15.40:0/1217619733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080d06f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67080b7b80 tx=0x7f66fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae1d320f45199d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f67077fe640 1 -- 172.21.15.40:0/1217619733 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc037070 con 0x7f670815f3a0 2024-04-26T21:35:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f67077fe640 1 -- 172.21.15.40:0/1217619733 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66fc032440 con 0x7f670815f3a0 2024-04-26T21:35:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f67077fe640 1 -- 172.21.15.40:0/1217619733 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc032810 con 0x7f670815f3a0 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f670eb4e640 1 -- 172.21.15.40:0/1217619733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 msgr2=0x7f67080d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f670eb4e640 1 --2- 172.21.15.40:0/1217619733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080d06f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67080b7b80 tx=0x7f66fc02a270 comp rx=0 tx=0).stop 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f670eb4e640 1 -- 172.21.15.40:0/1217619733 shutdown_connections 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f670eb4e640 1 --2- 172.21.15.40:0/1217619733 >> v2:172.21.15.184:3300/0 conn(0x7f67080d1770 0x7f67080d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f670eb4e640 1 --2- 172.21.15.40:0/1217619733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080d06f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f670eb4e640 1 -- 172.21.15.40:0/1217619733 >> 172.21.15.40:0/1217619733 conn(0x7f67080d4f90 msgr2=0x7f67080d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f670eb4e640 1 -- 172.21.15.40:0/1217619733 shutdown_connections 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.324+0000 7f670eb4e640 1 -- 172.21.15.40:0/1217619733 wait complete. 2024-04-26T21:35:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.325+0000 7f670eb4e640 1 Processor -- start 2024-04-26T21:35:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.325+0000 7f670eb4e640 1 -- start start 2024-04-26T21:35:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.326+0000 7f670eb4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67080d1770 0x7f67080b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670eb4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67080d2160 0x7f67080ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670eb4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670eb4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f670815f900 con 0x7f670815f3a0 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670eb4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f670815f780 con 0x7f67080d1770 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670eb4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67080b3860 con 0x7f67080d2160 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45370/0 (socket says 172.21.15.40:45370) 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 -- 172.21.15.40:0/3954862034 learned_addr learned my addr 172.21.15.40:0/3954862034 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670c8ea640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67080d1770 0x7f67080b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f6707fff640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67080d2160 0x7f67080ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 -- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67080d2160 msgr2=0x7f67080ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67080d2160 0x7f67080ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 -- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67080d1770 msgr2=0x7f67080b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67080d1770 0x7f67080b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 -- 172.21.15.40:0/3954862034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67080cd030 con 0x7f670815f3a0 2024-04-26T21:35:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670d0eb640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080ced00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f670000c7a0 tx=0x7f670000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f67057fa640 1 -- 172.21.15.40:0/3954862034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700003b70 con 0x7f670815f3a0 2024-04-26T21:35:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f67057fa640 1 -- 172.21.15.40:0/3954862034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6700003d30 con 0x7f670815f3a0 2024-04-26T21:35:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f67057fa640 1 -- 172.21.15.40:0/3954862034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670000aa90 con 0x7f670815f3a0 2024-04-26T21:35:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67080cd2e0 con 0x7f670815f3a0 2024-04-26T21:35:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.327+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67080cd8a0 con 0x7f670815f3a0 2024-04-26T21:35:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.328+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67080694f0 con 0x7f670815f3a0 2024-04-26T21:35:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.329+0000 7f67057fa640 1 -- 172.21.15.40:0/3954862034 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f670000ac30 con 0x7f670815f3a0 2024-04-26T21:35:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.329+0000 7f67057fa640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66ec06c8f0 0x7f66ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.329+0000 7f67057fa640 1 -- 172.21.15.40:0/3954862034 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f670008bcb0 con 0x7f670815f3a0 2024-04-26T21:35:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.330+0000 7f670c8ea640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66ec06c8f0 0x7f66ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.330+0000 7f670c8ea640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66ec06c8f0 0x7f66ec06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66fc034000 tx=0x7f66fc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.334+0000 7f67057fa640 1 -- 172.21.15.40:0/3954862034 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6700059740 con 0x7f670815f3a0 2024-04-26T21:35:32.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.555+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67080b6860 con 0x7f670815f3a0 2024-04-26T21:35:32.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.556+0000 7f67057fa640 1 -- 172.21.15.40:0/3954862034 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f670005da40 con 0x7f670815f3a0 2024-04-26T21:35:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:32.557 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:31.566271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:35:32.557 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.557+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66ec06c8f0 msgr2=0x7f66ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.557+0000 7f670eb4e640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66ec06c8f0 0x7f66ec06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66fc034000 tx=0x7f66fc02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.557+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 msgr2=0x7f67080ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.557+0000 7f670eb4e640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080ced00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f670000c7a0 tx=0x7f670000cc60 comp rx=0 tx=0).stop 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.558+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 shutdown_connections 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.558+0000 7f670eb4e640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66ec06c8f0 0x7f66ec06edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.558+0000 7f670eb4e640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670815f3a0 0x7f67080ced00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.558+0000 7f670eb4e640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67080d2160 0x7f67080ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.558+0000 7f670eb4e640 1 --2- 172.21.15.40:0/3954862034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67080d1770 0x7f67080b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.558+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 >> 172.21.15.40:0/3954862034 conn(0x7f67080d4f90 msgr2=0x7f670806dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:32.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.558+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 shutdown_connections 2024-04-26T21:35:32.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.558+0000 7f670eb4e640 1 -- 172.21.15.40:0/3954862034 wait complete. 2024-04-26T21:35:32.570 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.669+0000 7ff831862640 1 Processor -- start 2024-04-26T21:35:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.669+0000 7ff831862640 1 -- start start 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.669+0000 7ff831862640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff82c162880 0x7ff82c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.669+0000 7ff831862640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0d3bc0 0x7ff82c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.669+0000 7ff831862640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff82c0b7e50 con 0x7ff82c0b5f20 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.669+0000 7ff831862640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff82c0d0580 con 0x7ff82c162880 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.669+0000 7ff831862640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c0d1800 con 0x7ff82c0d3bc0 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.670+0000 7ff82affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff82c162880 0x7ff82c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.670+0000 7ff82affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff82c162880 0x7ff82c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45386/0 (socket says 172.21.15.40:45386) 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.670+0000 7ff82affd640 1 -- 172.21.15.40:0/2711628897 learned_addr learned my addr 172.21.15.40:0/2711628897 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.670+0000 7ff82b7fe640 1 --2- 172.21.15.40:0/2711628897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0d3bc0 0x7ff82c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.670+0000 7ff82affd640 1 -- 172.21.15.40:0/2711628897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0d3bc0 msgr2=0x7ff82c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.670+0000 7ff82affd640 1 --2- 172.21.15.40:0/2711628897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0d3bc0 0x7ff82c0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.670+0000 7ff82affd640 1 -- 172.21.15.40:0/2711628897 >> v1:172.21.15.184:6789/0 conn(0x7ff82c0b5f20 legacy=0x7ff82c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.670+0000 7ff82affd640 1 -- 172.21.15.40:0/2711628897 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c0d07d0 con 0x7ff82c162880 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff82affd640 1 --2- 172.21.15.40:0/2711628897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c0d2cb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009870 tx=0x7ff81c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a2b4612b5664a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff829ffb640 1 -- 172.21.15.40:0/2711628897 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c037070 con 0x7ff82c162880 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff829ffb640 1 -- 172.21.15.40:0/2711628897 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c032440 con 0x7ff82c162880 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff829ffb640 1 -- 172.21.15.40:0/2711628897 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c032810 con 0x7ff82c162880 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff831862640 1 -- 172.21.15.40:0/2711628897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 msgr2=0x7ff82c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff831862640 1 --2- 172.21.15.40:0/2711628897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c0d2cb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009870 tx=0x7ff81c02a270 comp rx=0 tx=0).stop 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff831862640 1 -- 172.21.15.40:0/2711628897 shutdown_connections 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff831862640 1 --2- 172.21.15.40:0/2711628897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0d3bc0 0x7ff82c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff831862640 1 --2- 172.21.15.40:0/2711628897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c0d2cb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff831862640 1 -- 172.21.15.40:0/2711628897 >> 172.21.15.40:0/2711628897 conn(0x7ff82c0d4f60 msgr2=0x7ff82c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff831862640 1 -- 172.21.15.40:0/2711628897 shutdown_connections 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.671+0000 7ff831862640 1 -- 172.21.15.40:0/2711628897 wait complete. 2024-04-26T21:35:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.673+0000 7ff831862640 1 Processor -- start 2024-04-26T21:35:32.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.673+0000 7ff831862640 1 -- start start 2024-04-26T21:35:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.673+0000 7ff831862640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0b5f20 0x7ff82c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.673+0000 7ff831862640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c0d3bc0 0x7ff82c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.673+0000 7ff831862640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.673+0000 7ff831862640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c162c60 con 0x7ff82c162880 2024-04-26T21:35:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.673+0000 7ff831862640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c0d0d90 con 0x7ff82c0d3bc0 2024-04-26T21:35:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.673+0000 7ff831862640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c162de0 con 0x7ff82c0b5f20 2024-04-26T21:35:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45400/0 (socket says 172.21.15.40:45400) 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 -- 172.21.15.40:0/4036919069 learned_addr learned my addr 172.21.15.40:0/4036919069 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82affd640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0b5f20 0x7ff82c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82a7fc640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c0d3bc0 0x7ff82c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 -- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0b5f20 msgr2=0x7ff82c0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0b5f20 0x7ff82c0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 -- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c0d3bc0 msgr2=0x7ff82c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c0d3bc0 0x7ff82c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 -- 172.21.15.40:0/4036919069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c0b30a0 con 0x7ff82c162880 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff82b7fe640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c151330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff824007d70 tx=0x7ff82400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff813fff640 1 -- 172.21.15.40:0/4036919069 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824014420 con 0x7ff82c162880 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff813fff640 1 -- 172.21.15.40:0/4036919069 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff824019070 con 0x7ff82c162880 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff813fff640 1 -- 172.21.15.40:0/4036919069 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824014820 con 0x7ff82c162880 2024-04-26T21:35:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c0b3380 con 0x7ff82c162880 2024-04-26T21:35:32.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.674+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c0b3940 con 0x7ff82c162880 2024-04-26T21:35:32.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.675+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82c0694f0 con 0x7ff82c162880 2024-04-26T21:35:32.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.676+0000 7ff813fff640 1 -- 172.21.15.40:0/4036919069 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8240149c0 con 0x7ff82c162880 2024-04-26T21:35:32.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.676+0000 7ff813fff640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff81806c8f0 0x7ff81806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.676+0000 7ff813fff640 1 -- 172.21.15.40:0/4036919069 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff82408c4b0 con 0x7ff82c162880 2024-04-26T21:35:32.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.676+0000 7ff82affd640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff81806c8f0 0x7ff81806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.677+0000 7ff82affd640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff81806c8f0 0x7ff81806edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff81c034000 tx=0x7ff81c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.681+0000 7ff813fff640 1 -- 172.21.15.40:0/4036919069 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff824059fe0 con 0x7ff82c162880 2024-04-26T21:35:32.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.869+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff82c0d3790 con 0x7ff82c162880 2024-04-26T21:35:32.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.870+0000 7ff813fff640 1 -- 172.21.15.40:0/4036919069 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7ff82405e2e0 con 0x7ff82c162880 2024-04-26T21:35:32.872 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:32.872 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff81806c8f0 msgr2=0x7ff81806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff81806c8f0 0x7ff81806edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff81c034000 tx=0x7ff81c02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 msgr2=0x7ff82c151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c151330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff824007d70 tx=0x7ff82400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 shutdown_connections 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff81806c8f0 0x7ff81806edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff82c162880 0x7ff82c151330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c0d3bc0 0x7ff82c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 --2- 172.21.15.40:0/4036919069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c0b5f20 0x7ff82c0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 >> 172.21.15.40:0/4036919069 conn(0x7ff82c0d4f60 msgr2=0x7ff82c06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 shutdown_connections 2024-04-26T21:35:32.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.872+0000 7ff831862640 1 -- 172.21.15.40:0/4036919069 wait complete. 2024-04-26T21:35:32.885 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:35:32.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:35:32.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.985+0000 7f80f9f51640 1 Processor -- start 2024-04-26T21:35:32.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.985+0000 7f80f9f51640 1 -- start start 2024-04-26T21:35:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f9f51640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f40d3bc0 0x7f80f40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f9f51640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f40b7e50 con 0x7f80f4162880 2024-04-26T21:35:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f9f51640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f40d1800 con 0x7f80f40b5f20 2024-04-26T21:35:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f9f51640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80f40d0580 con 0x7f80f40d3bc0 2024-04-26T21:35:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f80f4162880 0x7f80f40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40096/0 (socket says 172.21.15.40:40096) 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f37fe640 1 -- 172.21.15.40:0/1860299135 learned_addr learned my addr 172.21.15.40:0/1860299135 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f2ffd640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f40d3bc0 0x7f80f40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183642085 0 0) 0x7f80f40b7e50 con 0x7f80f4162880 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80dc003520 con 0x7f80f4162880 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1805641954 0 0) 0x7f80dc003520 con 0x7f80f4162880 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f40d3bc0 msgr2=0x7f80f40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f40d3bc0 0x7f80f40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 >> v1:172.21.15.184:6789/0 conn(0x7f80f40b5f20 legacy=0x7f80f40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f40d07d0 con 0x7f80f4162880 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f80e4001670 con 0x7f80f4162880 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 >> v1:172.21.15.40:6789/0 conn(0x7f80f4162880 legacy=0x7f80f40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.986+0000 7f80f27fc640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80dc0039e0 0x7f80dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f27fc640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f40d3bc0 0x7f80dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f27fc640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80dc004d40 0x7f80dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f80f40d0580 con 0x7f80dc0039e0 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc00ba70 con 0x7f80dc004d40 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80dc00bc70 con 0x7f80f40d3bc0 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f37fe640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80dc0039e0 0x7f80dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f2ffd640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f40d3bc0 0x7f80dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f3fff640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80dc004d40 0x7f80dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f37fe640 1 -- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f40d3bc0 msgr2=0x7f80dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f37fe640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f40d3bc0 0x7f80dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f37fe640 1 -- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80dc004d40 msgr2=0x7f80dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f37fe640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80dc004d40 0x7f80dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.987+0000 7f80f37fe640 1 -- 172.21.15.40:0/1860299135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc003520 con 0x7f80dc0039e0 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f37fe640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80dc0039e0 0x7f80dc003e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f80e400c210 tx=0x7f80dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f5db3eaa44ad71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4007d10 con 0x7f80dc0039e0 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e40058d0 con 0x7f80dc0039e0 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f27fc640 1 -- 172.21.15.40:0/1860299135 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4005d00 con 0x7f80dc0039e0 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f9f51640 1 -- 172.21.15.40:0/1860299135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80dc0039e0 msgr2=0x7f80dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f9f51640 1 --2- 172.21.15.40:0/1860299135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80dc0039e0 0x7f80dc003e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f80e400c210 tx=0x7f80dc005830 comp rx=0 tx=0).stop 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f3fff640 1 -- 172.21.15.40:0/1860299135 reap_dead start 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f9f51640 1 -- 172.21.15.40:0/1860299135 shutdown_connections 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f9f51640 1 -- 172.21.15.40:0/1860299135 >> 172.21.15.40:0/1860299135 conn(0x7f80f40d4f60 msgr2=0x7f80f40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f9f51640 1 -- 172.21.15.40:0/1860299135 shutdown_connections 2024-04-26T21:35:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.988+0000 7f80f9f51640 1 -- 172.21.15.40:0/1860299135 wait complete. 2024-04-26T21:35:32.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.990+0000 7f80f9f51640 1 Processor -- start 2024-04-26T21:35:32.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.990+0000 7f80f9f51640 1 -- start start 2024-04-26T21:35:32.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.990+0000 7f80f9f51640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80dc0039e0 0x7f80f4156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.990+0000 7f80f9f51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f41f4b10 0x7f80f40b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.990+0000 7f80f9f51640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40b4fa0 0x7f80f415f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.990+0000 7f80f9f51640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4162c60 con 0x7f80f41f4b10 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.990+0000 7f80f9f51640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80f40d0d90 con 0x7f80f40b4fa0 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f9f51640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80f4162de0 con 0x7f80dc0039e0 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f41f4b10 0x7f80f40b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f41f4b10 0x7f80f40b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45418/0 (socket says 172.21.15.40:45418) 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 -- 172.21.15.40:0/4228734765 learned_addr learned my addr 172.21.15.40:0/4228734765 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f37fe640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80dc0039e0 0x7f80f4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f3fff640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40b4fa0 0x7f80f415f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 -- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80dc0039e0 msgr2=0x7f80f4156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80dc0039e0 0x7f80f4156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 -- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40b4fa0 msgr2=0x7f80f415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40b4fa0 0x7f80f415f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 -- 172.21.15.40:0/4228734765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f40b5620 con 0x7f80f41f4b10 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f2ffd640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f41f4b10 0x7f80f40b4a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80e80119d0 tx=0x7f80e8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80dbfff640 1 -- 172.21.15.40:0/4228734765 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e800faf0 con 0x7f80f41f4b10 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80dbfff640 1 -- 172.21.15.40:0/4228734765 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e800fc90 con 0x7f80f41f4b10 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f40b5900 con 0x7f80f41f4b10 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80dbfff640 1 -- 172.21.15.40:0/4228734765 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8014440 con 0x7f80f41f4b10 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.991+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f40cda20 con 0x7f80f41f4b10 2024-04-26T21:35:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.992+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f40b5380 con 0x7f80f41f4b10 2024-04-26T21:35:32.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.993+0000 7f80dbfff640 1 -- 172.21.15.40:0/4228734765 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80e8012040 con 0x7f80f41f4b10 2024-04-26T21:35:32.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.993+0000 7f80dbfff640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80e006c8f0 0x7f80e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:32.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.993+0000 7f80dbfff640 1 -- 172.21.15.40:0/4228734765 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f80e808c410 con 0x7f80f41f4b10 2024-04-26T21:35:32.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.994+0000 7f80f37fe640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80e006c8f0 0x7f80e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:32.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.994+0000 7f80f37fe640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80e006c8f0 0x7f80e006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f80e4033000 tx=0x7f80e402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:32.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:32.998+0000 7f80dbfff640 1 -- 172.21.15.40:0/4228734765 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80e8019030 con 0x7f80f41f4b10 2024-04-26T21:35:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.181+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80f40b5ad0 con 0x7f80e006c8f0 2024-04-26T21:35:33.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.191+0000 7f80dbfff640 1 -- 172.21.15.40:0/4228734765 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f80e4000ed0 con 0x7f80e006c8f0 2024-04-26T21:35:33.196 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:33.197 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:35:33.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80e006c8f0 msgr2=0x7f80e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80e006c8f0 0x7f80e006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f80e4033000 tx=0x7f80e402e000 comp rx=0 tx=0).stop 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f41f4b10 msgr2=0x7f80f40b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f41f4b10 0x7f80f40b4a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80e80119d0 tx=0x7f80e8011e90 comp rx=0 tx=0).stop 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 shutdown_connections 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80e006c8f0 0x7f80e006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40b4fa0 0x7f80f415f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f41f4b10 0x7f80f40b4a60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 --2- 172.21.15.40:0/4228734765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80dc0039e0 0x7f80f4156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 >> 172.21.15.40:0/4228734765 conn(0x7f80f40d4f60 msgr2=0x7f80f40d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 shutdown_connections 2024-04-26T21:35:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.194+0000 7f80f9f51640 1 -- 172.21.15.40:0/4228734765 wait complete. 2024-04-26T21:35:33.207 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-26T21:35:31.998856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219400,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755320,"statfs":{"total":773094113280,"available":772869447680,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003561"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004303+0000","last_change":"2024-04-26T21:35:09.132596+0000","last_active":"2024-04-26T21:35:16.004303+0000","last_peered":"2024-04-26T21:35:16.004303+0000","last_clean":"2024-04-26T21:35:16.004303+0000","last_became_active":"2024-04-26T21:35:09.132270+0000","last_became_peered":"2024-04-26T21:35:09.132270+0000","last_unstale":"2024-04-26T21:35:16.004303+0000","last_undegraded":"2024-04-26T21:35:16.004303+0000","last_fullsized":"2024-04-26T21:35:16.004303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:22.652595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003605+0000","last_change":"2024-04-26T21:35:10.126640+0000","last_active":"2024-04-26T21:35:16.003605+0000","last_peered":"2024-04-26T21:35:16.003605+0000","last_clean":"2024-04-26T21:35:16.003605+0000","last_became_active":"2024-04-26T21:35:10.125603+0000","last_became_peered":"2024-04-26T21:35:10.125603+0000","last_unstale":"2024-04-26T21:35:16.003605+0000","last_undegraded":"2024-04-26T21:35:16.003605+0000","last_fullsized":"2024-04-26T21:35:16.003605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:27.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004487+0000","last_change":"2024-04-26T21:35:14.280943+0000","last_active":"2024-04-26T21:35:16.004487+0000","last_peered":"2024-04-26T21:35:16.004487+0000","last_clean":"2024-04-26T21:35:16.004487+0000","last_became_active":"2024-04-26T21:35:14.280611+0000","last_became_peered":"2024-04-26T21:35:14.280611+0000","last_unstale":"2024-04-26T21:35:16.004487+0000","last_undegraded":"2024-04-26T21:35:16.004487+0000","last_fullsized":"2024-04-26T21:35:16.004487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:28.734397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004488+0000","last_change":"2024-04-26T21:35:09.122996+0000","last_active":"2024-04-26T21:35:16.004488+0000","last_peered":"2024-04-26T21:35:16.004488+0000","last_clean":"2024-04-26T21:35:16.004488+0000","last_became_active":"2024-04-26T21:35:09.122682+0000","last_became_peered":"2024-04-26T21:35:09.122682+0000","last_unstale":"2024-04-26T21:35:16.004488+0000","last_undegraded":"2024-04-26T21:35:16.004488+0000","last_fullsized":"2024-04-26T21:35:16.004488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:44.358668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004023+0000","last_change":"2024-04-26T21:35:10.133464+0000","last_active":"2024-04-26T21:35:16.004023+0000","last_peered":"2024-04-26T21:35:16.004023+0000","last_clean":"2024-04-26T21:35:16.004023+0000","last_became_active":"2024-04-26T21:35:10.133170+0000","last_became_peered":"2024-04-26T21:35:10.133170+0000","last_unstale":"2024-04-26T21:35:16.004023+0000","last_undegraded":"2024-04-26T21:35:16.004023+0000","last_fullsized":"2024-04-26T21:35:16.004023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003606+0000","last_change":"2024-04-26T21:35:14.163746+0000","last_active":"2024-04-26T21:35:16.003606+0000","last_peered":"2024-04-26T21:35:16.003606+0000","last_clean":"2024-04-26T21:35:16.003606+0000","last_became_active":"2024-04-26T21:35:14.163263+0000","last_became_peered":"2024-04-26T21:35:14.163263+0000","last_unstale":"2024-04-26T21:35:16.003606+0000","last_undegraded":"2024-04-26T21:35:16.003606+0000","last_fullsized":"2024-04-26T21:35:16.003606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:28.472320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875887+0000","last_change":"2024-04-26T21:35:09.129656+0000","last_active":"2024-04-26T21:35:15.875887+0000","last_peered":"2024-04-26T21:35:15.875887+0000","last_clean":"2024-04-26T21:35:15.875887+0000","last_became_active":"2024-04-26T21:35:09.129379+0000","last_became_peered":"2024-04-26T21:35:09.129379+0000","last_unstale":"2024-04-26T21:35:15.875887+0000","last_undegraded":"2024-04-26T21:35:15.875887+0000","last_fullsized":"2024-04-26T21:35:15.875887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:51.639722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143824+0000","last_change":"2024-04-26T21:35:10.119557+0000","last_active":"2024-04-26T21:35:15.143824+0000","last_peered":"2024-04-26T21:35:15.143824+0000","last_clean":"2024-04-26T21:35:15.143824+0000","last_became_active":"2024-04-26T21:35:10.118407+0000","last_became_peered":"2024-04-26T21:35:10.118407+0000","last_unstale":"2024-04-26T21:35:15.143824+0000","last_undegraded":"2024-04-26T21:35:15.143824+0000","last_fullsized":"2024-04-26T21:35:15.143824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003821+0000","last_change":"2024-04-26T21:35:14.159166+0000","last_active":"2024-04-26T21:35:16.003821+0000","last_peered":"2024-04-26T21:35:16.003821+0000","last_clean":"2024-04-26T21:35:16.003821+0000","last_became_active":"2024-04-26T21:35:14.158592+0000","last_became_peered":"2024-04-26T21:35:14.158592+0000","last_unstale":"2024-04-26T21:35:16.003821+0000","last_undegraded":"2024-04-26T21:35:16.003821+0000","last_fullsized":"2024-04-26T21:35:16.003821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:31.201343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875768+0000","last_change":"2024-04-26T21:35:09.128770+0000","last_active":"2024-04-26T21:35:15.875768+0000","last_peered":"2024-04-26T21:35:15.875768+0000","last_clean":"2024-04-26T21:35:15.875768+0000","last_became_active":"2024-04-26T21:35:09.128385+0000","last_became_peered":"2024-04-26T21:35:09.128385+0000","last_unstale":"2024-04-26T21:35:15.875768+0000","last_undegraded":"2024-04-26T21:35:15.875768+0000","last_fullsized":"2024-04-26T21:35:15.875768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:20.201305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144355+0000","last_change":"2024-04-26T21:35:10.117537+0000","last_active":"2024-04-26T21:35:15.144355+0000","last_peered":"2024-04-26T21:35:15.144355+0000","last_clean":"2024-04-26T21:35:15.144355+0000","last_became_active":"2024-04-26T21:35:10.117267+0000","last_became_peered":"2024-04-26T21:35:10.117267+0000","last_unstale":"2024-04-26T21:35:15.144355+0000","last_undegraded":"2024-04-26T21:35:15.144355+0000","last_fullsized":"2024-04-26T21:35:15.144355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:01.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003840+0000","last_change":"2024-04-26T21:35:14.161136+0000","last_active":"2024-04-26T21:35:16.003840+0000","last_peered":"2024-04-26T21:35:16.003840+0000","last_clean":"2024-04-26T21:35:16.003840+0000","last_became_active":"2024-04-26T21:35:14.160648+0000","last_became_peered":"2024-04-26T21:35:14.160648+0000","last_unstale":"2024-04-26T21:35:16.003840+0000","last_undegraded":"2024-04-26T21:35:16.003840+0000","last_fullsized":"2024-04-26T21:35:16.003840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:10.381659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143344+0000","last_change":"2024-04-26T21:35:10.119355+0000","last_active":"2024-04-26T21:35:15.143344+0000","last_peered":"2024-04-26T21:35:15.143344+0000","last_clean":"2024-04-26T21:35:15.143344+0000","last_became_active":"2024-04-26T21:35:10.119067+0000","last_became_peered":"2024-04-26T21:35:10.119067+0000","last_unstale":"2024-04-26T21:35:15.143344+0000","last_undegraded":"2024-04-26T21:35:15.143344+0000","last_fullsized":"2024-04-26T21:35:15.143344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876041+0000","last_change":"2024-04-26T21:35:09.129853+0000","last_active":"2024-04-26T21:35:15.876041+0000","last_peered":"2024-04-26T21:35:15.876041+0000","last_clean":"2024-04-26T21:35:15.876041+0000","last_became_active":"2024-04-26T21:35:09.129500+0000","last_became_peered":"2024-04-26T21:35:09.129500+0000","last_unstale":"2024-04-26T21:35:15.876041+0000","last_undegraded":"2024-04-26T21:35:15.876041+0000","last_fullsized":"2024-04-26T21:35:15.876041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:54.605573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004307+0000","last_change":"2024-04-26T21:35:14.161264+0000","last_active":"2024-04-26T21:35:16.004307+0000","last_peered":"2024-04-26T21:35:16.004307+0000","last_clean":"2024-04-26T21:35:16.004307+0000","last_became_active":"2024-04-26T21:35:14.160617+0000","last_became_peered":"2024-04-26T21:35:14.160617+0000","last_unstale":"2024-04-26T21:35:16.004307+0000","last_undegraded":"2024-04-26T21:35:16.004307+0000","last_fullsized":"2024-04-26T21:35:16.004307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004050+0000","last_change":"2024-04-26T21:35:10.125998+0000","last_active":"2024-04-26T21:35:16.004050+0000","last_peered":"2024-04-26T21:35:16.004050+0000","last_clean":"2024-04-26T21:35:16.004050+0000","last_became_active":"2024-04-26T21:35:10.125704+0000","last_became_peered":"2024-04-26T21:35:10.125704+0000","last_unstale":"2024-04-26T21:35:16.004050+0000","last_undegraded":"2024-04-26T21:35:16.004050+0000","last_fullsized":"2024-04-26T21:35:16.004050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373953+0000","last_change":"2024-04-26T21:35:09.123003+0000","last_active":"2024-04-26T21:35:15.373953+0000","last_peered":"2024-04-26T21:35:15.373953+0000","last_clean":"2024-04-26T21:35:15.373953+0000","last_became_active":"2024-04-26T21:35:09.122610+0000","last_became_peered":"2024-04-26T21:35:09.122610+0000","last_unstale":"2024-04-26T21:35:15.373953+0000","last_undegraded":"2024-04-26T21:35:15.373953+0000","last_fullsized":"2024-04-26T21:35:15.373953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004080+0000","last_change":"2024-04-26T21:35:14.164010+0000","last_active":"2024-04-26T21:35:16.004080+0000","last_peered":"2024-04-26T21:35:16.004080+0000","last_clean":"2024-04-26T21:35:16.004080+0000","last_became_active":"2024-04-26T21:35:14.163262+0000","last_became_peered":"2024-04-26T21:35:14.163262+0000","last_unstale":"2024-04-26T21:35:16.004080+0000","last_undegraded":"2024-04-26T21:35:16.004080+0000","last_fullsized":"2024-04-26T21:35:16.004080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:45.829307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003626+0000","last_change":"2024-04-26T21:35:10.120565+0000","last_active":"2024-04-26T21:35:16.003626+0000","last_peered":"2024-04-26T21:35:16.003626+0000","last_clean":"2024-04-26T21:35:16.003626+0000","last_became_active":"2024-04-26T21:35:10.120133+0000","last_became_peered":"2024-04-26T21:35:10.120133+0000","last_unstale":"2024-04-26T21:35:16.003626+0000","last_undegraded":"2024-04-26T21:35:16.003626+0000","last_fullsized":"2024-04-26T21:35:16.003626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:06.354261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373691+0000","last_change":"2024-04-26T21:35:09.127198+0000","last_active":"2024-04-26T21:35:15.373691+0000","last_peered":"2024-04-26T21:35:15.373691+0000","last_clean":"2024-04-26T21:35:15.373691+0000","last_became_active":"2024-04-26T21:35:09.126870+0000","last_became_peered":"2024-04-26T21:35:09.126870+0000","last_unstale":"2024-04-26T21:35:15.373691+0000","last_undegraded":"2024-04-26T21:35:15.373691+0000","last_fullsized":"2024-04-26T21:35:15.373691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374142+0000","last_change":"2024-04-26T21:35:14.277851+0000","last_active":"2024-04-26T21:35:15.374142+0000","last_peered":"2024-04-26T21:35:15.374142+0000","last_clean":"2024-04-26T21:35:15.374142+0000","last_became_active":"2024-04-26T21:35:14.277461+0000","last_became_peered":"2024-04-26T21:35:14.277461+0000","last_unstale":"2024-04-26T21:35:15.374142+0000","last_undegraded":"2024-04-26T21:35:15.374142+0000","last_fullsized":"2024-04-26T21:35:15.374142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:20.536176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004022+0000","last_change":"2024-04-26T21:35:10.113247+0000","last_active":"2024-04-26T21:35:16.004022+0000","last_peered":"2024-04-26T21:35:16.004022+0000","last_clean":"2024-04-26T21:35:16.004022+0000","last_became_active":"2024-04-26T21:35:10.112937+0000","last_became_peered":"2024-04-26T21:35:10.112937+0000","last_unstale":"2024-04-26T21:35:16.004022+0000","last_undegraded":"2024-04-26T21:35:16.004022+0000","last_fullsized":"2024-04-26T21:35:16.004022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:47.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875388+0000","last_change":"2024-04-26T21:35:09.134099+0000","last_active":"2024-04-26T21:35:15.875388+0000","last_peered":"2024-04-26T21:35:15.875388+0000","last_clean":"2024-04-26T21:35:15.875388+0000","last_became_active":"2024-04-26T21:35:09.133869+0000","last_became_peered":"2024-04-26T21:35:09.133869+0000","last_unstale":"2024-04-26T21:35:15.875388+0000","last_undegraded":"2024-04-26T21:35:15.875388+0000","last_fullsized":"2024-04-26T21:35:15.875388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:16.219471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004280+0000","last_change":"2024-04-26T21:35:14.158160+0000","last_active":"2024-04-26T21:35:16.004280+0000","last_peered":"2024-04-26T21:35:16.004280+0000","last_clean":"2024-04-26T21:35:16.004280+0000","last_became_active":"2024-04-26T21:35:14.157711+0000","last_became_peered":"2024-04-26T21:35:14.157711+0000","last_unstale":"2024-04-26T21:35:16.004280+0000","last_undegraded":"2024-04-26T21:35:16.004280+0000","last_fullsized":"2024-04-26T21:35:16.004280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:26.153246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004196+0000","last_change":"2024-04-26T21:35:10.136384+0000","last_active":"2024-04-26T21:35:16.004196+0000","last_peered":"2024-04-26T21:35:16.004196+0000","last_clean":"2024-04-26T21:35:16.004196+0000","last_became_active":"2024-04-26T21:35:10.135589+0000","last_became_peered":"2024-04-26T21:35:10.135589+0000","last_unstale":"2024-04-26T21:35:16.004196+0000","last_undegraded":"2024-04-26T21:35:16.004196+0000","last_fullsized":"2024-04-26T21:35:16.004196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144087+0000","last_change":"2024-04-26T21:35:09.125486+0000","last_active":"2024-04-26T21:35:15.144087+0000","last_peered":"2024-04-26T21:35:15.144087+0000","last_clean":"2024-04-26T21:35:15.144087+0000","last_became_active":"2024-04-26T21:35:09.125019+0000","last_became_peered":"2024-04-26T21:35:09.125019+0000","last_unstale":"2024-04-26T21:35:15.144087+0000","last_undegraded":"2024-04-26T21:35:15.144087+0000","last_fullsized":"2024-04-26T21:35:15.144087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003847+0000","last_change":"2024-04-26T21:35:14.281054+0000","last_active":"2024-04-26T21:35:16.003847+0000","last_peered":"2024-04-26T21:35:16.003847+0000","last_clean":"2024-04-26T21:35:16.003847+0000","last_became_active":"2024-04-26T21:35:14.278082+0000","last_became_peered":"2024-04-26T21:35:14.278082+0000","last_unstale":"2024-04-26T21:35:16.003847+0000","last_undegraded":"2024-04-26T21:35:16.003847+0000","last_fullsized":"2024-04-26T21:35:16.003847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.959963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004866+0000","last_change":"2024-04-26T21:35:10.130908+0000","last_active":"2024-04-26T21:35:16.004866+0000","last_peered":"2024-04-26T21:35:16.004866+0000","last_clean":"2024-04-26T21:35:16.004866+0000","last_became_active":"2024-04-26T21:35:10.130448+0000","last_became_peered":"2024-04-26T21:35:10.130448+0000","last_unstale":"2024-04-26T21:35:16.004866+0000","last_undegraded":"2024-04-26T21:35:16.004866+0000","last_fullsized":"2024-04-26T21:35:16.004866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374192+0000","last_change":"2024-04-26T21:35:09.124758+0000","last_active":"2024-04-26T21:35:15.374192+0000","last_peered":"2024-04-26T21:35:15.374192+0000","last_clean":"2024-04-26T21:35:15.374192+0000","last_became_active":"2024-04-26T21:35:09.124446+0000","last_became_peered":"2024-04-26T21:35:09.124446+0000","last_unstale":"2024-04-26T21:35:15.374192+0000","last_undegraded":"2024-04-26T21:35:15.374192+0000","last_fullsized":"2024-04-26T21:35:15.374192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:16.219471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003317+0000","last_change":"2024-04-26T21:35:14.280487+0000","last_active":"2024-04-26T21:35:16.003317+0000","last_peered":"2024-04-26T21:35:16.003317+0000","last_clean":"2024-04-26T21:35:16.003317+0000","last_became_active":"2024-04-26T21:35:14.280075+0000","last_became_peered":"2024-04-26T21:35:14.280075+0000","last_unstale":"2024-04-26T21:35:16.003317+0000","last_undegraded":"2024-04-26T21:35:16.003317+0000","last_fullsized":"2024-04-26T21:35:16.003317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:49.955325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373962+0000","last_change":"2024-04-26T21:35:10.124638+0000","last_active":"2024-04-26T21:35:15.373962+0000","last_peered":"2024-04-26T21:35:15.373962+0000","last_clean":"2024-04-26T21:35:15.373962+0000","last_became_active":"2024-04-26T21:35:10.124298+0000","last_became_peered":"2024-04-26T21:35:10.124298+0000","last_unstale":"2024-04-26T21:35:15.373962+0000","last_undegraded":"2024-04-26T21:35:15.373962+0000","last_fullsized":"2024-04-26T21:35:15.373962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876132+0000","last_change":"2024-04-26T21:35:09.134096+0000","last_active":"2024-04-26T21:35:15.876132+0000","last_peered":"2024-04-26T21:35:15.876132+0000","last_clean":"2024-04-26T21:35:15.876132+0000","last_became_active":"2024-04-26T21:35:09.133816+0000","last_became_peered":"2024-04-26T21:35:09.133816+0000","last_unstale":"2024-04-26T21:35:15.876132+0000","last_undegraded":"2024-04-26T21:35:15.876132+0000","last_fullsized":"2024-04-26T21:35:15.876132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:39.887861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004297+0000","last_change":"2024-04-26T21:35:14.277888+0000","last_active":"2024-04-26T21:35:16.004297+0000","last_peered":"2024-04-26T21:35:16.004297+0000","last_clean":"2024-04-26T21:35:16.004297+0000","last_became_active":"2024-04-26T21:35:14.277473+0000","last_became_peered":"2024-04-26T21:35:14.277473+0000","last_unstale":"2024-04-26T21:35:16.004297+0000","last_undegraded":"2024-04-26T21:35:16.004297+0000","last_fullsized":"2024-04-26T21:35:16.004297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:20.536176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144502+0000","last_change":"2024-04-26T21:35:10.118766+0000","last_active":"2024-04-26T21:35:15.144502+0000","last_peered":"2024-04-26T21:35:15.144502+0000","last_clean":"2024-04-26T21:35:15.144502+0000","last_became_active":"2024-04-26T21:35:10.118480+0000","last_became_peered":"2024-04-26T21:35:10.118480+0000","last_unstale":"2024-04-26T21:35:15.144502+0000","last_undegraded":"2024-04-26T21:35:15.144502+0000","last_fullsized":"2024-04-26T21:35:15.144502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:33.320854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374086+0000","last_change":"2024-04-26T21:35:09.124939+0000","last_active":"2024-04-26T21:35:15.374086+0000","last_peered":"2024-04-26T21:35:15.374086+0000","last_clean":"2024-04-26T21:35:15.374086+0000","last_became_active":"2024-04-26T21:35:09.124630+0000","last_became_peered":"2024-04-26T21:35:09.124630+0000","last_unstale":"2024-04-26T21:35:15.374086+0000","last_undegraded":"2024-04-26T21:35:15.374086+0000","last_fullsized":"2024-04-26T21:35:15.374086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374121+0000","last_change":"2024-04-26T21:35:14.165015+0000","last_active":"2024-04-26T21:35:15.374121+0000","last_peered":"2024-04-26T21:35:15.374121+0000","last_clean":"2024-04-26T21:35:15.374121+0000","last_became_active":"2024-04-26T21:35:14.164615+0000","last_became_peered":"2024-04-26T21:35:14.164615+0000","last_unstale":"2024-04-26T21:35:15.374121+0000","last_undegraded":"2024-04-26T21:35:15.374121+0000","last_fullsized":"2024-04-26T21:35:15.374121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:37.348252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004379+0000","last_change":"2024-04-26T21:35:10.125981+0000","last_active":"2024-04-26T21:35:16.004379+0000","last_peered":"2024-04-26T21:35:16.004379+0000","last_clean":"2024-04-26T21:35:16.004379+0000","last_became_active":"2024-04-26T21:35:10.125718+0000","last_became_peered":"2024-04-26T21:35:10.125718+0000","last_unstale":"2024-04-26T21:35:16.004379+0000","last_undegraded":"2024-04-26T21:35:16.004379+0000","last_fullsized":"2024-04-26T21:35:16.004379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:09.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144282+0000","last_change":"2024-04-26T21:35:09.121086+0000","last_active":"2024-04-26T21:35:15.144282+0000","last_peered":"2024-04-26T21:35:15.144282+0000","last_clean":"2024-04-26T21:35:15.144282+0000","last_became_active":"2024-04-26T21:35:09.120443+0000","last_became_peered":"2024-04-26T21:35:09.120443+0000","last_unstale":"2024-04-26T21:35:15.144282+0000","last_undegraded":"2024-04-26T21:35:15.144282+0000","last_fullsized":"2024-04-26T21:35:15.144282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374085+0000","last_change":"2024-04-26T21:35:14.163974+0000","last_active":"2024-04-26T21:35:15.374085+0000","last_peered":"2024-04-26T21:35:15.374085+0000","last_clean":"2024-04-26T21:35:15.374085+0000","last_became_active":"2024-04-26T21:35:14.163558+0000","last_became_peered":"2024-04-26T21:35:14.163558+0000","last_unstale":"2024-04-26T21:35:15.374085+0000","last_undegraded":"2024-04-26T21:35:15.374085+0000","last_fullsized":"2024-04-26T21:35:15.374085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:01.158030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373556+0000","last_change":"2024-04-26T21:35:10.124855+0000","last_active":"2024-04-26T21:35:15.373556+0000","last_peered":"2024-04-26T21:35:15.373556+0000","last_clean":"2024-04-26T21:35:15.373556+0000","last_became_active":"2024-04-26T21:35:10.124588+0000","last_became_peered":"2024-04-26T21:35:10.124588+0000","last_unstale":"2024-04-26T21:35:15.373556+0000","last_undegraded":"2024-04-26T21:35:15.373556+0000","last_fullsized":"2024-04-26T21:35:15.373556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142012+0000","last_change":"2024-04-26T21:35:09.115909+0000","last_active":"2024-04-26T21:35:15.142012+0000","last_peered":"2024-04-26T21:35:15.142012+0000","last_clean":"2024-04-26T21:35:15.142012+0000","last_became_active":"2024-04-26T21:35:09.115120+0000","last_became_peered":"2024-04-26T21:35:09.115120+0000","last_unstale":"2024-04-26T21:35:15.142012+0000","last_undegraded":"2024-04-26T21:35:15.142012+0000","last_fullsized":"2024-04-26T21:35:15.142012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374302+0000","last_change":"2024-04-26T21:35:14.165160+0000","last_active":"2024-04-26T21:35:15.374302+0000","last_peered":"2024-04-26T21:35:15.374302+0000","last_clean":"2024-04-26T21:35:15.374302+0000","last_became_active":"2024-04-26T21:35:14.164683+0000","last_became_peered":"2024-04-26T21:35:14.164683+0000","last_unstale":"2024-04-26T21:35:15.374302+0000","last_undegraded":"2024-04-26T21:35:15.374302+0000","last_fullsized":"2024-04-26T21:35:15.374302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:37.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003512+0000","last_change":"2024-04-26T21:35:10.128846+0000","last_active":"2024-04-26T21:35:16.003512+0000","last_peered":"2024-04-26T21:35:16.003512+0000","last_clean":"2024-04-26T21:35:16.003512+0000","last_became_active":"2024-04-26T21:35:10.128576+0000","last_became_peered":"2024-04-26T21:35:10.128576+0000","last_unstale":"2024-04-26T21:35:16.003512+0000","last_undegraded":"2024-04-26T21:35:16.003512+0000","last_fullsized":"2024-04-26T21:35:16.003512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:40.892489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875388+0000","last_change":"2024-04-26T21:35:09.130374+0000","last_active":"2024-04-26T21:35:15.875388+0000","last_peered":"2024-04-26T21:35:15.875388+0000","last_clean":"2024-04-26T21:35:15.875388+0000","last_became_active":"2024-04-26T21:35:09.130122+0000","last_became_peered":"2024-04-26T21:35:09.130122+0000","last_unstale":"2024-04-26T21:35:15.875388+0000","last_undegraded":"2024-04-26T21:35:15.875388+0000","last_fullsized":"2024-04-26T21:35:15.875388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:25.035511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004078+0000","last_change":"2024-04-26T21:35:14.277956+0000","last_active":"2024-04-26T21:35:16.004078+0000","last_peered":"2024-04-26T21:35:16.004078+0000","last_clean":"2024-04-26T21:35:16.004078+0000","last_became_active":"2024-04-26T21:35:14.277555+0000","last_became_peered":"2024-04-26T21:35:14.277555+0000","last_unstale":"2024-04-26T21:35:16.004078+0000","last_undegraded":"2024-04-26T21:35:16.004078+0000","last_fullsized":"2024-04-26T21:35:16.004078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:37.348252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003728+0000","last_change":"2024-04-26T21:35:10.127546+0000","last_active":"2024-04-26T21:35:16.003728+0000","last_peered":"2024-04-26T21:35:16.003728+0000","last_clean":"2024-04-26T21:35:16.003728+0000","last_became_active":"2024-04-26T21:35:10.127263+0000","last_became_peered":"2024-04-26T21:35:10.127263+0000","last_unstale":"2024-04-26T21:35:16.003728+0000","last_undegraded":"2024-04-26T21:35:16.003728+0000","last_fullsized":"2024-04-26T21:35:16.003728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144006+0000","last_change":"2024-04-26T21:35:09.116290+0000","last_active":"2024-04-26T21:35:15.144006+0000","last_peered":"2024-04-26T21:35:15.144006+0000","last_clean":"2024-04-26T21:35:15.144006+0000","last_became_active":"2024-04-26T21:35:09.115972+0000","last_became_peered":"2024-04-26T21:35:09.115972+0000","last_unstale":"2024-04-26T21:35:15.144006+0000","last_undegraded":"2024-04-26T21:35:15.144006+0000","last_fullsized":"2024-04-26T21:35:15.144006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875528+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:35:15.875528+0000","last_peered":"2024-04-26T21:35:15.875528+0000","last_clean":"2024-04-26T21:35:15.875528+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:35:15.875528+0000","last_undegraded":"2024-04-26T21:35:15.875528+0000","last_fullsized":"2024-04-26T21:35:15.875528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143208+0000","last_change":"2024-04-26T21:35:14.279050+0000","last_active":"2024-04-26T21:35:15.143208+0000","last_peered":"2024-04-26T21:35:15.143208+0000","last_clean":"2024-04-26T21:35:15.143208+0000","last_became_active":"2024-04-26T21:35:14.278651+0000","last_became_peered":"2024-04-26T21:35:14.278651+0000","last_unstale":"2024-04-26T21:35:15.143208+0000","last_undegraded":"2024-04-26T21:35:15.143208+0000","last_fullsized":"2024-04-26T21:35:15.143208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:26.153246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875749+0000","last_change":"2024-04-26T21:35:10.130945+0000","last_active":"2024-04-26T21:35:15.875749+0000","last_peered":"2024-04-26T21:35:15.875749+0000","last_clean":"2024-04-26T21:35:15.875749+0000","last_became_active":"2024-04-26T21:35:10.130673+0000","last_became_peered":"2024-04-26T21:35:10.130673+0000","last_unstale":"2024-04-26T21:35:15.875749+0000","last_undegraded":"2024-04-26T21:35:15.875749+0000","last_fullsized":"2024-04-26T21:35:15.875749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:16.508778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142347+0000","last_change":"2024-04-26T21:35:09.118997+0000","last_active":"2024-04-26T21:35:15.142347+0000","last_peered":"2024-04-26T21:35:15.142347+0000","last_clean":"2024-04-26T21:35:15.142347+0000","last_became_active":"2024-04-26T21:35:09.118700+0000","last_became_peered":"2024-04-26T21:35:09.118700+0000","last_unstale":"2024-04-26T21:35:15.142347+0000","last_undegraded":"2024-04-26T21:35:15.142347+0000","last_fullsized":"2024-04-26T21:35:15.142347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143286+0000","last_change":"2024-04-26T21:35:14.277928+0000","last_active":"2024-04-26T21:35:15.143286+0000","last_peered":"2024-04-26T21:35:15.143286+0000","last_clean":"2024-04-26T21:35:15.143286+0000","last_became_active":"2024-04-26T21:35:14.277458+0000","last_became_peered":"2024-04-26T21:35:14.277458+0000","last_unstale":"2024-04-26T21:35:15.143286+0000","last_undegraded":"2024-04-26T21:35:15.143286+0000","last_fullsized":"2024-04-26T21:35:15.143286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:55.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144138+0000","last_change":"2024-04-26T21:35:10.120150+0000","last_active":"2024-04-26T21:35:15.144138+0000","last_peered":"2024-04-26T21:35:15.144138+0000","last_clean":"2024-04-26T21:35:15.144138+0000","last_became_active":"2024-04-26T21:35:10.119894+0000","last_became_peered":"2024-04-26T21:35:10.119894+0000","last_unstale":"2024-04-26T21:35:15.144138+0000","last_undegraded":"2024-04-26T21:35:15.144138+0000","last_fullsized":"2024-04-26T21:35:15.144138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.413835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374128+0000","last_change":"2024-04-26T21:35:09.125791+0000","last_active":"2024-04-26T21:35:15.374128+0000","last_peered":"2024-04-26T21:35:15.374128+0000","last_clean":"2024-04-26T21:35:15.374128+0000","last_became_active":"2024-04-26T21:35:09.125356+0000","last_became_peered":"2024-04-26T21:35:09.125356+0000","last_unstale":"2024-04-26T21:35:15.374128+0000","last_undegraded":"2024-04-26T21:35:15.374128+0000","last_fullsized":"2024-04-26T21:35:15.374128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:25.035511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004470+0000","last_change":"2024-04-26T21:35:14.160611+0000","last_active":"2024-04-26T21:35:16.004470+0000","last_peered":"2024-04-26T21:35:16.004470+0000","last_clean":"2024-04-26T21:35:16.004470+0000","last_became_active":"2024-04-26T21:35:14.160067+0000","last_became_peered":"2024-04-26T21:35:14.160067+0000","last_unstale":"2024-04-26T21:35:16.004470+0000","last_undegraded":"2024-04-26T21:35:16.004470+0000","last_fullsized":"2024-04-26T21:35:16.004470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:37.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876144+0000","last_change":"2024-04-26T21:35:10.136587+0000","last_active":"2024-04-26T21:35:15.876144+0000","last_peered":"2024-04-26T21:35:15.876144+0000","last_clean":"2024-04-26T21:35:15.876144+0000","last_became_active":"2024-04-26T21:35:10.136271+0000","last_became_peered":"2024-04-26T21:35:10.136271+0000","last_unstale":"2024-04-26T21:35:15.876144+0000","last_undegraded":"2024-04-26T21:35:15.876144+0000","last_fullsized":"2024-04-26T21:35:15.876144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:45.927928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142500+0000","last_change":"2024-04-26T21:35:09.115543+0000","last_active":"2024-04-26T21:35:15.142500+0000","last_peered":"2024-04-26T21:35:15.142500+0000","last_clean":"2024-04-26T21:35:15.142500+0000","last_became_active":"2024-04-26T21:35:09.115159+0000","last_became_peered":"2024-04-26T21:35:09.115159+0000","last_unstale":"2024-04-26T21:35:15.142500+0000","last_undegraded":"2024-04-26T21:35:15.142500+0000","last_fullsized":"2024-04-26T21:35:15.142500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142639+0000","last_change":"2024-04-26T21:35:14.278004+0000","last_active":"2024-04-26T21:35:15.142639+0000","last_peered":"2024-04-26T21:35:15.142639+0000","last_clean":"2024-04-26T21:35:15.142639+0000","last_became_active":"2024-04-26T21:35:14.277547+0000","last_became_peered":"2024-04-26T21:35:14.277547+0000","last_unstale":"2024-04-26T21:35:15.142639+0000","last_undegraded":"2024-04-26T21:35:15.142639+0000","last_fullsized":"2024-04-26T21:35:15.142639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:49.955325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875902+0000","last_change":"2024-04-26T21:35:10.131439+0000","last_active":"2024-04-26T21:35:15.875902+0000","last_peered":"2024-04-26T21:35:15.875902+0000","last_clean":"2024-04-26T21:35:15.875902+0000","last_became_active":"2024-04-26T21:35:10.130864+0000","last_became_peered":"2024-04-26T21:35:10.130864+0000","last_unstale":"2024-04-26T21:35:15.875902+0000","last_undegraded":"2024-04-26T21:35:15.875902+0000","last_fullsized":"2024-04-26T21:35:15.875902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.130632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004324+0000","last_change":"2024-04-26T21:35:09.123483+0000","last_active":"2024-04-26T21:35:16.004324+0000","last_peered":"2024-04-26T21:35:16.004324+0000","last_clean":"2024-04-26T21:35:16.004324+0000","last_became_active":"2024-04-26T21:35:09.123202+0000","last_became_peered":"2024-04-26T21:35:09.123202+0000","last_unstale":"2024-04-26T21:35:16.004324+0000","last_undegraded":"2024-04-26T21:35:16.004324+0000","last_fullsized":"2024-04-26T21:35:16.004324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:50.000306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003596+0000","last_change":"2024-04-26T21:35:14.163479+0000","last_active":"2024-04-26T21:35:16.003596+0000","last_peered":"2024-04-26T21:35:16.003596+0000","last_clean":"2024-04-26T21:35:16.003596+0000","last_became_active":"2024-04-26T21:35:14.163041+0000","last_became_peered":"2024-04-26T21:35:14.163041+0000","last_unstale":"2024-04-26T21:35:16.003596+0000","last_undegraded":"2024-04-26T21:35:16.003596+0000","last_fullsized":"2024-04-26T21:35:16.003596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:34.682922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875665+0000","last_change":"2024-04-26T21:35:10.129858+0000","last_active":"2024-04-26T21:35:15.875665+0000","last_peered":"2024-04-26T21:35:15.875665+0000","last_clean":"2024-04-26T21:35:15.875665+0000","last_became_active":"2024-04-26T21:35:10.129606+0000","last_became_peered":"2024-04-26T21:35:10.129606+0000","last_unstale":"2024-04-26T21:35:15.875665+0000","last_undegraded":"2024-04-26T21:35:15.875665+0000","last_fullsized":"2024-04-26T21:35:15.875665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.413835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.141883+0000","last_change":"2024-04-26T21:35:09.121200+0000","last_active":"2024-04-26T21:35:15.141883+0000","last_peered":"2024-04-26T21:35:15.141883+0000","last_clean":"2024-04-26T21:35:15.141883+0000","last_became_active":"2024-04-26T21:35:09.120700+0000","last_became_peered":"2024-04-26T21:35:09.120700+0000","last_unstale":"2024-04-26T21:35:15.141883+0000","last_undegraded":"2024-04-26T21:35:15.141883+0000","last_fullsized":"2024-04-26T21:35:15.141883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003924+0000","last_change":"2024-04-26T21:35:14.163658+0000","last_active":"2024-04-26T21:35:16.003924+0000","last_peered":"2024-04-26T21:35:16.003924+0000","last_clean":"2024-04-26T21:35:16.003924+0000","last_became_active":"2024-04-26T21:35:14.163215+0000","last_became_peered":"2024-04-26T21:35:14.163215+0000","last_unstale":"2024-04-26T21:35:16.003924+0000","last_undegraded":"2024-04-26T21:35:16.003924+0000","last_fullsized":"2024-04-26T21:35:16.003924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:01.158030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142807+0000","last_change":"2024-04-26T21:35:10.126750+0000","last_active":"2024-04-26T21:35:15.142807+0000","last_peered":"2024-04-26T21:35:15.142807+0000","last_clean":"2024-04-26T21:35:15.142807+0000","last_became_active":"2024-04-26T21:35:10.126096+0000","last_became_peered":"2024-04-26T21:35:10.126096+0000","last_unstale":"2024-04-26T21:35:15.142807+0000","last_undegraded":"2024-04-26T21:35:15.142807+0000","last_fullsized":"2024-04-26T21:35:15.142807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:46.899885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143301+0000","last_change":"2024-04-26T21:35:09.122401+0000","last_active":"2024-04-26T21:35:15.143301+0000","last_peered":"2024-04-26T21:35:15.143301+0000","last_clean":"2024-04-26T21:35:15.143301+0000","last_became_active":"2024-04-26T21:35:09.122067+0000","last_became_peered":"2024-04-26T21:35:09.122067+0000","last_unstale":"2024-04-26T21:35:15.143301+0000","last_undegraded":"2024-04-26T21:35:15.143301+0000","last_fullsized":"2024-04-26T21:35:15.143301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004812+0000","last_change":"2024-04-26T21:35:14.279383+0000","last_active":"2024-04-26T21:35:16.004812+0000","last_peered":"2024-04-26T21:35:16.004812+0000","last_clean":"2024-04-26T21:35:16.004812+0000","last_became_active":"2024-04-26T21:35:14.278985+0000","last_became_peered":"2024-04-26T21:35:14.278985+0000","last_unstale":"2024-04-26T21:35:16.004812+0000","last_undegraded":"2024-04-26T21:35:16.004812+0000","last_fullsized":"2024-04-26T21:35:16.004812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.757335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875771+0000","last_change":"2024-04-26T21:35:10.136295+0000","last_active":"2024-04-26T21:35:15.875771+0000","last_peered":"2024-04-26T21:35:15.875771+0000","last_clean":"2024-04-26T21:35:15.875771+0000","last_became_active":"2024-04-26T21:35:10.135975+0000","last_became_peered":"2024-04-26T21:35:10.135975+0000","last_unstale":"2024-04-26T21:35:15.875771+0000","last_undegraded":"2024-04-26T21:35:15.875771+0000","last_fullsized":"2024-04-26T21:35:15.875771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144056+0000","last_change":"2024-04-26T21:35:09.122493+0000","last_active":"2024-04-26T21:35:15.144056+0000","last_peered":"2024-04-26T21:35:15.144056+0000","last_clean":"2024-04-26T21:35:15.144056+0000","last_became_active":"2024-04-26T21:35:09.122211+0000","last_became_peered":"2024-04-26T21:35:09.122211+0000","last_unstale":"2024-04-26T21:35:15.144056+0000","last_undegraded":"2024-04-26T21:35:15.144056+0000","last_fullsized":"2024-04-26T21:35:15.144056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004937+0000","last_change":"2024-04-26T21:35:14.277801+0000","last_active":"2024-04-26T21:35:16.004937+0000","last_peered":"2024-04-26T21:35:16.004937+0000","last_clean":"2024-04-26T21:35:16.004937+0000","last_became_active":"2024-04-26T21:35:14.277417+0000","last_became_peered":"2024-04-26T21:35:14.277417+0000","last_unstale":"2024-04-26T21:35:16.004937+0000","last_undegraded":"2024-04-26T21:35:16.004937+0000","last_fullsized":"2024-04-26T21:35:16.004937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:15.246664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004133+0000","last_change":"2024-04-26T21:35:10.128386+0000","last_active":"2024-04-26T21:35:16.004133+0000","last_peered":"2024-04-26T21:35:16.004133+0000","last_clean":"2024-04-26T21:35:16.004133+0000","last_became_active":"2024-04-26T21:35:10.127067+0000","last_became_peered":"2024-04-26T21:35:10.127067+0000","last_unstale":"2024-04-26T21:35:16.004133+0000","last_undegraded":"2024-04-26T21:35:16.004133+0000","last_fullsized":"2024-04-26T21:35:16.004133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:46.899885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004551+0000","last_change":"2024-04-26T21:35:09.125422+0000","last_active":"2024-04-26T21:35:16.004551+0000","last_peered":"2024-04-26T21:35:16.004551+0000","last_clean":"2024-04-26T21:35:16.004551+0000","last_became_active":"2024-04-26T21:35:09.125086+0000","last_became_peered":"2024-04-26T21:35:09.125086+0000","last_unstale":"2024-04-26T21:35:16.004551+0000","last_undegraded":"2024-04-26T21:35:16.004551+0000","last_fullsized":"2024-04-26T21:35:16.004551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:44.358668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144502+0000","last_change":"2024-04-26T21:35:14.278312+0000","last_active":"2024-04-26T21:35:15.144502+0000","last_peered":"2024-04-26T21:35:15.144502+0000","last_clean":"2024-04-26T21:35:15.144502+0000","last_became_active":"2024-04-26T21:35:14.277836+0000","last_became_peered":"2024-04-26T21:35:14.277836+0000","last_unstale":"2024-04-26T21:35:15.144502+0000","last_undegraded":"2024-04-26T21:35:15.144502+0000","last_fullsized":"2024-04-26T21:35:15.144502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:20.215438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875616+0000","last_change":"2024-04-26T21:35:10.126791+0000","last_active":"2024-04-26T21:35:15.875616+0000","last_peered":"2024-04-26T21:35:15.875616+0000","last_clean":"2024-04-26T21:35:15.875616+0000","last_became_active":"2024-04-26T21:35:10.126515+0000","last_became_peered":"2024-04-26T21:35:10.126515+0000","last_unstale":"2024-04-26T21:35:15.875616+0000","last_undegraded":"2024-04-26T21:35:15.875616+0000","last_fullsized":"2024-04-26T21:35:15.875616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:01.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:19.559960+0000","last_change":"2024-04-26T21:35:09.126680+0000","last_active":"2024-04-26T21:35:19.559960+0000","last_peered":"2024-04-26T21:35:19.559960+0000","last_clean":"2024-04-26T21:35:19.559960+0000","last_became_active":"2024-04-26T21:35:09.126406+0000","last_became_peered":"2024-04-26T21:35:09.126406+0000","last_unstale":"2024-04-26T21:35:19.559960+0000","last_undegraded":"2024-04-26T21:35:19.559960+0000","last_fullsized":"2024-04-26T21:35:19.559960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373762+0000","last_change":"2024-04-26T21:35:14.163960+0000","last_active":"2024-04-26T21:35:15.373762+0000","last_peered":"2024-04-26T21:35:15.373762+0000","last_clean":"2024-04-26T21:35:15.373762+0000","last_became_active":"2024-04-26T21:35:14.163551+0000","last_became_peered":"2024-04-26T21:35:14.163551+0000","last_unstale":"2024-04-26T21:35:15.373762+0000","last_undegraded":"2024-04-26T21:35:15.373762+0000","last_fullsized":"2024-04-26T21:35:15.373762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004282+0000","last_change":"2024-04-26T21:35:10.129151+0000","last_active":"2024-04-26T21:35:16.004282+0000","last_peered":"2024-04-26T21:35:16.004282+0000","last_clean":"2024-04-26T21:35:16.004282+0000","last_became_active":"2024-04-26T21:35:10.128520+0000","last_became_peered":"2024-04-26T21:35:10.128520+0000","last_unstale":"2024-04-26T21:35:16.004282+0000","last_undegraded":"2024-04-26T21:35:16.004282+0000","last_fullsized":"2024-04-26T21:35:16.004282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:17.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374063+0000","last_change":"2024-04-26T21:35:09.128170+0000","last_active":"2024-04-26T21:35:15.374063+0000","last_peered":"2024-04-26T21:35:15.374063+0000","last_clean":"2024-04-26T21:35:15.374063+0000","last_became_active":"2024-04-26T21:35:09.127817+0000","last_became_peered":"2024-04-26T21:35:09.127817+0000","last_unstale":"2024-04-26T21:35:15.374063+0000","last_undegraded":"2024-04-26T21:35:15.374063+0000","last_fullsized":"2024-04-26T21:35:15.374063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004297+0000","last_change":"2024-04-26T21:35:14.157290+0000","last_active":"2024-04-26T21:35:16.004297+0000","last_peered":"2024-04-26T21:35:16.004297+0000","last_clean":"2024-04-26T21:35:16.004297+0000","last_became_active":"2024-04-26T21:35:14.156794+0000","last_became_peered":"2024-04-26T21:35:14.156794+0000","last_unstale":"2024-04-26T21:35:16.004297+0000","last_undegraded":"2024-04-26T21:35:16.004297+0000","last_fullsized":"2024-04-26T21:35:16.004297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:55.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004165+0000","last_change":"2024-04-26T21:35:10.132571+0000","last_active":"2024-04-26T21:35:16.004165+0000","last_peered":"2024-04-26T21:35:16.004165+0000","last_clean":"2024-04-26T21:35:16.004165+0000","last_became_active":"2024-04-26T21:35:10.132311+0000","last_became_peered":"2024-04-26T21:35:10.132311+0000","last_unstale":"2024-04-26T21:35:16.004165+0000","last_undegraded":"2024-04-26T21:35:16.004165+0000","last_fullsized":"2024-04-26T21:35:16.004165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143720+0000","last_change":"2024-04-26T21:35:09.124883+0000","last_active":"2024-04-26T21:35:15.143720+0000","last_peered":"2024-04-26T21:35:15.143720+0000","last_clean":"2024-04-26T21:35:15.143720+0000","last_became_active":"2024-04-26T21:35:09.124576+0000","last_became_peered":"2024-04-26T21:35:09.124576+0000","last_unstale":"2024-04-26T21:35:15.143720+0000","last_undegraded":"2024-04-26T21:35:15.143720+0000","last_fullsized":"2024-04-26T21:35:15.143720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003494+0000","last_change":"2024-04-26T21:35:14.158063+0000","last_active":"2024-04-26T21:35:16.003494+0000","last_peered":"2024-04-26T21:35:16.003494+0000","last_clean":"2024-04-26T21:35:16.003494+0000","last_became_active":"2024-04-26T21:35:14.157033+0000","last_became_peered":"2024-04-26T21:35:14.157033+0000","last_unstale":"2024-04-26T21:35:16.003494+0000","last_undegraded":"2024-04-26T21:35:16.003494+0000","last_fullsized":"2024-04-26T21:35:16.003494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:20.215438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143604+0000","last_change":"2024-04-26T21:35:10.121823+0000","last_active":"2024-04-26T21:35:15.143604+0000","last_peered":"2024-04-26T21:35:15.143604+0000","last_clean":"2024-04-26T21:35:15.143604+0000","last_became_active":"2024-04-26T21:35:10.121518+0000","last_became_peered":"2024-04-26T21:35:10.121518+0000","last_unstale":"2024-04-26T21:35:15.143604+0000","last_undegraded":"2024-04-26T21:35:15.143604+0000","last_fullsized":"2024-04-26T21:35:15.143604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:06.354261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143861+0000","last_change":"2024-04-26T21:35:09.127420+0000","last_active":"2024-04-26T21:35:15.143861+0000","last_peered":"2024-04-26T21:35:15.143861+0000","last_clean":"2024-04-26T21:35:15.143861+0000","last_became_active":"2024-04-26T21:35:09.127104+0000","last_became_peered":"2024-04-26T21:35:09.127104+0000","last_unstale":"2024-04-26T21:35:15.143861+0000","last_undegraded":"2024-04-26T21:35:15.143861+0000","last_fullsized":"2024-04-26T21:35:15.143861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004551+0000","last_change":"2024-04-26T21:35:14.281132+0000","last_active":"2024-04-26T21:35:16.004551+0000","last_peered":"2024-04-26T21:35:16.004551+0000","last_clean":"2024-04-26T21:35:16.004551+0000","last_became_active":"2024-04-26T21:35:14.280538+0000","last_became_peered":"2024-04-26T21:35:14.280538+0000","last_unstale":"2024-04-26T21:35:16.004551+0000","last_undegraded":"2024-04-26T21:35:16.004551+0000","last_fullsized":"2024-04-26T21:35:16.004551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:37.157683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003564+0000","last_change":"2024-04-26T21:35:14.279953+0000","last_active":"2024-04-26T21:35:16.003564+0000","last_peered":"2024-04-26T21:35:16.003564+0000","last_clean":"2024-04-26T21:35:16.003564+0000","last_became_active":"2024-04-26T21:35:14.279540+0000","last_became_peered":"2024-04-26T21:35:14.279540+0000","last_unstale":"2024-04-26T21:35:16.003564+0000","last_undegraded":"2024-04-26T21:35:16.003564+0000","last_fullsized":"2024-04-26T21:35:16.003564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373464+0000","last_change":"2024-04-26T21:35:10.121201+0000","last_active":"2024-04-26T21:35:15.373464+0000","last_peered":"2024-04-26T21:35:15.373464+0000","last_clean":"2024-04-26T21:35:15.373464+0000","last_became_active":"2024-04-26T21:35:10.120934+0000","last_became_peered":"2024-04-26T21:35:10.120934+0000","last_unstale":"2024-04-26T21:35:15.373464+0000","last_undegraded":"2024-04-26T21:35:15.373464+0000","last_fullsized":"2024-04-26T21:35:15.373464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142787+0000","last_change":"2024-04-26T21:35:09.119199+0000","last_active":"2024-04-26T21:35:15.142787+0000","last_peered":"2024-04-26T21:35:15.142787+0000","last_clean":"2024-04-26T21:35:15.142787+0000","last_became_active":"2024-04-26T21:35:09.118431+0000","last_became_peered":"2024-04-26T21:35:09.118431+0000","last_unstale":"2024-04-26T21:35:15.142787+0000","last_undegraded":"2024-04-26T21:35:15.142787+0000","last_fullsized":"2024-04-26T21:35:15.142787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144043+0000","last_change":"2024-04-26T21:35:14.160976+0000","last_active":"2024-04-26T21:35:15.144043+0000","last_peered":"2024-04-26T21:35:15.144043+0000","last_clean":"2024-04-26T21:35:15.144043+0000","last_became_active":"2024-04-26T21:35:14.160609+0000","last_became_peered":"2024-04-26T21:35:14.160609+0000","last_unstale":"2024-04-26T21:35:15.144043+0000","last_undegraded":"2024-04-26T21:35:15.144043+0000","last_fullsized":"2024-04-26T21:35:15.144043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003542+0000","last_change":"2024-04-26T21:35:10.134441+0000","last_active":"2024-04-26T21:35:16.003542+0000","last_peered":"2024-04-26T21:35:16.003542+0000","last_clean":"2024-04-26T21:35:16.003542+0000","last_became_active":"2024-04-26T21:35:10.134101+0000","last_became_peered":"2024-04-26T21:35:10.134101+0000","last_unstale":"2024-04-26T21:35:16.003542+0000","last_undegraded":"2024-04-26T21:35:16.003542+0000","last_fullsized":"2024-04-26T21:35:16.003542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004713+0000","last_change":"2024-04-26T21:35:09.124939+0000","last_active":"2024-04-26T21:35:16.004713+0000","last_peered":"2024-04-26T21:35:16.004713+0000","last_clean":"2024-04-26T21:35:16.004713+0000","last_became_active":"2024-04-26T21:35:09.124612+0000","last_became_peered":"2024-04-26T21:35:09.124612+0000","last_unstale":"2024-04-26T21:35:16.004713+0000","last_undegraded":"2024-04-26T21:35:16.004713+0000","last_fullsized":"2024-04-26T21:35:16.004713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143484+0000","last_change":"2024-04-26T21:35:09.122510+0000","last_active":"2024-04-26T21:35:15.143484+0000","last_peered":"2024-04-26T21:35:15.143484+0000","last_clean":"2024-04-26T21:35:15.143484+0000","last_became_active":"2024-04-26T21:35:09.122161+0000","last_became_peered":"2024-04-26T21:35:09.122161+0000","last_unstale":"2024-04-26T21:35:15.143484+0000","last_undegraded":"2024-04-26T21:35:15.143484+0000","last_fullsized":"2024-04-26T21:35:15.143484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004522+0000","last_change":"2024-04-26T21:35:10.128884+0000","last_active":"2024-04-26T21:35:16.004522+0000","last_peered":"2024-04-26T21:35:16.004522+0000","last_clean":"2024-04-26T21:35:16.004522+0000","last_became_active":"2024-04-26T21:35:10.128599+0000","last_became_peered":"2024-04-26T21:35:10.128599+0000","last_unstale":"2024-04-26T21:35:16.004522+0000","last_undegraded":"2024-04-26T21:35:16.004522+0000","last_fullsized":"2024-04-26T21:35:16.004522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142294+0000","last_change":"2024-04-26T21:35:14.158429+0000","last_active":"2024-04-26T21:35:15.142294+0000","last_peered":"2024-04-26T21:35:15.142294+0000","last_clean":"2024-04-26T21:35:15.142294+0000","last_became_active":"2024-04-26T21:35:14.158011+0000","last_became_peered":"2024-04-26T21:35:14.158011+0000","last_unstale":"2024-04-26T21:35:15.142294+0000","last_undegraded":"2024-04-26T21:35:15.142294+0000","last_fullsized":"2024-04-26T21:35:15.142294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:44.919887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004133+0000","last_change":"2024-04-26T21:35:09.127502+0000","last_active":"2024-04-26T21:35:16.004133+0000","last_peered":"2024-04-26T21:35:16.004133+0000","last_clean":"2024-04-26T21:35:16.004133+0000","last_became_active":"2024-04-26T21:35:09.127182+0000","last_became_peered":"2024-04-26T21:35:09.127182+0000","last_unstale":"2024-04-26T21:35:16.004133+0000","last_undegraded":"2024-04-26T21:35:16.004133+0000","last_fullsized":"2024-04-26T21:35:16.004133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003916+0000","last_change":"2024-04-26T21:35:10.120022+0000","last_active":"2024-04-26T21:35:16.003916+0000","last_peered":"2024-04-26T21:35:16.003916+0000","last_clean":"2024-04-26T21:35:16.003916+0000","last_became_active":"2024-04-26T21:35:10.119759+0000","last_became_peered":"2024-04-26T21:35:10.119759+0000","last_unstale":"2024-04-26T21:35:16.003916+0000","last_undegraded":"2024-04-26T21:35:16.003916+0000","last_fullsized":"2024-04-26T21:35:16.003916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:55.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004467+0000","last_change":"2024-04-26T21:35:14.157203+0000","last_active":"2024-04-26T21:35:16.004467+0000","last_peered":"2024-04-26T21:35:16.004467+0000","last_clean":"2024-04-26T21:35:16.004467+0000","last_became_active":"2024-04-26T21:35:14.156742+0000","last_became_peered":"2024-04-26T21:35:14.156742+0000","last_unstale":"2024-04-26T21:35:16.004467+0000","last_undegraded":"2024-04-26T21:35:16.004467+0000","last_fullsized":"2024-04-26T21:35:16.004467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:13.631171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142673+0000","last_change":"2024-04-26T21:35:10.126924+0000","last_active":"2024-04-26T21:35:15.142673+0000","last_peered":"2024-04-26T21:35:15.142673+0000","last_clean":"2024-04-26T21:35:15.142673+0000","last_became_active":"2024-04-26T21:35:10.126660+0000","last_became_peered":"2024-04-26T21:35:10.126660+0000","last_unstale":"2024-04-26T21:35:15.142673+0000","last_undegraded":"2024-04-26T21:35:15.142673+0000","last_fullsized":"2024-04-26T21:35:15.142673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:29.764634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142006+0000","last_change":"2024-04-26T21:35:10.116329+0000","last_active":"2024-04-26T21:35:15.142006+0000","last_peered":"2024-04-26T21:35:15.142006+0000","last_clean":"2024-04-26T21:35:15.142006+0000","last_became_active":"2024-04-26T21:35:10.116047+0000","last_became_peered":"2024-04-26T21:35:10.116047+0000","last_unstale":"2024-04-26T21:35:15.142006+0000","last_undegraded":"2024-04-26T21:35:15.142006+0000","last_fullsized":"2024-04-26T21:35:15.142006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:17.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143917+0000","last_change":"2024-04-26T21:35:10.121375+0000","last_active":"2024-04-26T21:35:15.143917+0000","last_peered":"2024-04-26T21:35:15.143917+0000","last_clean":"2024-04-26T21:35:15.143917+0000","last_became_active":"2024-04-26T21:35:10.121088+0000","last_became_peered":"2024-04-26T21:35:10.121088+0000","last_unstale":"2024-04-26T21:35:15.143917+0000","last_undegraded":"2024-04-26T21:35:15.143917+0000","last_fullsized":"2024-04-26T21:35:15.143917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004457+0000","last_change":"2024-04-26T21:35:10.131911+0000","last_active":"2024-04-26T21:35:16.004457+0000","last_peered":"2024-04-26T21:35:16.004457+0000","last_clean":"2024-04-26T21:35:16.004457+0000","last_became_active":"2024-04-26T21:35:10.131477+0000","last_became_peered":"2024-04-26T21:35:10.131477+0000","last_unstale":"2024-04-26T21:35:16.004457+0000","last_undegraded":"2024-04-26T21:35:16.004457+0000","last_fullsized":"2024-04-26T21:35:16.004457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.005022+0000","last_change":"2024-04-26T21:35:10.121507+0000","last_active":"2024-04-26T21:35:16.005022+0000","last_peered":"2024-04-26T21:35:16.005022+0000","last_clean":"2024-04-26T21:35:16.005022+0000","last_became_active":"2024-04-26T21:35:10.121208+0000","last_became_peered":"2024-04-26T21:35:10.121208+0000","last_unstale":"2024-04-26T21:35:16.005022+0000","last_undegraded":"2024-04-26T21:35:16.005022+0000","last_fullsized":"2024-04-26T21:35:16.005022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373872+0000","last_change":"2024-04-26T21:35:10.121106+0000","last_active":"2024-04-26T21:35:15.373872+0000","last_peered":"2024-04-26T21:35:15.373872+0000","last_clean":"2024-04-26T21:35:15.373872+0000","last_became_active":"2024-04-26T21:35:10.120857+0000","last_became_peered":"2024-04-26T21:35:10.120857+0000","last_unstale":"2024-04-26T21:35:15.373872+0000","last_undegraded":"2024-04-26T21:35:15.373872+0000","last_fullsized":"2024-04-26T21:35:15.373872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143866+0000","last_change":"2024-04-26T21:35:10.128212+0000","last_active":"2024-04-26T21:35:15.143866+0000","last_peered":"2024-04-26T21:35:15.143866+0000","last_clean":"2024-04-26T21:35:15.143866+0000","last_became_active":"2024-04-26T21:35:10.127868+0000","last_became_peered":"2024-04-26T21:35:10.127868+0000","last_unstale":"2024-04-26T21:35:15.143866+0000","last_undegraded":"2024-04-26T21:35:15.143866+0000","last_fullsized":"2024-04-26T21:35:15.143866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004798+0000","last_change":"2024-04-26T21:35:10.130695+0000","last_active":"2024-04-26T21:35:16.004798+0000","last_peered":"2024-04-26T21:35:16.004798+0000","last_clean":"2024-04-26T21:35:16.004798+0000","last_became_active":"2024-04-26T21:35:10.130084+0000","last_became_peered":"2024-04-26T21:35:10.130084+0000","last_unstale":"2024-04-26T21:35:16.004798+0000","last_undegraded":"2024-04-26T21:35:16.004798+0000","last_fullsized":"2024-04-26T21:35:16.004798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142712+0000","last_change":"2024-04-26T21:35:10.122732+0000","last_active":"2024-04-26T21:35:15.142712+0000","last_peered":"2024-04-26T21:35:15.142712+0000","last_clean":"2024-04-26T21:35:15.142712+0000","last_became_active":"2024-04-26T21:35:10.122457+0000","last_became_peered":"2024-04-26T21:35:10.122457+0000","last_unstale":"2024-04-26T21:35:15.142712+0000","last_undegraded":"2024-04-26T21:35:15.142712+0000","last_fullsized":"2024-04-26T21:35:15.142712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004615+0000","last_change":"2024-04-26T21:35:10.131503+0000","last_active":"2024-04-26T21:35:16.004615+0000","last_peered":"2024-04-26T21:35:16.004615+0000","last_clean":"2024-04-26T21:35:16.004615+0000","last_became_active":"2024-04-26T21:35:10.131262+0000","last_became_peered":"2024-04-26T21:35:10.131262+0000","last_unstale":"2024-04-26T21:35:16.004615+0000","last_undegraded":"2024-04-26T21:35:16.004615+0000","last_fullsized":"2024-04-26T21:35:16.004615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143913+0000","last_change":"2024-04-26T21:35:10.122046+0000","last_active":"2024-04-26T21:35:15.143913+0000","last_peered":"2024-04-26T21:35:15.143913+0000","last_clean":"2024-04-26T21:35:15.143913+0000","last_became_active":"2024-04-26T21:35:10.121759+0000","last_became_peered":"2024-04-26T21:35:10.121759+0000","last_unstale":"2024-04-26T21:35:15.143913+0000","last_undegraded":"2024-04-26T21:35:15.143913+0000","last_fullsized":"2024-04-26T21:35:15.143913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004361+0000","last_change":"2024-04-26T21:35:10.128039+0000","last_active":"2024-04-26T21:35:16.004361+0000","last_peered":"2024-04-26T21:35:16.004361+0000","last_clean":"2024-04-26T21:35:16.004361+0000","last_became_active":"2024-04-26T21:35:10.126920+0000","last_became_peered":"2024-04-26T21:35:10.126920+0000","last_unstale":"2024-04-26T21:35:16.004361+0000","last_undegraded":"2024-04-26T21:35:16.004361+0000","last_fullsized":"2024-04-26T21:35:16.004361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143654+0000","last_change":"2024-04-26T21:35:10.120371+0000","last_active":"2024-04-26T21:35:15.143654+0000","last_peered":"2024-04-26T21:35:15.143654+0000","last_clean":"2024-04-26T21:35:15.143654+0000","last_became_active":"2024-04-26T21:35:10.120020+0000","last_became_peered":"2024-04-26T21:35:10.120020+0000","last_unstale":"2024-04-26T21:35:15.143654+0000","last_undegraded":"2024-04-26T21:35:15.143654+0000","last_fullsized":"2024-04-26T21:35:15.143654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876023+0000","last_change":"2024-04-26T21:35:10.128002+0000","last_active":"2024-04-26T21:35:15.876023+0000","last_peered":"2024-04-26T21:35:15.876023+0000","last_clean":"2024-04-26T21:35:15.876023+0000","last_became_active":"2024-04-26T21:35:10.127763+0000","last_became_peered":"2024-04-26T21:35:10.127763+0000","last_unstale":"2024-04-26T21:35:15.876023+0000","last_undegraded":"2024-04-26T21:35:15.876023+0000","last_fullsized":"2024-04-26T21:35:15.876023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:09.615472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142501+0000","last_change":"2024-04-26T21:35:10.118765+0000","last_active":"2024-04-26T21:35:15.142501+0000","last_peered":"2024-04-26T21:35:15.142501+0000","last_clean":"2024-04-26T21:35:15.142501+0000","last_became_active":"2024-04-26T21:35:10.118481+0000","last_became_peered":"2024-04-26T21:35:10.118481+0000","last_unstale":"2024-04-26T21:35:15.142501+0000","last_undegraded":"2024-04-26T21:35:15.142501+0000","last_fullsized":"2024-04-26T21:35:15.142501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:55.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144424+0000","last_change":"2024-04-26T21:35:10.123877+0000","last_active":"2024-04-26T21:35:15.144424+0000","last_peered":"2024-04-26T21:35:15.144424+0000","last_clean":"2024-04-26T21:35:15.144424+0000","last_became_active":"2024-04-26T21:35:10.122628+0000","last_became_peered":"2024-04-26T21:35:10.122628+0000","last_unstale":"2024-04-26T21:35:15.144424+0000","last_undegraded":"2024-04-26T21:35:15.144424+0000","last_fullsized":"2024-04-26T21:35:15.144424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142605+0000","last_change":"2024-04-26T21:35:10.133644+0000","last_active":"2024-04-26T21:35:15.142605+0000","last_peered":"2024-04-26T21:35:15.142605+0000","last_clean":"2024-04-26T21:35:15.142605+0000","last_became_active":"2024-04-26T21:35:10.133364+0000","last_became_peered":"2024-04-26T21:35:10.133364+0000","last_unstale":"2024-04-26T21:35:15.142605+0000","last_undegraded":"2024-04-26T21:35:15.142605+0000","last_fullsized":"2024-04-26T21:35:15.142605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373575+0000","last_change":"2024-04-26T21:35:10.130778+0000","last_active":"2024-04-26T21:35:15.373575+0000","last_peered":"2024-04-26T21:35:15.373575+0000","last_clean":"2024-04-26T21:35:15.373575+0000","last_became_active":"2024-04-26T21:35:10.130501+0000","last_became_peered":"2024-04-26T21:35:10.130501+0000","last_unstale":"2024-04-26T21:35:15.373575+0000","last_undegraded":"2024-04-26T21:35:15.373575+0000","last_fullsized":"2024-04-26T21:35:15.373575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004023+0000","last_change":"2024-04-26T21:35:10.134920+0000","last_active":"2024-04-26T21:35:16.004023+0000","last_peered":"2024-04-26T21:35:16.004023+0000","last_clean":"2024-04-26T21:35:16.004023+0000","last_became_active":"2024-04-26T21:35:10.134647+0000","last_became_peered":"2024-04-26T21:35:10.134647+0000","last_unstale":"2024-04-26T21:35:16.004023+0000","last_undegraded":"2024-04-26T21:35:16.004023+0000","last_fullsized":"2024-04-26T21:35:16.004023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:19.754278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003968+0000","last_change":"2024-04-26T21:35:10.118720+0000","last_active":"2024-04-26T21:35:16.003968+0000","last_peered":"2024-04-26T21:35:16.003968+0000","last_clean":"2024-04-26T21:35:16.003968+0000","last_became_active":"2024-04-26T21:35:10.118418+0000","last_became_peered":"2024-04-26T21:35:10.118418+0000","last_unstale":"2024-04-26T21:35:16.003968+0000","last_undegraded":"2024-04-26T21:35:16.003968+0000","last_fullsized":"2024-04-26T21:35:16.003968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004237+0000","last_change":"2024-04-26T21:35:10.118157+0000","last_active":"2024-04-26T21:35:16.004237+0000","last_peered":"2024-04-26T21:35:16.004237+0000","last_clean":"2024-04-26T21:35:16.004237+0000","last_became_active":"2024-04-26T21:35:10.117862+0000","last_became_peered":"2024-04-26T21:35:10.117862+0000","last_unstale":"2024-04-26T21:35:16.004237+0000","last_undegraded":"2024-04-26T21:35:16.004237+0000","last_fullsized":"2024-04-26T21:35:16.004237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:26.040140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143343+0000","last_change":"2024-04-26T21:35:10.124503+0000","last_active":"2024-04-26T21:35:15.143343+0000","last_peered":"2024-04-26T21:35:15.143343+0000","last_clean":"2024-04-26T21:35:15.143343+0000","last_became_active":"2024-04-26T21:35:10.123919+0000","last_became_peered":"2024-04-26T21:35:10.123919+0000","last_unstale":"2024-04-26T21:35:15.143343+0000","last_undegraded":"2024-04-26T21:35:15.143343+0000","last_fullsized":"2024-04-26T21:35:15.143343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143962+0000","last_change":"2024-04-26T21:35:10.124821+0000","last_active":"2024-04-26T21:35:15.143962+0000","last_peered":"2024-04-26T21:35:15.143962+0000","last_clean":"2024-04-26T21:35:15.143962+0000","last_became_active":"2024-04-26T21:35:10.124245+0000","last_became_peered":"2024-04-26T21:35:10.124245+0000","last_unstale":"2024-04-26T21:35:15.143962+0000","last_undegraded":"2024-04-26T21:35:15.143962+0000","last_fullsized":"2024-04-26T21:35:15.143962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876099+0000","last_change":"2024-04-26T21:35:10.131624+0000","last_active":"2024-04-26T21:35:15.876099+0000","last_peered":"2024-04-26T21:35:15.876099+0000","last_clean":"2024-04-26T21:35:15.876099+0000","last_became_active":"2024-04-26T21:35:10.131392+0000","last_became_peered":"2024-04-26T21:35:10.131392+0000","last_unstale":"2024-04-26T21:35:15.876099+0000","last_undegraded":"2024-04-26T21:35:15.876099+0000","last_fullsized":"2024-04-26T21:35:15.876099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:33.396034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003337+0000","last_change":"2024-04-26T21:35:10.127875+0000","last_active":"2024-04-26T21:35:16.003337+0000","last_peered":"2024-04-26T21:35:16.003337+0000","last_clean":"2024-04-26T21:35:16.003337+0000","last_became_active":"2024-04-26T21:35:10.126566+0000","last_became_peered":"2024-04-26T21:35:10.126566+0000","last_unstale":"2024-04-26T21:35:16.003337+0000","last_undegraded":"2024-04-26T21:35:16.003337+0000","last_fullsized":"2024-04-26T21:35:16.003337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875494+0000","last_change":"2024-04-26T21:35:10.126677+0000","last_active":"2024-04-26T21:35:15.875494+0000","last_peered":"2024-04-26T21:35:15.875494+0000","last_clean":"2024-04-26T21:35:15.875494+0000","last_became_active":"2024-04-26T21:35:10.126388+0000","last_became_peered":"2024-04-26T21:35:10.126388+0000","last_unstale":"2024-04-26T21:35:15.875494+0000","last_undegraded":"2024-04-26T21:35:15.875494+0000","last_fullsized":"2024-04-26T21:35:15.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143757+0000","last_change":"2024-04-26T21:35:10.125406+0000","last_active":"2024-04-26T21:35:15.143757+0000","last_peered":"2024-04-26T21:35:15.143757+0000","last_clean":"2024-04-26T21:35:15.143757+0000","last_became_active":"2024-04-26T21:35:10.125143+0000","last_became_peered":"2024-04-26T21:35:10.125143+0000","last_unstale":"2024-04-26T21:35:15.143757+0000","last_undegraded":"2024-04-26T21:35:15.143757+0000","last_fullsized":"2024-04-26T21:35:15.143757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.130632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875494+0000","last_change":"2024-04-26T21:35:10.127947+0000","last_active":"2024-04-26T21:35:15.875494+0000","last_peered":"2024-04-26T21:35:15.875494+0000","last_clean":"2024-04-26T21:35:15.875494+0000","last_became_active":"2024-04-26T21:35:10.127700+0000","last_became_peered":"2024-04-26T21:35:10.127700+0000","last_unstale":"2024-04-26T21:35:15.875494+0000","last_undegraded":"2024-04-26T21:35:15.875494+0000","last_fullsized":"2024-04-26T21:35:15.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143721+0000","last_change":"2024-04-26T21:35:10.121445+0000","last_active":"2024-04-26T21:35:15.143721+0000","last_peered":"2024-04-26T21:35:15.143721+0000","last_clean":"2024-04-26T21:35:15.143721+0000","last_became_active":"2024-04-26T21:35:10.119356+0000","last_became_peered":"2024-04-26T21:35:10.119356+0000","last_unstale":"2024-04-26T21:35:15.143721+0000","last_undegraded":"2024-04-26T21:35:15.143721+0000","last_fullsized":"2024-04-26T21:35:15.143721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:27.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003601+0000","last_change":"2024-04-26T21:35:10.124758+0000","last_active":"2024-04-26T21:35:16.003601+0000","last_peered":"2024-04-26T21:35:16.003601+0000","last_clean":"2024-04-26T21:35:16.003601+0000","last_became_active":"2024-04-26T21:35:10.124480+0000","last_became_peered":"2024-04-26T21:35:10.124480+0000","last_unstale":"2024-04-26T21:35:16.003601+0000","last_undegraded":"2024-04-26T21:35:16.003601+0000","last_fullsized":"2024-04-26T21:35:16.003601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:29.764634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374098+0000","last_change":"2024-04-26T21:35:10.125937+0000","last_active":"2024-04-26T21:35:15.374098+0000","last_peered":"2024-04-26T21:35:15.374098+0000","last_clean":"2024-04-26T21:35:15.374098+0000","last_became_active":"2024-04-26T21:35:10.125640+0000","last_became_peered":"2024-04-26T21:35:10.125640+0000","last_unstale":"2024-04-26T21:35:15.374098+0000","last_undegraded":"2024-04-26T21:35:15.374098+0000","last_fullsized":"2024-04-26T21:35:15.374098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876035+0000","last_change":"2024-04-26T21:35:10.129050+0000","last_active":"2024-04-26T21:35:15.876035+0000","last_peered":"2024-04-26T21:35:15.876035+0000","last_clean":"2024-04-26T21:35:15.876035+0000","last_became_active":"2024-04-26T21:35:10.128720+0000","last_became_peered":"2024-04-26T21:35:10.128720+0000","last_unstale":"2024-04-26T21:35:15.876035+0000","last_undegraded":"2024-04-26T21:35:15.876035+0000","last_fullsized":"2024-04-26T21:35:15.876035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:33.320854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:35:33.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:35:33.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.346+0000 7f695fb2c640 1 Processor -- start 2024-04-26T21:35:33.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.346+0000 7f695fb2c640 1 -- start start 2024-04-26T21:35:33.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.347+0000 7f695fb2c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f69580b3860 0x7f695816a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.347+0000 7f695fb2c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69580d0ba0 con 0x7f695815f3a0 2024-04-26T21:35:33.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.347+0000 7f695fb2c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69580d2590 con 0x7f69580d06f0 2024-04-26T21:35:33.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.347+0000 7f695fb2c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f69580d0e20 con 0x7f69580b3860 2024-04-26T21:35:33.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.347+0000 7f695d8c8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f695815f3a0 0x7f69580d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40104/0 (socket says 172.21.15.40:40104) 2024-04-26T21:35:33.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.347+0000 7f695d8c8640 1 -- 172.21.15.40:0/2436196794 learned_addr learned my addr 172.21.15.40:0/2436196794 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:33.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.347+0000 7f695e0c9640 1 --2- 172.21.15.40:0/2436196794 >> v2:172.21.15.184:3300/0 conn(0x7f69580b3860 0x7f695816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.347+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1384109445 0 0) 0x7f6950001110 con 0x7f695815f3a0 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6940003520 con 0x7f695815f3a0 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385099009 0 0) 0x7f69580d2590 con 0x7f69580d06f0 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6950001110 con 0x7f69580d06f0 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3225403933 0 0) 0x7f6940003520 con 0x7f695815f3a0 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 >> v2:172.21.15.184:3300/0 conn(0x7f69580b3860 msgr2=0x7f695816a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 --2- 172.21.15.40:0/2436196794 >> v2:172.21.15.184:3300/0 conn(0x7f69580b3860 0x7f695816a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 >> v1:172.21.15.184:6789/0 conn(0x7f69580d06f0 legacy=0x7f69580d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69580b1d70 con 0x7f695815f3a0 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f6950001840 con 0x7f695815f3a0 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695e0c9640 1 --2- 172.21.15.40:0/2436196794 >> v2:172.21.15.184:3300/0 conn(0x7f69580b3860 0x7f695816a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 >> v1:172.21.15.40:6789/0 conn(0x7f695815f3a0 legacy=0x7f69580d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6940003b40 0x7f6940003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6940004800 0x7f6940004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69400051c0 0x7f694000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f69580d0e20 con 0x7f6940003b40 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f694000c0f0 con 0x7f69400051c0 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f694000c2f0 con 0x7f6940004800 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695d8c8640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6940003b40 0x7f6940003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695e0c9640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69400051c0 0x7f694000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695d0c7640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6940004800 0x7f6940004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695d8c8640 1 -- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6940004800 msgr2=0x7f6940004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695d8c8640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6940004800 0x7f6940004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695d8c8640 1 -- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69400051c0 msgr2=0x7f694000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695d8c8640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69400051c0 0x7f694000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695d8c8640 1 -- 172.21.15.40:0/2436196794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6940003520 con 0x7f6940003b40 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695e0c9640 1 -- 172.21.15.40:0/2436196794 reap_dead start 2024-04-26T21:35:33.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.348+0000 7f695d0c7640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6940004800 0x7f6940004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.349+0000 7f695d8c8640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6940003b40 0x7f6940003f40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f695000c490 tx=0x7f6940005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c99abc9e76d455 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.349+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695000c0e0 con 0x7f6940003b40 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.349+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6950034070 con 0x7f6940003b40 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.349+0000 7f695c8c6640 1 -- 172.21.15.40:0/2436196794 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950005d70 con 0x7f6940003b40 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.350+0000 7f695fb2c640 1 -- 172.21.15.40:0/2436196794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6940003b40 msgr2=0x7f6940003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.350+0000 7f695fb2c640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6940003b40 0x7f6940003f40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f695000c490 tx=0x7f6940005830 comp rx=0 tx=0).stop 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.350+0000 7f695fb2c640 1 -- 172.21.15.40:0/2436196794 shutdown_connections 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.350+0000 7f695fb2c640 1 --2- 172.21.15.40:0/2436196794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6940003b40 0x7f6940003f40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.350+0000 7f695fb2c640 1 -- 172.21.15.40:0/2436196794 >> 172.21.15.40:0/2436196794 conn(0x7f69580d4f90 msgr2=0x7f69580d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.350+0000 7f695fb2c640 1 -- 172.21.15.40:0/2436196794 shutdown_connections 2024-04-26T21:35:33.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.350+0000 7f695fb2c640 1 -- 172.21.15.40:0/2436196794 wait complete. 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.351+0000 7f695fb2c640 1 Processor -- start 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.351+0000 7f695fb2c640 1 -- start start 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695fb2c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6940003b40 0x7f69580cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695fb2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69581f58c0 0x7f6958161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695fb2c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581623f0 0x7f69581f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695fb2c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f695815f780 con 0x7f69581f58c0 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695fb2c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69580b2330 con 0x7f6940003b40 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695fb2c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f695815f900 con 0x7f69581623f0 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69581f58c0 0x7f6958161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69581f58c0 0x7f6958161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45450/0 (socket says 172.21.15.40:45450) 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 -- 172.21.15.40:0/286607042 learned_addr learned my addr 172.21.15.40:0/286607042 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d8c8640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6940003b40 0x7f69580cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695e0c9640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581623f0 0x7f69581f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 -- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581623f0 msgr2=0x7f69581f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581623f0 0x7f69581f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 -- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6940003b40 msgr2=0x7f69580cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6940003b40 0x7f69580cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 -- 172.21.15.40:0/286607042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69580c8e70 con 0x7f69581f58c0 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d8c8640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6940003b40 0x7f69580cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695d0c7640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69581f58c0 0x7f6958161eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f694c00e4a0 tx=0x7f694c00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f69567fc640 1 -- 172.21.15.40:0/286607042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c0052e0 con 0x7f69581f58c0 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f69567fc640 1 -- 172.21.15.40:0/286607042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f694c005480 con 0x7f69581f58c0 2024-04-26T21:35:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.352+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69580c9150 con 0x7f69581f58c0 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.353+0000 7f69567fc640 1 -- 172.21.15.40:0/286607042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c007020 con 0x7f69581f58c0 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.353+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69580c9710 con 0x7f69581f58c0 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.353+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69580694f0 con 0x7f69581f58c0 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.354+0000 7f69567fc640 1 -- 172.21.15.40:0/286607042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f694c01a020 con 0x7f69581f58c0 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.354+0000 7f69567fc640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f694406c8f0 0x7f694406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.355+0000 7f69567fc640 1 -- 172.21.15.40:0/286607042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f694c08b500 con 0x7f69581f58c0 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.355+0000 7f695d8c8640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f694406c8f0 0x7f694406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.355+0000 7f695d8c8640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f694406c8f0 0x7f694406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6950033000 tx=0x7f6950009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:33.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.359+0000 7f69567fc640 1 -- 172.21.15.40:0/286607042 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f694c059550 con 0x7f69581f58c0 2024-04-26T21:35:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.539+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6958162ac0 con 0x7f694406c8f0 2024-04-26T21:35:33.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.547+0000 7f69567fc640 1 -- 172.21.15.40:0/286607042 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f6958162ac0 con 0x7f694406c8f0 2024-04-26T21:35:33.552 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:33.553 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:35:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f694406c8f0 msgr2=0x7f694406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f694406c8f0 0x7f694406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6950033000 tx=0x7f6950009030 comp rx=0 tx=0).stop 2024-04-26T21:35:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69581f58c0 msgr2=0x7f6958161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69581f58c0 0x7f6958161eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f694c00e4a0 tx=0x7f694c00e960 comp rx=0 tx=0).stop 2024-04-26T21:35:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 shutdown_connections 2024-04-26T21:35:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f694406c8f0 0x7f694406edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581623f0 0x7f69581f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69581f58c0 0x7f6958161eb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.550+0000 7f695fb2c640 1 --2- 172.21.15.40:0/286607042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6940003b40 0x7f69580cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.551+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 >> 172.21.15.40:0/286607042 conn(0x7f69580d4f90 msgr2=0x7f69580b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.551+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 shutdown_connections 2024-04-26T21:35:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.551+0000 7f695fb2c640 1 -- 172.21.15.40:0/286607042 wait complete. 2024-04-26T21:35:33.562 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-26T21:35:31.998856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219400,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755320,"statfs":{"total":773094113280,"available":772869447680,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003561"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004303+0000","last_change":"2024-04-26T21:35:09.132596+0000","last_active":"2024-04-26T21:35:16.004303+0000","last_peered":"2024-04-26T21:35:16.004303+0000","last_clean":"2024-04-26T21:35:16.004303+0000","last_became_active":"2024-04-26T21:35:09.132270+0000","last_became_peered":"2024-04-26T21:35:09.132270+0000","last_unstale":"2024-04-26T21:35:16.004303+0000","last_undegraded":"2024-04-26T21:35:16.004303+0000","last_fullsized":"2024-04-26T21:35:16.004303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:22.652595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003605+0000","last_change":"2024-04-26T21:35:10.126640+0000","last_active":"2024-04-26T21:35:16.003605+0000","last_peered":"2024-04-26T21:35:16.003605+0000","last_clean":"2024-04-26T21:35:16.003605+0000","last_became_active":"2024-04-26T21:35:10.125603+0000","last_became_peered":"2024-04-26T21:35:10.125603+0000","last_unstale":"2024-04-26T21:35:16.003605+0000","last_undegraded":"2024-04-26T21:35:16.003605+0000","last_fullsized":"2024-04-26T21:35:16.003605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:27.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004487+0000","last_change":"2024-04-26T21:35:14.280943+0000","last_active":"2024-04-26T21:35:16.004487+0000","last_peered":"2024-04-26T21:35:16.004487+0000","last_clean":"2024-04-26T21:35:16.004487+0000","last_became_active":"2024-04-26T21:35:14.280611+0000","last_became_peered":"2024-04-26T21:35:14.280611+0000","last_unstale":"2024-04-26T21:35:16.004487+0000","last_undegraded":"2024-04-26T21:35:16.004487+0000","last_fullsized":"2024-04-26T21:35:16.004487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:28.734397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004488+0000","last_change":"2024-04-26T21:35:09.122996+0000","last_active":"2024-04-26T21:35:16.004488+0000","last_peered":"2024-04-26T21:35:16.004488+0000","last_clean":"2024-04-26T21:35:16.004488+0000","last_became_active":"2024-04-26T21:35:09.122682+0000","last_became_peered":"2024-04-26T21:35:09.122682+0000","last_unstale":"2024-04-26T21:35:16.004488+0000","last_undegraded":"2024-04-26T21:35:16.004488+0000","last_fullsized":"2024-04-26T21:35:16.004488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:44.358668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004023+0000","last_change":"2024-04-26T21:35:10.133464+0000","last_active":"2024-04-26T21:35:16.004023+0000","last_peered":"2024-04-26T21:35:16.004023+0000","last_clean":"2024-04-26T21:35:16.004023+0000","last_became_active":"2024-04-26T21:35:10.133170+0000","last_became_peered":"2024-04-26T21:35:10.133170+0000","last_unstale":"2024-04-26T21:35:16.004023+0000","last_undegraded":"2024-04-26T21:35:16.004023+0000","last_fullsized":"2024-04-26T21:35:16.004023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003606+0000","last_change":"2024-04-26T21:35:14.163746+0000","last_active":"2024-04-26T21:35:16.003606+0000","last_peered":"2024-04-26T21:35:16.003606+0000","last_clean":"2024-04-26T21:35:16.003606+0000","last_became_active":"2024-04-26T21:35:14.163263+0000","last_became_peered":"2024-04-26T21:35:14.163263+0000","last_unstale":"2024-04-26T21:35:16.003606+0000","last_undegraded":"2024-04-26T21:35:16.003606+0000","last_fullsized":"2024-04-26T21:35:16.003606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:28.472320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875887+0000","last_change":"2024-04-26T21:35:09.129656+0000","last_active":"2024-04-26T21:35:15.875887+0000","last_peered":"2024-04-26T21:35:15.875887+0000","last_clean":"2024-04-26T21:35:15.875887+0000","last_became_active":"2024-04-26T21:35:09.129379+0000","last_became_peered":"2024-04-26T21:35:09.129379+0000","last_unstale":"2024-04-26T21:35:15.875887+0000","last_undegraded":"2024-04-26T21:35:15.875887+0000","last_fullsized":"2024-04-26T21:35:15.875887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:51.639722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143824+0000","last_change":"2024-04-26T21:35:10.119557+0000","last_active":"2024-04-26T21:35:15.143824+0000","last_peered":"2024-04-26T21:35:15.143824+0000","last_clean":"2024-04-26T21:35:15.143824+0000","last_became_active":"2024-04-26T21:35:10.118407+0000","last_became_peered":"2024-04-26T21:35:10.118407+0000","last_unstale":"2024-04-26T21:35:15.143824+0000","last_undegraded":"2024-04-26T21:35:15.143824+0000","last_fullsized":"2024-04-26T21:35:15.143824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003821+0000","last_change":"2024-04-26T21:35:14.159166+0000","last_active":"2024-04-26T21:35:16.003821+0000","last_peered":"2024-04-26T21:35:16.003821+0000","last_clean":"2024-04-26T21:35:16.003821+0000","last_became_active":"2024-04-26T21:35:14.158592+0000","last_became_peered":"2024-04-26T21:35:14.158592+0000","last_unstale":"2024-04-26T21:35:16.003821+0000","last_undegraded":"2024-04-26T21:35:16.003821+0000","last_fullsized":"2024-04-26T21:35:16.003821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:31.201343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875768+0000","last_change":"2024-04-26T21:35:09.128770+0000","last_active":"2024-04-26T21:35:15.875768+0000","last_peered":"2024-04-26T21:35:15.875768+0000","last_clean":"2024-04-26T21:35:15.875768+0000","last_became_active":"2024-04-26T21:35:09.128385+0000","last_became_peered":"2024-04-26T21:35:09.128385+0000","last_unstale":"2024-04-26T21:35:15.875768+0000","last_undegraded":"2024-04-26T21:35:15.875768+0000","last_fullsized":"2024-04-26T21:35:15.875768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:20.201305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144355+0000","last_change":"2024-04-26T21:35:10.117537+0000","last_active":"2024-04-26T21:35:15.144355+0000","last_peered":"2024-04-26T21:35:15.144355+0000","last_clean":"2024-04-26T21:35:15.144355+0000","last_became_active":"2024-04-26T21:35:10.117267+0000","last_became_peered":"2024-04-26T21:35:10.117267+0000","last_unstale":"2024-04-26T21:35:15.144355+0000","last_undegraded":"2024-04-26T21:35:15.144355+0000","last_fullsized":"2024-04-26T21:35:15.144355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:01.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003840+0000","last_change":"2024-04-26T21:35:14.161136+0000","last_active":"2024-04-26T21:35:16.003840+0000","last_peered":"2024-04-26T21:35:16.003840+0000","last_clean":"2024-04-26T21:35:16.003840+0000","last_became_active":"2024-04-26T21:35:14.160648+0000","last_became_peered":"2024-04-26T21:35:14.160648+0000","last_unstale":"2024-04-26T21:35:16.003840+0000","last_undegraded":"2024-04-26T21:35:16.003840+0000","last_fullsized":"2024-04-26T21:35:16.003840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:10.381659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143344+0000","last_change":"2024-04-26T21:35:10.119355+0000","last_active":"2024-04-26T21:35:15.143344+0000","last_peered":"2024-04-26T21:35:15.143344+0000","last_clean":"2024-04-26T21:35:15.143344+0000","last_became_active":"2024-04-26T21:35:10.119067+0000","last_became_peered":"2024-04-26T21:35:10.119067+0000","last_unstale":"2024-04-26T21:35:15.143344+0000","last_undegraded":"2024-04-26T21:35:15.143344+0000","last_fullsized":"2024-04-26T21:35:15.143344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876041+0000","last_change":"2024-04-26T21:35:09.129853+0000","last_active":"2024-04-26T21:35:15.876041+0000","last_peered":"2024-04-26T21:35:15.876041+0000","last_clean":"2024-04-26T21:35:15.876041+0000","last_became_active":"2024-04-26T21:35:09.129500+0000","last_became_peered":"2024-04-26T21:35:09.129500+0000","last_unstale":"2024-04-26T21:35:15.876041+0000","last_undegraded":"2024-04-26T21:35:15.876041+0000","last_fullsized":"2024-04-26T21:35:15.876041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:54.605573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004307+0000","last_change":"2024-04-26T21:35:14.161264+0000","last_active":"2024-04-26T21:35:16.004307+0000","last_peered":"2024-04-26T21:35:16.004307+0000","last_clean":"2024-04-26T21:35:16.004307+0000","last_became_active":"2024-04-26T21:35:14.160617+0000","last_became_peered":"2024-04-26T21:35:14.160617+0000","last_unstale":"2024-04-26T21:35:16.004307+0000","last_undegraded":"2024-04-26T21:35:16.004307+0000","last_fullsized":"2024-04-26T21:35:16.004307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004050+0000","last_change":"2024-04-26T21:35:10.125998+0000","last_active":"2024-04-26T21:35:16.004050+0000","last_peered":"2024-04-26T21:35:16.004050+0000","last_clean":"2024-04-26T21:35:16.004050+0000","last_became_active":"2024-04-26T21:35:10.125704+0000","last_became_peered":"2024-04-26T21:35:10.125704+0000","last_unstale":"2024-04-26T21:35:16.004050+0000","last_undegraded":"2024-04-26T21:35:16.004050+0000","last_fullsized":"2024-04-26T21:35:16.004050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373953+0000","last_change":"2024-04-26T21:35:09.123003+0000","last_active":"2024-04-26T21:35:15.373953+0000","last_peered":"2024-04-26T21:35:15.373953+0000","last_clean":"2024-04-26T21:35:15.373953+0000","last_became_active":"2024-04-26T21:35:09.122610+0000","last_became_peered":"2024-04-26T21:35:09.122610+0000","last_unstale":"2024-04-26T21:35:15.373953+0000","last_undegraded":"2024-04-26T21:35:15.373953+0000","last_fullsized":"2024-04-26T21:35:15.373953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004080+0000","last_change":"2024-04-26T21:35:14.164010+0000","last_active":"2024-04-26T21:35:16.004080+0000","last_peered":"2024-04-26T21:35:16.004080+0000","last_clean":"2024-04-26T21:35:16.004080+0000","last_became_active":"2024-04-26T21:35:14.163262+0000","last_became_peered":"2024-04-26T21:35:14.163262+0000","last_unstale":"2024-04-26T21:35:16.004080+0000","last_undegraded":"2024-04-26T21:35:16.004080+0000","last_fullsized":"2024-04-26T21:35:16.004080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:45.829307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003626+0000","last_change":"2024-04-26T21:35:10.120565+0000","last_active":"2024-04-26T21:35:16.003626+0000","last_peered":"2024-04-26T21:35:16.003626+0000","last_clean":"2024-04-26T21:35:16.003626+0000","last_became_active":"2024-04-26T21:35:10.120133+0000","last_became_peered":"2024-04-26T21:35:10.120133+0000","last_unstale":"2024-04-26T21:35:16.003626+0000","last_undegraded":"2024-04-26T21:35:16.003626+0000","last_fullsized":"2024-04-26T21:35:16.003626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:06.354261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373691+0000","last_change":"2024-04-26T21:35:09.127198+0000","last_active":"2024-04-26T21:35:15.373691+0000","last_peered":"2024-04-26T21:35:15.373691+0000","last_clean":"2024-04-26T21:35:15.373691+0000","last_became_active":"2024-04-26T21:35:09.126870+0000","last_became_peered":"2024-04-26T21:35:09.126870+0000","last_unstale":"2024-04-26T21:35:15.373691+0000","last_undegraded":"2024-04-26T21:35:15.373691+0000","last_fullsized":"2024-04-26T21:35:15.373691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374142+0000","last_change":"2024-04-26T21:35:14.277851+0000","last_active":"2024-04-26T21:35:15.374142+0000","last_peered":"2024-04-26T21:35:15.374142+0000","last_clean":"2024-04-26T21:35:15.374142+0000","last_became_active":"2024-04-26T21:35:14.277461+0000","last_became_peered":"2024-04-26T21:35:14.277461+0000","last_unstale":"2024-04-26T21:35:15.374142+0000","last_undegraded":"2024-04-26T21:35:15.374142+0000","last_fullsized":"2024-04-26T21:35:15.374142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:20.536176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004022+0000","last_change":"2024-04-26T21:35:10.113247+0000","last_active":"2024-04-26T21:35:16.004022+0000","last_peered":"2024-04-26T21:35:16.004022+0000","last_clean":"2024-04-26T21:35:16.004022+0000","last_became_active":"2024-04-26T21:35:10.112937+0000","last_became_peered":"2024-04-26T21:35:10.112937+0000","last_unstale":"2024-04-26T21:35:16.004022+0000","last_undegraded":"2024-04-26T21:35:16.004022+0000","last_fullsized":"2024-04-26T21:35:16.004022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:47.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875388+0000","last_change":"2024-04-26T21:35:09.134099+0000","last_active":"2024-04-26T21:35:15.875388+0000","last_peered":"2024-04-26T21:35:15.875388+0000","last_clean":"2024-04-26T21:35:15.875388+0000","last_became_active":"2024-04-26T21:35:09.133869+0000","last_became_peered":"2024-04-26T21:35:09.133869+0000","last_unstale":"2024-04-26T21:35:15.875388+0000","last_undegraded":"2024-04-26T21:35:15.875388+0000","last_fullsized":"2024-04-26T21:35:15.875388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:16.219471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004280+0000","last_change":"2024-04-26T21:35:14.158160+0000","last_active":"2024-04-26T21:35:16.004280+0000","last_peered":"2024-04-26T21:35:16.004280+0000","last_clean":"2024-04-26T21:35:16.004280+0000","last_became_active":"2024-04-26T21:35:14.157711+0000","last_became_peered":"2024-04-26T21:35:14.157711+0000","last_unstale":"2024-04-26T21:35:16.004280+0000","last_undegraded":"2024-04-26T21:35:16.004280+0000","last_fullsized":"2024-04-26T21:35:16.004280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:26.153246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004196+0000","last_change":"2024-04-26T21:35:10.136384+0000","last_active":"2024-04-26T21:35:16.004196+0000","last_peered":"2024-04-26T21:35:16.004196+0000","last_clean":"2024-04-26T21:35:16.004196+0000","last_became_active":"2024-04-26T21:35:10.135589+0000","last_became_peered":"2024-04-26T21:35:10.135589+0000","last_unstale":"2024-04-26T21:35:16.004196+0000","last_undegraded":"2024-04-26T21:35:16.004196+0000","last_fullsized":"2024-04-26T21:35:16.004196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144087+0000","last_change":"2024-04-26T21:35:09.125486+0000","last_active":"2024-04-26T21:35:15.144087+0000","last_peered":"2024-04-26T21:35:15.144087+0000","last_clean":"2024-04-26T21:35:15.144087+0000","last_became_active":"2024-04-26T21:35:09.125019+0000","last_became_peered":"2024-04-26T21:35:09.125019+0000","last_unstale":"2024-04-26T21:35:15.144087+0000","last_undegraded":"2024-04-26T21:35:15.144087+0000","last_fullsized":"2024-04-26T21:35:15.144087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003847+0000","last_change":"2024-04-26T21:35:14.281054+0000","last_active":"2024-04-26T21:35:16.003847+0000","last_peered":"2024-04-26T21:35:16.003847+0000","last_clean":"2024-04-26T21:35:16.003847+0000","last_became_active":"2024-04-26T21:35:14.278082+0000","last_became_peered":"2024-04-26T21:35:14.278082+0000","last_unstale":"2024-04-26T21:35:16.003847+0000","last_undegraded":"2024-04-26T21:35:16.003847+0000","last_fullsized":"2024-04-26T21:35:16.003847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.959963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004866+0000","last_change":"2024-04-26T21:35:10.130908+0000","last_active":"2024-04-26T21:35:16.004866+0000","last_peered":"2024-04-26T21:35:16.004866+0000","last_clean":"2024-04-26T21:35:16.004866+0000","last_became_active":"2024-04-26T21:35:10.130448+0000","last_became_peered":"2024-04-26T21:35:10.130448+0000","last_unstale":"2024-04-26T21:35:16.004866+0000","last_undegraded":"2024-04-26T21:35:16.004866+0000","last_fullsized":"2024-04-26T21:35:16.004866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374192+0000","last_change":"2024-04-26T21:35:09.124758+0000","last_active":"2024-04-26T21:35:15.374192+0000","last_peered":"2024-04-26T21:35:15.374192+0000","last_clean":"2024-04-26T21:35:15.374192+0000","last_became_active":"2024-04-26T21:35:09.124446+0000","last_became_peered":"2024-04-26T21:35:09.124446+0000","last_unstale":"2024-04-26T21:35:15.374192+0000","last_undegraded":"2024-04-26T21:35:15.374192+0000","last_fullsized":"2024-04-26T21:35:15.374192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:16.219471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003317+0000","last_change":"2024-04-26T21:35:14.280487+0000","last_active":"2024-04-26T21:35:16.003317+0000","last_peered":"2024-04-26T21:35:16.003317+0000","last_clean":"2024-04-26T21:35:16.003317+0000","last_became_active":"2024-04-26T21:35:14.280075+0000","last_became_peered":"2024-04-26T21:35:14.280075+0000","last_unstale":"2024-04-26T21:35:16.003317+0000","last_undegraded":"2024-04-26T21:35:16.003317+0000","last_fullsized":"2024-04-26T21:35:16.003317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:49.955325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373962+0000","last_change":"2024-04-26T21:35:10.124638+0000","last_active":"2024-04-26T21:35:15.373962+0000","last_peered":"2024-04-26T21:35:15.373962+0000","last_clean":"2024-04-26T21:35:15.373962+0000","last_became_active":"2024-04-26T21:35:10.124298+0000","last_became_peered":"2024-04-26T21:35:10.124298+0000","last_unstale":"2024-04-26T21:35:15.373962+0000","last_undegraded":"2024-04-26T21:35:15.373962+0000","last_fullsized":"2024-04-26T21:35:15.373962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876132+0000","last_change":"2024-04-26T21:35:09.134096+0000","last_active":"2024-04-26T21:35:15.876132+0000","last_peered":"2024-04-26T21:35:15.876132+0000","last_clean":"2024-04-26T21:35:15.876132+0000","last_became_active":"2024-04-26T21:35:09.133816+0000","last_became_peered":"2024-04-26T21:35:09.133816+0000","last_unstale":"2024-04-26T21:35:15.876132+0000","last_undegraded":"2024-04-26T21:35:15.876132+0000","last_fullsized":"2024-04-26T21:35:15.876132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:39.887861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004297+0000","last_change":"2024-04-26T21:35:14.277888+0000","last_active":"2024-04-26T21:35:16.004297+0000","last_peered":"2024-04-26T21:35:16.004297+0000","last_clean":"2024-04-26T21:35:16.004297+0000","last_became_active":"2024-04-26T21:35:14.277473+0000","last_became_peered":"2024-04-26T21:35:14.277473+0000","last_unstale":"2024-04-26T21:35:16.004297+0000","last_undegraded":"2024-04-26T21:35:16.004297+0000","last_fullsized":"2024-04-26T21:35:16.004297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:20.536176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144502+0000","last_change":"2024-04-26T21:35:10.118766+0000","last_active":"2024-04-26T21:35:15.144502+0000","last_peered":"2024-04-26T21:35:15.144502+0000","last_clean":"2024-04-26T21:35:15.144502+0000","last_became_active":"2024-04-26T21:35:10.118480+0000","last_became_peered":"2024-04-26T21:35:10.118480+0000","last_unstale":"2024-04-26T21:35:15.144502+0000","last_undegraded":"2024-04-26T21:35:15.144502+0000","last_fullsized":"2024-04-26T21:35:15.144502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:33.320854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374086+0000","last_change":"2024-04-26T21:35:09.124939+0000","last_active":"2024-04-26T21:35:15.374086+0000","last_peered":"2024-04-26T21:35:15.374086+0000","last_clean":"2024-04-26T21:35:15.374086+0000","last_became_active":"2024-04-26T21:35:09.124630+0000","last_became_peered":"2024-04-26T21:35:09.124630+0000","last_unstale":"2024-04-26T21:35:15.374086+0000","last_undegraded":"2024-04-26T21:35:15.374086+0000","last_fullsized":"2024-04-26T21:35:15.374086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374121+0000","last_change":"2024-04-26T21:35:14.165015+0000","last_active":"2024-04-26T21:35:15.374121+0000","last_peered":"2024-04-26T21:35:15.374121+0000","last_clean":"2024-04-26T21:35:15.374121+0000","last_became_active":"2024-04-26T21:35:14.164615+0000","last_became_peered":"2024-04-26T21:35:14.164615+0000","last_unstale":"2024-04-26T21:35:15.374121+0000","last_undegraded":"2024-04-26T21:35:15.374121+0000","last_fullsized":"2024-04-26T21:35:15.374121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:37.348252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004379+0000","last_change":"2024-04-26T21:35:10.125981+0000","last_active":"2024-04-26T21:35:16.004379+0000","last_peered":"2024-04-26T21:35:16.004379+0000","last_clean":"2024-04-26T21:35:16.004379+0000","last_became_active":"2024-04-26T21:35:10.125718+0000","last_became_peered":"2024-04-26T21:35:10.125718+0000","last_unstale":"2024-04-26T21:35:16.004379+0000","last_undegraded":"2024-04-26T21:35:16.004379+0000","last_fullsized":"2024-04-26T21:35:16.004379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:09.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144282+0000","last_change":"2024-04-26T21:35:09.121086+0000","last_active":"2024-04-26T21:35:15.144282+0000","last_peered":"2024-04-26T21:35:15.144282+0000","last_clean":"2024-04-26T21:35:15.144282+0000","last_became_active":"2024-04-26T21:35:09.120443+0000","last_became_peered":"2024-04-26T21:35:09.120443+0000","last_unstale":"2024-04-26T21:35:15.144282+0000","last_undegraded":"2024-04-26T21:35:15.144282+0000","last_fullsized":"2024-04-26T21:35:15.144282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374085+0000","last_change":"2024-04-26T21:35:14.163974+0000","last_active":"2024-04-26T21:35:15.374085+0000","last_peered":"2024-04-26T21:35:15.374085+0000","last_clean":"2024-04-26T21:35:15.374085+0000","last_became_active":"2024-04-26T21:35:14.163558+0000","last_became_peered":"2024-04-26T21:35:14.163558+0000","last_unstale":"2024-04-26T21:35:15.374085+0000","last_undegraded":"2024-04-26T21:35:15.374085+0000","last_fullsized":"2024-04-26T21:35:15.374085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:01.158030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373556+0000","last_change":"2024-04-26T21:35:10.124855+0000","last_active":"2024-04-26T21:35:15.373556+0000","last_peered":"2024-04-26T21:35:15.373556+0000","last_clean":"2024-04-26T21:35:15.373556+0000","last_became_active":"2024-04-26T21:35:10.124588+0000","last_became_peered":"2024-04-26T21:35:10.124588+0000","last_unstale":"2024-04-26T21:35:15.373556+0000","last_undegraded":"2024-04-26T21:35:15.373556+0000","last_fullsized":"2024-04-26T21:35:15.373556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142012+0000","last_change":"2024-04-26T21:35:09.115909+0000","last_active":"2024-04-26T21:35:15.142012+0000","last_peered":"2024-04-26T21:35:15.142012+0000","last_clean":"2024-04-26T21:35:15.142012+0000","last_became_active":"2024-04-26T21:35:09.115120+0000","last_became_peered":"2024-04-26T21:35:09.115120+0000","last_unstale":"2024-04-26T21:35:15.142012+0000","last_undegraded":"2024-04-26T21:35:15.142012+0000","last_fullsized":"2024-04-26T21:35:15.142012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374302+0000","last_change":"2024-04-26T21:35:14.165160+0000","last_active":"2024-04-26T21:35:15.374302+0000","last_peered":"2024-04-26T21:35:15.374302+0000","last_clean":"2024-04-26T21:35:15.374302+0000","last_became_active":"2024-04-26T21:35:14.164683+0000","last_became_peered":"2024-04-26T21:35:14.164683+0000","last_unstale":"2024-04-26T21:35:15.374302+0000","last_undegraded":"2024-04-26T21:35:15.374302+0000","last_fullsized":"2024-04-26T21:35:15.374302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:37.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003512+0000","last_change":"2024-04-26T21:35:10.128846+0000","last_active":"2024-04-26T21:35:16.003512+0000","last_peered":"2024-04-26T21:35:16.003512+0000","last_clean":"2024-04-26T21:35:16.003512+0000","last_became_active":"2024-04-26T21:35:10.128576+0000","last_became_peered":"2024-04-26T21:35:10.128576+0000","last_unstale":"2024-04-26T21:35:16.003512+0000","last_undegraded":"2024-04-26T21:35:16.003512+0000","last_fullsized":"2024-04-26T21:35:16.003512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:40.892489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875388+0000","last_change":"2024-04-26T21:35:09.130374+0000","last_active":"2024-04-26T21:35:15.875388+0000","last_peered":"2024-04-26T21:35:15.875388+0000","last_clean":"2024-04-26T21:35:15.875388+0000","last_became_active":"2024-04-26T21:35:09.130122+0000","last_became_peered":"2024-04-26T21:35:09.130122+0000","last_unstale":"2024-04-26T21:35:15.875388+0000","last_undegraded":"2024-04-26T21:35:15.875388+0000","last_fullsized":"2024-04-26T21:35:15.875388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:25.035511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004078+0000","last_change":"2024-04-26T21:35:14.277956+0000","last_active":"2024-04-26T21:35:16.004078+0000","last_peered":"2024-04-26T21:35:16.004078+0000","last_clean":"2024-04-26T21:35:16.004078+0000","last_became_active":"2024-04-26T21:35:14.277555+0000","last_became_peered":"2024-04-26T21:35:14.277555+0000","last_unstale":"2024-04-26T21:35:16.004078+0000","last_undegraded":"2024-04-26T21:35:16.004078+0000","last_fullsized":"2024-04-26T21:35:16.004078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:37.348252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003728+0000","last_change":"2024-04-26T21:35:10.127546+0000","last_active":"2024-04-26T21:35:16.003728+0000","last_peered":"2024-04-26T21:35:16.003728+0000","last_clean":"2024-04-26T21:35:16.003728+0000","last_became_active":"2024-04-26T21:35:10.127263+0000","last_became_peered":"2024-04-26T21:35:10.127263+0000","last_unstale":"2024-04-26T21:35:16.003728+0000","last_undegraded":"2024-04-26T21:35:16.003728+0000","last_fullsized":"2024-04-26T21:35:16.003728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144006+0000","last_change":"2024-04-26T21:35:09.116290+0000","last_active":"2024-04-26T21:35:15.144006+0000","last_peered":"2024-04-26T21:35:15.144006+0000","last_clean":"2024-04-26T21:35:15.144006+0000","last_became_active":"2024-04-26T21:35:09.115972+0000","last_became_peered":"2024-04-26T21:35:09.115972+0000","last_unstale":"2024-04-26T21:35:15.144006+0000","last_undegraded":"2024-04-26T21:35:15.144006+0000","last_fullsized":"2024-04-26T21:35:15.144006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875528+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:35:15.875528+0000","last_peered":"2024-04-26T21:35:15.875528+0000","last_clean":"2024-04-26T21:35:15.875528+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:35:15.875528+0000","last_undegraded":"2024-04-26T21:35:15.875528+0000","last_fullsized":"2024-04-26T21:35:15.875528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143208+0000","last_change":"2024-04-26T21:35:14.279050+0000","last_active":"2024-04-26T21:35:15.143208+0000","last_peered":"2024-04-26T21:35:15.143208+0000","last_clean":"2024-04-26T21:35:15.143208+0000","last_became_active":"2024-04-26T21:35:14.278651+0000","last_became_peered":"2024-04-26T21:35:14.278651+0000","last_unstale":"2024-04-26T21:35:15.143208+0000","last_undegraded":"2024-04-26T21:35:15.143208+0000","last_fullsized":"2024-04-26T21:35:15.143208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:26.153246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875749+0000","last_change":"2024-04-26T21:35:10.130945+0000","last_active":"2024-04-26T21:35:15.875749+0000","last_peered":"2024-04-26T21:35:15.875749+0000","last_clean":"2024-04-26T21:35:15.875749+0000","last_became_active":"2024-04-26T21:35:10.130673+0000","last_became_peered":"2024-04-26T21:35:10.130673+0000","last_unstale":"2024-04-26T21:35:15.875749+0000","last_undegraded":"2024-04-26T21:35:15.875749+0000","last_fullsized":"2024-04-26T21:35:15.875749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:16.508778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142347+0000","last_change":"2024-04-26T21:35:09.118997+0000","last_active":"2024-04-26T21:35:15.142347+0000","last_peered":"2024-04-26T21:35:15.142347+0000","last_clean":"2024-04-26T21:35:15.142347+0000","last_became_active":"2024-04-26T21:35:09.118700+0000","last_became_peered":"2024-04-26T21:35:09.118700+0000","last_unstale":"2024-04-26T21:35:15.142347+0000","last_undegraded":"2024-04-26T21:35:15.142347+0000","last_fullsized":"2024-04-26T21:35:15.142347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143286+0000","last_change":"2024-04-26T21:35:14.277928+0000","last_active":"2024-04-26T21:35:15.143286+0000","last_peered":"2024-04-26T21:35:15.143286+0000","last_clean":"2024-04-26T21:35:15.143286+0000","last_became_active":"2024-04-26T21:35:14.277458+0000","last_became_peered":"2024-04-26T21:35:14.277458+0000","last_unstale":"2024-04-26T21:35:15.143286+0000","last_undegraded":"2024-04-26T21:35:15.143286+0000","last_fullsized":"2024-04-26T21:35:15.143286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:55.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144138+0000","last_change":"2024-04-26T21:35:10.120150+0000","last_active":"2024-04-26T21:35:15.144138+0000","last_peered":"2024-04-26T21:35:15.144138+0000","last_clean":"2024-04-26T21:35:15.144138+0000","last_became_active":"2024-04-26T21:35:10.119894+0000","last_became_peered":"2024-04-26T21:35:10.119894+0000","last_unstale":"2024-04-26T21:35:15.144138+0000","last_undegraded":"2024-04-26T21:35:15.144138+0000","last_fullsized":"2024-04-26T21:35:15.144138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.413835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374128+0000","last_change":"2024-04-26T21:35:09.125791+0000","last_active":"2024-04-26T21:35:15.374128+0000","last_peered":"2024-04-26T21:35:15.374128+0000","last_clean":"2024-04-26T21:35:15.374128+0000","last_became_active":"2024-04-26T21:35:09.125356+0000","last_became_peered":"2024-04-26T21:35:09.125356+0000","last_unstale":"2024-04-26T21:35:15.374128+0000","last_undegraded":"2024-04-26T21:35:15.374128+0000","last_fullsized":"2024-04-26T21:35:15.374128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:25.035511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004470+0000","last_change":"2024-04-26T21:35:14.160611+0000","last_active":"2024-04-26T21:35:16.004470+0000","last_peered":"2024-04-26T21:35:16.004470+0000","last_clean":"2024-04-26T21:35:16.004470+0000","last_became_active":"2024-04-26T21:35:14.160067+0000","last_became_peered":"2024-04-26T21:35:14.160067+0000","last_unstale":"2024-04-26T21:35:16.004470+0000","last_undegraded":"2024-04-26T21:35:16.004470+0000","last_fullsized":"2024-04-26T21:35:16.004470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:37.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876144+0000","last_change":"2024-04-26T21:35:10.136587+0000","last_active":"2024-04-26T21:35:15.876144+0000","last_peered":"2024-04-26T21:35:15.876144+0000","last_clean":"2024-04-26T21:35:15.876144+0000","last_became_active":"2024-04-26T21:35:10.136271+0000","last_became_peered":"2024-04-26T21:35:10.136271+0000","last_unstale":"2024-04-26T21:35:15.876144+0000","last_undegraded":"2024-04-26T21:35:15.876144+0000","last_fullsized":"2024-04-26T21:35:15.876144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:45.927928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142500+0000","last_change":"2024-04-26T21:35:09.115543+0000","last_active":"2024-04-26T21:35:15.142500+0000","last_peered":"2024-04-26T21:35:15.142500+0000","last_clean":"2024-04-26T21:35:15.142500+0000","last_became_active":"2024-04-26T21:35:09.115159+0000","last_became_peered":"2024-04-26T21:35:09.115159+0000","last_unstale":"2024-04-26T21:35:15.142500+0000","last_undegraded":"2024-04-26T21:35:15.142500+0000","last_fullsized":"2024-04-26T21:35:15.142500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142639+0000","last_change":"2024-04-26T21:35:14.278004+0000","last_active":"2024-04-26T21:35:15.142639+0000","last_peered":"2024-04-26T21:35:15.142639+0000","last_clean":"2024-04-26T21:35:15.142639+0000","last_became_active":"2024-04-26T21:35:14.277547+0000","last_became_peered":"2024-04-26T21:35:14.277547+0000","last_unstale":"2024-04-26T21:35:15.142639+0000","last_undegraded":"2024-04-26T21:35:15.142639+0000","last_fullsized":"2024-04-26T21:35:15.142639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:49.955325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875902+0000","last_change":"2024-04-26T21:35:10.131439+0000","last_active":"2024-04-26T21:35:15.875902+0000","last_peered":"2024-04-26T21:35:15.875902+0000","last_clean":"2024-04-26T21:35:15.875902+0000","last_became_active":"2024-04-26T21:35:10.130864+0000","last_became_peered":"2024-04-26T21:35:10.130864+0000","last_unstale":"2024-04-26T21:35:15.875902+0000","last_undegraded":"2024-04-26T21:35:15.875902+0000","last_fullsized":"2024-04-26T21:35:15.875902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.130632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004324+0000","last_change":"2024-04-26T21:35:09.123483+0000","last_active":"2024-04-26T21:35:16.004324+0000","last_peered":"2024-04-26T21:35:16.004324+0000","last_clean":"2024-04-26T21:35:16.004324+0000","last_became_active":"2024-04-26T21:35:09.123202+0000","last_became_peered":"2024-04-26T21:35:09.123202+0000","last_unstale":"2024-04-26T21:35:16.004324+0000","last_undegraded":"2024-04-26T21:35:16.004324+0000","last_fullsized":"2024-04-26T21:35:16.004324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:50.000306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003596+0000","last_change":"2024-04-26T21:35:14.163479+0000","last_active":"2024-04-26T21:35:16.003596+0000","last_peered":"2024-04-26T21:35:16.003596+0000","last_clean":"2024-04-26T21:35:16.003596+0000","last_became_active":"2024-04-26T21:35:14.163041+0000","last_became_peered":"2024-04-26T21:35:14.163041+0000","last_unstale":"2024-04-26T21:35:16.003596+0000","last_undegraded":"2024-04-26T21:35:16.003596+0000","last_fullsized":"2024-04-26T21:35:16.003596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:34.682922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875665+0000","last_change":"2024-04-26T21:35:10.129858+0000","last_active":"2024-04-26T21:35:15.875665+0000","last_peered":"2024-04-26T21:35:15.875665+0000","last_clean":"2024-04-26T21:35:15.875665+0000","last_became_active":"2024-04-26T21:35:10.129606+0000","last_became_peered":"2024-04-26T21:35:10.129606+0000","last_unstale":"2024-04-26T21:35:15.875665+0000","last_undegraded":"2024-04-26T21:35:15.875665+0000","last_fullsized":"2024-04-26T21:35:15.875665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.413835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.141883+0000","last_change":"2024-04-26T21:35:09.121200+0000","last_active":"2024-04-26T21:35:15.141883+0000","last_peered":"2024-04-26T21:35:15.141883+0000","last_clean":"2024-04-26T21:35:15.141883+0000","last_became_active":"2024-04-26T21:35:09.120700+0000","last_became_peered":"2024-04-26T21:35:09.120700+0000","last_unstale":"2024-04-26T21:35:15.141883+0000","last_undegraded":"2024-04-26T21:35:15.141883+0000","last_fullsized":"2024-04-26T21:35:15.141883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003924+0000","last_change":"2024-04-26T21:35:14.163658+0000","last_active":"2024-04-26T21:35:16.003924+0000","last_peered":"2024-04-26T21:35:16.003924+0000","last_clean":"2024-04-26T21:35:16.003924+0000","last_became_active":"2024-04-26T21:35:14.163215+0000","last_became_peered":"2024-04-26T21:35:14.163215+0000","last_unstale":"2024-04-26T21:35:16.003924+0000","last_undegraded":"2024-04-26T21:35:16.003924+0000","last_fullsized":"2024-04-26T21:35:16.003924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:01.158030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142807+0000","last_change":"2024-04-26T21:35:10.126750+0000","last_active":"2024-04-26T21:35:15.142807+0000","last_peered":"2024-04-26T21:35:15.142807+0000","last_clean":"2024-04-26T21:35:15.142807+0000","last_became_active":"2024-04-26T21:35:10.126096+0000","last_became_peered":"2024-04-26T21:35:10.126096+0000","last_unstale":"2024-04-26T21:35:15.142807+0000","last_undegraded":"2024-04-26T21:35:15.142807+0000","last_fullsized":"2024-04-26T21:35:15.142807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:46.899885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143301+0000","last_change":"2024-04-26T21:35:09.122401+0000","last_active":"2024-04-26T21:35:15.143301+0000","last_peered":"2024-04-26T21:35:15.143301+0000","last_clean":"2024-04-26T21:35:15.143301+0000","last_became_active":"2024-04-26T21:35:09.122067+0000","last_became_peered":"2024-04-26T21:35:09.122067+0000","last_unstale":"2024-04-26T21:35:15.143301+0000","last_undegraded":"2024-04-26T21:35:15.143301+0000","last_fullsized":"2024-04-26T21:35:15.143301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004812+0000","last_change":"2024-04-26T21:35:14.279383+0000","last_active":"2024-04-26T21:35:16.004812+0000","last_peered":"2024-04-26T21:35:16.004812+0000","last_clean":"2024-04-26T21:35:16.004812+0000","last_became_active":"2024-04-26T21:35:14.278985+0000","last_became_peered":"2024-04-26T21:35:14.278985+0000","last_unstale":"2024-04-26T21:35:16.004812+0000","last_undegraded":"2024-04-26T21:35:16.004812+0000","last_fullsized":"2024-04-26T21:35:16.004812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.757335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875771+0000","last_change":"2024-04-26T21:35:10.136295+0000","last_active":"2024-04-26T21:35:15.875771+0000","last_peered":"2024-04-26T21:35:15.875771+0000","last_clean":"2024-04-26T21:35:15.875771+0000","last_became_active":"2024-04-26T21:35:10.135975+0000","last_became_peered":"2024-04-26T21:35:10.135975+0000","last_unstale":"2024-04-26T21:35:15.875771+0000","last_undegraded":"2024-04-26T21:35:15.875771+0000","last_fullsized":"2024-04-26T21:35:15.875771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144056+0000","last_change":"2024-04-26T21:35:09.122493+0000","last_active":"2024-04-26T21:35:15.144056+0000","last_peered":"2024-04-26T21:35:15.144056+0000","last_clean":"2024-04-26T21:35:15.144056+0000","last_became_active":"2024-04-26T21:35:09.122211+0000","last_became_peered":"2024-04-26T21:35:09.122211+0000","last_unstale":"2024-04-26T21:35:15.144056+0000","last_undegraded":"2024-04-26T21:35:15.144056+0000","last_fullsized":"2024-04-26T21:35:15.144056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004937+0000","last_change":"2024-04-26T21:35:14.277801+0000","last_active":"2024-04-26T21:35:16.004937+0000","last_peered":"2024-04-26T21:35:16.004937+0000","last_clean":"2024-04-26T21:35:16.004937+0000","last_became_active":"2024-04-26T21:35:14.277417+0000","last_became_peered":"2024-04-26T21:35:14.277417+0000","last_unstale":"2024-04-26T21:35:16.004937+0000","last_undegraded":"2024-04-26T21:35:16.004937+0000","last_fullsized":"2024-04-26T21:35:16.004937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:15.246664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004133+0000","last_change":"2024-04-26T21:35:10.128386+0000","last_active":"2024-04-26T21:35:16.004133+0000","last_peered":"2024-04-26T21:35:16.004133+0000","last_clean":"2024-04-26T21:35:16.004133+0000","last_became_active":"2024-04-26T21:35:10.127067+0000","last_became_peered":"2024-04-26T21:35:10.127067+0000","last_unstale":"2024-04-26T21:35:16.004133+0000","last_undegraded":"2024-04-26T21:35:16.004133+0000","last_fullsized":"2024-04-26T21:35:16.004133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:46.899885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004551+0000","last_change":"2024-04-26T21:35:09.125422+0000","last_active":"2024-04-26T21:35:16.004551+0000","last_peered":"2024-04-26T21:35:16.004551+0000","last_clean":"2024-04-26T21:35:16.004551+0000","last_became_active":"2024-04-26T21:35:09.125086+0000","last_became_peered":"2024-04-26T21:35:09.125086+0000","last_unstale":"2024-04-26T21:35:16.004551+0000","last_undegraded":"2024-04-26T21:35:16.004551+0000","last_fullsized":"2024-04-26T21:35:16.004551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:44.358668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144502+0000","last_change":"2024-04-26T21:35:14.278312+0000","last_active":"2024-04-26T21:35:15.144502+0000","last_peered":"2024-04-26T21:35:15.144502+0000","last_clean":"2024-04-26T21:35:15.144502+0000","last_became_active":"2024-04-26T21:35:14.277836+0000","last_became_peered":"2024-04-26T21:35:14.277836+0000","last_unstale":"2024-04-26T21:35:15.144502+0000","last_undegraded":"2024-04-26T21:35:15.144502+0000","last_fullsized":"2024-04-26T21:35:15.144502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:20.215438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875616+0000","last_change":"2024-04-26T21:35:10.126791+0000","last_active":"2024-04-26T21:35:15.875616+0000","last_peered":"2024-04-26T21:35:15.875616+0000","last_clean":"2024-04-26T21:35:15.875616+0000","last_became_active":"2024-04-26T21:35:10.126515+0000","last_became_peered":"2024-04-26T21:35:10.126515+0000","last_unstale":"2024-04-26T21:35:15.875616+0000","last_undegraded":"2024-04-26T21:35:15.875616+0000","last_fullsized":"2024-04-26T21:35:15.875616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:01.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:19.559960+0000","last_change":"2024-04-26T21:35:09.126680+0000","last_active":"2024-04-26T21:35:19.559960+0000","last_peered":"2024-04-26T21:35:19.559960+0000","last_clean":"2024-04-26T21:35:19.559960+0000","last_became_active":"2024-04-26T21:35:09.126406+0000","last_became_peered":"2024-04-26T21:35:09.126406+0000","last_unstale":"2024-04-26T21:35:19.559960+0000","last_undegraded":"2024-04-26T21:35:19.559960+0000","last_fullsized":"2024-04-26T21:35:19.559960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373762+0000","last_change":"2024-04-26T21:35:14.163960+0000","last_active":"2024-04-26T21:35:15.373762+0000","last_peered":"2024-04-26T21:35:15.373762+0000","last_clean":"2024-04-26T21:35:15.373762+0000","last_became_active":"2024-04-26T21:35:14.163551+0000","last_became_peered":"2024-04-26T21:35:14.163551+0000","last_unstale":"2024-04-26T21:35:15.373762+0000","last_undegraded":"2024-04-26T21:35:15.373762+0000","last_fullsized":"2024-04-26T21:35:15.373762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004282+0000","last_change":"2024-04-26T21:35:10.129151+0000","last_active":"2024-04-26T21:35:16.004282+0000","last_peered":"2024-04-26T21:35:16.004282+0000","last_clean":"2024-04-26T21:35:16.004282+0000","last_became_active":"2024-04-26T21:35:10.128520+0000","last_became_peered":"2024-04-26T21:35:10.128520+0000","last_unstale":"2024-04-26T21:35:16.004282+0000","last_undegraded":"2024-04-26T21:35:16.004282+0000","last_fullsized":"2024-04-26T21:35:16.004282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:17.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374063+0000","last_change":"2024-04-26T21:35:09.128170+0000","last_active":"2024-04-26T21:35:15.374063+0000","last_peered":"2024-04-26T21:35:15.374063+0000","last_clean":"2024-04-26T21:35:15.374063+0000","last_became_active":"2024-04-26T21:35:09.127817+0000","last_became_peered":"2024-04-26T21:35:09.127817+0000","last_unstale":"2024-04-26T21:35:15.374063+0000","last_undegraded":"2024-04-26T21:35:15.374063+0000","last_fullsized":"2024-04-26T21:35:15.374063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004297+0000","last_change":"2024-04-26T21:35:14.157290+0000","last_active":"2024-04-26T21:35:16.004297+0000","last_peered":"2024-04-26T21:35:16.004297+0000","last_clean":"2024-04-26T21:35:16.004297+0000","last_became_active":"2024-04-26T21:35:14.156794+0000","last_became_peered":"2024-04-26T21:35:14.156794+0000","last_unstale":"2024-04-26T21:35:16.004297+0000","last_undegraded":"2024-04-26T21:35:16.004297+0000","last_fullsized":"2024-04-26T21:35:16.004297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:55.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004165+0000","last_change":"2024-04-26T21:35:10.132571+0000","last_active":"2024-04-26T21:35:16.004165+0000","last_peered":"2024-04-26T21:35:16.004165+0000","last_clean":"2024-04-26T21:35:16.004165+0000","last_became_active":"2024-04-26T21:35:10.132311+0000","last_became_peered":"2024-04-26T21:35:10.132311+0000","last_unstale":"2024-04-26T21:35:16.004165+0000","last_undegraded":"2024-04-26T21:35:16.004165+0000","last_fullsized":"2024-04-26T21:35:16.004165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143720+0000","last_change":"2024-04-26T21:35:09.124883+0000","last_active":"2024-04-26T21:35:15.143720+0000","last_peered":"2024-04-26T21:35:15.143720+0000","last_clean":"2024-04-26T21:35:15.143720+0000","last_became_active":"2024-04-26T21:35:09.124576+0000","last_became_peered":"2024-04-26T21:35:09.124576+0000","last_unstale":"2024-04-26T21:35:15.143720+0000","last_undegraded":"2024-04-26T21:35:15.143720+0000","last_fullsized":"2024-04-26T21:35:15.143720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003494+0000","last_change":"2024-04-26T21:35:14.158063+0000","last_active":"2024-04-26T21:35:16.003494+0000","last_peered":"2024-04-26T21:35:16.003494+0000","last_clean":"2024-04-26T21:35:16.003494+0000","last_became_active":"2024-04-26T21:35:14.157033+0000","last_became_peered":"2024-04-26T21:35:14.157033+0000","last_unstale":"2024-04-26T21:35:16.003494+0000","last_undegraded":"2024-04-26T21:35:16.003494+0000","last_fullsized":"2024-04-26T21:35:16.003494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:20.215438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143604+0000","last_change":"2024-04-26T21:35:10.121823+0000","last_active":"2024-04-26T21:35:15.143604+0000","last_peered":"2024-04-26T21:35:15.143604+0000","last_clean":"2024-04-26T21:35:15.143604+0000","last_became_active":"2024-04-26T21:35:10.121518+0000","last_became_peered":"2024-04-26T21:35:10.121518+0000","last_unstale":"2024-04-26T21:35:15.143604+0000","last_undegraded":"2024-04-26T21:35:15.143604+0000","last_fullsized":"2024-04-26T21:35:15.143604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:06.354261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143861+0000","last_change":"2024-04-26T21:35:09.127420+0000","last_active":"2024-04-26T21:35:15.143861+0000","last_peered":"2024-04-26T21:35:15.143861+0000","last_clean":"2024-04-26T21:35:15.143861+0000","last_became_active":"2024-04-26T21:35:09.127104+0000","last_became_peered":"2024-04-26T21:35:09.127104+0000","last_unstale":"2024-04-26T21:35:15.143861+0000","last_undegraded":"2024-04-26T21:35:15.143861+0000","last_fullsized":"2024-04-26T21:35:15.143861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004551+0000","last_change":"2024-04-26T21:35:14.281132+0000","last_active":"2024-04-26T21:35:16.004551+0000","last_peered":"2024-04-26T21:35:16.004551+0000","last_clean":"2024-04-26T21:35:16.004551+0000","last_became_active":"2024-04-26T21:35:14.280538+0000","last_became_peered":"2024-04-26T21:35:14.280538+0000","last_unstale":"2024-04-26T21:35:16.004551+0000","last_undegraded":"2024-04-26T21:35:16.004551+0000","last_fullsized":"2024-04-26T21:35:16.004551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:37.157683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003564+0000","last_change":"2024-04-26T21:35:14.279953+0000","last_active":"2024-04-26T21:35:16.003564+0000","last_peered":"2024-04-26T21:35:16.003564+0000","last_clean":"2024-04-26T21:35:16.003564+0000","last_became_active":"2024-04-26T21:35:14.279540+0000","last_became_peered":"2024-04-26T21:35:14.279540+0000","last_unstale":"2024-04-26T21:35:16.003564+0000","last_undegraded":"2024-04-26T21:35:16.003564+0000","last_fullsized":"2024-04-26T21:35:16.003564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373464+0000","last_change":"2024-04-26T21:35:10.121201+0000","last_active":"2024-04-26T21:35:15.373464+0000","last_peered":"2024-04-26T21:35:15.373464+0000","last_clean":"2024-04-26T21:35:15.373464+0000","last_became_active":"2024-04-26T21:35:10.120934+0000","last_became_peered":"2024-04-26T21:35:10.120934+0000","last_unstale":"2024-04-26T21:35:15.373464+0000","last_undegraded":"2024-04-26T21:35:15.373464+0000","last_fullsized":"2024-04-26T21:35:15.373464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142787+0000","last_change":"2024-04-26T21:35:09.119199+0000","last_active":"2024-04-26T21:35:15.142787+0000","last_peered":"2024-04-26T21:35:15.142787+0000","last_clean":"2024-04-26T21:35:15.142787+0000","last_became_active":"2024-04-26T21:35:09.118431+0000","last_became_peered":"2024-04-26T21:35:09.118431+0000","last_unstale":"2024-04-26T21:35:15.142787+0000","last_undegraded":"2024-04-26T21:35:15.142787+0000","last_fullsized":"2024-04-26T21:35:15.142787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144043+0000","last_change":"2024-04-26T21:35:14.160976+0000","last_active":"2024-04-26T21:35:15.144043+0000","last_peered":"2024-04-26T21:35:15.144043+0000","last_clean":"2024-04-26T21:35:15.144043+0000","last_became_active":"2024-04-26T21:35:14.160609+0000","last_became_peered":"2024-04-26T21:35:14.160609+0000","last_unstale":"2024-04-26T21:35:15.144043+0000","last_undegraded":"2024-04-26T21:35:15.144043+0000","last_fullsized":"2024-04-26T21:35:15.144043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003542+0000","last_change":"2024-04-26T21:35:10.134441+0000","last_active":"2024-04-26T21:35:16.003542+0000","last_peered":"2024-04-26T21:35:16.003542+0000","last_clean":"2024-04-26T21:35:16.003542+0000","last_became_active":"2024-04-26T21:35:10.134101+0000","last_became_peered":"2024-04-26T21:35:10.134101+0000","last_unstale":"2024-04-26T21:35:16.003542+0000","last_undegraded":"2024-04-26T21:35:16.003542+0000","last_fullsized":"2024-04-26T21:35:16.003542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004713+0000","last_change":"2024-04-26T21:35:09.124939+0000","last_active":"2024-04-26T21:35:16.004713+0000","last_peered":"2024-04-26T21:35:16.004713+0000","last_clean":"2024-04-26T21:35:16.004713+0000","last_became_active":"2024-04-26T21:35:09.124612+0000","last_became_peered":"2024-04-26T21:35:09.124612+0000","last_unstale":"2024-04-26T21:35:16.004713+0000","last_undegraded":"2024-04-26T21:35:16.004713+0000","last_fullsized":"2024-04-26T21:35:16.004713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143484+0000","last_change":"2024-04-26T21:35:09.122510+0000","last_active":"2024-04-26T21:35:15.143484+0000","last_peered":"2024-04-26T21:35:15.143484+0000","last_clean":"2024-04-26T21:35:15.143484+0000","last_became_active":"2024-04-26T21:35:09.122161+0000","last_became_peered":"2024-04-26T21:35:09.122161+0000","last_unstale":"2024-04-26T21:35:15.143484+0000","last_undegraded":"2024-04-26T21:35:15.143484+0000","last_fullsized":"2024-04-26T21:35:15.143484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004522+0000","last_change":"2024-04-26T21:35:10.128884+0000","last_active":"2024-04-26T21:35:16.004522+0000","last_peered":"2024-04-26T21:35:16.004522+0000","last_clean":"2024-04-26T21:35:16.004522+0000","last_became_active":"2024-04-26T21:35:10.128599+0000","last_became_peered":"2024-04-26T21:35:10.128599+0000","last_unstale":"2024-04-26T21:35:16.004522+0000","last_undegraded":"2024-04-26T21:35:16.004522+0000","last_fullsized":"2024-04-26T21:35:16.004522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142294+0000","last_change":"2024-04-26T21:35:14.158429+0000","last_active":"2024-04-26T21:35:15.142294+0000","last_peered":"2024-04-26T21:35:15.142294+0000","last_clean":"2024-04-26T21:35:15.142294+0000","last_became_active":"2024-04-26T21:35:14.158011+0000","last_became_peered":"2024-04-26T21:35:14.158011+0000","last_unstale":"2024-04-26T21:35:15.142294+0000","last_undegraded":"2024-04-26T21:35:15.142294+0000","last_fullsized":"2024-04-26T21:35:15.142294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:44.919887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004133+0000","last_change":"2024-04-26T21:35:09.127502+0000","last_active":"2024-04-26T21:35:16.004133+0000","last_peered":"2024-04-26T21:35:16.004133+0000","last_clean":"2024-04-26T21:35:16.004133+0000","last_became_active":"2024-04-26T21:35:09.127182+0000","last_became_peered":"2024-04-26T21:35:09.127182+0000","last_unstale":"2024-04-26T21:35:16.004133+0000","last_undegraded":"2024-04-26T21:35:16.004133+0000","last_fullsized":"2024-04-26T21:35:16.004133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003916+0000","last_change":"2024-04-26T21:35:10.120022+0000","last_active":"2024-04-26T21:35:16.003916+0000","last_peered":"2024-04-26T21:35:16.003916+0000","last_clean":"2024-04-26T21:35:16.003916+0000","last_became_active":"2024-04-26T21:35:10.119759+0000","last_became_peered":"2024-04-26T21:35:10.119759+0000","last_unstale":"2024-04-26T21:35:16.003916+0000","last_undegraded":"2024-04-26T21:35:16.003916+0000","last_fullsized":"2024-04-26T21:35:16.003916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:55.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004467+0000","last_change":"2024-04-26T21:35:14.157203+0000","last_active":"2024-04-26T21:35:16.004467+0000","last_peered":"2024-04-26T21:35:16.004467+0000","last_clean":"2024-04-26T21:35:16.004467+0000","last_became_active":"2024-04-26T21:35:14.156742+0000","last_became_peered":"2024-04-26T21:35:14.156742+0000","last_unstale":"2024-04-26T21:35:16.004467+0000","last_undegraded":"2024-04-26T21:35:16.004467+0000","last_fullsized":"2024-04-26T21:35:16.004467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:13.631171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142673+0000","last_change":"2024-04-26T21:35:10.126924+0000","last_active":"2024-04-26T21:35:15.142673+0000","last_peered":"2024-04-26T21:35:15.142673+0000","last_clean":"2024-04-26T21:35:15.142673+0000","last_became_active":"2024-04-26T21:35:10.126660+0000","last_became_peered":"2024-04-26T21:35:10.126660+0000","last_unstale":"2024-04-26T21:35:15.142673+0000","last_undegraded":"2024-04-26T21:35:15.142673+0000","last_fullsized":"2024-04-26T21:35:15.142673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:29.764634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142006+0000","last_change":"2024-04-26T21:35:10.116329+0000","last_active":"2024-04-26T21:35:15.142006+0000","last_peered":"2024-04-26T21:35:15.142006+0000","last_clean":"2024-04-26T21:35:15.142006+0000","last_became_active":"2024-04-26T21:35:10.116047+0000","last_became_peered":"2024-04-26T21:35:10.116047+0000","last_unstale":"2024-04-26T21:35:15.142006+0000","last_undegraded":"2024-04-26T21:35:15.142006+0000","last_fullsized":"2024-04-26T21:35:15.142006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:17.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143917+0000","last_change":"2024-04-26T21:35:10.121375+0000","last_active":"2024-04-26T21:35:15.143917+0000","last_peered":"2024-04-26T21:35:15.143917+0000","last_clean":"2024-04-26T21:35:15.143917+0000","last_became_active":"2024-04-26T21:35:10.121088+0000","last_became_peered":"2024-04-26T21:35:10.121088+0000","last_unstale":"2024-04-26T21:35:15.143917+0000","last_undegraded":"2024-04-26T21:35:15.143917+0000","last_fullsized":"2024-04-26T21:35:15.143917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004457+0000","last_change":"2024-04-26T21:35:10.131911+0000","last_active":"2024-04-26T21:35:16.004457+0000","last_peered":"2024-04-26T21:35:16.004457+0000","last_clean":"2024-04-26T21:35:16.004457+0000","last_became_active":"2024-04-26T21:35:10.131477+0000","last_became_peered":"2024-04-26T21:35:10.131477+0000","last_unstale":"2024-04-26T21:35:16.004457+0000","last_undegraded":"2024-04-26T21:35:16.004457+0000","last_fullsized":"2024-04-26T21:35:16.004457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.005022+0000","last_change":"2024-04-26T21:35:10.121507+0000","last_active":"2024-04-26T21:35:16.005022+0000","last_peered":"2024-04-26T21:35:16.005022+0000","last_clean":"2024-04-26T21:35:16.005022+0000","last_became_active":"2024-04-26T21:35:10.121208+0000","last_became_peered":"2024-04-26T21:35:10.121208+0000","last_unstale":"2024-04-26T21:35:16.005022+0000","last_undegraded":"2024-04-26T21:35:16.005022+0000","last_fullsized":"2024-04-26T21:35:16.005022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373872+0000","last_change":"2024-04-26T21:35:10.121106+0000","last_active":"2024-04-26T21:35:15.373872+0000","last_peered":"2024-04-26T21:35:15.373872+0000","last_clean":"2024-04-26T21:35:15.373872+0000","last_became_active":"2024-04-26T21:35:10.120857+0000","last_became_peered":"2024-04-26T21:35:10.120857+0000","last_unstale":"2024-04-26T21:35:15.373872+0000","last_undegraded":"2024-04-26T21:35:15.373872+0000","last_fullsized":"2024-04-26T21:35:15.373872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143866+0000","last_change":"2024-04-26T21:35:10.128212+0000","last_active":"2024-04-26T21:35:15.143866+0000","last_peered":"2024-04-26T21:35:15.143866+0000","last_clean":"2024-04-26T21:35:15.143866+0000","last_became_active":"2024-04-26T21:35:10.127868+0000","last_became_peered":"2024-04-26T21:35:10.127868+0000","last_unstale":"2024-04-26T21:35:15.143866+0000","last_undegraded":"2024-04-26T21:35:15.143866+0000","last_fullsized":"2024-04-26T21:35:15.143866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004798+0000","last_change":"2024-04-26T21:35:10.130695+0000","last_active":"2024-04-26T21:35:16.004798+0000","last_peered":"2024-04-26T21:35:16.004798+0000","last_clean":"2024-04-26T21:35:16.004798+0000","last_became_active":"2024-04-26T21:35:10.130084+0000","last_became_peered":"2024-04-26T21:35:10.130084+0000","last_unstale":"2024-04-26T21:35:16.004798+0000","last_undegraded":"2024-04-26T21:35:16.004798+0000","last_fullsized":"2024-04-26T21:35:16.004798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142712+0000","last_change":"2024-04-26T21:35:10.122732+0000","last_active":"2024-04-26T21:35:15.142712+0000","last_peered":"2024-04-26T21:35:15.142712+0000","last_clean":"2024-04-26T21:35:15.142712+0000","last_became_active":"2024-04-26T21:35:10.122457+0000","last_became_peered":"2024-04-26T21:35:10.122457+0000","last_unstale":"2024-04-26T21:35:15.142712+0000","last_undegraded":"2024-04-26T21:35:15.142712+0000","last_fullsized":"2024-04-26T21:35:15.142712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004615+0000","last_change":"2024-04-26T21:35:10.131503+0000","last_active":"2024-04-26T21:35:16.004615+0000","last_peered":"2024-04-26T21:35:16.004615+0000","last_clean":"2024-04-26T21:35:16.004615+0000","last_became_active":"2024-04-26T21:35:10.131262+0000","last_became_peered":"2024-04-26T21:35:10.131262+0000","last_unstale":"2024-04-26T21:35:16.004615+0000","last_undegraded":"2024-04-26T21:35:16.004615+0000","last_fullsized":"2024-04-26T21:35:16.004615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143913+0000","last_change":"2024-04-26T21:35:10.122046+0000","last_active":"2024-04-26T21:35:15.143913+0000","last_peered":"2024-04-26T21:35:15.143913+0000","last_clean":"2024-04-26T21:35:15.143913+0000","last_became_active":"2024-04-26T21:35:10.121759+0000","last_became_peered":"2024-04-26T21:35:10.121759+0000","last_unstale":"2024-04-26T21:35:15.143913+0000","last_undegraded":"2024-04-26T21:35:15.143913+0000","last_fullsized":"2024-04-26T21:35:15.143913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004361+0000","last_change":"2024-04-26T21:35:10.128039+0000","last_active":"2024-04-26T21:35:16.004361+0000","last_peered":"2024-04-26T21:35:16.004361+0000","last_clean":"2024-04-26T21:35:16.004361+0000","last_became_active":"2024-04-26T21:35:10.126920+0000","last_became_peered":"2024-04-26T21:35:10.126920+0000","last_unstale":"2024-04-26T21:35:16.004361+0000","last_undegraded":"2024-04-26T21:35:16.004361+0000","last_fullsized":"2024-04-26T21:35:16.004361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143654+0000","last_change":"2024-04-26T21:35:10.120371+0000","last_active":"2024-04-26T21:35:15.143654+0000","last_peered":"2024-04-26T21:35:15.143654+0000","last_clean":"2024-04-26T21:35:15.143654+0000","last_became_active":"2024-04-26T21:35:10.120020+0000","last_became_peered":"2024-04-26T21:35:10.120020+0000","last_unstale":"2024-04-26T21:35:15.143654+0000","last_undegraded":"2024-04-26T21:35:15.143654+0000","last_fullsized":"2024-04-26T21:35:15.143654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876023+0000","last_change":"2024-04-26T21:35:10.128002+0000","last_active":"2024-04-26T21:35:15.876023+0000","last_peered":"2024-04-26T21:35:15.876023+0000","last_clean":"2024-04-26T21:35:15.876023+0000","last_became_active":"2024-04-26T21:35:10.127763+0000","last_became_peered":"2024-04-26T21:35:10.127763+0000","last_unstale":"2024-04-26T21:35:15.876023+0000","last_undegraded":"2024-04-26T21:35:15.876023+0000","last_fullsized":"2024-04-26T21:35:15.876023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:09.615472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142501+0000","last_change":"2024-04-26T21:35:10.118765+0000","last_active":"2024-04-26T21:35:15.142501+0000","last_peered":"2024-04-26T21:35:15.142501+0000","last_clean":"2024-04-26T21:35:15.142501+0000","last_became_active":"2024-04-26T21:35:10.118481+0000","last_became_peered":"2024-04-26T21:35:10.118481+0000","last_unstale":"2024-04-26T21:35:15.142501+0000","last_undegraded":"2024-04-26T21:35:15.142501+0000","last_fullsized":"2024-04-26T21:35:15.142501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:55.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144424+0000","last_change":"2024-04-26T21:35:10.123877+0000","last_active":"2024-04-26T21:35:15.144424+0000","last_peered":"2024-04-26T21:35:15.144424+0000","last_clean":"2024-04-26T21:35:15.144424+0000","last_became_active":"2024-04-26T21:35:10.122628+0000","last_became_peered":"2024-04-26T21:35:10.122628+0000","last_unstale":"2024-04-26T21:35:15.144424+0000","last_undegraded":"2024-04-26T21:35:15.144424+0000","last_fullsized":"2024-04-26T21:35:15.144424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142605+0000","last_change":"2024-04-26T21:35:10.133644+0000","last_active":"2024-04-26T21:35:15.142605+0000","last_peered":"2024-04-26T21:35:15.142605+0000","last_clean":"2024-04-26T21:35:15.142605+0000","last_became_active":"2024-04-26T21:35:10.133364+0000","last_became_peered":"2024-04-26T21:35:10.133364+0000","last_unstale":"2024-04-26T21:35:15.142605+0000","last_undegraded":"2024-04-26T21:35:15.142605+0000","last_fullsized":"2024-04-26T21:35:15.142605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373575+0000","last_change":"2024-04-26T21:35:10.130778+0000","last_active":"2024-04-26T21:35:15.373575+0000","last_peered":"2024-04-26T21:35:15.373575+0000","last_clean":"2024-04-26T21:35:15.373575+0000","last_became_active":"2024-04-26T21:35:10.130501+0000","last_became_peered":"2024-04-26T21:35:10.130501+0000","last_unstale":"2024-04-26T21:35:15.373575+0000","last_undegraded":"2024-04-26T21:35:15.373575+0000","last_fullsized":"2024-04-26T21:35:15.373575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004023+0000","last_change":"2024-04-26T21:35:10.134920+0000","last_active":"2024-04-26T21:35:16.004023+0000","last_peered":"2024-04-26T21:35:16.004023+0000","last_clean":"2024-04-26T21:35:16.004023+0000","last_became_active":"2024-04-26T21:35:10.134647+0000","last_became_peered":"2024-04-26T21:35:10.134647+0000","last_unstale":"2024-04-26T21:35:16.004023+0000","last_undegraded":"2024-04-26T21:35:16.004023+0000","last_fullsized":"2024-04-26T21:35:16.004023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:19.754278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003968+0000","last_change":"2024-04-26T21:35:10.118720+0000","last_active":"2024-04-26T21:35:16.003968+0000","last_peered":"2024-04-26T21:35:16.003968+0000","last_clean":"2024-04-26T21:35:16.003968+0000","last_became_active":"2024-04-26T21:35:10.118418+0000","last_became_peered":"2024-04-26T21:35:10.118418+0000","last_unstale":"2024-04-26T21:35:16.003968+0000","last_undegraded":"2024-04-26T21:35:16.003968+0000","last_fullsized":"2024-04-26T21:35:16.003968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004237+0000","last_change":"2024-04-26T21:35:10.118157+0000","last_active":"2024-04-26T21:35:16.004237+0000","last_peered":"2024-04-26T21:35:16.004237+0000","last_clean":"2024-04-26T21:35:16.004237+0000","last_became_active":"2024-04-26T21:35:10.117862+0000","last_became_peered":"2024-04-26T21:35:10.117862+0000","last_unstale":"2024-04-26T21:35:16.004237+0000","last_undegraded":"2024-04-26T21:35:16.004237+0000","last_fullsized":"2024-04-26T21:35:16.004237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:26.040140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143343+0000","last_change":"2024-04-26T21:35:10.124503+0000","last_active":"2024-04-26T21:35:15.143343+0000","last_peered":"2024-04-26T21:35:15.143343+0000","last_clean":"2024-04-26T21:35:15.143343+0000","last_became_active":"2024-04-26T21:35:10.123919+0000","last_became_peered":"2024-04-26T21:35:10.123919+0000","last_unstale":"2024-04-26T21:35:15.143343+0000","last_undegraded":"2024-04-26T21:35:15.143343+0000","last_fullsized":"2024-04-26T21:35:15.143343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143962+0000","last_change":"2024-04-26T21:35:10.124821+0000","last_active":"2024-04-26T21:35:15.143962+0000","last_peered":"2024-04-26T21:35:15.143962+0000","last_clean":"2024-04-26T21:35:15.143962+0000","last_became_active":"2024-04-26T21:35:10.124245+0000","last_became_peered":"2024-04-26T21:35:10.124245+0000","last_unstale":"2024-04-26T21:35:15.143962+0000","last_undegraded":"2024-04-26T21:35:15.143962+0000","last_fullsized":"2024-04-26T21:35:15.143962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876099+0000","last_change":"2024-04-26T21:35:10.131624+0000","last_active":"2024-04-26T21:35:15.876099+0000","last_peered":"2024-04-26T21:35:15.876099+0000","last_clean":"2024-04-26T21:35:15.876099+0000","last_became_active":"2024-04-26T21:35:10.131392+0000","last_became_peered":"2024-04-26T21:35:10.131392+0000","last_unstale":"2024-04-26T21:35:15.876099+0000","last_undegraded":"2024-04-26T21:35:15.876099+0000","last_fullsized":"2024-04-26T21:35:15.876099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:33.396034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003337+0000","last_change":"2024-04-26T21:35:10.127875+0000","last_active":"2024-04-26T21:35:16.003337+0000","last_peered":"2024-04-26T21:35:16.003337+0000","last_clean":"2024-04-26T21:35:16.003337+0000","last_became_active":"2024-04-26T21:35:10.126566+0000","last_became_peered":"2024-04-26T21:35:10.126566+0000","last_unstale":"2024-04-26T21:35:16.003337+0000","last_undegraded":"2024-04-26T21:35:16.003337+0000","last_fullsized":"2024-04-26T21:35:16.003337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875494+0000","last_change":"2024-04-26T21:35:10.126677+0000","last_active":"2024-04-26T21:35:15.875494+0000","last_peered":"2024-04-26T21:35:15.875494+0000","last_clean":"2024-04-26T21:35:15.875494+0000","last_became_active":"2024-04-26T21:35:10.126388+0000","last_became_peered":"2024-04-26T21:35:10.126388+0000","last_unstale":"2024-04-26T21:35:15.875494+0000","last_undegraded":"2024-04-26T21:35:15.875494+0000","last_fullsized":"2024-04-26T21:35:15.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143757+0000","last_change":"2024-04-26T21:35:10.125406+0000","last_active":"2024-04-26T21:35:15.143757+0000","last_peered":"2024-04-26T21:35:15.143757+0000","last_clean":"2024-04-26T21:35:15.143757+0000","last_became_active":"2024-04-26T21:35:10.125143+0000","last_became_peered":"2024-04-26T21:35:10.125143+0000","last_unstale":"2024-04-26T21:35:15.143757+0000","last_undegraded":"2024-04-26T21:35:15.143757+0000","last_fullsized":"2024-04-26T21:35:15.143757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.130632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875494+0000","last_change":"2024-04-26T21:35:10.127947+0000","last_active":"2024-04-26T21:35:15.875494+0000","last_peered":"2024-04-26T21:35:15.875494+0000","last_clean":"2024-04-26T21:35:15.875494+0000","last_became_active":"2024-04-26T21:35:10.127700+0000","last_became_peered":"2024-04-26T21:35:10.127700+0000","last_unstale":"2024-04-26T21:35:15.875494+0000","last_undegraded":"2024-04-26T21:35:15.875494+0000","last_fullsized":"2024-04-26T21:35:15.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143721+0000","last_change":"2024-04-26T21:35:10.121445+0000","last_active":"2024-04-26T21:35:15.143721+0000","last_peered":"2024-04-26T21:35:15.143721+0000","last_clean":"2024-04-26T21:35:15.143721+0000","last_became_active":"2024-04-26T21:35:10.119356+0000","last_became_peered":"2024-04-26T21:35:10.119356+0000","last_unstale":"2024-04-26T21:35:15.143721+0000","last_undegraded":"2024-04-26T21:35:15.143721+0000","last_fullsized":"2024-04-26T21:35:15.143721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:27.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003601+0000","last_change":"2024-04-26T21:35:10.124758+0000","last_active":"2024-04-26T21:35:16.003601+0000","last_peered":"2024-04-26T21:35:16.003601+0000","last_clean":"2024-04-26T21:35:16.003601+0000","last_became_active":"2024-04-26T21:35:10.124480+0000","last_became_peered":"2024-04-26T21:35:10.124480+0000","last_unstale":"2024-04-26T21:35:16.003601+0000","last_undegraded":"2024-04-26T21:35:16.003601+0000","last_fullsized":"2024-04-26T21:35:16.003601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:29.764634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374098+0000","last_change":"2024-04-26T21:35:10.125937+0000","last_active":"2024-04-26T21:35:15.374098+0000","last_peered":"2024-04-26T21:35:15.374098+0000","last_clean":"2024-04-26T21:35:15.374098+0000","last_became_active":"2024-04-26T21:35:10.125640+0000","last_became_peered":"2024-04-26T21:35:10.125640+0000","last_unstale":"2024-04-26T21:35:15.374098+0000","last_undegraded":"2024-04-26T21:35:15.374098+0000","last_fullsized":"2024-04-26T21:35:15.374098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876035+0000","last_change":"2024-04-26T21:35:10.129050+0000","last_active":"2024-04-26T21:35:15.876035+0000","last_peered":"2024-04-26T21:35:15.876035+0000","last_clean":"2024-04-26T21:35:15.876035+0000","last_became_active":"2024-04-26T21:35:10.128720+0000","last_became_peered":"2024-04-26T21:35:10.128720+0000","last_unstale":"2024-04-26T21:35:15.876035+0000","last_undegraded":"2024-04-26T21:35:15.876035+0000","last_fullsized":"2024-04-26T21:35:15.876035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:33.320854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:35:33.567 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:35:33.568 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:35:33.568 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:35:33.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T21:35:33.568 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T21:35:33.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.703+0000 7f8a2cdbf640 1 Processor -- start 2024-04-26T21:35:33.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.703+0000 7f8a2cdbf640 1 -- start start 2024-04-26T21:35:33.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2cdbf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a28162810 0x7f8a28070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2cdbf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a280d1b30 0x7f8a280d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2cdbf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a280d10d0 con 0x7f8a280b2340 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2cdbf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8a280d0750 con 0x7f8a28162810 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2cdbf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8a280d16a0 con 0x7f8a280d1b30 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a28162810 0x7f8a28070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a28162810 0x7f8a28070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45452/0 (socket says 172.21.15.40:45452) 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2659c640 1 -- 172.21.15.40:0/3883656428 learned_addr learned my addr 172.21.15.40:0/3883656428 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a26d9d640 1 --2- 172.21.15.40:0/3883656428 >> v2:172.21.15.184:3300/0 conn(0x7f8a280d1b30 0x7f8a280d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2659c640 1 -- 172.21.15.40:0/3883656428 >> v2:172.21.15.184:3300/0 conn(0x7f8a280d1b30 msgr2=0x7f8a280d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2659c640 1 --2- 172.21.15.40:0/3883656428 >> v2:172.21.15.184:3300/0 conn(0x7f8a280d1b30 0x7f8a280d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2659c640 1 -- 172.21.15.40:0/3883656428 >> v1:172.21.15.184:6789/0 conn(0x7f8a280b2340 legacy=0x7f8a280cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.704+0000 7f8a2659c640 1 -- 172.21.15.40:0/3883656428 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a280d1370 con 0x7f8a28162810 2024-04-26T21:35:33.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.705+0000 7f8a2659c640 1 --2- 172.21.15.40:0/3883656428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a28070350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a28073330 tx=0x7f8a1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7a79078559ec82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:33.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.705+0000 7f8a25d9b640 1 -- 172.21.15.40:0/3883656428 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18037070 con 0x7f8a28162810 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.705+0000 7f8a25d9b640 1 -- 172.21.15.40:0/3883656428 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a18032440 con 0x7f8a28162810 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.705+0000 7f8a25d9b640 1 -- 172.21.15.40:0/3883656428 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18032810 con 0x7f8a28162810 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.706+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/3883656428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 msgr2=0x7f8a28070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.706+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/3883656428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a28070350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a28073330 tx=0x7f8a1802a270 comp rx=0 tx=0).stop 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.706+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/3883656428 shutdown_connections 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.706+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/3883656428 >> v2:172.21.15.184:3300/0 conn(0x7f8a280d1b30 0x7f8a280d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.706+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/3883656428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a28070350 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.706+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/3883656428 >> 172.21.15.40:0/3883656428 conn(0x7f8a280d2da0 msgr2=0x7f8a280d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.706+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/3883656428 shutdown_connections 2024-04-26T21:35:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.706+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/3883656428 wait complete. 2024-04-26T21:35:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.707+0000 7f8a2cdbf640 1 Processor -- start 2024-04-26T21:35:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.707+0000 7f8a2cdbf640 1 -- start start 2024-04-26T21:35:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a2cdbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a280b2340 0x7f8a2814bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a2cdbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a280d1b30 0x7f8a2806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a2cdbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a2806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a2cdbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28070890 con 0x7f8a28162810 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a2cdbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a280d0470 con 0x7f8a280b2340 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a2cdbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a280cdb00 con 0x7f8a280d1b30 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a2806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a2806d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45468/0 (socket says 172.21.15.40:45468) 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 -- 172.21.15.40:0/1039525340 learned_addr learned my addr 172.21.15.40:0/1039525340 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a2659c640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a280b2340 0x7f8a2814bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a1ffff640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a280d1b30 0x7f8a2806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 -- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a280d1b30 msgr2=0x7f8a2806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a280d1b30 0x7f8a2806d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 -- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a280b2340 msgr2=0x7f8a2814bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a280b2340 0x7f8a2814bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 -- 172.21.15.40:0/1039525340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a281f4dd0 con 0x7f8a28162810 2024-04-26T21:35:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a26d9d640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a2806d890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000efc0 tx=0x7f8a2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a1effd640 1 -- 172.21.15.40:0/1039525340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20018020 con 0x7f8a28162810 2024-04-26T21:35:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a1effd640 1 -- 172.21.15.40:0/1039525340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20014480 con 0x7f8a28162810 2024-04-26T21:35:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.708+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2814f320 con 0x7f8a28162810 2024-04-26T21:35:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.709+0000 7f8a1effd640 1 -- 172.21.15.40:0/1039525340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20014870 con 0x7f8a28162810 2024-04-26T21:35:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.709+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a281f5370 con 0x7f8a28162810 2024-04-26T21:35:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.709+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a28163a80 con 0x7f8a28162810 2024-04-26T21:35:33.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.710+0000 7f8a1effd640 1 -- 172.21.15.40:0/1039525340 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a20005d10 con 0x7f8a28162810 2024-04-26T21:35:33.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.710+0000 7f8a1effd640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0806c8f0 0x7f8a0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:33.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.710+0000 7f8a1effd640 1 -- 172.21.15.40:0/1039525340 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8a2008cb70 con 0x7f8a28162810 2024-04-26T21:35:33.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.710+0000 7f8a2659c640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0806c8f0 0x7f8a0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:33.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.711+0000 7f8a2659c640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0806c8f0 0x7f8a0806edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a18034000 tx=0x7f8a1802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:33.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.714+0000 7f8a1effd640 1 -- 172.21.15.40:0/1039525340 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a28163a80 con 0x7f8a28162810 2024-04-26T21:35:33.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.949+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8a280694f0 con 0x7f8a28162810 2024-04-26T21:35:33.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.952+0000 7f8a1effd640 1 -- 172.21.15.40:0/1039525340 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7f8a20022030 con 0x7f8a28162810 2024-04-26T21:35:33.955 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0806c8f0 msgr2=0x7f8a0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0806c8f0 0x7f8a0806edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a18034000 tx=0x7f8a1802e000 comp rx=0 tx=0).stop 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 msgr2=0x7f8a2806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a2806d890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000efc0 tx=0x7f8a2000c490 comp rx=0 tx=0).stop 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 shutdown_connections 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0806c8f0 0x7f8a0806edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a28162810 0x7f8a2806d890 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a280d1b30 0x7f8a2806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 --2- 172.21.15.40:0/1039525340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a280b2340 0x7f8a2814bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.954+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 >> 172.21.15.40:0/1039525340 conn(0x7f8a280d2da0 msgr2=0x7f8a280cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.955+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 shutdown_connections 2024-04-26T21:35:33.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:33.955+0000 7f8a2cdbf640 1 -- 172.21.15.40:0/1039525340 wait complete. 2024-04-26T21:35:33.966 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2311011869},{"type":"v1","addr":"172.21.15.184:6833","nonce":2311011869}]},"active_addr":"172.21.15.184:6833/2311011869","active_change":"2024-04-26T21:34:49.981927+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":4036537775}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":196313205}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":3568316643}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":3242939126}]}]} 2024-04-26T21:35:33.969 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T21:35:33.970 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T21:35:33.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.069+0000 7fd50d5bf640 1 Processor -- start 2024-04-26T21:35:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.069+0000 7fd50d5bf640 1 -- start start 2024-04-26T21:35:34.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.070+0000 7fd50d5bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 0x7fd508160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.070+0000 7fd50d5bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd508070930 0x7fd5080aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.070+0000 7fd50d5bf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5080b0990 con 0x7fd50806c240 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.070+0000 7fd50d5bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd5080af2d0 con 0x7fd508070930 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.070+0000 7fd50d5bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5080b0c70 con 0x7fd50815d100 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.070+0000 7fd5067fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd508070930 0x7fd5080aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.070+0000 7fd5077fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd50806c240 0x7fd508073160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40108/0 (socket says 172.21.15.40:40108) 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.070+0000 7fd5077fe640 1 -- 172.21.15.40:0/547779834 learned_addr learned my addr 172.21.15.40:0/547779834 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.071+0000 7fd506ffd640 1 --2- 172.21.15.40:0/547779834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 0x7fd508160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.071+0000 7fd5067fc640 1 -- 172.21.15.40:0/547779834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 msgr2=0x7fd508160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.071+0000 7fd5067fc640 1 --2- 172.21.15.40:0/547779834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 0x7fd508160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.071+0000 7fd5067fc640 1 -- 172.21.15.40:0/547779834 >> v1:172.21.15.40:6789/0 conn(0x7fd50806c240 legacy=0x7fd508073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.071+0000 7fd505ffb640 1 -- 172.21.15.40:0/547779834 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2568620939 0 0) 0x7fd5080b0990 con 0x7fd50806c240 2024-04-26T21:35:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.071+0000 7fd5067fc640 1 -- 172.21.15.40:0/547779834 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5080b0f50 con 0x7fd508070930 2024-04-26T21:35:34.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.071+0000 7fd5067fc640 1 --2- 172.21.15.40:0/547779834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd508070930 0x7fd5080aed90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00097a0 tx=0x7fd4f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb84a85c1683d45f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd505ffb640 1 -- 172.21.15.40:0/547779834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0037070 con 0x7fd508070930 2024-04-26T21:35:34.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd505ffb640 1 -- 172.21.15.40:0/547779834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f0032440 con 0x7fd508070930 2024-04-26T21:35:34.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd505ffb640 1 -- 172.21.15.40:0/547779834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0032810 con 0x7fd508070930 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd50d5bf640 1 -- 172.21.15.40:0/547779834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd508070930 msgr2=0x7fd5080aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/547779834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd508070930 0x7fd5080aed90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00097a0 tx=0x7fd4f002a180 comp rx=0 tx=0).stop 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd50d5bf640 1 -- 172.21.15.40:0/547779834 shutdown_connections 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/547779834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd508070930 0x7fd5080aed90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/547779834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 0x7fd508160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd50d5bf640 1 -- 172.21.15.40:0/547779834 >> 172.21.15.40:0/547779834 conn(0x7fd5080b39b0 msgr2=0x7fd5080b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd50d5bf640 1 -- 172.21.15.40:0/547779834 shutdown_connections 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.072+0000 7fd50d5bf640 1 -- 172.21.15.40:0/547779834 wait complete. 2024-04-26T21:35:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.073+0000 7fd50d5bf640 1 Processor -- start 2024-04-26T21:35:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd50d5bf640 1 -- start start 2024-04-26T21:35:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd50d5bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50806c240 0x7fd508152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd50d5bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd508070930 0x7fd508154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd50d5bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 0x7fd5080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd50d5bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd50806c6d0 con 0x7fd50806c240 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd50d5bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5080b1510 con 0x7fd508070930 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd50d5bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd50806c850 con 0x7fd50815d100 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd506ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50806c240 0x7fd508152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd506ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50806c240 0x7fd508152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45482/0 (socket says 172.21.15.40:45482) 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.074+0000 7fd506ffd640 1 -- 172.21.15.40:0/918865067 learned_addr learned my addr 172.21.15.40:0/918865067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd5077fe640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 0x7fd5080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd5067fc640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd508070930 0x7fd508154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd506ffd640 1 -- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 msgr2=0x7fd5080b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd506ffd640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 0x7fd5080b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd506ffd640 1 -- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd508070930 msgr2=0x7fd508154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd506ffd640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd508070930 0x7fd508154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd506ffd640 1 -- 172.21.15.40:0/918865067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50815a940 con 0x7fd50806c240 2024-04-26T21:35:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd5067fc640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd508070930 0x7fd508154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd506ffd640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50806c240 0x7fd508152a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd50000c7f0 tx=0x7fd50000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd4ff7fe640 1 -- 172.21.15.40:0/918865067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500015070 con 0x7fd50806c240 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd4ff7fe640 1 -- 172.21.15.40:0/918865067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd500004220 con 0x7fd50806c240 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50815ac20 con 0x7fd50806c240 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd4ff7fe640 1 -- 172.21.15.40:0/918865067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500007620 con 0x7fd50806c240 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.075+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50815b1e0 con 0x7fd50806c240 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.076+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5080694f0 con 0x7fd50806c240 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.077+0000 7fd4ff7fe640 1 -- 172.21.15.40:0/918865067 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5000079d0 con 0x7fd50806c240 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.077+0000 7fd4ff7fe640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4e806c8f0 0x7fd4e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.077+0000 7fd4ff7fe640 1 -- 172.21.15.40:0/918865067 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd50008c250 con 0x7fd50806c240 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.077+0000 7fd5067fc640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4e806c8f0 0x7fd4e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.078+0000 7fd5067fc640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4e806c8f0 0x7fd4e806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0034000 tx=0x7fd4f002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.081+0000 7fd4ff7fe640 1 -- 172.21.15.40:0/918865067 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd50005a340 con 0x7fd50806c240 2024-04-26T21:35:34.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.271+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd50806cd70 con 0x7fd50806c240 2024-04-26T21:35:34.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.272+0000 7fd4ff7fe640 1 -- 172.21.15.40:0/918865067 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7fd500059c40 con 0x7fd50806c240 2024-04-26T21:35:34.273 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:34.273 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4e806c8f0 msgr2=0x7fd4e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4e806c8f0 0x7fd4e806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0034000 tx=0x7fd4f002e000 comp rx=0 tx=0).stop 2024-04-26T21:35:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50806c240 msgr2=0x7fd508152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50806c240 0x7fd508152a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd50000c7f0 tx=0x7fd50000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 shutdown_connections 2024-04-26T21:35:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4e806c8f0 0x7fd4e806edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd50815d100 0x7fd5080b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd508070930 0x7fd508154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 --2- 172.21.15.40:0/918865067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50806c240 0x7fd508152a10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 >> 172.21.15.40:0/918865067 conn(0x7fd5080b39b0 msgr2=0x7fd508161310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 shutdown_connections 2024-04-26T21:35:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.274+0000 7fd50d5bf640 1 -- 172.21.15.40:0/918865067 wait complete. 2024-04-26T21:35:34.286 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T21:35:34.286 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.386+0000 7f91d01e4640 1 Processor -- start 2024-04-26T21:35:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.386+0000 7f91d01e4640 1 -- start start 2024-04-26T21:35:34.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91d01e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91c815f3a0 0x7f91c80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91d01e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d1770 0x7f91c80d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91d01e4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c80d25a0 con 0x7f91c80d2130 2024-04-26T21:35:34.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91d01e4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f91c80d0e20 con 0x7f91c815f3a0 2024-04-26T21:35:34.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91d01e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91c80b1d70 con 0x7f91c80d1770 2024-04-26T21:35:34.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91cdf80640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91c815f3a0 0x7f91c80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91cdf80640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91c815f3a0 0x7f91c80d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45488/0 (socket says 172.21.15.40:45488) 2024-04-26T21:35:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91cdf80640 1 -- 172.21.15.40:0/843162373 learned_addr learned my addr 172.21.15.40:0/843162373 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91cd77f640 1 --2- 172.21.15.40:0/843162373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d1770 0x7f91c80d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91cdf80640 1 -- 172.21.15.40:0/843162373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d1770 msgr2=0x7f91c80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91cdf80640 1 --2- 172.21.15.40:0/843162373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d1770 0x7f91c80d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91cdf80640 1 -- 172.21.15.40:0/843162373 >> v1:172.21.15.184:6789/0 conn(0x7f91c80d2130 legacy=0x7f91c80d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.387+0000 7f91cdf80640 1 -- 172.21.15.40:0/843162373 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c80b2050 con 0x7f91c815f3a0 2024-04-26T21:35:34.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.388+0000 7f91cdf80640 1 --2- 172.21.15.40:0/843162373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80d06f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009870 tx=0x7f91c402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd840bde5a421fff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.388+0000 7f91ccf7e640 1 -- 172.21.15.40:0/843162373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4037070 con 0x7f91c815f3a0 2024-04-26T21:35:34.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.388+0000 7f91ccf7e640 1 -- 172.21.15.40:0/843162373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c4032440 con 0x7f91c815f3a0 2024-04-26T21:35:34.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.388+0000 7f91ccf7e640 1 -- 172.21.15.40:0/843162373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4032810 con 0x7f91c815f3a0 2024-04-26T21:35:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.389+0000 7f91d01e4640 1 -- 172.21.15.40:0/843162373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 msgr2=0x7f91c80d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.389+0000 7f91d01e4640 1 --2- 172.21.15.40:0/843162373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80d06f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009870 tx=0x7f91c402a210 comp rx=0 tx=0).stop 2024-04-26T21:35:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.389+0000 7f91d01e4640 1 -- 172.21.15.40:0/843162373 shutdown_connections 2024-04-26T21:35:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.389+0000 7f91d01e4640 1 --2- 172.21.15.40:0/843162373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d1770 0x7f91c80d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.389+0000 7f91d01e4640 1 --2- 172.21.15.40:0/843162373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80d06f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.389+0000 7f91d01e4640 1 -- 172.21.15.40:0/843162373 >> 172.21.15.40:0/843162373 conn(0x7f91c80d4f90 msgr2=0x7f91c80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.389+0000 7f91d01e4640 1 -- 172.21.15.40:0/843162373 shutdown_connections 2024-04-26T21:35:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.389+0000 7f91d01e4640 1 -- 172.21.15.40:0/843162373 wait complete. 2024-04-26T21:35:34.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.390+0000 7f91d01e4640 1 Processor -- start 2024-04-26T21:35:34.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.390+0000 7f91d01e4640 1 -- start start 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91d01e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91c80d1770 0x7f91c8161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91d01e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d2130 0x7f91c80b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91d01e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91d01e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91c815f900 con 0x7f91c815f3a0 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91d01e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91c815f780 con 0x7f91c80d1770 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91d01e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91c80b3860 con 0x7f91c80d2130 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45498/0 (socket says 172.21.15.40:45498) 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 -- 172.21.15.40:0/1661796383 learned_addr learned my addr 172.21.15.40:0/1661796383 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91cd77f640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d2130 0x7f91c80b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91cdf80640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91c80d1770 0x7f91c8161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 -- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d2130 msgr2=0x7f91c80b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d2130 0x7f91c80b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 -- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91c80d1770 msgr2=0x7f91c8161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91c80d1770 0x7f91c8161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 -- 172.21.15.40:0/1661796383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c80b99d0 con 0x7f91c815f3a0 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91cdf80640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91c80d1770 0x7f91c8161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91cd77f640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d2130 0x7f91c80b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91ce781640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80b50a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91c000c7b0 tx=0x7f91c000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91be7fc640 1 -- 172.21.15.40:0/1661796383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0004b60 con 0x7f91c815f3a0 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91be7fc640 1 -- 172.21.15.40:0/1661796383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c0004d00 con 0x7f91c815f3a0 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c80b9cb0 con 0x7f91c815f3a0 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.391+0000 7f91be7fc640 1 -- 172.21.15.40:0/1661796383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c000aa90 con 0x7f91c815f3a0 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.392+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c80ba270 con 0x7f91c815f3a0 2024-04-26T21:35:34.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.392+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c80ba470 con 0x7f91c815f3a0 2024-04-26T21:35:34.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.393+0000 7f91be7fc640 1 -- 172.21.15.40:0/1661796383 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91c000ac30 con 0x7f91c815f3a0 2024-04-26T21:35:34.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.393+0000 7f91be7fc640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f91ac06c8f0 0x7f91ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.393+0000 7f91be7fc640 1 -- 172.21.15.40:0/1661796383 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f91c008c0f0 con 0x7f91c815f3a0 2024-04-26T21:35:34.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.394+0000 7f91cdf80640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f91ac06c8f0 0x7f91ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.394+0000 7f91cdf80640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f91ac06c8f0 0x7f91ac06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91c4034000 tx=0x7f91c402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.398+0000 7f91be7fc640 1 -- 172.21.15.40:0/1661796383 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91c80ba470 con 0x7f91c815f3a0 2024-04-26T21:35:34.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.584+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91c80694f0 con 0x7f91c815f3a0 2024-04-26T21:35:34.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.585+0000 7f91be7fc640 1 -- 172.21.15.40:0/1661796383 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f91c0020070 con 0x7f91c815f3a0 2024-04-26T21:35:34.587 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:34.587 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:34.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f91ac06c8f0 msgr2=0x7f91ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f91ac06c8f0 0x7f91ac06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91c4034000 tx=0x7f91c402e000 comp rx=0 tx=0).stop 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 msgr2=0x7f91c80b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80b50a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91c000c7b0 tx=0x7f91c000cc70 comp rx=0 tx=0).stop 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 shutdown_connections 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f91ac06c8f0 0x7f91ac06edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c815f3a0 0x7f91c80b50a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91c80d2130 0x7f91c80b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 --2- 172.21.15.40:0/1661796383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91c80d1770 0x7f91c8161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 >> 172.21.15.40:0/1661796383 conn(0x7f91c80d4f90 msgr2=0x7f91c80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 shutdown_connections 2024-04-26T21:35:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.587+0000 7f91d01e4640 1 -- 172.21.15.40:0/1661796383 wait complete. 2024-04-26T21:35:34.600 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T21:35:34.600 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T21:35:34.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T21:35:34.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T21:35:34.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T21:35:34.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T21:35:34.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T21:35:34.603 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T21:35:34.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.777+0000 7f2872459640 1 Processor -- start 2024-04-26T21:35:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.777+0000 7f2872459640 1 -- start start 2024-04-26T21:35:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.777+0000 7fa4f6d3b640 1 Processor -- start 2024-04-26T21:35:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7f5435f14640 1 Processor -- start 2024-04-26T21:35:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7f90f3e83640 1 Processor -- start 2024-04-26T21:35:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7fa4f6d3b640 1 -- start start 2024-04-26T21:35:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7f5435f14640 1 -- start start 2024-04-26T21:35:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7fb86bea9640 1 Processor -- start 2024-04-26T21:35:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7ff5cf59e640 1 Processor -- start 2024-04-26T21:35:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7fb86bea9640 1 -- start start 2024-04-26T21:35:34.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7fdf4d2da640 1 Processor -- start 2024-04-26T21:35:34.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7fdf4d2da640 1 -- start start 2024-04-26T21:35:34.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7fa4f6d3b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4f006f420 0x7fa4f01644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7fa4f6d3b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4f00b4e60 0x7fa4f0071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7fa4f6d3b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f006f800 con 0x7fa4f0072220 2024-04-26T21:35:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7fa4f6d3b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f0164a00 con 0x7fa4f006f420 2024-04-26T21:35:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f2872459640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f286c0d2830 0x7f286c0d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7fa4f6d3b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f0072600 con 0x7fa4f00b4e60 2024-04-26T21:35:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f286b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f286c163610 0x7f286c0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40116/0 (socket says 172.21.15.40:40116) 2024-04-26T21:35:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f2872459640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286c0d1a70 con 0x7f286c163610 2024-04-26T21:35:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f2872459640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286c0d1d50 con 0x7f286c15f5a0 2024-04-26T21:35:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f2872459640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f286c163d30 con 0x7f286c0d2830 2024-04-26T21:35:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f286b7fe640 1 -- 172.21.15.40:0/103579977 learned_addr learned my addr 172.21.15.40:0/103579977 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:34.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f5435f14640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f54300704b0 0x7f5430146460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f5435f14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5430072510 0x7f54300cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f5435f14640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54300c72d0 con 0x7f54300c62c0 2024-04-26T21:35:34.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f5435f14640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f54300c6a30 con 0x7f5430072510 2024-04-26T21:35:34.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f5435f14640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f54300c6d10 con 0x7f54300704b0 2024-04-26T21:35:34.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7f90f3e83640 1 -- start start 2024-04-26T21:35:34.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7fa4f5d39640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4f006f420 0x7fa4f01644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7fa4f5d39640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4f006f420 0x7fa4f01644c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45502/0 (socket says 172.21.15.40:45502) 2024-04-26T21:35:34.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7fa4f5d39640 1 -- 172.21.15.40:0/2960412192 learned_addr learned my addr 172.21.15.40:0/2960412192 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f90f3e83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90ec15d1c0 0x7f90ec0d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f90f3e83640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90ec0ce2f0 0x7f90ec0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f90f3e83640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90ec0d0a30 con 0x7f90ec160de0 2024-04-26T21:35:34.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f90f3e83640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f90ec0d1d30 con 0x7f90ec0ce2f0 2024-04-26T21:35:34.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.779+0000 7f90f3e83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90ec0d07e0 con 0x7f90ec15d1c0 2024-04-26T21:35:34.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.778+0000 7ff5cf59e640 1 -- start start 2024-04-26T21:35:34.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf4d2da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdf480d32a0 0x7fdf480d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fa4f5538640 1 --2- 172.21.15.40:0/2960412192 >> v2:172.21.15.184:3300/0 conn(0x7fa4f00b4e60 0x7fa4f0071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f542effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f54300c62c0 0x7f5430071e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40124/0 (socket says 172.21.15.40:40124) 2024-04-26T21:35:34.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f542effd640 1 -- 172.21.15.40:0/3840752912 learned_addr learned my addr 172.21.15.40:0/3840752912 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:34.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb86bea9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8641678f0 0x7fb8640cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb86bea9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb864163a50 con 0x7fb8640c41d0 2024-04-26T21:35:34.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb86bea9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8640ceb90 con 0x7fb864073f30 2024-04-26T21:35:34.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7ff5cf59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c8055720 0x7ff5c80509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7ff5ce59c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff5c80dfee0 0x7ff5c8053ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:57950/0 (socket says 172.21.15.40:57950) 2024-04-26T21:35:34.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7ff5ce59c640 1 -- 172.21.15.40:0/4149685684 learned_addr learned my addr 172.21.15.40:0/4149685684 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:34.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb21bfff640 1 Processor -- start 2024-04-26T21:35:34.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb86bea9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8640cf7a0 con 0x7fb8641678f0 2024-04-26T21:35:34.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f542f7fe640 1 --2- 172.21.15.40:0/3840752912 >> v2:172.21.15.184:3300/0 conn(0x7f54300704b0 0x7f5430146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7ff5cf59e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c80543b0 con 0x7ff5c803b920 2024-04-26T21:35:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7ff5cf59e640 1 -- 172.21.15.40:0/4149685684 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c8054680 con 0x7ff5c80dfee0 2024-04-26T21:35:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7ff5cf59e640 1 -- 172.21.15.40:0/4149685684 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c8054960 con 0x7ff5c8055720 2024-04-26T21:35:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf46ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdf480d32a0 0x7fdf480d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb86a446640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8641678f0 0x7fb8640cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb86a446640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8641678f0 0x7fb8640cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49588/0 (socket says 172.21.15.40:49588) 2024-04-26T21:35:34.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb86a446640 1 -- 172.21.15.40:0/725227924 learned_addr learned my addr 172.21.15.40:0/725227924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf4d2da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf480b62d0 0x7fdf48167b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf4d2da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf480b6780 con 0x7fdf480d5b20 2024-04-26T21:35:34.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf4d2da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdf480d1eb0 con 0x7fdf480d32a0 2024-04-26T21:35:34.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf467fc640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fdf480d5b20 0x7fdf48167470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:57974/0 (socket says 172.21.15.40:57974) 2024-04-26T21:35:34.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf467fc640 1 -- 172.21.15.40:0/309217124 learned_addr learned my addr 172.21.15.40:0/309217124 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:34.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7ff5cdd9b640 1 --2- 172.21.15.40:0/4149685684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c8055720 0x7ff5c80509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf477fe640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf480b62d0 0x7fdf48167b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184090129 0 0) 0x7fdf38001490 con 0x7fdf480d5b20 2024-04-26T21:35:34.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf4d2da640 1 -- 172.21.15.40:0/309217124 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf480d3db0 con 0x7fdf480b62d0 2024-04-26T21:35:34.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf34003520 con 0x7fdf480d5b20 2024-04-26T21:35:34.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7ff5cd59a640 1 -- 172.21.15.40:0/4149685684 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358383541 0 0) 0x7ff5c8054680 con 0x7ff5c80dfee0 2024-04-26T21:35:34.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fb21bfff640 1 -- start start 2024-04-26T21:35:34.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f28709f6640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f286c0d2830 0x7f286c0d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460769630 0 0) 0x7f286c0d1a70 con 0x7f286c163610 2024-04-26T21:35:34.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2854003520 con 0x7f286c163610 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1151912386 0 0) 0x7f2854003520 con 0x7f286c163610 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f286c0d2830 msgr2=0x7f286c0d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f286c0d2830 0x7f286c0d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 >> v1:172.21.15.184:6789/0 conn(0x7f286c15f5a0 legacy=0x7f286c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c0d2030 con 0x7f286c163610 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f2860001650 con 0x7f286c163610 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 >> v1:172.21.15.40:6789/0 conn(0x7f286c163610 legacy=0x7f286c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854003a00 0x7f2854003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2854004800 0x7f2854004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2854005210 0x7f285400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f285400c0c0 con 0x7f2854005210 2024-04-26T21:35:34.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f285400c270 con 0x7f2854003a00 2024-04-26T21:35:34.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f285400c470 con 0x7f2854004800 2024-04-26T21:35:34.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fa4f5538640 1 -- 172.21.15.40:0/2960412192 >> v2:172.21.15.40:3300/0 conn(0x7fa4f006f420 msgr2=0x7fa4f01644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f28709f6640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f286c0d2830 0x7f286c0d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:34.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7ff5cd59a640 1 -- 172.21.15.40:0/4149685684 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5b8003520 con 0x7ff5c80dfee0 2024-04-26T21:35:34.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f542ffff640 1 --2- 172.21.15.40:0/3840752912 >> v2:172.21.15.40:3300/0 conn(0x7f5430072510 0x7f54300cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f90f2420640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f90ec160de0 0x7f90ec0b5a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40148/0 (socket says 172.21.15.40:40148) 2024-04-26T21:35:34.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f90f2420640 1 -- 172.21.15.40:0/773140350 learned_addr learned my addr 172.21.15.40:0/773140350 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:34.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fa4f5538640 1 --2- 172.21.15.40:0/2960412192 >> v2:172.21.15.40:3300/0 conn(0x7fa4f006f420 0x7fa4f01644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fa4f5538640 1 -- 172.21.15.40:0/2960412192 >> v1:172.21.15.40:6789/0 conn(0x7fa4f0072220 legacy=0x7fa4f006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7fa4f5538640 1 -- 172.21.15.40:0/2960412192 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f00b3dc0 con 0x7fa4f00b4e60 2024-04-26T21:35:34.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fa4f5538640 1 --2- 172.21.15.40:0/2960412192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f00b4e60 0x7fa4f0071ce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00b0640 tx=0x7fa4e802a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a951014165190c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f28709f6640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2854005210 0x7f285400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.780+0000 7f90f141e640 1 --2- 172.21.15.40:0/773140350 >> v2:172.21.15.40:3300/0 conn(0x7f90ec0ce2f0 0x7f90ec0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f90f141e640 1 -- 172.21.15.40:0/773140350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90ec15d1c0 msgr2=0x7f90ec0d17f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:35:34.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f90f141e640 1 --2- 172.21.15.40:0/773140350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90ec15d1c0 0x7f90ec0d17f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f90f141e640 1 -- 172.21.15.40:0/773140350 >> v1:172.21.15.40:6789/0 conn(0x7f90ec160de0 legacy=0x7f90ec0b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f90f141e640 1 -- 172.21.15.40:0/773140350 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec0d0d10 con 0x7f90ec0ce2f0 2024-04-26T21:35:34.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2798012917 0 0) 0x7fdf34003520 con 0x7fdf480d5b20 2024-04-26T21:35:34.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf480b62d0 msgr2=0x7fdf48167b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf480b62d0 0x7fdf48167b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 >> v2:172.21.15.184:3300/0 conn(0x7fdf480d32a0 msgr2=0x7fdf480d3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 --2- 172.21.15.40:0/309217124 >> v2:172.21.15.184:3300/0 conn(0x7fdf480d32a0 0x7fdf480d3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f542f7fe640 1 -- 172.21.15.40:0/3840752912 >> v2:172.21.15.40:3300/0 conn(0x7f5430072510 msgr2=0x7f54300cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f542f7fe640 1 --2- 172.21.15.40:0/3840752912 >> v2:172.21.15.40:3300/0 conn(0x7f5430072510 0x7f54300cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f542f7fe640 1 -- 172.21.15.40:0/3840752912 >> v1:172.21.15.40:6789/0 conn(0x7f54300c62c0 legacy=0x7f5430071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f542f7fe640 1 -- 172.21.15.40:0/3840752912 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54300c6ff0 con 0x7f54300704b0 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf480d4090 con 0x7fdf480d5b20 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fdf38001bc0 con 0x7fdf480d5b20 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 >> v1:172.21.15.184:6789/0 conn(0x7fdf480d5b20 legacy=0x7fdf48167470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf340039b0 0x7fdf34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf34004800 0x7fdf34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf340051c0 0x7fdf3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf480d3db0 con 0x7fdf34004800 2024-04-26T21:35:34.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3400bf00 con 0x7fdf340039b0 2024-04-26T21:35:34.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3400c080 con 0x7fdf340051c0 2024-04-26T21:35:34.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7f542e7fc640 1 -- 172.21.15.40:0/3840752912 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1071409356 0 0) 0x7f54300c72d0 con 0x7f54300c62c0 2024-04-26T21:35:34.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fdf477fe640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf340051c0 0x7fdf3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fa4f4d37640 1 -- 172.21.15.40:0/2960412192 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8007740 con 0x7fa4f00b4e60 2024-04-26T21:35:34.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fa4f4d37640 1 -- 172.21.15.40:0/2960412192 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e80078e0 con 0x7fa4f00b4e60 2024-04-26T21:35:34.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fa4f4d37640 1 -- 172.21.15.40:0/2960412192 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8007740 con 0x7fa4f00b4e60 2024-04-26T21:35:34.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fdf467fc640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf34004800 0x7fdf34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fdf46ffd640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf340039b0 0x7fdf34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7ff5cd59a640 1 -- 172.21.15.40:0/4149685684 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023069060 0 0) 0x7ff5c80543b0 con 0x7ff5c803b920 2024-04-26T21:35:34.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7ff5cd59a640 1 -- 172.21.15.40:0/4149685684 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5c8054680 con 0x7ff5c803b920 2024-04-26T21:35:34.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.781+0000 7fb21bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb21c06f420 0x7fb21c1644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 0x7fb21c071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb21c072220 0x7fb21c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21bfff640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb21c072670 con 0x7fb21c072220 2024-04-26T21:35:34.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21bfff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb21c06f800 con 0x7fb21c06f420 2024-04-26T21:35:34.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb21c164a00 con 0x7fb21c0b4e60 2024-04-26T21:35:34.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21affd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb21c06f420 0x7fb21c1644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21affd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb21c06f420 0x7fb21c1644c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49592/0 (socket says 172.21.15.40:49592) 2024-04-26T21:35:34.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21affd640 1 -- 172.21.15.40:0/2762225553 learned_addr learned my addr 172.21.15.40:0/2762225553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21b7fe640 1 --2- 172.21.15.40:0/2762225553 >> v2:172.21.15.40:3300/0 conn(0x7fb21c072220 0x7fb21c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21affd640 1 -- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 msgr2=0x7fb21c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7f90f141e640 1 --2- 172.21.15.40:0/773140350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec0ce2f0 0x7f90ec0ce750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009e10 tx=0x7f90e4032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d0ad61cf0c4cca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7ff5cdd9b640 1 -- 172.21.15.40:0/4149685684 >> v1:172.21.15.184:6789/0 conn(0x7ff5c80dfee0 legacy=0x7ff5c8053ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7ff5cdd9b640 1 -- 172.21.15.40:0/4149685684 >> v1:172.21.15.40:6789/0 conn(0x7ff5c803b920 legacy=0x7ff5c8038d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7ff5cdd9b640 1 -- 172.21.15.40:0/4149685684 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c8054c40 con 0x7ff5c8055720 2024-04-26T21:35:34.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb86a446640 1 -- 172.21.15.40:0/725227924 >> v1:172.21.15.184:6789/0 conn(0x7fb864073f30 legacy=0x7fb8641671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb86a446640 1 -- 172.21.15.40:0/725227924 >> v1:172.21.15.40:6789/0 conn(0x7fb8640c41d0 legacy=0x7fb8640cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fb86a446640 1 -- 172.21.15.40:0/725227924 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8640cee70 con 0x7fb8641678f0 2024-04-26T21:35:34.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21a7fc640 1 --2- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 0x7fb21c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21affd640 1 --2- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 0x7fb21c071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21affd640 1 -- 172.21.15.40:0/2762225553 >> v2:172.21.15.40:3300/0 conn(0x7fb21c072220 msgr2=0x7fb21c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21affd640 1 --2- 172.21.15.40:0/2762225553 >> v2:172.21.15.40:3300/0 conn(0x7fb21c072220 0x7fb21c06dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fb21affd640 1 -- 172.21.15.40:0/2762225553 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c0b3dc0 con 0x7fb21c06f420 2024-04-26T21:35:34.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fb21a7fc640 1 --2- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 0x7fb21c071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:34.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f542f7fe640 1 --2- 172.21.15.40:0/3840752912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300704b0 0x7f5430146460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54300d0820 tx=0x7f5424032840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b30f6eb2ab5aab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f542e7fc640 1 -- 172.21.15.40:0/3840752912 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542403f070 con 0x7f54300704b0 2024-04-26T21:35:34.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f542e7fc640 1 -- 172.21.15.40:0/3840752912 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542400b650 con 0x7f54300704b0 2024-04-26T21:35:34.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f542e7fc640 1 -- 172.21.15.40:0/3840752912 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5424036050 con 0x7f54300704b0 2024-04-26T21:35:34.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f90f0c1d640 1 -- 172.21.15.40:0/773140350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4040070 con 0x7f90ec0ce2f0 2024-04-26T21:35:34.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f90f0c1d640 1 -- 172.21.15.40:0/773140350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e4033b40 con 0x7f90ec0ce2f0 2024-04-26T21:35:34.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f90f0c1d640 1 -- 172.21.15.40:0/773140350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e403b440 con 0x7f90ec0ce2f0 2024-04-26T21:35:34.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7fdf46ffd640 1 -- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf340051c0 msgr2=0x7fdf3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fdf46ffd640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf340051c0 0x7fdf3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fdf46ffd640 1 -- 172.21.15.40:0/309217124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf34004800 msgr2=0x7fdf34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fdf46ffd640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf34004800 0x7fdf34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fdf46ffd640 1 -- 172.21.15.40:0/309217124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf480d4090 con 0x7fdf340039b0 2024-04-26T21:35:34.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fdf467fc640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf34004800 0x7fdf34004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:34.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2960412192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f00b4e60 msgr2=0x7fa4f0071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fa4f6d3b640 1 --2- 172.21.15.40:0/2960412192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f00b4e60 0x7fa4f0071ce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00b0640 tx=0x7fa4e802a640 comp rx=0 tx=0).stop 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2960412192 shutdown_connections 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fa4f6d3b640 1 --2- 172.21.15.40:0/2960412192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f00b4e60 0x7fa4f0071ce0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fa4f6d3b640 1 --2- 172.21.15.40:0/2960412192 >> v2:172.21.15.40:3300/0 conn(0x7fa4f006f420 0x7fa4f01644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2960412192 >> 172.21.15.40:0/2960412192 conn(0x7fa4f00b2720 msgr2=0x7fa4f00b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.782+0000 7f286bfff640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854003a00 0x7f2854003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f286b7fe640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2854004800 0x7f2854004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f28709f6640 1 -- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2854004800 msgr2=0x7f2854004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f28709f6640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2854004800 0x7f2854004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f28709f6640 1 -- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854003a00 msgr2=0x7f2854003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:34.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f28709f6640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854003a00 0x7f2854003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7f28709f6640 1 -- 172.21.15.40:0/103579977 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c0d2030 con 0x7f2854005210 2024-04-26T21:35:34.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fb21affd640 1 --2- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 0x7fb21c1644c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb21c06daf0 tx=0x7fb21402a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f752dab4e40861 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7fb219ffb640 1 -- 172.21.15.40:0/2762225553 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214036070 con 0x7fb21c06f420 2024-04-26T21:35:34.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7fb219ffb640 1 -- 172.21.15.40:0/2762225553 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb214031600 con 0x7fb21c06f420 2024-04-26T21:35:34.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7fb219ffb640 1 -- 172.21.15.40:0/2762225553 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2140319d0 con 0x7fb21c06f420 2024-04-26T21:35:34.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7fb86a446640 1 --2- 172.21.15.40:0/725227924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8641678f0 0x7fb8640cc240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009e50 tx=0x7fb85c02a7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b705d984e8510c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f90f3e83640 1 -- 172.21.15.40:0/773140350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec0ce2f0 msgr2=0x7f90ec0ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f90f3e83640 1 --2- 172.21.15.40:0/773140350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec0ce2f0 0x7f90ec0ce750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009e10 tx=0x7f90e4032df0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7ff5cdd9b640 1 --2- 172.21.15.40:0/4149685684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c8055720 0x7ff5c80509a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097d0 tx=0x7ff5c402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d04d2476aff893e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7fb868c43640 1 -- 172.21.15.40:0/725227924 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c036070 con 0x7fb8641678f0 2024-04-26T21:35:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2960412192 shutdown_connections 2024-04-26T21:35:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.783+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2960412192 wait complete. 2024-04-26T21:35:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f286bfff640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854003a00 0x7f2854003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f90f3e83640 1 -- 172.21.15.40:0/773140350 shutdown_connections 2024-04-26T21:35:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f90f3e83640 1 --2- 172.21.15.40:0/773140350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec0ce2f0 0x7f90ec0ce750 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f90f3e83640 1 --2- 172.21.15.40:0/773140350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90ec15d1c0 0x7f90ec0d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f90f3e83640 1 -- 172.21.15.40:0/773140350 >> 172.21.15.40:0/773140350 conn(0x7f90ec0b4760 msgr2=0x7f90ec0b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7fb868c43640 1 -- 172.21.15.40:0/725227924 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c031600 con 0x7fb8641678f0 2024-04-26T21:35:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7fdf46ffd640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf340039b0 0x7fdf34003dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009af0 tx=0x7fdf34005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb9b8e26d4eba6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f28709f6640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2854005210 0x7f285400b9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f286c0d2c10 tx=0x7f2854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912def4bf8bd706e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7ff5cd59a640 1 -- 172.21.15.40:0/4149685684 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4037070 con 0x7ff5c8055720 2024-04-26T21:35:34.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7ff5cd59a640 1 -- 172.21.15.40:0/4149685684 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c4032440 con 0x7ff5c8055720 2024-04-26T21:35:34.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7ff5cd59a640 1 -- 172.21.15.40:0/4149685684 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4032810 con 0x7ff5c8055720 2024-04-26T21:35:34.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f5435f14640 1 -- 172.21.15.40:0/3840752912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300704b0 msgr2=0x7f5430146460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7f5435f14640 1 --2- 172.21.15.40:0/3840752912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300704b0 0x7f5430146460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54300d0820 tx=0x7f5424032840 comp rx=0 tx=0).stop 2024-04-26T21:35:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.784+0000 7fb868c43640 1 -- 172.21.15.40:0/725227924 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c036070 con 0x7fb8641678f0 2024-04-26T21:35:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f5435f14640 1 -- 172.21.15.40:0/3840752912 shutdown_connections 2024-04-26T21:35:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f5435f14640 1 --2- 172.21.15.40:0/3840752912 >> v2:172.21.15.40:3300/0 conn(0x7f5430072510 0x7f54300cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f5435f14640 1 --2- 172.21.15.40:0/3840752912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300704b0 0x7f5430146460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb21bfff640 1 -- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 msgr2=0x7fb21c1644c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb21bfff640 1 --2- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 0x7fb21c1644c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb21c06daf0 tx=0x7fb21402a690 comp rx=0 tx=0).stop 2024-04-26T21:35:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f5435f14640 1 -- 172.21.15.40:0/3840752912 >> 172.21.15.40:0/3840752912 conn(0x7f54300d09f0 msgr2=0x7f54300d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f5435f14640 1 -- 172.21.15.40:0/3840752912 shutdown_connections 2024-04-26T21:35:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864009b50 con 0x7f2854005210 2024-04-26T21:35:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2864003990 con 0x7f2854005210 2024-04-26T21:35:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f286affd640 1 -- 172.21.15.40:0/103579977 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864003d60 con 0x7f2854005210 2024-04-26T21:35:34.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb21bfff640 1 -- 172.21.15.40:0/2762225553 shutdown_connections 2024-04-26T21:35:34.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb21bfff640 1 --2- 172.21.15.40:0/2762225553 >> v2:172.21.15.40:3300/0 conn(0x7fb21c072220 0x7fb21c06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb21bfff640 1 --2- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 0x7fb21c071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb21bfff640 1 --2- 172.21.15.40:0/2762225553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 0x7fb21c1644c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb21bfff640 1 -- 172.21.15.40:0/2762225553 >> 172.21.15.40:0/2762225553 conn(0x7fb21c0b2720 msgr2=0x7fb21c0b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb86bea9640 1 -- 172.21.15.40:0/725227924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8641678f0 msgr2=0x7fb8640cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb86bea9640 1 --2- 172.21.15.40:0/725227924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8641678f0 0x7fb8640cc240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009e50 tx=0x7fb85c02a7b0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb86bea9640 1 -- 172.21.15.40:0/725227924 shutdown_connections 2024-04-26T21:35:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb86bea9640 1 --2- 172.21.15.40:0/725227924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8641678f0 0x7fb8640cc240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb86bea9640 1 -- 172.21.15.40:0/725227924 >> 172.21.15.40:0/725227924 conn(0x7fb8640d0be0 msgr2=0x7fb8640d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7f5435f14640 1 -- 172.21.15.40:0/3840752912 wait complete. 2024-04-26T21:35:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7ff5cf59e640 1 -- 172.21.15.40:0/4149685684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c8055720 msgr2=0x7ff5c80509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7ff5cf59e640 1 --2- 172.21.15.40:0/4149685684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c8055720 0x7ff5c80509a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097d0 tx=0x7ff5c402a250 comp rx=0 tx=0).stop 2024-04-26T21:35:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb86bea9640 1 -- 172.21.15.40:0/725227924 shutdown_connections 2024-04-26T21:35:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fb86bea9640 1 -- 172.21.15.40:0/725227924 wait complete. 2024-04-26T21:35:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40036070 con 0x7fdf340039b0 2024-04-26T21:35:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7ff5cf59e640 1 -- 172.21.15.40:0/4149685684 shutdown_connections 2024-04-26T21:35:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7ff5cf59e640 1 --2- 172.21.15.40:0/4149685684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c8055720 0x7ff5c80509a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7ff5cf59e640 1 -- 172.21.15.40:0/4149685684 >> 172.21.15.40:0/4149685684 conn(0x7ff5c8052650 msgr2=0x7ff5c8052a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf40031400 con 0x7fdf340039b0 2024-04-26T21:35:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f90f3e83640 1 -- 172.21.15.40:0/773140350 shutdown_connections 2024-04-26T21:35:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f2872459640 1 -- 172.21.15.40:0/103579977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2854005210 msgr2=0x7f285400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f2872459640 1 --2- 172.21.15.40:0/103579977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2854005210 0x7f285400b9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f286c0d2c10 tx=0x7f2854005830 comp rx=0 tx=0).stop 2024-04-26T21:35:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f28709f6640 1 -- 172.21.15.40:0/103579977 reap_dead start 2024-04-26T21:35:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f90f3e83640 1 -- 172.21.15.40:0/773140350 wait complete. 2024-04-26T21:35:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f2872459640 1 -- 172.21.15.40:0/103579977 shutdown_connections 2024-04-26T21:35:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f2872459640 1 -- 172.21.15.40:0/103579977 >> 172.21.15.40:0/103579977 conn(0x7f286c0d5190 msgr2=0x7f286c0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fdf45ffb640 1 -- 172.21.15.40:0/309217124 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40009690 con 0x7fdf340039b0 2024-04-26T21:35:34.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7fdf4d2da640 1 -- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf340039b0 msgr2=0x7fdf34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7fdf4d2da640 1 --2- 172.21.15.40:0/309217124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf340039b0 0x7fdf34003dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009af0 tx=0x7fdf34005830 comp rx=0 tx=0).stop 2024-04-26T21:35:34.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7fdf477fe640 1 -- 172.21.15.40:0/309217124 reap_dead start 2024-04-26T21:35:34.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f2872459640 1 -- 172.21.15.40:0/103579977 shutdown_connections 2024-04-26T21:35:34.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7f2872459640 1 -- 172.21.15.40:0/103579977 wait complete. 2024-04-26T21:35:34.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7ff5cf59e640 1 -- 172.21.15.40:0/4149685684 shutdown_connections 2024-04-26T21:35:34.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7fdf4d2da640 1 -- 172.21.15.40:0/309217124 shutdown_connections 2024-04-26T21:35:34.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7fdf4d2da640 1 -- 172.21.15.40:0/309217124 >> 172.21.15.40:0/309217124 conn(0x7fdf480d4e70 msgr2=0x7fdf480d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fa4f6d3b640 1 Processor -- start 2024-04-26T21:35:34.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.785+0000 7fa4f6d3b640 1 -- start start 2024-04-26T21:35:34.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7fa4f6d3b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f006f420 0x7fa4f0202070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7fa4f6d3b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f0072220 0x7fa4f015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f6d3b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f00b4e60 0x7fa4f01651c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fb21bfff640 1 -- 172.21.15.40:0/2762225553 shutdown_connections 2024-04-26T21:35:34.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fb21bfff640 1 -- 172.21.15.40:0/2762225553 wait complete. 2024-04-26T21:35:34.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7f5435f14640 1 Processor -- start 2024-04-26T21:35:34.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fb86bea9640 1 Processor -- start 2024-04-26T21:35:34.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7f5435f14640 1 -- start start 2024-04-26T21:35:34.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f653a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f00b4e60 0x7fa4f01651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f653a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f00b4e60 0x7fa4f01651c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45560/0 (socket says 172.21.15.40:45560) 2024-04-26T21:35:34.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f653a640 1 -- 172.21.15.40:0/2292674625 learned_addr learned my addr 172.21.15.40:0/2292674625 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00b4660 con 0x7fa4f00b4e60 2024-04-26T21:35:34.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00b4380 con 0x7fa4f0072220 2024-04-26T21:35:34.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f00b4940 con 0x7fa4f006f420 2024-04-26T21:35:34.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f5d39640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f006f420 0x7fa4f0202070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fa4f5538640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f0072220 0x7fa4f015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7f90f3e83640 1 Processor -- start 2024-04-26T21:35:34.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7f90f3e83640 1 -- start start 2024-04-26T21:35:34.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.787+0000 7fb86bea9640 1 -- start start 2024-04-26T21:35:34.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.786+0000 7ff5cf59e640 1 -- 172.21.15.40:0/4149685684 wait complete. 2024-04-26T21:35:34.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7ff5cf59e640 1 Processor -- start 2024-04-26T21:35:34.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7ff5cf59e640 1 -- start start 2024-04-26T21:35:34.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7fa4f653a640 1 -- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f006f420 msgr2=0x7fa4f0202070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7fa4f653a640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f006f420 0x7fa4f0202070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7fa4f653a640 1 -- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f0072220 msgr2=0x7fa4f015ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7fa4f653a640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f0072220 0x7fa4f015ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7fa4f653a640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f02025b0 con 0x7fa4f00b4e60 2024-04-26T21:35:34.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7fa4f653a640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f00b4e60 0x7fa4f01651c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c780 tx=0x7fa4e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7f2872459640 1 Processor -- start 2024-04-26T21:35:34.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0016020 con 0x7fa4f00b4e60 2024-04-26T21:35:34.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f2872459640 1 -- start start 2024-04-26T21:35:34.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.788+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e000a640 con 0x7fa4f00b4e60 2024-04-26T21:35:34.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f5435f14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54300704b0 0x7f543015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb86bea9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb864073f30 0x7fb8640b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb86bea9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8640c41d0 0x7fb864165110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb86bea9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8641678f0 0x7fb8640b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fdf4d2da640 1 -- 172.21.15.40:0/309217124 shutdown_connections 2024-04-26T21:35:34.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb21bfff640 1 Processor -- start 2024-04-26T21:35:34.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f5435f14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5430072510 0x7f54300b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fdf4d2da640 1 -- 172.21.15.40:0/309217124 wait complete. 2024-04-26T21:35:34.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb21bfff640 1 -- start start 2024-04-26T21:35:34.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f542f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54300704b0 0x7f543015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f542f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54300704b0 0x7f543015c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53650/0 (socket says 172.21.15.40:53650) 2024-04-26T21:35:34.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f542f7fe640 1 -- 172.21.15.40:0/1111270651 learned_addr learned my addr 172.21.15.40:0/1111270651 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f542effd640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5430072510 0x7f54300b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb86bea9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8640ce7b0 con 0x7fb8641678f0 2024-04-26T21:35:34.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb86bea9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8640cf430 con 0x7fb864073f30 2024-04-26T21:35:34.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb86bea9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8640ce930 con 0x7fb8640c41d0 2024-04-26T21:35:34.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb86a446640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8641678f0 0x7fb8640b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb869444640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8640c41d0 0x7fb864165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb869444640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8640c41d0 0x7fb864165110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53656/0 (socket says 172.21.15.40:53656) 2024-04-26T21:35:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb869444640 1 -- 172.21.15.40:0/1814141010 learned_addr learned my addr 172.21.15.40:0/1814141010 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f5435f14640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300c62c0 0x7f54300b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f54300cbbd0 con 0x7f5430072510 2024-04-26T21:35:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54300c6750 con 0x7f54300c62c0 2024-04-26T21:35:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f54300cbd50 con 0x7f54300704b0 2024-04-26T21:35:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f3e83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90ec0ce2f0 0x7f90ec0c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f3e83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90ec15d1c0 0x7f90ec15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f3e83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec160de0 0x7f90ec15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f3e83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec161ad0 con 0x7f90ec160de0 2024-04-26T21:35:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f3e83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec0d12d0 con 0x7f90ec0ce2f0 2024-04-26T21:35:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f3e83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90ec161c50 con 0x7f90ec15d1c0 2024-04-26T21:35:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f2420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec160de0 0x7f90ec15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f2420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec160de0 0x7f90ec15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45582/0 (socket says 172.21.15.40:45582) 2024-04-26T21:35:34.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7f90f2420640 1 -- 172.21.15.40:0/1471576597 learned_addr learned my addr 172.21.15.40:0/1471576597 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5cf59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c803b920 0x7ff5c8184ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.789+0000 7fb869c45640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb864073f30 0x7fb8640b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5cf59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8055720 0x7ff5c804f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f542ffff640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300c62c0 0x7f54300b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5cf59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80dfee0 0x7ff5c802e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5ce59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c803b920 0x7ff5c8184ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5cf59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c80e02c0 con 0x7ff5c8055720 2024-04-26T21:35:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5ce59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c803b920 0x7ff5c8184ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49652/0 (socket says 172.21.15.40:49652) 2024-04-26T21:35:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5ce59c640 1 -- 172.21.15.40:0/2147743229 learned_addr learned my addr 172.21.15.40:0/2147743229 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5cdd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8055720 0x7ff5c804f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f90f2420640 1 -- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90ec15d1c0 msgr2=0x7f90ec15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f90f2420640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90ec15d1c0 0x7f90ec15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f90f2420640 1 -- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90ec0ce2f0 msgr2=0x7f90ec0c8040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:35:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f90f2420640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90ec0ce2f0 0x7f90ec0c8040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f90f2420640 1 -- 172.21.15.40:0/1471576597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec1585f0 con 0x7f90ec160de0 2024-04-26T21:35:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8055200 con 0x7ff5c803b920 2024-04-26T21:35:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7fb86a446640 1 -- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8640c41d0 msgr2=0x7fb864165110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7fb86a446640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8640c41d0 0x7fb864165110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7fb86a446640 1 -- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb864073f30 msgr2=0x7fb8640b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7fb86a446640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb864073f30 0x7fb8640b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c8050ee0 con 0x7ff5c80dfee0 2024-04-26T21:35:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7fb86a446640 1 -- 172.21.15.40:0/1814141010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8640b5240 con 0x7fb8641678f0 2024-04-26T21:35:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7fb869444640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8640c41d0 0x7fb864165110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7fb869c45640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb864073f30 0x7fb8640b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7ff5ced9d640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80dfee0 0x7ff5c802e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f90f2420640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec160de0 0x7f90ec15b430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c3c0 tx=0x7f90dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7fb21bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 0x7fb21c1f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c072220 0x7fb21c159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f542ffff640 1 -- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54300704b0 msgr2=0x7f543015c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7ff5ce59c640 1 -- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80dfee0 msgr2=0x7ff5c802e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7ff5ce59c640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80dfee0 0x7ff5c802e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7ff5ce59c640 1 -- 172.21.15.40:0/2147743229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8055720 msgr2=0x7ff5c804f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7ff5ce59c640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8055720 0x7ff5c804f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21affd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 0x7fb21c1f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21affd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 0x7fb21c1f9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49666/0 (socket says 172.21.15.40:49666) 2024-04-26T21:35:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21affd640 1 -- 172.21.15.40:0/1322504185 learned_addr learned my addr 172.21.15.40:0/1322504185 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000aa30 con 0x7fa4f00b4e60 2024-04-26T21:35:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f015d350 con 0x7fa4f00b4e60 2024-04-26T21:35:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f0160af0 con 0x7fa4f00b4e60 2024-04-26T21:35:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f2872459640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854005210 0x7f286c1f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7ff5ce59c640 1 -- 172.21.15.40:0/2147743229 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c804da50 con 0x7ff5c803b920 2024-04-26T21:35:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f2872459640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f286c0c8cc0 0x7f286c0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21bfff640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 0x7fb21c1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f2872459640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f286c0cec60 0x7f286c1f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c0b4660 con 0x7fb21c072220 2024-04-26T21:35:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f2872459640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0d3c20 con 0x7f286c0cec60 2024-04-26T21:35:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c0b4380 con 0x7fb21c06f420 2024-04-26T21:35:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f2872459640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0d4190 con 0x7f2854005210 2024-04-26T21:35:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21b7fe640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 0x7fb21c1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f2872459640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f286c0d3da0 con 0x7f286c0c8cc0 2024-04-26T21:35:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb21c0b4940 con 0x7fb21c0b4e60 2024-04-26T21:35:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc017070 con 0x7f90ec160de0 2024-04-26T21:35:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90dc005ce0 con 0x7f90ec160de0 2024-04-26T21:35:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0072d0 con 0x7f90ec160de0 2024-04-26T21:35:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb21a7fc640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c072220 0x7fb21c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fdf4d2da640 1 Processor -- start 2024-04-26T21:35:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fdf4d2da640 1 -- start start 2024-04-26T21:35:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f286bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854005210 0x7f286c1f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7ff5ce59c640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c803b920 0x7ff5c8184ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0050be0 tx=0x7ff5d0075270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f286bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854005210 0x7f286c1f2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49668/0 (socket says 172.21.15.40:49668) 2024-04-26T21:35:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f286bfff640 1 -- 172.21.15.40:0/3293530914 learned_addr learned my addr 172.21.15.40:0/3293530914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb21affd640 1 -- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 msgr2=0x7fb21c1596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb21affd640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb21c0b4e60 0x7fb21c1596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb21affd640 1 -- 172.21.15.40:0/1322504185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c072220 msgr2=0x7fb21c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb21affd640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c072220 0x7fb21c159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb21affd640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c15a5d0 con 0x7fb21c06f420 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb21a7fc640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c072220 0x7fb21c159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f542ffff640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54300704b0 0x7f543015c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f542ffff640 1 -- 172.21.15.40:0/1111270651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5430072510 msgr2=0x7f54300b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f542ffff640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5430072510 0x7f54300b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f542ffff640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54300ca0b0 con 0x7f54300c62c0 2024-04-26T21:35:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.790+0000 7f542ffff640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300c62c0 0x7f54300b6590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f542000e7f0 tx=0x7f542000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420017070 con 0x7f54300c62c0 2024-04-26T21:35:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542000c940 con 0x7f54300c62c0 2024-04-26T21:35:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f90f3e83640 1 -- 172.21.15.40:0/1471576597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ec0ca700 con 0x7f90ec160de0 2024-04-26T21:35:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7fb86a446640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8641678f0 0x7fb8640b4ad0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb86406c4e0 tx=0x7fb85c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.791+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420017070 con 0x7f54300c62c0 2024-04-26T21:35:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54300ca390 con 0x7f54300c62c0 2024-04-26T21:35:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f90f3e83640 1 -- 172.21.15.40:0/1471576597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ec155e20 con 0x7f90ec160de0 2024-04-26T21:35:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0090070 con 0x7ff5c803b920 2024-04-26T21:35:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c036040 con 0x7fb8641678f0 2024-04-26T21:35:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54300ca950 con 0x7f54300c62c0 2024-04-26T21:35:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf4d2da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf340039b0 0x7fdf480b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf4d2da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf481626a0 0x7fdf48165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf4d2da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf481550e0 0x7fdf480d0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf467fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf481626a0 0x7fdf48165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf467fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf481626a0 0x7fdf48165bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49672/0 (socket says 172.21.15.40:49672) 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf4d2da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf480d2340 con 0x7fdf340039b0 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf4d2da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf48167b80 con 0x7fdf481626a0 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf4d2da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf480b8420 con 0x7fdf481550e0 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf467fc640 1 -- 172.21.15.40:0/588757789 learned_addr learned my addr 172.21.15.40:0/588757789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf46ffd640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf340039b0 0x7fdf480b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fdf477fe640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf481550e0 0x7fdf480d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f286bfff640 1 -- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f286c0c8cc0 msgr2=0x7f286c0ce720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f286bfff640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f286c0c8cc0 0x7f286c0ce720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f286bfff640 1 -- 172.21.15.40:0/3293530914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f286c0cec60 msgr2=0x7f286c1f4fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f286bfff640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f286c0cec60 0x7f286c1f4fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7f286bfff640 1 -- 172.21.15.40:0/3293530914 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c1f5d30 con 0x7f2854005210 2024-04-26T21:35:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c804dd30 con 0x7ff5c803b920 2024-04-26T21:35:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c804e2f0 con 0x7ff5c803b920 2024-04-26T21:35:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d00723d0 con 0x7ff5c803b920 2024-04-26T21:35:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fb21affd640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 0x7fb21c1f9cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb214000f80 tx=0x7fb214004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c031400 con 0x7fb8641678f0 2024-04-26T21:35:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.792+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c009510 con 0x7fb8641678f0 2024-04-26T21:35:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8640b5520 con 0x7fb8641678f0 2024-04-26T21:35:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8640b5ae0 con 0x7fb8641678f0 2024-04-26T21:35:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0072840 con 0x7ff5c803b920 2024-04-26T21:35:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7fdf467fc640 1 -- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf481550e0 msgr2=0x7fdf480d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7fdf467fc640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf481550e0 0x7fdf480d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7fdf467fc640 1 -- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf340039b0 msgr2=0x7fdf480b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7fdf467fc640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf340039b0 0x7fdf480b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7fdf467fc640 1 -- 172.21.15.40:0/588757789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf480b7b30 con 0x7fdf481626a0 2024-04-26T21:35:34.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7f90f3e83640 1 -- 172.21.15.40:0/1471576597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90ec0c8580 con 0x7f90ec160de0 2024-04-26T21:35:34.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7fdf46ffd640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf340039b0 0x7fdf480b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:34.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214036050 con 0x7fb21c06f420 2024-04-26T21:35:34.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb214042070 con 0x7fb21c06f420 2024-04-26T21:35:34.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214009510 con 0x7fb21c06f420 2024-04-26T21:35:34.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.794+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5b4001030 con 0x7ff5c803b920 2024-04-26T21:35:34.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.793+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4f0069370 con 0x7fa4f00b4e60 2024-04-26T21:35:34.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f286bfff640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854005210 0x7f286c1f2450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f285c00f910 tx=0x7f285c00fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb8640694f0 con 0x7fb8641678f0 2024-04-26T21:35:34.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7fdf467fc640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf481626a0 0x7fdf48165bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800aef0 tx=0x7fdf38007430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21c15a8b0 con 0x7fb21c06f420 2024-04-26T21:35:34.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f542001a070 con 0x7f54300c62c0 2024-04-26T21:35:34.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54300cad30 con 0x7f54300c62c0 2024-04-26T21:35:34.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21c15ae70 con 0x7fb21c06f420 2024-04-26T21:35:34.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0052e0 con 0x7f2854005210 2024-04-26T21:35:34.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c018070 con 0x7f2854005210 2024-04-26T21:35:34.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5d0093070 con 0x7ff5c803b920 2024-04-26T21:35:34.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c0cc260 con 0x7f2854005210 2024-04-26T21:35:34.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c0cc820 con 0x7f2854005210 2024-04-26T21:35:34.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7ff5beffd640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b006c9f0 0x7ff5b006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff5d0108b80 con 0x7ff5c803b920 2024-04-26T21:35:34.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7ff5beffd640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7ff5b0076170 0x7ff5b00785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5d0073e50 con 0x7ff5b0076170 2024-04-26T21:35:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff5d010d2a0 con 0x7ff5c803b920 2024-04-26T21:35:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7fb2017fa640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1f4001030 con 0x7fb21c06f420 2024-04-26T21:35:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7ff5ced9d640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7ff5b0076170 0x7ff5b00785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7fdf3f7fe640 1 -- 172.21.15.40:0/588757789 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38017020 con 0x7fdf481626a0 2024-04-26T21:35:34.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90dc01a070 con 0x7f90ec160de0 2024-04-26T21:35:34.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fdf3f7fe640 1 -- 172.21.15.40:0/588757789 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf38013600 con 0x7fdf481626a0 2024-04-26T21:35:34.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c00da10 con 0x7f2854005210 2024-04-26T21:35:34.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf480b7dd0 con 0x7fdf481626a0 2024-04-26T21:35:34.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7f90e27fc640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d006c9c0 0x7f90d006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f90dc08b9f0 con 0x7f90ec160de0 2024-04-26T21:35:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7f90f1c1f640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d006c9c0 0x7f90d006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7f90e27fc640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f90d0076140 0x7f90d0078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90d0078c50 con 0x7f90d0076140 2024-04-26T21:35:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f90dc08bde0 con 0x7f90ec160de0 2024-04-26T21:35:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fdf3f7fe640 1 -- 172.21.15.40:0/588757789 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf380139b0 con 0x7fdf481626a0 2024-04-26T21:35:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7ff5ced9d640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7ff5b0076170 0x7ff5b00785b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f5417fff640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f541c06ccf0 0x7f541c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7ff5cdd9b640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b006c9f0 0x7ff5b006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f542000f040 con 0x7f54300c62c0 2024-04-26T21:35:34.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f5417fff640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7f541c076370 0x7f541c0787b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.795+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5420015e50 con 0x7f541c076370 2024-04-26T21:35:34.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7f542f7fe640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f541c06ccf0 0x7f541c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7f542effd640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7f541c076370 0x7f541c0787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7f542f7fe640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f541c06ccf0 0x7f541c06f1b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f542400d440 tx=0x7f542400d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7f542effd640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7f541c076370 0x7f541c0787b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb85c043030 con 0x7fb8641678f0 2024-04-26T21:35:34.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fb85a7fc640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb84806c9c0 0x7fb84806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f286c0694f0 con 0x7f2854005210 2024-04-26T21:35:34.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb214047460 con 0x7fb21c06f420 2024-04-26T21:35:34.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb85c0ac350 con 0x7fb8641678f0 2024-04-26T21:35:34.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fb85a7fc640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7fb848076130 0x7fb848078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb85c008e50 con 0x7fb848076130 2024-04-26T21:35:34.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb85c0b3050 con 0x7fb8641678f0 2024-04-26T21:35:34.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5420055110 con 0x7f54300c62c0 2024-04-26T21:35:34.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5420015e50 con 0x7f541c076370 2024-04-26T21:35:34.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fb2037fe640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb20806c940 0x7fb20806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb2140ad0f0 con 0x7fb21c06f420 2024-04-26T21:35:34.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fb2037fe640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb2080760c0 0x7fb208078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb214031600 con 0x7fb2080760c0 2024-04-26T21:35:34.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb2140b4050 con 0x7fb21c06f420 2024-04-26T21:35:34.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7fb21a7fc640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb20806c940 0x7fb20806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f285c00dbb0 con 0x7f2854005210 2024-04-26T21:35:34.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.796+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4e0019020 con 0x7fa4f00b4e60 2024-04-26T21:35:34.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fa4e67fc640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4d406c9c0 0x7fa4d406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.797+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa4e008c2f0 con 0x7fa4f00b4e60 2024-04-26T21:35:34.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fa4e67fc640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fa4d4076140 0x7fa4d4078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4e000bf30 con 0x7fa4d4076140 2024-04-26T21:35:34.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa4e008c6e0 con 0x7fa4f00b4e60 2024-04-26T21:35:34.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fa4f5d39640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4d406c9c0 0x7fa4d406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fa4f5538640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fa4d4076140 0x7fa4d4078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7fb21b7fe640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb2080760c0 0x7fb208078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7fa4f5538640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fa4d4076140 0x7fa4d4078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5d0073e50 con 0x7ff5b0076170 2024-04-26T21:35:34.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7ff5cdd9b640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b006c9f0 0x7ff5b006eeb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4034000 tx=0x7ff5c402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7fdf3f7fe640 1 -- 172.21.15.40:0/588757789 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf3801a020 con 0x7fdf481626a0 2024-04-26T21:35:34.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7f2868ff9640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f285806c9f0 0x7f285806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7fa4f5d39640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4d406c9c0 0x7fa4d406ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009b30 tx=0x7fa4ec009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f285c08bdb0 con 0x7f2854005210 2024-04-26T21:35:34.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7fb21b7fe640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb2080760c0 0x7fb208078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7f286b7fe640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f285806c9f0 0x7f285806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7f2868ff9640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f2858076170 0x7f28580785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f285c005480 con 0x7f2858076170 2024-04-26T21:35:34.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f285c08e410 con 0x7f2854005210 2024-04-26T21:35:34.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7f90f1c1f640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d006c9c0 0x7f90d006ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009670 tx=0x7f90e8006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7f90f141e640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f90d0076140 0x7f90d0078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.799+0000 7f90f141e640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f90d0076140 0x7f90d0078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7fb869444640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7fb848076130 0x7fb848078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7fb869c45640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb84806c9c0 0x7fb84806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.798+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf481623e0 con 0x7fdf481626a0 2024-04-26T21:35:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7f28709f6640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f2858076170 0x7f28580785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7fdf3f7fe640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf2806c940 0x7fdf2806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb214031600 con 0x7fb2080760c0 2024-04-26T21:35:34.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7fdf3f7fe640 1 -- 172.21.15.40:0/588757789 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdf3808bcb0 con 0x7fdf481626a0 2024-04-26T21:35:34.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7fb21a7fc640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb20806c940 0x7fb20806ee00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb210006fd0 tx=0x7fb210008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.800+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 <== osd.3 v2:172.21.15.40:6824/4238362919 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4e000bf30 con 0x7fa4d4076140 2024-04-26T21:35:34.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.801+0000 7fdf46ffd640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf2806c940 0x7fdf2806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.801+0000 7f286b7fe640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f285806c9f0 0x7f285806eeb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2860000f60 tx=0x7f2860000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.801+0000 7f28709f6640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f2858076170 0x7f28580785b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.801+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90d0078c50 con 0x7f90d0076140 2024-04-26T21:35:34.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.801+0000 7fdf46ffd640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf2806c940 0x7fdf2806ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf40033000 tx=0x7fdf40008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.801+0000 7fb869c45640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb84806c9c0 0x7fb84806ee80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb854000f60 tx=0x7fb854009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.801+0000 7fdf4d2da640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fdf481554c0 0x7fdf480c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:34.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.801+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf48069a80 con 0x7fdf481554c0 2024-04-26T21:35:34.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.802+0000 7fb869444640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7fb848076130 0x7fb848078570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.802+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f285c005480 con 0x7f2858076170 2024-04-26T21:35:34.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.802+0000 7fdf477fe640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fdf481554c0 0x7fdf480c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:34.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.802+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb85c008e50 con 0x7fb848076130 2024-04-26T21:35:34.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.802+0000 7fdf477fe640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fdf481554c0 0x7fdf480c8b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:34.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.804+0000 7fdf3f7fe640 1 -- 172.21.15.40:0/588757789 <== osd.2 v2:172.21.15.40:6811/3280366756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdf48069a80 con 0x7fdf481554c0 2024-04-26T21:35:34.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.831+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff5b4002dd0 con 0x7ff5b0076170 2024-04-26T21:35:34.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.831+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5430069370 con 0x7f541c076370 2024-04-26T21:35:34.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7ff5beffd640 1 -- 172.21.15.40:0/2147743229 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff5b4002dd0 con 0x7ff5b0076170 2024-04-26T21:35:34.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7f5417fff640 1 -- 172.21.15.40:0/1111270651 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5430069370 con 0x7f541c076370 2024-04-26T21:35:34.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7ff5b0076170 msgr2=0x7ff5b00785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7ff5cf59e640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7ff5b0076170 0x7ff5b00785b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b006c9f0 msgr2=0x7ff5b006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7ff5cf59e640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b006c9f0 0x7ff5b006eeb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4034000 tx=0x7ff5c402e000 comp rx=0 tx=0).stop 2024-04-26T21:35:34.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c803b920 msgr2=0x7ff5c8184ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7ff5cf59e640 1 --2- 172.21.15.40:0/2147743229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c803b920 0x7ff5c8184ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0050be0 tx=0x7ff5d0075270 comp rx=0 tx=0).stop 2024-04-26T21:35:34.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7ff5ced9d640 1 -- 172.21.15.40:0/2147743229 reap_dead start 2024-04-26T21:35:34.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7f541c076370 msgr2=0x7f541c0787b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.832+0000 7f5435f14640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7f541c076370 0x7f541c0787b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f541c06ccf0 msgr2=0x7f541c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f5435f14640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f541c06ccf0 0x7f541c06f1b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f542400d440 tx=0x7f542400d000 comp rx=0 tx=0).stop 2024-04-26T21:35:34.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300c62c0 msgr2=0x7f54300b6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f5435f14640 1 --2- 172.21.15.40:0/1111270651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54300c62c0 0x7f54300b6590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f542000e7f0 tx=0x7f542000ecb0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 shutdown_connections 2024-04-26T21:35:34.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 >> 172.21.15.40:0/2147743229 conn(0x7ff5c8052650 msgr2=0x7ff5c803a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 shutdown_connections 2024-04-26T21:35:34.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7ff5cf59e640 1 -- 172.21.15.40:0/2147743229 wait complete. 2024-04-26T21:35:34.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f542ffff640 1 -- 172.21.15.40:0/1111270651 reap_dead start 2024-04-26T21:35:34.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 shutdown_connections 2024-04-26T21:35:34.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 >> 172.21.15.40:0/1111270651 conn(0x7f54300d09f0 msgr2=0x7f5430071e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 shutdown_connections 2024-04-26T21:35:34.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f5435f14640 1 -- 172.21.15.40:0/1111270651 wait complete. 2024-04-26T21:35:34.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.833+0000 7f90f3e83640 1 -- 172.21.15.40:0/1471576597 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90ec0ce6f0 con 0x7f90d0076140 2024-04-26T21:35:34.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.834+0000 7f90e27fc640 1 -- 172.21.15.40:0/1471576597 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f90ec0ce6f0 con 0x7f90d0076140 2024-04-26T21:35:34.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7f90cffff640 1 -- 172.21.15.40:0/1471576597 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f90d0076140 msgr2=0x7f90d0078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7f90cffff640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f90d0076140 0x7f90d0078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7f90cffff640 1 -- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d006c9c0 msgr2=0x7f90d006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7f90cffff640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90d006c9c0 0x7f90d006ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009670 tx=0x7f90e8006bd0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7f90cffff640 1 -- 172.21.15.40:0/1471576597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec160de0 msgr2=0x7f90ec15b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7f90cffff640 1 --2- 172.21.15.40:0/1471576597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90ec160de0 0x7f90ec15b430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c3c0 tx=0x7f90dc00c880 comp rx=0 tx=0).stop 2024-04-26T21:35:34.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7fb2017fa640 1 -- 172.21.15.40:0/1322504185 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1f4002dd0 con 0x7fb2080760c0 2024-04-26T21:35:34.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7fa4f6d3b640 1 -- 172.21.15.40:0/2292674625 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa4f01604c0 con 0x7fa4d4076140 2024-04-26T21:35:34.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7f90f2420640 1 -- 172.21.15.40:0/1471576597 reap_dead start 2024-04-26T21:35:34.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.835+0000 7fa4e67fc640 1 -- 172.21.15.40:0/2292674625 <== osd.3 v2:172.21.15.40:6824/4238362919 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa4f01604c0 con 0x7fa4d4076140 2024-04-26T21:35:34.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7fb2037fe640 1 -- 172.21.15.40:0/1322504185 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb1f4002dd0 con 0x7fb2080760c0 2024-04-26T21:35:34.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7f90cffff640 1 -- 172.21.15.40:0/1471576597 shutdown_connections 2024-04-26T21:35:34.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7f90cffff640 1 -- 172.21.15.40:0/1471576597 >> 172.21.15.40:0/1471576597 conn(0x7f90ec0b4760 msgr2=0x7f90ec0cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7f90cffff640 1 -- 172.21.15.40:0/1471576597 shutdown_connections 2024-04-26T21:35:34.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f286c068d30 con 0x7f2858076170 2024-04-26T21:35:34.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdf4806cfe0 con 0x7fdf481554c0 2024-04-26T21:35:34.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7f90cffff640 1 -- 172.21.15.40:0/1471576597 wait complete. 2024-04-26T21:35:34.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7fa4d3fff640 1 -- 172.21.15.40:0/2292674625 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fa4d4076140 msgr2=0x7fa4d4078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb2080760c0 msgr2=0x7fb208078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7fb21bfff640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb2080760c0 0x7fb208078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.836+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb8640d0430 con 0x7fb848076130 2024-04-26T21:35:34.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fa4d3fff640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fa4d4076140 0x7fa4d4078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb20806c940 msgr2=0x7fb20806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb21bfff640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb20806c940 0x7fb20806ee00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb210006fd0 tx=0x7fb210008040 comp rx=0 tx=0).stop 2024-04-26T21:35:34.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7f2868ff9640 1 -- 172.21.15.40:0/3293530914 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f286c068d30 con 0x7f2858076170 2024-04-26T21:35:34.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fa4d3fff640 1 -- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4d406c9c0 msgr2=0x7fa4d406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 msgr2=0x7fb21c1f9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fdf3f7fe640 1 -- 172.21.15.40:0/588757789 <== osd.2 v2:172.21.15.40:6811/3280366756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdf4806cfe0 con 0x7fdf481554c0 2024-04-26T21:35:34.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fa4d3fff640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4d406c9c0 0x7fa4d406ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009b30 tx=0x7fa4ec009240 comp rx=0 tx=0).stop 2024-04-26T21:35:34.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb21bfff640 1 --2- 172.21.15.40:0/1322504185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c06f420 0x7fb21c1f9cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb214000f80 tx=0x7fb214004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:34.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fa4d3fff640 1 -- 172.21.15.40:0/2292674625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f00b4e60 msgr2=0x7fa4f01651c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fa4d3fff640 1 --2- 172.21.15.40:0/2292674625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f00b4e60 0x7fa4f01651c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c780 tx=0x7fa4e000cc40 comp rx=0 tx=0).stop 2024-04-26T21:35:34.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb85a7fc640 1 -- 172.21.15.40:0/1814141010 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb8640d0430 con 0x7fb848076130 2024-04-26T21:35:34.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb21b7fe640 1 -- 172.21.15.40:0/1322504185 reap_dead start 2024-04-26T21:35:34.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fa4f653a640 1 -- 172.21.15.40:0/2292674625 reap_dead start 2024-04-26T21:35:34.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fa4d3fff640 1 -- 172.21.15.40:0/2292674625 shutdown_connections 2024-04-26T21:35:34.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fa4d3fff640 1 -- 172.21.15.40:0/2292674625 >> 172.21.15.40:0/2292674625 conn(0x7fa4f00b2720 msgr2=0x7fa4f015d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fdf481554c0 msgr2=0x7fdf480c8b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fdf4d2da640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fdf481554c0 0x7fdf480c8b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf2806c940 msgr2=0x7fdf2806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf4d2da640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf2806c940 0x7fdf2806ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf40033000 tx=0x7fdf40008030 comp rx=0 tx=0).stop 2024-04-26T21:35:34.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf481626a0 msgr2=0x7fdf48165bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 shutdown_connections 2024-04-26T21:35:34.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 >> 172.21.15.40:0/1322504185 conn(0x7fb21c0b2720 msgr2=0x7fb21c06dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7fb848076130 msgr2=0x7fb848078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.837+0000 7fb86bea9640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7fb848076130 0x7fb848078570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb84806c9c0 msgr2=0x7fb84806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb86bea9640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb84806c9c0 0x7fb84806ee80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb854000f60 tx=0x7fb854009040 comp rx=0 tx=0).stop 2024-04-26T21:35:34.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8641678f0 msgr2=0x7fb8640b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb86bea9640 1 --2- 172.21.15.40:0/1814141010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8641678f0 0x7fb8640b4ad0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb86406c4e0 tx=0x7fb85c004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:34.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf4d2da640 1 --2- 172.21.15.40:0/588757789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf481626a0 0x7fdf48165bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800aef0 tx=0x7fdf38007430 comp rx=0 tx=0).stop 2024-04-26T21:35:34.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb86a446640 1 -- 172.21.15.40:0/1814141010 reap_dead start 2024-04-26T21:35:34.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf477fe640 1 -- 172.21.15.40:0/588757789 reap_dead start 2024-04-26T21:35:34.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 shutdown_connections 2024-04-26T21:35:34.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f2858076170 msgr2=0x7f28580785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f2858076170 0x7f28580785b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f285806c9f0 msgr2=0x7f285806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f285806c9f0 0x7f285806eeb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2860000f60 tx=0x7f2860000ed0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 shutdown_connections 2024-04-26T21:35:34.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854005210 msgr2=0x7f286c1f2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:34.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 --2- 172.21.15.40:0/3293530914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2854005210 0x7f286c1f2450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f285c00f910 tx=0x7f285c00fdd0 comp rx=0 tx=0).stop 2024-04-26T21:35:34.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 >> 172.21.15.40:0/1814141010 conn(0x7fb8640d0be0 msgr2=0x7fb8640cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f28709f6640 1 -- 172.21.15.40:0/3293530914 reap_dead start 2024-04-26T21:35:34.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 shutdown_connections 2024-04-26T21:35:34.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fb21bfff640 1 -- 172.21.15.40:0/1322504185 wait complete. 2024-04-26T21:35:34.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 shutdown_connections 2024-04-26T21:35:34.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 >> 172.21.15.40:0/3293530914 conn(0x7f286c0d5190 msgr2=0x7f286c0d5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 >> 172.21.15.40:0/588757789 conn(0x7fdf480d4e70 msgr2=0x7fdf480ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:34.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 shutdown_connections 2024-04-26T21:35:34.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7f2872459640 1 -- 172.21.15.40:0/3293530914 wait complete. 2024-04-26T21:35:34.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 shutdown_connections 2024-04-26T21:35:34.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fa4d3fff640 1 -- 172.21.15.40:0/2292674625 shutdown_connections 2024-04-26T21:35:34.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fa4d3fff640 1 -- 172.21.15.40:0/2292674625 wait complete. 2024-04-26T21:35:34.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.838+0000 7fdf4d2da640 1 -- 172.21.15.40:0/588757789 wait complete. 2024-04-26T21:35:34.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.839+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 shutdown_connections 2024-04-26T21:35:34.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:34.839+0000 7fb86bea9640 1 -- 172.21.15.40:0/1814141010 wait complete. 2024-04-26T21:35:34.907 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:34.908 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T21:35:34.908 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:34.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T21:35:34.909 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:34.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T21:35:34.910 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:34.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T21:35:34.911 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:34.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T21:35:34.912 INFO:teuthology.orchestra.run.smithi040.stdout:55834574857 2024-04-26T21:35:34.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T21:35:34.913 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:34.914 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T21:35:34.915 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:34.915 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T21:35:35.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.123+0000 7f686d028640 1 Processor -- start 2024-04-26T21:35:35.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.123+0000 7fd019f64640 1 Processor -- start 2024-04-26T21:35:35.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.123+0000 7fc3b9c07640 1 Processor -- start 2024-04-26T21:35:35.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.123+0000 7fd019f64640 1 -- start start 2024-04-26T21:35:35.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.124+0000 7f1cf234d640 1 Processor -- start 2024-04-26T21:35:35.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.123+0000 7fc3b9c07640 1 -- start start 2024-04-26T21:35:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.123+0000 7f686d028640 1 -- start start 2024-04-26T21:35:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.124+0000 7f686d028640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f686815f190 0x7f68680d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.124+0000 7f686d028640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f686815f570 0x7f68680d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.124+0000 7f686d028640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68680d2ed0 con 0x7f68680d59c0 2024-04-26T21:35:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.124+0000 7f6866d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f686815f190 0x7f68680d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fd019f64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd00c0e30c0 0x7fd00c057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3b9c07640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3b40c45b0 0x7fc3b40cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.124+0000 7f686d028640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f68680d2c80 con 0x7f686815f190 2024-04-26T21:35:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.124+0000 7f686d028640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f68680b1c40 con 0x7f686815f570 2024-04-26T21:35:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3b9c07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b40cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f9ab9db4640 1 Processor -- start 2024-04-26T21:35:35.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3b9c07640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b4163c00 con 0x7fc3b40ce130 2024-04-26T21:35:35.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3b9c07640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b40ce9e0 con 0x7fc3b40c45b0 2024-04-26T21:35:35.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f0665c66640 1 Processor -- start 2024-04-26T21:35:35.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3b9c07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b40b1a20 con 0x7fc3b40cd3e0 2024-04-26T21:35:35.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3b37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3b40c45b0 0x7fc3b40cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3b37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3b40c45b0 0x7fc3b40cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45622/0 (socket says 172.21.15.40:45622) 2024-04-26T21:35:35.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3b37fe640 1 -- 172.21.15.40:0/2727855676 learned_addr learned my addr 172.21.15.40:0/2727855676 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fc3aaffd640 1 --2- 172.21.15.40:0/2727855676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b40cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.124+0000 7f1cf234d640 1 -- start start 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f1cf234d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ce40e7820 0x7f1ce4038280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f1cf234d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40eb300 0x7f1ce40eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f1cf234d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1ce40ebcc0 0x7f1ce4053520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f1cf234d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce40e7c00 con 0x7f1ce40e7820 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f1cf234d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce4057f20 con 0x7f1ce40ebcc0 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f1cf234d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce4058200 con 0x7f1ce40eb300 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f686759e640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f68680d59c0 0x7f68680cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:57982/0 (socket says 172.21.15.40:57982) 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f686759e640 1 -- 172.21.15.40:0/3532953989 learned_addr learned my addr 172.21.15.40:0/3532953989 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f686659c640 1 --2- 172.21.15.40:0/3532953989 >> v2:172.21.15.184:3300/0 conn(0x7f686815f570 0x7f68680d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f6865d9b640 1 -- 172.21.15.40:0/3532953989 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3674859686 0 0) 0x7f68680d2ed0 con 0x7f68680d59c0 2024-04-26T21:35:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f6865d9b640 1 -- 172.21.15.40:0/3532953989 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6850003520 con 0x7f68680d59c0 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f686659c640 1 -- 172.21.15.40:0/3532953989 >> v2:172.21.15.40:3300/0 conn(0x7f686815f190 msgr2=0x7f68680d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f686659c640 1 --2- 172.21.15.40:0/3532953989 >> v2:172.21.15.40:3300/0 conn(0x7f686815f190 0x7f68680d32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fd018f62640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd00c0e30c0 0x7fd00c057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fd018f62640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd00c0e30c0 0x7fd00c057d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49682/0 (socket says 172.21.15.40:49682) 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fd018f62640 1 -- 172.21.15.40:0/4269881498 learned_addr learned my addr 172.21.15.40:0/4269881498 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fd013fff640 1 --1- 172.21.15.40:0/4269881498 >> v1:172.21.15.184:6789/0 conn(0x7fd00c03e840 0x7fd00c053bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:57986/0 (socket says 172.21.15.40:57986) 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fd019f64640 1 -- 172.21.15.40:0/4269881498 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00c0582d0 con 0x7fd00c054300 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fd019f64640 1 -- 172.21.15.40:0/4269881498 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00c058570 con 0x7fd00c03e840 2024-04-26T21:35:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7fd019f64640 1 -- 172.21.15.40:0/4269881498 --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd00c058850 con 0x7fd00c0e30c0 2024-04-26T21:35:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f686659c640 1 -- 172.21.15.40:0/3532953989 >> v1:172.21.15.184:6789/0 conn(0x7f68680d59c0 legacy=0x7f68680cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f686659c640 1 -- 172.21.15.40:0/3532953989 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68680b1f20 con 0x7f686815f570 2024-04-26T21:35:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f6866d9d640 1 --2- 172.21.15.40:0/3532953989 >> v2:172.21.15.40:3300/0 conn(0x7f686815f190 0x7f68680d32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f1cebfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ce40e7820 0x7f1ce4038280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f1cf08ea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1ce40ebcc0 0x7f1ce4053520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fc3b2ffd640 1 -- 172.21.15.40:0/2727855676 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045260129 0 0) 0x7fc3b4163c00 con 0x7fc3b40ce130 2024-04-26T21:35:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fc3b2ffd640 1 -- 172.21.15.40:0/2727855676 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3a0003520 con 0x7fc3b40ce130 2024-04-26T21:35:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fc3aaffd640 1 -- 172.21.15.40:0/2727855676 >> v2:172.21.15.40:3300/0 conn(0x7fc3b40c45b0 msgr2=0x7fc3b40cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fc3aaffd640 1 --2- 172.21.15.40:0/2727855676 >> v2:172.21.15.40:3300/0 conn(0x7fc3b40c45b0 0x7fc3b40cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fc3aaffd640 1 -- 172.21.15.40:0/2727855676 >> v1:172.21.15.184:6789/0 conn(0x7fc3b40ce130 legacy=0x7fc3b40cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fc3aaffd640 1 -- 172.21.15.40:0/2727855676 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b40ce670 con 0x7fc3b40cd3e0 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40eb300 0x7f1ce40eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40eb300 0x7f1ce40eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53698/0 (socket says 172.21.15.40:53698) 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f1ceb7fe640 1 -- 172.21.15.40:0/223432014 learned_addr learned my addr 172.21.15.40:0/223432014 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.125+0000 7f0665c66640 1 -- start start 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f0665c66640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f066015b4d0 0x7f066006ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f0665c66640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06600727a0 0x7f066015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f0665c66640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f066015bdf0 0x7f06600b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f0665c66640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f06600b44c0 con 0x7f066015bdf0 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f0665c66640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f06600b5f30 con 0x7f066015b4d0 2024-04-26T21:35:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f0665c66640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0660072ff0 con 0x7f06600727a0 2024-04-26T21:35:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f1cf08ea640 1 -- 172.21.15.40:0/223432014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40eb300 msgr2=0x7f1ce40eb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f1cf08ea640 1 --2- 172.21.15.40:0/223432014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40eb300 0x7f1ce40eb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f1cf08ea640 1 -- 172.21.15.40:0/223432014 >> v2:172.21.15.40:3300/0 conn(0x7f1ce40e7820 msgr2=0x7f1ce4038280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2853133384 0 0) 0x7fd00c0582d0 con 0x7fd00c054300 2024-04-26T21:35:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd004003520 con 0x7fd00c054300 2024-04-26T21:35:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2946254661 0 0) 0x7fd004003520 con 0x7fd00c054300 2024-04-26T21:35:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 >> v2:172.21.15.184:3300/0 conn(0x7fd00c0e30c0 msgr2=0x7fd00c057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 --2- 172.21.15.40:0/4269881498 >> v2:172.21.15.184:3300/0 conn(0x7fd00c0e30c0 0x7fd00c057d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 >> v1:172.21.15.184:6789/0 conn(0x7fd00c03e840 legacy=0x7fd00c053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c058b30 con 0x7fd00c054300 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fd008001690 con 0x7fd00c054300 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 >> v1:172.21.15.40:6789/0 conn(0x7fd00c054300 legacy=0x7fd00c0068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0040039b0 0x7fd004003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd004004800 0x7fd004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0040051c0 0x7fd00400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c058850 con 0x7fd0040039b0 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd00400c0d0 con 0x7fd0040051c0 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd00400c2d0 con 0x7fd004004800 2024-04-26T21:35:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7fd019763640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0040051c0 0x7fd00400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7fadf836a640 1 Processor -- start 2024-04-26T21:35:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7fadf836a640 1 -- start start 2024-04-26T21:35:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f065ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06600727a0 0x7f066015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f686659c640 1 --2- 172.21.15.40:0/3532953989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 0x7f68680d3810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f68680b79a0 tx=0x7f685c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c23ba1c0a90d0ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f0664c64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f066015b4d0 0x7f066006ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7fd019763640 1 -- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd004004800 msgr2=0x7fd004004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:35:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7fd019763640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd004004800 0x7fd004004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7fd019763640 1 -- 172.21.15.40:0/4269881498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0040039b0 msgr2=0x7fd004003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:35:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7fd019763640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0040039b0 0x7fd004003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7fd019763640 1 -- 172.21.15.40:0/4269881498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c058b30 con 0x7fd0040051c0 2024-04-26T21:35:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f0664c64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f066015b4d0 0x7f066006ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49710/0 (socket says 172.21.15.40:49710) 2024-04-26T21:35:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f6865d9b640 1 -- 172.21.15.40:0/3532953989 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c037070 con 0x7f686815f570 2024-04-26T21:35:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f065ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06600727a0 0x7f066015b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53710/0 (socket says 172.21.15.40:53710) 2024-04-26T21:35:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.126+0000 7f9ab9db4640 1 -- start start 2024-04-26T21:35:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f9ab9db4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab40d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f9ab9db4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ab40d5b20 0x7f9ab40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f9ab9db4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab40d3db0 con 0x7f9ab4167470 2024-04-26T21:35:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f9ab9db4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab40d1eb0 con 0x7f9ab40d5b20 2024-04-26T21:35:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f9ab9db4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab40b8cd0 con 0x7f9ab40d32a0 2024-04-26T21:35:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f6865d9b640 1 -- 172.21.15.40:0/3532953989 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c032440 con 0x7f686815f570 2024-04-26T21:35:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f6865d9b640 1 -- 172.21.15.40:0/3532953989 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c02ae60 con 0x7f686815f570 2024-04-26T21:35:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f065ffff640 1 -- 172.21.15.40:0/1068256939 learned_addr learned my addr 172.21.15.40:0/1068256939 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab3fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f9ab4167470 0x7f9ab4167920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:58018/0 (socket says 172.21.15.40:58018) 2024-04-26T21:35:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab3fff640 1 -- 172.21.15.40:0/4081779519 learned_addr learned my addr 172.21.15.40:0/4081779519 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f1cf08ea640 1 --2- 172.21.15.40:0/223432014 >> v2:172.21.15.40:3300/0 conn(0x7f1ce40e7820 0x7f1ce4038280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.127+0000 7f1cf08ea640 1 -- 172.21.15.40:0/223432014 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce40584e0 con 0x7f1ce40ebcc0 2024-04-26T21:35:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7fc3aaffd640 1 --2- 172.21.15.40:0/2727855676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b40cf970 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0097a0 tx=0x7fc39c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccf282ced70d1f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f1cebfff640 1 --2- 172.21.15.40:0/223432014 >> v2:172.21.15.40:3300/0 conn(0x7f1ce40e7820 0x7f1ce4038280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab2ffd640 1 --2- 172.21.15.40:0/4081779519 >> v2:172.21.15.40:3300/0 conn(0x7f9ab40d5b20 0x7f9ab40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab37fe640 1 --2- 172.21.15.40:0/4081779519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab40d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f065ffff640 1 -- 172.21.15.40:0/1068256939 >> v2:172.21.15.184:3300/0 conn(0x7f066015b4d0 msgr2=0x7f066006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f0665465640 1 --2- 172.21.15.40:0/1068256939 >> v2:172.21.15.40:3300/0 conn(0x7f066015bdf0 0x7f06600b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f065ffff640 1 --2- 172.21.15.40:0/1068256939 >> v2:172.21.15.184:3300/0 conn(0x7f066015b4d0 0x7f066006ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f065ffff640 1 -- 172.21.15.40:0/1068256939 >> v2:172.21.15.40:3300/0 conn(0x7f066015bdf0 msgr2=0x7f06600b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f065ffff640 1 --2- 172.21.15.40:0/1068256939 >> v2:172.21.15.40:3300/0 conn(0x7f066015bdf0 0x7f06600b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f065ffff640 1 -- 172.21.15.40:0/1068256939 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06600732d0 con 0x7f06600727a0 2024-04-26T21:35:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f0665465640 1 --2- 172.21.15.40:0/1068256939 >> v2:172.21.15.40:3300/0 conn(0x7f066015bdf0 0x7f06600b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7fc3b2ffd640 1 -- 172.21.15.40:0/2727855676 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c037070 con 0x7fc3b40cd3e0 2024-04-26T21:35:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7fc3b2ffd640 1 -- 172.21.15.40:0/2727855676 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39c032440 con 0x7fc3b40cd3e0 2024-04-26T21:35:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab27fc640 1 -- 172.21.15.40:0/4081779519 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696685256 0 0) 0x7f9ab40d3db0 con 0x7f9ab4167470 2024-04-26T21:35:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab27fc640 1 -- 172.21.15.40:0/4081779519 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a9c003520 con 0x7f9ab4167470 2024-04-26T21:35:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab2ffd640 1 -- 172.21.15.40:0/4081779519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 msgr2=0x7f9ab40d3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab2ffd640 1 --2- 172.21.15.40:0/4081779519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab40d3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab2ffd640 1 -- 172.21.15.40:0/4081779519 >> v1:172.21.15.184:6789/0 conn(0x7f9ab4167470 legacy=0x7f9ab4167920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f9ab2ffd640 1 -- 172.21.15.40:0/4081779519 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab40d4090 con 0x7f9ab40d5b20 2024-04-26T21:35:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f4959727640 1 Processor -- start 2024-04-26T21:35:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f4959727640 1 -- start start 2024-04-26T21:35:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f686d028640 1 -- 172.21.15.40:0/3532953989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 msgr2=0x7f68680d3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f686d028640 1 --2- 172.21.15.40:0/3532953989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 0x7f68680d3810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f68680b79a0 tx=0x7f685c02a110 comp rx=0 tx=0).stop 2024-04-26T21:35:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f686d028640 1 -- 172.21.15.40:0/3532953989 shutdown_connections 2024-04-26T21:35:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f686d028640 1 --2- 172.21.15.40:0/3532953989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 0x7f68680d3810 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f686d028640 1 --2- 172.21.15.40:0/3532953989 >> v2:172.21.15.40:3300/0 conn(0x7f686815f190 0x7f68680d32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7f686d028640 1 -- 172.21.15.40:0/3532953989 >> 172.21.15.40:0/3532953989 conn(0x7f68680d4ea0 msgr2=0x7f68680d52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f686d028640 1 -- 172.21.15.40:0/3532953989 shutdown_connections 2024-04-26T21:35:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f686d028640 1 -- 172.21.15.40:0/3532953989 wait complete. 2024-04-26T21:35:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7fc3b2ffd640 1 -- 172.21.15.40:0/2727855676 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c032810 con 0x7fc3b40cd3e0 2024-04-26T21:35:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7fd019763640 1 -- 172.21.15.40:0/4269881498 reap_dead start 2024-04-26T21:35:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.128+0000 7fd019763640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0040051c0 0x7fd00400b9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd00c038990 tx=0x7fd004005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b52a69629232a4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008037070 con 0x7fd0040051c0 2024-04-26T21:35:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd008032410 con 0x7fd0040051c0 2024-04-26T21:35:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7fd0137fe640 1 -- 172.21.15.40:0/4269881498 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008007d70 con 0x7fd0040051c0 2024-04-26T21:35:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7fc3b9c07640 1 -- 172.21.15.40:0/2727855676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 msgr2=0x7fc3b40cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f9ab37fe640 1 --2- 172.21.15.40:0/4081779519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab40d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f4959727640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f49540c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f4959727640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49540c6ee0 0x7f495415c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f4959727640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49540c7510 con 0x7f4954072dd0 2024-04-26T21:35:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f4959727640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f495415ffb0 con 0x7f49540c6ee0 2024-04-26T21:35:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f4959727640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f495415f960 con 0x7f4954160440 2024-04-26T21:35:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/2727855676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b40cf970 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0097a0 tx=0x7fc39c02a180 comp rx=0 tx=0).stop 2024-04-26T21:35:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f1cf08ea640 1 --2- 172.21.15.40:0/223432014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce40ebcc0 0x7f1ce4053520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1cec06b190 tx=0x7f1cec0a6c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b717ed622bdc84 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fc3b9c07640 1 -- 172.21.15.40:0/2727855676 shutdown_connections 2024-04-26T21:35:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f065ffff640 1 --2- 172.21.15.40:0/1068256939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06600727a0 0x7f066015b8b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f06600b3770 tx=0x7f065402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78869334343c2e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/2727855676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b40cf970 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f9ab2ffd640 1 --2- 172.21.15.40:0/4081779519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab40d5b20 0x7f9ab40b62d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009d90 tx=0x7f9aac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f247189c6a28fd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/2727855676 >> v2:172.21.15.40:3300/0 conn(0x7fc3b40c45b0 0x7fc3b40cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fc3b9c07640 1 -- 172.21.15.40:0/2727855676 >> 172.21.15.40:0/2727855676 conn(0x7fc3b4073960 msgr2=0x7fc3b4073d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7fadf836a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fadf0162880 0x7fadf00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf6106640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fadf0162880 0x7fadf00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf6106640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fadf0162880 0x7fadf00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45654/0 (socket says 172.21.15.40:45654) 2024-04-26T21:35:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf6106640 1 -- 172.21.15.40:0/1993548060 learned_addr learned my addr 172.21.15.40:0/1993548060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf836a640 1 -- 172.21.15.40:0/1993548060 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf00b7e50 con 0x7fadf00b5f20 2024-04-26T21:35:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf836a640 1 -- 172.21.15.40:0/1993548060 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf00d1800 con 0x7fadf00d3bc0 2024-04-26T21:35:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf836a640 1 -- 172.21.15.40:0/1993548060 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fadf00d0580 con 0x7fadf0162880 2024-04-26T21:35:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fd019f64640 1 -- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0040051c0 msgr2=0x7fd00400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fd019f64640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0040051c0 0x7fd00400b9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd00c038990 tx=0x7fd004005830 comp rx=0 tx=0).stop 2024-04-26T21:35:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f1ceaffd640 1 -- 172.21.15.40:0/223432014 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0b4070 con 0x7f1ce40ebcc0 2024-04-26T21:35:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f1ceaffd640 1 -- 172.21.15.40:0/223432014 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec0a7a90 con 0x7f1ce40ebcc0 2024-04-26T21:35:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f1ceaffd640 1 -- 172.21.15.40:0/223432014 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0af440 con 0x7f1ce40ebcc0 2024-04-26T21:35:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fc3b9c07640 1 -- 172.21.15.40:0/2727855676 shutdown_connections 2024-04-26T21:35:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fd019f64640 1 -- 172.21.15.40:0/4269881498 shutdown_connections 2024-04-26T21:35:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fd019f64640 1 --2- 172.21.15.40:0/4269881498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0040051c0 0x7fd00400b9c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fd019f64640 1 -- 172.21.15.40:0/4269881498 >> 172.21.15.40:0/4269881498 conn(0x7fd00c0559e0 msgr2=0x7fd00c055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f49537fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f49540c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f49537fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f49540c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53726/0 (socket says 172.21.15.40:53726) 2024-04-26T21:35:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.129+0000 7f49537fe640 1 -- 172.21.15.40:0/2549542614 learned_addr learned my addr 172.21.15.40:0/2549542614 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f4958f26640 1 --2- 172.21.15.40:0/2549542614 >> v2:172.21.15.40:3300/0 conn(0x7f49540c6ee0 0x7f495415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fd019f64640 1 -- 172.21.15.40:0/4269881498 shutdown_connections 2024-04-26T21:35:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f065f7fe640 1 -- 172.21.15.40:0/1068256939 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654037070 con 0x7f06600727a0 2024-04-26T21:35:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fd019f64640 1 -- 172.21.15.40:0/4269881498 wait complete. 2024-04-26T21:35:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f9ab27fc640 1 -- 172.21.15.40:0/4081779519 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac007740 con 0x7f9ab40d5b20 2024-04-26T21:35:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f065f7fe640 1 -- 172.21.15.40:0/1068256939 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0654032440 con 0x7f06600727a0 2024-04-26T21:35:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f065f7fe640 1 -- 172.21.15.40:0/1068256939 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654032810 con 0x7f06600727a0 2024-04-26T21:35:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf6106640 1 -- 172.21.15.40:0/1993548060 >> v1:172.21.15.184:6789/0 conn(0x7fadf00d3bc0 legacy=0x7fadf00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf6106640 1 -- 172.21.15.40:0/1993548060 >> v1:172.21.15.40:6789/0 conn(0x7fadf00b5f20 legacy=0x7fadf00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f9ab27fc640 1 -- 172.21.15.40:0/4081779519 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aac0078e0 con 0x7f9ab40d5b20 2024-04-26T21:35:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fadf6106640 1 -- 172.21.15.40:0/1993548060 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf00d07d0 con 0x7fadf0162880 2024-04-26T21:35:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7fc3b9c07640 1 -- 172.21.15.40:0/2727855676 wait complete. 2024-04-26T21:35:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f686d028640 1 Processor -- start 2024-04-26T21:35:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f1cf234d640 1 -- 172.21.15.40:0/223432014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce40ebcc0 msgr2=0x7f1ce4053520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f1cf234d640 1 --2- 172.21.15.40:0/223432014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce40ebcc0 0x7f1ce4053520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1cec06b190 tx=0x7f1cec0a6c80 comp rx=0 tx=0).stop 2024-04-26T21:35:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f686d028640 1 -- start start 2024-04-26T21:35:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f1cf234d640 1 -- 172.21.15.40:0/223432014 shutdown_connections 2024-04-26T21:35:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f1cf234d640 1 --2- 172.21.15.40:0/223432014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce40ebcc0 0x7f1ce4053520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f1cf234d640 1 --2- 172.21.15.40:0/223432014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40eb300 0x7f1ce40eb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f1cf234d640 1 --2- 172.21.15.40:0/223432014 >> v2:172.21.15.40:3300/0 conn(0x7f1ce40e7820 0x7f1ce4038280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f1cf234d640 1 -- 172.21.15.40:0/223432014 >> 172.21.15.40:0/223432014 conn(0x7f1ce40559e0 msgr2=0x7f1ce4055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f4958f26640 1 -- 172.21.15.40:0/2549542614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 msgr2=0x7f49540c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f9ab27fc640 1 -- 172.21.15.40:0/4081779519 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac007cf0 con 0x7f9ab40d5b20 2024-04-26T21:35:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.131+0000 7f9ab9db4640 1 -- 172.21.15.40:0/4081779519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab40d5b20 msgr2=0x7f9ab40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f9ab9db4640 1 --2- 172.21.15.40:0/4081779519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab40d5b20 0x7f9ab40b62d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009d90 tx=0x7f9aac02a680 comp rx=0 tx=0).stop 2024-04-26T21:35:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7fadf6106640 1 --2- 172.21.15.40:0/1993548060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf0162880 0x7fadf00d2cb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1a646c61065468 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7fc3b9c07640 1 Processor -- start 2024-04-26T21:35:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f4958f26640 1 --2- 172.21.15.40:0/2549542614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f49540c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f4958f26640 1 -- 172.21.15.40:0/2549542614 >> v1:172.21.15.184:6789/0 conn(0x7f4954072dd0 legacy=0x7f49540731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.130+0000 7f4958f26640 1 -- 172.21.15.40:0/2549542614 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495415fc40 con 0x7f49540c6ee0 2024-04-26T21:35:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f4958f26640 1 --2- 172.21.15.40:0/2549542614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49540c6ee0 0x7f495415c200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f494c00f370 tx=0x7f494c02e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb93266c05363368 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f1cf234d640 1 -- 172.21.15.40:0/223432014 shutdown_connections 2024-04-26T21:35:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7fd019f64640 1 Processor -- start 2024-04-26T21:35:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7fc3b9c07640 1 -- start start 2024-04-26T21:35:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f9ab9db4640 1 -- 172.21.15.40:0/4081779519 shutdown_connections 2024-04-26T21:35:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f9ab9db4640 1 --2- 172.21.15.40:0/4081779519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab40d5b20 0x7f9ab40b62d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f9ab9db4640 1 --2- 172.21.15.40:0/4081779519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab40d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f9ab9db4640 1 -- 172.21.15.40:0/4081779519 >> 172.21.15.40:0/4081779519 conn(0x7f9ab40d4e70 msgr2=0x7f9ab40d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 -- 172.21.15.40:0/1068256939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06600727a0 msgr2=0x7f066015b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7fd019f64640 1 -- start start 2024-04-26T21:35:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 --2- 172.21.15.40:0/1068256939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06600727a0 0x7f066015b8b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f06600b3770 tx=0x7f065402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 -- 172.21.15.40:0/1068256939 shutdown_connections 2024-04-26T21:35:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 --2- 172.21.15.40:0/1068256939 >> v2:172.21.15.40:3300/0 conn(0x7f066015bdf0 0x7f06600b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 --2- 172.21.15.40:0/1068256939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06600727a0 0x7f066015b8b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 --2- 172.21.15.40:0/1068256939 >> v2:172.21.15.184:3300/0 conn(0x7f066015b4d0 0x7f066006ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 -- 172.21.15.40:0/1068256939 >> 172.21.15.40:0/1068256939 conn(0x7f06600b2770 msgr2=0x7f06600b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 -- 172.21.15.40:0/1068256939 shutdown_connections 2024-04-26T21:35:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f0665c66640 1 -- 172.21.15.40:0/1068256939 wait complete. 2024-04-26T21:35:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f686d028640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68680d59c0 0x7f68681f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f686d028640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686815f190 0x7f68680cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7fadf5104640 1 -- 172.21.15.40:0/1993548060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec037070 con 0x7fadf0162880 2024-04-26T21:35:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7fadf5104640 1 -- 172.21.15.40:0/1993548060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadec032440 con 0x7fadf0162880 2024-04-26T21:35:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7fadf5104640 1 -- 172.21.15.40:0/1993548060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec032810 con 0x7fadf0162880 2024-04-26T21:35:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f686d028640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 0x7f6868151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f1cf234d640 1 -- 172.21.15.40:0/223432014 wait complete. 2024-04-26T21:35:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f686659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686815f190 0x7f68680cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f686659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686815f190 0x7f68680cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45660/0 (socket says 172.21.15.40:45660) 2024-04-26T21:35:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fadf836a640 1 -- 172.21.15.40:0/1993548060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf0162880 msgr2=0x7fadf00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fadf836a640 1 --2- 172.21.15.40:0/1993548060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf0162880 0x7fadf00d2cb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02a270 comp rx=0 tx=0).stop 2024-04-26T21:35:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f4952ffd640 1 -- 172.21.15.40:0/2549542614 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c00d2e0 con 0x7f49540c6ee0 2024-04-26T21:35:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.132+0000 7f4952ffd640 1 -- 172.21.15.40:0/2549542614 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c039070 con 0x7f49540c6ee0 2024-04-26T21:35:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f4959727640 1 -- 172.21.15.40:0/2549542614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49540c6ee0 msgr2=0x7f495415c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f4959727640 1 --2- 172.21.15.40:0/2549542614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49540c6ee0 0x7f495415c200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f494c00f370 tx=0x7f494c02e980 comp rx=0 tx=0).stop 2024-04-26T21:35:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f686659c640 1 -- 172.21.15.40:0/2202312288 learned_addr learned my addr 172.21.15.40:0/2202312288 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f68680b35d0 con 0x7f686815f190 2024-04-26T21:35:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fadf836a640 1 -- 172.21.15.40:0/1993548060 shutdown_connections 2024-04-26T21:35:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fadf836a640 1 --2- 172.21.15.40:0/1993548060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf0162880 0x7fadf00d2cb0 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02a270 comp rx=0 tx=0).stop 2024-04-26T21:35:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fadf836a640 1 -- 172.21.15.40:0/1993548060 >> 172.21.15.40:0/1993548060 conn(0x7fadf00d4f60 msgr2=0x7fadf00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fd019f64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040051c0 0x7fd00c0e0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fd019f64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00c0dced0 0x7fd00c052790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fd019f64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c00a860 0x7fd00c0dda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fd019f64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c0e34a0 con 0x7fd00c0dced0 2024-04-26T21:35:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fd019f64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c03bfc0 con 0x7fd00c00a860 2024-04-26T21:35:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7fd019f64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd00c057d90 con 0x7fd0040051c0 2024-04-26T21:35:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f6866d9d640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68680d59c0 0x7f68681f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b9c07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c45b0 0x7fc3b40c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd013fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00c0dced0 0x7fd00c052790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f686759e640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 0x7f6868151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b9c07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b41666e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd013fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00c0dced0 0x7fd00c052790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45670/0 (socket says 172.21.15.40:45670) 2024-04-26T21:35:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd013fff640 1 -- 172.21.15.40:0/1544561686 learned_addr learned my addr 172.21.15.40:0/1544561686 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd018f62640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040051c0 0x7fd00c0e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd019763640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c00a860 0x7fd00c0dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f4959727640 1 -- 172.21.15.40:0/2549542614 shutdown_connections 2024-04-26T21:35:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f4959727640 1 --2- 172.21.15.40:0/2549542614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49540c6ee0 0x7f495415c200 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f4959727640 1 --2- 172.21.15.40:0/2549542614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f49540c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fadf836a640 1 -- 172.21.15.40:0/1993548060 shutdown_connections 2024-04-26T21:35:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f4959727640 1 -- 172.21.15.40:0/2549542614 >> 172.21.15.40:0/2549542614 conn(0x7f49540d1ae0 msgr2=0x7f49540d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b9c07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b40ce130 0x7fc3b40cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fadf836a640 1 -- 172.21.15.40:0/1993548060 wait complete. 2024-04-26T21:35:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c45b0 0x7fc3b40c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c45b0 0x7fc3b40c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45678/0 (socket says 172.21.15.40:45678) 2024-04-26T21:35:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68680b3450 con 0x7f686815f570 2024-04-26T21:35:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4074690 con 0x7fc3b40c45b0 2024-04-26T21:35:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f68680b3750 con 0x7f68680d59c0 2024-04-26T21:35:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f1cf234d640 1 Processor -- start 2024-04-26T21:35:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b37fe640 1 -- 172.21.15.40:0/1417894052 learned_addr learned my addr 172.21.15.40:0/1417894052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f0665c66640 1 Processor -- start 2024-04-26T21:35:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.133+0000 7f9ab9db4640 1 -- 172.21.15.40:0/4081779519 shutdown_connections 2024-04-26T21:35:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f9ab9db4640 1 -- 172.21.15.40:0/4081779519 wait complete. 2024-04-26T21:35:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd019763640 1 -- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040051c0 msgr2=0x7fd00c0e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd019763640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040051c0 0x7fd00c0e0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd019763640 1 -- 172.21.15.40:0/1544561686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00c0dced0 msgr2=0x7fd00c052790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd019763640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00c0dced0 0x7fd00c052790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd019763640 1 -- 172.21.15.40:0/1544561686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c0500d0 con 0x7fd00c00a860 2024-04-26T21:35:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fd013fff640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00c0dced0 0x7fd00c052790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b3fff640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b40ce130 0x7fc3b40cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7f0665c66640 1 -- start start 2024-04-26T21:35:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3aaffd640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b41666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.134+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4074510 con 0x7fc3b40ce130 2024-04-26T21:35:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b40b1060 con 0x7fc3b40cd3e0 2024-04-26T21:35:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f1cf234d640 1 -- start start 2024-04-26T21:35:35.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f4959727640 1 -- 172.21.15.40:0/2549542614 shutdown_connections 2024-04-26T21:35:35.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fd019763640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c00a860 0x7fd00c0dda40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd00c011e20 tx=0x7fd008001a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f6866d9d640 1 -- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 msgr2=0x7f6868151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f6866d9d640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 0x7f6868151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f6866d9d640 1 -- 172.21.15.40:0/2202312288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686815f190 msgr2=0x7f68680cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f6866d9d640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686815f190 0x7f68680cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f6866d9d640 1 -- 172.21.15.40:0/2202312288 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68681f59a0 con 0x7f68680d59c0 2024-04-26T21:35:35.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fd0117fa640 1 -- 172.21.15.40:0/1544561686 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008040020 con 0x7fd00c00a860 2024-04-26T21:35:35.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fd0117fa640 1 -- 172.21.15.40:0/1544561686 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd008005070 con 0x7fd00c00a860 2024-04-26T21:35:35.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f6866d9d640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68680d59c0 0x7f68681f5460 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f685800c7f0 tx=0x7f685800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7f4959727640 1 -- 172.21.15.40:0/2549542614 wait complete. 2024-04-26T21:35:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fc3b3fff640 1 -- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 msgr2=0x7fc3b41666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fc3b3fff640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b41666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fc3b3fff640 1 -- 172.21.15.40:0/1417894052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c45b0 msgr2=0x7fc3b40c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fc3b3fff640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c45b0 0x7fc3b40c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.135+0000 7fc3b3fff640 1 -- 172.21.15.40:0/1417894052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b40c5a60 con 0x7fc3b40ce130 2024-04-26T21:35:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7fd0117fa640 1 -- 172.21.15.40:0/1544561686 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008037040 con 0x7fd00c00a860 2024-04-26T21:35:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7fd019f64640 1 -- 172.21.15.40:0/1544561686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c0503b0 con 0x7fd00c00a860 2024-04-26T21:35:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7fd019f64640 1 -- 172.21.15.40:0/1544561686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c050970 con 0x7fd00c00a860 2024-04-26T21:35:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665c66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600727a0 0x7f0660159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665c66640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066015b4d0 0x7f0660139d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665c66640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066015bdf0 0x7f0660139360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7fc3b3fff640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b40ce130 0x7fc3b40cab50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00c3c0 tx=0x7fc3ac001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665c66640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f066015c200 con 0x7f06600727a0 2024-04-26T21:35:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665465640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066015bdf0 0x7f0660139360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665465640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066015bdf0 0x7f0660139360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53752/0 (socket says 172.21.15.40:53752) 2024-04-26T21:35:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665465640 1 -- 172.21.15.40:0/17281818 learned_addr learned my addr 172.21.15.40:0/17281818 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665c66640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06600b4760 con 0x7f066015b4d0 2024-04-26T21:35:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0665c66640 1 -- 172.21.15.40:0/17281818 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0660072c20 con 0x7f066015bdf0 2024-04-26T21:35:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f0664c64640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600727a0 0x7f0660159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f9ab9db4640 1 Processor -- start 2024-04-26T21:35:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f9ab9db4640 1 -- start start 2024-04-26T21:35:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f1cf234d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40e7820 0x7f1ce40daef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f684f7fe640 1 -- 172.21.15.40:0/2202312288 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858015070 con 0x7f68680d59c0 2024-04-26T21:35:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f684f7fe640 1 -- 172.21.15.40:0/2202312288 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6858004220 con 0x7f68680d59c0 2024-04-26T21:35:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686814ece0 con 0x7f68680d59c0 2024-04-26T21:35:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.136+0000 7f684f7fe640 1 -- 172.21.15.40:0/2202312288 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858007620 con 0x7f68680d59c0 2024-04-26T21:35:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7fc3b0ff9640 1 -- 172.21.15.40:0/1417894052 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac017070 con 0x7fc3b40ce130 2024-04-26T21:35:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7fc3b0ff9640 1 -- 172.21.15.40:0/1417894052 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ac0052f0 con 0x7fc3b40ce130 2024-04-26T21:35:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7fc3b0ff9640 1 -- 172.21.15.40:0/1417894052 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac003830 con 0x7fc3b40ce130 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f9ab9db4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab4159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f9ab9db4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab40d5b20 0x7f9ab41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f9ab9db4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab4167470 0x7f9ab41574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab9db4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab40d2340 con 0x7f9ab4167470 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab9db4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4168030 con 0x7f9ab40d5b20 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab9db4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab40c9d30 con 0x7f9ab40d32a0 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab2ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab40d5b20 0x7f9ab41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab2ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab40d5b20 0x7f9ab41567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49750/0 (socket says 172.21.15.40:49750) 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab2ffd640 1 -- 172.21.15.40:0/1204011326 learned_addr learned my addr 172.21.15.40:0/1204011326 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab3fff640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab4167470 0x7f9ab41574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab2ffd640 1 -- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 msgr2=0x7f9ab4159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab37fe640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f4959727640 1 Processor -- start 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40e7820 0x7f1ce40daef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f1cf234d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce40eb300 0x7f1ce4039930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f1cf234d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40ebcc0 0x7f1ce4039e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f1cf234d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4058ce0 con 0x7f1ce40e7820 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40e7820 0x7f1ce40daef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45700/0 (socket says 172.21.15.40:45700) 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f1cebfff640 1 -- 172.21.15.40:0/320251166 learned_addr learned my addr 172.21.15.40:0/320251166 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce40ec1a0 con 0x7f1ce40eb300 2024-04-26T21:35:35.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce40546f0 con 0x7f1ce40ebcc0 2024-04-26T21:35:35.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1cf08ea640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40ebcc0 0x7f1ce4039e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1cebfff640 1 -- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40ebcc0 msgr2=0x7f1ce4039e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1cebfff640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40ebcc0 0x7f1ce4039e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1cebfff640 1 -- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce40eb300 msgr2=0x7f1ce4039930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:35:35.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1cebfff640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce40eb300 0x7f1ce4039930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1cebfff640 1 -- 172.21.15.40:0/320251166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce40dc340 con 0x7f1ce40e7820 2024-04-26T21:35:35.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f686814e3b0 con 0x7f68680d59c0 2024-04-26T21:35:35.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1cebfff640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40e7820 0x7f1ce40daef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000e960 tx=0x7f1ce000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7fadf836a640 1 Processor -- start 2024-04-26T21:35:35.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f684d7fa640 1 -- 172.21.15.40:0/2202312288 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6840005290 con 0x7f68680d59c0 2024-04-26T21:35:35.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab2ffd640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab4159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab2ffd640 1 -- 172.21.15.40:0/1204011326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab4167470 msgr2=0x7f9ab41574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab2ffd640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab4167470 0x7f9ab41574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab2ffd640 1 -- 172.21.15.40:0/1204011326 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab40cb130 con 0x7f9ab40d5b20 2024-04-26T21:35:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f9ab3fff640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab4167470 0x7f9ab41574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f0664c64640 1 -- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066015bdf0 msgr2=0x7f0660139360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f0664c64640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066015bdf0 0x7f0660139360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f0664c64640 1 -- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066015b4d0 msgr2=0x7f0660139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f0664c64640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066015b4d0 0x7f0660139d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f0664c64640 1 -- 172.21.15.40:0/17281818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066015a190 con 0x7f06600727a0 2024-04-26T21:35:35.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7f0664c64640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600727a0 0x7f0660159a90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f065800ebc0 tx=0x7f065800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f065d7fa640 1 -- 172.21.15.40:0/17281818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658018020 con 0x7f06600727a0 2024-04-26T21:35:35.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f065d7fa640 1 -- 172.21.15.40:0/17281818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065800cda0 con 0x7f06600727a0 2024-04-26T21:35:35.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f065d7fa640 1 -- 172.21.15.40:0/17281818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06580146c0 con 0x7f06600727a0 2024-04-26T21:35:35.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f0665c66640 1 -- 172.21.15.40:0/17281818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066015a470 con 0x7f06600727a0 2024-04-26T21:35:35.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f0665c66640 1 -- 172.21.15.40:0/17281818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066015aa30 con 0x7f06600727a0 2024-04-26T21:35:35.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7fd0117fa640 1 -- 172.21.15.40:0/1544561686 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd008043020 con 0x7fd00c00a860 2024-04-26T21:35:35.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7f0665465640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066015bdf0 0x7f0660139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:35.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.137+0000 7fadf836a640 1 -- start start 2024-04-26T21:35:35.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b40c9210 con 0x7fc3b40ce130 2024-04-26T21:35:35.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1ce8ff9640 1 -- 172.21.15.40:0/320251166 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0017070 con 0x7f1ce40e7820 2024-04-26T21:35:35.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1ce8ff9640 1 -- 172.21.15.40:0/320251166 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce000ca00 con 0x7f1ce40e7820 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7f1ce8ff9640 1 -- 172.21.15.40:0/320251166 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000cdd0 con 0x7f1ce40e7820 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce4050140 con 0x7f1ce40e7820 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7fadf836a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf00b5f20 0x7fadf0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.138+0000 7fadf836a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadf00d3bc0 0x7fadf0155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf6106640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf00b5f20 0x7fadf0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf836a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadf01f8580 0x7fadf015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf836a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0162c60 con 0x7fadf00b5f20 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf6106640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf00b5f20 0x7fadf0155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45720/0 (socket says 172.21.15.40:45720) 2024-04-26T21:35:35.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf6106640 1 -- 172.21.15.40:0/811718643 learned_addr learned my addr 172.21.15.40:0/811718643 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b40c5d00 con 0x7fc3b40ce130 2024-04-26T21:35:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fd0117fa640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcff806c9c0 0x7fcff806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7f4959727640 1 -- start start 2024-04-26T21:35:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fd0117fa640 1 -- 172.21.15.40:0/1544561686 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd0080ab740 con 0x7fd00c00a860 2024-04-26T21:35:35.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7f9ab37fe640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab4159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:35.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce40506c0 con 0x7f1ce40e7820 2024-04-26T21:35:35.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fd019f64640 1 -- 172.21.15.40:0/1544561686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00c0057e0 con 0x7fd00c00a860 2024-04-26T21:35:35.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7f9ab2ffd640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab40d5b20 0x7f9ab41567d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009d90 tx=0x7f9aac004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7f9a9bfff640 1 -- 172.21.15.40:0/1204011326 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac040070 con 0x7f9ab40d5b20 2024-04-26T21:35:35.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7f684f7fe640 1 -- 172.21.15.40:0/2202312288 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6858007a50 con 0x7f68680d59c0 2024-04-26T21:35:35.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fd018f62640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcff806c9c0 0x7fcff806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf6907640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadf01f8580 0x7fadf015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7f0646ffd640 1 -- 172.21.15.40:0/17281818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0638005290 con 0x7f06600727a0 2024-04-26T21:35:35.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7f4959727640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4954072dd0 0x7f4954159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f4959727640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49540c6ee0 0x7f49540c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fadf00d0d90 con 0x7fadf01f8580 2024-04-26T21:35:35.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f4959727640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f495413dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fadf0162de0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f49537fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49540c6ee0 0x7f49540c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f49537fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49540c6ee0 0x7f49540c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49770/0 (socket says 172.21.15.40:49770) 2024-04-26T21:35:35.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.139+0000 7fadf5905640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadf00d3bc0 0x7fadf0155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fadf5905640 1 -- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadf01f8580 msgr2=0x7fadf015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f49537fe640 1 -- 172.21.15.40:0/2434971471 learned_addr learned my addr 172.21.15.40:0/2434971471 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:35.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f4958f26640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f495413dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7fc3b0ff9640 1 -- 172.21.15.40:0/1417894052 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3ac01a070 con 0x7fc3b40ce130 2024-04-26T21:35:35.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f495415c740 con 0x7f4954072dd0 2024-04-26T21:35:35.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f49540c7f20 con 0x7f49540c6ee0 2024-04-26T21:35:35.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f4953fff640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4954072dd0 0x7f4954159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f495415c8c0 con 0x7f4954160440 2024-04-26T21:35:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fadf5905640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadf01f8580 0x7fadf015aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fadf5905640 1 -- 172.21.15.40:0/811718643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf00b5f20 msgr2=0x7fadf0155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fadf5905640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf00b5f20 0x7fadf0155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fadf5905640 1 -- 172.21.15.40:0/811718643 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf01f8fd0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fadf6106640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf00b5f20 0x7fadf0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fadf6907640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadf01f8580 0x7fadf015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7fd018f62640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcff806c9c0 0x7fcff806ee80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd014051930 tx=0x7fd014098040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7fadf5905640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadf00d3bc0 0x7fadf0155490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fade000c440 tx=0x7fade000c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7f9ab9db4640 1 -- 172.21.15.40:0/1204011326 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab4153f70 con 0x7f9ab40d5b20 2024-04-26T21:35:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.140+0000 7f9ab9db4640 1 -- 172.21.15.40:0/1204011326 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab4154530 con 0x7f9ab40d5b20 2024-04-26T21:35:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.141+0000 7f9a9bfff640 1 -- 172.21.15.40:0/1204011326 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aac031bc0 con 0x7f9ab40d5b20 2024-04-26T21:35:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7fc3b0ff9640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc39406c9c0 0x7fc39406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f49537fe640 1 -- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 msgr2=0x7f495413dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f49537fe640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f495413dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f49537fe640 1 -- 172.21.15.40:0/2434971471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4954072dd0 msgr2=0x7f4954159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f49537fe640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4954072dd0 0x7f4954159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f49537fe640 1 -- 172.21.15.40:0/2434971471 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495413e510 con 0x7f49540c6ee0 2024-04-26T21:35:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f4953fff640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4954072dd0 0x7f4954159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:35.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7fade6ffd640 1 -- 172.21.15.40:0/811718643 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0017070 con 0x7fadf00d3bc0 2024-04-26T21:35:35.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7fc3b0ff9640 1 -- 172.21.15.40:0/1417894052 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc3ac08b040 con 0x7fc3b40ce130 2024-04-26T21:35:35.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7fade6ffd640 1 -- 172.21.15.40:0/811718643 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade00035f0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f49537fe640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49540c6ee0 0x7f49540c9f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f494800d9c0 tx=0x7f494800de80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f9a99ffb640 1 -- 172.21.15.40:0/1204011326 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab4069a80 con 0x7f9ab40d5b20 2024-04-26T21:35:35.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7fc3b37fe640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc39406c9c0 0x7fc39406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f9a9bfff640 1 -- 172.21.15.40:0/1204011326 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac045460 con 0x7f9ab40d5b20 2024-04-26T21:35:35.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7fade6ffd640 1 -- 172.21.15.40:0/811718643 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade00039e0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf01f92b0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf01599c0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7f065d7fa640 1 -- 172.21.15.40:0/17281818 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0658005d10 con 0x7f06600727a0 2024-04-26T21:35:35.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.142+0000 7f1ce8ff9640 1 -- 172.21.15.40:0/320251166 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ce001a070 con 0x7f1ce40e7820 2024-04-26T21:35:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7f684f7fe640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c9c0 0x7f685406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7f684f7fe640 1 -- 172.21.15.40:0/2202312288 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f685808be60 con 0x7f68680d59c0 2024-04-26T21:35:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7f1ce8ff9640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cd006c8f0 0x7f1cd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7f1ce8ff9640 1 -- 172.21.15.40:0/320251166 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1ce008bde0 con 0x7f1ce40e7820 2024-04-26T21:35:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce40057e0 con 0x7f1ce40e7820 2024-04-26T21:35:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b4069a80 con 0x7fc3b40ce130 2024-04-26T21:35:35.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7f065d7fa640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f064c06c9c0 0x7f064c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.143+0000 7f686659c640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c9c0 0x7f685406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.144+0000 7f1ceb7fe640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cd006c8f0 0x7f1cd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.144+0000 7f065d7fa640 1 -- 172.21.15.40:0/17281818 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f065808d160 con 0x7f06600727a0 2024-04-26T21:35:35.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.144+0000 7f065ffff640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f064c06c9c0 0x7f064c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.144+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf0159bc0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.144+0000 7fc3b37fe640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc39406c9c0 0x7fc39406ee80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40093e0 tx=0x7fc3a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.144+0000 7f065ffff640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f064c06c9c0 0x7f064c06ee80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0654034000 tx=0x7f065402c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.144+0000 7f686659c640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c9c0 0x7f685406ee80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f68680ba140 tx=0x7f685c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.144+0000 7f1ceb7fe640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cd006c8f0 0x7f1cd006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40e6b50 tx=0x7f1cd8009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.145+0000 7fd0117fa640 1 -- 172.21.15.40:0/1544561686 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd00804a030 con 0x7fd00c00a860 2024-04-26T21:35:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.146+0000 7f4950ff9640 1 -- 172.21.15.40:0/2434971471 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494800abe0 con 0x7f49540c6ee0 2024-04-26T21:35:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.146+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495413e790 con 0x7f49540c6ee0 2024-04-26T21:35:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.146+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495413ed50 con 0x7f49540c6ee0 2024-04-26T21:35:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.147+0000 7f9a9bfff640 1 -- 172.21.15.40:0/1204011326 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9aac039040 con 0x7f9ab40d5b20 2024-04-26T21:35:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.147+0000 7f684f7fe640 1 -- 172.21.15.40:0/2202312288 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68681f59a0 con 0x7f68680d59c0 2024-04-26T21:35:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.147+0000 7f9a9bfff640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9aa006c940 0x7f9aa006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.148+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4954069a80 con 0x7f49540c6ee0 2024-04-26T21:35:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.148+0000 7f9ab37fe640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9aa006c940 0x7f9aa006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.148+0000 7fade6ffd640 1 -- 172.21.15.40:0/811718643 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fade001a070 con 0x7fadf00d3bc0 2024-04-26T21:35:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.148+0000 7f065d7fa640 1 -- 172.21.15.40:0/17281818 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0658022030 con 0x7f06600727a0 2024-04-26T21:35:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.148+0000 7fade6ffd640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fadd406c970 0x7fadd406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.149+0000 7f9ab37fe640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9aa006c940 0x7f9aa006ee00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009ac0 tx=0x7f9aa8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.149+0000 7fadf6106640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fadd406c970 0x7fadd406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.149+0000 7fade6ffd640 1 -- 172.21.15.40:0/811718643 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fade008b6b0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.149+0000 7fadf6106640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fadd406c970 0x7fadd406ee30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fadec009630 tx=0x7fadec02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.150+0000 7f9a9bfff640 1 -- 172.21.15.40:0/1204011326 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9aac0797f0 con 0x7f9ab40d5b20 2024-04-26T21:35:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.150+0000 7f4950ff9640 1 -- 172.21.15.40:0/2434971471 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494800ad80 con 0x7f49540c6ee0 2024-04-26T21:35:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.150+0000 7f4950ff9640 1 -- 172.21.15.40:0/2434971471 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494800abe0 con 0x7f49540c6ee0 2024-04-26T21:35:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.151+0000 7fc3b0ff9640 1 -- 172.21.15.40:0/1417894052 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3ac058b50 con 0x7fc3b40ce130 2024-04-26T21:35:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.151+0000 7fade6ffd640 1 -- 172.21.15.40:0/811718643 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fadf0159bc0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.155+0000 7f9a9bfff640 1 -- 172.21.15.40:0/1204011326 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9aac054080 con 0x7f9ab40d5b20 2024-04-26T21:35:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.155+0000 7f1ce8ff9640 1 -- 172.21.15.40:0/320251166 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ce0059e30 con 0x7f1ce40e7820 2024-04-26T21:35:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.157+0000 7f4950ff9640 1 -- 172.21.15.40:0/2434971471 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4948019400 con 0x7f49540c6ee0 2024-04-26T21:35:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.157+0000 7f4950ff9640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f494006caf0 0x7f494006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:35.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.158+0000 7f4953fff640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f494006caf0 0x7f494006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:35.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.158+0000 7f4953fff640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f494006caf0 0x7f494006efb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4944003f00 tx=0x7f4944009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:35.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.158+0000 7f4950ff9640 1 -- 172.21.15.40:0/2434971471 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f49480580b0 con 0x7f49540c6ee0 2024-04-26T21:35:35.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.160+0000 7f4950ff9640 1 -- 172.21.15.40:0/2434971471 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4948027020 con 0x7f49540c6ee0 2024-04-26T21:35:35.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.576+0000 7f684d7fa640 1 -- 172.21.15.40:0/2202312288 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6840005580 con 0x7f68680d59c0 2024-04-26T21:35:35.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.577+0000 7f684f7fe640 1 -- 172.21.15.40:0/2202312288 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f68580598e0 con 0x7f68680d59c0 2024-04-26T21:35:35.579 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2024-04-26T21:35:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c9c0 msgr2=0x7f685406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c9c0 0x7f685406ee80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f68680ba140 tx=0x7f685c02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68680d59c0 msgr2=0x7f68681f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68680d59c0 0x7f68681f5460 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f685800c7f0 tx=0x7f685800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 shutdown_connections 2024-04-26T21:35:35.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c9c0 0x7f685406ee80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f686815f570 0x7f6868151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686815f190 0x7f68680cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 --2- 172.21.15.40:0/2202312288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68680d59c0 0x7f68681f5460 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 >> 172.21.15.40:0/2202312288 conn(0x7f68680d4ea0 msgr2=0x7f686806e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 shutdown_connections 2024-04-26T21:35:35.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.581+0000 7f686d028640 1 -- 172.21.15.40:0/2202312288 wait complete. 2024-04-26T21:35:35.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.583+0000 7fd019f64640 1 -- 172.21.15.40:0/1544561686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd00c056580 con 0x7fd00c00a860 2024-04-26T21:35:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.584+0000 7fd0117fa640 1 -- 172.21.15.40:0/1544561686 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd008079250 con 0x7fd00c00a860 2024-04-26T21:35:35.585 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2024-04-26T21:35:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.587+0000 7fcffeffd640 1 -- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcff806c9c0 msgr2=0x7fcff806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.587+0000 7fcffeffd640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcff806c9c0 0x7fcff806ee80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd014051930 tx=0x7fd014098040 comp rx=0 tx=0).stop 2024-04-26T21:35:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.587+0000 7fcffeffd640 1 -- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c00a860 msgr2=0x7fd00c0dda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.587+0000 7fcffeffd640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c00a860 0x7fd00c0dda40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd00c011e20 tx=0x7fd008001a60 comp rx=0 tx=0).stop 2024-04-26T21:35:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fcffeffd640 1 -- 172.21.15.40:0/1544561686 shutdown_connections 2024-04-26T21:35:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fcffeffd640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcff806c9c0 0x7fcff806ee80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fcffeffd640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c00a860 0x7fd00c0dda40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fcffeffd640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00c0dced0 0x7fd00c052790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fcffeffd640 1 --2- 172.21.15.40:0/1544561686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040051c0 0x7fd00c0e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fcffeffd640 1 -- 172.21.15.40:0/1544561686 >> 172.21.15.40:0/1544561686 conn(0x7fd00c0559e0 msgr2=0x7fd00c053ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fcffeffd640 1 -- 172.21.15.40:0/1544561686 shutdown_connections 2024-04-26T21:35:35.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fcffeffd640 1 -- 172.21.15.40:0/1544561686 wait complete. 2024-04-26T21:35:35.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.588+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc3b40b1640 con 0x7fc3b40ce130 2024-04-26T21:35:35.591 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2024-04-26T21:35:35.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.589+0000 7fc3b0ff9640 1 -- 172.21.15.40:0/1417894052 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3ac05ce50 con 0x7fc3b40ce130 2024-04-26T21:35:35.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.590+0000 7f0646ffd640 1 -- 172.21.15.40:0/17281818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0638005850 con 0x7f06600727a0 2024-04-26T21:35:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.591+0000 7f065d7fa640 1 -- 172.21.15.40:0/17281818 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0658010040 con 0x7f06600727a0 2024-04-26T21:35:35.592 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc39406c9c0 msgr2=0x7fc39406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc39406c9c0 0x7fc39406ee80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40093e0 tx=0x7fc3a4009040 comp rx=0 tx=0).stop 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b40ce130 msgr2=0x7fc3b40cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b40ce130 0x7fc3b40cab50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00c3c0 tx=0x7fc3ac001e70 comp rx=0 tx=0).stop 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 shutdown_connections 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc39406c9c0 0x7fc39406ee80 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40093e0 tx=0x7fc3a4009040 comp rx=0 tx=0).stop 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3b40ce130 0x7fc3b40cab50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3b40cd3e0 0x7fc3b41666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 --2- 172.21.15.40:0/1417894052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c45b0 0x7fc3b40c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.592+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 >> 172.21.15.40:0/1417894052 conn(0x7fc3b4073960 msgr2=0x7fc3b40b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.594+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 shutdown_connections 2024-04-26T21:35:35.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.594+0000 7fc3b9c07640 1 -- 172.21.15.40:0/1417894052 wait complete. 2024-04-26T21:35:35.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.594+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1ce403bcb0 con 0x7f1ce40e7820 2024-04-26T21:35:35.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.595+0000 7f9a99ffb640 1 -- 172.21.15.40:0/1204011326 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9ab40cb3c0 con 0x7f9ab40d5b20 2024-04-26T21:35:35.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.595+0000 7f1ce8ff9640 1 -- 172.21.15.40:0/320251166 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ce0059730 con 0x7f1ce40e7820 2024-04-26T21:35:35.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.595+0000 7f9a9bfff640 1 -- 172.21.15.40:0/1204011326 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9aac07be50 con 0x7f9ab40d5b20 2024-04-26T21:35:35.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.595+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f49540b7a70 con 0x7f49540c6ee0 2024-04-26T21:35:35.597 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2024-04-26T21:35:35.597 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2024-04-26T21:35:35.597 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2024-04-26T21:35:35.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.595+0000 7f4950ff9640 1 -- 172.21.15.40:0/2434971471 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4948027020 con 0x7f49540c6ee0 2024-04-26T21:35:35.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fadf00694f0 con 0x7fadf00d3bc0 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 -- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f064c06c9c0 msgr2=0x7f064c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f064c06c9c0 0x7f064c06ee80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0654034000 tx=0x7f065402c000 comp rx=0 tx=0).stop 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 -- 172.21.15.40:0/17281818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600727a0 msgr2=0x7f0660159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600727a0 0x7f0660159a90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f065800ebc0 tx=0x7f065800c490 comp rx=0 tx=0).stop 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 -- 172.21.15.40:0/17281818 shutdown_connections 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f064c06c9c0 0x7f064c06ee80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066015bdf0 0x7f0660139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066015b4d0 0x7f0660139d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.596+0000 7f0646ffd640 1 --2- 172.21.15.40:0/17281818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600727a0 0x7f0660159a90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.597+0000 7f0646ffd640 1 -- 172.21.15.40:0/17281818 >> 172.21.15.40:0/17281818 conn(0x7f06600b2770 msgr2=0x7f06600b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.597+0000 7f0646ffd640 1 -- 172.21.15.40:0/17281818 shutdown_connections 2024-04-26T21:35:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.597+0000 7f0646ffd640 1 -- 172.21.15.40:0/17281818 wait complete. 2024-04-26T21:35:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.597+0000 7fade6ffd640 1 -- 172.21.15.40:0/811718643 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fade0021020 con 0x7fadf00d3bc0 2024-04-26T21:35:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cd006c8f0 msgr2=0x7f1cd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cd006c8f0 0x7f1cd006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40e6b50 tx=0x7f1cd8009240 comp rx=0 tx=0).stop 2024-04-26T21:35:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40e7820 msgr2=0x7f1ce40daef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40e7820 0x7f1ce40daef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000e960 tx=0x7f1ce000ee20 comp rx=0 tx=0).stop 2024-04-26T21:35:35.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 shutdown_connections 2024-04-26T21:35:35.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cd006c8f0 0x7f1cd006edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ce40ebcc0 0x7f1ce4039e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce40eb300 0x7f1ce4039930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 --2- 172.21.15.40:0/320251166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40e7820 0x7f1ce40daef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.599+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 >> 172.21.15.40:0/320251166 conn(0x7f1ce40559e0 msgr2=0x7f1ce403c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.601 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2024-04-26T21:35:35.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 -- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9aa006c940 msgr2=0x7f9aa006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 shutdown_connections 2024-04-26T21:35:35.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f1cf234d640 1 -- 172.21.15.40:0/320251166 wait complete. 2024-04-26T21:35:35.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9aa006c940 0x7f9aa006ee00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009ac0 tx=0x7f9aa8008040 comp rx=0 tx=0).stop 2024-04-26T21:35:35.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 -- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab40d5b20 msgr2=0x7f9ab41567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab40d5b20 0x7f9ab41567d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009d90 tx=0x7f9aac004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:35.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f494006caf0 msgr2=0x7f494006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f494006caf0 0x7f494006efb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4944003f00 tx=0x7f4944009040 comp rx=0 tx=0).stop 2024-04-26T21:35:35.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-04-26T21:35:35.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49540c6ee0 msgr2=0x7f49540c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49540c6ee0 0x7f49540c9f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f494800d9c0 tx=0x7f494800de80 comp rx=0 tx=0).stop 2024-04-26T21:35:35.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 shutdown_connections 2024-04-26T21:35:35.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f494006caf0 0x7f494006efb0 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4944003f00 tx=0x7f4944009040 comp rx=0 tx=0).stop 2024-04-26T21:35:35.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4954160440 0x7f495413dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49540c6ee0 0x7f49540c9f10 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f494800d9c0 tx=0x7f494800de80 comp rx=0 tx=0).stop 2024-04-26T21:35:35.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 --2- 172.21.15.40:0/2434971471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4954072dd0 0x7f4954159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 >> 172.21.15.40:0/2434971471 conn(0x7f49540d1ae0 msgr2=0x7f4954074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 -- 172.21.15.40:0/1204011326 shutdown_connections 2024-04-26T21:35:35.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9aa006c940 0x7f9aa006ee00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab4167470 0x7f9ab41574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab40d5b20 0x7f9ab41567d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 --2- 172.21.15.40:0/1204011326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ab40d32a0 0x7f9ab4159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.600+0000 7f9a99ffb640 1 -- 172.21.15.40:0/1204011326 >> 172.21.15.40:0/1204011326 conn(0x7f9ab40d4e70 msgr2=0x7f9ab40b79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7f9a99ffb640 1 -- 172.21.15.40:0/1204011326 shutdown_connections 2024-04-26T21:35:35.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fadd406c970 msgr2=0x7fadd406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fadd406c970 0x7fadd406ee30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fadec009630 tx=0x7fadec02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:35.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadf00d3bc0 msgr2=0x7fadf0155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:35.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadf00d3bc0 0x7fadf0155490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fade000c440 tx=0x7fade000c840 comp rx=0 tx=0).stop 2024-04-26T21:35:35.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7f9a99ffb640 1 -- 172.21.15.40:0/1204011326 wait complete. 2024-04-26T21:35:35.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 shutdown_connections 2024-04-26T21:35:35.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 shutdown_connections 2024-04-26T21:35:35.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fadd406c970 0x7fadd406ee30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadf01f8580 0x7fadf015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadf00d3bc0 0x7fadf0155490 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 --2- 172.21.15.40:0/811718643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadf00b5f20 0x7fadf0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:35.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 >> 172.21.15.40:0/811718643 conn(0x7fadf00d4f60 msgr2=0x7fadf006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:35.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7f4959727640 1 -- 172.21.15.40:0/2434971471 wait complete. 2024-04-26T21:35:35.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 shutdown_connections 2024-04-26T21:35:35.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:35.601+0000 7fadf836a640 1 -- 172.21.15.40:0/811718643 wait complete. 2024-04-26T21:35:35.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-04-26T21:35:35.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-26T21:35:35.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-26T21:35:35.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2024-04-26T21:35:35.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-04-26T21:35:35.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-04-26T21:35:35.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-04-26T21:35:36.605 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T21:35:36.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T21:35:36.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T21:35:36.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T21:35:36.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T21:35:36.625 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T21:35:36.625 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T21:35:36.626 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T21:35:36.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.776+0000 7fbdee351640 1 Processor -- start 2024-04-26T21:35:36.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.776+0000 7fbdee351640 1 -- start start 2024-04-26T21:35:36.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.777+0000 7fbdee351640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbde80701f0 0x7fbde80cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.777+0000 7fbdee351640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80cb770 0x7fbde8073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.777+0000 7fbdee351640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde80cd490 con 0x7fbde806fe10 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.777+0000 7fbdee351640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbde80cec30 con 0x7fbde80701f0 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.777+0000 7fbdee351640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbde80b0520 con 0x7fbde80cb770 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.778+0000 7fbde77fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbde80701f0 0x7fbde80cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.778+0000 7fbde77fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbde80701f0 0x7fbde80cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49780/0 (socket says 172.21.15.40:49780) 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.778+0000 7fbde77fe640 1 -- 172.21.15.40:0/614199828 learned_addr learned my addr 172.21.15.40:0/614199828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.778+0000 7fbdec8ee640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80cb770 0x7fbde8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.778+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453479681 0 0) 0x7fbde80cd490 con 0x7fbde806fe10 2024-04-26T21:35:36.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdd0007930 con 0x7fbde806fe10 2024-04-26T21:35:36.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 181169151 0 0) 0x7fbdd0007930 con 0x7fbde806fe10 2024-04-26T21:35:36.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80cb770 msgr2=0x7fbde8073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80cb770 0x7fbde8073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 >> v2:172.21.15.184:3300/0 conn(0x7fbde80701f0 msgr2=0x7fbde80cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 --2- 172.21.15.40:0/614199828 >> v2:172.21.15.184:3300/0 conn(0x7fbde80701f0 0x7fbde80cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde80cced0 con 0x7fbde806fe10 2024-04-26T21:35:36.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fbddc001670 con 0x7fbde806fe10 2024-04-26T21:35:36.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 >> v1:172.21.15.40:6789/0 conn(0x7fbde806fe10 legacy=0x7fbde80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd0007dc0 0x7fbdd00081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0008c10 0x7fbdd0009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdd0007440 0x7fbdd00095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbde80cec30 con 0x7fbdd0008c10 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbde80b0520 con 0x7fbdd0007dc0 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.779+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd000c020 con 0x7fbdd0007440 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.780+0000 7fbde7fff640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd0007dc0 0x7fbdd00081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.780+0000 7fbde7fff640 1 -- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdd0007440 msgr2=0x7fbdd00095d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.780+0000 7fbde7fff640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdd0007440 0x7fbdd00095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.780+0000 7fbde7fff640 1 -- 172.21.15.40:0/614199828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0008c10 msgr2=0x7fbdd0009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.780+0000 7fbde7fff640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0008c10 0x7fbdd0009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.780+0000 7fbde7fff640 1 -- 172.21.15.40:0/614199828 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde80cced0 con 0x7fbdd0007dc0 2024-04-26T21:35:36.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.782+0000 7fbde7fff640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd0007dc0 0x7fbdd00081e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbddc007c80 tx=0x7fbdd0009c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdce24abfa5d7e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.782+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc058070 con 0x7fbdd0007dc0 2024-04-26T21:35:36.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.782+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbddc007270 con 0x7fbdd0007dc0 2024-04-26T21:35:36.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.782+0000 7fbde6ffd640 1 -- 172.21.15.40:0/614199828 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc007cc0 con 0x7fbdd0007dc0 2024-04-26T21:35:36.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.783+0000 7fbdee351640 1 -- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd0007dc0 msgr2=0x7fbdd00081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.783+0000 7fbdee351640 1 --2- 172.21.15.40:0/614199828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd0007dc0 0x7fbdd00081e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbddc007c80 tx=0x7fbdd0009c40 comp rx=0 tx=0).stop 2024-04-26T21:35:36.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.784+0000 7fbdec8ee640 1 -- 172.21.15.40:0/614199828 reap_dead start 2024-04-26T21:35:36.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.785+0000 7fbdee351640 1 -- 172.21.15.40:0/614199828 shutdown_connections 2024-04-26T21:35:36.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.785+0000 7fbdee351640 1 -- 172.21.15.40:0/614199828 >> 172.21.15.40:0/614199828 conn(0x7fbde80d0be0 msgr2=0x7fbde80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.785+0000 7fbdee351640 1 -- 172.21.15.40:0/614199828 shutdown_connections 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.785+0000 7fbdee351640 1 -- 172.21.15.40:0/614199828 wait complete. 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.787+0000 7fbdee351640 1 Processor -- start 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.787+0000 7fbdee351640 1 -- start start 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.788+0000 7fbdee351640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0007dc0 0x7fbde80af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.788+0000 7fbdee351640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde80b5a50 0x7fbde80b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.788+0000 7fbdee351640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80ae1e0 0x7fbde80ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.788+0000 7fbdee351640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbde80ccb60 con 0x7fbdd0007dc0 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.788+0000 7fbdee351640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbde80ce8c0 con 0x7fbde80b5a50 2024-04-26T21:35:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.788+0000 7fbdee351640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbde8072080 con 0x7fbde80ae1e0 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde80b5a50 0x7fbde80b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde80b5a50 0x7fbde80b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49786/0 (socket says 172.21.15.40:49786) 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde77fe640 1 -- 172.21.15.40:0/3042690873 learned_addr learned my addr 172.21.15.40:0/3042690873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbdec8ee640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80ae1e0 0x7fbde80ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde7fff640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0007dc0 0x7fbde80af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde77fe640 1 -- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80ae1e0 msgr2=0x7fbde80ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde77fe640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80ae1e0 0x7fbde80ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde77fe640 1 -- 172.21.15.40:0/3042690873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0007dc0 msgr2=0x7fbde80af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde77fe640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0007dc0 0x7fbde80af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.789+0000 7fbde77fe640 1 -- 172.21.15.40:0/3042690873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde808ad00 con 0x7fbde80b5a50 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.790+0000 7fbde7fff640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0007dc0 0x7fbde80af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.790+0000 7fbde77fe640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde80b5a50 0x7fbde80b5eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800a840 tx=0x7fbdd800ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.792+0000 7fbde4ff9640 1 -- 172.21.15.40:0/3042690873 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8017020 con 0x7fbde80b5a50 2024-04-26T21:35:36.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.792+0000 7fbdee351640 1 -- 172.21.15.40:0/3042690873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde808c040 con 0x7fbde80b5a50 2024-04-26T21:35:36.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.792+0000 7fbdee351640 1 -- 172.21.15.40:0/3042690873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde808b280 con 0x7fbde80b5a50 2024-04-26T21:35:36.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.793+0000 7fbde4ff9640 1 -- 172.21.15.40:0/3042690873 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd800ae50 con 0x7fbde80b5a50 2024-04-26T21:35:36.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.794+0000 7fbde4ff9640 1 -- 172.21.15.40:0/3042690873 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800dc70 con 0x7fbde80b5a50 2024-04-26T21:35:36.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.795+0000 7fbdee351640 1 -- 172.21.15.40:0/3042690873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc8005290 con 0x7fbde80b5a50 2024-04-26T21:35:36.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.796+0000 7fbde4ff9640 1 -- 172.21.15.40:0/3042690873 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbdd801a020 con 0x7fbde80b5a50 2024-04-26T21:35:36.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.797+0000 7fbde4ff9640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdd406c9c0 0x7fbdd406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.797+0000 7fbde4ff9640 1 -- 172.21.15.40:0/3042690873 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbdd808bf90 con 0x7fbde80b5a50 2024-04-26T21:35:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.798+0000 7fbde7fff640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdd406c9c0 0x7fbdd406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.798+0000 7fbde7fff640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdd406c9c0 0x7fbdd406ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbddc055000 tx=0x7fbddc009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.802+0000 7fbde4ff9640 1 -- 172.21.15.40:0/3042690873 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbdd8021020 con 0x7fbde80b5a50 2024-04-26T21:35:36.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.826+0000 7f6c44a52640 1 Processor -- start 2024-04-26T21:35:36.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.826+0000 7f6c44a52640 1 -- start start 2024-04-26T21:35:36.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f319184e640 1 Processor -- start 2024-04-26T21:35:36.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f319184e640 1 -- start start 2024-04-26T21:35:36.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f6c44a52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c400b56c0 0x7f6c400afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f6c44a52640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c400b5e30 con 0x7f6c40070750 2024-04-26T21:35:36.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f6c44a52640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c400b6110 con 0x7f6c40072fb0 2024-04-26T21:35:36.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f6c44a52640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6c4006f2b0 con 0x7f6c400b56c0 2024-04-26T21:35:36.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f6c3ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c400b56c0 0x7f6c400afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f6c3ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c400b56c0 0x7f6c400afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45754/0 (socket says 172.21.15.40:45754) 2024-04-26T21:35:36.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f6c3ed9d640 1 -- 172.21.15.40:0/859982190 learned_addr learned my addr 172.21.15.40:0/859982190 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.829+0000 7fbce95b5640 1 Processor -- start 2024-04-26T21:35:36.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.829+0000 7fbce95b5640 1 -- start start 2024-04-26T21:35:36.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.829+0000 7f6c3ed9d640 1 -- 172.21.15.40:0/859982190 >> v1:172.21.15.184:6789/0 conn(0x7f6c40072fb0 legacy=0x7f6c40073390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:35:36.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.829+0000 7f6c3ed9d640 1 -- 172.21.15.40:0/859982190 >> v1:172.21.15.40:6789/0 conn(0x7f6c40070750 legacy=0x7f6c400af6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:35:36.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.829+0000 7f6c3ed9d640 1 -- 172.21.15.40:0/859982190 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c400b63f0 con 0x7f6c400b56c0 2024-04-26T21:35:36.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.828+0000 7f888b9f2640 1 Processor -- start 2024-04-26T21:35:36.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7fbce95b5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce416fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7fbce95b5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbce40b4ee0 0x7fbce406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7fbce95b5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce40b5d30 con 0x7fbce416ff80 2024-04-26T21:35:36.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7fbce95b5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbce408c2e0 con 0x7fbce40b4ee0 2024-04-26T21:35:36.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7fbce95b5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbce408c580 con 0x7fbce416f660 2024-04-26T21:35:36.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.829+0000 7f888b9f2640 1 -- start start 2024-04-26T21:35:36.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f319184e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3184006af0 0x7f3184048fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f319184e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3184049720 0x7f3184006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7fbce37fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fbce416ff80 0x7fbce406e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:58044/0 (socket says 172.21.15.40:58044) 2024-04-26T21:35:36.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7fbce37fe640 1 -- 172.21.15.40:0/3961119181 learned_addr learned my addr 172.21.15.40:0/3961119181 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:36.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7fbce3fff640 1 --2- 172.21.15.40:0/3961119181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce416fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f319184e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3184007410 con 0x7f31840543b0 2024-04-26T21:35:36.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f319184e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3184048cc0 con 0x7f3184049720 2024-04-26T21:35:36.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f319184e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f31840538d0 con 0x7f3184006af0 2024-04-26T21:35:36.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f6c3ed9d640 1 --2- 172.21.15.40:0/859982190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c400b56c0 0x7f6c400afdf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009e50 tx=0x7f6c3802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4496b538f38633 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7fbce8db4640 1 --2- 172.21.15.40:0/3961119181 >> v2:172.21.15.40:3300/0 conn(0x7fbce40b4ee0 0x7fbce406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f318bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3184049720 0x7f3184006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7fbce2ffd640 1 -- 172.21.15.40:0/3961119181 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2523020183 0 0) 0x7fbce40b5d30 con 0x7fbce416ff80 2024-04-26T21:35:36.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7fbce2ffd640 1 -- 172.21.15.40:0/3961119181 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbccc003520 con 0x7fbce416ff80 2024-04-26T21:35:36.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f319084c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3184006af0 0x7f3184048fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f319084c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3184006af0 0x7f3184048fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53822/0 (socket says 172.21.15.40:53822) 2024-04-26T21:35:36.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f319084c640 1 -- 172.21.15.40:0/4272789895 learned_addr learned my addr 172.21.15.40:0/4272789895 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f888b9f2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f88840ce7b0 0x7f88840d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f888b9f2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88840c3900 0x7f88840ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f888b9f2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88840cf0d0 con 0x7f88840c3e50 2024-04-26T21:35:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f888b9f2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f88840721e0 con 0x7f88840c3900 2024-04-26T21:35:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f888b9f2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8884071f00 con 0x7f88840ce7b0 2024-04-26T21:35:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.830+0000 7f888978e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f88840ce7b0 0x7f88840d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f8889f8f640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f88840c3e50 0x7f88840cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:58058/0 (socket says 172.21.15.40:58058) 2024-04-26T21:35:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f8889f8f640 1 -- 172.21.15.40:0/2266314043 learned_addr learned my addr 172.21.15.40:0/2266314043 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f8888f8d640 1 --2- 172.21.15.40:0/2266314043 >> v2:172.21.15.40:3300/0 conn(0x7f88840c3900 0x7f88840ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f319084c640 1 -- 172.21.15.40:0/4272789895 >> v2:172.21.15.40:3300/0 conn(0x7f3184049720 msgr2=0x7f3184006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f319084c640 1 --2- 172.21.15.40:0/4272789895 >> v2:172.21.15.40:3300/0 conn(0x7f3184049720 0x7f3184006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f319084c640 1 -- 172.21.15.40:0/4272789895 >> v1:172.21.15.184:6789/0 conn(0x7f31840543b0 legacy=0x7f3184054840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f6c3d59a640 1 -- 172.21.15.40:0/859982190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38036070 con 0x7f6c400b56c0 2024-04-26T21:35:36.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f6c3d59a640 1 -- 172.21.15.40:0/859982190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c38031600 con 0x7f6c400b56c0 2024-04-26T21:35:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7fbce3fff640 1 -- 172.21.15.40:0/3961119181 >> v2:172.21.15.40:3300/0 conn(0x7fbce40b4ee0 msgr2=0x7fbce406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7fbce2ffd640 1 -- 172.21.15.40:0/3961119181 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1292944866 0 0) 0x7fbccc003520 con 0x7fbce416ff80 2024-04-26T21:35:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.831+0000 7f6c3d59a640 1 -- 172.21.15.40:0/859982190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38031a10 con 0x7f6c400b56c0 2024-04-26T21:35:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7f888978e640 1 -- 172.21.15.40:0/2266314043 >> v2:172.21.15.40:3300/0 conn(0x7f88840c3900 msgr2=0x7f88840ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7f888978e640 1 --2- 172.21.15.40:0/2266314043 >> v2:172.21.15.40:3300/0 conn(0x7f88840c3900 0x7f88840ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7f888978e640 1 -- 172.21.15.40:0/2266314043 >> v1:172.21.15.184:6789/0 conn(0x7f88840c3e50 legacy=0x7f88840cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7f888978e640 1 -- 172.21.15.40:0/2266314043 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88840cfa70 con 0x7f88840ce7b0 2024-04-26T21:35:36.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7fcc35db3640 1 Processor -- start 2024-04-26T21:35:36.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7fcc35db3640 1 -- start start 2024-04-26T21:35:36.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7f318b7fe640 1 -- 172.21.15.40:0/4272789895 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2519842474 0 0) 0x7f3184007410 con 0x7f31840543b0 2024-04-26T21:35:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7fbce3fff640 1 --2- 172.21.15.40:0/3961119181 >> v2:172.21.15.40:3300/0 conn(0x7fbce40b4ee0 0x7fbce406ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7fbce3fff640 1 -- 172.21.15.40:0/3961119181 >> v1:172.21.15.184:6789/0 conn(0x7fbce416ff80 legacy=0x7fbce406e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7fbce3fff640 1 -- 172.21.15.40:0/3961119181 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce408c860 con 0x7fbce416f660 2024-04-26T21:35:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f580aada640 1 Processor -- start 2024-04-26T21:35:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7f6c44a52640 1 -- 172.21.15.40:0/859982190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c400b56c0 msgr2=0x7f6c400afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7f6c44a52640 1 --2- 172.21.15.40:0/859982190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c400b56c0 0x7f6c400afdf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009e50 tx=0x7f6c3802a780 comp rx=0 tx=0).stop 2024-04-26T21:35:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f2d05431640 1 Processor -- start 2024-04-26T21:35:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7fbce8db4640 1 --2- 172.21.15.40:0/3961119181 >> v2:172.21.15.40:3300/0 conn(0x7fbce40b4ee0 0x7fbce406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f6c44a52640 1 -- 172.21.15.40:0/859982190 shutdown_connections 2024-04-26T21:35:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f6c44a52640 1 --2- 172.21.15.40:0/859982190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c400b56c0 0x7f6c400afdf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f6c44a52640 1 -- 172.21.15.40:0/859982190 >> 172.21.15.40:0/859982190 conn(0x7f6c400b39b0 msgr2=0x7f6c400b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.832+0000 7f319084c640 1 -- 172.21.15.40:0/4272789895 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3184053bb0 con 0x7f3184006af0 2024-04-26T21:35:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f2d05431640 1 -- start start 2024-04-26T21:35:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f888978e640 1 --2- 172.21.15.40:0/2266314043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88840ce7b0 0x7f88840d0550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f888406c4e0 tx=0x7f888002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f03f1fa1bf72d366 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f580aada640 1 -- start start 2024-04-26T21:35:36.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f6c44a52640 1 -- 172.21.15.40:0/859982190 shutdown_connections 2024-04-26T21:35:36.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.833+0000 7f6c44a52640 1 -- 172.21.15.40:0/859982190 wait complete. 2024-04-26T21:35:36.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f887bfff640 1 -- 172.21.15.40:0/2266314043 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880037070 con 0x7f88840ce7b0 2024-04-26T21:35:36.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f887bfff640 1 -- 172.21.15.40:0/2266314043 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8880032440 con 0x7f88840ce7b0 2024-04-26T21:35:36.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7fbce3fff640 1 --2- 172.21.15.40:0/3961119181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce416fa40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400ba50 tx=0x7fbcd402c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b77742868f86d7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7fcc35db3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc280df220 0x7fcc28038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7fbce2ffd640 1 -- 172.21.15.40:0/3961119181 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd403a070 con 0x7fbce416f660 2024-04-26T21:35:36.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7fbce2ffd640 1 -- 172.21.15.40:0/3961119181 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd4032480 con 0x7fbce416f660 2024-04-26T21:35:36.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7fcc35db3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc28053c70 0x7fcc2804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f580aada640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58040c41d0 0x7f58040cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f580aada640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 0x7f58041671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7fcc34db1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc280df220 0x7fcc28038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7fcc34db1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc280df220 0x7fcc28038080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53834/0 (socket says 172.21.15.40:53834) 2024-04-26T21:35:36.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f2d05431640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f2d05431640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2cf803e840 0x7f2cf8053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f2d05431640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cf8054130 0x7f2cf80e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f2d05431640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf80582d0 con 0x7f2cf8054130 2024-04-26T21:35:36.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f2d05431640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf8058570 con 0x7f2cf803e840 2024-04-26T21:35:36.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f2d05431640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf8058850 con 0x7f2cf80e30c0 2024-04-26T21:35:36.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fcc34db1640 1 -- 172.21.15.40:0/3802258397 learned_addr learned my addr 172.21.15.40:0/3802258397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fcc2ffff640 1 --2- 172.21.15.40:0/3802258397 >> v2:172.21.15.40:3300/0 conn(0x7fcc28053c70 0x7fcc2804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f580aada640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5804163a50 con 0x7f5804167720 2024-04-26T21:35:36.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f580aada640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f58040cf7a0 con 0x7f58040c41d0 2024-04-26T21:35:36.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.834+0000 7f580aada640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58040ceb90 con 0x7f5804073f30 2024-04-26T21:35:36.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f5808876640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58040c41d0 0x7f58040cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f5803fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 0x7f58041671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f5803fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 0x7f58041671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53840/0 (socket says 172.21.15.40:53840) 2024-04-26T21:35:36.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f5803fff640 1 -- 172.21.15.40:0/3213349554 learned_addr learned my addr 172.21.15.40:0/3213349554 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f2d04c30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cf8054130 0x7f2cf80e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f2d04c30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cf8054130 0x7f2cf80e6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45802/0 (socket says 172.21.15.40:45802) 2024-04-26T21:35:36.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f2d04c30640 1 -- 172.21.15.40:0/764467019 learned_addr learned my addr 172.21.15.40:0/764467019 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fbce95b5640 1 -- 172.21.15.40:0/3961119181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 msgr2=0x7fbce416fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f888b9f2640 1 -- 172.21.15.40:0/2266314043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88840ce7b0 msgr2=0x7f88840d0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f888b9f2640 1 --2- 172.21.15.40:0/2266314043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88840ce7b0 0x7f88840d0550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f888406c4e0 tx=0x7f888002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f888b9f2640 1 -- 172.21.15.40:0/2266314043 shutdown_connections 2024-04-26T21:35:36.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f888b9f2640 1 --2- 172.21.15.40:0/2266314043 >> v2:172.21.15.40:3300/0 conn(0x7f88840c3900 0x7f88840ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f888b9f2640 1 --2- 172.21.15.40:0/2266314043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88840ce7b0 0x7f88840d0550 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f888b9f2640 1 -- 172.21.15.40:0/2266314043 >> 172.21.15.40:0/2266314043 conn(0x7f88840d0be0 msgr2=0x7f88840d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:36.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fcc35db3640 1 -- 172.21.15.40:0/3802258397 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc280544c0 con 0x7fcc28033ef0 2024-04-26T21:35:36.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fcc35db3640 1 -- 172.21.15.40:0/3802258397 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcc280541e0 con 0x7fcc28053c70 2024-04-26T21:35:36.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fcc35db3640 1 -- 172.21.15.40:0/3802258397 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc28005f30 con 0x7fcc280df220 2024-04-26T21:35:36.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fbce95b5640 1 --2- 172.21.15.40:0/3961119181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce416fa40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400ba50 tx=0x7fbcd402c7d0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f2cff7fe640 1 --2- 172.21.15.40:0/764467019 >> v2:172.21.15.184:3300/0 conn(0x7f2cf803e840 0x7f2cf8053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f2cfffff640 1 --2- 172.21.15.40:0/764467019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f5808876640 1 -- 172.21.15.40:0/3213349554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 msgr2=0x7f58041671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f5808876640 1 --2- 172.21.15.40:0/3213349554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 0x7f58041671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fbce95b5640 1 -- 172.21.15.40:0/3961119181 shutdown_connections 2024-04-26T21:35:36.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fbce95b5640 1 --2- 172.21.15.40:0/3961119181 >> v2:172.21.15.40:3300/0 conn(0x7fbce40b4ee0 0x7fbce406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fbce95b5640 1 --2- 172.21.15.40:0/3961119181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce416fa40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fbce95b5640 1 -- 172.21.15.40:0/3961119181 >> 172.21.15.40:0/3961119181 conn(0x7fbce40b2720 msgr2=0x7fbce40b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7fbce95b5640 1 -- 172.21.15.40:0/3961119181 shutdown_connections 2024-04-26T21:35:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f5808876640 1 -- 172.21.15.40:0/3213349554 >> v1:172.21.15.184:6789/0 conn(0x7f5804167720 legacy=0x7f580406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f5808876640 1 -- 172.21.15.40:0/3213349554 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58040cee70 con 0x7f58040c41d0 2024-04-26T21:35:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f319084c640 1 --2- 172.21.15.40:0/4272789895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3184006af0 0x7f3184048fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f317c009870 tx=0x7f317c02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4708571a4cfccaf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f58037fe640 1 -- 172.21.15.40:0/3213349554 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924567216 0 0) 0x7f5804163a50 con 0x7f5804167720 2024-04-26T21:35:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7fcc34db1640 1 -- 172.21.15.40:0/3802258397 >> v2:172.21.15.40:3300/0 conn(0x7fcc28053c70 msgr2=0x7fcc2804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7fcc34db1640 1 --2- 172.21.15.40:0/3802258397 >> v2:172.21.15.40:3300/0 conn(0x7fcc28053c70 0x7fcc2804fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7fbce95b5640 1 -- 172.21.15.40:0/3961119181 wait complete. 2024-04-26T21:35:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f888b9f2640 1 -- 172.21.15.40:0/2266314043 shutdown_connections 2024-04-26T21:35:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.835+0000 7f888b9f2640 1 -- 172.21.15.40:0/2266314043 wait complete. 2024-04-26T21:35:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7f6c44a52640 1 Processor -- start 2024-04-26T21:35:36.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7f2d04c30640 1 -- 172.21.15.40:0/764467019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 msgr2=0x7f2cf8057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7f2d04c30640 1 --2- 172.21.15.40:0/764467019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8057d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7f2d04c30640 1 -- 172.21.15.40:0/764467019 >> v2:172.21.15.184:3300/0 conn(0x7f2cf803e840 msgr2=0x7f2cf8053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7f2d04c30640 1 --2- 172.21.15.40:0/764467019 >> v2:172.21.15.184:3300/0 conn(0x7f2cf803e840 0x7f2cf8053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7f2d04c30640 1 -- 172.21.15.40:0/764467019 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf8058b30 con 0x7f2cf8054130 2024-04-26T21:35:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f5808876640 1 --2- 172.21.15.40:0/3213349554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f58040cfc30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58040cf920 tx=0x7f57f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225a9b3ecba2821a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7fcc34db1640 1 -- 172.21.15.40:0/3802258397 >> v1:172.21.15.40:6789/0 conn(0x7fcc28033ef0 legacy=0x7fcc28006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7fcc34db1640 1 -- 172.21.15.40:0/3802258397 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc28006210 con 0x7fcc280df220 2024-04-26T21:35:36.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7fbce95b5640 1 Processor -- start 2024-04-26T21:35:36.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f2d04c30640 1 --2- 172.21.15.40:0/764467019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 0x7f2cf80e6c30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d0006b750 tx=0x7f2d000a6df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ab2d073b86e715 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7fcc34db1640 1 --2- 172.21.15.40:0/3802258397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc280df220 0x7fcc28038080 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2402c630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d45d5e472a58a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f58037fe640 1 -- 172.21.15.40:0/3213349554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8037070 con 0x7f58040c41d0 2024-04-26T21:35:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f58037fe640 1 -- 172.21.15.40:0/3213349554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f8032440 con 0x7f58040c41d0 2024-04-26T21:35:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f58037fe640 1 -- 172.21.15.40:0/3213349554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8032810 con 0x7f58040c41d0 2024-04-26T21:35:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f888b9f2640 1 Processor -- start 2024-04-26T21:35:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f888b9f2640 1 -- start start 2024-04-26T21:35:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7fcc2f7fe640 1 -- 172.21.15.40:0/3802258397 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24038070 con 0x7fcc280df220 2024-04-26T21:35:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7fcc2f7fe640 1 -- 172.21.15.40:0/3802258397 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2402ce50 con 0x7fcc280df220 2024-04-26T21:35:36.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7fcc2f7fe640 1 -- 172.21.15.40:0/3802258397 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24031970 con 0x7fcc280df220 2024-04-26T21:35:36.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f318b7fe640 1 -- 172.21.15.40:0/4272789895 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c037070 con 0x7f3184006af0 2024-04-26T21:35:36.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f318b7fe640 1 -- 172.21.15.40:0/4272789895 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317c032440 con 0x7f3184006af0 2024-04-26T21:35:36.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f318b7fe640 1 -- 172.21.15.40:0/4272789895 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c032810 con 0x7f3184006af0 2024-04-26T21:35:36.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.836+0000 7f6c44a52640 1 -- start start 2024-04-26T21:35:36.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f6c44a52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40070750 0x7f6c401fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f6c44a52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c40072fb0 0x7f6c400d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f6c44a52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c400b56c0 0x7f6c4014c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c44a52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4015c2e0 con 0x7f6c40070750 2024-04-26T21:35:36.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c44a52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4015c160 con 0x7f6c400b56c0 2024-04-26T21:35:36.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c44a52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6c400b0330 con 0x7f6c40072fb0 2024-04-26T21:35:36.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f580aada640 1 -- 172.21.15.40:0/3213349554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 msgr2=0x7f58040cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c3ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c400b56c0 0x7f6c4014c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c3ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c400b56c0 0x7f6c4014c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49804/0 (socket says 172.21.15.40:49804) 2024-04-26T21:35:36.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c3ed9d640 1 -- 172.21.15.40:0/431401689 learned_addr learned my addr 172.21.15.40:0/431401689 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c3dd9b640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c40072fb0 0x7f6c400d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f580aada640 1 --2- 172.21.15.40:0/3213349554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f58040cfc30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58040cf920 tx=0x7f57f802a180 comp rx=0 tx=0).stop 2024-04-26T21:35:36.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7fbce95b5640 1 -- start start 2024-04-26T21:35:36.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f2cfeffd640 1 -- 172.21.15.40:0/764467019 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000b4070 con 0x7f2cf8054130 2024-04-26T21:35:36.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f2cfeffd640 1 -- 172.21.15.40:0/764467019 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d000a7c00 con 0x7f2cf8054130 2024-04-26T21:35:36.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f2d05431640 1 -- 172.21.15.40:0/764467019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 msgr2=0x7f2cf80e6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f2d05431640 1 --2- 172.21.15.40:0/764467019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 0x7f2cf80e6c30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d0006b750 tx=0x7f2d000a6df0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.837+0000 7f2cfeffd640 1 -- 172.21.15.40:0/764467019 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000af660 con 0x7f2cf8054130 2024-04-26T21:35:36.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c3ed9d640 1 -- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c40072fb0 msgr2=0x7f6c400d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c3ed9d640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c40072fb0 0x7f6c400d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c3ed9d640 1 -- 172.21.15.40:0/431401689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40070750 msgr2=0x7f6c401fd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f6c3e59c640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40070750 0x7f6c401fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7fcc35db3640 1 -- 172.21.15.40:0/3802258397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc280df220 msgr2=0x7fcc28038080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7fcc35db3640 1 --2- 172.21.15.40:0/3802258397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc280df220 0x7fcc28038080 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2402c630 comp rx=0 tx=0).stop 2024-04-26T21:35:36.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f2d05431640 1 -- 172.21.15.40:0/764467019 shutdown_connections 2024-04-26T21:35:36.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f2d05431640 1 --2- 172.21.15.40:0/764467019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 0x7f2cf80e6c30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f2d05431640 1 --2- 172.21.15.40:0/764467019 >> v2:172.21.15.184:3300/0 conn(0x7f2cf803e840 0x7f2cf8053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f2d05431640 1 --2- 172.21.15.40:0/764467019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8057d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f2d05431640 1 -- 172.21.15.40:0/764467019 >> 172.21.15.40:0/764467019 conn(0x7f2cf80559e0 msgr2=0x7f2cf8055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:36.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f580aada640 1 -- 172.21.15.40:0/3213349554 shutdown_connections 2024-04-26T21:35:36.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f580aada640 1 --2- 172.21.15.40:0/3213349554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 0x7f58041671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f580aada640 1 --2- 172.21.15.40:0/3213349554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f58040cfc30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f580aada640 1 -- 172.21.15.40:0/3213349554 >> 172.21.15.40:0/3213349554 conn(0x7f58040d0be0 msgr2=0x7f58040d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:36.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f319184e640 1 -- 172.21.15.40:0/4272789895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3184006af0 msgr2=0x7f3184048fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.838+0000 7f319184e640 1 --2- 172.21.15.40:0/4272789895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3184006af0 0x7f3184048fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f317c009870 tx=0x7f317c02a130 comp rx=0 tx=0).stop 2024-04-26T21:35:36.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f6c3ed9d640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40070750 0x7f6c401fd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f6c3ed9d640 1 -- 172.21.15.40:0/431401689 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c400d19a0 con 0x7f6c400b56c0 2024-04-26T21:35:36.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f319184e640 1 -- 172.21.15.40:0/4272789895 shutdown_connections 2024-04-26T21:35:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f319184e640 1 --2- 172.21.15.40:0/4272789895 >> v2:172.21.15.40:3300/0 conn(0x7f3184049720 0x7f3184006ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fcc35db3640 1 -- 172.21.15.40:0/3802258397 shutdown_connections 2024-04-26T21:35:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fcc35db3640 1 --2- 172.21.15.40:0/3802258397 >> v2:172.21.15.40:3300/0 conn(0x7fcc28053c70 0x7fcc2804fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f319184e640 1 --2- 172.21.15.40:0/4272789895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3184006af0 0x7f3184048fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f319184e640 1 -- 172.21.15.40:0/4272789895 >> 172.21.15.40:0/4272789895 conn(0x7f3184055bd0 msgr2=0x7f3184055fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:36.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fcc35db3640 1 --2- 172.21.15.40:0/3802258397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc280df220 0x7fcc28038080 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fcc35db3640 1 -- 172.21.15.40:0/3802258397 >> 172.21.15.40:0/3802258397 conn(0x7fcc28009fb0 msgr2=0x7fcc2800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:36.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f580aada640 1 -- 172.21.15.40:0/3213349554 shutdown_connections 2024-04-26T21:35:36.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce95b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbce40b4ee0 0x7fbce40aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce95b5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fcc35db3640 1 -- 172.21.15.40:0/3802258397 shutdown_connections 2024-04-26T21:35:36.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f580aada640 1 -- 172.21.15.40:0/3213349554 wait complete. 2024-04-26T21:35:36.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fcc35db3640 1 -- 172.21.15.40:0/3802258397 wait complete. 2024-04-26T21:35:36.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f6c3ed9d640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c400b56c0 0x7f6c4014c6a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009f70 tx=0x7f6c38004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbce40b4ee0 0x7fbce40aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbce40b4ee0 0x7fbce40aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49810/0 (socket says 172.21.15.40:49810) 2024-04-26T21:35:36.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce3fff640 1 -- 172.21.15.40:0/1456748343 learned_addr learned my addr 172.21.15.40:0/1456748343 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce37fe640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce95b5640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbce416ff80 0x7fbce40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbce4170400 con 0x7fbce416ff80 2024-04-26T21:35:36.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbce406dde0 con 0x7fbce40b4ee0 2024-04-26T21:35:36.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbce4073a70 con 0x7fbce416f660 2024-04-26T21:35:36.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f888b9f2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88840c3900 0x7f88840afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7fbce8db4640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbce416ff80 0x7fbce40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f888b9f2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88840c3e50 0x7f88841514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888b9f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8884154630 0x7f88840c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888b9f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88840cf490 con 0x7f8884154630 2024-04-26T21:35:36.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888b9f2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88840d0030 con 0x7f88840c3e50 2024-04-26T21:35:36.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888b9f2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f88840cf610 con 0x7f88840c3900 2024-04-26T21:35:36.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888978e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88840c3900 0x7f88840afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888978e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88840c3900 0x7f88840afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53880/0 (socket says 172.21.15.40:53880) 2024-04-26T21:35:36.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888978e640 1 -- 172.21.15.40:0/166792646 learned_addr learned my addr 172.21.15.40:0/166792646 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f319184e640 1 -- 172.21.15.40:0/4272789895 shutdown_connections 2024-04-26T21:35:36.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f6c2effd640 1 -- 172.21.15.40:0/431401689 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38041020 con 0x7f6c400b56c0 2024-04-26T21:35:36.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.839+0000 7f6c2effd640 1 -- 172.21.15.40:0/431401689 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c38037040 con 0x7f6c400b56c0 2024-04-26T21:35:36.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f6c2effd640 1 -- 172.21.15.40:0/431401689 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38036070 con 0x7f6c400b56c0 2024-04-26T21:35:36.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c400ce620 con 0x7f6c400b56c0 2024-04-26T21:35:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.841+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c400cebe0 con 0x7f6c400b56c0 2024-04-26T21:35:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7fbce8db4640 1 -- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 msgr2=0x7fbce406ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7fbce8db4640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce406ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7fbce8db4640 1 -- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbce40b4ee0 msgr2=0x7fbce40aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7fbce8db4640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbce40b4ee0 0x7fbce40aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7fbce8db4640 1 -- 172.21.15.40:0/1456748343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce40ad550 con 0x7fbce416ff80 2024-04-26T21:35:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.841+0000 7fbce8db4640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbce416ff80 0x7fbce40ce1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800e7e0 tx=0x7fbcd800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888978e640 1 -- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88840c3e50 msgr2=0x7f88841514c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:35:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888978e640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88840c3e50 0x7f88841514c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888978e640 1 -- 172.21.15.40:0/166792646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8884154630 msgr2=0x7f88840c8d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:35:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888978e640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8884154630 0x7f88840c8d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f888978e640 1 -- 172.21.15.40:0/166792646 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8884151d10 con 0x7f88840c3900 2024-04-26T21:35:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f2d05431640 1 -- 172.21.15.40:0/764467019 shutdown_connections 2024-04-26T21:35:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f319184e640 1 -- 172.21.15.40:0/4272789895 wait complete. 2024-04-26T21:35:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.841+0000 7fcc35db3640 1 Processor -- start 2024-04-26T21:35:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.841+0000 7fcc35db3640 1 -- start start 2024-04-26T21:35:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.841+0000 7f580aada640 1 Processor -- start 2024-04-26T21:35:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.841+0000 7fbce0ff9640 1 -- 172.21.15.40:0/1456748343 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8017070 con 0x7fbce416ff80 2024-04-26T21:35:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.841+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce40ad830 con 0x7fbce416ff80 2024-04-26T21:35:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7f888978e640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88840c3900 0x7f88840afad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f888002a790 tx=0x7f88800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.840+0000 7f2d05431640 1 -- 172.21.15.40:0/764467019 wait complete. 2024-04-26T21:35:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7f2d05431640 1 Processor -- start 2024-04-26T21:35:36.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7f319184e640 1 Processor -- start 2024-04-26T21:35:36.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.841+0000 7f580aada640 1 -- start start 2024-04-26T21:35:36.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7f319184e640 1 -- start start 2024-04-26T21:35:36.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7fcc35db3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc28033ef0 0x7fcc2804e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7f580aada640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 0x7f5804158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7f580aada640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f580415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c400694f0 con 0x7f6c400b56c0 2024-04-26T21:35:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7fbce0ff9640 1 -- 172.21.15.40:0/1456748343 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd800c940 con 0x7fbce416ff80 2024-04-26T21:35:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce40ae0d0 con 0x7fbce416ff80 2024-04-26T21:35:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f8879ffb640 1 -- 172.21.15.40:0/166792646 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880037050 con 0x7f88840c3900 2024-04-26T21:35:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f8879ffb640 1 -- 172.21.15.40:0/166792646 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8880043070 con 0x7f88840c3900 2024-04-26T21:35:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f8879ffb640 1 -- 172.21.15.40:0/166792646 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880032440 con 0x7f88840c3900 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f319184e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3184006af0 0x7f31840e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f319184e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184049720 0x7f318417d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f319184e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31840543b0 0x7f318416fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc35db3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc28053c70 0x7fcc280c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc35db3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc280df220 0x7fcc28032050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc34db1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc28033ef0 0x7fcc2804e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc34db1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc28033ef0 0x7fcc2804e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53882/0 (socket says 172.21.15.40:53882) 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc34db1640 1 -- 172.21.15.40:0/1862168103 learned_addr learned my addr 172.21.15.40:0/1862168103 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc35db3640 1 -- 172.21.15.40:0/1862168103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28037a90 con 0x7fcc28053c70 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc35db3640 1 -- 172.21.15.40:0/1862168103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28037910 con 0x7fcc280df220 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc355b2640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc280df220 0x7fcc28032050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7fcc35db3640 1 -- 172.21.15.40:0/1862168103 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc280df600 con 0x7fcc28033ef0 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f5803fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f580415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f5803fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f580415a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45820/0 (socket says 172.21.15.40:45820) 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f5803fff640 1 -- 172.21.15.40:0/3883982652 learned_addr learned my addr 172.21.15.40:0/3883982652 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f580aada640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5804167720 0x7f580415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58040d0170 con 0x7f58040c41d0 2024-04-26T21:35:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58040cf430 con 0x7f5804167720 2024-04-26T21:35:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58040ce7b0 con 0x7f5804073f30 2024-04-26T21:35:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f5803fff640 1 -- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 msgr2=0x7f5804158370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:35:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f5803fff640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 0x7f5804158370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f5803fff640 1 -- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5804167720 msgr2=0x7f580415a6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:35:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f5803fff640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5804167720 0x7f580415a6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f5803fff640 1 -- 172.21.15.40:0/3883982652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580415c7c0 con 0x7f58040c41d0 2024-04-26T21:35:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f5803fff640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f580415a170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57f000ebe0 tx=0x7f57f000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f318bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184049720 0x7f318417d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f318bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184049720 0x7f318417d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49854/0 (socket says 172.21.15.40:49854) 2024-04-26T21:35:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f318bfff640 1 -- 172.21.15.40:0/2038128111 learned_addr learned my addr 172.21.15.40:0/2038128111 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f319104d640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31840543b0 0x7f318416fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31840558f0 con 0x7f3184006af0 2024-04-26T21:35:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31840e79f0 con 0x7f3184049720 2024-04-26T21:35:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3184054f50 con 0x7f31840543b0 2024-04-26T21:35:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f58017fa640 1 -- 172.21.15.40:0/3883982652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0017070 con 0x7f58040c41d0 2024-04-26T21:35:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f58017fa640 1 -- 172.21.15.40:0/3883982652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f000ed90 con 0x7f58040c41d0 2024-04-26T21:35:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5804166950 con 0x7f58040c41d0 2024-04-26T21:35:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f58017fa640 1 -- 172.21.15.40:0/3883982652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f000ce50 con 0x7f58040c41d0 2024-04-26T21:35:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580415ca80 con 0x7f58040c41d0 2024-04-26T21:35:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.842+0000 7f2d05431640 1 -- start start 2024-04-26T21:35:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f6c2effd640 1 -- 172.21.15.40:0/431401689 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c38044020 con 0x7f6c400b56c0 2024-04-26T21:35:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f6c2effd640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6c2006c9c0 0x7f6c2006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f6c2effd640 1 -- 172.21.15.40:0/431401689 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6c380ac9c0 con 0x7f6c400b56c0 2024-04-26T21:35:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f6c3e59c640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6c2006c9c0 0x7f6c2006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7fbce0ff9640 1 -- 172.21.15.40:0/1456748343 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800ccf0 con 0x7fbce416ff80 2024-04-26T21:35:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7fbce0ff9640 1 -- 172.21.15.40:0/1456748343 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcd801a070 con 0x7fbce416ff80 2024-04-26T21:35:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8884151ff0 con 0x7f88840c3900 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8884151a00 con 0x7f88840c3900 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f2d05431640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cf803e840 0x7f2cf800bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.843+0000 7f2d05431640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 0x7f2cf800c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f2cff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 0x7f2cf800c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f2cff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 0x7f2cf800c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45824/0 (socket says 172.21.15.40:45824) 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f2cff7fe640 1 -- 172.21.15.40:0/3736217412 learned_addr learned my addr 172.21.15.40:0/3736217412 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f2d05431640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8185770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf80e34a0 con 0x7f2cf8054130 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf803bfc0 con 0x7f2cf803e840 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf80e7170 con 0x7f2cf80e30c0 2024-04-26T21:35:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.844+0000 7f2d04c30640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8185770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f2cfffff640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cf803e840 0x7f2cf800bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f2d04c30640 1 -- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cf803e840 msgr2=0x7f2cf800bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f2d04c30640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cf803e840 0x7f2cf800bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f2d04c30640 1 -- 172.21.15.40:0/3736217412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 msgr2=0x7f2cf800c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f2d04c30640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 0x7f2cf800c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f2d04c30640 1 -- 172.21.15.40:0/3736217412 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf800c9f0 con 0x7f2cf80e30c0 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f318bfff640 1 -- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31840543b0 msgr2=0x7f318416fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f318bfff640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31840543b0 0x7f318416fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.845+0000 7f318bfff640 1 -- 172.21.15.40:0/2038128111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3184006af0 msgr2=0x7f31840e33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7f318bfff640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3184006af0 0x7f31840e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7f318bfff640 1 -- 172.21.15.40:0/2038128111 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3184046e90 con 0x7f3184049720 2024-04-26T21:35:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fbce0ff9640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcd006c940 0x7fbcd006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fcc355b2640 1 -- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc28033ef0 msgr2=0x7fcc2804e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fcc355b2640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc28033ef0 0x7fcc2804e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fcc355b2640 1 -- 172.21.15.40:0/1862168103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc28053c70 msgr2=0x7fcc280c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fcc355b2640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc28053c70 0x7fcc280c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fcc355b2640 1 -- 172.21.15.40:0/1862168103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc280e0ae0 con 0x7fcc280df220 2024-04-26T21:35:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fbce3fff640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcd006c940 0x7fbcd006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88840694f0 con 0x7f88840c3900 2024-04-26T21:35:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7f318bfff640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184049720 0x7f318417d3b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f318000d800 tx=0x7f318000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58040694f0 con 0x7f58040c41d0 2024-04-26T21:35:36.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fcc355b2640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc280df220 0x7fcc28032050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcc30069a20 tx=0x7fcc30075270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc4005290 con 0x7fbce416ff80 2024-04-26T21:35:36.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.846+0000 7f2d04c30640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8185770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d0006a760 tx=0x7f2d000661d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f2cfcff9640 1 -- 172.21.15.40:0/3736217412 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000b4040 con 0x7f2cf80e30c0 2024-04-26T21:35:36.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f2cfcff9640 1 -- 172.21.15.40:0/3736217412 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d00076450 con 0x7f2cf80e30c0 2024-04-26T21:35:36.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f2cfcff9640 1 -- 172.21.15.40:0/3736217412 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000b9480 con 0x7f2cf80e30c0 2024-04-26T21:35:36.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7fbce0ff9640 1 -- 172.21.15.40:0/1456748343 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbcd808baa0 con 0x7fbce416ff80 2024-04-26T21:35:36.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf800ccd0 con 0x7f2cf80e30c0 2024-04-26T21:35:36.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf800c710 con 0x7f2cf80e30c0 2024-04-26T21:35:36.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f6c3e59c640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6c2006c9c0 0x7f6c2006ee80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400a7d0 tx=0x7f6c34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7fbce3fff640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcd006c940 0x7fbcd006ee00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd4030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f8879ffb640 1 -- 172.21.15.40:0/166792646 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8880048460 con 0x7f88840c3900 2024-04-26T21:35:36.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f58017fa640 1 -- 172.21.15.40:0/3883982652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57f001a070 con 0x7f58040c41d0 2024-04-26T21:35:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.847+0000 7f8879ffb640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886806cb60 0x7f886806f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.848+0000 7f8879ffb640 1 -- 172.21.15.40:0/166792646 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f88800acd20 con 0x7f88840c3900 2024-04-26T21:35:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.848+0000 7f8888f8d640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886806cb60 0x7f886806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.848+0000 7f58017fa640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57e806ca90 0x7f57e806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.848+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf8007620 con 0x7f2cf80e30c0 2024-04-26T21:35:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.848+0000 7f58017fa640 1 -- 172.21.15.40:0/3883982652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f57f008c280 con 0x7f58040c41d0 2024-04-26T21:35:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.848+0000 7f5808876640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57e806ca90 0x7f57e806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.848+0000 7f8888f8d640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886806cb60 0x7f886806f020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88840b2440 tx=0x7f8874008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7f5808876640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57e806ca90 0x7f57e806ef50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f57f8034000 tx=0x7f57f802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7f31897fa640 1 -- 172.21.15.40:0/2038128111 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318000b6c0 con 0x7f3184049720 2024-04-26T21:35:36.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7fcc2d7fa640 1 -- 172.21.15.40:0/1862168103 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30094400 con 0x7fcc280df220 2024-04-26T21:35:36.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7fcc2d7fa640 1 -- 172.21.15.40:0/1862168103 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc300945a0 con 0x7fcc280df220 2024-04-26T21:35:36.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7fcc2d7fa640 1 -- 172.21.15.40:0/1862168103 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30094950 con 0x7fcc280df220 2024-04-26T21:35:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7f31897fa640 1 -- 172.21.15.40:0/2038128111 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3180019070 con 0x7f3184049720 2024-04-26T21:35:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7f31897fa640 1 -- 172.21.15.40:0/2038128111 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318000ba60 con 0x7f3184049720 2024-04-26T21:35:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318417de70 con 0x7f3184049720 2024-04-26T21:35:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3184047130 con 0x7f3184049720 2024-04-26T21:35:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7fcc35db3640 1 -- 172.21.15.40:0/1862168103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc280e0dc0 con 0x7fcc280df220 2024-04-26T21:35:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.849+0000 7fcc35db3640 1 -- 172.21.15.40:0/1862168103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc280e1380 con 0x7fcc280df220 2024-04-26T21:35:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.851+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3184048fa0 con 0x7f3184049720 2024-04-26T21:35:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.851+0000 7f6c2effd640 1 -- 172.21.15.40:0/431401689 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c3804b030 con 0x7f6c400b56c0 2024-04-26T21:35:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.852+0000 7f31897fa640 1 -- 172.21.15.40:0/2038128111 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f318000bc00 con 0x7f3184049720 2024-04-26T21:35:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.852+0000 7f31897fa640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.852+0000 7f31897fa640 1 -- 172.21.15.40:0/2038128111 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f318008c2d0 con 0x7f3184049720 2024-04-26T21:35:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.852+0000 7f319084c640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.852+0000 7fcc2d7fa640 1 -- 172.21.15.40:0/1862168103 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc3006f030 con 0x7fcc280df220 2024-04-26T21:35:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.853+0000 7fcc2d7fa640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcc1406c8f0 0x7fcc1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.853+0000 7fcc1affd640 1 -- 172.21.15.40:0/1862168103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc08005290 con 0x7fcc280df220 2024-04-26T21:35:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.853+0000 7fcc2d7fa640 1 -- 172.21.15.40:0/1862168103 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcc30108640 con 0x7fcc280df220 2024-04-26T21:35:36.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.853+0000 7f8879ffb640 1 -- 172.21.15.40:0/166792646 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f888007adf0 con 0x7f88840c3900 2024-04-26T21:35:36.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.853+0000 7f319084c640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f317806c920 0x7f317806ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f317c034000 tx=0x7f317c02c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.853+0000 7fcc34db1640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcc1406c8f0 0x7fcc1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.853+0000 7fcc34db1640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcc1406c8f0 0x7fcc1406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009280 tx=0x7fcc24008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.854+0000 7fbce0ff9640 1 -- 172.21.15.40:0/1456748343 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcd80093b0 con 0x7fbce416ff80 2024-04-26T21:35:36.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.855+0000 7f2cfcff9640 1 -- 172.21.15.40:0/3736217412 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d000b7070 con 0x7f2cf80e30c0 2024-04-26T21:35:36.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.855+0000 7f2cfcff9640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce406c9c0 0x7f2ce406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:36.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.856+0000 7f2cfcff9640 1 -- 172.21.15.40:0/3736217412 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2d001290d0 con 0x7f2cf80e30c0 2024-04-26T21:35:36.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.856+0000 7f2cfffff640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce406c9c0 0x7f2ce406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:36.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.856+0000 7f2cfffff640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce406c9c0 0x7f2ce406ee80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2cec006fd0 tx=0x7f2cec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:36.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.857+0000 7f31897fa640 1 -- 172.21.15.40:0/2038128111 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3184048fa0 con 0x7f3184049720 2024-04-26T21:35:36.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.858+0000 7f2cfcff9640 1 -- 172.21.15.40:0/3736217412 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d000bd020 con 0x7f2cf80e30c0 2024-04-26T21:35:36.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.861+0000 7f58017fa640 1 -- 172.21.15.40:0/3883982652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57f005a2d0 con 0x7f58040c41d0 2024-04-26T21:35:36.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:36.864+0000 7fcc2d7fa640 1 -- 172.21.15.40:0/1862168103 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcc30090030 con 0x7fcc280df220 2024-04-26T21:35:37.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.210+0000 7fbdee351640 1 -- 172.21.15.40:0/3042690873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fbdc8005580 con 0x7fbde80b5a50 2024-04-26T21:35:37.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.211+0000 7fbde4ff9640 1 -- 172.21.15.40:0/3042690873 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbdd805a060 con 0x7fbde80b5a50 2024-04-26T21:35:37.212 INFO:teuthology.orchestra.run.smithi040.stdout:55834574857 2024-04-26T21:35:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.214+0000 7fbdce7fc640 1 -- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdd406c9c0 msgr2=0x7fbdd406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.214+0000 7fbdce7fc640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdd406c9c0 0x7fbdd406ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbddc055000 tx=0x7fbddc009030 comp rx=0 tx=0).stop 2024-04-26T21:35:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 -- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde80b5a50 msgr2=0x7fbde80b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde80b5a50 0x7fbde80b5eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800a840 tx=0x7fbdd800ad00 comp rx=0 tx=0).stop 2024-04-26T21:35:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 -- 172.21.15.40:0/3042690873 shutdown_connections 2024-04-26T21:35:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdd406c9c0 0x7fbdd406ee80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde80ae1e0 0x7fbde80ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde80b5a50 0x7fbde80b5eb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 --2- 172.21.15.40:0/3042690873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd0007dc0 0x7fbde80af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 -- 172.21.15.40:0/3042690873 >> 172.21.15.40:0/3042690873 conn(0x7fbde80d0be0 msgr2=0x7fbde806f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 -- 172.21.15.40:0/3042690873 shutdown_connections 2024-04-26T21:35:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.215+0000 7fbdce7fc640 1 -- 172.21.15.40:0/3042690873 wait complete. 2024-04-26T21:35:37.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-04-26T21:35:37.233 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:37.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.258+0000 7fcc1affd640 1 -- 172.21.15.40:0/1862168103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcc08005580 con 0x7fcc280df220 2024-04-26T21:35:37.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.258+0000 7fcc2d7fa640 1 -- 172.21.15.40:0/1862168103 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcc300d6690 con 0x7fcc280df220 2024-04-26T21:35:37.260 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:37.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.260+0000 7fcc1affd640 1 -- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcc1406c8f0 msgr2=0x7fcc1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.260+0000 7fcc1affd640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcc1406c8f0 0x7fcc1406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009280 tx=0x7fcc24008030 comp rx=0 tx=0).stop 2024-04-26T21:35:37.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.260+0000 7fcc1affd640 1 -- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc280df220 msgr2=0x7fcc28032050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.260+0000 7fcc1affd640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc280df220 0x7fcc28032050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcc30069a20 tx=0x7fcc30075270 comp rx=0 tx=0).stop 2024-04-26T21:35:37.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.261+0000 7fcc1affd640 1 -- 172.21.15.40:0/1862168103 shutdown_connections 2024-04-26T21:35:37.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.261+0000 7fcc1affd640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcc1406c8f0 0x7fcc1406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.261+0000 7fcc1affd640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc280df220 0x7fcc28032050 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.261+0000 7fcc1affd640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc28053c70 0x7fcc280c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.261+0000 7fcc1affd640 1 --2- 172.21.15.40:0/1862168103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc28033ef0 0x7fcc2804e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.261+0000 7fcc1affd640 1 -- 172.21.15.40:0/1862168103 >> 172.21.15.40:0/1862168103 conn(0x7fcc28009fb0 msgr2=0x7fcc280dccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.261+0000 7fcc1affd640 1 -- 172.21.15.40:0/1862168103 shutdown_connections 2024-04-26T21:35:37.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.261+0000 7fcc1affd640 1 -- 172.21.15.40:0/1862168103 wait complete. 2024-04-26T21:35:37.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-26T21:35:37.273 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:37.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.288+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6c4015eb80 con 0x7f6c400b56c0 2024-04-26T21:35:37.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.289+0000 7f6c2effd640 1 -- 172.21.15.40:0/431401689 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6c3807ab50 con 0x7f6c400b56c0 2024-04-26T21:35:37.290 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6c2006c9c0 msgr2=0x7f6c2006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6c2006c9c0 0x7f6c2006ee80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400a7d0 tx=0x7f6c34008040 comp rx=0 tx=0).stop 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c400b56c0 msgr2=0x7f6c4014c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c400b56c0 0x7f6c4014c6a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009f70 tx=0x7f6c38004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 shutdown_connections 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6c2006c9c0 0x7f6c2006ee80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c400b56c0 0x7f6c4014c6a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c40072fb0 0x7f6c400d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 --2- 172.21.15.40:0/431401689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40070750 0x7f6c401fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.291+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 >> 172.21.15.40:0/431401689 conn(0x7f6c400b39b0 msgr2=0x7f6c4015e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.292+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 shutdown_connections 2024-04-26T21:35:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.292+0000 7f6c44a52640 1 -- 172.21.15.40:0/431401689 wait complete. 2024-04-26T21:35:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.293+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2cf800cea0 con 0x7f2cf80e30c0 2024-04-26T21:35:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.294+0000 7f2cfcff9640 1 -- 172.21.15.40:0/3736217412 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2d000f72c0 con 0x7f2cf80e30c0 2024-04-26T21:35:37.295 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:37.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.296+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce406c9c0 msgr2=0x7f2ce406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.296+0000 7f2d05431640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce406c9c0 0x7f2ce406ee80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2cec006fd0 tx=0x7f2cec008040 comp rx=0 tx=0).stop 2024-04-26T21:35:37.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.296+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 msgr2=0x7f2cf8185770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.296+0000 7f2d05431640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8185770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d0006a760 tx=0x7f2d000661d0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.296+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbcc4005850 con 0x7fbce416ff80 2024-04-26T21:35:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 shutdown_connections 2024-04-26T21:35:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f2d05431640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce406c9c0 0x7f2ce406ee80 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2cec006fd0 tx=0x7f2cec008040 comp rx=0 tx=0).stop 2024-04-26T21:35:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f2d05431640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cf80e30c0 0x7f2cf8185770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f2d05431640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cf8054130 0x7f2cf800c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f2d05431640 1 --2- 172.21.15.40:0/3736217412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cf803e840 0x7f2cf800bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 >> 172.21.15.40:0/3736217412 conn(0x7f2cf80559e0 msgr2=0x7f2cf8054ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7fbce0ff9640 1 -- 172.21.15.40:0/1456748343 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbcd8059720 con 0x7fbce416ff80 2024-04-26T21:35:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 shutdown_connections 2024-04-26T21:35:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f2d05431640 1 -- 172.21.15.40:0/3736217412 wait complete. 2024-04-26T21:35:37.300 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.297+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f88840b13c0 con 0x7f88840c3900 2024-04-26T21:35:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.298+0000 7f8879ffb640 1 -- 172.21.15.40:0/166792646 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f888007a6f0 con 0x7f88840c3900 2024-04-26T21:35:37.301 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:37.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.299+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f318417e390 con 0x7f3184049720 2024-04-26T21:35:37.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.299+0000 7f31897fa640 1 -- 172.21.15.40:0/2038128111 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f318005a320 con 0x7f3184049720 2024-04-26T21:35:37.302 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcd006c940 msgr2=0x7fbcd006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcd006c940 0x7fbcd006ee00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd4030000 comp rx=0 tx=0).stop 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbce416ff80 msgr2=0x7fbce40ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbce416ff80 0x7fbce40ce1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800e7e0 tx=0x7fbcd800eca0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 shutdown_connections 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcd006c940 0x7fbcd006ee00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbce416ff80 0x7fbce40ce1a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbce416f660 0x7fbce406ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 --2- 172.21.15.40:0/1456748343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbce40b4ee0 0x7fbce40aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.300+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 >> 172.21.15.40:0/1456748343 conn(0x7fbce40b2720 msgr2=0x7fbce406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 shutdown_connections 2024-04-26T21:35:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7fbce95b5640 1 -- 172.21.15.40:0/1456748343 wait complete. 2024-04-26T21:35:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886806cb60 msgr2=0x7f886806f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886806cb60 0x7f886806f020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88840b2440 tx=0x7f8874008040 comp rx=0 tx=0).stop 2024-04-26T21:35:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88840c3900 msgr2=0x7f88840afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88840c3900 0x7f88840afad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f888002a790 tx=0x7f88800046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 shutdown_connections 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886806cb60 0x7f886806f020 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8884154630 0x7f88840c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88840c3e50 0x7f88841514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 --2- 172.21.15.40:0/166792646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88840c3900 0x7f88840afad0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 >> 172.21.15.40:0/166792646 conn(0x7f88840d0be0 msgr2=0x7f8884166d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 shutdown_connections 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.301+0000 7f888b9f2640 1 -- 172.21.15.40:0/166792646 wait complete. 2024-04-26T21:35:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.302+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f58040c4a50 con 0x7f58040c41d0 2024-04-26T21:35:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f317806c920 msgr2=0x7f317806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f317806c920 0x7f317806ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f317c034000 tx=0x7f317c02c000 comp rx=0 tx=0).stop 2024-04-26T21:35:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184049720 msgr2=0x7f318417d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184049720 0x7f318417d3b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f318000d800 tx=0x7f318000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f58017fa640 1 -- 172.21.15.40:0/3883982652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f57f0059bd0 con 0x7f58040c41d0 2024-04-26T21:35:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 shutdown_connections 2024-04-26T21:35:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31840543b0 0x7f318416fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184049720 0x7f318417d3b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 --2- 172.21.15.40:0/2038128111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3184006af0 0x7f31840e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 >> 172.21.15.40:0/2038128111 conn(0x7f3184055bd0 msgr2=0x7f318404b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.307 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2024-04-26T21:35:37.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 shutdown_connections 2024-04-26T21:35:37.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.303+0000 7f319184e640 1 -- 172.21.15.40:0/2038128111 wait complete. 2024-04-26T21:35:37.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.306+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57e806ca90 msgr2=0x7f57e806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.306+0000 7f580aada640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57e806ca90 0x7f57e806ef50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f57f8034000 tx=0x7f57f802e000 comp rx=0 tx=0).stop 2024-04-26T21:35:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.306+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 msgr2=0x7f580415a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.306+0000 7f580aada640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f580415a170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57f000ebe0 tx=0x7f57f000c520 comp rx=0 tx=0).stop 2024-04-26T21:35:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.307+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 shutdown_connections 2024-04-26T21:35:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.307+0000 7f580aada640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57e806ca90 0x7f57e806ef50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.307+0000 7f580aada640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5804167720 0x7f580415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.307+0000 7f580aada640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58040c41d0 0x7f580415a170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-04-26T21:35:37.310 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.307+0000 7f580aada640 1 --2- 172.21.15.40:0/3883982652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5804073f30 0x7f5804158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.307+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 >> 172.21.15.40:0/3883982652 conn(0x7f58040d0be0 msgr2=0x7f580406ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.307+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 shutdown_connections 2024-04-26T21:35:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.307+0000 7f580aada640 1 -- 172.21.15.40:0/3883982652 wait complete. 2024-04-26T21:35:37.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-04-26T21:35:37.316 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:37.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-26T21:35:37.320 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:37.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-26T21:35:37.321 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:37.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-26T21:35:37.322 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:37.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-04-26T21:35:37.325 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:37.325 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:35:37.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:35:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.464+0000 7fb163c93640 1 Processor -- start 2024-04-26T21:35:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.465+0000 7fb163c93640 1 -- start start 2024-04-26T21:35:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.465+0000 7fb163c93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb15c15f5a0 0x7fb15c0d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.465+0000 7fb163c93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb15c0d2830 0x7fb15c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.465+0000 7fb163c93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb15c0d1a70 con 0x7fb15c0d3c20 2024-04-26T21:35:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.465+0000 7fb163c93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb15c163d30 con 0x7fb15c15f5a0 2024-04-26T21:35:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.465+0000 7fb163c93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb15c0d1d50 con 0x7fb15c0d2830 2024-04-26T21:35:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb161a2f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb15c15f5a0 0x7fb15c0d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb162230640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb15c0d3c20 0x7fb15c15f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40250/0 (socket says 172.21.15.40:40250) 2024-04-26T21:35:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb161a2f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb15c15f5a0 0x7fb15c0d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45848/0 (socket says 172.21.15.40:45848) 2024-04-26T21:35:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb162230640 1 -- 172.21.15.40:0/118758765 learned_addr learned my addr 172.21.15.40:0/118758765 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb16122e640 1 --2- 172.21.15.40:0/118758765 >> v2:172.21.15.184:3300/0 conn(0x7fb15c0d2830 0x7fb15c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb161a2f640 1 -- 172.21.15.40:0/118758765 >> v2:172.21.15.184:3300/0 conn(0x7fb15c0d2830 msgr2=0x7fb15c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb161a2f640 1 --2- 172.21.15.40:0/118758765 >> v2:172.21.15.184:3300/0 conn(0x7fb15c0d2830 0x7fb15c162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb161a2f640 1 -- 172.21.15.40:0/118758765 >> v1:172.21.15.40:6789/0 conn(0x7fb15c0d3c20 legacy=0x7fb15c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.466+0000 7fb161a2f640 1 -- 172.21.15.40:0/118758765 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15c0d2030 con 0x7fb15c15f5a0 2024-04-26T21:35:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb161a2f640 1 --2- 172.21.15.40:0/118758765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c15f5a0 0x7fb15c0d5cb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb15c08d2c0 tx=0x7fb15402a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc264e327d403ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb160a2d640 1 -- 172.21.15.40:0/118758765 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154037070 con 0x7fb15c15f5a0 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb160a2d640 1 -- 172.21.15.40:0/118758765 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15402aca0 con 0x7fb15c15f5a0 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb160a2d640 1 -- 172.21.15.40:0/118758765 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154032660 con 0x7fb15c15f5a0 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb163c93640 1 -- 172.21.15.40:0/118758765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c15f5a0 msgr2=0x7fb15c0d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb163c93640 1 --2- 172.21.15.40:0/118758765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c15f5a0 0x7fb15c0d5cb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb15c08d2c0 tx=0x7fb15402a010 comp rx=0 tx=0).stop 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb163c93640 1 -- 172.21.15.40:0/118758765 shutdown_connections 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb163c93640 1 --2- 172.21.15.40:0/118758765 >> v2:172.21.15.184:3300/0 conn(0x7fb15c0d2830 0x7fb15c162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb163c93640 1 --2- 172.21.15.40:0/118758765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c15f5a0 0x7fb15c0d5cb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb163c93640 1 -- 172.21.15.40:0/118758765 >> 172.21.15.40:0/118758765 conn(0x7fb15c0d5190 msgr2=0x7fb15c0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb163c93640 1 -- 172.21.15.40:0/118758765 shutdown_connections 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.467+0000 7fb163c93640 1 -- 172.21.15.40:0/118758765 wait complete. 2024-04-26T21:35:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb163c93640 1 Processor -- start 2024-04-26T21:35:37.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb163c93640 1 -- start start 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb163c93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c0d2830 0x7fb15c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb163c93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb15c0d3c20 0x7fb15c1614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb163c93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15c161a10 0x7fb15c0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb163c93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb15c163440 con 0x7fb15c0d2830 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb163c93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb15c0d4190 con 0x7fb15c0d3c20 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb163c93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb15c1635c0 con 0x7fb15c161a10 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb161a2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c0d2830 0x7fb15c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb161a2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c0d2830 0x7fb15c150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45864/0 (socket says 172.21.15.40:45864) 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.469+0000 7fb161a2f640 1 -- 172.21.15.40:0/2323569873 learned_addr learned my addr 172.21.15.40:0/2323569873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb16122e640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb15c0d3c20 0x7fb15c1614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb162230640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15c161a10 0x7fb15c0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb161a2f640 1 -- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15c161a10 msgr2=0x7fb15c0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb161a2f640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15c161a10 0x7fb15c0c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb161a2f640 1 -- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb15c0d3c20 msgr2=0x7fb15c1614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb161a2f640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb15c0d3c20 0x7fb15c1614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb161a2f640 1 -- 172.21.15.40:0/2323569873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15c0c9a60 con 0x7fb15c0d2830 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb16122e640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb15c0d3c20 0x7fb15c1614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb162230640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15c161a10 0x7fb15c0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb161a2f640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c0d2830 0x7fb15c150550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb154000c00 tx=0x7fb1540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb15a7fc640 1 -- 172.21.15.40:0/2323569873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154037040 con 0x7fb15c0d2830 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb15a7fc640 1 -- 172.21.15.40:0/2323569873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15402ae50 con 0x7fb15c0d2830 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15c0c9d40 con 0x7fb15c0d2830 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb15a7fc640 1 -- 172.21.15.40:0/2323569873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154004430 con 0x7fb15c0d2830 2024-04-26T21:35:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.470+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb15c0ca300 con 0x7fb15c0d2830 2024-04-26T21:35:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.471+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb15c161e20 con 0x7fb15c0d2830 2024-04-26T21:35:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.472+0000 7fb15a7fc640 1 -- 172.21.15.40:0/2323569873 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb15402ab60 con 0x7fb15c0d2830 2024-04-26T21:35:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.472+0000 7fb15a7fc640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.472+0000 7fb15a7fc640 1 -- 172.21.15.40:0/2323569873 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb1540ac6e0 con 0x7fb15c0d2830 2024-04-26T21:35:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.472+0000 7fb16122e640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.473+0000 7fb16122e640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb14806c8f0 0x7fb14806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0ba820 tx=0x7fb150008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:37.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.477+0000 7fb15a7fc640 1 -- 172.21.15.40:0/2323569873 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb15c161e20 con 0x7fb15c0d2830 2024-04-26T21:35:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.656+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb15c068d30 con 0x7fb14806c8f0 2024-04-26T21:35:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.665+0000 7fb15a7fc640 1 -- 172.21.15.40:0/2323569873 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7fb15c068d30 con 0x7fb14806c8f0 2024-04-26T21:35:37.670 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:35:37.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb14806c8f0 msgr2=0x7fb14806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb14806c8f0 0x7fb14806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0ba820 tx=0x7fb150008040 comp rx=0 tx=0).stop 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c0d2830 msgr2=0x7fb15c150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c0d2830 0x7fb15c150550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb154000c00 tx=0x7fb1540046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 shutdown_connections 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15c161a10 0x7fb15c0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb15c0d3c20 0x7fb15c1614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 --2- 172.21.15.40:0/2323569873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c0d2830 0x7fb15c150550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 >> 172.21.15.40:0/2323569873 conn(0x7fb15c0d5190 msgr2=0x7fb15c15d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 shutdown_connections 2024-04-26T21:35:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.668+0000 7fb163c93640 1 -- 172.21.15.40:0/2323569873 wait complete. 2024-04-26T21:35:37.680 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-26T21:35:35.999848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219400,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755320,"statfs":{"total":773094113280,"available":772869447680,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003455"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004303+0000","last_change":"2024-04-26T21:35:09.132596+0000","last_active":"2024-04-26T21:35:16.004303+0000","last_peered":"2024-04-26T21:35:16.004303+0000","last_clean":"2024-04-26T21:35:16.004303+0000","last_became_active":"2024-04-26T21:35:09.132270+0000","last_became_peered":"2024-04-26T21:35:09.132270+0000","last_unstale":"2024-04-26T21:35:16.004303+0000","last_undegraded":"2024-04-26T21:35:16.004303+0000","last_fullsized":"2024-04-26T21:35:16.004303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:22.652595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003605+0000","last_change":"2024-04-26T21:35:10.126640+0000","last_active":"2024-04-26T21:35:16.003605+0000","last_peered":"2024-04-26T21:35:16.003605+0000","last_clean":"2024-04-26T21:35:16.003605+0000","last_became_active":"2024-04-26T21:35:10.125603+0000","last_became_peered":"2024-04-26T21:35:10.125603+0000","last_unstale":"2024-04-26T21:35:16.003605+0000","last_undegraded":"2024-04-26T21:35:16.003605+0000","last_fullsized":"2024-04-26T21:35:16.003605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:27.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004487+0000","last_change":"2024-04-26T21:35:14.280943+0000","last_active":"2024-04-26T21:35:16.004487+0000","last_peered":"2024-04-26T21:35:16.004487+0000","last_clean":"2024-04-26T21:35:16.004487+0000","last_became_active":"2024-04-26T21:35:14.280611+0000","last_became_peered":"2024-04-26T21:35:14.280611+0000","last_unstale":"2024-04-26T21:35:16.004487+0000","last_undegraded":"2024-04-26T21:35:16.004487+0000","last_fullsized":"2024-04-26T21:35:16.004487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:28.734397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004488+0000","last_change":"2024-04-26T21:35:09.122996+0000","last_active":"2024-04-26T21:35:16.004488+0000","last_peered":"2024-04-26T21:35:16.004488+0000","last_clean":"2024-04-26T21:35:16.004488+0000","last_became_active":"2024-04-26T21:35:09.122682+0000","last_became_peered":"2024-04-26T21:35:09.122682+0000","last_unstale":"2024-04-26T21:35:16.004488+0000","last_undegraded":"2024-04-26T21:35:16.004488+0000","last_fullsized":"2024-04-26T21:35:16.004488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:44.358668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004023+0000","last_change":"2024-04-26T21:35:10.133464+0000","last_active":"2024-04-26T21:35:16.004023+0000","last_peered":"2024-04-26T21:35:16.004023+0000","last_clean":"2024-04-26T21:35:16.004023+0000","last_became_active":"2024-04-26T21:35:10.133170+0000","last_became_peered":"2024-04-26T21:35:10.133170+0000","last_unstale":"2024-04-26T21:35:16.004023+0000","last_undegraded":"2024-04-26T21:35:16.004023+0000","last_fullsized":"2024-04-26T21:35:16.004023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003606+0000","last_change":"2024-04-26T21:35:14.163746+0000","last_active":"2024-04-26T21:35:16.003606+0000","last_peered":"2024-04-26T21:35:16.003606+0000","last_clean":"2024-04-26T21:35:16.003606+0000","last_became_active":"2024-04-26T21:35:14.163263+0000","last_became_peered":"2024-04-26T21:35:14.163263+0000","last_unstale":"2024-04-26T21:35:16.003606+0000","last_undegraded":"2024-04-26T21:35:16.003606+0000","last_fullsized":"2024-04-26T21:35:16.003606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:28.472320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875887+0000","last_change":"2024-04-26T21:35:09.129656+0000","last_active":"2024-04-26T21:35:15.875887+0000","last_peered":"2024-04-26T21:35:15.875887+0000","last_clean":"2024-04-26T21:35:15.875887+0000","last_became_active":"2024-04-26T21:35:09.129379+0000","last_became_peered":"2024-04-26T21:35:09.129379+0000","last_unstale":"2024-04-26T21:35:15.875887+0000","last_undegraded":"2024-04-26T21:35:15.875887+0000","last_fullsized":"2024-04-26T21:35:15.875887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:51.639722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143824+0000","last_change":"2024-04-26T21:35:10.119557+0000","last_active":"2024-04-26T21:35:15.143824+0000","last_peered":"2024-04-26T21:35:15.143824+0000","last_clean":"2024-04-26T21:35:15.143824+0000","last_became_active":"2024-04-26T21:35:10.118407+0000","last_became_peered":"2024-04-26T21:35:10.118407+0000","last_unstale":"2024-04-26T21:35:15.143824+0000","last_undegraded":"2024-04-26T21:35:15.143824+0000","last_fullsized":"2024-04-26T21:35:15.143824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003821+0000","last_change":"2024-04-26T21:35:14.159166+0000","last_active":"2024-04-26T21:35:16.003821+0000","last_peered":"2024-04-26T21:35:16.003821+0000","last_clean":"2024-04-26T21:35:16.003821+0000","last_became_active":"2024-04-26T21:35:14.158592+0000","last_became_peered":"2024-04-26T21:35:14.158592+0000","last_unstale":"2024-04-26T21:35:16.003821+0000","last_undegraded":"2024-04-26T21:35:16.003821+0000","last_fullsized":"2024-04-26T21:35:16.003821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:31.201343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875768+0000","last_change":"2024-04-26T21:35:09.128770+0000","last_active":"2024-04-26T21:35:15.875768+0000","last_peered":"2024-04-26T21:35:15.875768+0000","last_clean":"2024-04-26T21:35:15.875768+0000","last_became_active":"2024-04-26T21:35:09.128385+0000","last_became_peered":"2024-04-26T21:35:09.128385+0000","last_unstale":"2024-04-26T21:35:15.875768+0000","last_undegraded":"2024-04-26T21:35:15.875768+0000","last_fullsized":"2024-04-26T21:35:15.875768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:20.201305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144355+0000","last_change":"2024-04-26T21:35:10.117537+0000","last_active":"2024-04-26T21:35:15.144355+0000","last_peered":"2024-04-26T21:35:15.144355+0000","last_clean":"2024-04-26T21:35:15.144355+0000","last_became_active":"2024-04-26T21:35:10.117267+0000","last_became_peered":"2024-04-26T21:35:10.117267+0000","last_unstale":"2024-04-26T21:35:15.144355+0000","last_undegraded":"2024-04-26T21:35:15.144355+0000","last_fullsized":"2024-04-26T21:35:15.144355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:01.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003840+0000","last_change":"2024-04-26T21:35:14.161136+0000","last_active":"2024-04-26T21:35:16.003840+0000","last_peered":"2024-04-26T21:35:16.003840+0000","last_clean":"2024-04-26T21:35:16.003840+0000","last_became_active":"2024-04-26T21:35:14.160648+0000","last_became_peered":"2024-04-26T21:35:14.160648+0000","last_unstale":"2024-04-26T21:35:16.003840+0000","last_undegraded":"2024-04-26T21:35:16.003840+0000","last_fullsized":"2024-04-26T21:35:16.003840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:10.381659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143344+0000","last_change":"2024-04-26T21:35:10.119355+0000","last_active":"2024-04-26T21:35:15.143344+0000","last_peered":"2024-04-26T21:35:15.143344+0000","last_clean":"2024-04-26T21:35:15.143344+0000","last_became_active":"2024-04-26T21:35:10.119067+0000","last_became_peered":"2024-04-26T21:35:10.119067+0000","last_unstale":"2024-04-26T21:35:15.143344+0000","last_undegraded":"2024-04-26T21:35:15.143344+0000","last_fullsized":"2024-04-26T21:35:15.143344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876041+0000","last_change":"2024-04-26T21:35:09.129853+0000","last_active":"2024-04-26T21:35:15.876041+0000","last_peered":"2024-04-26T21:35:15.876041+0000","last_clean":"2024-04-26T21:35:15.876041+0000","last_became_active":"2024-04-26T21:35:09.129500+0000","last_became_peered":"2024-04-26T21:35:09.129500+0000","last_unstale":"2024-04-26T21:35:15.876041+0000","last_undegraded":"2024-04-26T21:35:15.876041+0000","last_fullsized":"2024-04-26T21:35:15.876041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:54.605573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004307+0000","last_change":"2024-04-26T21:35:14.161264+0000","last_active":"2024-04-26T21:35:16.004307+0000","last_peered":"2024-04-26T21:35:16.004307+0000","last_clean":"2024-04-26T21:35:16.004307+0000","last_became_active":"2024-04-26T21:35:14.160617+0000","last_became_peered":"2024-04-26T21:35:14.160617+0000","last_unstale":"2024-04-26T21:35:16.004307+0000","last_undegraded":"2024-04-26T21:35:16.004307+0000","last_fullsized":"2024-04-26T21:35:16.004307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004050+0000","last_change":"2024-04-26T21:35:10.125998+0000","last_active":"2024-04-26T21:35:16.004050+0000","last_peered":"2024-04-26T21:35:16.004050+0000","last_clean":"2024-04-26T21:35:16.004050+0000","last_became_active":"2024-04-26T21:35:10.125704+0000","last_became_peered":"2024-04-26T21:35:10.125704+0000","last_unstale":"2024-04-26T21:35:16.004050+0000","last_undegraded":"2024-04-26T21:35:16.004050+0000","last_fullsized":"2024-04-26T21:35:16.004050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373953+0000","last_change":"2024-04-26T21:35:09.123003+0000","last_active":"2024-04-26T21:35:15.373953+0000","last_peered":"2024-04-26T21:35:15.373953+0000","last_clean":"2024-04-26T21:35:15.373953+0000","last_became_active":"2024-04-26T21:35:09.122610+0000","last_became_peered":"2024-04-26T21:35:09.122610+0000","last_unstale":"2024-04-26T21:35:15.373953+0000","last_undegraded":"2024-04-26T21:35:15.373953+0000","last_fullsized":"2024-04-26T21:35:15.373953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004080+0000","last_change":"2024-04-26T21:35:14.164010+0000","last_active":"2024-04-26T21:35:16.004080+0000","last_peered":"2024-04-26T21:35:16.004080+0000","last_clean":"2024-04-26T21:35:16.004080+0000","last_became_active":"2024-04-26T21:35:14.163262+0000","last_became_peered":"2024-04-26T21:35:14.163262+0000","last_unstale":"2024-04-26T21:35:16.004080+0000","last_undegraded":"2024-04-26T21:35:16.004080+0000","last_fullsized":"2024-04-26T21:35:16.004080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:45.829307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003626+0000","last_change":"2024-04-26T21:35:10.120565+0000","last_active":"2024-04-26T21:35:16.003626+0000","last_peered":"2024-04-26T21:35:16.003626+0000","last_clean":"2024-04-26T21:35:16.003626+0000","last_became_active":"2024-04-26T21:35:10.120133+0000","last_became_peered":"2024-04-26T21:35:10.120133+0000","last_unstale":"2024-04-26T21:35:16.003626+0000","last_undegraded":"2024-04-26T21:35:16.003626+0000","last_fullsized":"2024-04-26T21:35:16.003626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:06.354261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373691+0000","last_change":"2024-04-26T21:35:09.127198+0000","last_active":"2024-04-26T21:35:15.373691+0000","last_peered":"2024-04-26T21:35:15.373691+0000","last_clean":"2024-04-26T21:35:15.373691+0000","last_became_active":"2024-04-26T21:35:09.126870+0000","last_became_peered":"2024-04-26T21:35:09.126870+0000","last_unstale":"2024-04-26T21:35:15.373691+0000","last_undegraded":"2024-04-26T21:35:15.373691+0000","last_fullsized":"2024-04-26T21:35:15.373691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374142+0000","last_change":"2024-04-26T21:35:14.277851+0000","last_active":"2024-04-26T21:35:15.374142+0000","last_peered":"2024-04-26T21:35:15.374142+0000","last_clean":"2024-04-26T21:35:15.374142+0000","last_became_active":"2024-04-26T21:35:14.277461+0000","last_became_peered":"2024-04-26T21:35:14.277461+0000","last_unstale":"2024-04-26T21:35:15.374142+0000","last_undegraded":"2024-04-26T21:35:15.374142+0000","last_fullsized":"2024-04-26T21:35:15.374142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:20.536176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004022+0000","last_change":"2024-04-26T21:35:10.113247+0000","last_active":"2024-04-26T21:35:16.004022+0000","last_peered":"2024-04-26T21:35:16.004022+0000","last_clean":"2024-04-26T21:35:16.004022+0000","last_became_active":"2024-04-26T21:35:10.112937+0000","last_became_peered":"2024-04-26T21:35:10.112937+0000","last_unstale":"2024-04-26T21:35:16.004022+0000","last_undegraded":"2024-04-26T21:35:16.004022+0000","last_fullsized":"2024-04-26T21:35:16.004022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:47.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875388+0000","last_change":"2024-04-26T21:35:09.134099+0000","last_active":"2024-04-26T21:35:15.875388+0000","last_peered":"2024-04-26T21:35:15.875388+0000","last_clean":"2024-04-26T21:35:15.875388+0000","last_became_active":"2024-04-26T21:35:09.133869+0000","last_became_peered":"2024-04-26T21:35:09.133869+0000","last_unstale":"2024-04-26T21:35:15.875388+0000","last_undegraded":"2024-04-26T21:35:15.875388+0000","last_fullsized":"2024-04-26T21:35:15.875388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:16.219471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004280+0000","last_change":"2024-04-26T21:35:14.158160+0000","last_active":"2024-04-26T21:35:16.004280+0000","last_peered":"2024-04-26T21:35:16.004280+0000","last_clean":"2024-04-26T21:35:16.004280+0000","last_became_active":"2024-04-26T21:35:14.157711+0000","last_became_peered":"2024-04-26T21:35:14.157711+0000","last_unstale":"2024-04-26T21:35:16.004280+0000","last_undegraded":"2024-04-26T21:35:16.004280+0000","last_fullsized":"2024-04-26T21:35:16.004280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:26.153246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004196+0000","last_change":"2024-04-26T21:35:10.136384+0000","last_active":"2024-04-26T21:35:16.004196+0000","last_peered":"2024-04-26T21:35:16.004196+0000","last_clean":"2024-04-26T21:35:16.004196+0000","last_became_active":"2024-04-26T21:35:10.135589+0000","last_became_peered":"2024-04-26T21:35:10.135589+0000","last_unstale":"2024-04-26T21:35:16.004196+0000","last_undegraded":"2024-04-26T21:35:16.004196+0000","last_fullsized":"2024-04-26T21:35:16.004196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144087+0000","last_change":"2024-04-26T21:35:09.125486+0000","last_active":"2024-04-26T21:35:15.144087+0000","last_peered":"2024-04-26T21:35:15.144087+0000","last_clean":"2024-04-26T21:35:15.144087+0000","last_became_active":"2024-04-26T21:35:09.125019+0000","last_became_peered":"2024-04-26T21:35:09.125019+0000","last_unstale":"2024-04-26T21:35:15.144087+0000","last_undegraded":"2024-04-26T21:35:15.144087+0000","last_fullsized":"2024-04-26T21:35:15.144087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003847+0000","last_change":"2024-04-26T21:35:14.281054+0000","last_active":"2024-04-26T21:35:16.003847+0000","last_peered":"2024-04-26T21:35:16.003847+0000","last_clean":"2024-04-26T21:35:16.003847+0000","last_became_active":"2024-04-26T21:35:14.278082+0000","last_became_peered":"2024-04-26T21:35:14.278082+0000","last_unstale":"2024-04-26T21:35:16.003847+0000","last_undegraded":"2024-04-26T21:35:16.003847+0000","last_fullsized":"2024-04-26T21:35:16.003847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.959963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004866+0000","last_change":"2024-04-26T21:35:10.130908+0000","last_active":"2024-04-26T21:35:16.004866+0000","last_peered":"2024-04-26T21:35:16.004866+0000","last_clean":"2024-04-26T21:35:16.004866+0000","last_became_active":"2024-04-26T21:35:10.130448+0000","last_became_peered":"2024-04-26T21:35:10.130448+0000","last_unstale":"2024-04-26T21:35:16.004866+0000","last_undegraded":"2024-04-26T21:35:16.004866+0000","last_fullsized":"2024-04-26T21:35:16.004866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374192+0000","last_change":"2024-04-26T21:35:09.124758+0000","last_active":"2024-04-26T21:35:15.374192+0000","last_peered":"2024-04-26T21:35:15.374192+0000","last_clean":"2024-04-26T21:35:15.374192+0000","last_became_active":"2024-04-26T21:35:09.124446+0000","last_became_peered":"2024-04-26T21:35:09.124446+0000","last_unstale":"2024-04-26T21:35:15.374192+0000","last_undegraded":"2024-04-26T21:35:15.374192+0000","last_fullsized":"2024-04-26T21:35:15.374192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:16.219471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003317+0000","last_change":"2024-04-26T21:35:14.280487+0000","last_active":"2024-04-26T21:35:16.003317+0000","last_peered":"2024-04-26T21:35:16.003317+0000","last_clean":"2024-04-26T21:35:16.003317+0000","last_became_active":"2024-04-26T21:35:14.280075+0000","last_became_peered":"2024-04-26T21:35:14.280075+0000","last_unstale":"2024-04-26T21:35:16.003317+0000","last_undegraded":"2024-04-26T21:35:16.003317+0000","last_fullsized":"2024-04-26T21:35:16.003317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:49.955325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373962+0000","last_change":"2024-04-26T21:35:10.124638+0000","last_active":"2024-04-26T21:35:15.373962+0000","last_peered":"2024-04-26T21:35:15.373962+0000","last_clean":"2024-04-26T21:35:15.373962+0000","last_became_active":"2024-04-26T21:35:10.124298+0000","last_became_peered":"2024-04-26T21:35:10.124298+0000","last_unstale":"2024-04-26T21:35:15.373962+0000","last_undegraded":"2024-04-26T21:35:15.373962+0000","last_fullsized":"2024-04-26T21:35:15.373962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876132+0000","last_change":"2024-04-26T21:35:09.134096+0000","last_active":"2024-04-26T21:35:15.876132+0000","last_peered":"2024-04-26T21:35:15.876132+0000","last_clean":"2024-04-26T21:35:15.876132+0000","last_became_active":"2024-04-26T21:35:09.133816+0000","last_became_peered":"2024-04-26T21:35:09.133816+0000","last_unstale":"2024-04-26T21:35:15.876132+0000","last_undegraded":"2024-04-26T21:35:15.876132+0000","last_fullsized":"2024-04-26T21:35:15.876132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:39.887861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004297+0000","last_change":"2024-04-26T21:35:14.277888+0000","last_active":"2024-04-26T21:35:16.004297+0000","last_peered":"2024-04-26T21:35:16.004297+0000","last_clean":"2024-04-26T21:35:16.004297+0000","last_became_active":"2024-04-26T21:35:14.277473+0000","last_became_peered":"2024-04-26T21:35:14.277473+0000","last_unstale":"2024-04-26T21:35:16.004297+0000","last_undegraded":"2024-04-26T21:35:16.004297+0000","last_fullsized":"2024-04-26T21:35:16.004297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:20.536176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144502+0000","last_change":"2024-04-26T21:35:10.118766+0000","last_active":"2024-04-26T21:35:15.144502+0000","last_peered":"2024-04-26T21:35:15.144502+0000","last_clean":"2024-04-26T21:35:15.144502+0000","last_became_active":"2024-04-26T21:35:10.118480+0000","last_became_peered":"2024-04-26T21:35:10.118480+0000","last_unstale":"2024-04-26T21:35:15.144502+0000","last_undegraded":"2024-04-26T21:35:15.144502+0000","last_fullsized":"2024-04-26T21:35:15.144502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:33.320854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374086+0000","last_change":"2024-04-26T21:35:09.124939+0000","last_active":"2024-04-26T21:35:15.374086+0000","last_peered":"2024-04-26T21:35:15.374086+0000","last_clean":"2024-04-26T21:35:15.374086+0000","last_became_active":"2024-04-26T21:35:09.124630+0000","last_became_peered":"2024-04-26T21:35:09.124630+0000","last_unstale":"2024-04-26T21:35:15.374086+0000","last_undegraded":"2024-04-26T21:35:15.374086+0000","last_fullsized":"2024-04-26T21:35:15.374086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374121+0000","last_change":"2024-04-26T21:35:14.165015+0000","last_active":"2024-04-26T21:35:15.374121+0000","last_peered":"2024-04-26T21:35:15.374121+0000","last_clean":"2024-04-26T21:35:15.374121+0000","last_became_active":"2024-04-26T21:35:14.164615+0000","last_became_peered":"2024-04-26T21:35:14.164615+0000","last_unstale":"2024-04-26T21:35:15.374121+0000","last_undegraded":"2024-04-26T21:35:15.374121+0000","last_fullsized":"2024-04-26T21:35:15.374121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:37.348252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004379+0000","last_change":"2024-04-26T21:35:10.125981+0000","last_active":"2024-04-26T21:35:16.004379+0000","last_peered":"2024-04-26T21:35:16.004379+0000","last_clean":"2024-04-26T21:35:16.004379+0000","last_became_active":"2024-04-26T21:35:10.125718+0000","last_became_peered":"2024-04-26T21:35:10.125718+0000","last_unstale":"2024-04-26T21:35:16.004379+0000","last_undegraded":"2024-04-26T21:35:16.004379+0000","last_fullsized":"2024-04-26T21:35:16.004379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:09.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144282+0000","last_change":"2024-04-26T21:35:09.121086+0000","last_active":"2024-04-26T21:35:15.144282+0000","last_peered":"2024-04-26T21:35:15.144282+0000","last_clean":"2024-04-26T21:35:15.144282+0000","last_became_active":"2024-04-26T21:35:09.120443+0000","last_became_peered":"2024-04-26T21:35:09.120443+0000","last_unstale":"2024-04-26T21:35:15.144282+0000","last_undegraded":"2024-04-26T21:35:15.144282+0000","last_fullsized":"2024-04-26T21:35:15.144282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374085+0000","last_change":"2024-04-26T21:35:14.163974+0000","last_active":"2024-04-26T21:35:15.374085+0000","last_peered":"2024-04-26T21:35:15.374085+0000","last_clean":"2024-04-26T21:35:15.374085+0000","last_became_active":"2024-04-26T21:35:14.163558+0000","last_became_peered":"2024-04-26T21:35:14.163558+0000","last_unstale":"2024-04-26T21:35:15.374085+0000","last_undegraded":"2024-04-26T21:35:15.374085+0000","last_fullsized":"2024-04-26T21:35:15.374085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:01.158030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373556+0000","last_change":"2024-04-26T21:35:10.124855+0000","last_active":"2024-04-26T21:35:15.373556+0000","last_peered":"2024-04-26T21:35:15.373556+0000","last_clean":"2024-04-26T21:35:15.373556+0000","last_became_active":"2024-04-26T21:35:10.124588+0000","last_became_peered":"2024-04-26T21:35:10.124588+0000","last_unstale":"2024-04-26T21:35:15.373556+0000","last_undegraded":"2024-04-26T21:35:15.373556+0000","last_fullsized":"2024-04-26T21:35:15.373556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142012+0000","last_change":"2024-04-26T21:35:09.115909+0000","last_active":"2024-04-26T21:35:15.142012+0000","last_peered":"2024-04-26T21:35:15.142012+0000","last_clean":"2024-04-26T21:35:15.142012+0000","last_became_active":"2024-04-26T21:35:09.115120+0000","last_became_peered":"2024-04-26T21:35:09.115120+0000","last_unstale":"2024-04-26T21:35:15.142012+0000","last_undegraded":"2024-04-26T21:35:15.142012+0000","last_fullsized":"2024-04-26T21:35:15.142012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374302+0000","last_change":"2024-04-26T21:35:14.165160+0000","last_active":"2024-04-26T21:35:15.374302+0000","last_peered":"2024-04-26T21:35:15.374302+0000","last_clean":"2024-04-26T21:35:15.374302+0000","last_became_active":"2024-04-26T21:35:14.164683+0000","last_became_peered":"2024-04-26T21:35:14.164683+0000","last_unstale":"2024-04-26T21:35:15.374302+0000","last_undegraded":"2024-04-26T21:35:15.374302+0000","last_fullsized":"2024-04-26T21:35:15.374302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:37.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003512+0000","last_change":"2024-04-26T21:35:10.128846+0000","last_active":"2024-04-26T21:35:16.003512+0000","last_peered":"2024-04-26T21:35:16.003512+0000","last_clean":"2024-04-26T21:35:16.003512+0000","last_became_active":"2024-04-26T21:35:10.128576+0000","last_became_peered":"2024-04-26T21:35:10.128576+0000","last_unstale":"2024-04-26T21:35:16.003512+0000","last_undegraded":"2024-04-26T21:35:16.003512+0000","last_fullsized":"2024-04-26T21:35:16.003512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:40.892489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875388+0000","last_change":"2024-04-26T21:35:09.130374+0000","last_active":"2024-04-26T21:35:15.875388+0000","last_peered":"2024-04-26T21:35:15.875388+0000","last_clean":"2024-04-26T21:35:15.875388+0000","last_became_active":"2024-04-26T21:35:09.130122+0000","last_became_peered":"2024-04-26T21:35:09.130122+0000","last_unstale":"2024-04-26T21:35:15.875388+0000","last_undegraded":"2024-04-26T21:35:15.875388+0000","last_fullsized":"2024-04-26T21:35:15.875388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:25.035511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004078+0000","last_change":"2024-04-26T21:35:14.277956+0000","last_active":"2024-04-26T21:35:16.004078+0000","last_peered":"2024-04-26T21:35:16.004078+0000","last_clean":"2024-04-26T21:35:16.004078+0000","last_became_active":"2024-04-26T21:35:14.277555+0000","last_became_peered":"2024-04-26T21:35:14.277555+0000","last_unstale":"2024-04-26T21:35:16.004078+0000","last_undegraded":"2024-04-26T21:35:16.004078+0000","last_fullsized":"2024-04-26T21:35:16.004078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:37.348252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003728+0000","last_change":"2024-04-26T21:35:10.127546+0000","last_active":"2024-04-26T21:35:16.003728+0000","last_peered":"2024-04-26T21:35:16.003728+0000","last_clean":"2024-04-26T21:35:16.003728+0000","last_became_active":"2024-04-26T21:35:10.127263+0000","last_became_peered":"2024-04-26T21:35:10.127263+0000","last_unstale":"2024-04-26T21:35:16.003728+0000","last_undegraded":"2024-04-26T21:35:16.003728+0000","last_fullsized":"2024-04-26T21:35:16.003728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144006+0000","last_change":"2024-04-26T21:35:09.116290+0000","last_active":"2024-04-26T21:35:15.144006+0000","last_peered":"2024-04-26T21:35:15.144006+0000","last_clean":"2024-04-26T21:35:15.144006+0000","last_became_active":"2024-04-26T21:35:09.115972+0000","last_became_peered":"2024-04-26T21:35:09.115972+0000","last_unstale":"2024-04-26T21:35:15.144006+0000","last_undegraded":"2024-04-26T21:35:15.144006+0000","last_fullsized":"2024-04-26T21:35:15.144006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875528+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:35:15.875528+0000","last_peered":"2024-04-26T21:35:15.875528+0000","last_clean":"2024-04-26T21:35:15.875528+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:35:15.875528+0000","last_undegraded":"2024-04-26T21:35:15.875528+0000","last_fullsized":"2024-04-26T21:35:15.875528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143208+0000","last_change":"2024-04-26T21:35:14.279050+0000","last_active":"2024-04-26T21:35:15.143208+0000","last_peered":"2024-04-26T21:35:15.143208+0000","last_clean":"2024-04-26T21:35:15.143208+0000","last_became_active":"2024-04-26T21:35:14.278651+0000","last_became_peered":"2024-04-26T21:35:14.278651+0000","last_unstale":"2024-04-26T21:35:15.143208+0000","last_undegraded":"2024-04-26T21:35:15.143208+0000","last_fullsized":"2024-04-26T21:35:15.143208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:26.153246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875749+0000","last_change":"2024-04-26T21:35:10.130945+0000","last_active":"2024-04-26T21:35:15.875749+0000","last_peered":"2024-04-26T21:35:15.875749+0000","last_clean":"2024-04-26T21:35:15.875749+0000","last_became_active":"2024-04-26T21:35:10.130673+0000","last_became_peered":"2024-04-26T21:35:10.130673+0000","last_unstale":"2024-04-26T21:35:15.875749+0000","last_undegraded":"2024-04-26T21:35:15.875749+0000","last_fullsized":"2024-04-26T21:35:15.875749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:16.508778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142347+0000","last_change":"2024-04-26T21:35:09.118997+0000","last_active":"2024-04-26T21:35:15.142347+0000","last_peered":"2024-04-26T21:35:15.142347+0000","last_clean":"2024-04-26T21:35:15.142347+0000","last_became_active":"2024-04-26T21:35:09.118700+0000","last_became_peered":"2024-04-26T21:35:09.118700+0000","last_unstale":"2024-04-26T21:35:15.142347+0000","last_undegraded":"2024-04-26T21:35:15.142347+0000","last_fullsized":"2024-04-26T21:35:15.142347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143286+0000","last_change":"2024-04-26T21:35:14.277928+0000","last_active":"2024-04-26T21:35:15.143286+0000","last_peered":"2024-04-26T21:35:15.143286+0000","last_clean":"2024-04-26T21:35:15.143286+0000","last_became_active":"2024-04-26T21:35:14.277458+0000","last_became_peered":"2024-04-26T21:35:14.277458+0000","last_unstale":"2024-04-26T21:35:15.143286+0000","last_undegraded":"2024-04-26T21:35:15.143286+0000","last_fullsized":"2024-04-26T21:35:15.143286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:55.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144138+0000","last_change":"2024-04-26T21:35:10.120150+0000","last_active":"2024-04-26T21:35:15.144138+0000","last_peered":"2024-04-26T21:35:15.144138+0000","last_clean":"2024-04-26T21:35:15.144138+0000","last_became_active":"2024-04-26T21:35:10.119894+0000","last_became_peered":"2024-04-26T21:35:10.119894+0000","last_unstale":"2024-04-26T21:35:15.144138+0000","last_undegraded":"2024-04-26T21:35:15.144138+0000","last_fullsized":"2024-04-26T21:35:15.144138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.413835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374128+0000","last_change":"2024-04-26T21:35:09.125791+0000","last_active":"2024-04-26T21:35:15.374128+0000","last_peered":"2024-04-26T21:35:15.374128+0000","last_clean":"2024-04-26T21:35:15.374128+0000","last_became_active":"2024-04-26T21:35:09.125356+0000","last_became_peered":"2024-04-26T21:35:09.125356+0000","last_unstale":"2024-04-26T21:35:15.374128+0000","last_undegraded":"2024-04-26T21:35:15.374128+0000","last_fullsized":"2024-04-26T21:35:15.374128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:25.035511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004470+0000","last_change":"2024-04-26T21:35:14.160611+0000","last_active":"2024-04-26T21:35:16.004470+0000","last_peered":"2024-04-26T21:35:16.004470+0000","last_clean":"2024-04-26T21:35:16.004470+0000","last_became_active":"2024-04-26T21:35:14.160067+0000","last_became_peered":"2024-04-26T21:35:14.160067+0000","last_unstale":"2024-04-26T21:35:16.004470+0000","last_undegraded":"2024-04-26T21:35:16.004470+0000","last_fullsized":"2024-04-26T21:35:16.004470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:37.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876144+0000","last_change":"2024-04-26T21:35:10.136587+0000","last_active":"2024-04-26T21:35:15.876144+0000","last_peered":"2024-04-26T21:35:15.876144+0000","last_clean":"2024-04-26T21:35:15.876144+0000","last_became_active":"2024-04-26T21:35:10.136271+0000","last_became_peered":"2024-04-26T21:35:10.136271+0000","last_unstale":"2024-04-26T21:35:15.876144+0000","last_undegraded":"2024-04-26T21:35:15.876144+0000","last_fullsized":"2024-04-26T21:35:15.876144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:45.927928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142500+0000","last_change":"2024-04-26T21:35:09.115543+0000","last_active":"2024-04-26T21:35:15.142500+0000","last_peered":"2024-04-26T21:35:15.142500+0000","last_clean":"2024-04-26T21:35:15.142500+0000","last_became_active":"2024-04-26T21:35:09.115159+0000","last_became_peered":"2024-04-26T21:35:09.115159+0000","last_unstale":"2024-04-26T21:35:15.142500+0000","last_undegraded":"2024-04-26T21:35:15.142500+0000","last_fullsized":"2024-04-26T21:35:15.142500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142639+0000","last_change":"2024-04-26T21:35:14.278004+0000","last_active":"2024-04-26T21:35:15.142639+0000","last_peered":"2024-04-26T21:35:15.142639+0000","last_clean":"2024-04-26T21:35:15.142639+0000","last_became_active":"2024-04-26T21:35:14.277547+0000","last_became_peered":"2024-04-26T21:35:14.277547+0000","last_unstale":"2024-04-26T21:35:15.142639+0000","last_undegraded":"2024-04-26T21:35:15.142639+0000","last_fullsized":"2024-04-26T21:35:15.142639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:49.955325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875902+0000","last_change":"2024-04-26T21:35:10.131439+0000","last_active":"2024-04-26T21:35:15.875902+0000","last_peered":"2024-04-26T21:35:15.875902+0000","last_clean":"2024-04-26T21:35:15.875902+0000","last_became_active":"2024-04-26T21:35:10.130864+0000","last_became_peered":"2024-04-26T21:35:10.130864+0000","last_unstale":"2024-04-26T21:35:15.875902+0000","last_undegraded":"2024-04-26T21:35:15.875902+0000","last_fullsized":"2024-04-26T21:35:15.875902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.130632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004324+0000","last_change":"2024-04-26T21:35:09.123483+0000","last_active":"2024-04-26T21:35:16.004324+0000","last_peered":"2024-04-26T21:35:16.004324+0000","last_clean":"2024-04-26T21:35:16.004324+0000","last_became_active":"2024-04-26T21:35:09.123202+0000","last_became_peered":"2024-04-26T21:35:09.123202+0000","last_unstale":"2024-04-26T21:35:16.004324+0000","last_undegraded":"2024-04-26T21:35:16.004324+0000","last_fullsized":"2024-04-26T21:35:16.004324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:50.000306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003596+0000","last_change":"2024-04-26T21:35:14.163479+0000","last_active":"2024-04-26T21:35:16.003596+0000","last_peered":"2024-04-26T21:35:16.003596+0000","last_clean":"2024-04-26T21:35:16.003596+0000","last_became_active":"2024-04-26T21:35:14.163041+0000","last_became_peered":"2024-04-26T21:35:14.163041+0000","last_unstale":"2024-04-26T21:35:16.003596+0000","last_undegraded":"2024-04-26T21:35:16.003596+0000","last_fullsized":"2024-04-26T21:35:16.003596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:34.682922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875665+0000","last_change":"2024-04-26T21:35:10.129858+0000","last_active":"2024-04-26T21:35:15.875665+0000","last_peered":"2024-04-26T21:35:15.875665+0000","last_clean":"2024-04-26T21:35:15.875665+0000","last_became_active":"2024-04-26T21:35:10.129606+0000","last_became_peered":"2024-04-26T21:35:10.129606+0000","last_unstale":"2024-04-26T21:35:15.875665+0000","last_undegraded":"2024-04-26T21:35:15.875665+0000","last_fullsized":"2024-04-26T21:35:15.875665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.413835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.141883+0000","last_change":"2024-04-26T21:35:09.121200+0000","last_active":"2024-04-26T21:35:15.141883+0000","last_peered":"2024-04-26T21:35:15.141883+0000","last_clean":"2024-04-26T21:35:15.141883+0000","last_became_active":"2024-04-26T21:35:09.120700+0000","last_became_peered":"2024-04-26T21:35:09.120700+0000","last_unstale":"2024-04-26T21:35:15.141883+0000","last_undegraded":"2024-04-26T21:35:15.141883+0000","last_fullsized":"2024-04-26T21:35:15.141883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003924+0000","last_change":"2024-04-26T21:35:14.163658+0000","last_active":"2024-04-26T21:35:16.003924+0000","last_peered":"2024-04-26T21:35:16.003924+0000","last_clean":"2024-04-26T21:35:16.003924+0000","last_became_active":"2024-04-26T21:35:14.163215+0000","last_became_peered":"2024-04-26T21:35:14.163215+0000","last_unstale":"2024-04-26T21:35:16.003924+0000","last_undegraded":"2024-04-26T21:35:16.003924+0000","last_fullsized":"2024-04-26T21:35:16.003924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:01.158030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142807+0000","last_change":"2024-04-26T21:35:10.126750+0000","last_active":"2024-04-26T21:35:15.142807+0000","last_peered":"2024-04-26T21:35:15.142807+0000","last_clean":"2024-04-26T21:35:15.142807+0000","last_became_active":"2024-04-26T21:35:10.126096+0000","last_became_peered":"2024-04-26T21:35:10.126096+0000","last_unstale":"2024-04-26T21:35:15.142807+0000","last_undegraded":"2024-04-26T21:35:15.142807+0000","last_fullsized":"2024-04-26T21:35:15.142807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:46.899885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143301+0000","last_change":"2024-04-26T21:35:09.122401+0000","last_active":"2024-04-26T21:35:15.143301+0000","last_peered":"2024-04-26T21:35:15.143301+0000","last_clean":"2024-04-26T21:35:15.143301+0000","last_became_active":"2024-04-26T21:35:09.122067+0000","last_became_peered":"2024-04-26T21:35:09.122067+0000","last_unstale":"2024-04-26T21:35:15.143301+0000","last_undegraded":"2024-04-26T21:35:15.143301+0000","last_fullsized":"2024-04-26T21:35:15.143301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004812+0000","last_change":"2024-04-26T21:35:14.279383+0000","last_active":"2024-04-26T21:35:16.004812+0000","last_peered":"2024-04-26T21:35:16.004812+0000","last_clean":"2024-04-26T21:35:16.004812+0000","last_became_active":"2024-04-26T21:35:14.278985+0000","last_became_peered":"2024-04-26T21:35:14.278985+0000","last_unstale":"2024-04-26T21:35:16.004812+0000","last_undegraded":"2024-04-26T21:35:16.004812+0000","last_fullsized":"2024-04-26T21:35:16.004812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.757335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875771+0000","last_change":"2024-04-26T21:35:10.136295+0000","last_active":"2024-04-26T21:35:15.875771+0000","last_peered":"2024-04-26T21:35:15.875771+0000","last_clean":"2024-04-26T21:35:15.875771+0000","last_became_active":"2024-04-26T21:35:10.135975+0000","last_became_peered":"2024-04-26T21:35:10.135975+0000","last_unstale":"2024-04-26T21:35:15.875771+0000","last_undegraded":"2024-04-26T21:35:15.875771+0000","last_fullsized":"2024-04-26T21:35:15.875771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144056+0000","last_change":"2024-04-26T21:35:09.122493+0000","last_active":"2024-04-26T21:35:15.144056+0000","last_peered":"2024-04-26T21:35:15.144056+0000","last_clean":"2024-04-26T21:35:15.144056+0000","last_became_active":"2024-04-26T21:35:09.122211+0000","last_became_peered":"2024-04-26T21:35:09.122211+0000","last_unstale":"2024-04-26T21:35:15.144056+0000","last_undegraded":"2024-04-26T21:35:15.144056+0000","last_fullsized":"2024-04-26T21:35:15.144056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004937+0000","last_change":"2024-04-26T21:35:14.277801+0000","last_active":"2024-04-26T21:35:16.004937+0000","last_peered":"2024-04-26T21:35:16.004937+0000","last_clean":"2024-04-26T21:35:16.004937+0000","last_became_active":"2024-04-26T21:35:14.277417+0000","last_became_peered":"2024-04-26T21:35:14.277417+0000","last_unstale":"2024-04-26T21:35:16.004937+0000","last_undegraded":"2024-04-26T21:35:16.004937+0000","last_fullsized":"2024-04-26T21:35:16.004937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:15.246664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004133+0000","last_change":"2024-04-26T21:35:10.128386+0000","last_active":"2024-04-26T21:35:16.004133+0000","last_peered":"2024-04-26T21:35:16.004133+0000","last_clean":"2024-04-26T21:35:16.004133+0000","last_became_active":"2024-04-26T21:35:10.127067+0000","last_became_peered":"2024-04-26T21:35:10.127067+0000","last_unstale":"2024-04-26T21:35:16.004133+0000","last_undegraded":"2024-04-26T21:35:16.004133+0000","last_fullsized":"2024-04-26T21:35:16.004133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:46.899885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004551+0000","last_change":"2024-04-26T21:35:09.125422+0000","last_active":"2024-04-26T21:35:16.004551+0000","last_peered":"2024-04-26T21:35:16.004551+0000","last_clean":"2024-04-26T21:35:16.004551+0000","last_became_active":"2024-04-26T21:35:09.125086+0000","last_became_peered":"2024-04-26T21:35:09.125086+0000","last_unstale":"2024-04-26T21:35:16.004551+0000","last_undegraded":"2024-04-26T21:35:16.004551+0000","last_fullsized":"2024-04-26T21:35:16.004551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:44.358668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144502+0000","last_change":"2024-04-26T21:35:14.278312+0000","last_active":"2024-04-26T21:35:15.144502+0000","last_peered":"2024-04-26T21:35:15.144502+0000","last_clean":"2024-04-26T21:35:15.144502+0000","last_became_active":"2024-04-26T21:35:14.277836+0000","last_became_peered":"2024-04-26T21:35:14.277836+0000","last_unstale":"2024-04-26T21:35:15.144502+0000","last_undegraded":"2024-04-26T21:35:15.144502+0000","last_fullsized":"2024-04-26T21:35:15.144502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:20.215438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875616+0000","last_change":"2024-04-26T21:35:10.126791+0000","last_active":"2024-04-26T21:35:15.875616+0000","last_peered":"2024-04-26T21:35:15.875616+0000","last_clean":"2024-04-26T21:35:15.875616+0000","last_became_active":"2024-04-26T21:35:10.126515+0000","last_became_peered":"2024-04-26T21:35:10.126515+0000","last_unstale":"2024-04-26T21:35:15.875616+0000","last_undegraded":"2024-04-26T21:35:15.875616+0000","last_fullsized":"2024-04-26T21:35:15.875616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:01.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:19.559960+0000","last_change":"2024-04-26T21:35:09.126680+0000","last_active":"2024-04-26T21:35:19.559960+0000","last_peered":"2024-04-26T21:35:19.559960+0000","last_clean":"2024-04-26T21:35:19.559960+0000","last_became_active":"2024-04-26T21:35:09.126406+0000","last_became_peered":"2024-04-26T21:35:09.126406+0000","last_unstale":"2024-04-26T21:35:19.559960+0000","last_undegraded":"2024-04-26T21:35:19.559960+0000","last_fullsized":"2024-04-26T21:35:19.559960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373762+0000","last_change":"2024-04-26T21:35:14.163960+0000","last_active":"2024-04-26T21:35:15.373762+0000","last_peered":"2024-04-26T21:35:15.373762+0000","last_clean":"2024-04-26T21:35:15.373762+0000","last_became_active":"2024-04-26T21:35:14.163551+0000","last_became_peered":"2024-04-26T21:35:14.163551+0000","last_unstale":"2024-04-26T21:35:15.373762+0000","last_undegraded":"2024-04-26T21:35:15.373762+0000","last_fullsized":"2024-04-26T21:35:15.373762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004282+0000","last_change":"2024-04-26T21:35:10.129151+0000","last_active":"2024-04-26T21:35:16.004282+0000","last_peered":"2024-04-26T21:35:16.004282+0000","last_clean":"2024-04-26T21:35:16.004282+0000","last_became_active":"2024-04-26T21:35:10.128520+0000","last_became_peered":"2024-04-26T21:35:10.128520+0000","last_unstale":"2024-04-26T21:35:16.004282+0000","last_undegraded":"2024-04-26T21:35:16.004282+0000","last_fullsized":"2024-04-26T21:35:16.004282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:17.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374063+0000","last_change":"2024-04-26T21:35:09.128170+0000","last_active":"2024-04-26T21:35:15.374063+0000","last_peered":"2024-04-26T21:35:15.374063+0000","last_clean":"2024-04-26T21:35:15.374063+0000","last_became_active":"2024-04-26T21:35:09.127817+0000","last_became_peered":"2024-04-26T21:35:09.127817+0000","last_unstale":"2024-04-26T21:35:15.374063+0000","last_undegraded":"2024-04-26T21:35:15.374063+0000","last_fullsized":"2024-04-26T21:35:15.374063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004297+0000","last_change":"2024-04-26T21:35:14.157290+0000","last_active":"2024-04-26T21:35:16.004297+0000","last_peered":"2024-04-26T21:35:16.004297+0000","last_clean":"2024-04-26T21:35:16.004297+0000","last_became_active":"2024-04-26T21:35:14.156794+0000","last_became_peered":"2024-04-26T21:35:14.156794+0000","last_unstale":"2024-04-26T21:35:16.004297+0000","last_undegraded":"2024-04-26T21:35:16.004297+0000","last_fullsized":"2024-04-26T21:35:16.004297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:55.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004165+0000","last_change":"2024-04-26T21:35:10.132571+0000","last_active":"2024-04-26T21:35:16.004165+0000","last_peered":"2024-04-26T21:35:16.004165+0000","last_clean":"2024-04-26T21:35:16.004165+0000","last_became_active":"2024-04-26T21:35:10.132311+0000","last_became_peered":"2024-04-26T21:35:10.132311+0000","last_unstale":"2024-04-26T21:35:16.004165+0000","last_undegraded":"2024-04-26T21:35:16.004165+0000","last_fullsized":"2024-04-26T21:35:16.004165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143720+0000","last_change":"2024-04-26T21:35:09.124883+0000","last_active":"2024-04-26T21:35:15.143720+0000","last_peered":"2024-04-26T21:35:15.143720+0000","last_clean":"2024-04-26T21:35:15.143720+0000","last_became_active":"2024-04-26T21:35:09.124576+0000","last_became_peered":"2024-04-26T21:35:09.124576+0000","last_unstale":"2024-04-26T21:35:15.143720+0000","last_undegraded":"2024-04-26T21:35:15.143720+0000","last_fullsized":"2024-04-26T21:35:15.143720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003494+0000","last_change":"2024-04-26T21:35:14.158063+0000","last_active":"2024-04-26T21:35:16.003494+0000","last_peered":"2024-04-26T21:35:16.003494+0000","last_clean":"2024-04-26T21:35:16.003494+0000","last_became_active":"2024-04-26T21:35:14.157033+0000","last_became_peered":"2024-04-26T21:35:14.157033+0000","last_unstale":"2024-04-26T21:35:16.003494+0000","last_undegraded":"2024-04-26T21:35:16.003494+0000","last_fullsized":"2024-04-26T21:35:16.003494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:20.215438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143604+0000","last_change":"2024-04-26T21:35:10.121823+0000","last_active":"2024-04-26T21:35:15.143604+0000","last_peered":"2024-04-26T21:35:15.143604+0000","last_clean":"2024-04-26T21:35:15.143604+0000","last_became_active":"2024-04-26T21:35:10.121518+0000","last_became_peered":"2024-04-26T21:35:10.121518+0000","last_unstale":"2024-04-26T21:35:15.143604+0000","last_undegraded":"2024-04-26T21:35:15.143604+0000","last_fullsized":"2024-04-26T21:35:15.143604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:06.354261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143861+0000","last_change":"2024-04-26T21:35:09.127420+0000","last_active":"2024-04-26T21:35:15.143861+0000","last_peered":"2024-04-26T21:35:15.143861+0000","last_clean":"2024-04-26T21:35:15.143861+0000","last_became_active":"2024-04-26T21:35:09.127104+0000","last_became_peered":"2024-04-26T21:35:09.127104+0000","last_unstale":"2024-04-26T21:35:15.143861+0000","last_undegraded":"2024-04-26T21:35:15.143861+0000","last_fullsized":"2024-04-26T21:35:15.143861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004551+0000","last_change":"2024-04-26T21:35:14.281132+0000","last_active":"2024-04-26T21:35:16.004551+0000","last_peered":"2024-04-26T21:35:16.004551+0000","last_clean":"2024-04-26T21:35:16.004551+0000","last_became_active":"2024-04-26T21:35:14.280538+0000","last_became_peered":"2024-04-26T21:35:14.280538+0000","last_unstale":"2024-04-26T21:35:16.004551+0000","last_undegraded":"2024-04-26T21:35:16.004551+0000","last_fullsized":"2024-04-26T21:35:16.004551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:37.157683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003564+0000","last_change":"2024-04-26T21:35:14.279953+0000","last_active":"2024-04-26T21:35:16.003564+0000","last_peered":"2024-04-26T21:35:16.003564+0000","last_clean":"2024-04-26T21:35:16.003564+0000","last_became_active":"2024-04-26T21:35:14.279540+0000","last_became_peered":"2024-04-26T21:35:14.279540+0000","last_unstale":"2024-04-26T21:35:16.003564+0000","last_undegraded":"2024-04-26T21:35:16.003564+0000","last_fullsized":"2024-04-26T21:35:16.003564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373464+0000","last_change":"2024-04-26T21:35:10.121201+0000","last_active":"2024-04-26T21:35:15.373464+0000","last_peered":"2024-04-26T21:35:15.373464+0000","last_clean":"2024-04-26T21:35:15.373464+0000","last_became_active":"2024-04-26T21:35:10.120934+0000","last_became_peered":"2024-04-26T21:35:10.120934+0000","last_unstale":"2024-04-26T21:35:15.373464+0000","last_undegraded":"2024-04-26T21:35:15.373464+0000","last_fullsized":"2024-04-26T21:35:15.373464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142787+0000","last_change":"2024-04-26T21:35:09.119199+0000","last_active":"2024-04-26T21:35:15.142787+0000","last_peered":"2024-04-26T21:35:15.142787+0000","last_clean":"2024-04-26T21:35:15.142787+0000","last_became_active":"2024-04-26T21:35:09.118431+0000","last_became_peered":"2024-04-26T21:35:09.118431+0000","last_unstale":"2024-04-26T21:35:15.142787+0000","last_undegraded":"2024-04-26T21:35:15.142787+0000","last_fullsized":"2024-04-26T21:35:15.142787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144043+0000","last_change":"2024-04-26T21:35:14.160976+0000","last_active":"2024-04-26T21:35:15.144043+0000","last_peered":"2024-04-26T21:35:15.144043+0000","last_clean":"2024-04-26T21:35:15.144043+0000","last_became_active":"2024-04-26T21:35:14.160609+0000","last_became_peered":"2024-04-26T21:35:14.160609+0000","last_unstale":"2024-04-26T21:35:15.144043+0000","last_undegraded":"2024-04-26T21:35:15.144043+0000","last_fullsized":"2024-04-26T21:35:15.144043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003542+0000","last_change":"2024-04-26T21:35:10.134441+0000","last_active":"2024-04-26T21:35:16.003542+0000","last_peered":"2024-04-26T21:35:16.003542+0000","last_clean":"2024-04-26T21:35:16.003542+0000","last_became_active":"2024-04-26T21:35:10.134101+0000","last_became_peered":"2024-04-26T21:35:10.134101+0000","last_unstale":"2024-04-26T21:35:16.003542+0000","last_undegraded":"2024-04-26T21:35:16.003542+0000","last_fullsized":"2024-04-26T21:35:16.003542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004713+0000","last_change":"2024-04-26T21:35:09.124939+0000","last_active":"2024-04-26T21:35:16.004713+0000","last_peered":"2024-04-26T21:35:16.004713+0000","last_clean":"2024-04-26T21:35:16.004713+0000","last_became_active":"2024-04-26T21:35:09.124612+0000","last_became_peered":"2024-04-26T21:35:09.124612+0000","last_unstale":"2024-04-26T21:35:16.004713+0000","last_undegraded":"2024-04-26T21:35:16.004713+0000","last_fullsized":"2024-04-26T21:35:16.004713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143484+0000","last_change":"2024-04-26T21:35:09.122510+0000","last_active":"2024-04-26T21:35:15.143484+0000","last_peered":"2024-04-26T21:35:15.143484+0000","last_clean":"2024-04-26T21:35:15.143484+0000","last_became_active":"2024-04-26T21:35:09.122161+0000","last_became_peered":"2024-04-26T21:35:09.122161+0000","last_unstale":"2024-04-26T21:35:15.143484+0000","last_undegraded":"2024-04-26T21:35:15.143484+0000","last_fullsized":"2024-04-26T21:35:15.143484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004522+0000","last_change":"2024-04-26T21:35:10.128884+0000","last_active":"2024-04-26T21:35:16.004522+0000","last_peered":"2024-04-26T21:35:16.004522+0000","last_clean":"2024-04-26T21:35:16.004522+0000","last_became_active":"2024-04-26T21:35:10.128599+0000","last_became_peered":"2024-04-26T21:35:10.128599+0000","last_unstale":"2024-04-26T21:35:16.004522+0000","last_undegraded":"2024-04-26T21:35:16.004522+0000","last_fullsized":"2024-04-26T21:35:16.004522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142294+0000","last_change":"2024-04-26T21:35:14.158429+0000","last_active":"2024-04-26T21:35:15.142294+0000","last_peered":"2024-04-26T21:35:15.142294+0000","last_clean":"2024-04-26T21:35:15.142294+0000","last_became_active":"2024-04-26T21:35:14.158011+0000","last_became_peered":"2024-04-26T21:35:14.158011+0000","last_unstale":"2024-04-26T21:35:15.142294+0000","last_undegraded":"2024-04-26T21:35:15.142294+0000","last_fullsized":"2024-04-26T21:35:15.142294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:44.919887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004133+0000","last_change":"2024-04-26T21:35:09.127502+0000","last_active":"2024-04-26T21:35:16.004133+0000","last_peered":"2024-04-26T21:35:16.004133+0000","last_clean":"2024-04-26T21:35:16.004133+0000","last_became_active":"2024-04-26T21:35:09.127182+0000","last_became_peered":"2024-04-26T21:35:09.127182+0000","last_unstale":"2024-04-26T21:35:16.004133+0000","last_undegraded":"2024-04-26T21:35:16.004133+0000","last_fullsized":"2024-04-26T21:35:16.004133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003916+0000","last_change":"2024-04-26T21:35:10.120022+0000","last_active":"2024-04-26T21:35:16.003916+0000","last_peered":"2024-04-26T21:35:16.003916+0000","last_clean":"2024-04-26T21:35:16.003916+0000","last_became_active":"2024-04-26T21:35:10.119759+0000","last_became_peered":"2024-04-26T21:35:10.119759+0000","last_unstale":"2024-04-26T21:35:16.003916+0000","last_undegraded":"2024-04-26T21:35:16.003916+0000","last_fullsized":"2024-04-26T21:35:16.003916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:55.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004467+0000","last_change":"2024-04-26T21:35:14.157203+0000","last_active":"2024-04-26T21:35:16.004467+0000","last_peered":"2024-04-26T21:35:16.004467+0000","last_clean":"2024-04-26T21:35:16.004467+0000","last_became_active":"2024-04-26T21:35:14.156742+0000","last_became_peered":"2024-04-26T21:35:14.156742+0000","last_unstale":"2024-04-26T21:35:16.004467+0000","last_undegraded":"2024-04-26T21:35:16.004467+0000","last_fullsized":"2024-04-26T21:35:16.004467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:13.631171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142673+0000","last_change":"2024-04-26T21:35:10.126924+0000","last_active":"2024-04-26T21:35:15.142673+0000","last_peered":"2024-04-26T21:35:15.142673+0000","last_clean":"2024-04-26T21:35:15.142673+0000","last_became_active":"2024-04-26T21:35:10.126660+0000","last_became_peered":"2024-04-26T21:35:10.126660+0000","last_unstale":"2024-04-26T21:35:15.142673+0000","last_undegraded":"2024-04-26T21:35:15.142673+0000","last_fullsized":"2024-04-26T21:35:15.142673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:29.764634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142006+0000","last_change":"2024-04-26T21:35:10.116329+0000","last_active":"2024-04-26T21:35:15.142006+0000","last_peered":"2024-04-26T21:35:15.142006+0000","last_clean":"2024-04-26T21:35:15.142006+0000","last_became_active":"2024-04-26T21:35:10.116047+0000","last_became_peered":"2024-04-26T21:35:10.116047+0000","last_unstale":"2024-04-26T21:35:15.142006+0000","last_undegraded":"2024-04-26T21:35:15.142006+0000","last_fullsized":"2024-04-26T21:35:15.142006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:17.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143917+0000","last_change":"2024-04-26T21:35:10.121375+0000","last_active":"2024-04-26T21:35:15.143917+0000","last_peered":"2024-04-26T21:35:15.143917+0000","last_clean":"2024-04-26T21:35:15.143917+0000","last_became_active":"2024-04-26T21:35:10.121088+0000","last_became_peered":"2024-04-26T21:35:10.121088+0000","last_unstale":"2024-04-26T21:35:15.143917+0000","last_undegraded":"2024-04-26T21:35:15.143917+0000","last_fullsized":"2024-04-26T21:35:15.143917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004457+0000","last_change":"2024-04-26T21:35:10.131911+0000","last_active":"2024-04-26T21:35:16.004457+0000","last_peered":"2024-04-26T21:35:16.004457+0000","last_clean":"2024-04-26T21:35:16.004457+0000","last_became_active":"2024-04-26T21:35:10.131477+0000","last_became_peered":"2024-04-26T21:35:10.131477+0000","last_unstale":"2024-04-26T21:35:16.004457+0000","last_undegraded":"2024-04-26T21:35:16.004457+0000","last_fullsized":"2024-04-26T21:35:16.004457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.005022+0000","last_change":"2024-04-26T21:35:10.121507+0000","last_active":"2024-04-26T21:35:16.005022+0000","last_peered":"2024-04-26T21:35:16.005022+0000","last_clean":"2024-04-26T21:35:16.005022+0000","last_became_active":"2024-04-26T21:35:10.121208+0000","last_became_peered":"2024-04-26T21:35:10.121208+0000","last_unstale":"2024-04-26T21:35:16.005022+0000","last_undegraded":"2024-04-26T21:35:16.005022+0000","last_fullsized":"2024-04-26T21:35:16.005022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373872+0000","last_change":"2024-04-26T21:35:10.121106+0000","last_active":"2024-04-26T21:35:15.373872+0000","last_peered":"2024-04-26T21:35:15.373872+0000","last_clean":"2024-04-26T21:35:15.373872+0000","last_became_active":"2024-04-26T21:35:10.120857+0000","last_became_peered":"2024-04-26T21:35:10.120857+0000","last_unstale":"2024-04-26T21:35:15.373872+0000","last_undegraded":"2024-04-26T21:35:15.373872+0000","last_fullsized":"2024-04-26T21:35:15.373872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143866+0000","last_change":"2024-04-26T21:35:10.128212+0000","last_active":"2024-04-26T21:35:15.143866+0000","last_peered":"2024-04-26T21:35:15.143866+0000","last_clean":"2024-04-26T21:35:15.143866+0000","last_became_active":"2024-04-26T21:35:10.127868+0000","last_became_peered":"2024-04-26T21:35:10.127868+0000","last_unstale":"2024-04-26T21:35:15.143866+0000","last_undegraded":"2024-04-26T21:35:15.143866+0000","last_fullsized":"2024-04-26T21:35:15.143866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004798+0000","last_change":"2024-04-26T21:35:10.130695+0000","last_active":"2024-04-26T21:35:16.004798+0000","last_peered":"2024-04-26T21:35:16.004798+0000","last_clean":"2024-04-26T21:35:16.004798+0000","last_became_active":"2024-04-26T21:35:10.130084+0000","last_became_peered":"2024-04-26T21:35:10.130084+0000","last_unstale":"2024-04-26T21:35:16.004798+0000","last_undegraded":"2024-04-26T21:35:16.004798+0000","last_fullsized":"2024-04-26T21:35:16.004798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142712+0000","last_change":"2024-04-26T21:35:10.122732+0000","last_active":"2024-04-26T21:35:15.142712+0000","last_peered":"2024-04-26T21:35:15.142712+0000","last_clean":"2024-04-26T21:35:15.142712+0000","last_became_active":"2024-04-26T21:35:10.122457+0000","last_became_peered":"2024-04-26T21:35:10.122457+0000","last_unstale":"2024-04-26T21:35:15.142712+0000","last_undegraded":"2024-04-26T21:35:15.142712+0000","last_fullsized":"2024-04-26T21:35:15.142712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004615+0000","last_change":"2024-04-26T21:35:10.131503+0000","last_active":"2024-04-26T21:35:16.004615+0000","last_peered":"2024-04-26T21:35:16.004615+0000","last_clean":"2024-04-26T21:35:16.004615+0000","last_became_active":"2024-04-26T21:35:10.131262+0000","last_became_peered":"2024-04-26T21:35:10.131262+0000","last_unstale":"2024-04-26T21:35:16.004615+0000","last_undegraded":"2024-04-26T21:35:16.004615+0000","last_fullsized":"2024-04-26T21:35:16.004615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143913+0000","last_change":"2024-04-26T21:35:10.122046+0000","last_active":"2024-04-26T21:35:15.143913+0000","last_peered":"2024-04-26T21:35:15.143913+0000","last_clean":"2024-04-26T21:35:15.143913+0000","last_became_active":"2024-04-26T21:35:10.121759+0000","last_became_peered":"2024-04-26T21:35:10.121759+0000","last_unstale":"2024-04-26T21:35:15.143913+0000","last_undegraded":"2024-04-26T21:35:15.143913+0000","last_fullsized":"2024-04-26T21:35:15.143913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004361+0000","last_change":"2024-04-26T21:35:10.128039+0000","last_active":"2024-04-26T21:35:16.004361+0000","last_peered":"2024-04-26T21:35:16.004361+0000","last_clean":"2024-04-26T21:35:16.004361+0000","last_became_active":"2024-04-26T21:35:10.126920+0000","last_became_peered":"2024-04-26T21:35:10.126920+0000","last_unstale":"2024-04-26T21:35:16.004361+0000","last_undegraded":"2024-04-26T21:35:16.004361+0000","last_fullsized":"2024-04-26T21:35:16.004361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143654+0000","last_change":"2024-04-26T21:35:10.120371+0000","last_active":"2024-04-26T21:35:15.143654+0000","last_peered":"2024-04-26T21:35:15.143654+0000","last_clean":"2024-04-26T21:35:15.143654+0000","last_became_active":"2024-04-26T21:35:10.120020+0000","last_became_peered":"2024-04-26T21:35:10.120020+0000","last_unstale":"2024-04-26T21:35:15.143654+0000","last_undegraded":"2024-04-26T21:35:15.143654+0000","last_fullsized":"2024-04-26T21:35:15.143654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876023+0000","last_change":"2024-04-26T21:35:10.128002+0000","last_active":"2024-04-26T21:35:15.876023+0000","last_peered":"2024-04-26T21:35:15.876023+0000","last_clean":"2024-04-26T21:35:15.876023+0000","last_became_active":"2024-04-26T21:35:10.127763+0000","last_became_peered":"2024-04-26T21:35:10.127763+0000","last_unstale":"2024-04-26T21:35:15.876023+0000","last_undegraded":"2024-04-26T21:35:15.876023+0000","last_fullsized":"2024-04-26T21:35:15.876023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:09.615472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142501+0000","last_change":"2024-04-26T21:35:10.118765+0000","last_active":"2024-04-26T21:35:15.142501+0000","last_peered":"2024-04-26T21:35:15.142501+0000","last_clean":"2024-04-26T21:35:15.142501+0000","last_became_active":"2024-04-26T21:35:10.118481+0000","last_became_peered":"2024-04-26T21:35:10.118481+0000","last_unstale":"2024-04-26T21:35:15.142501+0000","last_undegraded":"2024-04-26T21:35:15.142501+0000","last_fullsized":"2024-04-26T21:35:15.142501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:55.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144424+0000","last_change":"2024-04-26T21:35:10.123877+0000","last_active":"2024-04-26T21:35:15.144424+0000","last_peered":"2024-04-26T21:35:15.144424+0000","last_clean":"2024-04-26T21:35:15.144424+0000","last_became_active":"2024-04-26T21:35:10.122628+0000","last_became_peered":"2024-04-26T21:35:10.122628+0000","last_unstale":"2024-04-26T21:35:15.144424+0000","last_undegraded":"2024-04-26T21:35:15.144424+0000","last_fullsized":"2024-04-26T21:35:15.144424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142605+0000","last_change":"2024-04-26T21:35:10.133644+0000","last_active":"2024-04-26T21:35:15.142605+0000","last_peered":"2024-04-26T21:35:15.142605+0000","last_clean":"2024-04-26T21:35:15.142605+0000","last_became_active":"2024-04-26T21:35:10.133364+0000","last_became_peered":"2024-04-26T21:35:10.133364+0000","last_unstale":"2024-04-26T21:35:15.142605+0000","last_undegraded":"2024-04-26T21:35:15.142605+0000","last_fullsized":"2024-04-26T21:35:15.142605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373575+0000","last_change":"2024-04-26T21:35:10.130778+0000","last_active":"2024-04-26T21:35:15.373575+0000","last_peered":"2024-04-26T21:35:15.373575+0000","last_clean":"2024-04-26T21:35:15.373575+0000","last_became_active":"2024-04-26T21:35:10.130501+0000","last_became_peered":"2024-04-26T21:35:10.130501+0000","last_unstale":"2024-04-26T21:35:15.373575+0000","last_undegraded":"2024-04-26T21:35:15.373575+0000","last_fullsized":"2024-04-26T21:35:15.373575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004023+0000","last_change":"2024-04-26T21:35:10.134920+0000","last_active":"2024-04-26T21:35:16.004023+0000","last_peered":"2024-04-26T21:35:16.004023+0000","last_clean":"2024-04-26T21:35:16.004023+0000","last_became_active":"2024-04-26T21:35:10.134647+0000","last_became_peered":"2024-04-26T21:35:10.134647+0000","last_unstale":"2024-04-26T21:35:16.004023+0000","last_undegraded":"2024-04-26T21:35:16.004023+0000","last_fullsized":"2024-04-26T21:35:16.004023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:19.754278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003968+0000","last_change":"2024-04-26T21:35:10.118720+0000","last_active":"2024-04-26T21:35:16.003968+0000","last_peered":"2024-04-26T21:35:16.003968+0000","last_clean":"2024-04-26T21:35:16.003968+0000","last_became_active":"2024-04-26T21:35:10.118418+0000","last_became_peered":"2024-04-26T21:35:10.118418+0000","last_unstale":"2024-04-26T21:35:16.003968+0000","last_undegraded":"2024-04-26T21:35:16.003968+0000","last_fullsized":"2024-04-26T21:35:16.003968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004237+0000","last_change":"2024-04-26T21:35:10.118157+0000","last_active":"2024-04-26T21:35:16.004237+0000","last_peered":"2024-04-26T21:35:16.004237+0000","last_clean":"2024-04-26T21:35:16.004237+0000","last_became_active":"2024-04-26T21:35:10.117862+0000","last_became_peered":"2024-04-26T21:35:10.117862+0000","last_unstale":"2024-04-26T21:35:16.004237+0000","last_undegraded":"2024-04-26T21:35:16.004237+0000","last_fullsized":"2024-04-26T21:35:16.004237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:26.040140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143343+0000","last_change":"2024-04-26T21:35:10.124503+0000","last_active":"2024-04-26T21:35:15.143343+0000","last_peered":"2024-04-26T21:35:15.143343+0000","last_clean":"2024-04-26T21:35:15.143343+0000","last_became_active":"2024-04-26T21:35:10.123919+0000","last_became_peered":"2024-04-26T21:35:10.123919+0000","last_unstale":"2024-04-26T21:35:15.143343+0000","last_undegraded":"2024-04-26T21:35:15.143343+0000","last_fullsized":"2024-04-26T21:35:15.143343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143962+0000","last_change":"2024-04-26T21:35:10.124821+0000","last_active":"2024-04-26T21:35:15.143962+0000","last_peered":"2024-04-26T21:35:15.143962+0000","last_clean":"2024-04-26T21:35:15.143962+0000","last_became_active":"2024-04-26T21:35:10.124245+0000","last_became_peered":"2024-04-26T21:35:10.124245+0000","last_unstale":"2024-04-26T21:35:15.143962+0000","last_undegraded":"2024-04-26T21:35:15.143962+0000","last_fullsized":"2024-04-26T21:35:15.143962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876099+0000","last_change":"2024-04-26T21:35:10.131624+0000","last_active":"2024-04-26T21:35:15.876099+0000","last_peered":"2024-04-26T21:35:15.876099+0000","last_clean":"2024-04-26T21:35:15.876099+0000","last_became_active":"2024-04-26T21:35:10.131392+0000","last_became_peered":"2024-04-26T21:35:10.131392+0000","last_unstale":"2024-04-26T21:35:15.876099+0000","last_undegraded":"2024-04-26T21:35:15.876099+0000","last_fullsized":"2024-04-26T21:35:15.876099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:33.396034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003337+0000","last_change":"2024-04-26T21:35:10.127875+0000","last_active":"2024-04-26T21:35:16.003337+0000","last_peered":"2024-04-26T21:35:16.003337+0000","last_clean":"2024-04-26T21:35:16.003337+0000","last_became_active":"2024-04-26T21:35:10.126566+0000","last_became_peered":"2024-04-26T21:35:10.126566+0000","last_unstale":"2024-04-26T21:35:16.003337+0000","last_undegraded":"2024-04-26T21:35:16.003337+0000","last_fullsized":"2024-04-26T21:35:16.003337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875494+0000","last_change":"2024-04-26T21:35:10.126677+0000","last_active":"2024-04-26T21:35:15.875494+0000","last_peered":"2024-04-26T21:35:15.875494+0000","last_clean":"2024-04-26T21:35:15.875494+0000","last_became_active":"2024-04-26T21:35:10.126388+0000","last_became_peered":"2024-04-26T21:35:10.126388+0000","last_unstale":"2024-04-26T21:35:15.875494+0000","last_undegraded":"2024-04-26T21:35:15.875494+0000","last_fullsized":"2024-04-26T21:35:15.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143757+0000","last_change":"2024-04-26T21:35:10.125406+0000","last_active":"2024-04-26T21:35:15.143757+0000","last_peered":"2024-04-26T21:35:15.143757+0000","last_clean":"2024-04-26T21:35:15.143757+0000","last_became_active":"2024-04-26T21:35:10.125143+0000","last_became_peered":"2024-04-26T21:35:10.125143+0000","last_unstale":"2024-04-26T21:35:15.143757+0000","last_undegraded":"2024-04-26T21:35:15.143757+0000","last_fullsized":"2024-04-26T21:35:15.143757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.130632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875494+0000","last_change":"2024-04-26T21:35:10.127947+0000","last_active":"2024-04-26T21:35:15.875494+0000","last_peered":"2024-04-26T21:35:15.875494+0000","last_clean":"2024-04-26T21:35:15.875494+0000","last_became_active":"2024-04-26T21:35:10.127700+0000","last_became_peered":"2024-04-26T21:35:10.127700+0000","last_unstale":"2024-04-26T21:35:15.875494+0000","last_undegraded":"2024-04-26T21:35:15.875494+0000","last_fullsized":"2024-04-26T21:35:15.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143721+0000","last_change":"2024-04-26T21:35:10.121445+0000","last_active":"2024-04-26T21:35:15.143721+0000","last_peered":"2024-04-26T21:35:15.143721+0000","last_clean":"2024-04-26T21:35:15.143721+0000","last_became_active":"2024-04-26T21:35:10.119356+0000","last_became_peered":"2024-04-26T21:35:10.119356+0000","last_unstale":"2024-04-26T21:35:15.143721+0000","last_undegraded":"2024-04-26T21:35:15.143721+0000","last_fullsized":"2024-04-26T21:35:15.143721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:27.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003601+0000","last_change":"2024-04-26T21:35:10.124758+0000","last_active":"2024-04-26T21:35:16.003601+0000","last_peered":"2024-04-26T21:35:16.003601+0000","last_clean":"2024-04-26T21:35:16.003601+0000","last_became_active":"2024-04-26T21:35:10.124480+0000","last_became_peered":"2024-04-26T21:35:10.124480+0000","last_unstale":"2024-04-26T21:35:16.003601+0000","last_undegraded":"2024-04-26T21:35:16.003601+0000","last_fullsized":"2024-04-26T21:35:16.003601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:29.764634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374098+0000","last_change":"2024-04-26T21:35:10.125937+0000","last_active":"2024-04-26T21:35:15.374098+0000","last_peered":"2024-04-26T21:35:15.374098+0000","last_clean":"2024-04-26T21:35:15.374098+0000","last_became_active":"2024-04-26T21:35:10.125640+0000","last_became_peered":"2024-04-26T21:35:10.125640+0000","last_unstale":"2024-04-26T21:35:15.374098+0000","last_undegraded":"2024-04-26T21:35:15.374098+0000","last_fullsized":"2024-04-26T21:35:15.374098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876035+0000","last_change":"2024-04-26T21:35:10.129050+0000","last_active":"2024-04-26T21:35:15.876035+0000","last_peered":"2024-04-26T21:35:15.876035+0000","last_clean":"2024-04-26T21:35:15.876035+0000","last_became_active":"2024-04-26T21:35:10.128720+0000","last_became_peered":"2024-04-26T21:35:10.128720+0000","last_unstale":"2024-04-26T21:35:15.876035+0000","last_undegraded":"2024-04-26T21:35:15.876035+0000","last_fullsized":"2024-04-26T21:35:15.876035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:33.320854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:35:37.685 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:35:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.821+0000 7f3c754b0640 1 Processor -- start 2024-04-26T21:35:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.821+0000 7f3c754b0640 1 -- start start 2024-04-26T21:35:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c754b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c7015f3a0 0x7f3c700d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c754b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3c700d1770 0x7f3c700d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c754b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c700d2590 con 0x7f3c700d2160 2024-04-26T21:35:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c754b0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3c700d0e20 con 0x7f3c7015f3a0 2024-04-26T21:35:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c754b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3c700b1d70 con 0x7f3c700d1770 2024-04-26T21:35:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c6effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c7015f3a0 0x7f3c700d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c6effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c7015f3a0 0x7f3c700d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45866/0 (socket says 172.21.15.40:45866) 2024-04-26T21:35:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c6effd640 1 -- 172.21.15.40:0/3869670215 learned_addr learned my addr 172.21.15.40:0/3869670215 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c6e7fc640 1 --2- 172.21.15.40:0/3869670215 >> v2:172.21.15.184:3300/0 conn(0x7f3c700d1770 0x7f3c700d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c6effd640 1 -- 172.21.15.40:0/3869670215 >> v2:172.21.15.184:3300/0 conn(0x7f3c700d1770 msgr2=0x7f3c700d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c6effd640 1 --2- 172.21.15.40:0/3869670215 >> v2:172.21.15.184:3300/0 conn(0x7f3c700d1770 0x7f3c700d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c6effd640 1 -- 172.21.15.40:0/3869670215 >> v1:172.21.15.184:6789/0 conn(0x7f3c700d2160 legacy=0x7f3c700d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.822+0000 7f3c6effd640 1 -- 172.21.15.40:0/3869670215 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700b2050 con 0x7f3c7015f3a0 2024-04-26T21:35:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.823+0000 7f3c6effd640 1 --2- 172.21.15.40:0/3869670215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700d06f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3c700b7b80 tx=0x7f3c6002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a76ae756194eb43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.823+0000 7f3c6dffb640 1 -- 172.21.15.40:0/3869670215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60037070 con 0x7f3c7015f3a0 2024-04-26T21:35:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.823+0000 7f3c6dffb640 1 -- 172.21.15.40:0/3869670215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c60032440 con 0x7f3c7015f3a0 2024-04-26T21:35:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.823+0000 7f3c6dffb640 1 -- 172.21.15.40:0/3869670215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60032810 con 0x7f3c7015f3a0 2024-04-26T21:35:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.824+0000 7f3c754b0640 1 -- 172.21.15.40:0/3869670215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 msgr2=0x7f3c700d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.824+0000 7f3c754b0640 1 --2- 172.21.15.40:0/3869670215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700d06f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3c700b7b80 tx=0x7f3c6002a230 comp rx=0 tx=0).stop 2024-04-26T21:35:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.824+0000 7f3c754b0640 1 -- 172.21.15.40:0/3869670215 shutdown_connections 2024-04-26T21:35:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.824+0000 7f3c754b0640 1 --2- 172.21.15.40:0/3869670215 >> v2:172.21.15.184:3300/0 conn(0x7f3c700d1770 0x7f3c700d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.824+0000 7f3c754b0640 1 --2- 172.21.15.40:0/3869670215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700d06f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.824+0000 7f3c754b0640 1 -- 172.21.15.40:0/3869670215 >> 172.21.15.40:0/3869670215 conn(0x7f3c700d4f90 msgr2=0x7f3c700d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.824+0000 7f3c754b0640 1 -- 172.21.15.40:0/3869670215 shutdown_connections 2024-04-26T21:35:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.824+0000 7f3c754b0640 1 -- 172.21.15.40:0/3869670215 wait complete. 2024-04-26T21:35:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.825+0000 7f3c754b0640 1 Processor -- start 2024-04-26T21:35:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.825+0000 7f3c754b0640 1 -- start start 2024-04-26T21:35:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c754b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c700d1770 0x7f3c701f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c754b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c700d2160 0x7f3c701f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c754b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c754b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7015f900 con 0x7f3c7015f3a0 2024-04-26T21:35:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c754b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7015f780 con 0x7f3c700d2160 2024-04-26T21:35:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c754b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3c700b3860 con 0x7f3c700d1770 2024-04-26T21:35:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45882/0 (socket says 172.21.15.40:45882) 2024-04-26T21:35:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 -- 172.21.15.40:0/2179962169 learned_addr learned my addr 172.21.15.40:0/2179962169 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6e7fc640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c700d2160 0x7f3c701f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6effd640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c700d1770 0x7f3c701f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 -- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c700d1770 msgr2=0x7f3c701f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c700d1770 0x7f3c701f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 -- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c700d2160 msgr2=0x7f3c701f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c700d2160 0x7f3c701f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 -- 172.21.15.40:0/2179962169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700b8e70 con 0x7f3c7015f3a0 2024-04-26T21:35:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.826+0000 7f3c6f7fe640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700b8930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800c780 tx=0x7f3c6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.827+0000 7f3c577fe640 1 -- 172.21.15.40:0/2179962169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68004b60 con 0x7f3c7015f3a0 2024-04-26T21:35:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.827+0000 7f3c577fe640 1 -- 172.21.15.40:0/2179962169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c68004d00 con 0x7f3c7015f3a0 2024-04-26T21:35:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.827+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c700b93d0 con 0x7f3c7015f3a0 2024-04-26T21:35:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.827+0000 7f3c577fe640 1 -- 172.21.15.40:0/2179962169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800aa90 con 0x7f3c7015f3a0 2024-04-26T21:35:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.827+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701f5020 con 0x7f3c7015f3a0 2024-04-26T21:35:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.828+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c700694f0 con 0x7f3c7015f3a0 2024-04-26T21:35:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.828+0000 7f3c577fe640 1 -- 172.21.15.40:0/2179962169 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c6800ac30 con 0x7f3c7015f3a0 2024-04-26T21:35:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.829+0000 7f3c577fe640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.829+0000 7f3c577fe640 1 -- 172.21.15.40:0/2179962169 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3c6808c240 con 0x7f3c7015f3a0 2024-04-26T21:35:37.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.829+0000 7f3c6effd640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:37.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.829+0000 7f3c6effd640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c60034000 tx=0x7f3c6002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:37.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:37.833+0000 7f3c577fe640 1 -- 172.21.15.40:0/2179962169 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c700b8e70 con 0x7f3c7015f3a0 2024-04-26T21:35:38.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.015+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c700b6860 con 0x7f3c5c06c8f0 2024-04-26T21:35:38.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.024+0000 7f3c577fe640 1 -- 172.21.15.40:0/2179962169 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f3c700b6860 con 0x7f3c5c06c8f0 2024-04-26T21:35:38.028 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:38.030 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c5c06c8f0 msgr2=0x7f3c5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c60034000 tx=0x7f3c6002e000 comp rx=0 tx=0).stop 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 msgr2=0x7f3c700b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700b8930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800c780 tx=0x7f3c6800cc40 comp rx=0 tx=0).stop 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 shutdown_connections 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c7015f3a0 0x7f3c700b8930 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c700d2160 0x7f3c701f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 --2- 172.21.15.40:0/2179962169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c700d1770 0x7f3c701f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 >> 172.21.15.40:0/2179962169 conn(0x7f3c700d4f90 msgr2=0x7f3c7006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 shutdown_connections 2024-04-26T21:35:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.027+0000 7f3c754b0640 1 -- 172.21.15.40:0/2179962169 wait complete. 2024-04-26T21:35:38.039 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-26T21:35:38.000477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219400,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755320,"statfs":{"total":773094113280,"available":772869447680,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003498"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004303+0000","last_change":"2024-04-26T21:35:09.132596+0000","last_active":"2024-04-26T21:35:16.004303+0000","last_peered":"2024-04-26T21:35:16.004303+0000","last_clean":"2024-04-26T21:35:16.004303+0000","last_became_active":"2024-04-26T21:35:09.132270+0000","last_became_peered":"2024-04-26T21:35:09.132270+0000","last_unstale":"2024-04-26T21:35:16.004303+0000","last_undegraded":"2024-04-26T21:35:16.004303+0000","last_fullsized":"2024-04-26T21:35:16.004303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:22.652595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003605+0000","last_change":"2024-04-26T21:35:10.126640+0000","last_active":"2024-04-26T21:35:16.003605+0000","last_peered":"2024-04-26T21:35:16.003605+0000","last_clean":"2024-04-26T21:35:16.003605+0000","last_became_active":"2024-04-26T21:35:10.125603+0000","last_became_peered":"2024-04-26T21:35:10.125603+0000","last_unstale":"2024-04-26T21:35:16.003605+0000","last_undegraded":"2024-04-26T21:35:16.003605+0000","last_fullsized":"2024-04-26T21:35:16.003605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:27.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004487+0000","last_change":"2024-04-26T21:35:14.280943+0000","last_active":"2024-04-26T21:35:16.004487+0000","last_peered":"2024-04-26T21:35:16.004487+0000","last_clean":"2024-04-26T21:35:16.004487+0000","last_became_active":"2024-04-26T21:35:14.280611+0000","last_became_peered":"2024-04-26T21:35:14.280611+0000","last_unstale":"2024-04-26T21:35:16.004487+0000","last_undegraded":"2024-04-26T21:35:16.004487+0000","last_fullsized":"2024-04-26T21:35:16.004487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:28.734397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004488+0000","last_change":"2024-04-26T21:35:09.122996+0000","last_active":"2024-04-26T21:35:16.004488+0000","last_peered":"2024-04-26T21:35:16.004488+0000","last_clean":"2024-04-26T21:35:16.004488+0000","last_became_active":"2024-04-26T21:35:09.122682+0000","last_became_peered":"2024-04-26T21:35:09.122682+0000","last_unstale":"2024-04-26T21:35:16.004488+0000","last_undegraded":"2024-04-26T21:35:16.004488+0000","last_fullsized":"2024-04-26T21:35:16.004488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:44.358668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004023+0000","last_change":"2024-04-26T21:35:10.133464+0000","last_active":"2024-04-26T21:35:16.004023+0000","last_peered":"2024-04-26T21:35:16.004023+0000","last_clean":"2024-04-26T21:35:16.004023+0000","last_became_active":"2024-04-26T21:35:10.133170+0000","last_became_peered":"2024-04-26T21:35:10.133170+0000","last_unstale":"2024-04-26T21:35:16.004023+0000","last_undegraded":"2024-04-26T21:35:16.004023+0000","last_fullsized":"2024-04-26T21:35:16.004023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003606+0000","last_change":"2024-04-26T21:35:14.163746+0000","last_active":"2024-04-26T21:35:16.003606+0000","last_peered":"2024-04-26T21:35:16.003606+0000","last_clean":"2024-04-26T21:35:16.003606+0000","last_became_active":"2024-04-26T21:35:14.163263+0000","last_became_peered":"2024-04-26T21:35:14.163263+0000","last_unstale":"2024-04-26T21:35:16.003606+0000","last_undegraded":"2024-04-26T21:35:16.003606+0000","last_fullsized":"2024-04-26T21:35:16.003606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:28.472320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875887+0000","last_change":"2024-04-26T21:35:09.129656+0000","last_active":"2024-04-26T21:35:15.875887+0000","last_peered":"2024-04-26T21:35:15.875887+0000","last_clean":"2024-04-26T21:35:15.875887+0000","last_became_active":"2024-04-26T21:35:09.129379+0000","last_became_peered":"2024-04-26T21:35:09.129379+0000","last_unstale":"2024-04-26T21:35:15.875887+0000","last_undegraded":"2024-04-26T21:35:15.875887+0000","last_fullsized":"2024-04-26T21:35:15.875887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:51.639722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143824+0000","last_change":"2024-04-26T21:35:10.119557+0000","last_active":"2024-04-26T21:35:15.143824+0000","last_peered":"2024-04-26T21:35:15.143824+0000","last_clean":"2024-04-26T21:35:15.143824+0000","last_became_active":"2024-04-26T21:35:10.118407+0000","last_became_peered":"2024-04-26T21:35:10.118407+0000","last_unstale":"2024-04-26T21:35:15.143824+0000","last_undegraded":"2024-04-26T21:35:15.143824+0000","last_fullsized":"2024-04-26T21:35:15.143824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003821+0000","last_change":"2024-04-26T21:35:14.159166+0000","last_active":"2024-04-26T21:35:16.003821+0000","last_peered":"2024-04-26T21:35:16.003821+0000","last_clean":"2024-04-26T21:35:16.003821+0000","last_became_active":"2024-04-26T21:35:14.158592+0000","last_became_peered":"2024-04-26T21:35:14.158592+0000","last_unstale":"2024-04-26T21:35:16.003821+0000","last_undegraded":"2024-04-26T21:35:16.003821+0000","last_fullsized":"2024-04-26T21:35:16.003821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:31.201343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875768+0000","last_change":"2024-04-26T21:35:09.128770+0000","last_active":"2024-04-26T21:35:15.875768+0000","last_peered":"2024-04-26T21:35:15.875768+0000","last_clean":"2024-04-26T21:35:15.875768+0000","last_became_active":"2024-04-26T21:35:09.128385+0000","last_became_peered":"2024-04-26T21:35:09.128385+0000","last_unstale":"2024-04-26T21:35:15.875768+0000","last_undegraded":"2024-04-26T21:35:15.875768+0000","last_fullsized":"2024-04-26T21:35:15.875768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:20.201305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144355+0000","last_change":"2024-04-26T21:35:10.117537+0000","last_active":"2024-04-26T21:35:15.144355+0000","last_peered":"2024-04-26T21:35:15.144355+0000","last_clean":"2024-04-26T21:35:15.144355+0000","last_became_active":"2024-04-26T21:35:10.117267+0000","last_became_peered":"2024-04-26T21:35:10.117267+0000","last_unstale":"2024-04-26T21:35:15.144355+0000","last_undegraded":"2024-04-26T21:35:15.144355+0000","last_fullsized":"2024-04-26T21:35:15.144355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:01.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003840+0000","last_change":"2024-04-26T21:35:14.161136+0000","last_active":"2024-04-26T21:35:16.003840+0000","last_peered":"2024-04-26T21:35:16.003840+0000","last_clean":"2024-04-26T21:35:16.003840+0000","last_became_active":"2024-04-26T21:35:14.160648+0000","last_became_peered":"2024-04-26T21:35:14.160648+0000","last_unstale":"2024-04-26T21:35:16.003840+0000","last_undegraded":"2024-04-26T21:35:16.003840+0000","last_fullsized":"2024-04-26T21:35:16.003840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:10.381659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143344+0000","last_change":"2024-04-26T21:35:10.119355+0000","last_active":"2024-04-26T21:35:15.143344+0000","last_peered":"2024-04-26T21:35:15.143344+0000","last_clean":"2024-04-26T21:35:15.143344+0000","last_became_active":"2024-04-26T21:35:10.119067+0000","last_became_peered":"2024-04-26T21:35:10.119067+0000","last_unstale":"2024-04-26T21:35:15.143344+0000","last_undegraded":"2024-04-26T21:35:15.143344+0000","last_fullsized":"2024-04-26T21:35:15.143344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876041+0000","last_change":"2024-04-26T21:35:09.129853+0000","last_active":"2024-04-26T21:35:15.876041+0000","last_peered":"2024-04-26T21:35:15.876041+0000","last_clean":"2024-04-26T21:35:15.876041+0000","last_became_active":"2024-04-26T21:35:09.129500+0000","last_became_peered":"2024-04-26T21:35:09.129500+0000","last_unstale":"2024-04-26T21:35:15.876041+0000","last_undegraded":"2024-04-26T21:35:15.876041+0000","last_fullsized":"2024-04-26T21:35:15.876041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:54.605573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004307+0000","last_change":"2024-04-26T21:35:14.161264+0000","last_active":"2024-04-26T21:35:16.004307+0000","last_peered":"2024-04-26T21:35:16.004307+0000","last_clean":"2024-04-26T21:35:16.004307+0000","last_became_active":"2024-04-26T21:35:14.160617+0000","last_became_peered":"2024-04-26T21:35:14.160617+0000","last_unstale":"2024-04-26T21:35:16.004307+0000","last_undegraded":"2024-04-26T21:35:16.004307+0000","last_fullsized":"2024-04-26T21:35:16.004307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004050+0000","last_change":"2024-04-26T21:35:10.125998+0000","last_active":"2024-04-26T21:35:16.004050+0000","last_peered":"2024-04-26T21:35:16.004050+0000","last_clean":"2024-04-26T21:35:16.004050+0000","last_became_active":"2024-04-26T21:35:10.125704+0000","last_became_peered":"2024-04-26T21:35:10.125704+0000","last_unstale":"2024-04-26T21:35:16.004050+0000","last_undegraded":"2024-04-26T21:35:16.004050+0000","last_fullsized":"2024-04-26T21:35:16.004050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373953+0000","last_change":"2024-04-26T21:35:09.123003+0000","last_active":"2024-04-26T21:35:15.373953+0000","last_peered":"2024-04-26T21:35:15.373953+0000","last_clean":"2024-04-26T21:35:15.373953+0000","last_became_active":"2024-04-26T21:35:09.122610+0000","last_became_peered":"2024-04-26T21:35:09.122610+0000","last_unstale":"2024-04-26T21:35:15.373953+0000","last_undegraded":"2024-04-26T21:35:15.373953+0000","last_fullsized":"2024-04-26T21:35:15.373953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004080+0000","last_change":"2024-04-26T21:35:14.164010+0000","last_active":"2024-04-26T21:35:16.004080+0000","last_peered":"2024-04-26T21:35:16.004080+0000","last_clean":"2024-04-26T21:35:16.004080+0000","last_became_active":"2024-04-26T21:35:14.163262+0000","last_became_peered":"2024-04-26T21:35:14.163262+0000","last_unstale":"2024-04-26T21:35:16.004080+0000","last_undegraded":"2024-04-26T21:35:16.004080+0000","last_fullsized":"2024-04-26T21:35:16.004080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:45.829307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003626+0000","last_change":"2024-04-26T21:35:10.120565+0000","last_active":"2024-04-26T21:35:16.003626+0000","last_peered":"2024-04-26T21:35:16.003626+0000","last_clean":"2024-04-26T21:35:16.003626+0000","last_became_active":"2024-04-26T21:35:10.120133+0000","last_became_peered":"2024-04-26T21:35:10.120133+0000","last_unstale":"2024-04-26T21:35:16.003626+0000","last_undegraded":"2024-04-26T21:35:16.003626+0000","last_fullsized":"2024-04-26T21:35:16.003626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:06.354261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373691+0000","last_change":"2024-04-26T21:35:09.127198+0000","last_active":"2024-04-26T21:35:15.373691+0000","last_peered":"2024-04-26T21:35:15.373691+0000","last_clean":"2024-04-26T21:35:15.373691+0000","last_became_active":"2024-04-26T21:35:09.126870+0000","last_became_peered":"2024-04-26T21:35:09.126870+0000","last_unstale":"2024-04-26T21:35:15.373691+0000","last_undegraded":"2024-04-26T21:35:15.373691+0000","last_fullsized":"2024-04-26T21:35:15.373691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374142+0000","last_change":"2024-04-26T21:35:14.277851+0000","last_active":"2024-04-26T21:35:15.374142+0000","last_peered":"2024-04-26T21:35:15.374142+0000","last_clean":"2024-04-26T21:35:15.374142+0000","last_became_active":"2024-04-26T21:35:14.277461+0000","last_became_peered":"2024-04-26T21:35:14.277461+0000","last_unstale":"2024-04-26T21:35:15.374142+0000","last_undegraded":"2024-04-26T21:35:15.374142+0000","last_fullsized":"2024-04-26T21:35:15.374142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:20.536176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004022+0000","last_change":"2024-04-26T21:35:10.113247+0000","last_active":"2024-04-26T21:35:16.004022+0000","last_peered":"2024-04-26T21:35:16.004022+0000","last_clean":"2024-04-26T21:35:16.004022+0000","last_became_active":"2024-04-26T21:35:10.112937+0000","last_became_peered":"2024-04-26T21:35:10.112937+0000","last_unstale":"2024-04-26T21:35:16.004022+0000","last_undegraded":"2024-04-26T21:35:16.004022+0000","last_fullsized":"2024-04-26T21:35:16.004022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:47.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875388+0000","last_change":"2024-04-26T21:35:09.134099+0000","last_active":"2024-04-26T21:35:15.875388+0000","last_peered":"2024-04-26T21:35:15.875388+0000","last_clean":"2024-04-26T21:35:15.875388+0000","last_became_active":"2024-04-26T21:35:09.133869+0000","last_became_peered":"2024-04-26T21:35:09.133869+0000","last_unstale":"2024-04-26T21:35:15.875388+0000","last_undegraded":"2024-04-26T21:35:15.875388+0000","last_fullsized":"2024-04-26T21:35:15.875388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:16.219471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004280+0000","last_change":"2024-04-26T21:35:14.158160+0000","last_active":"2024-04-26T21:35:16.004280+0000","last_peered":"2024-04-26T21:35:16.004280+0000","last_clean":"2024-04-26T21:35:16.004280+0000","last_became_active":"2024-04-26T21:35:14.157711+0000","last_became_peered":"2024-04-26T21:35:14.157711+0000","last_unstale":"2024-04-26T21:35:16.004280+0000","last_undegraded":"2024-04-26T21:35:16.004280+0000","last_fullsized":"2024-04-26T21:35:16.004280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:26.153246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004196+0000","last_change":"2024-04-26T21:35:10.136384+0000","last_active":"2024-04-26T21:35:16.004196+0000","last_peered":"2024-04-26T21:35:16.004196+0000","last_clean":"2024-04-26T21:35:16.004196+0000","last_became_active":"2024-04-26T21:35:10.135589+0000","last_became_peered":"2024-04-26T21:35:10.135589+0000","last_unstale":"2024-04-26T21:35:16.004196+0000","last_undegraded":"2024-04-26T21:35:16.004196+0000","last_fullsized":"2024-04-26T21:35:16.004196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144087+0000","last_change":"2024-04-26T21:35:09.125486+0000","last_active":"2024-04-26T21:35:15.144087+0000","last_peered":"2024-04-26T21:35:15.144087+0000","last_clean":"2024-04-26T21:35:15.144087+0000","last_became_active":"2024-04-26T21:35:09.125019+0000","last_became_peered":"2024-04-26T21:35:09.125019+0000","last_unstale":"2024-04-26T21:35:15.144087+0000","last_undegraded":"2024-04-26T21:35:15.144087+0000","last_fullsized":"2024-04-26T21:35:15.144087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003847+0000","last_change":"2024-04-26T21:35:14.281054+0000","last_active":"2024-04-26T21:35:16.003847+0000","last_peered":"2024-04-26T21:35:16.003847+0000","last_clean":"2024-04-26T21:35:16.003847+0000","last_became_active":"2024-04-26T21:35:14.278082+0000","last_became_peered":"2024-04-26T21:35:14.278082+0000","last_unstale":"2024-04-26T21:35:16.003847+0000","last_undegraded":"2024-04-26T21:35:16.003847+0000","last_fullsized":"2024-04-26T21:35:16.003847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.959963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004866+0000","last_change":"2024-04-26T21:35:10.130908+0000","last_active":"2024-04-26T21:35:16.004866+0000","last_peered":"2024-04-26T21:35:16.004866+0000","last_clean":"2024-04-26T21:35:16.004866+0000","last_became_active":"2024-04-26T21:35:10.130448+0000","last_became_peered":"2024-04-26T21:35:10.130448+0000","last_unstale":"2024-04-26T21:35:16.004866+0000","last_undegraded":"2024-04-26T21:35:16.004866+0000","last_fullsized":"2024-04-26T21:35:16.004866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374192+0000","last_change":"2024-04-26T21:35:09.124758+0000","last_active":"2024-04-26T21:35:15.374192+0000","last_peered":"2024-04-26T21:35:15.374192+0000","last_clean":"2024-04-26T21:35:15.374192+0000","last_became_active":"2024-04-26T21:35:09.124446+0000","last_became_peered":"2024-04-26T21:35:09.124446+0000","last_unstale":"2024-04-26T21:35:15.374192+0000","last_undegraded":"2024-04-26T21:35:15.374192+0000","last_fullsized":"2024-04-26T21:35:15.374192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:16.219471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003317+0000","last_change":"2024-04-26T21:35:14.280487+0000","last_active":"2024-04-26T21:35:16.003317+0000","last_peered":"2024-04-26T21:35:16.003317+0000","last_clean":"2024-04-26T21:35:16.003317+0000","last_became_active":"2024-04-26T21:35:14.280075+0000","last_became_peered":"2024-04-26T21:35:14.280075+0000","last_unstale":"2024-04-26T21:35:16.003317+0000","last_undegraded":"2024-04-26T21:35:16.003317+0000","last_fullsized":"2024-04-26T21:35:16.003317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:49.955325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373962+0000","last_change":"2024-04-26T21:35:10.124638+0000","last_active":"2024-04-26T21:35:15.373962+0000","last_peered":"2024-04-26T21:35:15.373962+0000","last_clean":"2024-04-26T21:35:15.373962+0000","last_became_active":"2024-04-26T21:35:10.124298+0000","last_became_peered":"2024-04-26T21:35:10.124298+0000","last_unstale":"2024-04-26T21:35:15.373962+0000","last_undegraded":"2024-04-26T21:35:15.373962+0000","last_fullsized":"2024-04-26T21:35:15.373962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876132+0000","last_change":"2024-04-26T21:35:09.134096+0000","last_active":"2024-04-26T21:35:15.876132+0000","last_peered":"2024-04-26T21:35:15.876132+0000","last_clean":"2024-04-26T21:35:15.876132+0000","last_became_active":"2024-04-26T21:35:09.133816+0000","last_became_peered":"2024-04-26T21:35:09.133816+0000","last_unstale":"2024-04-26T21:35:15.876132+0000","last_undegraded":"2024-04-26T21:35:15.876132+0000","last_fullsized":"2024-04-26T21:35:15.876132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:39.887861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004297+0000","last_change":"2024-04-26T21:35:14.277888+0000","last_active":"2024-04-26T21:35:16.004297+0000","last_peered":"2024-04-26T21:35:16.004297+0000","last_clean":"2024-04-26T21:35:16.004297+0000","last_became_active":"2024-04-26T21:35:14.277473+0000","last_became_peered":"2024-04-26T21:35:14.277473+0000","last_unstale":"2024-04-26T21:35:16.004297+0000","last_undegraded":"2024-04-26T21:35:16.004297+0000","last_fullsized":"2024-04-26T21:35:16.004297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:20.536176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144502+0000","last_change":"2024-04-26T21:35:10.118766+0000","last_active":"2024-04-26T21:35:15.144502+0000","last_peered":"2024-04-26T21:35:15.144502+0000","last_clean":"2024-04-26T21:35:15.144502+0000","last_became_active":"2024-04-26T21:35:10.118480+0000","last_became_peered":"2024-04-26T21:35:10.118480+0000","last_unstale":"2024-04-26T21:35:15.144502+0000","last_undegraded":"2024-04-26T21:35:15.144502+0000","last_fullsized":"2024-04-26T21:35:15.144502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:33.320854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374086+0000","last_change":"2024-04-26T21:35:09.124939+0000","last_active":"2024-04-26T21:35:15.374086+0000","last_peered":"2024-04-26T21:35:15.374086+0000","last_clean":"2024-04-26T21:35:15.374086+0000","last_became_active":"2024-04-26T21:35:09.124630+0000","last_became_peered":"2024-04-26T21:35:09.124630+0000","last_unstale":"2024-04-26T21:35:15.374086+0000","last_undegraded":"2024-04-26T21:35:15.374086+0000","last_fullsized":"2024-04-26T21:35:15.374086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374121+0000","last_change":"2024-04-26T21:35:14.165015+0000","last_active":"2024-04-26T21:35:15.374121+0000","last_peered":"2024-04-26T21:35:15.374121+0000","last_clean":"2024-04-26T21:35:15.374121+0000","last_became_active":"2024-04-26T21:35:14.164615+0000","last_became_peered":"2024-04-26T21:35:14.164615+0000","last_unstale":"2024-04-26T21:35:15.374121+0000","last_undegraded":"2024-04-26T21:35:15.374121+0000","last_fullsized":"2024-04-26T21:35:15.374121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:37.348252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004379+0000","last_change":"2024-04-26T21:35:10.125981+0000","last_active":"2024-04-26T21:35:16.004379+0000","last_peered":"2024-04-26T21:35:16.004379+0000","last_clean":"2024-04-26T21:35:16.004379+0000","last_became_active":"2024-04-26T21:35:10.125718+0000","last_became_peered":"2024-04-26T21:35:10.125718+0000","last_unstale":"2024-04-26T21:35:16.004379+0000","last_undegraded":"2024-04-26T21:35:16.004379+0000","last_fullsized":"2024-04-26T21:35:16.004379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:09.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144282+0000","last_change":"2024-04-26T21:35:09.121086+0000","last_active":"2024-04-26T21:35:15.144282+0000","last_peered":"2024-04-26T21:35:15.144282+0000","last_clean":"2024-04-26T21:35:15.144282+0000","last_became_active":"2024-04-26T21:35:09.120443+0000","last_became_peered":"2024-04-26T21:35:09.120443+0000","last_unstale":"2024-04-26T21:35:15.144282+0000","last_undegraded":"2024-04-26T21:35:15.144282+0000","last_fullsized":"2024-04-26T21:35:15.144282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374085+0000","last_change":"2024-04-26T21:35:14.163974+0000","last_active":"2024-04-26T21:35:15.374085+0000","last_peered":"2024-04-26T21:35:15.374085+0000","last_clean":"2024-04-26T21:35:15.374085+0000","last_became_active":"2024-04-26T21:35:14.163558+0000","last_became_peered":"2024-04-26T21:35:14.163558+0000","last_unstale":"2024-04-26T21:35:15.374085+0000","last_undegraded":"2024-04-26T21:35:15.374085+0000","last_fullsized":"2024-04-26T21:35:15.374085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:01.158030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373556+0000","last_change":"2024-04-26T21:35:10.124855+0000","last_active":"2024-04-26T21:35:15.373556+0000","last_peered":"2024-04-26T21:35:15.373556+0000","last_clean":"2024-04-26T21:35:15.373556+0000","last_became_active":"2024-04-26T21:35:10.124588+0000","last_became_peered":"2024-04-26T21:35:10.124588+0000","last_unstale":"2024-04-26T21:35:15.373556+0000","last_undegraded":"2024-04-26T21:35:15.373556+0000","last_fullsized":"2024-04-26T21:35:15.373556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142012+0000","last_change":"2024-04-26T21:35:09.115909+0000","last_active":"2024-04-26T21:35:15.142012+0000","last_peered":"2024-04-26T21:35:15.142012+0000","last_clean":"2024-04-26T21:35:15.142012+0000","last_became_active":"2024-04-26T21:35:09.115120+0000","last_became_peered":"2024-04-26T21:35:09.115120+0000","last_unstale":"2024-04-26T21:35:15.142012+0000","last_undegraded":"2024-04-26T21:35:15.142012+0000","last_fullsized":"2024-04-26T21:35:15.142012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374302+0000","last_change":"2024-04-26T21:35:14.165160+0000","last_active":"2024-04-26T21:35:15.374302+0000","last_peered":"2024-04-26T21:35:15.374302+0000","last_clean":"2024-04-26T21:35:15.374302+0000","last_became_active":"2024-04-26T21:35:14.164683+0000","last_became_peered":"2024-04-26T21:35:14.164683+0000","last_unstale":"2024-04-26T21:35:15.374302+0000","last_undegraded":"2024-04-26T21:35:15.374302+0000","last_fullsized":"2024-04-26T21:35:15.374302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:37.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003512+0000","last_change":"2024-04-26T21:35:10.128846+0000","last_active":"2024-04-26T21:35:16.003512+0000","last_peered":"2024-04-26T21:35:16.003512+0000","last_clean":"2024-04-26T21:35:16.003512+0000","last_became_active":"2024-04-26T21:35:10.128576+0000","last_became_peered":"2024-04-26T21:35:10.128576+0000","last_unstale":"2024-04-26T21:35:16.003512+0000","last_undegraded":"2024-04-26T21:35:16.003512+0000","last_fullsized":"2024-04-26T21:35:16.003512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:40.892489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875388+0000","last_change":"2024-04-26T21:35:09.130374+0000","last_active":"2024-04-26T21:35:15.875388+0000","last_peered":"2024-04-26T21:35:15.875388+0000","last_clean":"2024-04-26T21:35:15.875388+0000","last_became_active":"2024-04-26T21:35:09.130122+0000","last_became_peered":"2024-04-26T21:35:09.130122+0000","last_unstale":"2024-04-26T21:35:15.875388+0000","last_undegraded":"2024-04-26T21:35:15.875388+0000","last_fullsized":"2024-04-26T21:35:15.875388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:25.035511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004078+0000","last_change":"2024-04-26T21:35:14.277956+0000","last_active":"2024-04-26T21:35:16.004078+0000","last_peered":"2024-04-26T21:35:16.004078+0000","last_clean":"2024-04-26T21:35:16.004078+0000","last_became_active":"2024-04-26T21:35:14.277555+0000","last_became_peered":"2024-04-26T21:35:14.277555+0000","last_unstale":"2024-04-26T21:35:16.004078+0000","last_undegraded":"2024-04-26T21:35:16.004078+0000","last_fullsized":"2024-04-26T21:35:16.004078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:37.348252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003728+0000","last_change":"2024-04-26T21:35:10.127546+0000","last_active":"2024-04-26T21:35:16.003728+0000","last_peered":"2024-04-26T21:35:16.003728+0000","last_clean":"2024-04-26T21:35:16.003728+0000","last_became_active":"2024-04-26T21:35:10.127263+0000","last_became_peered":"2024-04-26T21:35:10.127263+0000","last_unstale":"2024-04-26T21:35:16.003728+0000","last_undegraded":"2024-04-26T21:35:16.003728+0000","last_fullsized":"2024-04-26T21:35:16.003728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144006+0000","last_change":"2024-04-26T21:35:09.116290+0000","last_active":"2024-04-26T21:35:15.144006+0000","last_peered":"2024-04-26T21:35:15.144006+0000","last_clean":"2024-04-26T21:35:15.144006+0000","last_became_active":"2024-04-26T21:35:09.115972+0000","last_became_peered":"2024-04-26T21:35:09.115972+0000","last_unstale":"2024-04-26T21:35:15.144006+0000","last_undegraded":"2024-04-26T21:35:15.144006+0000","last_fullsized":"2024-04-26T21:35:15.144006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875528+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:35:15.875528+0000","last_peered":"2024-04-26T21:35:15.875528+0000","last_clean":"2024-04-26T21:35:15.875528+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:35:15.875528+0000","last_undegraded":"2024-04-26T21:35:15.875528+0000","last_fullsized":"2024-04-26T21:35:15.875528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143208+0000","last_change":"2024-04-26T21:35:14.279050+0000","last_active":"2024-04-26T21:35:15.143208+0000","last_peered":"2024-04-26T21:35:15.143208+0000","last_clean":"2024-04-26T21:35:15.143208+0000","last_became_active":"2024-04-26T21:35:14.278651+0000","last_became_peered":"2024-04-26T21:35:14.278651+0000","last_unstale":"2024-04-26T21:35:15.143208+0000","last_undegraded":"2024-04-26T21:35:15.143208+0000","last_fullsized":"2024-04-26T21:35:15.143208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:26.153246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875749+0000","last_change":"2024-04-26T21:35:10.130945+0000","last_active":"2024-04-26T21:35:15.875749+0000","last_peered":"2024-04-26T21:35:15.875749+0000","last_clean":"2024-04-26T21:35:15.875749+0000","last_became_active":"2024-04-26T21:35:10.130673+0000","last_became_peered":"2024-04-26T21:35:10.130673+0000","last_unstale":"2024-04-26T21:35:15.875749+0000","last_undegraded":"2024-04-26T21:35:15.875749+0000","last_fullsized":"2024-04-26T21:35:15.875749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:16.508778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142347+0000","last_change":"2024-04-26T21:35:09.118997+0000","last_active":"2024-04-26T21:35:15.142347+0000","last_peered":"2024-04-26T21:35:15.142347+0000","last_clean":"2024-04-26T21:35:15.142347+0000","last_became_active":"2024-04-26T21:35:09.118700+0000","last_became_peered":"2024-04-26T21:35:09.118700+0000","last_unstale":"2024-04-26T21:35:15.142347+0000","last_undegraded":"2024-04-26T21:35:15.142347+0000","last_fullsized":"2024-04-26T21:35:15.142347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:50.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143286+0000","last_change":"2024-04-26T21:35:14.277928+0000","last_active":"2024-04-26T21:35:15.143286+0000","last_peered":"2024-04-26T21:35:15.143286+0000","last_clean":"2024-04-26T21:35:15.143286+0000","last_became_active":"2024-04-26T21:35:14.277458+0000","last_became_peered":"2024-04-26T21:35:14.277458+0000","last_unstale":"2024-04-26T21:35:15.143286+0000","last_undegraded":"2024-04-26T21:35:15.143286+0000","last_fullsized":"2024-04-26T21:35:15.143286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:55.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144138+0000","last_change":"2024-04-26T21:35:10.120150+0000","last_active":"2024-04-26T21:35:15.144138+0000","last_peered":"2024-04-26T21:35:15.144138+0000","last_clean":"2024-04-26T21:35:15.144138+0000","last_became_active":"2024-04-26T21:35:10.119894+0000","last_became_peered":"2024-04-26T21:35:10.119894+0000","last_unstale":"2024-04-26T21:35:15.144138+0000","last_undegraded":"2024-04-26T21:35:15.144138+0000","last_fullsized":"2024-04-26T21:35:15.144138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.413835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374128+0000","last_change":"2024-04-26T21:35:09.125791+0000","last_active":"2024-04-26T21:35:15.374128+0000","last_peered":"2024-04-26T21:35:15.374128+0000","last_clean":"2024-04-26T21:35:15.374128+0000","last_became_active":"2024-04-26T21:35:09.125356+0000","last_became_peered":"2024-04-26T21:35:09.125356+0000","last_unstale":"2024-04-26T21:35:15.374128+0000","last_undegraded":"2024-04-26T21:35:15.374128+0000","last_fullsized":"2024-04-26T21:35:15.374128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:25.035511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004470+0000","last_change":"2024-04-26T21:35:14.160611+0000","last_active":"2024-04-26T21:35:16.004470+0000","last_peered":"2024-04-26T21:35:16.004470+0000","last_clean":"2024-04-26T21:35:16.004470+0000","last_became_active":"2024-04-26T21:35:14.160067+0000","last_became_peered":"2024-04-26T21:35:14.160067+0000","last_unstale":"2024-04-26T21:35:16.004470+0000","last_undegraded":"2024-04-26T21:35:16.004470+0000","last_fullsized":"2024-04-26T21:35:16.004470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:37.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876144+0000","last_change":"2024-04-26T21:35:10.136587+0000","last_active":"2024-04-26T21:35:15.876144+0000","last_peered":"2024-04-26T21:35:15.876144+0000","last_clean":"2024-04-26T21:35:15.876144+0000","last_became_active":"2024-04-26T21:35:10.136271+0000","last_became_peered":"2024-04-26T21:35:10.136271+0000","last_unstale":"2024-04-26T21:35:15.876144+0000","last_undegraded":"2024-04-26T21:35:15.876144+0000","last_fullsized":"2024-04-26T21:35:15.876144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:45.927928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142500+0000","last_change":"2024-04-26T21:35:09.115543+0000","last_active":"2024-04-26T21:35:15.142500+0000","last_peered":"2024-04-26T21:35:15.142500+0000","last_clean":"2024-04-26T21:35:15.142500+0000","last_became_active":"2024-04-26T21:35:09.115159+0000","last_became_peered":"2024-04-26T21:35:09.115159+0000","last_unstale":"2024-04-26T21:35:15.142500+0000","last_undegraded":"2024-04-26T21:35:15.142500+0000","last_fullsized":"2024-04-26T21:35:15.142500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142639+0000","last_change":"2024-04-26T21:35:14.278004+0000","last_active":"2024-04-26T21:35:15.142639+0000","last_peered":"2024-04-26T21:35:15.142639+0000","last_clean":"2024-04-26T21:35:15.142639+0000","last_became_active":"2024-04-26T21:35:14.277547+0000","last_became_peered":"2024-04-26T21:35:14.277547+0000","last_unstale":"2024-04-26T21:35:15.142639+0000","last_undegraded":"2024-04-26T21:35:15.142639+0000","last_fullsized":"2024-04-26T21:35:15.142639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:49.955325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875902+0000","last_change":"2024-04-26T21:35:10.131439+0000","last_active":"2024-04-26T21:35:15.875902+0000","last_peered":"2024-04-26T21:35:15.875902+0000","last_clean":"2024-04-26T21:35:15.875902+0000","last_became_active":"2024-04-26T21:35:10.130864+0000","last_became_peered":"2024-04-26T21:35:10.130864+0000","last_unstale":"2024-04-26T21:35:15.875902+0000","last_undegraded":"2024-04-26T21:35:15.875902+0000","last_fullsized":"2024-04-26T21:35:15.875902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.130632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004324+0000","last_change":"2024-04-26T21:35:09.123483+0000","last_active":"2024-04-26T21:35:16.004324+0000","last_peered":"2024-04-26T21:35:16.004324+0000","last_clean":"2024-04-26T21:35:16.004324+0000","last_became_active":"2024-04-26T21:35:09.123202+0000","last_became_peered":"2024-04-26T21:35:09.123202+0000","last_unstale":"2024-04-26T21:35:16.004324+0000","last_undegraded":"2024-04-26T21:35:16.004324+0000","last_fullsized":"2024-04-26T21:35:16.004324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:50.000306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003596+0000","last_change":"2024-04-26T21:35:14.163479+0000","last_active":"2024-04-26T21:35:16.003596+0000","last_peered":"2024-04-26T21:35:16.003596+0000","last_clean":"2024-04-26T21:35:16.003596+0000","last_became_active":"2024-04-26T21:35:14.163041+0000","last_became_peered":"2024-04-26T21:35:14.163041+0000","last_unstale":"2024-04-26T21:35:16.003596+0000","last_undegraded":"2024-04-26T21:35:16.003596+0000","last_fullsized":"2024-04-26T21:35:16.003596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:34.682922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875665+0000","last_change":"2024-04-26T21:35:10.129858+0000","last_active":"2024-04-26T21:35:15.875665+0000","last_peered":"2024-04-26T21:35:15.875665+0000","last_clean":"2024-04-26T21:35:15.875665+0000","last_became_active":"2024-04-26T21:35:10.129606+0000","last_became_peered":"2024-04-26T21:35:10.129606+0000","last_unstale":"2024-04-26T21:35:15.875665+0000","last_undegraded":"2024-04-26T21:35:15.875665+0000","last_fullsized":"2024-04-26T21:35:15.875665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:51.413835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.141883+0000","last_change":"2024-04-26T21:35:09.121200+0000","last_active":"2024-04-26T21:35:15.141883+0000","last_peered":"2024-04-26T21:35:15.141883+0000","last_clean":"2024-04-26T21:35:15.141883+0000","last_became_active":"2024-04-26T21:35:09.120700+0000","last_became_peered":"2024-04-26T21:35:09.120700+0000","last_unstale":"2024-04-26T21:35:15.141883+0000","last_undegraded":"2024-04-26T21:35:15.141883+0000","last_fullsized":"2024-04-26T21:35:15.141883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003924+0000","last_change":"2024-04-26T21:35:14.163658+0000","last_active":"2024-04-26T21:35:16.003924+0000","last_peered":"2024-04-26T21:35:16.003924+0000","last_clean":"2024-04-26T21:35:16.003924+0000","last_became_active":"2024-04-26T21:35:14.163215+0000","last_became_peered":"2024-04-26T21:35:14.163215+0000","last_unstale":"2024-04-26T21:35:16.003924+0000","last_undegraded":"2024-04-26T21:35:16.003924+0000","last_fullsized":"2024-04-26T21:35:16.003924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:01.158030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142807+0000","last_change":"2024-04-26T21:35:10.126750+0000","last_active":"2024-04-26T21:35:15.142807+0000","last_peered":"2024-04-26T21:35:15.142807+0000","last_clean":"2024-04-26T21:35:15.142807+0000","last_became_active":"2024-04-26T21:35:10.126096+0000","last_became_peered":"2024-04-26T21:35:10.126096+0000","last_unstale":"2024-04-26T21:35:15.142807+0000","last_undegraded":"2024-04-26T21:35:15.142807+0000","last_fullsized":"2024-04-26T21:35:15.142807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:46.899885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143301+0000","last_change":"2024-04-26T21:35:09.122401+0000","last_active":"2024-04-26T21:35:15.143301+0000","last_peered":"2024-04-26T21:35:15.143301+0000","last_clean":"2024-04-26T21:35:15.143301+0000","last_became_active":"2024-04-26T21:35:09.122067+0000","last_became_peered":"2024-04-26T21:35:09.122067+0000","last_unstale":"2024-04-26T21:35:15.143301+0000","last_undegraded":"2024-04-26T21:35:15.143301+0000","last_fullsized":"2024-04-26T21:35:15.143301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004812+0000","last_change":"2024-04-26T21:35:14.279383+0000","last_active":"2024-04-26T21:35:16.004812+0000","last_peered":"2024-04-26T21:35:16.004812+0000","last_clean":"2024-04-26T21:35:16.004812+0000","last_became_active":"2024-04-26T21:35:14.278985+0000","last_became_peered":"2024-04-26T21:35:14.278985+0000","last_unstale":"2024-04-26T21:35:16.004812+0000","last_undegraded":"2024-04-26T21:35:16.004812+0000","last_fullsized":"2024-04-26T21:35:16.004812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:21.757335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875771+0000","last_change":"2024-04-26T21:35:10.136295+0000","last_active":"2024-04-26T21:35:15.875771+0000","last_peered":"2024-04-26T21:35:15.875771+0000","last_clean":"2024-04-26T21:35:15.875771+0000","last_became_active":"2024-04-26T21:35:10.135975+0000","last_became_peered":"2024-04-26T21:35:10.135975+0000","last_unstale":"2024-04-26T21:35:15.875771+0000","last_undegraded":"2024-04-26T21:35:15.875771+0000","last_fullsized":"2024-04-26T21:35:15.875771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144056+0000","last_change":"2024-04-26T21:35:09.122493+0000","last_active":"2024-04-26T21:35:15.144056+0000","last_peered":"2024-04-26T21:35:15.144056+0000","last_clean":"2024-04-26T21:35:15.144056+0000","last_became_active":"2024-04-26T21:35:09.122211+0000","last_became_peered":"2024-04-26T21:35:09.122211+0000","last_unstale":"2024-04-26T21:35:15.144056+0000","last_undegraded":"2024-04-26T21:35:15.144056+0000","last_fullsized":"2024-04-26T21:35:15.144056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004937+0000","last_change":"2024-04-26T21:35:14.277801+0000","last_active":"2024-04-26T21:35:16.004937+0000","last_peered":"2024-04-26T21:35:16.004937+0000","last_clean":"2024-04-26T21:35:16.004937+0000","last_became_active":"2024-04-26T21:35:14.277417+0000","last_became_peered":"2024-04-26T21:35:14.277417+0000","last_unstale":"2024-04-26T21:35:16.004937+0000","last_undegraded":"2024-04-26T21:35:16.004937+0000","last_fullsized":"2024-04-26T21:35:16.004937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:15.246664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004133+0000","last_change":"2024-04-26T21:35:10.128386+0000","last_active":"2024-04-26T21:35:16.004133+0000","last_peered":"2024-04-26T21:35:16.004133+0000","last_clean":"2024-04-26T21:35:16.004133+0000","last_became_active":"2024-04-26T21:35:10.127067+0000","last_became_peered":"2024-04-26T21:35:10.127067+0000","last_unstale":"2024-04-26T21:35:16.004133+0000","last_undegraded":"2024-04-26T21:35:16.004133+0000","last_fullsized":"2024-04-26T21:35:16.004133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:46.899885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004551+0000","last_change":"2024-04-26T21:35:09.125422+0000","last_active":"2024-04-26T21:35:16.004551+0000","last_peered":"2024-04-26T21:35:16.004551+0000","last_clean":"2024-04-26T21:35:16.004551+0000","last_became_active":"2024-04-26T21:35:09.125086+0000","last_became_peered":"2024-04-26T21:35:09.125086+0000","last_unstale":"2024-04-26T21:35:16.004551+0000","last_undegraded":"2024-04-26T21:35:16.004551+0000","last_fullsized":"2024-04-26T21:35:16.004551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:44.358668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144502+0000","last_change":"2024-04-26T21:35:14.278312+0000","last_active":"2024-04-26T21:35:15.144502+0000","last_peered":"2024-04-26T21:35:15.144502+0000","last_clean":"2024-04-26T21:35:15.144502+0000","last_became_active":"2024-04-26T21:35:14.277836+0000","last_became_peered":"2024-04-26T21:35:14.277836+0000","last_unstale":"2024-04-26T21:35:15.144502+0000","last_undegraded":"2024-04-26T21:35:15.144502+0000","last_fullsized":"2024-04-26T21:35:15.144502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:20.215438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875616+0000","last_change":"2024-04-26T21:35:10.126791+0000","last_active":"2024-04-26T21:35:15.875616+0000","last_peered":"2024-04-26T21:35:15.875616+0000","last_clean":"2024-04-26T21:35:15.875616+0000","last_became_active":"2024-04-26T21:35:10.126515+0000","last_became_peered":"2024-04-26T21:35:10.126515+0000","last_unstale":"2024-04-26T21:35:15.875616+0000","last_undegraded":"2024-04-26T21:35:15.875616+0000","last_fullsized":"2024-04-26T21:35:15.875616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:01.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:19.559960+0000","last_change":"2024-04-26T21:35:09.126680+0000","last_active":"2024-04-26T21:35:19.559960+0000","last_peered":"2024-04-26T21:35:19.559960+0000","last_clean":"2024-04-26T21:35:19.559960+0000","last_became_active":"2024-04-26T21:35:09.126406+0000","last_became_peered":"2024-04-26T21:35:09.126406+0000","last_unstale":"2024-04-26T21:35:19.559960+0000","last_undegraded":"2024-04-26T21:35:19.559960+0000","last_fullsized":"2024-04-26T21:35:19.559960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.081587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373762+0000","last_change":"2024-04-26T21:35:14.163960+0000","last_active":"2024-04-26T21:35:15.373762+0000","last_peered":"2024-04-26T21:35:15.373762+0000","last_clean":"2024-04-26T21:35:15.373762+0000","last_became_active":"2024-04-26T21:35:14.163551+0000","last_became_peered":"2024-04-26T21:35:14.163551+0000","last_unstale":"2024-04-26T21:35:15.373762+0000","last_undegraded":"2024-04-26T21:35:15.373762+0000","last_fullsized":"2024-04-26T21:35:15.373762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004282+0000","last_change":"2024-04-26T21:35:10.129151+0000","last_active":"2024-04-26T21:35:16.004282+0000","last_peered":"2024-04-26T21:35:16.004282+0000","last_clean":"2024-04-26T21:35:16.004282+0000","last_became_active":"2024-04-26T21:35:10.128520+0000","last_became_peered":"2024-04-26T21:35:10.128520+0000","last_unstale":"2024-04-26T21:35:16.004282+0000","last_undegraded":"2024-04-26T21:35:16.004282+0000","last_fullsized":"2024-04-26T21:35:16.004282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:17.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374063+0000","last_change":"2024-04-26T21:35:09.128170+0000","last_active":"2024-04-26T21:35:15.374063+0000","last_peered":"2024-04-26T21:35:15.374063+0000","last_clean":"2024-04-26T21:35:15.374063+0000","last_became_active":"2024-04-26T21:35:09.127817+0000","last_became_peered":"2024-04-26T21:35:09.127817+0000","last_unstale":"2024-04-26T21:35:15.374063+0000","last_undegraded":"2024-04-26T21:35:15.374063+0000","last_fullsized":"2024-04-26T21:35:15.374063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004297+0000","last_change":"2024-04-26T21:35:14.157290+0000","last_active":"2024-04-26T21:35:16.004297+0000","last_peered":"2024-04-26T21:35:16.004297+0000","last_clean":"2024-04-26T21:35:16.004297+0000","last_became_active":"2024-04-26T21:35:14.156794+0000","last_became_peered":"2024-04-26T21:35:14.156794+0000","last_unstale":"2024-04-26T21:35:16.004297+0000","last_undegraded":"2024-04-26T21:35:16.004297+0000","last_fullsized":"2024-04-26T21:35:16.004297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:55.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004165+0000","last_change":"2024-04-26T21:35:10.132571+0000","last_active":"2024-04-26T21:35:16.004165+0000","last_peered":"2024-04-26T21:35:16.004165+0000","last_clean":"2024-04-26T21:35:16.004165+0000","last_became_active":"2024-04-26T21:35:10.132311+0000","last_became_peered":"2024-04-26T21:35:10.132311+0000","last_unstale":"2024-04-26T21:35:16.004165+0000","last_undegraded":"2024-04-26T21:35:16.004165+0000","last_fullsized":"2024-04-26T21:35:16.004165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143720+0000","last_change":"2024-04-26T21:35:09.124883+0000","last_active":"2024-04-26T21:35:15.143720+0000","last_peered":"2024-04-26T21:35:15.143720+0000","last_clean":"2024-04-26T21:35:15.143720+0000","last_became_active":"2024-04-26T21:35:09.124576+0000","last_became_peered":"2024-04-26T21:35:09.124576+0000","last_unstale":"2024-04-26T21:35:15.143720+0000","last_undegraded":"2024-04-26T21:35:15.143720+0000","last_fullsized":"2024-04-26T21:35:15.143720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003494+0000","last_change":"2024-04-26T21:35:14.158063+0000","last_active":"2024-04-26T21:35:16.003494+0000","last_peered":"2024-04-26T21:35:16.003494+0000","last_clean":"2024-04-26T21:35:16.003494+0000","last_became_active":"2024-04-26T21:35:14.157033+0000","last_became_peered":"2024-04-26T21:35:14.157033+0000","last_unstale":"2024-04-26T21:35:16.003494+0000","last_undegraded":"2024-04-26T21:35:16.003494+0000","last_fullsized":"2024-04-26T21:35:16.003494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:20.215438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143604+0000","last_change":"2024-04-26T21:35:10.121823+0000","last_active":"2024-04-26T21:35:15.143604+0000","last_peered":"2024-04-26T21:35:15.143604+0000","last_clean":"2024-04-26T21:35:15.143604+0000","last_became_active":"2024-04-26T21:35:10.121518+0000","last_became_peered":"2024-04-26T21:35:10.121518+0000","last_unstale":"2024-04-26T21:35:15.143604+0000","last_undegraded":"2024-04-26T21:35:15.143604+0000","last_fullsized":"2024-04-26T21:35:15.143604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:06.354261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143861+0000","last_change":"2024-04-26T21:35:09.127420+0000","last_active":"2024-04-26T21:35:15.143861+0000","last_peered":"2024-04-26T21:35:15.143861+0000","last_clean":"2024-04-26T21:35:15.143861+0000","last_became_active":"2024-04-26T21:35:09.127104+0000","last_became_peered":"2024-04-26T21:35:09.127104+0000","last_unstale":"2024-04-26T21:35:15.143861+0000","last_undegraded":"2024-04-26T21:35:15.143861+0000","last_fullsized":"2024-04-26T21:35:15.143861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004551+0000","last_change":"2024-04-26T21:35:14.281132+0000","last_active":"2024-04-26T21:35:16.004551+0000","last_peered":"2024-04-26T21:35:16.004551+0000","last_clean":"2024-04-26T21:35:16.004551+0000","last_became_active":"2024-04-26T21:35:14.280538+0000","last_became_peered":"2024-04-26T21:35:14.280538+0000","last_unstale":"2024-04-26T21:35:16.004551+0000","last_undegraded":"2024-04-26T21:35:16.004551+0000","last_fullsized":"2024-04-26T21:35:16.004551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:37.157683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003564+0000","last_change":"2024-04-26T21:35:14.279953+0000","last_active":"2024-04-26T21:35:16.003564+0000","last_peered":"2024-04-26T21:35:16.003564+0000","last_clean":"2024-04-26T21:35:16.003564+0000","last_became_active":"2024-04-26T21:35:14.279540+0000","last_became_peered":"2024-04-26T21:35:14.279540+0000","last_unstale":"2024-04-26T21:35:16.003564+0000","last_undegraded":"2024-04-26T21:35:16.003564+0000","last_fullsized":"2024-04-26T21:35:16.003564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373464+0000","last_change":"2024-04-26T21:35:10.121201+0000","last_active":"2024-04-26T21:35:15.373464+0000","last_peered":"2024-04-26T21:35:15.373464+0000","last_clean":"2024-04-26T21:35:15.373464+0000","last_became_active":"2024-04-26T21:35:10.120934+0000","last_became_peered":"2024-04-26T21:35:10.120934+0000","last_unstale":"2024-04-26T21:35:15.373464+0000","last_undegraded":"2024-04-26T21:35:15.373464+0000","last_fullsized":"2024-04-26T21:35:15.373464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142787+0000","last_change":"2024-04-26T21:35:09.119199+0000","last_active":"2024-04-26T21:35:15.142787+0000","last_peered":"2024-04-26T21:35:15.142787+0000","last_clean":"2024-04-26T21:35:15.142787+0000","last_became_active":"2024-04-26T21:35:09.118431+0000","last_became_peered":"2024-04-26T21:35:09.118431+0000","last_unstale":"2024-04-26T21:35:15.142787+0000","last_undegraded":"2024-04-26T21:35:15.142787+0000","last_fullsized":"2024-04-26T21:35:15.142787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144043+0000","last_change":"2024-04-26T21:35:14.160976+0000","last_active":"2024-04-26T21:35:15.144043+0000","last_peered":"2024-04-26T21:35:15.144043+0000","last_clean":"2024-04-26T21:35:15.144043+0000","last_became_active":"2024-04-26T21:35:14.160609+0000","last_became_peered":"2024-04-26T21:35:14.160609+0000","last_unstale":"2024-04-26T21:35:15.144043+0000","last_undegraded":"2024-04-26T21:35:15.144043+0000","last_fullsized":"2024-04-26T21:35:15.144043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:05.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003542+0000","last_change":"2024-04-26T21:35:10.134441+0000","last_active":"2024-04-26T21:35:16.003542+0000","last_peered":"2024-04-26T21:35:16.003542+0000","last_clean":"2024-04-26T21:35:16.003542+0000","last_became_active":"2024-04-26T21:35:10.134101+0000","last_became_peered":"2024-04-26T21:35:10.134101+0000","last_unstale":"2024-04-26T21:35:16.003542+0000","last_undegraded":"2024-04-26T21:35:16.003542+0000","last_fullsized":"2024-04-26T21:35:16.003542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004713+0000","last_change":"2024-04-26T21:35:09.124939+0000","last_active":"2024-04-26T21:35:16.004713+0000","last_peered":"2024-04-26T21:35:16.004713+0000","last_clean":"2024-04-26T21:35:16.004713+0000","last_became_active":"2024-04-26T21:35:09.124612+0000","last_became_peered":"2024-04-26T21:35:09.124612+0000","last_unstale":"2024-04-26T21:35:16.004713+0000","last_undegraded":"2024-04-26T21:35:16.004713+0000","last_fullsized":"2024-04-26T21:35:16.004713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:44.387905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143484+0000","last_change":"2024-04-26T21:35:09.122510+0000","last_active":"2024-04-26T21:35:15.143484+0000","last_peered":"2024-04-26T21:35:15.143484+0000","last_clean":"2024-04-26T21:35:15.143484+0000","last_became_active":"2024-04-26T21:35:09.122161+0000","last_became_peered":"2024-04-26T21:35:09.122161+0000","last_unstale":"2024-04-26T21:35:15.143484+0000","last_undegraded":"2024-04-26T21:35:15.143484+0000","last_fullsized":"2024-04-26T21:35:15.143484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.872359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004522+0000","last_change":"2024-04-26T21:35:10.128884+0000","last_active":"2024-04-26T21:35:16.004522+0000","last_peered":"2024-04-26T21:35:16.004522+0000","last_clean":"2024-04-26T21:35:16.004522+0000","last_became_active":"2024-04-26T21:35:10.128599+0000","last_became_peered":"2024-04-26T21:35:10.128599+0000","last_unstale":"2024-04-26T21:35:16.004522+0000","last_undegraded":"2024-04-26T21:35:16.004522+0000","last_fullsized":"2024-04-26T21:35:16.004522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142294+0000","last_change":"2024-04-26T21:35:14.158429+0000","last_active":"2024-04-26T21:35:15.142294+0000","last_peered":"2024-04-26T21:35:15.142294+0000","last_clean":"2024-04-26T21:35:15.142294+0000","last_became_active":"2024-04-26T21:35:14.158011+0000","last_became_peered":"2024-04-26T21:35:14.158011+0000","last_unstale":"2024-04-26T21:35:15.142294+0000","last_undegraded":"2024-04-26T21:35:15.142294+0000","last_fullsized":"2024-04-26T21:35:15.142294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:44.919887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004133+0000","last_change":"2024-04-26T21:35:09.127502+0000","last_active":"2024-04-26T21:35:16.004133+0000","last_peered":"2024-04-26T21:35:16.004133+0000","last_clean":"2024-04-26T21:35:16.004133+0000","last_became_active":"2024-04-26T21:35:09.127182+0000","last_became_peered":"2024-04-26T21:35:09.127182+0000","last_unstale":"2024-04-26T21:35:16.004133+0000","last_undegraded":"2024-04-26T21:35:16.004133+0000","last_fullsized":"2024-04-26T21:35:16.004133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:08.093287+0000","last_clean_scrub_stamp":"2024-04-26T21:35:08.093287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:37.512436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003916+0000","last_change":"2024-04-26T21:35:10.120022+0000","last_active":"2024-04-26T21:35:16.003916+0000","last_peered":"2024-04-26T21:35:16.003916+0000","last_clean":"2024-04-26T21:35:16.003916+0000","last_became_active":"2024-04-26T21:35:10.119759+0000","last_became_peered":"2024-04-26T21:35:10.119759+0000","last_unstale":"2024-04-26T21:35:16.003916+0000","last_undegraded":"2024-04-26T21:35:16.003916+0000","last_fullsized":"2024-04-26T21:35:16.003916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:55.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004467+0000","last_change":"2024-04-26T21:35:14.157203+0000","last_active":"2024-04-26T21:35:16.004467+0000","last_peered":"2024-04-26T21:35:16.004467+0000","last_clean":"2024-04-26T21:35:16.004467+0000","last_became_active":"2024-04-26T21:35:14.156742+0000","last_became_peered":"2024-04-26T21:35:14.156742+0000","last_unstale":"2024-04-26T21:35:16.004467+0000","last_undegraded":"2024-04-26T21:35:16.004467+0000","last_fullsized":"2024-04-26T21:35:16.004467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:13.125313+0000","last_clean_scrub_stamp":"2024-04-26T21:35:13.125313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:13.631171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142673+0000","last_change":"2024-04-26T21:35:10.126924+0000","last_active":"2024-04-26T21:35:15.142673+0000","last_peered":"2024-04-26T21:35:15.142673+0000","last_clean":"2024-04-26T21:35:15.142673+0000","last_became_active":"2024-04-26T21:35:10.126660+0000","last_became_peered":"2024-04-26T21:35:10.126660+0000","last_unstale":"2024-04-26T21:35:15.142673+0000","last_undegraded":"2024-04-26T21:35:15.142673+0000","last_fullsized":"2024-04-26T21:35:15.142673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:29.764634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142006+0000","last_change":"2024-04-26T21:35:10.116329+0000","last_active":"2024-04-26T21:35:15.142006+0000","last_peered":"2024-04-26T21:35:15.142006+0000","last_clean":"2024-04-26T21:35:15.142006+0000","last_became_active":"2024-04-26T21:35:10.116047+0000","last_became_peered":"2024-04-26T21:35:10.116047+0000","last_unstale":"2024-04-26T21:35:15.142006+0000","last_undegraded":"2024-04-26T21:35:15.142006+0000","last_fullsized":"2024-04-26T21:35:15.142006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:17.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143917+0000","last_change":"2024-04-26T21:35:10.121375+0000","last_active":"2024-04-26T21:35:15.143917+0000","last_peered":"2024-04-26T21:35:15.143917+0000","last_clean":"2024-04-26T21:35:15.143917+0000","last_became_active":"2024-04-26T21:35:10.121088+0000","last_became_peered":"2024-04-26T21:35:10.121088+0000","last_unstale":"2024-04-26T21:35:15.143917+0000","last_undegraded":"2024-04-26T21:35:15.143917+0000","last_fullsized":"2024-04-26T21:35:15.143917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004457+0000","last_change":"2024-04-26T21:35:10.131911+0000","last_active":"2024-04-26T21:35:16.004457+0000","last_peered":"2024-04-26T21:35:16.004457+0000","last_clean":"2024-04-26T21:35:16.004457+0000","last_became_active":"2024-04-26T21:35:10.131477+0000","last_became_peered":"2024-04-26T21:35:10.131477+0000","last_unstale":"2024-04-26T21:35:16.004457+0000","last_undegraded":"2024-04-26T21:35:16.004457+0000","last_fullsized":"2024-04-26T21:35:16.004457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.005022+0000","last_change":"2024-04-26T21:35:10.121507+0000","last_active":"2024-04-26T21:35:16.005022+0000","last_peered":"2024-04-26T21:35:16.005022+0000","last_clean":"2024-04-26T21:35:16.005022+0000","last_became_active":"2024-04-26T21:35:10.121208+0000","last_became_peered":"2024-04-26T21:35:10.121208+0000","last_unstale":"2024-04-26T21:35:16.005022+0000","last_undegraded":"2024-04-26T21:35:16.005022+0000","last_fullsized":"2024-04-26T21:35:16.005022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373872+0000","last_change":"2024-04-26T21:35:10.121106+0000","last_active":"2024-04-26T21:35:15.373872+0000","last_peered":"2024-04-26T21:35:15.373872+0000","last_clean":"2024-04-26T21:35:15.373872+0000","last_became_active":"2024-04-26T21:35:10.120857+0000","last_became_peered":"2024-04-26T21:35:10.120857+0000","last_unstale":"2024-04-26T21:35:15.373872+0000","last_undegraded":"2024-04-26T21:35:15.373872+0000","last_fullsized":"2024-04-26T21:35:15.373872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143866+0000","last_change":"2024-04-26T21:35:10.128212+0000","last_active":"2024-04-26T21:35:15.143866+0000","last_peered":"2024-04-26T21:35:15.143866+0000","last_clean":"2024-04-26T21:35:15.143866+0000","last_became_active":"2024-04-26T21:35:10.127868+0000","last_became_peered":"2024-04-26T21:35:10.127868+0000","last_unstale":"2024-04-26T21:35:15.143866+0000","last_undegraded":"2024-04-26T21:35:15.143866+0000","last_fullsized":"2024-04-26T21:35:15.143866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004798+0000","last_change":"2024-04-26T21:35:10.130695+0000","last_active":"2024-04-26T21:35:16.004798+0000","last_peered":"2024-04-26T21:35:16.004798+0000","last_clean":"2024-04-26T21:35:16.004798+0000","last_became_active":"2024-04-26T21:35:10.130084+0000","last_became_peered":"2024-04-26T21:35:10.130084+0000","last_unstale":"2024-04-26T21:35:16.004798+0000","last_undegraded":"2024-04-26T21:35:16.004798+0000","last_fullsized":"2024-04-26T21:35:16.004798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142712+0000","last_change":"2024-04-26T21:35:10.122732+0000","last_active":"2024-04-26T21:35:15.142712+0000","last_peered":"2024-04-26T21:35:15.142712+0000","last_clean":"2024-04-26T21:35:15.142712+0000","last_became_active":"2024-04-26T21:35:10.122457+0000","last_became_peered":"2024-04-26T21:35:10.122457+0000","last_unstale":"2024-04-26T21:35:15.142712+0000","last_undegraded":"2024-04-26T21:35:15.142712+0000","last_fullsized":"2024-04-26T21:35:15.142712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004615+0000","last_change":"2024-04-26T21:35:10.131503+0000","last_active":"2024-04-26T21:35:16.004615+0000","last_peered":"2024-04-26T21:35:16.004615+0000","last_clean":"2024-04-26T21:35:16.004615+0000","last_became_active":"2024-04-26T21:35:10.131262+0000","last_became_peered":"2024-04-26T21:35:10.131262+0000","last_unstale":"2024-04-26T21:35:16.004615+0000","last_undegraded":"2024-04-26T21:35:16.004615+0000","last_fullsized":"2024-04-26T21:35:16.004615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143913+0000","last_change":"2024-04-26T21:35:10.122046+0000","last_active":"2024-04-26T21:35:15.143913+0000","last_peered":"2024-04-26T21:35:15.143913+0000","last_clean":"2024-04-26T21:35:15.143913+0000","last_became_active":"2024-04-26T21:35:10.121759+0000","last_became_peered":"2024-04-26T21:35:10.121759+0000","last_unstale":"2024-04-26T21:35:15.143913+0000","last_undegraded":"2024-04-26T21:35:15.143913+0000","last_fullsized":"2024-04-26T21:35:15.143913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:24.444922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004361+0000","last_change":"2024-04-26T21:35:10.128039+0000","last_active":"2024-04-26T21:35:16.004361+0000","last_peered":"2024-04-26T21:35:16.004361+0000","last_clean":"2024-04-26T21:35:16.004361+0000","last_became_active":"2024-04-26T21:35:10.126920+0000","last_became_peered":"2024-04-26T21:35:10.126920+0000","last_unstale":"2024-04-26T21:35:16.004361+0000","last_undegraded":"2024-04-26T21:35:16.004361+0000","last_fullsized":"2024-04-26T21:35:16.004361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143654+0000","last_change":"2024-04-26T21:35:10.120371+0000","last_active":"2024-04-26T21:35:15.143654+0000","last_peered":"2024-04-26T21:35:15.143654+0000","last_clean":"2024-04-26T21:35:15.143654+0000","last_became_active":"2024-04-26T21:35:10.120020+0000","last_became_peered":"2024-04-26T21:35:10.120020+0000","last_unstale":"2024-04-26T21:35:15.143654+0000","last_undegraded":"2024-04-26T21:35:15.143654+0000","last_fullsized":"2024-04-26T21:35:15.143654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.107201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876023+0000","last_change":"2024-04-26T21:35:10.128002+0000","last_active":"2024-04-26T21:35:15.876023+0000","last_peered":"2024-04-26T21:35:15.876023+0000","last_clean":"2024-04-26T21:35:15.876023+0000","last_became_active":"2024-04-26T21:35:10.127763+0000","last_became_peered":"2024-04-26T21:35:10.127763+0000","last_unstale":"2024-04-26T21:35:15.876023+0000","last_undegraded":"2024-04-26T21:35:15.876023+0000","last_fullsized":"2024-04-26T21:35:15.876023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:09.615472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142501+0000","last_change":"2024-04-26T21:35:10.118765+0000","last_active":"2024-04-26T21:35:15.142501+0000","last_peered":"2024-04-26T21:35:15.142501+0000","last_clean":"2024-04-26T21:35:15.142501+0000","last_became_active":"2024-04-26T21:35:10.118481+0000","last_became_peered":"2024-04-26T21:35:10.118481+0000","last_unstale":"2024-04-26T21:35:15.142501+0000","last_undegraded":"2024-04-26T21:35:15.142501+0000","last_fullsized":"2024-04-26T21:35:15.142501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:55.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.144424+0000","last_change":"2024-04-26T21:35:10.123877+0000","last_active":"2024-04-26T21:35:15.144424+0000","last_peered":"2024-04-26T21:35:15.144424+0000","last_clean":"2024-04-26T21:35:15.144424+0000","last_became_active":"2024-04-26T21:35:10.122628+0000","last_became_peered":"2024-04-26T21:35:10.122628+0000","last_unstale":"2024-04-26T21:35:15.144424+0000","last_undegraded":"2024-04-26T21:35:15.144424+0000","last_fullsized":"2024-04-26T21:35:15.144424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.125849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.142605+0000","last_change":"2024-04-26T21:35:10.133644+0000","last_active":"2024-04-26T21:35:15.142605+0000","last_peered":"2024-04-26T21:35:15.142605+0000","last_clean":"2024-04-26T21:35:15.142605+0000","last_became_active":"2024-04-26T21:35:10.133364+0000","last_became_peered":"2024-04-26T21:35:10.133364+0000","last_unstale":"2024-04-26T21:35:15.142605+0000","last_undegraded":"2024-04-26T21:35:15.142605+0000","last_fullsized":"2024-04-26T21:35:15.142605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.373575+0000","last_change":"2024-04-26T21:35:10.130778+0000","last_active":"2024-04-26T21:35:15.373575+0000","last_peered":"2024-04-26T21:35:15.373575+0000","last_clean":"2024-04-26T21:35:15.373575+0000","last_became_active":"2024-04-26T21:35:10.130501+0000","last_became_peered":"2024-04-26T21:35:10.130501+0000","last_unstale":"2024-04-26T21:35:15.373575+0000","last_undegraded":"2024-04-26T21:35:15.373575+0000","last_fullsized":"2024-04-26T21:35:15.373575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004023+0000","last_change":"2024-04-26T21:35:10.134920+0000","last_active":"2024-04-26T21:35:16.004023+0000","last_peered":"2024-04-26T21:35:16.004023+0000","last_clean":"2024-04-26T21:35:16.004023+0000","last_became_active":"2024-04-26T21:35:10.134647+0000","last_became_peered":"2024-04-26T21:35:10.134647+0000","last_unstale":"2024-04-26T21:35:16.004023+0000","last_undegraded":"2024-04-26T21:35:16.004023+0000","last_fullsized":"2024-04-26T21:35:16.004023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:19.754278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003968+0000","last_change":"2024-04-26T21:35:10.118720+0000","last_active":"2024-04-26T21:35:16.003968+0000","last_peered":"2024-04-26T21:35:16.003968+0000","last_clean":"2024-04-26T21:35:16.003968+0000","last_became_active":"2024-04-26T21:35:10.118418+0000","last_became_peered":"2024-04-26T21:35:10.118418+0000","last_unstale":"2024-04-26T21:35:16.003968+0000","last_undegraded":"2024-04-26T21:35:16.003968+0000","last_fullsized":"2024-04-26T21:35:16.003968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.004237+0000","last_change":"2024-04-26T21:35:10.118157+0000","last_active":"2024-04-26T21:35:16.004237+0000","last_peered":"2024-04-26T21:35:16.004237+0000","last_clean":"2024-04-26T21:35:16.004237+0000","last_became_active":"2024-04-26T21:35:10.117862+0000","last_became_peered":"2024-04-26T21:35:10.117862+0000","last_unstale":"2024-04-26T21:35:16.004237+0000","last_undegraded":"2024-04-26T21:35:16.004237+0000","last_fullsized":"2024-04-26T21:35:16.004237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:26.040140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143343+0000","last_change":"2024-04-26T21:35:10.124503+0000","last_active":"2024-04-26T21:35:15.143343+0000","last_peered":"2024-04-26T21:35:15.143343+0000","last_clean":"2024-04-26T21:35:15.143343+0000","last_became_active":"2024-04-26T21:35:10.123919+0000","last_became_peered":"2024-04-26T21:35:10.123919+0000","last_unstale":"2024-04-26T21:35:15.143343+0000","last_undegraded":"2024-04-26T21:35:15.143343+0000","last_fullsized":"2024-04-26T21:35:15.143343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:46.065896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143962+0000","last_change":"2024-04-26T21:35:10.124821+0000","last_active":"2024-04-26T21:35:15.143962+0000","last_peered":"2024-04-26T21:35:15.143962+0000","last_clean":"2024-04-26T21:35:15.143962+0000","last_became_active":"2024-04-26T21:35:10.124245+0000","last_became_peered":"2024-04-26T21:35:10.124245+0000","last_unstale":"2024-04-26T21:35:15.143962+0000","last_undegraded":"2024-04-26T21:35:15.143962+0000","last_fullsized":"2024-04-26T21:35:15.143962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876099+0000","last_change":"2024-04-26T21:35:10.131624+0000","last_active":"2024-04-26T21:35:15.876099+0000","last_peered":"2024-04-26T21:35:15.876099+0000","last_clean":"2024-04-26T21:35:15.876099+0000","last_became_active":"2024-04-26T21:35:10.131392+0000","last_became_peered":"2024-04-26T21:35:10.131392+0000","last_unstale":"2024-04-26T21:35:15.876099+0000","last_undegraded":"2024-04-26T21:35:15.876099+0000","last_fullsized":"2024-04-26T21:35:15.876099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:33.396034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003337+0000","last_change":"2024-04-26T21:35:10.127875+0000","last_active":"2024-04-26T21:35:16.003337+0000","last_peered":"2024-04-26T21:35:16.003337+0000","last_clean":"2024-04-26T21:35:16.003337+0000","last_became_active":"2024-04-26T21:35:10.126566+0000","last_became_peered":"2024-04-26T21:35:10.126566+0000","last_unstale":"2024-04-26T21:35:16.003337+0000","last_undegraded":"2024-04-26T21:35:16.003337+0000","last_fullsized":"2024-04-26T21:35:16.003337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:07.466638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875494+0000","last_change":"2024-04-26T21:35:10.126677+0000","last_active":"2024-04-26T21:35:15.875494+0000","last_peered":"2024-04-26T21:35:15.875494+0000","last_clean":"2024-04-26T21:35:15.875494+0000","last_became_active":"2024-04-26T21:35:10.126388+0000","last_became_peered":"2024-04-26T21:35:10.126388+0000","last_unstale":"2024-04-26T21:35:15.875494+0000","last_undegraded":"2024-04-26T21:35:15.875494+0000","last_fullsized":"2024-04-26T21:35:15.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:41.801909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143757+0000","last_change":"2024-04-26T21:35:10.125406+0000","last_active":"2024-04-26T21:35:15.143757+0000","last_peered":"2024-04-26T21:35:15.143757+0000","last_clean":"2024-04-26T21:35:15.143757+0000","last_became_active":"2024-04-26T21:35:10.125143+0000","last_became_peered":"2024-04-26T21:35:10.125143+0000","last_unstale":"2024-04-26T21:35:15.143757+0000","last_undegraded":"2024-04-26T21:35:15.143757+0000","last_fullsized":"2024-04-26T21:35:15.143757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.130632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.875494+0000","last_change":"2024-04-26T21:35:10.127947+0000","last_active":"2024-04-26T21:35:15.875494+0000","last_peered":"2024-04-26T21:35:15.875494+0000","last_clean":"2024-04-26T21:35:15.875494+0000","last_became_active":"2024-04-26T21:35:10.127700+0000","last_became_peered":"2024-04-26T21:35:10.127700+0000","last_unstale":"2024-04-26T21:35:15.875494+0000","last_undegraded":"2024-04-26T21:35:15.875494+0000","last_fullsized":"2024-04-26T21:35:15.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:09.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.143721+0000","last_change":"2024-04-26T21:35:10.121445+0000","last_active":"2024-04-26T21:35:15.143721+0000","last_peered":"2024-04-26T21:35:15.143721+0000","last_clean":"2024-04-26T21:35:15.143721+0000","last_became_active":"2024-04-26T21:35:10.119356+0000","last_became_peered":"2024-04-26T21:35:10.119356+0000","last_unstale":"2024-04-26T21:35:15.143721+0000","last_undegraded":"2024-04-26T21:35:15.143721+0000","last_fullsized":"2024-04-26T21:35:15.143721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:27.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:16.003601+0000","last_change":"2024-04-26T21:35:10.124758+0000","last_active":"2024-04-26T21:35:16.003601+0000","last_peered":"2024-04-26T21:35:16.003601+0000","last_clean":"2024-04-26T21:35:16.003601+0000","last_became_active":"2024-04-26T21:35:10.124480+0000","last_became_peered":"2024-04-26T21:35:10.124480+0000","last_unstale":"2024-04-26T21:35:16.003601+0000","last_undegraded":"2024-04-26T21:35:16.003601+0000","last_fullsized":"2024-04-26T21:35:16.003601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:29.764634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.374098+0000","last_change":"2024-04-26T21:35:10.125937+0000","last_active":"2024-04-26T21:35:15.374098+0000","last_peered":"2024-04-26T21:35:15.374098+0000","last_clean":"2024-04-26T21:35:15.374098+0000","last_became_active":"2024-04-26T21:35:10.125640+0000","last_became_peered":"2024-04-26T21:35:10.125640+0000","last_unstale":"2024-04-26T21:35:15.374098+0000","last_undegraded":"2024-04-26T21:35:15.374098+0000","last_fullsized":"2024-04-26T21:35:15.374098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:38.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T21:35:15.876035+0000","last_change":"2024-04-26T21:35:10.129050+0000","last_active":"2024-04-26T21:35:15.876035+0000","last_peered":"2024-04-26T21:35:15.876035+0000","last_clean":"2024-04-26T21:35:15.876035+0000","last_became_active":"2024-04-26T21:35:10.128720+0000","last_became_peered":"2024-04-26T21:35:10.128720+0000","last_unstale":"2024-04-26T21:35:15.876035+0000","last_undegraded":"2024-04-26T21:35:15.876035+0000","last_fullsized":"2024-04-26T21:35:15.876035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:35:09.097915+0000","last_clean_scrub_stamp":"2024-04-26T21:35:09.097915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:33.320854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:35:38.045 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:35:38.045 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:35:38.045 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:35:38.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:35:38.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.180+0000 7fbc3d225640 1 Processor -- start 2024-04-26T21:35:38.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.180+0000 7fbc3d225640 1 -- start start 2024-04-26T21:35:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc3d225640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380d3b80 0x7fbc380b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc3d225640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3ff0 0x7fbc380b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc3d225640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc380d2d70 con 0x7fbc3815f540 2024-04-26T21:35:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc3d225640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbc3815f920 con 0x7fbc380b3ff0 2024-04-26T21:35:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc3d225640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc380d0f30 con 0x7fbc380d3b80 2024-04-26T21:35:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380d3b80 0x7fbc380b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc3759e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3ff0 0x7fbc380b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc36d9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fbc3815f540 0x7fbc380d3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:58082/0 (socket says 172.21.15.40:58082) 2024-04-26T21:35:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.181+0000 7fbc36d9d640 1 -- 172.21.15.40:0/817652318 learned_addr learned my addr 172.21.15.40:0/817652318 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.182+0000 7fbc3659c640 1 -- 172.21.15.40:0/817652318 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1516064746 0 0) 0x7fbc380d2d70 con 0x7fbc3815f540 2024-04-26T21:35:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.182+0000 7fbc3659c640 1 -- 172.21.15.40:0/817652318 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc24003520 con 0x7fbc3815f540 2024-04-26T21:35:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.182+0000 7fbc2ffff640 1 -- 172.21.15.40:0/817652318 >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3ff0 msgr2=0x7fbc380b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.182+0000 7fbc2ffff640 1 --2- 172.21.15.40:0/817652318 >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3ff0 0x7fbc380b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.182+0000 7fbc2ffff640 1 -- 172.21.15.40:0/817652318 >> v1:172.21.15.184:6789/0 conn(0x7fbc3815f540 legacy=0x7fbc380d3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.182+0000 7fbc2ffff640 1 -- 172.21.15.40:0/817652318 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc380d3050 con 0x7fbc380d3b80 2024-04-26T21:35:38.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.182+0000 7fbc3759e640 1 --2- 172.21.15.40:0/817652318 >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3ff0 0x7fbc380b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:38.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.182+0000 7fbc2ffff640 1 --2- 172.21.15.40:0/817652318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380d3b80 0x7fbc380b4aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc200097a0 tx=0x7fbc2002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9519a382b817b15a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3659c640 1 -- 172.21.15.40:0/817652318 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20037070 con 0x7fbc380d3b80 2024-04-26T21:35:38.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3659c640 1 -- 172.21.15.40:0/817652318 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc20032440 con 0x7fbc380d3b80 2024-04-26T21:35:38.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3659c640 1 -- 172.21.15.40:0/817652318 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200327f0 con 0x7fbc380d3b80 2024-04-26T21:35:38.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3d225640 1 -- 172.21.15.40:0/817652318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380d3b80 msgr2=0x7fbc380b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3d225640 1 --2- 172.21.15.40:0/817652318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380d3b80 0x7fbc380b4aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc200097a0 tx=0x7fbc2002a180 comp rx=0 tx=0).stop 2024-04-26T21:35:38.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3d225640 1 -- 172.21.15.40:0/817652318 shutdown_connections 2024-04-26T21:35:38.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3d225640 1 --2- 172.21.15.40:0/817652318 >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3ff0 0x7fbc380b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3d225640 1 --2- 172.21.15.40:0/817652318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380d3b80 0x7fbc380b4aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3d225640 1 -- 172.21.15.40:0/817652318 >> 172.21.15.40:0/817652318 conn(0x7fbc380d4f90 msgr2=0x7fbc380d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3d225640 1 -- 172.21.15.40:0/817652318 shutdown_connections 2024-04-26T21:35:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.183+0000 7fbc3d225640 1 -- 172.21.15.40:0/817652318 wait complete. 2024-04-26T21:35:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc3d225640 1 Processor -- start 2024-04-26T21:35:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc3d225640 1 -- start start 2024-04-26T21:35:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc3d225640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ff0 0x7fbc380c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc3d225640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380d3b80 0x7fbc38153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc3d225640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3815f540 0x7fbc380cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc3d225640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38068e50 con 0x7fbc380d3b80 2024-04-26T21:35:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc3d225640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc380d0bc0 con 0x7fbc3815f540 2024-04-26T21:35:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc3d225640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc380d2780 con 0x7fbc380b3ff0 2024-04-26T21:35:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380d3b80 0x7fbc38153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.185+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380d3b80 0x7fbc38153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45896/0 (socket says 172.21.15.40:45896) 2024-04-26T21:35:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3023397475 learned_addr learned my addr 172.21.15.40:0/3023397475 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc36d9d640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ff0 0x7fbc380c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc3759e640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3815f540 0x7fbc380cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ff0 msgr2=0x7fbc380c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2ffff640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ff0 0x7fbc380c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3815f540 msgr2=0x7fbc380cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2ffff640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3815f540 0x7fbc380cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3023397475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc38153950 con 0x7fbc380d3b80 2024-04-26T21:35:38.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2ffff640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380d3b80 0x7fbc38153410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc200097a0 tx=0x7fbc200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2f7fe640 1 -- 172.21.15.40:0/3023397475 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20041070 con 0x7fbc380d3b80 2024-04-26T21:35:38.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2f7fe640 1 -- 172.21.15.40:0/3023397475 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc2003c870 con 0x7fbc380d3b80 2024-04-26T21:35:38.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc2f7fe640 1 -- 172.21.15.40:0/3023397475 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20037040 con 0x7fbc380d3b80 2024-04-26T21:35:38.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc380ca230 con 0x7fbc380d3b80 2024-04-26T21:35:38.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.186+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc380ca7f0 con 0x7fbc380d3b80 2024-04-26T21:35:38.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.187+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc380694f0 con 0x7fbc380d3b80 2024-04-26T21:35:38.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.188+0000 7fbc2f7fe640 1 -- 172.21.15.40:0/3023397475 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc20044070 con 0x7fbc380d3b80 2024-04-26T21:35:38.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.188+0000 7fbc2f7fe640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc1806c920 0x7fbc1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.188+0000 7fbc2f7fe640 1 -- 172.21.15.40:0/3023397475 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbc200ac790 con 0x7fbc380d3b80 2024-04-26T21:35:38.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.188+0000 7fbc36d9d640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc1806c920 0x7fbc1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.189+0000 7fbc36d9d640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc1806c920 0x7fbc1806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005880 tx=0x7fbc28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.193+0000 7fbc2f7fe640 1 -- 172.21.15.40:0/3023397475 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc38153950 con 0x7fbc380d3b80 2024-04-26T21:35:38.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.421+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbc380d4b20 con 0x7fbc380d3b80 2024-04-26T21:35:38.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.421+0000 7fbc2f7fe640 1 -- 172.21.15.40:0/3023397475 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbc2007a360 con 0x7fbc380d3b80 2024-04-26T21:35:38.423 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:38.423 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:35:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc1806c920 msgr2=0x7fbc1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc1806c920 0x7fbc1806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005880 tx=0x7fbc28009040 comp rx=0 tx=0).stop 2024-04-26T21:35:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380d3b80 msgr2=0x7fbc38153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380d3b80 0x7fbc38153410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc200097a0 tx=0x7fbc200046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 shutdown_connections 2024-04-26T21:35:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc1806c920 0x7fbc1806ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3815f540 0x7fbc380cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380d3b80 0x7fbc38153410 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 --2- 172.21.15.40:0/3023397475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ff0 0x7fbc380c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 >> 172.21.15.40:0/3023397475 conn(0x7fbc380d4f90 msgr2=0x7fbc380b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:38.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 shutdown_connections 2024-04-26T21:35:38.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.423+0000 7fbc3d225640 1 -- 172.21.15.40:0/3023397475 wait complete. 2024-04-26T21:35:38.435 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:35:38.436 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:38.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.535+0000 7fcd011a4640 1 Processor -- start 2024-04-26T21:35:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.535+0000 7fcd011a4640 1 -- start start 2024-04-26T21:35:38.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fcd011a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0d3bc0 0x7fccfc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fcd011a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fccfc0b5f20 0x7fccfc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fcd011a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccfc0b7e50 con 0x7fccfc162880 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fcd011a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fccfc0d0580 con 0x7fccfc0b5f20 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fcd011a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc0d1800 con 0x7fccfc0d3bc0 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfb59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fccfc0b5f20 0x7fccfc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fccfc162880 0x7fccfc0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40254/0 (socket says 172.21.15.40:40254) 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfad9d640 1 -- 172.21.15.40:0/2626755558 learned_addr learned my addr 172.21.15.40:0/2626755558 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfa59c640 1 --2- 172.21.15.40:0/2626755558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0d3bc0 0x7fccfc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfb59e640 1 -- 172.21.15.40:0/2626755558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0d3bc0 msgr2=0x7fccfc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfb59e640 1 --2- 172.21.15.40:0/2626755558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0d3bc0 0x7fccfc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfb59e640 1 -- 172.21.15.40:0/2626755558 >> v1:172.21.15.40:6789/0 conn(0x7fccfc162880 legacy=0x7fccfc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfb59e640 1 -- 172.21.15.40:0/2626755558 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc0d07d0 con 0x7fccfc0b5f20 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.536+0000 7fccfa59c640 1 --2- 172.21.15.40:0/2626755558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0d3bc0 0x7fccfc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.537+0000 7fccfb59e640 1 --2- 172.21.15.40:0/2626755558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc0b5f20 0x7fccfc0b8300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009d90 tx=0x7fccf402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e696f881e8e2c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.537+0000 7fccf9d9b640 1 -- 172.21.15.40:0/2626755558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4007740 con 0x7fccfc0b5f20 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.537+0000 7fccf9d9b640 1 -- 172.21.15.40:0/2626755558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf40078e0 con 0x7fccfc0b5f20 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.537+0000 7fccf9d9b640 1 -- 172.21.15.40:0/2626755558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4007cb0 con 0x7fccfc0b5f20 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.538+0000 7fcd011a4640 1 -- 172.21.15.40:0/2626755558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc0b5f20 msgr2=0x7fccfc0b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.538+0000 7fcd011a4640 1 --2- 172.21.15.40:0/2626755558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc0b5f20 0x7fccfc0b8300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009d90 tx=0x7fccf402a660 comp rx=0 tx=0).stop 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.538+0000 7fcd011a4640 1 -- 172.21.15.40:0/2626755558 shutdown_connections 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.538+0000 7fcd011a4640 1 --2- 172.21.15.40:0/2626755558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc0b5f20 0x7fccfc0b8300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.538+0000 7fcd011a4640 1 --2- 172.21.15.40:0/2626755558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0d3bc0 0x7fccfc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.538+0000 7fcd011a4640 1 -- 172.21.15.40:0/2626755558 >> 172.21.15.40:0/2626755558 conn(0x7fccfc0d4f60 msgr2=0x7fccfc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.538+0000 7fcd011a4640 1 -- 172.21.15.40:0/2626755558 shutdown_connections 2024-04-26T21:35:38.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.538+0000 7fcd011a4640 1 -- 172.21.15.40:0/2626755558 wait complete. 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.539+0000 7fcd011a4640 1 Processor -- start 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.539+0000 7fcd011a4640 1 -- start start 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fcd011a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0b5f20 0x7fccfc15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fcd011a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc0d3bc0 0x7fccfc1f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fcd011a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc162880 0x7fccfc1f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fcd011a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc162c60 con 0x7fccfc162880 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fcd011a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc0d0d90 con 0x7fccfc0d3bc0 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fcd011a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc162de0 con 0x7fccfc0b5f20 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc162880 0x7fccfc1f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc162880 0x7fccfc1f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45926/0 (socket says 172.21.15.40:45926) 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfb59e640 1 -- 172.21.15.40:0/422092985 learned_addr learned my addr 172.21.15.40:0/422092985 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfad9d640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0b5f20 0x7fccfc15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfa59c640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc0d3bc0 0x7fccfc1f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfb59e640 1 -- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0b5f20 msgr2=0x7fccfc15c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfb59e640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0b5f20 0x7fccfc15c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfb59e640 1 -- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc0d3bc0 msgr2=0x7fccfc1f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfb59e640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc0d3bc0 0x7fccfc1f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfb59e640 1 -- 172.21.15.40:0/422092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc1f4d50 con 0x7fccfc162880 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.540+0000 7fccfad9d640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0b5f20 0x7fccfc15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.541+0000 7fccfb59e640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc162880 0x7fccfc1f54e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fccf402ac50 tx=0x7fccf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.541+0000 7fccf37fe640 1 -- 172.21.15.40:0/422092985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf403f070 con 0x7fccfc162880 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.541+0000 7fccf37fe640 1 -- 172.21.15.40:0/422092985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4031c90 con 0x7fccfc162880 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.541+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc1f8af0 con 0x7fccfc162880 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.541+0000 7fccf37fe640 1 -- 172.21.15.40:0/422092985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4044680 con 0x7fccfc162880 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.541+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc1f90b0 con 0x7fccfc162880 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.542+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc0694f0 con 0x7fccfc162880 2024-04-26T21:35:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.542+0000 7fccf37fe640 1 -- 172.21.15.40:0/422092985 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccf403b410 con 0x7fccfc162880 2024-04-26T21:35:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.543+0000 7fccf37fe640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccdc06c8f0 0x7fccdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.543+0000 7fccf37fe640 1 -- 172.21.15.40:0/422092985 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fccf40ad720 con 0x7fccfc162880 2024-04-26T21:35:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.543+0000 7fccfad9d640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccdc06c8f0 0x7fccdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.543+0000 7fccfad9d640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccdc06c8f0 0x7fccdc06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccfc15fc00 tx=0x7fcce4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.547+0000 7fccf37fe640 1 -- 172.21.15.40:0/422092985 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fccf407b310 con 0x7fccfc162880 2024-04-26T21:35:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.732+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccfc0d3700 con 0x7fccfc162880 2024-04-26T21:35:38.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.733+0000 7fccf37fe640 1 -- 172.21.15.40:0/422092985 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7fccf407f610 con 0x7fccfc162880 2024-04-26T21:35:38.735 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:38.735 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccdc06c8f0 msgr2=0x7fccdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccdc06c8f0 0x7fccdc06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccfc15fc00 tx=0x7fcce4009040 comp rx=0 tx=0).stop 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc162880 msgr2=0x7fccfc1f54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc162880 0x7fccfc1f54e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fccf402ac50 tx=0x7fccf4004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 shutdown_connections 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccdc06c8f0 0x7fccdc06edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccfc162880 0x7fccfc1f54e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc0d3bc0 0x7fccfc1f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 --2- 172.21.15.40:0/422092985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc0b5f20 0x7fccfc15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.735+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 >> 172.21.15.40:0/422092985 conn(0x7fccfc0d4f60 msgr2=0x7fccfc06d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.736+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 shutdown_connections 2024-04-26T21:35:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.736+0000 7fcd011a4640 1 -- 172.21.15.40:0/422092985 wait complete. 2024-04-26T21:35:38.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:38.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.851+0000 7f48d46f1640 1 Processor -- start 2024-04-26T21:35:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.851+0000 7f48d46f1640 1 -- start start 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d46f1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f48cc15f550 0x7f48cc0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d46f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0d1040 con 0x7f48cc0d3bc0 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d46f1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0c9680 con 0x7f48cc15f170 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d46f1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f48cc0c9430 con 0x7f48cc15f550 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d2c8e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f48cc0d3bc0 0x7f48cc0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38628/0 (socket says 172.21.15.40:38628) 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d2c8e640 1 -- 172.21.15.40:0/1802363912 learned_addr learned my addr 172.21.15.40:0/1802363912 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d1c8c640 1 --2- 172.21.15.40:0/1802363912 >> v2:172.21.15.184:3300/0 conn(0x7f48cc15f550 0x7f48cc0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930279355 0 0) 0x7f48bc001490 con 0x7f48cc0d3bc0 2024-04-26T21:35:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.852+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48b4003520 con 0x7f48cc0d3bc0 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2798399827 0 0) 0x7f48b4003520 con 0x7f48cc0d3bc0 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 >> v2:172.21.15.184:3300/0 conn(0x7f48cc15f550 msgr2=0x7f48cc0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 --2- 172.21.15.40:0/1802363912 >> v2:172.21.15.184:3300/0 conn(0x7f48cc15f550 0x7f48cc0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 >> v1:172.21.15.184:6789/0 conn(0x7f48cc15f170 legacy=0x7f48cc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc0c9960 con 0x7f48cc0d3bc0 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f48bc001bc0 con 0x7f48cc0d3bc0 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 >> v1:172.21.15.40:6789/0 conn(0x7f48cc0d3bc0 legacy=0x7f48cc0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b40039e0 0x7f48b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b4004800 0x7f48b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40051c0 0x7f48b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc0c9430 con 0x7f48b40039e0 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f48b400c0c0 con 0x7f48b40051c0 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f48b400c2c0 con 0x7f48b4004800 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d248d640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b40039e0 0x7f48b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d2c8e640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40051c0 0x7f48b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d1c8c640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b4004800 0x7f48b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d248d640 1 -- 172.21.15.40:0/1802363912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b4004800 msgr2=0x7f48b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d248d640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b4004800 0x7f48b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d248d640 1 -- 172.21.15.40:0/1802363912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40051c0 msgr2=0x7f48b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d248d640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40051c0 0x7f48b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d2c8e640 1 -- 172.21.15.40:0/1802363912 reap_dead start 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.853+0000 7f48d248d640 1 -- 172.21.15.40:0/1802363912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b4003520 con 0x7f48b40039e0 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.854+0000 7f48d248d640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b40039e0 0x7f48b4003e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f48c4001750 tx=0x7f48b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cfe48c9a0fb501 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.854+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4001370 con 0x7f48b40039e0 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.854+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c40052d0 con 0x7f48b40039e0 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.854+0000 7f48d148b640 1 -- 172.21.15.40:0/1802363912 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c402abc0 con 0x7f48b40039e0 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.854+0000 7f48d46f1640 1 -- 172.21.15.40:0/1802363912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b40039e0 msgr2=0x7f48b4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.854+0000 7f48d46f1640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b40039e0 0x7f48b4003e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f48c4001750 tx=0x7f48b4005830 comp rx=0 tx=0).stop 2024-04-26T21:35:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.855+0000 7f48d46f1640 1 -- 172.21.15.40:0/1802363912 shutdown_connections 2024-04-26T21:35:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.855+0000 7f48d46f1640 1 --2- 172.21.15.40:0/1802363912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b40039e0 0x7f48b4003e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.855+0000 7f48d46f1640 1 -- 172.21.15.40:0/1802363912 >> 172.21.15.40:0/1802363912 conn(0x7f48cc0d4f60 msgr2=0x7f48cc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.855+0000 7f48d46f1640 1 -- 172.21.15.40:0/1802363912 shutdown_connections 2024-04-26T21:35:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.855+0000 7f48d46f1640 1 -- 172.21.15.40:0/1802363912 wait complete. 2024-04-26T21:35:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.856+0000 7f48d46f1640 1 Processor -- start 2024-04-26T21:35:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.856+0000 7f48d46f1640 1 -- start start 2024-04-26T21:35:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d46f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40039e0 0x7f48cc14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d46f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc15b300 0x7f48cc0a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d46f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48cc0b9010 0x7f48cc0b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d46f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc0d5c00 con 0x7f48cc15b300 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d46f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc0d5a80 con 0x7f48b40039e0 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d46f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc0d5d80 con 0x7f48cc0b9010 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc15b300 0x7f48cc0a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc15b300 0x7f48cc0a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41776/0 (socket says 172.21.15.40:41776) 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 -- 172.21.15.40:0/370751354 learned_addr learned my addr 172.21.15.40:0/370751354 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d248d640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40039e0 0x7f48cc14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d2c8e640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48cc0b9010 0x7f48cc0b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 -- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48cc0b9010 msgr2=0x7f48cc0b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48cc0b9010 0x7f48cc0b94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 -- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40039e0 msgr2=0x7f48cc14d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40039e0 0x7f48cc14d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 -- 172.21.15.40:0/370751354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc0a1de0 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d1c8c640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc15b300 0x7f48cc0a14e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48c0011540 tx=0x7f48c0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48caffd640 1 -- 172.21.15.40:0/370751354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0019070 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48caffd640 1 -- 172.21.15.40:0/370751354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c0003610 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc0a20c0 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48caffd640 1 -- 172.21.15.40:0/370751354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0003a00 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.857+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc0a2680 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.858+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc0694f0 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.859+0000 7f48caffd640 1 -- 172.21.15.40:0/370751354 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48c001c070 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.859+0000 7f48caffd640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48b806c920 0x7f48b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.859+0000 7f48caffd640 1 -- 172.21.15.40:0/370751354 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f48c008c180 con 0x7f48cc15b300 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.859+0000 7f48d248d640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48b806c920 0x7f48b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:38.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.860+0000 7f48d248d640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48b806c920 0x7f48b806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48c4032000 tx=0x7f48c402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:38.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:38.863+0000 7f48caffd640 1 -- 172.21.15.40:0/370751354 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48c005a300 con 0x7f48cc15b300 2024-04-26T21:35:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.046+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48cc0d0560 con 0x7f48cc15b300 2024-04-26T21:35:39.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.046+0000 7f48caffd640 1 -- 172.21.15.40:0/370751354 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f48c0059c00 con 0x7f48cc15b300 2024-04-26T21:35:39.048 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48b806c920 msgr2=0x7f48b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48b806c920 0x7f48b806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48c4032000 tx=0x7f48c402e000 comp rx=0 tx=0).stop 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc15b300 msgr2=0x7f48cc0a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc15b300 0x7f48cc0a14e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48c0011540 tx=0x7f48c0011a00 comp rx=0 tx=0).stop 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 shutdown_connections 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48b806c920 0x7f48b806ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48cc0b9010 0x7f48cc0b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc15b300 0x7f48cc0a14e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 --2- 172.21.15.40:0/370751354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b40039e0 0x7f48cc14d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 >> 172.21.15.40:0/370751354 conn(0x7f48cc0d4f60 msgr2=0x7f48cc0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:39.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 shutdown_connections 2024-04-26T21:35:39.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.048+0000 7f48d46f1640 1 -- 172.21.15.40:0/370751354 wait complete. 2024-04-26T21:35:39.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:39.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.163+0000 7f0458cda640 1 Processor -- start 2024-04-26T21:35:39.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.163+0000 7f0458cda640 1 -- start start 2024-04-26T21:35:39.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0458cda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f04540d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0458cda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f045415f550 0x7f04540d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0458cda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04540d1040 con 0x7f04540d34b0 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0458cda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f04540c9430 con 0x7f045415f550 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0458cda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04540c9680 con 0x7f045415f170 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0451d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f045415f550 0x7f04540d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0452d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f04540d34b0 0x7f04540cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38638/0 (socket says 172.21.15.40:38638) 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0452d9d640 1 -- 172.21.15.40:0/1726354118 learned_addr learned my addr 172.21.15.40:0/1726354118 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f045259c640 1 --2- 172.21.15.40:0/1726354118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f04540d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0451d9b640 1 -- 172.21.15.40:0/1726354118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 msgr2=0x7f04540d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0451d9b640 1 --2- 172.21.15.40:0/1726354118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f04540d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0451d9b640 1 -- 172.21.15.40:0/1726354118 >> v1:172.21.15.40:6789/0 conn(0x7f04540d34b0 legacy=0x7f04540cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f0451d9b640 1 -- 172.21.15.40:0/1726354118 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04540c9960 con 0x7f045415f550 2024-04-26T21:35:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.164+0000 7f045159a640 1 -- 172.21.15.40:0/1726354118 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2151632237 0 0) 0x7f0444001490 con 0x7f04540d34b0 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.165+0000 7f0451d9b640 1 --2- 172.21.15.40:0/1726354118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f045415f550 0x7f04540d2cb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f043c0097a0 tx=0x7f043c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757bced750a3d18c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.165+0000 7f045159a640 1 -- 172.21.15.40:0/1726354118 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c037070 con 0x7f045415f550 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.165+0000 7f045159a640 1 -- 172.21.15.40:0/1726354118 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043c032440 con 0x7f045415f550 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.165+0000 7f045159a640 1 -- 172.21.15.40:0/1726354118 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c032810 con 0x7f045415f550 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.166+0000 7f0458cda640 1 -- 172.21.15.40:0/1726354118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f045415f550 msgr2=0x7f04540d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.166+0000 7f0458cda640 1 --2- 172.21.15.40:0/1726354118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f045415f550 0x7f04540d2cb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f043c0097a0 tx=0x7f043c02a180 comp rx=0 tx=0).stop 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.166+0000 7f0458cda640 1 -- 172.21.15.40:0/1726354118 shutdown_connections 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.166+0000 7f0458cda640 1 --2- 172.21.15.40:0/1726354118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f045415f550 0x7f04540d2cb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.166+0000 7f0458cda640 1 --2- 172.21.15.40:0/1726354118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f04540d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.166+0000 7f0458cda640 1 -- 172.21.15.40:0/1726354118 >> 172.21.15.40:0/1726354118 conn(0x7f04540d4f60 msgr2=0x7f04540d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.166+0000 7f0458cda640 1 -- 172.21.15.40:0/1726354118 shutdown_connections 2024-04-26T21:35:39.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.166+0000 7f0458cda640 1 -- 172.21.15.40:0/1726354118 wait complete. 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.167+0000 7f0458cda640 1 Processor -- start 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.167+0000 7f0458cda640 1 -- start start 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f0458cda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04540d34b0 0x7f0454159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f0458cda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f045414f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f0458cda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04540b9b90 0x7f04540ba040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f0458cda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04540cb5d0 con 0x7f04540d34b0 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f0458cda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04540cb450 con 0x7f04540b9b90 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f0458cda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04540cb750 con 0x7f045415f170 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f045259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04540d34b0 0x7f0454159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f045259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04540d34b0 0x7f0454159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41798/0 (socket says 172.21.15.40:41798) 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f045259c640 1 -- 172.21.15.40:0/616964427 learned_addr learned my addr 172.21.15.40:0/616964427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f0452d9d640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04540b9b90 0x7f04540ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.168+0000 7f0451d9b640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f045414f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f045259c640 1 -- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 msgr2=0x7f045414f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f045259c640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f045414f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f045259c640 1 -- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04540b9b90 msgr2=0x7f04540ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f045259c640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04540b9b90 0x7f04540ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f045259c640 1 -- 172.21.15.40:0/616964427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045414f8e0 con 0x7f04540d34b0 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f0452d9d640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04540b9b90 0x7f04540ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f0451d9b640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f045414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f045259c640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04540d34b0 0x7f0454159f70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f044c00e9e0 tx=0x7f044c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f044affd640 1 -- 172.21.15.40:0/616964427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c017070 con 0x7f04540d34b0 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f044affd640 1 -- 172.21.15.40:0/616964427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c00c970 con 0x7f04540d34b0 2024-04-26T21:35:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f044affd640 1 -- 172.21.15.40:0/616964427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c00cd60 con 0x7f04540d34b0 2024-04-26T21:35:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0454158f10 con 0x7f04540d34b0 2024-04-26T21:35:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.169+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0454159490 con 0x7f04540d34b0 2024-04-26T21:35:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.170+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04540694f0 con 0x7f04540d34b0 2024-04-26T21:35:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.170+0000 7f044affd640 1 -- 172.21.15.40:0/616964427 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f044c01a070 con 0x7f04540d34b0 2024-04-26T21:35:39.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.171+0000 7f044affd640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f043406c8f0 0x7f043406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.171+0000 7f044affd640 1 -- 172.21.15.40:0/616964427 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f044c08be40 con 0x7f04540d34b0 2024-04-26T21:35:39.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.171+0000 7f0451d9b640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f043406c8f0 0x7f043406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.172+0000 7f0451d9b640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f043406c8f0 0x7f043406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f043c034000 tx=0x7f043c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.175+0000 7f044affd640 1 -- 172.21.15.40:0/616964427 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f045414f8e0 con 0x7f04540d34b0 2024-04-26T21:35:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.361+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f04540b6e40 con 0x7f04540d34b0 2024-04-26T21:35:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.361+0000 7f044affd640 1 -- 172.21.15.40:0/616964427 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f044c059f30 con 0x7f04540d34b0 2024-04-26T21:35:39.362 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f043406c8f0 msgr2=0x7f043406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f043406c8f0 0x7f043406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f043c034000 tx=0x7f043c02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04540d34b0 msgr2=0x7f0454159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04540d34b0 0x7f0454159f70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f044c00e9e0 tx=0x7f044c00eea0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 shutdown_connections 2024-04-26T21:35:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f043406c8f0 0x7f043406edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04540b9b90 0x7f04540ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f045415f170 0x7f045414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 --2- 172.21.15.40:0/616964427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04540d34b0 0x7f0454159f70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 >> 172.21.15.40:0/616964427 conn(0x7f04540d4f60 msgr2=0x7f04540d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 shutdown_connections 2024-04-26T21:35:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.363+0000 7f0458cda640 1 -- 172.21.15.40:0/616964427 wait complete. 2024-04-26T21:35:39.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.477+0000 7f907c740640 1 Processor -- start 2024-04-26T21:35:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.477+0000 7f907c740640 1 -- start start 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.478+0000 7f907c740640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f907415f770 0x7f90740b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.478+0000 7f907c740640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f90740d1920 0x7f90740cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.478+0000 7f907c740640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907406e580 con 0x7f90740d0d60 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.478+0000 7f907c740640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f907406f3e0 con 0x7f907415f770 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.478+0000 7f907c740640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f90740cd6e0 con 0x7f90740d1920 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.478+0000 7f907a4dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f907415f770 0x7f90740b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.478+0000 7f907a4dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f907415f770 0x7f90740b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41802/0 (socket says 172.21.15.40:41802) 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.478+0000 7f907a4dc640 1 -- 172.21.15.40:0/916737472 learned_addr learned my addr 172.21.15.40:0/916737472 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.479+0000 7f907acdd640 1 --2- 172.21.15.40:0/916737472 >> v2:172.21.15.184:3300/0 conn(0x7f90740d1920 0x7f90740cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.479+0000 7f907a4dc640 1 -- 172.21.15.40:0/916737472 >> v2:172.21.15.184:3300/0 conn(0x7f90740d1920 msgr2=0x7f90740cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.479+0000 7f907a4dc640 1 --2- 172.21.15.40:0/916737472 >> v2:172.21.15.184:3300/0 conn(0x7f90740d1920 0x7f90740cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.479+0000 7f907a4dc640 1 -- 172.21.15.40:0/916737472 >> v1:172.21.15.184:6789/0 conn(0x7f90740d0d60 legacy=0x7f90740d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.479+0000 7f907a4dc640 1 -- 172.21.15.40:0/916737472 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907406e860 con 0x7f907415f770 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.479+0000 7f907a4dc640 1 --2- 172.21.15.40:0/916737472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b1fd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9074073c60 tx=0x7f906c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe02d10d5dce2d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f90794da640 1 -- 172.21.15.40:0/916737472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c037070 con 0x7f907415f770 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f90794da640 1 -- 172.21.15.40:0/916737472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906c032440 con 0x7f907415f770 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f90794da640 1 -- 172.21.15.40:0/916737472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c032810 con 0x7f907415f770 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f907c740640 1 -- 172.21.15.40:0/916737472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 msgr2=0x7f90740b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f907c740640 1 --2- 172.21.15.40:0/916737472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b1fd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9074073c60 tx=0x7f906c02a270 comp rx=0 tx=0).stop 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f907c740640 1 -- 172.21.15.40:0/916737472 shutdown_connections 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f907c740640 1 --2- 172.21.15.40:0/916737472 >> v2:172.21.15.184:3300/0 conn(0x7f90740d1920 0x7f90740cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f907c740640 1 --2- 172.21.15.40:0/916737472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b1fd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f907c740640 1 -- 172.21.15.40:0/916737472 >> 172.21.15.40:0/916737472 conn(0x7f90740d2e60 msgr2=0x7f90740d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f907c740640 1 -- 172.21.15.40:0/916737472 shutdown_connections 2024-04-26T21:35:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.480+0000 7f907c740640 1 -- 172.21.15.40:0/916737472 wait complete. 2024-04-26T21:35:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.481+0000 7f907c740640 1 Processor -- start 2024-04-26T21:35:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.481+0000 7f907c740640 1 -- start start 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907c740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90740d0d60 0x7f90741f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907c740640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90740d1920 0x7f907415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907c740640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907c740640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90740ce230 con 0x7f907415f770 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907c740640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90740ce0b0 con 0x7f90740d1920 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907c740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90740ce3b0 con 0x7f90740d0d60 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907acdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907acdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41812/0 (socket says 172.21.15.40:41812) 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907acdd640 1 -- 172.21.15.40:0/2931719603 learned_addr learned my addr 172.21.15.40:0/2931719603 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f9079cdb640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90740d1920 0x7f907415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907acdd640 1 -- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90740d0d60 msgr2=0x7f90741f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.482+0000 7f907a4dc640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90740d0d60 0x7f90741f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f907acdd640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90740d0d60 0x7f90741f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f907acdd640 1 -- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90740d1920 msgr2=0x7f907415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f907acdd640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90740d1920 0x7f907415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f907acdd640 1 -- 172.21.15.40:0/2931719603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90740b75d0 con 0x7f907415f770 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f9079cdb640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90740d1920 0x7f907415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f907a4dc640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90740d0d60 0x7f90741f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f907acdd640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b5340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9064007e40 tx=0x7f906400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f9072ffd640 1 -- 172.21.15.40:0/2931719603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064014620 con 0x7f907415f770 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f9072ffd640 1 -- 172.21.15.40:0/2931719603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9064019070 con 0x7f907415f770 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90740b78b0 con 0x7f907415f770 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f9072ffd640 1 -- 172.21.15.40:0/2931719603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064014a20 con 0x7f907415f770 2024-04-26T21:35:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.483+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90740b7e70 con 0x7f907415f770 2024-04-26T21:35:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.484+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90740694f0 con 0x7f907415f770 2024-04-26T21:35:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.484+0000 7f9072ffd640 1 -- 172.21.15.40:0/2931719603 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9064014bc0 con 0x7f907415f770 2024-04-26T21:35:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.485+0000 7f9072ffd640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906006c8f0 0x7f906006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.485+0000 7f9072ffd640 1 -- 172.21.15.40:0/2931719603 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f906408c750 con 0x7f907415f770 2024-04-26T21:35:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.485+0000 7f907a4dc640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906006c8f0 0x7f906006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.486+0000 7f907a4dc640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906006c8f0 0x7f906006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f906c034000 tx=0x7f906c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.489+0000 7f9072ffd640 1 -- 172.21.15.40:0/2931719603 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f906405a840 con 0x7f907415f770 2024-04-26T21:35:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.674+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90740d3d90 con 0x7f907415f770 2024-04-26T21:35:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.675+0000 7f9072ffd640 1 -- 172.21.15.40:0/2931719603 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f906405a140 con 0x7f907415f770 2024-04-26T21:35:39.676 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906006c8f0 msgr2=0x7f906006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906006c8f0 0x7f906006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f906c034000 tx=0x7f906c02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 msgr2=0x7f90740b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b5340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9064007e40 tx=0x7f906400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 shutdown_connections 2024-04-26T21:35:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906006c8f0 0x7f906006edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907415f770 0x7f90740b5340 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90740d1920 0x7f907415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 --2- 172.21.15.40:0/2931719603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90740d0d60 0x7f90741f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 >> 172.21.15.40:0/2931719603 conn(0x7f90740d2e60 msgr2=0x7f907415e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 shutdown_connections 2024-04-26T21:35:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.677+0000 7f907c740640 1 -- 172.21.15.40:0/2931719603 wait complete. 2024-04-26T21:35:39.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.791+0000 7f78fd722640 1 Processor -- start 2024-04-26T21:35:39.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.791+0000 7f78fd722640 1 -- start start 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.792+0000 7f78fd722640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78f8072fb0 0x7f78f80b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.792+0000 7f78fd722640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f80b5e30 con 0x7f78f8073390 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.792+0000 7f78fd722640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f80b6110 con 0x7f78f80b06f0 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.792+0000 7f78fd722640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f78f80b0b80 con 0x7f78f8072fb0 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.792+0000 7f78f6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78f8072fb0 0x7f78f80b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.792+0000 7f78f67fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78f8073390 0x7f78f806f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38642/0 (socket says 172.21.15.40:38642) 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.792+0000 7f78f67fc640 1 -- 172.21.15.40:0/2753885487 learned_addr learned my addr 172.21.15.40:0/2753885487 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.793+0000 7f78f6ffd640 1 -- 172.21.15.40:0/2753885487 >> v1:172.21.15.184:6789/0 conn(0x7f78f80b06f0 legacy=0x7f78f806fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.793+0000 7f78f6ffd640 1 -- 172.21.15.40:0/2753885487 >> v1:172.21.15.40:6789/0 conn(0x7f78f8073390 legacy=0x7f78f806f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.793+0000 7f78f6ffd640 1 -- 172.21.15.40:0/2753885487 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f80b63f0 con 0x7f78f8072fb0 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.793+0000 7f78f6ffd640 1 --2- 172.21.15.40:0/2753885487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f80b56c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009d90 tx=0x7f78f002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8b6636e19c0a8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.793+0000 7f78f5ffb640 1 -- 172.21.15.40:0/2753885487 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0007740 con 0x7f78f8072fb0 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.793+0000 7f78f5ffb640 1 -- 172.21.15.40:0/2753885487 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f00078e0 con 0x7f78f8072fb0 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.793+0000 7f78f5ffb640 1 -- 172.21.15.40:0/2753885487 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0007cb0 con 0x7f78f8072fb0 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.794+0000 7f78fd722640 1 -- 172.21.15.40:0/2753885487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 msgr2=0x7f78f80b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.794+0000 7f78fd722640 1 --2- 172.21.15.40:0/2753885487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f80b56c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009d90 tx=0x7f78f002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.794+0000 7f78fd722640 1 -- 172.21.15.40:0/2753885487 shutdown_connections 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.794+0000 7f78fd722640 1 --2- 172.21.15.40:0/2753885487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f80b56c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.794+0000 7f78fd722640 1 -- 172.21.15.40:0/2753885487 >> 172.21.15.40:0/2753885487 conn(0x7f78f80b39b0 msgr2=0x7f78f80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.794+0000 7f78fd722640 1 -- 172.21.15.40:0/2753885487 shutdown_connections 2024-04-26T21:35:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.794+0000 7f78fd722640 1 -- 172.21.15.40:0/2753885487 wait complete. 2024-04-26T21:35:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.795+0000 7f78fd722640 1 Processor -- start 2024-04-26T21:35:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.795+0000 7f78fd722640 1 -- start start 2024-04-26T21:35:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78fd722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78fd722640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8073390 0x7f78f81f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78fd722640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f80b06f0 0x7f78f81f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78fd722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78f806f2b0 con 0x7f78f8072fb0 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78fd722640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78f815c160 con 0x7f78f80b06f0 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78fd722640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78f815c2e0 con 0x7f78f8073390 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f815ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41828/0 (socket says 172.21.15.40:41828) 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f6ffd640 1 -- 172.21.15.40:0/988174355 learned_addr learned my addr 172.21.15.40:0/988174355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f67fc640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8073390 0x7f78f81f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f77fe640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f80b06f0 0x7f78f81f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f6ffd640 1 -- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8073390 msgr2=0x7f78f81f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f6ffd640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8073390 0x7f78f81f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f6ffd640 1 -- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f80b06f0 msgr2=0x7f78f81f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f6ffd640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f80b06f0 0x7f78f81f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f6ffd640 1 -- 172.21.15.40:0/988174355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f815c4f0 con 0x7f78f8072fb0 2024-04-26T21:35:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f67fc640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8073390 0x7f78f81f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.796+0000 7f78f77fe640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f80b06f0 0x7f78f81f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.797+0000 7f78f6ffd640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f815ec20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009d90 tx=0x7f78f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.797+0000 7f78ef7fe640 1 -- 172.21.15.40:0/988174355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f003f070 con 0x7f78f8072fb0 2024-04-26T21:35:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.797+0000 7f78ef7fe640 1 -- 172.21.15.40:0/988174355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f0031bc0 con 0x7f78f8072fb0 2024-04-26T21:35:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.797+0000 7f78ef7fe640 1 -- 172.21.15.40:0/988174355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0044460 con 0x7f78f8072fb0 2024-04-26T21:35:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.797+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f815f2e0 con 0x7f78f8072fb0 2024-04-26T21:35:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.797+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f815ca90 con 0x7f78f8072fb0 2024-04-26T21:35:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.798+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f80694f0 con 0x7f78f8072fb0 2024-04-26T21:35:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.798+0000 7f78ef7fe640 1 -- 172.21.15.40:0/988174355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78f003b410 con 0x7f78f8072fb0 2024-04-26T21:35:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.799+0000 7f78ef7fe640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.799+0000 7f78ef7fe640 1 -- 172.21.15.40:0/988174355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f78f00ad850 con 0x7f78f8072fb0 2024-04-26T21:35:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.799+0000 7f78f67fc640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.799+0000 7f78f67fc640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f78f815e750 tx=0x7f78e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:39.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.803+0000 7f78ef7fe640 1 -- 172.21.15.40:0/988174355 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78f007b940 con 0x7f78f8072fb0 2024-04-26T21:35:39.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:35:39.966 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:35:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.989+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f78f8158920 con 0x7f78f8072fb0 2024-04-26T21:35:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.989+0000 7f78ef7fe640 1 -- 172.21.15.40:0/988174355 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f78f007b240 con 0x7f78f8072fb0 2024-04-26T21:35:39.990 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 msgr2=0x7f78d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f78f815e750 tx=0x7f78e0009040 comp rx=0 tx=0).stop 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 msgr2=0x7f78f815ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f815ec20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009d90 tx=0x7f78f0004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 shutdown_connections 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f80b06f0 0x7f78f81f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8073390 0x7f78f81f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 --2- 172.21.15.40:0/988174355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f8072fb0 0x7f78f815ec20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.991+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 >> 172.21.15.40:0/988174355 conn(0x7f78f80b39b0 msgr2=0x7f78f8159490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.992+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 shutdown_connections 2024-04-26T21:35:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:39.992+0000 7f78fd722640 1 -- 172.21.15.40:0/988174355 wait complete. 2024-04-26T21:35:40.003 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:35:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.173+0000 7f85af683640 1 Processor -- start 2024-04-26T21:35:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.173+0000 7f85af683640 1 -- start start 2024-04-26T21:35:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.174+0000 7f85af683640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f85a815ab60 0x7f85a806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.174+0000 7f85af683640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a81617f0 0x7f85a806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.174+0000 7f85af683640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a815af40 con 0x7f85a815d5f0 2024-04-26T21:35:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.174+0000 7f85af683640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f85a815e320 con 0x7f85a815ab60 2024-04-26T21:35:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.174+0000 7f85af683640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85a8160d10 con 0x7f85a81617f0 2024-04-26T21:35:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.175+0000 7f85ad41f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f85a815ab60 0x7f85a806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.175+0000 7f85acc1e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a81617f0 0x7f85a806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.175+0000 7f85adc20640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f85a815d5f0 0x7f85a815daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:50240/0 (socket says 172.21.15.40:50240) 2024-04-26T21:35:40.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.175+0000 7f85adc20640 1 -- 172.21.15.40:0/3460602330 learned_addr learned my addr 172.21.15.40:0/3460602330 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:40.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.175+0000 7f85acc1e640 1 -- 172.21.15.40:0/3460602330 >> v2:172.21.15.184:3300/0 conn(0x7f85a815ab60 msgr2=0x7f85a806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:40.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.175+0000 7f85acc1e640 1 --2- 172.21.15.40:0/3460602330 >> v2:172.21.15.184:3300/0 conn(0x7f85a815ab60 0x7f85a806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.175+0000 7f85acc1e640 1 -- 172.21.15.40:0/3460602330 >> v1:172.21.15.184:6789/0 conn(0x7f85a815d5f0 legacy=0x7f85a815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.175+0000 7f85acc1e640 1 -- 172.21.15.40:0/3460602330 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a8160ff0 con 0x7f85a81617f0 2024-04-26T21:35:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85acc1e640 1 --2- 172.21.15.40:0/3460602330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a81617f0 0x7f85a806c780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f859c0097a0 tx=0x7f859c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d293ad6ef0735793 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85a7fff640 1 -- 172.21.15.40:0/3460602330 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c037070 con 0x7f85a81617f0 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85a7fff640 1 -- 172.21.15.40:0/3460602330 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c032440 con 0x7f85a81617f0 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85a7fff640 1 -- 172.21.15.40:0/3460602330 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c032810 con 0x7f85a81617f0 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85af683640 1 -- 172.21.15.40:0/3460602330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a81617f0 msgr2=0x7f85a806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85af683640 1 --2- 172.21.15.40:0/3460602330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a81617f0 0x7f85a806c780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f859c0097a0 tx=0x7f859c02a180 comp rx=0 tx=0).stop 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85af683640 1 -- 172.21.15.40:0/3460602330 shutdown_connections 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85af683640 1 --2- 172.21.15.40:0/3460602330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a81617f0 0x7f85a806c780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85af683640 1 --2- 172.21.15.40:0/3460602330 >> v2:172.21.15.184:3300/0 conn(0x7f85a815ab60 0x7f85a806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85af683640 1 -- 172.21.15.40:0/3460602330 >> 172.21.15.40:0/3460602330 conn(0x7f85a80d1cf0 msgr2=0x7f85a80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85af683640 1 -- 172.21.15.40:0/3460602330 shutdown_connections 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.176+0000 7f85af683640 1 -- 172.21.15.40:0/3460602330 wait complete. 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85af683640 1 Processor -- start 2024-04-26T21:35:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85af683640 1 -- start start 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85af683640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a815ab60 0x7f85a80d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85af683640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a815d5f0 0x7f85a8143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85af683640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85a81617f0 0x7f85a81588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85af683640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85a80cc4c0 con 0x7f85a81617f0 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85af683640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85a80cc340 con 0x7f85a815d5f0 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85af683640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85a80cc640 con 0x7f85a815ab60 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85adc20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85a81617f0 0x7f85a81588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85adc20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85a81617f0 0x7f85a81588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41842/0 (socket says 172.21.15.40:41842) 2024-04-26T21:35:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.178+0000 7f85adc20640 1 -- 172.21.15.40:0/2704316390 learned_addr learned my addr 172.21.15.40:0/2704316390 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85acc1e640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a815d5f0 0x7f85a8143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85ad41f640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a815ab60 0x7f85a80d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85adc20640 1 -- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a815ab60 msgr2=0x7f85a80d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85adc20640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a815ab60 0x7f85a80d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85adc20640 1 -- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a815d5f0 msgr2=0x7f85a8143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85adc20640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a815d5f0 0x7f85a8143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85adc20640 1 -- 172.21.15.40:0/2704316390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a8143fc0 con 0x7f85a81617f0 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85adc20640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85a81617f0 0x7f85a81588a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f859800cbc0 tx=0x7f859800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85a5ffb640 1 -- 172.21.15.40:0/2704316390 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598017070 con 0x7f85a81617f0 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85a5ffb640 1 -- 172.21.15.40:0/2704316390 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859800ad40 con 0x7f85a81617f0 2024-04-26T21:35:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a80d0770 con 0x7f85a81617f0 2024-04-26T21:35:40.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85a5ffb640 1 -- 172.21.15.40:0/2704316390 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598012600 con 0x7f85a81617f0 2024-04-26T21:35:40.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.179+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a80b5dc0 con 0x7f85a81617f0 2024-04-26T21:35:40.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.181+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a80694f0 con 0x7f85a81617f0 2024-04-26T21:35:40.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.181+0000 7f85a5ffb640 1 -- 172.21.15.40:0/2704316390 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f859801a070 con 0x7f85a81617f0 2024-04-26T21:35:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.181+0000 7f85a5ffb640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859406c8f0 0x7f859406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.181+0000 7f85a5ffb640 1 -- 172.21.15.40:0/2704316390 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f859808bb50 con 0x7f85a81617f0 2024-04-26T21:35:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.181+0000 7f85ad41f640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859406c8f0 0x7f859406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.182+0000 7f85ad41f640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859406c8f0 0x7f859406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009b40 tx=0x7f85a0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.186+0000 7f85a5ffb640 1 -- 172.21.15.40:0/2704316390 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8598059600 con 0x7f85a81617f0 2024-04-26T21:35:40.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.408+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85a80b1c00 con 0x7f85a81617f0 2024-04-26T21:35:40.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.409+0000 7f85a5ffb640 1 -- 172.21.15.40:0/2704316390 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f859805d900 con 0x7f85a81617f0 2024-04-26T21:35:40.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:40.410 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:31.566271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:35:40.411 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-26T21:35:40.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859406c8f0 msgr2=0x7f859406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859406c8f0 0x7f859406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009b40 tx=0x7f85a0009190 comp rx=0 tx=0).stop 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85a81617f0 msgr2=0x7f85a81588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85a81617f0 0x7f85a81588a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f859800cbc0 tx=0x7f859800a430 comp rx=0 tx=0).stop 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 shutdown_connections 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859406c8f0 0x7f859406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85a81617f0 0x7f85a81588a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85a815d5f0 0x7f85a8143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 --2- 172.21.15.40:0/2704316390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85a815ab60 0x7f85a80d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 >> 172.21.15.40:0/2704316390 conn(0x7f85a80d1cf0 msgr2=0x7f85a80cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 shutdown_connections 2024-04-26T21:35:40.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.411+0000 7f85af683640 1 -- 172.21.15.40:0/2704316390 wait complete. 2024-04-26T21:35:40.423 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:40.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.524+0000 7f9e5282a640 1 Processor -- start 2024-04-26T21:35:40.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.524+0000 7f9e5282a640 1 -- start start 2024-04-26T21:35:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.524+0000 7f9e5282a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9e4c0d06f0 0x7f9e4c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.524+0000 7f9e5282a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c0b2850 0x7f9e4c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.524+0000 7f9e5282a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4c0d0ba0 con 0x7f9e4c15f3a0 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.524+0000 7f9e5282a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9e4c0d0e20 con 0x7f9e4c0d06f0 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.524+0000 7f9e5282a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c0b1d70 con 0x7f9e4c0b2850 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9e4c15f3a0 0x7f9e4c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38654/0 (socket says 172.21.15.40:38654) 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4bfff640 1 -- 172.21.15.40:0/1452893036 learned_addr learned my addr 172.21.15.40:0/1452893036 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4b7fe640 1 --2- 172.21.15.40:0/1452893036 >> v2:172.21.15.184:3300/0 conn(0x7f9e4c0d06f0 0x7f9e4c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e50dc7640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c0b2850 0x7f9e4c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844680294 0 0) 0x7f9e3c001110 con 0x7f9e4c15f3a0 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e34003520 con 0x7f9e4c15f3a0 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2490534132 0 0) 0x7f9e34003520 con 0x7f9e4c15f3a0 2024-04-26T21:35:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c0b2850 msgr2=0x7f9e4c0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c0b2850 0x7f9e4c0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 >> v2:172.21.15.184:3300/0 conn(0x7f9e4c0d06f0 msgr2=0x7f9e4c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 --2- 172.21.15.40:0/1452893036 >> v2:172.21.15.184:3300/0 conn(0x7f9e4c0d06f0 0x7f9e4c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c0b2050 con 0x7f9e4c15f3a0 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9e3c001840 con 0x7f9e4c15f3a0 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4b7fe640 1 --2- 172.21.15.40:0/1452893036 >> v2:172.21.15.184:3300/0 conn(0x7f9e4c0d06f0 0x7f9e4c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 >> v1:172.21.15.40:6789/0 conn(0x7f9e4c15f3a0 legacy=0x7f9e4c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c0b2850 0x7f9e340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e34003f30 0x7f9e34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e34004d40 0x7f9e340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c0d0e20 con 0x7f9e34004d40 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c0b1d70 con 0x7f9e34003f30 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3400b9c0 con 0x7f9e4c0b2850 2024-04-26T21:35:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e50dc7640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e34004d40 0x7f9e340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4bfff640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c0b2850 0x7f9e340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.525+0000 7f9e4b7fe640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e34003f30 0x7f9e34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.526+0000 7f9e50dc7640 1 -- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c0b2850 msgr2=0x7f9e340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.526+0000 7f9e50dc7640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c0b2850 0x7f9e340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.526+0000 7f9e50dc7640 1 -- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e34003f30 msgr2=0x7f9e34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.526+0000 7f9e50dc7640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e34003f30 0x7f9e34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.526+0000 7f9e50dc7640 1 -- 172.21.15.40:0/1452893036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e34003520 con 0x7f9e34004d40 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.526+0000 7f9e50dc7640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e34004d40 0x7f9e340051c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009fc0 tx=0x7f9e34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed16ac3d22bec64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44009c10 con 0x7f9e34004d40 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44036070 con 0x7f9e34004d40 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e4affd640 1 -- 172.21.15.40:0/1452893036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44031670 con 0x7f9e34004d40 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e5282a640 1 -- 172.21.15.40:0/1452893036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e34004d40 msgr2=0x7f9e340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e5282a640 1 --2- 172.21.15.40:0/1452893036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e34004d40 0x7f9e340051c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009fc0 tx=0x7f9e34005830 comp rx=0 tx=0).stop 2024-04-26T21:35:40.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e50dc7640 1 -- 172.21.15.40:0/1452893036 reap_dead start 2024-04-26T21:35:40.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e5282a640 1 -- 172.21.15.40:0/1452893036 shutdown_connections 2024-04-26T21:35:40.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e5282a640 1 -- 172.21.15.40:0/1452893036 >> 172.21.15.40:0/1452893036 conn(0x7f9e4c0d4f90 msgr2=0x7f9e4c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:40.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e5282a640 1 -- 172.21.15.40:0/1452893036 shutdown_connections 2024-04-26T21:35:40.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.527+0000 7f9e5282a640 1 -- 172.21.15.40:0/1452893036 wait complete. 2024-04-26T21:35:40.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.528+0000 7f9e5282a640 1 Processor -- start 2024-04-26T21:35:40.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e5282a640 1 -- start start 2024-04-26T21:35:40.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e5282a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c15d660 0x7f9e4c0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e5282a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e4c15b850 0x7f9e4c1f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e5282a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c161a90 0x7f9e4c161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e5282a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c15f900 con 0x7f9e4c15b850 2024-04-26T21:35:40.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e5282a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c15f780 con 0x7f9e4c161a90 2024-04-26T21:35:40.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e5282a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c0d23c0 con 0x7f9e4c15d660 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e4b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e4c15b850 0x7f9e4c1f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e4b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e4c15b850 0x7f9e4c1f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41854/0 (socket says 172.21.15.40:41854) 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.529+0000 7f9e4b7fe640 1 -- 172.21.15.40:0/2298858408 learned_addr learned my addr 172.21.15.40:0/2298858408 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e50dc7640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c161a90 0x7f9e4c161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e4bfff640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c15d660 0x7f9e4c0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e4b7fe640 1 -- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c15d660 msgr2=0x7f9e4c0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e4b7fe640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c15d660 0x7f9e4c0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e4b7fe640 1 -- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c161a90 msgr2=0x7f9e4c161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e4b7fe640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c161a90 0x7f9e4c161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e4b7fe640 1 -- 172.21.15.40:0/2298858408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c0ca070 con 0x7f9e4c15b850 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e4b7fe640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e4c15b850 0x7f9e4c1f8e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e40011d00 tx=0x7f9e4000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e48ff9640 1 -- 172.21.15.40:0/2298858408 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000fcc0 con 0x7f9e4c15b850 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e48ff9640 1 -- 172.21.15.40:0/2298858408 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4000fe60 con 0x7f9e4c15b850 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c0ca350 con 0x7f9e4c15b850 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e48ff9640 1 -- 172.21.15.40:0/2298858408 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40004270 con 0x7f9e4c15b850 2024-04-26T21:35:40.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.530+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c0ca910 con 0x7f9e4c15b850 2024-04-26T21:35:40.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.531+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4c0694f0 con 0x7f9e4c15b850 2024-04-26T21:35:40.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.532+0000 7f9e48ff9640 1 -- 172.21.15.40:0/2298858408 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e40012040 con 0x7f9e4c15b850 2024-04-26T21:35:40.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.532+0000 7f9e48ff9640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e3806c920 0x7f9e3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.532+0000 7f9e48ff9640 1 -- 172.21.15.40:0/2298858408 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9e4008c430 con 0x7f9e4c15b850 2024-04-26T21:35:40.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.532+0000 7f9e4bfff640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e3806c920 0x7f9e3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.533+0000 7f9e4bfff640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e3806c920 0x7f9e3806ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0017a0 tx=0x7f9e3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.536+0000 7f9e48ff9640 1 -- 172.21.15.40:0/2298858408 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e40019070 con 0x7f9e4c15b850 2024-04-26T21:35:40.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.724+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e4c0b6860 con 0x7f9e4c15b850 2024-04-26T21:35:40.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.725+0000 7f9e48ff9640 1 -- 172.21.15.40:0/2298858408 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f9e4005a480 con 0x7f9e4c15b850 2024-04-26T21:35:40.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:40.727 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e3806c920 msgr2=0x7f9e3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e3806c920 0x7f9e3806ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0017a0 tx=0x7f9e3c009040 comp rx=0 tx=0).stop 2024-04-26T21:35:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e4c15b850 msgr2=0x7f9e4c1f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e4c15b850 0x7f9e4c1f8e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e40011d00 tx=0x7f9e4000f610 comp rx=0 tx=0).stop 2024-04-26T21:35:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 shutdown_connections 2024-04-26T21:35:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e3806c920 0x7f9e3806ede0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c161a90 0x7f9e4c161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e4c15b850 0x7f9e4c1f8e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 --2- 172.21.15.40:0/2298858408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4c15d660 0x7f9e4c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 >> 172.21.15.40:0/2298858408 conn(0x7f9e4c0d4f90 msgr2=0x7f9e4c06e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:40.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 shutdown_connections 2024-04-26T21:35:40.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.727+0000 7f9e5282a640 1 -- 172.21.15.40:0/2298858408 wait complete. 2024-04-26T21:35:40.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:40.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:40.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.845+0000 7f4cd1612640 1 Processor -- start 2024-04-26T21:35:40.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.845+0000 7f4cd1612640 1 -- start start 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cd1612640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4ccc0d1a00 0x7f4ccc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cd1612640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ccc0d2320 0x7f4ccc1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cd1612640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 0x7f4ccc162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cd1612640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4ccc163070 con 0x7f4ccc0d2320 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cd1612640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4ccc0d0a10 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cd1612640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ccc0d0cf0 con 0x7f4ccc0b40d0 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cca7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ccc0d2320 0x7f4ccc1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cca7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ccc0d2320 0x7f4ccc1625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41868/0 (socket says 172.21.15.40:41868) 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cca7fc640 1 -- 172.21.15.40:0/2028647957 learned_addr learned my addr 172.21.15.40:0/2028647957 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4ccaffd640 1 --2- 172.21.15.40:0/2028647957 >> v2:172.21.15.184:3300/0 conn(0x7f4ccc0d1a00 0x7f4ccc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4ccb7fe640 1 --2- 172.21.15.40:0/2028647957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 0x7f4ccc162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cca7fc640 1 -- 172.21.15.40:0/2028647957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 msgr2=0x7f4ccc162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cca7fc640 1 --2- 172.21.15.40:0/2028647957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 0x7f4ccc162b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cca7fc640 1 -- 172.21.15.40:0/2028647957 >> v2:172.21.15.184:3300/0 conn(0x7f4ccc0d1a00 msgr2=0x7f4ccc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cca7fc640 1 --2- 172.21.15.40:0/2028647957 >> v2:172.21.15.184:3300/0 conn(0x7f4ccc0d1a00 0x7f4ccc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.846+0000 7f4cca7fc640 1 -- 172.21.15.40:0/2028647957 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc0d0fd0 con 0x7f4ccc0d2320 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.847+0000 7f4cca7fc640 1 --2- 172.21.15.40:0/2028647957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d2320 0x7f4ccc1625f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc06c650 tx=0x7f4cbc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6debf2280320308f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.847+0000 7f4cc9ffb640 1 -- 172.21.15.40:0/2028647957 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc037070 con 0x7f4ccc0d2320 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.847+0000 7f4cc9ffb640 1 -- 172.21.15.40:0/2028647957 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cbc02ae00 con 0x7f4ccc0d2320 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.847+0000 7f4cc9ffb640 1 -- 172.21.15.40:0/2028647957 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc032660 con 0x7f4ccc0d2320 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 -- 172.21.15.40:0/2028647957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d2320 msgr2=0x7f4ccc1625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 --2- 172.21.15.40:0/2028647957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d2320 0x7f4ccc1625f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc06c650 tx=0x7f4cbc02a0b0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 -- 172.21.15.40:0/2028647957 shutdown_connections 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 --2- 172.21.15.40:0/2028647957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 0x7f4ccc162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 --2- 172.21.15.40:0/2028647957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d2320 0x7f4ccc1625f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 --2- 172.21.15.40:0/2028647957 >> v2:172.21.15.184:3300/0 conn(0x7f4ccc0d1a00 0x7f4ccc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 -- 172.21.15.40:0/2028647957 >> 172.21.15.40:0/2028647957 conn(0x7f4ccc0d4fd0 msgr2=0x7f4ccc0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 -- 172.21.15.40:0/2028647957 shutdown_connections 2024-04-26T21:35:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.848+0000 7f4cd1612640 1 -- 172.21.15.40:0/2028647957 wait complete. 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.849+0000 7f4cd1612640 1 Processor -- start 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.849+0000 7f4cd1612640 1 -- start start 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cd1612640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 0x7f4ccc15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cd1612640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d1a00 0x7f4ccc0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cd1612640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ccc0d2320 0x7f4ccc0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cd1612640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc0d3050 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cd1612640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc0d2ed0 con 0x7f4ccc0d2320 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cd1612640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ccc0d31d0 con 0x7f4ccc0b40d0 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cca7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d1a00 0x7f4ccc0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cca7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d1a00 0x7f4ccc0ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41874/0 (socket says 172.21.15.40:41874) 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4cca7fc640 1 -- 172.21.15.40:0/869856898 learned_addr learned my addr 172.21.15.40:0/869856898 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4ccaffd640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 0x7f4ccc15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.850+0000 7f4ccb7fe640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ccc0d2320 0x7f4ccc0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cca7fc640 1 -- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 msgr2=0x7f4ccc15e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cca7fc640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 0x7f4ccc15e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cca7fc640 1 -- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ccc0d2320 msgr2=0x7f4ccc0b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cca7fc640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ccc0d2320 0x7f4ccc0b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cca7fc640 1 -- 172.21.15.40:0/869856898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc0cd190 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4ccb7fe640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ccc0d2320 0x7f4ccc0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cca7fc640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d1a00 0x7f4ccc0ce1f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc000c00 tx=0x7f4cbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cb37fe640 1 -- 172.21.15.40:0/869856898 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc041030 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cb37fe640 1 -- 172.21.15.40:0/869856898 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cbc02ac00 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ccc0cd430 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cb37fe640 1 -- 172.21.15.40:0/869856898 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc037040 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.851+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ccc0cd9f0 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.852+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ccc0694f0 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.853+0000 7f4cb37fe640 1 -- 172.21.15.40:0/869856898 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cbc044070 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.853+0000 7f4cb37fe640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4cb806c8f0 0x7f4cb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.853+0000 7f4cb37fe640 1 -- 172.21.15.40:0/869856898 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4cbc0ac180 con 0x7f4ccc0d1a00 2024-04-26T21:35:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.853+0000 7f4ccaffd640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4cb806c8f0 0x7f4cb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.854+0000 7f4ccaffd640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4cb806c8f0 0x7f4cb806edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009b40 tx=0x7f4cc0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:40.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:40.862+0000 7f4cb37fe640 1 -- 172.21.15.40:0/869856898 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4cbc04b070 con 0x7f4ccc0d1a00 2024-04-26T21:35:41.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.049+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4ccc162b10 con 0x7f4ccc0d1a00 2024-04-26T21:35:41.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.049+0000 7f4cb37fe640 1 -- 172.21.15.40:0/869856898 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f4cbc07a270 con 0x7f4ccc0d1a00 2024-04-26T21:35:41.050 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:41.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4cb806c8f0 msgr2=0x7f4cb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4cb806c8f0 0x7f4cb806edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009b40 tx=0x7f4cc0009190 comp rx=0 tx=0).stop 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d1a00 msgr2=0x7f4ccc0ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d1a00 0x7f4ccc0ce1f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc000c00 tx=0x7f4cbc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 shutdown_connections 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4cb806c8f0 0x7f4cb806edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ccc0d2320 0x7f4ccc0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc0d1a00 0x7f4ccc0ce1f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 --2- 172.21.15.40:0/869856898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ccc0b40d0 0x7f4ccc15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 >> 172.21.15.40:0/869856898 conn(0x7f4ccc0d4fd0 msgr2=0x7f4ccc0cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 shutdown_connections 2024-04-26T21:35:41.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.051+0000 7f4cd1612640 1 -- 172.21.15.40:0/869856898 wait complete. 2024-04-26T21:35:41.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.168+0000 7febfc607640 1 Processor -- start 2024-04-26T21:35:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.168+0000 7febfc607640 1 -- start start 2024-04-26T21:35:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febfc607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf40cc480 0x7febf4072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febfc607640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf415d7c0 con 0x7febf4072780 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febfc607640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf415daa0 con 0x7febf4071e60 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febfc607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7febf415dd80 con 0x7febf40cc480 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febfaba4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7febf4072780 0x7febf4166ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38656/0 (socket says 172.21.15.40:38656) 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febfaba4640 1 -- 172.21.15.40:0/23658834 learned_addr learned my addr 172.21.15.40:0/23658834 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febf9ba2640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf40cc480 0x7febf4072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767179142 0 0) 0x7febf415d7c0 con 0x7febf4072780 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.169+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febe0003520 con 0x7febf4072780 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2932351275 0 0) 0x7febe0003520 con 0x7febf4072780 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf40cc480 msgr2=0x7febf4072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf40cc480 0x7febf4072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 >> v1:172.21.15.184:6789/0 conn(0x7febf4071e60 legacy=0x7febf406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf415e060 con 0x7febf4072780 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7febec0019f0 con 0x7febf4072780 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 >> v1:172.21.15.40:6789/0 conn(0x7febf4072780 legacy=0x7febf4166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe00039e0 0x7febe0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe0004390 0x7febe0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf40cc480 0x7febe00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7febf415dd80 con 0x7febe00039e0 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe000b9c0 con 0x7febe0004390 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7febe000bbc0 con 0x7febf40cc480 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febfa3a3640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe00039e0 0x7febe0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febfaba4640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf40cc480 0x7febe00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febf9ba2640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe0004390 0x7febe0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febfa3a3640 1 -- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf40cc480 msgr2=0x7febe00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febfa3a3640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf40cc480 0x7febe00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febfa3a3640 1 -- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe0004390 msgr2=0x7febe0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febfa3a3640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe0004390 0x7febe0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.170+0000 7febfa3a3640 1 -- 172.21.15.40:0/23658834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe0003520 con 0x7febe00039e0 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.171+0000 7febfa3a3640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe00039e0 0x7febe0003dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7febf0001860 tx=0x7febe0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7265c801a6c65285 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.171+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0037070 con 0x7febe00039e0 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.171+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf002ac30 con 0x7febe00039e0 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.171+0000 7febf93a1640 1 -- 172.21.15.40:0/23658834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0001430 con 0x7febe00039e0 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.172+0000 7febfc607640 1 -- 172.21.15.40:0/23658834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe00039e0 msgr2=0x7febe0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.172+0000 7febfc607640 1 --2- 172.21.15.40:0/23658834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe00039e0 0x7febe0003dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7febf0001860 tx=0x7febe0005830 comp rx=0 tx=0).stop 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.172+0000 7febfaba4640 1 -- 172.21.15.40:0/23658834 reap_dead start 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.172+0000 7febfc607640 1 -- 172.21.15.40:0/23658834 shutdown_connections 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.172+0000 7febfc607640 1 -- 172.21.15.40:0/23658834 >> 172.21.15.40:0/23658834 conn(0x7febf40d0be0 msgr2=0x7febf40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.172+0000 7febfc607640 1 -- 172.21.15.40:0/23658834 shutdown_connections 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.172+0000 7febfc607640 1 -- 172.21.15.40:0/23658834 wait complete. 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.174+0000 7febfc607640 1 Processor -- start 2024-04-26T21:35:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.174+0000 7febfc607640 1 -- start start 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febfc607640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe00039e0 0x7febf4159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febfc607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febf40afee0 0x7febf40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febfc607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf415bff0 0x7febf415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febfc607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7febf415d3f0 con 0x7febf40afee0 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febfc607640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febf4072c10 con 0x7febe00039e0 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febfc607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7febf415d570 con 0x7febf415bff0 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febf9ba2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febf40afee0 0x7febf40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febf9ba2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febf40afee0 0x7febf40b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41892/0 (socket says 172.21.15.40:41892) 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febf9ba2640 1 -- 172.21.15.40:0/711408908 learned_addr learned my addr 172.21.15.40:0/711408908 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febfaba4640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf415bff0 0x7febf415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febfa3a3640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe00039e0 0x7febf4159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febf9ba2640 1 -- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf415bff0 msgr2=0x7febf415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febf9ba2640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf415bff0 0x7febf415c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febf9ba2640 1 -- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe00039e0 msgr2=0x7febf4159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.175+0000 7febf9ba2640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe00039e0 0x7febf4159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.176+0000 7febf9ba2640 1 -- 172.21.15.40:0/711408908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf40b55c0 con 0x7febf40afee0 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.176+0000 7febf9ba2640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febf40afee0 0x7febf40b4d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7febe40119a0 tx=0x7febe4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.176+0000 7febeaffd640 1 -- 172.21.15.40:0/711408908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe400fa30 con 0x7febf40afee0 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.176+0000 7febeaffd640 1 -- 172.21.15.40:0/711408908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe400fbd0 con 0x7febf40afee0 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.176+0000 7febeaffd640 1 -- 172.21.15.40:0/711408908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4014440 con 0x7febf40afee0 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.176+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf40b58a0 con 0x7febf40afee0 2024-04-26T21:35:41.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.176+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf4149150 con 0x7febf40afee0 2024-04-26T21:35:41.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.177+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf40694f0 con 0x7febf40afee0 2024-04-26T21:35:41.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.177+0000 7febeaffd640 1 -- 172.21.15.40:0/711408908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febe4012040 con 0x7febf40afee0 2024-04-26T21:35:41.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.178+0000 7febeaffd640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd806c8f0 0x7febd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.178+0000 7febeaffd640 1 -- 172.21.15.40:0/711408908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7febe408c450 con 0x7febf40afee0 2024-04-26T21:35:41.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.178+0000 7febfa3a3640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd806c8f0 0x7febd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.179+0000 7febfa3a3640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd806c8f0 0x7febd806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7febf0001860 tx=0x7febf0009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.182+0000 7febeaffd640 1 -- 172.21.15.40:0/711408908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7febe4019030 con 0x7febf40afee0 2024-04-26T21:35:41.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.368+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7febf41601c0 con 0x7febf40afee0 2024-04-26T21:35:41.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.369+0000 7febeaffd640 1 -- 172.21.15.40:0/711408908 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7febe405a540 con 0x7febf40afee0 2024-04-26T21:35:41.370 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:41.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd806c8f0 msgr2=0x7febd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd806c8f0 0x7febd806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7febf0001860 tx=0x7febf0009030 comp rx=0 tx=0).stop 2024-04-26T21:35:41.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febf40afee0 msgr2=0x7febf40b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febf40afee0 0x7febf40b4d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7febe40119a0 tx=0x7febe4011da0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 shutdown_connections 2024-04-26T21:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd806c8f0 0x7febd806edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febf415bff0 0x7febf415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febf40afee0 0x7febf40b4d50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 --2- 172.21.15.40:0/711408908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe00039e0 0x7febf4159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 >> 172.21.15.40:0/711408908 conn(0x7febf40d0be0 msgr2=0x7febf415e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 shutdown_connections 2024-04-26T21:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.371+0000 7febfc607640 1 -- 172.21.15.40:0/711408908 wait complete. 2024-04-26T21:35:41.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.486+0000 7f2008a99640 1 Processor -- start 2024-04-26T21:35:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.486+0000 7f2008a99640 1 -- start start 2024-04-26T21:35:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.486+0000 7f2008a99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20040ceb80 0x7f200416a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.486+0000 7f2008a99640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20040d1b80 con 0x7f200415f770 2024-04-26T21:35:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.486+0000 7f2008a99640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f200406e580 con 0x7f20040b1fd0 2024-04-26T21:35:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.486+0000 7f2008a99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f200406f3e0 con 0x7f20040ceb80 2024-04-26T21:35:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f200259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f200415f770 0x7f20040d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38670/0 (socket says 172.21.15.40:38670) 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f200259c640 1 -- 172.21.15.40:0/2841252607 learned_addr learned my addr 172.21.15.40:0/2841252607 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2002d9d640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20040ceb80 0x7f200416a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942131277 0 0) 0x7f20040d1b80 con 0x7f200415f770 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ff0003520 con 0x7f200415f770 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3182536548 0 0) 0x7f1ff0003520 con 0x7f200415f770 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20040ceb80 msgr2=0x7f200416a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20040ceb80 0x7f200416a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 >> v1:172.21.15.184:6789/0 conn(0x7f20040b1fd0 legacy=0x7f20040d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200406e860 con 0x7f200415f770 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1ff4001670 con 0x7f200415f770 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 >> v1:172.21.15.40:6789/0 conn(0x7f200415f770 legacy=0x7f20040d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ff00039e0 0x7f1ff0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff0004390 0x7f1ff0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20040ceb80 0x7f1ff00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f200406f3e0 con 0x7f1ff0004390 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff000b9f0 con 0x7f1ff00039e0 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff000bbf0 con 0x7f20040ceb80 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f200259c640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ff00039e0 0x7f1ff0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.487+0000 7f1ffbfff640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff0004390 0x7f1ff0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.488+0000 7f2002d9d640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20040ceb80 0x7f1ff00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.488+0000 7f1ffbfff640 1 -- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20040ceb80 msgr2=0x7f1ff00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.488+0000 7f1ffbfff640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20040ceb80 0x7f1ff00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.488+0000 7f1ffbfff640 1 -- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ff00039e0 msgr2=0x7f1ff0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.488+0000 7f1ffbfff640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ff00039e0 0x7f1ff0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.488+0000 7f1ffbfff640 1 -- 172.21.15.40:0/2841252607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff0003520 con 0x7f1ff0004390 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.488+0000 7f1ffbfff640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff0004390 0x7f1ff0004860 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001860 tx=0x7f1ff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b464e2fdc5794b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec037070 con 0x7f1ff0004390 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec02abf0 con 0x7f1ff0004390 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2001d9b640 1 -- 172.21.15.40:0/2841252607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec001430 con 0x7f1ff0004390 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2008a99640 1 -- 172.21.15.40:0/2841252607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff0004390 msgr2=0x7f1ff0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2008a99640 1 --2- 172.21.15.40:0/2841252607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff0004390 0x7f1ff0004860 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001860 tx=0x7f1ff0005830 comp rx=0 tx=0).stop 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2002d9d640 1 -- 172.21.15.40:0/2841252607 reap_dead start 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2008a99640 1 -- 172.21.15.40:0/2841252607 shutdown_connections 2024-04-26T21:35:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2008a99640 1 -- 172.21.15.40:0/2841252607 >> 172.21.15.40:0/2841252607 conn(0x7f20040d2e60 msgr2=0x7f20040d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:41.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2008a99640 1 -- 172.21.15.40:0/2841252607 shutdown_connections 2024-04-26T21:35:41.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.489+0000 7f2008a99640 1 -- 172.21.15.40:0/2841252607 wait complete. 2024-04-26T21:35:41.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.491+0000 7f2008a99640 1 Processor -- start 2024-04-26T21:35:41.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.492+0000 7f2008a99640 1 -- start start 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f2008a99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ff0004390 0x7f20040afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f2008a99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20041480e0 0x7f200415af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f2008a99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200415b490 0x7f2004158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f2008a99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20041681f0 con 0x7f20041480e0 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f2008a99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f20040cd6e0 con 0x7f200415b490 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f2008a99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2004168370 con 0x7f1ff0004390 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20041480e0 0x7f200415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20041480e0 0x7f200415af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41920/0 (socket says 172.21.15.40:41920) 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 -- 172.21.15.40:0/4233202680 learned_addr learned my addr 172.21.15.40:0/4233202680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f200259c640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ff0004390 0x7f20040afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f2002d9d640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200415b490 0x7f2004158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 -- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ff0004390 msgr2=0x7f20040afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ff0004390 0x7f20040afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 -- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200415b490 msgr2=0x7f2004158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200415b490 0x7f2004158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 -- 172.21.15.40:0/4233202680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200415d0a0 con 0x7f20041480e0 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f200259c640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ff0004390 0x7f20040afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f2002d9d640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200415b490 0x7f2004158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffbfff640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20041480e0 0x7f200415af50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001980 tx=0x7f1fec001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.493+0000 7f1ffaffd640 1 -- 172.21.15.40:0/4233202680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec03f020 con 0x7f20041480e0 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.494+0000 7f1ffaffd640 1 -- 172.21.15.40:0/4233202680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec005270 con 0x7f20041480e0 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.494+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200415d310 con 0x7f20041480e0 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.494+0000 7f1ffaffd640 1 -- 172.21.15.40:0/4233202680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec037040 con 0x7f20041480e0 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.494+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200415b930 con 0x7f20041480e0 2024-04-26T21:35:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.495+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20040694f0 con 0x7f20041480e0 2024-04-26T21:35:41.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.495+0000 7f1ffaffd640 1 -- 172.21.15.40:0/4233202680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fec042020 con 0x7f20041480e0 2024-04-26T21:35:41.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.496+0000 7f1ffaffd640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1fe406c920 0x7f1fe406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.496+0000 7f1ffaffd640 1 -- 172.21.15.40:0/4233202680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1fec0abb20 con 0x7f20041480e0 2024-04-26T21:35:41.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.496+0000 7f200259c640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1fe406c920 0x7f1fe406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.496+0000 7f200259c640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1fe406c920 0x7f1fe406ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4007cf0 tx=0x7f1ff4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.500+0000 7f1ffaffd640 1 -- 172.21.15.40:0/4233202680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fec049030 con 0x7f20041480e0 2024-04-26T21:35:41.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.684+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f200406fe50 con 0x7f20041480e0 2024-04-26T21:35:41.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.685+0000 7f1ffaffd640 1 -- 172.21.15.40:0/4233202680 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f1fec079ca0 con 0x7f20041480e0 2024-04-26T21:35:41.686 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1fe406c920 msgr2=0x7f1fe406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1fe406c920 0x7f1fe406ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4007cf0 tx=0x7f1ff4009040 comp rx=0 tx=0).stop 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20041480e0 msgr2=0x7f200415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20041480e0 0x7f200415af50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001980 tx=0x7f1fec001230 comp rx=0 tx=0).stop 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 shutdown_connections 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1fe406c920 0x7f1fe406ede0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200415b490 0x7f2004158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20041480e0 0x7f200415af50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 --2- 172.21.15.40:0/4233202680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ff0004390 0x7f20040afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 >> 172.21.15.40:0/4233202680 conn(0x7f20040d2e60 msgr2=0x7f20040d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:41.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 shutdown_connections 2024-04-26T21:35:41.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.687+0000 7f2008a99640 1 -- 172.21.15.40:0/4233202680 wait complete. 2024-04-26T21:35:41.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3fa0859640 1 Processor -- start 2024-04-26T21:35:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3fa0859640 1 -- start start 2024-04-26T21:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3fa0859640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f9c15f170 0x7f3f9c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3fa0859640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f9c15f550 0x7f3f9c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3fa0859640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f9c0d1040 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3fa0859640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9c0c9430 con 0x7f3f9c15f170 2024-04-26T21:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3fa0859640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9c0c9680 con 0x7f3f9c15f550 2024-04-26T21:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3f9a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f9c15f170 0x7f3f9c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3f9ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3f9c0d34b0 0x7f3f9c0cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38684/0 (socket says 172.21.15.40:38684) 2024-04-26T21:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.801+0000 7f3f9ad9d640 1 -- 172.21.15.40:0/2909285857 learned_addr learned my addr 172.21.15.40:0/2909285857 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.802+0000 7f3f93fff640 1 --2- 172.21.15.40:0/2909285857 >> v2:172.21.15.184:3300/0 conn(0x7f3f9c15f550 0x7f3f9c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.802+0000 7f3f9a59c640 1 -- 172.21.15.40:0/2909285857 >> v2:172.21.15.184:3300/0 conn(0x7f3f9c15f550 msgr2=0x7f3f9c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.802+0000 7f3f9a59c640 1 --2- 172.21.15.40:0/2909285857 >> v2:172.21.15.184:3300/0 conn(0x7f3f9c15f550 0x7f3f9c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.802+0000 7f3f9a59c640 1 -- 172.21.15.40:0/2909285857 >> v1:172.21.15.40:6789/0 conn(0x7f3f9c0d34b0 legacy=0x7f3f9c0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.802+0000 7f3f9a59c640 1 -- 172.21.15.40:0/2909285857 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c0c9960 con 0x7f3f9c15f170 2024-04-26T21:35:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.802+0000 7f3f99d9b640 1 -- 172.21.15.40:0/2909285857 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158561644 0 0) 0x7f3f94001490 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.802+0000 7f3f93fff640 1 --2- 172.21.15.40:0/2909285857 >> v2:172.21.15.184:3300/0 conn(0x7f3f9c15f550 0x7f3f9c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.802+0000 7f3f9a59c640 1 --2- 172.21.15.40:0/2909285857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c15f170 0x7f3f9c0d5a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0097a0 tx=0x7f3f8c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b4c40986944ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3f99d9b640 1 -- 172.21.15.40:0/2909285857 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c037070 con 0x7f3f9c15f170 2024-04-26T21:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3f99d9b640 1 -- 172.21.15.40:0/2909285857 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c032440 con 0x7f3f9c15f170 2024-04-26T21:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3f99d9b640 1 -- 172.21.15.40:0/2909285857 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c032810 con 0x7f3f9c15f170 2024-04-26T21:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3fa0859640 1 -- 172.21.15.40:0/2909285857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c15f170 msgr2=0x7f3f9c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3fa0859640 1 --2- 172.21.15.40:0/2909285857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c15f170 0x7f3f9c0d5a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0097a0 tx=0x7f3f8c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3fa0859640 1 -- 172.21.15.40:0/2909285857 shutdown_connections 2024-04-26T21:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3fa0859640 1 --2- 172.21.15.40:0/2909285857 >> v2:172.21.15.184:3300/0 conn(0x7f3f9c15f550 0x7f3f9c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3fa0859640 1 --2- 172.21.15.40:0/2909285857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c15f170 0x7f3f9c0d5a80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3fa0859640 1 -- 172.21.15.40:0/2909285857 >> 172.21.15.40:0/2909285857 conn(0x7f3f9c0d4f60 msgr2=0x7f3f9c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3fa0859640 1 -- 172.21.15.40:0/2909285857 shutdown_connections 2024-04-26T21:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.803+0000 7f3fa0859640 1 -- 172.21.15.40:0/2909285857 wait complete. 2024-04-26T21:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3fa0859640 1 Processor -- start 2024-04-26T21:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3fa0859640 1 -- start start 2024-04-26T21:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3fa0859640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c0d34b0 0x7f3f9c1f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3fa0859640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f9c15f170 0x7f3f9c0cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3fa0859640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f9c0ccef0 0x7f3f9c1520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3fa0859640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c0cb5d0 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3fa0859640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c0cb450 con 0x7f3f9c0ccef0 2024-04-26T21:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3fa0859640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c0cb750 con 0x7f3f9c15f170 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3f9a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c0d34b0 0x7f3f9c1f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3f9a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c0d34b0 0x7f3f9c1f7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41934/0 (socket says 172.21.15.40:41934) 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.805+0000 7f3f9a59c640 1 -- 172.21.15.40:0/3991169 learned_addr learned my addr 172.21.15.40:0/3991169 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f9ad9d640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f9c0ccef0 0x7f3f9c1520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f9a59c640 1 -- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f9c15f170 msgr2=0x7f3f9c0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f93fff640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f9c15f170 0x7f3f9c0cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f9a59c640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f9c15f170 0x7f3f9c0cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f9a59c640 1 -- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f9c0ccef0 msgr2=0x7f3f9c1520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f9a59c640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f9c0ccef0 0x7f3f9c1520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f9a59c640 1 -- 172.21.15.40:0/3991169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c1f8380 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f9ad9d640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f9c0ccef0 0x7f3f9c1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f9a59c640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c0d34b0 0x7f3f9c1f7e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c000c00 tx=0x7f3f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f92ffd640 1 -- 172.21.15.40:0/3991169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c037040 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f92ffd640 1 -- 172.21.15.40:0/3991169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c042070 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9c0cd450 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3f92ffd640 1 -- 172.21.15.40:0/3991169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c030a70 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.806+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9c150220 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.807+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9c150010 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.807+0000 7f3f92ffd640 1 -- 172.21.15.40:0/3991169 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f8c030c10 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.808+0000 7f3f92ffd640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f7c06c8f0 0x7f3f7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.808+0000 7f3f92ffd640 1 -- 172.21.15.40:0/3991169 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3f8c0ac8e0 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.808+0000 7f3f93fff640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f7c06c8f0 0x7f3f7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.809+0000 7f3f93fff640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f7c06c8f0 0x7f3f7c06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0cdbd0 tx=0x7f3f84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:41.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.812+0000 7f3f92ffd640 1 -- 172.21.15.40:0/3991169 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f8c045360 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.995+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3f9c0694f0 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.995+0000 7f3f92ffd640 1 -- 172.21.15.40:0/3991169 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f3f8c07aa60 con 0x7f3f9c0d34b0 2024-04-26T21:35:41.997 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f7c06c8f0 msgr2=0x7f3f7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f7c06c8f0 0x7f3f7c06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0cdbd0 tx=0x7f3f84008040 comp rx=0 tx=0).stop 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c0d34b0 msgr2=0x7f3f9c1f7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c0d34b0 0x7f3f9c1f7e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c000c00 tx=0x7f3f8c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 shutdown_connections 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f7c06c8f0 0x7f3f7c06edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f9c0ccef0 0x7f3f9c1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f9c15f170 0x7f3f9c0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 --2- 172.21.15.40:0/3991169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f9c0d34b0 0x7f3f9c1f7e40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.997+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 >> 172.21.15.40:0/3991169 conn(0x7f3f9c0d4f60 msgr2=0x7f3f9c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:42.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.998+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 shutdown_connections 2024-04-26T21:35:42.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:41.998+0000 7f3fa0859640 1 -- 172.21.15.40:0/3991169 wait complete. 2024-04-26T21:35:42.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:35:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.109+0000 7fbdf6020640 1 Processor -- start 2024-04-26T21:35:42.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.110+0000 7fbdf6020640 1 -- start start 2024-04-26T21:35:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.110+0000 7fbdf6020640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf0163610 0x7fbdf00d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.110+0000 7fbdf6020640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdf00d1a70 con 0x7fbdf015f5a0 2024-04-26T21:35:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.110+0000 7fbdf6020640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdf00d1d50 con 0x7fbdf00d2830 2024-04-26T21:35:42.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.110+0000 7fbdf6020640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf0163d30 con 0x7fbdf0163610 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.110+0000 7fbdef7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbdf015f5a0 0x7fbdf0162f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38686/0 (socket says 172.21.15.40:38686) 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.110+0000 7fbdef7fe640 1 -- 172.21.15.40:0/3485454270 learned_addr learned my addr 172.21.15.40:0/3485454270 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdeeffd640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf0163610 0x7fbdf00d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1617522339 0 0) 0x7fbdf00d1a70 con 0x7fbdf015f5a0 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdd8003520 con 0x7fbdf015f5a0 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3558812517 0 0) 0x7fbdd8003520 con 0x7fbdf015f5a0 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf0163610 msgr2=0x7fbdf00d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf0163610 0x7fbdf00d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 >> v1:172.21.15.184:6789/0 conn(0x7fbdf00d2830 legacy=0x7fbdf00cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf00d2030 con 0x7fbdf015f5a0 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fbde0001670 con 0x7fbdf015f5a0 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 >> v1:172.21.15.40:6789/0 conn(0x7fbdf015f5a0 legacy=0x7fbdf0162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdd8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf0163610 0x7fbdd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd8004d40 0x7fbdd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd800bac0 con 0x7fbdd80039e0 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd800bc70 con 0x7fbdd8004d40 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd800be70 con 0x7fbdf0163610 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdef7fe640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdd8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdeeffd640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf0163610 0x7fbdd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdeffff640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd8004d40 0x7fbdd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdef7fe640 1 -- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf0163610 msgr2=0x7fbdd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdef7fe640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf0163610 0x7fbdd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdef7fe640 1 -- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd8004d40 msgr2=0x7fbdd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdef7fe640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdd8004d40 0x7fbdd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.111+0000 7fbdef7fe640 1 -- 172.21.15.40:0/3485454270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd8003520 con 0x7fbdd80039e0 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.112+0000 7fbdef7fe640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdd8003e00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbde000c240 tx=0x7fbdd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2100f0750584492 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.112+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0007d10 con 0x7fbdd80039e0 2024-04-26T21:35:42.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.112+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde0005ad0 con 0x7fbdd80039e0 2024-04-26T21:35:42.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.112+0000 7fbdee7fc640 1 -- 172.21.15.40:0/3485454270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0007070 con 0x7fbdd80039e0 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.113+0000 7fbdf6020640 1 -- 172.21.15.40:0/3485454270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 msgr2=0x7fbdd8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.113+0000 7fbdf6020640 1 --2- 172.21.15.40:0/3485454270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdd8003e00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbde000c240 tx=0x7fbdd8005830 comp rx=0 tx=0).stop 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.113+0000 7fbdeffff640 1 -- 172.21.15.40:0/3485454270 reap_dead start 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.113+0000 7fbdf6020640 1 -- 172.21.15.40:0/3485454270 shutdown_connections 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.113+0000 7fbdf6020640 1 -- 172.21.15.40:0/3485454270 >> 172.21.15.40:0/3485454270 conn(0x7fbdf00d5190 msgr2=0x7fbdf00d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.113+0000 7fbdf6020640 1 -- 172.21.15.40:0/3485454270 shutdown_connections 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.113+0000 7fbdf6020640 1 -- 172.21.15.40:0/3485454270 wait complete. 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.114+0000 7fbdf6020640 1 Processor -- start 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.114+0000 7fbdf6020640 1 -- start start 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdf6020640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdf0155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdf6020640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf00b80b0 0x7fbdf00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdf6020640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdf00b8530 0x7fbdf00ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdf6020640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf00d3c20 con 0x7fbdd80039e0 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdf6020640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf00d4190 con 0x7fbdf00b8530 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdf6020640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf00d3da0 con 0x7fbdf00b80b0 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdf0155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdf0155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41954/0 (socket says 172.21.15.40:41954) 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdef7fe640 1 -- 172.21.15.40:0/4200579493 learned_addr learned my addr 172.21.15.40:0/4200579493 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdeffff640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdf00b8530 0x7fbdf00ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdeeffd640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf00b80b0 0x7fbdf00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdef7fe640 1 -- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf00b80b0 msgr2=0x7fbdf00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdef7fe640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf00b80b0 0x7fbdf00ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdef7fe640 1 -- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdf00b8530 msgr2=0x7fbdf00ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdef7fe640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdf00b8530 0x7fbdf00ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.115+0000 7fbdef7fe640 1 -- 172.21.15.40:0/4200579493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf00b8a90 con 0x7fbdd80039e0 2024-04-26T21:35:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.116+0000 7fbdef7fe640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdf0155250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbde0001970 tx=0x7fbdf008f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.116+0000 7fbdd7fff640 1 -- 172.21.15.40:0/4200579493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0040020 con 0x7fbdd80039e0 2024-04-26T21:35:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.116+0000 7fbdd7fff640 1 -- 172.21.15.40:0/4200579493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde0001430 con 0x7fbdd80039e0 2024-04-26T21:35:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.116+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf00cc300 con 0x7fbdd80039e0 2024-04-26T21:35:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.116+0000 7fbdd7fff640 1 -- 172.21.15.40:0/4200579493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0037040 con 0x7fbdd80039e0 2024-04-26T21:35:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.116+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf00b8d70 con 0x7fbdd80039e0 2024-04-26T21:35:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.117+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf00694f0 con 0x7fbdd80039e0 2024-04-26T21:35:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.117+0000 7fbdd7fff640 1 -- 172.21.15.40:0/4200579493 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbde0043020 con 0x7fbdd80039e0 2024-04-26T21:35:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.118+0000 7fbdd7fff640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbddc06c8e0 0x7fbddc06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.118+0000 7fbdd7fff640 1 -- 172.21.15.40:0/4200579493 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbde00ab440 con 0x7fbdd80039e0 2024-04-26T21:35:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.118+0000 7fbdeeffd640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbddc06c8e0 0x7fbddc06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.118+0000 7fbdeeffd640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbddc06c8e0 0x7fbddc06eda0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00cf450 tx=0x7fbde4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.122+0000 7fbdd7fff640 1 -- 172.21.15.40:0/4200579493 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbde0079490 con 0x7fbdd80039e0 2024-04-26T21:35:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.352+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdf0155d30 con 0x7fbdd80039e0 2024-04-26T21:35:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.352+0000 7fbdd7fff640 1 -- 172.21.15.40:0/4200579493 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7fbde0078d90 con 0x7fbdd80039e0 2024-04-26T21:35:42.353 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:42.354 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:31.566271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:35:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-26T21:35:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbddc06c8e0 msgr2=0x7fbddc06eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbddc06c8e0 0x7fbddc06eda0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00cf450 tx=0x7fbde4008040 comp rx=0 tx=0).stop 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 msgr2=0x7fbdf0155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdf0155250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbde0001970 tx=0x7fbdf008f2a0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 shutdown_connections 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbddc06c8e0 0x7fbddc06eda0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdf00b8530 0x7fbdf00ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdf00b80b0 0x7fbdf00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 --2- 172.21.15.40:0/4200579493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdd80039e0 0x7fbdf0155250 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 >> 172.21.15.40:0/4200579493 conn(0x7fbdf00d5190 msgr2=0x7fbdf00b5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 shutdown_connections 2024-04-26T21:35:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.354+0000 7fbdf6020640 1 -- 172.21.15.40:0/4200579493 wait complete. 2024-04-26T21:35:42.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:42.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.466+0000 7f964ae93640 1 Processor -- start 2024-04-26T21:35:42.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.466+0000 7f964ae93640 1 -- start start 2024-04-26T21:35:42.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f964ae93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9644162880 0x7f96440d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f964ae93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 0x7f96440d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f964ae93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f96440d39f0 0x7f96440b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f964ae93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f96440d0580 con 0x7f9644162880 2024-04-26T21:35:42.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f964ae93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f96440b7e50 con 0x7f96440d39f0 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f964ae93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96440d1800 con 0x7f96440b5f20 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f9648c2f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9644162880 0x7f96440d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f9648c2f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9644162880 0x7f96440d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41968/0 (socket says 172.21.15.40:41968) 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f9648c2f640 1 -- 172.21.15.40:0/3121089476 learned_addr learned my addr 172.21.15.40:0/3121089476 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f9648c2f640 1 -- 172.21.15.40:0/3121089476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 msgr2=0x7f96440d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f9648c2f640 1 --2- 172.21.15.40:0/3121089476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 0x7f96440d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f9648c2f640 1 -- 172.21.15.40:0/3121089476 >> v2:172.21.15.184:3300/0 conn(0x7f96440d39f0 msgr2=0x7f96440b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f9648c2f640 1 --2- 172.21.15.40:0/3121089476 >> v2:172.21.15.184:3300/0 conn(0x7f96440d39f0 0x7f96440b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.467+0000 7f9648c2f640 1 -- 172.21.15.40:0/3121089476 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440d07d0 con 0x7f9644162880 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.468+0000 7f9648c2f640 1 --2- 172.21.15.40:0/3121089476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96440d2cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f964408f770 tx=0x7f963802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2cbb56e8ce077a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.468+0000 7f96437fe640 1 -- 172.21.15.40:0/3121089476 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638037070 con 0x7f9644162880 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.468+0000 7f96437fe640 1 -- 172.21.15.40:0/3121089476 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9638032440 con 0x7f9644162880 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.468+0000 7f96437fe640 1 -- 172.21.15.40:0/3121089476 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638032810 con 0x7f9644162880 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.468+0000 7f964ae93640 1 -- 172.21.15.40:0/3121089476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 msgr2=0x7f96440d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.468+0000 7f964ae93640 1 --2- 172.21.15.40:0/3121089476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96440d2cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f964408f770 tx=0x7f963802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.469+0000 7f964ae93640 1 -- 172.21.15.40:0/3121089476 shutdown_connections 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.469+0000 7f964ae93640 1 --2- 172.21.15.40:0/3121089476 >> v2:172.21.15.184:3300/0 conn(0x7f96440d39f0 0x7f96440b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.469+0000 7f964ae93640 1 --2- 172.21.15.40:0/3121089476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 0x7f96440d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.469+0000 7f964ae93640 1 --2- 172.21.15.40:0/3121089476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96440d2cb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.469+0000 7f964ae93640 1 -- 172.21.15.40:0/3121089476 >> 172.21.15.40:0/3121089476 conn(0x7f96440d4f60 msgr2=0x7f96440d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.469+0000 7f964ae93640 1 -- 172.21.15.40:0/3121089476 shutdown_connections 2024-04-26T21:35:42.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.469+0000 7f964ae93640 1 -- 172.21.15.40:0/3121089476 wait complete. 2024-04-26T21:35:42.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.470+0000 7f964ae93640 1 Processor -- start 2024-04-26T21:35:42.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.470+0000 7f964ae93640 1 -- start start 2024-04-26T21:35:42.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f964ae93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 0x7f96440b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f964ae93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96440d39f0 0x7f96441f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f964ae93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96441615d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f964ae93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9644162c60 con 0x7f9644162880 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f964ae93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96440d0d90 con 0x7f96440d39f0 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f964ae93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9644162de0 con 0x7f96440b5f20 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96441615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96441615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41974/0 (socket says 172.21.15.40:41974) 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 -- 172.21.15.40:0/1816108584 learned_addr learned my addr 172.21.15.40:0/1816108584 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9643fff640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96440d39f0 0x7f96441f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9648c2f640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 0x7f96440b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 -- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 msgr2=0x7f96440b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 0x7f96440b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 -- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96440d39f0 msgr2=0x7f96441f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96440d39f0 0x7f96441f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 -- 172.21.15.40:0/1816108584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441f6190 con 0x7f9644162880 2024-04-26T21:35:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f9649430640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96441615d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f963c00b970 tx=0x7f963c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f96417fa640 1 -- 172.21.15.40:0/1816108584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c019070 con 0x7f9644162880 2024-04-26T21:35:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f96417fa640 1 -- 172.21.15.40:0/1816108584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963c00cb60 con 0x7f9644162880 2024-04-26T21:35:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f96417fa640 1 -- 172.21.15.40:0/1816108584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c014440 con 0x7f9644162880 2024-04-26T21:35:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.471+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441f6470 con 0x7f9644162880 2024-04-26T21:35:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.472+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441f6a30 con 0x7f9644162880 2024-04-26T21:35:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.472+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96440694f0 con 0x7f9644162880 2024-04-26T21:35:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.473+0000 7f96417fa640 1 -- 172.21.15.40:0/1816108584 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f963c01c070 con 0x7f9644162880 2024-04-26T21:35:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.473+0000 7f96417fa640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962806c920 0x7f962806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.473+0000 7f96417fa640 1 -- 172.21.15.40:0/1816108584 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f963c08bb20 con 0x7f9644162880 2024-04-26T21:35:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.474+0000 7f9648c2f640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962806c920 0x7f962806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.474+0000 7f9648c2f640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962806c920 0x7f962806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9638034000 tx=0x7f963802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.478+0000 7f96417fa640 1 -- 172.21.15.40:0/1816108584 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f963c0595b0 con 0x7f9644162880 2024-04-26T21:35:42.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.663+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9644069c20 con 0x7f9644162880 2024-04-26T21:35:42.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.664+0000 7f96417fa640 1 -- 172.21.15.40:0/1816108584 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f963c05d8b0 con 0x7f9644162880 2024-04-26T21:35:42.665 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:42.665 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:14.527620+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962806c920 msgr2=0x7f962806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962806c920 0x7f962806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9638034000 tx=0x7f963802e000 comp rx=0 tx=0).stop 2024-04-26T21:35:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 msgr2=0x7f96441615d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96441615d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f963c00b970 tx=0x7f963c00be30 comp rx=0 tx=0).stop 2024-04-26T21:35:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 shutdown_connections 2024-04-26T21:35:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962806c920 0x7f962806ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644162880 0x7f96441615d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96440d39f0 0x7f96441f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 --2- 172.21.15.40:0/1816108584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96440b5f20 0x7f96440b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 >> 172.21.15.40:0/1816108584 conn(0x7f96440d4f60 msgr2=0x7f9644160a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 shutdown_connections 2024-04-26T21:35:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.666+0000 7f964ae93640 1 -- 172.21.15.40:0/1816108584 wait complete. 2024-04-26T21:35:42.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:35:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.777+0000 7fd71de37640 1 Processor -- start 2024-04-26T21:35:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.777+0000 7fd71de37640 1 -- start start 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.778+0000 7fd71de37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180afda0 0x7fd718169c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.778+0000 7fd71de37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd718072fb0 con 0x7fd7180b0b70 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.778+0000 7fd71de37640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd718073290 con 0x7fd7180b4d40 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.778+0000 7fd71de37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd718073bc0 con 0x7fd7180afda0 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.778+0000 7fd7177fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd7180b0b70 0x7fd7180b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38700/0 (socket says 172.21.15.40:38700) 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.778+0000 7fd7177fe640 1 -- 172.21.15.40:0/2640301646 learned_addr learned my addr 172.21.15.40:0/2640301646 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.779+0000 7fd717fff640 1 --2- 172.21.15.40:0/2640301646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180afda0 0x7fd718169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.779+0000 7fd7167fc640 1 -- 172.21.15.40:0/2640301646 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3419677466 0 0) 0x7fd718072fb0 con 0x7fd7180b0b70 2024-04-26T21:35:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.779+0000 7fd7167fc640 1 -- 172.21.15.40:0/2640301646 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd700003520 con 0x7fd7180b0b70 2024-04-26T21:35:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.779+0000 7fd7167fc640 1 -- 172.21.15.40:0/2640301646 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541826223 0 0) 0x7fd718073290 con 0x7fd7180b4d40 2024-04-26T21:35:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.779+0000 7fd7167fc640 1 -- 172.21.15.40:0/2640301646 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd718072fb0 con 0x7fd7180b4d40 2024-04-26T21:35:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.779+0000 7fd717fff640 1 -- 172.21.15.40:0/2640301646 >> v1:172.21.15.184:6789/0 conn(0x7fd7180b4d40 legacy=0x7fd7180b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.779+0000 7fd717fff640 1 -- 172.21.15.40:0/2640301646 >> v1:172.21.15.40:6789/0 conn(0x7fd7180b0b70 legacy=0x7fd7180b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.779+0000 7fd717fff640 1 -- 172.21.15.40:0/2640301646 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd718073570 con 0x7fd7180afda0 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.780+0000 7fd717fff640 1 --2- 172.21.15.40:0/2640301646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180afda0 0x7fd718169c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd710009dc0 tx=0x7fd71002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=180ac7fcf774a072 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.780+0000 7fd7167fc640 1 -- 172.21.15.40:0/2640301646 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710007740 con 0x7fd7180afda0 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.780+0000 7fd7167fc640 1 -- 172.21.15.40:0/2640301646 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7100078e0 con 0x7fd7180afda0 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.780+0000 7fd7167fc640 1 -- 172.21.15.40:0/2640301646 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710007c90 con 0x7fd7180afda0 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.781+0000 7fd71de37640 1 -- 172.21.15.40:0/2640301646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180afda0 msgr2=0x7fd718169c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.781+0000 7fd71de37640 1 --2- 172.21.15.40:0/2640301646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180afda0 0x7fd718169c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd710009dc0 tx=0x7fd71002a6d0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.781+0000 7fd71de37640 1 -- 172.21.15.40:0/2640301646 shutdown_connections 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.781+0000 7fd71de37640 1 --2- 172.21.15.40:0/2640301646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180afda0 0x7fd718169c90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.781+0000 7fd71de37640 1 -- 172.21.15.40:0/2640301646 >> 172.21.15.40:0/2640301646 conn(0x7fd7180b39b0 msgr2=0x7fd7180b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:42.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.781+0000 7fd71de37640 1 -- 172.21.15.40:0/2640301646 shutdown_connections 2024-04-26T21:35:42.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.781+0000 7fd71de37640 1 -- 172.21.15.40:0/2640301646 wait complete. 2024-04-26T21:35:42.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.782+0000 7fd71de37640 1 Processor -- start 2024-04-26T21:35:42.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.782+0000 7fd71de37640 1 -- start start 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd71de37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7180afda0 0x7fd718156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd71de37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180b0b70 0x7fd7181f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd71de37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7180b4d40 0x7fd7181457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd71de37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd718160360 con 0x7fd7180afda0 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd71de37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd7180b6e20 con 0x7fd7180b4d40 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd71de37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd7181604e0 con 0x7fd7180b0b70 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7180afda0 0x7fd718156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7180afda0 0x7fd718156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41978/0 (socket says 172.21.15.40:41978) 2024-04-26T21:35:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 -- 172.21.15.40:0/3020856492 learned_addr learned my addr 172.21.15.40:0/3020856492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd716ffd640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180b0b70 0x7fd7181f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd717fff640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7180b4d40 0x7fd7181457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 -- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180b0b70 msgr2=0x7fd7181f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180b0b70 0x7fd7181f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 -- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7180b4d40 msgr2=0x7fd7181457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7180b4d40 0x7fd7181457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 -- 172.21.15.40:0/3020856492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd718155b20 con 0x7fd7180afda0 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd716ffd640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180b0b70 0x7fd7181f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.783+0000 7fd7177fe640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7180afda0 0x7fd718156a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd70800c500 tx=0x7fd70800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.784+0000 7fd6fffff640 1 -- 172.21.15.40:0/3020856492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708017070 con 0x7fd7180afda0 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.784+0000 7fd6fffff640 1 -- 172.21.15.40:0/3020856492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd708005ce0 con 0x7fd7180afda0 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.784+0000 7fd6fffff640 1 -- 172.21.15.40:0/3020856492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7080072f0 con 0x7fd7180afda0 2024-04-26T21:35:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.784+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd718143720 con 0x7fd7180afda0 2024-04-26T21:35:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.784+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd718143ce0 con 0x7fd7180afda0 2024-04-26T21:35:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.785+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7180694f0 con 0x7fd7180afda0 2024-04-26T21:35:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.785+0000 7fd6fffff640 1 -- 172.21.15.40:0/3020856492 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd70801a070 con 0x7fd7180afda0 2024-04-26T21:35:42.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.786+0000 7fd6fffff640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd70406c8f0 0x7fd70406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:42.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.786+0000 7fd6fffff640 1 -- 172.21.15.40:0/3020856492 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd70808b160 con 0x7fd7180afda0 2024-04-26T21:35:42.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.786+0000 7fd716ffd640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd70406c8f0 0x7fd70406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:42.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.786+0000 7fd716ffd640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd70406c8f0 0x7fd70406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7181f85c0 tx=0x7fd70c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:42.790+0000 7fd6fffff640 1 -- 172.21.15.40:0/3020856492 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd708058c90 con 0x7fd7180afda0 2024-04-26T21:35:43.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.019+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd718161a70 con 0x7fd7180afda0 2024-04-26T21:35:43.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.020+0000 7fd6fffff640 1 -- 172.21.15.40:0/3020856492 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7fd70805cf90 con 0x7fd7180afda0 2024-04-26T21:35:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:43.021 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:31.566271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:35:43.021 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-26T21:35:43.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd70406c8f0 msgr2=0x7fd70406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd70406c8f0 0x7fd70406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7181f85c0 tx=0x7fd70c009040 comp rx=0 tx=0).stop 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7180afda0 msgr2=0x7fd718156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7180afda0 0x7fd718156a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd70800c500 tx=0x7fd70800c9c0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 shutdown_connections 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd70406c8f0 0x7fd70406edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7180b4d40 0x7fd7181457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7180b0b70 0x7fd7181f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 --2- 172.21.15.40:0/3020856492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7180afda0 0x7fd718156a10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 >> 172.21.15.40:0/3020856492 conn(0x7fd7180b39b0 msgr2=0x7fd7180b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 shutdown_connections 2024-04-26T21:35:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.022+0000 7fd71de37640 1 -- 172.21.15.40:0/3020856492 wait complete. 2024-04-26T21:35:43.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:35:09.479293+0000', 'modified': '2024-04-26T21:35:31.566271+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6834/2862109951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 2862109951}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 2862109951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4293, 'qdb_cluster': [4293]} 2024-04-26T21:35:43.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:35:43.437 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:35:43.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.542+0000 7f3f7d36a640 1 Processor -- start 2024-04-26T21:35:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.542+0000 7f3f7d36a640 1 -- start start 2024-04-26T21:35:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.543+0000 7f3f7d36a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f78162850 0x7f3f78070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.543+0000 7f3f7d36a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f781608c0 0x7f3f78070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.543+0000 7f3f7d36a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f7815fc70 con 0x7f3f78162470 2024-04-26T21:35:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.543+0000 7f3f7d36a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3f7806cc80 con 0x7f3f781608c0 2024-04-26T21:35:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.543+0000 7f3f7d36a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3f7815ff50 con 0x7f3f78162850 2024-04-26T21:35:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.543+0000 7f3f76ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3f78162470 0x7f3f7806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38708/0 (socket says 172.21.15.40:38708) 2024-04-26T21:35:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.543+0000 7f3f76ffd640 1 -- 172.21.15.40:0/2537785189 learned_addr learned my addr 172.21.15.40:0/2537785189 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.543+0000 7f3f777fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f781608c0 0x7f3f78070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.544+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2537785189 >> v2:172.21.15.184:3300/0 conn(0x7f3f78162850 0x7f3f78070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.544+0000 7f3f777fe640 1 -- 172.21.15.40:0/2537785189 >> v2:172.21.15.184:3300/0 conn(0x7f3f78162850 msgr2=0x7f3f78070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.544+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2537785189 >> v2:172.21.15.184:3300/0 conn(0x7f3f78162850 0x7f3f78070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.544+0000 7f3f777fe640 1 -- 172.21.15.40:0/2537785189 >> v1:172.21.15.40:6789/0 conn(0x7f3f78162470 legacy=0x7f3f7806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.544+0000 7f3f75ffb640 1 -- 172.21.15.40:0/2537785189 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2153498768 0 0) 0x7f3f7815fc70 con 0x7f3f78162470 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.544+0000 7f3f777fe640 1 -- 172.21.15.40:0/2537785189 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f78160230 con 0x7f3f781608c0 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.544+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2537785189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f781608c0 0x7f3f78070aa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3f780b48b0 tx=0x7f3f7002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808fddce4818d6f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f75ffb640 1 -- 172.21.15.40:0/2537785189 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007740 con 0x7f3f781608c0 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f75ffb640 1 -- 172.21.15.40:0/2537785189 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f700078e0 con 0x7f3f781608c0 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f75ffb640 1 -- 172.21.15.40:0/2537785189 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007cb0 con 0x7f3f781608c0 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2537785189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f781608c0 msgr2=0x7f3f78070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2537785189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f781608c0 0x7f3f78070aa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3f780b48b0 tx=0x7f3f7002a680 comp rx=0 tx=0).stop 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2537785189 shutdown_connections 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2537785189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f781608c0 0x7f3f78070aa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2537785189 >> v2:172.21.15.184:3300/0 conn(0x7f3f78162850 0x7f3f78070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2537785189 >> 172.21.15.40:0/2537785189 conn(0x7f3f780b3940 msgr2=0x7f3f780b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2537785189 shutdown_connections 2024-04-26T21:35:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.545+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2537785189 wait complete. 2024-04-26T21:35:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.546+0000 7f3f7d36a640 1 Processor -- start 2024-04-26T21:35:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f7d36a640 1 -- start start 2024-04-26T21:35:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f7d36a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f781608c0 0x7f3f7815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f7d36a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f78162470 0x7f3f7814c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f7d36a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f78162850 0x7f3f78157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f7d36a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7815f880 con 0x7f3f78162850 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f7d36a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7815f700 con 0x7f3f78162470 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f7d36a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7815fa00 con 0x7f3f781608c0 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f78162850 0x7f3f78157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f78162850 0x7f3f78157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41988/0 (socket says 172.21.15.40:41988) 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.547+0000 7f3f777fe640 1 -- 172.21.15.40:0/2328881059 learned_addr learned my addr 172.21.15.40:0/2328881059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f78162470 0x7f3f7814c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f781608c0 0x7f3f7815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f777fe640 1 -- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f781608c0 msgr2=0x7f3f7815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f781608c0 0x7f3f7815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f777fe640 1 -- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f78162470 msgr2=0x7f3f7814c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f78162470 0x7f3f7814c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f777fe640 1 -- 172.21.15.40:0/2328881059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7815b730 con 0x7f3f78162850 2024-04-26T21:35:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f781608c0 0x7f3f7815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f78162470 0x7f3f7814c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f78162850 0x7f3f78157030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3f7002a6c0 tx=0x7f3f70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f5f7fe640 1 -- 172.21.15.40:0/2328881059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70040070 con 0x7f3f78162850 2024-04-26T21:35:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f5f7fe640 1 -- 172.21.15.40:0/2328881059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f70031bc0 con 0x7f3f78162850 2024-04-26T21:35:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f5f7fe640 1 -- 172.21.15.40:0/2328881059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70009450 con 0x7f3f78162850 2024-04-26T21:35:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2328881059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7815ba10 con 0x7f3f78162850 2024-04-26T21:35:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.548+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2328881059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f78156790 con 0x7f3f78162850 2024-04-26T21:35:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.550+0000 7f3f5f7fe640 1 -- 172.21.15.40:0/2328881059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f70039040 con 0x7f3f78162850 2024-04-26T21:35:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.550+0000 7f3f5f7fe640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f6406c8f0 0x7f3f6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.550+0000 7f3f5f7fe640 1 -- 172.21.15.40:0/2328881059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3f700acdb0 con 0x7f3f78162850 2024-04-26T21:35:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.551+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f6406c8f0 0x7f3f6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.551+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f6406c8f0 0x7f3f6406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f68000f60 tx=0x7f3f68007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.551+0000 7f3f80083740 1 Processor -- start 2024-04-26T21:35:43.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.551+0000 7f3f80083740 1 -- start start 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f80083740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2ede0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f80083740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 0x55bb2edfe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f80083740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edb6120 0x55bb2edfc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f80083740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ed1a740 con 0x55bb2edb6120 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f80083740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ecb63a0 con 0x55bb2edfdce0 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f80083740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bb2ed16700 con 0x55bb2ede0440 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edb6120 0x55bb2edfc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edb6120 0x55bb2edfc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41996/0 (socket says 172.21.15.40:41996) 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f777fe640 1 -- 172.21.15.40:0/1498635704 learned_addr learned my addr 172.21.15.40:0/1498635704 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2ede0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.552+0000 7f3f767fc640 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 0x55bb2edfe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f777fe640 1 -- 172.21.15.40:0/1498635704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 msgr2=0x55bb2ede0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f777fe640 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2ede0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f777fe640 1 -- 172.21.15.40:0/1498635704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 msgr2=0x55bb2edfe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f777fe640 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 0x55bb2edfe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f777fe640 1 -- 172.21.15.40:0/1498635704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb2ecd2750 con 0x55bb2edb6120 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f777fe640 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edb6120 0x55bb2edfc3a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f70082ee0 tx=0x7f3f7007a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9beedd474e49f8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f5cff9640 1 -- 172.21.15.40:0/1498635704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70049070 con 0x55bb2edb6120 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f5cff9640 1 -- 172.21.15.40:0/1498635704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f70081070 con 0x55bb2edb6120 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f5cff9640 1 -- 172.21.15.40:0/1498635704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70076ea0 con 0x55bb2edb6120 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f80083740 1 -- 172.21.15.40:0/1498635704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edb6120 msgr2=0x55bb2edfc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.553+0000 7f3f80083740 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edb6120 0x55bb2edfc3a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f70082ee0 tx=0x7f3f7007a4a0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 -- 172.21.15.40:0/1498635704 shutdown_connections 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edb6120 0x55bb2edfc3a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 0x55bb2edfe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 --2- 172.21.15.40:0/1498635704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2ede0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 -- 172.21.15.40:0/1498635704 >> 172.21.15.40:0/1498635704 conn(0x55bb2edb2a50 msgr2=0x55bb2edfb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:43.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 -- 172.21.15.40:0/1498635704 shutdown_connections 2024-04-26T21:35:43.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 -- 172.21.15.40:0/1498635704 wait complete. 2024-04-26T21:35:43.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 -- start start 2024-04-26T21:35:43.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.554+0000 7f3f80083740 1 Processor -- start 2024-04-26T21:35:43.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f80083740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2edda180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f80083740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 0x55bb2edf6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f80083740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2edf7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f80083740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ed1a740 con 0x55bb2edddde0 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f80083740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bb2eca1fe0 con 0x55bb2edfdce0 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f80083740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bb2edb6ec0 con 0x55bb2ede0440 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f80083740 10 client.0 _ll_register_callbacks cb 0x55bb2ecb8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2edf7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2edf7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42012/0 (socket says 172.21.15.40:42012) 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f777fe640 1 -- 172.21.15.40:0/3105083120 learned_addr learned my addr 172.21.15.40:0/3105083120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2edda180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f767fc640 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 0x55bb2edf6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f777fe640 1 -- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 msgr2=0x55bb2edda180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f777fe640 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2edda180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f777fe640 1 -- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 msgr2=0x55bb2edf6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f777fe640 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 0x55bb2edf6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f777fe640 1 -- 172.21.15.40:0/3105083120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2ea2c2d0 con 0x55bb2edddde0 2024-04-26T21:35:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.555+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2edda180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f777fe640 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2edf7310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ab080 tx=0x7f3f70009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55ab95459271c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f55ffb640 1 -- 172.21.15.40:0/3105083120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70099060 con 0x55bb2edddde0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f55ffb640 1 -- 172.21.15.40:0/3105083120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3f70082f80 con 0x55bb2edddde0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 1 -- 172.21.15.40:0/3105083120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f70081070 con 0x55bb2edddde0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f55ffb640 1 -- 172.21.15.40:0/3105083120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f3f70076ad0 con 0x55bb2edddde0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 10 client.4872 fetch_fsmap learned FSMap version 8 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 1 -- 172.21.15.40:0/3105083120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55bb2eaf66b0 con 0x55bb2edddde0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f55ffb640 1 -- 172.21.15.40:0/3105083120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f70081070 con 0x55bb2edddde0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f55ffb640 1 -- 172.21.15.40:0/3105083120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f3f7004c060 con 0x55bb2edddde0 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 10 client.4872 fetch_fsmap finished waiting for FSMap version 8 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 10 client.4872 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 20 client.4872 populate_metadata read hostname 'smithi040' 2024-04-26T21:35:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55bb2edf3230 0x55bb2ee0e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 4 client.4872 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f80083740 1 -- 172.21.15.40:0/3105083120 --> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb2ed28350 con 0x55bb2edf3230 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.556+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55bb2edf3230 0x55bb2ee0e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f55ffb640 10 client.4872 ms_handle_connect on v2:172.21.15.40:6832/2862109951 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55bb2edf3230 0x55bb2ee0e880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47de4b1806b04d41 server_cookie=d9630df106c7ca23 in_seq=0 out_seq=0 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f55ffb640 1 -- 172.21.15.40:0/3105083120 <== mds.0 v2:172.21.15.40:6832/2862109951 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb2ed28350 con 0x55bb2edf3230 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f55ffb640 10 client.4872 handle_command_reply: tid=0 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 client.4872 shutdown 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 -- 172.21.15.40:0/3105083120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 msgr2=0x55bb2edf7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2edf7310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ab080 tx=0x7f3f70009750 comp rx=0 tx=0).stop 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 -- 172.21.15.40:0/3105083120 shutdown_connections 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55bb2edf3230 0x55bb2ee0e880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2edf7310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edfdce0 0x55bb2edf6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 --2- 172.21.15.40:0/3105083120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2ede0440 0x55bb2edda180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f80083740 1 -- 172.21.15.40:0/3105083120 >> 172.21.15.40:0/3105083120 conn(0x55bb2edb6120 msgr2=0x55bb2edff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.557+0000 7f3f55ffb640 0 client.4872 ms_handle_reset on v2:172.21.15.40:6832/2862109951 2024-04-26T21:35:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.558+0000 7f3f80083740 1 -- 172.21.15.40:0/3105083120 shutdown_connections 2024-04-26T21:35:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.558+0000 7f3f80083740 1 -- 172.21.15.40:0/3105083120 wait complete. 2024-04-26T21:35:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.558+0000 7f3f80083740 20 client.4872 trim_cache size 0 max 16384 2024-04-26T21:35:43.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.576+0000 7f3f80083740 1 Processor -- start 2024-04-26T21:35:43.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.576+0000 7f3f80083740 1 -- start start 2024-04-26T21:35:43.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f80083740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edddde0 0x55bb2edf4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f80083740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2ede0440 0x55bb2edac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f80083740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee0a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f80083740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f78201a10 con 0x55bb2ede0440 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f80083740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f780b67a0 con 0x55bb2edddde0 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f80083740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f782013c0 con 0x55bb2edfdce0 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2ede0440 0x55bb2edac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2ede0440 0x55bb2edac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42026/0 (socket says 172.21.15.40:42026) 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 -- 172.21.15.40:0/2711617199 learned_addr learned my addr 172.21.15.40:0/2711617199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee0a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edddde0 0x55bb2edf4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 -- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 msgr2=0x55bb2ee0a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee0a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 -- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edddde0 msgr2=0x55bb2edf4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edddde0 0x55bb2edf4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 -- 172.21.15.40:0/2711617199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb2edb0d40 con 0x55bb2ede0440 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee0a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edddde0 0x55bb2edf4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2ede0440 0x55bb2edac010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3f7806d320 tx=0x7f3f6c00bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a52e75288567200 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f57fff640 1 -- 172.21.15.40:0/2711617199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c01a070 con 0x55bb2ede0440 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f57fff640 1 -- 172.21.15.40:0/2711617199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c014970 con 0x55bb2ede0440 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.577+0000 7f3f57fff640 1 -- 172.21.15.40:0/2711617199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c014d20 con 0x55bb2ede0440 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 -- 172.21.15.40:0/2711617199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2ede0440 msgr2=0x55bb2edac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2ede0440 0x55bb2edac010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3f7806d320 tx=0x7f3f6c00bd90 comp rx=0 tx=0).stop 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 -- 172.21.15.40:0/2711617199 shutdown_connections 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee0a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2ede0440 0x55bb2edac010 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 --2- 172.21.15.40:0/2711617199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2edddde0 0x55bb2edf4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 -- 172.21.15.40:0/2711617199 >> 172.21.15.40:0/2711617199 conn(0x55bb2edb6120 msgr2=0x55bb2edaf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 -- 172.21.15.40:0/2711617199 shutdown_connections 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 -- 172.21.15.40:0/2711617199 wait complete. 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 -- start start 2024-04-26T21:35:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.578+0000 7f3f80083740 1 Processor -- start 2024-04-26T21:35:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f80083740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2ee76850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f80083740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2ede0440 0x55bb2ee76d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f80083740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee772d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f80083740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ed1a740 con 0x55bb2edddde0 2024-04-26T21:35:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f80083740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bb2edb6ec0 con 0x55bb2ede0440 2024-04-26T21:35:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f80083740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bb2eca1fe0 con 0x55bb2edfdce0 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f80083740 10 client.0 _ll_register_callbacks cb 0x7f3f6c00b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f76ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2ee76850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f76ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2ee76850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42036/0 (socket says 172.21.15.40:42036) 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f76ffd640 1 -- 172.21.15.40:0/2072364323 learned_addr learned my addr 172.21.15.40:0/2072364323 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2ede0440 0x55bb2ee76d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.579+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f76ffd640 1 -- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 msgr2=0x55bb2ee772d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee772d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f76ffd640 1 -- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2ede0440 msgr2=0x55bb2ee76d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2ede0440 0x55bb2ee76d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f76ffd640 1 -- 172.21.15.40:0/2072364323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2edfe340 con 0x55bb2edddde0 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f777fe640 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f76ffd640 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2ee76850 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55bb2edf2b40 tx=0x7f3f68011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3e618ec1f51f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f55ffb640 1 -- 172.21.15.40:0/2072364323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680270e0 con 0x55bb2edddde0 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f55ffb640 1 -- 172.21.15.40:0/2072364323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3f68010040 con 0x55bb2edddde0 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 1 -- 172.21.15.40:0/2072364323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f6c014970 con 0x55bb2edddde0 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f55ffb640 1 -- 172.21.15.40:0/2072364323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f3f68027280 con 0x55bb2edddde0 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 10 client.4878 fetch_fsmap learned FSMap version 8 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 1 -- 172.21.15.40:0/2072364323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55bb2ede47a0 con 0x55bb2edddde0 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f55ffb640 1 -- 172.21.15.40:0/2072364323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c014970 con 0x55bb2edddde0 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f55ffb640 1 -- 172.21.15.40:0/2072364323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f3f68027420 con 0x55bb2edddde0 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 10 client.4878 fetch_fsmap finished waiting for FSMap version 8 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 10 client.4878 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 20 client.4878 populate_metadata read hostname 'smithi040' 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55bb2ede3b60 0x55bb2edde430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 4 client.4878 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.580+0000 7f3f80083740 1 -- 172.21.15.40:0/2072364323 --> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] -- command(tid 0: {"prefix": "status"}) -- 0x55bb2ed221c0 con 0x55bb2ede3b60 2024-04-26T21:35:43.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55bb2ede3b60 0x55bb2edde430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f55ffb640 10 client.4878 ms_handle_connect on v2:172.21.15.40:6832/2862109951 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f767fc640 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55bb2ede3b60 0x55bb2edde430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f76f2521662b5b0 server_cookie=6b2effe8088b6740 in_seq=0 out_seq=0 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f55ffb640 1 -- 172.21.15.40:0/2072364323 <== mds.0 v2:172.21.15.40:6832/2862109951 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55bb2ed221c0 con 0x55bb2ede3b60 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f55ffb640 10 client.4878 handle_command_reply: tid=0 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 client.4878 shutdown 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 -- 172.21.15.40:0/2072364323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 msgr2=0x55bb2ee76850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2ee76850 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55bb2edf2b40 tx=0x7f3f68011000 comp rx=0 tx=0).stop 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 -- 172.21.15.40:0/2072364323 shutdown_connections 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.40:6832/2862109951,v1:172.21.15.40:6834/2862109951] conn(0x55bb2ede3b60 0x55bb2edde430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb2edfdce0 0x55bb2ee772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb2ede0440 0x55bb2ee76d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f55ffb640 0 client.4878 ms_handle_reset on v2:172.21.15.40:6832/2862109951 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 --2- 172.21.15.40:0/2072364323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb2edddde0 0x55bb2ee76850 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 -- 172.21.15.40:0/2072364323 >> 172.21.15.40:0/2072364323 conn(0x55bb2edb6120 msgr2=0x55bb2ee0dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:43.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 -- 172.21.15.40:0/2072364323 shutdown_connections 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 1 -- 172.21.15.40:0/2072364323 wait complete. 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.581+0000 7f3f80083740 20 client.4878 trim_cache size 0 max 16384 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4293, 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 33.466495371000001, 2024-04-26T21:35:43.589 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 8, 2024-04-26T21:35:43.590 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 24, 2024-04-26T21:35:43.590 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 24, 2024-04-26T21:35:43.590 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 40.027265487999998 2024-04-26T21:35:43.590 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f6406c8f0 msgr2=0x7f3f6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f6406c8f0 0x7f3f6406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f68000f60 tx=0x7f3f68007c10 comp rx=0 tx=0).stop 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2328881059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f78162850 msgr2=0x7f3f78157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f78162850 0x7f3f78157030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3f7002a6c0 tx=0x7f3f70004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2328881059 shutdown_connections 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f6406c8f0 0x7f3f6406edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f78162850 0x7f3f78157030 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f78162470 0x7f3f7814c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 --2- 172.21.15.40:0/2328881059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f781608c0 0x7f3f7815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2328881059 >> 172.21.15.40:0/2328881059 conn(0x7f3f780b3940 msgr2=0x7f3f780b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2328881059 shutdown_connections 2024-04-26T21:35:43.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:43.582+0000 7f3f7d36a640 1 -- 172.21.15.40:0/2328881059 wait complete. 2024-04-26T21:35:43.594 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:35:43.602 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:35:43.602 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:35:43.603 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:35:43.603 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T21:35:43.657 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T21:35:43.657 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T21:35:43.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:43.658 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:35:43.673 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T21:35:43.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:43.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:35:43.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:43.738 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-26T21:35:43.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:43.842 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:35:43.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:43.857 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-26T21:35:43.923 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-04-26T21:35:43.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:35:43.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:43.925 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:35:43.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:43.943 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-04-26T21:35:44.014 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-04-26T21:35:44.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:35:44.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:44.016 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:35:44.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:44.074 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-04-26T21:35:44.143 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-04-26T21:35:44.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:35:44.145 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-26T21:35:44.145 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:35:44.145 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:35:44.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:35:44.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.248+0000 7f83db26b640 1 Processor -- start 2024-04-26T21:35:44.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.248+0000 7f83db26b640 1 -- start start 2024-04-26T21:35:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83db26b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83d40b5f20 0x7f83d40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83db26b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d40b7e50 con 0x7f83d40d3bc0 2024-04-26T21:35:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83db26b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d40d1800 con 0x7f83d4162880 2024-04-26T21:35:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83db26b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f83d40d0580 con 0x7f83d40b5f20 2024-04-26T21:35:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83d8806640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f83d40d3bc0 0x7f83d40b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38722/0 (socket says 172.21.15.40:38722) 2024-04-26T21:35:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83d9808640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83d40b5f20 0x7f83d40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83d8806640 1 -- 172.21.15.40:0/4083626544 learned_addr learned my addr 172.21.15.40:0/4083626544 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:44.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83d3fff640 1 -- 172.21.15.40:0/4083626544 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722468475 0 0) 0x7f83d40b7e50 con 0x7f83d40d3bc0 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83d3fff640 1 -- 172.21.15.40:0/4083626544 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83bc003520 con 0x7f83d40d3bc0 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.249+0000 7f83d9808640 1 -- 172.21.15.40:0/4083626544 >> v1:172.21.15.184:6789/0 conn(0x7f83d4162880 legacy=0x7f83d40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.250+0000 7f83d9808640 1 -- 172.21.15.40:0/4083626544 >> v1:172.21.15.40:6789/0 conn(0x7f83d40d3bc0 legacy=0x7f83d40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.250+0000 7f83d9808640 1 -- 172.21.15.40:0/4083626544 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40d07d0 con 0x7f83d40b5f20 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.250+0000 7f83d9808640 1 --2- 172.21.15.40:0/4083626544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40b5f20 0x7f83d40d2cb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0097a0 tx=0x7f83cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2460131c2d8d82b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.250+0000 7f83d3fff640 1 -- 172.21.15.40:0/4083626544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc037070 con 0x7f83d40b5f20 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.250+0000 7f83d3fff640 1 -- 172.21.15.40:0/4083626544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc032440 con 0x7f83d40b5f20 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.250+0000 7f83d3fff640 1 -- 172.21.15.40:0/4083626544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc032810 con 0x7f83d40b5f20 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.251+0000 7f83db26b640 1 -- 172.21.15.40:0/4083626544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40b5f20 msgr2=0x7f83d40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.251+0000 7f83db26b640 1 --2- 172.21.15.40:0/4083626544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40b5f20 0x7f83d40d2cb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0097a0 tx=0x7f83cc02a230 comp rx=0 tx=0).stop 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.251+0000 7f83db26b640 1 -- 172.21.15.40:0/4083626544 shutdown_connections 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.251+0000 7f83db26b640 1 --2- 172.21.15.40:0/4083626544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40b5f20 0x7f83d40d2cb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:44.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.251+0000 7f83db26b640 1 -- 172.21.15.40:0/4083626544 >> 172.21.15.40:0/4083626544 conn(0x7f83d40d4f60 msgr2=0x7f83d40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:44.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.251+0000 7f83db26b640 1 -- 172.21.15.40:0/4083626544 shutdown_connections 2024-04-26T21:35:44.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.251+0000 7f83db26b640 1 -- 172.21.15.40:0/4083626544 wait complete. 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.252+0000 7f83db26b640 1 Processor -- start 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.252+0000 7f83db26b640 1 -- start start 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83db26b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d40b5f20 0x7f83d4153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83db26b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3bc0 0x7f83d40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83db26b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d4162880 0x7f83d4158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83db26b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4162c60 con 0x7f83d4162880 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83db26b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40d0d90 con 0x7f83d40b5f20 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83db26b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f83d4162de0 con 0x7f83d40d3bc0 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9808640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d4162880 0x7f83d4158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9808640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d4162880 0x7f83d4158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42066/0 (socket says 172.21.15.40:42066) 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9808640 1 -- 172.21.15.40:0/363419086 learned_addr learned my addr 172.21.15.40:0/363419086 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9007640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d40b5f20 0x7f83d4153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d8806640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3bc0 0x7f83d40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9808640 1 -- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3bc0 msgr2=0x7f83d40cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9808640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3bc0 0x7f83d40cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9808640 1 -- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d40b5f20 msgr2=0x7f83d4153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9808640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d40b5f20 0x7f83d4153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d9808640 1 -- 172.21.15.40:0/363419086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d4154d10 con 0x7f83d4162880 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.253+0000 7f83d8806640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3bc0 0x7f83d40cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.254+0000 7f83d9808640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d4162880 0x7f83d4158b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f83cc035040 tx=0x7f83cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.254+0000 7f83d1ffb640 1 -- 172.21.15.40:0/363419086 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc037040 con 0x7f83d4162880 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.254+0000 7f83d1ffb640 1 -- 172.21.15.40:0/363419086 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc042070 con 0x7f83d4162880 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.254+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d40cc630 con 0x7f83d4162880 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.254+0000 7f83d1ffb640 1 -- 172.21.15.40:0/363419086 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0308a0 con 0x7f83d4162880 2024-04-26T21:35:44.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.254+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d40ccbf0 con 0x7f83d4162880 2024-04-26T21:35:44.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.255+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d40694f0 con 0x7f83d4162880 2024-04-26T21:35:44.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.255+0000 7f83d1ffb640 1 -- 172.21.15.40:0/363419086 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83cc030a40 con 0x7f83d4162880 2024-04-26T21:35:44.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.256+0000 7f83d1ffb640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f83c006c8f0 0x7f83c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:44.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.256+0000 7f83d1ffb640 1 -- 172.21.15.40:0/363419086 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f83cc0ac770 con 0x7f83d4162880 2024-04-26T21:35:44.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.256+0000 7f83d9007640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f83c006c8f0 0x7f83c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:44.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.257+0000 7f83d9007640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f83c006c8f0 0x7f83c006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83c4001e70 tx=0x7f83c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:44.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.260+0000 7f83d1ffb640 1 -- 172.21.15.40:0/363419086 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83cc07a7d0 con 0x7f83d4162880 2024-04-26T21:35:44.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.485+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f83d40d2f00 con 0x7f83d4162880 2024-04-26T21:35:44.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.485+0000 7f83d1ffb640 1 -- 172.21.15.40:0/363419086 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f83cc07a0d0 con 0x7f83d4162880 2024-04-26T21:35:44.486 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f83c006c8f0 msgr2=0x7f83c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f83c006c8f0 0x7f83c006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83c4001e70 tx=0x7f83c4009040 comp rx=0 tx=0).stop 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d4162880 msgr2=0x7f83d4158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d4162880 0x7f83d4158b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f83cc035040 tx=0x7f83cc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 shutdown_connections 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f83c006c8f0 0x7f83c006edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d4162880 0x7f83d4158b90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3bc0 0x7f83d40cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:44.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 --2- 172.21.15.40:0/363419086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d40b5f20 0x7f83d4153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:44.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 >> 172.21.15.40:0/363419086 conn(0x7f83d40d4f60 msgr2=0x7f83d40b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:44.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 shutdown_connections 2024-04-26T21:35:44.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44.487+0000 7f83db26b640 1 -- 172.21.15.40:0/363419086 wait complete. 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:35:44.499 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:35:44.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:44.500 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:35:44.515 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:44.515 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:35:44.516 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:35:44.517 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42703sec preferred_lft 42703sec 2024-04-26T21:35:44.517 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:35:44.517 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:44.517 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:44.517 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:44.517 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:35:44.517 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:35:44.517 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:44.517 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:44.517 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:35:44.517 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-04-26T21:35:44.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-04-26T21:35:44.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:35:44.518 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:44.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:44.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:44.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:44.655 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:35:44.677 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:35:44.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:44.679 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:35:44.736 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:35:44.736 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:35:44.736 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:35:44.736 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:35:44.736 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:35:44.736 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:44.736 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:44.736 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:35:44.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:35:44.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:35:44.737 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:44.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:44.863 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:35:44.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:44.879 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-04-26T21:35:44.935 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:44.935 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:44.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:35:44.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:35:44.935 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:45.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:45.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:45.053 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:35:45.053 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:45.053 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:45.053 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:35:45.053 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:35:45.053 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:35:45.053 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:35:45.053 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:35:45.053 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:45.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:45.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:45.166 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:45.166 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:45.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:35:45.167 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:35:45.167 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:45.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:45.250 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:35:45.270 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:35:45.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:45.270 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:35:45.329 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:35:45.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:45.330 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:45.419 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:45.419 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:35:45.538 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:45.538 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:35:45.538 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:35:45.538 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:35:45.538 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:35:45.539 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:35:45.539 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:35:45.539 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:35:45.539 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:35:45.539 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:35:45.540 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:35:45.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:45.541 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:35:45.606 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:45.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:45.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:45.703 INFO:teuthology.orchestra.run.smithi040.stdout:client4711 2024-04-26T21:35:45.703 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:35:45.703 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:35:45.720 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:45.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:45.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:45.816 INFO:teuthology.orchestra.run.smithi040.stdout:client4711 2024-04-26T21:35:45.817 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:45.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:45.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:45.873 INFO:teuthology.orchestra.run.smithi040.stdout:client4711 2024-04-26T21:35:45.873 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ec445c36-0c6e-44d6-884b-6999dea32f55.client4711/status') 2024-04-26T21:35:45.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:45.932 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4711 (3)172.21.15.40:0/3756386846 2024-04-26T21:35:45.932 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2024-04-26T21:35:45.932 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2024-04-26T21:35:45.932 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2024-04-26T21:35:45.932 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 2.6325e-05 s, 2.7 MB/s 2024-04-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:45.940 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-26T21:35:45.940 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:35:45.940 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T21:35:45.940 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:35:46.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.040+0000 7f730712b640 1 Processor -- start 2024-04-26T21:35:46.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.040+0000 7f730712b640 1 -- start start 2024-04-26T21:35:46.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.041+0000 7f730712b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300162810 0x7f7300070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:46.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.041+0000 7f730712b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73000d10d0 con 0x7f73000d1b30 2024-04-26T21:35:46.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.041+0000 7f730712b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73000d1370 con 0x7f73000b2340 2024-04-26T21:35:46.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.041+0000 7f730712b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73000d0750 con 0x7f7300162810 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.041+0000 7f73056c8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f73000d1b30 0x7f7300070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38728/0 (socket says 172.21.15.40:38728) 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.041+0000 7f73056c8640 1 -- 172.21.15.40:0/2773350334 learned_addr learned my addr 172.21.15.40:0/2773350334 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f7304ec7640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300162810 0x7f7300070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074046859 0 0) 0x7f73000d10d0 con 0x7f73000d1b30 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72ec003520 con 0x7f73000d1b30 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2243008132 0 0) 0x7f72ec003520 con 0x7f73000d1b30 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300162810 msgr2=0x7f7300070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300162810 0x7f7300070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 >> v1:172.21.15.184:6789/0 conn(0x7f73000b2340 legacy=0x7f73000cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73000d01b0 con 0x7f73000d1b30 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f72e8001670 con 0x7f73000d1b30 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 >> v1:172.21.15.40:6789/0 conn(0x7f73000d1b30 legacy=0x7f7300070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:46.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72ec0039e0 0x7f72ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:46.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72ec004390 0x7f72ec004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:46.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300162810 0x7f72ec0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:46.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73000d0750 con 0x7f72ec004390 2024-04-26T21:35:46.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec00b9c0 con 0x7f72ec0039e0 2024-04-26T21:35:46.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72ec00bbc0 con 0x7f7300162810 2024-04-26T21:35:46.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72fffff640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72ec004390 0x7f72ec004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:46.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f7304ec7640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72ec0039e0 0x7f72ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f73056c8640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300162810 0x7f72ec0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72fffff640 1 -- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300162810 msgr2=0x7f72ec0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72fffff640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300162810 0x7f72ec0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72fffff640 1 -- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72ec0039e0 msgr2=0x7f72ec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72fffff640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72ec0039e0 0x7f72ec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.042+0000 7f72fffff640 1 -- 172.21.15.40:0/2773350334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec003520 con 0x7f72ec004390 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.043+0000 7f72fffff640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72ec004390 0x7f72ec004860 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f72f0001f60 tx=0x7f72ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697bb4c24b433a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.043+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0001b30 con 0x7f72ec004390 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.043+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f0031620 con 0x7f72ec004390 2024-04-26T21:35:46.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.043+0000 7f72ff7fe640 1 -- 172.21.15.40:0/2773350334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0031a10 con 0x7f72ec004390 2024-04-26T21:35:46.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.044+0000 7f730712b640 1 -- 172.21.15.40:0/2773350334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72ec004390 msgr2=0x7f72ec004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:46.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.044+0000 7f730712b640 1 --2- 172.21.15.40:0/2773350334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72ec004390 0x7f72ec004860 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f72f0001f60 tx=0x7f72ec005830 comp rx=0 tx=0).stop 2024-04-26T21:35:46.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.044+0000 7f73056c8640 1 -- 172.21.15.40:0/2773350334 reap_dead start 2024-04-26T21:35:46.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.044+0000 7f730712b640 1 -- 172.21.15.40:0/2773350334 shutdown_connections 2024-04-26T21:35:46.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.044+0000 7f730712b640 1 -- 172.21.15.40:0/2773350334 >> 172.21.15.40:0/2773350334 conn(0x7f73000d2da0 msgr2=0x7f73000d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:46.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.044+0000 7f730712b640 1 -- 172.21.15.40:0/2773350334 shutdown_connections 2024-04-26T21:35:46.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.044+0000 7f730712b640 1 -- 172.21.15.40:0/2773350334 wait complete. 2024-04-26T21:35:46.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.045+0000 7f730712b640 1 Processor -- start 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.045+0000 7f730712b640 1 -- start start 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f730712b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72ec004390 0x7f73000cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f730712b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73001f1ec0 0x7f73001f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f730712b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f730015fad0 0x7f730015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f730712b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73000cdb00 con 0x7f730015fad0 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f730712b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73000d16a0 con 0x7f73001f1ec0 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f730712b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7300071260 con 0x7f72ec004390 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f73056c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f730015fad0 0x7f730015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f73056c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f730015fad0 0x7f730015ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42078/0 (socket says 172.21.15.40:42078) 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f73056c8640 1 -- 172.21.15.40:0/3783940495 learned_addr learned my addr 172.21.15.40:0/3783940495 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f72fffff640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73001f1ec0 0x7f73001f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f7304ec7640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72ec004390 0x7f73000cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f73056c8640 1 -- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72ec004390 msgr2=0x7f73000cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f73056c8640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72ec004390 0x7f73000cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f73056c8640 1 -- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73001f1ec0 msgr2=0x7f73001f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f73056c8640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73001f1ec0 0x7f73001f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f73056c8640 1 -- 172.21.15.40:0/3783940495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73001608c0 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.046+0000 7f7304ec7640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72ec004390 0x7f73000cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.047+0000 7f73056c8640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f730015fad0 0x7f730015ff50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72e8000c00 tx=0x7f72e8007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.047+0000 7f72fd7fa640 1 -- 172.21.15.40:0/3783940495 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8007740 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.047+0000 7f72fd7fa640 1 -- 172.21.15.40:0/3783940495 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e8018070 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.047+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7300160ba0 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.047+0000 7f72fd7fa640 1 -- 172.21.15.40:0/3783940495 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e800fab0 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.047+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7300161160 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.048+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73000694f0 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.048+0000 7f72fd7fa640 1 -- 172.21.15.40:0/3783940495 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72e800fc50 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.049+0000 7f72fd7fa640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72e006cac0 0x7f72e006ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.049+0000 7f72fd7fa640 1 -- 172.21.15.40:0/3783940495 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f72e808bbd0 con 0x7f730015fad0 2024-04-26T21:35:46.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.049+0000 7f7304ec7640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72e006cac0 0x7f72e006ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:46.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.049+0000 7f7304ec7640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72e006cac0 0x7f72e006ef80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009b40 tx=0x7f72f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:46.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.053+0000 7f72fd7fa640 1 -- 172.21.15.40:0/3783940495 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72e8059cc0 con 0x7f730015fad0 2024-04-26T21:35:46.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.282+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f73000cf480 con 0x7f730015fad0 2024-04-26T21:35:46.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.282+0000 7f72fd7fa640 1 -- 172.21.15.40:0/3783940495 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f72e80595c0 con 0x7f730015fad0 2024-04-26T21:35:46.284 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72e006cac0 msgr2=0x7f72e006ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72e006cac0 0x7f72e006ef80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009b40 tx=0x7f72f8008040 comp rx=0 tx=0).stop 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f730015fad0 msgr2=0x7f730015ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f730015fad0 0x7f730015ff50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72e8000c00 tx=0x7f72e8007430 comp rx=0 tx=0).stop 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 shutdown_connections 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72e006cac0 0x7f72e006ef80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f730015fad0 0x7f730015ff50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73001f1ec0 0x7f73001f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 --2- 172.21.15.40:0/3783940495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72ec004390 0x7f73000cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 >> 172.21.15.40:0/3783940495 conn(0x7f73000d2da0 msgr2=0x7f73000d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.284+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 shutdown_connections 2024-04-26T21:35:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46.285+0000 7f730712b640 1 -- 172.21.15.40:0/3783940495 wait complete. 2024-04-26T21:35:46.296 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:35:46.296 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:35:46.297 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:35:46.297 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T21:35:46.297 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T21:35:46.297 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:35:46.297 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:35:46.297 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:35:46.297 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T21:35:46.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:46.297 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:35:46.312 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:35:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42701sec preferred_lft 42701sec 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:35:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.315 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:35:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:35:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link tentative 2024-04-26T21:35:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:46.315 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:35:46.376 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:35:46.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:46.377 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:35:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:35:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:35:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:35:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:35:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:35:46.434 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:46.434 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:46.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:35:46.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:35:46.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:35:46.435 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:46.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:46.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:46.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:46.559 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:35:46.575 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:35:46.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:46.575 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-04-26T21:35:46.630 INFO:teuthology.orchestra.run.smithi040.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T21:35:46.631 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:46.631 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:46.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:35:46.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T21:35:46.631 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:46.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:46.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:46.749 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 4e:47:2e:b4:7d:d9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:35:46.773 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.774 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::4c47:2eff:feb4:7dd9/64 scope link tentative 2024-04-26T21:35:46.774 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:46.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T21:35:46.839 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:46.839 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:46.839 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:46.839 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.840 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:35:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 4e:47:2e:b4:7d:d9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:35:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:35:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::4c47:2eff:feb4:7dd9/64 scope link tentative 2024-04-26T21:35:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:46.841 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T21:35:46.841 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:46.841 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:46.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T21:35:46.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T21:35:46.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T21:35:46.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T21:35:46.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T21:35:46.842 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:46.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:46.992 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:46.992 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:35:46.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.1 up 2024-04-26T21:35:46.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T21:35:46.992 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:35:47.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:47.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:47.069 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T21:35:47.084 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T21:35:47.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:47.085 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T21:35:47.140 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-26T21:35:47.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:47.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:47.223 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:47.223 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:35:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:35:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:35:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 2@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:35:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:35:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T21:35:47.235 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:35:47.236 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:35:47.236 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:35:47.236 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:35:47.237 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:35:47.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:47.238 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T21:35:47.302 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:47.400 INFO:teuthology.orchestra.run.smithi040.stdout:client4580 2024-04-26T21:35:47.400 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:35:47.400 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:35:47.416 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:47.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:47.512 INFO:teuthology.orchestra.run.smithi040.stdout:client4580 2024-04-26T21:35:47.512 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:47.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:47.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:47.568 INFO:teuthology.orchestra.run.smithi040.stdout:client4580 2024-04-26T21:35:47.569 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ec445c36-0c6e-44d6-884b-6999dea32f55.client4580/status') 2024-04-26T21:35:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:47.627 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4580 (3)172.21.15.40:0/2444248473 2024-04-26T21:35:47.627 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2024-04-26T21:35:47.628 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2024-04-26T21:35:47.628 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2024-04-26T21:35:47.628 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 2.3487e-05 s, 3.0 MB/s 2024-04-26T21:35:47.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:47.634 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-26T21:35:47.635 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:35:47.635 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:35:47.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:35:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.734+0000 7f8511740640 1 Processor -- start 2024-04-26T21:35:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.734+0000 7f8511740640 1 -- start start 2024-04-26T21:35:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f8511740640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f850c0b5f20 0x7f850c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f8511740640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850c0b7e50 con 0x7f850c0d3bc0 2024-04-26T21:35:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f8511740640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850c0d1800 con 0x7f850c162880 2024-04-26T21:35:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f8511740640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f850c0d0580 con 0x7f850c0b5f20 2024-04-26T21:35:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f8503fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f850c0d3bc0 0x7f850c0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38738/0 (socket says 172.21.15.40:38738) 2024-04-26T21:35:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f850b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f850c0b5f20 0x7f850c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f8503fff640 1 -- 172.21.15.40:0/1603231395 learned_addr learned my addr 172.21.15.40:0/1603231395 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f850b7fe640 1 -- 172.21.15.40:0/1603231395 >> v1:172.21.15.184:6789/0 conn(0x7f850c162880 legacy=0x7f850c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f850b7fe640 1 -- 172.21.15.40:0/1603231395 >> v1:172.21.15.40:6789/0 conn(0x7f850c0d3bc0 legacy=0x7f850c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f850b7fe640 1 -- 172.21.15.40:0/1603231395 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c0d07d0 con 0x7f850c0b5f20 2024-04-26T21:35:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.735+0000 7f850a7fc640 1 -- 172.21.15.40:0/1603231395 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 774667931 0 0) 0x7f850c0b7e50 con 0x7f850c0d3bc0 2024-04-26T21:35:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.736+0000 7f850b7fe640 1 --2- 172.21.15.40:0/1603231395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0b5f20 0x7f850c0d2cb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8504009d90 tx=0x7f850402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9463cfa5f81e5bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.736+0000 7f850a7fc640 1 -- 172.21.15.40:0/1603231395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504007740 con 0x7f850c0b5f20 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.736+0000 7f850a7fc640 1 -- 172.21.15.40:0/1603231395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85040078e0 con 0x7f850c0b5f20 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.736+0000 7f850a7fc640 1 -- 172.21.15.40:0/1603231395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504007cb0 con 0x7f850c0b5f20 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.737+0000 7f8511740640 1 -- 172.21.15.40:0/1603231395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0b5f20 msgr2=0x7f850c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.737+0000 7f8511740640 1 --2- 172.21.15.40:0/1603231395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0b5f20 0x7f850c0d2cb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8504009d90 tx=0x7f850402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.737+0000 7f8511740640 1 -- 172.21.15.40:0/1603231395 shutdown_connections 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.737+0000 7f8511740640 1 --2- 172.21.15.40:0/1603231395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0b5f20 0x7f850c0d2cb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.737+0000 7f8511740640 1 -- 172.21.15.40:0/1603231395 >> 172.21.15.40:0/1603231395 conn(0x7f850c0d4f60 msgr2=0x7f850c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.737+0000 7f8511740640 1 -- 172.21.15.40:0/1603231395 shutdown_connections 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.737+0000 7f8511740640 1 -- 172.21.15.40:0/1603231395 wait complete. 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.738+0000 7f8511740640 1 Processor -- start 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.738+0000 7f8511740640 1 -- start start 2024-04-26T21:35:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8511740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f850c0b5f20 0x7f850c0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8511740640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0d3bc0 0x7f850c1f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8511740640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f850c162880 0x7f850c1f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8511740640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f850c162c60 con 0x7f850c0d3bc0 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8511740640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0d0d90 con 0x7f850c162880 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8511740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f850c162de0 con 0x7f850c0b5f20 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0d3bc0 0x7f850c1f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0d3bc0 0x7f850c1f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42090/0 (socket says 172.21.15.40:42090) 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 -- 172.21.15.40:0/940786292 learned_addr learned my addr 172.21.15.40:0/940786292 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f850b7fe640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f850c162880 0x7f850c1f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f850affd640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f850c0b5f20 0x7f850c0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 -- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f850c0b5f20 msgr2=0x7f850c0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f850c0b5f20 0x7f850c0b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 -- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f850c162880 msgr2=0x7f850c1f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f850c162880 0x7f850c1f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 -- 172.21.15.40:0/940786292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c1f4b80 con 0x7f850c0d3bc0 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f850b7fe640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f850c162880 0x7f850c1f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f8503fff640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0d3bc0 0x7f850c1f55f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c4d0 tx=0x7f84f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.739+0000 7f85037fe640 1 -- 172.21.15.40:0/940786292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4018070 con 0x7f850c0d3bc0 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.740+0000 7f85037fe640 1 -- 172.21.15.40:0/940786292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f40074e0 con 0x7f850c0d3bc0 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.740+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c1f8c40 con 0x7f850c0d3bc0 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.740+0000 7f85037fe640 1 -- 172.21.15.40:0/940786292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4004a20 con 0x7f850c0d3bc0 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.740+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c1f4df0 con 0x7f850c0d3bc0 2024-04-26T21:35:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.741+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c0694f0 con 0x7f850c0d3bc0 2024-04-26T21:35:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.741+0000 7f85037fe640 1 -- 172.21.15.40:0/940786292 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84f401b070 con 0x7f850c0d3bc0 2024-04-26T21:35:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.742+0000 7f85037fe640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84ec06c8f0 0x7f84ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.742+0000 7f85037fe640 1 -- 172.21.15.40:0/940786292 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f84f408b100 con 0x7f850c0d3bc0 2024-04-26T21:35:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.742+0000 7f850affd640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84ec06c8f0 0x7f84ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.742+0000 7f850affd640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84ec06c8f0 0x7f84ec06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f850c0cd0d0 tx=0x7f84fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.746+0000 7f85037fe640 1 -- 172.21.15.40:0/940786292 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f84f4059280 con 0x7f850c0d3bc0 2024-04-26T21:35:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.972+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f850c0cf390 con 0x7f850c0d3bc0 2024-04-26T21:35:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.973+0000 7f85037fe640 1 -- 172.21.15.40:0/940786292 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f84f4058b80 con 0x7f850c0d3bc0 2024-04-26T21:35:47.974 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T21:35:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84ec06c8f0 msgr2=0x7f84ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84ec06c8f0 0x7f84ec06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f850c0cd0d0 tx=0x7f84fc009040 comp rx=0 tx=0).stop 2024-04-26T21:35:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0d3bc0 msgr2=0x7f850c1f55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0d3bc0 0x7f850c1f55f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c4d0 tx=0x7f84f400c990 comp rx=0 tx=0).stop 2024-04-26T21:35:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 shutdown_connections 2024-04-26T21:35:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84ec06c8f0 0x7f84ec06edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f850c162880 0x7f850c1f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850c0d3bc0 0x7f850c1f55f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 --2- 172.21.15.40:0/940786292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f850c0b5f20 0x7f850c0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 >> 172.21.15.40:0/940786292 conn(0x7f850c0d4f60 msgr2=0x7f850c0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 shutdown_connections 2024-04-26T21:35:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:47.975+0000 7f8511740640 1 -- 172.21.15.40:0/940786292 wait complete. 2024-04-26T21:35:47.987 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:35:47.987 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:35:47.987 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:35:47.987 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:35:47.988 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:35:47.988 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:35:47.988 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:35:47.988 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:35:47.988 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:35:47.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:47.989 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:35:48.007 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:48.008 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:48.009 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:35:48.009 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:48.009 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:35:48.009 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42710sec preferred_lft 42710sec 2024-04-26T21:35:48.009 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::b78f:954:1ba9:31d4/64 scope link noprefixroute 2024-04-26T21:35:48.009 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:48.009 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:35:48.009 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:48.009 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:48.009 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:35:48.009 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr flush dev ceph-brx 2024-04-26T21:35:48.010 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set ceph-brx up 2024-04-26T21:35:48.010 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:35:48.010 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:48.104 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:48.195 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:48.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:48.202 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:35:48.226 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:35:48.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:48.228 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:35:48.297 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:35:48.297 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:35:48.297 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:35:48.298 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:35:48.298 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:35:48.298 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:48.298 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:48.298 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:35:48.298 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:35:48.298 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:35:48.298 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:48.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:48.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:48.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:48.562 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:35:48.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:48.576 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2024-04-26T21:35:48.630 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:48.630 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:48.631 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:35:48.631 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T21:35:48.631 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:48.719 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:48.741 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:48.747 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T21:35:48.747 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:48.747 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:48.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T21:35:48.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:35:48.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T21:35:48.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T21:35:48.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T21:35:48.748 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:48.797 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:48.866 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:48.875 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:48.875 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:48.875 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:35:48.875 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:35:48.875 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:48.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:48.945 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:48.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:48.954 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:35:48.967 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:35:48.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:48.968 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:35:49.017 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-26T21:35:49.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:49.017 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:49.103 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:49.103 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:35:49.211 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:49.211 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:35:49.211 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:35:49.211 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with new device syntax: 1@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:35:49.212 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:35:49.212 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:35:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:35:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:35:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:35:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:35:49.214 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:35:49.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:49.214 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:35:49.238 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:49.327 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:49.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:49.335 INFO:teuthology.orchestra.run.smithi184.stdout:client4905 2024-04-26T21:35:49.335 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:35:49.335 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:35:49.351 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:49.438 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:49.440 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:49.446 INFO:teuthology.orchestra.run.smithi184.stdout:client4905 2024-04-26T21:35:49.447 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:49.494 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:49.496 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:49.501 INFO:teuthology.orchestra.run.smithi184.stdout:client4905 2024-04-26T21:35:49.502 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ec445c36-0c6e-44d6-884b-6999dea32f55.client4905/status') 2024-04-26T21:35:49.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:49.559 INFO:teuthology.orchestra.run.smithi184.stdout:instance: client.4905 (3)172.21.15.184:0/1031440754 2024-04-26T21:35:49.559 INFO:teuthology.orchestra.run.smithi184.stdout:blocklisted: false 2024-04-26T21:35:49.559 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records in 2024-04-26T21:35:49.559 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records out 2024-04-26T21:35:49.560 INFO:teuthology.orchestra.run.smithi184.stderr:71 bytes copied, 2.2083e-05 s, 3.2 MB/s 2024-04-26T21:35:49.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:49.567 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-26T21:35:49.567 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:35:49.567 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T21:35:49.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:35:49.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.680+0000 7f193d324640 1 Processor -- start 2024-04-26T21:35:49.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.680+0000 7f193d324640 1 -- start start 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f193d324640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19380d3bc0 0x7f19380d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f193d324640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380b5f20 0x7f19380b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f193d324640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19380b7e50 con 0x7f1938162880 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f193d324640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f19380d0580 con 0x7f19380d3bc0 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f193d324640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19380d1800 con 0x7f19380b5f20 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f192ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19380d3bc0 0x7f19380d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f192ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19380d3bc0 0x7f19380d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55528/0 (socket says 172.21.15.40:55528) 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f192ffff640 1 -- 172.21.15.40:0/3353122471 learned_addr learned my addr 172.21.15.40:0/3353122471 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f19377fe640 1 --2- 172.21.15.40:0/3353122471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380b5f20 0x7f19380b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f192ffff640 1 -- 172.21.15.40:0/3353122471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380b5f20 msgr2=0x7f19380b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f192ffff640 1 --2- 172.21.15.40:0/3353122471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380b5f20 0x7f19380b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f192ffff640 1 -- 172.21.15.40:0/3353122471 >> v1:172.21.15.184:6789/0 conn(0x7f1938162880 legacy=0x7f19380d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:49.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.681+0000 7f192ffff640 1 -- 172.21.15.40:0/3353122471 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19380d07d0 con 0x7f19380d3bc0 2024-04-26T21:35:49.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.682+0000 7f192ffff640 1 --2- 172.21.15.40:0/3353122471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380d3bc0 0x7f19380d2cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1920009870 tx=0x7f192002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92db7d96c4e3c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:49.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.682+0000 7f19367fc640 1 -- 172.21.15.40:0/3353122471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920037070 con 0x7f19380d3bc0 2024-04-26T21:35:49.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.682+0000 7f19367fc640 1 -- 172.21.15.40:0/3353122471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1920032440 con 0x7f19380d3bc0 2024-04-26T21:35:49.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.682+0000 7f19367fc640 1 -- 172.21.15.40:0/3353122471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920032810 con 0x7f19380d3bc0 2024-04-26T21:35:49.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.683+0000 7f193d324640 1 -- 172.21.15.40:0/3353122471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380d3bc0 msgr2=0x7f19380d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.683+0000 7f193d324640 1 --2- 172.21.15.40:0/3353122471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380d3bc0 0x7f19380d2cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1920009870 tx=0x7f192002a210 comp rx=0 tx=0).stop 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.683+0000 7f193d324640 1 -- 172.21.15.40:0/3353122471 shutdown_connections 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.683+0000 7f193d324640 1 --2- 172.21.15.40:0/3353122471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380b5f20 0x7f19380b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.683+0000 7f193d324640 1 --2- 172.21.15.40:0/3353122471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380d3bc0 0x7f19380d2cb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.683+0000 7f193d324640 1 -- 172.21.15.40:0/3353122471 >> 172.21.15.40:0/3353122471 conn(0x7f19380d4f60 msgr2=0x7f19380d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.683+0000 7f193d324640 1 -- 172.21.15.40:0/3353122471 shutdown_connections 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.683+0000 7f193d324640 1 -- 172.21.15.40:0/3353122471 wait complete. 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.684+0000 7f193d324640 1 Processor -- start 2024-04-26T21:35:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.684+0000 7f193d324640 1 -- start start 2024-04-26T21:35:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f193d324640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380b5f20 0x7f19381f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f193d324640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19380d3bc0 0x7f193806de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f193d324640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1938162880 0x7f193806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f193d324640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1938162c60 con 0x7f19380b5f20 2024-04-26T21:35:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f193d324640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19380d0d90 con 0x7f19380d3bc0 2024-04-26T21:35:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f193d324640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1938162de0 con 0x7f1938162880 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f1936ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380b5f20 0x7f19381f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f1936ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380b5f20 0x7f19381f7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55544/0 (socket says 172.21.15.40:55544) 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f1936ffd640 1 -- 172.21.15.40:0/4229846586 learned_addr learned my addr 172.21.15.40:0/4229846586 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f19377fe640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1938162880 0x7f193806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.685+0000 7f192ffff640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19380d3bc0 0x7f193806de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f1936ffd640 1 -- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1938162880 msgr2=0x7f193806e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f1936ffd640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1938162880 0x7f193806e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f1936ffd640 1 -- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19380d3bc0 msgr2=0x7f193806de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f1936ffd640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19380d3bc0 0x7f193806de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f1936ffd640 1 -- 172.21.15.40:0/4229846586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19381f4e50 con 0x7f19380b5f20 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f1936ffd640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380b5f20 0x7f19381f7960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f192800c430 tx=0x7f192800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f192f7fe640 1 -- 172.21.15.40:0/4229846586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928011790 con 0x7f19380b5f20 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f192f7fe640 1 -- 172.21.15.40:0/4229846586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1928017070 con 0x7f19380b5f20 2024-04-26T21:35:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19381f7ea0 con 0x7f19380b5f20 2024-04-26T21:35:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f192f7fe640 1 -- 172.21.15.40:0/4229846586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928011bb0 con 0x7f19380b5f20 2024-04-26T21:35:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.686+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193815a630 con 0x7f19380b5f20 2024-04-26T21:35:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.687+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19380694f0 con 0x7f19380b5f20 2024-04-26T21:35:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.688+0000 7f192f7fe640 1 -- 172.21.15.40:0/4229846586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f192801c460 con 0x7f19380b5f20 2024-04-26T21:35:49.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.688+0000 7f192f7fe640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f191806c8f0 0x7f191806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:49.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.688+0000 7f192f7fe640 1 -- 172.21.15.40:0/4229846586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f192808bac0 con 0x7f19380b5f20 2024-04-26T21:35:49.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.688+0000 7f192ffff640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f191806c8f0 0x7f191806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:49.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.689+0000 7f192ffff640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f191806c8f0 0x7f191806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1920034000 tx=0x7f192002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.693+0000 7f192f7fe640 1 -- 172.21.15.40:0/4229846586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19280595f0 con 0x7f19380b5f20 2024-04-26T21:35:49.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.917+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1938074590 con 0x7f19380b5f20 2024-04-26T21:35:49.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.917+0000 7f192f7fe640 1 -- 172.21.15.40:0/4229846586 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f192805d8f0 con 0x7f19380b5f20 2024-04-26T21:35:49.919 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f191806c8f0 msgr2=0x7f191806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f191806c8f0 0x7f191806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1920034000 tx=0x7f192002e000 comp rx=0 tx=0).stop 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380b5f20 msgr2=0x7f19381f7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380b5f20 0x7f19381f7960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f192800c430 tx=0x7f192800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 shutdown_connections 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f191806c8f0 0x7f191806edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1938162880 0x7f193806e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19380d3bc0 0x7f193806de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 --2- 172.21.15.40:0/4229846586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19380b5f20 0x7f19381f7960 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 >> 172.21.15.40:0/4229846586 conn(0x7f19380d4f60 msgr2=0x7f19380d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 shutdown_connections 2024-04-26T21:35:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:49.919+0000 7f193d324640 1 -- 172.21.15.40:0/4229846586 wait complete. 2024-04-26T21:35:49.931 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:35:49.931 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:35:49.931 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:35:49.931 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T21:35:49.931 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T21:35:49.931 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:35:49.932 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:35:49.932 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:35:49.932 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T21:35:49.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:49.932 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:35:49.947 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:35:49.948 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42708sec preferred_lft 42708sec 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::b78f:954:1ba9:31d4/64 scope link noprefixroute 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 22:36:ab:29:ad:c5 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:49.949 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::4cc0:ff:fe94:cfeb/64 scope link 2024-04-26T21:35:49.950 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:49.950 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:35:49.950 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 22:36:ab:29:ad:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:35:49.950 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::2036:abff:fe29:adc5/64 scope link tentative 2024-04-26T21:35:49.950 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:49.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:49.950 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:35:50.011 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:35:50.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:50.012 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:35:50.069 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:35:50.070 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:35:50.070 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:35:50.070 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:35:50.070 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:35:50.070 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:50.070 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:50.070 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:35:50.070 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:35:50.071 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:35:50.071 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:50.157 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:50.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:50.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:50.193 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:35:50.208 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:35:50.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:50.209 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2024-04-26T21:35:50.266 INFO:teuthology.orchestra.run.smithi184.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T21:35:50.267 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:50.267 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:50.267 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:35:50.267 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T21:35:50.267 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:50.354 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:50.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:50.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:50.382 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 56:32:f8:23:8d:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:35:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::5432:f8ff:fe23:8da2/64 scope link tentative 2024-04-26T21:35:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:50.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:50.407 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T21:35:50.469 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:35:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:35:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 56:32:f8:23:8d:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::5432:f8ff:fe23:8da2/64 scope link tentative 2024-04-26T21:35:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:35:50.471 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T21:35:50.471 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:50.471 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:50.471 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T21:35:50.472 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T21:35:50.472 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T21:35:50.472 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T21:35:50.472 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T21:35:50.472 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:50.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:50.618 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:35:50.618 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:35:50.618 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.1 up 2024-04-26T21:35:50.618 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T21:35:50.618 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:35:50.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:50.686 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:50.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:50.693 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T21:35:50.706 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T21:35:50.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:35:50.707 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T21:35:50.761 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-26T21:35:50.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:50.761 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:50.842 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:50.843 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:35:50.850 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:35:50.850 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:35:50.850 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:35:50.851 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with new device syntax: 3@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:35:50.851 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:35:50.851 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T21:35:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:35:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:35:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:35:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:35:50.853 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:35:50.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:35:50.853 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T21:35:50.916 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:51.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:51.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:51.012 INFO:teuthology.orchestra.run.smithi184.stdout:client4914 2024-04-26T21:35:51.012 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:35:51.012 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:35:51.027 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:51.115 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:51.117 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:51.122 INFO:teuthology.orchestra.run.smithi184.stdout:client4914 2024-04-26T21:35:51.123 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:35:51.169 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:35:51.171 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:51.177 INFO:teuthology.orchestra.run.smithi184.stdout:client4914 2024-04-26T21:35:51.177 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ec445c36-0c6e-44d6-884b-6999dea32f55.client4914/status') 2024-04-26T21:35:51.225 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:35:51.234 INFO:teuthology.orchestra.run.smithi184.stdout:instance: client.4914 (3)172.21.15.184:0/2787305084 2024-04-26T21:35:51.234 INFO:teuthology.orchestra.run.smithi184.stdout:blocklisted: false 2024-04-26T21:35:51.235 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records in 2024-04-26T21:35:51.235 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records out 2024-04-26T21:35:51.235 INFO:teuthology.orchestra.run.smithi184.stderr:71 bytes copied, 2.1205e-05 s, 3.3 MB/s 2024-04-26T21:35:51.235 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:35:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:35:51.242 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T21:35:51.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:51.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.370+0000 7fdd83ecf640 1 Processor -- start 2024-04-26T21:35:51.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.370+0000 7fdd83ecf640 1 -- start start 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd83ecf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdd7c0d06f0 0x7fdd7c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd83ecf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdd7c0b2850 0x7fdd7c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd83ecf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd7c0d0ba0 con 0x7fdd7c15f3a0 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd83ecf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7c0d0e20 con 0x7fdd7c0b2850 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd83ecf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7c0b1d70 con 0x7fdd7c0d06f0 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd8246c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdd7c0b2850 0x7fdd7c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd8246c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdd7c0b2850 0x7fdd7c0b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55552/0 (socket says 172.21.15.40:55552) 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd8246c640 1 -- 172.21.15.40:0/2392483458 learned_addr learned my addr 172.21.15.40:0/2392483458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.371+0000 7fdd8146a640 1 --2- 172.21.15.40:0/2392483458 >> v2:172.21.15.184:3300/0 conn(0x7fdd7c0d06f0 0x7fdd7c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.372+0000 7fdd8246c640 1 -- 172.21.15.40:0/2392483458 >> v2:172.21.15.184:3300/0 conn(0x7fdd7c0d06f0 msgr2=0x7fdd7c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:51.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.372+0000 7fdd8246c640 1 --2- 172.21.15.40:0/2392483458 >> v2:172.21.15.184:3300/0 conn(0x7fdd7c0d06f0 0x7fdd7c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.372+0000 7fdd8246c640 1 -- 172.21.15.40:0/2392483458 >> v1:172.21.15.184:6789/0 conn(0x7fdd7c15f3a0 legacy=0x7fdd7c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.372+0000 7fdd8246c640 1 -- 172.21.15.40:0/2392483458 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c0b2050 con 0x7fdd7c0b2850 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.372+0000 7fdd8246c640 1 --2- 172.21.15.40:0/2392483458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0b2c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0b7b80 tx=0x7fdd6c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d93c2f8e97c8970 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.372+0000 7fdd80c69640 1 -- 172.21.15.40:0/2392483458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c036070 con 0x7fdd7c0b2850 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd80c69640 1 -- 172.21.15.40:0/2392483458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd6c031600 con 0x7fdd7c0b2850 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd80c69640 1 -- 172.21.15.40:0/2392483458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0319d0 con 0x7fdd7c0b2850 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2392483458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 msgr2=0x7fdd7c0b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2392483458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0b2c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0b7b80 tx=0x7fdd6c02a730 comp rx=0 tx=0).stop 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2392483458 shutdown_connections 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2392483458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0b2c80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2392483458 >> v2:172.21.15.184:3300/0 conn(0x7fdd7c0d06f0 0x7fdd7c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2392483458 >> 172.21.15.40:0/2392483458 conn(0x7fdd7c0d4f90 msgr2=0x7fdd7c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:51.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2392483458 shutdown_connections 2024-04-26T21:35:51.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.373+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2392483458 wait complete. 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.374+0000 7fdd83ecf640 1 Processor -- start 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.374+0000 7fdd83ecf640 1 -- start start 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd83ecf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd83ecf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c0d06f0 0x7fdd7c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd83ecf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd7c15f3a0 0x7fdd7c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd83ecf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c15f900 con 0x7fdd7c0b2850 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd83ecf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c15f780 con 0x7fdd7c0d06f0 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd83ecf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7c0d23c0 con 0x7fdd7c15f3a0 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd81c6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd81c6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55568/0 (socket says 172.21.15.40:55568) 2024-04-26T21:35:51.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd81c6b640 1 -- 172.21.15.40:0/2278723641 learned_addr learned my addr 172.21.15.40:0/2278723641 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd8146a640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c0d06f0 0x7fdd7c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.375+0000 7fdd8246c640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd7c15f3a0 0x7fdd7c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd81c6b640 1 -- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd7c15f3a0 msgr2=0x7fdd7c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd81c6b640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd7c15f3a0 0x7fdd7c15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd81c6b640 1 -- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c0d06f0 msgr2=0x7fdd7c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd81c6b640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c0d06f0 0x7fdd7c0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd81c6b640 1 -- 172.21.15.40:0/2278723641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c0c9e60 con 0x7fdd7c0b2850 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd81c6b640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0c9920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400c3c0 tx=0x7fdd7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd7a7fc640 1 -- 172.21.15.40:0/2278723641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd740052d0 con 0x7fdd7c0b2850 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd7a7fc640 1 -- 172.21.15.40:0/2278723641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd74005490 con 0x7fdd7c0b2850 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd7a7fc640 1 -- 172.21.15.40:0/2278723641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74007260 con 0x7fdd7c0b2850 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7c0cac70 con 0x7fdd7c0b2850 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.376+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd7c0c8e40 con 0x7fdd7c0b2850 2024-04-26T21:35:51.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.377+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd7c0694f0 con 0x7fdd7c0b2850 2024-04-26T21:35:51.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.377+0000 7fdd7a7fc640 1 -- 172.21.15.40:0/2278723641 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd74007400 con 0x7fdd7c0b2850 2024-04-26T21:35:51.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.378+0000 7fdd7a7fc640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd6806c8f0 0x7fdd6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.378+0000 7fdd7a7fc640 1 -- 172.21.15.40:0/2278723641 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdd7408b340 con 0x7fdd7c0b2850 2024-04-26T21:35:51.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.378+0000 7fdd8146a640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd6806c8f0 0x7fdd6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.379+0000 7fdd8146a640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd6806c8f0 0x7fdd6806edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c06d510 tx=0x7fdd70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:51.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.383+0000 7fdd7a7fc640 1 -- 172.21.15.40:0/2278723641 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd7c0c9e60 con 0x7fdd7c0b2850 2024-04-26T21:35:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.569+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd7c06d260 con 0x7fdd7c0b2850 2024-04-26T21:35:51.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.571+0000 7fdd7a7fc640 1 -- 172.21.15.40:0/2278723641 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7fdd7405f020 con 0x7fdd7c0b2850 2024-04-26T21:35:51.572 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:51.572 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:51.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd6806c8f0 msgr2=0x7fdd6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd6806c8f0 0x7fdd6806edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c06d510 tx=0x7fdd70008040 comp rx=0 tx=0).stop 2024-04-26T21:35:51.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 msgr2=0x7fdd7c0c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0c9920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400c3c0 tx=0x7fdd7400c880 comp rx=0 tx=0).stop 2024-04-26T21:35:51.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 shutdown_connections 2024-04-26T21:35:51.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdd6806c8f0 0x7fdd6806edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd7c15f3a0 0x7fdd7c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c0d06f0 0x7fdd7c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 --2- 172.21.15.40:0/2278723641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7c0b2850 0x7fdd7c0c9920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 >> 172.21.15.40:0/2278723641 conn(0x7fdd7c0d4f90 msgr2=0x7fdd7c0d1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:51.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 shutdown_connections 2024-04-26T21:35:51.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.573+0000 7fdd83ecf640 1 -- 172.21.15.40:0/2278723641 wait complete. 2024-04-26T21:35:51.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:51.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:51.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.688+0000 7f3e562c4640 1 Processor -- start 2024-04-26T21:35:51.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.688+0000 7f3e562c4640 1 -- start start 2024-04-26T21:35:51.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e562c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e500d3bc0 0x7f3e500d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e562c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e500d1040 con 0x7f3e5015f170 2024-04-26T21:35:51.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e562c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e500c9680 con 0x7f3e5015f550 2024-04-26T21:35:51.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e562c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3e500c9430 con 0x7f3e500d3bc0 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e4ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3e5015f170 0x7f3e500d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54662/0 (socket says 172.21.15.40:54662) 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e4ffff640 1 -- 172.21.15.40:0/1886727965 learned_addr learned my addr 172.21.15.40:0/1886727965 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e54861640 1 --2- 172.21.15.40:0/1886727965 >> v2:172.21.15.184:3300/0 conn(0x7f3e500d3bc0 0x7f3e500d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e4effd640 1 -- 172.21.15.40:0/1886727965 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311905825 0 0) 0x7f3e40001110 con 0x7f3e5015f170 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e4effd640 1 -- 172.21.15.40:0/1886727965 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e3c003520 con 0x7f3e5015f170 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e54861640 1 -- 172.21.15.40:0/1886727965 >> v1:172.21.15.184:6789/0 conn(0x7f3e5015f550 legacy=0x7f3e500ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e54861640 1 -- 172.21.15.40:0/1886727965 >> v1:172.21.15.40:6789/0 conn(0x7f3e5015f170 legacy=0x7f3e500d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.689+0000 7f3e54861640 1 -- 172.21.15.40:0/1886727965 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e500c9960 con 0x7f3e500d3bc0 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.690+0000 7f3e54861640 1 --2- 172.21.15.40:0/1886727965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e500d3bc0 0x7f3e500d5a80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3e500c95b0 tx=0x7f3e4802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea76b64885c287b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:51.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.690+0000 7f3e4effd640 1 -- 172.21.15.40:0/1886727965 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48007740 con 0x7f3e500d3bc0 2024-04-26T21:35:51.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.690+0000 7f3e4effd640 1 -- 172.21.15.40:0/1886727965 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e480078e0 con 0x7f3e500d3bc0 2024-04-26T21:35:51.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.690+0000 7f3e4effd640 1 -- 172.21.15.40:0/1886727965 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48007c90 con 0x7f3e500d3bc0 2024-04-26T21:35:51.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.691+0000 7f3e562c4640 1 -- 172.21.15.40:0/1886727965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e500d3bc0 msgr2=0x7f3e500d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.691+0000 7f3e562c4640 1 --2- 172.21.15.40:0/1886727965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e500d3bc0 0x7f3e500d5a80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3e500c95b0 tx=0x7f3e4802a710 comp rx=0 tx=0).stop 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.691+0000 7f3e562c4640 1 -- 172.21.15.40:0/1886727965 shutdown_connections 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.691+0000 7f3e562c4640 1 --2- 172.21.15.40:0/1886727965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e500d3bc0 0x7f3e500d5a80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.691+0000 7f3e562c4640 1 -- 172.21.15.40:0/1886727965 >> 172.21.15.40:0/1886727965 conn(0x7f3e500d4f60 msgr2=0x7f3e500d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.691+0000 7f3e562c4640 1 -- 172.21.15.40:0/1886727965 shutdown_connections 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.691+0000 7f3e562c4640 1 -- 172.21.15.40:0/1886727965 wait complete. 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.692+0000 7f3e562c4640 1 Processor -- start 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.692+0000 7f3e562c4640 1 -- start start 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e562c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e500d3bc0 0x7f3e5015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e562c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e5015f170 0x7f3e500a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e562c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e500a0950 0x7f3e500a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e562c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e500d2e30 con 0x7f3e500a0950 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e562c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3e500d2cb0 con 0x7f3e5015f170 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e562c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3e500d2fb0 con 0x7f3e500d3bc0 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e500a0950 0x7f3e500a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e500a0950 0x7f3e500a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55574/0 (socket says 172.21.15.40:55574) 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 -- 172.21.15.40:0/4148859463 learned_addr learned my addr 172.21.15.40:0/4148859463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e4ffff640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e500d3bc0 0x7f3e5015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e4f7fe640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e5015f170 0x7f3e500a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 -- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e500d3bc0 msgr2=0x7f3e5015d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e500d3bc0 0x7f3e5015d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 -- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e5015f170 msgr2=0x7f3e500a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e5015f170 0x7f3e500a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 -- 172.21.15.40:0/4148859463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e500a0e90 con 0x7f3e500a0950 2024-04-26T21:35:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e4ffff640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e500d3bc0 0x7f3e5015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.693+0000 7f3e54861640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e500a0950 0x7f3e500a21d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009d90 tx=0x7f3e48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.694+0000 7f3e4cff9640 1 -- 172.21.15.40:0/4148859463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48040070 con 0x7f3e500a0950 2024-04-26T21:35:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.694+0000 7f3e4cff9640 1 -- 172.21.15.40:0/4148859463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e48031c40 con 0x7f3e500a0950 2024-04-26T21:35:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.694+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e500a1170 con 0x7f3e500a0950 2024-04-26T21:35:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.694+0000 7f3e4cff9640 1 -- 172.21.15.40:0/4148859463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48009450 con 0x7f3e500a0950 2024-04-26T21:35:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.694+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e500a1730 con 0x7f3e500a0950 2024-04-26T21:35:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.695+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e500694f0 con 0x7f3e500a0950 2024-04-26T21:35:51.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.695+0000 7f3e4cff9640 1 -- 172.21.15.40:0/4148859463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e48039040 con 0x7f3e500a0950 2024-04-26T21:35:51.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.696+0000 7f3e4cff9640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e3006c8f0 0x7f3e3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:51.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.696+0000 7f3e4cff9640 1 -- 172.21.15.40:0/4148859463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3e480accd0 con 0x7f3e500a0950 2024-04-26T21:35:51.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.696+0000 7f3e4ffff640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e3006c8f0 0x7f3e3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:51.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.696+0000 7f3e4ffff640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e3006c8f0 0x7f3e3006edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3e40007fd0 tx=0x7f3e40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.700+0000 7f3e4cff9640 1 -- 172.21.15.40:0/4148859463 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e4807ad90 con 0x7f3e500a0950 2024-04-26T21:35:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.887+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e500bade0 con 0x7f3e500a0950 2024-04-26T21:35:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.888+0000 7f3e4cff9640 1 -- 172.21.15.40:0/4148859463 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f3e4807a690 con 0x7f3e500a0950 2024-04-26T21:35:51.889 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e3006c8f0 msgr2=0x7f3e3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e3006c8f0 0x7f3e3006edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3e40007fd0 tx=0x7f3e40009040 comp rx=0 tx=0).stop 2024-04-26T21:35:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e500a0950 msgr2=0x7f3e500a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e500a0950 0x7f3e500a21d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009d90 tx=0x7f3e48004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 shutdown_connections 2024-04-26T21:35:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e3006c8f0 0x7f3e3006edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e500a0950 0x7f3e500a21d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e5015f170 0x7f3e500a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 --2- 172.21.15.40:0/4148859463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e500d3bc0 0x7f3e5015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 >> 172.21.15.40:0/4148859463 conn(0x7f3e500d4f60 msgr2=0x7f3e500d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 shutdown_connections 2024-04-26T21:35:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:51.890+0000 7f3e562c4640 1 -- 172.21.15.40:0/4148859463 wait complete. 2024-04-26T21:35:51.903 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:52.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.006+0000 7f4801659640 1 Processor -- start 2024-04-26T21:35:52.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.006+0000 7f4801659640 1 -- start start 2024-04-26T21:35:52.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f4801659640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f47fc0d1a00 0x7f47fc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f4801659640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47fc0b40d0 0x7f47fc0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f4801659640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc0d0a10 con 0x7f47fc0d2320 2024-04-26T21:35:52.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f4801659640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f47fc0d0cf0 con 0x7f47fc0b40d0 2024-04-26T21:35:52.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f4801659640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f47fc0d0fd0 con 0x7f47fc0d1a00 2024-04-26T21:35:52.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47fb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47fc0b40d0 0x7f47fc0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47fa7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f47fc0d2320 0x7f47fc1625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54674/0 (socket says 172.21.15.40:54674) 2024-04-26T21:35:52.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47fa7fc640 1 -- 172.21.15.40:0/1695436192 learned_addr learned my addr 172.21.15.40:0/1695436192 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:52.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47faffd640 1 --2- 172.21.15.40:0/1695436192 >> v2:172.21.15.184:3300/0 conn(0x7f47fc0d1a00 0x7f47fc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47fb7fe640 1 -- 172.21.15.40:0/1695436192 >> v2:172.21.15.184:3300/0 conn(0x7f47fc0d1a00 msgr2=0x7f47fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:52.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47fb7fe640 1 --2- 172.21.15.40:0/1695436192 >> v2:172.21.15.184:3300/0 conn(0x7f47fc0d1a00 0x7f47fc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47fb7fe640 1 -- 172.21.15.40:0/1695436192 >> v1:172.21.15.40:6789/0 conn(0x7f47fc0d2320 legacy=0x7f47fc1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47fb7fe640 1 -- 172.21.15.40:0/1695436192 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc0d40a0 con 0x7f47fc0b40d0 2024-04-26T21:35:52.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.007+0000 7f47f9ffb640 1 -- 172.21.15.40:0/1695436192 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 283997206 0 0) 0x7f47fc0d0a10 con 0x7f47fc0d2320 2024-04-26T21:35:52.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.008+0000 7f47fb7fe640 1 --2- 172.21.15.40:0/1695436192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0b40d0 0x7f47fc0b4b00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47fc06c650 tx=0x7f47f402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ee08fe7b6a84d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.008+0000 7f47f9ffb640 1 -- 172.21.15.40:0/1695436192 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4007740 con 0x7f47fc0b40d0 2024-04-26T21:35:52.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.008+0000 7f47f9ffb640 1 -- 172.21.15.40:0/1695436192 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f40078e0 con 0x7f47fc0b40d0 2024-04-26T21:35:52.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.008+0000 7f47f9ffb640 1 -- 172.21.15.40:0/1695436192 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4007cb0 con 0x7f47fc0b40d0 2024-04-26T21:35:52.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.009+0000 7f4801659640 1 -- 172.21.15.40:0/1695436192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0b40d0 msgr2=0x7f47fc0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.009+0000 7f4801659640 1 --2- 172.21.15.40:0/1695436192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0b40d0 0x7f47fc0b4b00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47fc06c650 tx=0x7f47f402a560 comp rx=0 tx=0).stop 2024-04-26T21:35:52.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.009+0000 7f4801659640 1 -- 172.21.15.40:0/1695436192 shutdown_connections 2024-04-26T21:35:52.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.009+0000 7f4801659640 1 --2- 172.21.15.40:0/1695436192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0b40d0 0x7f47fc0b4b00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.009+0000 7f4801659640 1 --2- 172.21.15.40:0/1695436192 >> v2:172.21.15.184:3300/0 conn(0x7f47fc0d1a00 0x7f47fc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.009+0000 7f4801659640 1 -- 172.21.15.40:0/1695436192 >> 172.21.15.40:0/1695436192 conn(0x7f47fc0d4fd0 msgr2=0x7f47fc0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:52.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.009+0000 7f4801659640 1 -- 172.21.15.40:0/1695436192 shutdown_connections 2024-04-26T21:35:52.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.009+0000 7f4801659640 1 -- 172.21.15.40:0/1695436192 wait complete. 2024-04-26T21:35:52.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.010+0000 7f4801659640 1 Processor -- start 2024-04-26T21:35:52.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.010+0000 7f4801659640 1 -- start start 2024-04-26T21:35:52.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f4801659640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47fc0b40d0 0x7f47fc14f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f4801659640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0d1a00 0x7f47fc0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f4801659640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47fc0d2320 0x7f47fc151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f4801659640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc0b5040 con 0x7f47fc0d1a00 2024-04-26T21:35:52.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f4801659640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc162f50 con 0x7f47fc0b40d0 2024-04-26T21:35:52.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f4801659640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47fc0b51c0 con 0x7f47fc0d2320 2024-04-26T21:35:52.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0d1a00 0x7f47fc0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0d1a00 0x7f47fc0c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55594/0 (socket says 172.21.15.40:55594) 2024-04-26T21:35:52.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fa7fc640 1 -- 172.21.15.40:0/3304560095 learned_addr learned my addr 172.21.15.40:0/3304560095 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:52.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47faffd640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47fc0b40d0 0x7f47fc14f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fb7fe640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47fc0d2320 0x7f47fc151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fa7fc640 1 -- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47fc0d2320 msgr2=0x7f47fc151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fa7fc640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47fc0d2320 0x7f47fc151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fa7fc640 1 -- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47fc0b40d0 msgr2=0x7f47fc14f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fa7fc640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47fc0b40d0 0x7f47fc14f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47fa7fc640 1 -- 172.21.15.40:0/3304560095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc0cd190 con 0x7f47fc0d1a00 2024-04-26T21:35:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.011+0000 7f47faffd640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47fc0b40d0 0x7f47fc14f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.012+0000 7f47fa7fc640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0d1a00 0x7f47fc0c86f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c460 tx=0x7f47f000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.012+0000 7f47e37fe640 1 -- 172.21.15.40:0/3304560095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0018070 con 0x7f47fc0d1a00 2024-04-26T21:35:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.012+0000 7f47e37fe640 1 -- 172.21.15.40:0/3304560095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f0007500 con 0x7f47fc0d1a00 2024-04-26T21:35:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.012+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47fc0cd470 con 0x7f47fc0d1a00 2024-04-26T21:35:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.012+0000 7f47e37fe640 1 -- 172.21.15.40:0/3304560095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0004a20 con 0x7f47fc0d1a00 2024-04-26T21:35:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.012+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47fc0cda30 con 0x7f47fc0d1a00 2024-04-26T21:35:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.013+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47fc14fac0 con 0x7f47fc0d1a00 2024-04-26T21:35:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.013+0000 7f47e37fe640 1 -- 172.21.15.40:0/3304560095 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47f001b070 con 0x7f47fc0d1a00 2024-04-26T21:35:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.014+0000 7f47e37fe640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47e806c8f0 0x7f47e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.014+0000 7f47e37fe640 1 -- 172.21.15.40:0/3304560095 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f47f008b200 con 0x7f47fc0d1a00 2024-04-26T21:35:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.014+0000 7f47faffd640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47e806c8f0 0x7f47e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.014+0000 7f47faffd640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47e806c8f0 0x7f47e806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47ec006fd0 tx=0x7f47ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.018+0000 7f47e37fe640 1 -- 172.21.15.40:0/3304560095 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47f0059230 con 0x7f47fc0d1a00 2024-04-26T21:35:52.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.205+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47fc0694f0 con 0x7f47fc0d1a00 2024-04-26T21:35:52.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.206+0000 7f47e37fe640 1 -- 172.21.15.40:0/3304560095 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f47f0058b30 con 0x7f47fc0d1a00 2024-04-26T21:35:52.207 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47e806c8f0 msgr2=0x7f47e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47e806c8f0 0x7f47e806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47ec006fd0 tx=0x7f47ec008040 comp rx=0 tx=0).stop 2024-04-26T21:35:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0d1a00 msgr2=0x7f47fc0c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0d1a00 0x7f47fc0c86f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c460 tx=0x7f47f000c920 comp rx=0 tx=0).stop 2024-04-26T21:35:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 shutdown_connections 2024-04-26T21:35:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47e806c8f0 0x7f47e806edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47fc0d2320 0x7f47fc151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47fc0d1a00 0x7f47fc0c86f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 --2- 172.21.15.40:0/3304560095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47fc0b40d0 0x7f47fc14f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.207+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 >> 172.21.15.40:0/3304560095 conn(0x7f47fc0d4fd0 msgr2=0x7f47fc0b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:52.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.208+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 shutdown_connections 2024-04-26T21:35:52.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.208+0000 7f4801659640 1 -- 172.21.15.40:0/3304560095 wait complete. 2024-04-26T21:35:52.219 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.323+0000 7f13d1fc6640 1 Processor -- start 2024-04-26T21:35:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.323+0000 7f13d1fc6640 1 -- start start 2024-04-26T21:35:52.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.323+0000 7f13d1fc6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc0d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.323+0000 7f13d1fc6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13cc0cdb70 0x7f13cc0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.323+0000 7f13d1fc6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13cc06e580 con 0x7f13cc15f770 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.323+0000 7f13d1fc6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f13cc06f3e0 con 0x7f13cc0cdb70 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.323+0000 7f13d1fc6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc0cd6e0 con 0x7f13cc0b1fd0 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13caffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc0d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13cbfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13cc0cdb70 0x7f13cc0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13cb7fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f13cc15f770 0x7f13cc0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:47154/0 (socket says 172.21.15.40:47154) 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13cb7fe640 1 -- 172.21.15.40:0/2007934053 learned_addr learned my addr 172.21.15.40:0/2007934053 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13caffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc0d1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:46506/0 (socket says 172.21.15.40:46506) 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13ca7fc640 1 -- 172.21.15.40:0/2007934053 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2112376590 0 0) 0x7f13cc06e580 con 0x7f13cc15f770 2024-04-26T21:35:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13ca7fc640 1 -- 172.21.15.40:0/2007934053 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13b4003520 con 0x7f13cc15f770 2024-04-26T21:35:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13caffd640 1 -- 172.21.15.40:0/2007934053 >> v2:172.21.15.184:3300/0 conn(0x7f13cc0cdb70 msgr2=0x7f13cc0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13caffd640 1 --2- 172.21.15.40:0/2007934053 >> v2:172.21.15.184:3300/0 conn(0x7f13cc0cdb70 0x7f13cc0ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13caffd640 1 -- 172.21.15.40:0/2007934053 >> v1:172.21.15.184:6789/0 conn(0x7f13cc15f770 legacy=0x7f13cc0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13caffd640 1 -- 172.21.15.40:0/2007934053 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc06e860 con 0x7f13cc0b1fd0 2024-04-26T21:35:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.324+0000 7f13cbfff640 1 --2- 172.21.15.40:0/2007934053 >> v2:172.21.15.184:3300/0 conn(0x7f13cc0cdb70 0x7f13cc0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:52.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.325+0000 7f13caffd640 1 --2- 172.21.15.40:0/2007934053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc0d1470 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13c00097d0 tx=0x7f13c002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=893e0e286d312987 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.325+0000 7f13ca7fc640 1 -- 172.21.15.40:0/2007934053 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0037070 con 0x7f13cc0b1fd0 2024-04-26T21:35:52.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.325+0000 7f13ca7fc640 1 -- 172.21.15.40:0/2007934053 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c0032440 con 0x7f13cc0b1fd0 2024-04-26T21:35:52.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.325+0000 7f13ca7fc640 1 -- 172.21.15.40:0/2007934053 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0032810 con 0x7f13cc0b1fd0 2024-04-26T21:35:52.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.326+0000 7f13d1fc6640 1 -- 172.21.15.40:0/2007934053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 msgr2=0x7f13cc0d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.326+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/2007934053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc0d1470 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13c00097d0 tx=0x7f13c002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.326+0000 7f13d1fc6640 1 -- 172.21.15.40:0/2007934053 shutdown_connections 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.326+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/2007934053 >> v2:172.21.15.184:3300/0 conn(0x7f13cc0cdb70 0x7f13cc0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.326+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/2007934053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc0d1470 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.326+0000 7f13d1fc6640 1 -- 172.21.15.40:0/2007934053 >> 172.21.15.40:0/2007934053 conn(0x7f13cc0d2e60 msgr2=0x7f13cc0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.326+0000 7f13d1fc6640 1 -- 172.21.15.40:0/2007934053 shutdown_connections 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.326+0000 7f13d1fc6640 1 -- 172.21.15.40:0/2007934053 wait complete. 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.327+0000 7f13d1fc6640 1 Processor -- start 2024-04-26T21:35:52.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.327+0000 7f13d1fc6640 1 -- start start 2024-04-26T21:35:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13d1fc6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc1f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13d1fc6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc0cdb70 0x7f13cc0c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13d1fc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc15f770 0x7f13cc1f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13d1fc6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc0d1b30 con 0x7f13cc15f770 2024-04-26T21:35:52.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13d1fc6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc0d19b0 con 0x7f13cc0cdb70 2024-04-26T21:35:52.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13d1fc6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc0ce540 con 0x7f13cc0b1fd0 2024-04-26T21:35:52.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc15f770 0x7f13cc1f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc15f770 0x7f13cc1f8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55610/0 (socket says 172.21.15.40:55610) 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cbfff640 1 -- 172.21.15.40:0/3237096885 learned_addr learned my addr 172.21.15.40:0/3237096885 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cb7fe640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc1f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13caffd640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc0cdb70 0x7f13cc0c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cbfff640 1 -- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 msgr2=0x7f13cc1f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cbfff640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc1f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cbfff640 1 -- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc0cdb70 msgr2=0x7f13cc0c7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cbfff640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc0cdb70 0x7f13cc0c7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.328+0000 7f13cbfff640 1 -- 172.21.15.40:0/3237096885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc1f9b80 con 0x7f13cc15f770 2024-04-26T21:35:52.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.329+0000 7f13cbfff640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc15f770 0x7f13cc1f8d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009f00 tx=0x7f13c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.329+0000 7f13b3fff640 1 -- 172.21.15.40:0/3237096885 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4018030 con 0x7f13cc15f770 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.329+0000 7f13b3fff640 1 -- 172.21.15.40:0/3237096885 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c400ce50 con 0x7f13cc15f770 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.329+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc0ca260 con 0x7f13cc15f770 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.329+0000 7f13b3fff640 1 -- 172.21.15.40:0/3237096885 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4003bb0 con 0x7f13cc15f770 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.329+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc0c8d10 con 0x7f13cc15f770 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.330+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc0b8d00 con 0x7f13cc15f770 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.330+0000 7f13b3fff640 1 -- 172.21.15.40:0/3237096885 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13c401b070 con 0x7f13cc15f770 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.331+0000 7f13b3fff640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b806c8f0 0x7f13b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.331+0000 7f13b3fff640 1 -- 172.21.15.40:0/3237096885 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f13c408c880 con 0x7f13cc15f770 2024-04-26T21:35:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.331+0000 7f13cb7fe640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b806c8f0 0x7f13b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.332+0000 7f13cb7fe640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b806c8f0 0x7f13b806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f13bc005880 tx=0x7f13bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.335+0000 7f13b3fff640 1 -- 172.21.15.40:0/3237096885 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13cc0b8d00 con 0x7f13cc15f770 2024-04-26T21:35:52.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.519+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f13cc0694f0 con 0x7f13cc15f770 2024-04-26T21:35:52.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.520+0000 7f13b3fff640 1 -- 172.21.15.40:0/3237096885 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f13c4010040 con 0x7f13cc15f770 2024-04-26T21:35:52.521 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b806c8f0 msgr2=0x7f13b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b806c8f0 0x7f13b806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f13bc005880 tx=0x7f13bc009040 comp rx=0 tx=0).stop 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc15f770 msgr2=0x7f13cc1f8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc15f770 0x7f13cc1f8d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009f00 tx=0x7f13c400c490 comp rx=0 tx=0).stop 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 shutdown_connections 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b806c8f0 0x7f13b806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc15f770 0x7f13cc1f8d40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc0cdb70 0x7f13cc0c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.521+0000 7f13d1fc6640 1 --2- 172.21.15.40:0/3237096885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13cc0b1fd0 0x7f13cc1f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.522+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 >> 172.21.15.40:0/3237096885 conn(0x7f13cc0d2e60 msgr2=0x7f13cc0ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.522+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 shutdown_connections 2024-04-26T21:35:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.522+0000 7f13d1fc6640 1 -- 172.21.15.40:0/3237096885 wait complete. 2024-04-26T21:35:52.534 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.636+0000 7f3a0f7bf640 1 Processor -- start 2024-04-26T21:35:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.636+0000 7f3a0f7bf640 1 -- start start 2024-04-26T21:35:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0f7bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815f170 0x7f3a080d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0f7bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3a0815f550 0x7f3a080d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0f7bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a080d34b0 0x7f3a080d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0f7bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3a080c9430 con 0x7f3a080d34b0 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0f7bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3a080d3e20 con 0x7f3a0815f550 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0f7bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a080d1040 con 0x7f3a0815f170 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0dd5c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a080d34b0 0x7f3a080d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0dd5c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a080d34b0 0x7f3a080d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55626/0 (socket says 172.21.15.40:55626) 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0dd5c640 1 -- 172.21.15.40:0/1068324993 learned_addr learned my addr 172.21.15.40:0/1068324993 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0cd5a640 1 --2- 172.21.15.40:0/1068324993 >> v2:172.21.15.184:3300/0 conn(0x7f3a0815f550 0x7f3a080d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0d55b640 1 --2- 172.21.15.40:0/1068324993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815f170 0x7f3a080d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0dd5c640 1 -- 172.21.15.40:0/1068324993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815f170 msgr2=0x7f3a080d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0dd5c640 1 --2- 172.21.15.40:0/1068324993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815f170 0x7f3a080d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0dd5c640 1 -- 172.21.15.40:0/1068324993 >> v2:172.21.15.184:3300/0 conn(0x7f3a0815f550 msgr2=0x7f3a080d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0dd5c640 1 --2- 172.21.15.40:0/1068324993 >> v2:172.21.15.184:3300/0 conn(0x7f3a0815f550 0x7f3a080d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.637+0000 7f3a0dd5c640 1 -- 172.21.15.40:0/1068324993 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a080c9680 con 0x7f3a080d34b0 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.638+0000 7f3a0dd5c640 1 --2- 172.21.15.40:0/1068324993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a080d34b0 0x7f3a080d38e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3a0808e960 tx=0x7f3a0002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfa9de4c94136f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.638+0000 7f39fffff640 1 -- 172.21.15.40:0/1068324993 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00036070 con 0x7f3a080d34b0 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.638+0000 7f39fffff640 1 -- 172.21.15.40:0/1068324993 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a00031600 con 0x7f3a080d34b0 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.638+0000 7f39fffff640 1 -- 172.21.15.40:0/1068324993 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a000319d0 con 0x7f3a080d34b0 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/1068324993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a080d34b0 msgr2=0x7f3a080d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/1068324993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a080d34b0 0x7f3a080d38e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3a0808e960 tx=0x7f3a0002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/1068324993 shutdown_connections 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/1068324993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a080d34b0 0x7f3a080d38e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/1068324993 >> v2:172.21.15.184:3300/0 conn(0x7f3a0815f550 0x7f3a080d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/1068324993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815f170 0x7f3a080d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/1068324993 >> 172.21.15.40:0/1068324993 conn(0x7f3a080d4f60 msgr2=0x7f3a080d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/1068324993 shutdown_connections 2024-04-26T21:35:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.639+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/1068324993 wait complete. 2024-04-26T21:35:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.640+0000 7f3a0f7bf640 1 Processor -- start 2024-04-26T21:35:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.640+0000 7f3a0f7bf640 1 -- start start 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0f7bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a080d34b0 0x7f3a080b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0f7bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a0815f170 0x7f3a081f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0f7bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815d590 0x7f3a0815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0f7bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a080cfc50 con 0x7f3a0815f170 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0f7bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a080c9c40 con 0x7f3a080d34b0 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0f7bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a080cfdd0 con 0x7f3a0815d590 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a0815f170 0x7f3a081f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a0815f170 0x7f3a081f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55640/0 (socket says 172.21.15.40:55640) 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 -- 172.21.15.40:0/3745906664 learned_addr learned my addr 172.21.15.40:0/3745906664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 -- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815d590 msgr2=0x7f3a0815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0dd5c640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815d590 0x7f3a0815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0d55b640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a080d34b0 0x7f3a080b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815d590 0x7f3a0815c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 -- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a080d34b0 msgr2=0x7f3a080b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a080d34b0 0x7f3a080b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 -- 172.21.15.40:0/3745906664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a081f9540 con 0x7f3a0815f170 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.641+0000 7f3a0cd5a640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a0815f170 0x7f3a081f8d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39f800d5f0 tx=0x7f39f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.642+0000 7f39fdffb640 1 -- 172.21.15.40:0/3745906664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80125f0 con 0x7f3a0815f170 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.642+0000 7f39fdffb640 1 -- 172.21.15.40:0/3745906664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f8019070 con 0x7f3a0815f170 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.642+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a081f9820 con 0x7f3a0815f170 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.642+0000 7f39fdffb640 1 -- 172.21.15.40:0/3745906664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80129d0 con 0x7f3a0815f170 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.642+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a081f5a90 con 0x7f3a0815f170 2024-04-26T21:35:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.643+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a080694f0 con 0x7f3a0815f170 2024-04-26T21:35:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.643+0000 7f39fdffb640 1 -- 172.21.15.40:0/3745906664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39f8012b70 con 0x7f3a0815f170 2024-04-26T21:35:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.644+0000 7f39fdffb640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.644+0000 7f39fdffb640 1 -- 172.21.15.40:0/3745906664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f39f808c350 con 0x7f3a0815f170 2024-04-26T21:35:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.644+0000 7f3a0d55b640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.644+0000 7f3a0d55b640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ec06c8f0 0x7f39ec06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a04006fd0 tx=0x7f3a04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.648+0000 7f39fdffb640 1 -- 172.21.15.40:0/3745906664 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39f805a380 con 0x7f3a0815f170 2024-04-26T21:35:52.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.834+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3a080b6c90 con 0x7f3a0815f170 2024-04-26T21:35:52.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.835+0000 7f39fdffb640 1 -- 172.21.15.40:0/3745906664 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f39f8059c80 con 0x7f3a0815f170 2024-04-26T21:35:52.836 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ec06c8f0 msgr2=0x7f39ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ec06c8f0 0x7f39ec06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a04006fd0 tx=0x7f3a04008040 comp rx=0 tx=0).stop 2024-04-26T21:35:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a0815f170 msgr2=0x7f3a081f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a0815f170 0x7f3a081f8d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39f800d5f0 tx=0x7f39f800dab0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 shutdown_connections 2024-04-26T21:35:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a0815d590 0x7f3a0815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a0815f170 0x7f3a081f8d10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 --2- 172.21.15.40:0/3745906664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a080d34b0 0x7f3a080b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 >> 172.21.15.40:0/3745906664 conn(0x7f3a080d4f60 msgr2=0x7f3a0806d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:52.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 shutdown_connections 2024-04-26T21:35:52.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.837+0000 7f3a0f7bf640 1 -- 172.21.15.40:0/3745906664 wait complete. 2024-04-26T21:35:52.848 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.948+0000 7f17ef072640 1 Processor -- start 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.948+0000 7f17ef072640 1 -- start start 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ef072640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17e8162880 0x7f17e80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ef072640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f17e80b5f20 0x7f17e80d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ef072640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17e80b7e50 con 0x7f17e80d39f0 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ef072640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f17e80d3e40 con 0x7f17e8162880 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ef072640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f17e80d0580 con 0x7f17e80b5f20 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ece0e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17e8162880 0x7f17e80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ed60f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f17e80d39f0 0x7f17e80b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54676/0 (socket says 172.21.15.40:54676) 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ed60f640 1 -- 172.21.15.40:0/2227263554 learned_addr learned my addr 172.21.15.40:0/2227263554 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17e7fff640 1 --2- 172.21.15.40:0/2227263554 >> v2:172.21.15.184:3300/0 conn(0x7f17e80b5f20 0x7f17e80d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ece0e640 1 -- 172.21.15.40:0/2227263554 >> v2:172.21.15.184:3300/0 conn(0x7f17e80b5f20 msgr2=0x7f17e80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ece0e640 1 --2- 172.21.15.40:0/2227263554 >> v2:172.21.15.184:3300/0 conn(0x7f17e80b5f20 0x7f17e80d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ece0e640 1 -- 172.21.15.40:0/2227263554 >> v1:172.21.15.40:6789/0 conn(0x7f17e80d39f0 legacy=0x7f17e80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17e77fe640 1 -- 172.21.15.40:0/2227263554 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206361715 0 0) 0x7f17e80b7e50 con 0x7f17e80d39f0 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.949+0000 7f17ece0e640 1 -- 172.21.15.40:0/2227263554 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e80d1800 con 0x7f17e8162880 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.950+0000 7f17ece0e640 1 --2- 172.21.15.40:0/2227263554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e8162880 0x7f17e80d2cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17e808d3d0 tx=0x7f17dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d017e462618bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.950+0000 7f17e77fe640 1 -- 172.21.15.40:0/2227263554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc037070 con 0x7f17e8162880 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.950+0000 7f17e77fe640 1 -- 172.21.15.40:0/2227263554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17dc032440 con 0x7f17e8162880 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.950+0000 7f17e77fe640 1 -- 172.21.15.40:0/2227263554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc032810 con 0x7f17e8162880 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.950+0000 7f17ef072640 1 -- 172.21.15.40:0/2227263554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e8162880 msgr2=0x7f17e80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.950+0000 7f17ef072640 1 --2- 172.21.15.40:0/2227263554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e8162880 0x7f17e80d2cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17e808d3d0 tx=0x7f17dc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.951+0000 7f17ef072640 1 -- 172.21.15.40:0/2227263554 shutdown_connections 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.951+0000 7f17ef072640 1 --2- 172.21.15.40:0/2227263554 >> v2:172.21.15.184:3300/0 conn(0x7f17e80b5f20 0x7f17e80d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.951+0000 7f17ef072640 1 --2- 172.21.15.40:0/2227263554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e8162880 0x7f17e80d2cb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.951+0000 7f17ef072640 1 -- 172.21.15.40:0/2227263554 >> 172.21.15.40:0/2227263554 conn(0x7f17e80d4f60 msgr2=0x7f17e80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.951+0000 7f17ef072640 1 -- 172.21.15.40:0/2227263554 shutdown_connections 2024-04-26T21:35:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.951+0000 7f17ef072640 1 -- 172.21.15.40:0/2227263554 wait complete. 2024-04-26T21:35:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.952+0000 7f17ef072640 1 Processor -- start 2024-04-26T21:35:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.952+0000 7f17ef072640 1 -- start start 2024-04-26T21:35:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ef072640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e80b5f20 0x7f17e80cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ef072640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17e80d39f0 0x7f17e8160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ef072640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17e8162880 0x7f17e8160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ef072640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f17e80d0d90 con 0x7f17e80b5f20 2024-04-26T21:35:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ef072640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17e80d0ab0 con 0x7f17e8162880 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ef072640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17e8162c60 con 0x7f17e80d39f0 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e80b5f20 0x7f17e80cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e80b5f20 0x7f17e80cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55664/0 (socket says 172.21.15.40:55664) 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 -- 172.21.15.40:0/3816876385 learned_addr learned my addr 172.21.15.40:0/3816876385 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 -- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17e80d39f0 msgr2=0x7f17e8160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17e80d39f0 0x7f17e8160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 -- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17e8162880 msgr2=0x7f17e8160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17e8162880 0x7f17e8160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 -- 172.21.15.40:0/3816876385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e81524b0 con 0x7f17e80b5f20 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ece0e640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e80b5f20 0x7f17e80cb680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f17dc000c00 tx=0x7f17dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17e57fa640 1 -- 172.21.15.40:0/3816876385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc037040 con 0x7f17e80b5f20 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17e57fa640 1 -- 172.21.15.40:0/3816876385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17dc042070 con 0x7f17e80b5f20 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e8152790 con 0x7f17e80b5f20 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17e57fa640 1 -- 172.21.15.40:0/3816876385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc030bd0 con 0x7f17e80b5f20 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.953+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e8152d50 con 0x7f17e80b5f20 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.954+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e8075d70 con 0x7f17e80b5f20 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.955+0000 7f17e57fa640 1 -- 172.21.15.40:0/3816876385 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17dc047460 con 0x7f17e80b5f20 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.955+0000 7f17e57fa640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17cc06c8f0 0x7f17cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.955+0000 7f17e57fa640 1 -- 172.21.15.40:0/3816876385 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f17dc0acb20 con 0x7f17e80b5f20 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.956+0000 7f17e7fff640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17cc06c8f0 0x7f17cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.956+0000 7f17e7fff640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17cc06c8f0 0x7f17cc06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006fd0 tx=0x7f17d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:52.960+0000 7f17e57fa640 1 -- 172.21.15.40:0/3816876385 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17e8075d70 con 0x7f17e80b5f20 2024-04-26T21:35:53.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.147+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17e80694f0 con 0x7f17e80b5f20 2024-04-26T21:35:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.148+0000 7f17e57fa640 1 -- 172.21.15.40:0/3816876385 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7f17dc032e20 con 0x7f17e80b5f20 2024-04-26T21:35:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:53.149 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17cc06c8f0 msgr2=0x7f17cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17cc06c8f0 0x7f17cc06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006fd0 tx=0x7f17d4008040 comp rx=0 tx=0).stop 2024-04-26T21:35:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e80b5f20 msgr2=0x7f17e80cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e80b5f20 0x7f17e80cb680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f17dc000c00 tx=0x7f17dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 shutdown_connections 2024-04-26T21:35:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17cc06c8f0 0x7f17cc06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17e8162880 0x7f17e8160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17e80d39f0 0x7f17e8160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 --2- 172.21.15.40:0/3816876385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e80b5f20 0x7f17e80cb680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 >> 172.21.15.40:0/3816876385 conn(0x7f17e80d4f60 msgr2=0x7f17e8161480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 shutdown_connections 2024-04-26T21:35:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.150+0000 7f17ef072640 1 -- 172.21.15.40:0/3816876385 wait complete. 2024-04-26T21:35:53.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:53.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:53.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.265+0000 7f1e521b7640 1 Processor -- start 2024-04-26T21:35:53.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.265+0000 7f1e521b7640 1 -- start start 2024-04-26T21:35:53.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e521b7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e4c156460 0x7f1e4c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e521b7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c06fea0 0x7f1e4c072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e521b7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4c156840 con 0x7f1e4c0cba10 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e521b7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4c0cbf20 con 0x7f1e4c156460 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e521b7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4c0cc200 con 0x7f1e4c06fea0 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e4c156460 0x7f1e4c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e4c156460 0x7f1e4c0cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55676/0 (socket says 172.21.15.40:55676) 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c06fea0 0x7f1e4c072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4b7fe640 1 -- 172.21.15.40:0/3440037149 learned_addr learned my addr 172.21.15.40:0/3440037149 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4affd640 1 --1- 172.21.15.40:0/3440037149 >> v1:172.21.15.184:6789/0 conn(0x7f1e4c0cba10 0x7f1e4c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:47158/0 (socket says 172.21.15.40:47158) 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4b7fe640 1 -- 172.21.15.40:0/3440037149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c06fea0 msgr2=0x7f1e4c072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4b7fe640 1 --2- 172.21.15.40:0/3440037149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c06fea0 0x7f1e4c072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4b7fe640 1 -- 172.21.15.40:0/3440037149 >> v1:172.21.15.184:6789/0 conn(0x7f1e4c0cba10 legacy=0x7f1e4c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.266+0000 7f1e4b7fe640 1 -- 172.21.15.40:0/3440037149 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c0cc4e0 con 0x7f1e4c156460 2024-04-26T21:35:53.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.267+0000 7f1e4b7fe640 1 --2- 172.21.15.40:0/3440037149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c156460 0x7f1e4c0cb4d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009870 tx=0x7f1e3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79116802e77c0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.268+0000 7f1e4a7fc640 1 -- 172.21.15.40:0/3440037149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c037070 con 0x7f1e4c156460 2024-04-26T21:35:53.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.268+0000 7f1e4a7fc640 1 -- 172.21.15.40:0/3440037149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c032440 con 0x7f1e4c156460 2024-04-26T21:35:53.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.268+0000 7f1e4a7fc640 1 -- 172.21.15.40:0/3440037149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c032810 con 0x7f1e4c156460 2024-04-26T21:35:53.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.269+0000 7f1e521b7640 1 -- 172.21.15.40:0/3440037149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c156460 msgr2=0x7f1e4c0cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.269+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3440037149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c156460 0x7f1e4c0cb4d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009870 tx=0x7f1e3c02a270 comp rx=0 tx=0).stop 2024-04-26T21:35:53.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.269+0000 7f1e521b7640 1 -- 172.21.15.40:0/3440037149 shutdown_connections 2024-04-26T21:35:53.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.269+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3440037149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c06fea0 0x7f1e4c072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.269+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3440037149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c156460 0x7f1e4c0cb4d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.269+0000 7f1e521b7640 1 -- 172.21.15.40:0/3440037149 >> 172.21.15.40:0/3440037149 conn(0x7f1e4c0d0be0 msgr2=0x7f1e4c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:53.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.269+0000 7f1e521b7640 1 -- 172.21.15.40:0/3440037149 shutdown_connections 2024-04-26T21:35:53.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.269+0000 7f1e521b7640 1 -- 172.21.15.40:0/3440037149 wait complete. 2024-04-26T21:35:53.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.270+0000 7f1e521b7640 1 Processor -- start 2024-04-26T21:35:53.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e521b7640 1 -- start start 2024-04-26T21:35:53.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e521b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c06fea0 0x7f1e4c0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e521b7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e4c0cba10 0x7f1e4c0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e521b7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c156460 0x7f1e4c1f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e521b7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c06e260 con 0x7f1e4c06fea0 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e521b7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c072700 con 0x7f1e4c0cba10 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e521b7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4c06e3e0 con 0x7f1e4c156460 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e4b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c06fea0 0x7f1e4c0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e4b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c06fea0 0x7f1e4c0c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55682/0 (socket says 172.21.15.40:55682) 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e4b7fe640 1 -- 172.21.15.40:0/3912324113 learned_addr learned my addr 172.21.15.40:0/3912324113 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e4bfff640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c156460 0x7f1e4c1f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.271+0000 7f1e4affd640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e4c0cba10 0x7f1e4c0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e4b7fe640 1 -- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c156460 msgr2=0x7f1e4c1f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e4b7fe640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c156460 0x7f1e4c1f0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e4b7fe640 1 -- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e4c0cba10 msgr2=0x7f1e4c0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e4b7fe640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e4c0cba10 0x7f1e4c0ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e4b7fe640 1 -- 172.21.15.40:0/3912324113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c0c5a30 con 0x7f1e4c06fea0 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e4b7fe640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c06fea0 0x7f1e4c0c73e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c000c00 tx=0x7f1e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e33fff640 1 -- 172.21.15.40:0/3912324113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c037040 con 0x7f1e4c06fea0 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e33fff640 1 -- 172.21.15.40:0/3912324113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c042070 con 0x7f1e4c06fea0 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e33fff640 1 -- 172.21.15.40:0/3912324113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c030880 con 0x7f1e4c06fea0 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c0c9850 con 0x7f1e4c06fea0 2024-04-26T21:35:53.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.272+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c0c9e10 con 0x7f1e4c06fea0 2024-04-26T21:35:53.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.273+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c0c9680 con 0x7f1e4c06fea0 2024-04-26T21:35:53.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.274+0000 7f1e33fff640 1 -- 172.21.15.40:0/3912324113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e3c030a20 con 0x7f1e4c06fea0 2024-04-26T21:35:53.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.274+0000 7f1e33fff640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e3806c8f0 0x7f1e3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.274+0000 7f1e33fff640 1 -- 172.21.15.40:0/3912324113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1e3c0ac5c0 con 0x7f1e4c06fea0 2024-04-26T21:35:53.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.274+0000 7f1e4affd640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e3806c8f0 0x7f1e3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.275+0000 7f1e4affd640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e3806c8f0 0x7f1e3806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c14b210 tx=0x7f1e40007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.278+0000 7f1e33fff640 1 -- 172.21.15.40:0/3912324113 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e3c07a030 con 0x7f1e4c06fea0 2024-04-26T21:35:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.462+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e4c0694f0 con 0x7f1e4c06fea0 2024-04-26T21:35:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.463+0000 7f1e33fff640 1 -- 172.21.15.40:0/3912324113 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f1e3c07e330 con 0x7f1e4c06fea0 2024-04-26T21:35:53.464 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e3806c8f0 msgr2=0x7f1e3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e3806c8f0 0x7f1e3806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c14b210 tx=0x7f1e40007e80 comp rx=0 tx=0).stop 2024-04-26T21:35:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c06fea0 msgr2=0x7f1e4c0c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c06fea0 0x7f1e4c0c73e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c000c00 tx=0x7f1e3c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 shutdown_connections 2024-04-26T21:35:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e3806c8f0 0x7f1e3806edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e4c156460 0x7f1e4c1f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e4c0cba10 0x7f1e4c0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 --2- 172.21.15.40:0/3912324113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4c06fea0 0x7f1e4c0c73e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 >> 172.21.15.40:0/3912324113 conn(0x7f1e4c0d0be0 msgr2=0x7f1e4c06e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 shutdown_connections 2024-04-26T21:35:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.465+0000 7f1e521b7640 1 -- 172.21.15.40:0/3912324113 wait complete. 2024-04-26T21:35:53.477 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:53.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.580+0000 7f07ca1c1640 1 Processor -- start 2024-04-26T21:35:53.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.580+0000 7f07ca1c1640 1 -- start start 2024-04-26T21:35:53.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07ca1c1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f07c4162a20 0x7f07c40d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07ca1c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c40d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07ca1c1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c40c8bc0 con 0x7f07c40d2ed0 2024-04-26T21:35:53.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07ca1c1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f07c40d07a0 con 0x7f07c4162a20 2024-04-26T21:35:53.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07ca1c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f07c40d09f0 con 0x7f07c40d1c40 2024-04-26T21:35:53.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07c3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f07c40d2ed0 0x7f07c40d3360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54688/0 (socket says 172.21.15.40:54688) 2024-04-26T21:35:53.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07c3fff640 1 -- 172.21.15.40:0/1507974700 learned_addr learned my addr 172.21.15.40:0/1507974700 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07c37fe640 1 --2- 172.21.15.40:0/1507974700 >> v2:172.21.15.184:3300/0 conn(0x7f07c4162a20 0x7f07c40d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.581+0000 7f07c2ffd640 1 --2- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c40d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.582+0000 7f07c27fc640 1 -- 172.21.15.40:0/1507974700 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2912104965 0 0) 0x7f07c40c8bc0 con 0x7f07c40d2ed0 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.582+0000 7f07c27fc640 1 -- 172.21.15.40:0/1507974700 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07ac003520 con 0x7f07c40d2ed0 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.582+0000 7f07c37fe640 1 -- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 msgr2=0x7f07c40d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.582+0000 7f07c37fe640 1 --2- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c40d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.582+0000 7f07c37fe640 1 -- 172.21.15.40:0/1507974700 >> v1:172.21.15.40:6789/0 conn(0x7f07c40d2ed0 legacy=0x7f07c40d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.582+0000 7f07c37fe640 1 -- 172.21.15.40:0/1507974700 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c40d0cd0 con 0x7f07c4162a20 2024-04-26T21:35:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.582+0000 7f07c2ffd640 1 --2- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c40d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:53.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.582+0000 7f07c37fe640 1 --2- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 0x7f07c40d1700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f07b40097a0 tx=0x7f07b402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122b36f55d9dd7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07c27fc640 1 -- 172.21.15.40:0/1507974700 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4037070 con 0x7f07c4162a20 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07c27fc640 1 -- 172.21.15.40:0/1507974700 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b402ae50 con 0x7f07c4162a20 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07c27fc640 1 -- 172.21.15.40:0/1507974700 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4032640 con 0x7f07c4162a20 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07ca1c1640 1 -- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 msgr2=0x7f07c40d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 0x7f07c40d1700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f07b40097a0 tx=0x7f07b402a100 comp rx=0 tx=0).stop 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07ca1c1640 1 -- 172.21.15.40:0/1507974700 shutdown_connections 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c40d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/1507974700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 0x7f07c40d1700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07ca1c1640 1 -- 172.21.15.40:0/1507974700 >> 172.21.15.40:0/1507974700 conn(0x7f07c40d4fd0 msgr2=0x7f07c40d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07ca1c1640 1 -- 172.21.15.40:0/1507974700 shutdown_connections 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.583+0000 7f07ca1c1640 1 -- 172.21.15.40:0/1507974700 wait complete. 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.584+0000 7f07ca1c1640 1 Processor -- start 2024-04-26T21:35:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07ca1c1640 1 -- start start 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07ca1c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c4150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07ca1c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c40d2ed0 0x7f07c414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07ca1c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 0x7f07c40b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07ca1c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f07c40cbd90 con 0x7f07c40d2ed0 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07ca1c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07c40d0fb0 con 0x7f07c4162a20 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07ca1c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f07c4162e00 con 0x7f07c40d1c40 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c40d2ed0 0x7f07c414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c40d2ed0 0x7f07c414db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55698/0 (socket says 172.21.15.40:55698) 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.585+0000 7f07c2ffd640 1 -- 172.21.15.40:0/2743264258 learned_addr learned my addr 172.21.15.40:0/2743264258 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c3fff640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 0x7f07c40b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c37fe640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c4150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c2ffd640 1 -- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 msgr2=0x7f07c4150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c2ffd640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c4150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c2ffd640 1 -- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 msgr2=0x7f07c40b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c2ffd640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 0x7f07c40b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c2ffd640 1 -- 172.21.15.40:0/2743264258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c41513b0 con 0x7f07c40d2ed0 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c37fe640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c4150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c3fff640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 0x7f07c40b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07c2ffd640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c40d2ed0 0x7f07c414db10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e990 tx=0x7f07b800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07abfff640 1 -- 172.21.15.40:0/2743264258 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8017070 con 0x7f07c40d2ed0 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07abfff640 1 -- 172.21.15.40:0/2743264258 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b800c9e0 con 0x7f07c40d2ed0 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c414e920 con 0x7f07c40d2ed0 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07abfff640 1 -- 172.21.15.40:0/2743264258 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b800cdb0 con 0x7f07c40d2ed0 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.586+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c414eee0 con 0x7f07c40d2ed0 2024-04-26T21:35:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.587+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c40694f0 con 0x7f07c40d2ed0 2024-04-26T21:35:53.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.587+0000 7f07abfff640 1 -- 172.21.15.40:0/2743264258 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07b801a070 con 0x7f07c40d2ed0 2024-04-26T21:35:53.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.588+0000 7f07abfff640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f07b006c920 0x7f07b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.588+0000 7f07abfff640 1 -- 172.21.15.40:0/2743264258 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f07b808be00 con 0x7f07c40d2ed0 2024-04-26T21:35:53.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.588+0000 7f07c37fe640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f07b006c920 0x7f07b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.589+0000 7f07c37fe640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f07b006c920 0x7f07b006ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f07b4034000 tx=0x7f07b402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.592+0000 7f07abfff640 1 -- 172.21.15.40:0/2743264258 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07b8059e30 con 0x7f07c40d2ed0 2024-04-26T21:35:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.778+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f07c4161130 con 0x7f07c40d2ed0 2024-04-26T21:35:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.779+0000 7f07abfff640 1 -- 172.21.15.40:0/2743264258 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f07b8059730 con 0x7f07c40d2ed0 2024-04-26T21:35:53.780 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.780+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f07b006c920 msgr2=0x7f07b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.780+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f07b006c920 0x7f07b006ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f07b4034000 tx=0x7f07b402e000 comp rx=0 tx=0).stop 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.780+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c40d2ed0 msgr2=0x7f07c414db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.780+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c40d2ed0 0x7f07c414db10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e990 tx=0x7f07b800ee50 comp rx=0 tx=0).stop 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.781+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 shutdown_connections 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.781+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f07b006c920 0x7f07b006ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.781+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4162a20 0x7f07c40b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.781+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c40d2ed0 0x7f07c414db10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.781+0000 7f07ca1c1640 1 --2- 172.21.15.40:0/2743264258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07c40d1c40 0x7f07c4150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.781+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 >> 172.21.15.40:0/2743264258 conn(0x7f07c40d4fd0 msgr2=0x7f07c4160740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.781+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 shutdown_connections 2024-04-26T21:35:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.781+0000 7f07ca1c1640 1 -- 172.21.15.40:0/2743264258 wait complete. 2024-04-26T21:35:53.793 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.896+0000 7f71dfe64640 1 Processor -- start 2024-04-26T21:35:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.896+0000 7f71dfe64640 1 -- start start 2024-04-26T21:35:53.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71dfe64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71d80d1a00 0x7f71d80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71dfe64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71d80b40d0 0x7f71d80b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71dfe64640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d80d0a10 con 0x7f71d80d2320 2024-04-26T21:35:53.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71dfe64640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f71d80d0cf0 con 0x7f71d80b40d0 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71dfe64640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f71d80d0fd0 con 0x7f71d80d1a00 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71de401640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71d80b40d0 0x7f71d80b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71de401640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71d80b40d0 0x7f71d80b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55700/0 (socket says 172.21.15.40:55700) 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71de401640 1 -- 172.21.15.40:0/1685950521 learned_addr learned my addr 172.21.15.40:0/1685950521 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71ddc00640 1 --2- 172.21.15.40:0/1685950521 >> v2:172.21.15.184:3300/0 conn(0x7f71d80d1a00 0x7f71d80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71de401640 1 -- 172.21.15.40:0/1685950521 >> v2:172.21.15.184:3300/0 conn(0x7f71d80d1a00 msgr2=0x7f71d80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71de401640 1 --2- 172.21.15.40:0/1685950521 >> v2:172.21.15.184:3300/0 conn(0x7f71d80d1a00 0x7f71d80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71de401640 1 -- 172.21.15.40:0/1685950521 >> v1:172.21.15.184:6789/0 conn(0x7f71d80d2320 legacy=0x7f71d81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.897+0000 7f71de401640 1 -- 172.21.15.40:0/1685950521 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d80d40a0 con 0x7f71d80b40d0 2024-04-26T21:35:53.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.898+0000 7f71de401640 1 --2- 172.21.15.40:0/1685950521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80b40d0 0x7f71d80b4b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71d806c650 tx=0x7f71c802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e16258f67a0a0ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.898+0000 7f71dcbfe640 1 -- 172.21.15.40:0/1685950521 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8037070 con 0x7f71d80b40d0 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.898+0000 7f71dcbfe640 1 -- 172.21.15.40:0/1685950521 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c802ae20 con 0x7f71d80b40d0 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.898+0000 7f71dcbfe640 1 -- 172.21.15.40:0/1685950521 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8032660 con 0x7f71d80b40d0 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.899+0000 7f71dfe64640 1 -- 172.21.15.40:0/1685950521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80b40d0 msgr2=0x7f71d80b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.899+0000 7f71dfe64640 1 --2- 172.21.15.40:0/1685950521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80b40d0 0x7f71d80b4b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71d806c650 tx=0x7f71c802a110 comp rx=0 tx=0).stop 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.899+0000 7f71dfe64640 1 -- 172.21.15.40:0/1685950521 shutdown_connections 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.899+0000 7f71dfe64640 1 --2- 172.21.15.40:0/1685950521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80b40d0 0x7f71d80b4b00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.899+0000 7f71dfe64640 1 --2- 172.21.15.40:0/1685950521 >> v2:172.21.15.184:3300/0 conn(0x7f71d80d1a00 0x7f71d80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.899+0000 7f71dfe64640 1 -- 172.21.15.40:0/1685950521 >> 172.21.15.40:0/1685950521 conn(0x7f71d80d4fd0 msgr2=0x7f71d80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.899+0000 7f71dfe64640 1 -- 172.21.15.40:0/1685950521 shutdown_connections 2024-04-26T21:35:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.899+0000 7f71dfe64640 1 -- 172.21.15.40:0/1685950521 wait complete. 2024-04-26T21:35:53.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.900+0000 7f71dfe64640 1 Processor -- start 2024-04-26T21:35:53.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.900+0000 7f71dfe64640 1 -- start start 2024-04-26T21:35:53.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71dfe64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80b40d0 0x7f71d80b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71dfe64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80d1a00 0x7f71d80cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71dfe64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80d2320 0x7f71d814f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71dfe64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f71d80b5040 con 0x7f71d80d2320 2024-04-26T21:35:53.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71dfe64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8162f50 con 0x7f71d80b40d0 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71dfe64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71d80b51c0 con 0x7f71d80d1a00 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80d2320 0x7f71d814f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80d2320 0x7f71d814f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55702/0 (socket says 172.21.15.40:55702) 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 -- 172.21.15.40:0/2247587792 learned_addr learned my addr 172.21.15.40:0/2247587792 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71ddc00640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80b40d0 0x7f71d80b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71dd3ff640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80d1a00 0x7f71d80cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 -- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80d1a00 msgr2=0x7f71d80cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80d1a00 0x7f71d80cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 -- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80b40d0 msgr2=0x7f71d80b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80b40d0 0x7f71d80b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 -- 172.21.15.40:0/2247587792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d80b95e0 con 0x7f71d80d2320 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71ddc00640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80b40d0 0x7f71d80b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71dd3ff640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80d1a00 0x7f71d80cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.901+0000 7f71de401640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80d2320 0x7f71d814f480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f71c802a150 tx=0x7f71c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.902+0000 7f71d67fc640 1 -- 172.21.15.40:0/2247587792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8041070 con 0x7f71d80d2320 2024-04-26T21:35:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.902+0000 7f71d67fc640 1 -- 172.21.15.40:0/2247587792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c803c670 con 0x7f71d80d2320 2024-04-26T21:35:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.902+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d80b98c0 con 0x7f71d80d2320 2024-04-26T21:35:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.902+0000 7f71d67fc640 1 -- 172.21.15.40:0/2247587792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8037040 con 0x7f71d80d2320 2024-04-26T21:35:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.902+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d80b9e80 con 0x7f71d80d2320 2024-04-26T21:35:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.903+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d80694f0 con 0x7f71d80d2320 2024-04-26T21:35:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.903+0000 7f71d67fc640 1 -- 172.21.15.40:0/2247587792 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71c8044070 con 0x7f71d80d2320 2024-04-26T21:35:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.903+0000 7f71d67fc640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c406c8f0 0x7f71c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.904+0000 7f71d67fc640 1 -- 172.21.15.40:0/2247587792 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f71c80ac810 con 0x7f71d80d2320 2024-04-26T21:35:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.904+0000 7f71ddc00640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c406c8f0 0x7f71c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.904+0000 7f71ddc00640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c406c8f0 0x7f71c406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f71d0007330 tx=0x7f71d000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:53.908+0000 7f71d67fc640 1 -- 172.21.15.40:0/2247587792 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71c807a8c0 con 0x7f71d80d2320 2024-04-26T21:35:54.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.093+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71d80b37c0 con 0x7f71d80d2320 2024-04-26T21:35:54.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.094+0000 7f71d67fc640 1 -- 172.21.15.40:0/2247587792 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f71c802ae20 con 0x7f71d80d2320 2024-04-26T21:35:54.095 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c406c8f0 msgr2=0x7f71c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c406c8f0 0x7f71c406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f71d0007330 tx=0x7f71d000b040 comp rx=0 tx=0).stop 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80d2320 msgr2=0x7f71d814f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80d2320 0x7f71d814f480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f71c802a150 tx=0x7f71c80046c0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 shutdown_connections 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c406c8f0 0x7f71c406edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71d80d2320 0x7f71d814f480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80d1a00 0x7f71d80cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 --2- 172.21.15.40:0/2247587792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80b40d0 0x7f71d80b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 >> 172.21.15.40:0/2247587792 conn(0x7f71d80d4fd0 msgr2=0x7f71d806cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:54.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 shutdown_connections 2024-04-26T21:35:54.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.096+0000 7f71dfe64640 1 -- 172.21.15.40:0/2247587792 wait complete. 2024-04-26T21:35:54.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:54.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.213+0000 7f6feaacf640 1 Processor -- start 2024-04-26T21:35:54.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.213+0000 7f6feaacf640 1 -- start start 2024-04-26T21:35:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6feaacf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6fe415f170 0x7f6fe40d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6feaacf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe415f550 0x7f6fe40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6feaacf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6fe40d34b0 0x7f6fe40d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6feaacf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6fe40c9430 con 0x7f6fe415f170 2024-04-26T21:35:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6feaacf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6fe40d3e20 con 0x7f6fe40d34b0 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6feaacf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe40d1040 con 0x7f6fe415f550 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6fe886b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6fe415f170 0x7f6fe40d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6fe886b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6fe415f170 0x7f6fe40d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55706/0 (socket says 172.21.15.40:55706) 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6fe886b640 1 -- 172.21.15.40:0/857398392 learned_addr learned my addr 172.21.15.40:0/857398392 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6fe906c640 1 --2- 172.21.15.40:0/857398392 >> v2:172.21.15.184:3300/0 conn(0x7f6fe40d34b0 0x7f6fe40d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6fe3fff640 1 --2- 172.21.15.40:0/857398392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe415f550 0x7f6fe40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.214+0000 7f6fe886b640 1 -- 172.21.15.40:0/857398392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe415f550 msgr2=0x7f6fe40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe886b640 1 --2- 172.21.15.40:0/857398392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe415f550 0x7f6fe40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe886b640 1 -- 172.21.15.40:0/857398392 >> v2:172.21.15.184:3300/0 conn(0x7f6fe40d34b0 msgr2=0x7f6fe40d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe886b640 1 --2- 172.21.15.40:0/857398392 >> v2:172.21.15.184:3300/0 conn(0x7f6fe40d34b0 0x7f6fe40d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe886b640 1 -- 172.21.15.40:0/857398392 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe40c9680 con 0x7f6fe415f170 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe3fff640 1 --2- 172.21.15.40:0/857398392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe415f550 0x7f6fe40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe906c640 1 --2- 172.21.15.40:0/857398392 >> v2:172.21.15.184:3300/0 conn(0x7f6fe40d34b0 0x7f6fe40d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe886b640 1 --2- 172.21.15.40:0/857398392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe40d5a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6fe408f770 tx=0x7f6fdc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7059d862f3b4f11b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe37fe640 1 -- 172.21.15.40:0/857398392 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc037070 con 0x7f6fe415f170 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe37fe640 1 -- 172.21.15.40:0/857398392 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc032440 con 0x7f6fe415f170 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.215+0000 7f6fe37fe640 1 -- 172.21.15.40:0/857398392 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc032810 con 0x7f6fe415f170 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 -- 172.21.15.40:0/857398392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 msgr2=0x7f6fe40d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 --2- 172.21.15.40:0/857398392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe40d5a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6fe408f770 tx=0x7f6fdc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 -- 172.21.15.40:0/857398392 shutdown_connections 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 --2- 172.21.15.40:0/857398392 >> v2:172.21.15.184:3300/0 conn(0x7f6fe40d34b0 0x7f6fe40d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 --2- 172.21.15.40:0/857398392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe415f550 0x7f6fe40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 --2- 172.21.15.40:0/857398392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe40d5a80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 -- 172.21.15.40:0/857398392 >> 172.21.15.40:0/857398392 conn(0x7f6fe40d4f60 msgr2=0x7f6fe40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 -- 172.21.15.40:0/857398392 shutdown_connections 2024-04-26T21:35:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.216+0000 7f6feaacf640 1 -- 172.21.15.40:0/857398392 wait complete. 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.217+0000 7f6feaacf640 1 Processor -- start 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.217+0000 7f6feaacf640 1 -- start start 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6feaacf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fe40d34b0 0x7f6fe41f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6feaacf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe41f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6feaacf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe4150010 0x7f6fe41504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6feaacf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe40cfc50 con 0x7f6fe415f170 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6feaacf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe40c9c40 con 0x7f6fe40d34b0 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6feaacf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe40cfdd0 con 0x7f6fe4150010 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe41f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe41f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55716/0 (socket says 172.21.15.40:55716) 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe3fff640 1 -- 172.21.15.40:0/448254599 learned_addr learned my addr 172.21.15.40:0/448254599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe886b640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fe40d34b0 0x7f6fe41f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe906c640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe4150010 0x7f6fe41504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe3fff640 1 -- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe4150010 msgr2=0x7f6fe41504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe3fff640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe4150010 0x7f6fe41504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe3fff640 1 -- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fe40d34b0 msgr2=0x7f6fe41f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe3fff640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fe40d34b0 0x7f6fe41f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.218+0000 7f6fe3fff640 1 -- 172.21.15.40:0/448254599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe4150e00 con 0x7f6fe415f170 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.219+0000 7f6fe906c640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe4150010 0x7f6fe41504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.219+0000 7f6fe3fff640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe41f4a00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400ce00 tx=0x7f6fd400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.219+0000 7f6fe17fa640 1 -- 172.21.15.40:0/448254599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400f890 con 0x7f6fe415f170 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.219+0000 7f6fe17fa640 1 -- 172.21.15.40:0/448254599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd400fa30 con 0x7f6fe415f170 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.219+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe41510e0 con 0x7f6fe415f170 2024-04-26T21:35:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.219+0000 7f6fe17fa640 1 -- 172.21.15.40:0/448254599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400fe40 con 0x7f6fe415f170 2024-04-26T21:35:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.219+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe41516a0 con 0x7f6fe415f170 2024-04-26T21:35:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.220+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe40694f0 con 0x7f6fe415f170 2024-04-26T21:35:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.220+0000 7f6fe17fa640 1 -- 172.21.15.40:0/448254599 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fd4018400 con 0x7f6fe415f170 2024-04-26T21:35:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.221+0000 7f6fe17fa640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6fc406c8f0 0x7f6fc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.221+0000 7f6fe17fa640 1 -- 172.21.15.40:0/448254599 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6fd408ccd0 con 0x7f6fe415f170 2024-04-26T21:35:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.221+0000 7f6fe886b640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6fc406c8f0 0x7f6fc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.222+0000 7f6fe886b640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6fc406c8f0 0x7f6fc406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc034000 tx=0x7f6fdc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.225+0000 7f6fe17fa640 1 -- 172.21.15.40:0/448254599 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6fd4014030 con 0x7f6fe415f170 2024-04-26T21:35:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.413+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6fe4074590 con 0x7f6fe415f170 2024-04-26T21:35:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.413+0000 7f6fe17fa640 1 -- 172.21.15.40:0/448254599 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f6fd400ae20 con 0x7f6fe415f170 2024-04-26T21:35:54.414 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6fc406c8f0 msgr2=0x7f6fc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6fc406c8f0 0x7f6fc406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc034000 tx=0x7f6fdc02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 msgr2=0x7f6fe41f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe41f4a00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400ce00 tx=0x7f6fd400a650 comp rx=0 tx=0).stop 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 shutdown_connections 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6fc406c8f0 0x7f6fc406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fe4150010 0x7f6fe41504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe415f170 0x7f6fe41f4a00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 --2- 172.21.15.40:0/448254599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fe40d34b0 0x7f6fe41f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 >> 172.21.15.40:0/448254599 conn(0x7f6fe40d4f60 msgr2=0x7f6fe40d5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 shutdown_connections 2024-04-26T21:35:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.415+0000 7f6feaacf640 1 -- 172.21.15.40:0/448254599 wait complete. 2024-04-26T21:35:54.427 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:54.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.527+0000 7fadb2d71640 1 Processor -- start 2024-04-26T21:35:54.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.527+0000 7fadb2d71640 1 -- start start 2024-04-26T21:35:54.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadb2d71640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fadac06c950 0x7fadac0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadb2d71640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadac0cfd80 con 0x7fadac0b10b0 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadb2d71640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadac0d0020 con 0x7fadac0b0380 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadb2d71640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fadac0af8e0 con 0x7fadac06c950 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadb130e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fadac06c950 0x7fadac0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadabfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fadac0b10b0 0x7fadac15e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54698/0 (socket says 172.21.15.40:54698) 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadabfff640 1 -- 172.21.15.40:0/2573701509 learned_addr learned my addr 172.21.15.40:0/2573701509 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadb130e640 1 -- 172.21.15.40:0/2573701509 >> v1:172.21.15.184:6789/0 conn(0x7fadac0b0380 legacy=0x7fadac06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadb130e640 1 -- 172.21.15.40:0/2573701509 >> v1:172.21.15.40:6789/0 conn(0x7fadac0b10b0 legacy=0x7fadac15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadab7fe640 1 -- 172.21.15.40:0/2573701509 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073680171 0 0) 0x7fadac0cfd80 con 0x7fadac0b10b0 2024-04-26T21:35:54.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.528+0000 7fadb130e640 1 -- 172.21.15.40:0/2573701509 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac0d0300 con 0x7fadac06c950 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.529+0000 7fadb130e640 1 --2- 172.21.15.40:0/2573701509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0ccd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fad940097a0 tx=0x7fad9402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa834dcb9adc79a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.529+0000 7fadab7fe640 1 -- 172.21.15.40:0/2573701509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94037070 con 0x7fadac06c950 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.529+0000 7fadab7fe640 1 -- 172.21.15.40:0/2573701509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9402ae50 con 0x7fadac06c950 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.529+0000 7fadab7fe640 1 -- 172.21.15.40:0/2573701509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94032660 con 0x7fadac06c950 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.530+0000 7fadb2d71640 1 -- 172.21.15.40:0/2573701509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 msgr2=0x7fadac0ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.530+0000 7fadb2d71640 1 --2- 172.21.15.40:0/2573701509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0ccd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fad940097a0 tx=0x7fad9402a140 comp rx=0 tx=0).stop 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.530+0000 7fadb2d71640 1 -- 172.21.15.40:0/2573701509 shutdown_connections 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.530+0000 7fadb2d71640 1 --2- 172.21.15.40:0/2573701509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0ccd50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.530+0000 7fadb2d71640 1 -- 172.21.15.40:0/2573701509 >> 172.21.15.40:0/2573701509 conn(0x7fadac0d1cf0 msgr2=0x7fadac0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.530+0000 7fadb2d71640 1 -- 172.21.15.40:0/2573701509 shutdown_connections 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.530+0000 7fadb2d71640 1 -- 172.21.15.40:0/2573701509 wait complete. 2024-04-26T21:35:54.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.531+0000 7fadb2d71640 1 Processor -- start 2024-04-26T21:35:54.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.531+0000 7fadb2d71640 1 -- start start 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb2d71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb2d71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac0b0380 0x7fadac1f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb2d71640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac0b10b0 0x7fadac157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb2d71640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0b0760 con 0x7fadac06c950 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb2d71640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0d1980 con 0x7fadac0b10b0 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb2d71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fadac0d15c0 con 0x7fadac0b0380 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55738/0 (socket says 172.21.15.40:55738) 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 -- 172.21.15.40:0/1305043810 learned_addr learned my addr 172.21.15.40:0/1305043810 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb130e640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac0b10b0 0x7fadac157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 -- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac0b0380 msgr2=0x7fadac1f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac0b0380 0x7fadac1f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 -- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac0b10b0 msgr2=0x7fadac157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac0b10b0 0x7fadac157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 -- 172.21.15.40:0/1305043810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac1580b0 con 0x7fadac06c950 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb130e640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac0b10b0 0x7fadac157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.532+0000 7fadb0b0d640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0c84a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fada400c7d0 tx=0x7fada400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.533+0000 7fada97fa640 1 -- 172.21.15.40:0/1305043810 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4017070 con 0x7fadac06c950 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.533+0000 7fada97fa640 1 -- 172.21.15.40:0/1305043810 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada4003da0 con 0x7fadac06c950 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.533+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac0c8dc0 con 0x7fadac06c950 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.533+0000 7fada97fa640 1 -- 172.21.15.40:0/1305043810 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada40052b0 con 0x7fadac06c950 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.533+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac0c9380 con 0x7fadac06c950 2024-04-26T21:35:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.534+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadac0694f0 con 0x7fadac06c950 2024-04-26T21:35:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.534+0000 7fada97fa640 1 -- 172.21.15.40:0/1305043810 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fada401a070 con 0x7fadac06c950 2024-04-26T21:35:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.535+0000 7fada97fa640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad8c06c8f0 0x7fad8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.535+0000 7fada97fa640 1 -- 172.21.15.40:0/1305043810 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fada408b5f0 con 0x7fadac06c950 2024-04-26T21:35:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.535+0000 7fadabfff640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad8c06c8f0 0x7fad8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.535+0000 7fadabfff640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad8c06c8f0 0x7fad8c06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad9c005880 tx=0x7fad9c007950 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.539+0000 7fada97fa640 1 -- 172.21.15.40:0/1305043810 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fada40596b0 con 0x7fadac06c950 2024-04-26T21:35:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.725+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadac06c450 con 0x7fadac06c950 2024-04-26T21:35:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.726+0000 7fada97fa640 1 -- 172.21.15.40:0/1305043810 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7fada4058fb0 con 0x7fadac06c950 2024-04-26T21:35:54.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:54.727 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad8c06c8f0 msgr2=0x7fad8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad8c06c8f0 0x7fad8c06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad9c005880 tx=0x7fad9c007950 comp rx=0 tx=0).stop 2024-04-26T21:35:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 msgr2=0x7fadac0c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0c84a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fada400c7d0 tx=0x7fada400cc90 comp rx=0 tx=0).stop 2024-04-26T21:35:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 shutdown_connections 2024-04-26T21:35:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad8c06c8f0 0x7fad8c06edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac0b10b0 0x7fadac157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac0b0380 0x7fadac1f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 --2- 172.21.15.40:0/1305043810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadac06c950 0x7fadac0c84a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 >> 172.21.15.40:0/1305043810 conn(0x7fadac0d1cf0 msgr2=0x7fadac0b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 shutdown_connections 2024-04-26T21:35:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.728+0000 7fadb2d71640 1 -- 172.21.15.40:0/1305043810 wait complete. 2024-04-26T21:35:54.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:54.741 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:54.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.843+0000 7f86dad6a640 1 Processor -- start 2024-04-26T21:35:54.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.843+0000 7f86dad6a640 1 -- start start 2024-04-26T21:35:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.844+0000 7f86dad6a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86d415d100 0x7f86d4160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.844+0000 7f86dad6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d4070930 0x7f86d40aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.844+0000 7f86dad6a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86d406c240 0x7f86d406c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.844+0000 7f86dad6a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f86d40af2d0 con 0x7f86d406c240 2024-04-26T21:35:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.844+0000 7f86dad6a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f86d40b0990 con 0x7f86d415d100 2024-04-26T21:35:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.844+0000 7f86dad6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86d40b0c70 con 0x7f86d4070930 2024-04-26T21:35:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.844+0000 7f86d9307640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86d406c240 0x7f86d406c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d9307640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86d406c240 0x7f86d406c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55742/0 (socket says 172.21.15.40:55742) 2024-04-26T21:35:54.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d9307640 1 -- 172.21.15.40:0/2395931262 learned_addr learned my addr 172.21.15.40:0/2395931262 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:54.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d3fff640 1 --2- 172.21.15.40:0/2395931262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d4070930 0x7f86d40aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d8b06640 1 --2- 172.21.15.40:0/2395931262 >> v2:172.21.15.184:3300/0 conn(0x7f86d415d100 0x7f86d4160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d9307640 1 -- 172.21.15.40:0/2395931262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d4070930 msgr2=0x7f86d40aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:54.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d9307640 1 --2- 172.21.15.40:0/2395931262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d4070930 0x7f86d40aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d9307640 1 -- 172.21.15.40:0/2395931262 >> v2:172.21.15.184:3300/0 conn(0x7f86d415d100 msgr2=0x7f86d4160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:54.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d9307640 1 --2- 172.21.15.40:0/2395931262 >> v2:172.21.15.184:3300/0 conn(0x7f86d415d100 0x7f86d4160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d9307640 1 -- 172.21.15.40:0/2395931262 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d40b0f50 con 0x7f86d406c240 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.845+0000 7f86d9307640 1 --2- 172.21.15.40:0/2395931262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d406c240 0x7f86d406c6d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f86d40b17f0 tx=0x7f86cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527dc45f33d8bbac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86d37fe640 1 -- 172.21.15.40:0/2395931262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc036070 con 0x7f86d406c240 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86d37fe640 1 -- 172.21.15.40:0/2395931262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc031600 con 0x7f86d406c240 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86d37fe640 1 -- 172.21.15.40:0/2395931262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0319d0 con 0x7f86d406c240 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 -- 172.21.15.40:0/2395931262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d406c240 msgr2=0x7f86d406c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 --2- 172.21.15.40:0/2395931262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d406c240 0x7f86d406c6d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f86d40b17f0 tx=0x7f86cc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 -- 172.21.15.40:0/2395931262 shutdown_connections 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 --2- 172.21.15.40:0/2395931262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d406c240 0x7f86d406c6d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 --2- 172.21.15.40:0/2395931262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d4070930 0x7f86d40aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 --2- 172.21.15.40:0/2395931262 >> v2:172.21.15.184:3300/0 conn(0x7f86d415d100 0x7f86d4160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 -- 172.21.15.40:0/2395931262 >> 172.21.15.40:0/2395931262 conn(0x7f86d40b39b0 msgr2=0x7f86d40b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 -- 172.21.15.40:0/2395931262 shutdown_connections 2024-04-26T21:35:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.846+0000 7f86dad6a640 1 -- 172.21.15.40:0/2395931262 wait complete. 2024-04-26T21:35:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.847+0000 7f86dad6a640 1 Processor -- start 2024-04-26T21:35:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.847+0000 7f86dad6a640 1 -- start start 2024-04-26T21:35:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86dad6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d406c240 0x7f86d41e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86dad6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4070930 0x7f86d41576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86dad6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d415d100 0x7f86d415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86dad6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4073160 con 0x7f86d415d100 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86dad6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86d40b1510 con 0x7f86d4070930 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86dad6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86d40732e0 con 0x7f86d406c240 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86d9307640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d415d100 0x7f86d415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86d9307640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d415d100 0x7f86d415f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55748/0 (socket says 172.21.15.40:55748) 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.848+0000 7f86d9307640 1 -- 172.21.15.40:0/1914321293 learned_addr learned my addr 172.21.15.40:0/1914321293 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d3fff640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4070930 0x7f86d41576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d8b06640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d406c240 0x7f86d41e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d9307640 1 -- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d406c240 msgr2=0x7f86d41e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d9307640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d406c240 0x7f86d41e6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d9307640 1 -- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4070930 msgr2=0x7f86d41576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d9307640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4070930 0x7f86d41576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d9307640 1 -- 172.21.15.40:0/1914321293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d414fd30 con 0x7f86d415d100 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d9307640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d415d100 0x7f86d415f890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f86cc034040 tx=0x7f86cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d17fa640 1 -- 172.21.15.40:0/1914321293 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc041020 con 0x7f86d415d100 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d17fa640 1 -- 172.21.15.40:0/1914321293 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc0077a0 con 0x7f86d415d100 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86d17fa640 1 -- 172.21.15.40:0/1914321293 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc036040 con 0x7f86d415d100 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d4150010 con 0x7f86d415d100 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.849+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d41505d0 con 0x7f86d415d100 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.850+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d40694f0 con 0x7f86d415d100 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.851+0000 7f86d17fa640 1 -- 172.21.15.40:0/1914321293 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86cc044020 con 0x7f86d415d100 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.851+0000 7f86d17fa640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.851+0000 7f86d17fa640 1 -- 172.21.15.40:0/1914321293 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f86cc0aca10 con 0x7f86d415d100 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.851+0000 7f86d8b06640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.852+0000 7f86d8b06640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f86c006c920 0x7f86c006ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86c40079a0 tx=0x7f86c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:54.856+0000 7f86d17fa640 1 -- 172.21.15.40:0/1914321293 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86cc04b030 con 0x7f86d415d100 2024-04-26T21:35:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.037+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86d406cd70 con 0x7f86d415d100 2024-04-26T21:35:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.038+0000 7f86d17fa640 1 -- 172.21.15.40:0/1914321293 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f86cc07a480 con 0x7f86d415d100 2024-04-26T21:35:55.039 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f86c006c920 msgr2=0x7f86c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f86c006c920 0x7f86c006ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86c40079a0 tx=0x7f86c4008040 comp rx=0 tx=0).stop 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d415d100 msgr2=0x7f86d415f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d415d100 0x7f86d415f890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f86cc034040 tx=0x7f86cc004a40 comp rx=0 tx=0).stop 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 shutdown_connections 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d415d100 0x7f86d415f890 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d4070930 0x7f86d41576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 --2- 172.21.15.40:0/1914321293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86d406c240 0x7f86d41e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 >> 172.21.15.40:0/1914321293 conn(0x7f86d40b39b0 msgr2=0x7f86d4073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:55.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 shutdown_connections 2024-04-26T21:35:55.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.040+0000 7f86dad6a640 1 -- 172.21.15.40:0/1914321293 wait complete. 2024-04-26T21:35:55.052 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:55.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.155+0000 7f157c704640 1 Processor -- start 2024-04-26T21:35:55.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.156+0000 7f157c704640 1 -- start start 2024-04-26T21:35:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.156+0000 7f157c704640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1574162cd0 0x7f15740b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.156+0000 7f157c704640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f15740d2110 0x7f15740b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.156+0000 7f157c704640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15740d0890 con 0x7f15741628f0 2024-04-26T21:35:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.156+0000 7f157c704640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f15740d1130 con 0x7f15740d2110 2024-04-26T21:35:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.156+0000 7f157c704640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15740d0b70 con 0x7f1574162cd0 2024-04-26T21:35:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157a4a0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f15741628f0 0x7f15740d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54714/0 (socket says 172.21.15.40:54714) 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157a4a0640 1 -- 172.21.15.40:0/125298533 learned_addr learned my addr 172.21.15.40:0/125298533 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f1579c9f640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1574162cd0 0x7f15740b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201014607 0 0) 0x7f15740d0890 con 0x7f15741628f0 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157aca1640 1 --2- 172.21.15.40:0/125298533 >> v2:172.21.15.184:3300/0 conn(0x7f15740d2110 0x7f15740b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1560003520 con 0x7f15741628f0 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1850290963 0 0) 0x7f1560003520 con 0x7f15741628f0 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1574162cd0 msgr2=0x7f15740b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1574162cd0 0x7f15740b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 >> v2:172.21.15.184:3300/0 conn(0x7f15740d2110 msgr2=0x7f15740b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 --2- 172.21.15.40:0/125298533 >> v2:172.21.15.184:3300/0 conn(0x7f15740d2110 0x7f15740b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15740d0e50 con 0x7f15741628f0 2024-04-26T21:35:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1570001670 con 0x7f15741628f0 2024-04-26T21:35:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 >> v1:172.21.15.40:6789/0 conn(0x7f15741628f0 legacy=0x7f15740d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15600039b0 0x7f1560003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f1560004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1574162cd0 0x7f15600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15740d1130 con 0x7f1560004800 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15740d0b70 con 0x7f15600039b0 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f156000b9f0 con 0x7f1574162cd0 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f1579c9f640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f1560004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157a4a0640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15600039b0 0x7f1560003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.157+0000 7f157aca1640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1574162cd0 0x7f15600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.158+0000 7f1579c9f640 1 -- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1574162cd0 msgr2=0x7f15600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.158+0000 7f1579c9f640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1574162cd0 0x7f15600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.158+0000 7f1579c9f640 1 -- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15600039b0 msgr2=0x7f1560003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.158+0000 7f1579c9f640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15600039b0 0x7f1560003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.158+0000 7f1579c9f640 1 -- 172.21.15.40:0/125298533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1560003520 con 0x7f1560004800 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.158+0000 7f1579c9f640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f1560004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1564009a90 tx=0x7f1560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84e806f726cf8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.158+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564036070 con 0x7f1560004800 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.158+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1564031400 con 0x7f1560004800 2024-04-26T21:35:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.159+0000 7f157949e640 1 -- 172.21.15.40:0/125298533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564009690 con 0x7f1560004800 2024-04-26T21:35:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.159+0000 7f157c704640 1 -- 172.21.15.40:0/125298533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 msgr2=0x7f1560004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.159+0000 7f157c704640 1 --2- 172.21.15.40:0/125298533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f1560004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1564009a90 tx=0x7f1560005830 comp rx=0 tx=0).stop 2024-04-26T21:35:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.159+0000 7f157aca1640 1 -- 172.21.15.40:0/125298533 reap_dead start 2024-04-26T21:35:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.159+0000 7f157c704640 1 -- 172.21.15.40:0/125298533 shutdown_connections 2024-04-26T21:35:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.159+0000 7f157c704640 1 -- 172.21.15.40:0/125298533 >> 172.21.15.40:0/125298533 conn(0x7f15740d4ff0 msgr2=0x7f15740d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.159+0000 7f157c704640 1 -- 172.21.15.40:0/125298533 shutdown_connections 2024-04-26T21:35:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.159+0000 7f157c704640 1 -- 172.21.15.40:0/125298533 wait complete. 2024-04-26T21:35:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.160+0000 7f157c704640 1 Processor -- start 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.160+0000 7f157c704640 1 -- start start 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157c704640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f15741616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157c704640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1574161c20 0x7f157415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157c704640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f157415d300 0x7f15740cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157c704640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15740d16d0 con 0x7f1560004800 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157c704640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15740d1550 con 0x7f1574161c20 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157c704640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15740b3f50 con 0x7f157415d300 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157a4a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f15741616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157a4a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f15741616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55754/0 (socket says 172.21.15.40:55754) 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157a4a0640 1 -- 172.21.15.40:0/3430781446 learned_addr learned my addr 172.21.15.40:0/3430781446 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157a4a0640 1 -- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f157415d300 msgr2=0x7f15740cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157a4a0640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f157415d300 0x7f15740cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157a4a0640 1 -- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1574161c20 msgr2=0x7f157415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.161+0000 7f157a4a0640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1574161c20 0x7f157415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.162+0000 7f157a4a0640 1 -- 172.21.15.40:0/3430781446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157415e170 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.162+0000 7f157a4a0640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f15741616e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1570004ef0 tx=0x7f1570007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.162+0000 7f156affd640 1 -- 172.21.15.40:0/3430781446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157000f7b0 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.162+0000 7f156affd640 1 -- 172.21.15.40:0/3430781446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1570018070 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.162+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157415dbb0 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.162+0000 7f156affd640 1 -- 172.21.15.40:0/3430781446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157000fbb0 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.162+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15741f5ce0 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.163+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15740694f0 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.163+0000 7f156affd640 1 -- 172.21.15.40:0/3430781446 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f157000fd50 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.164+0000 7f156affd640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f155806c920 0x7f155806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.164+0000 7f156affd640 1 -- 172.21.15.40:0/3430781446 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f157008bf90 con 0x7f1560004800 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.164+0000 7f1579c9f640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f155806c920 0x7f155806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.164+0000 7f1579c9f640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f155806c920 0x7f155806ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1564033000 tx=0x7f156402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.168+0000 7f156affd640 1 -- 172.21.15.40:0/3430781446 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f15700598d0 con 0x7f1560004800 2024-04-26T21:35:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.353+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f15740b7430 con 0x7f1560004800 2024-04-26T21:35:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.354+0000 7f156affd640 1 -- 172.21.15.40:0/3430781446 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f157005dbd0 con 0x7f1560004800 2024-04-26T21:35:55.355 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f155806c920 msgr2=0x7f155806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f155806c920 0x7f155806ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1564033000 tx=0x7f156402e000 comp rx=0 tx=0).stop 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 msgr2=0x7f15741616e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f15741616e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1570004ef0 tx=0x7f1570007430 comp rx=0 tx=0).stop 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 shutdown_connections 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f155806c920 0x7f155806ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f157415d300 0x7f15740cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1574161c20 0x7f157415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 --2- 172.21.15.40:0/3430781446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1560004800 0x7f15741616e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 >> 172.21.15.40:0/3430781446 conn(0x7f15740d4ff0 msgr2=0x7f15740d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 shutdown_connections 2024-04-26T21:35:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.356+0000 7f157c704640 1 -- 172.21.15.40:0/3430781446 wait complete. 2024-04-26T21:35:55.368 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.471+0000 7fa417740640 1 Processor -- start 2024-04-26T21:35:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.471+0000 7fa417740640 1 -- start start 2024-04-26T21:35:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.471+0000 7fa417740640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4101628f0 0x7fa4100d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.471+0000 7fa417740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4100d1550 0x7fa4100d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.471+0000 7fa417740640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4100d0890 con 0x7fa410162cd0 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.471+0000 7fa417740640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa4100d1130 con 0x7fa4101628f0 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.471+0000 7fa417740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4100d0b70 con 0x7fa4100d1550 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.472+0000 7fa415cdd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4100d1550 0x7fa4100d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.472+0000 7fa415cdd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4100d1550 0x7fa4100d1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:46668/0 (socket says 172.21.15.40:46668) 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.472+0000 7fa415cdd640 1 -- 172.21.15.40:0/285599074 learned_addr learned my addr 172.21.15.40:0/285599074 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.472+0000 7fa4154dc640 1 --2- 172.21.15.40:0/285599074 >> v2:172.21.15.184:3300/0 conn(0x7fa4101628f0 0x7fa4100d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.472+0000 7fa415cdd640 1 -- 172.21.15.40:0/285599074 >> v2:172.21.15.184:3300/0 conn(0x7fa4101628f0 msgr2=0x7fa4100d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.472+0000 7fa415cdd640 1 --2- 172.21.15.40:0/285599074 >> v2:172.21.15.184:3300/0 conn(0x7fa4101628f0 0x7fa4100d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.472+0000 7fa415cdd640 1 -- 172.21.15.40:0/285599074 >> v1:172.21.15.184:6789/0 conn(0x7fa410162cd0 legacy=0x7fa4100d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.472+0000 7fa415cdd640 1 -- 172.21.15.40:0/285599074 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4100d0e50 con 0x7fa4100d1550 2024-04-26T21:35:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.473+0000 7fa415cdd640 1 --2- 172.21.15.40:0/285599074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4100d1550 0x7fa4100d1f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370ce65e039d6cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.473+0000 7fa407fff640 1 -- 172.21.15.40:0/285599074 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408036070 con 0x7fa4100d1550 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.473+0000 7fa407fff640 1 -- 172.21.15.40:0/285599074 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa408031600 con 0x7fa4100d1550 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.473+0000 7fa407fff640 1 -- 172.21.15.40:0/285599074 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080319d0 con 0x7fa4100d1550 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.474+0000 7fa417740640 1 -- 172.21.15.40:0/285599074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4100d1550 msgr2=0x7fa4100d1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.474+0000 7fa417740640 1 --2- 172.21.15.40:0/285599074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4100d1550 0x7fa4100d1f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40802a680 comp rx=0 tx=0).stop 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.474+0000 7fa417740640 1 -- 172.21.15.40:0/285599074 shutdown_connections 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.474+0000 7fa417740640 1 --2- 172.21.15.40:0/285599074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4100d1550 0x7fa4100d1f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.474+0000 7fa417740640 1 --2- 172.21.15.40:0/285599074 >> v2:172.21.15.184:3300/0 conn(0x7fa4101628f0 0x7fa4100d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.474+0000 7fa417740640 1 -- 172.21.15.40:0/285599074 >> 172.21.15.40:0/285599074 conn(0x7fa4100d4ff0 msgr2=0x7fa4100d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.474+0000 7fa417740640 1 -- 172.21.15.40:0/285599074 shutdown_connections 2024-04-26T21:35:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.474+0000 7fa417740640 1 -- 172.21.15.40:0/285599074 wait complete. 2024-04-26T21:35:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.475+0000 7fa417740640 1 Processor -- start 2024-04-26T21:35:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.475+0000 7fa417740640 1 -- start start 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa417740640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100d1550 0x7fa410154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa417740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4101628f0 0x7fa4100b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa417740640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa410162cd0 0x7fa4100ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa417740640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100b5df0 con 0x7fa4100d1550 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa417740640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100b5c70 con 0x7fa410162cd0 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa417740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4100b5f70 con 0x7fa4101628f0 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100d1550 0x7fa410154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100d1550 0x7fa410154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55756/0 (socket says 172.21.15.40:55756) 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 -- 172.21.15.40:0/3757878695 learned_addr learned my addr 172.21.15.40:0/3757878695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa414cdb640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4101628f0 0x7fa4100b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa415cdd640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa410162cd0 0x7fa4100ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 -- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4101628f0 msgr2=0x7fa4100b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4101628f0 0x7fa4100b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 -- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa410162cd0 msgr2=0x7fa4100ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa410162cd0 0x7fa4100ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 -- 172.21.15.40:0/3757878695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4100ccad0 con 0x7fa4100d1550 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa4154dc640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100d1550 0x7fa410154350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00e9b0 tx=0x7fa40c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.476+0000 7fa405ffb640 1 -- 172.21.15.40:0/3757878695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c017070 con 0x7fa4100d1550 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.477+0000 7fa405ffb640 1 -- 172.21.15.40:0/3757878695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c00c9a0 con 0x7fa4100d1550 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.477+0000 7fa405ffb640 1 -- 172.21.15.40:0/3757878695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00cd90 con 0x7fa4100d1550 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.477+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4100ccdb0 con 0x7fa4100d1550 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.477+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4100cd370 con 0x7fa4100d1550 2024-04-26T21:35:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.478+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4100694f0 con 0x7fa4100d1550 2024-04-26T21:35:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.478+0000 7fa405ffb640 1 -- 172.21.15.40:0/3757878695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa40c01a070 con 0x7fa4100d1550 2024-04-26T21:35:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.478+0000 7fa405ffb640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3f406c8f0 0x7fa3f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.479+0000 7fa405ffb640 1 -- 172.21.15.40:0/3757878695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa40c08c330 con 0x7fa4100d1550 2024-04-26T21:35:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.479+0000 7fa414cdb640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3f406c8f0 0x7fa3f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.479+0000 7fa414cdb640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3f406c8f0 0x7fa3f406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4100cbc20 tx=0x7fa400009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.483+0000 7fa405ffb640 1 -- 172.21.15.40:0/3757878695 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa40c059c90 con 0x7fa4100d1550 2024-04-26T21:35:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.668+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa4100d54e0 con 0x7fa4100d1550 2024-04-26T21:35:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.669+0000 7fa405ffb640 1 -- 172.21.15.40:0/3757878695 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa40c05df90 con 0x7fa4100d1550 2024-04-26T21:35:55.670 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3f406c8f0 msgr2=0x7fa3f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3f406c8f0 0x7fa3f406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4100cbc20 tx=0x7fa400009040 comp rx=0 tx=0).stop 2024-04-26T21:35:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100d1550 msgr2=0x7fa410154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100d1550 0x7fa410154350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00e9b0 tx=0x7fa40c00ee70 comp rx=0 tx=0).stop 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 shutdown_connections 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3f406c8f0 0x7fa3f406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa410162cd0 0x7fa4100ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4101628f0 0x7fa4100b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 --2- 172.21.15.40:0/3757878695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100d1550 0x7fa410154350 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 >> 172.21.15.40:0/3757878695 conn(0x7fa4100d4ff0 msgr2=0x7fa410161360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 shutdown_connections 2024-04-26T21:35:55.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.671+0000 7fa417740640 1 -- 172.21.15.40:0/3757878695 wait complete. 2024-04-26T21:35:55.683 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:55.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.787+0000 7fb98ce15640 1 Processor -- start 2024-04-26T21:35:55.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.787+0000 7fb98ce15640 1 -- start start 2024-04-26T21:35:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98ce15640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9880cfde0 0x7fb9880d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98ce15640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb988070350 0x7fb988070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98ce15640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9880d2390 con 0x7fb988071090 2024-04-26T21:35:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98ce15640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb9880afab0 con 0x7fb9880cfde0 2024-04-26T21:35:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98ce15640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb98806dd20 con 0x7fb988070350 2024-04-26T21:35:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9880cfde0 0x7fb9880d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9880cfde0 0x7fb9880d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55768/0 (socket says 172.21.15.40:55768) 2024-04-26T21:35:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98659c640 1 -- 172.21.15.40:0/4276519154 learned_addr learned my addr 172.21.15.40:0/4276519154 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb97dd9b640 1 --2- 172.21.15.40:0/4276519154 >> v2:172.21.15.184:3300/0 conn(0x7fb988070350 0x7fb988070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98659c640 1 -- 172.21.15.40:0/4276519154 >> v2:172.21.15.184:3300/0 conn(0x7fb988070350 msgr2=0x7fb988070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98659c640 1 --2- 172.21.15.40:0/4276519154 >> v2:172.21.15.184:3300/0 conn(0x7fb988070350 0x7fb988070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98659c640 1 -- 172.21.15.40:0/4276519154 >> v1:172.21.15.184:6789/0 conn(0x7fb988071090 legacy=0x7fb98815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98659c640 1 -- 172.21.15.40:0/4276519154 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98806d3f0 con 0x7fb9880cfde0 2024-04-26T21:35:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98659c640 1 --2- 172.21.15.40:0/4276519154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9880cfde0 0x7fb9880d1b60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb98806cb80 tx=0x7fb97802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823650b4ef0202c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb985d9b640 1 -- 172.21.15.40:0/4276519154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978037070 con 0x7fb9880cfde0 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb985d9b640 1 -- 172.21.15.40:0/4276519154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb978032440 con 0x7fb9880cfde0 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb985d9b640 1 -- 172.21.15.40:0/4276519154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978032810 con 0x7fb9880cfde0 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98ce15640 1 -- 172.21.15.40:0/4276519154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9880cfde0 msgr2=0x7fb9880d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.788+0000 7fb98ce15640 1 --2- 172.21.15.40:0/4276519154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9880cfde0 0x7fb9880d1b60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb98806cb80 tx=0x7fb97802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.789+0000 7fb98ce15640 1 -- 172.21.15.40:0/4276519154 shutdown_connections 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.789+0000 7fb98ce15640 1 --2- 172.21.15.40:0/4276519154 >> v2:172.21.15.184:3300/0 conn(0x7fb988070350 0x7fb988070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.789+0000 7fb98ce15640 1 --2- 172.21.15.40:0/4276519154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9880cfde0 0x7fb9880d1b60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.789+0000 7fb98ce15640 1 -- 172.21.15.40:0/4276519154 >> 172.21.15.40:0/4276519154 conn(0x7fb9880d2d80 msgr2=0x7fb9880d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.789+0000 7fb98ce15640 1 -- 172.21.15.40:0/4276519154 shutdown_connections 2024-04-26T21:35:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.789+0000 7fb98ce15640 1 -- 172.21.15.40:0/4276519154 wait complete. 2024-04-26T21:35:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.790+0000 7fb98ce15640 1 Processor -- start 2024-04-26T21:35:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.790+0000 7fb98ce15640 1 -- start start 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb98ce15640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb988070350 0x7fb988159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb98ce15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb988071090 0x7fb9880b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb98ce15640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9880cfde0 0x7fb9880b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb98ce15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb98806d9b0 con 0x7fb988071090 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb98ce15640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9880714c0 con 0x7fb988070350 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb98ce15640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9880d20a0 con 0x7fb9880cfde0 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb988071090 0x7fb9880b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb988071090 0x7fb9880b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55772/0 (socket says 172.21.15.40:55772) 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 -- 172.21.15.40:0/352481639 learned_addr learned my addr 172.21.15.40:0/352481639 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb986d9d640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9880cfde0 0x7fb9880b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb98659c640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb988070350 0x7fb988159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 -- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9880cfde0 msgr2=0x7fb9880b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9880cfde0 0x7fb9880b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 -- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb988070350 msgr2=0x7fb988159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb988070350 0x7fb988159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 -- 172.21.15.40:0/352481639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9880b48f0 con 0x7fb988071090 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb98659c640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb988070350 0x7fb988159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.791+0000 7fb97dd9b640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb988071090 0x7fb9880b5ec0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c7f0 tx=0x7fb97000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.792+0000 7fb97f7fe640 1 -- 172.21.15.40:0/352481639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970015070 con 0x7fb988071090 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.792+0000 7fb97f7fe640 1 -- 172.21.15.40:0/352481639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb970004220 con 0x7fb988071090 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.792+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9880b4bd0 con 0x7fb988071090 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.792+0000 7fb97f7fe640 1 -- 172.21.15.40:0/352481639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970007620 con 0x7fb988071090 2024-04-26T21:35:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.792+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9880b5190 con 0x7fb988071090 2024-04-26T21:35:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.793+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9880694f0 con 0x7fb988071090 2024-04-26T21:35:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.793+0000 7fb97f7fe640 1 -- 172.21.15.40:0/352481639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9700079d0 con 0x7fb988071090 2024-04-26T21:35:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.794+0000 7fb97f7fe640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb96806c8f0 0x7fb96806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.794+0000 7fb97f7fe640 1 -- 172.21.15.40:0/352481639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb97008c1d0 con 0x7fb988071090 2024-04-26T21:35:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.794+0000 7fb98659c640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb96806c8f0 0x7fb96806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.794+0000 7fb98659c640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb96806c8f0 0x7fb96806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb978034000 tx=0x7fb97802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.799+0000 7fb97f7fe640 1 -- 172.21.15.40:0/352481639 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb970059c40 con 0x7fb988071090 2024-04-26T21:35:55.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.984+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb9880cf300 con 0x7fb988071090 2024-04-26T21:35:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.984+0000 7fb97f7fe640 1 -- 172.21.15.40:0/352481639 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb97005df40 con 0x7fb988071090 2024-04-26T21:35:55.986 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:55.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.986+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb96806c8f0 msgr2=0x7fb96806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.986+0000 7fb98ce15640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb96806c8f0 0x7fb96806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb978034000 tx=0x7fb97802e000 comp rx=0 tx=0).stop 2024-04-26T21:35:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.986+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb988071090 msgr2=0x7fb9880b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.986+0000 7fb98ce15640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb988071090 0x7fb9880b5ec0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c7f0 tx=0x7fb97000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.987+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 shutdown_connections 2024-04-26T21:35:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.987+0000 7fb98ce15640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb96806c8f0 0x7fb96806edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.987+0000 7fb98ce15640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9880cfde0 0x7fb9880b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.987+0000 7fb98ce15640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb988071090 0x7fb9880b5ec0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.987+0000 7fb98ce15640 1 --2- 172.21.15.40:0/352481639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb988070350 0x7fb988159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.987+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 >> 172.21.15.40:0/352481639 conn(0x7fb9880d2d80 msgr2=0x7fb98806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.987+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 shutdown_connections 2024-04-26T21:35:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:55.987+0000 7fb98ce15640 1 -- 172.21.15.40:0/352481639 wait complete. 2024-04-26T21:35:55.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:56.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.099+0000 7fb1196b8640 1 Processor -- start 2024-04-26T21:35:56.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.099+0000 7fb1196b8640 1 -- start start 2024-04-26T21:35:56.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1196b8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb1140b0380 0x7fb1140ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1196b8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1140cfd80 con 0x7fb1140b10b0 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1196b8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1140d0020 con 0x7fb11406c950 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1196b8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb1140af8e0 con 0x7fb1140b0380 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb10bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb1140b10b0 0x7fb11406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54724/0 (socket says 172.21.15.40:54724) 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb10bfff640 1 -- 172.21.15.40:0/225642624 learned_addr learned my addr 172.21.15.40:0/225642624 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb112ffd640 1 --2- 172.21.15.40:0/225642624 >> v2:172.21.15.184:3300/0 conn(0x7fb1140b0380 0x7fb1140ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902537820 0 0) 0x7fb1140cfd80 con 0x7fb1140b10b0 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb100003520 con 0x7fb1140b10b0 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 11738542 0 0) 0x7fb100003520 con 0x7fb1140b10b0 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 >> v2:172.21.15.184:3300/0 conn(0x7fb1140b0380 msgr2=0x7fb1140ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.100+0000 7fb1127fc640 1 --2- 172.21.15.40:0/225642624 >> v2:172.21.15.184:3300/0 conn(0x7fb1140b0380 0x7fb1140ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 >> v1:172.21.15.184:6789/0 conn(0x7fb11406c950 legacy=0x7fb11415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1140d0300 con 0x7fb1140b10b0 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb0fc001690 con 0x7fb1140b10b0 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 >> v1:172.21.15.40:6789/0 conn(0x7fb1140b10b0 legacy=0x7fb11406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1000039b0 0x7fb100003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000043b0 0x7fb100004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb100005120 0x7fb10000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1140af8e0 con 0x7fb1000039b0 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb10000c060 con 0x7fb1000043b0 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb10000c260 con 0x7fb100005120 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb112ffd640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1000039b0 0x7fb100003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb10bfff640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000043b0 0x7fb100004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1137fe640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb100005120 0x7fb10000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb112ffd640 1 -- 172.21.15.40:0/225642624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb100005120 msgr2=0x7fb10000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb112ffd640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb100005120 0x7fb10000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb112ffd640 1 -- 172.21.15.40:0/225642624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000043b0 msgr2=0x7fb100004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb112ffd640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000043b0 0x7fb100004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb112ffd640 1 -- 172.21.15.40:0/225642624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb100003520 con 0x7fb1000039b0 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.101+0000 7fb1137fe640 1 -- 172.21.15.40:0/225642624 reap_dead start 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.102+0000 7fb112ffd640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1000039b0 0x7fb100003d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb104009ac0 tx=0x7fb100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0606fadd50a1b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.102+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104036070 con 0x7fb1000039b0 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.102+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb104031400 con 0x7fb1000039b0 2024-04-26T21:35:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.102+0000 7fb1127fc640 1 -- 172.21.15.40:0/225642624 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104009690 con 0x7fb1000039b0 2024-04-26T21:35:56.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.103+0000 7fb1196b8640 1 -- 172.21.15.40:0/225642624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1000039b0 msgr2=0x7fb100003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.103+0000 7fb1196b8640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1000039b0 0x7fb100003d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb104009ac0 tx=0x7fb100005830 comp rx=0 tx=0).stop 2024-04-26T21:35:56.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.103+0000 7fb1196b8640 1 -- 172.21.15.40:0/225642624 shutdown_connections 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.103+0000 7fb1196b8640 1 --2- 172.21.15.40:0/225642624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1000039b0 0x7fb100003d90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.103+0000 7fb1196b8640 1 -- 172.21.15.40:0/225642624 >> 172.21.15.40:0/225642624 conn(0x7fb1140d1cf0 msgr2=0x7fb1140d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.103+0000 7fb1196b8640 1 -- 172.21.15.40:0/225642624 shutdown_connections 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.103+0000 7fb1196b8640 1 -- 172.21.15.40:0/225642624 wait complete. 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.104+0000 7fb1196b8640 1 Processor -- start 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.104+0000 7fb1196b8640 1 -- start start 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000039b0 0x7fb114160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1140c5850 0x7fb11415d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb11415dea0 0x7fb1140c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1140b0760 con 0x7fb1140c5850 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb1140d1980 con 0x7fb1000039b0 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb1140d15c0 con 0x7fb11415dea0 2024-04-26T21:35:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1140c5850 0x7fb11415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1140c5850 0x7fb11415d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55782/0 (socket says 172.21.15.40:55782) 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 -- 172.21.15.40:0/1687027838 learned_addr learned my addr 172.21.15.40:0/1687027838 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb112ffd640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000039b0 0x7fb114160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1137fe640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb11415dea0 0x7fb1140c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 -- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb11415dea0 msgr2=0x7fb1140c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb11415dea0 0x7fb1140c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 -- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000039b0 msgr2=0x7fb114160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000039b0 0x7fb114160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 -- 172.21.15.40:0/1687027838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1141616e0 con 0x7fb1140c5850 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10bfff640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1140c5850 0x7fb11415d960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc004ec0 tx=0x7fb0fc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10b7fe640 1 -- 172.21.15.40:0/1687027838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00f730 con 0x7fb1140c5850 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10b7fe640 1 -- 172.21.15.40:0/1687027838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0fc018070 con 0x7fb1140c5850 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11415e6c0 con 0x7fb1140c5850 2024-04-26T21:35:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb10b7fe640 1 -- 172.21.15.40:0/1687027838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00fb50 con 0x7fb1140c5850 2024-04-26T21:35:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11415ec80 con 0x7fb1140c5850 2024-04-26T21:35:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.106+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11415ee80 con 0x7fb1140c5850 2024-04-26T21:35:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.107+0000 7fb10b7fe640 1 -- 172.21.15.40:0/1687027838 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0fc00fcf0 con 0x7fb1140c5850 2024-04-26T21:35:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.107+0000 7fb10b7fe640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0f406c920 0x7fb0f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.108+0000 7fb10b7fe640 1 -- 172.21.15.40:0/1687027838 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb0fc08be80 con 0x7fb1140c5850 2024-04-26T21:35:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.108+0000 7fb112ffd640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0f406c920 0x7fb0f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.108+0000 7fb112ffd640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0f406c920 0x7fb0f406ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb104033000 tx=0x7fb104008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.113+0000 7fb10b7fe640 1 -- 172.21.15.40:0/1687027838 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb11415ee80 con 0x7fb1140c5850 2024-04-26T21:35:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.301+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb11406ef90 con 0x7fb1140c5850 2024-04-26T21:35:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.302+0000 7fb10b7fe640 1 -- 172.21.15.40:0/1687027838 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7fb0fc059eb0 con 0x7fb1140c5850 2024-04-26T21:35:56.303 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:56.303 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0f406c920 msgr2=0x7fb0f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0f406c920 0x7fb0f406ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb104033000 tx=0x7fb104008030 comp rx=0 tx=0).stop 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1140c5850 msgr2=0x7fb11415d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1140c5850 0x7fb11415d960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc004ec0 tx=0x7fb0fc007430 comp rx=0 tx=0).stop 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 shutdown_connections 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0f406c920 0x7fb0f406ede0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb11415dea0 0x7fb1140c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1140c5850 0x7fb11415d960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 --2- 172.21.15.40:0/1687027838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1000039b0 0x7fb114160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 >> 172.21.15.40:0/1687027838 conn(0x7fb1140d1cf0 msgr2=0x7fb1140ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 shutdown_connections 2024-04-26T21:35:56.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.304+0000 7fb1196b8640 1 -- 172.21.15.40:0/1687027838 wait complete. 2024-04-26T21:35:56.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:56.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:56.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.420+0000 7f7f355da640 1 Processor -- start 2024-04-26T21:35:56.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.420+0000 7f7f355da640 1 -- start start 2024-04-26T21:35:56.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.420+0000 7f7f355da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7f300b40d0 0x7f7f301625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f355da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f300d0a10 con 0x7f7f30162b30 2024-04-26T21:35:56.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f355da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f300d0cf0 con 0x7f7f300d1a00 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f355da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7f300d0fd0 con 0x7f7f300b40d0 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7f30162b30 0x7f7f300b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54732/0 (socket says 172.21.15.40:54732) 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2f7fe640 1 -- 172.21.15.40:0/4077311960 learned_addr learned my addr 172.21.15.40:0/4077311960 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f27fff640 1 --2- 172.21.15.40:0/4077311960 >> v2:172.21.15.184:3300/0 conn(0x7f7f300b40d0 0x7f7f301625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658967678 0 0) 0x7f7f300d0a10 con 0x7f7f30162b30 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f1c003520 con 0x7f7f30162b30 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 832196597 0 0) 0x7f7f1c003520 con 0x7f7f30162b30 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 >> v2:172.21.15.184:3300/0 conn(0x7f7f300b40d0 msgr2=0x7f7f301625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 --2- 172.21.15.40:0/4077311960 >> v2:172.21.15.184:3300/0 conn(0x7f7f300b40d0 0x7f7f301625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 >> v1:172.21.15.184:6789/0 conn(0x7f7f300d1a00 legacy=0x7f7f300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f30162f80 con 0x7f7f30162b30 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f7f280019f0 con 0x7f7f30162b30 2024-04-26T21:35:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 >> v1:172.21.15.40:6789/0 conn(0x7f7f30162b30 legacy=0x7f7f300b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0039b0 0x7f7f1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0051c0 0x7f7f1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f300d0fd0 con 0x7f7f1c004800 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c00c160 con 0x7f7f1c0051c0 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f1c00c360 con 0x7f7f1c0039b0 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f27fff640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2f7fe640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0051c0 0x7f7f1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2effd640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0039b0 0x7f7f1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f27fff640 1 -- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0039b0 msgr2=0x7f7f1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f27fff640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0039b0 0x7f7f1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f27fff640 1 -- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0051c0 msgr2=0x7f7f1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f27fff640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0051c0 0x7f7f1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f27fff640 1 -- 172.21.15.40:0/4077311960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c003520 con 0x7f7f1c004800 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2effd640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0039b0 0x7f7f1c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2f7fe640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0051c0 0x7f7f1c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2f7fe640 1 -- 172.21.15.40:0/4077311960 reap_dead start 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f27fff640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009ac0 tx=0x7f7f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb949a0543cc7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.422+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18036070 con 0x7f7f1c004800 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f18031400 con 0x7f7f1c004800 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f2e7fc640 1 -- 172.21.15.40:0/4077311960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18009690 con 0x7f7f1c004800 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f355da640 1 -- 172.21.15.40:0/4077311960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 msgr2=0x7f7f1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f355da640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009ac0 tx=0x7f7f1c005830 comp rx=0 tx=0).stop 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f355da640 1 -- 172.21.15.40:0/4077311960 shutdown_connections 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f355da640 1 --2- 172.21.15.40:0/4077311960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f355da640 1 -- 172.21.15.40:0/4077311960 >> 172.21.15.40:0/4077311960 conn(0x7f7f300d4fd0 msgr2=0x7f7f300d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f355da640 1 -- 172.21.15.40:0/4077311960 shutdown_connections 2024-04-26T21:35:56.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.423+0000 7f7f355da640 1 -- 172.21.15.40:0/4077311960 wait complete. 2024-04-26T21:35:56.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.424+0000 7f7f355da640 1 Processor -- start 2024-04-26T21:35:56.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.424+0000 7f7f355da640 1 -- start start 2024-04-26T21:35:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f355da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f301f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f355da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f301518b0 0x7f7f300cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f355da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f300cd920 0x7f7f300b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f355da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30162770 con 0x7f7f1c004800 2024-04-26T21:35:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f355da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f301625f0 con 0x7f7f300cd920 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f355da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f301628f0 con 0x7f7f301518b0 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f301f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f301f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55792/0 (socket says 172.21.15.40:55792) 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2effd640 1 -- 172.21.15.40:0/2010528263 learned_addr learned my addr 172.21.15.40:0/2010528263 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f27fff640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f301518b0 0x7f7f300cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2effd640 1 -- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f301518b0 msgr2=0x7f7f300cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2effd640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f301518b0 0x7f7f300cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2effd640 1 -- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f300cd920 msgr2=0x7f7f300b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2f7fe640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f300cd920 0x7f7f300b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2effd640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f300cd920 0x7f7f300b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f2effd640 1 -- 172.21.15.40:0/2010528263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3014f210 con 0x7f7f1c004800 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.425+0000 7f7f27fff640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f301518b0 0x7f7f300cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.426+0000 7f7f2effd640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f301f4a40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000e4e0 tx=0x7f7f2000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.426+0000 7f7f277fe640 1 -- 172.21.15.40:0/2010528263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f200052e0 con 0x7f7f1c004800 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.426+0000 7f7f277fe640 1 -- 172.21.15.40:0/2010528263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f20007020 con 0x7f7f1c004800 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.426+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3014f4f0 con 0x7f7f1c004800 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.426+0000 7f7f277fe640 1 -- 172.21.15.40:0/2010528263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20007210 con 0x7f7f1c004800 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.426+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3014fab0 con 0x7f7f1c004800 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.427+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f300694f0 con 0x7f7f1c004800 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.427+0000 7f7f277fe640 1 -- 172.21.15.40:0/2010528263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f2001a020 con 0x7f7f1c004800 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.428+0000 7f7f277fe640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f1006c8f0 0x7f7f1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.428+0000 7f7f277fe640 1 -- 172.21.15.40:0/2010528263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7f2008b430 con 0x7f7f1c004800 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.428+0000 7f7f27fff640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f1006c8f0 0x7f7f1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.428+0000 7f7f27fff640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f1006c8f0 0x7f7f1006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f18033000 tx=0x7f7f18008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.433+0000 7f7f277fe640 1 -- 172.21.15.40:0/2010528263 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f2001a310 con 0x7f7f1c004800 2024-04-26T21:35:56.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.618+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7f300b5570 con 0x7f7f1c004800 2024-04-26T21:35:56.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.618+0000 7f7f277fe640 1 -- 172.21.15.40:0/2010528263 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f7f20059460 con 0x7f7f1c004800 2024-04-26T21:35:56.619 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f1006c8f0 msgr2=0x7f7f1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f1006c8f0 0x7f7f1006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f18033000 tx=0x7f7f18008030 comp rx=0 tx=0).stop 2024-04-26T21:35:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 msgr2=0x7f7f301f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f301f4a40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000e4e0 tx=0x7f7f2000e9a0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 shutdown_connections 2024-04-26T21:35:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f1006c8f0 0x7f7f1006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f300cd920 0x7f7f300b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f301518b0 0x7f7f300cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 --2- 172.21.15.40:0/2010528263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f1c004800 0x7f7f301f4a40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 >> 172.21.15.40:0/2010528263 conn(0x7f7f300d4fd0 msgr2=0x7f7f301616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 shutdown_connections 2024-04-26T21:35:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.620+0000 7f7f355da640 1 -- 172.21.15.40:0/2010528263 wait complete. 2024-04-26T21:35:56.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:56.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.735+0000 7f0e36b5f640 1 Processor -- start 2024-04-26T21:35:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.735+0000 7f0e36b5f640 1 -- start start 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e36b5f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0e300d1a00 0x7f0e300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e36b5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e300d2320 0x7f0e301625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e36b5f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e300d0a10 con 0x7f0e300b40d0 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e36b5f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0e300d0cf0 con 0x7f0e300d2320 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e36b5f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0e300d0fd0 con 0x7f0e300d1a00 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e2ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e300d2320 0x7f0e301625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e2ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e300d2320 0x7f0e301625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55796/0 (socket says 172.21.15.40:55796) 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e2ffff640 1 -- 172.21.15.40:0/2004266186 learned_addr learned my addr 172.21.15.40:0/2004266186 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.736+0000 7f0e348fb640 1 --2- 172.21.15.40:0/2004266186 >> v2:172.21.15.184:3300/0 conn(0x7f0e300d1a00 0x7f0e300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.737+0000 7f0e2ffff640 1 -- 172.21.15.40:0/2004266186 >> v2:172.21.15.184:3300/0 conn(0x7f0e300d1a00 msgr2=0x7f0e300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.737+0000 7f0e2ffff640 1 --2- 172.21.15.40:0/2004266186 >> v2:172.21.15.184:3300/0 conn(0x7f0e300d1a00 0x7f0e300d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.737+0000 7f0e2ffff640 1 -- 172.21.15.40:0/2004266186 >> v1:172.21.15.184:6789/0 conn(0x7f0e300b40d0 legacy=0x7f0e300d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.737+0000 7f0e2ffff640 1 -- 172.21.15.40:0/2004266186 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e300d40a0 con 0x7f0e300d2320 2024-04-26T21:35:56.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.737+0000 7f0e2ffff640 1 --2- 172.21.15.40:0/2004266186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e301625f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0e3006c650 tx=0x7f0e2402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc262bcd2ba1178 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.737+0000 7f0e2f7fe640 1 -- 172.21.15.40:0/2004266186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24037070 con 0x7f0e300d2320 2024-04-26T21:35:56.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.737+0000 7f0e2f7fe640 1 -- 172.21.15.40:0/2004266186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e24032440 con 0x7f0e300d2320 2024-04-26T21:35:56.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.737+0000 7f0e2f7fe640 1 -- 172.21.15.40:0/2004266186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24032810 con 0x7f0e300d2320 2024-04-26T21:35:56.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.738+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2004266186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 msgr2=0x7f0e301625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.738+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2004266186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e301625f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0e3006c650 tx=0x7f0e2402a150 comp rx=0 tx=0).stop 2024-04-26T21:35:56.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.738+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2004266186 shutdown_connections 2024-04-26T21:35:56.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.738+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2004266186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e301625f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.738+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2004266186 >> v2:172.21.15.184:3300/0 conn(0x7f0e300d1a00 0x7f0e300d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.738+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2004266186 >> 172.21.15.40:0/2004266186 conn(0x7f0e300d4fd0 msgr2=0x7f0e300d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:56.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.738+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2004266186 shutdown_connections 2024-04-26T21:35:56.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.738+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2004266186 wait complete. 2024-04-26T21:35:56.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.739+0000 7f0e36b5f640 1 Processor -- start 2024-04-26T21:35:56.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.739+0000 7f0e36b5f640 1 -- start start 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e36b5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e300b40d0 0x7f0e3015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e36b5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e300d1a00 0x7f0e301503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e36b5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e300ba070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e36b5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e30162b30 con 0x7f0e300d2320 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e36b5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e300b5d60 con 0x7f0e300d1a00 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e36b5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e30162cb0 con 0x7f0e300b40d0 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e350fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e300ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e350fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e300ba070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55802/0 (socket says 172.21.15.40:55802) 2024-04-26T21:35:56.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e350fc640 1 -- 172.21.15.40:0/2066601586 learned_addr learned my addr 172.21.15.40:0/2066601586 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e348fb640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e300b40d0 0x7f0e3015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.740+0000 7f0e2ffff640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e300d1a00 0x7f0e301503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e350fc640 1 -- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e300b40d0 msgr2=0x7f0e3015d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e350fc640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e300b40d0 0x7f0e3015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e350fc640 1 -- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e300d1a00 msgr2=0x7f0e301503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e350fc640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e300d1a00 0x7f0e301503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e350fc640 1 -- 172.21.15.40:0/2066601586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3015dd10 con 0x7f0e300d2320 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e350fc640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e300ba070 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c7a0 tx=0x7f0e2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e2d7fa640 1 -- 172.21.15.40:0/2066601586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28003b70 con 0x7f0e300d2320 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e2d7fa640 1 -- 172.21.15.40:0/2066601586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e28003d30 con 0x7f0e300d2320 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3015dff0 con 0x7f0e300d2320 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e2d7fa640 1 -- 172.21.15.40:0/2066601586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800aa90 con 0x7f0e300d2320 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.741+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3015e5b0 con 0x7f0e300d2320 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.742+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e300763f0 con 0x7f0e300d2320 2024-04-26T21:35:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.742+0000 7f0e2d7fa640 1 -- 172.21.15.40:0/2066601586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e2800ac30 con 0x7f0e300d2320 2024-04-26T21:35:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.743+0000 7f0e2d7fa640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e1c06c8f0 0x7f0e1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.743+0000 7f0e2d7fa640 1 -- 172.21.15.40:0/2066601586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0e2808beb0 con 0x7f0e300d2320 2024-04-26T21:35:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.743+0000 7f0e348fb640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e1c06c8f0 0x7f0e1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.744+0000 7f0e348fb640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e1c06c8f0 0x7f0e1c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009a10 tx=0x7f0e20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.747+0000 7f0e2d7fa640 1 -- 172.21.15.40:0/2066601586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e300763f0 con 0x7f0e300d2320 2024-04-26T21:35:56.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.932+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0e30161d50 con 0x7f0e300d2320 2024-04-26T21:35:56.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.932+0000 7f0e2d7fa640 1 -- 172.21.15.40:0/2066601586 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f0e30161d50 con 0x7f0e300d2320 2024-04-26T21:35:56.933 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:56.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e1c06c8f0 msgr2=0x7f0e1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e1c06c8f0 0x7f0e1c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009a10 tx=0x7f0e20008040 comp rx=0 tx=0).stop 2024-04-26T21:35:56.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 msgr2=0x7f0e300ba070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:56.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e300ba070 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c7a0 tx=0x7f0e2800cc60 comp rx=0 tx=0).stop 2024-04-26T21:35:56.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 shutdown_connections 2024-04-26T21:35:56.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e1c06c8f0 0x7f0e1c06edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e300d2320 0x7f0e300ba070 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e300d1a00 0x7f0e301503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 --2- 172.21.15.40:0/2066601586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e300b40d0 0x7f0e3015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 >> 172.21.15.40:0/2066601586 conn(0x7f0e300d4fd0 msgr2=0x7f0e3006cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 shutdown_connections 2024-04-26T21:35:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:56.934+0000 7f0e36b5f640 1 -- 172.21.15.40:0/2066601586 wait complete. 2024-04-26T21:35:56.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.050+0000 7f793e740640 1 Processor -- start 2024-04-26T21:35:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.050+0000 7f793e740640 1 -- start start 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f793e740640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7938070350 0x7f7938070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f793e740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938071090 0x7f793815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f793e740640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79380d2390 con 0x7f79380cfde0 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f793e740640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7938071500 con 0x7f7938070350 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f793e740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79380afab0 con 0x7f7938071090 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f79377fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7938070350 0x7f7938070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f7937fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f79380cfde0 0x7f79380d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54736/0 (socket says 172.21.15.40:54736) 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f7937fff640 1 -- 172.21.15.40:0/2365479276 learned_addr learned my addr 172.21.15.40:0/2365479276 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f79377fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7938070350 0x7f7938070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55808/0 (socket says 172.21.15.40:55808) 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f793ccdd640 1 --2- 172.21.15.40:0/2365479276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938071090 0x7f793815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f79377fe640 1 -- 172.21.15.40:0/2365479276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938071090 msgr2=0x7f793815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f79377fe640 1 --2- 172.21.15.40:0/2365479276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938071090 0x7f793815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f79377fe640 1 -- 172.21.15.40:0/2365479276 >> v1:172.21.15.40:6789/0 conn(0x7f79380cfde0 legacy=0x7f79380d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f7936ffd640 1 -- 172.21.15.40:0/2365479276 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075227319 0 0) 0x7f79380d2390 con 0x7f79380cfde0 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f79377fe640 1 -- 172.21.15.40:0/2365479276 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793806d3f0 con 0x7f7938070350 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.051+0000 7f793ccdd640 1 --2- 172.21.15.40:0/2365479276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938071090 0x7f793815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.052+0000 7f79377fe640 1 --2- 172.21.15.40:0/2365479276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7938070350 0x7f7938070b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f79200097a0 tx=0x7f792002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599e6f087a9ed47b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.052+0000 7f7936ffd640 1 -- 172.21.15.40:0/2365479276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920037070 con 0x7f7938070350 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.052+0000 7f7936ffd640 1 -- 172.21.15.40:0/2365479276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f792002ae50 con 0x7f7938070350 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.052+0000 7f7936ffd640 1 -- 172.21.15.40:0/2365479276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920032660 con 0x7f7938070350 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.053+0000 7f793e740640 1 -- 172.21.15.40:0/2365479276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7938070350 msgr2=0x7f7938070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.053+0000 7f793e740640 1 --2- 172.21.15.40:0/2365479276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7938070350 0x7f7938070b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f79200097a0 tx=0x7f792002a100 comp rx=0 tx=0).stop 2024-04-26T21:35:57.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.053+0000 7f793e740640 1 -- 172.21.15.40:0/2365479276 shutdown_connections 2024-04-26T21:35:57.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.053+0000 7f793e740640 1 --2- 172.21.15.40:0/2365479276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938071090 0x7f793815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.053+0000 7f793e740640 1 --2- 172.21.15.40:0/2365479276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7938070350 0x7f7938070b50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.053+0000 7f793e740640 1 -- 172.21.15.40:0/2365479276 >> 172.21.15.40:0/2365479276 conn(0x7f79380d2d80 msgr2=0x7f79380d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:57.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.053+0000 7f793e740640 1 -- 172.21.15.40:0/2365479276 shutdown_connections 2024-04-26T21:35:57.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.053+0000 7f793e740640 1 -- 172.21.15.40:0/2365479276 wait complete. 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.054+0000 7f793e740640 1 Processor -- start 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.054+0000 7f793e740640 1 -- start start 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793e740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938070350 0x7f793814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793e740640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938071090 0x7f79380c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793e740640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380cfde0 0x7f79380c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793e740640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f793806d9b0 con 0x7f79380cfde0 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793e740640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f793806dd20 con 0x7f7938071090 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793e740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f793815ef30 con 0x7f7938070350 2024-04-26T21:35:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380cfde0 0x7f79380c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380cfde0 0x7f79380c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55814/0 (socket says 172.21.15.40:55814) 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 -- 172.21.15.40:0/426096575 learned_addr learned my addr 172.21.15.40:0/426096575 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f7937fff640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938070350 0x7f793814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 -- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938070350 msgr2=0x7f793814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f79377fe640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938071090 0x7f79380c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938070350 0x7f793814b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 -- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938071090 msgr2=0x7f79380c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938071090 0x7f79380c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 -- 172.21.15.40:0/426096575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793809e9c0 con 0x7f79380cfde0 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f7937fff640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938070350 0x7f793814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f79377fe640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938071090 0x7f79380c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.055+0000 7f793ccdd640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380cfde0 0x7f79380c7f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f793000cf20 tx=0x7f793000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.056+0000 7f7934ff9640 1 -- 172.21.15.40:0/426096575 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793001a070 con 0x7f79380cfde0 2024-04-26T21:35:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.056+0000 7f7934ff9640 1 -- 172.21.15.40:0/426096575 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793000d840 con 0x7f79380cfde0 2024-04-26T21:35:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.056+0000 7f7934ff9640 1 -- 172.21.15.40:0/426096575 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793000dc30 con 0x7f79380cfde0 2024-04-26T21:35:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.056+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793814d3c0 con 0x7f79380cfde0 2024-04-26T21:35:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.056+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793814d980 con 0x7f79380cfde0 2024-04-26T21:35:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.057+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79380694f0 con 0x7f79380cfde0 2024-04-26T21:35:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.057+0000 7f7934ff9640 1 -- 172.21.15.40:0/426096575 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f793001d070 con 0x7f79380cfde0 2024-04-26T21:35:57.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.057+0000 7f7934ff9640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.058+0000 7f7934ff9640 1 -- 172.21.15.40:0/426096575 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f793008c780 con 0x7f79380cfde0 2024-04-26T21:35:57.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.058+0000 7f7937fff640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.058+0000 7f7937fff640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f791806c8f0 0x7f791806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7928000f60 tx=0x7f7928009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.062+0000 7f7934ff9640 1 -- 172.21.15.40:0/426096575 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f793809e9c0 con 0x7f79380cfde0 2024-04-26T21:35:57.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.248+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f79380cf590 con 0x7f79380cfde0 2024-04-26T21:35:57.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.249+0000 7f7934ff9640 1 -- 172.21.15.40:0/426096575 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f793005a840 con 0x7f79380cfde0 2024-04-26T21:35:57.249 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f791806c8f0 msgr2=0x7f791806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f791806c8f0 0x7f791806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7928000f60 tx=0x7f7928009040 comp rx=0 tx=0).stop 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380cfde0 msgr2=0x7f79380c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380cfde0 0x7f79380c7f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f793000cf20 tx=0x7f793000a7d0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 shutdown_connections 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380cfde0 0x7f79380c7f70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7938071090 0x7f79380c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 --2- 172.21.15.40:0/426096575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7938070350 0x7f793814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 >> 172.21.15.40:0/426096575 conn(0x7f79380d2d80 msgr2=0x7f793815e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 shutdown_connections 2024-04-26T21:35:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.251+0000 7f793e740640 1 -- 172.21.15.40:0/426096575 wait complete. 2024-04-26T21:35:57.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.366+0000 7f2e384b0640 1 Processor -- start 2024-04-26T21:35:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.366+0000 7f2e384b0640 1 -- start start 2024-04-26T21:35:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.367+0000 7f2e384b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e300b40d0 0x7f2e301625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.367+0000 7f2e384b0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e300d0a10 con 0x7f2e300d1a00 2024-04-26T21:35:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.367+0000 7f2e384b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e300d0cf0 con 0x7f2e30162b30 2024-04-26T21:35:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.367+0000 7f2e384b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2e300d0fd0 con 0x7f2e300b40d0 2024-04-26T21:35:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.367+0000 7f2e3624c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2e300d1a00 0x7f2e300d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54738/0 (socket says 172.21.15.40:54738) 2024-04-26T21:35:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.367+0000 7f2e3624c640 1 -- 172.21.15.40:0/2468972834 learned_addr learned my addr 172.21.15.40:0/2468972834 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1092187492 0 0) 0x7f2e300d0a10 con 0x7f2e300d1a00 2024-04-26T21:35:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e1c003520 con 0x7f2e300d1a00 2024-04-26T21:35:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e35a4b640 1 --2- 172.21.15.40:0/2468972834 >> v2:172.21.15.184:3300/0 conn(0x7f2e300b40d0 0x7f2e301625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2642225409 0 0) 0x7f2e1c003520 con 0x7f2e300d1a00 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 >> v2:172.21.15.184:3300/0 conn(0x7f2e300b40d0 msgr2=0x7f2e301625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 --2- 172.21.15.40:0/2468972834 >> v2:172.21.15.184:3300/0 conn(0x7f2e300b40d0 0x7f2e301625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 >> v1:172.21.15.184:6789/0 conn(0x7f2e30162b30 legacy=0x7f2e300b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e30162f80 con 0x7f2e300d1a00 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f2e2c001670 con 0x7f2e300d1a00 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 >> v1:172.21.15.40:6789/0 conn(0x7f2e300d1a00 legacy=0x7f2e300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e1c0039b0 0x7f2e1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c004800 0x7f2e1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c0051c0 0x7f2e1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300d0fd0 con 0x7f2e1c0039b0 2024-04-26T21:35:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c00c160 con 0x7f2e1c0051c0 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1c00c360 con 0x7f2e1c004800 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e3624c640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e1c0039b0 0x7f2e1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e36a4d640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c0051c0 0x7f2e1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.368+0000 7f2e35a4b640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c004800 0x7f2e1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3624c640 1 -- 172.21.15.40:0/2468972834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c004800 msgr2=0x7f2e1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3624c640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c004800 0x7f2e1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3624c640 1 -- 172.21.15.40:0/2468972834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c0051c0 msgr2=0x7f2e1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3624c640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c0051c0 0x7f2e1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e36a4d640 1 -- 172.21.15.40:0/2468972834 reap_dead start 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3624c640 1 -- 172.21.15.40:0/2468972834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c003520 con 0x7f2e1c0039b0 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3624c640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e1c0039b0 0x7f2e1c003dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00c240 tx=0x7f2e1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7862fc7073cc4dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c007d10 con 0x7f2e1c0039b0 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2c005ad0 con 0x7f2e1c0039b0 2024-04-26T21:35:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.369+0000 7f2e3524a640 1 -- 172.21.15.40:0/2468972834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c007070 con 0x7f2e1c0039b0 2024-04-26T21:35:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.370+0000 7f2e384b0640 1 -- 172.21.15.40:0/2468972834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e1c0039b0 msgr2=0x7f2e1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.370+0000 7f2e384b0640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e1c0039b0 0x7f2e1c003dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00c240 tx=0x7f2e1c005830 comp rx=0 tx=0).stop 2024-04-26T21:35:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.370+0000 7f2e384b0640 1 -- 172.21.15.40:0/2468972834 shutdown_connections 2024-04-26T21:35:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.370+0000 7f2e384b0640 1 --2- 172.21.15.40:0/2468972834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e1c0039b0 0x7f2e1c003dd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.370+0000 7f2e384b0640 1 -- 172.21.15.40:0/2468972834 >> 172.21.15.40:0/2468972834 conn(0x7f2e300d4fd0 msgr2=0x7f2e300d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.370+0000 7f2e384b0640 1 -- 172.21.15.40:0/2468972834 shutdown_connections 2024-04-26T21:35:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.370+0000 7f2e384b0640 1 -- 172.21.15.40:0/2468972834 wait complete. 2024-04-26T21:35:57.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.371+0000 7f2e384b0640 1 Processor -- start 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.371+0000 7f2e384b0640 1 -- start start 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e384b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c0039b0 0x7f2e300cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e384b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30153450 0x7f2e3015a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e384b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e300c86f0 0x7f2e300b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e384b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30162770 con 0x7f2e300c86f0 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e384b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301625f0 con 0x7f2e30153450 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e384b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e301628f0 con 0x7f2e1c0039b0 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e36a4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e300c86f0 0x7f2e300b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e36a4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e300c86f0 0x7f2e300b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55838/0 (socket says 172.21.15.40:55838) 2024-04-26T21:35:57.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e36a4d640 1 -- 172.21.15.40:0/1365089070 learned_addr learned my addr 172.21.15.40:0/1365089070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e3624c640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c0039b0 0x7f2e300cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e36a4d640 1 -- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c0039b0 msgr2=0x7f2e300cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e35a4b640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30153450 0x7f2e3015a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e36a4d640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c0039b0 0x7f2e300cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e36a4d640 1 -- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30153450 msgr2=0x7f2e3015a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e36a4d640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30153450 0x7f2e3015a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e36a4d640 1 -- 172.21.15.40:0/1365089070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3015b140 con 0x7f2e300c86f0 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e3624c640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c0039b0 0x7f2e300cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.372+0000 7f2e35a4b640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30153450 0x7f2e3015a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.373+0000 7f2e36a4d640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e300c86f0 0x7f2e300b5a10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800f750 tx=0x7f2e2800fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.373+0000 7f2e26ffd640 1 -- 172.21.15.40:0/1365089070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2801a070 con 0x7f2e300c86f0 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.373+0000 7f2e26ffd640 1 -- 172.21.15.40:0/1365089070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e28006d60 con 0x7f2e300c86f0 2024-04-26T21:35:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.373+0000 7f2e26ffd640 1 -- 172.21.15.40:0/1365089070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28003bb0 con 0x7f2e300c86f0 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.373+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3015b420 con 0x7f2e300c86f0 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.373+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3015b9e0 con 0x7f2e300c86f0 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.374+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e300694f0 con 0x7f2e300c86f0 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.374+0000 7f2e26ffd640 1 -- 172.21.15.40:0/1365089070 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e2801d070 con 0x7f2e300c86f0 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.375+0000 7f2e26ffd640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e1406c8f0 0x7f2e1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.375+0000 7f2e26ffd640 1 -- 172.21.15.40:0/1365089070 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2e2808b470 con 0x7f2e300c86f0 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.375+0000 7f2e3624c640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e1406c8f0 0x7f2e1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.376+0000 7f2e3624c640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e1406c8f0 0x7f2e1406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c033000 tx=0x7f2e2c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.379+0000 7f2e26ffd640 1 -- 172.21.15.40:0/1365089070 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e280594a0 con 0x7f2e300c86f0 2024-04-26T21:35:57.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.562+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2e300d55c0 con 0x7f2e300c86f0 2024-04-26T21:35:57.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.562+0000 7f2e26ffd640 1 -- 172.21.15.40:0/1365089070 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f2e28058da0 con 0x7f2e300c86f0 2024-04-26T21:35:57.563 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:57.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e1406c8f0 msgr2=0x7f2e1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e1406c8f0 0x7f2e1406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c033000 tx=0x7f2e2c02e000 comp rx=0 tx=0).stop 2024-04-26T21:35:57.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e300c86f0 msgr2=0x7f2e300b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e300c86f0 0x7f2e300b5a10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800f750 tx=0x7f2e2800fc10 comp rx=0 tx=0).stop 2024-04-26T21:35:57.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 shutdown_connections 2024-04-26T21:35:57.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e1406c8f0 0x7f2e1406edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e300c86f0 0x7f2e300b5a10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30153450 0x7f2e3015a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 --2- 172.21.15.40:0/1365089070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e1c0039b0 0x7f2e300cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 >> 172.21.15.40:0/1365089070 conn(0x7f2e300d4fd0 msgr2=0x7f2e300b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:57.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 shutdown_connections 2024-04-26T21:35:57.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.564+0000 7f2e384b0640 1 -- 172.21.15.40:0/1365089070 wait complete. 2024-04-26T21:35:57.598 INFO:tasks.cephfs_test_runner:Starting test: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2024-04-26T21:35:57.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:35:57.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.699+0000 7febf366d640 1 Processor -- start 2024-04-26T21:35:57.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.699+0000 7febf366d640 1 -- start start 2024-04-26T21:35:57.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf366d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7febec162810 0x7febec070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf366d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febec0d10d0 con 0x7febec0b2340 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf366d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febec0d1370 con 0x7febec0d1b30 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf366d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7febec0d0750 con 0x7febec162810 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf1409640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7febec162810 0x7febec070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf0c08640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7febec0b2340 0x7febec0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54744/0 (socket says 172.21.15.40:54744) 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf0c08640 1 -- 172.21.15.40:0/4009786364 learned_addr learned my addr 172.21.15.40:0/4009786364 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf1409640 1 -- 172.21.15.40:0/4009786364 >> v1:172.21.15.184:6789/0 conn(0x7febec0d1b30 legacy=0x7febec070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf1409640 1 -- 172.21.15.40:0/4009786364 >> v1:172.21.15.40:6789/0 conn(0x7febec0b2340 legacy=0x7febec0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.700+0000 7febf1409640 1 -- 172.21.15.40:0/4009786364 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec0d01b0 con 0x7febec162810 2024-04-26T21:35:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.701+0000 7febf1409640 1 --2- 172.21.15.40:0/4009786364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec162810 0x7febec070350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7febe80097a0 tx=0x7febe802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f33b9a19d2757d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.701+0000 7febe3fff640 1 -- 172.21.15.40:0/4009786364 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8037070 con 0x7febec162810 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.701+0000 7febe3fff640 1 -- 172.21.15.40:0/4009786364 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe8032440 con 0x7febec162810 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.701+0000 7febe3fff640 1 -- 172.21.15.40:0/4009786364 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8032810 con 0x7febec162810 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.702+0000 7febf366d640 1 -- 172.21.15.40:0/4009786364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec162810 msgr2=0x7febec070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.702+0000 7febf366d640 1 --2- 172.21.15.40:0/4009786364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec162810 0x7febec070350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7febe80097a0 tx=0x7febe802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.702+0000 7febf366d640 1 -- 172.21.15.40:0/4009786364 shutdown_connections 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.702+0000 7febf366d640 1 --2- 172.21.15.40:0/4009786364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec162810 0x7febec070350 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.702+0000 7febf366d640 1 -- 172.21.15.40:0/4009786364 >> 172.21.15.40:0/4009786364 conn(0x7febec0d2da0 msgr2=0x7febec0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.702+0000 7febf366d640 1 -- 172.21.15.40:0/4009786364 shutdown_connections 2024-04-26T21:35:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.702+0000 7febf366d640 1 -- 172.21.15.40:0/4009786364 wait complete. 2024-04-26T21:35:57.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.703+0000 7febf366d640 1 Processor -- start 2024-04-26T21:35:57.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.703+0000 7febf366d640 1 -- start start 2024-04-26T21:35:57.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf366d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febec0b2340 0x7febec1613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf366d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec0d1b30 0x7febec0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf366d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febec162810 0x7febec15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf366d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7febec070890 con 0x7febec0d1b30 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf366d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febec0d16a0 con 0x7febec162810 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf366d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7febec0cdb00 con 0x7febec0b2340 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec0d1b30 0x7febec0caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec0d1b30 0x7febec0caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55850/0 (socket says 172.21.15.40:55850) 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 -- 172.21.15.40:0/2655696007 learned_addr learned my addr 172.21.15.40:0/2655696007 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf1c0a640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febec162810 0x7febec15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf1409640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febec0b2340 0x7febec1613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 -- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febec0b2340 msgr2=0x7febec1613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febec0b2340 0x7febec1613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 -- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febec162810 msgr2=0x7febec15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febec162810 0x7febec15ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 -- 172.21.15.40:0/2655696007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec0c82a0 con 0x7febec0d1b30 2024-04-26T21:35:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf1c0a640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febec162810 0x7febec15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febf0c08640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec0d1b30 0x7febec0caa10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c400 tx=0x7febdc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.704+0000 7febe1ffb640 1 -- 172.21.15.40:0/2655696007 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc017070 con 0x7febec0d1b30 2024-04-26T21:35:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.705+0000 7febe1ffb640 1 -- 172.21.15.40:0/2655696007 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febdc005ce0 con 0x7febec0d1b30 2024-04-26T21:35:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.705+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec0cb3f0 con 0x7febec0d1b30 2024-04-26T21:35:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.705+0000 7febe1ffb640 1 -- 172.21.15.40:0/2655696007 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc0072f0 con 0x7febec0d1b30 2024-04-26T21:35:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.705+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec0c8840 con 0x7febec0d1b30 2024-04-26T21:35:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.706+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febec0694f0 con 0x7febec0d1b30 2024-04-26T21:35:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.706+0000 7febe1ffb640 1 -- 172.21.15.40:0/2655696007 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febdc01a070 con 0x7febec0d1b30 2024-04-26T21:35:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.707+0000 7febe1ffb640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd006c8f0 0x7febd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.707+0000 7febe1ffb640 1 -- 172.21.15.40:0/2655696007 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7febdc08b360 con 0x7febec0d1b30 2024-04-26T21:35:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.707+0000 7febf1409640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd006c8f0 0x7febd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.708+0000 7febf1409640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd006c8f0 0x7febd006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febe8034000 tx=0x7febe802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:57.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.711+0000 7febe1ffb640 1 -- 172.21.15.40:0/2655696007 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7febdc059390 con 0x7febec0d1b30 2024-04-26T21:35:57.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.896+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febec0d24b0 con 0x7febec0d1b30 2024-04-26T21:35:57.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.897+0000 7febe1ffb640 1 -- 172.21.15.40:0/2655696007 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7febdc058c90 con 0x7febec0d1b30 2024-04-26T21:35:57.897 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:35:57.898 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:35:57.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd006c8f0 msgr2=0x7febd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd006c8f0 0x7febd006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febe8034000 tx=0x7febe802e000 comp rx=0 tx=0).stop 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec0d1b30 msgr2=0x7febec0caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec0d1b30 0x7febec0caa10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c400 tx=0x7febdc00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 shutdown_connections 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7febd006c8f0 0x7febd006edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febec162810 0x7febec15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febec0d1b30 0x7febec0caa10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 --2- 172.21.15.40:0/2655696007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febec0b2340 0x7febec1613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 >> 172.21.15.40:0/2655696007 conn(0x7febec0d2da0 msgr2=0x7febec070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 shutdown_connections 2024-04-26T21:35:57.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:57.899+0000 7febf366d640 1 -- 172.21.15.40:0/2655696007 wait complete. 2024-04-26T21:35:57.910 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:35:57.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:35:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.015+0000 7ffba7e99640 1 Processor -- start 2024-04-26T21:35:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.015+0000 7ffba7e99640 1 -- start start 2024-04-26T21:35:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba7e99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 0x7ffba00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba7e99640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffba00d3bc0 0x7ffba00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba7e99640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba00d1040 con 0x7ffba015f550 2024-04-26T21:35:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba7e99640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffba00c9430 con 0x7ffba00d3bc0 2024-04-26T21:35:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba7e99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00c9680 con 0x7ffba015f170 2024-04-26T21:35:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba6436640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffba00d3bc0 0x7ffba00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba6436640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffba00d3bc0 0x7ffba00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55852/0 (socket says 172.21.15.40:55852) 2024-04-26T21:35:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba6436640 1 -- 172.21.15.40:0/3909539247 learned_addr learned my addr 172.21.15.40:0/3909539247 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:58.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba6436640 1 -- 172.21.15.40:0/3909539247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 msgr2=0x7ffba00d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:58.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba5c35640 1 --2- 172.21.15.40:0/3909539247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 0x7ffba00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba6436640 1 --2- 172.21.15.40:0/3909539247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 0x7ffba00d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.016+0000 7ffba6436640 1 -- 172.21.15.40:0/3909539247 >> v1:172.21.15.184:6789/0 conn(0x7ffba015f550 legacy=0x7ffba00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:58.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.017+0000 7ffba6436640 1 -- 172.21.15.40:0/3909539247 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba00c9960 con 0x7ffba00d3bc0 2024-04-26T21:35:58.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.017+0000 7ffba6436640 1 --2- 172.21.15.40:0/3909539247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00d2cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009e50 tx=0x7ffb9802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf9d3c557bcf3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.017+0000 7ffba4c33640 1 -- 172.21.15.40:0/3909539247 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98036070 con 0x7ffba00d3bc0 2024-04-26T21:35:58.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.017+0000 7ffba4c33640 1 -- 172.21.15.40:0/3909539247 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb98031600 con 0x7ffba00d3bc0 2024-04-26T21:35:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.017+0000 7ffba4c33640 1 -- 172.21.15.40:0/3909539247 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb980319d0 con 0x7ffba00d3bc0 2024-04-26T21:35:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.018+0000 7ffba7e99640 1 -- 172.21.15.40:0/3909539247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 msgr2=0x7ffba00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.018+0000 7ffba7e99640 1 --2- 172.21.15.40:0/3909539247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00d2cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009e50 tx=0x7ffb9802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.018+0000 7ffba7e99640 1 -- 172.21.15.40:0/3909539247 shutdown_connections 2024-04-26T21:35:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.018+0000 7ffba7e99640 1 --2- 172.21.15.40:0/3909539247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00d2cb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.018+0000 7ffba7e99640 1 --2- 172.21.15.40:0/3909539247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 0x7ffba00d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.018+0000 7ffba7e99640 1 -- 172.21.15.40:0/3909539247 >> 172.21.15.40:0/3909539247 conn(0x7ffba00d4f60 msgr2=0x7ffba00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.018+0000 7ffba7e99640 1 -- 172.21.15.40:0/3909539247 shutdown_connections 2024-04-26T21:35:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.018+0000 7ffba7e99640 1 -- 172.21.15.40:0/3909539247 wait complete. 2024-04-26T21:35:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.019+0000 7ffba7e99640 1 Processor -- start 2024-04-26T21:35:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.019+0000 7ffba7e99640 1 -- start start 2024-04-26T21:35:58.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba7e99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00b9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba7e99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 0x7ffba015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba7e99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba00ceb10 0x7ffba00a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba7e99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffba00ca5c0 con 0x7ffba00d3bc0 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba7e99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffba00ca440 con 0x7ffba00ceb10 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba7e99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00ca740 con 0x7ffba015f170 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba5c35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba5c35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55858/0 (socket says 172.21.15.40:55858) 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba5c35640 1 -- 172.21.15.40:0/159612450 learned_addr learned my addr 172.21.15.40:0/159612450 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba6436640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba00ceb10 0x7ffba00a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba5434640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 0x7ffba015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba5c35640 1 -- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 msgr2=0x7ffba015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.020+0000 7ffba5c35640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 0x7ffba015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffba5c35640 1 -- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba00ceb10 msgr2=0x7ffba00a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffba5c35640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba00ceb10 0x7ffba00a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffba5c35640 1 -- 172.21.15.40:0/159612450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba00b9f60 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffba6436640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba00ceb10 0x7ffba00a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffba5c35640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00b9710 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00c820 tx=0x7ffb9c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffb967fc640 1 -- 172.21.15.40:0/159612450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c015070 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffb967fc640 1 -- 172.21.15.40:0/159612450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c004240 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba00ba240 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffb967fc640 1 -- 172.21.15.40:0/159612450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00ce10 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.021+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba00ba800 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.022+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba00694f0 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.023+0000 7ffb967fc640 1 -- 172.21.15.40:0/159612450 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb9c007830 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.023+0000 7ffb967fc640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb8406c8f0 0x7ffb8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.023+0000 7ffb967fc640 1 -- 172.21.15.40:0/159612450 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ffb9c08bea0 con 0x7ffba00d3bc0 2024-04-26T21:35:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.023+0000 7ffba5434640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb8406c8f0 0x7ffb8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.024+0000 7ffba5434640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb8406c8f0 0x7ffb8406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb90006fd0 tx=0x7ffb90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.027+0000 7ffb967fc640 1 -- 172.21.15.40:0/159612450 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb9c059e30 con 0x7ffba00d3bc0 2024-04-26T21:35:58.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.217+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffba00c8750 con 0x7ffba00d3bc0 2024-04-26T21:35:58.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.217+0000 7ffb967fc640 1 -- 172.21.15.40:0/159612450 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ffb9c059730 con 0x7ffba00d3bc0 2024-04-26T21:35:58.217 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:35:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb8406c8f0 msgr2=0x7ffb8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb8406c8f0 0x7ffb8406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb90006fd0 tx=0x7ffb90008040 comp rx=0 tx=0).stop 2024-04-26T21:35:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 msgr2=0x7ffba00b9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00b9710 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00c820 tx=0x7ffb9c00cce0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 shutdown_connections 2024-04-26T21:35:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffb8406c8f0 0x7ffb8406edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba00ceb10 0x7ffba00a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba015f170 0x7ffba015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 --2- 172.21.15.40:0/159612450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffba00d3bc0 0x7ffba00b9710 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 >> 172.21.15.40:0/159612450 conn(0x7ffba00d4f60 msgr2=0x7ffba00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:58.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 shutdown_connections 2024-04-26T21:35:58.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.219+0000 7ffba7e99640 1 -- 172.21.15.40:0/159612450 wait complete. 2024-04-26T21:35:58.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:35:58.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.334+0000 7f19bdefc640 1 Processor -- start 2024-04-26T21:35:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.334+0000 7f19bdefc640 1 -- start start 2024-04-26T21:35:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.335+0000 7f19bdefc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f19b80d1a00 0x7f19b80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.335+0000 7f19bdefc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80b40d0 0x7f19b80b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.335+0000 7f19bdefc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19b80d0a10 con 0x7f19b80d2320 2024-04-26T21:35:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.335+0000 7f19bdefc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f19b80d0cf0 con 0x7f19b80d1a00 2024-04-26T21:35:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.335+0000 7f19bdefc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19b80d0fd0 con 0x7f19b80b40d0 2024-04-26T21:35:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80b40d0 0x7f19b80b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80b40d0 0x7f19b80b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:46810/0 (socket says 172.21.15.40:46810) 2024-04-26T21:35:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b77fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f19b80d1a00 0x7f19b80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b7fff640 1 -- 172.21.15.40:0/3045864564 learned_addr learned my addr 172.21.15.40:0/3045864564 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b7fff640 1 -- 172.21.15.40:0/3045864564 >> v2:172.21.15.184:3300/0 conn(0x7f19b80d1a00 msgr2=0x7f19b80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b7fff640 1 --2- 172.21.15.40:0/3045864564 >> v2:172.21.15.184:3300/0 conn(0x7f19b80d1a00 0x7f19b80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b7fff640 1 -- 172.21.15.40:0/3045864564 >> v1:172.21.15.184:6789/0 conn(0x7f19b80d2320 legacy=0x7f19b81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b7fff640 1 -- 172.21.15.40:0/3045864564 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b80d40a0 con 0x7f19b80b40d0 2024-04-26T21:35:58.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.336+0000 7f19b77fe640 1 --2- 172.21.15.40:0/3045864564 >> v2:172.21.15.184:3300/0 conn(0x7f19b80d1a00 0x7f19b80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19b7fff640 1 --2- 172.21.15.40:0/3045864564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80b40d0 0x7f19b80b4b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009e50 tx=0x7f19b002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a0364d0a1ce237 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19b67fc640 1 -- 172.21.15.40:0/3045864564 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0036070 con 0x7f19b80b40d0 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19b67fc640 1 -- 172.21.15.40:0/3045864564 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b0031600 con 0x7f19b80b40d0 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19b67fc640 1 -- 172.21.15.40:0/3045864564 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b00319d0 con 0x7f19b80b40d0 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19bdefc640 1 -- 172.21.15.40:0/3045864564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80b40d0 msgr2=0x7f19b80b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3045864564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80b40d0 0x7f19b80b4b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009e50 tx=0x7f19b002a5f0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19bdefc640 1 -- 172.21.15.40:0/3045864564 shutdown_connections 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3045864564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80b40d0 0x7f19b80b4b00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3045864564 >> v2:172.21.15.184:3300/0 conn(0x7f19b80d1a00 0x7f19b80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19bdefc640 1 -- 172.21.15.40:0/3045864564 >> 172.21.15.40:0/3045864564 conn(0x7f19b80d4fd0 msgr2=0x7f19b80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19bdefc640 1 -- 172.21.15.40:0/3045864564 shutdown_connections 2024-04-26T21:35:58.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.337+0000 7f19bdefc640 1 -- 172.21.15.40:0/3045864564 wait complete. 2024-04-26T21:35:58.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.339+0000 7f19bdefc640 1 Processor -- start 2024-04-26T21:35:58.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.339+0000 7f19bdefc640 1 -- start start 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.339+0000 7f19bdefc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b80b40d0 0x7f19b81539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.339+0000 7f19bdefc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b80d1a00 0x7f19b80b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.339+0000 7f19bdefc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80d2320 0x7f19b80b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.339+0000 7f19bdefc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19b80b5040 con 0x7f19b80d1a00 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.339+0000 7f19bdefc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8162f50 con 0x7f19b80b40d0 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.339+0000 7f19bdefc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19b80b51c0 con 0x7f19b80d2320 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b80d1a00 0x7f19b80b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b80d1a00 0x7f19b80b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55870/0 (socket says 172.21.15.40:55870) 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 -- 172.21.15.40:0/3210553602 learned_addr learned my addr 172.21.15.40:0/3210553602 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b7fff640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80d2320 0x7f19b80b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b77fe640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b80b40d0 0x7f19b81539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 -- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80d2320 msgr2=0x7f19b80b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80d2320 0x7f19b80b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 -- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b80b40d0 msgr2=0x7f19b81539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b80b40d0 0x7f19b81539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 -- 172.21.15.40:0/3210553602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b80b8ce0 con 0x7f19b80d1a00 2024-04-26T21:35:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19b6ffd640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b80d1a00 0x7f19b80b28c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00c3c0 tx=0x7f19ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f199ffff640 1 -- 172.21.15.40:0/3210553602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0052d0 con 0x7f19b80d1a00 2024-04-26T21:35:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f199ffff640 1 -- 172.21.15.40:0/3210553602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ac005490 con 0x7f19b80d1a00 2024-04-26T21:35:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b80b8fc0 con 0x7f19b80d1a00 2024-04-26T21:35:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f199ffff640 1 -- 172.21.15.40:0/3210553602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac007260 con 0x7f19b80d1a00 2024-04-26T21:35:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.340+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b80b9580 con 0x7f19b80d1a00 2024-04-26T21:35:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.342+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b80694f0 con 0x7f19b80d1a00 2024-04-26T21:35:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.342+0000 7f199ffff640 1 -- 172.21.15.40:0/3210553602 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19ac007400 con 0x7f19b80d1a00 2024-04-26T21:35:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.342+0000 7f199ffff640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19a406c8f0 0x7f19a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.342+0000 7f199ffff640 1 -- 172.21.15.40:0/3210553602 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f19ac08b2d0 con 0x7f19b80d1a00 2024-04-26T21:35:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.343+0000 7f19b77fe640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19a406c8f0 0x7f19a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.343+0000 7f19b77fe640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19a406c8f0 0x7f19a406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009560 tx=0x7f19a8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.347+0000 7f199ffff640 1 -- 172.21.15.40:0/3210553602 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19ac058d40 con 0x7f19b80d1a00 2024-04-26T21:35:58.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.531+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19b806d770 con 0x7f19b80d1a00 2024-04-26T21:35:58.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.532+0000 7f199ffff640 1 -- 172.21.15.40:0/3210553602 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f19ac08b0f0 con 0x7f19b80d1a00 2024-04-26T21:35:58.532 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:58.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19a406c8f0 msgr2=0x7f19a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19a406c8f0 0x7f19a406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009560 tx=0x7f19a8009190 comp rx=0 tx=0).stop 2024-04-26T21:35:58.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b80d1a00 msgr2=0x7f19b80b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b80d1a00 0x7f19b80b28c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00c3c0 tx=0x7f19ac00c880 comp rx=0 tx=0).stop 2024-04-26T21:35:58.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 shutdown_connections 2024-04-26T21:35:58.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19a406c8f0 0x7f19a406edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b80d2320 0x7f19b80b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b80d1a00 0x7f19b80b28c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 --2- 172.21.15.40:0/3210553602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b80b40d0 0x7f19b81539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 >> 172.21.15.40:0/3210553602 conn(0x7f19b80d4fd0 msgr2=0x7f19b80b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 shutdown_connections 2024-04-26T21:35:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.534+0000 7f19bdefc640 1 -- 172.21.15.40:0/3210553602 wait complete. 2024-04-26T21:35:58.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:35:58.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.649+0000 7f12b3ac5640 1 Processor -- start 2024-04-26T21:35:58.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.649+0000 7f12b3ac5640 1 -- start start 2024-04-26T21:35:58.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b3ac5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f12ac15f170 0x7f12ac0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b3ac5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12ac0d1040 con 0x7f12ac15f550 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b3ac5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12ac0c9680 con 0x7f12ac0d3bc0 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b3ac5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f12ac0c9430 con 0x7f12ac15f170 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b1060640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f12ac15f550 0x7f12ac0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34872/0 (socket says 172.21.15.40:34872) 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b1060640 1 -- 172.21.15.40:0/2401263265 learned_addr learned my addr 172.21.15.40:0/2401263265 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b1861640 1 --2- 172.21.15.40:0/2401263265 >> v2:172.21.15.184:3300/0 conn(0x7f12ac15f170 0x7f12ac0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b085f640 1 -- 172.21.15.40:0/2401263265 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2593309063 0 0) 0x7f129c001110 con 0x7f12ac15f550 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b085f640 1 -- 172.21.15.40:0/2401263265 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1298003520 con 0x7f12ac15f550 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b085f640 1 -- 172.21.15.40:0/2401263265 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 703035838 0 0) 0x7f1298003520 con 0x7f12ac15f550 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b1861640 1 -- 172.21.15.40:0/2401263265 >> v1:172.21.15.184:6789/0 conn(0x7f12ac0d3bc0 legacy=0x7f12ac0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b1861640 1 -- 172.21.15.40:0/2401263265 >> v1:172.21.15.40:6789/0 conn(0x7f12ac15f550 legacy=0x7f12ac0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.650+0000 7f12b1861640 1 -- 172.21.15.40:0/2401263265 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac0c9960 con 0x7f12ac15f170 2024-04-26T21:35:58.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.651+0000 7f12b1861640 1 --2- 172.21.15.40:0/2401263265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 0x7f12ac0d5a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0c95b0 tx=0x7f12a802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95050f422a47937a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.651+0000 7f12b085f640 1 -- 172.21.15.40:0/2401263265 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8037070 con 0x7f12ac15f170 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.651+0000 7f12b085f640 1 -- 172.21.15.40:0/2401263265 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a802ac70 con 0x7f12ac15f170 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.652+0000 7f12b085f640 1 -- 172.21.15.40:0/2401263265 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c001840 con 0x7f12ac15f170 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.652+0000 7f12b3ac5640 1 -- 172.21.15.40:0/2401263265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 msgr2=0x7f12ac0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.652+0000 7f12b3ac5640 1 --2- 172.21.15.40:0/2401263265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 0x7f12ac0d5a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0c95b0 tx=0x7f12a802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.652+0000 7f12b3ac5640 1 -- 172.21.15.40:0/2401263265 shutdown_connections 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.652+0000 7f12b3ac5640 1 --2- 172.21.15.40:0/2401263265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 0x7f12ac0d5a80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.652+0000 7f12b3ac5640 1 -- 172.21.15.40:0/2401263265 >> 172.21.15.40:0/2401263265 conn(0x7f12ac0d4f60 msgr2=0x7f12ac0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.652+0000 7f12b3ac5640 1 -- 172.21.15.40:0/2401263265 shutdown_connections 2024-04-26T21:35:58.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.652+0000 7f12b3ac5640 1 -- 172.21.15.40:0/2401263265 wait complete. 2024-04-26T21:35:58.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.653+0000 7f12b3ac5640 1 Processor -- start 2024-04-26T21:35:58.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.653+0000 7f12b3ac5640 1 -- start start 2024-04-26T21:35:58.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b3ac5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d3bc0 0x7f12ac158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b3ac5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 0x7f12ac0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b3ac5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0b28b0 0x7f12ac0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b3ac5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac0d2e30 con 0x7f12ac0b28b0 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b3ac5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac0d2cb0 con 0x7f12ac15f170 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b3ac5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac0d2fb0 con 0x7f12ac0d3bc0 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b2062640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0b28b0 0x7f12ac0b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b2062640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0b28b0 0x7f12ac0b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45412/0 (socket says 172.21.15.40:45412) 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b2062640 1 -- 172.21.15.40:0/1104261459 learned_addr learned my addr 172.21.15.40:0/1104261459 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b1060640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 0x7f12ac0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b1861640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d3bc0 0x7f12ac158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b2062640 1 -- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d3bc0 msgr2=0x7f12ac158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b2062640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d3bc0 0x7f12ac158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.654+0000 7f12b2062640 1 -- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 msgr2=0x7f12ac0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12b2062640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 0x7f12ac0b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12b2062640 1 -- 172.21.15.40:0/1104261459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac09ed00 con 0x7f12ac0b28b0 2024-04-26T21:35:58.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12b1060640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 0x7f12ac0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:58.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12b1861640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d3bc0 0x7f12ac158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:58.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12b2062640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0b28b0 0x7f12ac0b2d60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f12a400c7d0 tx=0x7f12a400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12a27fc640 1 -- 172.21.15.40:0/1104261459 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4003b70 con 0x7f12ac0b28b0 2024-04-26T21:35:58.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12a27fc640 1 -- 172.21.15.40:0/1104261459 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a4003d30 con 0x7f12ac0b28b0 2024-04-26T21:35:58.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac09efa0 con 0x7f12ac0b28b0 2024-04-26T21:35:58.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12a27fc640 1 -- 172.21.15.40:0/1104261459 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a400aa90 con 0x7f12ac0b28b0 2024-04-26T21:35:58.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.655+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac0a2c20 con 0x7f12ac0b28b0 2024-04-26T21:35:58.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.656+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac0694f0 con 0x7f12ac0b28b0 2024-04-26T21:35:58.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.656+0000 7f12a27fc640 1 -- 172.21.15.40:0/1104261459 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12a400ac30 con 0x7f12ac0b28b0 2024-04-26T21:35:58.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.657+0000 7f12a27fc640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.657+0000 7f12a27fc640 1 -- 172.21.15.40:0/1104261459 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f12a408bdd0 con 0x7f12ac0b28b0 2024-04-26T21:35:58.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.657+0000 7f12b1861640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.658+0000 7f12b1861640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12a8034000 tx=0x7f12a802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.661+0000 7f12a27fc640 1 -- 172.21.15.40:0/1104261459 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f12a4059e90 con 0x7f12ac0b28b0 2024-04-26T21:35:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.847+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f12ac06d390 con 0x7f12ac0b28b0 2024-04-26T21:35:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.847+0000 7f12a27fc640 1 -- 172.21.15.40:0/1104261459 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f12a408d020 con 0x7f12ac0b28b0 2024-04-26T21:35:58.847 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:35:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 msgr2=0x7f129006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12a8034000 tx=0x7f12a802e000 comp rx=0 tx=0).stop 2024-04-26T21:35:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0b28b0 msgr2=0x7f12ac0b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0b28b0 0x7f12ac0b2d60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f12a400c7d0 tx=0x7f12a400cc90 comp rx=0 tx=0).stop 2024-04-26T21:35:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 shutdown_connections 2024-04-26T21:35:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0b28b0 0x7f12ac0b2d60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac15f170 0x7f12ac0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 --2- 172.21.15.40:0/1104261459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d3bc0 0x7f12ac158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 >> 172.21.15.40:0/1104261459 conn(0x7f12ac0d4f60 msgr2=0x7f12ac06d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 shutdown_connections 2024-04-26T21:35:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.849+0000 7f12b3ac5640 1 -- 172.21.15.40:0/1104261459 wait complete. 2024-04-26T21:35:58.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:35:58.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.963+0000 7fae72607640 1 Processor -- start 2024-04-26T21:35:58.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.963+0000 7fae72607640 1 -- start start 2024-04-26T21:35:58.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.964+0000 7fae72607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c0d1240 0x7fae6c0d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.964+0000 7fae72607640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fae6c0b4360 0x7fae6c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.964+0000 7fae72607640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae6c0d0d60 0x7fae6c0cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.964+0000 7fae72607640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fae6c0cf210 con 0x7fae6c0d0d60 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.964+0000 7fae72607640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fae6c0d1b60 con 0x7fae6c0b4360 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.964+0000 7fae72607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c0ce5e0 con 0x7fae6c0d1240 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.964+0000 7fae70ba4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae6c0d0d60 0x7fae6c0cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.964+0000 7fae70ba4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae6c0d0d60 0x7fae6c0cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45420/0 (socket says 172.21.15.40:45420) 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.965+0000 7fae70ba4640 1 -- 172.21.15.40:0/2754842958 learned_addr learned my addr 172.21.15.40:0/2754842958 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.965+0000 7fae6bfff640 1 --2- 172.21.15.40:0/2754842958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c0d1240 0x7fae6c0d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.965+0000 7fae6b7fe640 1 --2- 172.21.15.40:0/2754842958 >> v2:172.21.15.184:3300/0 conn(0x7fae6c0b4360 0x7fae6c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.965+0000 7fae70ba4640 1 -- 172.21.15.40:0/2754842958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c0d1240 msgr2=0x7fae6c0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.965+0000 7fae70ba4640 1 --2- 172.21.15.40:0/2754842958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c0d1240 0x7fae6c0d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.965+0000 7fae70ba4640 1 -- 172.21.15.40:0/2754842958 >> v2:172.21.15.184:3300/0 conn(0x7fae6c0b4360 msgr2=0x7fae6c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.965+0000 7fae70ba4640 1 --2- 172.21.15.40:0/2754842958 >> v2:172.21.15.184:3300/0 conn(0x7fae6c0b4360 0x7fae6c0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.965+0000 7fae70ba4640 1 -- 172.21.15.40:0/2754842958 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c0ce8c0 con 0x7fae6c0d0d60 2024-04-26T21:35:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae70ba4640 1 --2- 172.21.15.40:0/2754842958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c0cfdb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fae6c06c520 tx=0x7fae6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffeb5d116ac2c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae6affd640 1 -- 172.21.15.40:0/2754842958 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64036070 con 0x7fae6c0d0d60 2024-04-26T21:35:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae6affd640 1 -- 172.21.15.40:0/2754842958 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae64031600 con 0x7fae6c0d0d60 2024-04-26T21:35:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae6affd640 1 -- 172.21.15.40:0/2754842958 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640319d0 con 0x7fae6c0d0d60 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 -- 172.21.15.40:0/2754842958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 msgr2=0x7fae6c0cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 --2- 172.21.15.40:0/2754842958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c0cfdb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fae6c06c520 tx=0x7fae6402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 -- 172.21.15.40:0/2754842958 shutdown_connections 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 --2- 172.21.15.40:0/2754842958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c0cfdb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 --2- 172.21.15.40:0/2754842958 >> v2:172.21.15.184:3300/0 conn(0x7fae6c0b4360 0x7fae6c0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 --2- 172.21.15.40:0/2754842958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c0d1240 0x7fae6c0d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 -- 172.21.15.40:0/2754842958 >> 172.21.15.40:0/2754842958 conn(0x7fae6c0d2e60 msgr2=0x7fae6c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 -- 172.21.15.40:0/2754842958 shutdown_connections 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.966+0000 7fae72607640 1 -- 172.21.15.40:0/2754842958 wait complete. 2024-04-26T21:35:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.967+0000 7fae72607640 1 Processor -- start 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae72607640 1 -- start start 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae72607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c09ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae72607640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c0d1240 0x7fae6c152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae72607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c15c7a0 0x7fae6c09d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae72607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0c6610 con 0x7fae6c0d0d60 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae72607640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0cf460 con 0x7fae6c0d1240 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae72607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c0c6790 con 0x7fae6c15c7a0 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae6bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c09ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae6bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c09ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45426/0 (socket says 172.21.15.40:45426) 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.968+0000 7fae6bfff640 1 -- 172.21.15.40:0/2835226395 learned_addr learned my addr 172.21.15.40:0/2835226395 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae6b7fe640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c0d1240 0x7fae6c152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae70ba4640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c15c7a0 0x7fae6c09d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae6bfff640 1 -- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c15c7a0 msgr2=0x7fae6c09d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae6bfff640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c15c7a0 0x7fae6c09d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae6bfff640 1 -- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c0d1240 msgr2=0x7fae6c152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae6bfff640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c0d1240 0x7fae6c152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae6bfff640 1 -- 172.21.15.40:0/2835226395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c0b54e0 con 0x7fae6c0d0d60 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae70ba4640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c15c7a0 0x7fae6c09d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae6b7fe640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c0d1240 0x7fae6c152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae6bfff640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c09ccc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00e870 tx=0x7fae5c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae68ff9640 1 -- 172.21.15.40:0/2835226395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c017070 con 0x7fae6c0d0d60 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae68ff9640 1 -- 172.21.15.40:0/2835226395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c00c8e0 con 0x7fae6c0d0d60 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c0b57c0 con 0x7fae6c0d0d60 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae68ff9640 1 -- 172.21.15.40:0/2835226395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00ccd0 con 0x7fae6c0d0d60 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.969+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c0b5d80 con 0x7fae6c0d0d60 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.970+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c0d1920 con 0x7fae6c0d0d60 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.971+0000 7fae68ff9640 1 -- 172.21.15.40:0/2835226395 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae5c01a070 con 0x7fae6c0d0d60 2024-04-26T21:35:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.971+0000 7fae68ff9640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5806c920 0x7fae5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.971+0000 7fae68ff9640 1 -- 172.21.15.40:0/2835226395 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fae5c08c0d0 con 0x7fae6c0d0d60 2024-04-26T21:35:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.971+0000 7fae6b7fe640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5806c920 0x7fae5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.972+0000 7fae6b7fe640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5806c920 0x7fae5806ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fae6c153d10 tx=0x7fae60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:58.976+0000 7fae68ff9640 1 -- 172.21.15.40:0/2835226395 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae6c0d1920 con 0x7fae6c0d0d60 2024-04-26T21:35:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.161+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fae6c0694f0 con 0x7fae6c0d0d60 2024-04-26T21:35:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.161+0000 7fae68ff9640 1 -- 172.21.15.40:0/2835226395 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fae5c05a100 con 0x7fae6c0d0d60 2024-04-26T21:35:59.161 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:35:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5806c920 msgr2=0x7fae5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5806c920 0x7fae5806ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fae6c153d10 tx=0x7fae60008040 comp rx=0 tx=0).stop 2024-04-26T21:35:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 msgr2=0x7fae6c09ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c09ccc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00e870 tx=0x7fae5c00ed30 comp rx=0 tx=0).stop 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 shutdown_connections 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5806c920 0x7fae5806ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae6c15c7a0 0x7fae6c09d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c0d1240 0x7fae6c152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 --2- 172.21.15.40:0/2835226395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d0d60 0x7fae6c09ccc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 >> 172.21.15.40:0/2835226395 conn(0x7fae6c0d2e60 msgr2=0x7fae6c15e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 shutdown_connections 2024-04-26T21:35:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.163+0000 7fae72607640 1 -- 172.21.15.40:0/2835226395 wait complete. 2024-04-26T21:35:59.174 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-04-26T21:35:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.275+0000 7fb3d5fc1640 1 Processor -- start 2024-04-26T21:35:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.275+0000 7fb3d5fc1640 1 -- start start 2024-04-26T21:35:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.275+0000 7fb3d5fc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d015f5a0 0x7fb3d00d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.275+0000 7fb3d5fc1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3d00d2830 0x7fb3d0162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.275+0000 7fb3d5fc1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d00d1a70 con 0x7fb3d00d3c20 2024-04-26T21:35:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.275+0000 7fb3d5fc1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d0163d30 con 0x7fb3d00d2830 2024-04-26T21:35:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.275+0000 7fb3d5fc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d00d1d50 con 0x7fb3d015f5a0 2024-04-26T21:35:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3ceffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3d00d2830 0x7fb3d0162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3cffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb3d00d3c20 0x7fb3d015f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34878/0 (socket says 172.21.15.40:34878) 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3cffff640 1 -- 172.21.15.40:0/3572534455 learned_addr learned my addr 172.21.15.40:0/3572534455 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/3572534455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d015f5a0 0x7fb3d00d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3ceffd640 1 -- 172.21.15.40:0/3572534455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d015f5a0 msgr2=0x7fb3d00d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/3572534455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d015f5a0 0x7fb3d00d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3ceffd640 1 -- 172.21.15.40:0/3572534455 >> v1:172.21.15.40:6789/0 conn(0x7fb3d00d3c20 legacy=0x7fb3d015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3ce7fc640 1 -- 172.21.15.40:0/3572534455 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450918373 0 0) 0x7fb3d00d1a70 con 0x7fb3d00d3c20 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3ceffd640 1 -- 172.21.15.40:0/3572534455 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d00d2030 con 0x7fb3d00d2830 2024-04-26T21:35:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.276+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/3572534455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d015f5a0 0x7fb3d00d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/3572534455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d0162f00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40097a0 tx=0x7fb3c402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa706b376b603791 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3ce7fc640 1 -- 172.21.15.40:0/3572534455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4037070 con 0x7fb3d00d2830 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3ce7fc640 1 -- 172.21.15.40:0/3572534455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c402ae50 con 0x7fb3d00d2830 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3ce7fc640 1 -- 172.21.15.40:0/3572534455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4032660 con 0x7fb3d00d2830 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/3572534455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 msgr2=0x7fb3d0162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/3572534455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d0162f00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40097a0 tx=0x7fb3c402a100 comp rx=0 tx=0).stop 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/3572534455 shutdown_connections 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/3572534455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d0162f00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/3572534455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d015f5a0 0x7fb3d00d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/3572534455 >> 172.21.15.40:0/3572534455 conn(0x7fb3d00d5190 msgr2=0x7fb3d00d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:35:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/3572534455 shutdown_connections 2024-04-26T21:35:59.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.277+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/3572534455 wait complete. 2024-04-26T21:35:59.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.279+0000 7fb3d5fc1640 1 Processor -- start 2024-04-26T21:35:59.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.279+0000 7fb3d5fc1640 1 -- start start 2024-04-26T21:35:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.279+0000 7fb3d5fc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d01eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.279+0000 7fb3d5fc1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d00d3c20 0x7fb3d01eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.279+0000 7fb3d5fc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d00c9cd0 0x7fb3d00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.279+0000 7fb3d5fc1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0163440 con 0x7fb3d00d2830 2024-04-26T21:35:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.279+0000 7fb3d5fc1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d00d4190 con 0x7fb3d00d3c20 2024-04-26T21:35:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.279+0000 7fb3d5fc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d01635c0 con 0x7fb3d00c9cd0 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d01eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d01eafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45438/0 (socket says 172.21.15.40:45438) 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 -- 172.21.15.40:0/1885613477 learned_addr learned my addr 172.21.15.40:0/1885613477 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cffff640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d00c9cd0 0x7fb3d00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d00d3c20 0x7fb3d01eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 -- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d00c9cd0 msgr2=0x7fb3d00ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d00c9cd0 0x7fb3d00ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 -- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d00d3c20 msgr2=0x7fb3d01eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d00d3c20 0x7fb3d01eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 -- 172.21.15.40:0/1885613477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d01582f0 con 0x7fb3d00d2830 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d01eafa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c7f0 tx=0x7fb3c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1885613477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0015070 con 0x7fb3d00d2830 2024-04-26T21:35:59.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1885613477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0007820 con 0x7fb3d00d2830 2024-04-26T21:35:59.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d01585d0 con 0x7fb3d00d2830 2024-04-26T21:35:59.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1885613477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0007bd0 con 0x7fb3d00d2830 2024-04-26T21:35:59.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.280+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d0158b90 con 0x7fb3d00d2830 2024-04-26T21:35:59.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.281+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d00694f0 con 0x7fb3d00d2830 2024-04-26T21:35:59.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.282+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1885613477 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3c0018070 con 0x7fb3d00d2830 2024-04-26T21:35:59.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.282+0000 7fb3b7fff640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:35:59.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.282+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1885613477 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb3c008bf80 con 0x7fb3d00d2830 2024-04-26T21:35:59.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.282+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:35:59.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.283+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4034000 tx=0x7fb3c402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:35:59.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.286+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1885613477 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3c00599f0 con 0x7fb3d00d2830 2024-04-26T21:35:59.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:35:59.543+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7fb3d0068d30 con 0x7fb3d00d2830 2024-04-26T21:36:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.293+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1885613477 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v69) ==== 137+0+0 (secure 0 0 0) 0x7fb3c005dcf0 con 0x7fb3d00d2830 2024-04-26T21:36:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 msgr2=0x7fb3bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4034000 tx=0x7fb3c402e000 comp rx=0 tx=0).stop 2024-04-26T21:36:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 msgr2=0x7fb3d01eafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d01eafa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c7f0 tx=0x7fb3c000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 shutdown_connections 2024-04-26T21:36:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d00c9cd0 0x7fb3d00ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d00d3c20 0x7fb3d01eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 --2- 172.21.15.40:0/1885613477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00d2830 0x7fb3d01eafa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 >> 172.21.15.40:0/1885613477 conn(0x7fb3d00d5190 msgr2=0x7fb3d006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 shutdown_connections 2024-04-26T21:36:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.296+0000 7fb3d5fc1640 1 -- 172.21.15.40:0/1885613477 wait complete. 2024-04-26T21:36:00.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:36:00.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.459+0000 7fef923b6640 1 Processor -- start 2024-04-26T21:36:00.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.460+0000 7fef923b6640 1 -- start start 2024-04-26T21:36:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.460+0000 7fef923b6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef8c0b40d0 0x7fef8c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.460+0000 7fef923b6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fef8c162b30 0x7fef8c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.460+0000 7fef923b6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef8c0d0a10 con 0x7fef8c0d1a00 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.460+0000 7fef923b6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fef8c0d0cf0 con 0x7fef8c0b40d0 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.460+0000 7fef923b6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fef8c0d0fd0 con 0x7fef8c162b30 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef8c0b40d0 0x7fef8c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fef8c0d1a00 0x7fef8c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34880/0 (socket says 172.21.15.40:34880) 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef8c0b40d0 0x7fef8c1625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45440/0 (socket says 172.21.15.40:45440) 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8bfff640 1 -- 172.21.15.40:0/950804393 learned_addr learned my addr 172.21.15.40:0/950804393 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef90953640 1 --2- 172.21.15.40:0/950804393 >> v2:172.21.15.184:3300/0 conn(0x7fef8c162b30 0x7fef8c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8b7fe640 1 -- 172.21.15.40:0/950804393 >> v2:172.21.15.184:3300/0 conn(0x7fef8c162b30 msgr2=0x7fef8c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8b7fe640 1 --2- 172.21.15.40:0/950804393 >> v2:172.21.15.184:3300/0 conn(0x7fef8c162b30 0x7fef8c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8b7fe640 1 -- 172.21.15.40:0/950804393 >> v1:172.21.15.40:6789/0 conn(0x7fef8c0d1a00 legacy=0x7fef8c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8b7fe640 1 -- 172.21.15.40:0/950804393 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c162f80 con 0x7fef8c0b40d0 2024-04-26T21:36:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.461+0000 7fef8affd640 1 -- 172.21.15.40:0/950804393 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1200638342 0 0) 0x7fef8c0d0a10 con 0x7fef8c0d1a00 2024-04-26T21:36:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef8b7fe640 1 --2- 172.21.15.40:0/950804393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0b40d0 0x7fef8c1625f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0b69a0 tx=0x7fef8002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccaee6a2aa9734ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef8affd640 1 -- 172.21.15.40:0/950804393 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80037070 con 0x7fef8c0b40d0 2024-04-26T21:36:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef8affd640 1 -- 172.21.15.40:0/950804393 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef80032440 con 0x7fef8c0b40d0 2024-04-26T21:36:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef8affd640 1 -- 172.21.15.40:0/950804393 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8002ae60 con 0x7fef8c0b40d0 2024-04-26T21:36:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef923b6640 1 -- 172.21.15.40:0/950804393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0b40d0 msgr2=0x7fef8c1625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef923b6640 1 --2- 172.21.15.40:0/950804393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0b40d0 0x7fef8c1625f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0b69a0 tx=0x7fef8002a110 comp rx=0 tx=0).stop 2024-04-26T21:36:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef923b6640 1 -- 172.21.15.40:0/950804393 shutdown_connections 2024-04-26T21:36:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef923b6640 1 --2- 172.21.15.40:0/950804393 >> v2:172.21.15.184:3300/0 conn(0x7fef8c162b30 0x7fef8c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef923b6640 1 --2- 172.21.15.40:0/950804393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0b40d0 0x7fef8c1625f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef923b6640 1 -- 172.21.15.40:0/950804393 >> 172.21.15.40:0/950804393 conn(0x7fef8c0d4fd0 msgr2=0x7fef8c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef923b6640 1 -- 172.21.15.40:0/950804393 shutdown_connections 2024-04-26T21:36:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.462+0000 7fef923b6640 1 -- 172.21.15.40:0/950804393 wait complete. 2024-04-26T21:36:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.464+0000 7fef923b6640 1 Processor -- start 2024-04-26T21:36:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.464+0000 7fef923b6640 1 -- start start 2024-04-26T21:36:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.464+0000 7fef923b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8c0b40d0 0x7fef8c1f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.464+0000 7fef923b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0d1a00 0x7fef8c0b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.464+0000 7fef923b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef8c162b30 0x7fef8c0c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef923b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c0d2670 con 0x7fef8c0d1a00 2024-04-26T21:36:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef923b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c0d24f0 con 0x7fef8c162b30 2024-04-26T21:36:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef923b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef8c0b5040 con 0x7fef8c0b40d0 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0d1a00 0x7fef8c0b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0d1a00 0x7fef8c0b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45444/0 (socket says 172.21.15.40:45444) 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 -- 172.21.15.40:0/297949370 learned_addr learned my addr 172.21.15.40:0/297949370 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef90953640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef8c162b30 0x7fef8c0c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8bfff640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8c0b40d0 0x7fef8c1f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 -- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8c0b40d0 msgr2=0x7fef8c1f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8c0b40d0 0x7fef8c1f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 -- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef8c162b30 msgr2=0x7fef8c0c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef8c162b30 0x7fef8c0c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 -- 172.21.15.40:0/297949370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c1f9450 con 0x7fef8c0d1a00 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8bfff640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8c0b40d0 0x7fef8c1f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef90953640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef8c162b30 0x7fef8c0c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef8b7fe640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0d1a00 0x7fef8c0b8590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef80009890 tx=0x7fef800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef88ff9640 1 -- 172.21.15.40:0/297949370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80037040 con 0x7fef8c0d1a00 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef88ff9640 1 -- 172.21.15.40:0/297949370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef80042070 con 0x7fef8c0d1a00 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c0b8ad0 con 0x7fef8c0d1a00 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef88ff9640 1 -- 172.21.15.40:0/297949370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80030bd0 con 0x7fef8c0d1a00 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.465+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c0b8120 con 0x7fef8c0d1a00 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.466+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8c0694f0 con 0x7fef8c0d1a00 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.467+0000 7fef88ff9640 1 -- 172.21.15.40:0/297949370 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef80047460 con 0x7fef8c0d1a00 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.467+0000 7fef88ff9640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef7806c8f0 0x7fef7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.467+0000 7fef88ff9640 1 -- 172.21.15.40:0/297949370 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fef800aca50 con 0x7fef8c0d1a00 2024-04-26T21:36:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.468+0000 7fef8bfff640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef7806c8f0 0x7fef7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.468+0000 7fef8bfff640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef7806c8f0 0x7fef7806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef7c000f60 tx=0x7fef7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:00.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.472+0000 7fef88ff9640 1 -- 172.21.15.40:0/297949370 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef80032e20 con 0x7fef8c0d1a00 2024-04-26T21:36:00.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.656+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fef8c0b78a0 con 0x7fef8c0d1a00 2024-04-26T21:36:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.661+0000 7fef88ff9640 1 -- 172.21.15.40:0/297949370 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fef8007aa80 con 0x7fef8c0d1a00 2024-04-26T21:36:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.663+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef7806c8f0 msgr2=0x7fef7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.663+0000 7fef923b6640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef7806c8f0 0x7fef7806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef7c000f60 tx=0x7fef7c009040 comp rx=0 tx=0).stop 2024-04-26T21:36:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.663+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0d1a00 msgr2=0x7fef8c0b8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.663+0000 7fef923b6640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0d1a00 0x7fef8c0b8590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef80009890 tx=0x7fef800046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.664+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 shutdown_connections 2024-04-26T21:36:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.664+0000 7fef923b6640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef7806c8f0 0x7fef7806edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.664+0000 7fef923b6640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef8c162b30 0x7fef8c0c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.664+0000 7fef923b6640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8c0d1a00 0x7fef8c0b8590 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.664+0000 7fef923b6640 1 --2- 172.21.15.40:0/297949370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8c0b40d0 0x7fef8c1f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.664+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 >> 172.21.15.40:0/297949370 conn(0x7fef8c0d4fd0 msgr2=0x7fef8c0d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.664+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 shutdown_connections 2024-04-26T21:36:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:00.664+0000 7fef923b6640 1 -- 172.21.15.40:0/297949370 wait complete. 2024-04-26T21:36:00.675 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:36:00.675 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:36:00.725 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:36:00.769 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:36:00.770 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:00.770 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:00.785 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:00.785 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:00.841 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:00.842 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:00.897 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:01.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.038+0000 7f18a28b0640 1 Processor -- start 2024-04-26T21:36:01.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.038+0000 7f18a28b0640 1 -- start start 2024-04-26T21:36:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f18a28b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f189c0b0380 0x7f189c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f18a28b0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189c0cfd80 con 0x7f189c06c950 2024-04-26T21:36:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f18a28b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189c0d0020 con 0x7f189c0b10b0 2024-04-26T21:36:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f18a28b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f189c0af8e0 con 0x7f189c0b0380 2024-04-26T21:36:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f18a0e4d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f189c06c950 0x7f189c15e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34886/0 (socket says 172.21.15.40:34886) 2024-04-26T21:36:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f18a0e4d640 1 -- 172.21.15.40:0/2218568646 learned_addr learned my addr 172.21.15.40:0/2218568646 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189bfff640 1 --2- 172.21.15.40:0/2218568646 >> v2:172.21.15.184:3300/0 conn(0x7f189c0b0380 0x7f189c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2264806881 0 0) 0x7f189c0cfd80 con 0x7f189c06c950 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1884003520 con 0x7f189c06c950 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3135147908 0 0) 0x7f1884003520 con 0x7f189c06c950 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 >> v2:172.21.15.184:3300/0 conn(0x7f189c0b0380 msgr2=0x7f189c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 --2- 172.21.15.40:0/2218568646 >> v2:172.21.15.184:3300/0 conn(0x7f189c0b0380 0x7f189c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 >> v1:172.21.15.184:6789/0 conn(0x7f189c0b10b0 legacy=0x7f189c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c0d0300 con 0x7f189c06c950 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1894001a10 con 0x7f189c06c950 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.039+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 >> v1:172.21.15.40:6789/0 conn(0x7f189c06c950 legacy=0x7f189c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189affd640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840039b0 0x7f1884003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189affd640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884004800 0x7f1884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189affd640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18840051c0 0x7f188400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f189c0af8e0 con 0x7f18840051c0 2024-04-26T21:36:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f188400c160 con 0x7f18840039b0 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f188400c360 con 0x7f1884004800 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f18a0e4d640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18840051c0 0x7f188400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189bfff640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840039b0 0x7f1884003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189b7fe640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884004800 0x7f1884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f18a0e4d640 1 -- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884004800 msgr2=0x7f1884004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f18a0e4d640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884004800 0x7f1884004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f18a0e4d640 1 -- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840039b0 msgr2=0x7f1884003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f18a0e4d640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840039b0 0x7f1884003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f18a0e4d640 1 -- 172.21.15.40:0/2218568646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1884003520 con 0x7f18840051c0 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.040+0000 7f189b7fe640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884004800 0x7f1884004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a0e4d640 1 -- 172.21.15.40:0/2218568646 reap_dead start 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a0e4d640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18840051c0 0x7f188400b9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f189c0cff50 tx=0x7f1884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848062587f720a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189400a4a0 con 0x7f18840051c0 2024-04-26T21:36:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189402f070 con 0x7f18840051c0 2024-04-26T21:36:01.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f189affd640 1 -- 172.21.15.40:0/2218568646 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189402bc70 con 0x7f18840051c0 2024-04-26T21:36:01.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a28b0640 1 -- 172.21.15.40:0/2218568646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18840051c0 msgr2=0x7f188400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a28b0640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18840051c0 0x7f188400b9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f189c0cff50 tx=0x7f1884005830 comp rx=0 tx=0).stop 2024-04-26T21:36:01.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a28b0640 1 -- 172.21.15.40:0/2218568646 shutdown_connections 2024-04-26T21:36:01.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a28b0640 1 --2- 172.21.15.40:0/2218568646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18840051c0 0x7f188400b9c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a28b0640 1 -- 172.21.15.40:0/2218568646 >> 172.21.15.40:0/2218568646 conn(0x7f189c0d1cf0 msgr2=0x7f189c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:01.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a28b0640 1 -- 172.21.15.40:0/2218568646 shutdown_connections 2024-04-26T21:36:01.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.041+0000 7f18a28b0640 1 -- 172.21.15.40:0/2218568646 wait complete. 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.042+0000 7f18a28b0640 1 Processor -- start 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.043+0000 7f18a28b0640 1 -- start start 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.043+0000 7f18a28b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18840051c0 0x7f189c1f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.043+0000 7f18a28b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c0cb8c0 0x7f189c0c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.043+0000 7f18a28b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1f5c60 0x7f189c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.043+0000 7f18a28b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f189c0b0760 con 0x7f189c1f5c60 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.043+0000 7f18a28b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f189c0d1980 con 0x7f189c0cb8c0 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.043+0000 7f18a28b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f189c0d15c0 con 0x7f18840051c0 2024-04-26T21:36:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1f5c60 0x7f189c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1f5c60 0x7f189c0b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45458/0 (socket says 172.21.15.40:45458) 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 -- 172.21.15.40:0/763276433 learned_addr learned my addr 172.21.15.40:0/763276433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f189bfff640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18840051c0 0x7f189c1f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f189b7fe640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c0cb8c0 0x7f189c0c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 -- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18840051c0 msgr2=0x7f189c1f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18840051c0 0x7f189c1f4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 -- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c0cb8c0 msgr2=0x7f189c0c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c0cb8c0 0x7f189c0c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 -- 172.21.15.40:0/763276433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1f5a40 con 0x7f189c1f5c60 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a0e4d640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1f5c60 0x7f189c0b2ac0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f18940012e0 tx=0x7f189402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f1898ff9640 1 -- 172.21.15.40:0/763276433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189400a4a0 con 0x7f189c1f5c60 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f1898ff9640 1 -- 172.21.15.40:0/763276433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1894004b60 con 0x7f189c1f5c60 2024-04-26T21:36:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c0cbd40 con 0x7f189c1f5c60 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f1898ff9640 1 -- 172.21.15.40:0/763276433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894034470 con 0x7f189c1f5c60 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.044+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1f5480 con 0x7f189c1f5c60 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.045+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c0694f0 con 0x7f189c1f5c60 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.046+0000 7f1898ff9640 1 -- 172.21.15.40:0/763276433 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1894007070 con 0x7f189c1f5c60 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.046+0000 7f1898ff9640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f188806c920 0x7f188806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.046+0000 7f1898ff9640 1 -- 172.21.15.40:0/763276433 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f18940abb30 con 0x7f189c1f5c60 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.046+0000 7f189bfff640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f188806c920 0x7f188806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.047+0000 7f189bfff640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f188806c920 0x7f188806ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f189c0b7fb0 tx=0x7f188c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.051+0000 7f1898ff9640 1 -- 172.21.15.40:0/763276433 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1894046060 con 0x7f189c1f5c60 2024-04-26T21:36:01.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.236+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f189c0694f0 con 0x7f189c1f5c60 2024-04-26T21:36:01.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.237+0000 7f1898ff9640 1 -- 172.21.15.40:0/763276433 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7f1894079bf0 con 0x7f189c1f5c60 2024-04-26T21:36:01.237 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:01.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f188806c920 msgr2=0x7f188806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f188806c920 0x7f188806ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f189c0b7fb0 tx=0x7f188c009190 comp rx=0 tx=0).stop 2024-04-26T21:36:01.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1f5c60 msgr2=0x7f189c0b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1f5c60 0x7f189c0b2ac0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f18940012e0 tx=0x7f189402d070 comp rx=0 tx=0).stop 2024-04-26T21:36:01.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 shutdown_connections 2024-04-26T21:36:01.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f188806c920 0x7f188806ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1f5c60 0x7f189c0b2ac0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c0cb8c0 0x7f189c0c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 --2- 172.21.15.40:0/763276433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18840051c0 0x7f189c1f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 >> 172.21.15.40:0/763276433 conn(0x7f189c0d1cf0 msgr2=0x7f189c0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:01.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 shutdown_connections 2024-04-26T21:36:01.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.239+0000 7f18a28b0640 1 -- 172.21.15.40:0/763276433 wait complete. 2024-04-26T21:36:01.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:36:01.251 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:36:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.355+0000 7f891602a640 1 Processor -- start 2024-04-26T21:36:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.355+0000 7f891602a640 1 -- start start 2024-04-26T21:36:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f891602a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f891015f120 0x7f891006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f891602a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f891006c950 0x7f89100af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f891602a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89100b49f0 con 0x7f89100b4480 2024-04-26T21:36:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f891602a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f891015f500 con 0x7f891015f120 2024-04-26T21:36:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f891602a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f89100ae8c0 con 0x7f891006c950 2024-04-26T21:36:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f890f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f891015f120 0x7f891006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f890f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f891015f120 0x7f891006f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45470/0 (socket says 172.21.15.40:45470) 2024-04-26T21:36:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f890f7fe640 1 -- 172.21.15.40:0/54852989 learned_addr learned my addr 172.21.15.40:0/54852989 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:01.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f890ffff640 1 --2- 172.21.15.40:0/54852989 >> v2:172.21.15.184:3300/0 conn(0x7f891006c950 0x7f89100af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f890f7fe640 1 -- 172.21.15.40:0/54852989 >> v2:172.21.15.184:3300/0 conn(0x7f891006c950 msgr2=0x7f89100af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:01.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f890f7fe640 1 --2- 172.21.15.40:0/54852989 >> v2:172.21.15.184:3300/0 conn(0x7f891006c950 0x7f89100af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f890f7fe640 1 -- 172.21.15.40:0/54852989 >> v1:172.21.15.184:6789/0 conn(0x7f89100b4480 legacy=0x7f891006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:01.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.356+0000 7f890f7fe640 1 -- 172.21.15.40:0/54852989 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89100b4cd0 con 0x7f891015f120 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.357+0000 7f890f7fe640 1 --2- 172.21.15.40:0/54852989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f891006f110 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8910070670 tx=0x7f890002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fc483b47e65f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.357+0000 7f890e7fc640 1 -- 172.21.15.40:0/54852989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900036070 con 0x7f891015f120 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.357+0000 7f890e7fc640 1 -- 172.21.15.40:0/54852989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8900031600 con 0x7f891015f120 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.357+0000 7f890e7fc640 1 -- 172.21.15.40:0/54852989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89000319d0 con 0x7f891015f120 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.358+0000 7f891602a640 1 -- 172.21.15.40:0/54852989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 msgr2=0x7f891006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.358+0000 7f891602a640 1 --2- 172.21.15.40:0/54852989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f891006f110 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8910070670 tx=0x7f890002a660 comp rx=0 tx=0).stop 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.358+0000 7f891602a640 1 -- 172.21.15.40:0/54852989 shutdown_connections 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.358+0000 7f891602a640 1 --2- 172.21.15.40:0/54852989 >> v2:172.21.15.184:3300/0 conn(0x7f891006c950 0x7f89100af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.358+0000 7f891602a640 1 --2- 172.21.15.40:0/54852989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f891006f110 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.358+0000 7f891602a640 1 -- 172.21.15.40:0/54852989 >> 172.21.15.40:0/54852989 conn(0x7f89100b3180 msgr2=0x7f89100b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.358+0000 7f891602a640 1 -- 172.21.15.40:0/54852989 shutdown_connections 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.358+0000 7f891602a640 1 -- 172.21.15.40:0/54852989 wait complete. 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.359+0000 7f891602a640 1 Processor -- start 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.359+0000 7f891602a640 1 -- start start 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f891602a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891006c950 0x7f891014ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f891602a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89100b4480 0x7f89100727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f891602a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f8910072d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f891602a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f891006f650 con 0x7f891015f120 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f891602a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89100aeba0 con 0x7f89100b4480 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f891602a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89100af570 con 0x7f891006c950 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f8910072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f8910072d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45476/0 (socket says 172.21.15.40:45476) 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 -- 172.21.15.40:0/1344878205 learned_addr learned my addr 172.21.15.40:0/1344878205 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890effd640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89100b4480 0x7f89100727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890f7fe640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891006c950 0x7f891014ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 -- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891006c950 msgr2=0x7f891014ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891006c950 0x7f891014ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 -- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89100b4480 msgr2=0x7f89100727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89100b4480 0x7f89100727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 -- 172.21.15.40:0/1344878205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89100c6490 con 0x7f891015f120 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890effd640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89100b4480 0x7f89100727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f890ffff640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f8910072d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f890800d5a0 tx=0x7f890800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.360+0000 7f88f7fff640 1 -- 172.21.15.40:0/1344878205 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89080125f0 con 0x7f891015f120 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.361+0000 7f88f7fff640 1 -- 172.21.15.40:0/1344878205 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8908019070 con 0x7f891015f120 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.361+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89100c6730 con 0x7f891015f120 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.361+0000 7f88f7fff640 1 -- 172.21.15.40:0/1344878205 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89080129d0 con 0x7f891015f120 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.361+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89100c6cf0 con 0x7f891015f120 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.362+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89100694f0 con 0x7f891015f120 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.362+0000 7f88f7fff640 1 -- 172.21.15.40:0/1344878205 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8908012b70 con 0x7f891015f120 2024-04-26T21:36:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.363+0000 7f88f7fff640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f88fc06c920 0x7f88fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.363+0000 7f88f7fff640 1 -- 172.21.15.40:0/1344878205 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f890808c2d0 con 0x7f891015f120 2024-04-26T21:36:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.363+0000 7f890f7fe640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f88fc06c920 0x7f88fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.363+0000 7f890f7fe640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f88fc06c920 0x7f88fc06ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8900033000 tx=0x7f8900009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.367+0000 7f88f7fff640 1 -- 172.21.15.40:0/1344878205 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f890805a300 con 0x7f891015f120 2024-04-26T21:36:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.558+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89100b09c0 con 0x7f891015f120 2024-04-26T21:36:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.558+0000 7f88f7fff640 1 -- 172.21.15.40:0/1344878205 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f890808c0f0 con 0x7f891015f120 2024-04-26T21:36:01.559 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:36:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f88fc06c920 msgr2=0x7f88fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f88fc06c920 0x7f88fc06ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8900033000 tx=0x7f8900009000 comp rx=0 tx=0).stop 2024-04-26T21:36:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 msgr2=0x7f8910072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f8910072d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f890800d5a0 tx=0x7f890800da60 comp rx=0 tx=0).stop 2024-04-26T21:36:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 shutdown_connections 2024-04-26T21:36:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f88fc06c920 0x7f88fc06ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891015f120 0x7f8910072d20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89100b4480 0x7f89100727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 --2- 172.21.15.40:0/1344878205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891006c950 0x7f891014ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 >> 172.21.15.40:0/1344878205 conn(0x7f89100b3180 msgr2=0x7f89100b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 shutdown_connections 2024-04-26T21:36:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.561+0000 7f891602a640 1 -- 172.21.15.40:0/1344878205 wait complete. 2024-04-26T21:36:01.572 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:36:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.676+0000 7fd56d425640 1 Processor -- start 2024-04-26T21:36:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.676+0000 7fd56d425640 1 -- start start 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd56d425640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd56815f2d0 0x7fd5681604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd56d425640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd568072f70 con 0x7fd56806fea0 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd56d425640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd568073250 con 0x7fd568072700 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd56d425640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd568074010 con 0x7fd56815f2d0 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5677fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd56806fea0 0x7fd5680ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34902/0 (socket says 172.21.15.40:34902) 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5677fe640 1 -- 172.21.15.40:0/976856842 learned_addr learned my addr 172.21.15.40:0/976856842 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd566ffd640 1 --2- 172.21.15.40:0/976856842 >> v2:172.21.15.184:3300/0 conn(0x7fd56815f2d0 0x7fd5681604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590631854 0 0) 0x7fd568072f70 con 0x7fd56806fea0 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd554003520 con 0x7fd56806fea0 2024-04-26T21:36:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 826524285 0 0) 0x7fd554003520 con 0x7fd56806fea0 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 >> v2:172.21.15.184:3300/0 conn(0x7fd56815f2d0 msgr2=0x7fd5681604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 --2- 172.21.15.40:0/976856842 >> v2:172.21.15.184:3300/0 conn(0x7fd56815f2d0 0x7fd5681604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 >> v1:172.21.15.184:6789/0 conn(0x7fd568072700 legacy=0x7fd56806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd568073530 con 0x7fd56806fea0 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fd5600019f0 con 0x7fd56806fea0 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 >> v1:172.21.15.40:6789/0 conn(0x7fd56806fea0 legacy=0x7fd5680ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540039b0 0x7fd554003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.677+0000 7fd5667fc640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554004800 0x7fd554004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5667fc640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5540051c0 0x7fd55400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd568074010 con 0x7fd5540051c0 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd55400c060 con 0x7fd554004800 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd55400c260 con 0x7fd5540039b0 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5677fe640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5540051c0 0x7fd55400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd566ffd640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540039b0 0x7fd554003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd55ffff640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554004800 0x7fd554004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5677fe640 1 -- 172.21.15.40:0/976856842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540039b0 msgr2=0x7fd554003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5677fe640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540039b0 0x7fd554003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5677fe640 1 -- 172.21.15.40:0/976856842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554004800 msgr2=0x7fd554004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5677fe640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554004800 0x7fd554004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.678+0000 7fd5677fe640 1 -- 172.21.15.40:0/976856842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd554003520 con 0x7fd5540051c0 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd5677fe640 1 -- 172.21.15.40:0/976856842 reap_dead start 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd5677fe640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5540051c0 0x7fd55400b9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd56000a860 tx=0x7fd554005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9576ea7d11c4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56000a480 con 0x7fd5540051c0 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56002f070 con 0x7fd5540051c0 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd5667fc640 1 -- 172.21.15.40:0/976856842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56002bcc0 con 0x7fd5540051c0 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd56d425640 1 -- 172.21.15.40:0/976856842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5540051c0 msgr2=0x7fd55400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd56d425640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5540051c0 0x7fd55400b9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd56000a860 tx=0x7fd554005830 comp rx=0 tx=0).stop 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd56d425640 1 -- 172.21.15.40:0/976856842 shutdown_connections 2024-04-26T21:36:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd56d425640 1 --2- 172.21.15.40:0/976856842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5540051c0 0x7fd55400b9c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd56d425640 1 -- 172.21.15.40:0/976856842 >> 172.21.15.40:0/976856842 conn(0x7fd5680d0be0 msgr2=0x7fd5680d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd56d425640 1 -- 172.21.15.40:0/976856842 shutdown_connections 2024-04-26T21:36:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.679+0000 7fd56d425640 1 -- 172.21.15.40:0/976856842 wait complete. 2024-04-26T21:36:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd56d425640 1 Processor -- start 2024-04-26T21:36:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd56d425640 1 -- start start 2024-04-26T21:36:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd56d425640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540051c0 0x7fd5680c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd56d425640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680c4620 0x7fd5680ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd56d425640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5680c4aa0 0x7fd568157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd56d425640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd56815f6b0 con 0x7fd5680c4aa0 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd56d425640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd568073af0 con 0x7fd5680c4620 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd56d425640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd568072b90 con 0x7fd5540051c0 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.681+0000 7fd5677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5680c4aa0 0x7fd568157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd5677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5680c4aa0 0x7fd568157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45490/0 (socket says 172.21.15.40:45490) 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd5677fe640 1 -- 172.21.15.40:0/1122523359 learned_addr learned my addr 172.21.15.40:0/1122523359 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd566ffd640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540051c0 0x7fd5680c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd55ffff640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680c4620 0x7fd5680ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd5677fe640 1 -- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540051c0 msgr2=0x7fd5680c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd5677fe640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540051c0 0x7fd5680c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd5677fe640 1 -- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680c4620 msgr2=0x7fd5680ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd5677fe640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680c4620 0x7fd5680ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd5677fe640 1 -- 172.21.15.40:0/1122523359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5680c6b50 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd5677fe640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5680c4aa0 0x7fd568157730 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd56000a980 tx=0x7fd568073420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd55f7fe640 1 -- 172.21.15.40:0/1122523359 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560044070 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd55f7fe640 1 -- 172.21.15.40:0/1122523359 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56002bdc0 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd568150220 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd55f7fe640 1 -- 172.21.15.40:0/1122523359 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56000a480 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.682+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5681507e0 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.683+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5681604e0 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.684+0000 7fd55f7fe640 1 -- 172.21.15.40:0/1122523359 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd560047070 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.684+0000 7fd55f7fe640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd54806c8f0 0x7fd54806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.684+0000 7fd55f7fe640 1 -- 172.21.15.40:0/1122523359 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd5600ab8e0 con 0x7fd5680c4aa0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.685+0000 7fd566ffd640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd54806c8f0 0x7fd54806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.685+0000 7fd566ffd640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd54806c8f0 0x7fd54806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd558009b40 tx=0x7fd558009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.689+0000 7fd55f7fe640 1 -- 172.21.15.40:0/1122523359 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5681604e0 con 0x7fd5680c4aa0 2024-04-26T21:36:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.874+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd5680694f0 con 0x7fd5680c4aa0 2024-04-26T21:36:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.874+0000 7fd55f7fe640 1 -- 172.21.15.40:0/1122523359 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fd5600792c0 con 0x7fd5680c4aa0 2024-04-26T21:36:01.875 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd54806c8f0 msgr2=0x7fd54806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd54806c8f0 0x7fd54806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd558009b40 tx=0x7fd558009190 comp rx=0 tx=0).stop 2024-04-26T21:36:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5680c4aa0 msgr2=0x7fd568157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5680c4aa0 0x7fd568157730 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd56000a980 tx=0x7fd568073420 comp rx=0 tx=0).stop 2024-04-26T21:36:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 shutdown_connections 2024-04-26T21:36:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd54806c8f0 0x7fd54806edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5680c4aa0 0x7fd568157730 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680c4620 0x7fd5680ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 --2- 172.21.15.40:0/1122523359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5540051c0 0x7fd5680c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 >> 172.21.15.40:0/1122523359 conn(0x7fd5680d0be0 msgr2=0x7fd5680d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 shutdown_connections 2024-04-26T21:36:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.877+0000 7fd56d425640 1 -- 172.21.15.40:0/1122523359 wait complete. 2024-04-26T21:36:01.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:36:01.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.992+0000 7f0a8a093640 1 Processor -- start 2024-04-26T21:36:01.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.992+0000 7f0a8a093640 1 -- start start 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a8a093640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0a840cc480 0x7f0a84072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a8a093640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a8415d7c0 con 0x7f0a84071e60 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a8a093640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a8415daa0 con 0x7f0a84072780 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a8a093640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0a8415dd80 con 0x7f0a840cc480 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a837fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0a84071e60 0x7f0a8406f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34908/0 (socket says 172.21.15.40:34908) 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a837fe640 1 -- 172.21.15.40:0/2748795974 learned_addr learned my addr 172.21.15.40:0/2748795974 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a7bfff640 1 --2- 172.21.15.40:0/2748795974 >> v2:172.21.15.184:3300/0 conn(0x7f0a840cc480 0x7f0a84072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a82ffd640 1 -- 172.21.15.40:0/2748795974 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081337282 0 0) 0x7f0a8415d7c0 con 0x7f0a84071e60 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.993+0000 7f0a82ffd640 1 -- 172.21.15.40:0/2748795974 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a70003520 con 0x7f0a84071e60 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.994+0000 7f0a7bfff640 1 -- 172.21.15.40:0/2748795974 >> v1:172.21.15.184:6789/0 conn(0x7f0a84072780 legacy=0x7f0a84166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.994+0000 7f0a7bfff640 1 -- 172.21.15.40:0/2748795974 >> v1:172.21.15.40:6789/0 conn(0x7f0a84071e60 legacy=0x7f0a8406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.994+0000 7f0a7bfff640 1 -- 172.21.15.40:0/2748795974 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8415e060 con 0x7f0a840cc480 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.994+0000 7f0a7bfff640 1 --2- 172.21.15.40:0/2748795974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a840cc480 0x7f0a84072240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0097a0 tx=0x7f0a6c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d6578872c27dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a82ffd640 1 -- 172.21.15.40:0/2748795974 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c037070 con 0x7f0a840cc480 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a82ffd640 1 -- 172.21.15.40:0/2748795974 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6c032440 con 0x7f0a840cc480 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a82ffd640 1 -- 172.21.15.40:0/2748795974 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c032810 con 0x7f0a840cc480 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a8a093640 1 -- 172.21.15.40:0/2748795974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a840cc480 msgr2=0x7f0a84072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a8a093640 1 --2- 172.21.15.40:0/2748795974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a840cc480 0x7f0a84072240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0097a0 tx=0x7f0a6c02a230 comp rx=0 tx=0).stop 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a8a093640 1 -- 172.21.15.40:0/2748795974 shutdown_connections 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a8a093640 1 --2- 172.21.15.40:0/2748795974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a840cc480 0x7f0a84072240 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a8a093640 1 -- 172.21.15.40:0/2748795974 >> 172.21.15.40:0/2748795974 conn(0x7f0a840d0be0 msgr2=0x7f0a840d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a8a093640 1 -- 172.21.15.40:0/2748795974 shutdown_connections 2024-04-26T21:36:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.995+0000 7f0a8a093640 1 -- 172.21.15.40:0/2748795974 wait complete. 2024-04-26T21:36:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.996+0000 7f0a8a093640 1 Processor -- start 2024-04-26T21:36:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.996+0000 7f0a8a093640 1 -- start start 2024-04-26T21:36:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a8a093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84071e60 0x7f0a840c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a8a093640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84072780 0x7f0a841f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a8a093640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a840cc480 0x7f0a841f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a8a093640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8415d3f0 con 0x7f0a840cc480 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a8a093640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84072bd0 con 0x7f0a84072780 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a8a093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8415d570 con 0x7f0a84071e60 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a83fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a840cc480 0x7f0a841f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a83fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a840cc480 0x7f0a841f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45506/0 (socket says 172.21.15.40:45506) 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a83fff640 1 -- 172.21.15.40:0/1154997465 learned_addr learned my addr 172.21.15.40:0/1154997465 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.997+0000 7f0a7bfff640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84072780 0x7f0a841f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a837fe640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84071e60 0x7f0a840c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a83fff640 1 -- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84071e60 msgr2=0x7f0a840c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a83fff640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84071e60 0x7f0a840c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a83fff640 1 -- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84072780 msgr2=0x7f0a841f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a83fff640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84072780 0x7f0a841f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a83fff640 1 -- 172.21.15.40:0/1154997465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a840b59a0 con 0x7f0a840cc480 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a83fff640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a840cc480 0x7f0a841f87f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00c7d0 tx=0x7f0a7c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a80ff9640 1 -- 172.21.15.40:0/1154997465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c003b70 con 0x7f0a840cc480 2024-04-26T21:36:02.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a80ff9640 1 -- 172.21.15.40:0/1154997465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a7c003d30 con 0x7f0a840cc480 2024-04-26T21:36:02.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a840b5c80 con 0x7f0a840cc480 2024-04-26T21:36:02.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a80ff9640 1 -- 172.21.15.40:0/1154997465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00aa90 con 0x7f0a840cc480 2024-04-26T21:36:02.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.998+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a840b6240 con 0x7f0a840cc480 2024-04-26T21:36:02.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:01.999+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a840694f0 con 0x7f0a840cc480 2024-04-26T21:36:02.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.000+0000 7f0a80ff9640 1 -- 172.21.15.40:0/1154997465 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a7c00ac30 con 0x7f0a840cc480 2024-04-26T21:36:02.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.000+0000 7f0a80ff9640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a6406c8f0 0x7f0a6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.000+0000 7f0a80ff9640 1 -- 172.21.15.40:0/1154997465 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0a7c08bd00 con 0x7f0a840cc480 2024-04-26T21:36:02.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.000+0000 7f0a837fe640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a6406c8f0 0x7f0a6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.001+0000 7f0a837fe640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a6406c8f0 0x7f0a6406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0a740011e0 tx=0x7f0a74007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:02.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.004+0000 7f0a80ff9640 1 -- 172.21.15.40:0/1154997465 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a7c0596e0 con 0x7f0a840cc480 2024-04-26T21:36:02.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.192+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a8415a4d0 con 0x7f0a840cc480 2024-04-26T21:36:02.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.193+0000 7f0a80ff9640 1 -- 172.21.15.40:0/1154997465 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f0a7c05d9e0 con 0x7f0a840cc480 2024-04-26T21:36:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:36:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a6406c8f0 msgr2=0x7f0a6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a6406c8f0 0x7f0a6406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0a740011e0 tx=0x7f0a74007d30 comp rx=0 tx=0).stop 2024-04-26T21:36:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a840cc480 msgr2=0x7f0a841f87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a840cc480 0x7f0a841f87f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00c7d0 tx=0x7f0a7c00cc90 comp rx=0 tx=0).stop 2024-04-26T21:36:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 shutdown_connections 2024-04-26T21:36:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a6406c8f0 0x7f0a6406edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a840cc480 0x7f0a841f87f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84072780 0x7f0a841f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 --2- 172.21.15.40:0/1154997465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84071e60 0x7f0a840c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 >> 172.21.15.40:0/1154997465 conn(0x7f0a840d0be0 msgr2=0x7f0a8415f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 shutdown_connections 2024-04-26T21:36:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.195+0000 7f0a8a093640 1 -- 172.21.15.40:0/1154997465 wait complete. 2024-04-26T21:36:02.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:36:02.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.311+0000 7fb45b893640 1 Processor -- start 2024-04-26T21:36:02.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.311+0000 7fb45b893640 1 -- start start 2024-04-26T21:36:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45b893640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4540d1a00 0x7fb4540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45b893640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb4540d2320 0x7fb4541625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45b893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540b40d0 0x7fb454162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45b893640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb454163070 con 0x7fb4540d1a00 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45b893640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb4540d0a10 con 0x7fb4540d2320 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45b893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4540d0cf0 con 0x7fb4540b40d0 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45962f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4540d1a00 0x7fb4540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45962f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4540d1a00 0x7fb4540d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45520/0 (socket says 172.21.15.40:45520) 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45962f640 1 -- 172.21.15.40:0/1278954698 learned_addr learned my addr 172.21.15.40:0/1278954698 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb458e2e640 1 --2- 172.21.15.40:0/1278954698 >> v2:172.21.15.184:3300/0 conn(0x7fb4540d2320 0x7fb4541625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb459e30640 1 --2- 172.21.15.40:0/1278954698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540b40d0 0x7fb454162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45962f640 1 -- 172.21.15.40:0/1278954698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540b40d0 msgr2=0x7fb454162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45962f640 1 --2- 172.21.15.40:0/1278954698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540b40d0 0x7fb454162b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45962f640 1 -- 172.21.15.40:0/1278954698 >> v2:172.21.15.184:3300/0 conn(0x7fb4540d2320 msgr2=0x7fb4541625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45962f640 1 --2- 172.21.15.40:0/1278954698 >> v2:172.21.15.184:3300/0 conn(0x7fb4540d2320 0x7fb4541625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.312+0000 7fb45962f640 1 -- 172.21.15.40:0/1278954698 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4540d0fd0 con 0x7fb4540d1a00 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.313+0000 7fb45962f640 1 --2- 172.21.15.40:0/1278954698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540d1a00 0x7fb4540d1de0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb45408d380 tx=0x7fb45002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877decc59a91d81d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.313+0000 7fb44bfff640 1 -- 172.21.15.40:0/1278954698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450037070 con 0x7fb4540d1a00 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.313+0000 7fb44bfff640 1 -- 172.21.15.40:0/1278954698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45002ae00 con 0x7fb4540d1a00 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.313+0000 7fb44bfff640 1 -- 172.21.15.40:0/1278954698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450032660 con 0x7fb4540d1a00 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 -- 172.21.15.40:0/1278954698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540d1a00 msgr2=0x7fb4540d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 --2- 172.21.15.40:0/1278954698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540d1a00 0x7fb4540d1de0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb45408d380 tx=0x7fb45002a0b0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 -- 172.21.15.40:0/1278954698 shutdown_connections 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 --2- 172.21.15.40:0/1278954698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540b40d0 0x7fb454162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 --2- 172.21.15.40:0/1278954698 >> v2:172.21.15.184:3300/0 conn(0x7fb4540d2320 0x7fb4541625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 --2- 172.21.15.40:0/1278954698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540d1a00 0x7fb4540d1de0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 -- 172.21.15.40:0/1278954698 >> 172.21.15.40:0/1278954698 conn(0x7fb4540d4fd0 msgr2=0x7fb4540d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 -- 172.21.15.40:0/1278954698 shutdown_connections 2024-04-26T21:36:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.314+0000 7fb45b893640 1 -- 172.21.15.40:0/1278954698 wait complete. 2024-04-26T21:36:02.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.315+0000 7fb45b893640 1 Processor -- start 2024-04-26T21:36:02.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.315+0000 7fb45b893640 1 -- start start 2024-04-26T21:36:02.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45b893640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540b40d0 0x7fb45415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45b893640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4540d1a00 0x7fb45415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45b893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540d2320 0x7fb4540cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45b893640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb4540d3050 con 0x7fb4540b40d0 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45b893640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4540d2ed0 con 0x7fb4540d1a00 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45b893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4540d31d0 con 0x7fb4540d2320 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540b40d0 0x7fb45415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540b40d0 0x7fb45415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45530/0 (socket says 172.21.15.40:45530) 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 -- 172.21.15.40:0/2177336394 learned_addr learned my addr 172.21.15.40:0/2177336394 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb458e2e640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4540d1a00 0x7fb45415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb459e30640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540d2320 0x7fb4540cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 -- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540d2320 msgr2=0x7fb4540cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540d2320 0x7fb4540cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 -- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4540d1a00 msgr2=0x7fb45415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4540d1a00 0x7fb45415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 -- 172.21.15.40:0/2177336394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4540cd570 con 0x7fb4540b40d0 2024-04-26T21:36:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb458e2e640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4540d1a00 0x7fb45415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.316+0000 7fb45962f640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540b40d0 0x7fb45415b890 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb45002a0f0 tx=0x7fb4500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.317+0000 7fb449ffb640 1 -- 172.21.15.40:0/2177336394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450041030 con 0x7fb4540b40d0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.317+0000 7fb449ffb640 1 -- 172.21.15.40:0/2177336394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45002ac00 con 0x7fb4540b40d0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.317+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4540b7e40 con 0x7fb4540b40d0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.317+0000 7fb449ffb640 1 -- 172.21.15.40:0/2177336394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450037040 con 0x7fb4540b40d0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.317+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4540b8400 con 0x7fb4540b40d0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.318+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4540694f0 con 0x7fb4540b40d0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.318+0000 7fb449ffb640 1 -- 172.21.15.40:0/2177336394 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb450044070 con 0x7fb4540b40d0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.319+0000 7fb449ffb640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43806c8f0 0x7fb43806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.319+0000 7fb449ffb640 1 -- 172.21.15.40:0/2177336394 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb4500ac380 con 0x7fb4540b40d0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.319+0000 7fb458e2e640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43806c8f0 0x7fb43806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.319+0000 7fb458e2e640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43806c8f0 0x7fb43806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb444006fd0 tx=0x7fb444008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:02.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.323+0000 7fb449ffb640 1 -- 172.21.15.40:0/2177336394 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb450044e40 con 0x7fb4540b40d0 2024-04-26T21:36:02.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.507+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb454162a20 con 0x7fb4540b40d0 2024-04-26T21:36:02.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.507+0000 7fb449ffb640 1 -- 172.21.15.40:0/2177336394 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb45007a3a0 con 0x7fb4540b40d0 2024-04-26T21:36:02.508 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43806c8f0 msgr2=0x7fb43806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43806c8f0 0x7fb43806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb444006fd0 tx=0x7fb444008040 comp rx=0 tx=0).stop 2024-04-26T21:36:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540b40d0 msgr2=0x7fb45415b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540b40d0 0x7fb45415b890 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb45002a0f0 tx=0x7fb4500046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 shutdown_connections 2024-04-26T21:36:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43806c8f0 0x7fb43806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4540d2320 0x7fb4540cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4540d1a00 0x7fb45415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 --2- 172.21.15.40:0/2177336394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4540b40d0 0x7fb45415b890 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.509+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 >> 172.21.15.40:0/2177336394 conn(0x7fb4540d4fd0 msgr2=0x7fb4540cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:02.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.510+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 shutdown_connections 2024-04-26T21:36:02.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.510+0000 7fb45b893640 1 -- 172.21.15.40:0/2177336394 wait complete. 2024-04-26T21:36:02.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:36:02.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.622+0000 7f832dd91640 1 Processor -- start 2024-04-26T21:36:02.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.622+0000 7f832dd91640 1 -- start start 2024-04-26T21:36:02.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f832dd91640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8328162880 0x7f83280d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f832dd91640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83280b5f20 0x7f83280d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f832dd91640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83280b7e50 con 0x7f83280d39f0 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f832dd91640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f83280d3e40 con 0x7f83280b5f20 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f832dd91640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f83280d0580 con 0x7f8328162880 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f8326ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83280b5f20 0x7f83280d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f8326ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83280b5f20 0x7f83280d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45544/0 (socket says 172.21.15.40:45544) 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f8326ffd640 1 -- 172.21.15.40:0/398723922 learned_addr learned my addr 172.21.15.40:0/398723922 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f83277fe640 1 --2- 172.21.15.40:0/398723922 >> v2:172.21.15.184:3300/0 conn(0x7f8328162880 0x7f83280d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f8326ffd640 1 -- 172.21.15.40:0/398723922 >> v2:172.21.15.184:3300/0 conn(0x7f8328162880 msgr2=0x7f83280d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f8326ffd640 1 --2- 172.21.15.40:0/398723922 >> v2:172.21.15.184:3300/0 conn(0x7f8328162880 0x7f83280d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.623+0000 7f8326ffd640 1 -- 172.21.15.40:0/398723922 >> v1:172.21.15.184:6789/0 conn(0x7f83280d39f0 legacy=0x7f83280b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.624+0000 7f8326ffd640 1 -- 172.21.15.40:0/398723922 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83280d1800 con 0x7f83280b5f20 2024-04-26T21:36:02.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.624+0000 7f83277fe640 1 --2- 172.21.15.40:0/398723922 >> v2:172.21.15.184:3300/0 conn(0x7f8328162880 0x7f83280d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:02.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.624+0000 7f8326ffd640 1 --2- 172.21.15.40:0/398723922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83280d34b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f832808d490 tx=0x7f831c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b2011824075ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:02.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.624+0000 7f83267fc640 1 -- 172.21.15.40:0/398723922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c037070 con 0x7f83280b5f20 2024-04-26T21:36:02.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.624+0000 7f83267fc640 1 -- 172.21.15.40:0/398723922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f831c032440 con 0x7f83280b5f20 2024-04-26T21:36:02.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.624+0000 7f83267fc640 1 -- 172.21.15.40:0/398723922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c032810 con 0x7f83280b5f20 2024-04-26T21:36:02.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.625+0000 7f832dd91640 1 -- 172.21.15.40:0/398723922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 msgr2=0x7f83280d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.625+0000 7f832dd91640 1 --2- 172.21.15.40:0/398723922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83280d34b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f832808d490 tx=0x7f831c02a270 comp rx=0 tx=0).stop 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.625+0000 7f832dd91640 1 -- 172.21.15.40:0/398723922 shutdown_connections 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.625+0000 7f832dd91640 1 --2- 172.21.15.40:0/398723922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83280d34b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.625+0000 7f832dd91640 1 --2- 172.21.15.40:0/398723922 >> v2:172.21.15.184:3300/0 conn(0x7f8328162880 0x7f83280d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.625+0000 7f832dd91640 1 -- 172.21.15.40:0/398723922 >> 172.21.15.40:0/398723922 conn(0x7f83280d4f60 msgr2=0x7f83280d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.625+0000 7f832dd91640 1 -- 172.21.15.40:0/398723922 shutdown_connections 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.625+0000 7f832dd91640 1 -- 172.21.15.40:0/398723922 wait complete. 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.626+0000 7f832dd91640 1 Processor -- start 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.626+0000 7f832dd91640 1 -- start start 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f832dd91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83281f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f832dd91640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83280d39f0 0x7f83280cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f832dd91640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8328162880 0x7f832815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f832dd91640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83280d0d90 con 0x7f83280b5f20 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f832dd91640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83280d0ab0 con 0x7f83280d39f0 2024-04-26T21:36:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f832dd91640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8328162c60 con 0x7f8328162880 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f83277fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83281f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f83277fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83281f4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45562/0 (socket says 172.21.15.40:45562) 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f83277fe640 1 -- 172.21.15.40:0/3852740240 learned_addr learned my addr 172.21.15.40:0/3852740240 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f8326ffd640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83280d39f0 0x7f83280cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.627+0000 7f8327fff640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8328162880 0x7f832815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f83277fe640 1 -- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8328162880 msgr2=0x7f832815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f83277fe640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8328162880 0x7f832815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f83277fe640 1 -- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83280d39f0 msgr2=0x7f83280cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f83277fe640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83280d39f0 0x7f83280cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f83277fe640 1 -- 172.21.15.40:0/3852740240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83280ce290 con 0x7f83280b5f20 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f83277fe640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83281f4a20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f831800e810 tx=0x7f831800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f830ffff640 1 -- 172.21.15.40:0/3852740240 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831800f040 con 0x7f83280b5f20 2024-04-26T21:36:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f830ffff640 1 -- 172.21.15.40:0/3852740240 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8318004030 con 0x7f83280b5f20 2024-04-26T21:36:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83281f59c0 con 0x7f83280b5f20 2024-04-26T21:36:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f830ffff640 1 -- 172.21.15.40:0/3852740240 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318004400 con 0x7f83280b5f20 2024-04-26T21:36:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.628+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83281f5400 con 0x7f83280b5f20 2024-04-26T21:36:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.629+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83280694f0 con 0x7f83280b5f20 2024-04-26T21:36:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.629+0000 7f830ffff640 1 -- 172.21.15.40:0/3852740240 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f831800c6d0 con 0x7f83280b5f20 2024-04-26T21:36:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.630+0000 7f830ffff640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f831406c8f0 0x7f831406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.630+0000 7f830ffff640 1 -- 172.21.15.40:0/3852740240 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f831808bc70 con 0x7f83280b5f20 2024-04-26T21:36:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.630+0000 7f8326ffd640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f831406c8f0 0x7f831406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.631+0000 7f8326ffd640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f831406c8f0 0x7f831406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f831c034000 tx=0x7f831c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:02.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.634+0000 7f830ffff640 1 -- 172.21.15.40:0/3852740240 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8318017020 con 0x7f83280b5f20 2024-04-26T21:36:02.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.831+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f83280cc1d0 con 0x7f83280b5f20 2024-04-26T21:36:02.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.831+0000 7f830ffff640 1 -- 172.21.15.40:0/3852740240 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f8318059640 con 0x7f83280b5f20 2024-04-26T21:36:02.831 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:36:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f831406c8f0 msgr2=0x7f831406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f831406c8f0 0x7f831406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f831c034000 tx=0x7f831c02e000 comp rx=0 tx=0).stop 2024-04-26T21:36:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 msgr2=0x7f83281f4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83281f4a20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f831800e810 tx=0x7f831800ecd0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 shutdown_connections 2024-04-26T21:36:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f831406c8f0 0x7f831406edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8328162880 0x7f832815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83280d39f0 0x7f83280cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 --2- 172.21.15.40:0/3852740240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83280b5f20 0x7f83281f4a20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 >> 172.21.15.40:0/3852740240 conn(0x7f83280d4f60 msgr2=0x7f832806e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 shutdown_connections 2024-04-26T21:36:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:02.833+0000 7f832dd91640 1 -- 172.21.15.40:0/3852740240 wait complete. 2024-04-26T21:36:02.844 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:36:02.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:02.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:36:02.869 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:36:02.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:02.869 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:36:02.925 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:36:02.925 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:02.944 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:36:02.944 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:03.003 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:36:03.003 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:03.062 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:03.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.163+0000 7f0bfe9f0640 1 Processor -- start 2024-04-26T21:36:03.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.163+0000 7f0bfe9f0640 1 -- start start 2024-04-26T21:36:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.164+0000 7f0bfe9f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0bf81608f0 0x7f0bf80d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.164+0000 7f0bfe9f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 0x7f0bf80b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.164+0000 7f0bfe9f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf80cde20 con 0x7f0bf80d0a90 2024-04-26T21:36:03.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.164+0000 7f0bfe9f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf80cea30 con 0x7f0bf81608f0 2024-04-26T21:36:03.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.164+0000 7f0bfe9f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf80ce100 con 0x7f0bf80cffa0 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.164+0000 7f0bf77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0bf80d0a90 0x7f0bf80cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34914/0 (socket says 172.21.15.40:34914) 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.164+0000 7f0bf77fe640 1 -- 172.21.15.40:0/1399844320 learned_addr learned my addr 172.21.15.40:0/1399844320 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bfcf8d640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 0x7f0bf80b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736306727 0 0) 0x7f0bf80cde20 con 0x7f0bf80d0a90 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be4003520 con 0x7f0bf80d0a90 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf7fff640 1 --2- 172.21.15.40:0/1399844320 >> v2:172.21.15.184:3300/0 conn(0x7f0bf81608f0 0x7f0bf80d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 293378739 0 0) 0x7f0be4003520 con 0x7f0bf80d0a90 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 msgr2=0x7f0bf80b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 0x7f0bf80b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 >> v2:172.21.15.184:3300/0 conn(0x7f0bf81608f0 msgr2=0x7f0bf80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 --2- 172.21.15.40:0/1399844320 >> v2:172.21.15.184:3300/0 conn(0x7f0bf81608f0 0x7f0bf80d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf80ce3e0 con 0x7f0bf80d0a90 2024-04-26T21:36:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f0be0001670 con 0x7f0bf80d0a90 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 >> v1:172.21.15.40:6789/0 conn(0x7f0bf80d0a90 legacy=0x7f0bf80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0be4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 0x7f0be4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0be4004d40 0x7f0be40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf80cea30 con 0x7f0be40039b0 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf80ce100 con 0x7f0be4004d40 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0be400b9c0 con 0x7f0bf80cffa0 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf7fff640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0be4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf77fe640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 0x7f0be4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bfcf8d640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0be4004d40 0x7f0be40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf7fff640 1 -- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 msgr2=0x7f0be4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf7fff640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 0x7f0be4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf7fff640 1 -- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0be4004d40 msgr2=0x7f0be40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf7fff640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0be4004d40 0x7f0be40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf7fff640 1 -- 172.21.15.40:0/1399844320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be4003520 con 0x7f0be40039b0 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.165+0000 7f0bf77fe640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80cffa0 0x7f0be4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.166+0000 7f0bf7fff640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0be4003dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009af0 tx=0x7f0be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83826275bc3b603 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.166+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8036070 con 0x7f0be40039b0 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.166+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be8031400 con 0x7f0be40039b0 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.166+0000 7f0bf6ffd640 1 -- 172.21.15.40:0/1399844320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8009690 con 0x7f0be40039b0 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.167+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/1399844320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 msgr2=0x7f0be4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.167+0000 7f0bfe9f0640 1 --2- 172.21.15.40:0/1399844320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0be4003dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009af0 tx=0x7f0be4005830 comp rx=0 tx=0).stop 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.167+0000 7f0bfcf8d640 1 -- 172.21.15.40:0/1399844320 reap_dead start 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.167+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/1399844320 shutdown_connections 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.167+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/1399844320 >> 172.21.15.40:0/1399844320 conn(0x7f0bf80d2ec0 msgr2=0x7f0bf80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.167+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/1399844320 shutdown_connections 2024-04-26T21:36:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.167+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/1399844320 wait complete. 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.168+0000 7f0bfe9f0640 1 Processor -- start 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.168+0000 7f0bfe9f0640 1 -- start start 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bfe9f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0bf81500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bfe9f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80b5ac0 0x7f0bf80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bfe9f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf80c80e0 0x7f0bf80c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bfe9f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf80ceec0 con 0x7f0be40039b0 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bfe9f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf80d0ef0 con 0x7f0bf80c80e0 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bfe9f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf80cda40 con 0x7f0bf80b5ac0 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0bf81500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0bf81500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45588/0 (socket says 172.21.15.40:45588) 2024-04-26T21:36:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf7fff640 1 -- 172.21.15.40:0/2511849838 learned_addr learned my addr 172.21.15.40:0/2511849838 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf77fe640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80b5ac0 0x7f0bf80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bfcf8d640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf80c80e0 0x7f0bf80c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf7fff640 1 -- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80b5ac0 msgr2=0x7f0bf80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf7fff640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80b5ac0 0x7f0bf80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf7fff640 1 -- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf80c80e0 msgr2=0x7f0bf80c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf7fff640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf80c80e0 0x7f0bf80c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.169+0000 7f0bf7fff640 1 -- 172.21.15.40:0/2511849838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf80c8aa0 con 0x7f0be40039b0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.170+0000 7f0bf7fff640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0bf81500f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009af0 tx=0x7f0be8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.170+0000 7f0bf4ff9640 1 -- 172.21.15.40:0/2511849838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8041020 con 0x7f0be40039b0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.170+0000 7f0bf4ff9640 1 -- 172.21.15.40:0/2511849838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be802ade0 con 0x7f0be40039b0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.170+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf80c8d40 con 0x7f0be40039b0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.170+0000 7f0bf4ff9640 1 -- 172.21.15.40:0/2511849838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8036040 con 0x7f0be40039b0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.170+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf815d580 con 0x7f0be40039b0 2024-04-26T21:36:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.171+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf815d370 con 0x7f0be40039b0 2024-04-26T21:36:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.171+0000 7f0bf4ff9640 1 -- 172.21.15.40:0/2511849838 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0be8044020 con 0x7f0be40039b0 2024-04-26T21:36:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.172+0000 7f0bf4ff9640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bd806c8f0 0x7f0bd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.172+0000 7f0bf4ff9640 1 -- 172.21.15.40:0/2511849838 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0be80ac550 con 0x7f0be40039b0 2024-04-26T21:36:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.172+0000 7f0bf77fe640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bd806c8f0 0x7f0bd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.172+0000 7f0bf77fe640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bd806c8f0 0x7f0bd806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0be0001eb0 tx=0x7f0be0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.176+0000 7f0bf4ff9640 1 -- 172.21.15.40:0/2511849838 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0be804b030 con 0x7f0be40039b0 2024-04-26T21:36:03.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.362+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bf80694f0 con 0x7f0be40039b0 2024-04-26T21:36:03.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.363+0000 7f0bf4ff9640 1 -- 172.21.15.40:0/2511849838 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7f0be807a4e0 con 0x7f0be40039b0 2024-04-26T21:36:03.364 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:03.364 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bd806c8f0 msgr2=0x7f0bd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bd806c8f0 0x7f0bd806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0be0001eb0 tx=0x7f0be0009040 comp rx=0 tx=0).stop 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 msgr2=0x7f0bf81500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0bf81500f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009af0 tx=0x7f0be8004730 comp rx=0 tx=0).stop 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 shutdown_connections 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bd806c8f0 0x7f0bd806edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf80c80e0 0x7f0bf80c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf80b5ac0 0x7f0bf80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 --2- 172.21.15.40:0/2511849838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0be40039b0 0x7f0bf81500f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 >> 172.21.15.40:0/2511849838 conn(0x7f0bf80d2ec0 msgr2=0x7f0bf80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 shutdown_connections 2024-04-26T21:36:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.365+0000 7f0bfe9f0640 1 -- 172.21.15.40:0/2511849838 wait complete. 2024-04-26T21:36:03.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:36:03.377 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:36:03.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.480+0000 7f335e047640 1 Processor -- start 2024-04-26T21:36:03.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.480+0000 7f335e047640 1 -- start start 2024-04-26T21:36:03.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.481+0000 7f335e047640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33580d3bc0 0x7f33580d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.481+0000 7f335e047640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33580d1040 con 0x7f335815f550 2024-04-26T21:36:03.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.481+0000 7f335e047640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33580c9680 con 0x7f335815f170 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.481+0000 7f335e047640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f33580c9430 con 0x7f33580d3bc0 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.481+0000 7f3357fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33580d3bc0 0x7f33580d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.481+0000 7f3357fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33580d3bc0 0x7f33580d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45602/0 (socket says 172.21.15.40:45602) 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.481+0000 7f3356ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f335815f550 0x7f33580ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34928/0 (socket says 172.21.15.40:34928) 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.481+0000 7f3357fff640 1 -- 172.21.15.40:0/1434407875 learned_addr learned my addr 172.21.15.40:0/1434407875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.482+0000 7f3357fff640 1 -- 172.21.15.40:0/1434407875 >> v1:172.21.15.184:6789/0 conn(0x7f335815f170 legacy=0x7f33580d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.482+0000 7f3357fff640 1 -- 172.21.15.40:0/1434407875 >> v1:172.21.15.40:6789/0 conn(0x7f335815f550 legacy=0x7f33580ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.482+0000 7f3357fff640 1 -- 172.21.15.40:0/1434407875 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33580c9960 con 0x7f33580d3bc0 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.482+0000 7f33567fc640 1 -- 172.21.15.40:0/1434407875 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672644818 0 0) 0x7f334c001110 con 0x7f335815f550 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.482+0000 7f3357fff640 1 --2- 172.21.15.40:0/1434407875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33580d3bc0 0x7f33580d5a80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f33580c95b0 tx=0x7f335002a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6f7c3e7ad15724 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.482+0000 7f33567fc640 1 -- 172.21.15.40:0/1434407875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350036070 con 0x7f33580d3bc0 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f33567fc640 1 -- 172.21.15.40:0/1434407875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3350031600 con 0x7f33580d3bc0 2024-04-26T21:36:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f33567fc640 1 -- 172.21.15.40:0/1434407875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33500319d0 con 0x7f33580d3bc0 2024-04-26T21:36:03.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f335e047640 1 -- 172.21.15.40:0/1434407875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33580d3bc0 msgr2=0x7f33580d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f335e047640 1 --2- 172.21.15.40:0/1434407875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33580d3bc0 0x7f33580d5a80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f33580c95b0 tx=0x7f335002a7e0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f335e047640 1 -- 172.21.15.40:0/1434407875 shutdown_connections 2024-04-26T21:36:03.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f335e047640 1 --2- 172.21.15.40:0/1434407875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33580d3bc0 0x7f33580d5a80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f335e047640 1 -- 172.21.15.40:0/1434407875 >> 172.21.15.40:0/1434407875 conn(0x7f33580d4f60 msgr2=0x7f33580d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:03.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f335e047640 1 -- 172.21.15.40:0/1434407875 shutdown_connections 2024-04-26T21:36:03.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.483+0000 7f335e047640 1 -- 172.21.15.40:0/1434407875 wait complete. 2024-04-26T21:36:03.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.484+0000 7f335e047640 1 Processor -- start 2024-04-26T21:36:03.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.484+0000 7f335e047640 1 -- start start 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f335e047640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33580d3bc0 0x7f335809f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f335e047640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f335815f170 0x7f33580cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f335e047640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33580bab30 0x7f335814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f335e047640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f33580d2e30 con 0x7f335815f170 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f335e047640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f33580d2cb0 con 0x7f33580d3bc0 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f335e047640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f33580d2fb0 con 0x7f33580bab30 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f335815f170 0x7f33580cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f335815f170 0x7f33580cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45608/0 (socket says 172.21.15.40:45608) 2024-04-26T21:36:03.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.485+0000 7f3356ffd640 1 -- 172.21.15.40:0/1890477874 learned_addr learned my addr 172.21.15.40:0/1890477874 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f3357fff640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33580bab30 0x7f335814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f33577fe640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33580d3bc0 0x7f335809f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f3356ffd640 1 -- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33580bab30 msgr2=0x7f335814fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f3356ffd640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33580bab30 0x7f335814fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f3356ffd640 1 -- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33580d3bc0 msgr2=0x7f335809f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f3356ffd640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33580d3bc0 0x7f335809f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f3356ffd640 1 -- 172.21.15.40:0/1890477874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33580cdf10 con 0x7f335815f170 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f33577fe640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33580d3bc0 0x7f335809f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f3356ffd640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f335815f170 0x7f33580cd9d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c390 tx=0x7f334c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f333ffff640 1 -- 172.21.15.40:0/1890477874 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c0074e0 con 0x7f335815f170 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f333ffff640 1 -- 172.21.15.40:0/1890477874 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f334c0047e0 con 0x7f335815f170 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f333ffff640 1 -- 172.21.15.40:0/1890477874 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c004bd0 con 0x7f335815f170 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33580ce1c0 con 0x7f335815f170 2024-04-26T21:36:03.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.486+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33580ce780 con 0x7f335815f170 2024-04-26T21:36:03.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.487+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335814dba0 con 0x7f335815f170 2024-04-26T21:36:03.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.488+0000 7f333ffff640 1 -- 172.21.15.40:0/1890477874 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f334c011070 con 0x7f335815f170 2024-04-26T21:36:03.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.488+0000 7f333ffff640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f334406c8f0 0x7f334406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.488+0000 7f333ffff640 1 -- 172.21.15.40:0/1890477874 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f334c08ba50 con 0x7f335815f170 2024-04-26T21:36:03.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.488+0000 7f33577fe640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f334406c8f0 0x7f334406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.489+0000 7f33577fe640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f334406c8f0 0x7f334406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3348000f60 tx=0x7f3348009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.492+0000 7f333ffff640 1 -- 172.21.15.40:0/1890477874 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f33580d1040 con 0x7f335815f170 2024-04-26T21:36:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.676+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f33580a2c30 con 0x7f335815f170 2024-04-26T21:36:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.676+0000 7f333ffff640 1 -- 172.21.15.40:0/1890477874 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f334c059b10 con 0x7f335815f170 2024-04-26T21:36:03.676 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f334406c8f0 msgr2=0x7f334406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f334406c8f0 0x7f334406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3348000f60 tx=0x7f3348009040 comp rx=0 tx=0).stop 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f335815f170 msgr2=0x7f33580cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f335815f170 0x7f33580cd9d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c390 tx=0x7f334c00c850 comp rx=0 tx=0).stop 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 shutdown_connections 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f334406c8f0 0x7f334406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33580bab30 0x7f335814fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f335815f170 0x7f33580cd9d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 --2- 172.21.15.40:0/1890477874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33580d3bc0 0x7f335809f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 >> 172.21.15.40:0/1890477874 conn(0x7f33580d4f60 msgr2=0x7f33580d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 shutdown_connections 2024-04-26T21:36:03.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.678+0000 7f335e047640 1 -- 172.21.15.40:0/1890477874 wait complete. 2024-04-26T21:36:03.689 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:36:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.793+0000 7f3014daf640 1 Processor -- start 2024-04-26T21:36:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.793+0000 7f3014daf640 1 -- start start 2024-04-26T21:36:03.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.794+0000 7f3014daf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3010071260 0x7f301015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.794+0000 7f3014daf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30100d2390 con 0x7f3010070350 2024-04-26T21:36:03.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.794+0000 7f3014daf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f301006d3f0 con 0x7f30100cfde0 2024-04-26T21:36:03.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.794+0000 7f3014daf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f30100afab0 con 0x7f3010071260 2024-04-26T21:36:03.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.794+0000 7f300dd9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3010070350 0x7f3010070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34934/0 (socket says 172.21.15.40:34934) 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.794+0000 7f300dd9b640 1 -- 172.21.15.40:0/2758068052 learned_addr learned my addr 172.21.15.40:0/2758068052 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.794+0000 7f300ed9d640 1 --2- 172.21.15.40:0/2758068052 >> v2:172.21.15.184:3300/0 conn(0x7f3010071260 0x7f301015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4204570347 0 0) 0x7f30100d2390 con 0x7f3010070350 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ffc003520 con 0x7f3010070350 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 192731325 0 0) 0x7f2ffc003520 con 0x7f3010070350 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 >> v2:172.21.15.184:3300/0 conn(0x7f3010071260 msgr2=0x7f301015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 --2- 172.21.15.40:0/2758068052 >> v2:172.21.15.184:3300/0 conn(0x7f3010071260 0x7f301015e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 >> v1:172.21.15.184:6789/0 conn(0x7f30100cfde0 legacy=0x7f30100d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301006d6d0 con 0x7f3010070350 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f2ff8001670 con 0x7f3010070350 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 >> v1:172.21.15.40:6789/0 conn(0x7f3010070350 legacy=0x7f3010070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f2ffc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc0043b0 0x7f2ffc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc005120 0x7f2ffc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00c030 con 0x7f2ffc0039b0 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00c1e0 con 0x7f2ffc0043b0 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2ffc00c3e0 con 0x7f2ffc005120 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300e59c640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f2ffc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300dd9b640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc0043b0 0x7f2ffc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300ed9d640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc005120 0x7f2ffc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300e59c640 1 -- 172.21.15.40:0/2758068052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc005120 msgr2=0x7f2ffc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300e59c640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc005120 0x7f2ffc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300e59c640 1 -- 172.21.15.40:0/2758068052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc0043b0 msgr2=0x7f2ffc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300e59c640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc0043b0 0x7f2ffc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300e59c640 1 -- 172.21.15.40:0/2758068052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc003520 con 0x7f2ffc0039b0 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.795+0000 7f300ed9d640 1 -- 172.21.15.40:0/2758068052 reap_dead start 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.796+0000 7f300e59c640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f2ffc003d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3008001860 tx=0x7f2ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0822c398282f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.796+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008037070 con 0x7f2ffc0039b0 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.796+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300802ac30 con 0x7f2ffc0039b0 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.796+0000 7f300d59a640 1 -- 172.21.15.40:0/2758068052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008001430 con 0x7f2ffc0039b0 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.797+0000 7f3014daf640 1 -- 172.21.15.40:0/2758068052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 msgr2=0x7f2ffc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.797+0000 7f3014daf640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f2ffc003d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3008001860 tx=0x7f2ffc005830 comp rx=0 tx=0).stop 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.797+0000 7f3014daf640 1 -- 172.21.15.40:0/2758068052 shutdown_connections 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.797+0000 7f3014daf640 1 --2- 172.21.15.40:0/2758068052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f2ffc003d90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.797+0000 7f3014daf640 1 -- 172.21.15.40:0/2758068052 >> 172.21.15.40:0/2758068052 conn(0x7f30100d2d80 msgr2=0x7f30100d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.797+0000 7f3014daf640 1 -- 172.21.15.40:0/2758068052 shutdown_connections 2024-04-26T21:36:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.797+0000 7f3014daf640 1 -- 172.21.15.40:0/2758068052 wait complete. 2024-04-26T21:36:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.798+0000 7f3014daf640 1 Processor -- start 2024-04-26T21:36:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.798+0000 7f3014daf640 1 -- start start 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f3014daf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f301009d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f3014daf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30100a0190 0x7f30100b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f3014daf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30100a05f0 0x7f30100b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f3014daf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3010070b50 con 0x7f2ffc0039b0 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f3014daf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f301006dd20 con 0x7f30100a05f0 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f3014daf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3010070cd0 con 0x7f30100a0190 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f301009d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f301009d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45626/0 (socket says 172.21.15.40:45626) 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300e59c640 1 -- 172.21.15.40:0/3293500598 learned_addr learned my addr 172.21.15.40:0/3293500598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300ed9d640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30100a05f0 0x7f30100b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300dd9b640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30100a0190 0x7f30100b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300e59c640 1 -- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30100a0190 msgr2=0x7f30100b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300e59c640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30100a0190 0x7f30100b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300e59c640 1 -- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30100a05f0 msgr2=0x7f30100b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300e59c640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30100a05f0 0x7f30100b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300e59c640 1 -- 172.21.15.40:0/3293500598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30100c8540 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.799+0000 7f300ed9d640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30100a05f0 0x7f30100b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.800+0000 7f300e59c640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f301009d500 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f300802a6c0 tx=0x7f3008001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.800+0000 7f3006ffd640 1 -- 172.21.15.40:0/3293500598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008040020 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.800+0000 7f3006ffd640 1 -- 172.21.15.40:0/3293500598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3008005270 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.800+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30100c8820 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.800+0000 7f3006ffd640 1 -- 172.21.15.40:0/3293500598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008037040 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.800+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30100c8de0 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.801+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30100694f0 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.801+0000 7f3006ffd640 1 -- 172.21.15.40:0/3293500598 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3008043020 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.802+0000 7f3006ffd640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ff006c920 0x7f2ff006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.802+0000 7f3006ffd640 1 -- 172.21.15.40:0/3293500598 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f30080abeb0 con 0x7f2ffc0039b0 2024-04-26T21:36:03.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.802+0000 7f300dd9b640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ff006c920 0x7f2ff006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:03.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.803+0000 7f300dd9b640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ff006c920 0x7f2ff006ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f301009fef0 tx=0x7f2ff8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:03.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.806+0000 7f3006ffd640 1 -- 172.21.15.40:0/3293500598 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f300804a030 con 0x7f2ffc0039b0 2024-04-26T21:36:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.994+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f30100b2da0 con 0x7f2ffc0039b0 2024-04-26T21:36:03.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.994+0000 7f3006ffd640 1 -- 172.21.15.40:0/3293500598 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f3008079ee0 con 0x7f2ffc0039b0 2024-04-26T21:36:03.995 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ff006c920 msgr2=0x7f2ff006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ff006c920 0x7f2ff006ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f301009fef0 tx=0x7f2ff8009040 comp rx=0 tx=0).stop 2024-04-26T21:36:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 msgr2=0x7f301009d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f301009d500 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f300802a6c0 tx=0x7f3008001230 comp rx=0 tx=0).stop 2024-04-26T21:36:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 shutdown_connections 2024-04-26T21:36:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ff006c920 0x7f2ff006ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30100a05f0 0x7f30100b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30100a0190 0x7f30100b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 --2- 172.21.15.40:0/3293500598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ffc0039b0 0x7f301009d500 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:03.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 >> 172.21.15.40:0/3293500598 conn(0x7f30100d2d80 msgr2=0x7f30101612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:03.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 shutdown_connections 2024-04-26T21:36:03.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:03.997+0000 7f3014daf640 1 -- 172.21.15.40:0/3293500598 wait complete. 2024-04-26T21:36:04.008 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:36:04.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.112+0000 7f5b8e7da640 1 Processor -- start 2024-04-26T21:36:04.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.112+0000 7f5b8e7da640 1 -- start start 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b8e7da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 0x7f5b880cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b8e7da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b880cd640 con 0x7f5b8815ea10 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b8e7da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b880cd920 con 0x7f5b880b8780 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b8e7da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b880cd3f0 con 0x7f5b880cf530 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b8cd77640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5b8815ea10 0x7f5b8815ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34948/0 (socket says 172.21.15.40:34948) 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b8cd77640 1 -- 172.21.15.40:0/2307980559 learned_addr learned my addr 172.21.15.40:0/2307980559 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b877fe640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 0x7f5b880cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609508908 0 0) 0x7f5b880cd640 con 0x7f5b8815ea10 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b70003520 con 0x7f5b8815ea10 2024-04-26T21:36:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2391747576 0 0) 0x7f5b70003520 con 0x7f5b8815ea10 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 msgr2=0x7f5b880cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b86ffd640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 0x7f5b880cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.113+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 >> v1:172.21.15.184:6789/0 conn(0x7f5b880b8780 legacy=0x7f5b880d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b880cdc00 con 0x7f5b8815ea10 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f5b800019f0 con 0x7f5b8815ea10 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 >> v1:172.21.15.40:6789/0 conn(0x7f5b8815ea10 legacy=0x7f5b8815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 0x7f5b700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b70003f20 0x7f5b70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b70004d40 0x7f5b700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880cd3f0 con 0x7f5b70004d40 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7000ba20 con 0x7f5b70003f20 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7000bc20 con 0x7f5b880cf530 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b8cd77640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b70004d40 0x7f5b700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b877fe640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b70003f20 0x7f5b70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b87fff640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 0x7f5b700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b8cd77640 1 -- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 msgr2=0x7f5b700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b8cd77640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 0x7f5b700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b8cd77640 1 -- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b70003f20 msgr2=0x7f5b70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b8cd77640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b70003f20 0x7f5b70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b8cd77640 1 -- 172.21.15.40:0/2307980559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b70003520 con 0x7f5b70004d40 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.114+0000 7f5b87fff640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b880cf530 0x7f5b700039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b8cd77640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b70004d40 0x7f5b700051c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000a920 tx=0x7f5b70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a9eedb185c778c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000a4c0 con 0x7f5b70004d40 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8002f070 con 0x7f5b70004d40 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b86ffd640 1 -- 172.21.15.40:0/2307980559 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8002bd20 con 0x7f5b70004d40 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b8e7da640 1 -- 172.21.15.40:0/2307980559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b70004d40 msgr2=0x7f5b700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b8e7da640 1 --2- 172.21.15.40:0/2307980559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b70004d40 0x7f5b700051c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000a920 tx=0x7f5b70005830 comp rx=0 tx=0).stop 2024-04-26T21:36:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b8cd77640 1 -- 172.21.15.40:0/2307980559 reap_dead start 2024-04-26T21:36:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b8e7da640 1 -- 172.21.15.40:0/2307980559 shutdown_connections 2024-04-26T21:36:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.115+0000 7f5b8e7da640 1 -- 172.21.15.40:0/2307980559 >> 172.21.15.40:0/2307980559 conn(0x7f5b880d2da0 msgr2=0x7f5b880d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.116+0000 7f5b8e7da640 1 -- 172.21.15.40:0/2307980559 shutdown_connections 2024-04-26T21:36:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.116+0000 7f5b8e7da640 1 -- 172.21.15.40:0/2307980559 wait complete. 2024-04-26T21:36:04.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.117+0000 7f5b8e7da640 1 Processor -- start 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.117+0000 7f5b8e7da640 1 -- start start 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8e7da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b70004d40 0x7f5b8814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8e7da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b88153d20 0x7f5b880afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8e7da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8806da10 0x7f5b8806dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8e7da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880d01f0 con 0x7f5b88153d20 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8e7da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880d2200 con 0x7f5b8806da10 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8e7da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b880d0370 con 0x7f5b70004d40 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b88153d20 0x7f5b880afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b88153d20 0x7f5b880afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45636/0 (socket says 172.21.15.40:45636) 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 -- 172.21.15.40:0/1318191274 learned_addr learned my addr 172.21.15.40:0/1318191274 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b87fff640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b70004d40 0x7f5b8814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8cd77640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8806da10 0x7f5b8806dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 -- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b70004d40 msgr2=0x7f5b8814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b70004d40 0x7f5b8814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 -- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8806da10 msgr2=0x7f5b8806dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8806da10 0x7f5b8806dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 -- 172.21.15.40:0/1318191274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8814fe10 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b877fe640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b88153d20 0x7f5b880afdd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0118e0 tx=0x7f5b7c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b84ff9640 1 -- 172.21.15.40:0/1318191274 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00faf0 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b84ff9640 1 -- 172.21.15.40:0/1318191274 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c00fc90 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b881500f0 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b84ff9640 1 -- 172.21.15.40:0/1318191274 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c014440 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.118+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b881506b0 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.119+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b880694f0 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.120+0000 7f5b84ff9640 1 -- 172.21.15.40:0/1318191274 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b7c012040 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.120+0000 7f5b84ff9640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b7406c8f0 0x7f5b7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.120+0000 7f5b84ff9640 1 -- 172.21.15.40:0/1318191274 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5b7c08c6d0 con 0x7f5b88153d20 2024-04-26T21:36:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.121+0000 7f5b87fff640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b7406c8f0 0x7f5b7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.121+0000 7f5b87fff640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b7406c8f0 0x7f5b7406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b88162370 tx=0x7f5b78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.125+0000 7f5b84ff9640 1 -- 172.21.15.40:0/1318191274 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b7c019030 con 0x7f5b88153d20 2024-04-26T21:36:04.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.309+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5b8815e3b0 con 0x7f5b88153d20 2024-04-26T21:36:04.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.309+0000 7f5b84ff9640 1 -- 172.21.15.40:0/1318191274 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f5b7c05a0b0 con 0x7f5b88153d20 2024-04-26T21:36:04.309 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:36:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b7406c8f0 msgr2=0x7f5b7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b7406c8f0 0x7f5b7406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b88162370 tx=0x7f5b78009040 comp rx=0 tx=0).stop 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b88153d20 msgr2=0x7f5b880afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b88153d20 0x7f5b880afdd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0118e0 tx=0x7f5b7c011da0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 shutdown_connections 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b7406c8f0 0x7f5b7406edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8806da10 0x7f5b8806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b88153d20 0x7f5b880afdd0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 --2- 172.21.15.40:0/1318191274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b70004d40 0x7f5b8814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 >> 172.21.15.40:0/1318191274 conn(0x7f5b880d2da0 msgr2=0x7f5b88070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 shutdown_connections 2024-04-26T21:36:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.311+0000 7f5b8e7da640 1 -- 172.21.15.40:0/1318191274 wait complete. 2024-04-26T21:36:04.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:36:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.426+0000 7f8233425640 1 Processor -- start 2024-04-26T21:36:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.426+0000 7f8233425640 1 -- start start 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f8233425640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f822c0b0380 0x7f822c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f8233425640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b10b0 0x7f822c06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f8233425640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f822c06c780 0x7f822c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f8233425640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f822c0cfd80 con 0x7f822c0b0380 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f8233425640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f822c0d0020 con 0x7f822c06c780 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f8233425640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f822c0d0300 con 0x7f822c0b10b0 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82311c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f822c0b0380 0x7f822c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82311c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f822c0b0380 0x7f822c0ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45648/0 (socket says 172.21.15.40:45648) 2024-04-26T21:36:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82311c1640 1 -- 172.21.15.40:0/3779358550 learned_addr learned my addr 172.21.15.40:0/3779358550 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82309c0640 1 --2- 172.21.15.40:0/3779358550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b10b0 0x7f822c06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82311c1640 1 -- 172.21.15.40:0/3779358550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b10b0 msgr2=0x7f822c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82311c1640 1 --2- 172.21.15.40:0/3779358550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b10b0 0x7f822c06c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82319c2640 1 --2- 172.21.15.40:0/3779358550 >> v2:172.21.15.184:3300/0 conn(0x7f822c06c780 0x7f822c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82311c1640 1 -- 172.21.15.40:0/3779358550 >> v2:172.21.15.184:3300/0 conn(0x7f822c06c780 msgr2=0x7f822c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82311c1640 1 --2- 172.21.15.40:0/3779358550 >> v2:172.21.15.184:3300/0 conn(0x7f822c06c780 0x7f822c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82311c1640 1 -- 172.21.15.40:0/3779358550 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c0d05e0 con 0x7f822c0b0380 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.427+0000 7f82319c2640 1 --2- 172.21.15.40:0/3779358550 >> v2:172.21.15.184:3300/0 conn(0x7f822c06c780 0x7f822c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.428+0000 7f82311c1640 1 --2- 172.21.15.40:0/3779358550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b0380 0x7f822c0ccd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f822c0c57c0 tx=0x7f822802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1c4364b6aaa648 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.428+0000 7f8223fff640 1 -- 172.21.15.40:0/3779358550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228037070 con 0x7f822c0b0380 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.428+0000 7f8223fff640 1 -- 172.21.15.40:0/3779358550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228032440 con 0x7f822c0b0380 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.428+0000 7f8223fff640 1 -- 172.21.15.40:0/3779358550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228032810 con 0x7f822c0b0380 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 -- 172.21.15.40:0/3779358550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b0380 msgr2=0x7f822c0ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 --2- 172.21.15.40:0/3779358550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b0380 0x7f822c0ccd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f822c0c57c0 tx=0x7f822802a140 comp rx=0 tx=0).stop 2024-04-26T21:36:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 -- 172.21.15.40:0/3779358550 shutdown_connections 2024-04-26T21:36:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 --2- 172.21.15.40:0/3779358550 >> v2:172.21.15.184:3300/0 conn(0x7f822c06c780 0x7f822c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 --2- 172.21.15.40:0/3779358550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b10b0 0x7f822c06c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 --2- 172.21.15.40:0/3779358550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b0380 0x7f822c0ccd50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 -- 172.21.15.40:0/3779358550 >> 172.21.15.40:0/3779358550 conn(0x7f822c0d1cf0 msgr2=0x7f822c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 -- 172.21.15.40:0/3779358550 shutdown_connections 2024-04-26T21:36:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.429+0000 7f8233425640 1 -- 172.21.15.40:0/3779358550 wait complete. 2024-04-26T21:36:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.430+0000 7f8233425640 1 Processor -- start 2024-04-26T21:36:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.430+0000 7f8233425640 1 -- start start 2024-04-26T21:36:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f8233425640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f822c06c780 0x7f822c0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f8233425640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b0380 0x7f822c0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f8233425640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b10b0 0x7f822c0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f8233425640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0d15c0 con 0x7f822c0b10b0 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f8233425640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0b0760 con 0x7f822c06c780 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f8233425640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f822c0d1740 con 0x7f822c0b0380 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b10b0 0x7f822c0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b10b0 0x7f822c0c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45656/0 (socket says 172.21.15.40:45656) 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 -- 172.21.15.40:0/3953264663 learned_addr learned my addr 172.21.15.40:0/3953264663 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82309c0640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b0380 0x7f822c0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82311c1640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f822c06c780 0x7f822c0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 -- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b0380 msgr2=0x7f822c0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b0380 0x7f822c0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 -- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f822c06c780 msgr2=0x7f822c0c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f822c06c780 0x7f822c0c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 -- 172.21.15.40:0/3953264663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c0cbcf0 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f82319c2640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b10b0 0x7f822c0c7440 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f822400cf20 tx=0x7f822400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f8221ffb640 1 -- 172.21.15.40:0/3953264663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822401a070 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.431+0000 7f8221ffb640 1 -- 172.21.15.40:0/3953264663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822400d840 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.432+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822c0c6880 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.432+0000 7f8221ffb640 1 -- 172.21.15.40:0/3953264663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822400dc30 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.432+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822c0c8e60 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.433+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822c0d0b00 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.433+0000 7f8221ffb640 1 -- 172.21.15.40:0/3953264663 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f822401d070 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.433+0000 7f8221ffb640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f821006c920 0x7f821006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.434+0000 7f8221ffb640 1 -- 172.21.15.40:0/3953264663 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f822408c720 con 0x7f822c0b10b0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.434+0000 7f82311c1640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f821006c920 0x7f821006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.434+0000 7f82311c1640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f821006c920 0x7f821006ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8228034000 tx=0x7f822802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.438+0000 7f8221ffb640 1 -- 172.21.15.40:0/3953264663 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f822c0d0b00 con 0x7f822c0b10b0 2024-04-26T21:36:04.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.621+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f822c0d2960 con 0x7f822c0b10b0 2024-04-26T21:36:04.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.622+0000 7f8221ffb640 1 -- 172.21.15.40:0/3953264663 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f822c0d2960 con 0x7f822c0b10b0 2024-04-26T21:36:04.622 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f821006c920 msgr2=0x7f821006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f821006c920 0x7f821006ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8228034000 tx=0x7f822802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b10b0 msgr2=0x7f822c0c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b10b0 0x7f822c0c7440 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f822400cf20 tx=0x7f822400a710 comp rx=0 tx=0).stop 2024-04-26T21:36:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 shutdown_connections 2024-04-26T21:36:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f821006c920 0x7f821006ede0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b10b0 0x7f822c0c7440 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822c0b0380 0x7f822c0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 --2- 172.21.15.40:0/3953264663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f822c06c780 0x7f822c0c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 >> 172.21.15.40:0/3953264663 conn(0x7f822c0d1cf0 msgr2=0x7f822c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 shutdown_connections 2024-04-26T21:36:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.624+0000 7f8233425640 1 -- 172.21.15.40:0/3953264663 wait complete. 2024-04-26T21:36:04.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:36:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.736+0000 7f3ad43e6640 1 Processor -- start 2024-04-26T21:36:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.736+0000 7f3ad43e6640 1 -- start start 2024-04-26T21:36:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad43e6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3acc0d06f0 0x7f3acc0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad43e6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3acc0b2850 0x7f3acc0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad43e6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc0d0ba0 con 0x7f3acc15f3a0 2024-04-26T21:36:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad43e6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3acc0d0e20 con 0x7f3acc0b2850 2024-04-26T21:36:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad43e6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3acc0b1d70 con 0x7f3acc0d06f0 2024-04-26T21:36:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad2983640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3acc0b2850 0x7f3acc0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad2182640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3acc15f3a0 0x7f3acc0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34954/0 (socket says 172.21.15.40:34954) 2024-04-26T21:36:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad2983640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3acc0b2850 0x7f3acc0b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45658/0 (socket says 172.21.15.40:45658) 2024-04-26T21:36:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.737+0000 7f3ad2182640 1 -- 172.21.15.40:0/835561599 learned_addr learned my addr 172.21.15.40:0/835561599 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.738+0000 7f3ad1981640 1 --2- 172.21.15.40:0/835561599 >> v2:172.21.15.184:3300/0 conn(0x7f3acc0d06f0 0x7f3acc0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.738+0000 7f3ad2983640 1 -- 172.21.15.40:0/835561599 >> v2:172.21.15.184:3300/0 conn(0x7f3acc0d06f0 msgr2=0x7f3acc0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.738+0000 7f3ad2983640 1 --2- 172.21.15.40:0/835561599 >> v2:172.21.15.184:3300/0 conn(0x7f3acc0d06f0 0x7f3acc0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.738+0000 7f3ad2983640 1 -- 172.21.15.40:0/835561599 >> v1:172.21.15.40:6789/0 conn(0x7f3acc15f3a0 legacy=0x7f3acc0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.738+0000 7f3ad2983640 1 -- 172.21.15.40:0/835561599 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc0b2050 con 0x7f3acc0b2850 2024-04-26T21:36:04.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.738+0000 7f3ad2983640 1 --2- 172.21.15.40:0/835561599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc0b2850 0x7f3acc0b2c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0b7b80 tx=0x7f3abc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e61f18aa6ebbd10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad1180640 1 -- 172.21.15.40:0/835561599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc007740 con 0x7f3acc0b2850 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad1180640 1 -- 172.21.15.40:0/835561599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc0078e0 con 0x7f3acc0b2850 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad1180640 1 -- 172.21.15.40:0/835561599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc007cb0 con 0x7f3acc0b2850 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad43e6640 1 -- 172.21.15.40:0/835561599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc0b2850 msgr2=0x7f3acc0b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/835561599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc0b2850 0x7f3acc0b2c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0b7b80 tx=0x7f3abc02a680 comp rx=0 tx=0).stop 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad43e6640 1 -- 172.21.15.40:0/835561599 shutdown_connections 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/835561599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc0b2850 0x7f3acc0b2c80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/835561599 >> v2:172.21.15.184:3300/0 conn(0x7f3acc0d06f0 0x7f3acc0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad43e6640 1 -- 172.21.15.40:0/835561599 >> 172.21.15.40:0/835561599 conn(0x7f3acc0d4f90 msgr2=0x7f3acc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad43e6640 1 -- 172.21.15.40:0/835561599 shutdown_connections 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.739+0000 7f3ad43e6640 1 -- 172.21.15.40:0/835561599 wait complete. 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad43e6640 1 Processor -- start 2024-04-26T21:36:04.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad43e6640 1 -- start start 2024-04-26T21:36:04.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad43e6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0b2850 0x7f3acc162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad43e6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc0d06f0 0x7f3acc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad43e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15f3a0 0x7f3acc15ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad43e6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc15f900 con 0x7f3acc15f3a0 2024-04-26T21:36:04.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad43e6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc15f780 con 0x7f3acc0b2850 2024-04-26T21:36:04.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad43e6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3acc0d23c0 con 0x7f3acc0d06f0 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad2983640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15f3a0 0x7f3acc15ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad2983640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15f3a0 0x7f3acc15ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45662/0 (socket says 172.21.15.40:45662) 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.741+0000 7f3ad2983640 1 -- 172.21.15.40:0/3682894748 learned_addr learned my addr 172.21.15.40:0/3682894748 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad1981640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc0d06f0 0x7f3acc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad2182640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0b2850 0x7f3acc162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad2983640 1 -- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc0d06f0 msgr2=0x7f3acc15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad2983640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc0d06f0 0x7f3acc15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad2983640 1 -- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0b2850 msgr2=0x7f3acc162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad2983640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0b2850 0x7f3acc162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad2983640 1 -- 172.21.15.40:0/3682894748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc15c870 con 0x7f3acc15f3a0 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad2983640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15f3a0 0x7f3acc15ffc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009e50 tx=0x7f3abc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3acaffd640 1 -- 172.21.15.40:0/3682894748 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc040070 con 0x7f3acc15f3a0 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3acaffd640 1 -- 172.21.15.40:0/3682894748 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc031bf0 con 0x7f3acc15f3a0 2024-04-26T21:36:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc161d50 con 0x7f3acc15f3a0 2024-04-26T21:36:04.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3acaffd640 1 -- 172.21.15.40:0/3682894748 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc045460 con 0x7f3acc15f3a0 2024-04-26T21:36:04.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.742+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc0b7cb0 con 0x7f3acc15f3a0 2024-04-26T21:36:04.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.743+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3acc0694f0 con 0x7f3acc15f3a0 2024-04-26T21:36:04.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.744+0000 7f3acaffd640 1 -- 172.21.15.40:0/3682894748 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3abc039040 con 0x7f3acc15f3a0 2024-04-26T21:36:04.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.744+0000 7f3acaffd640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ab806c8f0 0x7f3ab806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:04.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.744+0000 7f3acaffd640 1 -- 172.21.15.40:0/3682894748 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3abc0ad760 con 0x7f3acc15f3a0 2024-04-26T21:36:04.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.745+0000 7f3ad2182640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ab806c8f0 0x7f3ab806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:04.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.745+0000 7f3ad2182640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ab806c8f0 0x7f3ab806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4007fd0 tx=0x7f3ac4007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:04.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.749+0000 7f3acaffd640 1 -- 172.21.15.40:0/3682894748 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3acc15c870 con 0x7f3acc15f3a0 2024-04-26T21:36:04.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.947+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3acc0b6860 con 0x7f3acc15f3a0 2024-04-26T21:36:04.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.948+0000 7f3acaffd640 1 -- 172.21.15.40:0/3682894748 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f3abc07b130 con 0x7f3acc15f3a0 2024-04-26T21:36:04.948 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:36:04.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ab806c8f0 msgr2=0x7f3ab806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ab806c8f0 0x7f3ab806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4007fd0 tx=0x7f3ac4007e80 comp rx=0 tx=0).stop 2024-04-26T21:36:04.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15f3a0 msgr2=0x7f3acc15ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15f3a0 0x7f3acc15ffc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009e50 tx=0x7f3abc004a40 comp rx=0 tx=0).stop 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 shutdown_connections 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ab806c8f0 0x7f3ab806edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15f3a0 0x7f3acc15ffc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc0d06f0 0x7f3acc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 --2- 172.21.15.40:0/3682894748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0b2850 0x7f3acc162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 >> 172.21.15.40:0/3682894748 conn(0x7f3acc0d4f90 msgr2=0x7f3acc06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 shutdown_connections 2024-04-26T21:36:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:04.950+0000 7f3ad43e6640 1 -- 172.21.15.40:0/3682894748 wait complete. 2024-04-26T21:36:04.961 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:36:04.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:04.961 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T21:36:04.992 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:36:04.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:04.992 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:36:05.056 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:05.056 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:05.072 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:05.072 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:05.128 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:05.128 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:05.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.325+0000 7f13cfb22640 1 Processor -- start 2024-04-26T21:36:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.325+0000 7f13cfb22640 1 -- start start 2024-04-26T21:36:05.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cfb22640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13c80d1b30 0x7f13c80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cfb22640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c80d10d0 con 0x7f13c80b2340 2024-04-26T21:36:05.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cfb22640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c80d1370 con 0x7f13c8162810 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cfb22640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f13c80d0750 con 0x7f13c80d1b30 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cd0bd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f13c80b2340 0x7f13c80cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34962/0 (socket says 172.21.15.40:34962) 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cd0bd640 1 -- 172.21.15.40:0/2406280703 learned_addr learned my addr 172.21.15.40:0/2406280703 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024687186 0 0) 0x7f13c80d10d0 con 0x7f13c80b2340 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13b4003520 con 0x7f13c80b2340 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13ce0bf640 1 --2- 172.21.15.40:0/2406280703 >> v2:172.21.15.184:3300/0 conn(0x7f13c80d1b30 0x7f13c80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1581528805 0 0) 0x7f13b4003520 con 0x7f13c80b2340 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 >> v2:172.21.15.184:3300/0 conn(0x7f13c80d1b30 msgr2=0x7f13c80d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 --2- 172.21.15.40:0/2406280703 >> v2:172.21.15.184:3300/0 conn(0x7f13c80d1b30 0x7f13c80d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 >> v1:172.21.15.184:6789/0 conn(0x7f13c8162810 legacy=0x7f13c8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c80d01b0 con 0x7f13c80b2340 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f13b8001670 con 0x7f13c80b2340 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 >> v1:172.21.15.40:6789/0 conn(0x7f13c80b2340 legacy=0x7f13c80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4004800 0x7f13b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b40051c0 0x7f13b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80d0750 con 0x7f13b40039b0 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.326+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13b400c060 con 0x7f13b4004800 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13b400c260 con 0x7f13b40051c0 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cd8be640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cd0bd640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4004800 0x7f13b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13ce0bf640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b40051c0 0x7f13b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cd8be640 1 -- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b40051c0 msgr2=0x7f13b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cd8be640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b40051c0 0x7f13b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cd8be640 1 -- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4004800 msgr2=0x7f13b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cd8be640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4004800 0x7f13b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cd8be640 1 -- 172.21.15.40:0/2406280703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b4003520 con 0x7f13b40039b0 2024-04-26T21:36:05.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13ce0bf640 1 -- 172.21.15.40:0/2406280703 reap_dead start 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.327+0000 7f13cd0bd640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4004800 0x7f13b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cd8be640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13b4003dd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f13c4001750 tx=0x7f13b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc763b70faadca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4001370 con 0x7f13b40039b0 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c40052d0 con 0x7f13b40039b0 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cc8bc640 1 -- 172.21.15.40:0/2406280703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c402abe0 con 0x7f13b40039b0 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cfb22640 1 -- 172.21.15.40:0/2406280703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 msgr2=0x7f13b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cfb22640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13b4003dd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f13c4001750 tx=0x7f13b4005830 comp rx=0 tx=0).stop 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cfb22640 1 -- 172.21.15.40:0/2406280703 shutdown_connections 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cfb22640 1 --2- 172.21.15.40:0/2406280703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13b4003dd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cfb22640 1 -- 172.21.15.40:0/2406280703 >> 172.21.15.40:0/2406280703 conn(0x7f13c80d2da0 msgr2=0x7f13c80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cfb22640 1 -- 172.21.15.40:0/2406280703 shutdown_connections 2024-04-26T21:36:05.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.328+0000 7f13cfb22640 1 -- 172.21.15.40:0/2406280703 wait complete. 2024-04-26T21:36:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.329+0000 7f13cfb22640 1 Processor -- start 2024-04-26T21:36:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cfb22640 1 -- start start 2024-04-26T21:36:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cfb22640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13c81f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cfb22640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c8153d60 0x7f13c81541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cfb22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c80b0ed0 0x7f13c80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cfb22640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80d1fc0 con 0x7f13b40039b0 2024-04-26T21:36:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cfb22640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80d16a0 con 0x7f13c8153d60 2024-04-26T21:36:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cfb22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13c80d2140 con 0x7f13c80b0ed0 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cd8be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13c81f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cd8be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13c81f5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45676/0 (socket says 172.21.15.40:45676) 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.330+0000 7f13cd8be640 1 -- 172.21.15.40:0/2325444849 learned_addr learned my addr 172.21.15.40:0/2325444849 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13ce0bf640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c80b0ed0 0x7f13c80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cd0bd640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c8153d60 0x7f13c81541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cd8be640 1 -- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c80b0ed0 msgr2=0x7f13c80b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cd8be640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c80b0ed0 0x7f13c80b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cd8be640 1 -- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c8153d60 msgr2=0x7f13c81541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cd8be640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c8153d60 0x7f13c81541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cd8be640 1 -- 172.21.15.40:0/2325444849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c80c8f40 con 0x7f13b40039b0 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cd8be640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13c81f5190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f13c4001870 tx=0x7f13c4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13be7fc640 1 -- 172.21.15.40:0/2325444849 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c403f030 con 0x7f13b40039b0 2024-04-26T21:36:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13be7fc640 1 -- 172.21.15.40:0/2325444849 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c4005480 con 0x7f13b40039b0 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c80c9220 con 0x7f13b40039b0 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13be7fc640 1 -- 172.21.15.40:0/2325444849 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4037040 con 0x7f13b40039b0 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.331+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c80c97e0 con 0x7f13b40039b0 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.332+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c80b2070 con 0x7f13b40039b0 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.332+0000 7f13be7fc640 1 -- 172.21.15.40:0/2325444849 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13c4042070 con 0x7f13b40039b0 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.333+0000 7f13be7fc640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ac06c8f0 0x7f13ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.333+0000 7f13be7fc640 1 -- 172.21.15.40:0/2325444849 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f13c40ab4d0 con 0x7f13b40039b0 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.333+0000 7f13cd0bd640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ac06c8f0 0x7f13ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.334+0000 7f13cd0bd640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ac06c8f0 0x7f13ac06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13c80cd330 tx=0x7f13b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.337+0000 7f13be7fc640 1 -- 172.21.15.40:0/2325444849 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13c80b2070 con 0x7f13b40039b0 2024-04-26T21:36:05.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.526+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13c80694f0 con 0x7f13b40039b0 2024-04-26T21:36:05.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.527+0000 7f13be7fc640 1 -- 172.21.15.40:0/2325444849 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7f13c4079470 con 0x7f13b40039b0 2024-04-26T21:36:05.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:05.529 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ac06c8f0 msgr2=0x7f13ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ac06c8f0 0x7f13ac06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13c80cd330 tx=0x7f13b8009040 comp rx=0 tx=0).stop 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 msgr2=0x7f13c81f5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13c81f5190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f13c4001870 tx=0x7f13c4001120 comp rx=0 tx=0).stop 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 shutdown_connections 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ac06c8f0 0x7f13ac06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c80b0ed0 0x7f13c80b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c8153d60 0x7f13c81541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 --2- 172.21.15.40:0/2325444849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40039b0 0x7f13c81f5190 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 >> 172.21.15.40:0/2325444849 conn(0x7f13c80d2da0 msgr2=0x7f13c80cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 shutdown_connections 2024-04-26T21:36:05.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.529+0000 7f13cfb22640 1 -- 172.21.15.40:0/2325444849 wait complete. 2024-04-26T21:36:05.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:36:05.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:36:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.651+0000 7f270aff7640 1 Processor -- start 2024-04-26T21:36:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.652+0000 7f270aff7640 1 -- start start 2024-04-26T21:36:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.652+0000 7f270aff7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040ceb80 0x7f270416a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.652+0000 7f270aff7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27040d1b80 con 0x7f270415f770 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.652+0000 7f270aff7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270406e580 con 0x7f27040b1fd0 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.652+0000 7f270aff7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f270406f3e0 con 0x7f27040ceb80 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.652+0000 7f2708d93640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f270415f770 0x7f27040d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34966/0 (socket says 172.21.15.40:34966) 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.653+0000 7f2708d93640 1 -- 172.21.15.40:0/601153316 learned_addr learned my addr 172.21.15.40:0/601153316 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.653+0000 7f2709594640 1 --2- 172.21.15.40:0/601153316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040ceb80 0x7f270416a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.653+0000 7f27037fe640 1 -- 172.21.15.40:0/601153316 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1350801637 0 0) 0x7f27040d1b80 con 0x7f270415f770 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.653+0000 7f27037fe640 1 -- 172.21.15.40:0/601153316 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26f4003520 con 0x7f270415f770 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.653+0000 7f2709594640 1 -- 172.21.15.40:0/601153316 >> v1:172.21.15.184:6789/0 conn(0x7f27040b1fd0 legacy=0x7f27040d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.653+0000 7f2709594640 1 -- 172.21.15.40:0/601153316 >> v1:172.21.15.40:6789/0 conn(0x7f270415f770 legacy=0x7f27040d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.653+0000 7f2709594640 1 -- 172.21.15.40:0/601153316 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270406e860 con 0x7f27040ceb80 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.654+0000 7f2709594640 1 --2- 172.21.15.40:0/601153316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040ceb80 0x7f270416a200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009dc0 tx=0x7f26fc02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee14f3278a7f4644 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.654+0000 7f27037fe640 1 -- 172.21.15.40:0/601153316 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc007740 con 0x7f27040ceb80 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.654+0000 7f27037fe640 1 -- 172.21.15.40:0/601153316 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26fc0078e0 con 0x7f27040ceb80 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.654+0000 7f27037fe640 1 -- 172.21.15.40:0/601153316 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc007c90 con 0x7f27040ceb80 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.654+0000 7f270aff7640 1 -- 172.21.15.40:0/601153316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040ceb80 msgr2=0x7f270416a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.654+0000 7f270aff7640 1 --2- 172.21.15.40:0/601153316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040ceb80 0x7f270416a200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009dc0 tx=0x7f26fc02a6d0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.655+0000 7f270aff7640 1 -- 172.21.15.40:0/601153316 shutdown_connections 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.655+0000 7f270aff7640 1 --2- 172.21.15.40:0/601153316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040ceb80 0x7f270416a200 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.655+0000 7f270aff7640 1 -- 172.21.15.40:0/601153316 >> 172.21.15.40:0/601153316 conn(0x7f27040d2e60 msgr2=0x7f27040d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.655+0000 7f270aff7640 1 -- 172.21.15.40:0/601153316 shutdown_connections 2024-04-26T21:36:05.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.655+0000 7f270aff7640 1 -- 172.21.15.40:0/601153316 wait complete. 2024-04-26T21:36:05.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.656+0000 7f270aff7640 1 Processor -- start 2024-04-26T21:36:05.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.656+0000 7f270aff7640 1 -- start start 2024-04-26T21:36:05.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f270aff7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040b1fd0 0x7f27040c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f270aff7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27040ceb80 0x7f2704152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f270aff7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f270415f770 0x7f2704070650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f270aff7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f270416a740 con 0x7f27040ceb80 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f270aff7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27040cd6e0 con 0x7f270415f770 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f270aff7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f270416a8c0 con 0x7f27040b1fd0 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27040ceb80 0x7f2704152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27040ceb80 0x7f2704152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45688/0 (socket says 172.21.15.40:45688) 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 -- 172.21.15.40:0/529925982 learned_addr learned my addr 172.21.15.40:0/529925982 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2708d93640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040b1fd0 0x7f27040c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2709594640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f270415f770 0x7f2704070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 -- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040b1fd0 msgr2=0x7f27040c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040b1fd0 0x7f27040c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 -- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f270415f770 msgr2=0x7f2704070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f270415f770 0x7f2704070650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 -- 172.21.15.40:0/529925982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27040c7d80 con 0x7f27040ceb80 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f2703fff640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27040ceb80 0x7f2704152af0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c3c0 tx=0x7f26f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f27017fa640 1 -- 172.21.15.40:0/529925982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00052d0 con 0x7f27040ceb80 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f27017fa640 1 -- 172.21.15.40:0/529925982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f0005490 con 0x7f27040ceb80 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.657+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27040c8060 con 0x7f27040ceb80 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.658+0000 7f27017fa640 1 -- 172.21.15.40:0/529925982 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0007260 con 0x7f27040ceb80 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.658+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27040c8620 con 0x7f27040ceb80 2024-04-26T21:36:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.658+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27040694f0 con 0x7f27040ceb80 2024-04-26T21:36:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.659+0000 7f27017fa640 1 -- 172.21.15.40:0/529925982 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26f0007400 con 0x7f27040ceb80 2024-04-26T21:36:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.659+0000 7f27017fa640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e806c8f0 0x7f26e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.659+0000 7f27017fa640 1 -- 172.21.15.40:0/529925982 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f26f008b3a0 con 0x7f27040ceb80 2024-04-26T21:36:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.660+0000 7f2708d93640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e806c8f0 0x7f26e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.660+0000 7f2708d93640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e806c8f0 0x7f26e806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f26f80011e0 tx=0x7f26f8007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.664+0000 7f27017fa640 1 -- 172.21.15.40:0/529925982 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26f0058d80 con 0x7f27040ceb80 2024-04-26T21:36:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.846+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f270406fe50 con 0x7f27040ceb80 2024-04-26T21:36:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.847+0000 7f27017fa640 1 -- 172.21.15.40:0/529925982 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f26f005d080 con 0x7f27040ceb80 2024-04-26T21:36:05.847 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:36:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e806c8f0 msgr2=0x7f26e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e806c8f0 0x7f26e806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f26f80011e0 tx=0x7f26f8007d30 comp rx=0 tx=0).stop 2024-04-26T21:36:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27040ceb80 msgr2=0x7f2704152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27040ceb80 0x7f2704152af0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c3c0 tx=0x7f26f000c880 comp rx=0 tx=0).stop 2024-04-26T21:36:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 shutdown_connections 2024-04-26T21:36:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26e806c8f0 0x7f26e806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f270415f770 0x7f2704070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27040ceb80 0x7f2704152af0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 --2- 172.21.15.40:0/529925982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27040b1fd0 0x7f27040c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 >> 172.21.15.40:0/529925982 conn(0x7f27040d2e60 msgr2=0x7f2704158380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 shutdown_connections 2024-04-26T21:36:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.849+0000 7f270aff7640 1 -- 172.21.15.40:0/529925982 wait complete. 2024-04-26T21:36:05.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:36:05.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.964+0000 7fe0cab87640 1 Processor -- start 2024-04-26T21:36:05.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.964+0000 7fe0cab87640 1 -- start start 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.964+0000 7fe0cab87640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe0c415f170 0x7fe0c40d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.964+0000 7fe0cab87640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0c40d3bc0 0x7fe0c40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.964+0000 7fe0cab87640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c40d1040 con 0x7fe0c415f550 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.964+0000 7fe0cab87640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c40c9430 con 0x7fe0c40d3bc0 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.964+0000 7fe0cab87640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c40c9680 con 0x7fe0c415f170 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.965+0000 7fe0c9124640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0c40d3bc0 0x7fe0c40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.965+0000 7fe0c9124640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0c40d3bc0 0x7fe0c40d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45690/0 (socket says 172.21.15.40:45690) 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.965+0000 7fe0c9124640 1 -- 172.21.15.40:0/3242783187 learned_addr learned my addr 172.21.15.40:0/3242783187 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.965+0000 7fe0c8923640 1 --2- 172.21.15.40:0/3242783187 >> v2:172.21.15.184:3300/0 conn(0x7fe0c415f170 0x7fe0c40d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.965+0000 7fe0c9124640 1 -- 172.21.15.40:0/3242783187 >> v2:172.21.15.184:3300/0 conn(0x7fe0c415f170 msgr2=0x7fe0c40d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.965+0000 7fe0c9124640 1 --2- 172.21.15.40:0/3242783187 >> v2:172.21.15.184:3300/0 conn(0x7fe0c415f170 0x7fe0c40d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.965+0000 7fe0c9124640 1 -- 172.21.15.40:0/3242783187 >> v1:172.21.15.184:6789/0 conn(0x7fe0c415f550 legacy=0x7fe0c40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.965+0000 7fe0c9124640 1 -- 172.21.15.40:0/3242783187 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c40c9960 con 0x7fe0c40d3bc0 2024-04-26T21:36:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0c9124640 1 --2- 172.21.15.40:0/3242783187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c40d3bc0 0x7fe0c40d2cb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009e50 tx=0x7fe0bc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306414bab38a2258 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0c37fe640 1 -- 172.21.15.40:0/3242783187 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc036070 con 0x7fe0c40d3bc0 2024-04-26T21:36:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0c37fe640 1 -- 172.21.15.40:0/3242783187 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0bc031600 con 0x7fe0c40d3bc0 2024-04-26T21:36:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0c37fe640 1 -- 172.21.15.40:0/3242783187 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0319d0 con 0x7fe0c40d3bc0 2024-04-26T21:36:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0cab87640 1 -- 172.21.15.40:0/3242783187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c40d3bc0 msgr2=0x7fe0c40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0cab87640 1 --2- 172.21.15.40:0/3242783187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c40d3bc0 0x7fe0c40d2cb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009e50 tx=0x7fe0bc02a760 comp rx=0 tx=0).stop 2024-04-26T21:36:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0cab87640 1 -- 172.21.15.40:0/3242783187 shutdown_connections 2024-04-26T21:36:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0cab87640 1 --2- 172.21.15.40:0/3242783187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c40d3bc0 0x7fe0c40d2cb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0cab87640 1 --2- 172.21.15.40:0/3242783187 >> v2:172.21.15.184:3300/0 conn(0x7fe0c415f170 0x7fe0c40d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0cab87640 1 -- 172.21.15.40:0/3242783187 >> 172.21.15.40:0/3242783187 conn(0x7fe0c40d4f60 msgr2=0x7fe0c40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0cab87640 1 -- 172.21.15.40:0/3242783187 shutdown_connections 2024-04-26T21:36:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.966+0000 7fe0cab87640 1 -- 172.21.15.40:0/3242783187 wait complete. 2024-04-26T21:36:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.968+0000 7fe0cab87640 1 Processor -- start 2024-04-26T21:36:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.968+0000 7fe0cab87640 1 -- start start 2024-04-26T21:36:05.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.968+0000 7fe0cab87640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0c40d3bc0 0x7fe0c40a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.968+0000 7fe0cab87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c415f170 0x7fe0c4158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.968+0000 7fe0cab87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0c40a0970 0x7fe0c41593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.968+0000 7fe0cab87640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c40ca5c0 con 0x7fe0c415f170 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.968+0000 7fe0cab87640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c40ca440 con 0x7fe0c40a0970 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.968+0000 7fe0cab87640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c40ca740 con 0x7fe0c40d3bc0 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c415f170 0x7fe0c4158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c415f170 0x7fe0c4158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45706/0 (socket says 172.21.15.40:45706) 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 -- 172.21.15.40:0/570626059 learned_addr learned my addr 172.21.15.40:0/570626059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c8923640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0c40d3bc0 0x7fe0c40a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c9124640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0c40a0970 0x7fe0c41593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 -- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0c40d3bc0 msgr2=0x7fe0c40a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0c40d3bc0 0x7fe0c40a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 -- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0c40a0970 msgr2=0x7fe0c41593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0c40a0970 0x7fe0c41593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 -- 172.21.15.40:0/570626059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c40a0e80 con 0x7fe0c415f170 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c9124640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0c40a0970 0x7fe0c41593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c3fff640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c415f170 0x7fe0c4158e70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800c3c0 tx=0x7fe0b800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c17fa640 1 -- 172.21.15.40:0/570626059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80052d0 con 0x7fe0c415f170 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c17fa640 1 -- 172.21.15.40:0/570626059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b8005490 con 0x7fe0c415f170 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c40a1160 con 0x7fe0c415f170 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0c17fa640 1 -- 172.21.15.40:0/570626059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8007260 con 0x7fe0c415f170 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.969+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c40a1720 con 0x7fe0c415f170 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.970+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c40694f0 con 0x7fe0c415f170 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.971+0000 7fe0c17fa640 1 -- 172.21.15.40:0/570626059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0b8007400 con 0x7fe0c415f170 2024-04-26T21:36:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.971+0000 7fe0c17fa640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0b006c8f0 0x7fe0b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.971+0000 7fe0c17fa640 1 -- 172.21.15.40:0/570626059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe0b808b460 con 0x7fe0c415f170 2024-04-26T21:36:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.972+0000 7fe0c8923640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0b006c8f0 0x7fe0b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.972+0000 7fe0c8923640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0b006c8f0 0x7fe0b006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4006fd0 tx=0x7fe0b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:05.975+0000 7fe0c17fa640 1 -- 172.21.15.40:0/570626059 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0c40d1040 con 0x7fe0c415f170 2024-04-26T21:36:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.163+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0c406de10 con 0x7fe0c415f170 2024-04-26T21:36:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.163+0000 7fe0c17fa640 1 -- 172.21.15.40:0/570626059 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fe0b8059490 con 0x7fe0c415f170 2024-04-26T21:36:06.164 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:06.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0b006c8f0 msgr2=0x7fe0b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0b006c8f0 0x7fe0b006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4006fd0 tx=0x7fe0b4008040 comp rx=0 tx=0).stop 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c415f170 msgr2=0x7fe0c4158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c415f170 0x7fe0c4158e70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800c3c0 tx=0x7fe0b800c7c0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 shutdown_connections 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0b006c8f0 0x7fe0b006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0c40a0970 0x7fe0c41593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0c415f170 0x7fe0c4158e70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 --2- 172.21.15.40:0/570626059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0c40d3bc0 0x7fe0c40a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 >> 172.21.15.40:0/570626059 conn(0x7fe0c40d4f60 msgr2=0x7fe0c406d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 shutdown_connections 2024-04-26T21:36:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.165+0000 7fe0cab87640 1 -- 172.21.15.40:0/570626059 wait complete. 2024-04-26T21:36:06.176 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:36:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.280+0000 7f299a1da640 1 Processor -- start 2024-04-26T21:36:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.280+0000 7f299a1da640 1 -- start start 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f299a1da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29940b10b0 0x7f29940ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f299a1da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299406c950 0x7f299415e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f299a1da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29940cfd80 con 0x7f29940b0380 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f299a1da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f29940af8e0 con 0x7f29940b10b0 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f299a1da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f29940d0020 con 0x7f299406c950 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f2992ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29940b10b0 0x7f29940ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f2992ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29940b10b0 0x7f29940ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45716/0 (socket says 172.21.15.40:45716) 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f2992ffd640 1 -- 172.21.15.40:0/2056783491 learned_addr learned my addr 172.21.15.40:0/2056783491 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f2993fff640 1 --2- 172.21.15.40:0/2056783491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299406c950 0x7f299415e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f2992ffd640 1 -- 172.21.15.40:0/2056783491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299406c950 msgr2=0x7f299415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f2992ffd640 1 --2- 172.21.15.40:0/2056783491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299406c950 0x7f299415e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f2992ffd640 1 -- 172.21.15.40:0/2056783491 >> v1:172.21.15.184:6789/0 conn(0x7f29940b0380 legacy=0x7f299406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.281+0000 7f2992ffd640 1 -- 172.21.15.40:0/2056783491 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29940d0300 con 0x7f29940b10b0 2024-04-26T21:36:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f2992ffd640 1 --2- 172.21.15.40:0/2056783491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b10b0 0x7f29940ccd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f29940afa60 tx=0x7f298402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde07bf54255456c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f29927fc640 1 -- 172.21.15.40:0/2056783491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984037070 con 0x7f29940b10b0 2024-04-26T21:36:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f29927fc640 1 -- 172.21.15.40:0/2056783491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2984032440 con 0x7f29940b10b0 2024-04-26T21:36:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f29927fc640 1 -- 172.21.15.40:0/2056783491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298402ae60 con 0x7f29940b10b0 2024-04-26T21:36:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f299a1da640 1 -- 172.21.15.40:0/2056783491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b10b0 msgr2=0x7f29940ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f299a1da640 1 --2- 172.21.15.40:0/2056783491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b10b0 0x7f29940ccd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f29940afa60 tx=0x7f298402a150 comp rx=0 tx=0).stop 2024-04-26T21:36:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f299a1da640 1 -- 172.21.15.40:0/2056783491 shutdown_connections 2024-04-26T21:36:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f299a1da640 1 --2- 172.21.15.40:0/2056783491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299406c950 0x7f299415e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f299a1da640 1 --2- 172.21.15.40:0/2056783491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b10b0 0x7f29940ccd50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.282+0000 7f299a1da640 1 -- 172.21.15.40:0/2056783491 >> 172.21.15.40:0/2056783491 conn(0x7f29940d1cf0 msgr2=0x7f29940d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.283+0000 7f299a1da640 1 -- 172.21.15.40:0/2056783491 shutdown_connections 2024-04-26T21:36:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.283+0000 7f299a1da640 1 -- 172.21.15.40:0/2056783491 wait complete. 2024-04-26T21:36:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.284+0000 7f299a1da640 1 Processor -- start 2024-04-26T21:36:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.284+0000 7f299a1da640 1 -- start start 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f299a1da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299406c950 0x7f29940c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f299a1da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b0380 0x7f29940b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f299a1da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29940b10b0 0x7f2994161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f299a1da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f29940b0760 con 0x7f29940b0380 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f299a1da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29940d1980 con 0x7f299406c950 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f299a1da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f29940d15c0 con 0x7f29940b10b0 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b0380 0x7f29940b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b0380 0x7f29940b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45724/0 (socket says 172.21.15.40:45724) 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 -- 172.21.15.40:0/195038082 learned_addr learned my addr 172.21.15.40:0/195038082 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f29937fe640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299406c950 0x7f29940c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2993fff640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29940b10b0 0x7f2994161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 -- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29940b10b0 msgr2=0x7f2994161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29940b10b0 0x7f2994161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 -- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299406c950 msgr2=0x7f29940c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299406c950 0x7f29940c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 -- 172.21.15.40:0/195038082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29940c9300 con 0x7f29940b0380 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f29937fe640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299406c950 0x7f29940c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f2992ffd640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b0380 0x7f29940b2af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f298402a190 tx=0x7f29840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f298bfff640 1 -- 172.21.15.40:0/195038082 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984041070 con 0x7f29940b0380 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f298bfff640 1 -- 172.21.15.40:0/195038082 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f298403c870 con 0x7f29940b0380 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f298bfff640 1 -- 172.21.15.40:0/195038082 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984037040 con 0x7f29940b0380 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.285+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29940c95a0 con 0x7f29940b0380 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.286+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29940c9b60 con 0x7f29940b0380 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.287+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29940694f0 con 0x7f29940b0380 2024-04-26T21:36:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.287+0000 7f298bfff640 1 -- 172.21.15.40:0/195038082 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2984044070 con 0x7f29940b0380 2024-04-26T21:36:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.287+0000 7f298bfff640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f297406c8f0 0x7f297406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.287+0000 7f298bfff640 1 -- 172.21.15.40:0/195038082 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f29840acad0 con 0x7f29940b0380 2024-04-26T21:36:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.288+0000 7f29937fe640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f297406c8f0 0x7f297406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.288+0000 7f29937fe640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f297406c8f0 0x7f297406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f297c007fd0 tx=0x7f297c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.292+0000 7f298bfff640 1 -- 172.21.15.40:0/195038082 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f298407ac20 con 0x7f29940b0380 2024-04-26T21:36:06.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.476+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f299406ced0 con 0x7f29940b0380 2024-04-26T21:36:06.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.477+0000 7f298bfff640 1 -- 172.21.15.40:0/195038082 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f298407a520 con 0x7f29940b0380 2024-04-26T21:36:06.477 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:36:06.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f297406c8f0 msgr2=0x7f297406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f297406c8f0 0x7f297406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f297c007fd0 tx=0x7f297c009040 comp rx=0 tx=0).stop 2024-04-26T21:36:06.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b0380 msgr2=0x7f29940b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b0380 0x7f29940b2af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f298402a190 tx=0x7f29840046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 shutdown_connections 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f297406c8f0 0x7f297406edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29940b10b0 0x7f2994161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29940b0380 0x7f29940b2af0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 --2- 172.21.15.40:0/195038082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299406c950 0x7f29940c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 >> 172.21.15.40:0/195038082 conn(0x7f29940d1cf0 msgr2=0x7f29940709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 shutdown_connections 2024-04-26T21:36:06.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.479+0000 7f299a1da640 1 -- 172.21.15.40:0/195038082 wait complete. 2024-04-26T21:36:06.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:36:06.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.593+0000 7fbe14e85640 1 Processor -- start 2024-04-26T21:36:06.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.593+0000 7fbe14e85640 1 -- start start 2024-04-26T21:36:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.594+0000 7fbe14e85640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b4360 0x7fbe100b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.594+0000 7fbe14e85640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe100ce5e0 con 0x7fbe100d0d60 2024-04-26T21:36:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.594+0000 7fbe14e85640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe100ce8c0 con 0x7fbe100d1240 2024-04-26T21:36:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.594+0000 7fbe14e85640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe100cf210 con 0x7fbe100b4360 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.594+0000 7fbe0ed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbe100d0d60 0x7fbe100b2b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34976/0 (socket says 172.21.15.40:34976) 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.594+0000 7fbe0ed9d640 1 -- 172.21.15.40:0/1781081909 learned_addr learned my addr 172.21.15.40:0/1781081909 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0dd9b640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b4360 0x7fbe100b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3045231924 0 0) 0x7fbe100ce5e0 con 0x7fbe100d0d60 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdfc003520 con 0x7fbe100d0d60 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3157575920 0 0) 0x7fbdfc003520 con 0x7fbe100d0d60 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b4360 msgr2=0x7fbe100b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b4360 0x7fbe100b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 >> v1:172.21.15.184:6789/0 conn(0x7fbe100d1240 legacy=0x7fbe100d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe10160ab0 con 0x7fbe100d0d60 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fbe000019f0 con 0x7fbe100d0d60 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 >> v1:172.21.15.40:6789/0 conn(0x7fbe100d0d60 legacy=0x7fbe100b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdfc0039e0 0x7fbdfc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc004800 0x7fbdfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdfc0051c0 0x7fbdfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbe100cf210 con 0x7fbdfc0039e0 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc00bf00 con 0x7fbdfc004800 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbdfc00c080 con 0x7fbdfc0051c0 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0e59c640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdfc0039e0 0x7fbdfc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0dd9b640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc004800 0x7fbdfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0ed9d640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdfc0051c0 0x7fbdfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0e59c640 1 -- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdfc0051c0 msgr2=0x7fbdfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0e59c640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdfc0051c0 0x7fbdfc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0e59c640 1 -- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc004800 msgr2=0x7fbdfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0e59c640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc004800 0x7fbdfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.595+0000 7fbe0e59c640 1 -- 172.21.15.40:0/1781081909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc003520 con 0x7fbdfc0039e0 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.596+0000 7fbe0ed9d640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdfc0051c0 0x7fbdfc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.596+0000 7fbe0e59c640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdfc0039e0 0x7fbdfc003e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbe08001750 tx=0x7fbdfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4655c50bb8a12691 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.596+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08001370 con 0x7fbdfc0039e0 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.596+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe080052d0 con 0x7fbdfc0039e0 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.596+0000 7fbe0d59a640 1 -- 172.21.15.40:0/1781081909 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0802abe0 con 0x7fbdfc0039e0 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.597+0000 7fbe14e85640 1 -- 172.21.15.40:0/1781081909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdfc0039e0 msgr2=0x7fbdfc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.597+0000 7fbe14e85640 1 --2- 172.21.15.40:0/1781081909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdfc0039e0 0x7fbdfc003e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbe08001750 tx=0x7fbdfc005830 comp rx=0 tx=0).stop 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.597+0000 7fbe0ed9d640 1 -- 172.21.15.40:0/1781081909 reap_dead start 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.597+0000 7fbe14e85640 1 -- 172.21.15.40:0/1781081909 shutdown_connections 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.597+0000 7fbe14e85640 1 -- 172.21.15.40:0/1781081909 >> 172.21.15.40:0/1781081909 conn(0x7fbe100d2e60 msgr2=0x7fbe100d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:06.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.597+0000 7fbe14e85640 1 -- 172.21.15.40:0/1781081909 shutdown_connections 2024-04-26T21:36:06.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.597+0000 7fbe14e85640 1 -- 172.21.15.40:0/1781081909 wait complete. 2024-04-26T21:36:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.598+0000 7fbe14e85640 1 Processor -- start 2024-04-26T21:36:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.598+0000 7fbe14e85640 1 -- start start 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe14e85640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc0039e0 0x7fbe100cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe14e85640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b1ad0 0x7fbe10153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe14e85640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe100c7370 0x7fbe10071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe14e85640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbe100c6790 con 0x7fbe100c7370 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe14e85640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe100c6610 con 0x7fbdfc0039e0 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe14e85640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe100b2600 con 0x7fbe100b1ad0 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe0ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe100c7370 0x7fbe10071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe0ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe100c7370 0x7fbe10071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45752/0 (socket says 172.21.15.40:45752) 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe0ed9d640 1 -- 172.21.15.40:0/369491565 learned_addr learned my addr 172.21.15.40:0/369491565 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe0e59c640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc0039e0 0x7fbe100cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.599+0000 7fbe0dd9b640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b1ad0 0x7fbe10153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe0ed9d640 1 -- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b1ad0 msgr2=0x7fbe10153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe0ed9d640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b1ad0 0x7fbe10153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe0ed9d640 1 -- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc0039e0 msgr2=0x7fbe100cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe0ed9d640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc0039e0 0x7fbe100cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe0ed9d640 1 -- 172.21.15.40:0/369491565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe101f6230 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe0ed9d640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe100c7370 0x7fbe10071330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbe100767a0 tx=0x7fbe0000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe06ffd640 1 -- 172.21.15.40:0/369491565 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000ebc0 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe06ffd640 1 -- 172.21.15.40:0/369491565 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0000ed60 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe101f6510 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe06ffd640 1 -- 172.21.15.40:0/369491565 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00014670 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.600+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe101f6ad0 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.601+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe100694f0 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.602+0000 7fbe06ffd640 1 -- 172.21.15.40:0/369491565 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe00014810 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.602+0000 7fbe06ffd640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdf006c8f0 0x7fbdf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.602+0000 7fbe06ffd640 1 -- 172.21.15.40:0/369491565 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbe0008bd60 con 0x7fbe100c7370 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.602+0000 7fbe0e59c640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdf006c8f0 0x7fbdf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.603+0000 7fbe0e59c640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdf006c8f0 0x7fbdf006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbe08032000 tx=0x7fbe0802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.606+0000 7fbe06ffd640 1 -- 172.21.15.40:0/369491565 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe00059740 con 0x7fbe100c7370 2024-04-26T21:36:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.790+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbe100cd240 con 0x7fbe100c7370 2024-04-26T21:36:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.790+0000 7fbe06ffd640 1 -- 172.21.15.40:0/369491565 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fbe0005da40 con 0x7fbe100c7370 2024-04-26T21:36:06.791 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:06.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdf006c8f0 msgr2=0x7fbdf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdf006c8f0 0x7fbdf006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbe08032000 tx=0x7fbe0802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe100c7370 msgr2=0x7fbe10071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe100c7370 0x7fbe10071330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbe100767a0 tx=0x7fbe0000e530 comp rx=0 tx=0).stop 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 shutdown_connections 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbdf006c8f0 0x7fbdf006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe100c7370 0x7fbe10071330 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe100b1ad0 0x7fbe10153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 --2- 172.21.15.40:0/369491565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdfc0039e0 0x7fbe100cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 >> 172.21.15.40:0/369491565 conn(0x7fbe100d2e60 msgr2=0x7fbe10070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 shutdown_connections 2024-04-26T21:36:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.792+0000 7fbe14e85640 1 -- 172.21.15.40:0/369491565 wait complete. 2024-04-26T21:36:06.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:36:06.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.905+0000 7f37d2220640 1 Processor -- start 2024-04-26T21:36:06.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.905+0000 7f37d2220640 1 -- start start 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37d2220640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc15f3a0 0x7f37cc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37d2220640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f37cc0d2300 0x7f37cc0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37d2220640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37cc0d0e20 con 0x7f37cc0d1770 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37d2220640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f37cc0b1d70 con 0x7f37cc0d2300 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37d2220640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc0b2050 con 0x7f37cc15f3a0 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37caffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f37cc0d1770 0x7f37cc0d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34982/0 (socket says 172.21.15.40:34982) 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37caffd640 1 -- 172.21.15.40:0/3523316591 learned_addr learned my addr 172.21.15.40:0/3523316591 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37cb7fe640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc15f3a0 0x7f37cc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1603999850 0 0) 0x7f37cc0d0e20 con 0x7f37cc0d1770 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37b4003520 con 0x7f37cc0d1770 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37cbfff640 1 --2- 172.21.15.40:0/3523316591 >> v2:172.21.15.184:3300/0 conn(0x7f37cc0d2300 0x7f37cc0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4072165084 0 0) 0x7f37b4003520 con 0x7f37cc0d1770 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc15f3a0 msgr2=0x7f37cc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc15f3a0 0x7f37cc0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 >> v2:172.21.15.184:3300/0 conn(0x7f37cc0d2300 msgr2=0x7f37cc0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 --2- 172.21.15.40:0/3523316591 >> v2:172.21.15.184:3300/0 conn(0x7f37cc0d2300 0x7f37cc0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc0b2330 con 0x7f37cc0d1770 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f37c0001670 con 0x7f37cc0d1770 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.906+0000 7f37cbfff640 1 --2- 172.21.15.40:0/3523316591 >> v2:172.21.15.184:3300/0 conn(0x7f37cc0d2300 0x7f37cc0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 >> v1:172.21.15.40:6789/0 conn(0x7f37cc0d1770 legacy=0x7f37cc0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37ca7fc640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b40039b0 0x7f37b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37ca7fc640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b4004800 0x7f37b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37ca7fc640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc15f3a0 0x7f37b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc0b1d70 con 0x7f37b4004800 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc0b2050 con 0x7f37b40039b0 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37b400b9c0 con 0x7f37cc15f3a0 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37caffd640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b4004800 0x7f37b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37cb7fe640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b40039b0 0x7f37b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37cbfff640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc15f3a0 0x7f37b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37caffd640 1 -- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc15f3a0 msgr2=0x7f37b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37caffd640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc15f3a0 0x7f37b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37caffd640 1 -- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b40039b0 msgr2=0x7f37b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37caffd640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b40039b0 0x7f37b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.907+0000 7f37caffd640 1 -- 172.21.15.40:0/3523316591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b4003520 con 0x7f37b4004800 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37caffd640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b4004800 0x7f37b4004c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f37c0001050 tx=0x7f37b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d940e89d0466d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c002e070 con 0x7f37b4004800 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c0005af0 con 0x7f37b4004800 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37ca7fc640 1 -- 172.21.15.40:0/3523316591 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0007d70 con 0x7f37b4004800 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37d2220640 1 -- 172.21.15.40:0/3523316591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b4004800 msgr2=0x7f37b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37d2220640 1 --2- 172.21.15.40:0/3523316591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b4004800 0x7f37b4004c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f37c0001050 tx=0x7f37b4005830 comp rx=0 tx=0).stop 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37cbfff640 1 -- 172.21.15.40:0/3523316591 reap_dead start 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37d2220640 1 -- 172.21.15.40:0/3523316591 shutdown_connections 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.908+0000 7f37d2220640 1 -- 172.21.15.40:0/3523316591 >> 172.21.15.40:0/3523316591 conn(0x7f37cc0d4f90 msgr2=0x7f37cc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.909+0000 7f37d2220640 1 -- 172.21.15.40:0/3523316591 shutdown_connections 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.909+0000 7f37d2220640 1 -- 172.21.15.40:0/3523316591 wait complete. 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.910+0000 7f37d2220640 1 Processor -- start 2024-04-26T21:36:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.910+0000 7f37d2220640 1 -- start start 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37d2220640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b4004800 0x7f37cc1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37d2220640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37cc0b8540 0x7f37cc0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37d2220640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0ce920 0x7f37cc0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37d2220640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc0d0870 con 0x7f37cc0b8540 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37d2220640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc0d06f0 con 0x7f37b4004800 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37d2220640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc0d09f0 con 0x7f37cc0ce920 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37cc0b8540 0x7f37cc0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37cc0b8540 0x7f37cc0ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45780/0 (socket says 172.21.15.40:45780) 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 -- 172.21.15.40:0/410718970 learned_addr learned my addr 172.21.15.40:0/410718970 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37cb7fe640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b4004800 0x7f37cc1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37cbfff640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0ce920 0x7f37cc0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 -- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0ce920 msgr2=0x7f37cc0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0ce920 0x7f37cc0cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 -- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b4004800 msgr2=0x7f37cc1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b4004800 0x7f37cc1594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 -- 172.21.15.40:0/410718970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc0b8e20 con 0x7f37cc0b8540 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37caffd640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37cc0b8540 0x7f37cc0ce3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c300 tx=0x7f37c0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37b3fff640 1 -- 172.21.15.40:0/410718970 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0042070 con 0x7f37cc0b8540 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37b3fff640 1 -- 172.21.15.40:0/410718970 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c0007480 con 0x7f37cc0b8540 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc0b9100 con 0x7f37cc0b8540 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37b3fff640 1 -- 172.21.15.40:0/410718970 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0007d50 con 0x7f37cc0b8540 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.911+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc0cc060 con 0x7f37cc0b8540 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.912+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc0694f0 con 0x7f37cc0b8540 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.913+0000 7f37b3fff640 1 -- 172.21.15.40:0/410718970 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37c0045070 con 0x7f37cc0b8540 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.913+0000 7f37b3fff640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b806c8f0 0x7f37b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.913+0000 7f37b3fff640 1 -- 172.21.15.40:0/410718970 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f37c00ac930 con 0x7f37cc0b8540 2024-04-26T21:36:06.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.914+0000 7f37cb7fe640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b806c8f0 0x7f37b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:06.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.914+0000 7f37cb7fe640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b806c8f0 0x7f37b806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0099c0 tx=0x7f37bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:06.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:06.918+0000 7f37b3fff640 1 -- 172.21.15.40:0/410718970 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37c0005ac0 con 0x7f37cc0b8540 2024-04-26T21:36:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.114+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f37cc0b6860 con 0x7f37cc0b8540 2024-04-26T21:36:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.114+0000 7f37b3fff640 1 -- 172.21.15.40:0/410718970 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f37c007ab10 con 0x7f37cc0b8540 2024-04-26T21:36:07.115 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:36:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b806c8f0 msgr2=0x7f37b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b806c8f0 0x7f37b806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0099c0 tx=0x7f37bc008040 comp rx=0 tx=0).stop 2024-04-26T21:36:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37cc0b8540 msgr2=0x7f37cc0ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37cc0b8540 0x7f37cc0ce3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c300 tx=0x7f37c0000fe0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 shutdown_connections 2024-04-26T21:36:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b806c8f0 0x7f37b806edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0ce920 0x7f37cc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37cc0b8540 0x7f37cc0ce3e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 --2- 172.21.15.40:0/410718970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b4004800 0x7f37cc1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.116+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 >> 172.21.15.40:0/410718970 conn(0x7f37cc0d4f90 msgr2=0x7f37cc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.117+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 shutdown_connections 2024-04-26T21:36:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.117+0000 7f37d2220640 1 -- 172.21.15.40:0/410718970 wait complete. 2024-04-26T21:36:07.128 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T21:36:07.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:07.128 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T21:36:07.153 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:36:07.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:07.153 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T21:36:07.208 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:36:07.208 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:07.227 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:36:07.227 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:07.286 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:36:07.286 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:07.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:07.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.445+0000 7f15b98c4640 1 Processor -- start 2024-04-26T21:36:07.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.445+0000 7f15b98c4640 1 -- start start 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b98c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 0x7f15b406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b98c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f15b406fcd0 0x7f15b40ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b98c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b4160970 con 0x7f15b415f2d0 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b98c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f15b4074010 con 0x7f15b406fcd0 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b98c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15b4070100 con 0x7f15b41604e0 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f15b415f2d0 0x7f15b4072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34990/0 (socket says 172.21.15.40:34990) 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b2ffd640 1 -- 172.21.15.40:0/1630740820 learned_addr learned my addr 172.21.15.40:0/1630740820 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15abfff640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 0x7f15b406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b37fe640 1 --2- 172.21.15.40:0/1630740820 >> v2:172.21.15.184:3300/0 conn(0x7f15b406fcd0 0x7f15b40ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119571897 0 0) 0x7f15b4160970 con 0x7f15b415f2d0 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15a0003520 con 0x7f15b415f2d0 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.446+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3062034433 0 0) 0x7f15a0003520 con 0x7f15b415f2d0 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 msgr2=0x7f15b406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 0x7f15b406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 >> v2:172.21.15.184:3300/0 conn(0x7f15b406fcd0 msgr2=0x7f15b40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 --2- 172.21.15.40:0/1630740820 >> v2:172.21.15.184:3300/0 conn(0x7f15b406fcd0 0x7f15b40ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b4072f70 con 0x7f15b415f2d0 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f15a4001670 con 0x7f15b415f2d0 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15abfff640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 0x7f15b406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 >> v1:172.21.15.40:6789/0 conn(0x7f15b415f2d0 legacy=0x7f15b4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 0x7f15a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a0004d40 0x7f15a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4074010 con 0x7f15a00039b0 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4070100 con 0x7f15a0004d40 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15a000b9c0 con 0x7f15b41604e0 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b2ffd640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15abfff640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 0x7f15a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b37fe640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a0004d40 0x7f15a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b2ffd640 1 -- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 msgr2=0x7f15a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b2ffd640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b41604e0 0x7f15a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b2ffd640 1 -- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a0004d40 msgr2=0x7f15a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b2ffd640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a0004d40 0x7f15a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.447+0000 7f15b2ffd640 1 -- 172.21.15.40:0/1630740820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a0003520 con 0x7f15a00039b0 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b2ffd640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15a0003dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c290 tx=0x7f15a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85204155fcc438f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4007d50 con 0x7f15a00039b0 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a4005af0 con 0x7f15a00039b0 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b27fc640 1 -- 172.21.15.40:0/1630740820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4007070 con 0x7f15a00039b0 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b98c4640 1 -- 172.21.15.40:0/1630740820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 msgr2=0x7f15a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b98c4640 1 --2- 172.21.15.40:0/1630740820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15a0003dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c290 tx=0x7f15a0005830 comp rx=0 tx=0).stop 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b37fe640 1 -- 172.21.15.40:0/1630740820 reap_dead start 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b98c4640 1 -- 172.21.15.40:0/1630740820 shutdown_connections 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.448+0000 7f15b98c4640 1 -- 172.21.15.40:0/1630740820 >> 172.21.15.40:0/1630740820 conn(0x7f15b40d0be0 msgr2=0x7f15b40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.449+0000 7f15b98c4640 1 -- 172.21.15.40:0/1630740820 shutdown_connections 2024-04-26T21:36:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.449+0000 7f15b98c4640 1 -- 172.21.15.40:0/1630740820 wait complete. 2024-04-26T21:36:07.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.450+0000 7f15b98c4640 1 Processor -- start 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.450+0000 7f15b98c4640 1 -- start start 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.450+0000 7f15b98c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15b40cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.450+0000 7f15b98c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b415d2d0 0x7f15b415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.450+0000 7f15b98c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b40c8530 0x7f15b40c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.450+0000 7f15b98c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4073810 con 0x7f15a00039b0 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.450+0000 7f15b98c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4073530 con 0x7f15b415d2d0 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.450+0000 7f15b98c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15b4073af0 con 0x7f15b40c8530 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15b40cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15b40cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45806/0 (socket says 172.21.15.40:45806) 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 -- 172.21.15.40:0/3104996615 learned_addr learned my addr 172.21.15.40:0/3104996615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15abfff640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b415d2d0 0x7f15b415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b37fe640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b40c8530 0x7f15b40c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 -- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b40c8530 msgr2=0x7f15b40c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b40c8530 0x7f15b40c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 -- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b415d2d0 msgr2=0x7f15b415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b415d2d0 0x7f15b415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 -- 172.21.15.40:0/3104996615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b415d800 con 0x7f15a00039b0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15abfff640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b415d2d0 0x7f15b415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b2ffd640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15b40cfd30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f15a4001970 tx=0x7f15b40c7970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b0ff9640 1 -- 172.21.15.40:0/3104996615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4042070 con 0x7f15a00039b0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b0ff9640 1 -- 172.21.15.40:0/3104996615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a4007480 con 0x7f15a00039b0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b415dae0 con 0x7f15a00039b0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b0ff9640 1 -- 172.21.15.40:0/3104996615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4001c80 con 0x7f15a00039b0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.451+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b415e380 con 0x7f15a00039b0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.452+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b415ded0 con 0x7f15a00039b0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.453+0000 7f15b0ff9640 1 -- 172.21.15.40:0/3104996615 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15a4045070 con 0x7f15a00039b0 2024-04-26T21:36:07.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.453+0000 7f15b0ff9640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f159406c920 0x7f159406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.453+0000 7f15b0ff9640 1 -- 172.21.15.40:0/3104996615 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f15a40acc50 con 0x7f15a00039b0 2024-04-26T21:36:07.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.453+0000 7f15abfff640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f159406c920 0x7f159406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.454+0000 7f15abfff640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f159406c920 0x7f159406ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15b4160260 tx=0x7f159c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:07.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.457+0000 7f15b0ff9640 1 -- 172.21.15.40:0/3104996615 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f15a40b2020 con 0x7f15a00039b0 2024-04-26T21:36:07.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.645+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15b4070800 con 0x7f15a00039b0 2024-04-26T21:36:07.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.646+0000 7f15b0ff9640 1 -- 172.21.15.40:0/3104996615 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7f15b4070800 con 0x7f15a00039b0 2024-04-26T21:36:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:07.647 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f159406c920 msgr2=0x7f159406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f159406c920 0x7f159406ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15b4160260 tx=0x7f159c008040 comp rx=0 tx=0).stop 2024-04-26T21:36:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 msgr2=0x7f15b40cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15b40cfd30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f15a4001970 tx=0x7f15b40c7970 comp rx=0 tx=0).stop 2024-04-26T21:36:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 shutdown_connections 2024-04-26T21:36:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f159406c920 0x7f159406ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b40c8530 0x7f15b40c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b415d2d0 0x7f15b415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 --2- 172.21.15.40:0/3104996615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a00039b0 0x7f15b40cfd30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 >> 172.21.15.40:0/3104996615 conn(0x7f15b40d0be0 msgr2=0x7f15b40c6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:07.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 shutdown_connections 2024-04-26T21:36:07.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.649+0000 7f15b98c4640 1 -- 172.21.15.40:0/3104996615 wait complete. 2024-04-26T21:36:07.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:36:07.661 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:36:07.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.764+0000 7f54dc845640 1 Processor -- start 2024-04-26T21:36:07.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.764+0000 7f54dc845640 1 -- start start 2024-04-26T21:36:07.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54dc845640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54d80d1240 0x7f54d80d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54dc845640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f54d80b4360 0x7f54d80cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54dc845640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d80d1b60 con 0x7f54d80d0d60 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54dc845640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f54d80cf210 con 0x7f54d80d1240 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54dc845640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f54d80ce5e0 con 0x7f54d80b4360 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54d659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54d80d1240 0x7f54d80d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54d659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54d80d1240 0x7f54d80d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45822/0 (socket says 172.21.15.40:45822) 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54d659c640 1 -- 172.21.15.40:0/4244271758 learned_addr learned my addr 172.21.15.40:0/4244271758 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54d6d9d640 1 --1- 172.21.15.40:0/4244271758 >> v1:172.21.15.40:6789/0 conn(0x7f54d80d0d60 0x7f54d80cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34996/0 (socket says 172.21.15.40:34996) 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54cffff640 1 --2- 172.21.15.40:0/4244271758 >> v2:172.21.15.184:3300/0 conn(0x7f54d80b4360 0x7f54d80cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54d659c640 1 -- 172.21.15.40:0/4244271758 >> v2:172.21.15.184:3300/0 conn(0x7f54d80b4360 msgr2=0x7f54d80cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54d659c640 1 --2- 172.21.15.40:0/4244271758 >> v2:172.21.15.184:3300/0 conn(0x7f54d80b4360 0x7f54d80cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54d659c640 1 -- 172.21.15.40:0/4244271758 >> v1:172.21.15.40:6789/0 conn(0x7f54d80d0d60 legacy=0x7f54d80cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.765+0000 7f54d659c640 1 -- 172.21.15.40:0/4244271758 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d80ce8c0 con 0x7f54d80d1240 2024-04-26T21:36:07.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.766+0000 7f54d659c640 1 --2- 172.21.15.40:0/4244271758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d80d1620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f54d80b8f10 tx=0x7f54c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a2aeceec7acaa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:07.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.766+0000 7f54d5d9b640 1 -- 172.21.15.40:0/4244271758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8037070 con 0x7f54d80d1240 2024-04-26T21:36:07.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.766+0000 7f54d5d9b640 1 -- 172.21.15.40:0/4244271758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c8032440 con 0x7f54d80d1240 2024-04-26T21:36:07.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.766+0000 7f54d5d9b640 1 -- 172.21.15.40:0/4244271758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8032810 con 0x7f54d80d1240 2024-04-26T21:36:07.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.767+0000 7f54dc845640 1 -- 172.21.15.40:0/4244271758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 msgr2=0x7f54d80d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.767+0000 7f54dc845640 1 --2- 172.21.15.40:0/4244271758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d80d1620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f54d80b8f10 tx=0x7f54c802a270 comp rx=0 tx=0).stop 2024-04-26T21:36:07.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.767+0000 7f54dc845640 1 -- 172.21.15.40:0/4244271758 shutdown_connections 2024-04-26T21:36:07.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.767+0000 7f54dc845640 1 --2- 172.21.15.40:0/4244271758 >> v2:172.21.15.184:3300/0 conn(0x7f54d80b4360 0x7f54d80cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.767+0000 7f54dc845640 1 --2- 172.21.15.40:0/4244271758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d80d1620 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.767+0000 7f54dc845640 1 -- 172.21.15.40:0/4244271758 >> 172.21.15.40:0/4244271758 conn(0x7f54d80d2e60 msgr2=0x7f54d80d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:07.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.767+0000 7f54dc845640 1 -- 172.21.15.40:0/4244271758 shutdown_connections 2024-04-26T21:36:07.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.767+0000 7f54dc845640 1 -- 172.21.15.40:0/4244271758 wait complete. 2024-04-26T21:36:07.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.768+0000 7f54dc845640 1 Processor -- start 2024-04-26T21:36:07.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.768+0000 7f54dc845640 1 -- start start 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54dc845640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d80d0d60 0x7f54d81f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54dc845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54dc845640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54d80cb250 0x7f54d8153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54dc845640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f54d80c6610 con 0x7f54d80d1240 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54dc845640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54d80cf460 con 0x7f54d80d0d60 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54dc845640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f54d80c6790 con 0x7f54d80cb250 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d806d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45830/0 (socket says 172.21.15.40:45830) 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54cffff640 1 -- 172.21.15.40:0/1927651845 learned_addr learned my addr 172.21.15.40:0/1927651845 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54cffff640 1 -- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54d80cb250 msgr2=0x7f54d8153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54cffff640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54d80cb250 0x7f54d8153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54cffff640 1 -- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d80d0d60 msgr2=0x7f54d81f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54cffff640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d80d0d60 0x7f54d81f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.769+0000 7f54cffff640 1 -- 172.21.15.40:0/1927651845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d806d890 con 0x7f54d80d1240 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.770+0000 7f54cffff640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d806d350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f54c000d800 tx=0x7f54c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.770+0000 7f54ceffd640 1 -- 172.21.15.40:0/1927651845 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c000b6c0 con 0x7f54d80d1240 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.770+0000 7f54ceffd640 1 -- 172.21.15.40:0/1927651845 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c0019070 con 0x7f54d80d1240 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.770+0000 7f54ceffd640 1 -- 172.21.15.40:0/1927651845 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c000ba80 con 0x7f54d80d1240 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.770+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d806db30 con 0x7f54d80d1240 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.770+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d806e0f0 con 0x7f54d80d1240 2024-04-26T21:36:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.771+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d80694f0 con 0x7f54d80d1240 2024-04-26T21:36:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.771+0000 7f54ceffd640 1 -- 172.21.15.40:0/1927651845 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54c000bc20 con 0x7f54d80d1240 2024-04-26T21:36:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.772+0000 7f54ceffd640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54b806c8f0 0x7f54b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.772+0000 7f54ceffd640 1 -- 172.21.15.40:0/1927651845 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f54c008c370 con 0x7f54d80d1240 2024-04-26T21:36:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.772+0000 7f54d659c640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54b806c8f0 0x7f54b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.773+0000 7f54d659c640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54b806c8f0 0x7f54b806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54c8034000 tx=0x7f54c802c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:07.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.776+0000 7f54ceffd640 1 -- 172.21.15.40:0/1927651845 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54c0011340 con 0x7f54d80d1240 2024-04-26T21:36:07.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.963+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54d8070840 con 0x7f54d80d1240 2024-04-26T21:36:07.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.963+0000 7f54ceffd640 1 -- 172.21.15.40:0/1927651845 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f54c005a430 con 0x7f54d80d1240 2024-04-26T21:36:07.963 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54b806c8f0 msgr2=0x7f54b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54b806c8f0 0x7f54b806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54c8034000 tx=0x7f54c802c000 comp rx=0 tx=0).stop 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 msgr2=0x7f54d806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d806d350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f54c000d800 tx=0x7f54c000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 shutdown_connections 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54b806c8f0 0x7f54b806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54d80cb250 0x7f54d8153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d80d1240 0x7f54d806d350 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 --2- 172.21.15.40:0/1927651845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d80d0d60 0x7f54d81f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:07.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 >> 172.21.15.40:0/1927651845 conn(0x7f54d80d2e60 msgr2=0x7f54d815e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:07.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 shutdown_connections 2024-04-26T21:36:07.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:07.965+0000 7f54dc845640 1 -- 172.21.15.40:0/1927651845 wait complete. 2024-04-26T21:36:07.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:36:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.080+0000 7fdf60c6d640 1 Processor -- start 2024-04-26T21:36:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.080+0000 7fdf60c6d640 1 -- start start 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.081+0000 7fdf60c6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf5c0d1240 0x7fdf5c0d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.081+0000 7fdf60c6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0b3610 0x7fdf5c0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.081+0000 7fdf60c6d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf5c0d1b60 con 0x7fdf5c0b4360 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.081+0000 7fdf60c6d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdf5c0cf210 con 0x7fdf5c0d1240 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.081+0000 7fdf60c6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c0ce5e0 con 0x7fdf5c0b3610 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.081+0000 7fdf5a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf5c0d1240 0x7fdf5c0d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.082+0000 7fdf5a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf5c0d1240 0x7fdf5c0d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45836/0 (socket says 172.21.15.40:45836) 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.082+0000 7fdf5a59c640 1 -- 172.21.15.40:0/3156833739 learned_addr learned my addr 172.21.15.40:0/3156833739 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.082+0000 7fdf5ad9d640 1 --2- 172.21.15.40:0/3156833739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0b3610 0x7fdf5c0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.082+0000 7fdf5a59c640 1 -- 172.21.15.40:0/3156833739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0b3610 msgr2=0x7fdf5c0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.082+0000 7fdf5a59c640 1 --2- 172.21.15.40:0/3156833739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0b3610 0x7fdf5c0cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.082+0000 7fdf5a59c640 1 -- 172.21.15.40:0/3156833739 >> v1:172.21.15.184:6789/0 conn(0x7fdf5c0b4360 legacy=0x7fdf5c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.082+0000 7fdf5a59c640 1 -- 172.21.15.40:0/3156833739 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c0ce8c0 con 0x7fdf5c0d1240 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.082+0000 7fdf5a59c640 1 --2- 172.21.15.40:0/3156833739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0d1620 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009870 tx=0x7fdf5402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ebc7f01d7a52c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf5959a640 1 -- 172.21.15.40:0/3156833739 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54037070 con 0x7fdf5c0d1240 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf5959a640 1 -- 172.21.15.40:0/3156833739 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf54032440 con 0x7fdf5c0d1240 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf5959a640 1 -- 172.21.15.40:0/3156833739 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54032810 con 0x7fdf5c0d1240 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf60c6d640 1 -- 172.21.15.40:0/3156833739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 msgr2=0x7fdf5c0d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/3156833739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0d1620 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009870 tx=0x7fdf5402a210 comp rx=0 tx=0).stop 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf60c6d640 1 -- 172.21.15.40:0/3156833739 shutdown_connections 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/3156833739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0b3610 0x7fdf5c0cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/3156833739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0d1620 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf60c6d640 1 -- 172.21.15.40:0/3156833739 >> 172.21.15.40:0/3156833739 conn(0x7fdf5c0d2e60 msgr2=0x7fdf5c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf60c6d640 1 -- 172.21.15.40:0/3156833739 shutdown_connections 2024-04-26T21:36:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.083+0000 7fdf60c6d640 1 -- 172.21.15.40:0/3156833739 wait complete. 2024-04-26T21:36:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf60c6d640 1 Processor -- start 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf60c6d640 1 -- start start 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf60c6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5c0b3610 0x7fdf5c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf60c6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf60c6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0cae60 0x7fdf5c150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf60c6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c0c6610 con 0x7fdf5c0d1240 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf60c6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c0cf460 con 0x7fdf5c0b3610 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf60c6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c0c6790 con 0x7fdf5c0cae60 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.085+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45850/0 (socket says 172.21.15.40:45850) 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf59d9b640 1 -- 172.21.15.40:0/2494490991 learned_addr learned my addr 172.21.15.40:0/2494490991 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf5a59c640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5c0b3610 0x7fdf5c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf5ad9d640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0cae60 0x7fdf5c150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf59d9b640 1 -- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0cae60 msgr2=0x7fdf5c150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf59d9b640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0cae60 0x7fdf5c150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf59d9b640 1 -- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5c0b3610 msgr2=0x7fdf5c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf59d9b640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5c0b3610 0x7fdf5c0b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf59d9b640 1 -- 172.21.15.40:0/2494490991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c0cc9e0 con 0x7fdf5c0d1240 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf5ad9d640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0cae60 0x7fdf5c150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf59d9b640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0ca920 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c460 tx=0x7fdf4400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf52ffd640 1 -- 172.21.15.40:0/2494490991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44016020 con 0x7fdf5c0d1240 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf52ffd640 1 -- 172.21.15.40:0/2494490991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf44012870 con 0x7fdf5c0d1240 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c0cccc0 con 0x7fdf5c0d1240 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf52ffd640 1 -- 172.21.15.40:0/2494490991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44012c60 con 0x7fdf5c0d1240 2024-04-26T21:36:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.086+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c0b13b0 con 0x7fdf5c0d1240 2024-04-26T21:36:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.087+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c0694f0 con 0x7fdf5c0d1240 2024-04-26T21:36:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.088+0000 7fdf52ffd640 1 -- 172.21.15.40:0/2494490991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf44019020 con 0x7fdf5c0d1240 2024-04-26T21:36:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.088+0000 7fdf52ffd640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf3c06c8f0 0x7fdf3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.088+0000 7fdf52ffd640 1 -- 172.21.15.40:0/2494490991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdf4408bd70 con 0x7fdf5c0d1240 2024-04-26T21:36:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.089+0000 7fdf5a59c640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf3c06c8f0 0x7fdf3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.089+0000 7fdf5a59c640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf3c06c8f0 0x7fdf3c06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdf54034000 tx=0x7fdf5402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.092+0000 7fdf52ffd640 1 -- 172.21.15.40:0/2494490991 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf44020030 con 0x7fdf5c0d1240 2024-04-26T21:36:08.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.277+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdf5c0c8360 con 0x7fdf5c0d1240 2024-04-26T21:36:08.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.277+0000 7fdf52ffd640 1 -- 172.21.15.40:0/2494490991 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fdf44019310 con 0x7fdf5c0d1240 2024-04-26T21:36:08.278 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:08.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf3c06c8f0 msgr2=0x7fdf3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf3c06c8f0 0x7fdf3c06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdf54034000 tx=0x7fdf5402e000 comp rx=0 tx=0).stop 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 msgr2=0x7fdf5c0ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0ca920 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c460 tx=0x7fdf4400c920 comp rx=0 tx=0).stop 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 shutdown_connections 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf3c06c8f0 0x7fdf3c06edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf5c0cae60 0x7fdf5c150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf5c0d1240 0x7fdf5c0ca920 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 --2- 172.21.15.40:0/2494490991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5c0b3610 0x7fdf5c0b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 >> 172.21.15.40:0/2494490991 conn(0x7fdf5c0d2e60 msgr2=0x7fdf5c0cd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 shutdown_connections 2024-04-26T21:36:08.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.279+0000 7fdf60c6d640 1 -- 172.21.15.40:0/2494490991 wait complete. 2024-04-26T21:36:08.291 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:36:08.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.395+0000 7f4591c09640 1 Processor -- start 2024-04-26T21:36:08.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.395+0000 7f4591c09640 1 -- start start 2024-04-26T21:36:08.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f4591c09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f458c070350 0x7f458c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f4591c09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c0d2390 con 0x7f458c071090 2024-04-26T21:36:08.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f4591c09640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c06d3f0 con 0x7f458c0cfde0 2024-04-26T21:36:08.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f4591c09640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f458c0afab0 con 0x7f458c070350 2024-04-26T21:36:08.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f458affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f458c070350 0x7f458c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f458bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f458c071090 0x7f458c15e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35012/0 (socket says 172.21.15.40:35012) 2024-04-26T21:36:08.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f458bfff640 1 -- 172.21.15.40:0/3006062470 learned_addr learned my addr 172.21.15.40:0/3006062470 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:08.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f458affd640 1 -- 172.21.15.40:0/3006062470 >> v1:172.21.15.184:6789/0 conn(0x7f458c0cfde0 legacy=0x7f458c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:08.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f458affd640 1 -- 172.21.15.40:0/3006062470 >> v1:172.21.15.40:6789/0 conn(0x7f458c071090 legacy=0x7f458c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.396+0000 7f458affd640 1 -- 172.21.15.40:0/3006062470 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c06d6d0 con 0x7f458c070350 2024-04-26T21:36:08.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.397+0000 7f458affd640 1 --2- 172.21.15.40:0/3006062470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c070350 0x7f458c070b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f45800097a0 tx=0x7f458002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb19eb26d8d6c1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.397+0000 7f458a7fc640 1 -- 172.21.15.40:0/3006062470 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580037070 con 0x7f458c070350 2024-04-26T21:36:08.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.397+0000 7f458a7fc640 1 -- 172.21.15.40:0/3006062470 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458002ae50 con 0x7f458c070350 2024-04-26T21:36:08.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.397+0000 7f458a7fc640 1 -- 172.21.15.40:0/3006062470 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580032660 con 0x7f458c070350 2024-04-26T21:36:08.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.398+0000 7f4591c09640 1 -- 172.21.15.40:0/3006062470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c070350 msgr2=0x7f458c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.398+0000 7f4591c09640 1 --2- 172.21.15.40:0/3006062470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c070350 0x7f458c070b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f45800097a0 tx=0x7f458002a140 comp rx=0 tx=0).stop 2024-04-26T21:36:08.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.398+0000 7f4591c09640 1 -- 172.21.15.40:0/3006062470 shutdown_connections 2024-04-26T21:36:08.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.398+0000 7f4591c09640 1 --2- 172.21.15.40:0/3006062470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c070350 0x7f458c070b50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.398+0000 7f4591c09640 1 -- 172.21.15.40:0/3006062470 >> 172.21.15.40:0/3006062470 conn(0x7f458c0d2d80 msgr2=0x7f458c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:08.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.398+0000 7f4591c09640 1 -- 172.21.15.40:0/3006062470 shutdown_connections 2024-04-26T21:36:08.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.398+0000 7f4591c09640 1 -- 172.21.15.40:0/3006062470 wait complete. 2024-04-26T21:36:08.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.399+0000 7f4591c09640 1 Processor -- start 2024-04-26T21:36:08.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.399+0000 7f4591c09640 1 -- start start 2024-04-26T21:36:08.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f4591c09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c070350 0x7f458c15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f4591c09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c071090 0x7f458c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f4591c09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c0cfde0 0x7f458c14b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f4591c09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0714c0 con 0x7f458c0cfde0 2024-04-26T21:36:08.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f4591c09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f458c06dd20 con 0x7f458c071090 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f4591c09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f458c06f1c0 con 0x7f458c070350 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c0cfde0 0x7f458c14b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c0cfde0 0x7f458c14b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45870/0 (socket says 172.21.15.40:45870) 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 -- 172.21.15.40:0/3969983382 learned_addr learned my addr 172.21.15.40:0/3969983382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458affd640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c071090 0x7f458c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458b7fe640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c070350 0x7f458c15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 -- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c070350 msgr2=0x7f458c15f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c070350 0x7f458c15f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 -- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c071090 msgr2=0x7f458c15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c071090 0x7f458c15c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 -- 172.21.15.40:0/3969983382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c15cdf0 con 0x7f458c0cfde0 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.400+0000 7f458bfff640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c0cfde0 0x7f458c14b110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f458400cbc0 tx=0x7f458400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.401+0000 7f4573fff640 1 -- 172.21.15.40:0/3969983382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584017070 con 0x7f458c0cfde0 2024-04-26T21:36:08.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.401+0000 7f4573fff640 1 -- 172.21.15.40:0/3969983382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458400ad40 con 0x7f458c0cfde0 2024-04-26T21:36:08.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.401+0000 7f4573fff640 1 -- 172.21.15.40:0/3969983382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584012600 con 0x7f458c0cfde0 2024-04-26T21:36:08.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.401+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c0c7e90 con 0x7f458c0cfde0 2024-04-26T21:36:08.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.401+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c0c8450 con 0x7f458c0cfde0 2024-04-26T21:36:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.402+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c0694f0 con 0x7f458c0cfde0 2024-04-26T21:36:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.402+0000 7f4573fff640 1 -- 172.21.15.40:0/3969983382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f458401a070 con 0x7f458c0cfde0 2024-04-26T21:36:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.403+0000 7f4573fff640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c920 0x7f457806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.403+0000 7f4573fff640 1 -- 172.21.15.40:0/3969983382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f458408bb70 con 0x7f458c0cfde0 2024-04-26T21:36:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.403+0000 7f458b7fe640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c920 0x7f457806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.403+0000 7f458b7fe640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c920 0x7f457806ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f457c006fd0 tx=0x7f457c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.407+0000 7f4573fff640 1 -- 172.21.15.40:0/3969983382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4584059550 con 0x7f458c0cfde0 2024-04-26T21:36:08.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.590+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f458c06ce60 con 0x7f458c0cfde0 2024-04-26T21:36:08.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.590+0000 7f4573fff640 1 -- 172.21.15.40:0/3969983382 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f458405d850 con 0x7f458c0cfde0 2024-04-26T21:36:08.590 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:36:08.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c920 msgr2=0x7f457806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c920 0x7f457806ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f457c006fd0 tx=0x7f457c008040 comp rx=0 tx=0).stop 2024-04-26T21:36:08.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c0cfde0 msgr2=0x7f458c14b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c0cfde0 0x7f458c14b110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f458400cbc0 tx=0x7f458400a430 comp rx=0 tx=0).stop 2024-04-26T21:36:08.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 shutdown_connections 2024-04-26T21:36:08.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c920 0x7f457806ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f458c0cfde0 0x7f458c14b110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458c071090 0x7f458c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 --2- 172.21.15.40:0/3969983382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458c070350 0x7f458c15f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 >> 172.21.15.40:0/3969983382 conn(0x7f458c0d2d80 msgr2=0x7f458c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:08.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 shutdown_connections 2024-04-26T21:36:08.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.592+0000 7f4591c09640 1 -- 172.21.15.40:0/3969983382 wait complete. 2024-04-26T21:36:08.603 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:36:08.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.706+0000 7f819ffe7640 1 Processor -- start 2024-04-26T21:36:08.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.706+0000 7f819ffe7640 1 -- start start 2024-04-26T21:36:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.707+0000 7f819ffe7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81980d2110 0x7f81980b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.707+0000 7f819ffe7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81980d0890 con 0x7f81981628f0 2024-04-26T21:36:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.707+0000 7f819ffe7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81980d0b70 con 0x7f8198162cd0 2024-04-26T21:36:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.707+0000 7f819ffe7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f81980d1130 con 0x7f81980d2110 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.707+0000 7f819dd83640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f81981628f0 0x7f81980d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50106/0 (socket says 172.21.15.40:50106) 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.707+0000 7f819dd83640 1 -- 172.21.15.40:0/269904789 learned_addr learned my addr 172.21.15.40:0/269904789 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819e584640 1 --2- 172.21.15.40:0/269904789 >> v2:172.21.15.184:3300/0 conn(0x7f81980d2110 0x7f81980b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819cd81640 1 -- 172.21.15.40:0/269904789 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545364263 0 0) 0x7f81980d0890 con 0x7f81981628f0 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819cd81640 1 -- 172.21.15.40:0/269904789 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8180003520 con 0x7f81981628f0 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819cd81640 1 -- 172.21.15.40:0/269904789 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358793488 0 0) 0x7f81980d0b70 con 0x7f8198162cd0 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819cd81640 1 -- 172.21.15.40:0/269904789 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81980d0890 con 0x7f8198162cd0 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819cd81640 1 -- 172.21.15.40:0/269904789 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1109935578 0 0) 0x7f8180003520 con 0x7f81981628f0 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819e584640 1 -- 172.21.15.40:0/269904789 >> v1:172.21.15.184:6789/0 conn(0x7f8198162cd0 legacy=0x7f81980b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819e584640 1 -- 172.21.15.40:0/269904789 >> v1:172.21.15.40:6789/0 conn(0x7f81981628f0 legacy=0x7f81980d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.708+0000 7f819e584640 1 -- 172.21.15.40:0/269904789 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81980d0e50 con 0x7f81980d2110 2024-04-26T21:36:08.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819e584640 1 --2- 172.21.15.40:0/269904789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 0x7f81980b5370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81880097a0 tx=0x7f818802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4654f5b6ce4cb8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819cd81640 1 -- 172.21.15.40:0/269904789 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188037070 con 0x7f81980d2110 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819cd81640 1 -- 172.21.15.40:0/269904789 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818802abc0 con 0x7f81980d2110 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819cd81640 1 -- 172.21.15.40:0/269904789 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900019f0 con 0x7f81980d2110 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819ffe7640 1 -- 172.21.15.40:0/269904789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 msgr2=0x7f81980b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819ffe7640 1 --2- 172.21.15.40:0/269904789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 0x7f81980b5370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81880097a0 tx=0x7f818802a130 comp rx=0 tx=0).stop 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819ffe7640 1 -- 172.21.15.40:0/269904789 shutdown_connections 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819ffe7640 1 --2- 172.21.15.40:0/269904789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 0x7f81980b5370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.709+0000 7f819ffe7640 1 -- 172.21.15.40:0/269904789 >> 172.21.15.40:0/269904789 conn(0x7f81980d4ff0 msgr2=0x7f81980d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.710+0000 7f819ffe7640 1 -- 172.21.15.40:0/269904789 shutdown_connections 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.710+0000 7f819ffe7640 1 -- 172.21.15.40:0/269904789 wait complete. 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.711+0000 7f819ffe7640 1 Processor -- start 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.711+0000 7f819ffe7640 1 -- start start 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.711+0000 7f819ffe7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 0x7f81980b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.711+0000 7f819ffe7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81981628f0 0x7f81980b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.711+0000 7f819ffe7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8198162cd0 0x7f81980cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.711+0000 7f819ffe7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81980d16d0 con 0x7f8198162cd0 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.711+0000 7f819ffe7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81980d1550 con 0x7f81980d2110 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.711+0000 7f819ffe7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81980b3f50 con 0x7f81981628f0 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8198162cd0 0x7f81980cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8198162cd0 0x7f81980cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58168/0 (socket says 172.21.15.40:58168) 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 -- 172.21.15.40:0/1166334483 learned_addr learned my addr 172.21.15.40:0/1166334483 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819dd83640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 0x7f81980b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819d582640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81981628f0 0x7f81980b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 -- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81981628f0 msgr2=0x7f81980b9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81981628f0 0x7f81980b9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 -- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 msgr2=0x7f81980b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 0x7f81980b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 -- 172.21.15.40:0/1166334483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81980b22f0 con 0x7f8198162cd0 2024-04-26T21:36:08.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819d582640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81981628f0 0x7f81980b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:08.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819e584640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8198162cd0 0x7f81980cef90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f81880097a0 tx=0x7f81880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f81967fc640 1 -- 172.21.15.40:0/1166334483 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188041020 con 0x7f8198162cd0 2024-04-26T21:36:08.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f81967fc640 1 -- 172.21.15.40:0/1166334483 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818803d620 con 0x7f8198162cd0 2024-04-26T21:36:08.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81980b25d0 con 0x7f8198162cd0 2024-04-26T21:36:08.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f81967fc640 1 -- 172.21.15.40:0/1166334483 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188037040 con 0x7f8198162cd0 2024-04-26T21:36:08.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.712+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81980b2b90 con 0x7f8198162cd0 2024-04-26T21:36:08.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.713+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81980694f0 con 0x7f8198162cd0 2024-04-26T21:36:08.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.714+0000 7f81967fc640 1 -- 172.21.15.40:0/1166334483 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8188044020 con 0x7f8198162cd0 2024-04-26T21:36:08.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.714+0000 7f81967fc640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f818406c8f0 0x7f818406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:08.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.714+0000 7f81967fc640 1 -- 172.21.15.40:0/1166334483 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f81880ac550 con 0x7f8198162cd0 2024-04-26T21:36:08.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.714+0000 7f819dd83640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f818406c8f0 0x7f818406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:08.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.715+0000 7f819dd83640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f818406c8f0 0x7f818406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8190001560 tx=0x7f8190009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:08.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.718+0000 7f81967fc640 1 -- 172.21.15.40:0/1166334483 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f818804b030 con 0x7f8198162cd0 2024-04-26T21:36:08.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.904+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f81980d0030 con 0x7f8198162cd0 2024-04-26T21:36:08.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.904+0000 7f81967fc640 1 -- 172.21.15.40:0/1166334483 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f818807a4f0 con 0x7f8198162cd0 2024-04-26T21:36:08.904 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:08.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f818406c8f0 msgr2=0x7f818406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f818406c8f0 0x7f818406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8190001560 tx=0x7f8190009040 comp rx=0 tx=0).stop 2024-04-26T21:36:08.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8198162cd0 msgr2=0x7f81980cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:08.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8198162cd0 0x7f81980cef90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f81880097a0 tx=0x7f81880046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 shutdown_connections 2024-04-26T21:36:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f818406c8f0 0x7f818406edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8198162cd0 0x7f81980cef90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81981628f0 0x7f81980b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 --2- 172.21.15.40:0/1166334483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81980d2110 0x7f81980b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 >> 172.21.15.40:0/1166334483 conn(0x7f81980d4ff0 msgr2=0x7f81980b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 shutdown_connections 2024-04-26T21:36:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:08.906+0000 7f819ffe7640 1 -- 172.21.15.40:0/1166334483 wait complete. 2024-04-26T21:36:08.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:36:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.019+0000 7fcda0f24640 1 Processor -- start 2024-04-26T21:36:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.019+0000 7fcda0f24640 1 -- start start 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcda0f24640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcd9c0d3bc0 0x7fcd9c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcda0f24640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9c0b7e50 con 0x7fcd9c0b5f20 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcda0f24640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9c0d1800 con 0x7fcd9c162880 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcda0f24640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c0d0580 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcd9ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcd9c0b5f20 0x7fcd9c0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50114/0 (socket says 172.21.15.40:50114) 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcd9ad9d640 1 -- 172.21.15.40:0/2321078226 learned_addr learned my addr 172.21.15.40:0/2321078226 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcd93fff640 1 --2- 172.21.15.40:0/2321078226 >> v2:172.21.15.184:3300/0 conn(0x7fcd9c0d3bc0 0x7fcd9c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcd99d9b640 1 -- 172.21.15.40:0/2321078226 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 18185164 0 0) 0x7fcd9c0b7e50 con 0x7fcd9c0b5f20 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.020+0000 7fcd99d9b640 1 -- 172.21.15.40:0/2321078226 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd88003520 con 0x7fcd9c0b5f20 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.021+0000 7fcd99d9b640 1 -- 172.21.15.40:0/2321078226 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1557957086 0 0) 0x7fcd88003520 con 0x7fcd9c0b5f20 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.021+0000 7fcd93fff640 1 -- 172.21.15.40:0/2321078226 >> v1:172.21.15.184:6789/0 conn(0x7fcd9c162880 legacy=0x7fcd9c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.021+0000 7fcd93fff640 1 -- 172.21.15.40:0/2321078226 >> v1:172.21.15.40:6789/0 conn(0x7fcd9c0b5f20 legacy=0x7fcd9c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.021+0000 7fcd93fff640 1 -- 172.21.15.40:0/2321078226 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c0d07d0 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.021+0000 7fcd93fff640 1 --2- 172.21.15.40:0/2321078226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c0d2cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd840097a0 tx=0x7fcd8402a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390e77cb657c8126 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcd99d9b640 1 -- 172.21.15.40:0/2321078226 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84037070 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcd99d9b640 1 -- 172.21.15.40:0/2321078226 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8402ab60 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcd99d9b640 1 -- 172.21.15.40:0/2321078226 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd940019f0 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcda0f24640 1 -- 172.21.15.40:0/2321078226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0d3bc0 msgr2=0x7fcd9c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcda0f24640 1 --2- 172.21.15.40:0/2321078226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c0d2cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd840097a0 tx=0x7fcd8402a0d0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcda0f24640 1 -- 172.21.15.40:0/2321078226 shutdown_connections 2024-04-26T21:36:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcda0f24640 1 --2- 172.21.15.40:0/2321078226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c0d2cb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcda0f24640 1 -- 172.21.15.40:0/2321078226 >> 172.21.15.40:0/2321078226 conn(0x7fcd9c0d4f60 msgr2=0x7fcd9c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:09.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcda0f24640 1 -- 172.21.15.40:0/2321078226 shutdown_connections 2024-04-26T21:36:09.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.022+0000 7fcda0f24640 1 -- 172.21.15.40:0/2321078226 wait complete. 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.023+0000 7fcda0f24640 1 Processor -- start 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.023+0000 7fcda0f24640 1 -- start start 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcda0f24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0b5f20 0x7fcd9c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcda0f24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c06e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcda0f24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c162880 0x7fcd9c06e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcda0f24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c162c60 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcda0f24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c0d0d90 con 0x7fcd9c162880 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcda0f24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c162de0 con 0x7fcd9c0b5f20 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcd93fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c06e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcd93fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c06e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58178/0 (socket says 172.21.15.40:58178) 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcd93fff640 1 -- 172.21.15.40:0/3810032990 learned_addr learned my addr 172.21.15.40:0/3810032990 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0b5f20 0x7fcd9c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.024+0000 7fcd9ad9d640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c162880 0x7fcd9c06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd93fff640 1 -- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0b5f20 msgr2=0x7fcd9c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd93fff640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0b5f20 0x7fcd9c15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd93fff640 1 -- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c162880 msgr2=0x7fcd9c06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd93fff640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c162880 0x7fcd9c06e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd93fff640 1 -- 172.21.15.40:0/3810032990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c15cbe0 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0b5f20 0x7fcd9c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd93fff640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c06e030 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcd84035040 tx=0x7fcd840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd92ffd640 1 -- 172.21.15.40:0/3810032990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84037040 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd92ffd640 1 -- 172.21.15.40:0/3810032990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd84004030 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcd92ffd640 1 -- 172.21.15.40:0/3810032990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84004420 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c0b59e0 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.025+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c0b4f00 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.026+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c0694f0 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.026+0000 7fcd92ffd640 1 -- 172.21.15.40:0/3810032990 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd8403a040 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.027+0000 7fcd92ffd640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.027+0000 7fcd92ffd640 1 -- 172.21.15.40:0/3810032990 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcd840ac320 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.027+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.028+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c001a30 tx=0x7fcd8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.031+0000 7fcd92ffd640 1 -- 172.21.15.40:0/3810032990 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd84042020 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.228+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcd9c0d2de0 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.228+0000 7fcd92ffd640 1 -- 172.21.15.40:0/3810032990 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fcd8407a500 con 0x7fcd9c0d3bc0 2024-04-26T21:36:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd7c06c8f0 msgr2=0x7fcd7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c001a30 tx=0x7fcd8c009040 comp rx=0 tx=0).stop 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c0d3bc0 msgr2=0x7fcd9c06e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c06e030 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcd84035040 tx=0x7fcd840046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 shutdown_connections 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c162880 0x7fcd9c06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c0d3bc0 0x7fcd9c06e030 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 --2- 172.21.15.40:0/3810032990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0b5f20 0x7fcd9c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 >> 172.21.15.40:0/3810032990 conn(0x7fcd9c0d4f60 msgr2=0x7fcd9c0d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 shutdown_connections 2024-04-26T21:36:09.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.230+0000 7fcda0f24640 1 -- 172.21.15.40:0/3810032990 wait complete. 2024-04-26T21:36:09.241 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T21:36:09.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:09.241 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T21:36:09.272 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:36:09.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:09.272 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T21:36:09.331 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:36:09.331 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:09.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.432+0000 7f75e3f14640 1 Processor -- start 2024-04-26T21:36:09.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.432+0000 7f75e3f14640 1 -- start start 2024-04-26T21:36:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e3f14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75dc0b2340 0x7f75dc0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e3f14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e3f14640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75dc0d10d0 con 0x7f75dc162810 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e3f14640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f75dc0d0750 con 0x7f75dc0b2340 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e3f14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc0d16a0 con 0x7f75dc0cd930 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e14af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75dc0b2340 0x7f75dc0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e14af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75dc0b2340 0x7f75dc0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58188/0 (socket says 172.21.15.40:58188) 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e14af640 1 -- 172.21.15.40:0/2160038120 learned_addr learned my addr 172.21.15.40:0/2160038120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e24b1640 1 --2- 172.21.15.40:0/2160038120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e14af640 1 -- 172.21.15.40:0/2160038120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 msgr2=0x7f75dc0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e14af640 1 --2- 172.21.15.40:0/2160038120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e14af640 1 -- 172.21.15.40:0/2160038120 >> v1:172.21.15.184:6789/0 conn(0x7f75dc162810 legacy=0x7f75dc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.433+0000 7f75e14af640 1 -- 172.21.15.40:0/2160038120 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc0d1370 con 0x7f75dc0b2340 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.434+0000 7f75e14af640 1 --2- 172.21.15.40:0/2160038120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cd3f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0d1820 tx=0x7f75d002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcb065000ec29ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.434+0000 7f75e0cae640 1 -- 172.21.15.40:0/2160038120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0037070 con 0x7f75dc0b2340 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.434+0000 7f75e0cae640 1 -- 172.21.15.40:0/2160038120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d0032440 con 0x7f75dc0b2340 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.434+0000 7f75e0cae640 1 -- 172.21.15.40:0/2160038120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0032810 con 0x7f75dc0b2340 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.435+0000 7f75e3f14640 1 -- 172.21.15.40:0/2160038120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 msgr2=0x7f75dc0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.435+0000 7f75e3f14640 1 --2- 172.21.15.40:0/2160038120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cd3f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0d1820 tx=0x7f75d002a250 comp rx=0 tx=0).stop 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.435+0000 7f75e3f14640 1 -- 172.21.15.40:0/2160038120 shutdown_connections 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.435+0000 7f75e3f14640 1 --2- 172.21.15.40:0/2160038120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.435+0000 7f75e3f14640 1 --2- 172.21.15.40:0/2160038120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cd3f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.435+0000 7f75e3f14640 1 -- 172.21.15.40:0/2160038120 >> 172.21.15.40:0/2160038120 conn(0x7f75dc0d2da0 msgr2=0x7f75dc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.435+0000 7f75e3f14640 1 -- 172.21.15.40:0/2160038120 shutdown_connections 2024-04-26T21:36:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.435+0000 7f75e3f14640 1 -- 172.21.15.40:0/2160038120 wait complete. 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.436+0000 7f75e3f14640 1 Processor -- start 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.436+0000 7f75e3f14640 1 -- start start 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e3f14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e3f14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc14c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e3f14640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75dc162810 0x7f75dc0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e3f14640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc0d2070 con 0x7f75dc0b2340 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e3f14640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc0d0470 con 0x7f75dc162810 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e3f14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc0d21f0 con 0x7f75dc0cd930 2024-04-26T21:36:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58194/0 (socket says 172.21.15.40:58194) 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 -- 172.21.15.40:0/750197383 learned_addr learned my addr 172.21.15.40:0/750197383 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e24b1640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75dc162810 0x7f75dc0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e14af640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc14c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 -- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 msgr2=0x7f75dc14c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc14c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 -- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75dc162810 msgr2=0x7f75dc0cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75dc162810 0x7f75dc0cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 -- 172.21.15.40:0/750197383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc14cef0 con 0x7f75dc0b2340 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e24b1640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75dc162810 0x7f75dc0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e14af640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75e1cb0640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cc880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f75d400c3c0 tx=0x7f75d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.437+0000 7f75da7fc640 1 -- 172.21.15.40:0/750197383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d40052d0 con 0x7f75dc0b2340 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.438+0000 7f75da7fc640 1 -- 172.21.15.40:0/750197383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d4005490 con 0x7f75dc0b2340 2024-04-26T21:36:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.438+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc14d1d0 con 0x7f75dc0b2340 2024-04-26T21:36:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.438+0000 7f75da7fc640 1 -- 172.21.15.40:0/750197383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4007260 con 0x7f75dc0b2340 2024-04-26T21:36:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.438+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc14d790 con 0x7f75dc0b2340 2024-04-26T21:36:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.439+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc0694f0 con 0x7f75dc0b2340 2024-04-26T21:36:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.439+0000 7f75da7fc640 1 -- 172.21.15.40:0/750197383 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75d4007400 con 0x7f75dc0b2340 2024-04-26T21:36:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.439+0000 7f75da7fc640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.439+0000 7f75da7fc640 1 -- 172.21.15.40:0/750197383 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f75d408b3f0 con 0x7f75dc0b2340 2024-04-26T21:36:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.440+0000 7f75e14af640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.440+0000 7f75e14af640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f75d0034000 tx=0x7f75d002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.444+0000 7f75da7fc640 1 -- 172.21.15.40:0/750197383 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75d4059420 con 0x7f75dc0b2340 2024-04-26T21:36:09.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.668+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75dc06ce60 con 0x7f75dc0b2340 2024-04-26T21:36:09.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.668+0000 7f75da7fc640 1 -- 172.21.15.40:0/750197383 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f75d4058d20 con 0x7f75dc0b2340 2024-04-26T21:36:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:09.669 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:31.566271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:36:09.669 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-26T21:36:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 msgr2=0x7f75c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f75d0034000 tx=0x7f75d002e000 comp rx=0 tx=0).stop 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 msgr2=0x7f75dc0cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cc880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f75d400c3c0 tx=0x7f75d400c880 comp rx=0 tx=0).stop 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 shutdown_connections 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75dc162810 0x7f75dc0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75dc0cd930 0x7f75dc14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 --2- 172.21.15.40:0/750197383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75dc0b2340 0x7f75dc0cc880 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 >> 172.21.15.40:0/750197383 conn(0x7f75dc0d2da0 msgr2=0x7f75dc070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 shutdown_connections 2024-04-26T21:36:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.670+0000 7f75e3f14640 1 -- 172.21.15.40:0/750197383 wait complete. 2024-04-26T21:36:09.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.782+0000 7f14b98c4640 1 Processor -- start 2024-04-26T21:36:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.782+0000 7f14b98c4640 1 -- start start 2024-04-26T21:36:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.782+0000 7f14b98c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14b40d3bc0 0x7f14b40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b98c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b40b7e50 con 0x7f14b40b5f20 2024-04-26T21:36:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b98c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b40d1800 con 0x7f14b4162880 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b98c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f14b40d0580 con 0x7f14b40d3bc0 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f14b40b5f20 0x7f14b40b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50126/0 (socket says 172.21.15.40:50126) 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b37fe640 1 -- 172.21.15.40:0/839727389 learned_addr learned my addr 172.21.15.40:0/839727389 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b27fc640 1 --2- 172.21.15.40:0/839727389 >> v2:172.21.15.184:3300/0 conn(0x7f14b40d3bc0 0x7f14b40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733954610 0 0) 0x7f14b40b7e50 con 0x7f14b40b5f20 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f149c003520 con 0x7f14b40b5f20 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1385441334 0 0) 0x7f149c003520 con 0x7f14b40b5f20 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 >> v2:172.21.15.184:3300/0 conn(0x7f14b40d3bc0 msgr2=0x7f14b40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 --2- 172.21.15.40:0/839727389 >> v2:172.21.15.184:3300/0 conn(0x7f14b40d3bc0 0x7f14b40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 >> v1:172.21.15.184:6789/0 conn(0x7f14b4162880 legacy=0x7f14b40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b40d07d0 con 0x7f14b40b5f20 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f14ac0019f0 con 0x7f14b40b5f20 2024-04-26T21:36:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 >> v1:172.21.15.40:6789/0 conn(0x7f14b40b5f20 legacy=0x7f14b40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f149c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f149c004800 0x7f149c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149c0051c0 0x7f149c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14b40d0580 con 0x7f149c0039b0 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f149c00c060 con 0x7f149c0051c0 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f149c00c260 con 0x7f149c004800 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.783+0000 7f14b2ffd640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f149c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b37fe640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149c0051c0 0x7f149c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b27fc640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f149c004800 0x7f149c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b2ffd640 1 -- 172.21.15.40:0/839727389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f149c004800 msgr2=0x7f149c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b2ffd640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f149c004800 0x7f149c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b2ffd640 1 -- 172.21.15.40:0/839727389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149c0051c0 msgr2=0x7f149c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b2ffd640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149c0051c0 0x7f149c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b37fe640 1 -- 172.21.15.40:0/839727389 reap_dead start 2024-04-26T21:36:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b2ffd640 1 -- 172.21.15.40:0/839727389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c003520 con 0x7f149c0039b0 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.784+0000 7f14b2ffd640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f149c003dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f14a4001750 tx=0x7f149c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940c30b40874936f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4001370 con 0x7f149c0039b0 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a40052d0 con 0x7f149c0039b0 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b1ffb640 1 -- 172.21.15.40:0/839727389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a402abc0 con 0x7f149c0039b0 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b98c4640 1 -- 172.21.15.40:0/839727389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 msgr2=0x7f149c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b98c4640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f149c003dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f14a4001750 tx=0x7f149c005830 comp rx=0 tx=0).stop 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b98c4640 1 -- 172.21.15.40:0/839727389 shutdown_connections 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b98c4640 1 --2- 172.21.15.40:0/839727389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f149c003dd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b98c4640 1 -- 172.21.15.40:0/839727389 >> 172.21.15.40:0/839727389 conn(0x7f14b40d4f60 msgr2=0x7f14b40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b98c4640 1 -- 172.21.15.40:0/839727389 shutdown_connections 2024-04-26T21:36:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.785+0000 7f14b98c4640 1 -- 172.21.15.40:0/839727389 wait complete. 2024-04-26T21:36:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.786+0000 7f14b98c4640 1 Processor -- start 2024-04-26T21:36:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b98c4640 1 -- start start 2024-04-26T21:36:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b98c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f14b40b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b98c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b415c700 0x7f14b40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b98c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b406de90 0x7f14b406e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b98c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4162c60 con 0x7f149c0039b0 2024-04-26T21:36:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b98c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14b40d0d90 con 0x7f14b415c700 2024-04-26T21:36:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b98c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14b4162de0 con 0x7f14b406de90 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f14b40b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f14b40b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58224/0 (socket says 172.21.15.40:58224) 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b2ffd640 1 -- 172.21.15.40:0/3839071705 learned_addr learned my addr 172.21.15.40:0/3839071705 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.787+0000 7f14b27fc640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b415c700 0x7f14b40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b37fe640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b406de90 0x7f14b406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b2ffd640 1 -- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b406de90 msgr2=0x7f14b406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b2ffd640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b406de90 0x7f14b406e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b2ffd640 1 -- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b415c700 msgr2=0x7f14b40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b2ffd640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b415c700 0x7f14b40b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b2ffd640 1 -- 172.21.15.40:0/3839071705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b40b29b0 con 0x7f149c0039b0 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b37fe640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b406de90 0x7f14b406e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b2ffd640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f14b40b3a50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f14a4000c00 tx=0x7f14a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f149bfff640 1 -- 172.21.15.40:0/3839071705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4040030 con 0x7f149c0039b0 2024-04-26T21:36:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f149bfff640 1 -- 172.21.15.40:0/3839071705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a402be50 con 0x7f149c0039b0 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b40b2c90 con 0x7f149c0039b0 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f149bfff640 1 -- 172.21.15.40:0/3839071705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4001370 con 0x7f149c0039b0 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.788+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b40b3250 con 0x7f149c0039b0 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.789+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b40694f0 con 0x7f149c0039b0 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.790+0000 7f149bfff640 1 -- 172.21.15.40:0/3839071705 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14a4043070 con 0x7f149c0039b0 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.790+0000 7f149bfff640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14a006c8f0 0x7f14a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.790+0000 7f149bfff640 1 -- 172.21.15.40:0/3839071705 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f14a40ab780 con 0x7f149c0039b0 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.790+0000 7f14b27fc640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14a006c8f0 0x7f14a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.791+0000 7f14b27fc640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14a006c8f0 0x7f14a006edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14b40a2da0 tx=0x7f14a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.795+0000 7f149bfff640 1 -- 172.21.15.40:0/3839071705 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14a404b020 con 0x7f149c0039b0 2024-04-26T21:36:09.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.980+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14b40d36c0 con 0x7f149c0039b0 2024-04-26T21:36:09.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.981+0000 7f149bfff640 1 -- 172.21.15.40:0/3839071705 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7f14b40d1800 con 0x7f149c0039b0 2024-04-26T21:36:09.981 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:09.982 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:09.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14a006c8f0 msgr2=0x7f14a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14a006c8f0 0x7f14a006edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14b40a2da0 tx=0x7f14a8008040 comp rx=0 tx=0).stop 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 msgr2=0x7f14b40b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f14b40b3a50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f14a4000c00 tx=0x7f14a4001120 comp rx=0 tx=0).stop 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 shutdown_connections 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14a006c8f0 0x7f14a006edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b406de90 0x7f14b406e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b415c700 0x7f14b40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 --2- 172.21.15.40:0/3839071705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f149c0039b0 0x7f14b40b3a50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 >> 172.21.15.40:0/3839071705 conn(0x7f14b40d4f60 msgr2=0x7f14b40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 shutdown_connections 2024-04-26T21:36:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:09.983+0000 7f14b98c4640 1 -- 172.21.15.40:0/3839071705 wait complete. 2024-04-26T21:36:09.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:35:08.071299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:35:08.422512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:35:12.131050+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T21:36:09.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:36:10.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:36:10.040 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:36:10.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.100+0000 7f5085008640 1 Processor -- start 2024-04-26T21:36:10.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.100+0000 7f5085008640 1 -- start start 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5085008640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f50800b40d0 0x7f50801625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5085008640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5080162b30 0x7f50800b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5085008640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50800d0a10 con 0x7f50800d1a00 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5085008640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f50800d0cf0 con 0x7f50800b40d0 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5085008640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50800d0fd0 con 0x7f5080162b30 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f507f59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5080162b30 0x7f50800b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5077fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f50800b40d0 0x7f50801625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f507ed9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f50800d1a00 0x7f50800d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:40682/0 (socket says 172.21.15.40:40682) 2024-04-26T21:36:10.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f507ed9d640 1 -- 172.21.15.40:0/2411436735 learned_addr learned my addr 172.21.15.40:0/2411436735 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:10.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5077fff640 1 -- 172.21.15.40:0/2411436735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5080162b30 msgr2=0x7f50800b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:10.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5077fff640 1 --2- 172.21.15.40:0/2411436735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5080162b30 0x7f50800b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5077fff640 1 -- 172.21.15.40:0/2411436735 >> v1:172.21.15.184:6789/0 conn(0x7f50800d1a00 legacy=0x7f50800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f5077fff640 1 -- 172.21.15.40:0/2411436735 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5080162f80 con 0x7f50800b40d0 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.101+0000 7f507f59e640 1 --2- 172.21.15.40:0/2411436735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5080162b30 0x7f50800b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.102+0000 7f5077fff640 1 --2- 172.21.15.40:0/2411436735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50800b40d0 0x7f50801625f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50680097a0 tx=0x7f506802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37c55ffb1f2bec0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.102+0000 7f507e59c640 1 -- 172.21.15.40:0/2411436735 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068037070 con 0x7f50800b40d0 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.102+0000 7f507e59c640 1 -- 172.21.15.40:0/2411436735 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506802ae50 con 0x7f50800b40d0 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.102+0000 7f507e59c640 1 -- 172.21.15.40:0/2411436735 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068032660 con 0x7f50800b40d0 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.103+0000 7f5085008640 1 -- 172.21.15.40:0/2411436735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50800b40d0 msgr2=0x7f50801625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.103+0000 7f5085008640 1 --2- 172.21.15.40:0/2411436735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50800b40d0 0x7f50801625f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50680097a0 tx=0x7f506802a100 comp rx=0 tx=0).stop 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.103+0000 7f5085008640 1 -- 172.21.15.40:0/2411436735 shutdown_connections 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.103+0000 7f5085008640 1 --2- 172.21.15.40:0/2411436735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5080162b30 0x7f50800b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.103+0000 7f5085008640 1 --2- 172.21.15.40:0/2411436735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50800b40d0 0x7f50801625f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.103+0000 7f5085008640 1 -- 172.21.15.40:0/2411436735 >> 172.21.15.40:0/2411436735 conn(0x7f50800d4fd0 msgr2=0x7f50800d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.103+0000 7f5085008640 1 -- 172.21.15.40:0/2411436735 shutdown_connections 2024-04-26T21:36:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.103+0000 7f5085008640 1 -- 172.21.15.40:0/2411436735 wait complete. 2024-04-26T21:36:10.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.104+0000 7f5085008640 1 Processor -- start 2024-04-26T21:36:10.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.104+0000 7f5085008640 1 -- start start 2024-04-26T21:36:10.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5085008640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50800b40d0 0x7f508014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5085008640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50800d1a00 0x7f508014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5085008640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5080162b30 0x7f50800b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5085008640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50800d2670 con 0x7f50800d1a00 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5085008640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50800d24f0 con 0x7f5080162b30 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5085008640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50800b5040 con 0x7f50800b40d0 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5077fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50800d1a00 0x7f508014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5077fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50800d1a00 0x7f508014ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58230/0 (socket says 172.21.15.40:58230) 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5077fff640 1 -- 172.21.15.40:0/3846047472 learned_addr learned my addr 172.21.15.40:0/3846047472 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f507f59e640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5080162b30 0x7f50800b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f507ed9d640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50800b40d0 0x7f508014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5077fff640 1 -- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50800b40d0 msgr2=0x7f508014f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5077fff640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50800b40d0 0x7f508014f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5077fff640 1 -- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5080162b30 msgr2=0x7f50800b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5077fff640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5080162b30 0x7f50800b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.105+0000 7f5077fff640 1 -- 172.21.15.40:0/3846047472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508014fc10 con 0x7f50800d1a00 2024-04-26T21:36:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.106+0000 7f5077fff640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50800d1a00 0x7f508014ed50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f506802a140 tx=0x7f50680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.106+0000 7f50777fe640 1 -- 172.21.15.40:0/3846047472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068041070 con 0x7f50800d1a00 2024-04-26T21:36:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.106+0000 7f50777fe640 1 -- 172.21.15.40:0/3846047472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506803c670 con 0x7f50800d1a00 2024-04-26T21:36:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.106+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50801f6370 con 0x7f50800d1a00 2024-04-26T21:36:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.106+0000 7f50777fe640 1 -- 172.21.15.40:0/3846047472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068037040 con 0x7f50800d1a00 2024-04-26T21:36:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.106+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50800cac00 con 0x7f50800d1a00 2024-04-26T21:36:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.107+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50800ca750 con 0x7f50800d1a00 2024-04-26T21:36:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.107+0000 7f50777fe640 1 -- 172.21.15.40:0/3846047472 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5068044070 con 0x7f50800d1a00 2024-04-26T21:36:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.108+0000 7f50777fe640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f506006c8f0 0x7f506006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.108+0000 7f50777fe640 1 -- 172.21.15.40:0/3846047472 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f50680ac500 con 0x7f50800d1a00 2024-04-26T21:36:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.108+0000 7f507ed9d640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f506006c8f0 0x7f506006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.109+0000 7f507ed9d640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f506006c8f0 0x7f506006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5070007fd0 tx=0x7f5070007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.112+0000 7f50777fe640 1 -- 172.21.15.40:0/3846047472 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f506807a000 con 0x7f50800d1a00 2024-04-26T21:36:10.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.300+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50800694f0 con 0x7f50800d1a00 2024-04-26T21:36:10.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.300+0000 7f50777fe640 1 -- 172.21.15.40:0/3846047472 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f506807e300 con 0x7f50800d1a00 2024-04-26T21:36:10.301 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f506006c8f0 msgr2=0x7f506006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f506006c8f0 0x7f506006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5070007fd0 tx=0x7f5070007e80 comp rx=0 tx=0).stop 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50800d1a00 msgr2=0x7f508014ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50800d1a00 0x7f508014ed50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f506802a140 tx=0x7f50680046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 shutdown_connections 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f506006c8f0 0x7f506006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5080162b30 0x7f50800b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50800d1a00 0x7f508014ed50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 --2- 172.21.15.40:0/3846047472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50800b40d0 0x7f508014f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 >> 172.21.15.40:0/3846047472 conn(0x7f50800d4fd0 msgr2=0x7f50800d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:10.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.302+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 shutdown_connections 2024-04-26T21:36:10.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.303+0000 7f5085008640 1 -- 172.21.15.40:0/3846047472 wait complete. 2024-04-26T21:36:10.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:36:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.458+0000 7f67b522a640 1 Processor -- start 2024-04-26T21:36:10.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.459+0000 7f67b522a640 1 -- start start 2024-04-26T21:36:10.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.459+0000 7f67b522a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b00cf0e0 0x7f67b006f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.459+0000 7f67b522a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b015b9e0 con 0x7f67b00cf5e0 2024-04-26T21:36:10.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.459+0000 7f67b522a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b00d0740 con 0x7f67b015b600 2024-04-26T21:36:10.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.459+0000 7f67b522a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67b00cb9b0 con 0x7f67b00cf0e0 2024-04-26T21:36:10.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.459+0000 7f67ae59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f67b00cf5e0 0x7f67b00ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50136/0 (socket says 172.21.15.40:50136) 2024-04-26T21:36:10.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.459+0000 7f67ae59c640 1 -- 172.21.15.40:0/831181838 learned_addr learned my addr 172.21.15.40:0/831181838 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:10.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67af59e640 1 --2- 172.21.15.40:0/831181838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b00cf0e0 0x7f67b006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67add9b640 1 -- 172.21.15.40:0/831181838 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1057915592 0 0) 0x7f67b015b9e0 con 0x7f67b00cf5e0 2024-04-26T21:36:10.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67add9b640 1 -- 172.21.15.40:0/831181838 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6798003440 con 0x7f67b00cf5e0 2024-04-26T21:36:10.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67add9b640 1 -- 172.21.15.40:0/831181838 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1505909722 0 0) 0x7f67b00d0740 con 0x7f67b015b600 2024-04-26T21:36:10.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67add9b640 1 -- 172.21.15.40:0/831181838 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67b015b9e0 con 0x7f67b015b600 2024-04-26T21:36:10.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67add9b640 1 -- 172.21.15.40:0/831181838 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1939276717 0 0) 0x7f6798003440 con 0x7f67b00cf5e0 2024-04-26T21:36:10.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67af59e640 1 -- 172.21.15.40:0/831181838 >> v1:172.21.15.184:6789/0 conn(0x7f67b015b600 legacy=0x7f67b0071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67af59e640 1 -- 172.21.15.40:0/831181838 >> v1:172.21.15.40:6789/0 conn(0x7f67b00cf5e0 legacy=0x7f67b00ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.460+0000 7f67af59e640 1 -- 172.21.15.40:0/831181838 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b00cb360 con 0x7f67b00cf0e0 2024-04-26T21:36:10.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.461+0000 7f67af59e640 1 --2- 172.21.15.40:0/831181838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b00cf0e0 0x7f67b006f370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67a80097d0 tx=0x7f67a802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf46dc228d2a00d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.461+0000 7f67add9b640 1 -- 172.21.15.40:0/831181838 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8037070 con 0x7f67b00cf0e0 2024-04-26T21:36:10.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.461+0000 7f67add9b640 1 -- 172.21.15.40:0/831181838 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a802ac30 con 0x7f67b00cf0e0 2024-04-26T21:36:10.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.461+0000 7f67add9b640 1 -- 172.21.15.40:0/831181838 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4001670 con 0x7f67b00cf0e0 2024-04-26T21:36:10.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.462+0000 7f67b522a640 1 -- 172.21.15.40:0/831181838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b00cf0e0 msgr2=0x7f67b006f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.462+0000 7f67b522a640 1 --2- 172.21.15.40:0/831181838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b00cf0e0 0x7f67b006f370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67a80097d0 tx=0x7f67a802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.462+0000 7f67b522a640 1 -- 172.21.15.40:0/831181838 shutdown_connections 2024-04-26T21:36:10.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.462+0000 7f67b522a640 1 --2- 172.21.15.40:0/831181838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b00cf0e0 0x7f67b006f370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.462+0000 7f67b522a640 1 -- 172.21.15.40:0/831181838 >> 172.21.15.40:0/831181838 conn(0x7f67b00d0be0 msgr2=0x7f67b00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:10.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.462+0000 7f67b522a640 1 -- 172.21.15.40:0/831181838 shutdown_connections 2024-04-26T21:36:10.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.462+0000 7f67b522a640 1 -- 172.21.15.40:0/831181838 wait complete. 2024-04-26T21:36:10.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.463+0000 7f67b522a640 1 Processor -- start 2024-04-26T21:36:10.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.463+0000 7f67b522a640 1 -- start start 2024-04-26T21:36:10.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67b522a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00cf0e0 0x7f67b00b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67b522a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67b00cf5e0 0x7f67b014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67b522a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b015b600 0x7f67b01500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67b522a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f67b00d0470 con 0x7f67b00cf0e0 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67b522a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f67b00d02f0 con 0x7f67b00cf5e0 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67b522a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67b00cbbd0 con 0x7f67b015b600 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67aed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00cf0e0 0x7f67b00b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67aed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00cf0e0 0x7f67b00b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58244/0 (socket says 172.21.15.40:58244) 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67aed9d640 1 -- 172.21.15.40:0/2133394597 learned_addr learned my addr 172.21.15.40:0/2133394597 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67af59e640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b015b600 0x7f67b01500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67ae59c640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67b00cf5e0 0x7f67b014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67aed9d640 1 -- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b015b600 msgr2=0x7f67b01500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.464+0000 7f67aed9d640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b015b600 0x7f67b01500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67aed9d640 1 -- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67b00cf5e0 msgr2=0x7f67b014fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67aed9d640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67b00cf5e0 0x7f67b014fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67aed9d640 1 -- 172.21.15.40:0/2133394597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b00b4c00 con 0x7f67b00cf0e0 2024-04-26T21:36:10.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67ae59c640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67b00cf5e0 0x7f67b014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:10.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67aed9d640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00cf0e0 0x7f67b00b5ca0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f67a000c7a0 tx=0x7f67a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67977fe640 1 -- 172.21.15.40:0/2133394597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0003b70 con 0x7f67b00cf0e0 2024-04-26T21:36:10.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67977fe640 1 -- 172.21.15.40:0/2133394597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a0003d10 con 0x7f67b00cf0e0 2024-04-26T21:36:10.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b00b4ee0 con 0x7f67b00cf0e0 2024-04-26T21:36:10.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67977fe640 1 -- 172.21.15.40:0/2133394597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a000aa90 con 0x7f67b00cf0e0 2024-04-26T21:36:10.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.465+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b00b54a0 con 0x7f67b00cf0e0 2024-04-26T21:36:10.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.466+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b0069c90 con 0x7f67b00cf0e0 2024-04-26T21:36:10.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.466+0000 7f67977fe640 1 -- 172.21.15.40:0/2133394597 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67a000ac30 con 0x7f67b00cf0e0 2024-04-26T21:36:10.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.467+0000 7f67977fe640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f679c06c8f0 0x7f679c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.467+0000 7f67977fe640 1 -- 172.21.15.40:0/2133394597 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f67a008be30 con 0x7f67b00cf0e0 2024-04-26T21:36:10.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.467+0000 7f67ae59c640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f679c06c8f0 0x7f679c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.468+0000 7f67ae59c640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f679c06c8f0 0x7f679c06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f67b0141000 tx=0x7f67a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.471+0000 7f67977fe640 1 -- 172.21.15.40:0/2133394597 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67b0069c90 con 0x7f67b00cf0e0 2024-04-26T21:36:10.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.657+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67b00694f0 con 0x7f67b00cf0e0 2024-04-26T21:36:10.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.658+0000 7f67977fe640 1 -- 172.21.15.40:0/2133394597 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f67a0059e60 con 0x7f67b00cf0e0 2024-04-26T21:36:10.658 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f679c06c8f0 msgr2=0x7f679c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f679c06c8f0 0x7f679c06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f67b0141000 tx=0x7f67a4009040 comp rx=0 tx=0).stop 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00cf0e0 msgr2=0x7f67b00b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00cf0e0 0x7f67b00b5ca0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f67a000c7a0 tx=0x7f67a000cc60 comp rx=0 tx=0).stop 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 shutdown_connections 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f679c06c8f0 0x7f679c06edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67b015b600 0x7f67b01500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67b00cf5e0 0x7f67b014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 --2- 172.21.15.40:0/2133394597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00cf0e0 0x7f67b00b5ca0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 >> 172.21.15.40:0/2133394597 conn(0x7f67b00d0be0 msgr2=0x7f67b0072c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:10.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 shutdown_connections 2024-04-26T21:36:10.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.660+0000 7f67b522a640 1 -- 172.21.15.40:0/2133394597 wait complete. 2024-04-26T21:36:10.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:36:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.778+0000 7ff0e9885640 1 Processor -- start 2024-04-26T21:36:10.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.778+0000 7ff0e9885640 1 -- start start 2024-04-26T21:36:10.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e9885640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4072950 0x7ff0e4167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e9885640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e415d7c0 con 0x7ff0e4071e60 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e9885640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e415daa0 con 0x7ff0e40cc480 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e9885640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e415dd80 con 0x7ff0e4072950 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff0e4071e60 0x7ff0e406f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50146/0 (socket says 172.21.15.40:50146) 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e2ffd640 1 -- 172.21.15.40:0/1549385820 learned_addr learned my addr 172.21.15.40:0/1549385820 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e37fe640 1 --2- 172.21.15.40:0/1549385820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4072950 0x7ff0e4167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e1ffb640 1 -- 172.21.15.40:0/1549385820 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609486634 0 0) 0x7ff0e415d7c0 con 0x7ff0e4071e60 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e1ffb640 1 -- 172.21.15.40:0/1549385820 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0cc003520 con 0x7ff0e4071e60 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e1ffb640 1 -- 172.21.15.40:0/1549385820 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875145457 0 0) 0x7ff0e415daa0 con 0x7ff0e40cc480 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e1ffb640 1 -- 172.21.15.40:0/1549385820 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0e415d7c0 con 0x7ff0e40cc480 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e1ffb640 1 -- 172.21.15.40:0/1549385820 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2260257666 0 0) 0x7ff0cc003520 con 0x7ff0e4071e60 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e37fe640 1 -- 172.21.15.40:0/1549385820 >> v1:172.21.15.184:6789/0 conn(0x7ff0e40cc480 legacy=0x7ff0e4072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e37fe640 1 -- 172.21.15.40:0/1549385820 >> v1:172.21.15.40:6789/0 conn(0x7ff0e4071e60 legacy=0x7ff0e406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.780+0000 7ff0e37fe640 1 -- 172.21.15.40:0/1549385820 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e415e060 con 0x7ff0e4072950 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.781+0000 7ff0e37fe640 1 --2- 172.21.15.40:0/1549385820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4072950 0x7ff0e4167ae0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0097d0 tx=0x7ff0dc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ecd497933f12d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.781+0000 7ff0e1ffb640 1 -- 172.21.15.40:0/1549385820 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc037070 con 0x7ff0e4072950 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.781+0000 7ff0e1ffb640 1 -- 172.21.15.40:0/1549385820 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0dc02ab90 con 0x7ff0e4072950 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.781+0000 7ff0e1ffb640 1 -- 172.21.15.40:0/1549385820 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40019f0 con 0x7ff0e4072950 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.782+0000 7ff0e9885640 1 -- 172.21.15.40:0/1549385820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4072950 msgr2=0x7ff0e4167ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.782+0000 7ff0e9885640 1 --2- 172.21.15.40:0/1549385820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4072950 0x7ff0e4167ae0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0097d0 tx=0x7ff0dc02a100 comp rx=0 tx=0).stop 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.782+0000 7ff0e9885640 1 -- 172.21.15.40:0/1549385820 shutdown_connections 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.782+0000 7ff0e9885640 1 --2- 172.21.15.40:0/1549385820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4072950 0x7ff0e4167ae0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.782+0000 7ff0e9885640 1 -- 172.21.15.40:0/1549385820 >> 172.21.15.40:0/1549385820 conn(0x7ff0e40d0be0 msgr2=0x7ff0e40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.782+0000 7ff0e9885640 1 -- 172.21.15.40:0/1549385820 shutdown_connections 2024-04-26T21:36:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.782+0000 7ff0e9885640 1 -- 172.21.15.40:0/1549385820 wait complete. 2024-04-26T21:36:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.783+0000 7ff0e9885640 1 Processor -- start 2024-04-26T21:36:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.783+0000 7ff0e9885640 1 -- start start 2024-04-26T21:36:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e9885640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4071e60 0x7ff0e41f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e9885640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0e4072950 0x7ff0e4145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e9885640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e40cc480 0x7ff0e414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e9885640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e415d570 con 0x7ff0e4072950 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e9885640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e415d3f0 con 0x7ff0e40cc480 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e9885640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e406fff0 con 0x7ff0e4071e60 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0e4072950 0x7ff0e4145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0e4072950 0x7ff0e4145c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58248/0 (socket says 172.21.15.40:58248) 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e27fc640 1 -- 172.21.15.40:0/2607700374 learned_addr learned my addr 172.21.15.40:0/2607700374 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e2ffd640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4071e60 0x7ff0e41f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.784+0000 7ff0e37fe640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e40cc480 0x7ff0e414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0e27fc640 1 -- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4071e60 msgr2=0x7ff0e41f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0e27fc640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4071e60 0x7ff0e41f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0e27fc640 1 -- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e40cc480 msgr2=0x7ff0e414fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0e27fc640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e40cc480 0x7ff0e414fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0e27fc640 1 -- 172.21.15.40:0/2607700374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e414ffb0 con 0x7ff0e4072950 2024-04-26T21:36:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0e27fc640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0e4072950 0x7ff0e4145c30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c460 tx=0x7ff0d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0cbfff640 1 -- 172.21.15.40:0/2607700374 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80052d0 con 0x7ff0e4072950 2024-04-26T21:36:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0cbfff640 1 -- 172.21.15.40:0/2607700374 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d8005490 con 0x7ff0e4072950 2024-04-26T21:36:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e4152510 con 0x7ff0e4072950 2024-04-26T21:36:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0cbfff640 1 -- 172.21.15.40:0/2607700374 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8012870 con 0x7ff0e4072950 2024-04-26T21:36:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.785+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e4152ad0 con 0x7ff0e4072950 2024-04-26T21:36:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.786+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e40694f0 con 0x7ff0e4072950 2024-04-26T21:36:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.787+0000 7ff0cbfff640 1 -- 172.21.15.40:0/2607700374 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0d8012a10 con 0x7ff0e4072950 2024-04-26T21:36:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.787+0000 7ff0cbfff640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d006c8f0 0x7ff0d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.787+0000 7ff0cbfff640 1 -- 172.21.15.40:0/2607700374 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff0d808b960 con 0x7ff0e4072950 2024-04-26T21:36:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.787+0000 7ff0e2ffd640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d006c8f0 0x7ff0d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.788+0000 7ff0e2ffd640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d006c8f0 0x7ff0d006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4001b60 tx=0x7ff0d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.792+0000 7ff0cbfff640 1 -- 172.21.15.40:0/2607700374 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0e414ffb0 con 0x7ff0e4072950 2024-04-26T21:36:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.995+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff0e41501c0 con 0x7ff0e4072950 2024-04-26T21:36:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.996+0000 7ff0cbfff640 1 -- 172.21.15.40:0/2607700374 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7ff0d80593d0 con 0x7ff0e4072950 2024-04-26T21:36:10.996 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d006c8f0 msgr2=0x7ff0d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d006c8f0 0x7ff0d006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4001b60 tx=0x7ff0d4009040 comp rx=0 tx=0).stop 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0e4072950 msgr2=0x7ff0e4145c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0e4072950 0x7ff0e4145c30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c460 tx=0x7ff0d800c920 comp rx=0 tx=0).stop 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 shutdown_connections 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d006c8f0 0x7ff0d006edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e40cc480 0x7ff0e414fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0e4072950 0x7ff0e4145c30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 --2- 172.21.15.40:0/2607700374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e4071e60 0x7ff0e41f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 >> 172.21.15.40:0/2607700374 conn(0x7ff0e40d0be0 msgr2=0x7ff0e415f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 shutdown_connections 2024-04-26T21:36:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:10.998+0000 7ff0e9885640 1 -- 172.21.15.40:0/2607700374 wait complete. 2024-04-26T21:36:11.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T21:36:11.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.114+0000 7f20e22c4640 1 Processor -- start 2024-04-26T21:36:11.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.114+0000 7f20e22c4640 1 -- start start 2024-04-26T21:36:11.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.115+0000 7f20e22c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f20dc0b0b70 0x7f20dc0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.115+0000 7f20e22c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20dc0b4d40 0x7f20dc0b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.115+0000 7f20e22c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b6c50 0x7f20dc15f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.115+0000 7f20e22c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f20dc073bc0 con 0x7f20dc0b4d40 2024-04-26T21:36:11.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.115+0000 7f20e22c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f20dc0b1490 con 0x7f20dc0b0b70 2024-04-26T21:36:11.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.115+0000 7f20e22c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc072fb0 con 0x7f20dc0b6c50 2024-04-26T21:36:11.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.115+0000 7f20db7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20dc0b4d40 0x7f20dc0b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20db7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20dc0b4d40 0x7f20dc0b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58252/0 (socket says 172.21.15.40:58252) 2024-04-26T21:36:11.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20db7fe640 1 -- 172.21.15.40:0/2484564134 learned_addr learned my addr 172.21.15.40:0/2484564134 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:11.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20dbfff640 1 --2- 172.21.15.40:0/2484564134 >> v2:172.21.15.184:3300/0 conn(0x7f20dc0b0b70 0x7f20dc0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20e0861640 1 --2- 172.21.15.40:0/2484564134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b6c50 0x7f20dc15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20db7fe640 1 -- 172.21.15.40:0/2484564134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b6c50 msgr2=0x7f20dc15f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:11.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20db7fe640 1 --2- 172.21.15.40:0/2484564134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b6c50 0x7f20dc15f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20db7fe640 1 -- 172.21.15.40:0/2484564134 >> v2:172.21.15.184:3300/0 conn(0x7f20dc0b0b70 msgr2=0x7f20dc0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:11.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20db7fe640 1 --2- 172.21.15.40:0/2484564134 >> v2:172.21.15.184:3300/0 conn(0x7f20dc0b0b70 0x7f20dc0b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.116+0000 7f20db7fe640 1 -- 172.21.15.40:0/2484564134 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc073290 con 0x7f20dc0b4d40 2024-04-26T21:36:11.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20db7fe640 1 --2- 172.21.15.40:0/2484564134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b4d40 0x7f20dc0b6710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f20dc06edf0 tx=0x7f20c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749cc6176eb10927 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20daffd640 1 -- 172.21.15.40:0/2484564134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4037070 con 0x7f20dc0b4d40 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20daffd640 1 -- 172.21.15.40:0/2484564134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4032440 con 0x7f20dc0b4d40 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20daffd640 1 -- 172.21.15.40:0/2484564134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4032810 con 0x7f20dc0b4d40 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 -- 172.21.15.40:0/2484564134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b4d40 msgr2=0x7f20dc0b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 --2- 172.21.15.40:0/2484564134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b4d40 0x7f20dc0b6710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f20dc06edf0 tx=0x7f20c402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 -- 172.21.15.40:0/2484564134 shutdown_connections 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 --2- 172.21.15.40:0/2484564134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b6c50 0x7f20dc15f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 --2- 172.21.15.40:0/2484564134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b4d40 0x7f20dc0b6710 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 --2- 172.21.15.40:0/2484564134 >> v2:172.21.15.184:3300/0 conn(0x7f20dc0b0b70 0x7f20dc0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 -- 172.21.15.40:0/2484564134 >> 172.21.15.40:0/2484564134 conn(0x7f20dc0b39b0 msgr2=0x7f20dc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 -- 172.21.15.40:0/2484564134 shutdown_connections 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.117+0000 7f20e22c4640 1 -- 172.21.15.40:0/2484564134 wait complete. 2024-04-26T21:36:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.119+0000 7f20e22c4640 1 Processor -- start 2024-04-26T21:36:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.119+0000 7f20e22c4640 1 -- start start 2024-04-26T21:36:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.119+0000 7f20e22c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b0b70 0x7f20dc14eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.119+0000 7f20e22c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b4d40 0x7f20dc0afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.119+0000 7f20e22c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc0b6c50 0x7f20dc0d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.119+0000 7f20e22c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc15f890 con 0x7f20dc0b0b70 2024-04-26T21:36:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.119+0000 7f20e22c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc073850 con 0x7f20dc0b6c50 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.119+0000 7f20e22c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc15fa10 con 0x7f20dc0b4d40 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b0b70 0x7f20dc14eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b0b70 0x7f20dc14eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58260/0 (socket says 172.21.15.40:58260) 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 -- 172.21.15.40:0/3021121237 learned_addr learned my addr 172.21.15.40:0/3021121237 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20db7fe640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b4d40 0x7f20dc0afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20e0861640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc0b6c50 0x7f20dc0d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 -- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b4d40 msgr2=0x7f20dc0afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b4d40 0x7f20dc0afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 -- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc0b6c50 msgr2=0x7f20dc0d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc0b6c50 0x7f20dc0d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 -- 172.21.15.40:0/3021121237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc0d0730 con 0x7f20dc0b0b70 2024-04-26T21:36:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20dbfff640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b0b70 0x7f20dc14eb50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00e8c0 tx=0x7f20cc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20d8ff9640 1 -- 172.21.15.40:0/3021121237 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc017070 con 0x7f20dc0b0b70 2024-04-26T21:36:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc0d0a30 con 0x7f20dc0b0b70 2024-04-26T21:36:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20d8ff9640 1 -- 172.21.15.40:0/3021121237 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20cc00c940 con 0x7f20dc0b0b70 2024-04-26T21:36:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20d8ff9640 1 -- 172.21.15.40:0/3021121237 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00cd10 con 0x7f20dc0b0b70 2024-04-26T21:36:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.120+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc0d1270 con 0x7f20dc0b0b70 2024-04-26T21:36:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.121+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc0694f0 con 0x7f20dc0b0b70 2024-04-26T21:36:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.122+0000 7f20d8ff9640 1 -- 172.21.15.40:0/3021121237 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20cc01a070 con 0x7f20dc0b0b70 2024-04-26T21:36:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.122+0000 7f20d8ff9640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20bc06c8f0 0x7f20bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.122+0000 7f20d8ff9640 1 -- 172.21.15.40:0/3021121237 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f20cc08be90 con 0x7f20dc0b0b70 2024-04-26T21:36:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.123+0000 7f20db7fe640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20bc06c8f0 0x7f20bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.123+0000 7f20db7fe640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20bc06c8f0 0x7f20bc06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f20c4034000 tx=0x7f20c402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.131+0000 7f20d8ff9640 1 -- 172.21.15.40:0/3021121237 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20dc0d0a30 con 0x7f20dc0b0b70 2024-04-26T21:36:11.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.315+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f20dc06dfb0 con 0x7f20dc0b0b70 2024-04-26T21:36:11.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.315+0000 7f20d8ff9640 1 -- 172.21.15.40:0/3021121237 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f20cc059870 con 0x7f20dc0b0b70 2024-04-26T21:36:11.315 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:11.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20bc06c8f0 msgr2=0x7f20bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20bc06c8f0 0x7f20bc06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f20c4034000 tx=0x7f20c402e000 comp rx=0 tx=0).stop 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b0b70 msgr2=0x7f20dc14eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b0b70 0x7f20dc14eb50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00e8c0 tx=0x7f20cc00ed80 comp rx=0 tx=0).stop 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 shutdown_connections 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20bc06c8f0 0x7f20bc06edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc0b6c50 0x7f20dc0d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0b4d40 0x7f20dc0afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 --2- 172.21.15.40:0/3021121237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20dc0b0b70 0x7f20dc14eb50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 >> 172.21.15.40:0/3021121237 conn(0x7f20dc0b39b0 msgr2=0x7f20dc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 shutdown_connections 2024-04-26T21:36:11.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.317+0000 7f20e22c4640 1 -- 172.21.15.40:0/3021121237 wait complete. 2024-04-26T21:36:11.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.428+0000 7f2427e6d640 1 Processor -- start 2024-04-26T21:36:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.429+0000 7f2427e6d640 1 -- start start 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f2427e6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f242015d100 0x7f2420160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f2427e6d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f242006c950 0x7f24200aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f2427e6d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24200b0990 con 0x7f2420070930 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f2427e6d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f24200af2d0 con 0x7f242015d100 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f2427e6d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f24200b0c70 con 0x7f242006c950 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f2425c09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f242015d100 0x7f2420160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f2425c09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f242015d100 0x7f2420160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58274/0 (socket says 172.21.15.40:58274) 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f2425c09640 1 -- 172.21.15.40:0/611064769 learned_addr learned my addr 172.21.15.40:0/611064769 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.430+0000 7f242640a640 1 --2- 172.21.15.40:0/611064769 >> v2:172.21.15.184:3300/0 conn(0x7f242006c950 0x7f24200aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.431+0000 7f2425c09640 1 -- 172.21.15.40:0/611064769 >> v2:172.21.15.184:3300/0 conn(0x7f242006c950 msgr2=0x7f24200aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.431+0000 7f2425c09640 1 --2- 172.21.15.40:0/611064769 >> v2:172.21.15.184:3300/0 conn(0x7f242006c950 0x7f24200aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.431+0000 7f2425c09640 1 -- 172.21.15.40:0/611064769 >> v1:172.21.15.184:6789/0 conn(0x7f2420070930 legacy=0x7f242006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.431+0000 7f2425c09640 1 -- 172.21.15.40:0/611064769 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24200b0f50 con 0x7f242015d100 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.431+0000 7f2425c09640 1 --2- 172.21.15.40:0/611064769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f2420160ba0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f24200d0da0 tx=0x7f241402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ae72491eebbe56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.431+0000 7f2424c07640 1 -- 172.21.15.40:0/611064769 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414037070 con 0x7f242015d100 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2424c07640 1 -- 172.21.15.40:0/611064769 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241402ae10 con 0x7f242015d100 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2424c07640 1 -- 172.21.15.40:0/611064769 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414032660 con 0x7f242015d100 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2427e6d640 1 -- 172.21.15.40:0/611064769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 msgr2=0x7f2420160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2427e6d640 1 --2- 172.21.15.40:0/611064769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f2420160ba0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f24200d0da0 tx=0x7f241402a180 comp rx=0 tx=0).stop 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2427e6d640 1 -- 172.21.15.40:0/611064769 shutdown_connections 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2427e6d640 1 --2- 172.21.15.40:0/611064769 >> v2:172.21.15.184:3300/0 conn(0x7f242006c950 0x7f24200aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2427e6d640 1 --2- 172.21.15.40:0/611064769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f2420160ba0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2427e6d640 1 -- 172.21.15.40:0/611064769 >> 172.21.15.40:0/611064769 conn(0x7f24200b39b0 msgr2=0x7f24200b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2427e6d640 1 -- 172.21.15.40:0/611064769 shutdown_connections 2024-04-26T21:36:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.432+0000 7f2427e6d640 1 -- 172.21.15.40:0/611064769 wait complete. 2024-04-26T21:36:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.434+0000 7f2427e6d640 1 Processor -- start 2024-04-26T21:36:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.434+0000 7f2427e6d640 1 -- start start 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f2427e6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242006c950 0x7f24201f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f2427e6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2420070930 0x7f242009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f2427e6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f242009bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f2427e6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2420073160 con 0x7f242015d100 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f2427e6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24200b1510 con 0x7f242006c950 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f2427e6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24200732e0 con 0x7f2420070930 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f242640a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f242009bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f242640a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f242009bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58276/0 (socket says 172.21.15.40:58276) 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f242640a640 1 -- 172.21.15.40:0/3009256001 learned_addr learned my addr 172.21.15.40:0/3009256001 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.435+0000 7f2425c09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242006c950 0x7f24201f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f2425408640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2420070930 0x7f242009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f242640a640 1 -- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2420070930 msgr2=0x7f242009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f242640a640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2420070930 0x7f242009ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f242640a640 1 -- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242006c950 msgr2=0x7f24201f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f242640a640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242006c950 0x7f24201f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f242640a640 1 -- 172.21.15.40:0/3009256001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24201570f0 con 0x7f242015d100 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f2425c09640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242006c950 0x7f24201f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f242640a640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f242009bf90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f241800ef90 tx=0x7f241800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f24127fc640 1 -- 172.21.15.40:0/3009256001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418018020 con 0x7f242015d100 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f24127fc640 1 -- 172.21.15.40:0/3009256001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2418014480 con 0x7f242015d100 2024-04-26T21:36:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242009c720 con 0x7f242015d100 2024-04-26T21:36:11.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f24127fc640 1 -- 172.21.15.40:0/3009256001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418014870 con 0x7f242015d100 2024-04-26T21:36:11.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.436+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242009cca0 con 0x7f242015d100 2024-04-26T21:36:11.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.437+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24200694f0 con 0x7f242015d100 2024-04-26T21:36:11.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.438+0000 7f24127fc640 1 -- 172.21.15.40:0/3009256001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2418005d10 con 0x7f242015d100 2024-04-26T21:36:11.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.438+0000 7f24127fc640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f240406c8f0 0x7f240406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.439+0000 7f24127fc640 1 -- 172.21.15.40:0/3009256001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f241808ccb0 con 0x7f242015d100 2024-04-26T21:36:11.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.439+0000 7f2425c09640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f240406c8f0 0x7f240406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.439+0000 7f2425c09640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f240406c8f0 0x7f240406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2414034000 tx=0x7f241402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.443+0000 7f24127fc640 1 -- 172.21.15.40:0/3009256001 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2418022030 con 0x7f242015d100 2024-04-26T21:36:11.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.667+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24201604e0 con 0x7f242015d100 2024-04-26T21:36:11.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.667+0000 7f24127fc640 1 -- 172.21.15.40:0/3009256001 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f24201570f0 con 0x7f242015d100 2024-04-26T21:36:11.668 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:11.668 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:31.566271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:36:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f240406c8f0 msgr2=0x7f240406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f240406c8f0 0x7f240406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2414034000 tx=0x7f241402e000 comp rx=0 tx=0).stop 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 msgr2=0x7f242009bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f242009bf90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f241800ef90 tx=0x7f241800c490 comp rx=0 tx=0).stop 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 shutdown_connections 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f240406c8f0 0x7f240406edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242015d100 0x7f242009bf90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2420070930 0x7f242009ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 --2- 172.21.15.40:0/3009256001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242006c950 0x7f24201f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 >> 172.21.15.40:0/3009256001 conn(0x7f24200b39b0 msgr2=0x7f242006f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:11.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 shutdown_connections 2024-04-26T21:36:11.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.669+0000 7f2427e6d640 1 -- 172.21.15.40:0/3009256001 wait complete. 2024-04-26T21:36:11.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:11.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.781+0000 7fb3f69f0640 1 Processor -- start 2024-04-26T21:36:11.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.781+0000 7fb3f69f0640 1 -- start start 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3f69f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f0162810 0x7fb3f0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3f69f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3f00d1b30 0x7fb3f00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3f69f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f00d10d0 con 0x7fb3f00b2340 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3f69f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f00d0750 con 0x7fb3f00d1b30 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3f69f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f00d16a0 con 0x7fb3f0162810 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3ef7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb3f00b2340 0x7fb3f00cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50156/0 (socket says 172.21.15.40:50156) 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3ef7fe640 1 -- 172.21.15.40:0/2866195537 learned_addr learned my addr 172.21.15.40:0/2866195537 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3effff640 1 --2- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f0162810 0x7fb3f0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.782+0000 7fb3f4f8d640 1 --2- 172.21.15.40:0/2866195537 >> v2:172.21.15.184:3300/0 conn(0x7fb3f00d1b30 0x7fb3f00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.783+0000 7fb3eeffd640 1 -- 172.21.15.40:0/2866195537 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603231286 0 0) 0x7fb3f00d10d0 con 0x7fb3f00b2340 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.783+0000 7fb3eeffd640 1 -- 172.21.15.40:0/2866195537 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3d8003520 con 0x7fb3f00b2340 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.783+0000 7fb3f4f8d640 1 -- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f0162810 msgr2=0x7fb3f0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.783+0000 7fb3f4f8d640 1 --2- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f0162810 0x7fb3f0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.783+0000 7fb3f4f8d640 1 -- 172.21.15.40:0/2866195537 >> v1:172.21.15.40:6789/0 conn(0x7fb3f00b2340 legacy=0x7fb3f00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.783+0000 7fb3f4f8d640 1 -- 172.21.15.40:0/2866195537 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f00d1370 con 0x7fb3f00d1b30 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.783+0000 7fb3effff640 1 --2- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f0162810 0x7fb3f0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.783+0000 7fb3f4f8d640 1 --2- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00d1fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009d90 tx=0x7fb3e802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8925d98783438fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3eeffd640 1 -- 172.21.15.40:0/2866195537 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8007740 con 0x7fb3f00d1b30 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3eeffd640 1 -- 172.21.15.40:0/2866195537 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e80078e0 con 0x7fb3f00d1b30 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3eeffd640 1 -- 172.21.15.40:0/2866195537 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8007c90 con 0x7fb3f00d1b30 2024-04-26T21:36:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3f69f0640 1 -- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f00d1b30 msgr2=0x7fb3f00d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00d1fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009d90 tx=0x7fb3e802a660 comp rx=0 tx=0).stop 2024-04-26T21:36:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3f69f0640 1 -- 172.21.15.40:0/2866195537 shutdown_connections 2024-04-26T21:36:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00d1fc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/2866195537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f0162810 0x7fb3f0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3f69f0640 1 -- 172.21.15.40:0/2866195537 >> 172.21.15.40:0/2866195537 conn(0x7fb3f00d2da0 msgr2=0x7fb3f00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3f69f0640 1 -- 172.21.15.40:0/2866195537 shutdown_connections 2024-04-26T21:36:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.784+0000 7fb3f69f0640 1 -- 172.21.15.40:0/2866195537 wait complete. 2024-04-26T21:36:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.785+0000 7fb3f69f0640 1 Processor -- start 2024-04-26T21:36:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.786+0000 7fb3f69f0640 1 -- start start 2024-04-26T21:36:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.786+0000 7fb3f69f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f00b2340 0x7fb3f0152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.786+0000 7fb3f69f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.786+0000 7fb3f69f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f0162810 0x7fb3f00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.786+0000 7fb3f69f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0070890 con 0x7fb3f00d1b30 2024-04-26T21:36:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.786+0000 7fb3f69f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f00d0470 con 0x7fb3f0162810 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.786+0000 7fb3f69f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f00cdb00 con 0x7fb3f00b2340 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.786+0000 7fb3ef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58292/0 (socket says 172.21.15.40:58292) 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ef7fe640 1 -- 172.21.15.40:0/1401873160 learned_addr learned my addr 172.21.15.40:0/1401873160 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3effff640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f00b2340 0x7fb3f0152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3f4f8d640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f0162810 0x7fb3f00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ef7fe640 1 -- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f00b2340 msgr2=0x7fb3f0152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ef7fe640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f00b2340 0x7fb3f0152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ef7fe640 1 -- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f0162810 msgr2=0x7fb3f00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ef7fe640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f0162810 0x7fb3f00cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ef7fe640 1 -- 172.21.15.40:0/1401873160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f01525b0 con 0x7fb3f00d1b30 2024-04-26T21:36:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3f4f8d640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f0162810 0x7fb3f00cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ef7fe640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00c68b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000c460 tx=0x7fb3e000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ecff9640 1 -- 172.21.15.40:0/1401873160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0017070 con 0x7fb3f00d1b30 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ecff9640 1 -- 172.21.15.40:0/1401873160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e0005ce0 con 0x7fb3f00d1b30 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3ecff9640 1 -- 172.21.15.40:0/1401873160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00072f0 con 0x7fb3f00d1b30 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f015efe0 con 0x7fb3f00d1b30 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.787+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f015f5a0 con 0x7fb3f00d1b30 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.788+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f00694f0 con 0x7fb3f00d1b30 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.789+0000 7fb3ecff9640 1 -- 172.21.15.40:0/1401873160 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3e001a070 con 0x7fb3f00d1b30 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.789+0000 7fb3ecff9640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3dc06c8f0 0x7fb3dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.789+0000 7fb3ecff9640 1 -- 172.21.15.40:0/1401873160 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb3e008b220 con 0x7fb3f00d1b30 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.789+0000 7fb3effff640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3dc06c8f0 0x7fb3dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.790+0000 7fb3effff640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3dc06c8f0 0x7fb3dc06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3f015c4b0 tx=0x7fb3e4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.794+0000 7fb3ecff9640 1 -- 172.21.15.40:0/1401873160 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3e0058c90 con 0x7fb3f00d1b30 2024-04-26T21:36:11.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.979+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3f006e890 con 0x7fb3f00d1b30 2024-04-26T21:36:11.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.980+0000 7fb3ecff9640 1 -- 172.21.15.40:0/1401873160 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7fb3e005cf90 con 0x7fb3f00d1b30 2024-04-26T21:36:11.980 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:11.980 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3dc06c8f0 msgr2=0x7fb3dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3dc06c8f0 0x7fb3dc06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3f015c4b0 tx=0x7fb3e4009190 comp rx=0 tx=0).stop 2024-04-26T21:36:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f00d1b30 msgr2=0x7fb3f00c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00c68b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000c460 tx=0x7fb3e000c920 comp rx=0 tx=0).stop 2024-04-26T21:36:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 shutdown_connections 2024-04-26T21:36:11.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3dc06c8f0 0x7fb3dc06edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f0162810 0x7fb3f00cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f00d1b30 0x7fb3f00c68b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 --2- 172.21.15.40:0/1401873160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f00b2340 0x7fb3f0152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:11.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 >> 172.21.15.40:0/1401873160 conn(0x7fb3f00d2da0 msgr2=0x7fb3f006edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:11.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 shutdown_connections 2024-04-26T21:36:11.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:11.982+0000 7fb3f69f0640 1 -- 172.21.15.40:0/1401873160 wait complete. 2024-04-26T21:36:11.993 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:36:11.993 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.094+0000 7ff176c59640 1 Processor -- start 2024-04-26T21:36:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.094+0000 7ff176c59640 1 -- start start 2024-04-26T21:36:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.095+0000 7ff176c59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1700d2830 0x7ff1700d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.095+0000 7ff176c59640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1700d1a70 con 0x7ff170163610 2024-04-26T21:36:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.095+0000 7ff176c59640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1700d1d50 con 0x7ff17015f5a0 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.095+0000 7ff176c59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff170163d30 con 0x7ff1700d2830 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.095+0000 7ff16ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff170163610 0x7ff1700cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50164/0 (socket says 172.21.15.40:50164) 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.095+0000 7ff16ffff640 1 -- 172.21.15.40:0/319644570 learned_addr learned my addr 172.21.15.40:0/319644570 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff1751f6640 1 --2- 172.21.15.40:0/319644570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1700d2830 0x7ff1700d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff16f7fe640 1 -- 172.21.15.40:0/319644570 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4052880337 0 0) 0x7ff1700d1a70 con 0x7ff170163610 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff16f7fe640 1 -- 172.21.15.40:0/319644570 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff158003520 con 0x7ff170163610 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff16f7fe640 1 -- 172.21.15.40:0/319644570 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2109421295 0 0) 0x7ff1700d1d50 con 0x7ff17015f5a0 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff16f7fe640 1 -- 172.21.15.40:0/319644570 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1700d1a70 con 0x7ff17015f5a0 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff1751f6640 1 -- 172.21.15.40:0/319644570 >> v1:172.21.15.184:6789/0 conn(0x7ff17015f5a0 legacy=0x7ff170162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff16f7fe640 1 -- 172.21.15.40:0/319644570 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1047766636 0 0) 0x7ff158003520 con 0x7ff170163610 2024-04-26T21:36:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff1751f6640 1 -- 172.21.15.40:0/319644570 >> v1:172.21.15.40:6789/0 conn(0x7ff170163610 legacy=0x7ff1700cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.096+0000 7ff1751f6640 1 -- 172.21.15.40:0/319644570 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1700d2030 con 0x7ff1700d2830 2024-04-26T21:36:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff1751f6640 1 --2- 172.21.15.40:0/319644570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1700d2830 0x7ff1700d5cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1600097d0 tx=0x7ff16002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232e8d5b7dcb5656 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff16f7fe640 1 -- 172.21.15.40:0/319644570 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160037070 con 0x7ff1700d2830 2024-04-26T21:36:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff16f7fe640 1 -- 172.21.15.40:0/319644570 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16002ab90 con 0x7ff1700d2830 2024-04-26T21:36:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff16f7fe640 1 -- 172.21.15.40:0/319644570 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164001670 con 0x7ff1700d2830 2024-04-26T21:36:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff176c59640 1 -- 172.21.15.40:0/319644570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1700d2830 msgr2=0x7ff1700d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff176c59640 1 --2- 172.21.15.40:0/319644570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1700d2830 0x7ff1700d5cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1600097d0 tx=0x7ff16002a100 comp rx=0 tx=0).stop 2024-04-26T21:36:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff176c59640 1 -- 172.21.15.40:0/319644570 shutdown_connections 2024-04-26T21:36:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff176c59640 1 --2- 172.21.15.40:0/319644570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1700d2830 0x7ff1700d5cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff176c59640 1 -- 172.21.15.40:0/319644570 >> 172.21.15.40:0/319644570 conn(0x7ff1700d5190 msgr2=0x7ff1700d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:12.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff176c59640 1 -- 172.21.15.40:0/319644570 shutdown_connections 2024-04-26T21:36:12.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.097+0000 7ff176c59640 1 -- 172.21.15.40:0/319644570 wait complete. 2024-04-26T21:36:12.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.099+0000 7ff176c59640 1 Processor -- start 2024-04-26T21:36:12.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.099+0000 7ff176c59640 1 -- start start 2024-04-26T21:36:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.099+0000 7ff176c59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1700d2830 0x7ff17014e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.099+0000 7ff176c59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff170163610 0x7ff1700c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.099+0000 7ff176c59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17014f360 0x7ff1700b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.099+0000 7ff176c59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1700d3c20 con 0x7ff17014f360 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.099+0000 7ff176c59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1700d4190 con 0x7ff1700d2830 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.099+0000 7ff176c59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1700d3da0 con 0x7ff170163610 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17014f360 0x7ff1700b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17014f360 0x7ff1700b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58308/0 (socket says 172.21.15.40:58308) 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 -- 172.21.15.40:0/3799136775 learned_addr learned my addr 172.21.15.40:0/3799136775 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff16ffff640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff170163610 0x7ff1700c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1749f5640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1700d2830 0x7ff17014e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 -- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff170163610 msgr2=0x7ff1700c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff170163610 0x7ff1700c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 -- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1700d2830 msgr2=0x7ff17014e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1700d2830 0x7ff17014e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 -- 172.21.15.40:0/3799136775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17014f860 con 0x7ff17014f360 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff1751f6640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17014f360 0x7ff1700b7db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff1600097d0 tx=0x7ff1600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff16d7fa640 1 -- 172.21.15.40:0/3799136775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160041020 con 0x7ff17014f360 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff16d7fa640 1 -- 172.21.15.40:0/3799136775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16003d620 con 0x7ff17014f360 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17014fb40 con 0x7ff17014f360 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff16d7fa640 1 -- 172.21.15.40:0/3799136775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160037040 con 0x7ff17014f360 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.100+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff170150100 con 0x7ff17014f360 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.101+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1700694f0 con 0x7ff17014f360 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.102+0000 7ff16d7fa640 1 -- 172.21.15.40:0/3799136775 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff160044020 con 0x7ff17014f360 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.102+0000 7ff16d7fa640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff15c06c920 0x7ff15c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.102+0000 7ff16d7fa640 1 -- 172.21.15.40:0/3799136775 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff1600ac450 con 0x7ff17014f360 2024-04-26T21:36:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.102+0000 7ff1749f5640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff15c06c920 0x7ff15c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.103+0000 7ff1749f5640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff15c06c920 0x7ff15c06ede0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff168001560 tx=0x7ff168009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.106+0000 7ff16d7fa640 1 -- 172.21.15.40:0/3799136775 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff16004b030 con 0x7ff17014f360 2024-04-26T21:36:12.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.330+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff170068d30 con 0x7ff17014f360 2024-04-26T21:36:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.331+0000 7ff16d7fa640 1 -- 172.21.15.40:0/3799136775 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7ff16007a480 con 0x7ff17014f360 2024-04-26T21:36:12.331 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:12.331 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:31.566271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:36:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff15c06c920 msgr2=0x7ff15c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff15c06c920 0x7ff15c06ede0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff168001560 tx=0x7ff168009040 comp rx=0 tx=0).stop 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17014f360 msgr2=0x7ff1700b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17014f360 0x7ff1700b7db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff1600097d0 tx=0x7ff1600046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 shutdown_connections 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff15c06c920 0x7ff15c06ede0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17014f360 0x7ff1700b7db0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff170163610 0x7ff1700c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 --2- 172.21.15.40:0/3799136775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1700d2830 0x7ff17014e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 >> 172.21.15.40:0/3799136775 conn(0x7ff1700d5190 msgr2=0x7ff17006e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:12.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 shutdown_connections 2024-04-26T21:36:12.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.333+0000 7ff176c59640 1 -- 172.21.15.40:0/3799136775 wait complete. 2024-04-26T21:36:12.345 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:12.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.446+0000 7fdcbbe6d640 1 Processor -- start 2024-04-26T21:36:12.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.446+0000 7fdcbbe6d640 1 -- start start 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcbbe6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdcb40b2340 0x7fdcb40cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcbbe6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 0x7fdcb40d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcbbe6d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcb40d10d0 con 0x7fdcb4162810 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcbbe6d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdcb40d0750 con 0x7fdcb40b2340 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcbbe6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb40d16a0 con 0x7fdcb40cd930 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcb9408640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdcb40b2340 0x7fdcb40cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcb9408640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdcb40b2340 0x7fdcb40cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58322/0 (socket says 172.21.15.40:58322) 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcb9408640 1 -- 172.21.15.40:0/77016661 learned_addr learned my addr 172.21.15.40:0/77016661 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcba40a640 1 --2- 172.21.15.40:0/77016661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 0x7fdcb40d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcb9408640 1 -- 172.21.15.40:0/77016661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 msgr2=0x7fdcb40d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcb9408640 1 --2- 172.21.15.40:0/77016661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 0x7fdcb40d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcb9408640 1 -- 172.21.15.40:0/77016661 >> v1:172.21.15.184:6789/0 conn(0x7fdcb4162810 legacy=0x7fdcb4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.447+0000 7fdcb9408640 1 -- 172.21.15.40:0/77016661 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb40d1370 con 0x7fdcb40b2340 2024-04-26T21:36:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcb9408640 1 --2- 172.21.15.40:0/77016661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40cd3f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40d1820 tx=0x7fdca402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cd9d5062f2130d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcb8c07640 1 -- 172.21.15.40:0/77016661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4037070 con 0x7fdcb40b2340 2024-04-26T21:36:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcb8c07640 1 -- 172.21.15.40:0/77016661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca4032440 con 0x7fdcb40b2340 2024-04-26T21:36:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcb8c07640 1 -- 172.21.15.40:0/77016661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4032810 con 0x7fdcb40b2340 2024-04-26T21:36:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/77016661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 msgr2=0x7fdcb40cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/77016661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40cd3f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40d1820 tx=0x7fdca402a200 comp rx=0 tx=0).stop 2024-04-26T21:36:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/77016661 shutdown_connections 2024-04-26T21:36:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/77016661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 0x7fdcb40d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/77016661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40cd3f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.448+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/77016661 >> 172.21.15.40:0/77016661 conn(0x7fdcb40d2da0 msgr2=0x7fdcb40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.449+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/77016661 shutdown_connections 2024-04-26T21:36:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.449+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/77016661 wait complete. 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.450+0000 7fdcbbe6d640 1 Processor -- start 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.450+0000 7fdcbbe6d640 1 -- start start 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.450+0000 7fdcbbe6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.450+0000 7fdcbbe6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 0x7fdcb40ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.450+0000 7fdcbbe6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb4162810 0x7fdcb414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.450+0000 7fdcbbe6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb40d2070 con 0x7fdcb40b2340 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.450+0000 7fdcbbe6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb40d0470 con 0x7fdcb4162810 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.450+0000 7fdcbbe6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb40d21f0 con 0x7fdcb40cd930 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58330/0 (socket says 172.21.15.40:58330) 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 -- 172.21.15.40:0/367786725 learned_addr learned my addr 172.21.15.40:0/367786725 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9408640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 0x7fdcb40ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcba40a640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb4162810 0x7fdcb414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 -- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 msgr2=0x7fdcb40ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 0x7fdcb40ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 -- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb4162810 msgr2=0x7fdcb414a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb4162810 0x7fdcb414a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 -- 172.21.15.40:0/367786725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb40cb430 con 0x7fdcb40b2340 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcba40a640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb4162810 0x7fdcb414a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcb9c09640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40c68d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b3f0 tx=0x7fdcb000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcaa7fc640 1 -- 172.21.15.40:0/367786725 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0015020 con 0x7fdcb40b2340 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcaa7fc640 1 -- 172.21.15.40:0/367786725 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb0011830 con 0x7fdcb40b2340 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb40cb710 con 0x7fdcb40b2340 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcaa7fc640 1 -- 172.21.15.40:0/367786725 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0011c20 con 0x7fdcb40b2340 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.451+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb40cbcd0 con 0x7fdcb40b2340 2024-04-26T21:36:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.452+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb40694f0 con 0x7fdcb40b2340 2024-04-26T21:36:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.453+0000 7fdcaa7fc640 1 -- 172.21.15.40:0/367786725 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcb0018020 con 0x7fdcb40b2340 2024-04-26T21:36:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.453+0000 7fdcaa7fc640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc9806c8f0 0x7fdc9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.453+0000 7fdcaa7fc640 1 -- 172.21.15.40:0/367786725 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdcb008bb50 con 0x7fdcb40b2340 2024-04-26T21:36:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.453+0000 7fdcb9408640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc9806c8f0 0x7fdc9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.454+0000 7fdcb9408640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc9806c8f0 0x7fdc9806edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdca4034000 tx=0x7fdca402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.457+0000 7fdcaa7fc640 1 -- 172.21.15.40:0/367786725 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdcb001f030 con 0x7fdcb40b2340 2024-04-26T21:36:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.682+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcb4075ce0 con 0x7fdcb40b2340 2024-04-26T21:36:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.683+0000 7fdcaa7fc640 1 -- 172.21.15.40:0/367786725 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7fdcb0059b80 con 0x7fdcb40b2340 2024-04-26T21:36:12.683 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:12.683 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/780906898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":780906898},{"type":"v1","addr":"172.21.15.184:6835","nonce":780906898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3936779615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3936779615},{"type":"v1","addr":"172.21.15.40:6835","nonce":3936779615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2975364674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2975364674},{"type":"v1","addr":"172.21.15.184:6837","nonce":2975364674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:35:09.479293+0000","modified":"2024-04-26T21:35:31.566271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2862109951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2862109951},{"type":"v1","addr":"172.21.15.40:6834","nonce":2862109951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-26T21:36:12.683 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-26T21:36:12.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc9806c8f0 msgr2=0x7fdc9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc9806c8f0 0x7fdc9806edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdca4034000 tx=0x7fdca402e000 comp rx=0 tx=0).stop 2024-04-26T21:36:12.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 msgr2=0x7fdcb40c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40c68d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b3f0 tx=0x7fdcb000b8b0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 shutdown_connections 2024-04-26T21:36:12.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc9806c8f0 0x7fdc9806edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcb4162810 0x7fdcb414a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcb40cd930 0x7fdcb40ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 --2- 172.21.15.40:0/367786725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb40b2340 0x7fdcb40c68d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 >> 172.21.15.40:0/367786725 conn(0x7fdcb40d2da0 msgr2=0x7fdcb4070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:12.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 shutdown_connections 2024-04-26T21:36:12.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.685+0000 7fdcbbe6d640 1 -- 172.21.15.40:0/367786725 wait complete. 2024-04-26T21:36:12.696 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.796+0000 7f0408a47640 1 Processor -- start 2024-04-26T21:36:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.796+0000 7f0408a47640 1 -- start start 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0408a47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0404162810 0x7f0404070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0408a47640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04040d10d0 con 0x7f04040b2340 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0408a47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04040d1370 con 0x7f04040d1b30 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0408a47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04040d0750 con 0x7f0404162810 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f03fbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f04040b2340 0x7f04040cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50178/0 (socket says 172.21.15.40:50178) 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f03fbfff640 1 -- 172.21.15.40:0/292476997 learned_addr learned my addr 172.21.15.40:0/292476997 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f040259c640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0404162810 0x7f0404070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2751020422 0 0) 0x7f04040d10d0 con 0x7f04040b2340 2024-04-26T21:36:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03f0003520 con 0x7f04040b2340 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1662465251 0 0) 0x7f03f0003520 con 0x7f04040b2340 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0404162810 msgr2=0x7f0404070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0404162810 0x7f0404070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 >> v1:172.21.15.184:6789/0 conn(0x7f04040d1b30 legacy=0x7f0404070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04040d01b0 con 0x7f04040b2340 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f03ec001670 con 0x7f04040b2340 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.797+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 >> v1:172.21.15.40:6789/0 conn(0x7f04040b2340 legacy=0x7f04040cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f0401d9b640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f03f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f0401d9b640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0404162810 0x7f03f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f0401d9b640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f0004d40 0x7f03f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04040d0750 con 0x7f03f00039e0 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03f000ba70 con 0x7f03f0004d40 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03f000bc70 con 0x7f0404162810 2024-04-26T21:36:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f040259c640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f03f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f03fbfff640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0404162810 0x7f03f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f0402d9d640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f0004d40 0x7f03f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f040259c640 1 -- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0404162810 msgr2=0x7f03f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f040259c640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0404162810 0x7f03f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f040259c640 1 -- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f0004d40 msgr2=0x7f03f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f040259c640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f0004d40 0x7f03f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f040259c640 1 -- 172.21.15.40:0/292476997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f0003520 con 0x7f03f00039e0 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.798+0000 7f0402d9d640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f0004d40 0x7f03f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f040259c640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f03f0003e00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009a90 tx=0x7f03f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b226b7df4f4e9e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4036070 con 0x7f03f00039e0 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4031400 con 0x7f03f00039e0 2024-04-26T21:36:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0401d9b640 1 -- 172.21.15.40:0/292476997 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4009690 con 0x7f03f00039e0 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0408a47640 1 -- 172.21.15.40:0/292476997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 msgr2=0x7f03f0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0408a47640 1 --2- 172.21.15.40:0/292476997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f03f0003e00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009a90 tx=0x7f03f0005830 comp rx=0 tx=0).stop 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0402d9d640 1 -- 172.21.15.40:0/292476997 reap_dead start 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0408a47640 1 -- 172.21.15.40:0/292476997 shutdown_connections 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0408a47640 1 -- 172.21.15.40:0/292476997 >> 172.21.15.40:0/292476997 conn(0x7f04040d2da0 msgr2=0x7f04040d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0408a47640 1 -- 172.21.15.40:0/292476997 shutdown_connections 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.799+0000 7f0408a47640 1 -- 172.21.15.40:0/292476997 wait complete. 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.801+0000 7f0408a47640 1 Processor -- start 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.801+0000 7f0408a47640 1 -- start start 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.801+0000 7f0408a47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f04040cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f0408a47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04040ca030 0x7f040415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f0408a47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04040ce7d0 0x7f04040cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f0408a47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04040cdb00 con 0x7f03f00039e0 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f0408a47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04040d16a0 con 0x7f04040ca030 2024-04-26T21:36:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f0408a47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0404071260 con 0x7f04040ce7d0 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f04040cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f04040cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58358/0 (socket says 172.21.15.40:58358) 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 -- 172.21.15.40:0/2236840706 learned_addr learned my addr 172.21.15.40:0/2236840706 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f03fbfff640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04040ca030 0x7f040415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f0402d9d640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04040ce7d0 0x7f04040cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 -- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04040ce7d0 msgr2=0x7f04040cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04040ce7d0 0x7f04040cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 -- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04040ca030 msgr2=0x7f040415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04040ca030 0x7f040415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 -- 172.21.15.40:0/2236840706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04040cce30 con 0x7f03f00039e0 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f040259c640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f04040cc750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f03f4034040 tx=0x7f03f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f03faffd640 1 -- 172.21.15.40:0/2236840706 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4040020 con 0x7f03f00039e0 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f03faffd640 1 -- 172.21.15.40:0/2236840706 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4003bc0 con 0x7f03f00039e0 2024-04-26T21:36:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04040c7c50 con 0x7f03f00039e0 2024-04-26T21:36:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f03faffd640 1 -- 172.21.15.40:0/2236840706 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4036040 con 0x7f03f00039e0 2024-04-26T21:36:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.802+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04040c8210 con 0x7f03f00039e0 2024-04-26T21:36:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.803+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04040cf190 con 0x7f03f00039e0 2024-04-26T21:36:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.804+0000 7f03faffd640 1 -- 172.21.15.40:0/2236840706 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03f4043020 con 0x7f03f00039e0 2024-04-26T21:36:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.804+0000 7f03faffd640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03e406c920 0x7f03e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.804+0000 7f03faffd640 1 -- 172.21.15.40:0/2236840706 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f03f40ac010 con 0x7f03f00039e0 2024-04-26T21:36:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.804+0000 7f03fbfff640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03e406c920 0x7f03e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.805+0000 7f03fbfff640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03e406c920 0x7f03e406ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0404072330 tx=0x7f03ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.808+0000 7f03faffd640 1 -- 172.21.15.40:0/2236840706 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03f404a070 con 0x7f03f00039e0 2024-04-26T21:36:12.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.997+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04040c9c20 con 0x7f03f00039e0 2024-04-26T21:36:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:12.998+0000 7f03faffd640 1 -- 172.21.15.40:0/2236840706 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18867 (secure 0 0 0) 0x7f04040c9c20 con 0x7f03f00039e0 2024-04-26T21:36:13.002 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:13.002 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:35:51.251866+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:35:08.071299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:35:08.422512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:35:12.131050+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03e406c920 msgr2=0x7f03e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03e406c920 0x7f03e406ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0404072330 tx=0x7f03ec009040 comp rx=0 tx=0).stop 2024-04-26T21:36:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 msgr2=0x7f04040cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f04040cc750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f03f4034040 tx=0x7f03f4004730 comp rx=0 tx=0).stop 2024-04-26T21:36:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 shutdown_connections 2024-04-26T21:36:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03e406c920 0x7f03e406ede0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04040ce7d0 0x7f04040cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04040ca030 0x7f040415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 --2- 172.21.15.40:0/2236840706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f00039e0 0x7f04040cc750 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 >> 172.21.15.40:0/2236840706 conn(0x7f04040d2da0 msgr2=0x7f04040d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 shutdown_connections 2024-04-26T21:36:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.000+0000 7f0408a47640 1 -- 172.21.15.40:0/2236840706 wait complete. 2024-04-26T21:36:13.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:36:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.114+0000 7fac1deea640 1 Processor -- start 2024-04-26T21:36:13.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.114+0000 7fac1deea640 1 -- start start 2024-04-26T21:36:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac1deea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fac18162880 0x7fac180d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac1deea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac180b5f20 0x7fac180d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac1deea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac180b7e50 con 0x7fac180d39f0 2024-04-26T21:36:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac1deea640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fac180d0580 con 0x7fac18162880 2024-04-26T21:36:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac1deea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac180d1800 con 0x7fac180b5f20 2024-04-26T21:36:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac17fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fac180d39f0 0x7fac180b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50186/0 (socket says 172.21.15.40:50186) 2024-04-26T21:36:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac17fff640 1 -- 172.21.15.40:0/758713652 learned_addr learned my addr 172.21.15.40:0/758713652 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac177fe640 1 --2- 172.21.15.40:0/758713652 >> v2:172.21.15.184:3300/0 conn(0x7fac18162880 0x7fac180d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac16ffd640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac180b5f20 0x7fac180d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2710333087 0 0) 0x7fac180b7e50 con 0x7fac180d39f0 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.115+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac00003520 con 0x7fac180d39f0 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3668280320 0 0) 0x7fac00003520 con 0x7fac180d39f0 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac180b5f20 msgr2=0x7fac180d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac180b5f20 0x7fac180d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 >> v2:172.21.15.184:3300/0 conn(0x7fac18162880 msgr2=0x7fac180d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 --2- 172.21.15.40:0/758713652 >> v2:172.21.15.184:3300/0 conn(0x7fac18162880 0x7fac180d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac180d07d0 con 0x7fac180d39f0 2024-04-26T21:36:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fac100019f0 con 0x7fac180d39f0 2024-04-26T21:36:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac177fe640 1 --2- 172.21.15.40:0/758713652 >> v2:172.21.15.184:3300/0 conn(0x7fac18162880 0x7fac180d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 >> v1:172.21.15.40:6789/0 conn(0x7fac180d39f0 legacy=0x7fac180b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac000039b0 0x7fac00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac180b5f20 0x7fac00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac00004d40 0x7fac000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fac180d0580 con 0x7fac00004d40 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac180d1800 con 0x7fac000039b0 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac0000b9c0 con 0x7fac180b5f20 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac17fff640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac00004d40 0x7fac000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac177fe640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac000039b0 0x7fac00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac16ffd640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac180b5f20 0x7fac00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac17fff640 1 -- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac180b5f20 msgr2=0x7fac00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac17fff640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac180b5f20 0x7fac00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac17fff640 1 -- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac000039b0 msgr2=0x7fac00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac17fff640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac000039b0 0x7fac00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac17fff640 1 -- 172.21.15.40:0/758713652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac00003520 con 0x7fac00004d40 2024-04-26T21:36:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.116+0000 7fac177fe640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac000039b0 0x7fac00003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.117+0000 7fac17fff640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac00004d40 0x7fac000051c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fac1000a8a0 tx=0x7fac00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4304c072ce14b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.117+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1000a4c0 con 0x7fac00004d40 2024-04-26T21:36:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.117+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1002f070 con 0x7fac00004d40 2024-04-26T21:36:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.117+0000 7fac167fc640 1 -- 172.21.15.40:0/758713652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1002bd30 con 0x7fac00004d40 2024-04-26T21:36:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.118+0000 7fac1deea640 1 -- 172.21.15.40:0/758713652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac00004d40 msgr2=0x7fac000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.118+0000 7fac1deea640 1 --2- 172.21.15.40:0/758713652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac00004d40 0x7fac000051c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fac1000a8a0 tx=0x7fac00005830 comp rx=0 tx=0).stop 2024-04-26T21:36:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.118+0000 7fac17fff640 1 -- 172.21.15.40:0/758713652 reap_dead start 2024-04-26T21:36:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.118+0000 7fac1deea640 1 -- 172.21.15.40:0/758713652 shutdown_connections 2024-04-26T21:36:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.118+0000 7fac1deea640 1 -- 172.21.15.40:0/758713652 >> 172.21.15.40:0/758713652 conn(0x7fac180d4f60 msgr2=0x7fac180d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.118+0000 7fac1deea640 1 -- 172.21.15.40:0/758713652 shutdown_connections 2024-04-26T21:36:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.118+0000 7fac1deea640 1 -- 172.21.15.40:0/758713652 wait complete. 2024-04-26T21:36:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.119+0000 7fac1deea640 1 Processor -- start 2024-04-26T21:36:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.119+0000 7fac1deea640 1 -- start start 2024-04-26T21:36:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac1deea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac18161760 0x7fac180ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac1deea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac18161b40 0x7fac1815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac1deea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac1815fde0 0x7fac181f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac1deea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fac18162c60 con 0x7fac1815fde0 2024-04-26T21:36:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac1deea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac180d0d90 con 0x7fac18161760 2024-04-26T21:36:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac1deea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac18162de0 con 0x7fac18161b40 2024-04-26T21:36:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac1815fde0 0x7fac181f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac1815fde0 0x7fac181f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58372/0 (socket says 172.21.15.40:58372) 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 -- 172.21.15.40:0/357200655 learned_addr learned my addr 172.21.15.40:0/357200655 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac177fe640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac18161760 0x7fac180ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac16ffd640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac18161b40 0x7fac1815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 -- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac18161b40 msgr2=0x7fac1815f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac18161b40 0x7fac1815f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 -- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac18161760 msgr2=0x7fac180ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac18161760 0x7fac180ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 -- 172.21.15.40:0/357200655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac180cd530 con 0x7fac1815fde0 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.120+0000 7fac17fff640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac1815fde0 0x7fac181f40e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fac1000a8a0 tx=0x7fac10001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.121+0000 7fabfffff640 1 -- 172.21.15.40:0/357200655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10045070 con 0x7fac1815fde0 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.121+0000 7fabfffff640 1 -- 172.21.15.40:0/357200655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac10040480 con 0x7fac1815fde0 2024-04-26T21:36:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.121+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac180b1ec0 con 0x7fac1815fde0 2024-04-26T21:36:13.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.121+0000 7fabfffff640 1 -- 172.21.15.40:0/357200655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08009860 con 0x7fac1815fde0 2024-04-26T21:36:13.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.121+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac180b2480 con 0x7fac1815fde0 2024-04-26T21:36:13.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.122+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac180694f0 con 0x7fac1815fde0 2024-04-26T21:36:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.122+0000 7fabfffff640 1 -- 172.21.15.40:0/357200655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac10048070 con 0x7fac1815fde0 2024-04-26T21:36:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.123+0000 7fabfffff640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac0406c8f0 0x7fac0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.123+0000 7fabfffff640 1 -- 172.21.15.40:0/357200655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fac100abec0 con 0x7fac1815fde0 2024-04-26T21:36:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.123+0000 7fac177fe640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac0406c8f0 0x7fac0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.124+0000 7fac177fe640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac0406c8f0 0x7fac0406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac08009ef0 tx=0x7fac08009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.128+0000 7fabfffff640 1 -- 172.21.15.40:0/357200655 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac10079a80 con 0x7fac1815fde0 2024-04-26T21:36:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.352+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fac180d5680 con 0x7fac1815fde0 2024-04-26T21:36:13.353 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:36:13.352+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:36:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.354+0000 7fabfffff640 1 -- 172.21.15.40:0/357200655 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fac1007dd80 con 0x7fac1815fde0 2024-04-26T21:36:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:36:13.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.357+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac0406c8f0 msgr2=0x7fac0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.357+0000 7fac1deea640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac0406c8f0 0x7fac0406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac08009ef0 tx=0x7fac08009190 comp rx=0 tx=0).stop 2024-04-26T21:36:13.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.357+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac1815fde0 msgr2=0x7fac181f40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.357+0000 7fac1deea640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac1815fde0 0x7fac181f40e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fac1000a8a0 tx=0x7fac10001e30 comp rx=0 tx=0).stop 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.358+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 shutdown_connections 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.358+0000 7fac1deea640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac0406c8f0 0x7fac0406edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.358+0000 7fac1deea640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac1815fde0 0x7fac181f40e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.358+0000 7fac1deea640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac18161b40 0x7fac1815f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.358+0000 7fac1deea640 1 --2- 172.21.15.40:0/357200655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac18161760 0x7fac180ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.358+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 >> 172.21.15.40:0/357200655 conn(0x7fac180d4f60 msgr2=0x7fac180ba010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.358+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 shutdown_connections 2024-04-26T21:36:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.358+0000 7fac1deea640 1 -- 172.21.15.40:0/357200655 wait complete. 2024-04-26T21:36:13.360 INFO:tasks.ceph.mds.a.smithi040.stderr: -5467> 2024-04-26T21:35:14.564+0000 7f66caa24640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:36:13.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:36:13.391 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:36:13.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.472+0000 7f20d36c4640 1 Processor -- start 2024-04-26T21:36:13.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.473+0000 7f20d36c4640 1 -- start start 2024-04-26T21:36:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.473+0000 7f20d36c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f20cc15f5a0 0x7f20cc0d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.473+0000 7f20d36c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0d2830 0x7f20cc162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.473+0000 7f20d36c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20cc0d1a70 con 0x7f20cc0d3c20 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.473+0000 7f20d36c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f20cc163d30 con 0x7f20cc15f5a0 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.473+0000 7f20d36c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20cc0d1d50 con 0x7f20cc0d2830 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.473+0000 7f20d1c61640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f20cc0d3c20 0x7f20cc15f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50204/0 (socket says 172.21.15.40:50204) 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.473+0000 7f20d1c61640 1 -- 172.21.15.40:0/3138604201 learned_addr learned my addr 172.21.15.40:0/3138604201 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20d0c5f640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0d2830 0x7f20cc162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20d1460640 1 --2- 172.21.15.40:0/3138604201 >> v2:172.21.15.184:3300/0 conn(0x7f20cc15f5a0 0x7f20cc0d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979231184 0 0) 0x7f20cc0d1a70 con 0x7f20cc0d3c20 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20b8003520 con 0x7f20cc0d3c20 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 147363487 0 0) 0x7f20b8003520 con 0x7f20cc0d3c20 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0d2830 msgr2=0x7f20cc162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0d2830 0x7f20cc162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 >> v2:172.21.15.184:3300/0 conn(0x7f20cc15f5a0 msgr2=0x7f20cc0d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 --2- 172.21.15.40:0/3138604201 >> v2:172.21.15.184:3300/0 conn(0x7f20cc15f5a0 0x7f20cc0d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc0d2030 con 0x7f20cc0d3c20 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f20c40019f0 con 0x7f20cc0d3c20 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 >> v1:172.21.15.40:6789/0 conn(0x7f20cc0d3c20 legacy=0x7f20cc15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0d2830 0x7f20b80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8003f30 0x7f20b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8004d40 0x7f20b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0d1d50 con 0x7f20b8003f30 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f20b800b9c0 con 0x7f20b8004d40 2024-04-26T21:36:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20b800bb90 con 0x7f20cc0d2830 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20d0c5f640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8003f30 0x7f20b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20d1460640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0d2830 0x7f20b80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.474+0000 7f20d1c61640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8004d40 0x7f20b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20d0c5f640 1 -- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0d2830 msgr2=0x7f20b80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20d0c5f640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0d2830 0x7f20b80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20d0c5f640 1 -- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8004d40 msgr2=0x7f20b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20d0c5f640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8004d40 0x7f20b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20d0c5f640 1 -- 172.21.15.40:0/3138604201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b8003520 con 0x7f20b8003f30 2024-04-26T21:36:13.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20d1c61640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8004d40 0x7f20b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20d0c5f640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8003f30 0x7f20b8004800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009ac0 tx=0x7f20b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2956c01cd762cb70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8036070 con 0x7f20b8003f30 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c8031400 con 0x7f20b8003f30 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.475+0000 7f20bffff640 1 -- 172.21.15.40:0/3138604201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8009690 con 0x7f20b8003f30 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.476+0000 7f20d36c4640 1 -- 172.21.15.40:0/3138604201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8003f30 msgr2=0x7f20b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.476+0000 7f20d36c4640 1 --2- 172.21.15.40:0/3138604201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b8003f30 0x7f20b8004800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009ac0 tx=0x7f20b8005830 comp rx=0 tx=0).stop 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.476+0000 7f20d1c61640 1 -- 172.21.15.40:0/3138604201 reap_dead start 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.476+0000 7f20d36c4640 1 -- 172.21.15.40:0/3138604201 shutdown_connections 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.476+0000 7f20d36c4640 1 -- 172.21.15.40:0/3138604201 >> 172.21.15.40:0/3138604201 conn(0x7f20cc0d5190 msgr2=0x7f20cc0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.476+0000 7f20d36c4640 1 -- 172.21.15.40:0/3138604201 shutdown_connections 2024-04-26T21:36:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.476+0000 7f20d36c4640 1 -- 172.21.15.40:0/3138604201 wait complete. 2024-04-26T21:36:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.477+0000 7f20d36c4640 1 Processor -- start 2024-04-26T21:36:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.477+0000 7f20d36c4640 1 -- start start 2024-04-26T21:36:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d36c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8003f30 0x7f20cc0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d36c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20cc15e490 0x7f20cc15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d36c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0b9f30 0x7f20cc0ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d36c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1600a0 con 0x7f20cc15e490 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d36c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0d4190 con 0x7f20b8003f30 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d36c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20cc160220 con 0x7f20cc0b9f30 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20cc15e490 0x7f20cc15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20cc15e490 0x7f20cc15b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58430/0 (socket says 172.21.15.40:58430) 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 -- 172.21.15.40:0/585133164 learned_addr learned my addr 172.21.15.40:0/585133164 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 -- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0b9f30 msgr2=0x7f20cc0ba3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0b9f30 0x7f20cc0ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 -- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8003f30 msgr2=0x7f20cc0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d1460640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8003f30 0x7f20cc0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8003f30 0x7f20cc0cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 -- 172.21.15.40:0/585133164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc1e98b0 con 0x7f20cc15e490 2024-04-26T21:36:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.478+0000 7f20d0c5f640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20cc15e490 0x7f20cc15b2d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009ac0 tx=0x7f20c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.479+0000 7f20bdffb640 1 -- 172.21.15.40:0/585133164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8040020 con 0x7f20cc15e490 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.479+0000 7f20bdffb640 1 -- 172.21.15.40:0/585133164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c803b660 con 0x7f20cc15e490 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.479+0000 7f20bdffb640 1 -- 172.21.15.40:0/585133164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8036040 con 0x7f20cc15e490 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.479+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc1e9b90 con 0x7f20cc15e490 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.479+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc1ea150 con 0x7f20cc15e490 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.480+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20cc0cc130 con 0x7f20cc15e490 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.481+0000 7f20bdffb640 1 -- 172.21.15.40:0/585133164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20c8043020 con 0x7f20cc15e490 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.481+0000 7f20bdffb640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20b006c920 0x7f20b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.481+0000 7f20bdffb640 1 -- 172.21.15.40:0/585133164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f20c80ac590 con 0x7f20cc15e490 2024-04-26T21:36:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.481+0000 7f20d1460640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20b006c920 0x7f20b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:13.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.482+0000 7f20d1460640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20b006c920 0x7f20b006ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009a60 tx=0x7f20c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:13.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.486+0000 7f20bdffb640 1 -- 172.21.15.40:0/585133164 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20c804a030 con 0x7f20cc15e490 2024-04-26T21:36:13.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:13.709+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f20cc1ea530 con 0x7f20cc15e490 2024-04-26T21:36:14.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.361+0000 7f20bdffb640 1 -- 172.21.15.40:0/585133164 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f20cc1ea530 con 0x7f20cc15e490 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.364+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20b006c920 msgr2=0x7f20b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.364+0000 7f20d36c4640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20b006c920 0x7f20b006ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009a60 tx=0x7f20c0008040 comp rx=0 tx=0).stop 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.364+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20cc15e490 msgr2=0x7f20cc15b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.364+0000 7f20d36c4640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20cc15e490 0x7f20cc15b2d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009ac0 tx=0x7f20c8004730 comp rx=0 tx=0).stop 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.365+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 shutdown_connections 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.365+0000 7f20d36c4640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f20b006c920 0x7f20b006ede0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.365+0000 7f20d36c4640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20cc0b9f30 0x7f20cc0ba3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.365+0000 7f20d36c4640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20cc15e490 0x7f20cc15b2d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.365+0000 7f20d36c4640 1 --2- 172.21.15.40:0/585133164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20b8003f30 0x7f20cc0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.365+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 >> 172.21.15.40:0/585133164 conn(0x7f20cc0d5190 msgr2=0x7f20cc15f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.365+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 shutdown_connections 2024-04-26T21:36:14.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.365+0000 7f20d36c4640 1 -- 172.21.15.40:0/585133164 wait complete. 2024-04-26T21:36:14.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:36:14.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.533+0000 7ff7eca59640 1 Processor -- start 2024-04-26T21:36:14.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.533+0000 7ff7eca59640 1 -- start start 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7eca59640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7e81629a0 0x7ff7e80b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7eca59640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7e80b5fd0 0x7ff7e80d5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7eca59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80d1950 0x7ff7e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7eca59640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e8162d80 con 0x7ff7e80b5fd0 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7eca59640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e80d39d0 con 0x7ff7e81629a0 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7eca59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e80d3cb0 con 0x7ff7e80d1950 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e5d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7e80b5fd0 0x7ff7e80d5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e5d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7e80b5fd0 0x7ff7e80d5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58438/0 (socket says 172.21.15.40:58438) 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e5d9b640 1 -- 172.21.15.40:0/3436868625 learned_addr learned my addr 172.21.15.40:0/3436868625 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e5d9b640 1 -- 172.21.15.40:0/3436868625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80d1950 msgr2=0x7ff7e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e6d9d640 1 --2- 172.21.15.40:0/3436868625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80d1950 0x7ff7e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e659c640 1 --2- 172.21.15.40:0/3436868625 >> v2:172.21.15.184:3300/0 conn(0x7ff7e81629a0 0x7ff7e80b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e5d9b640 1 --2- 172.21.15.40:0/3436868625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80d1950 0x7ff7e80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e5d9b640 1 -- 172.21.15.40:0/3436868625 >> v2:172.21.15.184:3300/0 conn(0x7ff7e81629a0 msgr2=0x7ff7e80b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e5d9b640 1 --2- 172.21.15.40:0/3436868625 >> v2:172.21.15.184:3300/0 conn(0x7ff7e81629a0 0x7ff7e80b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e5d9b640 1 -- 172.21.15.40:0/3436868625 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e80cfda0 con 0x7ff7e80b5fd0 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e659c640 1 --2- 172.21.15.40:0/3436868625 >> v2:172.21.15.184:3300/0 conn(0x7ff7e81629a0 0x7ff7e80b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.534+0000 7ff7e6d9d640 1 --2- 172.21.15.40:0/3436868625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80d1950 0x7ff7e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.535+0000 7ff7e5d9b640 1 --2- 172.21.15.40:0/3436868625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80b5fd0 0x7ff7e80d5820 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff7e808ef60 tx=0x7ff7dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da31968872631c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.535+0000 7ff7e559a640 1 -- 172.21.15.40:0/3436868625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc037070 con 0x7ff7e80b5fd0 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.535+0000 7ff7e559a640 1 -- 172.21.15.40:0/3436868625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7dc032440 con 0x7ff7e80b5fd0 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.535+0000 7ff7e559a640 1 -- 172.21.15.40:0/3436868625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc032810 con 0x7ff7e80b5fd0 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 -- 172.21.15.40:0/3436868625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80b5fd0 msgr2=0x7ff7e80d5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3436868625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80b5fd0 0x7ff7e80d5820 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff7e808ef60 tx=0x7ff7dc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 -- 172.21.15.40:0/3436868625 shutdown_connections 2024-04-26T21:36:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3436868625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80d1950 0x7ff7e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3436868625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80b5fd0 0x7ff7e80d5820 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3436868625 >> v2:172.21.15.184:3300/0 conn(0x7ff7e81629a0 0x7ff7e80b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 -- 172.21.15.40:0/3436868625 >> 172.21.15.40:0/3436868625 conn(0x7ff7e809e3e0 msgr2=0x7ff7e80d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 -- 172.21.15.40:0/3436868625 shutdown_connections 2024-04-26T21:36:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.536+0000 7ff7eca59640 1 -- 172.21.15.40:0/3436868625 wait complete. 2024-04-26T21:36:14.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.537+0000 7ff7eca59640 1 Processor -- start 2024-04-26T21:36:14.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.537+0000 7ff7eca59640 1 -- start start 2024-04-26T21:36:14.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7eca59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80b5fd0 0x7ff7e814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7eca59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80d1950 0x7ff7e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7eca59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e81629a0 0x7ff7e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7eca59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e80d5d60 con 0x7ff7e80d1950 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7eca59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e80d3f90 con 0x7ff7e81629a0 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7eca59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e80d5ee0 con 0x7ff7e80b5fd0 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7e5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80d1950 0x7ff7e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7e5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80d1950 0x7ff7e806d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58452/0 (socket says 172.21.15.40:58452) 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7e5d9b640 1 -- 172.21.15.40:0/3629855652 learned_addr learned my addr 172.21.15.40:0/3629855652 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7e6d9d640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e81629a0 0x7ff7e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.538+0000 7ff7e659c640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80b5fd0 0x7ff7e814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7e5d9b640 1 -- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80b5fd0 msgr2=0x7ff7e814bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7e5d9b640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80b5fd0 0x7ff7e814bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7e5d9b640 1 -- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e81629a0 msgr2=0x7ff7e806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7e5d9b640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e81629a0 0x7ff7e806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7e5d9b640 1 -- 172.21.15.40:0/3629855652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e815b9a0 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7e5d9b640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80d1950 0x7ff7e806d450 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc02a220 tx=0x7ff7dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7d6ffd640 1 -- 172.21.15.40:0/3629855652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc037040 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7d6ffd640 1 -- 172.21.15.40:0/3629855652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7dc042070 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e80c9850 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7d6ffd640 1 -- 172.21.15.40:0/3629855652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc030bb0 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.539+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e814e790 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.540+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e8069600 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.541+0000 7ff7d6ffd640 1 -- 172.21.15.40:0/3629855652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7dc047460 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.541+0000 7ff7d6ffd640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7c806c920 0x7ff7c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.541+0000 7ff7d6ffd640 1 -- 172.21.15.40:0/3629855652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7ff7dc0ac9f0 con 0x7ff7e80d1950 2024-04-26T21:36:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.541+0000 7ff7e659c640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7c806c920 0x7ff7c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:14.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.542+0000 7ff7e659c640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7c806c920 0x7ff7c806ede0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000ab60 tx=0x7ff7e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.546+0000 7ff7d6ffd640 1 -- 172.21.15.40:0/3629855652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7dc07a400 con 0x7ff7e80d1950 2024-04-26T21:36:14.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:14.735+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff7e80cf4d0 con 0x7ff7e80d1950 2024-04-26T21:36:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.365+0000 7ff7d6ffd640 1 -- 172.21.15.40:0/3629855652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7ff7dc07e700 con 0x7ff7e80d1950 2024-04-26T21:36:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:36:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.368+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7c806c920 msgr2=0x7ff7c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.368+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7c806c920 0x7ff7c806ede0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000ab60 tx=0x7ff7e0008040 comp rx=0 tx=0).stop 2024-04-26T21:36:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.368+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80d1950 msgr2=0x7ff7e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.368+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80d1950 0x7ff7e806d450 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc02a220 tx=0x7ff7dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.369+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 shutdown_connections 2024-04-26T21:36:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.369+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7c806c920 0x7ff7c806ede0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.369+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e81629a0 0x7ff7e806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.369+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e80d1950 0x7ff7e806d450 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.369+0000 7ff7eca59640 1 --2- 172.21.15.40:0/3629855652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e80b5fd0 0x7ff7e814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.369+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 >> 172.21.15.40:0/3629855652 conn(0x7ff7e809e3e0 msgr2=0x7ff7e80b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.369+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 shutdown_connections 2024-04-26T21:36:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.369+0000 7ff7eca59640 1 -- 172.21.15.40:0/3629855652 wait complete. 2024-04-26T21:36:15.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:36:15.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.493+0000 7fcbdb9ec640 1 Processor -- start 2024-04-26T21:36:15.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.494+0000 7fcbdb9ec640 1 -- start start 2024-04-26T21:36:15.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.494+0000 7fcbdb9ec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcbd41489a0 0x7fcbd40710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:15.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.494+0000 7fcbdb9ec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcbd406c950 0x7fcbd40b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:15.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.494+0000 7fcbdb9ec640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd4148d80 con 0x7fcbd40b04b0 2024-04-26T21:36:15.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.494+0000 7fcbdb9ec640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd40b11d0 con 0x7fcbd41489a0 2024-04-26T21:36:15.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.494+0000 7fcbdb9ec640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd40b0f10 con 0x7fcbd406c950 2024-04-26T21:36:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.495+0000 7fcbd9788640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcbd41489a0 0x7fcbd40710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.495+0000 7fcbd9788640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcbd41489a0 0x7fcbd40710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58462/0 (socket says 172.21.15.40:58462) 2024-04-26T21:36:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.495+0000 7fcbd9788640 1 -- 172.21.15.40:0/1408502396 learned_addr learned my addr 172.21.15.40:0/1408502396 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.495+0000 7fcbd9788640 1 -- 172.21.15.40:0/1408502396 >> v2:172.21.15.184:3300/0 conn(0x7fcbd406c950 msgr2=0x7fcbd40b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.495+0000 7fcbd9f89640 1 --2- 172.21.15.40:0/1408502396 >> v2:172.21.15.184:3300/0 conn(0x7fcbd406c950 0x7fcbd40b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.495+0000 7fcbd9788640 1 --2- 172.21.15.40:0/1408502396 >> v2:172.21.15.184:3300/0 conn(0x7fcbd406c950 0x7fcbd40b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.495+0000 7fcbd9788640 1 -- 172.21.15.40:0/1408502396 >> v1:172.21.15.184:6789/0 conn(0x7fcbd40b04b0 legacy=0x7fcbd406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.495+0000 7fcbd9788640 1 -- 172.21.15.40:0/1408502396 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4070600 con 0x7fcbd41489a0 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbd9788640 1 --2- 172.21.15.40:0/1408502396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd41489a0 0x7fcbd40710e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40b1350 tx=0x7fcbd002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeafbfe06e13547 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbcbfff640 1 -- 172.21.15.40:0/1408502396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0037070 con 0x7fcbd41489a0 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbcbfff640 1 -- 172.21.15.40:0/1408502396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd0032440 con 0x7fcbd41489a0 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbcbfff640 1 -- 172.21.15.40:0/1408502396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0032810 con 0x7fcbd41489a0 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/1408502396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd41489a0 msgr2=0x7fcbd40710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/1408502396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd41489a0 0x7fcbd40710e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40b1350 tx=0x7fcbd002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/1408502396 shutdown_connections 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/1408502396 >> v2:172.21.15.184:3300/0 conn(0x7fcbd406c950 0x7fcbd40b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/1408502396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd41489a0 0x7fcbd40710e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.496+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/1408502396 >> 172.21.15.40:0/1408502396 conn(0x7fcbd40b3be0 msgr2=0x7fcbd40b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:15.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.497+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/1408502396 shutdown_connections 2024-04-26T21:36:15.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.497+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/1408502396 wait complete. 2024-04-26T21:36:15.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.498+0000 7fcbdb9ec640 1 Processor -- start 2024-04-26T21:36:15.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.498+0000 7fcbdb9ec640 1 -- start start 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbdb9ec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbd406c950 0x7fcbd4142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbdb9ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd40b04b0 0x7fcbd40b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbdb9ec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbd41489a0 0x7fcbd415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbdb9ec640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd4071620 con 0x7fcbd40b04b0 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbdb9ec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd4070bc0 con 0x7fcbd41489a0 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbdb9ec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd41604d0 con 0x7fcbd406c950 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd40b04b0 0x7fcbd40b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd40b04b0 0x7fcbd40b6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58474/0 (socket says 172.21.15.40:58474) 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 -- 172.21.15.40:0/4180291389 learned_addr learned my addr 172.21.15.40:0/4180291389 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd9788640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbd406c950 0x7fcbd4142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:15.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd9f89640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbd41489a0 0x7fcbd415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 -- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbd406c950 msgr2=0x7fcbd4142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbd406c950 0x7fcbd4142880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 -- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbd41489a0 msgr2=0x7fcbd415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbd41489a0 0x7fcbd415e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 -- 172.21.15.40:0/4180291389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4141fe0 con 0x7fcbd40b04b0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbd8f87640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd40b04b0 0x7fcbd40b6e20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c460 tx=0x7fcbc400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbc9ffb640 1 -- 172.21.15.40:0/4180291389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4016020 con 0x7fcbd40b04b0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbc9ffb640 1 -- 172.21.15.40:0/4180291389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc4012870 con 0x7fcbd40b04b0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd41422c0 con 0x7fcbd40b04b0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.499+0000 7fcbc9ffb640 1 -- 172.21.15.40:0/4180291389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4012c60 con 0x7fcbd40b04b0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.500+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd40cead0 con 0x7fcbd40b04b0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.501+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd4069600 con 0x7fcbd40b04b0 2024-04-26T21:36:15.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.501+0000 7fcbc9ffb640 1 -- 172.21.15.40:0/4180291389 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbc4019020 con 0x7fcbd40b04b0 2024-04-26T21:36:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.501+0000 7fcbc9ffb640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbb806c8f0 0x7fcbb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.501+0000 7fcbc9ffb640 1 -- 172.21.15.40:0/4180291389 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7fcbc408baa0 con 0x7fcbd40b04b0 2024-04-26T21:36:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.502+0000 7fcbd9788640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbb806c8f0 0x7fcbb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:15.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.502+0000 7fcbd9788640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbb806c8f0 0x7fcbb806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0034000 tx=0x7fcbd002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:15.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.506+0000 7fcbc9ffb640 1 -- 172.21.15.40:0/4180291389 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcbc4020030 con 0x7fcbd40b04b0 2024-04-26T21:36:15.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:15.690+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcbd40b49e0 con 0x7fcbd40b04b0 2024-04-26T21:36:16.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.370+0000 7fcbc9ffb640 1 -- 172.21.15.40:0/4180291389 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7fcbc4059cc0 con 0x7fcbd40b04b0 2024-04-26T21:36:16.371 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:36:16.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.375+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbb806c8f0 msgr2=0x7fcbb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:16.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.375+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbb806c8f0 0x7fcbb806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0034000 tx=0x7fcbd002e000 comp rx=0 tx=0).stop 2024-04-26T21:36:16.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.375+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd40b04b0 msgr2=0x7fcbd40b6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:16.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.375+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd40b04b0 0x7fcbd40b6e20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c460 tx=0x7fcbc400c920 comp rx=0 tx=0).stop 2024-04-26T21:36:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.377+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 shutdown_connections 2024-04-26T21:36:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.377+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbb806c8f0 0x7fcbb806edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.377+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbd41489a0 0x7fcbd415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.377+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd40b04b0 0x7fcbd40b6e20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.377+0000 7fcbdb9ec640 1 --2- 172.21.15.40:0/4180291389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbd406c950 0x7fcbd4142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.377+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 >> 172.21.15.40:0/4180291389 conn(0x7fcbd40b3be0 msgr2=0x7fcbd4161650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.378+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 shutdown_connections 2024-04-26T21:36:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.378+0000 7fcbdb9ec640 1 -- 172.21.15.40:0/4180291389 wait complete. 2024-04-26T21:36:16.395 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T21:36:16.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.511+0000 7fb2a0472640 1 Processor -- start 2024-04-26T21:36:16.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.511+0000 7fb2a0472640 1 -- start start 2024-04-26T21:36:16.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.512+0000 7fb2a0472640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb29815f3e0 0x7fb2980d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:16.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.512+0000 7fb2a0472640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2980d31e0 0x7fb2980d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:16.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.512+0000 7fb2a0472640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb298160930 con 0x7fb29815f7c0 2024-04-26T21:36:16.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.512+0000 7fb2a0472640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb29815fd20 con 0x7fb2980d31e0 2024-04-26T21:36:16.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.512+0000 7fb2a0472640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb298160000 con 0x7fb29815f3e0 2024-04-26T21:36:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.513+0000 7fb29da0d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2980d31e0 0x7fb2980d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.513+0000 7fb29da0d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2980d31e0 0x7fb2980d3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58484/0 (socket says 172.21.15.40:58484) 2024-04-26T21:36:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.513+0000 7fb29da0d640 1 -- 172.21.15.40:0/3117556223 learned_addr learned my addr 172.21.15.40:0/3117556223 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.513+0000 7fb29e20e640 1 --2- 172.21.15.40:0/3117556223 >> v2:172.21.15.184:3300/0 conn(0x7fb29815f3e0 0x7fb2980d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.513+0000 7fb29da0d640 1 -- 172.21.15.40:0/3117556223 >> v2:172.21.15.184:3300/0 conn(0x7fb29815f3e0 msgr2=0x7fb2980d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.513+0000 7fb29da0d640 1 --2- 172.21.15.40:0/3117556223 >> v2:172.21.15.184:3300/0 conn(0x7fb29815f3e0 0x7fb2980d2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.513+0000 7fb29da0d640 1 -- 172.21.15.40:0/3117556223 >> v1:172.21.15.184:6789/0 conn(0x7fb29815f7c0 legacy=0x7fb2981620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.513+0000 7fb29da0d640 1 -- 172.21.15.40:0/3117556223 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2981602e0 con 0x7fb2980d31e0 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb29da0d640 1 --2- 172.21.15.40:0/3117556223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980d3690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb29808d480 tx=0x7fb28802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6b67c8b729a3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb29d20c640 1 -- 172.21.15.40:0/3117556223 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288037070 con 0x7fb2980d31e0 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb29d20c640 1 -- 172.21.15.40:0/3117556223 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb288032440 con 0x7fb2980d31e0 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb29d20c640 1 -- 172.21.15.40:0/3117556223 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288032810 con 0x7fb2980d31e0 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb2a0472640 1 -- 172.21.15.40:0/3117556223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 msgr2=0x7fb2980d3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb2a0472640 1 --2- 172.21.15.40:0/3117556223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980d3690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb29808d480 tx=0x7fb28802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb2a0472640 1 -- 172.21.15.40:0/3117556223 shutdown_connections 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb2a0472640 1 --2- 172.21.15.40:0/3117556223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980d3690 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb2a0472640 1 --2- 172.21.15.40:0/3117556223 >> v2:172.21.15.184:3300/0 conn(0x7fb29815f3e0 0x7fb2980d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb2a0472640 1 -- 172.21.15.40:0/3117556223 >> 172.21.15.40:0/3117556223 conn(0x7fb29809e3c0 msgr2=0x7fb2980d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb2a0472640 1 -- 172.21.15.40:0/3117556223 shutdown_connections 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.514+0000 7fb2a0472640 1 -- 172.21.15.40:0/3117556223 wait complete. 2024-04-26T21:36:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.516+0000 7fb2a0472640 1 Processor -- start 2024-04-26T21:36:16.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.516+0000 7fb2a0472640 1 -- start start 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.516+0000 7fb2a0472640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.516+0000 7fb2a0472640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815f3e0 0x7fb2980d2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.516+0000 7fb2a0472640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29815f7c0 0x7fb2980b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.516+0000 7fb2a0472640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980d3bd0 con 0x7fb2980d31e0 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.516+0000 7fb2a0472640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb298160d40 con 0x7fb29815f7c0 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.516+0000 7fb2a0472640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2980d3d50 con 0x7fb29815f3e0 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58490/0 (socket says 172.21.15.40:58490) 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 -- 172.21.15.40:0/636091708 learned_addr learned my addr 172.21.15.40:0/636091708 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29da0d640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815f3e0 0x7fb2980d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29ea0f640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29815f7c0 0x7fb2980b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 -- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815f3e0 msgr2=0x7fb2980d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815f3e0 0x7fb2980d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 -- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29815f7c0 msgr2=0x7fb2980b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29815f7c0 0x7fb2980b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 -- 172.21.15.40:0/636091708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2980b97c0 con 0x7fb2980d31e0 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb29e20e640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980ba580 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb29400d800 tx=0x7fb29400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb28effd640 1 -- 172.21.15.40:0/636091708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29400b6c0 con 0x7fb2980d31e0 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb28effd640 1 -- 172.21.15.40:0/636091708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb294019070 con 0x7fb2980d31e0 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2980b9aa0 con 0x7fb2980d31e0 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb28effd640 1 -- 172.21.15.40:0/636091708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29400bac0 con 0x7fb2980d31e0 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.517+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981541a0 con 0x7fb2980d31e0 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.518+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29806d510 con 0x7fb2980d31e0 2024-04-26T21:36:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.519+0000 7fb28effd640 1 -- 172.21.15.40:0/636091708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb29400bc60 con 0x7fb2980d31e0 2024-04-26T21:36:16.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.519+0000 7fb28effd640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb27c06c8f0 0x7fb27c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:16.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.519+0000 7fb28effd640 1 -- 172.21.15.40:0/636091708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7fb29408c0d0 con 0x7fb2980d31e0 2024-04-26T21:36:16.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.519+0000 7fb29da0d640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb27c06c8f0 0x7fb27c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:16.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.520+0000 7fb29da0d640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb27c06c8f0 0x7fb27c06edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb288034000 tx=0x7fb28802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:16.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.524+0000 7fb28effd640 1 -- 172.21.15.40:0/636091708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb29806d510 con 0x7fb2980d31e0 2024-04-26T21:36:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:16.709+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb298069600 con 0x7fb2980d31e0 2024-04-26T21:36:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.372+0000 7fb28effd640 1 -- 172.21.15.40:0/636091708 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7fb29405a470 con 0x7fb2980d31e0 2024-04-26T21:36:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data_ec' removed 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb27c06c8f0 msgr2=0x7fb27c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb27c06c8f0 0x7fb27c06edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb288034000 tx=0x7fb28802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 msgr2=0x7fb2980ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980ba580 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb29400d800 tx=0x7fb29400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 shutdown_connections 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb27c06c8f0 0x7fb27c06edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29815f7c0 0x7fb2980b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815f3e0 0x7fb2980d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 --2- 172.21.15.40:0/636091708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980d31e0 0x7fb2980ba580 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.374+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 >> 172.21.15.40:0/636091708 conn(0x7fb29809e3c0 msgr2=0x7fb29806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.375+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 shutdown_connections 2024-04-26T21:36:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.375+0000 7fb2a0472640 1 -- 172.21.15.40:0/636091708 wait complete. 2024-04-26T21:36:17.391 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:36:17.391 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:36:17.391 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:36:17.391 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:36:17.391 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:36:17.391 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:36:17.392 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:36:17.392 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:36:17.392 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:36:17.392 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:36:17.392 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:36:17.392 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:36:17.392 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:36:17.392 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:36:17.393 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:36:17.393 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:36:17.403 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:36:17.403 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:36:17.404 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:36:17.404 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:36:17.405 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:36:17.405 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:36:17.405 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:36:17.406 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:36:17.445 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:36:17.446 DEBUG:teuthology.parallel:result is None 2024-04-26T21:36:17.447 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:36:17.447 DEBUG:teuthology.parallel:result is None 2024-04-26T21:36:17.448 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:36:17.448 DEBUG:teuthology.parallel:result is None 2024-04-26T21:36:17.449 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:36:17.449 DEBUG:teuthology.parallel:result is None 2024-04-26T21:36:17.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:36:17.527 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:36:17.528 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:36:17.529 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:36:17.529 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:36:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.560+0000 7f4c78c15640 1 Processor -- start 2024-04-26T21:36:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.560+0000 7f4c78c15640 1 -- start start 2024-04-26T21:36:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c78c15640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7415b600 0x7f4c74071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c78c15640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c74072450 con 0x7f4c740cf5e0 2024-04-26T21:36:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c78c15640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c7415b9e0 con 0x7f4c740cf0e0 2024-04-26T21:36:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c78c15640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4c740cb9b0 con 0x7f4c7415b600 2024-04-26T21:36:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c71d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4c740cf5e0 0x7f4c740ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50206/0 (socket says 172.21.15.40:50206) 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c71d9b640 1 -- 172.21.15.40:0/3983548780 learned_addr learned my addr 172.21.15.40:0/3983548780 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c7259c640 1 --2- 172.21.15.40:0/3983548780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7415b600 0x7f4c74071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c7159a640 1 -- 172.21.15.40:0/3983548780 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929154470 0 0) 0x7f4c68001110 con 0x7f4c740cf5e0 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.561+0000 7f4c7159a640 1 -- 172.21.15.40:0/3983548780 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c5c003520 con 0x7f4c740cf5e0 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.562+0000 7f4c7159a640 1 -- 172.21.15.40:0/3983548780 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2901569109 0 0) 0x7f4c7415b9e0 con 0x7f4c740cf0e0 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.562+0000 7f4c7159a640 1 -- 172.21.15.40:0/3983548780 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c68001110 con 0x7f4c740cf0e0 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.562+0000 7f4c7259c640 1 -- 172.21.15.40:0/3983548780 >> v1:172.21.15.184:6789/0 conn(0x7f4c740cf0e0 legacy=0x7f4c7406f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.562+0000 7f4c7259c640 1 -- 172.21.15.40:0/3983548780 >> v1:172.21.15.40:6789/0 conn(0x7f4c740cf5e0 legacy=0x7f4c740ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.562+0000 7f4c7159a640 1 -- 172.21.15.40:0/3983548780 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1565623715 0 0) 0x7f4c5c003520 con 0x7f4c740cf5e0 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.562+0000 7f4c7259c640 1 -- 172.21.15.40:0/3983548780 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c740d0740 con 0x7f4c7415b600 2024-04-26T21:36:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c7259c640 1 --2- 172.21.15.40:0/3983548780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7415b600 0x7f4c74071f10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c7415bbb0 tx=0x7f4c6402b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b289d0d6260c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:17.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c7159a640 1 -- 172.21.15.40:0/3983548780 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64008640 con 0x7f4c7415b600 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c7159a640 1 -- 172.21.15.40:0/3983548780 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c64030070 con 0x7f4c7415b600 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c7159a640 1 -- 172.21.15.40:0/3983548780 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64008a00 con 0x7f4c7415b600 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c78c15640 1 -- 172.21.15.40:0/3983548780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7415b600 msgr2=0x7f4c74071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c78c15640 1 --2- 172.21.15.40:0/3983548780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7415b600 0x7f4c74071f10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c7415bbb0 tx=0x7f4c6402b810 comp rx=0 tx=0).stop 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c78c15640 1 -- 172.21.15.40:0/3983548780 shutdown_connections 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c78c15640 1 --2- 172.21.15.40:0/3983548780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7415b600 0x7f4c74071f10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c78c15640 1 -- 172.21.15.40:0/3983548780 >> 172.21.15.40:0/3983548780 conn(0x7f4c740d0be0 msgr2=0x7f4c740d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c78c15640 1 -- 172.21.15.40:0/3983548780 shutdown_connections 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.563+0000 7f4c78c15640 1 -- 172.21.15.40:0/3983548780 wait complete. 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.565+0000 7f4c78c15640 1 Processor -- start 2024-04-26T21:36:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.565+0000 7f4c78c15640 1 -- start start 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.565+0000 7f4c78c15640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c740cf0e0 0x7f4c740c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.565+0000 7f4c78c15640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c740cf5e0 0x7f4c740b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.565+0000 7f4c78c15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c7415b600 0x7f4c740b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.565+0000 7f4c78c15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4c740d02f0 con 0x7f4c7415b600 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.565+0000 7f4c78c15640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4c740cb640 con 0x7f4c740cf0e0 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.565+0000 7f4c78c15640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4c740d0470 con 0x7f4c740cf5e0 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c7415b600 0x7f4c740b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c7259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c740cf0e0 0x7f4c740c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c7415b600 0x7f4c740b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58542/0 (socket says 172.21.15.40:58542) 2024-04-26T21:36:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 -- 172.21.15.40:0/2415308291 learned_addr learned my addr 172.21.15.40:0/2415308291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c71d9b640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c740cf5e0 0x7f4c740b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 -- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c740cf5e0 msgr2=0x7f4c740b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c740cf5e0 0x7f4c740b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 -- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c740cf0e0 msgr2=0x7f4c740c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c740cf0e0 0x7f4c740c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 -- 172.21.15.40:0/2415308291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c74150290 con 0x7f4c7415b600 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c72d9d640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c7415b600 0x7f4c740b1e20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00c340 tx=0x7f4c6c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c62ffd640 1 -- 172.21.15.40:0/2415308291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c017070 con 0x7f4c7415b600 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c62ffd640 1 -- 172.21.15.40:0/2415308291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6c0052d0 con 0x7f4c7415b600 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c74150570 con 0x7f4c7415b600 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c62ffd640 1 -- 172.21.15.40:0/2415308291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c003830 con 0x7f4c7415b600 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.566+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c74150b30 con 0x7f4c7415b600 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.567+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c740694f0 con 0x7f4c7415b600 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.568+0000 7f4c62ffd640 1 -- 172.21.15.40:0/2415308291 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c6c01a070 con 0x7f4c7415b600 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.569+0000 7f4c62ffd640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4c5406c8f0 0x7f4c5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:17.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.569+0000 7f4c62ffd640 1 -- 172.21.15.40:0/2415308291 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f4c6c08ac00 con 0x7f4c7415b600 2024-04-26T21:36:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.569+0000 7f4c7259c640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4c5406c8f0 0x7f4c5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.569+0000 7f4c7259c640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4c5406c8f0 0x7f4c5406edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c64035000 tx=0x7f4c6400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.573+0000 7f4c62ffd640 1 -- 172.21.15.40:0/2415308291 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c6c058b40 con 0x7f4c7415b600 2024-04-26T21:36:17.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:17.777+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4c740b6940 con 0x7f4c7415b600 2024-04-26T21:36:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.389+0000 7f4c62ffd640 1 -- 172.21.15.40:0/2415308291 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f4c6c05ce40 con 0x7f4c7415b600 2024-04-26T21:36:18.389 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:36:18.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4c5406c8f0 msgr2=0x7f4c5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4c5406c8f0 0x7f4c5406edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c64035000 tx=0x7f4c6400a000 comp rx=0 tx=0).stop 2024-04-26T21:36:18.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c7415b600 msgr2=0x7f4c740b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c7415b600 0x7f4c740b1e20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00c340 tx=0x7f4c6c001e70 comp rx=0 tx=0).stop 2024-04-26T21:36:18.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 shutdown_connections 2024-04-26T21:36:18.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4c5406c8f0 0x7f4c5406edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c7415b600 0x7f4c740b1e20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c740cf5e0 0x7f4c740b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 --2- 172.21.15.40:0/2415308291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c740cf0e0 0x7f4c740c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 >> 172.21.15.40:0/2415308291 conn(0x7f4c740d0be0 msgr2=0x7f4c74156cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:18.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 shutdown_connections 2024-04-26T21:36:18.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.392+0000 7f4c78c15640 1 -- 172.21.15.40:0/2415308291 wait complete. 2024-04-26T21:36:18.406 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:36:18.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.511+0000 7fb5e7899640 1 Processor -- start 2024-04-26T21:36:18.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.511+0000 7fb5e7899640 1 -- start start 2024-04-26T21:36:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e7899640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5e0162880 0x7fb5e00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e7899640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e00b7e50 con 0x7fb5e00b5f20 2024-04-26T21:36:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e7899640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e00d1800 con 0x7fb5e00d3bc0 2024-04-26T21:36:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e7899640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e00d0580 con 0x7fb5e0162880 2024-04-26T21:36:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e5635640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5e0162880 0x7fb5e00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e4e34640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb5e00b5f20 0x7fb5e00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50214/0 (socket says 172.21.15.40:50214) 2024-04-26T21:36:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e4e34640 1 -- 172.21.15.40:0/1766532020 learned_addr learned my addr 172.21.15.40:0/1766532020 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e5635640 1 -- 172.21.15.40:0/1766532020 >> v1:172.21.15.184:6789/0 conn(0x7fb5e00d3bc0 legacy=0x7fb5e00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e5635640 1 -- 172.21.15.40:0/1766532020 >> v1:172.21.15.40:6789/0 conn(0x7fb5e00b5f20 legacy=0x7fb5e00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5e5635640 1 -- 172.21.15.40:0/1766532020 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e00d07d0 con 0x7fb5e0162880 2024-04-26T21:36:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.512+0000 7fb5d7fff640 1 -- 172.21.15.40:0/1766532020 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 233813984 0 0) 0x7fb5e00b7e50 con 0x7fb5e00b5f20 2024-04-26T21:36:18.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.513+0000 7fb5e5635640 1 --2- 172.21.15.40:0/1766532020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e0162880 0x7fb5e00d2cb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0097a0 tx=0x7fb5dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2be99c913ceced server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:18.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.513+0000 7fb5d7fff640 1 -- 172.21.15.40:0/1766532020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc037070 con 0x7fb5e0162880 2024-04-26T21:36:18.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.513+0000 7fb5d7fff640 1 -- 172.21.15.40:0/1766532020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc032440 con 0x7fb5e0162880 2024-04-26T21:36:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.513+0000 7fb5d7fff640 1 -- 172.21.15.40:0/1766532020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc032810 con 0x7fb5e0162880 2024-04-26T21:36:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.514+0000 7fb5e7899640 1 -- 172.21.15.40:0/1766532020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e0162880 msgr2=0x7fb5e00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.514+0000 7fb5e7899640 1 --2- 172.21.15.40:0/1766532020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e0162880 0x7fb5e00d2cb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0097a0 tx=0x7fb5dc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.514+0000 7fb5e7899640 1 -- 172.21.15.40:0/1766532020 shutdown_connections 2024-04-26T21:36:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.514+0000 7fb5e7899640 1 --2- 172.21.15.40:0/1766532020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e0162880 0x7fb5e00d2cb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.514+0000 7fb5e7899640 1 -- 172.21.15.40:0/1766532020 >> 172.21.15.40:0/1766532020 conn(0x7fb5e00d4f60 msgr2=0x7fb5e00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.514+0000 7fb5e7899640 1 -- 172.21.15.40:0/1766532020 shutdown_connections 2024-04-26T21:36:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.514+0000 7fb5e7899640 1 -- 172.21.15.40:0/1766532020 wait complete. 2024-04-26T21:36:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.515+0000 7fb5e7899640 1 Processor -- start 2024-04-26T21:36:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.515+0000 7fb5e7899640 1 -- start start 2024-04-26T21:36:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e7899640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e00b5f20 0x7fb5e00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e7899640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e00d3bc0 0x7fb5e01f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e7899640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e0162880 0x7fb5e00b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e7899640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e0162c60 con 0x7fb5e00b5f20 2024-04-26T21:36:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e7899640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e00d0d90 con 0x7fb5e0162880 2024-04-26T21:36:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e7899640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e0162de0 con 0x7fb5e00d3bc0 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e00b5f20 0x7fb5e00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e00b5f20 0x7fb5e00b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58556/0 (socket says 172.21.15.40:58556) 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 -- 172.21.15.40:0/1253893504 learned_addr learned my addr 172.21.15.40:0/1253893504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 -- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e00d3bc0 msgr2=0x7fb5e01f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e00d3bc0 0x7fb5e01f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 -- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e0162880 msgr2=0x7fb5e00b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e0162880 0x7fb5e00b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 -- 172.21.15.40:0/1253893504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e00b21e0 con 0x7fb5e00b5f20 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e5635640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e00b5f20 0x7fb5e00b1b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc000c00 tx=0x7fb5dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5d5ffb640 1 -- 172.21.15.40:0/1253893504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc037040 con 0x7fb5e00b5f20 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5d5ffb640 1 -- 172.21.15.40:0/1253893504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc004230 con 0x7fb5e00b5f20 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e00b24c0 con 0x7fb5e00b5f20 2024-04-26T21:36:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.516+0000 7fb5d5ffb640 1 -- 172.21.15.40:0/1253893504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc032e40 con 0x7fb5e00b5f20 2024-04-26T21:36:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.517+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e00b2a80 con 0x7fb5e00b5f20 2024-04-26T21:36:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.518+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e00694f0 con 0x7fb5e00b5f20 2024-04-26T21:36:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.518+0000 7fb5d5ffb640 1 -- 172.21.15.40:0/1253893504 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5dc030770 con 0x7fb5e00b5f20 2024-04-26T21:36:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.519+0000 7fb5d5ffb640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5c406c8f0 0x7fb5c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.519+0000 7fb5d5ffb640 1 -- 172.21.15.40:0/1253893504 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fb5dc0acb30 con 0x7fb5e00b5f20 2024-04-26T21:36:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.519+0000 7fb5e4e34640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5c406c8f0 0x7fb5c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.519+0000 7fb5e4e34640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5c406c8f0 0x7fb5c406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0007fd0 tx=0x7fb5d0007960 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.523+0000 7fb5d5ffb640 1 -- 172.21.15.40:0/1253893504 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5e00d1800 con 0x7fb5e00b5f20 2024-04-26T21:36:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.781+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb5e00d1470 con 0x7fb5e00b5f20 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.782+0000 7fb5d5ffb640 1 -- 172.21.15.40:0/1253893504 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fb5dc07b1b0 con 0x7fb5e00b5f20 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmesG+ERAAJS53jYoKUawIOs2zrG5TLg==", 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmq9cvHRAAiYWrMjfAth8wMbs9YlfWgQ==", 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.783 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmLRmvGBAA6xAcjLDF7DMjfDqiUCexKA==", 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:36:18.784 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmXxkDJBAAbzzf+LUO3Rr8bdzXb+Gj2Q==", 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDYHSxmRkhFBhAAz1UfvJwhA8bolDiRN2eM+Q==", 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.785 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDbHSxmKAB/BBAAzW5dtz6eczjkDiTQBjfjuA==", 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDeHSxmrienAhAA2JS0l6/Uo9UPc0TeZ1RwRA==", 2024-04-26T21:36:18.786 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDhHSxmv40rARAAN8z+beJe/bTERvO2euMr6w==", 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDnHSxmxQ1XCBAAYptVM8WPir+zIHOjbX0tGg==", 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDqHSxmv32IBhAAMROkCLjWTOx3ELdkEiVSmw==", 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDtHSxmJdPTBBAAO32qvVLuAzqx1xdajL9CFw==", 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDwHSxmcUMYAxAAIvoAo82ApDXERodD+4+cBQ==", 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:36:18.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw==", 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmifWYMBAA6Nskvqt3MZSvFCMHjslELw==", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmi6l6LRAAF4EjChVEB8ReULEsl3YppA==", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:36:18.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmhSMFNhAAjCJvc4FknsSUJQ4NjXRV6Q==", 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDTHSxmQk1bMhAAixlszeAmlXc5cAmkFWjhfw==", 2024-04-26T21:36:18.792 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mds", 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD1HSxmBO0DLxAAZ6/7rjet45V5A+FHqvixsA==", 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD1HSxmWfYDLxAAaACyEanL6OZnr5uOymqEFQ==", 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-osd", 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD1HSxm4P0DLxAAle/GHidbtSd1O6osN2iNdw==", 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD1HSxm/wQELxAAgCmQOFtkoZxLdPdtqyTk5g==", 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD1HSxmdQwELxAALdDuUsqMvq+5VdhSoz4x3Q==", 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.795 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD1HSxmAhQELxAAn31A6OgLOLJyZl1IlQbqRQ==", 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmYjmPDhAADvvIHrcacI8kOAdeiJGd+Q==", 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:36:18.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmsiwICxAAmKAZ2KS3/dYubutkcGfolA==", 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:36:18.797 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:36:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.783+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5c406c8f0 msgr2=0x7fb5c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.783+0000 7fb5e7899640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5c406c8f0 0x7fb5c406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0007fd0 tx=0x7fb5d0007960 comp rx=0 tx=0).stop 2024-04-26T21:36:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.783+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e00b5f20 msgr2=0x7fb5e00b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.783+0000 7fb5e7899640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e00b5f20 0x7fb5e00b1b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc000c00 tx=0x7fb5dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.783+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 shutdown_connections 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.783+0000 7fb5e7899640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5c406c8f0 0x7fb5c406edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.784+0000 7fb5e7899640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e0162880 0x7fb5e00b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.784+0000 7fb5e7899640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e00d3bc0 0x7fb5e01f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.784+0000 7fb5e7899640 1 --2- 172.21.15.40:0/1253893504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e00b5f20 0x7fb5e00b1b50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.784+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 >> 172.21.15.40:0/1253893504 conn(0x7fb5e00d4f60 msgr2=0x7fb5e0161580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.784+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 shutdown_connections 2024-04-26T21:36:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.784+0000 7fb5e7899640 1 -- 172.21.15.40:0/1253893504 wait complete. 2024-04-26T21:36:18.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T21:36:18.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.901+0000 7f1caef4e640 1 Processor -- start 2024-04-26T21:36:18.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.901+0000 7f1caef4e640 1 -- start start 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1caef4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81604e0 0x7f1ca806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1caef4e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ca806fcd0 0x7f1ca80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1caef4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca8160970 con 0x7f1ca815f2d0 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1caef4e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca8074010 con 0x7f1ca806fcd0 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1caef4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca8070100 con 0x7f1ca81604e0 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1cad4eb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ca806fcd0 0x7f1ca80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1cad4eb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ca806fcd0 0x7f1ca80ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40800/0 (socket says 172.21.15.40:40800) 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1cad4eb640 1 -- 172.21.15.40:0/1827191881 learned_addr learned my addr 172.21.15.40:0/1827191881 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1ca7fff640 1 --2- 172.21.15.40:0/1827191881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81604e0 0x7f1ca806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1cad4eb640 1 -- 172.21.15.40:0/1827191881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81604e0 msgr2=0x7f1ca806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1cad4eb640 1 --2- 172.21.15.40:0/1827191881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81604e0 0x7f1ca806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1cad4eb640 1 -- 172.21.15.40:0/1827191881 >> v1:172.21.15.184:6789/0 conn(0x7f1ca815f2d0 legacy=0x7f1ca8072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1cad4eb640 1 -- 172.21.15.40:0/1827191881 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca8072f70 con 0x7f1ca806fcd0 2024-04-26T21:36:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.902+0000 7f1ca7fff640 1 --2- 172.21.15.40:0/1827191881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81604e0 0x7f1ca806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.903+0000 7f1cad4eb640 1 --2- 172.21.15.40:0/1827191881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca806fcd0 0x7f1ca80ce7b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009e50 tx=0x7f1ca002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36242225a3ebca2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.903+0000 7f1ca77fe640 1 -- 172.21.15.40:0/1827191881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0036070 con 0x7f1ca806fcd0 2024-04-26T21:36:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.903+0000 7f1ca77fe640 1 -- 172.21.15.40:0/1827191881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca0031600 con 0x7f1ca806fcd0 2024-04-26T21:36:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.903+0000 7f1ca77fe640 1 -- 172.21.15.40:0/1827191881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00319d0 con 0x7f1ca806fcd0 2024-04-26T21:36:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.903+0000 7f1caef4e640 1 -- 172.21.15.40:0/1827191881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca806fcd0 msgr2=0x7f1ca80ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.903+0000 7f1caef4e640 1 --2- 172.21.15.40:0/1827191881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca806fcd0 0x7f1ca80ce7b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009e50 tx=0x7f1ca002a710 comp rx=0 tx=0).stop 2024-04-26T21:36:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.904+0000 7f1caef4e640 1 -- 172.21.15.40:0/1827191881 shutdown_connections 2024-04-26T21:36:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.904+0000 7f1caef4e640 1 --2- 172.21.15.40:0/1827191881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca806fcd0 0x7f1ca80ce7b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.904+0000 7f1caef4e640 1 --2- 172.21.15.40:0/1827191881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81604e0 0x7f1ca806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.904+0000 7f1caef4e640 1 -- 172.21.15.40:0/1827191881 >> 172.21.15.40:0/1827191881 conn(0x7f1ca80d0be0 msgr2=0x7f1ca80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.904+0000 7f1caef4e640 1 -- 172.21.15.40:0/1827191881 shutdown_connections 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.904+0000 7f1caef4e640 1 -- 172.21.15.40:0/1827191881 wait complete. 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.905+0000 7f1caef4e640 1 Processor -- start 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.905+0000 7f1caef4e640 1 -- start start 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caef4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca806fcd0 0x7f1ca815b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caef4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca815f2d0 0x7f1ca81f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caef4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca81604e0 0x7f1ca815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caef4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8073810 con 0x7f1ca815f2d0 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caef4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8073530 con 0x7f1ca81604e0 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caef4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca8073af0 con 0x7f1ca806fcd0 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca815f2d0 0x7f1ca81f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca815f2d0 0x7f1ca81f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40810/0 (socket says 172.21.15.40:40810) 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 -- 172.21.15.40:0/2956396291 learned_addr learned my addr 172.21.15.40:0/2956396291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1cad4eb640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca81604e0 0x7f1ca815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caccea640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca806fcd0 0x7f1ca815b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 -- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca806fcd0 msgr2=0x7f1ca815b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca806fcd0 0x7f1ca815b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 -- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca81604e0 msgr2=0x7f1ca815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca81604e0 0x7f1ca815bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 -- 172.21.15.40:0/2956396291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca8147a30 con 0x7f1ca815f2d0 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1cad4eb640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca81604e0 0x7f1ca815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca7fff640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca815f2d0 0x7f1ca81f4bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400d7d0 tx=0x7f1c9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca57fa640 1 -- 172.21.15.40:0/2956396291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94004030 con 0x7f1ca815f2d0 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca57fa640 1 -- 172.21.15.40:0/2956396291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c940041d0 con 0x7f1ca815f2d0 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca815c3e0 con 0x7f1ca815f2d0 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1ca57fa640 1 -- 172.21.15.40:0/2956396291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400b6c0 con 0x7f1ca815f2d0 2024-04-26T21:36:18.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.906+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca8153c90 con 0x7f1ca815f2d0 2024-04-26T21:36:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.907+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca80694f0 con 0x7f1ca815f2d0 2024-04-26T21:36:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.908+0000 7f1ca57fa640 1 -- 172.21.15.40:0/2956396291 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c9400b860 con 0x7f1ca815f2d0 2024-04-26T21:36:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.908+0000 7f1ca57fa640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.908+0000 7f1ca57fa640 1 -- 172.21.15.40:0/2956396291 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f1c9408bc10 con 0x7f1ca815f2d0 2024-04-26T21:36:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.909+0000 7f1caccea640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.909+0000 7f1caccea640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0015a0 tx=0x7f1c9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:18.913+0000 7f1ca57fa640 1 -- 172.21.15.40:0/2956396291 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c9405a200 con 0x7f1ca815f2d0 2024-04-26T21:36:19.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.167+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f1ca806de80 con 0x7f1ca815f2d0 2024-04-26T21:36:19.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.169+0000 7f1ca57fa640 1 -- 172.21.15.40:0/2956396291 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f1c94059b00 con 0x7f1ca815f2d0 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.171+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c8c06c8f0 msgr2=0x7f1c8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.171+0000 7f1caef4e640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0015a0 tx=0x7f1c9c009040 comp rx=0 tx=0).stop 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.171+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca815f2d0 msgr2=0x7f1ca81f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.171+0000 7f1caef4e640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca815f2d0 0x7f1ca81f4bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400d7d0 tx=0x7f1c9400dc90 comp rx=0 tx=0).stop 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.172+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 shutdown_connections 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.172+0000 7f1caef4e640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.172+0000 7f1caef4e640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca81604e0 0x7f1ca815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.172+0000 7f1caef4e640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca815f2d0 0x7f1ca81f4bb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.172+0000 7f1caef4e640 1 --2- 172.21.15.40:0/2956396291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca806fcd0 0x7f1ca815b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.172+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 >> 172.21.15.40:0/2956396291 conn(0x7f1ca80d0be0 msgr2=0x7f1ca806f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.172+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 shutdown_connections 2024-04-26T21:36:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.172+0000 7f1caef4e640 1 -- 172.21.15.40:0/2956396291 wait complete. 2024-04-26T21:36:19.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T21:36:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.284+0000 7ff3df51b640 1 Processor -- start 2024-04-26T21:36:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.284+0000 7ff3df51b640 1 -- start start 2024-04-26T21:36:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3df51b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3d80d3bc0 0x7ff3d80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3df51b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d80b7e50 con 0x7ff3d8162880 2024-04-26T21:36:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3df51b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d80d1800 con 0x7ff3d80b5f20 2024-04-26T21:36:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3df51b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d80d0580 con 0x7ff3d80d3bc0 2024-04-26T21:36:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3dcab6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3d80d3bc0 0x7ff3d80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3dd2b7640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff3d8162880 0x7ff3d80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49968/0 (socket says 172.21.15.40:49968) 2024-04-26T21:36:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3dd2b7640 1 -- 172.21.15.40:0/965560783 learned_addr learned my addr 172.21.15.40:0/965560783 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3dcab6640 1 -- 172.21.15.40:0/965560783 >> v1:172.21.15.184:6789/0 conn(0x7ff3d80b5f20 legacy=0x7ff3d80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3dcab6640 1 -- 172.21.15.40:0/965560783 >> v1:172.21.15.40:6789/0 conn(0x7ff3d8162880 legacy=0x7ff3d80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.285+0000 7ff3dcab6640 1 -- 172.21.15.40:0/965560783 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d80d07d0 con 0x7ff3d80d3bc0 2024-04-26T21:36:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3dcab6640 1 --2- 172.21.15.40:0/965560783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80d3bc0 0x7ff3d80d2cb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80097a0 tx=0x7ff3c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f861917339a5f6cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3cffff640 1 -- 172.21.15.40:0/965560783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8037070 con 0x7ff3d80d3bc0 2024-04-26T21:36:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3cffff640 1 -- 172.21.15.40:0/965560783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c8032440 con 0x7ff3d80d3bc0 2024-04-26T21:36:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3cffff640 1 -- 172.21.15.40:0/965560783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8032810 con 0x7ff3d80d3bc0 2024-04-26T21:36:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3df51b640 1 -- 172.21.15.40:0/965560783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80d3bc0 msgr2=0x7ff3d80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3df51b640 1 --2- 172.21.15.40:0/965560783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80d3bc0 0x7ff3d80d2cb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80097a0 tx=0x7ff3c802a230 comp rx=0 tx=0).stop 2024-04-26T21:36:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3df51b640 1 -- 172.21.15.40:0/965560783 shutdown_connections 2024-04-26T21:36:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3df51b640 1 --2- 172.21.15.40:0/965560783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80d3bc0 0x7ff3d80d2cb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.286+0000 7ff3df51b640 1 -- 172.21.15.40:0/965560783 >> 172.21.15.40:0/965560783 conn(0x7ff3d80d4f60 msgr2=0x7ff3d80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.287+0000 7ff3df51b640 1 -- 172.21.15.40:0/965560783 shutdown_connections 2024-04-26T21:36:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.287+0000 7ff3df51b640 1 -- 172.21.15.40:0/965560783 wait complete. 2024-04-26T21:36:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3df51b640 1 Processor -- start 2024-04-26T21:36:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3df51b640 1 -- start start 2024-04-26T21:36:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3df51b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80b5f20 0x7ff3d80ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3df51b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d80d3bc0 0x7ff3d80b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3df51b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3d8162880 0x7ff3d80b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3df51b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d8162c60 con 0x7ff3d80b5f20 2024-04-26T21:36:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3df51b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d80d0d90 con 0x7ff3d80d3bc0 2024-04-26T21:36:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3df51b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d8162de0 con 0x7ff3d8162880 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.288+0000 7ff3dd2b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80b5f20 0x7ff3d80ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dd2b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80b5f20 0x7ff3d80ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40828/0 (socket says 172.21.15.40:40828) 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dd2b7640 1 -- 172.21.15.40:0/3684733163 learned_addr learned my addr 172.21.15.40:0/3684733163 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dcab6640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d80d3bc0 0x7ff3d80b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3ddab8640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3d8162880 0x7ff3d80b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dd2b7640 1 -- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3d8162880 msgr2=0x7ff3d80b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dd2b7640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3d8162880 0x7ff3d80b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dd2b7640 1 -- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d80d3bc0 msgr2=0x7ff3d80b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dd2b7640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d80d3bc0 0x7ff3d80b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dd2b7640 1 -- 172.21.15.40:0/3684733163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d80b3780 con 0x7ff3d80b5f20 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dcab6640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d80d3bc0 0x7ff3d80b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3dd2b7640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80b5f20 0x7ff3d80ce880 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400c480 tx=0x7ff3d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3cdffb640 1 -- 172.21.15.40:0/3684733163 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4017070 con 0x7ff3d80b5f20 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3cdffb640 1 -- 172.21.15.40:0/3684733163 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d4005ce0 con 0x7ff3d80b5f20 2024-04-26T21:36:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d80b3a60 con 0x7ff3d80b5f20 2024-04-26T21:36:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3cdffb640 1 -- 172.21.15.40:0/3684733163 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40072f0 con 0x7ff3d80b5f20 2024-04-26T21:36:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.289+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d80b4020 con 0x7ff3d80b5f20 2024-04-26T21:36:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.290+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d80d2f70 con 0x7ff3d80b5f20 2024-04-26T21:36:19.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.291+0000 7ff3cdffb640 1 -- 172.21.15.40:0/3684733163 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3d401a070 con 0x7ff3d80b5f20 2024-04-26T21:36:19.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.291+0000 7ff3cdffb640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3bc06c8f0 0x7ff3bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.291+0000 7ff3cdffb640 1 -- 172.21.15.40:0/3684733163 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7ff3d408acc0 con 0x7ff3d80b5f20 2024-04-26T21:36:19.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.291+0000 7ff3dcab6640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3bc06c8f0 0x7ff3bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.292+0000 7ff3dcab6640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3bc06c8f0 0x7ff3bc06edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8034000 tx=0x7ff3c802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.296+0000 7ff3cdffb640 1 -- 172.21.15.40:0/3684733163 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3d80d2f70 con 0x7ff3d80b5f20 2024-04-26T21:36:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.553+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7ff3d80694f0 con 0x7ff3d80b5f20 2024-04-26T21:36:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.555+0000 7ff3cdffb640 1 -- 172.21.15.40:0/3684733163 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7ff3d4091080 con 0x7ff3d80b5f20 2024-04-26T21:36:19.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3bc06c8f0 msgr2=0x7ff3bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3bc06c8f0 0x7ff3bc06edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8034000 tx=0x7ff3c802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80b5f20 msgr2=0x7ff3d80ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80b5f20 0x7ff3d80ce880 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400c480 tx=0x7ff3d400c940 comp rx=0 tx=0).stop 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 shutdown_connections 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3bc06c8f0 0x7ff3bc06edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3d8162880 0x7ff3d80b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d80d3bc0 0x7ff3d80b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 --2- 172.21.15.40:0/3684733163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3d80b5f20 0x7ff3d80ce880 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 >> 172.21.15.40:0/3684733163 conn(0x7ff3d80d4f60 msgr2=0x7ff3d80d21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 shutdown_connections 2024-04-26T21:36:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.558+0000 7ff3df51b640 1 -- 172.21.15.40:0/3684733163 wait complete. 2024-04-26T21:36:19.569 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T21:36:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.669+0000 7f439e093640 1 Processor -- start 2024-04-26T21:36:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.670+0000 7f439e093640 1 -- start start 2024-04-26T21:36:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.670+0000 7f439e093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4398162880 0x7f43980d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.670+0000 7f439e093640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43980b5f20 0x7f43980d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.670+0000 7f439e093640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43980b7e50 con 0x7f43980d39f0 2024-04-26T21:36:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.670+0000 7f439e093640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f43980d0580 con 0x7f43980b5f20 2024-04-26T21:36:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.670+0000 7f439e093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43980d1800 con 0x7f4398162880 2024-04-26T21:36:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f4396ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43980b5f20 0x7f43980d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f43977fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4398162880 0x7f43980d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f4397fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f43980d39f0 0x7f43980b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55726/0 (socket says 172.21.15.40:55726) 2024-04-26T21:36:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f4397fff640 1 -- 172.21.15.40:0/56210136 learned_addr learned my addr 172.21.15.40:0/56210136 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f43967fc640 1 -- 172.21.15.40:0/56210136 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839436775 0 0) 0x7f43980b7e50 con 0x7f43980d39f0 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f43967fc640 1 -- 172.21.15.40:0/56210136 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4380003520 con 0x7f43980d39f0 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f43977fe640 1 -- 172.21.15.40:0/56210136 >> v2:172.21.15.184:3300/0 conn(0x7f43980b5f20 msgr2=0x7f43980d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f43977fe640 1 --2- 172.21.15.40:0/56210136 >> v2:172.21.15.184:3300/0 conn(0x7f43980b5f20 0x7f43980d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f43977fe640 1 -- 172.21.15.40:0/56210136 >> v1:172.21.15.184:6789/0 conn(0x7f43980d39f0 legacy=0x7f43980b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f43977fe640 1 -- 172.21.15.40:0/56210136 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43980d07d0 con 0x7f4398162880 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.671+0000 7f4396ffd640 1 --2- 172.21.15.40:0/56210136 >> v2:172.21.15.184:3300/0 conn(0x7f43980b5f20 0x7f43980d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.672+0000 7f43977fe640 1 --2- 172.21.15.40:0/56210136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4398162880 0x7f43980d2cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43880097a0 tx=0x7f438802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29aec10ef94f2964 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.672+0000 7f43967fc640 1 -- 172.21.15.40:0/56210136 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388037070 con 0x7f4398162880 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.672+0000 7f43967fc640 1 -- 172.21.15.40:0/56210136 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4388032440 con 0x7f4398162880 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.672+0000 7f43967fc640 1 -- 172.21.15.40:0/56210136 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880327f0 con 0x7f4398162880 2024-04-26T21:36:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.672+0000 7f439e093640 1 -- 172.21.15.40:0/56210136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4398162880 msgr2=0x7f43980d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.672+0000 7f439e093640 1 --2- 172.21.15.40:0/56210136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4398162880 0x7f43980d2cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43880097a0 tx=0x7f438802a180 comp rx=0 tx=0).stop 2024-04-26T21:36:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.673+0000 7f439e093640 1 -- 172.21.15.40:0/56210136 shutdown_connections 2024-04-26T21:36:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.673+0000 7f439e093640 1 --2- 172.21.15.40:0/56210136 >> v2:172.21.15.184:3300/0 conn(0x7f43980b5f20 0x7f43980d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.673+0000 7f439e093640 1 --2- 172.21.15.40:0/56210136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4398162880 0x7f43980d2cb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.673+0000 7f439e093640 1 -- 172.21.15.40:0/56210136 >> 172.21.15.40:0/56210136 conn(0x7f43980d4f60 msgr2=0x7f43980d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.673+0000 7f439e093640 1 -- 172.21.15.40:0/56210136 shutdown_connections 2024-04-26T21:36:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.673+0000 7f439e093640 1 -- 172.21.15.40:0/56210136 wait complete. 2024-04-26T21:36:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.674+0000 7f439e093640 1 Processor -- start 2024-04-26T21:36:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.674+0000 7f439e093640 1 -- start start 2024-04-26T21:36:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f439e093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43980b5f20 0x7f43981f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f439e093640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43980d39f0 0x7f43980ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f439e093640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4398162880 0x7f43980b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f439e093640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4398162c60 con 0x7f4398162880 2024-04-26T21:36:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f439e093640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43980d0d90 con 0x7f43980d39f0 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f439e093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4398162de0 con 0x7f43980b5f20 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4398162880 0x7f43980b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4398162880 0x7f43980b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40830/0 (socket says 172.21.15.40:40830) 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 -- 172.21.15.40:0/2887146011 learned_addr learned my addr 172.21.15.40:0/2887146011 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f43977fe640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43980b5f20 0x7f43981f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4396ffd640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43980d39f0 0x7f43980ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 -- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43980b5f20 msgr2=0x7f43981f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43980b5f20 0x7f43981f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 -- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43980d39f0 msgr2=0x7f43980ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43980d39f0 0x7f43980ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 -- 172.21.15.40:0/2887146011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43980cda40 con 0x7f4398162880 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f4397fff640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4398162880 0x7f43980b44d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f439000cb40 tx=0x7f439000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.675+0000 7f437ffff640 1 -- 172.21.15.40:0/2887146011 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390017070 con 0x7f4398162880 2024-04-26T21:36:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.676+0000 7f437ffff640 1 -- 172.21.15.40:0/2887146011 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439000aca0 con 0x7f4398162880 2024-04-26T21:36:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.676+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43980cdd20 con 0x7f4398162880 2024-04-26T21:36:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.676+0000 7f437ffff640 1 -- 172.21.15.40:0/2887146011 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390012620 con 0x7f4398162880 2024-04-26T21:36:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.676+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43980ce2e0 con 0x7f4398162880 2024-04-26T21:36:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.677+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43980694f0 con 0x7f4398162880 2024-04-26T21:36:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.677+0000 7f437ffff640 1 -- 172.21.15.40:0/2887146011 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f439001a070 con 0x7f4398162880 2024-04-26T21:36:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.677+0000 7f437ffff640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f438406c920 0x7f438406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.678+0000 7f437ffff640 1 -- 172.21.15.40:0/2887146011 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f439008b7a0 con 0x7f4398162880 2024-04-26T21:36:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.678+0000 7f43977fe640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f438406c920 0x7f438406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.678+0000 7f43977fe640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f438406c920 0x7f438406ede0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4388034000 tx=0x7f438802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.682+0000 7f437ffff640 1 -- 172.21.15.40:0/2887146011 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43900596c0 con 0x7f4398162880 2024-04-26T21:36:19.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.938+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f439806e320 con 0x7f4398162880 2024-04-26T21:36:19.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.941+0000 7f437ffff640 1 -- 172.21.15.40:0/2887146011 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f439005d9c0 con 0x7f4398162880 2024-04-26T21:36:19.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f438406c920 msgr2=0x7f438406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f438406c920 0x7f438406ede0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4388034000 tx=0x7f438802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:19.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4398162880 msgr2=0x7f43980b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4398162880 0x7f43980b44d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f439000cb40 tx=0x7f439000a060 comp rx=0 tx=0).stop 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 shutdown_connections 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f438406c920 0x7f438406ede0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4398162880 0x7f43980b44d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43980d39f0 0x7f43980ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 --2- 172.21.15.40:0/2887146011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43980b5f20 0x7f43981f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 >> 172.21.15.40:0/2887146011 conn(0x7f43980d4f60 msgr2=0x7f439815f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 shutdown_connections 2024-04-26T21:36:19.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:19.943+0000 7f439e093640 1 -- 172.21.15.40:0/2887146011 wait complete. 2024-04-26T21:36:19.954 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T21:36:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.056+0000 7f4337ec8640 1 Processor -- start 2024-04-26T21:36:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.056+0000 7f4337ec8640 1 -- start start 2024-04-26T21:36:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4337ec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4337ec8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43300b5f20 0x7f43300d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4337ec8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300b7e50 con 0x7f43300d39f0 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4337ec8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f43300d0580 con 0x7f43300b5f20 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4337ec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43300d1800 con 0x7f4330162880 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4335463640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43300b5f20 0x7f43300d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4336465640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f43300d39f0 0x7f43300b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49972/0 (socket says 172.21.15.40:49972) 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4336465640 1 -- 172.21.15.40:0/15707701 learned_addr learned my addr 172.21.15.40:0/15707701 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4335c64640 1 --2- 172.21.15.40:0/15707701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4335463640 1 -- 172.21.15.40:0/15707701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 msgr2=0x7f43300d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4335463640 1 --2- 172.21.15.40:0/15707701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4335463640 1 -- 172.21.15.40:0/15707701 >> v1:172.21.15.40:6789/0 conn(0x7f43300d39f0 legacy=0x7f43300b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4335463640 1 -- 172.21.15.40:0/15707701 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43300d07d0 con 0x7f43300b5f20 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4334c62640 1 -- 172.21.15.40:0/15707701 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 564444443 0 0) 0x7f43300b7e50 con 0x7f43300d39f0 2024-04-26T21:36:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.057+0000 7f4335c64640 1 --2- 172.21.15.40:0/15707701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4335463640 1 --2- 172.21.15.40:0/15707701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300b5f20 0x7f43300d34b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f43200097a0 tx=0x7f432002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65de9cbedc907b4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4334c62640 1 -- 172.21.15.40:0/15707701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320037070 con 0x7f43300b5f20 2024-04-26T21:36:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4334c62640 1 -- 172.21.15.40:0/15707701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4320032440 con 0x7f43300b5f20 2024-04-26T21:36:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4334c62640 1 -- 172.21.15.40:0/15707701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320032810 con 0x7f43300b5f20 2024-04-26T21:36:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4337ec8640 1 -- 172.21.15.40:0/15707701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300b5f20 msgr2=0x7f43300d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4337ec8640 1 --2- 172.21.15.40:0/15707701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300b5f20 0x7f43300d34b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f43200097a0 tx=0x7f432002a180 comp rx=0 tx=0).stop 2024-04-26T21:36:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4337ec8640 1 -- 172.21.15.40:0/15707701 shutdown_connections 2024-04-26T21:36:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4337ec8640 1 --2- 172.21.15.40:0/15707701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300b5f20 0x7f43300d34b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4337ec8640 1 --2- 172.21.15.40:0/15707701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.058+0000 7f4337ec8640 1 -- 172.21.15.40:0/15707701 >> 172.21.15.40:0/15707701 conn(0x7f43300d4f60 msgr2=0x7f43300d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.059+0000 7f4337ec8640 1 -- 172.21.15.40:0/15707701 shutdown_connections 2024-04-26T21:36:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.059+0000 7f4337ec8640 1 -- 172.21.15.40:0/15707701 wait complete. 2024-04-26T21:36:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.060+0000 7f4337ec8640 1 Processor -- start 2024-04-26T21:36:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.060+0000 7f4337ec8640 1 -- start start 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.060+0000 7f4337ec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43300b5f20 0x7f43300b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.060+0000 7f4337ec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f43300ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.060+0000 7f4337ec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4330162880 0x7f433015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.060+0000 7f4337ec8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4330162c60 con 0x7f43300d39f0 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.060+0000 7f4337ec8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43300d0d90 con 0x7f4330162880 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.060+0000 7f4337ec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4330162de0 con 0x7f43300b5f20 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f43300ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f43300ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40834/0 (socket says 172.21.15.40:40834) 2024-04-26T21:36:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 -- 172.21.15.40:0/3311188463 learned_addr learned my addr 172.21.15.40:0/3311188463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335c64640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43300b5f20 0x7f43300b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4336465640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4330162880 0x7f433015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 -- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43300b5f20 msgr2=0x7f43300b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43300b5f20 0x7f43300b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 -- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4330162880 msgr2=0x7f433015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4330162880 0x7f433015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 -- 172.21.15.40:0/3311188463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43300cd760 con 0x7f43300d39f0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4335463640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f43300ccff0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f43200097a0 tx=0x7f43200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f43267fc640 1 -- 172.21.15.40:0/3311188463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320037050 con 0x7f43300d39f0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f43267fc640 1 -- 172.21.15.40:0/3311188463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4320038040 con 0x7f43300d39f0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f43267fc640 1 -- 172.21.15.40:0/3311188463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320004490 con 0x7f43300d39f0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43300cda40 con 0x7f43300d39f0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.061+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43300ce000 con 0x7f43300d39f0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.062+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43300694f0 con 0x7f43300d39f0 2024-04-26T21:36:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.063+0000 7f43267fc640 1 -- 172.21.15.40:0/3311188463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4320030980 con 0x7f43300d39f0 2024-04-26T21:36:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.063+0000 7f43267fc640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.063+0000 7f43267fc640 1 -- 172.21.15.40:0/3311188463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f43200ac5d0 con 0x7f43300d39f0 2024-04-26T21:36:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.064+0000 7f4335c64640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.064+0000 7f4335c64640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f432c008850 tx=0x7f432c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.067+0000 7f43267fc640 1 -- 172.21.15.40:0/3311188463 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f432007ace0 con 0x7f43300d39f0 2024-04-26T21:36:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.325+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f4330160c20 con 0x7f43300d39f0 2024-04-26T21:36:20.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.327+0000 7f43267fc640 1 -- 172.21.15.40:0/3311188463 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f432007a5e0 con 0x7f43300d39f0 2024-04-26T21:36:20.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 msgr2=0x7f431406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f432c008850 tx=0x7f432c008040 comp rx=0 tx=0).stop 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 msgr2=0x7f43300ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f43300ccff0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f43200097a0 tx=0x7f43200046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 shutdown_connections 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4330162880 0x7f433015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f43300ccff0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 --2- 172.21.15.40:0/3311188463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43300b5f20 0x7f43300b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 >> 172.21.15.40:0/3311188463 conn(0x7f43300d4f60 msgr2=0x7f433006d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 shutdown_connections 2024-04-26T21:36:20.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.329+0000 7f4337ec8640 1 -- 172.21.15.40:0/3311188463 wait complete. 2024-04-26T21:36:20.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T21:36:20.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.442+0000 7f94f61bf640 1 Processor -- start 2024-04-26T21:36:20.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.442+0000 7f94f61bf640 1 -- start start 2024-04-26T21:36:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94f61bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94f0162880 0x7f94f00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94f61bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f94f00b5f20 0x7f94f00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94f61bf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f00b7e50 con 0x7f94f00d39f0 2024-04-26T21:36:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94f61bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f94f00d3e40 con 0x7f94f0162880 2024-04-26T21:36:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94f61bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f94f00d0580 con 0x7f94f00b5f20 2024-04-26T21:36:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94ef7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94f0162880 0x7f94f00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94ef7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94f0162880 0x7f94f00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40850/0 (socket says 172.21.15.40:40850) 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94ef7fe640 1 -- 172.21.15.40:0/3212005422 learned_addr learned my addr 172.21.15.40:0/3212005422 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94eeffd640 1 --2- 172.21.15.40:0/3212005422 >> v2:172.21.15.184:3300/0 conn(0x7f94f00b5f20 0x7f94f00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94ef7fe640 1 -- 172.21.15.40:0/3212005422 >> v2:172.21.15.184:3300/0 conn(0x7f94f00b5f20 msgr2=0x7f94f00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94ef7fe640 1 --2- 172.21.15.40:0/3212005422 >> v2:172.21.15.184:3300/0 conn(0x7f94f00b5f20 0x7f94f00d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94ef7fe640 1 -- 172.21.15.40:0/3212005422 >> v1:172.21.15.184:6789/0 conn(0x7f94f00d39f0 legacy=0x7f94f00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.443+0000 7f94ef7fe640 1 -- 172.21.15.40:0/3212005422 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f00d1800 con 0x7f94f0162880 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.444+0000 7f94ef7fe640 1 --2- 172.21.15.40:0/3212005422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00d2cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f94f008d3d0 tx=0x7f94e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e512fc12ee7e120 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.444+0000 7f94ee7fc640 1 -- 172.21.15.40:0/3212005422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0037070 con 0x7f94f0162880 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.444+0000 7f94ee7fc640 1 -- 172.21.15.40:0/3212005422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e0032440 con 0x7f94f0162880 2024-04-26T21:36:20.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.444+0000 7f94ee7fc640 1 -- 172.21.15.40:0/3212005422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0032810 con 0x7f94f0162880 2024-04-26T21:36:20.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.445+0000 7f94f61bf640 1 -- 172.21.15.40:0/3212005422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 msgr2=0x7f94f00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.445+0000 7f94f61bf640 1 --2- 172.21.15.40:0/3212005422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00d2cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f94f008d3d0 tx=0x7f94e002a270 comp rx=0 tx=0).stop 2024-04-26T21:36:20.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.445+0000 7f94f61bf640 1 -- 172.21.15.40:0/3212005422 shutdown_connections 2024-04-26T21:36:20.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.445+0000 7f94f61bf640 1 --2- 172.21.15.40:0/3212005422 >> v2:172.21.15.184:3300/0 conn(0x7f94f00b5f20 0x7f94f00d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.445+0000 7f94f61bf640 1 --2- 172.21.15.40:0/3212005422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00d2cb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.445+0000 7f94f61bf640 1 -- 172.21.15.40:0/3212005422 >> 172.21.15.40:0/3212005422 conn(0x7f94f00d4f60 msgr2=0x7f94f00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.445+0000 7f94f61bf640 1 -- 172.21.15.40:0/3212005422 shutdown_connections 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.445+0000 7f94f61bf640 1 -- 172.21.15.40:0/3212005422 wait complete. 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.446+0000 7f94f61bf640 1 Processor -- start 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.446+0000 7f94f61bf640 1 -- start start 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94f61bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94f00b5f20 0x7f94f0151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94f61bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94f00d39f0 0x7f94f00b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94f61bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94f61bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94f00d0d90 con 0x7f94f0162880 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94f61bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94f00d0ab0 con 0x7f94f00d39f0 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94f61bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94f0162c60 con 0x7f94f00b5f20 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40858/0 (socket says 172.21.15.40:40858) 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 -- 172.21.15.40:0/1535965783 learned_addr learned my addr 172.21.15.40:0/1535965783 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94ef7fe640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94f00b5f20 0x7f94f0151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94eeffd640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94f00d39f0 0x7f94f00b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 -- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94f00b5f20 msgr2=0x7f94f0151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94f00b5f20 0x7f94f0151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 -- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94f00d39f0 msgr2=0x7f94f00b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94f00d39f0 0x7f94f00b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 -- 172.21.15.40:0/1535965783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f015daf0 con 0x7f94f0162880 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94ef7fe640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94f00b5f20 0x7f94f0151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.447+0000 7f94effff640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00b1e40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c6e0 tx=0x7f94e800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.448+0000 7f94d7fff640 1 -- 172.21.15.40:0/1535965783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8003b70 con 0x7f94f0162880 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.448+0000 7f94d7fff640 1 -- 172.21.15.40:0/1535965783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e8003d30 con 0x7f94f0162880 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.448+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f00b29b0 con 0x7f94f0162880 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.448+0000 7f94d7fff640 1 -- 172.21.15.40:0/1535965783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8005310 con 0x7f94f0162880 2024-04-26T21:36:20.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.448+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f00b2f70 con 0x7f94f0162880 2024-04-26T21:36:20.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.449+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f00694f0 con 0x7f94f0162880 2024-04-26T21:36:20.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.449+0000 7f94d7fff640 1 -- 172.21.15.40:0/1535965783 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94e80054b0 con 0x7f94f0162880 2024-04-26T21:36:20.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.450+0000 7f94d7fff640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94dc06c920 0x7f94dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.450+0000 7f94d7fff640 1 -- 172.21.15.40:0/1535965783 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f94e808b890 con 0x7f94f0162880 2024-04-26T21:36:20.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.450+0000 7f94ef7fe640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94dc06c920 0x7f94dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.450+0000 7f94ef7fe640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94dc06c920 0x7f94dc06ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94e0034000 tx=0x7f94e002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.454+0000 7f94d7fff640 1 -- 172.21.15.40:0/1535965783 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94e8059df0 con 0x7f94f0162880 2024-04-26T21:36:20.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.711+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f94f015f7d0 con 0x7f94f0162880 2024-04-26T21:36:20.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.713+0000 7f94d7fff640 1 -- 172.21.15.40:0/1535965783 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f94e80596f0 con 0x7f94f0162880 2024-04-26T21:36:20.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94dc06c920 msgr2=0x7f94dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94dc06c920 0x7f94dc06ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94e0034000 tx=0x7f94e002e000 comp rx=0 tx=0).stop 2024-04-26T21:36:20.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 msgr2=0x7f94f00b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00b1e40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c6e0 tx=0x7f94e800cba0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 shutdown_connections 2024-04-26T21:36:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94dc06c920 0x7f94dc06ede0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f0162880 0x7f94f00b1e40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94f00d39f0 0x7f94f00b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 --2- 172.21.15.40:0/1535965783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94f00b5f20 0x7f94f0151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 >> 172.21.15.40:0/1535965783 conn(0x7f94f00d4f60 msgr2=0x7f94f0160a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 shutdown_connections 2024-04-26T21:36:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.715+0000 7f94f61bf640 1 -- 172.21.15.40:0/1535965783 wait complete. 2024-04-26T21:36:20.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T21:36:20.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.827+0000 7f51aea47640 1 Processor -- start 2024-04-26T21:36:20.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.827+0000 7f51aea47640 1 -- start start 2024-04-26T21:36:20.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.828+0000 7f51aea47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a815f3a0 0x7f51a80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.828+0000 7f51aea47640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a80d0e20 con 0x7f51a80d1770 2024-04-26T21:36:20.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.828+0000 7f51aea47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a80b1d70 con 0x7f51a80d2300 2024-04-26T21:36:20.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.828+0000 7f51aea47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51a80b2050 con 0x7f51a815f3a0 2024-04-26T21:36:20.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.828+0000 7f51a77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51a80d1770 0x7f51a80d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49978/0 (socket says 172.21.15.40:49978) 2024-04-26T21:36:20.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.828+0000 7f51a77fe640 1 -- 172.21.15.40:0/989752913 learned_addr learned my addr 172.21.15.40:0/989752913 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a7fff640 1 --2- 172.21.15.40:0/989752913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a815f3a0 0x7f51a80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a6ffd640 1 -- 172.21.15.40:0/989752913 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 259280668 0 0) 0x7f51a80d0e20 con 0x7f51a80d1770 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a6ffd640 1 -- 172.21.15.40:0/989752913 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5194003520 con 0x7f51a80d1770 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a6ffd640 1 -- 172.21.15.40:0/989752913 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476260588 0 0) 0x7f51a80b1d70 con 0x7f51a80d2300 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a6ffd640 1 -- 172.21.15.40:0/989752913 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51a80d0e20 con 0x7f51a80d2300 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a7fff640 1 -- 172.21.15.40:0/989752913 >> v1:172.21.15.184:6789/0 conn(0x7f51a80d2300 legacy=0x7f51a80b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a7fff640 1 -- 172.21.15.40:0/989752913 >> v1:172.21.15.40:6789/0 conn(0x7f51a80d1770 legacy=0x7f51a80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a6ffd640 1 -- 172.21.15.40:0/989752913 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 149779688 0 0) 0x7f5194003520 con 0x7f51a80d1770 2024-04-26T21:36:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.829+0000 7f51a7fff640 1 -- 172.21.15.40:0/989752913 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a80b2330 con 0x7f51a815f3a0 2024-04-26T21:36:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.830+0000 7f51a7fff640 1 --2- 172.21.15.40:0/989752913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a815f3a0 0x7f51a80d06f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51900097d0 tx=0x7f519002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b5f73b3020014b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.830+0000 7f51a6ffd640 1 -- 172.21.15.40:0/989752913 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190037070 con 0x7f51a815f3a0 2024-04-26T21:36:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.830+0000 7f51a6ffd640 1 -- 172.21.15.40:0/989752913 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519002ac30 con 0x7f51a815f3a0 2024-04-26T21:36:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.830+0000 7f51a6ffd640 1 -- 172.21.15.40:0/989752913 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198001670 con 0x7f51a815f3a0 2024-04-26T21:36:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.831+0000 7f51aea47640 1 -- 172.21.15.40:0/989752913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a815f3a0 msgr2=0x7f51a80d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.831+0000 7f51aea47640 1 --2- 172.21.15.40:0/989752913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a815f3a0 0x7f51a80d06f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51900097d0 tx=0x7f519002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.831+0000 7f51aea47640 1 -- 172.21.15.40:0/989752913 shutdown_connections 2024-04-26T21:36:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.831+0000 7f51aea47640 1 --2- 172.21.15.40:0/989752913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a815f3a0 0x7f51a80d06f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.831+0000 7f51aea47640 1 -- 172.21.15.40:0/989752913 >> 172.21.15.40:0/989752913 conn(0x7f51a80d4f90 msgr2=0x7f51a80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.831+0000 7f51aea47640 1 -- 172.21.15.40:0/989752913 shutdown_connections 2024-04-26T21:36:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.831+0000 7f51aea47640 1 -- 172.21.15.40:0/989752913 wait complete. 2024-04-26T21:36:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.832+0000 7f51aea47640 1 Processor -- start 2024-04-26T21:36:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.832+0000 7f51aea47640 1 -- start start 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51aea47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a80d1770 0x7f51a815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51aea47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a80d2300 0x7f51a80b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51aea47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51a815f3a0 0x7f51a80b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51aea47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51a80b30e0 con 0x7f51a80d2300 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51aea47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51a80b2f60 con 0x7f51a815f3a0 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51aea47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51a80b3260 con 0x7f51a80d1770 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a80d2300 0x7f51a80b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a80d2300 0x7f51a80b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40868/0 (socket says 172.21.15.40:40868) 2024-04-26T21:36:20.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 -- 172.21.15.40:0/4172155034 learned_addr learned my addr 172.21.15.40:0/4172155034 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a7fff640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a80d1770 0x7f51a815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51acfe4640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51a815f3a0 0x7f51a80b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 -- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a80d1770 msgr2=0x7f51a815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a80d1770 0x7f51a815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 -- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51a815f3a0 msgr2=0x7f51a80b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51a815f3a0 0x7f51a80b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 -- 172.21.15.40:0/4172155034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a80cb720 con 0x7f51a80d2300 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a77fe640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a80d2300 0x7f51a80b5450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f519800c5a0 tx=0x7f519800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a4ff9640 1 -- 172.21.15.40:0/4172155034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198017070 con 0x7f51a80d2300 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a4ff9640 1 -- 172.21.15.40:0/4172155034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5198005ce0 con 0x7f51a80d2300 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a80b9e50 con 0x7f51a80d2300 2024-04-26T21:36:20.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.833+0000 7f51a4ff9640 1 -- 172.21.15.40:0/4172155034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51980072b0 con 0x7f51a80d2300 2024-04-26T21:36:20.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.834+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a80ba410 con 0x7f51a80d2300 2024-04-26T21:36:20.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.835+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a80d1bf0 con 0x7f51a80d2300 2024-04-26T21:36:20.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.835+0000 7f51a4ff9640 1 -- 172.21.15.40:0/4172155034 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f519801a070 con 0x7f51a80d2300 2024-04-26T21:36:20.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.836+0000 7f51a4ff9640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518806c8f0 0x7f518806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:20.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.836+0000 7f51a4ff9640 1 -- 172.21.15.40:0/4172155034 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f519808ae10 con 0x7f51a80d2300 2024-04-26T21:36:20.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.836+0000 7f51a7fff640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518806c8f0 0x7f518806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:20.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.836+0000 7f51a7fff640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518806c8f0 0x7f518806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5190034000 tx=0x7f519002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:20.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:20.840+0000 7f51a4ff9640 1 -- 172.21.15.40:0/4172155034 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51a80d1bf0 con 0x7f51a80d2300 2024-04-26T21:36:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.100+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f51a80694f0 con 0x7f51a80d2300 2024-04-26T21:36:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.103+0000 7f51a4ff9640 1 -- 172.21.15.40:0/4172155034 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f5198059490 con 0x7f51a80d2300 2024-04-26T21:36:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518806c8f0 msgr2=0x7f518806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518806c8f0 0x7f518806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5190034000 tx=0x7f519002e000 comp rx=0 tx=0).stop 2024-04-26T21:36:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a80d2300 msgr2=0x7f51a80b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a80d2300 0x7f51a80b5450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f519800c5a0 tx=0x7f519800ca60 comp rx=0 tx=0).stop 2024-04-26T21:36:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 shutdown_connections 2024-04-26T21:36:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518806c8f0 0x7f518806edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51a815f3a0 0x7f51a80b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a80d2300 0x7f51a80b5450 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 --2- 172.21.15.40:0/4172155034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51a80d1770 0x7f51a815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 >> 172.21.15.40:0/4172155034 conn(0x7f51a80d4f90 msgr2=0x7f51a80b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 shutdown_connections 2024-04-26T21:36:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.106+0000 7f51aea47640 1 -- 172.21.15.40:0/4172155034 wait complete. 2024-04-26T21:36:21.118 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:21.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.220+0000 7fbeb3719640 1 Processor -- start 2024-04-26T21:36:21.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.221+0000 7fbeb3719640 1 -- start start 2024-04-26T21:36:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.221+0000 7fbeb3719640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbeac0cffa0 0x7fbeac0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.221+0000 7fbeb3719640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d0a90 0x7fbeac0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.221+0000 7fbeb3719640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeac0d0f00 con 0x7fbeac1608f0 2024-04-26T21:36:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.221+0000 7fbeb3719640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbeac0cea30 con 0x7fbeac0cffa0 2024-04-26T21:36:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.221+0000 7fbeb3719640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac0cde20 con 0x7fbeac0d0a90 2024-04-26T21:36:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb0cb4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbeac0cffa0 0x7fbeac0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb14b5640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fbeac1608f0 0x7fbeac0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55744/0 (socket says 172.21.15.40:55744) 2024-04-26T21:36:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb14b5640 1 -- 172.21.15.40:0/4084202827 learned_addr learned my addr 172.21.15.40:0/4084202827 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb1cb6640 1 --2- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d0a90 0x7fbeac0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeabfff640 1 -- 172.21.15.40:0/4084202827 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2589737252 0 0) 0x7fbeac0d0f00 con 0x7fbeac1608f0 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeabfff640 1 -- 172.21.15.40:0/4084202827 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe94003520 con 0x7fbeac1608f0 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb0cb4640 1 -- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d0a90 msgr2=0x7fbeac0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb0cb4640 1 --2- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d0a90 0x7fbeac0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb0cb4640 1 -- 172.21.15.40:0/4084202827 >> v1:172.21.15.184:6789/0 conn(0x7fbeac1608f0 legacy=0x7fbeac0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb0cb4640 1 -- 172.21.15.40:0/4084202827 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac0ce100 con 0x7fbeac0cffa0 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.222+0000 7fbeb1cb6640 1 --2- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d0a90 0x7fbeac0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.223+0000 7fbeb0cb4640 1 --2- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac0cffa0 0x7fbeac0d1bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbea00097a0 tx=0x7fbea002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac570eda5de870a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.223+0000 7fbeabfff640 1 -- 172.21.15.40:0/4084202827 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0037070 con 0x7fbeac0cffa0 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.223+0000 7fbeabfff640 1 -- 172.21.15.40:0/4084202827 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea0032440 con 0x7fbeac0cffa0 2024-04-26T21:36:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.223+0000 7fbeabfff640 1 -- 172.21.15.40:0/4084202827 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00327f0 con 0x7fbeac0cffa0 2024-04-26T21:36:21.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.224+0000 7fbeb3719640 1 -- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac0cffa0 msgr2=0x7fbeac0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.224+0000 7fbeb3719640 1 --2- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac0cffa0 0x7fbeac0d1bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbea00097a0 tx=0x7fbea002a180 comp rx=0 tx=0).stop 2024-04-26T21:36:21.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.224+0000 7fbeb3719640 1 -- 172.21.15.40:0/4084202827 shutdown_connections 2024-04-26T21:36:21.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.224+0000 7fbeb3719640 1 --2- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d0a90 0x7fbeac0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.224+0000 7fbeb3719640 1 --2- 172.21.15.40:0/4084202827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac0cffa0 0x7fbeac0d1bf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.224+0000 7fbeb3719640 1 -- 172.21.15.40:0/4084202827 >> 172.21.15.40:0/4084202827 conn(0x7fbeac0d2ec0 msgr2=0x7fbeac0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:21.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.224+0000 7fbeb3719640 1 -- 172.21.15.40:0/4084202827 shutdown_connections 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.224+0000 7fbeb3719640 1 -- 172.21.15.40:0/4084202827 wait complete. 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.225+0000 7fbeb3719640 1 Processor -- start 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.225+0000 7fbeb3719640 1 -- start start 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb3719640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0cffa0 0x7fbeac0c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb3719640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0d0a90 0x7fbeac06efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb3719640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac1608f0 0x7fbeac1e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb3719640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0ceec0 con 0x7fbeac0d0a90 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb3719640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0ce6c0 con 0x7fbeac1608f0 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb3719640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac0cda40 con 0x7fbeac0cffa0 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0d0a90 0x7fbeac06efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0d0a90 0x7fbeac06efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40878/0 (socket says 172.21.15.40:40878) 2024-04-26T21:36:21.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 -- 172.21.15.40:0/1441220784 learned_addr learned my addr 172.21.15.40:0/1441220784 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb1cb6640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac1608f0 0x7fbeac1e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb14b5640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0cffa0 0x7fbeac0c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 -- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0cffa0 msgr2=0x7fbeac0c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0cffa0 0x7fbeac0c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 -- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac1608f0 msgr2=0x7fbeac1e8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac1608f0 0x7fbeac1e8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 -- 172.21.15.40:0/1441220784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac06df00 con 0x7fbeac0d0a90 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb0cb4640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0d0a90 0x7fbeac06efa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbea0035040 tx=0x7fbea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbea9ffb640 1 -- 172.21.15.40:0/1441220784 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0037040 con 0x7fbeac0d0a90 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbea9ffb640 1 -- 172.21.15.40:0/1441220784 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea00044c0 con 0x7fbeac0d0a90 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.226+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac06e1e0 con 0x7fbeac0d0a90 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.227+0000 7fbea9ffb640 1 -- 172.21.15.40:0/1441220784 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0006e10 con 0x7fbeac0d0a90 2024-04-26T21:36:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.227+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac06e7a0 con 0x7fbeac0d0a90 2024-04-26T21:36:21.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.228+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac0694f0 con 0x7fbeac0d0a90 2024-04-26T21:36:21.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.228+0000 7fbea9ffb640 1 -- 172.21.15.40:0/1441220784 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbea0044020 con 0x7fbeac0d0a90 2024-04-26T21:36:21.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.229+0000 7fbea9ffb640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.229+0000 7fbea9ffb640 1 -- 172.21.15.40:0/1441220784 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fbea00abab0 con 0x7fbeac0d0a90 2024-04-26T21:36:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.229+0000 7fbeb14b5640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.229+0000 7fbeb14b5640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbea4005880 tx=0x7fbea4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.233+0000 7fbea9ffb640 1 -- 172.21.15.40:0/1441220784 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbea007a0a0 con 0x7fbeac0d0a90 2024-04-26T21:36:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.418+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbeac0d0470 con 0x7fbeac0d0a90 2024-04-26T21:36:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.419+0000 7fbea9ffb640 1 -- 172.21.15.40:0/1441220784 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13128 (secure 0 0 0) 0x7fbea00799a0 con 0x7fbeac0d0a90 2024-04-26T21:36:21.419 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:21.419 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:18.396116+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:21.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 msgr2=0x7fbe9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbea4005880 tx=0x7fbea4009040 comp rx=0 tx=0).stop 2024-04-26T21:36:21.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0d0a90 msgr2=0x7fbeac06efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0d0a90 0x7fbeac06efa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbea0035040 tx=0x7fbea00046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 shutdown_connections 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac1608f0 0x7fbeac1e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0d0a90 0x7fbeac06efa0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 --2- 172.21.15.40:0/1441220784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0cffa0 0x7fbeac0c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 >> 172.21.15.40:0/1441220784 conn(0x7fbeac0d2ec0 msgr2=0x7fbeac0b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 shutdown_connections 2024-04-26T21:36:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.421+0000 7fbeb3719640 1 -- 172.21.15.40:0/1441220784 wait complete. 2024-04-26T21:36:21.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:36:21.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:36:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.536+0000 7f3fde45b640 1 Processor -- start 2024-04-26T21:36:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.536+0000 7f3fde45b640 1 -- start start 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fde45b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd815f170 0x7f3fd80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fde45b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3fd815f550 0x7f3fd80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fde45b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fd80d1040 con 0x7f3fd80d34b0 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fde45b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3fd80c9430 con 0x7f3fd815f550 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fde45b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd80c9680 con 0x7f3fd815f170 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fd7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd815f170 0x7f3fd80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fd77fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3fd815f550 0x7f3fd80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fd7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd815f170 0x7f3fd80d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:42794/0 (socket says 172.21.15.40:42794) 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fdc9f8640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f3fd80d34b0 0x7f3fd80cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55750/0 (socket says 172.21.15.40:55750) 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fd7fff640 1 -- 172.21.15.40:0/2313371860 learned_addr learned my addr 172.21.15.40:0/2313371860 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fd7fff640 1 -- 172.21.15.40:0/2313371860 >> v2:172.21.15.184:3300/0 conn(0x7f3fd815f550 msgr2=0x7f3fd80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fd7fff640 1 --2- 172.21.15.40:0/2313371860 >> v2:172.21.15.184:3300/0 conn(0x7f3fd815f550 0x7f3fd80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.537+0000 7f3fd7fff640 1 -- 172.21.15.40:0/2313371860 >> v1:172.21.15.184:6789/0 conn(0x7f3fd80d34b0 legacy=0x7f3fd80cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.538+0000 7f3fd7fff640 1 -- 172.21.15.40:0/2313371860 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd80c9960 con 0x7f3fd815f170 2024-04-26T21:36:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.538+0000 7f3fd77fe640 1 --2- 172.21.15.40:0/2313371860 >> v2:172.21.15.184:3300/0 conn(0x7f3fd815f550 0x7f3fd80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.538+0000 7f3fd7fff640 1 --2- 172.21.15.40:0/2313371860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd815f170 0x7f3fd80d5a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009870 tx=0x7f3fc802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9d97ec4a4bfccf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.538+0000 7f3fd6ffd640 1 -- 172.21.15.40:0/2313371860 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8037070 con 0x7f3fd815f170 2024-04-26T21:36:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fd6ffd640 1 -- 172.21.15.40:0/2313371860 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc8032440 con 0x7f3fd815f170 2024-04-26T21:36:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fd6ffd640 1 -- 172.21.15.40:0/2313371860 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8032810 con 0x7f3fd815f170 2024-04-26T21:36:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fde45b640 1 -- 172.21.15.40:0/2313371860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd815f170 msgr2=0x7f3fd80d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2313371860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd815f170 0x7f3fd80d5a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009870 tx=0x7f3fc802a270 comp rx=0 tx=0).stop 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fde45b640 1 -- 172.21.15.40:0/2313371860 shutdown_connections 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2313371860 >> v2:172.21.15.184:3300/0 conn(0x7f3fd815f550 0x7f3fd80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2313371860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd815f170 0x7f3fd80d5a80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fde45b640 1 -- 172.21.15.40:0/2313371860 >> 172.21.15.40:0/2313371860 conn(0x7f3fd80d4f60 msgr2=0x7f3fd80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fde45b640 1 -- 172.21.15.40:0/2313371860 shutdown_connections 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.539+0000 7f3fde45b640 1 -- 172.21.15.40:0/2313371860 wait complete. 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.540+0000 7f3fde45b640 1 Processor -- start 2024-04-26T21:36:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.540+0000 7f3fde45b640 1 -- start start 2024-04-26T21:36:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fde45b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd80d34b0 0x7f3fd8152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fde45b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd815f170 0x7f3fd80a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fde45b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fd80a2960 0x7f3fd80d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fde45b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd80cb5d0 con 0x7f3fd815f170 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fde45b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd80cb450 con 0x7f3fd80a2960 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fde45b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd80cb750 con 0x7f3fd80d34b0 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fd77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd815f170 0x7f3fd80a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fd77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd815f170 0x7f3fd80a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40884/0 (socket says 172.21.15.40:40884) 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fd77fe640 1 -- 172.21.15.40:0/2407356352 learned_addr learned my addr 172.21.15.40:0/2407356352 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fdc9f8640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fd80a2960 0x7f3fd80d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.541+0000 7f3fd7fff640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd80d34b0 0x7f3fd8152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd77fe640 1 -- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd80d34b0 msgr2=0x7f3fd8152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd77fe640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd80d34b0 0x7f3fd8152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd77fe640 1 -- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fd80a2960 msgr2=0x7f3fd80d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd77fe640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fd80a2960 0x7f3fd80d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd77fe640 1 -- 172.21.15.40:0/2407356352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd80cdaa0 con 0x7f3fd815f170 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd77fe640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd815f170 0x7f3fd80a2420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e960 tx=0x7f3fc000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd4ff9640 1 -- 172.21.15.40:0/2407356352 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0017070 con 0x7f3fd815f170 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd4ff9640 1 -- 172.21.15.40:0/2407356352 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc000ca00 con 0x7f3fd815f170 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fd4ff9640 1 -- 172.21.15.40:0/2407356352 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000cdd0 con 0x7f3fd815f170 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd80cdd80 con 0x7f3fd815f170 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.542+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd80ce340 con 0x7f3fd815f170 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.543+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd80694f0 con 0x7f3fd815f170 2024-04-26T21:36:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.544+0000 7f3fd4ff9640 1 -- 172.21.15.40:0/2407356352 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fc001a070 con 0x7f3fd815f170 2024-04-26T21:36:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.544+0000 7f3fd4ff9640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fb806c8f0 0x7f3fb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.544+0000 7f3fd4ff9640 1 -- 172.21.15.40:0/2407356352 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f3fc008bb50 con 0x7f3fd815f170 2024-04-26T21:36:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.544+0000 7f3fd7fff640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fb806c8f0 0x7f3fb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.545+0000 7f3fd7fff640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fb806c8f0 0x7f3fb806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8034000 tx=0x7f3fc802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.549+0000 7f3fd4ff9640 1 -- 172.21.15.40:0/2407356352 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3fc0059af0 con 0x7f3fd815f170 2024-04-26T21:36:21.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.735+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3fd80cf080 con 0x7f3fd815f170 2024-04-26T21:36:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.736+0000 7f3fd4ff9640 1 -- 172.21.15.40:0/2407356352 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f3fc005ddf0 con 0x7f3fd815f170 2024-04-26T21:36:21.736 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:36:21.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fb806c8f0 msgr2=0x7f3fb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fb806c8f0 0x7f3fb806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8034000 tx=0x7f3fc802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:21.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd815f170 msgr2=0x7f3fd80a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd815f170 0x7f3fd80a2420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e960 tx=0x7f3fc000ee20 comp rx=0 tx=0).stop 2024-04-26T21:36:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 shutdown_connections 2024-04-26T21:36:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fb806c8f0 0x7f3fb806edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fd80a2960 0x7f3fd80d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd815f170 0x7f3fd80a2420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 --2- 172.21.15.40:0/2407356352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fd80d34b0 0x7f3fd8152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 >> 172.21.15.40:0/2407356352 conn(0x7f3fd80d4f60 msgr2=0x7f3fd80c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 shutdown_connections 2024-04-26T21:36:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.738+0000 7f3fde45b640 1 -- 172.21.15.40:0/2407356352 wait complete. 2024-04-26T21:36:21.749 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:36:21.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.850+0000 7f4337ff0640 1 Processor -- start 2024-04-26T21:36:21.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.850+0000 7f4337ff0640 1 -- start start 2024-04-26T21:36:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f4337ff0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f4337ff0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43300b5f20 0x7f43300d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f4337ff0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300b7e50 con 0x7f43300d39f0 2024-04-26T21:36:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f4337ff0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f43300d3e40 con 0x7f43300b5f20 2024-04-26T21:36:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f4337ff0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f43300d0580 con 0x7f4330162880 2024-04-26T21:36:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f433558b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43300b5f20 0x7f43300d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f433558b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43300b5f20 0x7f43300d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40894/0 (socket says 172.21.15.40:40894) 2024-04-26T21:36:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f433558b640 1 -- 172.21.15.40:0/475884361 learned_addr learned my addr 172.21.15.40:0/475884361 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f4335d8c640 1 --2- 172.21.15.40:0/475884361 >> v2:172.21.15.184:3300/0 conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f433558b640 1 -- 172.21.15.40:0/475884361 >> v2:172.21.15.184:3300/0 conn(0x7f4330162880 msgr2=0x7f43300d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f433558b640 1 --2- 172.21.15.40:0/475884361 >> v2:172.21.15.184:3300/0 conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f433558b640 1 -- 172.21.15.40:0/475884361 >> v1:172.21.15.184:6789/0 conn(0x7f43300d39f0 legacy=0x7f43300b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f433558b640 1 -- 172.21.15.40:0/475884361 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43300d1800 con 0x7f43300b5f20 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.851+0000 7f4335d8c640 1 --2- 172.21.15.40:0/475884361 >> v2:172.21.15.184:3300/0 conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.852+0000 7f433558b640 1 --2- 172.21.15.40:0/475884361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300b5f20 0x7f43300d34b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f433008d490 tx=0x7f432002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcc761dd18dee21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.852+0000 7f4334d8a640 1 -- 172.21.15.40:0/475884361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320037070 con 0x7f43300b5f20 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.852+0000 7f4334d8a640 1 -- 172.21.15.40:0/475884361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4320032440 con 0x7f43300b5f20 2024-04-26T21:36:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.852+0000 7f4334d8a640 1 -- 172.21.15.40:0/475884361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320032810 con 0x7f43300b5f20 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.853+0000 7f4337ff0640 1 -- 172.21.15.40:0/475884361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300b5f20 msgr2=0x7f43300d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.853+0000 7f4337ff0640 1 --2- 172.21.15.40:0/475884361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300b5f20 0x7f43300d34b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f433008d490 tx=0x7f432002a230 comp rx=0 tx=0).stop 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.853+0000 7f4337ff0640 1 -- 172.21.15.40:0/475884361 shutdown_connections 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.853+0000 7f4337ff0640 1 --2- 172.21.15.40:0/475884361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300b5f20 0x7f43300d34b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.853+0000 7f4337ff0640 1 --2- 172.21.15.40:0/475884361 >> v2:172.21.15.184:3300/0 conn(0x7f4330162880 0x7f43300d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.853+0000 7f4337ff0640 1 -- 172.21.15.40:0/475884361 >> 172.21.15.40:0/475884361 conn(0x7f43300d4f60 msgr2=0x7f43300d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.853+0000 7f4337ff0640 1 -- 172.21.15.40:0/475884361 shutdown_connections 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.853+0000 7f4337ff0640 1 -- 172.21.15.40:0/475884361 wait complete. 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.854+0000 7f4337ff0640 1 Processor -- start 2024-04-26T21:36:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.854+0000 7f4337ff0640 1 -- start start 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4337ff0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43300b5f20 0x7f43300b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4337ff0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f4330153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4337ff0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4337ff0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43300d0d90 con 0x7f43300d39f0 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4337ff0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43300d0ab0 con 0x7f43300b5f20 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4337ff0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4330162c60 con 0x7f4330162880 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f4330153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f4330153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40900/0 (socket says 172.21.15.40:40900) 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 -- 172.21.15.40:0/3455996902 learned_addr learned my addr 172.21.15.40:0/3455996902 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4335d8c640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43300b5f20 0x7f43300b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433658d640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 -- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 msgr2=0x7f43300b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 -- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43300b5f20 msgr2=0x7f43300b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43300b5f20 0x7f43300b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 -- 172.21.15.40:0/3455996902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43300b1b50 con 0x7f43300d39f0 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4335d8c640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43300b5f20 0x7f43300b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433658d640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f433558b640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f4330153430 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4320000c00 tx=0x7f43200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f43267fc640 1 -- 172.21.15.40:0/3455996902 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320041020 con 0x7f43300d39f0 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f43267fc640 1 -- 172.21.15.40:0/3455996902 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4320030970 con 0x7f43300d39f0 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43300b3740 con 0x7f43300d39f0 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f43267fc640 1 -- 172.21.15.40:0/3455996902 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320037040 con 0x7f43300d39f0 2024-04-26T21:36:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.855+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43300b1df0 con 0x7f43300d39f0 2024-04-26T21:36:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.856+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43300694f0 con 0x7f43300d39f0 2024-04-26T21:36:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.857+0000 7f43267fc640 1 -- 172.21.15.40:0/3455996902 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4320044020 con 0x7f43300d39f0 2024-04-26T21:36:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.857+0000 7f43267fc640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.857+0000 7f43267fc640 1 -- 172.21.15.40:0/3455996902 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f43200ac370 con 0x7f43300d39f0 2024-04-26T21:36:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.857+0000 7f4335d8c640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.858+0000 7f4335d8c640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f432c006fd0 tx=0x7f432c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:21.861+0000 7f43267fc640 1 -- 172.21.15.40:0/3455996902 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f432004b030 con 0x7f43300d39f0 2024-04-26T21:36:22.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.051+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f433015eb20 con 0x7f43300d39f0 2024-04-26T21:36:22.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.051+0000 7f43267fc640 1 -- 172.21.15.40:0/3455996902 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f43300d1800 con 0x7f43300d39f0 2024-04-26T21:36:22.052 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:22.052 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:36:22.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:22.052 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:36:22.053 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:36:22.053 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:22.053 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:36:22.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 msgr2=0x7f431406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f432c006fd0 tx=0x7f432c008040 comp rx=0 tx=0).stop 2024-04-26T21:36:22.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 msgr2=0x7f4330153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f4330153430 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4320000c00 tx=0x7f43200046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 shutdown_connections 2024-04-26T21:36:22.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4330162880 0x7f43300b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43300d39f0 0x7f4330153430 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 --2- 172.21.15.40:0/3455996902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43300b5f20 0x7f43300b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 >> 172.21.15.40:0/3455996902 conn(0x7f43300d4f60 msgr2=0x7f43300d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:22.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 shutdown_connections 2024-04-26T21:36:22.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.053+0000 7f4337ff0640 1 -- 172.21.15.40:0/3455996902 wait complete. 2024-04-26T21:36:22.064 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:36:22.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:36:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.170+0000 7f8a88eda640 1 Processor -- start 2024-04-26T21:36:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.170+0000 7f8a88eda640 1 -- start start 2024-04-26T21:36:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a88eda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 0x7f8a840d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a88eda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a8415fe30 0x7f8a840cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a88eda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a84163160 con 0x7f8a841606c0 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a88eda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8a84162f10 con 0x7f8a8415fe30 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a88eda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a84163d70 con 0x7f8a8415f340 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a81d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8a841606c0 0x7f8a8415f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49984/0 (socket says 172.21.15.40:49984) 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a81d9b640 1 -- 172.21.15.40:0/1965149332 learned_addr learned my addr 172.21.15.40:0/1965149332 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a82d9d640 1 --2- 172.21.15.40:0/1965149332 >> v2:172.21.15.184:3300/0 conn(0x7f8a8415fe30 0x7f8a840cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.171+0000 7f8a8259c640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 0x7f8a840d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2301515022 0 0) 0x7f8a84163160 con 0x7f8a841606c0 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a70003520 con 0x7f8a841606c0 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2857527985 0 0) 0x7f8a70003520 con 0x7f8a841606c0 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 msgr2=0x7f8a840d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 0x7f8a840d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 >> v2:172.21.15.184:3300/0 conn(0x7f8a8415fe30 msgr2=0x7f8a840cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 --2- 172.21.15.40:0/1965149332 >> v2:172.21.15.184:3300/0 conn(0x7f8a8415fe30 0x7f8a840cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a84163440 con 0x7f8a841606c0 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f8a6c001670 con 0x7f8a841606c0 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 >> v1:172.21.15.40:6789/0 conn(0x7f8a841606c0 legacy=0x7f8a8415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a700039b0 0x7f8a70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 0x7f8a70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a70004d40 0x7f8a700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84162f10 con 0x7f8a700039b0 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84163d70 con 0x7f8a70004d40 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a7000b9c0 con 0x7f8a8415f340 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8259c640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a700039b0 0x7f8a70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a82d9d640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a70004d40 0x7f8a700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a81d9b640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 0x7f8a70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8259c640 1 -- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 msgr2=0x7f8a70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8259c640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 0x7f8a70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8259c640 1 -- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a70004d40 msgr2=0x7f8a700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8259c640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a70004d40 0x7f8a700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a8259c640 1 -- 172.21.15.40:0/1965149332 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a70003520 con 0x7f8a700039b0 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.172+0000 7f8a81d9b640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8415f340 0x7f8a70004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.173+0000 7f8a8259c640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a700039b0 0x7f8a70003dd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009fc0 tx=0x7f8a70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b54417323ec2c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.173+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74009c10 con 0x7f8a700039b0 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.173+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a74036070 con 0x7f8a700039b0 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.173+0000 7f8a8159a640 1 -- 172.21.15.40:0/1965149332 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74031880 con 0x7f8a700039b0 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.174+0000 7f8a88eda640 1 -- 172.21.15.40:0/1965149332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a700039b0 msgr2=0x7f8a70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.174+0000 7f8a88eda640 1 --2- 172.21.15.40:0/1965149332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a700039b0 0x7f8a70003dd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009fc0 tx=0x7f8a70005830 comp rx=0 tx=0).stop 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.174+0000 7f8a82d9d640 1 -- 172.21.15.40:0/1965149332 reap_dead start 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.174+0000 7f8a88eda640 1 -- 172.21.15.40:0/1965149332 shutdown_connections 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.174+0000 7f8a88eda640 1 -- 172.21.15.40:0/1965149332 >> 172.21.15.40:0/1965149332 conn(0x7f8a8409ba50 msgr2=0x7f8a8409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.174+0000 7f8a88eda640 1 -- 172.21.15.40:0/1965149332 shutdown_connections 2024-04-26T21:36:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.174+0000 7f8a88eda640 1 -- 172.21.15.40:0/1965149332 wait complete. 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.175+0000 7f8a88eda640 1 Processor -- start 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.175+0000 7f8a88eda640 1 -- start start 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a88eda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a700039b0 0x7f8a840ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a88eda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a840cc550 0x7f8a840b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a88eda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8414c480 0x7f8a8414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a88eda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841602a0 con 0x7f8a840cc550 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a88eda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84163a00 con 0x7f8a700039b0 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a88eda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a840d2550 con 0x7f8a8414c480 2024-04-26T21:36:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a840cc550 0x7f8a840b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a840cc550 0x7f8a840b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40922/0 (socket says 172.21.15.40:40922) 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 -- 172.21.15.40:0/4032925043 learned_addr learned my addr 172.21.15.40:0/4032925043 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a8259c640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a700039b0 0x7f8a840ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a82d9d640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8414c480 0x7f8a8414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 -- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8414c480 msgr2=0x7f8a8414c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8414c480 0x7f8a8414c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 -- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a700039b0 msgr2=0x7f8a840ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a700039b0 0x7f8a840ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 -- 172.21.15.40:0/4032925043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a840cd690 con 0x7f8a840cc550 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a82d9d640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8414c480 0x7f8a8414c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a8259c640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a700039b0 0x7f8a840ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.176+0000 7f8a81d9b640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a840cc550 0x7f8a840b7330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00e7f0 tx=0x7f8a6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.177+0000 7f8a7affd640 1 -- 172.21.15.40:0/4032925043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00c8d0 con 0x7f8a840cc550 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.177+0000 7f8a7affd640 1 -- 172.21.15.40:0/4032925043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a6c018070 con 0x7f8a840cc550 2024-04-26T21:36:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.177+0000 7f8a7affd640 1 -- 172.21.15.40:0/4032925043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00cc70 con 0x7f8a840cc550 2024-04-26T21:36:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.177+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8414ce40 con 0x7f8a840cc550 2024-04-26T21:36:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.177+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a840ccd60 con 0x7f8a840cc550 2024-04-26T21:36:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.178+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a84069600 con 0x7f8a840cc550 2024-04-26T21:36:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.178+0000 7f8a7affd640 1 -- 172.21.15.40:0/4032925043 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a6c01d460 con 0x7f8a840cc550 2024-04-26T21:36:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.179+0000 7f8a7affd640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a6406c920 0x7f8a6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.179+0000 7f8a7affd640 1 -- 172.21.15.40:0/4032925043 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f8a6c08b910 con 0x7f8a840cc550 2024-04-26T21:36:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.179+0000 7f8a8259c640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a6406c920 0x7f8a6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.179+0000 7f8a8259c640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a6406c920 0x7f8a6406ede0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8a74035000 tx=0x7f8a7402f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.183+0000 7f8a7affd640 1 -- 172.21.15.40:0/4032925043 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a6c059f00 con 0x7f8a840cc550 2024-04-26T21:36:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.366+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8a84068e00 con 0x7f8a840cc550 2024-04-26T21:36:22.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.400+0000 7f8a7affd640 1 -- 172.21.15.40:0/4032925043 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f8a6c0600d0 con 0x7f8a840cc550 2024-04-26T21:36:22.400 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:36:22.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a6406c920 msgr2=0x7f8a6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a6406c920 0x7f8a6406ede0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8a74035000 tx=0x7f8a7402f000 comp rx=0 tx=0).stop 2024-04-26T21:36:22.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a840cc550 msgr2=0x7f8a840b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a840cc550 0x7f8a840b7330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00e7f0 tx=0x7f8a6c00ecb0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 shutdown_connections 2024-04-26T21:36:22.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a6406c920 0x7f8a6406ede0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8414c480 0x7f8a8414c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a840cc550 0x7f8a840b7330 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 --2- 172.21.15.40:0/4032925043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a700039b0 0x7f8a840ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 >> 172.21.15.40:0/4032925043 conn(0x7f8a8409ba50 msgr2=0x7f8a84161fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:22.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 shutdown_connections 2024-04-26T21:36:22.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.402+0000 7f8a88eda640 1 -- 172.21.15.40:0/4032925043 wait complete. 2024-04-26T21:36:22.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:36:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.524+0000 7f37a6c99640 1 Processor -- start 2024-04-26T21:36:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.524+0000 7f37a6c99640 1 -- start start 2024-04-26T21:36:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f37a6c99640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f37a00cfde0 0x7f37a00d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f37a6c99640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37a0070350 0x7f37a0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f37a6c99640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37a00d2390 con 0x7f37a0071090 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f37a6c99640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f37a00afab0 con 0x7f37a0070350 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f37a6c99640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f37a006dd20 con 0x7f37a00cfde0 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f379ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37a0070350 0x7f37a0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f379ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37a0070350 0x7f37a0070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40924/0 (socket says 172.21.15.40:40924) 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f379ffff640 1 -- 172.21.15.40:0/925480871 learned_addr learned my addr 172.21.15.40:0/925480871 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f379ffff640 1 -- 172.21.15.40:0/925480871 >> v2:172.21.15.184:3300/0 conn(0x7f37a00cfde0 msgr2=0x7f37a00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f37a4a35640 1 --2- 172.21.15.40:0/925480871 >> v2:172.21.15.184:3300/0 conn(0x7f37a00cfde0 0x7f37a00d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f379ffff640 1 --2- 172.21.15.40:0/925480871 >> v2:172.21.15.184:3300/0 conn(0x7f37a00cfde0 0x7f37a00d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f379ffff640 1 -- 172.21.15.40:0/925480871 >> v1:172.21.15.184:6789/0 conn(0x7f37a0071090 legacy=0x7f37a015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f379ffff640 1 -- 172.21.15.40:0/925480871 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a006d3f0 con 0x7f37a0070350 2024-04-26T21:36:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.525+0000 7f37a4a35640 1 --2- 172.21.15.40:0/925480871 >> v2:172.21.15.184:3300/0 conn(0x7f37a00cfde0 0x7f37a00d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.526+0000 7f379ffff640 1 --2- 172.21.15.40:0/925480871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a0070b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f37a00762b0 tx=0x7f379402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93c225e70fba9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.526+0000 7f379f7fe640 1 -- 172.21.15.40:0/925480871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794037070 con 0x7f37a0070350 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.526+0000 7f379f7fe640 1 -- 172.21.15.40:0/925480871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3794032440 con 0x7f37a0070350 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.526+0000 7f379f7fe640 1 -- 172.21.15.40:0/925480871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794032810 con 0x7f37a0070350 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.527+0000 7f37a6c99640 1 -- 172.21.15.40:0/925480871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 msgr2=0x7f37a0070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.527+0000 7f37a6c99640 1 --2- 172.21.15.40:0/925480871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a0070b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f37a00762b0 tx=0x7f379402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.527+0000 7f37a6c99640 1 -- 172.21.15.40:0/925480871 shutdown_connections 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.527+0000 7f37a6c99640 1 --2- 172.21.15.40:0/925480871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a0070b50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.527+0000 7f37a6c99640 1 --2- 172.21.15.40:0/925480871 >> v2:172.21.15.184:3300/0 conn(0x7f37a00cfde0 0x7f37a00d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.527+0000 7f37a6c99640 1 -- 172.21.15.40:0/925480871 >> 172.21.15.40:0/925480871 conn(0x7f37a00d2d80 msgr2=0x7f37a00d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.527+0000 7f37a6c99640 1 -- 172.21.15.40:0/925480871 shutdown_connections 2024-04-26T21:36:22.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.527+0000 7f37a6c99640 1 -- 172.21.15.40:0/925480871 wait complete. 2024-04-26T21:36:22.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.528+0000 7f37a6c99640 1 Processor -- start 2024-04-26T21:36:22.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.528+0000 7f37a6c99640 1 -- start start 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a6c99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a01f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a6c99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0071090 0x7f37a01f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a6c99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a00cfde0 0x7f37a01f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a6c99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37a006d9b0 con 0x7f37a0070350 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a6c99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37a00714c0 con 0x7f37a00cfde0 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a6c99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37a00d20a0 con 0x7f37a0071090 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a01f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a01f4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40930/0 (socket says 172.21.15.40:40930) 2024-04-26T21:36:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 -- 172.21.15.40:0/822948273 learned_addr learned my addr 172.21.15.40:0/822948273 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a5236640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a00cfde0 0x7f37a01f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f379ffff640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0071090 0x7f37a01f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 -- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0071090 msgr2=0x7f37a01f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0071090 0x7f37a01f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 -- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a00cfde0 msgr2=0x7f37a01f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a00cfde0 0x7f37a01f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 -- 172.21.15.40:0/822948273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a01f8b90 con 0x7f37a0070350 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f379ffff640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0071090 0x7f37a01f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a4a35640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a01f4770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f379000c7f0 tx=0x7f379000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f379d7fa640 1 -- 172.21.15.40:0/822948273 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790015070 con 0x7f37a0070350 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f379d7fa640 1 -- 172.21.15.40:0/822948273 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3790007a30 con 0x7f37a0070350 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a01f8e70 con 0x7f37a0070350 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.529+0000 7f379d7fa640 1 -- 172.21.15.40:0/822948273 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790007e00 con 0x7f37a0070350 2024-04-26T21:36:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.530+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a01f4d80 con 0x7f37a0070350 2024-04-26T21:36:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.530+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a00694f0 con 0x7f37a0070350 2024-04-26T21:36:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.531+0000 7f379d7fa640 1 -- 172.21.15.40:0/822948273 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3790018070 con 0x7f37a0070350 2024-04-26T21:36:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.532+0000 7f379d7fa640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f378c06c8f0 0x7f378c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.532+0000 7f379d7fa640 1 -- 172.21.15.40:0/822948273 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f379008baf0 con 0x7f37a0070350 2024-04-26T21:36:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.532+0000 7f379ffff640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f378c06c8f0 0x7f378c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.532+0000 7f379ffff640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f378c06c8f0 0x7f378c06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3794034000 tx=0x7f379402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:22.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.536+0000 7f379d7fa640 1 -- 172.21.15.40:0/822948273 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3790059ef0 con 0x7f37a0070350 2024-04-26T21:36:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:22.721+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f37a00b2320 con 0x7f37a0070350 2024-04-26T21:36:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.402+0000 7f379d7fa640 1 -- 172.21.15.40:0/822948273 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f37900597f0 con 0x7f37a0070350 2024-04-26T21:36:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:36:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.405+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f378c06c8f0 msgr2=0x7f378c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.405+0000 7f37a6c99640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f378c06c8f0 0x7f378c06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3794034000 tx=0x7f379402e000 comp rx=0 tx=0).stop 2024-04-26T21:36:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.405+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 msgr2=0x7f37a01f4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.405+0000 7f37a6c99640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a01f4770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f379000c7f0 tx=0x7f379000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.406+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 shutdown_connections 2024-04-26T21:36:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.406+0000 7f37a6c99640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f378c06c8f0 0x7f378c06edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.406+0000 7f37a6c99640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a00cfde0 0x7f37a01f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.406+0000 7f37a6c99640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0071090 0x7f37a01f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.406+0000 7f37a6c99640 1 --2- 172.21.15.40:0/822948273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a0070350 0x7f37a01f4770 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.406+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 >> 172.21.15.40:0/822948273 conn(0x7f37a00d2d80 msgr2=0x7f37a00cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.406+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 shutdown_connections 2024-04-26T21:36:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.407+0000 7f37a6c99640 1 -- 172.21.15.40:0/822948273 wait complete. 2024-04-26T21:36:23.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:36:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.543+0000 7fe77f4d1640 1 Processor -- start 2024-04-26T21:36:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.543+0000 7fe77f4d1640 1 -- start start 2024-04-26T21:36:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77f4d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe778071e60 0x7fe77806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77f4d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 0x7fe778072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77f4d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe778072780 0x7fe7780cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77f4d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe77815d7c0 con 0x7fe778071e60 2024-04-26T21:36:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77f4d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe77815daa0 con 0x7fe778072780 2024-04-26T21:36:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77f4d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe77815dd80 con 0x7fe77806fe20 2024-04-26T21:36:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77d26d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe778071e60 0x7fe77806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77d26d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe778071e60 0x7fe77806f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40936/0 (socket says 172.21.15.40:40936) 2024-04-26T21:36:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77d26d640 1 -- 172.21.15.40:0/3541423149 learned_addr learned my addr 172.21.15.40:0/3541423149 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77da6e640 1 --2- 172.21.15.40:0/3541423149 >> v2:172.21.15.184:3300/0 conn(0x7fe778072780 0x7fe7780cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77ca6c640 1 --2- 172.21.15.40:0/3541423149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 0x7fe778072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77d26d640 1 -- 172.21.15.40:0/3541423149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 msgr2=0x7fe778072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77d26d640 1 --2- 172.21.15.40:0/3541423149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 0x7fe778072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77d26d640 1 -- 172.21.15.40:0/3541423149 >> v2:172.21.15.184:3300/0 conn(0x7fe778072780 msgr2=0x7fe7780cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77d26d640 1 --2- 172.21.15.40:0/3541423149 >> v2:172.21.15.184:3300/0 conn(0x7fe778072780 0x7fe7780cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.544+0000 7fe77d26d640 1 -- 172.21.15.40:0/3541423149 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77815e060 con 0x7fe778071e60 2024-04-26T21:36:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.545+0000 7fe77d26d640 1 --2- 172.21.15.40:0/3541423149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe77806f8e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe77806e5c0 tx=0x7fe77402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01fe5195b7759c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.545+0000 7fe76ffff640 1 -- 172.21.15.40:0/3541423149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774037070 con 0x7fe778071e60 2024-04-26T21:36:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.545+0000 7fe76ffff640 1 -- 172.21.15.40:0/3541423149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe774032440 con 0x7fe778071e60 2024-04-26T21:36:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.545+0000 7fe76ffff640 1 -- 172.21.15.40:0/3541423149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774032810 con 0x7fe778071e60 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.545+0000 7fe77f4d1640 1 -- 172.21.15.40:0/3541423149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 msgr2=0x7fe77806f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.545+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/3541423149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe77806f8e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe77806e5c0 tx=0x7fe77402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.546+0000 7fe77f4d1640 1 -- 172.21.15.40:0/3541423149 shutdown_connections 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.546+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/3541423149 >> v2:172.21.15.184:3300/0 conn(0x7fe778072780 0x7fe7780cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.546+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/3541423149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 0x7fe778072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.546+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/3541423149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe77806f8e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.546+0000 7fe77f4d1640 1 -- 172.21.15.40:0/3541423149 >> 172.21.15.40:0/3541423149 conn(0x7fe7780d0be0 msgr2=0x7fe7780d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.546+0000 7fe77f4d1640 1 -- 172.21.15.40:0/3541423149 shutdown_connections 2024-04-26T21:36:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.546+0000 7fe77f4d1640 1 -- 172.21.15.40:0/3541423149 wait complete. 2024-04-26T21:36:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.547+0000 7fe77f4d1640 1 Processor -- start 2024-04-26T21:36:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.547+0000 7fe77f4d1640 1 -- start start 2024-04-26T21:36:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77f4d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 0x7fe778151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77f4d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe7780af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77f4d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe778072780 0x7fe7780cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77f4d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe77815d3f0 con 0x7fe778071e60 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77f4d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe778072c10 con 0x7fe778072780 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77f4d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe77815d570 con 0x7fe77806fe20 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe7780af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe7780af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40946/0 (socket says 172.21.15.40:40946) 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 -- 172.21.15.40:0/563238153 learned_addr learned my addr 172.21.15.40:0/563238153 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77d26d640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 0x7fe778151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77da6e640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe778072780 0x7fe7780cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 -- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 msgr2=0x7fe778151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 0x7fe778151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 -- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe778072780 msgr2=0x7fe7780cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe778072780 0x7fe7780cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 -- 172.21.15.40:0/563238153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe778151750 con 0x7fe778071e60 2024-04-26T21:36:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77da6e640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe778072780 0x7fe7780cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77ca6c640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe7780af2e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe76800d800 tx=0x7fe76800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe76dffb640 1 -- 172.21.15.40:0/563238153 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768018020 con 0x7fe778071e60 2024-04-26T21:36:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe76dffb640 1 -- 172.21.15.40:0/563238153 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7680096a0 con 0x7fe778071e60 2024-04-26T21:36:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7780afda0 con 0x7fe778071e60 2024-04-26T21:36:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.548+0000 7fe76dffb640 1 -- 172.21.15.40:0/563238153 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768009a70 con 0x7fe778071e60 2024-04-26T21:36:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.549+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7780c7e50 con 0x7fe778071e60 2024-04-26T21:36:23.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.550+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7780b02c0 con 0x7fe778071e60 2024-04-26T21:36:23.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.550+0000 7fe76dffb640 1 -- 172.21.15.40:0/563238153 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe76801b020 con 0x7fe778071e60 2024-04-26T21:36:23.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.551+0000 7fe76dffb640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c8f0 0x7fe75c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:23.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.551+0000 7fe76dffb640 1 -- 172.21.15.40:0/563238153 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7fe76808bdb0 con 0x7fe778071e60 2024-04-26T21:36:23.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.551+0000 7fe77d26d640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c8f0 0x7fe75c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:23.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.552+0000 7fe77d26d640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c8f0 0x7fe75c06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe774034000 tx=0x7fe77402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.556+0000 7fe76dffb640 1 -- 172.21.15.40:0/563238153 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe768022030 con 0x7fe778071e60 2024-04-26T21:36:23.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:23.779+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe7780694f0 con 0x7fe778071e60 2024-04-26T21:36:24.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.424+0000 7fe76dffb640 1 -- 172.21.15.40:0/563238153 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7fe76805a140 con 0x7fe778071e60 2024-04-26T21:36:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 5 and data pool 6 2024-04-26T21:36:24.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.435+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c8f0 msgr2=0x7fe75c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.435+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c8f0 0x7fe75c06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe774034000 tx=0x7fe77402e000 comp rx=0 tx=0).stop 2024-04-26T21:36:24.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.435+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 msgr2=0x7fe7780af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.435+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe7780af2e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe76800d800 tx=0x7fe76800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.439+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 shutdown_connections 2024-04-26T21:36:24.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.439+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c8f0 0x7fe75c06edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.439+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe778072780 0x7fe7780cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.439+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe778071e60 0x7fe7780af2e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.439+0000 7fe77f4d1640 1 --2- 172.21.15.40:0/563238153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77806fe20 0x7fe778151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.439+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 >> 172.21.15.40:0/563238153 conn(0x7fe7780d0be0 msgr2=0x7fe77815ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.440+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 shutdown_connections 2024-04-26T21:36:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.440+0000 7fe77f4d1640 1 -- 172.21.15.40:0/563238153 wait complete. 2024-04-26T21:36:24.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:24.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.565+0000 7f75be420640 1 Processor -- start 2024-04-26T21:36:24.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.565+0000 7f75be420640 1 -- start start 2024-04-26T21:36:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.565+0000 7f75be420640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f75b815f3a0 0x7f75b80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.565+0000 7f75be420640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b80d0e20 con 0x7f75b80d2330 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.565+0000 7f75be420640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b80b1d70 con 0x7f75b80d1770 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.565+0000 7f75be420640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f75b80b2050 con 0x7f75b815f3a0 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75bc9bd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f75b80d2330 0x7f75b80b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49986/0 (socket says 172.21.15.40:49986) 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75bc9bd640 1 -- 172.21.15.40:0/919018745 learned_addr learned my addr 172.21.15.40:0/919018745 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b7fff640 1 --2- 172.21.15.40:0/919018745 >> v2:172.21.15.184:3300/0 conn(0x7f75b815f3a0 0x7f75b80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672384958 0 0) 0x7f75b80d0e20 con 0x7f75b80d2330 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75a4003520 con 0x7f75b80d2330 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2325153675 0 0) 0x7f75a4003520 con 0x7f75b80d2330 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 >> v2:172.21.15.184:3300/0 conn(0x7f75b815f3a0 msgr2=0x7f75b80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 --2- 172.21.15.40:0/919018745 >> v2:172.21.15.184:3300/0 conn(0x7f75b815f3a0 0x7f75b80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 >> v1:172.21.15.184:6789/0 conn(0x7f75b80d1770 legacy=0x7f75b80d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b80b2330 con 0x7f75b80d2330 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f75b0001a10 con 0x7f75b80d2330 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 >> v1:172.21.15.40:6789/0 conn(0x7f75b80d2330 legacy=0x7f75b80b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75a40039b0 0x7f75a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75a4004800 0x7f75a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75b80b2050 con 0x7f75a40051c0 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75a400c060 con 0x7f75a4004800 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75a400c260 con 0x7f75a40039b0 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75bc9bd640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b77fe640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75a4004800 0x7f75a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.566+0000 7f75b7fff640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75a40039b0 0x7f75a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.567+0000 7f75bc9bd640 1 -- 172.21.15.40:0/919018745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75a40039b0 msgr2=0x7f75a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.567+0000 7f75bc9bd640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75a40039b0 0x7f75a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.567+0000 7f75bc9bd640 1 -- 172.21.15.40:0/919018745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75a4004800 msgr2=0x7f75a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.567+0000 7f75bc9bd640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75a4004800 0x7f75a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.567+0000 7f75bc9bd640 1 -- 172.21.15.40:0/919018745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a4003520 con 0x7f75a40051c0 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.567+0000 7f75bc9bd640 1 -- 172.21.15.40:0/919018745 reap_dead start 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.567+0000 7f75bc9bd640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f75b000a8f0 tx=0x7f75a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159386b73a952f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b000a4c0 con 0x7f75a40051c0 2024-04-26T21:36:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b002f070 con 0x7f75a40051c0 2024-04-26T21:36:24.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75b6ffd640 1 -- 172.21.15.40:0/919018745 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b002bcb0 con 0x7f75a40051c0 2024-04-26T21:36:24.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75be420640 1 -- 172.21.15.40:0/919018745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 msgr2=0x7f75a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75be420640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f75b000a8f0 tx=0x7f75a4005830 comp rx=0 tx=0).stop 2024-04-26T21:36:24.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75be420640 1 -- 172.21.15.40:0/919018745 shutdown_connections 2024-04-26T21:36:24.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75be420640 1 --2- 172.21.15.40:0/919018745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75be420640 1 -- 172.21.15.40:0/919018745 >> 172.21.15.40:0/919018745 conn(0x7f75b80d4f90 msgr2=0x7f75b80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:24.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75be420640 1 -- 172.21.15.40:0/919018745 shutdown_connections 2024-04-26T21:36:24.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.568+0000 7f75be420640 1 -- 172.21.15.40:0/919018745 wait complete. 2024-04-26T21:36:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.569+0000 7f75be420640 1 Processor -- start 2024-04-26T21:36:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75be420640 1 -- start start 2024-04-26T21:36:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75be420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75b80b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75be420640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75b80b7b80 0x7f75b80ba200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75be420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75b80b5ea0 0x7f75b80ba740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75be420640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75b80d0870 con 0x7f75a40051c0 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75be420640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75b80d06f0 con 0x7f75b80b7b80 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75be420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75b80d09f0 con 0x7f75b80b5ea0 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75b80b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75b80b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40962/0 (socket says 172.21.15.40:40962) 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.570+0000 7f75b7fff640 1 -- 172.21.15.40:0/2878756757 learned_addr learned my addr 172.21.15.40:0/2878756757 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75bc9bd640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75b80b5ea0 0x7f75b80ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b77fe640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75b80b7b80 0x7f75b80ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b7fff640 1 -- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75b80b5ea0 msgr2=0x7f75b80ba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b7fff640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75b80b5ea0 0x7f75b80ba740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b7fff640 1 -- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75b80b7b80 msgr2=0x7f75b80ba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b7fff640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75b80b7b80 0x7f75b80ba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b7fff640 1 -- 172.21.15.40:0/2878756757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b80c8b60 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b7fff640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75b80b4c70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f75a00118e0 tx=0x7f75a0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b4ff9640 1 -- 172.21.15.40:0/2878756757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a000faf0 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b4ff9640 1 -- 172.21.15.40:0/2878756757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a000fc90 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75b4ff9640 1 -- 172.21.15.40:0/2878756757 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0014440 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b80b8010 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.571+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b80b85d0 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.572+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b80694f0 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.573+0000 7f75b4ff9640 1 -- 172.21.15.40:0/2878756757 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75a0012040 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.573+0000 7f75b4ff9640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f759806c8f0 0x7f759806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.573+0000 7f75b4ff9640 1 -- 172.21.15.40:0/2878756757 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f75a008c2d0 con 0x7f75a40051c0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.573+0000 7f75b77fe640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f759806c8f0 0x7f759806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.574+0000 7f75b77fe640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f759806c8f0 0x7f759806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f75b80ced20 tx=0x7f75a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.577+0000 7f75b4ff9640 1 -- 172.21.15.40:0/2878756757 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75a0019030 con 0x7f75a40051c0 2024-04-26T21:36:24.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.762+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75b80b6860 con 0x7f75a40051c0 2024-04-26T21:36:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.764+0000 7f75b4ff9640 1 -- 172.21.15.40:0/2878756757 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16919 (secure 0 0 0) 0x7f75a0059f20 con 0x7f75a40051c0 2024-04-26T21:36:24.764 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:24.764 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":37,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:23.780187+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:24.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f759806c8f0 msgr2=0x7f759806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f759806c8f0 0x7f759806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f75b80ced20 tx=0x7f75a8009040 comp rx=0 tx=0).stop 2024-04-26T21:36:24.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 msgr2=0x7f75b80b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75b80b4c70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f75a00118e0 tx=0x7f75a0011da0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 shutdown_connections 2024-04-26T21:36:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f759806c8f0 0x7f759806edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75b80b5ea0 0x7f75b80ba740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75b80b7b80 0x7f75b80ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 --2- 172.21.15.40:0/2878756757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75a40051c0 0x7f75b80b4c70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 >> 172.21.15.40:0/2878756757 conn(0x7f75b80d4f90 msgr2=0x7f75b80d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 shutdown_connections 2024-04-26T21:36:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.766+0000 7f75be420640 1 -- 172.21.15.40:0/2878756757 wait complete. 2024-04-26T21:36:24.777 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:24.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.878+0000 7f25040b0640 1 Processor -- start 2024-04-26T21:36:24.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.878+0000 7f25040b0640 1 -- start start 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f25040b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24fc0b5f20 0x7f24fc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f25040b0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24fc0b7e50 con 0x7f24fc0d3bc0 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f25040b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24fc0d1800 con 0x7f24fc162880 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f25040b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f24fc0d0580 con 0x7f24fc0b5f20 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f250164b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f24fc0d3bc0 0x7f24fc0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49988/0 (socket says 172.21.15.40:49988) 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f250164b640 1 -- 172.21.15.40:0/305634113 learned_addr learned my addr 172.21.15.40:0/305634113 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f250264d640 1 --2- 172.21.15.40:0/305634113 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0b5f20 0x7f24fc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f2500e4a640 1 -- 172.21.15.40:0/305634113 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176879636 0 0) 0x7f24fc0b7e50 con 0x7f24fc0d3bc0 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.879+0000 7f2500e4a640 1 -- 172.21.15.40:0/305634113 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24e4003520 con 0x7f24fc0d3bc0 2024-04-26T21:36:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.880+0000 7f2500e4a640 1 -- 172.21.15.40:0/305634113 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468240229 0 0) 0x7f24f4001110 con 0x7f24fc162880 2024-04-26T21:36:24.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.880+0000 7f2500e4a640 1 -- 172.21.15.40:0/305634113 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24fc0b7e50 con 0x7f24fc162880 2024-04-26T21:36:24.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.880+0000 7f250264d640 1 -- 172.21.15.40:0/305634113 >> v1:172.21.15.184:6789/0 conn(0x7f24fc162880 legacy=0x7f24fc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.880+0000 7f250264d640 1 -- 172.21.15.40:0/305634113 >> v1:172.21.15.40:6789/0 conn(0x7f24fc0d3bc0 legacy=0x7f24fc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.880+0000 7f250264d640 1 -- 172.21.15.40:0/305634113 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc0d07d0 con 0x7f24fc0b5f20 2024-04-26T21:36:24.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.880+0000 7f250264d640 1 --2- 172.21.15.40:0/305634113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 0x7f24fc0d2cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009d90 tx=0x7f24ec02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5de25042727ea19f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f2500e4a640 1 -- 172.21.15.40:0/305634113 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec007740 con 0x7f24fc0b5f20 2024-04-26T21:36:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f2500e4a640 1 -- 172.21.15.40:0/305634113 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ec0078e0 con 0x7f24fc0b5f20 2024-04-26T21:36:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f2500e4a640 1 -- 172.21.15.40:0/305634113 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec007c90 con 0x7f24fc0b5f20 2024-04-26T21:36:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f25040b0640 1 -- 172.21.15.40:0/305634113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 msgr2=0x7f24fc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f25040b0640 1 --2- 172.21.15.40:0/305634113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 0x7f24fc0d2cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009d90 tx=0x7f24ec02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f25040b0640 1 -- 172.21.15.40:0/305634113 shutdown_connections 2024-04-26T21:36:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f25040b0640 1 --2- 172.21.15.40:0/305634113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 0x7f24fc0d2cb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f25040b0640 1 -- 172.21.15.40:0/305634113 >> 172.21.15.40:0/305634113 conn(0x7f24fc0d4f60 msgr2=0x7f24fc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f25040b0640 1 -- 172.21.15.40:0/305634113 shutdown_connections 2024-04-26T21:36:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.881+0000 7f25040b0640 1 -- 172.21.15.40:0/305634113 wait complete. 2024-04-26T21:36:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.882+0000 7f25040b0640 1 Processor -- start 2024-04-26T21:36:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.882+0000 7f25040b0640 1 -- start start 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f25040b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 0x7f24fc1f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f25040b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3bc0 0x7f24fc1f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f25040b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc162880 0x7f24fc0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f25040b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc162c60 con 0x7f24fc162880 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f25040b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc0d0d90 con 0x7f24fc0b5f20 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f25040b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc162de0 con 0x7f24fc0d3bc0 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f250264d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc162880 0x7f24fc0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f250264d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc162880 0x7f24fc0cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40974/0 (socket says 172.21.15.40:40974) 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f250264d640 1 -- 172.21.15.40:0/1870683695 learned_addr learned my addr 172.21.15.40:0/1870683695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f2501e4c640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 0x7f24fc1f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.883+0000 7f250164b640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3bc0 0x7f24fc1f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f250264d640 1 -- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3bc0 msgr2=0x7f24fc1f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f250264d640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3bc0 0x7f24fc1f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f250264d640 1 -- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 msgr2=0x7f24fc1f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f250264d640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 0x7f24fc1f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f250264d640 1 -- 172.21.15.40:0/1870683695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc0cd9f0 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f2501e4c640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 0x7f24fc1f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f250264d640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc162880 0x7f24fc0cdc70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009eb0 tx=0x7f24ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f24fa7fc640 1 -- 172.21.15.40:0/1870683695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec009450 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f24fa7fc640 1 -- 172.21.15.40:0/1870683695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ec004260 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f24fa7fc640 1 -- 172.21.15.40:0/1870683695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec004650 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc0b1c80 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.884+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc0b2240 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.885+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24fc0694f0 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.886+0000 7f24fa7fc640 1 -- 172.21.15.40:0/1870683695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24ec039040 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.886+0000 7f24fa7fc640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24e806c920 0x7f24e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.886+0000 7f24fa7fc640 1 -- 172.21.15.40:0/1870683695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f24ec0acff0 con 0x7f24fc162880 2024-04-26T21:36:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.886+0000 7f2501e4c640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24e806c920 0x7f24e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:24.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.887+0000 7f2501e4c640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24e806c920 0x7f24e806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24f4007fd0 tx=0x7f24f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:24.891+0000 7f24fa7fc640 1 -- 172.21.15.40:0/1870683695 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24ec040030 con 0x7f24fc162880 2024-04-26T21:36:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.080+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24fc06e8d0 con 0x7f24fc162880 2024-04-26T21:36:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.081+0000 7f24fa7fc640 1 -- 172.21.15.40:0/1870683695 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16919 (secure 0 0 0) 0x7f24ec07adf0 con 0x7f24fc162880 2024-04-26T21:36:25.081 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:25.081 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":37,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:23.780187+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:25.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24e806c920 msgr2=0x7f24e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24e806c920 0x7f24e806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24f4007fd0 tx=0x7f24f4009040 comp rx=0 tx=0).stop 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc162880 msgr2=0x7f24fc0cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc162880 0x7f24fc0cdc70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009eb0 tx=0x7f24ec004a40 comp rx=0 tx=0).stop 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 shutdown_connections 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24e806c920 0x7f24e806ede0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc162880 0x7f24fc0cdc70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3bc0 0x7f24fc1f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 --2- 172.21.15.40:0/1870683695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b5f20 0x7f24fc1f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 >> 172.21.15.40:0/1870683695 conn(0x7f24fc0d4f60 msgr2=0x7f24fc0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 shutdown_connections 2024-04-26T21:36:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.083+0000 7f25040b0640 1 -- 172.21.15.40:0/1870683695 wait complete. 2024-04-26T21:36:25.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:36:25.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.198+0000 7fd3ceb24640 1 Processor -- start 2024-04-26T21:36:25.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.198+0000 7fd3ceb24640 1 -- start start 2024-04-26T21:36:25.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3ceb24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c815f170 0x7fd3c80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:25.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3ceb24640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd3c815f550 0x7fd3c80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:25.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3ceb24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3c80d34b0 0x7fd3c80d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:25.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3ceb24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c80c9430 con 0x7fd3c80d34b0 2024-04-26T21:36:25.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3ceb24640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c80d3e20 con 0x7fd3c815f550 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3ceb24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c80d1040 con 0x7fd3c815f170 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3cd0c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3c80d34b0 0x7fd3c80d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3cd0c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3c80d34b0 0x7fd3c80d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40984/0 (socket says 172.21.15.40:40984) 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3cd0c1640 1 -- 172.21.15.40:0/1352783052 learned_addr learned my addr 172.21.15.40:0/1352783052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3c7fff640 1 --2- 172.21.15.40:0/1352783052 >> v2:172.21.15.184:3300/0 conn(0x7fd3c815f550 0x7fd3c80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.199+0000 7fd3cc8c0640 1 --2- 172.21.15.40:0/1352783052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c815f170 0x7fd3c80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.200+0000 7fd3cd0c1640 1 -- 172.21.15.40:0/1352783052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c815f170 msgr2=0x7fd3c80d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.200+0000 7fd3cd0c1640 1 --2- 172.21.15.40:0/1352783052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c815f170 0x7fd3c80d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.200+0000 7fd3cd0c1640 1 -- 172.21.15.40:0/1352783052 >> v2:172.21.15.184:3300/0 conn(0x7fd3c815f550 msgr2=0x7fd3c80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.200+0000 7fd3cd0c1640 1 --2- 172.21.15.40:0/1352783052 >> v2:172.21.15.184:3300/0 conn(0x7fd3c815f550 0x7fd3c80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.200+0000 7fd3cd0c1640 1 -- 172.21.15.40:0/1352783052 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c80c9680 con 0x7fd3c80d34b0 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.200+0000 7fd3cd0c1640 1 --2- 172.21.15.40:0/1352783052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c80d34b0 0x7fd3c80d38e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd3c808e960 tx=0x7fd3b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eecd7fbc2fcd4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:25.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.200+0000 7fd3c77fe640 1 -- 172.21.15.40:0/1352783052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8037070 con 0x7fd3c80d34b0 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3c77fe640 1 -- 172.21.15.40:0/1352783052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8032440 con 0x7fd3c80d34b0 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3c77fe640 1 -- 172.21.15.40:0/1352783052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8032810 con 0x7fd3c80d34b0 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1352783052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c80d34b0 msgr2=0x7fd3c80d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1352783052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c80d34b0 0x7fd3c80d38e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd3c808e960 tx=0x7fd3b802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1352783052 shutdown_connections 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1352783052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c80d34b0 0x7fd3c80d38e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1352783052 >> v2:172.21.15.184:3300/0 conn(0x7fd3c815f550 0x7fd3c80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1352783052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c815f170 0x7fd3c80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1352783052 >> 172.21.15.40:0/1352783052 conn(0x7fd3c80d4f60 msgr2=0x7fd3c80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1352783052 shutdown_connections 2024-04-26T21:36:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.201+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1352783052 wait complete. 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.202+0000 7fd3ceb24640 1 Processor -- start 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.202+0000 7fd3ceb24640 1 -- start start 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3ceb24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c80d34b0 0x7fd3c81595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3ceb24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c815f170 0x7fd3c81f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3ceb24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c81f5710 0x7fd3c80cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3ceb24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80cfc50 con 0x7fd3c815f170 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3ceb24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80c9c40 con 0x7fd3c81f5710 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3ceb24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c80cfdd0 con 0x7fd3c80d34b0 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c815f170 0x7fd3c81f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c815f170 0x7fd3c81f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40996/0 (socket says 172.21.15.40:40996) 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3c7fff640 1 -- 172.21.15.40:0/1143235597 learned_addr learned my addr 172.21.15.40:0/1143235597 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3cc8c0640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c80d34b0 0x7fd3c81595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.203+0000 7fd3cd0c1640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c81f5710 0x7fd3c80cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c7fff640 1 -- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c80d34b0 msgr2=0x7fd3c81595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c7fff640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c80d34b0 0x7fd3c81595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c7fff640 1 -- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c81f5710 msgr2=0x7fd3c80cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c7fff640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c81f5710 0x7fd3c80cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c7fff640 1 -- 172.21.15.40:0/1143235597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c81e7c50 con 0x7fd3c815f170 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c7fff640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c815f170 0x7fd3c81f51d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e870 tx=0x7fd3bc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c57fa640 1 -- 172.21.15.40:0/1143235597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc017070 con 0x7fd3c815f170 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c57fa640 1 -- 172.21.15.40:0/1143235597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3bc00c940 con 0x7fd3c815f170 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c81e7f30 con 0x7fd3c815f170 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3c57fa640 1 -- 172.21.15.40:0/1143235597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00cd30 con 0x7fd3c815f170 2024-04-26T21:36:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.204+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c81e84f0 con 0x7fd3c815f170 2024-04-26T21:36:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.205+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c80694f0 con 0x7fd3c815f170 2024-04-26T21:36:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.205+0000 7fd3c57fa640 1 -- 172.21.15.40:0/1143235597 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3bc01a070 con 0x7fd3c815f170 2024-04-26T21:36:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.206+0000 7fd3c57fa640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3b406c8f0 0x7fd3b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.206+0000 7fd3c57fa640 1 -- 172.21.15.40:0/1143235597 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fd3bc08be30 con 0x7fd3c815f170 2024-04-26T21:36:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.206+0000 7fd3cc8c0640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3b406c8f0 0x7fd3b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.207+0000 7fd3cc8c0640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3b406c8f0 0x7fd3b406edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0011360 tx=0x7fd3c000d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.210+0000 7fd3c57fa640 1 -- 172.21.15.40:0/1143235597 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3bc059a80 con 0x7fd3c815f170 2024-04-26T21:36:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:25.441+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd3c8069ba0 con 0x7fd3c815f170 2024-04-26T21:36:26.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.432+0000 7fd3c57fa640 1 -- 172.21.15.40:0/1143235597 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fd3bc05dd80 con 0x7fd3c815f170 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3b406c8f0 msgr2=0x7fd3b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3b406c8f0 0x7fd3b406edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0011360 tx=0x7fd3c000d040 comp rx=0 tx=0).stop 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c815f170 msgr2=0x7fd3c81f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c815f170 0x7fd3c81f51d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e870 tx=0x7fd3bc00ed30 comp rx=0 tx=0).stop 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 shutdown_connections 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3b406c8f0 0x7fd3b406edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c81f5710 0x7fd3c80cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c815f170 0x7fd3c81f51d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 --2- 172.21.15.40:0/1143235597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3c80d34b0 0x7fd3c81595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 >> 172.21.15.40:0/1143235597 conn(0x7fd3c80d4f60 msgr2=0x7fd3c806e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 shutdown_connections 2024-04-26T21:36:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.435+0000 7fd3ceb24640 1 -- 172.21.15.40:0/1143235597 wait complete. 2024-04-26T21:36:26.449 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:36:26.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:36:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.558+0000 7fef8fe23640 1 Processor -- start 2024-04-26T21:36:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.559+0000 7fef8fe23640 1 -- start start 2024-04-26T21:36:26.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.559+0000 7fef8fe23640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef8815f170 0x7fef880d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:26.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8fe23640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fef8815f550 0x7fef880d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:26.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8fe23640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef880d1040 con 0x7fef880d34b0 2024-04-26T21:36:26.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8fe23640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fef880c9430 con 0x7fef8815f170 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8fe23640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fef880c9680 con 0x7fef8815f550 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8dbbf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef8815f170 0x7fef880d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8e3c0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fef880d34b0 0x7fef880cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50004/0 (socket says 172.21.15.40:50004) 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8e3c0640 1 -- 172.21.15.40:0/3369113486 learned_addr learned my addr 172.21.15.40:0/3369113486 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8d3be640 1 --2- 172.21.15.40:0/3369113486 >> v2:172.21.15.184:3300/0 conn(0x7fef8815f550 0x7fef880d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8dbbf640 1 -- 172.21.15.40:0/3369113486 >> v2:172.21.15.184:3300/0 conn(0x7fef8815f550 msgr2=0x7fef880d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8dbbf640 1 --2- 172.21.15.40:0/3369113486 >> v2:172.21.15.184:3300/0 conn(0x7fef8815f550 0x7fef880d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8dbbf640 1 -- 172.21.15.40:0/3369113486 >> v1:172.21.15.40:6789/0 conn(0x7fef880d34b0 legacy=0x7fef880cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8cbbd640 1 -- 172.21.15.40:0/3369113486 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3140551801 0 0) 0x7fef80001490 con 0x7fef880d34b0 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8dbbf640 1 -- 172.21.15.40:0/3369113486 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef880c9960 con 0x7fef8815f170 2024-04-26T21:36:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.560+0000 7fef8d3be640 1 --2- 172.21.15.40:0/3369113486 >> v2:172.21.15.184:3300/0 conn(0x7fef8815f550 0x7fef880d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.561+0000 7fef8dbbf640 1 --2- 172.21.15.40:0/3369113486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8815f170 0x7fef880d5a80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fef840097a0 tx=0x7fef8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89066f0566ae4030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.561+0000 7fef8cbbd640 1 -- 172.21.15.40:0/3369113486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84037070 con 0x7fef8815f170 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.561+0000 7fef8cbbd640 1 -- 172.21.15.40:0/3369113486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef84032440 con 0x7fef8815f170 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.561+0000 7fef8cbbd640 1 -- 172.21.15.40:0/3369113486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84032810 con 0x7fef8815f170 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.561+0000 7fef8fe23640 1 -- 172.21.15.40:0/3369113486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8815f170 msgr2=0x7fef880d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.561+0000 7fef8fe23640 1 --2- 172.21.15.40:0/3369113486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8815f170 0x7fef880d5a80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fef840097a0 tx=0x7fef8402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.562+0000 7fef8fe23640 1 -- 172.21.15.40:0/3369113486 shutdown_connections 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.562+0000 7fef8fe23640 1 --2- 172.21.15.40:0/3369113486 >> v2:172.21.15.184:3300/0 conn(0x7fef8815f550 0x7fef880d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.562+0000 7fef8fe23640 1 --2- 172.21.15.40:0/3369113486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef8815f170 0x7fef880d5a80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.562+0000 7fef8fe23640 1 -- 172.21.15.40:0/3369113486 >> 172.21.15.40:0/3369113486 conn(0x7fef880d4f60 msgr2=0x7fef880d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.562+0000 7fef8fe23640 1 -- 172.21.15.40:0/3369113486 shutdown_connections 2024-04-26T21:36:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.562+0000 7fef8fe23640 1 -- 172.21.15.40:0/3369113486 wait complete. 2024-04-26T21:36:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.563+0000 7fef8fe23640 1 Processor -- start 2024-04-26T21:36:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.563+0000 7fef8fe23640 1 -- start start 2024-04-26T21:36:26.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8fe23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef880d34b0 0x7fef881f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8fe23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8815f170 0x7fef880cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8fe23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880ccef0 0x7fef881520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8fe23640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef880cb5d0 con 0x7fef880ccef0 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8fe23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef880cb450 con 0x7fef880d34b0 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8fe23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef880cb750 con 0x7fef8815f170 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880ccef0 0x7fef881520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880ccef0 0x7fef881520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41018/0 (socket says 172.21.15.40:41018) 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 -- 172.21.15.40:0/2722364242 learned_addr learned my addr 172.21.15.40:0/2722364242 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 -- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8815f170 msgr2=0x7fef880cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8815f170 0x7fef880cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 -- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef880d34b0 msgr2=0x7fef881f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef880d34b0 0x7fef881f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 -- 172.21.15.40:0/2722364242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef881f8380 con 0x7fef880ccef0 2024-04-26T21:36:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8e3c0640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880ccef0 0x7fef881520c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c7f0 tx=0x7fef8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef7e7fc640 1 -- 172.21.15.40:0/2722364242 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80018070 con 0x7fef880ccef0 2024-04-26T21:36:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef7e7fc640 1 -- 172.21.15.40:0/2722364242 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8000a5e0 con 0x7fef880ccef0 2024-04-26T21:36:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef880cd450 con 0x7fef880ccef0 2024-04-26T21:36:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef7e7fc640 1 -- 172.21.15.40:0/2722364242 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8000aa10 con 0x7fef880ccef0 2024-04-26T21:36:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.564+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef88150220 con 0x7fef880ccef0 2024-04-26T21:36:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.565+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef88150010 con 0x7fef880ccef0 2024-04-26T21:36:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.566+0000 7fef7e7fc640 1 -- 172.21.15.40:0/2722364242 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef8001b070 con 0x7fef880ccef0 2024-04-26T21:36:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.566+0000 7fef7e7fc640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef6c06c8f0 0x7fef6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.566+0000 7fef7e7fc640 1 -- 172.21.15.40:0/2722364242 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fef8008b220 con 0x7fef880ccef0 2024-04-26T21:36:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.566+0000 7fef8dbbf640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef6c06c8f0 0x7fef6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.567+0000 7fef8dbbf640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef6c06c8f0 0x7fef6c06edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef84034000 tx=0x7fef8402c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:26.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.571+0000 7fef7e7fc640 1 -- 172.21.15.40:0/2722364242 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef880d1040 con 0x7fef880ccef0 2024-04-26T21:36:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:26.796+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fef8815ea90 con 0x7fef880ccef0 2024-04-26T21:36:27.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.436+0000 7fef7e7fc640 1 -- 172.21.15.40:0/2722364242 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fef80059550 con 0x7fef880ccef0 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef6c06c8f0 msgr2=0x7fef6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef6c06c8f0 0x7fef6c06edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef84034000 tx=0x7fef8402c000 comp rx=0 tx=0).stop 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880ccef0 msgr2=0x7fef881520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880ccef0 0x7fef881520c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c7f0 tx=0x7fef8000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 shutdown_connections 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef6c06c8f0 0x7fef6c06edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880ccef0 0x7fef881520c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef8815f170 0x7fef880cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 --2- 172.21.15.40:0/2722364242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef880d34b0 0x7fef881f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 >> 172.21.15.40:0/2722364242 conn(0x7fef880d4f60 msgr2=0x7fef880d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 shutdown_connections 2024-04-26T21:36:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.439+0000 7fef8fe23640 1 -- 172.21.15.40:0/2722364242 wait complete. 2024-04-26T21:36:27.454 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:36:27.454 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.561+0000 7fb69e55f640 1 Processor -- start 2024-04-26T21:36:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.561+0000 7fb69e55f640 1 -- start start 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb69e55f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6980d3bc0 0x7fb6980d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb69e55f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb6980b5f20 0x7fb6980b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb69e55f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6980b7e50 con 0x7fb698162880 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb69e55f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb6980d0580 con 0x7fb6980b5f20 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb69e55f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb6980d1800 con 0x7fb6980d3bc0 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb697fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb698162880 0x7fb6980d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50012/0 (socket says 172.21.15.40:50012) 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb697fff640 1 -- 172.21.15.40:0/4179433266 learned_addr learned my addr 172.21.15.40:0/4179433266 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565502180 0 0) 0x7fb6980b7e50 con 0x7fb698162880 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb69cafc640 1 --2- 172.21.15.40:0/4179433266 >> v2:172.21.15.184:3300/0 conn(0x7fb6980b5f20 0x7fb6980b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb6977fe640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6980d3bc0 0x7fb6980d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb684003520 con 0x7fb698162880 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3956154834 0 0) 0x7fb684003520 con 0x7fb698162880 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6980d3bc0 msgr2=0x7fb6980d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6980d3bc0 0x7fb6980d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 >> v2:172.21.15.184:3300/0 conn(0x7fb6980b5f20 msgr2=0x7fb6980b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 --2- 172.21.15.40:0/4179433266 >> v2:172.21.15.184:3300/0 conn(0x7fb6980b5f20 0x7fb6980b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6980d07d0 con 0x7fb698162880 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.562+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb680001670 con 0x7fb698162880 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 >> v1:172.21.15.40:6789/0 conn(0x7fb698162880 legacy=0x7fb6980d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb696ffd640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6840039b0 0x7fb684003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb696ffd640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6980d3bc0 0x7fb684004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb696ffd640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb684004d40 0x7fb6840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb6980d0580 con 0x7fb684004d40 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6980d1800 con 0x7fb6840039b0 2024-04-26T21:36:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb68400b9c0 con 0x7fb6980d3bc0 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb69cafc640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb684004d40 0x7fb6840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb697fff640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6840039b0 0x7fb684003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb6977fe640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6980d3bc0 0x7fb684004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb69cafc640 1 -- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6980d3bc0 msgr2=0x7fb684004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb69cafc640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6980d3bc0 0x7fb684004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb69cafc640 1 -- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6840039b0 msgr2=0x7fb684003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb69cafc640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6840039b0 0x7fb684003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.563+0000 7fb69cafc640 1 -- 172.21.15.40:0/4179433266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb684003520 con 0x7fb684004d40 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb69cafc640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb684004d40 0x7fb6840051c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb690009fc0 tx=0x7fb684005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ddb65b446b084d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690009c10 con 0x7fb684004d40 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb690036070 con 0x7fb684004d40 2024-04-26T21:36:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb696ffd640 1 -- 172.21.15.40:0/4179433266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69002d070 con 0x7fb684004d40 2024-04-26T21:36:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb69e55f640 1 -- 172.21.15.40:0/4179433266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb684004d40 msgr2=0x7fb6840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb69e55f640 1 --2- 172.21.15.40:0/4179433266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb684004d40 0x7fb6840051c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb690009fc0 tx=0x7fb684005830 comp rx=0 tx=0).stop 2024-04-26T21:36:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb69cafc640 1 -- 172.21.15.40:0/4179433266 reap_dead start 2024-04-26T21:36:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb69e55f640 1 -- 172.21.15.40:0/4179433266 shutdown_connections 2024-04-26T21:36:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb69e55f640 1 -- 172.21.15.40:0/4179433266 >> 172.21.15.40:0/4179433266 conn(0x7fb6980d4f60 msgr2=0x7fb6980d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.564+0000 7fb69e55f640 1 -- 172.21.15.40:0/4179433266 shutdown_connections 2024-04-26T21:36:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.565+0000 7fb69e55f640 1 -- 172.21.15.40:0/4179433266 wait complete. 2024-04-26T21:36:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.566+0000 7fb69e55f640 1 Processor -- start 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.566+0000 7fb69e55f640 1 -- start start 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69e55f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb684004d40 0x7fb6981f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69e55f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6980cc9a0 0x7fb6980cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69e55f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6980ce0a0 0x7fb698161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69e55f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb698162c60 con 0x7fb6980ce0a0 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69e55f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6980d0d90 con 0x7fb6980cc9a0 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69e55f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb698162de0 con 0x7fb684004d40 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6980ce0a0 0x7fb698161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6980ce0a0 0x7fb698161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41034/0 (socket says 172.21.15.40:41034) 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 -- 172.21.15.40:0/2656247159 learned_addr learned my addr 172.21.15.40:0/2656247159 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb6977fe640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6980cc9a0 0x7fb6980cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb697fff640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb684004d40 0x7fb6981f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 -- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb684004d40 msgr2=0x7fb6981f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb684004d40 0x7fb6981f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 -- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6980cc9a0 msgr2=0x7fb6980cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6980cc9a0 0x7fb6980cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 -- 172.21.15.40:0/2656247159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6981f8d30 con 0x7fb6980ce0a0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb697fff640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb684004d40 0x7fb6981f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb6977fe640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6980cc9a0 0x7fb6980cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb69cafc640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6980ce0a0 0x7fb698161150 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb690033040 tx=0x7fb690009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb694ff9640 1 -- 172.21.15.40:0/2656247159 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690009c10 con 0x7fb6980ce0a0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.567+0000 7fb694ff9640 1 -- 172.21.15.40:0/2656247159 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb690003b90 con 0x7fb6980ce0a0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.568+0000 7fb694ff9640 1 -- 172.21.15.40:0/2656247159 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690004020 con 0x7fb6980ce0a0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.568+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6981f9010 con 0x7fb6980ce0a0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.568+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6980cd360 con 0x7fb6980ce0a0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.569+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6980694f0 con 0x7fb6980ce0a0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.569+0000 7fb694ff9640 1 -- 172.21.15.40:0/2656247159 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb690005d10 con 0x7fb6980ce0a0 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.570+0000 7fb694ff9640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb67806c8f0 0x7fb67806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.570+0000 7fb694ff9640 1 -- 172.21.15.40:0/2656247159 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fb6900ad600 con 0x7fb6980ce0a0 2024-04-26T21:36:27.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.570+0000 7fb697fff640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb67806c8f0 0x7fb67806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.570+0000 7fb697fff640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb67806c8f0 0x7fb67806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb680000fb0 tx=0x7fb680009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:27.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.574+0000 7fb694ff9640 1 -- 172.21.15.40:0/2656247159 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb69004a060 con 0x7fb6980ce0a0 2024-04-26T21:36:27.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.798+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6980cbd40 con 0x7fb6980ce0a0 2024-04-26T21:36:27.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.799+0000 7fb694ff9640 1 -- 172.21.15.40:0/2656247159 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7fb69007b930 con 0x7fb6980ce0a0 2024-04-26T21:36:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:27.800 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:26.796847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3403660773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3403660773},{"type":"v1","addr":"172.21.15.40:6835","nonce":3403660773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5364,"qdb_cluster":[5364]},"id":2}]} 2024-04-26T21:36:27.800 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-04-26T21:36:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb67806c8f0 msgr2=0x7fb67806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb67806c8f0 0x7fb67806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb680000fb0 tx=0x7fb680009040 comp rx=0 tx=0).stop 2024-04-26T21:36:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6980ce0a0 msgr2=0x7fb698161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6980ce0a0 0x7fb698161150 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb690033040 tx=0x7fb690009750 comp rx=0 tx=0).stop 2024-04-26T21:36:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 shutdown_connections 2024-04-26T21:36:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb67806c8f0 0x7fb67806edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6980ce0a0 0x7fb698161150 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6980cc9a0 0x7fb6980cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 --2- 172.21.15.40:0/2656247159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb684004d40 0x7fb6981f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 >> 172.21.15.40:0/2656247159 conn(0x7fb6980d4f60 msgr2=0x7fb6980d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 shutdown_connections 2024-04-26T21:36:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.801+0000 7fb69e55f640 1 -- 172.21.15.40:0/2656247159 wait complete. 2024-04-26T21:36:27.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.913+0000 7f02af724640 1 Processor -- start 2024-04-26T21:36:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.913+0000 7f02af724640 1 -- start start 2024-04-26T21:36:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02af724640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f02a815ab60 0x7f02a806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02af724640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f02a815dd00 0x7f02a806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02af724640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a815af40 con 0x7f02a81617f0 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02af724640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f02a815e320 con 0x7f02a815ab60 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02af724640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f02a8160d10 con 0x7f02a815dd00 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02ad4c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f02a815ab60 0x7f02a806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02ad4c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f02a815ab60 0x7f02a806c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41044/0 (socket says 172.21.15.40:41044) 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02ad4c0640 1 -- 172.21.15.40:0/3031140415 learned_addr learned my addr 172.21.15.40:0/3031140415 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02ad4c0640 1 -- 172.21.15.40:0/3031140415 >> v2:172.21.15.184:3300/0 conn(0x7f02a815dd00 msgr2=0x7f02a806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02ad4c0640 1 --2- 172.21.15.40:0/3031140415 >> v2:172.21.15.184:3300/0 conn(0x7f02a815dd00 0x7f02a806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02ad4c0640 1 -- 172.21.15.40:0/3031140415 >> v1:172.21.15.184:6789/0 conn(0x7f02a81617f0 legacy=0x7f02a815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.914+0000 7f02ad4c0640 1 -- 172.21.15.40:0/3031140415 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a8160ff0 con 0x7f02a815ab60 2024-04-26T21:36:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f02ad4c0640 1 --2- 172.21.15.40:0/3031140415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a806c240 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f02a8074840 tx=0x7f02a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371cc63aa7c239e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f029ffff640 1 -- 172.21.15.40:0/3031140415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4037070 con 0x7f02a815ab60 2024-04-26T21:36:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f029ffff640 1 -- 172.21.15.40:0/3031140415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a4032440 con 0x7f02a815ab60 2024-04-26T21:36:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f029ffff640 1 -- 172.21.15.40:0/3031140415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4032810 con 0x7f02a815ab60 2024-04-26T21:36:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f02af724640 1 -- 172.21.15.40:0/3031140415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 msgr2=0x7f02a806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f02af724640 1 --2- 172.21.15.40:0/3031140415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a806c240 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f02a8074840 tx=0x7f02a402a230 comp rx=0 tx=0).stop 2024-04-26T21:36:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f02af724640 1 -- 172.21.15.40:0/3031140415 shutdown_connections 2024-04-26T21:36:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f02af724640 1 --2- 172.21.15.40:0/3031140415 >> v2:172.21.15.184:3300/0 conn(0x7f02a815dd00 0x7f02a806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f02af724640 1 --2- 172.21.15.40:0/3031140415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a806c240 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.915+0000 7f02af724640 1 -- 172.21.15.40:0/3031140415 >> 172.21.15.40:0/3031140415 conn(0x7f02a80d1cf0 msgr2=0x7f02a80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.916+0000 7f02af724640 1 -- 172.21.15.40:0/3031140415 shutdown_connections 2024-04-26T21:36:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.916+0000 7f02af724640 1 -- 172.21.15.40:0/3031140415 wait complete. 2024-04-26T21:36:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.917+0000 7f02af724640 1 Processor -- start 2024-04-26T21:36:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.917+0000 7f02af724640 1 -- start start 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.917+0000 7f02af724640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a81460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.917+0000 7f02af724640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a815dd00 0x7f02a8143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.917+0000 7f02af724640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a81617f0 0x7f02a80c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.917+0000 7f02af724640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f02a80cc4c0 con 0x7f02a815ab60 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.917+0000 7f02af724640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f02a80cc340 con 0x7f02a81617f0 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.917+0000 7f02af724640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f02a80cc640 con 0x7f02a815dd00 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a81460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a81460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41058/0 (socket says 172.21.15.40:41058) 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 -- 172.21.15.40:0/3350894617 learned_addr learned my addr 172.21.15.40:0/3350894617 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02accbf640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a815dd00 0x7f02a8143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02adcc1640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a81617f0 0x7f02a80c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 -- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a815dd00 msgr2=0x7f02a8143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a815dd00 0x7f02a8143d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 -- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a81617f0 msgr2=0x7f02a80c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a81617f0 0x7f02a80c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 -- 172.21.15.40:0/3350894617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a80b66f0 con 0x7f02a815ab60 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02accbf640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a815dd00 0x7f02a8143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02adcc1640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a81617f0 0x7f02a80c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02ad4c0640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a81460e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000c00 tx=0x7f02a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f029dffb640 1 -- 172.21.15.40:0/3350894617 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4041070 con 0x7f02a815ab60 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f029dffb640 1 -- 172.21.15.40:0/3350894617 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a403c870 con 0x7f02a815ab60 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a80b69d0 con 0x7f02a815ab60 2024-04-26T21:36:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.918+0000 7f029dffb640 1 -- 172.21.15.40:0/3350894617 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4037040 con 0x7f02a815ab60 2024-04-26T21:36:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.919+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a80b6f90 con 0x7f02a815ab60 2024-04-26T21:36:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.919+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a80694f0 con 0x7f02a815ab60 2024-04-26T21:36:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.920+0000 7f029dffb640 1 -- 172.21.15.40:0/3350894617 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02a4044070 con 0x7f02a815ab60 2024-04-26T21:36:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.920+0000 7f029dffb640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f028c06c8f0 0x7f028c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.920+0000 7f029dffb640 1 -- 172.21.15.40:0/3350894617 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f02a40ac8b0 con 0x7f02a815ab60 2024-04-26T21:36:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.921+0000 7f02accbf640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f028c06c8f0 0x7f028c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.921+0000 7f02accbf640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f028c06c8f0 0x7f028c06edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f02a80c4a10 tx=0x7f0298008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:27.925+0000 7f029dffb640 1 -- 172.21.15.40:0/3350894617 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02a407abe0 con 0x7f02a815ab60 2024-04-26T21:36:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.111+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02a81597f0 con 0x7f02a815ab60 2024-04-26T21:36:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.112+0000 7f029dffb640 1 -- 172.21.15.40:0/3350894617 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16906 (secure 0 0 0) 0x7f02a407a4e0 con 0x7f02a815ab60 2024-04-26T21:36:28.113 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:28.113 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:25.421233+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f028c06c8f0 msgr2=0x7f028c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f028c06c8f0 0x7f028c06edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f02a80c4a10 tx=0x7f0298008040 comp rx=0 tx=0).stop 2024-04-26T21:36:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 msgr2=0x7f02a81460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a81460e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000c00 tx=0x7f02a40046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 shutdown_connections 2024-04-26T21:36:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f028c06c8f0 0x7f028c06edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a81617f0 0x7f02a80c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a815dd00 0x7f02a8143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 --2- 172.21.15.40:0/3350894617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a815ab60 0x7f02a81460e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 >> 172.21.15.40:0/3350894617 conn(0x7f02a80d1cf0 msgr2=0x7f02a80cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:28.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 shutdown_connections 2024-04-26T21:36:28.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.114+0000 7f02af724640 1 -- 172.21.15.40:0/3350894617 wait complete. 2024-04-26T21:36:28.126 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:36:28.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:36:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.225+0000 7f0e98124640 1 Processor -- start 2024-04-26T21:36:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.226+0000 7f0e98124640 1 -- start start 2024-04-26T21:36:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.226+0000 7f0e98124640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0e90162810 0x7f0e90070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.226+0000 7f0e98124640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.226+0000 7f0e98124640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e900d10d0 con 0x7f0e900cd930 2024-04-26T21:36:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.226+0000 7f0e98124640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0e900d0750 con 0x7f0e90162810 2024-04-26T21:36:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.226+0000 7f0e98124640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e900d16a0 con 0x7f0e900b2340 2024-04-26T21:36:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e956bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e966c1640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f0e900cd930 0x7f0e900d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55814/0 (socket says 172.21.15.40:55814) 2024-04-26T21:36:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e95ec0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0e90162810 0x7f0e90070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e966c1640 1 -- 172.21.15.40:0/2647810285 learned_addr learned my addr 172.21.15.40:0/2647810285 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e94ebe640 1 -- 172.21.15.40:0/2647810285 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738888986 0 0) 0x7f0e900d10d0 con 0x7f0e900cd930 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e94ebe640 1 -- 172.21.15.40:0/2647810285 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e78003520 con 0x7f0e900cd930 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e956bf640 1 -- 172.21.15.40:0/2647810285 >> v2:172.21.15.184:3300/0 conn(0x7f0e90162810 msgr2=0x7f0e90070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e956bf640 1 --2- 172.21.15.40:0/2647810285 >> v2:172.21.15.184:3300/0 conn(0x7f0e90162810 0x7f0e90070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e956bf640 1 -- 172.21.15.40:0/2647810285 >> v1:172.21.15.184:6789/0 conn(0x7f0e900cd930 legacy=0x7f0e900d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e956bf640 1 -- 172.21.15.40:0/2647810285 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e900d1370 con 0x7f0e900b2340 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.227+0000 7f0e95ec0640 1 --2- 172.21.15.40:0/2647810285 >> v2:172.21.15.184:3300/0 conn(0x7f0e90162810 0x7f0e90070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.228+0000 7f0e956bf640 1 --2- 172.21.15.40:0/2647810285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900cd3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e840097a0 tx=0x7f0e8402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d47d4e6d8d2239 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.228+0000 7f0e94ebe640 1 -- 172.21.15.40:0/2647810285 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84037070 con 0x7f0e900b2340 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.228+0000 7f0e94ebe640 1 -- 172.21.15.40:0/2647810285 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e84032440 con 0x7f0e900b2340 2024-04-26T21:36:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.228+0000 7f0e94ebe640 1 -- 172.21.15.40:0/2647810285 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e840327f0 con 0x7f0e900b2340 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.229+0000 7f0e98124640 1 -- 172.21.15.40:0/2647810285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 msgr2=0x7f0e900cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.229+0000 7f0e98124640 1 --2- 172.21.15.40:0/2647810285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900cd3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e840097a0 tx=0x7f0e8402a180 comp rx=0 tx=0).stop 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.229+0000 7f0e98124640 1 -- 172.21.15.40:0/2647810285 shutdown_connections 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.229+0000 7f0e98124640 1 --2- 172.21.15.40:0/2647810285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900cd3f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.229+0000 7f0e98124640 1 --2- 172.21.15.40:0/2647810285 >> v2:172.21.15.184:3300/0 conn(0x7f0e90162810 0x7f0e90070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.229+0000 7f0e98124640 1 -- 172.21.15.40:0/2647810285 >> 172.21.15.40:0/2647810285 conn(0x7f0e900d2da0 msgr2=0x7f0e900d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.229+0000 7f0e98124640 1 -- 172.21.15.40:0/2647810285 shutdown_connections 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.229+0000 7f0e98124640 1 -- 172.21.15.40:0/2647810285 wait complete. 2024-04-26T21:36:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.230+0000 7f0e98124640 1 Processor -- start 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.230+0000 7f0e98124640 1 -- start start 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e98124640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e98124640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e900cd930 0x7f0e900cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e98124640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e90162810 0x7f0e900cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e98124640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90070890 con 0x7f0e900cd930 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e98124640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e900d0470 con 0x7f0e90162810 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e98124640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e900d2240 con 0x7f0e900b2340 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e900cd930 0x7f0e900cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e900cd930 0x7f0e900cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41072/0 (socket says 172.21.15.40:41072) 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 -- 172.21.15.40:0/423245871 learned_addr learned my addr 172.21.15.40:0/423245871 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e966c1640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e90162810 0x7f0e900cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e95ec0640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 -- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 msgr2=0x7f0e900b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 -- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e90162810 msgr2=0x7f0e900cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e90162810 0x7f0e900cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 -- 172.21.15.40:0/423245871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e900ccd70 con 0x7f0e900cd930 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.231+0000 7f0e956bf640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e900cd930 0x7f0e900cc160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0e84035040 tx=0x7f0e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.232+0000 7f0e8e7fc640 1 -- 172.21.15.40:0/423245871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84037040 con 0x7f0e900cd930 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.232+0000 7f0e8e7fc640 1 -- 172.21.15.40:0/423245871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e84042070 con 0x7f0e900cd930 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.232+0000 7f0e8e7fc640 1 -- 172.21.15.40:0/423245871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8403b930 con 0x7f0e900cd930 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.232+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e900c6ac0 con 0x7f0e900cd930 2024-04-26T21:36:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.232+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e900c7080 con 0x7f0e900cd930 2024-04-26T21:36:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.233+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e900694f0 con 0x7f0e900cd930 2024-04-26T21:36:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.233+0000 7f0e8e7fc640 1 -- 172.21.15.40:0/423245871 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e8403bad0 con 0x7f0e900cd930 2024-04-26T21:36:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.234+0000 7f0e8e7fc640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e7c06c920 0x7f0e7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.234+0000 7f0e8e7fc640 1 -- 172.21.15.40:0/423245871 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0e840ac5d0 con 0x7f0e900cd930 2024-04-26T21:36:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.234+0000 7f0e95ec0640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e7c06c920 0x7f0e7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.234+0000 7f0e95ec0640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e7c06c920 0x7f0e7c06ede0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e900b06a0 tx=0x7f0e88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.238+0000 7f0e8e7fc640 1 -- 172.21.15.40:0/423245871 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e8407a7f0 con 0x7f0e900cd930 2024-04-26T21:36:28.393 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:36:28.391+0000 7f2f2ca7a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:36:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.417+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e900c6890 con 0x7f0e7c06c920 2024-04-26T21:36:28.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.424+0000 7f0e8e7fc640 1 -- 172.21.15.40:0/423245871 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240074 (secure 0 0 0) 0x7f0e900c6890 con 0x7f0e7c06c920 2024-04-26T21:36:28.428 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:36:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e7c06c920 msgr2=0x7f0e7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e7c06c920 0x7f0e7c06ede0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e900b06a0 tx=0x7f0e88008040 comp rx=0 tx=0).stop 2024-04-26T21:36:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e900cd930 msgr2=0x7f0e900cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e900cd930 0x7f0e900cc160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0e84035040 tx=0x7f0e840046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 shutdown_connections 2024-04-26T21:36:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e7c06c920 0x7f0e7c06ede0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e90162810 0x7f0e900cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e900cd930 0x7f0e900cc160 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 --2- 172.21.15.40:0/423245871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e900b2340 0x7f0e900b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.426+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 >> 172.21.15.40:0/423245871 conn(0x7f0e900d2da0 msgr2=0x7f0e90070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:28.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.427+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 shutdown_connections 2024-04-26T21:36:28.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.427+0000 7f0e98124640 1 -- 172.21.15.40:0/423245871 wait complete. 2024-04-26T21:36:28.439 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T21:36:28.011710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220460,"kb_used_data":4260,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754260,"statfs":{"total":773094113280,"available":772868362240,"internally_reserved":0,"allocated":4362240,"data_stored":2518189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.609682"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408704+0000","last_change":"2024-04-26T21:36:22.408704+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408704+0000","last_undegraded":"2024-04-26T21:36:22.408704+0000","last_fullsized":"2024-04-26T21:36:22.408704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406003+0000","last_change":"2024-04-26T21:36:22.406003+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406003+0000","last_undegraded":"2024-04-26T21:36:22.406003+0000","last_fullsized":"2024-04-26T21:36:22.406003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406957+0000","last_change":"2024-04-26T21:36:22.406957+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406957+0000","last_undegraded":"2024-04-26T21:36:22.406957+0000","last_fullsized":"2024-04-26T21:36:22.406957+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404256+0000","last_change":"2024-04-26T21:36:22.404256+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404256+0000","last_undegraded":"2024-04-26T21:36:22.404256+0000","last_fullsized":"2024-04-26T21:36:22.404256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404676+0000","last_change":"2024-04-26T21:36:22.404676+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404676+0000","last_undegraded":"2024-04-26T21:36:22.404676+0000","last_fullsized":"2024-04-26T21:36:22.404676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407334+0000","last_change":"2024-04-26T21:36:22.407334+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407334+0000","last_undegraded":"2024-04-26T21:36:22.407334+0000","last_fullsized":"2024-04-26T21:36:22.407334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406817+0000","last_change":"2024-04-26T21:36:22.406817+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406817+0000","last_undegraded":"2024-04-26T21:36:22.406817+0000","last_fullsized":"2024-04-26T21:36:22.406817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404794+0000","last_change":"2024-04-26T21:36:22.404794+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404794+0000","last_undegraded":"2024-04-26T21:36:22.404794+0000","last_fullsized":"2024-04-26T21:36:22.404794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412333+0000","last_change":"2024-04-26T21:36:22.412333+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412333+0000","last_undegraded":"2024-04-26T21:36:22.412333+0000","last_fullsized":"2024-04-26T21:36:22.412333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404182+0000","last_change":"2024-04-26T21:36:22.404182+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404182+0000","last_undegraded":"2024-04-26T21:36:22.404182+0000","last_fullsized":"2024-04-26T21:36:22.404182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404209+0000","last_change":"2024-04-26T21:36:22.404209+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404209+0000","last_undegraded":"2024-04-26T21:36:22.404209+0000","last_fullsized":"2024-04-26T21:36:22.404209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412389+0000","last_change":"2024-04-26T21:36:22.412389+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412389+0000","last_undegraded":"2024-04-26T21:36:22.412389+0000","last_fullsized":"2024-04-26T21:36:22.412389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412388+0000","last_change":"2024-04-26T21:36:22.412388+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412388+0000","last_undegraded":"2024-04-26T21:36:22.412388+0000","last_fullsized":"2024-04-26T21:36:22.412388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406854+0000","last_change":"2024-04-26T21:36:22.406854+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406854+0000","last_undegraded":"2024-04-26T21:36:22.406854+0000","last_fullsized":"2024-04-26T21:36:22.406854+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408704+0000","last_change":"2024-04-26T21:36:22.408704+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408704+0000","last_undegraded":"2024-04-26T21:36:22.408704+0000","last_fullsized":"2024-04-26T21:36:22.408704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407136+0000","last_change":"2024-04-26T21:36:22.407136+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407136+0000","last_undegraded":"2024-04-26T21:36:22.407136+0000","last_fullsized":"2024-04-26T21:36:22.407136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404902+0000","last_change":"2024-04-26T21:36:22.404902+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404902+0000","last_undegraded":"2024-04-26T21:36:22.404902+0000","last_fullsized":"2024-04-26T21:36:22.404902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404184+0000","last_change":"2024-04-26T21:36:22.404184+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404184+0000","last_undegraded":"2024-04-26T21:36:22.404184+0000","last_fullsized":"2024-04-26T21:36:22.404184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403945+0000","last_change":"2024-04-26T21:36:22.403945+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403945+0000","last_undegraded":"2024-04-26T21:36:22.403945+0000","last_fullsized":"2024-04-26T21:36:22.403945+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407361+0000","last_change":"2024-04-26T21:36:22.407361+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407361+0000","last_undegraded":"2024-04-26T21:36:22.407361+0000","last_fullsized":"2024-04-26T21:36:22.407361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403862+0000","last_change":"2024-04-26T21:36:22.403862+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403862+0000","last_undegraded":"2024-04-26T21:36:22.403862+0000","last_fullsized":"2024-04-26T21:36:22.403862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403889+0000","last_change":"2024-04-26T21:36:22.403889+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403889+0000","last_undegraded":"2024-04-26T21:36:22.403889+0000","last_fullsized":"2024-04-26T21:36:22.403889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407307+0000","last_change":"2024-04-26T21:36:22.407307+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407307+0000","last_undegraded":"2024-04-26T21:36:22.407307+0000","last_fullsized":"2024-04-26T21:36:22.407307+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403907+0000","last_change":"2024-04-26T21:36:22.403907+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403907+0000","last_undegraded":"2024-04-26T21:36:22.403907+0000","last_fullsized":"2024-04-26T21:36:22.403907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404196+0000","last_change":"2024-04-26T21:36:22.404196+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404196+0000","last_undegraded":"2024-04-26T21:36:22.404196+0000","last_fullsized":"2024-04-26T21:36:22.404196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403911+0000","last_change":"2024-04-26T21:36:22.403911+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403911+0000","last_undegraded":"2024-04-26T21:36:22.403911+0000","last_fullsized":"2024-04-26T21:36:22.403911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.405895+0000","last_change":"2024-04-26T21:36:22.405895+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.405895+0000","last_undegraded":"2024-04-26T21:36:22.405895+0000","last_fullsized":"2024-04-26T21:36:22.405895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406117+0000","last_change":"2024-04-26T21:36:22.406117+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406117+0000","last_undegraded":"2024-04-26T21:36:22.406117+0000","last_fullsized":"2024-04-26T21:36:22.406117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408298+0000","last_change":"2024-04-26T21:36:22.408298+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408298+0000","last_undegraded":"2024-04-26T21:36:22.408298+0000","last_fullsized":"2024-04-26T21:36:22.408298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404711+0000","last_change":"2024-04-26T21:36:22.404711+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404711+0000","last_undegraded":"2024-04-26T21:36:22.404711+0000","last_fullsized":"2024-04-26T21:36:22.404711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412312+0000","last_change":"2024-04-26T21:36:22.412312+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412312+0000","last_undegraded":"2024-04-26T21:36:22.412312+0000","last_fullsized":"2024-04-26T21:36:22.412312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407200+0000","last_change":"2024-04-26T21:36:22.407200+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407200+0000","last_undegraded":"2024-04-26T21:36:22.407200+0000","last_fullsized":"2024-04-26T21:36:22.407200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T21:36:22.408318+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:36:22.408318+0000","last_peered":"2024-04-26T21:36:22.408318+0000","last_clean":"2024-04-26T21:36:22.408318+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:36:22.408318+0000","last_undegraded":"2024-04-26T21:36:22.408318+0000","last_fullsized":"2024-04-26T21:36:22.408318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":753669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":163301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":163301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:36:28.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:36:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.579+0000 7f68795a4640 1 Processor -- start 2024-04-26T21:36:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.579+0000 7f68795a4640 1 -- start start 2024-04-26T21:36:28.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.579+0000 7f68795a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6874162880 0x7f68740d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.579+0000 7f68795a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68740d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.579+0000 7f68795a4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68740b7e50 con 0x7f68740d39f0 2024-04-26T21:36:28.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.579+0000 7f68795a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f68740d0580 con 0x7f6874162880 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.579+0000 7f68795a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f68740d1800 con 0x7f68740b5f20 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f68727fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68740d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f68737fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f68740d39f0 0x7f68740b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55816/0 (socket says 172.21.15.40:55816) 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f68737fe640 1 -- 172.21.15.40:0/2685822450 learned_addr learned my addr 172.21.15.40:0/2685822450 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f6872ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6874162880 0x7f68740d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f6871ffb640 1 -- 172.21.15.40:0/2685822450 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 213574313 0 0) 0x7f68740b7e50 con 0x7f68740d39f0 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f6871ffb640 1 -- 172.21.15.40:0/2685822450 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6860003520 con 0x7f68740d39f0 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f6872ffd640 1 -- 172.21.15.40:0/2685822450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 msgr2=0x7f68740d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f6872ffd640 1 --2- 172.21.15.40:0/2685822450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68740d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f6872ffd640 1 -- 172.21.15.40:0/2685822450 >> v1:172.21.15.184:6789/0 conn(0x7f68740d39f0 legacy=0x7f68740b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f6872ffd640 1 -- 172.21.15.40:0/2685822450 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68740d07d0 con 0x7f6874162880 2024-04-26T21:36:28.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.580+0000 7f68727fc640 1 --2- 172.21.15.40:0/2685822450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68740d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:28.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.581+0000 7f6872ffd640 1 --2- 172.21.15.40:0/2685822450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 0x7f68740d2cb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f686c0097a0 tx=0x7f686c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130f13b0396f57b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.581+0000 7f6871ffb640 1 -- 172.21.15.40:0/2685822450 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c037070 con 0x7f6874162880 2024-04-26T21:36:28.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.581+0000 7f6871ffb640 1 -- 172.21.15.40:0/2685822450 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c032440 con 0x7f6874162880 2024-04-26T21:36:28.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.581+0000 7f6871ffb640 1 -- 172.21.15.40:0/2685822450 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0327f0 con 0x7f6874162880 2024-04-26T21:36:28.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.582+0000 7f68795a4640 1 -- 172.21.15.40:0/2685822450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 msgr2=0x7f68740d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.582+0000 7f68795a4640 1 --2- 172.21.15.40:0/2685822450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 0x7f68740d2cb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f686c0097a0 tx=0x7f686c02a180 comp rx=0 tx=0).stop 2024-04-26T21:36:28.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.582+0000 7f68795a4640 1 -- 172.21.15.40:0/2685822450 shutdown_connections 2024-04-26T21:36:28.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.582+0000 7f68795a4640 1 --2- 172.21.15.40:0/2685822450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68740d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.582+0000 7f68795a4640 1 --2- 172.21.15.40:0/2685822450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 0x7f68740d2cb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.582+0000 7f68795a4640 1 -- 172.21.15.40:0/2685822450 >> 172.21.15.40:0/2685822450 conn(0x7f68740d4f60 msgr2=0x7f68740d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:28.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.582+0000 7f68795a4640 1 -- 172.21.15.40:0/2685822450 shutdown_connections 2024-04-26T21:36:28.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.582+0000 7f68795a4640 1 -- 172.21.15.40:0/2685822450 wait complete. 2024-04-26T21:36:28.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.583+0000 7f68795a4640 1 Processor -- start 2024-04-26T21:36:28.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.583+0000 7f68795a4640 1 -- start start 2024-04-26T21:36:28.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68795a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68741f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68795a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68740d39f0 0x7f687415ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68795a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 0x7f6874160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68795a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6874162c60 con 0x7f68740d39f0 2024-04-26T21:36:28.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68795a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68740d0d90 con 0x7f6874162880 2024-04-26T21:36:28.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68795a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6874162de0 con 0x7f68740b5f20 2024-04-26T21:36:28.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68740d39f0 0x7f687415ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68740d39f0 0x7f687415ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41088/0 (socket says 172.21.15.40:41088) 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 -- 172.21.15.40:0/3010055397 learned_addr learned my addr 172.21.15.40:0/3010055397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68737fe640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 0x7f6874160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 -- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 msgr2=0x7f68741f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f6872ffd640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68741f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68741f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 -- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 msgr2=0x7f6874160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 0x7f6874160450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 -- 172.21.15.40:0/3010055397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68740ccb80 con 0x7f68740d39f0 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.584+0000 7f68727fc640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68740d39f0 0x7f687415ff10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f685c00e960 tx=0x7f685c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.585+0000 7f686b7fe640 1 -- 172.21.15.40:0/3010055397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c017070 con 0x7f68740d39f0 2024-04-26T21:36:28.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.585+0000 7f686b7fe640 1 -- 172.21.15.40:0/3010055397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c00ca00 con 0x7f68740d39f0 2024-04-26T21:36:28.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.585+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68740cce60 con 0x7f68740d39f0 2024-04-26T21:36:28.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.585+0000 7f686b7fe640 1 -- 172.21.15.40:0/3010055397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c00cdd0 con 0x7f68740d39f0 2024-04-26T21:36:28.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.585+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68740cd420 con 0x7f68740d39f0 2024-04-26T21:36:28.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.586+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687406e430 con 0x7f68740d39f0 2024-04-26T21:36:28.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.586+0000 7f686b7fe640 1 -- 172.21.15.40:0/3010055397 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f685c01a070 con 0x7f68740d39f0 2024-04-26T21:36:28.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.587+0000 7f686b7fe640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c8f0 0x7f685406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.587+0000 7f686b7fe640 1 -- 172.21.15.40:0/3010055397 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f685c08bf30 con 0x7f68740d39f0 2024-04-26T21:36:28.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.587+0000 7f6872ffd640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c8f0 0x7f685406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.588+0000 7f6872ffd640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c8f0 0x7f685406edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f686c034000 tx=0x7f686c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.591+0000 7f686b7fe640 1 -- 172.21.15.40:0/3010055397 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f687406e430 con 0x7f68740d39f0 2024-04-26T21:36:28.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.775+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68740694f0 con 0x7f685406c8f0 2024-04-26T21:36:28.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.781+0000 7f686b7fe640 1 -- 172.21.15.40:0/3010055397 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240074 (secure 0 0 0) 0x7f68740694f0 con 0x7f685406c8f0 2024-04-26T21:36:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:28.786 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:36:28.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c8f0 msgr2=0x7f685406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c8f0 0x7f685406edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f686c034000 tx=0x7f686c02e000 comp rx=0 tx=0).stop 2024-04-26T21:36:28.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68740d39f0 msgr2=0x7f687415ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68740d39f0 0x7f687415ff10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f685c00e960 tx=0x7f685c00ee20 comp rx=0 tx=0).stop 2024-04-26T21:36:28.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 shutdown_connections 2024-04-26T21:36:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685406c8f0 0x7f685406edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6874162880 0x7f6874160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68740d39f0 0x7f687415ff10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 --2- 172.21.15.40:0/3010055397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68740b5f20 0x7f68741f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 >> 172.21.15.40:0/3010055397 conn(0x7f68740d4f60 msgr2=0x7f6874074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 shutdown_connections 2024-04-26T21:36:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.784+0000 7f68795a4640 1 -- 172.21.15.40:0/3010055397 wait complete. 2024-04-26T21:36:28.796 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T21:36:28.011710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220460,"kb_used_data":4260,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754260,"statfs":{"total":773094113280,"available":772868362240,"internally_reserved":0,"allocated":4362240,"data_stored":2518189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.609682"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408704+0000","last_change":"2024-04-26T21:36:22.408704+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408704+0000","last_undegraded":"2024-04-26T21:36:22.408704+0000","last_fullsized":"2024-04-26T21:36:22.408704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406003+0000","last_change":"2024-04-26T21:36:22.406003+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406003+0000","last_undegraded":"2024-04-26T21:36:22.406003+0000","last_fullsized":"2024-04-26T21:36:22.406003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406957+0000","last_change":"2024-04-26T21:36:22.406957+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406957+0000","last_undegraded":"2024-04-26T21:36:22.406957+0000","last_fullsized":"2024-04-26T21:36:22.406957+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404256+0000","last_change":"2024-04-26T21:36:22.404256+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404256+0000","last_undegraded":"2024-04-26T21:36:22.404256+0000","last_fullsized":"2024-04-26T21:36:22.404256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404676+0000","last_change":"2024-04-26T21:36:22.404676+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404676+0000","last_undegraded":"2024-04-26T21:36:22.404676+0000","last_fullsized":"2024-04-26T21:36:22.404676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407334+0000","last_change":"2024-04-26T21:36:22.407334+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407334+0000","last_undegraded":"2024-04-26T21:36:22.407334+0000","last_fullsized":"2024-04-26T21:36:22.407334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406817+0000","last_change":"2024-04-26T21:36:22.406817+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406817+0000","last_undegraded":"2024-04-26T21:36:22.406817+0000","last_fullsized":"2024-04-26T21:36:22.406817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404794+0000","last_change":"2024-04-26T21:36:22.404794+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404794+0000","last_undegraded":"2024-04-26T21:36:22.404794+0000","last_fullsized":"2024-04-26T21:36:22.404794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412333+0000","last_change":"2024-04-26T21:36:22.412333+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412333+0000","last_undegraded":"2024-04-26T21:36:22.412333+0000","last_fullsized":"2024-04-26T21:36:22.412333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404182+0000","last_change":"2024-04-26T21:36:22.404182+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404182+0000","last_undegraded":"2024-04-26T21:36:22.404182+0000","last_fullsized":"2024-04-26T21:36:22.404182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404209+0000","last_change":"2024-04-26T21:36:22.404209+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404209+0000","last_undegraded":"2024-04-26T21:36:22.404209+0000","last_fullsized":"2024-04-26T21:36:22.404209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412389+0000","last_change":"2024-04-26T21:36:22.412389+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412389+0000","last_undegraded":"2024-04-26T21:36:22.412389+0000","last_fullsized":"2024-04-26T21:36:22.412389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412388+0000","last_change":"2024-04-26T21:36:22.412388+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412388+0000","last_undegraded":"2024-04-26T21:36:22.412388+0000","last_fullsized":"2024-04-26T21:36:22.412388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406854+0000","last_change":"2024-04-26T21:36:22.406854+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406854+0000","last_undegraded":"2024-04-26T21:36:22.406854+0000","last_fullsized":"2024-04-26T21:36:22.406854+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408704+0000","last_change":"2024-04-26T21:36:22.408704+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408704+0000","last_undegraded":"2024-04-26T21:36:22.408704+0000","last_fullsized":"2024-04-26T21:36:22.408704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407136+0000","last_change":"2024-04-26T21:36:22.407136+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407136+0000","last_undegraded":"2024-04-26T21:36:22.407136+0000","last_fullsized":"2024-04-26T21:36:22.407136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404902+0000","last_change":"2024-04-26T21:36:22.404902+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404902+0000","last_undegraded":"2024-04-26T21:36:22.404902+0000","last_fullsized":"2024-04-26T21:36:22.404902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404184+0000","last_change":"2024-04-26T21:36:22.404184+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404184+0000","last_undegraded":"2024-04-26T21:36:22.404184+0000","last_fullsized":"2024-04-26T21:36:22.404184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403945+0000","last_change":"2024-04-26T21:36:22.403945+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403945+0000","last_undegraded":"2024-04-26T21:36:22.403945+0000","last_fullsized":"2024-04-26T21:36:22.403945+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407361+0000","last_change":"2024-04-26T21:36:22.407361+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407361+0000","last_undegraded":"2024-04-26T21:36:22.407361+0000","last_fullsized":"2024-04-26T21:36:22.407361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403862+0000","last_change":"2024-04-26T21:36:22.403862+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403862+0000","last_undegraded":"2024-04-26T21:36:22.403862+0000","last_fullsized":"2024-04-26T21:36:22.403862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403889+0000","last_change":"2024-04-26T21:36:22.403889+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403889+0000","last_undegraded":"2024-04-26T21:36:22.403889+0000","last_fullsized":"2024-04-26T21:36:22.403889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407307+0000","last_change":"2024-04-26T21:36:22.407307+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407307+0000","last_undegraded":"2024-04-26T21:36:22.407307+0000","last_fullsized":"2024-04-26T21:36:22.407307+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403907+0000","last_change":"2024-04-26T21:36:22.403907+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403907+0000","last_undegraded":"2024-04-26T21:36:22.403907+0000","last_fullsized":"2024-04-26T21:36:22.403907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404196+0000","last_change":"2024-04-26T21:36:22.404196+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404196+0000","last_undegraded":"2024-04-26T21:36:22.404196+0000","last_fullsized":"2024-04-26T21:36:22.404196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403911+0000","last_change":"2024-04-26T21:36:22.403911+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403911+0000","last_undegraded":"2024-04-26T21:36:22.403911+0000","last_fullsized":"2024-04-26T21:36:22.403911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.405895+0000","last_change":"2024-04-26T21:36:22.405895+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.405895+0000","last_undegraded":"2024-04-26T21:36:22.405895+0000","last_fullsized":"2024-04-26T21:36:22.405895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406117+0000","last_change":"2024-04-26T21:36:22.406117+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406117+0000","last_undegraded":"2024-04-26T21:36:22.406117+0000","last_fullsized":"2024-04-26T21:36:22.406117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408298+0000","last_change":"2024-04-26T21:36:22.408298+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408298+0000","last_undegraded":"2024-04-26T21:36:22.408298+0000","last_fullsized":"2024-04-26T21:36:22.408298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404711+0000","last_change":"2024-04-26T21:36:22.404711+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404711+0000","last_undegraded":"2024-04-26T21:36:22.404711+0000","last_fullsized":"2024-04-26T21:36:22.404711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412312+0000","last_change":"2024-04-26T21:36:22.412312+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412312+0000","last_undegraded":"2024-04-26T21:36:22.412312+0000","last_fullsized":"2024-04-26T21:36:22.412312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407200+0000","last_change":"2024-04-26T21:36:22.407200+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407200+0000","last_undegraded":"2024-04-26T21:36:22.407200+0000","last_fullsized":"2024-04-26T21:36:22.407200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T21:36:22.408318+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:36:22.408318+0000","last_peered":"2024-04-26T21:36:22.408318+0000","last_clean":"2024-04-26T21:36:22.408318+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:36:22.408318+0000","last_undegraded":"2024-04-26T21:36:22.408318+0000","last_fullsized":"2024-04-26T21:36:22.408318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":753669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":163301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":163301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:36:28.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:36:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.936+0000 7fdb22e47640 1 Processor -- start 2024-04-26T21:36:28.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.936+0000 7fdb22e47640 1 -- start start 2024-04-26T21:36:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.937+0000 7fdb22e47640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdb1c15f3a0 0x7fdb1c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.937+0000 7fdb22e47640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1c0d0e20 con 0x7fdb1c0d2330 2024-04-26T21:36:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.937+0000 7fdb22e47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1c0b1d70 con 0x7fdb1c0d1770 2024-04-26T21:36:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.937+0000 7fdb22e47640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdb1c0b2050 con 0x7fdb1c15f3a0 2024-04-26T21:36:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.937+0000 7fdb213e4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdb1c0d2330 0x7fdb1c0b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47920/0 (socket says 172.21.15.40:47920) 2024-04-26T21:36:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.937+0000 7fdb213e4640 1 -- 172.21.15.40:0/414511909 learned_addr learned my addr 172.21.15.40:0/414511909 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb20be3640 1 --2- 172.21.15.40:0/414511909 >> v2:172.21.15.184:3300/0 conn(0x7fdb1c15f3a0 0x7fdb1c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455630858 0 0) 0x7fdb1c0d0e20 con 0x7fdb1c0d2330 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb08003520 con 0x7fdb1c0d2330 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4247851370 0 0) 0x7fdb08003520 con 0x7fdb1c0d2330 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 >> v2:172.21.15.184:3300/0 conn(0x7fdb1c15f3a0 msgr2=0x7fdb1c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 --2- 172.21.15.40:0/414511909 >> v2:172.21.15.184:3300/0 conn(0x7fdb1c15f3a0 0x7fdb1c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 >> v1:172.21.15.184:6789/0 conn(0x7fdb1c0d1770 legacy=0x7fdb1c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c0b2330 con 0x7fdb1c0d2330 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fdb0c001690 con 0x7fdb1c0d2330 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 >> v1:172.21.15.40:6789/0 conn(0x7fdb1c0d2330 legacy=0x7fdb1c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb080039b0 0x7fdb08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08004800 0x7fdb08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb080051c0 0x7fdb0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0b2050 con 0x7fdb080039b0 2024-04-26T21:36:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0800c060 con 0x7fdb08004800 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0800c260 con 0x7fdb080051c0 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb20be3640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb080039b0 0x7fdb08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb1bfff640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08004800 0x7fdb08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb213e4640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb080051c0 0x7fdb0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb20be3640 1 -- 172.21.15.40:0/414511909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb080051c0 msgr2=0x7fdb0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.938+0000 7fdb20be3640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb080051c0 0x7fdb0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.939+0000 7fdb20be3640 1 -- 172.21.15.40:0/414511909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08004800 msgr2=0x7fdb08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.939+0000 7fdb20be3640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08004800 0x7fdb08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.939+0000 7fdb20be3640 1 -- 172.21.15.40:0/414511909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb08003520 con 0x7fdb080039b0 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.939+0000 7fdb213e4640 1 -- 172.21.15.40:0/414511909 reap_dead start 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.939+0000 7fdb20be3640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb080039b0 0x7fdb08003dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009fc0 tx=0x7fdb08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ea7b1ab2c980a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.939+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14034020 con 0x7fdb080039b0 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.939+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb140305f0 con 0x7fdb080039b0 2024-04-26T21:36:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.939+0000 7fdb1b7fe640 1 -- 172.21.15.40:0/414511909 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14009c10 con 0x7fdb080039b0 2024-04-26T21:36:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.940+0000 7fdb22e47640 1 -- 172.21.15.40:0/414511909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb080039b0 msgr2=0x7fdb08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.940+0000 7fdb22e47640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb080039b0 0x7fdb08003dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009fc0 tx=0x7fdb08005830 comp rx=0 tx=0).stop 2024-04-26T21:36:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.940+0000 7fdb22e47640 1 -- 172.21.15.40:0/414511909 shutdown_connections 2024-04-26T21:36:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.940+0000 7fdb22e47640 1 --2- 172.21.15.40:0/414511909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb080039b0 0x7fdb08003dd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.940+0000 7fdb22e47640 1 -- 172.21.15.40:0/414511909 >> 172.21.15.40:0/414511909 conn(0x7fdb1c0d4f90 msgr2=0x7fdb1c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.940+0000 7fdb22e47640 1 -- 172.21.15.40:0/414511909 shutdown_connections 2024-04-26T21:36:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.940+0000 7fdb22e47640 1 -- 172.21.15.40:0/414511909 wait complete. 2024-04-26T21:36:28.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.941+0000 7fdb22e47640 1 Processor -- start 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.941+0000 7fdb22e47640 1 -- start start 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb22e47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb080039b0 0x7fdb1c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb22e47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c161c50 0x7fdb1c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb22e47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb1c0b5fd0 0x7fdb1c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb22e47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0d0870 con 0x7fdb1c161c50 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb22e47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0d06f0 con 0x7fdb080039b0 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb22e47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c0d09f0 con 0x7fdb1c0b5fd0 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb1bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c161c50 0x7fdb1c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb1bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c161c50 0x7fdb1c0b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52834/0 (socket says 172.21.15.40:52834) 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb1bfff640 1 -- 172.21.15.40:0/1665581199 learned_addr learned my addr 172.21.15.40:0/1665581199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb1bfff640 1 -- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb1c0b5fd0 msgr2=0x7fdb1c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb1bfff640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb1c0b5fd0 0x7fdb1c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb20be3640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb080039b0 0x7fdb1c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb1bfff640 1 -- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb080039b0 msgr2=0x7fdb1c159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb1bfff640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb080039b0 0x7fdb1c159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb1bfff640 1 -- 172.21.15.40:0/1665581199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c160bb0 con 0x7fdb1c161c50 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.942+0000 7fdb20be3640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb080039b0 0x7fdb1c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.943+0000 7fdb1bfff640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c161c50 0x7fdb1c0b5a90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400f7a0 tx=0x7fdb0400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.943+0000 7fdb197fa640 1 -- 172.21.15.40:0/1665581199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040050d0 con 0x7fdb1c161c50 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.943+0000 7fdb197fa640 1 -- 172.21.15.40:0/1665581199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb04018070 con 0x7fdb1c161c50 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.943+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c160e90 con 0x7fdb1c161c50 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.943+0000 7fdb197fa640 1 -- 172.21.15.40:0/1665581199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400d730 con 0x7fdb1c161c50 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.943+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c161450 con 0x7fdb1c161c50 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.944+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c0694f0 con 0x7fdb1c161c50 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.944+0000 7fdb197fa640 1 -- 172.21.15.40:0/1665581199 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb0400d8d0 con 0x7fdb1c161c50 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.945+0000 7fdb197fa640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdafc06c920 0x7fdafc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.945+0000 7fdb197fa640 1 -- 172.21.15.40:0/1665581199 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fdb0408bb90 con 0x7fdb1c161c50 2024-04-26T21:36:28.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.945+0000 7fdb20be3640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdafc06c920 0x7fdafc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:28.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.946+0000 7fdb20be3640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdafc06c920 0x7fdafc06ede0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb14035000 tx=0x7fdb1402f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:28.949+0000 7fdb197fa640 1 -- 172.21.15.40:0/1665581199 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb04059d90 con 0x7fdb1c161c50 2024-04-26T21:36:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.130+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb1c0b6860 con 0x7fdafc06c920 2024-04-26T21:36:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.136+0000 7fdb197fa640 1 -- 172.21.15.40:0/1665581199 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240074 (secure 0 0 0) 0x7fdb1c0b6860 con 0x7fdafc06c920 2024-04-26T21:36:29.140 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:29.141 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:36:29.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdafc06c920 msgr2=0x7fdafc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdafc06c920 0x7fdafc06ede0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb14035000 tx=0x7fdb1402f000 comp rx=0 tx=0).stop 2024-04-26T21:36:29.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c161c50 msgr2=0x7fdb1c0b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c161c50 0x7fdb1c0b5a90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400f7a0 tx=0x7fdb0400fc60 comp rx=0 tx=0).stop 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 shutdown_connections 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdafc06c920 0x7fdafc06ede0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb1c0b5fd0 0x7fdb1c0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c161c50 0x7fdb1c0b5a90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 --2- 172.21.15.40:0/1665581199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb080039b0 0x7fdb1c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 >> 172.21.15.40:0/1665581199 conn(0x7fdb1c0d4f90 msgr2=0x7fdb1c0b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 shutdown_connections 2024-04-26T21:36:29.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.139+0000 7fdb22e47640 1 -- 172.21.15.40:0/1665581199 wait complete. 2024-04-26T21:36:29.151 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T21:36:28.011710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220460,"kb_used_data":4260,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754260,"statfs":{"total":773094113280,"available":772868362240,"internally_reserved":0,"allocated":4362240,"data_stored":2518189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.609682"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408704+0000","last_change":"2024-04-26T21:36:22.408704+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408704+0000","last_undegraded":"2024-04-26T21:36:22.408704+0000","last_fullsized":"2024-04-26T21:36:22.408704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406003+0000","last_change":"2024-04-26T21:36:22.406003+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406003+0000","last_undegraded":"2024-04-26T21:36:22.406003+0000","last_fullsized":"2024-04-26T21:36:22.406003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406957+0000","last_change":"2024-04-26T21:36:22.406957+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406957+0000","last_undegraded":"2024-04-26T21:36:22.406957+0000","last_fullsized":"2024-04-26T21:36:22.406957+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404256+0000","last_change":"2024-04-26T21:36:22.404256+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404256+0000","last_undegraded":"2024-04-26T21:36:22.404256+0000","last_fullsized":"2024-04-26T21:36:22.404256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404676+0000","last_change":"2024-04-26T21:36:22.404676+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404676+0000","last_undegraded":"2024-04-26T21:36:22.404676+0000","last_fullsized":"2024-04-26T21:36:22.404676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407334+0000","last_change":"2024-04-26T21:36:22.407334+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407334+0000","last_undegraded":"2024-04-26T21:36:22.407334+0000","last_fullsized":"2024-04-26T21:36:22.407334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406817+0000","last_change":"2024-04-26T21:36:22.406817+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406817+0000","last_undegraded":"2024-04-26T21:36:22.406817+0000","last_fullsized":"2024-04-26T21:36:22.406817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404794+0000","last_change":"2024-04-26T21:36:22.404794+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404794+0000","last_undegraded":"2024-04-26T21:36:22.404794+0000","last_fullsized":"2024-04-26T21:36:22.404794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412333+0000","last_change":"2024-04-26T21:36:22.412333+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412333+0000","last_undegraded":"2024-04-26T21:36:22.412333+0000","last_fullsized":"2024-04-26T21:36:22.412333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404182+0000","last_change":"2024-04-26T21:36:22.404182+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404182+0000","last_undegraded":"2024-04-26T21:36:22.404182+0000","last_fullsized":"2024-04-26T21:36:22.404182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404209+0000","last_change":"2024-04-26T21:36:22.404209+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404209+0000","last_undegraded":"2024-04-26T21:36:22.404209+0000","last_fullsized":"2024-04-26T21:36:22.404209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412389+0000","last_change":"2024-04-26T21:36:22.412389+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412389+0000","last_undegraded":"2024-04-26T21:36:22.412389+0000","last_fullsized":"2024-04-26T21:36:22.412389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412388+0000","last_change":"2024-04-26T21:36:22.412388+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412388+0000","last_undegraded":"2024-04-26T21:36:22.412388+0000","last_fullsized":"2024-04-26T21:36:22.412388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406854+0000","last_change":"2024-04-26T21:36:22.406854+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406854+0000","last_undegraded":"2024-04-26T21:36:22.406854+0000","last_fullsized":"2024-04-26T21:36:22.406854+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408704+0000","last_change":"2024-04-26T21:36:22.408704+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408704+0000","last_undegraded":"2024-04-26T21:36:22.408704+0000","last_fullsized":"2024-04-26T21:36:22.408704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407136+0000","last_change":"2024-04-26T21:36:22.407136+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407136+0000","last_undegraded":"2024-04-26T21:36:22.407136+0000","last_fullsized":"2024-04-26T21:36:22.407136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404902+0000","last_change":"2024-04-26T21:36:22.404902+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404902+0000","last_undegraded":"2024-04-26T21:36:22.404902+0000","last_fullsized":"2024-04-26T21:36:22.404902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404184+0000","last_change":"2024-04-26T21:36:22.404184+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404184+0000","last_undegraded":"2024-04-26T21:36:22.404184+0000","last_fullsized":"2024-04-26T21:36:22.404184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403945+0000","last_change":"2024-04-26T21:36:22.403945+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403945+0000","last_undegraded":"2024-04-26T21:36:22.403945+0000","last_fullsized":"2024-04-26T21:36:22.403945+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407361+0000","last_change":"2024-04-26T21:36:22.407361+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407361+0000","last_undegraded":"2024-04-26T21:36:22.407361+0000","last_fullsized":"2024-04-26T21:36:22.407361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403862+0000","last_change":"2024-04-26T21:36:22.403862+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403862+0000","last_undegraded":"2024-04-26T21:36:22.403862+0000","last_fullsized":"2024-04-26T21:36:22.403862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403889+0000","last_change":"2024-04-26T21:36:22.403889+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403889+0000","last_undegraded":"2024-04-26T21:36:22.403889+0000","last_fullsized":"2024-04-26T21:36:22.403889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407307+0000","last_change":"2024-04-26T21:36:22.407307+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407307+0000","last_undegraded":"2024-04-26T21:36:22.407307+0000","last_fullsized":"2024-04-26T21:36:22.407307+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403907+0000","last_change":"2024-04-26T21:36:22.403907+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403907+0000","last_undegraded":"2024-04-26T21:36:22.403907+0000","last_fullsized":"2024-04-26T21:36:22.403907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404196+0000","last_change":"2024-04-26T21:36:22.404196+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404196+0000","last_undegraded":"2024-04-26T21:36:22.404196+0000","last_fullsized":"2024-04-26T21:36:22.404196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403911+0000","last_change":"2024-04-26T21:36:22.403911+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403911+0000","last_undegraded":"2024-04-26T21:36:22.403911+0000","last_fullsized":"2024-04-26T21:36:22.403911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.405895+0000","last_change":"2024-04-26T21:36:22.405895+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.405895+0000","last_undegraded":"2024-04-26T21:36:22.405895+0000","last_fullsized":"2024-04-26T21:36:22.405895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406117+0000","last_change":"2024-04-26T21:36:22.406117+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406117+0000","last_undegraded":"2024-04-26T21:36:22.406117+0000","last_fullsized":"2024-04-26T21:36:22.406117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408298+0000","last_change":"2024-04-26T21:36:22.408298+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408298+0000","last_undegraded":"2024-04-26T21:36:22.408298+0000","last_fullsized":"2024-04-26T21:36:22.408298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404711+0000","last_change":"2024-04-26T21:36:22.404711+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404711+0000","last_undegraded":"2024-04-26T21:36:22.404711+0000","last_fullsized":"2024-04-26T21:36:22.404711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412312+0000","last_change":"2024-04-26T21:36:22.412312+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412312+0000","last_undegraded":"2024-04-26T21:36:22.412312+0000","last_fullsized":"2024-04-26T21:36:22.412312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407200+0000","last_change":"2024-04-26T21:36:22.407200+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407200+0000","last_undegraded":"2024-04-26T21:36:22.407200+0000","last_fullsized":"2024-04-26T21:36:22.407200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T21:36:22.408318+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:36:22.408318+0000","last_peered":"2024-04-26T21:36:22.408318+0000","last_clean":"2024-04-26T21:36:22.408318+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:36:22.408318+0000","last_undegraded":"2024-04-26T21:36:22.408318+0000","last_fullsized":"2024-04-26T21:36:22.408318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":753669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":163301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":163301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:36:29.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-26T21:36:29.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-26T21:36:29.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-26T21:36:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-26T21:36:29.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.408704+0000', 'last_change': '2024-04-26T21:36:22.408704+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.408704+0000', 'last_undegraded': '2024-04-26T21:36:22.408704+0000', 'last_fullsized': '2024-04-26T21:36:22.408704+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.406003+0000', 'last_change': '2024-04-26T21:36:22.406003+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.406003+0000', 'last_undegraded': '2024-04-26T21:36:22.406003+0000', 'last_fullsized': '2024-04-26T21:36:22.406003+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.406957+0000', 'last_change': '2024-04-26T21:36:22.406957+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.406957+0000', 'last_undegraded': '2024-04-26T21:36:22.406957+0000', 'last_fullsized': '2024-04-26T21:36:22.406957+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404256+0000', 'last_change': '2024-04-26T21:36:22.404256+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404256+0000', 'last_undegraded': '2024-04-26T21:36:22.404256+0000', 'last_fullsized': '2024-04-26T21:36:22.404256+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404676+0000', 'last_change': '2024-04-26T21:36:22.404676+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404676+0000', 'last_undegraded': '2024-04-26T21:36:22.404676+0000', 'last_fullsized': '2024-04-26T21:36:22.404676+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:36:29.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.407334+0000', 'last_change': '2024-04-26T21:36:22.407334+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.407334+0000', 'last_undegraded': '2024-04-26T21:36:22.407334+0000', 'last_fullsized': '2024-04-26T21:36:22.407334+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.406817+0000', 'last_change': '2024-04-26T21:36:22.406817+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.406817+0000', 'last_undegraded': '2024-04-26T21:36:22.406817+0000', 'last_fullsized': '2024-04-26T21:36:22.406817+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404794+0000', 'last_change': '2024-04-26T21:36:22.404794+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404794+0000', 'last_undegraded': '2024-04-26T21:36:22.404794+0000', 'last_fullsized': '2024-04-26T21:36:22.404794+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:36:29.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.412333+0000', 'last_change': '2024-04-26T21:36:22.412333+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.412333+0000', 'last_undegraded': '2024-04-26T21:36:22.412333+0000', 'last_fullsized': '2024-04-26T21:36:22.412333+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404182+0000', 'last_change': '2024-04-26T21:36:22.404182+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404182+0000', 'last_undegraded': '2024-04-26T21:36:22.404182+0000', 'last_fullsized': '2024-04-26T21:36:22.404182+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-26T21:36:29.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404209+0000', 'last_change': '2024-04-26T21:36:22.404209+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404209+0000', 'last_undegraded': '2024-04-26T21:36:22.404209+0000', 'last_fullsized': '2024-04-26T21:36:22.404209+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.412389+0000', 'last_change': '2024-04-26T21:36:22.412389+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.412389+0000', 'last_undegraded': '2024-04-26T21:36:22.412389+0000', 'last_fullsized': '2024-04-26T21:36:22.412389+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.412388+0000', 'last_change': '2024-04-26T21:36:22.412388+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.412388+0000', 'last_undegraded': '2024-04-26T21:36:22.412388+0000', 'last_fullsized': '2024-04-26T21:36:22.412388+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.406854+0000', 'last_change': '2024-04-26T21:36:22.406854+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.406854+0000', 'last_undegraded': '2024-04-26T21:36:22.406854+0000', 'last_fullsized': '2024-04-26T21:36:22.406854+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.408704+0000', 'last_change': '2024-04-26T21:36:22.408704+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.408704+0000', 'last_undegraded': '2024-04-26T21:36:22.408704+0000', 'last_fullsized': '2024-04-26T21:36:22.408704+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.407136+0000', 'last_change': '2024-04-26T21:36:22.407136+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.407136+0000', 'last_undegraded': '2024-04-26T21:36:22.407136+0000', 'last_fullsized': '2024-04-26T21:36:22.407136+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-26T21:36:29.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404902+0000', 'last_change': '2024-04-26T21:36:22.404902+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404902+0000', 'last_undegraded': '2024-04-26T21:36:22.404902+0000', 'last_fullsized': '2024-04-26T21:36:22.404902+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404184+0000', 'last_change': '2024-04-26T21:36:22.404184+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404184+0000', 'last_undegraded': '2024-04-26T21:36:22.404184+0000', 'last_fullsized': '2024-04-26T21:36:22.404184+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.403945+0000', 'last_change': '2024-04-26T21:36:22.403945+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.403945+0000', 'last_undegraded': '2024-04-26T21:36:22.403945+0000', 'last_fullsized': '2024-04-26T21:36:22.403945+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:36:29.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.407361+0000', 'last_change': '2024-04-26T21:36:22.407361+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.407361+0000', 'last_undegraded': '2024-04-26T21:36:22.407361+0000', 'last_fullsized': '2024-04-26T21:36:22.407361+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.403862+0000', 'last_change': '2024-04-26T21:36:22.403862+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.403862+0000', 'last_undegraded': '2024-04-26T21:36:22.403862+0000', 'last_fullsized': '2024-04-26T21:36:22.403862+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-26T21:36:29.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.403889+0000', 'last_change': '2024-04-26T21:36:22.403889+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.403889+0000', 'last_undegraded': '2024-04-26T21:36:22.403889+0000', 'last_fullsized': '2024-04-26T21:36:22.403889+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.407307+0000', 'last_change': '2024-04-26T21:36:22.407307+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.407307+0000', 'last_undegraded': '2024-04-26T21:36:22.407307+0000', 'last_fullsized': '2024-04-26T21:36:22.407307+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.403907+0000', 'last_change': '2024-04-26T21:36:22.403907+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.403907+0000', 'last_undegraded': '2024-04-26T21:36:22.403907+0000', 'last_fullsized': '2024-04-26T21:36:22.403907+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-26T21:36:29.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404196+0000', 'last_change': '2024-04-26T21:36:22.404196+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404196+0000', 'last_undegraded': '2024-04-26T21:36:22.404196+0000', 'last_fullsized': '2024-04-26T21:36:22.404196+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.403911+0000', 'last_change': '2024-04-26T21:36:22.403911+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.403911+0000', 'last_undegraded': '2024-04-26T21:36:22.403911+0000', 'last_fullsized': '2024-04-26T21:36:22.403911+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.405895+0000', 'last_change': '2024-04-26T21:36:22.405895+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.405895+0000', 'last_undegraded': '2024-04-26T21:36:22.405895+0000', 'last_fullsized': '2024-04-26T21:36:22.405895+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:36:29.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.406117+0000', 'last_change': '2024-04-26T21:36:22.406117+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.406117+0000', 'last_undegraded': '2024-04-26T21:36:22.406117+0000', 'last_fullsized': '2024-04-26T21:36:22.406117+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.408298+0000', 'last_change': '2024-04-26T21:36:22.408298+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.408298+0000', 'last_undegraded': '2024-04-26T21:36:22.408298+0000', 'last_fullsized': '2024-04-26T21:36:22.408298+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.404711+0000', 'last_change': '2024-04-26T21:36:22.404711+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.404711+0000', 'last_undegraded': '2024-04-26T21:36:22.404711+0000', 'last_fullsized': '2024-04-26T21:36:22.404711+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.412312+0000', 'last_change': '2024-04-26T21:36:22.412312+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.412312+0000', 'last_undegraded': '2024-04-26T21:36:22.412312+0000', 'last_fullsized': '2024-04-26T21:36:22.412312+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:36:23.400474+0000', 'last_change': '2024-04-26T21:36:23.400474+0000', 'last_active': '2024-04-26T21:36:23.400474+0000', 'last_peered': '2024-04-26T21:36:23.400474+0000', 'last_clean': '2024-04-26T21:36:23.400474+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:23.400474+0000', 'last_undegraded': '2024-04-26T21:36:23.400474+0000', 'last_fullsized': '2024-04-26T21:36:23.400474+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:23.400474+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-26T21:36:29.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:36:22.407200+0000', 'last_change': '2024-04-26T21:36:22.407200+0000', 'last_active': '2024-04-26T21:36:22.395949+0000', 'last_peered': '2024-04-26T21:36:22.395949+0000', 'last_clean': '2024-04-26T21:36:22.395949+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:36:22.407200+0000', 'last_undegraded': '2024-04-26T21:36:22.407200+0000', 'last_fullsized': '2024-04-26T21:36:22.407200+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'last_clean_scrub_stamp': '2024-04-26T21:36:22.395949+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:36:29.173 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:36:29.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.290+0000 7f1a3eea2640 1 Processor -- start 2024-04-26T21:36:29.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.290+0000 7f1a3eea2640 1 -- start start 2024-04-26T21:36:29.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.291+0000 7f1a3eea2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1a3815f3a0 0x7f1a380d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.291+0000 7f1a3eea2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1a380d2330 0x7f1a380b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.291+0000 7f1a3eea2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a380d0e20 con 0x7f1a380d1770 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.291+0000 7f1a3eea2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1a380b1d70 con 0x7f1a3815f3a0 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.291+0000 7f1a3eea2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1a380b2050 con 0x7f1a380d2330 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.291+0000 7f1a3cc3e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1a3815f3a0 0x7f1a380d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.291+0000 7f1a3cc3e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1a3815f3a0 0x7f1a380d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52848/0 (socket says 172.21.15.40:52848) 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.291+0000 7f1a3cc3e640 1 -- 172.21.15.40:0/503425055 learned_addr learned my addr 172.21.15.40:0/503425055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.292+0000 7f1a3d43f640 1 --2- 172.21.15.40:0/503425055 >> v2:172.21.15.184:3300/0 conn(0x7f1a380d2330 0x7f1a380b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.292+0000 7f1a3cc3e640 1 -- 172.21.15.40:0/503425055 >> v2:172.21.15.184:3300/0 conn(0x7f1a380d2330 msgr2=0x7f1a380b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.292+0000 7f1a3cc3e640 1 --2- 172.21.15.40:0/503425055 >> v2:172.21.15.184:3300/0 conn(0x7f1a380d2330 0x7f1a380b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.292+0000 7f1a3cc3e640 1 -- 172.21.15.40:0/503425055 >> v1:172.21.15.184:6789/0 conn(0x7f1a380d1770 legacy=0x7f1a380d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.292+0000 7f1a3cc3e640 1 -- 172.21.15.40:0/503425055 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a380b2330 con 0x7f1a3815f3a0 2024-04-26T21:36:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.292+0000 7f1a3cc3e640 1 --2- 172.21.15.40:0/503425055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3815f3a0 0x7f1a380d06f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1a38073b50 tx=0x7f1a3002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38c7ea3cb6e535d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a377fe640 1 -- 172.21.15.40:0/503425055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30036070 con 0x7f1a3815f3a0 2024-04-26T21:36:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a377fe640 1 -- 172.21.15.40:0/503425055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a30031600 con 0x7f1a3815f3a0 2024-04-26T21:36:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a377fe640 1 -- 172.21.15.40:0/503425055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a300319d0 con 0x7f1a3815f3a0 2024-04-26T21:36:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a3eea2640 1 -- 172.21.15.40:0/503425055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3815f3a0 msgr2=0x7f1a380d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/503425055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3815f3a0 0x7f1a380d06f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1a38073b50 tx=0x7f1a3002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a3eea2640 1 -- 172.21.15.40:0/503425055 shutdown_connections 2024-04-26T21:36:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/503425055 >> v2:172.21.15.184:3300/0 conn(0x7f1a380d2330 0x7f1a380b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/503425055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3815f3a0 0x7f1a380d06f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a3eea2640 1 -- 172.21.15.40:0/503425055 >> 172.21.15.40:0/503425055 conn(0x7f1a380d4f90 msgr2=0x7f1a380d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a3eea2640 1 -- 172.21.15.40:0/503425055 shutdown_connections 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.293+0000 7f1a3eea2640 1 -- 172.21.15.40:0/503425055 wait complete. 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.294+0000 7f1a3eea2640 1 Processor -- start 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.294+0000 7f1a3eea2640 1 -- start start 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a3eea2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a380d1770 0x7f1a380cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a3eea2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a380d2330 0x7f1a381553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a3eea2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3815f3a0 0x7f1a380ba050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a3eea2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a380b2f10 con 0x7f1a380d2330 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a3eea2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a380b2d90 con 0x7f1a3815f3a0 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a3eea2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1a380b3090 con 0x7f1a380d1770 2024-04-26T21:36:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a380d2330 0x7f1a381553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a380d2330 0x7f1a381553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52850/0 (socket says 172.21.15.40:52850) 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a37fff640 1 -- 172.21.15.40:0/801261980 learned_addr learned my addr 172.21.15.40:0/801261980 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.295+0000 7f1a3d43f640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3815f3a0 0x7f1a380ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a3cc3e640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a380d1770 0x7f1a380cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a37fff640 1 -- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a380d1770 msgr2=0x7f1a380cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a37fff640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a380d1770 0x7f1a380cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a37fff640 1 -- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3815f3a0 msgr2=0x7f1a380ba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a37fff640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3815f3a0 0x7f1a380ba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a37fff640 1 -- 172.21.15.40:0/801261980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a380bae00 con 0x7f1a380d2330 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a37fff640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a380d2330 0x7f1a381553c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00c460 tx=0x7f1a2c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a357fa640 1 -- 172.21.15.40:0/801261980 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c011790 con 0x7f1a380d2330 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a357fa640 1 -- 172.21.15.40:0/801261980 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2c017070 con 0x7f1a380d2330 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a380ba590 con 0x7f1a380d2330 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a357fa640 1 -- 172.21.15.40:0/801261980 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c011bb0 con 0x7f1a380d2330 2024-04-26T21:36:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.296+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a380bab20 con 0x7f1a380d2330 2024-04-26T21:36:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.297+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a380694f0 con 0x7f1a380d2330 2024-04-26T21:36:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.297+0000 7f1a357fa640 1 -- 172.21.15.40:0/801261980 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a2c01c460 con 0x7f1a380d2330 2024-04-26T21:36:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.298+0000 7f1a357fa640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1a2406c8f0 0x7f1a2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.298+0000 7f1a357fa640 1 -- 172.21.15.40:0/801261980 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1a2c08bab0 con 0x7f1a380d2330 2024-04-26T21:36:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.298+0000 7f1a3cc3e640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1a2406c8f0 0x7f1a2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.299+0000 7f1a3cc3e640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1a2406c8f0 0x7f1a2406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a30033000 tx=0x7f1a30009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.302+0000 7f1a357fa640 1 -- 172.21.15.40:0/801261980 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a2c059de0 con 0x7f1a380d2330 2024-04-26T21:36:29.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.560+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1a38069dd0 con 0x7f1a380d2330 2024-04-26T21:36:29.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.561+0000 7f1a357fa640 1 -- 172.21.15.40:0/801261980 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1109 (secure 0 0 0) 0x7f1a2c0596e0 con 0x7f1a380d2330 2024-04-26T21:36:29.562 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:29.562 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":103,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":38,"num_osds":8,"num_up_osds":8,"osd_up_since":1714167297,"num_in_osds":8,"osd_in_since":1714167287,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225751040,"bytes_avail":772868362240,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5364}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-26T21:36:14.009276+0000","services":{}},"progress_events":{}} 2024-04-26T21:36:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1a2406c8f0 msgr2=0x7f1a2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1a2406c8f0 0x7f1a2406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a30033000 tx=0x7f1a30009000 comp rx=0 tx=0).stop 2024-04-26T21:36:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a380d2330 msgr2=0x7f1a381553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a380d2330 0x7f1a381553c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00c460 tx=0x7f1a2c00c920 comp rx=0 tx=0).stop 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 shutdown_connections 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1a2406c8f0 0x7f1a2406edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3815f3a0 0x7f1a380ba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a380d2330 0x7f1a381553c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 --2- 172.21.15.40:0/801261980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a380d1770 0x7f1a380cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 >> 172.21.15.40:0/801261980 conn(0x7f1a380d4f90 msgr2=0x7f1a380d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 shutdown_connections 2024-04-26T21:36:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.563+0000 7f1a3eea2640 1 -- 172.21.15.40:0/801261980 wait complete. 2024-04-26T21:36:29.575 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:36:29.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:36:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.675+0000 7f3e3d4dc640 1 Processor -- start 2024-04-26T21:36:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.675+0000 7f3e3d4dc640 1 -- start start 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e3d4dc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e38162810 0x7f3e38070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e3d4dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380b2340 0x7f3e380cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e3d4dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e380d10d0 con 0x7f3e380cd930 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e3d4dc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3e380d0750 con 0x7f3e38162810 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e3d4dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3e380d16a0 con 0x7f3e380b2340 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380b2340 0x7f3e380cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e36ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e38162810 0x7f3e38070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e377fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f3e380cd930 0x7f3e380d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39316/0 (socket says 172.21.15.40:39316) 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e377fe640 1 -- 172.21.15.40:0/2036785077 learned_addr learned my addr 172.21.15.40:0/2036785077 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e367fc640 1 -- 172.21.15.40:0/2036785077 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230353961 0 0) 0x7f3e380d10d0 con 0x7f3e380cd930 2024-04-26T21:36:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.676+0000 7f3e367fc640 1 -- 172.21.15.40:0/2036785077 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e24003520 con 0x7f3e380cd930 2024-04-26T21:36:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.677+0000 7f3e36ffd640 1 -- 172.21.15.40:0/2036785077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380b2340 msgr2=0x7f3e380cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.677+0000 7f3e36ffd640 1 --2- 172.21.15.40:0/2036785077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380b2340 0x7f3e380cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.677+0000 7f3e36ffd640 1 -- 172.21.15.40:0/2036785077 >> v1:172.21.15.184:6789/0 conn(0x7f3e380cd930 legacy=0x7f3e380d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.677+0000 7f3e36ffd640 1 -- 172.21.15.40:0/2036785077 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e380d1370 con 0x7f3e38162810 2024-04-26T21:36:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.677+0000 7f3e2ffff640 1 --2- 172.21.15.40:0/2036785077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380b2340 0x7f3e380cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.677+0000 7f3e36ffd640 1 --2- 172.21.15.40:0/2036785077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e38162810 0x7f3e38070350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e280097a0 tx=0x7f3e2802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75ac9ab95715745 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.677+0000 7f3e367fc640 1 -- 172.21.15.40:0/2036785077 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28037070 con 0x7f3e38162810 2024-04-26T21:36:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e367fc640 1 -- 172.21.15.40:0/2036785077 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e28032440 con 0x7f3e38162810 2024-04-26T21:36:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e367fc640 1 -- 172.21.15.40:0/2036785077 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28032810 con 0x7f3e38162810 2024-04-26T21:36:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2036785077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e38162810 msgr2=0x7f3e38070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2036785077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e38162810 0x7f3e38070350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e280097a0 tx=0x7f3e2802a180 comp rx=0 tx=0).stop 2024-04-26T21:36:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2036785077 shutdown_connections 2024-04-26T21:36:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2036785077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380b2340 0x7f3e380cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2036785077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e38162810 0x7f3e38070350 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2036785077 >> 172.21.15.40:0/2036785077 conn(0x7f3e380d2da0 msgr2=0x7f3e380d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2036785077 shutdown_connections 2024-04-26T21:36:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.678+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2036785077 wait complete. 2024-04-26T21:36:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.679+0000 7f3e3d4dc640 1 Processor -- start 2024-04-26T21:36:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.679+0000 7f3e3d4dc640 1 -- start start 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e3d4dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e380b2340 0x7f3e38153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e3d4dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380cd930 0x7f3e3815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e3d4dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e38162810 0x7f3e3815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e3d4dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38070890 con 0x7f3e38162810 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e3d4dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3e380d0470 con 0x7f3e380b2340 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e3d4dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3e380d2240 con 0x7f3e380cd930 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e38162810 0x7f3e3815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e38162810 0x7f3e3815bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52854/0 (socket says 172.21.15.40:52854) 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e377fe640 1 -- 172.21.15.40:0/2032060072 learned_addr learned my addr 172.21.15.40:0/2032060072 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e36ffd640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e380b2340 0x7f3e38153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.680+0000 7f3e2ffff640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380cd930 0x7f3e3815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e377fe640 1 -- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380cd930 msgr2=0x7f3e3815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e377fe640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380cd930 0x7f3e3815ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e377fe640 1 -- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e380b2340 msgr2=0x7f3e38153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e377fe640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e380b2340 0x7f3e38153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e377fe640 1 -- 172.21.15.40:0/2032060072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e38160900 con 0x7f3e38162810 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e36ffd640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e380b2340 0x7f3e38153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e2ffff640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380cd930 0x7f3e3815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e377fe640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e38162810 0x7f3e3815bc50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000cae0 tx=0x7f3e3000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e2f7fe640 1 -- 172.21.15.40:0/2032060072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30017070 con 0x7f3e38162810 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e2f7fe640 1 -- 172.21.15.40:0/2032060072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3000ac40 con 0x7f3e38162810 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e38160be0 con 0x7f3e38162810 2024-04-26T21:36:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e2f7fe640 1 -- 172.21.15.40:0/2032060072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30012640 con 0x7f3e38162810 2024-04-26T21:36:29.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.681+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e381611a0 con 0x7f3e38162810 2024-04-26T21:36:29.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.682+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3806f260 con 0x7f3e38162810 2024-04-26T21:36:29.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.683+0000 7f3e2f7fe640 1 -- 172.21.15.40:0/2032060072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e3001a070 con 0x7f3e38162810 2024-04-26T21:36:29.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.683+0000 7f3e2f7fe640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e1806c8f0 0x7f3e1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:29.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.683+0000 7f3e2f7fe640 1 -- 172.21.15.40:0/2032060072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f3e3008bbf0 con 0x7f3e38162810 2024-04-26T21:36:29.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.683+0000 7f3e36ffd640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e1806c8f0 0x7f3e1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:29.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.684+0000 7f3e36ffd640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e1806c8f0 0x7f3e1806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e28034000 tx=0x7f3e2802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:29.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.687+0000 7f3e2f7fe640 1 -- 172.21.15.40:0/2032060072 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e3806f260 con 0x7f3e38162810 2024-04-26T21:36:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.869+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e380694f0 con 0x7f3e1806c8f0 2024-04-26T21:36:29.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.875+0000 7f3e2f7fe640 1 -- 172.21.15.40:0/2032060072 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240074 (secure 0 0 0) 0x7f3e380694f0 con 0x7f3e1806c8f0 2024-04-26T21:36:29.879 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:36:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e1806c8f0 msgr2=0x7f3e1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e1806c8f0 0x7f3e1806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e28034000 tx=0x7f3e2802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e38162810 msgr2=0x7f3e3815bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e38162810 0x7f3e3815bc50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000cae0 tx=0x7f3e3000a060 comp rx=0 tx=0).stop 2024-04-26T21:36:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 shutdown_connections 2024-04-26T21:36:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e1806c8f0 0x7f3e1806edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e38162810 0x7f3e3815bc50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e380cd930 0x7f3e3815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 --2- 172.21.15.40:0/2032060072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e380b2340 0x7f3e38153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 >> 172.21.15.40:0/2032060072 conn(0x7f3e380d2da0 msgr2=0x7f3e380b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 shutdown_connections 2024-04-26T21:36:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:29.878+0000 7f3e3d4dc640 1 -- 172.21.15.40:0/2032060072 wait complete. 2024-04-26T21:36:29.890 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T21:36:28.011710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220460,"kb_used_data":4260,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754260,"statfs":{"total":773094113280,"available":772868362240,"internally_reserved":0,"allocated":4362240,"data_stored":2518189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.609682"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408704+0000","last_change":"2024-04-26T21:36:22.408704+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408704+0000","last_undegraded":"2024-04-26T21:36:22.408704+0000","last_fullsized":"2024-04-26T21:36:22.408704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406003+0000","last_change":"2024-04-26T21:36:22.406003+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406003+0000","last_undegraded":"2024-04-26T21:36:22.406003+0000","last_fullsized":"2024-04-26T21:36:22.406003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406957+0000","last_change":"2024-04-26T21:36:22.406957+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406957+0000","last_undegraded":"2024-04-26T21:36:22.406957+0000","last_fullsized":"2024-04-26T21:36:22.406957+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404256+0000","last_change":"2024-04-26T21:36:22.404256+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404256+0000","last_undegraded":"2024-04-26T21:36:22.404256+0000","last_fullsized":"2024-04-26T21:36:22.404256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404676+0000","last_change":"2024-04-26T21:36:22.404676+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404676+0000","last_undegraded":"2024-04-26T21:36:22.404676+0000","last_fullsized":"2024-04-26T21:36:22.404676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407334+0000","last_change":"2024-04-26T21:36:22.407334+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407334+0000","last_undegraded":"2024-04-26T21:36:22.407334+0000","last_fullsized":"2024-04-26T21:36:22.407334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406817+0000","last_change":"2024-04-26T21:36:22.406817+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406817+0000","last_undegraded":"2024-04-26T21:36:22.406817+0000","last_fullsized":"2024-04-26T21:36:22.406817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404794+0000","last_change":"2024-04-26T21:36:22.404794+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404794+0000","last_undegraded":"2024-04-26T21:36:22.404794+0000","last_fullsized":"2024-04-26T21:36:22.404794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412333+0000","last_change":"2024-04-26T21:36:22.412333+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412333+0000","last_undegraded":"2024-04-26T21:36:22.412333+0000","last_fullsized":"2024-04-26T21:36:22.412333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404182+0000","last_change":"2024-04-26T21:36:22.404182+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404182+0000","last_undegraded":"2024-04-26T21:36:22.404182+0000","last_fullsized":"2024-04-26T21:36:22.404182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404209+0000","last_change":"2024-04-26T21:36:22.404209+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404209+0000","last_undegraded":"2024-04-26T21:36:22.404209+0000","last_fullsized":"2024-04-26T21:36:22.404209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412389+0000","last_change":"2024-04-26T21:36:22.412389+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412389+0000","last_undegraded":"2024-04-26T21:36:22.412389+0000","last_fullsized":"2024-04-26T21:36:22.412389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412388+0000","last_change":"2024-04-26T21:36:22.412388+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412388+0000","last_undegraded":"2024-04-26T21:36:22.412388+0000","last_fullsized":"2024-04-26T21:36:22.412388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406854+0000","last_change":"2024-04-26T21:36:22.406854+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406854+0000","last_undegraded":"2024-04-26T21:36:22.406854+0000","last_fullsized":"2024-04-26T21:36:22.406854+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408704+0000","last_change":"2024-04-26T21:36:22.408704+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408704+0000","last_undegraded":"2024-04-26T21:36:22.408704+0000","last_fullsized":"2024-04-26T21:36:22.408704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407136+0000","last_change":"2024-04-26T21:36:22.407136+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407136+0000","last_undegraded":"2024-04-26T21:36:22.407136+0000","last_fullsized":"2024-04-26T21:36:22.407136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404902+0000","last_change":"2024-04-26T21:36:22.404902+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404902+0000","last_undegraded":"2024-04-26T21:36:22.404902+0000","last_fullsized":"2024-04-26T21:36:22.404902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404184+0000","last_change":"2024-04-26T21:36:22.404184+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404184+0000","last_undegraded":"2024-04-26T21:36:22.404184+0000","last_fullsized":"2024-04-26T21:36:22.404184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403945+0000","last_change":"2024-04-26T21:36:22.403945+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403945+0000","last_undegraded":"2024-04-26T21:36:22.403945+0000","last_fullsized":"2024-04-26T21:36:22.403945+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407361+0000","last_change":"2024-04-26T21:36:22.407361+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407361+0000","last_undegraded":"2024-04-26T21:36:22.407361+0000","last_fullsized":"2024-04-26T21:36:22.407361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403862+0000","last_change":"2024-04-26T21:36:22.403862+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403862+0000","last_undegraded":"2024-04-26T21:36:22.403862+0000","last_fullsized":"2024-04-26T21:36:22.403862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403889+0000","last_change":"2024-04-26T21:36:22.403889+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403889+0000","last_undegraded":"2024-04-26T21:36:22.403889+0000","last_fullsized":"2024-04-26T21:36:22.403889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407307+0000","last_change":"2024-04-26T21:36:22.407307+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407307+0000","last_undegraded":"2024-04-26T21:36:22.407307+0000","last_fullsized":"2024-04-26T21:36:22.407307+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403907+0000","last_change":"2024-04-26T21:36:22.403907+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403907+0000","last_undegraded":"2024-04-26T21:36:22.403907+0000","last_fullsized":"2024-04-26T21:36:22.403907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404196+0000","last_change":"2024-04-26T21:36:22.404196+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404196+0000","last_undegraded":"2024-04-26T21:36:22.404196+0000","last_fullsized":"2024-04-26T21:36:22.404196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.403911+0000","last_change":"2024-04-26T21:36:22.403911+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.403911+0000","last_undegraded":"2024-04-26T21:36:22.403911+0000","last_fullsized":"2024-04-26T21:36:22.403911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.405895+0000","last_change":"2024-04-26T21:36:22.405895+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.405895+0000","last_undegraded":"2024-04-26T21:36:22.405895+0000","last_fullsized":"2024-04-26T21:36:22.405895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.406117+0000","last_change":"2024-04-26T21:36:22.406117+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.406117+0000","last_undegraded":"2024-04-26T21:36:22.406117+0000","last_fullsized":"2024-04-26T21:36:22.406117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.408298+0000","last_change":"2024-04-26T21:36:22.408298+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.408298+0000","last_undegraded":"2024-04-26T21:36:22.408298+0000","last_fullsized":"2024-04-26T21:36:22.408298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.404711+0000","last_change":"2024-04-26T21:36:22.404711+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.404711+0000","last_undegraded":"2024-04-26T21:36:22.404711+0000","last_fullsized":"2024-04-26T21:36:22.404711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.412312+0000","last_change":"2024-04-26T21:36:22.412312+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.412312+0000","last_undegraded":"2024-04-26T21:36:22.412312+0000","last_fullsized":"2024-04-26T21:36:22.412312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:36:23.400474+0000","last_change":"2024-04-26T21:36:23.400474+0000","last_active":"2024-04-26T21:36:23.400474+0000","last_peered":"2024-04-26T21:36:23.400474+0000","last_clean":"2024-04-26T21:36:23.400474+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:23.400474+0000","last_undegraded":"2024-04-26T21:36:23.400474+0000","last_fullsized":"2024-04-26T21:36:23.400474+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T21:36:22.407200+0000","last_change":"2024-04-26T21:36:22.407200+0000","last_active":"2024-04-26T21:36:22.395949+0000","last_peered":"2024-04-26T21:36:22.395949+0000","last_clean":"2024-04-26T21:36:22.395949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:36:22.407200+0000","last_undegraded":"2024-04-26T21:36:22.407200+0000","last_fullsized":"2024-04-26T21:36:22.407200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T21:36:22.408318+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:36:22.408318+0000","last_peered":"2024-04-26T21:36:22.408318+0000","last_clean":"2024-04-26T21:36:22.408318+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:36:22.408318+0000","last_undegraded":"2024-04-26T21:36:22.408318+0000","last_fullsized":"2024-04-26T21:36:22.408318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":753669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":163301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":163301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:36:32.896 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:36:33.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.008+0000 7f5d588f1640 1 Processor -- start 2024-04-26T21:36:33.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.008+0000 7f5d588f1640 1 -- start start 2024-04-26T21:36:33.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d588f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5d540d06f0 0x7f5d540d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d588f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 0x7f5d540b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d588f1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d540d0ba0 con 0x7f5d5415f3a0 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d588f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5d540d0e20 con 0x7f5d540d06f0 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d588f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d540b1d70 con 0x7f5d540b2850 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d51d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5d540d06f0 0x7f5d540d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d51d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5d540d06f0 0x7f5d540d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52860/0 (socket says 172.21.15.40:52860) 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d51d9b640 1 -- 172.21.15.40:0/1693654305 learned_addr learned my addr 172.21.15.40:0/1693654305 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d52d9d640 1 --2- 172.21.15.40:0/1693654305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 0x7f5d540b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d51d9b640 1 -- 172.21.15.40:0/1693654305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 msgr2=0x7f5d540b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d51d9b640 1 --2- 172.21.15.40:0/1693654305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 0x7f5d540b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d51d9b640 1 -- 172.21.15.40:0/1693654305 >> v1:172.21.15.184:6789/0 conn(0x7f5d5415f3a0 legacy=0x7f5d540d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.009+0000 7f5d51d9b640 1 -- 172.21.15.40:0/1693654305 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d540b2050 con 0x7f5d540d06f0 2024-04-26T21:36:33.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.010+0000 7f5d51d9b640 1 --2- 172.21.15.40:0/1693654305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d540d1e80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009870 tx=0x7f5d4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12e7cc04149f834 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.010+0000 7f5d5159a640 1 -- 172.21.15.40:0/1693654305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48037070 con 0x7f5d540d06f0 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.010+0000 7f5d5159a640 1 -- 172.21.15.40:0/1693654305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d48032440 con 0x7f5d540d06f0 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.010+0000 7f5d5159a640 1 -- 172.21.15.40:0/1693654305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48032810 con 0x7f5d540d06f0 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.011+0000 7f5d588f1640 1 -- 172.21.15.40:0/1693654305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 msgr2=0x7f5d540d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.011+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1693654305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d540d1e80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009870 tx=0x7f5d4802a270 comp rx=0 tx=0).stop 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.011+0000 7f5d588f1640 1 -- 172.21.15.40:0/1693654305 shutdown_connections 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.011+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1693654305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 0x7f5d540b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.011+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1693654305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d540d1e80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.011+0000 7f5d588f1640 1 -- 172.21.15.40:0/1693654305 >> 172.21.15.40:0/1693654305 conn(0x7f5d540d4f90 msgr2=0x7f5d540d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.011+0000 7f5d588f1640 1 -- 172.21.15.40:0/1693654305 shutdown_connections 2024-04-26T21:36:33.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.011+0000 7f5d588f1640 1 -- 172.21.15.40:0/1693654305 wait complete. 2024-04-26T21:36:33.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.012+0000 7f5d588f1640 1 Processor -- start 2024-04-26T21:36:33.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d588f1640 1 -- start start 2024-04-26T21:36:33.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d588f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 0x7f5d540cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d588f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d54158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d588f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5415f3a0 0x7f5d5409ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d588f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5415f900 con 0x7f5d540d06f0 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d588f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5415f780 con 0x7f5d5415f3a0 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d588f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d540d23c0 con 0x7f5d540b2850 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d51d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d54158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d51d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d54158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52868/0 (socket says 172.21.15.40:52868) 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.013+0000 7f5d51d9b640 1 -- 172.21.15.40:0/1180502059 learned_addr learned my addr 172.21.15.40:0/1180502059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d5259c640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 0x7f5d540cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d52d9d640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5415f3a0 0x7f5d5409ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d51d9b640 1 -- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 msgr2=0x7f5d540cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d51d9b640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 0x7f5d540cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d51d9b640 1 -- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5415f3a0 msgr2=0x7f5d5409ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d51d9b640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5415f3a0 0x7f5d5409ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d51d9b640 1 -- 172.21.15.40:0/1180502059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d540cd590 con 0x7f5d540d06f0 2024-04-26T21:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d51d9b640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d54158110 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d48000c00 tx=0x7f5d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d42ffd640 1 -- 172.21.15.40:0/1180502059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48041020 con 0x7f5d540d06f0 2024-04-26T21:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d42ffd640 1 -- 172.21.15.40:0/1180502059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d480306c0 con 0x7f5d540d06f0 2024-04-26T21:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d42ffd640 1 -- 172.21.15.40:0/1180502059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48037040 con 0x7f5d540d06f0 2024-04-26T21:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d540cd870 con 0x7f5d540d06f0 2024-04-26T21:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.014+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d540cde30 con 0x7f5d540d06f0 2024-04-26T21:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.015+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d540694f0 con 0x7f5d540d06f0 2024-04-26T21:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.016+0000 7f5d42ffd640 1 -- 172.21.15.40:0/1180502059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d48044020 con 0x7f5d540d06f0 2024-04-26T21:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.016+0000 7f5d42ffd640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3406c8f0 0x7f5d3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.016+0000 7f5d42ffd640 1 -- 172.21.15.40:0/1180502059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5d480ac4e0 con 0x7f5d540d06f0 2024-04-26T21:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.017+0000 7f5d5259c640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3406c8f0 0x7f5d3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.017+0000 7f5d5259c640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3406c8f0 0x7f5d3406edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5d44001d40 tx=0x7f5d44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.021+0000 7f5d42ffd640 1 -- 172.21.15.40:0/1180502059 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d4804b030 con 0x7f5d540d06f0 2024-04-26T21:36:33.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.203+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d540b6860 con 0x7f5d3406c8f0 2024-04-26T21:36:33.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.209+0000 7f5d42ffd640 1 -- 172.21.15.40:0/1180502059 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f5d540b6860 con 0x7f5d3406c8f0 2024-04-26T21:36:33.213 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:33.214 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:36:33.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3406c8f0 msgr2=0x7f5d3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3406c8f0 0x7f5d3406edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5d44001d40 tx=0x7f5d44009040 comp rx=0 tx=0).stop 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 msgr2=0x7f5d54158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d54158110 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d48000c00 tx=0x7f5d480046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 shutdown_connections 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3406c8f0 0x7f5d3406edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5415f3a0 0x7f5d5409ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d540d06f0 0x7f5d54158110 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 --2- 172.21.15.40:0/1180502059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d540b2850 0x7f5d540cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 >> 172.21.15.40:0/1180502059 conn(0x7f5d540d4f90 msgr2=0x7f5d5406cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 shutdown_connections 2024-04-26T21:36:33.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.212+0000 7f5d588f1640 1 -- 172.21.15.40:0/1180502059 wait complete. 2024-04-26T21:36:33.224 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-26T21:36:32.012631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220808,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753912,"statfs":{"total":773094113280,"available":772868005888,"internally_reserved":0,"allocated":4759552,"data_stored":2705844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001560"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685956+0000","last_change":"2024-04-26T21:36:24.434845+0000","last_active":"2024-04-26T21:36:25.685956+0000","last_peered":"2024-04-26T21:36:25.685956+0000","last_clean":"2024-04-26T21:36:25.685956+0000","last_became_active":"2024-04-26T21:36:24.434566+0000","last_became_peered":"2024-04-26T21:36:24.434566+0000","last_unstale":"2024-04-26T21:36:25.685956+0000","last_undegraded":"2024-04-26T21:36:25.685956+0000","last_fullsized":"2024-04-26T21:36:25.685956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:38.122152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685114+0000","last_change":"2024-04-26T21:36:24.436323+0000","last_active":"2024-04-26T21:36:25.685114+0000","last_peered":"2024-04-26T21:36:25.685114+0000","last_clean":"2024-04-26T21:36:25.685114+0000","last_became_active":"2024-04-26T21:36:24.435743+0000","last_became_peered":"2024-04-26T21:36:24.435743+0000","last_unstale":"2024-04-26T21:36:25.685114+0000","last_undegraded":"2024-04-26T21:36:25.685114+0000","last_fullsized":"2024-04-26T21:36:25.685114+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:07.741788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685536+0000","last_change":"2024-04-26T21:36:24.437649+0000","last_active":"2024-04-26T21:36:25.685536+0000","last_peered":"2024-04-26T21:36:25.685536+0000","last_clean":"2024-04-26T21:36:25.685536+0000","last_became_active":"2024-04-26T21:36:24.437388+0000","last_became_peered":"2024-04-26T21:36:24.437388+0000","last_unstale":"2024-04-26T21:36:25.685536+0000","last_undegraded":"2024-04-26T21:36:25.685536+0000","last_fullsized":"2024-04-26T21:36:25.685536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:49.485899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427904+0000","last_change":"2024-04-26T21:36:24.431270+0000","last_active":"2024-04-26T21:36:25.427904+0000","last_peered":"2024-04-26T21:36:25.427904+0000","last_clean":"2024-04-26T21:36:25.427904+0000","last_became_active":"2024-04-26T21:36:24.430997+0000","last_became_peered":"2024-04-26T21:36:24.430997+0000","last_unstale":"2024-04-26T21:36:25.427904+0000","last_undegraded":"2024-04-26T21:36:25.427904+0000","last_fullsized":"2024-04-26T21:36:25.427904+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:11.624210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427703+0000","last_change":"2024-04-26T21:36:24.430888+0000","last_active":"2024-04-26T21:36:25.427703+0000","last_peered":"2024-04-26T21:36:25.427703+0000","last_clean":"2024-04-26T21:36:25.427703+0000","last_became_active":"2024-04-26T21:36:24.430603+0000","last_became_peered":"2024-04-26T21:36:24.430603+0000","last_unstale":"2024-04-26T21:36:25.427703+0000","last_undegraded":"2024-04-26T21:36:25.427703+0000","last_fullsized":"2024-04-26T21:36:25.427703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:23.591473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685327+0000","last_change":"2024-04-26T21:36:24.440304+0000","last_active":"2024-04-26T21:36:25.685327+0000","last_peered":"2024-04-26T21:36:25.685327+0000","last_clean":"2024-04-26T21:36:25.685327+0000","last_became_active":"2024-04-26T21:36:24.440003+0000","last_became_peered":"2024-04-26T21:36:24.440003+0000","last_unstale":"2024-04-26T21:36:25.685327+0000","last_undegraded":"2024-04-26T21:36:25.685327+0000","last_fullsized":"2024-04-26T21:36:25.685327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:43.185154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444335+0000","last_change":"2024-04-26T21:36:24.437722+0000","last_active":"2024-04-26T21:36:25.444335+0000","last_peered":"2024-04-26T21:36:25.444335+0000","last_clean":"2024-04-26T21:36:25.444335+0000","last_became_active":"2024-04-26T21:36:24.437431+0000","last_became_peered":"2024-04-26T21:36:24.437431+0000","last_unstale":"2024-04-26T21:36:25.444335+0000","last_undegraded":"2024-04-26T21:36:25.444335+0000","last_fullsized":"2024-04-26T21:36:25.444335+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:07.741788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684844+0000","last_change":"2024-04-26T21:36:24.428758+0000","last_active":"2024-04-26T21:36:25.684844+0000","last_peered":"2024-04-26T21:36:25.684844+0000","last_clean":"2024-04-26T21:36:25.684844+0000","last_became_active":"2024-04-26T21:36:24.428471+0000","last_became_peered":"2024-04-26T21:36:24.428471+0000","last_unstale":"2024-04-26T21:36:25.684844+0000","last_undegraded":"2024-04-26T21:36:25.684844+0000","last_fullsized":"2024-04-26T21:36:25.684844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:06.749542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685024+0000","last_change":"2024-04-26T21:36:24.436963+0000","last_active":"2024-04-26T21:36:25.685024+0000","last_peered":"2024-04-26T21:36:25.685024+0000","last_clean":"2024-04-26T21:36:25.685024+0000","last_became_active":"2024-04-26T21:36:24.436694+0000","last_became_peered":"2024-04-26T21:36:24.436694+0000","last_unstale":"2024-04-26T21:36:25.685024+0000","last_undegraded":"2024-04-26T21:36:25.685024+0000","last_fullsized":"2024-04-26T21:36:25.685024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:15.880910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444666+0000","last_change":"2024-04-26T21:36:24.433321+0000","last_active":"2024-04-26T21:36:25.444666+0000","last_peered":"2024-04-26T21:36:25.444666+0000","last_clean":"2024-04-26T21:36:25.444666+0000","last_became_active":"2024-04-26T21:36:24.432726+0000","last_became_peered":"2024-04-26T21:36:24.432726+0000","last_unstale":"2024-04-26T21:36:25.444666+0000","last_undegraded":"2024-04-26T21:36:25.444666+0000","last_fullsized":"2024-04-26T21:36:25.444666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:21.528915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444617+0000","last_change":"2024-04-26T21:36:24.429485+0000","last_active":"2024-04-26T21:36:25.444617+0000","last_peered":"2024-04-26T21:36:25.444617+0000","last_clean":"2024-04-26T21:36:25.444617+0000","last_became_active":"2024-04-26T21:36:24.429219+0000","last_became_peered":"2024-04-26T21:36:24.429219+0000","last_unstale":"2024-04-26T21:36:25.444617+0000","last_undegraded":"2024-04-26T21:36:25.444617+0000","last_fullsized":"2024-04-26T21:36:25.444617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:16.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684988+0000","last_change":"2024-04-26T21:36:24.428855+0000","last_active":"2024-04-26T21:36:25.684988+0000","last_peered":"2024-04-26T21:36:25.684988+0000","last_clean":"2024-04-26T21:36:25.684988+0000","last_became_active":"2024-04-26T21:36:24.428559+0000","last_became_peered":"2024-04-26T21:36:24.428559+0000","last_unstale":"2024-04-26T21:36:25.684988+0000","last_undegraded":"2024-04-26T21:36:25.684988+0000","last_fullsized":"2024-04-26T21:36:25.684988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:19.505451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444822+0000","last_change":"2024-04-26T21:36:24.430051+0000","last_active":"2024-04-26T21:36:25.444822+0000","last_peered":"2024-04-26T21:36:25.444822+0000","last_clean":"2024-04-26T21:36:25.444822+0000","last_became_active":"2024-04-26T21:36:24.429797+0000","last_became_peered":"2024-04-26T21:36:24.429797+0000","last_unstale":"2024-04-26T21:36:25.444822+0000","last_undegraded":"2024-04-26T21:36:25.444822+0000","last_fullsized":"2024-04-26T21:36:25.444822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:55.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684493+0000","last_change":"2024-04-26T21:36:24.430742+0000","last_active":"2024-04-26T21:36:25.684493+0000","last_peered":"2024-04-26T21:36:25.684493+0000","last_clean":"2024-04-26T21:36:25.684493+0000","last_became_active":"2024-04-26T21:36:24.430463+0000","last_became_peered":"2024-04-26T21:36:24.430463+0000","last_unstale":"2024-04-26T21:36:25.684493+0000","last_undegraded":"2024-04-26T21:36:25.684493+0000","last_fullsized":"2024-04-26T21:36:25.684493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:18.694992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444493+0000","last_change":"2024-04-26T21:36:24.418987+0000","last_active":"2024-04-26T21:36:25.444493+0000","last_peered":"2024-04-26T21:36:25.444493+0000","last_clean":"2024-04-26T21:36:25.444493+0000","last_became_active":"2024-04-26T21:36:24.418686+0000","last_became_peered":"2024-04-26T21:36:24.418686+0000","last_unstale":"2024-04-26T21:36:25.444493+0000","last_undegraded":"2024-04-26T21:36:25.444493+0000","last_fullsized":"2024-04-26T21:36:25.444493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:54.904873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431227+0000","last_change":"2024-04-26T21:36:24.435428+0000","last_active":"2024-04-26T21:36:25.431227+0000","last_peered":"2024-04-26T21:36:25.431227+0000","last_clean":"2024-04-26T21:36:25.431227+0000","last_became_active":"2024-04-26T21:36:24.435106+0000","last_became_peered":"2024-04-26T21:36:24.435106+0000","last_unstale":"2024-04-26T21:36:25.431227+0000","last_undegraded":"2024-04-26T21:36:25.431227+0000","last_fullsized":"2024-04-26T21:36:25.431227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.787863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431767+0000","last_change":"2024-04-26T21:36:24.435324+0000","last_active":"2024-04-26T21:36:25.431767+0000","last_peered":"2024-04-26T21:36:25.431767+0000","last_clean":"2024-04-26T21:36:25.431767+0000","last_became_active":"2024-04-26T21:36:24.434939+0000","last_became_peered":"2024-04-26T21:36:24.434939+0000","last_unstale":"2024-04-26T21:36:25.431767+0000","last_undegraded":"2024-04-26T21:36:25.431767+0000","last_fullsized":"2024-04-26T21:36:25.431767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:28.911774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431462+0000","last_change":"2024-04-26T21:36:24.428969+0000","last_active":"2024-04-26T21:36:25.431462+0000","last_peered":"2024-04-26T21:36:25.431462+0000","last_clean":"2024-04-26T21:36:25.431462+0000","last_became_active":"2024-04-26T21:36:24.428696+0000","last_became_peered":"2024-04-26T21:36:24.428696+0000","last_unstale":"2024-04-26T21:36:25.431462+0000","last_undegraded":"2024-04-26T21:36:25.431462+0000","last_fullsized":"2024-04-26T21:36:25.431462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:24.234568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427843+0000","last_change":"2024-04-26T21:36:24.426239+0000","last_active":"2024-04-26T21:36:25.427843+0000","last_peered":"2024-04-26T21:36:25.427843+0000","last_clean":"2024-04-26T21:36:25.427843+0000","last_became_active":"2024-04-26T21:36:24.425949+0000","last_became_peered":"2024-04-26T21:36:24.425949+0000","last_unstale":"2024-04-26T21:36:25.427843+0000","last_undegraded":"2024-04-26T21:36:25.427843+0000","last_fullsized":"2024-04-26T21:36:25.427843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:53.448677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431108+0000","last_change":"2024-04-26T21:36:24.429567+0000","last_active":"2024-04-26T21:36:25.431108+0000","last_peered":"2024-04-26T21:36:25.431108+0000","last_clean":"2024-04-26T21:36:25.431108+0000","last_became_active":"2024-04-26T21:36:24.429316+0000","last_became_peered":"2024-04-26T21:36:24.429316+0000","last_unstale":"2024-04-26T21:36:25.431108+0000","last_undegraded":"2024-04-26T21:36:25.431108+0000","last_fullsized":"2024-04-26T21:36:25.431108+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:18.694992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444472+0000","last_change":"2024-04-26T21:36:24.425668+0000","last_active":"2024-04-26T21:36:25.444472+0000","last_peered":"2024-04-26T21:36:25.444472+0000","last_clean":"2024-04-26T21:36:25.444472+0000","last_became_active":"2024-04-26T21:36:24.425318+0000","last_became_peered":"2024-04-26T21:36:24.425318+0000","last_unstale":"2024-04-26T21:36:25.444472+0000","last_undegraded":"2024-04-26T21:36:25.444472+0000","last_fullsized":"2024-04-26T21:36:25.444472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:57.787231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684798+0000","last_change":"2024-04-26T21:36:24.428196+0000","last_active":"2024-04-26T21:36:25.684798+0000","last_peered":"2024-04-26T21:36:25.684798+0000","last_clean":"2024-04-26T21:36:25.684798+0000","last_became_active":"2024-04-26T21:36:24.427899+0000","last_became_peered":"2024-04-26T21:36:24.427899+0000","last_unstale":"2024-04-26T21:36:25.684798+0000","last_undegraded":"2024-04-26T21:36:25.684798+0000","last_fullsized":"2024-04-26T21:36:25.684798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:07.044785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.430769+0000","last_change":"2024-04-26T21:36:24.431681+0000","last_active":"2024-04-26T21:36:25.430769+0000","last_peered":"2024-04-26T21:36:25.430769+0000","last_clean":"2024-04-26T21:36:25.430769+0000","last_became_active":"2024-04-26T21:36:24.431174+0000","last_became_peered":"2024-04-26T21:36:24.431174+0000","last_unstale":"2024-04-26T21:36:25.430769+0000","last_undegraded":"2024-04-26T21:36:25.430769+0000","last_fullsized":"2024-04-26T21:36:25.430769+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:29.981886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427703+0000","last_change":"2024-04-26T21:36:24.429256+0000","last_active":"2024-04-26T21:36:25.427703+0000","last_peered":"2024-04-26T21:36:25.427703+0000","last_clean":"2024-04-26T21:36:25.427703+0000","last_became_active":"2024-04-26T21:36:24.428942+0000","last_became_peered":"2024-04-26T21:36:24.428942+0000","last_unstale":"2024-04-26T21:36:25.427703+0000","last_undegraded":"2024-04-26T21:36:25.427703+0000","last_fullsized":"2024-04-26T21:36:25.427703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:39:47.142157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431434+0000","last_change":"2024-04-26T21:36:24.426410+0000","last_active":"2024-04-26T21:36:25.431434+0000","last_peered":"2024-04-26T21:36:25.431434+0000","last_clean":"2024-04-26T21:36:25.431434+0000","last_became_active":"2024-04-26T21:36:24.426131+0000","last_became_peered":"2024-04-26T21:36:24.426131+0000","last_unstale":"2024-04-26T21:36:25.431434+0000","last_undegraded":"2024-04-26T21:36:25.431434+0000","last_fullsized":"2024-04-26T21:36:25.431434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:07.044785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685995+0000","last_change":"2024-04-26T21:36:24.429976+0000","last_active":"2024-04-26T21:36:25.685995+0000","last_peered":"2024-04-26T21:36:25.685995+0000","last_clean":"2024-04-26T21:36:25.685995+0000","last_became_active":"2024-04-26T21:36:24.429688+0000","last_became_peered":"2024-04-26T21:36:24.429688+0000","last_unstale":"2024-04-26T21:36:25.685995+0000","last_undegraded":"2024-04-26T21:36:25.685995+0000","last_fullsized":"2024-04-26T21:36:25.685995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:29.981886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431181+0000","last_change":"2024-04-26T21:36:24.418927+0000","last_active":"2024-04-26T21:36:25.431181+0000","last_peered":"2024-04-26T21:36:25.431181+0000","last_clean":"2024-04-26T21:36:25.431181+0000","last_became_active":"2024-04-26T21:36:24.418633+0000","last_became_peered":"2024-04-26T21:36:24.418633+0000","last_unstale":"2024-04-26T21:36:25.431181+0000","last_undegraded":"2024-04-26T21:36:25.431181+0000","last_fullsized":"2024-04-26T21:36:25.431181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:39.843633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444766+0000","last_change":"2024-04-26T21:36:24.436221+0000","last_active":"2024-04-26T21:36:25.444766+0000","last_peered":"2024-04-26T21:36:25.444766+0000","last_clean":"2024-04-26T21:36:25.444766+0000","last_became_active":"2024-04-26T21:36:24.435933+0000","last_became_peered":"2024-04-26T21:36:24.435933+0000","last_unstale":"2024-04-26T21:36:25.444766+0000","last_undegraded":"2024-04-26T21:36:25.444766+0000","last_fullsized":"2024-04-26T21:36:25.444766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:43.319588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.430940+0000","last_change":"2024-04-26T21:36:24.426279+0000","last_active":"2024-04-26T21:36:25.430940+0000","last_peered":"2024-04-26T21:36:25.430940+0000","last_clean":"2024-04-26T21:36:25.430940+0000","last_became_active":"2024-04-26T21:36:24.425999+0000","last_became_peered":"2024-04-26T21:36:24.425999+0000","last_unstale":"2024-04-26T21:36:25.430940+0000","last_undegraded":"2024-04-26T21:36:25.430940+0000","last_fullsized":"2024-04-26T21:36:25.430940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:43.012435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.430684+0000","last_change":"2024-04-26T21:36:24.441231+0000","last_active":"2024-04-26T21:36:25.430684+0000","last_peered":"2024-04-26T21:36:25.430684+0000","last_clean":"2024-04-26T21:36:25.430684+0000","last_became_active":"2024-04-26T21:36:24.441014+0000","last_became_peered":"2024-04-26T21:36:24.441014+0000","last_unstale":"2024-04-26T21:36:25.430684+0000","last_undegraded":"2024-04-26T21:36:25.430684+0000","last_fullsized":"2024-04-26T21:36:25.430684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:18.189154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.430923+0000","last_change":"2024-04-26T21:36:24.433831+0000","last_active":"2024-04-26T21:36:25.430923+0000","last_peered":"2024-04-26T21:36:25.430923+0000","last_clean":"2024-04-26T21:36:25.430923+0000","last_became_active":"2024-04-26T21:36:24.431459+0000","last_became_peered":"2024-04-26T21:36:24.431459+0000","last_unstale":"2024-04-26T21:36:25.430923+0000","last_undegraded":"2024-04-26T21:36:25.430923+0000","last_fullsized":"2024-04-26T21:36:25.430923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:49.485899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685627+0000","last_change":"2024-04-26T21:36:24.432410+0000","last_active":"2024-04-26T21:36:25.685627+0000","last_peered":"2024-04-26T21:36:25.685627+0000","last_clean":"2024-04-26T21:36:25.685627+0000","last_became_active":"2024-04-26T21:36:24.432127+0000","last_became_peered":"2024-04-26T21:36:24.432127+0000","last_unstale":"2024-04-26T21:36:25.685627+0000","last_undegraded":"2024-04-26T21:36:25.685627+0000","last_fullsized":"2024-04-26T21:36:25.685627+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:19.505451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431877+0000","last_change":"2024-04-26T21:36:24.431040+0000","last_active":"2024-04-26T21:36:25.431877+0000","last_peered":"2024-04-26T21:36:25.431877+0000","last_clean":"2024-04-26T21:36:25.431877+0000","last_became_active":"2024-04-26T21:36:24.430749+0000","last_became_peered":"2024-04-26T21:36:24.430749+0000","last_unstale":"2024-04-26T21:36:25.431877+0000","last_undegraded":"2024-04-26T21:36:25.431877+0000","last_fullsized":"2024-04-26T21:36:25.431877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:57.787231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431422+0000","last_change":"2024-04-26T21:36:23.422809+0000","last_active":"2024-04-26T21:36:25.431422+0000","last_peered":"2024-04-26T21:36:25.431422+0000","last_clean":"2024-04-26T21:36:25.431422+0000","last_became_active":"2024-04-26T21:36:23.422538+0000","last_became_peered":"2024-04-26T21:36:23.422538+0000","last_unstale":"2024-04-26T21:36:25.431422+0000","last_undegraded":"2024-04-26T21:36:25.431422+0000","last_fullsized":"2024-04-26T21:36:25.431422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:22.586948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685320+0000","last_change":"2024-04-26T21:36:24.430548+0000","last_active":"2024-04-26T21:36:25.685320+0000","last_peered":"2024-04-26T21:36:25.685320+0000","last_clean":"2024-04-26T21:36:25.685320+0000","last_became_active":"2024-04-26T21:36:24.430272+0000","last_became_peered":"2024-04-26T21:36:24.430272+0000","last_unstale":"2024-04-26T21:36:25.685320+0000","last_undegraded":"2024-04-26T21:36:25.685320+0000","last_fullsized":"2024-04-26T21:36:25.685320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:41.965004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685714+0000","last_change":"2024-04-26T21:36:23.424920+0000","last_active":"2024-04-26T21:36:25.685714+0000","last_peered":"2024-04-26T21:36:25.685714+0000","last_clean":"2024-04-26T21:36:25.685714+0000","last_became_active":"2024-04-26T21:36:23.424381+0000","last_became_peered":"2024-04-26T21:36:23.424381+0000","last_unstale":"2024-04-26T21:36:25.685714+0000","last_undegraded":"2024-04-26T21:36:25.685714+0000","last_fullsized":"2024-04-26T21:36:25.685714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:38.989187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684875+0000","last_change":"2024-04-26T21:36:24.431082+0000","last_active":"2024-04-26T21:36:25.684875+0000","last_peered":"2024-04-26T21:36:25.684875+0000","last_clean":"2024-04-26T21:36:25.684875+0000","last_became_active":"2024-04-26T21:36:24.430399+0000","last_became_peered":"2024-04-26T21:36:24.430399+0000","last_unstale":"2024-04-26T21:36:25.684875+0000","last_undegraded":"2024-04-26T21:36:25.684875+0000","last_fullsized":"2024-04-26T21:36:25.684875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.327014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685466+0000","last_change":"2024-04-26T21:36:23.416963+0000","last_active":"2024-04-26T21:36:25.685466+0000","last_peered":"2024-04-26T21:36:25.685466+0000","last_clean":"2024-04-26T21:36:25.685466+0000","last_became_active":"2024-04-26T21:36:23.416695+0000","last_became_peered":"2024-04-26T21:36:23.416695+0000","last_unstale":"2024-04-26T21:36:25.685466+0000","last_undegraded":"2024-04-26T21:36:25.685466+0000","last_fullsized":"2024-04-26T21:36:25.685466+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:34.917784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685815+0000","last_change":"2024-04-26T21:36:24.433508+0000","last_active":"2024-04-26T21:36:25.685815+0000","last_peered":"2024-04-26T21:36:25.685815+0000","last_clean":"2024-04-26T21:36:25.685815+0000","last_became_active":"2024-04-26T21:36:24.433207+0000","last_became_peered":"2024-04-26T21:36:24.433207+0000","last_unstale":"2024-04-26T21:36:25.685815+0000","last_undegraded":"2024-04-26T21:36:25.685815+0000","last_fullsized":"2024-04-26T21:36:25.685815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:46.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684809+0000","last_change":"2024-04-26T21:36:23.417750+0000","last_active":"2024-04-26T21:36:25.684809+0000","last_peered":"2024-04-26T21:36:25.684809+0000","last_clean":"2024-04-26T21:36:25.684809+0000","last_became_active":"2024-04-26T21:36:23.417106+0000","last_became_peered":"2024-04-26T21:36:23.417106+0000","last_unstale":"2024-04-26T21:36:25.684809+0000","last_undegraded":"2024-04-26T21:36:25.684809+0000","last_fullsized":"2024-04-26T21:36:25.684809+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:38.989187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684561+0000","last_change":"2024-04-26T21:36:24.434576+0000","last_active":"2024-04-26T21:36:25.684561+0000","last_peered":"2024-04-26T21:36:25.684561+0000","last_clean":"2024-04-26T21:36:25.684561+0000","last_became_active":"2024-04-26T21:36:24.434248+0000","last_became_peered":"2024-04-26T21:36:24.434248+0000","last_unstale":"2024-04-26T21:36:25.684561+0000","last_undegraded":"2024-04-26T21:36:25.684561+0000","last_fullsized":"2024-04-26T21:36:25.684561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.327014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444660+0000","last_change":"2024-04-26T21:36:23.425516+0000","last_active":"2024-04-26T21:36:25.444660+0000","last_peered":"2024-04-26T21:36:25.444660+0000","last_clean":"2024-04-26T21:36:25.444660+0000","last_became_active":"2024-04-26T21:36:23.425189+0000","last_became_peered":"2024-04-26T21:36:23.425189+0000","last_unstale":"2024-04-26T21:36:25.444660+0000","last_undegraded":"2024-04-26T21:36:25.444660+0000","last_fullsized":"2024-04-26T21:36:25.444660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:42.007910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444818+0000","last_change":"2024-04-26T21:36:24.426551+0000","last_active":"2024-04-26T21:36:25.444818+0000","last_peered":"2024-04-26T21:36:25.444818+0000","last_clean":"2024-04-26T21:36:25.444818+0000","last_became_active":"2024-04-26T21:36:24.425430+0000","last_became_peered":"2024-04-26T21:36:24.425430+0000","last_unstale":"2024-04-26T21:36:25.444818+0000","last_undegraded":"2024-04-26T21:36:25.444818+0000","last_fullsized":"2024-04-26T21:36:25.444818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:24:40.963343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431257+0000","last_change":"2024-04-26T21:36:23.418388+0000","last_active":"2024-04-26T21:36:25.431257+0000","last_peered":"2024-04-26T21:36:25.431257+0000","last_clean":"2024-04-26T21:36:25.431257+0000","last_became_active":"2024-04-26T21:36:23.418113+0000","last_became_peered":"2024-04-26T21:36:23.418113+0000","last_unstale":"2024-04-26T21:36:25.431257+0000","last_undegraded":"2024-04-26T21:36:25.431257+0000","last_fullsized":"2024-04-26T21:36:25.431257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:42:16.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684872+0000","last_change":"2024-04-26T21:36:24.428788+0000","last_active":"2024-04-26T21:36:25.684872+0000","last_peered":"2024-04-26T21:36:25.684872+0000","last_clean":"2024-04-26T21:36:25.684872+0000","last_became_active":"2024-04-26T21:36:24.428539+0000","last_became_peered":"2024-04-26T21:36:24.428539+0000","last_unstale":"2024-04-26T21:36:25.684872+0000","last_undegraded":"2024-04-26T21:36:25.684872+0000","last_fullsized":"2024-04-26T21:36:25.684872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:41.965004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685076+0000","last_change":"2024-04-26T21:36:23.423093+0000","last_active":"2024-04-26T21:36:25.685076+0000","last_peered":"2024-04-26T21:36:25.685076+0000","last_clean":"2024-04-26T21:36:25.685076+0000","last_became_active":"2024-04-26T21:36:23.422794+0000","last_became_peered":"2024-04-26T21:36:23.422794+0000","last_unstale":"2024-04-26T21:36:25.685076+0000","last_undegraded":"2024-04-26T21:36:25.685076+0000","last_fullsized":"2024-04-26T21:36:25.685076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:10.619685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685218+0000","last_change":"2024-04-26T21:36:24.438829+0000","last_active":"2024-04-26T21:36:25.685218+0000","last_peered":"2024-04-26T21:36:25.685218+0000","last_clean":"2024-04-26T21:36:25.685218+0000","last_became_active":"2024-04-26T21:36:24.438077+0000","last_became_peered":"2024-04-26T21:36:24.438077+0000","last_unstale":"2024-04-26T21:36:25.685218+0000","last_undegraded":"2024-04-26T21:36:25.685218+0000","last_fullsized":"2024-04-26T21:36:25.685218+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:28.911774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684346+0000","last_change":"2024-04-26T21:36:23.420011+0000","last_active":"2024-04-26T21:36:25.684346+0000","last_peered":"2024-04-26T21:36:25.684346+0000","last_clean":"2024-04-26T21:36:25.684346+0000","last_became_active":"2024-04-26T21:36:23.419718+0000","last_became_peered":"2024-04-26T21:36:23.419718+0000","last_unstale":"2024-04-26T21:36:25.684346+0000","last_undegraded":"2024-04-26T21:36:25.684346+0000","last_fullsized":"2024-04-26T21:36:25.684346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:39:46.137633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427975+0000","last_change":"2024-04-26T21:36:24.431379+0000","last_active":"2024-04-26T21:36:25.427975+0000","last_peered":"2024-04-26T21:36:25.427975+0000","last_clean":"2024-04-26T21:36:25.427975+0000","last_became_active":"2024-04-26T21:36:24.431080+0000","last_became_peered":"2024-04-26T21:36:24.431080+0000","last_unstale":"2024-04-26T21:36:25.427975+0000","last_undegraded":"2024-04-26T21:36:25.427975+0000","last_fullsized":"2024-04-26T21:36:25.427975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:42:17.671596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.686008+0000","last_change":"2024-04-26T21:36:23.418185+0000","last_active":"2024-04-26T21:36:25.686008+0000","last_peered":"2024-04-26T21:36:25.686008+0000","last_clean":"2024-04-26T21:36:25.686008+0000","last_became_active":"2024-04-26T21:36:23.417205+0000","last_became_peered":"2024-04-26T21:36:23.417205+0000","last_unstale":"2024-04-26T21:36:25.686008+0000","last_undegraded":"2024-04-26T21:36:25.686008+0000","last_fullsized":"2024-04-26T21:36:25.686008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:38.839108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431534+0000","last_change":"2024-04-26T21:36:24.432784+0000","last_active":"2024-04-26T21:36:25.431534+0000","last_peered":"2024-04-26T21:36:25.431534+0000","last_clean":"2024-04-26T21:36:25.431534+0000","last_became_active":"2024-04-26T21:36:24.432464+0000","last_became_peered":"2024-04-26T21:36:24.432464+0000","last_unstale":"2024-04-26T21:36:25.431534+0000","last_undegraded":"2024-04-26T21:36:25.431534+0000","last_fullsized":"2024-04-26T21:36:25.431534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:42.014161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444916+0000","last_change":"2024-04-26T21:36:23.421919+0000","last_active":"2024-04-26T21:36:25.444916+0000","last_peered":"2024-04-26T21:36:25.444916+0000","last_clean":"2024-04-26T21:36:25.444916+0000","last_became_active":"2024-04-26T21:36:23.421668+0000","last_became_peered":"2024-04-26T21:36:23.421668+0000","last_unstale":"2024-04-26T21:36:25.444916+0000","last_undegraded":"2024-04-26T21:36:25.444916+0000","last_fullsized":"2024-04-26T21:36:25.444916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.322489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.686044+0000","last_change":"2024-04-26T21:36:24.435188+0000","last_active":"2024-04-26T21:36:25.686044+0000","last_peered":"2024-04-26T21:36:25.686044+0000","last_clean":"2024-04-26T21:36:25.686044+0000","last_became_active":"2024-04-26T21:36:24.434904+0000","last_became_peered":"2024-04-26T21:36:24.434904+0000","last_unstale":"2024-04-26T21:36:25.686044+0000","last_undegraded":"2024-04-26T21:36:25.686044+0000","last_fullsized":"2024-04-26T21:36:25.686044+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:10.463555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444695+0000","last_change":"2024-04-26T21:36:23.423299+0000","last_active":"2024-04-26T21:36:25.444695+0000","last_peered":"2024-04-26T21:36:25.444695+0000","last_clean":"2024-04-26T21:36:25.444695+0000","last_became_active":"2024-04-26T21:36:23.422981+0000","last_became_peered":"2024-04-26T21:36:23.422981+0000","last_unstale":"2024-04-26T21:36:25.444695+0000","last_undegraded":"2024-04-26T21:36:25.444695+0000","last_fullsized":"2024-04-26T21:36:25.444695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:17.690467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684890+0000","last_change":"2024-04-26T21:36:24.429327+0000","last_active":"2024-04-26T21:36:25.684890+0000","last_peered":"2024-04-26T21:36:25.684890+0000","last_clean":"2024-04-26T21:36:25.684890+0000","last_became_active":"2024-04-26T21:36:24.429062+0000","last_became_peered":"2024-04-26T21:36:24.429062+0000","last_unstale":"2024-04-26T21:36:25.684890+0000","last_undegraded":"2024-04-26T21:36:25.684890+0000","last_fullsized":"2024-04-26T21:36:25.684890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:43.185154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685671+0000","last_change":"2024-04-26T21:36:23.417102+0000","last_active":"2024-04-26T21:36:25.685671+0000","last_peered":"2024-04-26T21:36:25.685671+0000","last_clean":"2024-04-26T21:36:25.685671+0000","last_became_active":"2024-04-26T21:36:23.416827+0000","last_became_peered":"2024-04-26T21:36:23.416827+0000","last_unstale":"2024-04-26T21:36:25.685671+0000","last_undegraded":"2024-04-26T21:36:25.685671+0000","last_fullsized":"2024-04-26T21:36:25.685671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:26.322489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684578+0000","last_change":"2024-04-26T21:36:24.436465+0000","last_active":"2024-04-26T21:36:25.684578+0000","last_peered":"2024-04-26T21:36:25.684578+0000","last_clean":"2024-04-26T21:36:25.684578+0000","last_became_active":"2024-04-26T21:36:24.435941+0000","last_became_peered":"2024-04-26T21:36:24.435941+0000","last_unstale":"2024-04-26T21:36:25.684578+0000","last_undegraded":"2024-04-26T21:36:25.684578+0000","last_fullsized":"2024-04-26T21:36:25.684578+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:41.965004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685625+0000","last_change":"2024-04-26T21:36:23.426752+0000","last_active":"2024-04-26T21:36:25.685625+0000","last_peered":"2024-04-26T21:36:25.685625+0000","last_clean":"2024-04-26T21:36:25.685625+0000","last_became_active":"2024-04-26T21:36:23.426414+0000","last_became_peered":"2024-04-26T21:36:23.426414+0000","last_unstale":"2024-04-26T21:36:25.685625+0000","last_undegraded":"2024-04-26T21:36:25.685625+0000","last_fullsized":"2024-04-26T21:36:25.685625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:06.040261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685518+0000","last_change":"2024-04-26T21:36:24.434195+0000","last_active":"2024-04-26T21:36:25.685518+0000","last_peered":"2024-04-26T21:36:25.685518+0000","last_clean":"2024-04-26T21:36:25.685518+0000","last_became_active":"2024-04-26T21:36:24.433917+0000","last_became_peered":"2024-04-26T21:36:24.433917+0000","last_unstale":"2024-04-26T21:36:25.685518+0000","last_undegraded":"2024-04-26T21:36:25.685518+0000","last_fullsized":"2024-04-26T21:36:25.685518+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:16.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685388+0000","last_change":"2024-04-26T21:36:23.423041+0000","last_active":"2024-04-26T21:36:25.685388+0000","last_peered":"2024-04-26T21:36:25.685388+0000","last_clean":"2024-04-26T21:36:25.685388+0000","last_became_active":"2024-04-26T21:36:23.422755+0000","last_became_peered":"2024-04-26T21:36:23.422755+0000","last_unstale":"2024-04-26T21:36:25.685388+0000","last_undegraded":"2024-04-26T21:36:25.685388+0000","last_fullsized":"2024-04-26T21:36:25.685388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:46.400051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431200+0000","last_change":"2024-04-26T21:36:24.430919+0000","last_active":"2024-04-26T21:36:25.431200+0000","last_peered":"2024-04-26T21:36:25.431200+0000","last_clean":"2024-04-26T21:36:25.431200+0000","last_became_active":"2024-04-26T21:36:24.430654+0000","last_became_peered":"2024-04-26T21:36:24.430654+0000","last_unstale":"2024-04-26T21:36:25.431200+0000","last_undegraded":"2024-04-26T21:36:25.431200+0000","last_fullsized":"2024-04-26T21:36:25.431200+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:10.463555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.430458+0000","last_change":"2024-04-26T21:36:23.422867+0000","last_active":"2024-04-26T21:36:25.430458+0000","last_peered":"2024-04-26T21:36:25.430458+0000","last_clean":"2024-04-26T21:36:25.430458+0000","last_became_active":"2024-04-26T21:36:23.422525+0000","last_became_peered":"2024-04-26T21:36:23.422525+0000","last_unstale":"2024-04-26T21:36:25.430458+0000","last_undegraded":"2024-04-26T21:36:25.430458+0000","last_fullsized":"2024-04-26T21:36:25.430458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:39:46.137633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685375+0000","last_change":"2024-04-26T21:36:24.439072+0000","last_active":"2024-04-26T21:36:25.685375+0000","last_peered":"2024-04-26T21:36:25.685375+0000","last_clean":"2024-04-26T21:36:25.685375+0000","last_became_active":"2024-04-26T21:36:24.438520+0000","last_became_peered":"2024-04-26T21:36:24.438520+0000","last_unstale":"2024-04-26T21:36:25.685375+0000","last_undegraded":"2024-04-26T21:36:25.685375+0000","last_fullsized":"2024-04-26T21:36:25.685375+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:54.904873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431298+0000","last_change":"2024-04-26T21:36:23.418915+0000","last_active":"2024-04-26T21:36:25.431298+0000","last_peered":"2024-04-26T21:36:25.431298+0000","last_clean":"2024-04-26T21:36:25.431298+0000","last_became_active":"2024-04-26T21:36:23.418620+0000","last_became_peered":"2024-04-26T21:36:23.418620+0000","last_unstale":"2024-04-26T21:36:25.431298+0000","last_undegraded":"2024-04-26T21:36:25.431298+0000","last_fullsized":"2024-04-26T21:36:25.431298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:34.917784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685531+0000","last_change":"2024-04-26T21:36:24.436927+0000","last_active":"2024-04-26T21:36:25.685531+0000","last_peered":"2024-04-26T21:36:25.685531+0000","last_clean":"2024-04-26T21:36:25.685531+0000","last_became_active":"2024-04-26T21:36:24.434850+0000","last_became_peered":"2024-04-26T21:36:24.434850+0000","last_unstale":"2024-04-26T21:36:25.685531+0000","last_undegraded":"2024-04-26T21:36:25.685531+0000","last_fullsized":"2024-04-26T21:36:25.685531+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:21.528915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444770+0000","last_change":"2024-04-26T21:36:23.421096+0000","last_active":"2024-04-26T21:36:25.444770+0000","last_peered":"2024-04-26T21:36:25.444770+0000","last_clean":"2024-04-26T21:36:25.444770+0000","last_became_active":"2024-04-26T21:36:23.420835+0000","last_became_peered":"2024-04-26T21:36:23.420835+0000","last_unstale":"2024-04-26T21:36:25.444770+0000","last_undegraded":"2024-04-26T21:36:25.444770+0000","last_fullsized":"2024-04-26T21:36:25.444770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:38.839108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444726+0000","last_change":"2024-04-26T21:36:24.427350+0000","last_active":"2024-04-26T21:36:25.444726+0000","last_peered":"2024-04-26T21:36:25.444726+0000","last_clean":"2024-04-26T21:36:25.444726+0000","last_became_active":"2024-04-26T21:36:24.426625+0000","last_became_peered":"2024-04-26T21:36:24.426625+0000","last_unstale":"2024-04-26T21:36:25.444726+0000","last_undegraded":"2024-04-26T21:36:25.444726+0000","last_fullsized":"2024-04-26T21:36:25.444726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:46.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684942+0000","last_change":"2024-04-26T21:36:23.421658+0000","last_active":"2024-04-26T21:36:25.684942+0000","last_peered":"2024-04-26T21:36:25.684942+0000","last_clean":"2024-04-26T21:36:25.684942+0000","last_became_active":"2024-04-26T21:36:23.421292+0000","last_became_peered":"2024-04-26T21:36:23.421292+0000","last_unstale":"2024-04-26T21:36:25.684942+0000","last_undegraded":"2024-04-26T21:36:25.684942+0000","last_fullsized":"2024-04-26T21:36:25.684942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:52.444153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684940+0000","last_change":"2024-04-26T21:36:24.431646+0000","last_active":"2024-04-26T21:36:25.684940+0000","last_peered":"2024-04-26T21:36:25.684940+0000","last_clean":"2024-04-26T21:36:25.684940+0000","last_became_active":"2024-04-26T21:36:24.431400+0000","last_became_peered":"2024-04-26T21:36:24.431400+0000","last_unstale":"2024-04-26T21:36:25.684940+0000","last_undegraded":"2024-04-26T21:36:25.684940+0000","last_fullsized":"2024-04-26T21:36:25.684940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:19.505451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427884+0000","last_change":"2024-04-26T21:36:23.415735+0000","last_active":"2024-04-26T21:36:25.427884+0000","last_peered":"2024-04-26T21:36:25.427884+0000","last_clean":"2024-04-26T21:36:25.427884+0000","last_became_active":"2024-04-26T21:36:23.415430+0000","last_became_peered":"2024-04-26T21:36:23.415430+0000","last_unstale":"2024-04-26T21:36:25.427884+0000","last_undegraded":"2024-04-26T21:36:25.427884+0000","last_fullsized":"2024-04-26T21:36:25.427884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:53.946556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427850+0000","last_change":"2024-04-26T21:36:24.440666+0000","last_active":"2024-04-26T21:36:25.427850+0000","last_peered":"2024-04-26T21:36:25.427850+0000","last_clean":"2024-04-26T21:36:25.427850+0000","last_became_active":"2024-04-26T21:36:24.440399+0000","last_became_peered":"2024-04-26T21:36:24.440399+0000","last_unstale":"2024-04-26T21:36:25.427850+0000","last_undegraded":"2024-04-26T21:36:25.427850+0000","last_fullsized":"2024-04-26T21:36:25.427850+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:39.993712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685847+0000","last_change":"2024-04-26T21:36:23.416337+0000","last_active":"2024-04-26T21:36:25.685847+0000","last_peered":"2024-04-26T21:36:25.685847+0000","last_clean":"2024-04-26T21:36:25.685847+0000","last_became_active":"2024-04-26T21:36:23.415906+0000","last_became_peered":"2024-04-26T21:36:23.415906+0000","last_unstale":"2024-04-26T21:36:25.685847+0000","last_undegraded":"2024-04-26T21:36:25.685847+0000","last_fullsized":"2024-04-26T21:36:25.685847+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:42:16.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431515+0000","last_change":"2024-04-26T21:36:24.431507+0000","last_active":"2024-04-26T21:36:25.431515+0000","last_peered":"2024-04-26T21:36:25.431515+0000","last_clean":"2024-04-26T21:36:25.431515+0000","last_became_active":"2024-04-26T21:36:24.431230+0000","last_became_peered":"2024-04-26T21:36:24.431230+0000","last_unstale":"2024-04-26T21:36:25.431515+0000","last_undegraded":"2024-04-26T21:36:25.431515+0000","last_fullsized":"2024-04-26T21:36:25.431515+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:17.835049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427759+0000","last_change":"2024-04-26T21:36:23.426782+0000","last_active":"2024-04-26T21:36:25.427759+0000","last_peered":"2024-04-26T21:36:25.427759+0000","last_clean":"2024-04-26T21:36:25.427759+0000","last_became_active":"2024-04-26T21:36:23.426489+0000","last_became_peered":"2024-04-26T21:36:23.426489+0000","last_unstale":"2024-04-26T21:36:25.427759+0000","last_undegraded":"2024-04-26T21:36:25.427759+0000","last_fullsized":"2024-04-26T21:36:25.427759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:16.978574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685290+0000","last_change":"2024-04-26T21:36:24.435420+0000","last_active":"2024-04-26T21:36:25.685290+0000","last_peered":"2024-04-26T21:36:25.685290+0000","last_clean":"2024-04-26T21:36:25.685290+0000","last_became_active":"2024-04-26T21:36:24.435040+0000","last_became_peered":"2024-04-26T21:36:24.435040+0000","last_unstale":"2024-04-26T21:36:25.685290+0000","last_undegraded":"2024-04-26T21:36:25.685290+0000","last_fullsized":"2024-04-26T21:36:25.685290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:29.981886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427711+0000","last_change":"2024-04-26T21:36:23.415177+0000","last_active":"2024-04-26T21:36:25.427711+0000","last_peered":"2024-04-26T21:36:25.427711+0000","last_clean":"2024-04-26T21:36:25.427711+0000","last_became_active":"2024-04-26T21:36:23.414879+0000","last_became_peered":"2024-04-26T21:36:23.414879+0000","last_unstale":"2024-04-26T21:36:25.427711+0000","last_undegraded":"2024-04-26T21:36:25.427711+0000","last_fullsized":"2024-04-26T21:36:25.427711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:52.376377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444448+0000","last_change":"2024-04-26T21:36:24.419660+0000","last_active":"2024-04-26T21:36:25.444448+0000","last_peered":"2024-04-26T21:36:25.444448+0000","last_clean":"2024-04-26T21:36:25.444448+0000","last_became_active":"2024-04-26T21:36:24.419393+0000","last_became_peered":"2024-04-26T21:36:24.419393+0000","last_unstale":"2024-04-26T21:36:25.444448+0000","last_undegraded":"2024-04-26T21:36:25.444448+0000","last_fullsized":"2024-04-26T21:36:25.444448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:28.911774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.430977+0000","last_change":"2024-04-26T21:36:23.418911+0000","last_active":"2024-04-26T21:36:25.430977+0000","last_peered":"2024-04-26T21:36:25.430977+0000","last_clean":"2024-04-26T21:36:25.430977+0000","last_became_active":"2024-04-26T21:36:23.418632+0000","last_became_peered":"2024-04-26T21:36:23.418632+0000","last_unstale":"2024-04-26T21:36:25.430977+0000","last_undegraded":"2024-04-26T21:36:25.430977+0000","last_fullsized":"2024-04-26T21:36:25.430977+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:46.400051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685568+0000","last_change":"2024-04-26T21:36:24.437600+0000","last_active":"2024-04-26T21:36:25.685568+0000","last_peered":"2024-04-26T21:36:25.685568+0000","last_clean":"2024-04-26T21:36:25.685568+0000","last_became_active":"2024-04-26T21:36:24.437294+0000","last_became_peered":"2024-04-26T21:36:24.437294+0000","last_unstale":"2024-04-26T21:36:25.685568+0000","last_undegraded":"2024-04-26T21:36:25.685568+0000","last_fullsized":"2024-04-26T21:36:25.685568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:55.095852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.427558+0000","last_change":"2024-04-26T21:36:23.426136+0000","last_active":"2024-04-26T21:36:25.427558+0000","last_peered":"2024-04-26T21:36:25.427558+0000","last_clean":"2024-04-26T21:36:25.427558+0000","last_became_active":"2024-04-26T21:36:23.425853+0000","last_became_peered":"2024-04-26T21:36:23.425853+0000","last_unstale":"2024-04-26T21:36:25.427558+0000","last_undegraded":"2024-04-26T21:36:25.427558+0000","last_fullsized":"2024-04-26T21:36:25.427558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:26.201947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685841+0000","last_change":"2024-04-26T21:36:24.430412+0000","last_active":"2024-04-26T21:36:25.685841+0000","last_peered":"2024-04-26T21:36:25.685841+0000","last_clean":"2024-04-26T21:36:25.685841+0000","last_became_active":"2024-04-26T21:36:24.430149+0000","last_became_peered":"2024-04-26T21:36:24.430149+0000","last_unstale":"2024-04-26T21:36:25.685841+0000","last_undegraded":"2024-04-26T21:36:25.685841+0000","last_fullsized":"2024-04-26T21:36:25.685841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:15.880910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444876+0000","last_change":"2024-04-26T21:36:23.423497+0000","last_active":"2024-04-26T21:36:25.444876+0000","last_peered":"2024-04-26T21:36:25.444876+0000","last_clean":"2024-04-26T21:36:25.444876+0000","last_became_active":"2024-04-26T21:36:23.423224+0000","last_became_peered":"2024-04-26T21:36:23.423224+0000","last_unstale":"2024-04-26T21:36:25.444876+0000","last_undegraded":"2024-04-26T21:36:25.444876+0000","last_fullsized":"2024-04-26T21:36:25.444876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:06.040261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431709+0000","last_change":"2024-04-26T21:36:24.433002+0000","last_active":"2024-04-26T21:36:25.431709+0000","last_peered":"2024-04-26T21:36:25.431709+0000","last_clean":"2024-04-26T21:36:25.431709+0000","last_became_active":"2024-04-26T21:36:24.431830+0000","last_became_peered":"2024-04-26T21:36:24.431830+0000","last_unstale":"2024-04-26T21:36:25.431709+0000","last_undegraded":"2024-04-26T21:36:25.431709+0000","last_fullsized":"2024-04-26T21:36:25.431709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:06.749542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.428009+0000","last_change":"2024-04-26T21:36:23.422061+0000","last_active":"2024-04-26T21:36:25.428009+0000","last_peered":"2024-04-26T21:36:25.428009+0000","last_clean":"2024-04-26T21:36:25.428009+0000","last_became_active":"2024-04-26T21:36:23.421751+0000","last_became_peered":"2024-04-26T21:36:23.421751+0000","last_unstale":"2024-04-26T21:36:25.428009+0000","last_undegraded":"2024-04-26T21:36:25.428009+0000","last_fullsized":"2024-04-26T21:36:25.428009+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:50.745099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685261+0000","last_change":"2024-04-26T21:36:24.436017+0000","last_active":"2024-04-26T21:36:25.685261+0000","last_peered":"2024-04-26T21:36:25.685261+0000","last_clean":"2024-04-26T21:36:25.685261+0000","last_became_active":"2024-04-26T21:36:24.435408+0000","last_became_peered":"2024-04-26T21:36:24.435408+0000","last_unstale":"2024-04-26T21:36:25.685261+0000","last_undegraded":"2024-04-26T21:36:25.685261+0000","last_fullsized":"2024-04-26T21:36:25.685261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:57.787231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685586+0000","last_change":"2024-04-26T21:36:23.420995+0000","last_active":"2024-04-26T21:36:25.685586+0000","last_peered":"2024-04-26T21:36:25.685586+0000","last_clean":"2024-04-26T21:36:25.685586+0000","last_became_active":"2024-04-26T21:36:23.420695+0000","last_became_peered":"2024-04-26T21:36:23.420695+0000","last_unstale":"2024-04-26T21:36:25.685586+0000","last_undegraded":"2024-04-26T21:36:25.685586+0000","last_fullsized":"2024-04-26T21:36:25.685586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:10.619685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684726+0000","last_change":"2024-04-26T21:36:24.441442+0000","last_active":"2024-04-26T21:36:25.684726+0000","last_peered":"2024-04-26T21:36:25.684726+0000","last_clean":"2024-04-26T21:36:25.684726+0000","last_became_active":"2024-04-26T21:36:24.441151+0000","last_became_peered":"2024-04-26T21:36:24.441151+0000","last_unstale":"2024-04-26T21:36:25.684726+0000","last_undegraded":"2024-04-26T21:36:25.684726+0000","last_fullsized":"2024-04-26T21:36:25.684726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:43.185154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685938+0000","last_change":"2024-04-26T21:36:23.427646+0000","last_active":"2024-04-26T21:36:25.685938+0000","last_peered":"2024-04-26T21:36:25.685938+0000","last_clean":"2024-04-26T21:36:25.685938+0000","last_became_active":"2024-04-26T21:36:23.426902+0000","last_became_peered":"2024-04-26T21:36:23.426902+0000","last_unstale":"2024-04-26T21:36:25.685938+0000","last_undegraded":"2024-04-26T21:36:25.685938+0000","last_fullsized":"2024-04-26T21:36:25.685938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:42:16.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.685008+0000","last_change":"2024-04-26T21:36:24.431604+0000","last_active":"2024-04-26T21:36:25.685008+0000","last_peered":"2024-04-26T21:36:25.685008+0000","last_clean":"2024-04-26T21:36:25.685008+0000","last_became_active":"2024-04-26T21:36:24.431314+0000","last_became_peered":"2024-04-26T21:36:24.431314+0000","last_unstale":"2024-04-26T21:36:25.685008+0000","last_undegraded":"2024-04-26T21:36:25.685008+0000","last_fullsized":"2024-04-26T21:36:25.685008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.327014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.430745+0000","last_change":"2024-04-26T21:36:23.424581+0000","last_active":"2024-04-26T21:36:25.430745+0000","last_peered":"2024-04-26T21:36:25.430745+0000","last_clean":"2024-04-26T21:36:25.430745+0000","last_became_active":"2024-04-26T21:36:23.424295+0000","last_became_peered":"2024-04-26T21:36:23.424295+0000","last_unstale":"2024-04-26T21:36:25.430745+0000","last_undegraded":"2024-04-26T21:36:25.430745+0000","last_fullsized":"2024-04-26T21:36:25.430745+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:10.128213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684639+0000","last_change":"2024-04-26T21:36:24.440651+0000","last_active":"2024-04-26T21:36:25.684639+0000","last_peered":"2024-04-26T21:36:25.684639+0000","last_clean":"2024-04-26T21:36:25.684639+0000","last_became_active":"2024-04-26T21:36:24.440381+0000","last_became_peered":"2024-04-26T21:36:24.440381+0000","last_unstale":"2024-04-26T21:36:25.684639+0000","last_undegraded":"2024-04-26T21:36:25.684639+0000","last_fullsized":"2024-04-26T21:36:25.684639+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:39.843633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684360+0000","last_change":"2024-04-26T21:36:23.417617+0000","last_active":"2024-04-26T21:36:25.684360+0000","last_peered":"2024-04-26T21:36:25.684360+0000","last_clean":"2024-04-26T21:36:25.684360+0000","last_became_active":"2024-04-26T21:36:23.417315+0000","last_became_peered":"2024-04-26T21:36:23.417315+0000","last_unstale":"2024-04-26T21:36:25.684360+0000","last_undegraded":"2024-04-26T21:36:25.684360+0000","last_fullsized":"2024-04-26T21:36:25.684360+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:22.586948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431765+0000","last_change":"2024-04-26T21:36:24.434115+0000","last_active":"2024-04-26T21:36:25.431765+0000","last_peered":"2024-04-26T21:36:25.431765+0000","last_clean":"2024-04-26T21:36:25.431765+0000","last_became_active":"2024-04-26T21:36:24.431953+0000","last_became_peered":"2024-04-26T21:36:24.431953+0000","last_unstale":"2024-04-26T21:36:25.431765+0000","last_undegraded":"2024-04-26T21:36:25.431765+0000","last_fullsized":"2024-04-26T21:36:25.431765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:07.741788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.684891+0000","last_change":"2024-04-26T21:36:23.417713+0000","last_active":"2024-04-26T21:36:25.684891+0000","last_peered":"2024-04-26T21:36:25.684891+0000","last_clean":"2024-04-26T21:36:25.684891+0000","last_became_active":"2024-04-26T21:36:23.417444+0000","last_became_peered":"2024-04-26T21:36:23.417444+0000","last_unstale":"2024-04-26T21:36:25.684891+0000","last_undegraded":"2024-04-26T21:36:25.684891+0000","last_fullsized":"2024-04-26T21:36:25.684891+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:17.690467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.444735+0000","last_change":"2024-04-26T21:36:24.438144+0000","last_active":"2024-04-26T21:36:25.444735+0000","last_peered":"2024-04-26T21:36:25.444735+0000","last_clean":"2024-04-26T21:36:25.444735+0000","last_became_active":"2024-04-26T21:36:24.437845+0000","last_became_peered":"2024-04-26T21:36:24.437845+0000","last_unstale":"2024-04-26T21:36:25.444735+0000","last_undegraded":"2024-04-26T21:36:25.444735+0000","last_fullsized":"2024-04-26T21:36:25.444735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:23.400474+0000","last_clean_scrub_stamp":"2024-04-26T21:36:23.400474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:38.122152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431751+0000","last_change":"2024-04-26T21:36:23.415550+0000","last_active":"2024-04-26T21:36:25.431751+0000","last_peered":"2024-04-26T21:36:25.431751+0000","last_clean":"2024-04-26T21:36:25.431751+0000","last_became_active":"2024-04-26T21:36:23.415251+0000","last_became_peered":"2024-04-26T21:36:23.415251+0000","last_unstale":"2024-04-26T21:36:25.431751+0000","last_undegraded":"2024-04-26T21:36:25.431751+0000","last_fullsized":"2024-04-26T21:36:25.431751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:36:22.395949+0000","last_clean_scrub_stamp":"2024-04-26T21:36:22.395949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:17.184629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T21:36:25.431544+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:36:25.431544+0000","last_peered":"2024-04-26T21:36:25.431544+0000","last_clean":"2024-04-26T21:36:25.431544+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:36:25.431544+0000","last_undegraded":"2024-04-26T21:36:25.431544+0000","last_fullsized":"2024-04-26T21:36:25.431544+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":782086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":191682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":780438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:36:33.230 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:36:33.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:36:33.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.371+0000 7f459c083640 1 Processor -- start 2024-04-26T21:36:33.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.372+0000 7f459c083640 1 -- start start 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.372+0000 7f459c083640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4594071260 0x7f459415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.372+0000 7f459c083640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45940d2390 con 0x7f4594070350 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.372+0000 7f459c083640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f459406d3f0 con 0x7f45940cfde0 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.372+0000 7f459c083640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f45940afab0 con 0x7f4594071260 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.373+0000 7f459a620640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4594071260 0x7f459415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.373+0000 7f459961e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4594070350 0x7f4594070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47932/0 (socket says 172.21.15.40:47932) 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.373+0000 7f459961e640 1 -- 172.21.15.40:0/76540890 learned_addr learned my addr 172.21.15.40:0/76540890 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.373+0000 7f459a620640 1 -- 172.21.15.40:0/76540890 >> v1:172.21.15.184:6789/0 conn(0x7f45940cfde0 legacy=0x7f45940d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.373+0000 7f459a620640 1 -- 172.21.15.40:0/76540890 >> v1:172.21.15.40:6789/0 conn(0x7f4594070350 legacy=0x7f4594070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.373+0000 7f459a620640 1 -- 172.21.15.40:0/76540890 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459406d6d0 con 0x7f4594071260 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f459a620640 1 --2- 172.21.15.40:0/76540890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594071260 0x7f459415e9f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f458c009d90 tx=0x7f458c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa14d9e0f59a1f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f4598e1d640 1 -- 172.21.15.40:0/76540890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c007740 con 0x7f4594071260 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f4598e1d640 1 -- 172.21.15.40:0/76540890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458c0078e0 con 0x7f4594071260 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f4598e1d640 1 -- 172.21.15.40:0/76540890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c007cb0 con 0x7f4594071260 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f459c083640 1 -- 172.21.15.40:0/76540890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594071260 msgr2=0x7f459415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f459c083640 1 --2- 172.21.15.40:0/76540890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594071260 0x7f459415e9f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f458c009d90 tx=0x7f458c02a660 comp rx=0 tx=0).stop 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f459c083640 1 -- 172.21.15.40:0/76540890 shutdown_connections 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f459c083640 1 --2- 172.21.15.40:0/76540890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594071260 0x7f459415e9f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f459c083640 1 -- 172.21.15.40:0/76540890 >> 172.21.15.40:0/76540890 conn(0x7f45940d2d80 msgr2=0x7f45940d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f459c083640 1 -- 172.21.15.40:0/76540890 shutdown_connections 2024-04-26T21:36:33.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.374+0000 7f459c083640 1 -- 172.21.15.40:0/76540890 wait complete. 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f459c083640 1 Processor -- start 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f459c083640 1 -- start start 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f459c083640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594070350 0x7f4594158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f459c083640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4594071260 0x7f459415e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f459c083640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45940cfde0 0x7f459409d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f459c083640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f459406f1c0 con 0x7f4594070350 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f459c083640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f459406dd20 con 0x7f45940cfde0 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f459c083640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f459406f340 con 0x7f4594071260 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f4599e1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594070350 0x7f4594158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f4599e1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594070350 0x7f4594158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52886/0 (socket says 172.21.15.40:52886) 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.376+0000 7f4599e1f640 1 -- 172.21.15.40:0/2874050287 learned_addr learned my addr 172.21.15.40:0/2874050287 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f459961e640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4594071260 0x7f459415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f459a620640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45940cfde0 0x7f459409d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f4599e1f640 1 -- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4594071260 msgr2=0x7f459415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f4599e1f640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4594071260 0x7f459415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f4599e1f640 1 -- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45940cfde0 msgr2=0x7f459409d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f4599e1f640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45940cfde0 0x7f459409d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f4599e1f640 1 -- 172.21.15.40:0/2874050287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459409ccf0 con 0x7f4594070350 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f459a620640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45940cfde0 0x7f459409d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f4599e1f640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594070350 0x7f4594158520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f459000c400 tx=0x7f459000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f458a7fc640 1 -- 172.21.15.40:0/2874050287 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45900052d0 con 0x7f4594070350 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f458a7fc640 1 -- 172.21.15.40:0/2874050287 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4590005490 con 0x7f4594070350 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f458a7fc640 1 -- 172.21.15.40:0/2874050287 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590007260 con 0x7f4594070350 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45940a09e0 con 0x7f4594070350 2024-04-26T21:36:33.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.377+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45940a0420 con 0x7f4594070350 2024-04-26T21:36:33.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.378+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45940694f0 con 0x7f4594070350 2024-04-26T21:36:33.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.379+0000 7f458a7fc640 1 -- 172.21.15.40:0/2874050287 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4590007400 con 0x7f4594070350 2024-04-26T21:36:33.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.379+0000 7f458a7fc640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.379+0000 7f458a7fc640 1 -- 172.21.15.40:0/2874050287 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f459008b130 con 0x7f4594070350 2024-04-26T21:36:33.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.379+0000 7f459961e640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.380+0000 7f459961e640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f459415be10 tx=0x7f4584009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.384+0000 7f458a7fc640 1 -- 172.21.15.40:0/2874050287 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4590058e10 con 0x7f4594070350 2024-04-26T21:36:33.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.640+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f459409cec0 con 0x7f4594070350 2024-04-26T21:36:33.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.642+0000 7f458a7fc640 1 -- 172.21.15.40:0/2874050287 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f459005d110 con 0x7f4594070350 2024-04-26T21:36:33.644 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:36:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw== 2024-04-26T21:36:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:36:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:36:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:36:33.644 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:36:33.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 msgr2=0x7f457806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f459415be10 tx=0x7f4584009040 comp rx=0 tx=0).stop 2024-04-26T21:36:33.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594070350 msgr2=0x7f4594158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594070350 0x7f4594158520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f459000c400 tx=0x7f459000c8c0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 shutdown_connections 2024-04-26T21:36:33.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f457806c8f0 0x7f457806edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45940cfde0 0x7f459409d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4594071260 0x7f459415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 --2- 172.21.15.40:0/2874050287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4594070350 0x7f4594158520 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 >> 172.21.15.40:0/2874050287 conn(0x7f45940d2d80 msgr2=0x7f45940b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:33.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 shutdown_connections 2024-04-26T21:36:33.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.645+0000 7f459c083640 1 -- 172.21.15.40:0/2874050287 wait complete. 2024-04-26T21:36:33.657 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.756+0000 7f908972c640 1 Processor -- start 2024-04-26T21:36:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.756+0000 7f908972c640 1 -- start start 2024-04-26T21:36:33.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f908972c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9084162880 0x7f90840d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f908972c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90840b7e50 con 0x7f90840d3bc0 2024-04-26T21:36:33.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f908972c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90840d1800 con 0x7f90840b5f20 2024-04-26T21:36:33.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f908972c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90840d0580 con 0x7f9084162880 2024-04-26T21:36:33.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f90837fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f90840d3bc0 0x7f90840b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47940/0 (socket says 172.21.15.40:47940) 2024-04-26T21:36:33.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f90837fe640 1 -- 172.21.15.40:0/2851997093 learned_addr learned my addr 172.21.15.40:0/2851997093 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:33.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f9082ffd640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9084162880 0x7f90840d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606309817 0 0) 0x7f90840b7e50 con 0x7f90840d3bc0 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9070003520 con 0x7f90840d3bc0 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.757+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2558497541 0 0) 0x7f9070003520 con 0x7f90840d3bc0 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9084162880 msgr2=0x7f90840d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9084162880 0x7f90840d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 >> v1:172.21.15.184:6789/0 conn(0x7f90840b5f20 legacy=0x7f90840d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90840d07d0 con 0x7f90840d3bc0 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f90740019f0 con 0x7f90840d3bc0 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 >> v1:172.21.15.40:6789/0 conn(0x7f90840d3bc0 legacy=0x7f90840b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9084162880 0x7f90700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070003f20 0x7f9070004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070004d40 0x7f90700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90840d0580 con 0x7f9070004d40 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f907000ba20 con 0x7f9070003f20 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f907000bc20 con 0x7f9084162880 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f90837fe640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070004d40 0x7f90700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f9082ffd640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9084162880 0x7f90700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f90827fc640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070003f20 0x7f9070004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f90837fe640 1 -- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9084162880 msgr2=0x7f90700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f90837fe640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9084162880 0x7f90700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f90837fe640 1 -- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070003f20 msgr2=0x7f9070004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f90837fe640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070003f20 0x7f9070004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.758+0000 7f90837fe640 1 -- 172.21.15.40:0/2851997093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9070003520 con 0x7f9070004d40 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f90837fe640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070004d40 0x7f90700051c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90840b6930 tx=0x7f9070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7304fcb24cc22153 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907400a4a0 con 0x7f9070004d40 2024-04-26T21:36:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f907402f070 con 0x7f9070004d40 2024-04-26T21:36:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f9081ffb640 1 -- 172.21.15.40:0/2851997093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907402bcb0 con 0x7f9070004d40 2024-04-26T21:36:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f908972c640 1 -- 172.21.15.40:0/2851997093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070004d40 msgr2=0x7f90700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f908972c640 1 --2- 172.21.15.40:0/2851997093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070004d40 0x7f90700051c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90840b6930 tx=0x7f9070005830 comp rx=0 tx=0).stop 2024-04-26T21:36:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f90837fe640 1 -- 172.21.15.40:0/2851997093 reap_dead start 2024-04-26T21:36:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f908972c640 1 -- 172.21.15.40:0/2851997093 shutdown_connections 2024-04-26T21:36:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.759+0000 7f908972c640 1 -- 172.21.15.40:0/2851997093 >> 172.21.15.40:0/2851997093 conn(0x7f90840d4f60 msgr2=0x7f90840d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.760+0000 7f908972c640 1 -- 172.21.15.40:0/2851997093 shutdown_connections 2024-04-26T21:36:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.760+0000 7f908972c640 1 -- 172.21.15.40:0/2851997093 wait complete. 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.761+0000 7f908972c640 1 Processor -- start 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.761+0000 7f908972c640 1 -- start start 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f908972c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070004d40 0x7f90840ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f908972c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90840b2020 0x7f908415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f908972c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90840ba010 0x7f90840ba490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f908972c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9084162c60 con 0x7f90840b2020 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f908972c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90840d0d90 con 0x7f9070004d40 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f908972c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9084162de0 con 0x7f90840ba010 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90840b2020 0x7f908415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90840b2020 0x7f908415ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52898/0 (socket says 172.21.15.40:52898) 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 -- 172.21.15.40:0/1668785141 learned_addr learned my addr 172.21.15.40:0/1668785141 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90837fe640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90840ba010 0x7f90840ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f9082ffd640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070004d40 0x7f90840ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 -- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90840ba010 msgr2=0x7f90840ba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90840ba010 0x7f90840ba490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 -- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070004d40 msgr2=0x7f90840ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070004d40 0x7f90840ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 -- 172.21.15.40:0/1668785141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908415cb80 con 0x7f90840b2020 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f9082ffd640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070004d40 0x7f90840ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:33.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90837fe640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90840ba010 0x7f90840ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f90827fc640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90840b2020 0x7f908415ada0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e830 tx=0x7f906c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f907b7fe640 1 -- 172.21.15.40:0/1668785141 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c0050d0 con 0x7f90840b2020 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f907b7fe640 1 -- 172.21.15.40:0/1668785141 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906c018070 con 0x7f90840b2020 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90840b2570 con 0x7f90840b2020 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.762+0000 7f907b7fe640 1 -- 172.21.15.40:0/1668785141 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00c6c0 con 0x7f90840b2020 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.763+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90840b2b30 con 0x7f90840b2020 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.763+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90840694f0 con 0x7f90840b2020 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.764+0000 7f907b7fe640 1 -- 172.21.15.40:0/1668785141 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f906c00ca70 con 0x7f90840b2020 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.765+0000 7f907b7fe640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906406c8f0 0x7f906406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.765+0000 7f907b7fe640 1 -- 172.21.15.40:0/1668785141 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f906c08bd80 con 0x7f90840b2020 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.765+0000 7f9082ffd640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906406c8f0 0x7f906406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.765+0000 7f9082ffd640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906406c8f0 0x7f906406edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9084069fe0 tx=0x7f907c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.769+0000 7f907b7fe640 1 -- 172.21.15.40:0/1668785141 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90840d1800 con 0x7f90840b2020 2024-04-26T21:36:33.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.992+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90840b1e00 con 0x7f90840b2020 2024-04-26T21:36:33.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.993+0000 7f907b7fe640 1 -- 172.21.15.40:0/1668785141 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f906c01a310 con 0x7f90840b2020 2024-04-26T21:36:33.994 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:33.994 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:26.796847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3403660773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3403660773},{"type":"v1","addr":"172.21.15.40:6835","nonce":3403660773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5364,"qdb_cluster":[5364]},"id":2}]} 2024-04-26T21:36:33.994 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906406c8f0 msgr2=0x7f906406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906406c8f0 0x7f906406edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9084069fe0 tx=0x7f907c008040 comp rx=0 tx=0).stop 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90840b2020 msgr2=0x7f908415ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90840b2020 0x7f908415ada0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e830 tx=0x7f906c00ecf0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 shutdown_connections 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f906406c8f0 0x7f906406edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90840ba010 0x7f90840ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90840b2020 0x7f908415ada0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 --2- 172.21.15.40:0/1668785141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9070004d40 0x7f90840ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:33.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 >> 172.21.15.40:0/1668785141 conn(0x7f90840d4f60 msgr2=0x7f90840d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:33.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 shutdown_connections 2024-04-26T21:36:33.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:33.995+0000 7f908972c640 1 -- 172.21.15.40:0/1668785141 wait complete. 2024-04-26T21:36:34.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:36:23.780134+0000', 'modified': '2024-04-26T21:36:26.796847+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5364}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5364': {'gid': 5364, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/3403660773', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 3403660773}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3403660773}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5364, 'qdb_cluster': [5364]} 2024-04-26T21:36:34.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:36:34.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:36:34.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.516+0000 7fdc503e4640 1 Processor -- start 2024-04-26T21:36:34.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.516+0000 7fdc503e4640 1 -- start start 2024-04-26T21:36:34.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc503e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc48162810 0x7fdc48070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc503e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc480b2340 0x7fdc480cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc503e4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc480d10d0 con 0x7fdc480cd930 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc503e4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdc480d0750 con 0x7fdc480b2340 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc503e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdc480d16a0 con 0x7fdc48162810 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc4d97f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc480b2340 0x7fdc480cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc4d97f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc480b2340 0x7fdc480cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52906/0 (socket says 172.21.15.40:52906) 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc4d97f640 1 -- 172.21.15.40:0/3349771939 learned_addr learned my addr 172.21.15.40:0/3349771939 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc4e180640 1 --2- 172.21.15.40:0/3349771939 >> v2:172.21.15.184:3300/0 conn(0x7fdc48162810 0x7fdc48070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc4d97f640 1 -- 172.21.15.40:0/3349771939 >> v2:172.21.15.184:3300/0 conn(0x7fdc48162810 msgr2=0x7fdc48070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/3349771939 >> v2:172.21.15.184:3300/0 conn(0x7fdc48162810 0x7fdc48070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc4d97f640 1 -- 172.21.15.40:0/3349771939 >> v1:172.21.15.184:6789/0 conn(0x7fdc480cd930 legacy=0x7fdc480d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.517+0000 7fdc4d97f640 1 -- 172.21.15.40:0/3349771939 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc480d1370 con 0x7fdc480b2340 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.518+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/3349771939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc480b2340 0x7fdc480cd3f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc48073d90 tx=0x7fdc3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f097fd4d560f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.518+0000 7fdc4d17e640 1 -- 172.21.15.40:0/3349771939 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38037070 con 0x7fdc480b2340 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.518+0000 7fdc4d17e640 1 -- 172.21.15.40:0/3349771939 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc38032440 con 0x7fdc480b2340 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.518+0000 7fdc4d17e640 1 -- 172.21.15.40:0/3349771939 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38032810 con 0x7fdc480b2340 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.519+0000 7fdc503e4640 1 -- 172.21.15.40:0/3349771939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc480b2340 msgr2=0x7fdc480cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.519+0000 7fdc503e4640 1 --2- 172.21.15.40:0/3349771939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc480b2340 0x7fdc480cd3f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc48073d90 tx=0x7fdc3802a270 comp rx=0 tx=0).stop 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.519+0000 7fdc503e4640 1 -- 172.21.15.40:0/3349771939 shutdown_connections 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.519+0000 7fdc503e4640 1 --2- 172.21.15.40:0/3349771939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc480b2340 0x7fdc480cd3f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.519+0000 7fdc503e4640 1 --2- 172.21.15.40:0/3349771939 >> v2:172.21.15.184:3300/0 conn(0x7fdc48162810 0x7fdc48070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.519+0000 7fdc503e4640 1 -- 172.21.15.40:0/3349771939 >> 172.21.15.40:0/3349771939 conn(0x7fdc480d2da0 msgr2=0x7fdc480d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.519+0000 7fdc503e4640 1 -- 172.21.15.40:0/3349771939 shutdown_connections 2024-04-26T21:36:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.519+0000 7fdc503e4640 1 -- 172.21.15.40:0/3349771939 wait complete. 2024-04-26T21:36:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.520+0000 7fdc503e4640 1 Processor -- start 2024-04-26T21:36:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.520+0000 7fdc503e4640 1 -- start start 2024-04-26T21:36:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc503e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480b2340 0x7fdc48158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc503e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480cd930 0x7fdc481f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc503e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc48162810 0x7fdc480cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc503e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48070890 con 0x7fdc48162810 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc503e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc480d0470 con 0x7fdc480b2340 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc503e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc480d2240 con 0x7fdc480cd930 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc48162810 0x7fdc480cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc48162810 0x7fdc480cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52910/0 (socket says 172.21.15.40:52910) 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 -- 172.21.15.40:0/1962577484 learned_addr learned my addr 172.21.15.40:0/1962577484 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e180640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480b2340 0x7fdc48158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480cd930 0x7fdc481f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 -- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480cd930 msgr2=0x7fdc481f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480cd930 0x7fdc481f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 -- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480b2340 msgr2=0x7fdc48158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480b2340 0x7fdc48158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 -- 172.21.15.40:0/1962577484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc481f5010 con 0x7fdc48162810 2024-04-26T21:36:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e180640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480b2340 0x7fdc48158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480cd930 0x7fdc481f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc4e981640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc48162810 0x7fdc480cb3a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c760 tx=0x7fdc4000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc3effd640 1 -- 172.21.15.40:0/1962577484 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40003b70 con 0x7fdc48162810 2024-04-26T21:36:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc3effd640 1 -- 172.21.15.40:0/1962577484 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40003d10 con 0x7fdc48162810 2024-04-26T21:36:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc503e4640 1 -- 172.21.15.40:0/1962577484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc481f8d80 con 0x7fdc48162810 2024-04-26T21:36:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc3effd640 1 -- 172.21.15.40:0/1962577484 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000aa90 con 0x7fdc48162810 2024-04-26T21:36:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.521+0000 7fdc503e4640 1 -- 172.21.15.40:0/1962577484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc481f9340 con 0x7fdc48162810 2024-04-26T21:36:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.523+0000 7fdc3effd640 1 -- 172.21.15.40:0/1962577484 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc4000ac30 con 0x7fdc48162810 2024-04-26T21:36:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.523+0000 7fdc3effd640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc2c06c8f0 0x7fdc2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.523+0000 7fdc3effd640 1 -- 172.21.15.40:0/1962577484 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fdc4008bbe0 con 0x7fdc48162810 2024-04-26T21:36:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.524+0000 7fdc4e180640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc2c06c8f0 0x7fdc2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.524+0000 7fdc4e180640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc2c06c8f0 0x7fdc2c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdc480d29a0 tx=0x7fdc44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.524+0000 7fdc531d3740 1 Processor -- start 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.524+0000 7fdc531d3740 1 -- start start 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03be440 0x558aa03be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03dbce0 0x558aa03dc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa0394120 0x558aa03da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc531d3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558aa02f8740 con 0x558aa03dbce0 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc531d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558aa02943a0 con 0x558aa0394120 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc531d3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558aa02f4700 con 0x558aa03be440 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc4d97f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03dbce0 0x558aa03dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc4d97f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03dbce0 0x558aa03dc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52926/0 (socket says 172.21.15.40:52926) 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc4d97f640 1 -- 172.21.15.40:0/2773153712 learned_addr learned my addr 172.21.15.40:0/2773153712 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc4e981640 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa0394120 0x558aa03da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.525+0000 7fdc4e180640 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03be440 0x558aa03be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc4d97f640 1 -- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03be440 msgr2=0x558aa03be820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03be440 0x558aa03be820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc4d97f640 1 -- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa0394120 msgr2=0x558aa03da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa0394120 0x558aa03da3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc4d97f640 1 -- 172.21.15.40:0/2773153712 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa02b0750 con 0x558aa03dbce0 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc4e981640 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa0394120 0x558aa03da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03dbce0 0x558aa03dc230 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdc3802a2b0 tx=0x7fdc3802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d0b7ea1475a168 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc2bfff640 1 -- 172.21.15.40:0/2773153712 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38037040 con 0x558aa03dbce0 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc2bfff640 1 -- 172.21.15.40:0/2773153712 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc38032880 con 0x558aa03dbce0 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.526+0000 7fdc2bfff640 1 -- 172.21.15.40:0/2773153712 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38032c70 con 0x558aa03dbce0 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 -- 172.21.15.40:0/2773153712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03dbce0 msgr2=0x558aa03dc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03dbce0 0x558aa03dc230 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdc3802a2b0 tx=0x7fdc3802c000 comp rx=0 tx=0).stop 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 -- 172.21.15.40:0/2773153712 shutdown_connections 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa0394120 0x558aa03da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03dbce0 0x558aa03dc230 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 --2- 172.21.15.40:0/2773153712 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03be440 0x558aa03be820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 -- 172.21.15.40:0/2773153712 >> 172.21.15.40:0/2773153712 conn(0x558aa0390a50 msgr2=0x558aa03d9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 -- 172.21.15.40:0/2773153712 shutdown_connections 2024-04-26T21:36:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 -- 172.21.15.40:0/2773153712 wait complete. 2024-04-26T21:36:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 -- start start 2024-04-26T21:36:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.527+0000 7fdc531d3740 1 Processor -- start 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa03acc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc531d3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558aa0394ec0 con 0x558aa03be440 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc531d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558aa027ffe0 con 0x558aa03aff00 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc531d3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558aa038e980 con 0x558aa03dbce0 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc4e180640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc531d3740 10 client.0 _ll_register_callbacks cb 0x558aa0296150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc4e180640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52928/0 (socket says 172.21.15.40:52928) 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.528+0000 7fdc4e180640 1 -- 172.21.15.40:0/1590112509 learned_addr learned my addr 172.21.15.40:0/1590112509 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa03acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc4e981640 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc4e180640 1 -- 172.21.15.40:0/1590112509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 msgr2=0x558aa03acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc4e180640 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa03acc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc4e180640 1 -- 172.21.15.40:0/1590112509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 msgr2=0x558aa03d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc4e180640 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc4e180640 1 -- 172.21.15.40:0/1590112509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa000a2d0 con 0x558aa03be440 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc4e180640 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03ad3a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc44014770 tx=0x7fdc44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b6e1e07e73813e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc297fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc297fa640 1 -- 172.21.15.40:0/1590112509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44025020 con 0x558aa03be440 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc297fa640 1 -- 172.21.15.40:0/1590112509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fdc440158e0 con 0x558aa03be440 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc531d3740 1 -- 172.21.15.40:0/1590112509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc38032880 con 0x558aa03be440 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc297fa640 1 -- 172.21.15.40:0/1590112509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdc4402f050 con 0x558aa03be440 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc531d3740 10 client.5550 fetch_fsmap learned FSMap version 17 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc531d3740 1 -- 172.21.15.40:0/1590112509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x558aa00d46b0 con 0x558aa03be440 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.529+0000 7fdc297fa640 1 -- 172.21.15.40:0/1590112509 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc38032880 con 0x558aa03be440 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc297fa640 1 -- 172.21.15.40:0/1590112509 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fdc44032020 con 0x558aa03be440 2024-04-26T21:36:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc531d3740 10 client.5550 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc531d3740 10 client.5550 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc531d3740 20 client.5550 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc531d3740 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] conn(0x558aa03c0650 0x558aa03c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc531d3740 4 client.5550 mds_command: new command op to 5364 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc531d3740 1 -- 172.21.15.40:0/1590112509 --> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aa0306350 con 0x558aa03c0650 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] conn(0x558aa03c0650 0x558aa03c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc297fa640 10 client.5550 ms_handle_connect on v2:172.21.15.40:6833/3403660773 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.530+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] conn(0x558aa03c0650 0x558aa03c0a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b17b7bd37e2dbc9 server_cookie=21f6cc1f9bcbc5be in_seq=0 out_seq=0 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc297fa640 1 -- 172.21.15.40:0/1590112509 <== mds.0 v2:172.21.15.40:6833/3403660773 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558aa0306350 con 0x558aa03c0650 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc297fa640 10 client.5550 handle_command_reply: tid=0 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 client.5550 shutdown 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 -- 172.21.15.40:0/1590112509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 msgr2=0x558aa03ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03ad3a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc44014770 tx=0x7fdc44004730 comp rx=0 tx=0).stop 2024-04-26T21:36:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 -- 172.21.15.40:0/1590112509 shutdown_connections 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] conn(0x558aa03c0650 0x558aa03c0a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa03acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 --2- 172.21.15.40:0/1590112509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03ad3a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc297fa640 0 client.5550 ms_handle_reset on v2:172.21.15.40:6833/3403660773 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 -- 172.21.15.40:0/1590112509 >> 172.21.15.40:0/1590112509 conn(0x558aa0394120 msgr2=0x558aa03bd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 -- 172.21.15.40:0/1590112509 shutdown_connections 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 1 -- 172.21.15.40:0/1590112509 wait complete. 2024-04-26T21:36:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.531+0000 7fdc531d3740 20 client.5550 trim_cache size 0 max 16384 2024-04-26T21:36:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.549+0000 7fdc531d3740 1 Processor -- start 2024-04-26T21:36:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.549+0000 7fdc531d3740 1 -- start start 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa044dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc531d3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48200fc0 con 0x558aa03be440 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc531d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc480c71f0 con 0x558aa03aff00 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc531d3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc48200a10 con 0x558aa03dbce0 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4d97f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4d97f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52936/0 (socket says 172.21.15.40:52936) 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4d97f640 1 -- 172.21.15.40:0/4152572711 learned_addr learned my addr 172.21.15.40:0/4152572711 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4d97f640 1 -- 172.21.15.40:0/4152572711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 msgr2=0x558aa044dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/4152572711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa044dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4e180640 1 --2- 172.21.15.40:0/4152572711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4d97f640 1 -- 172.21.15.40:0/4152572711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 msgr2=0x558aa03d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/4152572711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03d89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.550+0000 7fdc4d97f640 1 -- 172.21.15.40:0/4152572711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa038f960 con 0x558aa03be440 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/4152572711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d1740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009630 tx=0x7fdc38009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8674d5af2720a70c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc2b7fe640 1 -- 172.21.15.40:0/4152572711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38037040 con 0x558aa03be440 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc2b7fe640 1 -- 172.21.15.40:0/4152572711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc38032a60 con 0x558aa03be440 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc2b7fe640 1 -- 172.21.15.40:0/4152572711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38032e50 con 0x558aa03be440 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 -- 172.21.15.40:0/4152572711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 msgr2=0x558aa03d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 --2- 172.21.15.40:0/4152572711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d1740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009630 tx=0x7fdc38009240 comp rx=0 tx=0).stop 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 -- 172.21.15.40:0/4152572711 shutdown_connections 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 --2- 172.21.15.40:0/4152572711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa044dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 --2- 172.21.15.40:0/4152572711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d1740 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 --2- 172.21.15.40:0/4152572711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 -- 172.21.15.40:0/4152572711 >> 172.21.15.40:0/4152572711 conn(0x558aa0394120 msgr2=0x558aa03bd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 -- 172.21.15.40:0/4152572711 shutdown_connections 2024-04-26T21:36:34.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.551+0000 7fdc531d3740 1 -- 172.21.15.40:0/4152572711 wait complete. 2024-04-26T21:36:34.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.552+0000 7fdc531d3740 1 -- start start 2024-04-26T21:36:34.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.552+0000 7fdc531d3740 1 Processor -- start 2024-04-26T21:36:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03dabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc531d3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa03d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc531d3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558aa03ad9d0 con 0x558aa03be440 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc531d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558aa03d0e10 con 0x558aa03aff00 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc531d3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558aa038d790 con 0x558aa03dbce0 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc531d3740 10 client.0 _ll_register_callbacks cb 0x7fdc4400b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52946/0 (socket says 172.21.15.40:52946) 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 -- 172.21.15.40:0/3290358456 learned_addr learned my addr 172.21.15.40:0/3290358456 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4e180640 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03dabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4e981640 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa03d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 -- 172.21.15.40:0/3290358456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 msgr2=0x558aa03d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa03d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 -- 172.21.15.40:0/3290358456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 msgr2=0x558aa03dabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03dabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 -- 172.21.15.40:0/3290358456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa03acff0 con 0x558aa03be440 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc4d97f640 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d2ba0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdc380090b0 tx=0x7fdc38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee0b6790a576a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.553+0000 7fdc297fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc297fa640 1 -- 172.21.15.40:0/3290358456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38037070 con 0x558aa03be440 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc297fa640 1 -- 172.21.15.40:0/3290358456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fdc38004040 con 0x558aa03be440 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 1 -- 172.21.15.40:0/3290358456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc38032a60 con 0x558aa03be440 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc297fa640 1 -- 172.21.15.40:0/3290358456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdc38057050 con 0x558aa03be440 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 10 client.5556 fetch_fsmap learned FSMap version 17 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 1 -- 172.21.15.40:0/3290358456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x558aa03d3670 con 0x558aa03be440 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc297fa640 1 -- 172.21.15.40:0/3290358456 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc38032a60 con 0x558aa03be440 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc297fa640 1 -- 172.21.15.40:0/3290358456 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fdc3805a020 con 0x558aa03be440 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 10 client.5556 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 10 client.5556 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:36:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 20 client.5556 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] conn(0x558aa03e8750 0x558aa03e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 4 client.5556 mds_command: new command op to 5364 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc531d3740 1 -- 172.21.15.40:0/3290358456 --> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] -- command(tid 0: {"prefix": "status"}) -- 0x558aa038a500 con 0x558aa03e8750 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc4e180640 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] conn(0x558aa03e8750 0x558aa03e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc4e180640 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] conn(0x558aa03e8750 0x558aa03e8b30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c3a7d72b7543e3 server_cookie=503e33578641f6b3 in_seq=0 out_seq=0 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.554+0000 7fdc297fa640 10 client.5556 ms_handle_connect on v2:172.21.15.40:6833/3403660773 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc297fa640 1 -- 172.21.15.40:0/3290358456 <== mds.0 v2:172.21.15.40:6833/3403660773 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x558aa038a500 con 0x558aa03e8750 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc297fa640 10 client.5556 handle_command_reply: tid=0 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 client.5556 shutdown 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 -- 172.21.15.40:0/3290358456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 msgr2=0x558aa03d2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d2ba0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdc380090b0 tx=0x7fdc38004730 comp rx=0 tx=0).stop 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 -- 172.21.15.40:0/3290358456 shutdown_connections 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.40:6833/3403660773,v1:172.21.15.40:6835/3403660773] conn(0x558aa03e8750 0x558aa03e8b30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558aa03dbce0 0x558aa03d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558aa03be440 0x558aa03d2ba0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 --2- 172.21.15.40:0/3290358456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558aa03aff00 0x558aa03dabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc297fa640 0 client.5556 ms_handle_reset on v2:172.21.15.40:6833/3403660773 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 -- 172.21.15.40:0/3290358456 >> 172.21.15.40:0/3290358456 conn(0x558aa0394120 msgr2=0x558aa03bdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 -- 172.21.15.40:0/3290358456 shutdown_connections 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 1 -- 172.21.15.40:0/3290358456 wait complete. 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc531d3740 20 client.5556 trim_cache size 0 max 16384 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:36:34.562 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5364, 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.133064442, 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 17, 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 38, 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 38, 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 17.026716539999999 2024-04-26T21:36:34.563 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:36:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc503e4640 1 -- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc2c06c8f0 msgr2=0x7fdc2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc503e4640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc2c06c8f0 0x7fdc2c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdc480d29a0 tx=0x7fdc44008040 comp rx=0 tx=0).stop 2024-04-26T21:36:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc503e4640 1 -- 172.21.15.40:0/1962577484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc48162810 msgr2=0x7fdc480cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.555+0000 7fdc503e4640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc48162810 0x7fdc480cb3a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c760 tx=0x7fdc4000cc20 comp rx=0 tx=0).stop 2024-04-26T21:36:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.556+0000 7fdc503e4640 1 -- 172.21.15.40:0/1962577484 shutdown_connections 2024-04-26T21:36:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.556+0000 7fdc503e4640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc2c06c8f0 0x7fdc2c06edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.556+0000 7fdc503e4640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc48162810 0x7fdc480cb3a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.556+0000 7fdc503e4640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480cd930 0x7fdc481f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.556+0000 7fdc503e4640 1 --2- 172.21.15.40:0/1962577484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480b2340 0x7fdc48158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.556+0000 7fdc503e4640 1 -- 172.21.15.40:0/1962577484 >> 172.21.15.40:0/1962577484 conn(0x7fdc480d2da0 msgr2=0x7fdc480d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:34.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.556+0000 7fdc503e4640 1 -- 172.21.15.40:0/1962577484 shutdown_connections 2024-04-26T21:36:34.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.556+0000 7fdc503e4640 1 -- 172.21.15.40:0/1962577484 wait complete. 2024-04-26T21:36:34.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:36:34.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.667+0000 7f556abf0640 1 Processor -- start 2024-04-26T21:36:34.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.668+0000 7f556abf0640 1 -- start start 2024-04-26T21:36:34.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.668+0000 7f556abf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5564162810 0x7f5564070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.668+0000 7f556abf0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f55640b2340 0x7f55640cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.668+0000 7f556abf0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55640d10d0 con 0x7f55640cd930 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.668+0000 7f556abf0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f55640d0750 con 0x7f55640b2340 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.668+0000 7f556abf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55640d16a0 con 0x7f5564162810 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.668+0000 7f5563fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f55640b2340 0x7f55640cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.668+0000 7f556918d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f55640cd930 0x7f55640d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47956/0 (socket says 172.21.15.40:47956) 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.669+0000 7f556918d640 1 -- 172.21.15.40:0/3114409786 learned_addr learned my addr 172.21.15.40:0/3114409786 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.669+0000 7f556898c640 1 --2- 172.21.15.40:0/3114409786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5564162810 0x7f5564070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.669+0000 7f5563fff640 1 -- 172.21.15.40:0/3114409786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5564162810 msgr2=0x7f5564070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.669+0000 7f5563fff640 1 --2- 172.21.15.40:0/3114409786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5564162810 0x7f5564070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.669+0000 7f5563fff640 1 -- 172.21.15.40:0/3114409786 >> v1:172.21.15.40:6789/0 conn(0x7f55640cd930 legacy=0x7f55640d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.669+0000 7f5563fff640 1 -- 172.21.15.40:0/3114409786 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55640d1370 con 0x7f55640b2340 2024-04-26T21:36:34.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.669+0000 7f55637fe640 1 -- 172.21.15.40:0/3114409786 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403595913 0 0) 0x7f55640d10d0 con 0x7f55640cd930 2024-04-26T21:36:34.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.669+0000 7f556898c640 1 --2- 172.21.15.40:0/3114409786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5564162810 0x7f5564070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:34.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f5563fff640 1 --2- 172.21.15.40:0/3114409786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55640b2340 0x7f55640cd3f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f55500097a0 tx=0x7f555002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3022b88fae7b7ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f55637fe640 1 -- 172.21.15.40:0/3114409786 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550037070 con 0x7f55640b2340 2024-04-26T21:36:34.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f55637fe640 1 -- 172.21.15.40:0/3114409786 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5550032440 con 0x7f55640b2340 2024-04-26T21:36:34.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f55637fe640 1 -- 172.21.15.40:0/3114409786 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550032810 con 0x7f55640b2340 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f556abf0640 1 -- 172.21.15.40:0/3114409786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55640b2340 msgr2=0x7f55640cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f556abf0640 1 --2- 172.21.15.40:0/3114409786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55640b2340 0x7f55640cd3f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f55500097a0 tx=0x7f555002a180 comp rx=0 tx=0).stop 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f556abf0640 1 -- 172.21.15.40:0/3114409786 shutdown_connections 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f556abf0640 1 --2- 172.21.15.40:0/3114409786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55640b2340 0x7f55640cd3f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f556abf0640 1 --2- 172.21.15.40:0/3114409786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5564162810 0x7f5564070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f556abf0640 1 -- 172.21.15.40:0/3114409786 >> 172.21.15.40:0/3114409786 conn(0x7f55640d2da0 msgr2=0x7f55640d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f556abf0640 1 -- 172.21.15.40:0/3114409786 shutdown_connections 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.670+0000 7f556abf0640 1 -- 172.21.15.40:0/3114409786 wait complete. 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.671+0000 7f556abf0640 1 Processor -- start 2024-04-26T21:36:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.672+0000 7f556abf0640 1 -- start start 2024-04-26T21:36:34.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.672+0000 7f556abf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55640b2340 0x7f55640c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.672+0000 7f556abf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55640cd930 0x7f55640ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.672+0000 7f556abf0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5564162810 0x7f55641f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.672+0000 7f556abf0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5564070890 con 0x7f5564162810 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.672+0000 7f556abf0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55640d0470 con 0x7f55640b2340 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.672+0000 7f556abf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55640d2240 con 0x7f55640cd930 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.672+0000 7f556918d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5564162810 0x7f55641f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556918d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5564162810 0x7f55641f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52968/0 (socket says 172.21.15.40:52968) 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556918d640 1 -- 172.21.15.40:0/1617166663 learned_addr learned my addr 172.21.15.40:0/1617166663 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556918d640 1 -- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55640cd930 msgr2=0x7f55640ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f5563fff640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55640cd930 0x7f55640ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556898c640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55640b2340 0x7f55640c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556918d640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55640cd930 0x7f55640ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556918d640 1 -- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55640b2340 msgr2=0x7f55640c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556918d640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55640b2340 0x7f55640c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556918d640 1 -- 172.21.15.40:0/1617166663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55640c8f70 con 0x7f5564162810 2024-04-26T21:36:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556898c640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55640b2340 0x7f55640c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556918d640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5564162810 0x7f55641f8270 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f555c00c730 tx=0x7f555c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f55617fa640 1 -- 172.21.15.40:0/1617166663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c018070 con 0x7f5564162810 2024-04-26T21:36:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f55617fa640 1 -- 172.21.15.40:0/1617166663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555c00ce40 con 0x7f5564162810 2024-04-26T21:36:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f55617fa640 1 -- 172.21.15.40:0/1617166663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00a860 con 0x7f5564162810 2024-04-26T21:36:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55640ca450 con 0x7f5564162810 2024-04-26T21:36:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.673+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55640c9510 con 0x7f5564162810 2024-04-26T21:36:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.674+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55640cd1b0 con 0x7f5564162810 2024-04-26T21:36:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.675+0000 7f55617fa640 1 -- 172.21.15.40:0/1617166663 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f555c01b070 con 0x7f5564162810 2024-04-26T21:36:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.675+0000 7f55617fa640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.675+0000 7f55617fa640 1 -- 172.21.15.40:0/1617166663 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f555c08b260 con 0x7f5564162810 2024-04-26T21:36:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.675+0000 7f556898c640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.676+0000 7f556898c640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554806c8f0 0x7f554806edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5558006fd0 tx=0x7f5558008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:34.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.679+0000 7f55617fa640 1 -- 172.21.15.40:0/1617166663 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f555c059500 con 0x7f5564162810 2024-04-26T21:36:34.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.904+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f55640694f0 con 0x7f5564162810 2024-04-26T21:36:34.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.904+0000 7f55617fa640 1 -- 172.21.15.40:0/1617166663 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f555c058e00 con 0x7f5564162810 2024-04-26T21:36:34.905 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:36:34.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554806c8f0 msgr2=0x7f554806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554806c8f0 0x7f554806edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5558006fd0 tx=0x7f5558008040 comp rx=0 tx=0).stop 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5564162810 msgr2=0x7f55641f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5564162810 0x7f55641f8270 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f555c00c730 tx=0x7f555c00cbf0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 shutdown_connections 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5564162810 0x7f55641f8270 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55640cd930 0x7f55640ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 --2- 172.21.15.40:0/1617166663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55640b2340 0x7f55640c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 >> 172.21.15.40:0/1617166663 conn(0x7f55640d2da0 msgr2=0x7f5564070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 shutdown_connections 2024-04-26T21:36:34.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:34.906+0000 7f556abf0640 1 -- 172.21.15.40:0/1617166663 wait complete. 2024-04-26T21:36:34.918 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:36:34.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:36:34.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:36:34.918 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:36:34.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:34.918 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:36:34.918 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:36:34.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:36:34.919 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:36:34.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:34.921 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:34.937 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42653sec preferred_lft 42653sec 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:34.938 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:36:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:34.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:34.940 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:36:35.000 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:36:35.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:35.001 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:36:35.058 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:36:35.058 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:36:35.058 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:36:35.058 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:36:35.058 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:36:35.059 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:36:35.059 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:36:35.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:36:35.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:36:35.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:36:35.059 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:36:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:36:35.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:36:35.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:35.185 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:36:35.200 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:36:35.200 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:36:35.200 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:36:35.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:35.201 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:36:35.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:36:35.265 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:35.320 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:36:35.320 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:36:35.321 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:35.375 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:36:35.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:35.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:36:35.458 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:36:35.459 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:36:35.467 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:36:35.467 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:36:35.467 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:36:35.467 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:36:35.467 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:36:35.468 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:36:35.469 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:36:35.469 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:36:35.469 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:36:35.469 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:36:35.470 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:36:35.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:35.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:35.541 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:36:35.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:36:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:36:35.642 INFO:teuthology.orchestra.run.smithi040.stdout:client5272 2024-04-26T21:36:35.642 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:35.642 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:35.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:35.658 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-26T21:36:35.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:35.719 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-26T21:36:35.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:35.776 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-26T21:36:35.833 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:36:35.833 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:36:35.833 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:36:35.833 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:36:35.833 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-26T21:36:35.833 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:36:35.939 INFO:teuthology.orchestra.run.smithi040.stdout:1099511627776 2024-04-26T21:36:35.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:36:35.948 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:35.948 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:36.004 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:36.004 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:36.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.200+0000 7fca54b87640 1 Processor -- start 2024-04-26T21:36:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.200+0000 7fca54b87640 1 -- start start 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca54b87640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca50162880 0x7fca500d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca54b87640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca500b7e50 con 0x7fca500b5f20 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca54b87640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca500d1800 con 0x7fca500d3bc0 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca54b87640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fca500d0580 con 0x7fca50162880 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca47fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fca500b5f20 0x7fca500d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47968/0 (socket says 172.21.15.40:47968) 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca47fff640 1 -- 172.21.15.40:0/3809346293 learned_addr learned my addr 172.21.15.40:0/3809346293 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1674738593 0 0) 0x7fca500b7e50 con 0x7fca500b5f20 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4e59c640 1 --2- 172.21.15.40:0/3809346293 >> v2:172.21.15.184:3300/0 conn(0x7fca50162880 0x7fca500d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca3c003520 con 0x7fca500b5f20 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1885121688 0 0) 0x7fca3c003520 con 0x7fca500b5f20 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 >> v2:172.21.15.184:3300/0 conn(0x7fca50162880 msgr2=0x7fca500d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 --2- 172.21.15.40:0/3809346293 >> v2:172.21.15.184:3300/0 conn(0x7fca50162880 0x7fca500d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 >> v1:172.21.15.184:6789/0 conn(0x7fca500d3bc0 legacy=0x7fca500b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca500d07d0 con 0x7fca500b5f20 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fca38001670 con 0x7fca500b5f20 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.201+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 >> v1:172.21.15.40:6789/0 conn(0x7fca500b5f20 legacy=0x7fca500d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4dd9b640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4dd9b640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4dd9b640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c0051c0 0x7fca3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca500d0580 con 0x7fca3c004800 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c00c060 con 0x7fca3c0051c0 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca3c00c260 con 0x7fca3c0039b0 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca47fff640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4ed9d640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c0051c0 0x7fca3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4e59c640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca47fff640 1 -- 172.21.15.40:0/3809346293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0039b0 msgr2=0x7fca3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca47fff640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca47fff640 1 -- 172.21.15.40:0/3809346293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c0051c0 msgr2=0x7fca3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca47fff640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c0051c0 0x7fca3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca47fff640 1 -- 172.21.15.40:0/3809346293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c003520 con 0x7fca3c004800 2024-04-26T21:36:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.202+0000 7fca4ed9d640 1 -- 172.21.15.40:0/3809346293 reap_dead start 2024-04-26T21:36:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca47fff640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c240 tx=0x7fca3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db61527fb75cca85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38007d10 con 0x7fca3c004800 2024-04-26T21:36:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca38005ad0 con 0x7fca3c004800 2024-04-26T21:36:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca4dd9b640 1 -- 172.21.15.40:0/3809346293 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38007070 con 0x7fca3c004800 2024-04-26T21:36:36.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca54b87640 1 -- 172.21.15.40:0/3809346293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 msgr2=0x7fca3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca54b87640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c240 tx=0x7fca3c005830 comp rx=0 tx=0).stop 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca54b87640 1 -- 172.21.15.40:0/3809346293 shutdown_connections 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca54b87640 1 --2- 172.21.15.40:0/3809346293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca54b87640 1 -- 172.21.15.40:0/3809346293 >> 172.21.15.40:0/3809346293 conn(0x7fca500d4f60 msgr2=0x7fca500d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca54b87640 1 -- 172.21.15.40:0/3809346293 shutdown_connections 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.203+0000 7fca54b87640 1 -- 172.21.15.40:0/3809346293 wait complete. 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.205+0000 7fca54b87640 1 Processor -- start 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.205+0000 7fca54b87640 1 -- start start 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.205+0000 7fca54b87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca501503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.205+0000 7fca54b87640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b5290 0x7fca500b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.205+0000 7fca54b87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca500b69e0 0x7fca500b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.205+0000 7fca54b87640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca50162c60 con 0x7fca3c004800 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.205+0000 7fca54b87640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca500d0d90 con 0x7fca500b69e0 2024-04-26T21:36:36.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.205+0000 7fca54b87640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca50162de0 con 0x7fca500b5290 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca501503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca501503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52978/0 (socket says 172.21.15.40:52978) 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 -- 172.21.15.40:0/2163496542 learned_addr learned my addr 172.21.15.40:0/2163496542 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca47fff640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b5290 0x7fca500b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4ed9d640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca500b69e0 0x7fca500b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 -- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b5290 msgr2=0x7fca500b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b5290 0x7fca500b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 -- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca500b69e0 msgr2=0x7fca500b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca500b69e0 0x7fca500b6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 -- 172.21.15.40:0/2163496542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca501508f0 con 0x7fca3c004800 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca4e59c640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca501503b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fca40011700 tx=0x7fca40011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca46ffd640 1 -- 172.21.15.40:0/2163496542 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4000f9c0 con 0x7fca3c004800 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca46ffd640 1 -- 172.21.15.40:0/2163496542 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca4000fb60 con 0x7fca3c004800 2024-04-26T21:36:36.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca500b1cd0 con 0x7fca3c004800 2024-04-26T21:36:36.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca46ffd640 1 -- 172.21.15.40:0/2163496542 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40015460 con 0x7fca3c004800 2024-04-26T21:36:36.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.206+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca500b2290 con 0x7fca3c004800 2024-04-26T21:36:36.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.207+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca500b73a0 con 0x7fca3c004800 2024-04-26T21:36:36.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.208+0000 7fca46ffd640 1 -- 172.21.15.40:0/2163496542 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca40015600 con 0x7fca3c004800 2024-04-26T21:36:36.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.208+0000 7fca46ffd640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.208+0000 7fca46ffd640 1 -- 172.21.15.40:0/2163496542 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fca4008c400 con 0x7fca3c004800 2024-04-26T21:36:36.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.209+0000 7fca47fff640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.209+0000 7fca47fff640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca3006c8f0 0x7fca3006edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fca38033000 tx=0x7fca3802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.213+0000 7fca46ffd640 1 -- 172.21.15.40:0/2163496542 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca501508f0 con 0x7fca3c004800 2024-04-26T21:36:36.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.399+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca500694f0 con 0x7fca3c004800 2024-04-26T21:36:36.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.400+0000 7fca46ffd640 1 -- 172.21.15.40:0/2163496542 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16906 (secure 0 0 0) 0x7fca40019030 con 0x7fca3c004800 2024-04-26T21:36:36.401 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:36.402 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:25.421233+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca3006c8f0 msgr2=0x7fca3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca3006c8f0 0x7fca3006edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fca38033000 tx=0x7fca3802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 msgr2=0x7fca501503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca501503b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fca40011700 tx=0x7fca40011bc0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 shutdown_connections 2024-04-26T21:36:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca500b69e0 0x7fca500b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b5290 0x7fca500b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 --2- 172.21.15.40:0/2163496542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004800 0x7fca501503b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 >> 172.21.15.40:0/2163496542 conn(0x7fca500d4f60 msgr2=0x7fca500ba210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 shutdown_connections 2024-04-26T21:36:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.402+0000 7fca54b87640 1 -- 172.21.15.40:0/2163496542 wait complete. 2024-04-26T21:36:36.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:36:22.370518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:36:22.724779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:36:36.415 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:36:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.519+0000 7fbe59b9e640 1 Processor -- start 2024-04-26T21:36:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.519+0000 7fbe59b9e640 1 -- start start 2024-04-26T21:36:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.519+0000 7fbe59b9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe540b5500 0x7fbe540b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.519+0000 7fbe59b9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.519+0000 7fbe59b9e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe540d3dc0 con 0x7fbe540b5120 2024-04-26T21:36:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.519+0000 7fbe59b9e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbe540d03b0 con 0x7fbe540b5500 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.519+0000 7fbe59b9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe540b75b0 con 0x7fbe540b59b0 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe53fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe540b5500 0x7fbe540b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe58b9c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbe540b5120 0x7fbe540b6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47980/0 (socket says 172.21.15.40:47980) 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe58b9c640 1 -- 172.21.15.40:0/3297221503 learned_addr learned my addr 172.21.15.40:0/3297221503 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe53fff640 1 -- 172.21.15.40:0/3297221503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 msgr2=0x7fbe540b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe5939d640 1 --2- 172.21.15.40:0/3297221503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe53fff640 1 --2- 172.21.15.40:0/3297221503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe53fff640 1 -- 172.21.15.40:0/3297221503 >> v1:172.21.15.40:6789/0 conn(0x7fbe540b5120 legacy=0x7fbe540b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe537fe640 1 -- 172.21.15.40:0/3297221503 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388146253 0 0) 0x7fbe540d3dc0 con 0x7fbe540b5120 2024-04-26T21:36:36.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe53fff640 1 -- 172.21.15.40:0/3297221503 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe540d0600 con 0x7fbe540b5500 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.520+0000 7fbe5939d640 1 --2- 172.21.15.40:0/3297221503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe53fff640 1 --2- 172.21.15.40:0/3297221503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5500 0x7fbe540b5ee0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0097d0 tx=0x7fbe3c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cc7224738404f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe537fe640 1 -- 172.21.15.40:0/3297221503 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c037070 con 0x7fbe540b5500 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe537fe640 1 -- 172.21.15.40:0/3297221503 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3c02adb0 con 0x7fbe540b5500 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe537fe640 1 -- 172.21.15.40:0/3297221503 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c032660 con 0x7fbe540b5500 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe59b9e640 1 -- 172.21.15.40:0/3297221503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5500 msgr2=0x7fbe540b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/3297221503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5500 0x7fbe540b5ee0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0097d0 tx=0x7fbe3c02a0a0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe59b9e640 1 -- 172.21.15.40:0/3297221503 shutdown_connections 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/3297221503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/3297221503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5500 0x7fbe540b5ee0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe59b9e640 1 -- 172.21.15.40:0/3297221503 >> 172.21.15.40:0/3297221503 conn(0x7fbe540d4db0 msgr2=0x7fbe540d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:36.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe59b9e640 1 -- 172.21.15.40:0/3297221503 shutdown_connections 2024-04-26T21:36:36.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.521+0000 7fbe59b9e640 1 -- 172.21.15.40:0/3297221503 wait complete. 2024-04-26T21:36:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.523+0000 7fbe59b9e640 1 Processor -- start 2024-04-26T21:36:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.523+0000 7fbe59b9e640 1 -- start start 2024-04-26T21:36:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe59b9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5120 0x7fbe541f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe59b9e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540b5500 0x7fbe540b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe59b9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe59b9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbe540b7c00 con 0x7fbe540b5120 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe59b9e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe540d0bc0 con 0x7fbe540b5500 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe59b9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe540b7d80 con 0x7fbe540b59b0 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5120 0x7fbe541f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5120 0x7fbe541f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53006/0 (socket says 172.21.15.40:53006) 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 -- 172.21.15.40:0/480555202 learned_addr learned my addr 172.21.15.40:0/480555202 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe53fff640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540b5500 0x7fbe540b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe5939d640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 -- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 msgr2=0x7fbe540b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 -- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540b5500 msgr2=0x7fbe540b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540b5500 0x7fbe540b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 -- 172.21.15.40:0/480555202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe541f4810 con 0x7fbe540b5120 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe58b9c640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5120 0x7fbe541f42d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00c890 tx=0x7fbe4c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe517fa640 1 -- 172.21.15.40:0/480555202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c018070 con 0x7fbe540b5120 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe517fa640 1 -- 172.21.15.40:0/480555202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4c00a640 con 0x7fbe540b5120 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe541f84c0 con 0x7fbe540b5120 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe517fa640 1 -- 172.21.15.40:0/480555202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00aa30 con 0x7fbe540b5120 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.524+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe541f8a80 con 0x7fbe540b5120 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.525+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe54069370 con 0x7fbe540b5120 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.526+0000 7fbe517fa640 1 -- 172.21.15.40:0/480555202 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe4c01b070 con 0x7fbe540b5120 2024-04-26T21:36:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.526+0000 7fbe517fa640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe3406c8f0 0x7fbe3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.526+0000 7fbe517fa640 1 -- 172.21.15.40:0/480555202 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fbe4c08b3b0 con 0x7fbe540b5120 2024-04-26T21:36:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.526+0000 7fbe53fff640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe3406c8f0 0x7fbe3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.527+0000 7fbe53fff640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe3406c8f0 0x7fbe3406edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c034000 tx=0x7fbe3c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.531+0000 7fbe517fa640 1 -- 172.21.15.40:0/480555202 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe541f4810 con 0x7fbe540b5120 2024-04-26T21:36:36.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.718+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe54074360 con 0x7fbe540b5120 2024-04-26T21:36:36.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.718+0000 7fbe517fa640 1 -- 172.21.15.40:0/480555202 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7fbe4c059640 con 0x7fbe540b5120 2024-04-26T21:36:36.720 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:36:36.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.720+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe3406c8f0 msgr2=0x7fbe3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.720+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe3406c8f0 0x7fbe3406edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c034000 tx=0x7fbe3c02e000 comp rx=0 tx=0).stop 2024-04-26T21:36:36.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.720+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5120 msgr2=0x7fbe541f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.720+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5120 0x7fbe541f42d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00c890 tx=0x7fbe4c00cd50 comp rx=0 tx=0).stop 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.720+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 shutdown_connections 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.720+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe3406c8f0 0x7fbe3406edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.721+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe540b59b0 0x7fbe540b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.721+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540b5500 0x7fbe540b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.721+0000 7fbe59b9e640 1 --2- 172.21.15.40:0/480555202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe540b5120 0x7fbe541f42d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.721+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 >> 172.21.15.40:0/480555202 conn(0x7fbe540d4db0 msgr2=0x7fbe540b87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.721+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 shutdown_connections 2024-04-26T21:36:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.721+0000 7fbe59b9e640 1 -- 172.21.15.40:0/480555202 wait complete. 2024-04-26T21:36:36.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:36:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.839+0000 7f41eda72640 1 Processor -- start 2024-04-26T21:36:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.839+0000 7f41eda72640 1 -- start start 2024-04-26T21:36:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41eda72640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f41e80cfde0 0x7f41e80d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41eda72640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41e8071260 0x7f41e815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41eda72640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e80d2390 con 0x7f41e8070350 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41eda72640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f41e80afab0 con 0x7f41e8071260 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41eda72640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f41e806dd20 con 0x7f41e80cfde0 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41e8071260 0x7f41e815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f41e8070350 0x7f41e8070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47982/0 (socket says 172.21.15.40:47982) 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e6ffd640 1 -- 172.21.15.40:0/2285659807 learned_addr learned my addr 172.21.15.40:0/2285659807 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e77fe640 1 --2- 172.21.15.40:0/2285659807 >> v2:172.21.15.184:3300/0 conn(0x7f41e80cfde0 0x7f41e80d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e7fff640 1 -- 172.21.15.40:0/2285659807 >> v2:172.21.15.184:3300/0 conn(0x7f41e80cfde0 msgr2=0x7f41e80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e7fff640 1 --2- 172.21.15.40:0/2285659807 >> v2:172.21.15.184:3300/0 conn(0x7f41e80cfde0 0x7f41e80d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e7fff640 1 -- 172.21.15.40:0/2285659807 >> v1:172.21.15.40:6789/0 conn(0x7f41e8070350 legacy=0x7f41e8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e67fc640 1 -- 172.21.15.40:0/2285659807 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670994995 0 0) 0x7f41e80d2390 con 0x7f41e8070350 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e7fff640 1 -- 172.21.15.40:0/2285659807 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e806d3f0 con 0x7f41e8071260 2024-04-26T21:36:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.840+0000 7f41e77fe640 1 --2- 172.21.15.40:0/2285659807 >> v2:172.21.15.184:3300/0 conn(0x7f41e80cfde0 0x7f41e80d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.841+0000 7f41e7fff640 1 --2- 172.21.15.40:0/2285659807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e815e9f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f41e80762b0 tx=0x7f41e002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4190ab23ff0471 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.841+0000 7f41e67fc640 1 -- 172.21.15.40:0/2285659807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0007740 con 0x7f41e8071260 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.841+0000 7f41e67fc640 1 -- 172.21.15.40:0/2285659807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e00078e0 con 0x7f41e8071260 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.841+0000 7f41e67fc640 1 -- 172.21.15.40:0/2285659807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0007cb0 con 0x7f41e8071260 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.842+0000 7f41eda72640 1 -- 172.21.15.40:0/2285659807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 msgr2=0x7f41e815e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.842+0000 7f41eda72640 1 --2- 172.21.15.40:0/2285659807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e815e9f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f41e80762b0 tx=0x7f41e002a5f0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.842+0000 7f41eda72640 1 -- 172.21.15.40:0/2285659807 shutdown_connections 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.842+0000 7f41eda72640 1 --2- 172.21.15.40:0/2285659807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e815e9f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.842+0000 7f41eda72640 1 --2- 172.21.15.40:0/2285659807 >> v2:172.21.15.184:3300/0 conn(0x7f41e80cfde0 0x7f41e80d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.842+0000 7f41eda72640 1 -- 172.21.15.40:0/2285659807 >> 172.21.15.40:0/2285659807 conn(0x7f41e80d2d80 msgr2=0x7f41e80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.842+0000 7f41eda72640 1 -- 172.21.15.40:0/2285659807 shutdown_connections 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.842+0000 7f41eda72640 1 -- 172.21.15.40:0/2285659807 wait complete. 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.843+0000 7f41eda72640 1 Processor -- start 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.843+0000 7f41eda72640 1 -- start start 2024-04-26T21:36:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41eda72640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e8070350 0x7f41e8161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41eda72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e8153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41eda72640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80cfde0 0x7f41e80b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41eda72640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41e80d20a0 con 0x7f41e8071260 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41eda72640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41e806d9b0 con 0x7f41e8070350 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41eda72640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41e815ef30 con 0x7f41e80cfde0 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e8153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e8153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53020/0 (socket says 172.21.15.40:53020) 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 -- 172.21.15.40:0/3094781998 learned_addr learned my addr 172.21.15.40:0/3094781998 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 -- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80cfde0 msgr2=0x7f41e80b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80cfde0 0x7f41e80b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 -- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e8070350 msgr2=0x7f41e8161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e77fe640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e8070350 0x7f41e8161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e8070350 0x7f41e8161cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 -- 172.21.15.40:0/3094781998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e809e9f0 con 0x7f41e8071260 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41e6ffd640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e8153fe0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00c4d0 tx=0x7f41dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41cffff640 1 -- 172.21.15.40:0/3094781998 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc018070 con 0x7f41e8071260 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.844+0000 7f41cffff640 1 -- 172.21.15.40:0/3094781998 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41dc007500 con 0x7f41e8071260 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.845+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e8162200 con 0x7f41e8071260 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.845+0000 7f41cffff640 1 -- 172.21.15.40:0/3094781998 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc004a20 con 0x7f41e8071260 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.845+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e809e1f0 con 0x7f41e8071260 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.846+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e809dd30 con 0x7f41e8071260 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.846+0000 7f41cffff640 1 -- 172.21.15.40:0/3094781998 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41dc01b070 con 0x7f41e8071260 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.846+0000 7f41cffff640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:36.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.846+0000 7f41cffff640 1 -- 172.21.15.40:0/3094781998 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f41dc08af60 con 0x7f41e8071260 2024-04-26T21:36:36.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.847+0000 7f41e77fe640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:36.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.847+0000 7f41e77fe640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41d406c8f0 0x7f41d406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f41d800b840 tx=0x7f41d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:36.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:36.851+0000 7f41cffff640 1 -- 172.21.15.40:0/3094781998 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41e809dd30 con 0x7f41e8071260 2024-04-26T21:36:37.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.036+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41e80b8ac0 con 0x7f41e8071260 2024-04-26T21:36:37.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.036+0000 7f41cffff640 1 -- 172.21.15.40:0/3094781998 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f41dc059200 con 0x7f41e8071260 2024-04-26T21:36:37.038 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41d406c8f0 msgr2=0x7f41d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41d406c8f0 0x7f41d406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f41d800b840 tx=0x7f41d8008040 comp rx=0 tx=0).stop 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 msgr2=0x7f41e8153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e8153fe0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00c4d0 tx=0x7f41dc00c990 comp rx=0 tx=0).stop 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 shutdown_connections 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80cfde0 0x7f41e80b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e8071260 0x7f41e8153fe0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 --2- 172.21.15.40:0/3094781998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e8070350 0x7f41e8161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 >> 172.21.15.40:0/3094781998 conn(0x7f41e80d2d80 msgr2=0x7f41e80c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 shutdown_connections 2024-04-26T21:36:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.038+0000 7f41eda72640 1 -- 172.21.15.40:0/3094781998 wait complete. 2024-04-26T21:36:37.051 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:36:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.195+0000 7f1f57b24640 1 Processor -- start 2024-04-26T21:36:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.196+0000 7f1f57b24640 1 -- start start 2024-04-26T21:36:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.196+0000 7f1f57b24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f50162cd0 0x7f1f500b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.196+0000 7f1f57b24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f500d2110 0x7f1f500b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.196+0000 7f1f57b24640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f500d0890 con 0x7f1f501628f0 2024-04-26T21:36:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.196+0000 7f1f57b24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1f500d1130 con 0x7f1f50162cd0 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.196+0000 7f1f57b24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f500d0b70 con 0x7f1f500d2110 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f550bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f50162cd0 0x7f1f500b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f558c0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1f501628f0 0x7f1f500d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47996/0 (socket says 172.21.15.40:47996) 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f558c0640 1 -- 172.21.15.40:0/2732733465 learned_addr learned my addr 172.21.15.40:0/2732733465 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f560c1640 1 --2- 172.21.15.40:0/2732733465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f500d2110 0x7f1f500b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f550bf640 1 -- 172.21.15.40:0/2732733465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f500d2110 msgr2=0x7f1f500b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f550bf640 1 --2- 172.21.15.40:0/2732733465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f500d2110 0x7f1f500b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f550bf640 1 -- 172.21.15.40:0/2732733465 >> v1:172.21.15.40:6789/0 conn(0x7f1f501628f0 legacy=0x7f1f500d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f550bf640 1 -- 172.21.15.40:0/2732733465 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f500d0e50 con 0x7f1f50162cd0 2024-04-26T21:36:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.197+0000 7f1f548be640 1 -- 172.21.15.40:0/2732733465 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1277044832 0 0) 0x7f1f500d0890 con 0x7f1f501628f0 2024-04-26T21:36:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f550bf640 1 --2- 172.21.15.40:0/2732733465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f50162cd0 0x7f1f500b4c60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1f400097a0 tx=0x7f1f4002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9153556926cdcbf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f548be640 1 -- 172.21.15.40:0/2732733465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40037070 con 0x7f1f50162cd0 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f548be640 1 -- 172.21.15.40:0/2732733465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4002ae50 con 0x7f1f50162cd0 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f548be640 1 -- 172.21.15.40:0/2732733465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40032660 con 0x7f1f50162cd0 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f57b24640 1 -- 172.21.15.40:0/2732733465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f50162cd0 msgr2=0x7f1f500b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f57b24640 1 --2- 172.21.15.40:0/2732733465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f50162cd0 0x7f1f500b4c60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1f400097a0 tx=0x7f1f4002a100 comp rx=0 tx=0).stop 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f57b24640 1 -- 172.21.15.40:0/2732733465 shutdown_connections 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f57b24640 1 --2- 172.21.15.40:0/2732733465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f500d2110 0x7f1f500b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f57b24640 1 --2- 172.21.15.40:0/2732733465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f50162cd0 0x7f1f500b4c60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.198+0000 7f1f57b24640 1 -- 172.21.15.40:0/2732733465 >> 172.21.15.40:0/2732733465 conn(0x7f1f500d4ff0 msgr2=0x7f1f500d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.199+0000 7f1f57b24640 1 -- 172.21.15.40:0/2732733465 shutdown_connections 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.199+0000 7f1f57b24640 1 -- 172.21.15.40:0/2732733465 wait complete. 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.200+0000 7f1f57b24640 1 Processor -- start 2024-04-26T21:36:37.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.200+0000 7f1f57b24640 1 -- start start 2024-04-26T21:36:37.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.200+0000 7f1f57b24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f500d2110 0x7f1f5015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.200+0000 7f1f57b24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f501628f0 0x7f1f500cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.200+0000 7f1f57b24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f50162cd0 0x7f1f500b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.200+0000 7f1f57b24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500d16d0 con 0x7f1f500d2110 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.200+0000 7f1f57b24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500d1550 con 0x7f1f501628f0 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.200+0000 7f1f57b24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f500b3f50 con 0x7f1f50162cd0 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f500d2110 0x7f1f5015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f500d2110 0x7f1f5015d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53034/0 (socket says 172.21.15.40:53034) 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 -- 172.21.15.40:0/3685082044 learned_addr learned my addr 172.21.15.40:0/3685082044 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f550bf640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f501628f0 0x7f1f500cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f560c1640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f50162cd0 0x7f1f500b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 -- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f50162cd0 msgr2=0x7f1f500b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f50162cd0 0x7f1f500b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 -- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f501628f0 msgr2=0x7f1f500cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f501628f0 0x7f1f500cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 -- 172.21.15.40:0/3685082044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f500cc2f0 con 0x7f1f500d2110 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f558c0640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f500d2110 0x7f1f5015d4f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c4d0 tx=0x7f1f4c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f467fc640 1 -- 172.21.15.40:0/3685082044 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c018070 con 0x7f1f500d2110 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f467fc640 1 -- 172.21.15.40:0/3685082044 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4c007500 con 0x7f1f500d2110 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f467fc640 1 -- 172.21.15.40:0/3685082044 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c004a20 con 0x7f1f500d2110 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f500cc5d0 con 0x7f1f500d2110 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.201+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f500ccb90 con 0x7f1f500d2110 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.202+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f500694f0 con 0x7f1f500d2110 2024-04-26T21:36:37.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.203+0000 7f1f467fc640 1 -- 172.21.15.40:0/3685082044 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f4c01b070 con 0x7f1f500d2110 2024-04-26T21:36:37.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.203+0000 7f1f467fc640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3406c8f0 0x7f1f3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.203+0000 7f1f467fc640 1 -- 172.21.15.40:0/3685082044 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1f4c08aff0 con 0x7f1f500d2110 2024-04-26T21:36:37.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.203+0000 7f1f550bf640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3406c8f0 0x7f1f3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.204+0000 7f1f550bf640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3406c8f0 0x7f1f3406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034000 tx=0x7f1f4002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.207+0000 7f1f467fc640 1 -- 172.21.15.40:0/3685082044 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f4c0592c0 con 0x7f1f500d2110 2024-04-26T21:36:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.397+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f500bae90 con 0x7f1f500d2110 2024-04-26T21:36:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.397+0000 7f1f467fc640 1 -- 172.21.15.40:0/3685082044 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f1f4c058bc0 con 0x7f1f500d2110 2024-04-26T21:36:37.398 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:36:37.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3406c8f0 msgr2=0x7f1f3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3406c8f0 0x7f1f3406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034000 tx=0x7f1f4002e000 comp rx=0 tx=0).stop 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f500d2110 msgr2=0x7f1f5015d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f500d2110 0x7f1f5015d4f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c4d0 tx=0x7f1f4c00c990 comp rx=0 tx=0).stop 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 shutdown_connections 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3406c8f0 0x7f1f3406edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f50162cd0 0x7f1f500b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f501628f0 0x7f1f500cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 --2- 172.21.15.40:0/3685082044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f500d2110 0x7f1f5015d4f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 >> 172.21.15.40:0/3685082044 conn(0x7f1f500d4ff0 msgr2=0x7f1f5015fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 shutdown_connections 2024-04-26T21:36:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.399+0000 7f1f57b24640 1 -- 172.21.15.40:0/3685082044 wait complete. 2024-04-26T21:36:37.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:36:37.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.511+0000 7f0dc9778640 1 Processor -- start 2024-04-26T21:36:37.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.511+0000 7f0dc9778640 1 -- start start 2024-04-26T21:36:37.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.512+0000 7f0dc9778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40d1bf0 0x7f0dc40d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.512+0000 7f0dc9778640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc40cde20 con 0x7f0dc41608f0 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.512+0000 7f0dc9778640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc40ce100 con 0x7f0dc40cffa0 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.512+0000 7f0dc9778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc40cea30 con 0x7f0dc40d1bf0 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.512+0000 7f0dc2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0dc41608f0 0x7f0dc40cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48002/0 (socket says 172.21.15.40:48002) 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.512+0000 7f0dc2ffd640 1 -- 172.21.15.40:0/1927947201 learned_addr learned my addr 172.21.15.40:0/1927947201 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2287551865 0 0) 0x7f0dc40cde20 con 0x7f0dc41608f0 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0db0003520 con 0x7f0dc41608f0 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc37fe640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40d1bf0 0x7f0dc40d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 335134498 0 0) 0x7f0db0003520 con 0x7f0dc41608f0 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40d1bf0 msgr2=0x7f0dc40d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40d1bf0 0x7f0dc40d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 >> v1:172.21.15.184:6789/0 conn(0x7f0dc40cffa0 legacy=0x7f0dc40b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc40ce3e0 con 0x7f0dc41608f0 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f0db40019f0 con 0x7f0dc41608f0 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 >> v1:172.21.15.40:6789/0 conn(0x7f0dc41608f0 legacy=0x7f0dc40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db00039e0 0x7f0db0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db0004390 0x7f0db0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40d1bf0 0x7f0db00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc40cea30 con 0x7f0db0004390 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0db000b9c0 con 0x7f0db00039e0 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0db000bbc0 con 0x7f0dc40d1bf0 2024-04-26T21:36:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc27fc640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db0004390 0x7f0db0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc2ffd640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db00039e0 0x7f0db0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.513+0000 7f0dc37fe640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40d1bf0 0x7f0db00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc27fc640 1 -- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40d1bf0 msgr2=0x7f0db00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc27fc640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40d1bf0 0x7f0db00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc27fc640 1 -- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db00039e0 msgr2=0x7f0db0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc27fc640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db00039e0 0x7f0db0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc27fc640 1 -- 172.21.15.40:0/1927947201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db0003520 con 0x7f0db0004390 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc27fc640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db0004390 0x7f0db0004860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0dac001860 tx=0x7f0db0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aa50d10a990689 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac037070 con 0x7f0db0004390 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dac02abf0 con 0x7f0db0004390 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.514+0000 7f0dc1ffb640 1 -- 172.21.15.40:0/1927947201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac001430 con 0x7f0db0004390 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.515+0000 7f0dc9778640 1 -- 172.21.15.40:0/1927947201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db0004390 msgr2=0x7f0db0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.515+0000 7f0dc9778640 1 --2- 172.21.15.40:0/1927947201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db0004390 0x7f0db0004860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0dac001860 tx=0x7f0db0005830 comp rx=0 tx=0).stop 2024-04-26T21:36:37.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.515+0000 7f0dc37fe640 1 -- 172.21.15.40:0/1927947201 reap_dead start 2024-04-26T21:36:37.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.515+0000 7f0dc9778640 1 -- 172.21.15.40:0/1927947201 shutdown_connections 2024-04-26T21:36:37.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.515+0000 7f0dc9778640 1 -- 172.21.15.40:0/1927947201 >> 172.21.15.40:0/1927947201 conn(0x7f0dc40d2ec0 msgr2=0x7f0dc40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:37.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.515+0000 7f0dc9778640 1 -- 172.21.15.40:0/1927947201 shutdown_connections 2024-04-26T21:36:37.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.515+0000 7f0dc9778640 1 -- 172.21.15.40:0/1927947201 wait complete. 2024-04-26T21:36:37.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.516+0000 7f0dc9778640 1 Processor -- start 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.516+0000 7f0dc9778640 1 -- start start 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc9778640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db0004390 0x7f0dc40c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc9778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40b8760 0x7f0dc4154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc9778640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dc40b6ee0 0x7f0dc40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc9778640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc40cda40 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc9778640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc40ceec0 con 0x7f0db0004390 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc9778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc40cdbc0 con 0x7f0dc40b8760 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dc40b6ee0 0x7f0dc40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dc40b6ee0 0x7f0dc40b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53056/0 (socket says 172.21.15.40:53056) 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc37fe640 1 -- 172.21.15.40:0/3693245648 learned_addr learned my addr 172.21.15.40:0/3693245648 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc2ffd640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db0004390 0x7f0dc40c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc27fc640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40b8760 0x7f0dc4154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc37fe640 1 -- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40b8760 msgr2=0x7f0dc4154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.517+0000 7f0dc37fe640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40b8760 0x7f0dc4154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dc37fe640 1 -- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db0004390 msgr2=0x7f0dc40c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dc37fe640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db0004390 0x7f0dc40c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dc37fe640 1 -- 172.21.15.40:0/3693245648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc40b76c0 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dc27fc640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40b8760 0x7f0dc4154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dc37fe640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dc40b6ee0 0x7f0dc40b0d10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc011860 tx=0x7f0dbc011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dbb7fe640 1 -- 172.21.15.40:0/3693245648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc004240 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dbb7fe640 1 -- 172.21.15.40:0/3693245648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc01b070 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc40b79a0 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dbb7fe640 1 -- 172.21.15.40:0/3693245648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00f7a0 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.518+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc40b7f60 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.519+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc40cde20 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.519+0000 7f0dbb7fe640 1 -- 172.21.15.40:0/3693245648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dbc012040 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.520+0000 7f0dbb7fe640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.520+0000 7f0dbb7fe640 1 -- 172.21.15.40:0/3693245648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0dbc08bbf0 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.520+0000 7f0dc2ffd640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.521+0000 7f0dc2ffd640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0da406c8f0 0x7f0da406edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000f80 tx=0x7f0db4006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.524+0000 7f0dbb7fe640 1 -- 172.21.15.40:0/3693245648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0dc40cde20 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.722+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0dc40694f0 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.723+0000 7f0dbb7fe640 1 -- 172.21.15.40:0/3693245648 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f0dbc059d80 con 0x7f0dc40b6ee0 2024-04-26T21:36:37.724 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0da406c8f0 msgr2=0x7f0da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0da406c8f0 0x7f0da406edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000f80 tx=0x7f0db4006040 comp rx=0 tx=0).stop 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dc40b6ee0 msgr2=0x7f0dc40b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dc40b6ee0 0x7f0dc40b0d10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc011860 tx=0x7f0dbc011d20 comp rx=0 tx=0).stop 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 shutdown_connections 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dc40b6ee0 0x7f0dc40b0d10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dc40b8760 0x7f0dc4154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 --2- 172.21.15.40:0/3693245648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db0004390 0x7f0dc40c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.724+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 >> 172.21.15.40:0/3693245648 conn(0x7f0dc40d2ec0 msgr2=0x7f0dc40b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:37.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.725+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 shutdown_connections 2024-04-26T21:36:37.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.725+0000 7f0dc9778640 1 -- 172.21.15.40:0/3693245648 wait complete. 2024-04-26T21:36:37.736 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:36:37.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:37.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:37.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:36:37.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:37.784 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:36:37.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:37.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.949+0000 7fdc9be09640 1 Processor -- start 2024-04-26T21:36:37.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.949+0000 7fdc9be09640 1 -- start start 2024-04-26T21:36:37.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.949+0000 7fdc9be09640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc94156460 0x7fdc940cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc9be09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc94156840 con 0x7fdc9406fea0 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc9be09640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc940cbf20 con 0x7fdc940cba10 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc9be09640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdc940cc200 con 0x7fdc94156460 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc9a3a6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdc9406fea0 0x7fdc94072c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48012/0 (socket says 172.21.15.40:48012) 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc9a3a6640 1 -- 172.21.15.40:0/3231373163 learned_addr learned my addr 172.21.15.40:0/3231373163 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc99ba5640 1 --2- 172.21.15.40:0/3231373163 >> v2:172.21.15.184:3300/0 conn(0x7fdc94156460 0x7fdc940cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc98ba3640 1 -- 172.21.15.40:0/3231373163 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3097398602 0 0) 0x7fdc94156840 con 0x7fdc9406fea0 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc98ba3640 1 -- 172.21.15.40:0/3231373163 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc80003520 con 0x7fdc9406fea0 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc98ba3640 1 -- 172.21.15.40:0/3231373163 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000182281 0 0) 0x7fdc940cbf20 con 0x7fdc940cba10 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc98ba3640 1 -- 172.21.15.40:0/3231373163 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc94156840 con 0x7fdc940cba10 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc99ba5640 1 -- 172.21.15.40:0/3231373163 >> v1:172.21.15.184:6789/0 conn(0x7fdc940cba10 legacy=0x7fdc9406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc99ba5640 1 -- 172.21.15.40:0/3231373163 >> v1:172.21.15.40:6789/0 conn(0x7fdc9406fea0 legacy=0x7fdc94072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc98ba3640 1 -- 172.21.15.40:0/3231373163 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1887141204 0 0) 0x7fdc80003520 con 0x7fdc9406fea0 2024-04-26T21:36:37.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.950+0000 7fdc99ba5640 1 -- 172.21.15.40:0/3231373163 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc940cc4e0 con 0x7fdc94156460 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.951+0000 7fdc99ba5640 1 --2- 172.21.15.40:0/3231373163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc94156460 0x7fdc940cb4d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0097a0 tx=0x7fdc8c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f29b64982568a6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.951+0000 7fdc98ba3640 1 -- 172.21.15.40:0/3231373163 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c037070 con 0x7fdc94156460 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.951+0000 7fdc98ba3640 1 -- 172.21.15.40:0/3231373163 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c02ac50 con 0x7fdc94156460 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.951+0000 7fdc98ba3640 1 -- 172.21.15.40:0/3231373163 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc900019f0 con 0x7fdc94156460 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.952+0000 7fdc9be09640 1 -- 172.21.15.40:0/3231373163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc94156460 msgr2=0x7fdc940cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.952+0000 7fdc9be09640 1 --2- 172.21.15.40:0/3231373163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc94156460 0x7fdc940cb4d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0097a0 tx=0x7fdc8c02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.952+0000 7fdc9be09640 1 -- 172.21.15.40:0/3231373163 shutdown_connections 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.952+0000 7fdc9be09640 1 --2- 172.21.15.40:0/3231373163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc94156460 0x7fdc940cb4d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.952+0000 7fdc9be09640 1 -- 172.21.15.40:0/3231373163 >> 172.21.15.40:0/3231373163 conn(0x7fdc940d0be0 msgr2=0x7fdc940d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:37.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.952+0000 7fdc9be09640 1 -- 172.21.15.40:0/3231373163 shutdown_connections 2024-04-26T21:36:37.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.952+0000 7fdc9be09640 1 -- 172.21.15.40:0/3231373163 wait complete. 2024-04-26T21:36:37.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.953+0000 7fdc9be09640 1 Processor -- start 2024-04-26T21:36:37.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.953+0000 7fdc9be09640 1 -- start start 2024-04-26T21:36:37.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc9be09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc9406fea0 0x7fdc94151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc9be09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc940cba10 0x7fdc940c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc9be09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc94156460 0x7fdc940c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc9be09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9406e260 con 0x7fdc9406fea0 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc9be09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94072700 con 0x7fdc940cba10 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc9be09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9406e3e0 con 0x7fdc94156460 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc99ba5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc9406fea0 0x7fdc94151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc99ba5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc9406fea0 0x7fdc94151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53064/0 (socket says 172.21.15.40:53064) 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc99ba5640 1 -- 172.21.15.40:0/3223440558 learned_addr learned my addr 172.21.15.40:0/3223440558 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc993a4640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc940cba10 0x7fdc940c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc9a3a6640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc94156460 0x7fdc940c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc99ba5640 1 -- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc94156460 msgr2=0x7fdc940c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc99ba5640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc94156460 0x7fdc940c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc99ba5640 1 -- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc940cba10 msgr2=0x7fdc940c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc99ba5640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc940cba10 0x7fdc940c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc99ba5640 1 -- 172.21.15.40:0/3223440558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc94153370 con 0x7fdc9406fea0 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.954+0000 7fdc993a4640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc940cba10 0x7fdc940c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:37.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.955+0000 7fdc99ba5640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc9406fea0 0x7fdc94151180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c035040 tx=0x7fdc8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.955+0000 7fdc8a7fc640 1 -- 172.21.15.40:0/3223440558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c041020 con 0x7fdc9406fea0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.955+0000 7fdc8a7fc640 1 -- 172.21.15.40:0/3223440558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c03d420 con 0x7fdc9406fea0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.955+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc94153610 con 0x7fdc9406fea0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.955+0000 7fdc8a7fc640 1 -- 172.21.15.40:0/3223440558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c037040 con 0x7fdc9406fea0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.955+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc94153bd0 con 0x7fdc9406fea0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.956+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc940694f0 con 0x7fdc9406fea0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.956+0000 7fdc8a7fc640 1 -- 172.21.15.40:0/3223440558 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc8c044020 con 0x7fdc9406fea0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.957+0000 7fdc8a7fc640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc7806c920 0x7fdc7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.957+0000 7fdc8a7fc640 1 -- 172.21.15.40:0/3223440558 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fdc8c0abd60 con 0x7fdc9406fea0 2024-04-26T21:36:37.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.957+0000 7fdc993a4640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc7806c920 0x7fdc7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:37.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.957+0000 7fdc993a4640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc7806c920 0x7fdc7806ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdc94151ef0 tx=0x7fdc84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:37.961+0000 7fdc8a7fc640 1 -- 172.21.15.40:0/3223440558 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc8c04b070 con 0x7fdc9406fea0 2024-04-26T21:36:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.184+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc9406de80 con 0x7fdc9406fea0 2024-04-26T21:36:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.185+0000 7fdc8a7fc640 1 -- 172.21.15.40:0/3223440558 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7fdc8c030e20 con 0x7fdc9406fea0 2024-04-26T21:36:38.186 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:38.186 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:26.796847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3403660773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3403660773},{"type":"v1","addr":"172.21.15.40:6835","nonce":3403660773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5364,"qdb_cluster":[5364]},"id":2}]} 2024-04-26T21:36:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc7806c920 msgr2=0x7fdc7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc7806c920 0x7fdc7806ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdc94151ef0 tx=0x7fdc84009040 comp rx=0 tx=0).stop 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc9406fea0 msgr2=0x7fdc94151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc9406fea0 0x7fdc94151180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c035040 tx=0x7fdc8c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 shutdown_connections 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc7806c920 0x7fdc7806ede0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc94156460 0x7fdc940c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc940cba10 0x7fdc940c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 --2- 172.21.15.40:0/3223440558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc9406fea0 0x7fdc94151180 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 >> 172.21.15.40:0/3223440558 conn(0x7fdc940d0be0 msgr2=0x7fdc940b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 shutdown_connections 2024-04-26T21:36:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.187+0000 7fdc9be09640 1 -- 172.21.15.40:0/3223440558 wait complete. 2024-04-26T21:36:38.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-26T21:36:38.326 INFO:teuthology.orchestra.run.smithi040.stdout:{"message":"","return_code":0} 2024-04-26T21:36:38.326 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:36:38.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:36:38.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.427+0000 7f624e0bd640 1 Processor -- start 2024-04-26T21:36:38.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.427+0000 7f624e0bd640 1 -- start start 2024-04-26T21:36:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.427+0000 7f624e0bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6248071e60 0x7f624806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.427+0000 7f624e0bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62480cc480 0x7f6248166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.427+0000 7f624e0bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f624815d7c0 con 0x7f624806fe20 2024-04-26T21:36:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.427+0000 7f624e0bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f624815daa0 con 0x7f6248071e60 2024-04-26T21:36:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.427+0000 7f624e0bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f624815dd80 con 0x7f62480cc480 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f6247fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62480cc480 0x7f6248166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f6246ffd640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f624806fe20 0x7f6248072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39366/0 (socket says 172.21.15.40:39366) 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f6246ffd640 1 -- 172.21.15.40:0/2999984151 learned_addr learned my addr 172.21.15.40:0/2999984151 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f62477fe640 1 --2- 172.21.15.40:0/2999984151 >> v2:172.21.15.184:3300/0 conn(0x7f6248071e60 0x7f624806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f62477fe640 1 -- 172.21.15.40:0/2999984151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62480cc480 msgr2=0x7f6248166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f62477fe640 1 --2- 172.21.15.40:0/2999984151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62480cc480 0x7f6248166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f62477fe640 1 -- 172.21.15.40:0/2999984151 >> v1:172.21.15.184:6789/0 conn(0x7f624806fe20 legacy=0x7f6248072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f62477fe640 1 -- 172.21.15.40:0/2999984151 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624815e060 con 0x7f6248071e60 2024-04-26T21:36:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.428+0000 7f6247fff640 1 --2- 172.21.15.40:0/2999984151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62480cc480 0x7f6248166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:38.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.429+0000 7f62477fe640 1 --2- 172.21.15.40:0/2999984151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6248071e60 0x7f624806f8e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f62380097a0 tx=0x7f623802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a97cd0d5df685f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.429+0000 7f62467fc640 1 -- 172.21.15.40:0/2999984151 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238037070 con 0x7f6248071e60 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.429+0000 7f62467fc640 1 -- 172.21.15.40:0/2999984151 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6238032440 con 0x7f6248071e60 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.429+0000 7f62467fc640 1 -- 172.21.15.40:0/2999984151 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238032810 con 0x7f6248071e60 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.430+0000 7f624e0bd640 1 -- 172.21.15.40:0/2999984151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6248071e60 msgr2=0x7f624806f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.430+0000 7f624e0bd640 1 --2- 172.21.15.40:0/2999984151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6248071e60 0x7f624806f8e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f62380097a0 tx=0x7f623802a180 comp rx=0 tx=0).stop 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.430+0000 7f624e0bd640 1 -- 172.21.15.40:0/2999984151 shutdown_connections 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.430+0000 7f624e0bd640 1 --2- 172.21.15.40:0/2999984151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62480cc480 0x7f6248166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.430+0000 7f624e0bd640 1 --2- 172.21.15.40:0/2999984151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6248071e60 0x7f624806f8e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.430+0000 7f624e0bd640 1 -- 172.21.15.40:0/2999984151 >> 172.21.15.40:0/2999984151 conn(0x7f62480d0be0 msgr2=0x7f62480d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.430+0000 7f624e0bd640 1 -- 172.21.15.40:0/2999984151 shutdown_connections 2024-04-26T21:36:38.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.430+0000 7f624e0bd640 1 -- 172.21.15.40:0/2999984151 wait complete. 2024-04-26T21:36:38.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.431+0000 7f624e0bd640 1 Processor -- start 2024-04-26T21:36:38.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.431+0000 7f624e0bd640 1 -- start start 2024-04-26T21:36:38.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f624e0bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f624806fe20 0x7f62481eed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f624e0bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6248071e60 0x7f62480cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f624e0bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62480cc480 0x7f62481f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f624e0bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f624815d570 con 0x7f624806fe20 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f624e0bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f624815d3f0 con 0x7f62480cc480 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f624e0bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6248072950 con 0x7f6248071e60 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f624806fe20 0x7f62481eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f624806fe20 0x7f62481eed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53080/0 (socket says 172.21.15.40:53080) 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 -- 172.21.15.40:0/1107769175 learned_addr learned my addr 172.21.15.40:0/1107769175 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f6246ffd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6248071e60 0x7f62480cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f6247fff640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62480cc480 0x7f62481f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 -- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6248071e60 msgr2=0x7f62480cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6248071e60 0x7f62480cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 -- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62480cc480 msgr2=0x7f62481f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62480cc480 0x7f62481f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 -- 172.21.15.40:0/1107769175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62481f59a0 con 0x7f624806fe20 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f6247fff640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62480cc480 0x7f62481f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f6246ffd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6248071e60 0x7f62480cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.432+0000 7f62477fe640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f624806fe20 0x7f62481eed70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f62380098c0 tx=0x7f62380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.433+0000 7f622ffff640 1 -- 172.21.15.40:0/1107769175 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238037040 con 0x7f624806fe20 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.433+0000 7f622ffff640 1 -- 172.21.15.40:0/1107769175 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6238030500 con 0x7f624806fe20 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.433+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62481f5c80 con 0x7f624806fe20 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.433+0000 7f622ffff640 1 -- 172.21.15.40:0/1107769175 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380308d0 con 0x7f624806fe20 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.433+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62480c8440 con 0x7f624806fe20 2024-04-26T21:36:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.434+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62480694f0 con 0x7f624806fe20 2024-04-26T21:36:38.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.434+0000 7f622ffff640 1 -- 172.21.15.40:0/1107769175 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6238044020 con 0x7f624806fe20 2024-04-26T21:36:38.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.434+0000 7f622ffff640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f623406c8f0 0x7f623406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.435+0000 7f622ffff640 1 -- 172.21.15.40:0/1107769175 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f62380abf10 con 0x7f624806fe20 2024-04-26T21:36:38.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.435+0000 7f6246ffd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f623406c8f0 0x7f623406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.435+0000 7f6246ffd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f623406c8f0 0x7f623406edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f62481f2ab0 tx=0x7f623c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.439+0000 7f622ffff640 1 -- 172.21.15.40:0/1107769175 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f623807a240 con 0x7f624806fe20 2024-04-26T21:36:38.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.664+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f62481f5e50 con 0x7f624806fe20 2024-04-26T21:36:38.666 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:36:38.665+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:36:38.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.668+0000 7f622ffff640 1 -- 172.21.15.40:0/1107769175 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f6238079b40 con 0x7f624806fe20 2024-04-26T21:36:38.669 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:36:38.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f623406c8f0 msgr2=0x7f623406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f623406c8f0 0x7f623406edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f62481f2ab0 tx=0x7f623c009040 comp rx=0 tx=0).stop 2024-04-26T21:36:38.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f624806fe20 msgr2=0x7f62481eed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f624806fe20 0x7f62481eed70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f62380098c0 tx=0x7f62380046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 shutdown_connections 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f623406c8f0 0x7f623406edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62480cc480 0x7f62481f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6248071e60 0x7f62480cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 --2- 172.21.15.40:0/1107769175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f624806fe20 0x7f62481eed70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 >> 172.21.15.40:0/1107769175 conn(0x7f62480d0be0 msgr2=0x7f624815ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 shutdown_connections 2024-04-26T21:36:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.670+0000 7f624e0bd640 1 -- 172.21.15.40:0/1107769175 wait complete. 2024-04-26T21:36:38.675 INFO:tasks.ceph.mds.a.smithi040.stderr: -4731> 2024-04-26T21:36:28.391+0000 7f2f2ca7a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:36:38.684 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:36:38.706 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:36:38.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.727+0000 7fdee5920f40 1 Processor -- start 2024-04-26T21:36:38.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.727+0000 7fdee5920f40 1 -- start start 2024-04-26T21:36:38.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee5920f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc284fd5b0 0x55bc284cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee5920f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bc285160d0 0x55bc285a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee5920f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285a2550 0x55bc2859b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee5920f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bc284cf370 con 0x55bc285a2550 2024-04-26T21:36:38.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee5920f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bc284fda20 con 0x55bc284fd5b0 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee5920f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bc28515d80 con 0x55bc285160d0 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee3300640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285a2550 0x55bc2859b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee3300640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285a2550 0x55bc2859b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44076/0 (socket says 172.21.15.40:44076) 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee3300640 1 -- 172.21.15.40:0/2015631088 learned_addr learned my addr 172.21.15.40:0/2015631088 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee3300640 1 -- 172.21.15.40:0/2015631088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bc285160d0 msgr2=0x55bc285a2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee3300640 1 --2- 172.21.15.40:0/2015631088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bc285160d0 0x55bc285a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee3300640 1 -- 172.21.15.40:0/2015631088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc284fd5b0 msgr2=0x55bc284cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee3300640 1 --2- 172.21.15.40:0/2015631088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc284fd5b0 0x55bc284cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.728+0000 7fdee3300640 1 -- 172.21.15.40:0/2015631088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc283f8cb0 con 0x55bc285a2550 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee3300640 1 --2- 172.21.15.40:0/2015631088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285a2550 0x55bc2859b5d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fded4005fd0 tx=0x7fded4050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562628f16dfcc946 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee12fc640 1 -- 172.21.15.40:0/2015631088 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4055030 con 0x55bc285a2550 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee12fc640 1 -- 172.21.15.40:0/2015631088 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded4051ba0 con 0x55bc285a2550 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee12fc640 1 -- 172.21.15.40:0/2015631088 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded405b430 con 0x55bc285a2550 2024-04-26T21:36:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 -- 172.21.15.40:0/2015631088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285a2550 msgr2=0x55bc2859b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 --2- 172.21.15.40:0/2015631088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285a2550 0x55bc2859b5d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fded4005fd0 tx=0x7fded4050000 comp rx=0 tx=0).stop 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 -- 172.21.15.40:0/2015631088 shutdown_connections 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 --2- 172.21.15.40:0/2015631088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285a2550 0x55bc2859b5d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 --2- 172.21.15.40:0/2015631088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bc285160d0 0x55bc285a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 --2- 172.21.15.40:0/2015631088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc284fd5b0 0x55bc284cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 -- 172.21.15.40:0/2015631088 >> 172.21.15.40:0/2015631088 conn(0x55bc284ce0e0 msgr2=0x55bc284cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 -- 172.21.15.40:0/2015631088 shutdown_connections 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.729+0000 7fdee5920f40 1 -- 172.21.15.40:0/2015631088 wait complete. 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.730+0000 7fdee5920f40 1 Processor -- start 2024-04-26T21:36:38.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.730+0000 7fdee5920f40 1 -- start start 2024-04-26T21:36:38.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee5920f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bc284fd5b0 0x55bc285915c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee5920f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285160d0 0x55bc285a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee5920f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc285a2550 0x55bc28590c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee5920f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bc285921f0 con 0x55bc285160d0 2024-04-26T21:36:38.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee5920f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bc284d0010 con 0x55bc285a2550 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee5920f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bc285a4b70 con 0x55bc284fd5b0 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee22fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285160d0 0x55bc285a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee22fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285160d0 0x55bc285a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44088/0 (socket says 172.21.15.40:44088) 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee22fe640 1 -- 172.21.15.40:0/3023118833 learned_addr learned my addr 172.21.15.40:0/3023118833 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee2aff640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bc284fd5b0 0x55bc285915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee3300640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc285a2550 0x55bc28590c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee22fe640 1 -- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bc284fd5b0 msgr2=0x55bc285915c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.731+0000 7fdee22fe640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bc284fd5b0 0x55bc285915c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdee22fe640 1 -- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc285a2550 msgr2=0x55bc28590c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdee22fe640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc285a2550 0x55bc28590c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdee22fe640 1 -- 172.21.15.40:0/3023118833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc285927f0 con 0x55bc285160d0 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdee22fe640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285160d0 0x55bc285a16f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00acd0 tx=0x7fdecc00adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdedbfff640 1 -- 172.21.15.40:0/3023118833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc01a070 con 0x55bc285160d0 2024-04-26T21:36:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdedbfff640 1 -- 172.21.15.40:0/3023118833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdecc008aa0 con 0x55bc285160d0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdedbfff640 1 -- 172.21.15.40:0/3023118833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc015420 con 0x55bc285160d0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc28592ad0 con 0x55bc285160d0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.732+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc28593090 con 0x55bc285160d0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.734+0000 7fdedbfff640 1 -- 172.21.15.40:0/3023118833 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdecc01d070 con 0x55bc285160d0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.734+0000 7fdedbfff640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdec406b300 0x7fdec406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.734+0000 7fdedbfff640 1 -- 172.21.15.40:0/3023118833 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fdecc089b00 con 0x55bc285160d0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.734+0000 7fdee5920f40 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bc2863c330 0x55bc2859cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.734+0000 7fdee2aff640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdec406b300 0x7fdec406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.734+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39) -- 0x55bc2859d070 con 0x55bc2863c330 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.735+0000 7fdee3300640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bc2863c330 0x55bc2859cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.735+0000 7fdee2aff640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdec406b300 0x7fdec406d7c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0030a0 tx=0x7fdedc0058c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.735+0000 7fdee3300640 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bc2863c330 0x55bc2859cb30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee3300640 1 -- 172.21.15.40:0/3023118833 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v39'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fded4051d50 con 0x55bc2863c330 2024-04-26T21:36:38.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bc2863c330 msgr2=0x55bc2859cb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bc2863c330 0x55bc2859cb30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdec406b300 msgr2=0x7fdec406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdec406b300 0x7fdec406d7c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0030a0 tx=0x7fdedc0058c0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285160d0 msgr2=0x55bc285a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 --2- 172.21.15.40:0/3023118833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bc285160d0 0x55bc285a16f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00acd0 tx=0x7fdecc00adc0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee3300640 1 -- 172.21.15.40:0/3023118833 reap_dead start 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 shutdown_connections 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 >> 172.21.15.40:0/3023118833 conn(0x55bc284ce0e0 msgr2=0x55bc284cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 shutdown_connections 2024-04-26T21:36:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.739+0000 7fdee5920f40 1 -- 172.21.15.40:0/3023118833 wait complete. 2024-04-26T21:36:38.742 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:36:38.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.886+0000 7f9a2adf7640 1 Processor -- start 2024-04-26T21:36:38.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.886+0000 7f9a2adf7640 1 -- start start 2024-04-26T21:36:38.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a2adf7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9a2415f170 0x7f9a240d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a2adf7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f550 0x7f9a240d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a2adf7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a240d1040 con 0x7f9a240d34b0 2024-04-26T21:36:38.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a2adf7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9a240c9430 con 0x7f9a2415f170 2024-04-26T21:36:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a2adf7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9a240c9680 con 0x7f9a2415f550 2024-04-26T21:36:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a28b93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9a2415f170 0x7f9a240d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a23fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f550 0x7f9a240d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a29394640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f9a240d34b0 0x7f9a240cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:44656/0 (socket says 172.21.15.40:44656) 2024-04-26T21:36:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.887+0000 7f9a29394640 1 -- 172.21.15.40:0/1274695359 learned_addr learned my addr 172.21.15.40:0/1274695359 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.888+0000 7f9a28b93640 1 -- 172.21.15.40:0/1274695359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f550 msgr2=0x7f9a240d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.888+0000 7f9a28b93640 1 --2- 172.21.15.40:0/1274695359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f550 0x7f9a240d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.888+0000 7f9a28b93640 1 -- 172.21.15.40:0/1274695359 >> v1:172.21.15.184:6789/0 conn(0x7f9a240d34b0 legacy=0x7f9a240cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.888+0000 7f9a28b93640 1 -- 172.21.15.40:0/1274695359 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a240c9960 con 0x7f9a2415f170 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.888+0000 7f9a28b93640 1 --2- 172.21.15.40:0/1274695359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2415f170 0x7f9a240d5a80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9a180097a0 tx=0x7f9a1802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f0d11f349f8850b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a237fe640 1 -- 172.21.15.40:0/1274695359 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18037070 con 0x7f9a2415f170 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a237fe640 1 -- 172.21.15.40:0/1274695359 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a18032440 con 0x7f9a2415f170 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a237fe640 1 -- 172.21.15.40:0/1274695359 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18032810 con 0x7f9a2415f170 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a2adf7640 1 -- 172.21.15.40:0/1274695359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2415f170 msgr2=0x7f9a240d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/1274695359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2415f170 0x7f9a240d5a80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9a180097a0 tx=0x7f9a1802a180 comp rx=0 tx=0).stop 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a2adf7640 1 -- 172.21.15.40:0/1274695359 shutdown_connections 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/1274695359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f550 0x7f9a240d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/1274695359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2415f170 0x7f9a240d5a80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a2adf7640 1 -- 172.21.15.40:0/1274695359 >> 172.21.15.40:0/1274695359 conn(0x7f9a240d4f60 msgr2=0x7f9a240d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a2adf7640 1 -- 172.21.15.40:0/1274695359 shutdown_connections 2024-04-26T21:36:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.889+0000 7f9a2adf7640 1 -- 172.21.15.40:0/1274695359 wait complete. 2024-04-26T21:36:38.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.890+0000 7f9a2adf7640 1 Processor -- start 2024-04-26T21:36:38.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.891+0000 7f9a2adf7640 1 -- start start 2024-04-26T21:36:38.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.891+0000 7f9a2adf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a240d34b0 0x7f9a24159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.891+0000 7f9a2adf7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f170 0x7f9a241522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.891+0000 7f9a2adf7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a240b9b90 0x7f9a240ba020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.891+0000 7f9a2adf7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9a240cb5d0 con 0x7f9a240b9b90 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.891+0000 7f9a2adf7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a240cb450 con 0x7f9a240d34b0 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.891+0000 7f9a2adf7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9a240cb750 con 0x7f9a2415f170 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a240b9b90 0x7f9a240ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a28b93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a240d34b0 0x7f9a24159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a240b9b90 0x7f9a240ba020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44092/0 (socket says 172.21.15.40:44092) 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 -- 172.21.15.40:0/318839673 learned_addr learned my addr 172.21.15.40:0/318839673 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a23fff640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f170 0x7f9a241522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 -- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f170 msgr2=0x7f9a241522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f170 0x7f9a241522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 -- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a240d34b0 msgr2=0x7f9a24159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a240d34b0 0x7f9a24159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 -- 172.21.15.40:0/318839673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a24158df0 con 0x7f9a240b9b90 2024-04-26T21:36:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a29394640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a240b9b90 0x7f9a240ba020 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00caa0 tx=0x7f9a1c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a217fa640 1 -- 172.21.15.40:0/318839673 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c017070 con 0x7f9a240b9b90 2024-04-26T21:36:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a241590d0 con 0x7f9a240b9b90 2024-04-26T21:36:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a217fa640 1 -- 172.21.15.40:0/318839673 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1c00ac00 con 0x7f9a240b9b90 2024-04-26T21:36:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a217fa640 1 -- 172.21.15.40:0/318839673 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c012620 con 0x7f9a240b9b90 2024-04-26T21:36:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.892+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a24159690 con 0x7f9a240b9b90 2024-04-26T21:36:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.893+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a240694f0 con 0x7f9a240b9b90 2024-04-26T21:36:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.894+0000 7f9a217fa640 1 -- 172.21.15.40:0/318839673 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a1c01a070 con 0x7f9a240b9b90 2024-04-26T21:36:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.894+0000 7f9a217fa640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a0806c8f0 0x7f9a0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.894+0000 7f9a217fa640 1 -- 172.21.15.40:0/318839673 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f9a1c08ba70 con 0x7f9a240b9b90 2024-04-26T21:36:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.895+0000 7f9a28b93640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a0806c8f0 0x7f9a0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.895+0000 7f9a28b93640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a0806c8f0 0x7f9a0806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9a18034000 tx=0x7f9a1802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:38.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:38.899+0000 7f9a217fa640 1 -- 172.21.15.40:0/318839673 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a240d1040 con 0x7f9a240b9b90 2024-04-26T21:36:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.129+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9a241527e0 con 0x7f9a240b9b90 2024-04-26T21:36:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.671+0000 7f9a217fa640 1 -- 172.21.15.40:0/318839673 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v19) ==== 162+0+0 (secure 0 0 0) 0x7f9a1c0595f0 con 0x7f9a240b9b90 2024-04-26T21:36:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.673+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a0806c8f0 msgr2=0x7f9a0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.673+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a0806c8f0 0x7f9a0806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9a18034000 tx=0x7f9a1802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.673+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a240b9b90 msgr2=0x7f9a240ba020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.673+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a240b9b90 0x7f9a240ba020 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00caa0 tx=0x7f9a1c00cf60 comp rx=0 tx=0).stop 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.674+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 shutdown_connections 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.674+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a0806c8f0 0x7f9a0806edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.674+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a240b9b90 0x7f9a240ba020 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.674+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a2415f170 0x7f9a241522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.674+0000 7f9a2adf7640 1 --2- 172.21.15.40:0/318839673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a240d34b0 0x7f9a24159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.674+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 >> 172.21.15.40:0/318839673 conn(0x7f9a240d4f60 msgr2=0x7f9a2406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.674+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 shutdown_connections 2024-04-26T21:36:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.674+0000 7f9a2adf7640 1 -- 172.21.15.40:0/318839673 wait complete. 2024-04-26T21:36:39.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:39.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.792+0000 7fdc383d4640 1 Processor -- start 2024-04-26T21:36:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.792+0000 7fdc383d4640 1 -- start start 2024-04-26T21:36:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.793+0000 7fdc383d4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc3015f3a0 0x7fdc300d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.793+0000 7fdc383d4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc300d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.793+0000 7fdc383d4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc300d25a0 con 0x7fdc300d2130 2024-04-26T21:36:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.793+0000 7fdc383d4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdc300d0e20 con 0x7fdc3015f3a0 2024-04-26T21:36:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.793+0000 7fdc383d4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc300b1d70 con 0x7fdc300d1770 2024-04-26T21:36:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc36170640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc3015f3a0 0x7fdc300d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc3596f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc300d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc36971640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fdc300d2130 0x7fdc300d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:44660/0 (socket says 172.21.15.40:44660) 2024-04-26T21:36:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc36971640 1 -- 172.21.15.40:0/987280584 learned_addr learned my addr 172.21.15.40:0/987280584 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc3596f640 1 -- 172.21.15.40:0/987280584 >> v2:172.21.15.184:3300/0 conn(0x7fdc3015f3a0 msgr2=0x7fdc300d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc3596f640 1 --2- 172.21.15.40:0/987280584 >> v2:172.21.15.184:3300/0 conn(0x7fdc3015f3a0 0x7fdc300d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc3596f640 1 -- 172.21.15.40:0/987280584 >> v1:172.21.15.184:6789/0 conn(0x7fdc300d2130 legacy=0x7fdc300d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc3596f640 1 -- 172.21.15.40:0/987280584 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc300b2050 con 0x7fdc300d1770 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.794+0000 7fdc36170640 1 --2- 172.21.15.40:0/987280584 >> v2:172.21.15.184:3300/0 conn(0x7fdc3015f3a0 0x7fdc300d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.795+0000 7fdc3596f640 1 --2- 172.21.15.40:0/987280584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc300d1bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc240097a0 tx=0x7fdc2402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba0f773bb2a800 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.795+0000 7fdc3516e640 1 -- 172.21.15.40:0/987280584 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24037070 con 0x7fdc300d1770 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.795+0000 7fdc3516e640 1 -- 172.21.15.40:0/987280584 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc24032440 con 0x7fdc300d1770 2024-04-26T21:36:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.795+0000 7fdc3516e640 1 -- 172.21.15.40:0/987280584 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24032810 con 0x7fdc300d1770 2024-04-26T21:36:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.795+0000 7fdc383d4640 1 -- 172.21.15.40:0/987280584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 msgr2=0x7fdc300d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.795+0000 7fdc383d4640 1 --2- 172.21.15.40:0/987280584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc300d1bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc240097a0 tx=0x7fdc2402a180 comp rx=0 tx=0).stop 2024-04-26T21:36:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.796+0000 7fdc383d4640 1 -- 172.21.15.40:0/987280584 shutdown_connections 2024-04-26T21:36:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.796+0000 7fdc383d4640 1 --2- 172.21.15.40:0/987280584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc300d1bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.796+0000 7fdc383d4640 1 --2- 172.21.15.40:0/987280584 >> v2:172.21.15.184:3300/0 conn(0x7fdc3015f3a0 0x7fdc300d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.796+0000 7fdc383d4640 1 -- 172.21.15.40:0/987280584 >> 172.21.15.40:0/987280584 conn(0x7fdc300d4f90 msgr2=0x7fdc300d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.796+0000 7fdc383d4640 1 -- 172.21.15.40:0/987280584 shutdown_connections 2024-04-26T21:36:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.796+0000 7fdc383d4640 1 -- 172.21.15.40:0/987280584 wait complete. 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.797+0000 7fdc383d4640 1 Processor -- start 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.797+0000 7fdc383d4640 1 -- start start 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.797+0000 7fdc383d4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc301f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.797+0000 7fdc383d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc300d2130 0x7fdc30152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.797+0000 7fdc383d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3015f3a0 0x7fdc300b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.797+0000 7fdc383d4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3015f900 con 0x7fdc300d2130 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.797+0000 7fdc383d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3015f780 con 0x7fdc3015f3a0 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.797+0000 7fdc383d4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc300b3860 con 0x7fdc300d1770 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc300d2130 0x7fdc30152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc300d2130 0x7fdc30152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44104/0 (socket says 172.21.15.40:44104) 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 -- 172.21.15.40:0/2607404531 learned_addr learned my addr 172.21.15.40:0/2607404531 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc36170640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc301f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc36971640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3015f3a0 0x7fdc300b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 -- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 msgr2=0x7fdc301f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc301f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 -- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3015f3a0 msgr2=0x7fdc300b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3015f3a0 0x7fdc300b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 -- 172.21.15.40:0/2607404531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc300ba480 con 0x7fdc300d2130 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc3596f640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc300d2130 0x7fdc30152160 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098c0 tx=0x7fdc240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc2effd640 1 -- 172.21.15.40:0/2607404531 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24037040 con 0x7fdc300d2130 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc2effd640 1 -- 172.21.15.40:0/2607404531 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc24030500 con 0x7fdc300d2130 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc2effd640 1 -- 172.21.15.40:0/2607404531 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc240308d0 con 0x7fdc300d2130 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc300b9110 con 0x7fdc300d2130 2024-04-26T21:36:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.798+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc300b96c0 con 0x7fdc300d2130 2024-04-26T21:36:39.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.799+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc300694f0 con 0x7fdc300d2130 2024-04-26T21:36:39.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.800+0000 7fdc2effd640 1 -- 172.21.15.40:0/2607404531 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc24044020 con 0x7fdc300d2130 2024-04-26T21:36:39.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.800+0000 7fdc2effd640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc1c06c8f0 0x7fdc1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:39.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.800+0000 7fdc2effd640 1 -- 172.21.15.40:0/2607404531 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fdc240ac030 con 0x7fdc300d2130 2024-04-26T21:36:39.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.801+0000 7fdc36170640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc1c06c8f0 0x7fdc1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:39.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.801+0000 7fdc36170640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc1c06c8f0 0x7fdc1c06edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc20007440 tx=0x7fdc2000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:39.805+0000 7fdc2effd640 1 -- 172.21.15.40:0/2607404531 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc2407a280 con 0x7fdc300d2130 2024-04-26T21:36:40.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.029+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc300b6890 con 0x7fdc300d2130 2024-04-26T21:36:40.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.029+0000 7fdc2effd640 1 -- 172.21.15.40:0/2607404531 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5178 (secure 0 0 0) 0x7fdc24079b80 con 0x7fdc300d2130 2024-04-26T21:36:40.030 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:39.674304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T21:36:40.031 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc1c06c8f0 msgr2=0x7fdc1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc1c06c8f0 0x7fdc1c06edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc20007440 tx=0x7fdc2000b040 comp rx=0 tx=0).stop 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc300d2130 msgr2=0x7fdc30152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc300d2130 0x7fdc30152160 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098c0 tx=0x7fdc240046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 shutdown_connections 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc1c06c8f0 0x7fdc1c06edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3015f3a0 0x7fdc300b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc300d2130 0x7fdc30152160 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 --2- 172.21.15.40:0/2607404531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc300d1770 0x7fdc301f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 >> 172.21.15.40:0/2607404531 conn(0x7fdc300d4f90 msgr2=0x7fdc300d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 shutdown_connections 2024-04-26T21:36:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:40.031+0000 7fdc383d4640 1 -- 172.21.15.40:0/2607404531 wait complete. 2024-04-26T21:36:40.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:36:40.106 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:36:40.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:36:23.780134+0000', 'modified': '2024-04-26T21:36:39.674304+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5619': {'gid': 5619, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6835/1366266306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 1366266306}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1366266306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:36:40.446 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5619:up:replay 2024-04-26T21:36:41.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:41.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.560+0000 7f2841a25640 1 Processor -- start 2024-04-26T21:36:41.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.560+0000 7f2841a25640 1 -- start start 2024-04-26T21:36:41.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f2841a25640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f283c162810 0x7f283c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:41.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f2841a25640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f283c0b2340 0x7f283c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f2841a25640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f283c0d10d0 con 0x7f283c0cd930 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f2841a25640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f283c0d0750 con 0x7f283c162810 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f2841a25640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f283c0d16a0 con 0x7f283c0b2340 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f283affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f283c162810 0x7f283c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f283affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f283c162810 0x7f283c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44110/0 (socket says 172.21.15.40:44110) 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f283affd640 1 -- 172.21.15.40:0/1215264776 learned_addr learned my addr 172.21.15.40:0/1215264776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f283a7fc640 1 --2- 172.21.15.40:0/1215264776 >> v2:172.21.15.184:3300/0 conn(0x7f283c0b2340 0x7f283c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.561+0000 7f283affd640 1 -- 172.21.15.40:0/1215264776 >> v2:172.21.15.184:3300/0 conn(0x7f283c0b2340 msgr2=0x7f283c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.562+0000 7f283affd640 1 --2- 172.21.15.40:0/1215264776 >> v2:172.21.15.184:3300/0 conn(0x7f283c0b2340 0x7f283c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.562+0000 7f283affd640 1 -- 172.21.15.40:0/1215264776 >> v1:172.21.15.184:6789/0 conn(0x7f283c0cd930 legacy=0x7f283c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.562+0000 7f283affd640 1 -- 172.21.15.40:0/1215264776 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c0d1370 con 0x7f283c162810 2024-04-26T21:36:41.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.562+0000 7f283affd640 1 --2- 172.21.15.40:0/1215264776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c070350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f283c0cd0a0 tx=0x7f282c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60279c329062a8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:41.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.562+0000 7f2839ffb640 1 -- 172.21.15.40:0/1215264776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c037070 con 0x7f283c162810 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.562+0000 7f2839ffb640 1 -- 172.21.15.40:0/1215264776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c032440 con 0x7f283c162810 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.562+0000 7f2839ffb640 1 -- 172.21.15.40:0/1215264776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c032810 con 0x7f283c162810 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.563+0000 7f2841a25640 1 -- 172.21.15.40:0/1215264776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 msgr2=0x7f283c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.563+0000 7f2841a25640 1 --2- 172.21.15.40:0/1215264776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c070350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f283c0cd0a0 tx=0x7f282c02a270 comp rx=0 tx=0).stop 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.563+0000 7f2841a25640 1 -- 172.21.15.40:0/1215264776 shutdown_connections 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.563+0000 7f2841a25640 1 --2- 172.21.15.40:0/1215264776 >> v2:172.21.15.184:3300/0 conn(0x7f283c0b2340 0x7f283c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.563+0000 7f2841a25640 1 --2- 172.21.15.40:0/1215264776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c070350 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.563+0000 7f2841a25640 1 -- 172.21.15.40:0/1215264776 >> 172.21.15.40:0/1215264776 conn(0x7f283c0d2da0 msgr2=0x7f283c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.563+0000 7f2841a25640 1 -- 172.21.15.40:0/1215264776 shutdown_connections 2024-04-26T21:36:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.563+0000 7f2841a25640 1 -- 172.21.15.40:0/1215264776 wait complete. 2024-04-26T21:36:41.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.564+0000 7f2841a25640 1 Processor -- start 2024-04-26T21:36:41.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.564+0000 7f2841a25640 1 -- start start 2024-04-26T21:36:41.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f2841a25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0b2340 0x7f283c1f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:41.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f2841a25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c0cd930 0x7f283c1f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:41.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f2841a25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f2841a25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f283c070890 con 0x7f283c162810 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f2841a25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f283c0d0470 con 0x7f283c0cd930 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f2841a25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f283c0d2240 con 0x7f283c0b2340 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f283b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f283b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c1f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44116/0 (socket says 172.21.15.40:44116) 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f283b7fe640 1 -- 172.21.15.40:0/3863077624 learned_addr learned my addr 172.21.15.40:0/3863077624 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f283affd640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0b2340 0x7f283c1f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.565+0000 7f283a7fc640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c0cd930 0x7f283c1f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f283b7fe640 1 -- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0b2340 msgr2=0x7f283c1f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f283b7fe640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0b2340 0x7f283c1f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f283b7fe640 1 -- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c0cd930 msgr2=0x7f283c1f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f283b7fe640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c0cd930 0x7f283c1f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f283b7fe640 1 -- 172.21.15.40:0/3863077624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c1f54e0 con 0x7f283c162810 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f283b7fe640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c1f4d70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f283400c760 tx=0x7f283400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f2823fff640 1 -- 172.21.15.40:0/3863077624 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834003b70 con 0x7f283c162810 2024-04-26T21:36:41.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f2823fff640 1 -- 172.21.15.40:0/3863077624 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2834003d10 con 0x7f283c162810 2024-04-26T21:36:41.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f2823fff640 1 -- 172.21.15.40:0/3863077624 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283400aa90 con 0x7f283c162810 2024-04-26T21:36:41.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c1f57c0 con 0x7f283c162810 2024-04-26T21:36:41.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.566+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c14c4d0 con 0x7f283c162810 2024-04-26T21:36:41.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.567+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c149ac0 con 0x7f283c162810 2024-04-26T21:36:41.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.567+0000 7f2823fff640 1 -- 172.21.15.40:0/3863077624 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f283400ac30 con 0x7f283c162810 2024-04-26T21:36:41.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.568+0000 7f2823fff640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282806c8f0 0x7f282806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:41.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.568+0000 7f2823fff640 1 -- 172.21.15.40:0/3863077624 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f283408bc30 con 0x7f283c162810 2024-04-26T21:36:41.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.568+0000 7f283affd640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282806c8f0 0x7f282806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:41.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.569+0000 7f283affd640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282806c8f0 0x7f282806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f282c034000 tx=0x7f282c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:41.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.572+0000 7f2823fff640 1 -- 172.21.15.40:0/3863077624 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f283c149ac0 con 0x7f283c162810 2024-04-26T21:36:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.797+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f283c0694f0 con 0x7f283c162810 2024-04-26T21:36:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.797+0000 7f2823fff640 1 -- 172.21.15.40:0/3863077624 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5178 (secure 0 0 0) 0x7f2834059e80 con 0x7f283c162810 2024-04-26T21:36:41.798 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:41.798 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:40.687072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T21:36:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-04-26T21:36:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282806c8f0 msgr2=0x7f282806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282806c8f0 0x7f282806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f282c034000 tx=0x7f282c02e000 comp rx=0 tx=0).stop 2024-04-26T21:36:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 msgr2=0x7f283c1f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c1f4d70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f283400c760 tx=0x7f283400cc20 comp rx=0 tx=0).stop 2024-04-26T21:36:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 shutdown_connections 2024-04-26T21:36:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282806c8f0 0x7f282806edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f283c162810 0x7f283c1f4d70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c0cd930 0x7f283c1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 --2- 172.21.15.40:0/3863077624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0b2340 0x7f283c1f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 >> 172.21.15.40:0/3863077624 conn(0x7f283c0d2da0 msgr2=0x7f283c070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 shutdown_connections 2024-04-26T21:36:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:41.799+0000 7f2841a25640 1 -- 172.21.15.40:0/3863077624 wait complete. 2024-04-26T21:36:42.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:36:23.780134+0000', 'modified': '2024-04-26T21:36:40.687072+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5619': {'gid': 5619, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.40:6835/1366266306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 1366266306}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1366266306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:36:42.214 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5619:up:rejoin 2024-04-26T21:36:43.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:43.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.327+0000 7fc9e4bbf640 1 Processor -- start 2024-04-26T21:36:43.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.327+0000 7fc9e4bbf640 1 -- start start 2024-04-26T21:36:43.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.328+0000 7fc9e4bbf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9e00d06f0 0x7fc9e00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:43.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.328+0000 7fc9e4bbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e00b2850 0x7fc9e00b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:43.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.328+0000 7fc9e4bbf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e00d0ba0 con 0x7fc9e015f3a0 2024-04-26T21:36:43.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.328+0000 7fc9e4bbf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e00d0e20 con 0x7fc9e00d06f0 2024-04-26T21:36:43.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.328+0000 7fc9e4bbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00b1d70 con 0x7fc9e00b2850 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ddd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9e00d06f0 0x7fc9e00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ddd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9e00d06f0 0x7fc9e00d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44128/0 (socket says 172.21.15.40:44128) 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ddd9b640 1 -- 172.21.15.40:0/1856981015 learned_addr learned my addr 172.21.15.40:0/1856981015 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ded9d640 1 --2- 172.21.15.40:0/1856981015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e00b2850 0x7fc9e00b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ddd9b640 1 -- 172.21.15.40:0/1856981015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e00b2850 msgr2=0x7fc9e00b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ddd9b640 1 --2- 172.21.15.40:0/1856981015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e00b2850 0x7fc9e00b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ddd9b640 1 -- 172.21.15.40:0/1856981015 >> v1:172.21.15.184:6789/0 conn(0x7fc9e015f3a0 legacy=0x7fc9e00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ddd9b640 1 -- 172.21.15.40:0/1856981015 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00b2050 con 0x7fc9e00d06f0 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.329+0000 7fc9ded9d640 1 --2- 172.21.15.40:0/1856981015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e00b2850 0x7fc9e00b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9ddd9b640 1 --2- 172.21.15.40:0/1856981015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e00d1e80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009870 tx=0x7fc9c802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2215c263fcc4032b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:43.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9dd59a640 1 -- 172.21.15.40:0/1856981015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8037070 con 0x7fc9e00d06f0 2024-04-26T21:36:43.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9dd59a640 1 -- 172.21.15.40:0/1856981015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c8032440 con 0x7fc9e00d06f0 2024-04-26T21:36:43.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9dd59a640 1 -- 172.21.15.40:0/1856981015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8032810 con 0x7fc9e00d06f0 2024-04-26T21:36:43.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/1856981015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 msgr2=0x7fc9e00d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:43.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/1856981015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e00d1e80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009870 tx=0x7fc9c802a210 comp rx=0 tx=0).stop 2024-04-26T21:36:43.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/1856981015 shutdown_connections 2024-04-26T21:36:43.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/1856981015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e00b2850 0x7fc9e00b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/1856981015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e00d1e80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/1856981015 >> 172.21.15.40:0/1856981015 conn(0x7fc9e00d4f90 msgr2=0x7fc9e00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:43.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/1856981015 shutdown_connections 2024-04-26T21:36:43.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.330+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/1856981015 wait complete. 2024-04-26T21:36:43.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.332+0000 7fc9e4bbf640 1 Processor -- start 2024-04-26T21:36:43.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.332+0000 7fc9e4bbf640 1 -- start start 2024-04-26T21:36:43.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9e4bbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e00b2850 0x7fc9e01503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9e4bbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e0160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9e4bbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e015f3a0 0x7fc9e0161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9e4bbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e015f900 con 0x7fc9e00d06f0 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9e4bbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e015f780 con 0x7fc9e00b2850 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9e4bbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00d23c0 con 0x7fc9e015f3a0 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e0160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e0160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44140/0 (socket says 172.21.15.40:44140) 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 -- 172.21.15.40:0/3469266059 learned_addr learned my addr 172.21.15.40:0/3469266059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ded9d640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e015f3a0 0x7fc9e0161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9de59c640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e00b2850 0x7fc9e01503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 -- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e015f3a0 msgr2=0x7fc9e0161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e015f3a0 0x7fc9e0161d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 -- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e00b2850 msgr2=0x7fc9e01503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e00b2850 0x7fc9e01503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 -- 172.21.15.40:0/3469266059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e0150920 con 0x7fc9e00d06f0 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ded9d640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e015f3a0 0x7fc9e0161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9de59c640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e00b2850 0x7fc9e01503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.333+0000 7fc9ddd9b640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e0160490 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009870 tx=0x7fc9c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.334+0000 7fc9d6ffd640 1 -- 172.21.15.40:0/3469266059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8037040 con 0x7fc9e00d06f0 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.334+0000 7fc9d6ffd640 1 -- 172.21.15.40:0/3469266059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c8042070 con 0x7fc9e00d06f0 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.334+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e0160ce0 con 0x7fc9e00d06f0 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.334+0000 7fc9d6ffd640 1 -- 172.21.15.40:0/3469266059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8030bb0 con 0x7fc9e00d06f0 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.334+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e01612a0 con 0x7fc9e00d06f0 2024-04-26T21:36:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.335+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e00694f0 con 0x7fc9e00d06f0 2024-04-26T21:36:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.335+0000 7fc9d6ffd640 1 -- 172.21.15.40:0/3469266059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9c8047460 con 0x7fc9e00d06f0 2024-04-26T21:36:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.336+0000 7fc9d6ffd640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc9c006c8f0 0x7fc9c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.336+0000 7fc9d6ffd640 1 -- 172.21.15.40:0/3469266059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fc9c80ac870 con 0x7fc9e00d06f0 2024-04-26T21:36:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.336+0000 7fc9de59c640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc9c006c8f0 0x7fc9c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.336+0000 7fc9de59c640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc9c006c8f0 0x7fc9c006edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8003eb0 tx=0x7fc9d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.340+0000 7fc9d6ffd640 1 -- 172.21.15.40:0/3469266059 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc9e0150920 con 0x7fc9e00d06f0 2024-04-26T21:36:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.564+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9e00d1fe0 con 0x7fc9e00d06f0 2024-04-26T21:36:43.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.565+0000 7fc9d6ffd640 1 -- 172.21.15.40:0/3469266059 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5185 (secure 0 0 0) 0x7fc9e00d0ba0 con 0x7fc9e00d06f0 2024-04-26T21:36:43.566 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:43.566 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:42.691649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5619,"qdb_cluster":[5619]},"id":2}]} 2024-04-26T21:36:43.566 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 23 2024-04-26T21:36:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc9c006c8f0 msgr2=0x7fc9c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc9c006c8f0 0x7fc9c006edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8003eb0 tx=0x7fc9d8009040 comp rx=0 tx=0).stop 2024-04-26T21:36:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 msgr2=0x7fc9e0160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e0160490 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009870 tx=0x7fc9c80046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 shutdown_connections 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc9c006c8f0 0x7fc9c006edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9e015f3a0 0x7fc9e0161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9e00d06f0 0x7fc9e0160490 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 --2- 172.21.15.40:0/3469266059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9e00b2850 0x7fc9e01503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 >> 172.21.15.40:0/3469266059 conn(0x7fc9e00d4f90 msgr2=0x7fc9e006d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 shutdown_connections 2024-04-26T21:36:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:43.567+0000 7fc9e4bbf640 1 -- 172.21.15.40:0/3469266059 wait complete. 2024-04-26T21:36:43.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:36:23.780134+0000', 'modified': '2024-04-26T21:36:42.691649+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5619': {'gid': 5619, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6835/1366266306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 1366266306}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1366266306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5619, 'qdb_cluster': [5619]} 2024-04-26T21:36:43.982 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:36:43.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:36:44.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.086+0000 7fea21b1b640 1 Processor -- start 2024-04-26T21:36:44.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.086+0000 7fea21b1b640 1 -- start start 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.087+0000 7fea21b1b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea1c0b4d40 0x7fea1c0b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.087+0000 7fea21b1b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fea1c0b6c50 0x7fea1c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.087+0000 7fea21b1b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea1c072fb0 con 0x7fea1c0b0b70 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.087+0000 7fea21b1b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fea1c073bc0 con 0x7fea1c0b4d40 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.087+0000 7fea21b1b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fea1c073290 con 0x7fea1c0b6c50 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.087+0000 7fea1affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea1c0b4d40 0x7fea1c0b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.087+0000 7fea1b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fea1c0b0b70 0x7fea1c0b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33452/0 (socket says 172.21.15.40:33452) 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.087+0000 7fea1b7fe640 1 -- 172.21.15.40:0/699417077 learned_addr learned my addr 172.21.15.40:0/699417077 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.088+0000 7fea1bfff640 1 --2- 172.21.15.40:0/699417077 >> v2:172.21.15.184:3300/0 conn(0x7fea1c0b6c50 0x7fea1c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.088+0000 7fea1affd640 1 -- 172.21.15.40:0/699417077 >> v2:172.21.15.184:3300/0 conn(0x7fea1c0b6c50 msgr2=0x7fea1c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.088+0000 7fea1affd640 1 --2- 172.21.15.40:0/699417077 >> v2:172.21.15.184:3300/0 conn(0x7fea1c0b6c50 0x7fea1c0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.088+0000 7fea1affd640 1 -- 172.21.15.40:0/699417077 >> v1:172.21.15.40:6789/0 conn(0x7fea1c0b0b70 legacy=0x7fea1c0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.088+0000 7fea1affd640 1 -- 172.21.15.40:0/699417077 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c073570 con 0x7fea1c0b4d40 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.088+0000 7fea1affd640 1 --2- 172.21.15.40:0/699417077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b4d40 0x7fea1c0b6710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0b1750 tx=0x7fea1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c55a7104541840d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea1a7fc640 1 -- 172.21.15.40:0/699417077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10037070 con 0x7fea1c0b4d40 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea1a7fc640 1 -- 172.21.15.40:0/699417077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea10032440 con 0x7fea1c0b4d40 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea1a7fc640 1 -- 172.21.15.40:0/699417077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10032810 con 0x7fea1c0b4d40 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea21b1b640 1 -- 172.21.15.40:0/699417077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b4d40 msgr2=0x7fea1c0b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea21b1b640 1 --2- 172.21.15.40:0/699417077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b4d40 0x7fea1c0b6710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0b1750 tx=0x7fea1002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea21b1b640 1 -- 172.21.15.40:0/699417077 shutdown_connections 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea21b1b640 1 --2- 172.21.15.40:0/699417077 >> v2:172.21.15.184:3300/0 conn(0x7fea1c0b6c50 0x7fea1c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea21b1b640 1 --2- 172.21.15.40:0/699417077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b4d40 0x7fea1c0b6710 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea21b1b640 1 -- 172.21.15.40:0/699417077 >> 172.21.15.40:0/699417077 conn(0x7fea1c0b39b0 msgr2=0x7fea1c0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea21b1b640 1 -- 172.21.15.40:0/699417077 shutdown_connections 2024-04-26T21:36:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.089+0000 7fea21b1b640 1 -- 172.21.15.40:0/699417077 wait complete. 2024-04-26T21:36:44.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.090+0000 7fea21b1b640 1 Processor -- start 2024-04-26T21:36:44.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea21b1b640 1 -- start start 2024-04-26T21:36:44.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea21b1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b0b70 0x7fea1c0d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea21b1b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea1c0b4d40 0x7fea1c14d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea21b1b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1c0b6c50 0x7fea1c0afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea21b1b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c1604e0 con 0x7fea1c0b0b70 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea21b1b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c160360 con 0x7fea1c0b4d40 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea21b1b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fea1c0af2d0 con 0x7fea1c0b6c50 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea1b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b0b70 0x7fea1c0d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea1b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b0b70 0x7fea1c0d0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44164/0 (socket says 172.21.15.40:44164) 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.091+0000 7fea1b7fe640 1 -- 172.21.15.40:0/1033398195 learned_addr learned my addr 172.21.15.40:0/1033398195 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1affd640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea1c0b4d40 0x7fea1c14d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1bfff640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1c0b6c50 0x7fea1c0afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1b7fe640 1 -- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1c0b6c50 msgr2=0x7fea1c0afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1c0b6c50 0x7fea1c0afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1b7fe640 1 -- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea1c0b4d40 msgr2=0x7fea1c14d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea1c0b4d40 0x7fea1c14d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1b7fe640 1 -- 172.21.15.40:0/1033398195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c0d1290 con 0x7fea1c0b0b70 2024-04-26T21:36:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1bfff640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1c0b6c50 0x7fea1c0afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b0b70 0x7fea1c0d0d50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c400 tx=0x7fea0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea03fff640 1 -- 172.21.15.40:0/1033398195 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c017070 con 0x7fea1c0b0b70 2024-04-26T21:36:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea03fff640 1 -- 172.21.15.40:0/1033398195 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c005ce0 con 0x7fea1c0b0b70 2024-04-26T21:36:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea21b1b640 1 -- 172.21.15.40:0/1033398195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1c0d1550 con 0x7fea1c0b0b70 2024-04-26T21:36:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea03fff640 1 -- 172.21.15.40:0/1033398195 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0072f0 con 0x7fea1c0b0b70 2024-04-26T21:36:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.092+0000 7fea21b1b640 1 -- 172.21.15.40:0/1033398195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea1c0d1b10 con 0x7fea1c0b0b70 2024-04-26T21:36:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.094+0000 7fea03fff640 1 -- 172.21.15.40:0/1033398195 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea0c01a070 con 0x7fea1c0b0b70 2024-04-26T21:36:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.094+0000 7fea03fff640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea0806c920 0x7fea0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.094+0000 7fea03fff640 1 -- 172.21.15.40:0/1033398195 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fea0c08b130 con 0x7fea1c0b0b70 2024-04-26T21:36:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.095+0000 7fea1affd640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea0806c920 0x7fea0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.095+0000 7fea24832740 1 Processor -- start 2024-04-26T21:36:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.095+0000 7fea24832740 1 -- start start 2024-04-26T21:36:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.095+0000 7fea1affd640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea0806c920 0x7fea0806ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea10034000 tx=0x7fea1002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea24832740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac69440 0x55da7ac69820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea24832740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac86ce0 0x55da7ac87230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea24832740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac3f120 0x55da7ac853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea24832740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55da7aba3740 con 0x55da7ac86ce0 2024-04-26T21:36:44.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea24832740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55da7ab3f3a0 con 0x55da7ac69440 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea24832740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55da7ab9f700 con 0x55da7ac3f120 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea1affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac86ce0 0x55da7ac87230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea1affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac86ce0 0x55da7ac87230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44174/0 (socket says 172.21.15.40:44174) 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.096+0000 7fea1affd640 1 -- 172.21.15.40:0/3325966556 learned_addr learned my addr 172.21.15.40:0/3325966556 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea1bfff640 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac3f120 0x55da7ac853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac69440 0x55da7ac69820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea1affd640 1 -- 172.21.15.40:0/3325966556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac3f120 msgr2=0x55da7ac853a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea1affd640 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac3f120 0x55da7ac853a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea1affd640 1 -- 172.21.15.40:0/3325966556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac69440 msgr2=0x55da7ac69820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea1affd640 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac69440 0x55da7ac69820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea1affd640 1 -- 172.21.15.40:0/3325966556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da7ab5b750 con 0x55da7ac86ce0 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea1affd640 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac86ce0 0x55da7ac87230 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fea10000c00 tx=0x7fea1002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1195fbbd552d10f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea017fa640 1 -- 172.21.15.40:0/3325966556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10037040 con 0x55da7ac86ce0 2024-04-26T21:36:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea017fa640 1 -- 172.21.15.40:0/3325966556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1004e070 con 0x55da7ac86ce0 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.097+0000 7fea017fa640 1 -- 172.21.15.40:0/3325966556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1003c440 con 0x55da7ac86ce0 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 -- 172.21.15.40:0/3325966556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac86ce0 msgr2=0x55da7ac87230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac86ce0 0x55da7ac87230 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fea10000c00 tx=0x7fea1002c000 comp rx=0 tx=0).stop 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 -- 172.21.15.40:0/3325966556 shutdown_connections 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac3f120 0x55da7ac853a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac86ce0 0x55da7ac87230 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 --2- 172.21.15.40:0/3325966556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac69440 0x55da7ac69820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 -- 172.21.15.40:0/3325966556 >> 172.21.15.40:0/3325966556 conn(0x55da7ac3ba50 msgr2=0x55da7ac84250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 -- 172.21.15.40:0/3325966556 shutdown_connections 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 -- 172.21.15.40:0/3325966556 wait complete. 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 -- start start 2024-04-26T21:36:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.098+0000 7fea24832740 1 Processor -- start 2024-04-26T21:36:44.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea24832740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac5dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea24832740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac86ce0 0x55da7ac7dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea24832740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac7c520 0x55da7ac83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea24832740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55da7ac3fec0 con 0x55da7ac69440 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea24832740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55da7ab2afe0 con 0x55da7ac7c520 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea24832740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55da7ac39980 con 0x55da7ac86ce0 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea24832740 10 client.0 _ll_register_callbacks cb 0x55da7ab41150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea1b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac5dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea1b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac5dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44190/0 (socket says 172.21.15.40:44190) 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea1b7fe640 1 -- 172.21.15.40:0/4057603907 learned_addr learned my addr 172.21.15.40:0/4057603907 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea1affd640 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac86ce0 0x55da7ac7dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.099+0000 7fea1bfff640 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac7c520 0x55da7ac83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea1b7fe640 1 -- 172.21.15.40:0/4057603907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac86ce0 msgr2=0x55da7ac7dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac86ce0 0x55da7ac7dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea1b7fe640 1 -- 172.21.15.40:0/4057603907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac7c520 msgr2=0x55da7ac83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac7c520 0x55da7ac83760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea1b7fe640 1 -- 172.21.15.40:0/4057603907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da7a8b52d0 con 0x55da7ac69440 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac5dc80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fea0c001eb0 tx=0x7fea0c058de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4e594f9a5b8aee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fe9fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/4057603907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c057060 con 0x55da7ac69440 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/4057603907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fea0c0076a0 con 0x55da7ac69440 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 1 -- 172.21.15.40:0/4057603907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1004e070 con 0x55da7ac69440 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/4057603907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fea0c0890d0 con 0x55da7ac69440 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 10 client.5661 fetch_fsmap learned FSMap version 23 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 1 -- 172.21.15.40:0/4057603907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55da7a97f6b0 con 0x55da7ac69440 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/4057603907 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1004e070 con 0x55da7ac69440 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/4057603907 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fea0c0580f0 con 0x55da7ac69440 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 10 client.5661 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 10 client.5661 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:36:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 20 client.5661 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55da7ac83ca0 0x55da7ac97d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 4 client.5661 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.100+0000 7fea24832740 1 -- 172.21.15.40:0/4057603907 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da7abb1350 con 0x55da7ac83ca0 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.101+0000 7fea1affd640 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55da7ac83ca0 0x55da7ac97d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.101+0000 7fe9fa7fc640 10 client.5661 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.101+0000 7fea1affd640 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55da7ac83ca0 0x55da7ac97d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67da3895300a4cd7 server_cookie=bc89cae182a8d844 in_seq=0 out_seq=0 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.101+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/4057603907 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da7abb1350 con 0x55da7ac83ca0 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.101+0000 7fe9fa7fc640 10 client.5661 handle_command_reply: tid=0 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.101+0000 7fea24832740 1 client.5661 shutdown 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 -- 172.21.15.40:0/4057603907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 msgr2=0x55da7ac5dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac5dc80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fea0c001eb0 tx=0x7fea0c058de0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 -- 172.21.15.40:0/4057603907 shutdown_connections 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55da7ac83ca0 0x55da7ac97d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac7c520 0x55da7ac83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac86ce0 0x55da7ac7dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fe9fa7fc640 0 client.5661 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 --2- 172.21.15.40:0/4057603907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac5dc80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 -- 172.21.15.40:0/4057603907 >> 172.21.15.40:0/4057603907 conn(0x55da7ac3f120 msgr2=0x55da7ac88060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 -- 172.21.15.40:0/4057603907 shutdown_connections 2024-04-26T21:36:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 1 -- 172.21.15.40:0/4057603907 wait complete. 2024-04-26T21:36:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.102+0000 7fea24832740 20 client.5661 trim_cache size 0 max 16384 2024-04-26T21:36:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.120+0000 7fea24832740 1 Processor -- start 2024-04-26T21:36:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.121+0000 7fea24832740 1 -- start start 2024-04-26T21:36:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.121+0000 7fea24832740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac7cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.121+0000 7fea24832740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac7c520 0x55da7ac7dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.121+0000 7fea24832740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac39cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.121+0000 7fea24832740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c201dc0 con 0x55da7ac69440 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.121+0000 7fea24832740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c200de0 con 0x55da7ac86ce0 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.121+0000 7fea24832740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fea1c158050 con 0x55da7ac7c520 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.121+0000 7fea1b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac7cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac7cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44204/0 (socket says 172.21.15.40:44204) 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1b7fe640 1 -- 172.21.15.40:0/2575527365 learned_addr learned my addr 172.21.15.40:0/2575527365 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1bfff640 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac39cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1affd640 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac7c520 0x55da7ac7dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1b7fe640 1 -- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac7c520 msgr2=0x55da7ac7dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac7c520 0x55da7ac7dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1b7fe640 1 -- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 msgr2=0x55da7ac39cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac39cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1b7fe640 1 -- 172.21.15.40:0/2575527365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da7aca8760 con 0x55da7ac69440 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1bfff640 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac39cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac7cc90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea0c05d140 tx=0x7fea0c05b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedcf0867af5cf78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea00ff9640 1 -- 172.21.15.40:0/2575527365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0580f0 con 0x55da7ac69440 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea00ff9640 1 -- 172.21.15.40:0/2575527365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c0549f0 con 0x55da7ac69440 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.122+0000 7fea00ff9640 1 -- 172.21.15.40:0/2575527365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c054be0 con 0x55da7ac69440 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 -- 172.21.15.40:0/2575527365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 msgr2=0x55da7ac7cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac7cc90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea0c05d140 tx=0x7fea0c05b000 comp rx=0 tx=0).stop 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 -- 172.21.15.40:0/2575527365 shutdown_connections 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac39cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac7c520 0x55da7ac7dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 --2- 172.21.15.40:0/2575527365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac69440 0x55da7ac7cc90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 -- 172.21.15.40:0/2575527365 >> 172.21.15.40:0/2575527365 conn(0x55da7ac3f120 msgr2=0x55da7ac67ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 -- 172.21.15.40:0/2575527365 shutdown_connections 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 -- 172.21.15.40:0/2575527365 wait complete. 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.123+0000 7fea24832740 1 -- start start 2024-04-26T21:36:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea24832740 1 Processor -- start 2024-04-26T21:36:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea24832740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac69440 0x55da7ac38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea24832740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac7c520 0x55da7ac38bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea24832740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac9a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea24832740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55da7ac7e5a0 con 0x55da7ac7c520 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea24832740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c200790 con 0x55da7ac86ce0 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea24832740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55da7ac7ca70 con 0x55da7ac69440 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea24832740 10 client.0 _ll_register_callbacks cb 0x7fea0c054f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea1affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac7c520 0x55da7ac38bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea1affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac7c520 0x55da7ac38bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44220/0 (socket says 172.21.15.40:44220) 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.124+0000 7fea1affd640 1 -- 172.21.15.40:0/828254549 learned_addr learned my addr 172.21.15.40:0/828254549 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1bfff640 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac9a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac69440 0x55da7ac38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1affd640 1 -- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac69440 msgr2=0x55da7ac38670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1affd640 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac69440 0x55da7ac38670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1affd640 1 -- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 msgr2=0x55da7ac9a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1affd640 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac9a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1affd640 1 -- 172.21.15.40:0/828254549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da7acfeb80 con 0x55da7ac7c520 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac69440 0x55da7ac38670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea1affd640 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac7c520 0x55da7ac38bb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea10000c00 tx=0x7fea10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f7857864d9f5e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fe9fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/828254549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10037040 con 0x55da7ac7c520 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/828254549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fea1005bbc0 con 0x55da7ac7c520 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea24832740 1 -- 172.21.15.40:0/828254549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0c0549f0 con 0x55da7ac7c520 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/828254549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fea1005bdc0 con 0x55da7ac7c520 2024-04-26T21:36:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea24832740 10 client.5667 fetch_fsmap learned FSMap version 23 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fea24832740 1 -- 172.21.15.40:0/828254549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55da7ac9cb60 con 0x55da7ac7c520 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.125+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/828254549 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c0549f0 con 0x55da7ac7c520 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/828254549 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fea1005b2e0 con 0x55da7ac7c520 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 10 client.5667 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 10 client.5667 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 20 client.5667 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55da7ac9af80 0x55da7ac7e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 4 client.5667 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 -- 172.21.15.40:0/828254549 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "status"}) -- 0x55da7abab1c0 con 0x55da7ac9af80 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55da7ac9af80 0x55da7ac7e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea1b7fe640 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55da7ac9af80 0x55da7ac7e5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703a17be2af290b server_cookie=da3924f2527077a in_seq=0 out_seq=0 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fe9fa7fc640 10 client.5667 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fe9fa7fc640 1 -- 172.21.15.40:0/828254549 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55da7abab1c0 con 0x55da7ac9af80 2024-04-26T21:36:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fe9fa7fc640 10 client.5667 handle_command_reply: tid=0 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 client.5667 shutdown 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 -- 172.21.15.40:0/828254549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac7c520 msgr2=0x55da7ac38bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac7c520 0x55da7ac38bb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea10000c00 tx=0x7fea10004730 comp rx=0 tx=0).stop 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 -- 172.21.15.40:0/828254549 shutdown_connections 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55da7ac9af80 0x55da7ac7e5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da7ac86ce0 0x55da7ac9a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55da7ac7c520 0x55da7ac38bb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fe9fa7fc640 0 client.5667 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 --2- 172.21.15.40:0/828254549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55da7ac69440 0x55da7ac38670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 -- 172.21.15.40:0/828254549 >> 172.21.15.40:0/828254549 conn(0x55da7ac3f120 msgr2=0x55da7ac82c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 -- 172.21.15.40:0/828254549 shutdown_connections 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.126+0000 7fea24832740 1 -- 172.21.15.40:0/828254549 wait complete. 2024-04-26T21:36:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea24832740 20 client.5667 trim_cache size 0 max 16384 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5619, 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 4.4482134020000004, 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 23, 2024-04-26T21:36:44.135 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 39, 2024-04-26T21:36:44.136 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 39, 2024-04-26T21:36:44.136 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 5.4197585049999999 2024-04-26T21:36:44.136 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:36:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 -- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea0806c920 msgr2=0x7fea0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea0806c920 0x7fea0806ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea10034000 tx=0x7fea1002e000 comp rx=0 tx=0).stop 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 -- 172.21.15.40:0/1033398195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b0b70 msgr2=0x7fea1c0d0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b0b70 0x7fea1c0d0d50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c400 tx=0x7fea0c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 -- 172.21.15.40:0/1033398195 shutdown_connections 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea0806c920 0x7fea0806ede0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1c0b6c50 0x7fea1c0afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea1c0b4d40 0x7fea1c14d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 --2- 172.21.15.40:0/1033398195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1c0b0b70 0x7fea1c0d0d50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 -- 172.21.15.40:0/1033398195 >> 172.21.15.40:0/1033398195 conn(0x7fea1c0b39b0 msgr2=0x7fea1c0b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 -- 172.21.15.40:0/1033398195 shutdown_connections 2024-04-26T21:36:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.127+0000 7fea21b1b640 1 -- 172.21.15.40:0/1033398195 wait complete. 2024-04-26T21:36:44.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:36:44.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.238+0000 7fee2a539640 1 Processor -- start 2024-04-26T21:36:44.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.238+0000 7fee2a539640 1 -- start start 2024-04-26T21:36:44.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee2a539640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee240b2340 0x7fee240cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee2a539640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240cd930 0x7fee240d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee2a539640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee240d10d0 con 0x7fee24162810 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee2a539640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fee240d0750 con 0x7fee240b2340 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee2a539640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee240d16a0 con 0x7fee240cd930 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee237fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee240b2340 0x7fee240cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee237fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee240b2340 0x7fee240cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44228/0 (socket says 172.21.15.40:44228) 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee237fe640 1 -- 172.21.15.40:0/3360410328 learned_addr learned my addr 172.21.15.40:0/3360410328 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee28ad6640 1 --2- 172.21.15.40:0/3360410328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240cd930 0x7fee240d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee237fe640 1 -- 172.21.15.40:0/3360410328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240cd930 msgr2=0x7fee240d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee237fe640 1 --2- 172.21.15.40:0/3360410328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240cd930 0x7fee240d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee237fe640 1 -- 172.21.15.40:0/3360410328 >> v1:172.21.15.184:6789/0 conn(0x7fee24162810 legacy=0x7fee24070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:44.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.239+0000 7fee237fe640 1 -- 172.21.15.40:0/3360410328 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee240d1370 con 0x7fee240b2340 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.240+0000 7fee237fe640 1 --2- 172.21.15.40:0/3360410328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee240b2340 0x7fee240cd3f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fee240d1820 tx=0x7fee1402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4182ec2ef15d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.240+0000 7fee22ffd640 1 -- 172.21.15.40:0/3360410328 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14037070 con 0x7fee240b2340 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.240+0000 7fee22ffd640 1 -- 172.21.15.40:0/3360410328 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee14032440 con 0x7fee240b2340 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.240+0000 7fee22ffd640 1 -- 172.21.15.40:0/3360410328 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14032810 con 0x7fee240b2340 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.241+0000 7fee2a539640 1 -- 172.21.15.40:0/3360410328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee240b2340 msgr2=0x7fee240cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.241+0000 7fee2a539640 1 --2- 172.21.15.40:0/3360410328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee240b2340 0x7fee240cd3f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fee240d1820 tx=0x7fee1402a200 comp rx=0 tx=0).stop 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.241+0000 7fee2a539640 1 -- 172.21.15.40:0/3360410328 shutdown_connections 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.241+0000 7fee2a539640 1 --2- 172.21.15.40:0/3360410328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240cd930 0x7fee240d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.241+0000 7fee2a539640 1 --2- 172.21.15.40:0/3360410328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee240b2340 0x7fee240cd3f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.241+0000 7fee2a539640 1 -- 172.21.15.40:0/3360410328 >> 172.21.15.40:0/3360410328 conn(0x7fee240d2da0 msgr2=0x7fee240d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:44.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.241+0000 7fee2a539640 1 -- 172.21.15.40:0/3360410328 shutdown_connections 2024-04-26T21:36:44.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.241+0000 7fee2a539640 1 -- 172.21.15.40:0/3360410328 wait complete. 2024-04-26T21:36:44.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.242+0000 7fee2a539640 1 Processor -- start 2024-04-26T21:36:44.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.242+0000 7fee2a539640 1 -- start start 2024-04-26T21:36:44.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee2a539640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240b2340 0x7fee241542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee2a539640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee240cd930 0x7fee240c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee2a539640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee24162810 0x7fee240c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee2a539640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee240d2070 con 0x7fee24162810 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee2a539640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee240d0470 con 0x7fee240cd930 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee2a539640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee240d21f0 con 0x7fee240b2340 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee24162810 0x7fee240c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee24162810 0x7fee240c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44238/0 (socket says 172.21.15.40:44238) 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 -- 172.21.15.40:0/1908710307 learned_addr learned my addr 172.21.15.40:0/1908710307 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee23fff640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240b2340 0x7fee241542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee237fe640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee240cd930 0x7fee240c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 -- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240b2340 msgr2=0x7fee241542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240b2340 0x7fee241542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 -- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee240cd930 msgr2=0x7fee240c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee240cd930 0x7fee240c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 -- 172.21.15.40:0/1908710307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee241f5360 con 0x7fee24162810 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee237fe640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee240cd930 0x7fee240c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee23fff640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240b2340 0x7fee241542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.243+0000 7fee28ad6640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee24162810 0x7fee240c8ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00cd40 tx=0x7fee1c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.244+0000 7fee20ff9640 1 -- 172.21.15.40:0/1908710307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c003b90 con 0x7fee24162810 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.244+0000 7fee20ff9640 1 -- 172.21.15.40:0/1908710307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee1c003d30 con 0x7fee24162810 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.244+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee241f5640 con 0x7fee24162810 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.244+0000 7fee20ff9640 1 -- 172.21.15.40:0/1908710307 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c004240 con 0x7fee24162810 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.244+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee241f5c00 con 0x7fee24162810 2024-04-26T21:36:44.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.245+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee240694f0 con 0x7fee24162810 2024-04-26T21:36:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.245+0000 7fee20ff9640 1 -- 172.21.15.40:0/1908710307 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee1c0043e0 con 0x7fee24162810 2024-04-26T21:36:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.245+0000 7fee20ff9640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.246+0000 7fee20ff9640 1 -- 172.21.15.40:0/1908710307 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fee1c08c930 con 0x7fee24162810 2024-04-26T21:36:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.246+0000 7fee23fff640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.246+0000 7fee23fff640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee1006c8f0 0x7fee1006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee18006fd0 tx=0x7fee18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.250+0000 7fee20ff9640 1 -- 172.21.15.40:0/1908710307 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee1c05aad0 con 0x7fee24162810 2024-04-26T21:36:44.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.479+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fee24069e40 con 0x7fee24162810 2024-04-26T21:36:44.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.479+0000 7fee20ff9640 1 -- 172.21.15.40:0/1908710307 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7fee1c05a3d0 con 0x7fee24162810 2024-04-26T21:36:44.480 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee1006c8f0 msgr2=0x7fee1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee1006c8f0 0x7fee1006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee18006fd0 tx=0x7fee18008040 comp rx=0 tx=0).stop 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee24162810 msgr2=0x7fee240c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee24162810 0x7fee240c8ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00cd40 tx=0x7fee1c00a660 comp rx=0 tx=0).stop 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 shutdown_connections 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee24162810 0x7fee240c8ea0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee240cd930 0x7fee240c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 --2- 172.21.15.40:0/1908710307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee240b2340 0x7fee241542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 >> 172.21.15.40:0/1908710307 conn(0x7fee240d2da0 msgr2=0x7fee240b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 shutdown_connections 2024-04-26T21:36:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44.481+0000 7fee2a539640 1 -- 172.21.15.40:0/1908710307 wait complete. 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:36:44.493 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:36:44.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:44.494 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:36:44.509 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:36:44.509 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:36:44.509 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:36:44.509 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42643sec preferred_lft 42643sec 2024-04-26T21:36:44.510 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:44.511 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:36:44.512 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:36:44.512 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:36:44.512 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:36:44.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:44.512 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:36:44.572 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:36:44.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:44.574 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:36:44.630 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:36:44.630 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:36:44.630 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:36:44.630 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:36:44.630 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:36:44.631 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:36:44.631 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:36:44.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:36:44.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:36:44.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:36:44.631 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:36:44.681 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:36:44.679+0000 7f7d240cc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:36:44.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:36:44.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:36:44.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:44.757 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:36:44.813 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:36:44.814 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:36:44.814 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:36:44.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:44.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:36:44.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:36:44.879 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:44.934 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:36:44.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:36:44.935 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:44.990 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:36:44.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:44.990 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:36:45.072 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:36:45.072 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:36:45.086 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:36:45.086 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:36:45.087 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:36:45.087 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:36:45.087 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:36:45.087 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:36:45.087 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:36:45.087 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:36:45.087 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:36:45.088 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:36:45.089 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:36:45.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:45.089 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:45.153 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:36:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:36:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:36:45.251 INFO:teuthology.orchestra.run.smithi040.stdout:client5009 2024-04-26T21:36:45.251 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:45.251 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:45.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:45.267 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-26T21:36:45.324 INFO:teuthology.orchestra.run.smithi040.stdout:file_undamaged 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> import json 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> sys.exit(e.errno) 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> print(json.dumps( 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T21:36:45.325 DEBUG:teuthology.orchestra.run.smithi040:> indent=2)) 2024-04-26T21:36:45.326 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:36:45.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-26T21:36:45.442 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-26T21:36:45.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:36:45.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.583+0000 7f5655111640 1 Processor -- start 2024-04-26T21:36:45.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.583+0000 7f5655111640 1 -- start start 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f5655111640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f565015ab60 0x7f565006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f5655111640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f565015dd00 0x7f565006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f5655111640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f565015af40 con 0x7f56501617f0 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f5655111640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f565015e320 con 0x7f565015ab60 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f5655111640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5650160d10 con 0x7f565015dd00 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f565015ab60 0x7f565006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f5647fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f56501617f0 0x7f565015d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33458/0 (socket says 172.21.15.40:33458) 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f565015ab60 0x7f565006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44252/0 (socket says 172.21.15.40:44252) 2024-04-26T21:36:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564ed9d640 1 -- 172.21.15.40:0/373726048 learned_addr learned my addr 172.21.15.40:0/373726048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564f59e640 1 --2- 172.21.15.40:0/373726048 >> v2:172.21.15.184:3300/0 conn(0x7f565015dd00 0x7f565006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564ed9d640 1 -- 172.21.15.40:0/373726048 >> v2:172.21.15.184:3300/0 conn(0x7f565015dd00 msgr2=0x7f565006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564ed9d640 1 --2- 172.21.15.40:0/373726048 >> v2:172.21.15.184:3300/0 conn(0x7f565015dd00 0x7f565006c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564ed9d640 1 -- 172.21.15.40:0/373726048 >> v1:172.21.15.40:6789/0 conn(0x7f56501617f0 legacy=0x7f565015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564e59c640 1 -- 172.21.15.40:0/373726048 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585530947 0 0) 0x7f565015af40 con 0x7f56501617f0 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.584+0000 7f564ed9d640 1 -- 172.21.15.40:0/373726048 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5650160ff0 con 0x7f565015ab60 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.585+0000 7f564ed9d640 1 --2- 172.21.15.40:0/373726048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015ab60 0x7f565006c240 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5650074840 tx=0x7f564002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3487e3bcb8d244d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.585+0000 7f564e59c640 1 -- 172.21.15.40:0/373726048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640037070 con 0x7f565015ab60 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.585+0000 7f564e59c640 1 -- 172.21.15.40:0/373726048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5640032440 con 0x7f565015ab60 2024-04-26T21:36:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.585+0000 7f564e59c640 1 -- 172.21.15.40:0/373726048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640032810 con 0x7f565015ab60 2024-04-26T21:36:45.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.586+0000 7f5655111640 1 -- 172.21.15.40:0/373726048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015ab60 msgr2=0x7f565006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.586+0000 7f5655111640 1 --2- 172.21.15.40:0/373726048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015ab60 0x7f565006c240 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5650074840 tx=0x7f564002a270 comp rx=0 tx=0).stop 2024-04-26T21:36:45.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.586+0000 7f5655111640 1 -- 172.21.15.40:0/373726048 shutdown_connections 2024-04-26T21:36:45.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.586+0000 7f5655111640 1 --2- 172.21.15.40:0/373726048 >> v2:172.21.15.184:3300/0 conn(0x7f565015dd00 0x7f565006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.586+0000 7f5655111640 1 --2- 172.21.15.40:0/373726048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015ab60 0x7f565006c240 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.586+0000 7f5655111640 1 -- 172.21.15.40:0/373726048 >> 172.21.15.40:0/373726048 conn(0x7f56500d1cf0 msgr2=0x7f56500d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:45.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.586+0000 7f5655111640 1 -- 172.21.15.40:0/373726048 shutdown_connections 2024-04-26T21:36:45.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.586+0000 7f5655111640 1 -- 172.21.15.40:0/373726048 wait complete. 2024-04-26T21:36:45.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.587+0000 7f5655111640 1 Processor -- start 2024-04-26T21:36:45.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.587+0000 7f5655111640 1 -- start start 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5655111640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f565015ab60 0x7f5650157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5655111640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015dd00 0x7f565015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5655111640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56501617f0 0x7f565015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5655111640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56500cc4c0 con 0x7f565015dd00 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5655111640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56500cc340 con 0x7f56501617f0 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5655111640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56500cc640 con 0x7f565015ab60 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015dd00 0x7f565015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015dd00 0x7f565015c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44260/0 (socket says 172.21.15.40:44260) 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 -- 172.21.15.40:0/238947409 learned_addr learned my addr 172.21.15.40:0/238947409 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f564ed9d640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f565015ab60 0x7f5650157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 -- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f565015ab60 msgr2=0x7f5650157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f565015ab60 0x7f5650157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 -- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56501617f0 msgr2=0x7f565015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56501617f0 0x7f565015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 -- 172.21.15.40:0/238947409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56500b4b40 con 0x7f565015dd00 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5647fff640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015dd00 0x7f565015c2b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f563800c430 tx=0x7f563800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f56477fe640 1 -- 172.21.15.40:0/238947409 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56380074e0 con 0x7f565015dd00 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f56477fe640 1 -- 172.21.15.40:0/238947409 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56380047e0 con 0x7f565015dd00 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56500b4e20 con 0x7f565015dd00 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.588+0000 7f56477fe640 1 -- 172.21.15.40:0/238947409 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638004bd0 con 0x7f565015dd00 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.589+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56500b5900 con 0x7f565015dd00 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.589+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56500b5bc0 con 0x7f565015dd00 2024-04-26T21:36:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.590+0000 7f56477fe640 1 -- 172.21.15.40:0/238947409 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5638011070 con 0x7f565015dd00 2024-04-26T21:36:45.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.590+0000 7f56477fe640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f563006c8f0 0x7f563006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.590+0000 7f56477fe640 1 -- 172.21.15.40:0/238947409 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f563808b9d0 con 0x7f565015dd00 2024-04-26T21:36:45.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.591+0000 7f564ed9d640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f563006c8f0 0x7f563006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.591+0000 7f564ed9d640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f563006c8f0 0x7f563006edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5640034000 tx=0x7f564002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.595+0000 7f56477fe640 1 -- 172.21.15.40:0/238947409 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5638059ba0 con 0x7f565015dd00 2024-04-26T21:36:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.819+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56500694f0 con 0x7f565015dd00 2024-04-26T21:36:45.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.820+0000 7f56477fe640 1 -- 172.21.15.40:0/238947409 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5185 (secure 0 0 0) 0x7f56380594a0 con 0x7f565015dd00 2024-04-26T21:36:45.821 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:45.821 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:42.691649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5619,"qdb_cluster":[5619]},"id":2}]} 2024-04-26T21:36:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 23 2024-04-26T21:36:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f563006c8f0 msgr2=0x7f563006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f563006c8f0 0x7f563006edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5640034000 tx=0x7f564002e000 comp rx=0 tx=0).stop 2024-04-26T21:36:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015dd00 msgr2=0x7f565015c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015dd00 0x7f565015c2b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f563800c430 tx=0x7f563800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 shutdown_connections 2024-04-26T21:36:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f563006c8f0 0x7f563006edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56501617f0 0x7f565015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f565015dd00 0x7f565015c2b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 --2- 172.21.15.40:0/238947409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f565015ab60 0x7f5650157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 >> 172.21.15.40:0/238947409 conn(0x7f56500d1cf0 msgr2=0x7f565006c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 shutdown_connections 2024-04-26T21:36:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.822+0000 7f5655111640 1 -- 172.21.15.40:0/238947409 wait complete. 2024-04-26T21:36:45.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-26T21:36:45.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.935+0000 7f9610cda640 1 Processor -- start 2024-04-26T21:36:45.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.935+0000 7f9610cda640 1 -- start start 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f9610cda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f960c15f3a0 0x7f960c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f9610cda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f960c0d2330 0x7f960c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f9610cda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f960c0d0e20 con 0x7f960c0d1770 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f9610cda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f960c0b1d70 con 0x7f960c0d2330 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f9610cda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f960c0b2050 con 0x7f960c15f3a0 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f960ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f960c0d2330 0x7f960c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f960ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f960c0d2330 0x7f960c0b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44266/0 (socket says 172.21.15.40:44266) 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f9609d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f960c0d1770 0x7f960c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33460/0 (socket says 172.21.15.40:33460) 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f960ad9d640 1 -- 172.21.15.40:0/1737020621 learned_addr learned my addr 172.21.15.40:0/1737020621 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.936+0000 7f960a59c640 1 --2- 172.21.15.40:0/1737020621 >> v2:172.21.15.184:3300/0 conn(0x7f960c15f3a0 0x7f960c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.937+0000 7f960ad9d640 1 -- 172.21.15.40:0/1737020621 >> v2:172.21.15.184:3300/0 conn(0x7f960c15f3a0 msgr2=0x7f960c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.937+0000 7f960ad9d640 1 --2- 172.21.15.40:0/1737020621 >> v2:172.21.15.184:3300/0 conn(0x7f960c15f3a0 0x7f960c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.937+0000 7f960ad9d640 1 -- 172.21.15.40:0/1737020621 >> v1:172.21.15.40:6789/0 conn(0x7f960c0d1770 legacy=0x7f960c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.937+0000 7f960ad9d640 1 -- 172.21.15.40:0/1737020621 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c0b2330 con 0x7f960c0d2330 2024-04-26T21:36:45.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.937+0000 7f960959a640 1 -- 172.21.15.40:0/1737020621 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 281831392 0 0) 0x7f960c0d0e20 con 0x7f960c0d1770 2024-04-26T21:36:45.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.937+0000 7f960ad9d640 1 --2- 172.21.15.40:0/1737020621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c0b2850 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f960c073b50 tx=0x7f95fc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b32a88168e0b023 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.937+0000 7f960959a640 1 -- 172.21.15.40:0/1737020621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc036070 con 0x7f960c0d2330 2024-04-26T21:36:45.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f960959a640 1 -- 172.21.15.40:0/1737020621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95fc031600 con 0x7f960c0d2330 2024-04-26T21:36:45.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f960959a640 1 -- 172.21.15.40:0/1737020621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0319d0 con 0x7f960c0d2330 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f9610cda640 1 -- 172.21.15.40:0/1737020621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 msgr2=0x7f960c0b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f9610cda640 1 --2- 172.21.15.40:0/1737020621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c0b2850 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f960c073b50 tx=0x7f95fc02a730 comp rx=0 tx=0).stop 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f9610cda640 1 -- 172.21.15.40:0/1737020621 shutdown_connections 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f9610cda640 1 --2- 172.21.15.40:0/1737020621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c0b2850 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f9610cda640 1 --2- 172.21.15.40:0/1737020621 >> v2:172.21.15.184:3300/0 conn(0x7f960c15f3a0 0x7f960c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f9610cda640 1 -- 172.21.15.40:0/1737020621 >> 172.21.15.40:0/1737020621 conn(0x7f960c0d4f90 msgr2=0x7f960c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f9610cda640 1 -- 172.21.15.40:0/1737020621 shutdown_connections 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.938+0000 7f9610cda640 1 -- 172.21.15.40:0/1737020621 wait complete. 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.939+0000 7f9610cda640 1 Processor -- start 2024-04-26T21:36:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9610cda640 1 -- start start 2024-04-26T21:36:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9610cda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960c0d1770 0x7f960c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9610cda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9610cda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960c15f3a0 0x7f960c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9610cda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f960c0b2f10 con 0x7f960c0d2330 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9610cda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f960c0b2d90 con 0x7f960c15f3a0 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9610cda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f960c0b3090 con 0x7f960c0d1770 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9609d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9609d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44278/0 (socket says 172.21.15.40:44278) 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.940+0000 7f9609d9b640 1 -- 172.21.15.40:0/1505137895 learned_addr learned my addr 172.21.15.40:0/1505137895 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f960ad9d640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960c15f3a0 0x7f960c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f960a59c640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960c0d1770 0x7f960c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9609d9b640 1 -- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960c0d1770 msgr2=0x7f960c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9609d9b640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960c0d1770 0x7f960c15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9609d9b640 1 -- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960c15f3a0 msgr2=0x7f960c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9609d9b640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960c15f3a0 0x7f960c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9609d9b640 1 -- 172.21.15.40:0/1505137895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c160090 con 0x7f960c0d2330 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f960a59c640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960c0d1770 0x7f960c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9609d9b640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c158780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f95f400c4b0 tx=0x7f95f400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9602ffd640 1 -- 172.21.15.40:0/1505137895 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4018070 con 0x7f960c0d2330 2024-04-26T21:36:45.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9602ffd640 1 -- 172.21.15.40:0/1505137895 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f4007480 con 0x7f960c0d2330 2024-04-26T21:36:45.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9602ffd640 1 -- 172.21.15.40:0/1505137895 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4004820 con 0x7f960c0d2330 2024-04-26T21:36:45.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9610cda640 1 -- 172.21.15.40:0/1505137895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c160370 con 0x7f960c0d2330 2024-04-26T21:36:45.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.941+0000 7f9610cda640 1 -- 172.21.15.40:0/1505137895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c158cc0 con 0x7f960c0d2330 2024-04-26T21:36:45.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.943+0000 7f9602ffd640 1 -- 172.21.15.40:0/1505137895 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95f401b070 con 0x7f960c0d2330 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.943+0000 7f9602ffd640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95ec06c920 0x7f95ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.943+0000 7f9602ffd640 1 -- 172.21.15.40:0/1505137895 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f95f408af20 con 0x7f960c0d2330 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.943+0000 7f960a59c640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95ec06c920 0x7f95ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.944+0000 7f960a59c640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95ec06c920 0x7f95ec06ede0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9604006fd0 tx=0x7f9604008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.944+0000 7f96139f9740 1 Processor -- start 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.944+0000 7f96139f9740 1 -- start start 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f96139f9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x5652948ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f96139f9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x5652948ea0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f96139f9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294914c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f96139f9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56529484eb80 con 0x5652949263c0 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f96139f9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5652947ea4d0 con 0x5652949147b0 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f96139f9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56529484ab40 con 0x5652948e9cf0 2024-04-26T21:36:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x5652948ea0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x5652948ea0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44294/0 (socket says 172.21.15.40:44294) 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 -- 172.21.15.40:0/662487623 learned_addr learned my addr 172.21.15.40:0/662487623 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f960a59c640 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x5652948ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f960ad9d640 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294914c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 -- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 msgr2=0x5652948ebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x5652948ebe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 -- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 msgr2=0x565294914c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294914c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 -- 172.21.15.40:0/662487623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565294817770 con 0x5652949263c0 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f960ad9d640 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294914c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.945+0000 7f9609d9b640 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x5652948ea0d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f95f40611d0 tx=0x7f95f405ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66147a5dfc8b2a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f95ebfff640 1 -- 172.21.15.40:0/662487623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4087070 con 0x5652949263c0 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f95ebfff640 1 -- 172.21.15.40:0/662487623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f4059e50 con 0x5652949263c0 2024-04-26T21:36:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f95ebfff640 1 -- 172.21.15.40:0/662487623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4061270 con 0x5652949263c0 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 -- 172.21.15.40:0/662487623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 msgr2=0x5652948ea0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x5652948ea0d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f95f40611d0 tx=0x7f95f405ced0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 -- 172.21.15.40:0/662487623 shutdown_connections 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294914c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x5652948ea0d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 --2- 172.21.15.40:0/662487623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x5652948ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 -- 172.21.15.40:0/662487623 >> 172.21.15.40:0/662487623 conn(0x56529487ee90 msgr2=0x56529492f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 -- 172.21.15.40:0/662487623 shutdown_connections 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.946+0000 7f96139f9740 1 -- 172.21.15.40:0/662487623 wait complete. 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.947+0000 7f96139f9740 1 -- start start 2024-04-26T21:36:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.947+0000 7f96139f9740 1 Processor -- start 2024-04-26T21:36:45.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f96139f9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294917620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f96139f9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294917b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f96139f9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x565294927a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f96139f9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5652947e7e70 con 0x5652949263c0 2024-04-26T21:36:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f96139f9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565294912300 con 0x5652949147b0 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f96139f9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x565294900f30 con 0x5652948e9cf0 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f96139f9740 10 client.0 _ll_register_callbacks cb 0x5652947f8ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x565294927a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x565294927a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44298/0 (socket says 172.21.15.40:44298) 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960ad9d640 1 -- 172.21.15.40:0/2729631009 learned_addr learned my addr 172.21.15.40:0/2729631009 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f9609d9b640 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294917b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960a59c640 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294917620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960ad9d640 1 -- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 msgr2=0x565294917620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960ad9d640 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294917620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960ad9d640 1 -- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 msgr2=0x565294917b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960ad9d640 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294917b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960ad9d640 1 -- 172.21.15.40:0/2729631009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56529462c7c0 con 0x5652949263c0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.948+0000 7f960a59c640 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294917620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f960ad9d640 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x565294927a10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f95fc034040 tx=0x7f95fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda4e18ff1195777 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f95e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f95e97fa640 1 -- 172.21.15.40:0/2729631009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc041070 con 0x5652949263c0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f95e97fa640 1 -- 172.21.15.40:0/2729631009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f95fc031d30 con 0x5652949263c0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 1 -- 172.21.15.40:0/2729631009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f4059e50 con 0x5652949263c0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f95e97fa640 1 -- 172.21.15.40:0/2729631009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f95fc04c050 con 0x5652949263c0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 10 client.5697 fetch_fsmap learned FSMap version 23 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 1 -- 172.21.15.40:0/2729631009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56529468cf50 con 0x5652949263c0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f95e97fa640 1 -- 172.21.15.40:0/2729631009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f4059e50 con 0x5652949263c0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f95e97fa640 1 -- 172.21.15.40:0/2729631009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f95fc04f020 con 0x5652949263c0 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 10 client.5697 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:36:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 10 client.5697 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 20 client.5697 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x5652948ffeb0 0x565294900290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 4 client.5697 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f96139f9740 1 -- 172.21.15.40:0/2729631009 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565294802240 con 0x5652948ffeb0 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.949+0000 7f960a59c640 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x5652948ffeb0 0x565294900290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.950+0000 7f95e97fa640 10 client.5697 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.950+0000 7f960a59c640 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x5652948ffeb0 0x565294900290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4be9ec7aa5f42e server_cookie=6e0555b7086924ff in_seq=0 out_seq=0 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.950+0000 7f95e97fa640 1 -- 172.21.15.40:0/2729631009 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565294802240 con 0x5652948ffeb0 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.950+0000 7f95e97fa640 10 client.5697 handle_command_reply: tid=0 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.950+0000 7f96139f9740 1 client.5697 shutdown 2024-04-26T21:36:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.950+0000 7f96139f9740 1 -- 172.21.15.40:0/2729631009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 msgr2=0x565294927a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.950+0000 7f96139f9740 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x565294927a10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f95fc034040 tx=0x7f95fc004ab0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 1 -- 172.21.15.40:0/2729631009 shutdown_connections 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x5652948ffeb0 0x565294900290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949263c0 0x565294927a10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949147b0 0x565294917b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f95e97fa640 0 client.5697 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 1 --2- 172.21.15.40:0/2729631009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294917620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 1 -- 172.21.15.40:0/2729631009 >> 172.21.15.40:0/2729631009 conn(0x56529487ee90 msgr2=0x5652949124e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 1 -- 172.21.15.40:0/2729631009 shutdown_connections 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 1 -- 172.21.15.40:0/2729631009 wait complete. 2024-04-26T21:36:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.951+0000 7f96139f9740 20 client.5697 trim_cache size 0 max 16384 2024-04-26T21:36:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.964+0000 7f96139f9740 1 Processor -- start 2024-04-26T21:36:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.964+0000 7f96139f9740 1 -- start start 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f96139f9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x56529493e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f96139f9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294929e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f96139f9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 0x565294929190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f96139f9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f960c200e40 con 0x5652949147b0 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f96139f9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f960c0c9cd0 con 0x5652949263c0 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f96139f9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f960c0ca320 con 0x5652948e9cf0 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294929e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294929e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44306/0 (socket says 172.21.15.40:44306) 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 -- 172.21.15.40:0/3639810433 learned_addr learned my addr 172.21.15.40:0/3639810433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f960a59c640 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x56529493e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f960ad9d640 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 0x565294929190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 -- 172.21.15.40:0/3639810433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 msgr2=0x56529493e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x56529493e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 -- 172.21.15.40:0/3639810433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 msgr2=0x565294929190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 0x565294929190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 -- 172.21.15.40:0/3639810433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565294927d80 con 0x5652949147b0 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.965+0000 7f9609d9b640 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294929e90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f95f4007fd0 tx=0x7f95f401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c2d154797280d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f95eb7fe640 1 -- 172.21.15.40:0/3639810433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4087040 con 0x5652949147b0 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f95eb7fe640 1 -- 172.21.15.40:0/3639810433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f4059e50 con 0x5652949147b0 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f95eb7fe640 1 -- 172.21.15.40:0/3639810433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4061270 con 0x5652949147b0 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 -- 172.21.15.40:0/3639810433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 msgr2=0x565294929e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294929e90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f95f4007fd0 tx=0x7f95f401e000 comp rx=0 tx=0).stop 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 -- 172.21.15.40:0/3639810433 shutdown_connections 2024-04-26T21:36:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 0x565294929190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294929e90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 --2- 172.21.15.40:0/3639810433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x56529493e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 -- 172.21.15.40:0/3639810433 >> 172.21.15.40:0/3639810433 conn(0x56529487ee90 msgr2=0x56529490f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 -- 172.21.15.40:0/3639810433 shutdown_connections 2024-04-26T21:36:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.966+0000 7f96139f9740 1 -- 172.21.15.40:0/3639810433 wait complete. 2024-04-26T21:36:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.967+0000 7f96139f9740 1 -- start start 2024-04-26T21:36:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.967+0000 7f96139f9740 1 Processor -- start 2024-04-26T21:36:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f96139f9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294913950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f96139f9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294900290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f96139f9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 0x5652949007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f96139f9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5652948fed70 con 0x5652949147b0 2024-04-26T21:36:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f96139f9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5652948ff430 con 0x5652949263c0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f96139f9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x565294901550 con 0x5652948e9cf0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f96139f9740 10 client.0 _ll_register_callbacks cb 0x7f95fc034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294900290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294900290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44314/0 (socket says 172.21.15.40:44314) 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 -- 172.21.15.40:0/906617064 learned_addr learned my addr 172.21.15.40:0/906617064 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f960a59c640 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294913950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f960ad9d640 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 0x5652949007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 -- 172.21.15.40:0/906617064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 msgr2=0x565294913950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294913950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 -- 172.21.15.40:0/906617064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 msgr2=0x5652949007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 0x5652949007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 -- 172.21.15.40:0/906617064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652945200b0 con 0x5652949147b0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f9609d9b640 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294900290 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f95f4058820 tx=0x7f95f4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18d470e1e367df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.968+0000 7f95e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f95e97fa640 1 -- 172.21.15.40:0/906617064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4087070 con 0x5652949147b0 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f95e97fa640 1 -- 172.21.15.40:0/906617064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f95f4082bc0 con 0x5652949147b0 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 1 -- 172.21.15.40:0/906617064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f4059e50 con 0x5652949147b0 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f95e97fa640 1 -- 172.21.15.40:0/906617064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f95f4068050 con 0x5652949147b0 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 10 client.5703 fetch_fsmap learned FSMap version 23 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 1 -- 172.21.15.40:0/906617064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56529493e710 con 0x5652949147b0 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f95e97fa640 1 -- 172.21.15.40:0/906617064 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f4059e50 con 0x5652949147b0 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f95e97fa640 1 -- 172.21.15.40:0/906617064 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f95f406b020 con 0x5652949147b0 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 10 client.5703 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 10 client.5703 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 20 client.5703 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x565294926ff0 0x5652949a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 4 client.5703 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f96139f9740 1 -- 172.21.15.40:0/906617064 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56529485c790 con 0x565294926ff0 2024-04-26T21:36:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.969+0000 7f960a59c640 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x565294926ff0 0x5652949a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f95e97fa640 10 client.5703 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f960a59c640 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x565294926ff0 0x5652949a93f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b31f250ab703e6f server_cookie=42daacb93999e3a6 in_seq=0 out_seq=0 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f95e97fa640 1 -- 172.21.15.40:0/906617064 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x56529485c790 con 0x565294926ff0 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f95e97fa640 10 client.5703 handle_command_reply: tid=0 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 client.5703 shutdown 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 -- 172.21.15.40:0/906617064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 msgr2=0x565294900290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294900290 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f95f4058820 tx=0x7f95f4000ed0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 -- 172.21.15.40:0/906617064 shutdown_connections 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x565294926ff0 0x5652949a93f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5652949263c0 0x5652949007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5652949147b0 0x565294900290 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 --2- 172.21.15.40:0/906617064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5652948e9cf0 0x565294913950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f95e97fa640 0 client.5703 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 -- 172.21.15.40:0/906617064 >> 172.21.15.40:0/906617064 conn(0x56529487ee90 msgr2=0x565294929190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 -- 172.21.15.40:0/906617064 shutdown_connections 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 1 -- 172.21.15.40:0/906617064 wait complete. 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.970+0000 7f96139f9740 20 client.5703 trim_cache size 0 max 16384 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "dentry", 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4285318383, 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627776, 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stdout: "frag": "*", 2024-04-26T21:36:45.978 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "file_to_be_damaged", 2024-04-26T21:36:45.979 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_id": "head", 2024-04-26T21:36:45.979 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-26T21:36:45.979 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:45.979 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:36:45.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 -- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95ec06c920 msgr2=0x7f95ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95ec06c920 0x7f95ec06ede0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9604006fd0 tx=0x7f9604008040 comp rx=0 tx=0).stop 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 -- 172.21.15.40:0/1505137895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 msgr2=0x7f960c158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c158780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f95f400c4b0 tx=0x7f95f400c970 comp rx=0 tx=0).stop 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 -- 172.21.15.40:0/1505137895 shutdown_connections 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95ec06c920 0x7f95ec06ede0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960c15f3a0 0x7f960c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960c0d2330 0x7f960c158780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 --2- 172.21.15.40:0/1505137895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960c0d1770 0x7f960c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 -- 172.21.15.40:0/1505137895 >> 172.21.15.40:0/1505137895 conn(0x7f960c0d4f90 msgr2=0x7f960c0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 -- 172.21.15.40:0/1505137895 shutdown_connections 2024-04-26T21:36:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:45.971+0000 7f9610cda640 1 -- 172.21.15.40:0/1505137895 wait complete. 2024-04-26T21:36:45.982 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:36:45.982 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:36:45.982 DEBUG:teuthology.orchestra.run.smithi040:> import errno 2024-04-26T21:36:45.982 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:36:45.983 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:36:45.983 DEBUG:teuthology.orchestra.run.smithi040:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-04-26T21:36:45.983 DEBUG:teuthology.orchestra.run.smithi040:> f.close() 2024-04-26T21:36:45.983 DEBUG:teuthology.orchestra.run.smithi040:> except IOError as e: 2024-04-26T21:36:45.983 DEBUG:teuthology.orchestra.run.smithi040:> sys.exit(errno.EIO) 2024-04-26T21:36:45.983 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:36:46.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-04-26T21:36:46.059 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-04-26T21:36:46.059 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> import json 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> sys.exit(e.errno) 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:36:46.060 DEBUG:teuthology.orchestra.run.smithi040:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T21:36:46.061 DEBUG:teuthology.orchestra.run.smithi040:> print(json.dumps( 2024-04-26T21:36:46.061 DEBUG:teuthology.orchestra.run.smithi040:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T21:36:46.061 DEBUG:teuthology.orchestra.run.smithi040:> indent=2)) 2024-04-26T21:36:46.061 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:36:46.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-04-26T21:36:46.177 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-04-26T21:36:46.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:46.178 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-26T21:36:46.234 INFO:teuthology.orchestra.run.smithi040.stdout:2 2024-04-26T21:36:46.234 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:46.234 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:46.292 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:36:46.292 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:36:46.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:36:46.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.488+0000 7fcaea1a4640 1 Processor -- start 2024-04-26T21:36:46.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.488+0000 7fcaea1a4640 1 -- start start 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcaea1a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcaea1a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae415d7c0 con 0x7fcae4071e60 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcaea1a4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae415daa0 con 0x7fcae4072780 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcaea1a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcae415dd80 con 0x7fcae40cc480 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcae4071e60 0x7fcae406f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33462/0 (socket says 172.21.15.40:33462) 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae37fe640 1 -- 172.21.15.40:0/1701516753 learned_addr learned my addr 172.21.15.40:0/1701516753 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae2ffd640 1 --2- 172.21.15.40:0/1701516753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae27fc640 1 -- 172.21.15.40:0/1701516753 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1869966619 0 0) 0x7fcae415d7c0 con 0x7fcae4071e60 2024-04-26T21:36:46.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae27fc640 1 -- 172.21.15.40:0/1701516753 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcad0003520 con 0x7fcae4071e60 2024-04-26T21:36:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae27fc640 1 -- 172.21.15.40:0/1701516753 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 574827482 0 0) 0x7fcae415daa0 con 0x7fcae4072780 2024-04-26T21:36:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae27fc640 1 -- 172.21.15.40:0/1701516753 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcae415d7c0 con 0x7fcae4072780 2024-04-26T21:36:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae2ffd640 1 -- 172.21.15.40:0/1701516753 >> v1:172.21.15.184:6789/0 conn(0x7fcae4072780 legacy=0x7fcae4166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae2ffd640 1 -- 172.21.15.40:0/1701516753 >> v1:172.21.15.40:6789/0 conn(0x7fcae4071e60 legacy=0x7fcae406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae27fc640 1 -- 172.21.15.40:0/1701516753 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2972034899 0 0) 0x7fcad0003520 con 0x7fcae4071e60 2024-04-26T21:36:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.489+0000 7fcae2ffd640 1 -- 172.21.15.40:0/1701516753 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae415e060 con 0x7fcae40cc480 2024-04-26T21:36:46.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.490+0000 7fcae2ffd640 1 --2- 172.21.15.40:0/1701516753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4072240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0097d0 tx=0x7fcacc02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1810b325831905 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.490+0000 7fcae27fc640 1 -- 172.21.15.40:0/1701516753 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc037070 con 0x7fcae40cc480 2024-04-26T21:36:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.490+0000 7fcae27fc640 1 -- 172.21.15.40:0/1701516753 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcacc02ac30 con 0x7fcae40cc480 2024-04-26T21:36:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.490+0000 7fcae27fc640 1 -- 172.21.15.40:0/1701516753 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4001670 con 0x7fcae40cc480 2024-04-26T21:36:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.491+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1701516753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 msgr2=0x7fcae4072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.491+0000 7fcaea1a4640 1 --2- 172.21.15.40:0/1701516753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4072240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0097d0 tx=0x7fcacc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.491+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1701516753 shutdown_connections 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.491+0000 7fcaea1a4640 1 --2- 172.21.15.40:0/1701516753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4072240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.491+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1701516753 >> 172.21.15.40:0/1701516753 conn(0x7fcae40d0be0 msgr2=0x7fcae40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.491+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1701516753 shutdown_connections 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.491+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1701516753 wait complete. 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.492+0000 7fcaea1a4640 1 Processor -- start 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.492+0000 7fcaea1a4640 1 -- start start 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcaea1a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcae4071e60 0x7fcae40b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcaea1a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcae4072780 0x7fcae4158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcaea1a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcaea1a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcae415d3f0 con 0x7fcae4071e60 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcaea1a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4072c10 con 0x7fcae4072780 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcaea1a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcae415d570 con 0x7fcae40cc480 2024-04-26T21:36:46.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcae37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcae4071e60 0x7fcae40b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcae37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcae4071e60 0x7fcae40b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44328/0 (socket says 172.21.15.40:44328) 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcae37fe640 1 -- 172.21.15.40:0/1797830010 learned_addr learned my addr 172.21.15.40:0/1797830010 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcae3fff640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.493+0000 7fcae2ffd640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcae4072780 0x7fcae4158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcae37fe640 1 -- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 msgr2=0x7fcae4153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcae37fe640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcae37fe640 1 -- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcae4072780 msgr2=0x7fcae4158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcae37fe640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcae4072780 0x7fcae4158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcae37fe640 1 -- 172.21.15.40:0/1797830010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae40b5a50 con 0x7fcae4071e60 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcae37fe640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcae4071e60 0x7fcae40b47a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcad400c5a0 tx=0x7fcad400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcadbfff640 1 -- 172.21.15.40:0/1797830010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4017070 con 0x7fcae4071e60 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcadbfff640 1 -- 172.21.15.40:0/1797830010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad4005ce0 con 0x7fcae4071e60 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcadbfff640 1 -- 172.21.15.40:0/1797830010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad40072b0 con 0x7fcae4071e60 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae40b5d30 con 0x7fcae4071e60 2024-04-26T21:36:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.494+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae40b62f0 con 0x7fcae4071e60 2024-04-26T21:36:46.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.495+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae40694f0 con 0x7fcae4071e60 2024-04-26T21:36:46.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.495+0000 7fcadbfff640 1 -- 172.21.15.40:0/1797830010 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcad401a070 con 0x7fcae4071e60 2024-04-26T21:36:46.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.496+0000 7fcadbfff640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.496+0000 7fcadbfff640 1 -- 172.21.15.40:0/1797830010 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fcad408b210 con 0x7fcae4071e60 2024-04-26T21:36:46.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.496+0000 7fcae2ffd640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.497+0000 7fcae2ffd640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcac406c8f0 0x7fcac406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcacc034000 tx=0x7fcacc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:46.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.500+0000 7fcadbfff640 1 -- 172.21.15.40:0/1797830010 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcad4058e10 con 0x7fcae4071e60 2024-04-26T21:36:46.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.685+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcae40d12f0 con 0x7fcae4071e60 2024-04-26T21:36:46.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.687+0000 7fcadbfff640 1 -- 172.21.15.40:0/1797830010 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17035 (secure 0 0 0) 0x7fcad405d110 con 0x7fcae4071e60 2024-04-26T21:36:46.688 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:36:46.688 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":39,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:38.666261+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3403660773":"2024-04-27T21:36:38.666240+0000","172.21.15.40:6833/3403660773":"2024-04-27T21:36:38.666240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.688+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcac406c8f0 msgr2=0x7fcac406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.688+0000 7fcaea1a4640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcac406c8f0 0x7fcac406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcacc034000 tx=0x7fcacc02e000 comp rx=0 tx=0).stop 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.688+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcae4071e60 msgr2=0x7fcae40b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.688+0000 7fcaea1a4640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcae4071e60 0x7fcae40b47a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcad400c5a0 tx=0x7fcad400ca60 comp rx=0 tx=0).stop 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.689+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 shutdown_connections 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.689+0000 7fcaea1a4640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.689+0000 7fcaea1a4640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcae40cc480 0x7fcae4153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.689+0000 7fcaea1a4640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcae4072780 0x7fcae4158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.689+0000 7fcaea1a4640 1 --2- 172.21.15.40:0/1797830010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcae4071e60 0x7fcae40b47a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.689+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 >> 172.21.15.40:0/1797830010 conn(0x7fcae40d0be0 msgr2=0x7fcae40b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.689+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 shutdown_connections 2024-04-26T21:36:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.689+0000 7fcaea1a4640 1 -- 172.21.15.40:0/1797830010 wait complete. 2024-04-26T21:36:46.701 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:36:22.370518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:36:22.724779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:36:46.701 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:36:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.805+0000 7f52e8d1c640 1 Processor -- start 2024-04-26T21:36:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.805+0000 7f52e8d1c640 1 -- start start 2024-04-26T21:36:46.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e8d1c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e4070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e8d1c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f52e4071090 0x7f52e415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e8d1c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e40d2390 con 0x7f52e40cfde0 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e8d1c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f52e4071500 con 0x7f52e4071090 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e8d1c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52e40afab0 con 0x7f52e4070350 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f52e40cfde0 0x7f52e40d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33470/0 (socket says 172.21.15.40:33470) 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e259c640 1 -- 172.21.15.40:0/2114266449 learned_addr learned my addr 172.21.15.40:0/2114266449 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e2d9d640 1 --2- 172.21.15.40:0/2114266449 >> v2:172.21.15.184:3300/0 conn(0x7f52e4071090 0x7f52e415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52dbfff640 1 --2- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e4070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e1d9b640 1 -- 172.21.15.40:0/2114266449 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511172559 0 0) 0x7f52e40d2390 con 0x7f52e40cfde0 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e1d9b640 1 -- 172.21.15.40:0/2114266449 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52d0003520 con 0x7f52e40cfde0 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e2d9d640 1 -- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 msgr2=0x7f52e4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e2d9d640 1 --2- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e4070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e2d9d640 1 -- 172.21.15.40:0/2114266449 >> v1:172.21.15.40:6789/0 conn(0x7f52e40cfde0 legacy=0x7f52e40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52e2d9d640 1 -- 172.21.15.40:0/2114266449 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e406d3f0 con 0x7f52e4071090 2024-04-26T21:36:46.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.806+0000 7f52dbfff640 1 --2- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e4070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.807+0000 7f52e2d9d640 1 --2- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 0x7f52e415e9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009d90 tx=0x7f52dc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=449a0d8da978a05e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.807+0000 7f52e1d9b640 1 -- 172.21.15.40:0/2114266449 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc031600 con 0x7f52e4071090 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.807+0000 7f52e1d9b640 1 -- 172.21.15.40:0/2114266449 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52dc0317a0 con 0x7f52e4071090 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.807+0000 7f52e1d9b640 1 -- 172.21.15.40:0/2114266449 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc031b70 con 0x7f52e4071090 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.808+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 msgr2=0x7f52e415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.808+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 0x7f52e415e9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009d90 tx=0x7f52dc02a5d0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.808+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2114266449 shutdown_connections 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.808+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 0x7f52e415e9f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.808+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2114266449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e4070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.808+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2114266449 >> 172.21.15.40:0/2114266449 conn(0x7f52e40d2d80 msgr2=0x7f52e40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.808+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2114266449 shutdown_connections 2024-04-26T21:36:46.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.808+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2114266449 wait complete. 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.809+0000 7f52e8d1c640 1 Processor -- start 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.809+0000 7f52e8d1c640 1 -- start start 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e8d1c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e41f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e8d1c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 0x7f52e415b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e8d1c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e40cfde0 0x7f52e409cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e8d1c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52e406d9b0 con 0x7f52e40cfde0 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e8d1c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52e406dd20 con 0x7f52e4071090 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e8d1c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52e415ef30 con 0x7f52e4070350 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e40cfde0 0x7f52e409cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e40cfde0 0x7f52e409cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44332/0 (socket says 172.21.15.40:44332) 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e2d9d640 1 -- 172.21.15.40:0/2391344451 learned_addr learned my addr 172.21.15.40:0/2391344451 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52dbfff640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 0x7f52e415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.810+0000 7f52e259c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e41f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52e2d9d640 1 -- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 msgr2=0x7f52e41f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52e2d9d640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e41f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52e2d9d640 1 -- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 msgr2=0x7f52e415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52e2d9d640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 0x7f52e415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52e2d9d640 1 -- 172.21.15.40:0/2391344451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e41f5820 con 0x7f52e40cfde0 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52e2d9d640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e40cfde0 0x7f52e409cbc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009eb0 tx=0x7f52dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52daffd640 1 -- 172.21.15.40:0/2391344451 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc040070 con 0x7f52e40cfde0 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52daffd640 1 -- 172.21.15.40:0/2391344451 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52dc03c850 con 0x7f52e40cfde0 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e40c8e20 con 0x7f52e40cfde0 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52daffd640 1 -- 172.21.15.40:0/2391344451 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc03cc40 con 0x7f52e40cfde0 2024-04-26T21:36:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.811+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e409f020 con 0x7f52e40cfde0 2024-04-26T21:36:46.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.812+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e40694f0 con 0x7f52e40cfde0 2024-04-26T21:36:46.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.812+0000 7f52daffd640 1 -- 172.21.15.40:0/2391344451 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52dc039040 con 0x7f52e40cfde0 2024-04-26T21:36:46.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.813+0000 7f52daffd640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c406c8f0 0x7f52c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:46.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.813+0000 7f52daffd640 1 -- 172.21.15.40:0/2391344451 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f52dc0acf20 con 0x7f52e40cfde0 2024-04-26T21:36:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.813+0000 7f52e259c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c406c8f0 0x7f52c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.813+0000 7f52e259c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c406c8f0 0x7f52c406edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f52d4005880 tx=0x7f52d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:46.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:46.817+0000 7f52daffd640 1 -- 172.21.15.40:0/2391344451 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f52e41f5820 con 0x7f52e40cfde0 2024-04-26T21:36:47.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.001+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f52e415b750 con 0x7f52e40cfde0 2024-04-26T21:36:47.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.001+0000 7f52daffd640 1 -- 172.21.15.40:0/2391344451 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f52dc038350 con 0x7f52e40cfde0 2024-04-26T21:36:47.003 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c406c8f0 msgr2=0x7f52c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c406c8f0 0x7f52c406edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f52d4005880 tx=0x7f52d4009040 comp rx=0 tx=0).stop 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e40cfde0 msgr2=0x7f52e409cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e40cfde0 0x7f52e409cbc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009eb0 tx=0x7f52dc004a40 comp rx=0 tx=0).stop 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 shutdown_connections 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c406c8f0 0x7f52c406edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e40cfde0 0x7f52e409cbc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e4071090 0x7f52e415b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 --2- 172.21.15.40:0/2391344451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e4070350 0x7f52e41f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 >> 172.21.15.40:0/2391344451 conn(0x7f52e40d2d80 msgr2=0x7f52e415e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:47.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 shutdown_connections 2024-04-26T21:36:47.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.003+0000 7f52e8d1c640 1 -- 172.21.15.40:0/2391344451 wait complete. 2024-04-26T21:36:47.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:36:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.118+0000 7fe7825f0640 1 Processor -- start 2024-04-26T21:36:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.118+0000 7fe7825f0640 1 -- start start 2024-04-26T21:36:47.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe7825f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f550 0x7fe77c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe7825f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe77c0d1040 con 0x7fe77c0d3bc0 2024-04-26T21:36:47.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe7825f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe77c0c9680 con 0x7fe77c15f170 2024-04-26T21:36:47.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe7825f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe77c0c9430 con 0x7fe77c15f550 2024-04-26T21:36:47.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe780b8d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe77c0d3bc0 0x7fe77c0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33472/0 (socket says 172.21.15.40:33472) 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe780b8d640 1 -- 172.21.15.40:0/2758636067 learned_addr learned my addr 172.21.15.40:0/2758636067 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe77b7fe640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f550 0x7fe77c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612026825 0 0) 0x7fe774001490 con 0x7fe77c0d3bc0 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe768003520 con 0x7fe77c0d3bc0 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371647186 0 0) 0x7fe77c0c9680 con 0x7fe77c15f170 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe774001490 con 0x7fe77c15f170 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2573260303 0 0) 0x7fe768003520 con 0x7fe77c0d3bc0 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f550 msgr2=0x7fe77c0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.119+0000 7fe77affd640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f550 0x7fe77c0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 >> v1:172.21.15.184:6789/0 conn(0x7fe77c15f170 legacy=0x7fe77c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c0c9960 con 0x7fe77c0d3bc0 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fe774001bc0 con 0x7fe77c0d3bc0 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77b7fe640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f550 0x7fe77c0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:36:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 >> v1:172.21.15.40:6789/0 conn(0x7fe77c0d3bc0 legacy=0x7fe77c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe768003b70 0x7fe768003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768004800 0x7fe768004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680051c0 0x7fe76800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c0c9430 con 0x7fe768003b70 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe76800bf00 con 0x7fe768004800 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe76800c080 con 0x7fe7680051c0 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77bfff640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe768003b70 0x7fe768003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77b7fe640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768004800 0x7fe768004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe780b8d640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680051c0 0x7fe76800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77bfff640 1 -- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680051c0 msgr2=0x7fe76800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77bfff640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680051c0 0x7fe76800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77bfff640 1 -- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768004800 msgr2=0x7fe768004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77bfff640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768004800 0x7fe768004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.120+0000 7fe77bfff640 1 -- 172.21.15.40:0/2758636067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe768003520 con 0x7fe768003b70 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe77bfff640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe768003b70 0x7fe768003f70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0018d0 tx=0x7fe768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd73bcc99d1311fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0014f0 con 0x7fe768003b70 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe76c0052d0 con 0x7fe768003b70 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe77affd640 1 -- 172.21.15.40:0/2758636067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c02aca0 con 0x7fe768003b70 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe7825f0640 1 -- 172.21.15.40:0/2758636067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe768003b70 msgr2=0x7fe768003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe7825f0640 1 --2- 172.21.15.40:0/2758636067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe768003b70 0x7fe768003f70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0018d0 tx=0x7fe768005830 comp rx=0 tx=0).stop 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe780b8d640 1 -- 172.21.15.40:0/2758636067 reap_dead start 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe7825f0640 1 -- 172.21.15.40:0/2758636067 shutdown_connections 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.121+0000 7fe7825f0640 1 -- 172.21.15.40:0/2758636067 >> 172.21.15.40:0/2758636067 conn(0x7fe77c0d4f60 msgr2=0x7fe77c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.122+0000 7fe7825f0640 1 -- 172.21.15.40:0/2758636067 shutdown_connections 2024-04-26T21:36:47.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.122+0000 7fe7825f0640 1 -- 172.21.15.40:0/2758636067 wait complete. 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.123+0000 7fe7825f0640 1 Processor -- start 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.123+0000 7fe7825f0640 1 -- start start 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe7825f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768003b70 0x7fe77c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe7825f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0cca20 0x7fe77c154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe7825f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0b8bd0 0x7fe77c0b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe7825f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c0d5c00 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe7825f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c0d5a80 con 0x7fe768003b70 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe7825f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe77c0d5d80 con 0x7fe77c0cca20 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0b8bd0 0x7fe77c0b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0b8bd0 0x7fe77c0b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44348/0 (socket says 172.21.15.40:44348) 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 -- 172.21.15.40:0/4155675103 learned_addr learned my addr 172.21.15.40:0/4155675103 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe77bfff640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768003b70 0x7fe77c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe77b7fe640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0cca20 0x7fe77c154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 -- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0cca20 msgr2=0x7fe77c154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0cca20 0x7fe77c154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 -- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768003b70 msgr2=0x7fe77c15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768003b70 0x7fe77c15b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 -- 172.21.15.40:0/4155675103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c15bff0 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe780b8d640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0b8bd0 0x7fe77c0b9050 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe774000f80 tx=0x7fe774007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe778ff9640 1 -- 172.21.15.40:0/4155675103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77400ae60 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe778ff9640 1 -- 172.21.15.40:0/4155675103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe774019070 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77c15c2d0 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe778ff9640 1 -- 172.21.15.40:0/4155675103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774013610 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.124+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77c15c890 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.126+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77c0694f0 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.126+0000 7fe778ff9640 1 -- 172.21.15.40:0/4155675103 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7740137b0 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.127+0000 7fe778ff9640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c920 0x7fe75c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.127+0000 7fe778ff9640 1 -- 172.21.15.40:0/4155675103 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fe77408c360 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.127+0000 7fe77bfff640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c920 0x7fe75c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.127+0000 7fe77bfff640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c920 0x7fe75c06ede0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe76c032000 tx=0x7fe76c009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.131+0000 7fe778ff9640 1 -- 172.21.15.40:0/4155675103 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe77c0d1040 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.318+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe77c0d1470 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.318+0000 7fe778ff9640 1 -- 172.21.15.40:0/4155675103 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) ==== 104+0+11 (secure 0 0 0) 0x7fe774059ee0 con 0x7fe77c0b8bd0 2024-04-26T21:36:47.319 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:36:47.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c920 msgr2=0x7fe75c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c920 0x7fe75c06ede0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe76c032000 tx=0x7fe76c009030 comp rx=0 tx=0).stop 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0b8bd0 msgr2=0x7fe77c0b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0b8bd0 0x7fe77c0b9050 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe774000f80 tx=0x7fe774007430 comp rx=0 tx=0).stop 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 shutdown_connections 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75c06c920 0x7fe75c06ede0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0b8bd0 0x7fe77c0b9050 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0cca20 0x7fe77c154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 --2- 172.21.15.40:0/4155675103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe768003b70 0x7fe77c15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 >> 172.21.15.40:0/4155675103 conn(0x7fe77c0d4f60 msgr2=0x7fe77c15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 shutdown_connections 2024-04-26T21:36:47.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.320+0000 7fe7825f0640 1 -- 172.21.15.40:0/4155675103 wait complete. 2024-04-26T21:36:47.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:36:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.436+0000 7f0d07e93640 1 Processor -- start 2024-04-26T21:36:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.436+0000 7f0d07e93640 1 -- start start 2024-04-26T21:36:47.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d07e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d000d3ee0 0x7f0d000d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d07e93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d000c8bc0 con 0x7f0d000c9350 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d07e93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d000d09f0 con 0x7f0d00162a20 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d07e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d000d07a0 con 0x7f0d000d3ee0 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d06430640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0d000c9350 0x7f0d000c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33484/0 (socket says 172.21.15.40:33484) 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d06430640 1 -- 172.21.15.40:0/903678309 learned_addr learned my addr 172.21.15.40:0/903678309 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d0542e640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d000d3ee0 0x7f0d000d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888154534 0 0) 0x7f0d000c8bc0 con 0x7f0d000c9350 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cec003520 con 0x7f0d000c9350 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2761729614 0 0) 0x7f0cec003520 con 0x7f0d000c9350 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d000d3ee0 msgr2=0x7f0d000d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d000d3ee0 0x7f0d000d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 >> v1:172.21.15.184:6789/0 conn(0x7f0d00162a20 legacy=0x7f0d000d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d000d0cd0 con 0x7f0d000c9350 2024-04-26T21:36:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f0cf80019f0 con 0x7f0d000c9350 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 >> v1:172.21.15.40:6789/0 conn(0x7f0d000c9350 legacy=0x7f0d000c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cec0039e0 0x7f0cec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d000d3ee0 0x7f0cec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cec004d40 0x7f0cec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d000d07a0 con 0x7f0cec0039e0 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec00baf0 con 0x7f0cec004d40 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.437+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec00bcf0 con 0x7f0d000d3ee0 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d05c2f640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cec0039e0 0x7f0cec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d0542e640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d000d3ee0 0x7f0cec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d06430640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cec004d40 0x7f0cec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d05c2f640 1 -- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d000d3ee0 msgr2=0x7f0cec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d05c2f640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d000d3ee0 0x7f0cec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d05c2f640 1 -- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cec004d40 msgr2=0x7f0cec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d05c2f640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cec004d40 0x7f0cec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d05c2f640 1 -- 172.21.15.40:0/903678309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec003520 con 0x7f0cec0039e0 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.438+0000 7f0d05c2f640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cec0039e0 0x7f0cec003e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc001750 tx=0x7f0cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72de3be2ed1bdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc001370 con 0x7f0cec0039e0 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cfc0052d0 con 0x7f0cec0039e0 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d04c2d640 1 -- 172.21.15.40:0/903678309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc02ab20 con 0x7f0cec0039e0 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d07e93640 1 -- 172.21.15.40:0/903678309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cec0039e0 msgr2=0x7f0cec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d07e93640 1 --2- 172.21.15.40:0/903678309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cec0039e0 0x7f0cec003e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc001750 tx=0x7f0cec005830 comp rx=0 tx=0).stop 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d06430640 1 -- 172.21.15.40:0/903678309 reap_dead start 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d07e93640 1 -- 172.21.15.40:0/903678309 shutdown_connections 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d07e93640 1 -- 172.21.15.40:0/903678309 >> 172.21.15.40:0/903678309 conn(0x7f0d000d4fd0 msgr2=0x7f0d000d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d07e93640 1 -- 172.21.15.40:0/903678309 shutdown_connections 2024-04-26T21:36:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.439+0000 7f0d07e93640 1 -- 172.21.15.40:0/903678309 wait complete. 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.440+0000 7f0d07e93640 1 Processor -- start 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.440+0000 7f0d07e93640 1 -- start start 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d07e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cec0039e0 0x7f0d00153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d07e93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0009fdb0 0x7f0d000b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d07e93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d000b2870 0x7f0d000cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d07e93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d000cbd90 con 0x7f0d0009fdb0 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d07e93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d000d0fb0 con 0x7f0d000b2870 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d07e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d00162e00 con 0x7f0cec0039e0 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d0542e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0009fdb0 0x7f0d000b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d0542e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0009fdb0 0x7f0d000b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44360/0 (socket says 172.21.15.40:44360) 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d0542e640 1 -- 172.21.15.40:0/690195130 learned_addr learned my addr 172.21.15.40:0/690195130 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d06430640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d000b2870 0x7f0d000cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.441+0000 7f0d05c2f640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cec0039e0 0x7f0d00153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d0542e640 1 -- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cec0039e0 msgr2=0x7f0d00153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d0542e640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cec0039e0 0x7f0d00153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d0542e640 1 -- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d000b2870 msgr2=0x7f0d000cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d0542e640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d000b2870 0x7f0d000cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d0542e640 1 -- 172.21.15.40:0/690195130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0015c890 con 0x7f0d0009fdb0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d06430640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d000b2870 0x7f0d000cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d0542e640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0009fdb0 0x7f0d000b2330 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00119d0 tx=0x7f0cf0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0cf67fc640 1 -- 172.21.15.40:0/690195130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000fa90 con 0x7f0d0009fdb0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0cf67fc640 1 -- 172.21.15.40:0/690195130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf000fc30 con 0x7f0d0009fdb0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d000b2ea0 con 0x7f0d0009fdb0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0cf67fc640 1 -- 172.21.15.40:0/690195130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0014440 con 0x7f0d0009fdb0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.442+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d000b3460 con 0x7f0d0009fdb0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.443+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d000694f0 con 0x7f0d0009fdb0 2024-04-26T21:36:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.443+0000 7f0cf67fc640 1 -- 172.21.15.40:0/690195130 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cf0012040 con 0x7f0d0009fdb0 2024-04-26T21:36:47.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.444+0000 7f0cf67fc640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0ce406c8f0 0x7f0ce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.444+0000 7f0cf67fc640 1 -- 172.21.15.40:0/690195130 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f0cf008c270 con 0x7f0d0009fdb0 2024-04-26T21:36:47.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.444+0000 7f0d05c2f640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0ce406c8f0 0x7f0ce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.445+0000 7f0d05c2f640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0ce406c8f0 0x7f0ce406edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc032000 tx=0x7f0cfc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.448+0000 7f0cf67fc640 1 -- 172.21.15.40:0/690195130 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0cf0019030 con 0x7f0d0009fdb0 2024-04-26T21:36:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.633+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d0006dd90 con 0x7f0d0009fdb0 2024-04-26T21:36:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.634+0000 7f0cf67fc640 1 -- 172.21.15.40:0/690195130 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) ==== 100+0+11 (secure 0 0 0) 0x7f0cf005a4c0 con 0x7f0d0009fdb0 2024-04-26T21:36:47.635 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:36:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0ce406c8f0 msgr2=0x7f0ce406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0ce406c8f0 0x7f0ce406edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc032000 tx=0x7f0cfc02e000 comp rx=0 tx=0).stop 2024-04-26T21:36:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0009fdb0 msgr2=0x7f0d000b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0009fdb0 0x7f0d000b2330 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00119d0 tx=0x7f0cf0011e90 comp rx=0 tx=0).stop 2024-04-26T21:36:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 shutdown_connections 2024-04-26T21:36:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0ce406c8f0 0x7f0ce406edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d000b2870 0x7f0d000cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0009fdb0 0x7f0d000b2330 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 --2- 172.21.15.40:0/690195130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cec0039e0 0x7f0d00153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 >> 172.21.15.40:0/690195130 conn(0x7f0d000d4fd0 msgr2=0x7f0d0006e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 shutdown_connections 2024-04-26T21:36:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.636+0000 7f0d07e93640 1 -- 172.21.15.40:0/690195130 wait complete. 2024-04-26T21:36:47.648 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:36:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.749+0000 7fbef5c93640 1 Processor -- start 2024-04-26T21:36:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.749+0000 7fbef5c93640 1 -- start start 2024-04-26T21:36:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbef5c93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbef00d3bc0 0x7fbef00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbef5c93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbef00b5f20 0x7fbef00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbef5c93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef00b7e50 con 0x7fbef0162880 2024-04-26T21:36:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbef5c93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbef00d0580 con 0x7fbef00d3bc0 2024-04-26T21:36:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbef5c93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbef00d1800 con 0x7fbef00b5f20 2024-04-26T21:36:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbeeeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbef00d3bc0 0x7fbef00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbeeeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbef00d3bc0 0x7fbef00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44362/0 (socket says 172.21.15.40:44362) 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbeeeffd640 1 -- 172.21.15.40:0/2042655411 learned_addr learned my addr 172.21.15.40:0/2042655411 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbeeffff640 1 --2- 172.21.15.40:0/2042655411 >> v2:172.21.15.184:3300/0 conn(0x7fbef00b5f20 0x7fbef00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbeeeffd640 1 -- 172.21.15.40:0/2042655411 >> v2:172.21.15.184:3300/0 conn(0x7fbef00b5f20 msgr2=0x7fbef00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbeeeffd640 1 --2- 172.21.15.40:0/2042655411 >> v2:172.21.15.184:3300/0 conn(0x7fbef00b5f20 0x7fbef00b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbeeeffd640 1 -- 172.21.15.40:0/2042655411 >> v1:172.21.15.184:6789/0 conn(0x7fbef0162880 legacy=0x7fbef00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.750+0000 7fbeeeffd640 1 -- 172.21.15.40:0/2042655411 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef00d07d0 con 0x7fbef00d3bc0 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.751+0000 7fbeeeffd640 1 --2- 172.21.15.40:0/2042655411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef00d2cb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbef008f770 tx=0x7fbee402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb3edd09cb7d54f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.751+0000 7fbeee7fc640 1 -- 172.21.15.40:0/2042655411 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4037070 con 0x7fbef00d3bc0 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.751+0000 7fbeee7fc640 1 -- 172.21.15.40:0/2042655411 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee4032440 con 0x7fbef00d3bc0 2024-04-26T21:36:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.751+0000 7fbeee7fc640 1 -- 172.21.15.40:0/2042655411 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4032810 con 0x7fbef00d3bc0 2024-04-26T21:36:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.752+0000 7fbef5c93640 1 -- 172.21.15.40:0/2042655411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 msgr2=0x7fbef00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.752+0000 7fbef5c93640 1 --2- 172.21.15.40:0/2042655411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef00d2cb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbef008f770 tx=0x7fbee402a230 comp rx=0 tx=0).stop 2024-04-26T21:36:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.752+0000 7fbef5c93640 1 -- 172.21.15.40:0/2042655411 shutdown_connections 2024-04-26T21:36:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.752+0000 7fbef5c93640 1 --2- 172.21.15.40:0/2042655411 >> v2:172.21.15.184:3300/0 conn(0x7fbef00b5f20 0x7fbef00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.752+0000 7fbef5c93640 1 --2- 172.21.15.40:0/2042655411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef00d2cb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.752+0000 7fbef5c93640 1 -- 172.21.15.40:0/2042655411 >> 172.21.15.40:0/2042655411 conn(0x7fbef00d4f60 msgr2=0x7fbef00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.752+0000 7fbef5c93640 1 -- 172.21.15.40:0/2042655411 shutdown_connections 2024-04-26T21:36:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.752+0000 7fbef5c93640 1 -- 172.21.15.40:0/2042655411 wait complete. 2024-04-26T21:36:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.753+0000 7fbef5c93640 1 Processor -- start 2024-04-26T21:36:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.753+0000 7fbef5c93640 1 -- start start 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbef5c93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbef00b5f20 0x7fbef00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbef5c93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef0150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbef5c93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0162880 0x7fbef00b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbef5c93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0162c60 con 0x7fbef00d3bc0 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbef5c93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbef00d0d90 con 0x7fbef0162880 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbef5c93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbef0162de0 con 0x7fbef00b5f20 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef0150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef0150420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44372/0 (socket says 172.21.15.40:44372) 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeeffd640 1 -- 172.21.15.40:0/1232909950 learned_addr learned my addr 172.21.15.40:0/1232909950 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeef7fe640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbef00b5f20 0x7fbef00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeffff640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0162880 0x7fbef00b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeeffd640 1 -- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbef00b5f20 msgr2=0x7fbef00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeeffd640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbef00b5f20 0x7fbef00ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeeffd640 1 -- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0162880 msgr2=0x7fbef00b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeeffd640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0162880 0x7fbef00b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.754+0000 7fbeeeffd640 1 -- 172.21.15.40:0/1232909950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef015b960 con 0x7fbef00d3bc0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.755+0000 7fbeeeffd640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef0150420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000c00 tx=0x7fbee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.755+0000 7fbed7fff640 1 -- 172.21.15.40:0/1232909950 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4041020 con 0x7fbef00d3bc0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.755+0000 7fbed7fff640 1 -- 172.21.15.40:0/1232909950 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee40306c0 con 0x7fbef00d3bc0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.755+0000 7fbed7fff640 1 -- 172.21.15.40:0/1232909950 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4037040 con 0x7fbef00d3bc0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.755+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef015bc40 con 0x7fbef00d3bc0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.755+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef015c200 con 0x7fbef00d3bc0 2024-04-26T21:36:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.756+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef00694f0 con 0x7fbef00d3bc0 2024-04-26T21:36:47.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.756+0000 7fbed7fff640 1 -- 172.21.15.40:0/1232909950 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbee4044020 con 0x7fbef00d3bc0 2024-04-26T21:36:47.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.757+0000 7fbed7fff640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbedc06c920 0x7fbedc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:47.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.757+0000 7fbed7fff640 1 -- 172.21.15.40:0/1232909950 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fbee40ac480 con 0x7fbef00d3bc0 2024-04-26T21:36:47.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.757+0000 7fbeef7fe640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbedc06c920 0x7fbedc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:47.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.757+0000 7fbeef7fe640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbedc06c920 0x7fbedc06ede0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbef00b5b60 tx=0x7fbee0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:47.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.761+0000 7fbed7fff640 1 -- 172.21.15.40:0/1232909950 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbee404b030 con 0x7fbef00d3bc0 2024-04-26T21:36:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.957+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbef015c3d0 con 0x7fbef00d3bc0 2024-04-26T21:36:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.957+0000 7fbed7fff640 1 -- 172.21.15.40:0/1232909950 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v39) ==== 80+0+122 (secure 0 0 0) 0x7fbee407a760 con 0x7fbef00d3bc0 2024-04-26T21:36:47.959 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/3403660773 2024-04-27T21:36:38.666240+0000 2024-04-26T21:36:47.959 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3403660773 2024-04-27T21:36:38.666240+0000 2024-04-26T21:36:47.959 INFO:teuthology.orchestra.run.smithi040.stderr:listed 2 entries 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbedc06c920 msgr2=0x7fbedc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbedc06c920 0x7fbedc06ede0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbef00b5b60 tx=0x7fbee0009040 comp rx=0 tx=0).stop 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 msgr2=0x7fbef0150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef0150420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000c00 tx=0x7fbee40046c0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 shutdown_connections 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbedc06c920 0x7fbedc06ede0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0162880 0x7fbef00b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef00d3bc0 0x7fbef0150420 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 --2- 172.21.15.40:0/1232909950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbef00b5f20 0x7fbef00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 >> 172.21.15.40:0/1232909950 conn(0x7fbef00d4f60 msgr2=0x7fbef00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.959+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 shutdown_connections 2024-04-26T21:36:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:47.960+0000 7fbef5c93640 1 -- 172.21.15.40:0/1232909950 wait complete. 2024-04-26T21:36:47.971 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:36:47.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:47.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:36:48.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:36:48.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:48.002 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:36:48.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:48.058 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start /subdir repair 2024-04-26T21:36:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.202+0000 7f316afb6640 1 Processor -- start 2024-04-26T21:36:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.202+0000 7f316afb6640 1 -- start start 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f316afb6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f316415f3e0 0x7f31640d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f316afb6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d3da0 0x7f31641620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f316afb6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3164160930 con 0x7f31640d31e0 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f316afb6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f316415fd20 con 0x7f316415f3e0 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f316afb6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3164160000 con 0x7f31640d3da0 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3168d52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f316415f3e0 0x7f31640d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3163fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f31640d31e0 0x7f31640d3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33500/0 (socket says 172.21.15.40:33500) 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3163fff640 1 -- 172.21.15.40:0/4278990724 learned_addr learned my addr 172.21.15.40:0/4278990724 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3169553640 1 --2- 172.21.15.40:0/4278990724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d3da0 0x7f31641620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3168d52640 1 -- 172.21.15.40:0/4278990724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d3da0 msgr2=0x7f31641620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3168d52640 1 --2- 172.21.15.40:0/4278990724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d3da0 0x7f31641620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3168d52640 1 -- 172.21.15.40:0/4278990724 >> v1:172.21.15.40:6789/0 conn(0x7f31640d31e0 legacy=0x7f31640d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f31637fe640 1 -- 172.21.15.40:0/4278990724 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 395634177 0 0) 0x7f3164160930 con 0x7f31640d31e0 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3168d52640 1 -- 172.21.15.40:0/4278990724 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31641602e0 con 0x7f316415f3e0 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.203+0000 7f3169553640 1 --2- 172.21.15.40:0/4278990724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d3da0 0x7f31641620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.204+0000 7f3168d52640 1 --2- 172.21.15.40:0/4278990724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f316415f3e0 0x7f31640d2ca0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f31580097a0 tx=0x7f315802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e6b5472b7676fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.204+0000 7f31637fe640 1 -- 172.21.15.40:0/4278990724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158037070 con 0x7f316415f3e0 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.204+0000 7f31637fe640 1 -- 172.21.15.40:0/4278990724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315802ae50 con 0x7f316415f3e0 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.204+0000 7f31637fe640 1 -- 172.21.15.40:0/4278990724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158032660 con 0x7f316415f3e0 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.204+0000 7f316afb6640 1 -- 172.21.15.40:0/4278990724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f316415f3e0 msgr2=0x7f31640d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.204+0000 7f316afb6640 1 --2- 172.21.15.40:0/4278990724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f316415f3e0 0x7f31640d2ca0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f31580097a0 tx=0x7f315802a100 comp rx=0 tx=0).stop 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.205+0000 7f316afb6640 1 -- 172.21.15.40:0/4278990724 shutdown_connections 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.205+0000 7f316afb6640 1 --2- 172.21.15.40:0/4278990724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d3da0 0x7f31641620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.205+0000 7f316afb6640 1 --2- 172.21.15.40:0/4278990724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f316415f3e0 0x7f31640d2ca0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.205+0000 7f316afb6640 1 -- 172.21.15.40:0/4278990724 >> 172.21.15.40:0/4278990724 conn(0x7f316409e3c0 msgr2=0x7f31640d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.205+0000 7f316afb6640 1 -- 172.21.15.40:0/4278990724 shutdown_connections 2024-04-26T21:36:48.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.205+0000 7f316afb6640 1 -- 172.21.15.40:0/4278990724 wait complete. 2024-04-26T21:36:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.206+0000 7f316afb6640 1 Processor -- start 2024-04-26T21:36:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.206+0000 7f316afb6640 1 -- start start 2024-04-26T21:36:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f316afb6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d31e0 0x7f316414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f316afb6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31640d3da0 0x7f31640ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f316afb6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316415f3e0 0x7f316414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f316afb6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f316415f7c0 con 0x7f31640d3da0 2024-04-26T21:36:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f316afb6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3164160d40 con 0x7f316415f3e0 2024-04-26T21:36:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f316afb6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f316415f940 con 0x7f31640d31e0 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31640d3da0 0x7f31640ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31640d3da0 0x7f31640ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44382/0 (socket says 172.21.15.40:44382) 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 -- 172.21.15.40:0/2595034710 learned_addr learned my addr 172.21.15.40:0/2595034710 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3169553640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316415f3e0 0x7f316414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3168d52640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d31e0 0x7f316414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 -- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d31e0 msgr2=0x7f316414c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d31e0 0x7f316414c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 -- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316415f3e0 msgr2=0x7f316414d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316415f3e0 0x7f316414d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 -- 172.21.15.40:0/2595034710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316414cb20 con 0x7f31640d3da0 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3168d52640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d31e0 0x7f316414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.207+0000 7f3163fff640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31640d3da0 0x7f31640ce830 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f315000c4b0 tx=0x7f315000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.208+0000 7f31617fa640 1 -- 172.21.15.40:0/2595034710 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150018070 con 0x7f31640d3da0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.208+0000 7f31617fa640 1 -- 172.21.15.40:0/2595034710 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3150007500 con 0x7f31640d3da0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.208+0000 7f31617fa640 1 -- 172.21.15.40:0/2595034710 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150004a20 con 0x7f31640d3da0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.208+0000 7f316afb6640 1 -- 172.21.15.40:0/2595034710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31640b80d0 con 0x7f31640d3da0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.208+0000 7f316afb6640 1 -- 172.21.15.40:0/2595034710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31640b8690 con 0x7f31640d3da0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.209+0000 7f31617fa640 1 -- 172.21.15.40:0/2595034710 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f315001b070 con 0x7f31640d3da0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.210+0000 7f31617fa640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f314806c8f0 0x7f314806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.210+0000 7f31617fa640 1 -- 172.21.15.40:0/2595034710 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f315008b090 con 0x7f31640d3da0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.210+0000 7f3168d52640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f314806c8f0 0x7f314806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.210+0000 7f3168d52640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f314806c8f0 0x7f314806edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3158034000 tx=0x7f315802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f316dda9740 1 Processor -- start 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f316dda9740 1 -- start start 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f316dda9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 0x558570a0b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f316dda9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a3b720 0x558570a3aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f316dda9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3afe0 0x558570a0fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f316dda9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558570974f80 con 0x558570a3b720 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f316dda9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558570910770 con 0x558570a3afe0 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f316dda9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558570970f40 con 0x558570a58600 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f3163fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a3b720 0x558570a3aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f3163fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a3b720 0x558570a3aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44390/0 (socket says 172.21.15.40:44390) 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f3163fff640 1 -- 172.21.15.40:0/1673598426 learned_addr learned my addr 172.21.15.40:0/1673598426 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f3168d52640 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 0x558570a0b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.211+0000 7f3169553640 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3afe0 0x558570a0fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f3163fff640 1 -- 172.21.15.40:0/1673598426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 msgr2=0x558570a0b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f3163fff640 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 0x558570a0b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f3163fff640 1 -- 172.21.15.40:0/1673598426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3afe0 msgr2=0x558570a0fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f3163fff640 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3afe0 0x558570a0fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f3163fff640 1 -- 172.21.15.40:0/1673598426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558570752a20 con 0x558570a3b720 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f3163fff640 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a3b720 0x558570a3aaa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3150001120 tx=0x7f3150058d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937a876cd4ba4f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f314e7fc640 1 -- 172.21.15.40:0/1673598426 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150087070 con 0x558570a3b720 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f314e7fc640 1 -- 172.21.15.40:0/1673598426 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3150056d30 con 0x558570a3b720 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.212+0000 7f314e7fc640 1 -- 172.21.15.40:0/1673598426 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150055300 con 0x558570a3b720 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 -- 172.21.15.40:0/1673598426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a3b720 msgr2=0x558570a3aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a3b720 0x558570a3aaa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3150001120 tx=0x7f3150058d50 comp rx=0 tx=0).stop 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 -- 172.21.15.40:0/1673598426 shutdown_connections 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3afe0 0x558570a0fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a3b720 0x558570a3aaa0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 --2- 172.21.15.40:0/1673598426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 0x558570a0b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 -- 172.21.15.40:0/1673598426 >> 172.21.15.40:0/1673598426 conn(0x558570a0d490 msgr2=0x558570a55aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 -- 172.21.15.40:0/1673598426 shutdown_connections 2024-04-26T21:36:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 -- 172.21.15.40:0/1673598426 wait complete. 2024-04-26T21:36:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 -- start start 2024-04-26T21:36:48.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.213+0000 7f316dda9740 1 Processor -- start 2024-04-26T21:36:48.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f316dda9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a3b720 0x558570a3d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f316dda9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a58600 0x558570a3dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f316dda9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f316dda9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558570a0f3b0 con 0x558570a37820 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f316dda9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55857090e110 con 0x558570a58600 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f316dda9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558570a0e7a0 con 0x558570a3b720 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f316dda9740 10 client.0 _ll_register_callbacks cb 0x55857096a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f3169553640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f3169553640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a37ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44406/0 (socket says 172.21.15.40:44406) 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.214+0000 7f3169553640 1 -- 172.21.15.40:0/771103424 learned_addr learned my addr 172.21.15.40:0/771103424 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f3168d52640 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a3b720 0x558570a3d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f3163fff640 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a58600 0x558570a3dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f3169553640 1 -- 172.21.15.40:0/771103424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a3b720 msgr2=0x558570a3d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f3169553640 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a3b720 0x558570a3d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f3169553640 1 -- 172.21.15.40:0/771103424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a58600 msgr2=0x558570a3dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f3169553640 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a58600 0x558570a3dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f3169553640 1 -- 172.21.15.40:0/771103424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558570646310 con 0x558570a37820 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f3169553640 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a37ca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f315c00ded0 tx=0x7f315c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba80569b9599c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f31437fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f31437fe640 1 -- 172.21.15.40:0/771103424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c018640 con 0x558570a37820 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f31437fe640 1 -- 172.21.15.40:0/771103424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f315c0187e0 con 0x558570a37820 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f316dda9740 1 -- 172.21.15.40:0/771103424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3150056d30 con 0x558570a37820 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f31437fe640 1 -- 172.21.15.40:0/771103424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f315c024050 con 0x558570a37820 2024-04-26T21:36:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f316dda9740 10 client.5748 fetch_fsmap learned FSMap version 23 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f316dda9740 1 -- 172.21.15.40:0/771103424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5585707b31b0 con 0x558570a37820 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f31437fe640 1 -- 172.21.15.40:0/771103424 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3150056d30 con 0x558570a37820 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f31437fe640 1 -- 172.21.15.40:0/771103424 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f315c027020 con 0x558570a37820 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f316dda9740 10 client.5748 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.215+0000 7f316dda9740 10 client.5748 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f316dda9740 20 client.5748 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f316dda9740 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x558570a0fd10 0x558570a10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f316dda9740 4 client.5748 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f316dda9740 1 -- 172.21.15.40:0/771103424 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558570982b90 con 0x558570a0fd10 2024-04-26T21:36:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f3168d52640 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x558570a0fd10 0x558570a10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f3168d52640 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x558570a0fd10 0x558570a10120 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb95117121073efa server_cookie=153b841797cc8cb9 in_seq=0 out_seq=0 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f31437fe640 10 client.5748 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f31437fe640 1 -- 172.21.15.40:0/771103424 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558570982b90 con 0x558570a0fd10 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.216+0000 7f31437fe640 10 client.5748 handle_command_reply: tid=0 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 client.5748 shutdown 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 -- 172.21.15.40:0/771103424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 msgr2=0x558570a37ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a37ca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f315c00ded0 tx=0x7f315c00c790 comp rx=0 tx=0).stop 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 -- 172.21.15.40:0/771103424 shutdown_connections 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x558570a0fd10 0x558570a10120 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a37ca0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a58600 0x558570a3dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f31437fe640 0 client.5748 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 --2- 172.21.15.40:0/771103424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a3b720 0x558570a3d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 -- 172.21.15.40:0/771103424 >> 172.21.15.40:0/771103424 conn(0x558570a3afe0 msgr2=0x558570a55aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 -- 172.21.15.40:0/771103424 shutdown_connections 2024-04-26T21:36:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 1 -- 172.21.15.40:0/771103424 wait complete. 2024-04-26T21:36:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.217+0000 7f316dda9740 20 client.5748 trim_cache size 0 max 16384 2024-04-26T21:36:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.234+0000 7f316dda9740 1 Processor -- start 2024-04-26T21:36:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f316dda9740 1 -- start start 2024-04-26T21:36:48.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f316dda9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a0b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f316dda9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 0x558570a558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f316dda9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 0x558570a55de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f316dda9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3164201d80 con 0x558570a37820 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f316dda9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3164200da0 con 0x558570a3b720 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f316dda9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f31640cca90 con 0x558570a58600 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f3168d52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a0b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f3168d52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a0b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44410/0 (socket says 172.21.15.40:44410) 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f3168d52640 1 -- 172.21.15.40:0/1248031804 learned_addr learned my addr 172.21.15.40:0/1248031804 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f3163fff640 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 0x558570a558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.235+0000 7f3169553640 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 0x558570a55de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f3168d52640 1 -- 172.21.15.40:0/1248031804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 msgr2=0x558570a55de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f3168d52640 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 0x558570a55de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f3168d52640 1 -- 172.21.15.40:0/1248031804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 msgr2=0x558570a558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f3168d52640 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 0x558570a558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f3168d52640 1 -- 172.21.15.40:0/1248031804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558570a12480 con 0x558570a37820 2024-04-26T21:36:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f3168d52640 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a0b0d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3158004650 tx=0x7f315802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaba2f34a017af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f314dffb640 1 -- 172.21.15.40:0/1248031804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158037040 con 0x558570a37820 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f314dffb640 1 -- 172.21.15.40:0/1248031804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3158003610 con 0x558570a37820 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f314dffb640 1 -- 172.21.15.40:0/1248031804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31580039c0 con 0x558570a37820 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f316dda9740 1 -- 172.21.15.40:0/1248031804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 msgr2=0x558570a0b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f316dda9740 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a0b0d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3158004650 tx=0x7f315802c000 comp rx=0 tx=0).stop 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f316dda9740 1 -- 172.21.15.40:0/1248031804 shutdown_connections 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f316dda9740 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a58600 0x558570a55de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f316dda9740 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 0x558570a558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f316dda9740 1 --2- 172.21.15.40:0/1248031804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a37820 0x558570a0b0d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.236+0000 7f316dda9740 1 -- 172.21.15.40:0/1248031804 >> 172.21.15.40:0/1248031804 conn(0x558570a3afe0 msgr2=0x558570a57df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.237+0000 7f316dda9740 1 -- 172.21.15.40:0/1248031804 shutdown_connections 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.237+0000 7f316dda9740 1 -- 172.21.15.40:0/1248031804 wait complete. 2024-04-26T21:36:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.237+0000 7f316dda9740 1 -- start start 2024-04-26T21:36:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.237+0000 7f316dda9740 1 Processor -- start 2024-04-26T21:36:48.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f316dda9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a37820 0x558570a26670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f316dda9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 0x558570a4be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f316dda9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a58600 0x558570a6ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f316dda9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31642009b0 con 0x558570a58600 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f316dda9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558570a0e330 con 0x558570a3b720 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f316dda9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558570a540d0 con 0x558570a37820 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f316dda9740 10 client.0 _ll_register_callbacks cb 0x7f315c007b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3169553640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a58600 0x558570a6ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3169553640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a58600 0x558570a6ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44420/0 (socket says 172.21.15.40:44420) 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3169553640 1 -- 172.21.15.40:0/223880750 learned_addr learned my addr 172.21.15.40:0/223880750 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3163fff640 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 0x558570a4be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3168d52640 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a37820 0x558570a26670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3169553640 1 -- 172.21.15.40:0/223880750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a37820 msgr2=0x558570a26670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3169553640 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a37820 0x558570a26670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3169553640 1 -- 172.21.15.40:0/223880750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 msgr2=0x558570a4be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3169553640 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 0x558570a4be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.238+0000 7f3169553640 1 -- 172.21.15.40:0/223880750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55857061f490 con 0x558570a58600 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f3169553640 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a58600 0x558570a6ee80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x558570a565f0 tx=0x7f315c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b0ee0982f45742 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f31437fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f31437fe640 1 -- 172.21.15.40:0/223880750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c00cc90 con 0x558570a58600 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f31437fe640 1 -- 172.21.15.40:0/223880750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f315c032ca0 con 0x558570a58600 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 1 -- 172.21.15.40:0/223880750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3158003610 con 0x558570a58600 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f31437fe640 1 -- 172.21.15.40:0/223880750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f315c018b50 con 0x558570a58600 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 10 client.5754 fetch_fsmap learned FSMap version 23 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 1 -- 172.21.15.40:0/223880750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x558570a10a80 con 0x558570a58600 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f31437fe640 1 -- 172.21.15.40:0/223880750 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3158003610 con 0x558570a58600 2024-04-26T21:36:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f31437fe640 1 -- 172.21.15.40:0/223880750 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f315c039050 con 0x558570a58600 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 10 client.5754 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 10 client.5754 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 20 client.5754 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x558570a6f3c0 0x558570a34e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 4 client.5754 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}] 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f316dda9740 1 -- 172.21.15.40:0/223880750 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}) -- 0x558570a0eb10 con 0x558570a6f3c0 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.239+0000 7f3168d52640 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x558570a6f3c0 0x558570a34e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f31437fe640 10 client.5754 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f3168d52640 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x558570a6f3c0 0x558570a34e00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137ce98d0d9eff66 server_cookie=f966332563bacb4c in_seq=0 out_seq=0 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f31437fe640 1 -- 172.21.15.40:0/223880750 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558570a0eb10 con 0x558570a6f3c0 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f31437fe640 10 client.5754 handle_command_reply: tid=0 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 client.5754 shutdown 2024-04-26T21:36:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 -- 172.21.15.40:0/223880750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a58600 msgr2=0x558570a6ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a58600 0x558570a6ee80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x558570a565f0 tx=0x7f315c014000 comp rx=0 tx=0).stop 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 -- 172.21.15.40:0/223880750 shutdown_connections 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x558570a6f3c0 0x558570a34e00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558570a58600 0x558570a6ee80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558570a3b720 0x558570a4be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 --2- 172.21.15.40:0/223880750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558570a37820 0x558570a26670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f31437fe640 0 client.5754 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.240+0000 7f316dda9740 1 -- 172.21.15.40:0/223880750 >> 172.21.15.40:0/223880750 conn(0x558570a3afe0 msgr2=0x558570a55490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316dda9740 1 -- 172.21.15.40:0/223880750 shutdown_connections 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316dda9740 1 -- 172.21.15.40:0/223880750 wait complete. 2024-04-26T21:36:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316dda9740 20 client.5754 trim_cache size 0 max 16384 2024-04-26T21:36:48.249 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:36:48.249 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0, 2024-04-26T21:36:48.249 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_tag": "e9f8ac55-1f55-475a-a7d7-aa7881d90858", 2024-04-26T21:36:48.249 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": "asynchronous" 2024-04-26T21:36:48.249 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 -- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f314806c8f0 msgr2=0x7f314806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f314806c8f0 0x7f314806edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3158034000 tx=0x7f315802e000 comp rx=0 tx=0).stop 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 -- 172.21.15.40:0/2595034710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31640d3da0 msgr2=0x7f31640ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31640d3da0 0x7f31640ce830 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f315000c4b0 tx=0x7f315000c970 comp rx=0 tx=0).stop 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 -- 172.21.15.40:0/2595034710 shutdown_connections 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f314806c8f0 0x7f314806edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316415f3e0 0x7f316414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31640d3da0 0x7f31640ce830 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 --2- 172.21.15.40:0/2595034710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31640d31e0 0x7f316414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 -- 172.21.15.40:0/2595034710 >> 172.21.15.40:0/2595034710 conn(0x7f316409e3c0 msgr2=0x7f3164161490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 -- 172.21.15.40:0/2595034710 shutdown_connections 2024-04-26T21:36:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.241+0000 7f316afb6640 1 -- 172.21.15.40:0/2595034710 wait complete. 2024-04-26T21:36:48.253 INFO:tasks.cephfs.test_damage:{ "return_code": 0, "scrub_tag": "e9f8ac55-1f55-475a-a7d7-aa7881d90858", "mode": "asynchronous" } 2024-04-26T21:36:48.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:36:48.253 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-26T21:36:48.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.355+0000 7f850b304640 1 Processor -- start 2024-04-26T21:36:48.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.355+0000 7f850b304640 1 -- start start 2024-04-26T21:36:48.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f850b304640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8504162810 0x7f8504070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f850b304640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f85040b2340 0x7f85040cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f850b304640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85040d10d0 con 0x7f85040cd930 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f850b304640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f85040d0750 con 0x7f8504162810 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f850b304640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f85040d16a0 con 0x7f85040b2340 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f85090a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8504162810 0x7f8504070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f85090a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8504162810 0x7f8504070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44422/0 (socket says 172.21.15.40:44422) 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f85090a0640 1 -- 172.21.15.40:0/654326429 learned_addr learned my addr 172.21.15.40:0/654326429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f850889f640 1 --2- 172.21.15.40:0/654326429 >> v2:172.21.15.184:3300/0 conn(0x7f85040b2340 0x7f85040cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f85090a0640 1 -- 172.21.15.40:0/654326429 >> v2:172.21.15.184:3300/0 conn(0x7f85040b2340 msgr2=0x7f85040cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f85090a0640 1 --2- 172.21.15.40:0/654326429 >> v2:172.21.15.184:3300/0 conn(0x7f85040b2340 0x7f85040cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f85090a0640 1 -- 172.21.15.40:0/654326429 >> v1:172.21.15.184:6789/0 conn(0x7f85040cd930 legacy=0x7f85040d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.356+0000 7f85090a0640 1 -- 172.21.15.40:0/654326429 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85040d1370 con 0x7f8504162810 2024-04-26T21:36:48.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.357+0000 7f85090a0640 1 --2- 172.21.15.40:0/654326429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f8504070350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f85040cd0a0 tx=0x7f84f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103e34bc86f9904 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.357+0000 7f84fbfff640 1 -- 172.21.15.40:0/654326429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4037070 con 0x7f8504162810 2024-04-26T21:36:48.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.357+0000 7f84fbfff640 1 -- 172.21.15.40:0/654326429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f4032440 con 0x7f8504162810 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.357+0000 7f84fbfff640 1 -- 172.21.15.40:0/654326429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4032810 con 0x7f8504162810 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.358+0000 7f850b304640 1 -- 172.21.15.40:0/654326429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 msgr2=0x7f8504070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.358+0000 7f850b304640 1 --2- 172.21.15.40:0/654326429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f8504070350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f85040cd0a0 tx=0x7f84f402a270 comp rx=0 tx=0).stop 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.358+0000 7f850b304640 1 -- 172.21.15.40:0/654326429 shutdown_connections 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.358+0000 7f850b304640 1 --2- 172.21.15.40:0/654326429 >> v2:172.21.15.184:3300/0 conn(0x7f85040b2340 0x7f85040cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.358+0000 7f850b304640 1 --2- 172.21.15.40:0/654326429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f8504070350 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.358+0000 7f850b304640 1 -- 172.21.15.40:0/654326429 >> 172.21.15.40:0/654326429 conn(0x7f85040d2da0 msgr2=0x7f85040d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.358+0000 7f850b304640 1 -- 172.21.15.40:0/654326429 shutdown_connections 2024-04-26T21:36:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.358+0000 7f850b304640 1 -- 172.21.15.40:0/654326429 wait complete. 2024-04-26T21:36:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.359+0000 7f850b304640 1 Processor -- start 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.359+0000 7f850b304640 1 -- start start 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f850b304640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85040b2340 0x7f850414cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f850b304640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85040cd930 0x7f850415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f850b304640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f850415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f850b304640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8504070890 con 0x7f8504162810 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f850b304640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85040d0470 con 0x7f85040b2340 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f850b304640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85040d2240 con 0x7f85040cd930 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85098a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f850415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85098a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f850415bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44438/0 (socket says 172.21.15.40:44438) 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85098a1640 1 -- 172.21.15.40:0/1394183573 learned_addr learned my addr 172.21.15.40:0/1394183573 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f850889f640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85040cd930 0x7f850415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85090a0640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85040b2340 0x7f850414cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85098a1640 1 -- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85040cd930 msgr2=0x7f850415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85098a1640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85040cd930 0x7f850415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85098a1640 1 -- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85040b2340 msgr2=0x7f850414cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85098a1640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85040b2340 0x7f850414cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85098a1640 1 -- 172.21.15.40:0/1394183573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8504160ba0 con 0x7f8504162810 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f85090a0640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85040b2340 0x7f850414cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.360+0000 7f850889f640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85040cd930 0x7f850415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.361+0000 7f85098a1640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f850415bd70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c730 tx=0x7f84fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.361+0000 7f84f9ffb640 1 -- 172.21.15.40:0/1394183573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc003b70 con 0x7f8504162810 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.361+0000 7f84f9ffb640 1 -- 172.21.15.40:0/1394183573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc003d10 con 0x7f8504162810 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.361+0000 7f84f9ffb640 1 -- 172.21.15.40:0/1394183573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00aa90 con 0x7f8504162810 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.361+0000 7f850b304640 1 -- 172.21.15.40:0/1394183573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8504160e80 con 0x7f8504162810 2024-04-26T21:36:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.361+0000 7f850b304640 1 -- 172.21.15.40:0/1394183573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8504161440 con 0x7f8504162810 2024-04-26T21:36:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.362+0000 7f84f9ffb640 1 -- 172.21.15.40:0/1394183573 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84fc00ac30 con 0x7f8504162810 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.363+0000 7f84f9ffb640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84e806c8f0 0x7f84e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.363+0000 7f84f9ffb640 1 -- 172.21.15.40:0/1394183573 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f84fc08bb50 con 0x7f8504162810 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.363+0000 7f85090a0640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84e806c8f0 0x7f84e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.364+0000 7f85090a0640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84e806c8f0 0x7f84e806edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.364+0000 7f850e019740 1 Processor -- start 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.364+0000 7f850e019740 1 -- start start 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850e019740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb1b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850e019740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb15e10 0x55e0ceb19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850e019740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb1a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850e019740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e0cea7fd80 con 0x55e0ceb15e10 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850e019740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e0cea398b0 con 0x55e0ceb1a0c0 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850e019740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e0cea7bd40 con 0x55e0ceb1af80 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb15e10 0x55e0ceb19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb15e10 0x55e0ceb19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44442/0 (socket says 172.21.15.40:44442) 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 -- 172.21.15.40:0/683821077 learned_addr learned my addr 172.21.15.40:0/683821077 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f85098a1640 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb1a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f85090a0640 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb1b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 -- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 msgr2=0x55e0ceb1b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb1b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 -- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 msgr2=0x55e0ceb1a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb1a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 -- 172.21.15.40:0/683821077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0cea0aa40 con 0x55e0ceb15e10 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f85090a0640 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb1b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.365+0000 7f850889f640 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb15e10 0x55e0ceb19b80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8500009be0 tx=0x7f8500009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab31b729ed494f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f84e6ffd640 1 -- 172.21.15.40:0/683821077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500019070 con 0x55e0ceb15e10 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f84e6ffd640 1 -- 172.21.15.40:0/683821077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500011cd0 con 0x55e0ceb15e10 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f84e6ffd640 1 -- 172.21.15.40:0/683821077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500004260 con 0x55e0ceb15e10 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 -- 172.21.15.40:0/683821077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb15e10 msgr2=0x55e0ceb19b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb15e10 0x55e0ceb19b80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8500009be0 tx=0x7f8500009190 comp rx=0 tx=0).stop 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 -- 172.21.15.40:0/683821077 shutdown_connections 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb15e10 0x55e0ceb19b80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 --2- 172.21.15.40:0/683821077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb1b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 -- 172.21.15.40:0/683821077 >> 172.21.15.40:0/683821077 conn(0x55e0ceb18080 msgr2=0x55e0ceb60b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 -- 172.21.15.40:0/683821077 shutdown_connections 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.366+0000 7f850e019740 1 -- 172.21.15.40:0/683821077 wait complete. 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.367+0000 7f850e019740 1 -- start start 2024-04-26T21:36:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.367+0000 7f850e019740 1 Processor -- start 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850e019740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850e019740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb3b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850e019740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb69030 0x55e0ceb3b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850e019740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e0ceb46b10 con 0x55e0ceb1af80 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850e019740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e0cea06d10 con 0x55e0ceb1a0c0 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850e019740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e0ceb45f90 con 0x55e0ceb69030 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850e019740 10 client.0 _ll_register_callbacks cb 0x55e0cea33210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb3b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb3b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44458/0 (socket says 172.21.15.40:44458) 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 -- 172.21.15.40:0/928385139 learned_addr learned my addr 172.21.15.40:0/928385139 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f85090a0640 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f85098a1640 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb69030 0x55e0ceb3b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 -- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb69030 msgr2=0x55e0ceb3b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb69030 0x55e0ceb3b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 -- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 msgr2=0x55e0ceb64790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb64790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 -- 172.21.15.40:0/928385139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0ce85b7d0 con 0x55e0ceb1af80 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f85098a1640 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb69030 0x55e0ceb3b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f850889f640 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb3b100 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8500011ac0 tx=0x7f8500004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fbf8792cfd7c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f84dffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f84dffff640 1 -- 172.21.15.40:0/928385139 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500019070 con 0x55e0ceb1af80 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.368+0000 7f84dffff640 1 -- 172.21.15.40:0/928385139 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f85000096d0 con 0x55e0ceb1af80 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 1 -- 172.21.15.40:0/928385139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8500011cd0 con 0x55e0ceb1af80 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f84dffff640 1 -- 172.21.15.40:0/928385139 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f850002f050 con 0x55e0ceb1af80 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 10 client.5766 fetch_fsmap learned FSMap version 23 2024-04-26T21:36:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 1 -- 172.21.15.40:0/928385139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55e0ce7913f0 con 0x55e0ceb1af80 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f84dffff640 1 -- 172.21.15.40:0/928385139 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500011cd0 con 0x55e0ceb1af80 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f84dffff640 1 -- 172.21.15.40:0/928385139 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f8500032020 con 0x55e0ceb1af80 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 10 client.5766 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 10 client.5766 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 20 client.5766 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55e0ceb651f0 0x55e0ceb655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 4 client.5766 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f850e019740 1 -- 172.21.15.40:0/928385139 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0cea8d990 con 0x55e0ceb651f0 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f85090a0640 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55e0ceb651f0 0x55e0ceb655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f84dffff640 10 client.5766 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.369+0000 7f85090a0640 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55e0ceb651f0 0x55e0ceb655f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b78303ef19f47 server_cookie=231cab99d21ca06 in_seq=0 out_seq=0 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f84dffff640 1 -- 172.21.15.40:0/928385139 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0cea8d990 con 0x55e0ceb651f0 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f84dffff640 10 client.5766 handle_command_reply: tid=0 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 client.5766 shutdown 2024-04-26T21:36:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 -- 172.21.15.40:0/928385139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 msgr2=0x55e0ceb3b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb3b100 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8500011ac0 tx=0x7f8500004730 comp rx=0 tx=0).stop 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 -- 172.21.15.40:0/928385139 shutdown_connections 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55e0ceb651f0 0x55e0ceb655f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb69030 0x55e0ceb3b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb3b100 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 --2- 172.21.15.40:0/928385139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f84dffff640 0 client.5766 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 -- 172.21.15.40:0/928385139 >> 172.21.15.40:0/928385139 conn(0x55e0ceb15e10 msgr2=0x55e0ceb1ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 -- 172.21.15.40:0/928385139 shutdown_connections 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 1 -- 172.21.15.40:0/928385139 wait complete. 2024-04-26T21:36:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.370+0000 7f850e019740 20 client.5766 trim_cache size 0 max 16384 2024-04-26T21:36:48.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.388+0000 7f850e019740 1 Processor -- start 2024-04-26T21:36:48.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.388+0000 7f850e019740 1 -- start start 2024-04-26T21:36:48.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850e019740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1a0c0 0x55e0ceb46180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850e019740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb1b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850e019740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb69030 0x55e0ceb14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850e019740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8504202050 con 0x55e0ceb1af80 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850e019740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8504201070 con 0x55e0ceb69030 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850e019740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f850414afa0 con 0x55e0ceb1a0c0 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb1b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb1b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44472/0 (socket says 172.21.15.40:44472) 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 -- 172.21.15.40:0/3262290702 learned_addr learned my addr 172.21.15.40:0/3262290702 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f85090a0640 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1a0c0 0x55e0ceb46180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f85098a1640 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb69030 0x55e0ceb14cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 -- 172.21.15.40:0/3262290702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1a0c0 msgr2=0x55e0ceb46180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1a0c0 0x55e0ceb46180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 -- 172.21.15.40:0/3262290702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb69030 msgr2=0x55e0ceb14cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb69030 0x55e0ceb14cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 -- 172.21.15.40:0/3262290702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0ce7510f0 con 0x55e0ceb1af80 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f850889f640 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb1b3f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8500006d90 tx=0x7f8500022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9485922ac86aa552 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.389+0000 7f84e67fc640 1 -- 172.21.15.40:0/3262290702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500038070 con 0x55e0ceb1af80 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f84e67fc640 1 -- 172.21.15.40:0/3262290702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500003910 con 0x55e0ceb1af80 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f84e67fc640 1 -- 172.21.15.40:0/3262290702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500003e20 con 0x55e0ceb1af80 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 -- 172.21.15.40:0/3262290702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 msgr2=0x55e0ceb1b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb1b3f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8500006d90 tx=0x7f8500022000 comp rx=0 tx=0).stop 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 -- 172.21.15.40:0/3262290702 shutdown_connections 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb69030 0x55e0ceb14cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb1af80 0x55e0ceb1b3f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 --2- 172.21.15.40:0/3262290702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1a0c0 0x55e0ceb46180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 -- 172.21.15.40:0/3262290702 >> 172.21.15.40:0/3262290702 conn(0x55e0ceb15e10 msgr2=0x55e0ceb681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 -- 172.21.15.40:0/3262290702 shutdown_connections 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.390+0000 7f850e019740 1 -- 172.21.15.40:0/3262290702 wait complete. 2024-04-26T21:36:48.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.391+0000 7f850e019740 1 -- start start 2024-04-26T21:36:48.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.391+0000 7f850e019740 1 Processor -- start 2024-04-26T21:36:48.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb83b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb69030 0x55e0ceb845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e0ceb63f80 con 0x55e0ceb69030 2024-04-26T21:36:48.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8504200a20 con 0x55e0ceb1a0c0 2024-04-26T21:36:48.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e0ceb1d520 con 0x55e0ceb1af80 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 10 client.0 _ll_register_callbacks cb 0x7f84fc060050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb69030 0x55e0ceb845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb69030 0x55e0ceb845e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44486/0 (socket says 172.21.15.40:44486) 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 -- 172.21.15.40:0/955499649 learned_addr learned my addr 172.21.15.40:0/955499649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85090a0640 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb83b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850889f640 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 -- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 msgr2=0x55e0ceb840a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb840a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 -- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 msgr2=0x55e0ceb83b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb83b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 -- 172.21.15.40:0/955499649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0cea404b0 con 0x55e0ceb69030 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85090a0640 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb83b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f85098a1640 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb69030 0x55e0ceb845e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0596c0 tx=0x7f84fc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3101ad47862862bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:36:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f84dffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f84dffff640 1 -- 172.21.15.40:0/955499649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc05dae0 con 0x55e0ceb69030 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f84dffff640 1 -- 172.21.15.40:0/955499649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f84fc061e80 con 0x55e0ceb69030 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 1 -- 172.21.15.40:0/955499649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8500003910 con 0x55e0ceb69030 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f84dffff640 1 -- 172.21.15.40:0/955499649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f84fc062310 con 0x55e0ceb69030 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 10 client.5772 fetch_fsmap learned FSMap version 23 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.392+0000 7f850e019740 1 -- 172.21.15.40:0/955499649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55e0ceb3ea90 con 0x55e0ceb69030 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f84dffff640 1 -- 172.21.15.40:0/955499649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500003910 con 0x55e0ceb69030 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f84dffff640 1 -- 172.21.15.40:0/955499649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f84fc058b80 con 0x55e0ceb69030 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 10 client.5772 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 10 client.5772 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 20 client.5772 populate_metadata read hostname 'smithi040' 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55e0cebcb3d0 0x55e0cebce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 4 client.5772 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 1 -- 172.21.15.40:0/955499649 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e0cea87800 con 0x55e0cebcb3d0 2024-04-26T21:36:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f85090a0640 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55e0cebcb3d0 0x55e0cebce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f85090a0640 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55e0cebcb3d0 0x55e0cebce620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8f9abc9bb4e771 server_cookie=fd084dd4643e129c in_seq=0 out_seq=0 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f84dffff640 10 client.5772 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f84dffff640 1 -- 172.21.15.40:0/955499649 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55e0cea87800 con 0x55e0cebcb3d0 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f84dffff640 10 client.5772 handle_command_reply: tid=0 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 1 client.5772 shutdown 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 1 -- 172.21.15.40:0/955499649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb69030 msgr2=0x55e0ceb845e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.393+0000 7f850e019740 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb69030 0x55e0ceb845e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0596c0 tx=0x7f84fc00cf00 comp rx=0 tx=0).stop 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 1 -- 172.21.15.40:0/955499649 shutdown_connections 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55e0cebcb3d0 0x55e0cebce620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0ceb69030 0x55e0ceb845e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0ceb1af80 0x55e0ceb840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f84dffff640 0 client.5772 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 1 --2- 172.21.15.40:0/955499649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0ceb1a0c0 0x55e0ceb83b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 1 -- 172.21.15.40:0/955499649 >> 172.21.15.40:0/955499649 conn(0x55e0ceb15e10 msgr2=0x55e0ceb681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 1 -- 172.21.15.40:0/955499649 shutdown_connections 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 1 -- 172.21.15.40:0/955499649 wait complete. 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850e019740 20 client.5772 trim_cache size 0 max 16384 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stdout: "e9f8ac55-1f55-475a-a7d7-aa7881d90858": { 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/subdir", 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "e9f8ac55-1f55-475a-a7d7-aa7881d90858", 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "repair" 2024-04-26T21:36:48.401 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:48.402 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:36:48.402 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 -- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84e806c8f0 msgr2=0x7f84e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84e806c8f0 0x7f84e806edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f402e000 comp rx=0 tx=0).stop 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 -- 172.21.15.40:0/1394183573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 msgr2=0x7f850415bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f850415bd70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c730 tx=0x7f84fc00cbf0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 -- 172.21.15.40:0/1394183573 shutdown_connections 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f84e806c8f0 0x7f84e806edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8504162810 0x7f850415bd70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85040cd930 0x7f850415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 --2- 172.21.15.40:0/1394183573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85040b2340 0x7f850414cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 -- 172.21.15.40:0/1394183573 >> 172.21.15.40:0/1394183573 conn(0x7f85040d2da0 msgr2=0x7f85040b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 -- 172.21.15.40:0/1394183573 shutdown_connections 2024-04-26T21:36:48.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:36:48.394+0000 7f850b304640 1 -- 172.21.15.40:0/1394183573 wait complete. 2024-04-26T21:36:48.406 INFO:tasks.cephfs.filesystem:scrub status for tag:e9f8ac55-1f55-475a-a7d7-aa7881d90858 - {'path': '/subdir', 'tag': 'e9f8ac55-1f55-475a-a7d7-aa7881d90858', 'options': 'repair'} 2024-04-26T21:37:10.233 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:37:10.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:37:18.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:18.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-26T21:37:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.523+0000 7fb3bbb6a640 1 Processor -- start 2024-04-26T21:37:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.523+0000 7fb3bbb6a640 1 -- start start 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3bbb6a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3b4162880 0x7fb3b40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3bbb6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b40b5f20 0x7fb3b40d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3bbb6a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b40b7e50 con 0x7fb3b40d39f0 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3bbb6a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b40d0580 con 0x7fb3b4162880 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3bbb6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b40d1800 con 0x7fb3b40b5f20 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3ba107640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb3b40d39f0 0x7fb3b40b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38546/0 (socket says 172.21.15.40:38546) 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3ba107640 1 -- 172.21.15.40:0/3747224396 learned_addr learned my addr 172.21.15.40:0/3747224396 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3b9105640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b40b5f20 0x7fb3b40d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.524+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3747224396 >> v2:172.21.15.184:3300/0 conn(0x7fb3b4162880 0x7fb3b40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402071586 0 0) 0x7fb3b40b7e50 con 0x7fb3b40d39f0 2024-04-26T21:37:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3a0003520 con 0x7fb3b40d39f0 2024-04-26T21:37:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3529615525 0 0) 0x7fb3a0003520 con 0x7fb3b40d39f0 2024-04-26T21:37:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b40b5f20 msgr2=0x7fb3b40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b40b5f20 0x7fb3b40d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 >> v2:172.21.15.184:3300/0 conn(0x7fb3b4162880 msgr2=0x7fb3b40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 --2- 172.21.15.40:0/3747224396 >> v2:172.21.15.184:3300/0 conn(0x7fb3b4162880 0x7fb3b40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b40d07d0 con 0x7fb3b40d39f0 2024-04-26T21:37:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb3ac0019f0 con 0x7fb3b40d39f0 2024-04-26T21:37:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3747224396 >> v2:172.21.15.184:3300/0 conn(0x7fb3b4162880 0x7fb3b40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 >> v1:172.21.15.40:6789/0 conn(0x7fb3b40d39f0 legacy=0x7fb3b40b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a00039b0 0x7fb3a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b40b5f20 0x7fb3a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0004d40 0x7fb3a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40d0580 con 0x7fb3a0004d40 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40d1800 con 0x7fb3a00039b0 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a000b9c0 con 0x7fb3b40b5f20 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3ba107640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0004d40 0x7fb3a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a00039b0 0x7fb3a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3b9105640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b40b5f20 0x7fb3a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3ba107640 1 -- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b40b5f20 msgr2=0x7fb3a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3ba107640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b40b5f20 0x7fb3a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3ba107640 1 -- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a00039b0 msgr2=0x7fb3a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3ba107640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a00039b0 0x7fb3a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.525+0000 7fb3ba107640 1 -- 172.21.15.40:0/3747224396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a0003520 con 0x7fb3a0004d40 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.526+0000 7fb3ba107640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0004d40 0x7fb3a00051c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0013d0 tx=0x7fb3a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a7c322f95bb93f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.526+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00a4e0 con 0x7fb3a0004d40 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.526+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac02f070 con 0x7fb3a0004d40 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.526+0000 7fb3b8904640 1 -- 172.21.15.40:0/3747224396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac02bce0 con 0x7fb3a0004d40 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.527+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/3747224396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0004d40 msgr2=0x7fb3a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.527+0000 7fb3bbb6a640 1 --2- 172.21.15.40:0/3747224396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0004d40 0x7fb3a00051c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0013d0 tx=0x7fb3a0005830 comp rx=0 tx=0).stop 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.527+0000 7fb3ba107640 1 -- 172.21.15.40:0/3747224396 reap_dead start 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.527+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/3747224396 shutdown_connections 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.527+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/3747224396 >> 172.21.15.40:0/3747224396 conn(0x7fb3b40d4f60 msgr2=0x7fb3b40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.527+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/3747224396 shutdown_connections 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.527+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/3747224396 wait complete. 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.528+0000 7fb3bbb6a640 1 Processor -- start 2024-04-26T21:37:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.528+0000 7fb3bbb6a640 1 -- start start 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3bbb6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a0004d40 0x7fb3b4157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3bbb6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b415bfb0 0x7fb3b4154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3bbb6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b415c410 0x7fb3b4155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3bbb6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4162c60 con 0x7fb3b415c410 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3bbb6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40d0d90 con 0x7fb3a0004d40 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3bbb6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b4162de0 con 0x7fb3b415bfb0 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3ba107640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b415c410 0x7fb3b4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3ba107640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b415c410 0x7fb3b4155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32832/0 (socket says 172.21.15.40:32832) 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3ba107640 1 -- 172.21.15.40:0/4267014846 learned_addr learned my addr 172.21.15.40:0/4267014846 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3b9906640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a0004d40 0x7fb3b4157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3b9105640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b415bfb0 0x7fb3b4154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3ba107640 1 -- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b415bfb0 msgr2=0x7fb3b4154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3ba107640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b415bfb0 0x7fb3b4154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3ba107640 1 -- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a0004d40 msgr2=0x7fb3b4157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3ba107640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a0004d40 0x7fb3b4157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.529+0000 7fb3ba107640 1 -- 172.21.15.40:0/4267014846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b40cc380 con 0x7fb3b415c410 2024-04-26T21:37:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.530+0000 7fb3ba107640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b415c410 0x7fb3b4155380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00a0b0 tx=0x7fb3ac001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.530+0000 7fb3aa7fc640 1 -- 172.21.15.40:0/4267014846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0019e0 con 0x7fb3b415c410 2024-04-26T21:37:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.530+0000 7fb3aa7fc640 1 -- 172.21.15.40:0/4267014846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac00a4c0 con 0x7fb3b415c410 2024-04-26T21:37:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.530+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/4267014846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b40cc660 con 0x7fb3b415c410 2024-04-26T21:37:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.530+0000 7fb3aa7fc640 1 -- 172.21.15.40:0/4267014846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac004d60 con 0x7fb3b415c410 2024-04-26T21:37:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.530+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/4267014846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b40ccc20 con 0x7fb3b415c410 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.532+0000 7fb3aa7fc640 1 -- 172.21.15.40:0/4267014846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3ac034470 con 0x7fb3b415c410 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.532+0000 7fb3aa7fc640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb39806ca90 0x7fb39806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.532+0000 7fb3aa7fc640 1 -- 172.21.15.40:0/4267014846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fb3ac0abbc0 con 0x7fb3b415c410 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.533+0000 7fb3b9906640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb39806ca90 0x7fb39806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.533+0000 7fb3b9906640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb39806ca90 0x7fb39806ef50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009bf0 tx=0x7fb3b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.533+0000 7fb3be522740 1 Processor -- start 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.533+0000 7fb3be522740 1 -- start start 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3be522740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3be522740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca99ce10 0x55b6ca9a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3be522740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 0x55b6ca9a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3be522740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca906d80 con 0x55b6ca99ce10 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3be522740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca8c08b0 con 0x55b6ca9a1f80 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3be522740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b6ca902d40 con 0x55b6ca9a10c0 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3b9105640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca99ce10 0x55b6ca9a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3b9105640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca99ce10 0x55b6ca9a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32848/0 (socket says 172.21.15.40:32848) 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.534+0000 7fb3b9105640 1 -- 172.21.15.40:0/3930711497 learned_addr learned my addr 172.21.15.40:0/3930711497 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3ba107640 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 0x55b6ca9a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3b9105640 1 -- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 msgr2=0x55b6ca9a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3b9105640 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 0x55b6ca9a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3b9105640 1 -- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 msgr2=0x55b6ca9a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3b9105640 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3b9105640 1 -- 172.21.15.40:0/3930711497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ca891a40 con 0x55b6ca99ce10 2024-04-26T21:37:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3ba107640 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 0x55b6ca9a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3b9105640 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca99ce10 0x55b6ca9a0b80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40099f0 tx=0x7fb3a4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f18593b12b4b641 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3977fe640 1 -- 172.21.15.40:0/3930711497 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a401a070 con 0x55b6ca99ce10 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3977fe640 1 -- 172.21.15.40:0/3930711497 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a4012d80 con 0x55b6ca99ce10 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.535+0000 7fb3977fe640 1 -- 172.21.15.40:0/3930711497 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4015690 con 0x55b6ca99ce10 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 -- 172.21.15.40:0/3930711497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca99ce10 msgr2=0x55b6ca9a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca99ce10 0x55b6ca9a0b80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40099f0 tx=0x7fb3a4009190 comp rx=0 tx=0).stop 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 -- 172.21.15.40:0/3930711497 shutdown_connections 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 0x55b6ca9a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca99ce10 0x55b6ca9a0b80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 --2- 172.21.15.40:0/3930711497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 -- 172.21.15.40:0/3930711497 >> 172.21.15.40:0/3930711497 conn(0x55b6ca99f080 msgr2=0x55b6ca9e7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 -- 172.21.15.40:0/3930711497 shutdown_connections 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 -- 172.21.15.40:0/3930711497 wait complete. 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 -- start start 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.536+0000 7fb3be522740 1 Processor -- start 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.537+0000 7fb3be522740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.537+0000 7fb3be522740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.537+0000 7fb3be522740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6ca9c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.537+0000 7fb3be522740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca9cdb10 con 0x55b6ca9a1f80 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.537+0000 7fb3be522740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca88dd10 con 0x55b6ca9a10c0 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.537+0000 7fb3be522740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b6ca9ccf90 con 0x55b6ca9bea50 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.537+0000 7fb3b9105640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.537+0000 7fb3be522740 10 client.0 _ll_register_callbacks cb 0x55b6ca8ba210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9105640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32864/0 (socket says 172.21.15.40:32864) 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9105640 1 -- 172.21.15.40:0/2317440909 learned_addr learned my addr 172.21.15.40:0/2317440909 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9906640 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3ba107640 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6ca9c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9105640 1 -- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 msgr2=0x55b6ca9c1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9105640 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6ca9c1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9105640 1 -- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 msgr2=0x55b6ca9eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9105640 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9105640 1 -- 172.21.15.40:0/2317440909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ca6e27d0 con 0x55b6ca9a1f80 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9906640 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3b9105640 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eb4c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4019000 tx=0x7fb3a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddba5aaff449bacd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb394ff9640 1 -- 172.21.15.40:0/2317440909 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a402d060 con 0x55b6ca9a1f80 2024-04-26T21:37:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb394ff9640 1 -- 172.21.15.40:0/2317440909 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fb3a4012b80 con 0x55b6ca9a1f80 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3be522740 1 -- 172.21.15.40:0/2317440909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a4012d80 con 0x55b6ca9a1f80 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb394ff9640 1 -- 172.21.15.40:0/2317440909 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb3a4033050 con 0x55b6ca9a1f80 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3be522740 10 client.5784 fetch_fsmap learned FSMap version 23 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.538+0000 7fb3be522740 1 -- 172.21.15.40:0/2317440909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55b6ca6183f0 con 0x55b6ca9a1f80 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb394ff9640 1 -- 172.21.15.40:0/2317440909 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a4012d80 con 0x55b6ca9a1f80 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb394ff9640 1 -- 172.21.15.40:0/2317440909 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fb3a4036020 con 0x55b6ca9a1f80 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb3be522740 10 client.5784 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb3be522740 10 client.5784 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb3be522740 20 client.5784 populate_metadata read hostname 'smithi040' 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb3be522740 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55b6ca9c22b0 0x55b6ca9ef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb3be522740 4 client.5784 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb3be522740 1 -- 172.21.15.40:0/2317440909 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ca914990 con 0x55b6ca9c22b0 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb3b9906640 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55b6ca9c22b0 0x55b6ca9ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb3b9906640 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55b6ca9c22b0 0x55b6ca9ef330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd044f2757c2e6fe server_cookie=f65fc7b6bae62ec1 in_seq=0 out_seq=0 2024-04-26T21:37:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.539+0000 7fb394ff9640 10 client.5784 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb394ff9640 1 -- 172.21.15.40:0/2317440909 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6ca914990 con 0x55b6ca9c22b0 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb394ff9640 10 client.5784 handle_command_reply: tid=0 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 client.5784 shutdown 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 -- 172.21.15.40:0/2317440909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 msgr2=0x55b6ca9eb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eb4c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4019000 tx=0x7fb3a40046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 -- 172.21.15.40:0/2317440909 shutdown_connections 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55b6ca9c22b0 0x55b6ca9ef330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6ca9c1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb394ff9640 0 client.5784 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eb4c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 --2- 172.21.15.40:0/2317440909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 -- 172.21.15.40:0/2317440909 >> 172.21.15.40:0/2317440909 conn(0x55b6ca99ce10 msgr2=0x55b6ca9a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 -- 172.21.15.40:0/2317440909 shutdown_connections 2024-04-26T21:37:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 1 -- 172.21.15.40:0/2317440909 wait complete. 2024-04-26T21:37:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.540+0000 7fb3be522740 20 client.5784 trim_cache size 0 max 16384 2024-04-26T21:37:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.558+0000 7fb3be522740 1 Processor -- start 2024-04-26T21:37:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.558+0000 7fb3be522740 1 -- start start 2024-04-26T21:37:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3be522740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 0x55b6ca9c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3be522740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3be522740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9bea50 0x55b6ca9ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3be522740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b42024f0 con 0x55b6ca9bea50 2024-04-26T21:37:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3be522740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4201550 con 0x55b6ca9a1f80 2024-04-26T21:37:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3be522740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b4156eb0 con 0x55b6ca9a10c0 2024-04-26T21:37:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9bea50 0x55b6ca9ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9bea50 0x55b6ca9ea500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32880/0 (socket says 172.21.15.40:32880) 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 -- 172.21.15.40:0/763590511 learned_addr learned my addr 172.21.15.40:0/763590511 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 -- 172.21.15.40:0/763590511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 msgr2=0x55b6ca9c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 --2- 172.21.15.40:0/763590511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 0x55b6ca9c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 -- 172.21.15.40:0/763590511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 msgr2=0x55b6ca9eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 --2- 172.21.15.40:0/763590511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 -- 172.21.15.40:0/763590511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ca5d80f0 con 0x55b6ca9bea50 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.559+0000 7fb3ba107640 1 --2- 172.21.15.40:0/763590511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9bea50 0x55b6ca9ea500 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0a9ef0 tx=0x7fb3ac079110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fa628b30ce35ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb396ffd640 1 -- 172.21.15.40:0/763590511 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac045040 con 0x55b6ca9bea50 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb396ffd640 1 -- 172.21.15.40:0/763590511 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac02f050 con 0x55b6ca9bea50 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb396ffd640 1 -- 172.21.15.40:0/763590511 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac07daa0 con 0x55b6ca9bea50 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 -- 172.21.15.40:0/763590511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9bea50 msgr2=0x55b6ca9ea500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 --2- 172.21.15.40:0/763590511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9bea50 0x55b6ca9ea500 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0a9ef0 tx=0x7fb3ac079110 comp rx=0 tx=0).stop 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 -- 172.21.15.40:0/763590511 shutdown_connections 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 --2- 172.21.15.40:0/763590511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9bea50 0x55b6ca9ea500 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 --2- 172.21.15.40:0/763590511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a1f80 0x55b6ca9eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 --2- 172.21.15.40:0/763590511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9a10c0 0x55b6ca9c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 -- 172.21.15.40:0/763590511 >> 172.21.15.40:0/763590511 conn(0x55b6ca99ce10 msgr2=0x55b6ca998580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 -- 172.21.15.40:0/763590511 shutdown_connections 2024-04-26T21:37:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.560+0000 7fb3be522740 1 -- 172.21.15.40:0/763590511 wait complete. 2024-04-26T21:37:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.561+0000 7fb3be522740 1 -- start start 2024-04-26T21:37:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.561+0000 7fb3be522740 1 Processor -- start 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3be522740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3be522740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6caa61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3be522740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6caa61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3be522740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4200f30 con 0x55b6ca9a1f80 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3be522740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca9e79e0 con 0x55b6ca9a10c0 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3be522740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b6ca9ecfb0 con 0x55b6ca9bea50 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3be522740 10 client.0 _ll_register_callbacks cb 0x7fb3a4009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6caa61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6caa61230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32888/0 (socket says 172.21.15.40:32888) 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 -- 172.21.15.40:0/3244312119 learned_addr learned my addr 172.21.15.40:0/3244312119 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3ba107640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6caa61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 -- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 msgr2=0x55b6caa61770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6caa61770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 -- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 msgr2=0x55b6ca9eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 -- 172.21.15.40:0/3244312119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ca8c74b0 con 0x55b6ca9a1f80 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3ba107640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6caa61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb3b9105640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6caa61230 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4013010 tx=0x7fb3a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1485523da79f94fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.562+0000 7fb394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb394ff9640 1 -- 172.21.15.40:0/3244312119 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a403b020 con 0x55b6ca9a1f80 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb394ff9640 1 -- 172.21.15.40:0/3244312119 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fb3a4038ca0 con 0x55b6ca9a1f80 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 1 -- 172.21.15.40:0/3244312119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3ac02f050 con 0x55b6ca9a1f80 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb394ff9640 1 -- 172.21.15.40:0/3244312119 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb3a401a040 con 0x55b6ca9a1f80 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 10 client.5790 fetch_fsmap learned FSMap version 23 2024-04-26T21:37:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 1 -- 172.21.15.40:0/3244312119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55b6ca9bef50 con 0x55b6ca9a1f80 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb394ff9640 1 -- 172.21.15.40:0/3244312119 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac02f050 con 0x55b6ca9a1f80 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb394ff9640 1 -- 172.21.15.40:0/3244312119 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fb3a4036040 con 0x55b6ca9a1f80 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 10 client.5790 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 10 client.5790 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 20 client.5790 populate_metadata read hostname 'smithi040' 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55b6caa61e70 0x55b6caa013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 4 client.5790 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3be522740 1 -- 172.21.15.40:0/3244312119 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6ca90e800 con 0x55b6caa61e70 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.563+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55b6caa61e70 0x55b6caa013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3b9906640 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55b6caa61e70 0x55b6caa013a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4c34ca1f0ae778f server_cookie=234bd4380184a7b1 in_seq=0 out_seq=0 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb394ff9640 10 client.5790 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb394ff9640 1 -- 172.21.15.40:0/3244312119 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b6ca90e800 con 0x55b6caa61e70 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb394ff9640 10 client.5790 handle_command_reply: tid=0 2024-04-26T21:37:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 client.5790 shutdown 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 -- 172.21.15.40:0/3244312119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 msgr2=0x55b6caa61230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6caa61230 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4013010 tx=0x7fb3a4004730 comp rx=0 tx=0).stop 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 -- 172.21.15.40:0/3244312119 shutdown_connections 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55b6caa61e70 0x55b6caa013a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b6ca9bea50 0x55b6caa61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b6ca9a1f80 0x55b6caa61230 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 --2- 172.21.15.40:0/3244312119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6ca9a10c0 0x55b6ca9eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb394ff9640 0 client.5790 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 -- 172.21.15.40:0/3244312119 >> 172.21.15.40:0/3244312119 conn(0x55b6ca99ce10 msgr2=0x55b6ca9c1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 -- 172.21.15.40:0/3244312119 shutdown_connections 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 1 -- 172.21.15.40:0/3244312119 wait complete. 2024-04-26T21:37:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.564+0000 7fb3be522740 20 client.5790 trim_cache size 0 max 16384 2024-04-26T21:37:18.572 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:37:18.572 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "no active scrubs running", 2024-04-26T21:37:18.572 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": {} 2024-04-26T21:37:18.572 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb39806ca90 msgr2=0x7fb39806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb39806ca90 0x7fb39806ef50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009bf0 tx=0x7fb3b0008040 comp rx=0 tx=0).stop 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/4267014846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b415c410 msgr2=0x7fb3b4155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b415c410 0x7fb3b4155380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00a0b0 tx=0x7fb3ac001dc0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/4267014846 shutdown_connections 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb39806ca90 0x7fb39806ef50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b415c410 0x7fb3b4155380 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3b415bfb0 0x7fb3b4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 --2- 172.21.15.40:0/4267014846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a0004d40 0x7fb3b4157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/4267014846 >> 172.21.15.40:0/4267014846 conn(0x7fb3b40d4f60 msgr2=0x7fb3b40b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/4267014846 shutdown_connections 2024-04-26T21:37:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.565+0000 7fb3bbb6a640 1 -- 172.21.15.40:0/4267014846 wait complete. 2024-04-26T21:37:18.576 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:37:18.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:37:18.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.676+0000 7f514eb12640 1 Processor -- start 2024-04-26T21:37:18.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.676+0000 7f514eb12640 1 -- start start 2024-04-26T21:37:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.677+0000 7f514eb12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51480d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.677+0000 7f514eb12640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 0x7f51480d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.677+0000 7f514eb12640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51480d25a0 con 0x7f51480d2130 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.677+0000 7f514eb12640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f51480d0e20 con 0x7f51480d1770 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.677+0000 7f514eb12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51480b1d70 con 0x7f514815f3a0 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.677+0000 7f514d0af640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51480d2130 0x7f51480d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42412/0 (socket says 172.21.15.40:42412) 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.677+0000 7f514d0af640 1 -- 172.21.15.40:0/474224369 learned_addr learned my addr 172.21.15.40:0/474224369 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f514c8ae640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51480d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f5147fff640 1 --2- 172.21.15.40:0/474224369 >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 0x7f51480d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2102587222 0 0) 0x7f5138001110 con 0x7f51480d2130 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5134003520 con 0x7f51480d2130 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1178586694 0 0) 0x7f5134003520 con 0x7f51480d2130 2024-04-26T21:37:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 msgr2=0x7f51480d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51480d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 msgr2=0x7f51480d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 --2- 172.21.15.40:0/474224369 >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 0x7f51480d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51480b2050 con 0x7f51480d2130 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f5138001840 con 0x7f51480d2130 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f5147fff640 1 --2- 172.21.15.40:0/474224369 >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 0x7f51480d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 >> v1:172.21.15.40:6789/0 conn(0x7f51480d2130 legacy=0x7f51480d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f5134004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5134004d40 0x7f51340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51480d0e20 con 0x7f5134003f30 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51480b1d70 con 0x7f5134004d40 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f513400b9c0 con 0x7f514815f3a0 2024-04-26T21:37:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f5147fff640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f5134004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f514c8ae640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f514d0af640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5134004d40 0x7f51340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f5147fff640 1 -- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 msgr2=0x7f51340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f5147fff640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f5147fff640 1 -- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5134004d40 msgr2=0x7f51340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f5147fff640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5134004d40 0x7f51340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.678+0000 7f5147fff640 1 -- 172.21.15.40:0/474224369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5134003520 con 0x7f5134003f30 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.679+0000 7f5147fff640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f5134004800 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5130009c90 tx=0x7f5134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2e6c5012da04b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.679+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130036070 con 0x7f5134003f30 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.679+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5130031400 con 0x7f5134003f30 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.679+0000 7f51477fe640 1 -- 172.21.15.40:0/474224369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130009860 con 0x7f5134003f30 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.680+0000 7f514eb12640 1 -- 172.21.15.40:0/474224369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 msgr2=0x7f5134004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.680+0000 7f514eb12640 1 --2- 172.21.15.40:0/474224369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f5134004800 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5130009c90 tx=0x7f5134005830 comp rx=0 tx=0).stop 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.680+0000 7f514d0af640 1 -- 172.21.15.40:0/474224369 reap_dead start 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.680+0000 7f514eb12640 1 -- 172.21.15.40:0/474224369 shutdown_connections 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.680+0000 7f514eb12640 1 -- 172.21.15.40:0/474224369 >> 172.21.15.40:0/474224369 conn(0x7f51480d4f90 msgr2=0x7f51480d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.680+0000 7f514eb12640 1 -- 172.21.15.40:0/474224369 shutdown_connections 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.680+0000 7f514eb12640 1 -- 172.21.15.40:0/474224369 wait complete. 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.681+0000 7f514eb12640 1 Processor -- start 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.681+0000 7f514eb12640 1 -- start start 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514eb12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f51481f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514eb12640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5148155750 0x7f514815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514eb12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815ffc0 0x7f5148161d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514eb12640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f514815f900 con 0x7f5134003f30 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514eb12640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f514815f780 con 0x7f5148155750 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514eb12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51480b3860 con 0x7f514815ffc0 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f51481f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f51481f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34944/0 (socket says 172.21.15.40:34944) 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 -- 172.21.15.40:0/3858816588 learned_addr learned my addr 172.21.15.40:0/3858816588 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 -- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815ffc0 msgr2=0x7f5148161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815ffc0 0x7f5148161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 -- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5148155750 msgr2=0x7f514815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5148155750 0x7f514815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 -- 172.21.15.40:0/3858816588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51481f5d30 con 0x7f5134003f30 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f514c8ae640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f51481f57f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5140007f30 tx=0x7f5140011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.682+0000 7f51457fa640 1 -- 172.21.15.40:0/3858816588 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514001a020 con 0x7f5134003f30 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.683+0000 7f51457fa640 1 -- 172.21.15.40:0/3858816588 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5140016470 con 0x7f5134003f30 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.683+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51480b9260 con 0x7f5134003f30 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.683+0000 7f51457fa640 1 -- 172.21.15.40:0/3858816588 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140016840 con 0x7f5134003f30 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.683+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51480b9820 con 0x7f5134003f30 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.683+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51481603e0 con 0x7f5134003f30 2024-04-26T21:37:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.684+0000 7f51457fa640 1 -- 172.21.15.40:0/3858816588 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5140005d10 con 0x7f5134003f30 2024-04-26T21:37:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.684+0000 7f51457fa640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512806c920 0x7f512806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.684+0000 7f51457fa640 1 -- 172.21.15.40:0/3858816588 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f514008cae0 con 0x7f5134003f30 2024-04-26T21:37:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.685+0000 7f5147fff640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512806c920 0x7f512806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.685+0000 7f5147fff640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512806c920 0x7f512806ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5130033000 tx=0x7f513002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.689+0000 7f51457fa640 1 -- 172.21.15.40:0/3858816588 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5140024030 con 0x7f5134003f30 2024-04-26T21:37:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.914+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f51480b6860 con 0x7f5134003f30 2024-04-26T21:37:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.914+0000 7f51457fa640 1 -- 172.21.15.40:0/3858816588 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f51481f5d30 con 0x7f5134003f30 2024-04-26T21:37:18.916 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:37:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512806c920 msgr2=0x7f512806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512806c920 0x7f512806ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5130033000 tx=0x7f513002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 msgr2=0x7f51481f57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f51481f57f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5140007f30 tx=0x7f5140011660 comp rx=0 tx=0).stop 2024-04-26T21:37:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 shutdown_connections 2024-04-26T21:37:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512806c920 0x7f512806ede0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815ffc0 0x7f5148161d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5148155750 0x7f514815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 --2- 172.21.15.40:0/3858816588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5134003f30 0x7f51481f57f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 >> 172.21.15.40:0/3858816588 conn(0x7f51480d4f90 msgr2=0x7f514806cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 shutdown_connections 2024-04-26T21:37:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:18.916+0000 7f514eb12640 1 -- 172.21.15.40:0/3858816588 wait complete. 2024-04-26T21:37:18.929 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:37:18.929 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:37:18.929 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:37:18.929 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:37:18.929 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:18.929 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:37:18.929 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:37:18.930 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:37:18.930 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:37:18.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:18.930 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:37:18.947 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:37:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:37:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:37:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:37:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:18.947 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:37:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42609sec preferred_lft 42609sec 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:18.948 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:37:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:18.950 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:37:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:37:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:37:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:18.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:18.950 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:37:19.011 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:37:19.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:19.013 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:37:19.069 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:37:19.070 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:37:19.070 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:37:19.070 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:37:19.070 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:37:19.070 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:37:19.070 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:37:19.070 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:37:19.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:37:19.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:37:19.071 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:37:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:37:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:37:19.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:19.196 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:37:19.211 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:37:19.211 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:37:19.211 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:37:19.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:19.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:37:19.276 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:37:19.276 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:19.332 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:37:19.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:37:19.332 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:19.388 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:37:19.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:19.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:37:19.470 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:37:19.471 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:37:19.478 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:37:19.478 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:37:19.478 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:37:19.478 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:37:19.478 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:37:19.479 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:37:19.479 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:37:19.479 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:37:19.479 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:37:19.479 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:37:19.481 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:37:19.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:19.481 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:19.553 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:37:19.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:37:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:37:19.650 INFO:teuthology.orchestra.run.smithi040.stdout:client5084 2024-04-26T21:37:19.650 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:37:19.650 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:19.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:19.666 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-26T21:37:19.725 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:37:19.725 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:37:19.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.809+0000 7fb4497b7f40 1 Processor -- start 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.809+0000 7fb4497b7f40 1 -- start start 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.809+0000 7fb4497b7f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b3d7560 0x55ad3b45fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.809+0000 7fb4497b7f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b461620 0x55ad3b464020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.809+0000 7fb4497b7f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b45edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.809+0000 7fb4497b7f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ad3b462ac0 con 0x55ad3b461a00 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.809+0000 7fb4497b7f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ad3b334e40 con 0x55ad3b461620 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.809+0000 7fb4497b7f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ad3b462190 con 0x55ad3b3d7560 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b45edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b45edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34956/0 (socket says 172.21.15.40:34956) 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 -- 172.21.15.40:0/2177573723 learned_addr learned my addr 172.21.15.40:0/2177573723 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb4472b3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b3d7560 0x55ad3b45fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb446ab2640 1 --2- 172.21.15.40:0/2177573723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b461620 0x55ad3b464020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 -- 172.21.15.40:0/2177573723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b3d7560 msgr2=0x55ad3b45fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 --2- 172.21.15.40:0/2177573723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b3d7560 0x55ad3b45fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 -- 172.21.15.40:0/2177573723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b461620 msgr2=0x55ad3b464020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 --2- 172.21.15.40:0/2177573723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b461620 0x55ad3b464020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 -- 172.21.15.40:0/2177573723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3b3bc4a0 con 0x55ad3b461a00 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb447ab4640 1 --2- 172.21.15.40:0/2177573723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b45edd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb4400016b0 tx=0x7fb440001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc658661a92070e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb445ab0640 1 -- 172.21.15.40:0/2177573723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400062e0 con 0x55ad3b461a00 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb445ab0640 1 -- 172.21.15.40:0/2177573723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb440003650 con 0x55ad3b461a00 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.810+0000 7fb445ab0640 1 -- 172.21.15.40:0/2177573723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440003a60 con 0x55ad3b461a00 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 -- 172.21.15.40:0/2177573723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 msgr2=0x55ad3b45edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 --2- 172.21.15.40:0/2177573723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b45edd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb4400016b0 tx=0x7fb440001b70 comp rx=0 tx=0).stop 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 -- 172.21.15.40:0/2177573723 shutdown_connections 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 --2- 172.21.15.40:0/2177573723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b45edd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 --2- 172.21.15.40:0/2177573723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b461620 0x55ad3b464020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 --2- 172.21.15.40:0/2177573723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b3d7560 0x55ad3b45fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 -- 172.21.15.40:0/2177573723 >> 172.21.15.40:0/2177573723 conn(0x55ad3b3b78b0 msgr2=0x55ad3b38fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 -- 172.21.15.40:0/2177573723 shutdown_connections 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 -- 172.21.15.40:0/2177573723 wait complete. 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 Processor -- start 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.811+0000 7fb4497b7f40 1 -- start start 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb4497b7f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b3d7560 0x55ad3b38fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb4497b7f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b461620 0x55ad3b45c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb4497b7f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b3d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb4497b7f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ad3b45f740 con 0x55ad3b461a00 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb4497b7f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ad3b45f460 con 0x55ad3b3d7560 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb4497b7f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ad3b3bc0e0 con 0x55ad3b461620 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb447ab4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b3d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb447ab4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b3d2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34960/0 (socket says 172.21.15.40:34960) 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb447ab4640 1 -- 172.21.15.40:0/128081072 learned_addr learned my addr 172.21.15.40:0/128081072 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb446ab2640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b461620 0x55ad3b45c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb4472b3640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b3d7560 0x55ad3b38fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb447ab4640 1 -- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b461620 msgr2=0x55ad3b45c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb447ab4640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ad3b461620 0x55ad3b45c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb447ab4640 1 -- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b3d7560 msgr2=0x55ad3b38fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.812+0000 7fb447ab4640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad3b3d7560 0x55ad3b38fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.813+0000 7fb447ab4640 1 -- 172.21.15.40:0/128081072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3b38c430 con 0x55ad3b461a00 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.813+0000 7fb447ab4640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b3d2de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55ad3b464590 tx=0x7fb440001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.813+0000 7fb444aae640 1 -- 172.21.15.40:0/128081072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440024070 con 0x55ad3b461a00 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.813+0000 7fb444aae640 1 -- 172.21.15.40:0/128081072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb440019040 con 0x55ad3b461a00 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.813+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ad3b38c710 con 0x55ad3b461a00 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.813+0000 7fb444aae640 1 -- 172.21.15.40:0/128081072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44001f6c0 con 0x55ad3b461a00 2024-04-26T21:37:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.813+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3b3cd960 con 0x55ad3b461a00 2024-04-26T21:37:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.814+0000 7fb444aae640 1 -- 172.21.15.40:0/128081072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb440010070 con 0x55ad3b461a00 2024-04-26T21:37:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.815+0000 7fb444aae640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43406b2d0 0x7fb43406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.815+0000 7fb444aae640 1 -- 172.21.15.40:0/128081072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fb44008ba80 con 0x55ad3b461a00 2024-04-26T21:37:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.815+0000 7fb4497b7f40 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ad3b4f8b00 0x55ad3b4f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.815+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55ad3b4f94a0 con 0x55ad3b4f8b00 2024-04-26T21:37:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.815+0000 7fb4472b3640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43406b2d0 0x7fb43406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.815+0000 7fb446ab2640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ad3b4f8b00 0x55ad3b4f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.816+0000 7fb4472b3640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43406b2d0 0x7fb43406d790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb438005cf0 tx=0x7fb43800c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.816+0000 7fb446ab2640 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ad3b4f8b00 0x55ad3b4f8f60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb446ab2640 1 -- 172.21.15.40:0/128081072 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v41'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb43c056450 con 0x55ad3b4f8b00 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ad3b4f8b00 msgr2=0x55ad3b4f8f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ad3b4f8b00 0x55ad3b4f8f60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43406b2d0 msgr2=0x7fb43406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43406b2d0 0x7fb43406d790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb438005cf0 tx=0x7fb43800c040 comp rx=0 tx=0).stop 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 msgr2=0x55ad3b3d2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 --2- 172.21.15.40:0/128081072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ad3b461a00 0x55ad3b3d2de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55ad3b464590 tx=0x7fb440001250 comp rx=0 tx=0).stop 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb447ab4640 1 -- 172.21.15.40:0/128081072 reap_dead start 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 shutdown_connections 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 >> 172.21.15.40:0/128081072 conn(0x55ad3b3b78b0 msgr2=0x55ad3b3ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 shutdown_connections 2024-04-26T21:37:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.817+0000 7fb4497b7f40 1 -- 172.21.15.40:0/128081072 wait complete. 2024-04-26T21:37:19.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:19.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.923+0000 7f3f0650a640 1 Processor -- start 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.923+0000 7f3f0650a640 1 -- start start 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3f0650a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f000b2340 0x7f3f000cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3f0650a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f000cd930 0x7f3f000d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3f0650a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f000d10d0 con 0x7f3f00162810 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3f0650a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3f000d0750 con 0x7f3f000b2340 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3f0650a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000d16a0 con 0x7f3f000cd930 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3eff7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f000b2340 0x7f3f000cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3f04aa7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f000cd930 0x7f3f000d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3efffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f3f00162810 0x7f3f00070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55628/0 (socket says 172.21.15.40:55628) 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3efffff640 1 -- 172.21.15.40:0/1625669626 learned_addr learned my addr 172.21.15.40:0/1625669626 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.924+0000 7f3efeffd640 1 -- 172.21.15.40:0/1625669626 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2571756209 0 0) 0x7f3f000d10d0 con 0x7f3f00162810 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.925+0000 7f3efeffd640 1 -- 172.21.15.40:0/1625669626 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3eec003520 con 0x7f3f00162810 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.925+0000 7f3eff7fe640 1 -- 172.21.15.40:0/1625669626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f000cd930 msgr2=0x7f3f000d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:19.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.925+0000 7f3eff7fe640 1 --2- 172.21.15.40:0/1625669626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f000cd930 0x7f3f000d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.925+0000 7f3eff7fe640 1 -- 172.21.15.40:0/1625669626 >> v1:172.21.15.184:6789/0 conn(0x7f3f00162810 legacy=0x7f3f00070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.925+0000 7f3eff7fe640 1 -- 172.21.15.40:0/1625669626 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f000d1370 con 0x7f3f000b2340 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.925+0000 7f3f04aa7640 1 --2- 172.21.15.40:0/1625669626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f000cd930 0x7f3f000d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.925+0000 7f3eff7fe640 1 --2- 172.21.15.40:0/1625669626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 0x7f3f000cd3f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f000d1820 tx=0x7f3ef002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3acbfe0161f2f08b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3efeffd640 1 -- 172.21.15.40:0/1625669626 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0037070 con 0x7f3f000b2340 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3efeffd640 1 -- 172.21.15.40:0/1625669626 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0032440 con 0x7f3f000b2340 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3efeffd640 1 -- 172.21.15.40:0/1625669626 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0032810 con 0x7f3f000b2340 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3f0650a640 1 -- 172.21.15.40:0/1625669626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 msgr2=0x7f3f000cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1625669626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 0x7f3f000cd3f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f000d1820 tx=0x7f3ef002a180 comp rx=0 tx=0).stop 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3f0650a640 1 -- 172.21.15.40:0/1625669626 shutdown_connections 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1625669626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f000cd930 0x7f3f000d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1625669626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 0x7f3f000cd3f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3f0650a640 1 -- 172.21.15.40:0/1625669626 >> 172.21.15.40:0/1625669626 conn(0x7f3f000d2da0 msgr2=0x7f3f000d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3f0650a640 1 -- 172.21.15.40:0/1625669626 shutdown_connections 2024-04-26T21:37:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.926+0000 7f3f0650a640 1 -- 172.21.15.40:0/1625669626 wait complete. 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.927+0000 7f3f0650a640 1 Processor -- start 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.927+0000 7f3f0650a640 1 -- start start 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3f0650a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 0x7f3f0014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3f0650a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cd930 0x7f3f00149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3f0650a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f00162810 0x7f3f001f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3f0650a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000d2070 con 0x7f3f000cd930 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3f0650a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000d0470 con 0x7f3f000b2340 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3f0650a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000d21f0 con 0x7f3f00162810 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cd930 0x7f3f00149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cd930 0x7f3f00149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34974/0 (socket says 172.21.15.40:34974) 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3eff7fe640 1 -- 172.21.15.40:0/1705078842 learned_addr learned my addr 172.21.15.40:0/1705078842 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3f04aa7640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f00162810 0x7f3f001f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.928+0000 7f3efffff640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 0x7f3f0014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3eff7fe640 1 -- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f00162810 msgr2=0x7f3f001f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3eff7fe640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f00162810 0x7f3f001f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3eff7fe640 1 -- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 msgr2=0x7f3f0014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3eff7fe640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 0x7f3f0014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3eff7fe640 1 -- 172.21.15.40:0/1705078842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f00161360 con 0x7f3f000cd930 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3f04aa7640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f00162810 0x7f3f001f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3eff7fe640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cd930 0x7f3f00149530 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0000c00 tx=0x7f3ef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3efcff9640 1 -- 172.21.15.40:0/1705078842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0041020 con 0x7f3f000cd930 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0014ca70 con 0x7f3f000cd930 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3efcff9640 1 -- 172.21.15.40:0/1705078842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef003d700 con 0x7f3f000cd930 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3efcff9640 1 -- 172.21.15.40:0/1705078842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0037040 con 0x7f3f000cd930 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.929+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0014fa80 con 0x7f3f000cd930 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.930+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f000694f0 con 0x7f3f000cd930 2024-04-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.930+0000 7f3efcff9640 1 -- 172.21.15.40:0/1705078842 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ef0044020 con 0x7f3f000cd930 2024-04-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.931+0000 7f3efcff9640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee006c8f0 0x7f3ee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.931+0000 7f3efcff9640 1 -- 172.21.15.40:0/1705078842 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f3ef00ac500 con 0x7f3f000cd930 2024-04-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.931+0000 7f3efffff640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee006c8f0 0x7f3ee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.932+0000 7f3efffff640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee006c8f0 0x7f3ee006edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005880 tx=0x7f3ee8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:19.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:19.935+0000 7f3efcff9640 1 -- 172.21.15.40:0/1705078842 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ef004b030 con 0x7f3f000cd930 2024-04-26T21:37:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.163+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f001f8230 con 0x7f3f000cd930 2024-04-26T21:37:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.163+0000 7f3efcff9640 1 -- 172.21.15.40:0/1705078842 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5185 (secure 0 0 0) 0x7f3ef007a740 con 0x7f3f000cd930 2024-04-26T21:37:20.165 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:20.165 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:42.691649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5619,"qdb_cluster":[5619]},"id":2}]} 2024-04-26T21:37:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 23 2024-04-26T21:37:20.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee006c8f0 msgr2=0x7f3ee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee006c8f0 0x7f3ee006edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005880 tx=0x7f3ee8009040 comp rx=0 tx=0).stop 2024-04-26T21:37:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cd930 msgr2=0x7f3f00149530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cd930 0x7f3f00149530 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0000c00 tx=0x7f3ef00046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 shutdown_connections 2024-04-26T21:37:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3ee006c8f0 0x7f3ee006edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f00162810 0x7f3f001f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cd930 0x7f3f00149530 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 --2- 172.21.15.40:0/1705078842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b2340 0x7f3f0014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.165+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 >> 172.21.15.40:0/1705078842 conn(0x7f3f000d2da0 msgr2=0x7f3f000b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.166+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 shutdown_connections 2024-04-26T21:37:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.166+0000 7f3f0650a640 1 -- 172.21.15.40:0/1705078842 wait complete. 2024-04-26T21:37:20.178 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 4285318383 2024-04-26T21:37:20.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.281+0000 7f391d597640 1 Processor -- start 2024-04-26T21:37:20.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.282+0000 7f391d597640 1 -- start start 2024-04-26T21:37:20.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.282+0000 7f391d597640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f39180cc480 0x7f3918072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.282+0000 7f391d597640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391815d7c0 con 0x7f3918072780 2024-04-26T21:37:20.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.282+0000 7f391d597640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391815daa0 con 0x7f3918071e60 2024-04-26T21:37:20.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.282+0000 7f391d597640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f391815dd80 con 0x7f39180cc480 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.282+0000 7f39177fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3918072780 0x7f3918166ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42414/0 (socket says 172.21.15.40:42414) 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.282+0000 7f39177fe640 1 -- 172.21.15.40:0/4174363661 learned_addr learned my addr 172.21.15.40:0/4174363661 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f39167fc640 1 --2- 172.21.15.40:0/4174363661 >> v2:172.21.15.184:3300/0 conn(0x7f39180cc480 0x7f3918072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f3915ffb640 1 -- 172.21.15.40:0/4174363661 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1170542996 0 0) 0x7f391815d7c0 con 0x7f3918072780 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f3915ffb640 1 -- 172.21.15.40:0/4174363661 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3904003520 con 0x7f3918072780 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f3915ffb640 1 -- 172.21.15.40:0/4174363661 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1555727138 0 0) 0x7f391815daa0 con 0x7f3918071e60 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f3915ffb640 1 -- 172.21.15.40:0/4174363661 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f391815d7c0 con 0x7f3918071e60 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f39167fc640 1 -- 172.21.15.40:0/4174363661 >> v1:172.21.15.184:6789/0 conn(0x7f3918071e60 legacy=0x7f391806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f3915ffb640 1 -- 172.21.15.40:0/4174363661 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3500292663 0 0) 0x7f3904003520 con 0x7f3918072780 2024-04-26T21:37:20.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f39167fc640 1 -- 172.21.15.40:0/4174363661 >> v1:172.21.15.40:6789/0 conn(0x7f3918072780 legacy=0x7f3918166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.283+0000 7f39167fc640 1 -- 172.21.15.40:0/4174363661 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391815e060 con 0x7f39180cc480 2024-04-26T21:37:20.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.284+0000 7f39167fc640 1 --2- 172.21.15.40:0/4174363661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39180cc480 0x7f3918072240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f39000097a0 tx=0x7f390002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6d033cf22b553ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.284+0000 7f3915ffb640 1 -- 172.21.15.40:0/4174363661 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900037070 con 0x7f39180cc480 2024-04-26T21:37:20.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.284+0000 7f3915ffb640 1 -- 172.21.15.40:0/4174363661 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390002ac50 con 0x7f39180cc480 2024-04-26T21:37:20.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.284+0000 7f3915ffb640 1 -- 172.21.15.40:0/4174363661 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39080019f0 con 0x7f39180cc480 2024-04-26T21:37:20.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.284+0000 7f391d597640 1 -- 172.21.15.40:0/4174363661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39180cc480 msgr2=0x7f3918072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.284+0000 7f391d597640 1 --2- 172.21.15.40:0/4174363661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39180cc480 0x7f3918072240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f39000097a0 tx=0x7f390002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.285+0000 7f391d597640 1 -- 172.21.15.40:0/4174363661 shutdown_connections 2024-04-26T21:37:20.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.285+0000 7f391d597640 1 --2- 172.21.15.40:0/4174363661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39180cc480 0x7f3918072240 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.285+0000 7f391d597640 1 -- 172.21.15.40:0/4174363661 >> 172.21.15.40:0/4174363661 conn(0x7f39180d0be0 msgr2=0x7f39180d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.285+0000 7f391d597640 1 -- 172.21.15.40:0/4174363661 shutdown_connections 2024-04-26T21:37:20.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.285+0000 7f391d597640 1 -- 172.21.15.40:0/4174363661 wait complete. 2024-04-26T21:37:20.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.286+0000 7f391d597640 1 Processor -- start 2024-04-26T21:37:20.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.286+0000 7f391d597640 1 -- start start 2024-04-26T21:37:20.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f391d597640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3918071e60 0x7f3918153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f391d597640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3918072780 0x7f39180c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f391d597640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39180cc480 0x7f391815ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f391d597640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f391815d3f0 con 0x7f3918072780 2024-04-26T21:37:20.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f391d597640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3918072bd0 con 0x7f39180cc480 2024-04-26T21:37:20.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f391d597640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f391815d570 con 0x7f3918071e60 2024-04-26T21:37:20.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3918072780 0x7f39180c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3918072780 0x7f39180c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34988/0 (socket says 172.21.15.40:34988) 2024-04-26T21:37:20.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 -- 172.21.15.40:0/2017471345 learned_addr learned my addr 172.21.15.40:0/2017471345 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:20.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f3916ffd640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3918071e60 0x7f3918153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 -- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3918071e60 msgr2=0x7f3918153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3918071e60 0x7f3918153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 -- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39180cc480 msgr2=0x7f391815ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39180cc480 0x7f391815ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 -- 172.21.15.40:0/2017471345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39180c9eb0 con 0x7f3918072780 2024-04-26T21:37:20.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f3916ffd640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3918071e60 0x7f3918153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:20.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f39167fc640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3918072780 0x7f39180c9420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3900035040 tx=0x7f39000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f390f7fe640 1 -- 172.21.15.40:0/2017471345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900037050 con 0x7f3918072780 2024-04-26T21:37:20.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f390f7fe640 1 -- 172.21.15.40:0/2017471345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3900043070 con 0x7f3918072780 2024-04-26T21:37:20.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f391d597640 1 -- 172.21.15.40:0/2017471345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39180ca190 con 0x7f3918072780 2024-04-26T21:37:20.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f390f7fe640 1 -- 172.21.15.40:0/2017471345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900006e10 con 0x7f3918072780 2024-04-26T21:37:20.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.287+0000 7f391d597640 1 -- 172.21.15.40:0/2017471345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39180ca750 con 0x7f3918072780 2024-04-26T21:37:20.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.289+0000 7f390f7fe640 1 -- 172.21.15.40:0/2017471345 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f390003c830 con 0x7f3918072780 2024-04-26T21:37:20.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.289+0000 7f390f7fe640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f38f806c8f0 0x7f38f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.289+0000 7f390f7fe640 1 -- 172.21.15.40:0/2017471345 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f39000ac680 con 0x7f3918072780 2024-04-26T21:37:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.290+0000 7f3916ffd640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f38f806c8f0 0x7f38f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.290+0000 7f3916ffd640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f38f806c8f0 0x7f38f806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f39100011e0 tx=0x7f3910009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.290+0000 7f3920385740 1 Processor -- start 2024-04-26T21:37:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.290+0000 7f3920385740 1 -- start start 2024-04-26T21:37:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f3920385740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bd4f0 0x55f52296fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f3920385740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f3920385740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f52299fed0 0x55f522974ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f3920385740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f5228d9ad0 con 0x55f5229a0610 2024-04-26T21:37:20.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f3920385740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f522893a00 con 0x55f5229bd4f0 2024-04-26T21:37:20.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f3920385740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f5228d5a90 con 0x55f52299fed0 2024-04-26T21:37:20.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f39167fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f39167fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35004/0 (socket says 172.21.15.40:35004) 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f39167fc640 1 -- 172.21.15.40:0/3100065447 learned_addr learned my addr 172.21.15.40:0/3100065447 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f3916ffd640 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bd4f0 0x55f52296fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f39177fe640 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f52299fed0 0x55f522974ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f39167fc640 1 -- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f52299fed0 msgr2=0x55f522974ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.291+0000 7f39167fc640 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f52299fed0 0x55f522974ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f39167fc640 1 -- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bd4f0 msgr2=0x55f52296fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f39167fc640 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bd4f0 0x55f52296fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f39167fc640 1 -- 172.21.15.40:0/3100065447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5226b7710 con 0x55f5229a0610 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f3916ffd640 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bd4f0 0x55f52296fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f39167fc640 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299f990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f39000aa850 tx=0x7f39000763b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae297ddd93697a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f390cff9640 1 -- 172.21.15.40:0/3100065447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390007a110 con 0x55f5229a0610 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f390cff9640 1 -- 172.21.15.40:0/3100065447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3900038040 con 0x55f5229a0610 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f390cff9640 1 -- 172.21.15.40:0/3100065447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390007e4c0 con 0x55f5229a0610 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f3920385740 1 -- 172.21.15.40:0/3100065447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 msgr2=0x55f52299f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.292+0000 7f3920385740 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299f990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f39000aa850 tx=0x7f39000763b0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 -- 172.21.15.40:0/3100065447 shutdown_connections 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f52299fed0 0x55f522974ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299f990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 --2- 172.21.15.40:0/3100065447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bd4f0 0x55f52296fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 -- 172.21.15.40:0/3100065447 >> 172.21.15.40:0/3100065447 conn(0x55f5229720f0 msgr2=0x55f5229ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 -- 172.21.15.40:0/3100065447 shutdown_connections 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 -- 172.21.15.40:0/3100065447 wait complete. 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 -- start start 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.293+0000 7f3920385740 1 Processor -- start 2024-04-26T21:37:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3920385740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3920385740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 0x55f52299f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3920385740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3920385740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f5229742a0 con 0x55f5229a0610 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3920385740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f522861000 con 0x55f5229bc1c0 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3920385740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f522973690 con 0x55f5229bd4f0 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3920385740 10 client.0 _ll_register_callbacks cb 0x55f52287c620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3916ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3916ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35014/0 (socket says 172.21.15.40:35014) 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3916ffd640 1 -- 172.21.15.40:0/3375940658 learned_addr learned my addr 172.21.15.40:0/3375940658 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f39167fc640 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 0x55f52299f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f39177fe640 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3916ffd640 1 -- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 msgr2=0x55f52299f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3916ffd640 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 0x55f52299f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3916ffd640 1 -- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 msgr2=0x55f5229ba940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3916ffd640 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229ba940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f3916ffd640 1 -- 172.21.15.40:0/3375940658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f522717ea0 con 0x55f5229a0610 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.294+0000 7f39177fe640 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229ba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3916ffd640 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299eb90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f391000dee0 tx=0x7f3910000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4337d65d023a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f38f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f38f5ffb640 1 -- 172.21.15.40:0/3375940658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910026060 con 0x55f5229a0610 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f38f5ffb640 1 -- 172.21.15.40:0/3375940658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f3910005320 con 0x55f5229a0610 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 1 -- 172.21.15.40:0/3375940658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3900038040 con 0x55f5229a0610 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f38f5ffb640 1 -- 172.21.15.40:0/3375940658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3910014e30 con 0x55f5229a0610 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 10 client.5817 fetch_fsmap learned FSMap version 23 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 1 -- 172.21.15.40:0/3375940658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55f5225ab000 con 0x55f5229a0610 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f38f5ffb640 1 -- 172.21.15.40:0/3375940658 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3900038040 con 0x55f5229a0610 2024-04-26T21:37:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f38f5ffb640 1 -- 172.21.15.40:0/3375940658 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f3910003610 con 0x55f5229a0610 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 10 client.5817 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 10 client.5817 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 20 client.5817 populate_metadata read hostname 'smithi040' 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55f52298e120 0x55f52298e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 4 client.5817 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f3920385740 1 -- 172.21.15.40:0/3375940658 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5228e76e0 con 0x55f52298e120 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.295+0000 7f39167fc640 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55f52298e120 0x55f52298e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f38f5ffb640 10 client.5817 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f39167fc640 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55f52298e120 0x55f52298e520 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23220d88950023d server_cookie=639afef69b47852b in_seq=0 out_seq=0 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f38f5ffb640 1 -- 172.21.15.40:0/3375940658 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5228e76e0 con 0x55f52298e120 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f38f5ffb640 10 client.5817 handle_command_reply: tid=0 2024-04-26T21:37:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f3920385740 1 client.5817 shutdown 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f3920385740 1 -- 172.21.15.40:0/3375940658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 msgr2=0x55f52299eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f3920385740 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299eb90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f391000dee0 tx=0x7f3910000ed0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f3920385740 1 -- 172.21.15.40:0/3375940658 shutdown_connections 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f3920385740 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55f52298e120 0x55f52298e520 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f3920385740 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229ba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f3920385740 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 0x55f52299f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f38f5ffb640 0 client.5817 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.296+0000 7f3920385740 1 --2- 172.21.15.40:0/3375940658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f52299eb90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.297+0000 7f3920385740 1 -- 172.21.15.40:0/3375940658 >> 172.21.15.40:0/3375940658 conn(0x55f52299fed0 msgr2=0x55f522974730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.297+0000 7f3920385740 1 -- 172.21.15.40:0/3375940658 shutdown_connections 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.297+0000 7f3920385740 1 -- 172.21.15.40:0/3375940658 wait complete. 2024-04-26T21:37:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.297+0000 7f3920385740 20 client.5817 trim_cache size 0 max 16384 2024-04-26T21:37:20.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.310+0000 7f3920385740 1 Processor -- start 2024-04-26T21:37:20.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.310+0000 7f3920385740 1 -- start start 2024-04-26T21:37:20.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f3920385740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229a0610 0x55f5229ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f3920385740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bc1c0 0x55f5229b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f3920385740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229bd4f0 0x55f52296fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f3920385740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39181fea20 con 0x55f5229bd4f0 2024-04-26T21:37:20.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f3920385740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39181f3d70 con 0x55f5229a0610 2024-04-26T21:37:20.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f3920385740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39181fe400 con 0x55f5229bc1c0 2024-04-26T21:37:20.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229bd4f0 0x55f52296fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229bd4f0 0x55f52296fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35028/0 (socket says 172.21.15.40:35028) 2024-04-26T21:37:20.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 -- 172.21.15.40:0/1190908325 learned_addr learned my addr 172.21.15.40:0/1190908325 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:20.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 -- 172.21.15.40:0/1190908325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bc1c0 msgr2=0x55f5229b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 --2- 172.21.15.40:0/1190908325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bc1c0 0x55f5229b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 -- 172.21.15.40:0/1190908325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229a0610 msgr2=0x55f5229ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 --2- 172.21.15.40:0/1190908325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229a0610 0x55f5229ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 -- 172.21.15.40:0/1190908325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f522973230 con 0x55f5229bd4f0 2024-04-26T21:37:20.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f39177fe640 1 --2- 172.21.15.40:0/1190908325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229bd4f0 0x55f52296fa20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3908002110 tx=0x7f3908007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef262463a5c4657 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f38f7fff640 1 -- 172.21.15.40:0/1190908325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908014400 con 0x55f5229bd4f0 2024-04-26T21:37:20.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.311+0000 7f38f7fff640 1 -- 172.21.15.40:0/1190908325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3908019070 con 0x55f5229bd4f0 2024-04-26T21:37:20.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f38f7fff640 1 -- 172.21.15.40:0/1190908325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39080147a0 con 0x55f5229bd4f0 2024-04-26T21:37:20.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 -- 172.21.15.40:0/1190908325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229bd4f0 msgr2=0x55f52296fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 --2- 172.21.15.40:0/1190908325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229bd4f0 0x55f52296fa20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3908002110 tx=0x7f3908007420 comp rx=0 tx=0).stop 2024-04-26T21:37:20.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 -- 172.21.15.40:0/1190908325 shutdown_connections 2024-04-26T21:37:20.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 --2- 172.21.15.40:0/1190908325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229bd4f0 0x55f52296fa20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 --2- 172.21.15.40:0/1190908325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bc1c0 0x55f5229b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 --2- 172.21.15.40:0/1190908325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229a0610 0x55f5229ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 -- 172.21.15.40:0/1190908325 >> 172.21.15.40:0/1190908325 conn(0x55f52299fed0 msgr2=0x55f522975850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 -- 172.21.15.40:0/1190908325 shutdown_connections 2024-04-26T21:37:20.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 -- 172.21.15.40:0/1190908325 wait complete. 2024-04-26T21:37:20.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.312+0000 7f3920385740 1 -- start start 2024-04-26T21:37:20.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3920385740 1 Processor -- start 2024-04-26T21:37:20.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3920385740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f522a223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3920385740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3920385740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 0x55f5229be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3920385740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f5229a0b00 con 0x55f5229a0610 2024-04-26T21:37:20.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3920385740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f52298ece0 con 0x55f5229bc1c0 2024-04-26T21:37:20.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3920385740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f5229bb140 con 0x55f5229bd4f0 2024-04-26T21:37:20.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3920385740 10 client.0 _ll_register_callbacks cb 0x7f391000dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:37:20.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3916ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f522a223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3916ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f522a223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35044/0 (socket says 172.21.15.40:35044) 2024-04-26T21:37:20.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f3916ffd640 1 -- 172.21.15.40:0/1752184407 learned_addr learned my addr 172.21.15.40:0/1752184407 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:20.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.313+0000 7f39177fe640 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 0x55f5229be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f39167fc640 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3916ffd640 1 -- 172.21.15.40:0/1752184407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 msgr2=0x55f5229be5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3916ffd640 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 0x55f5229be5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3916ffd640 1 -- 172.21.15.40:0/1752184407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 msgr2=0x55f5229c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3916ffd640 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3916ffd640 1 -- 172.21.15.40:0/1752184407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5229c95d0 con 0x55f5229a0610 2024-04-26T21:37:20.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3916ffd640 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f522a223f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3910000f10 tx=0x7f3910001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f67ac6c013579dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f38f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:37:20.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f38f5ffb640 1 -- 172.21.15.40:0/1752184407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910039060 con 0x55f5229a0610 2024-04-26T21:37:20.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f38f5ffb640 1 -- 172.21.15.40:0/1752184407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f3910031030 con 0x55f5229a0610 2024-04-26T21:37:20.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3920385740 1 -- 172.21.15.40:0/1752184407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3908019070 con 0x55f5229a0610 2024-04-26T21:37:20.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f38f5ffb640 1 -- 172.21.15.40:0/1752184407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f391003f050 con 0x55f5229a0610 2024-04-26T21:37:20.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3920385740 10 client.5823 fetch_fsmap learned FSMap version 23 2024-04-26T21:37:20.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f3920385740 1 -- 172.21.15.40:0/1752184407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55f5229b9e10 con 0x55f5229a0610 2024-04-26T21:37:20.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.314+0000 7f38f5ffb640 1 -- 172.21.15.40:0/1752184407 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3908019070 con 0x55f5229a0610 2024-04-26T21:37:20.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f38f5ffb640 1 -- 172.21.15.40:0/1752184407 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f3910042020 con 0x55f5229a0610 2024-04-26T21:37:20.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 10 client.5823 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:37:20.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 10 client.5823 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:37:20.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 20 client.5823 populate_metadata read hostname 'smithi040' 2024-04-26T21:37:20.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55f522974630 0x55f522974a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 4 client.5823 mds_command: new command op to 5619 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 4285318383}] 2024-04-26T21:37:20.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 -- 172.21.15.40:0/1752184407 --> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] -- command(tid 0: {"prefix": "damage rm", "damage_id": 4285318383}) -- 0x55f5228e1550 con 0x55f522974630 2024-04-26T21:37:20.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f39167fc640 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55f522974630 0x55f522974a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f39167fc640 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55f522974630 0x55f522974a10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e1cd91ff7ae024 server_cookie=80f39a61ee6cd47f in_seq=0 out_seq=0 2024-04-26T21:37:20.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f38f5ffb640 10 client.5823 ms_handle_connect on v2:172.21.15.40:6833/1366266306 2024-04-26T21:37:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f38f5ffb640 1 -- 172.21.15.40:0/1752184407 <== mds.0 v2:172.21.15.40:6833/1366266306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55f5228e1550 con 0x55f522974630 2024-04-26T21:37:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f38f5ffb640 10 client.5823 handle_command_reply: tid=0 2024-04-26T21:37:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 client.5823 shutdown 2024-04-26T21:37:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 -- 172.21.15.40:0/1752184407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 msgr2=0x55f522a223f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f522a223f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3910000f10 tx=0x7f3910001150 comp rx=0 tx=0).stop 2024-04-26T21:37:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 -- 172.21.15.40:0/1752184407 shutdown_connections 2024-04-26T21:37:20.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.40:6833/1366266306,v1:172.21.15.40:6835/1366266306] conn(0x55f522974630 0x55f522974a10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5229bd4f0 0x55f5229be5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f38f5ffb640 0 client.5823 ms_handle_reset on v2:172.21.15.40:6833/1366266306 2024-04-26T21:37:20.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5229bc1c0 0x55f5229c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 --2- 172.21.15.40:0/1752184407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5229a0610 0x55f522a223f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 -- 172.21.15.40:0/1752184407 >> 172.21.15.40:0/1752184407 conn(0x55f52299fed0 msgr2=0x55f52298ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 -- 172.21.15.40:0/1752184407 shutdown_connections 2024-04-26T21:37:20.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 1 -- 172.21.15.40:0/1752184407 wait complete. 2024-04-26T21:37:20.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.315+0000 7f3920385740 20 client.5823 trim_cache size 0 max 16384 2024-04-26T21:37:20.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 -- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f38f806c8f0 msgr2=0x7f38f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f38f806c8f0 0x7f38f806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f39100011e0 tx=0x7f3910009040 comp rx=0 tx=0).stop 2024-04-26T21:37:20.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 -- 172.21.15.40:0/2017471345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3918072780 msgr2=0x7f39180c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3918072780 0x7f39180c9420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3900035040 tx=0x7f39000046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 -- 172.21.15.40:0/2017471345 shutdown_connections 2024-04-26T21:37:20.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f38f806c8f0 0x7f38f806edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39180cc480 0x7f391815ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3918072780 0x7f39180c9420 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 --2- 172.21.15.40:0/2017471345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3918071e60 0x7f3918153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 -- 172.21.15.40:0/2017471345 >> 172.21.15.40:0/2017471345 conn(0x7f39180d0be0 msgr2=0x7f3918159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 -- 172.21.15.40:0/2017471345 shutdown_connections 2024-04-26T21:37:20.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.316+0000 7f391d597640 1 -- 172.21.15.40:0/2017471345 wait complete. 2024-04-26T21:37:20.348 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:37:20.348 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:37:20.348 DEBUG:teuthology.orchestra.run.smithi040:> import errno 2024-04-26T21:37:20.348 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:37:20.348 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:37:20.348 DEBUG:teuthology.orchestra.run.smithi040:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-04-26T21:37:20.349 DEBUG:teuthology.orchestra.run.smithi040:> f.close() 2024-04-26T21:37:20.349 DEBUG:teuthology.orchestra.run.smithi040:> except IOError as e: 2024-04-26T21:37:20.349 DEBUG:teuthology.orchestra.run.smithi040:> sys.exit(errno.EIO) 2024-04-26T21:37:20.349 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:37:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:37:20.430 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:37:20.430 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:37:20.510 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:37:20.563 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:37:20.564 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:37:20.564 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:37:20.564 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:20.580 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:37:20.580 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:20.636 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:20.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.775+0000 7f7f08751640 1 Processor -- start 2024-04-26T21:37:20.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.775+0000 7f7f08751640 1 -- start start 2024-04-26T21:37:20.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f08751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 0x7f7f000d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f08751640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f000d1770 0x7f7f000d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f08751640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f000d25a0 con 0x7f7f000d2130 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f08751640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7f000d0e20 con 0x7f7f000d1770 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f08751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f000b1d70 con 0x7f7f0015f3a0 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f05cec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f000d1770 0x7f7f000d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f06cee640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7f000d2130 0x7f7f000d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42426/0 (socket says 172.21.15.40:42426) 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f06cee640 1 -- 172.21.15.40:0/623062547 learned_addr learned my addr 172.21.15.40:0/623062547 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f064ed640 1 --2- 172.21.15.40:0/623062547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 0x7f7f000d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f05cec640 1 -- 172.21.15.40:0/623062547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 msgr2=0x7f7f000d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f05cec640 1 --2- 172.21.15.40:0/623062547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 0x7f7f000d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f05cec640 1 -- 172.21.15.40:0/623062547 >> v1:172.21.15.40:6789/0 conn(0x7f7f000d2130 legacy=0x7f7f000d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.776+0000 7f7f05cec640 1 -- 172.21.15.40:0/623062547 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f000b2050 con 0x7f7f000d1770 2024-04-26T21:37:20.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.777+0000 7f7f05cec640 1 --2- 172.21.15.40:0/623062547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d1770 0x7f7f000d1bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00097a0 tx=0x7f7ef002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be76083cf4586930 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.777+0000 7f7f054eb640 1 -- 172.21.15.40:0/623062547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0037070 con 0x7f7f000d1770 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.777+0000 7f7f054eb640 1 -- 172.21.15.40:0/623062547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef0032440 con 0x7f7f000d1770 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.777+0000 7f7f054eb640 1 -- 172.21.15.40:0/623062547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0032810 con 0x7f7f000d1770 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.778+0000 7f7f08751640 1 -- 172.21.15.40:0/623062547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d1770 msgr2=0x7f7f000d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.778+0000 7f7f08751640 1 --2- 172.21.15.40:0/623062547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d1770 0x7f7f000d1bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00097a0 tx=0x7f7ef002a180 comp rx=0 tx=0).stop 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.778+0000 7f7f08751640 1 -- 172.21.15.40:0/623062547 shutdown_connections 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.778+0000 7f7f08751640 1 --2- 172.21.15.40:0/623062547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d1770 0x7f7f000d1bf0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.778+0000 7f7f08751640 1 --2- 172.21.15.40:0/623062547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 0x7f7f000d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.778+0000 7f7f08751640 1 -- 172.21.15.40:0/623062547 >> 172.21.15.40:0/623062547 conn(0x7f7f000d4f90 msgr2=0x7f7f000d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.778+0000 7f7f08751640 1 -- 172.21.15.40:0/623062547 shutdown_connections 2024-04-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.778+0000 7f7f08751640 1 -- 172.21.15.40:0/623062547 wait complete. 2024-04-26T21:37:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.779+0000 7f7f08751640 1 Processor -- start 2024-04-26T21:37:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.779+0000 7f7f08751640 1 -- start start 2024-04-26T21:37:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f08751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f000d1770 0x7f7f000cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f08751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d2130 0x7f7f0014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f08751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 0x7f7f0014f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f08751640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0015f900 con 0x7f7f000d2130 2024-04-26T21:37:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f08751640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0015f780 con 0x7f7f000d1770 2024-04-26T21:37:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f08751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f000b3860 con 0x7f7f0015f3a0 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f05cec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d2130 0x7f7f0014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f05cec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d2130 0x7f7f0014eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35060/0 (socket says 172.21.15.40:35060) 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f05cec640 1 -- 172.21.15.40:0/2758133045 learned_addr learned my addr 172.21.15.40:0/2758133045 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f06cee640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 0x7f7f0014f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.780+0000 7f7f064ed640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f000d1770 0x7f7f000cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f05cec640 1 -- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 msgr2=0x7f7f0014f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f05cec640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 0x7f7f0014f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f05cec640 1 -- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f000d1770 msgr2=0x7f7f000cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f05cec640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f000d1770 0x7f7f000cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f05cec640 1 -- 172.21.15.40:0/2758133045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f000b97b0 con 0x7f7f000d2130 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f064ed640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f000d1770 0x7f7f000cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f05cec640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d2130 0x7f7f0014eae0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0035040 tx=0x7f7ef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7ef6ffd640 1 -- 172.21.15.40:0/2758133045 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0037040 con 0x7f7f000d2130 2024-04-26T21:37:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7ef6ffd640 1 -- 172.21.15.40:0/2758133045 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef0030500 con 0x7f7f000d2130 2024-04-26T21:37:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f000b9a90 con 0x7f7f000d2130 2024-04-26T21:37:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7ef6ffd640 1 -- 172.21.15.40:0/2758133045 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00308d0 con 0x7f7f000d2130 2024-04-26T21:37:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.781+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f000ba570 con 0x7f7f000d2130 2024-04-26T21:37:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.782+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f000694f0 con 0x7f7f000d2130 2024-04-26T21:37:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.783+0000 7f7ef6ffd640 1 -- 172.21.15.40:0/2758133045 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ef0044020 con 0x7f7f000d2130 2024-04-26T21:37:20.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.783+0000 7f7ef6ffd640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ee406c8f0 0x7f7ee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:20.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.783+0000 7f7ef6ffd640 1 -- 172.21.15.40:0/2758133045 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f7ef00abf50 con 0x7f7f000d2130 2024-04-26T21:37:20.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.783+0000 7f7f064ed640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ee406c8f0 0x7f7ee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:20.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.784+0000 7f7f064ed640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ee406c8f0 0x7f7ee406edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009af0 tx=0x7f7efc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:20.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.788+0000 7f7ef6ffd640 1 -- 172.21.15.40:0/2758133045 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ef007a180 con 0x7f7f000d2130 2024-04-26T21:37:20.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.972+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f000b6860 con 0x7f7f000d2130 2024-04-26T21:37:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.973+0000 7f7ef6ffd640 1 -- 172.21.15.40:0/2758133045 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17027 (secure 0 0 0) 0x7f7ef0079a80 con 0x7f7f000d2130 2024-04-26T21:37:20.974 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:20.975 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:51.722847+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3403660773":"2024-04-27T21:36:38.666240+0000","172.21.15.40:6833/3403660773":"2024-04-27T21:36:38.666240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ee406c8f0 msgr2=0x7f7ee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ee406c8f0 0x7f7ee406edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009af0 tx=0x7f7efc008040 comp rx=0 tx=0).stop 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d2130 msgr2=0x7f7f0014eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d2130 0x7f7f0014eae0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0035040 tx=0x7f7ef00046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 shutdown_connections 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ee406c8f0 0x7f7ee406edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f0015f3a0 0x7f7f0014f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f000d2130 0x7f7f0014eae0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 --2- 172.21.15.40:0/2758133045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f000d1770 0x7f7f000cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 >> 172.21.15.40:0/2758133045 conn(0x7f7f000d4f90 msgr2=0x7f7f000d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 shutdown_connections 2024-04-26T21:37:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:20.975+0000 7f7f08751640 1 -- 172.21.15.40:0/2758133045 wait complete. 2024-04-26T21:37:20.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:36:22.370518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:36:22.724779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:37:20.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:37:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.090+0000 7f52e3751640 1 Processor -- start 2024-04-26T21:37:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.090+0000 7f52e3751640 1 -- start start 2024-04-26T21:37:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52e3751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52dc0d3bc0 0x7f52dc0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52e3751640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52dc0d1040 con 0x7f52dc15f170 2024-04-26T21:37:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52e3751640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52dc0c9680 con 0x7f52dc15f550 2024-04-26T21:37:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52e3751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc0c9430 con 0x7f52dc0d3bc0 2024-04-26T21:37:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52e14ed640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f52dc15f170 0x7f52dc0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42442/0 (socket says 172.21.15.40:42442) 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52e14ed640 1 -- 172.21.15.40:0/480012747 learned_addr learned my addr 172.21.15.40:0/480012747 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3041088930 0 0) 0x7f52d8001110 con 0x7f52dc15f170 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52c8003520 con 0x7f52dc15f170 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.091+0000 7f52e1cee640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52dc0d3bc0 0x7f52dc0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2002564617 0 0) 0x7f52c8003520 con 0x7f52dc15f170 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52dc0d3bc0 msgr2=0x7f52dc0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52dc0d3bc0 0x7f52dc0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 >> v1:172.21.15.184:6789/0 conn(0x7f52dc15f550 legacy=0x7f52dc0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc0c9960 con 0x7f52dc15f170 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f52d8001840 con 0x7f52dc15f170 2024-04-26T21:37:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 >> v1:172.21.15.40:6789/0 conn(0x7f52dc15f170 legacy=0x7f52dc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c80039e0 0x7f52c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52c8004800 0x7f52c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80051c0 0x7f52c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc0c9430 con 0x7f52c80039e0 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52c800c060 con 0x7f52c8004800 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52c800c260 con 0x7f52c80051c0 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e14ed640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c80039e0 0x7f52c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e1cee640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52dc0d3bc0 0x7f52dc0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e0cec640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52c8004800 0x7f52c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e1cee640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80051c0 0x7f52c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e14ed640 1 -- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80051c0 msgr2=0x7f52c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e14ed640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80051c0 0x7f52c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e14ed640 1 -- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52c8004800 msgr2=0x7f52c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e14ed640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52c8004800 0x7f52c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e14ed640 1 -- 172.21.15.40:0/480012747 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c8003520 con 0x7f52c80039e0 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e1cee640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80051c0 0x7f52c800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.092+0000 7f52e0cec640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52c8004800 0x7f52c8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.093+0000 7f52e14ed640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c80039e0 0x7f52c8003e00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f52d800c4c0 tx=0x7f52c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d92d10b2d69afd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.093+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d800c0e0 con 0x7f52c80039e0 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.093+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d8034070 con 0x7f52c80039e0 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.093+0000 7f52d3fff640 1 -- 172.21.15.40:0/480012747 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8005d30 con 0x7f52c80039e0 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.093+0000 7f52e3751640 1 -- 172.21.15.40:0/480012747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c80039e0 msgr2=0x7f52c8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.093+0000 7f52e3751640 1 --2- 172.21.15.40:0/480012747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c80039e0 0x7f52c8003e00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f52d800c4c0 tx=0x7f52c8005830 comp rx=0 tx=0).stop 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.093+0000 7f52e1cee640 1 -- 172.21.15.40:0/480012747 reap_dead start 2024-04-26T21:37:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.094+0000 7f52e3751640 1 -- 172.21.15.40:0/480012747 shutdown_connections 2024-04-26T21:37:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.094+0000 7f52e3751640 1 -- 172.21.15.40:0/480012747 >> 172.21.15.40:0/480012747 conn(0x7f52dc0d4f60 msgr2=0x7f52dc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.094+0000 7f52e3751640 1 -- 172.21.15.40:0/480012747 shutdown_connections 2024-04-26T21:37:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.094+0000 7f52e3751640 1 -- 172.21.15.40:0/480012747 wait complete. 2024-04-26T21:37:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.095+0000 7f52e3751640 1 Processor -- start 2024-04-26T21:37:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.095+0000 7f52e3751640 1 -- start start 2024-04-26T21:37:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.095+0000 7f52e3751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80039e0 0x7f52dc1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e3751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52dc0a1bb0 0x7f52dc0a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e3751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52dc0b25d0 0x7f52dc0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e3751640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc0d5c00 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e3751640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc0d5a80 con 0x7f52dc0b25d0 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e3751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc0d5d80 con 0x7f52c80039e0 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52dc0a1bb0 0x7f52dc0a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52dc0a1bb0 0x7f52dc0a2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35074/0 (socket says 172.21.15.40:35074) 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 -- 172.21.15.40:0/4273986528 learned_addr learned my addr 172.21.15.40:0/4273986528 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e14ed640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80039e0 0x7f52dc1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e1cee640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52dc0b25d0 0x7f52dc0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 -- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80039e0 msgr2=0x7f52dc1583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80039e0 0x7f52dc1583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 -- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52dc0b25d0 msgr2=0x7f52dc0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52dc0b25d0 0x7f52dc0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 -- 172.21.15.40:0/4273986528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc1588e0 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e1cee640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52dc0b25d0 0x7f52dc0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e0cec640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52dc0a1bb0 0x7f52dc0a2030 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f52cc007960 tx=0x7f52cc00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52d1ffb640 1 -- 172.21.15.40:0/4273986528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0052e0 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52d1ffb640 1 -- 172.21.15.40:0/4273986528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52cc018070 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52d1ffb640 1 -- 172.21.15.40:0/4273986528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc003830 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.096+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc0a2670 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.097+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc09ed00 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.097+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc0694f0 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.098+0000 7f52d1ffb640 1 -- 172.21.15.40:0/4273986528 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52cc0039d0 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.098+0000 7f52d1ffb640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c006c920 0x7f52c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.099+0000 7f52d1ffb640 1 -- 172.21.15.40:0/4273986528 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f52cc08ba70 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.099+0000 7f52e14ed640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c006c920 0x7f52c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.099+0000 7f52e14ed640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c006c920 0x7f52c006ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52d8033000 tx=0x7f52d802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.103+0000 7f52d1ffb640 1 -- 172.21.15.40:0/4273986528 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f52cc059ca0 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.288+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f52dc0d2de0 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.289+0000 7f52d1ffb640 1 -- 172.21.15.40:0/4273986528 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f52cc0595a0 con 0x7f52dc0a1bb0 2024-04-26T21:37:21.292 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:37:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c006c920 msgr2=0x7f52c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c006c920 0x7f52c006ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52d8033000 tx=0x7f52d802e000 comp rx=0 tx=0).stop 2024-04-26T21:37:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52dc0a1bb0 msgr2=0x7f52dc0a2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52dc0a1bb0 0x7f52dc0a2030 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f52cc007960 tx=0x7f52cc00ea00 comp rx=0 tx=0).stop 2024-04-26T21:37:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 shutdown_connections 2024-04-26T21:37:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f52c006c920 0x7f52c006ede0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52dc0b25d0 0x7f52dc0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52dc0a1bb0 0x7f52dc0a2030 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 --2- 172.21.15.40:0/4273986528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52c80039e0 0x7f52dc1583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 >> 172.21.15.40:0/4273986528 conn(0x7f52dc0d4f60 msgr2=0x7f52dc0cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 shutdown_connections 2024-04-26T21:37:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.291+0000 7f52e3751640 1 -- 172.21.15.40:0/4273986528 wait complete. 2024-04-26T21:37:21.304 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:37:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.407+0000 7fd3545f7640 1 Processor -- start 2024-04-26T21:37:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.407+0000 7fd3545f7640 1 -- start start 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd3545f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 0x7fd34c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd3545f7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c0d1040 con 0x7fd34c15f550 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd3545f7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c0c9680 con 0x7fd34c0d3bc0 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd3545f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0c9430 con 0x7fd34c15f170 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd351b92640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd34c15f550 0x7fd34c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42448/0 (socket says 172.21.15.40:42448) 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd351b92640 1 -- 172.21.15.40:0/3810427039 learned_addr learned my addr 172.21.15.40:0/3810427039 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd352393640 1 --2- 172.21.15.40:0/3810427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 0x7fd34c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd351391640 1 -- 172.21.15.40:0/3810427039 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123597741 0 0) 0x7fd33c001110 con 0x7fd34c15f550 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd351391640 1 -- 172.21.15.40:0/3810427039 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd338003520 con 0x7fd34c15f550 2024-04-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd351391640 1 -- 172.21.15.40:0/3810427039 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4162045600 0 0) 0x7fd34c0c9680 con 0x7fd34c0d3bc0 2024-04-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd352393640 1 -- 172.21.15.40:0/3810427039 >> v1:172.21.15.184:6789/0 conn(0x7fd34c0d3bc0 legacy=0x7fd34c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd352393640 1 -- 172.21.15.40:0/3810427039 >> v1:172.21.15.40:6789/0 conn(0x7fd34c15f550 legacy=0x7fd34c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.408+0000 7fd352393640 1 -- 172.21.15.40:0/3810427039 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c0c9960 con 0x7fd34c15f170 2024-04-26T21:37:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.409+0000 7fd352393640 1 --2- 172.21.15.40:0/3810427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 0x7fd34c0d5a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3480097d0 tx=0x7fd34802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd3d50a6542518f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.409+0000 7fd351391640 1 -- 172.21.15.40:0/3810427039 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348037070 con 0x7fd34c15f170 2024-04-26T21:37:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.409+0000 7fd351391640 1 -- 172.21.15.40:0/3810427039 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd348032440 con 0x7fd34c15f170 2024-04-26T21:37:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.409+0000 7fd351391640 1 -- 172.21.15.40:0/3810427039 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480327f0 con 0x7fd34c15f170 2024-04-26T21:37:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.410+0000 7fd3545f7640 1 -- 172.21.15.40:0/3810427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 msgr2=0x7fd34c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.410+0000 7fd3545f7640 1 --2- 172.21.15.40:0/3810427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 0x7fd34c0d5a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3480097d0 tx=0x7fd34802a210 comp rx=0 tx=0).stop 2024-04-26T21:37:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.410+0000 7fd3545f7640 1 -- 172.21.15.40:0/3810427039 shutdown_connections 2024-04-26T21:37:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.410+0000 7fd3545f7640 1 --2- 172.21.15.40:0/3810427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 0x7fd34c0d5a80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.410+0000 7fd3545f7640 1 -- 172.21.15.40:0/3810427039 >> 172.21.15.40:0/3810427039 conn(0x7fd34c0d4f60 msgr2=0x7fd34c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.410+0000 7fd3545f7640 1 -- 172.21.15.40:0/3810427039 shutdown_connections 2024-04-26T21:37:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.410+0000 7fd3545f7640 1 -- 172.21.15.40:0/3810427039 wait complete. 2024-04-26T21:37:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.411+0000 7fd3545f7640 1 Processor -- start 2024-04-26T21:37:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.411+0000 7fd3545f7640 1 -- start start 2024-04-26T21:37:21.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd3545f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd34c0d3bc0 0x7fd34c1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd3545f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 0x7fd34c09f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd3545f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd34c0a1820 0x7fd34c09ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd3545f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0d2e30 con 0x7fd34c0a1820 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd3545f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0d2cb0 con 0x7fd34c0d3bc0 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd3545f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0d2fb0 con 0x7fd34c15f170 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352b94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd34c0a1820 0x7fd34c09ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352b94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd34c0a1820 0x7fd34c09ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35088/0 (socket says 172.21.15.40:35088) 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352b94640 1 -- 172.21.15.40:0/3182227500 learned_addr learned my addr 172.21.15.40:0/3182227500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352b94640 1 -- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 msgr2=0x7fd34c09f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352b94640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 0x7fd34c09f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352b94640 1 -- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd34c0d3bc0 msgr2=0x7fd34c1575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352393640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd34c0d3bc0 0x7fd34c1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352b94640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd34c0d3bc0 0x7fd34c1575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352b94640 1 -- 172.21.15.40:0/3182227500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c0a0780 con 0x7fd34c0a1820 2024-04-26T21:37:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.412+0000 7fd352393640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd34c0d3bc0 0x7fd34c1575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.413+0000 7fd352b94640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd34c0a1820 0x7fd34c09ff20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd34400c7e0 tx=0x7fd34400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.413+0000 7fd342ffd640 1 -- 172.21.15.40:0/3182227500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344017070 con 0x7fd34c0a1820 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.413+0000 7fd342ffd640 1 -- 172.21.15.40:0/3182227500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd344003da0 con 0x7fd34c0a1820 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.413+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c0a0a60 con 0x7fd34c0a1820 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.413+0000 7fd342ffd640 1 -- 172.21.15.40:0/3182227500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440052b0 con 0x7fd34c0a1820 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.413+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c0a1020 con 0x7fd34c0a1820 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.414+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c0694f0 con 0x7fd34c0a1820 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.414+0000 7fd342ffd640 1 -- 172.21.15.40:0/3182227500 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd34401a070 con 0x7fd34c0a1820 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.415+0000 7fd342ffd640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd33006c920 0x7fd33006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.415+0000 7fd342ffd640 1 -- 172.21.15.40:0/3182227500 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fd34408b450 con 0x7fd34c0a1820 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.415+0000 7fd352393640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd33006c920 0x7fd33006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.416+0000 7fd352393640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd33006c920 0x7fd33006ede0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd348034000 tx=0x7fd34802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.420+0000 7fd342ffd640 1 -- 172.21.15.40:0/3182227500 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd344059600 con 0x7fd34c0a1820 2024-04-26T21:37:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.604+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd34c0d5500 con 0x7fd34c0a1820 2024-04-26T21:37:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.604+0000 7fd342ffd640 1 -- 172.21.15.40:0/3182227500 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7fd344058f00 con 0x7fd34c0a1820 2024-04-26T21:37:21.605 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:37:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd33006c920 msgr2=0x7fd33006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd33006c920 0x7fd33006ede0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd348034000 tx=0x7fd34802e000 comp rx=0 tx=0).stop 2024-04-26T21:37:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd34c0a1820 msgr2=0x7fd34c09ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd34c0a1820 0x7fd34c09ff20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd34400c7e0 tx=0x7fd34400cca0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 shutdown_connections 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd33006c920 0x7fd33006ede0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd34c0a1820 0x7fd34c09ff20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd34c15f170 0x7fd34c09f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 --2- 172.21.15.40:0/3182227500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd34c0d3bc0 0x7fd34c1575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 >> 172.21.15.40:0/3182227500 conn(0x7fd34c0d4f60 msgr2=0x7fd34c0d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 shutdown_connections 2024-04-26T21:37:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.606+0000 7fd3545f7640 1 -- 172.21.15.40:0/3182227500 wait complete. 2024-04-26T21:37:21.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:37:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.721+0000 7fce20519640 1 Processor -- start 2024-04-26T21:37:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.721+0000 7fce20519640 1 -- start start 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce20519640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce18072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce20519640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fce18072780 0x7fce18166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce20519640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce1815d7c0 con 0x7fce18071e60 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce20519640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fce1815daa0 con 0x7fce18072780 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce20519640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fce1815dd80 con 0x7fce180cc480 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1e2b5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fce18071e60 0x7fce1806f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42460/0 (socket says 172.21.15.40:42460) 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1e2b5640 1 -- 172.21.15.40:0/397293385 learned_addr learned my addr 172.21.15.40:0/397293385 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1dab4640 1 --2- 172.21.15.40:0/397293385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce18072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1eab6640 1 --2- 172.21.15.40:0/397293385 >> v2:172.21.15.184:3300/0 conn(0x7fce18072780 0x7fce18166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1d2b3640 1 -- 172.21.15.40:0/397293385 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280387585 0 0) 0x7fce1815d7c0 con 0x7fce18071e60 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1d2b3640 1 -- 172.21.15.40:0/397293385 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce04003520 con 0x7fce18071e60 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1dab4640 1 -- 172.21.15.40:0/397293385 >> v2:172.21.15.184:3300/0 conn(0x7fce18072780 msgr2=0x7fce18166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1dab4640 1 --2- 172.21.15.40:0/397293385 >> v2:172.21.15.184:3300/0 conn(0x7fce18072780 0x7fce18166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1dab4640 1 -- 172.21.15.40:0/397293385 >> v1:172.21.15.40:6789/0 conn(0x7fce18071e60 legacy=0x7fce1806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1d2b3640 1 -- 172.21.15.40:0/397293385 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1863568842 0 0) 0x7fce04003520 con 0x7fce18071e60 2024-04-26T21:37:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1dab4640 1 -- 172.21.15.40:0/397293385 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1815e060 con 0x7fce180cc480 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1dab4640 1 --2- 172.21.15.40:0/397293385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce18072240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce080097a0 tx=0x7fce0802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf7a83c90b1ad3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.722+0000 7fce1d2b3640 1 -- 172.21.15.40:0/397293385 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08037070 con 0x7fce180cc480 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce1d2b3640 1 -- 172.21.15.40:0/397293385 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce0802aba0 con 0x7fce180cc480 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce1d2b3640 1 -- 172.21.15.40:0/397293385 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14001670 con 0x7fce180cc480 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce20519640 1 -- 172.21.15.40:0/397293385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 msgr2=0x7fce18072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce20519640 1 --2- 172.21.15.40:0/397293385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce18072240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce080097a0 tx=0x7fce0802a110 comp rx=0 tx=0).stop 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce20519640 1 -- 172.21.15.40:0/397293385 shutdown_connections 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce20519640 1 --2- 172.21.15.40:0/397293385 >> v2:172.21.15.184:3300/0 conn(0x7fce18072780 0x7fce18166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce20519640 1 --2- 172.21.15.40:0/397293385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce18072240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce20519640 1 -- 172.21.15.40:0/397293385 >> 172.21.15.40:0/397293385 conn(0x7fce180d0be0 msgr2=0x7fce180d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce20519640 1 -- 172.21.15.40:0/397293385 shutdown_connections 2024-04-26T21:37:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.723+0000 7fce20519640 1 -- 172.21.15.40:0/397293385 wait complete. 2024-04-26T21:37:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.724+0000 7fce20519640 1 Processor -- start 2024-04-26T21:37:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.724+0000 7fce20519640 1 -- start start 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce20519640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce18071e60 0x7fce1814f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce20519640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce18072780 0x7fce181553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce20519640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce181506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce20519640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fce1815d3f0 con 0x7fce18071e60 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce20519640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce18072c10 con 0x7fce18072780 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce20519640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fce1815d570 con 0x7fce180cc480 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce18071e60 0x7fce1814f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce18071e60 0x7fce1814f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35100/0 (socket says 172.21.15.40:35100) 2024-04-26T21:37:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 -- 172.21.15.40:0/3056459921 learned_addr learned my addr 172.21.15.40:0/3056459921 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1eab6640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce181506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 -- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 msgr2=0x7fce181506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1dab4640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce18072780 0x7fce181553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce181506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 -- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce18072780 msgr2=0x7fce181553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce18072780 0x7fce181553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 -- 172.21.15.40:0/3056459921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1814fee0 con 0x7fce18071e60 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1eab6640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce181506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1dab4640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce18072780 0x7fce181553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.725+0000 7fce1e2b5640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce18071e60 0x7fce1814f160 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fce1400c7b0 tx=0x7fce1400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.726+0000 7fce0effd640 1 -- 172.21.15.40:0/3056459921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14017070 con 0x7fce18071e60 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.726+0000 7fce0effd640 1 -- 172.21.15.40:0/3056459921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce14005ad0 con 0x7fce18071e60 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.726+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce18155d90 con 0x7fce18071e60 2024-04-26T21:37:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.726+0000 7fce0effd640 1 -- 172.21.15.40:0/3056459921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14007070 con 0x7fce18071e60 2024-04-26T21:37:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.726+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce18156350 con 0x7fce18071e60 2024-04-26T21:37:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.727+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce180694f0 con 0x7fce18071e60 2024-04-26T21:37:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.727+0000 7fce0effd640 1 -- 172.21.15.40:0/3056459921 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce1401a070 con 0x7fce18071e60 2024-04-26T21:37:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.728+0000 7fce0effd640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdfc06c8f0 0x7fcdfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.728+0000 7fce0effd640 1 -- 172.21.15.40:0/3056459921 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fce1408b230 con 0x7fce18071e60 2024-04-26T21:37:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.728+0000 7fce1dab4640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdfc06c8f0 0x7fcdfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.728+0000 7fce1dab4640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdfc06c8f0 0x7fcdfc06edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fce08034000 tx=0x7fce0802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:21.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.732+0000 7fce0effd640 1 -- 172.21.15.40:0/3056459921 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce14059460 con 0x7fce18071e60 2024-04-26T21:37:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.918+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce1815a290 con 0x7fce18071e60 2024-04-26T21:37:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.918+0000 7fce0effd640 1 -- 172.21.15.40:0/3056459921 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7fce14058d60 con 0x7fce18071e60 2024-04-26T21:37:21.920 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:37:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdfc06c8f0 msgr2=0x7fcdfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdfc06c8f0 0x7fcdfc06edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fce08034000 tx=0x7fce0802e000 comp rx=0 tx=0).stop 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce18071e60 msgr2=0x7fce1814f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce18071e60 0x7fce1814f160 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fce1400c7b0 tx=0x7fce1400cc70 comp rx=0 tx=0).stop 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 shutdown_connections 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdfc06c8f0 0x7fcdfc06edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce180cc480 0x7fce181506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce18072780 0x7fce181553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 --2- 172.21.15.40:0/3056459921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce18071e60 0x7fce1814f160 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 >> 172.21.15.40:0/3056459921 conn(0x7fce180d0be0 msgr2=0x7fce18159190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 shutdown_connections 2024-04-26T21:37:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:21.920+0000 7fce20519640 1 -- 172.21.15.40:0/3056459921 wait complete. 2024-04-26T21:37:21.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:37:22.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.033+0000 7f792302c640 1 Processor -- start 2024-04-26T21:37:22.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.033+0000 7f792302c640 1 -- start start 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f792302c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f791c15f3a0 0x7f791c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f792302c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f791c0d1770 0x7f791c0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f792302c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f791c0d2590 con 0x7f791c0d2160 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f792302c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f791c0d0e20 con 0x7f791c15f3a0 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f792302c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f791c0b1d70 con 0x7f791c0d1770 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f7920dc8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f791c15f3a0 0x7f791c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f79215c9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f791c0d2160 0x7f791c0d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42476/0 (socket says 172.21.15.40:42476) 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f7920dc8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f791c15f3a0 0x7f791c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35114/0 (socket says 172.21.15.40:35114) 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f7920dc8640 1 -- 172.21.15.40:0/2906047053 learned_addr learned my addr 172.21.15.40:0/2906047053 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f7920dc8640 1 -- 172.21.15.40:0/2906047053 >> v2:172.21.15.184:3300/0 conn(0x7f791c0d1770 msgr2=0x7f791c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.034+0000 7f7920dc8640 1 --2- 172.21.15.40:0/2906047053 >> v2:172.21.15.184:3300/0 conn(0x7f791c0d1770 0x7f791c0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.035+0000 7f7920dc8640 1 -- 172.21.15.40:0/2906047053 >> v1:172.21.15.40:6789/0 conn(0x7f791c0d2160 legacy=0x7f791c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.035+0000 7f7920dc8640 1 -- 172.21.15.40:0/2906047053 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c0b2050 con 0x7f791c15f3a0 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.035+0000 7f7920dc8640 1 --2- 172.21.15.40:0/2906047053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c15f3a0 0x7f791c0d06f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f791c0b7b80 tx=0x7f791402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9b0e487b23a9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.035+0000 7f791b7fe640 1 -- 172.21.15.40:0/2906047053 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914036070 con 0x7f791c15f3a0 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.035+0000 7f791b7fe640 1 -- 172.21.15.40:0/2906047053 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7914031600 con 0x7f791c15f3a0 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.035+0000 7f791b7fe640 1 -- 172.21.15.40:0/2906047053 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79140319d0 con 0x7f791c15f3a0 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.036+0000 7f792302c640 1 -- 172.21.15.40:0/2906047053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c15f3a0 msgr2=0x7f791c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.036+0000 7f792302c640 1 --2- 172.21.15.40:0/2906047053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c15f3a0 0x7f791c0d06f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f791c0b7b80 tx=0x7f791402a730 comp rx=0 tx=0).stop 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.036+0000 7f792302c640 1 -- 172.21.15.40:0/2906047053 shutdown_connections 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.036+0000 7f792302c640 1 --2- 172.21.15.40:0/2906047053 >> v2:172.21.15.184:3300/0 conn(0x7f791c0d1770 0x7f791c0d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.036+0000 7f792302c640 1 --2- 172.21.15.40:0/2906047053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c15f3a0 0x7f791c0d06f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.036+0000 7f792302c640 1 -- 172.21.15.40:0/2906047053 >> 172.21.15.40:0/2906047053 conn(0x7f791c0d4f90 msgr2=0x7f791c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.036+0000 7f792302c640 1 -- 172.21.15.40:0/2906047053 shutdown_connections 2024-04-26T21:37:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.036+0000 7f792302c640 1 -- 172.21.15.40:0/2906047053 wait complete. 2024-04-26T21:37:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.037+0000 7f792302c640 1 Processor -- start 2024-04-26T21:37:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.037+0000 7f792302c640 1 -- start start 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f792302c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f791c0d1770 0x7f791c0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f792302c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c0d2160 0x7f791c161af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f792302c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791c15f3a0 0x7f791c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f792302c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f791c15f900 con 0x7f791c0d2160 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f792302c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f791c15f780 con 0x7f791c0d1770 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f792302c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f791c0b3860 con 0x7f791c15f3a0 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f791bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c0d2160 0x7f791c161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f791bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c0d2160 0x7f791c161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35130/0 (socket says 172.21.15.40:35130) 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f791bfff640 1 -- 172.21.15.40:0/3235730751 learned_addr learned my addr 172.21.15.40:0/3235730751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f79215c9640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791c15f3a0 0x7f791c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f7920dc8640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f791c0d1770 0x7f791c0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f791bfff640 1 -- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791c15f3a0 msgr2=0x7f791c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f791bfff640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791c15f3a0 0x7f791c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f791bfff640 1 -- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f791c0d1770 msgr2=0x7f791c0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f791bfff640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f791c0d1770 0x7f791c0cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f791bfff640 1 -- 172.21.15.40:0/3235730751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c162260 con 0x7f791c0d2160 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f79215c9640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791c15f3a0 0x7f791c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.038+0000 7f7920dc8640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f791c0d1770 0x7f791c0cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.039+0000 7f791bfff640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c0d2160 0x7f791c161af0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f791000b4a0 tx=0x7f791000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.039+0000 7f79197fa640 1 -- 172.21.15.40:0/3235730751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910014070 con 0x7f791c0d2160 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.039+0000 7f79197fa640 1 -- 172.21.15.40:0/3235730751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7910010610 con 0x7f791c0d2160 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.039+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c162540 con 0x7f791c0d2160 2024-04-26T21:37:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.039+0000 7f79197fa640 1 -- 172.21.15.40:0/3235730751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910010a00 con 0x7f791c0d2160 2024-04-26T21:37:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.039+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c0b7d20 con 0x7f791c0d2160 2024-04-26T21:37:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.040+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c0b7f20 con 0x7f791c0d2160 2024-04-26T21:37:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.040+0000 7f79197fa640 1 -- 172.21.15.40:0/3235730751 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7910017070 con 0x7f791c0d2160 2024-04-26T21:37:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.041+0000 7f79197fa640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f790806c8f0 0x7f790806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.041+0000 7f79197fa640 1 -- 172.21.15.40:0/3235730751 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f791008b7a0 con 0x7f791c0d2160 2024-04-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.041+0000 7f7920dc8640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f790806c8f0 0x7f790806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.041+0000 7f7920dc8640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f790806c8f0 0x7f790806edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7914033000 tx=0x7f7914009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.046+0000 7f79197fa640 1 -- 172.21.15.40:0/3235730751 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f791c0b7f20 con 0x7f791c0d2160 2024-04-26T21:37:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.241+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f791c0694f0 con 0x7f791c0d2160 2024-04-26T21:37:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.242+0000 7f79197fa640 1 -- 172.21.15.40:0/3235730751 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v41) ==== 80+0+122 (secure 0 0 0) 0x7f79100599c0 con 0x7f791c0d2160 2024-04-26T21:37:22.243 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/3403660773 2024-04-27T21:36:38.666240+0000 2024-04-26T21:37:22.243 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3403660773 2024-04-27T21:36:38.666240+0000 2024-04-26T21:37:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:listed 2 entries 2024-04-26T21:37:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f790806c8f0 msgr2=0x7f790806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f790806c8f0 0x7f790806edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7914033000 tx=0x7f7914009000 comp rx=0 tx=0).stop 2024-04-26T21:37:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c0d2160 msgr2=0x7f791c161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c0d2160 0x7f791c161af0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f791000b4a0 tx=0x7f791000b960 comp rx=0 tx=0).stop 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 shutdown_connections 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f790806c8f0 0x7f790806edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f791c15f3a0 0x7f791c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f791c0d2160 0x7f791c161af0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 --2- 172.21.15.40:0/3235730751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f791c0d1770 0x7f791c0cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 >> 172.21.15.40:0/3235730751 conn(0x7f791c0d4f90 msgr2=0x7f791c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 shutdown_connections 2024-04-26T21:37:22.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.244+0000 7f792302c640 1 -- 172.21.15.40:0/3235730751 wait complete. 2024-04-26T21:37:22.256 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:37:22.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:22.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:22.284 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:37:22.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:22.284 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:37:22.341 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:37:22.341 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:37:22.341 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:22.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:37:22.361 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:37:22.361 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:37:22.361 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:22.396 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:37:22.396 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:37:22.396 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:37:22.397 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:22.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:37:22.418 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:22.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.536+0000 7f056370a640 1 Processor -- start 2024-04-26T21:37:22.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.536+0000 7f056370a640 1 -- start start 2024-04-26T21:37:22.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.537+0000 7f056370a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f055c15f3a0 0x7f055c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.537+0000 7f056370a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f055c0d2330 0x7f055c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.537+0000 7f056370a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f055c0d0e20 con 0x7f055c0d1770 2024-04-26T21:37:22.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.537+0000 7f056370a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f055c0b1d70 con 0x7f055c0d2330 2024-04-26T21:37:22.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.537+0000 7f056370a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f055c0b2050 con 0x7f055c15f3a0 2024-04-26T21:37:22.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.537+0000 7f0561ca7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f055c0d2330 0x7f055c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.537+0000 7f0560ca5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f055c0d1770 0x7f055c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42488/0 (socket says 172.21.15.40:42488) 2024-04-26T21:37:22.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.537+0000 7f0560ca5640 1 -- 172.21.15.40:0/987635113 learned_addr learned my addr 172.21.15.40:0/987635113 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:22.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.538+0000 7f0561ca7640 1 -- 172.21.15.40:0/987635113 >> v2:172.21.15.184:3300/0 conn(0x7f055c15f3a0 msgr2=0x7f055c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:22.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.538+0000 7f05614a6640 1 --2- 172.21.15.40:0/987635113 >> v2:172.21.15.184:3300/0 conn(0x7f055c15f3a0 0x7f055c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.538+0000 7f0561ca7640 1 --2- 172.21.15.40:0/987635113 >> v2:172.21.15.184:3300/0 conn(0x7f055c15f3a0 0x7f055c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.538+0000 7f0561ca7640 1 -- 172.21.15.40:0/987635113 >> v1:172.21.15.40:6789/0 conn(0x7f055c0d1770 legacy=0x7f055c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.538+0000 7f055bfff640 1 -- 172.21.15.40:0/987635113 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 34948604 0 0) 0x7f055c0d0e20 con 0x7f055c0d1770 2024-04-26T21:37:22.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.538+0000 7f0561ca7640 1 -- 172.21.15.40:0/987635113 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c0b2330 con 0x7f055c0d2330 2024-04-26T21:37:22.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.538+0000 7f0561ca7640 1 --2- 172.21.15.40:0/987635113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d2330 0x7f055c0b2850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f055c073b50 tx=0x7f054c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba19e8520d61e9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f055bfff640 1 -- 172.21.15.40:0/987635113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c037070 con 0x7f055c0d2330 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f055bfff640 1 -- 172.21.15.40:0/987635113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054c032440 con 0x7f055c0d2330 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f055bfff640 1 -- 172.21.15.40:0/987635113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c032810 con 0x7f055c0d2330 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f056370a640 1 -- 172.21.15.40:0/987635113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d2330 msgr2=0x7f055c0b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f056370a640 1 --2- 172.21.15.40:0/987635113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d2330 0x7f055c0b2850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f055c073b50 tx=0x7f054c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f056370a640 1 -- 172.21.15.40:0/987635113 shutdown_connections 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f056370a640 1 --2- 172.21.15.40:0/987635113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d2330 0x7f055c0b2850 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f056370a640 1 --2- 172.21.15.40:0/987635113 >> v2:172.21.15.184:3300/0 conn(0x7f055c15f3a0 0x7f055c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f056370a640 1 -- 172.21.15.40:0/987635113 >> 172.21.15.40:0/987635113 conn(0x7f055c0d4f90 msgr2=0x7f055c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f056370a640 1 -- 172.21.15.40:0/987635113 shutdown_connections 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.539+0000 7f056370a640 1 -- 172.21.15.40:0/987635113 wait complete. 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.540+0000 7f056370a640 1 Processor -- start 2024-04-26T21:37:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.540+0000 7f056370a640 1 -- start start 2024-04-26T21:37:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f056370a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d1770 0x7f055c153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f056370a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055c0d2330 0x7f055c0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f056370a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055c15f3a0 0x7f055c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f056370a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0b2f10 con 0x7f055c0d1770 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f056370a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0b2d90 con 0x7f055c15f3a0 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f056370a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f055c0b3090 con 0x7f055c0d2330 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f05614a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d1770 0x7f055c153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f05614a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d1770 0x7f055c153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35148/0 (socket says 172.21.15.40:35148) 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.541+0000 7f05614a6640 1 -- 172.21.15.40:0/2032859275 learned_addr learned my addr 172.21.15.40:0/2032859275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f05614a6640 1 -- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055c0d2330 msgr2=0x7f055c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f05614a6640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055c0d2330 0x7f055c0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f0561ca7640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055c15f3a0 0x7f055c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f05614a6640 1 -- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055c15f3a0 msgr2=0x7f055c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f05614a6640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055c15f3a0 0x7f055c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f05614a6640 1 -- 172.21.15.40:0/2032859275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c0b8e10 con 0x7f055c0d1770 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f0561ca7640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055c15f3a0 0x7f055c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f05614a6640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d1770 0x7f055c153460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f055400ccd0 tx=0x7f055400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f0559ffb640 1 -- 172.21.15.40:0/2032859275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554003990 con 0x7f055c0d1770 2024-04-26T21:37:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f0559ffb640 1 -- 172.21.15.40:0/2032859275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0554003b30 con 0x7f055c0d1770 2024-04-26T21:37:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c0b90f0 con 0x7f055c0d1770 2024-04-26T21:37:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f0559ffb640 1 -- 172.21.15.40:0/2032859275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554004020 con 0x7f055c0d1770 2024-04-26T21:37:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.542+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c0b96b0 con 0x7f055c0d1770 2024-04-26T21:37:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.543+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c0694f0 con 0x7f055c0d1770 2024-04-26T21:37:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.544+0000 7f0559ffb640 1 -- 172.21.15.40:0/2032859275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05540041c0 con 0x7f055c0d1770 2024-04-26T21:37:22.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.544+0000 7f0559ffb640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f054806c8f0 0x7f054806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.544+0000 7f0559ffb640 1 -- 172.21.15.40:0/2032859275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f055408c2e0 con 0x7f055c0d1770 2024-04-26T21:37:22.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.544+0000 7f0560ca5640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f054806c8f0 0x7f054806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.545+0000 7f0560ca5640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f054806c8f0 0x7f054806edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05500011e0 tx=0x7f0550007950 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.549+0000 7f0559ffb640 1 -- 172.21.15.40:0/2032859275 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0554014030 con 0x7f055c0d1770 2024-04-26T21:37:22.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.771+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f055c0b6860 con 0x7f055c0d1770 2024-04-26T21:37:22.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.772+0000 7f0559ffb640 1 -- 172.21.15.40:0/2032859275 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5185 (secure 0 0 0) 0x7f055405a510 con 0x7f055c0d1770 2024-04-26T21:37:22.773 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:22.773 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:42.691649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5619,"qdb_cluster":[5619]},"id":2}]} 2024-04-26T21:37:22.774 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 23 2024-04-26T21:37:22.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f054806c8f0 msgr2=0x7f054806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f054806c8f0 0x7f054806edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05500011e0 tx=0x7f0550007950 comp rx=0 tx=0).stop 2024-04-26T21:37:22.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d1770 msgr2=0x7f055c153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d1770 0x7f055c153460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f055400ccd0 tx=0x7f055400a5c0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 shutdown_connections 2024-04-26T21:37:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f054806c8f0 0x7f054806edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055c15f3a0 0x7f055c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055c0d2330 0x7f055c0b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 --2- 172.21.15.40:0/2032859275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f055c0d1770 0x7f055c153460 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 >> 172.21.15.40:0/2032859275 conn(0x7f055c0d4f90 msgr2=0x7f055c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 shutdown_connections 2024-04-26T21:37:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.774+0000 7f056370a640 1 -- 172.21.15.40:0/2032859275 wait complete. 2024-04-26T21:37:22.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fc2f9640 1 Processor -- start 2024-04-26T21:37:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fc2f9640 1 -- start start 2024-04-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fc2f9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f40b5f20 0x7f78f40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fc2f9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f40b7e50 con 0x7f78f40d3bc0 2024-04-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fc2f9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f40d1800 con 0x7f78f4162880 2024-04-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fc2f9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78f40d0580 con 0x7f78f40b5f20 2024-04-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78f9894640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78f40d3bc0 0x7f78f40b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42490/0 (socket says 172.21.15.40:42490) 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78f9894640 1 -- 172.21.15.40:0/3350754628 learned_addr learned my addr 172.21.15.40:0/3350754628 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fa896640 1 --2- 172.21.15.40:0/3350754628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f40b5f20 0x7f78f40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78f9093640 1 -- 172.21.15.40:0/3350754628 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460974003 0 0) 0x7f78f40b7e50 con 0x7f78f40d3bc0 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78f9093640 1 -- 172.21.15.40:0/3350754628 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78e0003520 con 0x7f78f40d3bc0 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78f9093640 1 -- 172.21.15.40:0/3350754628 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3511671971 0 0) 0x7f78f0001110 con 0x7f78f4162880 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78f9093640 1 -- 172.21.15.40:0/3350754628 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78f40b7e50 con 0x7f78f4162880 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fa896640 1 -- 172.21.15.40:0/3350754628 >> v1:172.21.15.184:6789/0 conn(0x7f78f4162880 legacy=0x7f78f40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fa896640 1 -- 172.21.15.40:0/3350754628 >> v1:172.21.15.40:6789/0 conn(0x7f78f40d3bc0 legacy=0x7f78f40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78f9093640 1 -- 172.21.15.40:0/3350754628 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 354780554 0 0) 0x7f78e0003520 con 0x7f78f40d3bc0 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.888+0000 7f78fa896640 1 -- 172.21.15.40:0/3350754628 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f40d07d0 con 0x7f78f40b5f20 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.889+0000 7f78fa896640 1 --2- 172.21.15.40:0/3350754628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f40b5f20 0x7f78f40d2cb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f78f40d0700 tx=0x7f78ec02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae86066f0825917 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.889+0000 7f78f9093640 1 -- 172.21.15.40:0/3350754628 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4001670 con 0x7f78f40b5f20 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.889+0000 7f78f9093640 1 -- 172.21.15.40:0/3350754628 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78ec007540 con 0x7f78f40b5f20 2024-04-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.889+0000 7f78f9093640 1 -- 172.21.15.40:0/3350754628 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0078f0 con 0x7f78f40b5f20 2024-04-26T21:37:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.890+0000 7f78fc2f9640 1 -- 172.21.15.40:0/3350754628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f40b5f20 msgr2=0x7f78f40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.890+0000 7f78fc2f9640 1 --2- 172.21.15.40:0/3350754628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f40b5f20 0x7f78f40d2cb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f78f40d0700 tx=0x7f78ec02a660 comp rx=0 tx=0).stop 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.890+0000 7f78fc2f9640 1 -- 172.21.15.40:0/3350754628 shutdown_connections 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.890+0000 7f78fc2f9640 1 --2- 172.21.15.40:0/3350754628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f40b5f20 0x7f78f40d2cb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.890+0000 7f78fc2f9640 1 -- 172.21.15.40:0/3350754628 >> 172.21.15.40:0/3350754628 conn(0x7f78f40d4f60 msgr2=0x7f78f40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.890+0000 7f78fc2f9640 1 -- 172.21.15.40:0/3350754628 shutdown_connections 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.890+0000 7f78fc2f9640 1 -- 172.21.15.40:0/3350754628 wait complete. 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.892+0000 7f78fc2f9640 1 Processor -- start 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.892+0000 7f78fc2f9640 1 -- start start 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fc2f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f40b5f20 0x7f78f4152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fc2f9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f40d3bc0 0x7f78f40b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fc2f9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f4162880 0x7f78f40b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fc2f9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78f4162c60 con 0x7f78f40b5f20 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fc2f9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78f40d0d90 con 0x7f78f40d3bc0 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fc2f9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78f4162de0 con 0x7f78f4162880 2024-04-26T21:37:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f40b5f20 0x7f78f4152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f40b5f20 0x7f78f4152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35158/0 (socket says 172.21.15.40:35158) 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 -- 172.21.15.40:0/4288957460 learned_addr learned my addr 172.21.15.40:0/4288957460 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa896640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f4162880 0x7f78f40b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78f9894640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f40d3bc0 0x7f78f40b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 -- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f4162880 msgr2=0x7f78f40b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f4162880 0x7f78f40b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 -- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f40d3bc0 msgr2=0x7f78f40b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f40d3bc0 0x7f78f40b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 -- 172.21.15.40:0/4288957460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f41f8750 con 0x7f78f40b5f20 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa896640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f4162880 0x7f78f40b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fa095640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f40b5f20 0x7f78f4152c40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c700 tx=0x7f78f000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78eaffd640 1 -- 172.21.15.40:0/4288957460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00052d0 con 0x7f78f40b5f20 2024-04-26T21:37:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78eaffd640 1 -- 172.21.15.40:0/4288957460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f0005470 con 0x7f78f40b5f20 2024-04-26T21:37:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f41f8a30 con 0x7f78f40b5f20 2024-04-26T21:37:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78eaffd640 1 -- 172.21.15.40:0/4288957460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f000a860 con 0x7f78f40b5f20 2024-04-26T21:37:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.893+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f4153180 con 0x7f78f40b5f20 2024-04-26T21:37:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.894+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f40694f0 con 0x7f78f40b5f20 2024-04-26T21:37:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.894+0000 7f78eaffd640 1 -- 172.21.15.40:0/4288957460 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78f000aa00 con 0x7f78f40b5f20 2024-04-26T21:37:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.895+0000 7f78eaffd640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.895+0000 7f78eaffd640 1 -- 172.21.15.40:0/4288957460 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f78f008b990 con 0x7f78f40b5f20 2024-04-26T21:37:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.895+0000 7f78f9894640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.895+0000 7f78f9894640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f78f40d29c0 tx=0x7f78e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:22.900+0000 7f78eaffd640 1 -- 172.21.15.40:0/4288957460 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78f40d1800 con 0x7f78f40b5f20 2024-04-26T21:37:23.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.085+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78f40d2f50 con 0x7f78f40b5f20 2024-04-26T21:37:23.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.086+0000 7f78eaffd640 1 -- 172.21.15.40:0/4288957460 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17027 (secure 0 0 0) 0x7f78f0059c50 con 0x7f78f40b5f20 2024-04-26T21:37:23.086 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:23.086 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:51.722847+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3403660773":"2024-04-27T21:36:38.666240+0000","172.21.15.40:6833/3403660773":"2024-04-27T21:36:38.666240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:23.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 msgr2=0x7f78d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f78f40d29c0 tx=0x7f78e4009040 comp rx=0 tx=0).stop 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f40b5f20 msgr2=0x7f78f4152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f40b5f20 0x7f78f4152c40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c700 tx=0x7f78f000cbc0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 shutdown_connections 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d806c8f0 0x7f78d806edb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f4162880 0x7f78f40b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f40d3bc0 0x7f78f40b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 --2- 172.21.15.40:0/4288957460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f40b5f20 0x7f78f4152c40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 >> 172.21.15.40:0/4288957460 conn(0x7f78f40d4f60 msgr2=0x7f78f40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 shutdown_connections 2024-04-26T21:37:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.088+0000 7f78fc2f9640 1 -- 172.21.15.40:0/4288957460 wait complete. 2024-04-26T21:37:23.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:36:22.370518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:36:22.724779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:37:23.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:37:23.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.203+0000 7f5f5a1f0640 1 Processor -- start 2024-04-26T21:37:23.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.204+0000 7f5f5a1f0640 1 -- start start 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.204+0000 7f5f5a1f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540b5b10 0x7f5f5416a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.204+0000 7f5f5a1f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f540d0a10 con 0x7f5f540d1a00 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.204+0000 7f5f5a1f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f540d0cf0 con 0x7f5f540b40d0 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.204+0000 7f5f5a1f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5f540d0fd0 con 0x7f5f540b5b10 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.204+0000 7f5f537fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5f540d1a00 0x7f5f540d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42494/0 (socket says 172.21.15.40:42494) 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.204+0000 7f5f537fe640 1 -- 172.21.15.40:0/3154006730 learned_addr learned my addr 172.21.15.40:0/3154006730 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.205+0000 7f5f53fff640 1 --2- 172.21.15.40:0/3154006730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540b5b10 0x7f5f5416a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.205+0000 7f5f527fc640 1 -- 172.21.15.40:0/3154006730 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470640649 0 0) 0x7f5f540d0a10 con 0x7f5f540d1a00 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.205+0000 7f5f527fc640 1 -- 172.21.15.40:0/3154006730 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f3c003520 con 0x7f5f540d1a00 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.205+0000 7f5f527fc640 1 -- 172.21.15.40:0/3154006730 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1834411726 0 0) 0x7f5f540d0cf0 con 0x7f5f540b40d0 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.205+0000 7f5f527fc640 1 -- 172.21.15.40:0/3154006730 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f540d0a10 con 0x7f5f540b40d0 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.205+0000 7f5f53fff640 1 -- 172.21.15.40:0/3154006730 >> v1:172.21.15.184:6789/0 conn(0x7f5f540b40d0 legacy=0x7f5f541625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.205+0000 7f5f53fff640 1 -- 172.21.15.40:0/3154006730 >> v1:172.21.15.40:6789/0 conn(0x7f5f540d1a00 legacy=0x7f5f540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.205+0000 7f5f53fff640 1 -- 172.21.15.40:0/3154006730 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f540d40a0 con 0x7f5f540b5b10 2024-04-26T21:37:23.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f53fff640 1 --2- 172.21.15.40:0/3154006730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540b5b10 0x7f5f5416a2e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009dc0 tx=0x7f5f4c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87fd88910c2e03f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f527fc640 1 -- 172.21.15.40:0/3154006730 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c02ae60 con 0x7f5f540b5b10 2024-04-26T21:37:23.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f527fc640 1 -- 172.21.15.40:0/3154006730 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f4c007740 con 0x7f5f540b5b10 2024-04-26T21:37:23.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f527fc640 1 -- 172.21.15.40:0/3154006730 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c007af0 con 0x7f5f540b5b10 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/3154006730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540b5b10 msgr2=0x7f5f5416a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f5a1f0640 1 --2- 172.21.15.40:0/3154006730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540b5b10 0x7f5f5416a2e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009dc0 tx=0x7f5f4c02a640 comp rx=0 tx=0).stop 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/3154006730 shutdown_connections 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f5a1f0640 1 --2- 172.21.15.40:0/3154006730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540b5b10 0x7f5f5416a2e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/3154006730 >> 172.21.15.40:0/3154006730 conn(0x7f5f540d4fd0 msgr2=0x7f5f540d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.206+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/3154006730 shutdown_connections 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.207+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/3154006730 wait complete. 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.208+0000 7f5f5a1f0640 1 Processor -- start 2024-04-26T21:37:23.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.208+0000 7f5f5a1f0640 1 -- start start 2024-04-26T21:37:23.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.208+0000 7f5f5a1f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f540b40d0 0x7f5f540b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.208+0000 7f5f5a1f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f540b5b10 0x7f5f540ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.208+0000 7f5f5a1f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540d1a00 0x7f5f540ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.208+0000 7f5f5a1f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5f54162e80 con 0x7f5f540b5b10 2024-04-26T21:37:23.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.208+0000 7f5f5a1f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f54162d00 con 0x7f5f540b40d0 2024-04-26T21:37:23.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.208+0000 7f5f5a1f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5f54163000 con 0x7f5f540d1a00 2024-04-26T21:37:23.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f540b5b10 0x7f5f540ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f540b5b10 0x7f5f540ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35162/0 (socket says 172.21.15.40:35162) 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 -- 172.21.15.40:0/1918392752 learned_addr learned my addr 172.21.15.40:0/1918392752 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f537fe640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f540b40d0 0x7f5f540b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f53fff640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540d1a00 0x7f5f540ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 -- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540d1a00 msgr2=0x7f5f540ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540d1a00 0x7f5f540ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 -- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f540b40d0 msgr2=0x7f5f540b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f540b40d0 0x7f5f540b8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 -- 172.21.15.40:0/1918392752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f540b9220 con 0x7f5f540b5b10 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f52ffd640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f540b5b10 0x7f5f540ce570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800c500 tx=0x7f5f4800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f3bfff640 1 -- 172.21.15.40:0/1918392752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48017070 con 0x7f5f540b5b10 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f3bfff640 1 -- 172.21.15.40:0/1918392752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f48005310 con 0x7f5f540b5b10 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f540b9500 con 0x7f5f540b5b10 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f3bfff640 1 -- 172.21.15.40:0/1918392752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48003830 con 0x7f5f540b5b10 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.209+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f540b9ac0 con 0x7f5f540b5b10 2024-04-26T21:37:23.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.210+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f540694f0 con 0x7f5f540b5b10 2024-04-26T21:37:23.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.211+0000 7f5f3bfff640 1 -- 172.21.15.40:0/1918392752 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f4801a070 con 0x7f5f540b5b10 2024-04-26T21:37:23.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.211+0000 7f5f3bfff640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f4006c8f0 0x7f5f4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.211+0000 7f5f3bfff640 1 -- 172.21.15.40:0/1918392752 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f5f4808b010 con 0x7f5f540b5b10 2024-04-26T21:37:23.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.212+0000 7f5f537fe640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f4006c8f0 0x7f5f4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.212+0000 7f5f537fe640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f4006c8f0 0x7f5f4006edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f440011e0 tx=0x7f5f44007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.216+0000 7f5f3bfff640 1 -- 172.21.15.40:0/1918392752 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5f48058c00 con 0x7f5f540b5b10 2024-04-26T21:37:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.403+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f540b7650 con 0x7f5f540b5b10 2024-04-26T21:37:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.404+0000 7f5f3bfff640 1 -- 172.21.15.40:0/1918392752 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f5f4805cf00 con 0x7f5f540b5b10 2024-04-26T21:37:23.404 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:37:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f4006c8f0 msgr2=0x7f5f4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f4006c8f0 0x7f5f4006edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f440011e0 tx=0x7f5f44007d30 comp rx=0 tx=0).stop 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f540b5b10 msgr2=0x7f5f540ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f540b5b10 0x7f5f540ce570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800c500 tx=0x7f5f4800c9c0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 shutdown_connections 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f4006c8f0 0x7f5f4006edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f540d1a00 0x7f5f540ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f540b5b10 0x7f5f540ce570 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 --2- 172.21.15.40:0/1918392752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f540b40d0 0x7f5f540b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 >> 172.21.15.40:0/1918392752 conn(0x7f5f540d4fd0 msgr2=0x7f5f5406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 shutdown_connections 2024-04-26T21:37:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.406+0000 7f5f5a1f0640 1 -- 172.21.15.40:0/1918392752 wait complete. 2024-04-26T21:37:23.417 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:37:23.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.521+0000 7f5857e6d640 1 Processor -- start 2024-04-26T21:37:23.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.521+0000 7f5857e6d640 1 -- start start 2024-04-26T21:37:23.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5857e6d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58500cfde0 0x7f58500d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5857e6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f5850070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5857e6d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58500d2390 con 0x7f5850071090 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5857e6d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5850071500 con 0x7f58500cfde0 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5857e6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58500afab0 con 0x7f5850070350 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5855408640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f5850070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5855c09640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58500cfde0 0x7f58500d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5855408640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f5850070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57018/0 (socket says 172.21.15.40:57018) 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f585640a640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f5850071090 0x7f585015e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55688/0 (socket says 172.21.15.40:55688) 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5855408640 1 -- 172.21.15.40:0/501711159 learned_addr learned my addr 172.21.15.40:0/501711159 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.522+0000 7f5855c09640 1 --2- 172.21.15.40:0/501711159 >> v2:172.21.15.184:3300/0 conn(0x7f58500cfde0 0x7f58500d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:52694/0 (socket says 172.21.15.40:52694) 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.523+0000 7f5855c09640 1 -- 172.21.15.40:0/501711159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 msgr2=0x7f5850070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.523+0000 7f5855c09640 1 --2- 172.21.15.40:0/501711159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f5850070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.523+0000 7f5855c09640 1 -- 172.21.15.40:0/501711159 >> v1:172.21.15.184:6789/0 conn(0x7f5850071090 legacy=0x7f585015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.523+0000 7f5855c09640 1 -- 172.21.15.40:0/501711159 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585006d3f0 con 0x7f58500cfde0 2024-04-26T21:37:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.523+0000 7f5854c07640 1 -- 172.21.15.40:0/501711159 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3957129429 0 0) 0x7f58500d2390 con 0x7f5850071090 2024-04-26T21:37:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.523+0000 7f5855408640 1 --2- 172.21.15.40:0/501711159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f5850070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.523+0000 7f5855c09640 1 --2- 172.21.15.40:0/501711159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 0x7f58500d1b60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f584c0097a0 tx=0x7f584c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a537163af270ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5854c07640 1 -- 172.21.15.40:0/501711159 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c037070 con 0x7f58500cfde0 2024-04-26T21:37:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5854c07640 1 -- 172.21.15.40:0/501711159 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584c02ae50 con 0x7f58500cfde0 2024-04-26T21:37:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5854c07640 1 -- 172.21.15.40:0/501711159 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c032660 con 0x7f58500cfde0 2024-04-26T21:37:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5857e6d640 1 -- 172.21.15.40:0/501711159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 msgr2=0x7f58500d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5857e6d640 1 --2- 172.21.15.40:0/501711159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 0x7f58500d1b60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f584c0097a0 tx=0x7f584c02a100 comp rx=0 tx=0).stop 2024-04-26T21:37:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5857e6d640 1 -- 172.21.15.40:0/501711159 shutdown_connections 2024-04-26T21:37:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5857e6d640 1 --2- 172.21.15.40:0/501711159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f5850070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5857e6d640 1 --2- 172.21.15.40:0/501711159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 0x7f58500d1b60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5857e6d640 1 -- 172.21.15.40:0/501711159 >> 172.21.15.40:0/501711159 conn(0x7f58500d2d80 msgr2=0x7f58500d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5857e6d640 1 -- 172.21.15.40:0/501711159 shutdown_connections 2024-04-26T21:37:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.524+0000 7f5857e6d640 1 -- 172.21.15.40:0/501711159 wait complete. 2024-04-26T21:37:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.525+0000 7f5857e6d640 1 Processor -- start 2024-04-26T21:37:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.525+0000 7f5857e6d640 1 -- start start 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5857e6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f58501f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5857e6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5850071090 0x7f585015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5857e6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 0x7f58500b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5857e6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f585006d9b0 con 0x7f5850071090 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5857e6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f585006dd20 con 0x7f58500cfde0 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5857e6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58500d20a0 con 0x7f5850070350 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5855408640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5850071090 0x7f585015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5855408640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5850071090 0x7f585015f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35174/0 (socket says 172.21.15.40:35174) 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.526+0000 7f5855408640 1 -- 172.21.15.40:0/1063973776 learned_addr learned my addr 172.21.15.40:0/1063973776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5855c09640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f58501f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5855408640 1 -- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 msgr2=0x7f58501f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f585640a640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 0x7f58500b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5855408640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f58501f8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5855408640 1 -- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 msgr2=0x7f58500b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5855408640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 0x7f58500b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5855408640 1 -- 172.21.15.40:0/1063973776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585009cc60 con 0x7f5850071090 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5855c09640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f58501f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f585640a640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 0x7f58500b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5855408640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5850071090 0x7f585015f910 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f584000e9b0 tx=0x7f584000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f58467fc640 1 -- 172.21.15.40:0/1063973776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840017070 con 0x7f5850071090 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f58467fc640 1 -- 172.21.15.40:0/1063973776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584000c9a0 con 0x7f5850071090 2024-04-26T21:37:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585015fe50 con 0x7f5850071090 2024-04-26T21:37:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f58467fc640 1 -- 172.21.15.40:0/1063973776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584000cd50 con 0x7f5850071090 2024-04-26T21:37:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.527+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585015f390 con 0x7f5850071090 2024-04-26T21:37:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.528+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58500694f0 con 0x7f5850071090 2024-04-26T21:37:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.528+0000 7f58467fc640 1 -- 172.21.15.40:0/1063973776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f584001a070 con 0x7f5850071090 2024-04-26T21:37:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.529+0000 7f58467fc640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f583406c8f0 0x7f583406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.529+0000 7f58467fc640 1 -- 172.21.15.40:0/1063973776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f584008bbf0 con 0x7f5850071090 2024-04-26T21:37:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.529+0000 7f5855c09640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f583406c8f0 0x7f583406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.529+0000 7f5855c09640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f583406c8f0 0x7f583406edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f584c034000 tx=0x7f584c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.533+0000 7f58467fc640 1 -- 172.21.15.40:0/1063973776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5840059e30 con 0x7f5850071090 2024-04-26T21:37:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.719+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f585009cfc0 con 0x7f5850071090 2024-04-26T21:37:23.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.720+0000 7f58467fc640 1 -- 172.21.15.40:0/1063973776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f5840059730 con 0x7f5850071090 2024-04-26T21:37:23.720 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.721+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f583406c8f0 msgr2=0x7f583406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.721+0000 7f5857e6d640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f583406c8f0 0x7f583406edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f584c034000 tx=0x7f584c02e000 comp rx=0 tx=0).stop 2024-04-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.721+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5850071090 msgr2=0x7f585015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.721+0000 7f5857e6d640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5850071090 0x7f585015f910 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f584000e9b0 tx=0x7f584000ee70 comp rx=0 tx=0).stop 2024-04-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.722+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 shutdown_connections 2024-04-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.722+0000 7f5857e6d640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f583406c8f0 0x7f583406edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.722+0000 7f5857e6d640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58500cfde0 0x7f58500b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.722+0000 7f5857e6d640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5850071090 0x7f585015f910 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.722+0000 7f5857e6d640 1 --2- 172.21.15.40:0/1063973776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850070350 0x7f58501f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.722+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 >> 172.21.15.40:0/1063973776 conn(0x7f58500d2d80 msgr2=0x7f58500c68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.722+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 shutdown_connections 2024-04-26T21:37:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.722+0000 7f5857e6d640 1 -- 172.21.15.40:0/1063973776 wait complete. 2024-04-26T21:37:23.732 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:37:23.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.836+0000 7ff5c67e4640 1 Processor -- start 2024-04-26T21:37:23.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.836+0000 7ff5c67e4640 1 -- start start 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.836+0000 7ff5c67e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c67e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5c015f550 0x7ff5c00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c67e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5c00d34b0 0x7ff5c00d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c67e4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c00c9430 con 0x7ff5c00d34b0 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c67e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c00d3e20 con 0x7ff5c015f550 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c67e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c00d1040 con 0x7ff5c015f170 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c4d81640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5c00d34b0 0x7ff5c00d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c4d81640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5c00d34b0 0x7ff5c00d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35190/0 (socket says 172.21.15.40:35190) 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c4d81640 1 -- 172.21.15.40:0/404161791 learned_addr learned my addr 172.21.15.40:0/404161791 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:23.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5bffff640 1 --2- 172.21.15.40:0/404161791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5bf7fe640 1 --2- 172.21.15.40:0/404161791 >> v2:172.21.15.184:3300/0 conn(0x7ff5c015f550 0x7ff5c00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c4d81640 1 -- 172.21.15.40:0/404161791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 msgr2=0x7ff5c00d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c4d81640 1 --2- 172.21.15.40:0/404161791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c4d81640 1 -- 172.21.15.40:0/404161791 >> v2:172.21.15.184:3300/0 conn(0x7ff5c015f550 msgr2=0x7ff5c00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c4d81640 1 --2- 172.21.15.40:0/404161791 >> v2:172.21.15.184:3300/0 conn(0x7ff5c015f550 0x7ff5c00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5c4d81640 1 -- 172.21.15.40:0/404161791 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c00c9680 con 0x7ff5c00d34b0 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.837+0000 7ff5bffff640 1 --2- 172.21.15.40:0/404161791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c4d81640 1 --2- 172.21.15.40:0/404161791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c00d34b0 0x7ff5c00d38e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff5c008e960 tx=0x7ff5b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792005e5e6482af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5beffd640 1 -- 172.21.15.40:0/404161791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8036070 con 0x7ff5c00d34b0 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5beffd640 1 -- 172.21.15.40:0/404161791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8031600 con 0x7ff5c00d34b0 2024-04-26T21:37:23.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5beffd640 1 -- 172.21.15.40:0/404161791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80319d0 con 0x7ff5c00d34b0 2024-04-26T21:37:23.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 -- 172.21.15.40:0/404161791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c00d34b0 msgr2=0x7ff5c00d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/404161791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c00d34b0 0x7ff5c00d38e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff5c008e960 tx=0x7ff5b802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 -- 172.21.15.40:0/404161791 shutdown_connections 2024-04-26T21:37:23.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/404161791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c00d34b0 0x7ff5c00d38e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/404161791 >> v2:172.21.15.184:3300/0 conn(0x7ff5c015f550 0x7ff5c00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/404161791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 -- 172.21.15.40:0/404161791 >> 172.21.15.40:0/404161791 conn(0x7ff5c00d4f60 msgr2=0x7ff5c00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 -- 172.21.15.40:0/404161791 shutdown_connections 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.838+0000 7ff5c67e4640 1 -- 172.21.15.40:0/404161791 wait complete. 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.840+0000 7ff5c67e4640 1 Processor -- start 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.840+0000 7ff5c67e4640 1 -- start start 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.840+0000 7ff5c67e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d34b0 0x7ff5c00cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.840+0000 7ff5c67e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.840+0000 7ff5c67e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c015b690 0x7ff5c009ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.840+0000 7ff5c67e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00cfc50 con 0x7ff5c015b690 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.840+0000 7ff5c67e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00c9c40 con 0x7ff5c00d34b0 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.840+0000 7ff5c67e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c00cfdd0 con 0x7ff5c015f170 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c015b690 0x7ff5c009ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c015b690 0x7ff5c009ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35198/0 (socket says 172.21.15.40:35198) 2024-04-26T21:37:23.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 -- 172.21.15.40:0/3006226277 learned_addr learned my addr 172.21.15.40:0/3006226277 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5bffff640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d34b0 0x7ff5c00cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5bf7fe640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 -- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 msgr2=0x7ff5c00a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 -- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d34b0 msgr2=0x7ff5c00cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d34b0 0x7ff5c00cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 -- 172.21.15.40:0/3006226277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c00ce4c0 con 0x7ff5c015b690 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5bffff640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d34b0 0x7ff5c00cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c4d81640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c015b690 0x7ff5c009ec60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff5b802ac90 tx=0x7ff5b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5bcff9640 1 -- 172.21.15.40:0/3006226277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b803f020 con 0x7ff5c015b690 2024-04-26T21:37:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5bcff9640 1 -- 172.21.15.40:0/3006226277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8037040 con 0x7ff5c015b690 2024-04-26T21:37:23.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c00ce7a0 con 0x7ff5c015b690 2024-04-26T21:37:23.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5bcff9640 1 -- 172.21.15.40:0/3006226277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8036070 con 0x7ff5c015b690 2024-04-26T21:37:23.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.841+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c00cc3b0 con 0x7ff5c015b690 2024-04-26T21:37:23.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.842+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c00cc5b0 con 0x7ff5c015b690 2024-04-26T21:37:23.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.843+0000 7ff5bcff9640 1 -- 172.21.15.40:0/3006226277 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5b8042020 con 0x7ff5c015b690 2024-04-26T21:37:23.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.843+0000 7ff5bcff9640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5ac06c8f0 0x7ff5ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:23.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.843+0000 7ff5bcff9640 1 -- 172.21.15.40:0/3006226277 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7ff5b80ac270 con 0x7ff5c015b690 2024-04-26T21:37:23.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.844+0000 7ff5bffff640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5ac06c8f0 0x7ff5ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:23.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.844+0000 7ff5bffff640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5ac06c8f0 0x7ff5ac06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006b10 tx=0x7ff5b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:23.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:23.848+0000 7ff5bcff9640 1 -- 172.21.15.40:0/3006226277 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5b8049030 con 0x7ff5c015b690 2024-04-26T21:37:24.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.032+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5c00694f0 con 0x7ff5c015b690 2024-04-26T21:37:24.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.032+0000 7ff5bcff9640 1 -- 172.21.15.40:0/3006226277 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7ff5b807a490 con 0x7ff5c015b690 2024-04-26T21:37:24.032 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:37:24.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5ac06c8f0 msgr2=0x7ff5ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5ac06c8f0 0x7ff5ac06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006b10 tx=0x7ff5b0008040 comp rx=0 tx=0).stop 2024-04-26T21:37:24.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c015b690 msgr2=0x7ff5c009ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c015b690 0x7ff5c009ec60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff5b802ac90 tx=0x7ff5b8004a40 comp rx=0 tx=0).stop 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 shutdown_connections 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5ac06c8f0 0x7ff5ac06edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c015b690 0x7ff5c009ec60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c015f170 0x7ff5c00a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 --2- 172.21.15.40:0/3006226277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d34b0 0x7ff5c00cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 >> 172.21.15.40:0/3006226277 conn(0x7ff5c00d4f60 msgr2=0x7ff5c006e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 shutdown_connections 2024-04-26T21:37:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.034+0000 7ff5c67e4640 1 -- 172.21.15.40:0/3006226277 wait complete. 2024-04-26T21:37:24.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:24.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.145+0000 7f87bd5c1640 1 Processor -- start 2024-04-26T21:37:24.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.145+0000 7f87bd5c1640 1 -- start start 2024-04-26T21:37:24.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.146+0000 7f87bd5c1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87b806d560 0x7f87b80d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.146+0000 7f87bd5c1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87b80d2420 con 0x7f87b80d0070 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.146+0000 7f87bd5c1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87b80cd6b0 con 0x7f87b806e270 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.146+0000 7f87bd5c1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f87b8162d00 con 0x7f87b806d560 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.146+0000 7f87b6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f87b80d0070 0x7f87b806db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42508/0 (socket says 172.21.15.40:42508) 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.146+0000 7f87b6ffd640 1 -- 172.21.15.40:0/672013871 learned_addr learned my addr 172.21.15.40:0/672013871 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.147+0000 7f87b77fe640 1 --2- 172.21.15.40:0/672013871 >> v2:172.21.15.184:3300/0 conn(0x7f87b806d560 0x7f87b80d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.147+0000 7f87b5ffb640 1 -- 172.21.15.40:0/672013871 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3735548578 0 0) 0x7f87b80d2420 con 0x7f87b80d0070 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.147+0000 7f87b5ffb640 1 -- 172.21.15.40:0/672013871 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87a4003520 con 0x7f87b80d0070 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.147+0000 7f87b77fe640 1 -- 172.21.15.40:0/672013871 >> v1:172.21.15.184:6789/0 conn(0x7f87b806e270 legacy=0x7f87b80ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.147+0000 7f87b77fe640 1 -- 172.21.15.40:0/672013871 >> v1:172.21.15.40:6789/0 conn(0x7f87b80d0070 legacy=0x7f87b806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.147+0000 7f87b77fe640 1 -- 172.21.15.40:0/672013871 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b80cd990 con 0x7f87b806d560 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87b77fe640 1 --2- 172.21.15.40:0/672013871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b806d560 0x7f87b80d1d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43b33546464e095 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87b5ffb640 1 -- 172.21.15.40:0/672013871 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8007740 con 0x7f87b806d560 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87b5ffb640 1 -- 172.21.15.40:0/672013871 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a80078e0 con 0x7f87b806d560 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87b5ffb640 1 -- 172.21.15.40:0/672013871 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8007c90 con 0x7f87b806d560 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87bd5c1640 1 -- 172.21.15.40:0/672013871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b806d560 msgr2=0x7f87b80d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87bd5c1640 1 --2- 172.21.15.40:0/672013871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b806d560 0x7f87b80d1d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87bd5c1640 1 -- 172.21.15.40:0/672013871 shutdown_connections 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87bd5c1640 1 --2- 172.21.15.40:0/672013871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b806d560 0x7f87b80d1d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87bd5c1640 1 -- 172.21.15.40:0/672013871 >> 172.21.15.40:0/672013871 conn(0x7f87b80d2ec0 msgr2=0x7f87b80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87bd5c1640 1 -- 172.21.15.40:0/672013871 shutdown_connections 2024-04-26T21:37:24.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.148+0000 7f87bd5c1640 1 -- 172.21.15.40:0/672013871 wait complete. 2024-04-26T21:37:24.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.150+0000 7f87bd5c1640 1 Processor -- start 2024-04-26T21:37:24.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.150+0000 7f87bd5c1640 1 -- start start 2024-04-26T21:37:24.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.150+0000 7f87bd5c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b806d560 0x7f87b814eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.150+0000 7f87bd5c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b806e270 0x7f87b80c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.150+0000 7f87bd5c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b80d0070 0x7f87b80c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.150+0000 7f87bd5c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f87b80ce8a0 con 0x7f87b806d560 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.150+0000 7f87bd5c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87b80d2d10 con 0x7f87b80d0070 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.150+0000 7f87bd5c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87b80cea20 con 0x7f87b806e270 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b806d560 0x7f87b814eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b806d560 0x7f87b814eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35214/0 (socket says 172.21.15.40:35214) 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 -- 172.21.15.40:0/2636370093 learned_addr learned my addr 172.21.15.40:0/2636370093 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b77fe640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b80d0070 0x7f87b80c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b67fc640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b806e270 0x7f87b80c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 -- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b806e270 msgr2=0x7f87b80c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b806e270 0x7f87b80c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 -- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b80d0070 msgr2=0x7f87b80c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b80d0070 0x7f87b80c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 -- 172.21.15.40:0/2636370093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b80c86c0 con 0x7f87b806d560 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87b6ffd640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b806d560 0x7f87b814eff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f87b000c440 tx=0x7f87b000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87af7fe640 1 -- 172.21.15.40:0/2636370093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0017070 con 0x7f87b806d560 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87af7fe640 1 -- 172.21.15.40:0/2636370093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b0005ce0 con 0x7f87b806d560 2024-04-26T21:37:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b80c89a0 con 0x7f87b806d560 2024-04-26T21:37:24.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87af7fe640 1 -- 172.21.15.40:0/2636370093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00072b0 con 0x7f87b806d560 2024-04-26T21:37:24.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.151+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b80c8f60 con 0x7f87b806d560 2024-04-26T21:37:24.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.152+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b80c9340 con 0x7f87b806d560 2024-04-26T21:37:24.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.153+0000 7f87af7fe640 1 -- 172.21.15.40:0/2636370093 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87b001a070 con 0x7f87b806d560 2024-04-26T21:37:24.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.153+0000 7f87af7fe640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f879806c8f0 0x7f879806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.153+0000 7f87af7fe640 1 -- 172.21.15.40:0/2636370093 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f87b008af80 con 0x7f87b806d560 2024-04-26T21:37:24.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.153+0000 7f87b67fc640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f879806c8f0 0x7f879806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.154+0000 7f87b67fc640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f879806c8f0 0x7f879806edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f87b8160940 tx=0x7f87a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.157+0000 7f87af7fe640 1 -- 172.21.15.40:0/2636370093 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f87b80c9340 con 0x7f87b806d560 2024-04-26T21:37:24.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.386+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87b80694f0 con 0x7f87b806d560 2024-04-26T21:37:24.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.386+0000 7f87af7fe640 1 -- 172.21.15.40:0/2636370093 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5185 (secure 0 0 0) 0x7f87b0058cb0 con 0x7f87b806d560 2024-04-26T21:37:24.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:24.387 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:42.691649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5619,"qdb_cluster":[5619]},"id":2}]} 2024-04-26T21:37:24.387 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 23 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.388+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f879806c8f0 msgr2=0x7f879806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.388+0000 7f87bd5c1640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f879806c8f0 0x7f879806edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f87b8160940 tx=0x7f87a0009040 comp rx=0 tx=0).stop 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.388+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b806d560 msgr2=0x7f87b814eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.388+0000 7f87bd5c1640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b806d560 0x7f87b814eff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f87b000c440 tx=0x7f87b000c900 comp rx=0 tx=0).stop 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.388+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 shutdown_connections 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.388+0000 7f87bd5c1640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f879806c8f0 0x7f879806edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.388+0000 7f87bd5c1640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b80d0070 0x7f87b80c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.388+0000 7f87bd5c1640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b806e270 0x7f87b80c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.389+0000 7f87bd5c1640 1 --2- 172.21.15.40:0/2636370093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b806d560 0x7f87b814eff0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.389+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 >> 172.21.15.40:0/2636370093 conn(0x7f87b80d2ec0 msgr2=0x7f87b8074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.389+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 shutdown_connections 2024-04-26T21:37:24.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.389+0000 7f87bd5c1640 1 -- 172.21.15.40:0/2636370093 wait complete. 2024-04-26T21:37:24.400 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.500+0000 7f1087012640 1 Processor -- start 2024-04-26T21:37:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.500+0000 7f1087012640 1 -- start start 2024-04-26T21:37:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f1087012640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10800d3bc0 0x7f10800d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f1087012640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10800b7e50 con 0x7f1080162880 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f1087012640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10800d1800 con 0x7f10800b5f20 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f1087012640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10800d0580 con 0x7f10800d3bc0 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f1084dae640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1080162880 0x7f10800d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42522/0 (socket says 172.21.15.40:42522) 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f1084dae640 1 -- 172.21.15.40:0/1054778908 learned_addr learned my addr 172.21.15.40:0/1054778908 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524711390 0 0) 0x7f10800b7e50 con 0x7f1080162880 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1068003520 con 0x7f1080162880 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.501+0000 7f107ffff640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10800d3bc0 0x7f10800d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2155774679 0 0) 0x7f1068003520 con 0x7f1080162880 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10800d3bc0 msgr2=0x7f10800d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10800d3bc0 0x7f10800d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 >> v1:172.21.15.184:6789/0 conn(0x7f10800b5f20 legacy=0x7f10800b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10800d07d0 con 0x7f1080162880 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1070001670 con 0x7f1080162880 2024-04-26T21:37:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 >> v1:172.21.15.40:6789/0 conn(0x7f1080162880 legacy=0x7f10800d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10680039e0 0x7f1068003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10800d3bc0 0x7f1068004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1068004d40 0x7f10680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10800d0580 con 0x7f1068004d40 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f106800ba70 con 0x7f10680039e0 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f106800bc70 con 0x7f10800d3bc0 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f10855af640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1068004d40 0x7f10680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f1084dae640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10680039e0 0x7f1068003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f107ffff640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10800d3bc0 0x7f1068004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f10855af640 1 -- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10800d3bc0 msgr2=0x7f1068004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f10855af640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10800d3bc0 0x7f1068004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f10855af640 1 -- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10680039e0 msgr2=0x7f1068003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f10855af640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10680039e0 0x7f1068003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.502+0000 7f10855af640 1 -- 172.21.15.40:0/1054778908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1068003520 con 0x7f1068004d40 2024-04-26T21:37:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f10855af640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1068004d40 0x7f10680051c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1078001e20 tx=0x7f1068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321d9941a1fb8823 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078001a70 con 0x7f1068004d40 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107802ae40 con 0x7f1068004d40 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f107f7fe640 1 -- 172.21.15.40:0/1054778908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078003bb0 con 0x7f1068004d40 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f1087012640 1 -- 172.21.15.40:0/1054778908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1068004d40 msgr2=0x7f10680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f1087012640 1 --2- 172.21.15.40:0/1054778908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1068004d40 0x7f10680051c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1078001e20 tx=0x7f1068005830 comp rx=0 tx=0).stop 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f10855af640 1 -- 172.21.15.40:0/1054778908 reap_dead start 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f1087012640 1 -- 172.21.15.40:0/1054778908 shutdown_connections 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.503+0000 7f1087012640 1 -- 172.21.15.40:0/1054778908 >> 172.21.15.40:0/1054778908 conn(0x7f10800d4f60 msgr2=0x7f10800d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.504+0000 7f1087012640 1 -- 172.21.15.40:0/1054778908 shutdown_connections 2024-04-26T21:37:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.504+0000 7f1087012640 1 -- 172.21.15.40:0/1054778908 wait complete. 2024-04-26T21:37:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.505+0000 7f1087012640 1 Processor -- start 2024-04-26T21:37:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.505+0000 7f1087012640 1 -- start start 2024-04-26T21:37:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1087012640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f108015dbf0 0x7f10800a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1087012640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10800a1de0 0x7f108006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1087012640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f108006d250 0x7f10801f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1087012640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1080162c60 con 0x7f10800a1de0 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1087012640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10800d0d90 con 0x7f108015dbf0 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1087012640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1080162de0 con 0x7f108006d250 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10800a1de0 0x7f108006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10800a1de0 0x7f108006cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35238/0 (socket says 172.21.15.40:35238) 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 -- 172.21.15.40:0/3648963791 learned_addr learned my addr 172.21.15.40:0/3648963791 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f10855af640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f108006d250 0x7f10801f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1084dae640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f108015dbf0 0x7f10800a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 -- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f108006d250 msgr2=0x7f10801f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f108006d250 0x7f10801f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 -- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f108015dbf0 msgr2=0x7f10800a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f108015dbf0 0x7f10800a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 -- 172.21.15.40:0/3648963791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10800a2360 con 0x7f10800a1de0 2024-04-26T21:37:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1084dae640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f108015dbf0 0x7f10800a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107ffff640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10800a1de0 0x7f108006cd10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1074011a00 tx=0x7f1074011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107d7fa640 1 -- 172.21.15.40:0/3648963791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107400faf0 con 0x7f10800a1de0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107d7fa640 1 -- 172.21.15.40:0/3648963791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107400fc90 con 0x7f10800a1de0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10800a2640 con 0x7f10800a1de0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.506+0000 7f107d7fa640 1 -- 172.21.15.40:0/3648963791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074014440 con 0x7f10800a1de0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.507+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108009efe0 con 0x7f10800a1de0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.507+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10800694f0 con 0x7f10800a1de0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.508+0000 7f107d7fa640 1 -- 172.21.15.40:0/3648963791 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1074012040 con 0x7f10800a1de0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.508+0000 7f107d7fa640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.508+0000 7f107d7fa640 1 -- 172.21.15.40:0/3648963791 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f107408c3f0 con 0x7f10800a1de0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.508+0000 7f1084dae640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.509+0000 7f1084dae640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106c06c8f0 0x7f106c06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1070004840 tx=0x7f1070009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.513+0000 7f107d7fa640 1 -- 172.21.15.40:0/3648963791 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1074019030 con 0x7f10800a1de0 2024-04-26T21:37:24.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.700+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10800d3550 con 0x7f10800a1de0 2024-04-26T21:37:24.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.701+0000 7f107d7fa640 1 -- 172.21.15.40:0/3648963791 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17027 (secure 0 0 0) 0x7f107405a580 con 0x7f10800a1de0 2024-04-26T21:37:24.702 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:24.702 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:51.722847+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3403660773":"2024-04-27T21:36:38.666240+0000","172.21.15.40:6833/3403660773":"2024-04-27T21:36:38.666240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106c06c8f0 msgr2=0x7f106c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106c06c8f0 0x7f106c06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1070004840 tx=0x7f1070009040 comp rx=0 tx=0).stop 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10800a1de0 msgr2=0x7f108006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10800a1de0 0x7f108006cd10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1074011a00 tx=0x7f1074011ec0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 shutdown_connections 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f108006d250 0x7f10801f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10800a1de0 0x7f108006cd10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 --2- 172.21.15.40:0/3648963791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f108015dbf0 0x7f10800a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.703+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 >> 172.21.15.40:0/3648963791 conn(0x7f10800d4f60 msgr2=0x7f108015f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.704+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 shutdown_connections 2024-04-26T21:37:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.704+0000 7f1087012640 1 -- 172.21.15.40:0/3648963791 wait complete. 2024-04-26T21:37:24.715 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:37:24.715 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:24.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.817+0000 7ff290cf8640 1 Processor -- start 2024-04-26T21:37:24.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.817+0000 7ff290cf8640 1 -- start start 2024-04-26T21:37:24.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff290cf8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff28c15f3a0 0x7ff28c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff290cf8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c0d2330 0x7ff28c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff290cf8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff28c0d0e20 con 0x7ff28c0d1770 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff290cf8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff28c0b1d70 con 0x7ff28c15f3a0 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff290cf8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c0b2050 con 0x7ff28c0d2330 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff289d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff28c0d1770 0x7ff28c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42534/0 (socket says 172.21.15.40:42534) 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff289d9b640 1 -- 172.21.15.40:0/845338797 learned_addr learned my addr 172.21.15.40:0/845338797 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28a59c640 1 --2- 172.21.15.40:0/845338797 >> v2:172.21.15.184:3300/0 conn(0x7ff28c15f3a0 0x7ff28c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28ad9d640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c0d2330 0x7ff28c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104240183 0 0) 0x7ff28c0d0e20 con 0x7ff28c0d1770 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff274003520 con 0x7ff28c0d1770 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3036851967 0 0) 0x7ff274003520 con 0x7ff28c0d1770 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c0d2330 msgr2=0x7ff28c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c0d2330 0x7ff28c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 >> v2:172.21.15.184:3300/0 conn(0x7ff28c15f3a0 msgr2=0x7ff28c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 --2- 172.21.15.40:0/845338797 >> v2:172.21.15.184:3300/0 conn(0x7ff28c15f3a0 0x7ff28c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c0b2330 con 0x7ff28c0d1770 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7ff280001670 con 0x7ff28c0d1770 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 >> v1:172.21.15.40:6789/0 conn(0x7ff28c0d1770 legacy=0x7ff28c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039b0 0x7ff274003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274004800 0x7ff274004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c0d2330 0x7ff2740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c0b1d70 con 0x7ff274004800 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c0b2050 con 0x7ff2740039b0 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff27400b9c0 con 0x7ff28c0d2330 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.818+0000 7ff28a59c640 1 --2- 172.21.15.40:0/845338797 >> v2:172.21.15.184:3300/0 conn(0x7ff28c15f3a0 0x7ff28c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff289d9b640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274004800 0x7ff274004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff28ad9d640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c0d2330 0x7ff2740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff28a59c640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039b0 0x7ff274003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff289d9b640 1 -- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c0d2330 msgr2=0x7ff2740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff289d9b640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c0d2330 0x7ff2740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff289d9b640 1 -- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039b0 msgr2=0x7ff274003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff289d9b640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039b0 0x7ff274003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff289d9b640 1 -- 172.21.15.40:0/845338797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274003520 con 0x7ff274004800 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.819+0000 7ff289d9b640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274004800 0x7ff274004c80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c210 tx=0x7ff274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82de3a910c3772b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280007d50 con 0x7ff274004800 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff280005ad0 con 0x7ff274004800 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff28959a640 1 -- 172.21.15.40:0/845338797 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280007070 con 0x7ff274004800 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff290cf8640 1 -- 172.21.15.40:0/845338797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274004800 msgr2=0x7ff274004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff290cf8640 1 --2- 172.21.15.40:0/845338797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274004800 0x7ff274004c80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c210 tx=0x7ff274005830 comp rx=0 tx=0).stop 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff28ad9d640 1 -- 172.21.15.40:0/845338797 reap_dead start 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff290cf8640 1 -- 172.21.15.40:0/845338797 shutdown_connections 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff290cf8640 1 -- 172.21.15.40:0/845338797 >> 172.21.15.40:0/845338797 conn(0x7ff28c0d4f90 msgr2=0x7ff28c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff290cf8640 1 -- 172.21.15.40:0/845338797 shutdown_connections 2024-04-26T21:37:24.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.820+0000 7ff290cf8640 1 -- 172.21.15.40:0/845338797 wait complete. 2024-04-26T21:37:24.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.822+0000 7ff290cf8640 1 Processor -- start 2024-04-26T21:37:24.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.822+0000 7ff290cf8640 1 -- start start 2024-04-26T21:37:24.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.822+0000 7ff290cf8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff274004800 0x7ff28c15c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.822+0000 7ff290cf8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff28c0ba920 0x7ff28c1f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.822+0000 7ff290cf8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c15d380 0x7ff28c15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.822+0000 7ff290cf8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c0b29d0 con 0x7ff28c0ba920 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.822+0000 7ff290cf8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c0b2850 con 0x7ff274004800 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.822+0000 7ff290cf8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c0b2b50 con 0x7ff28c15d380 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff28c0ba920 0x7ff28c1f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff28c0ba920 0x7ff28c1f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35252/0 (socket says 172.21.15.40:35252) 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 -- 172.21.15.40:0/2917169928 learned_addr learned my addr 172.21.15.40:0/2917169928 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff28ad9d640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c15d380 0x7ff28c15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff28a59c640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff274004800 0x7ff28c15c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 -- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c15d380 msgr2=0x7ff28c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c15d380 0x7ff28c15d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 -- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff274004800 msgr2=0x7ff28c15c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff274004800 0x7ff28c15c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 -- 172.21.15.40:0/2917169928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c15cbf0 con 0x7ff28c0ba920 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff289d9b640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff28c0ba920 0x7ff28c1f4fc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c210 tx=0x7ff280001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff27affd640 1 -- 172.21.15.40:0/2917169928 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280042070 con 0x7ff28c0ba920 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff27affd640 1 -- 172.21.15.40:0/2917169928 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff280007480 con 0x7ff28c0ba920 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff27affd640 1 -- 172.21.15.40:0/2917169928 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280001c90 con 0x7ff28c0ba920 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c1f5930 con 0x7ff28c0ba920 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.823+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1f5ef0 con 0x7ff28c0ba920 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.824+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c0694f0 con 0x7ff28c0ba920 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.825+0000 7ff27affd640 1 -- 172.21.15.40:0/2917169928 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff280045070 con 0x7ff28c0ba920 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.825+0000 7ff27affd640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff26c06c8f0 0x7ff26c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.825+0000 7ff27affd640 1 -- 172.21.15.40:0/2917169928 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7ff2800ac8a0 con 0x7ff28c0ba920 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.825+0000 7ff28a59c640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff26c06c8f0 0x7ff26c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.826+0000 7ff28a59c640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff26c06c8f0 0x7ff26c06edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff28c162ef0 tx=0x7ff27c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:24.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:24.829+0000 7ff27affd640 1 -- 172.21.15.40:0/2917169928 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff28004c030 con 0x7ff28c0ba920 2024-04-26T21:37:25.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.054+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff28c0b6860 con 0x7ff28c0ba920 2024-04-26T21:37:25.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.055+0000 7ff27affd640 1 -- 172.21.15.40:0/2917169928 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5185 (secure 0 0 0) 0x7ff28007a490 con 0x7ff28c0ba920 2024-04-26T21:37:25.055 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:25.055 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:42.691649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5619,"qdb_cluster":[5619]},"id":2}]} 2024-04-26T21:37:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 23 2024-04-26T21:37:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff26c06c8f0 msgr2=0x7ff26c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff26c06c8f0 0x7ff26c06edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff28c162ef0 tx=0x7ff27c008040 comp rx=0 tx=0).stop 2024-04-26T21:37:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff28c0ba920 msgr2=0x7ff28c1f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff28c0ba920 0x7ff28c1f4fc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c210 tx=0x7ff280001a40 comp rx=0 tx=0).stop 2024-04-26T21:37:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 shutdown_connections 2024-04-26T21:37:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff26c06c8f0 0x7ff26c06edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28c15d380 0x7ff28c15d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff28c0ba920 0x7ff28c1f4fc0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 --2- 172.21.15.40:0/2917169928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff274004800 0x7ff28c15c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 >> 172.21.15.40:0/2917169928 conn(0x7ff28c0d4f90 msgr2=0x7ff28c0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:25.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 shutdown_connections 2024-04-26T21:37:25.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.057+0000 7ff290cf8640 1 -- 172.21.15.40:0/2917169928 wait complete. 2024-04-26T21:37:25.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:25.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.169+0000 7fb46c437640 1 Processor -- start 2024-04-26T21:37:25.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.169+0000 7fb46c437640 1 -- start start 2024-04-26T21:37:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46c437640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb464162810 0x7fb464070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46c437640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb4640d1b30 0x7fb4640d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46c437640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4640d10d0 con 0x7fb4640b2340 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46c437640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb4640d0750 con 0x7fb4640d1b30 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46c437640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4640d16a0 con 0x7fb464162810 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb4699d2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb4640b2340 0x7fb4640cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42536/0 (socket says 172.21.15.40:42536) 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb4699d2640 1 -- 172.21.15.40:0/3612418910 learned_addr learned my addr 172.21.15.40:0/3612418910 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46a1d3640 1 --2- 172.21.15.40:0/3612418910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb464162810 0x7fb464070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46a9d4640 1 --2- 172.21.15.40:0/3612418910 >> v2:172.21.15.184:3300/0 conn(0x7fb4640d1b30 0x7fb4640d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb4691d1640 1 -- 172.21.15.40:0/3612418910 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593971159 0 0) 0x7fb4640d10d0 con 0x7fb4640b2340 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb4691d1640 1 -- 172.21.15.40:0/3612418910 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb450003520 con 0x7fb4640b2340 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46a1d3640 1 -- 172.21.15.40:0/3612418910 >> v2:172.21.15.184:3300/0 conn(0x7fb4640d1b30 msgr2=0x7fb4640d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46a1d3640 1 --2- 172.21.15.40:0/3612418910 >> v2:172.21.15.184:3300/0 conn(0x7fb4640d1b30 0x7fb4640d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46a1d3640 1 -- 172.21.15.40:0/3612418910 >> v1:172.21.15.40:6789/0 conn(0x7fb4640b2340 legacy=0x7fb4640cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.170+0000 7fb46a1d3640 1 -- 172.21.15.40:0/3612418910 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4640d1370 con 0x7fb464162810 2024-04-26T21:37:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.171+0000 7fb46a1d3640 1 --2- 172.21.15.40:0/3612418910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb464162810 0x7fb464070350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4600097a0 tx=0x7fb46002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc63e95a354e678 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.171+0000 7fb4691d1640 1 -- 172.21.15.40:0/3612418910 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460037070 con 0x7fb464162810 2024-04-26T21:37:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.171+0000 7fb4691d1640 1 -- 172.21.15.40:0/3612418910 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb460032440 con 0x7fb464162810 2024-04-26T21:37:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.171+0000 7fb4691d1640 1 -- 172.21.15.40:0/3612418910 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4600327f0 con 0x7fb464162810 2024-04-26T21:37:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.172+0000 7fb46c437640 1 -- 172.21.15.40:0/3612418910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb464162810 msgr2=0x7fb464070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.172+0000 7fb46c437640 1 --2- 172.21.15.40:0/3612418910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb464162810 0x7fb464070350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4600097a0 tx=0x7fb46002a180 comp rx=0 tx=0).stop 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.172+0000 7fb46c437640 1 -- 172.21.15.40:0/3612418910 shutdown_connections 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.172+0000 7fb46c437640 1 --2- 172.21.15.40:0/3612418910 >> v2:172.21.15.184:3300/0 conn(0x7fb4640d1b30 0x7fb4640d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.172+0000 7fb46c437640 1 --2- 172.21.15.40:0/3612418910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb464162810 0x7fb464070350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.172+0000 7fb46c437640 1 -- 172.21.15.40:0/3612418910 >> 172.21.15.40:0/3612418910 conn(0x7fb4640d2da0 msgr2=0x7fb4640d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.172+0000 7fb46c437640 1 -- 172.21.15.40:0/3612418910 shutdown_connections 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.172+0000 7fb46c437640 1 -- 172.21.15.40:0/3612418910 wait complete. 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.173+0000 7fb46c437640 1 Processor -- start 2024-04-26T21:37:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.173+0000 7fb46c437640 1 -- start start 2024-04-26T21:37:25.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46c437640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b2340 0x7fb464149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46c437640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4640d1b30 0x7fb46415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46c437640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb464162810 0x7fb46415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46c437640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb464070890 con 0x7fb464162810 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46c437640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4640d0470 con 0x7fb4640d1b30 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46c437640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4640cdb00 con 0x7fb4640b2340 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb464162810 0x7fb46415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb464162810 0x7fb46415c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35258/0 (socket says 172.21.15.40:35258) 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 -- 172.21.15.40:0/2420567920 learned_addr learned my addr 172.21.15.40:0/2420567920 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a1d3640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b2340 0x7fb464149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb4699d2640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4640d1b30 0x7fb46415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 -- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b2340 msgr2=0x7fb464149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b2340 0x7fb464149c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 -- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4640d1b30 msgr2=0x7fb46415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4640d1b30 0x7fb46415bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 -- 172.21.15.40:0/2420567920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46415ac00 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.174+0000 7fb46a9d4640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb464162810 0x7fb46415c1a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009dd0 tx=0x7fb45c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.175+0000 7fb45affd640 1 -- 172.21.15.40:0/2420567920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c018020 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.175+0000 7fb45affd640 1 -- 172.21.15.40:0/2420567920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c014480 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.175+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46415aea0 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.175+0000 7fb45affd640 1 -- 172.21.15.40:0/2420567920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c014870 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.175+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46415b460 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.176+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4640694f0 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.176+0000 7fb45affd640 1 -- 172.21.15.40:0/2420567920 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb45c005d10 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.176+0000 7fb45affd640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb44806c8f0 0x7fb44806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.177+0000 7fb45affd640 1 -- 172.21.15.40:0/2420567920 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fb45c08c8d0 con 0x7fb464162810 2024-04-26T21:37:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.177+0000 7fb46a1d3640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb44806c8f0 0x7fb44806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.177+0000 7fb46a1d3640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb44806c8f0 0x7fb44806edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb460034000 tx=0x7fb46002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.181+0000 7fb45affd640 1 -- 172.21.15.40:0/2420567920 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb45c022070 con 0x7fb464162810 2024-04-26T21:37:25.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.405+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb464069ca0 con 0x7fb464162810 2024-04-26T21:37:25.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.406+0000 7fb45affd640 1 -- 172.21.15.40:0/2420567920 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5185 (secure 0 0 0) 0x7fb45c010040 con 0x7fb464162810 2024-04-26T21:37:25.406 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:25.406 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:36:23.780134+0000","modified":"2024-04-26T21:36:42.691649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5619":{"gid":5619,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1366266306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1366266306},{"type":"v1","addr":"172.21.15.40:6835","nonce":1366266306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5619,"qdb_cluster":[5619]},"id":2}]} 2024-04-26T21:37:25.406 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 23 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb44806c8f0 msgr2=0x7fb44806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb44806c8f0 0x7fb44806edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb460034000 tx=0x7fb46002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb464162810 msgr2=0x7fb46415c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb464162810 0x7fb46415c1a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009dd0 tx=0x7fb45c00c490 comp rx=0 tx=0).stop 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 shutdown_connections 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb44806c8f0 0x7fb44806edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb464162810 0x7fb46415c1a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4640d1b30 0x7fb46415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 --2- 172.21.15.40:0/2420567920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b2340 0x7fb464149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 >> 172.21.15.40:0/2420567920 conn(0x7fb4640d2da0 msgr2=0x7fb4640b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:25.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 shutdown_connections 2024-04-26T21:37:25.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.408+0000 7fb46c437640 1 -- 172.21.15.40:0/2420567920 wait complete. 2024-04-26T21:37:25.419 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.519+0000 7fa8a3de4640 1 Processor -- start 2024-04-26T21:37:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.519+0000 7fa8a3de4640 1 -- start start 2024-04-26T21:37:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a3de4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c071e60 0x7fa89c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a3de4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa89c06fe20 0x7fa89c072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a3de4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89c15d7c0 con 0x7fa89c072780 2024-04-26T21:37:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a3de4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa89c15daa0 con 0x7fa89c06fe20 2024-04-26T21:37:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a3de4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c15dd80 con 0x7fa89c071e60 2024-04-26T21:37:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a137f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa89c06fe20 0x7fa89c072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a2381640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa89c072780 0x7fa89c0cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42552/0 (socket says 172.21.15.40:42552) 2024-04-26T21:37:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a2381640 1 -- 172.21.15.40:0/2686481854 learned_addr learned my addr 172.21.15.40:0/2686481854 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a1b80640 1 --2- 172.21.15.40:0/2686481854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c071e60 0x7fa89c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a137f640 1 -- 172.21.15.40:0/2686481854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c071e60 msgr2=0x7fa89c06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a137f640 1 --2- 172.21.15.40:0/2686481854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c071e60 0x7fa89c06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a137f640 1 -- 172.21.15.40:0/2686481854 >> v1:172.21.15.40:6789/0 conn(0x7fa89c072780 legacy=0x7fa89c0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a0b7e640 1 -- 172.21.15.40:0/2686481854 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054023360 0 0) 0x7fa89c15d7c0 con 0x7fa89c072780 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a137f640 1 -- 172.21.15.40:0/2686481854 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c15e060 con 0x7fa89c06fe20 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.520+0000 7fa8a1b80640 1 --2- 172.21.15.40:0/2686481854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c071e60 0x7fa89c06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.521+0000 7fa8a137f640 1 --2- 172.21.15.40:0/2686481854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c06fe20 0x7fa89c072240 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0097a0 tx=0x7fa88c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21f079c9de0b0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.521+0000 7fa8a0b7e640 1 -- 172.21.15.40:0/2686481854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c037070 con 0x7fa89c06fe20 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.521+0000 7fa8a0b7e640 1 -- 172.21.15.40:0/2686481854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88c032440 con 0x7fa89c06fe20 2024-04-26T21:37:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.521+0000 7fa8a0b7e640 1 -- 172.21.15.40:0/2686481854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c032810 con 0x7fa89c06fe20 2024-04-26T21:37:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.522+0000 7fa8a3de4640 1 -- 172.21.15.40:0/2686481854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c06fe20 msgr2=0x7fa89c072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.522+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/2686481854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c06fe20 0x7fa89c072240 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0097a0 tx=0x7fa88c02a180 comp rx=0 tx=0).stop 2024-04-26T21:37:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.522+0000 7fa8a3de4640 1 -- 172.21.15.40:0/2686481854 shutdown_connections 2024-04-26T21:37:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.522+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/2686481854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c06fe20 0x7fa89c072240 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.522+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/2686481854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c071e60 0x7fa89c06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.522+0000 7fa8a3de4640 1 -- 172.21.15.40:0/2686481854 >> 172.21.15.40:0/2686481854 conn(0x7fa89c0d0be0 msgr2=0x7fa89c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.522+0000 7fa8a3de4640 1 -- 172.21.15.40:0/2686481854 shutdown_connections 2024-04-26T21:37:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.522+0000 7fa8a3de4640 1 -- 172.21.15.40:0/2686481854 wait complete. 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.523+0000 7fa8a3de4640 1 Processor -- start 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.523+0000 7fa8a3de4640 1 -- start start 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a3de4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c06fe20 0x7fa89c0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a3de4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c071e60 0x7fa89c1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a3de4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa89c072780 0x7fa89c156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a3de4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c15d3f0 con 0x7fa89c071e60 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a3de4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c072c10 con 0x7fa89c072780 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a3de4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c15d570 con 0x7fa89c06fe20 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c071e60 0x7fa89c1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c071e60 0x7fa89c1560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35282/0 (socket says 172.21.15.40:35282) 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 -- 172.21.15.40:0/3331008431 learned_addr learned my addr 172.21.15.40:0/3331008431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a1b80640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c06fe20 0x7fa89c0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a2381640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa89c072780 0x7fa89c156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 -- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c06fe20 msgr2=0x7fa89c0c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c06fe20 0x7fa89c0c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 -- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa89c072780 msgr2=0x7fa89c156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa89c072780 0x7fa89c156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 -- 172.21.15.40:0/3331008431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c0ae8d0 con 0x7fa89c071e60 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a2381640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa89c072780 0x7fa89c156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.524+0000 7fa8a137f640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c071e60 0x7fa89c1560d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa88c035040 tx=0x7fa88c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.525+0000 7fa8927fc640 1 -- 172.21.15.40:0/3331008431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c037050 con 0x7fa89c071e60 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.525+0000 7fa8927fc640 1 -- 172.21.15.40:0/3331008431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88c038040 con 0x7fa89c071e60 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.525+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c0aebb0 con 0x7fa89c071e60 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.525+0000 7fa8927fc640 1 -- 172.21.15.40:0/3331008431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c004470 con 0x7fa89c071e60 2024-04-26T21:37:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.525+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c0af170 con 0x7fa89c071e60 2024-04-26T21:37:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.526+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa89c0694f0 con 0x7fa89c071e60 2024-04-26T21:37:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.526+0000 7fa8927fc640 1 -- 172.21.15.40:0/3331008431 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa88c030900 con 0x7fa89c071e60 2024-04-26T21:37:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.527+0000 7fa8927fc640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa88006c8f0 0x7fa88006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.527+0000 7fa8927fc640 1 -- 172.21.15.40:0/3331008431 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fa88c0ac880 con 0x7fa89c071e60 2024-04-26T21:37:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.527+0000 7fa8a1b80640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa88006c8f0 0x7fa88006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.528+0000 7fa8a1b80640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa88006c8f0 0x7fa88006edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa898006fd0 tx=0x7fa898008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.531+0000 7fa8927fc640 1 -- 172.21.15.40:0/3331008431 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa88c07aa10 con 0x7fa89c071e60 2024-04-26T21:37:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.719+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa89c15cc10 con 0x7fa89c071e60 2024-04-26T21:37:25.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.720+0000 7fa8927fc640 1 -- 172.21.15.40:0/3331008431 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17027 (secure 0 0 0) 0x7fa88c07a310 con 0x7fa89c071e60 2024-04-26T21:37:25.721 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:25.721 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:36:51.722847+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:36:22.370518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T21:36:22.724779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3403660773":"2024-04-27T21:36:38.666240+0000","172.21.15.40:6833/3403660773":"2024-04-27T21:36:38.666240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa88006c8f0 msgr2=0x7fa88006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa88006c8f0 0x7fa88006edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa898006fd0 tx=0x7fa898008040 comp rx=0 tx=0).stop 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c071e60 msgr2=0x7fa89c1560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c071e60 0x7fa89c1560d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa88c035040 tx=0x7fa88c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 shutdown_connections 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa88006c8f0 0x7fa88006edb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa89c072780 0x7fa89c156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa89c071e60 0x7fa89c1560d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 --2- 172.21.15.40:0/3331008431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa89c06fe20 0x7fa89c0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 >> 172.21.15.40:0/3331008431 conn(0x7fa89c0d0be0 msgr2=0x7fa89c159120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:25.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 shutdown_connections 2024-04-26T21:37:25.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.723+0000 7fa8a3de4640 1 -- 172.21.15.40:0/3331008431 wait complete. 2024-04-26T21:37:25.734 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:37:25.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.838+0000 7efdc6a7e640 1 Processor -- start 2024-04-26T21:37:25.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.838+0000 7efdc6a7e640 1 -- start start 2024-04-26T21:37:25.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.838+0000 7efdc6a7e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc015d1e0 0x7efdc00d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.838+0000 7efdc6a7e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efdc00d1a90 0x7efdc0070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.838+0000 7efdc6a7e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc00ce840 con 0x7efdc00d0f20 2024-04-26T21:37:25.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.838+0000 7efdc6a7e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efdc00cede0 con 0x7efdc00d1a90 2024-04-26T21:37:25.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.838+0000 7efdc6a7e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdc0070650 con 0x7efdc015d1e0 2024-04-26T21:37:25.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.839+0000 7efdc501b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efdc00d1a90 0x7efdc0070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.839+0000 7efdc481a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc015d1e0 0x7efdc00d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.839+0000 7efdbffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7efdc00d0f20 0x7efdc00d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55716/0 (socket says 172.21.15.40:55716) 2024-04-26T21:37:25.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.839+0000 7efdbffff640 1 -- 172.21.15.40:0/717956523 learned_addr learned my addr 172.21.15.40:0/717956523 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:25.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.839+0000 7efdc501b640 1 -- 172.21.15.40:0/717956523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc015d1e0 msgr2=0x7efdc00d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:25.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.839+0000 7efdc501b640 1 --2- 172.21.15.40:0/717956523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc015d1e0 0x7efdc00d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.839+0000 7efdc501b640 1 -- 172.21.15.40:0/717956523 >> v1:172.21.15.184:6789/0 conn(0x7efdc00d0f20 legacy=0x7efdc00d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.839+0000 7efdc501b640 1 -- 172.21.15.40:0/717956523 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc00ceb00 con 0x7efdc00d1a90 2024-04-26T21:37:25.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.840+0000 7efdc501b640 1 --2- 172.21.15.40:0/717956523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00d1a90 0x7efdc0070ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efdb00097a0 tx=0x7efdb002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=134846140ec0d336 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.840+0000 7efdbf7fe640 1 -- 172.21.15.40:0/717956523 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0037070 con 0x7efdc00d1a90 2024-04-26T21:37:25.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.840+0000 7efdbf7fe640 1 -- 172.21.15.40:0/717956523 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb0032440 con 0x7efdc00d1a90 2024-04-26T21:37:25.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.840+0000 7efdbf7fe640 1 -- 172.21.15.40:0/717956523 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0032810 con 0x7efdc00d1a90 2024-04-26T21:37:25.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.841+0000 7efdc6a7e640 1 -- 172.21.15.40:0/717956523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00d1a90 msgr2=0x7efdc0070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.841+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/717956523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00d1a90 0x7efdc0070ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efdb00097a0 tx=0x7efdb002a180 comp rx=0 tx=0).stop 2024-04-26T21:37:25.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.841+0000 7efdc6a7e640 1 -- 172.21.15.40:0/717956523 shutdown_connections 2024-04-26T21:37:25.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.841+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/717956523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00d1a90 0x7efdc0070ae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.841+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/717956523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc015d1e0 0x7efdc00d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.841+0000 7efdc6a7e640 1 -- 172.21.15.40:0/717956523 >> 172.21.15.40:0/717956523 conn(0x7efdc00d2da0 msgr2=0x7efdc00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:25.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.841+0000 7efdc6a7e640 1 -- 172.21.15.40:0/717956523 shutdown_connections 2024-04-26T21:37:25.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.841+0000 7efdc6a7e640 1 -- 172.21.15.40:0/717956523 wait complete. 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.842+0000 7efdc6a7e640 1 Processor -- start 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.842+0000 7efdc6a7e640 1 -- start start 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc6a7e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00d0f20 0x7efdc009db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc6a7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdc00d1a90 0x7efdc015c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc6a7e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc015d1e0 0x7efdc00c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc6a7e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0071020 con 0x7efdc00d1a90 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc6a7e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdc00ce4c0 con 0x7efdc015d1e0 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc6a7e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdc00711a0 con 0x7efdc00d0f20 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdc00d1a90 0x7efdc015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdc00d1a90 0x7efdc015c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35288/0 (socket says 172.21.15.40:35288) 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 -- 172.21.15.40:0/2850543096 learned_addr learned my addr 172.21.15.40:0/2850543096 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc481a640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00d0f20 0x7efdc009db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc501b640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc015d1e0 0x7efdc00c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 -- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00d0f20 msgr2=0x7efdc009db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00d0f20 0x7efdc009db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 -- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc015d1e0 msgr2=0x7efdc00c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc015d1e0 0x7efdc00c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 -- 172.21.15.40:0/2850543096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc009f500 con 0x7efdc00d1a90 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdc501b640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc015d1e0 0x7efdc00c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbffff640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdc00d1a90 0x7efdc015c320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efda800c440 tx=0x7efda8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.843+0000 7efdbd7fa640 1 -- 172.21.15.40:0/2850543096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8017070 con 0x7efdc00d1a90 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.844+0000 7efdbd7fa640 1 -- 172.21.15.40:0/2850543096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda8005ce0 con 0x7efdc00d1a90 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.844+0000 7efdbd7fa640 1 -- 172.21.15.40:0/2850543096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda80072f0 con 0x7efdc00d1a90 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.844+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc009f770 con 0x7efdc00d1a90 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.844+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc009fd30 con 0x7efdc00d1a90 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.845+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc00694f0 con 0x7efdc00d1a90 2024-04-26T21:37:25.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.845+0000 7efdbd7fa640 1 -- 172.21.15.40:0/2850543096 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efda801a070 con 0x7efdc00d1a90 2024-04-26T21:37:25.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.845+0000 7efdbd7fa640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:25.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.846+0000 7efdbd7fa640 1 -- 172.21.15.40:0/2850543096 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7efda808b070 con 0x7efdc00d1a90 2024-04-26T21:37:25.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.846+0000 7efdc481a640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:25.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.846+0000 7efdc481a640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efdb80073f0 tx=0x7efdb800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:25.850+0000 7efdbd7fa640 1 -- 172.21.15.40:0/2850543096 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efda8059330 con 0x7efdc00d1a90 2024-04-26T21:37:26.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.078+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efdc00b3bd0 con 0x7efdc00d1a90 2024-04-26T21:37:26.078 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:37:26.078+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:37:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.080+0000 7efdbd7fa640 1 -- 172.21.15.40:0/2850543096 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7efda8058c30 con 0x7efdc00d1a90 2024-04-26T21:37:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:37:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda006c8f0 msgr2=0x7efda006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efdb80073f0 tx=0x7efdb800b040 comp rx=0 tx=0).stop 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdc00d1a90 msgr2=0x7efdc015c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdc00d1a90 0x7efdc015c320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efda800c440 tx=0x7efda8001e70 comp rx=0 tx=0).stop 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 shutdown_connections 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc015d1e0 0x7efdc00c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdc00d1a90 0x7efdc015c320 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 --2- 172.21.15.40:0/2850543096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00d0f20 0x7efdc009db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 >> 172.21.15.40:0/2850543096 conn(0x7efdc00d2da0 msgr2=0x7efdc00b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 shutdown_connections 2024-04-26T21:37:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.082+0000 7efdc6a7e640 1 -- 172.21.15.40:0/2850543096 wait complete. 2024-04-26T21:37:26.085 INFO:tasks.ceph.mds.a.smithi040.stderr: -3528> 2024-04-26T21:36:44.679+0000 7f7d240cc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:37:26.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:37:26.117 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:37:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.199+0000 7f473d6d1640 1 Processor -- start 2024-04-26T21:37:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.199+0000 7f473d6d1640 1 -- start start 2024-04-26T21:37:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f473d6d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47380d06f0 0x7f47380d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f473d6d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f47380b2850 0x7f47380b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f473d6d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47380d0ba0 con 0x7f473815f3a0 2024-04-26T21:37:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f473d6d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f47380d0e20 con 0x7f47380d06f0 2024-04-26T21:37:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f473d6d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f47380b1d70 con 0x7f47380b2850 2024-04-26T21:37:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f47367fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47380d06f0 0x7f47380d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f4736ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f473815f3a0 0x7f47380d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42570/0 (socket says 172.21.15.40:42570) 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f4736ffd640 1 -- 172.21.15.40:0/3733755995 learned_addr learned my addr 172.21.15.40:0/3733755995 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f47377fe640 1 --2- 172.21.15.40:0/3733755995 >> v2:172.21.15.184:3300/0 conn(0x7f47380b2850 0x7f47380b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f47367fc640 1 -- 172.21.15.40:0/3733755995 >> v2:172.21.15.184:3300/0 conn(0x7f47380b2850 msgr2=0x7f47380b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f47367fc640 1 --2- 172.21.15.40:0/3733755995 >> v2:172.21.15.184:3300/0 conn(0x7f47380b2850 0x7f47380b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f47367fc640 1 -- 172.21.15.40:0/3733755995 >> v1:172.21.15.40:6789/0 conn(0x7f473815f3a0 legacy=0x7f47380d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f47367fc640 1 -- 172.21.15.40:0/3733755995 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47380b2050 con 0x7f47380d06f0 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f4735ffb640 1 -- 172.21.15.40:0/3733755995 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2702140039 0 0) 0x7f4728001110 con 0x7f473815f3a0 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.200+0000 7f47377fe640 1 --2- 172.21.15.40:0/3733755995 >> v2:172.21.15.184:3300/0 conn(0x7f47380b2850 0x7f47380b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.201+0000 7f47367fc640 1 --2- 172.21.15.40:0/3733755995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47380d06f0 0x7f47380d1e80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f47380c89f0 tx=0x7f472c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc4c2ebfea0d71c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.201+0000 7f4735ffb640 1 -- 172.21.15.40:0/3733755995 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c037070 con 0x7f47380d06f0 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.201+0000 7f4735ffb640 1 -- 172.21.15.40:0/3733755995 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472c032440 con 0x7f47380d06f0 2024-04-26T21:37:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.201+0000 7f4735ffb640 1 -- 172.21.15.40:0/3733755995 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c032810 con 0x7f47380d06f0 2024-04-26T21:37:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.202+0000 7f473d6d1640 1 -- 172.21.15.40:0/3733755995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47380d06f0 msgr2=0x7f47380d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.202+0000 7f473d6d1640 1 --2- 172.21.15.40:0/3733755995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47380d06f0 0x7f47380d1e80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f47380c89f0 tx=0x7f472c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.202+0000 7f473d6d1640 1 -- 172.21.15.40:0/3733755995 shutdown_connections 2024-04-26T21:37:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.202+0000 7f473d6d1640 1 --2- 172.21.15.40:0/3733755995 >> v2:172.21.15.184:3300/0 conn(0x7f47380b2850 0x7f47380b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.202+0000 7f473d6d1640 1 --2- 172.21.15.40:0/3733755995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47380d06f0 0x7f47380d1e80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.202+0000 7f473d6d1640 1 -- 172.21.15.40:0/3733755995 >> 172.21.15.40:0/3733755995 conn(0x7f47380d4f90 msgr2=0x7f47380d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.202+0000 7f473d6d1640 1 -- 172.21.15.40:0/3733755995 shutdown_connections 2024-04-26T21:37:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.202+0000 7f473d6d1640 1 -- 172.21.15.40:0/3733755995 wait complete. 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.203+0000 7f473d6d1640 1 Processor -- start 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.203+0000 7f473d6d1640 1 -- start start 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f473d6d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47380b2850 0x7f4738159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f473d6d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47380d06f0 0x7f47380b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f473d6d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473815f3a0 0x7f47380b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f473d6d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f473815f900 con 0x7f473815f3a0 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f473d6d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f473815f780 con 0x7f47380b2850 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f473d6d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47380d23c0 con 0x7f47380d06f0 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473815f3a0 0x7f47380b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473815f3a0 0x7f47380b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35318/0 (socket says 172.21.15.40:35318) 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 -- 172.21.15.40:0/2814008074 learned_addr learned my addr 172.21.15.40:0/2814008074 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f4736ffd640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47380b2850 0x7f4738159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47367fc640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47380d06f0 0x7f47380b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:26.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 -- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47380d06f0 msgr2=0x7f47380b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47380d06f0 0x7f47380b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 -- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47380b2850 msgr2=0x7f4738159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47380b2850 0x7f4738159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 -- 172.21.15.40:0/2814008074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4738159990 con 0x7f473815f3a0 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.204+0000 7f47377fe640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473815f3a0 0x7f47380b4c60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f473000cf20 tx=0x7f473000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.205+0000 7f471f7fe640 1 -- 172.21.15.40:0/2814008074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473001a070 con 0x7f473815f3a0 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.205+0000 7f471f7fe640 1 -- 172.21.15.40:0/2814008074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f473000d840 con 0x7f473815f3a0 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.205+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4738161a80 con 0x7f473815f3a0 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.205+0000 7f471f7fe640 1 -- 172.21.15.40:0/2814008074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473000dc30 con 0x7f473815f3a0 2024-04-26T21:37:26.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.205+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4738162040 con 0x7f473815f3a0 2024-04-26T21:37:26.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.206+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47380694f0 con 0x7f473815f3a0 2024-04-26T21:37:26.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.206+0000 7f471f7fe640 1 -- 172.21.15.40:0/2814008074 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f473001d070 con 0x7f473815f3a0 2024-04-26T21:37:26.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.207+0000 7f471f7fe640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472406c8f0 0x7f472406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:26.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.207+0000 7f471f7fe640 1 -- 172.21.15.40:0/2814008074 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6302+0+0 (secure 0 0 0) 0x7f473008c5b0 con 0x7f473815f3a0 2024-04-26T21:37:26.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.207+0000 7f4736ffd640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472406c8f0 0x7f472406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:26.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.207+0000 7f4736ffd640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472406c8f0 0x7f472406edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4728000f60 tx=0x7f4728007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:26.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.211+0000 7f471f7fe640 1 -- 172.21.15.40:0/2814008074 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f473005a820 con 0x7f473815f3a0 2024-04-26T21:37:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:26.436+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f47380b6860 con 0x7f473815f3a0 2024-04-26T21:37:27.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.081+0000 7f471f7fe640 1 -- 172.21.15.40:0/2814008074 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f473005a120 con 0x7f473815f3a0 2024-04-26T21:37:27.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472406c8f0 msgr2=0x7f472406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:27.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472406c8f0 0x7f472406edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4728000f60 tx=0x7f4728007c00 comp rx=0 tx=0).stop 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473815f3a0 msgr2=0x7f47380b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473815f3a0 0x7f47380b4c60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f473000cf20 tx=0x7f473000a7d0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 shutdown_connections 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472406c8f0 0x7f472406edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473815f3a0 0x7f47380b4c60 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47380d06f0 0x7f47380b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 --2- 172.21.15.40:0/2814008074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47380b2850 0x7f4738159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 >> 172.21.15.40:0/2814008074 conn(0x7f47380d4f90 msgr2=0x7f473806cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 shutdown_connections 2024-04-26T21:37:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.084+0000 7f473d6d1640 1 -- 172.21.15.40:0/2814008074 wait complete. 2024-04-26T21:37:27.099 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:37:27.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.251+0000 7f214bde7640 1 Processor -- start 2024-04-26T21:37:27.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.251+0000 7f214bde7640 1 -- start start 2024-04-26T21:37:27.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f214bde7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21440b5fd0 0x7f21440b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f214bde7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21440d1950 0x7f21440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f214bde7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2144162d80 con 0x7f21441629a0 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f214bde7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f21440d5f30 con 0x7f21440b5fd0 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f214bde7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f21440d39d0 con 0x7f21440d1950 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f2149382640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21440b5fd0 0x7f21440b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f2149b83640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f21441629a0 0x7f21440d5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42578/0 (socket says 172.21.15.40:42578) 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f2149b83640 1 -- 172.21.15.40:0/651710008 learned_addr learned my addr 172.21.15.40:0/651710008 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f2149382640 1 -- 172.21.15.40:0/651710008 >> v2:172.21.15.184:3300/0 conn(0x7f21440d1950 msgr2=0x7f21440d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f2149382640 1 --2- 172.21.15.40:0/651710008 >> v2:172.21.15.184:3300/0 conn(0x7f21440d1950 0x7f21440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f2149382640 1 -- 172.21.15.40:0/651710008 >> v1:172.21.15.40:6789/0 conn(0x7f21441629a0 legacy=0x7f21440d5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f2148b81640 1 -- 172.21.15.40:0/651710008 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2818319077 0 0) 0x7f2144162d80 con 0x7f21441629a0 2024-04-26T21:37:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.252+0000 7f2149382640 1 -- 172.21.15.40:0/651710008 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21440d3cb0 con 0x7f21440b5fd0 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.253+0000 7f2149382640 1 --2- 172.21.15.40:0/651710008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440b4890 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f214408f040 tx=0x7f213802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc0b6c38d3c96cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.253+0000 7f2148b81640 1 -- 172.21.15.40:0/651710008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138037070 con 0x7f21440b5fd0 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.253+0000 7f2148b81640 1 -- 172.21.15.40:0/651710008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2138032440 con 0x7f21440b5fd0 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.253+0000 7f2148b81640 1 -- 172.21.15.40:0/651710008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138032810 con 0x7f21440b5fd0 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.254+0000 7f214bde7640 1 -- 172.21.15.40:0/651710008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 msgr2=0x7f21440b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.254+0000 7f214bde7640 1 --2- 172.21.15.40:0/651710008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440b4890 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f214408f040 tx=0x7f213802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.254+0000 7f214bde7640 1 -- 172.21.15.40:0/651710008 shutdown_connections 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.254+0000 7f214bde7640 1 --2- 172.21.15.40:0/651710008 >> v2:172.21.15.184:3300/0 conn(0x7f21440d1950 0x7f21440d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.254+0000 7f214bde7640 1 --2- 172.21.15.40:0/651710008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440b4890 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.254+0000 7f214bde7640 1 -- 172.21.15.40:0/651710008 >> 172.21.15.40:0/651710008 conn(0x7f214409e3e0 msgr2=0x7f21440d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.254+0000 7f214bde7640 1 -- 172.21.15.40:0/651710008 shutdown_connections 2024-04-26T21:37:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.254+0000 7f214bde7640 1 -- 172.21.15.40:0/651710008 wait complete. 2024-04-26T21:37:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.255+0000 7f214bde7640 1 Processor -- start 2024-04-26T21:37:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.255+0000 7f214bde7640 1 -- start start 2024-04-26T21:37:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f214bde7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f214bde7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21440d1950 0x7f21440d0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f214bde7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21441629a0 0x7f21440caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f214bde7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21440d3f90 con 0x7f21440b5fd0 2024-04-26T21:37:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f214bde7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21440d36a0 con 0x7f21441629a0 2024-04-26T21:37:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f214bde7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21440d02c0 con 0x7f21440d1950 2024-04-26T21:37:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35338/0 (socket says 172.21.15.40:35338) 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 -- 172.21.15.40:0/2182948186 learned_addr learned my addr 172.21.15.40:0/2182948186 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 -- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21440d1950 msgr2=0x7f21440d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21440d1950 0x7f21440d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 -- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21441629a0 msgr2=0x7f21440caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21441629a0 0x7f21440caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 -- 172.21.15.40:0/2182948186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21440ca1c0 con 0x7f21440b5fd0 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.256+0000 7f2149b83640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440c9c80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c460 tx=0x7f213c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.257+0000 7f21427fc640 1 -- 172.21.15.40:0/2182948186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c018070 con 0x7f21440b5fd0 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.257+0000 7f21427fc640 1 -- 172.21.15.40:0/2182948186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213c007500 con 0x7f21440b5fd0 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.257+0000 7f21427fc640 1 -- 172.21.15.40:0/2182948186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c004a20 con 0x7f21440b5fd0 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.257+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21440c8be0 con 0x7f21440b5fd0 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.257+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21440c91a0 con 0x7f21440b5fd0 2024-04-26T21:37:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.258+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2144069600 con 0x7f21440b5fd0 2024-04-26T21:37:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.258+0000 7f21427fc640 1 -- 172.21.15.40:0/2182948186 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f213c01b070 con 0x7f21440b5fd0 2024-04-26T21:37:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.259+0000 7f21427fc640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f213006c8f0 0x7f213006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.259+0000 7f21427fc640 1 -- 172.21.15.40:0/2182948186 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6302+0+0 (secure 0 0 0) 0x7f213c08b000 con 0x7f21440b5fd0 2024-04-26T21:37:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.259+0000 7f2149382640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f213006c8f0 0x7f213006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.259+0000 7f2149382640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f213006c8f0 0x7f213006edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2138034000 tx=0x7f213802c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:27.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.263+0000 7f21427fc640 1 -- 172.21.15.40:0/2182948186 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f213c0591e0 con 0x7f21440b5fd0 2024-04-26T21:37:27.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:27.450+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f21440b69e0 con 0x7f21440b5fd0 2024-04-26T21:37:28.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.083+0000 7f21427fc640 1 -- 172.21.15.40:0/2182948186 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7f213c058ae0 con 0x7f21440b5fd0 2024-04-26T21:37:28.083 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:37:28.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f213006c8f0 msgr2=0x7f213006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:28.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f213006c8f0 0x7f213006edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2138034000 tx=0x7f213802c000 comp rx=0 tx=0).stop 2024-04-26T21:37:28.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 msgr2=0x7f21440c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:28.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440c9c80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c460 tx=0x7f213c00c920 comp rx=0 tx=0).stop 2024-04-26T21:37:28.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 shutdown_connections 2024-04-26T21:37:28.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f213006c8f0 0x7f213006edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21441629a0 0x7f21440caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21440d1950 0x7f21440d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 --2- 172.21.15.40:0/2182948186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21440b5fd0 0x7f21440c9c80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.086+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 >> 172.21.15.40:0/2182948186 conn(0x7f214409e3e0 msgr2=0x7f21440b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.087+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 shutdown_connections 2024-04-26T21:37:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.087+0000 7f214bde7640 1 -- 172.21.15.40:0/2182948186 wait complete. 2024-04-26T21:37:28.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:37:28.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.215+0000 7f72d8664640 1 Processor -- start 2024-04-26T21:37:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.215+0000 7f72d8664640 1 -- start start 2024-04-26T21:37:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.216+0000 7f72d8664640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d01489a0 0x7f72d00710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.216+0000 7f72d8664640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72d00b04b0 0x7f72d00b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.216+0000 7f72d8664640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d0148d80 con 0x7f72d006c240 2024-04-26T21:37:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.216+0000 7f72d8664640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f72d00b11d0 con 0x7f72d00b04b0 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.216+0000 7f72d8664640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72d00b0f10 con 0x7f72d01489a0 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d6c01640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f72d006c240 0x7f72d00ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42594/0 (socket says 172.21.15.40:42594) 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d6400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d01489a0 0x7f72d00710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d5bff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72d00b04b0 0x7f72d00b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d6c01640 1 -- 172.21.15.40:0/367272383 learned_addr learned my addr 172.21.15.40:0/367272383 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d53fe640 1 -- 172.21.15.40:0/367272383 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87531733 0 0) 0x7f72d0148d80 con 0x7f72d006c240 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d53fe640 1 -- 172.21.15.40:0/367272383 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72b8003520 con 0x7f72d006c240 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d5bff640 1 -- 172.21.15.40:0/367272383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d01489a0 msgr2=0x7f72d00710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d5bff640 1 --2- 172.21.15.40:0/367272383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d01489a0 0x7f72d00710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d5bff640 1 -- 172.21.15.40:0/367272383 >> v1:172.21.15.40:6789/0 conn(0x7f72d006c240 legacy=0x7f72d00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d5bff640 1 -- 172.21.15.40:0/367272383 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d0070600 con 0x7f72d00b04b0 2024-04-26T21:37:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.217+0000 7f72d6400640 1 --2- 172.21.15.40:0/367272383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d01489a0 0x7f72d00710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.218+0000 7f72d5bff640 1 --2- 172.21.15.40:0/367272383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d00b04b0 0x7f72d00b08f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72c40097a0 tx=0x7f72c402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ad3570746a1105f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.218+0000 7f72d53fe640 1 -- 172.21.15.40:0/367272383 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4037070 con 0x7f72d00b04b0 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.218+0000 7f72d53fe640 1 -- 172.21.15.40:0/367272383 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c4032440 con 0x7f72d00b04b0 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.218+0000 7f72d53fe640 1 -- 172.21.15.40:0/367272383 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c40327f0 con 0x7f72d00b04b0 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.219+0000 7f72d8664640 1 -- 172.21.15.40:0/367272383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d00b04b0 msgr2=0x7f72d00b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.219+0000 7f72d8664640 1 --2- 172.21.15.40:0/367272383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d00b04b0 0x7f72d00b08f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72c40097a0 tx=0x7f72c402a180 comp rx=0 tx=0).stop 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.219+0000 7f72d8664640 1 -- 172.21.15.40:0/367272383 shutdown_connections 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.219+0000 7f72d8664640 1 --2- 172.21.15.40:0/367272383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d00b04b0 0x7f72d00b08f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.219+0000 7f72d8664640 1 --2- 172.21.15.40:0/367272383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d01489a0 0x7f72d00710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.219+0000 7f72d8664640 1 -- 172.21.15.40:0/367272383 >> 172.21.15.40:0/367272383 conn(0x7f72d00b3be0 msgr2=0x7f72d00b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.219+0000 7f72d8664640 1 -- 172.21.15.40:0/367272383 shutdown_connections 2024-04-26T21:37:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.219+0000 7f72d8664640 1 -- 172.21.15.40:0/367272383 wait complete. 2024-04-26T21:37:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.220+0000 7f72d8664640 1 Processor -- start 2024-04-26T21:37:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.220+0000 7f72d8664640 1 -- start start 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d8664640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d006c240 0x7f72d01428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d8664640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d00b04b0 0x7f72d00b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d8664640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d01489a0 0x7f72d015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d8664640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0071620 con 0x7f72d006c240 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d8664640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0070bc0 con 0x7f72d01489a0 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d8664640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72d01604d0 con 0x7f72d00b04b0 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d006c240 0x7f72d01428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d006c240 0x7f72d01428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35342/0 (socket says 172.21.15.40:35342) 2024-04-26T21:37:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 -- 172.21.15.40:0/3150752481 learned_addr learned my addr 172.21.15.40:0/3150752481 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d5bff640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d00b04b0 0x7f72d00b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6c01640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d01489a0 0x7f72d015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 -- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d00b04b0 msgr2=0x7f72d00b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d00b04b0 0x7f72d00b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 -- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d01489a0 msgr2=0x7f72d015e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d01489a0 0x7f72d015e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 -- 172.21.15.40:0/3150752481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d00d01c0 con 0x7f72d006c240 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.221+0000 7f72d6400640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d006c240 0x7f72d01428d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009ec0 tx=0x7f72c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.222+0000 7f72ceffd640 1 -- 172.21.15.40:0/3150752481 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8018020 con 0x7f72d006c240 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.222+0000 7f72ceffd640 1 -- 172.21.15.40:0/3150752481 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c8014680 con 0x7f72d006c240 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.222+0000 7f72ceffd640 1 -- 172.21.15.40:0/3150752481 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8014a50 con 0x7f72d006c240 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.222+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d00b77d0 con 0x7f72d006c240 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.222+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d00d0430 con 0x7f72d006c240 2024-04-26T21:37:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.223+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d0142ed0 con 0x7f72d006c240 2024-04-26T21:37:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.223+0000 7f72ceffd640 1 -- 172.21.15.40:0/3150752481 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72c8005d10 con 0x7f72d006c240 2024-04-26T21:37:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.224+0000 7f72ceffd640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.224+0000 7f72ceffd640 1 -- 172.21.15.40:0/3150752481 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5864+0+0 (secure 0 0 0) 0x7f72c808c920 con 0x7f72d006c240 2024-04-26T21:37:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.224+0000 7f72d5bff640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.224+0000 7f72d5bff640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f72c4034000 tx=0x7f72c402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.228+0000 7f72ceffd640 1 -- 172.21.15.40:0/3150752481 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72c8022030 con 0x7f72d006c240 2024-04-26T21:37:28.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:28.416+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f72d00b49e0 con 0x7f72d006c240 2024-04-26T21:37:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.084+0000 7f72ceffd640 1 -- 172.21.15.40:0/3150752481 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7f72c8010040 con 0x7f72d006c240 2024-04-26T21:37:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:37:29.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 msgr2=0x7f72bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f72c4034000 tx=0x7f72c402e000 comp rx=0 tx=0).stop 2024-04-26T21:37:29.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d006c240 msgr2=0x7f72d01428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d006c240 0x7f72d01428d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009ec0 tx=0x7f72c800c490 comp rx=0 tx=0).stop 2024-04-26T21:37:29.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 shutdown_connections 2024-04-26T21:37:29.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72bc06c8f0 0x7f72bc06edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d01489a0 0x7f72d015e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72d00b04b0 0x7f72d00b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 --2- 172.21.15.40:0/3150752481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d006c240 0x7f72d01428d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 >> 172.21.15.40:0/3150752481 conn(0x7f72d00b3be0 msgr2=0x7f72d00afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:29.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 shutdown_connections 2024-04-26T21:37:29.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.088+0000 7f72d8664640 1 -- 172.21.15.40:0/3150752481 wait complete. 2024-04-26T21:37:29.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:37:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.212+0000 7f1b4de20640 1 Processor -- start 2024-04-26T21:37:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.212+0000 7f1b4de20640 1 -- start start 2024-04-26T21:37:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b4de20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b5f20 0x7f1b480d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b4de20640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b480b7e50 con 0x7f1b480d3bc0 2024-04-26T21:37:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b4de20640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b480d1800 con 0x7f1b48162880 2024-04-26T21:37:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b4de20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b480d0580 con 0x7f1b480b5f20 2024-04-26T21:37:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b46ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1b480d3bc0 0x7f1b480b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50742/0 (socket says 172.21.15.40:50742) 2024-04-26T21:37:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b46ffd640 1 -- 172.21.15.40:0/1741662523 learned_addr learned my addr 172.21.15.40:0/1741662523 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b47fff640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b5f20 0x7f1b480d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279893082 0 0) 0x7f1b480b7e50 con 0x7f1b480d3bc0 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.213+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b30003520 con 0x7f1b480d3bc0 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3662055853 0 0) 0x7f1b38001110 con 0x7f1b48162880 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b480b7e50 con 0x7f1b48162880 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3651163110 0 0) 0x7f1b30003520 con 0x7f1b480d3bc0 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b5f20 msgr2=0x7f1b480d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b5f20 0x7f1b480d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 >> v1:172.21.15.184:6789/0 conn(0x7f1b48162880 legacy=0x7f1b480d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b480d07d0 con 0x7f1b480d3bc0 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1b3c001670 con 0x7f1b480d3bc0 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 >> v1:172.21.15.40:6789/0 conn(0x7f1b480d3bc0 legacy=0x7f1b480b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b30003b70 0x7f1b30003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b30004800 0x7f1b30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b300051c0 0x7f1b3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1b480d0580 con 0x7f1b300051c0 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3000c010 con 0x7f1b30003b70 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3000c210 con 0x7f1b30004800 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b47fff640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b5f20 0x7f1b480d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b477fe640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b30003b70 0x7f1b30003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b47fff640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b300051c0 0x7f1b3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b46ffd640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b30004800 0x7f1b30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b47fff640 1 -- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b30004800 msgr2=0x7f1b30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b47fff640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b30004800 0x7f1b30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b47fff640 1 -- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b30003b70 msgr2=0x7f1b30003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b47fff640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b30003b70 0x7f1b30003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.214+0000 7f1b47fff640 1 -- 172.21.15.40:0/1741662523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b30003520 con 0x7f1b300051c0 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.215+0000 7f1b47fff640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b300051c0 0x7f1b3000b9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b480b6300 tx=0x7f1b30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dcff017c62e8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.215+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40036070 con 0x7f1b300051c0 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.215+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b40003c50 con 0x7f1b300051c0 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.215+0000 7f1b467fc640 1 -- 172.21.15.40:0/1741662523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40009d40 con 0x7f1b300051c0 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.216+0000 7f1b4de20640 1 -- 172.21.15.40:0/1741662523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b300051c0 msgr2=0x7f1b3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.216+0000 7f1b4de20640 1 --2- 172.21.15.40:0/1741662523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b300051c0 0x7f1b3000b9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b480b6300 tx=0x7f1b30005830 comp rx=0 tx=0).stop 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.216+0000 7f1b47fff640 1 -- 172.21.15.40:0/1741662523 reap_dead start 2024-04-26T21:37:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.216+0000 7f1b4de20640 1 -- 172.21.15.40:0/1741662523 shutdown_connections 2024-04-26T21:37:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.216+0000 7f1b4de20640 1 -- 172.21.15.40:0/1741662523 >> 172.21.15.40:0/1741662523 conn(0x7f1b480d4f60 msgr2=0x7f1b480d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.216+0000 7f1b4de20640 1 -- 172.21.15.40:0/1741662523 shutdown_connections 2024-04-26T21:37:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.216+0000 7f1b4de20640 1 -- 172.21.15.40:0/1741662523 wait complete. 2024-04-26T21:37:29.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.217+0000 7f1b4de20640 1 Processor -- start 2024-04-26T21:37:29.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.217+0000 7f1b4de20640 1 -- start start 2024-04-26T21:37:29.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b4de20640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b48153190 0x7f1b480b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b4de20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48153570 0x7f1b481f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b4de20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b3250 0x7f1b480b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b4de20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48162c60 con 0x7f1b48153570 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b4de20640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b480d0d90 con 0x7f1b48153190 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b4de20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b48162de0 con 0x7f1b480b3250 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b46ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48153570 0x7f1b481f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b46ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48153570 0x7f1b481f8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47214/0 (socket says 172.21.15.40:47214) 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b46ffd640 1 -- 172.21.15.40:0/1474065451 learned_addr learned my addr 172.21.15.40:0/1474065451 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b47fff640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b3250 0x7f1b480b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b477fe640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b48153190 0x7f1b480b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b46ffd640 1 -- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b3250 msgr2=0x7f1b480b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b46ffd640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b3250 0x7f1b480b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b46ffd640 1 -- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b48153190 msgr2=0x7f1b480b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b46ffd640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b48153190 0x7f1b480b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.218+0000 7f1b46ffd640 1 -- 172.21.15.40:0/1474065451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4815c740 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.219+0000 7f1b46ffd640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48153570 0x7f1b481f8f80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c004840 tx=0x7f1b3c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.219+0000 7f1b2ffff640 1 -- 172.21.15.40:0/1474065451 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00f940 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.219+0000 7f1b2ffff640 1 -- 172.21.15.40:0/1474065451 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3c018070 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.219+0000 7f1b2ffff640 1 -- 172.21.15.40:0/1474065451 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00fd60 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.219+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b480b37c0 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.219+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b480b3d80 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.220+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b480694f0 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.220+0000 7f1b2ffff640 1 -- 172.21.15.40:0/1474065451 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b3c01d460 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.221+0000 7f1b2ffff640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b3406c920 0x7f1b3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.221+0000 7f1b2ffff640 1 -- 172.21.15.40:0/1474065451 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f1b3c08bb50 con 0x7f1b48153570 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.221+0000 7f1b477fe640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b3406c920 0x7f1b3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.222+0000 7f1b477fe640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b3406c920 0x7f1b3406ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1b380012b0 tx=0x7f1b38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:29.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.225+0000 7f1b2ffff640 1 -- 172.21.15.40:0/1474065451 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b4815c740 con 0x7f1b48153570 2024-04-26T21:37:29.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.415+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1b481f88c0 con 0x7f1b48153570 2024-04-26T21:37:29.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.418+0000 7f1b2ffff640 1 -- 172.21.15.40:0/1474065451 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f1b3c05a000 con 0x7f1b48153570 2024-04-26T21:37:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.419+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b3406c920 msgr2=0x7f1b3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.419+0000 7f1b4de20640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b3406c920 0x7f1b3406ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1b380012b0 tx=0x7f1b38009040 comp rx=0 tx=0).stop 2024-04-26T21:37:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48153570 msgr2=0x7f1b481f8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48153570 0x7f1b481f8f80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c004840 tx=0x7f1b3c007720 comp rx=0 tx=0).stop 2024-04-26T21:37:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 shutdown_connections 2024-04-26T21:37:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b3406c920 0x7f1b3406ede0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b480b3250 0x7f1b480b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48153570 0x7f1b481f8f80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 --2- 172.21.15.40:0/1474065451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b48153190 0x7f1b480b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 >> 172.21.15.40:0/1474065451 conn(0x7f1b480d4f60 msgr2=0x7f1b48160a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 shutdown_connections 2024-04-26T21:37:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.420+0000 7f1b4de20640 1 -- 172.21.15.40:0/1474065451 wait complete. 2024-04-26T21:37:29.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-04-26T21:37:29.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.531+0000 7fd218faf640 1 Processor -- start 2024-04-26T21:37:29.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.531+0000 7fd218faf640 1 -- start start 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd218faf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd21415f3a0 0x7fd2140d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd218faf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd2140d1770 0x7fd2140d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd218faf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d2160 0x7fd2140d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd218faf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd2140d25d0 con 0x7fd2140d1770 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd218faf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd2140d0e20 con 0x7fd21415f3a0 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd218faf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2140b1d70 con 0x7fd2140d2160 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd211d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd2140d1770 0x7fd2140d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd211d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd2140d1770 0x7fd2140d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47224/0 (socket says 172.21.15.40:47224) 2024-04-26T21:37:29.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.532+0000 7fd211d9b640 1 -- 172.21.15.40:0/1075477 learned_addr learned my addr 172.21.15.40:0/1075477 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:29.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.533+0000 7fd211d9b640 1 -- 172.21.15.40:0/1075477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d2160 msgr2=0x7fd2140d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:29.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.533+0000 7fd211d9b640 1 --2- 172.21.15.40:0/1075477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d2160 0x7fd2140d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.533+0000 7fd21259c640 1 --2- 172.21.15.40:0/1075477 >> v2:172.21.15.184:3300/0 conn(0x7fd21415f3a0 0x7fd2140d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.533+0000 7fd211d9b640 1 -- 172.21.15.40:0/1075477 >> v2:172.21.15.184:3300/0 conn(0x7fd21415f3a0 msgr2=0x7fd2140d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:29.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.533+0000 7fd211d9b640 1 --2- 172.21.15.40:0/1075477 >> v2:172.21.15.184:3300/0 conn(0x7fd21415f3a0 0x7fd2140d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.533+0000 7fd211d9b640 1 -- 172.21.15.40:0/1075477 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2140b2050 con 0x7fd2140d1770 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.533+0000 7fd211d9b640 1 --2- 172.21.15.40:0/1075477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2140d1770 0x7fd2140d1c20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd214073b50 tx=0x7fd1fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5f30df6987e474 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.533+0000 7fd21159a640 1 -- 172.21.15.40:0/1075477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc037070 con 0x7fd2140d1770 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd21159a640 1 -- 172.21.15.40:0/1075477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc032440 con 0x7fd2140d1770 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd21159a640 1 -- 172.21.15.40:0/1075477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc032810 con 0x7fd2140d1770 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 -- 172.21.15.40:0/1075477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2140d1770 msgr2=0x7fd2140d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 --2- 172.21.15.40:0/1075477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2140d1770 0x7fd2140d1c20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd214073b50 tx=0x7fd1fc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 -- 172.21.15.40:0/1075477 shutdown_connections 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 --2- 172.21.15.40:0/1075477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d2160 0x7fd2140d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 --2- 172.21.15.40:0/1075477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2140d1770 0x7fd2140d1c20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 --2- 172.21.15.40:0/1075477 >> v2:172.21.15.184:3300/0 conn(0x7fd21415f3a0 0x7fd2140d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 -- 172.21.15.40:0/1075477 >> 172.21.15.40:0/1075477 conn(0x7fd2140d4f90 msgr2=0x7fd2140d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 -- 172.21.15.40:0/1075477 shutdown_connections 2024-04-26T21:37:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.534+0000 7fd218faf640 1 -- 172.21.15.40:0/1075477 wait complete. 2024-04-26T21:37:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.535+0000 7fd218faf640 1 Processor -- start 2024-04-26T21:37:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd218faf640 1 -- start start 2024-04-26T21:37:29.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd218faf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d1770 0x7fd21415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd218faf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2140d2160 0x7fd2141590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd218faf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd21415f3a0 0x7fd214159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd218faf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd21415f900 con 0x7fd21415f3a0 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd218faf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd21415f780 con 0x7fd2140d2160 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd218faf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2140d3390 con 0x7fd2140d1770 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd212d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd21415f3a0 0x7fd214159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd212d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd21415f3a0 0x7fd214159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47236/0 (socket says 172.21.15.40:47236) 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd212d9d640 1 -- 172.21.15.40:0/788280566 learned_addr learned my addr 172.21.15.40:0/788280566 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd21259c640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d1770 0x7fd21415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.536+0000 7fd211d9b640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2140d2160 0x7fd2141590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd212d9d640 1 -- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d1770 msgr2=0x7fd21415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd212d9d640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d1770 0x7fd21415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd212d9d640 1 -- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2140d2160 msgr2=0x7fd2141590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd212d9d640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2140d2160 0x7fd2141590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd212d9d640 1 -- 172.21.15.40:0/788280566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21415fbb0 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd212d9d640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd21415f3a0 0x7fd214159610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b450 tx=0x7fd20c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd20affd640 1 -- 172.21.15.40:0/788280566 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c015020 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd20affd640 1 -- 172.21.15.40:0/788280566 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c011890 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21415fe50 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd20affd640 1 -- 172.21.15.40:0/788280566 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c011c60 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.537+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd214160410 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.538+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2140694f0 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.539+0000 7fd20affd640 1 -- 172.21.15.40:0/788280566 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd20c018020 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.539+0000 7fd20affd640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1f406c8f0 0x7fd1f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.539+0000 7fd20affd640 1 -- 172.21.15.40:0/788280566 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fd20c08b9b0 con 0x7fd21415f3a0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.539+0000 7fd21259c640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1f406c8f0 0x7fd1f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.540+0000 7fd21259c640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1f406c8f0 0x7fd1f406edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd21415e640 tx=0x7fd204007710 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:29.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.543+0000 7fd20affd640 1 -- 172.21.15.40:0/788280566 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd20c01f020 con 0x7fd21415f3a0 2024-04-26T21:37:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:29.803+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7fd2140b6860 con 0x7fd21415f3a0 2024-04-26T21:37:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.098+0000 7fd20affd640 1 -- 172.21.15.40:0/788280566 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v145) ==== 134+0+0 (secure 0 0 0) 0x7fd20c00d040 con 0x7fd21415f3a0 2024-04-26T21:37:30.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1f406c8f0 msgr2=0x7fd1f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1f406c8f0 0x7fd1f406edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd21415e640 tx=0x7fd204007710 comp rx=0 tx=0).stop 2024-04-26T21:37:30.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd21415f3a0 msgr2=0x7fd214159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd21415f3a0 0x7fd214159610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b450 tx=0x7fd20c00b910 comp rx=0 tx=0).stop 2024-04-26T21:37:30.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 shutdown_connections 2024-04-26T21:37:30.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1f406c8f0 0x7fd1f406edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd21415f3a0 0x7fd214159610 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2140d2160 0x7fd2141590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 --2- 172.21.15.40:0/788280566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2140d1770 0x7fd21415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 >> 172.21.15.40:0/788280566 conn(0x7fd2140d4f90 msgr2=0x7fd21406dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:30.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 shutdown_connections 2024-04-26T21:37:30.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.101+0000 7fd218faf640 1 -- 172.21.15.40:0/788280566 wait complete. 2024-04-26T21:37:30.112 INFO:tasks.cephfs_test_runner:test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) ... ok 2024-04-26T21:37:30.113 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) That the MDS won't abort when the dentry is already known to be damaged. 2024-04-26T21:37:30.113 INFO:tasks.cephfs_test_runner:test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) 2024-04-26T21:37:30.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:30.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.215+0000 7f9865be9640 1 Processor -- start 2024-04-26T21:37:30.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.215+0000 7f9865be9640 1 -- start start 2024-04-26T21:37:30.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f9865be9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 0x7f98600d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f9865be9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98600b7e50 con 0x7f9860162880 2024-04-26T21:37:30.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f9865be9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98600d1800 con 0x7f98600b5f20 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f9865be9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98600d0580 con 0x7f98600d3bc0 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9860162880 0x7f98600d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50754/0 (socket says 172.21.15.40:50754) 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985f7fe640 1 -- 172.21.15.40:0/2732757223 learned_addr learned my addr 172.21.15.40:0/2732757223 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985effd640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 0x7f98600d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027309584 0 0) 0x7f98600b7e50 con 0x7f9860162880 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9848003520 con 0x7f9860162880 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 533263167 0 0) 0x7f9848003520 con 0x7f9860162880 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 msgr2=0x7f98600d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 0x7f98600d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 >> v1:172.21.15.184:6789/0 conn(0x7f98600b5f20 legacy=0x7f98600b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98600d07d0 con 0x7f9860162880 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9854001670 con 0x7f9860162880 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 >> v1:172.21.15.40:6789/0 conn(0x7f9860162880 legacy=0x7f98600d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98480039e0 0x7f9848003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9848004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 0x7f98480050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98600d0580 con 0x7f9848004390 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f984800b9c0 con 0x7f98480039e0 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f984800bbc0 con 0x7f98600d3bc0 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.216+0000 7f985effd640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9848004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985f7fe640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98480039e0 0x7f9848003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985ffff640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 0x7f98480050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985effd640 1 -- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 msgr2=0x7f98480050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985effd640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 0x7f98480050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985effd640 1 -- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98480039e0 msgr2=0x7f9848003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985effd640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98480039e0 0x7f9848003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985effd640 1 -- 172.21.15.40:0/2732757223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9848003520 con 0x7f9848004390 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985ffff640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98600d3bc0 0x7f98480050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:30.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.217+0000 7f985effd640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9848004860 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9850009ac0 tx=0x7f9848005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d831eb740a03e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850036070 con 0x7f9848004390 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9850031400 con 0x7f9848004390 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f985e7fc640 1 -- 172.21.15.40:0/2732757223 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850009690 con 0x7f9848004390 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f9865be9640 1 -- 172.21.15.40:0/2732757223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 msgr2=0x7f9848004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f9865be9640 1 --2- 172.21.15.40:0/2732757223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9848004860 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9850009ac0 tx=0x7f9848005830 comp rx=0 tx=0).stop 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f985ffff640 1 -- 172.21.15.40:0/2732757223 reap_dead start 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f9865be9640 1 -- 172.21.15.40:0/2732757223 shutdown_connections 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f9865be9640 1 -- 172.21.15.40:0/2732757223 >> 172.21.15.40:0/2732757223 conn(0x7f98600d4f60 msgr2=0x7f98600d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f9865be9640 1 -- 172.21.15.40:0/2732757223 shutdown_connections 2024-04-26T21:37:30.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.218+0000 7f9865be9640 1 -- 172.21.15.40:0/2732757223 wait complete. 2024-04-26T21:37:30.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.219+0000 7f9865be9640 1 Processor -- start 2024-04-26T21:37:30.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f9865be9640 1 -- start start 2024-04-26T21:37:30.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f9865be9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9860151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f9865be9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9860154470 0x7f986006ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f9865be9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986006d3a0 0x7f98600a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f9865be9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9860162c60 con 0x7f9848004390 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f9865be9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98600d0d90 con 0x7f9860154470 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f9865be9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9860162de0 con 0x7f986006d3a0 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f985f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9860151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f985f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9860151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47264/0 (socket says 172.21.15.40:47264) 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.220+0000 7f985f7fe640 1 -- 172.21.15.40:0/3600217718 learned_addr learned my addr 172.21.15.40:0/3600217718 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f985effd640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9860154470 0x7f986006ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f985f7fe640 1 -- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986006d3a0 msgr2=0x7f98600a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f985f7fe640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986006d3a0 0x7f98600a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f985f7fe640 1 -- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9860154470 msgr2=0x7f986006ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f985f7fe640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9860154470 0x7f986006ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f985f7fe640 1 -- 172.21.15.40:0/3600217718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9860153520 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f985effd640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9860154470 0x7f986006ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f985f7fe640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9860151330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9854001ba0 tx=0x7f9854007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f9847fff640 1 -- 172.21.15.40:0/3600217718 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985400f8e0 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f9847fff640 1 -- 172.21.15.40:0/3600217718 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9854018070 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9860153800 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f9847fff640 1 -- 172.21.15.40:0/3600217718 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985400fce0 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.221+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9860153dc0 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.222+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98600694f0 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.223+0000 7f9847fff640 1 -- 172.21.15.40:0/3600217718 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f985401d460 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.223+0000 7f9847fff640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f984c06c8f0 0x7f984c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.223+0000 7f9847fff640 1 -- 172.21.15.40:0/3600217718 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f985408bb80 con 0x7f9848004390 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.223+0000 7f985effd640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f984c06c8f0 0x7f984c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.224+0000 7f985effd640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f984c06c8f0 0x7f984c06edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9850009ac0 tx=0x7f9850008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.228+0000 7f9847fff640 1 -- 172.21.15.40:0/3600217718 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f985405a0d0 con 0x7f9848004390 2024-04-26T21:37:30.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.414+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f986006d7c0 con 0x7f9848004390 2024-04-26T21:37:30.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.414+0000 7f9847fff640 1 -- 172.21.15.40:0/3600217718 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13387 (secure 0 0 0) 0x7f98540599d0 con 0x7f9848004390 2024-04-26T21:37:30.414 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:30.415 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":44,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:29.083355+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/1366266306":"2024-04-27T21:37:26.078843+0000","172.21.15.40:6833/1366266306":"2024-04-27T21:37:26.078843+0000","172.21.15.40:6835/3403660773":"2024-04-27T21:36:38.666240+0000","172.21.15.40:6833/3403660773":"2024-04-27T21:36:38.666240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f984c06c8f0 msgr2=0x7f984c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f984c06c8f0 0x7f984c06edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9850009ac0 tx=0x7f9850008030 comp rx=0 tx=0).stop 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 msgr2=0x7f9860151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9860151330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9854001ba0 tx=0x7f9854007720 comp rx=0 tx=0).stop 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 shutdown_connections 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f984c06c8f0 0x7f984c06edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986006d3a0 0x7f98600a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9860154470 0x7f986006ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 --2- 172.21.15.40:0/3600217718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9848004390 0x7f9860151330 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.416+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 >> 172.21.15.40:0/3600217718 conn(0x7f98600d4f60 msgr2=0x7f98601604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:30.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.417+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 shutdown_connections 2024-04-26T21:37:30.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.417+0000 7f9865be9640 1 -- 172.21.15.40:0/3600217718 wait complete. 2024-04-26T21:37:30.428 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:37:30.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:37:30.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.533+0000 7f136bd12640 1 Processor -- start 2024-04-26T21:37:30.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.533+0000 7f136bd12640 1 -- start start 2024-04-26T21:37:30.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f136bd12640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f136415f550 0x7f13640d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f136bd12640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13640d1040 con 0x7f136415f170 2024-04-26T21:37:30.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f136bd12640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13640c9680 con 0x7f13640d3bc0 2024-04-26T21:37:30.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f136bd12640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f13640c9430 con 0x7f136415f550 2024-04-26T21:37:30.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f13692ad640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f136415f550 0x7f13640d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f1369aae640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f136415f170 0x7f13640d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50758/0 (socket says 172.21.15.40:50758) 2024-04-26T21:37:30.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f1369aae640 1 -- 172.21.15.40:0/797352580 learned_addr learned my addr 172.21.15.40:0/797352580 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:30.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f13692ad640 1 -- 172.21.15.40:0/797352580 >> v1:172.21.15.184:6789/0 conn(0x7f13640d3bc0 legacy=0x7f13640ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f13692ad640 1 -- 172.21.15.40:0/797352580 >> v1:172.21.15.40:6789/0 conn(0x7f136415f170 legacy=0x7f13640d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.534+0000 7f13692ad640 1 -- 172.21.15.40:0/797352580 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640c9960 con 0x7f136415f550 2024-04-26T21:37:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f13692ad640 1 --2- 172.21.15.40:0/797352580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f136415f550 0x7f13640d5a80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f13640c95b0 tx=0x7f135402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6501aa618cacd489 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f1368aac640 1 -- 172.21.15.40:0/797352580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354037070 con 0x7f136415f550 2024-04-26T21:37:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f1368aac640 1 -- 172.21.15.40:0/797352580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1354032440 con 0x7f136415f550 2024-04-26T21:37:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f1368aac640 1 -- 172.21.15.40:0/797352580 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354032810 con 0x7f136415f550 2024-04-26T21:37:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f136bd12640 1 -- 172.21.15.40:0/797352580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f136415f550 msgr2=0x7f13640d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f136bd12640 1 --2- 172.21.15.40:0/797352580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f136415f550 0x7f13640d5a80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f13640c95b0 tx=0x7f135402a250 comp rx=0 tx=0).stop 2024-04-26T21:37:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f136bd12640 1 -- 172.21.15.40:0/797352580 shutdown_connections 2024-04-26T21:37:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f136bd12640 1 --2- 172.21.15.40:0/797352580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f136415f550 0x7f13640d5a80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.535+0000 7f136bd12640 1 -- 172.21.15.40:0/797352580 >> 172.21.15.40:0/797352580 conn(0x7f13640d4f60 msgr2=0x7f13640d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.536+0000 7f136bd12640 1 -- 172.21.15.40:0/797352580 shutdown_connections 2024-04-26T21:37:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.536+0000 7f136bd12640 1 -- 172.21.15.40:0/797352580 wait complete. 2024-04-26T21:37:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.537+0000 7f136bd12640 1 Processor -- start 2024-04-26T21:37:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.537+0000 7f136bd12640 1 -- start start 2024-04-26T21:37:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.537+0000 7f136bd12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13640d3bc0 0x7f13640ba760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.537+0000 7f136bd12640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415f170 0x7f136414b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.537+0000 7f136bd12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13641583c0 0x7f13641ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f136bd12640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13640d2e30 con 0x7f13640d3bc0 2024-04-26T21:37:30.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f136bd12640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13640d2cb0 con 0x7f136415f170 2024-04-26T21:37:30.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f136bd12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13640d2fb0 con 0x7f13641583c0 2024-04-26T21:37:30.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13640d3bc0 0x7f13640ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13640d3bc0 0x7f13640ba760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47290/0 (socket says 172.21.15.40:47290) 2024-04-26T21:37:30.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 -- 172.21.15.40:0/3441837255 learned_addr learned my addr 172.21.15.40:0/3441837255 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f13692ad640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415f170 0x7f136414b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f136a2af640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13641583c0 0x7f13641ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 -- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13641583c0 msgr2=0x7f13641ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13641583c0 0x7f13641ea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 -- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415f170 msgr2=0x7f136414b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415f170 0x7f136414b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 -- 172.21.15.40:0/3441837255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1364159b20 con 0x7f13640d3bc0 2024-04-26T21:37:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f13692ad640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415f170 0x7f136414b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f1369aae640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13640d3bc0 0x7f13640ba760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f136000c4a0 tx=0x7f136000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f135a7fc640 1 -- 172.21.15.40:0/3441837255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360017070 con 0x7f13640d3bc0 2024-04-26T21:37:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f135a7fc640 1 -- 172.21.15.40:0/3441837255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1360005ce0 con 0x7f13640d3bc0 2024-04-26T21:37:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f135a7fc640 1 -- 172.21.15.40:0/3441837255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360007270 con 0x7f13640d3bc0 2024-04-26T21:37:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136414b880 con 0x7f13640d3bc0 2024-04-26T21:37:30.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.538+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136415a380 con 0x7f13640d3bc0 2024-04-26T21:37:30.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.540+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13640694f0 con 0x7f13640d3bc0 2024-04-26T21:37:30.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.540+0000 7f135a7fc640 1 -- 172.21.15.40:0/3441837255 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f136001a070 con 0x7f13640d3bc0 2024-04-26T21:37:30.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.540+0000 7f135a7fc640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f134806c8f0 0x7f134806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.540+0000 7f135a7fc640 1 -- 172.21.15.40:0/3441837255 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f136008acf0 con 0x7f13640d3bc0 2024-04-26T21:37:30.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.541+0000 7f13692ad640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f134806c8f0 0x7f134806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.541+0000 7f13692ad640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f134806c8f0 0x7f134806edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1354034000 tx=0x7f135402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.545+0000 7f135a7fc640 1 -- 172.21.15.40:0/3441837255 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f136008f070 con 0x7f13640d3bc0 2024-04-26T21:37:30.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.729+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f136415ea90 con 0x7f13640d3bc0 2024-04-26T21:37:30.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.729+0000 7f135a7fc640 1 -- 172.21.15.40:0/3441837255 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) ==== 93+0+10 (secure 0 0 0) 0x7f13640d1040 con 0x7f13640d3bc0 2024-04-26T21:37:30.729 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:37:30.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f134806c8f0 msgr2=0x7f134806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f134806c8f0 0x7f134806edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1354034000 tx=0x7f135402e000 comp rx=0 tx=0).stop 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13640d3bc0 msgr2=0x7f13640ba760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13640d3bc0 0x7f13640ba760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f136000c4a0 tx=0x7f136000c960 comp rx=0 tx=0).stop 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 shutdown_connections 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f134806c8f0 0x7f134806edb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13641583c0 0x7f13641ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415f170 0x7f136414b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 --2- 172.21.15.40:0/3441837255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13640d3bc0 0x7f13640ba760 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 >> 172.21.15.40:0/3441837255 conn(0x7f13640d4f60 msgr2=0x7f136406e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 shutdown_connections 2024-04-26T21:37:30.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.731+0000 7f136bd12640 1 -- 172.21.15.40:0/3441837255 wait complete. 2024-04-26T21:37:30.742 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-04-26T21:37:30.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.843+0000 7f58b44a2640 1 Processor -- start 2024-04-26T21:37:30.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.843+0000 7f58b44a2640 1 -- start start 2024-04-26T21:37:30.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b44a2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58ac0cf5e0 0x7f58ac0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b44a2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58ac15b9e0 con 0x7f58ac0cbbd0 2024-04-26T21:37:30.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b44a2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58ac0d0740 con 0x7f58ac15b600 2024-04-26T21:37:30.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b44a2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f58ac0cb9b0 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b1a3d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58ac0cf5e0 0x7f58ac0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b2a3f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f58ac0cbbd0 0x7f58ac06f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50774/0 (socket says 172.21.15.40:50774) 2024-04-26T21:37:30.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b1a3d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58ac0cf5e0 0x7f58ac0ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47306/0 (socket says 172.21.15.40:47306) 2024-04-26T21:37:30.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b2a3f640 1 -- 172.21.15.40:0/1518654345 learned_addr learned my addr 172.21.15.40:0/1518654345 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:30.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b1a3d640 1 -- 172.21.15.40:0/1518654345 >> v1:172.21.15.184:6789/0 conn(0x7f58ac15b600 legacy=0x7f58ac071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:30.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b1a3d640 1 -- 172.21.15.40:0/1518654345 >> v1:172.21.15.40:6789/0 conn(0x7f58ac0cbbd0 legacy=0x7f58ac06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b123c640 1 -- 172.21.15.40:0/1518654345 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 992433554 0 0) 0x7f58ac15b9e0 con 0x7f58ac0cbbd0 2024-04-26T21:37:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.844+0000 7f58b1a3d640 1 -- 172.21.15.40:0/1518654345 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac0cb360 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.845+0000 7f58b1a3d640 1 --2- 172.21.15.40:0/1518654345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac0ce9d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f589c0097a0 tx=0x7f589c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e541bb6fe7edfa06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.845+0000 7f58b123c640 1 -- 172.21.15.40:0/1518654345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c037070 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.845+0000 7f58b123c640 1 -- 172.21.15.40:0/1518654345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589c032440 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.845+0000 7f58b123c640 1 -- 172.21.15.40:0/1518654345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c032810 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.846+0000 7f58b44a2640 1 -- 172.21.15.40:0/1518654345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 msgr2=0x7f58ac0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.846+0000 7f58b44a2640 1 --2- 172.21.15.40:0/1518654345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac0ce9d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f589c0097a0 tx=0x7f589c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.846+0000 7f58b44a2640 1 -- 172.21.15.40:0/1518654345 shutdown_connections 2024-04-26T21:37:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.846+0000 7f58b44a2640 1 --2- 172.21.15.40:0/1518654345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac0ce9d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.846+0000 7f58b44a2640 1 -- 172.21.15.40:0/1518654345 >> 172.21.15.40:0/1518654345 conn(0x7f58ac0d0be0 msgr2=0x7f58ac0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.846+0000 7f58b44a2640 1 -- 172.21.15.40:0/1518654345 shutdown_connections 2024-04-26T21:37:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.846+0000 7f58b44a2640 1 -- 172.21.15.40:0/1518654345 wait complete. 2024-04-26T21:37:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.847+0000 7f58b44a2640 1 Processor -- start 2024-04-26T21:37:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.847+0000 7f58b44a2640 1 -- start start 2024-04-26T21:37:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b44a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58ac0cbbd0 0x7f58ac1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b44a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b44a2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58ac15b600 0x7f58ac147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b44a2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac0d0470 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b44a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac0d02f0 con 0x7f58ac0cbbd0 2024-04-26T21:37:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b44a2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac0cef10 con 0x7f58ac15b600 2024-04-26T21:37:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47322/0 (socket says 172.21.15.40:47322) 2024-04-26T21:37:30.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 -- 172.21.15.40:0/167799303 learned_addr learned my addr 172.21.15.40:0/167799303 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:30.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 -- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58ac15b600 msgr2=0x7f58ac147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58ac15b600 0x7f58ac147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 -- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58ac0cbbd0 msgr2=0x7f58ac1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:30.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b223e640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58ac0cbbd0 0x7f58ac1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58ac0cbbd0 0x7f58ac1509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 -- 172.21.15.40:0/167799303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac0af980 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.848+0000 7f58b1a3d640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac071d60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f589c035040 tx=0x7f589c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.849+0000 7f58a2ffd640 1 -- 172.21.15.40:0/167799303 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c037050 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.849+0000 7f58a2ffd640 1 -- 172.21.15.40:0/167799303 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589c038040 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.849+0000 7f58a2ffd640 1 -- 172.21.15.40:0/167799303 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c004490 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.849+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac0afbf0 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.849+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac0722a0 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.850+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ac0694f0 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.850+0000 7f58a2ffd640 1 -- 172.21.15.40:0/167799303 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f589c030900 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.850+0000 7f58a2ffd640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f589006c8f0 0x7f589006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:30.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.851+0000 7f58a2ffd640 1 -- 172.21.15.40:0/167799303 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f589c0ac800 con 0x7f58ac0cf5e0 2024-04-26T21:37:30.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.851+0000 7f58b223e640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f589006c8f0 0x7f589006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:30.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.851+0000 7f58b223e640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f589006c8f0 0x7f589006edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f58a8000f60 tx=0x7f58a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:30.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:30.855+0000 7f58a2ffd640 1 -- 172.21.15.40:0/167799303 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f589c07ae70 con 0x7f58ac0cf5e0 2024-04-26T21:37:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:31.115+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7f58ac14d470 con 0x7f58ac0cf5e0 2024-04-26T21:37:32.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.106+0000 7f58a2ffd640 1 -- 172.21.15.40:0/167799303 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v147) ==== 144+0+0 (secure 0 0 0) 0x7f589c07a770 con 0x7f58ac0cf5e0 2024-04-26T21:37:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f589006c8f0 msgr2=0x7f589006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f589006c8f0 0x7f589006edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f58a8000f60 tx=0x7f58a8009040 comp rx=0 tx=0).stop 2024-04-26T21:37:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 msgr2=0x7f58ac071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac071d60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f589c035040 tx=0x7f589c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 shutdown_connections 2024-04-26T21:37:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f589006c8f0 0x7f589006edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58ac15b600 0x7f58ac147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac0cf5e0 0x7f58ac071d60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 --2- 172.21.15.40:0/167799303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58ac0cbbd0 0x7f58ac1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 >> 172.21.15.40:0/167799303 conn(0x7f58ac0d0be0 msgr2=0x7f58ac0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:32.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 shutdown_connections 2024-04-26T21:37:32.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.109+0000 7f58b44a2640 1 -- 172.21.15.40:0/167799303 wait complete. 2024-04-26T21:37:32.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:37:32.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.233+0000 7f643b285640 1 Processor -- start 2024-04-26T21:37:32.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.233+0000 7f643b285640 1 -- start start 2024-04-26T21:37:32.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f643b285640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f643406c950 0x7f6434160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f643b285640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6434070930 0x7f64340aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f643b285640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64340b0990 con 0x7f643415d100 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f643b285640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f64340af2d0 con 0x7f643406c950 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f643b285640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f64340b0c70 con 0x7f6434070930 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f6438820640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f643406c950 0x7f6434160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f6439021640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f643415d100 0x7f643406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50780/0 (socket says 172.21.15.40:50780) 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f6439021640 1 -- 172.21.15.40:0/3402784460 learned_addr learned my addr 172.21.15.40:0/3402784460 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f6439822640 1 --2- 172.21.15.40:0/3402784460 >> v2:172.21.15.184:3300/0 conn(0x7f6434070930 0x7f64340aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f6438820640 1 -- 172.21.15.40:0/3402784460 >> v2:172.21.15.184:3300/0 conn(0x7f6434070930 msgr2=0x7f64340aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f6438820640 1 --2- 172.21.15.40:0/3402784460 >> v2:172.21.15.184:3300/0 conn(0x7f6434070930 0x7f64340aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f6438820640 1 -- 172.21.15.40:0/3402784460 >> v1:172.21.15.40:6789/0 conn(0x7f643415d100 legacy=0x7f643406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f6438820640 1 -- 172.21.15.40:0/3402784460 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64340b0f50 con 0x7f643406c950 2024-04-26T21:37:32.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.234+0000 7f642bfff640 1 -- 172.21.15.40:0/3402784460 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1353550635 0 0) 0x7f64340b0990 con 0x7f643415d100 2024-04-26T21:37:32.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.235+0000 7f6438820640 1 --2- 172.21.15.40:0/3402784460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643406c950 0x7f6434160ba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6434070320 tx=0x7f642402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330cabec3da46f87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:32.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.235+0000 7f642bfff640 1 -- 172.21.15.40:0/3402784460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424037070 con 0x7f643406c950 2024-04-26T21:37:32.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.235+0000 7f642bfff640 1 -- 172.21.15.40:0/3402784460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6424032440 con 0x7f643406c950 2024-04-26T21:37:32.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.235+0000 7f642bfff640 1 -- 172.21.15.40:0/3402784460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424032810 con 0x7f643406c950 2024-04-26T21:37:32.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.236+0000 7f643b285640 1 -- 172.21.15.40:0/3402784460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643406c950 msgr2=0x7f6434160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.236+0000 7f643b285640 1 --2- 172.21.15.40:0/3402784460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643406c950 0x7f6434160ba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6434070320 tx=0x7f642402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.236+0000 7f643b285640 1 -- 172.21.15.40:0/3402784460 shutdown_connections 2024-04-26T21:37:32.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.236+0000 7f643b285640 1 --2- 172.21.15.40:0/3402784460 >> v2:172.21.15.184:3300/0 conn(0x7f6434070930 0x7f64340aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.236+0000 7f643b285640 1 --2- 172.21.15.40:0/3402784460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643406c950 0x7f6434160ba0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.236+0000 7f643b285640 1 -- 172.21.15.40:0/3402784460 >> 172.21.15.40:0/3402784460 conn(0x7f64340b39b0 msgr2=0x7f64340b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:32.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.236+0000 7f643b285640 1 -- 172.21.15.40:0/3402784460 shutdown_connections 2024-04-26T21:37:32.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.236+0000 7f643b285640 1 -- 172.21.15.40:0/3402784460 wait complete. 2024-04-26T21:37:32.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.237+0000 7f643b285640 1 Processor -- start 2024-04-26T21:37:32.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.237+0000 7f643b285640 1 -- start start 2024-04-26T21:37:32.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f643b285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643406c950 0x7f643414b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f643b285640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434070930 0x7f64340b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f643b285640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f643415d100 0x7f6434158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f643b285640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6434073160 con 0x7f6434070930 2024-04-26T21:37:32.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f643b285640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64340b1510 con 0x7f643415d100 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f643b285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64340732e0 con 0x7f643406c950 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434070930 0x7f64340b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434070930 0x7f64340b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47336/0 (socket says 172.21.15.40:47336) 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 -- 172.21.15.40:0/1508604976 learned_addr learned my addr 172.21.15.40:0/1508604976 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6439021640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643406c950 0x7f643414b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6439822640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f643415d100 0x7f6434158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 -- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643406c950 msgr2=0x7f643414b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643406c950 0x7f643414b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 -- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f643415d100 msgr2=0x7f6434158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f643415d100 0x7f6434158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 -- 172.21.15.40:0/1508604976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643414da90 con 0x7f6434070930 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6439822640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f643415d100 0x7f6434158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.238+0000 7f6438820640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434070930 0x7f64340b6660 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6424035040 tx=0x7f64240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.239+0000 7f6429ffb640 1 -- 172.21.15.40:0/1508604976 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424037050 con 0x7f6434070930 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.239+0000 7f6429ffb640 1 -- 172.21.15.40:0/1508604976 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6424038040 con 0x7f6434070930 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.239+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64340b5190 con 0x7f6434070930 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.239+0000 7f6429ffb640 1 -- 172.21.15.40:0/1508604976 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424004470 con 0x7f6434070930 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.239+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64340b5750 con 0x7f6434070930 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.240+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64340694f0 con 0x7f6434070930 2024-04-26T21:37:32.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.240+0000 7f6429ffb640 1 -- 172.21.15.40:0/1508604976 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6424030900 con 0x7f6434070930 2024-04-26T21:37:32.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.241+0000 7f6429ffb640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641806c8f0 0x7f641806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.241+0000 7f6429ffb640 1 -- 172.21.15.40:0/1508604976 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f64240ac4c0 con 0x7f6434070930 2024-04-26T21:37:32.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.241+0000 7f6439021640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641806c8f0 0x7f641806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.242+0000 7f6439021640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641806c8f0 0x7f641806edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6430007fd0 tx=0x7f6430007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:32.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.245+0000 7f6429ffb640 1 -- 172.21.15.40:0/1508604976 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f642407aa10 con 0x7f6434070930 2024-04-26T21:37:32.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.432+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f643406cd70 con 0x7f6434070930 2024-04-26T21:37:32.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.435+0000 7f6429ffb640 1 -- 172.21.15.40:0/1508604976 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f642407a310 con 0x7f6434070930 2024-04-26T21:37:32.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641806c8f0 msgr2=0x7f641806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641806c8f0 0x7f641806edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6430007fd0 tx=0x7f6430007de0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434070930 msgr2=0x7f64340b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434070930 0x7f64340b6660 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6424035040 tx=0x7f64240046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 shutdown_connections 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641806c8f0 0x7f641806edb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f643415d100 0x7f6434158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434070930 0x7f64340b6660 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 --2- 172.21.15.40:0/1508604976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643406c950 0x7f643414b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 >> 172.21.15.40:0/1508604976 conn(0x7f64340b39b0 msgr2=0x7f6434161490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 shutdown_connections 2024-04-26T21:37:32.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.437+0000 7f643b285640 1 -- 172.21.15.40:0/1508604976 wait complete. 2024-04-26T21:37:32.448 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:37:32.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:37:32.491 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:37:32.530 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:37:32.530 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:37:32.531 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:32.547 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:37:32.547 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:37:32.547 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:32.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:37:32.589 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:37:32.589 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:32.604 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:37:32.605 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:37:32.605 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:32.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:37:32.645 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:37:32.646 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:32.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.746+0000 7fe2af596640 1 Processor -- start 2024-04-26T21:37:32.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.746+0000 7fe2af596640 1 -- start start 2024-04-26T21:37:32.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2af596640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2a806f790 0x7fe2a806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2af596640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8072c50 0x7fe2a80730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2af596640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a8156840 con 0x7fe2a8156460 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2af596640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a80cbf20 con 0x7fe2a806f790 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2af596640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a80cc200 con 0x7fe2a8072c50 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2acb31640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2a806f790 0x7fe2a806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2acb31640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2a806f790 0x7fe2a806fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47342/0 (socket says 172.21.15.40:47342) 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2acb31640 1 -- 172.21.15.40:0/923511372 learned_addr learned my addr 172.21.15.40:0/923511372 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2adb33640 1 --2- 172.21.15.40:0/923511372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8072c50 0x7fe2a80730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2acb31640 1 -- 172.21.15.40:0/923511372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8072c50 msgr2=0x7fe2a80730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2acb31640 1 --2- 172.21.15.40:0/923511372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8072c50 0x7fe2a80730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2acb31640 1 -- 172.21.15.40:0/923511372 >> v1:172.21.15.184:6789/0 conn(0x7fe2a8156460 legacy=0x7fe2a80cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.747+0000 7fe2acb31640 1 -- 172.21.15.40:0/923511372 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a80cc4e0 con 0x7fe2a806f790 2024-04-26T21:37:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.748+0000 7fe2acb31640 1 --2- 172.21.15.40:0/923511372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a806fc40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe298009870 tx=0x7fe29802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96307de83a74c44f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.748+0000 7fe29ffff640 1 -- 172.21.15.40:0/923511372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298037070 con 0x7fe2a806f790 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.748+0000 7fe29ffff640 1 -- 172.21.15.40:0/923511372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe298032440 con 0x7fe2a806f790 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.748+0000 7fe29ffff640 1 -- 172.21.15.40:0/923511372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298032810 con 0x7fe2a806f790 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.749+0000 7fe2af596640 1 -- 172.21.15.40:0/923511372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 msgr2=0x7fe2a806fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.749+0000 7fe2af596640 1 --2- 172.21.15.40:0/923511372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a806fc40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe298009870 tx=0x7fe29802a270 comp rx=0 tx=0).stop 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.749+0000 7fe2af596640 1 -- 172.21.15.40:0/923511372 shutdown_connections 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.749+0000 7fe2af596640 1 --2- 172.21.15.40:0/923511372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8072c50 0x7fe2a80730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.749+0000 7fe2af596640 1 --2- 172.21.15.40:0/923511372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a806fc40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.749+0000 7fe2af596640 1 -- 172.21.15.40:0/923511372 >> 172.21.15.40:0/923511372 conn(0x7fe2a80d0be0 msgr2=0x7fe2a80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.749+0000 7fe2af596640 1 -- 172.21.15.40:0/923511372 shutdown_connections 2024-04-26T21:37:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.749+0000 7fe2af596640 1 -- 172.21.15.40:0/923511372 wait complete. 2024-04-26T21:37:32.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.750+0000 7fe2af596640 1 Processor -- start 2024-04-26T21:37:32.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.750+0000 7fe2af596640 1 -- start start 2024-04-26T21:37:32.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2af596640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a81f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2af596640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8072c50 0x7fe2a81f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2af596640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 0x7fe2a80c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2af596640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a80cbbe0 con 0x7fe2a806f790 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2af596640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a8072700 con 0x7fe2a8072c50 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2af596640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a806e260 con 0x7fe2a8156460 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a81f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a81f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47352/0 (socket says 172.21.15.40:47352) 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 -- 172.21.15.40:0/4124072284 learned_addr learned my addr 172.21.15.40:0/4124072284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2adb33640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 0x7fe2a80c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2acb31640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8072c50 0x7fe2a81f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 -- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 msgr2=0x7fe2a80c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 0x7fe2a80c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 -- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8072c50 msgr2=0x7fe2a81f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8072c50 0x7fe2a81f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 -- 172.21.15.40:0/4124072284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a81f44c0 con 0x7fe2a806f790 2024-04-26T21:37:32.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2acb31640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8072c50 0x7fe2a81f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:32.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.751+0000 7fe2ad332640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a81f6750 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400c3c0 tx=0x7fe2a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:32.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.752+0000 7fe29dffb640 1 -- 172.21.15.40:0/4124072284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a40052d0 con 0x7fe2a806f790 2024-04-26T21:37:32.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.752+0000 7fe29dffb640 1 -- 172.21.15.40:0/4124072284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a4005490 con 0x7fe2a806f790 2024-04-26T21:37:32.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.752+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a81f47a0 con 0x7fe2a806f790 2024-04-26T21:37:32.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.752+0000 7fe29dffb640 1 -- 172.21.15.40:0/4124072284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4007260 con 0x7fe2a806f790 2024-04-26T21:37:32.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.752+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a81f4d60 con 0x7fe2a806f790 2024-04-26T21:37:32.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.753+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a80694f0 con 0x7fe2a806f790 2024-04-26T21:37:32.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.753+0000 7fe29dffb640 1 -- 172.21.15.40:0/4124072284 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2a4007400 con 0x7fe2a806f790 2024-04-26T21:37:32.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.753+0000 7fe29dffb640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:32.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.754+0000 7fe29dffb640 1 -- 172.21.15.40:0/4124072284 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fe2a408ae10 con 0x7fe2a806f790 2024-04-26T21:37:32.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.754+0000 7fe2acb31640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:32.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.754+0000 7fe2acb31640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe28c06c8f0 0x7fe28c06edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe298034000 tx=0x7fe29802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:32.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.758+0000 7fe29dffb640 1 -- 172.21.15.40:0/4124072284 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2a4058da0 con 0x7fe2a806f790 2024-04-26T21:37:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.980+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2a8155c10 con 0x7fe2a806f790 2024-04-26T21:37:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.981+0000 7fe29dffb640 1 -- 172.21.15.40:0/4124072284 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3872 (secure 0 0 0) 0x7fe2a405d0a0 con 0x7fe2a806f790 2024-04-26T21:37:32.981 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:32.981 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2962532870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2962532870},{"type":"v1","addr":"172.21.15.184:6836","nonce":2962532870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2377025756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2377025756},{"type":"v1","addr":"172.21.15.40:6834","nonce":2377025756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/454498269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":454498269},{"type":"v1","addr":"172.21.15.184:6837","nonce":454498269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3814559316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3814559316},{"type":"v1","addr":"172.21.15.40:6835","nonce":3814559316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[]} 2024-04-26T21:37:32.982 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2024-04-26T21:37:32.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe28c06c8f0 msgr2=0x7fe28c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe28c06c8f0 0x7fe28c06edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe298034000 tx=0x7fe29802e000 comp rx=0 tx=0).stop 2024-04-26T21:37:32.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 msgr2=0x7fe2a81f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:32.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a81f6750 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400c3c0 tx=0x7fe2a400c880 comp rx=0 tx=0).stop 2024-04-26T21:37:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 shutdown_connections 2024-04-26T21:37:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 0x7fe2a80c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8072c50 0x7fe2a81f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 --2- 172.21.15.40:0/4124072284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a806f790 0x7fe2a81f6750 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 >> 172.21.15.40:0/4124072284 conn(0x7fe2a80d0be0 msgr2=0x7fe2a80b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 shutdown_connections 2024-04-26T21:37:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:32.983+0000 7fe2af596640 1 -- 172.21.15.40:0/4124072284 wait complete. 2024-04-26T21:37:32.994 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:37:32.994 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:37:32.995 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:37:32.995 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:37:32.995 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:37:32.995 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:37:32.995 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:37:32.995 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:37:32.996 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:37:32.996 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:37:32.996 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:37:32.996 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:37:32.996 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:37:32.996 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:37:32.997 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:37:32.997 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:37:33.004 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:37:33.004 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:37:33.005 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:37:33.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:37:33.008 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:37:33.009 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:37:33.009 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:37:33.010 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:37:33.046 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:37:33.047 DEBUG:teuthology.parallel:result is None 2024-04-26T21:37:33.047 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:37:33.047 DEBUG:teuthology.parallel:result is None 2024-04-26T21:37:33.051 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:37:33.051 DEBUG:teuthology.parallel:result is None 2024-04-26T21:37:33.052 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:37:33.052 DEBUG:teuthology.parallel:result is None 2024-04-26T21:37:33.052 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:37:33.123 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:37:33.123 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:37:33.133 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:37:33.133 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:37:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.159+0000 7fe2142b8640 1 Processor -- start 2024-04-26T21:37:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.159+0000 7fe2142b8640 1 -- start start 2024-04-26T21:37:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe2142b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c163610 0x7fe20c0d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe2142b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe20c0d2830 0x7fe20c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe212054640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe20c15f5a0 0x7fe20c162f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50810/0 (socket says 172.21.15.40:50810) 2024-04-26T21:37:33.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe212855640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe20c0d2830 0x7fe20c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:33.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe212054640 1 -- 172.21.15.40:0/4067654078 learned_addr learned my addr 172.21.15.40:0/4067654078 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:33.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe2142b8640 1 -- 172.21.15.40:0/4067654078 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe20c0d1a70 con 0x7fe20c15f5a0 2024-04-26T21:37:33.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe2142b8640 1 -- 172.21.15.40:0/4067654078 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe20c163d30 con 0x7fe20c0d2830 2024-04-26T21:37:33.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe2142b8640 1 -- 172.21.15.40:0/4067654078 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe20c0d1d50 con 0x7fe20c163610 2024-04-26T21:37:33.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe211853640 1 --2- 172.21.15.40:0/4067654078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c163610 0x7fe20c0d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe211052640 1 -- 172.21.15.40:0/4067654078 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2298922918 0 0) 0x7fe20c0d1a70 con 0x7fe20c15f5a0 2024-04-26T21:37:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe211052640 1 -- 172.21.15.40:0/4067654078 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1f8003520 con 0x7fe20c15f5a0 2024-04-26T21:37:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe212855640 1 -- 172.21.15.40:0/4067654078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c163610 msgr2=0x7fe20c0d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe212855640 1 --2- 172.21.15.40:0/4067654078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c163610 0x7fe20c0d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.160+0000 7fe212855640 1 -- 172.21.15.40:0/4067654078 >> v1:172.21.15.40:6789/0 conn(0x7fe20c15f5a0 legacy=0x7fe20c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.161+0000 7fe212855640 1 -- 172.21.15.40:0/4067654078 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c0d2030 con 0x7fe20c0d2830 2024-04-26T21:37:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.161+0000 7fe212855640 1 --2- 172.21.15.40:0/4067654078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c0d2830 0x7fe20c0cfda0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe204009d90 tx=0x7fe20402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2675eb6778f972 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.161+0000 7fe211052640 1 -- 172.21.15.40:0/4067654078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204031600 con 0x7fe20c0d2830 2024-04-26T21:37:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.161+0000 7fe211052640 1 -- 172.21.15.40:0/4067654078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2040317a0 con 0x7fe20c0d2830 2024-04-26T21:37:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.161+0000 7fe211052640 1 -- 172.21.15.40:0/4067654078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204031b70 con 0x7fe20c0d2830 2024-04-26T21:37:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.162+0000 7fe2142b8640 1 -- 172.21.15.40:0/4067654078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c0d2830 msgr2=0x7fe20c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.162+0000 7fe2142b8640 1 --2- 172.21.15.40:0/4067654078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c0d2830 0x7fe20c0cfda0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe204009d90 tx=0x7fe20402a5d0 comp rx=0 tx=0).stop 2024-04-26T21:37:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.162+0000 7fe2142b8640 1 -- 172.21.15.40:0/4067654078 shutdown_connections 2024-04-26T21:37:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.162+0000 7fe2142b8640 1 --2- 172.21.15.40:0/4067654078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c0d2830 0x7fe20c0cfda0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.162+0000 7fe2142b8640 1 --2- 172.21.15.40:0/4067654078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c163610 0x7fe20c0d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.162+0000 7fe2142b8640 1 -- 172.21.15.40:0/4067654078 >> 172.21.15.40:0/4067654078 conn(0x7fe20c0d5190 msgr2=0x7fe20c0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.162+0000 7fe2142b8640 1 -- 172.21.15.40:0/4067654078 shutdown_connections 2024-04-26T21:37:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.162+0000 7fe2142b8640 1 -- 172.21.15.40:0/4067654078 wait complete. 2024-04-26T21:37:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.163+0000 7fe2142b8640 1 Processor -- start 2024-04-26T21:37:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.163+0000 7fe2142b8640 1 -- start start 2024-04-26T21:37:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe2142b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c0d2830 0x7fe20c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe2142b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c163610 0x7fe20c161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe2142b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1624d0 0x7fe20c1e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe2142b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c0d3c20 con 0x7fe20c163610 2024-04-26T21:37:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe2142b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c0d4190 con 0x7fe20c1624d0 2024-04-26T21:37:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe2142b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe20c0d3da0 con 0x7fe20c0d2830 2024-04-26T21:37:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe211853640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c163610 0x7fe20c161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe211853640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c163610 0x7fe20c161f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47422/0 (socket says 172.21.15.40:47422) 2024-04-26T21:37:33.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe211853640 1 -- 172.21.15.40:0/1875767433 learned_addr learned my addr 172.21.15.40:0/1875767433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:33.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe212054640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c0d2830 0x7fe20c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:33.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe212855640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1624d0 0x7fe20c1e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:33.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe211853640 1 -- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c0d2830 msgr2=0x7fe20c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:33.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe211853640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c0d2830 0x7fe20c153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:33.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe211853640 1 -- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1624d0 msgr2=0x7fe20c1e8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:33.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe211853640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1624d0 0x7fe20c1e8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:33.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe211853640 1 -- 172.21.15.40:0/1875767433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c1f9720 con 0x7fe20c163610 2024-04-26T21:37:33.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.164+0000 7fe212855640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1624d0 0x7fe20c1e8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:33.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.165+0000 7fe211853640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c163610 0x7fe20c161f90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe20800d620 tx=0x7fe20800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:33.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.165+0000 7fe202ffd640 1 -- 172.21.15.40:0/1875767433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20800b620 con 0x7fe20c163610 2024-04-26T21:37:33.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.165+0000 7fe202ffd640 1 -- 172.21.15.40:0/1875767433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe208019070 con 0x7fe20c163610 2024-04-26T21:37:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.165+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20c1f8960 con 0x7fe20c163610 2024-04-26T21:37:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.165+0000 7fe202ffd640 1 -- 172.21.15.40:0/1875767433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20800ba00 con 0x7fe20c163610 2024-04-26T21:37:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.165+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20c1f8f20 con 0x7fe20c163610 2024-04-26T21:37:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.166+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20c1628e0 con 0x7fe20c163610 2024-04-26T21:37:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.166+0000 7fe202ffd640 1 -- 172.21.15.40:0/1875767433 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe20800bba0 con 0x7fe20c163610 2024-04-26T21:37:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.167+0000 7fe202ffd640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1f006c8f0 0x7fe1f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.167+0000 7fe202ffd640 1 -- 172.21.15.40:0/1875767433 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fe20808be50 con 0x7fe20c163610 2024-04-26T21:37:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.167+0000 7fe212054640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1f006c8f0 0x7fe1f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.168+0000 7fe212054640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1f006c8f0 0x7fe1f006edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0011e0 tx=0x7fe1fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.171+0000 7fe202ffd640 1 -- 172.21.15.40:0/1875767433 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe20c1628e0 con 0x7fe20c163610 2024-04-26T21:37:33.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:33.369+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe20c0694f0 con 0x7fe20c163610 2024-04-26T21:37:34.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.121+0000 7fe202ffd640 1 -- 172.21.15.40:0/1875767433 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) ==== 97+0+0 (secure 0 0 0) 0x7fe20805a3b0 con 0x7fe20c163610 2024-04-26T21:37:34.121 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.123+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1f006c8f0 msgr2=0x7fe1f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1f006c8f0 0x7fe1f006edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0011e0 tx=0x7fe1fc009040 comp rx=0 tx=0).stop 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c163610 msgr2=0x7fe20c161f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c163610 0x7fe20c161f90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe20800d620 tx=0x7fe20800dae0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 shutdown_connections 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1f006c8f0 0x7fe1f006edb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1624d0 0x7fe20c1e8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe20c163610 0x7fe20c161f90 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 --2- 172.21.15.40:0/1875767433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe20c0d2830 0x7fe20c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 >> 172.21.15.40:0/1875767433 conn(0x7fe20c0d5190 msgr2=0x7fe20c0d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 shutdown_connections 2024-04-26T21:37:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.124+0000 7fe2142b8640 1 -- 172.21.15.40:0/1875767433 wait complete. 2024-04-26T21:37:34.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:37:34.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.246+0000 7fb50e1f7640 1 Processor -- start 2024-04-26T21:37:34.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.246+0000 7fb50e1f7640 1 -- start start 2024-04-26T21:37:34.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.247+0000 7fb50e1f7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb5080d1b30 0x7fb5080d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.247+0000 7fb50e1f7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5080d10d0 con 0x7fb5080b2340 2024-04-26T21:37:34.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.247+0000 7fb50e1f7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5080d1370 con 0x7fb508162810 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.247+0000 7fb50e1f7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb5080d0750 con 0x7fb5080d1b30 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.247+0000 7fb506ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb5080b2340 0x7fb5080cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50822/0 (socket says 172.21.15.40:50822) 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.247+0000 7fb506ffd640 1 -- 172.21.15.40:0/1072256571 learned_addr learned my addr 172.21.15.40:0/1072256571 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb507fff640 1 --2- 172.21.15.40:0/1072256571 >> v2:172.21.15.184:3300/0 conn(0x7fb5080d1b30 0x7fb5080d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1380676170 0 0) 0x7fb5080d10d0 con 0x7fb5080b2340 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4f0003520 con 0x7fb5080b2340 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3476951816 0 0) 0x7fb4f0003520 con 0x7fb5080b2340 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 >> v2:172.21.15.184:3300/0 conn(0x7fb5080d1b30 msgr2=0x7fb5080d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 --2- 172.21.15.40:0/1072256571 >> v2:172.21.15.184:3300/0 conn(0x7fb5080d1b30 0x7fb5080d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 >> v1:172.21.15.184:6789/0 conn(0x7fb508162810 legacy=0x7fb508070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5080d01b0 con 0x7fb5080b2340 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb4fc001670 con 0x7fb5080b2340 2024-04-26T21:37:34.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 >> v1:172.21.15.40:6789/0 conn(0x7fb5080b2340 legacy=0x7fb5080cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4f00039b0 0x7fb4f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f00051c0 0x7fb4f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5080d0750 con 0x7fb4f0004800 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f000c060 con 0x7fb4f00051c0 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f000c260 con 0x7fb4f00039b0 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb506ffd640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb507fff640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f00051c0 0x7fb4f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb5077fe640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4f00039b0 0x7fb4f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb506ffd640 1 -- 172.21.15.40:0/1072256571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4f00039b0 msgr2=0x7fb4f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb506ffd640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4f00039b0 0x7fb4f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb506ffd640 1 -- 172.21.15.40:0/1072256571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f00051c0 msgr2=0x7fb4f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb506ffd640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f00051c0 0x7fb4f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb506ffd640 1 -- 172.21.15.40:0/1072256571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f0003520 con 0x7fb4f0004800 2024-04-26T21:37:34.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.248+0000 7fb507fff640 1 -- 172.21.15.40:0/1072256571 reap_dead start 2024-04-26T21:37:34.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.249+0000 7fb506ffd640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00c210 tx=0x7fb4f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1db40467b6c488 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.249+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc007d10 con 0x7fb4f0004800 2024-04-26T21:37:34.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.249+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc005af0 con 0x7fb4f0004800 2024-04-26T21:37:34.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.249+0000 7fb5067fc640 1 -- 172.21.15.40:0/1072256571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc007070 con 0x7fb4f0004800 2024-04-26T21:37:34.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.250+0000 7fb50e1f7640 1 -- 172.21.15.40:0/1072256571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4f0004800 msgr2=0x7fb4f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.250+0000 7fb50e1f7640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00c210 tx=0x7fb4f0005830 comp rx=0 tx=0).stop 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.250+0000 7fb50e1f7640 1 -- 172.21.15.40:0/1072256571 shutdown_connections 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.250+0000 7fb50e1f7640 1 --2- 172.21.15.40:0/1072256571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.250+0000 7fb50e1f7640 1 -- 172.21.15.40:0/1072256571 >> 172.21.15.40:0/1072256571 conn(0x7fb5080d2da0 msgr2=0x7fb5080d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.250+0000 7fb50e1f7640 1 -- 172.21.15.40:0/1072256571 shutdown_connections 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.250+0000 7fb50e1f7640 1 -- 172.21.15.40:0/1072256571 wait complete. 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.251+0000 7fb50e1f7640 1 Processor -- start 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.251+0000 7fb50e1f7640 1 -- start start 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb50e1f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f0004800 0x7fb508153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb50e1f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50815b350 0x7fb5080cf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb50e1f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb50815b7d0 0x7fb5080cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb50e1f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5080d1fc0 con 0x7fb50815b7d0 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb50e1f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5080d16a0 con 0x7fb4f0004800 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb50e1f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5080d2140 con 0x7fb50815b350 2024-04-26T21:37:34.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb507fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb50815b7d0 0x7fb5080cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb507fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb50815b7d0 0x7fb5080cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47430/0 (socket says 172.21.15.40:47430) 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb507fff640 1 -- 172.21.15.40:0/585328009 learned_addr learned my addr 172.21.15.40:0/585328009 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb506ffd640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50815b350 0x7fb5080cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb5077fe640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f0004800 0x7fb508153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.252+0000 7fb507fff640 1 -- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50815b350 msgr2=0x7fb5080cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb507fff640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50815b350 0x7fb5080cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb507fff640 1 -- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f0004800 msgr2=0x7fb508153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb507fff640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f0004800 0x7fb508153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb507fff640 1 -- 172.21.15.40:0/585328009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5080c7eb0 con 0x7fb50815b7d0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb507fff640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb50815b7d0 0x7fb5080cf950 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb508073630 tx=0x7fb50000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb4effff640 1 -- 172.21.15.40:0/585328009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50000fc80 con 0x7fb50815b7d0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb4effff640 1 -- 172.21.15.40:0/585328009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50000fe20 con 0x7fb50815b7d0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb4effff640 1 -- 172.21.15.40:0/585328009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500015670 con 0x7fb50815b7d0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5080c8190 con 0x7fb50815b7d0 2024-04-26T21:37:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.253+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5080c8750 con 0x7fb50815b7d0 2024-04-26T21:37:34.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.254+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5080694f0 con 0x7fb50815b7d0 2024-04-26T21:37:34.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.254+0000 7fb4effff640 1 -- 172.21.15.40:0/585328009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb500015810 con 0x7fb50815b7d0 2024-04-26T21:37:34.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.255+0000 7fb4effff640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.255+0000 7fb4effff640 1 -- 172.21.15.40:0/585328009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7fb50008c880 con 0x7fb50815b7d0 2024-04-26T21:37:34.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.255+0000 7fb5077fe640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.256+0000 7fb5077fe640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8005590 tx=0x7fb4f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.259+0000 7fb4effff640 1 -- 172.21.15.40:0/585328009 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb500019030 con 0x7fb50815b7d0 2024-04-26T21:37:34.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.517+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb50806deb0 con 0x7fb50815b7d0 2024-04-26T21:37:34.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.518+0000 7fb4effff640 1 -- 172.21.15.40:0/585328009 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fb500010040 con 0x7fb50815b7d0 2024-04-26T21:37:34.518 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:34.518 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:37:34.518 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:37:34.518 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmesG+ERAAJS53jYoKUawIOs2zrG5TLg==", 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:37:34.519 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmq9cvHRAAiYWrMjfAth8wMbs9YlfWgQ==", 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmLRmvGBAA6xAcjLDF7DMjfDqiUCexKA==", 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.520 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmXxkDJBAAbzzf+LUO3Rr8bdzXb+Gj2Q==", 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.521 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDYHSxmRkhFBhAAz1UfvJwhA8bolDiRN2eM+Q==", 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:37:34.522 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDbHSxmKAB/BBAAzW5dtz6eczjkDiTQBjfjuA==", 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDeHSxmrienAhAA2JS0l6/Uo9UPc0TeZ1RwRA==", 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDhHSxmv40rARAAN8z+beJe/bTERvO2euMr6w==", 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDnHSxmxQ1XCBAAYptVM8WPir+zIHOjbX0tGg==", 2024-04-26T21:37:34.524 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDqHSxmv32IBhAAMROkCLjWTOx3ELdkEiVSmw==", 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDtHSxmJdPTBBAAO32qvVLuAzqx1xdajL9CFw==", 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDwHSxmcUMYAxAAIvoAo82ApDXERodD+4+cBQ==", 2024-04-26T21:37:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw==", 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmifWYMBAA6Nskvqt3MZSvFCMHjslELw==", 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmi6l6LRAAF4EjChVEB8ReULEsl3YppA==", 2024-04-26T21:37:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmhSMFNhAAjCJvc4FknsSUJQ4NjXRV6Q==", 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:37:34.529 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDTHSxmQk1bMhAAixlszeAmlXc5cAmkFWjhfw==", 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:37:34.530 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmYjmPDhAADvvIHrcacI8kOAdeiJGd+Q==", 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmsiwICxAAmKAZ2KS3/dYubutkcGfolA==", 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:37:34.531 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.532 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:34.532 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:37:34.532 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:37:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 msgr2=0x7fb4f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8005590 tx=0x7fb4f8009040 comp rx=0 tx=0).stop 2024-04-26T21:37:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb50815b7d0 msgr2=0x7fb5080cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb50815b7d0 0x7fb5080cf950 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb508073630 tx=0x7fb50000f1b0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 shutdown_connections 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb50815b7d0 0x7fb5080cf950 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50815b350 0x7fb5080cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 --2- 172.21.15.40:0/585328009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f0004800 0x7fb508153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 >> 172.21.15.40:0/585328009 conn(0x7fb5080d2da0 msgr2=0x7fb5080ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 shutdown_connections 2024-04-26T21:37:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.520+0000 7fb50e1f7640 1 -- 172.21.15.40:0/585328009 wait complete. 2024-04-26T21:37:34.535 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.635+0000 7f12a9b78640 1 Processor -- start 2024-04-26T21:37:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.635+0000 7f12a9b78640 1 -- start start 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a9b78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a4162880 0x7f12a40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a9b78640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f12a40b5f20 0x7f12a40d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a9b78640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12a40d39f0 0x7f12a40b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a9b78640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f12a40d0580 con 0x7f12a40d39f0 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a9b78640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f12a40b7e50 con 0x7f12a40b5f20 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a9b78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f12a40d1800 con 0x7f12a4162880 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12a40d39f0 0x7f12a40b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12a40d39f0 0x7f12a40b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47434/0 (socket says 172.21.15.40:47434) 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.636+0000 7f12a3fff640 1 -- 172.21.15.40:0/1997199596 learned_addr learned my addr 172.21.15.40:0/1997199596 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a37fe640 1 --2- 172.21.15.40:0/1997199596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a4162880 0x7f12a40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a2ffd640 1 --2- 172.21.15.40:0/1997199596 >> v2:172.21.15.184:3300/0 conn(0x7f12a40b5f20 0x7f12a40d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a3fff640 1 -- 172.21.15.40:0/1997199596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a4162880 msgr2=0x7f12a40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a3fff640 1 --2- 172.21.15.40:0/1997199596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a4162880 0x7f12a40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a3fff640 1 -- 172.21.15.40:0/1997199596 >> v2:172.21.15.184:3300/0 conn(0x7f12a40b5f20 msgr2=0x7f12a40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a3fff640 1 --2- 172.21.15.40:0/1997199596 >> v2:172.21.15.184:3300/0 conn(0x7f12a40b5f20 0x7f12a40d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a3fff640 1 -- 172.21.15.40:0/1997199596 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a40d07d0 con 0x7f12a40d39f0 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a37fe640 1 --2- 172.21.15.40:0/1997199596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a4162880 0x7f12a40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a2ffd640 1 --2- 172.21.15.40:0/1997199596 >> v2:172.21.15.184:3300/0 conn(0x7f12a40b5f20 0x7f12a40d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a3fff640 1 --2- 172.21.15.40:0/1997199596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a40d39f0 0x7f12a40b69e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f12a408e960 tx=0x7f129c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc880eb2bf09569 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.637+0000 7f12a27fc640 1 -- 172.21.15.40:0/1997199596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c036070 con 0x7f12a40d39f0 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a27fc640 1 -- 172.21.15.40:0/1997199596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c031600 con 0x7f12a40d39f0 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a27fc640 1 -- 172.21.15.40:0/1997199596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c0319d0 con 0x7f12a40d39f0 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 -- 172.21.15.40:0/1997199596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a40d39f0 msgr2=0x7f12a40b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 --2- 172.21.15.40:0/1997199596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a40d39f0 0x7f12a40b69e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f12a408e960 tx=0x7f129c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 -- 172.21.15.40:0/1997199596 shutdown_connections 2024-04-26T21:37:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 --2- 172.21.15.40:0/1997199596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a40d39f0 0x7f12a40b69e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 --2- 172.21.15.40:0/1997199596 >> v2:172.21.15.184:3300/0 conn(0x7f12a40b5f20 0x7f12a40d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 --2- 172.21.15.40:0/1997199596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a4162880 0x7f12a40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 -- 172.21.15.40:0/1997199596 >> 172.21.15.40:0/1997199596 conn(0x7f12a40d4f60 msgr2=0x7f12a40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 -- 172.21.15.40:0/1997199596 shutdown_connections 2024-04-26T21:37:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.638+0000 7f12a9b78640 1 -- 172.21.15.40:0/1997199596 wait complete. 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.639+0000 7f12a9b78640 1 Processor -- start 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a9b78640 1 -- start start 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a9b78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a40b5f20 0x7f12a40b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a9b78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a40d39f0 0x7f12a40b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a9b78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a4162880 0x7f12a4153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a9b78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4162c60 con 0x7f12a4162880 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a9b78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12a40d0d90 con 0x7f12a40b5f20 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a9b78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f12a4162de0 con 0x7f12a40d39f0 2024-04-26T21:37:34.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a4162880 0x7f12a4153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a4162880 0x7f12a4153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47448/0 (socket says 172.21.15.40:47448) 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.640+0000 7f12a3fff640 1 -- 172.21.15.40:0/3580535945 learned_addr learned my addr 172.21.15.40:0/3580535945 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a2ffd640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a40d39f0 0x7f12a40b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a37fe640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a40b5f20 0x7f12a40b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a3fff640 1 -- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a40d39f0 msgr2=0x7f12a40b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a3fff640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a40d39f0 0x7f12a40b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a3fff640 1 -- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a40b5f20 msgr2=0x7f12a40b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a3fff640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a40b5f20 0x7f12a40b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a3fff640 1 -- 172.21.15.40:0/3580535945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a40b2250 con 0x7f12a4162880 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a37fe640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a40b5f20 0x7f12a40b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a2ffd640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a40d39f0 0x7f12a40b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a3fff640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a4162880 0x7f12a4153220 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f129c034040 tx=0x7f129c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f128bfff640 1 -- 172.21.15.40:0/3580535945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c041020 con 0x7f12a4162880 2024-04-26T21:37:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f128bfff640 1 -- 172.21.15.40:0/3580535945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c007780 con 0x7f12a4162880 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a40b2530 con 0x7f12a4162880 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f128bfff640 1 -- 172.21.15.40:0/3580535945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c036040 con 0x7f12a4162880 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.641+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a40b2af0 con 0x7f12a4162880 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.642+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a40694f0 con 0x7f12a4162880 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.643+0000 7f128bfff640 1 -- 172.21.15.40:0/3580535945 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f129c044020 con 0x7f12a4162880 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.643+0000 7f128bfff640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.643+0000 7f128bfff640 1 -- 172.21.15.40:0/3580535945 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f129c0ac710 con 0x7f12a4162880 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.643+0000 7f12a37fe640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.644+0000 7f12a37fe640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1294006fd0 tx=0x7f1294008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.647+0000 7f128bfff640 1 -- 172.21.15.40:0/3580535945 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f129c04b030 con 0x7f12a4162880 2024-04-26T21:37:34.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.832+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12a4160fb0 con 0x7f12a4162880 2024-04-26T21:37:34.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.833+0000 7f128bfff640 1 -- 172.21.15.40:0/3580535945 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+13128 (secure 0 0 0) 0x7f129c07ae50 con 0x7f12a4162880 2024-04-26T21:37:34.833 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:34.833 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":48,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:34.135294+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 msgr2=0x7f129006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1294006fd0 tx=0x7f1294008040 comp rx=0 tx=0).stop 2024-04-26T21:37:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a4162880 msgr2=0x7f12a4153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a4162880 0x7f12a4153220 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f129c034040 tx=0x7f129c004a40 comp rx=0 tx=0).stop 2024-04-26T21:37:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 shutdown_connections 2024-04-26T21:37:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f129006c8f0 0x7f129006edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a4162880 0x7f12a4153220 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a40d39f0 0x7f12a40b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 --2- 172.21.15.40:0/3580535945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a40b5f20 0x7f12a40b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 >> 172.21.15.40:0/3580535945 conn(0x7f12a40d4f60 msgr2=0x7f12a41608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 shutdown_connections 2024-04-26T21:37:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.835+0000 7f12a9b78640 1 -- 172.21.15.40:0/3580535945 wait complete. 2024-04-26T21:37:34.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:37:34.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:37:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.951+0000 7f3b33151640 1 Processor -- start 2024-04-26T21:37:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.951+0000 7f3b33151640 1 -- start start 2024-04-26T21:37:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b33151640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c0d3bc0 0x7f3b2c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b33151640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c0d1040 con 0x7f3b2c15f170 2024-04-26T21:37:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b33151640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c0c9680 con 0x7f3b2c15f550 2024-04-26T21:37:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b33151640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c0c9430 con 0x7f3b2c0d3bc0 2024-04-26T21:37:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b30eed640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3b2c15f170 0x7f3b2c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50834/0 (socket says 172.21.15.40:50834) 2024-04-26T21:37:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b30eed640 1 -- 172.21.15.40:0/3770166160 learned_addr learned my addr 172.21.15.40:0/3770166160 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b316ee640 1 --2- 172.21.15.40:0/3770166160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c0d3bc0 0x7f3b2c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b2b7fe640 1 -- 172.21.15.40:0/3770166160 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 41935032 0 0) 0x7f3b1c001110 con 0x7f3b2c15f170 2024-04-26T21:37:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b2b7fe640 1 -- 172.21.15.40:0/3770166160 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b14003520 con 0x7f3b2c15f170 2024-04-26T21:37:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b2b7fe640 1 -- 172.21.15.40:0/3770166160 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368372317 0 0) 0x7f3b2c0c9680 con 0x7f3b2c15f550 2024-04-26T21:37:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b2b7fe640 1 -- 172.21.15.40:0/3770166160 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b1c001110 con 0x7f3b2c15f550 2024-04-26T21:37:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b2b7fe640 1 -- 172.21.15.40:0/3770166160 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1993632900 0 0) 0x7f3b14003520 con 0x7f3b2c15f170 2024-04-26T21:37:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b316ee640 1 -- 172.21.15.40:0/3770166160 >> v1:172.21.15.184:6789/0 conn(0x7f3b2c15f550 legacy=0x7f3b2c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.952+0000 7f3b316ee640 1 -- 172.21.15.40:0/3770166160 >> v1:172.21.15.40:6789/0 conn(0x7f3b2c15f170 legacy=0x7f3b2c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.953+0000 7f3b316ee640 1 -- 172.21.15.40:0/3770166160 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c0c9960 con 0x7f3b2c0d3bc0 2024-04-26T21:37:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.953+0000 7f3b316ee640 1 --2- 172.21.15.40:0/3770166160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c0d3bc0 0x7f3b2c0d5a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009df0 tx=0x7f3b2402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4736e7512e3ea00f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.953+0000 7f3b2b7fe640 1 -- 172.21.15.40:0/3770166160 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c001840 con 0x7f3b2c0d3bc0 2024-04-26T21:37:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.953+0000 7f3b2b7fe640 1 -- 172.21.15.40:0/3770166160 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24007540 con 0x7f3b2c0d3bc0 2024-04-26T21:37:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.953+0000 7f3b2b7fe640 1 -- 172.21.15.40:0/3770166160 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24007710 con 0x7f3b2c0d3bc0 2024-04-26T21:37:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.954+0000 7f3b33151640 1 -- 172.21.15.40:0/3770166160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c0d3bc0 msgr2=0x7f3b2c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.954+0000 7f3b33151640 1 --2- 172.21.15.40:0/3770166160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c0d3bc0 0x7f3b2c0d5a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009df0 tx=0x7f3b2402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.954+0000 7f3b33151640 1 -- 172.21.15.40:0/3770166160 shutdown_connections 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.954+0000 7f3b33151640 1 --2- 172.21.15.40:0/3770166160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c0d3bc0 0x7f3b2c0d5a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.954+0000 7f3b33151640 1 -- 172.21.15.40:0/3770166160 >> 172.21.15.40:0/3770166160 conn(0x7f3b2c0d4f60 msgr2=0x7f3b2c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.954+0000 7f3b33151640 1 -- 172.21.15.40:0/3770166160 shutdown_connections 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.954+0000 7f3b33151640 1 -- 172.21.15.40:0/3770166160 wait complete. 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.955+0000 7f3b33151640 1 Processor -- start 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.955+0000 7f3b33151640 1 -- start start 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b33151640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b2c0d3bc0 0x7f3b2c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b33151640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c15f170 0x7f3b2c15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b33151640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b2c15ccd0 0x7f3b2c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b33151640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c0d2e30 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b33151640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c0d2cb0 con 0x7f3b2c0d3bc0 2024-04-26T21:37:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b33151640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c0d2fb0 con 0x7f3b2c15f170 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b316ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b2c15ccd0 0x7f3b2c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b316ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b2c15ccd0 0x7f3b2c159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47450/0 (socket says 172.21.15.40:47450) 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b316ee640 1 -- 172.21.15.40:0/3841408512 learned_addr learned my addr 172.21.15.40:0/3841408512 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b2bfff640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c15f170 0x7f3b2c15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.956+0000 7f3b30eed640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b2c0d3bc0 0x7f3b2c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b316ee640 1 -- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c15f170 msgr2=0x7f3b2c15c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b316ee640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c15f170 0x7f3b2c15c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b316ee640 1 -- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b2c0d3bc0 msgr2=0x7f3b2c1522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b316ee640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b2c0d3bc0 0x7f3b2c1522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b316ee640 1 -- 172.21.15.40:0/3841408512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c1527e0 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b30eed640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b2c0d3bc0 0x7f3b2c1522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b316ee640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b2c15ccd0 0x7f3b2c159710 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009f10 tx=0x7f3b24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b297fa640 1 -- 172.21.15.40:0/3841408512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24040070 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b297fa640 1 -- 172.21.15.40:0/3841408512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2403c830 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c15a360 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b297fa640 1 -- 172.21.15.40:0/3841408512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2403cc00 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.957+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c15a920 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.958+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c0694f0 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.958+0000 7f3b297fa640 1 -- 172.21.15.40:0/3841408512 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b24039040 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.959+0000 7f3b297fa640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1806c920 0x7f3b1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.959+0000 7f3b297fa640 1 -- 172.21.15.40:0/3841408512 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f3b240ad0e0 con 0x7f3b2c15ccd0 2024-04-26T21:37:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.959+0000 7f3b30eed640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1806c920 0x7f3b1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.960+0000 7f3b30eed640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1806c920 0x7f3b1806ede0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c001eb0 tx=0x7f3b1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:34.963+0000 7f3b297fa640 1 -- 172.21.15.40:0/3841408512 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b2407b770 con 0x7f3b2c15ccd0 2024-04-26T21:37:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.148+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b2c0cb2f0 con 0x7f3b2c15ccd0 2024-04-26T21:37:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.149+0000 7f3b297fa640 1 -- 172.21.15.40:0/3841408512 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f3b2407b070 con 0x7f3b2c15ccd0 2024-04-26T21:37:35.149 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:37:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1806c920 msgr2=0x7f3b1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1806c920 0x7f3b1806ede0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c001eb0 tx=0x7f3b1c009040 comp rx=0 tx=0).stop 2024-04-26T21:37:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b2c15ccd0 msgr2=0x7f3b2c159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b2c15ccd0 0x7f3b2c159710 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009f10 tx=0x7f3b24004a40 comp rx=0 tx=0).stop 2024-04-26T21:37:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 shutdown_connections 2024-04-26T21:37:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1806c920 0x7f3b1806ede0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b2c15ccd0 0x7f3b2c159710 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b2c15f170 0x7f3b2c15c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 --2- 172.21.15.40:0/3841408512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b2c0d3bc0 0x7f3b2c1522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 >> 172.21.15.40:0/3841408512 conn(0x7f3b2c0d4f60 msgr2=0x7f3b2c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 shutdown_connections 2024-04-26T21:37:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.151+0000 7f3b33151640 1 -- 172.21.15.40:0/3841408512 wait complete. 2024-04-26T21:37:35.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:37:35.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.262+0000 7fbda2df7640 1 Processor -- start 2024-04-26T21:37:35.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.262+0000 7fbda2df7640 1 -- start start 2024-04-26T21:37:35.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbda2df7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbd9c06fc40 0x7fbd9c073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbda2df7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c156840 con 0x7fbd9c06f790 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbda2df7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c0cbf20 con 0x7fbd9c156460 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbda2df7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c0cc200 con 0x7fbd9c06fc40 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbda1394640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbd9c06fc40 0x7fbd9c073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbd9bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbd9c06f790 0x7fbd9c072c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50848/0 (socket says 172.21.15.40:50848) 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbd9bfff640 1 -- 172.21.15.40:0/3288163536 learned_addr learned my addr 172.21.15.40:0/3288163536 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbda1394640 1 -- 172.21.15.40:0/3288163536 >> v1:172.21.15.184:6789/0 conn(0x7fbd9c156460 legacy=0x7fbd9c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbda1394640 1 -- 172.21.15.40:0/3288163536 >> v1:172.21.15.40:6789/0 conn(0x7fbd9c06f790 legacy=0x7fbd9c072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbd9b7fe640 1 -- 172.21.15.40:0/3288163536 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3102294211 0 0) 0x7fbd9c156840 con 0x7fbd9c06f790 2024-04-26T21:37:35.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.263+0000 7fbda1394640 1 -- 172.21.15.40:0/3288163536 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c0cc4e0 con 0x7fbd9c06fc40 2024-04-26T21:37:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.264+0000 7fbda1394640 1 --2- 172.21.15.40:0/3288163536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c06fc40 0x7fbd9c073360 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009d90 tx=0x7fbd9402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a8cd1f2dc53229 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.264+0000 7fbd9b7fe640 1 -- 172.21.15.40:0/3288163536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94007740 con 0x7fbd9c06fc40 2024-04-26T21:37:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.264+0000 7fbd9b7fe640 1 -- 172.21.15.40:0/3288163536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd940078e0 con 0x7fbd9c06fc40 2024-04-26T21:37:35.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.264+0000 7fbd9b7fe640 1 -- 172.21.15.40:0/3288163536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94007cb0 con 0x7fbd9c06fc40 2024-04-26T21:37:35.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.265+0000 7fbda2df7640 1 -- 172.21.15.40:0/3288163536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c06fc40 msgr2=0x7fbd9c073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.265+0000 7fbda2df7640 1 --2- 172.21.15.40:0/3288163536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c06fc40 0x7fbd9c073360 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009d90 tx=0x7fbd9402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.265+0000 7fbda2df7640 1 -- 172.21.15.40:0/3288163536 shutdown_connections 2024-04-26T21:37:35.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.265+0000 7fbda2df7640 1 --2- 172.21.15.40:0/3288163536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c06fc40 0x7fbd9c073360 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.265+0000 7fbda2df7640 1 -- 172.21.15.40:0/3288163536 >> 172.21.15.40:0/3288163536 conn(0x7fbd9c0d0be0 msgr2=0x7fbd9c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:35.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.265+0000 7fbda2df7640 1 -- 172.21.15.40:0/3288163536 shutdown_connections 2024-04-26T21:37:35.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.265+0000 7fbda2df7640 1 -- 172.21.15.40:0/3288163536 wait complete. 2024-04-26T21:37:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.266+0000 7fbda2df7640 1 Processor -- start 2024-04-26T21:37:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.266+0000 7fbda2df7640 1 -- start start 2024-04-26T21:37:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda2df7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c06f790 0x7fbd9c14f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda2df7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd9c06fc40 0x7fbd9c1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda2df7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c156460 0x7fbd9c1f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda2df7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c070050 con 0x7fbd9c156460 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda2df7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c072700 con 0x7fbd9c06f790 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda2df7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c0cbbe0 con 0x7fbd9c06fc40 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda1394640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c156460 0x7fbd9c1f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda1394640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c156460 0x7fbd9c1f2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47482/0 (socket says 172.21.15.40:47482) 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda1394640 1 -- 172.21.15.40:0/1010960642 learned_addr learned my addr 172.21.15.40:0/1010960642 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbd9bfff640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd9c06fc40 0x7fbd9c1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda0b93640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c06f790 0x7fbd9c14f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda1394640 1 -- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd9c06fc40 msgr2=0x7fbd9c1f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda1394640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd9c06fc40 0x7fbd9c1f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda1394640 1 -- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c06f790 msgr2=0x7fbd9c14f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda1394640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c06f790 0x7fbd9c14f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda1394640 1 -- 172.21.15.40:0/1010960642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c1f2f10 con 0x7fbd9c156460 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbda0b93640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c06f790 0x7fbd9c14f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.267+0000 7fbd9bfff640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd9c06fc40 0x7fbd9c1f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:35.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.268+0000 7fbda1394640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c156460 0x7fbd9c1f2600 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbd9402ac90 tx=0x7fbd94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:35.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.268+0000 7fbd997fa640 1 -- 172.21.15.40:0/1010960642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94040070 con 0x7fbd9c156460 2024-04-26T21:37:35.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.268+0000 7fbd997fa640 1 -- 172.21.15.40:0/1010960642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94031c90 con 0x7fbd9c156460 2024-04-26T21:37:35.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.268+0000 7fbd997fa640 1 -- 172.21.15.40:0/1010960642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94045680 con 0x7fbd9c156460 2024-04-26T21:37:35.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.268+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c1f31f0 con 0x7fbd9c156460 2024-04-26T21:37:35.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.268+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c1f5cc0 con 0x7fbd9c156460 2024-04-26T21:37:35.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.269+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c0694f0 con 0x7fbd9c156460 2024-04-26T21:37:35.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.269+0000 7fbd997fa640 1 -- 172.21.15.40:0/1010960642 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd94039040 con 0x7fbd9c156460 2024-04-26T21:37:35.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.270+0000 7fbd997fa640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8806c8f0 0x7fbd8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.270+0000 7fbd997fa640 1 -- 172.21.15.40:0/1010960642 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7fbd940ad340 con 0x7fbd9c156460 2024-04-26T21:37:35.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.270+0000 7fbda0b93640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8806c8f0 0x7fbd8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.270+0000 7fbda0b93640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8806c8f0 0x7fbd8806edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c006fd0 tx=0x7fbd8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:35.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.274+0000 7fbd997fa640 1 -- 172.21.15.40:0/1010960642 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd9407ba80 con 0x7fbd9c156460 2024-04-26T21:37:35.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.463+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbd9c06ce60 con 0x7fbd9c156460 2024-04-26T21:37:35.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.464+0000 7fbd997fa640 1 -- 172.21.15.40:0/1010960642 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) ==== 84+0+65 (secure 0 0 0) 0x7fbd9407b380 con 0x7fbd9c156460 2024-04-26T21:37:35.464 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:35.464 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:37:35.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:37:35.464 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:37:35.464 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:37:35.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:37:35.464 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:37:35.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8806c8f0 msgr2=0x7fbd8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8806c8f0 0x7fbd8806edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c006fd0 tx=0x7fbd8c008040 comp rx=0 tx=0).stop 2024-04-26T21:37:35.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c156460 msgr2=0x7fbd9c1f2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c156460 0x7fbd9c1f2600 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbd9402ac90 tx=0x7fbd94004a40 comp rx=0 tx=0).stop 2024-04-26T21:37:35.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 shutdown_connections 2024-04-26T21:37:35.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8806c8f0 0x7fbd8806edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd9c156460 0x7fbd9c1f2600 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd9c06fc40 0x7fbd9c1f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 --2- 172.21.15.40:0/1010960642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd9c06f790 0x7fbd9c14f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 >> 172.21.15.40:0/1010960642 conn(0x7fbd9c0d0be0 msgr2=0x7fbd9c0b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:35.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 shutdown_connections 2024-04-26T21:37:35.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.466+0000 7fbda2df7640 1 -- 172.21.15.40:0/1010960642 wait complete. 2024-04-26T21:37:35.477 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:37:35.477 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:37:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.582+0000 7f36a4a09640 1 Processor -- start 2024-04-26T21:37:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.582+0000 7f36a4a09640 1 -- start start 2024-04-26T21:37:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f36a4a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a01606c0 0x7f36a015f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f36a4a09640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36a015fc60 0x7f36a00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f36a4a09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a0163160 con 0x7f36a015f340 2024-04-26T21:37:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f36a4a09640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f36a0162f10 con 0x7f36a015fc60 2024-04-26T21:37:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f36a4a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36a0163d70 con 0x7f36a01606c0 2024-04-26T21:37:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f36a015f340 0x7f36a00d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50862/0 (socket says 172.21.15.40:50862) 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369e59c640 1 -- 172.21.15.40:0/350416538 learned_addr learned my addr 172.21.15.40:0/350416538 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162953204 0 0) 0x7f36a0163160 con 0x7f36a015f340 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369ed9d640 1 --2- 172.21.15.40:0/350416538 >> v2:172.21.15.184:3300/0 conn(0x7f36a015fc60 0x7f36a00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f368c003520 con 0x7f36a015f340 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f3697fff640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a01606c0 0x7f36a015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1004698367 0 0) 0x7f368c003520 con 0x7f36a015f340 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a01606c0 msgr2=0x7f36a015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a01606c0 0x7f36a015f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 >> v2:172.21.15.184:3300/0 conn(0x7f36a015fc60 msgr2=0x7f36a00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 --2- 172.21.15.40:0/350416538 >> v2:172.21.15.184:3300/0 conn(0x7f36a015fc60 0x7f36a00cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a0163440 con 0x7f36a015f340 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.583+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f3690001670 con 0x7f36a015f340 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 >> v1:172.21.15.40:6789/0 conn(0x7f36a015f340 legacy=0x7f36a00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369dd9b640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f368c0039b0 0x7f368c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369dd9b640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a01606c0 0x7f368c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369dd9b640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f368c004d40 0x7f368c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0162f10 con 0x7f368c0039b0 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0163d70 con 0x7f368c004d40 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f368c00b9c0 con 0x7f36a01606c0 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369ed9d640 1 --2- 172.21.15.40:0/350416538 >> v2:172.21.15.184:3300/0 conn(0x7f36a015fc60 0x7f36a00cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369e59c640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f368c0039b0 0x7f368c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f3697fff640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a01606c0 0x7f368c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369ed9d640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f368c004d40 0x7f368c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369e59c640 1 -- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a01606c0 msgr2=0x7f368c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369e59c640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a01606c0 0x7f368c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369e59c640 1 -- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f368c004d40 msgr2=0x7f368c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369e59c640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f368c004d40 0x7f368c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.584+0000 7f369e59c640 1 -- 172.21.15.40:0/350416538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c003520 con 0x7f368c0039b0 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f369e59c640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f368c0039b0 0x7f368c003dd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f369000c270 tx=0x7f368c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620a02f1f6681b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690007d60 con 0x7f368c0039b0 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3690005ad0 con 0x7f368c0039b0 2024-04-26T21:37:35.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f369dd9b640 1 -- 172.21.15.40:0/350416538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690007070 con 0x7f368c0039b0 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f36a4a09640 1 -- 172.21.15.40:0/350416538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f368c0039b0 msgr2=0x7f368c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f36a4a09640 1 --2- 172.21.15.40:0/350416538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f368c0039b0 0x7f368c003dd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f369000c270 tx=0x7f368c005830 comp rx=0 tx=0).stop 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f369ed9d640 1 -- 172.21.15.40:0/350416538 reap_dead start 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f36a4a09640 1 -- 172.21.15.40:0/350416538 shutdown_connections 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f36a4a09640 1 -- 172.21.15.40:0/350416538 >> 172.21.15.40:0/350416538 conn(0x7f36a009ba50 msgr2=0x7f36a009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.585+0000 7f36a4a09640 1 -- 172.21.15.40:0/350416538 shutdown_connections 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.586+0000 7f36a4a09640 1 -- 172.21.15.40:0/350416538 wait complete. 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.587+0000 7f36a4a09640 1 Processor -- start 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.587+0000 7f36a4a09640 1 -- start start 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f36a4a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f368c0039b0 0x7f36a0153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f36a4a09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0155fd0 0x7f36a0156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f36a4a09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a0157df0 0x7f36a0158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f36a4a09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01600d0 con 0x7f36a0155fd0 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f36a4a09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0163a00 con 0x7f36a0157df0 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f36a4a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36a0160250 con 0x7f368c0039b0 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0155fd0 0x7f36a0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0155fd0 0x7f36a0156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47504/0 (socket says 172.21.15.40:47504) 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 -- 172.21.15.40:0/1115582524 learned_addr learned my addr 172.21.15.40:0/1115582524 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f369e59c640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f368c0039b0 0x7f36a0153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f369ed9d640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a0157df0 0x7f36a0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 -- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f368c0039b0 msgr2=0x7f36a0153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f368c0039b0 0x7f36a0153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 -- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a0157df0 msgr2=0x7f36a0158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a0157df0 0x7f36a0158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 -- 172.21.15.40:0/1115582524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a0156d30 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3697fff640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0155fd0 0x7f36a0156430 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f36880119a0 tx=0x7f3688011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3696ffd640 1 -- 172.21.15.40:0/1115582524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368800faf0 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3696ffd640 1 -- 172.21.15.40:0/1115582524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368800fc90 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a0157010 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f3696ffd640 1 -- 172.21.15.40:0/1115582524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688014440 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.588+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01578d0 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.589+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a00713e0 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.590+0000 7f3696ffd640 1 -- 172.21.15.40:0/1115582524 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3688012040 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.590+0000 7f3696ffd640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368006c8f0 0x7f368006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.590+0000 7f3696ffd640 1 -- 172.21.15.40:0/1115582524 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f368808bff0 con 0x7f36a0155fd0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.590+0000 7f369e59c640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368006c8f0 0x7f368006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:35.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.591+0000 7f369e59c640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368006c8f0 0x7f368006edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3690033000 tx=0x7f369002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:35.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.595+0000 7f3696ffd640 1 -- 172.21.15.40:0/1115582524 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36a00713e0 con 0x7f36a0155fd0 2024-04-26T21:37:35.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:35.780+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f36a0157530 con 0x7f36a0155fd0 2024-04-26T21:37:36.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.128+0000 7f3696ffd640 1 -- 172.21.15.40:0/1115582524 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) ==== 138+0+0 (secure 0 0 0) 0x7f3688019030 con 0x7f36a0155fd0 2024-04-26T21:37:36.129 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:37:36.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368006c8f0 msgr2=0x7f368006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368006c8f0 0x7f368006edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3690033000 tx=0x7f369002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0155fd0 msgr2=0x7f36a0156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0155fd0 0x7f36a0156430 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f36880119a0 tx=0x7f3688011e60 comp rx=0 tx=0).stop 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 shutdown_connections 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368006c8f0 0x7f368006edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a0157df0 0x7f36a0158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0155fd0 0x7f36a0156430 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 --2- 172.21.15.40:0/1115582524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f368c0039b0 0x7f36a0153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.132+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 >> 172.21.15.40:0/1115582524 conn(0x7f36a009ba50 msgr2=0x7f36a015a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.133+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 shutdown_connections 2024-04-26T21:37:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.133+0000 7f36a4a09640 1 -- 172.21.15.40:0/1115582524 wait complete. 2024-04-26T21:37:36.148 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:37:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.258+0000 7f2a9470a640 1 Processor -- start 2024-04-26T21:37:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.258+0000 7f2a9470a640 1 -- start start 2024-04-26T21:37:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a9470a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2a8c0b0380 0x7f2a8c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a9470a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a8c0b10b0 0x7f2a8c06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a9470a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a8c0cfd80 con 0x7f2a8c06c780 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a9470a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2a8c06cbd0 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a9470a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2a8c0af8e0 con 0x7f2a8c0b0380 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a91ca5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a8c0b10b0 0x7f2a8c06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a91ca5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a8c0b10b0 0x7f2a8c06c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47518/0 (socket says 172.21.15.40:47518) 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a91ca5640 1 -- 172.21.15.40:0/1494007319 learned_addr learned my addr 172.21.15.40:0/1494007319 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a924a6640 1 --2- 172.21.15.40:0/1494007319 >> v2:172.21.15.184:3300/0 conn(0x7f2a8c0b0380 0x7f2a8c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a91ca5640 1 -- 172.21.15.40:0/1494007319 >> v2:172.21.15.184:3300/0 conn(0x7f2a8c0b0380 msgr2=0x7f2a8c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a91ca5640 1 --2- 172.21.15.40:0/1494007319 >> v2:172.21.15.184:3300/0 conn(0x7f2a8c0b0380 0x7f2a8c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a91ca5640 1 -- 172.21.15.40:0/1494007319 >> v1:172.21.15.184:6789/0 conn(0x7f2a8c06c780 legacy=0x7f2a8c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.259+0000 7f2a91ca5640 1 -- 172.21.15.40:0/1494007319 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c0d0020 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.260+0000 7f2a91ca5640 1 --2- 172.21.15.40:0/1494007319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c06c240 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0b48c0 tx=0x7f2a8002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fc3648b2b18602 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.260+0000 7f2a914a4640 1 -- 172.21.15.40:0/1494007319 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80037070 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.260+0000 7f2a914a4640 1 -- 172.21.15.40:0/1494007319 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a80032440 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.260+0000 7f2a914a4640 1 -- 172.21.15.40:0/1494007319 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80032810 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.261+0000 7f2a9470a640 1 -- 172.21.15.40:0/1494007319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 msgr2=0x7f2a8c06c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.261+0000 7f2a9470a640 1 --2- 172.21.15.40:0/1494007319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c06c240 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0b48c0 tx=0x7f2a8002a160 comp rx=0 tx=0).stop 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.261+0000 7f2a9470a640 1 -- 172.21.15.40:0/1494007319 shutdown_connections 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.261+0000 7f2a9470a640 1 --2- 172.21.15.40:0/1494007319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c06c240 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.261+0000 7f2a9470a640 1 --2- 172.21.15.40:0/1494007319 >> v2:172.21.15.184:3300/0 conn(0x7f2a8c0b0380 0x7f2a8c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.261+0000 7f2a9470a640 1 -- 172.21.15.40:0/1494007319 >> 172.21.15.40:0/1494007319 conn(0x7f2a8c0d1cf0 msgr2=0x7f2a8c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.261+0000 7f2a9470a640 1 -- 172.21.15.40:0/1494007319 shutdown_connections 2024-04-26T21:37:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.261+0000 7f2a9470a640 1 -- 172.21.15.40:0/1494007319 wait complete. 2024-04-26T21:37:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.262+0000 7f2a9470a640 1 Processor -- start 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.262+0000 7f2a9470a640 1 -- start start 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a9470a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a8c06c780 0x7f2a8c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a9470a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a8c0b0380 0x7f2a8c0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a9470a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a9470a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c0d1980 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a9470a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c0d05e0 con 0x7f2a8c06c780 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a9470a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8c0b0760 con 0x7f2a8c0b0380 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a92ca7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a92ca7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c0b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47530/0 (socket says 172.21.15.40:47530) 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a92ca7640 1 -- 172.21.15.40:0/693796655 learned_addr learned my addr 172.21.15.40:0/693796655 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a91ca5640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a8c0b0380 0x7f2a8c0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a92ca7640 1 -- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a8c0b0380 msgr2=0x7f2a8c0b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a92ca7640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a8c0b0380 0x7f2a8c0b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a924a6640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a8c06c780 0x7f2a8c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a92ca7640 1 -- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a8c06c780 msgr2=0x7f2a8c0c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a92ca7640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a8c06c780 0x7f2a8c0c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.263+0000 7f2a92ca7640 1 -- 172.21.15.40:0/693796655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c0c6eb0 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.264+0000 7f2a92ca7640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c0b1e60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b970 tx=0x7f2a7c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.264+0000 7f2a8affd640 1 -- 172.21.15.40:0/693796655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c019070 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.264+0000 7f2a8affd640 1 -- 172.21.15.40:0/693796655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a7c00cb40 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.264+0000 7f2a8affd640 1 -- 172.21.15.40:0/693796655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c014440 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.264+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c0c7190 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.264+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c0c7750 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.265+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c0694f0 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.265+0000 7f2a8affd640 1 -- 172.21.15.40:0/693796655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a7c01c070 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.266+0000 7f2a8affd640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2a7806c8f0 0x7f2a7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.266+0000 7f2a8affd640 1 -- 172.21.15.40:0/693796655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5619+0+0 (secure 0 0 0) 0x7f2a7c08b850 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.266+0000 7f2a924a6640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2a7806c8f0 0x7f2a7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.267+0000 7f2a924a6640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2a7806c8f0 0x7f2a7806edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a84006fd0 tx=0x7f2a84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.271+0000 7f2a8affd640 1 -- 172.21.15.40:0/693796655 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a7c059c40 con 0x7f2a8c0b10b0 2024-04-26T21:37:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:36.456+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2a8c070fe0 con 0x7f2a8c0b10b0 2024-04-26T21:37:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.130+0000 7f2a8affd640 1 -- 172.21.15.40:0/693796655 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) ==== 170+0+0 (secure 0 0 0) 0x7f2a7c059540 con 0x7f2a8c0b10b0 2024-04-26T21:37:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:37:37.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2a7806c8f0 msgr2=0x7f2a7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2a7806c8f0 0x7f2a7806edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a84006fd0 tx=0x7f2a84008040 comp rx=0 tx=0).stop 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 msgr2=0x7f2a8c0b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c0b1e60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b970 tx=0x7f2a7c00be30 comp rx=0 tx=0).stop 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 shutdown_connections 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2a7806c8f0 0x7f2a7806edb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a8c0b10b0 0x7f2a8c0b1e60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a8c0b0380 0x7f2a8c0b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 --2- 172.21.15.40:0/693796655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a8c06c780 0x7f2a8c0c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 >> 172.21.15.40:0/693796655 conn(0x7f2a8c0d1cf0 msgr2=0x7f2a8c0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 shutdown_connections 2024-04-26T21:37:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.133+0000 7f2a9470a640 1 -- 172.21.15.40:0/693796655 wait complete. 2024-04-26T21:37:37.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:37:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.267+0000 7fc2328b0640 1 Processor -- start 2024-04-26T21:37:37.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.267+0000 7fc2328b0640 1 -- start start 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc2328b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc22c15f170 0x7fc22c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc2328b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c15f550 0x7fc22c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc2328b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22c0d1040 con 0x7fc22c0d34b0 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc2328b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc22c0c9430 con 0x7fc22c15f170 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc2328b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c0c9680 con 0x7fc22c15f550 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc22bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc22c15f170 0x7fc22c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc22b7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c15f550 0x7fc22c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc22bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc22c15f170 0x7fc22c0d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:33996/0 (socket says 172.21.15.40:33996) 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc22bfff640 1 -- 172.21.15.40:0/2634283303 learned_addr learned my addr 172.21.15.40:0/2634283303 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.268+0000 7fc230e4d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fc22c0d34b0 0x7fc22c0cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39500/0 (socket says 172.21.15.40:39500) 2024-04-26T21:37:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.269+0000 7fc22bfff640 1 -- 172.21.15.40:0/2634283303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c15f550 msgr2=0x7fc22c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.269+0000 7fc22bfff640 1 --2- 172.21.15.40:0/2634283303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c15f550 0x7fc22c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.269+0000 7fc22bfff640 1 -- 172.21.15.40:0/2634283303 >> v1:172.21.15.184:6789/0 conn(0x7fc22c0d34b0 legacy=0x7fc22c0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.269+0000 7fc22bfff640 1 -- 172.21.15.40:0/2634283303 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c0c9960 con 0x7fc22c15f170 2024-04-26T21:37:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.269+0000 7fc22b7fe640 1 --2- 172.21.15.40:0/2634283303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c15f550 0x7fc22c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc22bfff640 1 --2- 172.21.15.40:0/2634283303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c15f170 0x7fc22c0d5a80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc214009870 tx=0x7fc21402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2c056ce183c9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc22affd640 1 -- 172.21.15.40:0/2634283303 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214037070 con 0x7fc22c15f170 2024-04-26T21:37:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc22affd640 1 -- 172.21.15.40:0/2634283303 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc214032440 con 0x7fc22c15f170 2024-04-26T21:37:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc22affd640 1 -- 172.21.15.40:0/2634283303 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214032810 con 0x7fc22c15f170 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc2328b0640 1 -- 172.21.15.40:0/2634283303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c15f170 msgr2=0x7fc22c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2634283303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c15f170 0x7fc22c0d5a80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc214009870 tx=0x7fc21402a270 comp rx=0 tx=0).stop 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc2328b0640 1 -- 172.21.15.40:0/2634283303 shutdown_connections 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2634283303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c15f550 0x7fc22c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2634283303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c15f170 0x7fc22c0d5a80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc2328b0640 1 -- 172.21.15.40:0/2634283303 >> 172.21.15.40:0/2634283303 conn(0x7fc22c0d4f60 msgr2=0x7fc22c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc2328b0640 1 -- 172.21.15.40:0/2634283303 shutdown_connections 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.270+0000 7fc2328b0640 1 -- 172.21.15.40:0/2634283303 wait complete. 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.271+0000 7fc2328b0640 1 Processor -- start 2024-04-26T21:37:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.272+0000 7fc2328b0640 1 -- start start 2024-04-26T21:37:37.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.272+0000 7fc2328b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c0d34b0 0x7fc22c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:37.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.272+0000 7fc2328b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc22c15f170 0x7fc22c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:37.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.272+0000 7fc2328b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0b9b90 0x7fc22c0ba020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:37.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.272+0000 7fc2328b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c0cb5d0 con 0x7fc22c15f170 2024-04-26T21:37:37.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.272+0000 7fc2328b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c0cb450 con 0x7fc22c0b9b90 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.272+0000 7fc2328b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c0cb750 con 0x7fc22c0d34b0 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.272+0000 7fc22b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc22c15f170 0x7fc22c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc22c15f170 0x7fc22c1522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47544/0 (socket says 172.21.15.40:47544) 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22b7fe640 1 -- 172.21.15.40:0/2969281425 learned_addr learned my addr 172.21.15.40:0/2969281425 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22bfff640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c0d34b0 0x7fc22c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc230e4d640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0b9b90 0x7fc22c0ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22b7fe640 1 -- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c0d34b0 msgr2=0x7fc22c159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22b7fe640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c0d34b0 0x7fc22c159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22b7fe640 1 -- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0b9b90 msgr2=0x7fc22c0ba020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22b7fe640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0b9b90 0x7fc22c0ba020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22b7fe640 1 -- 172.21.15.40:0/2969281425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c158df0 con 0x7fc22c15f170 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc22b7fe640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc22c15f170 0x7fc22c1522a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00e960 tx=0x7fc21c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc228ff9640 1 -- 172.21.15.40:0/2969281425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c017070 con 0x7fc22c15f170 2024-04-26T21:37:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc228ff9640 1 -- 172.21.15.40:0/2969281425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c00ca00 con 0x7fc22c15f170 2024-04-26T21:37:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c1590d0 con 0x7fc22c15f170 2024-04-26T21:37:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc228ff9640 1 -- 172.21.15.40:0/2969281425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00cdd0 con 0x7fc22c15f170 2024-04-26T21:37:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.273+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c159690 con 0x7fc22c15f170 2024-04-26T21:37:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.274+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22c0694f0 con 0x7fc22c15f170 2024-04-26T21:37:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.275+0000 7fc228ff9640 1 -- 172.21.15.40:0/2969281425 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc21c01a070 con 0x7fc22c15f170 2024-04-26T21:37:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.275+0000 7fc228ff9640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc20c06c8f0 0x7fc20c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.275+0000 7fc228ff9640 1 -- 172.21.15.40:0/2969281425 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6001+0+0 (secure 0 0 0) 0x7fc21c08bec0 con 0x7fc22c15f170 2024-04-26T21:37:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.276+0000 7fc22bfff640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc20c06c8f0 0x7fc20c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.276+0000 7fc22bfff640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc20c06c8f0 0x7fc20c06edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc214034000 tx=0x7fc21402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:37.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.280+0000 7fc228ff9640 1 -- 172.21.15.40:0/2969281425 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc21c059af0 con 0x7fc22c15f170 2024-04-26T21:37:37.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:37.509+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc22c1599d0 con 0x7fc22c15f170 2024-04-26T21:37:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.143+0000 7fc228ff9640 1 -- 172.21.15.40:0/2969281425 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v28) ==== 172+0+0 (secure 0 0 0) 0x7fc21c05ddf0 con 0x7fc22c15f170 2024-04-26T21:37:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 7 and data pool 8 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc20c06c8f0 msgr2=0x7fc20c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc20c06c8f0 0x7fc20c06edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc214034000 tx=0x7fc21402e000 comp rx=0 tx=0).stop 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc22c15f170 msgr2=0x7fc22c1522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc22c15f170 0x7fc22c1522a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00e960 tx=0x7fc21c00ee20 comp rx=0 tx=0).stop 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 shutdown_connections 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc20c06c8f0 0x7fc20c06edb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0b9b90 0x7fc22c0ba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc22c15f170 0x7fc22c1522a0 secure :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00e960 tx=0x7fc21c00ee20 comp rx=0 tx=0).stop 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 --2- 172.21.15.40:0/2969281425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c0d34b0 0x7fc22c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.149+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 >> 172.21.15.40:0/2969281425 conn(0x7fc22c0d4f60 msgr2=0x7fc22c06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:38.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.154+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 shutdown_connections 2024-04-26T21:37:38.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.159+0000 7fc2328b0640 1 -- 172.21.15.40:0/2969281425 wait complete. 2024-04-26T21:37:38.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:38.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.294+0000 7f285dafc640 1 Processor -- start 2024-04-26T21:37:38.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.295+0000 7f285dafc640 1 -- start start 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.295+0000 7f285dafc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f28580d06f0 0x7f28580d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.295+0000 7f285dafc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28580b2850 0x7f28580b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.295+0000 7f285dafc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28580d0ba0 con 0x7f285815f3a0 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.295+0000 7f285dafc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f28580d0e20 con 0x7f28580b2850 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.295+0000 7f285dafc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f28580b1d70 con 0x7f28580d06f0 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.295+0000 7f2857fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28580b2850 0x7f28580b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.296+0000 7f28577fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f285815f3a0 0x7f28580d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50870/0 (socket says 172.21.15.40:50870) 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.296+0000 7f28577fe640 1 -- 172.21.15.40:0/1600901462 learned_addr learned my addr 172.21.15.40:0/1600901462 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.296+0000 7f2856ffd640 1 --2- 172.21.15.40:0/1600901462 >> v2:172.21.15.184:3300/0 conn(0x7f28580d06f0 0x7f28580d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.296+0000 7f2857fff640 1 -- 172.21.15.40:0/1600901462 >> v2:172.21.15.184:3300/0 conn(0x7f28580d06f0 msgr2=0x7f28580d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:38.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.296+0000 7f2857fff640 1 --2- 172.21.15.40:0/1600901462 >> v2:172.21.15.184:3300/0 conn(0x7f28580d06f0 0x7f28580d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.296+0000 7f2857fff640 1 -- 172.21.15.40:0/1600901462 >> v1:172.21.15.40:6789/0 conn(0x7f285815f3a0 legacy=0x7f28580d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.296+0000 7f2857fff640 1 -- 172.21.15.40:0/1600901462 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28580b2050 con 0x7f28580b2850 2024-04-26T21:37:38.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.296+0000 7f2856ffd640 1 --2- 172.21.15.40:0/1600901462 >> v2:172.21.15.184:3300/0 conn(0x7f28580d06f0 0x7f28580d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:38.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f2857fff640 1 --2- 172.21.15.40:0/1600901462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580b2850 0x7f28580b2c80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f28580b7b80 tx=0x7f285002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b00e2b7428251f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f28567fc640 1 -- 172.21.15.40:0/1600901462 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850007740 con 0x7f28580b2850 2024-04-26T21:37:38.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f28567fc640 1 -- 172.21.15.40:0/1600901462 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28500078e0 con 0x7f28580b2850 2024-04-26T21:37:38.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f28567fc640 1 -- 172.21.15.40:0/1600901462 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850007cb0 con 0x7f28580b2850 2024-04-26T21:37:38.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f285dafc640 1 -- 172.21.15.40:0/1600901462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580b2850 msgr2=0x7f28580b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f285dafc640 1 --2- 172.21.15.40:0/1600901462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580b2850 0x7f28580b2c80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f28580b7b80 tx=0x7f285002a680 comp rx=0 tx=0).stop 2024-04-26T21:37:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f285dafc640 1 -- 172.21.15.40:0/1600901462 shutdown_connections 2024-04-26T21:37:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f285dafc640 1 --2- 172.21.15.40:0/1600901462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580b2850 0x7f28580b2c80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f285dafc640 1 --2- 172.21.15.40:0/1600901462 >> v2:172.21.15.184:3300/0 conn(0x7f28580d06f0 0x7f28580d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f285dafc640 1 -- 172.21.15.40:0/1600901462 >> 172.21.15.40:0/1600901462 conn(0x7f28580d4f90 msgr2=0x7f28580d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f285dafc640 1 -- 172.21.15.40:0/1600901462 shutdown_connections 2024-04-26T21:37:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.297+0000 7f285dafc640 1 -- 172.21.15.40:0/1600901462 wait complete. 2024-04-26T21:37:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.298+0000 7f285dafc640 1 Processor -- start 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.299+0000 7f285dafc640 1 -- start start 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.299+0000 7f285dafc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28580b2850 0x7f285815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.299+0000 7f285dafc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d06f0 0x7f2858153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.299+0000 7f285dafc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285815f3a0 0x7f28581f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.299+0000 7f285dafc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f285815f900 con 0x7f28580d06f0 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.299+0000 7f285dafc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f285815f780 con 0x7f28580b2850 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.299+0000 7f285dafc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28580d23c0 con 0x7f285815f3a0 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.299+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d06f0 0x7f2858153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d06f0 0x7f2858153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47564/0 (socket says 172.21.15.40:47564) 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2856ffd640 1 -- 172.21.15.40:0/3667413495 learned_addr learned my addr 172.21.15.40:0/3667413495 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f28577fe640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28580b2850 0x7f285815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2857fff640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285815f3a0 0x7f28581f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2856ffd640 1 -- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285815f3a0 msgr2=0x7f28581f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2856ffd640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285815f3a0 0x7f28581f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2856ffd640 1 -- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28580b2850 msgr2=0x7f285815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2856ffd640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28580b2850 0x7f285815fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2856ffd640 1 -- 172.21.15.40:0/3667413495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285815ffc0 con 0x7f28580d06f0 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f2856ffd640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d06f0 0x7f2858153250 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c820 tx=0x7f284c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f283ffff640 1 -- 172.21.15.40:0/3667413495 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c015070 con 0x7f28580d06f0 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f283ffff640 1 -- 172.21.15.40:0/3667413495 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c007a30 con 0x7f28580d06f0 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2858160260 con 0x7f28580d06f0 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f283ffff640 1 -- 172.21.15.40:0/3667413495 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c007e00 con 0x7f28580d06f0 2024-04-26T21:37:38.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.300+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2858160820 con 0x7f28580d06f0 2024-04-26T21:37:38.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.301+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28580cb9a0 con 0x7f28580d06f0 2024-04-26T21:37:38.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.302+0000 7f283ffff640 1 -- 172.21.15.40:0/3667413495 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f284c018070 con 0x7f28580d06f0 2024-04-26T21:37:38.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.302+0000 7f283ffff640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f284406c8f0 0x7f284406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.302+0000 7f283ffff640 1 -- 172.21.15.40:0/3667413495 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f284c08bc00 con 0x7f28580d06f0 2024-04-26T21:37:38.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.302+0000 7f28577fe640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f284406c8f0 0x7f284406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.303+0000 7f28577fe640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f284406c8f0 0x7f284406edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2848007fd0 tx=0x7f2848007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.307+0000 7f283ffff640 1 -- 172.21.15.40:0/3667413495 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28580cb9a0 con 0x7f28580d06f0 2024-04-26T21:37:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.492+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28580b6860 con 0x7f28580d06f0 2024-04-26T21:37:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.493+0000 7f283ffff640 1 -- 172.21.15.40:0/3667413495 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16919 (secure 0 0 0) 0x7f284c092080 con 0x7f28580d06f0 2024-04-26T21:37:38.493 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:38.494 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":51,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:37.510485+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:37:35.784073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T21:37:36.460613+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f284406c8f0 msgr2=0x7f284406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f284406c8f0 0x7f284406edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2848007fd0 tx=0x7f2848007e80 comp rx=0 tx=0).stop 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d06f0 msgr2=0x7f2858153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d06f0 0x7f2858153250 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c820 tx=0x7f284c00cc20 comp rx=0 tx=0).stop 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 shutdown_connections 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f284406c8f0 0x7f284406edb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285815f3a0 0x7f28581f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d06f0 0x7f2858153250 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 --2- 172.21.15.40:0/3667413495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28580b2850 0x7f285815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 >> 172.21.15.40:0/3667413495 conn(0x7f28580d4f90 msgr2=0x7f28580caad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 shutdown_connections 2024-04-26T21:37:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.495+0000 7f285dafc640 1 -- 172.21.15.40:0/3667413495 wait complete. 2024-04-26T21:37:38.506 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.607+0000 7f2396bda640 1 Processor -- start 2024-04-26T21:37:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.607+0000 7f2396bda640 1 -- start start 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f2396bda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2390072950 0x7f2390167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f2396bda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f239015d7c0 con 0x7f2390071e60 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f2396bda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f239015daa0 con 0x7f23900cc480 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f2396bda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f239015dd80 con 0x7f2390072950 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f2394976640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2390071e60 0x7f239006f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50872/0 (socket says 172.21.15.40:50872) 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f2394976640 1 -- 172.21.15.40:0/4042160948 learned_addr learned my addr 172.21.15.40:0/4042160948 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 477571212 0 0) 0x7f239015d7c0 con 0x7f2390071e60 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f2395177640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2390072950 0x7f2390167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2380003520 con 0x7f2390071e60 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3244879672 0 0) 0x7f2380003520 con 0x7f2390071e60 2024-04-26T21:37:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2390072950 msgr2=0x7f2390167ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f238f7fe640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2390072950 0x7f2390167ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 >> v1:172.21.15.184:6789/0 conn(0x7f23900cc480 legacy=0x7f2390072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239015e060 con 0x7f2390071e60 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.608+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f2384001670 con 0x7f2390071e60 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 >> v1:172.21.15.40:6789/0 conn(0x7f2390071e60 legacy=0x7f239006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f238f7fe640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039e0 0x7f2380003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f238f7fe640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2390072950 0x7f2380004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f238f7fe640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2380004d40 0x7f23800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f239015dd80 con 0x7f23800039e0 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f238000baf0 con 0x7f2380004d40 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f238000bcf0 con 0x7f2390072950 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f2394976640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039e0 0x7f2380003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f238ffff640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2390072950 0x7f2380004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f2395177640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2380004d40 0x7f23800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f2394976640 1 -- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2390072950 msgr2=0x7f2380004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f2394976640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2390072950 0x7f2380004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f2394976640 1 -- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2380004d40 msgr2=0x7f23800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f2394976640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2380004d40 0x7f23800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f2394976640 1 -- 172.21.15.40:0/4042160948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2380003520 con 0x7f23800039e0 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.609+0000 7f2395177640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2380004d40 0x7f23800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f2394976640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039e0 0x7f2380003e00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f238400c240 tx=0x7f2380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e36a841d410e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384007d10 con 0x7f23800039e0 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2384005ad0 con 0x7f23800039e0 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f238f7fe640 1 -- 172.21.15.40:0/4042160948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384007070 con 0x7f23800039e0 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f2396bda640 1 -- 172.21.15.40:0/4042160948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039e0 msgr2=0x7f2380003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f2396bda640 1 --2- 172.21.15.40:0/4042160948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039e0 0x7f2380003e00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f238400c240 tx=0x7f2380005830 comp rx=0 tx=0).stop 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f2395177640 1 -- 172.21.15.40:0/4042160948 reap_dead start 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f2396bda640 1 -- 172.21.15.40:0/4042160948 shutdown_connections 2024-04-26T21:37:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f2396bda640 1 -- 172.21.15.40:0/4042160948 >> 172.21.15.40:0/4042160948 conn(0x7f23900d0be0 msgr2=0x7f23900d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:38.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f2396bda640 1 -- 172.21.15.40:0/4042160948 shutdown_connections 2024-04-26T21:37:38.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.610+0000 7f2396bda640 1 -- 172.21.15.40:0/4042160948 wait complete. 2024-04-26T21:37:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.612+0000 7f2396bda640 1 Processor -- start 2024-04-26T21:37:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.612+0000 7f2396bda640 1 -- start start 2024-04-26T21:37:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.612+0000 7f2396bda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039e0 0x7f23901f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.612+0000 7f2396bda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23901f7bf0 0x7f23900ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.612+0000 7f2396bda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23900cfb80 0x7f23900d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.612+0000 7f2396bda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f239015d570 con 0x7f23900cfb80 2024-04-26T21:37:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.612+0000 7f2396bda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f239015d3f0 con 0x7f23800039e0 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.612+0000 7f2396bda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f239006fff0 con 0x7f23901f7bf0 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23900cfb80 0x7f23900d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23900cfb80 0x7f23900d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47596/0 (socket says 172.21.15.40:47596) 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 -- 172.21.15.40:0/404765658 learned_addr learned my addr 172.21.15.40:0/404765658 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f238ffff640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23901f7bf0 0x7f23900ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2394976640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039e0 0x7f23901f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 -- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23901f7bf0 msgr2=0x7f23900ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23901f7bf0 0x7f23900ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 -- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039e0 msgr2=0x7f23901f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039e0 0x7f23901f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 -- 172.21.15.40:0/404765658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23900aee10 con 0x7f23900cfb80 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2395177640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23900cfb80 0x7f23900d0000 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f23900712b0 tx=0x7f2388010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f238d7fa640 1 -- 172.21.15.40:0/404765658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388010e60 con 0x7f23900cfb80 2024-04-26T21:37:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f238d7fa640 1 -- 172.21.15.40:0/404765658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2388003990 con 0x7f23900cfb80 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23900af0b0 con 0x7f23900cfb80 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f238d7fa640 1 -- 172.21.15.40:0/404765658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388003d60 con 0x7f23900cfb80 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.613+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23900af670 con 0x7f23900cfb80 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.614+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23900694f0 con 0x7f23900cfb80 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.615+0000 7f238d7fa640 1 -- 172.21.15.40:0/404765658 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2388005d10 con 0x7f23900cfb80 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.615+0000 7f238d7fa640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c920 0x7f237406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.615+0000 7f238d7fa640 1 -- 172.21.15.40:0/404765658 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f238808c200 con 0x7f23900cfb80 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.615+0000 7f2394976640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c920 0x7f237406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.616+0000 7f2394976640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c920 0x7f237406ede0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2384032000 tx=0x7f2384009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.620+0000 7f238d7fa640 1 -- 172.21.15.40:0/404765658 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2388025020 con 0x7f23900cfb80 2024-04-26T21:37:38.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.804+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f239015a040 con 0x7f23900cfb80 2024-04-26T21:37:38.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.805+0000 7f238d7fa640 1 -- 172.21.15.40:0/404765658 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16919 (secure 0 0 0) 0x7f2388059e50 con 0x7f23900cfb80 2024-04-26T21:37:38.806 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:38.806 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":51,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:37.510485+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:37:35.784073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T21:37:36.460613+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.807+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c920 msgr2=0x7f237406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.807+0000 7f2396bda640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c920 0x7f237406ede0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2384032000 tx=0x7f2384009030 comp rx=0 tx=0).stop 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.807+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23900cfb80 msgr2=0x7f23900d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.807+0000 7f2396bda640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23900cfb80 0x7f23900d0000 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f23900712b0 tx=0x7f2388010590 comp rx=0 tx=0).stop 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.808+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 shutdown_connections 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.808+0000 7f2396bda640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c920 0x7f237406ede0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.808+0000 7f2396bda640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23900cfb80 0x7f23900d0000 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.808+0000 7f2396bda640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23901f7bf0 0x7f23900ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.808+0000 7f2396bda640 1 --2- 172.21.15.40:0/404765658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039e0 0x7f23901f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.808+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 >> 172.21.15.40:0/404765658 conn(0x7f23900d0be0 msgr2=0x7f23901592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.808+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 shutdown_connections 2024-04-26T21:37:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.808+0000 7f2396bda640 1 -- 172.21.15.40:0/404765658 wait complete. 2024-04-26T21:37:38.819 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:37:38.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.922+0000 7fca42197640 1 Processor -- start 2024-04-26T21:37:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.923+0000 7fca42197640 1 -- start start 2024-04-26T21:37:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.923+0000 7fca42197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0b40d0 0x7fca3c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.923+0000 7fca42197640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca3c162b30 0x7fca3c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.923+0000 7fca42197640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c0d0a10 con 0x7fca3c0d1a00 2024-04-26T21:37:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.923+0000 7fca42197640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fca3c0d0cf0 con 0x7fca3c162b30 2024-04-26T21:37:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.923+0000 7fca42197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca3c0d0fd0 con 0x7fca3c0b40d0 2024-04-26T21:37:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fca3c0d1a00 0x7fca3c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60452/0 (socket says 172.21.15.40:60452) 2024-04-26T21:37:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3b7fe640 1 -- 172.21.15.40:0/2278491837 learned_addr learned my addr 172.21.15.40:0/2278491837 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca32ffd640 1 --2- 172.21.15.40:0/2278491837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0b40d0 0x7fca3c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3bfff640 1 --2- 172.21.15.40:0/2278491837 >> v2:172.21.15.184:3300/0 conn(0x7fca3c162b30 0x7fca3c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3affd640 1 -- 172.21.15.40:0/2278491837 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 127309221 0 0) 0x7fca3c0d0a10 con 0x7fca3c0d1a00 2024-04-26T21:37:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3affd640 1 -- 172.21.15.40:0/2278491837 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca28003520 con 0x7fca3c0d1a00 2024-04-26T21:37:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3bfff640 1 -- 172.21.15.40:0/2278491837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0b40d0 msgr2=0x7fca3c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3bfff640 1 --2- 172.21.15.40:0/2278491837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0b40d0 0x7fca3c1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3bfff640 1 -- 172.21.15.40:0/2278491837 >> v1:172.21.15.40:6789/0 conn(0x7fca3c0d1a00 legacy=0x7fca3c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.924+0000 7fca3bfff640 1 -- 172.21.15.40:0/2278491837 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c162fa0 con 0x7fca3c162b30 2024-04-26T21:37:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.925+0000 7fca3bfff640 1 --2- 172.21.15.40:0/2278491837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 0x7fca3c0b4b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca34009d90 tx=0x7fca3402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8f6e1a5ccfce45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.925+0000 7fca3affd640 1 -- 172.21.15.40:0/2278491837 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34031600 con 0x7fca3c162b30 2024-04-26T21:37:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.925+0000 7fca3affd640 1 -- 172.21.15.40:0/2278491837 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca340317a0 con 0x7fca3c162b30 2024-04-26T21:37:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.925+0000 7fca3affd640 1 -- 172.21.15.40:0/2278491837 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34031b70 con 0x7fca3c162b30 2024-04-26T21:37:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.925+0000 7fca42197640 1 -- 172.21.15.40:0/2278491837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 msgr2=0x7fca3c0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.925+0000 7fca42197640 1 --2- 172.21.15.40:0/2278491837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 0x7fca3c0b4b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca34009d90 tx=0x7fca3402a5d0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.926+0000 7fca42197640 1 -- 172.21.15.40:0/2278491837 shutdown_connections 2024-04-26T21:37:38.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.926+0000 7fca42197640 1 --2- 172.21.15.40:0/2278491837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 0x7fca3c0b4b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.926+0000 7fca42197640 1 --2- 172.21.15.40:0/2278491837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0b40d0 0x7fca3c1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.926+0000 7fca42197640 1 -- 172.21.15.40:0/2278491837 >> 172.21.15.40:0/2278491837 conn(0x7fca3c0d4fd0 msgr2=0x7fca3c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:38.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.926+0000 7fca42197640 1 -- 172.21.15.40:0/2278491837 shutdown_connections 2024-04-26T21:37:38.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.926+0000 7fca42197640 1 -- 172.21.15.40:0/2278491837 wait complete. 2024-04-26T21:37:38.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.927+0000 7fca42197640 1 Processor -- start 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.927+0000 7fca42197640 1 -- start start 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca42197640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c0b40d0 0x7fca3c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca42197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0d1a00 0x7fca3c0b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca42197640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 0x7fca3c14ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca42197640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c0d2670 con 0x7fca3c0b40d0 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca42197640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c0d24f0 con 0x7fca3c162b30 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca42197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca3c0b5040 con 0x7fca3c0d1a00 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c0b40d0 0x7fca3c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c0b40d0 0x7fca3c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56992/0 (socket says 172.21.15.40:56992) 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 -- 172.21.15.40:0/78734533 learned_addr learned my addr 172.21.15.40:0/78734533 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3bfff640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 0x7fca3c14ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca32ffd640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0d1a00 0x7fca3c0b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 -- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0d1a00 msgr2=0x7fca3c0b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0d1a00 0x7fca3c0b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 -- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 msgr2=0x7fca3c14ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 0x7fca3c14ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 -- 172.21.15.40:0/78734533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c0ba960 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca3b7fe640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c0b40d0 0x7fca3c0c8960 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c430 tx=0x7fca2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca38ff9640 1 -- 172.21.15.40:0/78734533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c017070 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca38ff9640 1 -- 172.21.15.40:0/78734533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2c005ce0 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca38ff9640 1 -- 172.21.15.40:0/78734533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0072f0 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.928+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c0b98c0 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.929+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c0b9e80 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.929+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3c0694f0 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.930+0000 7fca38ff9640 1 -- 172.21.15.40:0/78734533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca2c01a070 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.930+0000 7fca38ff9640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca1c06c8f0 0x7fca1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.931+0000 7fca38ff9640 1 -- 172.21.15.40:0/78734533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fca2c08b1a0 con 0x7fca3c0b40d0 2024-04-26T21:37:38.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.931+0000 7fca32ffd640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca1c06c8f0 0x7fca1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:38.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.931+0000 7fca32ffd640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca1c06c8f0 0x7fca1c06edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0ca670 tx=0x7fca24009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:38.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:38.935+0000 7fca38ff9640 1 -- 172.21.15.40:0/78734533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca2c058d60 con 0x7fca3c0b40d0 2024-04-26T21:37:39.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:39.163+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fca3c0d20c0 con 0x7fca3c0b40d0 2024-04-26T21:37:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.168+0000 7fca38ff9640 1 -- 172.21.15.40:0/78734533 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7fca2c05d060 con 0x7fca3c0b40d0 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca1c06c8f0 msgr2=0x7fca1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca1c06c8f0 0x7fca1c06edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0ca670 tx=0x7fca24009190 comp rx=0 tx=0).stop 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c0b40d0 msgr2=0x7fca3c0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c0b40d0 0x7fca3c0c8960 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c430 tx=0x7fca2c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 shutdown_connections 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca1c06c8f0 0x7fca1c06edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c162b30 0x7fca3c14ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca3c0d1a00 0x7fca3c0b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 --2- 172.21.15.40:0/78734533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c0b40d0 0x7fca3c0c8960 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 >> 172.21.15.40:0/78734533 conn(0x7fca3c0d4fd0 msgr2=0x7fca3c161570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:40.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 shutdown_connections 2024-04-26T21:37:40.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.171+0000 7fca42197640 1 -- 172.21.15.40:0/78734533 wait complete. 2024-04-26T21:37:40.184 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:37:40.184 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:37:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.294+0000 7f604c778640 1 Processor -- start 2024-04-26T21:37:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.294+0000 7f604c778640 1 -- start start 2024-04-26T21:37:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604c778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60440cfde0 0x7f60440d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604c778640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6044070350 0x7f6044070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604c778640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60440d2390 con 0x7f6044071090 2024-04-26T21:37:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604c778640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6044071500 con 0x7f6044070350 2024-04-26T21:37:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604c778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60440afab0 con 0x7f60440cfde0 2024-04-26T21:37:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604a514640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60440cfde0 0x7f60440d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f6049d13640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6044070350 0x7f6044070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604a514640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60440cfde0 0x7f60440d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:54126/0 (socket says 172.21.15.40:54126) 2024-04-26T21:37:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604ad15640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f6044071090 0x7f604415e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:44862/0 (socket says 172.21.15.40:44862) 2024-04-26T21:37:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f604a514640 1 -- 172.21.15.40:0/1428122631 learned_addr learned my addr 172.21.15.40:0/1428122631 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f6049d13640 1 -- 172.21.15.40:0/1428122631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60440cfde0 msgr2=0x7f60440d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f6049d13640 1 --2- 172.21.15.40:0/1428122631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60440cfde0 0x7f60440d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.295+0000 7f6049d13640 1 -- 172.21.15.40:0/1428122631 >> v1:172.21.15.184:6789/0 conn(0x7f6044071090 legacy=0x7f604415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.296+0000 7f6049d13640 1 -- 172.21.15.40:0/1428122631 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604406d3f0 con 0x7f6044070350 2024-04-26T21:37:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.296+0000 7f6049512640 1 -- 172.21.15.40:0/1428122631 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801232740 0 0) 0x7f60440d2390 con 0x7f6044071090 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.296+0000 7f604a514640 1 --2- 172.21.15.40:0/1428122631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60440cfde0 0x7f60440d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.296+0000 7f6049d13640 1 --2- 172.21.15.40:0/1428122631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 0x7f6044070b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f60340097a0 tx=0x7f603402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e84c37f326ad45a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.296+0000 7f6049512640 1 -- 172.21.15.40:0/1428122631 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034037070 con 0x7f6044070350 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.296+0000 7f6049512640 1 -- 172.21.15.40:0/1428122631 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603402ae50 con 0x7f6044070350 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.296+0000 7f6049512640 1 -- 172.21.15.40:0/1428122631 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034032660 con 0x7f6044070350 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.297+0000 7f604c778640 1 -- 172.21.15.40:0/1428122631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 msgr2=0x7f6044070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.297+0000 7f604c778640 1 --2- 172.21.15.40:0/1428122631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 0x7f6044070b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f60340097a0 tx=0x7f603402a100 comp rx=0 tx=0).stop 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.297+0000 7f604c778640 1 -- 172.21.15.40:0/1428122631 shutdown_connections 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.297+0000 7f604c778640 1 --2- 172.21.15.40:0/1428122631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 0x7f6044070b50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.297+0000 7f604c778640 1 --2- 172.21.15.40:0/1428122631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60440cfde0 0x7f60440d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.297+0000 7f604c778640 1 -- 172.21.15.40:0/1428122631 >> 172.21.15.40:0/1428122631 conn(0x7f60440d2d80 msgr2=0x7f60440d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.297+0000 7f604c778640 1 -- 172.21.15.40:0/1428122631 shutdown_connections 2024-04-26T21:37:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.297+0000 7f604c778640 1 -- 172.21.15.40:0/1428122631 wait complete. 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.298+0000 7f604c778640 1 Processor -- start 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.298+0000 7f604c778640 1 -- start start 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604c778640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 0x7f60441507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604c778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6044071090 0x7f604415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604c778640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60440cfde0 0x7f604409f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604c778640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f604406d9b0 con 0x7f60440cfde0 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604c778640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f604406dd20 con 0x7f6044070350 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604c778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60440d20a0 con 0x7f6044071090 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604ad15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60440cfde0 0x7f604409f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604ad15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60440cfde0 0x7f604409f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57006/0 (socket says 172.21.15.40:57006) 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604ad15640 1 -- 172.21.15.40:0/375209052 learned_addr learned my addr 172.21.15.40:0/375209052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f6049d13640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6044071090 0x7f604415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604a514640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 0x7f60441507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604ad15640 1 -- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6044071090 msgr2=0x7f604415b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604ad15640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6044071090 0x7f604415b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604ad15640 1 -- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 msgr2=0x7f60441507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.299+0000 7f604ad15640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 0x7f60441507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.300+0000 7f604ad15640 1 -- 172.21.15.40:0/375209052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60440a00d0 con 0x7f60440cfde0 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.300+0000 7f604ad15640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60440cfde0 0x7f604409f610 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f603c00c700 tx=0x7f603c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.300+0000 7f603affd640 1 -- 172.21.15.40:0/375209052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c018070 con 0x7f60440cfde0 2024-04-26T21:37:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.300+0000 7f603affd640 1 -- 172.21.15.40:0/375209052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603c00ce10 con 0x7f60440cfde0 2024-04-26T21:37:40.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.300+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60440b2880 con 0x7f60440cfde0 2024-04-26T21:37:40.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.300+0000 7f603affd640 1 -- 172.21.15.40:0/375209052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c00a860 con 0x7f60440cfde0 2024-04-26T21:37:40.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.300+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60440b2e40 con 0x7f60440cfde0 2024-04-26T21:37:40.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.301+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60440694f0 con 0x7f60440cfde0 2024-04-26T21:37:40.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.301+0000 7f603affd640 1 -- 172.21.15.40:0/375209052 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f603c00aa00 con 0x7f60440cfde0 2024-04-26T21:37:40.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.302+0000 7f603affd640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f602806c8f0 0x7f602806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:40.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.302+0000 7f603affd640 1 -- 172.21.15.40:0/375209052 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f603c08b220 con 0x7f60440cfde0 2024-04-26T21:37:40.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.302+0000 7f604a514640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f602806c8f0 0x7f602806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:40.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.302+0000 7f604a514640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f602806c8f0 0x7f602806edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6040006ca0 tx=0x7f6040009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:40.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.306+0000 7f603affd640 1 -- 172.21.15.40:0/375209052 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f603c058e80 con 0x7f60440cfde0 2024-04-26T21:37:40.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:37:40.365 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:37:40.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:40.532+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f60440a09e0 con 0x7f60440cfde0 2024-04-26T21:37:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.169+0000 7f603affd640 1 -- 172.21.15.40:0/375209052 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f603c05d180 con 0x7f60440cfde0 2024-04-26T21:37:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f602806c8f0 msgr2=0x7f602806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f602806c8f0 0x7f602806edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6040006ca0 tx=0x7f6040009240 comp rx=0 tx=0).stop 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60440cfde0 msgr2=0x7f604409f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60440cfde0 0x7f604409f610 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f603c00c700 tx=0x7f603c00cbc0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 shutdown_connections 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f602806c8f0 0x7f602806edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60440cfde0 0x7f604409f610 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6044071090 0x7f604415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 --2- 172.21.15.40:0/375209052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6044070350 0x7f60441507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 >> 172.21.15.40:0/375209052 conn(0x7f60440d2d80 msgr2=0x7f60440b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 shutdown_connections 2024-04-26T21:37:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.172+0000 7f604c778640 1 -- 172.21.15.40:0/375209052 wait complete. 2024-04-26T21:37:41.185 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:37:41.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:41.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.293+0000 7f7532852640 1 Processor -- start 2024-04-26T21:37:41.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.293+0000 7f7532852640 1 -- start start 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f7532852640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f752c162880 0x7f752c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f7532852640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0d3bc0 0x7f752c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f7532852640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c0b7e50 con 0x7f752c0b5f20 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f7532852640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f752c0d0580 con 0x7f752c162880 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f7532852640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f752c0d1800 con 0x7f752c0d3bc0 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f752bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f752c162880 0x7f752c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f752b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f752c0b5f20 0x7f752c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60458/0 (socket says 172.21.15.40:60458) 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f752b7fe640 1 -- 172.21.15.40:0/1115076919 learned_addr learned my addr 172.21.15.40:0/1115076919 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f7530def640 1 --2- 172.21.15.40:0/1115076919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0d3bc0 0x7f752c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f752bfff640 1 -- 172.21.15.40:0/1115076919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0d3bc0 msgr2=0x7f752c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f752bfff640 1 --2- 172.21.15.40:0/1115076919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0d3bc0 0x7f752c0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f752bfff640 1 -- 172.21.15.40:0/1115076919 >> v1:172.21.15.40:6789/0 conn(0x7f752c0b5f20 legacy=0x7f752c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f752bfff640 1 -- 172.21.15.40:0/1115076919 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c0d07d0 con 0x7f752c162880 2024-04-26T21:37:41.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.294+0000 7f7530def640 1 --2- 172.21.15.40:0/1115076919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0d3bc0 0x7f752c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.295+0000 7f752bfff640 1 --2- 172.21.15.40:0/1115076919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c0d2cb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f75200097a0 tx=0x7f752002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3c317602bb7dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.295+0000 7f752affd640 1 -- 172.21.15.40:0/1115076919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520037070 con 0x7f752c162880 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.295+0000 7f752affd640 1 -- 172.21.15.40:0/1115076919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7520032440 con 0x7f752c162880 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.295+0000 7f752affd640 1 -- 172.21.15.40:0/1115076919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520032810 con 0x7f752c162880 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.296+0000 7f7532852640 1 -- 172.21.15.40:0/1115076919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 msgr2=0x7f752c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.296+0000 7f7532852640 1 --2- 172.21.15.40:0/1115076919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c0d2cb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f75200097a0 tx=0x7f752002a180 comp rx=0 tx=0).stop 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.296+0000 7f7532852640 1 -- 172.21.15.40:0/1115076919 shutdown_connections 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.296+0000 7f7532852640 1 --2- 172.21.15.40:0/1115076919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0d3bc0 0x7f752c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.296+0000 7f7532852640 1 --2- 172.21.15.40:0/1115076919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c0d2cb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.296+0000 7f7532852640 1 -- 172.21.15.40:0/1115076919 >> 172.21.15.40:0/1115076919 conn(0x7f752c0d4f60 msgr2=0x7f752c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.296+0000 7f7532852640 1 -- 172.21.15.40:0/1115076919 shutdown_connections 2024-04-26T21:37:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.296+0000 7f7532852640 1 -- 172.21.15.40:0/1115076919 wait complete. 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.297+0000 7f7532852640 1 Processor -- start 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.297+0000 7f7532852640 1 -- start start 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7532852640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0b5f20 0x7f752c155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7532852640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f752c0d3bc0 0x7f752c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7532852640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7532852640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f752c162c60 con 0x7f752c162880 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7532852640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f752c0d0d90 con 0x7f752c0d3bc0 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7532852640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f752c162de0 con 0x7f752c0b5f20 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7530def640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7530def640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c15ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57014/0 (socket says 172.21.15.40:57014) 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f7530def640 1 -- 172.21.15.40:0/305005659 learned_addr learned my addr 172.21.15.40:0/305005659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.298+0000 7f752bfff640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0b5f20 0x7f752c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f752b7fe640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f752c0d3bc0 0x7f752c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7530def640 1 -- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0b5f20 msgr2=0x7f752c155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7530def640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0b5f20 0x7f752c155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7530def640 1 -- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f752c0d3bc0 msgr2=0x7f752c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7530def640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f752c0d3bc0 0x7f752c158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7530def640 1 -- 172.21.15.40:0/305005659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c0cf150 con 0x7f752c162880 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7530def640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c15ada0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f752400ccf0 tx=0x7f752400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7528ff9640 1 -- 172.21.15.40:0/305005659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400f680 con 0x7f752c162880 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7528ff9640 1 -- 172.21.15.40:0/305005659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f752400f840 con 0x7f752c162880 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c159a80 con 0x7f752c162880 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7528ff9640 1 -- 172.21.15.40:0/305005659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400fc30 con 0x7f752c162880 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.299+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c15a040 con 0x7f752c162880 2024-04-26T21:37:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.300+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c0694f0 con 0x7f752c162880 2024-04-26T21:37:41.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.300+0000 7f7528ff9640 1 -- 172.21.15.40:0/305005659 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75240213f0 con 0x7f752c162880 2024-04-26T21:37:41.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.301+0000 7f7528ff9640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f751806c8f0 0x7f751806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.301+0000 7f7528ff9640 1 -- 172.21.15.40:0/305005659 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f752408c920 con 0x7f752c162880 2024-04-26T21:37:41.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.301+0000 7f752bfff640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f751806c8f0 0x7f751806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.302+0000 7f752bfff640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f751806c8f0 0x7f751806edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7520034000 tx=0x7f752002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.305+0000 7f7528ff9640 1 -- 172.21.15.40:0/305005659 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f752c0cf150 con 0x7f752c162880 2024-04-26T21:37:41.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.531+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f752c159330 con 0x7f752c162880 2024-04-26T21:37:41.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.531+0000 7f7528ff9640 1 -- 172.21.15.40:0/305005659 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5181 (secure 0 0 0) 0x7f7524014030 con 0x7f752c162880 2024-04-26T21:37:41.531 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:41.532 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:37:40.533180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2276808574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2276808574},{"type":"v1","addr":"172.21.15.40:6834","nonce":2276808574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6015,"qdb_cluster":[6015]},"id":3}]} 2024-04-26T21:37:41.532 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 32 2024-04-26T21:37:41.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.533+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f751806c8f0 msgr2=0x7f751806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.533+0000 7f7532852640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f751806c8f0 0x7f751806edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7520034000 tx=0x7f752002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:41.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.533+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 msgr2=0x7f752c15ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.533+0000 7f7532852640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c15ada0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f752400ccf0 tx=0x7f752400a650 comp rx=0 tx=0).stop 2024-04-26T21:37:41.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.534+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 shutdown_connections 2024-04-26T21:37:41.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.534+0000 7f7532852640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f751806c8f0 0x7f751806edb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.534+0000 7f7532852640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f752c162880 0x7f752c15ada0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.534+0000 7f7532852640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f752c0d3bc0 0x7f752c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.534+0000 7f7532852640 1 --2- 172.21.15.40:0/305005659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f752c0b5f20 0x7f752c155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.534+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 >> 172.21.15.40:0/305005659 conn(0x7f752c0d4f60 msgr2=0x7f752c0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:41.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.534+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 shutdown_connections 2024-04-26T21:37:41.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.534+0000 7f7532852640 1 -- 172.21.15.40:0/305005659 wait complete. 2024-04-26T21:37:41.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:41.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.646+0000 7f107e3f0640 1 Processor -- start 2024-04-26T21:37:41.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.646+0000 7f107e3f0640 1 -- start start 2024-04-26T21:37:41.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.646+0000 7f107e3f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1078163610 0x7f10780d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.646+0000 7f107e3f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10780d1a70 con 0x7f107815f5a0 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.646+0000 7f107e3f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10780d1d50 con 0x7f10780d2830 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.646+0000 7f107e3f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1078163d30 con 0x7f1078163610 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.647+0000 7f10777fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1078163610 0x7f10780d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.647+0000 7f1077fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f107815f5a0 0x7f1078162f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60474/0 (socket says 172.21.15.40:60474) 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.647+0000 7f10777fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1078163610 0x7f10780d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57030/0 (socket says 172.21.15.40:57030) 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.647+0000 7f1077fff640 1 -- 172.21.15.40:0/867004547 learned_addr learned my addr 172.21.15.40:0/867004547 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.647+0000 7f10777fe640 1 -- 172.21.15.40:0/867004547 >> v1:172.21.15.184:6789/0 conn(0x7f10780d2830 legacy=0x7f10780cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.647+0000 7f10777fe640 1 -- 172.21.15.40:0/867004547 >> v1:172.21.15.40:6789/0 conn(0x7f107815f5a0 legacy=0x7f1078162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.647+0000 7f10777fe640 1 -- 172.21.15.40:0/867004547 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780d2030 con 0x7f1078163610 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f10777fe640 1 --2- 172.21.15.40:0/867004547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1078163610 0x7f10780d5cb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f106c0097a0 tx=0x7f106c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1545400495c72645 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f1076ffd640 1 -- 172.21.15.40:0/867004547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c037070 con 0x7f1078163610 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f1076ffd640 1 -- 172.21.15.40:0/867004547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c02ae50 con 0x7f1078163610 2024-04-26T21:37:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f1076ffd640 1 -- 172.21.15.40:0/867004547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c032660 con 0x7f1078163610 2024-04-26T21:37:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f107e3f0640 1 -- 172.21.15.40:0/867004547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1078163610 msgr2=0x7f10780d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f107e3f0640 1 --2- 172.21.15.40:0/867004547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1078163610 0x7f10780d5cb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f106c0097a0 tx=0x7f106c02a140 comp rx=0 tx=0).stop 2024-04-26T21:37:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f107e3f0640 1 -- 172.21.15.40:0/867004547 shutdown_connections 2024-04-26T21:37:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f107e3f0640 1 --2- 172.21.15.40:0/867004547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1078163610 0x7f10780d5cb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f107e3f0640 1 -- 172.21.15.40:0/867004547 >> 172.21.15.40:0/867004547 conn(0x7f10780d5190 msgr2=0x7f10780d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f107e3f0640 1 -- 172.21.15.40:0/867004547 shutdown_connections 2024-04-26T21:37:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.648+0000 7f107e3f0640 1 -- 172.21.15.40:0/867004547 wait complete. 2024-04-26T21:37:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.649+0000 7f107e3f0640 1 Processor -- start 2024-04-26T21:37:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f107e3f0640 1 -- start start 2024-04-26T21:37:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f107e3f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10780d2830 0x7f107814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f107e3f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1078163610 0x7f10780c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f107e3f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10780c91c0 0x7f10780cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f107e3f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10780d3c20 con 0x7f10780d2830 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f107e3f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10780d4190 con 0x7f10780c91c0 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f107e3f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10780d3da0 con 0x7f1078163610 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f1077fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10780d2830 0x7f107814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f1077fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10780d2830 0x7f107814b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57038/0 (socket says 172.21.15.40:57038) 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.650+0000 7f1077fff640 1 -- 172.21.15.40:0/1747359909 learned_addr learned my addr 172.21.15.40:0/1747359909 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f10777fe640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1078163610 0x7f10780c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f107c98d640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10780c91c0 0x7f10780cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1077fff640 1 -- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1078163610 msgr2=0x7f10780c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1077fff640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1078163610 0x7f10780c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1077fff640 1 -- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10780c91c0 msgr2=0x7f10780cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1077fff640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10780c91c0 0x7f10780cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1077fff640 1 -- 172.21.15.40:0/1747359909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780cb9f0 con 0x7f10780d2830 2024-04-26T21:37:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1077fff640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10780d2830 0x7f107814b790 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f106800c430 tx=0x7f106800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1074ff9640 1 -- 172.21.15.40:0/1747359909 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068017070 con 0x7f10780d2830 2024-04-26T21:37:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1074ff9640 1 -- 172.21.15.40:0/1747359909 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1068005ce0 con 0x7f10780d2830 2024-04-26T21:37:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10780cbcd0 con 0x7f10780d2830 2024-04-26T21:37:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f1074ff9640 1 -- 172.21.15.40:0/1747359909 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680072f0 con 0x7f10780d2830 2024-04-26T21:37:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.651+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107814c3e0 con 0x7f10780d2830 2024-04-26T21:37:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.652+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10780c95d0 con 0x7f10780d2830 2024-04-26T21:37:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.653+0000 7f1074ff9640 1 -- 172.21.15.40:0/1747359909 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f106801a070 con 0x7f10780d2830 2024-04-26T21:37:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.653+0000 7f1074ff9640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106406c8f0 0x7f106406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.653+0000 7f1074ff9640 1 -- 172.21.15.40:0/1747359909 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f106808b050 con 0x7f10780d2830 2024-04-26T21:37:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.653+0000 7f10777fe640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106406c8f0 0x7f106406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.654+0000 7f10777fe640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106406c8f0 0x7f106406edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f106c034000 tx=0x7f106c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.658+0000 7f1074ff9640 1 -- 172.21.15.40:0/1747359909 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10780c95d0 con 0x7f10780d2830 2024-04-26T21:37:41.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.844+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10780694f0 con 0x7f10780d2830 2024-04-26T21:37:41.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.845+0000 7f1074ff9640 1 -- 172.21.15.40:0/1747359909 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16906 (secure 0 0 0) 0x7f1068058ca0 con 0x7f10780d2830 2024-04-26T21:37:41.845 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:41.845 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":52,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:39.143951+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:37:35.784073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T21:37:36.460613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106406c8f0 msgr2=0x7f106406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106406c8f0 0x7f106406edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f106c034000 tx=0x7f106c02e000 comp rx=0 tx=0).stop 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10780d2830 msgr2=0x7f107814b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10780d2830 0x7f107814b790 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f106800c430 tx=0x7f106800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 shutdown_connections 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f106406c8f0 0x7f106406edb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10780c91c0 0x7f10780cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1078163610 0x7f10780c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 --2- 172.21.15.40:0/1747359909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10780d2830 0x7f107814b790 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 >> 172.21.15.40:0/1747359909 conn(0x7f10780d5190 msgr2=0x7f107815aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 shutdown_connections 2024-04-26T21:37:41.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.847+0000 7f107e3f0640 1 -- 172.21.15.40:0/1747359909 wait complete. 2024-04-26T21:37:41.859 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:37:41.859 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:37:41.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.959+0000 7fbeb2046640 1 Processor -- start 2024-04-26T21:37:41.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.959+0000 7fbeb2046640 1 -- start start 2024-04-26T21:37:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeb2046640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbeac162880 0x7fbeac0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeb2046640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbeac0b5f20 0x7fbeac0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeb2046640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 0x7fbeac0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeb2046640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbeac0d0580 con 0x7fbeac0b5f20 2024-04-26T21:37:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeb2046640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbeac0b7e50 con 0x7fbeac162880 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeb2046640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac0d1800 con 0x7fbeac0d39f0 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeaaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbeac0b5f20 0x7fbeac0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeaaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbeac0b5f20 0x7fbeac0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57048/0 (socket says 172.21.15.40:57048) 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeaaffd640 1 -- 172.21.15.40:0/2579514723 learned_addr learned my addr 172.21.15.40:0/2579514723 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeab7fe640 1 --2- 172.21.15.40:0/2579514723 >> v2:172.21.15.184:3300/0 conn(0x7fbeac162880 0x7fbeac0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeabfff640 1 --2- 172.21.15.40:0/2579514723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 0x7fbeac0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeaaffd640 1 -- 172.21.15.40:0/2579514723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 msgr2=0x7fbeac0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeaaffd640 1 --2- 172.21.15.40:0/2579514723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 0x7fbeac0b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeaaffd640 1 -- 172.21.15.40:0/2579514723 >> v2:172.21.15.184:3300/0 conn(0x7fbeac162880 msgr2=0x7fbeac0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeaaffd640 1 --2- 172.21.15.40:0/2579514723 >> v2:172.21.15.184:3300/0 conn(0x7fbeac162880 0x7fbeac0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeaaffd640 1 -- 172.21.15.40:0/2579514723 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac0d07d0 con 0x7fbeac0b5f20 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.960+0000 7fbeab7fe640 1 --2- 172.21.15.40:0/2579514723 >> v2:172.21.15.184:3300/0 conn(0x7fbeac162880 0x7fbeac0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.961+0000 7fbeaaffd640 1 --2- 172.21.15.40:0/2579514723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0d34b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbeac08d490 tx=0x7fbea002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4682e4935d0681d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.961+0000 7fbeaa7fc640 1 -- 172.21.15.40:0/2579514723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0037070 con 0x7fbeac0b5f20 2024-04-26T21:37:41.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.961+0000 7fbeaa7fc640 1 -- 172.21.15.40:0/2579514723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea0032440 con 0x7fbeac0b5f20 2024-04-26T21:37:41.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.961+0000 7fbeaa7fc640 1 -- 172.21.15.40:0/2579514723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0032810 con 0x7fbeac0b5f20 2024-04-26T21:37:41.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 -- 172.21.15.40:0/2579514723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 msgr2=0x7fbeac0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 --2- 172.21.15.40:0/2579514723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0d34b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbeac08d490 tx=0x7fbea002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 -- 172.21.15.40:0/2579514723 shutdown_connections 2024-04-26T21:37:41.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 --2- 172.21.15.40:0/2579514723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 0x7fbeac0b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 --2- 172.21.15.40:0/2579514723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0d34b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 --2- 172.21.15.40:0/2579514723 >> v2:172.21.15.184:3300/0 conn(0x7fbeac162880 0x7fbeac0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 -- 172.21.15.40:0/2579514723 >> 172.21.15.40:0/2579514723 conn(0x7fbeac0d4f60 msgr2=0x7fbeac0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 -- 172.21.15.40:0/2579514723 shutdown_connections 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.962+0000 7fbeb2046640 1 -- 172.21.15.40:0/2579514723 wait complete. 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.963+0000 7fbeb2046640 1 Processor -- start 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.963+0000 7fbeb2046640 1 -- start start 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeb2046640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeb2046640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 0x7fbeac151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeb2046640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac162880 0x7fbeac0a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeb2046640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac162c60 con 0x7fbeac0b5f20 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeb2046640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0d0d90 con 0x7fbeac162880 2024-04-26T21:37:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeb2046640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac162de0 con 0x7fbeac0d39f0 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0a27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57058/0 (socket says 172.21.15.40:57058) 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeab7fe640 1 -- 172.21.15.40:0/3867129238 learned_addr learned my addr 172.21.15.40:0/3867129238 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeabfff640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac162880 0x7fbeac0a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeaaffd640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 0x7fbeac151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeab7fe640 1 -- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 msgr2=0x7fbeac151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeab7fe640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 0x7fbeac151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeab7fe640 1 -- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac162880 msgr2=0x7fbeac0a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeab7fe640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac162880 0x7fbeac0a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.964+0000 7fbeab7fe640 1 -- 172.21.15.40:0/3867129238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac0a1700 con 0x7fbeac0b5f20 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.965+0000 7fbeab7fe640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0a27a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00e7e0 tx=0x7fbe9c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.965+0000 7fbe93fff640 1 -- 172.21.15.40:0/3867129238 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c00f040 con 0x7fbeac0b5f20 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.965+0000 7fbe93fff640 1 -- 172.21.15.40:0/3867129238 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe9c004030 con 0x7fbeac0b5f20 2024-04-26T21:37:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.965+0000 7fbe93fff640 1 -- 172.21.15.40:0/3867129238 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c004400 con 0x7fbeac0b5f20 2024-04-26T21:37:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.965+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac0a19e0 con 0x7fbeac0b5f20 2024-04-26T21:37:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.965+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac0a1fa0 con 0x7fbeac0b5f20 2024-04-26T21:37:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.966+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac0694f0 con 0x7fbeac0b5f20 2024-04-26T21:37:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.966+0000 7fbe93fff640 1 -- 172.21.15.40:0/3867129238 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe9c00c6d0 con 0x7fbeac0b5f20 2024-04-26T21:37:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.967+0000 7fbe93fff640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.967+0000 7fbe93fff640 1 -- 172.21.15.40:0/3867129238 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fbe9c08bb60 con 0x7fbeac0b5f20 2024-04-26T21:37:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.967+0000 7fbeaaffd640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.968+0000 7fbeaaffd640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbea0034000 tx=0x7fbea002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:41.971+0000 7fbe93fff640 1 -- 172.21.15.40:0/3867129238 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe9c017020 con 0x7fbeac0b5f20 2024-04-26T21:37:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.155+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbeac0d56c0 con 0x7fbe9806c8f0 2024-04-26T21:37:42.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.161+0000 7fbe93fff640 1 -- 172.21.15.40:0/3867129238 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246638 (secure 0 0 0) 0x7fbeac0d56c0 con 0x7fbe9806c8f0 2024-04-26T21:37:42.164 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:42.166 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:37:42.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 msgr2=0x7fbe9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbea0034000 tx=0x7fbea002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 msgr2=0x7fbeac0a27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0a27a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00e7e0 tx=0x7fbe9c00eca0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 shutdown_connections 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe9806c8f0 0x7fbe9806edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeac162880 0x7fbeac0a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbeac0d39f0 0x7fbeac151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 --2- 172.21.15.40:0/3867129238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeac0b5f20 0x7fbeac0a27a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 >> 172.21.15.40:0/3867129238 conn(0x7fbeac0d4f60 msgr2=0x7fbeac06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 shutdown_connections 2024-04-26T21:37:42.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.164+0000 7fbeb2046640 1 -- 172.21.15.40:0/3867129238 wait complete. 2024-04-26T21:37:42.176 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-04-26T21:37:42.029971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222012,"kb_used_data":5800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752708,"statfs":{"total":773094113280,"available":772866772992,"internally_reserved":0,"allocated":5939200,"data_stored":3288896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.900699"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.146297+0000","last_change":"2024-04-26T21:37:37.146297+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.146297+0000","last_undegraded":"2024-04-26T21:37:37.146297+0000","last_fullsized":"2024-04-26T21:37:37.146297+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138776+0000","last_change":"2024-04-26T21:37:37.138776+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138776+0000","last_undegraded":"2024-04-26T21:37:37.138776+0000","last_fullsized":"2024-04-26T21:37:37.138776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140361+0000","last_change":"2024-04-26T21:37:37.140361+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140361+0000","last_undegraded":"2024-04-26T21:37:37.140361+0000","last_fullsized":"2024-04-26T21:37:37.140361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139226+0000","last_change":"2024-04-26T21:37:37.139226+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139226+0000","last_undegraded":"2024-04-26T21:37:37.139226+0000","last_fullsized":"2024-04-26T21:37:37.139226+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137175+0000","last_change":"2024-04-26T21:37:37.137175+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137175+0000","last_undegraded":"2024-04-26T21:37:37.137175+0000","last_fullsized":"2024-04-26T21:37:37.137175+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.147679+0000","last_change":"2024-04-26T21:37:37.147679+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.147679+0000","last_undegraded":"2024-04-26T21:37:37.147679+0000","last_fullsized":"2024-04-26T21:37:37.147679+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139709+0000","last_change":"2024-04-26T21:37:37.139709+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139709+0000","last_undegraded":"2024-04-26T21:37:37.139709+0000","last_fullsized":"2024-04-26T21:37:37.139709+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139690+0000","last_change":"2024-04-26T21:37:37.139690+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139690+0000","last_undegraded":"2024-04-26T21:37:37.139690+0000","last_fullsized":"2024-04-26T21:37:37.139690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139335+0000","last_change":"2024-04-26T21:37:37.139335+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139335+0000","last_undegraded":"2024-04-26T21:37:37.139335+0000","last_fullsized":"2024-04-26T21:37:37.139335+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.148501+0000","last_change":"2024-04-26T21:37:37.148501+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.148501+0000","last_undegraded":"2024-04-26T21:37:37.148501+0000","last_fullsized":"2024-04-26T21:37:37.148501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143003+0000","last_change":"2024-04-26T21:37:37.143003+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143003+0000","last_undegraded":"2024-04-26T21:37:37.143003+0000","last_fullsized":"2024-04-26T21:37:37.143003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139262+0000","last_change":"2024-04-26T21:37:37.139262+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139262+0000","last_undegraded":"2024-04-26T21:37:37.139262+0000","last_fullsized":"2024-04-26T21:37:37.139262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145421+0000","last_change":"2024-04-26T21:37:37.145421+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145421+0000","last_undegraded":"2024-04-26T21:37:37.145421+0000","last_fullsized":"2024-04-26T21:37:37.145421+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140247+0000","last_change":"2024-04-26T21:37:37.140247+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140247+0000","last_undegraded":"2024-04-26T21:37:37.140247+0000","last_fullsized":"2024-04-26T21:37:37.140247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138351+0000","last_change":"2024-04-26T21:37:37.138351+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138351+0000","last_undegraded":"2024-04-26T21:37:37.138351+0000","last_fullsized":"2024-04-26T21:37:37.138351+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143870+0000","last_change":"2024-04-26T21:37:37.143870+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143870+0000","last_undegraded":"2024-04-26T21:37:37.143870+0000","last_fullsized":"2024-04-26T21:37:37.143870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144920+0000","last_change":"2024-04-26T21:37:37.144920+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144920+0000","last_undegraded":"2024-04-26T21:37:37.144920+0000","last_fullsized":"2024-04-26T21:37:37.144920+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138328+0000","last_change":"2024-04-26T21:37:37.138328+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138328+0000","last_undegraded":"2024-04-26T21:37:37.138328+0000","last_fullsized":"2024-04-26T21:37:37.138328+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139643+0000","last_change":"2024-04-26T21:37:37.139643+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139643+0000","last_undegraded":"2024-04-26T21:37:37.139643+0000","last_fullsized":"2024-04-26T21:37:37.139643+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143012+0000","last_change":"2024-04-26T21:37:37.143012+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143012+0000","last_undegraded":"2024-04-26T21:37:37.143012+0000","last_fullsized":"2024-04-26T21:37:37.143012+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140772+0000","last_change":"2024-04-26T21:37:37.140772+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140772+0000","last_undegraded":"2024-04-26T21:37:37.140772+0000","last_fullsized":"2024-04-26T21:37:37.140772+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138725+0000","last_change":"2024-04-26T21:37:37.138725+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138725+0000","last_undegraded":"2024-04-26T21:37:37.138725+0000","last_fullsized":"2024-04-26T21:37:37.138725+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137913+0000","last_change":"2024-04-26T21:37:37.137913+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137913+0000","last_undegraded":"2024-04-26T21:37:37.137913+0000","last_fullsized":"2024-04-26T21:37:37.137913+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138780+0000","last_change":"2024-04-26T21:37:37.138780+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138780+0000","last_undegraded":"2024-04-26T21:37:37.138780+0000","last_fullsized":"2024-04-26T21:37:37.138780+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144498+0000","last_change":"2024-04-26T21:37:37.144498+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144498+0000","last_undegraded":"2024-04-26T21:37:37.144498+0000","last_fullsized":"2024-04-26T21:37:37.144498+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138226+0000","last_change":"2024-04-26T21:37:37.138226+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138226+0000","last_undegraded":"2024-04-26T21:37:37.138226+0000","last_fullsized":"2024-04-26T21:37:37.138226+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.149883+0000","last_change":"2024-04-26T21:37:37.149883+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.149883+0000","last_undegraded":"2024-04-26T21:37:37.149883+0000","last_fullsized":"2024-04-26T21:37:37.149883+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.142613+0000","last_change":"2024-04-26T21:37:37.142613+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.142613+0000","last_undegraded":"2024-04-26T21:37:37.142613+0000","last_fullsized":"2024-04-26T21:37:37.142613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144029+0000","last_change":"2024-04-26T21:37:37.144029+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144029+0000","last_undegraded":"2024-04-26T21:37:37.144029+0000","last_fullsized":"2024-04-26T21:37:37.144029+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138351+0000","last_change":"2024-04-26T21:37:37.138351+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138351+0000","last_undegraded":"2024-04-26T21:37:37.138351+0000","last_fullsized":"2024-04-26T21:37:37.138351+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138627+0000","last_change":"2024-04-26T21:37:37.138627+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138627+0000","last_undegraded":"2024-04-26T21:37:37.138627+0000","last_fullsized":"2024-04-26T21:37:37.138627+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.135376+0000","last_change":"2024-04-26T21:37:37.135376+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.135376+0000","last_undegraded":"2024-04-26T21:37:37.135376+0000","last_fullsized":"2024-04-26T21:37:37.135376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138262+0000","last_change":"2024-04-26T21:37:37.138262+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138262+0000","last_undegraded":"2024-04-26T21:37:37.138262+0000","last_fullsized":"2024-04-26T21:37:37.138262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155881+0000","last_change":"2024-04-26T21:37:37.150925+0000","last_active":"2024-04-26T21:37:37.155881+0000","last_peered":"2024-04-26T21:37:37.155881+0000","last_clean":"2024-04-26T21:37:37.155881+0000","last_became_active":"2024-04-26T21:37:37.150597+0000","last_became_peered":"2024-04-26T21:37:37.150597+0000","last_unstale":"2024-04-26T21:37:37.155881+0000","last_undegraded":"2024-04-26T21:37:37.155881+0000","last_fullsized":"2024-04-26T21:37:37.155881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:53.683181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134054+0000","last_change":"2024-04-26T21:37:37.134054+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134054+0000","last_undegraded":"2024-04-26T21:37:37.134054+0000","last_fullsized":"2024-04-26T21:37:37.134054+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159403+0000","last_change":"2024-04-26T21:37:37.156594+0000","last_active":"2024-04-26T21:37:37.159403+0000","last_peered":"2024-04-26T21:37:37.159403+0000","last_clean":"2024-04-26T21:37:37.159403+0000","last_became_active":"2024-04-26T21:37:37.156312+0000","last_became_peered":"2024-04-26T21:37:37.156312+0000","last_unstale":"2024-04-26T21:37:37.159403+0000","last_undegraded":"2024-04-26T21:37:37.159403+0000","last_fullsized":"2024-04-26T21:37:37.159403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:43.766358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.142303+0000","last_change":"2024-04-26T21:37:37.142303+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.142303+0000","last_undegraded":"2024-04-26T21:37:37.142303+0000","last_fullsized":"2024-04-26T21:37:37.142303+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.162010+0000","last_change":"2024-04-26T21:37:37.158086+0000","last_active":"2024-04-26T21:37:37.162010+0000","last_peered":"2024-04-26T21:37:37.162010+0000","last_clean":"2024-04-26T21:37:37.162010+0000","last_became_active":"2024-04-26T21:37:37.157790+0000","last_became_peered":"2024-04-26T21:37:37.157790+0000","last_unstale":"2024-04-26T21:37:37.162010+0000","last_undegraded":"2024-04-26T21:37:37.162010+0000","last_fullsized":"2024-04-26T21:37:37.162010+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.123343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136890+0000","last_change":"2024-04-26T21:37:37.136890+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136890+0000","last_undegraded":"2024-04-26T21:37:37.136890+0000","last_fullsized":"2024-04-26T21:37:37.136890+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.147649+0000","last_change":"2024-04-26T21:37:37.140298+0000","last_active":"2024-04-26T21:37:37.147649+0000","last_peered":"2024-04-26T21:37:37.147649+0000","last_clean":"2024-04-26T21:37:37.147649+0000","last_became_active":"2024-04-26T21:37:37.139801+0000","last_became_peered":"2024-04-26T21:37:37.139801+0000","last_unstale":"2024-04-26T21:37:37.147649+0000","last_undegraded":"2024-04-26T21:37:37.147649+0000","last_fullsized":"2024-04-26T21:37:37.147649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137107+0000","last_change":"2024-04-26T21:37:37.137107+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137107+0000","last_undegraded":"2024-04-26T21:37:37.137107+0000","last_fullsized":"2024-04-26T21:37:37.137107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.162038+0000","last_change":"2024-04-26T21:37:37.158278+0000","last_active":"2024-04-26T21:37:37.162038+0000","last_peered":"2024-04-26T21:37:37.162038+0000","last_clean":"2024-04-26T21:37:37.162038+0000","last_became_active":"2024-04-26T21:37:37.154700+0000","last_became_peered":"2024-04-26T21:37:37.154700+0000","last_unstale":"2024-04-26T21:37:37.162038+0000","last_undegraded":"2024-04-26T21:37:37.162038+0000","last_fullsized":"2024-04-26T21:37:37.162038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:30.231390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139564+0000","last_change":"2024-04-26T21:37:37.139564+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139564+0000","last_undegraded":"2024-04-26T21:37:37.139564+0000","last_fullsized":"2024-04-26T21:37:37.139564+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.152274+0000","last_change":"2024-04-26T21:37:37.148427+0000","last_active":"2024-04-26T21:37:37.152274+0000","last_peered":"2024-04-26T21:37:37.152274+0000","last_clean":"2024-04-26T21:37:37.152274+0000","last_became_active":"2024-04-26T21:37:37.147872+0000","last_became_peered":"2024-04-26T21:37:37.147872+0000","last_unstale":"2024-04-26T21:37:37.152274+0000","last_undegraded":"2024-04-26T21:37:37.152274+0000","last_fullsized":"2024-04-26T21:37:37.152274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:14.846091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136935+0000","last_change":"2024-04-26T21:37:37.136935+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136935+0000","last_undegraded":"2024-04-26T21:37:37.136935+0000","last_fullsized":"2024-04-26T21:37:37.136935+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.149727+0000","last_change":"2024-04-26T21:37:37.147914+0000","last_active":"2024-04-26T21:37:37.149727+0000","last_peered":"2024-04-26T21:37:37.149727+0000","last_clean":"2024-04-26T21:37:37.149727+0000","last_became_active":"2024-04-26T21:37:37.147592+0000","last_became_peered":"2024-04-26T21:37:37.147592+0000","last_unstale":"2024-04-26T21:37:37.149727+0000","last_undegraded":"2024-04-26T21:37:37.149727+0000","last_fullsized":"2024-04-26T21:37:37.149727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:50.841990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138903+0000","last_change":"2024-04-26T21:37:37.138903+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138903+0000","last_undegraded":"2024-04-26T21:37:37.138903+0000","last_fullsized":"2024-04-26T21:37:37.138903+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158742+0000","last_change":"2024-04-26T21:37:37.153776+0000","last_active":"2024-04-26T21:37:37.158742+0000","last_peered":"2024-04-26T21:37:37.158742+0000","last_clean":"2024-04-26T21:37:37.158742+0000","last_became_active":"2024-04-26T21:37:37.153390+0000","last_became_peered":"2024-04-26T21:37:37.153390+0000","last_unstale":"2024-04-26T21:37:37.158742+0000","last_undegraded":"2024-04-26T21:37:37.158742+0000","last_fullsized":"2024-04-26T21:37:37.158742+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:43.797895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136442+0000","last_change":"2024-04-26T21:37:37.136442+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136442+0000","last_undegraded":"2024-04-26T21:37:37.136442+0000","last_fullsized":"2024-04-26T21:37:37.136442+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.154501+0000","last_change":"2024-04-26T21:37:37.153096+0000","last_active":"2024-04-26T21:37:37.154501+0000","last_peered":"2024-04-26T21:37:37.154501+0000","last_clean":"2024-04-26T21:37:37.154501+0000","last_became_active":"2024-04-26T21:37:37.152795+0000","last_became_peered":"2024-04-26T21:37:37.152795+0000","last_unstale":"2024-04-26T21:37:37.154501+0000","last_undegraded":"2024-04-26T21:37:37.154501+0000","last_fullsized":"2024-04-26T21:37:37.154501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:48.571013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134754+0000","last_change":"2024-04-26T21:37:37.134754+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134754+0000","last_undegraded":"2024-04-26T21:37:37.134754+0000","last_fullsized":"2024-04-26T21:37:37.134754+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158396+0000","last_change":"2024-04-26T21:37:37.154318+0000","last_active":"2024-04-26T21:37:37.158396+0000","last_peered":"2024-04-26T21:37:37.158396+0000","last_clean":"2024-04-26T21:37:37.158396+0000","last_became_active":"2024-04-26T21:37:37.154072+0000","last_became_peered":"2024-04-26T21:37:37.154072+0000","last_unstale":"2024-04-26T21:37:37.158396+0000","last_undegraded":"2024-04-26T21:37:37.158396+0000","last_fullsized":"2024-04-26T21:37:37.158396+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:08.561240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138381+0000","last_change":"2024-04-26T21:37:37.138381+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138381+0000","last_undegraded":"2024-04-26T21:37:37.138381+0000","last_fullsized":"2024-04-26T21:37:37.138381+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.156365+0000","last_change":"2024-04-26T21:37:37.154992+0000","last_active":"2024-04-26T21:37:37.156365+0000","last_peered":"2024-04-26T21:37:37.156365+0000","last_clean":"2024-04-26T21:37:37.156365+0000","last_became_active":"2024-04-26T21:37:37.154711+0000","last_became_peered":"2024-04-26T21:37:37.154711+0000","last_unstale":"2024-04-26T21:37:37.156365+0000","last_undegraded":"2024-04-26T21:37:37.156365+0000","last_fullsized":"2024-04-26T21:37:37.156365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:04.317873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136779+0000","last_change":"2024-04-26T21:37:37.136779+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136779+0000","last_undegraded":"2024-04-26T21:37:37.136779+0000","last_fullsized":"2024-04-26T21:37:37.136779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155175+0000","last_change":"2024-04-26T21:37:37.146807+0000","last_active":"2024-04-26T21:37:37.155175+0000","last_peered":"2024-04-26T21:37:37.155175+0000","last_clean":"2024-04-26T21:37:37.155175+0000","last_became_active":"2024-04-26T21:37:37.146479+0000","last_became_peered":"2024-04-26T21:37:37.146479+0000","last_unstale":"2024-04-26T21:37:37.155175+0000","last_undegraded":"2024-04-26T21:37:37.155175+0000","last_fullsized":"2024-04-26T21:37:37.155175+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.135431+0000","last_change":"2024-04-26T21:37:37.135431+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.135431+0000","last_undegraded":"2024-04-26T21:37:37.135431+0000","last_fullsized":"2024-04-26T21:37:37.135431+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158195+0000","last_change":"2024-04-26T21:37:37.152397+0000","last_active":"2024-04-26T21:37:37.158195+0000","last_peered":"2024-04-26T21:37:37.158195+0000","last_clean":"2024-04-26T21:37:37.158195+0000","last_became_active":"2024-04-26T21:37:37.152118+0000","last_became_peered":"2024-04-26T21:37:37.152118+0000","last_unstale":"2024-04-26T21:37:37.158195+0000","last_undegraded":"2024-04-26T21:37:37.158195+0000","last_fullsized":"2024-04-26T21:37:37.158195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:50.687873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139183+0000","last_change":"2024-04-26T21:37:37.139183+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139183+0000","last_undegraded":"2024-04-26T21:37:37.139183+0000","last_fullsized":"2024-04-26T21:37:37.139183+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158451+0000","last_change":"2024-04-26T21:37:37.155533+0000","last_active":"2024-04-26T21:37:37.158451+0000","last_peered":"2024-04-26T21:37:37.158451+0000","last_clean":"2024-04-26T21:37:37.158451+0000","last_became_active":"2024-04-26T21:37:37.155179+0000","last_became_peered":"2024-04-26T21:37:37.155179+0000","last_unstale":"2024-04-26T21:37:37.158451+0000","last_undegraded":"2024-04-26T21:37:37.158451+0000","last_fullsized":"2024-04-26T21:37:37.158451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:14.846091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137624+0000","last_change":"2024-04-26T21:37:37.137624+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137624+0000","last_undegraded":"2024-04-26T21:37:37.137624+0000","last_fullsized":"2024-04-26T21:37:37.137624+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.152746+0000","last_change":"2024-04-26T21:37:37.148935+0000","last_active":"2024-04-26T21:37:37.152746+0000","last_peered":"2024-04-26T21:37:37.152746+0000","last_clean":"2024-04-26T21:37:37.152746+0000","last_became_active":"2024-04-26T21:37:37.148635+0000","last_became_peered":"2024-04-26T21:37:37.148635+0000","last_unstale":"2024-04-26T21:37:37.152746+0000","last_undegraded":"2024-04-26T21:37:37.152746+0000","last_fullsized":"2024-04-26T21:37:37.152746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.883299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145387+0000","last_change":"2024-04-26T21:37:37.145387+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145387+0000","last_undegraded":"2024-04-26T21:37:37.145387+0000","last_fullsized":"2024-04-26T21:37:37.145387+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155439+0000","last_change":"2024-04-26T21:37:37.147944+0000","last_active":"2024-04-26T21:37:37.155439+0000","last_peered":"2024-04-26T21:37:37.155439+0000","last_clean":"2024-04-26T21:37:37.155439+0000","last_became_active":"2024-04-26T21:37:37.147675+0000","last_became_peered":"2024-04-26T21:37:37.147675+0000","last_unstale":"2024-04-26T21:37:37.155439+0000","last_undegraded":"2024-04-26T21:37:37.155439+0000","last_fullsized":"2024-04-26T21:37:37.155439+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:20.163566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136315+0000","last_change":"2024-04-26T21:37:37.136315+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136315+0000","last_undegraded":"2024-04-26T21:37:37.136315+0000","last_fullsized":"2024-04-26T21:37:37.136315+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158568+0000","last_change":"2024-04-26T21:37:37.153638+0000","last_active":"2024-04-26T21:37:37.158568+0000","last_peered":"2024-04-26T21:37:37.158568+0000","last_clean":"2024-04-26T21:37:37.158568+0000","last_became_active":"2024-04-26T21:37:37.153152+0000","last_became_peered":"2024-04-26T21:37:37.153152+0000","last_unstale":"2024-04-26T21:37:37.158568+0000","last_undegraded":"2024-04-26T21:37:37.158568+0000","last_fullsized":"2024-04-26T21:37:37.158568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:31.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144651+0000","last_change":"2024-04-26T21:37:37.144651+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144651+0000","last_undegraded":"2024-04-26T21:37:37.144651+0000","last_fullsized":"2024-04-26T21:37:37.144651+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.157061+0000","last_change":"2024-04-26T21:37:37.155432+0000","last_active":"2024-04-26T21:37:37.157061+0000","last_peered":"2024-04-26T21:37:37.157061+0000","last_clean":"2024-04-26T21:37:37.157061+0000","last_became_active":"2024-04-26T21:37:37.155163+0000","last_became_peered":"2024-04-26T21:37:37.155163+0000","last_unstale":"2024-04-26T21:37:37.157061+0000","last_undegraded":"2024-04-26T21:37:37.157061+0000","last_fullsized":"2024-04-26T21:37:37.157061+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:34.248752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136604+0000","last_change":"2024-04-26T21:37:37.136604+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136604+0000","last_undegraded":"2024-04-26T21:37:37.136604+0000","last_fullsized":"2024-04-26T21:37:37.136604+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.161212+0000","last_change":"2024-04-26T21:37:37.155985+0000","last_active":"2024-04-26T21:37:37.161212+0000","last_peered":"2024-04-26T21:37:37.161212+0000","last_clean":"2024-04-26T21:37:37.161212+0000","last_became_active":"2024-04-26T21:37:37.155643+0000","last_became_peered":"2024-04-26T21:37:37.155643+0000","last_unstale":"2024-04-26T21:37:37.161212+0000","last_undegraded":"2024-04-26T21:37:37.161212+0000","last_fullsized":"2024-04-26T21:37:37.161212+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:48.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145020+0000","last_change":"2024-04-26T21:37:37.145020+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145020+0000","last_undegraded":"2024-04-26T21:37:37.145020+0000","last_fullsized":"2024-04-26T21:37:37.145020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155757+0000","last_change":"2024-04-26T21:37:37.150088+0000","last_active":"2024-04-26T21:37:37.155757+0000","last_peered":"2024-04-26T21:37:37.155757+0000","last_clean":"2024-04-26T21:37:37.155757+0000","last_became_active":"2024-04-26T21:37:37.149820+0000","last_became_peered":"2024-04-26T21:37:37.149820+0000","last_unstale":"2024-04-26T21:37:37.155757+0000","last_undegraded":"2024-04-26T21:37:37.155757+0000","last_fullsized":"2024-04-26T21:37:37.155757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:56.039426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134473+0000","last_change":"2024-04-26T21:37:37.134473+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134473+0000","last_undegraded":"2024-04-26T21:37:37.134473+0000","last_fullsized":"2024-04-26T21:37:37.134473+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159394+0000","last_change":"2024-04-26T21:37:37.156465+0000","last_active":"2024-04-26T21:37:37.159394+0000","last_peered":"2024-04-26T21:37:37.159394+0000","last_clean":"2024-04-26T21:37:37.159394+0000","last_became_active":"2024-04-26T21:37:37.156194+0000","last_became_peered":"2024-04-26T21:37:37.156194+0000","last_unstale":"2024-04-26T21:37:37.159394+0000","last_undegraded":"2024-04-26T21:37:37.159394+0000","last_fullsized":"2024-04-26T21:37:37.159394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:28.173221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.141743+0000","last_change":"2024-04-26T21:37:37.141743+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.141743+0000","last_undegraded":"2024-04-26T21:37:37.141743+0000","last_fullsized":"2024-04-26T21:37:37.141743+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150052+0000","last_change":"2024-04-26T21:37:37.143737+0000","last_active":"2024-04-26T21:37:37.150052+0000","last_peered":"2024-04-26T21:37:37.150052+0000","last_clean":"2024-04-26T21:37:37.150052+0000","last_became_active":"2024-04-26T21:37:37.143432+0000","last_became_peered":"2024-04-26T21:37:37.143432+0000","last_unstale":"2024-04-26T21:37:37.150052+0000","last_undegraded":"2024-04-26T21:37:37.150052+0000","last_fullsized":"2024-04-26T21:37:37.150052+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:48.659187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139896+0000","last_change":"2024-04-26T21:37:37.139896+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139896+0000","last_undegraded":"2024-04-26T21:37:37.139896+0000","last_fullsized":"2024-04-26T21:37:37.139896+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158712+0000","last_change":"2024-04-26T21:37:37.156130+0000","last_active":"2024-04-26T21:37:37.158712+0000","last_peered":"2024-04-26T21:37:37.158712+0000","last_clean":"2024-04-26T21:37:37.158712+0000","last_became_active":"2024-04-26T21:37:37.154520+0000","last_became_peered":"2024-04-26T21:37:37.154520+0000","last_unstale":"2024-04-26T21:37:37.158712+0000","last_undegraded":"2024-04-26T21:37:37.158712+0000","last_fullsized":"2024-04-26T21:37:37.158712+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:38.183624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145831+0000","last_change":"2024-04-26T21:37:37.145831+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145831+0000","last_undegraded":"2024-04-26T21:37:37.145831+0000","last_fullsized":"2024-04-26T21:37:37.145831+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150738+0000","last_change":"2024-04-26T21:37:37.145621+0000","last_active":"2024-04-26T21:37:37.150738+0000","last_peered":"2024-04-26T21:37:37.150738+0000","last_clean":"2024-04-26T21:37:37.150738+0000","last_became_active":"2024-04-26T21:37:37.145215+0000","last_became_peered":"2024-04-26T21:37:37.145215+0000","last_unstale":"2024-04-26T21:37:37.150738+0000","last_undegraded":"2024-04-26T21:37:37.150738+0000","last_fullsized":"2024-04-26T21:37:37.150738+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.363875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134531+0000","last_change":"2024-04-26T21:37:37.134531+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134531+0000","last_undegraded":"2024-04-26T21:37:37.134531+0000","last_fullsized":"2024-04-26T21:37:37.134531+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.161206+0000","last_change":"2024-04-26T21:37:37.155721+0000","last_active":"2024-04-26T21:37:37.161206+0000","last_peered":"2024-04-26T21:37:37.161206+0000","last_clean":"2024-04-26T21:37:37.161206+0000","last_became_active":"2024-04-26T21:37:37.155438+0000","last_became_peered":"2024-04-26T21:37:37.155438+0000","last_unstale":"2024-04-26T21:37:37.161206+0000","last_undegraded":"2024-04-26T21:37:37.161206+0000","last_fullsized":"2024-04-26T21:37:37.161206+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:08.561240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.147246+0000","last_change":"2024-04-26T21:37:37.147246+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.147246+0000","last_undegraded":"2024-04-26T21:37:37.147246+0000","last_fullsized":"2024-04-26T21:37:37.147246+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.153071+0000","last_change":"2024-04-26T21:37:37.151412+0000","last_active":"2024-04-26T21:37:37.153071+0000","last_peered":"2024-04-26T21:37:37.153071+0000","last_clean":"2024-04-26T21:37:37.153071+0000","last_became_active":"2024-04-26T21:37:37.151157+0000","last_became_peered":"2024-04-26T21:37:37.151157+0000","last_unstale":"2024-04-26T21:37:37.153071+0000","last_undegraded":"2024-04-26T21:37:37.153071+0000","last_fullsized":"2024-04-26T21:37:37.153071+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:30.231390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136343+0000","last_change":"2024-04-26T21:37:37.136343+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136343+0000","last_undegraded":"2024-04-26T21:37:37.136343+0000","last_fullsized":"2024-04-26T21:37:37.136343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158678+0000","last_change":"2024-04-26T21:37:37.154574+0000","last_active":"2024-04-26T21:37:37.158678+0000","last_peered":"2024-04-26T21:37:37.158678+0000","last_clean":"2024-04-26T21:37:37.158678+0000","last_became_active":"2024-04-26T21:37:37.153518+0000","last_became_peered":"2024-04-26T21:37:37.153518+0000","last_unstale":"2024-04-26T21:37:37.158678+0000","last_undegraded":"2024-04-26T21:37:37.158678+0000","last_fullsized":"2024-04-26T21:37:37.158678+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:43.168309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138567+0000","last_change":"2024-04-26T21:37:37.138567+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138567+0000","last_undegraded":"2024-04-26T21:37:37.138567+0000","last_fullsized":"2024-04-26T21:37:37.138567+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.151260+0000","last_change":"2024-04-26T21:37:37.144706+0000","last_active":"2024-04-26T21:37:37.151260+0000","last_peered":"2024-04-26T21:37:37.151260+0000","last_clean":"2024-04-26T21:37:37.151260+0000","last_became_active":"2024-04-26T21:37:37.144369+0000","last_became_peered":"2024-04-26T21:37:37.144369+0000","last_unstale":"2024-04-26T21:37:37.151260+0000","last_undegraded":"2024-04-26T21:37:37.151260+0000","last_fullsized":"2024-04-26T21:37:37.151260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134559+0000","last_change":"2024-04-26T21:37:37.134559+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134559+0000","last_undegraded":"2024-04-26T21:37:37.134559+0000","last_fullsized":"2024-04-26T21:37:37.134559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.157301+0000","last_change":"2024-04-26T21:37:37.153455+0000","last_active":"2024-04-26T21:37:37.157301+0000","last_peered":"2024-04-26T21:37:37.157301+0000","last_clean":"2024-04-26T21:37:37.157301+0000","last_became_active":"2024-04-26T21:37:37.153181+0000","last_became_peered":"2024-04-26T21:37:37.153181+0000","last_unstale":"2024-04-26T21:37:37.157301+0000","last_undegraded":"2024-04-26T21:37:37.157301+0000","last_fullsized":"2024-04-26T21:37:37.157301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:47.599920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140819+0000","last_change":"2024-04-26T21:37:37.140819+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140819+0000","last_undegraded":"2024-04-26T21:37:37.140819+0000","last_fullsized":"2024-04-26T21:37:37.140819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150089+0000","last_change":"2024-04-26T21:37:37.144882+0000","last_active":"2024-04-26T21:37:37.150089+0000","last_peered":"2024-04-26T21:37:37.150089+0000","last_clean":"2024-04-26T21:37:37.150089+0000","last_became_active":"2024-04-26T21:37:37.144633+0000","last_became_peered":"2024-04-26T21:37:37.144633+0000","last_unstale":"2024-04-26T21:37:37.150089+0000","last_undegraded":"2024-04-26T21:37:37.150089+0000","last_fullsized":"2024-04-26T21:37:37.150089+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:53.683181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136338+0000","last_change":"2024-04-26T21:37:37.136338+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136338+0000","last_undegraded":"2024-04-26T21:37:37.136338+0000","last_fullsized":"2024-04-26T21:37:37.136338+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155915+0000","last_change":"2024-04-26T21:37:37.150695+0000","last_active":"2024-04-26T21:37:37.155915+0000","last_peered":"2024-04-26T21:37:37.155915+0000","last_clean":"2024-04-26T21:37:37.155915+0000","last_became_active":"2024-04-26T21:37:37.150398+0000","last_became_peered":"2024-04-26T21:37:37.150398+0000","last_unstale":"2024-04-26T21:37:37.155915+0000","last_undegraded":"2024-04-26T21:37:37.155915+0000","last_fullsized":"2024-04-26T21:37:37.155915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.141329+0000","last_change":"2024-04-26T21:37:37.141329+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.141329+0000","last_undegraded":"2024-04-26T21:37:37.141329+0000","last_fullsized":"2024-04-26T21:37:37.141329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159474+0000","last_change":"2024-04-26T21:37:37.156817+0000","last_active":"2024-04-26T21:37:37.159474+0000","last_peered":"2024-04-26T21:37:37.159474+0000","last_clean":"2024-04-26T21:37:37.159474+0000","last_became_active":"2024-04-26T21:37:37.156561+0000","last_became_peered":"2024-04-26T21:37:37.156561+0000","last_unstale":"2024-04-26T21:37:37.159474+0000","last_undegraded":"2024-04-26T21:37:37.159474+0000","last_fullsized":"2024-04-26T21:37:37.159474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.883299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":226,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.140105+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:37:37.140105+0000","last_peered":"2024-04-26T21:37:37.140105+0000","last_clean":"2024-04-26T21:37:37.140105+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:37:37.140105+0000","last_undegraded":"2024-04-26T21:37:37.140105+0000","last_fullsized":"2024-04-26T21:37:37.140105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:37:42.180 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:37:42.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.317+0000 7f0cd5ff0640 1 Processor -- start 2024-04-26T21:37:42.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.317+0000 7f0cd5ff0640 1 -- start start 2024-04-26T21:37:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0cd5ff0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cd0162810 0x7f0cd0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0cd5ff0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0cd00b2340 0x7f0cd00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0cd5ff0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd00d10d0 con 0x7f0cd00cd930 2024-04-26T21:37:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0cd5ff0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd00d0750 con 0x7f0cd0162810 2024-04-26T21:37:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0cd5ff0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd00d16a0 con 0x7f0cd00b2340 2024-04-26T21:37:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0ccf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cd0162810 0x7f0cd0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0ccffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0cd00cd930 0x7f0cd00d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60490/0 (socket says 172.21.15.40:60490) 2024-04-26T21:37:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0ccffff640 1 -- 172.21.15.40:0/3771099447 learned_addr learned my addr 172.21.15.40:0/3771099447 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0ccf7fe640 1 -- 172.21.15.40:0/3771099447 >> v2:172.21.15.184:3300/0 conn(0x7f0cd00b2340 msgr2=0x7f0cd00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0cceffd640 1 --2- 172.21.15.40:0/3771099447 >> v2:172.21.15.184:3300/0 conn(0x7f0cd00b2340 0x7f0cd00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0ccf7fe640 1 --2- 172.21.15.40:0/3771099447 >> v2:172.21.15.184:3300/0 conn(0x7f0cd00b2340 0x7f0cd00cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0ccf7fe640 1 -- 172.21.15.40:0/3771099447 >> v1:172.21.15.40:6789/0 conn(0x7f0cd00cd930 legacy=0x7f0cd00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0cce7fc640 1 -- 172.21.15.40:0/3771099447 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961643648 0 0) 0x7f0cd00d10d0 con 0x7f0cd00cd930 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0ccf7fe640 1 -- 172.21.15.40:0/3771099447 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd00d1370 con 0x7f0cd0162810 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.318+0000 7f0cceffd640 1 --2- 172.21.15.40:0/3771099447 >> v2:172.21.15.184:3300/0 conn(0x7f0cd00b2340 0x7f0cd00cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.319+0000 7f0ccf7fe640 1 --2- 172.21.15.40:0/3771099447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd0162810 0x7f0cd0070350 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00cd0a0 tx=0x7f0cc002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef77bcdcfe4576a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.319+0000 7f0cce7fc640 1 -- 172.21.15.40:0/3771099447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0037070 con 0x7f0cd0162810 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.319+0000 7f0cce7fc640 1 -- 172.21.15.40:0/3771099447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc0032440 con 0x7f0cd0162810 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.319+0000 7f0cce7fc640 1 -- 172.21.15.40:0/3771099447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0032810 con 0x7f0cd0162810 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.320+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/3771099447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd0162810 msgr2=0x7f0cd0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.320+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/3771099447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd0162810 0x7f0cd0070350 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00cd0a0 tx=0x7f0cc002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.320+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/3771099447 shutdown_connections 2024-04-26T21:37:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.320+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/3771099447 >> v2:172.21.15.184:3300/0 conn(0x7f0cd00b2340 0x7f0cd00cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.320+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/3771099447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd0162810 0x7f0cd0070350 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.320+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/3771099447 >> 172.21.15.40:0/3771099447 conn(0x7f0cd00d2da0 msgr2=0x7f0cd00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.320+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/3771099447 shutdown_connections 2024-04-26T21:37:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.320+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/3771099447 wait complete. 2024-04-26T21:37:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.321+0000 7f0cd5ff0640 1 Processor -- start 2024-04-26T21:37:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.321+0000 7f0cd5ff0640 1 -- start start 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cd5ff0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd00b2340 0x7f0cd0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cd5ff0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00cd930 0x7f0cd00c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cd5ff0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd0162810 0x7f0cd00cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cd5ff0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd0070890 con 0x7f0cd00cd930 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cd5ff0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd00d0470 con 0x7f0cd0162810 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cd5ff0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd00d2240 con 0x7f0cd00b2340 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00cd930 0x7f0cd00c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00cd930 0x7f0cd00c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57080/0 (socket says 172.21.15.40:57080) 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cceffd640 1 -- 172.21.15.40:0/1315726220 learned_addr learned my addr 172.21.15.40:0/1315726220 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0ccf7fe640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd00b2340 0x7f0cd0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0ccffff640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd0162810 0x7f0cd00cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cceffd640 1 -- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd00b2340 msgr2=0x7f0cd0152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cceffd640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd00b2340 0x7f0cd0152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cceffd640 1 -- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd0162810 msgr2=0x7f0cd00cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cceffd640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd0162810 0x7f0cd00cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0cceffd640 1 -- 172.21.15.40:0/1315726220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd0152770 con 0x7f0cd00cd930 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.322+0000 7f0ccffff640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd0162810 0x7f0cd00cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.323+0000 7f0cceffd640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00cd930 0x7f0cd00c72e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c820 tx=0x7f0cc400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.323+0000 7f0cb7fff640 1 -- 172.21.15.40:0/1315726220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4015070 con 0x7f0cd00cd930 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.323+0000 7f0cb7fff640 1 -- 172.21.15.40:0/1315726220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc4007a30 con 0x7f0cd00cd930 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.323+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd015e780 con 0x7f0cd00cd930 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.323+0000 7f0cb7fff640 1 -- 172.21.15.40:0/1315726220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4007e00 con 0x7f0cd00cd930 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.323+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd00c7f10 con 0x7f0cd00cd930 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.324+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd00694f0 con 0x7f0cd00cd930 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.324+0000 7f0cb7fff640 1 -- 172.21.15.40:0/1315726220 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cc4018070 con 0x7f0cd00cd930 2024-04-26T21:37:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.325+0000 7f0cb7fff640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c920 0x7f0cbc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.325+0000 7f0cb7fff640 1 -- 172.21.15.40:0/1315726220 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f0cc408bc10 con 0x7f0cd00cd930 2024-04-26T21:37:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.325+0000 7f0ccf7fe640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c920 0x7f0cbc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.326+0000 7f0ccf7fe640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c920 0x7f0cbc06ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0034000 tx=0x7f0cc002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.329+0000 7f0cb7fff640 1 -- 172.21.15.40:0/1315726220 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0cd0152770 con 0x7f0cd00cd930 2024-04-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.513+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cd015e950 con 0x7f0cbc06c920 2024-04-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.519+0000 7f0cb7fff640 1 -- 172.21.15.40:0/1315726220 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246638 (secure 0 0 0) 0x7f0cd015e950 con 0x7f0cbc06c920 2024-04-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:42.523 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c920 msgr2=0x7f0cbc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c920 0x7f0cbc06ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0034000 tx=0x7f0cc002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00cd930 msgr2=0x7f0cd00c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00cd930 0x7f0cd00c72e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c820 tx=0x7f0cc400cc20 comp rx=0 tx=0).stop 2024-04-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 shutdown_connections 2024-04-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c920 0x7f0cbc06ede0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd0162810 0x7f0cd00cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00cd930 0x7f0cd00c72e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 --2- 172.21.15.40:0/1315726220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd00b2340 0x7f0cd0152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 >> 172.21.15.40:0/1315726220 conn(0x7f0cd00d2da0 msgr2=0x7f0cd00b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:42.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 shutdown_connections 2024-04-26T21:37:42.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.522+0000 7f0cd5ff0640 1 -- 172.21.15.40:0/1315726220 wait complete. 2024-04-26T21:37:42.533 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-04-26T21:37:42.029971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222012,"kb_used_data":5800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752708,"statfs":{"total":773094113280,"available":772866772992,"internally_reserved":0,"allocated":5939200,"data_stored":3288896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.900699"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.146297+0000","last_change":"2024-04-26T21:37:37.146297+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.146297+0000","last_undegraded":"2024-04-26T21:37:37.146297+0000","last_fullsized":"2024-04-26T21:37:37.146297+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138776+0000","last_change":"2024-04-26T21:37:37.138776+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138776+0000","last_undegraded":"2024-04-26T21:37:37.138776+0000","last_fullsized":"2024-04-26T21:37:37.138776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140361+0000","last_change":"2024-04-26T21:37:37.140361+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140361+0000","last_undegraded":"2024-04-26T21:37:37.140361+0000","last_fullsized":"2024-04-26T21:37:37.140361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139226+0000","last_change":"2024-04-26T21:37:37.139226+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139226+0000","last_undegraded":"2024-04-26T21:37:37.139226+0000","last_fullsized":"2024-04-26T21:37:37.139226+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137175+0000","last_change":"2024-04-26T21:37:37.137175+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137175+0000","last_undegraded":"2024-04-26T21:37:37.137175+0000","last_fullsized":"2024-04-26T21:37:37.137175+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.147679+0000","last_change":"2024-04-26T21:37:37.147679+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.147679+0000","last_undegraded":"2024-04-26T21:37:37.147679+0000","last_fullsized":"2024-04-26T21:37:37.147679+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139709+0000","last_change":"2024-04-26T21:37:37.139709+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139709+0000","last_undegraded":"2024-04-26T21:37:37.139709+0000","last_fullsized":"2024-04-26T21:37:37.139709+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139690+0000","last_change":"2024-04-26T21:37:37.139690+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139690+0000","last_undegraded":"2024-04-26T21:37:37.139690+0000","last_fullsized":"2024-04-26T21:37:37.139690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139335+0000","last_change":"2024-04-26T21:37:37.139335+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139335+0000","last_undegraded":"2024-04-26T21:37:37.139335+0000","last_fullsized":"2024-04-26T21:37:37.139335+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.148501+0000","last_change":"2024-04-26T21:37:37.148501+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.148501+0000","last_undegraded":"2024-04-26T21:37:37.148501+0000","last_fullsized":"2024-04-26T21:37:37.148501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143003+0000","last_change":"2024-04-26T21:37:37.143003+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143003+0000","last_undegraded":"2024-04-26T21:37:37.143003+0000","last_fullsized":"2024-04-26T21:37:37.143003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139262+0000","last_change":"2024-04-26T21:37:37.139262+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139262+0000","last_undegraded":"2024-04-26T21:37:37.139262+0000","last_fullsized":"2024-04-26T21:37:37.139262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145421+0000","last_change":"2024-04-26T21:37:37.145421+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145421+0000","last_undegraded":"2024-04-26T21:37:37.145421+0000","last_fullsized":"2024-04-26T21:37:37.145421+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140247+0000","last_change":"2024-04-26T21:37:37.140247+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140247+0000","last_undegraded":"2024-04-26T21:37:37.140247+0000","last_fullsized":"2024-04-26T21:37:37.140247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138351+0000","last_change":"2024-04-26T21:37:37.138351+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138351+0000","last_undegraded":"2024-04-26T21:37:37.138351+0000","last_fullsized":"2024-04-26T21:37:37.138351+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143870+0000","last_change":"2024-04-26T21:37:37.143870+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143870+0000","last_undegraded":"2024-04-26T21:37:37.143870+0000","last_fullsized":"2024-04-26T21:37:37.143870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144920+0000","last_change":"2024-04-26T21:37:37.144920+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144920+0000","last_undegraded":"2024-04-26T21:37:37.144920+0000","last_fullsized":"2024-04-26T21:37:37.144920+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138328+0000","last_change":"2024-04-26T21:37:37.138328+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138328+0000","last_undegraded":"2024-04-26T21:37:37.138328+0000","last_fullsized":"2024-04-26T21:37:37.138328+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139643+0000","last_change":"2024-04-26T21:37:37.139643+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139643+0000","last_undegraded":"2024-04-26T21:37:37.139643+0000","last_fullsized":"2024-04-26T21:37:37.139643+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143012+0000","last_change":"2024-04-26T21:37:37.143012+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143012+0000","last_undegraded":"2024-04-26T21:37:37.143012+0000","last_fullsized":"2024-04-26T21:37:37.143012+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140772+0000","last_change":"2024-04-26T21:37:37.140772+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140772+0000","last_undegraded":"2024-04-26T21:37:37.140772+0000","last_fullsized":"2024-04-26T21:37:37.140772+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138725+0000","last_change":"2024-04-26T21:37:37.138725+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138725+0000","last_undegraded":"2024-04-26T21:37:37.138725+0000","last_fullsized":"2024-04-26T21:37:37.138725+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137913+0000","last_change":"2024-04-26T21:37:37.137913+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137913+0000","last_undegraded":"2024-04-26T21:37:37.137913+0000","last_fullsized":"2024-04-26T21:37:37.137913+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138780+0000","last_change":"2024-04-26T21:37:37.138780+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138780+0000","last_undegraded":"2024-04-26T21:37:37.138780+0000","last_fullsized":"2024-04-26T21:37:37.138780+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144498+0000","last_change":"2024-04-26T21:37:37.144498+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144498+0000","last_undegraded":"2024-04-26T21:37:37.144498+0000","last_fullsized":"2024-04-26T21:37:37.144498+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138226+0000","last_change":"2024-04-26T21:37:37.138226+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138226+0000","last_undegraded":"2024-04-26T21:37:37.138226+0000","last_fullsized":"2024-04-26T21:37:37.138226+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.149883+0000","last_change":"2024-04-26T21:37:37.149883+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.149883+0000","last_undegraded":"2024-04-26T21:37:37.149883+0000","last_fullsized":"2024-04-26T21:37:37.149883+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.142613+0000","last_change":"2024-04-26T21:37:37.142613+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.142613+0000","last_undegraded":"2024-04-26T21:37:37.142613+0000","last_fullsized":"2024-04-26T21:37:37.142613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144029+0000","last_change":"2024-04-26T21:37:37.144029+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144029+0000","last_undegraded":"2024-04-26T21:37:37.144029+0000","last_fullsized":"2024-04-26T21:37:37.144029+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138351+0000","last_change":"2024-04-26T21:37:37.138351+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138351+0000","last_undegraded":"2024-04-26T21:37:37.138351+0000","last_fullsized":"2024-04-26T21:37:37.138351+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138627+0000","last_change":"2024-04-26T21:37:37.138627+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138627+0000","last_undegraded":"2024-04-26T21:37:37.138627+0000","last_fullsized":"2024-04-26T21:37:37.138627+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.135376+0000","last_change":"2024-04-26T21:37:37.135376+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.135376+0000","last_undegraded":"2024-04-26T21:37:37.135376+0000","last_fullsized":"2024-04-26T21:37:37.135376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138262+0000","last_change":"2024-04-26T21:37:37.138262+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138262+0000","last_undegraded":"2024-04-26T21:37:37.138262+0000","last_fullsized":"2024-04-26T21:37:37.138262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155881+0000","last_change":"2024-04-26T21:37:37.150925+0000","last_active":"2024-04-26T21:37:37.155881+0000","last_peered":"2024-04-26T21:37:37.155881+0000","last_clean":"2024-04-26T21:37:37.155881+0000","last_became_active":"2024-04-26T21:37:37.150597+0000","last_became_peered":"2024-04-26T21:37:37.150597+0000","last_unstale":"2024-04-26T21:37:37.155881+0000","last_undegraded":"2024-04-26T21:37:37.155881+0000","last_fullsized":"2024-04-26T21:37:37.155881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:53.683181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134054+0000","last_change":"2024-04-26T21:37:37.134054+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134054+0000","last_undegraded":"2024-04-26T21:37:37.134054+0000","last_fullsized":"2024-04-26T21:37:37.134054+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159403+0000","last_change":"2024-04-26T21:37:37.156594+0000","last_active":"2024-04-26T21:37:37.159403+0000","last_peered":"2024-04-26T21:37:37.159403+0000","last_clean":"2024-04-26T21:37:37.159403+0000","last_became_active":"2024-04-26T21:37:37.156312+0000","last_became_peered":"2024-04-26T21:37:37.156312+0000","last_unstale":"2024-04-26T21:37:37.159403+0000","last_undegraded":"2024-04-26T21:37:37.159403+0000","last_fullsized":"2024-04-26T21:37:37.159403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:43.766358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.142303+0000","last_change":"2024-04-26T21:37:37.142303+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.142303+0000","last_undegraded":"2024-04-26T21:37:37.142303+0000","last_fullsized":"2024-04-26T21:37:37.142303+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.162010+0000","last_change":"2024-04-26T21:37:37.158086+0000","last_active":"2024-04-26T21:37:37.162010+0000","last_peered":"2024-04-26T21:37:37.162010+0000","last_clean":"2024-04-26T21:37:37.162010+0000","last_became_active":"2024-04-26T21:37:37.157790+0000","last_became_peered":"2024-04-26T21:37:37.157790+0000","last_unstale":"2024-04-26T21:37:37.162010+0000","last_undegraded":"2024-04-26T21:37:37.162010+0000","last_fullsized":"2024-04-26T21:37:37.162010+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.123343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136890+0000","last_change":"2024-04-26T21:37:37.136890+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136890+0000","last_undegraded":"2024-04-26T21:37:37.136890+0000","last_fullsized":"2024-04-26T21:37:37.136890+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.147649+0000","last_change":"2024-04-26T21:37:37.140298+0000","last_active":"2024-04-26T21:37:37.147649+0000","last_peered":"2024-04-26T21:37:37.147649+0000","last_clean":"2024-04-26T21:37:37.147649+0000","last_became_active":"2024-04-26T21:37:37.139801+0000","last_became_peered":"2024-04-26T21:37:37.139801+0000","last_unstale":"2024-04-26T21:37:37.147649+0000","last_undegraded":"2024-04-26T21:37:37.147649+0000","last_fullsized":"2024-04-26T21:37:37.147649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137107+0000","last_change":"2024-04-26T21:37:37.137107+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137107+0000","last_undegraded":"2024-04-26T21:37:37.137107+0000","last_fullsized":"2024-04-26T21:37:37.137107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.162038+0000","last_change":"2024-04-26T21:37:37.158278+0000","last_active":"2024-04-26T21:37:37.162038+0000","last_peered":"2024-04-26T21:37:37.162038+0000","last_clean":"2024-04-26T21:37:37.162038+0000","last_became_active":"2024-04-26T21:37:37.154700+0000","last_became_peered":"2024-04-26T21:37:37.154700+0000","last_unstale":"2024-04-26T21:37:37.162038+0000","last_undegraded":"2024-04-26T21:37:37.162038+0000","last_fullsized":"2024-04-26T21:37:37.162038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:30.231390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139564+0000","last_change":"2024-04-26T21:37:37.139564+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139564+0000","last_undegraded":"2024-04-26T21:37:37.139564+0000","last_fullsized":"2024-04-26T21:37:37.139564+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.152274+0000","last_change":"2024-04-26T21:37:37.148427+0000","last_active":"2024-04-26T21:37:37.152274+0000","last_peered":"2024-04-26T21:37:37.152274+0000","last_clean":"2024-04-26T21:37:37.152274+0000","last_became_active":"2024-04-26T21:37:37.147872+0000","last_became_peered":"2024-04-26T21:37:37.147872+0000","last_unstale":"2024-04-26T21:37:37.152274+0000","last_undegraded":"2024-04-26T21:37:37.152274+0000","last_fullsized":"2024-04-26T21:37:37.152274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:14.846091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136935+0000","last_change":"2024-04-26T21:37:37.136935+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136935+0000","last_undegraded":"2024-04-26T21:37:37.136935+0000","last_fullsized":"2024-04-26T21:37:37.136935+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.149727+0000","last_change":"2024-04-26T21:37:37.147914+0000","last_active":"2024-04-26T21:37:37.149727+0000","last_peered":"2024-04-26T21:37:37.149727+0000","last_clean":"2024-04-26T21:37:37.149727+0000","last_became_active":"2024-04-26T21:37:37.147592+0000","last_became_peered":"2024-04-26T21:37:37.147592+0000","last_unstale":"2024-04-26T21:37:37.149727+0000","last_undegraded":"2024-04-26T21:37:37.149727+0000","last_fullsized":"2024-04-26T21:37:37.149727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:50.841990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138903+0000","last_change":"2024-04-26T21:37:37.138903+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138903+0000","last_undegraded":"2024-04-26T21:37:37.138903+0000","last_fullsized":"2024-04-26T21:37:37.138903+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158742+0000","last_change":"2024-04-26T21:37:37.153776+0000","last_active":"2024-04-26T21:37:37.158742+0000","last_peered":"2024-04-26T21:37:37.158742+0000","last_clean":"2024-04-26T21:37:37.158742+0000","last_became_active":"2024-04-26T21:37:37.153390+0000","last_became_peered":"2024-04-26T21:37:37.153390+0000","last_unstale":"2024-04-26T21:37:37.158742+0000","last_undegraded":"2024-04-26T21:37:37.158742+0000","last_fullsized":"2024-04-26T21:37:37.158742+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:43.797895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136442+0000","last_change":"2024-04-26T21:37:37.136442+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136442+0000","last_undegraded":"2024-04-26T21:37:37.136442+0000","last_fullsized":"2024-04-26T21:37:37.136442+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.154501+0000","last_change":"2024-04-26T21:37:37.153096+0000","last_active":"2024-04-26T21:37:37.154501+0000","last_peered":"2024-04-26T21:37:37.154501+0000","last_clean":"2024-04-26T21:37:37.154501+0000","last_became_active":"2024-04-26T21:37:37.152795+0000","last_became_peered":"2024-04-26T21:37:37.152795+0000","last_unstale":"2024-04-26T21:37:37.154501+0000","last_undegraded":"2024-04-26T21:37:37.154501+0000","last_fullsized":"2024-04-26T21:37:37.154501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:48.571013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134754+0000","last_change":"2024-04-26T21:37:37.134754+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134754+0000","last_undegraded":"2024-04-26T21:37:37.134754+0000","last_fullsized":"2024-04-26T21:37:37.134754+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158396+0000","last_change":"2024-04-26T21:37:37.154318+0000","last_active":"2024-04-26T21:37:37.158396+0000","last_peered":"2024-04-26T21:37:37.158396+0000","last_clean":"2024-04-26T21:37:37.158396+0000","last_became_active":"2024-04-26T21:37:37.154072+0000","last_became_peered":"2024-04-26T21:37:37.154072+0000","last_unstale":"2024-04-26T21:37:37.158396+0000","last_undegraded":"2024-04-26T21:37:37.158396+0000","last_fullsized":"2024-04-26T21:37:37.158396+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:08.561240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138381+0000","last_change":"2024-04-26T21:37:37.138381+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138381+0000","last_undegraded":"2024-04-26T21:37:37.138381+0000","last_fullsized":"2024-04-26T21:37:37.138381+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.156365+0000","last_change":"2024-04-26T21:37:37.154992+0000","last_active":"2024-04-26T21:37:37.156365+0000","last_peered":"2024-04-26T21:37:37.156365+0000","last_clean":"2024-04-26T21:37:37.156365+0000","last_became_active":"2024-04-26T21:37:37.154711+0000","last_became_peered":"2024-04-26T21:37:37.154711+0000","last_unstale":"2024-04-26T21:37:37.156365+0000","last_undegraded":"2024-04-26T21:37:37.156365+0000","last_fullsized":"2024-04-26T21:37:37.156365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:04.317873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136779+0000","last_change":"2024-04-26T21:37:37.136779+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136779+0000","last_undegraded":"2024-04-26T21:37:37.136779+0000","last_fullsized":"2024-04-26T21:37:37.136779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155175+0000","last_change":"2024-04-26T21:37:37.146807+0000","last_active":"2024-04-26T21:37:37.155175+0000","last_peered":"2024-04-26T21:37:37.155175+0000","last_clean":"2024-04-26T21:37:37.155175+0000","last_became_active":"2024-04-26T21:37:37.146479+0000","last_became_peered":"2024-04-26T21:37:37.146479+0000","last_unstale":"2024-04-26T21:37:37.155175+0000","last_undegraded":"2024-04-26T21:37:37.155175+0000","last_fullsized":"2024-04-26T21:37:37.155175+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.135431+0000","last_change":"2024-04-26T21:37:37.135431+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.135431+0000","last_undegraded":"2024-04-26T21:37:37.135431+0000","last_fullsized":"2024-04-26T21:37:37.135431+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158195+0000","last_change":"2024-04-26T21:37:37.152397+0000","last_active":"2024-04-26T21:37:37.158195+0000","last_peered":"2024-04-26T21:37:37.158195+0000","last_clean":"2024-04-26T21:37:37.158195+0000","last_became_active":"2024-04-26T21:37:37.152118+0000","last_became_peered":"2024-04-26T21:37:37.152118+0000","last_unstale":"2024-04-26T21:37:37.158195+0000","last_undegraded":"2024-04-26T21:37:37.158195+0000","last_fullsized":"2024-04-26T21:37:37.158195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:50.687873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139183+0000","last_change":"2024-04-26T21:37:37.139183+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139183+0000","last_undegraded":"2024-04-26T21:37:37.139183+0000","last_fullsized":"2024-04-26T21:37:37.139183+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158451+0000","last_change":"2024-04-26T21:37:37.155533+0000","last_active":"2024-04-26T21:37:37.158451+0000","last_peered":"2024-04-26T21:37:37.158451+0000","last_clean":"2024-04-26T21:37:37.158451+0000","last_became_active":"2024-04-26T21:37:37.155179+0000","last_became_peered":"2024-04-26T21:37:37.155179+0000","last_unstale":"2024-04-26T21:37:37.158451+0000","last_undegraded":"2024-04-26T21:37:37.158451+0000","last_fullsized":"2024-04-26T21:37:37.158451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:14.846091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137624+0000","last_change":"2024-04-26T21:37:37.137624+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137624+0000","last_undegraded":"2024-04-26T21:37:37.137624+0000","last_fullsized":"2024-04-26T21:37:37.137624+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.152746+0000","last_change":"2024-04-26T21:37:37.148935+0000","last_active":"2024-04-26T21:37:37.152746+0000","last_peered":"2024-04-26T21:37:37.152746+0000","last_clean":"2024-04-26T21:37:37.152746+0000","last_became_active":"2024-04-26T21:37:37.148635+0000","last_became_peered":"2024-04-26T21:37:37.148635+0000","last_unstale":"2024-04-26T21:37:37.152746+0000","last_undegraded":"2024-04-26T21:37:37.152746+0000","last_fullsized":"2024-04-26T21:37:37.152746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.883299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145387+0000","last_change":"2024-04-26T21:37:37.145387+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145387+0000","last_undegraded":"2024-04-26T21:37:37.145387+0000","last_fullsized":"2024-04-26T21:37:37.145387+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155439+0000","last_change":"2024-04-26T21:37:37.147944+0000","last_active":"2024-04-26T21:37:37.155439+0000","last_peered":"2024-04-26T21:37:37.155439+0000","last_clean":"2024-04-26T21:37:37.155439+0000","last_became_active":"2024-04-26T21:37:37.147675+0000","last_became_peered":"2024-04-26T21:37:37.147675+0000","last_unstale":"2024-04-26T21:37:37.155439+0000","last_undegraded":"2024-04-26T21:37:37.155439+0000","last_fullsized":"2024-04-26T21:37:37.155439+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:20.163566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136315+0000","last_change":"2024-04-26T21:37:37.136315+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136315+0000","last_undegraded":"2024-04-26T21:37:37.136315+0000","last_fullsized":"2024-04-26T21:37:37.136315+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158568+0000","last_change":"2024-04-26T21:37:37.153638+0000","last_active":"2024-04-26T21:37:37.158568+0000","last_peered":"2024-04-26T21:37:37.158568+0000","last_clean":"2024-04-26T21:37:37.158568+0000","last_became_active":"2024-04-26T21:37:37.153152+0000","last_became_peered":"2024-04-26T21:37:37.153152+0000","last_unstale":"2024-04-26T21:37:37.158568+0000","last_undegraded":"2024-04-26T21:37:37.158568+0000","last_fullsized":"2024-04-26T21:37:37.158568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:31.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144651+0000","last_change":"2024-04-26T21:37:37.144651+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144651+0000","last_undegraded":"2024-04-26T21:37:37.144651+0000","last_fullsized":"2024-04-26T21:37:37.144651+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.157061+0000","last_change":"2024-04-26T21:37:37.155432+0000","last_active":"2024-04-26T21:37:37.157061+0000","last_peered":"2024-04-26T21:37:37.157061+0000","last_clean":"2024-04-26T21:37:37.157061+0000","last_became_active":"2024-04-26T21:37:37.155163+0000","last_became_peered":"2024-04-26T21:37:37.155163+0000","last_unstale":"2024-04-26T21:37:37.157061+0000","last_undegraded":"2024-04-26T21:37:37.157061+0000","last_fullsized":"2024-04-26T21:37:37.157061+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:34.248752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136604+0000","last_change":"2024-04-26T21:37:37.136604+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136604+0000","last_undegraded":"2024-04-26T21:37:37.136604+0000","last_fullsized":"2024-04-26T21:37:37.136604+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.161212+0000","last_change":"2024-04-26T21:37:37.155985+0000","last_active":"2024-04-26T21:37:37.161212+0000","last_peered":"2024-04-26T21:37:37.161212+0000","last_clean":"2024-04-26T21:37:37.161212+0000","last_became_active":"2024-04-26T21:37:37.155643+0000","last_became_peered":"2024-04-26T21:37:37.155643+0000","last_unstale":"2024-04-26T21:37:37.161212+0000","last_undegraded":"2024-04-26T21:37:37.161212+0000","last_fullsized":"2024-04-26T21:37:37.161212+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:48.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145020+0000","last_change":"2024-04-26T21:37:37.145020+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145020+0000","last_undegraded":"2024-04-26T21:37:37.145020+0000","last_fullsized":"2024-04-26T21:37:37.145020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155757+0000","last_change":"2024-04-26T21:37:37.150088+0000","last_active":"2024-04-26T21:37:37.155757+0000","last_peered":"2024-04-26T21:37:37.155757+0000","last_clean":"2024-04-26T21:37:37.155757+0000","last_became_active":"2024-04-26T21:37:37.149820+0000","last_became_peered":"2024-04-26T21:37:37.149820+0000","last_unstale":"2024-04-26T21:37:37.155757+0000","last_undegraded":"2024-04-26T21:37:37.155757+0000","last_fullsized":"2024-04-26T21:37:37.155757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:56.039426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134473+0000","last_change":"2024-04-26T21:37:37.134473+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134473+0000","last_undegraded":"2024-04-26T21:37:37.134473+0000","last_fullsized":"2024-04-26T21:37:37.134473+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159394+0000","last_change":"2024-04-26T21:37:37.156465+0000","last_active":"2024-04-26T21:37:37.159394+0000","last_peered":"2024-04-26T21:37:37.159394+0000","last_clean":"2024-04-26T21:37:37.159394+0000","last_became_active":"2024-04-26T21:37:37.156194+0000","last_became_peered":"2024-04-26T21:37:37.156194+0000","last_unstale":"2024-04-26T21:37:37.159394+0000","last_undegraded":"2024-04-26T21:37:37.159394+0000","last_fullsized":"2024-04-26T21:37:37.159394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:28.173221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.141743+0000","last_change":"2024-04-26T21:37:37.141743+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.141743+0000","last_undegraded":"2024-04-26T21:37:37.141743+0000","last_fullsized":"2024-04-26T21:37:37.141743+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150052+0000","last_change":"2024-04-26T21:37:37.143737+0000","last_active":"2024-04-26T21:37:37.150052+0000","last_peered":"2024-04-26T21:37:37.150052+0000","last_clean":"2024-04-26T21:37:37.150052+0000","last_became_active":"2024-04-26T21:37:37.143432+0000","last_became_peered":"2024-04-26T21:37:37.143432+0000","last_unstale":"2024-04-26T21:37:37.150052+0000","last_undegraded":"2024-04-26T21:37:37.150052+0000","last_fullsized":"2024-04-26T21:37:37.150052+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:48.659187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139896+0000","last_change":"2024-04-26T21:37:37.139896+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139896+0000","last_undegraded":"2024-04-26T21:37:37.139896+0000","last_fullsized":"2024-04-26T21:37:37.139896+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158712+0000","last_change":"2024-04-26T21:37:37.156130+0000","last_active":"2024-04-26T21:37:37.158712+0000","last_peered":"2024-04-26T21:37:37.158712+0000","last_clean":"2024-04-26T21:37:37.158712+0000","last_became_active":"2024-04-26T21:37:37.154520+0000","last_became_peered":"2024-04-26T21:37:37.154520+0000","last_unstale":"2024-04-26T21:37:37.158712+0000","last_undegraded":"2024-04-26T21:37:37.158712+0000","last_fullsized":"2024-04-26T21:37:37.158712+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:38.183624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145831+0000","last_change":"2024-04-26T21:37:37.145831+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145831+0000","last_undegraded":"2024-04-26T21:37:37.145831+0000","last_fullsized":"2024-04-26T21:37:37.145831+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150738+0000","last_change":"2024-04-26T21:37:37.145621+0000","last_active":"2024-04-26T21:37:37.150738+0000","last_peered":"2024-04-26T21:37:37.150738+0000","last_clean":"2024-04-26T21:37:37.150738+0000","last_became_active":"2024-04-26T21:37:37.145215+0000","last_became_peered":"2024-04-26T21:37:37.145215+0000","last_unstale":"2024-04-26T21:37:37.150738+0000","last_undegraded":"2024-04-26T21:37:37.150738+0000","last_fullsized":"2024-04-26T21:37:37.150738+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.363875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134531+0000","last_change":"2024-04-26T21:37:37.134531+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134531+0000","last_undegraded":"2024-04-26T21:37:37.134531+0000","last_fullsized":"2024-04-26T21:37:37.134531+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.161206+0000","last_change":"2024-04-26T21:37:37.155721+0000","last_active":"2024-04-26T21:37:37.161206+0000","last_peered":"2024-04-26T21:37:37.161206+0000","last_clean":"2024-04-26T21:37:37.161206+0000","last_became_active":"2024-04-26T21:37:37.155438+0000","last_became_peered":"2024-04-26T21:37:37.155438+0000","last_unstale":"2024-04-26T21:37:37.161206+0000","last_undegraded":"2024-04-26T21:37:37.161206+0000","last_fullsized":"2024-04-26T21:37:37.161206+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:08.561240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.147246+0000","last_change":"2024-04-26T21:37:37.147246+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.147246+0000","last_undegraded":"2024-04-26T21:37:37.147246+0000","last_fullsized":"2024-04-26T21:37:37.147246+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.153071+0000","last_change":"2024-04-26T21:37:37.151412+0000","last_active":"2024-04-26T21:37:37.153071+0000","last_peered":"2024-04-26T21:37:37.153071+0000","last_clean":"2024-04-26T21:37:37.153071+0000","last_became_active":"2024-04-26T21:37:37.151157+0000","last_became_peered":"2024-04-26T21:37:37.151157+0000","last_unstale":"2024-04-26T21:37:37.153071+0000","last_undegraded":"2024-04-26T21:37:37.153071+0000","last_fullsized":"2024-04-26T21:37:37.153071+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:30.231390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136343+0000","last_change":"2024-04-26T21:37:37.136343+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136343+0000","last_undegraded":"2024-04-26T21:37:37.136343+0000","last_fullsized":"2024-04-26T21:37:37.136343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158678+0000","last_change":"2024-04-26T21:37:37.154574+0000","last_active":"2024-04-26T21:37:37.158678+0000","last_peered":"2024-04-26T21:37:37.158678+0000","last_clean":"2024-04-26T21:37:37.158678+0000","last_became_active":"2024-04-26T21:37:37.153518+0000","last_became_peered":"2024-04-26T21:37:37.153518+0000","last_unstale":"2024-04-26T21:37:37.158678+0000","last_undegraded":"2024-04-26T21:37:37.158678+0000","last_fullsized":"2024-04-26T21:37:37.158678+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:43.168309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138567+0000","last_change":"2024-04-26T21:37:37.138567+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138567+0000","last_undegraded":"2024-04-26T21:37:37.138567+0000","last_fullsized":"2024-04-26T21:37:37.138567+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.151260+0000","last_change":"2024-04-26T21:37:37.144706+0000","last_active":"2024-04-26T21:37:37.151260+0000","last_peered":"2024-04-26T21:37:37.151260+0000","last_clean":"2024-04-26T21:37:37.151260+0000","last_became_active":"2024-04-26T21:37:37.144369+0000","last_became_peered":"2024-04-26T21:37:37.144369+0000","last_unstale":"2024-04-26T21:37:37.151260+0000","last_undegraded":"2024-04-26T21:37:37.151260+0000","last_fullsized":"2024-04-26T21:37:37.151260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134559+0000","last_change":"2024-04-26T21:37:37.134559+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134559+0000","last_undegraded":"2024-04-26T21:37:37.134559+0000","last_fullsized":"2024-04-26T21:37:37.134559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.157301+0000","last_change":"2024-04-26T21:37:37.153455+0000","last_active":"2024-04-26T21:37:37.157301+0000","last_peered":"2024-04-26T21:37:37.157301+0000","last_clean":"2024-04-26T21:37:37.157301+0000","last_became_active":"2024-04-26T21:37:37.153181+0000","last_became_peered":"2024-04-26T21:37:37.153181+0000","last_unstale":"2024-04-26T21:37:37.157301+0000","last_undegraded":"2024-04-26T21:37:37.157301+0000","last_fullsized":"2024-04-26T21:37:37.157301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:47.599920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140819+0000","last_change":"2024-04-26T21:37:37.140819+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140819+0000","last_undegraded":"2024-04-26T21:37:37.140819+0000","last_fullsized":"2024-04-26T21:37:37.140819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150089+0000","last_change":"2024-04-26T21:37:37.144882+0000","last_active":"2024-04-26T21:37:37.150089+0000","last_peered":"2024-04-26T21:37:37.150089+0000","last_clean":"2024-04-26T21:37:37.150089+0000","last_became_active":"2024-04-26T21:37:37.144633+0000","last_became_peered":"2024-04-26T21:37:37.144633+0000","last_unstale":"2024-04-26T21:37:37.150089+0000","last_undegraded":"2024-04-26T21:37:37.150089+0000","last_fullsized":"2024-04-26T21:37:37.150089+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:53.683181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136338+0000","last_change":"2024-04-26T21:37:37.136338+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136338+0000","last_undegraded":"2024-04-26T21:37:37.136338+0000","last_fullsized":"2024-04-26T21:37:37.136338+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155915+0000","last_change":"2024-04-26T21:37:37.150695+0000","last_active":"2024-04-26T21:37:37.155915+0000","last_peered":"2024-04-26T21:37:37.155915+0000","last_clean":"2024-04-26T21:37:37.155915+0000","last_became_active":"2024-04-26T21:37:37.150398+0000","last_became_peered":"2024-04-26T21:37:37.150398+0000","last_unstale":"2024-04-26T21:37:37.155915+0000","last_undegraded":"2024-04-26T21:37:37.155915+0000","last_fullsized":"2024-04-26T21:37:37.155915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.141329+0000","last_change":"2024-04-26T21:37:37.141329+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.141329+0000","last_undegraded":"2024-04-26T21:37:37.141329+0000","last_fullsized":"2024-04-26T21:37:37.141329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159474+0000","last_change":"2024-04-26T21:37:37.156817+0000","last_active":"2024-04-26T21:37:37.159474+0000","last_peered":"2024-04-26T21:37:37.159474+0000","last_clean":"2024-04-26T21:37:37.159474+0000","last_became_active":"2024-04-26T21:37:37.156561+0000","last_became_peered":"2024-04-26T21:37:37.156561+0000","last_unstale":"2024-04-26T21:37:37.159474+0000","last_undegraded":"2024-04-26T21:37:37.159474+0000","last_fullsized":"2024-04-26T21:37:37.159474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.883299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":226,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.140105+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:37:37.140105+0000","last_peered":"2024-04-26T21:37:37.140105+0000","last_clean":"2024-04-26T21:37:37.140105+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:37:37.140105+0000","last_undegraded":"2024-04-26T21:37:37.140105+0000","last_fullsized":"2024-04-26T21:37:37.140105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:37:42.539 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:37:42.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.675+0000 7fbb33b12640 1 Processor -- start 2024-04-26T21:37:42.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.675+0000 7fbb33b12640 1 -- start start 2024-04-26T21:37:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb33b12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0b0380 0x7fbb2c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb33b12640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2c0cfd80 con 0x7fbb2c06c950 2024-04-26T21:37:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb33b12640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2c0d0020 con 0x7fbb2c0b10b0 2024-04-26T21:37:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb33b12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c0af8e0 con 0x7fbb2c0b0380 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb320af640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbb2c06c950 0x7fbb2c15e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60494/0 (socket says 172.21.15.40:60494) 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb320af640 1 -- 172.21.15.40:0/1166766400 learned_addr learned my addr 172.21.15.40:0/1166766400 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb318ae640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0b0380 0x7fbb2c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1044150210 0 0) 0x7fbb2c0cfd80 con 0x7fbb2c06c950 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb18003520 con 0x7fbb2c06c950 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714376489 0 0) 0x7fbb2c0d0020 con 0x7fbb2c0b10b0 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb2c0cfd80 con 0x7fbb2c0b10b0 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2494848470 0 0) 0x7fbb18003520 con 0x7fbb2c06c950 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0b0380 msgr2=0x7fbb2c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0b0380 0x7fbb2c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 >> v1:172.21.15.184:6789/0 conn(0x7fbb2c0b10b0 legacy=0x7fbb2c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c0d0300 con 0x7fbb2c06c950 2024-04-26T21:37:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fbb240019f0 con 0x7fbb2c06c950 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 >> v1:172.21.15.40:6789/0 conn(0x7fbb2c06c950 legacy=0x7fbb2c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0b0380 0x7fbb18003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb180040d0 0x7fbb18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb18004d40 0x7fbb180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c0af8e0 con 0x7fbb180040d0 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1800bb50 con 0x7fbb18004d40 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.676+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1800bd20 con 0x7fbb2c0b0380 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb310ad640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb180040d0 0x7fbb18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb318ae640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0b0380 0x7fbb18003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb320af640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb18004d40 0x7fbb180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb310ad640 1 -- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0b0380 msgr2=0x7fbb18003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb310ad640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0b0380 0x7fbb18003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb310ad640 1 -- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb18004d40 msgr2=0x7fbb180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb310ad640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb18004d40 0x7fbb180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb310ad640 1 -- 172.21.15.40:0/1166766400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb18003520 con 0x7fbb180040d0 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.677+0000 7fbb310ad640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb180040d0 0x7fbb18004800 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0018d0 tx=0x7fbb18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf98832c6d388215 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0014f0 con 0x7fbb180040d0 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0052d0 con 0x7fbb180040d0 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb308ac640 1 -- 172.21.15.40:0/1166766400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c02aca0 con 0x7fbb180040d0 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb33b12640 1 -- 172.21.15.40:0/1166766400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb180040d0 msgr2=0x7fbb18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb33b12640 1 --2- 172.21.15.40:0/1166766400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb180040d0 0x7fbb18004800 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0018d0 tx=0x7fbb18005830 comp rx=0 tx=0).stop 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb320af640 1 -- 172.21.15.40:0/1166766400 reap_dead start 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb33b12640 1 -- 172.21.15.40:0/1166766400 shutdown_connections 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb33b12640 1 -- 172.21.15.40:0/1166766400 >> 172.21.15.40:0/1166766400 conn(0x7fbb2c0d1cf0 msgr2=0x7fbb2c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb33b12640 1 -- 172.21.15.40:0/1166766400 shutdown_connections 2024-04-26T21:37:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.678+0000 7fbb33b12640 1 -- 172.21.15.40:0/1166766400 wait complete. 2024-04-26T21:37:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.679+0000 7fbb33b12640 1 Processor -- start 2024-04-26T21:37:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb33b12640 1 -- start start 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb33b12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb180040d0 0x7fbb2c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb33b12640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c070810 0x7fbb2c070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb33b12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2c150ec0 0x7fbb2c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb33b12640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c0b0760 con 0x7fbb2c150ec0 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb33b12640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c0d1980 con 0x7fbb2c070810 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb33b12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c0d15c0 con 0x7fbb180040d0 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb320af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2c150ec0 0x7fbb2c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb320af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2c150ec0 0x7fbb2c151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57092/0 (socket says 172.21.15.40:57092) 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb320af640 1 -- 172.21.15.40:0/711940337 learned_addr learned my addr 172.21.15.40:0/711940337 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb310ad640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c070810 0x7fbb2c070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.680+0000 7fbb318ae640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb180040d0 0x7fbb2c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb320af640 1 -- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb180040d0 msgr2=0x7fbb2c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb320af640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb180040d0 0x7fbb2c0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb320af640 1 -- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c070810 msgr2=0x7fbb2c070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb320af640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c070810 0x7fbb2c070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb320af640 1 -- 172.21.15.40:0/711940337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c0cb350 con 0x7fbb2c150ec0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb310ad640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c070810 0x7fbb2c070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb320af640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2c150ec0 0x7fbb2c151340 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbb24002230 tx=0x7fbb24007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb227fc640 1 -- 172.21.15.40:0/711940337 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400ab30 con 0x7fbb2c150ec0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb227fc640 1 -- 172.21.15.40:0/711940337 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb24018070 con 0x7fbb2c150ec0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c0cb630 con 0x7fbb2c150ec0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb227fc640 1 -- 172.21.15.40:0/711940337 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb240133f0 con 0x7fbb2c150ec0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.681+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c0cbbf0 con 0x7fbb2c150ec0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.682+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb2c0d0b00 con 0x7fbb2c150ec0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.682+0000 7fbb227fc640 1 -- 172.21.15.40:0/711940337 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb24013590 con 0x7fbb2c150ec0 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.683+0000 7fbb227fc640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb1006c920 0x7fbb1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.683+0000 7fbb227fc640 1 -- 172.21.15.40:0/711940337 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fbb2408c190 con 0x7fbb2c150ec0 2024-04-26T21:37:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.683+0000 7fbb318ae640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb1006c920 0x7fbb1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.684+0000 7fbb318ae640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb1006c920 0x7fbb1006ede0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009a20 tx=0x7fbb28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.687+0000 7fbb227fc640 1 -- 172.21.15.40:0/711940337 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb2c0d0b00 con 0x7fbb2c150ec0 2024-04-26T21:37:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.867+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb2c0694f0 con 0x7fbb1006c920 2024-04-26T21:37:42.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.873+0000 7fbb227fc640 1 -- 172.21.15.40:0/711940337 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246638 (secure 0 0 0) 0x7fbb2c0694f0 con 0x7fbb1006c920 2024-04-26T21:37:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:42.877 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb1006c920 msgr2=0x7fbb1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb1006c920 0x7fbb1006ede0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009a20 tx=0x7fbb28008040 comp rx=0 tx=0).stop 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2c150ec0 msgr2=0x7fbb2c151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2c150ec0 0x7fbb2c151340 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbb24002230 tx=0x7fbb24007b00 comp rx=0 tx=0).stop 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 shutdown_connections 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb1006c920 0x7fbb1006ede0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2c150ec0 0x7fbb2c151340 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c070810 0x7fbb2c070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 --2- 172.21.15.40:0/711940337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb180040d0 0x7fbb2c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 >> 172.21.15.40:0/711940337 conn(0x7fbb2c0d1cf0 msgr2=0x7fbb2c073830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:42.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 shutdown_connections 2024-04-26T21:37:42.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:42.876+0000 7fbb33b12640 1 -- 172.21.15.40:0/711940337 wait complete. 2024-04-26T21:37:42.887 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-04-26T21:37:42.029971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222012,"kb_used_data":5800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752708,"statfs":{"total":773094113280,"available":772866772992,"internally_reserved":0,"allocated":5939200,"data_stored":3288896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.900699"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.146297+0000","last_change":"2024-04-26T21:37:37.146297+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.146297+0000","last_undegraded":"2024-04-26T21:37:37.146297+0000","last_fullsized":"2024-04-26T21:37:37.146297+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138776+0000","last_change":"2024-04-26T21:37:37.138776+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138776+0000","last_undegraded":"2024-04-26T21:37:37.138776+0000","last_fullsized":"2024-04-26T21:37:37.138776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140361+0000","last_change":"2024-04-26T21:37:37.140361+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140361+0000","last_undegraded":"2024-04-26T21:37:37.140361+0000","last_fullsized":"2024-04-26T21:37:37.140361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139226+0000","last_change":"2024-04-26T21:37:37.139226+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139226+0000","last_undegraded":"2024-04-26T21:37:37.139226+0000","last_fullsized":"2024-04-26T21:37:37.139226+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137175+0000","last_change":"2024-04-26T21:37:37.137175+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137175+0000","last_undegraded":"2024-04-26T21:37:37.137175+0000","last_fullsized":"2024-04-26T21:37:37.137175+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.147679+0000","last_change":"2024-04-26T21:37:37.147679+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.147679+0000","last_undegraded":"2024-04-26T21:37:37.147679+0000","last_fullsized":"2024-04-26T21:37:37.147679+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139709+0000","last_change":"2024-04-26T21:37:37.139709+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139709+0000","last_undegraded":"2024-04-26T21:37:37.139709+0000","last_fullsized":"2024-04-26T21:37:37.139709+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139690+0000","last_change":"2024-04-26T21:37:37.139690+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139690+0000","last_undegraded":"2024-04-26T21:37:37.139690+0000","last_fullsized":"2024-04-26T21:37:37.139690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139335+0000","last_change":"2024-04-26T21:37:37.139335+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139335+0000","last_undegraded":"2024-04-26T21:37:37.139335+0000","last_fullsized":"2024-04-26T21:37:37.139335+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.148501+0000","last_change":"2024-04-26T21:37:37.148501+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.148501+0000","last_undegraded":"2024-04-26T21:37:37.148501+0000","last_fullsized":"2024-04-26T21:37:37.148501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143003+0000","last_change":"2024-04-26T21:37:37.143003+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143003+0000","last_undegraded":"2024-04-26T21:37:37.143003+0000","last_fullsized":"2024-04-26T21:37:37.143003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139262+0000","last_change":"2024-04-26T21:37:37.139262+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139262+0000","last_undegraded":"2024-04-26T21:37:37.139262+0000","last_fullsized":"2024-04-26T21:37:37.139262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145421+0000","last_change":"2024-04-26T21:37:37.145421+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145421+0000","last_undegraded":"2024-04-26T21:37:37.145421+0000","last_fullsized":"2024-04-26T21:37:37.145421+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140247+0000","last_change":"2024-04-26T21:37:37.140247+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140247+0000","last_undegraded":"2024-04-26T21:37:37.140247+0000","last_fullsized":"2024-04-26T21:37:37.140247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138351+0000","last_change":"2024-04-26T21:37:37.138351+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138351+0000","last_undegraded":"2024-04-26T21:37:37.138351+0000","last_fullsized":"2024-04-26T21:37:37.138351+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143870+0000","last_change":"2024-04-26T21:37:37.143870+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143870+0000","last_undegraded":"2024-04-26T21:37:37.143870+0000","last_fullsized":"2024-04-26T21:37:37.143870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144920+0000","last_change":"2024-04-26T21:37:37.144920+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144920+0000","last_undegraded":"2024-04-26T21:37:37.144920+0000","last_fullsized":"2024-04-26T21:37:37.144920+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138328+0000","last_change":"2024-04-26T21:37:37.138328+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138328+0000","last_undegraded":"2024-04-26T21:37:37.138328+0000","last_fullsized":"2024-04-26T21:37:37.138328+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139643+0000","last_change":"2024-04-26T21:37:37.139643+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139643+0000","last_undegraded":"2024-04-26T21:37:37.139643+0000","last_fullsized":"2024-04-26T21:37:37.139643+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143012+0000","last_change":"2024-04-26T21:37:37.143012+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143012+0000","last_undegraded":"2024-04-26T21:37:37.143012+0000","last_fullsized":"2024-04-26T21:37:37.143012+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140772+0000","last_change":"2024-04-26T21:37:37.140772+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140772+0000","last_undegraded":"2024-04-26T21:37:37.140772+0000","last_fullsized":"2024-04-26T21:37:37.140772+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138725+0000","last_change":"2024-04-26T21:37:37.138725+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138725+0000","last_undegraded":"2024-04-26T21:37:37.138725+0000","last_fullsized":"2024-04-26T21:37:37.138725+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137913+0000","last_change":"2024-04-26T21:37:37.137913+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137913+0000","last_undegraded":"2024-04-26T21:37:37.137913+0000","last_fullsized":"2024-04-26T21:37:37.137913+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138780+0000","last_change":"2024-04-26T21:37:37.138780+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138780+0000","last_undegraded":"2024-04-26T21:37:37.138780+0000","last_fullsized":"2024-04-26T21:37:37.138780+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144498+0000","last_change":"2024-04-26T21:37:37.144498+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144498+0000","last_undegraded":"2024-04-26T21:37:37.144498+0000","last_fullsized":"2024-04-26T21:37:37.144498+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138226+0000","last_change":"2024-04-26T21:37:37.138226+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138226+0000","last_undegraded":"2024-04-26T21:37:37.138226+0000","last_fullsized":"2024-04-26T21:37:37.138226+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.149883+0000","last_change":"2024-04-26T21:37:37.149883+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.149883+0000","last_undegraded":"2024-04-26T21:37:37.149883+0000","last_fullsized":"2024-04-26T21:37:37.149883+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.142613+0000","last_change":"2024-04-26T21:37:37.142613+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.142613+0000","last_undegraded":"2024-04-26T21:37:37.142613+0000","last_fullsized":"2024-04-26T21:37:37.142613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144029+0000","last_change":"2024-04-26T21:37:37.144029+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144029+0000","last_undegraded":"2024-04-26T21:37:37.144029+0000","last_fullsized":"2024-04-26T21:37:37.144029+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138351+0000","last_change":"2024-04-26T21:37:37.138351+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138351+0000","last_undegraded":"2024-04-26T21:37:37.138351+0000","last_fullsized":"2024-04-26T21:37:37.138351+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138627+0000","last_change":"2024-04-26T21:37:37.138627+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138627+0000","last_undegraded":"2024-04-26T21:37:37.138627+0000","last_fullsized":"2024-04-26T21:37:37.138627+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.135376+0000","last_change":"2024-04-26T21:37:37.135376+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.135376+0000","last_undegraded":"2024-04-26T21:37:37.135376+0000","last_fullsized":"2024-04-26T21:37:37.135376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138262+0000","last_change":"2024-04-26T21:37:37.138262+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138262+0000","last_undegraded":"2024-04-26T21:37:37.138262+0000","last_fullsized":"2024-04-26T21:37:37.138262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155881+0000","last_change":"2024-04-26T21:37:37.150925+0000","last_active":"2024-04-26T21:37:37.155881+0000","last_peered":"2024-04-26T21:37:37.155881+0000","last_clean":"2024-04-26T21:37:37.155881+0000","last_became_active":"2024-04-26T21:37:37.150597+0000","last_became_peered":"2024-04-26T21:37:37.150597+0000","last_unstale":"2024-04-26T21:37:37.155881+0000","last_undegraded":"2024-04-26T21:37:37.155881+0000","last_fullsized":"2024-04-26T21:37:37.155881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:53.683181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134054+0000","last_change":"2024-04-26T21:37:37.134054+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134054+0000","last_undegraded":"2024-04-26T21:37:37.134054+0000","last_fullsized":"2024-04-26T21:37:37.134054+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159403+0000","last_change":"2024-04-26T21:37:37.156594+0000","last_active":"2024-04-26T21:37:37.159403+0000","last_peered":"2024-04-26T21:37:37.159403+0000","last_clean":"2024-04-26T21:37:37.159403+0000","last_became_active":"2024-04-26T21:37:37.156312+0000","last_became_peered":"2024-04-26T21:37:37.156312+0000","last_unstale":"2024-04-26T21:37:37.159403+0000","last_undegraded":"2024-04-26T21:37:37.159403+0000","last_fullsized":"2024-04-26T21:37:37.159403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:43.766358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.142303+0000","last_change":"2024-04-26T21:37:37.142303+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.142303+0000","last_undegraded":"2024-04-26T21:37:37.142303+0000","last_fullsized":"2024-04-26T21:37:37.142303+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.162010+0000","last_change":"2024-04-26T21:37:37.158086+0000","last_active":"2024-04-26T21:37:37.162010+0000","last_peered":"2024-04-26T21:37:37.162010+0000","last_clean":"2024-04-26T21:37:37.162010+0000","last_became_active":"2024-04-26T21:37:37.157790+0000","last_became_peered":"2024-04-26T21:37:37.157790+0000","last_unstale":"2024-04-26T21:37:37.162010+0000","last_undegraded":"2024-04-26T21:37:37.162010+0000","last_fullsized":"2024-04-26T21:37:37.162010+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.123343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136890+0000","last_change":"2024-04-26T21:37:37.136890+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136890+0000","last_undegraded":"2024-04-26T21:37:37.136890+0000","last_fullsized":"2024-04-26T21:37:37.136890+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.147649+0000","last_change":"2024-04-26T21:37:37.140298+0000","last_active":"2024-04-26T21:37:37.147649+0000","last_peered":"2024-04-26T21:37:37.147649+0000","last_clean":"2024-04-26T21:37:37.147649+0000","last_became_active":"2024-04-26T21:37:37.139801+0000","last_became_peered":"2024-04-26T21:37:37.139801+0000","last_unstale":"2024-04-26T21:37:37.147649+0000","last_undegraded":"2024-04-26T21:37:37.147649+0000","last_fullsized":"2024-04-26T21:37:37.147649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137107+0000","last_change":"2024-04-26T21:37:37.137107+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137107+0000","last_undegraded":"2024-04-26T21:37:37.137107+0000","last_fullsized":"2024-04-26T21:37:37.137107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.162038+0000","last_change":"2024-04-26T21:37:37.158278+0000","last_active":"2024-04-26T21:37:37.162038+0000","last_peered":"2024-04-26T21:37:37.162038+0000","last_clean":"2024-04-26T21:37:37.162038+0000","last_became_active":"2024-04-26T21:37:37.154700+0000","last_became_peered":"2024-04-26T21:37:37.154700+0000","last_unstale":"2024-04-26T21:37:37.162038+0000","last_undegraded":"2024-04-26T21:37:37.162038+0000","last_fullsized":"2024-04-26T21:37:37.162038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:30.231390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139564+0000","last_change":"2024-04-26T21:37:37.139564+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139564+0000","last_undegraded":"2024-04-26T21:37:37.139564+0000","last_fullsized":"2024-04-26T21:37:37.139564+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.152274+0000","last_change":"2024-04-26T21:37:37.148427+0000","last_active":"2024-04-26T21:37:37.152274+0000","last_peered":"2024-04-26T21:37:37.152274+0000","last_clean":"2024-04-26T21:37:37.152274+0000","last_became_active":"2024-04-26T21:37:37.147872+0000","last_became_peered":"2024-04-26T21:37:37.147872+0000","last_unstale":"2024-04-26T21:37:37.152274+0000","last_undegraded":"2024-04-26T21:37:37.152274+0000","last_fullsized":"2024-04-26T21:37:37.152274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:14.846091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136935+0000","last_change":"2024-04-26T21:37:37.136935+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136935+0000","last_undegraded":"2024-04-26T21:37:37.136935+0000","last_fullsized":"2024-04-26T21:37:37.136935+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.149727+0000","last_change":"2024-04-26T21:37:37.147914+0000","last_active":"2024-04-26T21:37:37.149727+0000","last_peered":"2024-04-26T21:37:37.149727+0000","last_clean":"2024-04-26T21:37:37.149727+0000","last_became_active":"2024-04-26T21:37:37.147592+0000","last_became_peered":"2024-04-26T21:37:37.147592+0000","last_unstale":"2024-04-26T21:37:37.149727+0000","last_undegraded":"2024-04-26T21:37:37.149727+0000","last_fullsized":"2024-04-26T21:37:37.149727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:50.841990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138903+0000","last_change":"2024-04-26T21:37:37.138903+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138903+0000","last_undegraded":"2024-04-26T21:37:37.138903+0000","last_fullsized":"2024-04-26T21:37:37.138903+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158742+0000","last_change":"2024-04-26T21:37:37.153776+0000","last_active":"2024-04-26T21:37:37.158742+0000","last_peered":"2024-04-26T21:37:37.158742+0000","last_clean":"2024-04-26T21:37:37.158742+0000","last_became_active":"2024-04-26T21:37:37.153390+0000","last_became_peered":"2024-04-26T21:37:37.153390+0000","last_unstale":"2024-04-26T21:37:37.158742+0000","last_undegraded":"2024-04-26T21:37:37.158742+0000","last_fullsized":"2024-04-26T21:37:37.158742+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:43.797895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136442+0000","last_change":"2024-04-26T21:37:37.136442+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136442+0000","last_undegraded":"2024-04-26T21:37:37.136442+0000","last_fullsized":"2024-04-26T21:37:37.136442+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.154501+0000","last_change":"2024-04-26T21:37:37.153096+0000","last_active":"2024-04-26T21:37:37.154501+0000","last_peered":"2024-04-26T21:37:37.154501+0000","last_clean":"2024-04-26T21:37:37.154501+0000","last_became_active":"2024-04-26T21:37:37.152795+0000","last_became_peered":"2024-04-26T21:37:37.152795+0000","last_unstale":"2024-04-26T21:37:37.154501+0000","last_undegraded":"2024-04-26T21:37:37.154501+0000","last_fullsized":"2024-04-26T21:37:37.154501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:48.571013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134754+0000","last_change":"2024-04-26T21:37:37.134754+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134754+0000","last_undegraded":"2024-04-26T21:37:37.134754+0000","last_fullsized":"2024-04-26T21:37:37.134754+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158396+0000","last_change":"2024-04-26T21:37:37.154318+0000","last_active":"2024-04-26T21:37:37.158396+0000","last_peered":"2024-04-26T21:37:37.158396+0000","last_clean":"2024-04-26T21:37:37.158396+0000","last_became_active":"2024-04-26T21:37:37.154072+0000","last_became_peered":"2024-04-26T21:37:37.154072+0000","last_unstale":"2024-04-26T21:37:37.158396+0000","last_undegraded":"2024-04-26T21:37:37.158396+0000","last_fullsized":"2024-04-26T21:37:37.158396+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:08.561240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138381+0000","last_change":"2024-04-26T21:37:37.138381+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138381+0000","last_undegraded":"2024-04-26T21:37:37.138381+0000","last_fullsized":"2024-04-26T21:37:37.138381+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.156365+0000","last_change":"2024-04-26T21:37:37.154992+0000","last_active":"2024-04-26T21:37:37.156365+0000","last_peered":"2024-04-26T21:37:37.156365+0000","last_clean":"2024-04-26T21:37:37.156365+0000","last_became_active":"2024-04-26T21:37:37.154711+0000","last_became_peered":"2024-04-26T21:37:37.154711+0000","last_unstale":"2024-04-26T21:37:37.156365+0000","last_undegraded":"2024-04-26T21:37:37.156365+0000","last_fullsized":"2024-04-26T21:37:37.156365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:04.317873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136779+0000","last_change":"2024-04-26T21:37:37.136779+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136779+0000","last_undegraded":"2024-04-26T21:37:37.136779+0000","last_fullsized":"2024-04-26T21:37:37.136779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155175+0000","last_change":"2024-04-26T21:37:37.146807+0000","last_active":"2024-04-26T21:37:37.155175+0000","last_peered":"2024-04-26T21:37:37.155175+0000","last_clean":"2024-04-26T21:37:37.155175+0000","last_became_active":"2024-04-26T21:37:37.146479+0000","last_became_peered":"2024-04-26T21:37:37.146479+0000","last_unstale":"2024-04-26T21:37:37.155175+0000","last_undegraded":"2024-04-26T21:37:37.155175+0000","last_fullsized":"2024-04-26T21:37:37.155175+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.135431+0000","last_change":"2024-04-26T21:37:37.135431+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.135431+0000","last_undegraded":"2024-04-26T21:37:37.135431+0000","last_fullsized":"2024-04-26T21:37:37.135431+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158195+0000","last_change":"2024-04-26T21:37:37.152397+0000","last_active":"2024-04-26T21:37:37.158195+0000","last_peered":"2024-04-26T21:37:37.158195+0000","last_clean":"2024-04-26T21:37:37.158195+0000","last_became_active":"2024-04-26T21:37:37.152118+0000","last_became_peered":"2024-04-26T21:37:37.152118+0000","last_unstale":"2024-04-26T21:37:37.158195+0000","last_undegraded":"2024-04-26T21:37:37.158195+0000","last_fullsized":"2024-04-26T21:37:37.158195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:50.687873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139183+0000","last_change":"2024-04-26T21:37:37.139183+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139183+0000","last_undegraded":"2024-04-26T21:37:37.139183+0000","last_fullsized":"2024-04-26T21:37:37.139183+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158451+0000","last_change":"2024-04-26T21:37:37.155533+0000","last_active":"2024-04-26T21:37:37.158451+0000","last_peered":"2024-04-26T21:37:37.158451+0000","last_clean":"2024-04-26T21:37:37.158451+0000","last_became_active":"2024-04-26T21:37:37.155179+0000","last_became_peered":"2024-04-26T21:37:37.155179+0000","last_unstale":"2024-04-26T21:37:37.158451+0000","last_undegraded":"2024-04-26T21:37:37.158451+0000","last_fullsized":"2024-04-26T21:37:37.158451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:14.846091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137624+0000","last_change":"2024-04-26T21:37:37.137624+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137624+0000","last_undegraded":"2024-04-26T21:37:37.137624+0000","last_fullsized":"2024-04-26T21:37:37.137624+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.152746+0000","last_change":"2024-04-26T21:37:37.148935+0000","last_active":"2024-04-26T21:37:37.152746+0000","last_peered":"2024-04-26T21:37:37.152746+0000","last_clean":"2024-04-26T21:37:37.152746+0000","last_became_active":"2024-04-26T21:37:37.148635+0000","last_became_peered":"2024-04-26T21:37:37.148635+0000","last_unstale":"2024-04-26T21:37:37.152746+0000","last_undegraded":"2024-04-26T21:37:37.152746+0000","last_fullsized":"2024-04-26T21:37:37.152746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.883299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145387+0000","last_change":"2024-04-26T21:37:37.145387+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145387+0000","last_undegraded":"2024-04-26T21:37:37.145387+0000","last_fullsized":"2024-04-26T21:37:37.145387+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155439+0000","last_change":"2024-04-26T21:37:37.147944+0000","last_active":"2024-04-26T21:37:37.155439+0000","last_peered":"2024-04-26T21:37:37.155439+0000","last_clean":"2024-04-26T21:37:37.155439+0000","last_became_active":"2024-04-26T21:37:37.147675+0000","last_became_peered":"2024-04-26T21:37:37.147675+0000","last_unstale":"2024-04-26T21:37:37.155439+0000","last_undegraded":"2024-04-26T21:37:37.155439+0000","last_fullsized":"2024-04-26T21:37:37.155439+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:20.163566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136315+0000","last_change":"2024-04-26T21:37:37.136315+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136315+0000","last_undegraded":"2024-04-26T21:37:37.136315+0000","last_fullsized":"2024-04-26T21:37:37.136315+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158568+0000","last_change":"2024-04-26T21:37:37.153638+0000","last_active":"2024-04-26T21:37:37.158568+0000","last_peered":"2024-04-26T21:37:37.158568+0000","last_clean":"2024-04-26T21:37:37.158568+0000","last_became_active":"2024-04-26T21:37:37.153152+0000","last_became_peered":"2024-04-26T21:37:37.153152+0000","last_unstale":"2024-04-26T21:37:37.158568+0000","last_undegraded":"2024-04-26T21:37:37.158568+0000","last_fullsized":"2024-04-26T21:37:37.158568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:31.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144651+0000","last_change":"2024-04-26T21:37:37.144651+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144651+0000","last_undegraded":"2024-04-26T21:37:37.144651+0000","last_fullsized":"2024-04-26T21:37:37.144651+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.157061+0000","last_change":"2024-04-26T21:37:37.155432+0000","last_active":"2024-04-26T21:37:37.157061+0000","last_peered":"2024-04-26T21:37:37.157061+0000","last_clean":"2024-04-26T21:37:37.157061+0000","last_became_active":"2024-04-26T21:37:37.155163+0000","last_became_peered":"2024-04-26T21:37:37.155163+0000","last_unstale":"2024-04-26T21:37:37.157061+0000","last_undegraded":"2024-04-26T21:37:37.157061+0000","last_fullsized":"2024-04-26T21:37:37.157061+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:34.248752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136604+0000","last_change":"2024-04-26T21:37:37.136604+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136604+0000","last_undegraded":"2024-04-26T21:37:37.136604+0000","last_fullsized":"2024-04-26T21:37:37.136604+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.161212+0000","last_change":"2024-04-26T21:37:37.155985+0000","last_active":"2024-04-26T21:37:37.161212+0000","last_peered":"2024-04-26T21:37:37.161212+0000","last_clean":"2024-04-26T21:37:37.161212+0000","last_became_active":"2024-04-26T21:37:37.155643+0000","last_became_peered":"2024-04-26T21:37:37.155643+0000","last_unstale":"2024-04-26T21:37:37.161212+0000","last_undegraded":"2024-04-26T21:37:37.161212+0000","last_fullsized":"2024-04-26T21:37:37.161212+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:48.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145020+0000","last_change":"2024-04-26T21:37:37.145020+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145020+0000","last_undegraded":"2024-04-26T21:37:37.145020+0000","last_fullsized":"2024-04-26T21:37:37.145020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155757+0000","last_change":"2024-04-26T21:37:37.150088+0000","last_active":"2024-04-26T21:37:37.155757+0000","last_peered":"2024-04-26T21:37:37.155757+0000","last_clean":"2024-04-26T21:37:37.155757+0000","last_became_active":"2024-04-26T21:37:37.149820+0000","last_became_peered":"2024-04-26T21:37:37.149820+0000","last_unstale":"2024-04-26T21:37:37.155757+0000","last_undegraded":"2024-04-26T21:37:37.155757+0000","last_fullsized":"2024-04-26T21:37:37.155757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:56.039426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134473+0000","last_change":"2024-04-26T21:37:37.134473+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134473+0000","last_undegraded":"2024-04-26T21:37:37.134473+0000","last_fullsized":"2024-04-26T21:37:37.134473+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159394+0000","last_change":"2024-04-26T21:37:37.156465+0000","last_active":"2024-04-26T21:37:37.159394+0000","last_peered":"2024-04-26T21:37:37.159394+0000","last_clean":"2024-04-26T21:37:37.159394+0000","last_became_active":"2024-04-26T21:37:37.156194+0000","last_became_peered":"2024-04-26T21:37:37.156194+0000","last_unstale":"2024-04-26T21:37:37.159394+0000","last_undegraded":"2024-04-26T21:37:37.159394+0000","last_fullsized":"2024-04-26T21:37:37.159394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:28.173221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.141743+0000","last_change":"2024-04-26T21:37:37.141743+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.141743+0000","last_undegraded":"2024-04-26T21:37:37.141743+0000","last_fullsized":"2024-04-26T21:37:37.141743+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150052+0000","last_change":"2024-04-26T21:37:37.143737+0000","last_active":"2024-04-26T21:37:37.150052+0000","last_peered":"2024-04-26T21:37:37.150052+0000","last_clean":"2024-04-26T21:37:37.150052+0000","last_became_active":"2024-04-26T21:37:37.143432+0000","last_became_peered":"2024-04-26T21:37:37.143432+0000","last_unstale":"2024-04-26T21:37:37.150052+0000","last_undegraded":"2024-04-26T21:37:37.150052+0000","last_fullsized":"2024-04-26T21:37:37.150052+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:48.659187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139896+0000","last_change":"2024-04-26T21:37:37.139896+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139896+0000","last_undegraded":"2024-04-26T21:37:37.139896+0000","last_fullsized":"2024-04-26T21:37:37.139896+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158712+0000","last_change":"2024-04-26T21:37:37.156130+0000","last_active":"2024-04-26T21:37:37.158712+0000","last_peered":"2024-04-26T21:37:37.158712+0000","last_clean":"2024-04-26T21:37:37.158712+0000","last_became_active":"2024-04-26T21:37:37.154520+0000","last_became_peered":"2024-04-26T21:37:37.154520+0000","last_unstale":"2024-04-26T21:37:37.158712+0000","last_undegraded":"2024-04-26T21:37:37.158712+0000","last_fullsized":"2024-04-26T21:37:37.158712+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:38.183624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145831+0000","last_change":"2024-04-26T21:37:37.145831+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145831+0000","last_undegraded":"2024-04-26T21:37:37.145831+0000","last_fullsized":"2024-04-26T21:37:37.145831+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150738+0000","last_change":"2024-04-26T21:37:37.145621+0000","last_active":"2024-04-26T21:37:37.150738+0000","last_peered":"2024-04-26T21:37:37.150738+0000","last_clean":"2024-04-26T21:37:37.150738+0000","last_became_active":"2024-04-26T21:37:37.145215+0000","last_became_peered":"2024-04-26T21:37:37.145215+0000","last_unstale":"2024-04-26T21:37:37.150738+0000","last_undegraded":"2024-04-26T21:37:37.150738+0000","last_fullsized":"2024-04-26T21:37:37.150738+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.363875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134531+0000","last_change":"2024-04-26T21:37:37.134531+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134531+0000","last_undegraded":"2024-04-26T21:37:37.134531+0000","last_fullsized":"2024-04-26T21:37:37.134531+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.161206+0000","last_change":"2024-04-26T21:37:37.155721+0000","last_active":"2024-04-26T21:37:37.161206+0000","last_peered":"2024-04-26T21:37:37.161206+0000","last_clean":"2024-04-26T21:37:37.161206+0000","last_became_active":"2024-04-26T21:37:37.155438+0000","last_became_peered":"2024-04-26T21:37:37.155438+0000","last_unstale":"2024-04-26T21:37:37.161206+0000","last_undegraded":"2024-04-26T21:37:37.161206+0000","last_fullsized":"2024-04-26T21:37:37.161206+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:08.561240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.147246+0000","last_change":"2024-04-26T21:37:37.147246+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.147246+0000","last_undegraded":"2024-04-26T21:37:37.147246+0000","last_fullsized":"2024-04-26T21:37:37.147246+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.153071+0000","last_change":"2024-04-26T21:37:37.151412+0000","last_active":"2024-04-26T21:37:37.153071+0000","last_peered":"2024-04-26T21:37:37.153071+0000","last_clean":"2024-04-26T21:37:37.153071+0000","last_became_active":"2024-04-26T21:37:37.151157+0000","last_became_peered":"2024-04-26T21:37:37.151157+0000","last_unstale":"2024-04-26T21:37:37.153071+0000","last_undegraded":"2024-04-26T21:37:37.153071+0000","last_fullsized":"2024-04-26T21:37:37.153071+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:30.231390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136343+0000","last_change":"2024-04-26T21:37:37.136343+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136343+0000","last_undegraded":"2024-04-26T21:37:37.136343+0000","last_fullsized":"2024-04-26T21:37:37.136343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158678+0000","last_change":"2024-04-26T21:37:37.154574+0000","last_active":"2024-04-26T21:37:37.158678+0000","last_peered":"2024-04-26T21:37:37.158678+0000","last_clean":"2024-04-26T21:37:37.158678+0000","last_became_active":"2024-04-26T21:37:37.153518+0000","last_became_peered":"2024-04-26T21:37:37.153518+0000","last_unstale":"2024-04-26T21:37:37.158678+0000","last_undegraded":"2024-04-26T21:37:37.158678+0000","last_fullsized":"2024-04-26T21:37:37.158678+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:43.168309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138567+0000","last_change":"2024-04-26T21:37:37.138567+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138567+0000","last_undegraded":"2024-04-26T21:37:37.138567+0000","last_fullsized":"2024-04-26T21:37:37.138567+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.151260+0000","last_change":"2024-04-26T21:37:37.144706+0000","last_active":"2024-04-26T21:37:37.151260+0000","last_peered":"2024-04-26T21:37:37.151260+0000","last_clean":"2024-04-26T21:37:37.151260+0000","last_became_active":"2024-04-26T21:37:37.144369+0000","last_became_peered":"2024-04-26T21:37:37.144369+0000","last_unstale":"2024-04-26T21:37:37.151260+0000","last_undegraded":"2024-04-26T21:37:37.151260+0000","last_fullsized":"2024-04-26T21:37:37.151260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134559+0000","last_change":"2024-04-26T21:37:37.134559+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134559+0000","last_undegraded":"2024-04-26T21:37:37.134559+0000","last_fullsized":"2024-04-26T21:37:37.134559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.157301+0000","last_change":"2024-04-26T21:37:37.153455+0000","last_active":"2024-04-26T21:37:37.157301+0000","last_peered":"2024-04-26T21:37:37.157301+0000","last_clean":"2024-04-26T21:37:37.157301+0000","last_became_active":"2024-04-26T21:37:37.153181+0000","last_became_peered":"2024-04-26T21:37:37.153181+0000","last_unstale":"2024-04-26T21:37:37.157301+0000","last_undegraded":"2024-04-26T21:37:37.157301+0000","last_fullsized":"2024-04-26T21:37:37.157301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:47.599920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140819+0000","last_change":"2024-04-26T21:37:37.140819+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140819+0000","last_undegraded":"2024-04-26T21:37:37.140819+0000","last_fullsized":"2024-04-26T21:37:37.140819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150089+0000","last_change":"2024-04-26T21:37:37.144882+0000","last_active":"2024-04-26T21:37:37.150089+0000","last_peered":"2024-04-26T21:37:37.150089+0000","last_clean":"2024-04-26T21:37:37.150089+0000","last_became_active":"2024-04-26T21:37:37.144633+0000","last_became_peered":"2024-04-26T21:37:37.144633+0000","last_unstale":"2024-04-26T21:37:37.150089+0000","last_undegraded":"2024-04-26T21:37:37.150089+0000","last_fullsized":"2024-04-26T21:37:37.150089+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:53.683181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136338+0000","last_change":"2024-04-26T21:37:37.136338+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136338+0000","last_undegraded":"2024-04-26T21:37:37.136338+0000","last_fullsized":"2024-04-26T21:37:37.136338+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155915+0000","last_change":"2024-04-26T21:37:37.150695+0000","last_active":"2024-04-26T21:37:37.155915+0000","last_peered":"2024-04-26T21:37:37.155915+0000","last_clean":"2024-04-26T21:37:37.155915+0000","last_became_active":"2024-04-26T21:37:37.150398+0000","last_became_peered":"2024-04-26T21:37:37.150398+0000","last_unstale":"2024-04-26T21:37:37.155915+0000","last_undegraded":"2024-04-26T21:37:37.155915+0000","last_fullsized":"2024-04-26T21:37:37.155915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.141329+0000","last_change":"2024-04-26T21:37:37.141329+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.141329+0000","last_undegraded":"2024-04-26T21:37:37.141329+0000","last_fullsized":"2024-04-26T21:37:37.141329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159474+0000","last_change":"2024-04-26T21:37:37.156817+0000","last_active":"2024-04-26T21:37:37.159474+0000","last_peered":"2024-04-26T21:37:37.159474+0000","last_clean":"2024-04-26T21:37:37.159474+0000","last_became_active":"2024-04-26T21:37:37.156561+0000","last_became_peered":"2024-04-26T21:37:37.156561+0000","last_unstale":"2024-04-26T21:37:37.159474+0000","last_undegraded":"2024-04-26T21:37:37.159474+0000","last_fullsized":"2024-04-26T21:37:37.159474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.883299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":226,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.140105+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:37:37.140105+0000","last_peered":"2024-04-26T21:37:37.140105+0000","last_clean":"2024-04-26T21:37:37.140105+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:37:37.140105+0000","last_undegraded":"2024-04-26T21:37:37.140105+0000","last_fullsized":"2024-04-26T21:37:37.140105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.146297+0000', 'last_change': '2024-04-26T21:37:37.146297+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.146297+0000', 'last_undegraded': '2024-04-26T21:37:37.146297+0000', 'last_fullsized': '2024-04-26T21:37:37.146297+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138776+0000', 'last_change': '2024-04-26T21:37:37.138776+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138776+0000', 'last_undegraded': '2024-04-26T21:37:37.138776+0000', 'last_fullsized': '2024-04-26T21:37:37.138776+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.140361+0000', 'last_change': '2024-04-26T21:37:37.140361+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.140361+0000', 'last_undegraded': '2024-04-26T21:37:37.140361+0000', 'last_fullsized': '2024-04-26T21:37:37.140361+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139226+0000', 'last_change': '2024-04-26T21:37:37.139226+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139226+0000', 'last_undegraded': '2024-04-26T21:37:37.139226+0000', 'last_fullsized': '2024-04-26T21:37:37.139226+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-04-26T21:37:42.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.137175+0000', 'last_change': '2024-04-26T21:37:37.137175+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.137175+0000', 'last_undegraded': '2024-04-26T21:37:37.137175+0000', 'last_fullsized': '2024-04-26T21:37:37.137175+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.147679+0000', 'last_change': '2024-04-26T21:37:37.147679+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.147679+0000', 'last_undegraded': '2024-04-26T21:37:37.147679+0000', 'last_fullsized': '2024-04-26T21:37:37.147679+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139709+0000', 'last_change': '2024-04-26T21:37:37.139709+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139709+0000', 'last_undegraded': '2024-04-26T21:37:37.139709+0000', 'last_fullsized': '2024-04-26T21:37:37.139709+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139690+0000', 'last_change': '2024-04-26T21:37:37.139690+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139690+0000', 'last_undegraded': '2024-04-26T21:37:37.139690+0000', 'last_fullsized': '2024-04-26T21:37:37.139690+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139335+0000', 'last_change': '2024-04-26T21:37:37.139335+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139335+0000', 'last_undegraded': '2024-04-26T21:37:37.139335+0000', 'last_fullsized': '2024-04-26T21:37:37.139335+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-04-26T21:37:42.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.148501+0000', 'last_change': '2024-04-26T21:37:37.148501+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.148501+0000', 'last_undegraded': '2024-04-26T21:37:37.148501+0000', 'last_fullsized': '2024-04-26T21:37:37.148501+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.143003+0000', 'last_change': '2024-04-26T21:37:37.143003+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.143003+0000', 'last_undegraded': '2024-04-26T21:37:37.143003+0000', 'last_fullsized': '2024-04-26T21:37:37.143003+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139262+0000', 'last_change': '2024-04-26T21:37:37.139262+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139262+0000', 'last_undegraded': '2024-04-26T21:37:37.139262+0000', 'last_fullsized': '2024-04-26T21:37:37.139262+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.145421+0000', 'last_change': '2024-04-26T21:37:37.145421+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.145421+0000', 'last_undegraded': '2024-04-26T21:37:37.145421+0000', 'last_fullsized': '2024-04-26T21:37:37.145421+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.140247+0000', 'last_change': '2024-04-26T21:37:37.140247+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.140247+0000', 'last_undegraded': '2024-04-26T21:37:37.140247+0000', 'last_fullsized': '2024-04-26T21:37:37.140247+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138351+0000', 'last_change': '2024-04-26T21:37:37.138351+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138351+0000', 'last_undegraded': '2024-04-26T21:37:37.138351+0000', 'last_fullsized': '2024-04-26T21:37:37.138351+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:37:42.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.143870+0000', 'last_change': '2024-04-26T21:37:37.143870+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.143870+0000', 'last_undegraded': '2024-04-26T21:37:37.143870+0000', 'last_fullsized': '2024-04-26T21:37:37.143870+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.144920+0000', 'last_change': '2024-04-26T21:37:37.144920+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.144920+0000', 'last_undegraded': '2024-04-26T21:37:37.144920+0000', 'last_fullsized': '2024-04-26T21:37:37.144920+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138328+0000', 'last_change': '2024-04-26T21:37:37.138328+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138328+0000', 'last_undegraded': '2024-04-26T21:37:37.138328+0000', 'last_fullsized': '2024-04-26T21:37:37.138328+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139643+0000', 'last_change': '2024-04-26T21:37:37.139643+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139643+0000', 'last_undegraded': '2024-04-26T21:37:37.139643+0000', 'last_fullsized': '2024-04-26T21:37:37.139643+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.143012+0000', 'last_change': '2024-04-26T21:37:37.143012+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.143012+0000', 'last_undegraded': '2024-04-26T21:37:37.143012+0000', 'last_fullsized': '2024-04-26T21:37:37.143012+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:37:42.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.140772+0000', 'last_change': '2024-04-26T21:37:37.140772+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.140772+0000', 'last_undegraded': '2024-04-26T21:37:37.140772+0000', 'last_fullsized': '2024-04-26T21:37:37.140772+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138725+0000', 'last_change': '2024-04-26T21:37:37.138725+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138725+0000', 'last_undegraded': '2024-04-26T21:37:37.138725+0000', 'last_fullsized': '2024-04-26T21:37:37.138725+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.137913+0000', 'last_change': '2024-04-26T21:37:37.137913+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.137913+0000', 'last_undegraded': '2024-04-26T21:37:37.137913+0000', 'last_fullsized': '2024-04-26T21:37:37.137913+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138780+0000', 'last_change': '2024-04-26T21:37:37.138780+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138780+0000', 'last_undegraded': '2024-04-26T21:37:37.138780+0000', 'last_fullsized': '2024-04-26T21:37:37.138780+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.144498+0000', 'last_change': '2024-04-26T21:37:37.144498+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.144498+0000', 'last_undegraded': '2024-04-26T21:37:37.144498+0000', 'last_fullsized': '2024-04-26T21:37:37.144498+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-04-26T21:37:42.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138226+0000', 'last_change': '2024-04-26T21:37:37.138226+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138226+0000', 'last_undegraded': '2024-04-26T21:37:37.138226+0000', 'last_fullsized': '2024-04-26T21:37:37.138226+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-04-26T21:37:42.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.149883+0000', 'last_change': '2024-04-26T21:37:37.149883+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.149883+0000', 'last_undegraded': '2024-04-26T21:37:37.149883+0000', 'last_fullsized': '2024-04-26T21:37:37.149883+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-04-26T21:37:42.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.142613+0000', 'last_change': '2024-04-26T21:37:37.142613+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.142613+0000', 'last_undegraded': '2024-04-26T21:37:37.142613+0000', 'last_fullsized': '2024-04-26T21:37:37.142613+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:37:42.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-04-26T21:37:42.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.144029+0000', 'last_change': '2024-04-26T21:37:37.144029+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.144029+0000', 'last_undegraded': '2024-04-26T21:37:37.144029+0000', 'last_fullsized': '2024-04-26T21:37:37.144029+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-04-26T21:37:42.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138351+0000', 'last_change': '2024-04-26T21:37:37.138351+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138351+0000', 'last_undegraded': '2024-04-26T21:37:37.138351+0000', 'last_fullsized': '2024-04-26T21:37:37.138351+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138627+0000', 'last_change': '2024-04-26T21:37:37.138627+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138627+0000', 'last_undegraded': '2024-04-26T21:37:37.138627+0000', 'last_fullsized': '2024-04-26T21:37:37.138627+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.135376+0000', 'last_change': '2024-04-26T21:37:37.135376+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.135376+0000', 'last_undegraded': '2024-04-26T21:37:37.135376+0000', 'last_fullsized': '2024-04-26T21:37:37.135376+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138262+0000', 'last_change': '2024-04-26T21:37:37.138262+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138262+0000', 'last_undegraded': '2024-04-26T21:37:37.138262+0000', 'last_fullsized': '2024-04-26T21:37:37.138262+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.134054+0000', 'last_change': '2024-04-26T21:37:37.134054+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.134054+0000', 'last_undegraded': '2024-04-26T21:37:37.134054+0000', 'last_fullsized': '2024-04-26T21:37:37.134054+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-26T21:37:42.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.142303+0000', 'last_change': '2024-04-26T21:37:37.142303+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.142303+0000', 'last_undegraded': '2024-04-26T21:37:37.142303+0000', 'last_fullsized': '2024-04-26T21:37:37.142303+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.136890+0000', 'last_change': '2024-04-26T21:37:37.136890+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.136890+0000', 'last_undegraded': '2024-04-26T21:37:37.136890+0000', 'last_fullsized': '2024-04-26T21:37:37.136890+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.137107+0000', 'last_change': '2024-04-26T21:37:37.137107+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.137107+0000', 'last_undegraded': '2024-04-26T21:37:37.137107+0000', 'last_fullsized': '2024-04-26T21:37:37.137107+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139564+0000', 'last_change': '2024-04-26T21:37:37.139564+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139564+0000', 'last_undegraded': '2024-04-26T21:37:37.139564+0000', 'last_fullsized': '2024-04-26T21:37:37.139564+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.136935+0000', 'last_change': '2024-04-26T21:37:37.136935+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.136935+0000', 'last_undegraded': '2024-04-26T21:37:37.136935+0000', 'last_fullsized': '2024-04-26T21:37:37.136935+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138903+0000', 'last_change': '2024-04-26T21:37:37.138903+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138903+0000', 'last_undegraded': '2024-04-26T21:37:37.138903+0000', 'last_fullsized': '2024-04-26T21:37:37.138903+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:37:42.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-26T21:37:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.136442+0000', 'last_change': '2024-04-26T21:37:37.136442+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.136442+0000', 'last_undegraded': '2024-04-26T21:37:37.136442+0000', 'last_fullsized': '2024-04-26T21:37:37.136442+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:37:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-26T21:37:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.134754+0000', 'last_change': '2024-04-26T21:37:37.134754+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.134754+0000', 'last_undegraded': '2024-04-26T21:37:37.134754+0000', 'last_fullsized': '2024-04-26T21:37:37.134754+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:37:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-26T21:37:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138381+0000', 'last_change': '2024-04-26T21:37:37.138381+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138381+0000', 'last_undegraded': '2024-04-26T21:37:37.138381+0000', 'last_fullsized': '2024-04-26T21:37:37.138381+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-26T21:37:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.136779+0000', 'last_change': '2024-04-26T21:37:37.136779+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.136779+0000', 'last_undegraded': '2024-04-26T21:37:37.136779+0000', 'last_fullsized': '2024-04-26T21:37:37.136779+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:37:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-26T21:37:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.135431+0000', 'last_change': '2024-04-26T21:37:37.135431+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.135431+0000', 'last_undegraded': '2024-04-26T21:37:37.135431+0000', 'last_fullsized': '2024-04-26T21:37:37.135431+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:37:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-26T21:37:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139183+0000', 'last_change': '2024-04-26T21:37:37.139183+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139183+0000', 'last_undegraded': '2024-04-26T21:37:37.139183+0000', 'last_fullsized': '2024-04-26T21:37:37.139183+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-26T21:37:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.137624+0000', 'last_change': '2024-04-26T21:37:37.137624+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.137624+0000', 'last_undegraded': '2024-04-26T21:37:37.137624+0000', 'last_fullsized': '2024-04-26T21:37:37.137624+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-26T21:37:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.145387+0000', 'last_change': '2024-04-26T21:37:37.145387+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.145387+0000', 'last_undegraded': '2024-04-26T21:37:37.145387+0000', 'last_fullsized': '2024-04-26T21:37:37.145387+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.136315+0000', 'last_change': '2024-04-26T21:37:37.136315+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.136315+0000', 'last_undegraded': '2024-04-26T21:37:37.136315+0000', 'last_fullsized': '2024-04-26T21:37:37.136315+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.144651+0000', 'last_change': '2024-04-26T21:37:37.144651+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.144651+0000', 'last_undegraded': '2024-04-26T21:37:37.144651+0000', 'last_fullsized': '2024-04-26T21:37:37.144651+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.136604+0000', 'last_change': '2024-04-26T21:37:37.136604+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.136604+0000', 'last_undegraded': '2024-04-26T21:37:37.136604+0000', 'last_fullsized': '2024-04-26T21:37:37.136604+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.145020+0000', 'last_change': '2024-04-26T21:37:37.145020+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.145020+0000', 'last_undegraded': '2024-04-26T21:37:37.145020+0000', 'last_fullsized': '2024-04-26T21:37:37.145020+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.134473+0000', 'last_change': '2024-04-26T21:37:37.134473+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.134473+0000', 'last_undegraded': '2024-04-26T21:37:37.134473+0000', 'last_fullsized': '2024-04-26T21:37:37.134473+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-26T21:37:42.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.141743+0000', 'last_change': '2024-04-26T21:37:37.141743+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.141743+0000', 'last_undegraded': '2024-04-26T21:37:37.141743+0000', 'last_fullsized': '2024-04-26T21:37:37.141743+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:37:42.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-26T21:37:42.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.139896+0000', 'last_change': '2024-04-26T21:37:37.139896+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.139896+0000', 'last_undegraded': '2024-04-26T21:37:37.139896+0000', 'last_fullsized': '2024-04-26T21:37:37.139896+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-26T21:37:42.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.145831+0000', 'last_change': '2024-04-26T21:37:37.145831+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.145831+0000', 'last_undegraded': '2024-04-26T21:37:37.145831+0000', 'last_fullsized': '2024-04-26T21:37:37.145831+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-26T21:37:42.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.134531+0000', 'last_change': '2024-04-26T21:37:37.134531+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.134531+0000', 'last_undegraded': '2024-04-26T21:37:37.134531+0000', 'last_fullsized': '2024-04-26T21:37:37.134531+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:37:42.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-26T21:37:42.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.147246+0000', 'last_change': '2024-04-26T21:37:37.147246+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.147246+0000', 'last_undegraded': '2024-04-26T21:37:37.147246+0000', 'last_fullsized': '2024-04-26T21:37:37.147246+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.136343+0000', 'last_change': '2024-04-26T21:37:37.136343+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.136343+0000', 'last_undegraded': '2024-04-26T21:37:37.136343+0000', 'last_fullsized': '2024-04-26T21:37:37.136343+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.138567+0000', 'last_change': '2024-04-26T21:37:37.138567+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.138567+0000', 'last_undegraded': '2024-04-26T21:37:37.138567+0000', 'last_fullsized': '2024-04-26T21:37:37.138567+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.134559+0000', 'last_change': '2024-04-26T21:37:37.134559+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.134559+0000', 'last_undegraded': '2024-04-26T21:37:37.134559+0000', 'last_fullsized': '2024-04-26T21:37:37.134559+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.140819+0000', 'last_change': '2024-04-26T21:37:37.140819+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.140819+0000', 'last_undegraded': '2024-04-26T21:37:37.140819+0000', 'last_fullsized': '2024-04-26T21:37:37.140819+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-26T21:37:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.136338+0000', 'last_change': '2024-04-26T21:37:37.136338+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.136338+0000', 'last_undegraded': '2024-04-26T21:37:37.136338+0000', 'last_fullsized': '2024-04-26T21:37:37.136338+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:37:42.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-26T21:37:42.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:37:37.141329+0000', 'last_change': '2024-04-26T21:37:37.141329+0000', 'last_active': '2024-04-26T21:37:37.128336+0000', 'last_peered': '2024-04-26T21:37:37.128336+0000', 'last_clean': '2024-04-26T21:37:37.128336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:37:37.141329+0000', 'last_undegraded': '2024-04-26T21:37:37.141329+0000', 'last_fullsized': '2024-04-26T21:37:37.141329+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'last_clean_scrub_stamp': '2024-04-26T21:37:37.128336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:37:42.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:37:43.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.027+0000 7f6ab359e640 1 Processor -- start 2024-04-26T21:37:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.027+0000 7f6ab359e640 1 -- start start 2024-04-26T21:37:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.028+0000 7f6ab359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6ab415ce80 0x7f6ab40cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.028+0000 7f6ab359e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ab4074120 0x7f6ab40cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.028+0000 7f6ab359e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab40ce340 con 0x7f6ab415d260 2024-04-26T21:37:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.028+0000 7f6ab359e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab40cf330 con 0x7f6ab4074120 2024-04-26T21:37:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.028+0000 7f6ab359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab40ce620 con 0x7f6ab415ce80 2024-04-26T21:37:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab2d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ab4074120 0x7f6ab40cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab1d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6ab415d260 0x7f6ab4073a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60506/0 (socket says 172.21.15.40:60506) 2024-04-26T21:37:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab1d9b640 1 -- 172.21.15.40:0/983721395 learned_addr learned my addr 172.21.15.40:0/983721395 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab259c640 1 --2- 172.21.15.40:0/983721395 >> v2:172.21.15.184:3300/0 conn(0x7f6ab415ce80 0x7f6ab40cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab2d9d640 1 -- 172.21.15.40:0/983721395 >> v2:172.21.15.184:3300/0 conn(0x7f6ab415ce80 msgr2=0x7f6ab40cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab2d9d640 1 --2- 172.21.15.40:0/983721395 >> v2:172.21.15.184:3300/0 conn(0x7f6ab415ce80 0x7f6ab40cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab2d9d640 1 -- 172.21.15.40:0/983721395 >> v1:172.21.15.40:6789/0 conn(0x7f6ab415d260 legacy=0x7f6ab4073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab2d9d640 1 -- 172.21.15.40:0/983721395 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab40ce900 con 0x7f6ab4074120 2024-04-26T21:37:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.029+0000 7f6ab159a640 1 -- 172.21.15.40:0/983721395 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317257691 0 0) 0x7f6ab40ce340 con 0x7f6ab415d260 2024-04-26T21:37:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab2d9d640 1 --2- 172.21.15.40:0/983721395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab4074120 0x7f6ab40cdd10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40d24c0 tx=0x7f6aa402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3edfeef6c7cc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab159a640 1 -- 172.21.15.40:0/983721395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4007740 con 0x7f6ab4074120 2024-04-26T21:37:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab159a640 1 -- 172.21.15.40:0/983721395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa40078e0 con 0x7f6ab4074120 2024-04-26T21:37:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab159a640 1 -- 172.21.15.40:0/983721395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4007cb0 con 0x7f6ab4074120 2024-04-26T21:37:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab359e640 1 -- 172.21.15.40:0/983721395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab4074120 msgr2=0x7f6ab40cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab359e640 1 --2- 172.21.15.40:0/983721395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab4074120 0x7f6ab40cdd10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40d24c0 tx=0x7f6aa402a5f0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab359e640 1 -- 172.21.15.40:0/983721395 shutdown_connections 2024-04-26T21:37:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab359e640 1 --2- 172.21.15.40:0/983721395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab4074120 0x7f6ab40cdd10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab359e640 1 --2- 172.21.15.40:0/983721395 >> v2:172.21.15.184:3300/0 conn(0x7f6ab415ce80 0x7f6ab40cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab359e640 1 -- 172.21.15.40:0/983721395 >> 172.21.15.40:0/983721395 conn(0x7f6ab40b2fe0 msgr2=0x7f6ab40b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab359e640 1 -- 172.21.15.40:0/983721395 shutdown_connections 2024-04-26T21:37:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.030+0000 7f6ab359e640 1 -- 172.21.15.40:0/983721395 wait complete. 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.032+0000 7f6ab359e640 1 Processor -- start 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.032+0000 7f6ab359e640 1 -- start start 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.032+0000 7f6ab359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ab4074120 0x7f6ab415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.032+0000 7f6ab359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ab415ce80 0x7f6ab40c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.032+0000 7f6ab359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab415d260 0x7f6ab40b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.032+0000 7f6ab359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab40cf7c0 con 0x7f6ab415d260 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.032+0000 7f6ab359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab40747d0 con 0x7f6ab4074120 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.032+0000 7f6ab359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab40cf940 con 0x7f6ab415ce80 2024-04-26T21:37:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab415d260 0x7f6ab40b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab415d260 0x7f6ab40b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57110/0 (socket says 172.21.15.40:57110) 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 -- 172.21.15.40:0/2379222703 learned_addr learned my addr 172.21.15.40:0/2379222703 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab259c640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ab4074120 0x7f6ab415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab1d9b640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ab415ce80 0x7f6ab40c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 -- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ab415ce80 msgr2=0x7f6ab40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ab415ce80 0x7f6ab40c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 -- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ab4074120 msgr2=0x7f6ab415a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ab4074120 0x7f6ab415a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 -- 172.21.15.40:0/2379222703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab40b44f0 con 0x7f6ab415d260 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab1d9b640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ab415ce80 0x7f6ab40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab2d9d640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab415d260 0x7f6ab40b1430 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6aa402abe0 tx=0x7f6aa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6aaaffd640 1 -- 172.21.15.40:0/2379222703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4040070 con 0x7f6ab415d260 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6aaaffd640 1 -- 172.21.15.40:0/2379222703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa4031bc0 con 0x7f6ab415d260 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab40ca2a0 con 0x7f6ab415d260 2024-04-26T21:37:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6aaaffd640 1 -- 172.21.15.40:0/2379222703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4045460 con 0x7f6ab415d260 2024-04-26T21:37:43.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.033+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab40c9a00 con 0x7f6ab415d260 2024-04-26T21:37:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.034+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab4069a80 con 0x7f6ab415d260 2024-04-26T21:37:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.035+0000 7f6aaaffd640 1 -- 172.21.15.40:0/2379222703 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6aa4039040 con 0x7f6ab415d260 2024-04-26T21:37:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.035+0000 7f6aaaffd640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a9406c8f0 0x7f6a9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.035+0000 7f6aaaffd640 1 -- 172.21.15.40:0/2379222703 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f6aa40ad4e0 con 0x7f6ab415d260 2024-04-26T21:37:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.036+0000 7f6ab259c640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a9406c8f0 0x7f6a9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.036+0000 7f6ab259c640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a9406c8f0 0x7f6a9406edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6aac006fd0 tx=0x7f6aac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:43.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.040+0000 7f6aaaffd640 1 -- 172.21.15.40:0/2379222703 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6aa407b810 con 0x7f6ab415d260 2024-04-26T21:37:43.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.296+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6ab40cebe0 con 0x7f6ab415d260 2024-04-26T21:37:43.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.297+0000 7f6aaaffd640 1 -- 172.21.15.40:0/2379222703 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1195 (secure 0 0 0) 0x7f6aa407b110 con 0x7f6ab415d260 2024-04-26T21:37:43.297 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:43.297 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":177,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1714167297,"num_in_osds":8,"osd_in_since":1714167287,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":227340288,"bytes_avail":772866772992,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":32,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6015}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-26T21:37:20.024709+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T21:37:43.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a9406c8f0 msgr2=0x7f6a9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a9406c8f0 0x7f6a9406edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6aac006fd0 tx=0x7f6aac008040 comp rx=0 tx=0).stop 2024-04-26T21:37:43.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab415d260 msgr2=0x7f6ab40b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab415d260 0x7f6ab40b1430 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6aa402abe0 tx=0x7f6aa4004a40 comp rx=0 tx=0).stop 2024-04-26T21:37:43.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 shutdown_connections 2024-04-26T21:37:43.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a9406c8f0 0x7f6a9406edb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab415d260 0x7f6ab40b1430 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ab415ce80 0x7f6ab40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 --2- 172.21.15.40:0/2379222703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ab4074120 0x7f6ab415a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 >> 172.21.15.40:0/2379222703 conn(0x7f6ab40b2fe0 msgr2=0x7f6ab40cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:43.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 shutdown_connections 2024-04-26T21:37:43.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.299+0000 7f6ab359e640 1 -- 172.21.15.40:0/2379222703 wait complete. 2024-04-26T21:37:43.310 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:37:43.310 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:37:43.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.411+0000 7fd817a25640 1 Processor -- start 2024-04-26T21:37:43.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.411+0000 7fd817a25640 1 -- start start 2024-04-26T21:37:43.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd817a25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81006f790 0x7fd81006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd817a25640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd810156840 con 0x7fd810156460 2024-04-26T21:37:43.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd817a25640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8100cbf20 con 0x7fd810072c50 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd817a25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8100cc200 con 0x7fd81006f790 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd8157c1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd810156460 0x7fd8100cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60516/0 (socket says 172.21.15.40:60516) 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd8157c1640 1 -- 172.21.15.40:0/3662311774 learned_addr learned my addr 172.21.15.40:0/3662311774 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3241308984 0 0) 0x7fd810156840 con 0x7fd810156460 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7f8003520 con 0x7fd810156460 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.412+0000 7fd814fc0640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81006f790 0x7fd81006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4039248991 0 0) 0x7fd7f8003520 con 0x7fd810156460 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81006f790 msgr2=0x7fd81006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81006f790 0x7fd81006fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 >> v1:172.21.15.184:6789/0 conn(0x7fd810072c50 legacy=0x7fd8100730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8100cc4e0 con 0x7fd810156460 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fd8000019f0 con 0x7fd810156460 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 >> v1:172.21.15.40:6789/0 conn(0x7fd810156460 legacy=0x7fd8100cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f80039e0 0x7fd7f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81006f790 0x7fd7f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f8004d40 0x7fd7f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8100cc200 con 0x7fd7f80039e0 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f800bb50 con 0x7fd7f8004d40 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f800df60 con 0x7fd81006f790 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd8157c1640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f80039e0 0x7fd7f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd814fc0640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81006f790 0x7fd7f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd815fc2640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f8004d40 0x7fd7f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd8157c1640 1 -- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81006f790 msgr2=0x7fd7f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd8157c1640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81006f790 0x7fd7f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd8157c1640 1 -- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f8004d40 msgr2=0x7fd7f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd8157c1640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f8004d40 0x7fd7f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.413+0000 7fd8157c1640 1 -- 172.21.15.40:0/3662311774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f8003520 con 0x7fd7f80039e0 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.414+0000 7fd8157c1640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f80039e0 0x7fd7f8003e00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd80000a860 tx=0x7fd7f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5aa7c2aa0da828 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.414+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000a480 con 0x7fd7f80039e0 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.414+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd80002f070 con 0x7fd7f80039e0 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.414+0000 7fd80ffff640 1 -- 172.21.15.40:0/3662311774 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80002bd00 con 0x7fd7f80039e0 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.414+0000 7fd817a25640 1 -- 172.21.15.40:0/3662311774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f80039e0 msgr2=0x7fd7f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.414+0000 7fd817a25640 1 --2- 172.21.15.40:0/3662311774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f80039e0 0x7fd7f8003e00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd80000a860 tx=0x7fd7f8005830 comp rx=0 tx=0).stop 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.414+0000 7fd815fc2640 1 -- 172.21.15.40:0/3662311774 reap_dead start 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.415+0000 7fd817a25640 1 -- 172.21.15.40:0/3662311774 shutdown_connections 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.415+0000 7fd817a25640 1 -- 172.21.15.40:0/3662311774 >> 172.21.15.40:0/3662311774 conn(0x7fd8100d0be0 msgr2=0x7fd8100d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.415+0000 7fd817a25640 1 -- 172.21.15.40:0/3662311774 shutdown_connections 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.415+0000 7fd817a25640 1 -- 172.21.15.40:0/3662311774 wait complete. 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.416+0000 7fd817a25640 1 Processor -- start 2024-04-26T21:37:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.416+0000 7fd817a25640 1 -- start start 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd817a25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80039e0 0x7fd81014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd817a25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8100c8670 0x7fd81015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd817a25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8101e6660 0x7fd8101e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd817a25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8100cbbe0 con 0x7fd8100c8670 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd817a25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd810072700 con 0x7fd8101e6660 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd817a25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd81006e260 con 0x7fd7f80039e0 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8100c8670 0x7fd81015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8100c8670 0x7fd81015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57128/0 (socket says 172.21.15.40:57128) 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 -- 172.21.15.40:0/893096553 learned_addr learned my addr 172.21.15.40:0/893096553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd815fc2640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8101e6660 0x7fd8101e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd8157c1640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80039e0 0x7fd81014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 -- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80039e0 msgr2=0x7fd81014e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80039e0 0x7fd81014e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 -- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8101e6660 msgr2=0x7fd8101e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8101e6660 0x7fd8101e6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 -- 172.21.15.40:0/893096553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8100c9440 con 0x7fd8100c8670 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd814fc0640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8100c8670 0x7fd81015ec80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd80400b820 tx=0x7fd80400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd80dffb640 1 -- 172.21.15.40:0/893096553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804018070 con 0x7fd8100c8670 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd80dffb640 1 -- 172.21.15.40:0/893096553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd80400be10 con 0x7fd8100c8670 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd80dffb640 1 -- 172.21.15.40:0/893096553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804010c00 con 0x7fd8100c8670 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.417+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8100c8bc0 con 0x7fd8100c8670 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.418+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81014e840 con 0x7fd8100c8670 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.418+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8100694f0 con 0x7fd8100c8670 2024-04-26T21:37:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.419+0000 7fd80dffb640 1 -- 172.21.15.40:0/893096553 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd80401b070 con 0x7fd8100c8670 2024-04-26T21:37:43.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.419+0000 7fd80dffb640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd7fc06c8f0 0x7fd7fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:43.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.419+0000 7fd80dffb640 1 -- 172.21.15.40:0/893096553 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fd80408bf60 con 0x7fd8100c8670 2024-04-26T21:37:43.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.420+0000 7fd8157c1640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd7fc06c8f0 0x7fd7fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:43.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.420+0000 7fd8157c1640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd7fc06c8f0 0x7fd7fc06edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd80000a860 tx=0x7fd800009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:43.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.424+0000 7fd80dffb640 1 -- 172.21.15.40:0/893096553 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd80405a200 con 0x7fd8100c8670 2024-04-26T21:37:43.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.605+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8100c90e0 con 0x7fd7fc06c8f0 2024-04-26T21:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.611+0000 7fd80dffb640 1 -- 172.21.15.40:0/893096553 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246638 (secure 0 0 0) 0x7fd80002f040 con 0x7fd7fc06c8f0 2024-04-26T21:37:43.614 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:43.615 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd7fc06c8f0 msgr2=0x7fd7fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd7fc06c8f0 0x7fd7fc06edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd80000a860 tx=0x7fd800009030 comp rx=0 tx=0).stop 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8100c8670 msgr2=0x7fd81015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8100c8670 0x7fd81015ec80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd80400b820 tx=0x7fd80400bce0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 shutdown_connections 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd7fc06c8f0 0x7fd7fc06edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8101e6660 0x7fd8101e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8100c8670 0x7fd81015ec80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 --2- 172.21.15.40:0/893096553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80039e0 0x7fd81014e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 >> 172.21.15.40:0/893096553 conn(0x7fd8100d0be0 msgr2=0x7fd8100b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 shutdown_connections 2024-04-26T21:37:43.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:43.614+0000 7fd817a25640 1 -- 172.21.15.40:0/893096553 wait complete. 2024-04-26T21:37:43.625 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-04-26T21:37:42.029971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222012,"kb_used_data":5800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752708,"statfs":{"total":773094113280,"available":772866772992,"internally_reserved":0,"allocated":5939200,"data_stored":3288896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.900699"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.146297+0000","last_change":"2024-04-26T21:37:37.146297+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.146297+0000","last_undegraded":"2024-04-26T21:37:37.146297+0000","last_fullsized":"2024-04-26T21:37:37.146297+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138776+0000","last_change":"2024-04-26T21:37:37.138776+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138776+0000","last_undegraded":"2024-04-26T21:37:37.138776+0000","last_fullsized":"2024-04-26T21:37:37.138776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140361+0000","last_change":"2024-04-26T21:37:37.140361+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140361+0000","last_undegraded":"2024-04-26T21:37:37.140361+0000","last_fullsized":"2024-04-26T21:37:37.140361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139226+0000","last_change":"2024-04-26T21:37:37.139226+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139226+0000","last_undegraded":"2024-04-26T21:37:37.139226+0000","last_fullsized":"2024-04-26T21:37:37.139226+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137175+0000","last_change":"2024-04-26T21:37:37.137175+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137175+0000","last_undegraded":"2024-04-26T21:37:37.137175+0000","last_fullsized":"2024-04-26T21:37:37.137175+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.147679+0000","last_change":"2024-04-26T21:37:37.147679+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.147679+0000","last_undegraded":"2024-04-26T21:37:37.147679+0000","last_fullsized":"2024-04-26T21:37:37.147679+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139709+0000","last_change":"2024-04-26T21:37:37.139709+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139709+0000","last_undegraded":"2024-04-26T21:37:37.139709+0000","last_fullsized":"2024-04-26T21:37:37.139709+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139690+0000","last_change":"2024-04-26T21:37:37.139690+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139690+0000","last_undegraded":"2024-04-26T21:37:37.139690+0000","last_fullsized":"2024-04-26T21:37:37.139690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139335+0000","last_change":"2024-04-26T21:37:37.139335+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139335+0000","last_undegraded":"2024-04-26T21:37:37.139335+0000","last_fullsized":"2024-04-26T21:37:37.139335+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.148501+0000","last_change":"2024-04-26T21:37:37.148501+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.148501+0000","last_undegraded":"2024-04-26T21:37:37.148501+0000","last_fullsized":"2024-04-26T21:37:37.148501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143003+0000","last_change":"2024-04-26T21:37:37.143003+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143003+0000","last_undegraded":"2024-04-26T21:37:37.143003+0000","last_fullsized":"2024-04-26T21:37:37.143003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139262+0000","last_change":"2024-04-26T21:37:37.139262+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139262+0000","last_undegraded":"2024-04-26T21:37:37.139262+0000","last_fullsized":"2024-04-26T21:37:37.139262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145421+0000","last_change":"2024-04-26T21:37:37.145421+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145421+0000","last_undegraded":"2024-04-26T21:37:37.145421+0000","last_fullsized":"2024-04-26T21:37:37.145421+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140247+0000","last_change":"2024-04-26T21:37:37.140247+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140247+0000","last_undegraded":"2024-04-26T21:37:37.140247+0000","last_fullsized":"2024-04-26T21:37:37.140247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138351+0000","last_change":"2024-04-26T21:37:37.138351+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138351+0000","last_undegraded":"2024-04-26T21:37:37.138351+0000","last_fullsized":"2024-04-26T21:37:37.138351+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143870+0000","last_change":"2024-04-26T21:37:37.143870+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143870+0000","last_undegraded":"2024-04-26T21:37:37.143870+0000","last_fullsized":"2024-04-26T21:37:37.143870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144920+0000","last_change":"2024-04-26T21:37:37.144920+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144920+0000","last_undegraded":"2024-04-26T21:37:37.144920+0000","last_fullsized":"2024-04-26T21:37:37.144920+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138328+0000","last_change":"2024-04-26T21:37:37.138328+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138328+0000","last_undegraded":"2024-04-26T21:37:37.138328+0000","last_fullsized":"2024-04-26T21:37:37.138328+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139643+0000","last_change":"2024-04-26T21:37:37.139643+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139643+0000","last_undegraded":"2024-04-26T21:37:37.139643+0000","last_fullsized":"2024-04-26T21:37:37.139643+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.143012+0000","last_change":"2024-04-26T21:37:37.143012+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.143012+0000","last_undegraded":"2024-04-26T21:37:37.143012+0000","last_fullsized":"2024-04-26T21:37:37.143012+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140772+0000","last_change":"2024-04-26T21:37:37.140772+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140772+0000","last_undegraded":"2024-04-26T21:37:37.140772+0000","last_fullsized":"2024-04-26T21:37:37.140772+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138725+0000","last_change":"2024-04-26T21:37:37.138725+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138725+0000","last_undegraded":"2024-04-26T21:37:37.138725+0000","last_fullsized":"2024-04-26T21:37:37.138725+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137913+0000","last_change":"2024-04-26T21:37:37.137913+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137913+0000","last_undegraded":"2024-04-26T21:37:37.137913+0000","last_fullsized":"2024-04-26T21:37:37.137913+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138780+0000","last_change":"2024-04-26T21:37:37.138780+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138780+0000","last_undegraded":"2024-04-26T21:37:37.138780+0000","last_fullsized":"2024-04-26T21:37:37.138780+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144498+0000","last_change":"2024-04-26T21:37:37.144498+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144498+0000","last_undegraded":"2024-04-26T21:37:37.144498+0000","last_fullsized":"2024-04-26T21:37:37.144498+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138226+0000","last_change":"2024-04-26T21:37:37.138226+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138226+0000","last_undegraded":"2024-04-26T21:37:37.138226+0000","last_fullsized":"2024-04-26T21:37:37.138226+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.149883+0000","last_change":"2024-04-26T21:37:37.149883+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.149883+0000","last_undegraded":"2024-04-26T21:37:37.149883+0000","last_fullsized":"2024-04-26T21:37:37.149883+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.142613+0000","last_change":"2024-04-26T21:37:37.142613+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.142613+0000","last_undegraded":"2024-04-26T21:37:37.142613+0000","last_fullsized":"2024-04-26T21:37:37.142613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144029+0000","last_change":"2024-04-26T21:37:37.144029+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144029+0000","last_undegraded":"2024-04-26T21:37:37.144029+0000","last_fullsized":"2024-04-26T21:37:37.144029+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138351+0000","last_change":"2024-04-26T21:37:37.138351+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138351+0000","last_undegraded":"2024-04-26T21:37:37.138351+0000","last_fullsized":"2024-04-26T21:37:37.138351+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138627+0000","last_change":"2024-04-26T21:37:37.138627+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138627+0000","last_undegraded":"2024-04-26T21:37:37.138627+0000","last_fullsized":"2024-04-26T21:37:37.138627+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.135376+0000","last_change":"2024-04-26T21:37:37.135376+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.135376+0000","last_undegraded":"2024-04-26T21:37:37.135376+0000","last_fullsized":"2024-04-26T21:37:37.135376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138262+0000","last_change":"2024-04-26T21:37:37.138262+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138262+0000","last_undegraded":"2024-04-26T21:37:37.138262+0000","last_fullsized":"2024-04-26T21:37:37.138262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155881+0000","last_change":"2024-04-26T21:37:37.150925+0000","last_active":"2024-04-26T21:37:37.155881+0000","last_peered":"2024-04-26T21:37:37.155881+0000","last_clean":"2024-04-26T21:37:37.155881+0000","last_became_active":"2024-04-26T21:37:37.150597+0000","last_became_peered":"2024-04-26T21:37:37.150597+0000","last_unstale":"2024-04-26T21:37:37.155881+0000","last_undegraded":"2024-04-26T21:37:37.155881+0000","last_fullsized":"2024-04-26T21:37:37.155881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:53.683181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134054+0000","last_change":"2024-04-26T21:37:37.134054+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134054+0000","last_undegraded":"2024-04-26T21:37:37.134054+0000","last_fullsized":"2024-04-26T21:37:37.134054+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159403+0000","last_change":"2024-04-26T21:37:37.156594+0000","last_active":"2024-04-26T21:37:37.159403+0000","last_peered":"2024-04-26T21:37:37.159403+0000","last_clean":"2024-04-26T21:37:37.159403+0000","last_became_active":"2024-04-26T21:37:37.156312+0000","last_became_peered":"2024-04-26T21:37:37.156312+0000","last_unstale":"2024-04-26T21:37:37.159403+0000","last_undegraded":"2024-04-26T21:37:37.159403+0000","last_fullsized":"2024-04-26T21:37:37.159403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:43.766358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.142303+0000","last_change":"2024-04-26T21:37:37.142303+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.142303+0000","last_undegraded":"2024-04-26T21:37:37.142303+0000","last_fullsized":"2024-04-26T21:37:37.142303+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.162010+0000","last_change":"2024-04-26T21:37:37.158086+0000","last_active":"2024-04-26T21:37:37.162010+0000","last_peered":"2024-04-26T21:37:37.162010+0000","last_clean":"2024-04-26T21:37:37.162010+0000","last_became_active":"2024-04-26T21:37:37.157790+0000","last_became_peered":"2024-04-26T21:37:37.157790+0000","last_unstale":"2024-04-26T21:37:37.162010+0000","last_undegraded":"2024-04-26T21:37:37.162010+0000","last_fullsized":"2024-04-26T21:37:37.162010+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.123343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136890+0000","last_change":"2024-04-26T21:37:37.136890+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136890+0000","last_undegraded":"2024-04-26T21:37:37.136890+0000","last_fullsized":"2024-04-26T21:37:37.136890+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.147649+0000","last_change":"2024-04-26T21:37:37.140298+0000","last_active":"2024-04-26T21:37:37.147649+0000","last_peered":"2024-04-26T21:37:37.147649+0000","last_clean":"2024-04-26T21:37:37.147649+0000","last_became_active":"2024-04-26T21:37:37.139801+0000","last_became_peered":"2024-04-26T21:37:37.139801+0000","last_unstale":"2024-04-26T21:37:37.147649+0000","last_undegraded":"2024-04-26T21:37:37.147649+0000","last_fullsized":"2024-04-26T21:37:37.147649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137107+0000","last_change":"2024-04-26T21:37:37.137107+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137107+0000","last_undegraded":"2024-04-26T21:37:37.137107+0000","last_fullsized":"2024-04-26T21:37:37.137107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.162038+0000","last_change":"2024-04-26T21:37:37.158278+0000","last_active":"2024-04-26T21:37:37.162038+0000","last_peered":"2024-04-26T21:37:37.162038+0000","last_clean":"2024-04-26T21:37:37.162038+0000","last_became_active":"2024-04-26T21:37:37.154700+0000","last_became_peered":"2024-04-26T21:37:37.154700+0000","last_unstale":"2024-04-26T21:37:37.162038+0000","last_undegraded":"2024-04-26T21:37:37.162038+0000","last_fullsized":"2024-04-26T21:37:37.162038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:30.231390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139564+0000","last_change":"2024-04-26T21:37:37.139564+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139564+0000","last_undegraded":"2024-04-26T21:37:37.139564+0000","last_fullsized":"2024-04-26T21:37:37.139564+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.152274+0000","last_change":"2024-04-26T21:37:37.148427+0000","last_active":"2024-04-26T21:37:37.152274+0000","last_peered":"2024-04-26T21:37:37.152274+0000","last_clean":"2024-04-26T21:37:37.152274+0000","last_became_active":"2024-04-26T21:37:37.147872+0000","last_became_peered":"2024-04-26T21:37:37.147872+0000","last_unstale":"2024-04-26T21:37:37.152274+0000","last_undegraded":"2024-04-26T21:37:37.152274+0000","last_fullsized":"2024-04-26T21:37:37.152274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:14.846091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136935+0000","last_change":"2024-04-26T21:37:37.136935+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136935+0000","last_undegraded":"2024-04-26T21:37:37.136935+0000","last_fullsized":"2024-04-26T21:37:37.136935+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.149727+0000","last_change":"2024-04-26T21:37:37.147914+0000","last_active":"2024-04-26T21:37:37.149727+0000","last_peered":"2024-04-26T21:37:37.149727+0000","last_clean":"2024-04-26T21:37:37.149727+0000","last_became_active":"2024-04-26T21:37:37.147592+0000","last_became_peered":"2024-04-26T21:37:37.147592+0000","last_unstale":"2024-04-26T21:37:37.149727+0000","last_undegraded":"2024-04-26T21:37:37.149727+0000","last_fullsized":"2024-04-26T21:37:37.149727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:50.841990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138903+0000","last_change":"2024-04-26T21:37:37.138903+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138903+0000","last_undegraded":"2024-04-26T21:37:37.138903+0000","last_fullsized":"2024-04-26T21:37:37.138903+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158742+0000","last_change":"2024-04-26T21:37:37.153776+0000","last_active":"2024-04-26T21:37:37.158742+0000","last_peered":"2024-04-26T21:37:37.158742+0000","last_clean":"2024-04-26T21:37:37.158742+0000","last_became_active":"2024-04-26T21:37:37.153390+0000","last_became_peered":"2024-04-26T21:37:37.153390+0000","last_unstale":"2024-04-26T21:37:37.158742+0000","last_undegraded":"2024-04-26T21:37:37.158742+0000","last_fullsized":"2024-04-26T21:37:37.158742+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:43.797895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136442+0000","last_change":"2024-04-26T21:37:37.136442+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136442+0000","last_undegraded":"2024-04-26T21:37:37.136442+0000","last_fullsized":"2024-04-26T21:37:37.136442+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.154501+0000","last_change":"2024-04-26T21:37:37.153096+0000","last_active":"2024-04-26T21:37:37.154501+0000","last_peered":"2024-04-26T21:37:37.154501+0000","last_clean":"2024-04-26T21:37:37.154501+0000","last_became_active":"2024-04-26T21:37:37.152795+0000","last_became_peered":"2024-04-26T21:37:37.152795+0000","last_unstale":"2024-04-26T21:37:37.154501+0000","last_undegraded":"2024-04-26T21:37:37.154501+0000","last_fullsized":"2024-04-26T21:37:37.154501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:48.571013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134754+0000","last_change":"2024-04-26T21:37:37.134754+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134754+0000","last_undegraded":"2024-04-26T21:37:37.134754+0000","last_fullsized":"2024-04-26T21:37:37.134754+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158396+0000","last_change":"2024-04-26T21:37:37.154318+0000","last_active":"2024-04-26T21:37:37.158396+0000","last_peered":"2024-04-26T21:37:37.158396+0000","last_clean":"2024-04-26T21:37:37.158396+0000","last_became_active":"2024-04-26T21:37:37.154072+0000","last_became_peered":"2024-04-26T21:37:37.154072+0000","last_unstale":"2024-04-26T21:37:37.158396+0000","last_undegraded":"2024-04-26T21:37:37.158396+0000","last_fullsized":"2024-04-26T21:37:37.158396+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:08.561240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138381+0000","last_change":"2024-04-26T21:37:37.138381+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138381+0000","last_undegraded":"2024-04-26T21:37:37.138381+0000","last_fullsized":"2024-04-26T21:37:37.138381+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.156365+0000","last_change":"2024-04-26T21:37:37.154992+0000","last_active":"2024-04-26T21:37:37.156365+0000","last_peered":"2024-04-26T21:37:37.156365+0000","last_clean":"2024-04-26T21:37:37.156365+0000","last_became_active":"2024-04-26T21:37:37.154711+0000","last_became_peered":"2024-04-26T21:37:37.154711+0000","last_unstale":"2024-04-26T21:37:37.156365+0000","last_undegraded":"2024-04-26T21:37:37.156365+0000","last_fullsized":"2024-04-26T21:37:37.156365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:04.317873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136779+0000","last_change":"2024-04-26T21:37:37.136779+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136779+0000","last_undegraded":"2024-04-26T21:37:37.136779+0000","last_fullsized":"2024-04-26T21:37:37.136779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155175+0000","last_change":"2024-04-26T21:37:37.146807+0000","last_active":"2024-04-26T21:37:37.155175+0000","last_peered":"2024-04-26T21:37:37.155175+0000","last_clean":"2024-04-26T21:37:37.155175+0000","last_became_active":"2024-04-26T21:37:37.146479+0000","last_became_peered":"2024-04-26T21:37:37.146479+0000","last_unstale":"2024-04-26T21:37:37.155175+0000","last_undegraded":"2024-04-26T21:37:37.155175+0000","last_fullsized":"2024-04-26T21:37:37.155175+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.135431+0000","last_change":"2024-04-26T21:37:37.135431+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.135431+0000","last_undegraded":"2024-04-26T21:37:37.135431+0000","last_fullsized":"2024-04-26T21:37:37.135431+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158195+0000","last_change":"2024-04-26T21:37:37.152397+0000","last_active":"2024-04-26T21:37:37.158195+0000","last_peered":"2024-04-26T21:37:37.158195+0000","last_clean":"2024-04-26T21:37:37.158195+0000","last_became_active":"2024-04-26T21:37:37.152118+0000","last_became_peered":"2024-04-26T21:37:37.152118+0000","last_unstale":"2024-04-26T21:37:37.158195+0000","last_undegraded":"2024-04-26T21:37:37.158195+0000","last_fullsized":"2024-04-26T21:37:37.158195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:50.687873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139183+0000","last_change":"2024-04-26T21:37:37.139183+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139183+0000","last_undegraded":"2024-04-26T21:37:37.139183+0000","last_fullsized":"2024-04-26T21:37:37.139183+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158451+0000","last_change":"2024-04-26T21:37:37.155533+0000","last_active":"2024-04-26T21:37:37.158451+0000","last_peered":"2024-04-26T21:37:37.158451+0000","last_clean":"2024-04-26T21:37:37.158451+0000","last_became_active":"2024-04-26T21:37:37.155179+0000","last_became_peered":"2024-04-26T21:37:37.155179+0000","last_unstale":"2024-04-26T21:37:37.158451+0000","last_undegraded":"2024-04-26T21:37:37.158451+0000","last_fullsized":"2024-04-26T21:37:37.158451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:14.846091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.137624+0000","last_change":"2024-04-26T21:37:37.137624+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.137624+0000","last_undegraded":"2024-04-26T21:37:37.137624+0000","last_fullsized":"2024-04-26T21:37:37.137624+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.152746+0000","last_change":"2024-04-26T21:37:37.148935+0000","last_active":"2024-04-26T21:37:37.152746+0000","last_peered":"2024-04-26T21:37:37.152746+0000","last_clean":"2024-04-26T21:37:37.152746+0000","last_became_active":"2024-04-26T21:37:37.148635+0000","last_became_peered":"2024-04-26T21:37:37.148635+0000","last_unstale":"2024-04-26T21:37:37.152746+0000","last_undegraded":"2024-04-26T21:37:37.152746+0000","last_fullsized":"2024-04-26T21:37:37.152746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.883299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145387+0000","last_change":"2024-04-26T21:37:37.145387+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145387+0000","last_undegraded":"2024-04-26T21:37:37.145387+0000","last_fullsized":"2024-04-26T21:37:37.145387+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155439+0000","last_change":"2024-04-26T21:37:37.147944+0000","last_active":"2024-04-26T21:37:37.155439+0000","last_peered":"2024-04-26T21:37:37.155439+0000","last_clean":"2024-04-26T21:37:37.155439+0000","last_became_active":"2024-04-26T21:37:37.147675+0000","last_became_peered":"2024-04-26T21:37:37.147675+0000","last_unstale":"2024-04-26T21:37:37.155439+0000","last_undegraded":"2024-04-26T21:37:37.155439+0000","last_fullsized":"2024-04-26T21:37:37.155439+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:20.163566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136315+0000","last_change":"2024-04-26T21:37:37.136315+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136315+0000","last_undegraded":"2024-04-26T21:37:37.136315+0000","last_fullsized":"2024-04-26T21:37:37.136315+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158568+0000","last_change":"2024-04-26T21:37:37.153638+0000","last_active":"2024-04-26T21:37:37.158568+0000","last_peered":"2024-04-26T21:37:37.158568+0000","last_clean":"2024-04-26T21:37:37.158568+0000","last_became_active":"2024-04-26T21:37:37.153152+0000","last_became_peered":"2024-04-26T21:37:37.153152+0000","last_unstale":"2024-04-26T21:37:37.158568+0000","last_undegraded":"2024-04-26T21:37:37.158568+0000","last_fullsized":"2024-04-26T21:37:37.158568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:31.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.144651+0000","last_change":"2024-04-26T21:37:37.144651+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.144651+0000","last_undegraded":"2024-04-26T21:37:37.144651+0000","last_fullsized":"2024-04-26T21:37:37.144651+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.157061+0000","last_change":"2024-04-26T21:37:37.155432+0000","last_active":"2024-04-26T21:37:37.157061+0000","last_peered":"2024-04-26T21:37:37.157061+0000","last_clean":"2024-04-26T21:37:37.157061+0000","last_became_active":"2024-04-26T21:37:37.155163+0000","last_became_peered":"2024-04-26T21:37:37.155163+0000","last_unstale":"2024-04-26T21:37:37.157061+0000","last_undegraded":"2024-04-26T21:37:37.157061+0000","last_fullsized":"2024-04-26T21:37:37.157061+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:34.248752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136604+0000","last_change":"2024-04-26T21:37:37.136604+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136604+0000","last_undegraded":"2024-04-26T21:37:37.136604+0000","last_fullsized":"2024-04-26T21:37:37.136604+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.161212+0000","last_change":"2024-04-26T21:37:37.155985+0000","last_active":"2024-04-26T21:37:37.161212+0000","last_peered":"2024-04-26T21:37:37.161212+0000","last_clean":"2024-04-26T21:37:37.161212+0000","last_became_active":"2024-04-26T21:37:37.155643+0000","last_became_peered":"2024-04-26T21:37:37.155643+0000","last_unstale":"2024-04-26T21:37:37.161212+0000","last_undegraded":"2024-04-26T21:37:37.161212+0000","last_fullsized":"2024-04-26T21:37:37.161212+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:48.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145020+0000","last_change":"2024-04-26T21:37:37.145020+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145020+0000","last_undegraded":"2024-04-26T21:37:37.145020+0000","last_fullsized":"2024-04-26T21:37:37.145020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155757+0000","last_change":"2024-04-26T21:37:37.150088+0000","last_active":"2024-04-26T21:37:37.155757+0000","last_peered":"2024-04-26T21:37:37.155757+0000","last_clean":"2024-04-26T21:37:37.155757+0000","last_became_active":"2024-04-26T21:37:37.149820+0000","last_became_peered":"2024-04-26T21:37:37.149820+0000","last_unstale":"2024-04-26T21:37:37.155757+0000","last_undegraded":"2024-04-26T21:37:37.155757+0000","last_fullsized":"2024-04-26T21:37:37.155757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:56.039426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134473+0000","last_change":"2024-04-26T21:37:37.134473+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134473+0000","last_undegraded":"2024-04-26T21:37:37.134473+0000","last_fullsized":"2024-04-26T21:37:37.134473+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159394+0000","last_change":"2024-04-26T21:37:37.156465+0000","last_active":"2024-04-26T21:37:37.159394+0000","last_peered":"2024-04-26T21:37:37.159394+0000","last_clean":"2024-04-26T21:37:37.159394+0000","last_became_active":"2024-04-26T21:37:37.156194+0000","last_became_peered":"2024-04-26T21:37:37.156194+0000","last_unstale":"2024-04-26T21:37:37.159394+0000","last_undegraded":"2024-04-26T21:37:37.159394+0000","last_fullsized":"2024-04-26T21:37:37.159394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:28.173221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.141743+0000","last_change":"2024-04-26T21:37:37.141743+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.141743+0000","last_undegraded":"2024-04-26T21:37:37.141743+0000","last_fullsized":"2024-04-26T21:37:37.141743+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150052+0000","last_change":"2024-04-26T21:37:37.143737+0000","last_active":"2024-04-26T21:37:37.150052+0000","last_peered":"2024-04-26T21:37:37.150052+0000","last_clean":"2024-04-26T21:37:37.150052+0000","last_became_active":"2024-04-26T21:37:37.143432+0000","last_became_peered":"2024-04-26T21:37:37.143432+0000","last_unstale":"2024-04-26T21:37:37.150052+0000","last_undegraded":"2024-04-26T21:37:37.150052+0000","last_fullsized":"2024-04-26T21:37:37.150052+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:48.659187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.139896+0000","last_change":"2024-04-26T21:37:37.139896+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.139896+0000","last_undegraded":"2024-04-26T21:37:37.139896+0000","last_fullsized":"2024-04-26T21:37:37.139896+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158712+0000","last_change":"2024-04-26T21:37:37.156130+0000","last_active":"2024-04-26T21:37:37.158712+0000","last_peered":"2024-04-26T21:37:37.158712+0000","last_clean":"2024-04-26T21:37:37.158712+0000","last_became_active":"2024-04-26T21:37:37.154520+0000","last_became_peered":"2024-04-26T21:37:37.154520+0000","last_unstale":"2024-04-26T21:37:37.158712+0000","last_undegraded":"2024-04-26T21:37:37.158712+0000","last_fullsized":"2024-04-26T21:37:37.158712+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:38.183624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.145831+0000","last_change":"2024-04-26T21:37:37.145831+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.145831+0000","last_undegraded":"2024-04-26T21:37:37.145831+0000","last_fullsized":"2024-04-26T21:37:37.145831+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150738+0000","last_change":"2024-04-26T21:37:37.145621+0000","last_active":"2024-04-26T21:37:37.150738+0000","last_peered":"2024-04-26T21:37:37.150738+0000","last_clean":"2024-04-26T21:37:37.150738+0000","last_became_active":"2024-04-26T21:37:37.145215+0000","last_became_peered":"2024-04-26T21:37:37.145215+0000","last_unstale":"2024-04-26T21:37:37.150738+0000","last_undegraded":"2024-04-26T21:37:37.150738+0000","last_fullsized":"2024-04-26T21:37:37.150738+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.363875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134531+0000","last_change":"2024-04-26T21:37:37.134531+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134531+0000","last_undegraded":"2024-04-26T21:37:37.134531+0000","last_fullsized":"2024-04-26T21:37:37.134531+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.161206+0000","last_change":"2024-04-26T21:37:37.155721+0000","last_active":"2024-04-26T21:37:37.161206+0000","last_peered":"2024-04-26T21:37:37.161206+0000","last_clean":"2024-04-26T21:37:37.161206+0000","last_became_active":"2024-04-26T21:37:37.155438+0000","last_became_peered":"2024-04-26T21:37:37.155438+0000","last_unstale":"2024-04-26T21:37:37.161206+0000","last_undegraded":"2024-04-26T21:37:37.161206+0000","last_fullsized":"2024-04-26T21:37:37.161206+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:08.561240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.147246+0000","last_change":"2024-04-26T21:37:37.147246+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.147246+0000","last_undegraded":"2024-04-26T21:37:37.147246+0000","last_fullsized":"2024-04-26T21:37:37.147246+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.153071+0000","last_change":"2024-04-26T21:37:37.151412+0000","last_active":"2024-04-26T21:37:37.153071+0000","last_peered":"2024-04-26T21:37:37.153071+0000","last_clean":"2024-04-26T21:37:37.153071+0000","last_became_active":"2024-04-26T21:37:37.151157+0000","last_became_peered":"2024-04-26T21:37:37.151157+0000","last_unstale":"2024-04-26T21:37:37.153071+0000","last_undegraded":"2024-04-26T21:37:37.153071+0000","last_fullsized":"2024-04-26T21:37:37.153071+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:30.231390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136343+0000","last_change":"2024-04-26T21:37:37.136343+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136343+0000","last_undegraded":"2024-04-26T21:37:37.136343+0000","last_fullsized":"2024-04-26T21:37:37.136343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.158678+0000","last_change":"2024-04-26T21:37:37.154574+0000","last_active":"2024-04-26T21:37:37.158678+0000","last_peered":"2024-04-26T21:37:37.158678+0000","last_clean":"2024-04-26T21:37:37.158678+0000","last_became_active":"2024-04-26T21:37:37.153518+0000","last_became_peered":"2024-04-26T21:37:37.153518+0000","last_unstale":"2024-04-26T21:37:37.158678+0000","last_undegraded":"2024-04-26T21:37:37.158678+0000","last_fullsized":"2024-04-26T21:37:37.158678+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:43.168309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.138567+0000","last_change":"2024-04-26T21:37:37.138567+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.138567+0000","last_undegraded":"2024-04-26T21:37:37.138567+0000","last_fullsized":"2024-04-26T21:37:37.138567+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.151260+0000","last_change":"2024-04-26T21:37:37.144706+0000","last_active":"2024-04-26T21:37:37.151260+0000","last_peered":"2024-04-26T21:37:37.151260+0000","last_clean":"2024-04-26T21:37:37.151260+0000","last_became_active":"2024-04-26T21:37:37.144369+0000","last_became_peered":"2024-04-26T21:37:37.144369+0000","last_unstale":"2024-04-26T21:37:37.151260+0000","last_undegraded":"2024-04-26T21:37:37.151260+0000","last_fullsized":"2024-04-26T21:37:37.151260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.134559+0000","last_change":"2024-04-26T21:37:37.134559+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.134559+0000","last_undegraded":"2024-04-26T21:37:37.134559+0000","last_fullsized":"2024-04-26T21:37:37.134559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.157301+0000","last_change":"2024-04-26T21:37:37.153455+0000","last_active":"2024-04-26T21:37:37.157301+0000","last_peered":"2024-04-26T21:37:37.157301+0000","last_clean":"2024-04-26T21:37:37.157301+0000","last_became_active":"2024-04-26T21:37:37.153181+0000","last_became_peered":"2024-04-26T21:37:37.153181+0000","last_unstale":"2024-04-26T21:37:37.157301+0000","last_undegraded":"2024-04-26T21:37:37.157301+0000","last_fullsized":"2024-04-26T21:37:37.157301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:47.599920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.140819+0000","last_change":"2024-04-26T21:37:37.140819+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.140819+0000","last_undegraded":"2024-04-26T21:37:37.140819+0000","last_fullsized":"2024-04-26T21:37:37.140819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.150089+0000","last_change":"2024-04-26T21:37:37.144882+0000","last_active":"2024-04-26T21:37:37.150089+0000","last_peered":"2024-04-26T21:37:37.150089+0000","last_clean":"2024-04-26T21:37:37.150089+0000","last_became_active":"2024-04-26T21:37:37.144633+0000","last_became_peered":"2024-04-26T21:37:37.144633+0000","last_unstale":"2024-04-26T21:37:37.150089+0000","last_undegraded":"2024-04-26T21:37:37.150089+0000","last_fullsized":"2024-04-26T21:37:37.150089+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:53.683181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.136338+0000","last_change":"2024-04-26T21:37:37.136338+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.136338+0000","last_undegraded":"2024-04-26T21:37:37.136338+0000","last_fullsized":"2024-04-26T21:37:37.136338+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.155915+0000","last_change":"2024-04-26T21:37:37.150695+0000","last_active":"2024-04-26T21:37:37.155915+0000","last_peered":"2024-04-26T21:37:37.155915+0000","last_clean":"2024-04-26T21:37:37.155915+0000","last_became_active":"2024-04-26T21:37:37.150398+0000","last_became_peered":"2024-04-26T21:37:37.150398+0000","last_unstale":"2024-04-26T21:37:37.155915+0000","last_undegraded":"2024-04-26T21:37:37.155915+0000","last_fullsized":"2024-04-26T21:37:37.155915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-04-26T21:37:37.141329+0000","last_change":"2024-04-26T21:37:37.141329+0000","last_active":"2024-04-26T21:37:37.128336+0000","last_peered":"2024-04-26T21:37:37.128336+0000","last_clean":"2024-04-26T21:37:37.128336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:37:37.141329+0000","last_undegraded":"2024-04-26T21:37:37.141329+0000","last_fullsized":"2024-04-26T21:37:37.141329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.159474+0000","last_change":"2024-04-26T21:37:37.156817+0000","last_active":"2024-04-26T21:37:37.159474+0000","last_peered":"2024-04-26T21:37:37.159474+0000","last_clean":"2024-04-26T21:37:37.159474+0000","last_became_active":"2024-04-26T21:37:37.156561+0000","last_became_peered":"2024-04-26T21:37:37.156561+0000","last_unstale":"2024-04-26T21:37:37.159474+0000","last_undegraded":"2024-04-26T21:37:37.159474+0000","last_fullsized":"2024-04-26T21:37:37.159474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.883299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":226,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:37:37.140105+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:37:37.140105+0000","last_peered":"2024-04-26T21:37:37.140105+0000","last_clean":"2024-04-26T21:37:37.140105+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:37:37.140105+0000","last_undegraded":"2024-04-26T21:37:37.140105+0000","last_fullsized":"2024-04-26T21:37:37.140105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":263520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":853888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:37:44.122 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:37:44.123+0000 7fb98f204640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:37:46.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:37:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.744+0000 7f5b799af640 1 Processor -- start 2024-04-26T21:37:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.744+0000 7f5b799af640 1 -- start start 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b799af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b74162810 0x7f5b74070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b799af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5b740d1b30 0x7f5b740d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b799af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b740d10d0 con 0x7f5b740b2340 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b799af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5b740d0750 con 0x7f5b740d1b30 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b799af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b740d16a0 con 0x7f5b74162810 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b727fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5b740b2340 0x7f5b740cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60530/0 (socket says 172.21.15.40:60530) 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b727fc640 1 -- 172.21.15.40:0/232287392 learned_addr learned my addr 172.21.15.40:0/232287392 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b737fe640 1 --2- 172.21.15.40:0/232287392 >> v2:172.21.15.184:3300/0 conn(0x7f5b740d1b30 0x7f5b740d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b72ffd640 1 --2- 172.21.15.40:0/232287392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b74162810 0x7f5b74070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b71ffb640 1 -- 172.21.15.40:0/232287392 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230063984 0 0) 0x7f5b740d10d0 con 0x7f5b740b2340 2024-04-26T21:37:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b71ffb640 1 -- 172.21.15.40:0/232287392 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b5c003520 con 0x7f5b740b2340 2024-04-26T21:37:46.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b71ffb640 1 -- 172.21.15.40:0/232287392 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1295916398 0 0) 0x7f5b5c003520 con 0x7f5b740b2340 2024-04-26T21:37:46.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b72ffd640 1 -- 172.21.15.40:0/232287392 >> v2:172.21.15.184:3300/0 conn(0x7f5b740d1b30 msgr2=0x7f5b740d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:46.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b72ffd640 1 --2- 172.21.15.40:0/232287392 >> v2:172.21.15.184:3300/0 conn(0x7f5b740d1b30 0x7f5b740d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b72ffd640 1 -- 172.21.15.40:0/232287392 >> v1:172.21.15.40:6789/0 conn(0x7f5b740b2340 legacy=0x7f5b740cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:46.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b72ffd640 1 -- 172.21.15.40:0/232287392 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b740d1370 con 0x7f5b74162810 2024-04-26T21:37:46.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.745+0000 7f5b737fe640 1 --2- 172.21.15.40:0/232287392 >> v2:172.21.15.184:3300/0 conn(0x7f5b740d1b30 0x7f5b740d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:46.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.746+0000 7f5b72ffd640 1 --2- 172.21.15.40:0/232287392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b74162810 0x7f5b74070350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097a0 tx=0x7f5b6402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4552b0cf5a62568a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:46.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b71ffb640 1 -- 172.21.15.40:0/232287392 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64037070 con 0x7f5b74162810 2024-04-26T21:37:46.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b71ffb640 1 -- 172.21.15.40:0/232287392 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6402aba0 con 0x7f5b74162810 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b71ffb640 1 -- 172.21.15.40:0/232287392 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68001670 con 0x7f5b74162810 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b799af640 1 -- 172.21.15.40:0/232287392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b74162810 msgr2=0x7f5b74070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b799af640 1 --2- 172.21.15.40:0/232287392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b74162810 0x7f5b74070350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097a0 tx=0x7f5b6402a110 comp rx=0 tx=0).stop 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b799af640 1 -- 172.21.15.40:0/232287392 shutdown_connections 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b799af640 1 --2- 172.21.15.40:0/232287392 >> v2:172.21.15.184:3300/0 conn(0x7f5b740d1b30 0x7f5b740d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b799af640 1 --2- 172.21.15.40:0/232287392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b74162810 0x7f5b74070350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b799af640 1 -- 172.21.15.40:0/232287392 >> 172.21.15.40:0/232287392 conn(0x7f5b740d2da0 msgr2=0x7f5b740d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b799af640 1 -- 172.21.15.40:0/232287392 shutdown_connections 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.747+0000 7f5b799af640 1 -- 172.21.15.40:0/232287392 wait complete. 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.748+0000 7f5b799af640 1 Processor -- start 2024-04-26T21:37:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b799af640 1 -- start start 2024-04-26T21:37:46.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b799af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740b2340 0x7f5b740c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:46.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b799af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b740d1b30 0x7f5b7414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:46.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b799af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b74162810 0x7f5b740c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:46.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b799af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b74070890 con 0x7f5b740b2340 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b799af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740d0470 con 0x7f5b74162810 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b799af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b740cdb00 con 0x7f5b740d1b30 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b72ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740b2340 0x7f5b740c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b72ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740b2340 0x7f5b740c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57132/0 (socket says 172.21.15.40:57132) 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.749+0000 7f5b72ffd640 1 -- 172.21.15.40:0/1704769693 learned_addr learned my addr 172.21.15.40:0/1704769693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b727fc640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b740d1b30 0x7f5b7414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b737fe640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b74162810 0x7f5b740c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b72ffd640 1 -- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b740d1b30 msgr2=0x7f5b7414db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b72ffd640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b740d1b30 0x7f5b7414db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b72ffd640 1 -- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b74162810 msgr2=0x7f5b740c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b72ffd640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b74162810 0x7f5b740c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b72ffd640 1 -- 172.21.15.40:0/1704769693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b740c7c70 con 0x7f5b740b2340 2024-04-26T21:37:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b72ffd640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740b2340 0x7f5b740c7730 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097a0 tx=0x7f5b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:46.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b5bfff640 1 -- 172.21.15.40:0/1704769693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64042020 con 0x7f5b740b2340 2024-04-26T21:37:46.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b5bfff640 1 -- 172.21.15.40:0/1704769693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6403d670 con 0x7f5b740b2340 2024-04-26T21:37:46.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b5bfff640 1 -- 172.21.15.40:0/1704769693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64037040 con 0x7f5b740b2340 2024-04-26T21:37:46.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b740cb0d0 con 0x7f5b740b2340 2024-04-26T21:37:46.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.750+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b740cb690 con 0x7f5b740b2340 2024-04-26T21:37:46.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.752+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b740cd7f0 con 0x7f5b740b2340 2024-04-26T21:37:46.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.752+0000 7f5b5bfff640 1 -- 172.21.15.40:0/1704769693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b64045020 con 0x7f5b740b2340 2024-04-26T21:37:46.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.752+0000 7f5b5bfff640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b6006c8f0 0x7f5b6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:46.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.753+0000 7f5b5bfff640 1 -- 172.21.15.40:0/1704769693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f5b640ac3b0 con 0x7f5b740b2340 2024-04-26T21:37:46.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.753+0000 7f5b727fc640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b6006c8f0 0x7f5b6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:46.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.753+0000 7f5b727fc640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b6006c8f0 0x7f5b6006edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b741f8a80 tx=0x7f5b680057f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:46.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.757+0000 7f5b5bfff640 1 -- 172.21.15.40:0/1704769693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b740cd7f0 con 0x7f5b740b2340 2024-04-26T21:37:46.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.938+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b740694f0 con 0x7f5b6006c8f0 2024-04-26T21:37:46.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.944+0000 7f5b5bfff640 1 -- 172.21.15.40:0/1704769693 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f5b740694f0 con 0x7f5b6006c8f0 2024-04-26T21:37:46.947 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:46.948 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:37:46.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b6006c8f0 msgr2=0x7f5b6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:46.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b6006c8f0 0x7f5b6006edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b741f8a80 tx=0x7f5b680057f0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740b2340 msgr2=0x7f5b740c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740b2340 0x7f5b740c7730 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097a0 tx=0x7f5b640046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 shutdown_connections 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b6006c8f0 0x7f5b6006edb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b74162810 0x7f5b740c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b740d1b30 0x7f5b7414db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 --2- 172.21.15.40:0/1704769693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740b2340 0x7f5b740c7730 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 >> 172.21.15.40:0/1704769693 conn(0x7f5b740d2da0 msgr2=0x7f5b74070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 shutdown_connections 2024-04-26T21:37:46.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:46.947+0000 7f5b799af640 1 -- 172.21.15.40:0/1704769693 wait complete. 2024-04-26T21:37:46.958 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-04-26T21:37:46.030945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222248,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752472,"statfs":{"total":773094113280,"available":772866531328,"internally_reserved":0,"allocated":6201344,"data_stored":3404588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002171"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147972+0000","last_change":"2024-04-26T21:37:38.183139+0000","last_active":"2024-04-26T21:37:39.147972+0000","last_peered":"2024-04-26T21:37:39.147972+0000","last_clean":"2024-04-26T21:37:39.147972+0000","last_became_active":"2024-04-26T21:37:38.182810+0000","last_became_peered":"2024-04-26T21:37:38.182810+0000","last_unstale":"2024-04-26T21:37:39.147972+0000","last_undegraded":"2024-04-26T21:37:39.147972+0000","last_fullsized":"2024-04-26T21:37:39.147972+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:44:11.899692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526216+0000","last_change":"2024-04-26T21:37:38.165038+0000","last_active":"2024-04-26T21:37:39.526216+0000","last_peered":"2024-04-26T21:37:39.526216+0000","last_clean":"2024-04-26T21:37:39.526216+0000","last_became_active":"2024-04-26T21:37:38.164776+0000","last_became_peered":"2024-04-26T21:37:38.164776+0000","last_unstale":"2024-04-26T21:37:39.526216+0000","last_undegraded":"2024-04-26T21:37:39.526216+0000","last_fullsized":"2024-04-26T21:37:39.526216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:37.279900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525573+0000","last_change":"2024-04-26T21:37:38.175271+0000","last_active":"2024-04-26T21:37:39.525573+0000","last_peered":"2024-04-26T21:37:39.525573+0000","last_clean":"2024-04-26T21:37:39.525573+0000","last_became_active":"2024-04-26T21:37:38.174573+0000","last_became_peered":"2024-04-26T21:37:38.174573+0000","last_unstale":"2024-04-26T21:37:39.525573+0000","last_undegraded":"2024-04-26T21:37:39.525573+0000","last_fullsized":"2024-04-26T21:37:39.525573+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:22.973015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526010+0000","last_change":"2024-04-26T21:37:38.170843+0000","last_active":"2024-04-26T21:37:39.526010+0000","last_peered":"2024-04-26T21:37:39.526010+0000","last_clean":"2024-04-26T21:37:39.526010+0000","last_became_active":"2024-04-26T21:37:38.170486+0000","last_became_peered":"2024-04-26T21:37:38.170486+0000","last_unstale":"2024-04-26T21:37:39.526010+0000","last_undegraded":"2024-04-26T21:37:39.526010+0000","last_fullsized":"2024-04-26T21:37:39.526010+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:41.814089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525119+0000","last_change":"2024-04-26T21:37:38.164045+0000","last_active":"2024-04-26T21:37:39.525119+0000","last_peered":"2024-04-26T21:37:39.525119+0000","last_clean":"2024-04-26T21:37:39.525119+0000","last_became_active":"2024-04-26T21:37:38.163769+0000","last_became_peered":"2024-04-26T21:37:38.163769+0000","last_unstale":"2024-04-26T21:37:39.525119+0000","last_undegraded":"2024-04-26T21:37:39.525119+0000","last_fullsized":"2024-04-26T21:37:39.525119+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:39.191649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147658+0000","last_change":"2024-04-26T21:37:38.172524+0000","last_active":"2024-04-26T21:37:39.147658+0000","last_peered":"2024-04-26T21:37:39.147658+0000","last_clean":"2024-04-26T21:37:39.147658+0000","last_became_active":"2024-04-26T21:37:38.172206+0000","last_became_peered":"2024-04-26T21:37:38.172206+0000","last_unstale":"2024-04-26T21:37:39.147658+0000","last_undegraded":"2024-04-26T21:37:39.147658+0000","last_fullsized":"2024-04-26T21:37:39.147658+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:55.364381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525158+0000","last_change":"2024-04-26T21:37:38.176018+0000","last_active":"2024-04-26T21:37:39.525158+0000","last_peered":"2024-04-26T21:37:39.525158+0000","last_clean":"2024-04-26T21:37:39.525158+0000","last_became_active":"2024-04-26T21:37:38.175707+0000","last_became_peered":"2024-04-26T21:37:38.175707+0000","last_unstale":"2024-04-26T21:37:39.525158+0000","last_undegraded":"2024-04-26T21:37:39.525158+0000","last_fullsized":"2024-04-26T21:37:39.525158+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:08.459021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524903+0000","last_change":"2024-04-26T21:37:38.171488+0000","last_active":"2024-04-26T21:37:39.524903+0000","last_peered":"2024-04-26T21:37:39.524903+0000","last_clean":"2024-04-26T21:37:39.524903+0000","last_became_active":"2024-04-26T21:37:38.169575+0000","last_became_peered":"2024-04-26T21:37:38.169575+0000","last_unstale":"2024-04-26T21:37:39.524903+0000","last_undegraded":"2024-04-26T21:37:39.524903+0000","last_fullsized":"2024-04-26T21:37:39.524903+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:08.710263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525685+0000","last_change":"2024-04-26T21:37:38.179434+0000","last_active":"2024-04-26T21:37:39.525685+0000","last_peered":"2024-04-26T21:37:39.525685+0000","last_clean":"2024-04-26T21:37:39.525685+0000","last_became_active":"2024-04-26T21:37:38.179140+0000","last_became_peered":"2024-04-26T21:37:38.179140+0000","last_unstale":"2024-04-26T21:37:39.525685+0000","last_undegraded":"2024-04-26T21:37:39.525685+0000","last_fullsized":"2024-04-26T21:37:39.525685+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:22.973015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147966+0000","last_change":"2024-04-26T21:37:38.171723+0000","last_active":"2024-04-26T21:37:39.147966+0000","last_peered":"2024-04-26T21:37:39.147966+0000","last_clean":"2024-04-26T21:37:39.147966+0000","last_became_active":"2024-04-26T21:37:38.171453+0000","last_became_peered":"2024-04-26T21:37:38.171453+0000","last_unstale":"2024-04-26T21:37:39.147966+0000","last_undegraded":"2024-04-26T21:37:39.147966+0000","last_fullsized":"2024-04-26T21:37:39.147966+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:52.866968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147332+0000","last_change":"2024-04-26T21:37:38.162518+0000","last_active":"2024-04-26T21:37:39.147332+0000","last_peered":"2024-04-26T21:37:39.147332+0000","last_clean":"2024-04-26T21:37:39.147332+0000","last_became_active":"2024-04-26T21:37:38.162224+0000","last_became_peered":"2024-04-26T21:37:38.162224+0000","last_unstale":"2024-04-26T21:37:39.147332+0000","last_undegraded":"2024-04-26T21:37:39.147332+0000","last_fullsized":"2024-04-26T21:37:39.147332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:09.569265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525599+0000","last_change":"2024-04-26T21:37:38.167099+0000","last_active":"2024-04-26T21:37:39.525599+0000","last_peered":"2024-04-26T21:37:39.525599+0000","last_clean":"2024-04-26T21:37:39.525599+0000","last_became_active":"2024-04-26T21:37:38.166815+0000","last_became_peered":"2024-04-26T21:37:38.166815+0000","last_unstale":"2024-04-26T21:37:39.525599+0000","last_undegraded":"2024-04-26T21:37:39.525599+0000","last_fullsized":"2024-04-26T21:37:39.525599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:50.180281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147688+0000","last_change":"2024-04-26T21:37:38.175892+0000","last_active":"2024-04-26T21:37:39.147688+0000","last_peered":"2024-04-26T21:37:39.147688+0000","last_clean":"2024-04-26T21:37:39.147688+0000","last_became_active":"2024-04-26T21:37:38.175595+0000","last_became_peered":"2024-04-26T21:37:38.175595+0000","last_unstale":"2024-04-26T21:37:39.147688+0000","last_undegraded":"2024-04-26T21:37:39.147688+0000","last_fullsized":"2024-04-26T21:37:39.147688+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:21.171590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525465+0000","last_change":"2024-04-26T21:37:38.174122+0000","last_active":"2024-04-26T21:37:39.525465+0000","last_peered":"2024-04-26T21:37:39.525465+0000","last_clean":"2024-04-26T21:37:39.525465+0000","last_became_active":"2024-04-26T21:37:38.173653+0000","last_became_peered":"2024-04-26T21:37:38.173653+0000","last_unstale":"2024-04-26T21:37:39.525465+0000","last_undegraded":"2024-04-26T21:37:39.525465+0000","last_fullsized":"2024-04-26T21:37:39.525465+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:10.522313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525723+0000","last_change":"2024-04-26T21:37:38.171839+0000","last_active":"2024-04-26T21:37:39.525723+0000","last_peered":"2024-04-26T21:37:39.525723+0000","last_clean":"2024-04-26T21:37:39.525723+0000","last_became_active":"2024-04-26T21:37:38.171525+0000","last_became_peered":"2024-04-26T21:37:38.171525+0000","last_unstale":"2024-04-26T21:37:39.525723+0000","last_undegraded":"2024-04-26T21:37:39.525723+0000","last_fullsized":"2024-04-26T21:37:39.525723+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:58.890761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147488+0000","last_change":"2024-04-26T21:37:38.172447+0000","last_active":"2024-04-26T21:37:39.147488+0000","last_peered":"2024-04-26T21:37:39.147488+0000","last_clean":"2024-04-26T21:37:39.147488+0000","last_became_active":"2024-04-26T21:37:38.172178+0000","last_became_peered":"2024-04-26T21:37:38.172178+0000","last_unstale":"2024-04-26T21:37:39.147488+0000","last_undegraded":"2024-04-26T21:37:39.147488+0000","last_fullsized":"2024-04-26T21:37:39.147488+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:49.667212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525722+0000","last_change":"2024-04-26T21:37:38.167632+0000","last_active":"2024-04-26T21:37:39.525722+0000","last_peered":"2024-04-26T21:37:39.525722+0000","last_clean":"2024-04-26T21:37:39.525722+0000","last_became_active":"2024-04-26T21:37:38.167342+0000","last_became_peered":"2024-04-26T21:37:38.167342+0000","last_unstale":"2024-04-26T21:37:39.525722+0000","last_undegraded":"2024-04-26T21:37:39.525722+0000","last_fullsized":"2024-04-26T21:37:39.525722+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:02.905026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526223+0000","last_change":"2024-04-26T21:37:38.165730+0000","last_active":"2024-04-26T21:37:39.526223+0000","last_peered":"2024-04-26T21:37:39.526223+0000","last_clean":"2024-04-26T21:37:39.526223+0000","last_became_active":"2024-04-26T21:37:38.165455+0000","last_became_peered":"2024-04-26T21:37:38.165455+0000","last_unstale":"2024-04-26T21:37:39.526223+0000","last_undegraded":"2024-04-26T21:37:39.526223+0000","last_fullsized":"2024-04-26T21:37:39.526223+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:34.453645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525611+0000","last_change":"2024-04-26T21:37:38.166488+0000","last_active":"2024-04-26T21:37:39.525611+0000","last_peered":"2024-04-26T21:37:39.525611+0000","last_clean":"2024-04-26T21:37:39.525611+0000","last_became_active":"2024-04-26T21:37:38.166179+0000","last_became_peered":"2024-04-26T21:37:38.166179+0000","last_unstale":"2024-04-26T21:37:39.525611+0000","last_undegraded":"2024-04-26T21:37:39.525611+0000","last_fullsized":"2024-04-26T21:37:39.525611+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:10.670811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525067+0000","last_change":"2024-04-26T21:37:38.167941+0000","last_active":"2024-04-26T21:37:39.525067+0000","last_peered":"2024-04-26T21:37:39.525067+0000","last_clean":"2024-04-26T21:37:39.525067+0000","last_became_active":"2024-04-26T21:37:38.167616+0000","last_became_peered":"2024-04-26T21:37:38.167616+0000","last_unstale":"2024-04-26T21:37:39.525067+0000","last_undegraded":"2024-04-26T21:37:39.525067+0000","last_fullsized":"2024-04-26T21:37:39.525067+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:49.667212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524798+0000","last_change":"2024-04-26T21:37:38.166491+0000","last_active":"2024-04-26T21:37:39.524798+0000","last_peered":"2024-04-26T21:37:39.524798+0000","last_clean":"2024-04-26T21:37:39.524798+0000","last_became_active":"2024-04-26T21:37:38.166108+0000","last_became_peered":"2024-04-26T21:37:38.166108+0000","last_unstale":"2024-04-26T21:37:39.524798+0000","last_undegraded":"2024-04-26T21:37:39.524798+0000","last_fullsized":"2024-04-26T21:37:39.524798+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:05.325898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524882+0000","last_change":"2024-04-26T21:37:38.169117+0000","last_active":"2024-04-26T21:37:39.524882+0000","last_peered":"2024-04-26T21:37:39.524882+0000","last_clean":"2024-04-26T21:37:39.524882+0000","last_became_active":"2024-04-26T21:37:38.168830+0000","last_became_peered":"2024-04-26T21:37:38.168830+0000","last_unstale":"2024-04-26T21:37:39.524882+0000","last_undegraded":"2024-04-26T21:37:39.524882+0000","last_fullsized":"2024-04-26T21:37:39.524882+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:08.459021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.146898+0000","last_change":"2024-04-26T21:37:38.167450+0000","last_active":"2024-04-26T21:37:39.146898+0000","last_peered":"2024-04-26T21:37:39.146898+0000","last_clean":"2024-04-26T21:37:39.146898+0000","last_became_active":"2024-04-26T21:37:38.167101+0000","last_became_peered":"2024-04-26T21:37:38.167101+0000","last_unstale":"2024-04-26T21:37:39.146898+0000","last_undegraded":"2024-04-26T21:37:39.146898+0000","last_fullsized":"2024-04-26T21:37:39.146898+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:15.854116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524687+0000","last_change":"2024-04-26T21:37:38.173305+0000","last_active":"2024-04-26T21:37:39.524687+0000","last_peered":"2024-04-26T21:37:39.524687+0000","last_clean":"2024-04-26T21:37:39.524687+0000","last_became_active":"2024-04-26T21:37:38.173028+0000","last_became_peered":"2024-04-26T21:37:38.173028+0000","last_unstale":"2024-04-26T21:37:39.524687+0000","last_undegraded":"2024-04-26T21:37:39.524687+0000","last_fullsized":"2024-04-26T21:37:39.524687+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:12.977121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525812+0000","last_change":"2024-04-26T21:37:38.171484+0000","last_active":"2024-04-26T21:37:39.525812+0000","last_peered":"2024-04-26T21:37:39.525812+0000","last_clean":"2024-04-26T21:37:39.525812+0000","last_became_active":"2024-04-26T21:37:38.171189+0000","last_became_peered":"2024-04-26T21:37:38.171189+0000","last_unstale":"2024-04-26T21:37:39.525812+0000","last_undegraded":"2024-04-26T21:37:39.525812+0000","last_fullsized":"2024-04-26T21:37:39.525812+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:21.141051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525956+0000","last_change":"2024-04-26T21:37:38.167891+0000","last_active":"2024-04-26T21:37:39.525956+0000","last_peered":"2024-04-26T21:37:39.525956+0000","last_clean":"2024-04-26T21:37:39.525956+0000","last_became_active":"2024-04-26T21:37:38.167618+0000","last_became_peered":"2024-04-26T21:37:38.167618+0000","last_unstale":"2024-04-26T21:37:39.525956+0000","last_undegraded":"2024-04-26T21:37:39.525956+0000","last_fullsized":"2024-04-26T21:37:39.525956+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:02.905026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147984+0000","last_change":"2024-04-26T21:37:38.181650+0000","last_active":"2024-04-26T21:37:39.147984+0000","last_peered":"2024-04-26T21:37:39.147984+0000","last_clean":"2024-04-26T21:37:39.147984+0000","last_became_active":"2024-04-26T21:37:38.181345+0000","last_became_peered":"2024-04-26T21:37:38.181345+0000","last_unstale":"2024-04-26T21:37:39.147984+0000","last_undegraded":"2024-04-26T21:37:39.147984+0000","last_fullsized":"2024-04-26T21:37:39.147984+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:20.164982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525066+0000","last_change":"2024-04-26T21:37:38.169420+0000","last_active":"2024-04-26T21:37:39.525066+0000","last_peered":"2024-04-26T21:37:39.525066+0000","last_clean":"2024-04-26T21:37:39.525066+0000","last_became_active":"2024-04-26T21:37:38.169165+0000","last_became_peered":"2024-04-26T21:37:38.169165+0000","last_unstale":"2024-04-26T21:37:39.525066+0000","last_undegraded":"2024-04-26T21:37:39.525066+0000","last_fullsized":"2024-04-26T21:37:39.525066+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:32.928106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525147+0000","last_change":"2024-04-26T21:37:38.173459+0000","last_active":"2024-04-26T21:37:39.525147+0000","last_peered":"2024-04-26T21:37:39.525147+0000","last_clean":"2024-04-26T21:37:39.525147+0000","last_became_active":"2024-04-26T21:37:38.173161+0000","last_became_peered":"2024-04-26T21:37:38.173161+0000","last_unstale":"2024-04-26T21:37:39.525147+0000","last_undegraded":"2024-04-26T21:37:39.525147+0000","last_fullsized":"2024-04-26T21:37:39.525147+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:49.021226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525935+0000","last_change":"2024-04-26T21:37:38.169219+0000","last_active":"2024-04-26T21:37:39.525935+0000","last_peered":"2024-04-26T21:37:39.525935+0000","last_clean":"2024-04-26T21:37:39.525935+0000","last_became_active":"2024-04-26T21:37:38.168947+0000","last_became_peered":"2024-04-26T21:37:38.168947+0000","last_unstale":"2024-04-26T21:37:39.525935+0000","last_undegraded":"2024-04-26T21:37:39.525935+0000","last_fullsized":"2024-04-26T21:37:39.525935+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:45.227011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525250+0000","last_change":"2024-04-26T21:37:38.175972+0000","last_active":"2024-04-26T21:37:39.525250+0000","last_peered":"2024-04-26T21:37:39.525250+0000","last_clean":"2024-04-26T21:37:39.525250+0000","last_became_active":"2024-04-26T21:37:38.175356+0000","last_became_peered":"2024-04-26T21:37:38.175356+0000","last_unstale":"2024-04-26T21:37:39.525250+0000","last_undegraded":"2024-04-26T21:37:39.525250+0000","last_fullsized":"2024-04-26T21:37:39.525250+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:08.710263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147851+0000","last_change":"2024-04-26T21:37:38.169481+0000","last_active":"2024-04-26T21:37:39.147851+0000","last_peered":"2024-04-26T21:37:39.147851+0000","last_clean":"2024-04-26T21:37:39.147851+0000","last_became_active":"2024-04-26T21:37:38.169185+0000","last_became_peered":"2024-04-26T21:37:38.169185+0000","last_unstale":"2024-04-26T21:37:39.147851+0000","last_undegraded":"2024-04-26T21:37:39.147851+0000","last_fullsized":"2024-04-26T21:37:39.147851+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:38.450634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525485+0000","last_change":"2024-04-26T21:37:38.162257+0000","last_active":"2024-04-26T21:37:39.525485+0000","last_peered":"2024-04-26T21:37:39.525485+0000","last_clean":"2024-04-26T21:37:39.525485+0000","last_became_active":"2024-04-26T21:37:38.161945+0000","last_became_peered":"2024-04-26T21:37:38.161945+0000","last_unstale":"2024-04-26T21:37:39.525485+0000","last_undegraded":"2024-04-26T21:37:39.525485+0000","last_fullsized":"2024-04-26T21:37:39.525485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:29.462561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525010+0000","last_change":"2024-04-26T21:37:37.150925+0000","last_active":"2024-04-26T21:37:39.525010+0000","last_peered":"2024-04-26T21:37:39.525010+0000","last_clean":"2024-04-26T21:37:39.525010+0000","last_became_active":"2024-04-26T21:37:37.150597+0000","last_became_peered":"2024-04-26T21:37:37.150597+0000","last_unstale":"2024-04-26T21:37:39.525010+0000","last_undegraded":"2024-04-26T21:37:39.525010+0000","last_fullsized":"2024-04-26T21:37:39.525010+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:43.766358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147856+0000","last_change":"2024-04-26T21:37:38.164546+0000","last_active":"2024-04-26T21:37:39.147856+0000","last_peered":"2024-04-26T21:37:39.147856+0000","last_clean":"2024-04-26T21:37:39.147856+0000","last_became_active":"2024-04-26T21:37:38.163478+0000","last_became_peered":"2024-04-26T21:37:38.163478+0000","last_unstale":"2024-04-26T21:37:39.147856+0000","last_undegraded":"2024-04-26T21:37:39.147856+0000","last_fullsized":"2024-04-26T21:37:39.147856+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:49.460135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.148066+0000","last_change":"2024-04-26T21:37:37.156594+0000","last_active":"2024-04-26T21:37:39.148066+0000","last_peered":"2024-04-26T21:37:39.148066+0000","last_clean":"2024-04-26T21:37:39.148066+0000","last_became_active":"2024-04-26T21:37:37.156312+0000","last_became_peered":"2024-04-26T21:37:37.156312+0000","last_unstale":"2024-04-26T21:37:39.148066+0000","last_undegraded":"2024-04-26T21:37:39.148066+0000","last_fullsized":"2024-04-26T21:37:39.148066+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:11.817532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524586+0000","last_change":"2024-04-26T21:37:38.171954+0000","last_active":"2024-04-26T21:37:39.524586+0000","last_peered":"2024-04-26T21:37:39.524586+0000","last_clean":"2024-04-26T21:37:39.524586+0000","last_became_active":"2024-04-26T21:37:38.171617+0000","last_became_peered":"2024-04-26T21:37:38.171617+0000","last_unstale":"2024-04-26T21:37:39.524586+0000","last_undegraded":"2024-04-26T21:37:39.524586+0000","last_fullsized":"2024-04-26T21:37:39.524586+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:31.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.148047+0000","last_change":"2024-04-26T21:37:37.158086+0000","last_active":"2024-04-26T21:37:39.148047+0000","last_peered":"2024-04-26T21:37:39.148047+0000","last_clean":"2024-04-26T21:37:39.148047+0000","last_became_active":"2024-04-26T21:37:37.157790+0000","last_became_peered":"2024-04-26T21:37:37.157790+0000","last_unstale":"2024-04-26T21:37:39.148047+0000","last_undegraded":"2024-04-26T21:37:39.148047+0000","last_fullsized":"2024-04-26T21:37:39.148047+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:50.687873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525011+0000","last_change":"2024-04-26T21:37:38.169705+0000","last_active":"2024-04-26T21:37:39.525011+0000","last_peered":"2024-04-26T21:37:39.525011+0000","last_clean":"2024-04-26T21:37:39.525011+0000","last_became_active":"2024-04-26T21:37:38.169424+0000","last_became_peered":"2024-04-26T21:37:38.169424+0000","last_unstale":"2024-04-26T21:37:39.525011+0000","last_undegraded":"2024-04-26T21:37:39.525011+0000","last_fullsized":"2024-04-26T21:37:39.525011+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:56.069223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.148058+0000","last_change":"2024-04-26T21:37:37.140298+0000","last_active":"2024-04-26T21:37:39.148058+0000","last_peered":"2024-04-26T21:37:39.148058+0000","last_clean":"2024-04-26T21:37:39.148058+0000","last_became_active":"2024-04-26T21:37:37.139801+0000","last_became_peered":"2024-04-26T21:37:37.139801+0000","last_unstale":"2024-04-26T21:37:39.148058+0000","last_undegraded":"2024-04-26T21:37:39.148058+0000","last_fullsized":"2024-04-26T21:37:39.148058+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:47.599920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525366+0000","last_change":"2024-04-26T21:37:38.167700+0000","last_active":"2024-04-26T21:37:39.525366+0000","last_peered":"2024-04-26T21:37:39.525366+0000","last_clean":"2024-04-26T21:37:39.525366+0000","last_became_active":"2024-04-26T21:37:38.167436+0000","last_became_peered":"2024-04-26T21:37:38.167436+0000","last_unstale":"2024-04-26T21:37:39.525366+0000","last_undegraded":"2024-04-26T21:37:39.525366+0000","last_fullsized":"2024-04-26T21:37:39.525366+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:31.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525244+0000","last_change":"2024-04-26T21:37:37.158278+0000","last_active":"2024-04-26T21:37:39.525244+0000","last_peered":"2024-04-26T21:37:39.525244+0000","last_clean":"2024-04-26T21:37:39.525244+0000","last_became_active":"2024-04-26T21:37:37.154700+0000","last_became_peered":"2024-04-26T21:37:37.154700+0000","last_unstale":"2024-04-26T21:37:39.525244+0000","last_undegraded":"2024-04-26T21:37:39.525244+0000","last_fullsized":"2024-04-26T21:37:39.525244+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.845057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526053+0000","last_change":"2024-04-26T21:37:38.180804+0000","last_active":"2024-04-26T21:37:39.526053+0000","last_peered":"2024-04-26T21:37:39.526053+0000","last_clean":"2024-04-26T21:37:39.526053+0000","last_became_active":"2024-04-26T21:37:38.180537+0000","last_became_peered":"2024-04-26T21:37:38.180537+0000","last_unstale":"2024-04-26T21:37:39.526053+0000","last_undegraded":"2024-04-26T21:37:39.526053+0000","last_fullsized":"2024-04-26T21:37:39.526053+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:38.450634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524849+0000","last_change":"2024-04-26T21:37:37.148427+0000","last_active":"2024-04-26T21:37:39.524849+0000","last_peered":"2024-04-26T21:37:39.524849+0000","last_clean":"2024-04-26T21:37:39.524849+0000","last_became_active":"2024-04-26T21:37:37.147872+0000","last_became_peered":"2024-04-26T21:37:37.147872+0000","last_unstale":"2024-04-26T21:37:39.524849+0000","last_undegraded":"2024-04-26T21:37:39.524849+0000","last_fullsized":"2024-04-26T21:37:39.524849+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:28.173221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526086+0000","last_change":"2024-04-26T21:37:38.158433+0000","last_active":"2024-04-26T21:37:39.526086+0000","last_peered":"2024-04-26T21:37:39.526086+0000","last_clean":"2024-04-26T21:37:39.526086+0000","last_became_active":"2024-04-26T21:37:38.157835+0000","last_became_peered":"2024-04-26T21:37:38.157835+0000","last_unstale":"2024-04-26T21:37:39.526086+0000","last_undegraded":"2024-04-26T21:37:39.526086+0000","last_fullsized":"2024-04-26T21:37:39.526086+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:23.209666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524509+0000","last_change":"2024-04-26T21:37:37.147914+0000","last_active":"2024-04-26T21:37:39.524509+0000","last_peered":"2024-04-26T21:37:39.524509+0000","last_clean":"2024-04-26T21:37:39.524509+0000","last_became_active":"2024-04-26T21:37:37.147592+0000","last_became_peered":"2024-04-26T21:37:37.147592+0000","last_unstale":"2024-04-26T21:37:39.524509+0000","last_undegraded":"2024-04-26T21:37:39.524509+0000","last_fullsized":"2024-04-26T21:37:39.524509+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:15.680185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526231+0000","last_change":"2024-04-26T21:37:38.178337+0000","last_active":"2024-04-26T21:37:39.526231+0000","last_peered":"2024-04-26T21:37:39.526231+0000","last_clean":"2024-04-26T21:37:39.526231+0000","last_became_active":"2024-04-26T21:37:38.178038+0000","last_became_peered":"2024-04-26T21:37:38.178038+0000","last_unstale":"2024-04-26T21:37:39.526231+0000","last_undegraded":"2024-04-26T21:37:39.526231+0000","last_fullsized":"2024-04-26T21:37:39.526231+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:49.460135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524706+0000","last_change":"2024-04-26T21:37:37.153776+0000","last_active":"2024-04-26T21:37:39.524706+0000","last_peered":"2024-04-26T21:37:39.524706+0000","last_clean":"2024-04-26T21:37:39.524706+0000","last_became_active":"2024-04-26T21:37:37.153390+0000","last_became_peered":"2024-04-26T21:37:37.153390+0000","last_unstale":"2024-04-26T21:37:39.524706+0000","last_undegraded":"2024-04-26T21:37:39.524706+0000","last_fullsized":"2024-04-26T21:37:39.524706+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:48.659187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524696+0000","last_change":"2024-04-26T21:37:38.167763+0000","last_active":"2024-04-26T21:37:39.524696+0000","last_peered":"2024-04-26T21:37:39.524696+0000","last_clean":"2024-04-26T21:37:39.524696+0000","last_became_active":"2024-04-26T21:37:38.167454+0000","last_became_peered":"2024-04-26T21:37:38.167454+0000","last_unstale":"2024-04-26T21:37:39.524696+0000","last_undegraded":"2024-04-26T21:37:39.524696+0000","last_fullsized":"2024-04-26T21:37:39.524696+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:44.805919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526135+0000","last_change":"2024-04-26T21:37:37.153096+0000","last_active":"2024-04-26T21:37:39.526135+0000","last_peered":"2024-04-26T21:37:39.526135+0000","last_clean":"2024-04-26T21:37:39.526135+0000","last_became_active":"2024-04-26T21:37:37.152795+0000","last_became_peered":"2024-04-26T21:37:37.152795+0000","last_unstale":"2024-04-26T21:37:39.526135+0000","last_undegraded":"2024-04-26T21:37:39.526135+0000","last_fullsized":"2024-04-26T21:37:39.526135+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.845057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.148035+0000","last_change":"2024-04-26T21:37:38.161117+0000","last_active":"2024-04-26T21:37:39.148035+0000","last_peered":"2024-04-26T21:37:39.148035+0000","last_clean":"2024-04-26T21:37:39.148035+0000","last_became_active":"2024-04-26T21:37:38.160841+0000","last_became_peered":"2024-04-26T21:37:38.160841+0000","last_unstale":"2024-04-26T21:37:39.148035+0000","last_undegraded":"2024-04-26T21:37:39.148035+0000","last_fullsized":"2024-04-26T21:37:39.148035+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:01.032700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525375+0000","last_change":"2024-04-26T21:37:37.154318+0000","last_active":"2024-04-26T21:37:39.525375+0000","last_peered":"2024-04-26T21:37:39.525375+0000","last_clean":"2024-04-26T21:37:39.525375+0000","last_became_active":"2024-04-26T21:37:37.154072+0000","last_became_peered":"2024-04-26T21:37:37.154072+0000","last_unstale":"2024-04-26T21:37:39.525375+0000","last_undegraded":"2024-04-26T21:37:39.525375+0000","last_fullsized":"2024-04-26T21:37:39.525375+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:48.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525763+0000","last_change":"2024-04-26T21:37:38.167726+0000","last_active":"2024-04-26T21:37:39.525763+0000","last_peered":"2024-04-26T21:37:39.525763+0000","last_clean":"2024-04-26T21:37:39.525763+0000","last_became_active":"2024-04-26T21:37:38.167408+0000","last_became_peered":"2024-04-26T21:37:38.167408+0000","last_unstale":"2024-04-26T21:37:39.525763+0000","last_undegraded":"2024-04-26T21:37:39.525763+0000","last_fullsized":"2024-04-26T21:37:39.525763+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:23.209666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526146+0000","last_change":"2024-04-26T21:37:37.154992+0000","last_active":"2024-04-26T21:37:39.526146+0000","last_peered":"2024-04-26T21:37:39.526146+0000","last_clean":"2024-04-26T21:37:39.526146+0000","last_became_active":"2024-04-26T21:37:37.154711+0000","last_became_peered":"2024-04-26T21:37:37.154711+0000","last_unstale":"2024-04-26T21:37:39.526146+0000","last_undegraded":"2024-04-26T21:37:39.526146+0000","last_fullsized":"2024-04-26T21:37:39.526146+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:31.920081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525259+0000","last_change":"2024-04-26T21:37:38.157939+0000","last_active":"2024-04-26T21:37:39.525259+0000","last_peered":"2024-04-26T21:37:39.525259+0000","last_clean":"2024-04-26T21:37:39.525259+0000","last_became_active":"2024-04-26T21:37:38.156966+0000","last_became_peered":"2024-04-26T21:37:38.156966+0000","last_unstale":"2024-04-26T21:37:39.525259+0000","last_undegraded":"2024-04-26T21:37:39.525259+0000","last_fullsized":"2024-04-26T21:37:39.525259+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.891323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525754+0000","last_change":"2024-04-26T21:37:37.146807+0000","last_active":"2024-04-26T21:37:39.525754+0000","last_peered":"2024-04-26T21:37:39.525754+0000","last_clean":"2024-04-26T21:37:39.525754+0000","last_became_active":"2024-04-26T21:37:37.146479+0000","last_became_peered":"2024-04-26T21:37:37.146479+0000","last_unstale":"2024-04-26T21:37:39.525754+0000","last_undegraded":"2024-04-26T21:37:39.525754+0000","last_fullsized":"2024-04-26T21:37:39.525754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:11.778057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.148050+0000","last_change":"2024-04-26T21:37:38.163450+0000","last_active":"2024-04-26T21:37:39.148050+0000","last_peered":"2024-04-26T21:37:39.148050+0000","last_clean":"2024-04-26T21:37:39.148050+0000","last_became_active":"2024-04-26T21:37:38.163202+0000","last_became_peered":"2024-04-26T21:37:38.163202+0000","last_unstale":"2024-04-26T21:37:39.148050+0000","last_undegraded":"2024-04-26T21:37:39.148050+0000","last_fullsized":"2024-04-26T21:37:39.148050+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:12.977121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525359+0000","last_change":"2024-04-26T21:37:37.152397+0000","last_active":"2024-04-26T21:37:39.525359+0000","last_peered":"2024-04-26T21:37:39.525359+0000","last_clean":"2024-04-26T21:37:39.525359+0000","last_became_active":"2024-04-26T21:37:37.152118+0000","last_became_peered":"2024-04-26T21:37:37.152118+0000","last_unstale":"2024-04-26T21:37:39.525359+0000","last_undegraded":"2024-04-26T21:37:39.525359+0000","last_fullsized":"2024-04-26T21:37:39.525359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525725+0000","last_change":"2024-04-26T21:37:38.175519+0000","last_active":"2024-04-26T21:37:39.525725+0000","last_peered":"2024-04-26T21:37:39.525725+0000","last_clean":"2024-04-26T21:37:39.525725+0000","last_became_active":"2024-04-26T21:37:38.174521+0000","last_became_peered":"2024-04-26T21:37:38.174521+0000","last_unstale":"2024-04-26T21:37:39.525725+0000","last_undegraded":"2024-04-26T21:37:39.525725+0000","last_fullsized":"2024-04-26T21:37:39.525725+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:18.772784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524929+0000","last_change":"2024-04-26T21:37:37.155533+0000","last_active":"2024-04-26T21:37:39.524929+0000","last_peered":"2024-04-26T21:37:39.524929+0000","last_clean":"2024-04-26T21:37:39.524929+0000","last_became_active":"2024-04-26T21:37:37.155179+0000","last_became_peered":"2024-04-26T21:37:37.155179+0000","last_unstale":"2024-04-26T21:37:39.524929+0000","last_undegraded":"2024-04-26T21:37:39.524929+0000","last_fullsized":"2024-04-26T21:37:39.524929+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:28.173221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525989+0000","last_change":"2024-04-26T21:37:38.163951+0000","last_active":"2024-04-26T21:37:39.525989+0000","last_peered":"2024-04-26T21:37:39.525989+0000","last_clean":"2024-04-26T21:37:39.525989+0000","last_became_active":"2024-04-26T21:37:38.163655+0000","last_became_peered":"2024-04-26T21:37:38.163655+0000","last_unstale":"2024-04-26T21:37:39.525989+0000","last_undegraded":"2024-04-26T21:37:39.525989+0000","last_fullsized":"2024-04-26T21:37:39.525989+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:18.772784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525744+0000","last_change":"2024-04-26T21:37:37.148935+0000","last_active":"2024-04-26T21:37:39.525744+0000","last_peered":"2024-04-26T21:37:39.525744+0000","last_clean":"2024-04-26T21:37:39.525744+0000","last_became_active":"2024-04-26T21:37:37.148635+0000","last_became_peered":"2024-04-26T21:37:37.148635+0000","last_unstale":"2024-04-26T21:37:39.525744+0000","last_undegraded":"2024-04-26T21:37:39.525744+0000","last_fullsized":"2024-04-26T21:37:39.525744+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:54.356357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525564+0000","last_change":"2024-04-26T21:37:38.177643+0000","last_active":"2024-04-26T21:37:39.525564+0000","last_peered":"2024-04-26T21:37:39.525564+0000","last_clean":"2024-04-26T21:37:39.525564+0000","last_became_active":"2024-04-26T21:37:38.177380+0000","last_became_peered":"2024-04-26T21:37:38.177380+0000","last_unstale":"2024-04-26T21:37:39.525564+0000","last_undegraded":"2024-04-26T21:37:39.525564+0000","last_fullsized":"2024-04-26T21:37:39.525564+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:50.180281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":22,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525163+0000","last_change":"2024-04-26T21:37:37.147944+0000","last_active":"2024-04-26T21:37:39.525163+0000","last_peered":"2024-04-26T21:37:39.525163+0000","last_clean":"2024-04-26T21:37:39.525163+0000","last_became_active":"2024-04-26T21:37:37.147675+0000","last_became_peered":"2024-04-26T21:37:37.147675+0000","last_unstale":"2024-04-26T21:37:39.525163+0000","last_undegraded":"2024-04-26T21:37:39.525163+0000","last_fullsized":"2024-04-26T21:37:39.525163+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:04.435279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525060+0000","last_change":"2024-04-26T21:37:38.165335+0000","last_active":"2024-04-26T21:37:39.525060+0000","last_peered":"2024-04-26T21:37:39.525060+0000","last_clean":"2024-04-26T21:37:39.525060+0000","last_became_active":"2024-04-26T21:37:38.165061+0000","last_became_peered":"2024-04-26T21:37:38.165061+0000","last_unstale":"2024-04-26T21:37:39.525060+0000","last_undegraded":"2024-04-26T21:37:39.525060+0000","last_fullsized":"2024-04-26T21:37:39.525060+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:10.522313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525833+0000","last_change":"2024-04-26T21:37:37.153638+0000","last_active":"2024-04-26T21:37:39.525833+0000","last_peered":"2024-04-26T21:37:39.525833+0000","last_clean":"2024-04-26T21:37:39.525833+0000","last_became_active":"2024-04-26T21:37:37.153152+0000","last_became_peered":"2024-04-26T21:37:37.153152+0000","last_unstale":"2024-04-26T21:37:39.525833+0000","last_undegraded":"2024-04-26T21:37:39.525833+0000","last_fullsized":"2024-04-26T21:37:39.525833+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:06.306838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147689+0000","last_change":"2024-04-26T21:37:38.180510+0000","last_active":"2024-04-26T21:37:39.147689+0000","last_peered":"2024-04-26T21:37:39.147689+0000","last_clean":"2024-04-26T21:37:39.147689+0000","last_became_active":"2024-04-26T21:37:38.180216+0000","last_became_peered":"2024-04-26T21:37:38.180216+0000","last_unstale":"2024-04-26T21:37:39.147689+0000","last_undegraded":"2024-04-26T21:37:39.147689+0000","last_fullsized":"2024-04-26T21:37:39.147689+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:07.314863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147084+0000","last_change":"2024-04-26T21:37:37.155432+0000","last_active":"2024-04-26T21:37:39.147084+0000","last_peered":"2024-04-26T21:37:39.147084+0000","last_clean":"2024-04-26T21:37:39.147084+0000","last_became_active":"2024-04-26T21:37:37.155163+0000","last_became_peered":"2024-04-26T21:37:37.155163+0000","last_unstale":"2024-04-26T21:37:39.147084+0000","last_undegraded":"2024-04-26T21:37:39.147084+0000","last_fullsized":"2024-04-26T21:37:39.147084+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:59.412059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524999+0000","last_change":"2024-04-26T21:37:38.169437+0000","last_active":"2024-04-26T21:37:39.524999+0000","last_peered":"2024-04-26T21:37:39.524999+0000","last_clean":"2024-04-26T21:37:39.524999+0000","last_became_active":"2024-04-26T21:37:38.169172+0000","last_became_peered":"2024-04-26T21:37:38.169172+0000","last_unstale":"2024-04-26T21:37:39.524999+0000","last_undegraded":"2024-04-26T21:37:39.524999+0000","last_fullsized":"2024-04-26T21:37:39.524999+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:44.176333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524933+0000","last_change":"2024-04-26T21:37:37.155985+0000","last_active":"2024-04-26T21:37:39.524933+0000","last_peered":"2024-04-26T21:37:39.524933+0000","last_clean":"2024-04-26T21:37:39.524933+0000","last_became_active":"2024-04-26T21:37:37.155643+0000","last_became_peered":"2024-04-26T21:37:37.155643+0000","last_unstale":"2024-04-26T21:37:39.524933+0000","last_undegraded":"2024-04-26T21:37:39.524933+0000","last_fullsized":"2024-04-26T21:37:39.524933+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.123343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147805+0000","last_change":"2024-04-26T21:37:38.182475+0000","last_active":"2024-04-26T21:37:39.147805+0000","last_peered":"2024-04-26T21:37:39.147805+0000","last_clean":"2024-04-26T21:37:39.147805+0000","last_became_active":"2024-04-26T21:37:38.182179+0000","last_became_peered":"2024-04-26T21:37:38.182179+0000","last_unstale":"2024-04-26T21:37:39.147805+0000","last_undegraded":"2024-04-26T21:37:39.147805+0000","last_fullsized":"2024-04-26T21:37:39.147805+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:48.607945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525187+0000","last_change":"2024-04-26T21:37:37.150088+0000","last_active":"2024-04-26T21:37:39.525187+0000","last_peered":"2024-04-26T21:37:39.525187+0000","last_clean":"2024-04-26T21:37:39.525187+0000","last_became_active":"2024-04-26T21:37:37.149820+0000","last_became_peered":"2024-04-26T21:37:37.149820+0000","last_unstale":"2024-04-26T21:37:39.525187+0000","last_undegraded":"2024-04-26T21:37:39.525187+0000","last_fullsized":"2024-04-26T21:37:39.525187+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:47.692897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147901+0000","last_change":"2024-04-26T21:37:38.161803+0000","last_active":"2024-04-26T21:37:39.147901+0000","last_peered":"2024-04-26T21:37:39.147901+0000","last_clean":"2024-04-26T21:37:39.147901+0000","last_became_active":"2024-04-26T21:37:38.161541+0000","last_became_peered":"2024-04-26T21:37:38.161541+0000","last_unstale":"2024-04-26T21:37:39.147901+0000","last_undegraded":"2024-04-26T21:37:39.147901+0000","last_fullsized":"2024-04-26T21:37:39.147901+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:34.453645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147820+0000","last_change":"2024-04-26T21:37:37.156465+0000","last_active":"2024-04-26T21:37:39.147820+0000","last_peered":"2024-04-26T21:37:39.147820+0000","last_clean":"2024-04-26T21:37:39.147820+0000","last_became_active":"2024-04-26T21:37:37.156194+0000","last_became_peered":"2024-04-26T21:37:37.156194+0000","last_unstale":"2024-04-26T21:37:39.147820+0000","last_undegraded":"2024-04-26T21:37:39.147820+0000","last_fullsized":"2024-04-26T21:37:39.147820+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:47.692897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524845+0000","last_change":"2024-04-26T21:37:38.166953+0000","last_active":"2024-04-26T21:37:39.524845+0000","last_peered":"2024-04-26T21:37:39.524845+0000","last_clean":"2024-04-26T21:37:39.524845+0000","last_became_active":"2024-04-26T21:37:38.166218+0000","last_became_peered":"2024-04-26T21:37:38.166218+0000","last_unstale":"2024-04-26T21:37:39.524845+0000","last_undegraded":"2024-04-26T21:37:39.524845+0000","last_fullsized":"2024-04-26T21:37:39.524845+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.853081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524987+0000","last_change":"2024-04-26T21:37:37.143737+0000","last_active":"2024-04-26T21:37:39.524987+0000","last_peered":"2024-04-26T21:37:39.524987+0000","last_clean":"2024-04-26T21:37:39.524987+0000","last_became_active":"2024-04-26T21:37:37.143432+0000","last_became_peered":"2024-04-26T21:37:37.143432+0000","last_unstale":"2024-04-26T21:37:39.524987+0000","last_undegraded":"2024-04-26T21:37:39.524987+0000","last_fullsized":"2024-04-26T21:37:39.524987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:51.858944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525528+0000","last_change":"2024-04-26T21:37:38.169909+0000","last_active":"2024-04-26T21:37:39.525528+0000","last_peered":"2024-04-26T21:37:39.525528+0000","last_clean":"2024-04-26T21:37:39.525528+0000","last_became_active":"2024-04-26T21:37:38.169593+0000","last_became_peered":"2024-04-26T21:37:38.169593+0000","last_unstale":"2024-04-26T21:37:39.525528+0000","last_undegraded":"2024-04-26T21:37:39.525528+0000","last_fullsized":"2024-04-26T21:37:39.525528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:42.473250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525252+0000","last_change":"2024-04-26T21:37:37.156130+0000","last_active":"2024-04-26T21:37:39.525252+0000","last_peered":"2024-04-26T21:37:39.525252+0000","last_clean":"2024-04-26T21:37:39.525252+0000","last_became_active":"2024-04-26T21:37:37.154520+0000","last_became_peered":"2024-04-26T21:37:37.154520+0000","last_unstale":"2024-04-26T21:37:39.525252+0000","last_undegraded":"2024-04-26T21:37:39.525252+0000","last_fullsized":"2024-04-26T21:37:39.525252+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:48.571013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.148119+0000","last_change":"2024-04-26T21:37:38.170860+0000","last_active":"2024-04-26T21:37:39.148119+0000","last_peered":"2024-04-26T21:37:39.148119+0000","last_clean":"2024-04-26T21:37:39.148119+0000","last_became_active":"2024-04-26T21:37:38.170600+0000","last_became_peered":"2024-04-26T21:37:38.170600+0000","last_unstale":"2024-04-26T21:37:39.148119+0000","last_undegraded":"2024-04-26T21:37:39.148119+0000","last_fullsized":"2024-04-26T21:37:39.148119+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:12.786082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.146805+0000","last_change":"2024-04-26T21:37:37.145621+0000","last_active":"2024-04-26T21:37:39.146805+0000","last_peered":"2024-04-26T21:37:39.146805+0000","last_clean":"2024-04-26T21:37:39.146805+0000","last_became_active":"2024-04-26T21:37:37.145215+0000","last_became_peered":"2024-04-26T21:37:37.145215+0000","last_unstale":"2024-04-26T21:37:39.146805+0000","last_undegraded":"2024-04-26T21:37:39.146805+0000","last_fullsized":"2024-04-26T21:37:39.146805+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:07.815690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147857+0000","last_change":"2024-04-26T21:37:38.169448+0000","last_active":"2024-04-26T21:37:39.147857+0000","last_peered":"2024-04-26T21:37:39.147857+0000","last_clean":"2024-04-26T21:37:39.147857+0000","last_became_active":"2024-04-26T21:37:38.169023+0000","last_became_peered":"2024-04-26T21:37:38.169023+0000","last_unstale":"2024-04-26T21:37:39.147857+0000","last_undegraded":"2024-04-26T21:37:39.147857+0000","last_fullsized":"2024-04-26T21:37:39.147857+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:37.279900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525144+0000","last_change":"2024-04-26T21:37:37.155721+0000","last_active":"2024-04-26T21:37:39.525144+0000","last_peered":"2024-04-26T21:37:39.525144+0000","last_clean":"2024-04-26T21:37:39.525144+0000","last_became_active":"2024-04-26T21:37:37.155438+0000","last_became_peered":"2024-04-26T21:37:37.155438+0000","last_unstale":"2024-04-26T21:37:39.525144+0000","last_undegraded":"2024-04-26T21:37:39.525144+0000","last_fullsized":"2024-04-26T21:37:39.525144+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:11.817532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147816+0000","last_change":"2024-04-26T21:37:38.183271+0000","last_active":"2024-04-26T21:37:39.147816+0000","last_peered":"2024-04-26T21:37:39.147816+0000","last_clean":"2024-04-26T21:37:39.147816+0000","last_became_active":"2024-04-26T21:37:38.182966+0000","last_became_peered":"2024-04-26T21:37:38.182966+0000","last_unstale":"2024-04-26T21:37:39.147816+0000","last_undegraded":"2024-04-26T21:37:39.147816+0000","last_fullsized":"2024-04-26T21:37:39.147816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:05.443303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.526186+0000","last_change":"2024-04-26T21:37:37.151412+0000","last_active":"2024-04-26T21:37:39.526186+0000","last_peered":"2024-04-26T21:37:39.526186+0000","last_clean":"2024-04-26T21:37:39.526186+0000","last_became_active":"2024-04-26T21:37:37.151157+0000","last_became_peered":"2024-04-26T21:37:37.151157+0000","last_unstale":"2024-04-26T21:37:39.526186+0000","last_undegraded":"2024-04-26T21:37:39.526186+0000","last_fullsized":"2024-04-26T21:37:39.526186+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:48.659187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525158+0000","last_change":"2024-04-26T21:37:38.164589+0000","last_active":"2024-04-26T21:37:39.525158+0000","last_peered":"2024-04-26T21:37:39.525158+0000","last_clean":"2024-04-26T21:37:39.525158+0000","last_became_active":"2024-04-26T21:37:38.164300+0000","last_became_peered":"2024-04-26T21:37:38.164300+0000","last_unstale":"2024-04-26T21:37:39.525158+0000","last_undegraded":"2024-04-26T21:37:39.525158+0000","last_fullsized":"2024-04-26T21:37:39.525158+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:42.473250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524938+0000","last_change":"2024-04-26T21:37:37.154574+0000","last_active":"2024-04-26T21:37:39.524938+0000","last_peered":"2024-04-26T21:37:39.524938+0000","last_clean":"2024-04-26T21:37:39.524938+0000","last_became_active":"2024-04-26T21:37:37.153518+0000","last_became_peered":"2024-04-26T21:37:37.153518+0000","last_unstale":"2024-04-26T21:37:39.524938+0000","last_undegraded":"2024-04-26T21:37:39.524938+0000","last_fullsized":"2024-04-26T21:37:39.524938+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:31.920081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525327+0000","last_change":"2024-04-26T21:37:38.172828+0000","last_active":"2024-04-26T21:37:39.525327+0000","last_peered":"2024-04-26T21:37:39.525327+0000","last_clean":"2024-04-26T21:37:39.525327+0000","last_became_active":"2024-04-26T21:37:38.172544+0000","last_became_peered":"2024-04-26T21:37:38.172544+0000","last_unstale":"2024-04-26T21:37:39.525327+0000","last_undegraded":"2024-04-26T21:37:39.525327+0000","last_fullsized":"2024-04-26T21:37:39.525327+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:56.069223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525190+0000","last_change":"2024-04-26T21:37:37.144706+0000","last_active":"2024-04-26T21:37:39.525190+0000","last_peered":"2024-04-26T21:37:39.525190+0000","last_clean":"2024-04-26T21:37:39.525190+0000","last_became_active":"2024-04-26T21:37:37.144369+0000","last_became_peered":"2024-04-26T21:37:37.144369+0000","last_unstale":"2024-04-26T21:37:39.525190+0000","last_undegraded":"2024-04-26T21:37:39.525190+0000","last_fullsized":"2024-04-26T21:37:39.525190+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:19.156957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.148138+0000","last_change":"2024-04-26T21:37:38.162848+0000","last_active":"2024-04-26T21:37:39.148138+0000","last_peered":"2024-04-26T21:37:39.148138+0000","last_clean":"2024-04-26T21:37:39.148138+0000","last_became_active":"2024-04-26T21:37:38.162583+0000","last_became_peered":"2024-04-26T21:37:38.162583+0000","last_unstale":"2024-04-26T21:37:39.148138+0000","last_undegraded":"2024-04-26T21:37:39.148138+0000","last_fullsized":"2024-04-26T21:37:39.148138+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:49.021226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525082+0000","last_change":"2024-04-26T21:37:37.153455+0000","last_active":"2024-04-26T21:37:39.525082+0000","last_peered":"2024-04-26T21:37:39.525082+0000","last_clean":"2024-04-26T21:37:39.525082+0000","last_became_active":"2024-04-26T21:37:37.153181+0000","last_became_peered":"2024-04-26T21:37:37.153181+0000","last_unstale":"2024-04-26T21:37:39.525082+0000","last_undegraded":"2024-04-26T21:37:39.525082+0000","last_fullsized":"2024-04-26T21:37:39.525082+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:44:10.891667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524906+0000","last_change":"2024-04-26T21:37:38.174487+0000","last_active":"2024-04-26T21:37:39.524906+0000","last_peered":"2024-04-26T21:37:39.524906+0000","last_clean":"2024-04-26T21:37:39.524906+0000","last_became_active":"2024-04-26T21:37:38.173873+0000","last_became_peered":"2024-04-26T21:37:38.173873+0000","last_unstale":"2024-04-26T21:37:39.524906+0000","last_undegraded":"2024-04-26T21:37:39.524906+0000","last_fullsized":"2024-04-26T21:37:39.524906+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:01.032700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524810+0000","last_change":"2024-04-26T21:37:37.144882+0000","last_active":"2024-04-26T21:37:39.524810+0000","last_peered":"2024-04-26T21:37:39.524810+0000","last_clean":"2024-04-26T21:37:39.524810+0000","last_became_active":"2024-04-26T21:37:37.144633+0000","last_became_peered":"2024-04-26T21:37:37.144633+0000","last_unstale":"2024-04-26T21:37:39.524810+0000","last_undegraded":"2024-04-26T21:37:39.524810+0000","last_fullsized":"2024-04-26T21:37:39.524810+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:15.680185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525825+0000","last_change":"2024-04-26T21:37:38.158117+0000","last_active":"2024-04-26T21:37:39.525825+0000","last_peered":"2024-04-26T21:37:39.525825+0000","last_clean":"2024-04-26T21:37:39.525825+0000","last_became_active":"2024-04-26T21:37:38.157800+0000","last_became_peered":"2024-04-26T21:37:38.157800+0000","last_unstale":"2024-04-26T21:37:39.525825+0000","last_undegraded":"2024-04-26T21:37:39.525825+0000","last_fullsized":"2024-04-26T21:37:39.525825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:10.670811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.147971+0000","last_change":"2024-04-26T21:37:37.150695+0000","last_active":"2024-04-26T21:37:39.147971+0000","last_peered":"2024-04-26T21:37:39.147971+0000","last_clean":"2024-04-26T21:37:39.147971+0000","last_became_active":"2024-04-26T21:37:37.150398+0000","last_became_peered":"2024-04-26T21:37:37.150398+0000","last_unstale":"2024-04-26T21:37:39.147971+0000","last_undegraded":"2024-04-26T21:37:39.147971+0000","last_fullsized":"2024-04-26T21:37:39.147971+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:04.435279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.524665+0000","last_change":"2024-04-26T21:37:38.165076+0000","last_active":"2024-04-26T21:37:39.524665+0000","last_peered":"2024-04-26T21:37:39.524665+0000","last_clean":"2024-04-26T21:37:39.524665+0000","last_became_active":"2024-04-26T21:37:38.162761+0000","last_became_peered":"2024-04-26T21:37:38.162761+0000","last_unstale":"2024-04-26T21:37:39.524665+0000","last_undegraded":"2024-04-26T21:37:39.524665+0000","last_fullsized":"2024-04-26T21:37:39.524665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:37.128336+0000","last_clean_scrub_stamp":"2024-04-26T21:37:37.128336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:49.579038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"52'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525122+0000","last_change":"2024-04-26T21:37:37.156817+0000","last_active":"2024-04-26T21:37:39.525122+0000","last_peered":"2024-04-26T21:37:39.525122+0000","last_clean":"2024-04-26T21:37:39.525122+0000","last_became_active":"2024-04-26T21:37:37.156561+0000","last_became_peered":"2024-04-26T21:37:37.156561+0000","last_unstale":"2024-04-26T21:37:39.525122+0000","last_undegraded":"2024-04-26T21:37:39.525122+0000","last_fullsized":"2024-04-26T21:37:39.525122+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:37:36.120312+0000","last_clean_scrub_stamp":"2024-04-26T21:37:36.120312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:04.317873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":230,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T21:37:39.525479+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:37:39.525479+0000","last_peered":"2024-04-26T21:37:39.525479+0000","last_clean":"2024-04-26T21:37:39.525479+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:37:39.525479+0000","last_undegraded":"2024-04-26T21:37:39.525479+0000","last_fullsized":"2024-04-26T21:37:39.525479+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":867871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":278031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":618496,"data_stored":277379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":277961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":279013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":278991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":277539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":867803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:37:46.962 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:37:46.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:37:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.107+0000 7f7d27ad1640 1 Processor -- start 2024-04-26T21:37:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.107+0000 7f7d27ad1640 1 -- start start 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d27ad1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2015f550 0x7f7d200d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d27ad1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d200d3bc0 0x7f7d200d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d27ad1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d200d1040 con 0x7f7d2015f170 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d27ad1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7d200c9430 con 0x7f7d200d3bc0 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d27ad1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d200c9680 con 0x7f7d2015f550 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d2606e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d200d3bc0 0x7f7d200d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d2586d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7d2015f170 0x7f7d200d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60542/0 (socket says 172.21.15.40:60542) 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d2586d640 1 -- 172.21.15.40:0/2586441519 learned_addr learned my addr 172.21.15.40:0/2586441519 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.108+0000 7f7d2506c640 1 --2- 172.21.15.40:0/2586441519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2015f550 0x7f7d200d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.109+0000 7f7d2606e640 1 -- 172.21.15.40:0/2586441519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2015f550 msgr2=0x7f7d200d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.109+0000 7f7d2606e640 1 --2- 172.21.15.40:0/2586441519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2015f550 0x7f7d200d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.109+0000 7f7d2606e640 1 -- 172.21.15.40:0/2586441519 >> v1:172.21.15.40:6789/0 conn(0x7f7d2015f170 legacy=0x7f7d200d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.109+0000 7f7d2606e640 1 -- 172.21.15.40:0/2586441519 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d200c9960 con 0x7f7d200d3bc0 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.109+0000 7f7d2606e640 1 --2- 172.21.15.40:0/2586441519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d200d3bc0 0x7f7d200d2cb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009d90 tx=0x7f7d1002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d34e1b840e0722 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.109+0000 7f7d2486b640 1 -- 172.21.15.40:0/2586441519 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10007740 con 0x7f7d200d3bc0 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.109+0000 7f7d2486b640 1 -- 172.21.15.40:0/2586441519 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d100078e0 con 0x7f7d200d3bc0 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.109+0000 7f7d2486b640 1 -- 172.21.15.40:0/2586441519 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10007cb0 con 0x7f7d200d3bc0 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.110+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2586441519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d200d3bc0 msgr2=0x7f7d200d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.110+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2586441519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d200d3bc0 0x7f7d200d2cb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009d90 tx=0x7f7d1002a660 comp rx=0 tx=0).stop 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.110+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2586441519 shutdown_connections 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.110+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2586441519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d200d3bc0 0x7f7d200d2cb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.110+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2586441519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2015f550 0x7f7d200d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.110+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2586441519 >> 172.21.15.40:0/2586441519 conn(0x7f7d200d4f60 msgr2=0x7f7d200d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.110+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2586441519 shutdown_connections 2024-04-26T21:37:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.110+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2586441519 wait complete. 2024-04-26T21:37:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.111+0000 7f7d27ad1640 1 Processor -- start 2024-04-26T21:37:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.111+0000 7f7d27ad1640 1 -- start start 2024-04-26T21:37:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d27ad1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d200d3bc0 0x7f7d200cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d27ad1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d2015f170 0x7f7d201522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d27ad1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d201597a0 0x7f7d2015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d27ad1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d200ca5c0 con 0x7f7d2015f170 2024-04-26T21:37:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d27ad1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d200ca440 con 0x7f7d200d3bc0 2024-04-26T21:37:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d27ad1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d200ca740 con 0x7f7d201597a0 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d2506c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d2015f170 0x7f7d201522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d2506c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d2015f170 0x7f7d201522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57160/0 (socket says 172.21.15.40:57160) 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d2506c640 1 -- 172.21.15.40:0/2749176078 learned_addr learned my addr 172.21.15.40:0/2749176078 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d2586d640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d200d3bc0 0x7f7d200cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.112+0000 7f7d2606e640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d201597a0 0x7f7d2015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d2506c640 1 -- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d201597a0 msgr2=0x7f7d2015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d2506c640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d201597a0 0x7f7d2015a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d2506c640 1 -- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d200d3bc0 msgr2=0x7f7d200cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d2506c640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d200d3bc0 0x7f7d200cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d2506c640 1 -- 172.21.15.40:0/2749176078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d200cc410 con 0x7f7d2015f170 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d2586d640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d200d3bc0 0x7f7d200cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d2506c640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d2015f170 0x7f7d201522a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400d5f0 tx=0x7f7d1400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d1e7fc640 1 -- 172.21.15.40:0/2749176078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d140125f0 con 0x7f7d2015f170 2024-04-26T21:37:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d1e7fc640 1 -- 172.21.15.40:0/2749176078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d14019070 con 0x7f7d2015f170 2024-04-26T21:37:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d1e7fc640 1 -- 172.21.15.40:0/2749176078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d140129d0 con 0x7f7d2015f170 2024-04-26T21:37:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d200cc6f0 con 0x7f7d2015f170 2024-04-26T21:37:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.113+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d200cccb0 con 0x7f7d2015f170 2024-04-26T21:37:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.114+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d200694f0 con 0x7f7d2015f170 2024-04-26T21:37:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.114+0000 7f7d1e7fc640 1 -- 172.21.15.40:0/2749176078 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d14012b70 con 0x7f7d2015f170 2024-04-26T21:37:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.115+0000 7f7d1e7fc640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7d0c06c8f0 0x7f7d0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.115+0000 7f7d1e7fc640 1 -- 172.21.15.40:0/2749176078 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f7d1408c050 con 0x7f7d2015f170 2024-04-26T21:37:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.115+0000 7f7d2586d640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7d0c06c8f0 0x7f7d0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.116+0000 7f7d2586d640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7d0c06c8f0 0x7f7d0c06edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000f60 tx=0x7f7d18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.119+0000 7f7d1e7fc640 1 -- 172.21.15.40:0/2749176078 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d1405a380 con 0x7f7d2015f170 2024-04-26T21:37:47.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.375+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7d200b6e40 con 0x7f7d2015f170 2024-04-26T21:37:47.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.378+0000 7f7d1e7fc640 1 -- 172.21.15.40:0/2749176078 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f7d14059c80 con 0x7f7d2015f170 2024-04-26T21:37:47.378 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:37:47.378 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw== 2024-04-26T21:37:47.378 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:37:47.378 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:37:47.378 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:37:47.379 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:37:47.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7d0c06c8f0 msgr2=0x7f7d0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7d0c06c8f0 0x7f7d0c06edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000f60 tx=0x7f7d18009040 comp rx=0 tx=0).stop 2024-04-26T21:37:47.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d2015f170 msgr2=0x7f7d201522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d2015f170 0x7f7d201522a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400d5f0 tx=0x7f7d1400dab0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 shutdown_connections 2024-04-26T21:37:47.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7d0c06c8f0 0x7f7d0c06edb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d201597a0 0x7f7d2015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d2015f170 0x7f7d201522a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 --2- 172.21.15.40:0/2749176078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d200d3bc0 0x7f7d200cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 >> 172.21.15.40:0/2749176078 conn(0x7f7d200d4f60 msgr2=0x7f7d2006e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:47.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 shutdown_connections 2024-04-26T21:37:47.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.380+0000 7f7d27ad1640 1 -- 172.21.15.40:0/2749176078 wait complete. 2024-04-26T21:37:47.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:47.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.492+0000 7f944563f640 1 Processor -- start 2024-04-26T21:37:47.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.492+0000 7f944563f640 1 -- start start 2024-04-26T21:37:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f944563f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94400cc480 0x7f9440072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f944563f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440072780 0x7f9440166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f944563f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944015d7c0 con 0x7f9440071e60 2024-04-26T21:37:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f944563f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f944015daa0 con 0x7f94400cc480 2024-04-26T21:37:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f944563f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f944015dd80 con 0x7f9440072780 2024-04-26T21:37:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f943e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94400cc480 0x7f9440072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f943e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94400cc480 0x7f9440072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57168/0 (socket says 172.21.15.40:57168) 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f943e7fc640 1 -- 172.21.15.40:0/3137894045 learned_addr learned my addr 172.21.15.40:0/3137894045 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f943f7fe640 1 --2- 172.21.15.40:0/3137894045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440072780 0x7f9440166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f943e7fc640 1 -- 172.21.15.40:0/3137894045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440072780 msgr2=0x7f9440166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f943e7fc640 1 --2- 172.21.15.40:0/3137894045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440072780 0x7f9440166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f943e7fc640 1 -- 172.21.15.40:0/3137894045 >> v1:172.21.15.184:6789/0 conn(0x7f9440071e60 legacy=0x7f944006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.493+0000 7f943e7fc640 1 -- 172.21.15.40:0/3137894045 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944015e060 con 0x7f94400cc480 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.494+0000 7f943e7fc640 1 --2- 172.21.15.40:0/3137894045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f9440072240 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9434009870 tx=0x7f943402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6ef744d49d92f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.494+0000 7f943dffb640 1 -- 172.21.15.40:0/3137894045 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434037070 con 0x7f94400cc480 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.494+0000 7f943dffb640 1 -- 172.21.15.40:0/3137894045 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9434032440 con 0x7f94400cc480 2024-04-26T21:37:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.494+0000 7f943dffb640 1 -- 172.21.15.40:0/3137894045 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434032810 con 0x7f94400cc480 2024-04-26T21:37:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.495+0000 7f944563f640 1 -- 172.21.15.40:0/3137894045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 msgr2=0x7f9440072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.495+0000 7f944563f640 1 --2- 172.21.15.40:0/3137894045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f9440072240 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9434009870 tx=0x7f943402a270 comp rx=0 tx=0).stop 2024-04-26T21:37:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.495+0000 7f944563f640 1 -- 172.21.15.40:0/3137894045 shutdown_connections 2024-04-26T21:37:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.495+0000 7f944563f640 1 --2- 172.21.15.40:0/3137894045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440072780 0x7f9440166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.495+0000 7f944563f640 1 --2- 172.21.15.40:0/3137894045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f9440072240 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.495+0000 7f944563f640 1 -- 172.21.15.40:0/3137894045 >> 172.21.15.40:0/3137894045 conn(0x7f94400d0be0 msgr2=0x7f94400d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.495+0000 7f944563f640 1 -- 172.21.15.40:0/3137894045 shutdown_connections 2024-04-26T21:37:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.495+0000 7f944563f640 1 -- 172.21.15.40:0/3137894045 wait complete. 2024-04-26T21:37:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.496+0000 7f944563f640 1 Processor -- start 2024-04-26T21:37:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.496+0000 7f944563f640 1 -- start start 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f944563f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440071e60 0x7f94401f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f944563f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9440072780 0x7f94401f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f944563f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f944015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f944563f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f944015d3f0 con 0x7f94400cc480 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f944563f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9440072c10 con 0x7f9440072780 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f944563f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f944015d570 con 0x7f9440071e60 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f944015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f944015ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57174/0 (socket says 172.21.15.40:57174) 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943f7fe640 1 -- 172.21.15.40:0/4117534355 learned_addr learned my addr 172.21.15.40:0/4117534355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943e7fc640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9440072780 0x7f94401f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943effd640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440071e60 0x7f94401f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943f7fe640 1 -- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440071e60 msgr2=0x7f94401f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943f7fe640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440071e60 0x7f94401f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943f7fe640 1 -- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9440072780 msgr2=0x7f94401f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943f7fe640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9440072780 0x7f94401f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943f7fe640 1 -- 172.21.15.40:0/4117534355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94400af870 con 0x7f94400cc480 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943e7fc640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9440072780 0x7f94401f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.497+0000 7f943effd640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440071e60 0x7f94401f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.498+0000 7f943f7fe640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f944015ba30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9438007e40 tx=0x7f943800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.498+0000 7f94277fe640 1 -- 172.21.15.40:0/4117534355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438014620 con 0x7f94400cc480 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.498+0000 7f94277fe640 1 -- 172.21.15.40:0/4117534355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9438019070 con 0x7f94400cc480 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.498+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94400ce7b0 con 0x7f94400cc480 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.498+0000 7f94277fe640 1 -- 172.21.15.40:0/4117534355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438014a20 con 0x7f94400cc480 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.498+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94400ced70 con 0x7f94400cc480 2024-04-26T21:37:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.499+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94400694f0 con 0x7f94400cc480 2024-04-26T21:37:47.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.499+0000 7f94277fe640 1 -- 172.21.15.40:0/4117534355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9438014bc0 con 0x7f94400cc480 2024-04-26T21:37:47.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.500+0000 7f94277fe640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:47.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.500+0000 7f94277fe640 1 -- 172.21.15.40:0/4117534355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f943808c5f0 con 0x7f94400cc480 2024-04-26T21:37:47.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.500+0000 7f943effd640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.500+0000 7f943effd640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f942c06c8f0 0x7f942c06edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f944015a500 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.504+0000 7f94277fe640 1 -- 172.21.15.40:0/4117534355 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f943805a890 con 0x7f94400cc480 2024-04-26T21:37:47.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.730+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9440069e20 con 0x7f94400cc480 2024-04-26T21:37:47.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.730+0000 7f94277fe640 1 -- 172.21.15.40:0/4117534355 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5181 (secure 0 0 0) 0x7f943805a190 con 0x7f94400cc480 2024-04-26T21:37:47.731 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:47.731 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:37:40.533180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2276808574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2276808574},{"type":"v1","addr":"172.21.15.40:6834","nonce":2276808574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6015,"qdb_cluster":[6015]},"id":3}]} 2024-04-26T21:37:47.731 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 32 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f942c06c8f0 msgr2=0x7f942c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f942c06c8f0 0x7f942c06edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f944015a500 tx=0x7f9430009040 comp rx=0 tx=0).stop 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 msgr2=0x7f944015ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f944015ba30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9438007e40 tx=0x7f943800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 shutdown_connections 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f942c06c8f0 0x7f942c06edb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94400cc480 0x7f944015ba30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9440072780 0x7f94401f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 --2- 172.21.15.40:0/4117534355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9440071e60 0x7f94401f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.732+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 >> 172.21.15.40:0/4117534355 conn(0x7f94400d0be0 msgr2=0x7f94400b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.733+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 shutdown_connections 2024-04-26T21:37:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:47.733+0000 7f944563f640 1 -- 172.21.15.40:0/4117534355 wait complete. 2024-04-26T21:37:48.146 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:37:37.510449+0000', 'modified': '2024-04-26T21:37:40.533180+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6015}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6015': {'gid': 6015, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6834/2276808574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 2276808574}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 2276808574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6015, 'qdb_cluster': [6015]} 2024-04-26T21:37:48.146 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:37:48.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:37:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.252+0000 7fb6543bd640 1 Processor -- start 2024-04-26T21:37:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.252+0000 7fb6543bd640 1 -- start start 2024-04-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb6543bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb64c0b2340 0x7fb64c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb6543bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64c0d10d0 con 0x7fb64c162810 2024-04-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb6543bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64c0d1370 con 0x7fb64c0cd930 2024-04-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb6543bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb64c0d0750 con 0x7fb64c0b2340 2024-04-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb652159640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb64c162810 0x7fb64c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60558/0 (socket says 172.21.15.40:60558) 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb652159640 1 -- 172.21.15.40:0/3460784124 learned_addr learned my addr 172.21.15.40:0/3460784124 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4262933096 0 0) 0x7fb64c0d10d0 con 0x7fb64c162810 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb638003520 con 0x7fb64c162810 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb651958640 1 --2- 172.21.15.40:0/3460784124 >> v2:172.21.15.184:3300/0 conn(0x7fb64c0b2340 0x7fb64c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.253+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 656951572 0 0) 0x7fb638003520 con 0x7fb64c162810 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 >> v2:172.21.15.184:3300/0 conn(0x7fb64c0b2340 msgr2=0x7fb64c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 --2- 172.21.15.40:0/3460784124 >> v2:172.21.15.184:3300/0 conn(0x7fb64c0b2340 0x7fb64c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 >> v1:172.21.15.184:6789/0 conn(0x7fb64c0cd930 legacy=0x7fb64c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c0d01b0 con 0x7fb64c162810 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb648001670 con 0x7fb64c162810 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 >> v1:172.21.15.40:6789/0 conn(0x7fb64c162810 legacy=0x7fb64c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380039b0 0x7fb638003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb638004800 0x7fb638004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb638005210 0x7fb63800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c0d0750 con 0x7fb6380039b0 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb63800c140 con 0x7fb638005210 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb63800c340 con 0x7fb638004800 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb652159640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380039b0 0x7fb638003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb651958640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb638004800 0x7fb638004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb65295a640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb638005210 0x7fb63800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb652159640 1 -- 172.21.15.40:0/3460784124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb638004800 msgr2=0x7fb638004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb652159640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb638004800 0x7fb638004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb652159640 1 -- 172.21.15.40:0/3460784124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb638005210 msgr2=0x7fb63800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb652159640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb638005210 0x7fb63800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb652159640 1 -- 172.21.15.40:0/3460784124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb638003520 con 0x7fb6380039b0 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.254+0000 7fb65295a640 1 -- 172.21.15.40:0/3460784124 reap_dead start 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.255+0000 7fb652159640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380039b0 0x7fb638003df0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb64800c240 tx=0x7fb638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b385a6054899936d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.255+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648007d10 con 0x7fb6380039b0 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.255+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb648005ad0 con 0x7fb6380039b0 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.255+0000 7fb651157640 1 -- 172.21.15.40:0/3460784124 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648007070 con 0x7fb6380039b0 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.255+0000 7fb6543bd640 1 -- 172.21.15.40:0/3460784124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380039b0 msgr2=0x7fb638003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.255+0000 7fb6543bd640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380039b0 0x7fb638003df0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb64800c240 tx=0x7fb638005830 comp rx=0 tx=0).stop 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.256+0000 7fb6543bd640 1 -- 172.21.15.40:0/3460784124 shutdown_connections 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.256+0000 7fb6543bd640 1 --2- 172.21.15.40:0/3460784124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380039b0 0x7fb638003df0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.256+0000 7fb6543bd640 1 -- 172.21.15.40:0/3460784124 >> 172.21.15.40:0/3460784124 conn(0x7fb64c0d2da0 msgr2=0x7fb64c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.256+0000 7fb6543bd640 1 -- 172.21.15.40:0/3460784124 shutdown_connections 2024-04-26T21:37:48.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.256+0000 7fb6543bd640 1 -- 172.21.15.40:0/3460784124 wait complete. 2024-04-26T21:37:48.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.257+0000 7fb6543bd640 1 Processor -- start 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.257+0000 7fb6543bd640 1 -- start start 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb6543bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6380039b0 0x7fb64c156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb6543bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c0ca560 0x7fb64c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb6543bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb64c0ca9e0 0x7fb64c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb6543bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c0cd3f0 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb6543bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c0d16a0 con 0x7fb64c0ca560 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb6543bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb64c0cd570 con 0x7fb6380039b0 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb64c0ca9e0 0x7fb64c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb64c0ca9e0 0x7fb64c0cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57196/0 (socket says 172.21.15.40:57196) 2024-04-26T21:37:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 -- 172.21.15.40:0/37867119 learned_addr learned my addr 172.21.15.40:0/37867119 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb652159640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6380039b0 0x7fb64c156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb651958640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c0ca560 0x7fb64c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 -- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6380039b0 msgr2=0x7fb64c156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6380039b0 0x7fb64c156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 -- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c0ca560 msgr2=0x7fb64c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c0ca560 0x7fb64c0cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 -- 172.21.15.40:0/37867119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c0b04c0 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb651958640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c0ca560 0x7fb64c0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb65295a640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb64c0ca9e0 0x7fb64c0cc7b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0cf3e0 tx=0x7fb64400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb642ffd640 1 -- 172.21.15.40:0/37867119 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64400fe90 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb642ffd640 1 -- 172.21.15.40:0/37867119 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb644018070 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb642ffd640 1 -- 172.21.15.40:0/37867119 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644003da0 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb6543bd640 1 -- 172.21.15.40:0/37867119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c0b07a0 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.258+0000 7fb6543bd640 1 -- 172.21.15.40:0/37867119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c0b0d60 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.260+0000 7fb642ffd640 1 -- 172.21.15.40:0/37867119 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6440050d0 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.260+0000 7fb642ffd640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb63006ca90 0x7fb63006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.260+0000 7fb642ffd640 1 -- 172.21.15.40:0/37867119 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fb64408b9b0 con 0x7fb64c0ca9e0 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.261+0000 7fb652159640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb63006ca90 0x7fb63006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.261+0000 7fb6571b0740 1 Processor -- start 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.261+0000 7fb652159640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb63006ca90 0x7fb63006ef50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb648033000 tx=0x7fb64802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.261+0000 7fb6571b0740 1 -- start start 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 0x557d32948820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32965ce0 0x557d32966230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3291e120 0x557d329643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb6571b0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557d32882740 con 0x557d3291e120 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb6571b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557d3281e3a0 con 0x557d32965ce0 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb6571b0740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557d3287e700 con 0x557d32948440 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3291e120 0x557d329643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3291e120 0x557d329643a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57210/0 (socket says 172.21.15.40:57210) 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb65295a640 1 -- 172.21.15.40:0/1468014510 learned_addr learned my addr 172.21.15.40:0/1468014510 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb652159640 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 0x557d32948820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb651958640 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32965ce0 0x557d32966230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb65295a640 1 -- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 msgr2=0x557d32948820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.262+0000 7fb65295a640 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 0x557d32948820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.263+0000 7fb65295a640 1 -- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32965ce0 msgr2=0x557d32966230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.263+0000 7fb65295a640 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32965ce0 0x557d32966230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.263+0000 7fb65295a640 1 -- 172.21.15.40:0/1468014510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3283a750 con 0x557d3291e120 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.263+0000 7fb651958640 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32965ce0 0x557d32966230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.263+0000 7fb65295a640 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3291e120 0x557d329643a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb644061c90 tx=0x7fb644059ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28290ed89e7b161 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.263+0000 7fb62ffff640 1 -- 172.21.15.40:0/1468014510 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644088070 con 0x557d3291e120 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.263+0000 7fb62ffff640 1 -- 172.21.15.40:0/1468014510 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64405d990 con 0x557d3291e120 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.263+0000 7fb62ffff640 1 -- 172.21.15.40:0/1468014510 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64408b070 con 0x557d3291e120 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 -- 172.21.15.40:0/1468014510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3291e120 msgr2=0x557d329643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3291e120 0x557d329643a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb644061c90 tx=0x7fb644059ce0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 -- 172.21.15.40:0/1468014510 shutdown_connections 2024-04-26T21:37:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3291e120 0x557d329643a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32965ce0 0x557d32966230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1468014510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 0x557d32948820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 -- 172.21.15.40:0/1468014510 >> 172.21.15.40:0/1468014510 conn(0x557d3291aa50 msgr2=0x557d32963250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 -- 172.21.15.40:0/1468014510 shutdown_connections 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 -- 172.21.15.40:0/1468014510 wait complete. 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 -- start start 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.264+0000 7fb6571b0740 1 Processor -- start 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 0x557d3294b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32965ce0 0x557d3295b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3295acd0 0x557d3295b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb6571b0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557d3291eec0 con 0x557d3295acd0 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb6571b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557d32809fe0 con 0x557d32948440 2024-04-26T21:37:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb6571b0740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557d32918980 con 0x557d32965ce0 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb6571b0740 10 client.0 _ll_register_callbacks cb 0x557d32820150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3295acd0 0x557d3295b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3295acd0 0x557d3295b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57224/0 (socket says 172.21.15.40:57224) 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb65295a640 1 -- 172.21.15.40:0/557230366 learned_addr learned my addr 172.21.15.40:0/557230366 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb652159640 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 0x557d3294b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.265+0000 7fb651958640 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32965ce0 0x557d3295b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb65295a640 1 -- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32965ce0 msgr2=0x557d3295b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb65295a640 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32965ce0 0x557d3295b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb65295a640 1 -- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 msgr2=0x557d3294b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb65295a640 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 0x557d3294b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb65295a640 1 -- 172.21.15.40:0/557230366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d325942d0 con 0x557d3295acd0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb651958640 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32965ce0 0x557d3295b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb65295a640 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3295acd0 0x557d3295b9e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb644059570 tx=0x7fb64405c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c859e9abec93edb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb62d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb62d7fa640 1 -- 172.21.15.40:0/557230366 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644088070 con 0x557d3295acd0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb62d7fa640 1 -- 172.21.15.40:0/557230366 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fb644058c30 con 0x557d3295acd0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 1 -- 172.21.15.40:0/557230366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64405d990 con 0x557d3295acd0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb62d7fa640 1 -- 172.21.15.40:0/557230366 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb64405eb20 con 0x557d3295acd0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 10 client.6195 fetch_fsmap learned FSMap version 32 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 1 -- 172.21.15.40:0/557230366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x557d3265e6b0 con 0x557d3295acd0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb62d7fa640 1 -- 172.21.15.40:0/557230366 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64405d990 con 0x557d3295acd0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb62d7fa640 1 -- 172.21.15.40:0/557230366 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7fb644055330 con 0x557d3295acd0 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 10 client.6195 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T21:37:48.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 10 client.6195 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 20 client.6195 populate_metadata read hostname 'smithi040' 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x557d3294a930 0x557d329737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 4 client.6195 mds_command: new command op to 6015 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.266+0000 7fb6571b0740 1 -- 172.21.15.40:0/557230366 --> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d32890350 con 0x557d3294a930 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.267+0000 7fb652159640 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x557d3294a930 0x557d329737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.267+0000 7fb652159640 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x557d3294a930 0x557d329737f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a555a7d51fa7395a server_cookie=e48668fad696448 in_seq=0 out_seq=0 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.267+0000 7fb62d7fa640 10 client.6195 ms_handle_connect on v2:172.21.15.40:6832/2276808574 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.267+0000 7fb62d7fa640 1 -- 172.21.15.40:0/557230366 <== mds.0 v2:172.21.15.40:6832/2276808574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d32890350 con 0x557d3294a930 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.267+0000 7fb62d7fa640 10 client.6195 handle_command_reply: tid=0 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.267+0000 7fb6571b0740 1 client.6195 shutdown 2024-04-26T21:37:48.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 -- 172.21.15.40:0/557230366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3295acd0 msgr2=0x557d3295b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3295acd0 0x557d3295b9e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb644059570 tx=0x7fb64405c070 comp rx=0 tx=0).stop 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 -- 172.21.15.40:0/557230366 shutdown_connections 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x557d3294a930 0x557d329737f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d3295acd0 0x557d3295b9e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32965ce0 0x557d3295b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 --2- 172.21.15.40:0/557230366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 0x557d3294b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb62d7fa640 0 client.6195 ms_handle_reset on v2:172.21.15.40:6832/2276808574 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 -- 172.21.15.40:0/557230366 >> 172.21.15.40:0/557230366 conn(0x557d3291e120 msgr2=0x557d32967060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 -- 172.21.15.40:0/557230366 shutdown_connections 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 1 -- 172.21.15.40:0/557230366 wait complete. 2024-04-26T21:37:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.268+0000 7fb6571b0740 20 client.6195 trim_cache size 0 max 16384 2024-04-26T21:37:48.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.286+0000 7fb6571b0740 1 Processor -- start 2024-04-26T21:37:48.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.286+0000 7fb6571b0740 1 -- start start 2024-04-26T21:37:48.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 0x557d3295a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d3295acd0 0x557d3295bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d32917f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb6571b0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c201580 con 0x557d32965ce0 2024-04-26T21:37:48.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb6571b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c2008e0 con 0x557d3295acd0 2024-04-26T21:37:48.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb6571b0740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb64c200f30 con 0x557d32948440 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d32917f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d32917f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57228/0 (socket says 172.21.15.40:57228) 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb65295a640 1 -- 172.21.15.40:0/4082552845 learned_addr learned my addr 172.21.15.40:0/4082552845 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb651958640 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d3295acd0 0x557d3295bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb652159640 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 0x557d3295a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb65295a640 1 -- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 msgr2=0x557d3295a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb65295a640 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 0x557d3295a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb65295a640 1 -- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d3295acd0 msgr2=0x557d3295bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb65295a640 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d3295acd0 0x557d3295bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb65295a640 1 -- 172.21.15.40:0/4082552845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d32944110 con 0x557d32965ce0 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.287+0000 7fb651958640 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d3295acd0 0x557d3295bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb65295a640 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d32917f30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb6440559f0 tx=0x7fb644084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc5001a16036b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb62f7fe640 1 -- 172.21.15.40:0/4082552845 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644088070 con 0x557d32965ce0 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb62f7fe640 1 -- 172.21.15.40:0/4082552845 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64405d990 con 0x557d32965ce0 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb62f7fe640 1 -- 172.21.15.40:0/4082552845 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440593b0 con 0x557d32965ce0 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 -- 172.21.15.40:0/4082552845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 msgr2=0x557d32917f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d32917f30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb6440559f0 tx=0x7fb644084000 comp rx=0 tx=0).stop 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 -- 172.21.15.40:0/4082552845 shutdown_connections 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d32917f30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d3295acd0 0x557d3295bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 --2- 172.21.15.40:0/4082552845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d32948440 0x557d3295a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 -- 172.21.15.40:0/4082552845 >> 172.21.15.40:0/4082552845 conn(0x557d3291e120 msgr2=0x557d3291f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 -- 172.21.15.40:0/4082552845 shutdown_connections 2024-04-26T21:37:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.288+0000 7fb6571b0740 1 -- 172.21.15.40:0/4082552845 wait complete. 2024-04-26T21:37:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.289+0000 7fb6571b0740 1 -- start start 2024-04-26T21:37:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.289+0000 7fb6571b0740 1 Processor -- start 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 0x557d329e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d3295acd0 0x557d32986390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb6571b0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d329868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb6571b0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557d3291eec0 con 0x557d32965ce0 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb6571b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557d32949a60 con 0x557d32948440 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb6571b0740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557d32809fe0 con 0x557d3295acd0 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb6571b0740 10 client.0 _ll_register_callbacks cb 0x7fb644086070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d329868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d329868d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57230/0 (socket says 172.21.15.40:57230) 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 -- 172.21.15.40:0/1149413345 learned_addr learned my addr 172.21.15.40:0/1149413345 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:48.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb652159640 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 0x557d329e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb651958640 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d3295acd0 0x557d32986390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 -- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d3295acd0 msgr2=0x557d32986390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d3295acd0 0x557d32986390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 -- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 msgr2=0x557d329e0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 0x557d329e0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 -- 172.21.15.40:0/1149413345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3291d410 con 0x557d32965ce0 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb651958640 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d3295acd0 0x557d32986390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb65295a640 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d329868d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb644055a40 tx=0x7fb644001390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2021df029eebb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.290+0000 7fb62d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb62d7fa640 1 -- 172.21.15.40:0/1149413345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64407d070 con 0x557d32965ce0 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb62d7fa640 1 -- 172.21.15.40:0/1149413345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fb6440053c0 con 0x557d32965ce0 2024-04-26T21:37:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 1 -- 172.21.15.40:0/1149413345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64405d990 con 0x557d32965ce0 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb62d7fa640 1 -- 172.21.15.40:0/1149413345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb6440593b0 con 0x557d32965ce0 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 10 client.6201 fetch_fsmap learned FSMap version 32 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 1 -- 172.21.15.40:0/1149413345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x557d3291d410 con 0x557d32965ce0 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb62d7fa640 1 -- 172.21.15.40:0/1149413345 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64405d990 con 0x557d32965ce0 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb62d7fa640 1 -- 172.21.15.40:0/1149413345 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7fb6440055c0 con 0x557d32965ce0 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 10 client.6201 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 10 client.6201 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 20 client.6201 populate_metadata read hostname 'smithi040' 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x557d3297b360 0x557d3297b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 4 client.6201 mds_command: new command op to 6015 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb6571b0740 1 -- 172.21.15.40:0/1149413345 --> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] -- command(tid 0: {"prefix": "status"}) -- 0x557d3288a1c0 con 0x557d3297b360 2024-04-26T21:37:48.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb652159640 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x557d3297b360 0x557d3297b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb62d7fa640 10 client.6201 ms_handle_connect on v2:172.21.15.40:6832/2276808574 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.291+0000 7fb652159640 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x557d3297b360 0x557d3297b740 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b2382d8a43a5239 server_cookie=5c027bc55e0f3c91 in_seq=0 out_seq=0 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb62d7fa640 1 -- 172.21.15.40:0/1149413345 <== mds.0 v2:172.21.15.40:6832/2276808574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x557d3288a1c0 con 0x557d3297b360 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb62d7fa640 10 client.6201 handle_command_reply: tid=0 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 client.6201 shutdown 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 -- 172.21.15.40:0/1149413345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 msgr2=0x557d329868d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d329868d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb644055a40 tx=0x7fb644001390 comp rx=0 tx=0).stop 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 -- 172.21.15.40:0/1149413345 shutdown_connections 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x557d3297b360 0x557d3297b740 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557d32965ce0 0x557d329868d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557d3295acd0 0x557d32986390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb62d7fa640 0 client.6201 ms_handle_reset on v2:172.21.15.40:6832/2276808574 2024-04-26T21:37:48.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 --2- 172.21.15.40:0/1149413345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d32948440 0x557d329e0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 -- 172.21.15.40:0/1149413345 >> 172.21.15.40:0/1149413345 conn(0x557d3291e120 msgr2=0x557d32946670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:48.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 -- 172.21.15.40:0/1149413345 shutdown_connections 2024-04-26T21:37:48.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 1 -- 172.21.15.40:0/1149413345 wait complete. 2024-04-26T21:37:48.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6571b0740 20 client.6201 trim_cache size 0 max 16384 2024-04-26T21:37:48.300 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6015, 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.147820958000001, 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 32, 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 52, 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 52, 2024-04-26T21:37:48.301 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.167717501 2024-04-26T21:37:48.302 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:37:48.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6543bd640 1 -- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb63006ca90 msgr2=0x7fb63006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.292+0000 7fb6543bd640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb63006ca90 0x7fb63006ef50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb648033000 tx=0x7fb64802e000 comp rx=0 tx=0).stop 2024-04-26T21:37:48.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 -- 172.21.15.40:0/37867119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb64c0ca9e0 msgr2=0x7fb64c0cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb64c0ca9e0 0x7fb64c0cc7b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0cf3e0 tx=0x7fb64400fbe0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 -- 172.21.15.40:0/37867119 shutdown_connections 2024-04-26T21:37:48.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb63006ca90 0x7fb63006ef50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb64c0ca9e0 0x7fb64c0cc7b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c0ca560 0x7fb64c0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 --2- 172.21.15.40:0/37867119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6380039b0 0x7fb64c156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 -- 172.21.15.40:0/37867119 >> 172.21.15.40:0/37867119 conn(0x7fb64c0d2da0 msgr2=0x7fb64c06d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 -- 172.21.15.40:0/37867119 shutdown_connections 2024-04-26T21:37:48.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.293+0000 7fb6543bd640 1 -- 172.21.15.40:0/37867119 wait complete. 2024-04-26T21:37:48.304 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:37:48.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.407+0000 7f94f3052640 1 Processor -- start 2024-04-26T21:37:48.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.407+0000 7f94f3052640 1 -- start start 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94f3052640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94ec1604e0 0x7f94ec06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94f3052640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f94ec06fcd0 0x7f94ec0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94f3052640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94ec160970 con 0x7f94ec15f2d0 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94f3052640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f94ec074010 con 0x7f94ec1604e0 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94f3052640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f94ec070100 con 0x7f94ec06fcd0 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94ebfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94ec1604e0 0x7f94ec06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94ebfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94ec1604e0 0x7f94ec06f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57236/0 (socket says 172.21.15.40:57236) 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94ebfff640 1 -- 172.21.15.40:0/3917058595 learned_addr learned my addr 172.21.15.40:0/3917058595 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94f0dee640 1 --1- 172.21.15.40:0/3917058595 >> v1:172.21.15.40:6789/0 conn(0x7f94ec15f2d0 0x7f94ec072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60564/0 (socket says 172.21.15.40:60564) 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94f15ef640 1 --2- 172.21.15.40:0/3917058595 >> v2:172.21.15.184:3300/0 conn(0x7f94ec06fcd0 0x7f94ec0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94ebfff640 1 -- 172.21.15.40:0/3917058595 >> v2:172.21.15.184:3300/0 conn(0x7f94ec06fcd0 msgr2=0x7f94ec0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94ebfff640 1 --2- 172.21.15.40:0/3917058595 >> v2:172.21.15.184:3300/0 conn(0x7f94ec06fcd0 0x7f94ec0ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94ebfff640 1 -- 172.21.15.40:0/3917058595 >> v1:172.21.15.40:6789/0 conn(0x7f94ec15f2d0 legacy=0x7f94ec072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94ebfff640 1 -- 172.21.15.40:0/3917058595 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec072f70 con 0x7f94ec1604e0 2024-04-26T21:37:48.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.408+0000 7f94f15ef640 1 --2- 172.21.15.40:0/3917058595 >> v2:172.21.15.184:3300/0 conn(0x7f94ec06fcd0 0x7f94ec0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:48.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.409+0000 7f94ebfff640 1 --2- 172.21.15.40:0/3917058595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec06f790 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f94ec071590 tx=0x7f94d802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccff900c88996be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.409+0000 7f94eb7fe640 1 -- 172.21.15.40:0/3917058595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8037070 con 0x7f94ec1604e0 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.409+0000 7f94eb7fe640 1 -- 172.21.15.40:0/3917058595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d8032440 con 0x7f94ec1604e0 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.409+0000 7f94eb7fe640 1 -- 172.21.15.40:0/3917058595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8032810 con 0x7f94ec1604e0 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.410+0000 7f94f3052640 1 -- 172.21.15.40:0/3917058595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 msgr2=0x7f94ec06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.410+0000 7f94f3052640 1 --2- 172.21.15.40:0/3917058595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec06f790 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f94ec071590 tx=0x7f94d802a2a0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.410+0000 7f94f3052640 1 -- 172.21.15.40:0/3917058595 shutdown_connections 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.410+0000 7f94f3052640 1 --2- 172.21.15.40:0/3917058595 >> v2:172.21.15.184:3300/0 conn(0x7f94ec06fcd0 0x7f94ec0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.410+0000 7f94f3052640 1 --2- 172.21.15.40:0/3917058595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec06f790 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.410+0000 7f94f3052640 1 -- 172.21.15.40:0/3917058595 >> 172.21.15.40:0/3917058595 conn(0x7f94ec0d0be0 msgr2=0x7f94ec0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:48.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.410+0000 7f94f3052640 1 -- 172.21.15.40:0/3917058595 shutdown_connections 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.410+0000 7f94f3052640 1 -- 172.21.15.40:0/3917058595 wait complete. 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.411+0000 7f94f3052640 1 Processor -- start 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.411+0000 7f94f3052640 1 -- start start 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f3052640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ec06fcd0 0x7f94ec14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f3052640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94ec15f2d0 0x7f94ec0c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f3052640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec1f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f3052640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec073810 con 0x7f94ec1604e0 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f3052640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec073530 con 0x7f94ec06fcd0 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f3052640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94ec073af0 con 0x7f94ec15f2d0 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec1f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec1f0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57246/0 (socket says 172.21.15.40:57246) 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 -- 172.21.15.40:0/1523377399 learned_addr learned my addr 172.21.15.40:0/1523377399 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94ebfff640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94ec15f2d0 0x7f94ec0c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f0dee640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ec06fcd0 0x7f94ec14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 -- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94ec15f2d0 msgr2=0x7f94ec0c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94ec15f2d0 0x7f94ec0c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 -- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ec06fcd0 msgr2=0x7f94ec14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ec06fcd0 0x7f94ec14c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 -- 172.21.15.40:0/1523377399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec154ab0 con 0x7f94ec1604e0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.412+0000 7f94f15ef640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec1f0e40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f94e400cf20 tx=0x7f94e400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.413+0000 7f94e97fa640 1 -- 172.21.15.40:0/1523377399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e401a070 con 0x7f94ec1604e0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.413+0000 7f94e97fa640 1 -- 172.21.15.40:0/1523377399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e400d840 con 0x7f94ec1604e0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.413+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec154d90 con 0x7f94ec1604e0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.413+0000 7f94e97fa640 1 -- 172.21.15.40:0/1523377399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e400dc30 con 0x7f94ec1604e0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.413+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec1f1380 con 0x7f94ec1604e0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.414+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ec0694f0 con 0x7f94ec1604e0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.414+0000 7f94e97fa640 1 -- 172.21.15.40:0/1523377399 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94e401d070 con 0x7f94ec1604e0 2024-04-26T21:37:48.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.415+0000 7f94e97fa640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94d006c8f0 0x7f94d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:48.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.415+0000 7f94e97fa640 1 -- 172.21.15.40:0/1523377399 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f94e408c560 con 0x7f94ec1604e0 2024-04-26T21:37:48.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.415+0000 7f94f0dee640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94d006c8f0 0x7f94d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:48.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.416+0000 7f94f0dee640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94d006c8f0 0x7f94d006edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f94e0000f60 tx=0x7f94e000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:48.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.419+0000 7f94e97fa640 1 -- 172.21.15.40:0/1523377399 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94e405a800 con 0x7f94ec1604e0 2024-04-26T21:37:48.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.647+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f94ec0c7a70 con 0x7f94ec1604e0 2024-04-26T21:37:48.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.647+0000 7f94e97fa640 1 -- 172.21.15.40:0/1523377399 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) ==== 53+0+73 (secure 0 0 0) 0x7f94e405a100 con 0x7f94ec1604e0 2024-04-26T21:37:48.647 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94d006c8f0 msgr2=0x7f94d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94d006c8f0 0x7f94d006edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f94e0000f60 tx=0x7f94e000b040 comp rx=0 tx=0).stop 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 msgr2=0x7f94ec1f0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec1f0e40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f94e400cf20 tx=0x7f94e400a7d0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 shutdown_connections 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94d006c8f0 0x7f94d006edb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec1604e0 0x7f94ec1f0e40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94ec15f2d0 0x7f94ec0c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 --2- 172.21.15.40:0/1523377399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ec06fcd0 0x7f94ec14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:48.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 >> 172.21.15.40:0/1523377399 conn(0x7f94ec0d0be0 msgr2=0x7f94ec06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:48.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.649+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 shutdown_connections 2024-04-26T21:37:48.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48.650+0000 7f94f3052640 1 -- 172.21.15.40:0/1523377399 wait complete. 2024-04-26T21:37:48.660 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:37:48.660 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:37:48.660 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:37:48.660 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:37:48.661 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:48.661 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:37:48.661 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:37:48.661 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:37:48.661 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:37:48.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:48.661 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:48.684 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:37:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:37:48.686 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:37:48.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:48.686 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:37:48.746 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:37:48.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:48.748 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:37:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:37:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:37:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:37:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:37:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:37:48.806 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:37:48.806 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:37:48.806 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:37:48.806 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:37:48.806 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:37:48.806 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:37:48.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:37:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:37:48.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:48.933 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:37:48.948 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:37:48.948 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:37:48.948 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:37:48.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:48.948 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:37:49.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:37:49.012 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:49.068 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:37:49.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:37:49.068 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:49.124 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:37:49.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:49.124 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:37:49.206 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:37:49.206 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:37:49.213 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:37:49.213 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:37:49.214 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:37:49.214 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:37:49.214 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:37:49.214 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:37:49.214 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:37:49.215 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:37:49.215 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:37:49.215 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:37:49.216 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:37:49.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:49.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:49.282 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:37:49.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:37:49.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:37:49.383 INFO:teuthology.orchestra.run.smithi040.stdout:client6213 2024-04-26T21:37:49.383 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:37:49.383 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:49.399 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/b') 2024-04-26T21:37:49.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:37:49.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:37:49.502 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-04-26T21:37:49.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.606+0000 7ff0b9637640 1 Processor -- start 2024-04-26T21:37:49.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.606+0000 7ff0b9637640 1 -- start start 2024-04-26T21:37:49.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b9637640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff0b415d1e0 0x7ff0b40d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b9637640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b40d1a90 0x7ff0b4070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b9637640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b40ce840 con 0x7ff0b40d0f20 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b9637640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b40cede0 con 0x7ff0b415d1e0 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b9637640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b4070650 con 0x7ff0b40d1a90 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b2ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff0b415d1e0 0x7ff0b40d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b37fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b40d1a90 0x7ff0b4070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b2ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff0b415d1e0 0x7ff0b40d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:56472/0 (socket says 172.21.15.40:56472) 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/2702325612 learned_addr learned my addr 172.21.15.40:0/2702325612 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b37fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b40d1a90 0x7ff0b4070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:58944/0 (socket says 172.21.15.40:58944) 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0aa7fc640 1 --1- 172.21.15.40:0/2702325612 >> v1:172.21.15.184:6789/0 conn(0x7ff0b40d0f20 0x7ff0b40d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:34494/0 (socket says 172.21.15.40:34494) 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/2702325612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b40d1a90 msgr2=0x7ff0b4070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/2702325612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b40d1a90 0x7ff0b4070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/2702325612 >> v1:172.21.15.184:6789/0 conn(0x7ff0b40d0f20 legacy=0x7ff0b40d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/2702325612 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b40ceb00 con 0x7ff0b415d1e0 2024-04-26T21:37:49.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.607+0000 7ff0b37fe640 1 --2- 172.21.15.40:0/2702325612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b40d1a90 0x7ff0b4070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.608+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/2702325612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b415d1e0 0x7ff0b40d09e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009870 tx=0x7ff0a402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad00adab5b4a53e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.608+0000 7ff0b27fc640 1 -- 172.21.15.40:0/2702325612 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4037070 con 0x7ff0b415d1e0 2024-04-26T21:37:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.608+0000 7ff0b27fc640 1 -- 172.21.15.40:0/2702325612 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a4032440 con 0x7ff0b415d1e0 2024-04-26T21:37:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.608+0000 7ff0b27fc640 1 -- 172.21.15.40:0/2702325612 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4032810 con 0x7ff0b415d1e0 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.609+0000 7ff0b9637640 1 -- 172.21.15.40:0/2702325612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b415d1e0 msgr2=0x7ff0b40d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.609+0000 7ff0b9637640 1 --2- 172.21.15.40:0/2702325612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b415d1e0 0x7ff0b40d09e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009870 tx=0x7ff0a402a270 comp rx=0 tx=0).stop 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.609+0000 7ff0b9637640 1 -- 172.21.15.40:0/2702325612 shutdown_connections 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.609+0000 7ff0b9637640 1 --2- 172.21.15.40:0/2702325612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b40d1a90 0x7ff0b4070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.609+0000 7ff0b9637640 1 --2- 172.21.15.40:0/2702325612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b415d1e0 0x7ff0b40d09e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.609+0000 7ff0b9637640 1 -- 172.21.15.40:0/2702325612 >> 172.21.15.40:0/2702325612 conn(0x7ff0b40d2da0 msgr2=0x7ff0b40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.609+0000 7ff0b9637640 1 -- 172.21.15.40:0/2702325612 shutdown_connections 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.609+0000 7ff0b9637640 1 -- 172.21.15.40:0/2702325612 wait complete. 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.610+0000 7ff0b9637640 1 Processor -- start 2024-04-26T21:37:49.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.610+0000 7ff0b9637640 1 -- start start 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b9637640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40d0f20 0x7ff0b414a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b9637640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b40d1a90 0x7ff0b40c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b9637640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b415d1e0 0x7ff0b40cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b9637640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4071020 con 0x7ff0b40d0f20 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b9637640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b40ce4c0 con 0x7ff0b40d1a90 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b9637640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b40711a0 con 0x7ff0b415d1e0 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40d0f20 0x7ff0b414a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40d0f20 0x7ff0b414a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52726/0 (socket says 172.21.15.40:52726) 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/1473834727 learned_addr learned my addr 172.21.15.40:0/1473834727 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b40d1a90 0x7ff0b40c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.611+0000 7ff0b37fe640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b415d1e0 0x7ff0b40cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b415d1e0 msgr2=0x7ff0b40cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b415d1e0 0x7ff0b40cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b40d1a90 msgr2=0x7ff0b40c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b40d1a90 0x7ff0b40c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/1473834727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b4157b10 con 0x7ff0b40d0f20 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b40d1a90 0x7ff0b40c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40d0f20 0x7ff0b414a2b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4000c00 tx=0x7ff0a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0abfff640 1 -- 172.21.15.40:0/1473834727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4037040 con 0x7ff0b40d0f20 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0abfff640 1 -- 172.21.15.40:0/1473834727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a4042070 con 0x7ff0b40d0f20 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0b9637640 1 -- 172.21.15.40:0/1473834727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b4158c50 con 0x7ff0b40d0f20 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0abfff640 1 -- 172.21.15.40:0/1473834727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4030880 con 0x7ff0b40d0f20 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.612+0000 7ff0b9637640 1 -- 172.21.15.40:0/1473834727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b4159210 con 0x7ff0b40d0f20 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.614+0000 7ff0abfff640 1 -- 172.21.15.40:0/1473834727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0a4030a20 con 0x7ff0b40d0f20 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.614+0000 7ff0abfff640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff09406c8f0 0x7ff09406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.614+0000 7ff0abfff640 1 -- 172.21.15.40:0/1473834727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7ff0a40ac6a0 con 0x7ff0b40d0f20 2024-04-26T21:37:49.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.614+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff09406c8f0 0x7ff09406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.615+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff09406c8f0 0x7ff09406edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff09c007fd0 tx=0x7ff09c007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.615+0000 7ff0bc42d740 1 Processor -- start 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.615+0000 7ff0bc42d740 1 -- start start 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbcef80 0x55f61fbcf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbc9e10 0x55f61fbcdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbce0c0 0x55f61fbce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f61fb33d80 con 0x55f61fbc9e10 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f61faed8b0 con 0x55f61fbce0c0 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f61fb2fd40 con 0x55f61fbcef80 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbc9e10 0x55f61fbcdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbc9e10 0x55f61fbcdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52736/0 (socket says 172.21.15.40:52736) 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 -- 172.21.15.40:0/774072220 learned_addr learned my addr 172.21.15.40:0/774072220 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0b37fe640 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbce0c0 0x55f61fbce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbcef80 0x55f61fbcf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 -- 172.21.15.40:0/774072220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbcef80 msgr2=0x55f61fbcf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbcef80 0x55f61fbcf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 -- 172.21.15.40:0/774072220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbce0c0 msgr2=0x55f61fbce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbce0c0 0x55f61fbce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 -- 172.21.15.40:0/774072220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61fabea40 con 0x55f61fbc9e10 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.616+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbc9e10 0x55f61fbcdb80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff09c005fd0 tx=0x7ff09c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a1d1cfd9257596 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0a8ff9640 1 -- 172.21.15.40:0/774072220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c015110 con 0x55f61fbc9e10 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0a8ff9640 1 -- 172.21.15.40:0/774072220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09c015990 con 0x55f61fbc9e10 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0a8ff9640 1 -- 172.21.15.40:0/774072220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c015d40 con 0x55f61fbc9e10 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 -- 172.21.15.40:0/774072220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbc9e10 msgr2=0x55f61fbcdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbc9e10 0x55f61fbcdb80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff09c005fd0 tx=0x7ff09c012000 comp rx=0 tx=0).stop 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 -- 172.21.15.40:0/774072220 shutdown_connections 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbce0c0 0x55f61fbce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbc9e10 0x55f61fbcdb80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/774072220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbcef80 0x55f61fbcf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 -- 172.21.15.40:0/774072220 >> 172.21.15.40:0/774072220 conn(0x55f61fbcc080 msgr2=0x55f61fc14b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 -- 172.21.15.40:0/774072220 shutdown_connections 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.617+0000 7ff0bc42d740 1 -- 172.21.15.40:0/774072220 wait complete. 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.618+0000 7ff0bc42d740 1 -- start start 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.618+0000 7ff0bc42d740 1 Processor -- start 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 0x55f61fbe2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fc16900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fc1b520 0x55f61fc1b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f61fbfab10 con 0x55f61fc1b520 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f61fabad10 con 0x55f61fbcef80 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f61fbf9f90 con 0x55f61fbce0c0 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0bc42d740 10 client.0 _ll_register_callbacks cb 0x55f61fae7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fc1b520 0x55f61fc1b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fc1b520 0x55f61fc1b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52748/0 (socket says 172.21.15.40:52748) 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 -- 172.21.15.40:0/3616607275 learned_addr learned my addr 172.21.15.40:0/3616607275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 0x55f61fbe2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fc16900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 -- 172.21.15.40:0/3616607275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 msgr2=0x55f61fbe2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 0x55f61fbe2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 -- 172.21.15.40:0/3616607275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 msgr2=0x55f61fc16900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fc16900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 -- 172.21.15.40:0/3616607275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61f90f7d0 con 0x55f61fc1b520 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff0b37fe640 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fc1b520 0x55f61fc1b9a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac007da0 tx=0x7ff0ac014690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ba999c448bd4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.619+0000 7ff091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff091ffb640 1 -- 172.21.15.40:0/3616607275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac01c020 con 0x55f61fc1b520 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff091ffb640 1 -- 172.21.15.40:0/3616607275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7ff0ac012040 con 0x55f61fc1b520 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3616607275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff09c015990 con 0x55f61fc1b520 2024-04-26T21:37:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff091ffb640 1 -- 172.21.15.40:0/3616607275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff0ac026050 con 0x55f61fc1b520 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 10 client.6222 fetch_fsmap learned FSMap version 32 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3616607275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55f61f8453f0 con 0x55f61fc1b520 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff091ffb640 1 -- 172.21.15.40:0/3616607275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09c015990 con 0x55f61fc1b520 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff091ffb640 1 -- 172.21.15.40:0/3616607275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7ff0ac029020 con 0x55f61fc1b520 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 10 client.6222 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 10 client.6222 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 20 client.6222 populate_metadata read hostname 'smithi040' 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x55f61fbf09c0 0x55f61fbf0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 4 client.6222 mds_command: new command op to 6015 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3616607275 --> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61fb41990 con 0x55f61fbf09c0 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.620+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x55f61fbf09c0 0x55f61fbf0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff091ffb640 10 client.6222 ms_handle_connect on v2:172.21.15.40:6832/2276808574 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x55f61fbf09c0 0x55f61fbf0da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c2d1af899dbe13 server_cookie=5e20bc59f5d9e72a in_seq=0 out_seq=0 2024-04-26T21:37:49.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff091ffb640 1 -- 172.21.15.40:0/3616607275 <== mds.0 v2:172.21.15.40:6832/2276808574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f61fb41990 con 0x55f61fbf09c0 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff091ffb640 10 client.6222 handle_command_reply: tid=0 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 client.6222 shutdown 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3616607275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fc1b520 msgr2=0x55f61fc1b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fc1b520 0x55f61fc1b9a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac007da0 tx=0x7ff0ac014690 comp rx=0 tx=0).stop 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3616607275 shutdown_connections 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x55f61fbf09c0 0x55f61fbf0da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fc1b520 0x55f61fc1b9a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff091ffb640 0 client.6222 ms_handle_reset on v2:172.21.15.40:6832/2276808574 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fc16900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3616607275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 0x55f61fbe2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3616607275 >> 172.21.15.40:0/3616607275 conn(0x55f61fbc9e10 msgr2=0x55f61fbcfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3616607275 shutdown_connections 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.621+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3616607275 wait complete. 2024-04-26T21:37:49.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.622+0000 7ff0bc42d740 20 client.6222 trim_cache size 0 max 16384 2024-04-26T21:37:49.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.636+0000 7ff0bc42d740 1 Processor -- start 2024-04-26T21:37:49.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.636+0000 7ff0bc42d740 1 -- start start 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbce0c0 0x55f61fbc8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fbf9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fc1b520 0x55f61fbfa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41ffbc0 con 0x55f61fbce0c0 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41fec70 con 0x55f61fbcef80 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b415ac70 con 0x55f61fc1b520 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbce0c0 0x55f61fbc8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbce0c0 0x55f61fbc8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52752/0 (socket says 172.21.15.40:52752) 2024-04-26T21:37:49.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/232361311 learned_addr learned my addr 172.21.15.40:0/232361311 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fbf9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b37fe640 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fc1b520 0x55f61fbfa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fc1b520 msgr2=0x55f61fbfa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fc1b520 0x55f61fbfa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 msgr2=0x55f61fbf9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fbf9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0b2ffd640 1 -- 172.21.15.40:0/232361311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61fbf1110 con 0x55f61fbce0c0 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.637+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fbf9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbce0c0 0x55f61fbc8eb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff0a407a0b0 tx=0x7ff0a40aa870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87c1f54d7ed5e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff093fff640 1 -- 172.21.15.40:0/232361311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4097070 con 0x55f61fbce0c0 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff093fff640 1 -- 172.21.15.40:0/232361311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a4092070 con 0x55f61fbce0c0 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff093fff640 1 -- 172.21.15.40:0/232361311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a407a120 con 0x55f61fbce0c0 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 -- 172.21.15.40:0/232361311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbce0c0 msgr2=0x55f61fbc8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbce0c0 0x55f61fbc8eb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff0a407a0b0 tx=0x7ff0a40aa870 comp rx=0 tx=0).stop 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 -- 172.21.15.40:0/232361311 shutdown_connections 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fc1b520 0x55f61fbfa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fbcef80 0x55f61fbf9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/232361311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbce0c0 0x55f61fbc8eb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 -- 172.21.15.40:0/232361311 >> 172.21.15.40:0/232361311 conn(0x55f61fbc9e10 msgr2=0x55f61fc1b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 -- 172.21.15.40:0/232361311 shutdown_connections 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.638+0000 7ff0bc42d740 1 -- 172.21.15.40:0/232361311 wait complete. 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.639+0000 7ff0bc42d740 1 -- start start 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.639+0000 7ff0bc42d740 1 Processor -- start 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 0x55f61fc8dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbcef80 0x55f61fbee980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0bc42d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fc1b520 0x55f61fbfa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41fe620 con 0x55f61fbcef80 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f61fbccf10 con 0x55f61fc1b520 2024-04-26T21:37:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0bc42d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f61fc14b30 con 0x55f61fbce0c0 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0bc42d740 10 client.0 _ll_register_callbacks cb 0x7ff0ac009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbcef80 0x55f61fbee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbcef80 0x55f61fbee980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52754/0 (socket says 172.21.15.40:52754) 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 -- 172.21.15.40:0/3715540529 learned_addr learned my addr 172.21.15.40:0/3715540529 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 0x55f61fc8dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0b37fe640 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fc1b520 0x55f61fbfa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 -- 172.21.15.40:0/3715540529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 msgr2=0x55f61fc8dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 0x55f61fc8dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 -- 172.21.15.40:0/3715540529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fc1b520 msgr2=0x55f61fbfa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fc1b520 0x55f61fbfa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 -- 172.21.15.40:0/3715540529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61f8050f0 con 0x55f61fbcef80 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0aa7fc640 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbcef80 0x55f61fbee980 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40d1300 tx=0x7ff09c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930ec81c12ae8e7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:37:49.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff091ffb640 1 -- 172.21.15.40:0/3715540529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c015b20 con 0x55f61fbcef80 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff091ffb640 1 -- 172.21.15.40:0/3715540529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7ff09c015cc0 con 0x55f61fbcef80 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.640+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3715540529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a4092070 con 0x55f61fbcef80 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff091ffb640 1 -- 172.21.15.40:0/3715540529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff09c021450 con 0x55f61fbcef80 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0bc42d740 10 client.6228 fetch_fsmap learned FSMap version 32 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3715540529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55f61faf44b0 con 0x55f61fbcef80 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff091ffb640 1 -- 172.21.15.40:0/3715540529 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a4092070 con 0x55f61fbcef80 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff091ffb640 1 -- 172.21.15.40:0/3715540529 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7ff09c036050 con 0x55f61fbcef80 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0bc42d740 10 client.6228 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0bc42d740 10 client.6228 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0bc42d740 20 client.6228 populate_metadata read hostname 'smithi040' 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x55f61fbeeec0 0x55f61fbfa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0bc42d740 4 client.6228 mds_command: new command op to 6015 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3715540529 --> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f61fb3b800 con 0x55f61fbeeec0 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x55f61fbeeec0 0x55f61fbfa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff091ffb640 10 client.6228 ms_handle_connect on v2:172.21.15.40:6832/2276808574 2024-04-26T21:37:49.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.641+0000 7ff0b2ffd640 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x55f61fbeeec0 0x55f61fbfa5b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5d0d2ba4a59ff server_cookie=353119290011fab6 in_seq=0 out_seq=0 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff091ffb640 1 -- 172.21.15.40:0/3715540529 <== mds.0 v2:172.21.15.40:6832/2276808574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f61fb3b800 con 0x55f61fbeeec0 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff091ffb640 10 client.6228 handle_command_reply: tid=0 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 client.6228 shutdown 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3715540529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbcef80 msgr2=0x55f61fbee980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbcef80 0x55f61fbee980 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40d1300 tx=0x7ff09c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3715540529 shutdown_connections 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.40:6832/2276808574,v1:172.21.15.40:6834/2276808574] conn(0x55f61fbeeec0 0x55f61fbfa5b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f61fc1b520 0x55f61fbfa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f61fbcef80 0x55f61fbee980 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff091ffb640 0 client.6228 ms_handle_reset on v2:172.21.15.40:6832/2276808574 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 --2- 172.21.15.40:0/3715540529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f61fbce0c0 0x55f61fc8dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3715540529 >> 172.21.15.40:0/3715540529 conn(0x55f61fbc9e10 msgr2=0x55f61fc16880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3715540529 shutdown_connections 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.652+0000 7ff0bc42d740 1 -- 172.21.15.40:0/3715540529 wait complete. 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0bc42d740 20 client.6228 trim_cache size 0 max 16384 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0 2024-04-26T21:37:49.654 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:37:49.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 -- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff09406c8f0 msgr2=0x7ff09406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff09406c8f0 0x7ff09406edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff09c007fd0 tx=0x7ff09c007d60 comp rx=0 tx=0).stop 2024-04-26T21:37:49.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 -- 172.21.15.40:0/1473834727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40d0f20 msgr2=0x7ff0b414a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40d0f20 0x7ff0b414a2b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4000c00 tx=0x7ff0a40046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 -- 172.21.15.40:0/1473834727 shutdown_connections 2024-04-26T21:37:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff09406c8f0 0x7ff09406edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b415d1e0 0x7ff0b40cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b40d1a90 0x7ff0b40c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 --2- 172.21.15.40:0/1473834727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40d0f20 0x7ff0b414a2b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 -- 172.21.15.40:0/1473834727 >> 172.21.15.40:0/1473834727 conn(0x7ff0b40d2da0 msgr2=0x7ff0b40afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 -- 172.21.15.40:0/1473834727 shutdown_connections 2024-04-26T21:37:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.653+0000 7ff0b9637640 1 -- 172.21.15.40:0/1473834727 wait complete. 2024-04-26T21:37:49.664 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-04-26T21:37:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.768+0000 7f541d445640 1 Processor -- start 2024-04-26T21:37:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.768+0000 7f541d445640 1 -- start start 2024-04-26T21:37:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f541d445640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5418071260 0x7f541815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f541d445640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54180d2390 con 0x7f5418070350 2024-04-26T21:37:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f541d445640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f541806d3f0 con 0x7f54180cfde0 2024-04-26T21:37:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f541d445640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f54180afab0 con 0x7f5418071260 2024-04-26T21:37:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f54177fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5418071260 0x7f541815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f540ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5418070350 0x7f5418070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53540/0 (socket says 172.21.15.40:53540) 2024-04-26T21:37:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f540ffff640 1 -- 172.21.15.40:0/2656333092 learned_addr learned my addr 172.21.15.40:0/2656333092 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f54177fe640 1 -- 172.21.15.40:0/2656333092 >> v1:172.21.15.184:6789/0 conn(0x7f54180cfde0 legacy=0x7f54180d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f54177fe640 1 -- 172.21.15.40:0/2656333092 >> v1:172.21.15.40:6789/0 conn(0x7f5418070350 legacy=0x7f5418070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f54177fe640 1 -- 172.21.15.40:0/2656333092 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541806d6d0 con 0x7f5418071260 2024-04-26T21:37:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.769+0000 7f54167fc640 1 -- 172.21.15.40:0/2656333092 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540201764 0 0) 0x7f54180d2390 con 0x7f5418070350 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.770+0000 7f54177fe640 1 --2- 172.21.15.40:0/2656333092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5418071260 0x7f541815e9f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d90 tx=0x7f541002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9219f759854f9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.770+0000 7f54167fc640 1 -- 172.21.15.40:0/2656333092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410007740 con 0x7f5418071260 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.770+0000 7f54167fc640 1 -- 172.21.15.40:0/2656333092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54100078e0 con 0x7f5418071260 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.770+0000 7f54167fc640 1 -- 172.21.15.40:0/2656333092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410007cb0 con 0x7f5418071260 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.771+0000 7f541d445640 1 -- 172.21.15.40:0/2656333092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5418071260 msgr2=0x7f541815e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.771+0000 7f541d445640 1 --2- 172.21.15.40:0/2656333092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5418071260 0x7f541815e9f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d90 tx=0x7f541002a660 comp rx=0 tx=0).stop 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.771+0000 7f541d445640 1 -- 172.21.15.40:0/2656333092 shutdown_connections 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.771+0000 7f541d445640 1 --2- 172.21.15.40:0/2656333092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5418071260 0x7f541815e9f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.771+0000 7f541d445640 1 -- 172.21.15.40:0/2656333092 >> 172.21.15.40:0/2656333092 conn(0x7f54180d2d80 msgr2=0x7f54180d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.771+0000 7f541d445640 1 -- 172.21.15.40:0/2656333092 shutdown_connections 2024-04-26T21:37:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.771+0000 7f541d445640 1 -- 172.21.15.40:0/2656333092 wait complete. 2024-04-26T21:37:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.772+0000 7f541d445640 1 Processor -- start 2024-04-26T21:37:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.772+0000 7f541d445640 1 -- start start 2024-04-26T21:37:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f541d445640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5418070350 0x7f54181f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f541d445640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5418071260 0x7f54180c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f541d445640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54180cfde0 0x7f54181f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f541d445640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f541806f1c0 con 0x7f54180cfde0 2024-04-26T21:37:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f541d445640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f541806dd20 con 0x7f5418070350 2024-04-26T21:37:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f541d445640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f541806f340 con 0x7f5418071260 2024-04-26T21:37:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54180cfde0 0x7f54181f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54180cfde0 0x7f54181f7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52778/0 (socket says 172.21.15.40:52778) 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 -- 172.21.15.40:0/3325918703 learned_addr learned my addr 172.21.15.40:0/3325918703 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f540ffff640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5418071260 0x7f54180c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f5416ffd640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5418070350 0x7f54181f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 -- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5418071260 msgr2=0x7f54180c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5418071260 0x7f54180c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 -- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5418070350 msgr2=0x7f54181f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5418070350 0x7f54181f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 -- 172.21.15.40:0/3325918703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54181f8e10 con 0x7f54180cfde0 2024-04-26T21:37:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f540ffff640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5418071260 0x7f54180c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:49.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f5416ffd640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5418070350 0x7f54181f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:49.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.773+0000 7f54177fe640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54180cfde0 0x7f54181f7fc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d90 tx=0x7f5410004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.774+0000 7f540f7fe640 1 -- 172.21.15.40:0/3325918703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410040070 con 0x7f54180cfde0 2024-04-26T21:37:49.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.774+0000 7f540f7fe640 1 -- 172.21.15.40:0/3325918703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5410031c90 con 0x7f54180cfde0 2024-04-26T21:37:49.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.774+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54180a0710 con 0x7f54180cfde0 2024-04-26T21:37:49.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.774+0000 7f540f7fe640 1 -- 172.21.15.40:0/3325918703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541003c640 con 0x7f54180cfde0 2024-04-26T21:37:49.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.774+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54181f4ae0 con 0x7f54180cfde0 2024-04-26T21:37:49.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.775+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54180694f0 con 0x7f54180cfde0 2024-04-26T21:37:49.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.775+0000 7f540f7fe640 1 -- 172.21.15.40:0/3325918703 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5410039040 con 0x7f54180cfde0 2024-04-26T21:37:49.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.776+0000 7f540f7fe640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53f806c8f0 0x7f53f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:49.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.776+0000 7f540f7fe640 1 -- 172.21.15.40:0/3325918703 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f54100ad2f0 con 0x7f54180cfde0 2024-04-26T21:37:49.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.776+0000 7f5416ffd640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53f806c8f0 0x7f53f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:49.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.777+0000 7f5416ffd640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53f806c8f0 0x7f53f806edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f54180cf6e0 tx=0x7f5408009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:49.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.780+0000 7f540f7fe640 1 -- 172.21.15.40:0/3325918703 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f541007b6e0 con 0x7f54180cfde0 2024-04-26T21:37:49.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.964+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7f54180c9170 con 0x7f54180cfde0 2024-04-26T21:37:49.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.967+0000 7f540f7fe640 1 -- 172.21.15.40:0/3325918703 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v13)=0 v13) ==== 135+0+0 (secure 0 0 0) 0x7f541007afe0 con 0x7f54180cfde0 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53f806c8f0 msgr2=0x7f53f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53f806c8f0 0x7f53f806edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f54180cf6e0 tx=0x7f5408009040 comp rx=0 tx=0).stop 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54180cfde0 msgr2=0x7f54181f7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54180cfde0 0x7f54181f7fc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d90 tx=0x7f5410004a40 comp rx=0 tx=0).stop 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 shutdown_connections 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53f806c8f0 0x7f53f806edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54180cfde0 0x7f54181f7fc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5418071260 0x7f54180c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 --2- 172.21.15.40:0/3325918703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5418070350 0x7f54181f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 >> 172.21.15.40:0/3325918703 conn(0x7f54180d2d80 msgr2=0x7f54180b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 shutdown_connections 2024-04-26T21:37:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:49.969+0000 7f541d445640 1 -- 172.21.15.40:0/3325918703 wait complete. 2024-04-26T21:37:49.980 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-04-26T21:37:50.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.084+0000 7f09ecde2640 1 Processor -- start 2024-04-26T21:37:50.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.084+0000 7f09ecde2640 1 -- start start 2024-04-26T21:37:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09ecde2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09e80cfde0 0x7f09e80d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09ecde2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f09e8070350 0x7f09e8070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09ecde2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e80d2390 con 0x7f09e8071090 2024-04-26T21:37:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09ecde2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f09e80afab0 con 0x7f09e80cfde0 2024-04-26T21:37:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09ecde2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f09e806dd20 con 0x7f09e8070350 2024-04-26T21:37:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09e659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09e80cfde0 0x7f09e80d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09e6d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f09e8071090 0x7f09e815e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53552/0 (socket says 172.21.15.40:53552) 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09e6d9d640 1 -- 172.21.15.40:0/3476717663 learned_addr learned my addr 172.21.15.40:0/3476717663 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09dffff640 1 --2- 172.21.15.40:0/3476717663 >> v2:172.21.15.184:3300/0 conn(0x7f09e8070350 0x7f09e8070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09e659c640 1 -- 172.21.15.40:0/3476717663 >> v2:172.21.15.184:3300/0 conn(0x7f09e8070350 msgr2=0x7f09e8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09e659c640 1 --2- 172.21.15.40:0/3476717663 >> v2:172.21.15.184:3300/0 conn(0x7f09e8070350 0x7f09e8070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09e659c640 1 -- 172.21.15.40:0/3476717663 >> v1:172.21.15.40:6789/0 conn(0x7f09e8071090 legacy=0x7f09e815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09e659c640 1 -- 172.21.15.40:0/3476717663 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e806d3f0 con 0x7f09e80cfde0 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09e5d9b640 1 -- 172.21.15.40:0/3476717663 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 636767767 0 0) 0x7f09e80d2390 con 0x7f09e8071090 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.085+0000 7f09dffff640 1 --2- 172.21.15.40:0/3476717663 >> v2:172.21.15.184:3300/0 conn(0x7f09e8070350 0x7f09e8070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.086+0000 7f09e659c640 1 --2- 172.21.15.40:0/3476717663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e80cfde0 0x7f09e80d1b60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f09e806cb80 tx=0x7f09d802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9488040f09673d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.086+0000 7f09e5d9b640 1 -- 172.21.15.40:0/3476717663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8037070 con 0x7f09e80cfde0 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.086+0000 7f09e5d9b640 1 -- 172.21.15.40:0/3476717663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d8032440 con 0x7f09e80cfde0 2024-04-26T21:37:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.086+0000 7f09e5d9b640 1 -- 172.21.15.40:0/3476717663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d802ae60 con 0x7f09e80cfde0 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.087+0000 7f09ecde2640 1 -- 172.21.15.40:0/3476717663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e80cfde0 msgr2=0x7f09e80d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.087+0000 7f09ecde2640 1 --2- 172.21.15.40:0/3476717663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e80cfde0 0x7f09e80d1b60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f09e806cb80 tx=0x7f09d802a110 comp rx=0 tx=0).stop 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.087+0000 7f09ecde2640 1 -- 172.21.15.40:0/3476717663 shutdown_connections 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.087+0000 7f09ecde2640 1 --2- 172.21.15.40:0/3476717663 >> v2:172.21.15.184:3300/0 conn(0x7f09e8070350 0x7f09e8070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.087+0000 7f09ecde2640 1 --2- 172.21.15.40:0/3476717663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e80cfde0 0x7f09e80d1b60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.087+0000 7f09ecde2640 1 -- 172.21.15.40:0/3476717663 >> 172.21.15.40:0/3476717663 conn(0x7f09e80d2d80 msgr2=0x7f09e80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.087+0000 7f09ecde2640 1 -- 172.21.15.40:0/3476717663 shutdown_connections 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.087+0000 7f09ecde2640 1 -- 172.21.15.40:0/3476717663 wait complete. 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.088+0000 7f09ecde2640 1 Processor -- start 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.088+0000 7f09ecde2640 1 -- start start 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09ecde2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e8070350 0x7f09e81f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09ecde2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8071090 0x7f09e815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09ecde2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09e80cfde0 0x7f09e815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09ecde2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f09e806d9b0 con 0x7f09e8070350 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09ecde2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09e80714c0 con 0x7f09e8071090 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09ecde2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80d20a0 con 0x7f09e80cfde0 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e8070350 0x7f09e81f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e8070350 0x7f09e81f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52802/0 (socket says 172.21.15.40:52802) 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e659c640 1 -- 172.21.15.40:0/1429068742 learned_addr learned my addr 172.21.15.40:0/1429068742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e6d9d640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09e80cfde0 0x7f09e815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09dffff640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8071090 0x7f09e815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e659c640 1 -- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09e80cfde0 msgr2=0x7f09e815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e659c640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09e80cfde0 0x7f09e815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e659c640 1 -- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8071090 msgr2=0x7f09e815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e659c640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8071090 0x7f09e815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e659c640 1 -- 172.21.15.40:0/1429068742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e8161950 con 0x7f09e8070350 2024-04-26T21:37:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09e6d9d640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09e80cfde0 0x7f09e815ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:50.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.089+0000 7f09dffff640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8071090 0x7f09e815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:50.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.090+0000 7f09e659c640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e8070350 0x7f09e81f5480 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c00 tx=0x7f09d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:50.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.090+0000 7f09deffd640 1 -- 172.21.15.40:0/1429068742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8037040 con 0x7f09e8070350 2024-04-26T21:37:50.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.090+0000 7f09deffd640 1 -- 172.21.15.40:0/1429068742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d8042070 con 0x7f09e8070350 2024-04-26T21:37:50.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.090+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e8161390 con 0x7f09e8070350 2024-04-26T21:37:50.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.090+0000 7f09deffd640 1 -- 172.21.15.40:0/1429068742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8030a50 con 0x7f09e8070350 2024-04-26T21:37:50.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.090+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e815d700 con 0x7f09e8070350 2024-04-26T21:37:50.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.091+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e80694f0 con 0x7f09e8070350 2024-04-26T21:37:50.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.091+0000 7f09deffd640 1 -- 172.21.15.40:0/1429068742 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09d8030bf0 con 0x7f09e8070350 2024-04-26T21:37:50.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.092+0000 7f09deffd640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f09c806c8f0 0x7f09c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.092+0000 7f09deffd640 1 -- 172.21.15.40:0/1429068742 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f09d80ac770 con 0x7f09e8070350 2024-04-26T21:37:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.092+0000 7f09dffff640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f09c806c8f0 0x7f09c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.092+0000 7f09dffff640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f09c806c8f0 0x7f09c806edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f09d0006fd0 tx=0x7f09d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:50.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.096+0000 7f09deffd640 1 -- 172.21.15.40:0/1429068742 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09d807aa10 con 0x7f09e8070350 2024-04-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.281+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7f09e815d8d0 con 0x7f09e8070350 2024-04-26T21:37:50.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.284+0000 7f09deffd640 1 -- 172.21.15.40:0/1429068742 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v14)=0 v14) ==== 138+0+0 (secure 0 0 0) 0x7f09d807a310 con 0x7f09e8070350 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f09c806c8f0 msgr2=0x7f09c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f09c806c8f0 0x7f09c806edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f09d0006fd0 tx=0x7f09d0008040 comp rx=0 tx=0).stop 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e8070350 msgr2=0x7f09e81f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e8070350 0x7f09e81f5480 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c00 tx=0x7f09d80046c0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 shutdown_connections 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f09c806c8f0 0x7f09c806edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09e80cfde0 0x7f09e815ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09e8071090 0x7f09e815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 --2- 172.21.15.40:0/1429068742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09e8070350 0x7f09e81f5480 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 >> 172.21.15.40:0/1429068742 conn(0x7f09e80d2d80 msgr2=0x7f09e806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 shutdown_connections 2024-04-26T21:37:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:50.286+0000 7f09ecde2640 1 -- 172.21.15.40:0/1429068742 wait complete. 2024-04-26T21:37:55.298 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mv a/b a/c; sync .') 2024-04-26T21:37:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:37:55.359 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:37:55.359+0000 7fb991a09640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6213:44 nref=7 cr=0x5634baf36700)) (dversion lock w=1 last_client=6213) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5634bae82000] 2024-04-26T21:37:55.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:37:55.367 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:37:55.368 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:37:55.385 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:37:55.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.527+0000 7fa366eda640 1 Processor -- start 2024-04-26T21:37:55.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.527+0000 7fa366eda640 1 -- start start 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa366eda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa3600d1b30 0x7fa3600d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa366eda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3600d10d0 con 0x7fa360162810 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa366eda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3600d1370 con 0x7fa3600b2340 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa366eda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa3600d0750 con 0x7fa3600d1b30 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa364c76640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa360162810 0x7fa360070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53564/0 (socket says 172.21.15.40:53564) 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa364c76640 1 -- 172.21.15.40:0/3386608748 learned_addr learned my addr 172.21.15.40:0/3386608748 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2546067621 0 0) 0x7fa3600d10d0 con 0x7fa360162810 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa348003520 con 0x7fa360162810 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.528+0000 7fa365477640 1 --2- 172.21.15.40:0/3386608748 >> v2:172.21.15.184:3300/0 conn(0x7fa3600d1b30 0x7fa3600d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3582671722 0 0) 0x7fa348003520 con 0x7fa360162810 2024-04-26T21:37:55.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 >> v2:172.21.15.184:3300/0 conn(0x7fa3600d1b30 msgr2=0x7fa3600d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 --2- 172.21.15.40:0/3386608748 >> v2:172.21.15.184:3300/0 conn(0x7fa3600d1b30 0x7fa3600d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 >> v1:172.21.15.184:6789/0 conn(0x7fa3600b2340 legacy=0x7fa3600cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3600d01b0 con 0x7fa360162810 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fa350001670 con 0x7fa360162810 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 >> v1:172.21.15.40:6789/0 conn(0x7fa360162810 legacy=0x7fa360070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3600d0750 con 0x7fa3480039b0 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa34800c110 con 0x7fa348004800 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa34800c310 con 0x7fa3480051c0 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa364c76640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa35ffff640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa365477640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa364c76640 1 -- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480051c0 msgr2=0x7fa34800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa364c76640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa364c76640 1 -- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa348004800 msgr2=0x7fa348004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa364c76640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa364c76640 1 -- 172.21.15.40:0/3386608748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa348003520 con 0x7fa3480039b0 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa365477640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.529+0000 7fa365477640 1 -- 172.21.15.40:0/3386608748 reap_dead start 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.530+0000 7fa364c76640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa35000c240 tx=0x7fa348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c945bf09deeeb5d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.530+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350007d10 con 0x7fa3480039b0 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.530+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3500058d0 con 0x7fa3480039b0 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.530+0000 7fa35f7fe640 1 -- 172.21.15.40:0/3386608748 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350005d40 con 0x7fa3480039b0 2024-04-26T21:37:55.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.530+0000 7fa366eda640 1 -- 172.21.15.40:0/3386608748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3480039b0 msgr2=0x7fa348003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.530+0000 7fa366eda640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa35000c240 tx=0x7fa348005830 comp rx=0 tx=0).stop 2024-04-26T21:37:55.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.530+0000 7fa366eda640 1 -- 172.21.15.40:0/3386608748 shutdown_connections 2024-04-26T21:37:55.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.530+0000 7fa366eda640 1 --2- 172.21.15.40:0/3386608748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.531+0000 7fa366eda640 1 -- 172.21.15.40:0/3386608748 >> 172.21.15.40:0/3386608748 conn(0x7fa3600d2da0 msgr2=0x7fa3600d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:55.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.531+0000 7fa366eda640 1 -- 172.21.15.40:0/3386608748 shutdown_connections 2024-04-26T21:37:55.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.531+0000 7fa366eda640 1 -- 172.21.15.40:0/3386608748 wait complete. 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.532+0000 7fa366eda640 1 Processor -- start 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.532+0000 7fa366eda640 1 -- start start 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa366eda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480039b0 0x7fa3601f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa366eda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3601f64f0 0x7fa3601f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa366eda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3601f58b0 0x7fa3600c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa366eda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3600d1fc0 con 0x7fa3601f58b0 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa366eda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3600d16a0 con 0x7fa3480039b0 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa366eda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa3600d2140 con 0x7fa3601f64f0 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3601f58b0 0x7fa3600c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3601f58b0 0x7fa3600c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52822/0 (socket says 172.21.15.40:52822) 2024-04-26T21:37:55.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 -- 172.21.15.40:0/1867021309 learned_addr learned my addr 172.21.15.40:0/1867021309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa35ffff640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3601f64f0 0x7fa3601f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa364c76640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480039b0 0x7fa3601f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 -- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3601f64f0 msgr2=0x7fa3601f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3601f64f0 0x7fa3601f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 -- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480039b0 msgr2=0x7fa3601f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480039b0 0x7fa3601f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 -- 172.21.15.40:0/1867021309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3601f5e80 con 0x7fa3601f58b0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa365477640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3601f58b0 0x7fa3600c8770 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa360073630 tx=0x7fa358004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa35d7fa640 1 -- 172.21.15.40:0/1867021309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358007cb0 con 0x7fa3601f58b0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa35d7fa640 1 -- 172.21.15.40:0/1867021309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa358019070 con 0x7fa3601f58b0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3601f6160 con 0x7fa3601f58b0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa35d7fa640 1 -- 172.21.15.40:0/1867021309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358014650 con 0x7fa3601f58b0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.533+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa360158c90 con 0x7fa3601f58b0 2024-04-26T21:37:55.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.534+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3600694f0 con 0x7fa3601f58b0 2024-04-26T21:37:55.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.535+0000 7fa35d7fa640 1 -- 172.21.15.40:0/1867021309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3580147f0 con 0x7fa3601f58b0 2024-04-26T21:37:55.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.535+0000 7fa35d7fa640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa34c06c8f0 0x7fa34c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.536+0000 7fa35d7fa640 1 -- 172.21.15.40:0/1867021309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6159+0+0 (secure 0 0 0) 0x7fa35808c390 con 0x7fa3601f58b0 2024-04-26T21:37:55.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.536+0000 7fa364c76640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa34c06c8f0 0x7fa34c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.536+0000 7fa364c76640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa34c06c8f0 0x7fa34c06edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa350033000 tx=0x7fa350035040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:55.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.540+0000 7fa35d7fa640 1 -- 172.21.15.40:0/1867021309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa358023020 con 0x7fa3601f58b0 2024-04-26T21:37:55.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.729+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa36006dbf0 con 0x7fa3601f58b0 2024-04-26T21:37:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.730+0000 7fa35d7fa640 1 -- 172.21.15.40:0/1867021309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+17001 (secure 0 0 0) 0x7fa35805a690 con 0x7fa3601f58b0 2024-04-26T21:37:55.731 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:55.731 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":54,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:52.285426+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:37:35.784073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T21:37:36.460613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:37:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa34c06c8f0 msgr2=0x7fa34c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa34c06c8f0 0x7fa34c06edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa350033000 tx=0x7fa350035040 comp rx=0 tx=0).stop 2024-04-26T21:37:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3601f58b0 msgr2=0x7fa3600c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3601f58b0 0x7fa3600c8770 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa360073630 tx=0x7fa358004840 comp rx=0 tx=0).stop 2024-04-26T21:37:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 shutdown_connections 2024-04-26T21:37:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa34c06c8f0 0x7fa34c06edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3601f58b0 0x7fa3600c8770 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3601f64f0 0x7fa3601f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 --2- 172.21.15.40:0/1867021309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480039b0 0x7fa3601f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 >> 172.21.15.40:0/1867021309 conn(0x7fa3600d2da0 msgr2=0x7fa3600cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 shutdown_connections 2024-04-26T21:37:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.732+0000 7fa366eda640 1 -- 172.21.15.40:0/1867021309 wait complete. 2024-04-26T21:37:55.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:37:35.784073+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:37:36.460613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:37:55.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:37:55.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.847+0000 7f9926a52640 1 Processor -- start 2024-04-26T21:37:55.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.848+0000 7f9926a52640 1 -- start start 2024-04-26T21:37:55.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.848+0000 7f9926a52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200cfde0 0x7f99200d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.848+0000 7f9926a52640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9920070350 0x7f9920070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.848+0000 7f9926a52640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99200d2390 con 0x7f9920071090 2024-04-26T21:37:55.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.848+0000 7f9926a52640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9920071500 con 0x7f9920070350 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.848+0000 7f9926a52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99200afab0 con 0x7f99200cfde0 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.848+0000 7f9924fef640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9920071090 0x7f992015e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53568/0 (socket says 172.21.15.40:53568) 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f9924fef640 1 -- 172.21.15.40:0/2647323321 learned_addr learned my addr 172.21.15.40:0/2647323321 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991ffff640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200cfde0 0x7f99200d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1145050979 0 0) 0x7f99200d2390 con 0x7f9920071090 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991f7fe640 1 --2- 172.21.15.40:0/2647323321 >> v2:172.21.15.184:3300/0 conn(0x7f9920070350 0x7f9920070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9908003520 con 0x7f9920071090 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2921718747 0 0) 0x7f9908003520 con 0x7f9920071090 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200cfde0 msgr2=0x7f99200d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200cfde0 0x7f99200d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 >> v2:172.21.15.184:3300/0 conn(0x7f9920070350 msgr2=0x7f9920070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 --2- 172.21.15.40:0/2647323321 >> v2:172.21.15.184:3300/0 conn(0x7f9920070350 0x7f9920070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992006d3f0 con 0x7f9920071090 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f99180019f0 con 0x7f9920071090 2024-04-26T21:37:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 >> v1:172.21.15.40:6789/0 conn(0x7f9920071090 legacy=0x7f992015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99080039b0 0x7f9908003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200cfde0 0x7f9908004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9908004d40 0x7f99080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9920071500 con 0x7f9908004d40 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f990800b9c0 con 0x7f99080039b0 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f990800bb40 con 0x7f99200cfde0 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f9924fef640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9908004d40 0x7f99080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991ffff640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99080039b0 0x7f9908003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.849+0000 7f991f7fe640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200cfde0 0x7f9908004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f9924fef640 1 -- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200cfde0 msgr2=0x7f9908004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f9924fef640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200cfde0 0x7f9908004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f9924fef640 1 -- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99080039b0 msgr2=0x7f9908003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f9924fef640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99080039b0 0x7f9908003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f9924fef640 1 -- 172.21.15.40:0/2647323321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9908003520 con 0x7f9908004d40 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f9924fef640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9908004d40 0x7f99080051c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f991800a8a0 tx=0x7f9908005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9611fd2e15c41ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991800a4c0 con 0x7f9908004d40 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991802f070 con 0x7f9908004d40 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.850+0000 7f991effd640 1 -- 172.21.15.40:0/2647323321 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991802bd30 con 0x7f9908004d40 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.851+0000 7f9926a52640 1 -- 172.21.15.40:0/2647323321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9908004d40 msgr2=0x7f99080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.851+0000 7f9926a52640 1 --2- 172.21.15.40:0/2647323321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9908004d40 0x7f99080051c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f991800a8a0 tx=0x7f9908005830 comp rx=0 tx=0).stop 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.851+0000 7f9924fef640 1 -- 172.21.15.40:0/2647323321 reap_dead start 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.851+0000 7f9926a52640 1 -- 172.21.15.40:0/2647323321 shutdown_connections 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.851+0000 7f9926a52640 1 -- 172.21.15.40:0/2647323321 >> 172.21.15.40:0/2647323321 conn(0x7f99200d2d80 msgr2=0x7f99200d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.851+0000 7f9926a52640 1 -- 172.21.15.40:0/2647323321 shutdown_connections 2024-04-26T21:37:55.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.851+0000 7f9926a52640 1 -- 172.21.15.40:0/2647323321 wait complete. 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.852+0000 7f9926a52640 1 Processor -- start 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.852+0000 7f9926a52640 1 -- start start 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f9926a52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200c9620 0x7f992009f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f9926a52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99200c9a00 0x7f992009cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f9926a52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992009d170 0x7f99200b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f9926a52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f992006d9b0 con 0x7f99200c9a00 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f9926a52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f992006dd20 con 0x7f992009d170 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f9926a52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99200d1b60 con 0x7f99200c9620 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f991f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99200c9a00 0x7f992009cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f991f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99200c9a00 0x7f992009cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52836/0 (socket says 172.21.15.40:52836) 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f991f7fe640 1 -- 172.21.15.40:0/589421362 learned_addr learned my addr 172.21.15.40:0/589421362 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f991f7fe640 1 -- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200c9620 msgr2=0x7f992009f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f991f7fe640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200c9620 0x7f992009f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f991f7fe640 1 -- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992009d170 msgr2=0x7f99200b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.853+0000 7f991f7fe640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992009d170 0x7f99200b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.854+0000 7f991f7fe640 1 -- 172.21.15.40:0/589421362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992009fc20 con 0x7f99200c9a00 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.854+0000 7f991f7fe640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99200c9a00 0x7f992009cc30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9914011a70 tx=0x7f9914011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.854+0000 7f991cff9640 1 -- 172.21.15.40:0/589421362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991400fad0 con 0x7f99200c9a00 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.854+0000 7f991cff9640 1 -- 172.21.15.40:0/589421362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991400fc70 con 0x7f99200c9a00 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.854+0000 7f991cff9640 1 -- 172.21.15.40:0/589421362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914014440 con 0x7f99200c9a00 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.854+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99201f51d0 con 0x7f99200c9a00 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.854+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201f5790 con 0x7f99200c9a00 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.855+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99200c68b0 con 0x7f99200c9a00 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.855+0000 7f991cff9640 1 -- 172.21.15.40:0/589421362 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9914012040 con 0x7f99200c9a00 2024-04-26T21:37:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.856+0000 7f991cff9640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990c06c8f0 0x7f990c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.856+0000 7f991cff9640 1 -- 172.21.15.40:0/589421362 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6159+0+0 (secure 0 0 0) 0x7f991408c370 con 0x7f99200c9a00 2024-04-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.856+0000 7f991ffff640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990c06c8f0 0x7f990c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.856+0000 7f991ffff640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990c06c8f0 0x7f990c06edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f99200b8860 tx=0x7f9910009320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:55.860+0000 7f991cff9640 1 -- 172.21.15.40:0/589421362 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f992009fc20 con 0x7f99200c9a00 2024-04-26T21:37:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.045+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99201e6e70 con 0x7f99200c9a00 2024-04-26T21:37:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.045+0000 7f991cff9640 1 -- 172.21.15.40:0/589421362 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f9914019030 con 0x7f99200c9a00 2024-04-26T21:37:56.046 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:37:56.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990c06c8f0 msgr2=0x7f990c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990c06c8f0 0x7f990c06edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f99200b8860 tx=0x7f9910009320 comp rx=0 tx=0).stop 2024-04-26T21:37:56.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99200c9a00 msgr2=0x7f992009cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99200c9a00 0x7f992009cc30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9914011a70 tx=0x7f9914011f30 comp rx=0 tx=0).stop 2024-04-26T21:37:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 shutdown_connections 2024-04-26T21:37:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990c06c8f0 0x7f990c06edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f992009d170 0x7f99200b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99200c9a00 0x7f992009cc30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 --2- 172.21.15.40:0/589421362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99200c9620 0x7f992009f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 >> 172.21.15.40:0/589421362 conn(0x7f99200d2d80 msgr2=0x7f99200afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 shutdown_connections 2024-04-26T21:37:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.047+0000 7f9926a52640 1 -- 172.21.15.40:0/589421362 wait complete. 2024-04-26T21:37:56.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:37:56.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.162+0000 7f8639283640 1 Processor -- start 2024-04-26T21:37:56.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.162+0000 7f8639283640 1 -- start start 2024-04-26T21:37:56.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f8639283640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340d1a00 0x7f86340d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f8639283640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86340b40d0 0x7f86340b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f8639283640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86340d0a10 con 0x7f86340d2320 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f8639283640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f86340d0cf0 con 0x7f86340b40d0 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f8639283640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86340d0fd0 con 0x7f86340d1a00 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f86337fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86340b40d0 0x7f86340b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f86327fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f86340d2320 0x7f86341625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53576/0 (socket says 172.21.15.40:53576) 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f86327fc640 1 -- 172.21.15.40:0/1440027955 learned_addr learned my addr 172.21.15.40:0/1440027955 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.163+0000 7f8632ffd640 1 --2- 172.21.15.40:0/1440027955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340d1a00 0x7f86340d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.164+0000 7f86337fe640 1 -- 172.21.15.40:0/1440027955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340d1a00 msgr2=0x7f86340d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.164+0000 7f86337fe640 1 --2- 172.21.15.40:0/1440027955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340d1a00 0x7f86340d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.164+0000 7f86337fe640 1 -- 172.21.15.40:0/1440027955 >> v1:172.21.15.40:6789/0 conn(0x7f86340d2320 legacy=0x7f86341625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.164+0000 7f86337fe640 1 -- 172.21.15.40:0/1440027955 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86340d40a0 con 0x7f86340b40d0 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.164+0000 7f86337fe640 1 --2- 172.21.15.40:0/1440027955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340b40d0 0x7f86340b4b00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8624009d90 tx=0x7f862402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc0c713f59b341d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8631ffb640 1 -- 172.21.15.40:0/1440027955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624031600 con 0x7f86340b40d0 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8631ffb640 1 -- 172.21.15.40:0/1440027955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86240317a0 con 0x7f86340b40d0 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8631ffb640 1 -- 172.21.15.40:0/1440027955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624031b70 con 0x7f86340b40d0 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8639283640 1 -- 172.21.15.40:0/1440027955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340b40d0 msgr2=0x7f86340b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8639283640 1 --2- 172.21.15.40:0/1440027955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340b40d0 0x7f86340b4b00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8624009d90 tx=0x7f862402a5d0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8639283640 1 -- 172.21.15.40:0/1440027955 shutdown_connections 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8639283640 1 --2- 172.21.15.40:0/1440027955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340b40d0 0x7f86340b4b00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8639283640 1 --2- 172.21.15.40:0/1440027955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340d1a00 0x7f86340d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8639283640 1 -- 172.21.15.40:0/1440027955 >> 172.21.15.40:0/1440027955 conn(0x7f86340d4fd0 msgr2=0x7f86340d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:56.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8639283640 1 -- 172.21.15.40:0/1440027955 shutdown_connections 2024-04-26T21:37:56.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.165+0000 7f8639283640 1 -- 172.21.15.40:0/1440027955 wait complete. 2024-04-26T21:37:56.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f8639283640 1 Processor -- start 2024-04-26T21:37:56.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f8639283640 1 -- start start 2024-04-26T21:37:56.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f8639283640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340b40d0 0x7f863415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f8639283640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340d1a00 0x7f86340c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f8639283640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86340d2320 0x7f86340b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f8639283640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86340b5040 con 0x7f86340d1a00 2024-04-26T21:37:56.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f8639283640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8634162f50 con 0x7f86340d2320 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f8639283640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86340b51c0 con 0x7f86340b40d0 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f86327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340d1a00 0x7f86340c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.167+0000 7f86327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340d1a00 0x7f86340c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52864/0 (socket says 172.21.15.40:52864) 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86327fc640 1 -- 172.21.15.40:0/2854339275 learned_addr learned my addr 172.21.15.40:0/2854339275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f8632ffd640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340b40d0 0x7f863415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86337fe640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86340d2320 0x7f86340b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86327fc640 1 -- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340b40d0 msgr2=0x7f863415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86327fc640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340b40d0 0x7f863415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86327fc640 1 -- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86340d2320 msgr2=0x7f86340b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86327fc640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86340d2320 0x7f86340b8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86327fc640 1 -- 172.21.15.40:0/2854339275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86340ba620 con 0x7f86340d1a00 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86337fe640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86340d2320 0x7f86340b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f86327fc640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340d1a00 0x7f86340c9340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c3c0 tx=0x7f861c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f862b7fe640 1 -- 172.21.15.40:0/2854339275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c017070 con 0x7f86340d1a00 2024-04-26T21:37:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f862b7fe640 1 -- 172.21.15.40:0/2854339275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c005ce0 con 0x7f86340d1a00 2024-04-26T21:37:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86340b9880 con 0x7f86340d1a00 2024-04-26T21:37:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f862b7fe640 1 -- 172.21.15.40:0/2854339275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c0072f0 con 0x7f86340d1a00 2024-04-26T21:37:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.168+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86340cc380 con 0x7f86340d1a00 2024-04-26T21:37:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.169+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86340694f0 con 0x7f86340d1a00 2024-04-26T21:37:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.170+0000 7f862b7fe640 1 -- 172.21.15.40:0/2854339275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f861c01a070 con 0x7f86340d1a00 2024-04-26T21:37:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.170+0000 7f862b7fe640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f861406c8f0 0x7f861406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.170+0000 7f862b7fe640 1 -- 172.21.15.40:0/2854339275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6159+0+0 (secure 0 0 0) 0x7f861c08afb0 con 0x7f86340d1a00 2024-04-26T21:37:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.170+0000 7f8632ffd640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f861406c8f0 0x7f861406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.171+0000 7f8632ffd640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f861406c8f0 0x7f861406edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f862c009b40 tx=0x7f862c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.175+0000 7f862b7fe640 1 -- 172.21.15.40:0/2854339275 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86340b9880 con 0x7f86340d1a00 2024-04-26T21:37:56.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.361+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f86340b1fe0 con 0x7f86340d1a00 2024-04-26T21:37:56.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.362+0000 7f862b7fe640 1 -- 172.21.15.40:0/2854339275 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7f86340ba620 con 0x7f86340d1a00 2024-04-26T21:37:56.363 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f861406c8f0 msgr2=0x7f861406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f861406c8f0 0x7f861406edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f862c009b40 tx=0x7f862c009190 comp rx=0 tx=0).stop 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340d1a00 msgr2=0x7f86340c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340d1a00 0x7f86340c9340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c3c0 tx=0x7f861c00c880 comp rx=0 tx=0).stop 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 shutdown_connections 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f861406c8f0 0x7f861406edb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86340d2320 0x7f86340b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86340d1a00 0x7f86340c9340 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 --2- 172.21.15.40:0/2854339275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86340b40d0 0x7f863415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 >> 172.21.15.40:0/2854339275 conn(0x7f86340d4fd0 msgr2=0x7f863406e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:56.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 shutdown_connections 2024-04-26T21:37:56.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.364+0000 7f8639283640 1 -- 172.21.15.40:0/2854339275 wait complete. 2024-04-26T21:37:56.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:37:56.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.478+0000 7f602d9f0640 1 Processor -- start 2024-04-26T21:37:56.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.478+0000 7f602d9f0640 1 -- start start 2024-04-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f602d9f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f602815f550 0x7f60280d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f602d9f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60280d1040 con 0x7f60280d3bc0 2024-04-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f602d9f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60280c9680 con 0x7f602815f170 2024-04-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f602d9f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f60280c9430 con 0x7f602815f550 2024-04-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f601ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f602815f550 0x7f60280d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f60277fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f60280d3bc0 0x7f60280ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53578/0 (socket says 172.21.15.40:53578) 2024-04-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f60277fe640 1 -- 172.21.15.40:0/2622032060 learned_addr learned my addr 172.21.15.40:0/2622032060 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f601ffff640 1 -- 172.21.15.40:0/2622032060 >> v1:172.21.15.184:6789/0 conn(0x7f602815f170 legacy=0x7f60280d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f601ffff640 1 -- 172.21.15.40:0/2622032060 >> v1:172.21.15.40:6789/0 conn(0x7f60280d3bc0 legacy=0x7f60280ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f60267fc640 1 -- 172.21.15.40:0/2622032060 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1270771861 0 0) 0x7f6020001490 con 0x7f60280d3bc0 2024-04-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.479+0000 7f601ffff640 1 -- 172.21.15.40:0/2622032060 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60280c9960 con 0x7f602815f550 2024-04-26T21:37:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.480+0000 7f601ffff640 1 --2- 172.21.15.40:0/2622032060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f602815f550 0x7f60280d5a80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f60280c95b0 tx=0x7f601002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9460637c3a278c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.480+0000 7f60267fc640 1 -- 172.21.15.40:0/2622032060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010037070 con 0x7f602815f550 2024-04-26T21:37:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.480+0000 7f60267fc640 1 -- 172.21.15.40:0/2622032060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6010032440 con 0x7f602815f550 2024-04-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.480+0000 7f60267fc640 1 -- 172.21.15.40:0/2622032060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010032810 con 0x7f602815f550 2024-04-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.481+0000 7f602d9f0640 1 -- 172.21.15.40:0/2622032060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f602815f550 msgr2=0x7f60280d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.481+0000 7f602d9f0640 1 --2- 172.21.15.40:0/2622032060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f602815f550 0x7f60280d5a80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f60280c95b0 tx=0x7f601002a250 comp rx=0 tx=0).stop 2024-04-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.481+0000 7f602d9f0640 1 -- 172.21.15.40:0/2622032060 shutdown_connections 2024-04-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.481+0000 7f602d9f0640 1 --2- 172.21.15.40:0/2622032060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f602815f550 0x7f60280d5a80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.481+0000 7f602d9f0640 1 -- 172.21.15.40:0/2622032060 >> 172.21.15.40:0/2622032060 conn(0x7f60280d4f60 msgr2=0x7f60280d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.481+0000 7f602d9f0640 1 -- 172.21.15.40:0/2622032060 shutdown_connections 2024-04-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.481+0000 7f602d9f0640 1 -- 172.21.15.40:0/2622032060 wait complete. 2024-04-26T21:37:56.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.482+0000 7f602d9f0640 1 Processor -- start 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.482+0000 7f602d9f0640 1 -- start start 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f602d9f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60280d3bc0 0x7f60280a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f602d9f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602815f170 0x7f602809fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f602d9f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60280a00c0 0x7f6028159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f602d9f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f60280d2e30 con 0x7f60280d3bc0 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f602d9f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60280d2cb0 con 0x7f602815f170 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f602d9f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60280d2fb0 con 0x7f60280a00c0 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60280d3bc0 0x7f60280a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60280d3bc0 0x7f60280a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52892/0 (socket says 172.21.15.40:52892) 2024-04-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 -- 172.21.15.40:0/981752467 learned_addr learned my addr 172.21.15.40:0/981752467 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f60277fe640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60280a00c0 0x7f6028159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f601ffff640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602815f170 0x7f602809fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 -- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60280a00c0 msgr2=0x7f6028159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60280a00c0 0x7f6028159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 -- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602815f170 msgr2=0x7f602809fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602815f170 0x7f602809fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 -- 172.21.15.40:0/981752467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6028158e40 con 0x7f60280d3bc0 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f60277fe640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60280a00c0 0x7f6028159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f601ffff640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602815f170 0x7f602809fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6026ffd640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60280d3bc0 0x7f60280a1970 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f601800c480 tx=0x7f601800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6024ff9640 1 -- 172.21.15.40:0/981752467 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180052d0 con 0x7f60280d3bc0 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6024ff9640 1 -- 172.21.15.40:0/981752467 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6018005470 con 0x7f60280d3bc0 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60280a08d0 con 0x7f60280d3bc0 2024-04-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f6024ff9640 1 -- 172.21.15.40:0/981752467 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018012870 con 0x7f60280d3bc0 2024-04-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.483+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60280a0e90 con 0x7f60280d3bc0 2024-04-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.484+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60280694f0 con 0x7f60280d3bc0 2024-04-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.485+0000 7f6024ff9640 1 -- 172.21.15.40:0/981752467 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6018012a10 con 0x7f60280d3bc0 2024-04-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.485+0000 7f6024ff9640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f600806c920 0x7f600806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.485+0000 7f6024ff9640 1 -- 172.21.15.40:0/981752467 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6159+0+0 (secure 0 0 0) 0x7f601808b910 con 0x7f60280d3bc0 2024-04-26T21:37:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.485+0000 7f601ffff640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f600806c920 0x7f600806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.486+0000 7f601ffff640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f600806c920 0x7f600806ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6010034000 tx=0x7f601002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.489+0000 7f6024ff9640 1 -- 172.21.15.40:0/981752467 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60180595c0 con 0x7f60280d3bc0 2024-04-26T21:37:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.676+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6028069cd0 con 0x7f60280d3bc0 2024-04-26T21:37:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.676+0000 7f6024ff9640 1 -- 172.21.15.40:0/981752467 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7f601805d8c0 con 0x7f60280d3bc0 2024-04-26T21:37:56.677 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f600806c920 msgr2=0x7f600806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f600806c920 0x7f600806ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6010034000 tx=0x7f601002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60280d3bc0 msgr2=0x7f60280a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60280d3bc0 0x7f60280a1970 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f601800c480 tx=0x7f601800c940 comp rx=0 tx=0).stop 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 shutdown_connections 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f600806c920 0x7f600806ede0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60280a00c0 0x7f6028159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602815f170 0x7f602809fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 --2- 172.21.15.40:0/981752467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60280d3bc0 0x7f60280a1970 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 >> 172.21.15.40:0/981752467 conn(0x7f60280d4f60 msgr2=0x7f60280d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:56.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 shutdown_connections 2024-04-26T21:37:56.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.679+0000 7f602d9f0640 1 -- 172.21.15.40:0/981752467 wait complete. 2024-04-26T21:37:56.691 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:37:56.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.791+0000 7f0faeb78640 1 Processor -- start 2024-04-26T21:37:56.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.791+0000 7f0faeb78640 1 -- start start 2024-04-26T21:37:56.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0faeb78640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0fa8162810 0x7f0fa8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0faeb78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80b2340 0x7f0fa80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0faeb78640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa80d10d0 con 0x7f0fa80cd930 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0faeb78640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa80d0750 con 0x7f0fa8162810 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0faeb78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa80d16a0 con 0x7f0fa80b2340 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0fa7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80b2340 0x7f0fa80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0fac914640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0fa8162810 0x7f0fa8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0fad115640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f0fa80cd930 0x7f0fa80d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:34536/0 (socket says 172.21.15.40:34536) 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0fad115640 1 -- 172.21.15.40:0/60907063 learned_addr learned my addr 172.21.15.40:0/60907063 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0fac914640 1 -- 172.21.15.40:0/60907063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80b2340 msgr2=0x7f0fa80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0fac914640 1 --2- 172.21.15.40:0/60907063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80b2340 0x7f0fa80cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0fac914640 1 -- 172.21.15.40:0/60907063 >> v1:172.21.15.184:6789/0 conn(0x7f0fa80cd930 legacy=0x7f0fa80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.793+0000 7f0fac914640 1 -- 172.21.15.40:0/60907063 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa80d1370 con 0x7f0fa8162810 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.792+0000 7f0fa77fe640 1 -- 172.21.15.40:0/60907063 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3551161981 0 0) 0x7f0fa80d10d0 con 0x7f0fa80cd930 2024-04-26T21:37:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.793+0000 7f0fa7fff640 1 --2- 172.21.15.40:0/60907063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80b2340 0x7f0fa80cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.793+0000 7f0fac914640 1 --2- 172.21.15.40:0/60907063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8162810 0x7f0fa8070350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0097a0 tx=0x7f0f9c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba34f7c26e1af1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.793+0000 7f0fa77fe640 1 -- 172.21.15.40:0/60907063 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c037070 con 0x7f0fa8162810 2024-04-26T21:37:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.793+0000 7f0fa77fe640 1 -- 172.21.15.40:0/60907063 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9c032440 con 0x7f0fa8162810 2024-04-26T21:37:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.793+0000 7f0fa77fe640 1 -- 172.21.15.40:0/60907063 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c032810 con 0x7f0fa8162810 2024-04-26T21:37:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.794+0000 7f0faeb78640 1 -- 172.21.15.40:0/60907063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8162810 msgr2=0x7f0fa8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.794+0000 7f0faeb78640 1 --2- 172.21.15.40:0/60907063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8162810 0x7f0fa8070350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0097a0 tx=0x7f0f9c02a180 comp rx=0 tx=0).stop 2024-04-26T21:37:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.794+0000 7f0faeb78640 1 -- 172.21.15.40:0/60907063 shutdown_connections 2024-04-26T21:37:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.794+0000 7f0faeb78640 1 --2- 172.21.15.40:0/60907063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80b2340 0x7f0fa80cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.794+0000 7f0faeb78640 1 --2- 172.21.15.40:0/60907063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8162810 0x7f0fa8070350 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.794+0000 7f0faeb78640 1 -- 172.21.15.40:0/60907063 >> 172.21.15.40:0/60907063 conn(0x7f0fa80d2da0 msgr2=0x7f0fa80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.794+0000 7f0faeb78640 1 -- 172.21.15.40:0/60907063 shutdown_connections 2024-04-26T21:37:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.794+0000 7f0faeb78640 1 -- 172.21.15.40:0/60907063 wait complete. 2024-04-26T21:37:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.795+0000 7f0faeb78640 1 Processor -- start 2024-04-26T21:37:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.795+0000 7f0faeb78640 1 -- start start 2024-04-26T21:37:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0faeb78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80b2340 0x7f0fa8149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0faeb78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80cd930 0x7f0fa814c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0faeb78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa8162810 0x7f0fa814a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0faeb78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8070890 con 0x7f0fa8162810 2024-04-26T21:37:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0faeb78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa80d0470 con 0x7f0fa80b2340 2024-04-26T21:37:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0faeb78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa80d2240 con 0x7f0fa80cd930 2024-04-26T21:37:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0fad115640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa8162810 0x7f0fa814a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0fad115640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa8162810 0x7f0fa814a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52902/0 (socket says 172.21.15.40:52902) 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0fad115640 1 -- 172.21.15.40:0/2492437630 learned_addr learned my addr 172.21.15.40:0/2492437630 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0fa7fff640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80cd930 0x7f0fa814c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.796+0000 7f0fac914640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80b2340 0x7f0fa8149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fad115640 1 -- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80cd930 msgr2=0x7f0fa814c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fad115640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80cd930 0x7f0fa814c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fad115640 1 -- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80b2340 msgr2=0x7f0fa8149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fad115640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80b2340 0x7f0fa8149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fad115640 1 -- 172.21.15.40:0/2492437630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa814a070 con 0x7f0fa8162810 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fac914640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80b2340 0x7f0fa8149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fad115640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa8162810 0x7f0fa814a380 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000c760 tx=0x7f0fa000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fa57fa640 1 -- 172.21.15.40:0/2492437630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0018070 con 0x7f0fa8162810 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fa57fa640 1 -- 172.21.15.40:0/2492437630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa000a640 con 0x7f0fa8162810 2024-04-26T21:37:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0fa57fa640 1 -- 172.21.15.40:0/2492437630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa000aa30 con 0x7f0fa8162810 2024-04-26T21:37:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa814ccf0 con 0x7f0fa8162810 2024-04-26T21:37:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.797+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa814d2b0 con 0x7f0fa8162810 2024-04-26T21:37:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.798+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa80694f0 con 0x7f0fa8162810 2024-04-26T21:37:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.799+0000 7f0fa57fa640 1 -- 172.21.15.40:0/2492437630 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fa001b070 con 0x7f0fa8162810 2024-04-26T21:37:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.799+0000 7f0fa57fa640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.799+0000 7f0fa57fa640 1 -- 172.21.15.40:0/2492437630 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6159+0+0 (secure 0 0 0) 0x7f0fa008b260 con 0x7f0fa8162810 2024-04-26T21:37:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.799+0000 7f0fac914640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.800+0000 7f0fac914640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c034000 tx=0x7f0f9c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:56.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:56.804+0000 7f0fa57fa640 1 -- 172.21.15.40:0/2492437630 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0fa0058dd0 con 0x7f0fa8162810 2024-04-26T21:37:57.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.004+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0fa80ced10 con 0x7f0fa8162810 2024-04-26T21:37:57.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.004+0000 7f0fa57fa640 1 -- 172.21.15.40:0/2492437630 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7f0fa005d0d0 con 0x7f0fa8162810 2024-04-26T21:37:57.005 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f8c06c8f0 msgr2=0x7f0f8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c034000 tx=0x7f0f9c02e000 comp rx=0 tx=0).stop 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa8162810 msgr2=0x7f0fa814a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa8162810 0x7f0fa814a380 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000c760 tx=0x7f0fa000cc20 comp rx=0 tx=0).stop 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 shutdown_connections 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa8162810 0x7f0fa814a380 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fa80cd930 0x7f0fa814c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 --2- 172.21.15.40:0/2492437630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80b2340 0x7f0fa8149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 >> 172.21.15.40:0/2492437630 conn(0x7f0fa80d2da0 msgr2=0x7f0fa80cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 shutdown_connections 2024-04-26T21:37:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.006+0000 7f0faeb78640 1 -- 172.21.15.40:0/2492437630 wait complete. 2024-04-26T21:37:57.018 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:37:57.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:57.018 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:37:57.048 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:37:57.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:37:57.048 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:37:57.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.243+0000 7fed11808640 1 Processor -- start 2024-04-26T21:37:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.243+0000 7fed11808640 1 -- start start 2024-04-26T21:37:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.244+0000 7fed11808640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fed0c0d06f0 0x7fed0c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.244+0000 7fed11808640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed0c0b2850 0x7fed0c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.244+0000 7fed11808640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed0c0d0ba0 con 0x7fed0c15f3a0 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.244+0000 7fed11808640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fed0c0d0e20 con 0x7fed0c0b2850 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.244+0000 7fed11808640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fed0c0b1d70 con 0x7fed0c0d06f0 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.244+0000 7fed0b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed0c0b2850 0x7fed0c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.244+0000 7fed0affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fed0c15f3a0 0x7fed0c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53588/0 (socket says 172.21.15.40:53588) 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.244+0000 7fed0affd640 1 -- 172.21.15.40:0/709822896 learned_addr learned my addr 172.21.15.40:0/709822896 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.245+0000 7fed0a7fc640 1 --2- 172.21.15.40:0/709822896 >> v2:172.21.15.184:3300/0 conn(0x7fed0c0d06f0 0x7fed0c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.245+0000 7fed0b7fe640 1 -- 172.21.15.40:0/709822896 >> v2:172.21.15.184:3300/0 conn(0x7fed0c0d06f0 msgr2=0x7fed0c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.245+0000 7fed0b7fe640 1 --2- 172.21.15.40:0/709822896 >> v2:172.21.15.184:3300/0 conn(0x7fed0c0d06f0 0x7fed0c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.245+0000 7fed0b7fe640 1 -- 172.21.15.40:0/709822896 >> v1:172.21.15.40:6789/0 conn(0x7fed0c15f3a0 legacy=0x7fed0c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.245+0000 7fed09ffb640 1 -- 172.21.15.40:0/709822896 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763221866 0 0) 0x7fed04001110 con 0x7fed0c15f3a0 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.245+0000 7fed0b7fe640 1 -- 172.21.15.40:0/709822896 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c0b2050 con 0x7fed0c0b2850 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.245+0000 7fed0a7fc640 1 --2- 172.21.15.40:0/709822896 >> v2:172.21.15.184:3300/0 conn(0x7fed0c0d06f0 0x7fed0c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.245+0000 7fed0b7fe640 1 --2- 172.21.15.40:0/709822896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c0b2850 0x7fed0c0b2c80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0b7b80 tx=0x7fecfc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f6ef0701b50686 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed09ffb640 1 -- 172.21.15.40:0/709822896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc007740 con 0x7fed0c0b2850 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed09ffb640 1 -- 172.21.15.40:0/709822896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc0078e0 con 0x7fed0c0b2850 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed09ffb640 1 -- 172.21.15.40:0/709822896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc007cb0 con 0x7fed0c0b2850 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed11808640 1 -- 172.21.15.40:0/709822896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c0b2850 msgr2=0x7fed0c0b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed11808640 1 --2- 172.21.15.40:0/709822896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c0b2850 0x7fed0c0b2c80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0b7b80 tx=0x7fecfc02a680 comp rx=0 tx=0).stop 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed11808640 1 -- 172.21.15.40:0/709822896 shutdown_connections 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed11808640 1 --2- 172.21.15.40:0/709822896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c0b2850 0x7fed0c0b2c80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed11808640 1 --2- 172.21.15.40:0/709822896 >> v2:172.21.15.184:3300/0 conn(0x7fed0c0d06f0 0x7fed0c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed11808640 1 -- 172.21.15.40:0/709822896 >> 172.21.15.40:0/709822896 conn(0x7fed0c0d4f90 msgr2=0x7fed0c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed11808640 1 -- 172.21.15.40:0/709822896 shutdown_connections 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.246+0000 7fed11808640 1 -- 172.21.15.40:0/709822896 wait complete. 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.247+0000 7fed11808640 1 Processor -- start 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed11808640 1 -- start start 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed11808640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed0c0b2850 0x7fed0c14ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed11808640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0c0d06f0 0x7fed0c0b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed11808640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c15f3a0 0x7fed0c0b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed11808640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c15f900 con 0x7fed0c15f3a0 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed11808640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c15f780 con 0x7fed0c0b2850 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed11808640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c0d23c0 con 0x7fed0c0d06f0 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c15f3a0 0x7fed0c0b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c15f3a0 0x7fed0c0b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52916/0 (socket says 172.21.15.40:52916) 2024-04-26T21:37:57.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.248+0000 7fed0b7fe640 1 -- 172.21.15.40:0/1025291033 learned_addr learned my addr 172.21.15.40:0/1025291033 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:57.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0a7fc640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0c0d06f0 0x7fed0c0b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0affd640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed0c0b2850 0x7fed0c14ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0b7fe640 1 -- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0c0d06f0 msgr2=0x7fed0c0b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0b7fe640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0c0d06f0 0x7fed0c0b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0b7fe640 1 -- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed0c0b2850 msgr2=0x7fed0c14ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0b7fe640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed0c0b2850 0x7fed0c14ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0b7fe640 1 -- 172.21.15.40:0/1025291033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c0b9790 con 0x7fed0c15f3a0 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0affd640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed0c0b2850 0x7fed0c14ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed0b7fe640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c15f3a0 0x7fed0c0b80c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009e80 tx=0x7fecfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed03fff640 1 -- 172.21.15.40:0/1025291033 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc040070 con 0x7fed0c15f3a0 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed03fff640 1 -- 172.21.15.40:0/1025291033 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc031c40 con 0x7fed0c15f3a0 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed03fff640 1 -- 172.21.15.40:0/1025291033 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc003b90 con 0x7fed0c15f3a0 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c0b86f0 con 0x7fed0c15f3a0 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.249+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c0b8cb0 con 0x7fed0c15f3a0 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.250+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0c0694f0 con 0x7fed0c15f3a0 2024-04-26T21:37:57.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.250+0000 7fed03fff640 1 -- 172.21.15.40:0/1025291033 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecfc039040 con 0x7fed0c15f3a0 2024-04-26T21:37:57.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.251+0000 7fed03fff640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecec06c8f0 0x7fecec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.251+0000 7fed03fff640 1 -- 172.21.15.40:0/1025291033 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6159+0+0 (secure 0 0 0) 0x7fecfc0ad070 con 0x7fed0c15f3a0 2024-04-26T21:37:57.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.251+0000 7fed0affd640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecec06c8f0 0x7fecec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.252+0000 7fed0affd640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecec06c8f0 0x7fecec06edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed04000f60 tx=0x7fed04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:57.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.255+0000 7fed03fff640 1 -- 172.21.15.40:0/1025291033 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecfc07b2e0 con 0x7fed0c15f3a0 2024-04-26T21:37:57.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.480+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed0c1507b0 con 0x7fed0c15f3a0 2024-04-26T21:37:57.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.481+0000 7fed03fff640 1 -- 172.21.15.40:0/1025291033 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5181 (secure 0 0 0) 0x7fecfc07abe0 con 0x7fed0c15f3a0 2024-04-26T21:37:57.482 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:57.482 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:37:40.533180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2276808574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2276808574},{"type":"v1","addr":"172.21.15.40:6834","nonce":2276808574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6015,"qdb_cluster":[6015]},"id":3}]} 2024-04-26T21:37:57.483 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 32 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecec06c8f0 msgr2=0x7fecec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecec06c8f0 0x7fecec06edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed04000f60 tx=0x7fed04009040 comp rx=0 tx=0).stop 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c15f3a0 msgr2=0x7fed0c0b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c15f3a0 0x7fed0c0b80c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009e80 tx=0x7fecfc004a40 comp rx=0 tx=0).stop 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 shutdown_connections 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecec06c8f0 0x7fecec06edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed0c15f3a0 0x7fed0c0b80c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0c0d06f0 0x7fed0c0b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 --2- 172.21.15.40:0/1025291033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed0c0b2850 0x7fed0c14ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 >> 172.21.15.40:0/1025291033 conn(0x7fed0c0d4f90 msgr2=0x7fed0c0b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 shutdown_connections 2024-04-26T21:37:57.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.483+0000 7fed11808640 1 -- 172.21.15.40:0/1025291033 wait complete. 2024-04-26T21:37:57.495 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:37:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.595+0000 7fcaaede2640 1 Processor -- start 2024-04-26T21:37:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.595+0000 7fcaaede2640 1 -- start start 2024-04-26T21:37:57.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaaede2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcaa80d1b30 0x7fcaa80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaaede2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa80d10d0 con 0x7fcaa8162810 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaaede2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa80d1370 con 0x7fcaa80b2340 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaaede2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa80d0750 con 0x7fcaa80d1b30 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaacb7e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcaa8162810 0x7fcaa8070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53590/0 (socket says 172.21.15.40:53590) 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaad37f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcaa80d1b30 0x7fcaa80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaacb7e640 1 -- 172.21.15.40:0/2459887260 learned_addr learned my addr 172.21.15.40:0/2459887260 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaad37f640 1 -- 172.21.15.40:0/2459887260 >> v1:172.21.15.184:6789/0 conn(0x7fcaa80b2340 legacy=0x7fcaa80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaad37f640 1 -- 172.21.15.40:0/2459887260 >> v1:172.21.15.40:6789/0 conn(0x7fcaa8162810 legacy=0x7fcaa8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaad37f640 1 -- 172.21.15.40:0/2459887260 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa80d01b0 con 0x7fcaa80d1b30 2024-04-26T21:37:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.596+0000 7fcaa77fe640 1 -- 172.21.15.40:0/2459887260 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2735965089 0 0) 0x7fcaa80d10d0 con 0x7fcaa8162810 2024-04-26T21:37:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.597+0000 7fcaad37f640 1 --2- 172.21.15.40:0/2459887260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80d1b30 0x7fcaa80d1fc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009d90 tx=0x7fcaa002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2eeed35dc8fce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.597+0000 7fcaa77fe640 1 -- 172.21.15.40:0/2459887260 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0007740 con 0x7fcaa80d1b30 2024-04-26T21:37:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.597+0000 7fcaa77fe640 1 -- 172.21.15.40:0/2459887260 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa00078e0 con 0x7fcaa80d1b30 2024-04-26T21:37:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.597+0000 7fcaa77fe640 1 -- 172.21.15.40:0/2459887260 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0007cb0 con 0x7fcaa80d1b30 2024-04-26T21:37:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.598+0000 7fcaaede2640 1 -- 172.21.15.40:0/2459887260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80d1b30 msgr2=0x7fcaa80d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.598+0000 7fcaaede2640 1 --2- 172.21.15.40:0/2459887260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80d1b30 0x7fcaa80d1fc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009d90 tx=0x7fcaa002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.598+0000 7fcaaede2640 1 -- 172.21.15.40:0/2459887260 shutdown_connections 2024-04-26T21:37:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.598+0000 7fcaaede2640 1 --2- 172.21.15.40:0/2459887260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80d1b30 0x7fcaa80d1fc0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.598+0000 7fcaaede2640 1 -- 172.21.15.40:0/2459887260 >> 172.21.15.40:0/2459887260 conn(0x7fcaa80d2da0 msgr2=0x7fcaa80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:57.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.598+0000 7fcaaede2640 1 -- 172.21.15.40:0/2459887260 shutdown_connections 2024-04-26T21:37:57.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.598+0000 7fcaaede2640 1 -- 172.21.15.40:0/2459887260 wait complete. 2024-04-26T21:37:57.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.599+0000 7fcaaede2640 1 Processor -- start 2024-04-26T21:37:57.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.599+0000 7fcaaede2640 1 -- start start 2024-04-26T21:37:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaaede2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80b2340 0x7fcaa81f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaaede2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa80d1b30 0x7fcaa80c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaaede2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa8162810 0x7fcaa81f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaaede2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa80cdb00 con 0x7fcaa80b2340 2024-04-26T21:37:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaaede2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa80d16a0 con 0x7fcaa8162810 2024-04-26T21:37:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaaede2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa8070b50 con 0x7fcaa80d1b30 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80b2340 0x7fcaa81f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80b2340 0x7fcaa81f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52942/0 (socket says 172.21.15.40:52942) 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 -- 172.21.15.40:0/738946829 learned_addr learned my addr 172.21.15.40:0/738946829 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaad37f640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa8162810 0x7fcaa81f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaa7fff640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa80d1b30 0x7fcaa80c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 -- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa80d1b30 msgr2=0x7fcaa80c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa80d1b30 0x7fcaa80c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 -- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa8162810 msgr2=0x7fcaa81f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa8162810 0x7fcaa81f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 -- 172.21.15.40:0/738946829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa80c7800 con 0x7fcaa80b2340 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.600+0000 7fcaacb7e640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80b2340 0x7fcaa81f52f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fca9800c420 tx=0x7fca9800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.601+0000 7fcaa57fa640 1 -- 172.21.15.40:0/738946829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980074e0 con 0x7fcaa80b2340 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.601+0000 7fcaa57fa640 1 -- 172.21.15.40:0/738946829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca980047e0 con 0x7fcaa80b2340 2024-04-26T21:37:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.601+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa80c7e90 con 0x7fcaa80b2340 2024-04-26T21:37:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.601+0000 7fcaa57fa640 1 -- 172.21.15.40:0/738946829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98004c10 con 0x7fcaa80b2340 2024-04-26T21:37:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.601+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa80c8450 con 0x7fcaa80b2340 2024-04-26T21:37:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.602+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa80694f0 con 0x7fcaa80b2340 2024-04-26T21:37:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.602+0000 7fcaa57fa640 1 -- 172.21.15.40:0/738946829 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca98011070 con 0x7fcaa80b2340 2024-04-26T21:37:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.603+0000 7fcaa57fa640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca9406c8f0 0x7fca9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.603+0000 7fcaa57fa640 1 -- 172.21.15.40:0/738946829 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6159+0+0 (secure 0 0 0) 0x7fca9808b8d0 con 0x7fcaa80b2340 2024-04-26T21:37:57.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.603+0000 7fcaa7fff640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca9406c8f0 0x7fca9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:57.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.604+0000 7fcaa7fff640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca9406c8f0 0x7fca9406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcaa815c6c0 tx=0x7fca9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.607+0000 7fcaa57fa640 1 -- 172.21.15.40:0/738946829 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcaa80c7800 con 0x7fcaa80b2340 2024-04-26T21:37:57.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.835+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaa80b1ce0 con 0x7fcaa80b2340 2024-04-26T21:37:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.835+0000 7fcaa57fa640 1 -- 172.21.15.40:0/738946829 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5181 (secure 0 0 0) 0x7fca98059b40 con 0x7fcaa80b2340 2024-04-26T21:37:57.836 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:37:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:37:40.533180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/2276808574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2276808574},{"type":"v1","addr":"172.21.15.40:6834","nonce":2276808574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6015,"qdb_cluster":[6015]},"id":3}]} 2024-04-26T21:37:57.837 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 32 2024-04-26T21:37:57.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca9406c8f0 msgr2=0x7fca9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca9406c8f0 0x7fca9406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcaa815c6c0 tx=0x7fca9c009040 comp rx=0 tx=0).stop 2024-04-26T21:37:57.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80b2340 msgr2=0x7fcaa81f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80b2340 0x7fcaa81f52f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fca9800c420 tx=0x7fca9800c8e0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 shutdown_connections 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca9406c8f0 0x7fca9406edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa8162810 0x7fcaa81f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa80d1b30 0x7fcaa80c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 --2- 172.21.15.40:0/738946829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaa80b2340 0x7fcaa81f52f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 >> 172.21.15.40:0/738946829 conn(0x7fcaa80d2da0 msgr2=0x7fcaa80b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 shutdown_connections 2024-04-26T21:37:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:57.837+0000 7fcaaede2640 1 -- 172.21.15.40:0/738946829 wait complete. 2024-04-26T21:37:57.849 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /a 0 2024-04-26T21:37:57.965 INFO:teuthology.orchestra.run.smithi040.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-04-26T21:37:55.358972+0000","btime":"2024-04-26T21:37:49.493040+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T21:37:55.358972+0000","atime":"2024-04-26T21:37:49.493040+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T21:37:55.358972+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-26T21:37:55.358972+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-26T21:37:55.358972+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":0,"dirtyparent":0,"dirty":1,"authpin":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"3","committed_version":"3","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"a/c","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":0,"dirty":1,"authpin":0},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-26T21:37:57.966 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-26T21:37:55.358972+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-04-26T21:37:49.493040+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T21:37:49.493040+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-04-26T21:37:55.358972+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "3", "committing_version": "3", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 0, "lock": 0, "request": 0 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/a", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-04-26T21:37:55.358972+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T21:37:55.358972+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 0, "dirfrag": 1, "dirty": 1, "dirtyparent": 0, "lock": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-26T21:37:55.358972+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } ] 2024-04-26T21:37:57.966 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-04-26T21:37:55.358972+0000', 'btime': '2024-04-26T21:37:49.493040+0000', 'mode': 16877, 'uid': 1000, 'gid': 1316, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-04-26T21:37:55.358972+0000', 'atime': '2024-04-26T21:37:49.493040+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-04-26T21:37:55.358972+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-04-26T21:37:55.358972+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-04-26T21:37:55.358972+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [{'key': 'security.selinux', 'val': 'unconfined_u:object_r:unlabeled_t:s0\x00'}], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'dirfrag': 1, 'caps': 0, 'dirtyparent': 0, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '3', 'committed_version': '3', 'is_rep': False, 'dir_auth': '', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'dentries': [{'path': 'a/c', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 0, 'dirty': 1, 'authpin': 0}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-04-26T21:37:57.967 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:37:58.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.068+0000 7fe51af89640 1 Processor -- start 2024-04-26T21:37:58.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.068+0000 7fe51af89640 1 -- start start 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe51af89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe514072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe51af89640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe514072780 0x7fe514166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe51af89640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51415d7c0 con 0x7fe514071e60 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe51af89640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe51415daa0 con 0x7fe514072780 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe51af89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe51415dd80 con 0x7fe5140cc480 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe513fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe514072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe519526640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe514072780 0x7fe514166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe513fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe514072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:59174/0 (socket says 172.21.15.40:59174) 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.069+0000 7fe513fff640 1 -- 172.21.15.40:0/2255456358 learned_addr learned my addr 172.21.15.40:0/2255456358 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.070+0000 7fe513fff640 1 -- 172.21.15.40:0/2255456358 >> v2:172.21.15.184:3300/0 conn(0x7fe514072780 msgr2=0x7fe514166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.070+0000 7fe513fff640 1 --2- 172.21.15.40:0/2255456358 >> v2:172.21.15.184:3300/0 conn(0x7fe514072780 0x7fe514166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.070+0000 7fe513fff640 1 -- 172.21.15.40:0/2255456358 >> v1:172.21.15.184:6789/0 conn(0x7fe514071e60 legacy=0x7fe51406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.070+0000 7fe5137fe640 1 -- 172.21.15.40:0/2255456358 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242352011 0 0) 0x7fe51415d7c0 con 0x7fe514071e60 2024-04-26T21:37:58.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.070+0000 7fe513fff640 1 -- 172.21.15.40:0/2255456358 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51415e060 con 0x7fe5140cc480 2024-04-26T21:37:58.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.070+0000 7fe519526640 1 --2- 172.21.15.40:0/2255456358 >> v2:172.21.15.184:3300/0 conn(0x7fe514072780 0x7fe514166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.070+0000 7fe513fff640 1 --2- 172.21.15.40:0/2255456358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe514072240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe500009870 tx=0x7fe50002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63314bd5e60fda34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.070+0000 7fe5137fe640 1 -- 172.21.15.40:0/2255456358 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500037070 con 0x7fe5140cc480 2024-04-26T21:37:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe5137fe640 1 -- 172.21.15.40:0/2255456358 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe500032440 con 0x7fe5140cc480 2024-04-26T21:37:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe5137fe640 1 -- 172.21.15.40:0/2255456358 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500032810 con 0x7fe5140cc480 2024-04-26T21:37:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe51af89640 1 -- 172.21.15.40:0/2255456358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 msgr2=0x7fe514072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe51af89640 1 --2- 172.21.15.40:0/2255456358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe514072240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe500009870 tx=0x7fe50002a270 comp rx=0 tx=0).stop 2024-04-26T21:37:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe51af89640 1 -- 172.21.15.40:0/2255456358 shutdown_connections 2024-04-26T21:37:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe51af89640 1 --2- 172.21.15.40:0/2255456358 >> v2:172.21.15.184:3300/0 conn(0x7fe514072780 0x7fe514166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe51af89640 1 --2- 172.21.15.40:0/2255456358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe514072240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe51af89640 1 -- 172.21.15.40:0/2255456358 >> 172.21.15.40:0/2255456358 conn(0x7fe5140d0be0 msgr2=0x7fe5140d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe51af89640 1 -- 172.21.15.40:0/2255456358 shutdown_connections 2024-04-26T21:37:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.071+0000 7fe51af89640 1 -- 172.21.15.40:0/2255456358 wait complete. 2024-04-26T21:37:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.072+0000 7fe51af89640 1 Processor -- start 2024-04-26T21:37:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.072+0000 7fe51af89640 1 -- start start 2024-04-26T21:37:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe51af89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe514071e60 0x7fe51415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe51af89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe514072780 0x7fe5140ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe51af89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe5140c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe51af89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe51415d3f0 con 0x7fe514071e60 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe51af89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe514072c10 con 0x7fe514072780 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe51af89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe51415d570 con 0x7fe5140cc480 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe518d25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe514071e60 0x7fe51415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe518d25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe514071e60 0x7fe51415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52944/0 (socket says 172.21.15.40:52944) 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe518d25640 1 -- 172.21.15.40:0/3852991443 learned_addr learned my addr 172.21.15.40:0/3852991443 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe513fff640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe514072780 0x7fe5140ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.073+0000 7fe519526640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe5140c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe518d25640 1 -- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 msgr2=0x7fe5140c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe518d25640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe5140c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe518d25640 1 -- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe514072780 msgr2=0x7fe5140ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe518d25640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe514072780 0x7fe5140ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe518d25640 1 -- 172.21.15.40:0/3852991443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5140af830 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe519526640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe5140c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe513fff640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe514072780 0x7fe5140ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe518d25640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe514071e60 0x7fe51415a1a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c4e0 tx=0x7fe50800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe5117fa640 1 -- 172.21.15.40:0/3852991443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508017070 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe5117fa640 1 -- 172.21.15.40:0/3852991443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5080035f0 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5141f4ad0 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe5117fa640 1 -- 172.21.15.40:0/3852991443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5080039e0 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.074+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5140aeea0 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.075+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5141f48c0 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.075+0000 7fe5117fa640 1 -- 172.21.15.40:0/3852991443 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe50801a070 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.076+0000 7fe5117fa640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4f806c8f0 0x7fe4f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.076+0000 7fe5117fa640 1 -- 172.21.15.40:0/3852991443 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6159+0+0 (secure 0 0 0) 0x7fe50808aff0 con 0x7fe514071e60 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.076+0000 7fe513fff640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4f806c8f0 0x7fe4f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.076+0000 7fe513fff640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4f806c8f0 0x7fe4f806edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe500034000 tx=0x7fe50002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.080+0000 7fe5117fa640 1 -- 172.21.15.40:0/3852991443 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe508058c10 con 0x7fe514071e60 2024-04-26T21:37:58.299 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:37:58.298+0000 7f86209e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-26T21:37:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.304+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe5140694f0 con 0x7fe514071e60 2024-04-26T21:37:58.305 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:37:58.305+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:37:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.306+0000 7fe5117fa640 1 -- 172.21.15.40:0/3852991443 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7fe50805cf10 con 0x7fe514071e60 2024-04-26T21:37:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.308+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4f806c8f0 msgr2=0x7fe4f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.308+0000 7fe51af89640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4f806c8f0 0x7fe4f806edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe500034000 tx=0x7fe50002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.308+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe514071e60 msgr2=0x7fe51415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.308+0000 7fe51af89640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe514071e60 0x7fe51415a1a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c4e0 tx=0x7fe50800c9a0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.309+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 shutdown_connections 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.309+0000 7fe51af89640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4f806c8f0 0x7fe4f806edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.309+0000 7fe51af89640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5140cc480 0x7fe5140c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.309+0000 7fe51af89640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe514072780 0x7fe5140ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.309+0000 7fe51af89640 1 --2- 172.21.15.40:0/3852991443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe514071e60 0x7fe51415a1a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.309+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 >> 172.21.15.40:0/3852991443 conn(0x7fe5140d0be0 msgr2=0x7fe51415f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.309+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 shutdown_connections 2024-04-26T21:37:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.309+0000 7fe51af89640 1 -- 172.21.15.40:0/3852991443 wait complete. 2024-04-26T21:37:58.312 INFO:tasks.ceph.mds.c.smithi040.stderr: -4242> 2024-04-26T21:37:44.123+0000 7fb98f204640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:37:58.317 INFO:tasks.ceph.mds.c.smithi040.stderr: -409> 2024-04-26T21:37:55.359+0000 7fb991a09640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6213:44 nref=7 cr=0x5634baf36700)) (dversion lock w=1 last_client=6213) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x5634bae82000] 2024-04-26T21:37:58.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-04-26T21:37:58.343 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:37:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.424+0000 7f42579bf640 1 Processor -- start 2024-04-26T21:37:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.424+0000 7f42579bf640 1 -- start start 2024-04-26T21:37:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f42579bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4250162810 0x7f4250070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f42579bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500b2340 0x7f42500cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f42579bf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42500d10d0 con 0x7f42500cd930 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f42579bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f42500d0750 con 0x7f4250162810 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f42579bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f42500d16a0 con 0x7f42500b2340 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f4254f5a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500b2340 0x7f42500cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f425575b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4250162810 0x7f4250070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f4254f5a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500b2340 0x7f42500cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:59238/0 (socket says 172.21.15.40:59238) 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f425575b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4250162810 0x7f4250070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:56736/0 (socket says 172.21.15.40:56736) 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f425575b640 1 -- 172.21.15.40:0/1863647104 learned_addr learned my addr 172.21.15.40:0/1863647104 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f425575b640 1 -- 172.21.15.40:0/1863647104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500b2340 msgr2=0x7f42500cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f425575b640 1 --2- 172.21.15.40:0/1863647104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500b2340 0x7f42500cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f425575b640 1 -- 172.21.15.40:0/1863647104 >> v1:172.21.15.184:6789/0 conn(0x7f42500cd930 legacy=0x7f42500d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f425575b640 1 -- 172.21.15.40:0/1863647104 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42500d1370 con 0x7f4250162810 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f4247fff640 1 -- 172.21.15.40:0/1863647104 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917827676 0 0) 0x7f42500d10d0 con 0x7f42500cd930 2024-04-26T21:37:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.425+0000 7f4254f5a640 1 --2- 172.21.15.40:0/1863647104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500b2340 0x7f42500cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:37:58.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.426+0000 7f425575b640 1 --2- 172.21.15.40:0/1863647104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4250162810 0x7f4250070350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f424c009870 tx=0x7f424c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37cfb1a769f2cd77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.426+0000 7f4247fff640 1 -- 172.21.15.40:0/1863647104 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c037070 con 0x7f4250162810 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.426+0000 7f4247fff640 1 -- 172.21.15.40:0/1863647104 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424c032440 con 0x7f4250162810 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.426+0000 7f4247fff640 1 -- 172.21.15.40:0/1863647104 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c032810 con 0x7f4250162810 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.427+0000 7f42579bf640 1 -- 172.21.15.40:0/1863647104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4250162810 msgr2=0x7f4250070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.427+0000 7f42579bf640 1 --2- 172.21.15.40:0/1863647104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4250162810 0x7f4250070350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f424c009870 tx=0x7f424c02a270 comp rx=0 tx=0).stop 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.427+0000 7f42579bf640 1 -- 172.21.15.40:0/1863647104 shutdown_connections 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.427+0000 7f42579bf640 1 --2- 172.21.15.40:0/1863647104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500b2340 0x7f42500cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.427+0000 7f42579bf640 1 --2- 172.21.15.40:0/1863647104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4250162810 0x7f4250070350 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.427+0000 7f42579bf640 1 -- 172.21.15.40:0/1863647104 >> 172.21.15.40:0/1863647104 conn(0x7f42500d2da0 msgr2=0x7f42500d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.427+0000 7f42579bf640 1 -- 172.21.15.40:0/1863647104 shutdown_connections 2024-04-26T21:37:58.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.427+0000 7f42579bf640 1 -- 172.21.15.40:0/1863647104 wait complete. 2024-04-26T21:37:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.428+0000 7f42579bf640 1 Processor -- start 2024-04-26T21:37:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.428+0000 7f42579bf640 1 -- start start 2024-04-26T21:37:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f42579bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42500b2340 0x7f42501f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f42579bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500cd930 0x7f42501f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f42579bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4250162810 0x7f4250160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f42579bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4250070890 con 0x7f4250162810 2024-04-26T21:37:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f42579bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f42500d0470 con 0x7f42500b2340 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f42579bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f42500d2240 con 0x7f42500cd930 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4250162810 0x7f4250160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4250162810 0x7f4250160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52978/0 (socket says 172.21.15.40:52978) 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 -- 172.21.15.40:0/534358380 learned_addr learned my addr 172.21.15.40:0/534358380 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4254f5a640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500cd930 0x7f42501f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f425575b640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42500b2340 0x7f42501f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 -- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500cd930 msgr2=0x7f42501f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500cd930 0x7f42501f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 -- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42500b2340 msgr2=0x7f42501f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42500b2340 0x7f42501f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 -- 172.21.15.40:0/534358380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42501f5d80 con 0x7f4250162810 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f425575b640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42500b2340 0x7f42501f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.429+0000 7f4255f5c640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4250162810 0x7f4250160630 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f424800c700 tx=0x7f424800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.430+0000 7f4245ffb640 1 -- 172.21.15.40:0/534358380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248003da0 con 0x7f4250162810 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.430+0000 7f4245ffb640 1 -- 172.21.15.40:0/534358380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424800ce10 con 0x7f4250162810 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.430+0000 7f4245ffb640 1 -- 172.21.15.40:0/534358380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248005270 con 0x7f4250162810 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.430+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425015fdf0 con 0x7f4250162810 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.430+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425015f030 con 0x7f4250162810 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.431+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42500694f0 con 0x7f4250162810 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.431+0000 7f4245ffb640 1 -- 172.21.15.40:0/534358380 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4248005410 con 0x7f4250162810 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.431+0000 7f4245ffb640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f423406c8f0 0x7f423406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.432+0000 7f4245ffb640 1 -- 172.21.15.40:0/534358380 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f424808bc40 con 0x7f4250162810 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.432+0000 7f425575b640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f423406c8f0 0x7f423406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.432+0000 7f425575b640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f423406c8f0 0x7f423406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f424c034000 tx=0x7f424c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.436+0000 7f4245ffb640 1 -- 172.21.15.40:0/534358380 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42501f5d80 con 0x7f4250162810 2024-04-26T21:37:58.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.628+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f425015ea00 con 0x7f4250162810 2024-04-26T21:37:58.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.631+0000 7f4245ffb640 1 -- 172.21.15.40:0/534358380 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f4248059ee0 con 0x7f4250162810 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f423406c8f0 msgr2=0x7f423406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f423406c8f0 0x7f423406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f424c034000 tx=0x7f424c02e000 comp rx=0 tx=0).stop 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4250162810 msgr2=0x7f4250160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4250162810 0x7f4250160630 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f424800c700 tx=0x7f424800cbc0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 shutdown_connections 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f423406c8f0 0x7f423406edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4250162810 0x7f4250160630 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42500cd930 0x7f42501f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 --2- 172.21.15.40:0/534358380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42500b2340 0x7f42501f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.633+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 >> 172.21.15.40:0/534358380 conn(0x7f42500d2da0 msgr2=0x7f42500b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.634+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 shutdown_connections 2024-04-26T21:37:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.634+0000 7f42579bf640 1 -- 172.21.15.40:0/534358380 wait complete. 2024-04-26T21:37:58.645 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-04-26T21:37:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.788+0000 7fdb6f5f0640 1 Processor -- start 2024-04-26T21:37:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.788+0000 7fdb6f5f0640 1 -- start start 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.789+0000 7fdb6f5f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdb680b1fd0 0x7fdb680d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.789+0000 7fdb6f5f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb6806e580 con 0x7fdb6815f770 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.789+0000 7fdb6f5f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb6806e860 con 0x7fdb680cdb70 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.789+0000 7fdb6f5f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdb6806f3e0 con 0x7fdb680b1fd0 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.789+0000 7fdb6d38c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdb6815f770 0x7fdb680d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51800/0 (socket says 172.21.15.40:51800) 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.789+0000 7fdb6d38c640 1 -- 172.21.15.40:0/1494587981 learned_addr learned my addr 172.21.15.40:0/1494587981 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425820052 0 0) 0x7fdb6806e580 con 0x7fdb6815f770 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb50003520 con 0x7fdb6815f770 2024-04-26T21:37:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6cb8b640 1 --2- 172.21.15.40:0/1494587981 >> v2:172.21.15.184:3300/0 conn(0x7fdb680b1fd0 0x7fdb680d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1403150311 0 0) 0x7fdb50003520 con 0x7fdb6815f770 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 >> v2:172.21.15.184:3300/0 conn(0x7fdb680b1fd0 msgr2=0x7fdb680d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 --2- 172.21.15.40:0/1494587981 >> v2:172.21.15.184:3300/0 conn(0x7fdb680b1fd0 0x7fdb680d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 >> v1:172.21.15.184:6789/0 conn(0x7fdb680cdb70 legacy=0x7fdb680ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6806eb40 con 0x7fdb6815f770 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fdb60001670 con 0x7fdb6815f770 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 >> v1:172.21.15.40:6789/0 conn(0x7fdb6815f770 legacy=0x7fdb680d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb500039b0 0x7fdb50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb50004800 0x7fdb50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500051c0 0x7fdb5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6806f3e0 con 0x7fdb500039b0 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5000c060 con 0x7fdb50004800 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5000c260 con 0x7fdb500051c0 2024-04-26T21:37:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6d38c640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb500039b0 0x7fdb50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6cb8b640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb50004800 0x7fdb50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6db8d640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500051c0 0x7fdb5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6d38c640 1 -- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500051c0 msgr2=0x7fdb5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6d38c640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500051c0 0x7fdb5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6d38c640 1 -- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb50004800 msgr2=0x7fdb50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6d38c640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb50004800 0x7fdb50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6d38c640 1 -- 172.21.15.40:0/1494587981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb50003520 con 0x7fdb500039b0 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6db8d640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500051c0 0x7fdb5000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.790+0000 7fdb6db8d640 1 -- 172.21.15.40:0/1494587981 reap_dead start 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.791+0000 7fdb6d38c640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb500039b0 0x7fdb50003dd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000c210 tx=0x7fdb50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385b3a79f94c125f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.791+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60007d30 con 0x7fdb500039b0 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.791+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb60005ad0 con 0x7fdb500039b0 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.791+0000 7fdb67fff640 1 -- 172.21.15.40:0/1494587981 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60007070 con 0x7fdb500039b0 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.792+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/1494587981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb500039b0 msgr2=0x7fdb50003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.792+0000 7fdb6f5f0640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb500039b0 0x7fdb50003dd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000c210 tx=0x7fdb50005830 comp rx=0 tx=0).stop 2024-04-26T21:37:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.792+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/1494587981 shutdown_connections 2024-04-26T21:37:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.792+0000 7fdb6f5f0640 1 --2- 172.21.15.40:0/1494587981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb500039b0 0x7fdb50003dd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.792+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/1494587981 >> 172.21.15.40:0/1494587981 conn(0x7fdb680d2e60 msgr2=0x7fdb680d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.792+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/1494587981 shutdown_connections 2024-04-26T21:37:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.792+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/1494587981 wait complete. 2024-04-26T21:37:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.793+0000 7fdb6f5f0640 1 Processor -- start 2024-04-26T21:37:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.793+0000 7fdb6f5f0640 1 -- start start 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6f5f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500039b0 0x7fdb681523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6f5f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb6815fb50 0x7fdb680c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6f5f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb680b0700 0x7fdb680b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6f5f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdb680d10f0 con 0x7fdb680b0700 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6f5f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb680d0f70 con 0x7fdb6815fb50 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6f5f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdb680d1270 con 0x7fdb500039b0 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb680b0700 0x7fdb680b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb680b0700 0x7fdb680b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56690/0 (socket says 172.21.15.40:56690) 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 -- 172.21.15.40:0/759897172 learned_addr learned my addr 172.21.15.40:0/759897172 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6d38c640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500039b0 0x7fdb681523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6cb8b640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb6815fb50 0x7fdb680c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 -- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500039b0 msgr2=0x7fdb681523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500039b0 0x7fdb681523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 -- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb6815fb50 msgr2=0x7fdb680c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb6815fb50 0x7fdb680c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 -- 172.21.15.40:0/759897172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb681600c0 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb6db8d640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb680b0700 0x7fdb680b0b80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdb680731b0 tx=0x7fdb5800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.794+0000 7fdb65ffb640 1 -- 172.21.15.40:0/759897172 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58003970 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.795+0000 7fdb65ffb640 1 -- 172.21.15.40:0/759897172 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb58018070 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.795+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb681603a0 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.795+0000 7fdb65ffb640 1 -- 172.21.15.40:0/759897172 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58003d50 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.795+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb68160960 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.796+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb680694f0 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.796+0000 7fdb65ffb640 1 -- 172.21.15.40:0/759897172 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb580050d0 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.797+0000 7fdb65ffb640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdb5406c920 0x7fdb5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.797+0000 7fdb65ffb640 1 -- 172.21.15.40:0/759897172 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7fdb5808b760 con 0x7fdb680b0700 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.797+0000 7fdb6d38c640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdb5406c920 0x7fdb5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.798+0000 7fdb6d38c640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdb5406c920 0x7fdb5406ede0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdb60033000 tx=0x7fdb6002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.801+0000 7fdb65ffb640 1 -- 172.21.15.40:0/759897172 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb58022020 con 0x7fdb680b0700 2024-04-26T21:37:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.985+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7fdb6806fe50 con 0x7fdb680b0700 2024-04-26T21:37:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.985+0000 7fdb65ffb640 1 -- 172.21.15.40:0/759897172 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v15)=0 v15) ==== 135+0+0 (secure 0 0 0) 0x7fdb580598d0 con 0x7fdb680b0700 2024-04-26T21:37:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdb5406c920 msgr2=0x7fdb5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdb5406c920 0x7fdb5406ede0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdb60033000 tx=0x7fdb6002e000 comp rx=0 tx=0).stop 2024-04-26T21:37:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb680b0700 msgr2=0x7fdb680b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb680b0700 0x7fdb680b0b80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdb680731b0 tx=0x7fdb5800eb90 comp rx=0 tx=0).stop 2024-04-26T21:37:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 shutdown_connections 2024-04-26T21:37:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdb5406c920 0x7fdb5406ede0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb680b0700 0x7fdb680b0b80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb6815fb50 0x7fdb680c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 --2- 172.21.15.40:0/759897172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb500039b0 0x7fdb681523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 >> 172.21.15.40:0/759897172 conn(0x7fdb680d2e60 msgr2=0x7fdb6806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 shutdown_connections 2024-04-26T21:37:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:58.987+0000 7fdb6f5f0640 1 -- 172.21.15.40:0/759897172 wait complete. 2024-04-26T21:37:58.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:37:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.102+0000 7ff5c590a640 1 Processor -- start 2024-04-26T21:37:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.102+0000 7ff5c590a640 1 -- start start 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5c590a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5c0162cd0 0x7ff5c00b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5c590a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5c00d2110 0x7ff5c00b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5c590a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c00d0890 con 0x7ff5c01628f0 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5c590a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c00d1130 con 0x7ff5c00d2110 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5c590a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c00d0b70 con 0x7ff5c0162cd0 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5bf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5c00d2110 0x7ff5c00b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5beffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff5c01628f0 0x7ff5c00d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51802/0 (socket says 172.21.15.40:51802) 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5beffd640 1 -- 172.21.15.40:0/3611934462 learned_addr learned my addr 172.21.15.40:0/3611934462 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.103+0000 7ff5be7fc640 1 --2- 172.21.15.40:0/3611934462 >> v2:172.21.15.184:3300/0 conn(0x7ff5c0162cd0 0x7ff5c00b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.104+0000 7ff5bf7fe640 1 -- 172.21.15.40:0/3611934462 >> v2:172.21.15.184:3300/0 conn(0x7ff5c0162cd0 msgr2=0x7ff5c00b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.104+0000 7ff5bf7fe640 1 --2- 172.21.15.40:0/3611934462 >> v2:172.21.15.184:3300/0 conn(0x7ff5c0162cd0 0x7ff5c00b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.104+0000 7ff5bf7fe640 1 -- 172.21.15.40:0/3611934462 >> v1:172.21.15.40:6789/0 conn(0x7ff5c01628f0 legacy=0x7ff5c00d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.104+0000 7ff5bf7fe640 1 -- 172.21.15.40:0/3611934462 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c00d0e50 con 0x7ff5c00d2110 2024-04-26T21:37:59.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.104+0000 7ff5bdffb640 1 -- 172.21.15.40:0/3611934462 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920441004 0 0) 0x7ff5c00d0890 con 0x7ff5c01628f0 2024-04-26T21:37:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.104+0000 7ff5bf7fe640 1 --2- 172.21.15.40:0/3611934462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c00d2110 0x7ff5c00b51a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00d4250 tx=0x7ff5b002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962c805f2e905fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.104+0000 7ff5bdffb640 1 -- 172.21.15.40:0/3611934462 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0007740 con 0x7ff5c00d2110 2024-04-26T21:37:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.104+0000 7ff5bdffb640 1 -- 172.21.15.40:0/3611934462 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b00078e0 con 0x7ff5c00d2110 2024-04-26T21:37:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5bdffb640 1 -- 172.21.15.40:0/3611934462 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0007cb0 con 0x7ff5c00d2110 2024-04-26T21:37:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5c590a640 1 -- 172.21.15.40:0/3611934462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c00d2110 msgr2=0x7ff5c00b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5c590a640 1 --2- 172.21.15.40:0/3611934462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c00d2110 0x7ff5c00b51a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00d4250 tx=0x7ff5b002a5f0 comp rx=0 tx=0).stop 2024-04-26T21:37:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5c590a640 1 -- 172.21.15.40:0/3611934462 shutdown_connections 2024-04-26T21:37:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5c590a640 1 --2- 172.21.15.40:0/3611934462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c00d2110 0x7ff5c00b51a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5c590a640 1 --2- 172.21.15.40:0/3611934462 >> v2:172.21.15.184:3300/0 conn(0x7ff5c0162cd0 0x7ff5c00b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5c590a640 1 -- 172.21.15.40:0/3611934462 >> 172.21.15.40:0/3611934462 conn(0x7ff5c00d4ff0 msgr2=0x7ff5c00d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:37:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5c590a640 1 -- 172.21.15.40:0/3611934462 shutdown_connections 2024-04-26T21:37:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.105+0000 7ff5c590a640 1 -- 172.21.15.40:0/3611934462 wait complete. 2024-04-26T21:37:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.106+0000 7ff5c590a640 1 Processor -- start 2024-04-26T21:37:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.106+0000 7ff5c590a640 1 -- start start 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5c590a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d2110 0x7ff5c01f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5c590a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c01628f0 0x7ff5c0153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5c590a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c0162cd0 0x7ff5c00cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5c590a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00d16d0 con 0x7ff5c01628f0 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5c590a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00d1550 con 0x7ff5c00d2110 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5c590a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c00b3f50 con 0x7ff5c0162cd0 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5be7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c01628f0 0x7ff5c0153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5be7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c01628f0 0x7ff5c0153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56694/0 (socket says 172.21.15.40:56694) 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5be7fc640 1 -- 172.21.15.40:0/2800703185 learned_addr learned my addr 172.21.15.40:0/2800703185 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.107+0000 7ff5bf7fe640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c0162cd0 0x7ff5c00cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5beffd640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d2110 0x7ff5c01f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5be7fc640 1 -- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c0162cd0 msgr2=0x7ff5c00cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5be7fc640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c0162cd0 0x7ff5c00cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5be7fc640 1 -- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d2110 msgr2=0x7ff5c01f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5be7fc640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d2110 0x7ff5c01f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5be7fc640 1 -- 172.21.15.40:0/2800703185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c00ce0a0 con 0x7ff5c01628f0 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5bf7fe640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c0162cd0 0x7ff5c00cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5beffd640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d2110 0x7ff5c01f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5be7fc640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c01628f0 0x7ff5c0153230 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800d5f0 tx=0x7ff5a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5b7fff640 1 -- 172.21.15.40:0/2800703185 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80125f0 con 0x7ff5c01628f0 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5b7fff640 1 -- 172.21.15.40:0/2800703185 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a8019070 con 0x7ff5c01628f0 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c00ce380 con 0x7ff5c01628f0 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5b7fff640 1 -- 172.21.15.40:0/2800703185 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80129d0 con 0x7ff5c01628f0 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.108+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c00ce940 con 0x7ff5c01628f0 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.109+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c00694f0 con 0x7ff5c01628f0 2024-04-26T21:37:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.109+0000 7ff5b7fff640 1 -- 172.21.15.40:0/2800703185 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5a8012b70 con 0x7ff5c01628f0 2024-04-26T21:37:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.110+0000 7ff5b7fff640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a006c8f0 0x7ff5a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:37:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.110+0000 7ff5b7fff640 1 -- 172.21.15.40:0/2800703185 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7ff5a808c1c0 con 0x7ff5c01628f0 2024-04-26T21:37:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.110+0000 7ff5beffd640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a006c8f0 0x7ff5a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:37:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.111+0000 7ff5beffd640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a006c8f0 0x7ff5a006edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000f60 tx=0x7ff5b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:37:59.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.114+0000 7ff5b7fff640 1 -- 172.21.15.40:0/2800703185 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5a805a3d0 con 0x7ff5c01628f0 2024-04-26T21:37:59.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:37:59.341+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff5c006e910 con 0x7ff5c01628f0 2024-04-26T21:38:00.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.315+0000 7ff5b7fff640 1 -- 172.21.15.40:0/2800703185 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v36) ==== 162+0+0 (secure 0 0 0) 0x7ff5a8059cd0 con 0x7ff5c01628f0 2024-04-26T21:38:00.317 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:38:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a006c8f0 msgr2=0x7ff5a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a006c8f0 0x7ff5a006edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000f60 tx=0x7ff5b8009040 comp rx=0 tx=0).stop 2024-04-26T21:38:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c01628f0 msgr2=0x7ff5c0153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c01628f0 0x7ff5c0153230 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800d5f0 tx=0x7ff5a800dab0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 shutdown_connections 2024-04-26T21:38:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a006c8f0 0x7ff5a006edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c0162cd0 0x7ff5c00cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c01628f0 0x7ff5c0153230 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 --2- 172.21.15.40:0/2800703185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c00d2110 0x7ff5c01f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 >> 172.21.15.40:0/2800703185 conn(0x7ff5c00d4ff0 msgr2=0x7ff5c00b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 shutdown_connections 2024-04-26T21:38:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.319+0000 7ff5c590a640 1 -- 172.21.15.40:0/2800703185 wait complete. 2024-04-26T21:38:00.336 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.442+0000 7fa77bee1640 1 Processor -- start 2024-04-26T21:38:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.442+0000 7fa77bee1640 1 -- start start 2024-04-26T21:38:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa77bee1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa774162810 0x7fa774070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa77bee1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7740d10d0 con 0x7fa7740d1b30 2024-04-26T21:38:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa77bee1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7740d1370 con 0x7fa7740b2340 2024-04-26T21:38:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa77bee1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa7740d0750 con 0x7fa774162810 2024-04-26T21:38:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa779c7d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa774162810 0x7fa774070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa77a47e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa7740d1b30 0x7fa774070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51806/0 (socket says 172.21.15.40:51806) 2024-04-26T21:38:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa77a47e640 1 -- 172.21.15.40:0/657437815 learned_addr learned my addr 172.21.15.40:0/657437815 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa779c7d640 1 -- 172.21.15.40:0/657437815 >> v1:172.21.15.184:6789/0 conn(0x7fa7740b2340 legacy=0x7fa7740cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa779c7d640 1 -- 172.21.15.40:0/657437815 >> v1:172.21.15.40:6789/0 conn(0x7fa7740d1b30 legacy=0x7fa774070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.443+0000 7fa778c7b640 1 -- 172.21.15.40:0/657437815 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 872685771 0 0) 0x7fa7740d10d0 con 0x7fa7740d1b30 2024-04-26T21:38:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.444+0000 7fa779c7d640 1 -- 172.21.15.40:0/657437815 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7740d01b0 con 0x7fa774162810 2024-04-26T21:38:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.444+0000 7fa779c7d640 1 --2- 172.21.15.40:0/657437815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa774070350 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa7700097a0 tx=0x7fa77002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6509a9d6a6b58b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.444+0000 7fa778c7b640 1 -- 172.21.15.40:0/657437815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770037070 con 0x7fa774162810 2024-04-26T21:38:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.444+0000 7fa778c7b640 1 -- 172.21.15.40:0/657437815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa770032440 con 0x7fa774162810 2024-04-26T21:38:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.444+0000 7fa778c7b640 1 -- 172.21.15.40:0/657437815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770032810 con 0x7fa774162810 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.445+0000 7fa77bee1640 1 -- 172.21.15.40:0/657437815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 msgr2=0x7fa774070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.445+0000 7fa77bee1640 1 --2- 172.21.15.40:0/657437815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa774070350 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa7700097a0 tx=0x7fa77002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.445+0000 7fa77bee1640 1 -- 172.21.15.40:0/657437815 shutdown_connections 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.445+0000 7fa77bee1640 1 --2- 172.21.15.40:0/657437815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa774070350 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.445+0000 7fa77bee1640 1 -- 172.21.15.40:0/657437815 >> 172.21.15.40:0/657437815 conn(0x7fa7740d2da0 msgr2=0x7fa7740d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.445+0000 7fa77bee1640 1 -- 172.21.15.40:0/657437815 shutdown_connections 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.445+0000 7fa77bee1640 1 -- 172.21.15.40:0/657437815 wait complete. 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.446+0000 7fa77bee1640 1 Processor -- start 2024-04-26T21:38:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.446+0000 7fa77bee1640 1 -- start start 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77bee1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7740b2340 0x7fa77414bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77bee1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7740d1b30 0x7fa77414e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77bee1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa77414e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77bee1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa774070890 con 0x7fa774162810 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77bee1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7740d16a0 con 0x7fa7740d1b30 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77bee1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7740cdb00 con 0x7fa7740b2340 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77a47e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa77414e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77a47e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa77414e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56708/0 (socket says 172.21.15.40:56708) 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77a47e640 1 -- 172.21.15.40:0/581255815 learned_addr learned my addr 172.21.15.40:0/581255815 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77a47e640 1 -- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7740b2340 msgr2=0x7fa77414bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77a47e640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7740b2340 0x7fa77414bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77a47e640 1 -- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7740d1b30 msgr2=0x7fa77414e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77a47e640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7740d1b30 0x7fa77414e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.447+0000 7fa77a47e640 1 -- 172.21.15.40:0/581255815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77414c090 con 0x7fa774162810 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.448+0000 7fa77a47e640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa77414e970 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00c7d0 tx=0x7fa76c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.448+0000 7fa76a7fc640 1 -- 172.21.15.40:0/581255815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c018070 con 0x7fa774162810 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.448+0000 7fa76a7fc640 1 -- 172.21.15.40:0/581255815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c00a640 con 0x7fa774162810 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.448+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77414cfb0 con 0x7fa774162810 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.448+0000 7fa76a7fc640 1 -- 172.21.15.40:0/581255815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00aa70 con 0x7fa774162810 2024-04-26T21:38:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.448+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77414d570 con 0x7fa774162810 2024-04-26T21:38:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.449+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7740694f0 con 0x7fa774162810 2024-04-26T21:38:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.449+0000 7fa76a7fc640 1 -- 172.21.15.40:0/581255815 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa76c01b070 con 0x7fa774162810 2024-04-26T21:38:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.450+0000 7fa76a7fc640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa75806c920 0x7fa75806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.450+0000 7fa76a7fc640 1 -- 172.21.15.40:0/581255815 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7fa76c08b3b0 con 0x7fa774162810 2024-04-26T21:38:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.450+0000 7fa779c7d640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa75806c920 0x7fa75806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.450+0000 7fa779c7d640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa75806c920 0x7fa75806ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa770034000 tx=0x7fa77002c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.454+0000 7fa76a7fc640 1 -- 172.21.15.40:0/581255815 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa76c059530 con 0x7fa774162810 2024-04-26T21:38:00.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.681+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7740b2c10 con 0x7fa774162810 2024-04-26T21:38:00.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.681+0000 7fa76a7fc640 1 -- 172.21.15.40:0/581255815 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5175 (secure 0 0 0) 0x7fa76c058e30 con 0x7fa774162810 2024-04-26T21:38:00.683 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:00.683 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:38:00.322094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"c","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6834/4223063167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4223063167},{"type":"v1","addr":"172.21.15.40:6834","nonce":4223063167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-26T21:38:00.683 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 37 2024-04-26T21:38:00.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa75806c920 msgr2=0x7fa75806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa75806c920 0x7fa75806ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa770034000 tx=0x7fa77002c000 comp rx=0 tx=0).stop 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 msgr2=0x7fa77414e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa77414e970 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00c7d0 tx=0x7fa76c00cc90 comp rx=0 tx=0).stop 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 shutdown_connections 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa75806c920 0x7fa75806ede0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa774162810 0x7fa77414e970 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7740d1b30 0x7fa77414e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 --2- 172.21.15.40:0/581255815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7740b2340 0x7fa77414bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 >> 172.21.15.40:0/581255815 conn(0x7fa7740d2da0 msgr2=0x7fa77415cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 shutdown_connections 2024-04-26T21:38:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.683+0000 7fa77bee1640 1 -- 172.21.15.40:0/581255815 wait complete. 2024-04-26T21:38:00.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-04-26T21:38:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.797+0000 7f1440220640 1 Processor -- start 2024-04-26T21:38:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.797+0000 7f1440220640 1 -- start start 2024-04-26T21:38:00.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.798+0000 7f1440220640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 0x7f14380d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.798+0000 7f1440220640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14380d1770 0x7f14380d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.798+0000 7f1440220640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14380d25a0 con 0x7f14380d2130 2024-04-26T21:38:00.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.798+0000 7f1440220640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f14380d0e20 con 0x7f14380d1770 2024-04-26T21:38:00.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.798+0000 7f1440220640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14380b1d70 con 0x7f143815f3a0 2024-04-26T21:38:00.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.798+0000 7f143e7bd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f14380d2130 0x7f14380d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51820/0 (socket says 172.21.15.40:51820) 2024-04-26T21:38:00.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.798+0000 7f143e7bd640 1 -- 172.21.15.40:0/3668238892 learned_addr learned my addr 172.21.15.40:0/3668238892 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 0x7f14380d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3972371456 0 0) 0x7f1428001490 con 0x7f14380d2130 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1420003520 con 0x7f14380d2130 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143d7bb640 1 --2- 172.21.15.40:0/3668238892 >> v2:172.21.15.184:3300/0 conn(0x7f14380d1770 0x7f14380d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 127822310 0 0) 0x7f1420003520 con 0x7f14380d2130 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 msgr2=0x7f14380d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 0x7f14380d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 >> v2:172.21.15.184:3300/0 conn(0x7f14380d1770 msgr2=0x7f14380d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 --2- 172.21.15.40:0/3668238892 >> v2:172.21.15.184:3300/0 conn(0x7f14380d1770 0x7f14380d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14380b2050 con 0x7f14380d2130 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1428001bc0 con 0x7f14380d2130 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 >> v1:172.21.15.40:6789/0 conn(0x7f14380d2130 legacy=0x7f14380d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14200039b0 0x7f1420003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1420004800 0x7f1420004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 0x7f14200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14380d0e20 con 0x7f14200039b0 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14380b1d70 con 0x7f1420004800 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f142000b9c0 con 0x7f143815f3a0 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14200039b0 0x7f1420003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143d7bb640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1420004800 0x7f1420004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143e7bd640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 0x7f14200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143dfbc640 1 -- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 msgr2=0x7f14200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 0x7f14200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143dfbc640 1 -- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1420004800 msgr2=0x7f1420004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1420004800 0x7f1420004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143dfbc640 1 -- 172.21.15.40:0/3668238892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1420003520 con 0x7f14200039b0 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143d7bb640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1420004800 0x7f1420004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:00.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.799+0000 7f143e7bd640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f143815f3a0 0x7f14200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.800+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14200039b0 0x7f1420003dd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1430009ac0 tx=0x7f1420005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b7193db0d90e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.800+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430036070 con 0x7f14200039b0 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.800+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1430031400 con 0x7f14200039b0 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.800+0000 7f143cfba640 1 -- 172.21.15.40:0/3668238892 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430009690 con 0x7f14200039b0 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.801+0000 7f1440220640 1 -- 172.21.15.40:0/3668238892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14200039b0 msgr2=0x7f1420003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.801+0000 7f1440220640 1 --2- 172.21.15.40:0/3668238892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14200039b0 0x7f1420003dd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1430009ac0 tx=0x7f1420005830 comp rx=0 tx=0).stop 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.801+0000 7f143e7bd640 1 -- 172.21.15.40:0/3668238892 reap_dead start 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.801+0000 7f1440220640 1 -- 172.21.15.40:0/3668238892 shutdown_connections 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.801+0000 7f1440220640 1 -- 172.21.15.40:0/3668238892 >> 172.21.15.40:0/3668238892 conn(0x7f14380d4f90 msgr2=0x7f14380d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.801+0000 7f1440220640 1 -- 172.21.15.40:0/3668238892 shutdown_connections 2024-04-26T21:38:00.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.801+0000 7f1440220640 1 -- 172.21.15.40:0/3668238892 wait complete. 2024-04-26T21:38:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.802+0000 7f1440220640 1 Processor -- start 2024-04-26T21:38:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.802+0000 7f1440220640 1 -- start start 2024-04-26T21:38:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f1440220640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14200039b0 0x7f14380ba210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f1440220640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f143815d210 0x7f143815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f1440220640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14380cd4c0 0x7f14380cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f1440220640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f143815f900 con 0x7f14380cd4c0 2024-04-26T21:38:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f1440220640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f143815f780 con 0x7f143815d210 2024-04-26T21:38:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f1440220640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14380b3860 con 0x7f14200039b0 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143e7bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14380cd4c0 0x7f14380cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143e7bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14380cd4c0 0x7f14380cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56730/0 (socket says 172.21.15.40:56730) 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143e7bd640 1 -- 172.21.15.40:0/1789217058 learned_addr learned my addr 172.21.15.40:0/1789217058 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143dfbc640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14200039b0 0x7f14380ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143d7bb640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f143815d210 0x7f143815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143e7bd640 1 -- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14200039b0 msgr2=0x7f14380ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143e7bd640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14200039b0 0x7f14380ba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143e7bd640 1 -- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f143815d210 msgr2=0x7f143815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143e7bd640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f143815d210 0x7f143815c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.803+0000 7f143e7bd640 1 -- 172.21.15.40:0/1789217058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14380ba750 con 0x7f14380cd4c0 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.804+0000 7f143e7bd640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14380cd4c0 0x7f14380cd940 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f14280013d0 tx=0x7f142800a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.804+0000 7f14367fc640 1 -- 172.21.15.40:0/1789217058 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142800ac20 con 0x7f14380cd4c0 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.804+0000 7f14367fc640 1 -- 172.21.15.40:0/1789217058 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f142800ade0 con 0x7f14380cd4c0 2024-04-26T21:38:00.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.804+0000 7f1440220640 1 -- 172.21.15.40:0/1789217058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14380baa00 con 0x7f14380cd4c0 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.804+0000 7f14367fc640 1 -- 172.21.15.40:0/1789217058 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428014670 con 0x7f14380cd4c0 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.804+0000 7f1440220640 1 -- 172.21.15.40:0/1789217058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14380b7f90 con 0x7f14380cd4c0 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.805+0000 7f14367fc640 1 -- 172.21.15.40:0/1789217058 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1428014810 con 0x7f14380cd4c0 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.806+0000 7f14367fc640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f142406c8f0 0x7f142406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.806+0000 7f14367fc640 1 -- 172.21.15.40:0/1789217058 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f142808bc60 con 0x7f14380cd4c0 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.806+0000 7f143dfbc640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f142406c8f0 0x7f142406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.807+0000 7f143dfbc640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f142406c8f0 0x7f142406edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1430033000 tx=0x7f1430008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.807+0000 7f1443012740 1 Processor -- start 2024-04-26T21:38:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.807+0000 7f1443012740 1 -- start start 2024-04-26T21:38:00.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f1443012740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46eef0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f1443012740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeae10 0x55a46eeeeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f1443012740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eeef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f1443012740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a46ee54d80 con 0x55a46eeeff80 2024-04-26T21:38:00.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f1443012740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a46ee0e8b0 con 0x55a46eeef0c0 2024-04-26T21:38:00.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f1443012740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a46ee50d40 con 0x55a46eeeae10 2024-04-26T21:38:00.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46eef0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46eef0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56742/0 (socket says 172.21.15.40:56742) 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 -- 172.21.15.40:0/724578438 learned_addr learned my addr 172.21.15.40:0/724578438 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143d7bb640 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeae10 0x55a46eeeeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143e7bd640 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eeef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 -- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeae10 msgr2=0x55a46eeeeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeae10 0x55a46eeeeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 -- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 msgr2=0x55a46eeef500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eeef500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 -- 172.21.15.40:0/724578438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46eddfa40 con 0x55a46eeeff80 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143e7bd640 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eeef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f143dfbc640 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46eef0360 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f143002abb0 tx=0x7f143002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1204601372b4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f141f7fe640 1 -- 172.21.15.40:0/724578438 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430040040 con 0x55a46eeeff80 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f141f7fe640 1 -- 172.21.15.40:0/724578438 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1430003d60 con 0x55a46eeeff80 2024-04-26T21:38:00.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.808+0000 7f141f7fe640 1 -- 172.21.15.40:0/724578438 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430036040 con 0x55a46eeeff80 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 -- 172.21.15.40:0/724578438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 msgr2=0x55a46eef0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46eef0360 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f143002abb0 tx=0x7f143002e000 comp rx=0 tx=0).stop 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 -- 172.21.15.40:0/724578438 shutdown_connections 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eeef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeae10 0x55a46eeeeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 --2- 172.21.15.40:0/724578438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46eef0360 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 -- 172.21.15.40:0/724578438 >> 172.21.15.40:0/724578438 conn(0x55a46eeed080 msgr2=0x55a46ef35b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 -- 172.21.15.40:0/724578438 shutdown_connections 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.809+0000 7f1443012740 1 -- 172.21.15.40:0/724578438 wait complete. 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.810+0000 7f1443012740 1 -- start start 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.810+0000 7f1443012740 1 Processor -- start 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f1443012740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46ef0f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f1443012740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46ef10b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f1443012740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 0x55a46ef38f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f1443012740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a46ef1bb10 con 0x55a46eeeff80 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f1443012740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a46eddbd10 con 0x55a46eeef0c0 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f1443012740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a46ef1af90 con 0x55a46eefbce0 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f1443012740 10 client.0 _ll_register_callbacks cb 0x55a46ee08210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46ef10b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46ef10b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56748/0 (socket says 172.21.15.40:56748) 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 -- 172.21.15.40:0/3316811441 learned_addr learned my addr 172.21.15.40:0/3316811441 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143e7bd640 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 0x55a46ef38f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46ef0f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 -- 172.21.15.40:0/3316811441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 msgr2=0x55a46ef38f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 0x55a46ef38f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 -- 172.21.15.40:0/3316811441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 msgr2=0x55a46ef0f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46ef0f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 -- 172.21.15.40:0/3316811441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46ec307d0 con 0x55a46eeeff80 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f143d7bb640 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46ef10b10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f142c009d50 tx=0x7f142c0167a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b0a67656418743 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f141cff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f141cff9640 1 -- 172.21.15.40:0/3316811441 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c017640 con 0x55a46eeeff80 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f141cff9640 1 -- 172.21.15.40:0/3316811441 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f142c021d00 con 0x55a46eeeff80 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.811+0000 7f1443012740 1 -- 172.21.15.40:0/3316811441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1430003d60 con 0x55a46eeeff80 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f141cff9640 1 -- 172.21.15.40:0/3316811441 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f142c009e40 con 0x55a46eeeff80 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f1443012740 10 client.6345 fetch_fsmap learned FSMap version 37 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f1443012740 1 -- 172.21.15.40:0/3316811441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55a46eb663f0 con 0x55a46eeeff80 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f141cff9640 1 -- 172.21.15.40:0/3316811441 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1430003d60 con 0x55a46eeeff80 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f141cff9640 1 -- 172.21.15.40:0/3316811441 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) ==== 3486+0+0 (secure 0 0 0) 0x7f142c028050 con 0x55a46eeeff80 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f1443012740 10 client.6345 fetch_fsmap finished waiting for FSMap version 37 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f1443012740 10 client.6345 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T21:38:00.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f1443012740 20 client.6345 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f1443012740 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] conn(0x55a46ef3d3e0 0x55a46ef3d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f1443012740 4 client.6345 mds_command: new command op to 6306 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f1443012740 1 -- 172.21.15.40:0/3316811441 --> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a46ee62990 con 0x55a46ef3d3e0 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] conn(0x55a46ef3d3e0 0x55a46ef3d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f141cff9640 10 client.6345 ms_handle_connect on v2:172.21.15.40:6832/4223063167 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.812+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] conn(0x55a46ef3d3e0 0x55a46ef3d820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c721526f9e2a99 server_cookie=f03013a907f8f25c in_seq=0 out_seq=0 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f141cff9640 1 -- 172.21.15.40:0/3316811441 <== mds.0 v2:172.21.15.40:6832/4223063167 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a46ee62990 con 0x55a46ef3d3e0 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f141cff9640 10 client.6345 handle_command_reply: tid=0 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 client.6345 shutdown 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 -- 172.21.15.40:0/3316811441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 msgr2=0x55a46ef10b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46ef10b10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f142c009d50 tx=0x7f142c0167a0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 -- 172.21.15.40:0/3316811441 shutdown_connections 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] conn(0x55a46ef3d3e0 0x55a46ef3d820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 0x55a46ef38f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f141cff9640 0 client.6345 ms_handle_reset on v2:172.21.15.40:6832/4223063167 2024-04-26T21:38:00.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeeff80 0x55a46ef10b10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 --2- 172.21.15.40:0/3316811441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46ef0f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 -- 172.21.15.40:0/3316811441 >> 172.21.15.40:0/3316811441 conn(0x55a46eeeae10 msgr2=0x55a46eef0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 -- 172.21.15.40:0/3316811441 shutdown_connections 2024-04-26T21:38:00.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 1 -- 172.21.15.40:0/3316811441 wait complete. 2024-04-26T21:38:00.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.813+0000 7f1443012740 20 client.6345 trim_cache size 0 max 16384 2024-04-26T21:38:00.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.828+0000 7f1443012740 1 Processor -- start 2024-04-26T21:38:00.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.828+0000 7f1443012740 1 -- start start 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f1443012740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeef0c0 0x55a46eeea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f1443012740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeeff80 0x55a46efa6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f1443012740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 0x55a46eefad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f1443012740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14382026c0 con 0x55a46eeef0c0 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f1443012740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1438201720 con 0x55a46eeeff80 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f1443012740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1438200c10 con 0x55a46eefbce0 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f143dfbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeef0c0 0x55a46eeea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f143dfbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeef0c0 0x55a46eeea280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56762/0 (socket says 172.21.15.40:56762) 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f143dfbc640 1 -- 172.21.15.40:0/2202817425 learned_addr learned my addr 172.21.15.40:0/2202817425 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f143dfbc640 1 -- 172.21.15.40:0/2202817425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 msgr2=0x55a46eefad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f143dfbc640 1 --2- 172.21.15.40:0/2202817425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 0x55a46eefad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f143dfbc640 1 -- 172.21.15.40:0/2202817425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeeff80 msgr2=0x55a46efa6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.829+0000 7f143dfbc640 1 --2- 172.21.15.40:0/2202817425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeeff80 0x55a46efa6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f143dfbc640 1 -- 172.21.15.40:0/2202817425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46eb260f0 con 0x55a46eeef0c0 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f143dfbc640 1 --2- 172.21.15.40:0/2202817425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeef0c0 0x55a46eeea280 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1430006f80 tx=0x7f1430009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de740873ae2994 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f141effd640 1 -- 172.21.15.40:0/2202817425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143005a070 con 0x55a46eeef0c0 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f141effd640 1 -- 172.21.15.40:0/2202817425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1430055b50 con 0x55a46eeef0c0 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f141effd640 1 -- 172.21.15.40:0/2202817425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430036070 con 0x55a46eeef0c0 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 -- 172.21.15.40:0/2202817425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeef0c0 msgr2=0x55a46eeea280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 --2- 172.21.15.40:0/2202817425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeef0c0 0x55a46eeea280 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1430006f80 tx=0x7f1430009190 comp rx=0 tx=0).stop 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 -- 172.21.15.40:0/2202817425 shutdown_connections 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 --2- 172.21.15.40:0/2202817425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eefbce0 0x55a46eefad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 --2- 172.21.15.40:0/2202817425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeeff80 0x55a46efa6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 --2- 172.21.15.40:0/2202817425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eeef0c0 0x55a46eeea280 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 -- 172.21.15.40:0/2202817425 >> 172.21.15.40:0/2202817425 conn(0x55a46eeeae10 msgr2=0x55a46ef0f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 -- 172.21.15.40:0/2202817425 shutdown_connections 2024-04-26T21:38:00.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.830+0000 7f1443012740 1 -- 172.21.15.40:0/2202817425 wait complete. 2024-04-26T21:38:00.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.831+0000 7f1443012740 1 -- start start 2024-04-26T21:38:00.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.831+0000 7f1443012740 1 Processor -- start 2024-04-26T21:38:00.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f1443012740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eefb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f1443012740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeff80 0x55a46ef13160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f1443012740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eefbce0 0x55a46ef136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f1443012740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a46ef3c580 con 0x55a46eefbce0 2024-04-26T21:38:00.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f1443012740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1438201100 con 0x55a46eeef0c0 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f1443012740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a46eeea4a0 con 0x55a46eeeff80 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f1443012740 10 client.0 _ll_register_callbacks cb 0x7f142c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eefbce0 0x55a46ef136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eefbce0 0x55a46ef136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56778/0 (socket says 172.21.15.40:56778) 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 -- 172.21.15.40:0/3097394166 learned_addr learned my addr 172.21.15.40:0/3097394166 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143d7bb640 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeff80 0x55a46ef13160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eefb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 -- 172.21.15.40:0/3097394166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeff80 msgr2=0x55a46ef13160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeff80 0x55a46ef13160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 -- 172.21.15.40:0/3097394166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 msgr2=0x55a46eefb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eefb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 -- 172.21.15.40:0/3097394166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46ee154b0 con 0x55a46eefbce0 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f143e7bd640 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eefbce0 0x55a46ef136a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f14280595c0 tx=0x7f142805a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f0059d4841f5bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.832+0000 7f141cff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f141cff9640 1 -- 172.21.15.40:0/3097394166 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428088070 con 0x55a46eefbce0 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f141cff9640 1 -- 172.21.15.40:0/3097394166 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f14280600b0 con 0x55a46eefbce0 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 1 -- 172.21.15.40:0/3097394166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1430055b50 con 0x55a46eefbce0 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f141cff9640 1 -- 172.21.15.40:0/3097394166 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1428055f30 con 0x55a46eefbce0 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 10 client.6351 fetch_fsmap learned FSMap version 37 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 1 -- 172.21.15.40:0/3097394166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55a46efa4560 con 0x55a46eefbce0 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f141cff9640 1 -- 172.21.15.40:0/3097394166 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1430055b50 con 0x55a46eefbce0 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f141cff9640 1 -- 172.21.15.40:0/3097394166 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) ==== 3486+0+0 (secure 0 0 0) 0x7f1428058d10 con 0x55a46eefbce0 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 10 client.6351 fetch_fsmap finished waiting for FSMap version 37 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 10 client.6351 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 20 client.6351 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] conn(0x55a46eee9cc0 0x55a46efa1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 4 client.6351 mds_command: new command op to 6306 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T21:38:00.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f1443012740 1 -- 172.21.15.40:0/3097394166 --> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a46ee5c800 con 0x55a46eee9cc0 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.833+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] conn(0x55a46eee9cc0 0x55a46efa1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f141cff9640 10 client.6351 ms_handle_connect on v2:172.21.15.40:6832/4223063167 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f143dfbc640 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] conn(0x55a46eee9cc0 0x55a46efa1f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e1cbe6809e4656 server_cookie=34d911515a9c6f91 in_seq=0 out_seq=0 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f141cff9640 1 -- 172.21.15.40:0/3097394166 <== mds.0 v2:172.21.15.40:6832/4223063167 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a46ee5c800 con 0x55a46eee9cc0 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f141cff9640 10 client.6351 handle_command_reply: tid=0 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 client.6351 shutdown 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 -- 172.21.15.40:0/3097394166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eefbce0 msgr2=0x55a46ef136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eefbce0 0x55a46ef136a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f14280595c0 tx=0x7f142805a120 comp rx=0 tx=0).stop 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 -- 172.21.15.40:0/3097394166 shutdown_connections 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a46eefbce0 0x55a46ef136a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.40:6832/4223063167,v1:172.21.15.40:6834/4223063167] conn(0x55a46eee9cc0 0x55a46efa1f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a46eeeff80 0x55a46ef13160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 --2- 172.21.15.40:0/3097394166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a46eeef0c0 0x55a46eefb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f141cff9640 0 client.6351 ms_handle_reset on v2:172.21.15.40:6832/4223063167 2024-04-26T21:38:00.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 -- 172.21.15.40:0/3097394166 >> 172.21.15.40:0/3097394166 conn(0x55a46eeeae10 msgr2=0x55a46eef0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 -- 172.21.15.40:0/3097394166 shutdown_connections 2024-04-26T21:38:00.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 1 -- 172.21.15.40:0/3097394166 wait complete. 2024-04-26T21:38:00.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.834+0000 7f1443012740 20 client.6351 trim_cache size 0 max 16384 2024-04-26T21:38:00.841 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:38:00.841 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-04-26T21:38:00.841 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0 2024-04-26T21:38:00.841 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:38:00.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 -- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f142406c8f0 msgr2=0x7f142406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f142406c8f0 0x7f142406edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1430033000 tx=0x7f1430008030 comp rx=0 tx=0).stop 2024-04-26T21:38:00.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 -- 172.21.15.40:0/1789217058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14380cd4c0 msgr2=0x7f14380cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14380cd4c0 0x7f14380cd940 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f14280013d0 tx=0x7f142800a590 comp rx=0 tx=0).stop 2024-04-26T21:38:00.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 -- 172.21.15.40:0/1789217058 shutdown_connections 2024-04-26T21:38:00.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f142406c8f0 0x7f142406edb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14380cd4c0 0x7f14380cd940 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f143815d210 0x7f143815c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 --2- 172.21.15.40:0/1789217058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14200039b0 0x7f14380ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 -- 172.21.15.40:0/1789217058 >> 172.21.15.40:0/1789217058 conn(0x7f14380d4f90 msgr2=0x7f143815e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 -- 172.21.15.40:0/1789217058 shutdown_connections 2024-04-26T21:38:00.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.835+0000 7f1440220640 1 -- 172.21.15.40:0/1789217058 wait complete. 2024-04-26T21:38:00.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:00.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.946+0000 7f99289c6640 1 Processor -- start 2024-04-26T21:38:00.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.946+0000 7f99289c6640 1 -- start start 2024-04-26T21:38:00.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.946+0000 7f99289c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99240d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.946+0000 7f99289c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99240b5f20 0x7f99240d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.946+0000 7f99289c6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99240b7e50 con 0x7f99240d39f0 2024-04-26T21:38:00.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.946+0000 7f99289c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f99240d0580 con 0x7f99240b5f20 2024-04-26T21:38:00.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.946+0000 7f99289c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99240d1800 con 0x7f9924162880 2024-04-26T21:38:00.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f991bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99240b5f20 0x7f99240d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f991bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99240b5f20 0x7f99240d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56788/0 (socket says 172.21.15.40:56788) 2024-04-26T21:38:00.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f9922d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f99240d39f0 0x7f99240b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51834/0 (socket says 172.21.15.40:51834) 2024-04-26T21:38:00.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f991bfff640 1 -- 172.21.15.40:0/3814510404 learned_addr learned my addr 172.21.15.40:0/3814510404 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:00.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f992259c640 1 --2- 172.21.15.40:0/3814510404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99240d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f991bfff640 1 -- 172.21.15.40:0/3814510404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 msgr2=0x7f99240d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:00.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f991bfff640 1 --2- 172.21.15.40:0/3814510404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99240d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f991bfff640 1 -- 172.21.15.40:0/3814510404 >> v1:172.21.15.40:6789/0 conn(0x7f99240d39f0 legacy=0x7f99240b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f991bfff640 1 -- 172.21.15.40:0/3814510404 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99240d07d0 con 0x7f99240b5f20 2024-04-26T21:38:00.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.947+0000 7f9921d9b640 1 -- 172.21.15.40:0/3814510404 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3227462100 0 0) 0x7f99240b7e50 con 0x7f99240d39f0 2024-04-26T21:38:00.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f991bfff640 1 --2- 172.21.15.40:0/3814510404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240b5f20 0x7f99240d34b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f990c009870 tx=0x7f990c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4380031607ebe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f9921d9b640 1 -- 172.21.15.40:0/3814510404 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c037070 con 0x7f99240b5f20 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f9921d9b640 1 -- 172.21.15.40:0/3814510404 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c032440 con 0x7f99240b5f20 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f9921d9b640 1 -- 172.21.15.40:0/3814510404 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c032810 con 0x7f99240b5f20 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f99289c6640 1 -- 172.21.15.40:0/3814510404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240b5f20 msgr2=0x7f99240d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f99289c6640 1 --2- 172.21.15.40:0/3814510404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240b5f20 0x7f99240d34b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f990c009870 tx=0x7f990c02a270 comp rx=0 tx=0).stop 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f99289c6640 1 -- 172.21.15.40:0/3814510404 shutdown_connections 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f99289c6640 1 --2- 172.21.15.40:0/3814510404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240b5f20 0x7f99240d34b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f99289c6640 1 --2- 172.21.15.40:0/3814510404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99240d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f99289c6640 1 -- 172.21.15.40:0/3814510404 >> 172.21.15.40:0/3814510404 conn(0x7f99240d4f60 msgr2=0x7f99240d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f99289c6640 1 -- 172.21.15.40:0/3814510404 shutdown_connections 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.948+0000 7f99289c6640 1 -- 172.21.15.40:0/3814510404 wait complete. 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.950+0000 7f99289c6640 1 Processor -- start 2024-04-26T21:38:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.950+0000 7f99289c6640 1 -- start start 2024-04-26T21:38:00.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.950+0000 7f99289c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99240b5f20 0x7f992415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.950+0000 7f99289c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240d39f0 0x7f99240ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.950+0000 7f99289c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99241f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.950+0000 7f99289c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9924162c60 con 0x7f99240d39f0 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.950+0000 7f99289c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99240d0d90 con 0x7f99240b5f20 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.950+0000 7f99289c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9924162de0 con 0x7f9924162880 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240d39f0 0x7f99240ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240d39f0 0x7f99240ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56800/0 (socket says 172.21.15.40:56800) 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 -- 172.21.15.40:0/1591373881 learned_addr learned my addr 172.21.15.40:0/1591373881 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f9922d9d640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99241f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f992259c640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99240b5f20 0x7f992415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 -- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 msgr2=0x7f99241f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99241f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 -- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99240b5f20 msgr2=0x7f992415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99240b5f20 0x7f992415d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 -- 172.21.15.40:0/1591373881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99240b1d60 con 0x7f99240d39f0 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f9922d9d640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99241f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f992259c640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99240b5f20 0x7f992415d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991bfff640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240d39f0 0x7f99240ceb70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f990c02a2b0 tx=0x7f990c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991affd640 1 -- 172.21.15.40:0/1591373881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c041020 con 0x7f99240d39f0 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991affd640 1 -- 172.21.15.40:0/1591373881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c0306c0 con 0x7f99240d39f0 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99240b2040 con 0x7f99240d39f0 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f991affd640 1 -- 172.21.15.40:0/1591373881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c037040 con 0x7f99240d39f0 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.951+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992415d930 con 0x7f99240d39f0 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.952+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99240694f0 con 0x7f99240d39f0 2024-04-26T21:38:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.953+0000 7f991affd640 1 -- 172.21.15.40:0/1591373881 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f990c044020 con 0x7f99240d39f0 2024-04-26T21:38:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.953+0000 7f991affd640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990406c8f0 0x7f990406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.953+0000 7f991affd640 1 -- 172.21.15.40:0/1591373881 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f990c0ac530 con 0x7f99240d39f0 2024-04-26T21:38:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.953+0000 7f992259c640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990406c8f0 0x7f990406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.954+0000 7f992259c640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990406c8f0 0x7f990406edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9914006fd0 tx=0x7f9914008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:00.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:00.958+0000 7f991affd640 1 -- 172.21.15.40:0/1591373881 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f990c04b030 con 0x7f99240d39f0 2024-04-26T21:38:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.185+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9924074590 con 0x7f99240d39f0 2024-04-26T21:38:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.185+0000 7f991affd640 1 -- 172.21.15.40:0/1591373881 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5175 (secure 0 0 0) 0x7f990c07a740 con 0x7f99240d39f0 2024-04-26T21:38:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:01.187 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:38:00.322094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"c","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6834/4223063167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4223063167},{"type":"v1","addr":"172.21.15.40:6834","nonce":4223063167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-26T21:38:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 37 2024-04-26T21:38:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990406c8f0 msgr2=0x7f990406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990406c8f0 0x7f990406edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9914006fd0 tx=0x7f9914008040 comp rx=0 tx=0).stop 2024-04-26T21:38:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240d39f0 msgr2=0x7f99240ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240d39f0 0x7f99240ceb70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f990c02a2b0 tx=0x7f990c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 shutdown_connections 2024-04-26T21:38:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f990406c8f0 0x7f990406edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9924162880 0x7f99241f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99240d39f0 0x7f99240ceb70 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 --2- 172.21.15.40:0/1591373881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99240b5f20 0x7f992415d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 >> 172.21.15.40:0/1591373881 conn(0x7f99240d4f60 msgr2=0x7f99240d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 shutdown_connections 2024-04-26T21:38:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.188+0000 7f99289c6640 1 -- 172.21.15.40:0/1591373881 wait complete. 2024-04-26T21:38:01.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:01.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.299+0000 7f7950d40640 1 Processor -- start 2024-04-26T21:38:01.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.300+0000 7f7950d40640 1 -- start start 2024-04-26T21:38:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.300+0000 7f7950d40640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f794c0b2340 0x7f794c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.300+0000 7f7950d40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f794c0cd930 0x7f794c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.300+0000 7f7950d40640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f794c0d10d0 con 0x7f794c162810 2024-04-26T21:38:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.300+0000 7f7950d40640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f794c0d0750 con 0x7f794c0cd930 2024-04-26T21:38:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.300+0000 7f7950d40640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f794c0d16a0 con 0x7f794c0b2340 2024-04-26T21:38:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.300+0000 7f794ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f794c0cd930 0x7f794c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f794ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f794c0cd930 0x7f794c0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56804/0 (socket says 172.21.15.40:56804) 2024-04-26T21:38:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f794a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f794c162810 0x7f794c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51840/0 (socket says 172.21.15.40:51840) 2024-04-26T21:38:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f794ad9d640 1 -- 172.21.15.40:0/2959022442 learned_addr learned my addr 172.21.15.40:0/2959022442 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f7943fff640 1 --2- 172.21.15.40:0/2959022442 >> v2:172.21.15.184:3300/0 conn(0x7f794c0b2340 0x7f794c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f794ad9d640 1 -- 172.21.15.40:0/2959022442 >> v2:172.21.15.184:3300/0 conn(0x7f794c0b2340 msgr2=0x7f794c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f794ad9d640 1 --2- 172.21.15.40:0/2959022442 >> v2:172.21.15.184:3300/0 conn(0x7f794c0b2340 0x7f794c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f794ad9d640 1 -- 172.21.15.40:0/2959022442 >> v1:172.21.15.40:6789/0 conn(0x7f794c162810 legacy=0x7f794c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f794ad9d640 1 -- 172.21.15.40:0/2959022442 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c0d1370 con 0x7f794c0cd930 2024-04-26T21:38:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.301+0000 7f7949d9b640 1 -- 172.21.15.40:0/2959022442 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 740310958 0 0) 0x7f794c0d10d0 con 0x7f794c162810 2024-04-26T21:38:01.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f794ad9d640 1 --2- 172.21.15.40:0/2959022442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0d1b30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f794c0cd0a0 tx=0x7f794402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18ffea542f1b3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:01.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7949d9b640 1 -- 172.21.15.40:0/2959022442 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944036070 con 0x7f794c0cd930 2024-04-26T21:38:01.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7949d9b640 1 -- 172.21.15.40:0/2959022442 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7944031600 con 0x7f794c0cd930 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7949d9b640 1 -- 172.21.15.40:0/2959022442 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440319d0 con 0x7f794c0cd930 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7950d40640 1 -- 172.21.15.40:0/2959022442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 msgr2=0x7f794c0d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7950d40640 1 --2- 172.21.15.40:0/2959022442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0d1b30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f794c0cd0a0 tx=0x7f794402a730 comp rx=0 tx=0).stop 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7950d40640 1 -- 172.21.15.40:0/2959022442 shutdown_connections 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7950d40640 1 --2- 172.21.15.40:0/2959022442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0d1b30 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7950d40640 1 --2- 172.21.15.40:0/2959022442 >> v2:172.21.15.184:3300/0 conn(0x7f794c0b2340 0x7f794c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7950d40640 1 -- 172.21.15.40:0/2959022442 >> 172.21.15.40:0/2959022442 conn(0x7f794c0d2da0 msgr2=0x7f794c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7950d40640 1 -- 172.21.15.40:0/2959022442 shutdown_connections 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.302+0000 7f7950d40640 1 -- 172.21.15.40:0/2959022442 wait complete. 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7950d40640 1 Processor -- start 2024-04-26T21:38:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7950d40640 1 -- start start 2024-04-26T21:38:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7950d40640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c0b2340 0x7f794c1f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7950d40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7950d40640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c162810 0x7f794c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7950d40640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0d2070 con 0x7f794c0cd930 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7950d40640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0d0470 con 0x7f794c0b2340 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7950d40640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f794c0d21f0 con 0x7f794c162810 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7943fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7943fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56812/0 (socket says 172.21.15.40:56812) 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.304+0000 7f7943fff640 1 -- 172.21.15.40:0/2391632783 learned_addr learned my addr 172.21.15.40:0/2391632783 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f794a59c640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c0b2340 0x7f794c1f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f794ad9d640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c162810 0x7f794c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7943fff640 1 -- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c162810 msgr2=0x7f794c0c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7943fff640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c162810 0x7f794c0c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7943fff640 1 -- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c0b2340 msgr2=0x7f794c1f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7943fff640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c0b2340 0x7f794c1f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7943fff640 1 -- 172.21.15.40:0/2391632783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c0ca050 con 0x7f794c0cd930 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7943fff640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0c8070 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f793400e9b0 tx=0x7f793400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7942ffd640 1 -- 172.21.15.40:0/2391632783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934017070 con 0x7f794c0cd930 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7942ffd640 1 -- 172.21.15.40:0/2391632783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793400c9a0 con 0x7f794c0cd930 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7942ffd640 1 -- 172.21.15.40:0/2391632783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793400cd90 con 0x7f794c0cd930 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794c0ca330 con 0x7f794c0cd930 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.305+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794c0ca8f0 con 0x7f794c0cd930 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.306+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794c0694f0 con 0x7f794c0cd930 2024-04-26T21:38:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.307+0000 7f7942ffd640 1 -- 172.21.15.40:0/2391632783 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f793401a070 con 0x7f794c0cd930 2024-04-26T21:38:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.307+0000 7f7942ffd640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f792c06c8f0 0x7f792c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.307+0000 7f7942ffd640 1 -- 172.21.15.40:0/2391632783 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f793408bc30 con 0x7f794c0cd930 2024-04-26T21:38:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.307+0000 7f794a59c640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f792c06c8f0 0x7f792c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.308+0000 7f794a59c640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f792c06c8f0 0x7f792c06edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f793c007fd0 tx=0x7f793c007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:01.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.312+0000 7f7942ffd640 1 -- 172.21.15.40:0/2391632783 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7934059880 con 0x7f794c0cd930 2024-04-26T21:38:01.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.536+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f794c0b3320 con 0x7f794c0cd930 2024-04-26T21:38:01.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.537+0000 7f7942ffd640 1 -- 172.21.15.40:0/2391632783 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5178 (secure 0 0 0) 0x7f793405db80 con 0x7f794c0cd930 2024-04-26T21:38:01.538 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:01.538 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:38:00.331181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.40:6834/4223063167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4223063167},{"type":"v1","addr":"172.21.15.40:6834","nonce":4223063167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-26T21:38:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.538+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f792c06c8f0 msgr2=0x7f792c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.538+0000 7f7950d40640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f792c06c8f0 0x7f792c06edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f793c007fd0 tx=0x7f793c007de0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.538+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 msgr2=0x7f794c0c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.538+0000 7f7950d40640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0c8070 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f793400e9b0 tx=0x7f793400ee70 comp rx=0 tx=0).stop 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.538+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 shutdown_connections 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.538+0000 7f7950d40640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f792c06c8f0 0x7f792c06edb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.538+0000 7f7950d40640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c162810 0x7f794c0c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.539+0000 7f7950d40640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c0cd930 0x7f794c0c8070 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.539+0000 7f7950d40640 1 --2- 172.21.15.40:0/2391632783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c0b2340 0x7f794c1f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.539+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 >> 172.21.15.40:0/2391632783 conn(0x7f794c0d2da0 msgr2=0x7f794c070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.539+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 shutdown_connections 2024-04-26T21:38:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.539+0000 7f7950d40640 1 -- 172.21.15.40:0/2391632783 wait complete. 2024-04-26T21:38:01.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:01.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.650+0000 7f4526285640 1 Processor -- start 2024-04-26T21:38:01.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.650+0000 7f4526285640 1 -- start start 2024-04-26T21:38:01.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f4526285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4520162880 0x7f45200d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f4526285640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f45200b5f20 0x7f45200d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f4526285640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45200b7e50 con 0x7f45200d39f0 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f4526285640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f45200d0580 con 0x7f45200b5f20 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f4526285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45200d1800 con 0x7f4520162880 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f451f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f45200b5f20 0x7f45200d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f451f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f45200b5f20 0x7f45200d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56814/0 (socket says 172.21.15.40:56814) 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f451f7fe640 1 -- 172.21.15.40:0/1513229425 learned_addr learned my addr 172.21.15.40:0/1513229425 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f4524822640 1 --1- 172.21.15.40:0/1513229425 >> v1:172.21.15.40:6789/0 conn(0x7f45200d39f0 0x7f45200b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51846/0 (socket says 172.21.15.40:51846) 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f451ffff640 1 --2- 172.21.15.40:0/1513229425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4520162880 0x7f45200d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f451f7fe640 1 -- 172.21.15.40:0/1513229425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4520162880 msgr2=0x7f45200d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.651+0000 7f451f7fe640 1 --2- 172.21.15.40:0/1513229425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4520162880 0x7f45200d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.652+0000 7f451f7fe640 1 -- 172.21.15.40:0/1513229425 >> v1:172.21.15.40:6789/0 conn(0x7f45200d39f0 legacy=0x7f45200b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.652+0000 7f451f7fe640 1 -- 172.21.15.40:0/1513229425 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45200d07d0 con 0x7f45200b5f20 2024-04-26T21:38:01.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.652+0000 7f451f7fe640 1 --2- 172.21.15.40:0/1513229425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45200d34b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4510009870 tx=0x7f451002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894b0db917b9fad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.652+0000 7f451effd640 1 -- 172.21.15.40:0/1513229425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510037070 con 0x7f45200b5f20 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.652+0000 7f451effd640 1 -- 172.21.15.40:0/1513229425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4510032440 con 0x7f45200b5f20 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.652+0000 7f451effd640 1 -- 172.21.15.40:0/1513229425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510032810 con 0x7f45200b5f20 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.653+0000 7f4526285640 1 -- 172.21.15.40:0/1513229425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 msgr2=0x7f45200d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.653+0000 7f4526285640 1 --2- 172.21.15.40:0/1513229425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45200d34b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4510009870 tx=0x7f451002a270 comp rx=0 tx=0).stop 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.653+0000 7f4526285640 1 -- 172.21.15.40:0/1513229425 shutdown_connections 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.653+0000 7f4526285640 1 --2- 172.21.15.40:0/1513229425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45200d34b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.653+0000 7f4526285640 1 --2- 172.21.15.40:0/1513229425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4520162880 0x7f45200d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.653+0000 7f4526285640 1 -- 172.21.15.40:0/1513229425 >> 172.21.15.40:0/1513229425 conn(0x7f45200d4f60 msgr2=0x7f45200d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.653+0000 7f4526285640 1 -- 172.21.15.40:0/1513229425 shutdown_connections 2024-04-26T21:38:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.653+0000 7f4526285640 1 -- 172.21.15.40:0/1513229425 wait complete. 2024-04-26T21:38:01.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.654+0000 7f4526285640 1 Processor -- start 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.654+0000 7f4526285640 1 -- start start 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f4526285640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45201519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f4526285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45200d39f0 0x7f45200a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f4526285640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4520162880 0x7f45200a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f4526285640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4520162c60 con 0x7f45200b5f20 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f4526285640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45200d0d90 con 0x7f4520162880 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f4526285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4520162de0 con 0x7f45200d39f0 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45201519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45201519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56824/0 (socket says 172.21.15.40:56824) 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451ffff640 1 -- 172.21.15.40:0/2072995825 learned_addr learned my addr 172.21.15.40:0/2072995825 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451f7fe640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45200d39f0 0x7f45200a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f4524822640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4520162880 0x7f45200a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451ffff640 1 -- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45200d39f0 msgr2=0x7f45200a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451ffff640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45200d39f0 0x7f45200a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451ffff640 1 -- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4520162880 msgr2=0x7f45200a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451ffff640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4520162880 0x7f45200a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f451ffff640 1 -- 172.21.15.40:0/2072995825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45200b5720 con 0x7f45200b5f20 2024-04-26T21:38:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.655+0000 7f4524822640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4520162880 0x7f45200a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:01.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.656+0000 7f451ffff640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45201519a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f450800d800 tx=0x7f450800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:01.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.656+0000 7f451cff9640 1 -- 172.21.15.40:0/2072995825 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450800b6c0 con 0x7f45200b5f20 2024-04-26T21:38:01.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.656+0000 7f451cff9640 1 -- 172.21.15.40:0/2072995825 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4508019070 con 0x7f45200b5f20 2024-04-26T21:38:01.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.656+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4520151ee0 con 0x7f45200b5f20 2024-04-26T21:38:01.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.656+0000 7f451cff9640 1 -- 172.21.15.40:0/2072995825 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450800ba80 con 0x7f45200b5f20 2024-04-26T21:38:01.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.656+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452014f670 con 0x7f45200b5f20 2024-04-26T21:38:01.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.657+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45200694f0 con 0x7f45200b5f20 2024-04-26T21:38:01.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.657+0000 7f451cff9640 1 -- 172.21.15.40:0/2072995825 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f450800bc20 con 0x7f45200b5f20 2024-04-26T21:38:01.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.658+0000 7f451cff9640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f450006c8f0 0x7f450006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:01.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.658+0000 7f451cff9640 1 -- 172.21.15.40:0/2072995825 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f450808c260 con 0x7f45200b5f20 2024-04-26T21:38:01.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.658+0000 7f451f7fe640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f450006c8f0 0x7f450006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:01.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.658+0000 7f451f7fe640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f450006c8f0 0x7f450006edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4510034000 tx=0x7f451002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:01.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.662+0000 7f451cff9640 1 -- 172.21.15.40:0/2072995825 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f450801c360 con 0x7f45200b5f20 2024-04-26T21:38:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.890+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f452006db30 con 0x7f45200b5f20 2024-04-26T21:38:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.890+0000 7f451cff9640 1 -- 172.21.15.40:0/2072995825 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5178 (secure 0 0 0) 0x7f450805a470 con 0x7f45200b5f20 2024-04-26T21:38:01.891 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:01.891 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:38:00.331181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.40:6834/4223063167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4223063167},{"type":"v1","addr":"172.21.15.40:6834","nonce":4223063167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-26T21:38:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f450006c8f0 msgr2=0x7f450006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f450006c8f0 0x7f450006edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4510034000 tx=0x7f451002e000 comp rx=0 tx=0).stop 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 msgr2=0x7f45201519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45201519a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f450800d800 tx=0x7f450800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 shutdown_connections 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f450006c8f0 0x7f450006edb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4520162880 0x7f45200a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45200d39f0 0x7f45200a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 --2- 172.21.15.40:0/2072995825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200b5f20 0x7f45201519a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.892+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 >> 172.21.15.40:0/2072995825 conn(0x7f45200d4f60 msgr2=0x7f452006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.893+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 shutdown_connections 2024-04-26T21:38:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:01.893+0000 7f4526285640 1 -- 172.21.15.40:0/2072995825 wait complete. 2024-04-26T21:38:01.905 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /a 0 2024-04-26T21:38:02.021 INFO:teuthology.orchestra.run.smithi040.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-04-26T21:37:55.358972+0000","btime":"2024-04-26T21:37:49.493040+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T21:37:55.358972+0000","atime":"2024-04-26T21:37:49.493040+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T21:37:55.358972+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-26T21:37:55.358972+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-26T21:37:55.358972+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirfrag":1,"dirty":1},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1},"nref":1,"dentries":[{"path":"a/c","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"inodepin":1,"dirty":1},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirty":1},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-26T21:38:02.022 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-26T21:37:55.358972+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-04-26T21:37:49.493040+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T21:37:49.493040+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-04-26T21:37:55.358972+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "dirty": 1, "inodepin": 1 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "dirty": 1 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/a", "pins": { "child": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-04-26T21:37:55.358972+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T21:37:55.358972+0000", "nestlock": {}, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "dirfrag": 1, "dirty": 1 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-26T21:37:55.358972+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } ] 2024-04-26T21:38:02.023 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-04-26T21:37:55.358972+0000', 'btime': '2024-04-26T21:37:49.493040+0000', 'mode': 16877, 'uid': 1000, 'gid': 1316, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-04-26T21:37:55.358972+0000', 'atime': '2024-04-26T21:37:49.493040+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-04-26T21:37:55.358972+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-04-26T21:37:55.358972+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-04-26T21:37:55.358972+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [{'key': 'security.selinux', 'val': 'unconfined_u:object_r:unlabeled_t:s0\x00'}], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirfrag': 1, 'dirty': 1}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '0', 'committed_version': '0', 'is_rep': False, 'dir_auth': '', 'states': ['auth'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1}, 'nref': 1, 'dentries': [{'path': 'a/c', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'inodepin': 1, 'dirty': 1}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirty': 1}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-04-26T21:38:02.023 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:38:02.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:38:02.069 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:38:02.116 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:38:02.116 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:38:02.117 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:38:02.117 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:02.132 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:38:02.132 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:38:02.133 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:38:02.133 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:02.150 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:38:02.150 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:38:02.150 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:38:02.150 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:02.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:38:02.188 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:38:02.189 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:38:02.189 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:02.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:38:02.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:02.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.329+0000 7f2d0ea93640 1 Processor -- start 2024-04-26T21:38:02.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.329+0000 7f2d0ea93640 1 -- start start 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.330+0000 7f2d0ea93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806e270 0x7f2d080d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.330+0000 7f2d0ea93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d0806d560 0x7f2d080ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.330+0000 7f2d0ea93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d080d2420 con 0x7f2d080d0070 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.330+0000 7f2d0ea93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2d08162d00 con 0x7f2d0806d560 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.330+0000 7f2d0ea93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d080d2d10 con 0x7f2d0806e270 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.330+0000 7f2d0d030640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d0806d560 0x7f2d080ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.330+0000 7f2d0d030640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d0806d560 0x7f2d080ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56832/0 (socket says 172.21.15.40:56832) 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.330+0000 7f2d0d030640 1 -- 172.21.15.40:0/1616566278 learned_addr learned my addr 172.21.15.40:0/1616566278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.331+0000 7f2d07fff640 1 --2- 172.21.15.40:0/1616566278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806e270 0x7f2d080d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.331+0000 7f2d0d030640 1 -- 172.21.15.40:0/1616566278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806e270 msgr2=0x7f2d080d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:02.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.331+0000 7f2d0d030640 1 --2- 172.21.15.40:0/1616566278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806e270 0x7f2d080d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.331+0000 7f2d0d030640 1 -- 172.21.15.40:0/1616566278 >> v1:172.21.15.184:6789/0 conn(0x7f2d080d0070 legacy=0x7f2d0806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.331+0000 7f2d0d030640 1 -- 172.21.15.40:0/1616566278 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d080cd6b0 con 0x7f2d0806d560 2024-04-26T21:38:02.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.332+0000 7f2d0d030640 1 --2- 172.21.15.40:0/1616566278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d0806d560 0x7f2d080ce190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009e50 tx=0x7f2d0002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6498e2dbd0499561 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.333+0000 7f2d077fe640 1 -- 172.21.15.40:0/1616566278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00036070 con 0x7f2d0806d560 2024-04-26T21:38:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d077fe640 1 -- 172.21.15.40:0/1616566278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d00031600 con 0x7f2d0806d560 2024-04-26T21:38:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d077fe640 1 -- 172.21.15.40:0/1616566278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000319d0 con 0x7f2d0806d560 2024-04-26T21:38:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d0ea93640 1 -- 172.21.15.40:0/1616566278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d0806d560 msgr2=0x7f2d080ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/1616566278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d0806d560 0x7f2d080ce190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009e50 tx=0x7f2d0002a7a0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d0ea93640 1 -- 172.21.15.40:0/1616566278 shutdown_connections 2024-04-26T21:38:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/1616566278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d0806d560 0x7f2d080ce190 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/1616566278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806e270 0x7f2d080d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d0ea93640 1 -- 172.21.15.40:0/1616566278 >> 172.21.15.40:0/1616566278 conn(0x7f2d080d2ec0 msgr2=0x7f2d080d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d0ea93640 1 -- 172.21.15.40:0/1616566278 shutdown_connections 2024-04-26T21:38:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.334+0000 7f2d0ea93640 1 -- 172.21.15.40:0/1616566278 wait complete. 2024-04-26T21:38:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.335+0000 7f2d0ea93640 1 Processor -- start 2024-04-26T21:38:02.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0ea93640 1 -- start start 2024-04-26T21:38:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0ea93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806d560 0x7f2d080c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0ea93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0806e270 0x7f2d0814b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0ea93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d0070 0x7f2d0814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0ea93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080ce6d0 con 0x7f2d080d0070 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0ea93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080cdc70 con 0x7f2d0806e270 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0ea93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d080ce850 con 0x7f2d0806d560 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0d030640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d0070 0x7f2d0814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0d030640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d0070 0x7f2d0814b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56842/0 (socket says 172.21.15.40:56842) 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.336+0000 7f2d0d030640 1 -- 172.21.15.40:0/3397770830 learned_addr learned my addr 172.21.15.40:0/3397770830 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0c82f640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806d560 0x7f2d080c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d07fff640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0806e270 0x7f2d0814b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0d030640 1 -- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806d560 msgr2=0x7f2d080c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0d030640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806d560 0x7f2d080c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0d030640 1 -- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0806e270 msgr2=0x7f2d0814b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0d030640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0806e270 0x7f2d0814b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0d030640 1 -- 172.21.15.40:0/3397770830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0814dc70 con 0x7f2d080d0070 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0d030640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d0070 0x7f2d0814b630 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009e50 tx=0x7f2d00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d057fa640 1 -- 172.21.15.40:0/3397770830 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00041020 con 0x7f2d080d0070 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d057fa640 1 -- 172.21.15.40:0/3397770830 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d00007800 con 0x7f2d080d0070 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0814df50 con 0x7f2d080d0070 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d057fa640 1 -- 172.21.15.40:0/3397770830 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00036040 con 0x7f2d080d0070 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.337+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0814e510 con 0x7f2d080d0070 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.338+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d080694f0 con 0x7f2d080d0070 2024-04-26T21:38:02.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.339+0000 7f2d057fa640 1 -- 172.21.15.40:0/3397770830 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d00044020 con 0x7f2d080d0070 2024-04-26T21:38:02.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.339+0000 7f2d057fa640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce806c8f0 0x7f2ce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.339+0000 7f2d057fa640 1 -- 172.21.15.40:0/3397770830 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f2d000ac990 con 0x7f2d080d0070 2024-04-26T21:38:02.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.339+0000 7f2d0c82f640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce806c8f0 0x7f2ce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.340+0000 7f2d0c82f640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce806c8f0 0x7f2ce806edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80016e0 tx=0x7f2cf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:02.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.343+0000 7f2d057fa640 1 -- 172.21.15.40:0/3397770830 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d0004b030 con 0x7f2d080d0070 2024-04-26T21:38:02.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.568+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d0806efb0 con 0x7f2d080d0070 2024-04-26T21:38:02.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.568+0000 7f2d057fa640 1 -- 172.21.15.40:0/3397770830 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5175 (secure 0 0 0) 0x7f2d0007a4c0 con 0x7f2d080d0070 2024-04-26T21:38:02.569 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:02.569 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:38:01.327743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"c","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.40:6834/4223063167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4223063167},{"type":"v1","addr":"172.21.15.40:6834","nonce":4223063167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-26T21:38:02.570 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 39 2024-04-26T21:38:02.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce806c8f0 msgr2=0x7f2ce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce806c8f0 0x7f2ce806edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80016e0 tx=0x7f2cf8009040 comp rx=0 tx=0).stop 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d0070 msgr2=0x7f2d0814b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d0070 0x7f2d0814b630 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009e50 tx=0x7f2d00004a40 comp rx=0 tx=0).stop 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 shutdown_connections 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ce806c8f0 0x7f2ce806edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d0070 0x7f2d0814b630 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0806e270 0x7f2d0814b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 --2- 172.21.15.40:0/3397770830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0806d560 0x7f2d080c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 >> 172.21.15.40:0/3397770830 conn(0x7f2d080d2ec0 msgr2=0x7f2d08074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 shutdown_connections 2024-04-26T21:38:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.570+0000 7f2d0ea93640 1 -- 172.21.15.40:0/3397770830 wait complete. 2024-04-26T21:38:02.583 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:02.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.683+0000 7f354e2da640 1 Processor -- start 2024-04-26T21:38:02.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.684+0000 7f354e2da640 1 -- start start 2024-04-26T21:38:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.684+0000 7f354e2da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f354815f3a0 0x7f35480d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.684+0000 7f354e2da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35480d0e20 con 0x7f35480d2330 2024-04-26T21:38:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.684+0000 7f354e2da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35480b1d70 con 0x7f35480d1770 2024-04-26T21:38:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.684+0000 7f354e2da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f35480b2050 con 0x7f354815f3a0 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.684+0000 7f354c877640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f35480d2330 0x7f35480b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51862/0 (socket says 172.21.15.40:51862) 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.684+0000 7f354c877640 1 -- 172.21.15.40:0/113975668 learned_addr learned my addr 172.21.15.40:0/113975668 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3547fff640 1 --2- 172.21.15.40:0/113975668 >> v2:172.21.15.184:3300/0 conn(0x7f354815f3a0 0x7f35480d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009246885 0 0) 0x7f35480d0e20 con 0x7f35480d2330 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3530003520 con 0x7f35480d2330 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3615765409 0 0) 0x7f3530003520 con 0x7f35480d2330 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 >> v2:172.21.15.184:3300/0 conn(0x7f354815f3a0 msgr2=0x7f35480d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 --2- 172.21.15.40:0/113975668 >> v2:172.21.15.184:3300/0 conn(0x7f354815f3a0 0x7f35480d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 >> v1:172.21.15.184:6789/0 conn(0x7f35480d1770 legacy=0x7f35480d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35480b2330 con 0x7f35480d2330 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f3540001a10 con 0x7f35480d2330 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 >> v1:172.21.15.40:6789/0 conn(0x7f35480d2330 legacy=0x7f35480b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35300039b0 0x7f3530003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f3530004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35300051c0 0x7f353000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35480b2050 con 0x7f3530004800 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f353000c060 con 0x7f35300039b0 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f353000c260 con 0x7f35300051c0 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f35477fe640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f3530004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f3547fff640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35300039b0 0x7f3530003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.685+0000 7f354c877640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35300051c0 0x7f353000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f35477fe640 1 -- 172.21.15.40:0/113975668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35300051c0 msgr2=0x7f353000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f35477fe640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35300051c0 0x7f353000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f35477fe640 1 -- 172.21.15.40:0/113975668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35300039b0 msgr2=0x7f3530003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f35477fe640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35300039b0 0x7f3530003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f354c877640 1 -- 172.21.15.40:0/113975668 reap_dead start 2024-04-26T21:38:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f35477fe640 1 -- 172.21.15.40:0/113975668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3530003520 con 0x7f3530004800 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f35477fe640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f3530004c80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3538001750 tx=0x7f3530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59f1ca04ba6dd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538001370 con 0x7f3530004800 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35380052d0 con 0x7f3530004800 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.686+0000 7f3546ffd640 1 -- 172.21.15.40:0/113975668 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353802abc0 con 0x7f3530004800 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.687+0000 7f354e2da640 1 -- 172.21.15.40:0/113975668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 msgr2=0x7f3530004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.687+0000 7f354e2da640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f3530004c80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3538001750 tx=0x7f3530005830 comp rx=0 tx=0).stop 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.687+0000 7f354e2da640 1 -- 172.21.15.40:0/113975668 shutdown_connections 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.687+0000 7f354e2da640 1 --2- 172.21.15.40:0/113975668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f3530004c80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.687+0000 7f354e2da640 1 -- 172.21.15.40:0/113975668 >> 172.21.15.40:0/113975668 conn(0x7f35480d4f90 msgr2=0x7f35480d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.687+0000 7f354e2da640 1 -- 172.21.15.40:0/113975668 shutdown_connections 2024-04-26T21:38:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.687+0000 7f354e2da640 1 -- 172.21.15.40:0/113975668 wait complete. 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.688+0000 7f354e2da640 1 Processor -- start 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.688+0000 7f354e2da640 1 -- start start 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f354e2da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f35480cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f354e2da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480cb770 0x7f35481f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f354e2da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35481f5950 0x7f35480b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f354e2da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35480d0870 con 0x7f3530004800 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f354e2da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35480d06f0 con 0x7f35480cb770 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f354e2da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35480d09f0 con 0x7f35481f5950 2024-04-26T21:38:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f3547fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f35480cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f3547fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f35480cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56858/0 (socket says 172.21.15.40:56858) 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f3547fff640 1 -- 172.21.15.40:0/3834291085 learned_addr learned my addr 172.21.15.40:0/3834291085 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f35477fe640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480cb770 0x7f35481f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.689+0000 7f354c877640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35481f5950 0x7f35480b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3547fff640 1 -- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35481f5950 msgr2=0x7f35480b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3547fff640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35481f5950 0x7f35480b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3547fff640 1 -- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480cb770 msgr2=0x7f35481f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3547fff640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480cb770 0x7f35481f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3547fff640 1 -- 172.21.15.40:0/3834291085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35480c9800 con 0x7f3530004800 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3547fff640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f35480cb230 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f353c011890 tx=0x7f353c011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3544ff9640 1 -- 172.21.15.40:0/3834291085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c019070 con 0x7f3530004800 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3544ff9640 1 -- 172.21.15.40:0/3834291085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c004240 con 0x7f3530004800 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35480ca470 con 0x7f3530004800 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f3544ff9640 1 -- 172.21.15.40:0/3834291085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c00f820 con 0x7f3530004800 2024-04-26T21:38:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.690+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35480caa30 con 0x7f3530004800 2024-04-26T21:38:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.691+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35480694f0 con 0x7f3530004800 2024-04-26T21:38:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.692+0000 7f3544ff9640 1 -- 172.21.15.40:0/3834291085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f353c01c070 con 0x7f3530004800 2024-04-26T21:38:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.692+0000 7f3544ff9640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.692+0000 7f3544ff9640 1 -- 172.21.15.40:0/3834291085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f353c08c1d0 con 0x7f3530004800 2024-04-26T21:38:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.692+0000 7f35477fe640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.693+0000 7f35477fe640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f353406c8f0 0x7f353406edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3538032000 tx=0x7f353802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:02.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.697+0000 7f3544ff9640 1 -- 172.21.15.40:0/3834291085 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f353c05a3e0 con 0x7f3530004800 2024-04-26T21:38:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.885+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35480b6860 con 0x7f3530004800 2024-04-26T21:38:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.886+0000 7f3544ff9640 1 -- 172.21.15.40:0/3834291085 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+17130 (secure 0 0 0) 0x7f353c059ce0 con 0x7f3530004800 2024-04-26T21:38:02.888 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:02.888 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":55,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:58.306304+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:37:35.784073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T21:37:36.460613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6834/2276808574":"2024-04-27T21:37:58.306284+0000","172.21.15.40:6832/2276808574":"2024-04-27T21:37:58.306284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f353406c8f0 msgr2=0x7f353406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f353406c8f0 0x7f353406edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3538032000 tx=0x7f353802e000 comp rx=0 tx=0).stop 2024-04-26T21:38:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 msgr2=0x7f35480cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f35480cb230 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f353c011890 tx=0x7f353c011d50 comp rx=0 tx=0).stop 2024-04-26T21:38:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 shutdown_connections 2024-04-26T21:38:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35481f5950 0x7f35480b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480cb770 0x7f35481f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 --2- 172.21.15.40:0/3834291085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3530004800 0x7f35480cb230 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:02.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 >> 172.21.15.40:0/3834291085 conn(0x7f35480d4f90 msgr2=0x7f354806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:02.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 shutdown_connections 2024-04-26T21:38:02.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:02.888+0000 7f354e2da640 1 -- 172.21.15.40:0/3834291085 wait complete. 2024-04-26T21:38:02.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:37:35.784073+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:37:36.460613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:38:02.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:38:03.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.004+0000 7fb0219c6640 1 Processor -- start 2024-04-26T21:38:03.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.004+0000 7fb0219c6640 1 -- start start 2024-04-26T21:38:03.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.005+0000 7fb0219c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c162cd0 0x7fb01c0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.005+0000 7fb0219c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb01c0d2110 0x7fb01c0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.005+0000 7fb0219c6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb01c0d0890 con 0x7fb01c1628f0 2024-04-26T21:38:03.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.005+0000 7fb0219c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb01c0d1130 con 0x7fb01c0d2110 2024-04-26T21:38:03.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.005+0000 7fb0219c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb01c0d0b70 con 0x7fb01c162cd0 2024-04-26T21:38:03.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01a7fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c162cd0 0x7fb01c0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01b7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb01c0d2110 0x7fb01c0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01b7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb01c0d2110 0x7fb01c0b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:43852/0 (socket says 172.21.15.40:43852) 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01a7fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c162cd0 0x7fb01c0b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:52060/0 (socket says 172.21.15.40:52060) 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01affd640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb01c1628f0 0x7fb01c0d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:44712/0 (socket says 172.21.15.40:44712) 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01b7fe640 1 -- 172.21.15.40:0/2364658724 learned_addr learned my addr 172.21.15.40:0/2364658724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01b7fe640 1 -- 172.21.15.40:0/2364658724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c162cd0 msgr2=0x7fb01c0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01b7fe640 1 --2- 172.21.15.40:0/2364658724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c162cd0 0x7fb01c0b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01b7fe640 1 -- 172.21.15.40:0/2364658724 >> v1:172.21.15.184:6789/0 conn(0x7fb01c1628f0 legacy=0x7fb01c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.006+0000 7fb01b7fe640 1 -- 172.21.15.40:0/2364658724 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c0d0e50 con 0x7fb01c0d2110 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.007+0000 7fb01b7fe640 1 --2- 172.21.15.40:0/2364658724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 0x7fb01c0b51a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb014009e50 tx=0x7fb01402a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee1dacf48418e28f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.007+0000 7fb019ffb640 1 -- 172.21.15.40:0/2364658724 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014036070 con 0x7fb01c0d2110 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.007+0000 7fb019ffb640 1 -- 172.21.15.40:0/2364658724 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb014031600 con 0x7fb01c0d2110 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.007+0000 7fb019ffb640 1 -- 172.21.15.40:0/2364658724 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140319d0 con 0x7fb01c0d2110 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.007+0000 7fb0219c6640 1 -- 172.21.15.40:0/2364658724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 msgr2=0x7fb01c0b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.007+0000 7fb0219c6640 1 --2- 172.21.15.40:0/2364658724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 0x7fb01c0b51a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb014009e50 tx=0x7fb01402a590 comp rx=0 tx=0).stop 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.008+0000 7fb0219c6640 1 -- 172.21.15.40:0/2364658724 shutdown_connections 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.008+0000 7fb0219c6640 1 --2- 172.21.15.40:0/2364658724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 0x7fb01c0b51a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.008+0000 7fb0219c6640 1 --2- 172.21.15.40:0/2364658724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c162cd0 0x7fb01c0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.008+0000 7fb0219c6640 1 -- 172.21.15.40:0/2364658724 >> 172.21.15.40:0/2364658724 conn(0x7fb01c0d4ff0 msgr2=0x7fb01c0d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:03.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.008+0000 7fb0219c6640 1 -- 172.21.15.40:0/2364658724 shutdown_connections 2024-04-26T21:38:03.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.008+0000 7fb0219c6640 1 -- 172.21.15.40:0/2364658724 wait complete. 2024-04-26T21:38:03.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.009+0000 7fb0219c6640 1 Processor -- start 2024-04-26T21:38:03.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.009+0000 7fb0219c6640 1 -- start start 2024-04-26T21:38:03.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb0219c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 0x7fb01c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb0219c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c1628f0 0x7fb01c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb0219c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c162cd0 0x7fb01c14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb0219c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c0d16d0 con 0x7fb01c162cd0 2024-04-26T21:38:03.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb0219c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c0d1550 con 0x7fb01c0d2110 2024-04-26T21:38:03.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb0219c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb01c0b3f50 con 0x7fb01c1628f0 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c162cd0 0x7fb01c14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c162cd0 0x7fb01c14eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56862/0 (socket says 172.21.15.40:56862) 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 -- 172.21.15.40:0/3688641635 learned_addr learned my addr 172.21.15.40:0/3688641635 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01a7fc640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c1628f0 0x7fb01c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01affd640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 0x7fb01c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 -- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c1628f0 msgr2=0x7fb01c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c1628f0 0x7fb01c0cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 -- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 msgr2=0x7fb01c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 0x7fb01c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 -- 172.21.15.40:0/3688641635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c0ceee0 con 0x7fb01c162cd0 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01a7fc640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c1628f0 0x7fb01c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb01b7fe640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c162cd0 0x7fb01c14eb40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb014009f70 tx=0x7fb014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.010+0000 7fb013fff640 1 -- 172.21.15.40:0/3688641635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014036050 con 0x7fb01c162cd0 2024-04-26T21:38:03.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.011+0000 7fb013fff640 1 -- 172.21.15.40:0/3688641635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb014037040 con 0x7fb01c162cd0 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.011+0000 7fb013fff640 1 -- 172.21.15.40:0/3688641635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014009510 con 0x7fb01c162cd0 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.011+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c0cf1c0 con 0x7fb01c162cd0 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.011+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c0cf780 con 0x7fb01c162cd0 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.012+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c0694f0 con 0x7fb01c162cd0 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.012+0000 7fb013fff640 1 -- 172.21.15.40:0/3688641635 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb014007c70 con 0x7fb01c162cd0 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.012+0000 7fb013fff640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faffc06c8f0 0x7faffc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.013+0000 7fb013fff640 1 -- 172.21.15.40:0/3688641635 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7fb0140ac8e0 con 0x7fb01c162cd0 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.013+0000 7fb01affd640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faffc06c8f0 0x7faffc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.013+0000 7fb01affd640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faffc06c8f0 0x7faffc06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb00c000f60 tx=0x7fb00c007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.017+0000 7fb013fff640 1 -- 172.21.15.40:0/3688641635 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb01407aae0 con 0x7fb01c162cd0 2024-04-26T21:38:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.201+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb01c0745d0 con 0x7fb01c162cd0 2024-04-26T21:38:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.202+0000 7fb013fff640 1 -- 172.21.15.40:0/3688641635 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) ==== 93+0+10 (secure 0 0 0) 0x7fb01407a3e0 con 0x7fb01c162cd0 2024-04-26T21:38:03.203 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:38:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.203+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faffc06c8f0 msgr2=0x7faffc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faffc06c8f0 0x7faffc06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb00c000f60 tx=0x7fb00c007ca0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c162cd0 msgr2=0x7fb01c14eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c162cd0 0x7fb01c14eb40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb014009f70 tx=0x7fb014004a40 comp rx=0 tx=0).stop 2024-04-26T21:38:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 shutdown_connections 2024-04-26T21:38:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faffc06c8f0 0x7faffc06edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c162cd0 0x7fb01c14eb40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c1628f0 0x7fb01c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 --2- 172.21.15.40:0/3688641635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c0d2110 0x7fb01c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 >> 172.21.15.40:0/3688641635 conn(0x7fb01c0d4ff0 msgr2=0x7fb01c06e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 shutdown_connections 2024-04-26T21:38:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.204+0000 7fb0219c6640 1 -- 172.21.15.40:0/3688641635 wait complete. 2024-04-26T21:38:03.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:38:03.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.318+0000 7ff2a1a2a640 1 Processor -- start 2024-04-26T21:38:03.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.318+0000 7ff2a1a2a640 1 -- start start 2024-04-26T21:38:03.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff2a1a2a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff29c0cfde0 0x7ff29c0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff2a1a2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff29c070350 0x7ff29c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff2a1a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff2a1a2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff29c071500 con 0x7ff29c070350 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff2a1a2a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff29c0afab0 con 0x7ff29c0cfde0 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff2a1a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff29c06dd20 con 0x7ff29c071090 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff29c070350 0x7ff29c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff29c070350 0x7ff29c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56876/0 (socket says 172.21.15.40:56876) 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29a7fc640 1 -- 172.21.15.40:0/3385675757 learned_addr learned my addr 172.21.15.40:0/3385675757 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29b7fe640 1 --2- 172.21.15.40:0/3385675757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29affd640 1 --2- 172.21.15.40:0/3385675757 >> v2:172.21.15.184:3300/0 conn(0x7ff29c0cfde0 0x7ff29c0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29a7fc640 1 -- 172.21.15.40:0/3385675757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 msgr2=0x7ff29c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29a7fc640 1 --2- 172.21.15.40:0/3385675757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29a7fc640 1 -- 172.21.15.40:0/3385675757 >> v2:172.21.15.184:3300/0 conn(0x7ff29c0cfde0 msgr2=0x7ff29c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:03.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29a7fc640 1 --2- 172.21.15.40:0/3385675757 >> v2:172.21.15.184:3300/0 conn(0x7ff29c0cfde0 0x7ff29c0d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.319+0000 7ff29a7fc640 1 -- 172.21.15.40:0/3385675757 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c0d2390 con 0x7ff29c070350 2024-04-26T21:38:03.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.320+0000 7ff29a7fc640 1 --2- 172.21.15.40:0/3385675757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c070b50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0762b0 tx=0x7ff28402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9608c351da7f197 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.320+0000 7ff299ffb640 1 -- 172.21.15.40:0/3385675757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284037070 con 0x7ff29c070350 2024-04-26T21:38:03.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.320+0000 7ff299ffb640 1 -- 172.21.15.40:0/3385675757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff284032440 con 0x7ff29c070350 2024-04-26T21:38:03.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.320+0000 7ff299ffb640 1 -- 172.21.15.40:0/3385675757 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284032810 con 0x7ff29c070350 2024-04-26T21:38:03.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/3385675757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 msgr2=0x7ff29c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/3385675757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c070b50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0762b0 tx=0x7ff28402a140 comp rx=0 tx=0).stop 2024-04-26T21:38:03.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/3385675757 shutdown_connections 2024-04-26T21:38:03.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/3385675757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/3385675757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c070b50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/3385675757 >> v2:172.21.15.184:3300/0 conn(0x7ff29c0cfde0 0x7ff29c0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/3385675757 >> 172.21.15.40:0/3385675757 conn(0x7ff29c0d2d80 msgr2=0x7ff29c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:03.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/3385675757 shutdown_connections 2024-04-26T21:38:03.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.321+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/3385675757 wait complete. 2024-04-26T21:38:03.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.322+0000 7ff2a1a2a640 1 Processor -- start 2024-04-26T21:38:03.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.322+0000 7ff2a1a2a640 1 -- start start 2024-04-26T21:38:03.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff2a1a2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff2a1a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c09f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff2a1a2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff29c0cfde0 0x7ff29c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff2a1a2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c06d9b0 con 0x7ff29c070350 2024-04-26T21:38:03.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff2a1a2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c06d6d0 con 0x7ff29c0cfde0 2024-04-26T21:38:03.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff2a1a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff29c0d20a0 con 0x7ff29c071090 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c0b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56882/0 (socket says 172.21.15.40:56882) 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29affd640 1 -- 172.21.15.40:0/1989988851 learned_addr learned my addr 172.21.15.40:0/1989988851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29a7fc640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c09f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29b7fe640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff29c0cfde0 0x7ff29c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29affd640 1 -- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 msgr2=0x7ff29c09f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29affd640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c09f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29affd640 1 -- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff29c0cfde0 msgr2=0x7ff29c0b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29affd640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff29c0cfde0 0x7ff29c0b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29affd640 1 -- 172.21.15.40:0/1989988851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c0b4850 con 0x7ff29c070350 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.323+0000 7ff29a7fc640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c09f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.324+0000 7ff29b7fe640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff29c0cfde0 0x7ff29c0b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.324+0000 7ff29affd640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c0b52f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c7f0 tx=0x7ff29400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.324+0000 7ff293fff640 1 -- 172.21.15.40:0/1989988851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294015070 con 0x7ff29c070350 2024-04-26T21:38:03.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.324+0000 7ff293fff640 1 -- 172.21.15.40:0/1989988851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff294004240 con 0x7ff29c070350 2024-04-26T21:38:03.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.324+0000 7ff293fff640 1 -- 172.21.15.40:0/1989988851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294007620 con 0x7ff29c070350 2024-04-26T21:38:03.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.324+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c0b6b10 con 0x7ff29c070350 2024-04-26T21:38:03.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.324+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c0b4af0 con 0x7ff29c070350 2024-04-26T21:38:03.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.325+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29c0694f0 con 0x7ff29c070350 2024-04-26T21:38:03.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.325+0000 7ff293fff640 1 -- 172.21.15.40:0/1989988851 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2940079d0 con 0x7ff29c070350 2024-04-26T21:38:03.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.326+0000 7ff293fff640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.326+0000 7ff293fff640 1 -- 172.21.15.40:0/1989988851 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7ff29408c200 con 0x7ff29c070350 2024-04-26T21:38:03.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.326+0000 7ff29a7fc640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.327+0000 7ff29a7fc640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff27c06c8f0 0x7ff27c06edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff284034000 tx=0x7ff28402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.330+0000 7ff293fff640 1 -- 172.21.15.40:0/1989988851 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff29405a410 con 0x7ff29c070350 2024-04-26T21:38:03.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.515+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff29c162790 con 0x7ff29c070350 2024-04-26T21:38:03.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.516+0000 7ff293fff640 1 -- 172.21.15.40:0/1989988851 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) ==== 104+0+11 (secure 0 0 0) 0x7ff294059d10 con 0x7ff29c070350 2024-04-26T21:38:03.517 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff27c06c8f0 msgr2=0x7ff27c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff27c06c8f0 0x7ff27c06edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff284034000 tx=0x7ff28402e000 comp rx=0 tx=0).stop 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 msgr2=0x7ff29c0b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c0b52f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c7f0 tx=0x7ff29400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 shutdown_connections 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff29c0cfde0 0x7ff29c0b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff29c071090 0x7ff29c09f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 --2- 172.21.15.40:0/1989988851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c070350 0x7ff29c0b52f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.517+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 >> 172.21.15.40:0/1989988851 conn(0x7ff29c0d2d80 msgr2=0x7ff29c0afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:03.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.518+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 shutdown_connections 2024-04-26T21:38:03.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.518+0000 7ff2a1a2a640 1 -- 172.21.15.40:0/1989988851 wait complete. 2024-04-26T21:38:03.529 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:38:03.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.632+0000 7f195ec2a640 1 Processor -- start 2024-04-26T21:38:03.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.632+0000 7f195ec2a640 1 -- start start 2024-04-26T21:38:03.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.632+0000 7f195ec2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19580d1a00 0x7f19580d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.632+0000 7f195ec2a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19580d0a10 con 0x7f19580d2320 2024-04-26T21:38:03.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.632+0000 7f195ec2a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19580d0cf0 con 0x7f19580b40d0 2024-04-26T21:38:03.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.632+0000 7f195ec2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f19580d0fd0 con 0x7f19580d1a00 2024-04-26T21:38:03.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.633+0000 7f195c9c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19580d1a00 0x7f19580d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.633+0000 7f1957fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f19580d2320 0x7f19581625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51866/0 (socket says 172.21.15.40:51866) 2024-04-26T21:38:03.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.633+0000 7f1957fff640 1 -- 172.21.15.40:0/3178286235 learned_addr learned my addr 172.21.15.40:0/3178286235 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:03.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.633+0000 7f195c9c6640 1 -- 172.21.15.40:0/3178286235 >> v1:172.21.15.184:6789/0 conn(0x7f19580b40d0 legacy=0x7f19580b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.633+0000 7f195c9c6640 1 -- 172.21.15.40:0/3178286235 >> v1:172.21.15.40:6789/0 conn(0x7f19580d2320 legacy=0x7f19581625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.633+0000 7f19577fe640 1 -- 172.21.15.40:0/3178286235 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2294528390 0 0) 0x7f19580d0a10 con 0x7f19580d2320 2024-04-26T21:38:03.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.633+0000 7f195c9c6640 1 -- 172.21.15.40:0/3178286235 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19580d40a0 con 0x7f19580d1a00 2024-04-26T21:38:03.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f195c9c6640 1 --2- 172.21.15.40:0/3178286235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580d1a00 0x7f19580d1de0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f194c0097a0 tx=0x7f194c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff473dcd3f60480 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f19577fe640 1 -- 172.21.15.40:0/3178286235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c037070 con 0x7f19580d1a00 2024-04-26T21:38:03.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f19577fe640 1 -- 172.21.15.40:0/3178286235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c032440 con 0x7f19580d1a00 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f19577fe640 1 -- 172.21.15.40:0/3178286235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c02ae60 con 0x7f19580d1a00 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f195ec2a640 1 -- 172.21.15.40:0/3178286235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580d1a00 msgr2=0x7f19580d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f195ec2a640 1 --2- 172.21.15.40:0/3178286235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580d1a00 0x7f19580d1de0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f194c0097a0 tx=0x7f194c02a110 comp rx=0 tx=0).stop 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f195ec2a640 1 -- 172.21.15.40:0/3178286235 shutdown_connections 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f195ec2a640 1 --2- 172.21.15.40:0/3178286235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580d1a00 0x7f19580d1de0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f195ec2a640 1 -- 172.21.15.40:0/3178286235 >> 172.21.15.40:0/3178286235 conn(0x7f19580d4fd0 msgr2=0x7f19580d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f195ec2a640 1 -- 172.21.15.40:0/3178286235 shutdown_connections 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.634+0000 7f195ec2a640 1 -- 172.21.15.40:0/3178286235 wait complete. 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.636+0000 7f195ec2a640 1 Processor -- start 2024-04-26T21:38:03.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.636+0000 7f195ec2a640 1 -- start start 2024-04-26T21:38:03.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.636+0000 7f195ec2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580b40d0 0x7f195814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.636+0000 7f195ec2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19580d1a00 0x7f19581514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.636+0000 7f195ec2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19580d2320 0x7f1958151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.636+0000 7f195ec2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19580b5210 con 0x7f19580b40d0 2024-04-26T21:38:03.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.636+0000 7f195ec2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1958162f50 con 0x7f19580d1a00 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.636+0000 7f195ec2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19580b5390 con 0x7f19580d2320 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580b40d0 0x7f195814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580b40d0 0x7f195814eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56890/0 (socket says 172.21.15.40:56890) 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 -- 172.21.15.40:0/3862039064 learned_addr learned my addr 172.21.15.40:0/3862039064 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f1957fff640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19580d1a00 0x7f19581514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195d1c7640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19580d2320 0x7f1958151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 -- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19580d2320 msgr2=0x7f1958151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19580d2320 0x7f1958151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 -- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19580d1a00 msgr2=0x7f19581514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19580d1a00 0x7f19581514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 -- 172.21.15.40:0/3862039064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19580ba1d0 con 0x7f19580b40d0 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195c9c6640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580b40d0 0x7f195814eb20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f194c0098c0 tx=0x7f194c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f19557fa640 1 -- 172.21.15.40:0/3862039064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c037040 con 0x7f19580b40d0 2024-04-26T21:38:03.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f19557fa640 1 -- 172.21.15.40:0/3862039064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c042070 con 0x7f19580b40d0 2024-04-26T21:38:03.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f19557fa640 1 -- 172.21.15.40:0/3862039064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c030c70 con 0x7f19580b40d0 2024-04-26T21:38:03.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19580cbe60 con 0x7f19580b40d0 2024-04-26T21:38:03.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.637+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19580cb5c0 con 0x7f19580b40d0 2024-04-26T21:38:03.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.638+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19580694f0 con 0x7f19580b40d0 2024-04-26T21:38:03.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.639+0000 7f19557fa640 1 -- 172.21.15.40:0/3862039064 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f194c047460 con 0x7f19580b40d0 2024-04-26T21:38:03.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.639+0000 7f19557fa640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f193c06c920 0x7f193c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.639+0000 7f19557fa640 1 -- 172.21.15.40:0/3862039064 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f194c0ac960 con 0x7f19580b40d0 2024-04-26T21:38:03.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.640+0000 7f1957fff640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f193c06c920 0x7f193c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.640+0000 7f1957fff640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f193c06c920 0x7f193c06ede0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1958155a80 tx=0x7f1944007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.644+0000 7f19557fa640 1 -- 172.21.15.40:0/3862039064 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f194c07a480 con 0x7f19580b40d0 2024-04-26T21:38:03.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.830+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f19580bae30 con 0x7f19580b40d0 2024-04-26T21:38:03.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.830+0000 7f19557fa640 1 -- 172.21.15.40:0/3862039064 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) ==== 100+0+11 (secure 0 0 0) 0x7f194c07e780 con 0x7f19580b40d0 2024-04-26T21:38:03.831 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f193c06c920 msgr2=0x7f193c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f193c06c920 0x7f193c06ede0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1958155a80 tx=0x7f1944007e40 comp rx=0 tx=0).stop 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580b40d0 msgr2=0x7f195814eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580b40d0 0x7f195814eb20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f194c0098c0 tx=0x7f194c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 shutdown_connections 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f193c06c920 0x7f193c06ede0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19580d2320 0x7f1958151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19580d1a00 0x7f19581514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 --2- 172.21.15.40:0/3862039064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19580b40d0 0x7f195814eb20 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 >> 172.21.15.40:0/3862039064 conn(0x7f19580d4fd0 msgr2=0x7f19580d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 shutdown_connections 2024-04-26T21:38:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.832+0000 7f195ec2a640 1 -- 172.21.15.40:0/3862039064 wait complete. 2024-04-26T21:38:03.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:03.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.944+0000 7faee68b0640 1 Processor -- start 2024-04-26T21:38:03.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.944+0000 7faee68b0640 1 -- start start 2024-04-26T21:38:03.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee68b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faee00b2340 0x7faee00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee68b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee00cd930 0x7faee00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee68b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee00d10d0 con 0x7faee0162810 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee68b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faee00d0750 con 0x7faee00b2340 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee68b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faee00d16a0 con 0x7faee00cd930 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee4e4d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee00cd930 0x7faee00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faedf7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faee00b2340 0x7faee00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee4e4d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee00cd930 0x7faee00d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:52110/0 (socket says 172.21.15.40:52110) 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faedf7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faee00b2340 0x7faee00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:43892/0 (socket says 172.21.15.40:43892) 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee4e4d640 1 -- 172.21.15.40:0/4203829280 learned_addr learned my addr 172.21.15.40:0/4203829280 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faedf7fe640 1 -- 172.21.15.40:0/4203829280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee00cd930 msgr2=0x7faee00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faedf7fe640 1 --2- 172.21.15.40:0/4203829280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee00cd930 0x7faee00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faedf7fe640 1 -- 172.21.15.40:0/4203829280 >> v1:172.21.15.184:6789/0 conn(0x7faee0162810 legacy=0x7faee0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faedf7fe640 1 -- 172.21.15.40:0/4203829280 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee00d1370 con 0x7faee00b2340 2024-04-26T21:38:03.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.945+0000 7faee4e4d640 1 --2- 172.21.15.40:0/4203829280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee00cd930 0x7faee00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.946+0000 7faedf7fe640 1 --2- 172.21.15.40:0/4203829280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 0x7faee00cd3f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faee00d1820 tx=0x7faed402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfcdb949374afdae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.946+0000 7faedeffd640 1 -- 172.21.15.40:0/4203829280 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037070 con 0x7faee00b2340 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.946+0000 7faedeffd640 1 -- 172.21.15.40:0/4203829280 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4032440 con 0x7faee00b2340 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.946+0000 7faedeffd640 1 -- 172.21.15.40:0/4203829280 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4032810 con 0x7faee00b2340 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.947+0000 7faee68b0640 1 -- 172.21.15.40:0/4203829280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 msgr2=0x7faee00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.947+0000 7faee68b0640 1 --2- 172.21.15.40:0/4203829280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 0x7faee00cd3f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faee00d1820 tx=0x7faed402a180 comp rx=0 tx=0).stop 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.947+0000 7faee68b0640 1 -- 172.21.15.40:0/4203829280 shutdown_connections 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.947+0000 7faee68b0640 1 --2- 172.21.15.40:0/4203829280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee00cd930 0x7faee00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.947+0000 7faee68b0640 1 --2- 172.21.15.40:0/4203829280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 0x7faee00cd3f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.947+0000 7faee68b0640 1 -- 172.21.15.40:0/4203829280 >> 172.21.15.40:0/4203829280 conn(0x7faee00d2da0 msgr2=0x7faee00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:03.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.947+0000 7faee68b0640 1 -- 172.21.15.40:0/4203829280 shutdown_connections 2024-04-26T21:38:03.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.947+0000 7faee68b0640 1 -- 172.21.15.40:0/4203829280 wait complete. 2024-04-26T21:38:03.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.948+0000 7faee68b0640 1 Processor -- start 2024-04-26T21:38:03.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.948+0000 7faee68b0640 1 -- start start 2024-04-26T21:38:03.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faee68b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 0x7faee01e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faee68b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faee00cd930 0x7faee00c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faee68b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee0162810 0x7faee0151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faee68b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faee00d2070 con 0x7faee00cd930 2024-04-26T21:38:03.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faee68b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faee00d0470 con 0x7faee00b2340 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faee68b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faee00d21f0 con 0x7faee0162810 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faee00cd930 0x7faee00c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faee00cd930 0x7faee00c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56892/0 (socket says 172.21.15.40:56892) 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedf7fe640 1 -- 172.21.15.40:0/424232789 learned_addr learned my addr 172.21.15.40:0/424232789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faee4e4d640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee0162810 0x7faee0151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedffff640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 0x7faee01e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedf7fe640 1 -- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee0162810 msgr2=0x7faee0151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedf7fe640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee0162810 0x7faee0151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedf7fe640 1 -- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 msgr2=0x7faee01e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedf7fe640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 0x7faee01e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.949+0000 7faedf7fe640 1 -- 172.21.15.40:0/424232789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee00ccf60 con 0x7faee00cd930 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.950+0000 7faedf7fe640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faee00cd930 0x7faee00c9e70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faed4009890 tx=0x7faed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.950+0000 7faedcff9640 1 -- 172.21.15.40:0/424232789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037040 con 0x7faee00cd930 2024-04-26T21:38:03.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.950+0000 7faedcff9640 1 -- 172.21.15.40:0/424232789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4042070 con 0x7faee00cd930 2024-04-26T21:38:03.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.950+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee00cd240 con 0x7faee00cd930 2024-04-26T21:38:03.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.950+0000 7faedcff9640 1 -- 172.21.15.40:0/424232789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4030960 con 0x7faee00cd930 2024-04-26T21:38:03.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.950+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee01e8e60 con 0x7faee00cd930 2024-04-26T21:38:03.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.951+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee00694f0 con 0x7faee00cd930 2024-04-26T21:38:03.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.951+0000 7faedcff9640 1 -- 172.21.15.40:0/424232789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faed4030b00 con 0x7faee00cd930 2024-04-26T21:38:03.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.952+0000 7faedcff9640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faecc06c8f0 0x7faecc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:03.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.952+0000 7faedcff9640 1 -- 172.21.15.40:0/424232789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7faed40ac540 con 0x7faee00cd930 2024-04-26T21:38:03.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.952+0000 7faedffff640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faecc06c8f0 0x7faecc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:03.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.952+0000 7faedffff640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faecc06c8f0 0x7faecc06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faed00016e0 tx=0x7faed0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:03.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:03.956+0000 7faedcff9640 1 -- 172.21.15.40:0/424232789 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faed407a190 con 0x7faee00cd930 2024-04-26T21:38:04.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.179+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faee00b1e60 con 0x7faee00cd930 2024-04-26T21:38:04.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.180+0000 7faedcff9640 1 -- 172.21.15.40:0/424232789 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5182 (secure 0 0 0) 0x7faed407e490 con 0x7faee00cd930 2024-04-26T21:38:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:04.181 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:38:03.333495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/4223063167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4223063167},{"type":"v1","addr":"172.21.15.40:6834","nonce":4223063167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6306,"qdb_cluster":[6306]},"id":3}]} 2024-04-26T21:38:04.181 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 40 2024-04-26T21:38:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faecc06c8f0 msgr2=0x7faecc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faecc06c8f0 0x7faecc06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faed00016e0 tx=0x7faed0009040 comp rx=0 tx=0).stop 2024-04-26T21:38:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faee00cd930 msgr2=0x7faee00c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faee00cd930 0x7faee00c9e70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faed4009890 tx=0x7faed40046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 shutdown_connections 2024-04-26T21:38:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faecc06c8f0 0x7faecc06edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee0162810 0x7faee0151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faee00cd930 0x7faee00c9e70 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 --2- 172.21.15.40:0/424232789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee00b2340 0x7faee01e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 >> 172.21.15.40:0/424232789 conn(0x7faee00d2da0 msgr2=0x7faee00cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:04.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 shutdown_connections 2024-04-26T21:38:04.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.182+0000 7faee68b0640 1 -- 172.21.15.40:0/424232789 wait complete. 2024-04-26T21:38:04.194 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:04.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.293+0000 7f78f317f640 1 Processor -- start 2024-04-26T21:38:04.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.293+0000 7f78f317f640 1 -- start start 2024-04-26T21:38:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.294+0000 7f78f317f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f78ec071e60 0x7f78ec06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.294+0000 7f78f317f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78ec15d7c0 con 0x7f78ec0cc480 2024-04-26T21:38:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.294+0000 7f78f317f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78ec15daa0 con 0x7f78ec06fe20 2024-04-26T21:38:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.294+0000 7f78f317f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f78ec15dd80 con 0x7f78ec071e60 2024-04-26T21:38:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.294+0000 7f78f171c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78ec0cc480 0x7f78ec166ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51880/0 (socket says 172.21.15.40:51880) 2024-04-26T21:38:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.294+0000 7f78f171c640 1 -- 172.21.15.40:0/3858695791 learned_addr learned my addr 172.21.15.40:0/3858695791 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78f0f1b640 1 --2- 172.21.15.40:0/3858695791 >> v2:172.21.15.184:3300/0 conn(0x7f78ec071e60 0x7f78ec06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708770243 0 0) 0x7f78ec15d7c0 con 0x7f78ec0cc480 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78dc003520 con 0x7f78ec0cc480 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2356081220 0 0) 0x7f78dc003520 con 0x7f78ec0cc480 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 >> v2:172.21.15.184:3300/0 conn(0x7f78ec071e60 msgr2=0x7f78ec06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 --2- 172.21.15.40:0/3858695791 >> v2:172.21.15.184:3300/0 conn(0x7f78ec071e60 0x7f78ec06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 >> v1:172.21.15.184:6789/0 conn(0x7f78ec06fe20 legacy=0x7f78ec072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec15e060 con 0x7f78ec0cc480 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f78e40019f0 con 0x7f78ec0cc480 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 >> v1:172.21.15.40:6789/0 conn(0x7f78ec0cc480 legacy=0x7f78ec166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0039b0 0x7f78dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78dc004800 0x7f78dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78dc0051c0 0x7f78dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec15dd80 con 0x7f78dc0051c0 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc00c060 con 0x7f78dc0039b0 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78dc00c260 con 0x7f78dc004800 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78f171c640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78dc0051c0 0x7f78dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78f0f1b640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0039b0 0x7f78dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78ebfff640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78dc004800 0x7f78dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78f171c640 1 -- 172.21.15.40:0/3858695791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78dc004800 msgr2=0x7f78dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78f171c640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78dc004800 0x7f78dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78f171c640 1 -- 172.21.15.40:0/3858695791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0039b0 msgr2=0x7f78dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78f171c640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0039b0 0x7f78dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.295+0000 7f78f171c640 1 -- 172.21.15.40:0/3858695791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc003520 con 0x7f78dc0051c0 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.296+0000 7f78f171c640 1 -- 172.21.15.40:0/3858695791 reap_dead start 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.296+0000 7f78f171c640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78dc0051c0 0x7f78dc00b9c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f78e400a840 tx=0x7f78dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1db24bb0d2712ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.296+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400a460 con 0x7f78dc0051c0 2024-04-26T21:38:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.296+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e402f070 con 0x7f78dc0051c0 2024-04-26T21:38:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.296+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3858695791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e402bcb0 con 0x7f78dc0051c0 2024-04-26T21:38:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.297+0000 7f78f317f640 1 -- 172.21.15.40:0/3858695791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78dc0051c0 msgr2=0x7f78dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.297+0000 7f78f317f640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78dc0051c0 0x7f78dc00b9c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f78e400a840 tx=0x7f78dc005830 comp rx=0 tx=0).stop 2024-04-26T21:38:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.297+0000 7f78f317f640 1 -- 172.21.15.40:0/3858695791 shutdown_connections 2024-04-26T21:38:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.297+0000 7f78f317f640 1 --2- 172.21.15.40:0/3858695791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78dc0051c0 0x7f78dc00b9c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.297+0000 7f78f317f640 1 -- 172.21.15.40:0/3858695791 >> 172.21.15.40:0/3858695791 conn(0x7f78ec0d0be0 msgr2=0x7f78ec0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.297+0000 7f78f317f640 1 -- 172.21.15.40:0/3858695791 shutdown_connections 2024-04-26T21:38:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.297+0000 7f78f317f640 1 -- 172.21.15.40:0/3858695791 wait complete. 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.298+0000 7f78f317f640 1 Processor -- start 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.298+0000 7f78f317f640 1 -- start start 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78f317f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0051c0 0x7f78ec15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78f317f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec14cc30 0x7f78ec14d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78f317f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ec0c8920 0x7f78ec0c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78f317f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec15d570 con 0x7f78ec14cc30 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78f317f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec15d3f0 con 0x7f78dc0051c0 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78f317f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78ec06f8e0 con 0x7f78ec0c8920 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec14cc30 0x7f78ec14d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec14cc30 0x7f78ec14d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56908/0 (socket says 172.21.15.40:56908) 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78ebfff640 1 -- 172.21.15.40:0/786689883 learned_addr learned my addr 172.21.15.40:0/786689883 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78f171c640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ec0c8920 0x7f78ec0c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78f0f1b640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0051c0 0x7f78ec15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78ebfff640 1 -- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ec0c8920 msgr2=0x7f78ec0c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78ebfff640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ec0c8920 0x7f78ec0c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78ebfff640 1 -- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0051c0 msgr2=0x7f78ec15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78ebfff640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0051c0 0x7f78ec15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.299+0000 7f78ebfff640 1 -- 172.21.15.40:0/786689883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec14d5f0 con 0x7f78ec14cc30 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.300+0000 7f78ebfff640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec14cc30 0x7f78ec14d0b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f78d800e480 tx=0x7f78d800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.300+0000 7f78e97fa640 1 -- 172.21.15.40:0/786689883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80052e0 con 0x7f78ec14cc30 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.300+0000 7f78e97fa640 1 -- 172.21.15.40:0/786689883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d8018070 con 0x7f78ec14cc30 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.300+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec0c98f0 con 0x7f78ec14cc30 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.300+0000 7f78e97fa640 1 -- 172.21.15.40:0/786689883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8003830 con 0x7f78ec14cc30 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.300+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec1596a0 con 0x7f78ec14cc30 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.301+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ec0694f0 con 0x7f78ec14cc30 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.301+0000 7f78e97fa640 1 -- 172.21.15.40:0/786689883 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78d80039d0 con 0x7f78ec14cc30 2024-04-26T21:38:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.302+0000 7f78e97fa640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d006c8f0 0x7f78d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.302+0000 7f78e97fa640 1 -- 172.21.15.40:0/786689883 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f78d808b910 con 0x7f78ec14cc30 2024-04-26T21:38:04.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.302+0000 7f78f0f1b640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d006c8f0 0x7f78d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.302+0000 7f78f0f1b640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d006c8f0 0x7f78d006edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009a80 tx=0x7f78e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.306+0000 7f78e97fa640 1 -- 172.21.15.40:0/786689883 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78d8059bb0 con 0x7f78ec14cc30 2024-04-26T21:38:04.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.495+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78ec15fc80 con 0x7f78ec14cc30 2024-04-26T21:38:04.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.496+0000 7f78e97fa640 1 -- 172.21.15.40:0/786689883 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+17130 (secure 0 0 0) 0x7f78d80594b0 con 0x7f78ec14cc30 2024-04-26T21:38:04.497 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:04.497 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":55,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:58.306304+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:37:35.784073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T21:37:36.460613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6834/2276808574":"2024-04-27T21:37:58.306284+0000","172.21.15.40:6832/2276808574":"2024-04-27T21:37:58.306284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:04.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d006c8f0 msgr2=0x7f78d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d006c8f0 0x7f78d006edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009a80 tx=0x7f78e0008040 comp rx=0 tx=0).stop 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec14cc30 msgr2=0x7f78ec14d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec14cc30 0x7f78ec14d0b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f78d800e480 tx=0x7f78d800e940 comp rx=0 tx=0).stop 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 shutdown_connections 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78d006c8f0 0x7f78d006edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ec0c8920 0x7f78ec0c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec14cc30 0x7f78ec14d0b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 --2- 172.21.15.40:0/786689883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78dc0051c0 0x7f78ec15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 >> 172.21.15.40:0/786689883 conn(0x7f78ec0d0be0 msgr2=0x7f78ec06e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 shutdown_connections 2024-04-26T21:38:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.498+0000 7f78f317f640 1 -- 172.21.15.40:0/786689883 wait complete. 2024-04-26T21:38:04.510 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:38:04.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:04.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.610+0000 7f92f5a25640 1 Processor -- start 2024-04-26T21:38:04.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.610+0000 7f92f5a25640 1 -- start start 2024-04-26T21:38:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92f5a25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f0162880 0x7f92f00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92f5a25640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92f00b5f20 0x7f92f00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92f5a25640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f00b7e50 con 0x7f92f00d39f0 2024-04-26T21:38:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92f5a25640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f92f00d0580 con 0x7f92f00b5f20 2024-04-26T21:38:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92f5a25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92f00d1800 con 0x7f92f0162880 2024-04-26T21:38:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92ee7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92f00b5f20 0x7f92f00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92ee7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92f00b5f20 0x7f92f00d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56916/0 (socket says 172.21.15.40:56916) 2024-04-26T21:38:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92ee7fc640 1 -- 172.21.15.40:0/146545837 learned_addr learned my addr 172.21.15.40:0/146545837 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92ee7fc640 1 -- 172.21.15.40:0/146545837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f0162880 msgr2=0x7f92f00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92eeffd640 1 --2- 172.21.15.40:0/146545837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f0162880 0x7f92f00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92ee7fc640 1 --2- 172.21.15.40:0/146545837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f0162880 0x7f92f00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92ee7fc640 1 -- 172.21.15.40:0/146545837 >> v1:172.21.15.184:6789/0 conn(0x7f92f00d39f0 legacy=0x7f92f00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92ee7fc640 1 -- 172.21.15.40:0/146545837 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f00d07d0 con 0x7f92f00b5f20 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.611+0000 7f92eeffd640 1 --2- 172.21.15.40:0/146545837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f0162880 0x7f92f00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.612+0000 7f92ee7fc640 1 --2- 172.21.15.40:0/146545837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00b5f20 0x7f92f00d34b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009870 tx=0x7f92e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a36d1f8218674b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.612+0000 7f92edffb640 1 -- 172.21.15.40:0/146545837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4037070 con 0x7f92f00b5f20 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.612+0000 7f92edffb640 1 -- 172.21.15.40:0/146545837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e4032440 con 0x7f92f00b5f20 2024-04-26T21:38:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.612+0000 7f92edffb640 1 -- 172.21.15.40:0/146545837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4032810 con 0x7f92f00b5f20 2024-04-26T21:38:04.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.613+0000 7f92f5a25640 1 -- 172.21.15.40:0/146545837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00b5f20 msgr2=0x7f92f00d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.613+0000 7f92f5a25640 1 --2- 172.21.15.40:0/146545837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00b5f20 0x7f92f00d34b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009870 tx=0x7f92e402a210 comp rx=0 tx=0).stop 2024-04-26T21:38:04.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.613+0000 7f92f5a25640 1 -- 172.21.15.40:0/146545837 shutdown_connections 2024-04-26T21:38:04.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.613+0000 7f92f5a25640 1 --2- 172.21.15.40:0/146545837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00b5f20 0x7f92f00d34b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.613+0000 7f92f5a25640 1 --2- 172.21.15.40:0/146545837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f0162880 0x7f92f00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.613+0000 7f92f5a25640 1 -- 172.21.15.40:0/146545837 >> 172.21.15.40:0/146545837 conn(0x7f92f00d4f60 msgr2=0x7f92f00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:04.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.613+0000 7f92f5a25640 1 -- 172.21.15.40:0/146545837 shutdown_connections 2024-04-26T21:38:04.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.613+0000 7f92f5a25640 1 -- 172.21.15.40:0/146545837 wait complete. 2024-04-26T21:38:04.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.614+0000 7f92f5a25640 1 Processor -- start 2024-04-26T21:38:04.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.614+0000 7f92f5a25640 1 -- start start 2024-04-26T21:38:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92f5a25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f00b5f20 0x7f92f01594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92f5a25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d39f0 0x7f92f00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92f5a25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f0162880 0x7f92f015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92f5a25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0162c60 con 0x7f92f00d39f0 2024-04-26T21:38:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92f5a25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92f00d0d90 con 0x7f92f0162880 2024-04-26T21:38:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92f5a25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92f0162de0 con 0x7f92f00b5f20 2024-04-26T21:38:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d39f0 0x7f92f00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d39f0 0x7f92f00ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56930/0 (socket says 172.21.15.40:56930) 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ee7fc640 1 -- 172.21.15.40:0/2250381690 learned_addr learned my addr 172.21.15.40:0/2250381690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92eeffd640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f00b5f20 0x7f92f01594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ef7fe640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f0162880 0x7f92f015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ee7fc640 1 -- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f00b5f20 msgr2=0x7f92f01594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ee7fc640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f00b5f20 0x7f92f01594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ee7fc640 1 -- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f0162880 msgr2=0x7f92f015bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ee7fc640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f0162880 0x7f92f015bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92ee7fc640 1 -- 172.21.15.40:0/2250381690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f00cce90 con 0x7f92f00d39f0 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.615+0000 7f92eeffd640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f00b5f20 0x7f92f01594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.616+0000 7f92ee7fc640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d39f0 0x7f92f00ce800 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.616+0000 7f92d7fff640 1 -- 172.21.15.40:0/2250381690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4037040 con 0x7f92f00d39f0 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.616+0000 7f92d7fff640 1 -- 172.21.15.40:0/2250381690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e4042070 con 0x7f92f00d39f0 2024-04-26T21:38:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.616+0000 7f92d7fff640 1 -- 172.21.15.40:0/2250381690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4030bd0 con 0x7f92f00d39f0 2024-04-26T21:38:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.616+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f00cd170 con 0x7f92f00d39f0 2024-04-26T21:38:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.616+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f00cd730 con 0x7f92f00d39f0 2024-04-26T21:38:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.617+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f00694f0 con 0x7f92f00d39f0 2024-04-26T21:38:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.617+0000 7f92d7fff640 1 -- 172.21.15.40:0/2250381690 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92e4047460 con 0x7f92f00d39f0 2024-04-26T21:38:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.618+0000 7f92d7fff640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92dc06c8f0 0x7f92dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.618+0000 7f92d7fff640 1 -- 172.21.15.40:0/2250381690 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f92e40ac8b0 con 0x7f92f00d39f0 2024-04-26T21:38:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.618+0000 7f92eeffd640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92dc06c8f0 0x7f92dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.618+0000 7f92eeffd640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92dc06c8f0 0x7f92dc06edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92e800aa30 tx=0x7f92e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.622+0000 7f92d7fff640 1 -- 172.21.15.40:0/2250381690 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92e4045360 con 0x7f92f00d39f0 2024-04-26T21:38:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.846+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92f00d5340 con 0x7f92f00d39f0 2024-04-26T21:38:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.846+0000 7f92d7fff640 1 -- 172.21.15.40:0/2250381690 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5182 (secure 0 0 0) 0x7f92e407ab50 con 0x7f92f00d39f0 2024-04-26T21:38:04.848 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:04.848 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:38:03.333495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/4223063167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4223063167},{"type":"v1","addr":"172.21.15.40:6834","nonce":4223063167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6306,"qdb_cluster":[6306]},"id":3}]} 2024-04-26T21:38:04.848 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 40 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92dc06c8f0 msgr2=0x7f92dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92dc06c8f0 0x7f92dc06edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92e800aa30 tx=0x7f92e8008040 comp rx=0 tx=0).stop 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d39f0 msgr2=0x7f92f00ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d39f0 0x7f92f00ce800 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e40046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 shutdown_connections 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92dc06c8f0 0x7f92dc06edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f0162880 0x7f92f015bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d39f0 0x7f92f00ce800 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 --2- 172.21.15.40:0/2250381690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92f00b5f20 0x7f92f01594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.848+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 >> 172.21.15.40:0/2250381690 conn(0x7f92f00d4f60 msgr2=0x7f92f006d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:04.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.849+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 shutdown_connections 2024-04-26T21:38:04.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.849+0000 7f92f5a25640 1 -- 172.21.15.40:0/2250381690 wait complete. 2024-04-26T21:38:04.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:04.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.960+0000 7fa199d9d640 1 Processor -- start 2024-04-26T21:38:04.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.960+0000 7fa199d9d640 1 -- start start 2024-04-26T21:38:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa199d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1940d3bc0 0x7fa1940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa199d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1940b5f20 0x7fa1940b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa199d9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1940b7e50 con 0x7fa194162880 2024-04-26T21:38:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa199d9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa1940d0580 con 0x7fa1940b5f20 2024-04-26T21:38:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa199d9d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa1940d1800 con 0x7fa1940d3bc0 2024-04-26T21:38:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa193fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1940b5f20 0x7fa1940b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa193fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1940b5f20 0x7fa1940b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56938/0 (socket says 172.21.15.40:56938) 2024-04-26T21:38:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa193fff640 1 -- 172.21.15.40:0/2527258631 learned_addr learned my addr 172.21.15.40:0/2527258631 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa18bfff640 1 --2- 172.21.15.40:0/2527258631 >> v2:172.21.15.184:3300/0 conn(0x7fa1940d3bc0 0x7fa1940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa193fff640 1 -- 172.21.15.40:0/2527258631 >> v2:172.21.15.184:3300/0 conn(0x7fa1940d3bc0 msgr2=0x7fa1940d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa193fff640 1 --2- 172.21.15.40:0/2527258631 >> v2:172.21.15.184:3300/0 conn(0x7fa1940d3bc0 0x7fa1940d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa193fff640 1 -- 172.21.15.40:0/2527258631 >> v1:172.21.15.184:6789/0 conn(0x7fa194162880 legacy=0x7fa1940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.961+0000 7fa193fff640 1 -- 172.21.15.40:0/2527258631 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1940d07d0 con 0x7fa1940b5f20 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.962+0000 7fa193fff640 1 --2- 172.21.15.40:0/2527258631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940b8300 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa19408d3d0 tx=0x7fa18c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5782f297520b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.962+0000 7fa192ffd640 1 -- 172.21.15.40:0/2527258631 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c036070 con 0x7fa1940b5f20 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.962+0000 7fa192ffd640 1 -- 172.21.15.40:0/2527258631 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18c031600 con 0x7fa1940b5f20 2024-04-26T21:38:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.962+0000 7fa192ffd640 1 -- 172.21.15.40:0/2527258631 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0319d0 con 0x7fa1940b5f20 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.963+0000 7fa199d9d640 1 -- 172.21.15.40:0/2527258631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 msgr2=0x7fa1940b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.963+0000 7fa199d9d640 1 --2- 172.21.15.40:0/2527258631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940b8300 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa19408d3d0 tx=0x7fa18c02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.963+0000 7fa199d9d640 1 -- 172.21.15.40:0/2527258631 shutdown_connections 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.963+0000 7fa199d9d640 1 --2- 172.21.15.40:0/2527258631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940b8300 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.963+0000 7fa199d9d640 1 --2- 172.21.15.40:0/2527258631 >> v2:172.21.15.184:3300/0 conn(0x7fa1940d3bc0 0x7fa1940d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.963+0000 7fa199d9d640 1 -- 172.21.15.40:0/2527258631 >> 172.21.15.40:0/2527258631 conn(0x7fa1940d4f60 msgr2=0x7fa1940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.963+0000 7fa199d9d640 1 -- 172.21.15.40:0/2527258631 shutdown_connections 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.963+0000 7fa199d9d640 1 -- 172.21.15.40:0/2527258631 wait complete. 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.964+0000 7fa199d9d640 1 Processor -- start 2024-04-26T21:38:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.964+0000 7fa199d9d640 1 -- start start 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa199d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa199d9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940d3bc0 0x7fa19415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa199d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194162880 0x7fa194150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa199d9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa194162c60 con 0x7fa1940b5f20 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa199d9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1940d0d90 con 0x7fa194162880 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa199d9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa194162de0 con 0x7fa1940d3bc0 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56948/0 (socket says 172.21.15.40:56948) 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 -- 172.21.15.40:0/171089432 learned_addr learned my addr 172.21.15.40:0/171089432 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa18bfff640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940d3bc0 0x7fa19415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa193fff640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194162880 0x7fa194150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 -- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940d3bc0 msgr2=0x7fa19415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940d3bc0 0x7fa19415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 -- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194162880 msgr2=0x7fa194150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:04.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194162880 0x7fa194150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:04.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 -- 172.21.15.40:0/171089432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1941f5350 con 0x7fa1940b5f20 2024-04-26T21:38:04.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa193fff640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194162880 0x7fa194150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:04.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa1937fe640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940ce950 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa18400c430 tx=0x7fa18400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa190ff9640 1 -- 172.21.15.40:0/171089432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184011790 con 0x7fa1940b5f20 2024-04-26T21:38:04.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa190ff9640 1 -- 172.21.15.40:0/171089432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa184017070 con 0x7fa1940b5f20 2024-04-26T21:38:04.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.965+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1941f5630 con 0x7fa1940b5f20 2024-04-26T21:38:04.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.966+0000 7fa190ff9640 1 -- 172.21.15.40:0/171089432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184011b70 con 0x7fa1940b5f20 2024-04-26T21:38:04.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.966+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1941f5bf0 con 0x7fa1940b5f20 2024-04-26T21:38:04.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.966+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1940694f0 con 0x7fa1940b5f20 2024-04-26T21:38:04.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.967+0000 7fa190ff9640 1 -- 172.21.15.40:0/171089432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa184011d10 con 0x7fa1940b5f20 2024-04-26T21:38:04.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.967+0000 7fa190ff9640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa17406c8f0 0x7fa17406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:04.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.968+0000 7fa190ff9640 1 -- 172.21.15.40:0/171089432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7fa18408bb40 con 0x7fa1940b5f20 2024-04-26T21:38:04.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.968+0000 7fa18bfff640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa17406c8f0 0x7fa17406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:04.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.968+0000 7fa18bfff640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa17406c8f0 0x7fa17406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa194069f80 tx=0x7fa17c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:04.972+0000 7fa190ff9640 1 -- 172.21.15.40:0/171089432 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa184059d50 con 0x7fa1940b5f20 2024-04-26T21:38:05.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.199+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1940d0000 con 0x7fa1940b5f20 2024-04-26T21:38:05.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.200+0000 7fa190ff9640 1 -- 172.21.15.40:0/171089432 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5182 (secure 0 0 0) 0x7fa184059650 con 0x7fa1940b5f20 2024-04-26T21:38:05.201 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:05.201 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:37:37.510449+0000","modified":"2024-04-26T21:38:03.333495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/4223063167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4223063167},{"type":"v1","addr":"172.21.15.40:6834","nonce":4223063167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6306,"qdb_cluster":[6306]},"id":3}]} 2024-04-26T21:38:05.202 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 40 2024-04-26T21:38:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa17406c8f0 msgr2=0x7fa17406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa17406c8f0 0x7fa17406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa194069f80 tx=0x7fa17c008040 comp rx=0 tx=0).stop 2024-04-26T21:38:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 msgr2=0x7fa1940ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940ce950 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa18400c430 tx=0x7fa18400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 shutdown_connections 2024-04-26T21:38:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa17406c8f0 0x7fa17406edb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194162880 0x7fa194150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940d3bc0 0x7fa19415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 --2- 172.21.15.40:0/171089432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1940b5f20 0x7fa1940ce950 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 >> 172.21.15.40:0/171089432 conn(0x7fa1940d4f60 msgr2=0x7fa1940d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 shutdown_connections 2024-04-26T21:38:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.202+0000 7fa199d9d640 1 -- 172.21.15.40:0/171089432 wait complete. 2024-04-26T21:38:05.214 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.314+0000 7f53c8f04640 1 Processor -- start 2024-04-26T21:38:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.314+0000 7f53c8f04640 1 -- start start 2024-04-26T21:38:05.316 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:38:05.314+0000 7f8cf965a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.315+0000 7f53c8f04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c40d1380 0x7f53c40d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.315+0000 7f53c8f04640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c40d1ce0 con 0x7f53c4070ae0 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.315+0000 7f53c8f04640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c40ce840 con 0x7f53c415d1e0 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.315+0000 7f53c8f04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f53c40cede0 con 0x7f53c40d1380 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.315+0000 7f53c2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f53c4070ae0 0x7f53c4070f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51896/0 (socket says 172.21.15.40:51896) 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.315+0000 7f53c2d9d640 1 -- 172.21.15.40:0/533011642 learned_addr learned my addr 172.21.15.40:0/533011642 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.315+0000 7f53c1d9b640 1 --2- 172.21.15.40:0/533011642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c40d1380 0x7f53c40d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.316+0000 7f53c159a640 1 -- 172.21.15.40:0/533011642 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 315201340 0 0) 0x7f53c40d1ce0 con 0x7f53c4070ae0 2024-04-26T21:38:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.316+0000 7f53c159a640 1 -- 172.21.15.40:0/533011642 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53b0003520 con 0x7f53c4070ae0 2024-04-26T21:38:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.316+0000 7f53c1d9b640 1 -- 172.21.15.40:0/533011642 >> v1:172.21.15.184:6789/0 conn(0x7f53c415d1e0 legacy=0x7f53c40d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.316+0000 7f53c159a640 1 -- 172.21.15.40:0/533011642 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310782939 0 0) 0x7f53c40ce840 con 0x7f53c415d1e0 2024-04-26T21:38:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.316+0000 7f53c1d9b640 1 -- 172.21.15.40:0/533011642 >> v1:172.21.15.40:6789/0 conn(0x7f53c4070ae0 legacy=0x7f53c4070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.316+0000 7f53c1d9b640 1 -- 172.21.15.40:0/533011642 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c40ceb00 con 0x7f53c40d1380 2024-04-26T21:38:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.316+0000 7f53c1d9b640 1 --2- 172.21.15.40:0/533011642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c40d1380 0x7f53c40d17a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0097d0 tx=0x7f53ac02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3309b3eeb83939 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c159a640 1 -- 172.21.15.40:0/533011642 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac037070 con 0x7f53c40d1380 2024-04-26T21:38:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c159a640 1 -- 172.21.15.40:0/533011642 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ac032440 con 0x7f53c40d1380 2024-04-26T21:38:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c159a640 1 -- 172.21.15.40:0/533011642 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac032810 con 0x7f53c40d1380 2024-04-26T21:38:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c8f04640 1 -- 172.21.15.40:0/533011642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c40d1380 msgr2=0x7f53c40d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c8f04640 1 --2- 172.21.15.40:0/533011642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c40d1380 0x7f53c40d17a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0097d0 tx=0x7f53ac02a210 comp rx=0 tx=0).stop 2024-04-26T21:38:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c8f04640 1 -- 172.21.15.40:0/533011642 shutdown_connections 2024-04-26T21:38:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c8f04640 1 --2- 172.21.15.40:0/533011642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c40d1380 0x7f53c40d17a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c8f04640 1 -- 172.21.15.40:0/533011642 >> 172.21.15.40:0/533011642 conn(0x7f53c40d2da0 msgr2=0x7f53c40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c8f04640 1 -- 172.21.15.40:0/533011642 shutdown_connections 2024-04-26T21:38:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.317+0000 7f53c8f04640 1 -- 172.21.15.40:0/533011642 wait complete. 2024-04-26T21:38:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.319+0000 7f53c8f04640 1 Processor -- start 2024-04-26T21:38:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.319+0000 7f53c8f04640 1 -- start start 2024-04-26T21:38:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.319+0000 7f53c8f04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c4070ae0 0x7f53c4156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.319+0000 7f53c8f04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53c40d1380 0x7f53c40cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.319+0000 7f53c8f04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c415d1e0 0x7f53c4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.319+0000 7f53c8f04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53c40d10f0 con 0x7f53c4070ae0 2024-04-26T21:38:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.319+0000 7f53c8f04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4070650 con 0x7f53c40d1380 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.319+0000 7f53c8f04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f53c40ce4c0 con 0x7f53c415d1e0 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c4070ae0 0x7f53c4156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c4070ae0 0x7f53c4156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56962/0 (socket says 172.21.15.40:56962) 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 -- 172.21.15.40:0/3622087242 learned_addr learned my addr 172.21.15.40:0/3622087242 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c2d9d640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c415d1e0 0x7f53c4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c1d9b640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53c40d1380 0x7f53c40cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 -- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c415d1e0 msgr2=0x7f53c4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c415d1e0 0x7f53c4070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 -- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53c40d1380 msgr2=0x7f53c40cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53c40d1380 0x7f53c40cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 -- 172.21.15.40:0/3622087242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c40b2790 con 0x7f53c4070ae0 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c259c640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c4070ae0 0x7f53c4156fe0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c440 tx=0x7f53bc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53baffd640 1 -- 172.21.15.40:0/3622087242 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc017070 con 0x7f53c4070ae0 2024-04-26T21:38:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53baffd640 1 -- 172.21.15.40:0/3622087242 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53bc005480 con 0x7f53c4070ae0 2024-04-26T21:38:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c40b2a70 con 0x7f53c4070ae0 2024-04-26T21:38:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53baffd640 1 -- 172.21.15.40:0/3622087242 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc003830 con 0x7f53c4070ae0 2024-04-26T21:38:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.320+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c40b3030 con 0x7f53c4070ae0 2024-04-26T21:38:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.321+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c40d1d50 con 0x7f53c4070ae0 2024-04-26T21:38:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.322+0000 7f53baffd640 1 -- 172.21.15.40:0/3622087242 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53bc01a070 con 0x7f53c4070ae0 2024-04-26T21:38:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.322+0000 7f53baffd640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53a406c8f0 0x7f53a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.322+0000 7f53baffd640 1 -- 172.21.15.40:0/3622087242 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7f53bc08afe0 con 0x7f53c4070ae0 2024-04-26T21:38:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.322+0000 7f53c1d9b640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53a406c8f0 0x7f53a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.323+0000 7f53c1d9b640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53a406c8f0 0x7f53a406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f53ac034000 tx=0x7f53ac02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:05.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.327+0000 7f53baffd640 1 -- 172.21.15.40:0/3622087242 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53c40d1d50 con 0x7f53c4070ae0 2024-04-26T21:38:05.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.515+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53c40694f0 con 0x7f53c4070ae0 2024-04-26T21:38:05.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.516+0000 7f53baffd640 1 -- 172.21.15.40:0/3622087242 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+17130 (secure 0 0 0) 0x7f53bc058ba0 con 0x7f53c4070ae0 2024-04-26T21:38:05.518 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:05.518 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":55,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:37:58.306304+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:37:35.784073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T21:37:36.460613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6834/2276808574":"2024-04-27T21:37:58.306284+0000","172.21.15.40:6832/2276808574":"2024-04-27T21:37:58.306284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53a406c8f0 msgr2=0x7f53a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53a406c8f0 0x7f53a406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f53ac034000 tx=0x7f53ac02e000 comp rx=0 tx=0).stop 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c4070ae0 msgr2=0x7f53c4156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c4070ae0 0x7f53c4156fe0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c440 tx=0x7f53bc00c900 comp rx=0 tx=0).stop 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 shutdown_connections 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53a406c8f0 0x7f53a406edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53c415d1e0 0x7f53c4070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53c40d1380 0x7f53c40cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 --2- 172.21.15.40:0/3622087242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c4070ae0 0x7f53c4156fe0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 >> 172.21.15.40:0/3622087242 conn(0x7f53c40d2da0 msgr2=0x7f53c415c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:05.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 shutdown_connections 2024-04-26T21:38:05.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.518+0000 7f53c8f04640 1 -- 172.21.15.40:0/3622087242 wait complete. 2024-04-26T21:38:05.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:38:05.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.631+0000 7fb3e05f0640 1 Processor -- start 2024-04-26T21:38:05.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.631+0000 7fb3e05f0640 1 -- start start 2024-04-26T21:38:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3e05f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3d8162880 0x7fb3d80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3e05f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3d80d3bc0 0x7fb3d80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3e05f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d80b7e50 con 0x7fb3d80b5f20 2024-04-26T21:38:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3e05f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d80d0580 con 0x7fb3d80d3bc0 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3e05f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d80d1800 con 0x7fb3d8162880 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3deb8d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3d80d3bc0 0x7fb3d80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3ddb8b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb3d80b5f20 0x7fb3d80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51908/0 (socket says 172.21.15.40:51908) 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3ddb8b640 1 -- 172.21.15.40:0/2013877144 learned_addr learned my addr 172.21.15.40:0/2013877144 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3deb8d640 1 -- 172.21.15.40:0/2013877144 >> v2:172.21.15.184:3300/0 conn(0x7fb3d8162880 msgr2=0x7fb3d80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3deb8d640 1 --2- 172.21.15.40:0/2013877144 >> v2:172.21.15.184:3300/0 conn(0x7fb3d8162880 0x7fb3d80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3deb8d640 1 -- 172.21.15.40:0/2013877144 >> v1:172.21.15.40:6789/0 conn(0x7fb3d80b5f20 legacy=0x7fb3d80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3deb8d640 1 -- 172.21.15.40:0/2013877144 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d80d07d0 con 0x7fb3d80d3bc0 2024-04-26T21:38:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.632+0000 7fb3dd38a640 1 -- 172.21.15.40:0/2013877144 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733296641 0 0) 0x7fb3d80b7e50 con 0x7fb3d80b5f20 2024-04-26T21:38:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.633+0000 7fb3deb8d640 1 --2- 172.21.15.40:0/2013877144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80d3bc0 0x7fb3d80b8300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb3d808d490 tx=0x7fb3d002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a0fde1967724f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.633+0000 7fb3dd38a640 1 -- 172.21.15.40:0/2013877144 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0007740 con 0x7fb3d80d3bc0 2024-04-26T21:38:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.633+0000 7fb3dd38a640 1 -- 172.21.15.40:0/2013877144 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d00078e0 con 0x7fb3d80d3bc0 2024-04-26T21:38:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.633+0000 7fb3dd38a640 1 -- 172.21.15.40:0/2013877144 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0007cb0 con 0x7fb3d80d3bc0 2024-04-26T21:38:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.634+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2013877144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80d3bc0 msgr2=0x7fb3d80b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.634+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2013877144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80d3bc0 0x7fb3d80b8300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb3d808d490 tx=0x7fb3d002a680 comp rx=0 tx=0).stop 2024-04-26T21:38:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.634+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2013877144 shutdown_connections 2024-04-26T21:38:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.634+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2013877144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80d3bc0 0x7fb3d80b8300 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.634+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2013877144 >> v2:172.21.15.184:3300/0 conn(0x7fb3d8162880 0x7fb3d80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.634+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2013877144 >> 172.21.15.40:0/2013877144 conn(0x7fb3d80d4f60 msgr2=0x7fb3d80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.634+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2013877144 shutdown_connections 2024-04-26T21:38:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.634+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2013877144 wait complete. 2024-04-26T21:38:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.635+0000 7fb3e05f0640 1 Processor -- start 2024-04-26T21:38:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.635+0000 7fb3e05f0640 1 -- start start 2024-04-26T21:38:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3e05f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80b5f20 0x7fb3d815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3e05f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d80d3bc0 0x7fb3d815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3e05f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d8162880 0x7fb3d80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3e05f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8162c60 con 0x7fb3d80b5f20 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3e05f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d80d0d90 con 0x7fb3d80d3bc0 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3e05f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d8162de0 con 0x7fb3d8162880 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80b5f20 0x7fb3d815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80b5f20 0x7fb3d815bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56982/0 (socket says 172.21.15.40:56982) 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 -- 172.21.15.40:0/2791268921 learned_addr learned my addr 172.21.15.40:0/2791268921 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 -- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d8162880 msgr2=0x7fb3d80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3deb8d640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d8162880 0x7fb3d80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3ddb8b640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d80d3bc0 0x7fb3d815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d8162880 0x7fb3d80ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 -- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d80d3bc0 msgr2=0x7fb3d815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d80d3bc0 0x7fb3d815c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 -- 172.21.15.40:0/2791268921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d815c830 con 0x7fb3d80b5f20 2024-04-26T21:38:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3deb8d640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d8162880 0x7fb3d80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3ddb8b640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d80d3bc0 0x7fb3d815c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3de38c640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80b5f20 0x7fb3d815bac0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400b4a0 tx=0x7fb3d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3ceffd640 1 -- 172.21.15.40:0/2791268921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4014070 con 0x7fb3d80b5f20 2024-04-26T21:38:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.636+0000 7fb3ceffd640 1 -- 172.21.15.40:0/2791268921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d4010610 con 0x7fb3d80b5f20 2024-04-26T21:38:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.637+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d815cb10 con 0x7fb3d80b5f20 2024-04-26T21:38:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.637+0000 7fb3ceffd640 1 -- 172.21.15.40:0/2791268921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4010a00 con 0x7fb3d80b5f20 2024-04-26T21:38:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.637+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d815d3b0 con 0x7fb3d80b5f20 2024-04-26T21:38:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.638+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d80694f0 con 0x7fb3d80b5f20 2024-04-26T21:38:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.638+0000 7fb3ceffd640 1 -- 172.21.15.40:0/2791268921 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3d4017070 con 0x7fb3d80b5f20 2024-04-26T21:38:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.638+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.639+0000 7fb3ceffd640 1 -- 172.21.15.40:0/2791268921 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6245+0+0 (secure 0 0 0) 0x7fb3d408b800 con 0x7fb3d80b5f20 2024-04-26T21:38:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.639+0000 7fb3ddb8b640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.639+0000 7fb3ddb8b640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb3d806a070 tx=0x7fb3c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:05.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.643+0000 7fb3ceffd640 1 -- 172.21.15.40:0/2791268921 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3d4059a20 con 0x7fb3d80b5f20 2024-04-26T21:38:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.867+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb3d815d580 con 0x7fb3d80b5f20 2024-04-26T21:38:05.869 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:38:05.868+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:38:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.871+0000 7fb3ceffd640 1 -- 172.21.15.40:0/2791268921 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7fb3d4059320 con 0x7fb3d80b5f20 2024-04-26T21:38:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.873+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 msgr2=0x7fb3bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.873+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb3d806a070 tx=0x7fb3c8009040 comp rx=0 tx=0).stop 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80b5f20 msgr2=0x7fb3d815bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80b5f20 0x7fb3d815bac0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400b4a0 tx=0x7fb3d400b960 comp rx=0 tx=0).stop 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 shutdown_connections 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 secure :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb3d806a070 tx=0x7fb3c8009040 comp rx=0 tx=0).stop 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3d8162880 0x7fb3d80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3d80d3bc0 0x7fb3d815c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 --2- 172.21.15.40:0/2791268921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d80b5f20 0x7fb3d815bac0 secure :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400b4a0 tx=0x7fb3d400b960 comp rx=0 tx=0).stop 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 >> 172.21.15.40:0/2791268921 conn(0x7fb3d80d4f60 msgr2=0x7fb3d806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 shutdown_connections 2024-04-26T21:38:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.874+0000 7fb3e05f0640 1 -- 172.21.15.40:0/2791268921 wait complete. 2024-04-26T21:38:05.876 INFO:tasks.ceph.mds.c.smithi040.stderr: -37> 2024-04-26T21:38:05.314+0000 7f8cf965a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:38:05.886 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:38:05.901 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:38:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.989+0000 7f9faed87640 1 Processor -- start 2024-04-26T21:38:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.989+0000 7f9faed87640 1 -- start start 2024-04-26T21:38:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9faed87640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa815ab60 0x7f9fa806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9faed87640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9fa81617f0 0x7f9fa806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9faed87640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa815af40 con 0x7f9fa815d5f0 2024-04-26T21:38:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9faed87640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa815e320 con 0x7f9fa81617f0 2024-04-26T21:38:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9faed87640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa8160d10 con 0x7f9fa815ab60 2024-04-26T21:38:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9fa7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9fa81617f0 0x7f9fa806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9fa7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9fa81617f0 0x7f9fa806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57002/0 (socket says 172.21.15.40:57002) 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9fa7fff640 1 -- 172.21.15.40:0/1983107583 learned_addr learned my addr 172.21.15.40:0/1983107583 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9facb23640 1 --2- 172.21.15.40:0/1983107583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa815ab60 0x7f9fa806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9fa7fff640 1 -- 172.21.15.40:0/1983107583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa815ab60 msgr2=0x7f9fa806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9fa7fff640 1 --2- 172.21.15.40:0/1983107583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa815ab60 0x7f9fa806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9fa7fff640 1 -- 172.21.15.40:0/1983107583 >> v1:172.21.15.184:6789/0 conn(0x7f9fa815d5f0 legacy=0x7f9fa815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.990+0000 7f9fa7fff640 1 -- 172.21.15.40:0/1983107583 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa8160ff0 con 0x7f9fa81617f0 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.991+0000 7f9fa7fff640 1 --2- 172.21.15.40:0/1983107583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa81617f0 0x7f9fa806c780 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009870 tx=0x7f9f9402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9398b0d7ccce596 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.991+0000 7f9fa77fe640 1 -- 172.21.15.40:0/1983107583 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94037070 con 0x7f9fa81617f0 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.991+0000 7f9fa77fe640 1 -- 172.21.15.40:0/1983107583 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f94032440 con 0x7f9fa81617f0 2024-04-26T21:38:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.991+0000 7f9fa77fe640 1 -- 172.21.15.40:0/1983107583 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94032810 con 0x7f9fa81617f0 2024-04-26T21:38:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.992+0000 7f9faed87640 1 -- 172.21.15.40:0/1983107583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa81617f0 msgr2=0x7f9fa806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.992+0000 7f9faed87640 1 --2- 172.21.15.40:0/1983107583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa81617f0 0x7f9fa806c780 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009870 tx=0x7f9f9402a210 comp rx=0 tx=0).stop 2024-04-26T21:38:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.992+0000 7f9faed87640 1 -- 172.21.15.40:0/1983107583 shutdown_connections 2024-04-26T21:38:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.992+0000 7f9faed87640 1 --2- 172.21.15.40:0/1983107583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa81617f0 0x7f9fa806c780 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.992+0000 7f9faed87640 1 --2- 172.21.15.40:0/1983107583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa815ab60 0x7f9fa806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.992+0000 7f9faed87640 1 -- 172.21.15.40:0/1983107583 >> 172.21.15.40:0/1983107583 conn(0x7f9fa80d1cf0 msgr2=0x7f9fa80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.992+0000 7f9faed87640 1 -- 172.21.15.40:0/1983107583 shutdown_connections 2024-04-26T21:38:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.992+0000 7f9faed87640 1 -- 172.21.15.40:0/1983107583 wait complete. 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.993+0000 7f9faed87640 1 Processor -- start 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.993+0000 7f9faed87640 1 -- start start 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9faed87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa815ab60 0x7f9fa8159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9faed87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa815d5f0 0x7f9fa80d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9faed87640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa81617f0 0x7f9fa80b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9faed87640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80cc4c0 con 0x7f9fa815ab60 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9faed87640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80cc340 con 0x7f9fa815d5f0 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9faed87640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa80cc640 con 0x7f9fa81617f0 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa815ab60 0x7f9fa8159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa815ab60 0x7f9fa8159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57016/0 (socket says 172.21.15.40:57016) 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 -- 172.21.15.40:0/3530624991 learned_addr learned my addr 172.21.15.40:0/3530624991 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9fad324640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa81617f0 0x7f9fa80b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9fa7fff640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa815d5f0 0x7f9fa80d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 -- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa81617f0 msgr2=0x7f9fa80b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa81617f0 0x7f9fa80b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 -- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa815d5f0 msgr2=0x7f9fa80d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa815d5f0 0x7f9fa80d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 -- 172.21.15.40:0/3530624991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa80aea70 con 0x7f9fa815ab60 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.994+0000 7f9facb23640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa815ab60 0x7f9fa8159680 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00d800 tx=0x7f9f9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.995+0000 7f9fa57fa640 1 -- 172.21.15.40:0/3530624991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00b6c0 con 0x7f9fa815ab60 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.995+0000 7f9fa57fa640 1 -- 172.21.15.40:0/3530624991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9c019070 con 0x7f9fa815ab60 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.995+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa80aed50 con 0x7f9fa815ab60 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.995+0000 7f9fa57fa640 1 -- 172.21.15.40:0/3530624991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00ba80 con 0x7f9fa815ab60 2024-04-26T21:38:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.995+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa80af310 con 0x7f9fa815ab60 2024-04-26T21:38:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.996+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa80694f0 con 0x7f9fa815ab60 2024-04-26T21:38:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.996+0000 7f9fa57fa640 1 -- 172.21.15.40:0/3530624991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f9c00bc20 con 0x7f9fa815ab60 2024-04-26T21:38:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.997+0000 7f9fa57fa640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.997+0000 7f9fa57fa640 1 -- 172.21.15.40:0/3530624991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6331+0+0 (secure 0 0 0) 0x7f9f9c08c260 con 0x7f9fa815ab60 2024-04-26T21:38:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.997+0000 7f9fa7fff640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:05.997+0000 7f9fa7fff640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9f94034000 tx=0x7f9f9402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.001+0000 7f9fa57fa640 1 -- 172.21.15.40:0/3530624991 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f9c01c360 con 0x7f9fa815ab60 2024-04-26T21:38:06.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.227+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9fa8071420 con 0x7f9fa815ab60 2024-04-26T21:38:06.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.876+0000 7f9fa57fa640 1 -- 172.21.15.40:0/3530624991 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f9f9c05a420 con 0x7f9fa815ab60 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9f8c06c8f0 msgr2=0x7f9f8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9f94034000 tx=0x7f9f9402e000 comp rx=0 tx=0).stop 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa815ab60 msgr2=0x7f9fa8159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa815ab60 0x7f9fa8159680 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00d800 tx=0x7f9f9c00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 shutdown_connections 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fa81617f0 0x7f9fa80b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa815d5f0 0x7f9fa80d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:06.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 --2- 172.21.15.40:0/3530624991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa815ab60 0x7f9fa8159680 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:06.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 >> 172.21.15.40:0/3530624991 conn(0x7f9fa80d1cf0 msgr2=0x7f9fa80cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:06.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 shutdown_connections 2024-04-26T21:38:06.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:06.879+0000 7f9faed87640 1 -- 172.21.15.40:0/3530624991 wait complete. 2024-04-26T21:38:06.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:38:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.047+0000 7f24f4bbf640 1 Processor -- start 2024-04-26T21:38:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.047+0000 7f24f4bbf640 1 -- start start 2024-04-26T21:38:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.048+0000 7f24f4bbf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24f015f2b0 0x7f24f00d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.048+0000 7f24f4bbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d2080 0x7f24f00d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.048+0000 7f24f4bbf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f00d2e50 con 0x7f24f00d3ef0 2024-04-26T21:38:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.048+0000 7f24f4bbf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f24f00d2c00 con 0x7f24f015f2b0 2024-04-26T21:38:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.048+0000 7f24f4bbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24f00d3a60 con 0x7f24f00d2080 2024-04-26T21:38:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24ee59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24f015f2b0 0x7f24f00d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24eed9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d2080 0x7f24f00d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24edd9b640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f24f00d3ef0 0x7f24f00d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:44798/0 (socket says 172.21.15.40:44798) 2024-04-26T21:38:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24eed9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d2080 0x7f24f00d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:52276/0 (socket says 172.21.15.40:52276) 2024-04-26T21:38:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24ee59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24f015f2b0 0x7f24f00d0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:43998/0 (socket says 172.21.15.40:43998) 2024-04-26T21:38:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24edd9b640 1 -- 172.21.15.40:0/197499626 learned_addr learned my addr 172.21.15.40:0/197499626 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:07.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24ee59c640 1 -- 172.21.15.40:0/197499626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d2080 msgr2=0x7f24f00d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24ee59c640 1 --2- 172.21.15.40:0/197499626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d2080 0x7f24f00d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24ee59c640 1 -- 172.21.15.40:0/197499626 >> v1:172.21.15.184:6789/0 conn(0x7f24f00d3ef0 legacy=0x7f24f00d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.049+0000 7f24ee59c640 1 -- 172.21.15.40:0/197499626 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f00d3130 con 0x7f24f015f2b0 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24ee59c640 1 --2- 172.21.15.40:0/197499626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 0x7f24f00d0750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f24e00097a0 tx=0x7f24e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfbcbc26a0d97c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24ed59a640 1 -- 172.21.15.40:0/197499626 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0037070 con 0x7f24f015f2b0 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24ed59a640 1 -- 172.21.15.40:0/197499626 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e0032440 con 0x7f24f015f2b0 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24ed59a640 1 -- 172.21.15.40:0/197499626 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0032810 con 0x7f24f015f2b0 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24f4bbf640 1 -- 172.21.15.40:0/197499626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 msgr2=0x7f24f00d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/197499626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 0x7f24f00d0750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f24e00097a0 tx=0x7f24e002a180 comp rx=0 tx=0).stop 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24f4bbf640 1 -- 172.21.15.40:0/197499626 shutdown_connections 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/197499626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d2080 0x7f24f00d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.050+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/197499626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 0x7f24f00d0750 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.051+0000 7f24f4bbf640 1 -- 172.21.15.40:0/197499626 >> 172.21.15.40:0/197499626 conn(0x7f24f00d5030 msgr2=0x7f24f00d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.051+0000 7f24f4bbf640 1 -- 172.21.15.40:0/197499626 shutdown_connections 2024-04-26T21:38:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.051+0000 7f24f4bbf640 1 -- 172.21.15.40:0/197499626 wait complete. 2024-04-26T21:38:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.052+0000 7f24f4bbf640 1 Processor -- start 2024-04-26T21:38:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.052+0000 7f24f4bbf640 1 -- start start 2024-04-26T21:38:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24f4bbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24f00d2080 0x7f24f00a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24f4bbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d3ef0 0x7f24f00a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24f4bbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 0x7f24f0155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24f4bbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f24f015f690 con 0x7f24f00d2080 2024-04-26T21:38:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24f4bbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24f00d36f0 con 0x7f24f015f2b0 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24f4bbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24f00b7e80 con 0x7f24f00d3ef0 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24f00d2080 0x7f24f00a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24f00d2080 0x7f24f00a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57020/0 (socket says 172.21.15.40:57020) 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 -- 172.21.15.40:0/1678308387 learned_addr learned my addr 172.21.15.40:0/1678308387 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24edd9b640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d3ef0 0x7f24f00a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24eed9d640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 0x7f24f0155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 -- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d3ef0 msgr2=0x7f24f00a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d3ef0 0x7f24f00a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 -- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 msgr2=0x7f24f0155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 0x7f24f0155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 -- 172.21.15.40:0/1678308387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f009fe00 con 0x7f24f00d2080 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24edd9b640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d3ef0 0x7f24f00a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24eed9d640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 0x7f24f0155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.053+0000 7f24ee59c640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24f00d2080 0x7f24f00a1c10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f24e0035040 tx=0x7f24e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.054+0000 7f24deffd640 1 -- 172.21.15.40:0/1678308387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0037040 con 0x7f24f00d2080 2024-04-26T21:38:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.054+0000 7f24deffd640 1 -- 172.21.15.40:0/1678308387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e0030500 con 0x7f24f00d2080 2024-04-26T21:38:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.054+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f00a00e0 con 0x7f24f00d2080 2024-04-26T21:38:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.054+0000 7f24deffd640 1 -- 172.21.15.40:0/1678308387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0030910 con 0x7f24f00d2080 2024-04-26T21:38:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.054+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f00a0980 con 0x7f24f00d2080 2024-04-26T21:38:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.055+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f0069600 con 0x7f24f00d2080 2024-04-26T21:38:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.055+0000 7f24deffd640 1 -- 172.21.15.40:0/1678308387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24e0044020 con 0x7f24f00d2080 2024-04-26T21:38:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.056+0000 7f24deffd640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24d006c8f0 0x7f24d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.056+0000 7f24deffd640 1 -- 172.21.15.40:0/1678308387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6331+0+0 (secure 0 0 0) 0x7f24e00ac060 con 0x7f24f00d2080 2024-04-26T21:38:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.056+0000 7f24edd9b640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24d006c8f0 0x7f24d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.057+0000 7f24edd9b640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24d006c8f0 0x7f24d006edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f24f00a1970 tx=0x7f24e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.060+0000 7f24deffd640 1 -- 172.21.15.40:0/1678308387 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24e007a220 con 0x7f24f00d2080 2024-04-26T21:38:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.245+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f24f008eb50 con 0x7f24f00d2080 2024-04-26T21:38:07.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.877+0000 7f24deffd640 1 -- 172.21.15.40:0/1678308387 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) ==== 181+0+0 (secure 0 0 0) 0x7f24e0079b20 con 0x7f24f00d2080 2024-04-26T21:38:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:38:07.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24d006c8f0 msgr2=0x7f24d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:07.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24d006c8f0 0x7f24d006edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f24f00a1970 tx=0x7f24e4009040 comp rx=0 tx=0).stop 2024-04-26T21:38:07.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24f00d2080 msgr2=0x7f24f00a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24f00d2080 0x7f24f00a1c10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f24e0035040 tx=0x7f24e00046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 shutdown_connections 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f24d006c8f0 0x7f24d006edb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24f015f2b0 0x7f24f0155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24f00d3ef0 0x7f24f00a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 --2- 172.21.15.40:0/1678308387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24f00d2080 0x7f24f00a1c10 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 >> 172.21.15.40:0/1678308387 conn(0x7f24f00d5030 msgr2=0x7f24f00b82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 shutdown_connections 2024-04-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:07.881+0000 7f24f4bbf640 1 -- 172.21.15.40:0/1678308387 wait complete. 2024-04-26T21:38:07.896 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:38:08.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.006+0000 7fa87fb9d640 1 Processor -- start 2024-04-26T21:38:08.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.007+0000 7fa87fb9d640 1 -- start start 2024-04-26T21:38:08.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.007+0000 7fa87fb9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa87806c950 0x7fa8780b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.007+0000 7fa87fb9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780b66d0 0x7fa87806f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.007+0000 7fa87fb9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8780b5450 con 0x7fa87815f280 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.007+0000 7fa87fb9d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa87815f660 con 0x7fa87806c950 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.007+0000 7fa87fb9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8780b0f00 con 0x7fa8780b66d0 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87d939640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa87815f280 0x7fa87806c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51934/0 (socket says 172.21.15.40:51934) 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87d939640 1 -- 172.21.15.40:0/58610832 learned_addr learned my addr 172.21.15.40:0/58610832 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546690823 0 0) 0x7fa8780b5450 con 0x7fa87815f280 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa864003520 con 0x7fa87815f280 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87e13a640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780b66d0 0x7fa87806f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87d138640 1 --2- 172.21.15.40:0/58610832 >> v2:172.21.15.184:3300/0 conn(0x7fa87806c950 0x7fa8780b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1642406093 0 0) 0x7fa864003520 con 0x7fa87815f280 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780b66d0 msgr2=0x7fa87806f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780b66d0 0x7fa87806f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 >> v2:172.21.15.184:3300/0 conn(0x7fa87806c950 msgr2=0x7fa8780b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 --2- 172.21.15.40:0/58610832 >> v2:172.21.15.184:3300/0 conn(0x7fa87806c950 0x7fa8780b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8780b11a0 con 0x7fa87815f280 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fa874001690 con 0x7fa87815f280 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87d138640 1 --2- 172.21.15.40:0/58610832 >> v2:172.21.15.184:3300/0 conn(0x7fa87806c950 0x7fa8780b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 >> v1:172.21.15.40:6789/0 conn(0x7fa87815f280 legacy=0x7fa87806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780b66d0 0x7fa8640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa864003f30 0x7fa864004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864004d40 0x7fa8640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8780b0f00 con 0x7fa864003f30 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400b9c0 con 0x7fa864004d40 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa86400bb90 con 0x7fa8780b66d0 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87d138640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa864003f30 0x7fa864004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87d939640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780b66d0 0x7fa8640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.008+0000 7fa87e13a640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864004d40 0x7fa8640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.009+0000 7fa87d138640 1 -- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780b66d0 msgr2=0x7fa8640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.009+0000 7fa87d138640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780b66d0 0x7fa8640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.009+0000 7fa87d138640 1 -- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864004d40 msgr2=0x7fa8640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.009+0000 7fa87d138640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864004d40 0x7fa8640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.009+0000 7fa87d138640 1 -- 172.21.15.40:0/58610832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa864003520 con 0x7fa864003f30 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.009+0000 7fa87d138640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa864003f30 0x7fa864004800 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa8680098a0 tx=0x7fa864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87043802301c38f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86800f040 con 0x7fa864003f30 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86802ad20 con 0x7fa864003f30 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87c937640 1 -- 172.21.15.40:0/58610832 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868004250 con 0x7fa864003f30 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87fb9d640 1 -- 172.21.15.40:0/58610832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa864003f30 msgr2=0x7fa864004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87fb9d640 1 --2- 172.21.15.40:0/58610832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa864003f30 0x7fa864004800 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa8680098a0 tx=0x7fa864005830 comp rx=0 tx=0).stop 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87e13a640 1 -- 172.21.15.40:0/58610832 reap_dead start 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87fb9d640 1 -- 172.21.15.40:0/58610832 shutdown_connections 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87fb9d640 1 -- 172.21.15.40:0/58610832 >> 172.21.15.40:0/58610832 conn(0x7fa87806dbd0 msgr2=0x7fa8780b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87fb9d640 1 -- 172.21.15.40:0/58610832 shutdown_connections 2024-04-26T21:38:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.010+0000 7fa87fb9d640 1 -- 172.21.15.40:0/58610832 wait complete. 2024-04-26T21:38:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.011+0000 7fa87fb9d640 1 Processor -- start 2024-04-26T21:38:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.011+0000 7fa87fb9d640 1 -- start start 2024-04-26T21:38:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87fb9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864003f30 0x7fa8780cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87fb9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8781f5560 0x7fa8780cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87fb9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa87815de20 0x7fa8781f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87fb9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8780af9d0 con 0x7fa87815de20 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87fb9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8780b59a0 con 0x7fa864003f30 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87fb9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8780b1480 con 0x7fa8781f5560 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87e13a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa87815de20 0x7fa8781f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87e13a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa87815de20 0x7fa8781f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57026/0 (socket says 172.21.15.40:57026) 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.012+0000 7fa87e13a640 1 -- 172.21.15.40:0/3810945479 learned_addr learned my addr 172.21.15.40:0/3810945479 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87d939640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864003f30 0x7fa8780cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87d138640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8781f5560 0x7fa8780cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87e13a640 1 -- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8781f5560 msgr2=0x7fa8780cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87e13a640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8781f5560 0x7fa8780cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87e13a640 1 -- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864003f30 msgr2=0x7fa8780cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87e13a640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864003f30 0x7fa8780cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87e13a640 1 -- 172.21.15.40:0/3810945479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8780cf890 con 0x7fa87815de20 2024-04-26T21:38:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87d939640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864003f30 0x7fa8780cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87e13a640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa87815de20 0x7fa8781f3cc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa8780714c0 tx=0x7fa87000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa86e7fc640 1 -- 172.21.15.40:0/3810945479 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000fde0 con 0x7fa87815de20 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa86e7fc640 1 -- 172.21.15.40:0/3810945479 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870015430 con 0x7fa87815de20 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa86e7fc640 1 -- 172.21.15.40:0/3810945479 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870015820 con 0x7fa87815de20 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8780d0650 con 0x7fa87815de20 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.013+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8780ceb50 con 0x7fa87815de20 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.014+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa878069600 con 0x7fa87815de20 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.015+0000 7fa86e7fc640 1 -- 172.21.15.40:0/3810945479 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8700159c0 con 0x7fa87815de20 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.015+0000 7fa86e7fc640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85c06c920 0x7fa85c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.015+0000 7fa86e7fc640 1 -- 172.21.15.40:0/3810945479 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5893+0+0 (secure 0 0 0) 0x7fa87008c920 con 0x7fa87815de20 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.015+0000 7fa87d939640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85c06c920 0x7fa85c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.016+0000 7fa87d939640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85c06c920 0x7fa85c06ede0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa87806d260 tx=0x7fa874009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.020+0000 7fa86e7fc640 1 -- 172.21.15.40:0/3810945479 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa870019030 con 0x7fa87815de20 2024-04-26T21:38:08.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.207+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8780d0360 con 0x7fa87815de20 2024-04-26T21:38:08.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.885+0000 7fa86e7fc640 1 -- 172.21.15.40:0/3810945479 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) ==== 169+0+0 (secure 0 0 0) 0x7fa870010040 con 0x7fa87815de20 2024-04-26T21:38:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:38:08.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85c06c920 msgr2=0x7fa85c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:08.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85c06c920 0x7fa85c06ede0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa87806d260 tx=0x7fa874009040 comp rx=0 tx=0).stop 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa87815de20 msgr2=0x7fa8781f3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa87815de20 0x7fa8781f3cc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa8780714c0 tx=0x7fa87000f580 comp rx=0 tx=0).stop 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 shutdown_connections 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85c06c920 0x7fa85c06ede0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa87815de20 0x7fa8781f3cc0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8781f5560 0x7fa8780cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 --2- 172.21.15.40:0/3810945479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa864003f30 0x7fa8780cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.890+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 >> 172.21.15.40:0/3810945479 conn(0x7fa87806dbd0 msgr2=0x7fa87815b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.891+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 shutdown_connections 2024-04-26T21:38:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:08.891+0000 7fa87fb9d640 1 -- 172.21.15.40:0/3810945479 wait complete. 2024-04-26T21:38:08.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:38:09.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.023+0000 7f95bac85640 1 Processor -- start 2024-04-26T21:38:09.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.024+0000 7f95bac85640 1 -- start start 2024-04-26T21:38:09.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.024+0000 7f95bac85640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95b415f3a0 0x7f95b40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.024+0000 7f95bac85640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.024+0000 7f95bac85640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95b40d25a0 con 0x7f95b40d2130 2024-04-26T21:38:09.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.024+0000 7f95bac85640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95b40d0e20 con 0x7f95b415f3a0 2024-04-26T21:38:09.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.024+0000 7f95bac85640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95b40b1d70 con 0x7f95b40d1770 2024-04-26T21:38:09.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.024+0000 7f95b8a21640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95b415f3a0 0x7f95b40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.025+0000 7f95b8a21640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95b415f3a0 0x7f95b40d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56378/0 (socket says 172.21.15.40:56378) 2024-04-26T21:38:09.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.025+0000 7f95b8a21640 1 -- 172.21.15.40:0/483552914 learned_addr learned my addr 172.21.15.40:0/483552914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:09.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.025+0000 7f95b3fff640 1 --2- 172.21.15.40:0/483552914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.025+0000 7f95b8a21640 1 -- 172.21.15.40:0/483552914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 msgr2=0x7f95b40d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:09.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.025+0000 7f95b8a21640 1 --2- 172.21.15.40:0/483552914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.025+0000 7f95b8a21640 1 -- 172.21.15.40:0/483552914 >> v1:172.21.15.184:6789/0 conn(0x7f95b40d2130 legacy=0x7f95b40d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:09.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.025+0000 7f95b8a21640 1 -- 172.21.15.40:0/483552914 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b40b2050 con 0x7f95b415f3a0 2024-04-26T21:38:09.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.025+0000 7f95b3fff640 1 --2- 172.21.15.40:0/483552914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:09.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95b8a21640 1 --2- 172.21.15.40:0/483552914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b40d06f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009870 tx=0x7f95a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a16c38fd0ddc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95b37fe640 1 -- 172.21.15.40:0/483552914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8037070 con 0x7f95b415f3a0 2024-04-26T21:38:09.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95b37fe640 1 -- 172.21.15.40:0/483552914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a8032440 con 0x7f95b415f3a0 2024-04-26T21:38:09.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95b37fe640 1 -- 172.21.15.40:0/483552914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8032810 con 0x7f95b415f3a0 2024-04-26T21:38:09.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95bac85640 1 -- 172.21.15.40:0/483552914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 msgr2=0x7f95b40d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95bac85640 1 --2- 172.21.15.40:0/483552914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b40d06f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009870 tx=0x7f95a802a210 comp rx=0 tx=0).stop 2024-04-26T21:38:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95bac85640 1 -- 172.21.15.40:0/483552914 shutdown_connections 2024-04-26T21:38:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95bac85640 1 --2- 172.21.15.40:0/483552914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95bac85640 1 --2- 172.21.15.40:0/483552914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b40d06f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95bac85640 1 -- 172.21.15.40:0/483552914 >> 172.21.15.40:0/483552914 conn(0x7f95b40d4f90 msgr2=0x7f95b40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95bac85640 1 -- 172.21.15.40:0/483552914 shutdown_connections 2024-04-26T21:38:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.026+0000 7f95bac85640 1 -- 172.21.15.40:0/483552914 wait complete. 2024-04-26T21:38:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.027+0000 7f95bac85640 1 Processor -- start 2024-04-26T21:38:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.028+0000 7f95bac85640 1 -- start start 2024-04-26T21:38:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.028+0000 7f95bac85640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.028+0000 7f95bac85640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95b40d2130 0x7f95b415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.028+0000 7f95bac85640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.028+0000 7f95bac85640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95b415f900 con 0x7f95b415f3a0 2024-04-26T21:38:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.028+0000 7f95bac85640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95b415f780 con 0x7f95b40d2130 2024-04-26T21:38:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.028+0000 7f95bac85640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95b40b3860 con 0x7f95b40d1770 2024-04-26T21:38:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b4160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56380/0 (socket says 172.21.15.40:56380) 2024-04-26T21:38:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 -- 172.21.15.40:0/1793417015 learned_addr learned my addr 172.21.15.40:0/1793417015 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b8a21640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b3fff640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95b40d2130 0x7f95b415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 -- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 msgr2=0x7f95b40ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 -- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95b40d2130 msgr2=0x7f95b415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95b40d2130 0x7f95b415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 -- 172.21.15.40:0/1793417015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b40cef20 con 0x7f95b415f3a0 2024-04-26T21:38:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b9222640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b4160d00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c7b0 tx=0x7f95ac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b17fa640 1 -- 172.21.15.40:0/1793417015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac004b60 con 0x7f95b415f3a0 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b17fa640 1 -- 172.21.15.40:0/1793417015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ac004d00 con 0x7f95b415f3a0 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b40ba4b0 con 0x7f95b415f3a0 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95b17fa640 1 -- 172.21.15.40:0/1793417015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00aa90 con 0x7f95b415f3a0 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.029+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b40b7dd0 con 0x7f95b415f3a0 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.030+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b40694f0 con 0x7f95b415f3a0 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.031+0000 7f95b17fa640 1 -- 172.21.15.40:0/1793417015 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95ac00ac30 con 0x7f95b415f3a0 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.031+0000 7f95b17fa640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.031+0000 7f95b17fa640 1 -- 172.21.15.40:0/1793417015 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f95ac08bbb0 con 0x7f95b415f3a0 2024-04-26T21:38:09.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.032+0000 7f95b8a21640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.032+0000 7f95b8a21640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f959806c8f0 0x7f959806edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95a8034000 tx=0x7f95a802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.036+0000 7f95b17fa640 1 -- 172.21.15.40:0/1793417015 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95ac020070 con 0x7f95b415f3a0 2024-04-26T21:38:09.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.220+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f95b40b6860 con 0x7f95b415f3a0 2024-04-26T21:38:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.223+0000 7f95b17fa640 1 -- 172.21.15.40:0/1793417015 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7f95b40cef20 con 0x7f95b415f3a0 2024-04-26T21:38:09.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f959806c8f0 msgr2=0x7f959806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f959806c8f0 0x7f959806edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95a8034000 tx=0x7f95a802e000 comp rx=0 tx=0).stop 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 msgr2=0x7f95b4160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b4160d00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c7b0 tx=0x7f95ac00cc70 comp rx=0 tx=0).stop 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 shutdown_connections 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95b415f3a0 0x7f95b4160d00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95b40d2130 0x7f95b415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 --2- 172.21.15.40:0/1793417015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95b40d1770 0x7f95b40ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 >> 172.21.15.40:0/1793417015 conn(0x7f95b40d4f90 msgr2=0x7f95b40d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 shutdown_connections 2024-04-26T21:38:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.225+0000 7f95bac85640 1 -- 172.21.15.40:0/1793417015 wait complete. 2024-04-26T21:38:09.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_abort_on_newly_corrupt_dentry 2024-04-26T21:38:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.339+0000 7f7f80664640 1 Processor -- start 2024-04-26T21:38:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.339+0000 7f7f80664640 1 -- start start 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f80664640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f7815d1e0 0x7f7f780d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f80664640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7f780d0f20 0x7f7f780d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f80664640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f780ce840 con 0x7f7f780d18c0 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f80664640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7f780cede0 con 0x7f7f780d0f20 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f80664640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f78070650 con 0x7f7f7815d1e0 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f7ec01640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7f780d18c0 0x7f7f780cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39870/0 (socket says 172.21.15.40:39870) 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f7ec01640 1 -- 172.21.15.40:0/4043190033 learned_addr learned my addr 172.21.15.40:0/4043190033 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f7dbff640 1 --2- 172.21.15.40:0/4043190033 >> v2:172.21.15.184:3300/0 conn(0x7f7f780d0f20 0x7f7f780d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f7e400640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f7815d1e0 0x7f7f780d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1780176981 0 0) 0x7f7f780ce840 con 0x7f7f780d18c0 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.340+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f64003520 con 0x7f7f780d18c0 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3717287586 0 0) 0x7f7f64003520 con 0x7f7f780d18c0 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f7815d1e0 msgr2=0x7f7f780d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f7815d1e0 0x7f7f780d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 >> v2:172.21.15.184:3300/0 conn(0x7f7f780d0f20 msgr2=0x7f7f780d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 --2- 172.21.15.40:0/4043190033 >> v2:172.21.15.184:3300/0 conn(0x7f7f780d0f20 0x7f7f780d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f780ceb00 con 0x7f7f780d18c0 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f7f700019f0 con 0x7f7f780d18c0 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7dbff640 1 --2- 172.21.15.40:0/4043190033 >> v2:172.21.15.184:3300/0 conn(0x7f7f780d0f20 0x7f7f780d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 >> v1:172.21.15.40:6789/0 conn(0x7f7f780d18c0 legacy=0x7f7f780cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f640039b0 0x7f7f64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f7815d1e0 0x7f7f64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f64004d40 0x7f7f640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f780cede0 con 0x7f7f64004d40 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f78070650 con 0x7f7f640039b0 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6400b9c0 con 0x7f7f7815d1e0 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7ec01640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f64004d40 0x7f7f640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7e400640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f640039b0 0x7f7f64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7dbff640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f7815d1e0 0x7f7f64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7ec01640 1 -- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f7815d1e0 msgr2=0x7f7f64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7ec01640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f7815d1e0 0x7f7f64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7ec01640 1 -- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f640039b0 msgr2=0x7f7f64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7ec01640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f640039b0 0x7f7f64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.341+0000 7f7f7ec01640 1 -- 172.21.15.40:0/4043190033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f64003520 con 0x7f7f64004d40 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.342+0000 7f7f7ec01640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f64004d40 0x7f7f640051c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7f700013d0 tx=0x7f7f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01a5306a01bb954 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.342+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000a4e0 con 0x7f7f64004d40 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.342+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7002f070 con 0x7f7f64004d40 2024-04-26T21:38:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.342+0000 7f7f7d3fe640 1 -- 172.21.15.40:0/4043190033 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7002bce0 con 0x7f7f64004d40 2024-04-26T21:38:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.343+0000 7f7f80664640 1 -- 172.21.15.40:0/4043190033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f64004d40 msgr2=0x7f7f640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.343+0000 7f7f80664640 1 --2- 172.21.15.40:0/4043190033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f64004d40 0x7f7f640051c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7f700013d0 tx=0x7f7f64005830 comp rx=0 tx=0).stop 2024-04-26T21:38:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.343+0000 7f7f7ec01640 1 -- 172.21.15.40:0/4043190033 reap_dead start 2024-04-26T21:38:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.343+0000 7f7f80664640 1 -- 172.21.15.40:0/4043190033 shutdown_connections 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.343+0000 7f7f80664640 1 -- 172.21.15.40:0/4043190033 >> 172.21.15.40:0/4043190033 conn(0x7f7f780d2da0 msgr2=0x7f7f780d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.343+0000 7f7f80664640 1 -- 172.21.15.40:0/4043190033 shutdown_connections 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.343+0000 7f7f80664640 1 -- 172.21.15.40:0/4043190033 wait complete. 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.344+0000 7f7f80664640 1 Processor -- start 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.344+0000 7f7f80664640 1 -- start start 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f80664640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f64004d40 0x7f7f780c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f80664640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f780cc040 0x7f7f780cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f80664640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f781f29a0 0x7f7f781f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f80664640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f78071af0 con 0x7f7f780cc040 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f80664640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f780ce4c0 con 0x7f7f781f29a0 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f80664640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f78071c70 con 0x7f7f64004d40 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f780cc040 0x7f7f780cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f780cc040 0x7f7f780cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56396/0 (socket says 172.21.15.40:56396) 2024-04-26T21:38:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 -- 172.21.15.40:0/3469314800 learned_addr learned my addr 172.21.15.40:0/3469314800 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7e400640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f64004d40 0x7f7f780c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7ec01640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f781f29a0 0x7f7f781f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 -- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f64004d40 msgr2=0x7f7f780c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f64004d40 0x7f7f780c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 -- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f781f29a0 msgr2=0x7f7f781f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f781f29a0 0x7f7f781f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 -- 172.21.15.40:0/3469314800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f780c74d0 con 0x7f7f780cc040 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f7dbff640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f780cc040 0x7f7f780cc4a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7f74011cd0 tx=0x7f7f7400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f6effd640 1 -- 172.21.15.40:0/3469314800 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400fd20 con 0x7f7f780cc040 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f6effd640 1 -- 172.21.15.40:0/3469314800 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f74004030 con 0x7f7f780cc040 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f781f3360 con 0x7f7f780cc040 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f6effd640 1 -- 172.21.15.40:0/3469314800 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74004400 con 0x7f7f780cc040 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.345+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f780c6710 con 0x7f7f780cc040 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.346+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f780cc9e0 con 0x7f7f780cc040 2024-04-26T21:38:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.347+0000 7f7f6effd640 1 -- 172.21.15.40:0/3469314800 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f74012040 con 0x7f7f780cc040 2024-04-26T21:38:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.347+0000 7f7f6effd640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.347+0000 7f7f6effd640 1 -- 172.21.15.40:0/3469314800 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f7f7408c030 con 0x7f7f780cc040 2024-04-26T21:38:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.348+0000 7f7f7e400640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.348+0000 7f7f7e400640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009a80 tx=0x7f7f68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.352+0000 7f7f6effd640 1 -- 172.21.15.40:0/3469314800 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f780cc9e0 con 0x7f7f780cc040 2024-04-26T21:38:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.536+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"} v 0) -- 0x7f7f780694f0 con 0x7f7f780cc040 2024-04-26T21:38:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.539+0000 7f7f6effd640 1 -- 172.21.15.40:0/3469314800 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"}]=0 v17) ==== 116+0+0 (secure 0 0 0) 0x7f7f781f3360 con 0x7f7f780cc040 2024-04-26T21:38:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f5c06c8f0 msgr2=0x7f7f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009a80 tx=0x7f7f68008040 comp rx=0 tx=0).stop 2024-04-26T21:38:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f780cc040 msgr2=0x7f7f780cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f780cc040 0x7f7f780cc4a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7f74011cd0 tx=0x7f7f7400f610 comp rx=0 tx=0).stop 2024-04-26T21:38:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 shutdown_connections 2024-04-26T21:38:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f781f29a0 0x7f7f781f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f780cc040 0x7f7f780cc4a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 --2- 172.21.15.40:0/3469314800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f64004d40 0x7f7f780c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 >> 172.21.15.40:0/3469314800 conn(0x7f7f780d2da0 msgr2=0x7f7f780b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 shutdown_connections 2024-04-26T21:38:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.541+0000 7f7f80664640 1 -- 172.21.15.40:0/3469314800 wait complete. 2024-04-26T21:38:09.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-04-26T21:38:09.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.655+0000 7fe54abaf640 1 Processor -- start 2024-04-26T21:38:09.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.655+0000 7fe54abaf640 1 -- start start 2024-04-26T21:38:09.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe54abaf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe54415f290 0x7fe5440cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe54abaf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe5440cd060 0x7fe544162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe54abaf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe54406f5a0 con 0x7fe5441633c0 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe54abaf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe54406f880 con 0x7fe5440cd060 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe54abaf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe54406fb60 con 0x7fe54415f290 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe543fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe5440cd060 0x7fe544162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe54914c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe5441633c0 0x7fe54406e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39880/0 (socket says 172.21.15.40:39880) 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe543fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe5440cd060 0x7fe544162e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56412/0 (socket says 172.21.15.40:56412) 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe54914c640 1 -- 172.21.15.40:0/399231218 learned_addr learned my addr 172.21.15.40:0/399231218 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe54894b640 1 --2- 172.21.15.40:0/399231218 >> v2:172.21.15.184:3300/0 conn(0x7fe54415f290 0x7fe5440cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe543fff640 1 -- 172.21.15.40:0/399231218 >> v2:172.21.15.184:3300/0 conn(0x7fe54415f290 msgr2=0x7fe5440cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe543fff640 1 --2- 172.21.15.40:0/399231218 >> v2:172.21.15.184:3300/0 conn(0x7fe54415f290 0x7fe5440cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe543fff640 1 -- 172.21.15.40:0/399231218 >> v1:172.21.15.40:6789/0 conn(0x7fe5441633c0 legacy=0x7fe54406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe5437fe640 1 -- 172.21.15.40:0/399231218 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757586756 0 0) 0x7fe54406f5a0 con 0x7fe5441633c0 2024-04-26T21:38:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.656+0000 7fe543fff640 1 -- 172.21.15.40:0/399231218 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54415f670 con 0x7fe5440cd060 2024-04-26T21:38:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.657+0000 7fe543fff640 1 --2- 172.21.15.40:0/399231218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440cd060 0x7fe544162e80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe544075b00 tx=0x7fe53002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf02ec77b5f0112 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.657+0000 7fe5437fe640 1 -- 172.21.15.40:0/399231218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530037070 con 0x7fe5440cd060 2024-04-26T21:38:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.657+0000 7fe5437fe640 1 -- 172.21.15.40:0/399231218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe530032440 con 0x7fe5440cd060 2024-04-26T21:38:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.657+0000 7fe5437fe640 1 -- 172.21.15.40:0/399231218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530032810 con 0x7fe5440cd060 2024-04-26T21:38:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.658+0000 7fe54abaf640 1 -- 172.21.15.40:0/399231218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440cd060 msgr2=0x7fe544162e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.658+0000 7fe54abaf640 1 --2- 172.21.15.40:0/399231218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440cd060 0x7fe544162e80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe544075b00 tx=0x7fe53002a230 comp rx=0 tx=0).stop 2024-04-26T21:38:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.658+0000 7fe54abaf640 1 -- 172.21.15.40:0/399231218 shutdown_connections 2024-04-26T21:38:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.658+0000 7fe54abaf640 1 --2- 172.21.15.40:0/399231218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440cd060 0x7fe544162e80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.658+0000 7fe54abaf640 1 --2- 172.21.15.40:0/399231218 >> v2:172.21.15.184:3300/0 conn(0x7fe54415f290 0x7fe5440cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.658+0000 7fe54abaf640 1 -- 172.21.15.40:0/399231218 >> 172.21.15.40:0/399231218 conn(0x7fe5440d0fd0 msgr2=0x7fe5440d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.658+0000 7fe54abaf640 1 -- 172.21.15.40:0/399231218 shutdown_connections 2024-04-26T21:38:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.658+0000 7fe54abaf640 1 -- 172.21.15.40:0/399231218 wait complete. 2024-04-26T21:38:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.659+0000 7fe54abaf640 1 Processor -- start 2024-04-26T21:38:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.659+0000 7fe54abaf640 1 -- start start 2024-04-26T21:38:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54abaf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440cd060 0x7fe544153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54abaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe54415f290 0x7fe544162260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54abaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5441633c0 0x7fe5440b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54abaf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe5440736d0 con 0x7fe5441633c0 2024-04-26T21:38:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54abaf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5440cd6e0 con 0x7fe5440cd060 2024-04-26T21:38:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54abaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe544073850 con 0x7fe54415f290 2024-04-26T21:38:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5441633c0 0x7fe5440b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5441633c0 0x7fe5440b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56418/0 (socket says 172.21.15.40:56418) 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 -- 172.21.15.40:0/1135441268 learned_addr learned my addr 172.21.15.40:0/1135441268 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54894b640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440cd060 0x7fe544153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe543fff640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe54415f290 0x7fe544162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 -- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe54415f290 msgr2=0x7fe544162260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe54415f290 0x7fe544162260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 -- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440cd060 msgr2=0x7fe544153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440cd060 0x7fe544153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 -- 172.21.15.40:0/1135441268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5441611c0 con 0x7fe5441633c0 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe543fff640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe54415f290 0x7fe544162260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe54914c640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5441633c0 0x7fe5440b4900 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c7a0 tx=0x7fe53c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.660+0000 7fe5417fa640 1 -- 172.21.15.40:0/1135441268 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c018070 con 0x7fe5441633c0 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.661+0000 7fe5417fa640 1 -- 172.21.15.40:0/1135441268 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53c00a640 con 0x7fe5441633c0 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.661+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5441614a0 con 0x7fe5441633c0 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.661+0000 7fe5417fa640 1 -- 172.21.15.40:0/1135441268 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c00aab0 con 0x7fe5441633c0 2024-04-26T21:38:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.661+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe544161a60 con 0x7fe5441633c0 2024-04-26T21:38:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.662+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5440694f0 con 0x7fe5441633c0 2024-04-26T21:38:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.662+0000 7fe5417fa640 1 -- 172.21.15.40:0/1135441268 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe53c01b070 con 0x7fe5441633c0 2024-04-26T21:38:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.662+0000 7fe5417fa640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.663+0000 7fe5417fa640 1 -- 172.21.15.40:0/1135441268 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7fe53c08afb0 con 0x7fe5441633c0 2024-04-26T21:38:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.663+0000 7fe54894b640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.663+0000 7fe54894b640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe538006fd0 tx=0x7fe538008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.667+0000 7fe5417fa640 1 -- 172.21.15.40:0/1135441268 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe53c058eb0 con 0x7fe5441633c0 2024-04-26T21:38:09.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.850+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7fe54415d8f0 con 0x7fe5441633c0 2024-04-26T21:38:09.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.850+0000 7fe5417fa640 1 -- 172.21.15.40:0/1135441268 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7fe53c05d1b0 con 0x7fe5441633c0 2024-04-26T21:38:09.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 msgr2=0x7fe52806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe538006fd0 tx=0x7fe538008040 comp rx=0 tx=0).stop 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5441633c0 msgr2=0x7fe5440b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5441633c0 0x7fe5440b4900 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c7a0 tx=0x7fe53c00cc60 comp rx=0 tx=0).stop 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 shutdown_connections 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5441633c0 0x7fe5440b4900 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe54415f290 0x7fe544162260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 --2- 172.21.15.40:0/1135441268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440cd060 0x7fe544153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 >> 172.21.15.40:0/1135441268 conn(0x7fe5440d0fd0 msgr2=0x7fe5440d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 shutdown_connections 2024-04-26T21:38:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.852+0000 7fe54abaf640 1 -- 172.21.15.40:0/1135441268 wait complete. 2024-04-26T21:38:09.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-04-26T21:38:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.965+0000 7fb7f2b9d640 1 Processor -- start 2024-04-26T21:38:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.965+0000 7fb7f2b9d640 1 -- start start 2024-04-26T21:38:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7f2b9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7ec162810 0x7fb7ec070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7f2b9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7ec0d10d0 con 0x7fb7ec0b2340 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7f2b9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7ec0d1370 con 0x7fb7ec0d1b30 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7f2b9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb7ec0d0750 con 0x7fb7ec162810 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7f0939640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7ec162810 0x7fb7ec070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7ebfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb7ec0b2340 0x7fb7ec0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39886/0 (socket says 172.21.15.40:39886) 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7ebfff640 1 -- 172.21.15.40:0/3998949173 learned_addr learned my addr 172.21.15.40:0/3998949173 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7f0939640 1 -- 172.21.15.40:0/3998949173 >> v1:172.21.15.184:6789/0 conn(0x7fb7ec0d1b30 legacy=0x7fb7ec070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7f0939640 1 -- 172.21.15.40:0/3998949173 >> v1:172.21.15.40:6789/0 conn(0x7fb7ec0b2340 legacy=0x7fb7ec0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7f0939640 1 -- 172.21.15.40:0/3998949173 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec0d01b0 con 0x7fb7ec162810 2024-04-26T21:38:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.966+0000 7fb7eb7fe640 1 -- 172.21.15.40:0/3998949173 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2968141137 0 0) 0x7fb7ec0d10d0 con 0x7fb7ec0b2340 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.967+0000 7fb7f0939640 1 --2- 172.21.15.40:0/3998949173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec162810 0x7fb7ec070350 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0097a0 tx=0x7fb7dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72edeedafa3a803 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.967+0000 7fb7eb7fe640 1 -- 172.21.15.40:0/3998949173 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc037070 con 0x7fb7ec162810 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.967+0000 7fb7eb7fe640 1 -- 172.21.15.40:0/3998949173 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc032440 con 0x7fb7ec162810 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.967+0000 7fb7eb7fe640 1 -- 172.21.15.40:0/3998949173 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc032810 con 0x7fb7ec162810 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.968+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/3998949173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec162810 msgr2=0x7fb7ec070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.968+0000 7fb7f2b9d640 1 --2- 172.21.15.40:0/3998949173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec162810 0x7fb7ec070350 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0097a0 tx=0x7fb7dc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.968+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/3998949173 shutdown_connections 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.968+0000 7fb7f2b9d640 1 --2- 172.21.15.40:0/3998949173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec162810 0x7fb7ec070350 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.968+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/3998949173 >> 172.21.15.40:0/3998949173 conn(0x7fb7ec0d2da0 msgr2=0x7fb7ec0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.968+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/3998949173 shutdown_connections 2024-04-26T21:38:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.968+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/3998949173 wait complete. 2024-04-26T21:38:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.969+0000 7fb7f2b9d640 1 Processor -- start 2024-04-26T21:38:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.969+0000 7fb7f2b9d640 1 -- start start 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f2b9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec0b2340 0x7fb7ec155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f2b9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0d1b30 0x7fb7ec1f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f2b9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec162810 0x7fb7ec06e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f2b9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec070890 con 0x7fb7ec0b2340 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f2b9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0d16a0 con 0x7fb7ec162810 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f2b9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec0cdb00 con 0x7fb7ec0d1b30 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f0939640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec0b2340 0x7fb7ec155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f0939640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec0b2340 0x7fb7ec155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56434/0 (socket says 172.21.15.40:56434) 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f0939640 1 -- 172.21.15.40:0/609467422 learned_addr learned my addr 172.21.15.40:0/609467422 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7ebfff640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0d1b30 0x7fb7ec1f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f113a640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec162810 0x7fb7ec06e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f0939640 1 -- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0d1b30 msgr2=0x7fb7ec1f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f0939640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0d1b30 0x7fb7ec1f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f0939640 1 -- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec162810 msgr2=0x7fb7ec06e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f0939640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec162810 0x7fb7ec06e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.970+0000 7fb7f0939640 1 -- 172.21.15.40:0/609467422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec1f9740 con 0x7fb7ec0b2340 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.971+0000 7fb7f0939640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec0b2340 0x7fb7ec155e20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0097a0 tx=0x7fb7dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.971+0000 7fb7e97fa640 1 -- 172.21.15.40:0/609467422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc037040 con 0x7fb7ec0b2340 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.971+0000 7fb7e97fa640 1 -- 172.21.15.40:0/609467422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc004250 con 0x7fb7ec0b2340 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.971+0000 7fb7e97fa640 1 -- 172.21.15.40:0/609467422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc032e20 con 0x7fb7ec0b2340 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.971+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec0c87a0 con 0x7fb7ec0b2340 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.971+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec0c8d60 con 0x7fb7ec0b2340 2024-04-26T21:38:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.972+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ec0694f0 con 0x7fb7ec0b2340 2024-04-26T21:38:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.972+0000 7fb7e97fa640 1 -- 172.21.15.40:0/609467422 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7dc030770 con 0x7fb7ec0b2340 2024-04-26T21:38:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.973+0000 7fb7e97fa640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d806c8f0 0x7fb7d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.973+0000 7fb7e97fa640 1 -- 172.21.15.40:0/609467422 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7fb7dc0accd0 con 0x7fb7ec0b2340 2024-04-26T21:38:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.973+0000 7fb7ebfff640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d806c8f0 0x7fb7d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.974+0000 7fb7ebfff640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d806c8f0 0x7fb7d806edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec15b260 tx=0x7fb7e0007b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:09.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:09.977+0000 7fb7e97fa640 1 -- 172.21.15.40:0/609467422 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb7ec1f9740 con 0x7fb7ec0b2340 2024-04-26T21:38:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.234+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7fb7ec0b3e50 con 0x7fb7ec0b2340 2024-04-26T21:38:10.418 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:38:10.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:38:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.896+0000 7fb7e97fa640 1 -- 172.21.15.40:0/609467422 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v182) ==== 141+0+0 (secure 0 0 0) 0x7fb7dc07b200 con 0x7fb7ec0b2340 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d806c8f0 msgr2=0x7fb7d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d806c8f0 0x7fb7d806edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec15b260 tx=0x7fb7e0007b50 comp rx=0 tx=0).stop 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec0b2340 msgr2=0x7fb7ec155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec0b2340 0x7fb7ec155e20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0097a0 tx=0x7fb7dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 shutdown_connections 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d806c8f0 0x7fb7d806edb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec162810 0x7fb7ec06e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0d1b30 0x7fb7ec1f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 --2- 172.21.15.40:0/609467422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7ec0b2340 0x7fb7ec155e20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 >> 172.21.15.40:0/609467422 conn(0x7fb7ec0d2da0 msgr2=0x7fb7ec0cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 shutdown_connections 2024-04-26T21:38:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:10.898+0000 7fb7f2b9d640 1 -- 172.21.15.40:0/609467422 wait complete. 2024-04-26T21:38:10.912 INFO:tasks.cephfs_test_runner:That the MDS won't abort when the dentry is already known to be damaged. ... ok 2024-04-26T21:38:10.912 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-04-26T21:38:10.912 INFO:tasks.cephfs_test_runner:test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) 2024-04-26T21:38:10.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:11.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.015+0000 7ff4985c6640 1 Processor -- start 2024-04-26T21:38:11.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.015+0000 7ff4985c6640 1 -- start start 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.015+0000 7ff4985c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff4900cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.015+0000 7ff4985c6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900d10d0 con 0x7ff4900cd930 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.015+0000 7ff4985c6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900d1370 con 0x7ff490162810 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.015+0000 7ff4985c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900d0750 con 0x7ff4900b2340 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff496b63640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff4900cd930 0x7ff4900d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39890/0 (socket says 172.21.15.40:39890) 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff496b63640 1 -- 172.21.15.40:0/4204950600 learned_addr learned my addr 172.21.15.40:0/4204950600 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff495b61640 1 --2- 172.21.15.40:0/4204950600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff4900cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff495360640 1 -- 172.21.15.40:0/4204950600 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1611060338 0 0) 0x7ff4900d10d0 con 0x7ff4900cd930 2024-04-26T21:38:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff495360640 1 -- 172.21.15.40:0/4204950600 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff47c003520 con 0x7ff4900cd930 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff495360640 1 -- 172.21.15.40:0/4204950600 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3114425802 0 0) 0x7ff4900d1370 con 0x7ff490162810 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff495360640 1 -- 172.21.15.40:0/4204950600 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4900d10d0 con 0x7ff490162810 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff495b61640 1 -- 172.21.15.40:0/4204950600 >> v1:172.21.15.184:6789/0 conn(0x7ff490162810 legacy=0x7ff490070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff495b61640 1 -- 172.21.15.40:0/4204950600 >> v1:172.21.15.40:6789/0 conn(0x7ff4900cd930 legacy=0x7ff4900d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.016+0000 7ff495b61640 1 -- 172.21.15.40:0/4204950600 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900d01b0 con 0x7ff4900b2340 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.017+0000 7ff495b61640 1 --2- 172.21.15.40:0/4204950600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff4900cd3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4900d1540 tx=0x7ff48002a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba66339e2e123d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.017+0000 7ff495360640 1 -- 172.21.15.40:0/4204950600 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480037070 con 0x7ff4900b2340 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.017+0000 7ff495360640 1 -- 172.21.15.40:0/4204950600 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff480032440 con 0x7ff4900b2340 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.017+0000 7ff495360640 1 -- 172.21.15.40:0/4204950600 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800327f0 con 0x7ff4900b2340 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.018+0000 7ff4985c6640 1 -- 172.21.15.40:0/4204950600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 msgr2=0x7ff4900cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.018+0000 7ff4985c6640 1 --2- 172.21.15.40:0/4204950600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff4900cd3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4900d1540 tx=0x7ff48002a200 comp rx=0 tx=0).stop 2024-04-26T21:38:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.018+0000 7ff4985c6640 1 -- 172.21.15.40:0/4204950600 shutdown_connections 2024-04-26T21:38:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.018+0000 7ff4985c6640 1 --2- 172.21.15.40:0/4204950600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff4900cd3f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.018+0000 7ff4985c6640 1 -- 172.21.15.40:0/4204950600 >> 172.21.15.40:0/4204950600 conn(0x7ff4900d2da0 msgr2=0x7ff4900d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.018+0000 7ff4985c6640 1 -- 172.21.15.40:0/4204950600 shutdown_connections 2024-04-26T21:38:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.018+0000 7ff4985c6640 1 -- 172.21.15.40:0/4204950600 wait complete. 2024-04-26T21:38:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.019+0000 7ff4985c6640 1 Processor -- start 2024-04-26T21:38:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.019+0000 7ff4985c6640 1 -- start start 2024-04-26T21:38:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff4985c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff49015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff4985c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900cd930 0x7ff4901601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff4985c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff490162810 0x7ff490152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff4985c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900d2240 con 0x7ff4900cd930 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff4985c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900d16a0 con 0x7ff490162810 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff4985c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900d23c0 con 0x7ff4900b2340 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900cd930 0x7ff4901601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900cd930 0x7ff4901601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56438/0 (socket says 172.21.15.40:56438) 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 -- 172.21.15.40:0/1321861534 learned_addr learned my addr 172.21.15.40:0/1321861534 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff496362640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff49015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff496b63640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff490162810 0x7ff490152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 -- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 msgr2=0x7ff49015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff49015e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 -- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff490162810 msgr2=0x7ff490152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff490162810 0x7ff490152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 -- 172.21.15.40:0/1321861534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff490160a10 con 0x7ff4900cd930 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff496b63640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff490162810 0x7ff490152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.020+0000 7ff495b61640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900cd930 0x7ff4901601a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff48002a240 tx=0x7ff4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.021+0000 7ff486ffd640 1 -- 172.21.15.40:0/1321861534 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480041020 con 0x7ff4900cd930 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.021+0000 7ff486ffd640 1 -- 172.21.15.40:0/1321861534 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48003d620 con 0x7ff4900cd930 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.021+0000 7ff486ffd640 1 -- 172.21.15.40:0/1321861534 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480037040 con 0x7ff4900cd930 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.021+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff490160cf0 con 0x7ff4900cd930 2024-04-26T21:38:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.021+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901612b0 con 0x7ff4900cd930 2024-04-26T21:38:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.022+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4900d2020 con 0x7ff4900cd930 2024-04-26T21:38:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.022+0000 7ff486ffd640 1 -- 172.21.15.40:0/1321861534 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff480044020 con 0x7ff4900cd930 2024-04-26T21:38:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.022+0000 7ff486ffd640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.023+0000 7ff486ffd640 1 -- 172.21.15.40:0/1321861534 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7ff4800ac170 con 0x7ff4900cd930 2024-04-26T21:38:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.023+0000 7ff496362640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.023+0000 7ff496362640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007fd0 tx=0x7ff48c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.027+0000 7ff486ffd640 1 -- 172.21.15.40:0/1321861534 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4900d2020 con 0x7ff4900cd930 2024-04-26T21:38:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.212+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4900694f0 con 0x7ff4900cd930 2024-04-26T21:38:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.213+0000 7ff486ffd640 1 -- 172.21.15.40:0/1321861534 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+13387 (secure 0 0 0) 0x7ff48004b030 con 0x7ff4900cd930 2024-04-26T21:38:11.214 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:11.214 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":58,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:38:08.879480+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6834/4223063167":"2024-04-27T21:38:05.869619+0000","172.21.15.40:6832/4223063167":"2024-04-27T21:38:05.869619+0000","172.21.15.40:6834/2276808574":"2024-04-27T21:37:58.306284+0000","172.21.15.40:6832/2276808574":"2024-04-27T21:37:58.306284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff47406c8f0 msgr2=0x7ff47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007fd0 tx=0x7ff48c009040 comp rx=0 tx=0).stop 2024-04-26T21:38:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900cd930 msgr2=0x7ff4901601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900cd930 0x7ff4901601a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff48002a240 tx=0x7ff4800046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 shutdown_connections 2024-04-26T21:38:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff490162810 0x7ff490152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900cd930 0x7ff4901601a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 --2- 172.21.15.40:0/1321861534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900b2340 0x7ff49015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 >> 172.21.15.40:0/1321861534 conn(0x7ff4900d2da0 msgr2=0x7ff49006e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 shutdown_connections 2024-04-26T21:38:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.215+0000 7ff4985c6640 1 -- 172.21.15.40:0/1321861534 wait complete. 2024-04-26T21:38:11.227 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:38:11.227 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:38:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.330+0000 7f8a26597640 1 Processor -- start 2024-04-26T21:38:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.330+0000 7f8a26597640 1 -- start start 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a26597640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200d3bc0 0x7f8a200d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a26597640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a200d1040 con 0x7f8a2015f170 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a26597640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a200c9680 con 0x7f8a2015f550 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a26597640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a200c9430 con 0x7f8a200d3bc0 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a1ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8a2015f170 0x7f8a200d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39892/0 (socket says 172.21.15.40:39892) 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a1ffff640 1 -- 172.21.15.40:0/50962608 learned_addr learned my addr 172.21.15.40:0/50962608 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a24b34640 1 --2- 172.21.15.40:0/50962608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200d3bc0 0x7f8a200d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a1effd640 1 -- 172.21.15.40:0/50962608 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 119230954 0 0) 0x7f8a14001110 con 0x7f8a2015f170 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.331+0000 7f8a1effd640 1 -- 172.21.15.40:0/50962608 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a08003520 con 0x7f8a2015f170 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.332+0000 7f8a24b34640 1 -- 172.21.15.40:0/50962608 >> v1:172.21.15.184:6789/0 conn(0x7f8a2015f550 legacy=0x7f8a200ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.332+0000 7f8a24b34640 1 -- 172.21.15.40:0/50962608 >> v1:172.21.15.40:6789/0 conn(0x7f8a2015f170 legacy=0x7f8a200d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.332+0000 7f8a24b34640 1 -- 172.21.15.40:0/50962608 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a200c9960 con 0x7f8a200d3bc0 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.332+0000 7f8a24b34640 1 --2- 172.21.15.40:0/50962608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200d3bc0 0x7f8a200d5a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009df0 tx=0x7f8a1802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87bc52e09adb2d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a1effd640 1 -- 172.21.15.40:0/50962608 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18007740 con 0x7f8a200d3bc0 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a1effd640 1 -- 172.21.15.40:0/50962608 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a180078e0 con 0x7f8a200d3bc0 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a1effd640 1 -- 172.21.15.40:0/50962608 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18007c90 con 0x7f8a200d3bc0 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a26597640 1 -- 172.21.15.40:0/50962608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200d3bc0 msgr2=0x7f8a200d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a26597640 1 --2- 172.21.15.40:0/50962608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200d3bc0 0x7f8a200d5a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009df0 tx=0x7f8a1802a710 comp rx=0 tx=0).stop 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a26597640 1 -- 172.21.15.40:0/50962608 shutdown_connections 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a26597640 1 --2- 172.21.15.40:0/50962608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200d3bc0 0x7f8a200d5a80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a26597640 1 -- 172.21.15.40:0/50962608 >> 172.21.15.40:0/50962608 conn(0x7f8a200d4f60 msgr2=0x7f8a200d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a26597640 1 -- 172.21.15.40:0/50962608 shutdown_connections 2024-04-26T21:38:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.333+0000 7f8a26597640 1 -- 172.21.15.40:0/50962608 wait complete. 2024-04-26T21:38:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.334+0000 7f8a26597640 1 Processor -- start 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a26597640 1 -- start start 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a26597640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d3bc0 0x7f8a200cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a26597640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2015f170 0x7f8a201f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a26597640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20158f30 0x7f8a201521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a26597640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a200d2e30 con 0x7f8a2015f170 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a26597640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a200d2cb0 con 0x7f8a200d3bc0 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a26597640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a200d2fb0 con 0x7f8a20158f30 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2015f170 0x7f8a201f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a1f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2015f170 0x7f8a201f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56454/0 (socket says 172.21.15.40:56454) 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.335+0000 7f8a1f7fe640 1 -- 172.21.15.40:0/80082412 learned_addr learned my addr 172.21.15.40:0/80082412 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1ffff640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d3bc0 0x7f8a200cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a24b34640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20158f30 0x7f8a201521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1f7fe640 1 -- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20158f30 msgr2=0x7f8a201521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1f7fe640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20158f30 0x7f8a201521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1f7fe640 1 -- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d3bc0 msgr2=0x7f8a200cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1f7fe640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d3bc0 0x7f8a200cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1f7fe640 1 -- 172.21.15.40:0/80082412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a200cf290 con 0x7f8a2015f170 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1ffff640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d3bc0 0x7f8a200cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1f7fe640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2015f170 0x7f8a201f91a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000c3c0 tx=0x7f8a1000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1cff9640 1 -- 172.21.15.40:0/80082412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a100052d0 con 0x7f8a2015f170 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1cff9640 1 -- 172.21.15.40:0/80082412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a10005490 con 0x7f8a2015f170 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a1cff9640 1 -- 172.21.15.40:0/80082412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10007260 con 0x7f8a2015f170 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a200cc040 con 0x7f8a2015f170 2024-04-26T21:38:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.336+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a200cc600 con 0x7f8a2015f170 2024-04-26T21:38:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.337+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a200694f0 con 0x7f8a2015f170 2024-04-26T21:38:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.338+0000 7f8a1cff9640 1 -- 172.21.15.40:0/80082412 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a10007400 con 0x7f8a2015f170 2024-04-26T21:38:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.338+0000 7f8a1cff9640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.338+0000 7f8a1cff9640 1 -- 172.21.15.40:0/80082412 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f8a1008ace0 con 0x7f8a2015f170 2024-04-26T21:38:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.338+0000 7f8a1ffff640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.339+0000 7f8a1ffff640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8a14007fd0 tx=0x7f8a14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.342+0000 7f8a1cff9640 1 -- 172.21.15.40:0/80082412 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a10058c70 con 0x7f8a2015f170 2024-04-26T21:38:11.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.525+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a200d5be0 con 0x7f8a2015f170 2024-04-26T21:38:11.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.526+0000 7f8a1cff9640 1 -- 172.21.15.40:0/80082412 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f8a1005cf70 con 0x7f8a2015f170 2024-04-26T21:38:11.527 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:38:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 msgr2=0x7f8a0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8a14007fd0 tx=0x7f8a14009040 comp rx=0 tx=0).stop 2024-04-26T21:38:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2015f170 msgr2=0x7f8a201f91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2015f170 0x7f8a201f91a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000c3c0 tx=0x7f8a1000c7c0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 shutdown_connections 2024-04-26T21:38:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20158f30 0x7f8a201521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2015f170 0x7f8a201f91a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 --2- 172.21.15.40:0/80082412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d3bc0 0x7f8a200cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 >> 172.21.15.40:0/80082412 conn(0x7f8a200d4f60 msgr2=0x7f8a200d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:11.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 shutdown_connections 2024-04-26T21:38:11.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.528+0000 7f8a26597640 1 -- 172.21.15.40:0/80082412 wait complete. 2024-04-26T21:38:11.540 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-04-26T21:38:11.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.639+0000 7f1de061d640 1 Processor -- start 2024-04-26T21:38:11.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.639+0000 7f1de061d640 1 -- start start 2024-04-26T21:38:11.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1de061d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1dd8162880 0x7f1dd80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1de061d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1de061d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd80b7e50 con 0x7f1dd80d39f0 2024-04-26T21:38:11.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1de061d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd80d0580 con 0x7f1dd8162880 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1de061d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd80d1800 con 0x7f1dd80b5f20 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1dde3b9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1dd8162880 0x7f1dd80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1dde3b9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1dd8162880 0x7f1dd80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56464/0 (socket says 172.21.15.40:56464) 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1dde3b9640 1 -- 172.21.15.40:0/1656816327 learned_addr learned my addr 172.21.15.40:0/1656816327 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1ddebba640 1 --1- 172.21.15.40:0/1656816327 >> v1:172.21.15.40:6789/0 conn(0x7f1dd80d39f0 0x7f1dd80b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39904/0 (socket says 172.21.15.40:39904) 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1dddbb8640 1 --2- 172.21.15.40:0/1656816327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1dde3b9640 1 -- 172.21.15.40:0/1656816327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 msgr2=0x7f1dd80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.640+0000 7f1dde3b9640 1 --2- 172.21.15.40:0/1656816327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.641+0000 7f1dde3b9640 1 -- 172.21.15.40:0/1656816327 >> v1:172.21.15.40:6789/0 conn(0x7f1dd80d39f0 legacy=0x7f1dd80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.641+0000 7f1dde3b9640 1 -- 172.21.15.40:0/1656816327 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd80d07d0 con 0x7f1dd8162880 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.641+0000 7f1ddd3b7640 1 -- 172.21.15.40:0/1656816327 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727284034 0 0) 0x7f1dd80b7e50 con 0x7f1dd80d39f0 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.641+0000 7f1dddbb8640 1 --2- 172.21.15.40:0/1656816327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.641+0000 7f1dde3b9640 1 --2- 172.21.15.40:0/1656816327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd80d2cb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009870 tx=0x7f1dd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105ab4222465ca41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.641+0000 7f1ddd3b7640 1 -- 172.21.15.40:0/1656816327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0037070 con 0x7f1dd8162880 2024-04-26T21:38:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.641+0000 7f1ddd3b7640 1 -- 172.21.15.40:0/1656816327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd0032440 con 0x7f1dd8162880 2024-04-26T21:38:11.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.641+0000 7f1ddd3b7640 1 -- 172.21.15.40:0/1656816327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0032810 con 0x7f1dd8162880 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.642+0000 7f1de061d640 1 -- 172.21.15.40:0/1656816327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 msgr2=0x7f1dd80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.642+0000 7f1de061d640 1 --2- 172.21.15.40:0/1656816327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd80d2cb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009870 tx=0x7f1dd002a270 comp rx=0 tx=0).stop 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.642+0000 7f1de061d640 1 -- 172.21.15.40:0/1656816327 shutdown_connections 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.642+0000 7f1de061d640 1 --2- 172.21.15.40:0/1656816327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.642+0000 7f1de061d640 1 --2- 172.21.15.40:0/1656816327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd80d2cb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.642+0000 7f1de061d640 1 -- 172.21.15.40:0/1656816327 >> 172.21.15.40:0/1656816327 conn(0x7f1dd80d4f60 msgr2=0x7f1dd80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.642+0000 7f1de061d640 1 -- 172.21.15.40:0/1656816327 shutdown_connections 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.642+0000 7f1de061d640 1 -- 172.21.15.40:0/1656816327 wait complete. 2024-04-26T21:38:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.643+0000 7f1de061d640 1 Processor -- start 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.643+0000 7f1de061d640 1 -- start start 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1de061d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1de061d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd80d39f0 0x7f1dd80b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1de061d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd81603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1de061d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8162c60 con 0x7f1dd8162880 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1de061d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80d0d90 con 0x7f1dd80d39f0 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1de061d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd8162de0 con 0x7f1dd80b5f20 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd81603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd81603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56470/0 (socket says 172.21.15.40:56470) 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 -- 172.21.15.40:0/529884496 learned_addr learned my addr 172.21.15.40:0/529884496 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1dde3b9640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1dddbb8640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd80d39f0 0x7f1dd80b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 -- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 msgr2=0x7f1dd80b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 -- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd80d39f0 msgr2=0x7f1dd80b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd80d39f0 0x7f1dd80b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 -- 172.21.15.40:0/529884496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd80ccbb0 con 0x7f1dd8162880 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1dde3b9640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1dddbb8640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd80d39f0 0x7f1dd80b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.644+0000 7f1ddebba640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd81603e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800c7a0 tx=0x7f1dc800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.645+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/529884496 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8018070 con 0x7f1dd8162880 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.645+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/529884496 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc800a640 con 0x7f1dd8162880 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.645+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd80cce90 con 0x7f1dd8162880 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.645+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/529884496 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800aab0 con 0x7f1dd8162880 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.645+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd80cdb90 con 0x7f1dd8162880 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.646+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd80694f0 con 0x7f1dd8162880 2024-04-26T21:38:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.646+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/529884496 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dc801b070 con 0x7f1dd8162880 2024-04-26T21:38:11.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.647+0000 7f1dd6ffd640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:11.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.647+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/529884496 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f1dc808af50 con 0x7f1dd8162880 2024-04-26T21:38:11.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.647+0000 7f1dde3b9640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:11.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.647+0000 7f1dde3b9640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0034000 tx=0x7f1dd002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:11.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.651+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/529884496 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1dc8059530 con 0x7f1dd8162880 2024-04-26T21:38:11.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:11.909+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7f1dd80b5a40 con 0x7f1dd8162880 2024-04-26T21:38:12.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.902+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/529884496 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v184) ==== 145+0+0 (secure 0 0 0) 0x7f1dc8058e30 con 0x7f1dd8162880 2024-04-26T21:38:12.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 msgr2=0x7f1dc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:12.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0034000 tx=0x7f1dd002e000 comp rx=0 tx=0).stop 2024-04-26T21:38:12.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 msgr2=0x7f1dd81603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:12.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd81603e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800c7a0 tx=0x7f1dc800cc60 comp rx=0 tx=0).stop 2024-04-26T21:38:12.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 shutdown_connections 2024-04-26T21:38:12.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:12.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd8162880 0x7f1dd81603e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:12.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd80d39f0 0x7f1dd80b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:12.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 --2- 172.21.15.40:0/529884496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80b5f20 0x7f1dd80b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:12.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 >> 172.21.15.40:0/529884496 conn(0x7f1dd80d4f60 msgr2=0x7f1dd80b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:12.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 shutdown_connections 2024-04-26T21:38:12.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:12.905+0000 7f1de061d640 1 -- 172.21.15.40:0/529884496 wait complete. 2024-04-26T21:38:12.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:38:13.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.029+0000 7fbcc2293640 1 Processor -- start 2024-04-26T21:38:13.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.030+0000 7fbcc2293640 1 -- start start 2024-04-26T21:38:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.030+0000 7fbcc2293640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcbc0d1240 0x7fbcbc0d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.030+0000 7fbcc2293640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b3610 0x7fbcbc0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.030+0000 7fbcc2293640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcbc0d1b60 con 0x7fbcbc0b4360 2024-04-26T21:38:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.030+0000 7fbcc2293640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbcbc0cf210 con 0x7fbcbc0d1240 2024-04-26T21:38:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.030+0000 7fbcc2293640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc0ce5e0 con 0x7fbcbc0b3610 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcbbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcbc0d1240 0x7fbcbc0d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcbb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbcbc0b4360 0x7fbcbc0cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39918/0 (socket says 172.21.15.40:39918) 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcbb7fe640 1 -- 172.21.15.40:0/293771120 learned_addr learned my addr 172.21.15.40:0/293771120 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcc0830640 1 --2- 172.21.15.40:0/293771120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b3610 0x7fbcbc0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcbbfff640 1 -- 172.21.15.40:0/293771120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b3610 msgr2=0x7fbcbc0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcbbfff640 1 --2- 172.21.15.40:0/293771120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b3610 0x7fbcbc0cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcbbfff640 1 -- 172.21.15.40:0/293771120 >> v1:172.21.15.40:6789/0 conn(0x7fbcbc0b4360 legacy=0x7fbcbc0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcbaffd640 1 -- 172.21.15.40:0/293771120 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335853084 0 0) 0x7fbcbc0d1b60 con 0x7fbcbc0b4360 2024-04-26T21:38:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.031+0000 7fbcbbfff640 1 -- 172.21.15.40:0/293771120 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc0ce8c0 con 0x7fbcbc0d1240 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcbbfff640 1 --2- 172.21.15.40:0/293771120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0d1240 0x7fbcbc0d1620 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0097a0 tx=0x7fbcac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58d6fa1bb04a030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcbaffd640 1 -- 172.21.15.40:0/293771120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac037070 con 0x7fbcbc0d1240 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcbaffd640 1 -- 172.21.15.40:0/293771120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcac032440 con 0x7fbcbc0d1240 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcbaffd640 1 -- 172.21.15.40:0/293771120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac032810 con 0x7fbcbc0d1240 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcc2293640 1 -- 172.21.15.40:0/293771120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0d1240 msgr2=0x7fbcbc0d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcc2293640 1 --2- 172.21.15.40:0/293771120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0d1240 0x7fbcbc0d1620 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0097a0 tx=0x7fbcac02a180 comp rx=0 tx=0).stop 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcc2293640 1 -- 172.21.15.40:0/293771120 shutdown_connections 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcc2293640 1 --2- 172.21.15.40:0/293771120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b3610 0x7fbcbc0cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcc2293640 1 --2- 172.21.15.40:0/293771120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0d1240 0x7fbcbc0d1620 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcc2293640 1 -- 172.21.15.40:0/293771120 >> 172.21.15.40:0/293771120 conn(0x7fbcbc0d2e60 msgr2=0x7fbcbc0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcc2293640 1 -- 172.21.15.40:0/293771120 shutdown_connections 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.032+0000 7fbcc2293640 1 -- 172.21.15.40:0/293771120 wait complete. 2024-04-26T21:38:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.034+0000 7fbcc2293640 1 Processor -- start 2024-04-26T21:38:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.034+0000 7fbcc2293640 1 -- start start 2024-04-26T21:38:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcc2293640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3610 0x7fbcbc0b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcc2293640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0d1240 0x7fbcbc159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcc2293640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b6900 0x7fbcbc15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcc2293640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc0c6610 con 0x7fbcbc0b3610 2024-04-26T21:38:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcc2293640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc0cf460 con 0x7fbcbc0d1240 2024-04-26T21:38:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcc2293640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc0c6790 con 0x7fbcbc0b6900 2024-04-26T21:38:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3610 0x7fbcbc0b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3610 0x7fbcbc0b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56478/0 (socket says 172.21.15.40:56478) 2024-04-26T21:38:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 -- 172.21.15.40:0/4249328695 learned_addr learned my addr 172.21.15.40:0/4249328695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcc0830640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b6900 0x7fbcbc15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbb7fe640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0d1240 0x7fbcbc159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 -- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b6900 msgr2=0x7fbcbc15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b6900 0x7fbcbc15ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 -- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0d1240 msgr2=0x7fbcbc159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0d1240 0x7fbcbc159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 -- 172.21.15.40:0/4249328695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc0b77c0 con 0x7fbcbc0b3610 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcbbfff640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3610 0x7fbcbc0b71e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbcac000c00 tx=0x7fbcac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcb8ff9640 1 -- 172.21.15.40:0/4249328695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac037040 con 0x7fbcbc0b3610 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcb8ff9640 1 -- 172.21.15.40:0/4249328695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcac042070 con 0x7fbcbc0b3610 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcb8ff9640 1 -- 172.21.15.40:0/4249328695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac030bb0 con 0x7fbcbc0b3610 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.035+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc0b7aa0 con 0x7fbcbc0b3610 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.036+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc0b6db0 con 0x7fbcbc0b3610 2024-04-26T21:38:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.036+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc0694f0 con 0x7fbcbc0b3610 2024-04-26T21:38:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.037+0000 7fbcb8ff9640 1 -- 172.21.15.40:0/4249328695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcac047460 con 0x7fbcbc0b3610 2024-04-26T21:38:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.037+0000 7fbcb8ff9640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbca806c8f0 0x7fbca806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.037+0000 7fbcb8ff9640 1 -- 172.21.15.40:0/4249328695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7fbcac0ac400 con 0x7fbcbc0b3610 2024-04-26T21:38:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.038+0000 7fbcbb7fe640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbca806c8f0 0x7fbca806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.038+0000 7fbcbb7fe640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbca806c8f0 0x7fbca806edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0a0f00 tx=0x7fbcb0007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.042+0000 7fbcb8ff9640 1 -- 172.21.15.40:0/4249328695 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcac032e20 con 0x7fbcbc0b3610 2024-04-26T21:38:13.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.228+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbcbc15dcc0 con 0x7fbcbc0b3610 2024-04-26T21:38:13.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.231+0000 7fbcb8ff9640 1 -- 172.21.15.40:0/4249328695 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fbcac07a390 con 0x7fbcbc0b3610 2024-04-26T21:38:13.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbca806c8f0 msgr2=0x7fbca806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbca806c8f0 0x7fbca806edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0a0f00 tx=0x7fbcb0007d50 comp rx=0 tx=0).stop 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3610 msgr2=0x7fbcbc0b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3610 0x7fbcbc0b71e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbcac000c00 tx=0x7fbcac0046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 shutdown_connections 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbca806c8f0 0x7fbca806edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b6900 0x7fbcbc15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0d1240 0x7fbcbc159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 --2- 172.21.15.40:0/4249328695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3610 0x7fbcbc0b71e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 >> 172.21.15.40:0/4249328695 conn(0x7fbcbc0d2e60 msgr2=0x7fbcbc0b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 shutdown_connections 2024-04-26T21:38:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.233+0000 7fbcc2293640 1 -- 172.21.15.40:0/4249328695 wait complete. 2024-04-26T21:38:13.245 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:38:13.246 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:38:13.291 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:38:13.336 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:38:13.336 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:38:13.336 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:13.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:38:13.353 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:38:13.354 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:13.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:38:13.372 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:38:13.372 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:13.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:38:13.408 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:38:13.408 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:13.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:38:13.428 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:38:13.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.547+0000 7f0b97119640 1 Processor -- start 2024-04-26T21:38:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.547+0000 7f0b97119640 1 -- start start 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b97119640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b97119640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0b900cd930 0x7f0b900d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b97119640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b900d10d0 con 0x7f0b90162810 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b97119640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0b900d0750 con 0x7f0b900cd930 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b97119640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b900d16a0 con 0x7f0b900b2340 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b94eb5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0b90162810 0x7f0b90070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39928/0 (socket says 172.21.15.40:39928) 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b94eb5640 1 -- 172.21.15.40:0/3370793787 learned_addr learned my addr 172.21.15.40:0/3370793787 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.548+0000 7f0b956b6640 1 --2- 172.21.15.40:0/3370793787 >> v2:172.21.15.184:3300/0 conn(0x7f0b900cd930 0x7f0b900d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.549+0000 7f0b8f7fe640 1 -- 172.21.15.40:0/3370793787 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268701695 0 0) 0x7f0b900d10d0 con 0x7f0b90162810 2024-04-26T21:38:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.549+0000 7f0b8f7fe640 1 -- 172.21.15.40:0/3370793787 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b78003520 con 0x7f0b90162810 2024-04-26T21:38:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.549+0000 7f0b956b6640 1 -- 172.21.15.40:0/3370793787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 msgr2=0x7f0b900cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.549+0000 7f0b956b6640 1 --2- 172.21.15.40:0/3370793787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.549+0000 7f0b956b6640 1 -- 172.21.15.40:0/3370793787 >> v1:172.21.15.40:6789/0 conn(0x7f0b90162810 legacy=0x7f0b90070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.549+0000 7f0b8f7fe640 1 -- 172.21.15.40:0/3370793787 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1862447621 0 0) 0x7f0b78003520 con 0x7f0b90162810 2024-04-26T21:38:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.549+0000 7f0b956b6640 1 -- 172.21.15.40:0/3370793787 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b900d1370 con 0x7f0b900cd930 2024-04-26T21:38:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.549+0000 7f0b8ffff640 1 --2- 172.21.15.40:0/3370793787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b956b6640 1 --2- 172.21.15.40:0/3370793787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b900cd930 0x7f0b900d1b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b88009d90 tx=0x7f0b8802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1947ee8a1ddc2f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b8f7fe640 1 -- 172.21.15.40:0/3370793787 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80001670 con 0x7f0b900cd930 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b8f7fe640 1 -- 172.21.15.40:0/3370793787 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b88007540 con 0x7f0b900cd930 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b8f7fe640 1 -- 172.21.15.40:0/3370793787 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b880078f0 con 0x7f0b900cd930 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b97119640 1 -- 172.21.15.40:0/3370793787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b900cd930 msgr2=0x7f0b900d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b97119640 1 --2- 172.21.15.40:0/3370793787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b900cd930 0x7f0b900d1b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b88009d90 tx=0x7f0b8802a5f0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b97119640 1 -- 172.21.15.40:0/3370793787 shutdown_connections 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b97119640 1 --2- 172.21.15.40:0/3370793787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b900cd930 0x7f0b900d1b30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b97119640 1 --2- 172.21.15.40:0/3370793787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b97119640 1 -- 172.21.15.40:0/3370793787 >> 172.21.15.40:0/3370793787 conn(0x7f0b900d2da0 msgr2=0x7f0b900d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b97119640 1 -- 172.21.15.40:0/3370793787 shutdown_connections 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.550+0000 7f0b97119640 1 -- 172.21.15.40:0/3370793787 wait complete. 2024-04-26T21:38:13.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.552+0000 7f0b97119640 1 Processor -- start 2024-04-26T21:38:13.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.552+0000 7f0b97119640 1 -- start start 2024-04-26T21:38:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b97119640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b97119640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b900cd930 0x7f0b9015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b97119640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b90162810 0x7f0b9015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b97119640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0b900d2070 con 0x7f0b900cd930 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b97119640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b900d0470 con 0x7f0b90162810 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b97119640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b900d21f0 con 0x7f0b900b2340 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b900cd930 0x7f0b9015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b900cd930 0x7f0b9015c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56484/0 (socket says 172.21.15.40:56484) 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 -- 172.21.15.40:0/4081672129 learned_addr learned my addr 172.21.15.40:0/4081672129 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b94eb5640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b956b6640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b90162810 0x7f0b9015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 -- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 msgr2=0x7f0b900c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 -- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b90162810 msgr2=0x7f0b9015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b90162810 0x7f0b9015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 -- 172.21.15.40:0/4081672129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9015c7f0 con 0x7f0b900cd930 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8ffff640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b900cd930 0x7f0b9015c2b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400ebe0 tx=0x7f0b8400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8d7fa640 1 -- 172.21.15.40:0/4081672129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84017070 con 0x7f0b900cd930 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8d7fa640 1 -- 172.21.15.40:0/4081672129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8400cc30 con 0x7f0b900cd930 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b901f8680 con 0x7f0b900cd930 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b8d7fa640 1 -- 172.21.15.40:0/4081672129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84013470 con 0x7f0b900cd930 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.553+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b901f8c40 con 0x7f0b900cd930 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.555+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b901f9300 con 0x7f0b900cd930 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.555+0000 7f0b8d7fa640 1 -- 172.21.15.40:0/4081672129 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b8401a070 con 0x7f0b900cd930 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.555+0000 7f0b8d7fa640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b7c06c8f0 0x7f0b7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.555+0000 7f0b8d7fa640 1 -- 172.21.15.40:0/4081672129 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f0b8408b8a0 con 0x7f0b900cd930 2024-04-26T21:38:13.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.556+0000 7f0b94eb5640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b7c06c8f0 0x7f0b7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.556+0000 7f0b94eb5640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b7c06c8f0 0x7f0b7c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b80005880 tx=0x7f0b800057f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.559+0000 7f0b8d7fa640 1 -- 172.21.15.40:0/4081672129 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b901f9300 con 0x7f0b900cd930 2024-04-26T21:38:13.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.781+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b900694f0 con 0x7f0b900cd930 2024-04-26T21:38:13.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.782+0000 7f0b8d7fa640 1 -- 172.21.15.40:0/4081672129 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+3869 (secure 0 0 0) 0x7f0b9015c7f0 con 0x7f0b900cd930 2024-04-26T21:38:13.783 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:13.783 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/638752118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":638752118},{"type":"v1","addr":"172.21.15.184:6836","nonce":638752118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/983596258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":983596258},{"type":"v1","addr":"172.21.15.184:6837","nonce":983596258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1478523170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1478523170},{"type":"v1","addr":"172.21.15.40:6835","nonce":1478523170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2620286726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2620286726},{"type":"v1","addr":"172.21.15.40:6834","nonce":2620286726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[]} 2024-04-26T21:38:13.783 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 42 2024-04-26T21:38:13.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b7c06c8f0 msgr2=0x7f0b7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b7c06c8f0 0x7f0b7c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b80005880 tx=0x7f0b800057f0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b900cd930 msgr2=0x7f0b9015c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b900cd930 0x7f0b9015c2b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400ebe0 tx=0x7f0b8400c520 comp rx=0 tx=0).stop 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 shutdown_connections 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b7c06c8f0 0x7f0b7c06edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b90162810 0x7f0b9015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b900cd930 0x7f0b9015c2b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 --2- 172.21.15.40:0/4081672129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b900b2340 0x7f0b900c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 >> 172.21.15.40:0/4081672129 conn(0x7f0b900d2da0 msgr2=0x7f0b900d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 shutdown_connections 2024-04-26T21:38:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.784+0000 7f0b97119640 1 -- 172.21.15.40:0/4081672129 wait complete. 2024-04-26T21:38:13.796 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:38:13.796 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:38:13.796 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:38:13.797 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:38:13.797 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:38:13.797 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:38:13.797 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:38:13.797 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:38:13.797 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:38:13.797 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:38:13.798 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:38:13.798 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:38:13.798 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:38:13.798 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:38:13.798 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:38:13.798 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:38:13.806 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:38:13.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:38:13.807 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:38:13.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:38:13.810 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:38:13.810 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:38:13.811 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:38:13.811 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:38:13.849 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:38:13.849 DEBUG:teuthology.parallel:result is None 2024-04-26T21:38:13.850 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:38:13.850 DEBUG:teuthology.parallel:result is None 2024-04-26T21:38:13.853 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:38:13.853 DEBUG:teuthology.parallel:result is None 2024-04-26T21:38:13.853 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:38:13.854 DEBUG:teuthology.parallel:result is None 2024-04-26T21:38:13.854 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:38:13.926 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:38:13.926 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:38:13.931 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:38:13.932 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:38:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.959+0000 7fa439b9d640 1 Processor -- start 2024-04-26T21:38:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.959+0000 7fa439b9d640 1 -- start start 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa439b9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa43415f3a0 0x7fa4340d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa439b9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d2330 0x7fa4340b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa439b9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4340d0e20 con 0x7fa4340d1770 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa439b9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa4340b1d70 con 0x7fa43415f3a0 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa439b9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4340b2050 con 0x7fa4340d2330 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa4337fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa43415f3a0 0x7fa4340d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa4337fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa43415f3a0 0x7fa4340d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56512/0 (socket says 172.21.15.40:56512) 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa4337fe640 1 -- 172.21.15.40:0/1262353056 learned_addr learned my addr 172.21.15.40:0/1262353056 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa433fff640 1 --2- 172.21.15.40:0/1262353056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d2330 0x7fa4340b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa4337fe640 1 -- 172.21.15.40:0/1262353056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d2330 msgr2=0x7fa4340b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa4337fe640 1 --2- 172.21.15.40:0/1262353056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d2330 0x7fa4340b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa4337fe640 1 -- 172.21.15.40:0/1262353056 >> v1:172.21.15.40:6789/0 conn(0x7fa4340d1770 legacy=0x7fa4340d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.960+0000 7fa4337fe640 1 -- 172.21.15.40:0/1262353056 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4340b2330 con 0x7fa43415f3a0 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.961+0000 7fa4337fe640 1 --2- 172.21.15.40:0/1262353056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa4340d06f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa424009870 tx=0x7fa42402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c470ace47279a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.961+0000 7fa4327fc640 1 -- 172.21.15.40:0/1262353056 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424037070 con 0x7fa43415f3a0 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.961+0000 7fa4327fc640 1 -- 172.21.15.40:0/1262353056 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa424032440 con 0x7fa43415f3a0 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.961+0000 7fa4327fc640 1 -- 172.21.15.40:0/1262353056 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424032810 con 0x7fa43415f3a0 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.962+0000 7fa439b9d640 1 -- 172.21.15.40:0/1262353056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 msgr2=0x7fa4340d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.962+0000 7fa439b9d640 1 --2- 172.21.15.40:0/1262353056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa4340d06f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa424009870 tx=0x7fa42402a270 comp rx=0 tx=0).stop 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.962+0000 7fa439b9d640 1 -- 172.21.15.40:0/1262353056 shutdown_connections 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.962+0000 7fa439b9d640 1 --2- 172.21.15.40:0/1262353056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d2330 0x7fa4340b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.962+0000 7fa439b9d640 1 --2- 172.21.15.40:0/1262353056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa4340d06f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.962+0000 7fa439b9d640 1 -- 172.21.15.40:0/1262353056 >> 172.21.15.40:0/1262353056 conn(0x7fa4340d4f90 msgr2=0x7fa4340d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.962+0000 7fa439b9d640 1 -- 172.21.15.40:0/1262353056 shutdown_connections 2024-04-26T21:38:13.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.962+0000 7fa439b9d640 1 -- 172.21.15.40:0/1262353056 wait complete. 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.963+0000 7fa439b9d640 1 Processor -- start 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.963+0000 7fa439b9d640 1 -- start start 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa439b9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d1770 0x7fa4340cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa439b9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4340d2330 0x7fa434150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa439b9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa434151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa439b9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa4340b2f10 con 0x7fa43415f3a0 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa439b9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4340b2d90 con 0x7fa4340d2330 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa439b9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4340b3090 con 0x7fa4340d1770 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa434151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa434151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56516/0 (socket says 172.21.15.40:56516) 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 -- 172.21.15.40:0/3870446129 learned_addr learned my addr 172.21.15.40:0/3870446129 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa4337fe640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d1770 0x7fa4340cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa432ffd640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4340d2330 0x7fa434150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 -- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d1770 msgr2=0x7fa4340cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d1770 0x7fa4340cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 -- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4340d2330 msgr2=0x7fa434150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4340d2330 0x7fa434150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 -- 172.21.15.40:0/3870446129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43414efd0 con 0x7fa43415f3a0 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa432ffd640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4340d2330 0x7fa434150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa433fff640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa434151470 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa42c007e60 tx=0x7fa42c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa41bfff640 1 -- 172.21.15.40:0/3870446129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c014620 con 0x7fa43415f3a0 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43414f2b0 con 0x7fa43415f3a0 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.964+0000 7fa41bfff640 1 -- 172.21.15.40:0/3870446129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa42c019070 con 0x7fa43415f3a0 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.965+0000 7fa41bfff640 1 -- 172.21.15.40:0/3870446129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c014a00 con 0x7fa43415f3a0 2024-04-26T21:38:13.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.965+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43415e1a0 con 0x7fa43415f3a0 2024-04-26T21:38:13.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.965+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4340694f0 con 0x7fa43415f3a0 2024-04-26T21:38:13.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.966+0000 7fa41bfff640 1 -- 172.21.15.40:0/3870446129 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa42c014ba0 con 0x7fa43415f3a0 2024-04-26T21:38:13.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.967+0000 7fa41bfff640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa42006c920 0x7fa42006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:13.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.967+0000 7fa41bfff640 1 -- 172.21.15.40:0/3870446129 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7fa42c08c260 con 0x7fa43415f3a0 2024-04-26T21:38:13.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.967+0000 7fa4337fe640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa42006c920 0x7fa42006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:13.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.967+0000 7fa4337fe640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa42006c920 0x7fa42006ede0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa424034000 tx=0x7fa42402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:13.971+0000 7fa41bfff640 1 -- 172.21.15.40:0/3870446129 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa42c05a7c0 con 0x7fa43415f3a0 2024-04-26T21:38:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.170+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa4340cbe20 con 0x7fa43415f3a0 2024-04-26T21:38:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.915+0000 7fa41bfff640 1 -- 172.21.15.40:0/3870446129 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) ==== 97+0+0 (secure 0 0 0) 0x7fa42c05a0c0 con 0x7fa43415f3a0 2024-04-26T21:38:14.917 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:38:14.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa42006c920 msgr2=0x7fa42006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:14.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa42006c920 0x7fa42006ede0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa424034000 tx=0x7fa42402e000 comp rx=0 tx=0).stop 2024-04-26T21:38:14.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 msgr2=0x7fa434151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:14.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa434151470 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa42c007e60 tx=0x7fa42c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:38:14.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 shutdown_connections 2024-04-26T21:38:14.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa42006c920 0x7fa42006ede0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:14.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43415f3a0 0x7fa434151470 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:14.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4340d2330 0x7fa434150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:14.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 --2- 172.21.15.40:0/3870446129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4340d1770 0x7fa4340cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:14.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 >> 172.21.15.40:0/3870446129 conn(0x7fa4340d4f90 msgr2=0x7fa4340d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:14.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 shutdown_connections 2024-04-26T21:38:14.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:14.919+0000 7fa439b9d640 1 -- 172.21.15.40:0/3870446129 wait complete. 2024-04-26T21:38:14.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:38:15.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.041+0000 7f39d15a4640 1 Processor -- start 2024-04-26T21:38:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.041+0000 7f39d15a4640 1 -- start start 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.042+0000 7f39d15a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f39cc1608f0 0x7f39cc0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.042+0000 7f39d15a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.042+0000 7f39d15a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39cc0cde20 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.042+0000 7f39d15a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f39cc0cea30 con 0x7f39cc1608f0 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.042+0000 7f39d15a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39cc0ce100 con 0x7f39cc0d0a90 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.042+0000 7f39cb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f39cc0cfdd0 0x7f39cc160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39956/0 (socket says 172.21.15.40:39956) 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.042+0000 7f39cb7fe640 1 -- 172.21.15.40:0/505001478 learned_addr learned my addr 172.21.15.40:0/505001478 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39ca7fc640 1 --2- 172.21.15.40:0/505001478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39caffd640 1 --2- 172.21.15.40:0/505001478 >> v2:172.21.15.184:3300/0 conn(0x7f39cc1608f0 0x7f39cc0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39c9ffb640 1 -- 172.21.15.40:0/505001478 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084386904 0 0) 0x7f39cc0cde20 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39c9ffb640 1 -- 172.21.15.40:0/505001478 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39b8003520 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39ca7fc640 1 -- 172.21.15.40:0/505001478 >> v2:172.21.15.184:3300/0 conn(0x7f39cc1608f0 msgr2=0x7f39cc0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39ca7fc640 1 --2- 172.21.15.40:0/505001478 >> v2:172.21.15.184:3300/0 conn(0x7f39cc1608f0 0x7f39cc0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39c9ffb640 1 -- 172.21.15.40:0/505001478 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2674319857 0 0) 0x7f39b8003520 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39ca7fc640 1 -- 172.21.15.40:0/505001478 >> v1:172.21.15.40:6789/0 conn(0x7f39cc0cfdd0 legacy=0x7f39cc160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39ca7fc640 1 -- 172.21.15.40:0/505001478 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc0ce3e0 con 0x7f39cc0d0a90 2024-04-26T21:38:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.043+0000 7f39caffd640 1 --2- 172.21.15.40:0/505001478 >> v2:172.21.15.184:3300/0 conn(0x7f39cc1608f0 0x7f39cc0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39ca7fc640 1 --2- 172.21.15.40:0/505001478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cf890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39b40097a0 tx=0x7f39b402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aeed3d8ff72ab99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39c9ffb640 1 -- 172.21.15.40:0/505001478 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4037070 con 0x7f39cc0d0a90 2024-04-26T21:38:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39c9ffb640 1 -- 172.21.15.40:0/505001478 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b402aba0 con 0x7f39cc0d0a90 2024-04-26T21:38:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39c9ffb640 1 -- 172.21.15.40:0/505001478 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0019f0 con 0x7f39cc0d0a90 2024-04-26T21:38:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39d15a4640 1 -- 172.21.15.40:0/505001478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 msgr2=0x7f39cc0cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39d15a4640 1 --2- 172.21.15.40:0/505001478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cf890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39b40097a0 tx=0x7f39b402a110 comp rx=0 tx=0).stop 2024-04-26T21:38:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39d15a4640 1 -- 172.21.15.40:0/505001478 shutdown_connections 2024-04-26T21:38:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39d15a4640 1 --2- 172.21.15.40:0/505001478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cf890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39d15a4640 1 --2- 172.21.15.40:0/505001478 >> v2:172.21.15.184:3300/0 conn(0x7f39cc1608f0 0x7f39cc0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39d15a4640 1 -- 172.21.15.40:0/505001478 >> 172.21.15.40:0/505001478 conn(0x7f39cc0d2ec0 msgr2=0x7f39cc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.044+0000 7f39d15a4640 1 -- 172.21.15.40:0/505001478 shutdown_connections 2024-04-26T21:38:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.045+0000 7f39d15a4640 1 -- 172.21.15.40:0/505001478 wait complete. 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.046+0000 7f39d15a4640 1 Processor -- start 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.046+0000 7f39d15a4640 1 -- start start 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39d15a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39cc0cfdd0 0x7f39cc1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39d15a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39d15a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39cc1608f0 0x7f39cc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39d15a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc0b3310 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39d15a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc0d0ef0 con 0x7f39cc1608f0 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39d15a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39cc0ceec0 con 0x7f39cc0d0a90 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39cc0cfdd0 0x7f39cc1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39cc0cfdd0 0x7f39cc1581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56518/0 (socket says 172.21.15.40:56518) 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 -- 172.21.15.40:0/3586917888 learned_addr learned my addr 172.21.15.40:0/3586917888 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39ca7fc640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39cb7fe640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39cc1608f0 0x7f39cc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 -- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 msgr2=0x7f39cc0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 -- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39cc1608f0 msgr2=0x7f39cc0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39cc1608f0 0x7f39cc0cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 -- 172.21.15.40:0/3586917888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc0cac10 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39caffd640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39cc0cfdd0 0x7f39cc1581f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e960 tx=0x7f39c400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39c37fe640 1 -- 172.21.15.40:0/3586917888 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4017070 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39c37fe640 1 -- 172.21.15.40:0/3586917888 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c400ca00 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.047+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc0caef0 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.048+0000 7f39c37fe640 1 -- 172.21.15.40:0/3586917888 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c400cdd0 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.048+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc0cb4b0 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.048+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc0694f0 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.049+0000 7f39c37fe640 1 -- 172.21.15.40:0/3586917888 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39c401a070 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.049+0000 7f39c37fe640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ac06c8f0 0x7f39ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.049+0000 7f39c37fe640 1 -- 172.21.15.40:0/3586917888 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5249+0+0 (secure 0 0 0) 0x7f39c408bd00 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.050+0000 7f39ca7fc640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ac06c8f0 0x7f39ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.050+0000 7f39ca7fc640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ac06c8f0 0x7f39ac06edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f39b4034000 tx=0x7f39b402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.054+0000 7f39c37fe640 1 -- 172.21.15.40:0/3586917888 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39c405a1d0 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.311+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f39cc0cd1f0 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.311+0000 7f39c37fe640 1 -- 172.21.15.40:0/3586917888 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f39c4059ad0 con 0x7f39cc0cfdd0 2024-04-26T21:38:15.312 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmesG+ERAAJS53jYoKUawIOs2zrG5TLg==", 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmq9cvHRAAiYWrMjfAth8wMbs9YlfWgQ==", 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmLRmvGBAA6xAcjLDF7DMjfDqiUCexKA==", 2024-04-26T21:38:15.314 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmXxkDJBAAbzzf+LUO3Rr8bdzXb+Gj2Q==", 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDYHSxmRkhFBhAAz1UfvJwhA8bolDiRN2eM+Q==", 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:38:15.316 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDbHSxmKAB/BBAAzW5dtz6eczjkDiTQBjfjuA==", 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDeHSxmrienAhAA2JS0l6/Uo9UPc0TeZ1RwRA==", 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDhHSxmv40rARAAN8z+beJe/bTERvO2euMr6w==", 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:38:15.318 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDnHSxmxQ1XCBAAYptVM8WPir+zIHOjbX0tGg==", 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDqHSxmv32IBhAAMROkCLjWTOx3ELdkEiVSmw==", 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.319 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDtHSxmJdPTBBAAO32qvVLuAzqx1xdajL9CFw==", 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDwHSxmcUMYAxAAIvoAo82ApDXERodD+4+cBQ==", 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.320 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw==", 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmifWYMBAA6Nskvqt3MZSvFCMHjslELw==", 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmi6l6LRAAF4EjChVEB8ReULEsl3YppA==", 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmhSMFNhAAjCJvc4FknsSUJQ4NjXRV6Q==", 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDTHSxmQk1bMhAAixlszeAmlXc5cAmkFWjhfw==", 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmYjmPDhAADvvIHrcacI8kOAdeiJGd+Q==", 2024-04-26T21:38:15.324 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmsiwICxAAmKAZ2KS3/dYubutkcGfolA==", 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.325 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:15.326 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:38:15.326 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:38:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ac06c8f0 msgr2=0x7f39ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ac06c8f0 0x7f39ac06edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f39b4034000 tx=0x7f39b402e000 comp rx=0 tx=0).stop 2024-04-26T21:38:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39cc0cfdd0 msgr2=0x7f39cc1581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39cc0cfdd0 0x7f39cc1581f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e960 tx=0x7f39c400ee20 comp rx=0 tx=0).stop 2024-04-26T21:38:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 shutdown_connections 2024-04-26T21:38:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39ac06c8f0 0x7f39ac06edb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39cc1608f0 0x7f39cc0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39cc0d0a90 0x7f39cc0cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 --2- 172.21.15.40:0/3586917888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39cc0cfdd0 0x7f39cc1581f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 >> 172.21.15.40:0/3586917888 conn(0x7f39cc0d2ec0 msgr2=0x7f39cc070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 shutdown_connections 2024-04-26T21:38:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.313+0000 7f39d15a4640 1 -- 172.21.15.40:0/3586917888 wait complete. 2024-04-26T21:38:15.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:15.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.429+0000 7fd4833af640 1 Processor -- start 2024-04-26T21:38:15.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.429+0000 7fd4833af640 1 -- start start 2024-04-26T21:38:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.429+0000 7fd4833af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd47c162880 0x7fd47c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.429+0000 7fd4833af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c0b5f20 0x7fd47c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd4833af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c0b7e50 con 0x7fd47c0d39f0 2024-04-26T21:38:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd4833af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd47c0d0580 con 0x7fd47c162880 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd4833af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd47c0d1800 con 0x7fd47c0b5f20 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd48114b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd47c162880 0x7fd47c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd48114b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd47c162880 0x7fd47c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56530/0 (socket says 172.21.15.40:56530) 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd48114b640 1 -- 172.21.15.40:0/1992958112 learned_addr learned my addr 172.21.15.40:0/1992958112 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd48094a640 1 --2- 172.21.15.40:0/1992958112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c0b5f20 0x7fd47c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd48114b640 1 -- 172.21.15.40:0/1992958112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c0b5f20 msgr2=0x7fd47c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd48114b640 1 --2- 172.21.15.40:0/1992958112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c0b5f20 0x7fd47c0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd48114b640 1 -- 172.21.15.40:0/1992958112 >> v1:172.21.15.184:6789/0 conn(0x7fd47c0d39f0 legacy=0x7fd47c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.430+0000 7fd48114b640 1 -- 172.21.15.40:0/1992958112 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c0d07d0 con 0x7fd47c162880 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.431+0000 7fd48114b640 1 --2- 172.21.15.40:0/1992958112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c162880 0x7fd47c0d2cb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd478009870 tx=0x7fd47802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab63800d75813b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.431+0000 7fd473fff640 1 -- 172.21.15.40:0/1992958112 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478037070 con 0x7fd47c162880 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.431+0000 7fd473fff640 1 -- 172.21.15.40:0/1992958112 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd478032440 con 0x7fd47c162880 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.431+0000 7fd473fff640 1 -- 172.21.15.40:0/1992958112 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478032810 con 0x7fd47c162880 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.431+0000 7fd4833af640 1 -- 172.21.15.40:0/1992958112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c162880 msgr2=0x7fd47c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.431+0000 7fd4833af640 1 --2- 172.21.15.40:0/1992958112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c162880 0x7fd47c0d2cb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd478009870 tx=0x7fd47802a270 comp rx=0 tx=0).stop 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.431+0000 7fd4833af640 1 -- 172.21.15.40:0/1992958112 shutdown_connections 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.432+0000 7fd4833af640 1 --2- 172.21.15.40:0/1992958112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c0b5f20 0x7fd47c0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.432+0000 7fd4833af640 1 --2- 172.21.15.40:0/1992958112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c162880 0x7fd47c0d2cb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.432+0000 7fd4833af640 1 -- 172.21.15.40:0/1992958112 >> 172.21.15.40:0/1992958112 conn(0x7fd47c0d4f60 msgr2=0x7fd47c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.432+0000 7fd4833af640 1 -- 172.21.15.40:0/1992958112 shutdown_connections 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.432+0000 7fd4833af640 1 -- 172.21.15.40:0/1992958112 wait complete. 2024-04-26T21:38:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.433+0000 7fd4833af640 1 Processor -- start 2024-04-26T21:38:15.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.433+0000 7fd4833af640 1 -- start start 2024-04-26T21:38:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.433+0000 7fd4833af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c0b5f20 0x7fd47c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.433+0000 7fd4833af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd47c0d39f0 0x7fd47c0b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.433+0000 7fd4833af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c162880 0x7fd47c0ba230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.433+0000 7fd4833af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c162c60 con 0x7fd47c0b5f20 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.433+0000 7fd4833af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c0d0d90 con 0x7fd47c0d39f0 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.433+0000 7fd4833af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd47c162de0 con 0x7fd47c162880 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c0b5f20 0x7fd47c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c0b5f20 0x7fd47c159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56538/0 (socket says 172.21.15.40:56538) 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 -- 172.21.15.40:0/2157974727 learned_addr learned my addr 172.21.15.40:0/2157974727 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48094a640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd47c0d39f0 0x7fd47c0b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 -- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c162880 msgr2=0x7fd47c0ba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48194c640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c162880 0x7fd47c0ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c162880 0x7fd47c0ba230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 -- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd47c0d39f0 msgr2=0x7fd47c0b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd47c0d39f0 0x7fd47c0b9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 -- 172.21.15.40:0/2157974727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c0ce140 con 0x7fd47c0b5f20 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48094a640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd47c0d39f0 0x7fd47c0b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd48114b640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c0b5f20 0x7fd47c159390 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd478000c00 tx=0x7fd4780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd471ffb640 1 -- 172.21.15.40:0/2157974727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478041020 con 0x7fd47c0b5f20 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd471ffb640 1 -- 172.21.15.40:0/2157974727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4780306c0 con 0x7fd47c0b5f20 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47c0ccbf0 con 0x7fd47c0b5f20 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd471ffb640 1 -- 172.21.15.40:0/2157974727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478037040 con 0x7fd47c0b5f20 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.434+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47c15a0c0 con 0x7fd47c0b5f20 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.435+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd47c0694f0 con 0x7fd47c0b5f20 2024-04-26T21:38:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.436+0000 7fd471ffb640 1 -- 172.21.15.40:0/2157974727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd478044020 con 0x7fd47c0b5f20 2024-04-26T21:38:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.436+0000 7fd471ffb640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd46006c8f0 0x7fd46006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.436+0000 7fd471ffb640 1 -- 172.21.15.40:0/2157974727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5249+0+0 (secure 0 0 0) 0x7fd4780ac190 con 0x7fd47c0b5f20 2024-04-26T21:38:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.436+0000 7fd48094a640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd46006c8f0 0x7fd46006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.437+0000 7fd48094a640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd46006c8f0 0x7fd46006edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd46c006fd0 tx=0x7fd46c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.440+0000 7fd471ffb640 1 -- 172.21.15.40:0/2157974727 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd47804b030 con 0x7fd47c0b5f20 2024-04-26T21:38:15.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.631+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd47c15f830 con 0x7fd47c0b5f20 2024-04-26T21:38:15.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.631+0000 7fd471ffb640 1 -- 172.21.15.40:0/2157974727 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+13128 (secure 0 0 0) 0x7fd47807a780 con 0x7fd47c0b5f20 2024-04-26T21:38:15.633 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:15.633 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":62,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:38:14.928455+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd46006c8f0 msgr2=0x7fd46006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd46006c8f0 0x7fd46006edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd46c006fd0 tx=0x7fd46c008040 comp rx=0 tx=0).stop 2024-04-26T21:38:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c0b5f20 msgr2=0x7fd47c159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c0b5f20 0x7fd47c159390 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd478000c00 tx=0x7fd4780046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 shutdown_connections 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd46006c8f0 0x7fd46006edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd47c162880 0x7fd47c0ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd47c0d39f0 0x7fd47c0b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 --2- 172.21.15.40:0/2157974727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd47c0b5f20 0x7fd47c159390 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 >> 172.21.15.40:0/2157974727 conn(0x7fd47c0d4f60 msgr2=0x7fd47c0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 shutdown_connections 2024-04-26T21:38:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.633+0000 7fd4833af640 1 -- 172.21.15.40:0/2157974727 wait complete. 2024-04-26T21:38:15.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:38:15.646 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:38:15.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.749+0000 7efc8db71640 1 Processor -- start 2024-04-26T21:38:15.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.749+0000 7efc8db71640 1 -- start start 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc8db71640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc88070350 0x7efc88070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc8db71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 0x7efc8815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc8db71640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc880d2390 con 0x7efc880cfde0 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc8db71640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efc88071500 con 0x7efc88070350 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc8db71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efc880afab0 con 0x7efc88071090 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc877fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efc880cfde0 0x7efc880d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39962/0 (socket says 172.21.15.40:39962) 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc877fe640 1 -- 172.21.15.40:0/3528692239 learned_addr learned my addr 172.21.15.40:0/3528692239 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617874956 0 0) 0x7efc880d2390 con 0x7efc880cfde0 2024-04-26T21:38:15.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc87fff640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 0x7efc8815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc86ffd640 1 --2- 172.21.15.40:0/3528692239 >> v2:172.21.15.184:3300/0 conn(0x7efc88070350 0x7efc88070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.750+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc70003520 con 0x7efc880cfde0 2024-04-26T21:38:15.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 675859595 0 0) 0x7efc70003520 con 0x7efc880cfde0 2024-04-26T21:38:15.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 msgr2=0x7efc8815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:15.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 0x7efc8815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 >> v2:172.21.15.184:3300/0 conn(0x7efc88070350 msgr2=0x7efc88070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:15.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 --2- 172.21.15.40:0/3528692239 >> v2:172.21.15.184:3300/0 conn(0x7efc88070350 0x7efc88070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8806d3f0 con 0x7efc880cfde0 2024-04-26T21:38:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7efc78001670 con 0x7efc880cfde0 2024-04-26T21:38:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 >> v1:172.21.15.40:6789/0 conn(0x7efc880cfde0 legacy=0x7efc880d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 0x7efc700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc70003f30 0x7efc70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc70004d40 0x7efc700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efc88071500 con 0x7efc70003f30 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efc7000b9c0 con 0x7efc70004d40 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efc7000bb90 con 0x7efc88071090 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc86ffd640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc70003f30 0x7efc70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc877fe640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 0x7efc700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc87fff640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc70004d40 0x7efc700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc86ffd640 1 -- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 msgr2=0x7efc700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc86ffd640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 0x7efc700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc86ffd640 1 -- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc70004d40 msgr2=0x7efc700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc86ffd640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc70004d40 0x7efc700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc86ffd640 1 -- 172.21.15.40:0/3528692239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc70003520 con 0x7efc70003f30 2024-04-26T21:38:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc87fff640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc70004d40 0x7efc700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.751+0000 7efc877fe640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc88071090 0x7efc700039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc86ffd640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc70003f30 0x7efc70004800 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009af0 tx=0x7efc70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ef2c23e2b98aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c036070 con 0x7efc70003f30 2024-04-26T21:38:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c031400 con 0x7efc70003f30 2024-04-26T21:38:15.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc867fc640 1 -- 172.21.15.40:0/3528692239 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c009690 con 0x7efc70003f30 2024-04-26T21:38:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc8db71640 1 -- 172.21.15.40:0/3528692239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc70003f30 msgr2=0x7efc70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc8db71640 1 --2- 172.21.15.40:0/3528692239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc70003f30 0x7efc70004800 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009af0 tx=0x7efc70005830 comp rx=0 tx=0).stop 2024-04-26T21:38:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc87fff640 1 -- 172.21.15.40:0/3528692239 reap_dead start 2024-04-26T21:38:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc8db71640 1 -- 172.21.15.40:0/3528692239 shutdown_connections 2024-04-26T21:38:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc8db71640 1 -- 172.21.15.40:0/3528692239 >> 172.21.15.40:0/3528692239 conn(0x7efc880d2d80 msgr2=0x7efc880d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.752+0000 7efc8db71640 1 -- 172.21.15.40:0/3528692239 shutdown_connections 2024-04-26T21:38:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.753+0000 7efc8db71640 1 -- 172.21.15.40:0/3528692239 wait complete. 2024-04-26T21:38:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.754+0000 7efc8db71640 1 Processor -- start 2024-04-26T21:38:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.754+0000 7efc8db71640 1 -- start start 2024-04-26T21:38:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.754+0000 7efc8db71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc70003f30 0x7efc881541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc8db71640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc881f5f40 0x7efc880b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc8db71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc880b3f30 0x7efc881f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc8db71640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efc8806d9b0 con 0x7efc880b3f30 2024-04-26T21:38:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc8db71640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efc8806dd20 con 0x7efc881f5f40 2024-04-26T21:38:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc8db71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efc8815e9f0 con 0x7efc70003f30 2024-04-26T21:38:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc880b3f30 0x7efc881f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc880b3f30 0x7efc881f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56560/0 (socket says 172.21.15.40:56560) 2024-04-26T21:38:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 -- 172.21.15.40:0/1543445946 learned_addr learned my addr 172.21.15.40:0/1543445946 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc877fe640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc70003f30 0x7efc881541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc86ffd640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc881f5f40 0x7efc880b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 -- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc70003f30 msgr2=0x7efc881541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc70003f30 0x7efc881541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 -- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc881f5f40 msgr2=0x7efc880b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc881f5f40 0x7efc880b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 -- 172.21.15.40:0/1543445946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc88161ca0 con 0x7efc880b3f30 2024-04-26T21:38:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc877fe640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc70003f30 0x7efc881541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc87fff640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc880b3f30 0x7efc881f4d30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7efc8806c4e0 tx=0x7efc80010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc6ffff640 1 -- 172.21.15.40:0/1543445946 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80010d80 con 0x7efc880b3f30 2024-04-26T21:38:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc6ffff640 1 -- 172.21.15.40:0/1543445946 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc80003990 con 0x7efc880b3f30 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc881f5960 con 0x7efc880b3f30 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc6ffff640 1 -- 172.21.15.40:0/1543445946 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80003d80 con 0x7efc880b3f30 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.755+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8809cbc0 con 0x7efc880b3f30 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.756+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc880694f0 con 0x7efc880b3f30 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.757+0000 7efc6ffff640 1 -- 172.21.15.40:0/1543445946 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc80005d10 con 0x7efc880b3f30 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.757+0000 7efc6ffff640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc7406c920 0x7efc7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.757+0000 7efc6ffff640 1 -- 172.21.15.40:0/1543445946 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5249+0+0 (secure 0 0 0) 0x7efc8008bf90 con 0x7efc880b3f30 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.758+0000 7efc877fe640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc7406c920 0x7efc7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.758+0000 7efc877fe640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc7406c920 0x7efc7406ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efc78001050 tx=0x7efc78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.762+0000 7efc6ffff640 1 -- 172.21.15.40:0/1543445946 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc80025020 con 0x7efc880b3f30 2024-04-26T21:38:15.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.949+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc8815e410 con 0x7efc880b3f30 2024-04-26T21:38:15.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.950+0000 7efc6ffff640 1 -- 172.21.15.40:0/1543445946 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7efc8005a580 con 0x7efc880b3f30 2024-04-26T21:38:15.951 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:38:15.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.951+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc7406c920 msgr2=0x7efc7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.951+0000 7efc8db71640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc7406c920 0x7efc7406ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efc78001050 tx=0x7efc78009040 comp rx=0 tx=0).stop 2024-04-26T21:38:15.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.951+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc880b3f30 msgr2=0x7efc881f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:15.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.951+0000 7efc8db71640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc880b3f30 0x7efc881f4d30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7efc8806c4e0 tx=0x7efc80010590 comp rx=0 tx=0).stop 2024-04-26T21:38:15.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.952+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 shutdown_connections 2024-04-26T21:38:15.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.952+0000 7efc8db71640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc7406c920 0x7efc7406ede0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.952+0000 7efc8db71640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc880b3f30 0x7efc881f4d30 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.952+0000 7efc8db71640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc881f5f40 0x7efc880b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.952+0000 7efc8db71640 1 --2- 172.21.15.40:0/1543445946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc70003f30 0x7efc881541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:15.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.952+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 >> 172.21.15.40:0/1543445946 conn(0x7efc880d2d80 msgr2=0x7efc880b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:15.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.952+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 shutdown_connections 2024-04-26T21:38:15.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:15.952+0000 7efc8db71640 1 -- 172.21.15.40:0/1543445946 wait complete. 2024-04-26T21:38:15.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:38:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.064+0000 7fdff5351640 1 Processor -- start 2024-04-26T21:38:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.064+0000 7fdff5351640 1 -- start start 2024-04-26T21:38:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdff5351640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdff0162880 0x7fdff00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdff5351640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff00d3bc0 0x7fdff00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdff5351640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff00b7e50 con 0x7fdff00b5f20 2024-04-26T21:38:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdff5351640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdff00d0580 con 0x7fdff0162880 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdff5351640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdff00d1800 con 0x7fdff00d3bc0 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfe7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdff00b5f20 0x7fdff00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39970/0 (socket says 172.21.15.40:39970) 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfe7fff640 1 -- 172.21.15.40:0/1966732375 learned_addr learned my addr 172.21.15.40:0/1966732375 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfeeffd640 1 --2- 172.21.15.40:0/1966732375 >> v2:172.21.15.184:3300/0 conn(0x7fdff0162880 0x7fdff00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865325787 0 0) 0x7fdff00b7e50 con 0x7fdff00b5f20 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfef7fe640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff00d3bc0 0x7fdff00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfdc003520 con 0x7fdff00b5f20 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2058438638 0 0) 0x7fdfdc003520 con 0x7fdff00b5f20 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff00d3bc0 msgr2=0x7fdff00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfee7fc640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff00d3bc0 0x7fdff00b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.065+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 >> v2:172.21.15.184:3300/0 conn(0x7fdff0162880 msgr2=0x7fdff00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 --2- 172.21.15.40:0/1966732375 >> v2:172.21.15.184:3300/0 conn(0x7fdff0162880 0x7fdff00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff00d07d0 con 0x7fdff00b5f20 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fdfd8001670 con 0x7fdff00b5f20 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 >> v1:172.21.15.40:6789/0 conn(0x7fdff00b5f20 legacy=0x7fdff00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc0039b0 0x7fdfdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfdc004800 0x7fdfdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff00d3bc0 0x7fdfdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdff00d0580 con 0x7fdfdc004800 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdff00d1800 con 0x7fdfdc0039b0 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdfdc00b9c0 con 0x7fdff00d3bc0 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfe7fff640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfdc004800 0x7fdfdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfeeffd640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc0039b0 0x7fdfdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfef7fe640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff00d3bc0 0x7fdfdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfe7fff640 1 -- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff00d3bc0 msgr2=0x7fdfdc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfe7fff640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff00d3bc0 0x7fdfdc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfe7fff640 1 -- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc0039b0 msgr2=0x7fdfdc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfe7fff640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc0039b0 0x7fdfdc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.066+0000 7fdfe7fff640 1 -- 172.21.15.40:0/1966732375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc003520 con 0x7fdfdc004800 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.067+0000 7fdfe7fff640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfdc004800 0x7fdfdc004c80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001050 tx=0x7fdfdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeda4de1c0f8aed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.067+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd802e070 con 0x7fdfdc004800 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.067+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd8005af0 con 0x7fdfdc004800 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.067+0000 7fdfee7fc640 1 -- 172.21.15.40:0/1966732375 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8007d70 con 0x7fdfdc004800 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.067+0000 7fdff5351640 1 -- 172.21.15.40:0/1966732375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfdc004800 msgr2=0x7fdfdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.067+0000 7fdff5351640 1 --2- 172.21.15.40:0/1966732375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfdc004800 0x7fdfdc004c80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001050 tx=0x7fdfdc005830 comp rx=0 tx=0).stop 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.067+0000 7fdfef7fe640 1 -- 172.21.15.40:0/1966732375 reap_dead start 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.068+0000 7fdff5351640 1 -- 172.21.15.40:0/1966732375 shutdown_connections 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.068+0000 7fdff5351640 1 -- 172.21.15.40:0/1966732375 >> 172.21.15.40:0/1966732375 conn(0x7fdff00d4f60 msgr2=0x7fdff00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:16.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.068+0000 7fdff5351640 1 -- 172.21.15.40:0/1966732375 shutdown_connections 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.068+0000 7fdff5351640 1 -- 172.21.15.40:0/1966732375 wait complete. 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.069+0000 7fdff5351640 1 Processor -- start 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.069+0000 7fdff5351640 1 -- start start 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.069+0000 7fdff5351640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc004800 0x7fdff00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.069+0000 7fdff5351640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff01f52d0 0x7fdff00cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.069+0000 7fdff5351640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff015ce00 0x7fdff00ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.069+0000 7fdff5351640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0162c60 con 0x7fdff01f52d0 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.069+0000 7fdff5351640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdff00d0d90 con 0x7fdfdc004800 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.069+0000 7fdff5351640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdff0162de0 con 0x7fdff015ce00 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff01f52d0 0x7fdff00cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff01f52d0 0x7fdff00cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56584/0 (socket says 172.21.15.40:56584) 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 -- 172.21.15.40:0/702709680 learned_addr learned my addr 172.21.15.40:0/702709680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfeeffd640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc004800 0x7fdff00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfef7fe640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff015ce00 0x7fdff00ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 -- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff015ce00 msgr2=0x7fdff00ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff015ce00 0x7fdff00ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 -- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc004800 msgr2=0x7fdff00b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc004800 0x7fdff00b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 -- 172.21.15.40:0/702709680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff01f5730 con 0x7fdff01f52d0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe7fff640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff01f52d0 0x7fdff00cc800 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8034040 tx=0x7fdfd8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe77fe640 1 -- 172.21.15.40:0/702709680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8042070 con 0x7fdff01f52d0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe77fe640 1 -- 172.21.15.40:0/702709680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd8007480 con 0x7fdff01f52d0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff015ecc0 con 0x7fdff01f52d0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdfe77fe640 1 -- 172.21.15.40:0/702709680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8007d50 con 0x7fdff01f52d0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.070+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff015f280 con 0x7fdff01f52d0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.071+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff00694f0 con 0x7fdff01f52d0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.072+0000 7fdfe77fe640 1 -- 172.21.15.40:0/702709680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfd8045070 con 0x7fdff01f52d0 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.072+0000 7fdfe77fe640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfd006c920 0x7fdfd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.072+0000 7fdfe77fe640 1 -- 172.21.15.40:0/702709680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5249+0+0 (secure 0 0 0) 0x7fdfd80ac380 con 0x7fdff01f52d0 2024-04-26T21:38:16.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.072+0000 7fdfeeffd640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfd006c920 0x7fdfd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.073+0000 7fdfeeffd640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfd006c920 0x7fdfd006ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009bc0 tx=0x7fdfe0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:16.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.076+0000 7fdfe77fe640 1 -- 172.21.15.40:0/702709680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdfd8005ac0 con 0x7fdff01f52d0 2024-04-26T21:38:16.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.263+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdff0074590 con 0x7fdff01f52d0 2024-04-26T21:38:16.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.264+0000 7fdfe77fe640 1 -- 172.21.15.40:0/702709680 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) ==== 84+0+65 (secure 0 0 0) 0x7fdfd807a440 con 0x7fdff01f52d0 2024-04-26T21:38:16.265 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:16.265 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:38:16.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:38:16.265 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:38:16.265 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:38:16.265 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:38:16.266 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfd006c920 msgr2=0x7fdfd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfd006c920 0x7fdfd006ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009bc0 tx=0x7fdfe0009190 comp rx=0 tx=0).stop 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff01f52d0 msgr2=0x7fdff00cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff01f52d0 0x7fdff00cc800 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8034040 tx=0x7fdfd8000fe0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 shutdown_connections 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfd006c920 0x7fdfd006ede0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdff015ce00 0x7fdff00ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff01f52d0 0x7fdff00cc800 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 --2- 172.21.15.40:0/702709680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfdc004800 0x7fdff00b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 >> 172.21.15.40:0/702709680 conn(0x7fdff00d4f60 msgr2=0x7fdff015e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 shutdown_connections 2024-04-26T21:38:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.266+0000 7fdff5351640 1 -- 172.21.15.40:0/702709680 wait complete. 2024-04-26T21:38:16.277 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:38:16.278 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:38:16.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.383+0000 7f0fede46640 1 Processor -- start 2024-04-26T21:38:16.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.383+0000 7f0fede46640 1 -- start start 2024-04-26T21:38:16.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fede46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe815f460 0x7f0fe80d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fede46640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0fe80d0760 0x7f0fe80d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fede46640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0fe80d1a40 0x7f0fe80d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fede46640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe80d41c0 con 0x7f0fe80d0760 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fede46640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe80d37f0 con 0x7f0fe80d1a40 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fede46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe815f840 con 0x7f0fe815f460 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0fe80d0760 0x7f0fe80d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0fe80d0760 0x7f0fe80d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56596/0 (socket says 172.21.15.40:56596) 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe6ffd640 1 -- 172.21.15.40:0/156264417 learned_addr learned my addr 172.21.15.40:0/156264417 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe7fff640 1 --2- 172.21.15.40:0/156264417 >> v2:172.21.15.184:3300/0 conn(0x7f0fe80d1a40 0x7f0fe80d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe77fe640 1 --2- 172.21.15.40:0/156264417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe815f460 0x7f0fe80d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe6ffd640 1 -- 172.21.15.40:0/156264417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe815f460 msgr2=0x7f0fe80d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe6ffd640 1 --2- 172.21.15.40:0/156264417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe815f460 0x7f0fe80d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe6ffd640 1 -- 172.21.15.40:0/156264417 >> v2:172.21.15.184:3300/0 conn(0x7f0fe80d1a40 msgr2=0x7f0fe80d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe6ffd640 1 --2- 172.21.15.40:0/156264417 >> v2:172.21.15.184:3300/0 conn(0x7f0fe80d1a40 0x7f0fe80d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.384+0000 7f0fe6ffd640 1 -- 172.21.15.40:0/156264417 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe80d2ec0 con 0x7f0fe80d0760 2024-04-26T21:38:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.385+0000 7f0fe6ffd640 1 --2- 172.21.15.40:0/156264417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe80d0760 0x7f0fe80d0bc0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80c8b60 tx=0x7f0fdc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d430cf8f5055d820 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.385+0000 7f0fe67fc640 1 -- 172.21.15.40:0/156264417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc037070 con 0x7f0fe80d0760 2024-04-26T21:38:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.385+0000 7f0fe67fc640 1 -- 172.21.15.40:0/156264417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc032440 con 0x7f0fe80d0760 2024-04-26T21:38:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.385+0000 7f0fe67fc640 1 -- 172.21.15.40:0/156264417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc032810 con 0x7f0fe80d0760 2024-04-26T21:38:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 -- 172.21.15.40:0/156264417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe80d0760 msgr2=0x7f0fe80d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 --2- 172.21.15.40:0/156264417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe80d0760 0x7f0fe80d0bc0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80c8b60 tx=0x7f0fdc02a140 comp rx=0 tx=0).stop 2024-04-26T21:38:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 -- 172.21.15.40:0/156264417 shutdown_connections 2024-04-26T21:38:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 --2- 172.21.15.40:0/156264417 >> v2:172.21.15.184:3300/0 conn(0x7f0fe80d1a40 0x7f0fe80d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 --2- 172.21.15.40:0/156264417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe80d0760 0x7f0fe80d0bc0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 --2- 172.21.15.40:0/156264417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe815f460 0x7f0fe80d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 -- 172.21.15.40:0/156264417 >> 172.21.15.40:0/156264417 conn(0x7f0fe809e3f0 msgr2=0x7f0fe80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 -- 172.21.15.40:0/156264417 shutdown_connections 2024-04-26T21:38:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.386+0000 7f0fede46640 1 -- 172.21.15.40:0/156264417 wait complete. 2024-04-26T21:38:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.387+0000 7f0fede46640 1 Processor -- start 2024-04-26T21:38:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.387+0000 7f0fede46640 1 -- start start 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fede46640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fe80d0760 0x7f0fe80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fede46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe80d1a40 0x7f0fe80cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fede46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe815f460 0x7f0fe815c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fede46640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80d5f30 con 0x7f0fe815f460 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fede46640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80d3480 con 0x7f0fe80d0760 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fede46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe806ce20 con 0x7f0fe80d1a40 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe815f460 0x7f0fe815c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe815f460 0x7f0fe815c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56600/0 (socket says 172.21.15.40:56600) 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 -- 172.21.15.40:0/219781813 learned_addr learned my addr 172.21.15.40:0/219781813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 -- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe80d1a40 msgr2=0x7f0fe80cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe6ffd640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe80d1a40 0x7f0fe80cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe77fe640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fe80d0760 0x7f0fe80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe80d1a40 0x7f0fe80cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 -- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fe80d0760 msgr2=0x7f0fe80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fe80d0760 0x7f0fe80b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 -- 172.21.15.40:0/219781813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe81f56e0 con 0x7f0fe815f460 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fe7fff640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe815f460 0x7f0fe815c3d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0007e60 tx=0x7f0fe000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fcffff640 1 -- 172.21.15.40:0/219781813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0014620 con 0x7f0fe815f460 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fcffff640 1 -- 172.21.15.40:0/219781813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe0019070 con 0x7f0fe815f460 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fcffff640 1 -- 172.21.15.40:0/219781813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0014a20 con 0x7f0fe815f460 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.388+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe81f5c70 con 0x7f0fe815f460 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.389+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe81f6230 con 0x7f0fe815f460 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.389+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe8069600 con 0x7f0fe815f460 2024-04-26T21:38:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.390+0000 7f0fcffff640 1 -- 172.21.15.40:0/219781813 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fe0014bc0 con 0x7f0fe815f460 2024-04-26T21:38:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.390+0000 7f0fcffff640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fd406c8f0 0x7f0fd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.390+0000 7f0fcffff640 1 -- 172.21.15.40:0/219781813 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5249+0+0 (secure 0 0 0) 0x7f0fe008c150 con 0x7f0fe815f460 2024-04-26T21:38:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.391+0000 7f0fe77fe640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fd406c8f0 0x7f0fd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.391+0000 7f0fe77fe640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fd406c8f0 0x7f0fd406edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8006fd0 tx=0x7f0fd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.395+0000 7f0fcffff640 1 -- 172.21.15.40:0/219781813 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0fe005a7d0 con 0x7f0fe815f460 2024-04-26T21:38:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.579+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0fe80d29a0 con 0x7f0fe815f460 2024-04-26T21:38:16.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.919+0000 7f0fcffff640 1 -- 172.21.15.40:0/219781813 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) ==== 138+0+0 (secure 0 0 0) 0x7f0fe005a0d0 con 0x7f0fe815f460 2024-04-26T21:38:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fd406c8f0 msgr2=0x7f0fd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fd406c8f0 0x7f0fd406edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8006fd0 tx=0x7f0fd8008040 comp rx=0 tx=0).stop 2024-04-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe815f460 msgr2=0x7f0fe815c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe815f460 0x7f0fe815c3d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0007e60 tx=0x7f0fe000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 shutdown_connections 2024-04-26T21:38:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fd406c8f0 0x7f0fd406edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fe815f460 0x7f0fe815c3d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fe80d1a40 0x7f0fe80cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 --2- 172.21.15.40:0/219781813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fe80d0760 0x7f0fe80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 >> 172.21.15.40:0/219781813 conn(0x7f0fe809e3f0 msgr2=0x7f0fe80b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:16.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 shutdown_connections 2024-04-26T21:38:16.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:16.922+0000 7f0fede46640 1 -- 172.21.15.40:0/219781813 wait complete. 2024-04-26T21:38:16.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:38:17.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.048+0000 7f0704ada640 1 Processor -- start 2024-04-26T21:38:17.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.048+0000 7f0704ada640 1 -- start start 2024-04-26T21:38:17.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.049+0000 7f0704ada640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07000b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:17.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.049+0000 7f0704ada640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07000b4d40 0x7f07000b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.049+0000 7f0704ada640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07000b1490 con 0x7f07000b6c50 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.049+0000 7f0704ada640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0700073bc0 con 0x7f07000b4d40 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.049+0000 7f0704ada640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0700072fb0 con 0x7f07000b0b70 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.049+0000 7f06fdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07000b4d40 0x7f07000b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.049+0000 7f06fdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07000b4d40 0x7f07000b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56612/0 (socket says 172.21.15.40:56612) 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.049+0000 7f06fdd9b640 1 -- 172.21.15.40:0/3540644380 learned_addr learned my addr 172.21.15.40:0/3540644380 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.050+0000 7f06fdd9b640 1 -- 172.21.15.40:0/3540644380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 msgr2=0x7f07000b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.050+0000 7f06fe59c640 1 --2- 172.21.15.40:0/3540644380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07000b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.050+0000 7f06fdd9b640 1 --2- 172.21.15.40:0/3540644380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07000b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.050+0000 7f06fdd9b640 1 -- 172.21.15.40:0/3540644380 >> v1:172.21.15.40:6789/0 conn(0x7f07000b6c50 legacy=0x7f070015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.050+0000 7f06fdd9b640 1 -- 172.21.15.40:0/3540644380 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0700073290 con 0x7f07000b4d40 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.050+0000 7f06fe59c640 1 --2- 172.21.15.40:0/3540644380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07000b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.050+0000 7f06fdd9b640 1 --2- 172.21.15.40:0/3540644380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b4d40 0x7f07000b6710 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009870 tx=0x7f06e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd833698ecf2d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:17.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f06fd59a640 1 -- 172.21.15.40:0/3540644380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8037070 con 0x7f07000b4d40 2024-04-26T21:38:17.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f06fd59a640 1 -- 172.21.15.40:0/3540644380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e8032440 con 0x7f07000b4d40 2024-04-26T21:38:17.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f06fd59a640 1 -- 172.21.15.40:0/3540644380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8032810 con 0x7f07000b4d40 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f0704ada640 1 -- 172.21.15.40:0/3540644380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b4d40 msgr2=0x7f07000b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f0704ada640 1 --2- 172.21.15.40:0/3540644380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b4d40 0x7f07000b6710 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009870 tx=0x7f06e802a270 comp rx=0 tx=0).stop 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f0704ada640 1 -- 172.21.15.40:0/3540644380 shutdown_connections 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f0704ada640 1 --2- 172.21.15.40:0/3540644380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b4d40 0x7f07000b6710 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f0704ada640 1 --2- 172.21.15.40:0/3540644380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07000b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f0704ada640 1 -- 172.21.15.40:0/3540644380 >> 172.21.15.40:0/3540644380 conn(0x7f07000b39b0 msgr2=0x7f07000b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f0704ada640 1 -- 172.21.15.40:0/3540644380 shutdown_connections 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.051+0000 7f0704ada640 1 -- 172.21.15.40:0/3540644380 wait complete. 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f0704ada640 1 Processor -- start 2024-04-26T21:38:17.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f0704ada640 1 -- start start 2024-04-26T21:38:17.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f0704ada640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07001493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:17.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f0704ada640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07000b4d40 0x7f070014c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:17.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f0704ada640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b6c50 0x7f070015daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:17.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f0704ada640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f07000afda0 con 0x7f07000b6c50 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f0704ada640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0700073850 con 0x7f07000b4d40 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f0704ada640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f07000aff20 con 0x7f07000b0b70 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.053+0000 7f06fed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b6c50 0x7f070015daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b6c50 0x7f070015daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56622/0 (socket says 172.21.15.40:56622) 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fed9d640 1 -- 172.21.15.40:0/2087097354 learned_addr learned my addr 172.21.15.40:0/2087097354 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fe59c640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07001493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fdd9b640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07000b4d40 0x7f070014c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fed9d640 1 -- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 msgr2=0x7f07001493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fed9d640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07001493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fed9d640 1 -- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07000b4d40 msgr2=0x7f070014c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fed9d640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07000b4d40 0x7f070014c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fed9d640 1 -- 172.21.15.40:0/2087097354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070014cac0 con 0x7f07000b6c50 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fdd9b640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07000b4d40 0x7f070014c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fe59c640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07001493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:17.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06fed9d640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b6c50 0x7f070015daa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c7a0 tx=0x7f06f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:17.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06f6ffd640 1 -- 172.21.15.40:0/2087097354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0003b70 con 0x7f07000b6c50 2024-04-26T21:38:17.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06f6ffd640 1 -- 172.21.15.40:0/2087097354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f0003d30 con 0x7f07000b6c50 2024-04-26T21:38:17.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f06f6ffd640 1 -- 172.21.15.40:0/2087097354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f000aa90 con 0x7f07000b6c50 2024-04-26T21:38:17.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07000d1130 con 0x7f07000b6c50 2024-04-26T21:38:17.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.054+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07000d16f0 con 0x7f07000b6c50 2024-04-26T21:38:17.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.055+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07000694f0 con 0x7f07000b6c50 2024-04-26T21:38:17.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.056+0000 7f06f6ffd640 1 -- 172.21.15.40:0/2087097354 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06f000ac30 con 0x7f07000b6c50 2024-04-26T21:38:17.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.057+0000 7f06f6ffd640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f06e006c8f0 0x7f06e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:17.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.057+0000 7f06f6ffd640 1 -- 172.21.15.40:0/2087097354 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5619+0+0 (secure 0 0 0) 0x7f06f008bb40 con 0x7f07000b6c50 2024-04-26T21:38:17.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.057+0000 7f06fe59c640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f06e006c8f0 0x7f06e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:17.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.057+0000 7f06fe59c640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f06e006c8f0 0x7f06e006edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f06f800a990 tx=0x7f06f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:17.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.061+0000 7f06f6ffd640 1 -- 172.21.15.40:0/2087097354 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06f0059f20 con 0x7f07000b6c50 2024-04-26T21:38:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.249+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f070006dfb0 con 0x7f07000b6c50 2024-04-26T21:38:17.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.927+0000 7f06f6ffd640 1 -- 172.21.15.40:0/2087097354 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) ==== 170+0+0 (secure 0 0 0) 0x7f06f0059820 con 0x7f07000b6c50 2024-04-26T21:38:17.928 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:38:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.930+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f06e006c8f0 msgr2=0x7f06e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.930+0000 7f0704ada640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f06e006c8f0 0x7f06e006edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f06f800a990 tx=0x7f06f8008040 comp rx=0 tx=0).stop 2024-04-26T21:38:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.930+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b6c50 msgr2=0x7f070015daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.930+0000 7f0704ada640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b6c50 0x7f070015daa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c7a0 tx=0x7f06f000cc60 comp rx=0 tx=0).stop 2024-04-26T21:38:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.931+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 shutdown_connections 2024-04-26T21:38:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.931+0000 7f0704ada640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f06e006c8f0 0x7f06e006edb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.931+0000 7f0704ada640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000b6c50 0x7f070015daa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.931+0000 7f0704ada640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07000b4d40 0x7f070014c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.931+0000 7f0704ada640 1 --2- 172.21.15.40:0/2087097354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07000b0b70 0x7f07001493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:17.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.931+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 >> 172.21.15.40:0/2087097354 conn(0x7f07000b39b0 msgr2=0x7f07000708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.933+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 shutdown_connections 2024-04-26T21:38:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:17.933+0000 7f0704ada640 1 -- 172.21.15.40:0/2087097354 wait complete. 2024-04-26T21:38:17.958 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:38:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.067+0000 7f34988e1640 1 Processor -- start 2024-04-26T21:38:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.068+0000 7f34988e1640 1 -- start start 2024-04-26T21:38:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.068+0000 7f34988e1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f34940d1a00 0x7f34940d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.068+0000 7f34988e1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940d2320 0x7f34941625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.068+0000 7f34988e1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34940d0a10 con 0x7f34940b40d0 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.068+0000 7f34988e1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f34940d0cf0 con 0x7f34940d1a00 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.068+0000 7f34988e1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34940d0fd0 con 0x7f34940d2320 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f3492d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f34940b40d0 0x7f34940d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39986/0 (socket says 172.21.15.40:39986) 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f3492d9d640 1 -- 172.21.15.40:0/1501903747 learned_addr learned my addr 172.21.15.40:0/1501903747 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f349259c640 1 --2- 172.21.15.40:0/1501903747 >> v2:172.21.15.184:3300/0 conn(0x7f34940d1a00 0x7f34940d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f3491d9b640 1 --2- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940d2320 0x7f34941625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f349159a640 1 -- 172.21.15.40:0/1501903747 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193823271 0 0) 0x7f34940d0a10 con 0x7f34940b40d0 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f349159a640 1 -- 172.21.15.40:0/1501903747 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3480003520 con 0x7f34940b40d0 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f349159a640 1 -- 172.21.15.40:0/1501903747 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3492962074 0 0) 0x7f3480003520 con 0x7f34940b40d0 2024-04-26T21:38:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f349259c640 1 -- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940d2320 msgr2=0x7f34941625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f349259c640 1 --2- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940d2320 0x7f34941625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f349259c640 1 -- 172.21.15.40:0/1501903747 >> v1:172.21.15.40:6789/0 conn(0x7f34940b40d0 legacy=0x7f34940d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f349259c640 1 -- 172.21.15.40:0/1501903747 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34940d40a0 con 0x7f34940d1a00 2024-04-26T21:38:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.069+0000 7f3491d9b640 1 --2- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940d2320 0x7f34941625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:18.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.070+0000 7f349259c640 1 --2- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 0x7f34940d1de0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f348c0097a0 tx=0x7f348c02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1589256de7cc738a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:18.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.070+0000 7f349159a640 1 -- 172.21.15.40:0/1501903747 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c037070 con 0x7f34940d1a00 2024-04-26T21:38:18.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.070+0000 7f349159a640 1 -- 172.21.15.40:0/1501903747 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348c02ab10 con 0x7f34940d1a00 2024-04-26T21:38:18.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.070+0000 7f349159a640 1 -- 172.21.15.40:0/1501903747 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34840019f0 con 0x7f34940d1a00 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.071+0000 7f34988e1640 1 -- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 msgr2=0x7f34940d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.071+0000 7f34988e1640 1 --2- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 0x7f34940d1de0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f348c0097a0 tx=0x7f348c02a080 comp rx=0 tx=0).stop 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.071+0000 7f34988e1640 1 -- 172.21.15.40:0/1501903747 shutdown_connections 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.071+0000 7f34988e1640 1 --2- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940d2320 0x7f34941625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.071+0000 7f34988e1640 1 --2- 172.21.15.40:0/1501903747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 0x7f34940d1de0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.071+0000 7f34988e1640 1 -- 172.21.15.40:0/1501903747 >> 172.21.15.40:0/1501903747 conn(0x7f34940d4fd0 msgr2=0x7f34940d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.071+0000 7f34988e1640 1 -- 172.21.15.40:0/1501903747 shutdown_connections 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.071+0000 7f34988e1640 1 -- 172.21.15.40:0/1501903747 wait complete. 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.072+0000 7f34988e1640 1 Processor -- start 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.072+0000 7f34988e1640 1 -- start start 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f34988e1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940b40d0 0x7f34940cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f34988e1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 0x7f34940ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f34988e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34940d2320 0x7f34940ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f34988e1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3494162b30 con 0x7f34940d2320 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f34988e1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34940b5d20 con 0x7f34940d1a00 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f34988e1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3494162cb0 con 0x7f34940b40d0 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3492d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34940d2320 0x7f34940ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3492d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34940d2320 0x7f34940ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56638/0 (socket says 172.21.15.40:56638) 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3492d9d640 1 -- 172.21.15.40:0/2414267127 learned_addr learned my addr 172.21.15.40:0/2414267127 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f349259c640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940b40d0 0x7f34940cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3492d9d640 1 -- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940b40d0 msgr2=0x7f34940cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3492d9d640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940b40d0 0x7f34940cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3492d9d640 1 -- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 msgr2=0x7f34940ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3491d9b640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 0x7f34940ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3492d9d640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 0x7f34940ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.073+0000 7f3492d9d640 1 -- 172.21.15.40:0/2414267127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349415e4f0 con 0x7f34940d2320 2024-04-26T21:38:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.074+0000 7f3492d9d640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34940d2320 0x7f34940ca250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3484001b60 tx=0x7f348400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.074+0000 7f348affd640 1 -- 172.21.15.40:0/2414267127 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484017070 con 0x7f34940d2320 2024-04-26T21:38:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.074+0000 7f348affd640 1 -- 172.21.15.40:0/2414267127 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3484012400 con 0x7f34940d2320 2024-04-26T21:38:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.074+0000 7f348affd640 1 -- 172.21.15.40:0/2414267127 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34840127b0 con 0x7f34940d2320 2024-04-26T21:38:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.074+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34940c8940 con 0x7f34940d2320 2024-04-26T21:38:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.074+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34940b7bf0 con 0x7f34940d2320 2024-04-26T21:38:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.075+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34940b7df0 con 0x7f34940d2320 2024-04-26T21:38:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.075+0000 7f348affd640 1 -- 172.21.15.40:0/2414267127 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f348401a070 con 0x7f34940d2320 2024-04-26T21:38:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.076+0000 7f348affd640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347406c8f0 0x7f347406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.076+0000 7f348affd640 1 -- 172.21.15.40:0/2414267127 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6001+0+0 (secure 0 0 0) 0x7f348408bad0 con 0x7f34940d2320 2024-04-26T21:38:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.076+0000 7f349259c640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347406c8f0 0x7f347406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.077+0000 7f349259c640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347406c8f0 0x7f347406edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f348c034000 tx=0x7f348c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:18.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.080+0000 7f348affd640 1 -- 172.21.15.40:0/2414267127 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34940b7df0 con 0x7f34940d2320 2024-04-26T21:38:18.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.308+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f34940b6730 con 0x7f34940d2320 2024-04-26T21:38:18.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.935+0000 7f348affd640 1 -- 172.21.15.40:0/2414267127 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v45) ==== 173+0+0 (secure 0 0 0) 0x7f3484059810 con 0x7f34940d2320 2024-04-26T21:38:18.936 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 9 and data pool 10 2024-04-26T21:38:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.952+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347406c8f0 msgr2=0x7f347406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.952+0000 7f34988e1640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347406c8f0 0x7f347406edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f348c034000 tx=0x7f348c02e000 comp rx=0 tx=0).stop 2024-04-26T21:38:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.952+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34940d2320 msgr2=0x7f34940ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.952+0000 7f34988e1640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34940d2320 0x7f34940ca250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3484001b60 tx=0x7f348400a430 comp rx=0 tx=0).stop 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.953+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 shutdown_connections 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.953+0000 7f34988e1640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347406c8f0 0x7f347406edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.953+0000 7f34988e1640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34940d2320 0x7f34940ca250 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.953+0000 7f34988e1640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34940d1a00 0x7f34940ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.953+0000 7f34988e1640 1 --2- 172.21.15.40:0/2414267127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34940b40d0 0x7f34940cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.953+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 >> 172.21.15.40:0/2414267127 conn(0x7f34940d4fd0 msgr2=0x7f34940d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.953+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 shutdown_connections 2024-04-26T21:38:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:18.953+0000 7f34988e1640 1 -- 172.21.15.40:0/2414267127 wait complete. 2024-04-26T21:38:18.973 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.078+0000 7fc4bf1c6640 1 Processor -- start 2024-04-26T21:38:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.078+0000 7fc4bf1c6640 1 -- start start 2024-04-26T21:38:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.078+0000 7fc4bf1c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f3a0 0x7fc4b80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.078+0000 7fc4bf1c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4b80d1770 0x7fc4b80d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.078+0000 7fc4bf1c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 0x7fc4b80d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.078+0000 7fc4bf1c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b80d25d0 con 0x7fc4b80d1770 2024-04-26T21:38:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.078+0000 7fc4bf1c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b80d0e20 con 0x7fc4b815f3a0 2024-04-26T21:38:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.078+0000 7fc4bf1c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b80b1d70 con 0x7fc4b80d2160 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4b7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4b80d1770 0x7fc4b80d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4b7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4b80d1770 0x7fc4b80d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46726/0 (socket says 172.21.15.40:46726) 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4b7fff640 1 -- 172.21.15.40:0/2014337365 learned_addr learned my addr 172.21.15.40:0/2014337365 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4bd763640 1 --2- 172.21.15.40:0/2014337365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 0x7fc4b80d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4bcf62640 1 --2- 172.21.15.40:0/2014337365 >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f3a0 0x7fc4b80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4b7fff640 1 -- 172.21.15.40:0/2014337365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 msgr2=0x7fc4b80d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4b7fff640 1 --2- 172.21.15.40:0/2014337365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 0x7fc4b80d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4b7fff640 1 -- 172.21.15.40:0/2014337365 >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f3a0 msgr2=0x7fc4b80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4b7fff640 1 --2- 172.21.15.40:0/2014337365 >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f3a0 0x7fc4b80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.079+0000 7fc4b7fff640 1 -- 172.21.15.40:0/2014337365 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b80b2050 con 0x7fc4b80d1770 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4b7fff640 1 --2- 172.21.15.40:0/2014337365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b80d1c20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8073b50 tx=0x7fc4a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715bca813d2a8a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4b77fe640 1 -- 172.21.15.40:0/2014337365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4037070 con 0x7fc4b80d1770 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4b77fe640 1 -- 172.21.15.40:0/2014337365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a4032440 con 0x7fc4b80d1770 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4b77fe640 1 -- 172.21.15.40:0/2014337365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4032810 con 0x7fc4b80d1770 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/2014337365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 msgr2=0x7fc4b80d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/2014337365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b80d1c20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8073b50 tx=0x7fc4a402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/2014337365 shutdown_connections 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/2014337365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 0x7fc4b80d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/2014337365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b80d1c20 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/2014337365 >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f3a0 0x7fc4b80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/2014337365 >> 172.21.15.40:0/2014337365 conn(0x7fc4b80d4f90 msgr2=0x7fc4b80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/2014337365 shutdown_connections 2024-04-26T21:38:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.080+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/2014337365 wait complete. 2024-04-26T21:38:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.082+0000 7fc4bf1c6640 1 Processor -- start 2024-04-26T21:38:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.082+0000 7fc4bf1c6640 1 -- start start 2024-04-26T21:38:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bf1c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b814cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bf1c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 0x7fc4b80cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bf1c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f3a0 0x7fc4b80cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bf1c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b815f900 con 0x7fc4b80d1770 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bf1c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b815f780 con 0x7fc4b815f3a0 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bf1c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b80d3390 con 0x7fc4b80d2160 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b814cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b814cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46734/0 (socket says 172.21.15.40:46734) 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 -- 172.21.15.40:0/384632801 learned_addr learned my addr 172.21.15.40:0/384632801 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4b7fff640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 0x7fc4b80cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bd763640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f3a0 0x7fc4b80cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 -- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 msgr2=0x7fc4b80cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 0x7fc4b80cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 -- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f3a0 msgr2=0x7fc4b80cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f3a0 0x7fc4b80cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 -- 172.21.15.40:0/384632801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b81f9450 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bcf62640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b814cff0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00d5f0 tx=0x7fc4ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4b57fa640 1 -- 172.21.15.40:0/384632801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac0125f0 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4b57fa640 1 -- 172.21.15.40:0/384632801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ac019070 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4b57fa640 1 -- 172.21.15.40:0/384632801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac0129d0 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.083+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b80ca5f0 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.084+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b80cabb0 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.084+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b80694f0 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.085+0000 7fc4b57fa640 1 -- 172.21.15.40:0/384632801 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4ac012b70 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.085+0000 7fc4b57fa640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc49c06c8f0 0x7fc49c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.085+0000 7fc4b57fa640 1 -- 172.21.15.40:0/384632801 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fc4ac08bfa0 con 0x7fc4b80d1770 2024-04-26T21:38:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.085+0000 7fc4b7fff640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc49c06c8f0 0x7fc49c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.086+0000 7fc4b7fff640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc49c06c8f0 0x7fc49c06edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4034000 tx=0x7fc4a402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.090+0000 7fc4b57fa640 1 -- 172.21.15.40:0/384632801 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4ac059be0 con 0x7fc4b80d1770 2024-04-26T21:38:19.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.275+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4b80c85c0 con 0x7fc4b80d1770 2024-04-26T21:38:19.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.276+0000 7fc4b57fa640 1 -- 172.21.15.40:0/384632801 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16861 (secure 0 0 0) 0x7fc4ac05dee0 con 0x7fc4b80d1770 2024-04-26T21:38:19.278 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:19.278 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":65,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:38:18.310396+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc49c06c8f0 msgr2=0x7fc49c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc49c06c8f0 0x7fc49c06edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4034000 tx=0x7fc4a402e000 comp rx=0 tx=0).stop 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 msgr2=0x7fc4b814cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b814cff0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00d5f0 tx=0x7fc4ac00dab0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 shutdown_connections 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc49c06c8f0 0x7fc49c06edb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f3a0 0x7fc4b80cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d2160 0x7fc4b80cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 --2- 172.21.15.40:0/384632801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b80d1770 0x7fc4b814cff0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 >> 172.21.15.40:0/384632801 conn(0x7fc4b80d4f90 msgr2=0x7fc4b80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 shutdown_connections 2024-04-26T21:38:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.278+0000 7fc4bf1c6640 1 -- 172.21.15.40:0/384632801 wait complete. 2024-04-26T21:38:19.291 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:19.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.390+0000 7f733f914640 1 Processor -- start 2024-04-26T21:38:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.391+0000 7f733f914640 1 -- start start 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.391+0000 7f733f914640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.391+0000 7f733f914640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73380d1770 0x7f73380d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.391+0000 7f733f914640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73380d2130 0x7f73380d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.391+0000 7f733f914640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f73380d25a0 con 0x7f73380d2130 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.391+0000 7f733f914640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f73380d0e20 con 0x7f73380d1770 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.391+0000 7f733f914640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73380b1d70 con 0x7f733815f3a0 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.391+0000 7f733deb1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73380d2130 0x7f73380d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733deb1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73380d2130 0x7f73380d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46740/0 (socket says 172.21.15.40:46740) 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733deb1640 1 -- 172.21.15.40:0/3224884209 learned_addr learned my addr 172.21.15.40:0/3224884209 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733ceaf640 1 --2- 172.21.15.40:0/3224884209 >> v2:172.21.15.184:3300/0 conn(0x7f73380d1770 0x7f73380d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733d6b0640 1 --2- 172.21.15.40:0/3224884209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733deb1640 1 -- 172.21.15.40:0/3224884209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 msgr2=0x7f73380d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733deb1640 1 --2- 172.21.15.40:0/3224884209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733deb1640 1 -- 172.21.15.40:0/3224884209 >> v2:172.21.15.184:3300/0 conn(0x7f73380d1770 msgr2=0x7f73380d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733deb1640 1 --2- 172.21.15.40:0/3224884209 >> v2:172.21.15.184:3300/0 conn(0x7f73380d1770 0x7f73380d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733deb1640 1 -- 172.21.15.40:0/3224884209 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73380b2050 con 0x7f73380d2130 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733d6b0640 1 --2- 172.21.15.40:0/3224884209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.392+0000 7f733deb1640 1 --2- 172.21.15.40:0/3224884209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d2130 0x7f73380d2e50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7338073c50 tx=0x7f732802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6620ce404f3c1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f7337fff640 1 -- 172.21.15.40:0/3224884209 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328036070 con 0x7f73380d2130 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f7337fff640 1 -- 172.21.15.40:0/3224884209 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7328031600 con 0x7f73380d2130 2024-04-26T21:38:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f7337fff640 1 -- 172.21.15.40:0/3224884209 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73280319d0 con 0x7f73380d2130 2024-04-26T21:38:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 -- 172.21.15.40:0/3224884209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d2130 msgr2=0x7f73380d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 --2- 172.21.15.40:0/3224884209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d2130 0x7f73380d2e50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7338073c50 tx=0x7f732802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 -- 172.21.15.40:0/3224884209 shutdown_connections 2024-04-26T21:38:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 --2- 172.21.15.40:0/3224884209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d2130 0x7f73380d2e50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 --2- 172.21.15.40:0/3224884209 >> v2:172.21.15.184:3300/0 conn(0x7f73380d1770 0x7f73380d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 --2- 172.21.15.40:0/3224884209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 -- 172.21.15.40:0/3224884209 >> 172.21.15.40:0/3224884209 conn(0x7f73380d4f90 msgr2=0x7f73380d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 -- 172.21.15.40:0/3224884209 shutdown_connections 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.393+0000 7f733f914640 1 -- 172.21.15.40:0/3224884209 wait complete. 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.394+0000 7f733f914640 1 Processor -- start 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733f914640 1 -- start start 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733f914640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d1770 0x7f7338159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733f914640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73380d2130 0x7f73380b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733f914640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733f914640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f733815f900 con 0x7f73380d1770 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733f914640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f733815f780 con 0x7f73380d2130 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733f914640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73380d3390 con 0x7f733815f3a0 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733d6b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d1770 0x7f7338159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733d6b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d1770 0x7f7338159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46746/0 (socket says 172.21.15.40:46746) 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.395+0000 7f733d6b0640 1 -- 172.21.15.40:0/1012455508 learned_addr learned my addr 172.21.15.40:0/1012455508 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733deb1640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733ceaf640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73380d2130 0x7f73380b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733d6b0640 1 -- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 msgr2=0x7f73380b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733d6b0640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733d6b0640 1 -- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73380d2130 msgr2=0x7f73380b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733d6b0640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73380d2130 0x7f73380b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733d6b0640 1 -- 172.21.15.40:0/1012455508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73380b9880 con 0x7f73380d1770 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733deb1640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733ceaf640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73380d2130 0x7f73380b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733d6b0640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d1770 0x7f7338159600 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f733000c7f0 tx=0x7f733000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f7335ffb640 1 -- 172.21.15.40:0/1012455508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330015070 con 0x7f73380d1770 2024-04-26T21:38:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f7335ffb640 1 -- 172.21.15.40:0/1012455508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7330007820 con 0x7f73380d1770 2024-04-26T21:38:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73380b8210 con 0x7f73380d1770 2024-04-26T21:38:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f7335ffb640 1 -- 172.21.15.40:0/1012455508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330007bd0 con 0x7f73380d1770 2024-04-26T21:38:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.396+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73380b87d0 con 0x7f73380d1770 2024-04-26T21:38:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.397+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73380694f0 con 0x7f73380d1770 2024-04-26T21:38:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.398+0000 7f7335ffb640 1 -- 172.21.15.40:0/1012455508 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7330018070 con 0x7f73380d1770 2024-04-26T21:38:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.398+0000 7f7335ffb640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f732406c8f0 0x7f732406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.398+0000 7f7335ffb640 1 -- 172.21.15.40:0/1012455508 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f733008bd60 con 0x7f73380d1770 2024-04-26T21:38:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.398+0000 7f733ceaf640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f732406c8f0 0x7f732406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.399+0000 7f733ceaf640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f732406c8f0 0x7f732406edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f73380c8500 tx=0x7f732c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.402+0000 7f7335ffb640 1 -- 172.21.15.40:0/1012455508 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f733005a090 con 0x7f73380d1770 2024-04-26T21:38:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.590+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73380b6860 con 0x7f73380d1770 2024-04-26T21:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.592+0000 7f7335ffb640 1 -- 172.21.15.40:0/1012455508 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16861 (secure 0 0 0) 0x7f7330059990 con 0x7f73380d1770 2024-04-26T21:38:19.594 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:19.594 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":65,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:38:18.310396+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.593+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f732406c8f0 msgr2=0x7f732406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.593+0000 7f733f914640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f732406c8f0 0x7f732406edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f73380c8500 tx=0x7f732c008040 comp rx=0 tx=0).stop 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.593+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d1770 msgr2=0x7f7338159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.593+0000 7f733f914640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d1770 0x7f7338159600 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f733000c7f0 tx=0x7f733000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.594+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 shutdown_connections 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.594+0000 7f733f914640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f732406c8f0 0x7f732406edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.594+0000 7f733f914640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f733815f3a0 0x7f73380b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.594+0000 7f733f914640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73380d2130 0x7f73380b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.594+0000 7f733f914640 1 --2- 172.21.15.40:0/1012455508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73380d1770 0x7f7338159600 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.594+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 >> 172.21.15.40:0/1012455508 conn(0x7f73380d4f90 msgr2=0x7f73380b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.594+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 shutdown_connections 2024-04-26T21:38:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.594+0000 7f733f914640 1 -- 172.21.15.40:0/1012455508 wait complete. 2024-04-26T21:38:19.606 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:38:19.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.708+0000 7fa10d21d640 1 Processor -- start 2024-04-26T21:38:19.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.708+0000 7fa10d21d640 1 -- start start 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10d21d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa10815f170 0x7fa1080d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10d21d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa10815f550 0x7fa1080d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10d21d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1080d34b0 0x7fa1080d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10d21d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa1080c9430 con 0x7fa1080d34b0 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10d21d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa1080d3e20 con 0x7fa10815f170 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10d21d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa1080d1040 con 0x7fa10815f550 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1080d34b0 0x7fa1080d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1080d34b0 0x7fa1080d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46748/0 (socket says 172.21.15.40:46748) 2024-04-26T21:38:19.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10759e640 1 -- 172.21.15.40:0/858831762 learned_addr learned my addr 172.21.15.40:0/858831762 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10759e640 1 -- 172.21.15.40:0/858831762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa10815f550 msgr2=0x7fa1080d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10759e640 1 --2- 172.21.15.40:0/858831762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa10815f550 0x7fa1080d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10759e640 1 -- 172.21.15.40:0/858831762 >> v2:172.21.15.184:3300/0 conn(0x7fa10815f170 msgr2=0x7fa1080d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa106d9d640 1 --2- 172.21.15.40:0/858831762 >> v2:172.21.15.184:3300/0 conn(0x7fa10815f170 0x7fa1080d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10759e640 1 --2- 172.21.15.40:0/858831762 >> v2:172.21.15.184:3300/0 conn(0x7fa10815f170 0x7fa1080d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa10759e640 1 -- 172.21.15.40:0/858831762 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1080c9680 con 0x7fa1080d34b0 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.709+0000 7fa106d9d640 1 --2- 172.21.15.40:0/858831762 >> v2:172.21.15.184:3300/0 conn(0x7fa10815f170 0x7fa1080d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.710+0000 7fa10759e640 1 --2- 172.21.15.40:0/858831762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080d34b0 0x7fa1080d38e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa10808d450 tx=0x7fa10002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb22bc115e882da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.710+0000 7fa105d9b640 1 -- 172.21.15.40:0/858831762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100036070 con 0x7fa1080d34b0 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.710+0000 7fa105d9b640 1 -- 172.21.15.40:0/858831762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa100031600 con 0x7fa1080d34b0 2024-04-26T21:38:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.710+0000 7fa105d9b640 1 -- 172.21.15.40:0/858831762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1000319d0 con 0x7fa1080d34b0 2024-04-26T21:38:19.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.710+0000 7fa10d21d640 1 -- 172.21.15.40:0/858831762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080d34b0 msgr2=0x7fa1080d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.710+0000 7fa10d21d640 1 --2- 172.21.15.40:0/858831762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080d34b0 0x7fa1080d38e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa10808d450 tx=0x7fa10002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.711+0000 7fa10d21d640 1 -- 172.21.15.40:0/858831762 shutdown_connections 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.711+0000 7fa10d21d640 1 --2- 172.21.15.40:0/858831762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080d34b0 0x7fa1080d38e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.711+0000 7fa10d21d640 1 --2- 172.21.15.40:0/858831762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa10815f550 0x7fa1080d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.711+0000 7fa10d21d640 1 --2- 172.21.15.40:0/858831762 >> v2:172.21.15.184:3300/0 conn(0x7fa10815f170 0x7fa1080d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.711+0000 7fa10d21d640 1 -- 172.21.15.40:0/858831762 >> 172.21.15.40:0/858831762 conn(0x7fa1080d4f60 msgr2=0x7fa1080d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.711+0000 7fa10d21d640 1 -- 172.21.15.40:0/858831762 shutdown_connections 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.711+0000 7fa10d21d640 1 -- 172.21.15.40:0/858831762 wait complete. 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.712+0000 7fa10d21d640 1 Processor -- start 2024-04-26T21:38:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.712+0000 7fa10d21d640 1 -- start start 2024-04-26T21:38:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10d21d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1080d34b0 0x7fa108155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10d21d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10815f170 0x7fa1080b9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10d21d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1080b9cb0 0x7fa1080a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10d21d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1080cfc50 con 0x7fa10815f170 2024-04-26T21:38:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10d21d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1080c9c40 con 0x7fa1080d34b0 2024-04-26T21:38:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10d21d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa1080cfdd0 con 0x7fa1080b9cb0 2024-04-26T21:38:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10815f170 0x7fa1080b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10815f170 0x7fa1080b9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46758/0 (socket says 172.21.15.40:46758) 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10659c640 1 -- 172.21.15.40:0/2084180412 learned_addr learned my addr 172.21.15.40:0/2084180412 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa106d9d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1080d34b0 0x7fa108155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10759e640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1080b9cb0 0x7fa1080a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10659c640 1 -- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1080b9cb0 msgr2=0x7fa1080a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10659c640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1080b9cb0 0x7fa1080a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10659c640 1 -- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1080d34b0 msgr2=0x7fa108155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10659c640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1080d34b0 0x7fa108155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.713+0000 7fa10659c640 1 -- 172.21.15.40:0/2084180412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10815b9c0 con 0x7fa10815f170 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.714+0000 7fa10659c640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10815f170 0x7fa1080b9770 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00b410 tx=0x7fa0fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.714+0000 7fa0ef7fe640 1 -- 172.21.15.40:0/2084180412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc015020 con 0x7fa10815f170 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.714+0000 7fa0ef7fe640 1 -- 172.21.15.40:0/2084180412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0fc011850 con 0x7fa10815f170 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.714+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1080ba350 con 0x7fa10815f170 2024-04-26T21:38:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.714+0000 7fa0ef7fe640 1 -- 172.21.15.40:0/2084180412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc011c40 con 0x7fa10815f170 2024-04-26T21:38:19.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.714+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10809efe0 con 0x7fa10815f170 2024-04-26T21:38:19.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.715+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1080ba0c0 con 0x7fa10815f170 2024-04-26T21:38:19.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.715+0000 7fa0ef7fe640 1 -- 172.21.15.40:0/2084180412 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0fc018020 con 0x7fa10815f170 2024-04-26T21:38:19.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.716+0000 7fa0ef7fe640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa0f406c920 0x7fa0f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.716+0000 7fa0ef7fe640 1 -- 172.21.15.40:0/2084180412 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fa0fc08ba50 con 0x7fa10815f170 2024-04-26T21:38:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.716+0000 7fa106d9d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa0f406c920 0x7fa0f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.717+0000 7fa106d9d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa0f406c920 0x7fa0f406ede0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800ab60 tx=0x7fa0f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:19.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.720+0000 7fa0ef7fe640 1 -- 172.21.15.40:0/2084180412 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0fc01f020 con 0x7fa10815f170 2024-04-26T21:38:19.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:19.949+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa1080694f0 con 0x7fa10815f170 2024-04-26T21:38:20.919 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:38:20.918+0000 7ffae98d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:38:20.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.949+0000 7fa0ef7fe640 1 -- 172.21.15.40:0/2084180412 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) ==== 118+0+0 (secure 0 0 0) 0x7fa10815b9c0 con 0x7fa10815f170 2024-04-26T21:38:20.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa0f406c920 msgr2=0x7fa0f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa0f406c920 0x7fa0f406ede0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800ab60 tx=0x7fa0f8008040 comp rx=0 tx=0).stop 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10815f170 msgr2=0x7fa1080b9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10815f170 0x7fa1080b9770 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00b410 tx=0x7fa0fc00b8d0 comp rx=0 tx=0).stop 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 shutdown_connections 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa0f406c920 0x7fa0f406ede0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1080b9cb0 0x7fa1080a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10815f170 0x7fa1080b9770 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 --2- 172.21.15.40:0/2084180412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1080d34b0 0x7fa108155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 >> 172.21.15.40:0/2084180412 conn(0x7fa1080d4f60 msgr2=0x7fa10806e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 shutdown_connections 2024-04-26T21:38:20.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:20.952+0000 7fa10d21d640 1 -- 172.21.15.40:0/2084180412 wait complete. 2024-04-26T21:38:20.967 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:38:20.967 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:38:21.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.076+0000 7f0d84464640 1 Processor -- start 2024-04-26T21:38:21.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.076+0000 7f0d84464640 1 -- start start 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.077+0000 7f0d84464640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d7c0cfde0 0x7f0d7c0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.077+0000 7f0d84464640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0d7c070350 0x7f0d7c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.077+0000 7f0d84464640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d7c0d2390 con 0x7f0d7c071090 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.077+0000 7f0d84464640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0d7c0afab0 con 0x7f0d7c0cfde0 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.077+0000 7f0d84464640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0d7c06dd20 con 0x7f0d7c070350 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.078+0000 7f0d82200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d7c0cfde0 0x7f0d7c0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.078+0000 7f0d82200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d7c0cfde0 0x7f0d7c0d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46774/0 (socket says 172.21.15.40:46774) 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.078+0000 7f0d82200640 1 -- 172.21.15.40:0/187315811 learned_addr learned my addr 172.21.15.40:0/187315811 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.078+0000 7f0d819ff640 1 --2- 172.21.15.40:0/187315811 >> v2:172.21.15.184:3300/0 conn(0x7f0d7c070350 0x7f0d7c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.078+0000 7f0d82200640 1 -- 172.21.15.40:0/187315811 >> v2:172.21.15.184:3300/0 conn(0x7f0d7c070350 msgr2=0x7f0d7c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.078+0000 7f0d82200640 1 --2- 172.21.15.40:0/187315811 >> v2:172.21.15.184:3300/0 conn(0x7f0d7c070350 0x7f0d7c070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.078+0000 7f0d82200640 1 -- 172.21.15.40:0/187315811 >> v1:172.21.15.40:6789/0 conn(0x7f0d7c071090 legacy=0x7f0d7c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.078+0000 7f0d82200640 1 -- 172.21.15.40:0/187315811 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c06d3f0 con 0x7f0d7c0cfde0 2024-04-26T21:38:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.079+0000 7f0d82200640 1 --2- 172.21.15.40:0/187315811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c0cfde0 0x7f0d7c0d1b60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c06cb80 tx=0x7f0d7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c20ed685b363b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.079+0000 7f0d811fe640 1 -- 172.21.15.40:0/187315811 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78036070 con 0x7f0d7c0cfde0 2024-04-26T21:38:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.079+0000 7f0d811fe640 1 -- 172.21.15.40:0/187315811 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d78031600 con 0x7f0d7c0cfde0 2024-04-26T21:38:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.079+0000 7f0d811fe640 1 -- 172.21.15.40:0/187315811 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d780319d0 con 0x7f0d7c0cfde0 2024-04-26T21:38:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.079+0000 7f0d84464640 1 -- 172.21.15.40:0/187315811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c0cfde0 msgr2=0x7f0d7c0d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.079+0000 7f0d84464640 1 --2- 172.21.15.40:0/187315811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c0cfde0 0x7f0d7c0d1b60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c06cb80 tx=0x7f0d7802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.080+0000 7f0d84464640 1 -- 172.21.15.40:0/187315811 shutdown_connections 2024-04-26T21:38:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.080+0000 7f0d84464640 1 --2- 172.21.15.40:0/187315811 >> v2:172.21.15.184:3300/0 conn(0x7f0d7c070350 0x7f0d7c070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.080+0000 7f0d84464640 1 --2- 172.21.15.40:0/187315811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c0cfde0 0x7f0d7c0d1b60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.080+0000 7f0d84464640 1 -- 172.21.15.40:0/187315811 >> 172.21.15.40:0/187315811 conn(0x7f0d7c0d2d80 msgr2=0x7f0d7c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.080+0000 7f0d84464640 1 -- 172.21.15.40:0/187315811 shutdown_connections 2024-04-26T21:38:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.080+0000 7f0d84464640 1 -- 172.21.15.40:0/187315811 wait complete. 2024-04-26T21:38:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.081+0000 7f0d84464640 1 Processor -- start 2024-04-26T21:38:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.081+0000 7f0d84464640 1 -- start start 2024-04-26T21:38:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d84464640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c070350 0x7f0d7c1e5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d84464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c071090 0x7f0d7c1db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d84464640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0cfde0 0x7f0d7c1db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d84464640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c06d9b0 con 0x7f0d7c071090 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d84464640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c0714c0 con 0x7f0d7c070350 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d84464640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7c0d20a0 con 0x7f0d7c0cfde0 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c071090 0x7f0d7c1db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c071090 0x7f0d7c1db0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46778/0 (socket says 172.21.15.40:46778) 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 -- 172.21.15.40:0/4128389252 learned_addr learned my addr 172.21.15.40:0/4128389252 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d82200640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c070350 0x7f0d7c1e5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d82a01640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0cfde0 0x7f0d7c1db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 -- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0cfde0 msgr2=0x7f0d7c1db5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0cfde0 0x7f0d7c1db5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 -- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c070350 msgr2=0x7f0d7c1e5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c070350 0x7f0d7c1e5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 -- 172.21.15.40:0/4128389252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c09ec00 con 0x7f0d7c071090 2024-04-26T21:38:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.082+0000 7f0d819ff640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c071090 0x7f0d7c1db0a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d5f0 tx=0x7f0d7000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.083+0000 7f0d6effd640 1 -- 172.21.15.40:0/4128389252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700125f0 con 0x7f0d7c071090 2024-04-26T21:38:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.083+0000 7f0d6effd640 1 -- 172.21.15.40:0/4128389252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d70019070 con 0x7f0d7c071090 2024-04-26T21:38:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.083+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c09e920 con 0x7f0d7c071090 2024-04-26T21:38:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.083+0000 7f0d6effd640 1 -- 172.21.15.40:0/4128389252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700129d0 con 0x7f0d7c071090 2024-04-26T21:38:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.083+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c0b4bd0 con 0x7f0d7c071090 2024-04-26T21:38:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.084+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7c070eb0 con 0x7f0d7c071090 2024-04-26T21:38:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.084+0000 7f0d6effd640 1 -- 172.21.15.40:0/4128389252 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d70012b70 con 0x7f0d7c071090 2024-04-26T21:38:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.085+0000 7f0d6effd640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d6006c8f0 0x7f0d6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.085+0000 7f0d6effd640 1 -- 172.21.15.40:0/4128389252 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f0d7008bff0 con 0x7f0d7c071090 2024-04-26T21:38:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.085+0000 7f0d82200640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d6006c8f0 0x7f0d6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.086+0000 7f0d82200640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d6006c8f0 0x7f0d6006edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0d78033000 tx=0x7f0d78009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.089+0000 7f0d6effd640 1 -- 172.21.15.40:0/4128389252 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d7c070eb0 con 0x7f0d7c071090 2024-04-26T21:38:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.314+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0d7c0694f0 con 0x7f0d7c071090 2024-04-26T21:38:21.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.956+0000 7f0d6effd640 1 -- 172.21.15.40:0/4128389252 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f0d70059c40 con 0x7f0d7c071090 2024-04-26T21:38:21.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d6006c8f0 msgr2=0x7f0d6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:21.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d6006c8f0 0x7f0d6006edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0d78033000 tx=0x7f0d78009000 comp rx=0 tx=0).stop 2024-04-26T21:38:21.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c071090 msgr2=0x7f0d7c1db0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:21.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c071090 0x7f0d7c1db0a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d5f0 tx=0x7f0d7000dab0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 shutdown_connections 2024-04-26T21:38:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d6006c8f0 0x7f0d6006edb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0cfde0 0x7f0d7c1db5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d7c071090 0x7f0d7c1db0a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 --2- 172.21.15.40:0/4128389252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c070350 0x7f0d7c1e5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 >> 172.21.15.40:0/4128389252 conn(0x7f0d7c0d2d80 msgr2=0x7f0d7c0afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 shutdown_connections 2024-04-26T21:38:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:21.960+0000 7f0d84464640 1 -- 172.21.15.40:0/4128389252 wait complete. 2024-04-26T21:38:21.975 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:38:21.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:22.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.084+0000 7f94b7d24640 1 Processor -- start 2024-04-26T21:38:22.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.084+0000 7f94b7d24640 1 -- start start 2024-04-26T21:38:22.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b7d24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94b00b2340 0x7f94b00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b7d24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00cd930 0x7f94b00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b7d24640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b00d10d0 con 0x7f94b0162810 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b7d24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f94b00d0750 con 0x7f94b00b2340 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b7d24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94b00d16a0 con 0x7f94b00cd930 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b52bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94b00b2340 0x7f94b00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b52bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94b00b2340 0x7f94b00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46784/0 (socket says 172.21.15.40:46784) 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b52bf640 1 -- 172.21.15.40:0/617412824 learned_addr learned my addr 172.21.15.40:0/617412824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b62c1640 1 --2- 172.21.15.40:0/617412824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00cd930 0x7f94b00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b52bf640 1 -- 172.21.15.40:0/617412824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00cd930 msgr2=0x7f94b00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b52bf640 1 --2- 172.21.15.40:0/617412824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00cd930 0x7f94b00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b52bf640 1 -- 172.21.15.40:0/617412824 >> v1:172.21.15.40:6789/0 conn(0x7f94b0162810 legacy=0x7f94b0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.085+0000 7f94b52bf640 1 -- 172.21.15.40:0/617412824 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b00d1370 con 0x7f94b00b2340 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.086+0000 7f94b62c1640 1 --2- 172.21.15.40:0/617412824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00cd930 0x7f94b00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:22.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.086+0000 7f94b52bf640 1 --2- 172.21.15.40:0/617412824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00b2340 0x7f94b00cd3f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f94b00d1820 tx=0x7f94a002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0cbf5f0da05e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.086+0000 7f94b4abe640 1 -- 172.21.15.40:0/617412824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0037070 con 0x7f94b00b2340 2024-04-26T21:38:22.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.086+0000 7f94b4abe640 1 -- 172.21.15.40:0/617412824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a0032440 con 0x7f94b00b2340 2024-04-26T21:38:22.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.086+0000 7f94b4abe640 1 -- 172.21.15.40:0/617412824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0032810 con 0x7f94b00b2340 2024-04-26T21:38:22.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.087+0000 7f94b7d24640 1 -- 172.21.15.40:0/617412824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00b2340 msgr2=0x7f94b00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.087+0000 7f94b7d24640 1 --2- 172.21.15.40:0/617412824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00b2340 0x7f94b00cd3f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f94b00d1820 tx=0x7f94a002a250 comp rx=0 tx=0).stop 2024-04-26T21:38:22.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.087+0000 7f94b7d24640 1 -- 172.21.15.40:0/617412824 shutdown_connections 2024-04-26T21:38:22.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.087+0000 7f94b7d24640 1 --2- 172.21.15.40:0/617412824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00cd930 0x7f94b00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.087+0000 7f94b7d24640 1 --2- 172.21.15.40:0/617412824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00b2340 0x7f94b00cd3f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.087+0000 7f94b7d24640 1 -- 172.21.15.40:0/617412824 >> 172.21.15.40:0/617412824 conn(0x7f94b00d2da0 msgr2=0x7f94b00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.087+0000 7f94b7d24640 1 -- 172.21.15.40:0/617412824 shutdown_connections 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.087+0000 7f94b7d24640 1 -- 172.21.15.40:0/617412824 wait complete. 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.088+0000 7f94b7d24640 1 Processor -- start 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.088+0000 7f94b7d24640 1 -- start start 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b7d24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00b2340 0x7f94b00c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b7d24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00cd930 0x7f94b00afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b7d24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94b0162810 0x7f94b0153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b7d24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94b00d2070 con 0x7f94b00cd930 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b7d24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94b00d0470 con 0x7f94b0162810 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b7d24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94b00d21f0 con 0x7f94b00b2340 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b52bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00cd930 0x7f94b00afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b52bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00cd930 0x7f94b00afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46792/0 (socket says 172.21.15.40:46792) 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b52bf640 1 -- 172.21.15.40:0/2419321862 learned_addr learned my addr 172.21.15.40:0/2419321862 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b52bf640 1 -- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00b2340 msgr2=0x7f94b00c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b52bf640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00b2340 0x7f94b00c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b52bf640 1 -- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94b0162810 msgr2=0x7f94b0153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b62c1640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94b0162810 0x7f94b0153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b52bf640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94b0162810 0x7f94b0153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b52bf640 1 -- 172.21.15.40:0/2419321862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b01f50e0 con 0x7f94b00cd930 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.089+0000 7f94b62c1640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94b0162810 0x7f94b0153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.090+0000 7f94b52bf640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00cd930 0x7f94b00afd20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f94a002a290 tx=0x7f94a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.090+0000 7f94a67fc640 1 -- 172.21.15.40:0/2419321862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0037040 con 0x7f94b00cd930 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.090+0000 7f94a67fc640 1 -- 172.21.15.40:0/2419321862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a0042070 con 0x7f94b00cd930 2024-04-26T21:38:22.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.090+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b015ee10 con 0x7f94b00cd930 2024-04-26T21:38:22.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.090+0000 7f94a67fc640 1 -- 172.21.15.40:0/2419321862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0030bb0 con 0x7f94b00cd930 2024-04-26T21:38:22.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.090+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b015f4d0 con 0x7f94b00cd930 2024-04-26T21:38:22.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.091+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b00694f0 con 0x7f94b00cd930 2024-04-26T21:38:22.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.091+0000 7f94a67fc640 1 -- 172.21.15.40:0/2419321862 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94a0047460 con 0x7f94b00cd930 2024-04-26T21:38:22.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.092+0000 7f94a67fc640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.092+0000 7f94a67fc640 1 -- 172.21.15.40:0/2419321862 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f94a00ac950 con 0x7f94b00cd930 2024-04-26T21:38:22.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.092+0000 7f94b5ac0640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.092+0000 7f94b5ac0640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f949406c8f0 0x7f949406edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f94ac000f60 tx=0x7f94ac0079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.096+0000 7f94a67fc640 1 -- 172.21.15.40:0/2419321862 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94a0045360 con 0x7f94b00cd930 2024-04-26T21:38:22.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.324+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94b00cfb70 con 0x7f94b00cd930 2024-04-26T21:38:22.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.325+0000 7f94a67fc640 1 -- 172.21.15.40:0/2419321862 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5185 (secure 0 0 0) 0x7f94a007abf0 con 0x7f94b00cd930 2024-04-26T21:38:22.326 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:22.326 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:21.316626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1667529627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1667529627},{"type":"v1","addr":"172.21.15.40:6835","nonce":1667529627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":4}]} 2024-04-26T21:38:22.326 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2024-04-26T21:38:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f949406c8f0 msgr2=0x7f949406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f949406c8f0 0x7f949406edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f94ac000f60 tx=0x7f94ac0079d0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00cd930 msgr2=0x7f94b00afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00cd930 0x7f94b00afd20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f94a002a290 tx=0x7f94a00046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 shutdown_connections 2024-04-26T21:38:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94b0162810 0x7f94b0153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b00cd930 0x7f94b00afd20 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 --2- 172.21.15.40:0/2419321862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b00b2340 0x7f94b00c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 >> 172.21.15.40:0/2419321862 conn(0x7f94b00d2da0 msgr2=0x7f94b00d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 shutdown_connections 2024-04-26T21:38:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.327+0000 7f94b7d24640 1 -- 172.21.15.40:0/2419321862 wait complete. 2024-04-26T21:38:22.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:22.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.438+0000 7f319e82a640 1 Processor -- start 2024-04-26T21:38:22.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319e82a640 1 -- start start 2024-04-26T21:38:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319e82a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3198071e60 0x7f319806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319e82a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31980cc480 0x7f3198166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319e82a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f319815d7c0 con 0x7f319806fe20 2024-04-26T21:38:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319e82a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f319815daa0 con 0x7f31980cc480 2024-04-26T21:38:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319e82a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f319815dd80 con 0x7f3198071e60 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319cdc7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31980cc480 0x7f3198166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319cdc7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31980cc480 0x7f3198166ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46798/0 (socket says 172.21.15.40:46798) 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.439+0000 7f319cdc7640 1 -- 172.21.15.40:0/2587098318 learned_addr learned my addr 172.21.15.40:0/2587098318 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.440+0000 7f3197fff640 1 --2- 172.21.15.40:0/2587098318 >> v2:172.21.15.184:3300/0 conn(0x7f3198071e60 0x7f319806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.440+0000 7f319cdc7640 1 -- 172.21.15.40:0/2587098318 >> v2:172.21.15.184:3300/0 conn(0x7f3198071e60 msgr2=0x7f319806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.440+0000 7f319cdc7640 1 --2- 172.21.15.40:0/2587098318 >> v2:172.21.15.184:3300/0 conn(0x7f3198071e60 0x7f319806f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.440+0000 7f319cdc7640 1 -- 172.21.15.40:0/2587098318 >> v1:172.21.15.184:6789/0 conn(0x7f319806fe20 legacy=0x7f3198072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.440+0000 7f319cdc7640 1 -- 172.21.15.40:0/2587098318 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319815e060 con 0x7f31980cc480 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.440+0000 7f3197fff640 1 --2- 172.21.15.40:0/2587098318 >> v2:172.21.15.184:3300/0 conn(0x7f3198071e60 0x7f319806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.440+0000 7f319cdc7640 1 --2- 172.21.15.40:0/2587098318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31980cc480 0x7f3198166ad0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f31980b3700 tx=0x7f319002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e63af104b9b826 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f3196ffd640 1 -- 172.21.15.40:0/2587098318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190036070 con 0x7f31980cc480 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f3196ffd640 1 -- 172.21.15.40:0/2587098318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3190031600 con 0x7f31980cc480 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f3196ffd640 1 -- 172.21.15.40:0/2587098318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31900319d0 con 0x7f31980cc480 2024-04-26T21:38:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f319e82a640 1 -- 172.21.15.40:0/2587098318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31980cc480 msgr2=0x7f3198166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f319e82a640 1 --2- 172.21.15.40:0/2587098318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31980cc480 0x7f3198166ad0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f31980b3700 tx=0x7f319002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f319e82a640 1 -- 172.21.15.40:0/2587098318 shutdown_connections 2024-04-26T21:38:22.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f319e82a640 1 --2- 172.21.15.40:0/2587098318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31980cc480 0x7f3198166ad0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f319e82a640 1 --2- 172.21.15.40:0/2587098318 >> v2:172.21.15.184:3300/0 conn(0x7f3198071e60 0x7f319806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f319e82a640 1 -- 172.21.15.40:0/2587098318 >> 172.21.15.40:0/2587098318 conn(0x7f31980d0be0 msgr2=0x7f31980d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:22.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f319e82a640 1 -- 172.21.15.40:0/2587098318 shutdown_connections 2024-04-26T21:38:22.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.441+0000 7f319e82a640 1 -- 172.21.15.40:0/2587098318 wait complete. 2024-04-26T21:38:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.442+0000 7f319e82a640 1 Processor -- start 2024-04-26T21:38:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f319e82a640 1 -- start start 2024-04-26T21:38:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f319e82a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319806fe20 0x7f31980b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f319e82a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3198071e60 0x7f319814dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f319e82a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31980cc480 0x7f31980cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f319e82a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f319815d570 con 0x7f3198071e60 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f319e82a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f319815d3f0 con 0x7f31980cc480 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f319e82a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3198072950 con 0x7f319806fe20 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f31977fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3198071e60 0x7f319814dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f31977fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3198071e60 0x7f319814dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46814/0 (socket says 172.21.15.40:46814) 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.443+0000 7f31977fe640 1 -- 172.21.15.40:0/3937156317 learned_addr learned my addr 172.21.15.40:0/3937156317 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f319cdc7640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31980cc480 0x7f31980cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f3197fff640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319806fe20 0x7f31980b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f31977fe640 1 -- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319806fe20 msgr2=0x7f31980b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f31977fe640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319806fe20 0x7f31980b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f31977fe640 1 -- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31980cc480 msgr2=0x7f31980cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f31977fe640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31980cc480 0x7f31980cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f31977fe640 1 -- 172.21.15.40:0/3937156317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31980b4820 con 0x7f3198071e60 2024-04-26T21:38:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f31977fe640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3198071e60 0x7f319814dfc0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f318c00c430 tx=0x7f318c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f3194ff9640 1 -- 172.21.15.40:0/3937156317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c011790 con 0x7f3198071e60 2024-04-26T21:38:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f3194ff9640 1 -- 172.21.15.40:0/3937156317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c017070 con 0x7f3198071e60 2024-04-26T21:38:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31980b0ea0 con 0x7f3198071e60 2024-04-26T21:38:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f3194ff9640 1 -- 172.21.15.40:0/3937156317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c011bb0 con 0x7f3198071e60 2024-04-26T21:38:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.444+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31980b1460 con 0x7f3198071e60 2024-04-26T21:38:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.445+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31980b1660 con 0x7f3198071e60 2024-04-26T21:38:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.446+0000 7f3194ff9640 1 -- 172.21.15.40:0/3937156317 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f318c01c460 con 0x7f3198071e60 2024-04-26T21:38:22.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.446+0000 7f3194ff9640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318406c8f0 0x7f318406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.446+0000 7f3194ff9640 1 -- 172.21.15.40:0/3937156317 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f318c08b9a0 con 0x7f3198071e60 2024-04-26T21:38:22.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.446+0000 7f3197fff640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318406c8f0 0x7f318406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.447+0000 7f3197fff640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318406c8f0 0x7f318406edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f318800ab60 tx=0x7f3188008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.450+0000 7f3194ff9640 1 -- 172.21.15.40:0/3937156317 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31980b1660 con 0x7f3198071e60 2024-04-26T21:38:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.635+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31980694f0 con 0x7f3198071e60 2024-04-26T21:38:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.637+0000 7f3194ff9640 1 -- 172.21.15.40:0/3937156317 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16848 (secure 0 0 0) 0x7f318c059680 con 0x7f3198071e60 2024-04-26T21:38:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:22.638 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:38:19.929996+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318406c8f0 msgr2=0x7f318406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318406c8f0 0x7f318406edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f318800ab60 tx=0x7f3188008040 comp rx=0 tx=0).stop 2024-04-26T21:38:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3198071e60 msgr2=0x7f319814dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3198071e60 0x7f319814dfc0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f318c00c430 tx=0x7f318c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 shutdown_connections 2024-04-26T21:38:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318406c8f0 0x7f318406edb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31980cc480 0x7f31980cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3198071e60 0x7f319814dfc0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 --2- 172.21.15.40:0/3937156317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319806fe20 0x7f31980b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 >> 172.21.15.40:0/3937156317 conn(0x7f31980d0be0 msgr2=0x7f3198159430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 shutdown_connections 2024-04-26T21:38:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.639+0000 7f319e82a640 1 -- 172.21.15.40:0/3937156317 wait complete. 2024-04-26T21:38:22.651 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:38:22.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:38:22.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.750+0000 7fd05ffe4640 1 Processor -- start 2024-04-26T21:38:22.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.751+0000 7fd05ffe4640 1 -- start start 2024-04-26T21:38:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.751+0000 7fd05ffe4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd05806f790 0x7fd05806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.751+0000 7fd05ffe4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd058072c50 0x7fd058073060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.751+0000 7fd05ffe4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd058156840 con 0x7fd058156460 2024-04-26T21:38:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.751+0000 7fd05ffe4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0580cbf20 con 0x7fd058072c50 2024-04-26T21:38:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.751+0000 7fd05ffe4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd0580cc200 con 0x7fd05806f790 2024-04-26T21:38:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05e581640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd058072c50 0x7fd058073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05e581640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd058072c50 0x7fd058073060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46822/0 (socket says 172.21.15.40:46822) 2024-04-26T21:38:22.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05e581640 1 -- 172.21.15.40:0/3561262600 learned_addr learned my addr 172.21.15.40:0/3561262600 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:22.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05d57f640 1 --2- 172.21.15.40:0/3561262600 >> v2:172.21.15.184:3300/0 conn(0x7fd05806f790 0x7fd05806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05e581640 1 -- 172.21.15.40:0/3561262600 >> v2:172.21.15.184:3300/0 conn(0x7fd05806f790 msgr2=0x7fd05806fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:22.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05e581640 1 --2- 172.21.15.40:0/3561262600 >> v2:172.21.15.184:3300/0 conn(0x7fd05806f790 0x7fd05806fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05e581640 1 -- 172.21.15.40:0/3561262600 >> v1:172.21.15.40:6789/0 conn(0x7fd058156460 legacy=0x7fd0580cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05e581640 1 -- 172.21.15.40:0/3561262600 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0580cc4e0 con 0x7fd058072c50 2024-04-26T21:38:22.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.752+0000 7fd05d57f640 1 --2- 172.21.15.40:0/3561262600 >> v2:172.21.15.184:3300/0 conn(0x7fd05806f790 0x7fd05806fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05e581640 1 --2- 172.21.15.40:0/3561262600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd058073060 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd0580d0b90 tx=0x7fd05402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a5658ecec7f1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05cd7e640 1 -- 172.21.15.40:0/3561262600 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054036070 con 0x7fd058072c50 2024-04-26T21:38:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05cd7e640 1 -- 172.21.15.40:0/3561262600 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd054031600 con 0x7fd058072c50 2024-04-26T21:38:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05cd7e640 1 -- 172.21.15.40:0/3561262600 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540319d0 con 0x7fd058072c50 2024-04-26T21:38:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05ffe4640 1 -- 172.21.15.40:0/3561262600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 msgr2=0x7fd058073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/3561262600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd058073060 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd0580d0b90 tx=0x7fd05402a730 comp rx=0 tx=0).stop 2024-04-26T21:38:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05ffe4640 1 -- 172.21.15.40:0/3561262600 shutdown_connections 2024-04-26T21:38:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/3561262600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd058073060 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/3561262600 >> v2:172.21.15.184:3300/0 conn(0x7fd05806f790 0x7fd05806fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.753+0000 7fd05ffe4640 1 -- 172.21.15.40:0/3561262600 >> 172.21.15.40:0/3561262600 conn(0x7fd0580d0be0 msgr2=0x7fd0580d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.754+0000 7fd05ffe4640 1 -- 172.21.15.40:0/3561262600 shutdown_connections 2024-04-26T21:38:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.754+0000 7fd05ffe4640 1 -- 172.21.15.40:0/3561262600 wait complete. 2024-04-26T21:38:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.755+0000 7fd05ffe4640 1 Processor -- start 2024-04-26T21:38:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.755+0000 7fd05ffe4640 1 -- start start 2024-04-26T21:38:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.755+0000 7fd05ffe4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05806f790 0x7fd058146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.755+0000 7fd05ffe4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd0581541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05ffe4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd058156460 0x7fd0581546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05ffe4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0580cbbe0 con 0x7fd058072c50 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05ffe4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd058072700 con 0x7fd05806f790 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05ffe4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd05806e260 con 0x7fd058156460 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd0581541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd0581541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46832/0 (socket says 172.21.15.40:46832) 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 -- 172.21.15.40:0/246679666 learned_addr learned my addr 172.21.15.40:0/246679666 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05e581640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd058156460 0x7fd0581546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05dd80640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05806f790 0x7fd058146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 -- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd058156460 msgr2=0x7fd0581546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd058156460 0x7fd0581546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 -- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05806f790 msgr2=0x7fd058146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05806f790 0x7fd058146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 -- 172.21.15.40:0/246679666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0580c4c10 con 0x7fd058072c50 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05dd80640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05806f790 0x7fd058146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05e581640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd058156460 0x7fd0581546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05d57f640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd0581541b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e810 tx=0x7fd04800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd04e7fc640 1 -- 172.21.15.40:0/246679666 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04800f040 con 0x7fd058072c50 2024-04-26T21:38:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd04e7fc640 1 -- 172.21.15.40:0/246679666 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd048004030 con 0x7fd058072c50 2024-04-26T21:38:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0580c4ef0 con 0x7fd058072c50 2024-04-26T21:38:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd04e7fc640 1 -- 172.21.15.40:0/246679666 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048004400 con 0x7fd058072c50 2024-04-26T21:38:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.756+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0580c54b0 con 0x7fd058072c50 2024-04-26T21:38:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.757+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0580694f0 con 0x7fd058072c50 2024-04-26T21:38:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.758+0000 7fd04e7fc640 1 -- 172.21.15.40:0/246679666 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd04800c6d0 con 0x7fd058072c50 2024-04-26T21:38:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.758+0000 7fd04e7fc640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd03c06c8f0 0x7fd03c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.759+0000 7fd04e7fc640 1 -- 172.21.15.40:0/246679666 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fd04808ba60 con 0x7fd058072c50 2024-04-26T21:38:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.759+0000 7fd05dd80640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd03c06c8f0 0x7fd03c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.759+0000 7fd05dd80640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd03c06c8f0 0x7fd03c06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd050001660 tx=0x7fd050009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:22.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.763+0000 7fd04e7fc640 1 -- 172.21.15.40:0/246679666 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd048017020 con 0x7fd058072c50 2024-04-26T21:38:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.949+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd05806de80 con 0x7fd03c06c8f0 2024-04-26T21:38:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.955+0000 7fd04e7fc640 1 -- 172.21.15.40:0/246679666 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246709 (secure 0 0 0) 0x7fd05806de80 con 0x7fd03c06c8f0 2024-04-26T21:38:22.959 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:22.960 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:38:22.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd03c06c8f0 msgr2=0x7fd03c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd03c06c8f0 0x7fd03c06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd050001660 tx=0x7fd050009040 comp rx=0 tx=0).stop 2024-04-26T21:38:22.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 msgr2=0x7fd0581541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:22.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd0581541b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e810 tx=0x7fd04800ecd0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 shutdown_connections 2024-04-26T21:38:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd03c06c8f0 0x7fd03c06edb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd058156460 0x7fd0581546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd058072c50 0x7fd0581541b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 --2- 172.21.15.40:0/246679666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05806f790 0x7fd058146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 >> 172.21.15.40:0/246679666 conn(0x7fd0580d0be0 msgr2=0x7fd0580cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 shutdown_connections 2024-04-26T21:38:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:22.958+0000 7fd05ffe4640 1 -- 172.21.15.40:0/246679666 wait complete. 2024-04-26T21:38:22.971 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-04-26T21:38:22.039727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223304,"kb_used_data":7136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751416,"statfs":{"total":773094113280,"available":772865449984,"internally_reserved":0,"allocated":7307264,"data_stored":3948070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114996"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935415+0000","last_change":"2024-04-26T21:38:17.935415+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935415+0000","last_undegraded":"2024-04-26T21:38:17.935415+0000","last_fullsized":"2024-04-26T21:38:17.935415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933853+0000","last_change":"2024-04-26T21:38:17.933853+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933853+0000","last_undegraded":"2024-04-26T21:38:17.933853+0000","last_fullsized":"2024-04-26T21:38:17.933853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940393+0000","last_change":"2024-04-26T21:38:17.940393+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940393+0000","last_undegraded":"2024-04-26T21:38:17.940393+0000","last_fullsized":"2024-04-26T21:38:17.940393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939593+0000","last_change":"2024-04-26T21:38:17.939593+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939593+0000","last_undegraded":"2024-04-26T21:38:17.939593+0000","last_fullsized":"2024-04-26T21:38:17.939593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937487+0000","last_change":"2024-04-26T21:38:17.937487+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937487+0000","last_undegraded":"2024-04-26T21:38:17.937487+0000","last_fullsized":"2024-04-26T21:38:17.937487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940592+0000","last_change":"2024-04-26T21:38:17.940592+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940592+0000","last_undegraded":"2024-04-26T21:38:17.940592+0000","last_fullsized":"2024-04-26T21:38:17.940592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937934+0000","last_change":"2024-04-26T21:38:17.937934+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937934+0000","last_undegraded":"2024-04-26T21:38:17.937934+0000","last_fullsized":"2024-04-26T21:38:17.937934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934359+0000","last_change":"2024-04-26T21:38:17.934359+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934359+0000","last_undegraded":"2024-04-26T21:38:17.934359+0000","last_fullsized":"2024-04-26T21:38:17.934359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940540+0000","last_change":"2024-04-26T21:38:17.940540+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940540+0000","last_undegraded":"2024-04-26T21:38:17.940540+0000","last_fullsized":"2024-04-26T21:38:17.940540+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939851+0000","last_change":"2024-04-26T21:38:17.939851+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939851+0000","last_undegraded":"2024-04-26T21:38:17.939851+0000","last_fullsized":"2024-04-26T21:38:17.939851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939391+0000","last_change":"2024-04-26T21:38:17.939391+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939391+0000","last_undegraded":"2024-04-26T21:38:17.939391+0000","last_fullsized":"2024-04-26T21:38:17.939391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934346+0000","last_change":"2024-04-26T21:38:17.934346+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934346+0000","last_undegraded":"2024-04-26T21:38:17.934346+0000","last_fullsized":"2024-04-26T21:38:17.934346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936529+0000","last_change":"2024-04-26T21:38:17.936529+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936529+0000","last_undegraded":"2024-04-26T21:38:17.936529+0000","last_fullsized":"2024-04-26T21:38:17.936529+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938238+0000","last_change":"2024-04-26T21:38:17.938238+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938238+0000","last_undegraded":"2024-04-26T21:38:17.938238+0000","last_fullsized":"2024-04-26T21:38:17.938238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938428+0000","last_change":"2024-04-26T21:38:17.938428+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938428+0000","last_undegraded":"2024-04-26T21:38:17.938428+0000","last_fullsized":"2024-04-26T21:38:17.938428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938569+0000","last_change":"2024-04-26T21:38:17.938569+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938569+0000","last_undegraded":"2024-04-26T21:38:17.938569+0000","last_fullsized":"2024-04-26T21:38:17.938569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934251+0000","last_change":"2024-04-26T21:38:17.934251+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934251+0000","last_undegraded":"2024-04-26T21:38:17.934251+0000","last_fullsized":"2024-04-26T21:38:17.934251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938225+0000","last_change":"2024-04-26T21:38:17.938225+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938225+0000","last_undegraded":"2024-04-26T21:38:17.938225+0000","last_fullsized":"2024-04-26T21:38:17.938225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938791+0000","last_change":"2024-04-26T21:38:17.938791+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938791+0000","last_undegraded":"2024-04-26T21:38:17.938791+0000","last_fullsized":"2024-04-26T21:38:17.938791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934564+0000","last_change":"2024-04-26T21:38:17.934564+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934564+0000","last_undegraded":"2024-04-26T21:38:17.934564+0000","last_fullsized":"2024-04-26T21:38:17.934564+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934207+0000","last_change":"2024-04-26T21:38:17.934207+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934207+0000","last_undegraded":"2024-04-26T21:38:17.934207+0000","last_fullsized":"2024-04-26T21:38:17.934207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934474+0000","last_change":"2024-04-26T21:38:17.934474+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934474+0000","last_undegraded":"2024-04-26T21:38:17.934474+0000","last_fullsized":"2024-04-26T21:38:17.934474+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933962+0000","last_change":"2024-04-26T21:38:17.933962+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933962+0000","last_undegraded":"2024-04-26T21:38:17.933962+0000","last_fullsized":"2024-04-26T21:38:17.933962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937683+0000","last_change":"2024-04-26T21:38:17.937683+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937683+0000","last_undegraded":"2024-04-26T21:38:17.937683+0000","last_fullsized":"2024-04-26T21:38:17.937683+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934845+0000","last_change":"2024-04-26T21:38:17.934845+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934845+0000","last_undegraded":"2024-04-26T21:38:17.934845+0000","last_fullsized":"2024-04-26T21:38:17.934845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935655+0000","last_change":"2024-04-26T21:38:17.935655+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935655+0000","last_undegraded":"2024-04-26T21:38:17.935655+0000","last_fullsized":"2024-04-26T21:38:17.935655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939017+0000","last_change":"2024-04-26T21:38:17.939017+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939017+0000","last_undegraded":"2024-04-26T21:38:17.939017+0000","last_fullsized":"2024-04-26T21:38:17.939017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936461+0000","last_change":"2024-04-26T21:38:17.936461+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936461+0000","last_undegraded":"2024-04-26T21:38:17.936461+0000","last_fullsized":"2024-04-26T21:38:17.936461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938837+0000","last_change":"2024-04-26T21:38:17.938837+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938837+0000","last_undegraded":"2024-04-26T21:38:17.938837+0000","last_fullsized":"2024-04-26T21:38:17.938837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935425+0000","last_change":"2024-04-26T21:38:17.935425+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935425+0000","last_undegraded":"2024-04-26T21:38:17.935425+0000","last_fullsized":"2024-04-26T21:38:17.935425+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937570+0000","last_change":"2024-04-26T21:38:17.937570+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937570+0000","last_undegraded":"2024-04-26T21:38:17.937570+0000","last_fullsized":"2024-04-26T21:38:17.937570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941238+0000","last_change":"2024-04-26T21:38:17.941238+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941238+0000","last_undegraded":"2024-04-26T21:38:17.941238+0000","last_fullsized":"2024-04-26T21:38:17.941238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937920+0000","last_change":"2024-04-26T21:38:17.937920+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937920+0000","last_undegraded":"2024-04-26T21:38:17.937920+0000","last_fullsized":"2024-04-26T21:38:17.937920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950615+0000","last_change":"2024-04-26T21:38:17.948310+0000","last_active":"2024-04-26T21:38:17.950615+0000","last_peered":"2024-04-26T21:38:17.950615+0000","last_clean":"2024-04-26T21:38:17.950615+0000","last_became_active":"2024-04-26T21:38:17.948052+0000","last_became_peered":"2024-04-26T21:38:17.948052+0000","last_unstale":"2024-04-26T21:38:17.950615+0000","last_undegraded":"2024-04-26T21:38:17.950615+0000","last_fullsized":"2024-04-26T21:38:17.950615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:29.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938659+0000","last_change":"2024-04-26T21:38:17.938659+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938659+0000","last_undegraded":"2024-04-26T21:38:17.938659+0000","last_fullsized":"2024-04-26T21:38:17.938659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951738+0000","last_change":"2024-04-26T21:38:17.949729+0000","last_active":"2024-04-26T21:38:17.951738+0000","last_peered":"2024-04-26T21:38:17.951738+0000","last_clean":"2024-04-26T21:38:17.951738+0000","last_became_active":"2024-04-26T21:38:17.949412+0000","last_became_peered":"2024-04-26T21:38:17.949412+0000","last_unstale":"2024-04-26T21:38:17.951738+0000","last_undegraded":"2024-04-26T21:38:17.951738+0000","last_fullsized":"2024-04-26T21:38:17.951738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:20.434769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941869+0000","last_change":"2024-04-26T21:38:17.941869+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941869+0000","last_undegraded":"2024-04-26T21:38:17.941869+0000","last_fullsized":"2024-04-26T21:38:17.941869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950645+0000","last_change":"2024-04-26T21:38:17.947141+0000","last_active":"2024-04-26T21:38:17.950645+0000","last_peered":"2024-04-26T21:38:17.950645+0000","last_clean":"2024-04-26T21:38:17.950645+0000","last_became_active":"2024-04-26T21:38:17.946847+0000","last_became_peered":"2024-04-26T21:38:17.946847+0000","last_unstale":"2024-04-26T21:38:17.950645+0000","last_undegraded":"2024-04-26T21:38:17.950645+0000","last_fullsized":"2024-04-26T21:38:17.950645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.686151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940906+0000","last_change":"2024-04-26T21:38:17.940906+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940906+0000","last_undegraded":"2024-04-26T21:38:17.940906+0000","last_fullsized":"2024-04-26T21:38:17.940906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951770+0000","last_change":"2024-04-26T21:38:17.949992+0000","last_active":"2024-04-26T21:38:17.951770+0000","last_peered":"2024-04-26T21:38:17.951770+0000","last_clean":"2024-04-26T21:38:17.951770+0000","last_became_active":"2024-04-26T21:38:17.949701+0000","last_became_peered":"2024-04-26T21:38:17.949701+0000","last_unstale":"2024-04-26T21:38:17.951770+0000","last_undegraded":"2024-04-26T21:38:17.951770+0000","last_fullsized":"2024-04-26T21:38:17.951770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:12.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938112+0000","last_change":"2024-04-26T21:38:17.938112+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938112+0000","last_undegraded":"2024-04-26T21:38:17.938112+0000","last_fullsized":"2024-04-26T21:38:17.938112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950255+0000","last_change":"2024-04-26T21:38:17.947017+0000","last_active":"2024-04-26T21:38:17.950255+0000","last_peered":"2024-04-26T21:38:17.950255+0000","last_clean":"2024-04-26T21:38:17.950255+0000","last_became_active":"2024-04-26T21:38:17.946744+0000","last_became_peered":"2024-04-26T21:38:17.946744+0000","last_unstale":"2024-04-26T21:38:17.950255+0000","last_undegraded":"2024-04-26T21:38:17.950255+0000","last_fullsized":"2024-04-26T21:38:17.950255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:22.534915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933792+0000","last_change":"2024-04-26T21:38:17.933792+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933792+0000","last_undegraded":"2024-04-26T21:38:17.933792+0000","last_fullsized":"2024-04-26T21:38:17.933792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948630+0000","last_change":"2024-04-26T21:38:17.944477+0000","last_active":"2024-04-26T21:38:17.948630+0000","last_peered":"2024-04-26T21:38:17.948630+0000","last_clean":"2024-04-26T21:38:17.948630+0000","last_became_active":"2024-04-26T21:38:17.944149+0000","last_became_peered":"2024-04-26T21:38:17.944149+0000","last_unstale":"2024-04-26T21:38:17.948630+0000","last_undegraded":"2024-04-26T21:38:17.948630+0000","last_fullsized":"2024-04-26T21:38:17.948630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:03.182220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939989+0000","last_change":"2024-04-26T21:38:17.939989+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939989+0000","last_undegraded":"2024-04-26T21:38:17.939989+0000","last_fullsized":"2024-04-26T21:38:17.939989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948727+0000","last_change":"2024-04-26T21:38:17.945215+0000","last_active":"2024-04-26T21:38:17.948727+0000","last_peered":"2024-04-26T21:38:17.948727+0000","last_clean":"2024-04-26T21:38:17.948727+0000","last_became_active":"2024-04-26T21:38:17.944579+0000","last_became_peered":"2024-04-26T21:38:17.944579+0000","last_unstale":"2024-04-26T21:38:17.948727+0000","last_undegraded":"2024-04-26T21:38:17.948727+0000","last_fullsized":"2024-04-26T21:38:17.948727+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:00.507529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933926+0000","last_change":"2024-04-26T21:38:17.933926+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933926+0000","last_undegraded":"2024-04-26T21:38:17.933926+0000","last_fullsized":"2024-04-26T21:38:17.933926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952152+0000","last_change":"2024-04-26T21:38:17.949823+0000","last_active":"2024-04-26T21:38:17.952152+0000","last_peered":"2024-04-26T21:38:17.952152+0000","last_clean":"2024-04-26T21:38:17.952152+0000","last_became_active":"2024-04-26T21:38:17.949531+0000","last_became_peered":"2024-04-26T21:38:17.949531+0000","last_unstale":"2024-04-26T21:38:17.952152+0000","last_undegraded":"2024-04-26T21:38:17.952152+0000","last_fullsized":"2024-04-26T21:38:17.952152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:17.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935879+0000","last_change":"2024-04-26T21:38:17.935879+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935879+0000","last_undegraded":"2024-04-26T21:38:17.935879+0000","last_fullsized":"2024-04-26T21:38:17.935879+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953189+0000","last_change":"2024-04-26T21:38:17.951863+0000","last_active":"2024-04-26T21:38:17.953189+0000","last_peered":"2024-04-26T21:38:17.953189+0000","last_clean":"2024-04-26T21:38:17.953189+0000","last_became_active":"2024-04-26T21:38:17.951250+0000","last_became_peered":"2024-04-26T21:38:17.951250+0000","last_unstale":"2024-04-26T21:38:17.953189+0000","last_undegraded":"2024-04-26T21:38:17.953189+0000","last_fullsized":"2024-04-26T21:38:17.953189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:57.060894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933756+0000","last_change":"2024-04-26T21:38:17.933756+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933756+0000","last_undegraded":"2024-04-26T21:38:17.933756+0000","last_fullsized":"2024-04-26T21:38:17.933756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953111+0000","last_change":"2024-04-26T21:38:17.952295+0000","last_active":"2024-04-26T21:38:17.953111+0000","last_peered":"2024-04-26T21:38:17.953111+0000","last_clean":"2024-04-26T21:38:17.953111+0000","last_became_active":"2024-04-26T21:38:17.951993+0000","last_became_peered":"2024-04-26T21:38:17.951993+0000","last_unstale":"2024-04-26T21:38:17.953111+0000","last_undegraded":"2024-04-26T21:38:17.953111+0000","last_fullsized":"2024-04-26T21:38:17.953111+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:44:51.686328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938886+0000","last_change":"2024-04-26T21:38:17.938886+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938886+0000","last_undegraded":"2024-04-26T21:38:17.938886+0000","last_fullsized":"2024-04-26T21:38:17.938886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953151+0000","last_change":"2024-04-26T21:38:17.951334+0000","last_active":"2024-04-26T21:38:17.953151+0000","last_peered":"2024-04-26T21:38:17.953151+0000","last_clean":"2024-04-26T21:38:17.953151+0000","last_became_active":"2024-04-26T21:38:17.951044+0000","last_became_peered":"2024-04-26T21:38:17.951044+0000","last_unstale":"2024-04-26T21:38:17.953151+0000","last_undegraded":"2024-04-26T21:38:17.953151+0000","last_fullsized":"2024-04-26T21:38:17.953151+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:47.039536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939860+0000","last_change":"2024-04-26T21:38:17.939860+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939860+0000","last_undegraded":"2024-04-26T21:38:17.939860+0000","last_fullsized":"2024-04-26T21:38:17.939860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950495+0000","last_change":"2024-04-26T21:38:17.945882+0000","last_active":"2024-04-26T21:38:17.950495+0000","last_peered":"2024-04-26T21:38:17.950495+0000","last_clean":"2024-04-26T21:38:17.950495+0000","last_became_active":"2024-04-26T21:38:17.945594+0000","last_became_peered":"2024-04-26T21:38:17.945594+0000","last_unstale":"2024-04-26T21:38:17.950495+0000","last_undegraded":"2024-04-26T21:38:17.950495+0000","last_fullsized":"2024-04-26T21:38:17.950495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:50.308949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936947+0000","last_change":"2024-04-26T21:38:17.936947+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936947+0000","last_undegraded":"2024-04-26T21:38:17.936947+0000","last_fullsized":"2024-04-26T21:38:17.936947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948676+0000","last_change":"2024-04-26T21:38:17.947105+0000","last_active":"2024-04-26T21:38:17.948676+0000","last_peered":"2024-04-26T21:38:17.948676+0000","last_clean":"2024-04-26T21:38:17.948676+0000","last_became_active":"2024-04-26T21:38:17.946804+0000","last_became_peered":"2024-04-26T21:38:17.946804+0000","last_unstale":"2024-04-26T21:38:17.948676+0000","last_undegraded":"2024-04-26T21:38:17.948676+0000","last_fullsized":"2024-04-26T21:38:17.948676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:58.559420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935577+0000","last_change":"2024-04-26T21:38:17.935577+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935577+0000","last_undegraded":"2024-04-26T21:38:17.935577+0000","last_fullsized":"2024-04-26T21:38:17.935577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945985+0000","last_change":"2024-04-26T21:38:17.944988+0000","last_active":"2024-04-26T21:38:17.945985+0000","last_peered":"2024-04-26T21:38:17.945985+0000","last_clean":"2024-04-26T21:38:17.945985+0000","last_became_active":"2024-04-26T21:38:17.944724+0000","last_became_peered":"2024-04-26T21:38:17.944724+0000","last_unstale":"2024-04-26T21:38:17.945985+0000","last_undegraded":"2024-04-26T21:38:17.945985+0000","last_fullsized":"2024-04-26T21:38:17.945985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939621+0000","last_change":"2024-04-26T21:38:17.939621+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939621+0000","last_undegraded":"2024-04-26T21:38:17.939621+0000","last_fullsized":"2024-04-26T21:38:17.939621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950969+0000","last_change":"2024-04-26T21:38:17.947884+0000","last_active":"2024-04-26T21:38:17.950969+0000","last_peered":"2024-04-26T21:38:17.950969+0000","last_clean":"2024-04-26T21:38:17.950969+0000","last_became_active":"2024-04-26T21:38:17.947599+0000","last_became_peered":"2024-04-26T21:38:17.947599+0000","last_unstale":"2024-04-26T21:38:17.950969+0000","last_undegraded":"2024-04-26T21:38:17.950969+0000","last_fullsized":"2024-04-26T21:38:17.950969+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:02.996302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933961+0000","last_change":"2024-04-26T21:38:17.933961+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933961+0000","last_undegraded":"2024-04-26T21:38:17.933961+0000","last_fullsized":"2024-04-26T21:38:17.933961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952474+0000","last_change":"2024-04-26T21:38:17.948438+0000","last_active":"2024-04-26T21:38:17.952474+0000","last_peered":"2024-04-26T21:38:17.952474+0000","last_clean":"2024-04-26T21:38:17.952474+0000","last_became_active":"2024-04-26T21:38:17.948196+0000","last_became_peered":"2024-04-26T21:38:17.948196+0000","last_unstale":"2024-04-26T21:38:17.952474+0000","last_undegraded":"2024-04-26T21:38:17.952474+0000","last_fullsized":"2024-04-26T21:38:17.952474+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:32.653604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937129+0000","last_change":"2024-04-26T21:38:17.937129+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937129+0000","last_undegraded":"2024-04-26T21:38:17.937129+0000","last_fullsized":"2024-04-26T21:38:17.937129+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945381+0000","last_change":"2024-04-26T21:38:17.942502+0000","last_active":"2024-04-26T21:38:17.945381+0000","last_peered":"2024-04-26T21:38:17.945381+0000","last_clean":"2024-04-26T21:38:17.945381+0000","last_became_active":"2024-04-26T21:38:17.942225+0000","last_became_peered":"2024-04-26T21:38:17.942225+0000","last_unstale":"2024-04-26T21:38:17.945381+0000","last_undegraded":"2024-04-26T21:38:17.945381+0000","last_fullsized":"2024-04-26T21:38:17.945381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:02.996302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938374+0000","last_change":"2024-04-26T21:38:17.938374+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938374+0000","last_undegraded":"2024-04-26T21:38:17.938374+0000","last_fullsized":"2024-04-26T21:38:17.938374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952434+0000","last_change":"2024-04-26T21:38:17.949980+0000","last_active":"2024-04-26T21:38:17.952434+0000","last_peered":"2024-04-26T21:38:17.952434+0000","last_clean":"2024-04-26T21:38:17.952434+0000","last_became_active":"2024-04-26T21:38:17.949710+0000","last_became_peered":"2024-04-26T21:38:17.949710+0000","last_unstale":"2024-04-26T21:38:17.952434+0000","last_undegraded":"2024-04-26T21:38:17.952434+0000","last_fullsized":"2024-04-26T21:38:17.952434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937829+0000","last_change":"2024-04-26T21:38:17.937829+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937829+0000","last_undegraded":"2024-04-26T21:38:17.937829+0000","last_fullsized":"2024-04-26T21:38:17.937829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953177+0000","last_change":"2024-04-26T21:38:17.951499+0000","last_active":"2024-04-26T21:38:17.953177+0000","last_peered":"2024-04-26T21:38:17.953177+0000","last_clean":"2024-04-26T21:38:17.953177+0000","last_became_active":"2024-04-26T21:38:17.951190+0000","last_became_peered":"2024-04-26T21:38:17.951190+0000","last_unstale":"2024-04-26T21:38:17.953177+0000","last_undegraded":"2024-04-26T21:38:17.953177+0000","last_fullsized":"2024-04-26T21:38:17.953177+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:12.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938444+0000","last_change":"2024-04-26T21:38:17.938444+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938444+0000","last_undegraded":"2024-04-26T21:38:17.938444+0000","last_fullsized":"2024-04-26T21:38:17.938444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948659+0000","last_change":"2024-04-26T21:38:17.943246+0000","last_active":"2024-04-26T21:38:17.948659+0000","last_peered":"2024-04-26T21:38:17.948659+0000","last_clean":"2024-04-26T21:38:17.948659+0000","last_became_active":"2024-04-26T21:38:17.942975+0000","last_became_peered":"2024-04-26T21:38:17.942975+0000","last_unstale":"2024-04-26T21:38:17.948659+0000","last_undegraded":"2024-04-26T21:38:17.948659+0000","last_fullsized":"2024-04-26T21:38:17.948659+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:44.198607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940171+0000","last_change":"2024-04-26T21:38:17.940171+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940171+0000","last_undegraded":"2024-04-26T21:38:17.940171+0000","last_fullsized":"2024-04-26T21:38:17.940171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950719+0000","last_change":"2024-04-26T21:38:17.946995+0000","last_active":"2024-04-26T21:38:17.950719+0000","last_peered":"2024-04-26T21:38:17.950719+0000","last_clean":"2024-04-26T21:38:17.950719+0000","last_became_active":"2024-04-26T21:38:17.946697+0000","last_became_peered":"2024-04-26T21:38:17.946697+0000","last_unstale":"2024-04-26T21:38:17.950719+0000","last_undegraded":"2024-04-26T21:38:17.950719+0000","last_fullsized":"2024-04-26T21:38:17.950719+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:56.530385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934101+0000","last_change":"2024-04-26T21:38:17.934101+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934101+0000","last_undegraded":"2024-04-26T21:38:17.934101+0000","last_fullsized":"2024-04-26T21:38:17.934101+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945837+0000","last_change":"2024-04-26T21:38:17.942558+0000","last_active":"2024-04-26T21:38:17.945837+0000","last_peered":"2024-04-26T21:38:17.945837+0000","last_clean":"2024-04-26T21:38:17.945837+0000","last_became_active":"2024-04-26T21:38:17.942267+0000","last_became_peered":"2024-04-26T21:38:17.942267+0000","last_unstale":"2024-04-26T21:38:17.945837+0000","last_undegraded":"2024-04-26T21:38:17.945837+0000","last_fullsized":"2024-04-26T21:38:17.945837+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:00.927686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939224+0000","last_change":"2024-04-26T21:38:17.939224+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939224+0000","last_undegraded":"2024-04-26T21:38:17.939224+0000","last_fullsized":"2024-04-26T21:38:17.939224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950979+0000","last_change":"2024-04-26T21:38:17.948494+0000","last_active":"2024-04-26T21:38:17.950979+0000","last_peered":"2024-04-26T21:38:17.950979+0000","last_clean":"2024-04-26T21:38:17.950979+0000","last_became_active":"2024-04-26T21:38:17.948212+0000","last_became_peered":"2024-04-26T21:38:17.948212+0000","last_unstale":"2024-04-26T21:38:17.950979+0000","last_undegraded":"2024-04-26T21:38:17.950979+0000","last_fullsized":"2024-04-26T21:38:17.950979+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:00.507529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939650+0000","last_change":"2024-04-26T21:38:17.939650+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939650+0000","last_undegraded":"2024-04-26T21:38:17.939650+0000","last_fullsized":"2024-04-26T21:38:17.939650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.955821+0000","last_change":"2024-04-26T21:38:17.951721+0000","last_active":"2024-04-26T21:38:17.955821+0000","last_peered":"2024-04-26T21:38:17.955821+0000","last_clean":"2024-04-26T21:38:17.955821+0000","last_became_active":"2024-04-26T21:38:17.951400+0000","last_became_peered":"2024-04-26T21:38:17.951400+0000","last_unstale":"2024-04-26T21:38:17.955821+0000","last_undegraded":"2024-04-26T21:38:17.955821+0000","last_fullsized":"2024-04-26T21:38:17.955821+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:12.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939217+0000","last_change":"2024-04-26T21:38:17.939217+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939217+0000","last_undegraded":"2024-04-26T21:38:17.939217+0000","last_fullsized":"2024-04-26T21:38:17.939217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.947698+0000","last_change":"2024-04-26T21:38:17.945997+0000","last_active":"2024-04-26T21:38:17.947698+0000","last_peered":"2024-04-26T21:38:17.947698+0000","last_clean":"2024-04-26T21:38:17.947698+0000","last_became_active":"2024-04-26T21:38:17.945730+0000","last_became_peered":"2024-04-26T21:38:17.945730+0000","last_unstale":"2024-04-26T21:38:17.947698+0000","last_undegraded":"2024-04-26T21:38:17.947698+0000","last_fullsized":"2024-04-26T21:38:17.947698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:47.039536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934677+0000","last_change":"2024-04-26T21:38:17.934677+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934677+0000","last_undegraded":"2024-04-26T21:38:17.934677+0000","last_fullsized":"2024-04-26T21:38:17.934677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951528+0000","last_change":"2024-04-26T21:38:17.949610+0000","last_active":"2024-04-26T21:38:17.951528+0000","last_peered":"2024-04-26T21:38:17.951528+0000","last_clean":"2024-04-26T21:38:17.951528+0000","last_became_active":"2024-04-26T21:38:17.949247+0000","last_became_peered":"2024-04-26T21:38:17.949247+0000","last_unstale":"2024-04-26T21:38:17.951528+0000","last_undegraded":"2024-04-26T21:38:17.951528+0000","last_fullsized":"2024-04-26T21:38:17.951528+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:22.534915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938020+0000","last_change":"2024-04-26T21:38:17.938020+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938020+0000","last_undegraded":"2024-04-26T21:38:17.938020+0000","last_fullsized":"2024-04-26T21:38:17.938020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950521+0000","last_change":"2024-04-26T21:38:17.946463+0000","last_active":"2024-04-26T21:38:17.950521+0000","last_peered":"2024-04-26T21:38:17.950521+0000","last_clean":"2024-04-26T21:38:17.950521+0000","last_became_active":"2024-04-26T21:38:17.946182+0000","last_became_peered":"2024-04-26T21:38:17.946182+0000","last_unstale":"2024-04-26T21:38:17.950521+0000","last_undegraded":"2024-04-26T21:38:17.950521+0000","last_fullsized":"2024-04-26T21:38:17.950521+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:22.259886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941345+0000","last_change":"2024-04-26T21:38:17.941345+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941345+0000","last_undegraded":"2024-04-26T21:38:17.941345+0000","last_fullsized":"2024-04-26T21:38:17.941345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951656+0000","last_change":"2024-04-26T21:38:17.949728+0000","last_active":"2024-04-26T21:38:17.951656+0000","last_peered":"2024-04-26T21:38:17.951656+0000","last_clean":"2024-04-26T21:38:17.951656+0000","last_became_active":"2024-04-26T21:38:17.949450+0000","last_became_peered":"2024-04-26T21:38:17.949450+0000","last_unstale":"2024-04-26T21:38:17.951656+0000","last_undegraded":"2024-04-26T21:38:17.951656+0000","last_fullsized":"2024-04-26T21:38:17.951656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:00.927686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937164+0000","last_change":"2024-04-26T21:38:17.937164+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937164+0000","last_undegraded":"2024-04-26T21:38:17.937164+0000","last_fullsized":"2024-04-26T21:38:17.937164+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.940335+0000","last_change":"2024-04-26T21:38:17.937457+0000","last_active":"2024-04-26T21:38:17.940335+0000","last_peered":"2024-04-26T21:38:17.940335+0000","last_clean":"2024-04-26T21:38:17.940335+0000","last_became_active":"2024-04-26T21:38:17.937039+0000","last_became_peered":"2024-04-26T21:38:17.937039+0000","last_unstale":"2024-04-26T21:38:17.940335+0000","last_undegraded":"2024-04-26T21:38:17.940335+0000","last_fullsized":"2024-04-26T21:38:17.940335+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:46.534568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934742+0000","last_change":"2024-04-26T21:38:17.934742+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934742+0000","last_undegraded":"2024-04-26T21:38:17.934742+0000","last_fullsized":"2024-04-26T21:38:17.934742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948683+0000","last_change":"2024-04-26T21:38:17.944966+0000","last_active":"2024-04-26T21:38:17.948683+0000","last_peered":"2024-04-26T21:38:17.948683+0000","last_clean":"2024-04-26T21:38:17.948683+0000","last_became_active":"2024-04-26T21:38:17.944185+0000","last_became_peered":"2024-04-26T21:38:17.944185+0000","last_unstale":"2024-04-26T21:38:17.948683+0000","last_undegraded":"2024-04-26T21:38:17.948683+0000","last_fullsized":"2024-04-26T21:38:17.948683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:56.530385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938690+0000","last_change":"2024-04-26T21:38:17.938690+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938690+0000","last_undegraded":"2024-04-26T21:38:17.938690+0000","last_fullsized":"2024-04-26T21:38:17.938690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950383+0000","last_change":"2024-04-26T21:38:17.945734+0000","last_active":"2024-04-26T21:38:17.950383+0000","last_peered":"2024-04-26T21:38:17.950383+0000","last_clean":"2024-04-26T21:38:17.950383+0000","last_became_active":"2024-04-26T21:38:17.945450+0000","last_became_peered":"2024-04-26T21:38:17.945450+0000","last_unstale":"2024-04-26T21:38:17.950383+0000","last_undegraded":"2024-04-26T21:38:17.950383+0000","last_fullsized":"2024-04-26T21:38:17.950383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:35.855859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936991+0000","last_change":"2024-04-26T21:38:17.936991+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936991+0000","last_undegraded":"2024-04-26T21:38:17.936991+0000","last_fullsized":"2024-04-26T21:38:17.936991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948681+0000","last_change":"2024-04-26T21:38:17.944876+0000","last_active":"2024-04-26T21:38:17.948681+0000","last_peered":"2024-04-26T21:38:17.948681+0000","last_clean":"2024-04-26T21:38:17.948681+0000","last_became_active":"2024-04-26T21:38:17.944590+0000","last_became_peered":"2024-04-26T21:38:17.944590+0000","last_unstale":"2024-04-26T21:38:17.948681+0000","last_undegraded":"2024-04-26T21:38:17.948681+0000","last_fullsized":"2024-04-26T21:38:17.948681+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.686151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":251,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.938085+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:38:17.938085+0000","last_peered":"2024-04-26T21:38:17.938085+0000","last_clean":"2024-04-26T21:38:17.938085+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:38:17.938085+0000","last_undegraded":"2024-04-26T21:38:17.938085+0000","last_fullsized":"2024-04-26T21:38:17.938085+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343480,"statfs":{"total":96636764160,"available":96607723520,"internally_reserved":0,"allocated":1368064,"data_stored":941279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343480,"statfs":{"total":96636764160,"available":96607723520,"internally_reserved":0,"allocated":1368064,"data_stored":941279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:38:22.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:38:23.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.111+0000 7f5cbda09640 1 Processor -- start 2024-04-26T21:38:23.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.111+0000 7f5cbda09640 1 -- start start 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.112+0000 7f5cbda09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cb815f5a0 0x7f5cb80d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.112+0000 7f5cbda09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d2830 0x7f5cb8162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.112+0000 7f5cbda09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb80d1a70 con 0x7f5cb80d3c20 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.112+0000 7f5cbda09640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb8163d30 con 0x7f5cb815f5a0 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.112+0000 7f5cbda09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb80d1d50 con 0x7f5cb80d2830 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.112+0000 7f5cb6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cb815f5a0 0x7f5cb80d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.112+0000 7f5cb6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cb815f5a0 0x7f5cb80d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46848/0 (socket says 172.21.15.40:46848) 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.112+0000 7f5cb6ffd640 1 -- 172.21.15.40:0/760874061 learned_addr learned my addr 172.21.15.40:0/760874061 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.113+0000 7f5cb67fc640 1 --2- 172.21.15.40:0/760874061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d2830 0x7f5cb8162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.113+0000 7f5cb6ffd640 1 -- 172.21.15.40:0/760874061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d2830 msgr2=0x7f5cb8162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:23.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.113+0000 7f5cb6ffd640 1 --2- 172.21.15.40:0/760874061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d2830 0x7f5cb8162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.113+0000 7f5cb6ffd640 1 -- 172.21.15.40:0/760874061 >> v1:172.21.15.40:6789/0 conn(0x7f5cb80d3c20 legacy=0x7f5cb815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.113+0000 7f5cb6ffd640 1 -- 172.21.15.40:0/760874061 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb80d2030 con 0x7f5cb815f5a0 2024-04-26T21:38:23.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.113+0000 7f5cb67fc640 1 --2- 172.21.15.40:0/760874061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d2830 0x7f5cb8162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.113+0000 7f5cb6ffd640 1 --2- 172.21.15.40:0/760874061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb815f5a0 0x7f5cb80d5cb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009870 tx=0x7f5ca802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40d584ce00eb9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cb5ffb640 1 -- 172.21.15.40:0/760874061 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8037070 con 0x7f5cb815f5a0 2024-04-26T21:38:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cb5ffb640 1 -- 172.21.15.40:0/760874061 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca8032440 con 0x7f5cb815f5a0 2024-04-26T21:38:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cb5ffb640 1 -- 172.21.15.40:0/760874061 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8032810 con 0x7f5cb815f5a0 2024-04-26T21:38:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cbda09640 1 -- 172.21.15.40:0/760874061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb815f5a0 msgr2=0x7f5cb80d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cbda09640 1 --2- 172.21.15.40:0/760874061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb815f5a0 0x7f5cb80d5cb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009870 tx=0x7f5ca802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cbda09640 1 -- 172.21.15.40:0/760874061 shutdown_connections 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cbda09640 1 --2- 172.21.15.40:0/760874061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d2830 0x7f5cb8162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cbda09640 1 --2- 172.21.15.40:0/760874061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb815f5a0 0x7f5cb80d5cb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cbda09640 1 -- 172.21.15.40:0/760874061 >> 172.21.15.40:0/760874061 conn(0x7f5cb80d5190 msgr2=0x7f5cb80d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cbda09640 1 -- 172.21.15.40:0/760874061 shutdown_connections 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.114+0000 7f5cbda09640 1 -- 172.21.15.40:0/760874061 wait complete. 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.115+0000 7f5cbda09640 1 Processor -- start 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cbda09640 1 -- start start 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cbda09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb80d2830 0x7f5cb80cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cbda09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d3c20 0x7f5cb81f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cbda09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb80ce260 0x7f5cb81f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cbda09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb8163440 con 0x7f5cb80ce260 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cbda09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb80d4190 con 0x7f5cb80d2830 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cbda09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb81635c0 con 0x7f5cb80d3c20 2024-04-26T21:38:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cb77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb80ce260 0x7f5cb81f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cb77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb80ce260 0x7f5cb81f5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46860/0 (socket says 172.21.15.40:46860) 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.116+0000 7f5cb77fe640 1 -- 172.21.15.40:0/550063011 learned_addr learned my addr 172.21.15.40:0/550063011 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cb67fc640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d3c20 0x7f5cb81f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cb6ffd640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb80d2830 0x7f5cb80cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cb77fe640 1 -- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d3c20 msgr2=0x7f5cb81f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cb77fe640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d3c20 0x7f5cb81f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cb77fe640 1 -- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb80d2830 msgr2=0x7f5cb80cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cb77fe640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb80d2830 0x7f5cb80cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cb77fe640 1 -- 172.21.15.40:0/550063011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb815afd0 con 0x7f5cb80ce260 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cb77fe640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb80ce260 0x7f5cb81f5dd0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000c7a0 tx=0x7f5cb000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5c9ffff640 1 -- 172.21.15.40:0/550063011 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0003b70 con 0x7f5cb80ce260 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5c9ffff640 1 -- 172.21.15.40:0/550063011 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb0003d30 con 0x7f5cb80ce260 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb80ccb40 con 0x7f5cb80ce260 2024-04-26T21:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5c9ffff640 1 -- 172.21.15.40:0/550063011 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000aa90 con 0x7f5cb80ce260 2024-04-26T21:38:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.117+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb80cd100 con 0x7f5cb80ce260 2024-04-26T21:38:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.118+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb80694f0 con 0x7f5cb80ce260 2024-04-26T21:38:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.118+0000 7f5c9ffff640 1 -- 172.21.15.40:0/550063011 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5cb000ac30 con 0x7f5cb80ce260 2024-04-26T21:38:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.119+0000 7f5c9ffff640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca406c8f0 0x7f5ca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.119+0000 7f5c9ffff640 1 -- 172.21.15.40:0/550063011 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f5cb008bc80 con 0x7f5cb80ce260 2024-04-26T21:38:23.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.119+0000 7f5cb6ffd640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca406c8f0 0x7f5ca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.120+0000 7f5cb6ffd640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca406c8f0 0x7f5ca406edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8034000 tx=0x7f5ca802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.123+0000 7f5c9ffff640 1 -- 172.21.15.40:0/550063011 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5cb0059f20 con 0x7f5cb80ce260 2024-04-26T21:38:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.306+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cb80cc020 con 0x7f5ca406c8f0 2024-04-26T21:38:23.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.313+0000 7f5c9ffff640 1 -- 172.21.15.40:0/550063011 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246709 (secure 0 0 0) 0x7f5cb80cc020 con 0x7f5ca406c8f0 2024-04-26T21:38:23.317 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:23.318 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:38:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca406c8f0 msgr2=0x7f5ca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca406c8f0 0x7f5ca406edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8034000 tx=0x7f5ca802e000 comp rx=0 tx=0).stop 2024-04-26T21:38:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb80ce260 msgr2=0x7f5cb81f5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb80ce260 0x7f5cb81f5dd0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000c7a0 tx=0x7f5cb000cc60 comp rx=0 tx=0).stop 2024-04-26T21:38:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 shutdown_connections 2024-04-26T21:38:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca406c8f0 0x7f5ca406edb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb80ce260 0x7f5cb81f5dd0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb80d3c20 0x7f5cb81f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 --2- 172.21.15.40:0/550063011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb80d2830 0x7f5cb80cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 >> 172.21.15.40:0/550063011 conn(0x7f5cb80d5190 msgr2=0x7f5cb80cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 shutdown_connections 2024-04-26T21:38:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.316+0000 7f5cbda09640 1 -- 172.21.15.40:0/550063011 wait complete. 2024-04-26T21:38:23.328 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-04-26T21:38:22.039727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223304,"kb_used_data":7136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751416,"statfs":{"total":773094113280,"available":772865449984,"internally_reserved":0,"allocated":7307264,"data_stored":3948070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114996"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935415+0000","last_change":"2024-04-26T21:38:17.935415+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935415+0000","last_undegraded":"2024-04-26T21:38:17.935415+0000","last_fullsized":"2024-04-26T21:38:17.935415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933853+0000","last_change":"2024-04-26T21:38:17.933853+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933853+0000","last_undegraded":"2024-04-26T21:38:17.933853+0000","last_fullsized":"2024-04-26T21:38:17.933853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940393+0000","last_change":"2024-04-26T21:38:17.940393+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940393+0000","last_undegraded":"2024-04-26T21:38:17.940393+0000","last_fullsized":"2024-04-26T21:38:17.940393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939593+0000","last_change":"2024-04-26T21:38:17.939593+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939593+0000","last_undegraded":"2024-04-26T21:38:17.939593+0000","last_fullsized":"2024-04-26T21:38:17.939593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937487+0000","last_change":"2024-04-26T21:38:17.937487+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937487+0000","last_undegraded":"2024-04-26T21:38:17.937487+0000","last_fullsized":"2024-04-26T21:38:17.937487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940592+0000","last_change":"2024-04-26T21:38:17.940592+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940592+0000","last_undegraded":"2024-04-26T21:38:17.940592+0000","last_fullsized":"2024-04-26T21:38:17.940592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937934+0000","last_change":"2024-04-26T21:38:17.937934+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937934+0000","last_undegraded":"2024-04-26T21:38:17.937934+0000","last_fullsized":"2024-04-26T21:38:17.937934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934359+0000","last_change":"2024-04-26T21:38:17.934359+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934359+0000","last_undegraded":"2024-04-26T21:38:17.934359+0000","last_fullsized":"2024-04-26T21:38:17.934359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940540+0000","last_change":"2024-04-26T21:38:17.940540+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940540+0000","last_undegraded":"2024-04-26T21:38:17.940540+0000","last_fullsized":"2024-04-26T21:38:17.940540+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939851+0000","last_change":"2024-04-26T21:38:17.939851+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939851+0000","last_undegraded":"2024-04-26T21:38:17.939851+0000","last_fullsized":"2024-04-26T21:38:17.939851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939391+0000","last_change":"2024-04-26T21:38:17.939391+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939391+0000","last_undegraded":"2024-04-26T21:38:17.939391+0000","last_fullsized":"2024-04-26T21:38:17.939391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934346+0000","last_change":"2024-04-26T21:38:17.934346+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934346+0000","last_undegraded":"2024-04-26T21:38:17.934346+0000","last_fullsized":"2024-04-26T21:38:17.934346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936529+0000","last_change":"2024-04-26T21:38:17.936529+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936529+0000","last_undegraded":"2024-04-26T21:38:17.936529+0000","last_fullsized":"2024-04-26T21:38:17.936529+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938238+0000","last_change":"2024-04-26T21:38:17.938238+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938238+0000","last_undegraded":"2024-04-26T21:38:17.938238+0000","last_fullsized":"2024-04-26T21:38:17.938238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938428+0000","last_change":"2024-04-26T21:38:17.938428+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938428+0000","last_undegraded":"2024-04-26T21:38:17.938428+0000","last_fullsized":"2024-04-26T21:38:17.938428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938569+0000","last_change":"2024-04-26T21:38:17.938569+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938569+0000","last_undegraded":"2024-04-26T21:38:17.938569+0000","last_fullsized":"2024-04-26T21:38:17.938569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934251+0000","last_change":"2024-04-26T21:38:17.934251+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934251+0000","last_undegraded":"2024-04-26T21:38:17.934251+0000","last_fullsized":"2024-04-26T21:38:17.934251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938225+0000","last_change":"2024-04-26T21:38:17.938225+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938225+0000","last_undegraded":"2024-04-26T21:38:17.938225+0000","last_fullsized":"2024-04-26T21:38:17.938225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938791+0000","last_change":"2024-04-26T21:38:17.938791+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938791+0000","last_undegraded":"2024-04-26T21:38:17.938791+0000","last_fullsized":"2024-04-26T21:38:17.938791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934564+0000","last_change":"2024-04-26T21:38:17.934564+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934564+0000","last_undegraded":"2024-04-26T21:38:17.934564+0000","last_fullsized":"2024-04-26T21:38:17.934564+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934207+0000","last_change":"2024-04-26T21:38:17.934207+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934207+0000","last_undegraded":"2024-04-26T21:38:17.934207+0000","last_fullsized":"2024-04-26T21:38:17.934207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934474+0000","last_change":"2024-04-26T21:38:17.934474+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934474+0000","last_undegraded":"2024-04-26T21:38:17.934474+0000","last_fullsized":"2024-04-26T21:38:17.934474+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933962+0000","last_change":"2024-04-26T21:38:17.933962+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933962+0000","last_undegraded":"2024-04-26T21:38:17.933962+0000","last_fullsized":"2024-04-26T21:38:17.933962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937683+0000","last_change":"2024-04-26T21:38:17.937683+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937683+0000","last_undegraded":"2024-04-26T21:38:17.937683+0000","last_fullsized":"2024-04-26T21:38:17.937683+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934845+0000","last_change":"2024-04-26T21:38:17.934845+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934845+0000","last_undegraded":"2024-04-26T21:38:17.934845+0000","last_fullsized":"2024-04-26T21:38:17.934845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935655+0000","last_change":"2024-04-26T21:38:17.935655+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935655+0000","last_undegraded":"2024-04-26T21:38:17.935655+0000","last_fullsized":"2024-04-26T21:38:17.935655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939017+0000","last_change":"2024-04-26T21:38:17.939017+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939017+0000","last_undegraded":"2024-04-26T21:38:17.939017+0000","last_fullsized":"2024-04-26T21:38:17.939017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936461+0000","last_change":"2024-04-26T21:38:17.936461+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936461+0000","last_undegraded":"2024-04-26T21:38:17.936461+0000","last_fullsized":"2024-04-26T21:38:17.936461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938837+0000","last_change":"2024-04-26T21:38:17.938837+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938837+0000","last_undegraded":"2024-04-26T21:38:17.938837+0000","last_fullsized":"2024-04-26T21:38:17.938837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935425+0000","last_change":"2024-04-26T21:38:17.935425+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935425+0000","last_undegraded":"2024-04-26T21:38:17.935425+0000","last_fullsized":"2024-04-26T21:38:17.935425+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937570+0000","last_change":"2024-04-26T21:38:17.937570+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937570+0000","last_undegraded":"2024-04-26T21:38:17.937570+0000","last_fullsized":"2024-04-26T21:38:17.937570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941238+0000","last_change":"2024-04-26T21:38:17.941238+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941238+0000","last_undegraded":"2024-04-26T21:38:17.941238+0000","last_fullsized":"2024-04-26T21:38:17.941238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937920+0000","last_change":"2024-04-26T21:38:17.937920+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937920+0000","last_undegraded":"2024-04-26T21:38:17.937920+0000","last_fullsized":"2024-04-26T21:38:17.937920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950615+0000","last_change":"2024-04-26T21:38:17.948310+0000","last_active":"2024-04-26T21:38:17.950615+0000","last_peered":"2024-04-26T21:38:17.950615+0000","last_clean":"2024-04-26T21:38:17.950615+0000","last_became_active":"2024-04-26T21:38:17.948052+0000","last_became_peered":"2024-04-26T21:38:17.948052+0000","last_unstale":"2024-04-26T21:38:17.950615+0000","last_undegraded":"2024-04-26T21:38:17.950615+0000","last_fullsized":"2024-04-26T21:38:17.950615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:29.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938659+0000","last_change":"2024-04-26T21:38:17.938659+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938659+0000","last_undegraded":"2024-04-26T21:38:17.938659+0000","last_fullsized":"2024-04-26T21:38:17.938659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951738+0000","last_change":"2024-04-26T21:38:17.949729+0000","last_active":"2024-04-26T21:38:17.951738+0000","last_peered":"2024-04-26T21:38:17.951738+0000","last_clean":"2024-04-26T21:38:17.951738+0000","last_became_active":"2024-04-26T21:38:17.949412+0000","last_became_peered":"2024-04-26T21:38:17.949412+0000","last_unstale":"2024-04-26T21:38:17.951738+0000","last_undegraded":"2024-04-26T21:38:17.951738+0000","last_fullsized":"2024-04-26T21:38:17.951738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:20.434769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941869+0000","last_change":"2024-04-26T21:38:17.941869+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941869+0000","last_undegraded":"2024-04-26T21:38:17.941869+0000","last_fullsized":"2024-04-26T21:38:17.941869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950645+0000","last_change":"2024-04-26T21:38:17.947141+0000","last_active":"2024-04-26T21:38:17.950645+0000","last_peered":"2024-04-26T21:38:17.950645+0000","last_clean":"2024-04-26T21:38:17.950645+0000","last_became_active":"2024-04-26T21:38:17.946847+0000","last_became_peered":"2024-04-26T21:38:17.946847+0000","last_unstale":"2024-04-26T21:38:17.950645+0000","last_undegraded":"2024-04-26T21:38:17.950645+0000","last_fullsized":"2024-04-26T21:38:17.950645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.686151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940906+0000","last_change":"2024-04-26T21:38:17.940906+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940906+0000","last_undegraded":"2024-04-26T21:38:17.940906+0000","last_fullsized":"2024-04-26T21:38:17.940906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951770+0000","last_change":"2024-04-26T21:38:17.949992+0000","last_active":"2024-04-26T21:38:17.951770+0000","last_peered":"2024-04-26T21:38:17.951770+0000","last_clean":"2024-04-26T21:38:17.951770+0000","last_became_active":"2024-04-26T21:38:17.949701+0000","last_became_peered":"2024-04-26T21:38:17.949701+0000","last_unstale":"2024-04-26T21:38:17.951770+0000","last_undegraded":"2024-04-26T21:38:17.951770+0000","last_fullsized":"2024-04-26T21:38:17.951770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:12.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938112+0000","last_change":"2024-04-26T21:38:17.938112+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938112+0000","last_undegraded":"2024-04-26T21:38:17.938112+0000","last_fullsized":"2024-04-26T21:38:17.938112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950255+0000","last_change":"2024-04-26T21:38:17.947017+0000","last_active":"2024-04-26T21:38:17.950255+0000","last_peered":"2024-04-26T21:38:17.950255+0000","last_clean":"2024-04-26T21:38:17.950255+0000","last_became_active":"2024-04-26T21:38:17.946744+0000","last_became_peered":"2024-04-26T21:38:17.946744+0000","last_unstale":"2024-04-26T21:38:17.950255+0000","last_undegraded":"2024-04-26T21:38:17.950255+0000","last_fullsized":"2024-04-26T21:38:17.950255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:22.534915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933792+0000","last_change":"2024-04-26T21:38:17.933792+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933792+0000","last_undegraded":"2024-04-26T21:38:17.933792+0000","last_fullsized":"2024-04-26T21:38:17.933792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948630+0000","last_change":"2024-04-26T21:38:17.944477+0000","last_active":"2024-04-26T21:38:17.948630+0000","last_peered":"2024-04-26T21:38:17.948630+0000","last_clean":"2024-04-26T21:38:17.948630+0000","last_became_active":"2024-04-26T21:38:17.944149+0000","last_became_peered":"2024-04-26T21:38:17.944149+0000","last_unstale":"2024-04-26T21:38:17.948630+0000","last_undegraded":"2024-04-26T21:38:17.948630+0000","last_fullsized":"2024-04-26T21:38:17.948630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:03.182220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939989+0000","last_change":"2024-04-26T21:38:17.939989+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939989+0000","last_undegraded":"2024-04-26T21:38:17.939989+0000","last_fullsized":"2024-04-26T21:38:17.939989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948727+0000","last_change":"2024-04-26T21:38:17.945215+0000","last_active":"2024-04-26T21:38:17.948727+0000","last_peered":"2024-04-26T21:38:17.948727+0000","last_clean":"2024-04-26T21:38:17.948727+0000","last_became_active":"2024-04-26T21:38:17.944579+0000","last_became_peered":"2024-04-26T21:38:17.944579+0000","last_unstale":"2024-04-26T21:38:17.948727+0000","last_undegraded":"2024-04-26T21:38:17.948727+0000","last_fullsized":"2024-04-26T21:38:17.948727+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:00.507529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933926+0000","last_change":"2024-04-26T21:38:17.933926+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933926+0000","last_undegraded":"2024-04-26T21:38:17.933926+0000","last_fullsized":"2024-04-26T21:38:17.933926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952152+0000","last_change":"2024-04-26T21:38:17.949823+0000","last_active":"2024-04-26T21:38:17.952152+0000","last_peered":"2024-04-26T21:38:17.952152+0000","last_clean":"2024-04-26T21:38:17.952152+0000","last_became_active":"2024-04-26T21:38:17.949531+0000","last_became_peered":"2024-04-26T21:38:17.949531+0000","last_unstale":"2024-04-26T21:38:17.952152+0000","last_undegraded":"2024-04-26T21:38:17.952152+0000","last_fullsized":"2024-04-26T21:38:17.952152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:17.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935879+0000","last_change":"2024-04-26T21:38:17.935879+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935879+0000","last_undegraded":"2024-04-26T21:38:17.935879+0000","last_fullsized":"2024-04-26T21:38:17.935879+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953189+0000","last_change":"2024-04-26T21:38:17.951863+0000","last_active":"2024-04-26T21:38:17.953189+0000","last_peered":"2024-04-26T21:38:17.953189+0000","last_clean":"2024-04-26T21:38:17.953189+0000","last_became_active":"2024-04-26T21:38:17.951250+0000","last_became_peered":"2024-04-26T21:38:17.951250+0000","last_unstale":"2024-04-26T21:38:17.953189+0000","last_undegraded":"2024-04-26T21:38:17.953189+0000","last_fullsized":"2024-04-26T21:38:17.953189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:57.060894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933756+0000","last_change":"2024-04-26T21:38:17.933756+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933756+0000","last_undegraded":"2024-04-26T21:38:17.933756+0000","last_fullsized":"2024-04-26T21:38:17.933756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953111+0000","last_change":"2024-04-26T21:38:17.952295+0000","last_active":"2024-04-26T21:38:17.953111+0000","last_peered":"2024-04-26T21:38:17.953111+0000","last_clean":"2024-04-26T21:38:17.953111+0000","last_became_active":"2024-04-26T21:38:17.951993+0000","last_became_peered":"2024-04-26T21:38:17.951993+0000","last_unstale":"2024-04-26T21:38:17.953111+0000","last_undegraded":"2024-04-26T21:38:17.953111+0000","last_fullsized":"2024-04-26T21:38:17.953111+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:44:51.686328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938886+0000","last_change":"2024-04-26T21:38:17.938886+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938886+0000","last_undegraded":"2024-04-26T21:38:17.938886+0000","last_fullsized":"2024-04-26T21:38:17.938886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953151+0000","last_change":"2024-04-26T21:38:17.951334+0000","last_active":"2024-04-26T21:38:17.953151+0000","last_peered":"2024-04-26T21:38:17.953151+0000","last_clean":"2024-04-26T21:38:17.953151+0000","last_became_active":"2024-04-26T21:38:17.951044+0000","last_became_peered":"2024-04-26T21:38:17.951044+0000","last_unstale":"2024-04-26T21:38:17.953151+0000","last_undegraded":"2024-04-26T21:38:17.953151+0000","last_fullsized":"2024-04-26T21:38:17.953151+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:47.039536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939860+0000","last_change":"2024-04-26T21:38:17.939860+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939860+0000","last_undegraded":"2024-04-26T21:38:17.939860+0000","last_fullsized":"2024-04-26T21:38:17.939860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950495+0000","last_change":"2024-04-26T21:38:17.945882+0000","last_active":"2024-04-26T21:38:17.950495+0000","last_peered":"2024-04-26T21:38:17.950495+0000","last_clean":"2024-04-26T21:38:17.950495+0000","last_became_active":"2024-04-26T21:38:17.945594+0000","last_became_peered":"2024-04-26T21:38:17.945594+0000","last_unstale":"2024-04-26T21:38:17.950495+0000","last_undegraded":"2024-04-26T21:38:17.950495+0000","last_fullsized":"2024-04-26T21:38:17.950495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:50.308949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936947+0000","last_change":"2024-04-26T21:38:17.936947+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936947+0000","last_undegraded":"2024-04-26T21:38:17.936947+0000","last_fullsized":"2024-04-26T21:38:17.936947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948676+0000","last_change":"2024-04-26T21:38:17.947105+0000","last_active":"2024-04-26T21:38:17.948676+0000","last_peered":"2024-04-26T21:38:17.948676+0000","last_clean":"2024-04-26T21:38:17.948676+0000","last_became_active":"2024-04-26T21:38:17.946804+0000","last_became_peered":"2024-04-26T21:38:17.946804+0000","last_unstale":"2024-04-26T21:38:17.948676+0000","last_undegraded":"2024-04-26T21:38:17.948676+0000","last_fullsized":"2024-04-26T21:38:17.948676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:58.559420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935577+0000","last_change":"2024-04-26T21:38:17.935577+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935577+0000","last_undegraded":"2024-04-26T21:38:17.935577+0000","last_fullsized":"2024-04-26T21:38:17.935577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945985+0000","last_change":"2024-04-26T21:38:17.944988+0000","last_active":"2024-04-26T21:38:17.945985+0000","last_peered":"2024-04-26T21:38:17.945985+0000","last_clean":"2024-04-26T21:38:17.945985+0000","last_became_active":"2024-04-26T21:38:17.944724+0000","last_became_peered":"2024-04-26T21:38:17.944724+0000","last_unstale":"2024-04-26T21:38:17.945985+0000","last_undegraded":"2024-04-26T21:38:17.945985+0000","last_fullsized":"2024-04-26T21:38:17.945985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939621+0000","last_change":"2024-04-26T21:38:17.939621+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939621+0000","last_undegraded":"2024-04-26T21:38:17.939621+0000","last_fullsized":"2024-04-26T21:38:17.939621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950969+0000","last_change":"2024-04-26T21:38:17.947884+0000","last_active":"2024-04-26T21:38:17.950969+0000","last_peered":"2024-04-26T21:38:17.950969+0000","last_clean":"2024-04-26T21:38:17.950969+0000","last_became_active":"2024-04-26T21:38:17.947599+0000","last_became_peered":"2024-04-26T21:38:17.947599+0000","last_unstale":"2024-04-26T21:38:17.950969+0000","last_undegraded":"2024-04-26T21:38:17.950969+0000","last_fullsized":"2024-04-26T21:38:17.950969+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:02.996302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933961+0000","last_change":"2024-04-26T21:38:17.933961+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933961+0000","last_undegraded":"2024-04-26T21:38:17.933961+0000","last_fullsized":"2024-04-26T21:38:17.933961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952474+0000","last_change":"2024-04-26T21:38:17.948438+0000","last_active":"2024-04-26T21:38:17.952474+0000","last_peered":"2024-04-26T21:38:17.952474+0000","last_clean":"2024-04-26T21:38:17.952474+0000","last_became_active":"2024-04-26T21:38:17.948196+0000","last_became_peered":"2024-04-26T21:38:17.948196+0000","last_unstale":"2024-04-26T21:38:17.952474+0000","last_undegraded":"2024-04-26T21:38:17.952474+0000","last_fullsized":"2024-04-26T21:38:17.952474+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:32.653604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937129+0000","last_change":"2024-04-26T21:38:17.937129+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937129+0000","last_undegraded":"2024-04-26T21:38:17.937129+0000","last_fullsized":"2024-04-26T21:38:17.937129+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945381+0000","last_change":"2024-04-26T21:38:17.942502+0000","last_active":"2024-04-26T21:38:17.945381+0000","last_peered":"2024-04-26T21:38:17.945381+0000","last_clean":"2024-04-26T21:38:17.945381+0000","last_became_active":"2024-04-26T21:38:17.942225+0000","last_became_peered":"2024-04-26T21:38:17.942225+0000","last_unstale":"2024-04-26T21:38:17.945381+0000","last_undegraded":"2024-04-26T21:38:17.945381+0000","last_fullsized":"2024-04-26T21:38:17.945381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:02.996302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938374+0000","last_change":"2024-04-26T21:38:17.938374+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938374+0000","last_undegraded":"2024-04-26T21:38:17.938374+0000","last_fullsized":"2024-04-26T21:38:17.938374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952434+0000","last_change":"2024-04-26T21:38:17.949980+0000","last_active":"2024-04-26T21:38:17.952434+0000","last_peered":"2024-04-26T21:38:17.952434+0000","last_clean":"2024-04-26T21:38:17.952434+0000","last_became_active":"2024-04-26T21:38:17.949710+0000","last_became_peered":"2024-04-26T21:38:17.949710+0000","last_unstale":"2024-04-26T21:38:17.952434+0000","last_undegraded":"2024-04-26T21:38:17.952434+0000","last_fullsized":"2024-04-26T21:38:17.952434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937829+0000","last_change":"2024-04-26T21:38:17.937829+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937829+0000","last_undegraded":"2024-04-26T21:38:17.937829+0000","last_fullsized":"2024-04-26T21:38:17.937829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953177+0000","last_change":"2024-04-26T21:38:17.951499+0000","last_active":"2024-04-26T21:38:17.953177+0000","last_peered":"2024-04-26T21:38:17.953177+0000","last_clean":"2024-04-26T21:38:17.953177+0000","last_became_active":"2024-04-26T21:38:17.951190+0000","last_became_peered":"2024-04-26T21:38:17.951190+0000","last_unstale":"2024-04-26T21:38:17.953177+0000","last_undegraded":"2024-04-26T21:38:17.953177+0000","last_fullsized":"2024-04-26T21:38:17.953177+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:12.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938444+0000","last_change":"2024-04-26T21:38:17.938444+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938444+0000","last_undegraded":"2024-04-26T21:38:17.938444+0000","last_fullsized":"2024-04-26T21:38:17.938444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948659+0000","last_change":"2024-04-26T21:38:17.943246+0000","last_active":"2024-04-26T21:38:17.948659+0000","last_peered":"2024-04-26T21:38:17.948659+0000","last_clean":"2024-04-26T21:38:17.948659+0000","last_became_active":"2024-04-26T21:38:17.942975+0000","last_became_peered":"2024-04-26T21:38:17.942975+0000","last_unstale":"2024-04-26T21:38:17.948659+0000","last_undegraded":"2024-04-26T21:38:17.948659+0000","last_fullsized":"2024-04-26T21:38:17.948659+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:44.198607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940171+0000","last_change":"2024-04-26T21:38:17.940171+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940171+0000","last_undegraded":"2024-04-26T21:38:17.940171+0000","last_fullsized":"2024-04-26T21:38:17.940171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950719+0000","last_change":"2024-04-26T21:38:17.946995+0000","last_active":"2024-04-26T21:38:17.950719+0000","last_peered":"2024-04-26T21:38:17.950719+0000","last_clean":"2024-04-26T21:38:17.950719+0000","last_became_active":"2024-04-26T21:38:17.946697+0000","last_became_peered":"2024-04-26T21:38:17.946697+0000","last_unstale":"2024-04-26T21:38:17.950719+0000","last_undegraded":"2024-04-26T21:38:17.950719+0000","last_fullsized":"2024-04-26T21:38:17.950719+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:56.530385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934101+0000","last_change":"2024-04-26T21:38:17.934101+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934101+0000","last_undegraded":"2024-04-26T21:38:17.934101+0000","last_fullsized":"2024-04-26T21:38:17.934101+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945837+0000","last_change":"2024-04-26T21:38:17.942558+0000","last_active":"2024-04-26T21:38:17.945837+0000","last_peered":"2024-04-26T21:38:17.945837+0000","last_clean":"2024-04-26T21:38:17.945837+0000","last_became_active":"2024-04-26T21:38:17.942267+0000","last_became_peered":"2024-04-26T21:38:17.942267+0000","last_unstale":"2024-04-26T21:38:17.945837+0000","last_undegraded":"2024-04-26T21:38:17.945837+0000","last_fullsized":"2024-04-26T21:38:17.945837+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:00.927686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939224+0000","last_change":"2024-04-26T21:38:17.939224+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939224+0000","last_undegraded":"2024-04-26T21:38:17.939224+0000","last_fullsized":"2024-04-26T21:38:17.939224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950979+0000","last_change":"2024-04-26T21:38:17.948494+0000","last_active":"2024-04-26T21:38:17.950979+0000","last_peered":"2024-04-26T21:38:17.950979+0000","last_clean":"2024-04-26T21:38:17.950979+0000","last_became_active":"2024-04-26T21:38:17.948212+0000","last_became_peered":"2024-04-26T21:38:17.948212+0000","last_unstale":"2024-04-26T21:38:17.950979+0000","last_undegraded":"2024-04-26T21:38:17.950979+0000","last_fullsized":"2024-04-26T21:38:17.950979+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:00.507529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939650+0000","last_change":"2024-04-26T21:38:17.939650+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939650+0000","last_undegraded":"2024-04-26T21:38:17.939650+0000","last_fullsized":"2024-04-26T21:38:17.939650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.955821+0000","last_change":"2024-04-26T21:38:17.951721+0000","last_active":"2024-04-26T21:38:17.955821+0000","last_peered":"2024-04-26T21:38:17.955821+0000","last_clean":"2024-04-26T21:38:17.955821+0000","last_became_active":"2024-04-26T21:38:17.951400+0000","last_became_peered":"2024-04-26T21:38:17.951400+0000","last_unstale":"2024-04-26T21:38:17.955821+0000","last_undegraded":"2024-04-26T21:38:17.955821+0000","last_fullsized":"2024-04-26T21:38:17.955821+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:12.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939217+0000","last_change":"2024-04-26T21:38:17.939217+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939217+0000","last_undegraded":"2024-04-26T21:38:17.939217+0000","last_fullsized":"2024-04-26T21:38:17.939217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.947698+0000","last_change":"2024-04-26T21:38:17.945997+0000","last_active":"2024-04-26T21:38:17.947698+0000","last_peered":"2024-04-26T21:38:17.947698+0000","last_clean":"2024-04-26T21:38:17.947698+0000","last_became_active":"2024-04-26T21:38:17.945730+0000","last_became_peered":"2024-04-26T21:38:17.945730+0000","last_unstale":"2024-04-26T21:38:17.947698+0000","last_undegraded":"2024-04-26T21:38:17.947698+0000","last_fullsized":"2024-04-26T21:38:17.947698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:47.039536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934677+0000","last_change":"2024-04-26T21:38:17.934677+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934677+0000","last_undegraded":"2024-04-26T21:38:17.934677+0000","last_fullsized":"2024-04-26T21:38:17.934677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951528+0000","last_change":"2024-04-26T21:38:17.949610+0000","last_active":"2024-04-26T21:38:17.951528+0000","last_peered":"2024-04-26T21:38:17.951528+0000","last_clean":"2024-04-26T21:38:17.951528+0000","last_became_active":"2024-04-26T21:38:17.949247+0000","last_became_peered":"2024-04-26T21:38:17.949247+0000","last_unstale":"2024-04-26T21:38:17.951528+0000","last_undegraded":"2024-04-26T21:38:17.951528+0000","last_fullsized":"2024-04-26T21:38:17.951528+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:22.534915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938020+0000","last_change":"2024-04-26T21:38:17.938020+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938020+0000","last_undegraded":"2024-04-26T21:38:17.938020+0000","last_fullsized":"2024-04-26T21:38:17.938020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950521+0000","last_change":"2024-04-26T21:38:17.946463+0000","last_active":"2024-04-26T21:38:17.950521+0000","last_peered":"2024-04-26T21:38:17.950521+0000","last_clean":"2024-04-26T21:38:17.950521+0000","last_became_active":"2024-04-26T21:38:17.946182+0000","last_became_peered":"2024-04-26T21:38:17.946182+0000","last_unstale":"2024-04-26T21:38:17.950521+0000","last_undegraded":"2024-04-26T21:38:17.950521+0000","last_fullsized":"2024-04-26T21:38:17.950521+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:22.259886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941345+0000","last_change":"2024-04-26T21:38:17.941345+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941345+0000","last_undegraded":"2024-04-26T21:38:17.941345+0000","last_fullsized":"2024-04-26T21:38:17.941345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951656+0000","last_change":"2024-04-26T21:38:17.949728+0000","last_active":"2024-04-26T21:38:17.951656+0000","last_peered":"2024-04-26T21:38:17.951656+0000","last_clean":"2024-04-26T21:38:17.951656+0000","last_became_active":"2024-04-26T21:38:17.949450+0000","last_became_peered":"2024-04-26T21:38:17.949450+0000","last_unstale":"2024-04-26T21:38:17.951656+0000","last_undegraded":"2024-04-26T21:38:17.951656+0000","last_fullsized":"2024-04-26T21:38:17.951656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:00.927686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937164+0000","last_change":"2024-04-26T21:38:17.937164+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937164+0000","last_undegraded":"2024-04-26T21:38:17.937164+0000","last_fullsized":"2024-04-26T21:38:17.937164+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.940335+0000","last_change":"2024-04-26T21:38:17.937457+0000","last_active":"2024-04-26T21:38:17.940335+0000","last_peered":"2024-04-26T21:38:17.940335+0000","last_clean":"2024-04-26T21:38:17.940335+0000","last_became_active":"2024-04-26T21:38:17.937039+0000","last_became_peered":"2024-04-26T21:38:17.937039+0000","last_unstale":"2024-04-26T21:38:17.940335+0000","last_undegraded":"2024-04-26T21:38:17.940335+0000","last_fullsized":"2024-04-26T21:38:17.940335+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:46.534568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934742+0000","last_change":"2024-04-26T21:38:17.934742+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934742+0000","last_undegraded":"2024-04-26T21:38:17.934742+0000","last_fullsized":"2024-04-26T21:38:17.934742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948683+0000","last_change":"2024-04-26T21:38:17.944966+0000","last_active":"2024-04-26T21:38:17.948683+0000","last_peered":"2024-04-26T21:38:17.948683+0000","last_clean":"2024-04-26T21:38:17.948683+0000","last_became_active":"2024-04-26T21:38:17.944185+0000","last_became_peered":"2024-04-26T21:38:17.944185+0000","last_unstale":"2024-04-26T21:38:17.948683+0000","last_undegraded":"2024-04-26T21:38:17.948683+0000","last_fullsized":"2024-04-26T21:38:17.948683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:56.530385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938690+0000","last_change":"2024-04-26T21:38:17.938690+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938690+0000","last_undegraded":"2024-04-26T21:38:17.938690+0000","last_fullsized":"2024-04-26T21:38:17.938690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950383+0000","last_change":"2024-04-26T21:38:17.945734+0000","last_active":"2024-04-26T21:38:17.950383+0000","last_peered":"2024-04-26T21:38:17.950383+0000","last_clean":"2024-04-26T21:38:17.950383+0000","last_became_active":"2024-04-26T21:38:17.945450+0000","last_became_peered":"2024-04-26T21:38:17.945450+0000","last_unstale":"2024-04-26T21:38:17.950383+0000","last_undegraded":"2024-04-26T21:38:17.950383+0000","last_fullsized":"2024-04-26T21:38:17.950383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:35.855859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936991+0000","last_change":"2024-04-26T21:38:17.936991+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936991+0000","last_undegraded":"2024-04-26T21:38:17.936991+0000","last_fullsized":"2024-04-26T21:38:17.936991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948681+0000","last_change":"2024-04-26T21:38:17.944876+0000","last_active":"2024-04-26T21:38:17.948681+0000","last_peered":"2024-04-26T21:38:17.948681+0000","last_clean":"2024-04-26T21:38:17.948681+0000","last_became_active":"2024-04-26T21:38:17.944590+0000","last_became_peered":"2024-04-26T21:38:17.944590+0000","last_unstale":"2024-04-26T21:38:17.948681+0000","last_undegraded":"2024-04-26T21:38:17.948681+0000","last_fullsized":"2024-04-26T21:38:17.948681+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.686151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":251,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.938085+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:38:17.938085+0000","last_peered":"2024-04-26T21:38:17.938085+0000","last_clean":"2024-04-26T21:38:17.938085+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:38:17.938085+0000","last_undegraded":"2024-04-26T21:38:17.938085+0000","last_fullsized":"2024-04-26T21:38:17.938085+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343480,"statfs":{"total":96636764160,"available":96607723520,"internally_reserved":0,"allocated":1368064,"data_stored":941279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343480,"statfs":{"total":96636764160,"available":96607723520,"internally_reserved":0,"allocated":1368064,"data_stored":941279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:38:23.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:38:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.469+0000 7f1accf7f640 1 Processor -- start 2024-04-26T21:38:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.469+0000 7f1accf7f640 1 -- start start 2024-04-26T21:38:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.469+0000 7f1accf7f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac80d06f0 0x7f1ac80d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.469+0000 7f1accf7f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1ac80b2850 0x7f1ac80b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.469+0000 7f1accf7f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac80d0b70 con 0x7f1ac815f3a0 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.469+0000 7f1accf7f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac80d0e20 con 0x7f1ac80b2850 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.469+0000 7f1accf7f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac80b1d70 con 0x7f1ac80d06f0 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1ac659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1ac815f3a0 0x7f1ac80d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39746/0 (socket says 172.21.15.40:39746) 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1ac659c640 1 -- 172.21.15.40:0/2866258247 learned_addr learned my addr 172.21.15.40:0/2866258247 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1abffff640 1 --2- 172.21.15.40:0/2866258247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac80d06f0 0x7f1ac80d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1ac6d9d640 1 --2- 172.21.15.40:0/2866258247 >> v2:172.21.15.184:3300/0 conn(0x7f1ac80b2850 0x7f1ac80b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1ac5d9b640 1 -- 172.21.15.40:0/2866258247 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311612043 0 0) 0x7f1ac80d0b70 con 0x7f1ac815f3a0 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1ac5d9b640 1 -- 172.21.15.40:0/2866258247 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ab4003520 con 0x7f1ac815f3a0 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1abffff640 1 -- 172.21.15.40:0/2866258247 >> v2:172.21.15.184:3300/0 conn(0x7f1ac80b2850 msgr2=0x7f1ac80b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1abffff640 1 --2- 172.21.15.40:0/2866258247 >> v2:172.21.15.184:3300/0 conn(0x7f1ac80b2850 0x7f1ac80b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1abffff640 1 -- 172.21.15.40:0/2866258247 >> v1:172.21.15.40:6789/0 conn(0x7f1ac815f3a0 legacy=0x7f1ac80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1abffff640 1 -- 172.21.15.40:0/2866258247 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac80b2050 con 0x7f1ac80d06f0 2024-04-26T21:38:23.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.470+0000 7f1ac6d9d640 1 --2- 172.21.15.40:0/2866258247 >> v2:172.21.15.184:3300/0 conn(0x7f1ac80b2850 0x7f1ac80b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:23.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.471+0000 7f1abffff640 1 --2- 172.21.15.40:0/2866258247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac80d06f0 0x7f1ac80d1e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00097a0 tx=0x7f1ab002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2888101f579532d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.471+0000 7f1ac5d9b640 1 -- 172.21.15.40:0/2866258247 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0037070 con 0x7f1ac80d06f0 2024-04-26T21:38:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.471+0000 7f1ac5d9b640 1 -- 172.21.15.40:0/2866258247 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0032440 con 0x7f1ac80d06f0 2024-04-26T21:38:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.471+0000 7f1ac5d9b640 1 -- 172.21.15.40:0/2866258247 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00327f0 con 0x7f1ac80d06f0 2024-04-26T21:38:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.472+0000 7f1accf7f640 1 -- 172.21.15.40:0/2866258247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac80d06f0 msgr2=0x7f1ac80d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.472+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2866258247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac80d06f0 0x7f1ac80d1e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00097a0 tx=0x7f1ab002a180 comp rx=0 tx=0).stop 2024-04-26T21:38:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.472+0000 7f1accf7f640 1 -- 172.21.15.40:0/2866258247 shutdown_connections 2024-04-26T21:38:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.472+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2866258247 >> v2:172.21.15.184:3300/0 conn(0x7f1ac80b2850 0x7f1ac80b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.472+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2866258247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac80d06f0 0x7f1ac80d1e80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.472+0000 7f1accf7f640 1 -- 172.21.15.40:0/2866258247 >> 172.21.15.40:0/2866258247 conn(0x7f1ac80d4f90 msgr2=0x7f1ac80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.472+0000 7f1accf7f640 1 -- 172.21.15.40:0/2866258247 shutdown_connections 2024-04-26T21:38:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.472+0000 7f1accf7f640 1 -- 172.21.15.40:0/2866258247 wait complete. 2024-04-26T21:38:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.473+0000 7f1accf7f640 1 Processor -- start 2024-04-26T21:38:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.473+0000 7f1accf7f640 1 -- start start 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1accf7f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac80b2850 0x7f1ac81f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1accf7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac80d06f0 0x7f1ac8151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1accf7f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac815f3a0 0x7f1ac80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1accf7f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac815f900 con 0x7f1ac80d06f0 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1accf7f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac815f780 con 0x7f1ac80b2850 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1accf7f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac80d23c0 con 0x7f1ac815f3a0 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac80d06f0 0x7f1ac8151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac80d06f0 0x7f1ac8151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46870/0 (socket says 172.21.15.40:46870) 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 -- 172.21.15.40:0/2424339342 learned_addr learned my addr 172.21.15.40:0/2424339342 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1ac659c640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac80b2850 0x7f1ac81f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1ac6d9d640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac815f3a0 0x7f1ac80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 -- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac815f3a0 msgr2=0x7f1ac80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac815f3a0 0x7f1ac80b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 -- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac80b2850 msgr2=0x7f1ac81f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac80b2850 0x7f1ac81f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 -- 172.21.15.40:0/2424339342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac81f8b10 con 0x7f1ac80d06f0 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1ac6d9d640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac815f3a0 0x7f1ac80b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abffff640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac80d06f0 0x7f1ac8151610 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0035040 tx=0x7f1ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.474+0000 7f1abeffd640 1 -- 172.21.15.40:0/2424339342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0037040 con 0x7f1ac80d06f0 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.475+0000 7f1abeffd640 1 -- 172.21.15.40:0/2424339342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0042070 con 0x7f1ac80d06f0 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.475+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac81f8860 con 0x7f1ac80d06f0 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.475+0000 7f1abeffd640 1 -- 172.21.15.40:0/2424339342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab003b930 con 0x7f1ac80d06f0 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.475+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81f9090 con 0x7f1ac80d06f0 2024-04-26T21:38:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.476+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac80cd770 con 0x7f1ac80d06f0 2024-04-26T21:38:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.476+0000 7f1abeffd640 1 -- 172.21.15.40:0/2424339342 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ab003bad0 con 0x7f1ac80d06f0 2024-04-26T21:38:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.476+0000 7f1abeffd640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.476+0000 7f1abeffd640 1 -- 172.21.15.40:0/2424339342 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f1ab00ac490 con 0x7f1ac80d06f0 2024-04-26T21:38:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.477+0000 7f1ac659c640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.477+0000 7f1ac659c640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa806c8f0 0x7f1aa806edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005880 tx=0x7f1ab8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.481+0000 7f1abeffd640 1 -- 172.21.15.40:0/2424339342 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ab007a7c0 con 0x7f1ac80d06f0 2024-04-26T21:38:23.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.662+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ac80694f0 con 0x7f1aa806c8f0 2024-04-26T21:38:23.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.669+0000 7f1abeffd640 1 -- 172.21.15.40:0/2424339342 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246709 (secure 0 0 0) 0x7f1ac80694f0 con 0x7f1aa806c8f0 2024-04-26T21:38:23.672 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:23.674 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:38:23.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa806c8f0 msgr2=0x7f1aa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa806c8f0 0x7f1aa806edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005880 tx=0x7f1ab8009040 comp rx=0 tx=0).stop 2024-04-26T21:38:23.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac80d06f0 msgr2=0x7f1ac8151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac80d06f0 0x7f1ac8151610 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0035040 tx=0x7f1ab00046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 shutdown_connections 2024-04-26T21:38:23.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac815f3a0 0x7f1ac80b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac80d06f0 0x7f1ac8151610 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 --2- 172.21.15.40:0/2424339342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac80b2850 0x7f1ac81f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 >> 172.21.15.40:0/2424339342 conn(0x7f1ac80d4f90 msgr2=0x7f1ac806dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:23.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 shutdown_connections 2024-04-26T21:38:23.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.672+0000 7f1accf7f640 1 -- 172.21.15.40:0/2424339342 wait complete. 2024-04-26T21:38:23.684 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-04-26T21:38:22.039727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223304,"kb_used_data":7136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751416,"statfs":{"total":773094113280,"available":772865449984,"internally_reserved":0,"allocated":7307264,"data_stored":3948070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114996"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935415+0000","last_change":"2024-04-26T21:38:17.935415+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935415+0000","last_undegraded":"2024-04-26T21:38:17.935415+0000","last_fullsized":"2024-04-26T21:38:17.935415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933853+0000","last_change":"2024-04-26T21:38:17.933853+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933853+0000","last_undegraded":"2024-04-26T21:38:17.933853+0000","last_fullsized":"2024-04-26T21:38:17.933853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940393+0000","last_change":"2024-04-26T21:38:17.940393+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940393+0000","last_undegraded":"2024-04-26T21:38:17.940393+0000","last_fullsized":"2024-04-26T21:38:17.940393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939593+0000","last_change":"2024-04-26T21:38:17.939593+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939593+0000","last_undegraded":"2024-04-26T21:38:17.939593+0000","last_fullsized":"2024-04-26T21:38:17.939593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937487+0000","last_change":"2024-04-26T21:38:17.937487+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937487+0000","last_undegraded":"2024-04-26T21:38:17.937487+0000","last_fullsized":"2024-04-26T21:38:17.937487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940592+0000","last_change":"2024-04-26T21:38:17.940592+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940592+0000","last_undegraded":"2024-04-26T21:38:17.940592+0000","last_fullsized":"2024-04-26T21:38:17.940592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937934+0000","last_change":"2024-04-26T21:38:17.937934+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937934+0000","last_undegraded":"2024-04-26T21:38:17.937934+0000","last_fullsized":"2024-04-26T21:38:17.937934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934359+0000","last_change":"2024-04-26T21:38:17.934359+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934359+0000","last_undegraded":"2024-04-26T21:38:17.934359+0000","last_fullsized":"2024-04-26T21:38:17.934359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940540+0000","last_change":"2024-04-26T21:38:17.940540+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940540+0000","last_undegraded":"2024-04-26T21:38:17.940540+0000","last_fullsized":"2024-04-26T21:38:17.940540+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939851+0000","last_change":"2024-04-26T21:38:17.939851+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939851+0000","last_undegraded":"2024-04-26T21:38:17.939851+0000","last_fullsized":"2024-04-26T21:38:17.939851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939391+0000","last_change":"2024-04-26T21:38:17.939391+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939391+0000","last_undegraded":"2024-04-26T21:38:17.939391+0000","last_fullsized":"2024-04-26T21:38:17.939391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934346+0000","last_change":"2024-04-26T21:38:17.934346+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934346+0000","last_undegraded":"2024-04-26T21:38:17.934346+0000","last_fullsized":"2024-04-26T21:38:17.934346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936529+0000","last_change":"2024-04-26T21:38:17.936529+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936529+0000","last_undegraded":"2024-04-26T21:38:17.936529+0000","last_fullsized":"2024-04-26T21:38:17.936529+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938238+0000","last_change":"2024-04-26T21:38:17.938238+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938238+0000","last_undegraded":"2024-04-26T21:38:17.938238+0000","last_fullsized":"2024-04-26T21:38:17.938238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938428+0000","last_change":"2024-04-26T21:38:17.938428+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938428+0000","last_undegraded":"2024-04-26T21:38:17.938428+0000","last_fullsized":"2024-04-26T21:38:17.938428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938569+0000","last_change":"2024-04-26T21:38:17.938569+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938569+0000","last_undegraded":"2024-04-26T21:38:17.938569+0000","last_fullsized":"2024-04-26T21:38:17.938569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934251+0000","last_change":"2024-04-26T21:38:17.934251+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934251+0000","last_undegraded":"2024-04-26T21:38:17.934251+0000","last_fullsized":"2024-04-26T21:38:17.934251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938225+0000","last_change":"2024-04-26T21:38:17.938225+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938225+0000","last_undegraded":"2024-04-26T21:38:17.938225+0000","last_fullsized":"2024-04-26T21:38:17.938225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938791+0000","last_change":"2024-04-26T21:38:17.938791+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938791+0000","last_undegraded":"2024-04-26T21:38:17.938791+0000","last_fullsized":"2024-04-26T21:38:17.938791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934564+0000","last_change":"2024-04-26T21:38:17.934564+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934564+0000","last_undegraded":"2024-04-26T21:38:17.934564+0000","last_fullsized":"2024-04-26T21:38:17.934564+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934207+0000","last_change":"2024-04-26T21:38:17.934207+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934207+0000","last_undegraded":"2024-04-26T21:38:17.934207+0000","last_fullsized":"2024-04-26T21:38:17.934207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934474+0000","last_change":"2024-04-26T21:38:17.934474+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934474+0000","last_undegraded":"2024-04-26T21:38:17.934474+0000","last_fullsized":"2024-04-26T21:38:17.934474+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933962+0000","last_change":"2024-04-26T21:38:17.933962+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933962+0000","last_undegraded":"2024-04-26T21:38:17.933962+0000","last_fullsized":"2024-04-26T21:38:17.933962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937683+0000","last_change":"2024-04-26T21:38:17.937683+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937683+0000","last_undegraded":"2024-04-26T21:38:17.937683+0000","last_fullsized":"2024-04-26T21:38:17.937683+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934845+0000","last_change":"2024-04-26T21:38:17.934845+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934845+0000","last_undegraded":"2024-04-26T21:38:17.934845+0000","last_fullsized":"2024-04-26T21:38:17.934845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935655+0000","last_change":"2024-04-26T21:38:17.935655+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935655+0000","last_undegraded":"2024-04-26T21:38:17.935655+0000","last_fullsized":"2024-04-26T21:38:17.935655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939017+0000","last_change":"2024-04-26T21:38:17.939017+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939017+0000","last_undegraded":"2024-04-26T21:38:17.939017+0000","last_fullsized":"2024-04-26T21:38:17.939017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936461+0000","last_change":"2024-04-26T21:38:17.936461+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936461+0000","last_undegraded":"2024-04-26T21:38:17.936461+0000","last_fullsized":"2024-04-26T21:38:17.936461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938837+0000","last_change":"2024-04-26T21:38:17.938837+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938837+0000","last_undegraded":"2024-04-26T21:38:17.938837+0000","last_fullsized":"2024-04-26T21:38:17.938837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935425+0000","last_change":"2024-04-26T21:38:17.935425+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935425+0000","last_undegraded":"2024-04-26T21:38:17.935425+0000","last_fullsized":"2024-04-26T21:38:17.935425+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937570+0000","last_change":"2024-04-26T21:38:17.937570+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937570+0000","last_undegraded":"2024-04-26T21:38:17.937570+0000","last_fullsized":"2024-04-26T21:38:17.937570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941238+0000","last_change":"2024-04-26T21:38:17.941238+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941238+0000","last_undegraded":"2024-04-26T21:38:17.941238+0000","last_fullsized":"2024-04-26T21:38:17.941238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937920+0000","last_change":"2024-04-26T21:38:17.937920+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937920+0000","last_undegraded":"2024-04-26T21:38:17.937920+0000","last_fullsized":"2024-04-26T21:38:17.937920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950615+0000","last_change":"2024-04-26T21:38:17.948310+0000","last_active":"2024-04-26T21:38:17.950615+0000","last_peered":"2024-04-26T21:38:17.950615+0000","last_clean":"2024-04-26T21:38:17.950615+0000","last_became_active":"2024-04-26T21:38:17.948052+0000","last_became_peered":"2024-04-26T21:38:17.948052+0000","last_unstale":"2024-04-26T21:38:17.950615+0000","last_undegraded":"2024-04-26T21:38:17.950615+0000","last_fullsized":"2024-04-26T21:38:17.950615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:29.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938659+0000","last_change":"2024-04-26T21:38:17.938659+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938659+0000","last_undegraded":"2024-04-26T21:38:17.938659+0000","last_fullsized":"2024-04-26T21:38:17.938659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951738+0000","last_change":"2024-04-26T21:38:17.949729+0000","last_active":"2024-04-26T21:38:17.951738+0000","last_peered":"2024-04-26T21:38:17.951738+0000","last_clean":"2024-04-26T21:38:17.951738+0000","last_became_active":"2024-04-26T21:38:17.949412+0000","last_became_peered":"2024-04-26T21:38:17.949412+0000","last_unstale":"2024-04-26T21:38:17.951738+0000","last_undegraded":"2024-04-26T21:38:17.951738+0000","last_fullsized":"2024-04-26T21:38:17.951738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:20.434769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941869+0000","last_change":"2024-04-26T21:38:17.941869+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941869+0000","last_undegraded":"2024-04-26T21:38:17.941869+0000","last_fullsized":"2024-04-26T21:38:17.941869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950645+0000","last_change":"2024-04-26T21:38:17.947141+0000","last_active":"2024-04-26T21:38:17.950645+0000","last_peered":"2024-04-26T21:38:17.950645+0000","last_clean":"2024-04-26T21:38:17.950645+0000","last_became_active":"2024-04-26T21:38:17.946847+0000","last_became_peered":"2024-04-26T21:38:17.946847+0000","last_unstale":"2024-04-26T21:38:17.950645+0000","last_undegraded":"2024-04-26T21:38:17.950645+0000","last_fullsized":"2024-04-26T21:38:17.950645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.686151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940906+0000","last_change":"2024-04-26T21:38:17.940906+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940906+0000","last_undegraded":"2024-04-26T21:38:17.940906+0000","last_fullsized":"2024-04-26T21:38:17.940906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951770+0000","last_change":"2024-04-26T21:38:17.949992+0000","last_active":"2024-04-26T21:38:17.951770+0000","last_peered":"2024-04-26T21:38:17.951770+0000","last_clean":"2024-04-26T21:38:17.951770+0000","last_became_active":"2024-04-26T21:38:17.949701+0000","last_became_peered":"2024-04-26T21:38:17.949701+0000","last_unstale":"2024-04-26T21:38:17.951770+0000","last_undegraded":"2024-04-26T21:38:17.951770+0000","last_fullsized":"2024-04-26T21:38:17.951770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:12.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938112+0000","last_change":"2024-04-26T21:38:17.938112+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938112+0000","last_undegraded":"2024-04-26T21:38:17.938112+0000","last_fullsized":"2024-04-26T21:38:17.938112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950255+0000","last_change":"2024-04-26T21:38:17.947017+0000","last_active":"2024-04-26T21:38:17.950255+0000","last_peered":"2024-04-26T21:38:17.950255+0000","last_clean":"2024-04-26T21:38:17.950255+0000","last_became_active":"2024-04-26T21:38:17.946744+0000","last_became_peered":"2024-04-26T21:38:17.946744+0000","last_unstale":"2024-04-26T21:38:17.950255+0000","last_undegraded":"2024-04-26T21:38:17.950255+0000","last_fullsized":"2024-04-26T21:38:17.950255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:22.534915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933792+0000","last_change":"2024-04-26T21:38:17.933792+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933792+0000","last_undegraded":"2024-04-26T21:38:17.933792+0000","last_fullsized":"2024-04-26T21:38:17.933792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948630+0000","last_change":"2024-04-26T21:38:17.944477+0000","last_active":"2024-04-26T21:38:17.948630+0000","last_peered":"2024-04-26T21:38:17.948630+0000","last_clean":"2024-04-26T21:38:17.948630+0000","last_became_active":"2024-04-26T21:38:17.944149+0000","last_became_peered":"2024-04-26T21:38:17.944149+0000","last_unstale":"2024-04-26T21:38:17.948630+0000","last_undegraded":"2024-04-26T21:38:17.948630+0000","last_fullsized":"2024-04-26T21:38:17.948630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:03.182220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939989+0000","last_change":"2024-04-26T21:38:17.939989+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939989+0000","last_undegraded":"2024-04-26T21:38:17.939989+0000","last_fullsized":"2024-04-26T21:38:17.939989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948727+0000","last_change":"2024-04-26T21:38:17.945215+0000","last_active":"2024-04-26T21:38:17.948727+0000","last_peered":"2024-04-26T21:38:17.948727+0000","last_clean":"2024-04-26T21:38:17.948727+0000","last_became_active":"2024-04-26T21:38:17.944579+0000","last_became_peered":"2024-04-26T21:38:17.944579+0000","last_unstale":"2024-04-26T21:38:17.948727+0000","last_undegraded":"2024-04-26T21:38:17.948727+0000","last_fullsized":"2024-04-26T21:38:17.948727+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:00.507529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933926+0000","last_change":"2024-04-26T21:38:17.933926+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933926+0000","last_undegraded":"2024-04-26T21:38:17.933926+0000","last_fullsized":"2024-04-26T21:38:17.933926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952152+0000","last_change":"2024-04-26T21:38:17.949823+0000","last_active":"2024-04-26T21:38:17.952152+0000","last_peered":"2024-04-26T21:38:17.952152+0000","last_clean":"2024-04-26T21:38:17.952152+0000","last_became_active":"2024-04-26T21:38:17.949531+0000","last_became_peered":"2024-04-26T21:38:17.949531+0000","last_unstale":"2024-04-26T21:38:17.952152+0000","last_undegraded":"2024-04-26T21:38:17.952152+0000","last_fullsized":"2024-04-26T21:38:17.952152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:17.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935879+0000","last_change":"2024-04-26T21:38:17.935879+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935879+0000","last_undegraded":"2024-04-26T21:38:17.935879+0000","last_fullsized":"2024-04-26T21:38:17.935879+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953189+0000","last_change":"2024-04-26T21:38:17.951863+0000","last_active":"2024-04-26T21:38:17.953189+0000","last_peered":"2024-04-26T21:38:17.953189+0000","last_clean":"2024-04-26T21:38:17.953189+0000","last_became_active":"2024-04-26T21:38:17.951250+0000","last_became_peered":"2024-04-26T21:38:17.951250+0000","last_unstale":"2024-04-26T21:38:17.953189+0000","last_undegraded":"2024-04-26T21:38:17.953189+0000","last_fullsized":"2024-04-26T21:38:17.953189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:57.060894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933756+0000","last_change":"2024-04-26T21:38:17.933756+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933756+0000","last_undegraded":"2024-04-26T21:38:17.933756+0000","last_fullsized":"2024-04-26T21:38:17.933756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953111+0000","last_change":"2024-04-26T21:38:17.952295+0000","last_active":"2024-04-26T21:38:17.953111+0000","last_peered":"2024-04-26T21:38:17.953111+0000","last_clean":"2024-04-26T21:38:17.953111+0000","last_became_active":"2024-04-26T21:38:17.951993+0000","last_became_peered":"2024-04-26T21:38:17.951993+0000","last_unstale":"2024-04-26T21:38:17.953111+0000","last_undegraded":"2024-04-26T21:38:17.953111+0000","last_fullsized":"2024-04-26T21:38:17.953111+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:44:51.686328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938886+0000","last_change":"2024-04-26T21:38:17.938886+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938886+0000","last_undegraded":"2024-04-26T21:38:17.938886+0000","last_fullsized":"2024-04-26T21:38:17.938886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953151+0000","last_change":"2024-04-26T21:38:17.951334+0000","last_active":"2024-04-26T21:38:17.953151+0000","last_peered":"2024-04-26T21:38:17.953151+0000","last_clean":"2024-04-26T21:38:17.953151+0000","last_became_active":"2024-04-26T21:38:17.951044+0000","last_became_peered":"2024-04-26T21:38:17.951044+0000","last_unstale":"2024-04-26T21:38:17.953151+0000","last_undegraded":"2024-04-26T21:38:17.953151+0000","last_fullsized":"2024-04-26T21:38:17.953151+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:47.039536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939860+0000","last_change":"2024-04-26T21:38:17.939860+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939860+0000","last_undegraded":"2024-04-26T21:38:17.939860+0000","last_fullsized":"2024-04-26T21:38:17.939860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950495+0000","last_change":"2024-04-26T21:38:17.945882+0000","last_active":"2024-04-26T21:38:17.950495+0000","last_peered":"2024-04-26T21:38:17.950495+0000","last_clean":"2024-04-26T21:38:17.950495+0000","last_became_active":"2024-04-26T21:38:17.945594+0000","last_became_peered":"2024-04-26T21:38:17.945594+0000","last_unstale":"2024-04-26T21:38:17.950495+0000","last_undegraded":"2024-04-26T21:38:17.950495+0000","last_fullsized":"2024-04-26T21:38:17.950495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:50.308949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936947+0000","last_change":"2024-04-26T21:38:17.936947+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936947+0000","last_undegraded":"2024-04-26T21:38:17.936947+0000","last_fullsized":"2024-04-26T21:38:17.936947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948676+0000","last_change":"2024-04-26T21:38:17.947105+0000","last_active":"2024-04-26T21:38:17.948676+0000","last_peered":"2024-04-26T21:38:17.948676+0000","last_clean":"2024-04-26T21:38:17.948676+0000","last_became_active":"2024-04-26T21:38:17.946804+0000","last_became_peered":"2024-04-26T21:38:17.946804+0000","last_unstale":"2024-04-26T21:38:17.948676+0000","last_undegraded":"2024-04-26T21:38:17.948676+0000","last_fullsized":"2024-04-26T21:38:17.948676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:58.559420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.935577+0000","last_change":"2024-04-26T21:38:17.935577+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.935577+0000","last_undegraded":"2024-04-26T21:38:17.935577+0000","last_fullsized":"2024-04-26T21:38:17.935577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945985+0000","last_change":"2024-04-26T21:38:17.944988+0000","last_active":"2024-04-26T21:38:17.945985+0000","last_peered":"2024-04-26T21:38:17.945985+0000","last_clean":"2024-04-26T21:38:17.945985+0000","last_became_active":"2024-04-26T21:38:17.944724+0000","last_became_peered":"2024-04-26T21:38:17.944724+0000","last_unstale":"2024-04-26T21:38:17.945985+0000","last_undegraded":"2024-04-26T21:38:17.945985+0000","last_fullsized":"2024-04-26T21:38:17.945985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939621+0000","last_change":"2024-04-26T21:38:17.939621+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939621+0000","last_undegraded":"2024-04-26T21:38:17.939621+0000","last_fullsized":"2024-04-26T21:38:17.939621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950969+0000","last_change":"2024-04-26T21:38:17.947884+0000","last_active":"2024-04-26T21:38:17.950969+0000","last_peered":"2024-04-26T21:38:17.950969+0000","last_clean":"2024-04-26T21:38:17.950969+0000","last_became_active":"2024-04-26T21:38:17.947599+0000","last_became_peered":"2024-04-26T21:38:17.947599+0000","last_unstale":"2024-04-26T21:38:17.950969+0000","last_undegraded":"2024-04-26T21:38:17.950969+0000","last_fullsized":"2024-04-26T21:38:17.950969+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:02.996302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.933961+0000","last_change":"2024-04-26T21:38:17.933961+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.933961+0000","last_undegraded":"2024-04-26T21:38:17.933961+0000","last_fullsized":"2024-04-26T21:38:17.933961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952474+0000","last_change":"2024-04-26T21:38:17.948438+0000","last_active":"2024-04-26T21:38:17.952474+0000","last_peered":"2024-04-26T21:38:17.952474+0000","last_clean":"2024-04-26T21:38:17.952474+0000","last_became_active":"2024-04-26T21:38:17.948196+0000","last_became_peered":"2024-04-26T21:38:17.948196+0000","last_unstale":"2024-04-26T21:38:17.952474+0000","last_undegraded":"2024-04-26T21:38:17.952474+0000","last_fullsized":"2024-04-26T21:38:17.952474+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:32.653604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937129+0000","last_change":"2024-04-26T21:38:17.937129+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937129+0000","last_undegraded":"2024-04-26T21:38:17.937129+0000","last_fullsized":"2024-04-26T21:38:17.937129+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945381+0000","last_change":"2024-04-26T21:38:17.942502+0000","last_active":"2024-04-26T21:38:17.945381+0000","last_peered":"2024-04-26T21:38:17.945381+0000","last_clean":"2024-04-26T21:38:17.945381+0000","last_became_active":"2024-04-26T21:38:17.942225+0000","last_became_peered":"2024-04-26T21:38:17.942225+0000","last_unstale":"2024-04-26T21:38:17.945381+0000","last_undegraded":"2024-04-26T21:38:17.945381+0000","last_fullsized":"2024-04-26T21:38:17.945381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:02.996302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938374+0000","last_change":"2024-04-26T21:38:17.938374+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938374+0000","last_undegraded":"2024-04-26T21:38:17.938374+0000","last_fullsized":"2024-04-26T21:38:17.938374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.952434+0000","last_change":"2024-04-26T21:38:17.949980+0000","last_active":"2024-04-26T21:38:17.952434+0000","last_peered":"2024-04-26T21:38:17.952434+0000","last_clean":"2024-04-26T21:38:17.952434+0000","last_became_active":"2024-04-26T21:38:17.949710+0000","last_became_peered":"2024-04-26T21:38:17.949710+0000","last_unstale":"2024-04-26T21:38:17.952434+0000","last_undegraded":"2024-04-26T21:38:17.952434+0000","last_fullsized":"2024-04-26T21:38:17.952434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937829+0000","last_change":"2024-04-26T21:38:17.937829+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937829+0000","last_undegraded":"2024-04-26T21:38:17.937829+0000","last_fullsized":"2024-04-26T21:38:17.937829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.953177+0000","last_change":"2024-04-26T21:38:17.951499+0000","last_active":"2024-04-26T21:38:17.953177+0000","last_peered":"2024-04-26T21:38:17.953177+0000","last_clean":"2024-04-26T21:38:17.953177+0000","last_became_active":"2024-04-26T21:38:17.951190+0000","last_became_peered":"2024-04-26T21:38:17.951190+0000","last_unstale":"2024-04-26T21:38:17.953177+0000","last_undegraded":"2024-04-26T21:38:17.953177+0000","last_fullsized":"2024-04-26T21:38:17.953177+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:12.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938444+0000","last_change":"2024-04-26T21:38:17.938444+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938444+0000","last_undegraded":"2024-04-26T21:38:17.938444+0000","last_fullsized":"2024-04-26T21:38:17.938444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948659+0000","last_change":"2024-04-26T21:38:17.943246+0000","last_active":"2024-04-26T21:38:17.948659+0000","last_peered":"2024-04-26T21:38:17.948659+0000","last_clean":"2024-04-26T21:38:17.948659+0000","last_became_active":"2024-04-26T21:38:17.942975+0000","last_became_peered":"2024-04-26T21:38:17.942975+0000","last_unstale":"2024-04-26T21:38:17.948659+0000","last_undegraded":"2024-04-26T21:38:17.948659+0000","last_fullsized":"2024-04-26T21:38:17.948659+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:44.198607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.940171+0000","last_change":"2024-04-26T21:38:17.940171+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.940171+0000","last_undegraded":"2024-04-26T21:38:17.940171+0000","last_fullsized":"2024-04-26T21:38:17.940171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950719+0000","last_change":"2024-04-26T21:38:17.946995+0000","last_active":"2024-04-26T21:38:17.950719+0000","last_peered":"2024-04-26T21:38:17.950719+0000","last_clean":"2024-04-26T21:38:17.950719+0000","last_became_active":"2024-04-26T21:38:17.946697+0000","last_became_peered":"2024-04-26T21:38:17.946697+0000","last_unstale":"2024-04-26T21:38:17.950719+0000","last_undegraded":"2024-04-26T21:38:17.950719+0000","last_fullsized":"2024-04-26T21:38:17.950719+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:56.530385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934101+0000","last_change":"2024-04-26T21:38:17.934101+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934101+0000","last_undegraded":"2024-04-26T21:38:17.934101+0000","last_fullsized":"2024-04-26T21:38:17.934101+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.945837+0000","last_change":"2024-04-26T21:38:17.942558+0000","last_active":"2024-04-26T21:38:17.945837+0000","last_peered":"2024-04-26T21:38:17.945837+0000","last_clean":"2024-04-26T21:38:17.945837+0000","last_became_active":"2024-04-26T21:38:17.942267+0000","last_became_peered":"2024-04-26T21:38:17.942267+0000","last_unstale":"2024-04-26T21:38:17.945837+0000","last_undegraded":"2024-04-26T21:38:17.945837+0000","last_fullsized":"2024-04-26T21:38:17.945837+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:00.927686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939224+0000","last_change":"2024-04-26T21:38:17.939224+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939224+0000","last_undegraded":"2024-04-26T21:38:17.939224+0000","last_fullsized":"2024-04-26T21:38:17.939224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950979+0000","last_change":"2024-04-26T21:38:17.948494+0000","last_active":"2024-04-26T21:38:17.950979+0000","last_peered":"2024-04-26T21:38:17.950979+0000","last_clean":"2024-04-26T21:38:17.950979+0000","last_became_active":"2024-04-26T21:38:17.948212+0000","last_became_peered":"2024-04-26T21:38:17.948212+0000","last_unstale":"2024-04-26T21:38:17.950979+0000","last_undegraded":"2024-04-26T21:38:17.950979+0000","last_fullsized":"2024-04-26T21:38:17.950979+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:00.507529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939650+0000","last_change":"2024-04-26T21:38:17.939650+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939650+0000","last_undegraded":"2024-04-26T21:38:17.939650+0000","last_fullsized":"2024-04-26T21:38:17.939650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.955821+0000","last_change":"2024-04-26T21:38:17.951721+0000","last_active":"2024-04-26T21:38:17.955821+0000","last_peered":"2024-04-26T21:38:17.955821+0000","last_clean":"2024-04-26T21:38:17.955821+0000","last_became_active":"2024-04-26T21:38:17.951400+0000","last_became_peered":"2024-04-26T21:38:17.951400+0000","last_unstale":"2024-04-26T21:38:17.955821+0000","last_undegraded":"2024-04-26T21:38:17.955821+0000","last_fullsized":"2024-04-26T21:38:17.955821+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:12.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.939217+0000","last_change":"2024-04-26T21:38:17.939217+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.939217+0000","last_undegraded":"2024-04-26T21:38:17.939217+0000","last_fullsized":"2024-04-26T21:38:17.939217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.947698+0000","last_change":"2024-04-26T21:38:17.945997+0000","last_active":"2024-04-26T21:38:17.947698+0000","last_peered":"2024-04-26T21:38:17.947698+0000","last_clean":"2024-04-26T21:38:17.947698+0000","last_became_active":"2024-04-26T21:38:17.945730+0000","last_became_peered":"2024-04-26T21:38:17.945730+0000","last_unstale":"2024-04-26T21:38:17.947698+0000","last_undegraded":"2024-04-26T21:38:17.947698+0000","last_fullsized":"2024-04-26T21:38:17.947698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:47.039536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934677+0000","last_change":"2024-04-26T21:38:17.934677+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934677+0000","last_undegraded":"2024-04-26T21:38:17.934677+0000","last_fullsized":"2024-04-26T21:38:17.934677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951528+0000","last_change":"2024-04-26T21:38:17.949610+0000","last_active":"2024-04-26T21:38:17.951528+0000","last_peered":"2024-04-26T21:38:17.951528+0000","last_clean":"2024-04-26T21:38:17.951528+0000","last_became_active":"2024-04-26T21:38:17.949247+0000","last_became_peered":"2024-04-26T21:38:17.949247+0000","last_unstale":"2024-04-26T21:38:17.951528+0000","last_undegraded":"2024-04-26T21:38:17.951528+0000","last_fullsized":"2024-04-26T21:38:17.951528+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:22.534915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938020+0000","last_change":"2024-04-26T21:38:17.938020+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938020+0000","last_undegraded":"2024-04-26T21:38:17.938020+0000","last_fullsized":"2024-04-26T21:38:17.938020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950521+0000","last_change":"2024-04-26T21:38:17.946463+0000","last_active":"2024-04-26T21:38:17.950521+0000","last_peered":"2024-04-26T21:38:17.950521+0000","last_clean":"2024-04-26T21:38:17.950521+0000","last_became_active":"2024-04-26T21:38:17.946182+0000","last_became_peered":"2024-04-26T21:38:17.946182+0000","last_unstale":"2024-04-26T21:38:17.950521+0000","last_undegraded":"2024-04-26T21:38:17.950521+0000","last_fullsized":"2024-04-26T21:38:17.950521+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:22.259886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.941345+0000","last_change":"2024-04-26T21:38:17.941345+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.941345+0000","last_undegraded":"2024-04-26T21:38:17.941345+0000","last_fullsized":"2024-04-26T21:38:17.941345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.951656+0000","last_change":"2024-04-26T21:38:17.949728+0000","last_active":"2024-04-26T21:38:17.951656+0000","last_peered":"2024-04-26T21:38:17.951656+0000","last_clean":"2024-04-26T21:38:17.951656+0000","last_became_active":"2024-04-26T21:38:17.949450+0000","last_became_peered":"2024-04-26T21:38:17.949450+0000","last_unstale":"2024-04-26T21:38:17.951656+0000","last_undegraded":"2024-04-26T21:38:17.951656+0000","last_fullsized":"2024-04-26T21:38:17.951656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:00.927686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.937164+0000","last_change":"2024-04-26T21:38:17.937164+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.937164+0000","last_undegraded":"2024-04-26T21:38:17.937164+0000","last_fullsized":"2024-04-26T21:38:17.937164+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.940335+0000","last_change":"2024-04-26T21:38:17.937457+0000","last_active":"2024-04-26T21:38:17.940335+0000","last_peered":"2024-04-26T21:38:17.940335+0000","last_clean":"2024-04-26T21:38:17.940335+0000","last_became_active":"2024-04-26T21:38:17.937039+0000","last_became_peered":"2024-04-26T21:38:17.937039+0000","last_unstale":"2024-04-26T21:38:17.940335+0000","last_undegraded":"2024-04-26T21:38:17.940335+0000","last_fullsized":"2024-04-26T21:38:17.940335+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:46.534568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.934742+0000","last_change":"2024-04-26T21:38:17.934742+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.934742+0000","last_undegraded":"2024-04-26T21:38:17.934742+0000","last_fullsized":"2024-04-26T21:38:17.934742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948683+0000","last_change":"2024-04-26T21:38:17.944966+0000","last_active":"2024-04-26T21:38:17.948683+0000","last_peered":"2024-04-26T21:38:17.948683+0000","last_clean":"2024-04-26T21:38:17.948683+0000","last_became_active":"2024-04-26T21:38:17.944185+0000","last_became_peered":"2024-04-26T21:38:17.944185+0000","last_unstale":"2024-04-26T21:38:17.948683+0000","last_undegraded":"2024-04-26T21:38:17.948683+0000","last_fullsized":"2024-04-26T21:38:17.948683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:56.530385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.938690+0000","last_change":"2024-04-26T21:38:17.938690+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.938690+0000","last_undegraded":"2024-04-26T21:38:17.938690+0000","last_fullsized":"2024-04-26T21:38:17.938690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.950383+0000","last_change":"2024-04-26T21:38:17.945734+0000","last_active":"2024-04-26T21:38:17.950383+0000","last_peered":"2024-04-26T21:38:17.950383+0000","last_clean":"2024-04-26T21:38:17.950383+0000","last_became_active":"2024-04-26T21:38:17.945450+0000","last_became_peered":"2024-04-26T21:38:17.945450+0000","last_unstale":"2024-04-26T21:38:17.950383+0000","last_undegraded":"2024-04-26T21:38:17.950383+0000","last_fullsized":"2024-04-26T21:38:17.950383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:35.855859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-04-26T21:38:17.936991+0000","last_change":"2024-04-26T21:38:17.936991+0000","last_active":"2024-04-26T21:38:17.921449+0000","last_peered":"2024-04-26T21:38:17.921449+0000","last_clean":"2024-04-26T21:38:17.921449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:38:17.936991+0000","last_undegraded":"2024-04-26T21:38:17.936991+0000","last_fullsized":"2024-04-26T21:38:17.936991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.948681+0000","last_change":"2024-04-26T21:38:17.944876+0000","last_active":"2024-04-26T21:38:17.948681+0000","last_peered":"2024-04-26T21:38:17.948681+0000","last_clean":"2024-04-26T21:38:17.948681+0000","last_became_active":"2024-04-26T21:38:17.944590+0000","last_became_peered":"2024-04-26T21:38:17.944590+0000","last_unstale":"2024-04-26T21:38:17.948681+0000","last_undegraded":"2024-04-26T21:38:17.948681+0000","last_fullsized":"2024-04-26T21:38:17.948681+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:46.686151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":251,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T21:38:17.938085+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:38:17.938085+0000","last_peered":"2024-04-26T21:38:17.938085+0000","last_clean":"2024-04-26T21:38:17.938085+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:38:17.938085+0000","last_undegraded":"2024-04-26T21:38:17.938085+0000","last_fullsized":"2024-04-26T21:38:17.938085+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343480,"statfs":{"total":96636764160,"available":96607723520,"internally_reserved":0,"allocated":1368064,"data_stored":941279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":344252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343480,"statfs":{"total":96636764160,"available":96607723520,"internally_reserved":0,"allocated":1368064,"data_stored":941279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:38:23.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2024-04-26T21:38:23.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.935415+0000', 'last_change': '2024-04-26T21:38:17.935415+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.935415+0000', 'last_undegraded': '2024-04-26T21:38:17.935415+0000', 'last_fullsized': '2024-04-26T21:38:17.935415+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:38:23.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2024-04-26T21:38:23.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.933853+0000', 'last_change': '2024-04-26T21:38:17.933853+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.933853+0000', 'last_undegraded': '2024-04-26T21:38:17.933853+0000', 'last_fullsized': '2024-04-26T21:38:17.933853+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:38:23.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-04-26T21:38:23.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.940393+0000', 'last_change': '2024-04-26T21:38:17.940393+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.940393+0000', 'last_undegraded': '2024-04-26T21:38:17.940393+0000', 'last_fullsized': '2024-04-26T21:38:17.940393+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939593+0000', 'last_change': '2024-04-26T21:38:17.939593+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939593+0000', 'last_undegraded': '2024-04-26T21:38:17.939593+0000', 'last_fullsized': '2024-04-26T21:38:17.939593+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.937487+0000', 'last_change': '2024-04-26T21:38:17.937487+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.937487+0000', 'last_undegraded': '2024-04-26T21:38:17.937487+0000', 'last_fullsized': '2024-04-26T21:38:17.937487+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.940592+0000', 'last_change': '2024-04-26T21:38:17.940592+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.940592+0000', 'last_undegraded': '2024-04-26T21:38:17.940592+0000', 'last_fullsized': '2024-04-26T21:38:17.940592+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.937934+0000', 'last_change': '2024-04-26T21:38:17.937934+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.937934+0000', 'last_undegraded': '2024-04-26T21:38:17.937934+0000', 'last_fullsized': '2024-04-26T21:38:17.937934+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934359+0000', 'last_change': '2024-04-26T21:38:17.934359+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934359+0000', 'last_undegraded': '2024-04-26T21:38:17.934359+0000', 'last_fullsized': '2024-04-26T21:38:17.934359+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-04-26T21:38:23.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.940540+0000', 'last_change': '2024-04-26T21:38:17.940540+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.940540+0000', 'last_undegraded': '2024-04-26T21:38:17.940540+0000', 'last_fullsized': '2024-04-26T21:38:17.940540+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939851+0000', 'last_change': '2024-04-26T21:38:17.939851+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939851+0000', 'last_undegraded': '2024-04-26T21:38:17.939851+0000', 'last_fullsized': '2024-04-26T21:38:17.939851+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939391+0000', 'last_change': '2024-04-26T21:38:17.939391+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939391+0000', 'last_undegraded': '2024-04-26T21:38:17.939391+0000', 'last_fullsized': '2024-04-26T21:38:17.939391+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934346+0000', 'last_change': '2024-04-26T21:38:17.934346+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934346+0000', 'last_undegraded': '2024-04-26T21:38:17.934346+0000', 'last_fullsized': '2024-04-26T21:38:17.934346+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.936529+0000', 'last_change': '2024-04-26T21:38:17.936529+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.936529+0000', 'last_undegraded': '2024-04-26T21:38:17.936529+0000', 'last_fullsized': '2024-04-26T21:38:17.936529+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938238+0000', 'last_change': '2024-04-26T21:38:17.938238+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938238+0000', 'last_undegraded': '2024-04-26T21:38:17.938238+0000', 'last_fullsized': '2024-04-26T21:38:17.938238+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938428+0000', 'last_change': '2024-04-26T21:38:17.938428+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938428+0000', 'last_undegraded': '2024-04-26T21:38:17.938428+0000', 'last_fullsized': '2024-04-26T21:38:17.938428+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938569+0000', 'last_change': '2024-04-26T21:38:17.938569+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938569+0000', 'last_undegraded': '2024-04-26T21:38:17.938569+0000', 'last_fullsized': '2024-04-26T21:38:17.938569+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934251+0000', 'last_change': '2024-04-26T21:38:17.934251+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934251+0000', 'last_undegraded': '2024-04-26T21:38:17.934251+0000', 'last_fullsized': '2024-04-26T21:38:17.934251+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938225+0000', 'last_change': '2024-04-26T21:38:17.938225+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938225+0000', 'last_undegraded': '2024-04-26T21:38:17.938225+0000', 'last_fullsized': '2024-04-26T21:38:17.938225+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-04-26T21:38:23.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938791+0000', 'last_change': '2024-04-26T21:38:17.938791+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938791+0000', 'last_undegraded': '2024-04-26T21:38:17.938791+0000', 'last_fullsized': '2024-04-26T21:38:17.938791+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934564+0000', 'last_change': '2024-04-26T21:38:17.934564+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934564+0000', 'last_undegraded': '2024-04-26T21:38:17.934564+0000', 'last_fullsized': '2024-04-26T21:38:17.934564+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934207+0000', 'last_change': '2024-04-26T21:38:17.934207+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934207+0000', 'last_undegraded': '2024-04-26T21:38:17.934207+0000', 'last_fullsized': '2024-04-26T21:38:17.934207+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934474+0000', 'last_change': '2024-04-26T21:38:17.934474+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934474+0000', 'last_undegraded': '2024-04-26T21:38:17.934474+0000', 'last_fullsized': '2024-04-26T21:38:17.934474+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.933962+0000', 'last_change': '2024-04-26T21:38:17.933962+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.933962+0000', 'last_undegraded': '2024-04-26T21:38:17.933962+0000', 'last_fullsized': '2024-04-26T21:38:17.933962+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.937683+0000', 'last_change': '2024-04-26T21:38:17.937683+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.937683+0000', 'last_undegraded': '2024-04-26T21:38:17.937683+0000', 'last_fullsized': '2024-04-26T21:38:17.937683+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934845+0000', 'last_change': '2024-04-26T21:38:17.934845+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934845+0000', 'last_undegraded': '2024-04-26T21:38:17.934845+0000', 'last_fullsized': '2024-04-26T21:38:17.934845+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.935655+0000', 'last_change': '2024-04-26T21:38:17.935655+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.935655+0000', 'last_undegraded': '2024-04-26T21:38:17.935655+0000', 'last_fullsized': '2024-04-26T21:38:17.935655+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939017+0000', 'last_change': '2024-04-26T21:38:17.939017+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939017+0000', 'last_undegraded': '2024-04-26T21:38:17.939017+0000', 'last_fullsized': '2024-04-26T21:38:17.939017+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.936461+0000', 'last_change': '2024-04-26T21:38:17.936461+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.936461+0000', 'last_undegraded': '2024-04-26T21:38:17.936461+0000', 'last_fullsized': '2024-04-26T21:38:17.936461+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938837+0000', 'last_change': '2024-04-26T21:38:17.938837+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938837+0000', 'last_undegraded': '2024-04-26T21:38:17.938837+0000', 'last_fullsized': '2024-04-26T21:38:17.938837+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-04-26T21:38:23.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.935425+0000', 'last_change': '2024-04-26T21:38:17.935425+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.935425+0000', 'last_undegraded': '2024-04-26T21:38:17.935425+0000', 'last_fullsized': '2024-04-26T21:38:17.935425+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.937570+0000', 'last_change': '2024-04-26T21:38:17.937570+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.937570+0000', 'last_undegraded': '2024-04-26T21:38:17.937570+0000', 'last_fullsized': '2024-04-26T21:38:17.937570+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.941238+0000', 'last_change': '2024-04-26T21:38:17.941238+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.941238+0000', 'last_undegraded': '2024-04-26T21:38:17.941238+0000', 'last_fullsized': '2024-04-26T21:38:17.941238+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.937920+0000', 'last_change': '2024-04-26T21:38:17.937920+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.937920+0000', 'last_undegraded': '2024-04-26T21:38:17.937920+0000', 'last_fullsized': '2024-04-26T21:38:17.937920+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938659+0000', 'last_change': '2024-04-26T21:38:17.938659+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938659+0000', 'last_undegraded': '2024-04-26T21:38:17.938659+0000', 'last_fullsized': '2024-04-26T21:38:17.938659+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-04-26T21:38:23.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.941869+0000', 'last_change': '2024-04-26T21:38:17.941869+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.941869+0000', 'last_undegraded': '2024-04-26T21:38:17.941869+0000', 'last_fullsized': '2024-04-26T21:38:17.941869+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.940906+0000', 'last_change': '2024-04-26T21:38:17.940906+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.940906+0000', 'last_undegraded': '2024-04-26T21:38:17.940906+0000', 'last_fullsized': '2024-04-26T21:38:17.940906+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938112+0000', 'last_change': '2024-04-26T21:38:17.938112+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938112+0000', 'last_undegraded': '2024-04-26T21:38:17.938112+0000', 'last_fullsized': '2024-04-26T21:38:17.938112+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.933792+0000', 'last_change': '2024-04-26T21:38:17.933792+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.933792+0000', 'last_undegraded': '2024-04-26T21:38:17.933792+0000', 'last_fullsized': '2024-04-26T21:38:17.933792+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939989+0000', 'last_change': '2024-04-26T21:38:17.939989+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939989+0000', 'last_undegraded': '2024-04-26T21:38:17.939989+0000', 'last_fullsized': '2024-04-26T21:38:17.939989+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-04-26T21:38:23.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.933926+0000', 'last_change': '2024-04-26T21:38:17.933926+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.933926+0000', 'last_undegraded': '2024-04-26T21:38:17.933926+0000', 'last_fullsized': '2024-04-26T21:38:17.933926+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.935879+0000', 'last_change': '2024-04-26T21:38:17.935879+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.935879+0000', 'last_undegraded': '2024-04-26T21:38:17.935879+0000', 'last_fullsized': '2024-04-26T21:38:17.935879+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.933756+0000', 'last_change': '2024-04-26T21:38:17.933756+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.933756+0000', 'last_undegraded': '2024-04-26T21:38:17.933756+0000', 'last_fullsized': '2024-04-26T21:38:17.933756+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938886+0000', 'last_change': '2024-04-26T21:38:17.938886+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938886+0000', 'last_undegraded': '2024-04-26T21:38:17.938886+0000', 'last_fullsized': '2024-04-26T21:38:17.938886+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939860+0000', 'last_change': '2024-04-26T21:38:17.939860+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939860+0000', 'last_undegraded': '2024-04-26T21:38:17.939860+0000', 'last_fullsized': '2024-04-26T21:38:17.939860+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-04-26T21:38:23.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.936947+0000', 'last_change': '2024-04-26T21:38:17.936947+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.936947+0000', 'last_undegraded': '2024-04-26T21:38:17.936947+0000', 'last_fullsized': '2024-04-26T21:38:17.936947+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.935577+0000', 'last_change': '2024-04-26T21:38:17.935577+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.935577+0000', 'last_undegraded': '2024-04-26T21:38:17.935577+0000', 'last_fullsized': '2024-04-26T21:38:17.935577+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939621+0000', 'last_change': '2024-04-26T21:38:17.939621+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939621+0000', 'last_undegraded': '2024-04-26T21:38:17.939621+0000', 'last_fullsized': '2024-04-26T21:38:17.939621+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.933961+0000', 'last_change': '2024-04-26T21:38:17.933961+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.933961+0000', 'last_undegraded': '2024-04-26T21:38:17.933961+0000', 'last_fullsized': '2024-04-26T21:38:17.933961+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.937129+0000', 'last_change': '2024-04-26T21:38:17.937129+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.937129+0000', 'last_undegraded': '2024-04-26T21:38:17.937129+0000', 'last_fullsized': '2024-04-26T21:38:17.937129+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938374+0000', 'last_change': '2024-04-26T21:38:17.938374+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938374+0000', 'last_undegraded': '2024-04-26T21:38:17.938374+0000', 'last_fullsized': '2024-04-26T21:38:17.938374+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-04-26T21:38:23.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.937829+0000', 'last_change': '2024-04-26T21:38:17.937829+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.937829+0000', 'last_undegraded': '2024-04-26T21:38:17.937829+0000', 'last_fullsized': '2024-04-26T21:38:17.937829+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938444+0000', 'last_change': '2024-04-26T21:38:17.938444+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938444+0000', 'last_undegraded': '2024-04-26T21:38:17.938444+0000', 'last_fullsized': '2024-04-26T21:38:17.938444+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.940171+0000', 'last_change': '2024-04-26T21:38:17.940171+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.940171+0000', 'last_undegraded': '2024-04-26T21:38:17.940171+0000', 'last_fullsized': '2024-04-26T21:38:17.940171+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934101+0000', 'last_change': '2024-04-26T21:38:17.934101+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934101+0000', 'last_undegraded': '2024-04-26T21:38:17.934101+0000', 'last_fullsized': '2024-04-26T21:38:17.934101+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939224+0000', 'last_change': '2024-04-26T21:38:17.939224+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939224+0000', 'last_undegraded': '2024-04-26T21:38:17.939224+0000', 'last_fullsized': '2024-04-26T21:38:17.939224+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-04-26T21:38:23.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939650+0000', 'last_change': '2024-04-26T21:38:17.939650+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939650+0000', 'last_undegraded': '2024-04-26T21:38:17.939650+0000', 'last_fullsized': '2024-04-26T21:38:17.939650+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.939217+0000', 'last_change': '2024-04-26T21:38:17.939217+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.939217+0000', 'last_undegraded': '2024-04-26T21:38:17.939217+0000', 'last_fullsized': '2024-04-26T21:38:17.939217+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934677+0000', 'last_change': '2024-04-26T21:38:17.934677+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934677+0000', 'last_undegraded': '2024-04-26T21:38:17.934677+0000', 'last_fullsized': '2024-04-26T21:38:17.934677+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938020+0000', 'last_change': '2024-04-26T21:38:17.938020+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938020+0000', 'last_undegraded': '2024-04-26T21:38:17.938020+0000', 'last_fullsized': '2024-04-26T21:38:17.938020+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.941345+0000', 'last_change': '2024-04-26T21:38:17.941345+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.941345+0000', 'last_undegraded': '2024-04-26T21:38:17.941345+0000', 'last_fullsized': '2024-04-26T21:38:17.941345+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.937164+0000', 'last_change': '2024-04-26T21:38:17.937164+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.937164+0000', 'last_undegraded': '2024-04-26T21:38:17.937164+0000', 'last_fullsized': '2024-04-26T21:38:17.937164+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:38:23.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-04-26T21:38:23.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.934742+0000', 'last_change': '2024-04-26T21:38:17.934742+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.934742+0000', 'last_undegraded': '2024-04-26T21:38:17.934742+0000', 'last_fullsized': '2024-04-26T21:38:17.934742+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:38:23.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-04-26T21:38:23.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.938690+0000', 'last_change': '2024-04-26T21:38:17.938690+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.938690+0000', 'last_undegraded': '2024-04-26T21:38:17.938690+0000', 'last_fullsized': '2024-04-26T21:38:17.938690+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:38:23.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-04-26T21:38:23.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:38:17.936991+0000', 'last_change': '2024-04-26T21:38:17.936991+0000', 'last_active': '2024-04-26T21:38:17.921449+0000', 'last_peered': '2024-04-26T21:38:17.921449+0000', 'last_clean': '2024-04-26T21:38:17.921449+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:38:17.936991+0000', 'last_undegraded': '2024-04-26T21:38:17.936991+0000', 'last_fullsized': '2024-04-26T21:38:17.936991+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'last_clean_scrub_stamp': '2024-04-26T21:38:17.921449+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:38:23.700 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:38:23.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.823+0000 7f41dad39640 1 Processor -- start 2024-04-26T21:38:23.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.824+0000 7f41dad39640 1 -- start start 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.824+0000 7f41dad39640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f41d40cc480 0x7f41d4072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.824+0000 7f41dad39640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41d4072780 0x7f41d4166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.824+0000 7f41dad39640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d415d7c0 con 0x7f41d4071e60 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.824+0000 7f41dad39640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f41d415daa0 con 0x7f41d4072780 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.824+0000 7f41dad39640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f41d415dd80 con 0x7f41d40cc480 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.824+0000 7f41d92d6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41d4072780 0x7f41d4166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d92d6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41d4072780 0x7f41d4166ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46878/0 (socket says 172.21.15.40:46878) 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d92d6640 1 -- 172.21.15.40:0/2189500521 learned_addr learned my addr 172.21.15.40:0/2189500521 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d3fff640 1 --2- 172.21.15.40:0/2189500521 >> v2:172.21.15.184:3300/0 conn(0x7f41d40cc480 0x7f41d4072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d92d6640 1 -- 172.21.15.40:0/2189500521 >> v2:172.21.15.184:3300/0 conn(0x7f41d40cc480 msgr2=0x7f41d4072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d92d6640 1 --2- 172.21.15.40:0/2189500521 >> v2:172.21.15.184:3300/0 conn(0x7f41d40cc480 0x7f41d4072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d92d6640 1 -- 172.21.15.40:0/2189500521 >> v1:172.21.15.40:6789/0 conn(0x7f41d4071e60 legacy=0x7f41d406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d92d6640 1 -- 172.21.15.40:0/2189500521 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d415e060 con 0x7f41d4072780 2024-04-26T21:38:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d3fff640 1 --2- 172.21.15.40:0/2189500521 >> v2:172.21.15.184:3300/0 conn(0x7f41d40cc480 0x7f41d4072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.825+0000 7f41d92d6640 1 --2- 172.21.15.40:0/2189500521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d4072780 0x7f41d4166ad0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f41d40b3820 tx=0x7f41cc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e367231c82cb7722 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41d37fe640 1 -- 172.21.15.40:0/2189500521 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc036070 con 0x7f41d4072780 2024-04-26T21:38:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41d37fe640 1 -- 172.21.15.40:0/2189500521 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc031600 con 0x7f41d4072780 2024-04-26T21:38:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41d37fe640 1 -- 172.21.15.40:0/2189500521 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0319d0 con 0x7f41d4072780 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41dad39640 1 -- 172.21.15.40:0/2189500521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d4072780 msgr2=0x7f41d4166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41dad39640 1 --2- 172.21.15.40:0/2189500521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d4072780 0x7f41d4166ad0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f41d40b3820 tx=0x7f41cc02a730 comp rx=0 tx=0).stop 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41dad39640 1 -- 172.21.15.40:0/2189500521 shutdown_connections 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41dad39640 1 --2- 172.21.15.40:0/2189500521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d4072780 0x7f41d4166ad0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41dad39640 1 --2- 172.21.15.40:0/2189500521 >> v2:172.21.15.184:3300/0 conn(0x7f41d40cc480 0x7f41d4072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41dad39640 1 -- 172.21.15.40:0/2189500521 >> 172.21.15.40:0/2189500521 conn(0x7f41d40d0be0 msgr2=0x7f41d40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41dad39640 1 -- 172.21.15.40:0/2189500521 shutdown_connections 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.826+0000 7f41dad39640 1 -- 172.21.15.40:0/2189500521 wait complete. 2024-04-26T21:38:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.827+0000 7f41dad39640 1 Processor -- start 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.827+0000 7f41dad39640 1 -- start start 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41dad39640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41d4071e60 0x7f41d415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41dad39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41d4072780 0x7f41d40aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41dad39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d40cc480 0x7f41d41f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41dad39640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41d415d3f0 con 0x7f41d40cc480 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41dad39640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4072bd0 con 0x7f41d4071e60 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41dad39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41d415d570 con 0x7f41d4072780 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41d92d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d40cc480 0x7f41d41f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41d92d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d40cc480 0x7f41d41f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46886/0 (socket says 172.21.15.40:46886) 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.828+0000 7f41d92d6640 1 -- 172.21.15.40:0/2277073320 learned_addr learned my addr 172.21.15.40:0/2277073320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d8ad5640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41d4071e60 0x7f41d415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d3fff640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41d4072780 0x7f41d40aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d92d6640 1 -- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41d4072780 msgr2=0x7f41d40aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d92d6640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41d4072780 0x7f41d40aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d92d6640 1 -- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41d4071e60 msgr2=0x7f41d415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d92d6640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41d4071e60 0x7f41d415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d92d6640 1 -- 172.21.15.40:0/2277073320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d40ab700 con 0x7f41d40cc480 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d92d6640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d40cc480 0x7f41d41f5b30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009f40 tx=0x7f41cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d17fa640 1 -- 172.21.15.40:0/2277073320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc036050 con 0x7f41d40cc480 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d17fa640 1 -- 172.21.15.40:0/2277073320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc042070 con 0x7f41d40cc480 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d40ab9e0 con 0x7f41d40cc480 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41d17fa640 1 -- 172.21.15.40:0/2277073320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc009510 con 0x7f41d40cc480 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.829+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d40abfa0 con 0x7f41d40cc480 2024-04-26T21:38:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.830+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d40694f0 con 0x7f41d40cc480 2024-04-26T21:38:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.831+0000 7f41d17fa640 1 -- 172.21.15.40:0/2277073320 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41cc007bf0 con 0x7f41d40cc480 2024-04-26T21:38:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.831+0000 7f41d17fa640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.831+0000 7f41d17fa640 1 -- 172.21.15.40:0/2277073320 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f41cc0acb60 con 0x7f41d40cc480 2024-04-26T21:38:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.831+0000 7f41d8ad5640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.832+0000 7f41d8ad5640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41b806c8f0 0x7f41b806edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41c8001640 tx=0x7f41c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:23.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:23.835+0000 7f41d17fa640 1 -- 172.21.15.40:0/2277073320 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41cc045360 con 0x7f41d40cc480 2024-04-26T21:38:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.092+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f41d40b6720 con 0x7f41d40cc480 2024-04-26T21:38:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.092+0000 7f41d17fa640 1 -- 172.21.15.40:0/2277073320 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1453 (secure 0 0 0) 0x7f41cc07ae00 con 0x7f41d40cc480 2024-04-26T21:38:24.094 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:24.094 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":218,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":8,"num_up_osds":8,"osd_up_since":1714167297,"num_in_osds":8,"osd_in_since":1714167287,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":228663296,"bytes_avail":772865449984,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":49,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6546}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-26T21:38:06.036193+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T21:38:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.094+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41b806c8f0 msgr2=0x7f41b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.094+0000 7f41dad39640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41b806c8f0 0x7f41b806edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41c8001640 tx=0x7f41c8009040 comp rx=0 tx=0).stop 2024-04-26T21:38:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.094+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d40cc480 msgr2=0x7f41d41f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.094+0000 7f41dad39640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d40cc480 0x7f41d41f5b30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009f40 tx=0x7f41cc004a40 comp rx=0 tx=0).stop 2024-04-26T21:38:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.095+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 shutdown_connections 2024-04-26T21:38:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.095+0000 7f41dad39640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.095+0000 7f41dad39640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d40cc480 0x7f41d41f5b30 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.095+0000 7f41dad39640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41d4072780 0x7f41d40aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.095+0000 7f41dad39640 1 --2- 172.21.15.40:0/2277073320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41d4071e60 0x7f41d415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.095+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 >> 172.21.15.40:0/2277073320 conn(0x7f41d40d0be0 msgr2=0x7f41d40b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:24.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.095+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 shutdown_connections 2024-04-26T21:38:24.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.095+0000 7f41dad39640 1 -- 172.21.15.40:0/2277073320 wait complete. 2024-04-26T21:38:24.107 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:38:24.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:38:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.207+0000 7fafe6adc640 1 Processor -- start 2024-04-26T21:38:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.207+0000 7fafe6adc640 1 -- start start 2024-04-26T21:38:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafe6adc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafe00d3bc0 0x7fafe00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafe6adc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe00b7e50 con 0x7fafe00b5f20 2024-04-26T21:38:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafe6adc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe00d1800 con 0x7fafe0162880 2024-04-26T21:38:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafe6adc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fafe00d0580 con 0x7fafe00d3bc0 2024-04-26T21:38:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafdffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafe00d3bc0 0x7fafe00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafdffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafe00d3bc0 0x7fafe00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46888/0 (socket says 172.21.15.40:46888) 2024-04-26T21:38:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafdffff640 1 -- 172.21.15.40:0/901732904 learned_addr learned my addr 172.21.15.40:0/901732904 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafdffff640 1 -- 172.21.15.40:0/901732904 >> v1:172.21.15.184:6789/0 conn(0x7fafe0162880 legacy=0x7fafe00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafdffff640 1 -- 172.21.15.40:0/901732904 >> v1:172.21.15.40:6789/0 conn(0x7fafe00b5f20 legacy=0x7fafe00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.208+0000 7fafdffff640 1 -- 172.21.15.40:0/901732904 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe00d07d0 con 0x7fafe00d3bc0 2024-04-26T21:38:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafdffff640 1 --2- 172.21.15.40:0/901732904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe00d3bc0 0x7fafe00d2cb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009870 tx=0x7fafcc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a1f5712b6dd66e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafdf7fe640 1 -- 172.21.15.40:0/901732904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc037070 con 0x7fafe00d3bc0 2024-04-26T21:38:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafdf7fe640 1 -- 172.21.15.40:0/901732904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafcc032440 con 0x7fafe00d3bc0 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafdf7fe640 1 -- 172.21.15.40:0/901732904 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc032810 con 0x7fafe00d3bc0 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafe6adc640 1 -- 172.21.15.40:0/901732904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe00d3bc0 msgr2=0x7fafe00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafe6adc640 1 --2- 172.21.15.40:0/901732904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe00d3bc0 0x7fafe00d2cb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009870 tx=0x7fafcc02a2a0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafe6adc640 1 -- 172.21.15.40:0/901732904 shutdown_connections 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafe6adc640 1 --2- 172.21.15.40:0/901732904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe00d3bc0 0x7fafe00d2cb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafe6adc640 1 -- 172.21.15.40:0/901732904 >> 172.21.15.40:0/901732904 conn(0x7fafe00d4f60 msgr2=0x7fafe00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafe6adc640 1 -- 172.21.15.40:0/901732904 shutdown_connections 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.209+0000 7fafe6adc640 1 -- 172.21.15.40:0/901732904 wait complete. 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.211+0000 7fafe6adc640 1 Processor -- start 2024-04-26T21:38:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.211+0000 7fafe6adc640 1 -- start start 2024-04-26T21:38:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.211+0000 7fafe6adc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe00b5f20 0x7fafe0151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.211+0000 7fafe6adc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafe00d3bc0 0x7fafe009ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.211+0000 7fafe6adc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe0162880 0x7fafe00b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.211+0000 7fafe6adc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0162c60 con 0x7fafe0162880 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.211+0000 7fafe6adc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafe00d0d90 con 0x7fafe00b5f20 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.211+0000 7fafe6adc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafe0162de0 con 0x7fafe00d3bc0 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe0162880 0x7fafe00b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe0162880 0x7fafe00b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46896/0 (socket says 172.21.15.40:46896) 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 -- 172.21.15.40:0/1514357474 learned_addr learned my addr 172.21.15.40:0/1514357474 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe4878640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe00b5f20 0x7fafe0151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafdffff640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafe00d3bc0 0x7fafe009ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 -- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafe00d3bc0 msgr2=0x7fafe009ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafe00d3bc0 0x7fafe009ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 -- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe00b5f20 msgr2=0x7fafe0151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe00b5f20 0x7fafe0151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 -- 172.21.15.40:0/1514357474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe009f5d0 con 0x7fafe0162880 2024-04-26T21:38:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe5079640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe0162880 0x7fafe00b69e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fafd800c7a0 tx=0x7fafd800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafdd7fa640 1 -- 172.21.15.40:0/1514357474 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8003b70 con 0x7fafe0162880 2024-04-26T21:38:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafdd7fa640 1 -- 172.21.15.40:0/1514357474 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd8003d30 con 0x7fafe0162880 2024-04-26T21:38:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe009f8b0 con 0x7fafe0162880 2024-04-26T21:38:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafdd7fa640 1 -- 172.21.15.40:0/1514357474 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd800aa90 con 0x7fafe0162880 2024-04-26T21:38:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.212+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe009fe70 con 0x7fafe0162880 2024-04-26T21:38:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.213+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe00694f0 con 0x7fafe0162880 2024-04-26T21:38:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.214+0000 7fafdd7fa640 1 -- 172.21.15.40:0/1514357474 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafd800ac30 con 0x7fafe0162880 2024-04-26T21:38:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.214+0000 7fafdd7fa640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafc406c8f0 0x7fafc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.214+0000 7fafdd7fa640 1 -- 172.21.15.40:0/1514357474 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fafd808bbf0 con 0x7fafe0162880 2024-04-26T21:38:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.214+0000 7fafe4878640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafc406c8f0 0x7fafc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.215+0000 7fafe4878640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafc406c8f0 0x7fafc406edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fafd4006fd0 tx=0x7fafd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.218+0000 7fafdd7fa640 1 -- 172.21.15.40:0/1514357474 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fafd8059f20 con 0x7fafe0162880 2024-04-26T21:38:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.403+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafe00cb980 con 0x7fafc406c8f0 2024-04-26T21:38:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.410+0000 7fafdd7fa640 1 -- 172.21.15.40:0/1514357474 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252019 (secure 0 0 0) 0x7fafe00cb980 con 0x7fafc406c8f0 2024-04-26T21:38:24.414 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:24.415 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafc406c8f0 msgr2=0x7fafc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafc406c8f0 0x7fafc406edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fafd4006fd0 tx=0x7fafd4008040 comp rx=0 tx=0).stop 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe0162880 msgr2=0x7fafe00b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe0162880 0x7fafe00b69e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fafd800c7a0 tx=0x7fafd800cc60 comp rx=0 tx=0).stop 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 shutdown_connections 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafc406c8f0 0x7fafc406edb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe0162880 0x7fafe00b69e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafe00d3bc0 0x7fafe009ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 --2- 172.21.15.40:0/1514357474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe00b5f20 0x7fafe0151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 >> 172.21.15.40:0/1514357474 conn(0x7fafe00d4f60 msgr2=0x7fafe00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 shutdown_connections 2024-04-26T21:38:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:24.413+0000 7fafe6adc640 1 -- 172.21.15.40:0/1514357474 wait complete. 2024-04-26T21:38:24.425 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-04-26T21:38:24.040138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223620,"kb_used_data":7452,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751100,"statfs":{"total":773094113280,"available":772865126400,"internally_reserved":0,"allocated":7630848,"data_stored":4097024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.108295"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934419+0000","last_change":"2024-04-26T21:38:18.959438+0000","last_active":"2024-04-26T21:38:19.934419+0000","last_peered":"2024-04-26T21:38:19.934419+0000","last_clean":"2024-04-26T21:38:19.934419+0000","last_became_active":"2024-04-26T21:38:18.959052+0000","last_became_peered":"2024-04-26T21:38:18.959052+0000","last_unstale":"2024-04-26T21:38:19.934419+0000","last_undegraded":"2024-04-26T21:38:19.934419+0000","last_fullsized":"2024-04-26T21:38:19.934419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.692628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934276+0000","last_change":"2024-04-26T21:38:18.956022+0000","last_active":"2024-04-26T21:38:19.934276+0000","last_peered":"2024-04-26T21:38:19.934276+0000","last_clean":"2024-04-26T21:38:19.934276+0000","last_became_active":"2024-04-26T21:38:18.955671+0000","last_became_peered":"2024-04-26T21:38:18.955671+0000","last_unstale":"2024-04-26T21:38:19.934276+0000","last_undegraded":"2024-04-26T21:38:19.934276+0000","last_fullsized":"2024-04-26T21:38:19.934276+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:47.541044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099392+0000","last_change":"2024-04-26T21:38:18.957492+0000","last_active":"2024-04-26T21:38:21.099392+0000","last_peered":"2024-04-26T21:38:21.099392+0000","last_clean":"2024-04-26T21:38:21.099392+0000","last_became_active":"2024-04-26T21:38:18.957167+0000","last_became_peered":"2024-04-26T21:38:18.957167+0000","last_unstale":"2024-04-26T21:38:21.099392+0000","last_undegraded":"2024-04-26T21:38:21.099392+0000","last_fullsized":"2024-04-26T21:38:21.099392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:28.266053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200146+0000","last_change":"2024-04-26T21:38:18.948395+0000","last_active":"2024-04-26T21:38:22.200146+0000","last_peered":"2024-04-26T21:38:22.200146+0000","last_clean":"2024-04-26T21:38:22.200146+0000","last_became_active":"2024-04-26T21:38:18.948141+0000","last_became_peered":"2024-04-26T21:38:18.948141+0000","last_unstale":"2024-04-26T21:38:22.200146+0000","last_undegraded":"2024-04-26T21:38:22.200146+0000","last_fullsized":"2024-04-26T21:38:22.200146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:36.862335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953081+0000","last_change":"2024-04-26T21:38:18.952537+0000","last_active":"2024-04-26T21:38:19.953081+0000","last_peered":"2024-04-26T21:38:19.953081+0000","last_clean":"2024-04-26T21:38:19.953081+0000","last_became_active":"2024-04-26T21:38:18.952286+0000","last_became_peered":"2024-04-26T21:38:18.952286+0000","last_unstale":"2024-04-26T21:38:19.953081+0000","last_undegraded":"2024-04-26T21:38:19.953081+0000","last_fullsized":"2024-04-26T21:38:19.953081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:48.185126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199646+0000","last_change":"2024-04-26T21:38:18.951837+0000","last_active":"2024-04-26T21:38:22.199646+0000","last_peered":"2024-04-26T21:38:22.199646+0000","last_clean":"2024-04-26T21:38:22.199646+0000","last_became_active":"2024-04-26T21:38:18.951438+0000","last_became_peered":"2024-04-26T21:38:18.951438+0000","last_unstale":"2024-04-26T21:38:22.199646+0000","last_undegraded":"2024-04-26T21:38:22.199646+0000","last_fullsized":"2024-04-26T21:38:22.199646+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:50.681412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933590+0000","last_change":"2024-04-26T21:38:18.944281+0000","last_active":"2024-04-26T21:38:19.933590+0000","last_peered":"2024-04-26T21:38:19.933590+0000","last_clean":"2024-04-26T21:38:19.933590+0000","last_became_active":"2024-04-26T21:38:18.943978+0000","last_became_peered":"2024-04-26T21:38:18.943978+0000","last_unstale":"2024-04-26T21:38:19.933590+0000","last_undegraded":"2024-04-26T21:38:19.933590+0000","last_fullsized":"2024-04-26T21:38:19.933590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:36.157494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933412+0000","last_change":"2024-04-26T21:38:18.943968+0000","last_active":"2024-04-26T21:38:19.933412+0000","last_peered":"2024-04-26T21:38:19.933412+0000","last_clean":"2024-04-26T21:38:19.933412+0000","last_became_active":"2024-04-26T21:38:18.943684+0000","last_became_peered":"2024-04-26T21:38:18.943684+0000","last_unstale":"2024-04-26T21:38:19.933412+0000","last_undegraded":"2024-04-26T21:38:19.933412+0000","last_fullsized":"2024-04-26T21:38:19.933412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:53.023152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099820+0000","last_change":"2024-04-26T21:38:18.955952+0000","last_active":"2024-04-26T21:38:21.099820+0000","last_peered":"2024-04-26T21:38:21.099820+0000","last_clean":"2024-04-26T21:38:21.099820+0000","last_became_active":"2024-04-26T21:38:18.955643+0000","last_became_peered":"2024-04-26T21:38:18.955643+0000","last_unstale":"2024-04-26T21:38:21.099820+0000","last_undegraded":"2024-04-26T21:38:21.099820+0000","last_fullsized":"2024-04-26T21:38:21.099820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:17.991870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099673+0000","last_change":"2024-04-26T21:38:18.957998+0000","last_active":"2024-04-26T21:38:21.099673+0000","last_peered":"2024-04-26T21:38:21.099673+0000","last_clean":"2024-04-26T21:38:21.099673+0000","last_became_active":"2024-04-26T21:38:18.957675+0000","last_became_peered":"2024-04-26T21:38:18.957675+0000","last_unstale":"2024-04-26T21:38:21.099673+0000","last_undegraded":"2024-04-26T21:38:21.099673+0000","last_fullsized":"2024-04-26T21:38:21.099673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:11.176375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200224+0000","last_change":"2024-04-26T21:38:18.959855+0000","last_active":"2024-04-26T21:38:22.200224+0000","last_peered":"2024-04-26T21:38:22.200224+0000","last_clean":"2024-04-26T21:38:22.200224+0000","last_became_active":"2024-04-26T21:38:18.959580+0000","last_became_peered":"2024-04-26T21:38:18.959580+0000","last_unstale":"2024-04-26T21:38:22.200224+0000","last_undegraded":"2024-04-26T21:38:22.200224+0000","last_fullsized":"2024-04-26T21:38:22.200224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:51.315426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933506+0000","last_change":"2024-04-26T21:38:18.942690+0000","last_active":"2024-04-26T21:38:19.933506+0000","last_peered":"2024-04-26T21:38:19.933506+0000","last_clean":"2024-04-26T21:38:19.933506+0000","last_became_active":"2024-04-26T21:38:18.942377+0000","last_became_peered":"2024-04-26T21:38:18.942377+0000","last_unstale":"2024-04-26T21:38:19.933506+0000","last_undegraded":"2024-04-26T21:38:19.933506+0000","last_fullsized":"2024-04-26T21:38:19.933506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:32.347741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933919+0000","last_change":"2024-04-26T21:38:18.947832+0000","last_active":"2024-04-26T21:38:19.933919+0000","last_peered":"2024-04-26T21:38:19.933919+0000","last_clean":"2024-04-26T21:38:19.933919+0000","last_became_active":"2024-04-26T21:38:18.947561+0000","last_became_peered":"2024-04-26T21:38:18.947561+0000","last_unstale":"2024-04-26T21:38:19.933919+0000","last_undegraded":"2024-04-26T21:38:19.933919+0000","last_fullsized":"2024-04-26T21:38:19.933919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:31.910540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099598+0000","last_change":"2024-04-26T21:38:18.952884+0000","last_active":"2024-04-26T21:38:21.099598+0000","last_peered":"2024-04-26T21:38:21.099598+0000","last_clean":"2024-04-26T21:38:21.099598+0000","last_became_active":"2024-04-26T21:38:18.952624+0000","last_became_peered":"2024-04-26T21:38:18.952624+0000","last_unstale":"2024-04-26T21:38:21.099598+0000","last_undegraded":"2024-04-26T21:38:21.099598+0000","last_fullsized":"2024-04-26T21:38:21.099598+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:31.910540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099381+0000","last_change":"2024-04-26T21:38:18.960701+0000","last_active":"2024-04-26T21:38:21.099381+0000","last_peered":"2024-04-26T21:38:21.099381+0000","last_clean":"2024-04-26T21:38:21.099381+0000","last_became_active":"2024-04-26T21:38:18.960425+0000","last_became_peered":"2024-04-26T21:38:18.960425+0000","last_unstale":"2024-04-26T21:38:21.099381+0000","last_undegraded":"2024-04-26T21:38:21.099381+0000","last_fullsized":"2024-04-26T21:38:21.099381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:32.347741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933471+0000","last_change":"2024-04-26T21:38:18.946641+0000","last_active":"2024-04-26T21:38:19.933471+0000","last_peered":"2024-04-26T21:38:19.933471+0000","last_clean":"2024-04-26T21:38:19.933471+0000","last_became_active":"2024-04-26T21:38:18.946372+0000","last_became_peered":"2024-04-26T21:38:18.946372+0000","last_unstale":"2024-04-26T21:38:19.933471+0000","last_undegraded":"2024-04-26T21:38:19.933471+0000","last_fullsized":"2024-04-26T21:38:19.933471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:30.460324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934456+0000","last_change":"2024-04-26T21:38:18.949643+0000","last_active":"2024-04-26T21:38:19.934456+0000","last_peered":"2024-04-26T21:38:19.934456+0000","last_clean":"2024-04-26T21:38:19.934456+0000","last_became_active":"2024-04-26T21:38:18.949291+0000","last_became_peered":"2024-04-26T21:38:18.949291+0000","last_unstale":"2024-04-26T21:38:19.934456+0000","last_undegraded":"2024-04-26T21:38:19.934456+0000","last_fullsized":"2024-04-26T21:38:19.934456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.536861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200309+0000","last_change":"2024-04-26T21:38:18.948352+0000","last_active":"2024-04-26T21:38:22.200309+0000","last_peered":"2024-04-26T21:38:22.200309+0000","last_clean":"2024-04-26T21:38:22.200309+0000","last_became_active":"2024-04-26T21:38:18.947746+0000","last_became_peered":"2024-04-26T21:38:18.947746+0000","last_unstale":"2024-04-26T21:38:22.200309+0000","last_undegraded":"2024-04-26T21:38:22.200309+0000","last_fullsized":"2024-04-26T21:38:22.200309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:23.266362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953127+0000","last_change":"2024-04-26T21:38:18.947674+0000","last_active":"2024-04-26T21:38:19.953127+0000","last_peered":"2024-04-26T21:38:19.953127+0000","last_clean":"2024-04-26T21:38:19.953127+0000","last_became_active":"2024-04-26T21:38:18.947397+0000","last_became_peered":"2024-04-26T21:38:18.947397+0000","last_unstale":"2024-04-26T21:38:19.953127+0000","last_undegraded":"2024-04-26T21:38:19.953127+0000","last_fullsized":"2024-04-26T21:38:19.953127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:39.442390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760361+0000","last_change":"2024-04-26T21:38:18.951143+0000","last_active":"2024-04-26T21:38:21.760361+0000","last_peered":"2024-04-26T21:38:21.760361+0000","last_clean":"2024-04-26T21:38:21.760361+0000","last_became_active":"2024-04-26T21:38:18.950851+0000","last_became_peered":"2024-04-26T21:38:18.950851+0000","last_unstale":"2024-04-26T21:38:21.760361+0000","last_undegraded":"2024-04-26T21:38:21.760361+0000","last_fullsized":"2024-04-26T21:38:21.760361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:20.945070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933961+0000","last_change":"2024-04-26T21:38:18.941201+0000","last_active":"2024-04-26T21:38:19.933961+0000","last_peered":"2024-04-26T21:38:19.933961+0000","last_clean":"2024-04-26T21:38:19.933961+0000","last_became_active":"2024-04-26T21:38:18.940870+0000","last_became_peered":"2024-04-26T21:38:18.940870+0000","last_unstale":"2024-04-26T21:38:19.933961+0000","last_undegraded":"2024-04-26T21:38:19.933961+0000","last_fullsized":"2024-04-26T21:38:19.933961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:11.176375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760066+0000","last_change":"2024-04-26T21:38:18.951917+0000","last_active":"2024-04-26T21:38:21.760066+0000","last_peered":"2024-04-26T21:38:21.760066+0000","last_clean":"2024-04-26T21:38:21.760066+0000","last_became_active":"2024-04-26T21:38:18.951657+0000","last_became_peered":"2024-04-26T21:38:18.951657+0000","last_unstale":"2024-04-26T21:38:21.760066+0000","last_undegraded":"2024-04-26T21:38:21.760066+0000","last_fullsized":"2024-04-26T21:38:21.760066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:50.681412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760043+0000","last_change":"2024-04-26T21:38:18.952718+0000","last_active":"2024-04-26T21:38:21.760043+0000","last_peered":"2024-04-26T21:38:21.760043+0000","last_clean":"2024-04-26T21:38:21.760043+0000","last_became_active":"2024-04-26T21:38:18.952436+0000","last_became_peered":"2024-04-26T21:38:18.952436+0000","last_unstale":"2024-04-26T21:38:21.760043+0000","last_undegraded":"2024-04-26T21:38:21.760043+0000","last_fullsized":"2024-04-26T21:38:21.760043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:48.116434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099278+0000","last_change":"2024-04-26T21:38:18.954925+0000","last_active":"2024-04-26T21:38:21.099278+0000","last_peered":"2024-04-26T21:38:21.099278+0000","last_clean":"2024-04-26T21:38:21.099278+0000","last_became_active":"2024-04-26T21:38:18.953754+0000","last_became_peered":"2024-04-26T21:38:18.953754+0000","last_unstale":"2024-04-26T21:38:21.099278+0000","last_undegraded":"2024-04-26T21:38:21.099278+0000","last_fullsized":"2024-04-26T21:38:21.099278+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:14.110732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933683+0000","last_change":"2024-04-26T21:38:18.943836+0000","last_active":"2024-04-26T21:38:19.933683+0000","last_peered":"2024-04-26T21:38:19.933683+0000","last_clean":"2024-04-26T21:38:19.933683+0000","last_became_active":"2024-04-26T21:38:18.943496+0000","last_became_peered":"2024-04-26T21:38:18.943496+0000","last_unstale":"2024-04-26T21:38:19.933683+0000","last_undegraded":"2024-04-26T21:38:19.933683+0000","last_fullsized":"2024-04-26T21:38:19.933683+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:58.368085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934025+0000","last_change":"2024-04-26T21:38:18.944578+0000","last_active":"2024-04-26T21:38:19.934025+0000","last_peered":"2024-04-26T21:38:19.934025+0000","last_clean":"2024-04-26T21:38:19.934025+0000","last_became_active":"2024-04-26T21:38:18.944333+0000","last_became_peered":"2024-04-26T21:38:18.944333+0000","last_unstale":"2024-04-26T21:38:19.934025+0000","last_undegraded":"2024-04-26T21:38:19.934025+0000","last_fullsized":"2024-04-26T21:38:19.934025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:55.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200345+0000","last_change":"2024-04-26T21:38:18.946326+0000","last_active":"2024-04-26T21:38:22.200345+0000","last_peered":"2024-04-26T21:38:22.200345+0000","last_clean":"2024-04-26T21:38:22.200345+0000","last_became_active":"2024-04-26T21:38:18.945972+0000","last_became_peered":"2024-04-26T21:38:18.945972+0000","last_unstale":"2024-04-26T21:38:22.200345+0000","last_undegraded":"2024-04-26T21:38:22.200345+0000","last_fullsized":"2024-04-26T21:38:22.200345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:30.973393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934409+0000","last_change":"2024-04-26T21:38:18.962609+0000","last_active":"2024-04-26T21:38:19.934409+0000","last_peered":"2024-04-26T21:38:19.934409+0000","last_clean":"2024-04-26T21:38:19.934409+0000","last_became_active":"2024-04-26T21:38:18.962235+0000","last_became_peered":"2024-04-26T21:38:18.962235+0000","last_unstale":"2024-04-26T21:38:19.934409+0000","last_undegraded":"2024-04-26T21:38:19.934409+0000","last_fullsized":"2024-04-26T21:38:19.934409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:45.205083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099445+0000","last_change":"2024-04-26T21:38:18.952731+0000","last_active":"2024-04-26T21:38:21.099445+0000","last_peered":"2024-04-26T21:38:21.099445+0000","last_clean":"2024-04-26T21:38:21.099445+0000","last_became_active":"2024-04-26T21:38:18.952474+0000","last_became_peered":"2024-04-26T21:38:18.952474+0000","last_unstale":"2024-04-26T21:38:21.099445+0000","last_undegraded":"2024-04-26T21:38:21.099445+0000","last_fullsized":"2024-04-26T21:38:21.099445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.399686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933694+0000","last_change":"2024-04-26T21:38:18.943991+0000","last_active":"2024-04-26T21:38:19.933694+0000","last_peered":"2024-04-26T21:38:19.933694+0000","last_clean":"2024-04-26T21:38:19.933694+0000","last_became_active":"2024-04-26T21:38:18.943564+0000","last_became_peered":"2024-04-26T21:38:18.943564+0000","last_unstale":"2024-04-26T21:38:19.933694+0000","last_undegraded":"2024-04-26T21:38:19.933694+0000","last_fullsized":"2024-04-26T21:38:19.933694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:14.110732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199910+0000","last_change":"2024-04-26T21:38:18.944660+0000","last_active":"2024-04-26T21:38:22.199910+0000","last_peered":"2024-04-26T21:38:22.199910+0000","last_clean":"2024-04-26T21:38:22.199910+0000","last_became_active":"2024-04-26T21:38:18.944340+0000","last_became_peered":"2024-04-26T21:38:18.944340+0000","last_unstale":"2024-04-26T21:38:22.199910+0000","last_undegraded":"2024-04-26T21:38:22.199910+0000","last_fullsized":"2024-04-26T21:38:22.199910+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:29.814338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199386+0000","last_change":"2024-04-26T21:38:18.950340+0000","last_active":"2024-04-26T21:38:22.199386+0000","last_peered":"2024-04-26T21:38:22.199386+0000","last_clean":"2024-04-26T21:38:22.199386+0000","last_became_active":"2024-04-26T21:38:18.950032+0000","last_became_peered":"2024-04-26T21:38:18.950032+0000","last_unstale":"2024-04-26T21:38:22.199386+0000","last_undegraded":"2024-04-26T21:38:22.199386+0000","last_fullsized":"2024-04-26T21:38:22.199386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:07.586134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952983+0000","last_change":"2024-04-26T21:38:18.963455+0000","last_active":"2024-04-26T21:38:19.952983+0000","last_peered":"2024-04-26T21:38:19.952983+0000","last_clean":"2024-04-26T21:38:19.952983+0000","last_became_active":"2024-04-26T21:38:18.963181+0000","last_became_peered":"2024-04-26T21:38:18.963181+0000","last_unstale":"2024-04-26T21:38:19.952983+0000","last_undegraded":"2024-04-26T21:38:19.952983+0000","last_fullsized":"2024-04-26T21:38:19.952983+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:48.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934525+0000","last_change":"2024-04-26T21:38:17.948310+0000","last_active":"2024-04-26T21:38:19.934525+0000","last_peered":"2024-04-26T21:38:19.934525+0000","last_clean":"2024-04-26T21:38:19.934525+0000","last_became_active":"2024-04-26T21:38:17.948052+0000","last_became_peered":"2024-04-26T21:38:17.948052+0000","last_unstale":"2024-04-26T21:38:19.934525+0000","last_undegraded":"2024-04-26T21:38:19.934525+0000","last_fullsized":"2024-04-26T21:38:19.934525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:02.996302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200398+0000","last_change":"2024-04-26T21:38:18.951143+0000","last_active":"2024-04-26T21:38:22.200398+0000","last_peered":"2024-04-26T21:38:22.200398+0000","last_clean":"2024-04-26T21:38:22.200398+0000","last_became_active":"2024-04-26T21:38:18.950744+0000","last_became_peered":"2024-04-26T21:38:18.950744+0000","last_unstale":"2024-04-26T21:38:22.200398+0000","last_undegraded":"2024-04-26T21:38:22.200398+0000","last_fullsized":"2024-04-26T21:38:22.200398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:51.463923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199839+0000","last_change":"2024-04-26T21:38:17.949729+0000","last_active":"2024-04-26T21:38:22.199839+0000","last_peered":"2024-04-26T21:38:22.199839+0000","last_clean":"2024-04-26T21:38:22.199839+0000","last_became_active":"2024-04-26T21:38:17.949412+0000","last_became_peered":"2024-04-26T21:38:17.949412+0000","last_unstale":"2024-04-26T21:38:22.199839+0000","last_undegraded":"2024-04-26T21:38:22.199839+0000","last_fullsized":"2024-04-26T21:38:22.199839+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:44.393210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199802+0000","last_change":"2024-04-26T21:38:18.951483+0000","last_active":"2024-04-26T21:38:22.199802+0000","last_peered":"2024-04-26T21:38:22.199802+0000","last_clean":"2024-04-26T21:38:22.199802+0000","last_became_active":"2024-04-26T21:38:18.951192+0000","last_became_peered":"2024-04-26T21:38:18.951192+0000","last_unstale":"2024-04-26T21:38:22.199802+0000","last_undegraded":"2024-04-26T21:38:22.199802+0000","last_fullsized":"2024-04-26T21:38:22.199802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:33.979629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199707+0000","last_change":"2024-04-26T21:38:17.947141+0000","last_active":"2024-04-26T21:38:22.199707+0000","last_peered":"2024-04-26T21:38:22.199707+0000","last_clean":"2024-04-26T21:38:22.199707+0000","last_became_active":"2024-04-26T21:38:17.946847+0000","last_became_peered":"2024-04-26T21:38:17.946847+0000","last_unstale":"2024-04-26T21:38:22.199707+0000","last_undegraded":"2024-04-26T21:38:22.199707+0000","last_fullsized":"2024-04-26T21:38:22.199707+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:27.259577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099803+0000","last_change":"2024-04-26T21:38:18.961238+0000","last_active":"2024-04-26T21:38:21.099803+0000","last_peered":"2024-04-26T21:38:21.099803+0000","last_clean":"2024-04-26T21:38:21.099803+0000","last_became_active":"2024-04-26T21:38:18.960945+0000","last_became_peered":"2024-04-26T21:38:18.960945+0000","last_unstale":"2024-04-26T21:38:21.099803+0000","last_undegraded":"2024-04-26T21:38:21.099803+0000","last_fullsized":"2024-04-26T21:38:21.099803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:55.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199536+0000","last_change":"2024-04-26T21:38:17.949992+0000","last_active":"2024-04-26T21:38:22.199536+0000","last_peered":"2024-04-26T21:38:22.199536+0000","last_clean":"2024-04-26T21:38:22.199536+0000","last_became_active":"2024-04-26T21:38:17.949701+0000","last_became_peered":"2024-04-26T21:38:17.949701+0000","last_unstale":"2024-04-26T21:38:22.199536+0000","last_undegraded":"2024-04-26T21:38:22.199536+0000","last_fullsized":"2024-04-26T21:38:22.199536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:52.016675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933606+0000","last_change":"2024-04-26T21:38:18.936571+0000","last_active":"2024-04-26T21:38:19.933606+0000","last_peered":"2024-04-26T21:38:19.933606+0000","last_clean":"2024-04-26T21:38:19.933606+0000","last_became_active":"2024-04-26T21:38:18.936209+0000","last_became_peered":"2024-04-26T21:38:18.936209+0000","last_unstale":"2024-04-26T21:38:19.933606+0000","last_undegraded":"2024-04-26T21:38:19.933606+0000","last_fullsized":"2024-04-26T21:38:19.933606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:48.107975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760229+0000","last_change":"2024-04-26T21:38:17.947017+0000","last_active":"2024-04-26T21:38:21.760229+0000","last_peered":"2024-04-26T21:38:21.760229+0000","last_clean":"2024-04-26T21:38:21.760229+0000","last_became_active":"2024-04-26T21:38:17.946744+0000","last_became_peered":"2024-04-26T21:38:17.946744+0000","last_unstale":"2024-04-26T21:38:21.760229+0000","last_undegraded":"2024-04-26T21:38:21.760229+0000","last_fullsized":"2024-04-26T21:38:21.760229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:13.104255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933550+0000","last_change":"2024-04-26T21:38:18.945973+0000","last_active":"2024-04-26T21:38:19.933550+0000","last_peered":"2024-04-26T21:38:19.933550+0000","last_clean":"2024-04-26T21:38:19.933550+0000","last_became_active":"2024-04-26T21:38:18.945676+0000","last_became_peered":"2024-04-26T21:38:18.945676+0000","last_unstale":"2024-04-26T21:38:19.933550+0000","last_undegraded":"2024-04-26T21:38:19.933550+0000","last_fullsized":"2024-04-26T21:38:19.933550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:39.442390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.759997+0000","last_change":"2024-04-26T21:38:17.944477+0000","last_active":"2024-04-26T21:38:21.759997+0000","last_peered":"2024-04-26T21:38:21.759997+0000","last_clean":"2024-04-26T21:38:21.759997+0000","last_became_active":"2024-04-26T21:38:17.944149+0000","last_became_peered":"2024-04-26T21:38:17.944149+0000","last_unstale":"2024-04-26T21:38:21.759997+0000","last_undegraded":"2024-04-26T21:38:21.759997+0000","last_fullsized":"2024-04-26T21:38:21.759997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:52.016675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952780+0000","last_change":"2024-04-26T21:38:18.954534+0000","last_active":"2024-04-26T21:38:19.952780+0000","last_peered":"2024-04-26T21:38:19.952780+0000","last_clean":"2024-04-26T21:38:19.952780+0000","last_became_active":"2024-04-26T21:38:18.954227+0000","last_became_peered":"2024-04-26T21:38:18.954227+0000","last_unstale":"2024-04-26T21:38:19.952780+0000","last_undegraded":"2024-04-26T21:38:19.952780+0000","last_fullsized":"2024-04-26T21:38:19.952780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:01.934163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760193+0000","last_change":"2024-04-26T21:38:17.945215+0000","last_active":"2024-04-26T21:38:21.760193+0000","last_peered":"2024-04-26T21:38:21.760193+0000","last_clean":"2024-04-26T21:38:21.760193+0000","last_became_active":"2024-04-26T21:38:17.944579+0000","last_became_peered":"2024-04-26T21:38:17.944579+0000","last_unstale":"2024-04-26T21:38:21.760193+0000","last_undegraded":"2024-04-26T21:38:21.760193+0000","last_fullsized":"2024-04-26T21:38:21.760193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:57.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760177+0000","last_change":"2024-04-26T21:38:18.953771+0000","last_active":"2024-04-26T21:38:21.760177+0000","last_peered":"2024-04-26T21:38:21.760177+0000","last_clean":"2024-04-26T21:38:21.760177+0000","last_became_active":"2024-04-26T21:38:18.953342+0000","last_became_peered":"2024-04-26T21:38:18.953342+0000","last_unstale":"2024-04-26T21:38:21.760177+0000","last_undegraded":"2024-04-26T21:38:21.760177+0000","last_fullsized":"2024-04-26T21:38:21.760177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:47.732055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952723+0000","last_change":"2024-04-26T21:38:17.949823+0000","last_active":"2024-04-26T21:38:19.952723+0000","last_peered":"2024-04-26T21:38:19.952723+0000","last_clean":"2024-04-26T21:38:19.952723+0000","last_became_active":"2024-04-26T21:38:17.949531+0000","last_became_peered":"2024-04-26T21:38:17.949531+0000","last_unstale":"2024-04-26T21:38:19.952723+0000","last_undegraded":"2024-04-26T21:38:19.952723+0000","last_fullsized":"2024-04-26T21:38:19.952723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:14.240281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934418+0000","last_change":"2024-04-26T21:38:18.951856+0000","last_active":"2024-04-26T21:38:19.934418+0000","last_peered":"2024-04-26T21:38:19.934418+0000","last_clean":"2024-04-26T21:38:19.934418+0000","last_became_active":"2024-04-26T21:38:18.951557+0000","last_became_peered":"2024-04-26T21:38:18.951557+0000","last_unstale":"2024-04-26T21:38:19.934418+0000","last_undegraded":"2024-04-26T21:38:19.934418+0000","last_fullsized":"2024-04-26T21:38:19.934418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:04.188697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099892+0000","last_change":"2024-04-26T21:38:17.951863+0000","last_active":"2024-04-26T21:38:21.099892+0000","last_peered":"2024-04-26T21:38:21.099892+0000","last_clean":"2024-04-26T21:38:21.099892+0000","last_became_active":"2024-04-26T21:38:17.951250+0000","last_became_peered":"2024-04-26T21:38:17.951250+0000","last_unstale":"2024-04-26T21:38:21.099892+0000","last_undegraded":"2024-04-26T21:38:21.099892+0000","last_fullsized":"2024-04-26T21:38:21.099892+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:01.859943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760193+0000","last_change":"2024-04-26T21:38:18.950273+0000","last_active":"2024-04-26T21:38:21.760193+0000","last_peered":"2024-04-26T21:38:21.760193+0000","last_clean":"2024-04-26T21:38:21.760193+0000","last_became_active":"2024-04-26T21:38:18.949963+0000","last_became_peered":"2024-04-26T21:38:18.949963+0000","last_unstale":"2024-04-26T21:38:21.760193+0000","last_undegraded":"2024-04-26T21:38:21.760193+0000","last_fullsized":"2024-04-26T21:38:21.760193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:33.979629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200433+0000","last_change":"2024-04-26T21:38:17.952295+0000","last_active":"2024-04-26T21:38:22.200433+0000","last_peered":"2024-04-26T21:38:22.200433+0000","last_clean":"2024-04-26T21:38:22.200433+0000","last_became_active":"2024-04-26T21:38:17.951993+0000","last_became_peered":"2024-04-26T21:38:17.951993+0000","last_unstale":"2024-04-26T21:38:22.200433+0000","last_undegraded":"2024-04-26T21:38:22.200433+0000","last_fullsized":"2024-04-26T21:38:22.200433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:09.249196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953020+0000","last_change":"2024-04-26T21:38:18.951823+0000","last_active":"2024-04-26T21:38:19.953020+0000","last_peered":"2024-04-26T21:38:19.953020+0000","last_clean":"2024-04-26T21:38:19.953020+0000","last_became_active":"2024-04-26T21:38:18.951577+0000","last_became_peered":"2024-04-26T21:38:18.951577+0000","last_unstale":"2024-04-26T21:38:19.953020+0000","last_undegraded":"2024-04-26T21:38:19.953020+0000","last_fullsized":"2024-04-26T21:38:19.953020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:13.413771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099364+0000","last_change":"2024-04-26T21:38:17.951334+0000","last_active":"2024-04-26T21:38:21.099364+0000","last_peered":"2024-04-26T21:38:21.099364+0000","last_clean":"2024-04-26T21:38:21.099364+0000","last_became_active":"2024-04-26T21:38:17.951044+0000","last_became_peered":"2024-04-26T21:38:17.951044+0000","last_unstale":"2024-04-26T21:38:21.099364+0000","last_undegraded":"2024-04-26T21:38:21.099364+0000","last_fullsized":"2024-04-26T21:38:21.099364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:39.512854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200456+0000","last_change":"2024-04-26T21:38:18.947673+0000","last_active":"2024-04-26T21:38:22.200456+0000","last_peered":"2024-04-26T21:38:22.200456+0000","last_clean":"2024-04-26T21:38:22.200456+0000","last_became_active":"2024-04-26T21:38:18.947045+0000","last_became_peered":"2024-04-26T21:38:18.947045+0000","last_unstale":"2024-04-26T21:38:22.200456+0000","last_undegraded":"2024-04-26T21:38:22.200456+0000","last_fullsized":"2024-04-26T21:38:22.200456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:43.698138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934535+0000","last_change":"2024-04-26T21:38:17.945882+0000","last_active":"2024-04-26T21:38:19.934535+0000","last_peered":"2024-04-26T21:38:19.934535+0000","last_clean":"2024-04-26T21:38:19.934535+0000","last_became_active":"2024-04-26T21:38:17.945594+0000","last_became_peered":"2024-04-26T21:38:17.945594+0000","last_unstale":"2024-04-26T21:38:19.934535+0000","last_undegraded":"2024-04-26T21:38:19.934535+0000","last_fullsized":"2024-04-26T21:38:19.934535+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:42.691662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199909+0000","last_change":"2024-04-26T21:38:18.951616+0000","last_active":"2024-04-26T21:38:22.199909+0000","last_peered":"2024-04-26T21:38:22.199909+0000","last_clean":"2024-04-26T21:38:22.199909+0000","last_became_active":"2024-04-26T21:38:18.951329+0000","last_became_peered":"2024-04-26T21:38:18.951329+0000","last_unstale":"2024-04-26T21:38:22.199909+0000","last_undegraded":"2024-04-26T21:38:22.199909+0000","last_fullsized":"2024-04-26T21:38:22.199909+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:49.252133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099618+0000","last_change":"2024-04-26T21:38:17.947105+0000","last_active":"2024-04-26T21:38:21.099618+0000","last_peered":"2024-04-26T21:38:21.099618+0000","last_clean":"2024-04-26T21:38:21.099618+0000","last_became_active":"2024-04-26T21:38:17.946804+0000","last_became_peered":"2024-04-26T21:38:17.946804+0000","last_unstale":"2024-04-26T21:38:21.099618+0000","last_undegraded":"2024-04-26T21:38:21.099618+0000","last_fullsized":"2024-04-26T21:38:21.099618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:47.178650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933578+0000","last_change":"2024-04-26T21:38:18.946576+0000","last_active":"2024-04-26T21:38:19.933578+0000","last_peered":"2024-04-26T21:38:19.933578+0000","last_clean":"2024-04-26T21:38:19.933578+0000","last_became_active":"2024-04-26T21:38:18.946238+0000","last_became_peered":"2024-04-26T21:38:18.946238+0000","last_unstale":"2024-04-26T21:38:19.933578+0000","last_undegraded":"2024-04-26T21:38:19.933578+0000","last_fullsized":"2024-04-26T21:38:19.933578+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:28.266053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933737+0000","last_change":"2024-04-26T21:38:17.944988+0000","last_active":"2024-04-26T21:38:19.933737+0000","last_peered":"2024-04-26T21:38:19.933737+0000","last_clean":"2024-04-26T21:38:19.933737+0000","last_became_active":"2024-04-26T21:38:17.944724+0000","last_became_peered":"2024-04-26T21:38:17.944724+0000","last_unstale":"2024-04-26T21:38:19.933737+0000","last_undegraded":"2024-04-26T21:38:19.933737+0000","last_fullsized":"2024-04-26T21:38:19.933737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:57.060894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952799+0000","last_change":"2024-04-26T21:38:18.954305+0000","last_active":"2024-04-26T21:38:19.952799+0000","last_peered":"2024-04-26T21:38:19.952799+0000","last_clean":"2024-04-26T21:38:19.952799+0000","last_became_active":"2024-04-26T21:38:18.953647+0000","last_became_peered":"2024-04-26T21:38:18.953647+0000","last_unstale":"2024-04-26T21:38:19.952799+0000","last_undegraded":"2024-04-26T21:38:19.952799+0000","last_fullsized":"2024-04-26T21:38:19.952799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:29.752423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099401+0000","last_change":"2024-04-26T21:38:17.947884+0000","last_active":"2024-04-26T21:38:21.099401+0000","last_peered":"2024-04-26T21:38:21.099401+0000","last_clean":"2024-04-26T21:38:21.099401+0000","last_became_active":"2024-04-26T21:38:17.947599+0000","last_became_peered":"2024-04-26T21:38:17.947599+0000","last_unstale":"2024-04-26T21:38:21.099401+0000","last_undegraded":"2024-04-26T21:38:21.099401+0000","last_fullsized":"2024-04-26T21:38:21.099401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:40.097539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934158+0000","last_change":"2024-04-26T21:38:18.946460+0000","last_active":"2024-04-26T21:38:19.934158+0000","last_peered":"2024-04-26T21:38:19.934158+0000","last_clean":"2024-04-26T21:38:19.934158+0000","last_became_active":"2024-04-26T21:38:18.946145+0000","last_became_peered":"2024-04-26T21:38:18.946145+0000","last_unstale":"2024-04-26T21:38:19.934158+0000","last_undegraded":"2024-04-26T21:38:19.934158+0000","last_fullsized":"2024-04-26T21:38:19.934158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:02.866419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199824+0000","last_change":"2024-04-26T21:38:17.948438+0000","last_active":"2024-04-26T21:38:22.199824+0000","last_peered":"2024-04-26T21:38:22.199824+0000","last_clean":"2024-04-26T21:38:22.199824+0000","last_became_active":"2024-04-26T21:38:17.948196+0000","last_became_peered":"2024-04-26T21:38:17.948196+0000","last_unstale":"2024-04-26T21:38:22.199824+0000","last_undegraded":"2024-04-26T21:38:22.199824+0000","last_fullsized":"2024-04-26T21:38:22.199824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:59.951617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099420+0000","last_change":"2024-04-26T21:38:18.953516+0000","last_active":"2024-04-26T21:38:21.099420+0000","last_peered":"2024-04-26T21:38:21.099420+0000","last_clean":"2024-04-26T21:38:21.099420+0000","last_became_active":"2024-04-26T21:38:18.953197+0000","last_became_peered":"2024-04-26T21:38:18.953197+0000","last_unstale":"2024-04-26T21:38:21.099420+0000","last_undegraded":"2024-04-26T21:38:21.099420+0000","last_fullsized":"2024-04-26T21:38:21.099420+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:58.368085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933508+0000","last_change":"2024-04-26T21:38:17.942502+0000","last_active":"2024-04-26T21:38:19.933508+0000","last_peered":"2024-04-26T21:38:19.933508+0000","last_clean":"2024-04-26T21:38:19.933508+0000","last_became_active":"2024-04-26T21:38:17.942225+0000","last_became_peered":"2024-04-26T21:38:17.942225+0000","last_unstale":"2024-04-26T21:38:19.933508+0000","last_undegraded":"2024-04-26T21:38:19.933508+0000","last_fullsized":"2024-04-26T21:38:19.933508+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:40.097539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953140+0000","last_change":"2024-04-26T21:38:18.948264+0000","last_active":"2024-04-26T21:38:19.953140+0000","last_peered":"2024-04-26T21:38:19.953140+0000","last_clean":"2024-04-26T21:38:19.953140+0000","last_became_active":"2024-04-26T21:38:18.947965+0000","last_became_peered":"2024-04-26T21:38:18.947965+0000","last_unstale":"2024-04-26T21:38:19.953140+0000","last_undegraded":"2024-04-26T21:38:19.953140+0000","last_fullsized":"2024-04-26T21:38:19.953140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:40.519330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099697+0000","last_change":"2024-04-26T21:38:17.949980+0000","last_active":"2024-04-26T21:38:21.099697+0000","last_peered":"2024-04-26T21:38:21.099697+0000","last_clean":"2024-04-26T21:38:21.099697+0000","last_became_active":"2024-04-26T21:38:17.949710+0000","last_became_peered":"2024-04-26T21:38:17.949710+0000","last_unstale":"2024-04-26T21:38:21.099697+0000","last_undegraded":"2024-04-26T21:38:21.099697+0000","last_fullsized":"2024-04-26T21:38:21.099697+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:02.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200405+0000","last_change":"2024-04-26T21:38:18.948878+0000","last_active":"2024-04-26T21:38:22.200405+0000","last_peered":"2024-04-26T21:38:22.200405+0000","last_clean":"2024-04-26T21:38:22.200405+0000","last_became_active":"2024-04-26T21:38:18.948616+0000","last_became_peered":"2024-04-26T21:38:18.948616+0000","last_unstale":"2024-04-26T21:38:22.200405+0000","last_undegraded":"2024-04-26T21:38:22.200405+0000","last_fullsized":"2024-04-26T21:38:22.200405+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:03.766127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099576+0000","last_change":"2024-04-26T21:38:17.951499+0000","last_active":"2024-04-26T21:38:21.099576+0000","last_peered":"2024-04-26T21:38:21.099576+0000","last_clean":"2024-04-26T21:38:21.099576+0000","last_became_active":"2024-04-26T21:38:17.951190+0000","last_became_peered":"2024-04-26T21:38:17.951190+0000","last_unstale":"2024-04-26T21:38:21.099576+0000","last_undegraded":"2024-04-26T21:38:21.099576+0000","last_fullsized":"2024-04-26T21:38:21.099576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:46.534568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952994+0000","last_change":"2024-04-26T21:38:18.955078+0000","last_active":"2024-04-26T21:38:19.952994+0000","last_peered":"2024-04-26T21:38:19.952994+0000","last_clean":"2024-04-26T21:38:19.952994+0000","last_became_active":"2024-04-26T21:38:18.954813+0000","last_became_peered":"2024-04-26T21:38:18.954813+0000","last_unstale":"2024-04-26T21:38:19.952994+0000","last_undegraded":"2024-04-26T21:38:19.952994+0000","last_fullsized":"2024-04-26T21:38:19.952994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:58.067370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760316+0000","last_change":"2024-04-26T21:38:17.943246+0000","last_active":"2024-04-26T21:38:21.760316+0000","last_peered":"2024-04-26T21:38:21.760316+0000","last_clean":"2024-04-26T21:38:21.760316+0000","last_became_active":"2024-04-26T21:38:17.942975+0000","last_became_peered":"2024-04-26T21:38:17.942975+0000","last_unstale":"2024-04-26T21:38:21.760316+0000","last_undegraded":"2024-04-26T21:38:21.760316+0000","last_fullsized":"2024-04-26T21:38:21.760316+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.865600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199459+0000","last_change":"2024-04-26T21:38:18.948522+0000","last_active":"2024-04-26T21:38:22.199459+0000","last_peered":"2024-04-26T21:38:22.199459+0000","last_clean":"2024-04-26T21:38:22.199459+0000","last_became_active":"2024-04-26T21:38:18.948236+0000","last_became_peered":"2024-04-26T21:38:18.948236+0000","last_unstale":"2024-04-26T21:38:22.199459+0000","last_undegraded":"2024-04-26T21:38:22.199459+0000","last_fullsized":"2024-04-26T21:38:22.199459+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:48.116434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199539+0000","last_change":"2024-04-26T21:38:17.946995+0000","last_active":"2024-04-26T21:38:22.199539+0000","last_peered":"2024-04-26T21:38:22.199539+0000","last_clean":"2024-04-26T21:38:22.199539+0000","last_became_active":"2024-04-26T21:38:17.946697+0000","last_became_peered":"2024-04-26T21:38:17.946697+0000","last_unstale":"2024-04-26T21:38:22.199539+0000","last_undegraded":"2024-04-26T21:38:22.199539+0000","last_fullsized":"2024-04-26T21:38:22.199539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:31.341265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933414+0000","last_change":"2024-04-26T21:38:18.945327+0000","last_active":"2024-04-26T21:38:19.933414+0000","last_peered":"2024-04-26T21:38:19.933414+0000","last_clean":"2024-04-26T21:38:19.933414+0000","last_became_active":"2024-04-26T21:38:18.945055+0000","last_became_peered":"2024-04-26T21:38:18.945055+0000","last_unstale":"2024-04-26T21:38:19.933414+0000","last_undegraded":"2024-04-26T21:38:19.933414+0000","last_fullsized":"2024-04-26T21:38:19.933414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:54.872076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933870+0000","last_change":"2024-04-26T21:38:17.942558+0000","last_active":"2024-04-26T21:38:19.933870+0000","last_peered":"2024-04-26T21:38:19.933870+0000","last_clean":"2024-04-26T21:38:19.933870+0000","last_became_active":"2024-04-26T21:38:17.942267+0000","last_became_peered":"2024-04-26T21:38:17.942267+0000","last_unstale":"2024-04-26T21:38:19.933870+0000","last_undegraded":"2024-04-26T21:38:19.933870+0000","last_fullsized":"2024-04-26T21:38:19.933870+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:02.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953093+0000","last_change":"2024-04-26T21:38:18.954514+0000","last_active":"2024-04-26T21:38:19.953093+0000","last_peered":"2024-04-26T21:38:19.953093+0000","last_clean":"2024-04-26T21:38:19.953093+0000","last_became_active":"2024-04-26T21:38:18.953995+0000","last_became_peered":"2024-04-26T21:38:18.953995+0000","last_unstale":"2024-04-26T21:38:19.953093+0000","last_undegraded":"2024-04-26T21:38:19.953093+0000","last_fullsized":"2024-04-26T21:38:19.953093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:41.104015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199362+0000","last_change":"2024-04-26T21:38:17.948494+0000","last_active":"2024-04-26T21:38:22.199362+0000","last_peered":"2024-04-26T21:38:22.199362+0000","last_clean":"2024-04-26T21:38:22.199362+0000","last_became_active":"2024-04-26T21:38:17.948212+0000","last_became_peered":"2024-04-26T21:38:17.948212+0000","last_unstale":"2024-04-26T21:38:22.199362+0000","last_undegraded":"2024-04-26T21:38:22.199362+0000","last_fullsized":"2024-04-26T21:38:22.199362+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.865600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199695+0000","last_change":"2024-04-26T21:38:18.951827+0000","last_active":"2024-04-26T21:38:22.199695+0000","last_peered":"2024-04-26T21:38:22.199695+0000","last_clean":"2024-04-26T21:38:22.199695+0000","last_became_active":"2024-04-26T21:38:18.951536+0000","last_became_peered":"2024-04-26T21:38:18.951536+0000","last_unstale":"2024-04-26T21:38:22.199695+0000","last_undegraded":"2024-04-26T21:38:22.199695+0000","last_fullsized":"2024-04-26T21:38:22.199695+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:47.732055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933770+0000","last_change":"2024-04-26T21:38:17.951721+0000","last_active":"2024-04-26T21:38:19.933770+0000","last_peered":"2024-04-26T21:38:19.933770+0000","last_clean":"2024-04-26T21:38:19.933770+0000","last_became_active":"2024-04-26T21:38:17.951400+0000","last_became_peered":"2024-04-26T21:38:17.951400+0000","last_unstale":"2024-04-26T21:38:19.933770+0000","last_undegraded":"2024-04-26T21:38:19.933770+0000","last_fullsized":"2024-04-26T21:38:19.933770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:38.435914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933808+0000","last_change":"2024-04-26T21:38:18.954039+0000","last_active":"2024-04-26T21:38:19.933808+0000","last_peered":"2024-04-26T21:38:19.933808+0000","last_clean":"2024-04-26T21:38:19.933808+0000","last_became_active":"2024-04-26T21:38:18.953771+0000","last_became_peered":"2024-04-26T21:38:18.953771+0000","last_unstale":"2024-04-26T21:38:19.933808+0000","last_undegraded":"2024-04-26T21:38:19.933808+0000","last_fullsized":"2024-04-26T21:38:19.933808+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:29.401057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933943+0000","last_change":"2024-04-26T21:38:17.945997+0000","last_active":"2024-04-26T21:38:19.933943+0000","last_peered":"2024-04-26T21:38:19.933943+0000","last_clean":"2024-04-26T21:38:19.933943+0000","last_became_active":"2024-04-26T21:38:17.945730+0000","last_became_peered":"2024-04-26T21:38:17.945730+0000","last_unstale":"2024-04-26T21:38:19.933943+0000","last_undegraded":"2024-04-26T21:38:19.933943+0000","last_fullsized":"2024-04-26T21:38:19.933943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:47.178650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933714+0000","last_change":"2024-04-26T21:38:18.955861+0000","last_active":"2024-04-26T21:38:19.933714+0000","last_peered":"2024-04-26T21:38:19.933714+0000","last_clean":"2024-04-26T21:38:19.933714+0000","last_became_active":"2024-04-26T21:38:18.955456+0000","last_became_peered":"2024-04-26T21:38:18.955456+0000","last_unstale":"2024-04-26T21:38:19.933714+0000","last_undegraded":"2024-04-26T21:38:19.933714+0000","last_fullsized":"2024-04-26T21:38:19.933714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.399686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199740+0000","last_change":"2024-04-26T21:38:17.949610+0000","last_active":"2024-04-26T21:38:22.199740+0000","last_peered":"2024-04-26T21:38:22.199740+0000","last_clean":"2024-04-26T21:38:22.199740+0000","last_became_active":"2024-04-26T21:38:17.949247+0000","last_became_peered":"2024-04-26T21:38:17.949247+0000","last_unstale":"2024-04-26T21:38:22.199740+0000","last_undegraded":"2024-04-26T21:38:22.199740+0000","last_fullsized":"2024-04-26T21:38:22.199740+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:57.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953206+0000","last_change":"2024-04-26T21:38:18.955515+0000","last_active":"2024-04-26T21:38:19.953206+0000","last_peered":"2024-04-26T21:38:19.953206+0000","last_clean":"2024-04-26T21:38:19.953206+0000","last_became_active":"2024-04-26T21:38:18.955243+0000","last_became_peered":"2024-04-26T21:38:18.955243+0000","last_unstale":"2024-04-26T21:38:19.953206+0000","last_undegraded":"2024-04-26T21:38:19.953206+0000","last_fullsized":"2024-04-26T21:38:19.953206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:02.866419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934541+0000","last_change":"2024-04-26T21:38:17.946463+0000","last_active":"2024-04-26T21:38:19.934541+0000","last_peered":"2024-04-26T21:38:19.934541+0000","last_clean":"2024-04-26T21:38:19.934541+0000","last_became_active":"2024-04-26T21:38:17.946182+0000","last_became_peered":"2024-04-26T21:38:17.946182+0000","last_unstale":"2024-04-26T21:38:19.934541+0000","last_undegraded":"2024-04-26T21:38:19.934541+0000","last_fullsized":"2024-04-26T21:38:19.934541+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:58.559420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952722+0000","last_change":"2024-04-26T21:38:18.956532+0000","last_active":"2024-04-26T21:38:19.952722+0000","last_peered":"2024-04-26T21:38:19.952722+0000","last_clean":"2024-04-26T21:38:19.952722+0000","last_became_active":"2024-04-26T21:38:18.956278+0000","last_became_peered":"2024-04-26T21:38:18.956278+0000","last_unstale":"2024-04-26T21:38:19.952722+0000","last_undegraded":"2024-04-26T21:38:19.952722+0000","last_fullsized":"2024-04-26T21:38:19.952722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:04.002778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099629+0000","last_change":"2024-04-26T21:38:17.949728+0000","last_active":"2024-04-26T21:38:21.099629+0000","last_peered":"2024-04-26T21:38:21.099629+0000","last_clean":"2024-04-26T21:38:21.099629+0000","last_became_active":"2024-04-26T21:38:17.949450+0000","last_became_peered":"2024-04-26T21:38:17.949450+0000","last_unstale":"2024-04-26T21:38:21.099629+0000","last_undegraded":"2024-04-26T21:38:21.099629+0000","last_fullsized":"2024-04-26T21:38:21.099629+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:38.435914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200066+0000","last_change":"2024-04-26T21:38:18.952550+0000","last_active":"2024-04-26T21:38:22.200066+0000","last_peered":"2024-04-26T21:38:22.200066+0000","last_clean":"2024-04-26T21:38:22.200066+0000","last_became_active":"2024-04-26T21:38:18.952263+0000","last_became_peered":"2024-04-26T21:38:18.952263+0000","last_unstale":"2024-04-26T21:38:22.200066+0000","last_undegraded":"2024-04-26T21:38:22.200066+0000","last_fullsized":"2024-04-26T21:38:22.200066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:49.503375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760181+0000","last_change":"2024-04-26T21:38:17.937457+0000","last_active":"2024-04-26T21:38:21.760181+0000","last_peered":"2024-04-26T21:38:21.760181+0000","last_clean":"2024-04-26T21:38:21.760181+0000","last_became_active":"2024-04-26T21:38:17.937039+0000","last_became_peered":"2024-04-26T21:38:17.937039+0000","last_unstale":"2024-04-26T21:38:21.760181+0000","last_undegraded":"2024-04-26T21:38:21.760181+0000","last_fullsized":"2024-04-26T21:38:21.760181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:31.341265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933979+0000","last_change":"2024-04-26T21:38:18.946583+0000","last_active":"2024-04-26T21:38:19.933979+0000","last_peered":"2024-04-26T21:38:19.933979+0000","last_clean":"2024-04-26T21:38:19.933979+0000","last_became_active":"2024-04-26T21:38:18.946216+0000","last_became_peered":"2024-04-26T21:38:18.946216+0000","last_unstale":"2024-04-26T21:38:19.933979+0000","last_undegraded":"2024-04-26T21:38:19.933979+0000","last_fullsized":"2024-04-26T21:38:19.933979+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:40.519330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760157+0000","last_change":"2024-04-26T21:38:17.944966+0000","last_active":"2024-04-26T21:38:21.760157+0000","last_peered":"2024-04-26T21:38:21.760157+0000","last_clean":"2024-04-26T21:38:21.760157+0000","last_became_active":"2024-04-26T21:38:17.944185+0000","last_became_peered":"2024-04-26T21:38:17.944185+0000","last_unstale":"2024-04-26T21:38:21.760157+0000","last_undegraded":"2024-04-26T21:38:21.760157+0000","last_fullsized":"2024-04-26T21:38:21.760157+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:44.393210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933759+0000","last_change":"2024-04-26T21:38:18.945797+0000","last_active":"2024-04-26T21:38:19.933759+0000","last_peered":"2024-04-26T21:38:19.933759+0000","last_clean":"2024-04-26T21:38:19.933759+0000","last_became_active":"2024-04-26T21:38:18.945522+0000","last_became_peered":"2024-04-26T21:38:18.945522+0000","last_unstale":"2024-04-26T21:38:19.933759+0000","last_undegraded":"2024-04-26T21:38:19.933759+0000","last_fullsized":"2024-04-26T21:38:19.933759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:01.964703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933793+0000","last_change":"2024-04-26T21:38:17.945734+0000","last_active":"2024-04-26T21:38:19.933793+0000","last_peered":"2024-04-26T21:38:19.933793+0000","last_clean":"2024-04-26T21:38:19.933793+0000","last_became_active":"2024-04-26T21:38:17.945450+0000","last_became_peered":"2024-04-26T21:38:17.945450+0000","last_unstale":"2024-04-26T21:38:19.933793+0000","last_undegraded":"2024-04-26T21:38:19.933793+0000","last_fullsized":"2024-04-26T21:38:19.933793+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:50.457447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953062+0000","last_change":"2024-04-26T21:38:18.957994+0000","last_active":"2024-04-26T21:38:19.953062+0000","last_peered":"2024-04-26T21:38:19.953062+0000","last_clean":"2024-04-26T21:38:19.953062+0000","last_became_active":"2024-04-26T21:38:18.957733+0000","last_became_peered":"2024-04-26T21:38:18.957733+0000","last_unstale":"2024-04-26T21:38:19.953062+0000","last_undegraded":"2024-04-26T21:38:19.953062+0000","last_fullsized":"2024-04-26T21:38:19.953062+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:03.784664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.759951+0000","last_change":"2024-04-26T21:38:17.944876+0000","last_active":"2024-04-26T21:38:21.759951+0000","last_peered":"2024-04-26T21:38:21.759951+0000","last_clean":"2024-04-26T21:38:21.759951+0000","last_became_active":"2024-04-26T21:38:17.944590+0000","last_became_peered":"2024-04-26T21:38:17.944590+0000","last_unstale":"2024-04-26T21:38:21.759951+0000","last_undegraded":"2024-04-26T21:38:21.759951+0000","last_fullsized":"2024-04-26T21:38:21.759951+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:27.259577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":255,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099207+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:38:21.099207+0000","last_peered":"2024-04-26T21:38:21.099207+0000","last_clean":"2024-04-26T21:38:21.099207+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:38:21.099207+0000","last_undegraded":"2024-04-26T21:38:21.099207+0000","last_fullsized":"2024-04-26T21:38:21.099207+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28396,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343444,"statfs":{"total":96636764160,"available":96607686656,"internally_reserved":0,"allocated":1409024,"data_stored":955914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":364804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":364792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":366382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":364770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":364770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":367146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28376,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343464,"statfs":{"total":96636764160,"available":96607707136,"internally_reserved":0,"allocated":1380352,"data_stored":948446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:38:27.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:38:27.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.542+0000 7fbbfd8f1640 1 Processor -- start 2024-04-26T21:38:27.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.542+0000 7fbbfd8f1640 1 -- start start 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbfd8f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbbf8156460 0x7fbbf80cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbfd8f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf806fea0 0x7fbbf8072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbfd8f1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf8156840 con 0x7fbbf80cba10 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbfd8f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbbf80cbf20 con 0x7fbbf8156460 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbfd8f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf80cc200 con 0x7fbbf806fea0 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbf6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbbf8156460 0x7fbbf80cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbf6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbbf8156460 0x7fbbf80cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46898/0 (socket says 172.21.15.40:46898) 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbf6ffd640 1 -- 172.21.15.40:0/756203074 learned_addr learned my addr 172.21.15.40:0/756203074 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.543+0000 7fbbf77fe640 1 --2- 172.21.15.40:0/756203074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf806fea0 0x7fbbf8072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.544+0000 7fbbf6ffd640 1 -- 172.21.15.40:0/756203074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf806fea0 msgr2=0x7fbbf8072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:27.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.544+0000 7fbbf6ffd640 1 --2- 172.21.15.40:0/756203074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf806fea0 0x7fbbf8072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.544+0000 7fbbf6ffd640 1 -- 172.21.15.40:0/756203074 >> v1:172.21.15.184:6789/0 conn(0x7fbbf80cba10 legacy=0x7fbbf806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.544+0000 7fbbf6ffd640 1 -- 172.21.15.40:0/756203074 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf80cc4e0 con 0x7fbbf8156460 2024-04-26T21:38:27.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.544+0000 7fbbf6ffd640 1 --2- 172.21.15.40:0/756203074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf8156460 0x7fbbf80cb4d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800bb10 tx=0x7fbbe802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f356b005b32914f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:27.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbf5ffb640 1 -- 172.21.15.40:0/756203074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8036020 con 0x7fbbf8156460 2024-04-26T21:38:27.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbf5ffb640 1 -- 172.21.15.40:0/756203074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe8032820 con 0x7fbbf8156460 2024-04-26T21:38:27.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbf5ffb640 1 -- 172.21.15.40:0/756203074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8032bf0 con 0x7fbbf8156460 2024-04-26T21:38:27.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/756203074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf8156460 msgr2=0x7fbbf80cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/756203074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf8156460 0x7fbbf80cb4d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800bb10 tx=0x7fbbe802c920 comp rx=0 tx=0).stop 2024-04-26T21:38:27.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/756203074 shutdown_connections 2024-04-26T21:38:27.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/756203074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf806fea0 0x7fbbf8072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/756203074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf8156460 0x7fbbf80cb4d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/756203074 >> 172.21.15.40:0/756203074 conn(0x7fbbf80d0be0 msgr2=0x7fbbf80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:27.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/756203074 shutdown_connections 2024-04-26T21:38:27.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.545+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/756203074 wait complete. 2024-04-26T21:38:27.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.546+0000 7fbbfd8f1640 1 Processor -- start 2024-04-26T21:38:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.547+0000 7fbbfd8f1640 1 -- start start 2024-04-26T21:38:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.547+0000 7fbbfd8f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf806fea0 0x7fbbf80c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.547+0000 7fbbfd8f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf80cba10 0x7fbbf81f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.547+0000 7fbbfd8f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbf8156460 0x7fbbf80cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.547+0000 7fbbfd8f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf806e260 con 0x7fbbf806fea0 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.547+0000 7fbbfd8f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8072700 con 0x7fbbf8156460 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.547+0000 7fbbfd8f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf806e3e0 con 0x7fbbf80cba10 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.547+0000 7fbbf6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf806fea0 0x7fbbf80c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf806fea0 0x7fbbf80c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46908/0 (socket says 172.21.15.40:46908) 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf6ffd640 1 -- 172.21.15.40:0/983190192 learned_addr learned my addr 172.21.15.40:0/983190192 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf67fc640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf80cba10 0x7fbbf81f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf77fe640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbf8156460 0x7fbbf80cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf6ffd640 1 -- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf80cba10 msgr2=0x7fbbf81f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf6ffd640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf80cba10 0x7fbbf81f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf6ffd640 1 -- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbf8156460 msgr2=0x7fbbf80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf6ffd640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbf8156460 0x7fbbf80cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf6ffd640 1 -- 172.21.15.40:0/983190192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf80c9400 con 0x7fbbf806fea0 2024-04-26T21:38:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbf6ffd640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf806fea0 0x7fbbf80c8ba0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbbe802c960 tx=0x7fbbe8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:27.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbdffff640 1 -- 172.21.15.40:0/983190192 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8042070 con 0x7fbbf806fea0 2024-04-26T21:38:27.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbdffff640 1 -- 172.21.15.40:0/983190192 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe8031cb0 con 0x7fbbf806fea0 2024-04-26T21:38:27.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf80c96e0 con 0x7fbbf806fea0 2024-04-26T21:38:27.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbdffff640 1 -- 172.21.15.40:0/983190192 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8036040 con 0x7fbbf806fea0 2024-04-26T21:38:27.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.548+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf80c9ca0 con 0x7fbbf806fea0 2024-04-26T21:38:27.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.549+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf80694f0 con 0x7fbbf806fea0 2024-04-26T21:38:27.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.550+0000 7fbbdffff640 1 -- 172.21.15.40:0/983190192 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbe8045070 con 0x7fbbf806fea0 2024-04-26T21:38:27.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.550+0000 7fbbdffff640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe406c8f0 0x7fbbe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.550+0000 7fbbdffff640 1 -- 172.21.15.40:0/983190192 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fbbe80ac7c0 con 0x7fbbf806fea0 2024-04-26T21:38:27.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.551+0000 7fbbf67fc640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe406c8f0 0x7fbbe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.551+0000 7fbbf67fc640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe406c8f0 0x7fbbe406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbbf81f8780 tx=0x7fbbec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.555+0000 7fbbdffff640 1 -- 172.21.15.40:0/983190192 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbbe802f070 con 0x7fbbf806fea0 2024-04-26T21:38:27.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.736+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbbf8073ae0 con 0x7fbbe406c8f0 2024-04-26T21:38:27.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.743+0000 7fbbdffff640 1 -- 172.21.15.40:0/983190192 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252019 (secure 0 0 0) 0x7fbbf8073ae0 con 0x7fbbe406c8f0 2024-04-26T21:38:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:38:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe406c8f0 msgr2=0x7fbbe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe406c8f0 0x7fbbe406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbbf81f8780 tx=0x7fbbec009040 comp rx=0 tx=0).stop 2024-04-26T21:38:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf806fea0 msgr2=0x7fbbf80c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf806fea0 0x7fbbf80c8ba0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbbe802c960 tx=0x7fbbe8004730 comp rx=0 tx=0).stop 2024-04-26T21:38:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 shutdown_connections 2024-04-26T21:38:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe406c8f0 0x7fbbe406edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbf8156460 0x7fbbf80cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbf80cba10 0x7fbbf81f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 --2- 172.21.15.40:0/983190192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbf806fea0 0x7fbbf80c8ba0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 >> 172.21.15.40:0/983190192 conn(0x7fbbf80d0be0 msgr2=0x7fbbf8072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 shutdown_connections 2024-04-26T21:38:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.746+0000 7fbbfd8f1640 1 -- 172.21.15.40:0/983190192 wait complete. 2024-04-26T21:38:27.758 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-04-26T21:38:26.040966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223620,"kb_used_data":7452,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751100,"statfs":{"total":773094113280,"available":772865126400,"internally_reserved":0,"allocated":7630848,"data_stored":4097024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002239"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934419+0000","last_change":"2024-04-26T21:38:18.959438+0000","last_active":"2024-04-26T21:38:19.934419+0000","last_peered":"2024-04-26T21:38:19.934419+0000","last_clean":"2024-04-26T21:38:19.934419+0000","last_became_active":"2024-04-26T21:38:18.959052+0000","last_became_peered":"2024-04-26T21:38:18.959052+0000","last_unstale":"2024-04-26T21:38:19.934419+0000","last_undegraded":"2024-04-26T21:38:19.934419+0000","last_fullsized":"2024-04-26T21:38:19.934419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.692628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934276+0000","last_change":"2024-04-26T21:38:18.956022+0000","last_active":"2024-04-26T21:38:19.934276+0000","last_peered":"2024-04-26T21:38:19.934276+0000","last_clean":"2024-04-26T21:38:19.934276+0000","last_became_active":"2024-04-26T21:38:18.955671+0000","last_became_peered":"2024-04-26T21:38:18.955671+0000","last_unstale":"2024-04-26T21:38:19.934276+0000","last_undegraded":"2024-04-26T21:38:19.934276+0000","last_fullsized":"2024-04-26T21:38:19.934276+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:47.541044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099392+0000","last_change":"2024-04-26T21:38:18.957492+0000","last_active":"2024-04-26T21:38:21.099392+0000","last_peered":"2024-04-26T21:38:21.099392+0000","last_clean":"2024-04-26T21:38:21.099392+0000","last_became_active":"2024-04-26T21:38:18.957167+0000","last_became_peered":"2024-04-26T21:38:18.957167+0000","last_unstale":"2024-04-26T21:38:21.099392+0000","last_undegraded":"2024-04-26T21:38:21.099392+0000","last_fullsized":"2024-04-26T21:38:21.099392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:28.266053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200146+0000","last_change":"2024-04-26T21:38:18.948395+0000","last_active":"2024-04-26T21:38:22.200146+0000","last_peered":"2024-04-26T21:38:22.200146+0000","last_clean":"2024-04-26T21:38:22.200146+0000","last_became_active":"2024-04-26T21:38:18.948141+0000","last_became_peered":"2024-04-26T21:38:18.948141+0000","last_unstale":"2024-04-26T21:38:22.200146+0000","last_undegraded":"2024-04-26T21:38:22.200146+0000","last_fullsized":"2024-04-26T21:38:22.200146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:36.862335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953081+0000","last_change":"2024-04-26T21:38:18.952537+0000","last_active":"2024-04-26T21:38:19.953081+0000","last_peered":"2024-04-26T21:38:19.953081+0000","last_clean":"2024-04-26T21:38:19.953081+0000","last_became_active":"2024-04-26T21:38:18.952286+0000","last_became_peered":"2024-04-26T21:38:18.952286+0000","last_unstale":"2024-04-26T21:38:19.953081+0000","last_undegraded":"2024-04-26T21:38:19.953081+0000","last_fullsized":"2024-04-26T21:38:19.953081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:48.185126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199646+0000","last_change":"2024-04-26T21:38:18.951837+0000","last_active":"2024-04-26T21:38:22.199646+0000","last_peered":"2024-04-26T21:38:22.199646+0000","last_clean":"2024-04-26T21:38:22.199646+0000","last_became_active":"2024-04-26T21:38:18.951438+0000","last_became_peered":"2024-04-26T21:38:18.951438+0000","last_unstale":"2024-04-26T21:38:22.199646+0000","last_undegraded":"2024-04-26T21:38:22.199646+0000","last_fullsized":"2024-04-26T21:38:22.199646+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:50.681412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933590+0000","last_change":"2024-04-26T21:38:18.944281+0000","last_active":"2024-04-26T21:38:19.933590+0000","last_peered":"2024-04-26T21:38:19.933590+0000","last_clean":"2024-04-26T21:38:19.933590+0000","last_became_active":"2024-04-26T21:38:18.943978+0000","last_became_peered":"2024-04-26T21:38:18.943978+0000","last_unstale":"2024-04-26T21:38:19.933590+0000","last_undegraded":"2024-04-26T21:38:19.933590+0000","last_fullsized":"2024-04-26T21:38:19.933590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:36.157494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933412+0000","last_change":"2024-04-26T21:38:18.943968+0000","last_active":"2024-04-26T21:38:19.933412+0000","last_peered":"2024-04-26T21:38:19.933412+0000","last_clean":"2024-04-26T21:38:19.933412+0000","last_became_active":"2024-04-26T21:38:18.943684+0000","last_became_peered":"2024-04-26T21:38:18.943684+0000","last_unstale":"2024-04-26T21:38:19.933412+0000","last_undegraded":"2024-04-26T21:38:19.933412+0000","last_fullsized":"2024-04-26T21:38:19.933412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:53.023152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099820+0000","last_change":"2024-04-26T21:38:18.955952+0000","last_active":"2024-04-26T21:38:21.099820+0000","last_peered":"2024-04-26T21:38:21.099820+0000","last_clean":"2024-04-26T21:38:21.099820+0000","last_became_active":"2024-04-26T21:38:18.955643+0000","last_became_peered":"2024-04-26T21:38:18.955643+0000","last_unstale":"2024-04-26T21:38:21.099820+0000","last_undegraded":"2024-04-26T21:38:21.099820+0000","last_fullsized":"2024-04-26T21:38:21.099820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:17.991870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099673+0000","last_change":"2024-04-26T21:38:18.957998+0000","last_active":"2024-04-26T21:38:21.099673+0000","last_peered":"2024-04-26T21:38:21.099673+0000","last_clean":"2024-04-26T21:38:21.099673+0000","last_became_active":"2024-04-26T21:38:18.957675+0000","last_became_peered":"2024-04-26T21:38:18.957675+0000","last_unstale":"2024-04-26T21:38:21.099673+0000","last_undegraded":"2024-04-26T21:38:21.099673+0000","last_fullsized":"2024-04-26T21:38:21.099673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:11.176375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200224+0000","last_change":"2024-04-26T21:38:18.959855+0000","last_active":"2024-04-26T21:38:22.200224+0000","last_peered":"2024-04-26T21:38:22.200224+0000","last_clean":"2024-04-26T21:38:22.200224+0000","last_became_active":"2024-04-26T21:38:18.959580+0000","last_became_peered":"2024-04-26T21:38:18.959580+0000","last_unstale":"2024-04-26T21:38:22.200224+0000","last_undegraded":"2024-04-26T21:38:22.200224+0000","last_fullsized":"2024-04-26T21:38:22.200224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:51.315426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933506+0000","last_change":"2024-04-26T21:38:18.942690+0000","last_active":"2024-04-26T21:38:19.933506+0000","last_peered":"2024-04-26T21:38:19.933506+0000","last_clean":"2024-04-26T21:38:19.933506+0000","last_became_active":"2024-04-26T21:38:18.942377+0000","last_became_peered":"2024-04-26T21:38:18.942377+0000","last_unstale":"2024-04-26T21:38:19.933506+0000","last_undegraded":"2024-04-26T21:38:19.933506+0000","last_fullsized":"2024-04-26T21:38:19.933506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:32.347741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933919+0000","last_change":"2024-04-26T21:38:18.947832+0000","last_active":"2024-04-26T21:38:19.933919+0000","last_peered":"2024-04-26T21:38:19.933919+0000","last_clean":"2024-04-26T21:38:19.933919+0000","last_became_active":"2024-04-26T21:38:18.947561+0000","last_became_peered":"2024-04-26T21:38:18.947561+0000","last_unstale":"2024-04-26T21:38:19.933919+0000","last_undegraded":"2024-04-26T21:38:19.933919+0000","last_fullsized":"2024-04-26T21:38:19.933919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:31.910540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099598+0000","last_change":"2024-04-26T21:38:18.952884+0000","last_active":"2024-04-26T21:38:21.099598+0000","last_peered":"2024-04-26T21:38:21.099598+0000","last_clean":"2024-04-26T21:38:21.099598+0000","last_became_active":"2024-04-26T21:38:18.952624+0000","last_became_peered":"2024-04-26T21:38:18.952624+0000","last_unstale":"2024-04-26T21:38:21.099598+0000","last_undegraded":"2024-04-26T21:38:21.099598+0000","last_fullsized":"2024-04-26T21:38:21.099598+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:31.910540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099381+0000","last_change":"2024-04-26T21:38:18.960701+0000","last_active":"2024-04-26T21:38:21.099381+0000","last_peered":"2024-04-26T21:38:21.099381+0000","last_clean":"2024-04-26T21:38:21.099381+0000","last_became_active":"2024-04-26T21:38:18.960425+0000","last_became_peered":"2024-04-26T21:38:18.960425+0000","last_unstale":"2024-04-26T21:38:21.099381+0000","last_undegraded":"2024-04-26T21:38:21.099381+0000","last_fullsized":"2024-04-26T21:38:21.099381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:32.347741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933471+0000","last_change":"2024-04-26T21:38:18.946641+0000","last_active":"2024-04-26T21:38:19.933471+0000","last_peered":"2024-04-26T21:38:19.933471+0000","last_clean":"2024-04-26T21:38:19.933471+0000","last_became_active":"2024-04-26T21:38:18.946372+0000","last_became_peered":"2024-04-26T21:38:18.946372+0000","last_unstale":"2024-04-26T21:38:19.933471+0000","last_undegraded":"2024-04-26T21:38:19.933471+0000","last_fullsized":"2024-04-26T21:38:19.933471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:30.460324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934456+0000","last_change":"2024-04-26T21:38:18.949643+0000","last_active":"2024-04-26T21:38:19.934456+0000","last_peered":"2024-04-26T21:38:19.934456+0000","last_clean":"2024-04-26T21:38:19.934456+0000","last_became_active":"2024-04-26T21:38:18.949291+0000","last_became_peered":"2024-04-26T21:38:18.949291+0000","last_unstale":"2024-04-26T21:38:19.934456+0000","last_undegraded":"2024-04-26T21:38:19.934456+0000","last_fullsized":"2024-04-26T21:38:19.934456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:57.536861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200309+0000","last_change":"2024-04-26T21:38:18.948352+0000","last_active":"2024-04-26T21:38:22.200309+0000","last_peered":"2024-04-26T21:38:22.200309+0000","last_clean":"2024-04-26T21:38:22.200309+0000","last_became_active":"2024-04-26T21:38:18.947746+0000","last_became_peered":"2024-04-26T21:38:18.947746+0000","last_unstale":"2024-04-26T21:38:22.200309+0000","last_undegraded":"2024-04-26T21:38:22.200309+0000","last_fullsized":"2024-04-26T21:38:22.200309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:23.266362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953127+0000","last_change":"2024-04-26T21:38:18.947674+0000","last_active":"2024-04-26T21:38:19.953127+0000","last_peered":"2024-04-26T21:38:19.953127+0000","last_clean":"2024-04-26T21:38:19.953127+0000","last_became_active":"2024-04-26T21:38:18.947397+0000","last_became_peered":"2024-04-26T21:38:18.947397+0000","last_unstale":"2024-04-26T21:38:19.953127+0000","last_undegraded":"2024-04-26T21:38:19.953127+0000","last_fullsized":"2024-04-26T21:38:19.953127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:39.442390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760361+0000","last_change":"2024-04-26T21:38:18.951143+0000","last_active":"2024-04-26T21:38:21.760361+0000","last_peered":"2024-04-26T21:38:21.760361+0000","last_clean":"2024-04-26T21:38:21.760361+0000","last_became_active":"2024-04-26T21:38:18.950851+0000","last_became_peered":"2024-04-26T21:38:18.950851+0000","last_unstale":"2024-04-26T21:38:21.760361+0000","last_undegraded":"2024-04-26T21:38:21.760361+0000","last_fullsized":"2024-04-26T21:38:21.760361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:20.945070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933961+0000","last_change":"2024-04-26T21:38:18.941201+0000","last_active":"2024-04-26T21:38:19.933961+0000","last_peered":"2024-04-26T21:38:19.933961+0000","last_clean":"2024-04-26T21:38:19.933961+0000","last_became_active":"2024-04-26T21:38:18.940870+0000","last_became_peered":"2024-04-26T21:38:18.940870+0000","last_unstale":"2024-04-26T21:38:19.933961+0000","last_undegraded":"2024-04-26T21:38:19.933961+0000","last_fullsized":"2024-04-26T21:38:19.933961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:11.176375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760066+0000","last_change":"2024-04-26T21:38:18.951917+0000","last_active":"2024-04-26T21:38:21.760066+0000","last_peered":"2024-04-26T21:38:21.760066+0000","last_clean":"2024-04-26T21:38:21.760066+0000","last_became_active":"2024-04-26T21:38:18.951657+0000","last_became_peered":"2024-04-26T21:38:18.951657+0000","last_unstale":"2024-04-26T21:38:21.760066+0000","last_undegraded":"2024-04-26T21:38:21.760066+0000","last_fullsized":"2024-04-26T21:38:21.760066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:50.681412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760043+0000","last_change":"2024-04-26T21:38:18.952718+0000","last_active":"2024-04-26T21:38:21.760043+0000","last_peered":"2024-04-26T21:38:21.760043+0000","last_clean":"2024-04-26T21:38:21.760043+0000","last_became_active":"2024-04-26T21:38:18.952436+0000","last_became_peered":"2024-04-26T21:38:18.952436+0000","last_unstale":"2024-04-26T21:38:21.760043+0000","last_undegraded":"2024-04-26T21:38:21.760043+0000","last_fullsized":"2024-04-26T21:38:21.760043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:48.116434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099278+0000","last_change":"2024-04-26T21:38:18.954925+0000","last_active":"2024-04-26T21:38:21.099278+0000","last_peered":"2024-04-26T21:38:21.099278+0000","last_clean":"2024-04-26T21:38:21.099278+0000","last_became_active":"2024-04-26T21:38:18.953754+0000","last_became_peered":"2024-04-26T21:38:18.953754+0000","last_unstale":"2024-04-26T21:38:21.099278+0000","last_undegraded":"2024-04-26T21:38:21.099278+0000","last_fullsized":"2024-04-26T21:38:21.099278+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:14.110732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933683+0000","last_change":"2024-04-26T21:38:18.943836+0000","last_active":"2024-04-26T21:38:19.933683+0000","last_peered":"2024-04-26T21:38:19.933683+0000","last_clean":"2024-04-26T21:38:19.933683+0000","last_became_active":"2024-04-26T21:38:18.943496+0000","last_became_peered":"2024-04-26T21:38:18.943496+0000","last_unstale":"2024-04-26T21:38:19.933683+0000","last_undegraded":"2024-04-26T21:38:19.933683+0000","last_fullsized":"2024-04-26T21:38:19.933683+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:58.368085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934025+0000","last_change":"2024-04-26T21:38:18.944578+0000","last_active":"2024-04-26T21:38:19.934025+0000","last_peered":"2024-04-26T21:38:19.934025+0000","last_clean":"2024-04-26T21:38:19.934025+0000","last_became_active":"2024-04-26T21:38:18.944333+0000","last_became_peered":"2024-04-26T21:38:18.944333+0000","last_unstale":"2024-04-26T21:38:19.934025+0000","last_undegraded":"2024-04-26T21:38:19.934025+0000","last_fullsized":"2024-04-26T21:38:19.934025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:55.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200345+0000","last_change":"2024-04-26T21:38:18.946326+0000","last_active":"2024-04-26T21:38:22.200345+0000","last_peered":"2024-04-26T21:38:22.200345+0000","last_clean":"2024-04-26T21:38:22.200345+0000","last_became_active":"2024-04-26T21:38:18.945972+0000","last_became_peered":"2024-04-26T21:38:18.945972+0000","last_unstale":"2024-04-26T21:38:22.200345+0000","last_undegraded":"2024-04-26T21:38:22.200345+0000","last_fullsized":"2024-04-26T21:38:22.200345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:30.973393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934409+0000","last_change":"2024-04-26T21:38:18.962609+0000","last_active":"2024-04-26T21:38:19.934409+0000","last_peered":"2024-04-26T21:38:19.934409+0000","last_clean":"2024-04-26T21:38:19.934409+0000","last_became_active":"2024-04-26T21:38:18.962235+0000","last_became_peered":"2024-04-26T21:38:18.962235+0000","last_unstale":"2024-04-26T21:38:19.934409+0000","last_undegraded":"2024-04-26T21:38:19.934409+0000","last_fullsized":"2024-04-26T21:38:19.934409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:45.205083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099445+0000","last_change":"2024-04-26T21:38:18.952731+0000","last_active":"2024-04-26T21:38:21.099445+0000","last_peered":"2024-04-26T21:38:21.099445+0000","last_clean":"2024-04-26T21:38:21.099445+0000","last_became_active":"2024-04-26T21:38:18.952474+0000","last_became_peered":"2024-04-26T21:38:18.952474+0000","last_unstale":"2024-04-26T21:38:21.099445+0000","last_undegraded":"2024-04-26T21:38:21.099445+0000","last_fullsized":"2024-04-26T21:38:21.099445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.399686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933694+0000","last_change":"2024-04-26T21:38:18.943991+0000","last_active":"2024-04-26T21:38:19.933694+0000","last_peered":"2024-04-26T21:38:19.933694+0000","last_clean":"2024-04-26T21:38:19.933694+0000","last_became_active":"2024-04-26T21:38:18.943564+0000","last_became_peered":"2024-04-26T21:38:18.943564+0000","last_unstale":"2024-04-26T21:38:19.933694+0000","last_undegraded":"2024-04-26T21:38:19.933694+0000","last_fullsized":"2024-04-26T21:38:19.933694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:14.110732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199910+0000","last_change":"2024-04-26T21:38:18.944660+0000","last_active":"2024-04-26T21:38:22.199910+0000","last_peered":"2024-04-26T21:38:22.199910+0000","last_clean":"2024-04-26T21:38:22.199910+0000","last_became_active":"2024-04-26T21:38:18.944340+0000","last_became_peered":"2024-04-26T21:38:18.944340+0000","last_unstale":"2024-04-26T21:38:22.199910+0000","last_undegraded":"2024-04-26T21:38:22.199910+0000","last_fullsized":"2024-04-26T21:38:22.199910+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:29.814338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199386+0000","last_change":"2024-04-26T21:38:18.950340+0000","last_active":"2024-04-26T21:38:22.199386+0000","last_peered":"2024-04-26T21:38:22.199386+0000","last_clean":"2024-04-26T21:38:22.199386+0000","last_became_active":"2024-04-26T21:38:18.950032+0000","last_became_peered":"2024-04-26T21:38:18.950032+0000","last_unstale":"2024-04-26T21:38:22.199386+0000","last_undegraded":"2024-04-26T21:38:22.199386+0000","last_fullsized":"2024-04-26T21:38:22.199386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:07.586134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952983+0000","last_change":"2024-04-26T21:38:18.963455+0000","last_active":"2024-04-26T21:38:19.952983+0000","last_peered":"2024-04-26T21:38:19.952983+0000","last_clean":"2024-04-26T21:38:19.952983+0000","last_became_active":"2024-04-26T21:38:18.963181+0000","last_became_peered":"2024-04-26T21:38:18.963181+0000","last_unstale":"2024-04-26T21:38:19.952983+0000","last_undegraded":"2024-04-26T21:38:19.952983+0000","last_fullsized":"2024-04-26T21:38:19.952983+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:48.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934525+0000","last_change":"2024-04-26T21:38:17.948310+0000","last_active":"2024-04-26T21:38:19.934525+0000","last_peered":"2024-04-26T21:38:19.934525+0000","last_clean":"2024-04-26T21:38:19.934525+0000","last_became_active":"2024-04-26T21:38:17.948052+0000","last_became_peered":"2024-04-26T21:38:17.948052+0000","last_unstale":"2024-04-26T21:38:19.934525+0000","last_undegraded":"2024-04-26T21:38:19.934525+0000","last_fullsized":"2024-04-26T21:38:19.934525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:02.996302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200398+0000","last_change":"2024-04-26T21:38:18.951143+0000","last_active":"2024-04-26T21:38:22.200398+0000","last_peered":"2024-04-26T21:38:22.200398+0000","last_clean":"2024-04-26T21:38:22.200398+0000","last_became_active":"2024-04-26T21:38:18.950744+0000","last_became_peered":"2024-04-26T21:38:18.950744+0000","last_unstale":"2024-04-26T21:38:22.200398+0000","last_undegraded":"2024-04-26T21:38:22.200398+0000","last_fullsized":"2024-04-26T21:38:22.200398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:51.463923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199839+0000","last_change":"2024-04-26T21:38:17.949729+0000","last_active":"2024-04-26T21:38:22.199839+0000","last_peered":"2024-04-26T21:38:22.199839+0000","last_clean":"2024-04-26T21:38:22.199839+0000","last_became_active":"2024-04-26T21:38:17.949412+0000","last_became_peered":"2024-04-26T21:38:17.949412+0000","last_unstale":"2024-04-26T21:38:22.199839+0000","last_undegraded":"2024-04-26T21:38:22.199839+0000","last_fullsized":"2024-04-26T21:38:22.199839+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:44.393210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199802+0000","last_change":"2024-04-26T21:38:18.951483+0000","last_active":"2024-04-26T21:38:22.199802+0000","last_peered":"2024-04-26T21:38:22.199802+0000","last_clean":"2024-04-26T21:38:22.199802+0000","last_became_active":"2024-04-26T21:38:18.951192+0000","last_became_peered":"2024-04-26T21:38:18.951192+0000","last_unstale":"2024-04-26T21:38:22.199802+0000","last_undegraded":"2024-04-26T21:38:22.199802+0000","last_fullsized":"2024-04-26T21:38:22.199802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:33.979629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199707+0000","last_change":"2024-04-26T21:38:17.947141+0000","last_active":"2024-04-26T21:38:22.199707+0000","last_peered":"2024-04-26T21:38:22.199707+0000","last_clean":"2024-04-26T21:38:22.199707+0000","last_became_active":"2024-04-26T21:38:17.946847+0000","last_became_peered":"2024-04-26T21:38:17.946847+0000","last_unstale":"2024-04-26T21:38:22.199707+0000","last_undegraded":"2024-04-26T21:38:22.199707+0000","last_fullsized":"2024-04-26T21:38:22.199707+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:27.259577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099803+0000","last_change":"2024-04-26T21:38:18.961238+0000","last_active":"2024-04-26T21:38:21.099803+0000","last_peered":"2024-04-26T21:38:21.099803+0000","last_clean":"2024-04-26T21:38:21.099803+0000","last_became_active":"2024-04-26T21:38:18.960945+0000","last_became_peered":"2024-04-26T21:38:18.960945+0000","last_unstale":"2024-04-26T21:38:21.099803+0000","last_undegraded":"2024-04-26T21:38:21.099803+0000","last_fullsized":"2024-04-26T21:38:21.099803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:55.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199536+0000","last_change":"2024-04-26T21:38:17.949992+0000","last_active":"2024-04-26T21:38:22.199536+0000","last_peered":"2024-04-26T21:38:22.199536+0000","last_clean":"2024-04-26T21:38:22.199536+0000","last_became_active":"2024-04-26T21:38:17.949701+0000","last_became_peered":"2024-04-26T21:38:17.949701+0000","last_unstale":"2024-04-26T21:38:22.199536+0000","last_undegraded":"2024-04-26T21:38:22.199536+0000","last_fullsized":"2024-04-26T21:38:22.199536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:52.016675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933606+0000","last_change":"2024-04-26T21:38:18.936571+0000","last_active":"2024-04-26T21:38:19.933606+0000","last_peered":"2024-04-26T21:38:19.933606+0000","last_clean":"2024-04-26T21:38:19.933606+0000","last_became_active":"2024-04-26T21:38:18.936209+0000","last_became_peered":"2024-04-26T21:38:18.936209+0000","last_unstale":"2024-04-26T21:38:19.933606+0000","last_undegraded":"2024-04-26T21:38:19.933606+0000","last_fullsized":"2024-04-26T21:38:19.933606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:48.107975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760229+0000","last_change":"2024-04-26T21:38:17.947017+0000","last_active":"2024-04-26T21:38:21.760229+0000","last_peered":"2024-04-26T21:38:21.760229+0000","last_clean":"2024-04-26T21:38:21.760229+0000","last_became_active":"2024-04-26T21:38:17.946744+0000","last_became_peered":"2024-04-26T21:38:17.946744+0000","last_unstale":"2024-04-26T21:38:21.760229+0000","last_undegraded":"2024-04-26T21:38:21.760229+0000","last_fullsized":"2024-04-26T21:38:21.760229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:13.104255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933550+0000","last_change":"2024-04-26T21:38:18.945973+0000","last_active":"2024-04-26T21:38:19.933550+0000","last_peered":"2024-04-26T21:38:19.933550+0000","last_clean":"2024-04-26T21:38:19.933550+0000","last_became_active":"2024-04-26T21:38:18.945676+0000","last_became_peered":"2024-04-26T21:38:18.945676+0000","last_unstale":"2024-04-26T21:38:19.933550+0000","last_undegraded":"2024-04-26T21:38:19.933550+0000","last_fullsized":"2024-04-26T21:38:19.933550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:39.442390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.759997+0000","last_change":"2024-04-26T21:38:17.944477+0000","last_active":"2024-04-26T21:38:21.759997+0000","last_peered":"2024-04-26T21:38:21.759997+0000","last_clean":"2024-04-26T21:38:21.759997+0000","last_became_active":"2024-04-26T21:38:17.944149+0000","last_became_peered":"2024-04-26T21:38:17.944149+0000","last_unstale":"2024-04-26T21:38:21.759997+0000","last_undegraded":"2024-04-26T21:38:21.759997+0000","last_fullsized":"2024-04-26T21:38:21.759997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:52.016675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952780+0000","last_change":"2024-04-26T21:38:18.954534+0000","last_active":"2024-04-26T21:38:19.952780+0000","last_peered":"2024-04-26T21:38:19.952780+0000","last_clean":"2024-04-26T21:38:19.952780+0000","last_became_active":"2024-04-26T21:38:18.954227+0000","last_became_peered":"2024-04-26T21:38:18.954227+0000","last_unstale":"2024-04-26T21:38:19.952780+0000","last_undegraded":"2024-04-26T21:38:19.952780+0000","last_fullsized":"2024-04-26T21:38:19.952780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:01.934163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760193+0000","last_change":"2024-04-26T21:38:17.945215+0000","last_active":"2024-04-26T21:38:21.760193+0000","last_peered":"2024-04-26T21:38:21.760193+0000","last_clean":"2024-04-26T21:38:21.760193+0000","last_became_active":"2024-04-26T21:38:17.944579+0000","last_became_peered":"2024-04-26T21:38:17.944579+0000","last_unstale":"2024-04-26T21:38:21.760193+0000","last_undegraded":"2024-04-26T21:38:21.760193+0000","last_fullsized":"2024-04-26T21:38:21.760193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:57.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760177+0000","last_change":"2024-04-26T21:38:18.953771+0000","last_active":"2024-04-26T21:38:21.760177+0000","last_peered":"2024-04-26T21:38:21.760177+0000","last_clean":"2024-04-26T21:38:21.760177+0000","last_became_active":"2024-04-26T21:38:18.953342+0000","last_became_peered":"2024-04-26T21:38:18.953342+0000","last_unstale":"2024-04-26T21:38:21.760177+0000","last_undegraded":"2024-04-26T21:38:21.760177+0000","last_fullsized":"2024-04-26T21:38:21.760177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:47.732055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952723+0000","last_change":"2024-04-26T21:38:17.949823+0000","last_active":"2024-04-26T21:38:19.952723+0000","last_peered":"2024-04-26T21:38:19.952723+0000","last_clean":"2024-04-26T21:38:19.952723+0000","last_became_active":"2024-04-26T21:38:17.949531+0000","last_became_peered":"2024-04-26T21:38:17.949531+0000","last_unstale":"2024-04-26T21:38:19.952723+0000","last_undegraded":"2024-04-26T21:38:19.952723+0000","last_fullsized":"2024-04-26T21:38:19.952723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:14.240281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934418+0000","last_change":"2024-04-26T21:38:18.951856+0000","last_active":"2024-04-26T21:38:19.934418+0000","last_peered":"2024-04-26T21:38:19.934418+0000","last_clean":"2024-04-26T21:38:19.934418+0000","last_became_active":"2024-04-26T21:38:18.951557+0000","last_became_peered":"2024-04-26T21:38:18.951557+0000","last_unstale":"2024-04-26T21:38:19.934418+0000","last_undegraded":"2024-04-26T21:38:19.934418+0000","last_fullsized":"2024-04-26T21:38:19.934418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:04.188697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099892+0000","last_change":"2024-04-26T21:38:17.951863+0000","last_active":"2024-04-26T21:38:21.099892+0000","last_peered":"2024-04-26T21:38:21.099892+0000","last_clean":"2024-04-26T21:38:21.099892+0000","last_became_active":"2024-04-26T21:38:17.951250+0000","last_became_peered":"2024-04-26T21:38:17.951250+0000","last_unstale":"2024-04-26T21:38:21.099892+0000","last_undegraded":"2024-04-26T21:38:21.099892+0000","last_fullsized":"2024-04-26T21:38:21.099892+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:01.859943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760193+0000","last_change":"2024-04-26T21:38:18.950273+0000","last_active":"2024-04-26T21:38:21.760193+0000","last_peered":"2024-04-26T21:38:21.760193+0000","last_clean":"2024-04-26T21:38:21.760193+0000","last_became_active":"2024-04-26T21:38:18.949963+0000","last_became_peered":"2024-04-26T21:38:18.949963+0000","last_unstale":"2024-04-26T21:38:21.760193+0000","last_undegraded":"2024-04-26T21:38:21.760193+0000","last_fullsized":"2024-04-26T21:38:21.760193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:33.979629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200433+0000","last_change":"2024-04-26T21:38:17.952295+0000","last_active":"2024-04-26T21:38:22.200433+0000","last_peered":"2024-04-26T21:38:22.200433+0000","last_clean":"2024-04-26T21:38:22.200433+0000","last_became_active":"2024-04-26T21:38:17.951993+0000","last_became_peered":"2024-04-26T21:38:17.951993+0000","last_unstale":"2024-04-26T21:38:22.200433+0000","last_undegraded":"2024-04-26T21:38:22.200433+0000","last_fullsized":"2024-04-26T21:38:22.200433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:09.249196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953020+0000","last_change":"2024-04-26T21:38:18.951823+0000","last_active":"2024-04-26T21:38:19.953020+0000","last_peered":"2024-04-26T21:38:19.953020+0000","last_clean":"2024-04-26T21:38:19.953020+0000","last_became_active":"2024-04-26T21:38:18.951577+0000","last_became_peered":"2024-04-26T21:38:18.951577+0000","last_unstale":"2024-04-26T21:38:19.953020+0000","last_undegraded":"2024-04-26T21:38:19.953020+0000","last_fullsized":"2024-04-26T21:38:19.953020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:13.413771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099364+0000","last_change":"2024-04-26T21:38:17.951334+0000","last_active":"2024-04-26T21:38:21.099364+0000","last_peered":"2024-04-26T21:38:21.099364+0000","last_clean":"2024-04-26T21:38:21.099364+0000","last_became_active":"2024-04-26T21:38:17.951044+0000","last_became_peered":"2024-04-26T21:38:17.951044+0000","last_unstale":"2024-04-26T21:38:21.099364+0000","last_undegraded":"2024-04-26T21:38:21.099364+0000","last_fullsized":"2024-04-26T21:38:21.099364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:39.512854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200456+0000","last_change":"2024-04-26T21:38:18.947673+0000","last_active":"2024-04-26T21:38:22.200456+0000","last_peered":"2024-04-26T21:38:22.200456+0000","last_clean":"2024-04-26T21:38:22.200456+0000","last_became_active":"2024-04-26T21:38:18.947045+0000","last_became_peered":"2024-04-26T21:38:18.947045+0000","last_unstale":"2024-04-26T21:38:22.200456+0000","last_undegraded":"2024-04-26T21:38:22.200456+0000","last_fullsized":"2024-04-26T21:38:22.200456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:43.698138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934535+0000","last_change":"2024-04-26T21:38:17.945882+0000","last_active":"2024-04-26T21:38:19.934535+0000","last_peered":"2024-04-26T21:38:19.934535+0000","last_clean":"2024-04-26T21:38:19.934535+0000","last_became_active":"2024-04-26T21:38:17.945594+0000","last_became_peered":"2024-04-26T21:38:17.945594+0000","last_unstale":"2024-04-26T21:38:19.934535+0000","last_undegraded":"2024-04-26T21:38:19.934535+0000","last_fullsized":"2024-04-26T21:38:19.934535+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:42.691662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199909+0000","last_change":"2024-04-26T21:38:18.951616+0000","last_active":"2024-04-26T21:38:22.199909+0000","last_peered":"2024-04-26T21:38:22.199909+0000","last_clean":"2024-04-26T21:38:22.199909+0000","last_became_active":"2024-04-26T21:38:18.951329+0000","last_became_peered":"2024-04-26T21:38:18.951329+0000","last_unstale":"2024-04-26T21:38:22.199909+0000","last_undegraded":"2024-04-26T21:38:22.199909+0000","last_fullsized":"2024-04-26T21:38:22.199909+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:49.252133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099618+0000","last_change":"2024-04-26T21:38:17.947105+0000","last_active":"2024-04-26T21:38:21.099618+0000","last_peered":"2024-04-26T21:38:21.099618+0000","last_clean":"2024-04-26T21:38:21.099618+0000","last_became_active":"2024-04-26T21:38:17.946804+0000","last_became_peered":"2024-04-26T21:38:17.946804+0000","last_unstale":"2024-04-26T21:38:21.099618+0000","last_undegraded":"2024-04-26T21:38:21.099618+0000","last_fullsized":"2024-04-26T21:38:21.099618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:47.178650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933578+0000","last_change":"2024-04-26T21:38:18.946576+0000","last_active":"2024-04-26T21:38:19.933578+0000","last_peered":"2024-04-26T21:38:19.933578+0000","last_clean":"2024-04-26T21:38:19.933578+0000","last_became_active":"2024-04-26T21:38:18.946238+0000","last_became_peered":"2024-04-26T21:38:18.946238+0000","last_unstale":"2024-04-26T21:38:19.933578+0000","last_undegraded":"2024-04-26T21:38:19.933578+0000","last_fullsized":"2024-04-26T21:38:19.933578+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:28.266053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933737+0000","last_change":"2024-04-26T21:38:17.944988+0000","last_active":"2024-04-26T21:38:19.933737+0000","last_peered":"2024-04-26T21:38:19.933737+0000","last_clean":"2024-04-26T21:38:19.933737+0000","last_became_active":"2024-04-26T21:38:17.944724+0000","last_became_peered":"2024-04-26T21:38:17.944724+0000","last_unstale":"2024-04-26T21:38:19.933737+0000","last_undegraded":"2024-04-26T21:38:19.933737+0000","last_fullsized":"2024-04-26T21:38:19.933737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:57.060894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952799+0000","last_change":"2024-04-26T21:38:18.954305+0000","last_active":"2024-04-26T21:38:19.952799+0000","last_peered":"2024-04-26T21:38:19.952799+0000","last_clean":"2024-04-26T21:38:19.952799+0000","last_became_active":"2024-04-26T21:38:18.953647+0000","last_became_peered":"2024-04-26T21:38:18.953647+0000","last_unstale":"2024-04-26T21:38:19.952799+0000","last_undegraded":"2024-04-26T21:38:19.952799+0000","last_fullsized":"2024-04-26T21:38:19.952799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:29.752423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099401+0000","last_change":"2024-04-26T21:38:17.947884+0000","last_active":"2024-04-26T21:38:21.099401+0000","last_peered":"2024-04-26T21:38:21.099401+0000","last_clean":"2024-04-26T21:38:21.099401+0000","last_became_active":"2024-04-26T21:38:17.947599+0000","last_became_peered":"2024-04-26T21:38:17.947599+0000","last_unstale":"2024-04-26T21:38:21.099401+0000","last_undegraded":"2024-04-26T21:38:21.099401+0000","last_fullsized":"2024-04-26T21:38:21.099401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:40.097539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934158+0000","last_change":"2024-04-26T21:38:18.946460+0000","last_active":"2024-04-26T21:38:19.934158+0000","last_peered":"2024-04-26T21:38:19.934158+0000","last_clean":"2024-04-26T21:38:19.934158+0000","last_became_active":"2024-04-26T21:38:18.946145+0000","last_became_peered":"2024-04-26T21:38:18.946145+0000","last_unstale":"2024-04-26T21:38:19.934158+0000","last_undegraded":"2024-04-26T21:38:19.934158+0000","last_fullsized":"2024-04-26T21:38:19.934158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:02.866419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199824+0000","last_change":"2024-04-26T21:38:17.948438+0000","last_active":"2024-04-26T21:38:22.199824+0000","last_peered":"2024-04-26T21:38:22.199824+0000","last_clean":"2024-04-26T21:38:22.199824+0000","last_became_active":"2024-04-26T21:38:17.948196+0000","last_became_peered":"2024-04-26T21:38:17.948196+0000","last_unstale":"2024-04-26T21:38:22.199824+0000","last_undegraded":"2024-04-26T21:38:22.199824+0000","last_fullsized":"2024-04-26T21:38:22.199824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:59.951617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099420+0000","last_change":"2024-04-26T21:38:18.953516+0000","last_active":"2024-04-26T21:38:21.099420+0000","last_peered":"2024-04-26T21:38:21.099420+0000","last_clean":"2024-04-26T21:38:21.099420+0000","last_became_active":"2024-04-26T21:38:18.953197+0000","last_became_peered":"2024-04-26T21:38:18.953197+0000","last_unstale":"2024-04-26T21:38:21.099420+0000","last_undegraded":"2024-04-26T21:38:21.099420+0000","last_fullsized":"2024-04-26T21:38:21.099420+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:58.368085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933508+0000","last_change":"2024-04-26T21:38:17.942502+0000","last_active":"2024-04-26T21:38:19.933508+0000","last_peered":"2024-04-26T21:38:19.933508+0000","last_clean":"2024-04-26T21:38:19.933508+0000","last_became_active":"2024-04-26T21:38:17.942225+0000","last_became_peered":"2024-04-26T21:38:17.942225+0000","last_unstale":"2024-04-26T21:38:19.933508+0000","last_undegraded":"2024-04-26T21:38:19.933508+0000","last_fullsized":"2024-04-26T21:38:19.933508+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:40.097539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953140+0000","last_change":"2024-04-26T21:38:18.948264+0000","last_active":"2024-04-26T21:38:19.953140+0000","last_peered":"2024-04-26T21:38:19.953140+0000","last_clean":"2024-04-26T21:38:19.953140+0000","last_became_active":"2024-04-26T21:38:18.947965+0000","last_became_peered":"2024-04-26T21:38:18.947965+0000","last_unstale":"2024-04-26T21:38:19.953140+0000","last_undegraded":"2024-04-26T21:38:19.953140+0000","last_fullsized":"2024-04-26T21:38:19.953140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:40.519330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099697+0000","last_change":"2024-04-26T21:38:17.949980+0000","last_active":"2024-04-26T21:38:21.099697+0000","last_peered":"2024-04-26T21:38:21.099697+0000","last_clean":"2024-04-26T21:38:21.099697+0000","last_became_active":"2024-04-26T21:38:17.949710+0000","last_became_peered":"2024-04-26T21:38:17.949710+0000","last_unstale":"2024-04-26T21:38:21.099697+0000","last_undegraded":"2024-04-26T21:38:21.099697+0000","last_fullsized":"2024-04-26T21:38:21.099697+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:02.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200405+0000","last_change":"2024-04-26T21:38:18.948878+0000","last_active":"2024-04-26T21:38:22.200405+0000","last_peered":"2024-04-26T21:38:22.200405+0000","last_clean":"2024-04-26T21:38:22.200405+0000","last_became_active":"2024-04-26T21:38:18.948616+0000","last_became_peered":"2024-04-26T21:38:18.948616+0000","last_unstale":"2024-04-26T21:38:22.200405+0000","last_undegraded":"2024-04-26T21:38:22.200405+0000","last_fullsized":"2024-04-26T21:38:22.200405+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:03.766127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099576+0000","last_change":"2024-04-26T21:38:17.951499+0000","last_active":"2024-04-26T21:38:21.099576+0000","last_peered":"2024-04-26T21:38:21.099576+0000","last_clean":"2024-04-26T21:38:21.099576+0000","last_became_active":"2024-04-26T21:38:17.951190+0000","last_became_peered":"2024-04-26T21:38:17.951190+0000","last_unstale":"2024-04-26T21:38:21.099576+0000","last_undegraded":"2024-04-26T21:38:21.099576+0000","last_fullsized":"2024-04-26T21:38:21.099576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:46.534568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952994+0000","last_change":"2024-04-26T21:38:18.955078+0000","last_active":"2024-04-26T21:38:19.952994+0000","last_peered":"2024-04-26T21:38:19.952994+0000","last_clean":"2024-04-26T21:38:19.952994+0000","last_became_active":"2024-04-26T21:38:18.954813+0000","last_became_peered":"2024-04-26T21:38:18.954813+0000","last_unstale":"2024-04-26T21:38:19.952994+0000","last_undegraded":"2024-04-26T21:38:19.952994+0000","last_fullsized":"2024-04-26T21:38:19.952994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:58.067370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760316+0000","last_change":"2024-04-26T21:38:17.943246+0000","last_active":"2024-04-26T21:38:21.760316+0000","last_peered":"2024-04-26T21:38:21.760316+0000","last_clean":"2024-04-26T21:38:21.760316+0000","last_became_active":"2024-04-26T21:38:17.942975+0000","last_became_peered":"2024-04-26T21:38:17.942975+0000","last_unstale":"2024-04-26T21:38:21.760316+0000","last_undegraded":"2024-04-26T21:38:21.760316+0000","last_fullsized":"2024-04-26T21:38:21.760316+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.865600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199459+0000","last_change":"2024-04-26T21:38:18.948522+0000","last_active":"2024-04-26T21:38:22.199459+0000","last_peered":"2024-04-26T21:38:22.199459+0000","last_clean":"2024-04-26T21:38:22.199459+0000","last_became_active":"2024-04-26T21:38:18.948236+0000","last_became_peered":"2024-04-26T21:38:18.948236+0000","last_unstale":"2024-04-26T21:38:22.199459+0000","last_undegraded":"2024-04-26T21:38:22.199459+0000","last_fullsized":"2024-04-26T21:38:22.199459+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:36:48.116434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199539+0000","last_change":"2024-04-26T21:38:17.946995+0000","last_active":"2024-04-26T21:38:22.199539+0000","last_peered":"2024-04-26T21:38:22.199539+0000","last_clean":"2024-04-26T21:38:22.199539+0000","last_became_active":"2024-04-26T21:38:17.946697+0000","last_became_peered":"2024-04-26T21:38:17.946697+0000","last_unstale":"2024-04-26T21:38:22.199539+0000","last_undegraded":"2024-04-26T21:38:22.199539+0000","last_fullsized":"2024-04-26T21:38:22.199539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:31.341265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933414+0000","last_change":"2024-04-26T21:38:18.945327+0000","last_active":"2024-04-26T21:38:19.933414+0000","last_peered":"2024-04-26T21:38:19.933414+0000","last_clean":"2024-04-26T21:38:19.933414+0000","last_became_active":"2024-04-26T21:38:18.945055+0000","last_became_peered":"2024-04-26T21:38:18.945055+0000","last_unstale":"2024-04-26T21:38:19.933414+0000","last_undegraded":"2024-04-26T21:38:19.933414+0000","last_fullsized":"2024-04-26T21:38:19.933414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:54.872076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933870+0000","last_change":"2024-04-26T21:38:17.942558+0000","last_active":"2024-04-26T21:38:19.933870+0000","last_peered":"2024-04-26T21:38:19.933870+0000","last_clean":"2024-04-26T21:38:19.933870+0000","last_became_active":"2024-04-26T21:38:17.942267+0000","last_became_peered":"2024-04-26T21:38:17.942267+0000","last_unstale":"2024-04-26T21:38:19.933870+0000","last_undegraded":"2024-04-26T21:38:19.933870+0000","last_fullsized":"2024-04-26T21:38:19.933870+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:02.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953093+0000","last_change":"2024-04-26T21:38:18.954514+0000","last_active":"2024-04-26T21:38:19.953093+0000","last_peered":"2024-04-26T21:38:19.953093+0000","last_clean":"2024-04-26T21:38:19.953093+0000","last_became_active":"2024-04-26T21:38:18.953995+0000","last_became_peered":"2024-04-26T21:38:18.953995+0000","last_unstale":"2024-04-26T21:38:19.953093+0000","last_undegraded":"2024-04-26T21:38:19.953093+0000","last_fullsized":"2024-04-26T21:38:19.953093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:41.104015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199362+0000","last_change":"2024-04-26T21:38:17.948494+0000","last_active":"2024-04-26T21:38:22.199362+0000","last_peered":"2024-04-26T21:38:22.199362+0000","last_clean":"2024-04-26T21:38:22.199362+0000","last_became_active":"2024-04-26T21:38:17.948212+0000","last_became_peered":"2024-04-26T21:38:17.948212+0000","last_unstale":"2024-04-26T21:38:22.199362+0000","last_undegraded":"2024-04-26T21:38:22.199362+0000","last_fullsized":"2024-04-26T21:38:22.199362+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.865600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199695+0000","last_change":"2024-04-26T21:38:18.951827+0000","last_active":"2024-04-26T21:38:22.199695+0000","last_peered":"2024-04-26T21:38:22.199695+0000","last_clean":"2024-04-26T21:38:22.199695+0000","last_became_active":"2024-04-26T21:38:18.951536+0000","last_became_peered":"2024-04-26T21:38:18.951536+0000","last_unstale":"2024-04-26T21:38:22.199695+0000","last_undegraded":"2024-04-26T21:38:22.199695+0000","last_fullsized":"2024-04-26T21:38:22.199695+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:47.732055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933770+0000","last_change":"2024-04-26T21:38:17.951721+0000","last_active":"2024-04-26T21:38:19.933770+0000","last_peered":"2024-04-26T21:38:19.933770+0000","last_clean":"2024-04-26T21:38:19.933770+0000","last_became_active":"2024-04-26T21:38:17.951400+0000","last_became_peered":"2024-04-26T21:38:17.951400+0000","last_unstale":"2024-04-26T21:38:19.933770+0000","last_undegraded":"2024-04-26T21:38:19.933770+0000","last_fullsized":"2024-04-26T21:38:19.933770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:38.435914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933808+0000","last_change":"2024-04-26T21:38:18.954039+0000","last_active":"2024-04-26T21:38:19.933808+0000","last_peered":"2024-04-26T21:38:19.933808+0000","last_clean":"2024-04-26T21:38:19.933808+0000","last_became_active":"2024-04-26T21:38:18.953771+0000","last_became_peered":"2024-04-26T21:38:18.953771+0000","last_unstale":"2024-04-26T21:38:19.933808+0000","last_undegraded":"2024-04-26T21:38:19.933808+0000","last_fullsized":"2024-04-26T21:38:19.933808+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:29.401057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933943+0000","last_change":"2024-04-26T21:38:17.945997+0000","last_active":"2024-04-26T21:38:19.933943+0000","last_peered":"2024-04-26T21:38:19.933943+0000","last_clean":"2024-04-26T21:38:19.933943+0000","last_became_active":"2024-04-26T21:38:17.945730+0000","last_became_peered":"2024-04-26T21:38:17.945730+0000","last_unstale":"2024-04-26T21:38:19.933943+0000","last_undegraded":"2024-04-26T21:38:19.933943+0000","last_fullsized":"2024-04-26T21:38:19.933943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:47.178650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933714+0000","last_change":"2024-04-26T21:38:18.955861+0000","last_active":"2024-04-26T21:38:19.933714+0000","last_peered":"2024-04-26T21:38:19.933714+0000","last_clean":"2024-04-26T21:38:19.933714+0000","last_became_active":"2024-04-26T21:38:18.955456+0000","last_became_peered":"2024-04-26T21:38:18.955456+0000","last_unstale":"2024-04-26T21:38:19.933714+0000","last_undegraded":"2024-04-26T21:38:19.933714+0000","last_fullsized":"2024-04-26T21:38:19.933714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.399686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.199740+0000","last_change":"2024-04-26T21:38:17.949610+0000","last_active":"2024-04-26T21:38:22.199740+0000","last_peered":"2024-04-26T21:38:22.199740+0000","last_clean":"2024-04-26T21:38:22.199740+0000","last_became_active":"2024-04-26T21:38:17.949247+0000","last_became_peered":"2024-04-26T21:38:17.949247+0000","last_unstale":"2024-04-26T21:38:22.199740+0000","last_undegraded":"2024-04-26T21:38:22.199740+0000","last_fullsized":"2024-04-26T21:38:22.199740+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:57.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953206+0000","last_change":"2024-04-26T21:38:18.955515+0000","last_active":"2024-04-26T21:38:19.953206+0000","last_peered":"2024-04-26T21:38:19.953206+0000","last_clean":"2024-04-26T21:38:19.953206+0000","last_became_active":"2024-04-26T21:38:18.955243+0000","last_became_peered":"2024-04-26T21:38:18.955243+0000","last_unstale":"2024-04-26T21:38:19.953206+0000","last_undegraded":"2024-04-26T21:38:19.953206+0000","last_fullsized":"2024-04-26T21:38:19.953206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:02.866419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.934541+0000","last_change":"2024-04-26T21:38:17.946463+0000","last_active":"2024-04-26T21:38:19.934541+0000","last_peered":"2024-04-26T21:38:19.934541+0000","last_clean":"2024-04-26T21:38:19.934541+0000","last_became_active":"2024-04-26T21:38:17.946182+0000","last_became_peered":"2024-04-26T21:38:17.946182+0000","last_unstale":"2024-04-26T21:38:19.934541+0000","last_undegraded":"2024-04-26T21:38:19.934541+0000","last_fullsized":"2024-04-26T21:38:19.934541+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:58.559420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.952722+0000","last_change":"2024-04-26T21:38:18.956532+0000","last_active":"2024-04-26T21:38:19.952722+0000","last_peered":"2024-04-26T21:38:19.952722+0000","last_clean":"2024-04-26T21:38:19.952722+0000","last_became_active":"2024-04-26T21:38:18.956278+0000","last_became_peered":"2024-04-26T21:38:18.956278+0000","last_unstale":"2024-04-26T21:38:19.952722+0000","last_undegraded":"2024-04-26T21:38:19.952722+0000","last_fullsized":"2024-04-26T21:38:19.952722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:04.002778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099629+0000","last_change":"2024-04-26T21:38:17.949728+0000","last_active":"2024-04-26T21:38:21.099629+0000","last_peered":"2024-04-26T21:38:21.099629+0000","last_clean":"2024-04-26T21:38:21.099629+0000","last_became_active":"2024-04-26T21:38:17.949450+0000","last_became_peered":"2024-04-26T21:38:17.949450+0000","last_unstale":"2024-04-26T21:38:21.099629+0000","last_undegraded":"2024-04-26T21:38:21.099629+0000","last_fullsized":"2024-04-26T21:38:21.099629+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:38.435914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:22.200066+0000","last_change":"2024-04-26T21:38:18.952550+0000","last_active":"2024-04-26T21:38:22.200066+0000","last_peered":"2024-04-26T21:38:22.200066+0000","last_clean":"2024-04-26T21:38:22.200066+0000","last_became_active":"2024-04-26T21:38:18.952263+0000","last_became_peered":"2024-04-26T21:38:18.952263+0000","last_unstale":"2024-04-26T21:38:22.200066+0000","last_undegraded":"2024-04-26T21:38:22.200066+0000","last_fullsized":"2024-04-26T21:38:22.200066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:49.503375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760181+0000","last_change":"2024-04-26T21:38:17.937457+0000","last_active":"2024-04-26T21:38:21.760181+0000","last_peered":"2024-04-26T21:38:21.760181+0000","last_clean":"2024-04-26T21:38:21.760181+0000","last_became_active":"2024-04-26T21:38:17.937039+0000","last_became_peered":"2024-04-26T21:38:17.937039+0000","last_unstale":"2024-04-26T21:38:21.760181+0000","last_undegraded":"2024-04-26T21:38:21.760181+0000","last_fullsized":"2024-04-26T21:38:21.760181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:31.341265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933979+0000","last_change":"2024-04-26T21:38:18.946583+0000","last_active":"2024-04-26T21:38:19.933979+0000","last_peered":"2024-04-26T21:38:19.933979+0000","last_clean":"2024-04-26T21:38:19.933979+0000","last_became_active":"2024-04-26T21:38:18.946216+0000","last_became_peered":"2024-04-26T21:38:18.946216+0000","last_unstale":"2024-04-26T21:38:19.933979+0000","last_undegraded":"2024-04-26T21:38:19.933979+0000","last_fullsized":"2024-04-26T21:38:19.933979+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:40.519330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.760157+0000","last_change":"2024-04-26T21:38:17.944966+0000","last_active":"2024-04-26T21:38:21.760157+0000","last_peered":"2024-04-26T21:38:21.760157+0000","last_clean":"2024-04-26T21:38:21.760157+0000","last_became_active":"2024-04-26T21:38:17.944185+0000","last_became_peered":"2024-04-26T21:38:17.944185+0000","last_unstale":"2024-04-26T21:38:21.760157+0000","last_undegraded":"2024-04-26T21:38:21.760157+0000","last_fullsized":"2024-04-26T21:38:21.760157+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:44.393210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933759+0000","last_change":"2024-04-26T21:38:18.945797+0000","last_active":"2024-04-26T21:38:19.933759+0000","last_peered":"2024-04-26T21:38:19.933759+0000","last_clean":"2024-04-26T21:38:19.933759+0000","last_became_active":"2024-04-26T21:38:18.945522+0000","last_became_peered":"2024-04-26T21:38:18.945522+0000","last_unstale":"2024-04-26T21:38:19.933759+0000","last_undegraded":"2024-04-26T21:38:19.933759+0000","last_fullsized":"2024-04-26T21:38:19.933759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:01.964703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.933793+0000","last_change":"2024-04-26T21:38:17.945734+0000","last_active":"2024-04-26T21:38:19.933793+0000","last_peered":"2024-04-26T21:38:19.933793+0000","last_clean":"2024-04-26T21:38:19.933793+0000","last_became_active":"2024-04-26T21:38:17.945450+0000","last_became_peered":"2024-04-26T21:38:17.945450+0000","last_unstale":"2024-04-26T21:38:19.933793+0000","last_undegraded":"2024-04-26T21:38:19.933793+0000","last_fullsized":"2024-04-26T21:38:19.933793+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:50.457447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:19.953062+0000","last_change":"2024-04-26T21:38:18.957994+0000","last_active":"2024-04-26T21:38:19.953062+0000","last_peered":"2024-04-26T21:38:19.953062+0000","last_clean":"2024-04-26T21:38:19.953062+0000","last_became_active":"2024-04-26T21:38:18.957733+0000","last_became_peered":"2024-04-26T21:38:18.957733+0000","last_unstale":"2024-04-26T21:38:19.953062+0000","last_undegraded":"2024-04-26T21:38:19.953062+0000","last_fullsized":"2024-04-26T21:38:19.953062+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:17.921449+0000","last_clean_scrub_stamp":"2024-04-26T21:38:17.921449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:03.784664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.759951+0000","last_change":"2024-04-26T21:38:17.944876+0000","last_active":"2024-04-26T21:38:21.759951+0000","last_peered":"2024-04-26T21:38:21.759951+0000","last_clean":"2024-04-26T21:38:21.759951+0000","last_became_active":"2024-04-26T21:38:17.944590+0000","last_became_peered":"2024-04-26T21:38:17.944590+0000","last_unstale":"2024-04-26T21:38:21.759951+0000","last_undegraded":"2024-04-26T21:38:21.759951+0000","last_fullsized":"2024-04-26T21:38:21.759951+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:38:16.914972+0000","last_clean_scrub_stamp":"2024-04-26T21:38:16.914972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:27.259577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":255,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-26T21:38:21.099207+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:38:21.099207+0000","last_peered":"2024-04-26T21:38:21.099207+0000","last_clean":"2024-04-26T21:38:21.099207+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:38:21.099207+0000","last_undegraded":"2024-04-26T21:38:21.099207+0000","last_fullsized":"2024-04-26T21:38:21.099207+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28396,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343444,"statfs":{"total":96636764160,"available":96607686656,"internally_reserved":0,"allocated":1409024,"data_stored":955914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":364804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":364792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":366382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":364770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":364770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":367146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28376,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343464,"statfs":{"total":96636764160,"available":96607707136,"internally_reserved":0,"allocated":1380352,"data_stored":948446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:38:27.761 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:38:27.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:38:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.904+0000 7ff6c1499640 1 Processor -- start 2024-04-26T21:38:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.904+0000 7ff6c1499640 1 -- start start 2024-04-26T21:38:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.905+0000 7ff6c1499640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff6bc15f770 0x7ff6bc0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.905+0000 7ff6c1499640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6bc06e580 con 0x7ff6bc0d1920 2024-04-26T21:38:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.905+0000 7ff6c1499640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6bc06e860 con 0x7ff6bc0d0d60 2024-04-26T21:38:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.905+0000 7ff6c1499640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff6bc06f3e0 con 0x7ff6bc15f770 2024-04-26T21:38:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.905+0000 7ff6bb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff6bc0d1920 0x7ff6bc0cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39774/0 (socket says 172.21.15.40:39774) 2024-04-26T21:38:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.905+0000 7ff6bb7fe640 1 -- 172.21.15.40:0/3330080578 learned_addr learned my addr 172.21.15.40:0/3330080578 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078963876 0 0) 0x7ff6bc06e580 con 0x7ff6bc0d1920 2024-04-26T21:38:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6a8003520 con 0x7ff6bc0d1920 2024-04-26T21:38:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6baffd640 1 --2- 172.21.15.40:0/3330080578 >> v2:172.21.15.184:3300/0 conn(0x7ff6bc15f770 0x7ff6bc0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2991575415 0 0) 0x7ff6a8003520 con 0x7ff6bc0d1920 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 >> v2:172.21.15.184:3300/0 conn(0x7ff6bc15f770 msgr2=0x7ff6bc0b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 --2- 172.21.15.40:0/3330080578 >> v2:172.21.15.184:3300/0 conn(0x7ff6bc15f770 0x7ff6bc0b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 >> v1:172.21.15.184:6789/0 conn(0x7ff6bc0d0d60 legacy=0x7ff6bc0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc06eb40 con 0x7ff6bc0d1920 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7ff6ac0019f0 con 0x7ff6bc0d1920 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 >> v1:172.21.15.40:6789/0 conn(0x7ff6bc0d1920 legacy=0x7ff6bc0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6a80039b0 0x7ff6a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a8004800 0x7ff6a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc06f3e0 con 0x7ff6a80051c0 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a800c060 con 0x7ff6a8004800 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a800c260 con 0x7ff6a80039b0 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6bb7fe640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6baffd640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6a80039b0 0x7ff6a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6ba7fc640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a8004800 0x7ff6a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6bb7fe640 1 -- 172.21.15.40:0/3330080578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6a80039b0 msgr2=0x7ff6a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6bb7fe640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6a80039b0 0x7ff6a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6bb7fe640 1 -- 172.21.15.40:0/3330080578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a8004800 msgr2=0x7ff6a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6bb7fe640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6a8004800 0x7ff6a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.906+0000 7ff6bb7fe640 1 -- 172.21.15.40:0/3330080578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a8003520 con 0x7ff6a80051c0 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.907+0000 7ff6bb7fe640 1 -- 172.21.15.40:0/3330080578 reap_dead start 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.907+0000 7ff6bb7fe640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6a800b9c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00a850 tx=0x7ff6a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c390c3f49c221b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.907+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac00a4a0 con 0x7ff6a80051c0 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.907+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ac02f070 con 0x7ff6a80051c0 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.907+0000 7ff6b9ffb640 1 -- 172.21.15.40:0/3330080578 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac02bce0 con 0x7ff6a80051c0 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.908+0000 7ff6c1499640 1 -- 172.21.15.40:0/3330080578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 msgr2=0x7ff6a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.908+0000 7ff6c1499640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6a800b9c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00a850 tx=0x7ff6a8005830 comp rx=0 tx=0).stop 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.908+0000 7ff6c1499640 1 -- 172.21.15.40:0/3330080578 shutdown_connections 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.908+0000 7ff6c1499640 1 --2- 172.21.15.40:0/3330080578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6a800b9c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.908+0000 7ff6c1499640 1 -- 172.21.15.40:0/3330080578 >> 172.21.15.40:0/3330080578 conn(0x7ff6bc0d2e60 msgr2=0x7ff6bc0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.908+0000 7ff6c1499640 1 -- 172.21.15.40:0/3330080578 shutdown_connections 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.908+0000 7ff6c1499640 1 -- 172.21.15.40:0/3330080578 wait complete. 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.909+0000 7ff6c1499640 1 Processor -- start 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.909+0000 7ff6c1499640 1 -- start start 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6c1499640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6bc15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6c1499640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6bc0b5a80 0x7ff6bc0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6c1499640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6bc0b13b0 0x7ff6bc0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6c1499640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc0b2360 con 0x7ff6a80051c0 2024-04-26T21:38:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6c1499640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc0b21e0 con 0x7ff6bc0b13b0 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6c1499640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff6bc0cd6e0 con 0x7ff6bc0b5a80 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6baffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6bc15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6baffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6bc15c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46914/0 (socket says 172.21.15.40:46914) 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6baffd640 1 -- 172.21.15.40:0/3310304276 learned_addr learned my addr 172.21.15.40:0/3310304276 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6ba7fc640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6bc0b5a80 0x7ff6bc0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6bb7fe640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6bc0b13b0 0x7ff6bc0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6baffd640 1 -- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6bc0b5a80 msgr2=0x7ff6bc0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6baffd640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6bc0b5a80 0x7ff6bc0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6baffd640 1 -- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6bc0b13b0 msgr2=0x7ff6bc0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6baffd640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6bc0b13b0 0x7ff6bc0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6baffd640 1 -- 172.21.15.40:0/3310304276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc0b5f60 con 0x7ff6a80051c0 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.910+0000 7ff6bb7fe640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6bc0b13b0 0x7ff6bc0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.911+0000 7ff6baffd640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6bc15c0b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4011910 tx=0x7ff6b4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.911+0000 7ff6b37fe640 1 -- 172.21.15.40:0/3310304276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400faf0 con 0x7ff6a80051c0 2024-04-26T21:38:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.911+0000 7ff6b37fe640 1 -- 172.21.15.40:0/3310304276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b400fc90 con 0x7ff6a80051c0 2024-04-26T21:38:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.911+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6bc0b6240 con 0x7ff6a80051c0 2024-04-26T21:38:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.911+0000 7ff6b37fe640 1 -- 172.21.15.40:0/3310304276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4014440 con 0x7ff6a80051c0 2024-04-26T21:38:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.911+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6bc0b6800 con 0x7ff6a80051c0 2024-04-26T21:38:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.912+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6bc0694f0 con 0x7ff6a80051c0 2024-04-26T21:38:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.912+0000 7ff6b37fe640 1 -- 172.21.15.40:0/3310304276 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6b4012040 con 0x7ff6a80051c0 2024-04-26T21:38:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.913+0000 7ff6b37fe640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff69c06c8f0 0x7ff69c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.913+0000 7ff6b37fe640 1 -- 172.21.15.40:0/3310304276 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7ff6b408c330 con 0x7ff6a80051c0 2024-04-26T21:38:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.913+0000 7ff6ba7fc640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff69c06c8f0 0x7ff69c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.913+0000 7ff6ba7fc640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff69c06c8f0 0x7ff69c06edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc1f8680 tx=0x7ff6a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:27.917+0000 7ff6b37fe640 1 -- 172.21.15.40:0/3310304276 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6b4019030 con 0x7ff6a80051c0 2024-04-26T21:38:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.172+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff6bc075aa0 con 0x7ff6a80051c0 2024-04-26T21:38:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.174+0000 7ff6b37fe640 1 -- 172.21.15.40:0/3310304276 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7ff6b405a5d0 con 0x7ff6a80051c0 2024-04-26T21:38:28.175 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:38:28.175 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw== 2024-04-26T21:38:28.175 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:38:28.175 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:38:28.175 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:38:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:38:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff69c06c8f0 msgr2=0x7ff69c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff69c06c8f0 0x7ff69c06edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc1f8680 tx=0x7ff6a4009040 comp rx=0 tx=0).stop 2024-04-26T21:38:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 msgr2=0x7ff6bc15c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6bc15c0b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4011910 tx=0x7ff6b4011dd0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 shutdown_connections 2024-04-26T21:38:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff69c06c8f0 0x7ff69c06edb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6bc0b13b0 0x7ff6bc0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6bc0b5a80 0x7ff6bc0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 --2- 172.21.15.40:0/3310304276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a80051c0 0x7ff6bc15c0b0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 >> 172.21.15.40:0/3310304276 conn(0x7ff6bc0d2e60 msgr2=0x7ff6bc0703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 shutdown_connections 2024-04-26T21:38:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.176+0000 7ff6c1499640 1 -- 172.21.15.40:0/3310304276 wait complete. 2024-04-26T21:38:28.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.288+0000 7f74dd19d640 1 Processor -- start 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.288+0000 7f74dd19d640 1 -- start start 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.288+0000 7f74dd19d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f74d8162880 0x7f74d80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.288+0000 7f74dd19d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74d80b5f20 0x7f74d80d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.288+0000 7f74dd19d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b7e50 con 0x7f74d80d39f0 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.288+0000 7f74dd19d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f74d80d3e40 con 0x7f74d80b5f20 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.288+0000 7f74dd19d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f74d80d0580 con 0x7f74d8162880 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.289+0000 7f74d659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74d80b5f20 0x7f74d80d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.289+0000 7f74d659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74d80b5f20 0x7f74d80d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46918/0 (socket says 172.21.15.40:46918) 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.289+0000 7f74d659c640 1 -- 172.21.15.40:0/692924866 learned_addr learned my addr 172.21.15.40:0/692924866 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.289+0000 7f74d6d9d640 1 --2- 172.21.15.40:0/692924866 >> v2:172.21.15.184:3300/0 conn(0x7f74d8162880 0x7f74d80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.289+0000 7f74d659c640 1 -- 172.21.15.40:0/692924866 >> v2:172.21.15.184:3300/0 conn(0x7f74d8162880 msgr2=0x7f74d80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.289+0000 7f74d659c640 1 --2- 172.21.15.40:0/692924866 >> v2:172.21.15.184:3300/0 conn(0x7f74d8162880 0x7f74d80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.289+0000 7f74d659c640 1 -- 172.21.15.40:0/692924866 >> v1:172.21.15.40:6789/0 conn(0x7f74d80d39f0 legacy=0x7f74d80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.289+0000 7f74d659c640 1 -- 172.21.15.40:0/692924866 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d80d1800 con 0x7f74d80b5f20 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74d659c640 1 --2- 172.21.15.40:0/692924866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d80d34b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f74d808d490 tx=0x7f74cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50e03f1a21512ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74d5d9b640 1 -- 172.21.15.40:0/692924866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc037070 con 0x7f74d80b5f20 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74d5d9b640 1 -- 172.21.15.40:0/692924866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74cc032440 con 0x7f74d80b5f20 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74d5d9b640 1 -- 172.21.15.40:0/692924866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc032810 con 0x7f74d80b5f20 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74dd19d640 1 -- 172.21.15.40:0/692924866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 msgr2=0x7f74d80d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74dd19d640 1 --2- 172.21.15.40:0/692924866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d80d34b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f74d808d490 tx=0x7f74cc02a270 comp rx=0 tx=0).stop 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74dd19d640 1 -- 172.21.15.40:0/692924866 shutdown_connections 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74dd19d640 1 --2- 172.21.15.40:0/692924866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d80d34b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74dd19d640 1 --2- 172.21.15.40:0/692924866 >> v2:172.21.15.184:3300/0 conn(0x7f74d8162880 0x7f74d80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74dd19d640 1 -- 172.21.15.40:0/692924866 >> 172.21.15.40:0/692924866 conn(0x7f74d80d4f60 msgr2=0x7f74d80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74dd19d640 1 -- 172.21.15.40:0/692924866 shutdown_connections 2024-04-26T21:38:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.290+0000 7f74dd19d640 1 -- 172.21.15.40:0/692924866 wait complete. 2024-04-26T21:38:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.292+0000 7f74dd19d640 1 Processor -- start 2024-04-26T21:38:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.292+0000 7f74dd19d640 1 -- start start 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.292+0000 7f74dd19d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d8155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.292+0000 7f74dd19d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d80d39f0 0x7f74d806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.292+0000 7f74dd19d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74d8162880 0x7f74d806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.292+0000 7f74dd19d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74d80d0d90 con 0x7f74d80b5f20 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.292+0000 7f74dd19d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74d80d0ab0 con 0x7f74d80d39f0 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.292+0000 7f74dd19d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74d8162c60 con 0x7f74d8162880 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d8155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d8155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46932/0 (socket says 172.21.15.40:46932) 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 -- 172.21.15.40:0/2404472116 learned_addr learned my addr 172.21.15.40:0/2404472116 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d659c640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d80d39f0 0x7f74d806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d759e640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74d8162880 0x7f74d806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 -- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74d8162880 msgr2=0x7f74d806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74d8162880 0x7f74d806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 -- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d80d39f0 msgr2=0x7f74d806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d80d39f0 0x7f74d806dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 -- 172.21.15.40:0/2404472116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d815a260 con 0x7f74d80b5f20 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d659c640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d80d39f0 0x7f74d806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d759e640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74d8162880 0x7f74d806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74d6d9d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d8155600 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f74c800d830 tx=0x7f74c800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2404472116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c800b6c0 con 0x7f74d80b5f20 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2404472116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c8019070 con 0x7f74d80b5f20 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d815a540 con 0x7f74d80b5f20 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2404472116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c800ba80 con 0x7f74d80b5f20 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.293+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d815ab00 con 0x7f74d80b5f20 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.294+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d80694f0 con 0x7f74d80b5f20 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.295+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2404472116 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74c801e460 con 0x7f74d80b5f20 2024-04-26T21:38:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.295+0000 7f74bf7fe640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74c406c920 0x7f74c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.295+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2404472116 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f74c808c300 con 0x7f74d80b5f20 2024-04-26T21:38:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.295+0000 7f74d659c640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74c406c920 0x7f74c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.296+0000 7f74d659c640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74c406c920 0x7f74c406ede0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f74cc034000 tx=0x7f74cc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.299+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2404472116 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74c805a630 con 0x7f74d80b5f20 2024-04-26T21:38:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.524+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74d8160c30 con 0x7f74d80b5f20 2024-04-26T21:38:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.525+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2404472116 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5185 (secure 0 0 0) 0x7f74c8059f30 con 0x7f74d80b5f20 2024-04-26T21:38:28.526 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:28.526 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:21.316626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1667529627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1667529627},{"type":"v1","addr":"172.21.15.40:6835","nonce":1667529627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":4}]} 2024-04-26T21:38:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2024-04-26T21:38:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74c406c920 msgr2=0x7f74c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74c406c920 0x7f74c406ede0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f74cc034000 tx=0x7f74cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:38:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 msgr2=0x7f74d8155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d8155600 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f74c800d830 tx=0x7f74c800dcf0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 shutdown_connections 2024-04-26T21:38:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74c406c920 0x7f74c406ede0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74d8162880 0x7f74d806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d80d39f0 0x7f74d806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 --2- 172.21.15.40:0/2404472116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74d80b5f20 0x7f74d8155600 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 >> 172.21.15.40:0/2404472116 conn(0x7f74d80d4f60 msgr2=0x7f74d80b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 shutdown_connections 2024-04-26T21:38:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:28.527+0000 7f74dd19d640 1 -- 172.21.15.40:0/2404472116 wait complete. 2024-04-26T21:38:28.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:38:18.310357+0000', 'modified': '2024-04-26T21:38:21.316626+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6546}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6546': {'gid': 6546, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/1667529627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1667529627}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1667529627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6546, 'qdb_cluster': [6546]} 2024-04-26T21:38:28.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:38:28.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:38:29.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.047+0000 7fcceccda640 1 Processor -- start 2024-04-26T21:38:29.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.047+0000 7fcceccda640 1 -- start start 2024-04-26T21:38:29.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcceccda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcce80d3bc0 0x7fcce80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcceccda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce80b7e50 con 0x7fcce8162880 2024-04-26T21:38:29.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcceccda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce80d1800 con 0x7fcce80b5f20 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcceccda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcce80d0580 con 0x7fcce80d3bc0 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcce659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcce8162880 0x7fcce80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58754/0 (socket says 172.21.15.40:58754) 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcce659c640 1 -- 172.21.15.40:0/3703894763 learned_addr learned my addr 172.21.15.40:0/3703894763 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447242994 0 0) 0x7fcce80b7e50 con 0x7fcce8162880 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fccdffff640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcce80d3bc0 0x7fcce80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccd4003520 con 0x7fcce8162880 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 201268324 0 0) 0x7fccd4003520 con 0x7fcce8162880 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcce80d3bc0 msgr2=0x7fcce80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcce5d9b640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcce80d3bc0 0x7fcce80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.048+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 >> v1:172.21.15.184:6789/0 conn(0x7fcce80b5f20 legacy=0x7fcce80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce80d07d0 con 0x7fcce8162880 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fccd8001670 con 0x7fcce8162880 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 >> v1:172.21.15.40:6789/0 conn(0x7fcce8162880 legacy=0x7fcce80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcce80d3bc0 0x7fccd40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccd4003fb0 0x7fccd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd4004d40 0x7fccd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcce80d0580 con 0x7fccd4003fb0 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccd400bb00 con 0x7fccd4004d40 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fccd400bd00 con 0x7fcce80d3bc0 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fccdffff640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccd4003fb0 0x7fccd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce659c640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcce80d3bc0 0x7fccd40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd4004d40 0x7fccd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fccdffff640 1 -- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcce80d3bc0 msgr2=0x7fccd40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fccdffff640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcce80d3bc0 0x7fccd40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fccdffff640 1 -- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd4004d40 msgr2=0x7fccd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fccdffff640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd4004d40 0x7fccd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fccdffff640 1 -- 172.21.15.40:0/3703894763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd4003520 con 0x7fccd4003fb0 2024-04-26T21:38:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.049+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd4004d40 0x7fccd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fccdffff640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccd4003fb0 0x7fccd4004800 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009ac0 tx=0x7fccd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584ec33f54af5009 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0036070 con 0x7fccd4003fb0 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd0031400 con 0x7fccd4003fb0 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcce5d9b640 1 -- 172.21.15.40:0/3703894763 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0009690 con 0x7fccd4003fb0 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcceccda640 1 -- 172.21.15.40:0/3703894763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccd4003fb0 msgr2=0x7fccd4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcceccda640 1 --2- 172.21.15.40:0/3703894763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccd4003fb0 0x7fccd4004800 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009ac0 tx=0x7fccd4005830 comp rx=0 tx=0).stop 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3703894763 reap_dead start 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcceccda640 1 -- 172.21.15.40:0/3703894763 shutdown_connections 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcceccda640 1 -- 172.21.15.40:0/3703894763 >> 172.21.15.40:0/3703894763 conn(0x7fcce80d4f60 msgr2=0x7fcce80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcceccda640 1 -- 172.21.15.40:0/3703894763 shutdown_connections 2024-04-26T21:38:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.050+0000 7fcceccda640 1 -- 172.21.15.40:0/3703894763 wait complete. 2024-04-26T21:38:29.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.052+0000 7fcceccda640 1 Processor -- start 2024-04-26T21:38:29.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.052+0000 7fcceccda640 1 -- start start 2024-04-26T21:38:29.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcceccda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4003fb0 0x7fcce8154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcceccda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce80ce170 0x7fcce80ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcceccda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcce80a1df0 0x7fcce80a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcceccda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8162c60 con 0x7fcce80a1df0 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcceccda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcce80d0d90 con 0x7fcce80ce170 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcceccda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcce8162de0 con 0x7fccd4003fb0 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcce80a1df0 0x7fcce80a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcce80a1df0 0x7fcce80a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50164/0 (socket says 172.21.15.40:50164) 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3138445764 learned_addr learned my addr 172.21.15.40:0/3138445764 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fccdffff640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce80ce170 0x7fcce80ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce659c640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4003fb0 0x7fcce8154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4003fb0 msgr2=0x7fcce8154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4003fb0 0x7fcce8154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce80ce170 msgr2=0x7fcce80ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce80ce170 0x7fcce80ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3138445764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce80a2b70 con 0x7fcce80a1df0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce659c640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4003fb0 0x7fcce8154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcce80a1df0 0x7fcce80a2250 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcce0000f80 tx=0x7fcce000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fccdeffd640 1 -- 172.21.15.40:0/3138445764 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0006b50 con 0x7fcce80a1df0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fccdeffd640 1 -- 172.21.15.40:0/3138445764 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce0006d10 con 0x7fcce80a1df0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fcceccda640 1 -- 172.21.15.40:0/3138445764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce8151eb0 con 0x7fcce80a1df0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.053+0000 7fccdeffd640 1 -- 172.21.15.40:0/3138445764 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0003970 con 0x7fcce80a1df0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.054+0000 7fcceccda640 1 -- 172.21.15.40:0/3138445764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce8152470 con 0x7fcce80a1df0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.055+0000 7fccdeffd640 1 -- 172.21.15.40:0/3138445764 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcce001a020 con 0x7fcce80a1df0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.055+0000 7fccdeffd640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccc806c920 0x7fccc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.055+0000 7fccdeffd640 1 -- 172.21.15.40:0/3138445764 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fcce008b280 con 0x7fcce80a1df0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.056+0000 7fcce659c640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccc806c920 0x7fccc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.056+0000 7fcce659c640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccc806c920 0x7fccc806ede0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcce8160080 tx=0x7fccd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.056+0000 7fccef9f8740 1 Processor -- start 2024-04-26T21:38:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccef9f8740 1 -- start start 2024-04-26T21:38:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 0x558032a71820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a8f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a47120 0x558032a8d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccef9f8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5580329ab740 con 0x558032a8ece0 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccef9f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5580329473a0 con 0x558032a71440 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccef9f8740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5580329a7700 con 0x558032a47120 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a8f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.057+0000 7fccdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a8f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50168/0 (socket says 172.21.15.40:50168) 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccdffff640 1 -- 172.21.15.40:0/3226882926 learned_addr learned my addr 172.21.15.40:0/3226882926 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fcce659c640 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 0x558032a71820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a47120 0x558032a8d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccdffff640 1 -- 172.21.15.40:0/3226882926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a47120 msgr2=0x558032a8d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccdffff640 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a47120 0x558032a8d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccdffff640 1 -- 172.21.15.40:0/3226882926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 msgr2=0x558032a71820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccdffff640 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 0x558032a71820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccdffff640 1 -- 172.21.15.40:0/3226882926 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558032963750 con 0x558032a8ece0 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccdffff640 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a8f230 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009ac0 tx=0x7fccd002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269f51f5b0fefc61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccc7fff640 1 -- 172.21.15.40:0/3226882926 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0036040 con 0x558032a8ece0 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccc7fff640 1 -- 172.21.15.40:0/3226882926 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd0004380 con 0x558032a8ece0 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.058+0000 7fccc7fff640 1 -- 172.21.15.40:0/3226882926 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0003810 con 0x558032a8ece0 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 -- 172.21.15.40:0/3226882926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 msgr2=0x558032a8f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a8f230 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009ac0 tx=0x7fccd002e000 comp rx=0 tx=0).stop 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 -- 172.21.15.40:0/3226882926 shutdown_connections 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a47120 0x558032a8d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a8f230 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3226882926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 0x558032a71820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 -- 172.21.15.40:0/3226882926 >> 172.21.15.40:0/3226882926 conn(0x558032a43a50 msgr2=0x558032a8c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 -- 172.21.15.40:0/3226882926 shutdown_connections 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 -- 172.21.15.40:0/3226882926 wait complete. 2024-04-26T21:38:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.059+0000 7fccef9f8740 1 -- start start 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.060+0000 7fccef9f8740 1 Processor -- start 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.060+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 0x558032a6fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.060+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.060+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032a5d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.060+0000 7fccef9f8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558032a47ec0 con 0x558032a8ece0 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.060+0000 7fccef9f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558032932fe0 con 0x558032a5b380 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.060+0000 7fccef9f8740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558032a41980 con 0x558032a71440 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccef9f8740 10 client.0 _ll_register_callbacks cb 0x558032949150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50170/0 (socket says 172.21.15.40:50170) 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 -- 172.21.15.40:0/3547217215 learned_addr learned my addr 172.21.15.40:0/3547217215 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fcce659c640 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 0x558032a6fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032a5d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 -- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 msgr2=0x558032a6fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 0x558032a6fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 -- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 msgr2=0x558032a5d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032a5d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 -- 172.21.15.40:0/3547217215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580326bd2d0 con 0x558032a8ece0 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032a5d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccdffff640 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5ae40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009b20 tx=0x7fccd002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ada42e0936b0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccc57fa640 1 -- 172.21.15.40:0/3547217215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0036070 con 0x558032a8ece0 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccc57fa640 1 -- 172.21.15.40:0/3547217215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fccd0005ba0 con 0x558032a8ece0 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccef9f8740 1 -- 172.21.15.40:0/3547217215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd0004380 con 0x558032a8ece0 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccc57fa640 1 -- 172.21.15.40:0/3547217215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fccd0055050 con 0x558032a8ece0 2024-04-26T21:38:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccef9f8740 10 client.6711 fetch_fsmap learned FSMap version 49 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.061+0000 7fccef9f8740 1 -- 172.21.15.40:0/3547217215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x5580327876b0 con 0x558032a8ece0 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccc57fa640 1 -- 172.21.15.40:0/3547217215 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd0004380 con 0x558032a8ece0 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccc57fa640 1 -- 172.21.15.40:0/3547217215 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7fccd0058020 con 0x558032a8ece0 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccef9f8740 10 client.6711 fetch_fsmap finished waiting for FSMap version 49 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccef9f8740 10 client.6711 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccef9f8740 20 client.6711 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x558032a5e230 0x558032a5e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccef9f8740 4 client.6711 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccef9f8740 1 -- 172.21.15.40:0/3547217215 --> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580329b9350 con 0x558032a5e230 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fcce659c640 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x558032a5e230 0x558032a5e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fcce659c640 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x558032a5e230 0x558032a5e610 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b05408aa78bc82d8 server_cookie=3d0984ac83a56145 in_seq=0 out_seq=0 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.062+0000 7fccc57fa640 10 client.6711 ms_handle_connect on v2:172.21.15.40:6832/1667529627 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccc57fa640 1 -- 172.21.15.40:0/3547217215 <== mds.0 v2:172.21.15.40:6832/1667529627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580329b9350 con 0x558032a5e230 2024-04-26T21:38:29.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccc57fa640 10 client.6711 handle_command_reply: tid=0 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 client.6711 shutdown 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 -- 172.21.15.40:0/3547217215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 msgr2=0x558032a5ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5ae40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009b20 tx=0x7fccd002c000 comp rx=0 tx=0).stop 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 -- 172.21.15.40:0/3547217215 shutdown_connections 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x558032a5e230 0x558032a5e610 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032a5d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5ae40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccc57fa640 0 client.6711 ms_handle_reset on v2:172.21.15.40:6832/1667529627 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3547217215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 0x558032a6fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 -- 172.21.15.40:0/3547217215 >> 172.21.15.40:0/3547217215 conn(0x558032a47120 msgr2=0x558032a8f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 -- 172.21.15.40:0/3547217215 shutdown_connections 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 1 -- 172.21.15.40:0/3547217215 wait complete. 2024-04-26T21:38:29.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.063+0000 7fccef9f8740 20 client.6711 trim_cache size 0 max 16384 2024-04-26T21:38:29.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.082+0000 7fccef9f8740 1 Processor -- start 2024-04-26T21:38:29.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.082+0000 7fccef9f8740 1 -- start start 2024-04-26T21:38:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.082+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a5b380 0x558032a445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 0x558032a5ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032ab0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fccef9f8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8201650 con 0x558032a8ece0 2024-04-26T21:38:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fccef9f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcce82009b0 con 0x558032a71440 2024-04-26T21:38:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fccef9f8740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcce8201000 con 0x558032a5b380 2024-04-26T21:38:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032ab0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032ab0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50174/0 (socket says 172.21.15.40:50174) 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3368499019 learned_addr learned my addr 172.21.15.40:0/3368499019 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3368499019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a5b380 msgr2=0x558032a445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3368499019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a5b380 0x558032a445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3368499019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 msgr2=0x558032a5ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3368499019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 0x558032a5ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 -- 172.21.15.40:0/3368499019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580329fefc0 con 0x558032a8ece0 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/3368499019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032ab0a90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcce00013d0 tx=0x7fcce0058a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab17fc74bb2c5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fccc77fe640 1 -- 172.21.15.40:0/3368499019 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce008b0c0 con 0x558032a8ece0 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fccc77fe640 1 -- 172.21.15.40:0/3368499019 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce005d1d0 con 0x558032a8ece0 2024-04-26T21:38:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.083+0000 7fccc77fe640 1 -- 172.21.15.40:0/3368499019 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0061a00 con 0x558032a8ece0 2024-04-26T21:38:29.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 -- 172.21.15.40:0/3368499019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 msgr2=0x558032ab0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3368499019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032ab0a90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcce00013d0 tx=0x7fcce0058a90 comp rx=0 tx=0).stop 2024-04-26T21:38:29.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 -- 172.21.15.40:0/3368499019 shutdown_connections 2024-04-26T21:38:29.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3368499019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032ab0a90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3368499019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a71440 0x558032a5ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 --2- 172.21.15.40:0/3368499019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a5b380 0x558032a445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 -- 172.21.15.40:0/3368499019 >> 172.21.15.40:0/3368499019 conn(0x558032a47120 msgr2=0x558032a6d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 -- 172.21.15.40:0/3368499019 shutdown_connections 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 -- 172.21.15.40:0/3368499019 wait complete. 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 -- start start 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.084+0000 7fccef9f8740 1 Processor -- start 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.085+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032b08710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.085+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 0x558032a5a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.085+0000 7fccef9f8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.085+0000 7fccef9f8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558032a47ec0 con 0x558032a8ece0 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.085+0000 7fccef9f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558032932fe0 con 0x558032a5b380 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.085+0000 7fccef9f8740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558032a60670 con 0x558032a71440 2024-04-26T21:38:29.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.085+0000 7fccef9f8740 10 client.0 _ll_register_callbacks cb 0x7fccd8000f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50176/0 (socket says 172.21.15.40:50176) 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 -- 172.21.15.40:0/1069198795 learned_addr learned my addr 172.21.15.40:0/1069198795 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce659c640 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032b08710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccdffff640 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 0x558032a5a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 -- 172.21.15.40:0/1069198795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 msgr2=0x558032a5a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 0x558032a5a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 -- 172.21.15.40:0/1069198795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 msgr2=0x558032b08710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032b08710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 -- 172.21.15.40:0/1069198795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558032aafb20 con 0x558032a8ece0 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fcce6d9d640 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5a850 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcce00895f0 tx=0x7fcce0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b098a3a727ea444 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccc57fa640 1 -- 172.21.15.40:0/1069198795 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0089170 con 0x558032a8ece0 2024-04-26T21:38:29.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccc57fa640 1 -- 172.21.15.40:0/1069198795 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fcce0072e40 con 0x558032a8ece0 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccef9f8740 1 -- 172.21.15.40:0/1069198795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce005d1d0 con 0x558032a8ece0 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccc57fa640 1 -- 172.21.15.40:0/1069198795 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcce0089310 con 0x558032a8ece0 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccef9f8740 10 client.6717 fetch_fsmap learned FSMap version 49 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccef9f8740 1 -- 172.21.15.40:0/1069198795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x558032a81be0 con 0x558032a8ece0 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.086+0000 7fccc57fa640 1 -- 172.21.15.40:0/1069198795 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce005d1d0 con 0x558032a8ece0 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccc57fa640 1 -- 172.21.15.40:0/1069198795 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7fcce0078300 con 0x558032a8ece0 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 10 client.6717 fetch_fsmap finished waiting for FSMap version 49 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 10 client.6717 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 20 client.6717 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x558032a8d940 0x558032a74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 4 client.6717 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 -- 172.21.15.40:0/1069198795 --> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] -- command(tid 0: {"prefix": "status"}) -- 0x5580329b31c0 con 0x558032a8d940 2024-04-26T21:38:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fcce659c640 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x558032a8d940 0x558032a74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccc57fa640 10 client.6717 ms_handle_connect on v2:172.21.15.40:6832/1667529627 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fcce659c640 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x558032a8d940 0x558032a74210 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=170a26addaba9195 server_cookie=f85b42947d770d50 in_seq=0 out_seq=0 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccc57fa640 1 -- 172.21.15.40:0/1069198795 <== mds.0 v2:172.21.15.40:6832/1667529627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5580329b31c0 con 0x558032a8d940 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccc57fa640 10 client.6717 handle_command_reply: tid=0 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 client.6717 shutdown 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 -- 172.21.15.40:0/1069198795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 msgr2=0x558032a5a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5a850 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcce00895f0 tx=0x7fcce0001360 comp rx=0 tx=0).stop 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 -- 172.21.15.40:0/1069198795 shutdown_connections 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558032a8ece0 0x558032a5a850 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558032a71440 0x558032a5a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.087+0000 7fccef9f8740 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x558032a8d940 0x558032a74210 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fccef9f8740 1 --2- 172.21.15.40:0/1069198795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558032a5b380 0x558032b08710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fccc57fa640 0 client.6717 ms_handle_reset on v2:172.21.15.40:6832/1667529627 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fccef9f8740 1 -- 172.21.15.40:0/1069198795 >> 172.21.15.40:0/1069198795 conn(0x558032a47120 msgr2=0x558032ab0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fccef9f8740 1 -- 172.21.15.40:0/1069198795 shutdown_connections 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fccef9f8740 1 -- 172.21.15.40:0/1069198795 wait complete. 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fccef9f8740 20 client.6717 trim_cache size 0 max 16384 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6546, 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:38:29.096 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.151477996000001, 2024-04-26T21:38:29.097 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 49, 2024-04-26T21:38:29.097 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 66, 2024-04-26T21:38:29.097 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 66, 2024-04-26T21:38:29.097 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.160533792000001 2024-04-26T21:38:29.097 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 -- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccc806c920 msgr2=0x7fccc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccc806c920 0x7fccc806ede0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcce8160080 tx=0x7fccd8009040 comp rx=0 tx=0).stop 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 -- 172.21.15.40:0/3138445764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcce80a1df0 msgr2=0x7fcce80a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcce80a1df0 0x7fcce80a2250 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcce0000f80 tx=0x7fcce000eb90 comp rx=0 tx=0).stop 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 -- 172.21.15.40:0/3138445764 shutdown_connections 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fccc806c920 0x7fccc806ede0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcce80a1df0 0x7fcce80a2250 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce80ce170 0x7fcce80ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 --2- 172.21.15.40:0/3138445764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4003fb0 0x7fcce8154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 -- 172.21.15.40:0/3138445764 >> 172.21.15.40:0/3138445764 conn(0x7fcce80d4f60 msgr2=0x7fcce8160140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 -- 172.21.15.40:0/3138445764 shutdown_connections 2024-04-26T21:38:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.088+0000 7fcceccda640 1 -- 172.21.15.40:0/3138445764 wait complete. 2024-04-26T21:38:29.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:38:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.200+0000 7efd33607640 1 Processor -- start 2024-04-26T21:38:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.200+0000 7efd33607640 1 -- start start 2024-04-26T21:38:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.201+0000 7efd33607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 0x7efd2c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.201+0000 7efd33607640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efd2c0b2850 0x7efd2c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.201+0000 7efd33607640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd2c0d0b70 con 0x7efd2c15f3a0 2024-04-26T21:38:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.201+0000 7efd33607640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efd2c0d0e20 con 0x7efd2c0b2850 2024-04-26T21:38:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.201+0000 7efd33607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c0b1d70 con 0x7efd2c0d06f0 2024-04-26T21:38:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.201+0000 7efd313a3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd2c15f3a0 0x7efd2c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58764/0 (socket says 172.21.15.40:58764) 2024-04-26T21:38:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.201+0000 7efd313a3640 1 -- 172.21.15.40:0/3384197794 learned_addr learned my addr 172.21.15.40:0/3384197794 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd31ba4640 1 --2- 172.21.15.40:0/3384197794 >> v2:172.21.15.184:3300/0 conn(0x7efd2c0b2850 0x7efd2c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd30ba2640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 0x7efd2c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687490597 0 0) 0x7efd2c0d0b70 con 0x7efd2c15f3a0 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd14003520 con 0x7efd2c15f3a0 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3428455511 0 0) 0x7efd14003520 con 0x7efd2c15f3a0 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 msgr2=0x7efd2c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 0x7efd2c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 >> v2:172.21.15.184:3300/0 conn(0x7efd2c0b2850 msgr2=0x7efd2c0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 --2- 172.21.15.40:0/3384197794 >> v2:172.21.15.184:3300/0 conn(0x7efd2c0b2850 0x7efd2c0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c0b2050 con 0x7efd2c15f3a0 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7efd24001670 con 0x7efd2c15f3a0 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 >> v1:172.21.15.40:6789/0 conn(0x7efd2c15f3a0 legacy=0x7efd2c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140039b0 0x7efd14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14004800 0x7efd14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 0x7efd140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c0d0e20 con 0x7efd140039b0 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c0b1d70 con 0x7efd14004800 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd1400b9c0 con 0x7efd2c0d06f0 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd313a3640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140039b0 0x7efd14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd30ba2640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14004800 0x7efd14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd31ba4640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 0x7efd140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd313a3640 1 -- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 msgr2=0x7efd140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd313a3640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 0x7efd140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd313a3640 1 -- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14004800 msgr2=0x7efd14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd313a3640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14004800 0x7efd14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd313a3640 1 -- 172.21.15.40:0/3384197794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd14003520 con 0x7efd140039b0 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.202+0000 7efd31ba4640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd2c0d06f0 0x7efd140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.203+0000 7efd313a3640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140039b0 0x7efd14003dd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7efd24001050 tx=0x7efd14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2422f5e9d5f23a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.203+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2402e070 con 0x7efd140039b0 2024-04-26T21:38:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.203+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd24005af0 con 0x7efd140039b0 2024-04-26T21:38:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.203+0000 7efd2bfff640 1 -- 172.21.15.40:0/3384197794 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24007d70 con 0x7efd140039b0 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.204+0000 7efd33607640 1 -- 172.21.15.40:0/3384197794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140039b0 msgr2=0x7efd14003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.204+0000 7efd33607640 1 --2- 172.21.15.40:0/3384197794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140039b0 0x7efd14003dd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7efd24001050 tx=0x7efd14005830 comp rx=0 tx=0).stop 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.204+0000 7efd31ba4640 1 -- 172.21.15.40:0/3384197794 reap_dead start 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.204+0000 7efd33607640 1 -- 172.21.15.40:0/3384197794 shutdown_connections 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.204+0000 7efd33607640 1 -- 172.21.15.40:0/3384197794 >> 172.21.15.40:0/3384197794 conn(0x7efd2c0d4f90 msgr2=0x7efd2c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.204+0000 7efd33607640 1 -- 172.21.15.40:0/3384197794 shutdown_connections 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.204+0000 7efd33607640 1 -- 172.21.15.40:0/3384197794 wait complete. 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.205+0000 7efd33607640 1 Processor -- start 2024-04-26T21:38:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.205+0000 7efd33607640 1 -- start start 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd33607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140039b0 0x7efd2c161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd33607640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd2c14bdc0 0x7efd2c1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd33607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd2c15c200 0x7efd2c15c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd33607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c15f900 con 0x7efd2c15c200 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd33607640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c15f780 con 0x7efd2c14bdc0 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd33607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c0d1e80 con 0x7efd140039b0 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd2c15c200 0x7efd2c15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd2c15c200 0x7efd2c15c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50186/0 (socket says 172.21.15.40:50186) 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 -- 172.21.15.40:0/3140153905 learned_addr learned my addr 172.21.15.40:0/3140153905 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd313a3640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140039b0 0x7efd2c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd30ba2640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd2c14bdc0 0x7efd2c1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 -- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140039b0 msgr2=0x7efd2c161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140039b0 0x7efd2c161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 -- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd2c14bdc0 msgr2=0x7efd2c1f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd2c14bdc0 0x7efd2c1f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 -- 172.21.15.40:0/3140153905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c161fd0 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd30ba2640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd2c14bdc0 0x7efd2c1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.206+0000 7efd31ba4640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd2c15c200 0x7efd2c15c680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0d2ae0 tx=0x7efd1c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.207+0000 7efd29ffb640 1 -- 172.21.15.40:0/3140153905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c010e80 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.207+0000 7efd29ffb640 1 -- 172.21.15.40:0/3140153905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1c003990 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.207+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c1622b0 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.207+0000 7efd29ffb640 1 -- 172.21.15.40:0/3140153905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c003da0 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.207+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c162870 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.208+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2c0694f0 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.208+0000 7efd29ffb640 1 -- 172.21.15.40:0/3140153905 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd1c01e020 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.209+0000 7efd29ffb640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd1806c8f0 0x7efd1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.209+0000 7efd29ffb640 1 -- 172.21.15.40:0/3140153905 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7efd1c08c410 con 0x7efd2c15c200 2024-04-26T21:38:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.209+0000 7efd313a3640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd1806c8f0 0x7efd1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:29.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.210+0000 7efd313a3640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd1806c8f0 0x7efd1806edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efd24033000 tx=0x7efd24009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:29.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.213+0000 7efd29ffb640 1 -- 172.21.15.40:0/3140153905 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd1c05a610 con 0x7efd2c15c200 2024-04-26T21:38:29.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.440+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efd2c0b6860 con 0x7efd2c15c200 2024-04-26T21:38:29.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.440+0000 7efd29ffb640 1 -- 172.21.15.40:0/3140153905 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7efd1c059f10 con 0x7efd2c15c200 2024-04-26T21:38:29.442 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:38:29.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd1806c8f0 msgr2=0x7efd1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd1806c8f0 0x7efd1806edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efd24033000 tx=0x7efd24009030 comp rx=0 tx=0).stop 2024-04-26T21:38:29.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd2c15c200 msgr2=0x7efd2c15c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:29.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd2c15c200 0x7efd2c15c680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0d2ae0 tx=0x7efd1c009f90 comp rx=0 tx=0).stop 2024-04-26T21:38:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 shutdown_connections 2024-04-26T21:38:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd1806c8f0 0x7efd1806edb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd2c15c200 0x7efd2c15c680 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd2c14bdc0 0x7efd2c1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 --2- 172.21.15.40:0/3140153905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140039b0 0x7efd2c161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 >> 172.21.15.40:0/3140153905 conn(0x7efd2c0d4f90 msgr2=0x7efd2c06e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 shutdown_connections 2024-04-26T21:38:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29.443+0000 7efd33607640 1 -- 172.21.15.40:0/3140153905 wait complete. 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:38:29.455 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:38:29.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:29.456 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:29.471 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42538sec preferred_lft 42538sec 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:29.472 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:38:29.473 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:38:29.474 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:29.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:29.474 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:38:29.535 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:38:29.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:29.537 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:38:29.593 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:38:29.593 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:38:29.593 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:38:29.593 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:38:29.593 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:38:29.594 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:38:29.594 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:38:29.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:38:29.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:38:29.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:38:29.594 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:38:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:38:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:38:29.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:29.720 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:38:29.736 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:38:29.736 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:38:29.736 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:38:29.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:29.736 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:38:29.799 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:38:29.800 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:29.855 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:38:29.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:38:29.855 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:29.912 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:38:29.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:29.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:38:29.994 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:38:29.994 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:38:30.002 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:38:30.002 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:38:30.002 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:38:30.002 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:38:30.002 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:38:30.003 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:38:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:38:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:38:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:38:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:38:30.004 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:38:30.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:30.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:30.069 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:38:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:38:30.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:38:30.172 INFO:teuthology.orchestra.run.smithi040.stdout:client6726 2024-04-26T21:38:30.172 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:38:30.172 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:30.188 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d; sync .') 2024-04-26T21:38:30.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:38:30.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:38:30.297 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:38:30.297 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:30.312 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:30.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.453+0000 7ff2c8637640 1 Processor -- start 2024-04-26T21:38:30.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.453+0000 7ff2c8637640 1 -- start start 2024-04-26T21:38:30.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c8637640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c0162880 0x7ff2c00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c8637640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2c00b5f20 0x7ff2c00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c8637640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c00b7e50 con 0x7ff2c00d39f0 2024-04-26T21:38:30.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c8637640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c00d0580 con 0x7ff2c00b5f20 2024-04-26T21:38:30.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c8637640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c00d1800 con 0x7ff2c0162880 2024-04-26T21:38:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c5bd2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2c00b5f20 0x7ff2c00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c5bd2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2c00b5f20 0x7ff2c00d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50200/0 (socket says 172.21.15.40:50200) 2024-04-26T21:38:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c5bd2640 1 -- 172.21.15.40:0/2649306488 learned_addr learned my addr 172.21.15.40:0/2649306488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c63d3640 1 --2- 172.21.15.40:0/2649306488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c0162880 0x7ff2c00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c5bd2640 1 -- 172.21.15.40:0/2649306488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c0162880 msgr2=0x7ff2c00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c5bd2640 1 --2- 172.21.15.40:0/2649306488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c0162880 0x7ff2c00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c5bd2640 1 -- 172.21.15.40:0/2649306488 >> v1:172.21.15.184:6789/0 conn(0x7ff2c00d39f0 legacy=0x7ff2c00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.454+0000 7ff2c5bd2640 1 -- 172.21.15.40:0/2649306488 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c00d07d0 con 0x7ff2c00b5f20 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.455+0000 7ff2c5bd2640 1 --2- 172.21.15.40:0/2649306488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c00d34b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009870 tx=0x7ff2b002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9980711d1904f261 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.455+0000 7ff2c53d1640 1 -- 172.21.15.40:0/2649306488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0037070 con 0x7ff2c00b5f20 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.455+0000 7ff2c53d1640 1 -- 172.21.15.40:0/2649306488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b0032440 con 0x7ff2c00b5f20 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.455+0000 7ff2c53d1640 1 -- 172.21.15.40:0/2649306488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0032810 con 0x7ff2c00b5f20 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.455+0000 7ff2c8637640 1 -- 172.21.15.40:0/2649306488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 msgr2=0x7ff2c00d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.455+0000 7ff2c8637640 1 --2- 172.21.15.40:0/2649306488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c00d34b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009870 tx=0x7ff2b002a210 comp rx=0 tx=0).stop 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.456+0000 7ff2c8637640 1 -- 172.21.15.40:0/2649306488 shutdown_connections 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.456+0000 7ff2c8637640 1 --2- 172.21.15.40:0/2649306488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c00d34b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.456+0000 7ff2c8637640 1 --2- 172.21.15.40:0/2649306488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c0162880 0x7ff2c00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.456+0000 7ff2c8637640 1 -- 172.21.15.40:0/2649306488 >> 172.21.15.40:0/2649306488 conn(0x7ff2c00d4f60 msgr2=0x7ff2c00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.456+0000 7ff2c8637640 1 -- 172.21.15.40:0/2649306488 shutdown_connections 2024-04-26T21:38:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.456+0000 7ff2c8637640 1 -- 172.21.15.40:0/2649306488 wait complete. 2024-04-26T21:38:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.457+0000 7ff2c8637640 1 Processor -- start 2024-04-26T21:38:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.457+0000 7ff2c8637640 1 -- start start 2024-04-26T21:38:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.457+0000 7ff2c8637640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.457+0000 7ff2c8637640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c00d39f0 0x7ff2c00ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c8637640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2c0162880 0x7ff2c00cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c8637640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0162c60 con 0x7ff2c00b5f20 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c8637640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c00d0d90 con 0x7ff2c0162880 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c8637640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c0162de0 con 0x7ff2c00d39f0 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c015a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50206/0 (socket says 172.21.15.40:50206) 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 -- 172.21.15.40:0/363772304 learned_addr learned my addr 172.21.15.40:0/363772304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c5bd2640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c00d39f0 0x7ff2c00ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c6bd4640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2c0162880 0x7ff2c00cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 -- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c00d39f0 msgr2=0x7ff2c00ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c00d39f0 0x7ff2c00ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 -- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2c0162880 msgr2=0x7ff2c00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2c0162880 0x7ff2c00cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 -- 172.21.15.40:0/363772304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c00b3a40 con 0x7ff2c00b5f20 2024-04-26T21:38:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c63d3640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c015a3e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00c7f0 tx=0x7ff2bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2b6ffd640 1 -- 172.21.15.40:0/363772304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc015070 con 0x7ff2c00b5f20 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2b6ffd640 1 -- 172.21.15.40:0/363772304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2bc004240 con 0x7ff2c00b5f20 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c00b3d20 con 0x7ff2c00b5f20 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2b6ffd640 1 -- 172.21.15.40:0/363772304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc007620 con 0x7ff2c00b5f20 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.458+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c00b42e0 con 0x7ff2c00b5f20 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.459+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c00694f0 con 0x7ff2c00b5f20 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.460+0000 7ff2b6ffd640 1 -- 172.21.15.40:0/363772304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2bc0079d0 con 0x7ff2c00b5f20 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.460+0000 7ff2b6ffd640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff2a406c8f0 0x7ff2a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.460+0000 7ff2b6ffd640 1 -- 172.21.15.40:0/363772304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7ff2bc08c170 con 0x7ff2c00b5f20 2024-04-26T21:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.460+0000 7ff2c5bd2640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff2a406c8f0 0x7ff2a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.461+0000 7ff2c5bd2640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff2a406c8f0 0x7ff2a406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0034000 tx=0x7ff2b002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:30.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.464+0000 7ff2b6ffd640 1 -- 172.21.15.40:0/363772304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2bc05a410 con 0x7ff2c00b5f20 2024-04-26T21:38:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.653+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2c00badb0 con 0x7ff2c00b5f20 2024-04-26T21:38:30.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.654+0000 7ff2b6ffd640 1 -- 172.21.15.40:0/363772304 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16848 (secure 0 0 0) 0x7ff2bc059d10 con 0x7ff2c00b5f20 2024-04-26T21:38:30.655 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:30.655 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:38:19.929996+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff2a406c8f0 msgr2=0x7ff2a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff2a406c8f0 0x7ff2a406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0034000 tx=0x7ff2b002e000 comp rx=0 tx=0).stop 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 msgr2=0x7ff2c015a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c015a3e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00c7f0 tx=0x7ff2bc00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 shutdown_connections 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff2a406c8f0 0x7ff2a406edb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2c0162880 0x7ff2c00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2c00d39f0 0x7ff2c00ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 --2- 172.21.15.40:0/363772304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2c00b5f20 0x7ff2c015a3e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 >> 172.21.15.40:0/363772304 conn(0x7ff2c00d4f60 msgr2=0x7ff2c006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 shutdown_connections 2024-04-26T21:38:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.656+0000 7ff2c8637640 1 -- 172.21.15.40:0/363772304 wait complete. 2024-04-26T21:38:30.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:38:16.583866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:38:17.253849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:38:30.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:38:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.772+0000 7f26efc99640 1 Processor -- start 2024-04-26T21:38:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.772+0000 7f26efc99640 1 -- start start 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26efc99640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26e80cfde0 0x7f26e80d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26efc99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8071260 0x7f26e815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26efc99640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e80d2390 con 0x7f26e8070350 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26efc99640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f26e80afab0 con 0x7f26e80cfde0 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26efc99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26e806dd20 con 0x7f26e8071260 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26eda35640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26e80cfde0 0x7f26e80d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26ed234640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f26e8070350 0x7f26e8070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58776/0 (socket says 172.21.15.40:58776) 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26ed234640 1 -- 172.21.15.40:0/1772809246 learned_addr learned my addr 172.21.15.40:0/1772809246 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26ee236640 1 --2- 172.21.15.40:0/1772809246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8071260 0x7f26e815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26eda35640 1 -- 172.21.15.40:0/1772809246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8071260 msgr2=0x7f26e815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26eda35640 1 --2- 172.21.15.40:0/1772809246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8071260 0x7f26e815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26eda35640 1 -- 172.21.15.40:0/1772809246 >> v1:172.21.15.40:6789/0 conn(0x7f26e8070350 legacy=0x7f26e8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26eca33640 1 -- 172.21.15.40:0/1772809246 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463161304 0 0) 0x7f26e80d2390 con 0x7f26e8070350 2024-04-26T21:38:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26eda35640 1 -- 172.21.15.40:0/1772809246 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e806d3f0 con 0x7f26e80cfde0 2024-04-26T21:38:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.773+0000 7f26ee236640 1 --2- 172.21.15.40:0/1772809246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8071260 0x7f26e815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.774+0000 7f26eda35640 1 --2- 172.21.15.40:0/1772809246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e80d1b60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f26e40097a0 tx=0x7f26e402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1170d6a7c876581b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.774+0000 7f26eca33640 1 -- 172.21.15.40:0/1772809246 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4037070 con 0x7f26e80cfde0 2024-04-26T21:38:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.774+0000 7f26eca33640 1 -- 172.21.15.40:0/1772809246 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e402ae50 con 0x7f26e80cfde0 2024-04-26T21:38:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.774+0000 7f26eca33640 1 -- 172.21.15.40:0/1772809246 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4032660 con 0x7f26e80cfde0 2024-04-26T21:38:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.774+0000 7f26efc99640 1 -- 172.21.15.40:0/1772809246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 msgr2=0x7f26e80d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.774+0000 7f26efc99640 1 --2- 172.21.15.40:0/1772809246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e80d1b60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f26e40097a0 tx=0x7f26e402a100 comp rx=0 tx=0).stop 2024-04-26T21:38:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.775+0000 7f26efc99640 1 -- 172.21.15.40:0/1772809246 shutdown_connections 2024-04-26T21:38:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.775+0000 7f26efc99640 1 --2- 172.21.15.40:0/1772809246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8071260 0x7f26e815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.775+0000 7f26efc99640 1 --2- 172.21.15.40:0/1772809246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e80d1b60 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.775+0000 7f26efc99640 1 -- 172.21.15.40:0/1772809246 >> 172.21.15.40:0/1772809246 conn(0x7f26e80d2d80 msgr2=0x7f26e80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.775+0000 7f26efc99640 1 -- 172.21.15.40:0/1772809246 shutdown_connections 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.775+0000 7f26efc99640 1 -- 172.21.15.40:0/1772809246 wait complete. 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.776+0000 7f26efc99640 1 Processor -- start 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.776+0000 7f26efc99640 1 -- start start 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.776+0000 7f26efc99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8070350 0x7f26e81603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.776+0000 7f26efc99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e8071260 0x7f26e809f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.776+0000 7f26efc99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.776+0000 7f26efc99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80d20a0 con 0x7f26e80cfde0 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.776+0000 7f26efc99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26e806d9b0 con 0x7f26e8071260 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.776+0000 7f26efc99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26e815ef30 con 0x7f26e8070350 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e814ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50226/0 (socket says 172.21.15.40:50226) 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 -- 172.21.15.40:0/2410546816 learned_addr learned my addr 172.21.15.40:0/2410546816 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26eda35640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8070350 0x7f26e81603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ed234640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e8071260 0x7f26e809f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 -- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8070350 msgr2=0x7f26e81603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8070350 0x7f26e81603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 -- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e8071260 msgr2=0x7f26e809f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e8071260 0x7f26e809f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 -- 172.21.15.40:0/2410546816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e809fe90 con 0x7f26e80cfde0 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26ee236640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e814ffd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007e40 tx=0x7f26e000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26de7fc640 1 -- 172.21.15.40:0/2410546816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e001a070 con 0x7f26e80cfde0 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26de7fc640 1 -- 172.21.15.40:0/2410546816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0015600 con 0x7f26e80cfde0 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26de7fc640 1 -- 172.21.15.40:0/2410546816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00159f0 con 0x7f26e80cfde0 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e8160ad0 con 0x7f26e80cfde0 2024-04-26T21:38:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.777+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e8161090 con 0x7f26e80cfde0 2024-04-26T21:38:30.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.778+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e80694f0 con 0x7f26e80cfde0 2024-04-26T21:38:30.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.779+0000 7f26de7fc640 1 -- 172.21.15.40:0/2410546816 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26e001d070 con 0x7f26e80cfde0 2024-04-26T21:38:30.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.779+0000 7f26de7fc640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26cc06c8f0 0x7f26cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:30.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.779+0000 7f26de7fc640 1 -- 172.21.15.40:0/2410546816 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f26e008bf30 con 0x7f26e80cfde0 2024-04-26T21:38:30.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.779+0000 7f26eda35640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26cc06c8f0 0x7f26cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:30.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.780+0000 7f26eda35640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26cc06c8f0 0x7f26cc06edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f26e4034000 tx=0x7f26e402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:30.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.784+0000 7f26de7fc640 1 -- 172.21.15.40:0/2410546816 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26e0010070 con 0x7f26e80cfde0 2024-04-26T21:38:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.968+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26e8071810 con 0x7f26e80cfde0 2024-04-26T21:38:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.969+0000 7f26de7fc640 1 -- 172.21.15.40:0/2410546816 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f26e0059b80 con 0x7f26e80cfde0 2024-04-26T21:38:30.970 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26cc06c8f0 msgr2=0x7f26cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26cc06c8f0 0x7f26cc06edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f26e4034000 tx=0x7f26e402e000 comp rx=0 tx=0).stop 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 msgr2=0x7f26e814ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e814ffd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007e40 tx=0x7f26e000d4a0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 shutdown_connections 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26cc06c8f0 0x7f26cc06edb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e80cfde0 0x7f26e814ffd0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e8071260 0x7f26e809f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 --2- 172.21.15.40:0/2410546816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e8070350 0x7f26e81603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 >> 172.21.15.40:0/2410546816 conn(0x7f26e80d2d80 msgr2=0x7f26e80c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 shutdown_connections 2024-04-26T21:38:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:30.971+0000 7f26efc99640 1 -- 172.21.15.40:0/2410546816 wait complete. 2024-04-26T21:38:30.983 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:38:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.085+0000 7fbf42f7f640 1 Processor -- start 2024-04-26T21:38:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.085+0000 7fbf42f7f640 1 -- start start 2024-04-26T21:38:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf42f7f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf3c0d3bc0 0x7fbf3c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf42f7f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf3c0d1040 con 0x7fbf3c15f550 2024-04-26T21:38:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf42f7f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf3c0c9680 con 0x7fbf3c15f170 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf42f7f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3c0c9430 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf4151c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf3c0d3bc0 0x7fbf3c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf3bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf3c15f550 0x7fbf3c0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58782/0 (socket says 172.21.15.40:58782) 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf3bfff640 1 -- 172.21.15.40:0/2387598203 learned_addr learned my addr 172.21.15.40:0/2387598203 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf4151c640 1 -- 172.21.15.40:0/2387598203 >> v1:172.21.15.184:6789/0 conn(0x7fbf3c15f170 legacy=0x7fbf3c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf4151c640 1 -- 172.21.15.40:0/2387598203 >> v1:172.21.15.40:6789/0 conn(0x7fbf3c15f550 legacy=0x7fbf3c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf4151c640 1 -- 172.21.15.40:0/2387598203 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c0c9960 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.086+0000 7fbf3b7fe640 1 -- 172.21.15.40:0/2387598203 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752372970 0 0) 0x7fbf30001110 con 0x7fbf3c15f550 2024-04-26T21:38:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.087+0000 7fbf4151c640 1 --2- 172.21.15.40:0/2387598203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c0d5a80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0c95b0 tx=0x7fbf3402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e31b9d54c7ed28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.087+0000 7fbf3b7fe640 1 -- 172.21.15.40:0/2387598203 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34007740 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.087+0000 7fbf3b7fe640 1 -- 172.21.15.40:0/2387598203 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf340078e0 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.087+0000 7fbf3b7fe640 1 -- 172.21.15.40:0/2387598203 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34007cb0 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.088+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2387598203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 msgr2=0x7fbf3c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.088+0000 7fbf42f7f640 1 --2- 172.21.15.40:0/2387598203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c0d5a80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0c95b0 tx=0x7fbf3402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.088+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2387598203 shutdown_connections 2024-04-26T21:38:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.088+0000 7fbf42f7f640 1 --2- 172.21.15.40:0/2387598203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c0d5a80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.088+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2387598203 >> 172.21.15.40:0/2387598203 conn(0x7fbf3c0d4f60 msgr2=0x7fbf3c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.088+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2387598203 shutdown_connections 2024-04-26T21:38:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.088+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2387598203 wait complete. 2024-04-26T21:38:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.089+0000 7fbf42f7f640 1 Processor -- start 2024-04-26T21:38:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.089+0000 7fbf42f7f640 1 -- start start 2024-04-26T21:38:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf42f7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf42f7f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf3c15f170 0x7fbf3c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf42f7f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf3c159ca0 0x7fbf3c0b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf42f7f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c0d2e30 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf42f7f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c0d2cb0 con 0x7fbf3c15f170 2024-04-26T21:38:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf42f7f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c0d2fb0 con 0x7fbf3c159ca0 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50240/0 (socket says 172.21.15.40:50240) 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 -- 172.21.15.40:0/2541077433 learned_addr learned my addr 172.21.15.40:0/2541077433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 -- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf3c159ca0 msgr2=0x7fbf3c0b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf3c159ca0 0x7fbf3c0b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 -- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf3c15f170 msgr2=0x7fbf3c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf3c15f170 0x7fbf3c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 -- 172.21.15.40:0/2541077433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c150a80 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.090+0000 7fbf40d1b640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c150540 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00c410 tx=0x7fbf2c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.091+0000 7fbf397fa640 1 -- 172.21.15.40:0/2541077433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c017070 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.091+0000 7fbf397fa640 1 -- 172.21.15.40:0/2541077433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2c0035f0 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.091+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c1f4ef0 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.091+0000 7fbf397fa640 1 -- 172.21.15.40:0/2541077433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c003a20 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.091+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c0cd2b0 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.092+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3c0cce00 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.092+0000 7fbf397fa640 1 -- 172.21.15.40:0/2541077433 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf2c01a070 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.092+0000 7fbf397fa640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbf2806c8f0 0x7fbf2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.092+0000 7fbf397fa640 1 -- 172.21.15.40:0/2541077433 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fbf2c08b1f0 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.093+0000 7fbf3bfff640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbf2806c8f0 0x7fbf2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.093+0000 7fbf3bfff640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbf2806c8f0 0x7fbf2806edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbf30007fd0 tx=0x7fbf30007960 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.097+0000 7fbf397fa640 1 -- 172.21.15.40:0/2541077433 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf2c059490 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.284+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbf3c0d36c0 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.284+0000 7fbf397fa640 1 -- 172.21.15.40:0/2541077433 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7fbf2c0010c0 con 0x7fbf3c0d3bc0 2024-04-26T21:38:31.286 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:38:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.286+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbf2806c8f0 msgr2=0x7fbf2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.286+0000 7fbf42f7f640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbf2806c8f0 0x7fbf2806edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbf30007fd0 tx=0x7fbf30007960 comp rx=0 tx=0).stop 2024-04-26T21:38:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.286+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 msgr2=0x7fbf3c150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.286+0000 7fbf42f7f640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c150540 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00c410 tx=0x7fbf2c00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.287+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 shutdown_connections 2024-04-26T21:38:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.287+0000 7fbf42f7f640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbf2806c8f0 0x7fbf2806edb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.287+0000 7fbf42f7f640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf3c159ca0 0x7fbf3c0b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.287+0000 7fbf42f7f640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf3c15f170 0x7fbf3c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.287+0000 7fbf42f7f640 1 --2- 172.21.15.40:0/2541077433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c0d3bc0 0x7fbf3c150540 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.287+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 >> 172.21.15.40:0/2541077433 conn(0x7fbf3c0d4f60 msgr2=0x7fbf3c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.287+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 shutdown_connections 2024-04-26T21:38:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.287+0000 7fbf42f7f640 1 -- 172.21.15.40:0/2541077433 wait complete. 2024-04-26T21:38:31.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:38:31.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.401+0000 7f28be2fc640 1 Processor -- start 2024-04-26T21:38:31.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.401+0000 7f28be2fc640 1 -- start start 2024-04-26T21:38:31.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.401+0000 7f28be2fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f28b815f170 0x7f28b80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.401+0000 7f28be2fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b80d1040 con 0x7f28b815f550 2024-04-26T21:38:31.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.401+0000 7f28be2fc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b80c9680 con 0x7f28b80d3bc0 2024-04-26T21:38:31.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.401+0000 7f28be2fc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f28b80c9430 con 0x7f28b815f170 2024-04-26T21:38:31.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f28b815f550 0x7f28b80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58788/0 (socket says 172.21.15.40:58788) 2024-04-26T21:38:31.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b77fe640 1 -- 172.21.15.40:0/2837283418 learned_addr learned my addr 172.21.15.40:0/2837283418 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:31.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 661378641 0 0) 0x7f28ac001110 con 0x7f28b815f550 2024-04-26T21:38:31.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28a0003520 con 0x7f28b815f550 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b7fff640 1 --2- 172.21.15.40:0/2837283418 >> v2:172.21.15.184:3300/0 conn(0x7f28b815f170 0x7f28b80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 759831611 0 0) 0x7f28a0003520 con 0x7f28b815f550 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 >> v2:172.21.15.184:3300/0 conn(0x7f28b815f170 msgr2=0x7f28b80d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 --2- 172.21.15.40:0/2837283418 >> v2:172.21.15.184:3300/0 conn(0x7f28b815f170 0x7f28b80d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 >> v1:172.21.15.184:6789/0 conn(0x7f28b80d3bc0 legacy=0x7f28b80ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b80c9960 con 0x7f28b815f550 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f28ac001840 con 0x7f28b815f550 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 >> v1:172.21.15.40:6789/0 conn(0x7f28b815f550 legacy=0x7f28b80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a00039b0 0x7f28a0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a0004800 0x7f28a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a00051c0 0x7f28a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80c9430 con 0x7f28a0004800 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28a000c090 con 0x7f28a00039b0 2024-04-26T21:38:31.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28a000c290 con 0x7f28a00051c0 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b77fe640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a0004800 0x7f28a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28b7fff640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a00039b0 0x7f28a0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.402+0000 7f28bc899640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a00051c0 0x7f28a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.403+0000 7f28b77fe640 1 -- 172.21.15.40:0/2837283418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a00051c0 msgr2=0x7f28a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.403+0000 7f28b77fe640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a00051c0 0x7f28a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.403+0000 7f28b77fe640 1 -- 172.21.15.40:0/2837283418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a00039b0 msgr2=0x7f28a0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.403+0000 7f28b77fe640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a00039b0 0x7f28a0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.403+0000 7f28b77fe640 1 -- 172.21.15.40:0/2837283418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a0003520 con 0x7f28a0004800 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.403+0000 7f28bc899640 1 -- 172.21.15.40:0/2837283418 reap_dead start 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.403+0000 7f28b77fe640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a0004800 0x7f28a0004c80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f28b80c8990 tx=0x7f28a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726cadbcb0536b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00c0e0 con 0x7f28a0004800 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ac02c040 con 0x7f28a0004800 2024-04-26T21:38:31.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28b6ffd640 1 -- 172.21.15.40:0/2837283418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac005d30 con 0x7f28a0004800 2024-04-26T21:38:31.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28be2fc640 1 -- 172.21.15.40:0/2837283418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a0004800 msgr2=0x7f28a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28be2fc640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a0004800 0x7f28a0004c80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f28b80c8990 tx=0x7f28a0005830 comp rx=0 tx=0).stop 2024-04-26T21:38:31.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28be2fc640 1 -- 172.21.15.40:0/2837283418 shutdown_connections 2024-04-26T21:38:31.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28be2fc640 1 --2- 172.21.15.40:0/2837283418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a0004800 0x7f28a0004c80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28be2fc640 1 -- 172.21.15.40:0/2837283418 >> 172.21.15.40:0/2837283418 conn(0x7f28b80d4f60 msgr2=0x7f28b80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:31.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28be2fc640 1 -- 172.21.15.40:0/2837283418 shutdown_connections 2024-04-26T21:38:31.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.404+0000 7f28be2fc640 1 -- 172.21.15.40:0/2837283418 wait complete. 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.405+0000 7f28be2fc640 1 Processor -- start 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.405+0000 7f28be2fc640 1 -- start start 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28be2fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a0004800 0x7f28b809ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28be2fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b80b1d10 0x7f28b80b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28be2fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b8158020 0x7f28b81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28be2fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80d5c00 con 0x7f28b8158020 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28be2fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80d5a80 con 0x7f28a0004800 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28be2fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28b80d5d80 con 0x7f28b80b1d10 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28bc899640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b8158020 0x7f28b81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28bc899640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b8158020 0x7f28b81584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50262/0 (socket says 172.21.15.40:50262) 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28bc899640 1 -- 172.21.15.40:0/272365314 learned_addr learned my addr 172.21.15.40:0/272365314 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28b77fe640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b80b1d10 0x7f28b80b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.406+0000 7f28b7fff640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a0004800 0x7f28b809ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28bc899640 1 -- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b80b1d10 msgr2=0x7f28b80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28bc899640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b80b1d10 0x7f28b80b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28bc899640 1 -- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a0004800 msgr2=0x7f28b809ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28bc899640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a0004800 0x7f28b809ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28bc899640 1 -- 172.21.15.40:0/272365314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b81589e0 con 0x7f28b8158020 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28bc899640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b8158020 0x7f28b81584a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f28b8073bc0 tx=0x7f28b000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28b4ff9640 1 -- 172.21.15.40:0/272365314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0003b80 con 0x7f28b8158020 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28b4ff9640 1 -- 172.21.15.40:0/272365314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b0018070 con 0x7f28b8158020 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b809f540 con 0x7f28b8158020 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28b4ff9640 1 -- 172.21.15.40:0/272365314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00050d0 con 0x7f28b8158020 2024-04-26T21:38:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.407+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b809fde0 con 0x7f28b8158020 2024-04-26T21:38:31.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.408+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b80694f0 con 0x7f28b8158020 2024-04-26T21:38:31.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.409+0000 7f28b4ff9640 1 -- 172.21.15.40:0/272365314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28b0005270 con 0x7f28b8158020 2024-04-26T21:38:31.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.409+0000 7f28b4ff9640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.409+0000 7f28b4ff9640 1 -- 172.21.15.40:0/272365314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f28b008bb20 con 0x7f28b8158020 2024-04-26T21:38:31.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.409+0000 7f28b7fff640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.410+0000 7f28b7fff640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28a406c8f0 0x7f28a406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f28b8052070 tx=0x7f28a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.413+0000 7f28b4ff9640 1 -- 172.21.15.40:0/272365314 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28b001a310 con 0x7f28b8158020 2024-04-26T21:38:31.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.599+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28b8074590 con 0x7f28b8158020 2024-04-26T21:38:31.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.599+0000 7f28b4ff9640 1 -- 172.21.15.40:0/272365314 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7f28b0059e50 con 0x7f28b8158020 2024-04-26T21:38:31.600 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:38:31.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28a406c8f0 msgr2=0x7f28a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28a406c8f0 0x7f28a406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f28b8052070 tx=0x7f28a8008040 comp rx=0 tx=0).stop 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b8158020 msgr2=0x7f28b81584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b8158020 0x7f28b81584a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f28b8073bc0 tx=0x7f28b000fd40 comp rx=0 tx=0).stop 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 shutdown_connections 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b8158020 0x7f28b81584a0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b80b1d10 0x7f28b80b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 --2- 172.21.15.40:0/272365314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a0004800 0x7f28b809ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 >> 172.21.15.40:0/272365314 conn(0x7f28b80d4f60 msgr2=0x7f28b80b9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 shutdown_connections 2024-04-26T21:38:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.601+0000 7f28be2fc640 1 -- 172.21.15.40:0/272365314 wait complete. 2024-04-26T21:38:31.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:38:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.714+0000 7ff5ce1a4640 1 Processor -- start 2024-04-26T21:38:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.714+0000 7ff5ce1a4640 1 -- start start 2024-04-26T21:38:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5ce1a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5c80d3bc0 0x7ff5c80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5ce1a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5ce1a4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c80b7e50 con 0x7ff5c8162880 2024-04-26T21:38:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5ce1a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c80d0580 con 0x7ff5c80d3bc0 2024-04-26T21:38:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5ce1a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c80d1800 con 0x7ff5c80b5f20 2024-04-26T21:38:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5c6ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5c80d3bc0 0x7ff5c80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c80b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:55122/0 (socket says 172.21.15.40:55122) 2024-04-26T21:38:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5c7fff640 1 -- 172.21.15.40:0/3874522472 learned_addr learned my addr 172.21.15.40:0/3874522472 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5c7fff640 1 -- 172.21.15.40:0/3874522472 >> v2:172.21.15.184:3300/0 conn(0x7ff5c80d3bc0 msgr2=0x7ff5c80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5c7fff640 1 --2- 172.21.15.40:0/3874522472 >> v2:172.21.15.184:3300/0 conn(0x7ff5c80d3bc0 0x7ff5c80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5c7fff640 1 -- 172.21.15.40:0/3874522472 >> v1:172.21.15.184:6789/0 conn(0x7ff5c8162880 legacy=0x7ff5c80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.715+0000 7ff5c7fff640 1 -- 172.21.15.40:0/3874522472 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80d07d0 con 0x7ff5c80b5f20 2024-04-26T21:38:31.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.716+0000 7ff5c7fff640 1 --2- 172.21.15.40:0/3874522472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c80b8300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009e50 tx=0x7ff5c002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e41f78100bd8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.716+0000 7ff5c67fc640 1 -- 172.21.15.40:0/3874522472 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0036070 con 0x7ff5c80b5f20 2024-04-26T21:38:31.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.716+0000 7ff5c67fc640 1 -- 172.21.15.40:0/3874522472 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c0031600 con 0x7ff5c80b5f20 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.716+0000 7ff5c67fc640 1 -- 172.21.15.40:0/3874522472 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c00319d0 con 0x7ff5c80b5f20 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.717+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/3874522472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 msgr2=0x7ff5c80b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.717+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/3874522472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c80b8300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009e50 tx=0x7ff5c002a710 comp rx=0 tx=0).stop 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.717+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/3874522472 shutdown_connections 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.717+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/3874522472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c80b8300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.717+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/3874522472 >> v2:172.21.15.184:3300/0 conn(0x7ff5c80d3bc0 0x7ff5c80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.717+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/3874522472 >> 172.21.15.40:0/3874522472 conn(0x7ff5c80d4f60 msgr2=0x7ff5c80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.717+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/3874522472 shutdown_connections 2024-04-26T21:38:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.717+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/3874522472 wait complete. 2024-04-26T21:38:31.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.718+0000 7ff5ce1a4640 1 Processor -- start 2024-04-26T21:38:31.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.718+0000 7ff5ce1a4640 1 -- start start 2024-04-26T21:38:31.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5ce1a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c81f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5ce1a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c80d3bc0 0x7ff5c80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5ce1a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8162880 0x7ff5c80b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5ce1a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8162c60 con 0x7ff5c8162880 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5ce1a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c80d0d90 con 0x7ff5c80d3bc0 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5ce1a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c8162de0 con 0x7ff5c80b5f20 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8162880 0x7ff5c80b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8162880 0x7ff5c80b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50266/0 (socket says 172.21.15.40:50266) 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5c7fff640 1 -- 172.21.15.40:0/2735517656 learned_addr learned my addr 172.21.15.40:0/2735517656 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5c77fe640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c81f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.719+0000 7ff5c6ffd640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c80d3bc0 0x7ff5c80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5c7fff640 1 -- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 msgr2=0x7ff5c81f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5c7fff640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c81f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5c7fff640 1 -- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c80d3bc0 msgr2=0x7ff5c80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5c7fff640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c80d3bc0 0x7ff5c80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5c7fff640 1 -- 172.21.15.40:0/2735517656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80b3800 con 0x7ff5c8162880 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5c7fff640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8162880 0x7ff5c80b4c60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0034040 tx=0x7ff5c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5affff640 1 -- 172.21.15.40:0/2735517656 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0036050 con 0x7ff5c8162880 2024-04-26T21:38:31.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5affff640 1 -- 172.21.15.40:0/2735517656 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c0042070 con 0x7ff5c8162880 2024-04-26T21:38:31.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5affff640 1 -- 172.21.15.40:0/2735517656 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0009510 con 0x7ff5c8162880 2024-04-26T21:38:31.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c80b56d0 con 0x7ff5c8162880 2024-04-26T21:38:31.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.720+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c81f7ff0 con 0x7ff5c8162880 2024-04-26T21:38:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.721+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c80694f0 con 0x7ff5c8162880 2024-04-26T21:38:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.722+0000 7ff5affff640 1 -- 172.21.15.40:0/2735517656 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5c0007c70 con 0x7ff5c8162880 2024-04-26T21:38:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.722+0000 7ff5affff640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b406c8f0 0x7ff5b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.722+0000 7ff5affff640 1 -- 172.21.15.40:0/2735517656 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7ff5c00ac950 con 0x7ff5c8162880 2024-04-26T21:38:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.722+0000 7ff5c77fe640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b406c8f0 0x7ff5b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.723+0000 7ff5c77fe640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b406c8f0 0x7ff5b406edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc001780 tx=0x7ff5bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:31.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.726+0000 7ff5affff640 1 -- 172.21.15.40:0/2735517656 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5c80b3800 con 0x7ff5c8162880 2024-04-26T21:38:31.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.930+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff5c8069c90 con 0x7ff5c8162880 2024-04-26T21:38:31.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.931+0000 7ff5affff640 1 -- 172.21.15.40:0/2735517656 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) ==== 80+0+0 (secure 0 0 0) 0x7ff5c007a630 con 0x7ff5c8162880 2024-04-26T21:38:31.932 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:38:31.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b406c8f0 msgr2=0x7ff5b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b406c8f0 0x7ff5b406edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc001780 tx=0x7ff5bc009040 comp rx=0 tx=0).stop 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8162880 msgr2=0x7ff5c80b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8162880 0x7ff5c80b4c60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0034040 tx=0x7ff5c0004a40 comp rx=0 tx=0).stop 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 shutdown_connections 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5b406c8f0 0x7ff5b406edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5c8162880 0x7ff5c80b4c60 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5c80d3bc0 0x7ff5c80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 --2- 172.21.15.40:0/2735517656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5c80b5f20 0x7ff5c81f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 >> 172.21.15.40:0/2735517656 conn(0x7ff5c80d4f60 msgr2=0x7ff5c8160320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 shutdown_connections 2024-04-26T21:38:31.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:31.933+0000 7ff5ce1a4640 1 -- 172.21.15.40:0/2735517656 wait complete. 2024-04-26T21:38:31.945 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:38:31.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:31.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:31.975 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:38:31.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:31.975 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:38:32.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-04-26T21:38:32.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.174+0000 7fb5ad39e640 1 Processor -- start 2024-04-26T21:38:32.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.174+0000 7fb5ad39e640 1 -- start start 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5ad39e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb5a80d3930 0x7fb5a80cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5ad39e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a80d5ab0 con 0x7fb5a815f200 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5ad39e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a80d2d70 con 0x7fb5a80d1f30 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5ad39e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb5a80d5d90 con 0x7fb5a80d3930 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb5a815f200 0x7fb5a80d3220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58804/0 (socket says 172.21.15.40:58804) 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a7fff640 1 -- 172.21.15.40:0/3454908415 learned_addr learned my addr 172.21.15.40:0/3454908415 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/3454908415 >> v2:172.21.15.184:3300/0 conn(0x7fb5a80d3930 0x7fb5a80cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2896460799 0 0) 0x7fb5a80d5ab0 con 0x7fb5a815f200 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb594003520 con 0x7fb5a815f200 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2488222560 0 0) 0x7fb594003520 con 0x7fb5a815f200 2024-04-26T21:38:32.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 >> v2:172.21.15.184:3300/0 conn(0x7fb5a80d3930 msgr2=0x7fb5a80cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 --2- 172.21.15.40:0/3454908415 >> v2:172.21.15.184:3300/0 conn(0x7fb5a80d3930 0x7fb5a80cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 >> v1:172.21.15.184:6789/0 conn(0x7fb5a80d1f30 legacy=0x7fb5a80cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a80b3dc0 con 0x7fb5a815f200 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb590001670 con 0x7fb5a815f200 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/3454908415 >> v2:172.21.15.184:3300/0 conn(0x7fb5a80d3930 0x7fb5a80cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 >> v1:172.21.15.40:6789/0 conn(0x7fb5a815f200 legacy=0x7fb5a80d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940039b0 0x7fb594003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb594004800 0x7fb594004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5940051c0 0x7fb59400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb59400c130 con 0x7fb5940051c0 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb59400c2e0 con 0x7fb594004800 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.175+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb59400c4e0 con 0x7fb5940039b0 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5940051c0 0x7fb59400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb594004800 0x7fb594004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940039b0 0x7fb594003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5acb9d640 1 -- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940039b0 msgr2=0x7fb594003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940039b0 0x7fb594003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5acb9d640 1 -- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb594004800 msgr2=0x7fb594004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb594004800 0x7fb594004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5acb9d640 1 -- 172.21.15.40:0/3454908415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb594003520 con 0x7fb5940051c0 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.176+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb594004800 0x7fb594004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5acb9d640 1 -- 172.21.15.40:0/3454908415 reap_dead start 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5940051c0 0x7fb59400b9c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0001e50 tx=0x7fb594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95706a57b8ec2a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598009860 con 0x7fb5940051c0 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a0003970 con 0x7fb5940051c0 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5a6ffd640 1 -- 172.21.15.40:0/3454908415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0001a70 con 0x7fb5940051c0 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5ad39e640 1 -- 172.21.15.40:0/3454908415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5940051c0 msgr2=0x7fb59400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5ad39e640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5940051c0 0x7fb59400b9c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0001e50 tx=0x7fb594005830 comp rx=0 tx=0).stop 2024-04-26T21:38:32.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5ad39e640 1 -- 172.21.15.40:0/3454908415 shutdown_connections 2024-04-26T21:38:32.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5ad39e640 1 --2- 172.21.15.40:0/3454908415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5940051c0 0x7fb59400b9c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5ad39e640 1 -- 172.21.15.40:0/3454908415 >> 172.21.15.40:0/3454908415 conn(0x7fb5a80d4df0 msgr2=0x7fb5a80d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:32.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5ad39e640 1 -- 172.21.15.40:0/3454908415 shutdown_connections 2024-04-26T21:38:32.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.177+0000 7fb5ad39e640 1 -- 172.21.15.40:0/3454908415 wait complete. 2024-04-26T21:38:32.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.179+0000 7fb5ad39e640 1 Processor -- start 2024-04-26T21:38:32.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.179+0000 7fb5ad39e640 1 -- start start 2024-04-26T21:38:32.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5ad39e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940051c0 0x7fb5a81f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5ad39e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a815f7f0 0x7fb5a8161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5ad39e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5a8154110 0x7fb5a8154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5ad39e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a80d3050 con 0x7fb5a8154110 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5ad39e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a80d3220 con 0x7fb5a815f7f0 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5ad39e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a80d33a0 con 0x7fb5940051c0 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5a8154110 0x7fb5a8154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5a8154110 0x7fb5a8154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50282/0 (socket says 172.21.15.40:50282) 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 -- 172.21.15.40:0/1056416050 learned_addr learned my addr 172.21.15.40:0/1056416050 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a815f7f0 0x7fb5a8161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940051c0 0x7fb5a81f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 -- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940051c0 msgr2=0x7fb5a81f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940051c0 0x7fb5a81f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 -- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a815f7f0 msgr2=0x7fb5a8161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a815f7f0 0x7fb5a8161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 -- 172.21.15.40:0/1056416050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a8154ad0 con 0x7fb5a8154110 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940051c0 0x7fb5a81f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5a8154110 0x7fb5a8154590 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb5a002a920 tx=0x7fb5a00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5a4ff9640 1 -- 172.21.15.40:0/1056416050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0001a70 con 0x7fb5a8154110 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5a4ff9640 1 -- 172.21.15.40:0/1056416050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a002ac70 con 0x7fb5a8154110 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5ad39e640 1 -- 172.21.15.40:0/1056416050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a81f5d00 con 0x7fb5a8154110 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5a4ff9640 1 -- 172.21.15.40:0/1056416050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0006d70 con 0x7fb5a8154110 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.180+0000 7fb5ad39e640 1 -- 172.21.15.40:0/1056416050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a8160070 con 0x7fb5a8154110 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.182+0000 7fb5a4ff9640 1 -- 172.21.15.40:0/1056416050 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5a0007030 con 0x7fb5a8154110 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.182+0000 7fb5a4ff9640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb58806c8f0 0x7fb58806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.182+0000 7fb5a4ff9640 1 -- 172.21.15.40:0/1056416050 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fb5a00ab870 con 0x7fb5a8154110 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.183+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb58806c8f0 0x7fb58806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.183+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb58806c8f0 0x7fb58806edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80cfed0 tx=0x7fb590009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.183+0000 7fb5b13f2740 1 Processor -- start 2024-04-26T21:38:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5b13f2740 1 -- start start 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 0x562a55a63530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a5dfe0 0x562a55a61d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a62290 0x562a55a626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562a559c7d80 con 0x562a55a5dfe0 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a559818b0 con 0x562a55a62290 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562a559c3d40 con 0x562a55a63150 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a5dfe0 0x562a55a61d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a5dfe0 0x562a55a61d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50294/0 (socket says 172.21.15.40:50294) 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5a77fe640 1 -- 172.21.15.40:0/3501041386 learned_addr learned my addr 172.21.15.40:0/3501041386 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 0x562a55a63530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.184+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a62290 0x562a55a626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb5a77fe640 1 -- 172.21.15.40:0/3501041386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 msgr2=0x562a55a63530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 0x562a55a63530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb5a77fe640 1 -- 172.21.15.40:0/3501041386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a62290 msgr2=0x562a55a626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a62290 0x562a55a626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb5a77fe640 1 -- 172.21.15.40:0/3501041386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a55952a40 con 0x562a55a5dfe0 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a5dfe0 0x562a55a61d50 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb598009cc0 tx=0x7fb598009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdee16985c7dcea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb59dffb640 1 -- 172.21.15.40:0/3501041386 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59801f480 con 0x562a55a5dfe0 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb59dffb640 1 -- 172.21.15.40:0/3501041386 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59801f620 con 0x562a55a5dfe0 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.185+0000 7fb59dffb640 1 -- 172.21.15.40:0/3501041386 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59801f9f0 con 0x562a55a5dfe0 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 -- 172.21.15.40:0/3501041386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a5dfe0 msgr2=0x562a55a61d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a5dfe0 0x562a55a61d50 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb598009cc0 tx=0x7fb598009190 comp rx=0 tx=0).stop 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 -- 172.21.15.40:0/3501041386 shutdown_connections 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a62290 0x562a55a626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a5dfe0 0x562a55a61d50 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/3501041386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 0x562a55a63530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 -- 172.21.15.40:0/3501041386 >> 172.21.15.40:0/3501041386 conn(0x562a55a60250 msgr2=0x562a55aa8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 -- 172.21.15.40:0/3501041386 shutdown_connections 2024-04-26T21:38:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 -- 172.21.15.40:0/3501041386 wait complete. 2024-04-26T21:38:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 -- start start 2024-04-26T21:38:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.186+0000 7fb5b13f2740 1 Processor -- start 2024-04-26T21:38:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 0x562a55aab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a63150 0x562a55a842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55aa9ed0 0x562a55aac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562a55a8ece0 con 0x562a55aa9ed0 2024-04-26T21:38:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a5594ed10 con 0x562a55a63150 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562a55a8e160 con 0x562a55a62290 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5b13f2740 10 client.0 _ll_register_callbacks cb 0x562a5597b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5acb9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55aa9ed0 0x562a55aac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5acb9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55aa9ed0 0x562a55aac5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50298/0 (socket says 172.21.15.40:50298) 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5acb9d640 1 -- 172.21.15.40:0/852530692 learned_addr learned my addr 172.21.15.40:0/852530692 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 0x562a55aab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.187+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a63150 0x562a55a842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5acb9d640 1 -- 172.21.15.40:0/852530692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 msgr2=0x562a55aab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 0x562a55aab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5acb9d640 1 -- 172.21.15.40:0/852530692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a63150 msgr2=0x562a55a842b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a63150 0x562a55a842b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5acb9d640 1 -- 172.21.15.40:0/852530692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a557a37d0 con 0x562a55aa9ed0 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55aa9ed0 0x562a55aac5c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0081b50 tx=0x7fb5a0079050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fec624830caf9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb586ffd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb586ffd640 1 -- 172.21.15.40:0/852530692 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0081df0 con 0x562a55aa9ed0 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb586ffd640 1 -- 172.21.15.40:0/852530692 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fb5a00788f0 con 0x562a55aa9ed0 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5b13f2740 1 -- 172.21.15.40:0/852530692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb59801f620 con 0x562a55aa9ed0 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb586ffd640 1 -- 172.21.15.40:0/852530692 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb5a00794a0 con 0x562a55aa9ed0 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5b13f2740 10 client.6771 fetch_fsmap learned FSMap version 49 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5b13f2740 1 -- 172.21.15.40:0/852530692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x562a556d93f0 con 0x562a55aa9ed0 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb586ffd640 1 -- 172.21.15.40:0/852530692 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59801f620 con 0x562a55aa9ed0 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb586ffd640 1 -- 172.21.15.40:0/852530692 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7fb5a0046070 con 0x562a55aa9ed0 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5b13f2740 10 client.6771 fetch_fsmap finished waiting for FSMap version 49 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5b13f2740 10 client.6771 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.188+0000 7fb5b13f2740 20 client.6771 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x562a55aacb00 0x562a55a9fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb5b13f2740 4 client.6771 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:38:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb5b13f2740 1 -- 172.21.15.40:0/852530692 --> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a559d5990 con 0x562a55aacb00 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x562a55aacb00 0x562a55a9fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x562a55aacb00 0x562a55a9fdc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc80920a47e96bad server_cookie=1c006fa7d1439cf0 in_seq=0 out_seq=0 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb586ffd640 10 client.6771 ms_handle_connect on v2:172.21.15.40:6832/1667529627 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb586ffd640 1 -- 172.21.15.40:0/852530692 <== mds.0 v2:172.21.15.40:6832/1667529627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a559d5990 con 0x562a55aacb00 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb586ffd640 10 client.6771 handle_command_reply: tid=0 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.189+0000 7fb5b13f2740 1 client.6771 shutdown 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 -- 172.21.15.40:0/852530692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55aa9ed0 msgr2=0x562a55aac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55aa9ed0 0x562a55aac5c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0081b50 tx=0x7fb5a0079050 comp rx=0 tx=0).stop 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 -- 172.21.15.40:0/852530692 shutdown_connections 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55aa9ed0 0x562a55aac5c0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x562a55aacb00 0x562a55a9fdc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55a63150 0x562a55a842b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/852530692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 0x562a55aab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb586ffd640 0 client.6771 ms_handle_reset on v2:172.21.15.40:6832/1667529627 2024-04-26T21:38:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 -- 172.21.15.40:0/852530692 >> 172.21.15.40:0/852530692 conn(0x562a55a5dfe0 msgr2=0x562a55a63c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 -- 172.21.15.40:0/852530692 shutdown_connections 2024-04-26T21:38:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 1 -- 172.21.15.40:0/852530692 wait complete. 2024-04-26T21:38:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.190+0000 7fb5b13f2740 20 client.6771 trim_cache size 0 max 16384 2024-04-26T21:38:32.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.205+0000 7fb5b13f2740 1 Processor -- start 2024-04-26T21:38:32.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.205+0000 7fb5b13f2740 1 -- start start 2024-04-26T21:38:32.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.205+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 0x562a55aab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.205+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a63150 0x562a55a7b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.205+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55a5d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.205+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a82020a0 con 0x562a55a63150 2024-04-26T21:38:32.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.205+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a82010a0 con 0x562a55aa9ed0 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.205+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a80b8c60 con 0x562a55a62290 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a63150 0x562a55a7b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a63150 0x562a55a7b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50310/0 (socket says 172.21.15.40:50310) 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 -- 172.21.15.40:0/2005626421 learned_addr learned my addr 172.21.15.40:0/2005626421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 0x562a55aab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55a5d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 -- 172.21.15.40:0/2005626421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 msgr2=0x562a55aab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 0x562a55aab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 -- 172.21.15.40:0/2005626421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 msgr2=0x562a55a5d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55a5d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 -- 172.21.15.40:0/2005626421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a556990f0 con 0x562a55a63150 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a63150 0x562a55a7b670 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb5980097a0 tx=0x7fb598019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac709b6ef8197928 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb59d7fa640 1 -- 172.21.15.40:0/2005626421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598004240 con 0x562a55a63150 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb59d7fa640 1 -- 172.21.15.40:0/2005626421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59802f070 con 0x562a55a63150 2024-04-26T21:38:32.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.206+0000 7fb59d7fa640 1 -- 172.21.15.40:0/2005626421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598006a20 con 0x562a55a63150 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 -- 172.21.15.40:0/2005626421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a63150 msgr2=0x562a55a7b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a63150 0x562a55a7b670 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb5980097a0 tx=0x7fb598019000 comp rx=0 tx=0).stop 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 -- 172.21.15.40:0/2005626421 shutdown_connections 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55a5d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a63150 0x562a55a7b670 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/2005626421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a62290 0x562a55aab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 -- 172.21.15.40:0/2005626421 >> 172.21.15.40:0/2005626421 conn(0x562a55a5dfe0 msgr2=0x562a55aafda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 -- 172.21.15.40:0/2005626421 shutdown_connections 2024-04-26T21:38:32.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 -- 172.21.15.40:0/2005626421 wait complete. 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.207+0000 7fb5b13f2740 1 -- start start 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5b13f2740 1 Processor -- start 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a62290 0x562a55aaf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 0x562a55b20180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5b13f2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55b22c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562a55a64270 con 0x562a55a62290 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a8200a50 con 0x562a55aa9ed0 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5b13f2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562a55a63dd0 con 0x562a55a63150 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5b13f2740 10 client.0 _ll_register_callbacks cb 0x7fb5a0079360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a62290 0x562a55aaf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a62290 0x562a55aaf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50326/0 (socket says 172.21.15.40:50326) 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.208+0000 7fb5a7fff640 1 -- 172.21.15.40:0/4292228275 learned_addr learned my addr 172.21.15.40:0/4292228275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 0x562a55b20180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55b22c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5a7fff640 1 -- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 msgr2=0x562a55b20180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 0x562a55b20180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5a7fff640 1 -- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 msgr2=0x562a55b22c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55b22c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5a7fff640 1 -- 172.21.15.40:0/4292228275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a559884b0 con 0x562a55a62290 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5acb9d640 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55b22c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5a7fff640 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a62290 0x562a55aaf3a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x562a55aacf70 tx=0x7fb590001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212d45a3c1b97117 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb586ffd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb586ffd640 1 -- 172.21.15.40:0/4292228275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590006310 con 0x562a55a62290 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb586ffd640 1 -- 172.21.15.40:0/4292228275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fb590035c30 con 0x562a55a62290 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5b13f2740 1 -- 172.21.15.40:0/4292228275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb59802f070 con 0x562a55a62290 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb586ffd640 1 -- 172.21.15.40:0/4292228275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb590035e30 con 0x562a55a62290 2024-04-26T21:38:32.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5b13f2740 10 client.6777 fetch_fsmap learned FSMap version 49 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb5b13f2740 1 -- 172.21.15.40:0/4292228275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x562a55aad4f0 con 0x562a55a62290 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.209+0000 7fb586ffd640 1 -- 172.21.15.40:0/4292228275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59802f070 con 0x562a55a62290 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb586ffd640 1 -- 172.21.15.40:0/4292228275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7fb590021040 con 0x562a55a62290 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb5b13f2740 10 client.6777 fetch_fsmap finished waiting for FSMap version 49 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb5b13f2740 10 client.6777 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb5b13f2740 20 client.6777 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x562a55aa5360 0x562a55aae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb5b13f2740 4 client.6777 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb5b13f2740 1 -- 172.21.15.40:0/4292228275 --> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a559cf800 con 0x562a55aa5360 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x562a55aa5360 0x562a55aae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb5a77fe640 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x562a55aa5360 0x562a55aae680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b117ffaf78c97a13 server_cookie=5dffddff9a919305 in_seq=0 out_seq=0 2024-04-26T21:38:32.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.210+0000 7fb586ffd640 10 client.6777 ms_handle_connect on v2:172.21.15.40:6832/1667529627 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb586ffd640 1 -- 172.21.15.40:0/4292228275 <== mds.0 v2:172.21.15.40:6832/1667529627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a559cf800 con 0x562a55aa5360 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb586ffd640 10 client.6777 handle_command_reply: tid=0 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 client.6777 shutdown 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 -- 172.21.15.40:0/4292228275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a62290 msgr2=0x562a55aaf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a62290 0x562a55aaf3a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x562a55aacf70 tx=0x7fb590001a40 comp rx=0 tx=0).stop 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 -- 172.21.15.40:0/4292228275 shutdown_connections 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.40:6832/1667529627,v1:172.21.15.40:6835/1667529627] conn(0x562a55aa5360 0x562a55aae680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a55aa9ed0 0x562a55b22c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb586ffd640 0 client.6777 ms_handle_reset on v2:172.21.15.40:6832/1667529627 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562a55a63150 0x562a55b20180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 --2- 172.21.15.40:0/4292228275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562a55a62290 0x562a55aaf3a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 -- 172.21.15.40:0/4292228275 >> 172.21.15.40:0/4292228275 conn(0x562a55a5dfe0 msgr2=0x562a55a5d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:32.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 -- 172.21.15.40:0/4292228275 shutdown_connections 2024-04-26T21:38:32.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.221+0000 7fb5b13f2740 1 -- 172.21.15.40:0/4292228275 wait complete. 2024-04-26T21:38:32.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5b13f2740 20 client.6777 trim_cache size 0 max 16384 2024-04-26T21:38:32.224 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:38:32.224 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-04-26T21:38:32.224 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0 2024-04-26T21:38:32.224 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:38:32.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 -- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb58806c8f0 msgr2=0x7fb58806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb58806c8f0 0x7fb58806edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80cfed0 tx=0x7fb590009040 comp rx=0 tx=0).stop 2024-04-26T21:38:32.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 -- 172.21.15.40:0/1056416050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5a8154110 msgr2=0x7fb5a8154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5a8154110 0x7fb5a8154590 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb5a002a920 tx=0x7fb5a00014a0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 -- 172.21.15.40:0/1056416050 shutdown_connections 2024-04-26T21:38:32.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb58806c8f0 0x7fb58806edb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5a8154110 0x7fb5a8154590 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a815f7f0 0x7fb5a8161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 --2- 172.21.15.40:0/1056416050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5940051c0 0x7fb5a81f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 -- 172.21.15.40:0/1056416050 >> 172.21.15.40:0/1056416050 conn(0x7fb5a80d4df0 msgr2=0x7fb5a815e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:32.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 -- 172.21.15.40:0/1056416050 shutdown_connections 2024-04-26T21:38:32.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.222+0000 7fb5ad39e640 1 -- 172.21.15.40:0/1056416050 wait complete. 2024-04-26T21:38:32.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:38:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.333+0000 7f9881351640 1 Processor -- start 2024-04-26T21:38:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.333+0000 7f9881351640 1 -- start start 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.334+0000 7f9881351640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0b6760 0x7f987c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.334+0000 7f9881351640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f987c0d3470 0x7f987c1673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.334+0000 7f9881351640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f987c0d2aa0 con 0x7f987c0d0a20 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.334+0000 7f9881351640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f987c0d2570 con 0x7f987c0d3470 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.334+0000 7f9881351640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f987c0d0590 con 0x7f987c0b6760 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.334+0000 7f987b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f987c0d3470 0x7f987c1673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.334+0000 7f987b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f987c0d3470 0x7f987c1673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50334/0 (socket says 172.21.15.40:50334) 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.334+0000 7f987b7fe640 1 -- 172.21.15.40:0/2826646132 learned_addr learned my addr 172.21.15.40:0/2826646132 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.335+0000 7f987b7fe640 1 -- 172.21.15.40:0/2826646132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0b6760 msgr2=0x7f987c0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.335+0000 7f987affd640 1 --2- 172.21.15.40:0/2826646132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0b6760 0x7f987c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.335+0000 7f987b7fe640 1 --2- 172.21.15.40:0/2826646132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0b6760 0x7f987c0c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.335+0000 7f987b7fe640 1 -- 172.21.15.40:0/2826646132 >> v1:172.21.15.184:6789/0 conn(0x7f987c0d0a20 legacy=0x7f987c0d2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.335+0000 7f987b7fe640 1 -- 172.21.15.40:0/2826646132 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c0d27c0 con 0x7f987c0d3470 2024-04-26T21:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.335+0000 7f987b7fe640 1 --2- 172.21.15.40:0/2826646132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0d3470 0x7f987c1673a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f987c0d26f0 tx=0x7f986c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d92e46b975a5fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9879ffb640 1 -- 172.21.15.40:0/2826646132 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c036070 con 0x7f987c0d3470 2024-04-26T21:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9879ffb640 1 -- 172.21.15.40:0/2826646132 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c031600 con 0x7f987c0d3470 2024-04-26T21:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9879ffb640 1 -- 172.21.15.40:0/2826646132 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c0319d0 con 0x7f987c0d3470 2024-04-26T21:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9881351640 1 -- 172.21.15.40:0/2826646132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0d3470 msgr2=0x7f987c1673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9881351640 1 --2- 172.21.15.40:0/2826646132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0d3470 0x7f987c1673a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f987c0d26f0 tx=0x7f986c02a630 comp rx=0 tx=0).stop 2024-04-26T21:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9881351640 1 -- 172.21.15.40:0/2826646132 shutdown_connections 2024-04-26T21:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9881351640 1 --2- 172.21.15.40:0/2826646132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0d3470 0x7f987c1673a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9881351640 1 --2- 172.21.15.40:0/2826646132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0b6760 0x7f987c0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9881351640 1 -- 172.21.15.40:0/2826646132 >> 172.21.15.40:0/2826646132 conn(0x7f987c0d5190 msgr2=0x7f987c0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9881351640 1 -- 172.21.15.40:0/2826646132 shutdown_connections 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.336+0000 7f9881351640 1 -- 172.21.15.40:0/2826646132 wait complete. 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f9881351640 1 Processor -- start 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f9881351640 1 -- start start 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f9881351640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0b6760 0x7f987c155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f9881351640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0d0a20 0x7f987c1f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f9881351640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987c0c98c0 0x7f987c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f9881351640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f987c0d0ea0 con 0x7f987c0b6760 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f9881351640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f987c0d0220 con 0x7f987c0c98c0 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f9881351640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f987c0d1020 con 0x7f987c0d0a20 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f987affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0b6760 0x7f987c155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f987affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0b6760 0x7f987c155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50336/0 (socket says 172.21.15.40:50336) 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.338+0000 7f987affd640 1 -- 172.21.15.40:0/550532881 learned_addr learned my addr 172.21.15.40:0/550532881 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f987b7fe640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987c0c98c0 0x7f987c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f987a7fc640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0d0a20 0x7f987c1f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f987affd640 1 -- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0d0a20 msgr2=0x7f987c1f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f987affd640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0d0a20 0x7f987c1f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f987affd640 1 -- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987c0c98c0 msgr2=0x7f987c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f987affd640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987c0c98c0 0x7f987c0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f987affd640 1 -- 172.21.15.40:0/550532881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c0ca8e0 con 0x7f987c0b6760 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f987affd640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0b6760 0x7f987c155be0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f987400c7f0 tx=0x7f987400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f98737fe640 1 -- 172.21.15.40:0/550532881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874015070 con 0x7f987c0b6760 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f98737fe640 1 -- 172.21.15.40:0/550532881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9874004220 con 0x7f987c0b6760 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f98737fe640 1 -- 172.21.15.40:0/550532881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874007620 con 0x7f987c0b6760 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c0c9d70 con 0x7f987c0b6760 2024-04-26T21:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.339+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c0ca320 con 0x7f987c0b6760 2024-04-26T21:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.340+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987c0694f0 con 0x7f987c0b6760 2024-04-26T21:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.341+0000 7f98737fe640 1 -- 172.21.15.40:0/550532881 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98740079d0 con 0x7f987c0b6760 2024-04-26T21:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.341+0000 7f98737fe640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f985c06c8f0 0x7f985c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.341+0000 7f98737fe640 1 -- 172.21.15.40:0/550532881 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f987408c0a0 con 0x7f987c0b6760 2024-04-26T21:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.341+0000 7f987a7fc640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f985c06c8f0 0x7f985c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:32.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.342+0000 7f987a7fc640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f985c06c8f0 0x7f985c06edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f987c0b6170 tx=0x7f9864009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.345+0000 7f98737fe640 1 -- 172.21.15.40:0/550532881 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f987405a340 con 0x7f987c0b6760 2024-04-26T21:38:32.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.574+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f987c068d30 con 0x7f987c0b6760 2024-04-26T21:38:32.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.574+0000 7f98737fe640 1 -- 172.21.15.40:0/550532881 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7f9874059c40 con 0x7f987c0b6760 2024-04-26T21:38:32.575 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:38:32.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f985c06c8f0 msgr2=0x7f985c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f985c06c8f0 0x7f985c06edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f987c0b6170 tx=0x7f9864009040 comp rx=0 tx=0).stop 2024-04-26T21:38:32.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0b6760 msgr2=0x7f987c155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0b6760 0x7f987c155be0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f987400c7f0 tx=0x7f987400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 shutdown_connections 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f985c06c8f0 0x7f985c06edb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987c0c98c0 0x7f987c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987c0d0a20 0x7f987c1f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 --2- 172.21.15.40:0/550532881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f987c0b6760 0x7f987c155be0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 >> 172.21.15.40:0/550532881 conn(0x7f987c0d5190 msgr2=0x7f987c06e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 shutdown_connections 2024-04-26T21:38:32.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32.576+0000 7f9881351640 1 -- 172.21.15.40:0/550532881 wait complete. 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:38:32.588 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:38:32.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:32.589 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:38:32.604 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:38:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42535sec preferred_lft 42535sec 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:32.606 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:38:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:38:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:38:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:32.607 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:38:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:38:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:38:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:38:32.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:32.607 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:38:32.668 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:38:32.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:32.669 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:38:32.725 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:38:32.725 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:38:32.726 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:38:32.726 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:38:32.726 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:38:32.726 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:38:32.726 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:38:32.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:38:32.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:38:32.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:38:32.726 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:38:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:38:32.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:38:32.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:32.852 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:38:32.867 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:38:32.867 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:38:32.867 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:38:32.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:32.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:38:32.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:38:32.931 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:32.987 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:38:32.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:38:32.987 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:33.042 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:38:33.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:33.042 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:38:33.125 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:38:33.125 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:38:33.136 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:38:33.136 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:38:33.136 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:38:33.136 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:38:33.136 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:38:33.137 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:38:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:38:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:38:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:38:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:38:33.138 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:38:33.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:33.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:33.203 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:38:33.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:38:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:38:33.301 INFO:teuthology.orchestra.run.smithi040.stdout:client6786 2024-04-26T21:38:33.301 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:38:33.301 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:33.317 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 711 a/b/d; sync .') 2024-04-26T21:38:33.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:38:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:38:33.415 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:38:33.415 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:38:33.432 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:38:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.571+0000 7fbad5e93640 1 Processor -- start 2024-04-26T21:38:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.571+0000 7fbad5e93640 1 -- start start 2024-04-26T21:38:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbad5e93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbad00b2340 0x7fbad00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbad5e93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbad00cd930 0x7fbad00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbad5e93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad00d10d0 con 0x7fbad0162810 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbad5e93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbad00d0750 con 0x7fbad00b2340 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbad5e93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbad00d16a0 con 0x7fbad00cd930 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbacf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbad0162810 0x7fbad0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58814/0 (socket says 172.21.15.40:58814) 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbacf7fe640 1 -- 172.21.15.40:0/736546279 learned_addr learned my addr 172.21.15.40:0/736546279 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbaceffd640 1 --2- 172.21.15.40:0/736546279 >> v2:172.21.15.40:3300/0 conn(0x7fbad00b2340 0x7fbad00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.572+0000 7fbacffff640 1 --2- 172.21.15.40:0/736546279 >> v2:172.21.15.184:3300/0 conn(0x7fbad00cd930 0x7fbad00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbaceffd640 1 -- 172.21.15.40:0/736546279 >> v2:172.21.15.184:3300/0 conn(0x7fbad00cd930 msgr2=0x7fbad00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbaceffd640 1 --2- 172.21.15.40:0/736546279 >> v2:172.21.15.184:3300/0 conn(0x7fbad00cd930 0x7fbad00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbaceffd640 1 -- 172.21.15.40:0/736546279 >> v1:172.21.15.40:6789/0 conn(0x7fbad0162810 legacy=0x7fbad0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbaceffd640 1 -- 172.21.15.40:0/736546279 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad00d1370 con 0x7fbad00b2340 2024-04-26T21:38:33.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbace7fc640 1 -- 172.21.15.40:0/736546279 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694503003 0 0) 0x7fbad00d10d0 con 0x7fbad0162810 2024-04-26T21:38:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbaceffd640 1 --2- 172.21.15.40:0/736546279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00cd3f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbad00b2280 tx=0x7fbac402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da95ba7b4db88ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbace7fc640 1 -- 172.21.15.40:0/736546279 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4037070 con 0x7fbad00b2340 2024-04-26T21:38:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbace7fc640 1 -- 172.21.15.40:0/736546279 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac4032440 con 0x7fbad00b2340 2024-04-26T21:38:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.573+0000 7fbace7fc640 1 -- 172.21.15.40:0/736546279 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4032810 con 0x7fbad00b2340 2024-04-26T21:38:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.574+0000 7fbad5e93640 1 -- 172.21.15.40:0/736546279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 msgr2=0x7fbad00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.574+0000 7fbad5e93640 1 --2- 172.21.15.40:0/736546279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00cd3f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbad00b2280 tx=0x7fbac402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.574+0000 7fbad5e93640 1 -- 172.21.15.40:0/736546279 shutdown_connections 2024-04-26T21:38:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.574+0000 7fbad5e93640 1 --2- 172.21.15.40:0/736546279 >> v2:172.21.15.184:3300/0 conn(0x7fbad00cd930 0x7fbad00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.574+0000 7fbad5e93640 1 --2- 172.21.15.40:0/736546279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00cd3f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.574+0000 7fbad5e93640 1 -- 172.21.15.40:0/736546279 >> 172.21.15.40:0/736546279 conn(0x7fbad00d2da0 msgr2=0x7fbad00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.574+0000 7fbad5e93640 1 -- 172.21.15.40:0/736546279 shutdown_connections 2024-04-26T21:38:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.574+0000 7fbad5e93640 1 -- 172.21.15.40:0/736546279 wait complete. 2024-04-26T21:38:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.575+0000 7fbad5e93640 1 Processor -- start 2024-04-26T21:38:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.575+0000 7fbad5e93640 1 -- start start 2024-04-26T21:38:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbad5e93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbad5e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbad00cd930 0x7fbad00c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbad5e93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad0162810 0x7fbad00b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbad5e93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbad00d2070 con 0x7fbad00b2340 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbad5e93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbad00d0470 con 0x7fbad0162810 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbad5e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbad00d21f0 con 0x7fbad00cd930 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50354/0 (socket says 172.21.15.40:50354) 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacf7fe640 1 -- 172.21.15.40:0/4078704139 learned_addr learned my addr 172.21.15.40:0/4078704139 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbaceffd640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbad00cd930 0x7fbad00c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacffff640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad0162810 0x7fbad00b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacf7fe640 1 -- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbad00cd930 msgr2=0x7fbad00c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacf7fe640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbad00cd930 0x7fbad00c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacf7fe640 1 -- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad0162810 msgr2=0x7fbad00b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacf7fe640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad0162810 0x7fbad00b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacf7fe640 1 -- 172.21.15.40:0/4078704139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad015a2a0 con 0x7fbad00b2340 2024-04-26T21:38:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbacffff640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad0162810 0x7fbad00b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.576+0000 7fbaceffd640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbad00cd930 0x7fbad00c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.577+0000 7fbacf7fe640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00c6e70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbac000c460 tx=0x7fbac000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.577+0000 7fbab7fff640 1 -- 172.21.15.40:0/4078704139 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0018070 con 0x7fbad00b2340 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.577+0000 7fbab7fff640 1 -- 172.21.15.40:0/4078704139 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac0007500 con 0x7fbad00b2340 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.577+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad014cb40 con 0x7fbad00b2340 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.577+0000 7fbab7fff640 1 -- 172.21.15.40:0/4078704139 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0004a20 con 0x7fbad00b2340 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.577+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad00c8a20 con 0x7fbad00b2340 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.578+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad00720a0 con 0x7fbad00b2340 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.578+0000 7fbab7fff640 1 -- 172.21.15.40:0/4078704139 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbac001b070 con 0x7fbad00b2340 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.579+0000 7fbab7fff640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbabc06c8f0 0x7fbabc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.579+0000 7fbab7fff640 1 -- 172.21.15.40:0/4078704139 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fbac008af40 con 0x7fbad00b2340 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.579+0000 7fbaceffd640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbabc06c8f0 0x7fbabc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.580+0000 7fbaceffd640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbabc06c8f0 0x7fbabc06edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbac4034000 tx=0x7fbac402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:33.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.583+0000 7fbab7fff640 1 -- 172.21.15.40:0/4078704139 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbad00720a0 con 0x7fbad00b2340 2024-04-26T21:38:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.772+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbad00694f0 con 0x7fbad00b2340 2024-04-26T21:38:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.773+0000 7fbab7fff640 1 -- 172.21.15.40:0/4078704139 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16848 (secure 0 0 0) 0x7fbad015a2a0 con 0x7fbad00b2340 2024-04-26T21:38:33.774 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:33.774 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:38:19.929996+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:38:33.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbabc06c8f0 msgr2=0x7fbabc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbabc06c8f0 0x7fbabc06edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbac4034000 tx=0x7fbac402e000 comp rx=0 tx=0).stop 2024-04-26T21:38:33.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 msgr2=0x7fbad00c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00c6e70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbac000c460 tx=0x7fbac000c920 comp rx=0 tx=0).stop 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 shutdown_connections 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbabc06c8f0 0x7fbabc06edb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad0162810 0x7fbad00b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbad00cd930 0x7fbad00c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 --2- 172.21.15.40:0/4078704139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00b2340 0x7fbad00c6e70 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 >> 172.21.15.40:0/4078704139 conn(0x7fbad00d2da0 msgr2=0x7fbad00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 shutdown_connections 2024-04-26T21:38:33.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.775+0000 7fbad5e93640 1 -- 172.21.15.40:0/4078704139 wait complete. 2024-04-26T21:38:33.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:38:16.583866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:38:17.253849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:38:33.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:38:33.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.890+0000 7f5d1af58640 1 Processor -- start 2024-04-26T21:38:33.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.890+0000 7f5d1af58640 1 -- start start 2024-04-26T21:38:33.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d1af58640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5d1415d250 0x7f5d140d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d1af58640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5d140d1720 0x7f5d14070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d1af58640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d140701e0 con 0x7f5d140d39f0 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d1af58640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5d1415d630 con 0x7f5d1415d250 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d1af58640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5d140704e0 con 0x7f5d140d1720 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d18cf4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5d1415d250 0x7f5d140d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d18cf4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5d1415d250 0x7f5d140d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50370/0 (socket says 172.21.15.40:50370) 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d18cf4640 1 -- 172.21.15.40:0/3508246949 learned_addr learned my addr 172.21.15.40:0/3508246949 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.891+0000 7f5d13fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5d140d39f0 0x7f5d140d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58830/0 (socket says 172.21.15.40:58830) 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.892+0000 7f5d18cf4640 1 -- 172.21.15.40:0/3508246949 >> v2:172.21.15.184:3300/0 conn(0x7f5d140d1720 msgr2=0x7f5d14070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.892+0000 7f5d194f5640 1 --2- 172.21.15.40:0/3508246949 >> v2:172.21.15.184:3300/0 conn(0x7f5d140d1720 0x7f5d14070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.892+0000 7f5d18cf4640 1 --2- 172.21.15.40:0/3508246949 >> v2:172.21.15.184:3300/0 conn(0x7f5d140d1720 0x7f5d14070970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.892+0000 7f5d137fe640 1 -- 172.21.15.40:0/3508246949 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1185968385 0 0) 0x7f5d140701e0 con 0x7f5d140d39f0 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.892+0000 7f5d18cf4640 1 -- 172.21.15.40:0/3508246949 >> v1:172.21.15.40:6789/0 conn(0x7f5d140d39f0 legacy=0x7f5d140d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.892+0000 7f5d18cf4640 1 -- 172.21.15.40:0/3508246949 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d140ce370 con 0x7f5d1415d250 2024-04-26T21:38:33.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.892+0000 7f5d18cf4640 1 --2- 172.21.15.40:0/3508246949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d1415d250 0x7f5d140d0950 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5d140b4800 tx=0x7f5d0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6547e9a9174c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d137fe640 1 -- 172.21.15.40:0/3508246949 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c037070 con 0x7f5d1415d250 2024-04-26T21:38:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d137fe640 1 -- 172.21.15.40:0/3508246949 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d0c032440 con 0x7f5d1415d250 2024-04-26T21:38:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d137fe640 1 -- 172.21.15.40:0/3508246949 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c032810 con 0x7f5d1415d250 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d1af58640 1 -- 172.21.15.40:0/3508246949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d1415d250 msgr2=0x7f5d140d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d1af58640 1 --2- 172.21.15.40:0/3508246949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d1415d250 0x7f5d140d0950 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5d140b4800 tx=0x7f5d0c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d1af58640 1 -- 172.21.15.40:0/3508246949 shutdown_connections 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d1af58640 1 --2- 172.21.15.40:0/3508246949 >> v2:172.21.15.184:3300/0 conn(0x7f5d140d1720 0x7f5d14070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d1af58640 1 --2- 172.21.15.40:0/3508246949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d1415d250 0x7f5d140d0950 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d1af58640 1 -- 172.21.15.40:0/3508246949 >> 172.21.15.40:0/3508246949 conn(0x7f5d140d2d80 msgr2=0x7f5d140d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d1af58640 1 -- 172.21.15.40:0/3508246949 shutdown_connections 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.893+0000 7f5d1af58640 1 -- 172.21.15.40:0/3508246949 wait complete. 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.894+0000 7f5d1af58640 1 Processor -- start 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.894+0000 7f5d1af58640 1 -- start start 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d1af58640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d140d1720 0x7f5d140cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d1af58640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d140d39f0 0x7f5d1406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d1af58640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d1415d250 0x7f5d140ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d1af58640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5d140cebb0 con 0x7f5d140d39f0 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d1af58640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d140ce8d0 con 0x7f5d140d1720 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d1af58640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d14070eb0 con 0x7f5d1415d250 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d13fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d140d39f0 0x7f5d1406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d13fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d140d39f0 0x7f5d1406d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50382/0 (socket says 172.21.15.40:50382) 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d13fff640 1 -- 172.21.15.40:0/2039357960 learned_addr learned my addr 172.21.15.40:0/2039357960 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.895+0000 7f5d18cf4640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d140d1720 0x7f5d140cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d194f5640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d1415d250 0x7f5d140ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d13fff640 1 -- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d1415d250 msgr2=0x7f5d140ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d13fff640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d1415d250 0x7f5d140ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d13fff640 1 -- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d140d1720 msgr2=0x7f5d140cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d13fff640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d140d1720 0x7f5d140cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d13fff640 1 -- 172.21.15.40:0/2039357960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1406da50 con 0x7f5d140d39f0 2024-04-26T21:38:33.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d18cf4640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d140d1720 0x7f5d140cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:33.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d13fff640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d140d39f0 0x7f5d1406d350 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c890 tx=0x7f5d0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:33.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d117fa640 1 -- 172.21.15.40:0/2039357960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04018070 con 0x7f5d140d39f0 2024-04-26T21:38:33.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d117fa640 1 -- 172.21.15.40:0/2039357960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d0400a640 con 0x7f5d140d39f0 2024-04-26T21:38:33.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1406dd30 con 0x7f5d140d39f0 2024-04-26T21:38:33.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d117fa640 1 -- 172.21.15.40:0/2039357960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400aa30 con 0x7f5d140d39f0 2024-04-26T21:38:33.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.896+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1406e5d0 con 0x7f5d140d39f0 2024-04-26T21:38:33.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.897+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d1406df00 con 0x7f5d140d39f0 2024-04-26T21:38:33.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.897+0000 7f5d117fa640 1 -- 172.21.15.40:0/2039357960 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d0401b070 con 0x7f5d140d39f0 2024-04-26T21:38:33.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.898+0000 7f5d117fa640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5cf406c8f0 0x7f5cf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:33.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.898+0000 7f5d117fa640 1 -- 172.21.15.40:0/2039357960 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f5d0408b490 con 0x7f5d140d39f0 2024-04-26T21:38:33.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.898+0000 7f5d18cf4640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5cf406c8f0 0x7f5cf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:33.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.899+0000 7f5d18cf4640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5cf406c8f0 0x7f5cf406edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c034000 tx=0x7f5d0c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:33.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:33.902+0000 7f5d117fa640 1 -- 172.21.15.40:0/2039357960 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d04059690 con 0x7f5d140d39f0 2024-04-26T21:38:34.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.090+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d140694f0 con 0x7f5d140d39f0 2024-04-26T21:38:34.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.091+0000 7f5d117fa640 1 -- 172.21.15.40:0/2039357960 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f5d04058f90 con 0x7f5d140d39f0 2024-04-26T21:38:34.092 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5cf406c8f0 msgr2=0x7f5cf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5cf406c8f0 0x7f5cf406edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c034000 tx=0x7f5d0c02e000 comp rx=0 tx=0).stop 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d140d39f0 msgr2=0x7f5d1406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d140d39f0 0x7f5d1406d350 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c890 tx=0x7f5d0400cd50 comp rx=0 tx=0).stop 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 shutdown_connections 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5cf406c8f0 0x7f5cf406edb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d1415d250 0x7f5d140ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d140d39f0 0x7f5d1406d350 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 --2- 172.21.15.40:0/2039357960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d140d1720 0x7f5d140cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 >> 172.21.15.40:0/2039357960 conn(0x7f5d140d2d80 msgr2=0x7f5d14071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:34.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 shutdown_connections 2024-04-26T21:38:34.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.093+0000 7f5d1af58640 1 -- 172.21.15.40:0/2039357960 wait complete. 2024-04-26T21:38:34.105 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:38:34.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.208+0000 7fb8835e4640 1 Processor -- start 2024-04-26T21:38:34.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb8835e4640 1 -- start start 2024-04-26T21:38:34.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb8835e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb8835e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb87c06c950 0x7fb87c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb8835e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb87c0cfd80 con 0x7fb87c0b0380 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb8835e4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb87c0af8e0 con 0x7fb87c06c950 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb8835e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c0d0020 con 0x7fb87c0b10b0 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb881b81640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb87c06c950 0x7fb87c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb881380640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb87c0b0380 0x7fb87c06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58836/0 (socket says 172.21.15.40:58836) 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.209+0000 7fb881380640 1 -- 172.21.15.40:0/2822707174 learned_addr learned my addr 172.21.15.40:0/2822707174 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.210+0000 7fb881b81640 1 -- 172.21.15.40:0/2822707174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 msgr2=0x7fb87c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.210+0000 7fb880b7f640 1 --2- 172.21.15.40:0/2822707174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.210+0000 7fb87bfff640 1 -- 172.21.15.40:0/2822707174 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2012371530 0 0) 0x7fb87c0cfd80 con 0x7fb87c0b0380 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.210+0000 7fb881b81640 1 --2- 172.21.15.40:0/2822707174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.210+0000 7fb881b81640 1 -- 172.21.15.40:0/2822707174 >> v1:172.21.15.40:6789/0 conn(0x7fb87c0b0380 legacy=0x7fb87c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.210+0000 7fb881b81640 1 -- 172.21.15.40:0/2822707174 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c0d0300 con 0x7fb87c06c950 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.210+0000 7fb880b7f640 1 --2- 172.21.15.40:0/2822707174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.210+0000 7fb881b81640 1 --2- 172.21.15.40:0/2822707174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c06c950 0x7fb87c15e970 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0afa60 tx=0x7fb87402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cd1ad79809c59f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb87bfff640 1 -- 172.21.15.40:0/2822707174 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874037070 con 0x7fb87c06c950 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb87bfff640 1 -- 172.21.15.40:0/2822707174 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb87402ae50 con 0x7fb87c06c950 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb87bfff640 1 -- 172.21.15.40:0/2822707174 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874032660 con 0x7fb87c06c950 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb8835e4640 1 -- 172.21.15.40:0/2822707174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c06c950 msgr2=0x7fb87c15e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb8835e4640 1 --2- 172.21.15.40:0/2822707174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c06c950 0x7fb87c15e970 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0afa60 tx=0x7fb87402a100 comp rx=0 tx=0).stop 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb8835e4640 1 -- 172.21.15.40:0/2822707174 shutdown_connections 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb8835e4640 1 --2- 172.21.15.40:0/2822707174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c06c950 0x7fb87c15e970 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb8835e4640 1 --2- 172.21.15.40:0/2822707174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb8835e4640 1 -- 172.21.15.40:0/2822707174 >> 172.21.15.40:0/2822707174 conn(0x7fb87c0d1cf0 msgr2=0x7fb87c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb8835e4640 1 -- 172.21.15.40:0/2822707174 shutdown_connections 2024-04-26T21:38:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.211+0000 7fb8835e4640 1 -- 172.21.15.40:0/2822707174 wait complete. 2024-04-26T21:38:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.212+0000 7fb8835e4640 1 Processor -- start 2024-04-26T21:38:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb8835e4640 1 -- start start 2024-04-26T21:38:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb8835e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c06c950 0x7fb87c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb8835e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0b0380 0x7fb87c0b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb8835e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c1f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb8835e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c0b0760 con 0x7fb87c0b0380 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb8835e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c0d1980 con 0x7fb87c06c950 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb8835e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c0d15c0 con 0x7fb87c0b10b0 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb880b7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0b0380 0x7fb87c0b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb880b7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0b0380 0x7fb87c0b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50396/0 (socket says 172.21.15.40:50396) 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.213+0000 7fb880b7f640 1 -- 172.21.15.40:0/726755477 learned_addr learned my addr 172.21.15.40:0/726755477 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb881b81640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c1f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb881380640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c06c950 0x7fb87c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb880b7f640 1 -- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 msgr2=0x7fb87c1f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb880b7f640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c1f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb880b7f640 1 -- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c06c950 msgr2=0x7fb87c155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb880b7f640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c06c950 0x7fb87c155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb880b7f640 1 -- 172.21.15.40:0/726755477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c1f8a40 con 0x7fb87c0b0380 2024-04-26T21:38:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb880b7f640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0b0380 0x7fb87c0b7450 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb87000c7f0 tx=0x7fb87000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb879ffb640 1 -- 172.21.15.40:0/726755477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870015070 con 0x7fb87c0b0380 2024-04-26T21:38:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb879ffb640 1 -- 172.21.15.40:0/726755477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb870007820 con 0x7fb87c0b0380 2024-04-26T21:38:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c1f8d20 con 0x7fb87c0b0380 2024-04-26T21:38:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb879ffb640 1 -- 172.21.15.40:0/726755477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870007bd0 con 0x7fb87c0b0380 2024-04-26T21:38:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.214+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1582c0 con 0x7fb87c0b0380 2024-04-26T21:38:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.215+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87c0694f0 con 0x7fb87c0b0380 2024-04-26T21:38:34.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.216+0000 7fb879ffb640 1 -- 172.21.15.40:0/726755477 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb870018070 con 0x7fb87c0b0380 2024-04-26T21:38:34.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.216+0000 7fb879ffb640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb86806c920 0x7fb86806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.216+0000 7fb879ffb640 1 -- 172.21.15.40:0/726755477 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fb87008bd50 con 0x7fb87c0b0380 2024-04-26T21:38:34.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.217+0000 7fb881380640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb86806c920 0x7fb86806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.217+0000 7fb881380640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb86806c920 0x7fb86806ede0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0012e0 tx=0x7fb86c001700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.221+0000 7fb879ffb640 1 -- 172.21.15.40:0/726755477 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb870059990 con 0x7fb87c0b0380 2024-04-26T21:38:34.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.407+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb87c0c4d40 con 0x7fb87c0b0380 2024-04-26T21:38:34.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.407+0000 7fb879ffb640 1 -- 172.21.15.40:0/726755477 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7fb87005ee20 con 0x7fb87c0b0380 2024-04-26T21:38:34.408 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb86806c920 msgr2=0x7fb86806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb86806c920 0x7fb86806ede0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0012e0 tx=0x7fb86c001700 comp rx=0 tx=0).stop 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0b0380 msgr2=0x7fb87c0b7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0b0380 0x7fb87c0b7450 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb87000c7f0 tx=0x7fb87000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 shutdown_connections 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb86806c920 0x7fb86806ede0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb87c0b10b0 0x7fb87c1f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0b0380 0x7fb87c0b7450 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 --2- 172.21.15.40:0/726755477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb87c06c950 0x7fb87c155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.409+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 >> 172.21.15.40:0/726755477 conn(0x7fb87c0d1cf0 msgr2=0x7fb87c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.410+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 shutdown_connections 2024-04-26T21:38:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.410+0000 7fb8835e4640 1 -- 172.21.15.40:0/726755477 wait complete. 2024-04-26T21:38:34.421 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:38:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.524+0000 7f14aabc1640 1 Processor -- start 2024-04-26T21:38:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.524+0000 7f14aabc1640 1 -- start start 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14aabc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 0x7f14a415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14aabc1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a40d2390 con 0x7f14a40cfde0 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14aabc1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a406d3f0 con 0x7f14a4070350 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14aabc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14a40afab0 con 0x7f14a4071260 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a895d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f14a40cfde0 0x7f14a40d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58840/0 (socket says 172.21.15.40:58840) 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a895d640 1 -- 172.21.15.40:0/888246099 learned_addr learned my addr 172.21.15.40:0/888246099 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a915e640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 0x7f14a415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3622991301 0 0) 0x7f14a40d2390 con 0x7f14a40cfde0 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f148c003520 con 0x7f14a40cfde0 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 831803363 0 0) 0x7f148c003520 con 0x7f14a40cfde0 2024-04-26T21:38:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 msgr2=0x7f14a415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 0x7f14a415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 >> v1:172.21.15.184:6789/0 conn(0x7f14a4070350 legacy=0x7f14a4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a406d6d0 con 0x7f14a40cfde0 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1494001670 con 0x7f14a40cfde0 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a915e640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 0x7f14a415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 >> v1:172.21.15.40:6789/0 conn(0x7f14a40cfde0 legacy=0x7f14a40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0039e0 0x7f148c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.525+0000 7f14a37fe640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f148c004390 0x7f148c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a37fe640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 0x7f148c005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00ba10 con 0x7f148c0039e0 2024-04-26T21:38:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00bbc0 con 0x7f148c004390 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f148c00bdc0 con 0x7f14a4071260 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a895d640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0039e0 0x7f148c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a3fff640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f148c004390 0x7f148c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a915e640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 0x7f148c005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a895d640 1 -- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 msgr2=0x7f148c005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a895d640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a4071260 0x7f148c005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a895d640 1 -- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f148c004390 msgr2=0x7f148c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a895d640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f148c004390 0x7f148c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.526+0000 7f14a895d640 1 -- 172.21.15.40:0/888246099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c003520 con 0x7f148c0039e0 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.527+0000 7f14a895d640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0039e0 0x7f148c003dc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f149400c190 tx=0x7f148c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933f4cbbf49306a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.527+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494007cc0 con 0x7f148c0039e0 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.527+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1494005ad0 con 0x7f148c0039e0 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.527+0000 7f14a37fe640 1 -- 172.21.15.40:0/888246099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494007070 con 0x7f148c0039e0 2024-04-26T21:38:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.527+0000 7f14aabc1640 1 -- 172.21.15.40:0/888246099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0039e0 msgr2=0x7f148c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.527+0000 7f14aabc1640 1 --2- 172.21.15.40:0/888246099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0039e0 0x7f148c003dc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f149400c190 tx=0x7f148c005830 comp rx=0 tx=0).stop 2024-04-26T21:38:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.527+0000 7f14a915e640 1 -- 172.21.15.40:0/888246099 reap_dead start 2024-04-26T21:38:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.528+0000 7f14aabc1640 1 -- 172.21.15.40:0/888246099 shutdown_connections 2024-04-26T21:38:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.528+0000 7f14aabc1640 1 -- 172.21.15.40:0/888246099 >> 172.21.15.40:0/888246099 conn(0x7f14a40d2d80 msgr2=0x7f14a40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.528+0000 7f14aabc1640 1 -- 172.21.15.40:0/888246099 shutdown_connections 2024-04-26T21:38:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.528+0000 7f14aabc1640 1 -- 172.21.15.40:0/888246099 wait complete. 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.529+0000 7f14aabc1640 1 Processor -- start 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.529+0000 7f14aabc1640 1 -- start start 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14aabc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f148c0039e0 0x7f14a40c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14aabc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a415da40 0x7f14a415f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14aabc1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a4157d00 0x7f14a41608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14aabc1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14a406f1c0 con 0x7f14a415da40 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14aabc1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14a406dd20 con 0x7f14a4157d00 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14aabc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14a406f340 con 0x7f148c0039e0 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a415da40 0x7f14a415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a415da40 0x7f14a415f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50414/0 (socket says 172.21.15.40:50414) 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 -- 172.21.15.40:0/670370824 learned_addr learned my addr 172.21.15.40:0/670370824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a895d640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f148c0039e0 0x7f14a40c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a915e640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a4157d00 0x7f14a41608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 -- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f148c0039e0 msgr2=0x7f14a40c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f148c0039e0 0x7f14a40c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 -- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a4157d00 msgr2=0x7f14a41608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a4157d00 0x7f14a41608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 -- 172.21.15.40:0/670370824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a415dfb0 con 0x7f14a415da40 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.530+0000 7f14a3fff640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a415da40 0x7f14a415f050 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f149800f870 tx=0x7f149800fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.531+0000 7f14a17fa640 1 -- 172.21.15.40:0/670370824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149801a070 con 0x7f14a415da40 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.531+0000 7f14a17fa640 1 -- 172.21.15.40:0/670370824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1498005320 con 0x7f14a415da40 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.531+0000 7f14a17fa640 1 -- 172.21.15.40:0/670370824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498007270 con 0x7f14a415da40 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.531+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a415e290 con 0x7f14a415da40 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.531+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a415e850 con 0x7f14a415da40 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.532+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a40694f0 con 0x7f14a415da40 2024-04-26T21:38:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.532+0000 7f14a17fa640 1 -- 172.21.15.40:0/670370824 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f149801d070 con 0x7f14a415da40 2024-04-26T21:38:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.533+0000 7f14a17fa640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f149006c920 0x7f149006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.533+0000 7f14a17fa640 1 -- 172.21.15.40:0/670370824 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f149808b470 con 0x7f14a415da40 2024-04-26T21:38:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.533+0000 7f14a895d640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f149006c920 0x7f149006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.533+0000 7f14a895d640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f149006c920 0x7f149006ede0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1494032000 tx=0x7f149402d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.537+0000 7f14a17fa640 1 -- 172.21.15.40:0/670370824 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14980597a0 con 0x7f14a415da40 2024-04-26T21:38:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.720+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14a40d12f0 con 0x7f14a415da40 2024-04-26T21:38:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.720+0000 7f14a17fa640 1 -- 172.21.15.40:0/670370824 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7f14980590a0 con 0x7f14a415da40 2024-04-26T21:38:34.722 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:38:34.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f149006c920 msgr2=0x7f149006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f149006c920 0x7f149006ede0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1494032000 tx=0x7f149402d000 comp rx=0 tx=0).stop 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a415da40 msgr2=0x7f14a415f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a415da40 0x7f14a415f050 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f149800f870 tx=0x7f149800fd30 comp rx=0 tx=0).stop 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 shutdown_connections 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f149006c920 0x7f149006ede0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a4157d00 0x7f14a41608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a415da40 0x7f14a415f050 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 --2- 172.21.15.40:0/670370824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f148c0039e0 0x7f14a40c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 >> 172.21.15.40:0/670370824 conn(0x7f14a40d2d80 msgr2=0x7f14a40b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 shutdown_connections 2024-04-26T21:38:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.722+0000 7f14aabc1640 1 -- 172.21.15.40:0/670370824 wait complete. 2024-04-26T21:38:34.734 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:38:34.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.835+0000 7f36b1739640 1 Processor -- start 2024-04-26T21:38:34.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.835+0000 7f36b1739640 1 -- start start 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36b1739640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 0x7f36ac0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36b1739640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36ac0cd930 0x7f36ac0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36b1739640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36ac0d10d0 con 0x7f36ac162810 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36b1739640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f36ac0d0750 con 0x7f36ac0cd930 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36b1739640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac0d16a0 con 0x7f36ac0b2340 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36aaffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f36ac162810 0x7f36ac070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58854/0 (socket says 172.21.15.40:58854) 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36aaffd640 1 -- 172.21.15.40:0/2077959844 learned_addr learned my addr 172.21.15.40:0/2077959844 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36ab7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36ac0cd930 0x7f36ac0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36ab7fe640 1 -- 172.21.15.40:0/2077959844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 msgr2=0x7f36ac0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:34.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36ab7fe640 1 --2- 172.21.15.40:0/2077959844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 0x7f36ac0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36ab7fe640 1 -- 172.21.15.40:0/2077959844 >> v1:172.21.15.40:6789/0 conn(0x7f36ac162810 legacy=0x7f36ac070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36ab7fe640 1 -- 172.21.15.40:0/2077959844 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac0d1370 con 0x7f36ac0cd930 2024-04-26T21:38:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.836+0000 7f36a9ffb640 1 -- 172.21.15.40:0/2077959844 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3623007033 0 0) 0x7f36ac0d10d0 con 0x7f36ac162810 2024-04-26T21:38:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.837+0000 7f36ab7fe640 1 --2- 172.21.15.40:0/2077959844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0cd930 0x7f36ac0d1b30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009d90 tx=0x7f36a402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81758efe945f385 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.837+0000 7f36a9ffb640 1 -- 172.21.15.40:0/2077959844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4007740 con 0x7f36ac0cd930 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.837+0000 7f36a9ffb640 1 -- 172.21.15.40:0/2077959844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a40078e0 con 0x7f36ac0cd930 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.837+0000 7f36a9ffb640 1 -- 172.21.15.40:0/2077959844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4007cb0 con 0x7f36ac0cd930 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.838+0000 7f36b1739640 1 -- 172.21.15.40:0/2077959844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0cd930 msgr2=0x7f36ac0d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.838+0000 7f36b1739640 1 --2- 172.21.15.40:0/2077959844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0cd930 0x7f36ac0d1b30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009d90 tx=0x7f36a402a660 comp rx=0 tx=0).stop 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.838+0000 7f36b1739640 1 -- 172.21.15.40:0/2077959844 shutdown_connections 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.838+0000 7f36b1739640 1 --2- 172.21.15.40:0/2077959844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0cd930 0x7f36ac0d1b30 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.838+0000 7f36b1739640 1 --2- 172.21.15.40:0/2077959844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 0x7f36ac0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.838+0000 7f36b1739640 1 -- 172.21.15.40:0/2077959844 >> 172.21.15.40:0/2077959844 conn(0x7f36ac0d2da0 msgr2=0x7f36ac0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.838+0000 7f36b1739640 1 -- 172.21.15.40:0/2077959844 shutdown_connections 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.838+0000 7f36b1739640 1 -- 172.21.15.40:0/2077959844 wait complete. 2024-04-26T21:38:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.839+0000 7f36b1739640 1 Processor -- start 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.839+0000 7f36b1739640 1 -- start start 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36b1739640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 0x7f36ac0a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36b1739640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac0cd930 0x7f36ac0c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36b1739640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac162810 0x7f36ac0c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36b1739640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac0d2070 con 0x7f36ac162810 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36b1739640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac0d0470 con 0x7f36ac0cd930 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36b1739640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac0d21f0 con 0x7f36ac0b2340 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac162810 0x7f36ac0c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac162810 0x7f36ac0c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50432/0 (socket says 172.21.15.40:50432) 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 -- 172.21.15.40:0/1981591264 learned_addr learned my addr 172.21.15.40:0/1981591264 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36aa7fc640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac0cd930 0x7f36ac0c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36aaffd640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 0x7f36ac0a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 -- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 msgr2=0x7f36ac0a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 0x7f36ac0a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 -- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac0cd930 msgr2=0x7f36ac0c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac0cd930 0x7f36ac0c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 -- 172.21.15.40:0/1981591264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac15fc60 con 0x7f36ac162810 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36aa7fc640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac0cd930 0x7f36ac0c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36aaffd640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 0x7f36ac0a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.840+0000 7f36ab7fe640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac162810 0x7f36ac0c68b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009eb0 tx=0x7f36a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.841+0000 7f36937fe640 1 -- 172.21.15.40:0/1981591264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4040070 con 0x7f36ac162810 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.841+0000 7f36937fe640 1 -- 172.21.15.40:0/1981591264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a4031c90 con 0x7f36ac162810 2024-04-26T21:38:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.841+0000 7f36937fe640 1 -- 172.21.15.40:0/1981591264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4045680 con 0x7f36ac162810 2024-04-26T21:38:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.841+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac0c9540 con 0x7f36ac162810 2024-04-26T21:38:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.841+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac0c9b00 con 0x7f36ac162810 2024-04-26T21:38:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.842+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ac0764d0 con 0x7f36ac162810 2024-04-26T21:38:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.842+0000 7f36937fe640 1 -- 172.21.15.40:0/1981591264 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36a4039040 con 0x7f36ac162810 2024-04-26T21:38:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.843+0000 7f36937fe640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369806c8f0 0x7f369806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.843+0000 7f36937fe640 1 -- 172.21.15.40:0/1981591264 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f36a40ad660 con 0x7f36ac162810 2024-04-26T21:38:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.843+0000 7f36aaffd640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369806c8f0 0x7f369806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.843+0000 7f36aaffd640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369806c8f0 0x7f369806edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f369c000f60 tx=0x7f369c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:34.847+0000 7f36937fe640 1 -- 172.21.15.40:0/1981591264 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36ac0764d0 con 0x7f36ac162810 2024-04-26T21:38:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.047+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f36ac0694f0 con 0x7f36ac162810 2024-04-26T21:38:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.048+0000 7f36937fe640 1 -- 172.21.15.40:0/1981591264 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) ==== 80+0+0 (secure 0 0 0) 0x7f36a40abe20 con 0x7f36ac162810 2024-04-26T21:38:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:38:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369806c8f0 msgr2=0x7f369806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369806c8f0 0x7f369806edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f369c000f60 tx=0x7f369c009040 comp rx=0 tx=0).stop 2024-04-26T21:38:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac162810 msgr2=0x7f36ac0c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac162810 0x7f36ac0c68b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009eb0 tx=0x7f36a4004a40 comp rx=0 tx=0).stop 2024-04-26T21:38:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 shutdown_connections 2024-04-26T21:38:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369806c8f0 0x7f369806edb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac162810 0x7f36ac0c68b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac0cd930 0x7f36ac0c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 --2- 172.21.15.40:0/1981591264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0b2340 0x7f36ac0a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 >> 172.21.15.40:0/1981591264 conn(0x7f36ac0d2da0 msgr2=0x7f36ac070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 shutdown_connections 2024-04-26T21:38:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.050+0000 7f36b1739640 1 -- 172.21.15.40:0/1981591264 wait complete. 2024-04-26T21:38:35.061 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:38:35.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:35.062 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:38:35.089 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:38:35.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:38:35.089 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:38:35.146 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:38:35.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.286+0000 7fe782d4e640 1 Processor -- start 2024-04-26T21:38:35.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.287+0000 7fe782d4e640 1 -- start start 2024-04-26T21:38:35.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.287+0000 7fe782d4e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe77c15f3a0 0x7fe77c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.287+0000 7fe782d4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0d1770 0x7fe77c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.287+0000 7fe782d4e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe77c0d2130 0x7fe77c0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.287+0000 7fe782d4e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe77c0d25a0 con 0x7fe77c0d2130 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.287+0000 7fe782d4e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe77c0d0e20 con 0x7fe77c15f3a0 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.287+0000 7fe782d4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe77c0b1d70 con 0x7fe77c0d1770 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.287+0000 7fe7812eb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe77c0d2130 0x7fe77c0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe7812eb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe77c0d2130 0x7fe77c0d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50436/0 (socket says 172.21.15.40:50436) 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe7812eb640 1 -- 172.21.15.40:0/4030139753 learned_addr learned my addr 172.21.15.40:0/4030139753 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe780aea640 1 --2- 172.21.15.40:0/4030139753 >> v2:172.21.15.184:3300/0 conn(0x7fe77c15f3a0 0x7fe77c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe77bfff640 1 --2- 172.21.15.40:0/4030139753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0d1770 0x7fe77c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe7812eb640 1 -- 172.21.15.40:0/4030139753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0d1770 msgr2=0x7fe77c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe7812eb640 1 --2- 172.21.15.40:0/4030139753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0d1770 0x7fe77c0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe7812eb640 1 -- 172.21.15.40:0/4030139753 >> v2:172.21.15.184:3300/0 conn(0x7fe77c15f3a0 msgr2=0x7fe77c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe7812eb640 1 --2- 172.21.15.40:0/4030139753 >> v2:172.21.15.184:3300/0 conn(0x7fe77c15f3a0 0x7fe77c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe7812eb640 1 -- 172.21.15.40:0/4030139753 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c0b2050 con 0x7fe77c0d2130 2024-04-26T21:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.288+0000 7fe77bfff640 1 --2- 172.21.15.40:0/4030139753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0d1770 0x7fe77c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe7812eb640 1 --2- 172.21.15.40:0/4030139753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d2130 0x7fe77c0d2e50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe77c073b50 tx=0x7fe77402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8816ee22127e92f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe77b7fe640 1 -- 172.21.15.40:0/4030139753 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774036070 con 0x7fe77c0d2130 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe77b7fe640 1 -- 172.21.15.40:0/4030139753 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe774031600 con 0x7fe77c0d2130 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe77b7fe640 1 -- 172.21.15.40:0/4030139753 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7740319d0 con 0x7fe77c0d2130 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 -- 172.21.15.40:0/4030139753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d2130 msgr2=0x7fe77c0d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 --2- 172.21.15.40:0/4030139753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d2130 0x7fe77c0d2e50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe77c073b50 tx=0x7fe77402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 -- 172.21.15.40:0/4030139753 shutdown_connections 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 --2- 172.21.15.40:0/4030139753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d2130 0x7fe77c0d2e50 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 --2- 172.21.15.40:0/4030139753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c0d1770 0x7fe77c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 --2- 172.21.15.40:0/4030139753 >> v2:172.21.15.184:3300/0 conn(0x7fe77c15f3a0 0x7fe77c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 -- 172.21.15.40:0/4030139753 >> 172.21.15.40:0/4030139753 conn(0x7fe77c0d4f90 msgr2=0x7fe77c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 -- 172.21.15.40:0/4030139753 shutdown_connections 2024-04-26T21:38:35.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.289+0000 7fe782d4e640 1 -- 172.21.15.40:0/4030139753 wait complete. 2024-04-26T21:38:35.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.290+0000 7fe782d4e640 1 Processor -- start 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe782d4e640 1 -- start start 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe782d4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d1770 0x7fe77c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe782d4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe77c0d2130 0x7fe77c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe782d4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f3a0 0x7fe77c1f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe782d4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c15f900 con 0x7fe77c0d1770 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe782d4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c15f780 con 0x7fe77c0d2130 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe782d4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe77c0d3390 con 0x7fe77c15f3a0 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe780aea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d1770 0x7fe77c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe780aea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d1770 0x7fe77c0cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50446/0 (socket says 172.21.15.40:50446) 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.291+0000 7fe780aea640 1 -- 172.21.15.40:0/91710154 learned_addr learned my addr 172.21.15.40:0/91710154 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe7812eb640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f3a0 0x7fe77c1f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe77bfff640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe77c0d2130 0x7fe77c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe780aea640 1 -- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f3a0 msgr2=0x7fe77c1f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe780aea640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f3a0 0x7fe77c1f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe780aea640 1 -- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe77c0d2130 msgr2=0x7fe77c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe780aea640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe77c0d2130 0x7fe77c0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe780aea640 1 -- 172.21.15.40:0/91710154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c0cd5b0 con 0x7fe77c0d1770 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe780aea640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d1770 0x7fe77c0cca70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe77000d5f0 tx=0x7fe77000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe7797fa640 1 -- 172.21.15.40:0/91710154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7700125f0 con 0x7fe77c0d1770 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe7797fa640 1 -- 172.21.15.40:0/91710154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe770019070 con 0x7fe77c0d1770 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77c0c8bf0 con 0x7fe77c0d1770 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe7797fa640 1 -- 172.21.15.40:0/91710154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7700129d0 con 0x7fe77c0d1770 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.292+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77c0c91b0 con 0x7fe77c0d1770 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.293+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77c0694f0 con 0x7fe77c0d1770 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.294+0000 7fe7797fa640 1 -- 172.21.15.40:0/91710154 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe770012b70 con 0x7fe77c0d1770 2024-04-26T21:38:35.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.294+0000 7fe7797fa640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe76006c8f0 0x7fe76006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.294+0000 7fe7797fa640 1 -- 172.21.15.40:0/91710154 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7fe77008c050 con 0x7fe77c0d1770 2024-04-26T21:38:35.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.295+0000 7fe77bfff640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe76006c8f0 0x7fe76006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.295+0000 7fe77bfff640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe76006c8f0 0x7fe76006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe77c154c80 tx=0x7fe768008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:35.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.299+0000 7fe7797fa640 1 -- 172.21.15.40:0/91710154 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe77005a380 con 0x7fe77c0d1770 2024-04-26T21:38:35.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.523+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe77c0b6860 con 0x7fe77c0d1770 2024-04-26T21:38:35.525 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:38:35.524+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:38:35.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.525+0000 7fe7797fa640 1 -- 172.21.15.40:0/91710154 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7fe770059c80 con 0x7fe77c0d1770 2024-04-26T21:38:35.529 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:38:35.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe76006c8f0 msgr2=0x7fe76006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe76006c8f0 0x7fe76006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe77c154c80 tx=0x7fe768008040 comp rx=0 tx=0).stop 2024-04-26T21:38:35.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d1770 msgr2=0x7fe77c0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d1770 0x7fe77c0cca70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe77000d5f0 tx=0x7fe77000dab0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 shutdown_connections 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe76006c8f0 0x7fe76006edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe77c15f3a0 0x7fe77c1f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe77c0d2130 0x7fe77c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 --2- 172.21.15.40:0/91710154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0d1770 0x7fe77c0cca70 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 >> 172.21.15.40:0/91710154 conn(0x7fe77c0d4f90 msgr2=0x7fe77c06e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 shutdown_connections 2024-04-26T21:38:35.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.528+0000 7fe782d4e640 1 -- 172.21.15.40:0/91710154 wait complete. 2024-04-26T21:38:35.532 INFO:tasks.ceph.mds.a.smithi040.stderr: -5563> 2024-04-26T21:38:20.918+0000 7ffae98d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:38:35.541 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_journal_corrupt_dentry_first 1.0 2024-04-26T21:38:35.563 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:38:35.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.646+0000 7fca4656a640 1 Processor -- start 2024-04-26T21:38:35.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.646+0000 7fca4656a640 1 -- start start 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca4656a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca400cf0e0 0x7fca4006f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca4656a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4015b9e0 con 0x7fca400cf5e0 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca4656a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca400d0740 con 0x7fca4015b600 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca4656a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fca400cb9b0 con 0x7fca400cf0e0 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fca400cf5e0 0x7fca400ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58858/0 (socket says 172.21.15.40:58858) 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3f7fe640 1 -- 172.21.15.40:0/2803398412 learned_addr learned my addr 172.21.15.40:0/2803398412 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca44b07640 1 --2- 172.21.15.40:0/2803398412 >> v2:172.21.15.184:3300/0 conn(0x7fca400cf0e0 0x7fca4006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2117610023 0 0) 0x7fca4015b9e0 con 0x7fca400cf5e0 2024-04-26T21:38:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca2c003520 con 0x7fca400cf5e0 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 723995780 0 0) 0x7fca2c003520 con 0x7fca400cf5e0 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 >> v2:172.21.15.184:3300/0 conn(0x7fca400cf0e0 msgr2=0x7fca4006f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 --2- 172.21.15.40:0/2803398412 >> v2:172.21.15.184:3300/0 conn(0x7fca400cf0e0 0x7fca4006f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 >> v1:172.21.15.184:6789/0 conn(0x7fca4015b600 legacy=0x7fca40071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca400cb360 con 0x7fca400cf5e0 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fca30001690 con 0x7fca400cf5e0 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 >> v1:172.21.15.40:6789/0 conn(0x7fca400cf5e0 legacy=0x7fca400ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca2c004800 0x7fca2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca2c0051c0 0x7fca2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca400cb9b0 con 0x7fca2c0039b0 2024-04-26T21:38:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c00c060 con 0x7fca2c004800 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.647+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca2c00c260 con 0x7fca2c0051c0 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca3ffff640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca44b07640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca2c0051c0 0x7fca2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca3f7fe640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca2c004800 0x7fca2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca3ffff640 1 -- 172.21.15.40:0/2803398412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca2c0051c0 msgr2=0x7fca2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca3ffff640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca2c0051c0 0x7fca2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca3ffff640 1 -- 172.21.15.40:0/2803398412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca2c004800 msgr2=0x7fca2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca3ffff640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca2c004800 0x7fca2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca3ffff640 1 -- 172.21.15.40:0/2803398412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c003520 con 0x7fca2c0039b0 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.648+0000 7fca44b07640 1 -- 172.21.15.40:0/2803398412 reap_dead start 2024-04-26T21:38:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca3ffff640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca2c003dd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fca28001750 tx=0x7fca2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d504981aed335b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28001370 con 0x7fca2c0039b0 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca280052d0 con 0x7fca2c0039b0 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca3effd640 1 -- 172.21.15.40:0/2803398412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2802abe0 con 0x7fca2c0039b0 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca4656a640 1 -- 172.21.15.40:0/2803398412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 msgr2=0x7fca2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca4656a640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca2c003dd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fca28001750 tx=0x7fca2c005830 comp rx=0 tx=0).stop 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca4656a640 1 -- 172.21.15.40:0/2803398412 shutdown_connections 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca4656a640 1 --2- 172.21.15.40:0/2803398412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca2c003dd0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca4656a640 1 -- 172.21.15.40:0/2803398412 >> 172.21.15.40:0/2803398412 conn(0x7fca400d0be0 msgr2=0x7fca400d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca4656a640 1 -- 172.21.15.40:0/2803398412 shutdown_connections 2024-04-26T21:38:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.649+0000 7fca4656a640 1 -- 172.21.15.40:0/2803398412 wait complete. 2024-04-26T21:38:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.650+0000 7fca4656a640 1 Processor -- start 2024-04-26T21:38:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca4656a640 1 -- start start 2024-04-26T21:38:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca4656a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca400b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca4656a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca400b3dc0 0x7fca401456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca4656a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca40145c20 0x7fca400b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca4656a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca400d0470 con 0x7fca2c0039b0 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca4656a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca400d02f0 con 0x7fca400b3dc0 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca4656a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca400cbbd0 con 0x7fca40145c20 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca400b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca400b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50484/0 (socket says 172.21.15.40:50484) 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.651+0000 7fca3ffff640 1 -- 172.21.15.40:0/1732316159 learned_addr learned my addr 172.21.15.40:0/1732316159 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca44b07640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca40145c20 0x7fca400b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3f7fe640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca400b3dc0 0x7fca401456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3ffff640 1 -- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca40145c20 msgr2=0x7fca400b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3ffff640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca40145c20 0x7fca400b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3ffff640 1 -- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca400b3dc0 msgr2=0x7fca401456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3ffff640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca400b3dc0 0x7fca401456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3ffff640 1 -- 172.21.15.40:0/1732316159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca400c7f00 con 0x7fca2c0039b0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3ffff640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca400b3880 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fca28001870 tx=0x7fca28001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3cff9640 1 -- 172.21.15.40:0/1732316159 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2803f030 con 0x7fca2c0039b0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3cff9640 1 -- 172.21.15.40:0/1732316159 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca28005480 con 0x7fca2c0039b0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca400c81e0 con 0x7fca2c0039b0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca3cff9640 1 -- 172.21.15.40:0/1732316159 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28037040 con 0x7fca2c0039b0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.652+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca400c87a0 con 0x7fca2c0039b0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.653+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca400694f0 con 0x7fca2c0039b0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.654+0000 7fca3cff9640 1 -- 172.21.15.40:0/1732316159 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca28042070 con 0x7fca2c0039b0 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.654+0000 7fca3cff9640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca2006c8f0 0x7fca2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.654+0000 7fca3cff9640 1 -- 172.21.15.40:0/1732316159 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7fca280ab250 con 0x7fca2c0039b0 2024-04-26T21:38:35.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.654+0000 7fca3f7fe640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca2006c8f0 0x7fca2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:35.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.655+0000 7fca3f7fe640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca2006c8f0 0x7fca2006edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fca30001050 tx=0x7fca300078f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:35.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.659+0000 7fca3cff9640 1 -- 172.21.15.40:0/1732316159 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca28078d20 con 0x7fca2c0039b0 2024-04-26T21:38:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.844+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}] v 0) -- 0x7fca4015ee20 con 0x7fca2c0039b0 2024-04-26T21:38:35.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.849+0000 7fca3cff9640 1 -- 172.21.15.40:0/1732316159 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}]=0 v19)=0 v19) ==== 139+0+0 (secure 0 0 0) 0x7fca2807d020 con 0x7fca2c0039b0 2024-04-26T21:38:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca2006c8f0 msgr2=0x7fca2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca2006c8f0 0x7fca2006edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fca30001050 tx=0x7fca300078f0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 msgr2=0x7fca400b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca400b3880 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fca28001870 tx=0x7fca28001120 comp rx=0 tx=0).stop 2024-04-26T21:38:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 shutdown_connections 2024-04-26T21:38:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca2006c8f0 0x7fca2006edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca40145c20 0x7fca400b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca400b3dc0 0x7fca401456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 --2- 172.21.15.40:0/1732316159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0039b0 0x7fca400b3880 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 >> 172.21.15.40:0/1732316159 conn(0x7fca400d0be0 msgr2=0x7fca4015a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 shutdown_connections 2024-04-26T21:38:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:35.851+0000 7fca4656a640 1 -- 172.21.15.40:0/1732316159 wait complete. 2024-04-26T21:38:40.548 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:38:40.551 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:38:40.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:38:40.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.979+0000 7f7904738640 1 Processor -- start 2024-04-26T21:38:40.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.979+0000 7f7904738640 1 -- start start 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f7904738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc15f170 0x7f78fc0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f7904738640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78fc0d1040 con 0x7f78fc15f550 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f7904738640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78fc0c9680 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f7904738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78fc0c9430 con 0x7f78fc15f170 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f7901cd3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78fc15f550 0x7f78fc0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44048/0 (socket says 172.21.15.40:44048) 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f7901cd3640 1 -- 172.21.15.40:0/658307316 learned_addr learned my addr 172.21.15.40:0/658307316 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f79024d4640 1 --2- 172.21.15.40:0/658307316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc15f170 0x7f78fc0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f79014d2640 1 -- 172.21.15.40:0/658307316 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453891119 0 0) 0x7f78ec001110 con 0x7f78fc15f550 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.980+0000 7f79014d2640 1 -- 172.21.15.40:0/658307316 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78e8003520 con 0x7f78fc15f550 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.981+0000 7f79014d2640 1 -- 172.21.15.40:0/658307316 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139783407 0 0) 0x7f78fc0c9680 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.981+0000 7f79014d2640 1 -- 172.21.15.40:0/658307316 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78ec001110 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.981+0000 7f79014d2640 1 -- 172.21.15.40:0/658307316 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 345390365 0 0) 0x7f78e8003520 con 0x7f78fc15f550 2024-04-26T21:38:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.981+0000 7f79024d4640 1 -- 172.21.15.40:0/658307316 >> v1:172.21.15.184:6789/0 conn(0x7f78fc0d3bc0 legacy=0x7f78fc0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.981+0000 7f79024d4640 1 -- 172.21.15.40:0/658307316 >> v1:172.21.15.40:6789/0 conn(0x7f78fc15f550 legacy=0x7f78fc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.981+0000 7f79024d4640 1 -- 172.21.15.40:0/658307316 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc0c9960 con 0x7f78fc15f170 2024-04-26T21:38:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.981+0000 7f79024d4640 1 --2- 172.21.15.40:0/658307316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc15f170 0x7f78fc0d5a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78f80097d0 tx=0x7f78f802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb0c8cdf5dc28bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f79014d2640 1 -- 172.21.15.40:0/658307316 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8037070 con 0x7f78fc15f170 2024-04-26T21:38:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f79014d2640 1 -- 172.21.15.40:0/658307316 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f802ac30 con 0x7f78fc15f170 2024-04-26T21:38:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f79014d2640 1 -- 172.21.15.40:0/658307316 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec001840 con 0x7f78fc15f170 2024-04-26T21:38:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f7904738640 1 -- 172.21.15.40:0/658307316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc15f170 msgr2=0x7f78fc0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f7904738640 1 --2- 172.21.15.40:0/658307316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc15f170 0x7f78fc0d5a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78f80097d0 tx=0x7f78f802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:38:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f7904738640 1 -- 172.21.15.40:0/658307316 shutdown_connections 2024-04-26T21:38:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f7904738640 1 --2- 172.21.15.40:0/658307316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc15f170 0x7f78fc0d5a80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f7904738640 1 -- 172.21.15.40:0/658307316 >> 172.21.15.40:0/658307316 conn(0x7f78fc0d4f60 msgr2=0x7f78fc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f7904738640 1 -- 172.21.15.40:0/658307316 shutdown_connections 2024-04-26T21:38:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.982+0000 7f7904738640 1 -- 172.21.15.40:0/658307316 wait complete. 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f7904738640 1 Processor -- start 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f7904738640 1 -- start start 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f7904738640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78fc0d3bc0 0x7f78fc15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f7904738640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78fc15f170 0x7f78fc0a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f7904738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc0a2240 0x7f78fc09f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f7904738640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0d2e30 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f7904738640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0d2cb0 con 0x7f78fc15f170 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f7904738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78fc0d2fb0 con 0x7f78fc0a2240 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f79024d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78fc0d3bc0 0x7f78fc15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f79024d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78fc0d3bc0 0x7f78fc15caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36874/0 (socket says 172.21.15.40:36874) 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.984+0000 7f79024d4640 1 -- 172.21.15.40:0/1689335978 learned_addr learned my addr 172.21.15.40:0/1689335978 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f7901cd3640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78fc15f170 0x7f78fc0a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f7902cd5640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc0a2240 0x7f78fc09f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f79024d4640 1 -- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc0a2240 msgr2=0x7f78fc09f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f79024d4640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc0a2240 0x7f78fc09f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f79024d4640 1 -- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78fc15f170 msgr2=0x7f78fc0a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f79024d4640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78fc15f170 0x7f78fc0a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f79024d4640 1 -- 172.21.15.40:0/1689335978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc15cfe0 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f7902cd5640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc0a2240 0x7f78fc09f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f79024d4640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78fc0d3bc0 0x7f78fc15caa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f78f8035040 tx=0x7f78f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f78f2ffd640 1 -- 172.21.15.40:0/1689335978 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8041020 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f78f2ffd640 1 -- 172.21.15.40:0/1689335978 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f803d640 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f78f2ffd640 1 -- 172.21.15.40:0/1689335978 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8037040 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78fc0a0110 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.985+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78fc0a06d0 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.986+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78fc0a2650 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.987+0000 7f78f2ffd640 1 -- 172.21.15.40:0/1689335978 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78f8044020 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.987+0000 7f78f2ffd640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e006c920 0x7f78e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.987+0000 7f78f2ffd640 1 -- 172.21.15.40:0/1689335978 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f78f80ac260 con 0x7f78fc0d3bc0 2024-04-26T21:38:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.987+0000 7f7901cd3640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e006c920 0x7f78e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.988+0000 7f7901cd3640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e006c920 0x7f78e006ede0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0cefa0 tx=0x7f78ec000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:40.992+0000 7f78f2ffd640 1 -- 172.21.15.40:0/1689335978 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78fc0a2650 con 0x7f78fc0d3bc0 2024-04-26T21:38:41.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.216+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f78fc0d0890 con 0x7f78fc0d3bc0 2024-04-26T21:38:41.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.536+0000 7f78f2ffd640 1 -- 172.21.15.40:0/1689335978 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v52) ==== 162+0+0 (secure 0 0 0) 0x7f78f804b030 con 0x7f78fc0d3bc0 2024-04-26T21:38:41.537 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:38:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e006c920 msgr2=0x7f78e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e006c920 0x7f78e006ede0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0cefa0 tx=0x7f78ec000ed0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78fc0d3bc0 msgr2=0x7f78fc15caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78fc0d3bc0 0x7f78fc15caa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f78f8035040 tx=0x7f78f80046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 shutdown_connections 2024-04-26T21:38:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e006c920 0x7f78e006ede0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78fc0a2240 0x7f78fc09f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78fc15f170 0x7f78fc0a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 --2- 172.21.15.40:0/1689335978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78fc0d3bc0 0x7f78fc15caa0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 >> 172.21.15.40:0/1689335978 conn(0x7f78fc0d4f60 msgr2=0x7f78fc0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 shutdown_connections 2024-04-26T21:38:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.538+0000 7f7904738640 1 -- 172.21.15.40:0/1689335978 wait complete. 2024-04-26T21:38:41.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.653+0000 7f1330bda640 1 Processor -- start 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.653+0000 7f1330bda640 1 -- start start 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f1330bda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c15d1e0 0x7f132c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f1330bda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f132c0d1a90 0x7f132c070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f1330bda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f132c0ce840 con 0x7f132c0d0f20 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f1330bda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f132c0cede0 con 0x7f132c0d1a90 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f1330bda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f132c070650 con 0x7f132c15d1e0 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132ad9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f132c0d1a90 0x7f132c070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132a59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c15d1e0 0x7f132c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132ad9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f132c0d1a90 0x7f132c070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:52166/0 (socket says 172.21.15.40:52166) 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132ad9d640 1 -- 172.21.15.40:0/1371578151 learned_addr learned my addr 172.21.15.40:0/1371578151 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f1323fff640 1 --1- 172.21.15.40:0/1371578151 >> v1:172.21.15.184:6789/0 conn(0x7f132c0d0f20 0x7f132c0d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:35512/0 (socket says 172.21.15.40:35512) 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132ad9d640 1 -- 172.21.15.40:0/1371578151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c15d1e0 msgr2=0x7f132c0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132ad9d640 1 --2- 172.21.15.40:0/1371578151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c15d1e0 0x7f132c0d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132ad9d640 1 -- 172.21.15.40:0/1371578151 >> v1:172.21.15.184:6789/0 conn(0x7f132c0d0f20 legacy=0x7f132c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:41.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132ad9d640 1 -- 172.21.15.40:0/1371578151 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c0ceb00 con 0x7f132c0d1a90 2024-04-26T21:38:41.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.654+0000 7f132a59c640 1 --2- 172.21.15.40:0/1371578151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c15d1e0 0x7f132c0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.655+0000 7f132ad9d640 1 --2- 172.21.15.40:0/1371578151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c0d1a90 0x7f132c070ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1324009e50 tx=0x7f132402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4e5aa45309d4b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.655+0000 7f1329d9b640 1 -- 172.21.15.40:0/1371578151 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324036070 con 0x7f132c0d1a90 2024-04-26T21:38:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.655+0000 7f1329d9b640 1 -- 172.21.15.40:0/1371578151 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1324031600 con 0x7f132c0d1a90 2024-04-26T21:38:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.655+0000 7f1329d9b640 1 -- 172.21.15.40:0/1371578151 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13240319d0 con 0x7f132c0d1a90 2024-04-26T21:38:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.656+0000 7f1330bda640 1 -- 172.21.15.40:0/1371578151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c0d1a90 msgr2=0x7f132c070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.656+0000 7f1330bda640 1 --2- 172.21.15.40:0/1371578151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c0d1a90 0x7f132c070ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1324009e50 tx=0x7f132402a710 comp rx=0 tx=0).stop 2024-04-26T21:38:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.656+0000 7f1330bda640 1 -- 172.21.15.40:0/1371578151 shutdown_connections 2024-04-26T21:38:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.656+0000 7f1330bda640 1 --2- 172.21.15.40:0/1371578151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c0d1a90 0x7f132c070ae0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.656+0000 7f1330bda640 1 --2- 172.21.15.40:0/1371578151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c15d1e0 0x7f132c0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.656+0000 7f1330bda640 1 -- 172.21.15.40:0/1371578151 >> 172.21.15.40:0/1371578151 conn(0x7f132c0d2da0 msgr2=0x7f132c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.656+0000 7f1330bda640 1 -- 172.21.15.40:0/1371578151 shutdown_connections 2024-04-26T21:38:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.656+0000 7f1330bda640 1 -- 172.21.15.40:0/1371578151 wait complete. 2024-04-26T21:38:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.657+0000 7f1330bda640 1 Processor -- start 2024-04-26T21:38:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.657+0000 7f1330bda640 1 -- start start 2024-04-26T21:38:41.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1330bda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c0d0f20 0x7f132c0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:41.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1330bda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c0d1a90 0x7f132c1600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:41.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1330bda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c15d1e0 0x7f132c0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:41.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1330bda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f132c071020 con 0x7f132c0d1a90 2024-04-26T21:38:41.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1330bda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f132c0ce4c0 con 0x7f132c15d1e0 2024-04-26T21:38:41.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1330bda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f132c0711a0 con 0x7f132c0d0f20 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1323fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c0d1a90 0x7f132c1600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1323fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c0d1a90 0x7f132c1600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36890/0 (socket says 172.21.15.40:36890) 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f1323fff640 1 -- 172.21.15.40:0/790151797 learned_addr learned my addr 172.21.15.40:0/790151797 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f132a59c640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c0d0f20 0x7f132c0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.658+0000 7f132ad9d640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c15d1e0 0x7f132c0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1323fff640 1 -- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c0d0f20 msgr2=0x7f132c0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1323fff640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c0d0f20 0x7f132c0c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1323fff640 1 -- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c15d1e0 msgr2=0x7f132c0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1323fff640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c15d1e0 0x7f132c0c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1323fff640 1 -- 172.21.15.40:0/790151797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c0c6ed0 con 0x7f132c0d1a90 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f132ad9d640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c15d1e0 0x7f132c0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1323fff640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c0d1a90 0x7f132c1600b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f131400c440 tx=0x7f1314001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1322ffd640 1 -- 172.21.15.40:0/790151797 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13140074e0 con 0x7f132c0d1a90 2024-04-26T21:38:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1322ffd640 1 -- 172.21.15.40:0/790151797 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1314007680 con 0x7f132c0d1a90 2024-04-26T21:38:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c0c71b0 con 0x7f132c0d1a90 2024-04-26T21:38:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1322ffd640 1 -- 172.21.15.40:0/790151797 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314004a20 con 0x7f132c0d1a90 2024-04-26T21:38:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.659+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c1511b0 con 0x7f132c0d1a90 2024-04-26T21:38:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.660+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c0694f0 con 0x7f132c0d1a90 2024-04-26T21:38:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.661+0000 7f1322ffd640 1 -- 172.21.15.40:0/790151797 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f131401a020 con 0x7f132c0d1a90 2024-04-26T21:38:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.661+0000 7f1322ffd640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f130c06c920 0x7f130c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.661+0000 7f1322ffd640 1 -- 172.21.15.40:0/790151797 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f131408b330 con 0x7f132c0d1a90 2024-04-26T21:38:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.662+0000 7f132a59c640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f130c06c920 0x7f130c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.662+0000 7f132a59c640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f130c06c920 0x7f130c06ede0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f131c009560 tx=0x7f131c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:41.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.666+0000 7f1322ffd640 1 -- 172.21.15.40:0/790151797 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1314058fc0 con 0x7f132c0d1a90 2024-04-26T21:38:41.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.890+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f132c0c8ad0 con 0x7f132c0d1a90 2024-04-26T21:38:41.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.890+0000 7f1322ffd640 1 -- 172.21.15.40:0/790151797 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5179 (secure 0 0 0) 0x7f131405d2c0 con 0x7f132c0d1a90 2024-04-26T21:38:41.892 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:41.892 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":53,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:41.539601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-26T21:38:41.892 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 53 2024-04-26T21:38:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f130c06c920 msgr2=0x7f130c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f130c06c920 0x7f130c06ede0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f131c009560 tx=0x7f131c009190 comp rx=0 tx=0).stop 2024-04-26T21:38:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c0d1a90 msgr2=0x7f132c1600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c0d1a90 0x7f132c1600b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f131400c440 tx=0x7f1314001e70 comp rx=0 tx=0).stop 2024-04-26T21:38:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 shutdown_connections 2024-04-26T21:38:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f130c06c920 0x7f130c06ede0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f132c15d1e0 0x7f132c0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c0d1a90 0x7f132c1600b0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 --2- 172.21.15.40:0/790151797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f132c0d0f20 0x7f132c0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:41.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 >> 172.21.15.40:0/790151797 conn(0x7f132c0d2da0 msgr2=0x7f132c0b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:41.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 shutdown_connections 2024-04-26T21:38:41.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:41.893+0000 7f1330bda640 1 -- 172.21.15.40:0/790151797 wait complete. 2024-04-26T21:38:42.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:38:18.310357+0000', 'modified': '2024-04-26T21:38:41.539601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 67, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6840}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6840': {'gid': 6840, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6835/1699369057', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1699369057}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1699369057}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:38:42.308 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6840:up:replay 2024-04-26T21:38:43.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:43.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.422+0000 7f518d71c640 1 Processor -- start 2024-04-26T21:38:43.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.422+0000 7f518d71c640 1 -- start start 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.422+0000 7f518d71c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5188162810 0x7f5188070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.422+0000 7f518d71c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f51880b2340 0x7f51880cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.422+0000 7f518d71c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51880cd930 0x7f51880d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.422+0000 7f518d71c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f51880d0750 con 0x7f51880cd930 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.422+0000 7f518d71c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f51880d16a0 con 0x7f51880b2340 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.422+0000 7f518d71c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51880d10d0 con 0x7f5188162810 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51877fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51880cd930 0x7f51880d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51877fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51880cd930 0x7f51880d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36900/0 (socket says 172.21.15.40:36900) 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51877fe640 1 -- 172.21.15.40:0/824857138 learned_addr learned my addr 172.21.15.40:0/824857138 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:43.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51877fe640 1 -- 172.21.15.40:0/824857138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5188162810 msgr2=0x7f5188070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:43.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51867fc640 1 --2- 172.21.15.40:0/824857138 >> v2:172.21.15.184:3300/0 conn(0x7f51880b2340 0x7f51880cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:43.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51877fe640 1 --2- 172.21.15.40:0/824857138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5188162810 0x7f5188070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51877fe640 1 -- 172.21.15.40:0/824857138 >> v2:172.21.15.184:3300/0 conn(0x7f51880b2340 msgr2=0x7f51880cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:43.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51877fe640 1 --2- 172.21.15.40:0/824857138 >> v2:172.21.15.184:3300/0 conn(0x7f51880b2340 0x7f51880cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.423+0000 7f51877fe640 1 -- 172.21.15.40:0/824857138 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51880d1370 con 0x7f51880cd930 2024-04-26T21:38:43.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f51877fe640 1 --2- 172.21.15.40:0/824857138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880cd930 0x7f51880d1b30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f518806c4e0 tx=0x7f518002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e016fe528ae9256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:43.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f5185ffb640 1 -- 172.21.15.40:0/824857138 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180036070 con 0x7f51880cd930 2024-04-26T21:38:43.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f5185ffb640 1 -- 172.21.15.40:0/824857138 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5180031600 con 0x7f51880cd930 2024-04-26T21:38:43.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f5185ffb640 1 -- 172.21.15.40:0/824857138 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800319d0 con 0x7f51880cd930 2024-04-26T21:38:43.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f518d71c640 1 -- 172.21.15.40:0/824857138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880cd930 msgr2=0x7f51880d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f518d71c640 1 --2- 172.21.15.40:0/824857138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880cd930 0x7f51880d1b30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f518806c4e0 tx=0x7f518002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f518d71c640 1 -- 172.21.15.40:0/824857138 shutdown_connections 2024-04-26T21:38:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f518d71c640 1 --2- 172.21.15.40:0/824857138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880cd930 0x7f51880d1b30 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f518d71c640 1 --2- 172.21.15.40:0/824857138 >> v2:172.21.15.184:3300/0 conn(0x7f51880b2340 0x7f51880cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f518d71c640 1 --2- 172.21.15.40:0/824857138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5188162810 0x7f5188070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.424+0000 7f518d71c640 1 -- 172.21.15.40:0/824857138 >> 172.21.15.40:0/824857138 conn(0x7f51880d2da0 msgr2=0x7f51880d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.425+0000 7f518d71c640 1 -- 172.21.15.40:0/824857138 shutdown_connections 2024-04-26T21:38:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.425+0000 7f518d71c640 1 -- 172.21.15.40:0/824857138 wait complete. 2024-04-26T21:38:43.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.426+0000 7f518d71c640 1 Processor -- start 2024-04-26T21:38:43.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.426+0000 7f518d71c640 1 -- start start 2024-04-26T21:38:43.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f518d71c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880b2340 0x7f518815d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:43.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f518d71c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51880cd930 0x7f518814ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f518d71c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5188162810 0x7f518814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f518d71c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5188070890 con 0x7f51880b2340 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f518d71c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51880d0470 con 0x7f5188162810 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f518d71c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51880d2070 con 0x7f51880cd930 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880b2340 0x7f518815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880b2340 0x7f518815d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36910/0 (socket says 172.21.15.40:36910) 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 -- 172.21.15.40:0/2029132554 learned_addr learned my addr 172.21.15.40:0/2029132554 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f51877fe640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5188162810 0x7f518814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f51867fc640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51880cd930 0x7f518814ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 -- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51880cd930 msgr2=0x7f518814ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51880cd930 0x7f518814ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 -- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5188162810 msgr2=0x7f518814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5188162810 0x7f518814b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 -- 172.21.15.40:0/2029132554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51880c8be0 con 0x7f51880b2340 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f51867fc640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51880cd930 0x7f518814ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f5186ffd640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880b2340 0x7f518815d540 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f517800b440 tx=0x7f517800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.427+0000 7f516f7fe640 1 -- 172.21.15.40:0/2029132554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178015020 con 0x7f51880b2340 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.428+0000 7f516f7fe640 1 -- 172.21.15.40:0/2029132554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5178011890 con 0x7f51880b2340 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.428+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51880c8ec0 con 0x7f51880b2340 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.428+0000 7f516f7fe640 1 -- 172.21.15.40:0/2029132554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178011c80 con 0x7f51880b2340 2024-04-26T21:38:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.428+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51880c9480 con 0x7f51880b2340 2024-04-26T21:38:43.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.429+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51880694f0 con 0x7f51880b2340 2024-04-26T21:38:43.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.429+0000 7f516f7fe640 1 -- 172.21.15.40:0/2029132554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5178018020 con 0x7f51880b2340 2024-04-26T21:38:43.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.430+0000 7f516f7fe640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f517406c8f0 0x7f517406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:43.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.430+0000 7f516f7fe640 1 -- 172.21.15.40:0/2029132554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f517808bd40 con 0x7f51880b2340 2024-04-26T21:38:43.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.430+0000 7f51867fc640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f517406c8f0 0x7f517406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:43.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.431+0000 7f51867fc640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f517406c8f0 0x7f517406edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51881f83e0 tx=0x7f517c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:43.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.434+0000 7f516f7fe640 1 -- 172.21.15.40:0/2029132554 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f517801f020 con 0x7f51880b2340 2024-04-26T21:38:43.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.662+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f518806f070 con 0x7f51880b2340 2024-04-26T21:38:43.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.662+0000 7f516f7fe640 1 -- 172.21.15.40:0/2029132554 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5179 (secure 0 0 0) 0x7f517800d040 con 0x7f51880b2340 2024-04-26T21:38:43.663 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:43.663 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:42.547983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-26T21:38:43.664 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 55 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f517406c8f0 msgr2=0x7f517406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f517406c8f0 0x7f517406edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51881f83e0 tx=0x7f517c008040 comp rx=0 tx=0).stop 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880b2340 msgr2=0x7f518815d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880b2340 0x7f518815d540 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f517800b440 tx=0x7f517800b900 comp rx=0 tx=0).stop 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 shutdown_connections 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f517406c8f0 0x7f517406edb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5188162810 0x7f518814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51880cd930 0x7f518814ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 --2- 172.21.15.40:0/2029132554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51880b2340 0x7f518815d540 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 >> 172.21.15.40:0/2029132554 conn(0x7f51880d2da0 msgr2=0x7f51880cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 shutdown_connections 2024-04-26T21:38:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:43.664+0000 7f518d71c640 1 -- 172.21.15.40:0/2029132554 wait complete. 2024-04-26T21:38:44.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:38:18.310357+0000', 'modified': '2024-04-26T21:38:42.547983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 67, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6840}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6840': {'gid': 6840, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.40:6835/1699369057', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1699369057}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1699369057}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:38:44.079 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6840:up:rejoin 2024-04-26T21:38:45.080 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.192+0000 7f72eb285640 1 Processor -- start 2024-04-26T21:38:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.193+0000 7f72eb285640 1 -- start start 2024-04-26T21:38:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.193+0000 7f72eb285640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72e40d0070 0x7f72e40d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.193+0000 7f72eb285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e40ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.193+0000 7f72eb285640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72e40d2420 con 0x7f72e406d560 2024-04-26T21:38:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.193+0000 7f72eb285640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f72e4162d00 con 0x7f72e40d0070 2024-04-26T21:38:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.193+0000 7f72eb285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72e40d2d10 con 0x7f72e406e270 2024-04-26T21:38:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9021640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72e40d0070 0x7f72e40d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9822640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e40ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9021640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72e40d0070 0x7f72e40d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:52208/0 (socket says 172.21.15.40:52208) 2024-04-26T21:38:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9021640 1 -- 172.21.15.40:0/2704792545 learned_addr learned my addr 172.21.15.40:0/2704792545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e8820640 1 --1- 172.21.15.40:0/2704792545 >> v1:172.21.15.184:6789/0 conn(0x7f72e406d560 0x7f72e406db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:35518/0 (socket says 172.21.15.40:35518) 2024-04-26T21:38:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9822640 1 -- 172.21.15.40:0/2704792545 >> v2:172.21.15.184:3300/0 conn(0x7f72e40d0070 msgr2=0x7f72e40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9822640 1 --2- 172.21.15.40:0/2704792545 >> v2:172.21.15.184:3300/0 conn(0x7f72e40d0070 0x7f72e40d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9822640 1 -- 172.21.15.40:0/2704792545 >> v1:172.21.15.184:6789/0 conn(0x7f72e406d560 legacy=0x7f72e406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9822640 1 -- 172.21.15.40:0/2704792545 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e40cd6b0 con 0x7f72e406e270 2024-04-26T21:38:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.194+0000 7f72e9021640 1 --2- 172.21.15.40:0/2704792545 >> v2:172.21.15.184:3300/0 conn(0x7f72e40d0070 0x7f72e40d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.195+0000 7f72e9822640 1 --2- 172.21.15.40:0/2704792545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e40ce190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009d90 tx=0x7f72dc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bb0e8ee88620da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.195+0000 7f72d7fff640 1 -- 172.21.15.40:0/2704792545 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc007740 con 0x7f72e406e270 2024-04-26T21:38:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.195+0000 7f72d7fff640 1 -- 172.21.15.40:0/2704792545 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72dc0078e0 con 0x7f72e406e270 2024-04-26T21:38:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.195+0000 7f72d7fff640 1 -- 172.21.15.40:0/2704792545 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc007cb0 con 0x7f72e406e270 2024-04-26T21:38:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.196+0000 7f72eb285640 1 -- 172.21.15.40:0/2704792545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 msgr2=0x7f72e40ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.196+0000 7f72eb285640 1 --2- 172.21.15.40:0/2704792545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e40ce190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009d90 tx=0x7f72dc02a660 comp rx=0 tx=0).stop 2024-04-26T21:38:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.196+0000 7f72eb285640 1 -- 172.21.15.40:0/2704792545 shutdown_connections 2024-04-26T21:38:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.196+0000 7f72eb285640 1 --2- 172.21.15.40:0/2704792545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e40ce190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.196+0000 7f72eb285640 1 --2- 172.21.15.40:0/2704792545 >> v2:172.21.15.184:3300/0 conn(0x7f72e40d0070 0x7f72e40d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.196+0000 7f72eb285640 1 -- 172.21.15.40:0/2704792545 >> 172.21.15.40:0/2704792545 conn(0x7f72e40d2ec0 msgr2=0x7f72e40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.196+0000 7f72eb285640 1 -- 172.21.15.40:0/2704792545 shutdown_connections 2024-04-26T21:38:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.196+0000 7f72eb285640 1 -- 172.21.15.40:0/2704792545 wait complete. 2024-04-26T21:38:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.197+0000 7f72eb285640 1 Processor -- start 2024-04-26T21:38:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.197+0000 7f72eb285640 1 -- start start 2024-04-26T21:38:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72eb285640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e406d560 0x7f72e415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72eb285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72eb285640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e40d0070 0x7f72e40cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72eb285640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f72e40ce6d0 con 0x7f72e406d560 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72eb285640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72e40cdc70 con 0x7f72e40d0070 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72eb285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72e40ce850 con 0x7f72e406e270 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9021640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e406d560 0x7f72e415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9021640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e406d560 0x7f72e415dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36920/0 (socket says 172.21.15.40:36920) 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9021640 1 -- 172.21.15.40:0/2413321441 learned_addr learned my addr 172.21.15.40:0/2413321441 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9822640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e40d0070 0x7f72e40cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e8820640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9021640 1 -- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 msgr2=0x7f72e415a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9021640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e415a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9021640 1 -- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e40d0070 msgr2=0x7f72e40cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9021640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e40d0070 0x7f72e40cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.198+0000 7f72e9021640 1 -- 172.21.15.40:0/2413321441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e415b560 con 0x7f72e406d560 2024-04-26T21:38:45.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.199+0000 7f72e9021640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e406d560 0x7f72e415dda0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f72e000e9b0 tx=0x7f72e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.199+0000 7f72d5ffb640 1 -- 172.21.15.40:0/2413321441 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0017070 con 0x7f72e406d560 2024-04-26T21:38:45.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.199+0000 7f72d5ffb640 1 -- 172.21.15.40:0/2413321441 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e000c9a0 con 0x7f72e406d560 2024-04-26T21:38:45.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.199+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e415b280 con 0x7f72e406d560 2024-04-26T21:38:45.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.199+0000 7f72d5ffb640 1 -- 172.21.15.40:0/2413321441 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e000cd50 con 0x7f72e406d560 2024-04-26T21:38:45.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.199+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e40c6d30 con 0x7f72e406d560 2024-04-26T21:38:45.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.200+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e40694f0 con 0x7f72e406d560 2024-04-26T21:38:45.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.201+0000 7f72d5ffb640 1 -- 172.21.15.40:0/2413321441 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72e001a070 con 0x7f72e406d560 2024-04-26T21:38:45.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.201+0000 7f72d5ffb640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.201+0000 7f72d5ffb640 1 -- 172.21.15.40:0/2413321441 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f72e008bb30 con 0x7f72e406d560 2024-04-26T21:38:45.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.202+0000 7f72e8820640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.202+0000 7f72e8820640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f72e40d16c0 tx=0x7f72d8007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.206+0000 7f72d5ffb640 1 -- 172.21.15.40:0/2413321441 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72e415b560 con 0x7f72e406d560 2024-04-26T21:38:45.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.434+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72e40d0450 con 0x7f72e406d560 2024-04-26T21:38:45.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.434+0000 7f72d5ffb640 1 -- 172.21.15.40:0/2413321441 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5186 (secure 0 0 0) 0x7f72e0059e10 con 0x7f72e406d560 2024-04-26T21:38:45.435 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:45.435 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:44.553747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6840,"qdb_cluster":[6840]},"id":4}]} 2024-04-26T21:38:45.436 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 56 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.436+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 msgr2=0x7f72c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.436+0000 7f72eb285640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f72e40d16c0 tx=0x7f72d8007cc0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.436+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e406d560 msgr2=0x7f72e415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.436+0000 7f72eb285640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e406d560 0x7f72e415dda0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f72e000e9b0 tx=0x7f72e000ee70 comp rx=0 tx=0).stop 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.437+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 shutdown_connections 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.437+0000 7f72eb285640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.437+0000 7f72eb285640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e40d0070 0x7f72e40cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.437+0000 7f72eb285640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72e406e270 0x7f72e415a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.437+0000 7f72eb285640 1 --2- 172.21.15.40:0/2413321441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e406d560 0x7f72e415dda0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.437+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 >> 172.21.15.40:0/2413321441 conn(0x7f72e40d2ec0 msgr2=0x7f72e406db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:45.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.437+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 shutdown_connections 2024-04-26T21:38:45.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.437+0000 7f72eb285640 1 -- 172.21.15.40:0/2413321441 wait complete. 2024-04-26T21:38:45.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:38:18.310357+0000', 'modified': '2024-04-26T21:38:44.553747+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 67, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6840}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6840': {'gid': 6840, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.40:6835/1699369057', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1699369057}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1699369057}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6840, 'qdb_cluster': [6840]} 2024-04-26T21:38:45.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:38:45.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:38:45.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.955+0000 7fe52d17f640 1 Processor -- start 2024-04-26T21:38:45.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.955+0000 7fe52d17f640 1 -- start start 2024-04-26T21:38:45.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.955+0000 7fe52d17f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5280d06f0 0x7fe5280d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.955+0000 7fe52d17f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe5280b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.955+0000 7fe52d17f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5280d0ba0 con 0x7fe52815f3a0 2024-04-26T21:38:45.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.955+0000 7fe52d17f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe5280d0e20 con 0x7fe5280d06f0 2024-04-26T21:38:45.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.955+0000 7fe52d17f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5280b1d70 con 0x7fe5280b2850 2024-04-26T21:38:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe51ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5280d06f0 0x7fe5280d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe52759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe5280b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe51ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5280d06f0 0x7fe5280d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:52220/0 (socket says 172.21.15.40:52220) 2024-04-26T21:38:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe51ffff640 1 -- 172.21.15.40:0/1375872290 learned_addr learned my addr 172.21.15.40:0/1375872290 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe52759e640 1 -- 172.21.15.40:0/1375872290 >> v2:172.21.15.184:3300/0 conn(0x7fe5280d06f0 msgr2=0x7fe5280d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe52759e640 1 --2- 172.21.15.40:0/1375872290 >> v2:172.21.15.184:3300/0 conn(0x7fe5280d06f0 0x7fe5280d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe52759e640 1 -- 172.21.15.40:0/1375872290 >> v1:172.21.15.184:6789/0 conn(0x7fe52815f3a0 legacy=0x7fe5280d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe52759e640 1 -- 172.21.15.40:0/1375872290 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5280b2050 con 0x7fe5280b2850 2024-04-26T21:38:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.956+0000 7fe51ffff640 1 --2- 172.21.15.40:0/1375872290 >> v2:172.21.15.184:3300/0 conn(0x7fe5280d06f0 0x7fe5280d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.957+0000 7fe52759e640 1 --2- 172.21.15.40:0/1375872290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe5280b2cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe520009d90 tx=0x7fe52002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383ebf7fde2297fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.957+0000 7fe52659c640 1 -- 172.21.15.40:0/1375872290 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520007740 con 0x7fe5280b2850 2024-04-26T21:38:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.957+0000 7fe52659c640 1 -- 172.21.15.40:0/1375872290 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5200078e0 con 0x7fe5280b2850 2024-04-26T21:38:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.957+0000 7fe52659c640 1 -- 172.21.15.40:0/1375872290 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520007cb0 con 0x7fe5280b2850 2024-04-26T21:38:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.958+0000 7fe52d17f640 1 -- 172.21.15.40:0/1375872290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 msgr2=0x7fe5280b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.958+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1375872290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe5280b2cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe520009d90 tx=0x7fe52002a660 comp rx=0 tx=0).stop 2024-04-26T21:38:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.958+0000 7fe52d17f640 1 -- 172.21.15.40:0/1375872290 shutdown_connections 2024-04-26T21:38:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.958+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1375872290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe5280b2cc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.958+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1375872290 >> v2:172.21.15.184:3300/0 conn(0x7fe5280d06f0 0x7fe5280d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.958+0000 7fe52d17f640 1 -- 172.21.15.40:0/1375872290 >> 172.21.15.40:0/1375872290 conn(0x7fe5280d4f90 msgr2=0x7fe5280d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.958+0000 7fe52d17f640 1 -- 172.21.15.40:0/1375872290 shutdown_connections 2024-04-26T21:38:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.958+0000 7fe52d17f640 1 -- 172.21.15.40:0/1375872290 wait complete. 2024-04-26T21:38:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.959+0000 7fe52d17f640 1 Processor -- start 2024-04-26T21:38:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.959+0000 7fe52d17f640 1 -- start start 2024-04-26T21:38:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52d17f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe52815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52d17f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5280d06f0 0x7fe5280cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52d17f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe52815f3a0 0x7fe5280cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52d17f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe52815f900 con 0x7fe52815f3a0 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52d17f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe52815f780 con 0x7fe5280d06f0 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52d17f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5280d23c0 con 0x7fe5280b2850 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe52815f3a0 0x7fe5280cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe52815f3a0 0x7fe5280cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36928/0 (socket says 172.21.15.40:36928) 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52759e640 1 -- 172.21.15.40:0/1016742353 learned_addr learned my addr 172.21.15.40:0/1016742353 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe526d9d640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe52815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe51ffff640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5280d06f0 0x7fe5280cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52759e640 1 -- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 msgr2=0x7fe52815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52759e640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe52815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52759e640 1 -- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5280d06f0 msgr2=0x7fe5280cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52759e640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5280d06f0 0x7fe5280cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.960+0000 7fe52759e640 1 -- 172.21.15.40:0/1016742353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5280c9dd0 con 0x7fe52815f3a0 2024-04-26T21:38:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.961+0000 7fe52759e640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe52815f3a0 0x7fe5280cae70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe520009eb0 tx=0x7fe520004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.961+0000 7fe51f7fe640 1 -- 172.21.15.40:0/1016742353 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52003f070 con 0x7fe52815f3a0 2024-04-26T21:38:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.961+0000 7fe51f7fe640 1 -- 172.21.15.40:0/1016742353 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe520031c60 con 0x7fe52815f3a0 2024-04-26T21:38:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.961+0000 7fe52d17f640 1 -- 172.21.15.40:0/1016742353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5280ca0b0 con 0x7fe52815f3a0 2024-04-26T21:38:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.961+0000 7fe51f7fe640 1 -- 172.21.15.40:0/1016742353 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520009450 con 0x7fe52815f3a0 2024-04-26T21:38:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.961+0000 7fe52d17f640 1 -- 172.21.15.40:0/1016742353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5280ca670 con 0x7fe52815f3a0 2024-04-26T21:38:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.962+0000 7fe51f7fe640 1 -- 172.21.15.40:0/1016742353 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe52003b410 con 0x7fe52815f3a0 2024-04-26T21:38:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.963+0000 7fe51f7fe640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe50806c8f0 0x7fe50806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.963+0000 7fe51f7fe640 1 -- 172.21.15.40:0/1016742353 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7fe5200acf50 con 0x7fe52815f3a0 2024-04-26T21:38:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.963+0000 7fe526d9d640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe50806c8f0 0x7fe50806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.964+0000 7fe526d9d640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe50806c8f0 0x7fe50806edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe518001780 tx=0x7fe518009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.964+0000 7fe52ff73740 1 Processor -- start 2024-04-26T21:38:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.964+0000 7fe52ff73740 1 -- start start 2024-04-26T21:38:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 0x55e0b61d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61f1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61a9120 0x55e0b61ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe52ff73740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e0b610d740 con 0x55e0b61f0ce0 2024-04-26T21:38:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe52ff73740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e0b60a93a0 con 0x55e0b61a9120 2024-04-26T21:38:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe52ff73740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e0b6109700 con 0x55e0b61d3440 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61f1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36942/0 (socket says 172.21.15.40:36942) 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe51ffff640 1 -- 172.21.15.40:0/1016562615 learned_addr learned my addr 172.21.15.40:0/1016562615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe526d9d640 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 0x55e0b61d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe52759e640 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61a9120 0x55e0b61ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe51ffff640 1 -- 172.21.15.40:0/1016562615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 msgr2=0x55e0b61d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe51ffff640 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 0x55e0b61d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe51ffff640 1 -- 172.21.15.40:0/1016562615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61a9120 msgr2=0x55e0b61ef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe51ffff640 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61a9120 0x55e0b61ef3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.965+0000 7fe51ffff640 1 -- 172.21.15.40:0/1016562615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b60c5750 con 0x55e0b61f0ce0 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe51ffff640 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61f1230 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe510009b00 tx=0x7fe5100112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aa382cc7f6c11a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe51cff9640 1 -- 172.21.15.40:0/1016562615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510011d00 con 0x55e0b61f0ce0 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe51cff9640 1 -- 172.21.15.40:0/1016562615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe510014440 con 0x55e0b61f0ce0 2024-04-26T21:38:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe51cff9640 1 -- 172.21.15.40:0/1016562615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510011ea0 con 0x55e0b61f0ce0 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe52ff73740 1 -- 172.21.15.40:0/1016562615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 msgr2=0x55e0b61f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61f1230 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe510009b00 tx=0x7fe5100112c0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe52ff73740 1 -- 172.21.15.40:0/1016562615 shutdown_connections 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61a9120 0x55e0b61ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61f1230 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1016562615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 0x55e0b61d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.966+0000 7fe52ff73740 1 -- 172.21.15.40:0/1016562615 >> 172.21.15.40:0/1016562615 conn(0x55e0b61a5a50 msgr2=0x55e0b61ee250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.967+0000 7fe52ff73740 1 -- 172.21.15.40:0/1016562615 shutdown_connections 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.967+0000 7fe52ff73740 1 -- 172.21.15.40:0/1016562615 wait complete. 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.967+0000 7fe52ff73740 1 -- start start 2024-04-26T21:38:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.967+0000 7fe52ff73740 1 Processor -- start 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b61d5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61d5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b6201780 0x55e0b61ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe52ff73740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e0b61a9ec0 con 0x55e0b61f0ce0 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe52ff73740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e0b6094fe0 con 0x55e0b61d3440 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe52ff73740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e0b61a3980 con 0x55e0b6201780 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe52ff73740 10 client.0 _ll_register_callbacks cb 0x55e0b60ab150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61d5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61d5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36952/0 (socket says 172.21.15.40:36952) 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe51ffff640 1 -- 172.21.15.40:0/2086676693 learned_addr learned my addr 172.21.15.40:0/2086676693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe526d9d640 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b61d5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe52759e640 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b6201780 0x55e0b61ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe51ffff640 1 -- 172.21.15.40:0/2086676693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b6201780 msgr2=0x55e0b61ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe51ffff640 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b6201780 0x55e0b61ef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe51ffff640 1 -- 172.21.15.40:0/2086676693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 msgr2=0x55e0b61d5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe51ffff640 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b61d5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.968+0000 7fe51ffff640 1 -- 172.21.15.40:0/2086676693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b5e1f2d0 con 0x55e0b61f0ce0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe51ffff640 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61d5e60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe5100117c0 tx=0x7fe510004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64841195232e036b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe505ffb640 1 -- 172.21.15.40:0/2086676693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510014b60 con 0x55e0b61f0ce0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe505ffb640 1 -- 172.21.15.40:0/2086676693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7fe510014d00 con 0x55e0b61f0ce0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 1 -- 172.21.15.40:0/2086676693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe510014440 con 0x55e0b61f0ce0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe505ffb640 1 -- 172.21.15.40:0/2086676693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe51002d050 con 0x55e0b61f0ce0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 10 client.6873 fetch_fsmap learned FSMap version 56 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 1 -- 172.21.15.40:0/2086676693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55e0b5ee96b0 con 0x55e0b61f0ce0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe505ffb640 1 -- 172.21.15.40:0/2086676693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe510014440 con 0x55e0b61f0ce0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe505ffb640 1 -- 172.21.15.40:0/2086676693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7fe510030020 con 0x55e0b61f0ce0 2024-04-26T21:38:45.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 10 client.6873 fetch_fsmap finished waiting for FSMap version 56 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 10 client.6873 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 20 client.6873 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x55e0b61d41a0 0x55e0b61d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 4 client.6873 mds_command: new command op to 6840 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe52ff73740 1 -- 172.21.15.40:0/2086676693 --> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0b611b350 con 0x55e0b61d41a0 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.969+0000 7fe526d9d640 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x55e0b61d41a0 0x55e0b61d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.970+0000 7fe526d9d640 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x55e0b61d41a0 0x55e0b61d4580 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b31ea1ee7418bd server_cookie=9cf361fae9803017 in_seq=0 out_seq=0 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.970+0000 7fe505ffb640 10 client.6873 ms_handle_connect on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.970+0000 7fe505ffb640 1 -- 172.21.15.40:0/2086676693 <== mds.0 v2:172.21.15.40:6832/1699369057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0b611b350 con 0x55e0b61d41a0 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.970+0000 7fe505ffb640 10 client.6873 handle_command_reply: tid=0 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.970+0000 7fe52ff73740 1 client.6873 shutdown 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.970+0000 7fe52ff73740 1 -- 172.21.15.40:0/2086676693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 msgr2=0x55e0b61d5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.970+0000 7fe52ff73740 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61d5e60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe5100117c0 tx=0x7fe510004730 comp rx=0 tx=0).stop 2024-04-26T21:38:45.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 1 -- 172.21.15.40:0/2086676693 shutdown_connections 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x55e0b61d41a0 0x55e0b61d4580 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b6201780 0x55e0b61ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b61f0ce0 0x55e0b61d5e60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 1 --2- 172.21.15.40:0/2086676693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b61d5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe505ffb640 0 client.6873 ms_handle_reset on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 1 -- 172.21.15.40:0/2086676693 >> 172.21.15.40:0/2086676693 conn(0x55e0b61a9120 msgr2=0x55e0b61f2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 1 -- 172.21.15.40:0/2086676693 shutdown_connections 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 1 -- 172.21.15.40:0/2086676693 wait complete. 2024-04-26T21:38:45.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.971+0000 7fe52ff73740 20 client.6873 trim_cache size 0 max 16384 2024-04-26T21:38:45.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.989+0000 7fe52ff73740 1 Processor -- start 2024-04-26T21:38:45.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.989+0000 7fe52ff73740 1 -- start start 2024-04-26T21:38:45.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 0x55e0b61e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61f0ce0 0x55e0b6208090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b61d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52ff73740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe528205550 con 0x55e0b6201780 2024-04-26T21:38:45.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52ff73740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5280ccb50 con 0x55e0b61f0ce0 2024-04-26T21:38:45.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52ff73740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe528204f00 con 0x55e0b61d3440 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b61d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b61d5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36962/0 (socket says 172.21.15.40:36962) 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 -- 172.21.15.40:0/805474335 learned_addr learned my addr 172.21.15.40:0/805474335 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 -- 172.21.15.40:0/805474335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 msgr2=0x55e0b61e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 --2- 172.21.15.40:0/805474335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 0x55e0b61e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 -- 172.21.15.40:0/805474335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61f0ce0 msgr2=0x55e0b6208090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 --2- 172.21.15.40:0/805474335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61f0ce0 0x55e0b6208090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 -- 172.21.15.40:0/805474335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b6208b90 con 0x55e0b6201780 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe52759e640 1 --2- 172.21.15.40:0/805474335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b61d5c30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe520009e80 tx=0x7fe520009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6596366b3b462c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe507fff640 1 -- 172.21.15.40:0/805474335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52007be60 con 0x55e0b6201780 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe507fff640 1 -- 172.21.15.40:0/805474335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe520083710 con 0x55e0b6201780 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.990+0000 7fe507fff640 1 -- 172.21.15.40:0/805474335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520031e10 con 0x55e0b6201780 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 -- 172.21.15.40:0/805474335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 msgr2=0x55e0b61d5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 --2- 172.21.15.40:0/805474335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b61d5c30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe520009e80 tx=0x7fe520009b20 comp rx=0 tx=0).stop 2024-04-26T21:38:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 -- 172.21.15.40:0/805474335 shutdown_connections 2024-04-26T21:38:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 --2- 172.21.15.40:0/805474335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b61d5c30 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 --2- 172.21.15.40:0/805474335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61f0ce0 0x55e0b6208090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 --2- 172.21.15.40:0/805474335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61d3440 0x55e0b61e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 -- 172.21.15.40:0/805474335 >> 172.21.15.40:0/805474335 conn(0x55e0b61a9120 msgr2=0x55e0b619ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 -- 172.21.15.40:0/805474335 shutdown_connections 2024-04-26T21:38:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 -- 172.21.15.40:0/805474335 wait complete. 2024-04-26T21:38:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 -- start start 2024-04-26T21:38:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.991+0000 7fe52ff73740 1 Processor -- start 2024-04-26T21:38:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b6207d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61f0ce0 0x55e0b62082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52ff73740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b6206040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52ff73740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e0b61fe890 con 0x55e0b6201780 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52ff73740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e0b62026b0 con 0x55e0b61d3440 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52ff73740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e0b61e5740 con 0x55e0b61f0ce0 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52ff73740 10 client.0 _ll_register_callbacks cb 0x7fe520009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b6206040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b6206040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36966/0 (socket says 172.21.15.40:36966) 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe52759e640 1 -- 172.21.15.40:0/1373269663 learned_addr learned my addr 172.21.15.40:0/1373269663 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe51ffff640 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61f0ce0 0x55e0b62082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.992+0000 7fe526d9d640 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b6207d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52759e640 1 -- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61f0ce0 msgr2=0x55e0b62082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52759e640 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61f0ce0 0x55e0b62082a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52759e640 1 -- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 msgr2=0x55e0b6207d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52759e640 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b6207d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52759e640 1 -- 172.21.15.40:0/1373269663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b62628c0 con 0x55e0b6201780 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe526d9d640 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b6207d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52759e640 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b6206040 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe5200ab380 tx=0x7fe520009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de83111a63a6f5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe505ffb640 1 -- 172.21.15.40:0/1373269663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52009a060 con 0x55e0b6201780 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe505ffb640 1 -- 172.21.15.40:0/1373269663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7fe52007ae40 con 0x55e0b6201780 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 1 -- 172.21.15.40:0/1373269663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe520083710 con 0x55e0b6201780 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe505ffb640 1 -- 172.21.15.40:0/1373269663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe52007be60 con 0x55e0b6201780 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 10 client.6879 fetch_fsmap learned FSMap version 56 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 1 -- 172.21.15.40:0/1373269663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55e0b6260790 con 0x55e0b6201780 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe505ffb640 1 -- 172.21.15.40:0/1373269663 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe520083710 con 0x55e0b6201780 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe505ffb640 1 -- 172.21.15.40:0/1373269663 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7fe52009bce0 con 0x55e0b6201780 2024-04-26T21:38:45.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 10 client.6879 fetch_fsmap finished waiting for FSMap version 56 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 10 client.6879 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 20 client.6879 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x55e0b6206580 0x55e0b62110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 4 client.6879 mds_command: new command op to 6840 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe52ff73740 1 -- 172.21.15.40:0/1373269663 --> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] -- command(tid 0: {"prefix": "status"}) -- 0x55e0b61151c0 con 0x55e0b6206580 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.993+0000 7fe526d9d640 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x55e0b6206580 0x55e0b62110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe526d9d640 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x55e0b6206580 0x55e0b62110f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f9201ae1717f5b server_cookie=3aa5021363c51f3 in_seq=0 out_seq=0 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe505ffb640 10 client.6879 ms_handle_connect on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe505ffb640 1 -- 172.21.15.40:0/1373269663 <== mds.0 v2:172.21.15.40:6832/1699369057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e0b61151c0 con 0x55e0b6206580 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe505ffb640 10 client.6879 handle_command_reply: tid=0 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 client.6879 shutdown 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 -- 172.21.15.40:0/1373269663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 msgr2=0x55e0b6206040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b6206040 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe5200ab380 tx=0x7fe520009750 comp rx=0 tx=0).stop 2024-04-26T21:38:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 -- 172.21.15.40:0/1373269663 shutdown_connections 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x55e0b6206580 0x55e0b62110f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e0b6201780 0x55e0b6206040 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e0b61f0ce0 0x55e0b62082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe505ffb640 0 client.6879 ms_handle_reset on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 --2- 172.21.15.40:0/1373269663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e0b61d3440 0x55e0b6207d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 -- 172.21.15.40:0/1373269663 >> 172.21.15.40:0/1373269663 conn(0x55e0b61a9120 msgr2=0x55e0b61e6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 -- 172.21.15.40:0/1373269663 shutdown_connections 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 1 -- 172.21.15.40:0/1373269663 wait complete. 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.994+0000 7fe52ff73740 20 client.6879 trim_cache size 0 max 16384 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:38:46.001 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6840, 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 4.4543834139999996, 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 56, 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 67, 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 67, 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.430193242 2024-04-26T21:38:46.002 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:38:46.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 -- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe50806c8f0 msgr2=0x7fe50806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe50806c8f0 0x7fe50806edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe518001780 tx=0x7fe518009040 comp rx=0 tx=0).stop 2024-04-26T21:38:46.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 -- 172.21.15.40:0/1016742353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe52815f3a0 msgr2=0x7fe5280cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe52815f3a0 0x7fe5280cae70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe520009eb0 tx=0x7fe520004a40 comp rx=0 tx=0).stop 2024-04-26T21:38:46.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 -- 172.21.15.40:0/1016742353 shutdown_connections 2024-04-26T21:38:46.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe50806c8f0 0x7fe50806edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe52815f3a0 0x7fe5280cae70 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5280d06f0 0x7fe5280cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 --2- 172.21.15.40:0/1016742353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5280b2850 0x7fe52815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 -- 172.21.15.40:0/1016742353 >> 172.21.15.40:0/1016742353 conn(0x7fe5280d4f90 msgr2=0x7fe52806e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:46.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 -- 172.21.15.40:0/1016742353 shutdown_connections 2024-04-26T21:38:46.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:45.995+0000 7fe52d17f640 1 -- 172.21.15.40:0/1016742353 wait complete. 2024-04-26T21:38:46.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.106+0000 7f2cc7d19640 1 Processor -- start 2024-04-26T21:38:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.106+0000 7f2cc7d19640 1 -- start start 2024-04-26T21:38:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc7d19640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cc015f3a0 0x7f2cc00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc7d19640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2cc00d2330 0x7f2cc00b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc7d19640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cc00d0e20 con 0x7f2cc00d1770 2024-04-26T21:38:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc7d19640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc00b1d70 con 0x7f2cc015f3a0 2024-04-26T21:38:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc7d19640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc00b2050 con 0x7f2cc00d2330 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc5ab5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cc015f3a0 0x7f2cc00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc5ab5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cc015f3a0 0x7f2cc00d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36974/0 (socket says 172.21.15.40:36974) 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc5ab5640 1 -- 172.21.15.40:0/2146895302 learned_addr learned my addr 172.21.15.40:0/2146895302 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.107+0000 7f2cc52b4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2cc00d1770 0x7f2cc00d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44058/0 (socket says 172.21.15.40:44058) 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.108+0000 7f2cc62b6640 1 --2- 172.21.15.40:0/2146895302 >> v2:172.21.15.184:3300/0 conn(0x7f2cc00d2330 0x7f2cc00b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.108+0000 7f2cc5ab5640 1 -- 172.21.15.40:0/2146895302 >> v2:172.21.15.184:3300/0 conn(0x7f2cc00d2330 msgr2=0x7f2cc00b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.108+0000 7f2cc5ab5640 1 --2- 172.21.15.40:0/2146895302 >> v2:172.21.15.184:3300/0 conn(0x7f2cc00d2330 0x7f2cc00b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.108+0000 7f2cc5ab5640 1 -- 172.21.15.40:0/2146895302 >> v1:172.21.15.40:6789/0 conn(0x7f2cc00d1770 legacy=0x7f2cc00d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.108+0000 7f2cc4ab3640 1 -- 172.21.15.40:0/2146895302 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666502190 0 0) 0x7f2cc00d0e20 con 0x7f2cc00d1770 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.108+0000 7f2cc5ab5640 1 -- 172.21.15.40:0/2146895302 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc00b2330 con 0x7f2cc015f3a0 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.108+0000 7f2cc62b6640 1 --2- 172.21.15.40:0/2146895302 >> v2:172.21.15.184:3300/0 conn(0x7f2cc00d2330 0x7f2cc00b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc5ab5640 1 --2- 172.21.15.40:0/2146895302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00d06f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00c89f0 tx=0x7f2cb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f413fe2e48dd1171 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc4ab3640 1 -- 172.21.15.40:0/2146895302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8037070 con 0x7f2cc015f3a0 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc4ab3640 1 -- 172.21.15.40:0/2146895302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb8032440 con 0x7f2cc015f3a0 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc4ab3640 1 -- 172.21.15.40:0/2146895302 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8032810 con 0x7f2cc015f3a0 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc7d19640 1 -- 172.21.15.40:0/2146895302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 msgr2=0x7f2cc00d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/2146895302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00d06f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00c89f0 tx=0x7f2cb802a270 comp rx=0 tx=0).stop 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc7d19640 1 -- 172.21.15.40:0/2146895302 shutdown_connections 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/2146895302 >> v2:172.21.15.184:3300/0 conn(0x7f2cc00d2330 0x7f2cc00b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/2146895302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00d06f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc7d19640 1 -- 172.21.15.40:0/2146895302 >> 172.21.15.40:0/2146895302 conn(0x7f2cc00d4f90 msgr2=0x7f2cc00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc7d19640 1 -- 172.21.15.40:0/2146895302 shutdown_connections 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.109+0000 7f2cc7d19640 1 -- 172.21.15.40:0/2146895302 wait complete. 2024-04-26T21:38:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc7d19640 1 Processor -- start 2024-04-26T21:38:46.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc7d19640 1 -- start start 2024-04-26T21:38:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc7d19640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc00d1770 0x7f2cc0157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc7d19640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc00d2330 0x7f2cc01f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc7d19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00ba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc7d19640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc00b2f10 con 0x7f2cc015f3a0 2024-04-26T21:38:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc7d19640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc00b2d90 con 0x7f2cc00d2330 2024-04-26T21:38:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc7d19640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc00b3090 con 0x7f2cc00d1770 2024-04-26T21:38:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc62b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00ba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc62b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00ba0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36986/0 (socket says 172.21.15.40:36986) 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.111+0000 7f2cc62b6640 1 -- 172.21.15.40:0/3931312298 learned_addr learned my addr 172.21.15.40:0/3931312298 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc52b4640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc00d2330 0x7f2cc01f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc5ab5640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc00d1770 0x7f2cc0157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc62b6640 1 -- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc00d1770 msgr2=0x7f2cc0157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc62b6640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc00d1770 0x7f2cc0157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc62b6640 1 -- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc00d2330 msgr2=0x7f2cc01f8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc62b6640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc00d2330 0x7f2cc01f8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc62b6640 1 -- 172.21.15.40:0/3931312298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc01f8a50 con 0x7f2cc015f3a0 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc52b4640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc00d2330 0x7f2cc01f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc5ab5640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc00d1770 0x7f2cc0157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc62b6640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00ba0e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0007ef0 tx=0x7f2cb000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/3931312298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb001a070 con 0x7f2cc015f3a0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/3931312298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb0003b90 con 0x7f2cc015f3a0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01f8d30 con 0x7f2cc015f3a0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/3931312298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0004030 con 0x7f2cc015f3a0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.112+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc00b8010 con 0x7f2cc015f3a0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.113+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc00b8210 con 0x7f2cc015f3a0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.114+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/3931312298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cb001d070 con 0x7f2cc015f3a0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.114+0000 7f2cbe7fc640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.114+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/3931312298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f2cb008be80 con 0x7f2cc015f3a0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.114+0000 7f2cc5ab5640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.115+0000 7f2cc5ab5640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8034000 tx=0x7f2cb802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.119+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/3931312298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cc00b8210 con 0x7f2cc015f3a0 2024-04-26T21:38:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.340+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cc00694f0 con 0x7f2cc015f3a0 2024-04-26T21:38:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.341+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/3931312298 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5186 (secure 0 0 0) 0x7f2cb0010070 con 0x7f2cc015f3a0 2024-04-26T21:38:46.342 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:46.343 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:44.553747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6840,"qdb_cluster":[6840]},"id":4}]} 2024-04-26T21:38:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 56 2024-04-26T21:38:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 msgr2=0x7f2cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8034000 tx=0x7f2cb802e000 comp rx=0 tx=0).stop 2024-04-26T21:38:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 msgr2=0x7f2cc00ba0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00ba0e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0007ef0 tx=0x7f2cb000d4a0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 shutdown_connections 2024-04-26T21:38:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f3a0 0x7f2cc00ba0e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc00d2330 0x7f2cc01f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 --2- 172.21.15.40:0/3931312298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc00d1770 0x7f2cc0157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 >> 172.21.15.40:0/3931312298 conn(0x7f2cc00d4f90 msgr2=0x7f2cc006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 shutdown_connections 2024-04-26T21:38:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.343+0000 7f2cc7d19640 1 -- 172.21.15.40:0/3931312298 wait complete. 2024-04-26T21:38:46.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 true 2024-04-26T21:38:46.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.457+0000 7fd3e66fc640 1 Processor -- start 2024-04-26T21:38:46.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.457+0000 7fd3e66fc640 1 -- start start 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3e66fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3e00d3bc0 0x7fd3e00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3e66fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e00b5f20 0x7fd3e00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3e66fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e00b7e50 con 0x7fd3e0162880 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3e66fc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e00d0580 con 0x7fd3e00d3bc0 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3e66fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e00d1800 con 0x7fd3e00b5f20 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3df7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3e00d3bc0 0x7fd3e00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3dffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd3e0162880 0x7fd3e00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44060/0 (socket says 172.21.15.40:44060) 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3dffff640 1 -- 172.21.15.40:0/1353001845 learned_addr learned my addr 172.21.15.40:0/1353001845 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3e4c99640 1 --2- 172.21.15.40:0/1353001845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e00b5f20 0x7fd3e00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3df7fe640 1 -- 172.21.15.40:0/1353001845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e00b5f20 msgr2=0x7fd3e00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3df7fe640 1 --2- 172.21.15.40:0/1353001845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e00b5f20 0x7fd3e00b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3df7fe640 1 -- 172.21.15.40:0/1353001845 >> v1:172.21.15.40:6789/0 conn(0x7fd3e0162880 legacy=0x7fd3e00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3df7fe640 1 -- 172.21.15.40:0/1353001845 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e00d07d0 con 0x7fd3e00d3bc0 2024-04-26T21:38:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3deffd640 1 -- 172.21.15.40:0/1353001845 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4270999881 0 0) 0x7fd3e00b7e50 con 0x7fd3e0162880 2024-04-26T21:38:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.458+0000 7fd3e4c99640 1 --2- 172.21.15.40:0/1353001845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e00b5f20 0x7fd3e00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3df7fe640 1 --2- 172.21.15.40:0/1353001845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00d3bc0 0x7fd3e00d2cb0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40097a0 tx=0x7fd3d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17771356b75175df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3deffd640 1 -- 172.21.15.40:0/1353001845 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4037070 con 0x7fd3e00d3bc0 2024-04-26T21:38:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3deffd640 1 -- 172.21.15.40:0/1353001845 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d4032440 con 0x7fd3e00d3bc0 2024-04-26T21:38:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3deffd640 1 -- 172.21.15.40:0/1353001845 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4032810 con 0x7fd3e00d3bc0 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3e66fc640 1 -- 172.21.15.40:0/1353001845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00d3bc0 msgr2=0x7fd3e00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/1353001845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00d3bc0 0x7fd3e00d2cb0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40097a0 tx=0x7fd3d402a180 comp rx=0 tx=0).stop 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3e66fc640 1 -- 172.21.15.40:0/1353001845 shutdown_connections 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/1353001845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e00b5f20 0x7fd3e00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/1353001845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00d3bc0 0x7fd3e00d2cb0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.459+0000 7fd3e66fc640 1 -- 172.21.15.40:0/1353001845 >> 172.21.15.40:0/1353001845 conn(0x7fd3e00d4f60 msgr2=0x7fd3e00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.460+0000 7fd3e66fc640 1 -- 172.21.15.40:0/1353001845 shutdown_connections 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.460+0000 7fd3e66fc640 1 -- 172.21.15.40:0/1353001845 wait complete. 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.461+0000 7fd3e66fc640 1 Processor -- start 2024-04-26T21:38:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.461+0000 7fd3e66fc640 1 -- start start 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.461+0000 7fd3e66fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00b5f20 0x7fd3e0154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.461+0000 7fd3e66fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e00d3bc0 0x7fd3e006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.461+0000 7fd3e66fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e0162880 0x7fd3e006e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3e66fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0162c60 con 0x7fd3e00b5f20 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3e66fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e00d0d90 con 0x7fd3e00d3bc0 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3e66fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e0162de0 con 0x7fd3e0162880 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00b5f20 0x7fd3e0154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00b5f20 0x7fd3e0154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37012/0 (socket says 172.21.15.40:37012) 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 -- 172.21.15.40:0/3243749922 learned_addr learned my addr 172.21.15.40:0/3243749922 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3e4c99640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e0162880 0x7fd3e006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3df7fe640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e00d3bc0 0x7fd3e006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 -- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e0162880 msgr2=0x7fd3e006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e0162880 0x7fd3e006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 -- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e00d3bc0 msgr2=0x7fd3e006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e00d3bc0 0x7fd3e006dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 -- 172.21.15.40:0/3243749922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e00b27f0 con 0x7fd3e00b5f20 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3e4c99640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e0162880 0x7fd3e006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3df7fe640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e00d3bc0 0x7fd3e006dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dffff640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00b5f20 0x7fd3e0154550 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000c500 tx=0x7fd3d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dcff9640 1 -- 172.21.15.40:0/3243749922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0018070 con 0x7fd3e00b5f20 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dcff9640 1 -- 172.21.15.40:0/3243749922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d0007580 con 0x7fd3e00b5f20 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e00b2ad0 con 0x7fd3e00b5f20 2024-04-26T21:38:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3dcff9640 1 -- 172.21.15.40:0/3243749922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0004a20 con 0x7fd3e00b5f20 2024-04-26T21:38:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.462+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e00b3090 con 0x7fd3e00b5f20 2024-04-26T21:38:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.463+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e00694f0 con 0x7fd3e00b5f20 2024-04-26T21:38:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.464+0000 7fd3dcff9640 1 -- 172.21.15.40:0/3243749922 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3d0004bc0 con 0x7fd3e00b5f20 2024-04-26T21:38:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.464+0000 7fd3dcff9640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3cc06c8f0 0x7fd3cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.464+0000 7fd3dcff9640 1 -- 172.21.15.40:0/3243749922 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7fd3d008b030 con 0x7fd3e00b5f20 2024-04-26T21:38:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.464+0000 7fd3df7fe640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3cc06c8f0 0x7fd3cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.465+0000 7fd3df7fe640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3cc06c8f0 0x7fd3cc06edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4034000 tx=0x7fd3d402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:46.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.468+0000 7fd3dcff9640 1 -- 172.21.15.40:0/3243749922 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3d0059290 con 0x7fd3e00b5f20 2024-04-26T21:38:46.536 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:38:46.534+0000 7f2030ff2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:38:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:46.729+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"} v 0) -- 0x7fd3e015e200 con 0x7fd3e00b5f20 2024-04-26T21:38:47.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.565+0000 7fd3dcff9640 1 -- 172.21.15.40:0/3243749922 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"}]=0 freezing mds.6840 v57) ==== 112+0+0 (secure 0 0 0) 0x7fd3d0058b90 con 0x7fd3e00b5f20 2024-04-26T21:38:47.567 INFO:teuthology.orchestra.run.smithi040.stderr:freezing mds.6840 2024-04-26T21:38:47.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3cc06c8f0 msgr2=0x7fd3cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3cc06c8f0 0x7fd3cc06edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4034000 tx=0x7fd3d402e000 comp rx=0 tx=0).stop 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00b5f20 msgr2=0x7fd3e0154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00b5f20 0x7fd3e0154550 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000c500 tx=0x7fd3d000c9c0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 shutdown_connections 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd3cc06c8f0 0x7fd3cc06edb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e0162880 0x7fd3e006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e00d3bc0 0x7fd3e006dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 --2- 172.21.15.40:0/3243749922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e00b5f20 0x7fd3e0154550 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 >> 172.21.15.40:0/3243749922 conn(0x7fd3e00d4f60 msgr2=0x7fd3e00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 shutdown_connections 2024-04-26T21:38:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.568+0000 7fd3e66fc640 1 -- 172.21.15.40:0/3243749922 wait complete. 2024-04-26T21:38:47.583 DEBUG:teuthology.orchestra.run.smithi040:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-26T21:38:47.586 DEBUG:teuthology.orchestra.run.smithi040:> timeout 30 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --connect-timeout=60 tell mds.4:0 flush journal 2024-04-26T21:38:47.588 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-26T21:38:47.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.697+0000 7f36b8740640 1 Processor -- start 2024-04-26T21:38:47.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.697+0000 7f36b8740640 1 -- start start 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b8740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 0x7f36b00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b8740640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b00d0b70 con 0x7f36b015f3a0 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b8740640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b00d0e20 con 0x7f36b00b2850 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b8740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36b00b1d70 con 0x7f36b00d06f0 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b64dc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f36b015f3a0 0x7f36b00d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44062/0 (socket says 172.21.15.40:44062) 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b64dc640 1 -- 172.21.15.40:0/1688986102 learned_addr learned my addr 172.21.15.40:0/1688986102 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4137126046 0 0) 0x7f36b00d0b70 con 0x7f36b015f3a0 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f369c003520 con 0x7f36b015f3a0 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 0x7f36b00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 779620365 0 0) 0x7f369c003520 con 0x7f36b015f3a0 2024-04-26T21:38:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 msgr2=0x7f36b00d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 0x7f36b00d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 >> v1:172.21.15.184:6789/0 conn(0x7f36b00b2850 legacy=0x7f36b00b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b00b2050 con 0x7f36b015f3a0 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f36ac001670 con 0x7f36b015f3a0 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 >> v1:172.21.15.40:6789/0 conn(0x7f36b015f3a0 legacy=0x7f36b00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 0x7f369c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.698+0000 7f36b54da640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f369c003fb0 0x7f369c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b54da640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369c004d40 0x7f369c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36b00b1d70 con 0x7f369c003fb0 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f369c00bb00 con 0x7f369c004d40 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f369c00bd00 con 0x7f36b00d06f0 2024-04-26T21:38:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f369c003fb0 0x7f369c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369c004d40 0x7f369c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b64dc640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 0x7f369c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b5cdb640 1 -- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 msgr2=0x7f369c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 0x7f369c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b5cdb640 1 -- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369c004d40 msgr2=0x7f369c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369c004d40 0x7f369c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b5cdb640 1 -- 172.21.15.40:0/1688986102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c003520 con 0x7f369c003fb0 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.699+0000 7f36b64dc640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00d06f0 0x7f369c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.700+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f369c003fb0 0x7f369c004800 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009ac0 tx=0x7f369c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2666df873e1f5cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.700+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0036070 con 0x7f369c003fb0 2024-04-26T21:38:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.700+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a0031400 con 0x7f369c003fb0 2024-04-26T21:38:47.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.700+0000 7f36b54da640 1 -- 172.21.15.40:0/1688986102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0009690 con 0x7f369c003fb0 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.701+0000 7f36b8740640 1 -- 172.21.15.40:0/1688986102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f369c003fb0 msgr2=0x7f369c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.701+0000 7f36b8740640 1 --2- 172.21.15.40:0/1688986102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f369c003fb0 0x7f369c004800 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009ac0 tx=0x7f369c005830 comp rx=0 tx=0).stop 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.701+0000 7f36b6cdd640 1 -- 172.21.15.40:0/1688986102 reap_dead start 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.701+0000 7f36b8740640 1 -- 172.21.15.40:0/1688986102 shutdown_connections 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.701+0000 7f36b8740640 1 -- 172.21.15.40:0/1688986102 >> 172.21.15.40:0/1688986102 conn(0x7f36b00d4f90 msgr2=0x7f36b00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.701+0000 7f36b8740640 1 -- 172.21.15.40:0/1688986102 shutdown_connections 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.701+0000 7f36b8740640 1 -- 172.21.15.40:0/1688986102 wait complete. 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.703+0000 7f36b8740640 1 Processor -- start 2024-04-26T21:38:47.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.703+0000 7f36b8740640 1 -- start start 2024-04-26T21:38:47.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.704+0000 7f36b8740640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369c003fb0 0x7f36b015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.704+0000 7f36b8740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00c8b20 0x7f36b00cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.704+0000 7f36b8740640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b00b4ba0 0x7f36b00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.704+0000 7f36b8740640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36b015f900 con 0x7f36b00b4ba0 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.704+0000 7f36b8740640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36b015f780 con 0x7f369c003fb0 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.704+0000 7f36b8740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36b00d1e80 con 0x7f36b00c8b20 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b00b4ba0 0x7f36b00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b00b4ba0 0x7f36b00b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37032/0 (socket says 172.21.15.40:37032) 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 -- 172.21.15.40:0/2734675717 learned_addr learned my addr 172.21.15.40:0/2734675717 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b64dc640 1 --2- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369c003fb0 0x7f36b015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00c8b20 0x7f36b00cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 -- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00c8b20 msgr2=0x7f36b00cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b00c8b20 0x7f36b00cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 -- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369c003fb0 msgr2=0x7f36b015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369c003fb0 0x7f36b015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 -- 172.21.15.40:0/2734675717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b015fb20 con 0x7f36b00b4ba0 2024-04-26T21:38:47.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/2734675717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b00b4ba0 0x7f36b00b5030 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f36b0073b20 tx=0x7f36a800fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36a6ffd640 1 -- 172.21.15.40:0/2734675717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a801a070 con 0x7f36b00b4ba0 2024-04-26T21:38:47.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36a6ffd640 1 -- 172.21.15.40:0/2734675717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a8006d60 con 0x7f36b00b4ba0 2024-04-26T21:38:47.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36b8740640 1 -- 172.21.15.40:0/2734675717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b015fe00 con 0x7f36b00b4ba0 2024-04-26T21:38:47.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.705+0000 7f36a6ffd640 1 -- 172.21.15.40:0/2734675717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8003b90 con 0x7f36b00b4ba0 2024-04-26T21:38:47.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.706+0000 7f36b8740640 1 -- 172.21.15.40:0/2734675717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b01603c0 con 0x7f36b00b4ba0 2024-04-26T21:38:47.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.707+0000 7f36a6ffd640 1 -- 172.21.15.40:0/2734675717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36a801d070 con 0x7f36b00b4ba0 2024-04-26T21:38:47.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.708+0000 7f36a6ffd640 1 --2- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369406c8f0 0x7f369406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.708+0000 7f36a6ffd640 1 -- 172.21.15.40:0/2734675717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f36a808b2a0 con 0x7f36b00b4ba0 2024-04-26T21:38:47.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.708+0000 7f36b64dc640 1 --2- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369406c8f0 0x7f369406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.709+0000 7f36b64dc640 1 --2- 172.21.15.40:0/2734675717 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f369406c8f0 0x7f369406edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f36b0162ed0 tx=0x7f36ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.709+0000 7f36bb456740 1 Processor -- start 2024-04-26T21:38:47.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.709+0000 7f36bb456740 1 -- start start 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36bb456740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e33d1a0 0x56135e340920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36bb456740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e35cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36bb456740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e35d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36bb456740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56135e273880 con 0x56135e35d2e0 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36bb456740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56135e27ae10 con 0x56135e340e60 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36bb456740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56135e2167b0 con 0x56135e33d1a0 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36b6cdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e35d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36b6cdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e35d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37038/0 (socket says 172.21.15.40:37038) 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36b6cdd640 1 -- 172.21.15.40:0/4284639346 learned_addr learned my addr 172.21.15.40:0/4284639346 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e35cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.710+0000 7f36b64dc640 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e33d1a0 0x56135e340920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f36b6cdd640 1 -- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e33d1a0 msgr2=0x56135e340920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e33d1a0 0x56135e340920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f36b6cdd640 1 -- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 msgr2=0x56135e35cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e35cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f36b6cdd640 1 -- 172.21.15.40:0/4284639346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135e058a30 con 0x56135e35d2e0 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e35cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e35d740 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f36a8089520 tx=0x7f36a8058e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f09410674f8811 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f3693fff640 1 -- 172.21.15.40:0/4284639346 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8057190 con 0x56135e35d2e0 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f3693fff640 1 -- 172.21.15.40:0/4284639346 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a8055500 con 0x56135e35d2e0 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.711+0000 7f3693fff640 1 -- 172.21.15.40:0/4284639346 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8071320 con 0x56135e35d2e0 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 -- 172.21.15.40:0/4284639346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 msgr2=0x56135e35d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e35d740 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f36a8089520 tx=0x7f36a8058e90 comp rx=0 tx=0).stop 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 -- 172.21.15.40:0/4284639346 shutdown_connections 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e35d740 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e35cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 --2- 172.21.15.40:0/4284639346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e33d1a0 0x56135e340920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 -- 172.21.15.40:0/4284639346 >> 172.21.15.40:0/4284639346 conn(0x56135e313180 msgr2=0x56135e314180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 -- 172.21.15.40:0/4284639346 shutdown_connections 2024-04-26T21:38:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 -- 172.21.15.40:0/4284639346 wait complete. 2024-04-26T21:38:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.712+0000 7f36bb456740 1 -- start start 2024-04-26T21:38:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.713+0000 7f36bb456740 1 Processor -- start 2024-04-26T21:38:47.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.713+0000 7f36bb456740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e340e60 0x56135e342d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.713+0000 7f36bb456740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e35d2e0 0x56135e343280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.713+0000 7f36bb456740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e36b9b0 0x56135e357cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.713+0000 7f36bb456740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56135e214140 con 0x56135e36b9b0 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36bb456740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56135e276dd0 con 0x56135e35d2e0 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36bb456740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56135e310fb0 con 0x56135e340e60 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36bb456740 10 client.0 _ll_register_callbacks cb 0x56135e2003c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e36b9b0 0x56135e357cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e36b9b0 0x56135e357cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37042/0 (socket says 172.21.15.40:37042) 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 -- 172.21.15.40:0/551968769 learned_addr learned my addr 172.21.15.40:0/551968769 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b64dc640 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e340e60 0x56135e342d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e35d2e0 0x56135e343280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 -- 172.21.15.40:0/551968769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e340e60 msgr2=0x56135e342d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e340e60 0x56135e342d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 -- 172.21.15.40:0/551968769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e35d2e0 msgr2=0x56135e343280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e35d2e0 0x56135e343280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 -- 172.21.15.40:0/551968769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135df8c620 con 0x56135e36b9b0 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e36b9b0 0x56135e357cc0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f36a8000f80 tx=0x7f36a8017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82d347fbef1250d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36917fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36917fa640 1 -- 172.21.15.40:0/551968769 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a80892b0 con 0x56135e36b9b0 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36917fa640 1 -- 172.21.15.40:0/551968769 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f36a8078220 con 0x56135e36b9b0 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36bb456740 1 -- 172.21.15.40:0/551968769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a8055500 con 0x56135e36b9b0 2024-04-26T21:38:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36917fa640 1 -- 172.21.15.40:0/551968769 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f36a8080050 con 0x56135e36b9b0 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.714+0000 7f36bb456740 10 client.6909 fetch_fsmap learned FSMap version 57 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36bb456740 1 -- 172.21.15.40:0/551968769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x56135df4c320 con 0x56135e36b9b0 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36917fa640 1 -- 172.21.15.40:0/551968769 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a8055500 con 0x56135e36b9b0 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36917fa640 1 -- 172.21.15.40:0/551968769 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f36a80582e0 con 0x56135e36b9b0 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36bb456740 10 client.6909 fetch_fsmap finished waiting for FSMap version 57 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36bb456740 10 client.6909 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36bb456740 20 client.6909 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36bb456740 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e3764e0 0x56135e3788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36bb456740 4 client.6909 mds_command: new command op to 6840 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36bb456740 1 -- 172.21.15.40:0/551968769 --> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56135e288a20 con 0x56135e3764e0 2024-04-26T21:38:47.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36b64dc640 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e3764e0 0x56135e3788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36b64dc640 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e3764e0 0x56135e3788d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7dc52c2a40d2347 server_cookie=a3ebeceb1c717243 in_seq=0 out_seq=0 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.715+0000 7f36917fa640 10 client.6909 ms_handle_connect on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.716+0000 7f36917fa640 1 -- 172.21.15.40:0/551968769 <== mds.0 v2:172.21.15.40:6832/1699369057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56135e288a20 con 0x56135e3764e0 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.716+0000 7f36917fa640 10 client.6909 handle_command_reply: tid=0 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.716+0000 7f36bb456740 1 client.6909 shutdown 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.716+0000 7f36bb456740 1 -- 172.21.15.40:0/551968769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e36b9b0 msgr2=0x56135e357cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.716+0000 7f36bb456740 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e36b9b0 0x56135e357cc0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f36a8000f80 tx=0x7f36a8017040 comp rx=0 tx=0).stop 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 1 -- 172.21.15.40:0/551968769 shutdown_connections 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e3764e0 0x56135e3788d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e36b9b0 0x56135e357cc0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e35d2e0 0x56135e343280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 1 --2- 172.21.15.40:0/551968769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e340e60 0x56135e342d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 1 -- 172.21.15.40:0/551968769 >> 172.21.15.40:0/551968769 conn(0x56135e33d1a0 msgr2=0x56135e30fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36917fa640 0 client.6909 ms_handle_reset on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 1 -- 172.21.15.40:0/551968769 shutdown_connections 2024-04-26T21:38:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 1 -- 172.21.15.40:0/551968769 wait complete. 2024-04-26T21:38:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.717+0000 7f36bb456740 20 client.6909 trim_cache size 0 max 16384 2024-04-26T21:38:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.722+0000 7f06b4499640 1 Processor -- start 2024-04-26T21:38:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.722+0000 7f06b4499640 1 -- start start 2024-04-26T21:38:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b4499640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f06ac160710 0x7f06ac0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b4499640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06ac0d02d0 0x7f06ac0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b4499640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06ac0d1c70 con 0x7f06ac0cfc00 2024-04-26T21:38:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b4499640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f06ac070140 con 0x7f06ac0d02d0 2024-04-26T21:38:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b4499640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f06ac070370 con 0x7f06ac160710 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b1a34640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06ac0d02d0 0x7f06ac0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b1a34640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06ac0d02d0 0x7f06ac0cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37046/0 (socket says 172.21.15.40:37046) 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b1a34640 1 -- 172.21.15.40:0/2361743561 learned_addr learned my addr 172.21.15.40:0/2361743561 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.723+0000 7f06b2235640 1 --2- 172.21.15.40:0/2361743561 >> v2:172.21.15.184:3300/0 conn(0x7f06ac160710 0x7f06ac0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.724+0000 7f06b1a34640 1 -- 172.21.15.40:0/2361743561 >> v2:172.21.15.184:3300/0 conn(0x7f06ac160710 msgr2=0x7f06ac0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.724+0000 7f06b1a34640 1 --2- 172.21.15.40:0/2361743561 >> v2:172.21.15.184:3300/0 conn(0x7f06ac160710 0x7f06ac0d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.724+0000 7f06b1a34640 1 -- 172.21.15.40:0/2361743561 >> v1:172.21.15.184:6789/0 conn(0x7f06ac0cfc00 legacy=0x7f06ac0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.724+0000 7f06b1a34640 1 -- 172.21.15.40:0/2361743561 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac070610 con 0x7f06ac0d02d0 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.724+0000 7f06b1a34640 1 --2- 172.21.15.40:0/2361743561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d02d0 0x7f06ac0cf6c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f06ac076130 tx=0x7f069c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d904e57e3d84f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.725+0000 7f06b1233640 1 -- 172.21.15.40:0/2361743561 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c037070 con 0x7f06ac0d02d0 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.725+0000 7f06b1233640 1 -- 172.21.15.40:0/2361743561 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c032440 con 0x7f06ac0d02d0 2024-04-26T21:38:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.725+0000 7f06b1233640 1 -- 172.21.15.40:0/2361743561 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c032810 con 0x7f06ac0d02d0 2024-04-26T21:38:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.725+0000 7f06b4499640 1 -- 172.21.15.40:0/2361743561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d02d0 msgr2=0x7f06ac0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.725+0000 7f06b4499640 1 --2- 172.21.15.40:0/2361743561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d02d0 0x7f06ac0cf6c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f06ac076130 tx=0x7f069c02a270 comp rx=0 tx=0).stop 2024-04-26T21:38:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.726+0000 7f06b4499640 1 -- 172.21.15.40:0/2361743561 shutdown_connections 2024-04-26T21:38:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.726+0000 7f06b4499640 1 --2- 172.21.15.40:0/2361743561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d02d0 0x7f06ac0cf6c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.726+0000 7f06b4499640 1 --2- 172.21.15.40:0/2361743561 >> v2:172.21.15.184:3300/0 conn(0x7f06ac160710 0x7f06ac0d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.726+0000 7f06b4499640 1 -- 172.21.15.40:0/2361743561 >> 172.21.15.40:0/2361743561 conn(0x7f06ac0d2d80 msgr2=0x7f06ac0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.726+0000 7f06b4499640 1 -- 172.21.15.40:0/2361743561 shutdown_connections 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.726+0000 7f06b4499640 1 -- 172.21.15.40:0/2361743561 wait complete. 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.728+0000 7f06b4499640 1 Processor -- start 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.728+0000 7f06b4499640 1 -- start start 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b4499640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0cfc00 0x7f06ac0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b4499640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06ac0d02d0 0x7f06ac14c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b4499640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac160710 0x7f06ac15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b4499640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac0d1a20 con 0x7f06ac0cfc00 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b4499640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac0d18a0 con 0x7f06ac0d02d0 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b4499640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac071e20 con 0x7f06ac160710 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2235640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0cfc00 0x7f06ac0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2235640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0cfc00 0x7f06ac0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37060/0 (socket says 172.21.15.40:37060) 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2235640 1 -- 172.21.15.40:0/3174926586 learned_addr learned my addr 172.21.15.40:0/3174926586 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2a36640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac160710 0x7f06ac15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b1a34640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06ac0d02d0 0x7f06ac14c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2235640 1 -- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac160710 msgr2=0x7f06ac15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2235640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac160710 0x7f06ac15b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2235640 1 -- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06ac0d02d0 msgr2=0x7f06ac14c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2235640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06ac0d02d0 0x7f06ac14c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2235640 1 -- 172.21.15.40:0/3174926586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac0c7310 con 0x7f06ac0cfc00 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.729+0000 7f06b2a36640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac160710 0x7f06ac15b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.730+0000 7f06b2235640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0cfc00 0x7f06ac0c6dd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f06a800e870 tx=0x7f06a800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.730+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8017070 con 0x7f06ac0cfc00 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.730+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a800c940 con 0x7f06ac0cfc00 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.730+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a800cd30 con 0x7f06ac0cfc00 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.730+0000 7f06b4499640 1 -- 172.21.15.40:0/3174926586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac0c75b0 con 0x7f06ac0cfc00 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.730+0000 7f06b4499640 1 -- 172.21.15.40:0/3174926586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac0c7b70 con 0x7f06ac0cfc00 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.731+0000 7f06b4499640 1 -- 172.21.15.40:0/3174926586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f06ac069590 con 0x7f06ac0cfc00 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.732+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06a801a070 con 0x7f06ac0cfc00 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.732+0000 7f06a2ffd640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069006c8f0 0x7f069006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.732+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f06a808c480 con 0x7f06ac0cfc00 2024-04-26T21:38:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.732+0000 7f06b1a34640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069006c8f0 0x7f069006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.733+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+513 (secure 0 0 0) 0x7f06a808c850 con 0x7f06ac0cfc00 2024-04-26T21:38:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.733+0000 7f06b4499640 1 -- 172.21.15.40:0/3174926586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f06ac0ca5b0 con 0x7f06ac0cfc00 2024-04-26T21:38:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.733+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 7 ==== log(1 entries from seq 716 at 2024-04-26T21:38:46.731179+0000) ==== 309+0+0 (secure 0 0 0) 0x7f06a8057fd0 con 0x7f06ac0cfc00 2024-04-26T21:38:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.733+0000 7f06b1a34640 1 --2- 172.21.15.40:0/3174926586 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069006c8f0 0x7f069006edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f069c034000 tx=0x7f069c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36bb456740 1 Processor -- start 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36bb456740 1 -- start start 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36bb456740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e33a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36bb456740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e358b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36bb456740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e36b9b0 0x56135e3590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36b5cdb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e358b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36b5cdb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e358b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37064/0 (socket says 172.21.15.40:37064) 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36b5cdb640 1 -- 172.21.15.40:0/542462134 learned_addr learned my addr 172.21.15.40:0/542462134 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36bb456740 1 -- 172.21.15.40:0/542462134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36b02026d0 con 0x56135e35d2e0 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36bb456740 1 -- 172.21.15.40:0/542462134 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36b02016f0 con 0x56135e340e60 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.736+0000 7f36bb456740 1 -- 172.21.15.40:0/542462134 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36b0200a80 con 0x56135e36b9b0 2024-04-26T21:38:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36b64dc640 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e33a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e36b9b0 0x56135e3590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36b5cdb640 1 -- 172.21.15.40:0/542462134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e36b9b0 msgr2=0x56135e3590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e36b9b0 0x56135e3590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36b5cdb640 1 -- 172.21.15.40:0/542462134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 msgr2=0x56135e33a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e33a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36b5cdb640 1 -- 172.21.15.40:0/542462134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135e318210 con 0x56135e35d2e0 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e358b60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f36a002aae0 tx=0x7f36a002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c20e0dd74d0845 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36937fe640 1 -- 172.21.15.40:0/542462134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0036040 con 0x56135e35d2e0 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36937fe640 1 -- 172.21.15.40:0/542462134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a0004300 con 0x56135e35d2e0 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.737+0000 7f36937fe640 1 -- 172.21.15.40:0/542462134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0003610 con 0x56135e35d2e0 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 -- 172.21.15.40:0/542462134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 msgr2=0x56135e358b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e358b60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f36a002aae0 tx=0x7f36a002e000 comp rx=0 tx=0).stop 2024-04-26T21:38:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 -- 172.21.15.40:0/542462134 shutdown_connections 2024-04-26T21:38:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e36b9b0 0x56135e3590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e35d2e0 0x56135e358b60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 --2- 172.21.15.40:0/542462134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e340e60 0x56135e33a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 -- 172.21.15.40:0/542462134 >> 172.21.15.40:0/542462134 conn(0x56135e33d1a0 msgr2=0x56135e355ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 -- 172.21.15.40:0/542462134 shutdown_connections 2024-04-26T21:38:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.738+0000 7f36bb456740 1 -- 172.21.15.40:0/542462134 wait complete. 2024-04-26T21:38:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.739+0000 7f36bb456740 1 -- start start 2024-04-26T21:38:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.739+0000 7f36bb456740 1 Processor -- start 2024-04-26T21:38:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36bb456740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e340e60 0x56135e36e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36bb456740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e35d2e0 0x56135e3101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36bb456740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e36b9b0 0x56135e37c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36bb456740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36b02010a0 con 0x56135e340e60 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b64dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e340e60 0x56135e36e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36bb456740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56135e37d530 con 0x56135e36b9b0 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36bb456740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56135e37dd40 con 0x56135e35d2e0 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b64dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e340e60 0x56135e36e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37078/0 (socket says 172.21.15.40:37078) 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36bb456740 10 client.0 _ll_register_callbacks cb 0x7f36a8070040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b64dc640 1 -- 172.21.15.40:0/12978009 learned_addr learned my addr 172.21.15.40:0/12978009 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e36b9b0 0x56135e37c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e35d2e0 0x56135e3101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b64dc640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e35d2e0 msgr2=0x56135e3101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b64dc640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e35d2e0 0x56135e3101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b64dc640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e36b9b0 msgr2=0x56135e37c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b64dc640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e36b9b0 0x56135e37c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.740+0000 7f36b64dc640 1 -- 172.21.15.40:0/12978009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135e356550 con 0x56135e340e60 2024-04-26T21:38:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36b6cdd640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135e36b9b0 0x56135e37c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135e35d2e0 0x56135e3101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36b64dc640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135e340e60 0x56135e36e890 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x56135e3187a0 tx=0x7f36ac001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292bda4557601b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36917fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:38:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36917fa640 1 -- 172.21.15.40:0/12978009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac023800 con 0x56135e340e60 2024-04-26T21:38:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36917fa640 1 -- 172.21.15.40:0/12978009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f36ac0239a0 con 0x56135e340e60 2024-04-26T21:38:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 1 -- 172.21.15.40:0/12978009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a0004300 con 0x56135e340e60 2024-04-26T21:38:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36917fa640 1 -- 172.21.15.40:0/12978009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f36ac037050 con 0x56135e340e60 2024-04-26T21:38:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 10 client.6921 fetch_fsmap learned FSMap version 57 2024-04-26T21:38:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 1 -- 172.21.15.40:0/12978009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x56135e37d270 con 0x56135e340e60 2024-04-26T21:38:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36917fa640 1 -- 172.21.15.40:0/12978009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a0004300 con 0x56135e340e60 2024-04-26T21:38:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36917fa640 1 -- 172.21.15.40:0/12978009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f36ac03a020 con 0x56135e340e60 2024-04-26T21:38:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 10 client.6921 fetch_fsmap finished waiting for FSMap version 57 2024-04-26T21:38:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 10 client.6921 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-26T21:38:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 20 client.6921 populate_metadata read hostname 'smithi040' 2024-04-26T21:38:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 4 client.6921 mds_command: new command op to 6840 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T21:38:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36bb456740 1 -- 172.21.15.40:0/12978009 --> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56135e282890 con 0x56135e315f60 2024-04-26T21:38:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.741+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.742+0000 7f36917fa640 10 client.6921 ms_handle_connect on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.742+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc95ba5ec1a99d28 server_cookie=b0e194ed906d57c7 in_seq=0 out_seq=0 2024-04-26T21:38:47.752 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:38:47.744+0000 7f202cfea640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55663dec6f00] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:38:47.744+0000 7f202cfea640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55663dec6f00] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f202cfea640 time 2024-04-26T21:38:47.746129+0000 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f20395825ed] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55663ae6ec21] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55663b094514] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 4: ceph-mds(+0x3bc70c) [0x55663b0ae70c] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 5: (CDir::_omap_commit(int)+0x93a) [0x55663b0b003a] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55663b0a3875] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xae7) [0x55663aeb4977] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55663b1349e2] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 9: (MDLog::trim_all()+0x769) [0x55663b139619] 2024-04-26T21:38:47.753 INFO:tasks.ceph.mds.a.smithi040.stderr: 10: ceph-mds(+0x1b1e81) [0x55663aea3e81] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 11: ceph-mds(+0x15431d) [0x55663ae4631d] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 13: ceph-mds(+0x1545e2) [0x55663ae465e2] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 15: (MDSIOContextBase::complete(int)+0x31c) [0x55663b12e82c] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f2039630585] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 17: /lib64/libc.so.6(+0x89c02) [0x7f2038c89c02] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 18: /lib64/libc.so.6(+0x10ec40) [0x7f2038d0ec40] 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:38:47.745+0000 7f202cfea640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f202cfea640 time 2024-04-26T21:38:47.746129+0000 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.754 INFO:tasks.ceph.mds.a.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f20395825ed] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55663ae6ec21] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55663b094514] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 4: ceph-mds(+0x3bc70c) [0x55663b0ae70c] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 5: (CDir::_omap_commit(int)+0x93a) [0x55663b0b003a] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55663b0a3875] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xae7) [0x55663aeb4977] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55663b1349e2] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 9: (MDLog::trim_all()+0x769) [0x55663b139619] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 10: ceph-mds(+0x1b1e81) [0x55663aea3e81] 2024-04-26T21:38:47.755 INFO:tasks.ceph.mds.a.smithi040.stderr: 11: ceph-mds(+0x15431d) [0x55663ae4631d] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 13: ceph-mds(+0x1545e2) [0x55663ae465e2] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 15: (MDSIOContextBase::complete(int)+0x31c) [0x55663b12e82c] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f2039630585] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 17: /lib64/libc.so.6(+0x89c02) [0x7f2038c89c02] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 18: /lib64/libc.so.6(+0x10ec40) [0x7f2038d0ec40] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr:*** Caught signal (Aborted) ** 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: in thread 7f202cfea640 thread_name:MR_Finisher 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f2038c3e6f0] 2024-04-26T21:38:47.756 INFO:tasks.ceph.mds.a.smithi040.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f2038c8b94c] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 3: raise() 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 4: abort() 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7f20395826aa] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55663ae6ec21] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55663b094514] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 8: ceph-mds(+0x3bc70c) [0x55663b0ae70c] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 9: (CDir::_omap_commit(int)+0x93a) [0x55663b0b003a] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55663b0a3875] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xae7) [0x55663aeb4977] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55663b1349e2] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 13: (MDLog::trim_all()+0x769) [0x55663b139619] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 14: ceph-mds(+0x1b1e81) [0x55663aea3e81] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 15: ceph-mds(+0x15431d) [0x55663ae4631d] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.757 INFO:tasks.ceph.mds.a.smithi040.stderr: 17: ceph-mds(+0x1545e2) [0x55663ae465e2] 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55663b12e82c] 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f2039630585] 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 21: /lib64/libc.so.6(+0x89c02) [0x7f2038c89c02] 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 22: /lib64/libc.so.6(+0x10ec40) [0x7f2038d0ec40] 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:38:47.747+0000 7f202cfea640 -1 *** Caught signal (Aborted) ** 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: in thread 7f202cfea640 thread_name:MR_Finisher 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f2038c3e6f0] 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f2038c8b94c] 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 3: raise() 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 4: abort() 2024-04-26T21:38:47.758 INFO:tasks.ceph.mds.a.smithi040.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7f20395826aa] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55663ae6ec21] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55663b094514] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 8: ceph-mds(+0x3bc70c) [0x55663b0ae70c] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 9: (CDir::_omap_commit(int)+0x93a) [0x55663b0b003a] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55663b0a3875] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xae7) [0x55663aeb4977] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55663b1349e2] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 13: (MDLog::trim_all()+0x769) [0x55663b139619] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 14: ceph-mds(+0x1b1e81) [0x55663aea3e81] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 15: ceph-mds(+0x15431d) [0x55663ae4631d] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 17: ceph-mds(+0x1545e2) [0x55663ae465e2] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55663b12e82c] 2024-04-26T21:38:47.759 INFO:tasks.ceph.mds.a.smithi040.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f2039630585] 2024-04-26T21:38:47.760 INFO:tasks.ceph.mds.a.smithi040.stderr: 21: /lib64/libc.so.6(+0x89c02) [0x7f2038c89c02] 2024-04-26T21:38:47.760 INFO:tasks.ceph.mds.a.smithi040.stderr: 22: /lib64/libc.so.6(+0x10ec40) [0x7f2038d0ec40] 2024-04-26T21:38:47.760 INFO:tasks.ceph.mds.a.smithi040.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T21:38:47.760 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.760 INFO:teuthology.orchestra.run.smithi040.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-26T21:38:47.761 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-26T21:38:47.761 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:47.762 INFO:tasks.ceph.mds.a.smithi040.stderr: -188> 2024-04-26T21:38:46.534+0000 7f2030ff2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:38:47.762 INFO:tasks.ceph.mds.a.smithi040.stderr: -10> 2024-04-26T21:38:47.744+0000 7f202cfea640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55663dec6f00] 2024-04-26T21:38:47.762 INFO:tasks.ceph.mds.a.smithi040.stderr: -8> 2024-04-26T21:38:47.744+0000 7f202cfea640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55663dec6f00] 2024-04-26T21:38:47.762 INFO:tasks.ceph.mds.a.smithi040.stderr: -2> 2024-04-26T21:38:47.745+0000 7f202cfea640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f202cfea640 time 2024-04-26T21:38:47.746129+0000 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f20395825ed] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55663ae6ec21] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55663b094514] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 4: ceph-mds(+0x3bc70c) [0x55663b0ae70c] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 5: (CDir::_omap_commit(int)+0x93a) [0x55663b0b003a] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55663b0a3875] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xae7) [0x55663aeb4977] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55663b1349e2] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 9: (MDLog::trim_all()+0x769) [0x55663b139619] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 10: ceph-mds(+0x1b1e81) [0x55663aea3e81] 2024-04-26T21:38:47.763 INFO:tasks.ceph.mds.a.smithi040.stderr: 11: ceph-mds(+0x15431d) [0x55663ae4631d] 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 13: ceph-mds(+0x1545e2) [0x55663ae465e2] 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 15: (MDSIOContextBase::complete(int)+0x31c) [0x55663b12e82c] 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f2039630585] 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 17: /lib64/libc.so.6(+0x89c02) [0x7f2038c89c02] 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 18: /lib64/libc.so.6(+0x10ec40) [0x7f2038d0ec40] 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 0> 2024-04-26T21:38:47.747+0000 7f202cfea640 -1 *** Caught signal (Aborted) ** 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: in thread 7f202cfea640 thread_name:MR_Finisher 2024-04-26T21:38:47.764 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f2038c3e6f0] 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f2038c8b94c] 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 3: raise() 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 4: abort() 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7f20395826aa] 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55663ae6ec21] 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55663b094514] 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 8: ceph-mds(+0x3bc70c) [0x55663b0ae70c] 2024-04-26T21:38:47.765 INFO:tasks.ceph.mds.a.smithi040.stderr: 9: (CDir::_omap_commit(int)+0x93a) [0x55663b0b003a] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55663b0a3875] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xae7) [0x55663aeb4977] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55663b1349e2] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 13: (MDLog::trim_all()+0x769) [0x55663b139619] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 14: ceph-mds(+0x1b1e81) [0x55663aea3e81] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 15: ceph-mds(+0x15431d) [0x55663ae4631d] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 17: ceph-mds(+0x1545e2) [0x55663ae465e2] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55663b12e82c] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f2039630585] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 21: /lib64/libc.so.6(+0x89c02) [0x7f2038c89c02] 2024-04-26T21:38:47.766 INFO:tasks.ceph.mds.a.smithi040.stderr: 22: /lib64/libc.so.6(+0x10ec40) [0x7f2038d0ec40] 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: -189> 2024-04-26T21:38:46.534+0000 7f2030ff2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: -11> 2024-04-26T21:38:47.744+0000 7f202cfea640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55663dec6f00] 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: -9> 2024-04-26T21:38:47.744+0000 7f202cfea640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55663dec6f00] 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: -3> 2024-04-26T21:38:47.745+0000 7f202cfea640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f202cfea640 time 2024-04-26T21:38:47.746129+0000 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f20395825ed] 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55663ae6ec21] 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55663b094514] 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: 4: ceph-mds(+0x3bc70c) [0x55663b0ae70c] 2024-04-26T21:38:47.767 INFO:tasks.ceph.mds.a.smithi040.stderr: 5: (CDir::_omap_commit(int)+0x93a) [0x55663b0b003a] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55663b0a3875] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xae7) [0x55663aeb4977] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55663b1349e2] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 9: (MDLog::trim_all()+0x769) [0x55663b139619] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 10: ceph-mds(+0x1b1e81) [0x55663aea3e81] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 11: ceph-mds(+0x15431d) [0x55663ae4631d] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 13: ceph-mds(+0x1545e2) [0x55663ae465e2] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 15: (MDSIOContextBase::complete(int)+0x31c) [0x55663b12e82c] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f2039630585] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 17: /lib64/libc.so.6(+0x89c02) [0x7f2038c89c02] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 18: /lib64/libc.so.6(+0x10ec40) [0x7f2038d0ec40] 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.768 INFO:tasks.ceph.mds.a.smithi040.stderr: -1> 2024-04-26T21:38:47.747+0000 7f202cfea640 -1 *** Caught signal (Aborted) ** 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: in thread 7f202cfea640 thread_name:MR_Finisher 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f2038c3e6f0] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f2038c8b94c] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 3: raise() 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 4: abort() 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7f20395826aa] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55663ae6ec21] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55663b094514] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 8: ceph-mds(+0x3bc70c) [0x55663b0ae70c] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 9: (CDir::_omap_commit(int)+0x93a) [0x55663b0b003a] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55663b0a3875] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xae7) [0x55663aeb4977] 2024-04-26T21:38:47.769 INFO:tasks.ceph.mds.a.smithi040.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55663b1349e2] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 13: (MDLog::trim_all()+0x769) [0x55663b139619] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 14: ceph-mds(+0x1b1e81) [0x55663aea3e81] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 15: ceph-mds(+0x15431d) [0x55663ae4631d] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 17: ceph-mds(+0x1545e2) [0x55663ae465e2] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55663b12e3dc] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55663b12e82c] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f2039630585] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 21: /lib64/libc.so.6(+0x89c02) [0x7f2038c89c02] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 22: /lib64/libc.so.6(+0x10ec40) [0x7f2038d0ec40] 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T21:38:47.770 INFO:tasks.ceph.mds.a.smithi040.stderr: 2024-04-26T21:38:47.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.772+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-04-26T21:38:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.772+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:38:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.772+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:38:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.773+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T21:38:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.773+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.773+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=14 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:38:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.773+0000 7f36917fa640 1 client.6921 ms_handle_refused on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:47.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.872+0000 7f0324dda640 1 Processor -- start 2024-04-26T21:38:47.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.872+0000 7f0324dda640 1 -- start start 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.872+0000 7f0324dda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f03200d06f0 0x7f03200d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.872+0000 7f0324dda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03200b2850 0x7f03200b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f0324dda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03200d0ba0 con 0x7f032015f3a0 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f0324dda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f03200d0e20 con 0x7f03200b2850 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f0324dda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f03200b1d70 con 0x7f03200d06f0 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03200b2850 0x7f03200b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f032015f3a0 0x7f03200d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44076/0 (socket says 172.21.15.40:44076) 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031e59c640 1 -- 172.21.15.40:0/1814864497 learned_addr learned my addr 172.21.15.40:0/1814864497 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031ed9d640 1 -- 172.21.15.40:0/1814864497 >> v2:172.21.15.184:3300/0 conn(0x7f03200d06f0 msgr2=0x7f03200d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031dd9b640 1 --2- 172.21.15.40:0/1814864497 >> v2:172.21.15.184:3300/0 conn(0x7f03200d06f0 0x7f03200d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031d59a640 1 -- 172.21.15.40:0/1814864497 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785420409 0 0) 0x7f0310001110 con 0x7f032015f3a0 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031ed9d640 1 --2- 172.21.15.40:0/1814864497 >> v2:172.21.15.184:3300/0 conn(0x7f03200d06f0 0x7f03200d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031ed9d640 1 -- 172.21.15.40:0/1814864497 >> v1:172.21.15.40:6789/0 conn(0x7f032015f3a0 legacy=0x7f03200d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031ed9d640 1 -- 172.21.15.40:0/1814864497 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03200b2050 con 0x7f03200b2850 2024-04-26T21:38:47.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.873+0000 7f031dd9b640 1 --2- 172.21.15.40:0/1814864497 >> v2:172.21.15.184:3300/0 conn(0x7f03200d06f0 0x7f03200d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:38:47.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.874+0000 7f031ed9d640 1 --2- 172.21.15.40:0/1814864497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200b2850 0x7f03200b2c80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f03200b7b80 tx=0x7f031802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256bb86670806ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.874+0000 7f031d59a640 1 -- 172.21.15.40:0/1814864497 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318007740 con 0x7f03200b2850 2024-04-26T21:38:47.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.874+0000 7f031d59a640 1 -- 172.21.15.40:0/1814864497 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03180078e0 con 0x7f03200b2850 2024-04-26T21:38:47.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.874+0000 7f031d59a640 1 -- 172.21.15.40:0/1814864497 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318007cb0 con 0x7f03200b2850 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.874+0000 7f0324dda640 1 -- 172.21.15.40:0/1814864497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200b2850 msgr2=0x7f03200b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.874+0000 7f0324dda640 1 --2- 172.21.15.40:0/1814864497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200b2850 0x7f03200b2c80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f03200b7b80 tx=0x7f031802a680 comp rx=0 tx=0).stop 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.875+0000 7f0324dda640 1 -- 172.21.15.40:0/1814864497 shutdown_connections 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.875+0000 7f0324dda640 1 --2- 172.21.15.40:0/1814864497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200b2850 0x7f03200b2c80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.875+0000 7f0324dda640 1 --2- 172.21.15.40:0/1814864497 >> v2:172.21.15.184:3300/0 conn(0x7f03200d06f0 0x7f03200d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.875+0000 7f0324dda640 1 -- 172.21.15.40:0/1814864497 >> 172.21.15.40:0/1814864497 conn(0x7f03200d4f90 msgr2=0x7f03200d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.875+0000 7f0324dda640 1 -- 172.21.15.40:0/1814864497 shutdown_connections 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.875+0000 7f0324dda640 1 -- 172.21.15.40:0/1814864497 wait complete. 2024-04-26T21:38:47.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.876+0000 7f0324dda640 1 Processor -- start 2024-04-26T21:38:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.876+0000 7f0324dda640 1 -- start start 2024-04-26T21:38:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f0324dda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200b2850 0x7f032014ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f0324dda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d06f0 0x7f03200b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f0324dda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f032015f3a0 0x7f03200b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f0324dda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f032015f900 con 0x7f032015f3a0 2024-04-26T21:38:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f0324dda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f032015f780 con 0x7f03200d06f0 2024-04-26T21:38:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f0324dda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03200d23c0 con 0x7f03200b2850 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f032015f3a0 0x7f03200b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f032015f3a0 0x7f03200b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37098/0 (socket says 172.21.15.40:37098) 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 -- 172.21.15.40:0/1041102800 learned_addr learned my addr 172.21.15.40:0/1041102800 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031e59c640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200b2850 0x7f032014ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031dd9b640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d06f0 0x7f03200b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 -- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200b2850 msgr2=0x7f032014ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200b2850 0x7f032014ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 -- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d06f0 msgr2=0x7f03200b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d06f0 0x7f03200b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 -- 172.21.15.40:0/1041102800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03200b9790 con 0x7f032015f3a0 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f031ed9d640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f032015f3a0 0x7f03200b80c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f031802ac70 tx=0x7f0318004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f030effd640 1 -- 172.21.15.40:0/1041102800 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318040070 con 0x7f032015f3a0 2024-04-26T21:38:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f030effd640 1 -- 172.21.15.40:0/1041102800 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0318031c90 con 0x7f032015f3a0 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03200b86f0 con 0x7f032015f3a0 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.877+0000 7f030effd640 1 -- 172.21.15.40:0/1041102800 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318045680 con 0x7f032015f3a0 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.878+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03200b8cb0 con 0x7f032015f3a0 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.878+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03200694f0 con 0x7f032015f3a0 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.879+0000 7f030effd640 1 -- 172.21.15.40:0/1041102800 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0318039040 con 0x7f032015f3a0 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.879+0000 7f030effd640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.879+0000 7f030effd640 1 -- 172.21.15.40:0/1041102800 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6187+0+0 (secure 0 0 0) 0x7f03180ad640 con 0x7f032015f3a0 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.879+0000 7f031e59c640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:47.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.880+0000 7f031e59c640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f030006c8f0 0x7f030006edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0310007fd0 tx=0x7f0310007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:47.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.885+0000 7f030effd640 1 -- 172.21.15.40:0/1041102800 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f031807b300 con 0x7f032015f3a0 2024-04-26T21:38:47.947 INFO:tasks.ceph.mds.a.smithi040.stderr:daemon-helper: command crashed with signal 6 2024-04-26T21:38:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.974+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.974+0000 7f36917fa640 1 client.6921 ms_handle_refused on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:47.974+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=14 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:38:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.108+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03201507b0 con 0x7f032015f3a0 2024-04-26T21:38:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.108+0000 7f030effd640 1 -- 172.21.15.40:0/1041102800 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5186 (secure 0 0 0) 0x7f031807f600 con 0x7f032015f3a0 2024-04-26T21:38:48.110 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:48.111 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:46.731288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6840,"qdb_cluster":[6840]},"id":4}]} 2024-04-26T21:38:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f030006c8f0 msgr2=0x7f030006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f030006c8f0 0x7f030006edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0310007fd0 tx=0x7f0310007de0 comp rx=0 tx=0).stop 2024-04-26T21:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f032015f3a0 msgr2=0x7f03200b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f032015f3a0 0x7f03200b80c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f031802ac70 tx=0x7f0318004a40 comp rx=0 tx=0).stop 2024-04-26T21:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 shutdown_connections 2024-04-26T21:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f032015f3a0 0x7f03200b80c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d06f0 0x7f03200b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 --2- 172.21.15.40:0/1041102800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200b2850 0x7f032014ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 >> 172.21.15.40:0/1041102800 conn(0x7f03200d4f90 msgr2=0x7f03200b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 shutdown_connections 2024-04-26T21:38:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.110+0000 7f0324dda640 1 -- 172.21.15.40:0/1041102800 wait complete. 2024-04-26T21:38:48.123 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T21:38:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.375+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.375+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=14 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:38:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.375+0000 7f36917fa640 1 client.6921 ms_handle_refused on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:48.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:48.571+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 8 ==== log(2 entries from seq 717 at 2024-04-26T21:38:47.566968+0000) ==== 635+0+0 (secure 0 0 0) 0x7f06a8058230 con 0x7f06ac0cfc00 2024-04-26T21:38:48.572 INFO:teuthology.orchestra.run.smithi040.stdout: cluster: 2024-04-26T21:38:48.572 INFO:teuthology.orchestra.run.smithi040.stdout: id: ec445c36-0c6e-44d6-884b-6999dea32f55 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: health: HEALTH_OK 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: services: 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: mon: 3 daemons, quorum a,b,c (age 4m) 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: mgr: x(active, since 3m), standbys: y 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: mds: 1/1 daemons up, 3 standby 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: data: 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: volumes: 1/1 healthy 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: pools: 3 pools, 97 pgs 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: objects: 29 objects, 602 KiB 2024-04-26T21:38:48.573 INFO:teuthology.orchestra.run.smithi040.stdout: usage: 223 MiB used, 720 GiB / 720 GiB avail 2024-04-26T21:38:48.574 INFO:teuthology.orchestra.run.smithi040.stdout: pgs: 97 active+clean 2024-04-26T21:38:48.574 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:48.574 INFO:teuthology.orchestra.run.smithi040.stdout: io: 2024-04-26T21:38:48.574 INFO:teuthology.orchestra.run.smithi040.stdout: client: 2.1 KiB/s rd, 876 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-26T21:38:48.574 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:48.574 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:48.574 INFO:teuthology.orchestra.run.smithi040.stdout:2024-04-26T21:38:47.746104+0000 mds.a [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55663dec6f00] 2024-04-26T21:38:49.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:49.176+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:49.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:49.176+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=14 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:38:49.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:49.176+0000 7f36917fa640 1 client.6921 ms_handle_refused on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:49.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-26T21:38:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:50.778+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:50.778+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=14 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:38:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:50.778+0000 7f36917fa640 1 client.6921 ms_handle_refused on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:51.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:51.583+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 9 ==== log(8 entries from seq 130 at 2024-04-26T21:38:51.300162+0000) ==== 2210+0+0 (secure 0 0 0) 0x7f06a808a840 con 0x7f06ac0cfc00 2024-04-26T21:38:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:52.596+0000 7f06a2ffd640 1 -- 172.21.15.40:0/3174926586 <== mon.0 v2:172.21.15.40:3300/0 10 ==== log(4 entries from seq 725 at 2024-04-26T21:38:51.576562+0000) ==== 1080+0+0 (secure 0 0 0) 0x7f06a80565a0 con 0x7f06ac0cfc00 2024-04-26T21:38:53.124 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:53.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.238+0000 7f8b31311640 1 Processor -- start 2024-04-26T21:38:53.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.238+0000 7f8b31311640 1 -- start start 2024-04-26T21:38:53.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.239+0000 7f8b31311640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8b2c163610 0x7f8b2c0d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:53.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.239+0000 7f8b31311640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2c0d1a70 con 0x7f8b2c0d2830 2024-04-26T21:38:53.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.239+0000 7f8b31311640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2c0d1d50 con 0x7f8b2c15f5a0 2024-04-26T21:38:53.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.239+0000 7f8b31311640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8b2c163d30 con 0x7f8b2c163610 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.239+0000 7f8b2b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8b2c0d2830 0x7f8b2c0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42618/0 (socket says 172.21.15.40:42618) 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.239+0000 7f8b2b7fe640 1 -- 172.21.15.40:0/907952523 learned_addr learned my addr 172.21.15.40:0/907952523 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.240+0000 7f8b2a7fc640 1 --2- 172.21.15.40:0/907952523 >> v2:172.21.15.184:3300/0 conn(0x7f8b2c163610 0x7f8b2c0d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.240+0000 7f8b29ffb640 1 -- 172.21.15.40:0/907952523 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250844508 0 0) 0x7f8b2c0d1a70 con 0x7f8b2c0d2830 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.240+0000 7f8b29ffb640 1 -- 172.21.15.40:0/907952523 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b14003520 con 0x7f8b2c0d2830 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.240+0000 7f8b2a7fc640 1 -- 172.21.15.40:0/907952523 >> v1:172.21.15.184:6789/0 conn(0x7f8b2c15f5a0 legacy=0x7f8b2c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.240+0000 7f8b2a7fc640 1 -- 172.21.15.40:0/907952523 >> v1:172.21.15.40:6789/0 conn(0x7f8b2c0d2830 legacy=0x7f8b2c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.240+0000 7f8b29ffb640 1 -- 172.21.15.40:0/907952523 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2411382873 0 0) 0x7f8b2c0d1d50 con 0x7f8b2c15f5a0 2024-04-26T21:38:53.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.240+0000 7f8b2a7fc640 1 -- 172.21.15.40:0/907952523 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c0d2030 con 0x7f8b2c163610 2024-04-26T21:38:53.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.241+0000 7f8b2a7fc640 1 --2- 172.21.15.40:0/907952523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c163610 0x7f8b2c0d5cb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8b200097a0 tx=0x7f8b2002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe64fc10b63166ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:53.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.241+0000 7f8b29ffb640 1 -- 172.21.15.40:0/907952523 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20037070 con 0x7f8b2c163610 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.241+0000 7f8b29ffb640 1 -- 172.21.15.40:0/907952523 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b20032440 con 0x7f8b2c163610 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.241+0000 7f8b29ffb640 1 -- 172.21.15.40:0/907952523 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20032810 con 0x7f8b2c163610 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.241+0000 7f8b31311640 1 -- 172.21.15.40:0/907952523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c163610 msgr2=0x7f8b2c0d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.241+0000 7f8b31311640 1 --2- 172.21.15.40:0/907952523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c163610 0x7f8b2c0d5cb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8b200097a0 tx=0x7f8b2002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.241+0000 7f8b31311640 1 -- 172.21.15.40:0/907952523 shutdown_connections 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.242+0000 7f8b31311640 1 --2- 172.21.15.40:0/907952523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c163610 0x7f8b2c0d5cb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.242+0000 7f8b31311640 1 -- 172.21.15.40:0/907952523 >> 172.21.15.40:0/907952523 conn(0x7f8b2c0d5190 msgr2=0x7f8b2c0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.242+0000 7f8b31311640 1 -- 172.21.15.40:0/907952523 shutdown_connections 2024-04-26T21:38:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.242+0000 7f8b31311640 1 -- 172.21.15.40:0/907952523 wait complete. 2024-04-26T21:38:53.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.243+0000 7f8b31311640 1 Processor -- start 2024-04-26T21:38:53.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.243+0000 7f8b31311640 1 -- start start 2024-04-26T21:38:53.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b31311640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c0d2830 0x7f8b2c1f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:53.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b31311640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b2c163610 0x7f8b2c1f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:53.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b31311640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2c0c8c80 0x7f8b2c0ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:53.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b31311640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c0d3c20 con 0x7f8b2c163610 2024-04-26T21:38:53.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b31311640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c0d4190 con 0x7f8b2c0d2830 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b31311640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c0d3da0 con 0x7f8b2c0c8c80 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b2c163610 0x7f8b2c1f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b2c163610 0x7f8b2c1f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51532/0 (socket says 172.21.15.40:51532) 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 -- 172.21.15.40:0/4082756638 learned_addr learned my addr 172.21.15.40:0/4082756638 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2affd640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c0d2830 0x7f8b2c1f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2b7fe640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2c0c8c80 0x7f8b2c0ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 -- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2c0c8c80 msgr2=0x7f8b2c0ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2c0c8c80 0x7f8b2c0ba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 -- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c0d2830 msgr2=0x7f8b2c1f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c0d2830 0x7f8b2c1f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 -- 172.21.15.40:0/4082756638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c0cac70 con 0x7f8b2c163610 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2affd640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c0d2830 0x7f8b2c1f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:38:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.244+0000 7f8b2a7fc640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b2c163610 0x7f8b2c1f2430 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8b20000c00 tx=0x7f8b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.245+0000 7f8b137fe640 1 -- 172.21.15.40:0/4082756638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20037050 con 0x7f8b2c163610 2024-04-26T21:38:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.245+0000 7f8b137fe640 1 -- 172.21.15.40:0/4082756638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b20043070 con 0x7f8b2c163610 2024-04-26T21:38:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.245+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c1f2970 con 0x7f8b2c163610 2024-04-26T21:38:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.245+0000 7f8b137fe640 1 -- 172.21.15.40:0/4082756638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2003cb10 con 0x7f8b2c163610 2024-04-26T21:38:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.245+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c0c94b0 con 0x7f8b2c163610 2024-04-26T21:38:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.246+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c0694f0 con 0x7f8b2c163610 2024-04-26T21:38:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.246+0000 7f8b137fe640 1 -- 172.21.15.40:0/4082756638 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b2003ccb0 con 0x7f8b2c163610 2024-04-26T21:38:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.246+0000 7f8b137fe640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b1806c8f0 0x7f8b1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.247+0000 7f8b137fe640 1 -- 172.21.15.40:0/4082756638 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6303+0+0 (secure 0 0 0) 0x7f8b200acb10 con 0x7f8b2c163610 2024-04-26T21:38:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.247+0000 7f8b2affd640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b1806c8f0 0x7f8b1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.247+0000 7f8b2affd640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b1806c8f0 0x7f8b1806edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8b24000f60 tx=0x7f8b24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:53.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.251+0000 7f8b137fe640 1 -- 172.21.15.40:0/4082756638 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b2c0cac70 con 0x7f8b2c163610 2024-04-26T21:38:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.474+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b2c068d30 con 0x7f8b2c163610 2024-04-26T21:38:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.474+0000 7f8b137fe640 1 -- 172.21.15.40:0/4082756638 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5186 (secure 0 0 0) 0x7f8b20046360 con 0x7f8b2c163610 2024-04-26T21:38:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:53.475 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:46.731288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6840,"qdb_cluster":[6840]},"id":4}]} 2024-04-26T21:38:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b1806c8f0 msgr2=0x7f8b1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b1806c8f0 0x7f8b1806edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8b24000f60 tx=0x7f8b24009040 comp rx=0 tx=0).stop 2024-04-26T21:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b2c163610 msgr2=0x7f8b2c1f2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b2c163610 0x7f8b2c1f2430 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8b20000c00 tx=0x7f8b200046c0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 shutdown_connections 2024-04-26T21:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b1806c8f0 0x7f8b1806edb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2c0c8c80 0x7f8b2c0ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b2c163610 0x7f8b2c1f2430 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 --2- 172.21.15.40:0/4082756638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b2c0d2830 0x7f8b2c1f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 >> 172.21.15.40:0/4082756638 conn(0x7f8b2c0d5190 msgr2=0x7f8b2c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 shutdown_connections 2024-04-26T21:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.476+0000 7f8b31311640 1 -- 172.21.15.40:0/4082756638 wait complete. 2024-04-26T21:38:53.487 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T21:38:53.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.980+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:53.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.980+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=14 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-26T21:38:53.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:53.980+0000 7f36917fa640 1 client.6921 ms_handle_refused on v2:172.21.15.40:6832/1699369057 2024-04-26T21:38:56.463 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-04-26T21:38:58.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:38:58.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.602+0000 7f63be1da640 1 Processor -- start 2024-04-26T21:38:58.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.602+0000 7f63be1da640 1 -- start start 2024-04-26T21:38:58.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63be1da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63b8162880 0x7f63b80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:58.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63be1da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b80b7e50 con 0x7f63b80d3bc0 2024-04-26T21:38:58.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63be1da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b80d1800 con 0x7f63b80b5f20 2024-04-26T21:38:58.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63be1da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63b80d0580 con 0x7f63b8162880 2024-04-26T21:38:58.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63b80d3bc0 0x7f63b80b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42628/0 (socket says 172.21.15.40:42628) 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b7fff640 1 -- 172.21.15.40:0/2372240650 learned_addr learned my addr 172.21.15.40:0/2372240650 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b77fe640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63b8162880 0x7f63b80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3332450169 0 0) 0x7f63b80b7e50 con 0x7f63b80d3bc0 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63a0003520 con 0x7f63b80d3bc0 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1950059125 0 0) 0x7f63a0003520 con 0x7f63b80d3bc0 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63b8162880 msgr2=0x7f63b80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63b8162880 0x7f63b80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 >> v1:172.21.15.184:6789/0 conn(0x7f63b80b5f20 legacy=0x7f63b80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b80d07d0 con 0x7f63b80d3bc0 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f63b00019f0 con 0x7f63b80d3bc0 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.603+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 >> v1:172.21.15.40:6789/0 conn(0x7f63b80d3bc0 legacy=0x7f63b80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b67fc640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a00039e0 0x7f63a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b67fc640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a0004800 0x7f63a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:58.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b67fc640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63b8162880 0x7f63a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f63b80d0580 con 0x7f63a00039e0 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63a000ba70 con 0x7f63a0004800 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63a000bc70 con 0x7f63b8162880 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b77fe640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a00039e0 0x7f63a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b7fff640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63b8162880 0x7f63a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b6ffd640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a0004800 0x7f63a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b77fe640 1 -- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63b8162880 msgr2=0x7f63a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b77fe640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63b8162880 0x7f63a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b77fe640 1 -- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a0004800 msgr2=0x7f63a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b77fe640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a0004800 0x7f63a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.604+0000 7f63b77fe640 1 -- 172.21.15.40:0/2372240650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a0003520 con 0x7f63a00039e0 2024-04-26T21:38:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.605+0000 7f63b77fe640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a00039e0 0x7f63a0003e00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009a90 tx=0x7f63a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0988845d9d69cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac036070 con 0x7f63a00039e0 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac031400 con 0x7f63a00039e0 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63b67fc640 1 -- 172.21.15.40:0/2372240650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac009690 con 0x7f63a00039e0 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63be1da640 1 -- 172.21.15.40:0/2372240650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a00039e0 msgr2=0x7f63a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63be1da640 1 --2- 172.21.15.40:0/2372240650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a00039e0 0x7f63a0003e00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009a90 tx=0x7f63a0005830 comp rx=0 tx=0).stop 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63b7fff640 1 -- 172.21.15.40:0/2372240650 reap_dead start 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63be1da640 1 -- 172.21.15.40:0/2372240650 shutdown_connections 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63be1da640 1 -- 172.21.15.40:0/2372240650 >> 172.21.15.40:0/2372240650 conn(0x7f63b80d4f60 msgr2=0x7f63b80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63be1da640 1 -- 172.21.15.40:0/2372240650 shutdown_connections 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.606+0000 7f63be1da640 1 -- 172.21.15.40:0/2372240650 wait complete. 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.608+0000 7f63be1da640 1 Processor -- start 2024-04-26T21:38:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.608+0000 7f63be1da640 1 -- start start 2024-04-26T21:38:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63be1da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a00039e0 0x7f63b81f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63be1da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63b80b3140 0x7f63b80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63be1da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b81605b0 0x7f63b815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63be1da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8162c60 con 0x7f63b81605b0 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63be1da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63b80d0d90 con 0x7f63b80b3140 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63be1da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63b8162de0 con 0x7f63a00039e0 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b81605b0 0x7f63b815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b81605b0 0x7f63b815c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51550/0 (socket says 172.21.15.40:51550) 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 -- 172.21.15.40:0/335424941 learned_addr learned my addr 172.21.15.40:0/335424941 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b77fe640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a00039e0 0x7f63b81f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b6ffd640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63b80b3140 0x7f63b80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 -- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a00039e0 msgr2=0x7f63b81f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a00039e0 0x7f63b81f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 -- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63b80b3140 msgr2=0x7f63b80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63b80b3140 0x7f63b80cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 -- 172.21.15.40:0/335424941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b81f9ea0 con 0x7f63b81605b0 2024-04-26T21:38:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63b7fff640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b81605b0 0x7f63b815c300 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f63b0000f80 tx=0x7f63b000a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f639ffff640 1 -- 172.21.15.40:0/335424941 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b000ad30 con 0x7f63b81605b0 2024-04-26T21:38:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f639ffff640 1 -- 172.21.15.40:0/335424941 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b0019070 con 0x7f63b81605b0 2024-04-26T21:38:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.609+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b81f9600 con 0x7f63b81605b0 2024-04-26T21:38:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.610+0000 7f639ffff640 1 -- 172.21.15.40:0/335424941 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0013630 con 0x7f63b81605b0 2024-04-26T21:38:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.610+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b81f9bc0 con 0x7f63b81605b0 2024-04-26T21:38:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.611+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b80694f0 con 0x7f63b81605b0 2024-04-26T21:38:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.611+0000 7f639ffff640 1 -- 172.21.15.40:0/335424941 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63b00137d0 con 0x7f63b81605b0 2024-04-26T21:38:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.611+0000 7f639ffff640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63a406c920 0x7f63a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:38:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.612+0000 7f639ffff640 1 -- 172.21.15.40:0/335424941 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6303+0+0 (secure 0 0 0) 0x7f63b008c3b0 con 0x7f63b81605b0 2024-04-26T21:38:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.612+0000 7f63b77fe640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63a406c920 0x7f63a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:38:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.612+0000 7f63b77fe640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63a406c920 0x7f63a406ede0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f63ac033000 tx=0x7f63ac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:38:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.616+0000 7f639ffff640 1 -- 172.21.15.40:0/335424941 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f63b005a000 con 0x7f63b81605b0 2024-04-26T21:38:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.839+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63b8069c90 con 0x7f63b81605b0 2024-04-26T21:38:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.840+0000 7f639ffff640 1 -- 172.21.15.40:0/335424941 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5186 (secure 0 0 0) 0x7f63b005e300 con 0x7f63b81605b0 2024-04-26T21:38:58.840 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:38:58.840 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:46.731288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6840,"qdb_cluster":[6840]},"id":4}]} 2024-04-26T21:38:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:38:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63a406c920 msgr2=0x7f63a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63a406c920 0x7f63a406ede0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f63ac033000 tx=0x7f63ac008030 comp rx=0 tx=0).stop 2024-04-26T21:38:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b81605b0 msgr2=0x7f63b815c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b81605b0 0x7f63b815c300 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f63b0000f80 tx=0x7f63b000a730 comp rx=0 tx=0).stop 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 shutdown_connections 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63a406c920 0x7f63a406ede0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b81605b0 0x7f63b815c300 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63b80b3140 0x7f63b80cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 --2- 172.21.15.40:0/335424941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a00039e0 0x7f63b81f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 >> 172.21.15.40:0/335424941 conn(0x7f63b80d4f60 msgr2=0x7f63b80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 shutdown_connections 2024-04-26T21:38:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:38:58.842+0000 7f63be1da640 1 -- 172.21.15.40:0/335424941 wait complete. 2024-04-26T21:38:58.853 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T21:39:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:00.386+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:6832/1699369057 2024-04-26T21:39:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:00.386+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=14 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-04-26T21:39:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:00.386+0000 7f36917fa640 1 client.6921 ms_handle_refused on v2:172.21.15.40:6832/1699369057 2024-04-26T21:39:03.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-04-26T21:39:03.854 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:39:03.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.968+0000 7fe55972b640 1 Processor -- start 2024-04-26T21:39:03.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.968+0000 7fe55972b640 1 -- start start 2024-04-26T21:39:03.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.968+0000 7fe55972b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe554162880 0x7fe5540d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:03.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.968+0000 7fe55972b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 0x7fe5540b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:03.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.968+0000 7fe55972b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5540b7e50 con 0x7fe5540b5f20 2024-04-26T21:39:03.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.968+0000 7fe55972b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe5540d0580 con 0x7fe554162880 2024-04-26T21:39:03.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.968+0000 7fe55972b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5540d1800 con 0x7fe5540d3bc0 2024-04-26T21:39:03.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe552ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe554162880 0x7fe5540d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:03.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe5527fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe5540b5f20 0x7fe5540d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35154/0 (socket says 172.21.15.40:35154) 2024-04-26T21:39:03.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe552ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe554162880 0x7fe5540d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56034/0 (socket says 172.21.15.40:56034) 2024-04-26T21:39:03.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe5527fc640 1 -- 172.21.15.40:0/761381188 learned_addr learned my addr 172.21.15.40:0/761381188 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:39:03.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe5537fe640 1 --2- 172.21.15.40:0/761381188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 0x7fe5540b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:03.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe552ffd640 1 -- 172.21.15.40:0/761381188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 msgr2=0x7fe5540b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:03.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe552ffd640 1 --2- 172.21.15.40:0/761381188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 0x7fe5540b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:03.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe552ffd640 1 -- 172.21.15.40:0/761381188 >> v1:172.21.15.40:6789/0 conn(0x7fe5540b5f20 legacy=0x7fe5540d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:03.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.969+0000 7fe552ffd640 1 -- 172.21.15.40:0/761381188 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540d07d0 con 0x7fe554162880 2024-04-26T21:39:03.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe552ffd640 1 --2- 172.21.15.40:0/761381188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe554162880 0x7fe5540d2cb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0097a0 tx=0x7fe54c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b03349ef30167a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:03.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe551ffb640 1 -- 172.21.15.40:0/761381188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c037070 con 0x7fe554162880 2024-04-26T21:39:03.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe551ffb640 1 -- 172.21.15.40:0/761381188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54c032440 con 0x7fe554162880 2024-04-26T21:39:03.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe551ffb640 1 -- 172.21.15.40:0/761381188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c032810 con 0x7fe554162880 2024-04-26T21:39:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe55972b640 1 -- 172.21.15.40:0/761381188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe554162880 msgr2=0x7fe5540d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe55972b640 1 --2- 172.21.15.40:0/761381188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe554162880 0x7fe5540d2cb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0097a0 tx=0x7fe54c02a180 comp rx=0 tx=0).stop 2024-04-26T21:39:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe55972b640 1 -- 172.21.15.40:0/761381188 shutdown_connections 2024-04-26T21:39:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe55972b640 1 --2- 172.21.15.40:0/761381188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 0x7fe5540b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe55972b640 1 --2- 172.21.15.40:0/761381188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe554162880 0x7fe5540d2cb0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.970+0000 7fe55972b640 1 -- 172.21.15.40:0/761381188 >> 172.21.15.40:0/761381188 conn(0x7fe5540d4f60 msgr2=0x7fe5540d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.971+0000 7fe55972b640 1 -- 172.21.15.40:0/761381188 shutdown_connections 2024-04-26T21:39:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.971+0000 7fe55972b640 1 -- 172.21.15.40:0/761381188 wait complete. 2024-04-26T21:39:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.972+0000 7fe55972b640 1 Processor -- start 2024-04-26T21:39:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.972+0000 7fe55972b640 1 -- start start 2024-04-26T21:39:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe55972b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540b5f20 0x7fe5540cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe55972b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 0x7fe5540cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe55972b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe554162880 0x7fe5540cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe55972b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe554162c60 con 0x7fe5540b5f20 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe55972b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540d0d90 con 0x7fe554162880 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe55972b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe554162de0 con 0x7fe5540d3bc0 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540b5f20 0x7fe5540cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540b5f20 0x7fe5540cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56050/0 (socket says 172.21.15.40:56050) 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 -- 172.21.15.40:0/201757294 learned_addr learned my addr 172.21.15.40:0/201757294 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 -- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 msgr2=0x7fe5540cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 0x7fe5540cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 -- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe554162880 msgr2=0x7fe5540cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe554162880 0x7fe5540cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 -- 172.21.15.40:0/201757294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540cc5f0 con 0x7fe5540b5f20 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe552ffd640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540b5f20 0x7fe5540cd690 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0098c0 tx=0x7fe54c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe54b7fe640 1 -- 172.21.15.40:0/201757294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c037040 con 0x7fe5540b5f20 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.973+0000 7fe54b7fe640 1 -- 172.21.15.40:0/201757294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54c030770 con 0x7fe5540b5f20 2024-04-26T21:39:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.974+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5540cc8d0 con 0x7fe5540b5f20 2024-04-26T21:39:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.974+0000 7fe54b7fe640 1 -- 172.21.15.40:0/201757294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c030b40 con 0x7fe5540b5f20 2024-04-26T21:39:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.974+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5540cce90 con 0x7fe5540b5f20 2024-04-26T21:39:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.975+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5540694f0 con 0x7fe5540b5f20 2024-04-26T21:39:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.975+0000 7fe54b7fe640 1 -- 172.21.15.40:0/201757294 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe54c044020 con 0x7fe5540b5f20 2024-04-26T21:39:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.975+0000 7fe54b7fe640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe53406c8f0 0x7fe53406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.976+0000 7fe54b7fe640 1 -- 172.21.15.40:0/201757294 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6303+0+0 (secure 0 0 0) 0x7fe54c0ac2e0 con 0x7fe5540b5f20 2024-04-26T21:39:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.976+0000 7fe5527fc640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe53406c8f0 0x7fe53406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.976+0000 7fe5527fc640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe53406c8f0 0x7fe53406edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe53c000f60 tx=0x7fe53c007a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:03.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:03.980+0000 7fe54b7fe640 1 -- 172.21.15.40:0/201757294 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe54c032e20 con 0x7fe5540b5f20 2024-04-26T21:39:04.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.203+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5540d3550 con 0x7fe5540b5f20 2024-04-26T21:39:04.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.204+0000 7fe54b7fe640 1 -- 172.21.15.40:0/201757294 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5186 (secure 0 0 0) 0x7fe54c07a430 con 0x7fe5540b5f20 2024-04-26T21:39:04.204 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:39:04.204 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:38:46.731288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6840,"qdb_cluster":[6840]},"id":4}]} 2024-04-26T21:39:04.204 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:39:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe53406c8f0 msgr2=0x7fe53406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe53406c8f0 0x7fe53406edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe53c000f60 tx=0x7fe53c007a80 comp rx=0 tx=0).stop 2024-04-26T21:39:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540b5f20 msgr2=0x7fe5540cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540b5f20 0x7fe5540cd690 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0098c0 tx=0x7fe54c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:39:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 shutdown_connections 2024-04-26T21:39:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe53406c8f0 0x7fe53406edb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe554162880 0x7fe5540cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5540d3bc0 0x7fe5540cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:04.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 --2- 172.21.15.40:0/201757294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540b5f20 0x7fe5540cd690 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:04.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 >> 172.21.15.40:0/201757294 conn(0x7fe5540d4f60 msgr2=0x7fe5540ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:04.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 shutdown_connections 2024-04-26T21:39:04.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:04.206+0000 7fe55972b640 1 -- 172.21.15.40:0/201757294 wait complete. 2024-04-26T21:39:04.217 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T21:39:09.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:39:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.333+0000 7f550acf8640 1 Processor -- start 2024-04-26T21:39:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.333+0000 7f550acf8640 1 -- start start 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f550acf8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f55040cc480 0x7f5504072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f550acf8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f550415d7c0 con 0x7f5504072780 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f550acf8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f550415daa0 con 0x7f5504071e60 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f550acf8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f550415dd80 con 0x7f55040cc480 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f5509295640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5504072780 0x7f5504166ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35712/0 (socket says 172.21.15.40:35712) 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f5509295640 1 -- 172.21.15.40:0/2619576887 learned_addr learned my addr 172.21.15.40:0/2619576887 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f5503fff640 1 --2- 172.21.15.40:0/2619576887 >> v2:172.21.15.184:3300/0 conn(0x7f55040cc480 0x7f5504072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f55037fe640 1 -- 172.21.15.40:0/2619576887 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584800360 0 0) 0x7f550415d7c0 con 0x7f5504072780 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f55037fe640 1 -- 172.21.15.40:0/2619576887 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54f4003520 con 0x7f5504072780 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f55037fe640 1 -- 172.21.15.40:0/2619576887 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2446916185 0 0) 0x7f550415daa0 con 0x7f5504071e60 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.334+0000 7f55037fe640 1 -- 172.21.15.40:0/2619576887 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f550415d7c0 con 0x7f5504071e60 2024-04-26T21:39:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.335+0000 7f5503fff640 1 -- 172.21.15.40:0/2619576887 >> v1:172.21.15.184:6789/0 conn(0x7f5504071e60 legacy=0x7f550406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.335+0000 7f55037fe640 1 -- 172.21.15.40:0/2619576887 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 587798873 0 0) 0x7f54f4003520 con 0x7f5504072780 2024-04-26T21:39:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.335+0000 7f5503fff640 1 -- 172.21.15.40:0/2619576887 >> v1:172.21.15.40:6789/0 conn(0x7f5504072780 legacy=0x7f5504166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.335+0000 7f5503fff640 1 -- 172.21.15.40:0/2619576887 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550415e060 con 0x7f55040cc480 2024-04-26T21:39:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.335+0000 7f5503fff640 1 --2- 172.21.15.40:0/2619576887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 0x7f5504072240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097a0 tx=0x7f54f002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f882b954bac88e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f55037fe640 1 -- 172.21.15.40:0/2619576887 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0037070 con 0x7f55040cc480 2024-04-26T21:39:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f55037fe640 1 -- 172.21.15.40:0/2619576887 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f002ac50 con 0x7f55040cc480 2024-04-26T21:39:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f55037fe640 1 -- 172.21.15.40:0/2619576887 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc0019f0 con 0x7f55040cc480 2024-04-26T21:39:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f550acf8640 1 -- 172.21.15.40:0/2619576887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 msgr2=0x7f5504072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f550acf8640 1 --2- 172.21.15.40:0/2619576887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 0x7f5504072240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097a0 tx=0x7f54f002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f550acf8640 1 -- 172.21.15.40:0/2619576887 shutdown_connections 2024-04-26T21:39:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f550acf8640 1 --2- 172.21.15.40:0/2619576887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 0x7f5504072240 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f550acf8640 1 -- 172.21.15.40:0/2619576887 >> 172.21.15.40:0/2619576887 conn(0x7f55040d0be0 msgr2=0x7f55040d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f550acf8640 1 -- 172.21.15.40:0/2619576887 shutdown_connections 2024-04-26T21:39:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.336+0000 7f550acf8640 1 -- 172.21.15.40:0/2619576887 wait complete. 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.337+0000 7f550acf8640 1 Processor -- start 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.338+0000 7f550acf8640 1 -- start start 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.338+0000 7f550acf8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5504071e60 0x7f55040cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.338+0000 7f550acf8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5504072780 0x7f550414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.338+0000 7f550acf8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 0x7f550414ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.338+0000 7f550acf8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f550415d3f0 con 0x7f5504072780 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.338+0000 7f550acf8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5504072bd0 con 0x7f55040cc480 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.338+0000 7f550acf8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f550415d570 con 0x7f5504071e60 2024-04-26T21:39:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5504072780 0x7f550414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5504072780 0x7f550414a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40886/0 (socket says 172.21.15.40:40886) 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 -- 172.21.15.40:0/3541272009 learned_addr learned my addr 172.21.15.40:0/3541272009 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5509295640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 0x7f550414ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5508a94640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5504071e60 0x7f55040cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 -- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5504071e60 msgr2=0x7f55040cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5504071e60 0x7f55040cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 -- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 msgr2=0x7f550414ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 0x7f550414ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 -- 172.21.15.40:0/3541272009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55040d0400 con 0x7f5504072780 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f5503fff640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5504072780 0x7f550414a3a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f54f0000c00 tx=0x7f54f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f55017fa640 1 -- 172.21.15.40:0/3541272009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0037050 con 0x7f5504072780 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f55017fa640 1 -- 172.21.15.40:0/3541272009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f0043070 con 0x7f5504072780 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550414a8e0 con 0x7f5504072780 2024-04-26T21:39:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.339+0000 7f55017fa640 1 -- 172.21.15.40:0/3541272009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f003c830 con 0x7f5504072780 2024-04-26T21:39:09.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.340+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55040ae8d0 con 0x7f5504072780 2024-04-26T21:39:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.341+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55040694f0 con 0x7f5504072780 2024-04-26T21:39:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.341+0000 7f55017fa640 1 -- 172.21.15.40:0/3541272009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54f003c9d0 con 0x7f5504072780 2024-04-26T21:39:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.341+0000 7f55017fa640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54e806c8f0 0x7f54e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.342+0000 7f55017fa640 1 -- 172.21.15.40:0/3541272009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6303+0+0 (secure 0 0 0) 0x7f54f00ac550 con 0x7f5504072780 2024-04-26T21:39:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.342+0000 7f5508a94640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54e806c8f0 0x7f54e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.342+0000 7f5508a94640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54e806c8f0 0x7f54e806edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f54f80011e0 tx=0x7f54f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.346+0000 7f55017fa640 1 -- 172.21.15.40:0/3541272009 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54f007a1f0 con 0x7f5504072780 2024-04-26T21:39:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.576+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f550414b150 con 0x7f5504072780 2024-04-26T21:39:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.576+0000 7f55017fa640 1 -- 172.21.15.40:0/3541272009 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5234 (secure 0 0 0) 0x7f54f007e4f0 con 0x7f5504072780 2024-04-26T21:39:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:39:09.577 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:39:05.610046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6835/1699369057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1699369057},{"type":"v1","addr":"172.21.15.40:6835","nonce":1699369057}]},"join_fscid":-1,"laggy_since":"2024-04-26T21:39:05.610046+0000","export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6840,"qdb_cluster":[6840]},"id":4}]} 2024-04-26T21:39:09.577 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 58 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54e806c8f0 msgr2=0x7f54e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54e806c8f0 0x7f54e806edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f54f80011e0 tx=0x7f54f8009040 comp rx=0 tx=0).stop 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5504072780 msgr2=0x7f550414a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5504072780 0x7f550414a3a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f54f0000c00 tx=0x7f54f00046c0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 shutdown_connections 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54e806c8f0 0x7f54e806edb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040cc480 0x7f550414ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5504072780 0x7f550414a3a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 --2- 172.21.15.40:0/3541272009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5504071e60 0x7f55040cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 >> 172.21.15.40:0/3541272009 conn(0x7f55040d0be0 msgr2=0x7f55041592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 shutdown_connections 2024-04-26T21:39:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.578+0000 7f550acf8640 1 -- 172.21.15.40:0/3541272009 wait complete. 2024-04-26T21:39:09.590 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-26T21:39:09.691 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-04-26T21:39:09.691 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2024-04-26T21:39:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.794+0000 7ff0f1bda640 1 Processor -- start 2024-04-26T21:39:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.794+0000 7ff0f1bda640 1 -- start start 2024-04-26T21:39:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0f1bda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec162810 0x7ff0ec070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0f1bda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff0ec0d1b30 0x7ff0ec0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0f1bda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ec0d10d0 con 0x7ff0ec0b2340 2024-04-26T21:39:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0f1bda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ec0d0750 con 0x7ff0ec0d1b30 2024-04-26T21:39:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0f1bda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec0d16a0 con 0x7ff0ec162810 2024-04-26T21:39:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0ebfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff0ec0d1b30 0x7ff0ec0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0ebfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff0ec0d1b30 0x7ff0ec0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40890/0 (socket says 172.21.15.40:40890) 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0ebfff640 1 -- 172.21.15.40:0/2922179542 learned_addr learned my addr 172.21.15.40:0/2922179542 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0eb7fe640 1 --2- 172.21.15.40:0/2922179542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec162810 0x7ff0ec070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0ebfff640 1 -- 172.21.15.40:0/2922179542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec162810 msgr2=0x7ff0ec070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0ebfff640 1 --2- 172.21.15.40:0/2922179542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec162810 0x7ff0ec070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0ebfff640 1 -- 172.21.15.40:0/2922179542 >> v1:172.21.15.40:6789/0 conn(0x7ff0ec0b2340 legacy=0x7ff0ec0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.795+0000 7ff0ebfff640 1 -- 172.21.15.40:0/2922179542 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec0d1370 con 0x7ff0ec0d1b30 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.796+0000 7ff0ebfff640 1 --2- 172.21.15.40:0/2922179542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0d1b30 0x7ff0ec0d1fc0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009e50 tx=0x7ff0e402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcff9e85b005d7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.796+0000 7ff0ea7fc640 1 -- 172.21.15.40:0/2922179542 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4036070 con 0x7ff0ec0d1b30 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.796+0000 7ff0ea7fc640 1 -- 172.21.15.40:0/2922179542 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e4031600 con 0x7ff0ec0d1b30 2024-04-26T21:39:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.796+0000 7ff0ea7fc640 1 -- 172.21.15.40:0/2922179542 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40319d0 con 0x7ff0ec0d1b30 2024-04-26T21:39:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.797+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2922179542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0d1b30 msgr2=0x7ff0ec0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.797+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2922179542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0d1b30 0x7ff0ec0d1fc0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009e50 tx=0x7ff0e402a710 comp rx=0 tx=0).stop 2024-04-26T21:39:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.797+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2922179542 shutdown_connections 2024-04-26T21:39:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.797+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2922179542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0d1b30 0x7ff0ec0d1fc0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.797+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2922179542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec162810 0x7ff0ec070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.797+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2922179542 >> 172.21.15.40:0/2922179542 conn(0x7ff0ec0d2da0 msgr2=0x7ff0ec0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.797+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2922179542 shutdown_connections 2024-04-26T21:39:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.797+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2922179542 wait complete. 2024-04-26T21:39:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.798+0000 7ff0f1bda640 1 Processor -- start 2024-04-26T21:39:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.798+0000 7ff0f1bda640 1 -- start start 2024-04-26T21:39:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0f1bda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0b2340 0x7ff0ec1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0f1bda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec0d1b30 0x7ff0ec0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0f1bda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0ec162810 0x7ff0ec15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0f1bda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec070890 con 0x7ff0ec0b2340 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0f1bda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec0d0470 con 0x7ff0ec162810 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0f1bda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec0cdb00 con 0x7ff0ec0d1b30 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0b2340 0x7ff0ec1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0b2340 0x7ff0ec1501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40902/0 (socket says 172.21.15.40:40902) 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eb7fe640 1 -- 172.21.15.40:0/2839453011 learned_addr learned my addr 172.21.15.40:0/2839453011 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0ebfff640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0ec162810 0x7ff0ec15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eaffd640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec0d1b30 0x7ff0ec0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eb7fe640 1 -- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec0d1b30 msgr2=0x7ff0ec0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eb7fe640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec0d1b30 0x7ff0ec0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eb7fe640 1 -- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0ec162810 msgr2=0x7ff0ec15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eb7fe640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0ec162810 0x7ff0ec15bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eb7fe640 1 -- 172.21.15.40:0/2839453011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec160a10 con 0x7ff0ec0b2340 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.799+0000 7ff0eaffd640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec0d1b30 0x7ff0ec0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.800+0000 7ff0eb7fe640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0b2340 0x7ff0ec1501a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00d5f0 tx=0x7ff0dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.800+0000 7ff0d3fff640 1 -- 172.21.15.40:0/2839453011 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0125f0 con 0x7ff0ec0b2340 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.800+0000 7ff0d3fff640 1 -- 172.21.15.40:0/2839453011 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0dc019070 con 0x7ff0ec0b2340 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.800+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec160cd0 con 0x7ff0ec0b2340 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.800+0000 7ff0d3fff640 1 -- 172.21.15.40:0/2839453011 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0129d0 con 0x7ff0ec0b2340 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.800+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec161290 con 0x7ff0ec0b2340 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.801+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec0694f0 con 0x7ff0ec0b2340 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.801+0000 7ff0d3fff640 1 -- 172.21.15.40:0/2839453011 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0dc012b70 con 0x7ff0ec0b2340 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.802+0000 7ff0d3fff640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d806c8f0 0x7ff0d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.802+0000 7ff0d3fff640 1 -- 172.21.15.40:0/2839453011 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6303+0+0 (secure 0 0 0) 0x7ff0dc08c250 con 0x7ff0ec0b2340 2024-04-26T21:39:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.802+0000 7ff0eaffd640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d806c8f0 0x7ff0d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.803+0000 7ff0eaffd640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d806c8f0 0x7ff0d806edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec06f4f0 tx=0x7ff0e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.806+0000 7ff0d3fff640 1 -- 172.21.15.40:0/2839453011 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0dc05a300 con 0x7ff0ec0b2340 2024-04-26T21:39:09.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.991+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7ff0ec0b88b0 con 0x7ff0ec0b2340 2024-04-26T21:39:09.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.994+0000 7ff0d3fff640 1 -- 172.21.15.40:0/2839453011 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7ff0dc059c00 con 0x7ff0ec0b2340 2024-04-26T21:39:09.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d806c8f0 msgr2=0x7ff0d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d806c8f0 0x7ff0d806edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec06f4f0 tx=0x7ff0e0009040 comp rx=0 tx=0).stop 2024-04-26T21:39:09.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0b2340 msgr2=0x7ff0ec1501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:09.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0b2340 0x7ff0ec1501a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00d5f0 tx=0x7ff0dc00dab0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 shutdown_connections 2024-04-26T21:39:09.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff0d806c8f0 0x7ff0d806edb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0ec162810 0x7ff0ec15bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0ec0d1b30 0x7ff0ec0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 --2- 172.21.15.40:0/2839453011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0ec0b2340 0x7ff0ec1501a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:09.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 >> 172.21.15.40:0/2839453011 conn(0x7ff0ec0d2da0 msgr2=0x7ff0ec070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:09.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 shutdown_connections 2024-04-26T21:39:09.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:09.996+0000 7ff0f1bda640 1 -- 172.21.15.40:0/2839453011 wait complete. 2024-04-26T21:39:10.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 false 2024-04-26T21:39:10.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.110+0000 7f3a9bac8640 1 Processor -- start 2024-04-26T21:39:10.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.110+0000 7f3a9bac8640 1 -- start start 2024-04-26T21:39:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a9bac8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a940d3bc0 0x7f3a940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a9bac8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a940b7e50 con 0x7f3a94162880 2024-04-26T21:39:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a9bac8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a940d1800 con 0x7f3a940b5f20 2024-04-26T21:39:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a9bac8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3a940d0580 con 0x7f3a940d3bc0 2024-04-26T21:39:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a99063640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a940d3bc0 0x7f3a940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a99063640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a940d3bc0 0x7f3a940d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40904/0 (socket says 172.21.15.40:40904) 2024-04-26T21:39:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a99063640 1 -- 172.21.15.40:0/2263614354 learned_addr learned my addr 172.21.15.40:0/2263614354 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a99063640 1 -- 172.21.15.40:0/2263614354 >> v1:172.21.15.184:6789/0 conn(0x7f3a940b5f20 legacy=0x7f3a940b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a99063640 1 -- 172.21.15.40:0/2263614354 >> v1:172.21.15.40:6789/0 conn(0x7f3a94162880 legacy=0x7f3a940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.111+0000 7f3a99063640 1 -- 172.21.15.40:0/2263614354 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a940d07d0 con 0x7f3a940d3bc0 2024-04-26T21:39:10.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.112+0000 7f3a99063640 1 --2- 172.21.15.40:0/2263614354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a940d2cb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a8402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8695a6d1fba2044c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:10.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.112+0000 7f3a98862640 1 -- 172.21.15.40:0/2263614354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84037070 con 0x7f3a940d3bc0 2024-04-26T21:39:10.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.112+0000 7f3a98862640 1 -- 172.21.15.40:0/2263614354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a84032440 con 0x7f3a940d3bc0 2024-04-26T21:39:10.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.112+0000 7f3a98862640 1 -- 172.21.15.40:0/2263614354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84032810 con 0x7f3a940d3bc0 2024-04-26T21:39:10.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.113+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2263614354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 msgr2=0x7f3a940d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.113+0000 7f3a9bac8640 1 --2- 172.21.15.40:0/2263614354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a940d2cb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a8402a2a0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.113+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2263614354 shutdown_connections 2024-04-26T21:39:10.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.113+0000 7f3a9bac8640 1 --2- 172.21.15.40:0/2263614354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a940d2cb0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.113+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2263614354 >> 172.21.15.40:0/2263614354 conn(0x7f3a940d4f60 msgr2=0x7f3a940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:10.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.113+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2263614354 shutdown_connections 2024-04-26T21:39:10.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.113+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2263614354 wait complete. 2024-04-26T21:39:10.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.114+0000 7f3a9bac8640 1 Processor -- start 2024-04-26T21:39:10.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.114+0000 7f3a9bac8640 1 -- start start 2024-04-26T21:39:10.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a9bac8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940b5f20 0x7f3a941f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a9bac8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a941f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a9bac8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94162880 0x7f3a940b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a9bac8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94162c60 con 0x7f3a940d3bc0 2024-04-26T21:39:10.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a9bac8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a940d0d90 con 0x7f3a940b5f20 2024-04-26T21:39:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a9bac8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a94162de0 con 0x7f3a94162880 2024-04-26T21:39:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a941f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a941f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40920/0 (socket says 172.21.15.40:40920) 2024-04-26T21:39:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 -- 172.21.15.40:0/2078363582 learned_addr learned my addr 172.21.15.40:0/2078363582 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99864640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940b5f20 0x7f3a941f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a9a065640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94162880 0x7f3a940b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 -- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94162880 msgr2=0x7f3a940b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94162880 0x7f3a940b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 -- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940b5f20 msgr2=0x7f3a941f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940b5f20 0x7f3a941f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 -- 172.21.15.40:0/2078363582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a940b30d0 con 0x7f3a940d3bc0 2024-04-26T21:39:10.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a99063640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a941f76a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:10.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.115+0000 7f3a8a7fc640 1 -- 172.21.15.40:0/2078363582 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84041020 con 0x7f3a940d3bc0 2024-04-26T21:39:10.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.116+0000 7f3a8a7fc640 1 -- 172.21.15.40:0/2078363582 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a84030970 con 0x7f3a940d3bc0 2024-04-26T21:39:10.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.116+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a940b33b0 con 0x7f3a940d3bc0 2024-04-26T21:39:10.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.116+0000 7f3a8a7fc640 1 -- 172.21.15.40:0/2078363582 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84037040 con 0x7f3a940d3bc0 2024-04-26T21:39:10.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.116+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a940b3970 con 0x7f3a940d3bc0 2024-04-26T21:39:10.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.117+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a940b3e40 con 0x7f3a940d3bc0 2024-04-26T21:39:10.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.117+0000 7f3a8a7fc640 1 -- 172.21.15.40:0/2078363582 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a84044020 con 0x7f3a940d3bc0 2024-04-26T21:39:10.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.118+0000 7f3a8a7fc640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a7806c8f0 0x7f3a7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.118+0000 7f3a8a7fc640 1 -- 172.21.15.40:0/2078363582 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6303+0+0 (secure 0 0 0) 0x7f3a840ac700 con 0x7f3a940d3bc0 2024-04-26T21:39:10.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.118+0000 7f3a99864640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a7806c8f0 0x7f3a7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.119+0000 7f3a99864640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a7806c8f0 0x7f3a7806edb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a90007fd0 tx=0x7f3a90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:10.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.122+0000 7f3a8a7fc640 1 -- 172.21.15.40:0/2078363582 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a8404b030 con 0x7f3a940d3bc0 2024-04-26T21:39:10.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.388+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"} v 0) -- 0x7f3a940694f0 con 0x7f3a940d3bc0 2024-04-26T21:39:10.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.612+0000 7f3a8a7fc640 1 -- 172.21.15.40:0/2078363582 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"}]=0 unfreezing mds.6840 v59) ==== 115+0+0 (secure 0 0 0) 0x7f3a940d1800 con 0x7f3a940d3bc0 2024-04-26T21:39:10.612 INFO:teuthology.orchestra.run.smithi040.stderr:unfreezing mds.6840 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.615+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a7806c8f0 msgr2=0x7f3a7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.615+0000 7f3a9bac8640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a7806c8f0 0x7f3a7806edb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a90007fd0 tx=0x7f3a90009040 comp rx=0 tx=0).stop 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.615+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 msgr2=0x7f3a941f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.615+0000 7f3a9bac8640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a941f76a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a840046c0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.616+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 shutdown_connections 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.616+0000 7f3a9bac8640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a7806c8f0 0x7f3a7806edb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.616+0000 7f3a9bac8640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94162880 0x7f3a940b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.616+0000 7f3a9bac8640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940d3bc0 0x7f3a941f76a0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.616+0000 7f3a9bac8640 1 --2- 172.21.15.40:0/2078363582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940b5f20 0x7f3a941f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.616+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 >> 172.21.15.40:0/2078363582 conn(0x7f3a940d4f60 msgr2=0x7f3a9406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.616+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 shutdown_connections 2024-04-26T21:39:10.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.616+0000 7f3a9bac8640 1 -- 172.21.15.40:0/2078363582 wait complete. 2024-04-26T21:39:10.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -n kernel.core_pattern 2024-04-26T21:39:10.655 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:39:10.657 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:39:10.657 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-04-26T21:39:10.725 INFO:teuthology.orchestra.run.smithi040.stdout:1714167527.57838.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2024-04-26T21:39:10.727 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1714167527.57838.core'] 2024-04-26T21:39:10.727 INFO:tasks.cephfs.cephfs_test_case:Found core file 1714167527.57838.core, deleting it 2024-04-26T21:39:10.728 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1714167527.57838.core 2024-04-26T21:39:10.786 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:39:10.795 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:39:10.795 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:39:10.795 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:39:10.796 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:39:10.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:39:10.796 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi040 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-04-26T21:39:10.797 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:39:10.797 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:39:10.838 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:39:10.839 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:39:10.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:39:10.915 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:39:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.983+0000 7f12b2625640 1 Processor -- start 2024-04-26T21:39:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.983+0000 7f12b2625640 1 -- start start 2024-04-26T21:39:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b2625640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac15f3a0 0x7f12ac0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b2625640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f12ac0d1770 0x7f12ac0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b2625640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12ac0d2130 0x7f12ac0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b2625640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f12ac0d25a0 con 0x7f12ac0d2130 2024-04-26T21:39:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b2625640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f12ac0d0e20 con 0x7f12ac0d1770 2024-04-26T21:39:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b2625640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac0b1d70 con 0x7f12ac15f3a0 2024-04-26T21:39:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b0bc2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12ac0d2130 0x7f12ac0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b0bc2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12ac0d2130 0x7f12ac0d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40942/0 (socket says 172.21.15.40:40942) 2024-04-26T21:39:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12b0bc2640 1 -- 172.21.15.40:0/3832002513 learned_addr learned my addr 172.21.15.40:0/3832002513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12ab7fe640 1 --2- 172.21.15.40:0/3832002513 >> v2:172.21.15.184:3300/0 conn(0x7f12ac0d1770 0x7f12ac0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.984+0000 7f12abfff640 1 --2- 172.21.15.40:0/3832002513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac15f3a0 0x7f12ac0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12b0bc2640 1 -- 172.21.15.40:0/3832002513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac15f3a0 msgr2=0x7f12ac0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12b0bc2640 1 --2- 172.21.15.40:0/3832002513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac15f3a0 0x7f12ac0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12b0bc2640 1 -- 172.21.15.40:0/3832002513 >> v2:172.21.15.184:3300/0 conn(0x7f12ac0d1770 msgr2=0x7f12ac0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12b0bc2640 1 --2- 172.21.15.40:0/3832002513 >> v2:172.21.15.184:3300/0 conn(0x7f12ac0d1770 0x7f12ac0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12b0bc2640 1 -- 172.21.15.40:0/3832002513 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac0b2050 con 0x7f12ac0d2130 2024-04-26T21:39:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12abfff640 1 --2- 172.21.15.40:0/3832002513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac15f3a0 0x7f12ac0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:39:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12b0bc2640 1 --2- 172.21.15.40:0/3832002513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0d2130 0x7f12ac0d2e50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f12ac073c50 tx=0x7f12a402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9476243f5edee7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12aaffd640 1 -- 172.21.15.40:0/3832002513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4036070 con 0x7f12ac0d2130 2024-04-26T21:39:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12aaffd640 1 -- 172.21.15.40:0/3832002513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a4031600 con 0x7f12ac0d2130 2024-04-26T21:39:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.985+0000 7f12aaffd640 1 -- 172.21.15.40:0/3832002513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a40319d0 con 0x7f12ac0d2130 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 -- 172.21.15.40:0/3832002513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0d2130 msgr2=0x7f12ac0d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 --2- 172.21.15.40:0/3832002513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0d2130 0x7f12ac0d2e50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f12ac073c50 tx=0x7f12a402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 -- 172.21.15.40:0/3832002513 shutdown_connections 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 --2- 172.21.15.40:0/3832002513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac0d2130 0x7f12ac0d2e50 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 --2- 172.21.15.40:0/3832002513 >> v2:172.21.15.184:3300/0 conn(0x7f12ac0d1770 0x7f12ac0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 --2- 172.21.15.40:0/3832002513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac15f3a0 0x7f12ac0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 -- 172.21.15.40:0/3832002513 >> 172.21.15.40:0/3832002513 conn(0x7f12ac0d4f90 msgr2=0x7f12ac0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 -- 172.21.15.40:0/3832002513 shutdown_connections 2024-04-26T21:39:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.986+0000 7f12b2625640 1 -- 172.21.15.40:0/3832002513 wait complete. 2024-04-26T21:39:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.987+0000 7f12b2625640 1 Processor -- start 2024-04-26T21:39:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.987+0000 7f12b2625640 1 -- start start 2024-04-26T21:39:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b2625640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac0d1770 0x7f12ac15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b2625640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d2130 0x7f12ac159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b2625640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac15f3a0 0x7f12ac0b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b2625640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac15f900 con 0x7f12ac15f3a0 2024-04-26T21:39:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b2625640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac15f780 con 0x7f12ac0d1770 2024-04-26T21:39:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b2625640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac0d3390 con 0x7f12ac0d2130 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b0bc2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac15f3a0 0x7f12ac0b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b0bc2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac15f3a0 0x7f12ac0b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40946/0 (socket says 172.21.15.40:40946) 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12b0bc2640 1 -- 172.21.15.40:0/2096655632 learned_addr learned my addr 172.21.15.40:0/2096655632 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12abfff640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac0d1770 0x7f12ac15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.988+0000 7f12ab7fe640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d2130 0x7f12ac159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12b0bc2640 1 -- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d2130 msgr2=0x7f12ac159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12b0bc2640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d2130 0x7f12ac159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12b0bc2640 1 -- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac0d1770 msgr2=0x7f12ac15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12b0bc2640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac0d1770 0x7f12ac15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12b0bc2640 1 -- 172.21.15.40:0/2096655632 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac15fc20 con 0x7f12ac15f3a0 2024-04-26T21:39:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12abfff640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac0d1770 0x7f12ac15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12b0bc2640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac15f3a0 0x7f12ac0b54d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f12a4034040 tx=0x7f12a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12a8ff9640 1 -- 172.21.15.40:0/2096655632 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4036050 con 0x7f12ac15f3a0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12a8ff9640 1 -- 172.21.15.40:0/2096655632 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a4042070 con 0x7f12ac15f3a0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac15ff00 con 0x7f12ac15f3a0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12a8ff9640 1 -- 172.21.15.40:0/2096655632 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4009510 con 0x7f12ac15f3a0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.989+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac1604c0 con 0x7f12ac15f3a0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.990+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac162d20 con 0x7f12ac15f3a0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.991+0000 7f12a8ff9640 1 -- 172.21.15.40:0/2096655632 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12a4007bf0 con 0x7f12ac15f3a0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.991+0000 7f12a8ff9640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f128c06c8f0 0x7f128c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.991+0000 7f12a8ff9640 1 -- 172.21.15.40:0/2096655632 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f12a40acb00 con 0x7f12ac15f3a0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.991+0000 7f12abfff640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f128c06c8f0 0x7f128c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.992+0000 7f12abfff640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f128c06c8f0 0x7f128c06edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f12ac15d1e0 tx=0x7f1294008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:10.995+0000 7f12a8ff9640 1 -- 172.21.15.40:0/2096655632 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f12a407ad10 con 0x7f12ac15f3a0 2024-04-26T21:39:11.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.224+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12ac0694f0 con 0x7f12ac15f3a0 2024-04-26T21:39:11.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.225+0000 7f12a8ff9640 1 -- 172.21.15.40:0/2096655632 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+4354 (secure 0 0 0) 0x7f12a407a610 con 0x7f12ac15f3a0 2024-04-26T21:39:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:39:11.226 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:39:10.611364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-26T21:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 59 2024-04-26T21:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f128c06c8f0 msgr2=0x7f128c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f128c06c8f0 0x7f128c06edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f12ac15d1e0 tx=0x7f1294008040 comp rx=0 tx=0).stop 2024-04-26T21:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac15f3a0 msgr2=0x7f12ac0b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac15f3a0 0x7f12ac0b54d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f12a4034040 tx=0x7f12a4004a40 comp rx=0 tx=0).stop 2024-04-26T21:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 shutdown_connections 2024-04-26T21:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f128c06c8f0 0x7f128c06edb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12ac15f3a0 0x7f12ac0b54d0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12ac0d2130 0x7f12ac159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 --2- 172.21.15.40:0/2096655632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12ac0d1770 0x7f12ac15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 >> 172.21.15.40:0/2096655632 conn(0x7f12ac0d4f90 msgr2=0x7f12ac0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 shutdown_connections 2024-04-26T21:39:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:11.227+0000 7f12b2625640 1 -- 172.21.15.40:0/2096655632 wait complete. 2024-04-26T21:39:11.641 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:38:18.310357+0000', 'modified': '2024-04-26T21:39:10.611364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6540}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6540': {'gid': 6540, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6834/425620740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 425620740}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 425620740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:39:11.641 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6540:up:replay 2024-04-26T21:39:12.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:39:12.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.756+0000 7f53ab119640 1 Processor -- start 2024-04-26T21:39:12.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.756+0000 7f53ab119640 1 -- start start 2024-04-26T21:39:12.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.756+0000 7f53ab119640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53a415f3a0 0x7f53a40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:12.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.756+0000 7f53ab119640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a40d1770 0x7f53a40d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:12.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.756+0000 7f53ab119640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a40d25a0 con 0x7f53a40d2130 2024-04-26T21:39:12.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.756+0000 7f53ab119640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f53a40d0e20 con 0x7f53a415f3a0 2024-04-26T21:39:12.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.756+0000 7f53ab119640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f53a40b1d70 con 0x7f53a40d1770 2024-04-26T21:39:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a8eb5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53a415f3a0 0x7f53a40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a40d1770 0x7f53a40d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a8eb5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53a415f3a0 0x7f53a40d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:53766/0 (socket says 172.21.15.40:53766) 2024-04-26T21:39:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a40d1770 0x7f53a40d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:55464/0 (socket says 172.21.15.40:55464) 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a3fff640 1 -- 172.21.15.40:0/846588492 learned_addr learned my addr 172.21.15.40:0/846588492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a96b6640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f53a40d2130 0x7f53a40d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39282/0 (socket says 172.21.15.40:39282) 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a37fe640 1 -- 172.21.15.40:0/846588492 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3137831421 0 0) 0x7f539c001490 con 0x7f53a40d2130 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a37fe640 1 -- 172.21.15.40:0/846588492 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5390003520 con 0x7f53a40d2130 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a3fff640 1 -- 172.21.15.40:0/846588492 >> v2:172.21.15.184:3300/0 conn(0x7f53a415f3a0 msgr2=0x7f53a40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a3fff640 1 --2- 172.21.15.40:0/846588492 >> v2:172.21.15.184:3300/0 conn(0x7f53a415f3a0 0x7f53a40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a3fff640 1 -- 172.21.15.40:0/846588492 >> v1:172.21.15.184:6789/0 conn(0x7f53a40d2130 legacy=0x7f53a40d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a3fff640 1 -- 172.21.15.40:0/846588492 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a40b2050 con 0x7f53a40d1770 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.757+0000 7f53a8eb5640 1 --2- 172.21.15.40:0/846588492 >> v2:172.21.15.184:3300/0 conn(0x7f53a415f3a0 0x7f53a40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.758+0000 7f53a3fff640 1 --2- 172.21.15.40:0/846588492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a40d1770 0x7f53a40d1bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f538c009870 tx=0x7f538c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90911c340b63649 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.758+0000 7f53a37fe640 1 -- 172.21.15.40:0/846588492 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c037070 con 0x7f53a40d1770 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.758+0000 7f53a37fe640 1 -- 172.21.15.40:0/846588492 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c032440 con 0x7f53a40d1770 2024-04-26T21:39:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.758+0000 7f53a37fe640 1 -- 172.21.15.40:0/846588492 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c032810 con 0x7f53a40d1770 2024-04-26T21:39:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.759+0000 7f53ab119640 1 -- 172.21.15.40:0/846588492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a40d1770 msgr2=0x7f53a40d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.759+0000 7f53ab119640 1 --2- 172.21.15.40:0/846588492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a40d1770 0x7f53a40d1bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f538c009870 tx=0x7f538c02a270 comp rx=0 tx=0).stop 2024-04-26T21:39:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.759+0000 7f53ab119640 1 -- 172.21.15.40:0/846588492 shutdown_connections 2024-04-26T21:39:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.759+0000 7f53ab119640 1 --2- 172.21.15.40:0/846588492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a40d1770 0x7f53a40d1bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.759+0000 7f53ab119640 1 --2- 172.21.15.40:0/846588492 >> v2:172.21.15.184:3300/0 conn(0x7f53a415f3a0 0x7f53a40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.759+0000 7f53ab119640 1 -- 172.21.15.40:0/846588492 >> 172.21.15.40:0/846588492 conn(0x7f53a40d4f90 msgr2=0x7f53a40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.759+0000 7f53ab119640 1 -- 172.21.15.40:0/846588492 shutdown_connections 2024-04-26T21:39:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.759+0000 7f53ab119640 1 -- 172.21.15.40:0/846588492 wait complete. 2024-04-26T21:39:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.760+0000 7f53ab119640 1 Processor -- start 2024-04-26T21:39:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.760+0000 7f53ab119640 1 -- start start 2024-04-26T21:39:12.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.761+0000 7f53ab119640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40d1770 0x7f53a41534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:12.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.761+0000 7f53ab119640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a40d2130 0x7f53a4153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:12.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.761+0000 7f53ab119640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a415f3a0 0x7f53a40b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.761+0000 7f53ab119640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53a415f900 con 0x7f53a40d2130 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.761+0000 7f53ab119640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f53a415f780 con 0x7f53a40d1770 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.761+0000 7f53ab119640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f53a40b3860 con 0x7f53a415f3a0 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.761+0000 7f53a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a40d2130 0x7f53a4153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a40d2130 0x7f53a4153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40962/0 (socket says 172.21.15.40:40962) 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a3fff640 1 -- 172.21.15.40:0/2093861100 learned_addr learned my addr 172.21.15.40:0/2093861100 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a8eb5640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40d1770 0x7f53a41534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a96b6640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a415f3a0 0x7f53a40b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a3fff640 1 -- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a415f3a0 msgr2=0x7f53a40b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a3fff640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a415f3a0 0x7f53a40b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a3fff640 1 -- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40d1770 msgr2=0x7f53a41534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a3fff640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40d1770 0x7f53a41534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a3fff640 1 -- 172.21.15.40:0/2093861100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a40b8b70 con 0x7f53a40d2130 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a3fff640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a40d2130 0x7f53a4153c30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f538c02a2b0 tx=0x7f538c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a17fa640 1 -- 172.21.15.40:0/2093861100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c041020 con 0x7f53a40d2130 2024-04-26T21:39:12.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a17fa640 1 -- 172.21.15.40:0/2093861100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c03d620 con 0x7f53a40d2130 2024-04-26T21:39:12.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a40b85b0 con 0x7f53a40d2130 2024-04-26T21:39:12.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53a17fa640 1 -- 172.21.15.40:0/2093861100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c037040 con 0x7f53a40d2130 2024-04-26T21:39:12.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.762+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a40b92e0 con 0x7f53a40d2130 2024-04-26T21:39:12.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.763+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a40b6860 con 0x7f53a40d2130 2024-04-26T21:39:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.764+0000 7f53a17fa640 1 -- 172.21.15.40:0/2093861100 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f538c044020 con 0x7f53a40d2130 2024-04-26T21:39:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.764+0000 7f53a17fa640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f538406c8f0 0x7f538406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.765+0000 7f53a17fa640 1 -- 172.21.15.40:0/2093861100 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f538c0ac4b0 con 0x7f53a40d2130 2024-04-26T21:39:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.765+0000 7f53a8eb5640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f538406c8f0 0x7f538406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.765+0000 7f53a8eb5640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f538406c8f0 0x7f538406edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5394009560 tx=0x7f5394009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:12.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.769+0000 7f53a17fa640 1 -- 172.21.15.40:0/2093861100 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f538c04b030 con 0x7f53a40d2130 2024-04-26T21:39:12.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.993+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53a40694d0 con 0x7f53a40d2130 2024-04-26T21:39:12.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.993+0000 7f53a17fa640 1 -- 172.21.15.40:0/2093861100 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5180 (secure 0 0 0) 0x7f538c07a5b0 con 0x7f53a40d2130 2024-04-26T21:39:12.994 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:39:12.994 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6975,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3716769386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3716769386},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716769386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:39:11.616332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":59,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-26T21:39:12.994 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 61 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f538406c8f0 msgr2=0x7f538406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f538406c8f0 0x7f538406edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5394009560 tx=0x7f5394009190 comp rx=0 tx=0).stop 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a40d2130 msgr2=0x7f53a4153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a40d2130 0x7f53a4153c30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f538c02a2b0 tx=0x7f538c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 shutdown_connections 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f538406c8f0 0x7f538406edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a415f3a0 0x7f53a40b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a40d2130 0x7f53a4153c30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 --2- 172.21.15.40:0/2093861100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40d1770 0x7f53a41534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 >> 172.21.15.40:0/2093861100 conn(0x7f53a40d4f90 msgr2=0x7f53a40b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 shutdown_connections 2024-04-26T21:39:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:12.995+0000 7f53ab119640 1 -- 172.21.15.40:0/2093861100 wait complete. 2024-04-26T21:39:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:13.200+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=BANNER_CONNECTING pgs=14 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:13.200+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_RECONNECTING pgs=14 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2024-04-26T21:39:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:13.200+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_RECONNECTING pgs=14 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2024-04-26T21:39:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:13.200+0000 7f36917fa640 0 client.6921 ms_handle_remote_reset on v2:172.21.15.40:6832/1699369057 2024-04-26T21:39:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:13.200+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-04-26T21:39:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:13.200+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:39:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:13.200+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:39:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:13.200+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-26T21:39:13.409 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:38:18.310357+0000', 'modified': '2024-04-26T21:39:11.616332+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6540}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6540': {'gid': 6540, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:rejoin', 'state_seq': 17, 'addr': '172.21.15.40:6834/425620740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 425620740}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 425620740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:39:13.409 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6540:up:rejoin 2024-04-26T21:39:14.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:39:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.524+0000 7f8db4a93640 1 Processor -- start 2024-04-26T21:39:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.524+0000 7f8db4a93640 1 -- start start 2024-04-26T21:39:14.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8db4a93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8db015f380 0x7f8db00cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:14.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8db4a93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8db00d0620 0x7f8db00cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:14.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8db4a93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8db0072c00 con 0x7f8db0073b70 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8db4a93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8db00729b0 con 0x7f8db015f380 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8db4a93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8db0072ee0 con 0x7f8db00d0620 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8dae59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8db015f380 0x7f8db00cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8dae59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8db015f380 0x7f8db00cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40974/0 (socket says 172.21.15.40:40974) 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8dae59c640 1 -- 172.21.15.40:0/2099061998 learned_addr learned my addr 172.21.15.40:0/2099061998 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8daed9d640 1 --2- 172.21.15.40:0/2099061998 >> v2:172.21.15.184:3300/0 conn(0x7f8db00d0620 0x7f8db00cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8dae59c640 1 -- 172.21.15.40:0/2099061998 >> v2:172.21.15.184:3300/0 conn(0x7f8db00d0620 msgr2=0x7f8db00cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8dae59c640 1 --2- 172.21.15.40:0/2099061998 >> v2:172.21.15.184:3300/0 conn(0x7f8db00d0620 0x7f8db00cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8dae59c640 1 -- 172.21.15.40:0/2099061998 >> v1:172.21.15.184:6789/0 conn(0x7f8db0073b70 legacy=0x7f8db00cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.525+0000 7f8dae59c640 1 -- 172.21.15.40:0/2099061998 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db00731c0 con 0x7f8db015f380 2024-04-26T21:39:14.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.526+0000 7f8dae59c640 1 --2- 172.21.15.40:0/2099061998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db015f380 0x7f8db00cb440 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8db00735d0 tx=0x7f8da002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae2528135a698c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:14.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.526+0000 7f8dadd9b640 1 -- 172.21.15.40:0/2099061998 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0037070 con 0x7f8db015f380 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.526+0000 7f8dadd9b640 1 -- 172.21.15.40:0/2099061998 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da0032440 con 0x7f8db015f380 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.526+0000 7f8dadd9b640 1 -- 172.21.15.40:0/2099061998 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0032810 con 0x7f8db015f380 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.527+0000 7f8db4a93640 1 -- 172.21.15.40:0/2099061998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db015f380 msgr2=0x7f8db00cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.527+0000 7f8db4a93640 1 --2- 172.21.15.40:0/2099061998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db015f380 0x7f8db00cb440 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8db00735d0 tx=0x7f8da002a230 comp rx=0 tx=0).stop 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.527+0000 7f8db4a93640 1 -- 172.21.15.40:0/2099061998 shutdown_connections 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.527+0000 7f8db4a93640 1 --2- 172.21.15.40:0/2099061998 >> v2:172.21.15.184:3300/0 conn(0x7f8db00d0620 0x7f8db00cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.527+0000 7f8db4a93640 1 --2- 172.21.15.40:0/2099061998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db015f380 0x7f8db00cb440 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.527+0000 7f8db4a93640 1 -- 172.21.15.40:0/2099061998 >> 172.21.15.40:0/2099061998 conn(0x7f8db00d0be0 msgr2=0x7f8db00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.527+0000 7f8db4a93640 1 -- 172.21.15.40:0/2099061998 shutdown_connections 2024-04-26T21:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.527+0000 7f8db4a93640 1 -- 172.21.15.40:0/2099061998 wait complete. 2024-04-26T21:39:14.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.528+0000 7f8db4a93640 1 Processor -- start 2024-04-26T21:39:14.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.528+0000 7f8db4a93640 1 -- start start 2024-04-26T21:39:14.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8db4a93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db00d0620 0x7f8db00c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:14.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8db4a93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db015f380 0x7f8db006f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:14.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8db4a93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db01f8950 0x7f8db00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:14.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8db4a93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8db0073620 con 0x7f8db01f8950 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8db4a93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8db00aff50 con 0x7f8db015f380 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8db4a93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8db00737a0 con 0x7f8db00d0620 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8daed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db01f8950 0x7f8db00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8daed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db01f8950 0x7f8db00c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40990/0 (socket says 172.21.15.40:40990) 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8daed9d640 1 -- 172.21.15.40:0/3710133801 learned_addr learned my addr 172.21.15.40:0/3710133801 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8da7fff640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db015f380 0x7f8db006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.529+0000 7f8dae59c640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db00d0620 0x7f8db00c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8daed9d640 1 -- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db00d0620 msgr2=0x7f8db00c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8daed9d640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db00d0620 0x7f8db00c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8daed9d640 1 -- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db015f380 msgr2=0x7f8db006f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8daed9d640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db015f380 0x7f8db006f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8daed9d640 1 -- 172.21.15.40:0/3710133801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db01f8de0 con 0x7f8db01f8950 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8daed9d640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db01f8950 0x7f8db00c9770 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8da800cd70 tx=0x7f8da800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3710133801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8003b90 con 0x7f8db01f8950 2024-04-26T21:39:14.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3710133801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da8003d30 con 0x7f8db01f8950 2024-04-26T21:39:14.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db01f90c0 con 0x7f8db01f8950 2024-04-26T21:39:14.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3710133801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8004240 con 0x7f8db01f8950 2024-04-26T21:39:14.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.530+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db01f9680 con 0x7f8db01f8950 2024-04-26T21:39:14.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.531+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db00694f0 con 0x7f8db01f8950 2024-04-26T21:39:14.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.532+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3710133801 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8da80043e0 con 0x7f8db01f8950 2024-04-26T21:39:14.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.532+0000 7f8da6ffd640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d9006c8f0 0x7f8d9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:14.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.532+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3710133801 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f8da808c930 con 0x7f8db01f8950 2024-04-26T21:39:14.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.532+0000 7f8dae59c640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d9006c8f0 0x7f8d9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:14.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.533+0000 7f8dae59c640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d9006c8f0 0x7f8d9006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8da0034000 tx=0x7f8da002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.536+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3710133801 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8da801d340 con 0x7f8db01f8950 2024-04-26T21:39:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.761+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8db0075540 con 0x7f8db01f8950 2024-04-26T21:39:14.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.762+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3710133801 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5187 (secure 0 0 0) 0x7f8da805aab0 con 0x7f8db01f8950 2024-04-26T21:39:14.762 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:39:14.763 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6975,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3716769386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3716769386},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716769386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:39:13.615988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6540,"qdb_cluster":[6540]},"id":4}]} 2024-04-26T21:39:14.763 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 62 2024-04-26T21:39:14.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d9006c8f0 msgr2=0x7f8d9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:14.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d9006c8f0 0x7f8d9006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8da0034000 tx=0x7f8da002e000 comp rx=0 tx=0).stop 2024-04-26T21:39:14.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db01f8950 msgr2=0x7f8db00c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:14.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db01f8950 0x7f8db00c9770 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8da800cd70 tx=0x7f8da800a660 comp rx=0 tx=0).stop 2024-04-26T21:39:14.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 shutdown_connections 2024-04-26T21:39:14.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d9006c8f0 0x7f8d9006edb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db01f8950 0x7f8db00c9770 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db015f380 0x7f8db006f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 --2- 172.21.15.40:0/3710133801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db00d0620 0x7f8db00c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:14.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 >> 172.21.15.40:0/3710133801 conn(0x7f8db00d0be0 msgr2=0x7f8db00b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:14.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 shutdown_connections 2024-04-26T21:39:14.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:14.764+0000 7f8db4a93640 1 -- 172.21.15.40:0/3710133801 wait complete. 2024-04-26T21:39:14.922 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:39:14.922+0000 7f41571d2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:39:15.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:38:18.310357+0000', 'modified': '2024-04-26T21:39:13.615988+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6540}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6540': {'gid': 6540, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.40:6834/425620740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 425620740}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 425620740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6540, 'qdb_cluster': [6540]} 2024-04-26T21:39:15.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:39:15.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:39:15.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.282+0000 7f5acd693640 1 Processor -- start 2024-04-26T21:39:15.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.282+0000 7f5acd693640 1 -- start start 2024-04-26T21:39:15.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5acd693640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ac80b3860 0x7f5ac816a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5acd693640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac80d0ba0 con 0x7f5ac80d06f0 2024-04-26T21:39:15.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5acd693640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac80d2590 con 0x7f5ac815f3a0 2024-04-26T21:39:15.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5acd693640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac80d0e20 con 0x7f5ac80b3860 2024-04-26T21:39:15.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5ac77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ac80b3860 0x7f5ac816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5abffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5ac80d06f0 0x7f5ac80d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35772/0 (socket says 172.21.15.40:35772) 2024-04-26T21:39:15.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5abffff640 1 -- 172.21.15.40:0/1443787195 learned_addr learned my addr 172.21.15.40:0/1443787195 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:39:15.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5ac77fe640 1 -- 172.21.15.40:0/1443787195 >> v1:172.21.15.184:6789/0 conn(0x7f5ac815f3a0 legacy=0x7f5ac80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5ac77fe640 1 -- 172.21.15.40:0/1443787195 >> v1:172.21.15.40:6789/0 conn(0x7f5ac80d06f0 legacy=0x7f5ac80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5ac67fc640 1 -- 172.21.15.40:0/1443787195 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045163408 0 0) 0x7f5ab0001110 con 0x7f5ac80d06f0 2024-04-26T21:39:15.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.283+0000 7f5ac77fe640 1 -- 172.21.15.40:0/1443787195 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac80b1d70 con 0x7f5ac80b3860 2024-04-26T21:39:15.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.284+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/1443787195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac80b3860 0x7f5ac816a000 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009d90 tx=0x7f5ac002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72ea9103403cde5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:15.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.284+0000 7f5ac67fc640 1 -- 172.21.15.40:0/1443787195 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0031600 con 0x7f5ac80b3860 2024-04-26T21:39:15.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.284+0000 7f5ac67fc640 1 -- 172.21.15.40:0/1443787195 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac00317a0 con 0x7f5ac80b3860 2024-04-26T21:39:15.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.284+0000 7f5ac67fc640 1 -- 172.21.15.40:0/1443787195 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0031b70 con 0x7f5ac80b3860 2024-04-26T21:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.285+0000 7f5acd693640 1 -- 172.21.15.40:0/1443787195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac80b3860 msgr2=0x7f5ac816a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.285+0000 7f5acd693640 1 --2- 172.21.15.40:0/1443787195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac80b3860 0x7f5ac816a000 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009d90 tx=0x7f5ac002a610 comp rx=0 tx=0).stop 2024-04-26T21:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.285+0000 7f5acd693640 1 -- 172.21.15.40:0/1443787195 shutdown_connections 2024-04-26T21:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.285+0000 7f5acd693640 1 --2- 172.21.15.40:0/1443787195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac80b3860 0x7f5ac816a000 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.285+0000 7f5acd693640 1 -- 172.21.15.40:0/1443787195 >> 172.21.15.40:0/1443787195 conn(0x7f5ac80d4f90 msgr2=0x7f5ac80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.285+0000 7f5acd693640 1 -- 172.21.15.40:0/1443787195 shutdown_connections 2024-04-26T21:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.285+0000 7f5acd693640 1 -- 172.21.15.40:0/1443787195 wait complete. 2024-04-26T21:39:15.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.286+0000 7f5acd693640 1 Processor -- start 2024-04-26T21:39:15.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.286+0000 7f5acd693640 1 -- start start 2024-04-26T21:39:15.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5acd693640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ac80b3860 0x7f5ac80a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5acd693640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ac80d06f0 0x7f5ac809ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5acd693640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac815f3a0 0x7f5ac809f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5acd693640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac815f780 con 0x7f5ac815f3a0 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5acd693640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80b2330 con 0x7f5ac80b3860 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5acd693640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac815f900 con 0x7f5ac80d06f0 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac815f3a0 0x7f5ac809f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac815f3a0 0x7f5ac809f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41006/0 (socket says 172.21.15.40:41006) 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 -- 172.21.15.40:0/3943143071 learned_addr learned my addr 172.21.15.40:0/3943143071 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5abffff640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ac80d06f0 0x7f5ac809ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ac80b3860 0x7f5ac80a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 -- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ac80d06f0 msgr2=0x7f5ac809ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ac80d06f0 0x7f5ac809ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 -- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ac80b3860 msgr2=0x7f5ac80a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ac80b3860 0x7f5ac80a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 -- 172.21.15.40:0/3943143071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac81f9730 con 0x7f5ac815f3a0 2024-04-26T21:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ac80b3860 0x7f5ac80a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac815f3a0 0x7f5ac809f1b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009eb0 tx=0x7f5ac0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5abf7fe640 1 -- 172.21.15.40:0/3943143071 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0040070 con 0x7f5ac815f3a0 2024-04-26T21:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5abf7fe640 1 -- 172.21.15.40:0/3943143071 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac0030be0 con 0x7f5ac815f3a0 2024-04-26T21:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.287+0000 7f5acd693640 1 -- 172.21.15.40:0/3943143071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac81f8970 con 0x7f5ac815f3a0 2024-04-26T21:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.288+0000 7f5abf7fe640 1 -- 172.21.15.40:0/3943143071 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0009450 con 0x7f5ac815f3a0 2024-04-26T21:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.288+0000 7f5acd693640 1 -- 172.21.15.40:0/3943143071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81f8f30 con 0x7f5ac815f3a0 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.289+0000 7f5abf7fe640 1 -- 172.21.15.40:0/3943143071 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ac0039040 con 0x7f5ac815f3a0 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.289+0000 7f5abf7fe640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5aa806c8f0 0x7f5aa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.290+0000 7f5abf7fe640 1 -- 172.21.15.40:0/3943143071 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f5ac00acf40 con 0x7f5ac815f3a0 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.290+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5aa806c8f0 0x7f5aa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.290+0000 7f5ad03b4740 1 Processor -- start 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.290+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5aa806c8f0 0x7f5aa806edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800a830 tx=0x7f5ab8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5ad03b4740 1 -- start start 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a4c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a6a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a22120 0x55ada8a683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ada8986740 con 0x55ada8a69ce0 2024-04-26T21:39:15.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ada89223a0 con 0x55ada8a4c440 2024-04-26T21:39:15.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ada8982700 con 0x55ada8a22120 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5abffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a6a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5ac6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a4c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5abffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a6a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41016/0 (socket says 172.21.15.40:41016) 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.291+0000 7f5abffff640 1 -- 172.21.15.40:0/2296026131 learned_addr learned my addr 172.21.15.40:0/2296026131 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a22120 0x55ada8a683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abffff640 1 -- 172.21.15.40:0/2296026131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a22120 msgr2=0x55ada8a683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abffff640 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a22120 0x55ada8a683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abffff640 1 -- 172.21.15.40:0/2296026131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 msgr2=0x55ada8a4c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abffff640 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a4c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abffff640 1 -- 172.21.15.40:0/2296026131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ada893e750 con 0x55ada8a69ce0 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a22120 0x55ada8a683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:39:15.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abffff640 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a6a230 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007260 tx=0x7f5ab0007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2431c9b9f0d3f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abcff9640 1 -- 172.21.15.40:0/2296026131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0019070 con 0x55ada8a69ce0 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abcff9640 1 -- 172.21.15.40:0/2296026131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0001690 con 0x55ada8a69ce0 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.292+0000 7f5abcff9640 1 -- 172.21.15.40:0/2296026131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0001a60 con 0x55ada8a69ce0 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2296026131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 msgr2=0x55ada8a6a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a6a230 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007260 tx=0x7f5ab0007720 comp rx=0 tx=0).stop 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2296026131 shutdown_connections 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a22120 0x55ada8a683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a6a230 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2296026131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a4c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2296026131 >> 172.21.15.40:0/2296026131 conn(0x55ada8a1ea50 msgr2=0x55ada8a67250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2296026131 shutdown_connections 2024-04-26T21:39:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2296026131 wait complete. 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.293+0000 7f5ad03b4740 1 -- start start 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5ad03b4740 1 Processor -- start 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a47e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a5f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a5f840 0x55ada8a682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ada8a22ec0 con 0x55ada8a69ce0 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ada890dfe0 con 0x55ada8a4c440 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ada8a1c980 con 0x55ada8a5f840 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5ad03b4740 10 client.0 _ll_register_callbacks cb 0x55ada8924150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5abffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a5f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5abffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a5f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41032/0 (socket says 172.21.15.40:41032) 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.294+0000 7f5abffff640 1 -- 172.21.15.40:0/4136949759 learned_addr learned my addr 172.21.15.40:0/4136949759 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a5f840 0x55ada8a682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a47e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5abffff640 1 -- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a5f840 msgr2=0x55ada8a682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5abffff640 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a5f840 0x55ada8a682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5abffff640 1 -- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 msgr2=0x55ada8a47e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5abffff640 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a47e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5abffff640 1 -- 172.21.15.40:0/4136949759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ada86982d0 con 0x55ada8a69ce0 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a47e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5abffff640 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a5f300 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007760 tx=0x7f5ab0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17beff2599c7520 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5aa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/4136949759 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0019070 con 0x55ada8a69ce0 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/4136949759 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f5ab000f050 con 0x55ada8a69ce0 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5ad03b4740 1 -- 172.21.15.40:0/4136949759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab0001690 con 0x55ada8a69ce0 2024-04-26T21:39:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/4136949759 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f5ab002a050 con 0x55ada8a69ce0 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5ad03b4740 10 client.7005 fetch_fsmap learned FSMap version 62 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5ad03b4740 1 -- 172.21.15.40:0/4136949759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55ada87626b0 con 0x55ada8a69ce0 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/4136949759 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0001690 con 0x55ada8a69ce0 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/4136949759 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f5ab002d020 con 0x55ada8a69ce0 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.295+0000 7f5ad03b4740 10 client.7005 fetch_fsmap finished waiting for FSMap version 62 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.296+0000 7f5ad03b4740 10 client.7005 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.296+0000 7f5ad03b4740 20 client.7005 populate_metadata read hostname 'smithi040' 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.296+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55ada8a77ac0 0x55ada8a7aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.296+0000 7f5ad03b4740 4 client.7005 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.296+0000 7f5ad03b4740 1 -- 172.21.15.40:0/4136949759 --> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ada8994350 con 0x55ada8a77ac0 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.296+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55ada8a77ac0 0x55ada8a7aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.296+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55ada8a77ac0 0x55ada8a7aee0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc65a908fae64ca2 server_cookie=5f3934014c273535 in_seq=0 out_seq=0 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.296+0000 7f5aa5ffb640 10 client.7005 ms_handle_connect on v2:172.21.15.40:6833/425620740 2024-04-26T21:39:15.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/4136949759 <== mds.0 v2:172.21.15.40:6833/425620740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ada8994350 con 0x55ada8a77ac0 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5aa5ffb640 10 client.7005 handle_command_reply: tid=0 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 client.7005 shutdown 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 -- 172.21.15.40:0/4136949759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 msgr2=0x55ada8a5f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a5f300 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007760 tx=0x7f5ab0000ed0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 -- 172.21.15.40:0/4136949759 shutdown_connections 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55ada8a77ac0 0x55ada8a7aee0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a5f840 0x55ada8a682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5aa5ffb640 0 client.7005 ms_handle_reset on v2:172.21.15.40:6833/425620740 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a5f300 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/4136949759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a47e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 -- 172.21.15.40:0/4136949759 >> 172.21.15.40:0/4136949759 conn(0x55ada8a22120 msgr2=0x55ada8a6b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 -- 172.21.15.40:0/4136949759 shutdown_connections 2024-04-26T21:39:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 1 -- 172.21.15.40:0/4136949759 wait complete. 2024-04-26T21:39:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.297+0000 7f5ad03b4740 20 client.7005 trim_cache size 0 max 16384 2024-04-26T21:39:15.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.315+0000 7f5ad03b4740 1 Processor -- start 2024-04-26T21:39:15.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.315+0000 7f5ad03b4740 1 -- start start 2024-04-26T21:39:15.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a69450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a5f840 0x55ada8a65270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a69ce0 0x55ada8a657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8201190 con 0x55ada8a5f840 2024-04-26T21:39:15.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80a1910 con 0x55ada8a4c440 2024-04-26T21:39:15.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac8200c00 con 0x55ada8a69ce0 2024-04-26T21:39:15.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a5f840 0x55ada8a65270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a5f840 0x55ada8a65270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41034/0 (socket says 172.21.15.40:41034) 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 -- 172.21.15.40:0/520123802 learned_addr learned my addr 172.21.15.40:0/520123802 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a69450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a69ce0 0x55ada8a657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 -- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a69ce0 msgr2=0x55ada8a657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a69ce0 0x55ada8a657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 -- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 msgr2=0x55ada8a69450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a69450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 -- 172.21.15.40:0/520123802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ada8a5f670 con 0x55ada8a5f840 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a69ce0 0x55ada8a657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a69450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.316+0000 7f5abffff640 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a5f840 0x55ada8a65270 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007eb0 tx=0x7f5ab001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05606257e05b968 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5aa7fff640 1 -- 172.21.15.40:0/520123802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0001690 con 0x55ada8a5f840 2024-04-26T21:39:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5aa7fff640 1 -- 172.21.15.40:0/520123802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab001a040 con 0x55ada8a5f840 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5aa7fff640 1 -- 172.21.15.40:0/520123802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0025300 con 0x55ada8a5f840 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 -- 172.21.15.40:0/520123802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a5f840 msgr2=0x55ada8a65270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a5f840 0x55ada8a65270 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007eb0 tx=0x7f5ab001b000 comp rx=0 tx=0).stop 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 -- 172.21.15.40:0/520123802 shutdown_connections 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a69ce0 0x55ada8a657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a5f840 0x55ada8a65270 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/520123802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a4c440 0x55ada8a69450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 -- 172.21.15.40:0/520123802 >> 172.21.15.40:0/520123802 conn(0x55ada8a22120 msgr2=0x55ada8a21740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 -- 172.21.15.40:0/520123802 shutdown_connections 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.317+0000 7f5ad03b4740 1 -- 172.21.15.40:0/520123802 wait complete. 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.318+0000 7f5ad03b4740 1 -- start start 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.318+0000 7f5ad03b4740 1 Processor -- start 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.318+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a4c440 0x55ada8ac7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a5f840 0x55ada8ae54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a81250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ada8a77940 con 0x55ada8a69ce0 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ada8a5ee80 con 0x55ada8a5f840 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ada8a785e0 con 0x55ada8a4c440 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 10 client.0 _ll_register_callbacks cb 0x7f5ab0017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a81250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a81250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41036/0 (socket says 172.21.15.40:41036) 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 -- 172.21.15.40:0/2951680701 learned_addr learned my addr 172.21.15.40:0/2951680701 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5abffff640 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a5f840 0x55ada8ae54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a4c440 0x55ada8ac7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 -- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a4c440 msgr2=0x55ada8ac7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a4c440 0x55ada8ac7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 -- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a5f840 msgr2=0x55ada8ae54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a5f840 0x55ada8ae54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 -- 172.21.15.40:0/2951680701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ada8a1bb30 con 0x55ada8a69ce0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a4c440 0x55ada8ac7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ac77fe640 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a81250 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5ac002ac00 tx=0x7f5ac007ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e6bb42dc8623ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5aa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/2951680701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0049040 con 0x55ada8a69ce0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/2951680701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f5ac00810b0 con 0x55ada8a69ce0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2951680701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab001a040 con 0x55ada8a69ce0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/2951680701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f5ac00812b0 con 0x55ada8a69ce0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 10 client.7011 fetch_fsmap learned FSMap version 62 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.319+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2951680701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55ada8a692a0 con 0x55ada8a69ce0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/2951680701 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab001a040 con 0x55ada8a69ce0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/2951680701 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f5ac0083100 con 0x55ada8a69ce0 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 10 client.7011 fetch_fsmap finished waiting for FSMap version 62 2024-04-26T21:39:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 10 client.7011 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 20 client.7011 populate_metadata read hostname 'smithi040' 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55ada8ae1bf0 0x55ada8a4d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 4 client.7011 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2951680701 --> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] -- command(tid 0: {"prefix": "status"}) -- 0x55ada898e1c0 con 0x55ada8ae1bf0 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55ada8ae1bf0 0x55ada8a4d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ac6ffd640 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55ada8ae1bf0 0x55ada8a4d830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3b7c16b3b1b553c server_cookie=ace37c53e263aad0 in_seq=0 out_seq=0 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5aa5ffb640 10 client.7011 ms_handle_connect on v2:172.21.15.40:6833/425620740 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5aa5ffb640 1 -- 172.21.15.40:0/2951680701 <== mds.0 v2:172.21.15.40:6833/425620740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ada898e1c0 con 0x55ada8ae1bf0 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5aa5ffb640 10 client.7011 handle_command_reply: tid=0 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 client.7011 shutdown 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2951680701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 msgr2=0x55ada8a81250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a81250 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5ac002ac00 tx=0x7f5ac007ee20 comp rx=0 tx=0).stop 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2951680701 shutdown_connections 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55ada8ae1bf0 0x55ada8a4d830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ada8a69ce0 0x55ada8a81250 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ada8a5f840 0x55ada8ae54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5aa5ffb640 0 client.7011 ms_handle_reset on v2:172.21.15.40:6833/425620740 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 --2- 172.21.15.40:0/2951680701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ada8a4c440 0x55ada8ac7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2951680701 >> 172.21.15.40:0/2951680701 conn(0x55ada8a22120 msgr2=0x55ada8a60110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.320+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2951680701 shutdown_connections 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5ad03b4740 1 -- 172.21.15.40:0/2951680701 wait complete. 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5ad03b4740 20 client.7011 trim_cache size 0 max 16384 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:39:15.328 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6540, 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 4.7077959199999997, 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 62, 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 70, 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 70, 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 61.392599844000003 2024-04-26T21:39:15.329 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:39:15.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 -- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5aa806c8f0 msgr2=0x7f5aa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5aa806c8f0 0x7f5aa806edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800a830 tx=0x7f5ab8008040 comp rx=0 tx=0).stop 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 -- 172.21.15.40:0/3943143071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac815f3a0 msgr2=0x7f5ac809f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac815f3a0 0x7f5ac809f1b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009eb0 tx=0x7f5ac0004a40 comp rx=0 tx=0).stop 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 -- 172.21.15.40:0/3943143071 shutdown_connections 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5aa806c8f0 0x7f5aa806edb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac815f3a0 0x7f5ac809f1b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ac80d06f0 0x7f5ac809ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 --2- 172.21.15.40:0/3943143071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ac80b3860 0x7f5ac80a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 -- 172.21.15.40:0/3943143071 >> 172.21.15.40:0/3943143071 conn(0x7f5ac80d4f90 msgr2=0x7f5ac80d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:39:15.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 -- 172.21.15.40:0/3943143071 shutdown_connections 2024-04-26T21:39:15.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:15.321+0000 7f5acd693640 1 -- 172.21.15.40:0/3943143071 wait complete. 2024-04-26T21:39:16.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-26T21:39:28.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:28.215+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:28.215+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-04-26T21:39:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:28.215+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:39:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:28.215+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:39:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:28.215+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-26T21:39:40.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:39:40.894 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:39:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:43.224+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:43.224+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-04-26T21:39:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:43.224+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:39:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:43.224+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:39:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:43.224+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-26T21:39:58.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:58.234+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:39:58.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:58.234+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-04-26T21:39:58.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:58.234+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:39:58.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:58.234+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:39:58.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:39:58.234+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-26T21:40:11.046 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:40:11.049 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:40:13.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:13.235+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:13.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:13.235+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-04-26T21:40:13.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:13.235+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:40:13.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:13.235+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:40:13.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:13.235+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-26T21:40:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:28.243+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:28.243+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-04-26T21:40:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:28.243+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:40:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:28.243+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:40:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:28.243+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-26T21:40:41.318 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:40:41.322 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:40:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:43.256+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:43.257+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-04-26T21:40:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:43.257+0000 7f36b5cdb640 1 -- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 msgr2=0x56135e369f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:40:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:43.257+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:40:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:43.257+0000 7f36b5cdb640 1 --2- 172.21.15.40:0/12978009 >> [v2:172.21.15.40:6832/1699369057,v1:172.21.15.40:6835/1699369057] conn(0x56135e315f60 0x56135e369f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-26T21:40:47.618 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-26T21:40:47.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:40:47.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.730+0000 7f26e75f0640 1 Processor -- start 2024-04-26T21:40:47.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.730+0000 7f26e75f0640 1 -- start start 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.731+0000 7f26e75f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e006f790 0x7f26e006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.731+0000 7f26e75f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e0156840 con 0x7f26e0156460 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.731+0000 7f26e75f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e00cbf20 con 0x7f26e0072c50 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.731+0000 7f26e75f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26e00cc200 con 0x7f26e006f790 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.731+0000 7f26e538c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f26e0156460 0x7f26e00cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52690/0 (socket says 172.21.15.40:52690) 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.731+0000 7f26e538c640 1 -- 172.21.15.40:0/462017846 learned_addr learned my addr 172.21.15.40:0/462017846 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e4b8b640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e006f790 0x7f26e006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123370361 0 0) 0x7f26e0156840 con 0x7f26e0156460 2024-04-26T21:40:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26cc005730 con 0x7f26e0156460 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3722225022 0 0) 0x7f26cc005730 con 0x7f26e0156460 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e006f790 msgr2=0x7f26e006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e006f790 0x7f26e006fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 >> v1:172.21.15.184:6789/0 conn(0x7f26e0072c50 legacy=0x7f26e00730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e00cc4e0 con 0x7f26e0156460 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f26dc001670 con 0x7f26e0156460 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 >> v1:172.21.15.40:6789/0 conn(0x7f26e0156460 legacy=0x7f26e00cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e006f790 0x7f26cc005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006130 0x7f26cc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc006f50 0x7f26cc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26e00cc200 con 0x7f26cc006f50 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc00dc30 con 0x7f26cc006130 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26cc00de30 con 0x7f26e006f790 2024-04-26T21:40:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e5b8d640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc006f50 0x7f26cc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e538c640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e006f790 0x7f26cc005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e4b8b640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006130 0x7f26cc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e5b8d640 1 -- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e006f790 msgr2=0x7f26cc005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e5b8d640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e006f790 0x7f26cc005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e5b8d640 1 -- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006130 msgr2=0x7f26cc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e5b8d640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006130 0x7f26cc006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.732+0000 7f26e5b8d640 1 -- 172.21.15.40:0/462017846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc005730 con 0x7f26cc006f50 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.733+0000 7f26e5b8d640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc006f50 0x7f26cc0073d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f26d8001f60 tx=0x7f26cc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9144c1d5ad12a8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.733+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8001b30 con 0x7f26cc006f50 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.733+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d8031620 con 0x7f26cc006f50 2024-04-26T21:40:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.733+0000 7f26d7fff640 1 -- 172.21.15.40:0/462017846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8031a10 con 0x7f26cc006f50 2024-04-26T21:40:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.734+0000 7f26e75f0640 1 -- 172.21.15.40:0/462017846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc006f50 msgr2=0x7f26cc0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.734+0000 7f26e75f0640 1 --2- 172.21.15.40:0/462017846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc006f50 0x7f26cc0073d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f26d8001f60 tx=0x7f26cc007a40 comp rx=0 tx=0).stop 2024-04-26T21:40:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.734+0000 7f26e5b8d640 1 -- 172.21.15.40:0/462017846 reap_dead start 2024-04-26T21:40:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.734+0000 7f26e75f0640 1 -- 172.21.15.40:0/462017846 shutdown_connections 2024-04-26T21:40:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.734+0000 7f26e75f0640 1 -- 172.21.15.40:0/462017846 >> 172.21.15.40:0/462017846 conn(0x7f26e00d0be0 msgr2=0x7f26e00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.734+0000 7f26e75f0640 1 -- 172.21.15.40:0/462017846 shutdown_connections 2024-04-26T21:40:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.734+0000 7f26e75f0640 1 -- 172.21.15.40:0/462017846 wait complete. 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.735+0000 7f26e75f0640 1 Processor -- start 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.735+0000 7f26e75f0640 1 -- start start 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e75f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006f50 0x7f26e014e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e75f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ae60 0x7f26e015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e75f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e01f4ee0 0x7f26e01f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e75f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26e00cbbe0 con 0x7f26e015ae60 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e75f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0072700 con 0x7f26cc006f50 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e75f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26e006e260 con 0x7f26e01f4ee0 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e4b8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ae60 0x7f26e015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e4b8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ae60 0x7f26e015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46038/0 (socket says 172.21.15.40:46038) 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e4b8b640 1 -- 172.21.15.40:0/1122203991 learned_addr learned my addr 172.21.15.40:0/1122203991 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e538c640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006f50 0x7f26e014e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.736+0000 7f26e5b8d640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e01f4ee0 0x7f26e01f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26e4b8b640 1 -- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e01f4ee0 msgr2=0x7f26e01f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26e4b8b640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e01f4ee0 0x7f26e01f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26e4b8b640 1 -- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006f50 msgr2=0x7f26e014e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26e4b8b640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006f50 0x7f26e014e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26e4b8b640 1 -- 172.21.15.40:0/1122203991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e01f5fa0 con 0x7f26e015ae60 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26e4b8b640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ae60 0x7f26e015b2e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f26d00119d0 tx=0x7f26d0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26d5ffb640 1 -- 172.21.15.40:0/1122203991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d000fb50 con 0x7f26e015ae60 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26d5ffb640 1 -- 172.21.15.40:0/1122203991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d000fcf0 con 0x7f26e015ae60 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26d5ffb640 1 -- 172.21.15.40:0/1122203991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d000fea0 con 0x7f26e015ae60 2024-04-26T21:40:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e01f6280 con 0x7f26e015ae60 2024-04-26T21:40:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.737+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e01f6840 con 0x7f26e015ae60 2024-04-26T21:40:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.738+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e015b820 con 0x7f26e015ae60 2024-04-26T21:40:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.739+0000 7f26d5ffb640 1 -- 172.21.15.40:0/1122203991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26d0012040 con 0x7f26e015ae60 2024-04-26T21:40:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.739+0000 7f26d5ffb640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c406c8f0 0x7f26c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.739+0000 7f26d5ffb640 1 -- 172.21.15.40:0/1122203991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f26d008c390 con 0x7f26e015ae60 2024-04-26T21:40:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.739+0000 7f26e538c640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c406c8f0 0x7f26c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.740+0000 7f26e538c640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c406c8f0 0x7f26c406edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f26dc001050 tx=0x7f26dc000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.744+0000 7f26d5ffb640 1 -- 172.21.15.40:0/1122203991 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26d0019030 con 0x7f26e015ae60 2024-04-26T21:40:47.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.971+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f26e00694f0 con 0x7f26e015ae60 2024-04-26T21:40:47.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.971+0000 7f26d5ffb640 1 -- 172.21.15.40:0/1122203991 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f26d0059ec0 con 0x7f26e015ae60 2024-04-26T21:40:47.972 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:40:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c406c8f0 msgr2=0x7f26c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c406c8f0 0x7f26c406edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f26dc001050 tx=0x7f26dc000f80 comp rx=0 tx=0).stop 2024-04-26T21:40:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ae60 msgr2=0x7f26e015b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ae60 0x7f26e015b2e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f26d00119d0 tx=0x7f26d0011e90 comp rx=0 tx=0).stop 2024-04-26T21:40:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 shutdown_connections 2024-04-26T21:40:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c406c8f0 0x7f26c406edb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e01f4ee0 0x7f26e01f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ae60 0x7f26e015b2e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 --2- 172.21.15.40:0/1122203991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26cc006f50 0x7f26e014e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 >> 172.21.15.40:0/1122203991 conn(0x7f26e00d0be0 msgr2=0x7f26e00d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 shutdown_connections 2024-04-26T21:40:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:47.973+0000 7f26e75f0640 1 -- 172.21.15.40:0/1122203991 wait complete. 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:40:47.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:40:47.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:47.986 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:40:48.002 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:40:48.002 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:40:48.003 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42400sec preferred_lft 42400sec 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:40:48.004 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:40:48.005 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:40:48.005 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:40:48.005 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:40:48.005 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:40:48.005 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:40:48.005 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:40:48.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:48.005 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:40:48.065 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:40:48.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:48.067 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:40:48.124 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:40:48.124 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:40:48.124 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:40:48.124 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:40:48.124 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:40:48.125 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:40:48.125 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:40:48.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:40:48.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:40:48.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:40:48.125 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:40:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:40:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:40:48.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:48.252 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:40:48.267 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:40:48.267 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:40:48.267 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:40:48.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:48.268 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:40:48.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:40:48.332 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:40:48.388 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:40:48.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:40:48.388 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:40:48.443 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:40:48.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:48.444 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:40:48.526 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:40:48.526 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:40:48.534 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:40:48.534 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:40:48.534 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:40:48.534 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:40:48.534 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:40:48.535 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:40:48.535 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:40:48.535 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:40:48.535 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:40:48.535 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:40:48.537 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:40:48.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:48.537 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:40:48.601 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:40:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:40:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:40:48.698 INFO:teuthology.orchestra.run.smithi040.stdout:client7020 2024-04-26T21:40:48.699 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:40:48.699 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:40:48.714 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-04-26T21:40:48.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout: File: a/ 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 2bh/43d Inode: 1099511627776 Links: 4 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1316/ ubuntu) 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:38:30.283567934 +0000 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:38:30.287567887 +0000 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:38:30.287567887 +0000 2024-04-26T21:40:48.807 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:38:30.283567934 +0000 2024-04-26T21:40:48.812 INFO:teuthology.orchestra.run.smithi040.stdout:a/ 2024-04-26T21:40:48.812 INFO:teuthology.orchestra.run.smithi040.stdout:a/c 2024-04-26T21:40:48.812 INFO:teuthology.orchestra.run.smithi040.stdout:a/c/d 2024-04-26T21:40:48.812 INFO:teuthology.orchestra.run.smithi040.stdout:a/b 2024-04-26T21:40:48.812 INFO:teuthology.orchestra.run.smithi040.stdout:a/b/d 2024-04-26T21:40:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:40:48.818 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-04-26T21:40:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.920+0000 7f21f3aa2640 1 Processor -- start 2024-04-26T21:40:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.921+0000 7f21f3aa2640 1 -- start start 2024-04-26T21:40:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f3aa2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f3aa2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d1b30 0x7f21ec0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f3aa2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec0d10d0 con 0x7f21ec0b2340 2024-04-26T21:40:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f3aa2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec0d0750 con 0x7f21ec0d1b30 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f3aa2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0d16a0 con 0x7f21ec162810 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f183e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f203f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d1b30 0x7f21ec0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f103d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f21ec0b2340 0x7f21ec0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:42558/0 (socket says 172.21.15.40:42558) 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f183e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:58458/0 (socket says 172.21.15.40:58458) 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f203f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d1b30 0x7f21ec0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48038/0 (socket says 172.21.15.40:48038) 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f103d640 1 -- 172.21.15.40:0/4167032241 learned_addr learned my addr 172.21.15.40:0/4167032241 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f083c640 1 -- 172.21.15.40:0/4167032241 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297413139 0 0) 0x7f21ec0d10d0 con 0x7f21ec0b2340 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f083c640 1 -- 172.21.15.40:0/4167032241 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21d4003520 con 0x7f21ec0b2340 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f183e640 1 -- 172.21.15.40:0/4167032241 >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d1b30 msgr2=0x7f21ec0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f183e640 1 --2- 172.21.15.40:0/4167032241 >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d1b30 0x7f21ec0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f183e640 1 -- 172.21.15.40:0/4167032241 >> v1:172.21.15.184:6789/0 conn(0x7f21ec0b2340 legacy=0x7f21ec0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f183e640 1 -- 172.21.15.40:0/4167032241 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0d1370 con 0x7f21ec162810 2024-04-26T21:40:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.922+0000 7f21f203f640 1 --2- 172.21.15.40:0/4167032241 >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d1b30 0x7f21ec0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:40:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.923+0000 7f21f183e640 1 --2- 172.21.15.40:0/4167032241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec070350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21e40097a0 tx=0x7f21e402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea4d23bd3399ce1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.923+0000 7f21f083c640 1 -- 172.21.15.40:0/4167032241 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4037070 con 0x7f21ec162810 2024-04-26T21:40:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.923+0000 7f21f083c640 1 -- 172.21.15.40:0/4167032241 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4032440 con 0x7f21ec162810 2024-04-26T21:40:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.923+0000 7f21f083c640 1 -- 172.21.15.40:0/4167032241 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40327f0 con 0x7f21ec162810 2024-04-26T21:40:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.924+0000 7f21f3aa2640 1 -- 172.21.15.40:0/4167032241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 msgr2=0x7f21ec070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.924+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/4167032241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec070350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21e40097a0 tx=0x7f21e402a180 comp rx=0 tx=0).stop 2024-04-26T21:40:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.924+0000 7f21f3aa2640 1 -- 172.21.15.40:0/4167032241 shutdown_connections 2024-04-26T21:40:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.924+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/4167032241 >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d1b30 0x7f21ec0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.924+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/4167032241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec070350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.924+0000 7f21f3aa2640 1 -- 172.21.15.40:0/4167032241 >> 172.21.15.40:0/4167032241 conn(0x7f21ec0d2da0 msgr2=0x7f21ec0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:48.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.924+0000 7f21f3aa2640 1 -- 172.21.15.40:0/4167032241 shutdown_connections 2024-04-26T21:40:48.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.924+0000 7f21f3aa2640 1 -- 172.21.15.40:0/4167032241 wait complete. 2024-04-26T21:40:48.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.925+0000 7f21f3aa2640 1 Processor -- start 2024-04-26T21:40:48.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.925+0000 7f21f3aa2640 1 -- start start 2024-04-26T21:40:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f3aa2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0b2340 0x7f21ec14c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f3aa2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec0d1b30 0x7f21ec149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f3aa2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec15d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f3aa2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec070890 con 0x7f21ec0d1b30 2024-04-26T21:40:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f3aa2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0d0470 con 0x7f21ec0b2340 2024-04-26T21:40:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f3aa2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0cdb00 con 0x7f21ec162810 2024-04-26T21:40:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f103d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec0d1b30 0x7f21ec149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f103d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec0d1b30 0x7f21ec149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54144/0 (socket says 172.21.15.40:54144) 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f103d640 1 -- 172.21.15.40:0/1650682838 learned_addr learned my addr 172.21.15.40:0/1650682838 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f203f640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f183e640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0b2340 0x7f21ec14c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f103d640 1 -- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 msgr2=0x7f21ec15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f103d640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec15d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f103d640 1 -- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0b2340 msgr2=0x7f21ec14c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.926+0000 7f21f103d640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0b2340 0x7f21ec14c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.927+0000 7f21f103d640 1 -- 172.21.15.40:0/1650682838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec14cc60 con 0x7f21ec0d1b30 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.927+0000 7f21f183e640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0b2340 0x7f21ec14c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.927+0000 7f21f103d640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec0d1b30 0x7f21ec149310 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f21e000c460 tx=0x7f21e000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.927+0000 7f21ea7fc640 1 -- 172.21.15.40:0/1650682838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0017070 con 0x7f21ec0d1b30 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.927+0000 7f21ea7fc640 1 -- 172.21.15.40:0/1650682838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e0005ce0 con 0x7f21ec0d1b30 2024-04-26T21:40:48.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.927+0000 7f21f3aa2640 1 -- 172.21.15.40:0/1650682838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec1f82f0 con 0x7f21ec0d1b30 2024-04-26T21:40:48.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.927+0000 7f21ea7fc640 1 -- 172.21.15.40:0/1650682838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0007290 con 0x7f21ec0d1b30 2024-04-26T21:40:48.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.927+0000 7f21f3aa2640 1 -- 172.21.15.40:0/1650682838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec1f88b0 con 0x7f21ec0d1b30 2024-04-26T21:40:48.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.928+0000 7f21ea7fc640 1 -- 172.21.15.40:0/1650682838 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21e001a070 con 0x7f21ec0d1b30 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.929+0000 7f21ea7fc640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.929+0000 7f21ea7fc640 1 -- 172.21.15.40:0/1650682838 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f21e008b1b0 con 0x7f21ec0d1b30 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.929+0000 7f21f183e640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.930+0000 7f21f183e640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21d806c8f0 0x7f21d806edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f21e4034000 tx=0x7f21e402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.930+0000 7f21f67bd740 1 Processor -- start 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.930+0000 7f21f67bd740 1 -- start start 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d26019af80 0x55d26019b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d260195e10 0x55d260199b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d26019a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f67bd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d2600ffd80 con 0x55d260195e10 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f67bd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d2600b98b0 con 0x55d26019af80 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f67bd740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d2600fbd40 con 0x55d26019a0c0 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d260195e10 0x55d260199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d260195e10 0x55d260199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54154/0 (socket says 172.21.15.40:54154) 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 -- 172.21.15.40:0/4135925711 learned_addr learned my addr 172.21.15.40:0/4135925711 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f183e640 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d26019af80 0x55d26019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f203f640 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d26019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 -- 172.21.15.40:0/4135925711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 msgr2=0x55d26019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d26019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 -- 172.21.15.40:0/4135925711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d26019af80 msgr2=0x55d26019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d26019af80 0x55d26019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 -- 172.21.15.40:0/4135925711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d26008aa40 con 0x55d260195e10 2024-04-26T21:40:48.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.931+0000 7f21f103d640 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d260195e10 0x55d260199b80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f21e000c430 tx=0x7f21e0059320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a199b9f460b0e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21d37fe640 1 -- 172.21.15.40:0/4135925711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e00711c0 con 0x55d260195e10 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21d37fe640 1 -- 172.21.15.40:0/4135925711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e005cfd0 con 0x55d260195e10 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21d37fe640 1 -- 172.21.15.40:0/4135925711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0061800 con 0x55d260195e10 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 -- 172.21.15.40:0/4135925711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d260195e10 msgr2=0x55d260199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d260195e10 0x55d260199b80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f21e000c430 tx=0x7f21e0059320 comp rx=0 tx=0).stop 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 -- 172.21.15.40:0/4135925711 shutdown_connections 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d26019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d260195e10 0x55d260199b80 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4135925711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d26019af80 0x55d26019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 -- 172.21.15.40:0/4135925711 >> 172.21.15.40:0/4135925711 conn(0x55d260198080 msgr2=0x55d2601e0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 -- 172.21.15.40:0/4135925711 shutdown_connections 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.932+0000 7f21f67bd740 1 -- 172.21.15.40:0/4135925711 wait complete. 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.933+0000 7f21f67bd740 1 -- start start 2024-04-26T21:40:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.933+0000 7f21f67bd740 1 Processor -- start 2024-04-26T21:40:48.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d2601a6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2601e5e10 0x55d2601e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f67bd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d2601c6b10 con 0x55d26019af80 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f67bd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d260086d10 con 0x55d2601e5e10 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f67bd740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d2601c5f90 con 0x55d26019a0c0 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f67bd740 10 client.0 _ll_register_callbacks cb 0x55d2600b3210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f103d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f103d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601e58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54166/0 (socket says 172.21.15.40:54166) 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f103d640 1 -- 172.21.15.40:0/4034743085 learned_addr learned my addr 172.21.15.40:0/4034743085 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f203f640 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2601e5e10 0x55d2601e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f183e640 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d2601a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f103d640 1 -- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 msgr2=0x55d2601a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f103d640 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d2601a6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f103d640 1 -- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2601e5e10 msgr2=0x55d2601e0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f103d640 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2601e5e10 0x55d2601e0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f103d640 1 -- 172.21.15.40:0/4034743085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25fedb7d0 con 0x55d26019af80 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.934+0000 7f21f183e640 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d2601a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f103d640 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601e58d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f21e0089470 tx=0x7f21e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598d65c13df2d0e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21d0ff9640 1 -- 172.21.15.40:0/4034743085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0075070 con 0x55d26019af80 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21d0ff9640 1 -- 172.21.15.40:0/4034743085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f21e0078070 con 0x55d26019af80 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 1 -- 172.21.15.40:0/4034743085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e005cfd0 con 0x55d26019af80 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21d0ff9640 1 -- 172.21.15.40:0/4034743085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f21e0078290 con 0x55d26019af80 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 10 client.7029 fetch_fsmap learned FSMap version 62 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 1 -- 172.21.15.40:0/4034743085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55d25fe113f0 con 0x55d26019af80 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21d0ff9640 1 -- 172.21.15.40:0/4034743085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e005cfd0 con 0x55d26019af80 2024-04-26T21:40:48.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21d0ff9640 1 -- 172.21.15.40:0/4034743085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f21e0061370 con 0x55d26019af80 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 10 client.7029 fetch_fsmap finished waiting for FSMap version 62 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 10 client.7029 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 20 client.7029 populate_metadata read hostname 'smithi040' 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55d2601e6cb0 0x55d2601e0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 4 client.7029 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f67bd740 1 -- 172.21.15.40:0/4034743085 --> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d26010d990 con 0x55d2601e6cb0 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.935+0000 7f21f183e640 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55d2601e6cb0 0x55d2601e0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f183e640 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55d2601e6cb0 0x55d2601e0950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecaaeb21dfd3eee server_cookie=ad61835b846a89fc in_seq=0 out_seq=0 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21d0ff9640 10 client.7029 ms_handle_connect on v2:172.21.15.40:6833/425620740 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21d0ff9640 1 -- 172.21.15.40:0/4034743085 <== mds.0 v2:172.21.15.40:6833/425620740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d26010d990 con 0x55d2601e6cb0 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21d0ff9640 10 client.7029 handle_command_reply: tid=0 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 client.7029 shutdown 2024-04-26T21:40:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 -- 172.21.15.40:0/4034743085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 msgr2=0x55d2601e58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601e58d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f21e0089470 tx=0x7f21e0000bc0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 -- 172.21.15.40:0/4034743085 shutdown_connections 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55d2601e6cb0 0x55d2601e0950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2601e5e10 0x55d2601e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601e58d0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 --2- 172.21.15.40:0/4034743085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d2601a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.936+0000 7f21f67bd740 1 -- 172.21.15.40:0/4034743085 >> 172.21.15.40:0/4034743085 conn(0x55d260195e10 msgr2=0x55d260195550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.937+0000 7f21d0ff9640 0 client.7029 ms_handle_reset on v2:172.21.15.40:6833/425620740 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.937+0000 7f21f67bd740 1 -- 172.21.15.40:0/4034743085 shutdown_connections 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.937+0000 7f21f67bd740 1 -- 172.21.15.40:0/4034743085 wait complete. 2024-04-26T21:40:48.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.937+0000 7f21f67bd740 20 client.7029 trim_cache size 0 max 16384 2024-04-26T21:40:48.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f67bd740 1 Processor -- start 2024-04-26T21:40:48.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f67bd740 1 -- start start 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019a0c0 0x55d26019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019af80 0x55d260253eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 0x55d26019b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f67bd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec201dd0 con 0x55d26019a0c0 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f67bd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec200e80 con 0x55d260196a80 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f67bd740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0cc6b0 con 0x55d26019af80 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f183e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019a0c0 0x55d26019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f183e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019a0c0 0x55d26019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54174/0 (socket says 172.21.15.40:54174) 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.952+0000 7f21f183e640 1 -- 172.21.15.40:0/2586422199 learned_addr learned my addr 172.21.15.40:0/2586422199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21f183e640 1 -- 172.21.15.40:0/2586422199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019af80 msgr2=0x55d260253eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21f183e640 1 --2- 172.21.15.40:0/2586422199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019af80 0x55d260253eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21f183e640 1 -- 172.21.15.40:0/2586422199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 msgr2=0x55d26019b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21f183e640 1 --2- 172.21.15.40:0/2586422199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 0x55d26019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21f183e640 1 -- 172.21.15.40:0/2586422199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2601991a0 con 0x55d26019a0c0 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21f183e640 1 --2- 172.21.15.40:0/2586422199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019a0c0 0x55d26019a7e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f21e402a770 tx=0x7f21e402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921a9bbc5964db66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21d2ffd640 1 -- 172.21.15.40:0/2586422199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4037040 con 0x55d26019a0c0 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21d2ffd640 1 -- 172.21.15.40:0/2586422199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4054070 con 0x55d26019a0c0 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21d2ffd640 1 -- 172.21.15.40:0/2586422199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4032c60 con 0x55d26019a0c0 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21f67bd740 1 -- 172.21.15.40:0/2586422199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019a0c0 msgr2=0x55d26019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.953+0000 7f21f67bd740 1 --2- 172.21.15.40:0/2586422199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019a0c0 0x55d26019a7e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f21e402a770 tx=0x7f21e402c000 comp rx=0 tx=0).stop 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 -- 172.21.15.40:0/2586422199 shutdown_connections 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 --2- 172.21.15.40:0/2586422199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 0x55d26019b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 --2- 172.21.15.40:0/2586422199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019af80 0x55d260253eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 --2- 172.21.15.40:0/2586422199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019a0c0 0x55d26019a7e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 -- 172.21.15.40:0/2586422199 >> 172.21.15.40:0/2586422199 conn(0x55d260195e10 msgr2=0x55d26019c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 -- 172.21.15.40:0/2586422199 shutdown_connections 2024-04-26T21:40:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 -- 172.21.15.40:0/2586422199 wait complete. 2024-04-26T21:40:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 -- start start 2024-04-26T21:40:48.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.954+0000 7f21f67bd740 1 Processor -- start 2024-04-26T21:40:48.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 0x55d2601e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d260251680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f67bd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f67bd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d2601c62e0 con 0x55d26019af80 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f67bd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec200830 con 0x55d260196a80 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f67bd740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d2601e0850 con 0x55d26019a0c0 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f67bd740 10 client.0 _ll_register_callbacks cb 0x7f21e0059f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f203f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f203f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601b7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54190/0 (socket says 172.21.15.40:54190) 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f203f640 1 -- 172.21.15.40:0/3213861124 learned_addr learned my addr 172.21.15.40:0/3213861124 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.955+0000 7f21f103d640 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d260251680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f183e640 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 0x55d2601e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f203f640 1 -- 172.21.15.40:0/3213861124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 msgr2=0x55d260251680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f203f640 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d260251680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f203f640 1 -- 172.21.15.40:0/3213861124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 msgr2=0x55d2601e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:48.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f203f640 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 0x55d2601e7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f203f640 1 -- 172.21.15.40:0/3213861124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25fdd10f0 con 0x55d26019af80 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f203f640 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601b7eb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x55d2601e6290 tx=0x7f21dc012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb476f6250bec72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21d0ff9640 1 -- 172.21.15.40:0/3213861124 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc01e070 con 0x55d26019af80 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21d0ff9640 1 -- 172.21.15.40:0/3213861124 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f21dc017640 con 0x55d26019af80 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 1 -- 172.21.15.40:0/3213861124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e4054070 con 0x55d26019af80 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21d0ff9640 1 -- 172.21.15.40:0/3213861124 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f21dc012e80 con 0x55d26019af80 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 10 client.7035 fetch_fsmap learned FSMap version 62 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 1 -- 172.21.15.40:0/3213861124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55d2600c04b0 con 0x55d26019af80 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21d0ff9640 1 -- 172.21.15.40:0/3213861124 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4054070 con 0x55d26019af80 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21d0ff9640 1 -- 172.21.15.40:0/3213861124 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f21dc02c080 con 0x55d26019af80 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 10 client.7035 fetch_fsmap finished waiting for FSMap version 62 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 10 client.7035 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-26T21:40:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 20 client.7035 populate_metadata read hostname 'smithi040' 2024-04-26T21:40:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55d2601dd6f0 0x55d2601b9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 4 client.7035 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T21:40:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.956+0000 7f21f67bd740 1 -- 172.21.15.40:0/3213861124 --> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d260107800 con 0x55d2601dd6f0 2024-04-26T21:40:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.957+0000 7f21f183e640 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55d2601dd6f0 0x55d2601b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.957+0000 7f21d0ff9640 10 client.7035 ms_handle_connect on v2:172.21.15.40:6833/425620740 2024-04-26T21:40:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.957+0000 7f21f183e640 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55d2601dd6f0 0x55d2601b9020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd8a1e33b77ff98 server_cookie=f832f3ff519a1715 in_seq=0 out_seq=0 2024-04-26T21:40:48.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21d0ff9640 1 -- 172.21.15.40:0/3213861124 <== mds.0 v2:172.21.15.40:6833/425620740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d260107800 con 0x55d2601dd6f0 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21d0ff9640 10 client.7035 handle_command_reply: tid=0 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 client.7035 shutdown 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 -- 172.21.15.40:0/3213861124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 msgr2=0x55d2601b7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601b7eb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x55d2601e6290 tx=0x7f21dc012610 comp rx=0 tx=0).stop 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 -- 172.21.15.40:0/3213861124 shutdown_connections 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.40:6833/425620740,v1:172.21.15.40:6834/425620740] conn(0x55d2601dd6f0 0x55d2601b9020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d26019af80 0x55d2601b7eb0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d26019a0c0 0x55d260251680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21d0ff9640 0 client.7035 ms_handle_reset on v2:172.21.15.40:6833/425620740 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 --2- 172.21.15.40:0/3213861124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d260196a80 0x55d2601e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 -- 172.21.15.40:0/3213861124 >> 172.21.15.40:0/3213861124 conn(0x55d260195e10 msgr2=0x55d26019b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 -- 172.21.15.40:0/3213861124 shutdown_connections 2024-04-26T21:40:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 1 -- 172.21.15.40:0/3213861124 wait complete. 2024-04-26T21:40:48.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.973+0000 7f21f67bd740 20 client.7035 trim_cache size 0 max 16384 2024-04-26T21:40:48.976 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:40:48.976 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-04-26T21:40:48.976 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0 2024-04-26T21:40:48.976 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:40:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 -- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21d806c8f0 msgr2=0x7f21d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21d806c8f0 0x7f21d806edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f21e4034000 tx=0x7f21e402e000 comp rx=0 tx=0).stop 2024-04-26T21:40:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 -- 172.21.15.40:0/1650682838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec0d1b30 msgr2=0x7f21ec149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec0d1b30 0x7f21ec149310 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f21e000c460 tx=0x7f21e000c920 comp rx=0 tx=0).stop 2024-04-26T21:40:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 -- 172.21.15.40:0/1650682838 shutdown_connections 2024-04-26T21:40:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec162810 0x7f21ec15d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec0d1b30 0x7f21ec149310 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 --2- 172.21.15.40:0/1650682838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0b2340 0x7f21ec14c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:48.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 -- 172.21.15.40:0/1650682838 >> 172.21.15.40:0/1650682838 conn(0x7f21ec0d2da0 msgr2=0x7f21ec0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:48.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 -- 172.21.15.40:0/1650682838 shutdown_connections 2024-04-26T21:40:48.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:48.974+0000 7f21f3aa2640 1 -- 172.21.15.40:0/1650682838 wait complete. 2024-04-26T21:40:48.986 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:40:48.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:40:49.026 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:40:49.070 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:40:49.070 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:40:49.070 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:40:49.071 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:40:49.086 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:40:49.086 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:40:49.142 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:40:49.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.283+0000 7f3495ec8640 1 Processor -- start 2024-04-26T21:40:49.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.283+0000 7f3495ec8640 1 -- start start 2024-04-26T21:40:49.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.283+0000 7f3495ec8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f349015f3a0 0x7f34900d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.283+0000 7f3495ec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 0x7f34900d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.283+0000 7f3495ec8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34900d25a0 con 0x7f34900d2130 2024-04-26T21:40:49.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.283+0000 7f3495ec8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f34900d0e20 con 0x7f349015f3a0 2024-04-26T21:40:49.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.283+0000 7f3495ec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34900b1d70 con 0x7f34900d1770 2024-04-26T21:40:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f34900d2130 0x7f34900d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43358/0 (socket says 172.21.15.40:43358) 2024-04-26T21:40:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348ffff640 1 -- 172.21.15.40:0/3735517533 learned_addr learned my addr 172.21.15.40:0/3735517533 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348effd640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 0x7f34900d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3735517533 >> v2:172.21.15.184:3300/0 conn(0x7f349015f3a0 0x7f34900d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 679570051 0 0) 0x7f3488001490 con 0x7f34900d2130 2024-04-26T21:40:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3478003520 con 0x7f34900d2130 2024-04-26T21:40:49.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3382228286 0 0) 0x7f3478003520 con 0x7f34900d2130 2024-04-26T21:40:49.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 msgr2=0x7f34900d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:49.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 0x7f34900d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 >> v2:172.21.15.184:3300/0 conn(0x7f349015f3a0 msgr2=0x7f34900d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:49.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 --2- 172.21.15.40:0/3735517533 >> v2:172.21.15.184:3300/0 conn(0x7f349015f3a0 0x7f34900d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34900b2050 con 0x7f34900d2130 2024-04-26T21:40:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f3488001bc0 con 0x7f34900d2130 2024-04-26T21:40:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 >> v1:172.21.15.40:6789/0 conn(0x7f34900d2130 legacy=0x7f34900d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34780039b0 0x7f3478003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3478004800 0x7f3478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 0x7f34780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34900d0e20 con 0x7f34780039b0 2024-04-26T21:40:49.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34900b1d70 con 0x7f3478004800 2024-04-26T21:40:49.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f347800b9c0 con 0x7f34900d1770 2024-04-26T21:40:49.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34780039b0 0x7f3478003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348effd640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3478004800 0x7f3478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.284+0000 7f348ffff640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 0x7f34780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348f7fe640 1 -- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 msgr2=0x7f34780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 0x7f34780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348f7fe640 1 -- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3478004800 msgr2=0x7f3478004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3478004800 0x7f3478004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348f7fe640 1 -- 172.21.15.40:0/3735517533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3478003520 con 0x7f34780039b0 2024-04-26T21:40:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348ffff640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34900d1770 0x7f34780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348effd640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3478004800 0x7f3478004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34780039b0 0x7f3478003dd0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3480009ac0 tx=0x7f3478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cb820804611c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480036070 con 0x7f34780039b0 2024-04-26T21:40:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3480031400 con 0x7f34780039b0 2024-04-26T21:40:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.285+0000 7f348e7fc640 1 -- 172.21.15.40:0/3735517533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480009690 con 0x7f34780039b0 2024-04-26T21:40:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.286+0000 7f3495ec8640 1 -- 172.21.15.40:0/3735517533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34780039b0 msgr2=0x7f3478003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.286+0000 7f3495ec8640 1 --2- 172.21.15.40:0/3735517533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34780039b0 0x7f3478003dd0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3480009ac0 tx=0x7f3478005830 comp rx=0 tx=0).stop 2024-04-26T21:40:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.286+0000 7f348ffff640 1 -- 172.21.15.40:0/3735517533 reap_dead start 2024-04-26T21:40:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.286+0000 7f3495ec8640 1 -- 172.21.15.40:0/3735517533 shutdown_connections 2024-04-26T21:40:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.286+0000 7f3495ec8640 1 -- 172.21.15.40:0/3735517533 >> 172.21.15.40:0/3735517533 conn(0x7f34900d4f90 msgr2=0x7f34900d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.286+0000 7f3495ec8640 1 -- 172.21.15.40:0/3735517533 shutdown_connections 2024-04-26T21:40:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.286+0000 7f3495ec8640 1 -- 172.21.15.40:0/3735517533 wait complete. 2024-04-26T21:40:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.287+0000 7f3495ec8640 1 Processor -- start 2024-04-26T21:40:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.287+0000 7f3495ec8640 1 -- start start 2024-04-26T21:40:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f3495ec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34780039b0 0x7f34901f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f3495ec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34900b7bf0 0x7f34900b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f3495ec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34900cac10 0x7f34900cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f3495ec8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f349015f900 con 0x7f34900b7bf0 2024-04-26T21:40:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f3495ec8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f349015f780 con 0x7f34900cac10 2024-04-26T21:40:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f3495ec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34900b3860 con 0x7f34780039b0 2024-04-26T21:40:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f348effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34900b7bf0 0x7f34900b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f348effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34900b7bf0 0x7f34900b8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54206/0 (socket says 172.21.15.40:54206) 2024-04-26T21:40:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f348effd640 1 -- 172.21.15.40:0/3910234218 learned_addr learned my addr 172.21.15.40:0/3910234218 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34780039b0 0x7f34901f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.288+0000 7f348ffff640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34900cac10 0x7f34900cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f348effd640 1 -- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34780039b0 msgr2=0x7f34901f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f348effd640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34780039b0 0x7f34901f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f348effd640 1 -- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34900cac10 msgr2=0x7f34900cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f348effd640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34900cac10 0x7f34900cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f348effd640 1 -- 172.21.15.40:0/3910234218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34900bad10 con 0x7f34900b7bf0 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34780039b0 0x7f34901f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f348effd640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34900b7bf0 0x7f34900b8070 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3484011730 tx=0x7f3484011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f3477fff640 1 -- 172.21.15.40:0/3910234218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348400f9c0 con 0x7f34900b7bf0 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f3477fff640 1 -- 172.21.15.40:0/3910234218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348400fb60 con 0x7f34900b7bf0 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f3477fff640 1 -- 172.21.15.40:0/3910234218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484015460 con 0x7f34900b7bf0 2024-04-26T21:40:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34900ba750 con 0x7f34900b7bf0 2024-04-26T21:40:49.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.289+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349014be80 con 0x7f34900b7bf0 2024-04-26T21:40:49.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.290+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34900694f0 con 0x7f34900b7bf0 2024-04-26T21:40:49.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.290+0000 7f3477fff640 1 -- 172.21.15.40:0/3910234218 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3484015600 con 0x7f34900b7bf0 2024-04-26T21:40:49.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.291+0000 7f3477fff640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347c06c8f0 0x7f347c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.291+0000 7f3477fff640 1 -- 172.21.15.40:0/3910234218 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f348408c7a0 con 0x7f34900b7bf0 2024-04-26T21:40:49.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.291+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347c06c8f0 0x7f347c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.292+0000 7f348f7fe640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347c06c8f0 0x7f347c06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3480033000 tx=0x7f3480008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.295+0000 7f3477fff640 1 -- 172.21.15.40:0/3910234218 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3484019030 con 0x7f34900b7bf0 2024-04-26T21:40:49.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.481+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34900b6860 con 0x7f34900b7bf0 2024-04-26T21:40:49.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.482+0000 7f3477fff640 1 -- 172.21.15.40:0/3910234218 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17299 (secure 0 0 0) 0x7f348405a9b0 con 0x7f34900b7bf0 2024-04-26T21:40:49.484 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:40:49.484 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":70,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:39:10.611194+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6832/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6835/1667529627":"2024-04-27T21:38:35.525579+0000","172.21.15.40:6832/1667529627":"2024-04-27T21:38:35.525579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:40:49.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347c06c8f0 msgr2=0x7f347c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347c06c8f0 0x7f347c06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3480033000 tx=0x7f3480008030 comp rx=0 tx=0).stop 2024-04-26T21:40:49.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34900b7bf0 msgr2=0x7f34900b8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34900b7bf0 0x7f34900b8070 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3484011730 tx=0x7f3484011bf0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 shutdown_connections 2024-04-26T21:40:49.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f347c06c8f0 0x7f347c06edb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34900cac10 0x7f34900cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34900b7bf0 0x7f34900b8070 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 --2- 172.21.15.40:0/3910234218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34780039b0 0x7f34901f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 >> 172.21.15.40:0/3910234218 conn(0x7f34900d4f90 msgr2=0x7f34900d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:49.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 shutdown_connections 2024-04-26T21:40:49.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.484+0000 7f3495ec8640 1 -- 172.21.15.40:0/3910234218 wait complete. 2024-04-26T21:40:49.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:38:16.583866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:38:17.253849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:40:49.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:40:49.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.599+0000 7fc126e3f640 1 Processor -- start 2024-04-26T21:40:49.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.599+0000 7fc126e3f640 1 -- start start 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.600+0000 7fc126e3f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc120071e60 0x7fc12006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.600+0000 7fc126e3f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc12006fe20 0x7fc120072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.600+0000 7fc126e3f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12015d7c0 con 0x7fc120072780 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.600+0000 7fc126e3f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc12015daa0 con 0x7fc120071e60 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.600+0000 7fc126e3f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc12015dd80 con 0x7fc12006fe20 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.600+0000 7fc1253dc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc120072780 0x7fc1200cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43370/0 (socket says 172.21.15.40:43370) 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.600+0000 7fc1253dc640 1 -- 172.21.15.40:0/2977388205 learned_addr learned my addr 172.21.15.40:0/2977388205 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc124bdb640 1 --2- 172.21.15.40:0/2977388205 >> v2:172.21.15.184:3300/0 conn(0x7fc120071e60 0x7fc12006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc11ffff640 1 --2- 172.21.15.40:0/2977388205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc12006fe20 0x7fc120072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc11f7fe640 1 -- 172.21.15.40:0/2977388205 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804046695 0 0) 0x7fc12015d7c0 con 0x7fc120072780 2024-04-26T21:40:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc11f7fe640 1 -- 172.21.15.40:0/2977388205 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc108003520 con 0x7fc120072780 2024-04-26T21:40:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc11f7fe640 1 -- 172.21.15.40:0/2977388205 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2077742654 0 0) 0x7fc108003520 con 0x7fc120072780 2024-04-26T21:40:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc11ffff640 1 -- 172.21.15.40:0/2977388205 >> v2:172.21.15.184:3300/0 conn(0x7fc120071e60 msgr2=0x7fc12006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc11ffff640 1 --2- 172.21.15.40:0/2977388205 >> v2:172.21.15.184:3300/0 conn(0x7fc120071e60 0x7fc12006f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc11ffff640 1 -- 172.21.15.40:0/2977388205 >> v1:172.21.15.40:6789/0 conn(0x7fc120072780 legacy=0x7fc1200cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.601+0000 7fc11ffff640 1 -- 172.21.15.40:0/2977388205 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12015e060 con 0x7fc12006fe20 2024-04-26T21:40:49.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.602+0000 7fc11ffff640 1 --2- 172.21.15.40:0/2977388205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc12006fe20 0x7fc120072240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097a0 tx=0x7fc11402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50280839a7f375f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.602+0000 7fc11f7fe640 1 -- 172.21.15.40:0/2977388205 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114037070 con 0x7fc12006fe20 2024-04-26T21:40:49.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.602+0000 7fc11f7fe640 1 -- 172.21.15.40:0/2977388205 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc11402aba0 con 0x7fc12006fe20 2024-04-26T21:40:49.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.602+0000 7fc11f7fe640 1 -- 172.21.15.40:0/2977388205 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110001670 con 0x7fc12006fe20 2024-04-26T21:40:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.603+0000 7fc126e3f640 1 -- 172.21.15.40:0/2977388205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc12006fe20 msgr2=0x7fc120072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.603+0000 7fc126e3f640 1 --2- 172.21.15.40:0/2977388205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc12006fe20 0x7fc120072240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097a0 tx=0x7fc11402a110 comp rx=0 tx=0).stop 2024-04-26T21:40:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.603+0000 7fc126e3f640 1 -- 172.21.15.40:0/2977388205 shutdown_connections 2024-04-26T21:40:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.603+0000 7fc126e3f640 1 --2- 172.21.15.40:0/2977388205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc12006fe20 0x7fc120072240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.603+0000 7fc126e3f640 1 --2- 172.21.15.40:0/2977388205 >> v2:172.21.15.184:3300/0 conn(0x7fc120071e60 0x7fc12006f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.603+0000 7fc126e3f640 1 -- 172.21.15.40:0/2977388205 >> 172.21.15.40:0/2977388205 conn(0x7fc1200d0be0 msgr2=0x7fc1200d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.603+0000 7fc126e3f640 1 -- 172.21.15.40:0/2977388205 shutdown_connections 2024-04-26T21:40:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.603+0000 7fc126e3f640 1 -- 172.21.15.40:0/2977388205 wait complete. 2024-04-26T21:40:49.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.604+0000 7fc126e3f640 1 Processor -- start 2024-04-26T21:40:49.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.604+0000 7fc126e3f640 1 -- start start 2024-04-26T21:40:49.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc126e3f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc12006fe20 0x7fc1201f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc126e3f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc120071e60 0x7fc1201f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc126e3f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc120072780 0x7fc1200af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc126e3f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc12015d3f0 con 0x7fc120071e60 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc126e3f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc120072c10 con 0x7fc12006fe20 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc126e3f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc12015d570 con 0x7fc120072780 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc120071e60 0x7fc1201f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc120071e60 0x7fc1201f8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54212/0 (socket says 172.21.15.40:54212) 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 -- 172.21.15.40:0/1346597010 learned_addr learned my addr 172.21.15.40:0/1346597010 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc1253dc640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc120072780 0x7fc1200af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc124bdb640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc12006fe20 0x7fc1201f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 -- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc120072780 msgr2=0x7fc1200af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc120072780 0x7fc1200af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 -- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc12006fe20 msgr2=0x7fc1201f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc12006fe20 0x7fc1201f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 -- 172.21.15.40:0/1346597010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1200b0240 con 0x7fc120071e60 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.605+0000 7fc11ffff640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc120071e60 0x7fc1201f8960 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc114035040 tx=0x7fc1140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.606+0000 7fc11d7fa640 1 -- 172.21.15.40:0/1346597010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114041030 con 0x7fc120071e60 2024-04-26T21:40:49.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.606+0000 7fc11d7fa640 1 -- 172.21.15.40:0/1346597010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc11403d420 con 0x7fc120071e60 2024-04-26T21:40:49.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.606+0000 7fc11d7fa640 1 -- 172.21.15.40:0/1346597010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114037040 con 0x7fc120071e60 2024-04-26T21:40:49.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.606+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1200aff60 con 0x7fc120071e60 2024-04-26T21:40:49.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.606+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1200c80a0 con 0x7fc120071e60 2024-04-26T21:40:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.607+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1200694f0 con 0x7fc120071e60 2024-04-26T21:40:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.607+0000 7fc11d7fa640 1 -- 172.21.15.40:0/1346597010 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc114044070 con 0x7fc120071e60 2024-04-26T21:40:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.608+0000 7fc11d7fa640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc10c06c920 0x7fc10c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.608+0000 7fc11d7fa640 1 -- 172.21.15.40:0/1346597010 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7fc1140ac2e0 con 0x7fc120071e60 2024-04-26T21:40:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.608+0000 7fc124bdb640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc10c06c920 0x7fc10c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.608+0000 7fc124bdb640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc10c06c920 0x7fc10c06ede0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc1200b6840 tx=0x7fc11800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.612+0000 7fc11d7fa640 1 -- 172.21.15.40:0/1346597010 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc114044e40 con 0x7fc120071e60 2024-04-26T21:40:49.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.801+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc120159590 con 0x7fc120071e60 2024-04-26T21:40:49.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.801+0000 7fc11d7fa640 1 -- 172.21.15.40:0/1346597010 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fc11407a340 con 0x7fc120071e60 2024-04-26T21:40:49.802 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:40:49.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc10c06c920 msgr2=0x7fc10c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc10c06c920 0x7fc10c06ede0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc1200b6840 tx=0x7fc11800b040 comp rx=0 tx=0).stop 2024-04-26T21:40:49.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc120071e60 msgr2=0x7fc1201f8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc120071e60 0x7fc1201f8960 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc114035040 tx=0x7fc1140046c0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 shutdown_connections 2024-04-26T21:40:49.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc10c06c920 0x7fc10c06ede0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc120072780 0x7fc1200af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc120071e60 0x7fc1201f8960 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 --2- 172.21.15.40:0/1346597010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc12006fe20 0x7fc1201f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 >> 172.21.15.40:0/1346597010 conn(0x7fc1200d0be0 msgr2=0x7fc1200b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:49.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 shutdown_connections 2024-04-26T21:40:49.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.803+0000 7fc126e3f640 1 -- 172.21.15.40:0/1346597010 wait complete. 2024-04-26T21:40:49.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:40:49.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.918+0000 7fab9f02a640 1 Processor -- start 2024-04-26T21:40:49.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.918+0000 7fab9f02a640 1 -- start start 2024-04-26T21:40:49.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.918+0000 7fab9f02a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab980d1a00 0x7fab980d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.918+0000 7fab9f02a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 0x7fab981625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.918+0000 7fab9f02a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab980d0a10 con 0x7fab980b40d0 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.918+0000 7fab9f02a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fab980d0cf0 con 0x7fab980d1a00 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.918+0000 7fab9f02a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab980d0fd0 con 0x7fab980d2320 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.919+0000 7fab9cdc6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab980d1a00 0x7fab980d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.919+0000 7fab9cdc6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab980d1a00 0x7fab980d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54222/0 (socket says 172.21.15.40:54222) 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.919+0000 7fab9cdc6640 1 -- 172.21.15.40:0/3840400186 learned_addr learned my addr 172.21.15.40:0/3840400186 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.919+0000 7fab97fff640 1 --2- 172.21.15.40:0/3840400186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 0x7fab981625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.919+0000 7fab9cdc6640 1 -- 172.21.15.40:0/3840400186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 msgr2=0x7fab981625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.919+0000 7fab9cdc6640 1 --2- 172.21.15.40:0/3840400186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 0x7fab981625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.919+0000 7fab9cdc6640 1 -- 172.21.15.40:0/3840400186 >> v1:172.21.15.184:6789/0 conn(0x7fab980b40d0 legacy=0x7fab980d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.919+0000 7fab9cdc6640 1 -- 172.21.15.40:0/3840400186 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab980d40a0 con 0x7fab980d1a00 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9cdc6640 1 --2- 172.21.15.40:0/3840400186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980d1de0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009870 tx=0x7fab8c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed1c036fe0085d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab977fe640 1 -- 172.21.15.40:0/3840400186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c037070 con 0x7fab980d1a00 2024-04-26T21:40:49.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab977fe640 1 -- 172.21.15.40:0/3840400186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c032440 con 0x7fab980d1a00 2024-04-26T21:40:49.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab977fe640 1 -- 172.21.15.40:0/3840400186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c032810 con 0x7fab980d1a00 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9f02a640 1 -- 172.21.15.40:0/3840400186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 msgr2=0x7fab980d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9f02a640 1 --2- 172.21.15.40:0/3840400186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980d1de0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009870 tx=0x7fab8c02a180 comp rx=0 tx=0).stop 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9f02a640 1 -- 172.21.15.40:0/3840400186 shutdown_connections 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9f02a640 1 --2- 172.21.15.40:0/3840400186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 0x7fab981625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9f02a640 1 --2- 172.21.15.40:0/3840400186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980d1de0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9f02a640 1 -- 172.21.15.40:0/3840400186 >> 172.21.15.40:0/3840400186 conn(0x7fab980d4fd0 msgr2=0x7fab980d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9f02a640 1 -- 172.21.15.40:0/3840400186 shutdown_connections 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.920+0000 7fab9f02a640 1 -- 172.21.15.40:0/3840400186 wait complete. 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.921+0000 7fab9f02a640 1 Processor -- start 2024-04-26T21:40:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.922+0000 7fab9f02a640 1 -- start start 2024-04-26T21:40:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.922+0000 7fab9f02a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab980b40d0 0x7fab980cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.922+0000 7fab9f02a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.922+0000 7fab9f02a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 0x7fab980cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.922+0000 7fab9f02a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab98162b30 con 0x7fab980d1a00 2024-04-26T21:40:49.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.922+0000 7fab9f02a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab980b5d60 con 0x7fab980b40d0 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.922+0000 7fab9f02a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab98162cb0 con 0x7fab980d2320 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54224/0 (socket says 172.21.15.40:54224) 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 -- 172.21.15.40:0/1837285994 learned_addr learned my addr 172.21.15.40:0/1837285994 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab9cdc6640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab980b40d0 0x7fab980cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab9d5c7640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 0x7fab980cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 -- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 msgr2=0x7fab980cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 0x7fab980cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 -- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab980b40d0 msgr2=0x7fab980cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab980b40d0 0x7fab980cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 -- 172.21.15.40:0/1837285994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab980cefc0 con 0x7fab980d1a00 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab97fff640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980c9090 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fab8400c7f0 tx=0x7fab8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab957fa640 1 -- 172.21.15.40:0/1837285994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84015070 con 0x7fab980d1a00 2024-04-26T21:40:49.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab957fa640 1 -- 172.21.15.40:0/1837285994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab84004220 con 0x7fab980d1a00 2024-04-26T21:40:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab980b9980 con 0x7fab980d1a00 2024-04-26T21:40:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab957fa640 1 -- 172.21.15.40:0/1837285994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84007620 con 0x7fab980d1a00 2024-04-26T21:40:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.923+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab980ba1c0 con 0x7fab980d1a00 2024-04-26T21:40:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.924+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab980694f0 con 0x7fab980d1a00 2024-04-26T21:40:49.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.925+0000 7fab957fa640 1 -- 172.21.15.40:0/1837285994 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab84007a90 con 0x7fab980d1a00 2024-04-26T21:40:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.925+0000 7fab957fa640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab7c06c920 0x7fab7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.925+0000 7fab957fa640 1 -- 172.21.15.40:0/1837285994 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7fab8408bc50 con 0x7fab980d1a00 2024-04-26T21:40:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.925+0000 7fab9cdc6640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab7c06c920 0x7fab7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:49.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.926+0000 7fab9cdc6640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab7c06c920 0x7fab7c06ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fab8c034000 tx=0x7fab8c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:49.929+0000 7fab957fa640 1 -- 172.21.15.40:0/1837285994 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab84091020 con 0x7fab980d1a00 2024-04-26T21:40:50.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.115+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fab9806cdd0 con 0x7fab980d1a00 2024-04-26T21:40:50.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.116+0000 7fab957fa640 1 -- 172.21.15.40:0/1837285994 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7fab980b9980 con 0x7fab980d1a00 2024-04-26T21:40:50.117 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.117+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab7c06c920 msgr2=0x7fab7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.117+0000 7fab9f02a640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab7c06c920 0x7fab7c06ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fab8c034000 tx=0x7fab8c02e000 comp rx=0 tx=0).stop 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.117+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 msgr2=0x7fab980c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.117+0000 7fab9f02a640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980c9090 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fab8400c7f0 tx=0x7fab8400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.118+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 shutdown_connections 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.118+0000 7fab9f02a640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab7c06c920 0x7fab7c06ede0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.118+0000 7fab9f02a640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab980d2320 0x7fab980cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.118+0000 7fab9f02a640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab980d1a00 0x7fab980c9090 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.118+0000 7fab9f02a640 1 --2- 172.21.15.40:0/1837285994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab980b40d0 0x7fab980cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.118+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 >> 172.21.15.40:0/1837285994 conn(0x7fab980d4fd0 msgr2=0x7fab980d3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.118+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 shutdown_connections 2024-04-26T21:40:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.118+0000 7fab9f02a640 1 -- 172.21.15.40:0/1837285994 wait complete. 2024-04-26T21:40:50.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:40:50.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.233+0000 7f643f69b640 1 Processor -- start 2024-04-26T21:40:50.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.233+0000 7f643f69b640 1 -- start start 2024-04-26T21:40:50.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643f69b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f643815f170 0x7f64380d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643f69b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 0x7f64380d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643f69b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64380d1040 con 0x7f643815f550 2024-04-26T21:40:50.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643f69b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f64380c9430 con 0x7f643815f170 2024-04-26T21:40:50.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643f69b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64380c9680 con 0x7f64380d3bc0 2024-04-26T21:40:50.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643d437640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f643815f170 0x7f64380d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643d437640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f643815f170 0x7f64380d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54228/0 (socket says 172.21.15.40:54228) 2024-04-26T21:40:50.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643d437640 1 -- 172.21.15.40:0/3677943019 learned_addr learned my addr 172.21.15.40:0/3677943019 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:50.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643d437640 1 -- 172.21.15.40:0/3677943019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 msgr2=0x7f64380d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:50.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643d437640 1 --2- 172.21.15.40:0/3677943019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 0x7f64380d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643d437640 1 -- 172.21.15.40:0/3677943019 >> v1:172.21.15.184:6789/0 conn(0x7f643815f550 legacy=0x7f64380d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:50.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.234+0000 7f643d437640 1 -- 172.21.15.40:0/3677943019 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64380c9960 con 0x7f643815f170 2024-04-26T21:40:50.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.235+0000 7f643d437640 1 --2- 172.21.15.40:0/3677943019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380d5a80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f64340098a0 tx=0x7f643402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f80ffe17e762bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:50.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.235+0000 7f642ffff640 1 -- 172.21.15.40:0/3677943019 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434037070 con 0x7f643815f170 2024-04-26T21:40:50.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.235+0000 7f642ffff640 1 -- 172.21.15.40:0/3677943019 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6434032440 con 0x7f643815f170 2024-04-26T21:40:50.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.235+0000 7f642ffff640 1 -- 172.21.15.40:0/3677943019 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434032810 con 0x7f643815f170 2024-04-26T21:40:50.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.235+0000 7f643f69b640 1 -- 172.21.15.40:0/3677943019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 msgr2=0x7f64380d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.235+0000 7f643f69b640 1 --2- 172.21.15.40:0/3677943019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380d5a80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f64340098a0 tx=0x7f643402a250 comp rx=0 tx=0).stop 2024-04-26T21:40:50.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.236+0000 7f643f69b640 1 -- 172.21.15.40:0/3677943019 shutdown_connections 2024-04-26T21:40:50.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.236+0000 7f643f69b640 1 --2- 172.21.15.40:0/3677943019 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 0x7f64380d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.236+0000 7f643f69b640 1 --2- 172.21.15.40:0/3677943019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380d5a80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.236+0000 7f643f69b640 1 -- 172.21.15.40:0/3677943019 >> 172.21.15.40:0/3677943019 conn(0x7f64380d4f60 msgr2=0x7f64380d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:50.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.236+0000 7f643f69b640 1 -- 172.21.15.40:0/3677943019 shutdown_connections 2024-04-26T21:40:50.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.236+0000 7f643f69b640 1 -- 172.21.15.40:0/3677943019 wait complete. 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.237+0000 7f643f69b640 1 Processor -- start 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.237+0000 7f643f69b640 1 -- start start 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.237+0000 7f643f69b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 0x7f6438158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.237+0000 7f643f69b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643f69b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380ce750 0x7f643815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643f69b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f64380ca5c0 con 0x7f643815f170 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643f69b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64380ca440 con 0x7f64380ce750 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643f69b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64380ca740 con 0x7f64380d3bc0 2024-04-26T21:40:50.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54240/0 (socket says 172.21.15.40:54240) 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 -- 172.21.15.40:0/3131656379 learned_addr learned my addr 172.21.15.40:0/3131656379 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643d437640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 0x7f6438158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643dc38640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380ce750 0x7f643815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 -- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 msgr2=0x7f6438158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 0x7f6438158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 -- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380ce750 msgr2=0x7f643815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380ce750 0x7f643815b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 -- 172.21.15.40:0/3131656379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643815bca0 con 0x7f643815f170 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643cc36640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380ce210 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f6428009fd0 tx=0x7f642800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f642dffb640 1 -- 172.21.15.40:0/3131656379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428015020 con 0x7f643815f170 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f642dffb640 1 -- 172.21.15.40:0/3131656379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6428011830 con 0x7f643815f170 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643815bf80 con 0x7f643815f170 2024-04-26T21:40:50.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.238+0000 7f642dffb640 1 -- 172.21.15.40:0/3131656379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428011c00 con 0x7f643815f170 2024-04-26T21:40:50.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.239+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6438158fd0 con 0x7f643815f170 2024-04-26T21:40:50.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.239+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64380ceb60 con 0x7f643815f170 2024-04-26T21:40:50.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.240+0000 7f642dffb640 1 -- 172.21.15.40:0/3131656379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6428018020 con 0x7f643815f170 2024-04-26T21:40:50.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.240+0000 7f642dffb640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641c06c8f0 0x7f641c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.240+0000 7f642dffb640 1 -- 172.21.15.40:0/3131656379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f642808bcc0 con 0x7f643815f170 2024-04-26T21:40:50.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.240+0000 7f643d437640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641c06c8f0 0x7f641c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.241+0000 7f643d437640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641c06c8f0 0x7f641c06edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6434034000 tx=0x7f643402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:50.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.245+0000 7f642dffb640 1 -- 172.21.15.40:0/3131656379 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64380ceb60 con 0x7f643815f170 2024-04-26T21:40:50.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.433+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f64380694f0 con 0x7f643815f170 2024-04-26T21:40:50.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.433+0000 7f642dffb640 1 -- 172.21.15.40:0/3131656379 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7f642801f020 con 0x7f643815f170 2024-04-26T21:40:50.434 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641c06c8f0 msgr2=0x7f641c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641c06c8f0 0x7f641c06edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6434034000 tx=0x7f643402e000 comp rx=0 tx=0).stop 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 msgr2=0x7f64380ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380ce210 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f6428009fd0 tx=0x7f642800b870 comp rx=0 tx=0).stop 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 shutdown_connections 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f641c06c8f0 0x7f641c06edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380ce750 0x7f643815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643815f170 0x7f64380ce210 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 --2- 172.21.15.40:0/3131656379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64380d3bc0 0x7f6438158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 >> 172.21.15.40:0/3131656379 conn(0x7f64380d4f60 msgr2=0x7f643806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 shutdown_connections 2024-04-26T21:40:50.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.435+0000 7f643f69b640 1 -- 172.21.15.40:0/3131656379 wait complete. 2024-04-26T21:40:50.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:40:50.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.548+0000 7f577474e640 1 Processor -- start 2024-04-26T21:40:50.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.548+0000 7f577474e640 1 -- start start 2024-04-26T21:40:50.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f577474e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f576c0d3bc0 0x7f576c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f577474e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f576c0b5f20 0x7f576c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f577474e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f576c0b7e50 con 0x7f576c162880 2024-04-26T21:40:50.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f577474e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f576c0d0580 con 0x7f576c0d3bc0 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f577474e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f576c0d1800 con 0x7f576c0b5f20 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f5771ce9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f576c0d3bc0 0x7f576c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f5771ce9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f576c0d3bc0 0x7f576c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54242/0 (socket says 172.21.15.40:54242) 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f5771ce9640 1 -- 172.21.15.40:0/3200233229 learned_addr learned my addr 172.21.15.40:0/3200233229 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f5772ceb640 1 --2- 172.21.15.40:0/3200233229 >> v2:172.21.15.184:3300/0 conn(0x7f576c0b5f20 0x7f576c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f5771ce9640 1 -- 172.21.15.40:0/3200233229 >> v2:172.21.15.184:3300/0 conn(0x7f576c0b5f20 msgr2=0x7f576c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f5771ce9640 1 --2- 172.21.15.40:0/3200233229 >> v2:172.21.15.184:3300/0 conn(0x7f576c0b5f20 0x7f576c0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f5771ce9640 1 -- 172.21.15.40:0/3200233229 >> v1:172.21.15.184:6789/0 conn(0x7f576c162880 legacy=0x7f576c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.549+0000 7f5771ce9640 1 -- 172.21.15.40:0/3200233229 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c0d07d0 con 0x7f576c0d3bc0 2024-04-26T21:40:50.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.550+0000 7f5771ce9640 1 --2- 172.21.15.40:0/3200233229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0d2cb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f576c08f770 tx=0x7f576002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f32aceef34b64b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:50.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.550+0000 7f57714e8640 1 -- 172.21.15.40:0/3200233229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760037070 con 0x7f576c0d3bc0 2024-04-26T21:40:50.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.550+0000 7f57714e8640 1 -- 172.21.15.40:0/3200233229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5760032440 con 0x7f576c0d3bc0 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.550+0000 7f57714e8640 1 -- 172.21.15.40:0/3200233229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760032810 con 0x7f576c0d3bc0 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.551+0000 7f577474e640 1 -- 172.21.15.40:0/3200233229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 msgr2=0x7f576c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.551+0000 7f577474e640 1 --2- 172.21.15.40:0/3200233229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0d2cb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f576c08f770 tx=0x7f576002a230 comp rx=0 tx=0).stop 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.551+0000 7f577474e640 1 -- 172.21.15.40:0/3200233229 shutdown_connections 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.551+0000 7f577474e640 1 --2- 172.21.15.40:0/3200233229 >> v2:172.21.15.184:3300/0 conn(0x7f576c0b5f20 0x7f576c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.551+0000 7f577474e640 1 --2- 172.21.15.40:0/3200233229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0d2cb0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.551+0000 7f577474e640 1 -- 172.21.15.40:0/3200233229 >> 172.21.15.40:0/3200233229 conn(0x7f576c0d4f60 msgr2=0x7f576c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.551+0000 7f577474e640 1 -- 172.21.15.40:0/3200233229 shutdown_connections 2024-04-26T21:40:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.551+0000 7f577474e640 1 -- 172.21.15.40:0/3200233229 wait complete. 2024-04-26T21:40:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.552+0000 7f577474e640 1 Processor -- start 2024-04-26T21:40:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.552+0000 7f577474e640 1 -- start start 2024-04-26T21:40:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f577474e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f576c0b5f20 0x7f576c0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f577474e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f577474e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f576c162880 0x7f576c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f577474e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f576c162c60 con 0x7f576c0d3bc0 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f577474e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0d0d90 con 0x7f576c162880 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f577474e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f576c162de0 con 0x7f576c0b5f20 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5771ce9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5771ce9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54250/0 (socket says 172.21.15.40:54250) 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5771ce9640 1 -- 172.21.15.40:0/3407167308 learned_addr learned my addr 172.21.15.40:0/3407167308 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f57724ea640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f576c0b5f20 0x7f576c0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5772ceb640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f576c162880 0x7f576c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5771ce9640 1 -- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f576c0b5f20 msgr2=0x7f576c0b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5771ce9640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f576c0b5f20 0x7f576c0b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5771ce9640 1 -- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f576c162880 msgr2=0x7f576c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5771ce9640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f576c162880 0x7f576c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.553+0000 7f5771ce9640 1 -- 172.21.15.40:0/3407167308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c0b2f90 con 0x7f576c0d3bc0 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.554+0000 7f5771ce9640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0b5500 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f576002a270 tx=0x7f57600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.554+0000 7f576affd640 1 -- 172.21.15.40:0/3407167308 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760041020 con 0x7f576c0d3bc0 2024-04-26T21:40:50.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.554+0000 7f576affd640 1 -- 172.21.15.40:0/3407167308 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57600306c0 con 0x7f576c0d3bc0 2024-04-26T21:40:50.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.554+0000 7f576affd640 1 -- 172.21.15.40:0/3407167308 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760037040 con 0x7f576c0d3bc0 2024-04-26T21:40:50.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.554+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c0b3270 con 0x7f576c0d3bc0 2024-04-26T21:40:50.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.554+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c0b3830 con 0x7f576c0d3bc0 2024-04-26T21:40:50.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.555+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576c0694f0 con 0x7f576c0d3bc0 2024-04-26T21:40:50.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.555+0000 7f576affd640 1 -- 172.21.15.40:0/3407167308 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5760044020 con 0x7f576c0d3bc0 2024-04-26T21:40:50.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.556+0000 7f576affd640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f575806c8f0 0x7f575806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:50.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.556+0000 7f576affd640 1 -- 172.21.15.40:0/3407167308 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f57600ac560 con 0x7f576c0d3bc0 2024-04-26T21:40:50.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.556+0000 7f57724ea640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f575806c8f0 0x7f575806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:50.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.557+0000 7f57724ea640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f575806c8f0 0x7f575806edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5764004f60 tx=0x7f5764007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:50.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.560+0000 7f576affd640 1 -- 172.21.15.40:0/3407167308 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f576004b030 con 0x7f576c0d3bc0 2024-04-26T21:40:50.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.759+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f576c0d3550 con 0x7f576c0d3bc0 2024-04-26T21:40:50.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.759+0000 7f576affd640 1 -- 172.21.15.40:0/3407167308 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v70) ==== 80+0+244 (secure 0 0 0) 0x7f576007a770 con 0x7f576c0d3bc0 2024-04-26T21:40:50.761 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1699369057 2024-04-27T21:39:10.611149+0000 2024-04-26T21:40:50.761 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1699369057 2024-04-27T21:39:10.611149+0000 2024-04-26T21:40:50.761 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1667529627 2024-04-27T21:38:35.525579+0000 2024-04-26T21:40:50.761 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1667529627 2024-04-27T21:38:35.525579+0000 2024-04-26T21:40:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:listed 4 entries 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f575806c8f0 msgr2=0x7f575806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f575806c8f0 0x7f575806edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5764004f60 tx=0x7f5764007e30 comp rx=0 tx=0).stop 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 msgr2=0x7f576c0b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0b5500 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f576002a270 tx=0x7f57600046c0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 shutdown_connections 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f575806c8f0 0x7f575806edb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f576c162880 0x7f576c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f576c0d3bc0 0x7f576c0b5500 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.761+0000 7f577474e640 1 --2- 172.21.15.40:0/3407167308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f576c0b5f20 0x7f576c0b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.762+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 >> 172.21.15.40:0/3407167308 conn(0x7f576c0d4f60 msgr2=0x7f576c0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.762+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 shutdown_connections 2024-04-26T21:40:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:50.762+0000 7f577474e640 1 -- 172.21.15.40:0/3407167308 wait complete. 2024-04-26T21:40:50.773 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:40:50.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:50.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:40:50.801 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:40:50.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:40:50.802 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:40:50.857 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:40:50.857 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:40:50.857 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:40:50.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:40:50.875 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:40:50.876 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:40:50.876 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:40:50.913 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:40:50.914 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:40:50.914 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:40:50.914 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:40:50.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:40:50.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:40:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.053+0000 7fac2a4f1640 1 Processor -- start 2024-04-26T21:40:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.053+0000 7fac2a4f1640 1 -- start start 2024-04-26T21:40:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac2a4f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac24162880 0x7fac240d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac2a4f1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fac240b5f20 0x7fac240d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac2a4f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac240b7e50 con 0x7fac240d39f0 2024-04-26T21:40:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac2a4f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fac240d3e40 con 0x7fac24162880 2024-04-26T21:40:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac2a4f1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fac240d0580 con 0x7fac240b5f20 2024-04-26T21:40:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac23fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac24162880 0x7fac240d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac28a8e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fac240d39f0 0x7fac240b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43380/0 (socket says 172.21.15.40:43380) 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac28a8e640 1 -- 172.21.15.40:0/2284707284 learned_addr learned my addr 172.21.15.40:0/2284707284 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac237fe640 1 --2- 172.21.15.40:0/2284707284 >> v2:172.21.15.184:3300/0 conn(0x7fac240b5f20 0x7fac240d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.054+0000 7fac23fff640 1 -- 172.21.15.40:0/2284707284 >> v2:172.21.15.184:3300/0 conn(0x7fac240b5f20 msgr2=0x7fac240d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.055+0000 7fac23fff640 1 --2- 172.21.15.40:0/2284707284 >> v2:172.21.15.184:3300/0 conn(0x7fac240b5f20 0x7fac240d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.055+0000 7fac23fff640 1 -- 172.21.15.40:0/2284707284 >> v1:172.21.15.40:6789/0 conn(0x7fac240d39f0 legacy=0x7fac240b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.055+0000 7fac22ffd640 1 -- 172.21.15.40:0/2284707284 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1365482494 0 0) 0x7fac240b7e50 con 0x7fac240d39f0 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.055+0000 7fac23fff640 1 -- 172.21.15.40:0/2284707284 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac240d1800 con 0x7fac24162880 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.055+0000 7fac23fff640 1 --2- 172.21.15.40:0/2284707284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac24162880 0x7fac240d2cb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fac2408d3d0 tx=0x7fac1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b21524504bbc67c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.055+0000 7fac22ffd640 1 -- 172.21.15.40:0/2284707284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18037070 con 0x7fac24162880 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.055+0000 7fac22ffd640 1 -- 172.21.15.40:0/2284707284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac18032440 con 0x7fac24162880 2024-04-26T21:40:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.055+0000 7fac22ffd640 1 -- 172.21.15.40:0/2284707284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18032810 con 0x7fac24162880 2024-04-26T21:40:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.056+0000 7fac2a4f1640 1 -- 172.21.15.40:0/2284707284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac24162880 msgr2=0x7fac240d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.056+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/2284707284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac24162880 0x7fac240d2cb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fac2408d3d0 tx=0x7fac1802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.056+0000 7fac2a4f1640 1 -- 172.21.15.40:0/2284707284 shutdown_connections 2024-04-26T21:40:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.056+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/2284707284 >> v2:172.21.15.184:3300/0 conn(0x7fac240b5f20 0x7fac240d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.056+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/2284707284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac24162880 0x7fac240d2cb0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.056+0000 7fac2a4f1640 1 -- 172.21.15.40:0/2284707284 >> 172.21.15.40:0/2284707284 conn(0x7fac240d4f60 msgr2=0x7fac240d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.056+0000 7fac2a4f1640 1 -- 172.21.15.40:0/2284707284 shutdown_connections 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.056+0000 7fac2a4f1640 1 -- 172.21.15.40:0/2284707284 wait complete. 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.057+0000 7fac2a4f1640 1 Processor -- start 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.057+0000 7fac2a4f1640 1 -- start start 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac2a4f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac240b5f20 0x7fac2415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac2a4f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac240d39f0 0x7fac240b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac2a4f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac24162880 0x7fac241ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac2a4f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fac240d0d90 con 0x7fac240b5f20 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac2a4f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac240d0ab0 con 0x7fac240d39f0 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac2a4f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac24162c60 con 0x7fac24162880 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac240b5f20 0x7fac2415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac240b5f20 0x7fac2415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54266/0 (socket says 172.21.15.40:54266) 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac23fff640 1 -- 172.21.15.40:0/4015117480 learned_addr learned my addr 172.21.15.40:0/4015117480 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac237fe640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac240d39f0 0x7fac240b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac28a8e640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac24162880 0x7fac241ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac23fff640 1 -- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac24162880 msgr2=0x7fac241ea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.058+0000 7fac23fff640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac24162880 0x7fac241ea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac23fff640 1 -- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac240d39f0 msgr2=0x7fac240b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac23fff640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac240d39f0 0x7fac240b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac23fff640 1 -- 172.21.15.40:0/4015117480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac240cc140 con 0x7fac240b5f20 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac23fff640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac240b5f20 0x7fac2415c470 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fac18000c00 tx=0x7fac180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac20ff9640 1 -- 172.21.15.40:0/4015117480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18037040 con 0x7fac240b5f20 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac20ff9640 1 -- 172.21.15.40:0/4015117480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac18042070 con 0x7fac240b5f20 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac240cc420 con 0x7fac240b5f20 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac20ff9640 1 -- 172.21.15.40:0/4015117480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18030920 con 0x7fac240b5f20 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.059+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac240cc9e0 con 0x7fac240b5f20 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.060+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac240694f0 con 0x7fac240b5f20 2024-04-26T21:40:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.060+0000 7fac20ff9640 1 -- 172.21.15.40:0/4015117480 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac18030ac0 con 0x7fac240b5f20 2024-04-26T21:40:51.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.061+0000 7fac20ff9640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac1006c8f0 0x7fac1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.061+0000 7fac20ff9640 1 -- 172.21.15.40:0/4015117480 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7fac180ac720 con 0x7fac240b5f20 2024-04-26T21:40:51.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.061+0000 7fac237fe640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac1006c8f0 0x7fac1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.061+0000 7fac237fe640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac1006c8f0 0x7fac1006edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fac240a2ea0 tx=0x7fac14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.065+0000 7fac20ff9640 1 -- 172.21.15.40:0/4015117480 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac240d1800 con 0x7fac240b5f20 2024-04-26T21:40:51.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.287+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac24074590 con 0x7fac240b5f20 2024-04-26T21:40:51.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.288+0000 7fac20ff9640 1 -- 172.21.15.40:0/4015117480 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5187 (secure 0 0 0) 0x7fac1807a250 con 0x7fac240b5f20 2024-04-26T21:40:51.289 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:40:51.289 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6975,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3716769386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3716769386},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716769386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:39:13.615988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6540,"qdb_cluster":[6540]},"id":4}]} 2024-04-26T21:40:51.289 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 62 2024-04-26T21:40:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac1006c8f0 msgr2=0x7fac1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac1006c8f0 0x7fac1006edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fac240a2ea0 tx=0x7fac14008040 comp rx=0 tx=0).stop 2024-04-26T21:40:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac240b5f20 msgr2=0x7fac2415c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac240b5f20 0x7fac2415c470 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fac18000c00 tx=0x7fac180046c0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 shutdown_connections 2024-04-26T21:40:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fac1006c8f0 0x7fac1006edb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac24162880 0x7fac241ea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac240d39f0 0x7fac240b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 --2- 172.21.15.40:0/4015117480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac240b5f20 0x7fac2415c470 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 >> 172.21.15.40:0/4015117480 conn(0x7fac240d4f60 msgr2=0x7fac2415f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 shutdown_connections 2024-04-26T21:40:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.290+0000 7fac2a4f1640 1 -- 172.21.15.40:0/4015117480 wait complete. 2024-04-26T21:40:51.302 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:40:51.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.402+0000 7f751107e640 1 Processor -- start 2024-04-26T21:40:51.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.402+0000 7f751107e640 1 -- start start 2024-04-26T21:40:51.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f751107e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f750c162810 0x7f750c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f751107e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0b2340 0x7f750c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f751107e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f750c0d10d0 con 0x7f750c0cd930 2024-04-26T21:40:51.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f751107e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f750c0d0750 con 0x7f750c162810 2024-04-26T21:40:51.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f751107e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f750c0d16a0 con 0x7f750c0b2340 2024-04-26T21:40:51.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f750ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f750c162810 0x7f750c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f750ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f750c162810 0x7f750c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54276/0 (socket says 172.21.15.40:54276) 2024-04-26T21:40:51.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f750ad9d640 1 -- 172.21.15.40:0/3279256711 learned_addr learned my addr 172.21.15.40:0/3279256711 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:51.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f7503fff640 1 --2- 172.21.15.40:0/3279256711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0b2340 0x7f750c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f750ad9d640 1 -- 172.21.15.40:0/3279256711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0b2340 msgr2=0x7f750c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:51.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f750ad9d640 1 --2- 172.21.15.40:0/3279256711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0b2340 0x7f750c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f750ad9d640 1 -- 172.21.15.40:0/3279256711 >> v1:172.21.15.184:6789/0 conn(0x7f750c0cd930 legacy=0x7f750c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.403+0000 7f750ad9d640 1 -- 172.21.15.40:0/3279256711 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c0d1370 con 0x7f750c162810 2024-04-26T21:40:51.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.404+0000 7f750ad9d640 1 --2- 172.21.15.40:0/3279256711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c162810 0x7f750c070350 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009870 tx=0x7f74fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dfa9c43088106c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.404+0000 7f750a59c640 1 -- 172.21.15.40:0/3279256711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc037070 con 0x7f750c162810 2024-04-26T21:40:51.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.404+0000 7f750a59c640 1 -- 172.21.15.40:0/3279256711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74fc032440 con 0x7f750c162810 2024-04-26T21:40:51.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.404+0000 7f750a59c640 1 -- 172.21.15.40:0/3279256711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc032810 con 0x7f750c162810 2024-04-26T21:40:51.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.405+0000 7f751107e640 1 -- 172.21.15.40:0/3279256711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c162810 msgr2=0x7f750c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.405+0000 7f751107e640 1 --2- 172.21.15.40:0/3279256711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c162810 0x7f750c070350 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009870 tx=0x7f74fc02a270 comp rx=0 tx=0).stop 2024-04-26T21:40:51.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.405+0000 7f751107e640 1 -- 172.21.15.40:0/3279256711 shutdown_connections 2024-04-26T21:40:51.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.405+0000 7f751107e640 1 --2- 172.21.15.40:0/3279256711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0b2340 0x7f750c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.405+0000 7f751107e640 1 --2- 172.21.15.40:0/3279256711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c162810 0x7f750c070350 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.405+0000 7f751107e640 1 -- 172.21.15.40:0/3279256711 >> 172.21.15.40:0/3279256711 conn(0x7f750c0d2da0 msgr2=0x7f750c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:51.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.405+0000 7f751107e640 1 -- 172.21.15.40:0/3279256711 shutdown_connections 2024-04-26T21:40:51.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.405+0000 7f751107e640 1 -- 172.21.15.40:0/3279256711 wait complete. 2024-04-26T21:40:51.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.406+0000 7f751107e640 1 Processor -- start 2024-04-26T21:40:51.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.406+0000 7f751107e640 1 -- start start 2024-04-26T21:40:51.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f751107e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c0b2340 0x7f750c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f751107e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0cd930 0x7f750c0c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f751107e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750c162810 0x7f750c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f751107e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f750c070890 con 0x7f750c0b2340 2024-04-26T21:40:51.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f751107e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f750c0d0470 con 0x7f750c162810 2024-04-26T21:40:51.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f751107e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f750c0d2240 con 0x7f750c0cd930 2024-04-26T21:40:51.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c0b2340 0x7f750c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c0b2340 0x7f750c0cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54278/0 (socket says 172.21.15.40:54278) 2024-04-26T21:40:51.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 -- 172.21.15.40:0/3405253681 learned_addr learned my addr 172.21.15.40:0/3405253681 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:51.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f7503fff640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0cd930 0x7f750c0c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750b59e640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750c162810 0x7f750c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 -- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0cd930 msgr2=0x7f750c0c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0cd930 0x7f750c0c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 -- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750c162810 msgr2=0x7f750c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750c162810 0x7f750c15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 -- 172.21.15.40:0/3405253681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c0c6950 con 0x7f750c0b2340 2024-04-26T21:40:51.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f750ad9d640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c0b2340 0x7f750c0cf800 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f74fc000c00 tx=0x7f74fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.407+0000 7f75037fe640 1 -- 172.21.15.40:0/3405253681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc041020 con 0x7f750c0b2340 2024-04-26T21:40:51.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.408+0000 7f75037fe640 1 -- 172.21.15.40:0/3405253681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74fc0306c0 con 0x7f750c0b2340 2024-04-26T21:40:51.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.408+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c0c6bc0 con 0x7f750c0b2340 2024-04-26T21:40:51.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.408+0000 7f75037fe640 1 -- 172.21.15.40:0/3405253681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc037040 con 0x7f750c0b2340 2024-04-26T21:40:51.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.408+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c0c7180 con 0x7f750c0b2340 2024-04-26T21:40:51.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.409+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750c0694f0 con 0x7f750c0b2340 2024-04-26T21:40:51.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.409+0000 7f75037fe640 1 -- 172.21.15.40:0/3405253681 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74fc044020 con 0x7f750c0b2340 2024-04-26T21:40:51.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.410+0000 7f75037fe640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74ec06c8f0 0x7f74ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.410+0000 7f75037fe640 1 -- 172.21.15.40:0/3405253681 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f74fc0ac580 con 0x7f750c0b2340 2024-04-26T21:40:51.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.410+0000 7f7503fff640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74ec06c8f0 0x7f74ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.410+0000 7f7503fff640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74ec06c8f0 0x7f74ec06edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f74f4006fd0 tx=0x7f74f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.414+0000 7f75037fe640 1 -- 172.21.15.40:0/3405253681 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74fc04b030 con 0x7f750c0b2340 2024-04-26T21:40:51.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.600+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f750c0b0ac0 con 0x7f750c0b2340 2024-04-26T21:40:51.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.601+0000 7f75037fe640 1 -- 172.21.15.40:0/3405253681 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17299 (secure 0 0 0) 0x7f74fc07a790 con 0x7f750c0b2340 2024-04-26T21:40:51.603 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:40:51.603 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":70,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:39:10.611194+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6832/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6835/1667529627":"2024-04-27T21:38:35.525579+0000","172.21.15.40:6832/1667529627":"2024-04-27T21:38:35.525579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74ec06c8f0 msgr2=0x7f74ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74ec06c8f0 0x7f74ec06edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f74f4006fd0 tx=0x7f74f4008040 comp rx=0 tx=0).stop 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c0b2340 msgr2=0x7f750c0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c0b2340 0x7f750c0cf800 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f74fc000c00 tx=0x7f74fc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 shutdown_connections 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f74ec06c8f0 0x7f74ec06edb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750c162810 0x7f750c15cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f750c0cd930 0x7f750c0c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 --2- 172.21.15.40:0/3405253681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f750c0b2340 0x7f750c0cf800 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 >> 172.21.15.40:0/3405253681 conn(0x7f750c0d2da0 msgr2=0x7f750c1618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:51.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 shutdown_connections 2024-04-26T21:40:51.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.604+0000 7f751107e640 1 -- 172.21.15.40:0/3405253681 wait complete. 2024-04-26T21:40:51.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:38:16.583866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:38:17.253849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:40:51.616 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:40:51.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.720+0000 7fecce96a640 1 Processor -- start 2024-04-26T21:40:51.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.720+0000 7fecce96a640 1 -- start start 2024-04-26T21:40:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecce96a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fecc815f170 0x7fecc80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecce96a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc815f550 0x7fecc80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecce96a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc80d1040 con 0x7fecc80d34b0 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecce96a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fecc80c9430 con 0x7fecc815f170 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecce96a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fecc80c9680 con 0x7fecc815f550 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fecc815f170 0x7fecc80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fecc815f170 0x7fecc80d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54290/0 (socket says 172.21.15.40:54290) 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc7fff640 1 -- 172.21.15.40:0/3145250086 learned_addr learned my addr 172.21.15.40:0/3145250086 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc77fe640 1 --2- 172.21.15.40:0/3145250086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc815f550 0x7fecc80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc7fff640 1 -- 172.21.15.40:0/3145250086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc815f550 msgr2=0x7fecc80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc7fff640 1 --2- 172.21.15.40:0/3145250086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc815f550 0x7fecc80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc7fff640 1 -- 172.21.15.40:0/3145250086 >> v1:172.21.15.184:6789/0 conn(0x7fecc80d34b0 legacy=0x7fecc80cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc7fff640 1 -- 172.21.15.40:0/3145250086 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc80c9960 con 0x7fecc815f170 2024-04-26T21:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.721+0000 7fecc77fe640 1 --2- 172.21.15.40:0/3145250086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc815f550 0x7fecc80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:40:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.722+0000 7fecc7fff640 1 --2- 172.21.15.40:0/3145250086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815f170 0x7fecc80d5a80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009870 tx=0x7fecb802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd97da7382c1bac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.722+0000 7fecc6ffd640 1 -- 172.21.15.40:0/3145250086 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8037070 con 0x7fecc815f170 2024-04-26T21:40:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.722+0000 7fecc6ffd640 1 -- 172.21.15.40:0/3145250086 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb8032440 con 0x7fecc815f170 2024-04-26T21:40:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.722+0000 7fecc6ffd640 1 -- 172.21.15.40:0/3145250086 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8032810 con 0x7fecc815f170 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.722+0000 7fecce96a640 1 -- 172.21.15.40:0/3145250086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815f170 msgr2=0x7fecc80d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.722+0000 7fecce96a640 1 --2- 172.21.15.40:0/3145250086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815f170 0x7fecc80d5a80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009870 tx=0x7fecb802a210 comp rx=0 tx=0).stop 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.723+0000 7fecce96a640 1 -- 172.21.15.40:0/3145250086 shutdown_connections 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.723+0000 7fecce96a640 1 --2- 172.21.15.40:0/3145250086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc815f550 0x7fecc80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.723+0000 7fecce96a640 1 --2- 172.21.15.40:0/3145250086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815f170 0x7fecc80d5a80 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.723+0000 7fecce96a640 1 -- 172.21.15.40:0/3145250086 >> 172.21.15.40:0/3145250086 conn(0x7fecc80d4f60 msgr2=0x7fecc80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.723+0000 7fecce96a640 1 -- 172.21.15.40:0/3145250086 shutdown_connections 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.723+0000 7fecce96a640 1 -- 172.21.15.40:0/3145250086 wait complete. 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.724+0000 7fecce96a640 1 Processor -- start 2024-04-26T21:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.724+0000 7fecce96a640 1 -- start start 2024-04-26T21:40:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecce96a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc80d34b0 0x7fecc81524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecce96a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc815f170 0x7fecc815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecce96a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815b6c0 0x7fecc806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecce96a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fecc80cb5d0 con 0x7fecc815b6c0 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecce96a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fecc80cb450 con 0x7fecc815f170 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecce96a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fecc80cb750 con 0x7fecc80d34b0 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815b6c0 0x7fecc806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815b6c0 0x7fecc806d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54300/0 (socket says 172.21.15.40:54300) 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 -- 172.21.15.40:0/2849171164 learned_addr learned my addr 172.21.15.40:0/2849171164 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecc7fff640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc80d34b0 0x7fecc81524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecc77fe640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc815f170 0x7fecc815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 -- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc80d34b0 msgr2=0x7fecc81524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc80d34b0 0x7fecc81524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 -- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc815f170 msgr2=0x7fecc815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc815f170 0x7fecc815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 -- 172.21.15.40:0/2849171164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc80cd360 con 0x7fecc815b6c0 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecccf07640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815b6c0 0x7fecc806d140 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c780 tx=0x7fecc000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.725+0000 7fecc4ff9640 1 -- 172.21.15.40:0/2849171164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0004b60 con 0x7fecc815b6c0 2024-04-26T21:40:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.726+0000 7fecc4ff9640 1 -- 172.21.15.40:0/2849171164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc0004d00 con 0x7fecc815b6c0 2024-04-26T21:40:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.726+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc80cd640 con 0x7fecc815b6c0 2024-04-26T21:40:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.726+0000 7fecc4ff9640 1 -- 172.21.15.40:0/2849171164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc000aa30 con 0x7fecc815b6c0 2024-04-26T21:40:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.726+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc80cdc00 con 0x7fecc815b6c0 2024-04-26T21:40:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.727+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc80694f0 con 0x7fecc815b6c0 2024-04-26T21:40:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.727+0000 7fecc4ff9640 1 -- 172.21.15.40:0/2849171164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecc000abd0 con 0x7fecc815b6c0 2024-04-26T21:40:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.727+0000 7fecc4ff9640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecb406c920 0x7fecb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.728+0000 7fecc4ff9640 1 -- 172.21.15.40:0/2849171164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7fecc008c190 con 0x7fecc815b6c0 2024-04-26T21:40:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.728+0000 7fecc7fff640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecb406c920 0x7fecb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:51.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.728+0000 7fecc7fff640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecb406c920 0x7fecb406ede0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fecb8034000 tx=0x7fecb802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.732+0000 7fecc4ff9640 1 -- 172.21.15.40:0/2849171164 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecc0059cc0 con 0x7fecc815b6c0 2024-04-26T21:40:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.917+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fecc815ea90 con 0x7fecc815b6c0 2024-04-26T21:40:51.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.918+0000 7fecc4ff9640 1 -- 172.21.15.40:0/2849171164 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fecc005dfc0 con 0x7fecc815b6c0 2024-04-26T21:40:51.919 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.919+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecb406c920 msgr2=0x7fecb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.919+0000 7fecce96a640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecb406c920 0x7fecb406ede0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fecb8034000 tx=0x7fecb802e000 comp rx=0 tx=0).stop 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.919+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815b6c0 msgr2=0x7fecc806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.919+0000 7fecce96a640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815b6c0 0x7fecc806d140 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c780 tx=0x7fecc000cc40 comp rx=0 tx=0).stop 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.920+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 shutdown_connections 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.920+0000 7fecce96a640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecb406c920 0x7fecb406ede0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.920+0000 7fecce96a640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc815b6c0 0x7fecc806d140 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.920+0000 7fecce96a640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc815f170 0x7fecc815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.920+0000 7fecce96a640 1 --2- 172.21.15.40:0/2849171164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc80d34b0 0x7fecc81524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.920+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 >> 172.21.15.40:0/2849171164 conn(0x7fecc80d4f60 msgr2=0x7fecc80c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.920+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 shutdown_connections 2024-04-26T21:40:51.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:51.920+0000 7fecce96a640 1 -- 172.21.15.40:0/2849171164 wait complete. 2024-04-26T21:40:51.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:40:52.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.035+0000 7f45d3b0a640 1 Processor -- start 2024-04-26T21:40:52.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.035+0000 7f45d3b0a640 1 -- start start 2024-04-26T21:40:52.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d3b0a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45cc15f170 0x7f45cc0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d3b0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d3b0a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45cc0d1040 con 0x7f45cc15f550 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d3b0a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f45cc0c9430 con 0x7f45cc15f170 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d3b0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc0c9680 con 0x7f45cc0d3bc0 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d20a7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d18a6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45cc15f170 0x7f45cc0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d10a5640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f45cc15f550 0x7f45cc0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:42618/0 (socket says 172.21.15.40:42618) 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d20a7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:58622/0 (socket says 172.21.15.40:58622) 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d10a5640 1 -- 172.21.15.40:0/2302720378 learned_addr learned my addr 172.21.15.40:0/2302720378 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d18a6640 1 -- 172.21.15.40:0/2302720378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 msgr2=0x7f45cc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d18a6640 1 --2- 172.21.15.40:0/2302720378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d18a6640 1 -- 172.21.15.40:0/2302720378 >> v1:172.21.15.184:6789/0 conn(0x7f45cc15f550 legacy=0x7f45cc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d18a6640 1 -- 172.21.15.40:0/2302720378 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc0c9960 con 0x7f45cc15f170 2024-04-26T21:40:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.036+0000 7f45d08a4640 1 -- 172.21.15.40:0/2302720378 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3507775731 0 0) 0x7f45bc001110 con 0x7f45cc15f550 2024-04-26T21:40:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.037+0000 7f45d18a6640 1 --2- 172.21.15.40:0/2302720378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 0x7f45cc0d5a80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f45c80097d0 tx=0x7f45c802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e838df8c3ade657 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.037+0000 7f45d08a4640 1 -- 172.21.15.40:0/2302720378 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8037070 con 0x7f45cc15f170 2024-04-26T21:40:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.037+0000 7f45d08a4640 1 -- 172.21.15.40:0/2302720378 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c8032440 con 0x7f45cc15f170 2024-04-26T21:40:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.037+0000 7f45d08a4640 1 -- 172.21.15.40:0/2302720378 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8032810 con 0x7f45cc15f170 2024-04-26T21:40:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.038+0000 7f45d3b0a640 1 -- 172.21.15.40:0/2302720378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 msgr2=0x7f45cc0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.038+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/2302720378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 0x7f45cc0d5a80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f45c80097d0 tx=0x7f45c802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.038+0000 7f45d3b0a640 1 -- 172.21.15.40:0/2302720378 shutdown_connections 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.038+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/2302720378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.038+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/2302720378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 0x7f45cc0d5a80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.038+0000 7f45d3b0a640 1 -- 172.21.15.40:0/2302720378 >> 172.21.15.40:0/2302720378 conn(0x7f45cc0d4f60 msgr2=0x7f45cc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.038+0000 7f45d3b0a640 1 -- 172.21.15.40:0/2302720378 shutdown_connections 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.038+0000 7f45d3b0a640 1 -- 172.21.15.40:0/2302720378 wait complete. 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.039+0000 7f45d3b0a640 1 Processor -- start 2024-04-26T21:40:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.039+0000 7f45d3b0a640 1 -- start start 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d3b0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d3b0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 0x7f45cc0b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d3b0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc0ba0d0 0x7f45cc153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d3b0a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0ca5c0 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d3b0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0ca440 con 0x7f45cc15f170 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d3b0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc0ca740 con 0x7f45cc0d3bc0 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc0ba0d0 0x7f45cc153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc0ba0d0 0x7f45cc153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54314/0 (socket says 172.21.15.40:54314) 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 -- 172.21.15.40:0/3505870261 learned_addr learned my addr 172.21.15.40:0/3505870261 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d10a5640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 0x7f45cc0b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d18a6640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 -- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 msgr2=0x7f45cc150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 -- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 msgr2=0x7f45cc0b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 0x7f45cc0b9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 -- 172.21.15.40:0/3505870261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc1515b0 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.040+0000 7f45d20a7640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc0ba0d0 0x7f45cc153720 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f45c400efc0 tx=0x7f45c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.041+0000 7f45c27fc640 1 -- 172.21.15.40:0/3505870261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4018020 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.041+0000 7f45c27fc640 1 -- 172.21.15.40:0/3505870261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c4014480 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.041+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc154470 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.041+0000 7f45c27fc640 1 -- 172.21.15.40:0/3505870261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4014870 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.041+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc151b50 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.042+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc0a2c60 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.042+0000 7f45c27fc640 1 -- 172.21.15.40:0/3505870261 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45c4005d10 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.043+0000 7f45c27fc640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f45b006c920 0x7f45b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.043+0000 7f45c27fc640 1 -- 172.21.15.40:0/3505870261 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f45c408cc80 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.043+0000 7f45d18a6640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f45b006c920 0x7f45b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.043+0000 7f45d18a6640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f45b006c920 0x7f45b006ede0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f45c8034000 tx=0x7f45c802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.047+0000 7f45c27fc640 1 -- 172.21.15.40:0/3505870261 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45c4022030 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.233+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f45cc0694f0 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.233+0000 7f45c27fc640 1 -- 172.21.15.40:0/3505870261 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f45c4010040 con 0x7f45cc0ba0d0 2024-04-26T21:40:52.235 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f45b006c920 msgr2=0x7f45b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f45b006c920 0x7f45b006ede0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f45c8034000 tx=0x7f45c802e000 comp rx=0 tx=0).stop 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc0ba0d0 msgr2=0x7f45cc153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc0ba0d0 0x7f45cc153720 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f45c400efc0 tx=0x7f45c400c490 comp rx=0 tx=0).stop 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 shutdown_connections 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f45b006c920 0x7f45b006ede0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc0ba0d0 0x7f45cc153720 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc15f170 0x7f45cc0b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 --2- 172.21.15.40:0/3505870261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45cc0d3bc0 0x7f45cc150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 >> 172.21.15.40:0/3505870261 conn(0x7f45cc0d4f60 msgr2=0x7f45cc0c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.235+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 shutdown_connections 2024-04-26T21:40:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.236+0000 7f45d3b0a640 1 -- 172.21.15.40:0/3505870261 wait complete. 2024-04-26T21:40:52.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:40:52.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.350+0000 7fce3e358640 1 Processor -- start 2024-04-26T21:40:52.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.351+0000 7fce3e358640 1 -- start start 2024-04-26T21:40:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.351+0000 7fce3e358640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce380d3bc0 0x7fce380d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.351+0000 7fce3e358640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce380d1040 con 0x7fce3815f170 2024-04-26T21:40:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.351+0000 7fce3e358640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce380c9680 con 0x7fce3815f550 2024-04-26T21:40:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.351+0000 7fce3e358640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fce380c9430 con 0x7fce380d3bc0 2024-04-26T21:40:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.351+0000 7fce37fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fce3815f170 0x7fce380d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43394/0 (socket says 172.21.15.40:43394) 2024-04-26T21:40:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.351+0000 7fce37fff640 1 -- 172.21.15.40:0/4230590698 learned_addr learned my addr 172.21.15.40:0/4230590698 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.351+0000 7fce3c8f5640 1 --2- 172.21.15.40:0/4230590698 >> v2:172.21.15.40:3300/0 conn(0x7fce380d3bc0 0x7fce380d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.352+0000 7fce36ffd640 1 -- 172.21.15.40:0/4230590698 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929368128 0 0) 0x7fce20001110 con 0x7fce3815f170 2024-04-26T21:40:52.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.352+0000 7fce36ffd640 1 -- 172.21.15.40:0/4230590698 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce24003520 con 0x7fce3815f170 2024-04-26T21:40:52.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.352+0000 7fce3c8f5640 1 -- 172.21.15.40:0/4230590698 >> v1:172.21.15.184:6789/0 conn(0x7fce3815f550 legacy=0x7fce380ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.352+0000 7fce3c8f5640 1 -- 172.21.15.40:0/4230590698 >> v1:172.21.15.40:6789/0 conn(0x7fce3815f170 legacy=0x7fce380d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.352+0000 7fce3c8f5640 1 -- 172.21.15.40:0/4230590698 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce380c9960 con 0x7fce380d3bc0 2024-04-26T21:40:52.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.352+0000 7fce3c8f5640 1 --2- 172.21.15.40:0/4230590698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce380d3bc0 0x7fce380d5a80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fce380c95b0 tx=0x7fce3002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510030fa83217d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce36ffd640 1 -- 172.21.15.40:0/4230590698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30007740 con 0x7fce380d3bc0 2024-04-26T21:40:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce36ffd640 1 -- 172.21.15.40:0/4230590698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce300078e0 con 0x7fce380d3bc0 2024-04-26T21:40:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce36ffd640 1 -- 172.21.15.40:0/4230590698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30007cb0 con 0x7fce380d3bc0 2024-04-26T21:40:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce3e358640 1 -- 172.21.15.40:0/4230590698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce380d3bc0 msgr2=0x7fce380d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce3e358640 1 --2- 172.21.15.40:0/4230590698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce380d3bc0 0x7fce380d5a80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fce380c95b0 tx=0x7fce3002a710 comp rx=0 tx=0).stop 2024-04-26T21:40:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce3e358640 1 -- 172.21.15.40:0/4230590698 shutdown_connections 2024-04-26T21:40:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce3e358640 1 --2- 172.21.15.40:0/4230590698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce380d3bc0 0x7fce380d5a80 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce3e358640 1 -- 172.21.15.40:0/4230590698 >> 172.21.15.40:0/4230590698 conn(0x7fce380d4f60 msgr2=0x7fce380d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce3e358640 1 -- 172.21.15.40:0/4230590698 shutdown_connections 2024-04-26T21:40:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.353+0000 7fce3e358640 1 -- 172.21.15.40:0/4230590698 wait complete. 2024-04-26T21:40:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.354+0000 7fce3e358640 1 Processor -- start 2024-04-26T21:40:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce3e358640 1 -- start start 2024-04-26T21:40:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce3e358640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce380d3bc0 0x7fce3815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce3e358640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3815f170 0x7fce38158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce3e358640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce381528f0 0x7fce380ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce3e358640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fce380d2e30 con 0x7fce3815f170 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce3e358640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce380d2cb0 con 0x7fce380d3bc0 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce3e358640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fce380d2fb0 con 0x7fce381528f0 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3815f170 0x7fce38158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3815f170 0x7fce38158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54330/0 (socket says 172.21.15.40:54330) 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.355+0000 7fce377fe640 1 -- 172.21.15.40:0/2417941976 learned_addr learned my addr 172.21.15.40:0/2417941976 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce37fff640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce380d3bc0 0x7fce3815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce3c8f5640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce381528f0 0x7fce380ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce377fe640 1 -- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce381528f0 msgr2=0x7fce380ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce377fe640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce381528f0 0x7fce380ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce377fe640 1 -- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce380d3bc0 msgr2=0x7fce3815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce377fe640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce380d3bc0 0x7fce3815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce377fe640 1 -- 172.21.15.40:0/2417941976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3815b520 con 0x7fce3815f170 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce3c8f5640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce381528f0 0x7fce380ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce377fe640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3815f170 0x7fce38158c70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fce2800c3f0 tx=0x7fce2800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce34ff9640 1 -- 172.21.15.40:0/2417941976 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280052d0 con 0x7fce3815f170 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce34ff9640 1 -- 172.21.15.40:0/2417941976 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce28005490 con 0x7fce3815f170 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce38159580 con 0x7fce3815f170 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce34ff9640 1 -- 172.21.15.40:0/2417941976 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28007260 con 0x7fce3815f170 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.356+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce38159b40 con 0x7fce3815f170 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.357+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce380d5a80 con 0x7fce3815f170 2024-04-26T21:40:52.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.358+0000 7fce34ff9640 1 -- 172.21.15.40:0/2417941976 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce28007400 con 0x7fce3815f170 2024-04-26T21:40:52.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.358+0000 7fce34ff9640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce1806c8f0 0x7fce1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.358+0000 7fce34ff9640 1 -- 172.21.15.40:0/2417941976 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7fce2808b100 con 0x7fce3815f170 2024-04-26T21:40:52.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.358+0000 7fce37fff640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce1806c8f0 0x7fce1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.359+0000 7fce37fff640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce1806c8f0 0x7fce1806edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fce200012b0 tx=0x7fce20007b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.363+0000 7fce34ff9640 1 -- 172.21.15.40:0/2417941976 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce380d5a80 con 0x7fce3815f170 2024-04-26T21:40:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.549+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce3806d370 con 0x7fce3815f170 2024-04-26T21:40:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.549+0000 7fce34ff9640 1 -- 172.21.15.40:0/2417941976 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7fce28058cc0 con 0x7fce3815f170 2024-04-26T21:40:52.550 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce1806c8f0 msgr2=0x7fce1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce1806c8f0 0x7fce1806edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fce200012b0 tx=0x7fce20007b70 comp rx=0 tx=0).stop 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3815f170 msgr2=0x7fce38158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3815f170 0x7fce38158c70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fce2800c3f0 tx=0x7fce2800c7f0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 shutdown_connections 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce1806c8f0 0x7fce1806edb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce381528f0 0x7fce380ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3815f170 0x7fce38158c70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 --2- 172.21.15.40:0/2417941976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce380d3bc0 0x7fce3815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 >> 172.21.15.40:0/2417941976 conn(0x7fce380d4f60 msgr2=0x7fce380d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 shutdown_connections 2024-04-26T21:40:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.551+0000 7fce3e358640 1 -- 172.21.15.40:0/2417941976 wait complete. 2024-04-26T21:40:52.563 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:40:52.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.663+0000 7f5f18bc9640 1 Processor -- start 2024-04-26T21:40:52.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.663+0000 7f5f18bc9640 1 -- start start 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.664+0000 7f5f18bc9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5f1406c240 0x7f5f1406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.664+0000 7f5f18bc9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f140b4480 con 0x7f5f140aed50 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.664+0000 7f5f18bc9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f140b4720 con 0x7f5f1415f1d0 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.664+0000 7f5f18bc9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5f1406cbe0 con 0x7f5f1406c240 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.664+0000 7f5f12d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5f140aed50 0x7f5f140af1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43410/0 (socket says 172.21.15.40:43410) 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.664+0000 7f5f12d9d640 1 -- 172.21.15.40:0/77186732 learned_addr learned my addr 172.21.15.40:0/77186732 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.664+0000 7f5f11d9b640 1 --2- 172.21.15.40:0/77186732 >> v2:172.21.15.184:3300/0 conn(0x7f5f1406c240 0x7f5f1406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.665+0000 7f5f1159a640 1 -- 172.21.15.40:0/77186732 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3142792382 0 0) 0x7f5f140b4480 con 0x7f5f140aed50 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.665+0000 7f5f1159a640 1 -- 172.21.15.40:0/77186732 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5efc003520 con 0x7f5f140aed50 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.665+0000 7f5f11d9b640 1 -- 172.21.15.40:0/77186732 >> v1:172.21.15.184:6789/0 conn(0x7f5f1415f1d0 legacy=0x7f5f1406f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.665+0000 7f5f11d9b640 1 -- 172.21.15.40:0/77186732 >> v1:172.21.15.40:6789/0 conn(0x7f5f140aed50 legacy=0x7f5f140af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.665+0000 7f5f1159a640 1 -- 172.21.15.40:0/77186732 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170608071 0 0) 0x7f5f140b4720 con 0x7f5f1415f1d0 2024-04-26T21:40:52.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.665+0000 7f5f11d9b640 1 -- 172.21.15.40:0/77186732 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f140b4a00 con 0x7f5f1406c240 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f11d9b640 1 --2- 172.21.15.40:0/77186732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f1406c240 0x7f5f1406c6a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5f080097a0 tx=0x7f5f0802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f224067e6de346d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f1159a640 1 -- 172.21.15.40:0/77186732 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08037070 con 0x7f5f1406c240 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f1159a640 1 -- 172.21.15.40:0/77186732 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0802ac30 con 0x7f5f1406c240 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f1159a640 1 -- 172.21.15.40:0/77186732 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040019f0 con 0x7f5f1406c240 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f18bc9640 1 -- 172.21.15.40:0/77186732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f1406c240 msgr2=0x7f5f1406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f18bc9640 1 --2- 172.21.15.40:0/77186732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f1406c240 0x7f5f1406c6a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5f080097a0 tx=0x7f5f0802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f18bc9640 1 -- 172.21.15.40:0/77186732 shutdown_connections 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f18bc9640 1 --2- 172.21.15.40:0/77186732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f1406c240 0x7f5f1406c6a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f18bc9640 1 -- 172.21.15.40:0/77186732 >> 172.21.15.40:0/77186732 conn(0x7f5f140b3180 msgr2=0x7f5f140b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f18bc9640 1 -- 172.21.15.40:0/77186732 shutdown_connections 2024-04-26T21:40:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.666+0000 7f5f18bc9640 1 -- 172.21.15.40:0/77186732 wait complete. 2024-04-26T21:40:52.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.668+0000 7f5f18bc9640 1 Processor -- start 2024-04-26T21:40:52.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.668+0000 7f5f18bc9640 1 -- start start 2024-04-26T21:40:52.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f18bc9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1406c240 0x7f5f1414b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f18bc9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f140aed50 0x7f5f1414dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f18bc9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f1415f1d0 0x7f5f1414a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f18bc9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5f140b4fc0 con 0x7f5f1406c240 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f18bc9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f140ae8c0 con 0x7f5f140aed50 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f18bc9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5f14075ea0 con 0x7f5f1415f1d0 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1406c240 0x7f5f1414b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1406c240 0x7f5f1414b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54338/0 (socket says 172.21.15.40:54338) 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 -- 172.21.15.40:0/3750443529 learned_addr learned my addr 172.21.15.40:0/3750443529 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 -- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f1415f1d0 msgr2=0x7f5f1414a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f1415f1d0 0x7f5f1414a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 -- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f140aed50 msgr2=0x7f5f1414dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f140aed50 0x7f5f1414dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 -- 172.21.15.40:0/3750443529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1414e000 con 0x7f5f1406c240 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f1259c640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1406c240 0x7f5f1414b020 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c4b0 tx=0x7f5f0c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f02ffd640 1 -- 172.21.15.40:0/3750443529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c017070 con 0x7f5f1406c240 2024-04-26T21:40:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f02ffd640 1 -- 172.21.15.40:0/3750443529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0c005480 con 0x7f5f1406c240 2024-04-26T21:40:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1414b600 con 0x7f5f1406c240 2024-04-26T21:40:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f02ffd640 1 -- 172.21.15.40:0/3750443529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c003830 con 0x7f5f1406c240 2024-04-26T21:40:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.669+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1414bea0 con 0x7f5f1406c240 2024-04-26T21:40:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.670+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f14068fc0 con 0x7f5f1406c240 2024-04-26T21:40:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.671+0000 7f5f02ffd640 1 -- 172.21.15.40:0/3750443529 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f0c01a070 con 0x7f5f1406c240 2024-04-26T21:40:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.671+0000 7f5f02ffd640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.672+0000 7f5f02ffd640 1 -- 172.21.15.40:0/3750443529 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f5f0c08b350 con 0x7f5f1406c240 2024-04-26T21:40:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.672+0000 7f5f11d9b640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.672+0000 7f5f11d9b640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ef406c8f0 0x7f5ef406edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5f08034000 tx=0x7f5f0802c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:52.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.676+0000 7f5f02ffd640 1 -- 172.21.15.40:0/3750443529 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5f1414e000 con 0x7f5f1406c240 2024-04-26T21:40:52.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.899+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f141629f0 con 0x7f5f1406c240 2024-04-26T21:40:52.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.900+0000 7f5f02ffd640 1 -- 172.21.15.40:0/3750443529 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5187 (secure 0 0 0) 0x7f5f0c0593a0 con 0x7f5f1406c240 2024-04-26T21:40:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:40:52.901 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6975,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3716769386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3716769386},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716769386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:39:13.615988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6540,"qdb_cluster":[6540]},"id":4}]} 2024-04-26T21:40:52.901 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 62 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ef406c8f0 msgr2=0x7f5ef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ef406c8f0 0x7f5ef406edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5f08034000 tx=0x7f5f0802c000 comp rx=0 tx=0).stop 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1406c240 msgr2=0x7f5f1414b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1406c240 0x7f5f1414b020 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c4b0 tx=0x7f5f0c00c970 comp rx=0 tx=0).stop 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 shutdown_connections 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f1415f1d0 0x7f5f1414a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f140aed50 0x7f5f1414dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 --2- 172.21.15.40:0/3750443529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1406c240 0x7f5f1414b020 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 >> 172.21.15.40:0/3750443529 conn(0x7f5f140b3180 msgr2=0x7f5f14072e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 shutdown_connections 2024-04-26T21:40:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:52.902+0000 7f5f18bc9640 1 -- 172.21.15.40:0/3750443529 wait complete. 2024-04-26T21:40:52.914 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:40:53.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.013+0000 7f1d8b31b640 1 Processor -- start 2024-04-26T21:40:53.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.014+0000 7f1d8b31b640 1 -- start start 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.014+0000 7f1d8b31b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d840b3860 0x7f1d8416a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.014+0000 7f1d8b31b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d840d0ba0 con 0x7f1d840d06f0 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.014+0000 7f1d8b31b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d840d2590 con 0x7f1d8415f3a0 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.014+0000 7f1d8b31b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1d840d0e20 con 0x7f1d840b3860 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.014+0000 7f1d898b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d840b3860 0x7f1d8416a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.014+0000 7f1d888b6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d840d06f0 0x7f1d840d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43416/0 (socket says 172.21.15.40:43416) 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.014+0000 7f1d888b6640 1 -- 172.21.15.40:0/311487394 learned_addr learned my addr 172.21.15.40:0/311487394 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.015+0000 7f1d898b8640 1 -- 172.21.15.40:0/311487394 >> v1:172.21.15.184:6789/0 conn(0x7f1d8415f3a0 legacy=0x7f1d840d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.015+0000 7f1d898b8640 1 -- 172.21.15.40:0/311487394 >> v1:172.21.15.40:6789/0 conn(0x7f1d840d06f0 legacy=0x7f1d840d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.015+0000 7f1d898b8640 1 -- 172.21.15.40:0/311487394 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d840b1d70 con 0x7f1d840b3860 2024-04-26T21:40:53.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.015+0000 7f1d898b8640 1 --2- 172.21.15.40:0/311487394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d840b3860 0x7f1d8416a000 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009d90 tx=0x7f1d7402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368d3ddc8b7c727e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d83fff640 1 -- 172.21.15.40:0/311487394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74007740 con 0x7f1d840b3860 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d83fff640 1 -- 172.21.15.40:0/311487394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d740078e0 con 0x7f1d840b3860 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d83fff640 1 -- 172.21.15.40:0/311487394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74007cb0 con 0x7f1d840b3860 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d8b31b640 1 -- 172.21.15.40:0/311487394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d840b3860 msgr2=0x7f1d8416a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d8b31b640 1 --2- 172.21.15.40:0/311487394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d840b3860 0x7f1d8416a000 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009d90 tx=0x7f1d7402a660 comp rx=0 tx=0).stop 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d8b31b640 1 -- 172.21.15.40:0/311487394 shutdown_connections 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d8b31b640 1 --2- 172.21.15.40:0/311487394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d840b3860 0x7f1d8416a000 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d8b31b640 1 -- 172.21.15.40:0/311487394 >> 172.21.15.40:0/311487394 conn(0x7f1d840d4f90 msgr2=0x7f1d840d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d8b31b640 1 -- 172.21.15.40:0/311487394 shutdown_connections 2024-04-26T21:40:53.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.016+0000 7f1d8b31b640 1 -- 172.21.15.40:0/311487394 wait complete. 2024-04-26T21:40:53.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.017+0000 7f1d8b31b640 1 Processor -- start 2024-04-26T21:40:53.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.017+0000 7f1d8b31b640 1 -- start start 2024-04-26T21:40:53.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d8b31b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d840b3860 0x7f1d840cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d8b31b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d840d06f0 0x7f1d8409ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d8b31b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d8415f3a0 0x7f1d841504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d8b31b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8415f780 con 0x7f1d8415f3a0 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d8b31b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d840b2330 con 0x7f1d840d06f0 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d8b31b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d8415f900 con 0x7f1d840b3860 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d898b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d8415f3a0 0x7f1d841504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d898b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d8415f3a0 0x7f1d841504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54362/0 (socket says 172.21.15.40:54362) 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.018+0000 7f1d898b8640 1 -- 172.21.15.40:0/2463181007 learned_addr learned my addr 172.21.15.40:0/2463181007 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d888b6640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d840d06f0 0x7f1d8409ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d890b7640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d840b3860 0x7f1d840cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d898b8640 1 -- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d840b3860 msgr2=0x7f1d840cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d898b8640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d840b3860 0x7f1d840cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d898b8640 1 -- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d840d06f0 msgr2=0x7f1d8409ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d898b8640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d840d06f0 0x7f1d8409ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d898b8640 1 -- 172.21.15.40:0/2463181007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d84150a20 con 0x7f1d8415f3a0 2024-04-26T21:40:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d898b8640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d8415f3a0 0x7f1d841504e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1d7402ac50 tx=0x7f1d74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d81ffb640 1 -- 172.21.15.40:0/2463181007 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74040070 con 0x7f1d8415f3a0 2024-04-26T21:40:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d81ffb640 1 -- 172.21.15.40:0/2463181007 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d74031c90 con 0x7f1d8415f3a0 2024-04-26T21:40:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d84150f50 con 0x7f1d8415f3a0 2024-04-26T21:40:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d81ffb640 1 -- 172.21.15.40:0/2463181007 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74045680 con 0x7f1d8415f3a0 2024-04-26T21:40:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.019+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d84151510 con 0x7f1d8415f3a0 2024-04-26T21:40:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.020+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d840694f0 con 0x7f1d8415f3a0 2024-04-26T21:40:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.021+0000 7f1d81ffb640 1 -- 172.21.15.40:0/2463181007 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d74039040 con 0x7f1d8415f3a0 2024-04-26T21:40:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.021+0000 7f1d81ffb640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1d7006c8f0 0x7f1d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.021+0000 7f1d81ffb640 1 -- 172.21.15.40:0/2463181007 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f1d740ad7a0 con 0x7f1d8415f3a0 2024-04-26T21:40:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.021+0000 7f1d890b7640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1d7006c8f0 0x7f1d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.022+0000 7f1d890b7640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1d7006c8f0 0x7f1d7006edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c001500 tx=0x7f1d7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.025+0000 7f1d81ffb640 1 -- 172.21.15.40:0/2463181007 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d84150a20 con 0x7f1d8415f3a0 2024-04-26T21:40:53.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.214+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d840b6860 con 0x7f1d8415f3a0 2024-04-26T21:40:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.215+0000 7f1d81ffb640 1 -- 172.21.15.40:0/2463181007 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17299 (secure 0 0 0) 0x7f1d74043360 con 0x7f1d8415f3a0 2024-04-26T21:40:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:40:53.217 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":70,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:39:10.611194+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6832/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6835/1667529627":"2024-04-27T21:38:35.525579+0000","172.21.15.40:6832/1667529627":"2024-04-27T21:38:35.525579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:40:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1d7006c8f0 msgr2=0x7f1d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1d7006c8f0 0x7f1d7006edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c001500 tx=0x7f1d7c009040 comp rx=0 tx=0).stop 2024-04-26T21:40:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d8415f3a0 msgr2=0x7f1d841504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d8415f3a0 0x7f1d841504e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1d7402ac50 tx=0x7f1d74004a40 comp rx=0 tx=0).stop 2024-04-26T21:40:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 shutdown_connections 2024-04-26T21:40:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1d7006c8f0 0x7f1d7006edb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d8415f3a0 0x7f1d841504e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d840d06f0 0x7f1d8409ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 --2- 172.21.15.40:0/2463181007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d840b3860 0x7f1d840cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 >> 172.21.15.40:0/2463181007 conn(0x7f1d840d4f90 msgr2=0x7f1d8406cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 shutdown_connections 2024-04-26T21:40:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.218+0000 7f1d8b31b640 1 -- 172.21.15.40:0/2463181007 wait complete. 2024-04-26T21:40:53.230 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:40:53.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:40:53.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.330+0000 7f7fbb1e7640 1 Processor -- start 2024-04-26T21:40:53.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.330+0000 7f7fbb1e7640 1 -- start start 2024-04-26T21:40:53.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fbb1e7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fb40d1b30 0x7f7fb40d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fbb1e7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fb40d10d0 con 0x7f7fb40b2340 2024-04-26T21:40:53.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fbb1e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fb40d1370 con 0x7f7fb4162810 2024-04-26T21:40:53.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fbb1e7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7fb40d0750 con 0x7f7fb40d1b30 2024-04-26T21:40:53.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fb3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7fb40b2340 0x7f7fb40cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43424/0 (socket says 172.21.15.40:43424) 2024-04-26T21:40:53.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fb9784640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fb40d1b30 0x7f7fb40d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fb3fff640 1 -- 172.21.15.40:0/2024487733 learned_addr learned my addr 172.21.15.40:0/2024487733 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:53.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fb9784640 1 -- 172.21.15.40:0/2024487733 >> v1:172.21.15.184:6789/0 conn(0x7f7fb4162810 legacy=0x7f7fb4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:53.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fb9784640 1 -- 172.21.15.40:0/2024487733 >> v1:172.21.15.40:6789/0 conn(0x7f7fb40b2340 legacy=0x7f7fb40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.331+0000 7f7fb9784640 1 -- 172.21.15.40:0/2024487733 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb40d01b0 con 0x7f7fb40d1b30 2024-04-26T21:40:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.332+0000 7f7fb9784640 1 --2- 172.21.15.40:0/2024487733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb40d1b30 0x7f7fb40d1fc0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009d90 tx=0x7f7fac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9e6ebb987a0b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.332+0000 7f7fb37fe640 1 -- 172.21.15.40:0/2024487733 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac007740 con 0x7f7fb40d1b30 2024-04-26T21:40:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.332+0000 7f7fb37fe640 1 -- 172.21.15.40:0/2024487733 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac0078e0 con 0x7f7fb40d1b30 2024-04-26T21:40:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.332+0000 7f7fb37fe640 1 -- 172.21.15.40:0/2024487733 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac007cb0 con 0x7f7fb40d1b30 2024-04-26T21:40:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.333+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/2024487733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb40d1b30 msgr2=0x7f7fb40d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.333+0000 7f7fbb1e7640 1 --2- 172.21.15.40:0/2024487733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb40d1b30 0x7f7fb40d1fc0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009d90 tx=0x7f7fac02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.333+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/2024487733 shutdown_connections 2024-04-26T21:40:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.333+0000 7f7fbb1e7640 1 --2- 172.21.15.40:0/2024487733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb40d1b30 0x7f7fb40d1fc0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.333+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/2024487733 >> 172.21.15.40:0/2024487733 conn(0x7f7fb40d2da0 msgr2=0x7f7fb40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.333+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/2024487733 shutdown_connections 2024-04-26T21:40:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.333+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/2024487733 wait complete. 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.334+0000 7f7fbb1e7640 1 Processor -- start 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.334+0000 7f7fbb1e7640 1 -- start start 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fbb1e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb40b2340 0x7f7fb41e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fbb1e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40d1b30 0x7f7fb40c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fbb1e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb4162810 0x7f7fb40cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fbb1e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb40cdb00 con 0x7f7fb4162810 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fbb1e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb40d16a0 con 0x7f7fb40b2340 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fbb1e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb4070b50 con 0x7f7fb40d1b30 2024-04-26T21:40:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb4162810 0x7f7fb40cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb4162810 0x7f7fb40cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54388/0 (socket says 172.21.15.40:54388) 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 -- 172.21.15.40:0/3090825054 learned_addr learned my addr 172.21.15.40:0/3090825054 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb3fff640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40d1b30 0x7f7fb40c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb8f83640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb40b2340 0x7f7fb41e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 -- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40d1b30 msgr2=0x7f7fb40c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40d1b30 0x7f7fb40c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 -- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb40b2340 msgr2=0x7f7fb41e95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb40b2340 0x7f7fb41e95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 -- 172.21.15.40:0/3090825054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb40ca5c0 con 0x7f7fb4162810 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb3fff640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40d1b30 0x7f7fb40c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb8f83640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb40b2340 0x7f7fb41e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.335+0000 7f7fb9784640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb4162810 0x7f7fb40cb660 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009eb0 tx=0x7f7fac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.336+0000 7f7fb17fa640 1 -- 172.21.15.40:0/3090825054 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac040070 con 0x7f7fb4162810 2024-04-26T21:40:53.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.336+0000 7f7fb17fa640 1 -- 172.21.15.40:0/3090825054 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac031c90 con 0x7f7fb4162810 2024-04-26T21:40:53.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.336+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb40ca8a0 con 0x7f7fb4162810 2024-04-26T21:40:53.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.336+0000 7f7fb17fa640 1 -- 172.21.15.40:0/3090825054 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac03c640 con 0x7f7fb4162810 2024-04-26T21:40:53.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.336+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb40cb140 con 0x7f7fb4162810 2024-04-26T21:40:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.337+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb40caa70 con 0x7f7fb4162810 2024-04-26T21:40:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.337+0000 7f7fb17fa640 1 -- 172.21.15.40:0/3090825054 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fac039040 con 0x7f7fb4162810 2024-04-26T21:40:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.337+0000 7f7fb17fa640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.338+0000 7f7fb17fa640 1 -- 172.21.15.40:0/3090825054 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f7fac0ad440 con 0x7f7fb4162810 2024-04-26T21:40:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.338+0000 7f7fb8f83640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.338+0000 7f7fb8f83640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f9806c8f0 0x7f7f9806edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800a830 tx=0x7f7fa8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.342+0000 7f7fb17fa640 1 -- 172.21.15.40:0/3090825054 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7fac07b680 con 0x7f7fb4162810 2024-04-26T21:40:53.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.565+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fb40694f0 con 0x7f7fb4162810 2024-04-26T21:40:53.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.565+0000 7f7fb17fa640 1 -- 172.21.15.40:0/3090825054 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5187 (secure 0 0 0) 0x7f7fac07af80 con 0x7f7fb4162810 2024-04-26T21:40:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:40:53.567 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6975,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3716769386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3716769386},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716769386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:39:13.615988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6540,"qdb_cluster":[6540]},"id":4}]} 2024-04-26T21:40:53.567 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 62 2024-04-26T21:40:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f9806c8f0 msgr2=0x7f7f9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f9806c8f0 0x7f7f9806edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800a830 tx=0x7f7fa8008040 comp rx=0 tx=0).stop 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb4162810 msgr2=0x7f7fb40cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb4162810 0x7f7fb40cb660 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009eb0 tx=0x7f7fac004a40 comp rx=0 tx=0).stop 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 shutdown_connections 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb4162810 0x7f7fb40cb660 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40d1b30 0x7f7fb40c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 --2- 172.21.15.40:0/3090825054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb40b2340 0x7f7fb41e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 >> 172.21.15.40:0/3090825054 conn(0x7f7fb40d2da0 msgr2=0x7f7fb40b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 shutdown_connections 2024-04-26T21:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.567+0000 7f7fbb1e7640 1 -- 172.21.15.40:0/3090825054 wait complete. 2024-04-26T21:40:53.579 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:40:53.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.679+0000 7f5e1279d640 1 Processor -- start 2024-04-26T21:40:53.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.679+0000 7f5e1279d640 1 -- start start 2024-04-26T21:40:53.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e1279d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5e0c162810 0x7f5e0c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e1279d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e0c0d1b30 0x7f5e0c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e1279d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0c0d10d0 con 0x7f5e0c0b2340 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e1279d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5e0c0d0750 con 0x7f5e0c0d1b30 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e1279d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5e0c0d16a0 con 0x7f5e0c162810 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e10d3a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e0c0d1b30 0x7f5e0c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e0b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5e0c0b2340 0x7f5e0c0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43436/0 (socket says 172.21.15.40:43436) 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e0b7fe640 1 -- 172.21.15.40:0/3411338938 learned_addr learned my addr 172.21.15.40:0/3411338938 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e10d3a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e0c0d1b30 0x7f5e0c0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54400/0 (socket says 172.21.15.40:54400) 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e0bfff640 1 --2- 172.21.15.40:0/3411338938 >> v2:172.21.15.184:3300/0 conn(0x7f5e0c162810 0x7f5e0c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e10d3a640 1 -- 172.21.15.40:0/3411338938 >> v2:172.21.15.184:3300/0 conn(0x7f5e0c162810 msgr2=0x7f5e0c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e10d3a640 1 --2- 172.21.15.40:0/3411338938 >> v2:172.21.15.184:3300/0 conn(0x7f5e0c162810 0x7f5e0c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e10d3a640 1 -- 172.21.15.40:0/3411338938 >> v1:172.21.15.40:6789/0 conn(0x7f5e0c0b2340 legacy=0x7f5e0c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.680+0000 7f5e10d3a640 1 -- 172.21.15.40:0/3411338938 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c0d1370 con 0x7f5e0c0d1b30 2024-04-26T21:40:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.681+0000 7f5e10d3a640 1 --2- 172.21.15.40:0/3411338938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0d1b30 0x7f5e0c0d1fc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c073d90 tx=0x7f5e0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda0f1bf2c3eb627 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.681+0000 7f5e0affd640 1 -- 172.21.15.40:0/3411338938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04007740 con 0x7f5e0c0d1b30 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.681+0000 7f5e0affd640 1 -- 172.21.15.40:0/3411338938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e040078e0 con 0x7f5e0c0d1b30 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.681+0000 7f5e0affd640 1 -- 172.21.15.40:0/3411338938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04007cb0 con 0x7f5e0c0d1b30 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.681+0000 7f5e1279d640 1 -- 172.21.15.40:0/3411338938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0d1b30 msgr2=0x7f5e0c0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.681+0000 7f5e1279d640 1 --2- 172.21.15.40:0/3411338938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0d1b30 0x7f5e0c0d1fc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c073d90 tx=0x7f5e0402a680 comp rx=0 tx=0).stop 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.682+0000 7f5e1279d640 1 -- 172.21.15.40:0/3411338938 shutdown_connections 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.682+0000 7f5e1279d640 1 --2- 172.21.15.40:0/3411338938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0d1b30 0x7f5e0c0d1fc0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.682+0000 7f5e1279d640 1 --2- 172.21.15.40:0/3411338938 >> v2:172.21.15.184:3300/0 conn(0x7f5e0c162810 0x7f5e0c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.682+0000 7f5e1279d640 1 -- 172.21.15.40:0/3411338938 >> 172.21.15.40:0/3411338938 conn(0x7f5e0c0d2da0 msgr2=0x7f5e0c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.682+0000 7f5e1279d640 1 -- 172.21.15.40:0/3411338938 shutdown_connections 2024-04-26T21:40:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.682+0000 7f5e1279d640 1 -- 172.21.15.40:0/3411338938 wait complete. 2024-04-26T21:40:53.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.683+0000 7f5e1279d640 1 Processor -- start 2024-04-26T21:40:53.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.683+0000 7f5e1279d640 1 -- start start 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e1279d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0b2340 0x7f5e0c159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e1279d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e0c0d1b30 0x7f5e0c0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e1279d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c162810 0x7f5e0c1f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e1279d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c070890 con 0x7f5e0c0b2340 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e1279d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c0d0470 con 0x7f5e0c162810 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e1279d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c0cdb00 con 0x7f5e0c0d1b30 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0b2340 0x7f5e0c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0b2340 0x7f5e0c159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54402/0 (socket says 172.21.15.40:54402) 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 -- 172.21.15.40:0/2736468249 learned_addr learned my addr 172.21.15.40:0/2736468249 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e10d3a640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c162810 0x7f5e0c1f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0b7fe640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e0c0d1b30 0x7f5e0c0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 -- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e0c0d1b30 msgr2=0x7f5e0c0c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e0c0d1b30 0x7f5e0c0c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 -- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c162810 msgr2=0x7f5e0c1f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c162810 0x7f5e0c1f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 -- 172.21.15.40:0/2736468249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c159750 con 0x7f5e0c0b2340 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e10d3a640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c162810 0x7f5e0c1f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0b7fe640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e0c0d1b30 0x7f5e0c0c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e0bfff640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0b2340 0x7f5e0c159210 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5df400d5f0 tx=0x7f5df400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e08ff9640 1 -- 172.21.15.40:0/2736468249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40125f0 con 0x7f5e0c0b2340 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e08ff9640 1 -- 172.21.15.40:0/2736468249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df4019070 con 0x7f5e0c0b2340 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e08ff9640 1 -- 172.21.15.40:0/2736468249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40129d0 con 0x7f5e0c0b2340 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.684+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c15ca50 con 0x7f5e0c0b2340 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.685+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1f5ad0 con 0x7f5e0c0b2340 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.685+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c0694f0 con 0x7f5e0c0b2340 2024-04-26T21:40:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.686+0000 7f5e08ff9640 1 -- 172.21.15.40:0/2736468249 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5df4012b70 con 0x7f5e0c0b2340 2024-04-26T21:40:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.686+0000 7f5e08ff9640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5dec06c8f0 0x7f5dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.686+0000 7f5e08ff9640 1 -- 172.21.15.40:0/2736468249 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f5df408c1c0 con 0x7f5e0c0b2340 2024-04-26T21:40:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.687+0000 7f5e0b7fe640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5dec06c8f0 0x7f5dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.687+0000 7f5e0b7fe640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5dec06c8f0 0x7f5dec06edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0b47c0 tx=0x7f5dfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:53.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.691+0000 7f5e08ff9640 1 -- 172.21.15.40:0/2736468249 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e0c159750 con 0x7f5e0c0b2340 2024-04-26T21:40:53.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.923+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e0c0b1f30 con 0x7f5e0c0b2340 2024-04-26T21:40:53.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.924+0000 7f5e08ff9640 1 -- 172.21.15.40:0/2736468249 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5187 (secure 0 0 0) 0x7f5df405a340 con 0x7f5e0c0b2340 2024-04-26T21:40:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:40:53.925 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6975,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3716769386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3716769386},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716769386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:38:18.310357+0000","modified":"2024-04-26T21:39:13.615988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6834/425620740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":425620740},{"type":"v1","addr":"172.21.15.40:6834","nonce":425620740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6540,"qdb_cluster":[6540]},"id":4}]} 2024-04-26T21:40:53.925 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 62 2024-04-26T21:40:53.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5dec06c8f0 msgr2=0x7f5dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5dec06c8f0 0x7f5dec06edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0b47c0 tx=0x7f5dfc009040 comp rx=0 tx=0).stop 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0b2340 msgr2=0x7f5e0c159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0b2340 0x7f5e0c159210 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5df400d5f0 tx=0x7f5df400dab0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 shutdown_connections 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5dec06c8f0 0x7f5dec06edb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c162810 0x7f5e0c1f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e0c0d1b30 0x7f5e0c0c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 --2- 172.21.15.40:0/2736468249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0c0b2340 0x7f5e0c159210 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 >> 172.21.15.40:0/2736468249 conn(0x7f5e0c0d2da0 msgr2=0x7f5e0c0cf8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 shutdown_connections 2024-04-26T21:40:53.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:53.926+0000 7f5e1279d640 1 -- 172.21.15.40:0/2736468249 wait complete. 2024-04-26T21:40:53.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:40:54.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.038+0000 7f1cd51c6640 1 Processor -- start 2024-04-26T21:40:54.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.038+0000 7f1cd51c6640 1 -- start start 2024-04-26T21:40:54.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1cd51c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd0162880 0x7f1cd00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1cd51c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1cd00b5f20 0x7f1cd00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1cd51c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cd00d39f0 0x7f1cd00b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1cd51c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd00d0580 con 0x7f1cd00d39f0 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1cd51c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd00b7e50 con 0x7f1cd00b5f20 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1cd51c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd00d1800 con 0x7f1cd0162880 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1ccf59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cd00d39f0 0x7f1cd00b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1ccf59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cd00d39f0 0x7f1cd00b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54410/0 (socket says 172.21.15.40:54410) 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1ccf59e640 1 -- 172.21.15.40:0/3751021487 learned_addr learned my addr 172.21.15.40:0/3751021487 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1ccf59e640 1 -- 172.21.15.40:0/3751021487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd0162880 msgr2=0x7f1cd00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1ccf59e640 1 --2- 172.21.15.40:0/3751021487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd0162880 0x7f1cd00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1ccf59e640 1 -- 172.21.15.40:0/3751021487 >> v2:172.21.15.184:3300/0 conn(0x7f1cd00b5f20 msgr2=0x7f1cd00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1ccf59e640 1 --2- 172.21.15.40:0/3751021487 >> v2:172.21.15.184:3300/0 conn(0x7f1cd00b5f20 0x7f1cd00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.039+0000 7f1ccf59e640 1 -- 172.21.15.40:0/3751021487 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd00d07d0 con 0x7f1cd00d39f0 2024-04-26T21:40:54.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.040+0000 7f1ccf59e640 1 --2- 172.21.15.40:0/3751021487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00d39f0 0x7f1cd00b69e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1cd008e960 tx=0x7f1cc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97f025a9cee0ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.040+0000 7f1ccdd9b640 1 -- 172.21.15.40:0/3751021487 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8037070 con 0x7f1cd00d39f0 2024-04-26T21:40:54.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.040+0000 7f1ccdd9b640 1 -- 172.21.15.40:0/3751021487 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc8032440 con 0x7f1cd00d39f0 2024-04-26T21:40:54.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.040+0000 7f1ccdd9b640 1 -- 172.21.15.40:0/3751021487 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8032810 con 0x7f1cd00d39f0 2024-04-26T21:40:54.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 -- 172.21.15.40:0/3751021487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00d39f0 msgr2=0x7f1cd00b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/3751021487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00d39f0 0x7f1cd00b69e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1cd008e960 tx=0x7f1cc802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 -- 172.21.15.40:0/3751021487 shutdown_connections 2024-04-26T21:40:54.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/3751021487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00d39f0 0x7f1cd00b69e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/3751021487 >> v2:172.21.15.184:3300/0 conn(0x7f1cd00b5f20 0x7f1cd00d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/3751021487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd0162880 0x7f1cd00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 -- 172.21.15.40:0/3751021487 >> 172.21.15.40:0/3751021487 conn(0x7f1cd00d4f60 msgr2=0x7f1cd00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:54.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 -- 172.21.15.40:0/3751021487 shutdown_connections 2024-04-26T21:40:54.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.041+0000 7f1cd51c6640 1 -- 172.21.15.40:0/3751021487 wait complete. 2024-04-26T21:40:54.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.042+0000 7f1cd51c6640 1 Processor -- start 2024-04-26T21:40:54.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.042+0000 7f1cd51c6640 1 -- start start 2024-04-26T21:40:54.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cd51c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00b5f20 0x7f1cd009f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cd51c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d39f0 0x7f1cd015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cd51c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd0162880 0x7f1cd00b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cd51c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0162c60 con 0x7f1cd00b5f20 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cd51c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd00d0d90 con 0x7f1cd0162880 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cd51c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd0162de0 con 0x7f1cd00d39f0 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cced9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00b5f20 0x7f1cd009f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cced9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00b5f20 0x7f1cd009f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54422/0 (socket says 172.21.15.40:54422) 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cced9d640 1 -- 172.21.15.40:0/478258253 learned_addr learned my addr 172.21.15.40:0/478258253 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1ccf59e640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd0162880 0x7f1cd00b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cce59c640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d39f0 0x7f1cd015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cced9d640 1 -- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d39f0 msgr2=0x7f1cd015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cced9d640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d39f0 0x7f1cd015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cced9d640 1 -- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd0162880 msgr2=0x7f1cd00b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cced9d640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd0162880 0x7f1cd00b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1cced9d640 1 -- 172.21.15.40:0/478258253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd00b4a10 con 0x7f1cd00b5f20 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.043+0000 7f1ccf59e640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd0162880 0x7f1cd00b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.044+0000 7f1cced9d640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00b5f20 0x7f1cd009f040 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000ba50 tx=0x7f1cc000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.044+0000 7f1cc77fe640 1 -- 172.21.15.40:0/478258253 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0019070 con 0x7f1cd00b5f20 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.044+0000 7f1cc77fe640 1 -- 172.21.15.40:0/478258253 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc000cba0 con 0x7f1cd00b5f20 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.044+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd00b4170 con 0x7f1cd00b5f20 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.044+0000 7f1cc77fe640 1 -- 172.21.15.40:0/478258253 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0014440 con 0x7f1cd00b5f20 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.044+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd00b4730 con 0x7f1cd00b5f20 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.045+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd00694f0 con 0x7f1cd00b5f20 2024-04-26T21:40:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.045+0000 7f1cc77fe640 1 -- 172.21.15.40:0/478258253 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cc001c070 con 0x7f1cd00b5f20 2024-04-26T21:40:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.046+0000 7f1cc77fe640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cb006c8f0 0x7f1cb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.046+0000 7f1cc77fe640 1 -- 172.21.15.40:0/478258253 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f1cc008bb20 con 0x7f1cd00b5f20 2024-04-26T21:40:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.046+0000 7f1cce59c640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cb006c8f0 0x7f1cb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.047+0000 7f1cce59c640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cb006c8f0 0x7f1cb006edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00cbf10 tx=0x7f1cb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.050+0000 7f1cc77fe640 1 -- 172.21.15.40:0/478258253 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cc0059ca0 con 0x7f1cd00b5f20 2024-04-26T21:40:54.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.236+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cd00baa00 con 0x7f1cd00b5f20 2024-04-26T21:40:54.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.237+0000 7f1cc77fe640 1 -- 172.21.15.40:0/478258253 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17299 (secure 0 0 0) 0x7f1cc00595a0 con 0x7f1cd00b5f20 2024-04-26T21:40:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:40:54.238 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":70,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:39:10.611194+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:38:16.583866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-26T21:38:17.253849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6832/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6835/1667529627":"2024-04-27T21:38:35.525579+0000","172.21.15.40:6832/1667529627":"2024-04-27T21:38:35.525579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:40:54.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.239+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cb006c8f0 msgr2=0x7f1cb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.239+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cb006c8f0 0x7f1cb006edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00cbf10 tx=0x7f1cb8008040 comp rx=0 tx=0).stop 2024-04-26T21:40:54.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.239+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00b5f20 msgr2=0x7f1cd009f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.239+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00b5f20 0x7f1cd009f040 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000ba50 tx=0x7f1cc000bf10 comp rx=0 tx=0).stop 2024-04-26T21:40:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.239+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 shutdown_connections 2024-04-26T21:40:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.240+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cb006c8f0 0x7f1cb006edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.240+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd0162880 0x7f1cd00b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.240+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d39f0 0x7f1cd015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.240+0000 7f1cd51c6640 1 --2- 172.21.15.40:0/478258253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd00b5f20 0x7f1cd009f040 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.240+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 >> 172.21.15.40:0/478258253 conn(0x7f1cd00d4f60 msgr2=0x7f1cd006cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.240+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 shutdown_connections 2024-04-26T21:40:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.240+0000 7f1cd51c6640 1 -- 172.21.15.40:0/478258253 wait complete. 2024-04-26T21:40:54.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:40:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.353+0000 7f103756a640 1 Processor -- start 2024-04-26T21:40:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.353+0000 7f103756a640 1 -- start start 2024-04-26T21:40:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f103756a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10300b5f20 0x7f10300d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f103756a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10300b7e50 con 0x7f1030162880 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f103756a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10300d1800 con 0x7f10300d3bc0 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f103756a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f10300d0580 con 0x7f10300b5f20 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f1035306640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1030162880 0x7f10300d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43440/0 (socket says 172.21.15.40:43440) 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f1035306640 1 -- 172.21.15.40:0/278044842 learned_addr learned my addr 172.21.15.40:0/278044842 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f1035b07640 1 --2- 172.21.15.40:0/278044842 >> v2:172.21.15.40:3300/0 conn(0x7f10300b5f20 0x7f10300d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f1027fff640 1 -- 172.21.15.40:0/278044842 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3960291896 0 0) 0x7f10300b7e50 con 0x7f1030162880 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f1027fff640 1 -- 172.21.15.40:0/278044842 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f101c003520 con 0x7f1030162880 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f1035b07640 1 -- 172.21.15.40:0/278044842 >> v1:172.21.15.184:6789/0 conn(0x7f10300d3bc0 legacy=0x7f10300b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f1035b07640 1 -- 172.21.15.40:0/278044842 >> v1:172.21.15.40:6789/0 conn(0x7f1030162880 legacy=0x7f10300d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.354+0000 7f1035b07640 1 -- 172.21.15.40:0/278044842 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10300d07d0 con 0x7f10300b5f20 2024-04-26T21:40:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.355+0000 7f1035b07640 1 --2- 172.21.15.40:0/278044842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300b5f20 0x7f10300d2cb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1028009d90 tx=0x7f102802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944a7e19fc12252b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.355+0000 7f1027fff640 1 -- 172.21.15.40:0/278044842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028007740 con 0x7f10300b5f20 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.355+0000 7f1027fff640 1 -- 172.21.15.40:0/278044842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10280078e0 con 0x7f10300b5f20 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.355+0000 7f1027fff640 1 -- 172.21.15.40:0/278044842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028007cb0 con 0x7f10300b5f20 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.356+0000 7f103756a640 1 -- 172.21.15.40:0/278044842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300b5f20 msgr2=0x7f10300d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.356+0000 7f103756a640 1 --2- 172.21.15.40:0/278044842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300b5f20 0x7f10300d2cb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1028009d90 tx=0x7f102802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.356+0000 7f103756a640 1 -- 172.21.15.40:0/278044842 shutdown_connections 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.356+0000 7f103756a640 1 --2- 172.21.15.40:0/278044842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300b5f20 0x7f10300d2cb0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.356+0000 7f103756a640 1 -- 172.21.15.40:0/278044842 >> 172.21.15.40:0/278044842 conn(0x7f10300d4f60 msgr2=0x7f10300d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.356+0000 7f103756a640 1 -- 172.21.15.40:0/278044842 shutdown_connections 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.356+0000 7f103756a640 1 -- 172.21.15.40:0/278044842 wait complete. 2024-04-26T21:40:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.357+0000 7f103756a640 1 Processor -- start 2024-04-26T21:40:54.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.357+0000 7f103756a640 1 -- start start 2024-04-26T21:40:54.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f103756a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10300b5f20 0x7f1030150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f103756a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300d3bc0 0x7f10300cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f103756a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1030162880 0x7f10300b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f103756a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1030162c60 con 0x7f10300d3bc0 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f103756a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10300d0d90 con 0x7f1030162880 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f103756a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1030162de0 con 0x7f10300b5f20 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300d3bc0 0x7f10300cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300d3bc0 0x7f10300cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54452/0 (socket says 172.21.15.40:54452) 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 -- 172.21.15.40:0/1190002181 learned_addr learned my addr 172.21.15.40:0/1190002181 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1035306640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10300b5f20 0x7f1030150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1035b07640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1030162880 0x7f10300b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 -- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10300b5f20 msgr2=0x7f1030150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10300b5f20 0x7f1030150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 -- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1030162880 msgr2=0x7f10300b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1030162880 0x7f10300b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 -- 172.21.15.40:0/1190002181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10300cefc0 con 0x7f10300d3bc0 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1034b05640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300d3bc0 0x7f10300cea80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f102000c430 tx=0x7f102000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1025ffb640 1 -- 172.21.15.40:0/1190002181 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020011790 con 0x7f10300d3bc0 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1025ffb640 1 -- 172.21.15.40:0/1190002181 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1020017070 con 0x7f10300d3bc0 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f1025ffb640 1 -- 172.21.15.40:0/1190002181 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020011b70 con 0x7f10300d3bc0 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.358+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10300ce4d0 con 0x7f10300d3bc0 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.359+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10300b1f60 con 0x7f10300d3bc0 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.360+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10300694f0 con 0x7f10300d3bc0 2024-04-26T21:40:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.360+0000 7f1025ffb640 1 -- 172.21.15.40:0/1190002181 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1020011d10 con 0x7f10300d3bc0 2024-04-26T21:40:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.360+0000 7f1025ffb640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f101406c8f0 0x7f101406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.361+0000 7f1025ffb640 1 -- 172.21.15.40:0/1190002181 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6389+0+0 (secure 0 0 0) 0x7f102008bce0 con 0x7f10300d3bc0 2024-04-26T21:40:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.361+0000 7f1035306640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f101406c8f0 0x7f101406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.361+0000 7f1035306640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f101406c8f0 0x7f101406edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f102c005880 tx=0x7f102c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.365+0000 7f1025ffb640 1 -- 172.21.15.40:0/1190002181 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10300cefc0 con 0x7f10300d3bc0 2024-04-26T21:40:54.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.587+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1030074590 con 0x7f10300d3bc0 2024-04-26T21:40:54.588 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:40:54.587+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:40:54.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.589+0000 7f1025ffb640 1 -- 172.21.15.40:0/1190002181 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7f1020059e50 con 0x7f10300d3bc0 2024-04-26T21:40:54.590 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.591+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f101406c8f0 msgr2=0x7f101406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.591+0000 7f103756a640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f101406c8f0 0x7f101406edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f102c005880 tx=0x7f102c009040 comp rx=0 tx=0).stop 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.591+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300d3bc0 msgr2=0x7f10300cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.591+0000 7f103756a640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300d3bc0 0x7f10300cea80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f102000c430 tx=0x7f102000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.592+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 shutdown_connections 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.592+0000 7f103756a640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f101406c8f0 0x7f101406edb0 secure :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f102c005880 tx=0x7f102c009040 comp rx=0 tx=0).stop 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.592+0000 7f103756a640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1030162880 0x7f10300b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.592+0000 7f103756a640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10300d3bc0 0x7f10300cea80 secure :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f102000c430 tx=0x7f102000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.592+0000 7f103756a640 1 --2- 172.21.15.40:0/1190002181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10300b5f20 0x7f1030150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.592+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 >> 172.21.15.40:0/1190002181 conn(0x7f10300d4f60 msgr2=0x7f10300d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:54.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.592+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 shutdown_connections 2024-04-26T21:40:54.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.592+0000 7f103756a640 1 -- 172.21.15.40:0/1190002181 wait complete. 2024-04-26T21:40:54.596 INFO:tasks.ceph.mds.c.smithi040.stderr: -4253> 2024-04-26T21:39:14.922+0000 7f41571d2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:40:54.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:40:54.625 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:40:54.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.707+0000 7feceb4fc640 1 Processor -- start 2024-04-26T21:40:54.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.707+0000 7feceb4fc640 1 -- start start 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7feceb4fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece4162880 0x7fece40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7feceb4fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece40b7e50 con 0x7fece40b5f20 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7feceb4fc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece40d1800 con 0x7fece40d3bc0 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7feceb4fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fece40d0580 con 0x7fece4162880 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7fece8a97640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fece40b5f20 0x7fece40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43456/0 (socket says 172.21.15.40:43456) 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7fece8a97640 1 -- 172.21.15.40:0/222474502 learned_addr learned my addr 172.21.15.40:0/222474502 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7fece9298640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece4162880 0x7fece40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 692442112 0 0) 0x7fece40b7e50 con 0x7fece40b5f20 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.708+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feccc003520 con 0x7fece40b5f20 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1021774882 0 0) 0x7feccc003520 con 0x7fece40b5f20 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece4162880 msgr2=0x7fece40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:54.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece4162880 0x7fece40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 >> v1:172.21.15.184:6789/0 conn(0x7fece40d3bc0 legacy=0x7fece40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece40d07d0 con 0x7fece40b5f20 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fecd8001670 con 0x7fece40b5f20 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 >> v1:172.21.15.40:6789/0 conn(0x7fece40b5f20 legacy=0x7fece40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece4162880 0x7feccc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feccc003f60 0x7feccc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc004d40 0x7feccc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fece40d0580 con 0x7feccc003f60 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feccc00baa0 con 0x7feccc004d40 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feccc00bca0 con 0x7fece4162880 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece8a97640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feccc003f60 0x7feccc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece9298640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece4162880 0x7feccc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece9a99640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc004d40 0x7feccc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece8a97640 1 -- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece4162880 msgr2=0x7feccc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece8a97640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece4162880 0x7feccc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece8a97640 1 -- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc004d40 msgr2=0x7feccc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece8a97640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc004d40 0x7feccc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece8a97640 1 -- 172.21.15.40:0/222474502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc003520 con 0x7feccc003f60 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.709+0000 7fece9a99640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc004d40 0x7feccc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.710+0000 7fece8a97640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feccc003f60 0x7feccc004800 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fecd800c240 tx=0x7feccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d7705aaec8878e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.710+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8007d10 con 0x7feccc003f60 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.710+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd8005ad0 con 0x7feccc003f60 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.710+0000 7fece3fff640 1 -- 172.21.15.40:0/222474502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8007070 con 0x7feccc003f60 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.711+0000 7feceb4fc640 1 -- 172.21.15.40:0/222474502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feccc003f60 msgr2=0x7feccc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.711+0000 7feceb4fc640 1 --2- 172.21.15.40:0/222474502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feccc003f60 0x7feccc004800 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fecd800c240 tx=0x7feccc005830 comp rx=0 tx=0).stop 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.711+0000 7fece9a99640 1 -- 172.21.15.40:0/222474502 reap_dead start 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.711+0000 7feceb4fc640 1 -- 172.21.15.40:0/222474502 shutdown_connections 2024-04-26T21:40:54.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.711+0000 7feceb4fc640 1 -- 172.21.15.40:0/222474502 >> 172.21.15.40:0/222474502 conn(0x7fece40d4f60 msgr2=0x7fece40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:54.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.711+0000 7feceb4fc640 1 -- 172.21.15.40:0/222474502 shutdown_connections 2024-04-26T21:40:54.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.711+0000 7feceb4fc640 1 -- 172.21.15.40:0/222474502 wait complete. 2024-04-26T21:40:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.712+0000 7feceb4fc640 1 Processor -- start 2024-04-26T21:40:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.712+0000 7feceb4fc640 1 -- start start 2024-04-26T21:40:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7feceb4fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc003f60 0x7fece415c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7feceb4fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece40cdff0 0x7fece41f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7feceb4fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece40b97f0 0x7fece40b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7feceb4fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fece4162c60 con 0x7fece40b97f0 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7feceb4fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fece40d0d90 con 0x7feccc003f60 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7feceb4fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fece4162de0 con 0x7fece40cdff0 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece40b97f0 0x7fece40b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece40b97f0 0x7fece40b9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54484/0 (socket says 172.21.15.40:54484) 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 -- 172.21.15.40:0/494474498 learned_addr learned my addr 172.21.15.40:0/494474498 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9298640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc003f60 0x7fece415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece8a97640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece40cdff0 0x7fece41f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 -- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece40cdff0 msgr2=0x7fece41f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece40cdff0 0x7fece41f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 -- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc003f60 msgr2=0x7fece415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc003f60 0x7fece415c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 -- 172.21.15.40:0/494474498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece409ec60 con 0x7fece40b97f0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9298640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc003f60 0x7fece415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece8a97640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece40cdff0 0x7fece41f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece9a99640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece40b97f0 0x7fece40b9ca0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fecd4000f80 tx=0x7fecd400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece1ffb640 1 -- 172.21.15.40:0/494474498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4006b50 con 0x7fece40b97f0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece1ffb640 1 -- 172.21.15.40:0/494474498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd4006d10 con 0x7fece40b97f0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece41f5750 con 0x7fece40b97f0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7fece1ffb640 1 -- 172.21.15.40:0/494474498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4003970 con 0x7fece40b97f0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.713+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece41f5d10 con 0x7fece40b97f0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.715+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece40694f0 con 0x7fece40b97f0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.715+0000 7fece1ffb640 1 -- 172.21.15.40:0/494474498 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecd401a020 con 0x7fece40b97f0 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.715+0000 7fece1ffb640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecd006c920 0x7fecd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:54.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.716+0000 7fece1ffb640 1 -- 172.21.15.40:0/494474498 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6475+0+0 (secure 0 0 0) 0x7fecd408b4b0 con 0x7fece40b97f0 2024-04-26T21:40:54.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.716+0000 7fece9298640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecd006c920 0x7fecd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:54.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.716+0000 7fece9298640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecd006c920 0x7fecd006ede0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fece40cdf10 tx=0x7fecdc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:54.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.720+0000 7fece1ffb640 1 -- 172.21.15.40:0/494474498 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fece40d1800 con 0x7fece40b97f0 2024-04-26T21:40:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:54.952+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fece40ce6f0 con 0x7fece40b97f0 2024-04-26T21:40:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.590+0000 7fece1ffb640 1 -- 172.21.15.40:0/494474498 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7fecd4059670 con 0x7fece40b97f0 2024-04-26T21:40:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecd006c920 msgr2=0x7fecd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecd006c920 0x7fecd006ede0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fece40cdf10 tx=0x7fecdc008040 comp rx=0 tx=0).stop 2024-04-26T21:40:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece40b97f0 msgr2=0x7fece40b9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece40b97f0 0x7fece40b9ca0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fecd4000f80 tx=0x7fecd400eb90 comp rx=0 tx=0).stop 2024-04-26T21:40:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 shutdown_connections 2024-04-26T21:40:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fecd006c920 0x7fecd006ede0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece40b97f0 0x7fece40b9ca0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece40cdff0 0x7fece41f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 --2- 172.21.15.40:0/494474498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feccc003f60 0x7fece415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 >> 172.21.15.40:0/494474498 conn(0x7fece40d4f60 msgr2=0x7fece4160620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 shutdown_connections 2024-04-26T21:40:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.593+0000 7feceb4fc640 1 -- 172.21.15.40:0/494474498 wait complete. 2024-04-26T21:40:55.607 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:40:55.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.761+0000 7efd00ebd640 1 Processor -- start 2024-04-26T21:40:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.761+0000 7efd00ebd640 1 -- start start 2024-04-26T21:40:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.761+0000 7efd00ebd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efcfc15f3e0 0x7efcfc0d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.761+0000 7efd00ebd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 0x7efcfc0d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.761+0000 7efd00ebd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc160930 con 0x7efcfc15f7c0 2024-04-26T21:40:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.761+0000 7efd00ebd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efcfc15fd20 con 0x7efcfc15f3e0 2024-04-26T21:40:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.761+0000 7efd00ebd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efcfc160000 con 0x7efcfc0d31e0 2024-04-26T21:40:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcfa59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efcfc15f3e0 0x7efcfc0d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcfad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efcfc15f7c0 0x7efcfc1620a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43458/0 (socket says 172.21.15.40:43458) 2024-04-26T21:40:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcfad9d640 1 -- 172.21.15.40:0/2303149184 learned_addr learned my addr 172.21.15.40:0/2303149184 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcf3fff640 1 --2- 172.21.15.40:0/2303149184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 0x7efcfc0d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcfa59c640 1 -- 172.21.15.40:0/2303149184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 msgr2=0x7efcfc0d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcfa59c640 1 --2- 172.21.15.40:0/2303149184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 0x7efcfc0d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcfa59c640 1 -- 172.21.15.40:0/2303149184 >> v1:172.21.15.40:6789/0 conn(0x7efcfc15f7c0 legacy=0x7efcfc1620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcf9d9b640 1 -- 172.21.15.40:0/2303149184 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 829230459 0 0) 0x7efcfc160930 con 0x7efcfc15f7c0 2024-04-26T21:40:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.762+0000 7efcfa59c640 1 -- 172.21.15.40:0/2303149184 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc1602e0 con 0x7efcfc15f3e0 2024-04-26T21:40:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efcfa59c640 1 --2- 172.21.15.40:0/2303149184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0d2ca0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efcec0097a0 tx=0x7efcec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d469a1bcf44fe201 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efcf9d9b640 1 -- 172.21.15.40:0/2303149184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec037070 con 0x7efcfc15f3e0 2024-04-26T21:40:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efcf9d9b640 1 -- 172.21.15.40:0/2303149184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcec02ae50 con 0x7efcfc15f3e0 2024-04-26T21:40:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efcf9d9b640 1 -- 172.21.15.40:0/2303149184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec032660 con 0x7efcfc15f3e0 2024-04-26T21:40:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efd00ebd640 1 -- 172.21.15.40:0/2303149184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 msgr2=0x7efcfc0d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efd00ebd640 1 --2- 172.21.15.40:0/2303149184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0d2ca0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efcec0097a0 tx=0x7efcec02a100 comp rx=0 tx=0).stop 2024-04-26T21:40:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efd00ebd640 1 -- 172.21.15.40:0/2303149184 shutdown_connections 2024-04-26T21:40:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efd00ebd640 1 --2- 172.21.15.40:0/2303149184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 0x7efcfc0d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efd00ebd640 1 --2- 172.21.15.40:0/2303149184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0d2ca0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.763+0000 7efd00ebd640 1 -- 172.21.15.40:0/2303149184 >> 172.21.15.40:0/2303149184 conn(0x7efcfc09e3c0 msgr2=0x7efcfc0d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.764+0000 7efd00ebd640 1 -- 172.21.15.40:0/2303149184 shutdown_connections 2024-04-26T21:40:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.764+0000 7efd00ebd640 1 -- 172.21.15.40:0/2303149184 wait complete. 2024-04-26T21:40:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.765+0000 7efd00ebd640 1 Processor -- start 2024-04-26T21:40:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.765+0000 7efd00ebd640 1 -- start start 2024-04-26T21:40:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efd00ebd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 0x7efcfc15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efd00ebd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efd00ebd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcfc15f7c0 0x7efcfc0b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efd00ebd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc0d3bd0 con 0x7efcfc15f3e0 2024-04-26T21:40:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efd00ebd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc160d40 con 0x7efcfc15f7c0 2024-04-26T21:40:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efd00ebd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efcfc0d3d50 con 0x7efcfc0d31e0 2024-04-26T21:40:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54502/0 (socket says 172.21.15.40:54502) 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 -- 172.21.15.40:0/4091961449 learned_addr learned my addr 172.21.15.40:0/4091961449 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcfa59c640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 0x7efcfc15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcfad9d640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcfc15f7c0 0x7efcfc0b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 -- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 msgr2=0x7efcfc15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 0x7efcfc15c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 -- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcfc15f7c0 msgr2=0x7efcfc0b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcfc15f7c0 0x7efcfc0b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 -- 172.21.15.40:0/4091961449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc0cba50 con 0x7efcfc15f3e0 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf3fff640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0c9f60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efce400d7d0 tx=0x7efce400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf2ffd640 1 -- 172.21.15.40:0/4091961449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4004020 con 0x7efcfc15f3e0 2024-04-26T21:40:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf2ffd640 1 -- 172.21.15.40:0/4091961449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce40041e0 con 0x7efcfc15f3e0 2024-04-26T21:40:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc0ca9b0 con 0x7efcfc15f3e0 2024-04-26T21:40:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.766+0000 7efcf2ffd640 1 -- 172.21.15.40:0/4091961449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce40043d0 con 0x7efcfc15f3e0 2024-04-26T21:40:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.767+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc0caf70 con 0x7efcfc15f3e0 2024-04-26T21:40:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.768+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcfc069600 con 0x7efcfc15f3e0 2024-04-26T21:40:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.768+0000 7efcf2ffd640 1 -- 172.21.15.40:0/4091961449 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efce401b020 con 0x7efcfc15f3e0 2024-04-26T21:40:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.768+0000 7efcf2ffd640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcdc06c920 0x7efcdc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.768+0000 7efcf2ffd640 1 -- 172.21.15.40:0/4091961449 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6475+0+0 (secure 0 0 0) 0x7efce408bb20 con 0x7efcfc15f3e0 2024-04-26T21:40:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.769+0000 7efcfa59c640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcdc06c920 0x7efcdc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.769+0000 7efcfa59c640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcdc06c920 0x7efcdc06ede0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efcec034000 tx=0x7efcec02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.773+0000 7efcf2ffd640 1 -- 172.21.15.40:0/4091961449 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efce4059c50 con 0x7efcfc15f3e0 2024-04-26T21:40:55.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:55.960+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efcfc06d9b0 con 0x7efcfc15f3e0 2024-04-26T21:40:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.804+0000 7efcf2ffd640 1 -- 172.21.15.40:0/4091961449 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) ==== 181+0+0 (secure 0 0 0) 0x7efce4059550 con 0x7efcfc15f3e0 2024-04-26T21:40:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:40:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcdc06c920 msgr2=0x7efcdc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcdc06c920 0x7efcdc06ede0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efcec034000 tx=0x7efcec02e000 comp rx=0 tx=0).stop 2024-04-26T21:40:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 msgr2=0x7efcfc0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0c9f60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efce400d7d0 tx=0x7efce400dc90 comp rx=0 tx=0).stop 2024-04-26T21:40:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 shutdown_connections 2024-04-26T21:40:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcdc06c920 0x7efcdc06ede0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcfc15f7c0 0x7efcfc0b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc15f3e0 0x7efcfc0c9f60 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 --2- 172.21.15.40:0/4091961449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcfc0d31e0 0x7efcfc15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 >> 172.21.15.40:0/4091961449 conn(0x7efcfc09e3c0 msgr2=0x7efcfc06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 shutdown_connections 2024-04-26T21:40:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.807+0000 7efd00ebd640 1 -- 172.21.15.40:0/4091961449 wait complete. 2024-04-26T21:40:56.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:40:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.935+0000 7fc07349c640 1 Processor -- start 2024-04-26T21:40:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.935+0000 7fc07349c640 1 -- start start 2024-04-26T21:40:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc07349c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc06c1629a0 0x7fc06c0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc07349c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 0x7fc06c0d5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc07349c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc06c162d80 con 0x7fc06c0d1950 2024-04-26T21:40:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc07349c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc06c0d39d0 con 0x7fc06c1629a0 2024-04-26T21:40:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc07349c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc06c0d3cb0 con 0x7fc06c0b5fd0 2024-04-26T21:40:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc070a37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 0x7fc06c0d5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc071238640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc06c1629a0 0x7fc06c0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc071238640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc06c1629a0 0x7fc06c0b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48362/0 (socket says 172.21.15.40:48362) 2024-04-26T21:40:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.936+0000 7fc071238640 1 -- 172.21.15.40:0/1446353078 learned_addr learned my addr 172.21.15.40:0/1446353078 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.937+0000 7fc071238640 1 -- 172.21.15.40:0/1446353078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 msgr2=0x7fc06c0d5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.937+0000 7fc071238640 1 --2- 172.21.15.40:0/1446353078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 0x7fc06c0d5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.937+0000 7fc071238640 1 -- 172.21.15.40:0/1446353078 >> v1:172.21.15.184:6789/0 conn(0x7fc06c0d1950 legacy=0x7fc06c0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.937+0000 7fc05ffff640 1 -- 172.21.15.40:0/1446353078 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810186219 0 0) 0x7fc06c162d80 con 0x7fc06c0d1950 2024-04-26T21:40:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.937+0000 7fc071238640 1 -- 172.21.15.40:0/1446353078 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c0cfda0 con 0x7fc06c1629a0 2024-04-26T21:40:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.937+0000 7fc071238640 1 --2- 172.21.15.40:0/1446353078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c1629a0 0x7fc06c0b4890 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc068009870 tx=0x7fc06802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a0428a80616ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc05ffff640 1 -- 172.21.15.40:0/1446353078 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068037070 con 0x7fc06c1629a0 2024-04-26T21:40:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc05ffff640 1 -- 172.21.15.40:0/1446353078 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc068032440 con 0x7fc06c1629a0 2024-04-26T21:40:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc05ffff640 1 -- 172.21.15.40:0/1446353078 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068032810 con 0x7fc06c1629a0 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc07349c640 1 -- 172.21.15.40:0/1446353078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c1629a0 msgr2=0x7fc06c0b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc07349c640 1 --2- 172.21.15.40:0/1446353078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c1629a0 0x7fc06c0b4890 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc068009870 tx=0x7fc06802a270 comp rx=0 tx=0).stop 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc07349c640 1 -- 172.21.15.40:0/1446353078 shutdown_connections 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc07349c640 1 --2- 172.21.15.40:0/1446353078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 0x7fc06c0d5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc07349c640 1 --2- 172.21.15.40:0/1446353078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c1629a0 0x7fc06c0b4890 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc07349c640 1 -- 172.21.15.40:0/1446353078 >> 172.21.15.40:0/1446353078 conn(0x7fc06c09e3e0 msgr2=0x7fc06c0d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc07349c640 1 -- 172.21.15.40:0/1446353078 shutdown_connections 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.938+0000 7fc07349c640 1 -- 172.21.15.40:0/1446353078 wait complete. 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc07349c640 1 Processor -- start 2024-04-26T21:40:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc07349c640 1 -- start start 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc07349c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 0x7fc06c1602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc07349c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c0d1950 0x7fc06c0cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc07349c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc06c1629a0 0x7fc06c0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc07349c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c0d5d60 con 0x7fc06c1629a0 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc07349c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c0d3f90 con 0x7fc06c0d1950 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc07349c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc06c0d5ee0 con 0x7fc06c0b5fd0 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc071a39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc06c1629a0 0x7fc06c0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc071238640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 0x7fc06c1602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc071a39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc06c1629a0 0x7fc06c0ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54508/0 (socket says 172.21.15.40:54508) 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.940+0000 7fc071a39640 1 -- 172.21.15.40:0/493720208 learned_addr learned my addr 172.21.15.40:0/493720208 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc070a37640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c0d1950 0x7fc06c0cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc071a39640 1 -- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 msgr2=0x7fc06c1602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc071a39640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 0x7fc06c1602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc071a39640 1 -- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c0d1950 msgr2=0x7fc06c0cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc071a39640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c0d1950 0x7fc06c0cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc071a39640 1 -- 172.21.15.40:0/493720208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c15f4e0 con 0x7fc06c1629a0 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc071a39640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc06c1629a0 0x7fc06c0ca710 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c7b0 tx=0x7fc06400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc05dffb640 1 -- 172.21.15.40:0/493720208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064003ba0 con 0x7fc06c1629a0 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc05dffb640 1 -- 172.21.15.40:0/493720208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc064003d40 con 0x7fc06c1629a0 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c15f7c0 con 0x7fc06c1629a0 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc05dffb640 1 -- 172.21.15.40:0/493720208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064005270 con 0x7fc06c1629a0 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.941+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c152450 con 0x7fc06c1629a0 2024-04-26T21:40:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.942+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06c069600 con 0x7fc06c1629a0 2024-04-26T21:40:56.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.943+0000 7fc05dffb640 1 -- 172.21.15.40:0/493720208 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc064005410 con 0x7fc06c1629a0 2024-04-26T21:40:56.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.943+0000 7fc05dffb640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:56.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.943+0000 7fc05dffb640 1 -- 172.21.15.40:0/493720208 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6008+0+0 (secure 0 0 0) 0x7fc06408b970 con 0x7fc06c1629a0 2024-04-26T21:40:56.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.943+0000 7fc071238640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:56.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.944+0000 7fc071238640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc05006c8f0 0x7fc05006edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc068034000 tx=0x7fc06802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:56.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:56.948+0000 7fc05dffb640 1 -- 172.21.15.40:0/493720208 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0640596b0 con 0x7fc06c1629a0 2024-04-26T21:40:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.138+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc06c0cbfa0 con 0x7fc06c1629a0 2024-04-26T21:40:57.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.806+0000 7fc05dffb640 1 -- 172.21.15.40:0/493720208 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) ==== 169+0+0 (secure 0 0 0) 0x7fc06405d9b0 con 0x7fc06c1629a0 2024-04-26T21:40:57.807 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.809+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc05006c8f0 msgr2=0x7fc05006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.809+0000 7fc07349c640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc05006c8f0 0x7fc05006edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc068034000 tx=0x7fc06802e000 comp rx=0 tx=0).stop 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.809+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc06c1629a0 msgr2=0x7fc06c0ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.809+0000 7fc07349c640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc06c1629a0 0x7fc06c0ca710 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c7b0 tx=0x7fc06400cbb0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.810+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 shutdown_connections 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.810+0000 7fc07349c640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.810+0000 7fc07349c640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc06c1629a0 0x7fc06c0ca710 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.810+0000 7fc07349c640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc06c0d1950 0x7fc06c0cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.810+0000 7fc07349c640 1 --2- 172.21.15.40:0/493720208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc06c0b5fd0 0x7fc06c1602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.810+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 >> 172.21.15.40:0/493720208 conn(0x7fc06c09e3e0 msgr2=0x7fc06c0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.810+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 shutdown_connections 2024-04-26T21:40:57.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.810+0000 7fc07349c640 1 -- 172.21.15.40:0/493720208 wait complete. 2024-04-26T21:40:57.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:40:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.939+0000 7f9da72fc640 1 Processor -- start 2024-04-26T21:40:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.940+0000 7f9da72fc640 1 -- start start 2024-04-26T21:40:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.940+0000 7f9da72fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9da00b2340 0x7f9da00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.940+0000 7f9da72fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00cd930 0x7f9da00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.940+0000 7f9da72fc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da00d10d0 con 0x7f9da0162810 2024-04-26T21:40:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.940+0000 7f9da72fc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9da00d0750 con 0x7f9da00b2340 2024-04-26T21:40:57.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.940+0000 7f9da72fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9da00d16a0 con 0x7f9da00cd930 2024-04-26T21:40:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da5899640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00cd930 0x7f9da00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da4897640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9da00b2340 0x7f9da00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da5098640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f9da0162810 0x7f9da0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:42674/0 (socket says 172.21.15.40:42674) 2024-04-26T21:40:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da5899640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00cd930 0x7f9da00d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:58772/0 (socket says 172.21.15.40:58772) 2024-04-26T21:40:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da5098640 1 -- 172.21.15.40:0/3088001432 learned_addr learned my addr 172.21.15.40:0/3088001432 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da5899640 1 -- 172.21.15.40:0/3088001432 >> v2:172.21.15.184:3300/0 conn(0x7f9da00b2340 msgr2=0x7f9da00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da5899640 1 --2- 172.21.15.40:0/3088001432 >> v2:172.21.15.184:3300/0 conn(0x7f9da00b2340 0x7f9da00cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da5899640 1 -- 172.21.15.40:0/3088001432 >> v1:172.21.15.184:6789/0 conn(0x7f9da0162810 legacy=0x7f9da0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9d97fff640 1 -- 172.21.15.40:0/3088001432 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450738934 0 0) 0x7f9da00d10d0 con 0x7f9da0162810 2024-04-26T21:40:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da5899640 1 -- 172.21.15.40:0/3088001432 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da00d1370 con 0x7f9da00cd930 2024-04-26T21:40:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.941+0000 7f9da4897640 1 --2- 172.21.15.40:0/3088001432 >> v2:172.21.15.184:3300/0 conn(0x7f9da00b2340 0x7f9da00cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:40:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.942+0000 7f9da5899640 1 --2- 172.21.15.40:0/3088001432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00cd930 0x7f9da00d1b30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9da00d1820 tx=0x7f9d9802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a54103bc0d543b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.942+0000 7f9d97fff640 1 -- 172.21.15.40:0/3088001432 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9802ae60 con 0x7f9da00cd930 2024-04-26T21:40:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.942+0000 7f9d97fff640 1 -- 172.21.15.40:0/3088001432 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98007740 con 0x7f9da00cd930 2024-04-26T21:40:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.942+0000 7f9d97fff640 1 -- 172.21.15.40:0/3088001432 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98007b10 con 0x7f9da00cd930 2024-04-26T21:40:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.943+0000 7f9da72fc640 1 -- 172.21.15.40:0/3088001432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00cd930 msgr2=0x7f9da00d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.943+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3088001432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00cd930 0x7f9da00d1b30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9da00d1820 tx=0x7f9d9802a640 comp rx=0 tx=0).stop 2024-04-26T21:40:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.943+0000 7f9da72fc640 1 -- 172.21.15.40:0/3088001432 shutdown_connections 2024-04-26T21:40:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.943+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3088001432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00cd930 0x7f9da00d1b30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.943+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3088001432 >> v2:172.21.15.184:3300/0 conn(0x7f9da00b2340 0x7f9da00cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.943+0000 7f9da72fc640 1 -- 172.21.15.40:0/3088001432 >> 172.21.15.40:0/3088001432 conn(0x7f9da00d2da0 msgr2=0x7f9da00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.943+0000 7f9da72fc640 1 -- 172.21.15.40:0/3088001432 shutdown_connections 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.943+0000 7f9da72fc640 1 -- 172.21.15.40:0/3088001432 wait complete. 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.944+0000 7f9da72fc640 1 Processor -- start 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.944+0000 7f9da72fc640 1 -- start start 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da72fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00b2340 0x7f9da015c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da72fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da00cd930 0x7f9da006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da72fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da0162810 0x7f9da006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da72fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9da00d2070 con 0x7f9da00cd930 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da72fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9da00d0470 con 0x7f9da0162810 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da72fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9da00d21f0 con 0x7f9da00b2340 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da4897640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da00cd930 0x7f9da006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da4897640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da00cd930 0x7f9da006e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54514/0 (socket says 172.21.15.40:54514) 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da4897640 1 -- 172.21.15.40:0/3550704116 learned_addr learned my addr 172.21.15.40:0/3550704116 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da5899640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da0162810 0x7f9da006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da5098640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00b2340 0x7f9da015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da4897640 1 -- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00b2340 msgr2=0x7f9da015c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da4897640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00b2340 0x7f9da015c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da4897640 1 -- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da0162810 msgr2=0x7f9da006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da4897640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da0162810 0x7f9da006ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.945+0000 7f9da4897640 1 -- 172.21.15.40:0/3550704116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da00cb190 con 0x7f9da00cd930 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.946+0000 7f9da4897640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da00cd930 0x7f9da006e4d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000ec10 tx=0x7f9d9000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.946+0000 7f9d95ffb640 1 -- 172.21.15.40:0/3550704116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90017070 con 0x7f9da00cd930 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.946+0000 7f9d95ffb640 1 -- 172.21.15.40:0/3550704116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9000cc30 con 0x7f9da00cd930 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.946+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da00cc9c0 con 0x7f9da00cd930 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.946+0000 7f9d95ffb640 1 -- 172.21.15.40:0/3550704116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90013470 con 0x7f9da00cd930 2024-04-26T21:40:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.946+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da00c6b90 con 0x7f9da00cd930 2024-04-26T21:40:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.947+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da00694f0 con 0x7f9da00cd930 2024-04-26T21:40:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.948+0000 7f9d95ffb640 1 -- 172.21.15.40:0/3550704116 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d9001a070 con 0x7f9da00cd930 2024-04-26T21:40:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.948+0000 7f9d95ffb640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8406c8f0 0x7f9d8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:57.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.948+0000 7f9d95ffb640 1 -- 172.21.15.40:0/3550704116 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7f9d9008bcf0 con 0x7f9da00cd930 2024-04-26T21:40:57.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.948+0000 7f9da5098640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8406c8f0 0x7f9d8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:57.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.949+0000 7f9da5098640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8406c8f0 0x7f9d8406edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c007fd0 tx=0x7f9d9c007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:57.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:57.952+0000 7f9d95ffb640 1 -- 172.21.15.40:0/3550704116 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d90090070 con 0x7f9da00cd930 2024-04-26T21:40:58.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.141+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9da00b2a60 con 0x7f9da00cd930 2024-04-26T21:40:58.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.144+0000 7f9d95ffb640 1 -- 172.21.15.40:0/3550704116 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f9d90059b90 con 0x7f9da00cd930 2024-04-26T21:40:58.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8406c8f0 msgr2=0x7f9d8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8406c8f0 0x7f9d8406edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c007fd0 tx=0x7f9d9c007de0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da00cd930 msgr2=0x7f9da006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da00cd930 0x7f9da006e4d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000ec10 tx=0x7f9d9000c520 comp rx=0 tx=0).stop 2024-04-26T21:40:58.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 shutdown_connections 2024-04-26T21:40:58.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8406c8f0 0x7f9d8406edb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da0162810 0x7f9da006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da00cd930 0x7f9da006e4d0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 --2- 172.21.15.40:0/3550704116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da00b2340 0x7f9da015c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.146+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 >> 172.21.15.40:0/3550704116 conn(0x7f9da00d2da0 msgr2=0x7f9da00cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:58.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.147+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 shutdown_connections 2024-04-26T21:40:58.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.147+0000 7f9da72fc640 1 -- 172.21.15.40:0/3550704116 wait complete. 2024-04-26T21:40:58.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-04-26T21:40:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.261+0000 7fe643339640 1 Processor -- start 2024-04-26T21:40:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.261+0000 7fe643339640 1 -- start start 2024-04-26T21:40:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe643339640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe63c0b5f20 0x7fe63c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe643339640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe63c0b7e50 con 0x7fe63c0d3bc0 2024-04-26T21:40:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe643339640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe63c0d1800 con 0x7fe63c162880 2024-04-26T21:40:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe643339640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe63c0d0580 con 0x7fe63c0b5f20 2024-04-26T21:40:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe6418d6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe63c0b5f20 0x7fe63c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe6408d4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe63c0d3bc0 0x7fe63c0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43466/0 (socket says 172.21.15.40:43466) 2024-04-26T21:40:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe6408d4640 1 -- 172.21.15.40:0/3316313795 learned_addr learned my addr 172.21.15.40:0/3316313795 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe6418d6640 1 -- 172.21.15.40:0/3316313795 >> v1:172.21.15.184:6789/0 conn(0x7fe63c162880 legacy=0x7fe63c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe6418d6640 1 -- 172.21.15.40:0/3316313795 >> v1:172.21.15.40:6789/0 conn(0x7fe63c0d3bc0 legacy=0x7fe63c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe6418d6640 1 -- 172.21.15.40:0/3316313795 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c0d07d0 con 0x7fe63c0b5f20 2024-04-26T21:40:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.262+0000 7fe63bfff640 1 -- 172.21.15.40:0/3316313795 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 548207646 0 0) 0x7fe63c0b7e50 con 0x7fe63c0d3bc0 2024-04-26T21:40:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.263+0000 7fe6418d6640 1 --2- 172.21.15.40:0/3316313795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c0b5f20 0x7fe63c0d2cb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009d90 tx=0x7fe62c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c9437a21b54329 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.263+0000 7fe63bfff640 1 -- 172.21.15.40:0/3316313795 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c007740 con 0x7fe63c0b5f20 2024-04-26T21:40:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.263+0000 7fe63bfff640 1 -- 172.21.15.40:0/3316313795 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c0078e0 con 0x7fe63c0b5f20 2024-04-26T21:40:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.263+0000 7fe63bfff640 1 -- 172.21.15.40:0/3316313795 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c007cb0 con 0x7fe63c0b5f20 2024-04-26T21:40:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.264+0000 7fe643339640 1 -- 172.21.15.40:0/3316313795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c0b5f20 msgr2=0x7fe63c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.264+0000 7fe643339640 1 --2- 172.21.15.40:0/3316313795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c0b5f20 0x7fe63c0d2cb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009d90 tx=0x7fe62c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.264+0000 7fe643339640 1 -- 172.21.15.40:0/3316313795 shutdown_connections 2024-04-26T21:40:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.264+0000 7fe643339640 1 --2- 172.21.15.40:0/3316313795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c0b5f20 0x7fe63c0d2cb0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.264+0000 7fe643339640 1 -- 172.21.15.40:0/3316313795 >> 172.21.15.40:0/3316313795 conn(0x7fe63c0d4f60 msgr2=0x7fe63c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.264+0000 7fe643339640 1 -- 172.21.15.40:0/3316313795 shutdown_connections 2024-04-26T21:40:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.264+0000 7fe643339640 1 -- 172.21.15.40:0/3316313795 wait complete. 2024-04-26T21:40:58.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.265+0000 7fe643339640 1 Processor -- start 2024-04-26T21:40:58.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.265+0000 7fe643339640 1 -- start start 2024-04-26T21:40:58.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe643339640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe63c0b5f20 0x7fe63c0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe643339640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe63c0d3bc0 0x7fe63c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe643339640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c162880 0x7fe63c1500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe643339640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c162c60 con 0x7fe63c162880 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe643339640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c0d0d90 con 0x7fe63c0d3bc0 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe643339640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe63c162de0 con 0x7fe63c0b5f20 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c162880 0x7fe63c1500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c162880 0x7fe63c1500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54542/0 (socket says 172.21.15.40:54542) 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 -- 172.21.15.40:0/3070345618 learned_addr learned my addr 172.21.15.40:0/3070345618 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6410d5640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe63c0b5f20 0x7fe63c0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6408d4640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe63c0d3bc0 0x7fe63c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 -- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe63c0b5f20 msgr2=0x7fe63c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe63c0b5f20 0x7fe63c0ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 -- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe63c0d3bc0 msgr2=0x7fe63c14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe63c0d3bc0 0x7fe63c14e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 -- 172.21.15.40:0/3070345618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c14f020 con 0x7fe63c162880 2024-04-26T21:40:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.266+0000 7fe6418d6640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c162880 0x7fe63c1500c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009d90 tx=0x7fe62c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.267+0000 7fe639ffb640 1 -- 172.21.15.40:0/3070345618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c040070 con 0x7fe63c162880 2024-04-26T21:40:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.267+0000 7fe639ffb640 1 -- 172.21.15.40:0/3070345618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c031c40 con 0x7fe63c162880 2024-04-26T21:40:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.267+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63c14f300 con 0x7fe63c162880 2024-04-26T21:40:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.267+0000 7fe639ffb640 1 -- 172.21.15.40:0/3070345618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c003990 con 0x7fe63c162880 2024-04-26T21:40:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.267+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63c14f8c0 con 0x7fe63c162880 2024-04-26T21:40:58.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.268+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63c0694f0 con 0x7fe63c162880 2024-04-26T21:40:58.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.268+0000 7fe639ffb640 1 -- 172.21.15.40:0/3070345618 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe62c039040 con 0x7fe63c162880 2024-04-26T21:40:58.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.269+0000 7fe639ffb640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe62806c8f0 0x7fe62806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.269+0000 7fe639ffb640 1 -- 172.21.15.40:0/3070345618 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7fe62c0acac0 con 0x7fe63c162880 2024-04-26T21:40:58.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.269+0000 7fe6410d5640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe62806c8f0 0x7fe62806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.269+0000 7fe6410d5640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe62806c8f0 0x7fe62806edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe634006fd0 tx=0x7fe634008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.273+0000 7fe639ffb640 1 -- 172.21.15.40:0/3070345618 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe63c0d1800 con 0x7fe63c162880 2024-04-26T21:40:58.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.459+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7fe63c0cfd70 con 0x7fe63c162880 2024-04-26T21:40:58.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.459+0000 7fe639ffb640 1 -- 172.21.15.40:0/3070345618 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v21) ==== 122+0+0 (secure 0 0 0) 0x7fe62c07afb0 con 0x7fe63c162880 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe62806c8f0 msgr2=0x7fe62806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe62806c8f0 0x7fe62806edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe634006fd0 tx=0x7fe634008040 comp rx=0 tx=0).stop 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c162880 msgr2=0x7fe63c1500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c162880 0x7fe63c1500c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009d90 tx=0x7fe62c004a40 comp rx=0 tx=0).stop 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 shutdown_connections 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe62806c8f0 0x7fe62806edb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe63c162880 0x7fe63c1500c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe63c0d3bc0 0x7fe63c14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 --2- 172.21.15.40:0/3070345618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe63c0b5f20 0x7fe63c0ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 >> 172.21.15.40:0/3070345618 conn(0x7fe63c0d4f60 msgr2=0x7fe63c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.461+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 shutdown_connections 2024-04-26T21:40:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.462+0000 7fe643339640 1 -- 172.21.15.40:0/3070345618 wait complete. 2024-04-26T21:40:58.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-04-26T21:40:58.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.573+0000 7faf5642c640 1 Processor -- start 2024-04-26T21:40:58.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.573+0000 7faf5642c640 1 -- start start 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.573+0000 7faf5642c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf500d3bc0 0x7faf500d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.573+0000 7faf5642c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf500b5f20 0x7faf500b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.573+0000 7faf5642c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf500b7e50 con 0x7faf50162880 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.573+0000 7faf5642c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7faf500d0580 con 0x7faf500d3bc0 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.573+0000 7faf5642c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faf500d1800 con 0x7faf500b5f20 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf500d3bc0 0x7faf500d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faf50162880 0x7faf500d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43476/0 (socket says 172.21.15.40:43476) 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf500d3bc0 0x7faf500d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54546/0 (socket says 172.21.15.40:54546) 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4ffff640 1 -- 172.21.15.40:0/2666065168 learned_addr learned my addr 172.21.15.40:0/2666065168 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:58.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf549c9640 1 --2- 172.21.15.40:0/2666065168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf500b5f20 0x7faf500b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4f7fe640 1 -- 172.21.15.40:0/2666065168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf500b5f20 msgr2=0x7faf500b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:58.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4f7fe640 1 --2- 172.21.15.40:0/2666065168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf500b5f20 0x7faf500b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4f7fe640 1 -- 172.21.15.40:0/2666065168 >> v1:172.21.15.40:6789/0 conn(0x7faf50162880 legacy=0x7faf500d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4f7fe640 1 -- 172.21.15.40:0/2666065168 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500d07d0 con 0x7faf500d3bc0 2024-04-26T21:40:58.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.574+0000 7faf4effd640 1 -- 172.21.15.40:0/2666065168 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3936902424 0 0) 0x7faf500b7e50 con 0x7faf50162880 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf4f7fe640 1 --2- 172.21.15.40:0/2666065168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500d2cb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7faf400097a0 tx=0x7faf4002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce6cf3ff6c70e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf4effd640 1 -- 172.21.15.40:0/2666065168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40037070 con 0x7faf500d3bc0 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf4effd640 1 -- 172.21.15.40:0/2666065168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf40032440 con 0x7faf500d3bc0 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf4effd640 1 -- 172.21.15.40:0/2666065168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40032810 con 0x7faf500d3bc0 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf5642c640 1 -- 172.21.15.40:0/2666065168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 msgr2=0x7faf500d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf5642c640 1 --2- 172.21.15.40:0/2666065168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500d2cb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7faf400097a0 tx=0x7faf4002a180 comp rx=0 tx=0).stop 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf5642c640 1 -- 172.21.15.40:0/2666065168 shutdown_connections 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf5642c640 1 --2- 172.21.15.40:0/2666065168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf500b5f20 0x7faf500b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf5642c640 1 --2- 172.21.15.40:0/2666065168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500d2cb0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf5642c640 1 -- 172.21.15.40:0/2666065168 >> 172.21.15.40:0/2666065168 conn(0x7faf500d4f60 msgr2=0x7faf500d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:58.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.575+0000 7faf5642c640 1 -- 172.21.15.40:0/2666065168 shutdown_connections 2024-04-26T21:40:58.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.576+0000 7faf5642c640 1 -- 172.21.15.40:0/2666065168 wait complete. 2024-04-26T21:40:58.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.577+0000 7faf5642c640 1 Processor -- start 2024-04-26T21:40:58.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.577+0000 7faf5642c640 1 -- start start 2024-04-26T21:40:58.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.577+0000 7faf5642c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf500b5f20 0x7faf5015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.577+0000 7faf5642c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.577+0000 7faf5642c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf50162880 0x7faf500b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.577+0000 7faf5642c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faf50162c60 con 0x7faf500d3bc0 2024-04-26T21:40:58.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf5642c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf500d0d90 con 0x7faf500b5f20 2024-04-26T21:40:58.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf5642c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faf50162de0 con 0x7faf50162880 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54558/0 (socket says 172.21.15.40:54558) 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 -- 172.21.15.40:0/3706478309 learned_addr learned my addr 172.21.15.40:0/3706478309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4ffff640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf500b5f20 0x7faf5015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf549c9640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf50162880 0x7faf500b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 -- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf50162880 msgr2=0x7faf500b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf50162880 0x7faf500b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 -- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf500b5f20 msgr2=0x7faf5015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf500b5f20 0x7faf5015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 -- 172.21.15.40:0/3706478309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500cc140 con 0x7faf500d3bc0 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4f7fe640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500b1b50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7faf40035040 tx=0x7faf400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4cff9640 1 -- 172.21.15.40:0/3706478309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40037040 con 0x7faf500d3bc0 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4cff9640 1 -- 172.21.15.40:0/3706478309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf40030770 con 0x7faf500d3bc0 2024-04-26T21:40:58.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf4cff9640 1 -- 172.21.15.40:0/3706478309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40030b40 con 0x7faf500d3bc0 2024-04-26T21:40:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf500cc3e0 con 0x7faf500d3bc0 2024-04-26T21:40:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.578+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf500cc9a0 con 0x7faf500d3bc0 2024-04-26T21:40:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.579+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf500694f0 con 0x7faf500d3bc0 2024-04-26T21:40:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.580+0000 7faf4cff9640 1 -- 172.21.15.40:0/3706478309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf40044020 con 0x7faf500d3bc0 2024-04-26T21:40:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.580+0000 7faf4cff9640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf3c06c8f0 0x7faf3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.580+0000 7faf4cff9640 1 -- 172.21.15.40:0/3706478309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7faf400abf90 con 0x7faf500d3bc0 2024-04-26T21:40:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.580+0000 7faf4ffff640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf3c06c8f0 0x7faf3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.581+0000 7faf4ffff640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf3c06c8f0 0x7faf3c06edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faf44000f60 tx=0x7faf44007b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:58.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.584+0000 7faf4cff9640 1 -- 172.21.15.40:0/3706478309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf40032e20 con 0x7faf500d3bc0 2024-04-26T21:40:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:58.841+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7faf50074590 con 0x7faf500d3bc0 2024-04-26T21:40:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.819+0000 7faf4cff9640 1 -- 172.21.15.40:0/3706478309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v293) ==== 142+0+0 (secure 0 0 0) 0x7faf4007a480 con 0x7faf500d3bc0 2024-04-26T21:40:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf3c06c8f0 msgr2=0x7faf3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf3c06c8f0 0x7faf3c06edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faf44000f60 tx=0x7faf44007b50 comp rx=0 tx=0).stop 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 msgr2=0x7faf500b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500b1b50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7faf40035040 tx=0x7faf400046c0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 shutdown_connections 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf3c06c8f0 0x7faf3c06edb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf50162880 0x7faf500b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf500d3bc0 0x7faf500b1b50 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 --2- 172.21.15.40:0/3706478309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf500b5f20 0x7faf5015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 >> 172.21.15.40:0/3706478309 conn(0x7faf500d4f60 msgr2=0x7faf500d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 shutdown_connections 2024-04-26T21:40:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.822+0000 7faf5642c640 1 -- 172.21.15.40:0/3706478309 wait complete. 2024-04-26T21:40:59.837 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-04-26T21:40:59.837 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-04-26T21:40:59.837 INFO:tasks.cephfs_test_runner:test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) 2024-04-26T21:40:59.837 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:40:59.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.944+0000 7f91a4009640 1 Processor -- start 2024-04-26T21:40:59.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.944+0000 7f91a4009640 1 -- start start 2024-04-26T21:40:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.945+0000 7f91a4009640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f919c156460 0x7f919c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.945+0000 7f91a4009640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 0x7f919c072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.945+0000 7f91a4009640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c156840 con 0x7f919c0cba10 2024-04-26T21:40:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.945+0000 7f91a4009640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f919c0cbf20 con 0x7f919c156460 2024-04-26T21:40:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.945+0000 7f91a4009640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f919c0cc200 con 0x7f919c06fea0 2024-04-26T21:40:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.945+0000 7f91a15a4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f919c0cba10 0x7f919c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58666/0 (socket says 172.21.15.40:58666) 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.945+0000 7f91a15a4640 1 -- 172.21.15.40:0/2485017692 learned_addr learned my addr 172.21.15.40:0/2485017692 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.945+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090868356 0 0) 0x7f919c156840 con 0x7f919c0cba10 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9188003520 con 0x7f919c0cba10 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a25a6640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 0x7f919c072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a1da5640 1 --2- 172.21.15.40:0/2485017692 >> v2:172.21.15.184:3300/0 conn(0x7f919c156460 0x7f919c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3118344411 0 0) 0x7f9188003520 con 0x7f919c0cba10 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 msgr2=0x7f919c072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 0x7f919c072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 >> v2:172.21.15.184:3300/0 conn(0x7f919c156460 msgr2=0x7f919c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 --2- 172.21.15.40:0/2485017692 >> v2:172.21.15.184:3300/0 conn(0x7f919c156460 0x7f919c0cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c0cc4e0 con 0x7f919c0cba10 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f918c001670 con 0x7f919c0cba10 2024-04-26T21:40:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 >> v1:172.21.15.40:6789/0 conn(0x7f919c0cba10 legacy=0x7f919c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 0x7f91880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9188003f30 0x7f9188004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9188004d40 0x7f91880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0cbf20 con 0x7f9188003f30 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0cc200 con 0x7f9188004d40 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f918800b9c0 con 0x7f919c06fea0 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a15a4640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9188003f30 0x7f9188004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a25a6640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9188004d40 0x7f91880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a1da5640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 0x7f91880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a15a4640 1 -- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 msgr2=0x7f91880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a15a4640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 0x7f91880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a15a4640 1 -- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9188004d40 msgr2=0x7f91880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:40:59.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a15a4640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9188004d40 0x7f91880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a15a4640 1 -- 172.21.15.40:0/2485017692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9188003520 con 0x7f9188003f30 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.946+0000 7f91a1da5640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c06fea0 0x7f91880039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.947+0000 7f91a15a4640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9188003f30 0x7f9188004800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f918c00c240 tx=0x7f9188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40eb3db024922e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.947+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c007d30 con 0x7f9188003f30 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.947+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918c005ad0 con 0x7f9188003f30 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.947+0000 7f91a0da3640 1 -- 172.21.15.40:0/2485017692 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c007070 con 0x7f9188003f30 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.948+0000 7f91a4009640 1 -- 172.21.15.40:0/2485017692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9188003f30 msgr2=0x7f9188004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.948+0000 7f91a4009640 1 --2- 172.21.15.40:0/2485017692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9188003f30 0x7f9188004800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f918c00c240 tx=0x7f9188005830 comp rx=0 tx=0).stop 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.948+0000 7f91a25a6640 1 -- 172.21.15.40:0/2485017692 reap_dead start 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.948+0000 7f91a4009640 1 -- 172.21.15.40:0/2485017692 shutdown_connections 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.948+0000 7f91a4009640 1 -- 172.21.15.40:0/2485017692 >> 172.21.15.40:0/2485017692 conn(0x7f919c0d0be0 msgr2=0x7f919c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.948+0000 7f91a4009640 1 -- 172.21.15.40:0/2485017692 shutdown_connections 2024-04-26T21:40:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.948+0000 7f91a4009640 1 -- 172.21.15.40:0/2485017692 wait complete. 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.949+0000 7f91a4009640 1 Processor -- start 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.949+0000 7f91a4009640 1 -- start start 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a4009640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9188003f30 0x7f919c151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a4009640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c15dd60 0x7f919c1e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a4009640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f919c1e5500 0x7f919c15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a4009640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f919c06e260 con 0x7f919c1e5500 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a4009640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f919c072700 con 0x7f9188003f30 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a4009640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f919c06e3e0 con 0x7f919c15dd60 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f919c1e5500 0x7f919c15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f919c1e5500 0x7f919c15ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47756/0 (socket says 172.21.15.40:47756) 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 -- 172.21.15.40:0/2722933294 learned_addr learned my addr 172.21.15.40:0/2722933294 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 -- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c15dd60 msgr2=0x7f919c1e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c15dd60 0x7f919c1e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 -- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9188003f30 msgr2=0x7f919c151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9188003f30 0x7f919c151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 -- 172.21.15.40:0/2722933294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c15cfa0 con 0x7f919c1e5500 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.950+0000 7f91a25a6640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f919c1e5500 0x7f919c15ae40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f919c0d0b90 tx=0x7f919400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.951+0000 7f91927fc640 1 -- 172.21.15.40:0/2722933294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194011e90 con 0x7f919c1e5500 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.951+0000 7f91927fc640 1 -- 172.21.15.40:0/2722933294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919401c070 con 0x7f919c1e5500 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.951+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c15d280 con 0x7f919c1e5500 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.951+0000 7f91927fc640 1 -- 172.21.15.40:0/2722933294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194003bd0 con 0x7f919c1e5500 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.951+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c15d840 con 0x7f919c1e5500 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.952+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c0694f0 con 0x7f919c1e5500 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.952+0000 7f91927fc640 1 -- 172.21.15.40:0/2722933294 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9194004030 con 0x7f919c1e5500 2024-04-26T21:40:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.953+0000 7f91927fc640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918006ca90 0x7f918006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:40:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.953+0000 7f91927fc640 1 -- 172.21.15.40:0/2722933294 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7f919408c390 con 0x7f919c1e5500 2024-04-26T21:40:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.953+0000 7f91a1da5640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918006ca90 0x7f918006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:40:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.954+0000 7f91a1da5640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918006ca90 0x7f918006ef50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f91980098b0 tx=0x7f9198008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:40:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:40:59.957+0000 7f91927fc640 1 -- 172.21.15.40:0/2722933294 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9194010040 con 0x7f919c1e5500 2024-04-26T21:41:00.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.148+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f919c15da10 con 0x7f919c1e5500 2024-04-26T21:41:00.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.149+0000 7f91927fc640 1 -- 172.21.15.40:0/2722933294 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+13516 (secure 0 0 0) 0x7f919405a880 con 0x7f919c1e5500 2024-04-26T21:41:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:00.150 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":73,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:40:57.805687+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6834/425620740":"2024-04-27T21:40:54.588991+0000","172.21.15.40:6835/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6832/1699369057":"2024-04-27T21:39:10.611149+0000","172.21.15.40:6833/425620740":"2024-04-27T21:40:54.588991+0000","172.21.15.40:6835/1667529627":"2024-04-27T21:38:35.525579+0000","172.21.15.40:6832/1667529627":"2024-04-27T21:38:35.525579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:00.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918006ca90 msgr2=0x7f918006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918006ca90 0x7f918006ef50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f91980098b0 tx=0x7f9198008040 comp rx=0 tx=0).stop 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f919c1e5500 msgr2=0x7f919c15ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f919c1e5500 0x7f919c15ae40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f919c0d0b90 tx=0x7f919400f580 comp rx=0 tx=0).stop 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 shutdown_connections 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918006ca90 0x7f918006ef50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f919c1e5500 0x7f919c15ae40 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f919c15dd60 0x7f919c1e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 --2- 172.21.15.40:0/2722933294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9188003f30 0x7f919c151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 >> 172.21.15.40:0/2722933294 conn(0x7f919c0d0be0 msgr2=0x7f919c15edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 shutdown_connections 2024-04-26T21:41:00.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.151+0000 7f91a4009640 1 -- 172.21.15.40:0/2722933294 wait complete. 2024-04-26T21:41:00.163 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:41:00.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:41:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.266+0000 7f4fb5abd640 1 Processor -- start 2024-04-26T21:41:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.266+0000 7f4fb5abd640 1 -- start start 2024-04-26T21:41:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.267+0000 7f4fb5abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb01604e0 0x7f4fb006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.267+0000 7f4fb5abd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4fb006fcd0 0x7f4fb00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.267+0000 7f4fb5abd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb0160970 con 0x7f4fb015f2d0 2024-04-26T21:41:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.267+0000 7f4fb5abd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb0074010 con 0x7f4fb006fcd0 2024-04-26T21:41:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.267+0000 7f4fb5abd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb0070100 con 0x7f4fb01604e0 2024-04-26T21:41:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4fb006fcd0 0x7f4fb00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faeffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb01604e0 0x7f4fb006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faeffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb01604e0 0x7f4fb006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:48320/0 (socket says 172.21.15.40:48320) 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4fb006fcd0 0x7f4fb00ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49492/0 (socket says 172.21.15.40:49492) 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faf7fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f4fb015f2d0 0x7f4fb0072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39794/0 (socket says 172.21.15.40:39794) 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faeffd640 1 -- 172.21.15.40:0/1238044144 learned_addr learned my addr 172.21.15.40:0/1238044144 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faeffd640 1 -- 172.21.15.40:0/1238044144 >> v2:172.21.15.184:3300/0 conn(0x7f4fb006fcd0 msgr2=0x7f4fb00ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faeffd640 1 --2- 172.21.15.40:0/1238044144 >> v2:172.21.15.184:3300/0 conn(0x7f4fb006fcd0 0x7f4fb00ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faeffd640 1 -- 172.21.15.40:0/1238044144 >> v1:172.21.15.184:6789/0 conn(0x7f4fb015f2d0 legacy=0x7f4fb0072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faeffd640 1 -- 172.21.15.40:0/1238044144 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb0072f70 con 0x7f4fb01604e0 2024-04-26T21:41:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.268+0000 7f4faffff640 1 --2- 172.21.15.40:0/1238044144 >> v2:172.21.15.184:3300/0 conn(0x7f4fb006fcd0 0x7f4fb00ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4faeffd640 1 --2- 172.21.15.40:0/1238044144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb01604e0 0x7f4fb006f790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009870 tx=0x7f4fa402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d29d014b391f92c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fae7fc640 1 -- 172.21.15.40:0/1238044144 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4037070 con 0x7f4fb01604e0 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fae7fc640 1 -- 172.21.15.40:0/1238044144 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa4032440 con 0x7f4fb01604e0 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fae7fc640 1 -- 172.21.15.40:0/1238044144 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4032810 con 0x7f4fb01604e0 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fb5abd640 1 -- 172.21.15.40:0/1238044144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb01604e0 msgr2=0x7f4fb006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/1238044144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb01604e0 0x7f4fb006f790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009870 tx=0x7f4fa402a270 comp rx=0 tx=0).stop 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fb5abd640 1 -- 172.21.15.40:0/1238044144 shutdown_connections 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/1238044144 >> v2:172.21.15.184:3300/0 conn(0x7f4fb006fcd0 0x7f4fb00ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/1238044144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb01604e0 0x7f4fb006f790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.269+0000 7f4fb5abd640 1 -- 172.21.15.40:0/1238044144 >> 172.21.15.40:0/1238044144 conn(0x7f4fb00d0be0 msgr2=0x7f4fb00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.270+0000 7f4fb5abd640 1 -- 172.21.15.40:0/1238044144 shutdown_connections 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.270+0000 7f4fb5abd640 1 -- 172.21.15.40:0/1238044144 wait complete. 2024-04-26T21:41:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.271+0000 7f4fb5abd640 1 Processor -- start 2024-04-26T21:41:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.271+0000 7f4fb5abd640 1 -- start start 2024-04-26T21:41:00.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4fb5abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb006fcd0 0x7f4fb0154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4fb5abd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fb015f2d0 0x7f4fb00c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4fb5abd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb01604e0 0x7f4fb01538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4fb5abd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0073810 con 0x7f4fb015f2d0 2024-04-26T21:41:00.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4fb5abd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0073530 con 0x7f4fb01604e0 2024-04-26T21:41:00.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4fb5abd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb0073af0 con 0x7f4fb006fcd0 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fb015f2d0 0x7f4fb00c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fb015f2d0 0x7f4fb00c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47762/0 (socket says 172.21.15.40:47762) 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 -- 172.21.15.40:0/422174669 learned_addr learned my addr 172.21.15.40:0/422174669 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faf7fe640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb006fcd0 0x7f4fb0154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faffff640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb01604e0 0x7f4fb01538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 -- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb006fcd0 msgr2=0x7f4fb0154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb006fcd0 0x7f4fb0154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 -- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb01604e0 msgr2=0x7f4fb01538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb01604e0 0x7f4fb01538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 -- 172.21.15.40:0/422174669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb0152840 con 0x7f4fb015f2d0 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faffff640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb01604e0 0x7f4fb01538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4faeffd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fb015f2d0 0x7f4fb00c7cb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4fa402a2b0 tx=0x7f4fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4f97fff640 1 -- 172.21.15.40:0/422174669 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4041030 con 0x7f4fb015f2d0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.272+0000 7f4f97fff640 1 -- 172.21.15.40:0/422174669 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa4030440 con 0x7f4fb015f2d0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.273+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb0152b20 con 0x7f4fb015f2d0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.273+0000 7f4f97fff640 1 -- 172.21.15.40:0/422174669 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4037040 con 0x7f4fb015f2d0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.273+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb01530e0 con 0x7f4fb015f2d0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.273+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb00694f0 con 0x7f4fb015f2d0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.274+0000 7f4f97fff640 1 -- 172.21.15.40:0/422174669 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fa4044070 con 0x7f4fb015f2d0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.274+0000 7f4f97fff640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.274+0000 7f4f97fff640 1 -- 172.21.15.40:0/422174669 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7f4fa40abdb0 con 0x7f4fb015f2d0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.275+0000 7f4faf7fe640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.275+0000 7f4faf7fe640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0007fd0 tx=0x7f4fa0007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:00.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.279+0000 7f4f97fff640 1 -- 172.21.15.40:0/422174669 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4fa404b070 con 0x7f4fb015f2d0 2024-04-26T21:41:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.465+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4fb006eac0 con 0x7f4fb015f2d0 2024-04-26T21:41:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.466+0000 7f4f97fff640 1 -- 172.21.15.40:0/422174669 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f4fa407a2a0 con 0x7f4fb015f2d0 2024-04-26T21:41:00.467 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f9c06c8f0 msgr2=0x7f4f9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0007fd0 tx=0x7f4fa0007e80 comp rx=0 tx=0).stop 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fb015f2d0 msgr2=0x7f4fb00c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fb015f2d0 0x7f4fb00c7cb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4fa402a2b0 tx=0x7f4fa40046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 shutdown_connections 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb01604e0 0x7f4fb01538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fb015f2d0 0x7f4fb00c7cb0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 --2- 172.21.15.40:0/422174669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fb006fcd0 0x7f4fb0154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 >> 172.21.15.40:0/422174669 conn(0x7f4fb00d0be0 msgr2=0x7f4fb0151220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 shutdown_connections 2024-04-26T21:41:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.468+0000 7f4fb5abd640 1 -- 172.21.15.40:0/422174669 wait complete. 2024-04-26T21:41:00.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-04-26T21:41:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.579+0000 7f2e011c6640 1 Processor -- start 2024-04-26T21:41:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.579+0000 7f2e011c6640 1 -- start start 2024-04-26T21:41:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2e011c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2dfc162810 0x7f2dfc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2e011c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2dfc0d1b30 0x7f2dfc0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2e011c6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dfc0d10d0 con 0x7f2dfc0b2340 2024-04-26T21:41:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2e011c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2dfc0d0750 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2e011c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2dfc0d16a0 con 0x7f2dfc162810 2024-04-26T21:41:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2dfb59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2dfc0d1b30 0x7f2dfc0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2dfb59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2dfc0d1b30 0x7f2dfc0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47768/0 (socket says 172.21.15.40:47768) 2024-04-26T21:41:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2dfb59e640 1 -- 172.21.15.40:0/3291125244 learned_addr learned my addr 172.21.15.40:0/3291125244 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2dfad9d640 1 --2- 172.21.15.40:0/3291125244 >> v2:172.21.15.184:3300/0 conn(0x7f2dfc162810 0x7f2dfc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2dfb59e640 1 -- 172.21.15.40:0/3291125244 >> v2:172.21.15.184:3300/0 conn(0x7f2dfc162810 msgr2=0x7f2dfc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2dfb59e640 1 --2- 172.21.15.40:0/3291125244 >> v2:172.21.15.184:3300/0 conn(0x7f2dfc162810 0x7f2dfc070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2dfb59e640 1 -- 172.21.15.40:0/3291125244 >> v1:172.21.15.184:6789/0 conn(0x7f2dfc0b2340 legacy=0x7f2dfc0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.580+0000 7f2dfb59e640 1 -- 172.21.15.40:0/3291125244 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc0d1370 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.581+0000 7f2dfb59e640 1 --2- 172.21.15.40:0/3291125244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc0d1fc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc073d90 tx=0x7f2df402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fab1533334ee41e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:00.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.581+0000 7f2df9d9b640 1 -- 172.21.15.40:0/3291125244 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4036070 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.581+0000 7f2df9d9b640 1 -- 172.21.15.40:0/3291125244 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4031600 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.581+0000 7f2df9d9b640 1 -- 172.21.15.40:0/3291125244 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df40319d0 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.582+0000 7f2e011c6640 1 -- 172.21.15.40:0/3291125244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 msgr2=0x7f2dfc0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.582+0000 7f2e011c6640 1 --2- 172.21.15.40:0/3291125244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc0d1fc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc073d90 tx=0x7f2df402a730 comp rx=0 tx=0).stop 2024-04-26T21:41:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.582+0000 7f2e011c6640 1 -- 172.21.15.40:0/3291125244 shutdown_connections 2024-04-26T21:41:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.582+0000 7f2e011c6640 1 --2- 172.21.15.40:0/3291125244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc0d1fc0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.582+0000 7f2e011c6640 1 --2- 172.21.15.40:0/3291125244 >> v2:172.21.15.184:3300/0 conn(0x7f2dfc162810 0x7f2dfc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.582+0000 7f2e011c6640 1 -- 172.21.15.40:0/3291125244 >> 172.21.15.40:0/3291125244 conn(0x7f2dfc0d2da0 msgr2=0x7f2dfc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.582+0000 7f2e011c6640 1 -- 172.21.15.40:0/3291125244 shutdown_connections 2024-04-26T21:41:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.582+0000 7f2e011c6640 1 -- 172.21.15.40:0/3291125244 wait complete. 2024-04-26T21:41:00.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.583+0000 7f2e011c6640 1 Processor -- start 2024-04-26T21:41:00.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.583+0000 7f2e011c6640 1 -- start start 2024-04-26T21:41:00.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2e011c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dfc0b2340 0x7f2dfc15bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2e011c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2e011c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dfc162810 0x7f2dfc159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2e011c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc070890 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2e011c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc0d0470 con 0x7f2dfc0b2340 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2e011c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2dfc0cdb00 con 0x7f2dfc162810 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2dfa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2dfa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc1598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47772/0 (socket says 172.21.15.40:47772) 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2dfa59c640 1 -- 172.21.15.40:0/188098300 learned_addr learned my addr 172.21.15.40:0/188098300 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2dfad9d640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dfc0b2340 0x7f2dfc15bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2dfb59e640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dfc162810 0x7f2dfc159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2dfa59c640 1 -- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dfc162810 msgr2=0x7f2dfc159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2dfa59c640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dfc162810 0x7f2dfc159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.584+0000 7f2dfa59c640 1 -- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dfc0b2340 msgr2=0x7f2dfc15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2dfa59c640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dfc0b2340 0x7f2dfc15bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2dfa59c640 1 -- 172.21.15.40:0/188098300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc148ff0 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2dfad9d640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dfc0b2340 0x7f2dfc15bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:00.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2dfa59c640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc1598d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2df000c430 tx=0x7f2df000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:00.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2de37fe640 1 -- 172.21.15.40:0/188098300 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df00052d0 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2de37fe640 1 -- 172.21.15.40:0/188098300 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df0005490 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc149260 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2de37fe640 1 -- 172.21.15.40:0/188098300 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0007260 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.585+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc149820 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.586+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dfc0b4640 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.586+0000 7f2de37fe640 1 -- 172.21.15.40:0/188098300 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2df0007400 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.587+0000 7f2de37fe640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2de806c920 0x7f2de806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:00.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.587+0000 7f2de37fe640 1 -- 172.21.15.40:0/188098300 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7f2df008aff0 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.587+0000 7f2dfad9d640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2de806c920 0x7f2de806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:00.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.587+0000 7f2dfad9d640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2de806c920 0x7f2de806ede0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009af0 tx=0x7f2dec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:00.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.591+0000 7f2de37fe640 1 -- 172.21.15.40:0/188098300 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2dfc0b4640 con 0x7f2dfc0d1b30 2024-04-26T21:41:00.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:00.850+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7f2dfc0694f0 con 0x7f2dfc0d1b30 2024-04-26T21:41:01.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.827+0000 7f2de37fe640 1 -- 172.21.15.40:0/188098300 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v295) ==== 144+0+0 (secure 0 0 0) 0x7f2df00594e0 con 0x7f2dfc0d1b30 2024-04-26T21:41:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2de806c920 msgr2=0x7f2de806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2de806c920 0x7f2de806ede0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009af0 tx=0x7f2dec008040 comp rx=0 tx=0).stop 2024-04-26T21:41:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 msgr2=0x7f2dfc1598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc1598d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2df000c430 tx=0x7f2df000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 shutdown_connections 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2de806c920 0x7f2de806ede0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dfc162810 0x7f2dfc159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dfc0d1b30 0x7f2dfc1598d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 --2- 172.21.15.40:0/188098300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dfc0b2340 0x7f2dfc15bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 >> 172.21.15.40:0/188098300 conn(0x7f2dfc0d2da0 msgr2=0x7f2dfc0b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 shutdown_connections 2024-04-26T21:41:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.830+0000 7f2e011c6640 1 -- 172.21.15.40:0/188098300 wait complete. 2024-04-26T21:41:01.845 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:41:01.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.955+0000 7f62081a5640 1 Processor -- start 2024-04-26T21:41:01.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.955+0000 7f62081a5640 1 -- start start 2024-04-26T21:41:01.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f62081a5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:01.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f62081a5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f62000d3bc0 0x7f62000d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:01.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f62081a5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62000d1040 con 0x7f620015f550 2024-04-26T21:41:01.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f62081a5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f62000c9430 con 0x7f62000d3bc0 2024-04-26T21:41:01.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f62081a5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f62000c9680 con 0x7f620015f170 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f6205f41640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f6206742640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f62000d3bc0 0x7f62000d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f6206742640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f62000d3bc0 0x7f62000d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49530/0 (socket says 172.21.15.40:49530) 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f6206742640 1 -- 172.21.15.40:0/1637064713 learned_addr learned my addr 172.21.15.40:0/1637064713 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f6205740640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f620015f550 0x7f62000d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39814/0 (socket says 172.21.15.40:39814) 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f6206742640 1 -- 172.21.15.40:0/1637064713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 msgr2=0x7f62000d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.956+0000 7f6206742640 1 --2- 172.21.15.40:0/1637064713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.957+0000 7f6206742640 1 -- 172.21.15.40:0/1637064713 >> v1:172.21.15.184:6789/0 conn(0x7f620015f550 legacy=0x7f62000d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.957+0000 7f6206742640 1 -- 172.21.15.40:0/1637064713 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62000c9960 con 0x7f62000d3bc0 2024-04-26T21:41:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.957+0000 7f6204f3f640 1 -- 172.21.15.40:0/1637064713 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1582859581 0 0) 0x7f61f0001110 con 0x7f620015f550 2024-04-26T21:41:01.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.957+0000 7f6205f41640 1 --2- 172.21.15.40:0/1637064713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:41:01.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.957+0000 7f6206742640 1 --2- 172.21.15.40:0/1637064713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62000d3bc0 0x7f62000d2cb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009e50 tx=0x7f61f802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=931e425182b14282 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.957+0000 7f6204f3f640 1 -- 172.21.15.40:0/1637064713 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8036070 con 0x7f62000d3bc0 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.957+0000 7f6204f3f640 1 -- 172.21.15.40:0/1637064713 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f8031600 con 0x7f62000d3bc0 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f6204f3f640 1 -- 172.21.15.40:0/1637064713 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f80319d0 con 0x7f62000d3bc0 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f62081a5640 1 -- 172.21.15.40:0/1637064713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62000d3bc0 msgr2=0x7f62000d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f62081a5640 1 --2- 172.21.15.40:0/1637064713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62000d3bc0 0x7f62000d2cb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009e50 tx=0x7f61f802a710 comp rx=0 tx=0).stop 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f62081a5640 1 -- 172.21.15.40:0/1637064713 shutdown_connections 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f62081a5640 1 --2- 172.21.15.40:0/1637064713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62000d3bc0 0x7f62000d2cb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f62081a5640 1 --2- 172.21.15.40:0/1637064713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f62081a5640 1 -- 172.21.15.40:0/1637064713 >> 172.21.15.40:0/1637064713 conn(0x7f62000d4f60 msgr2=0x7f62000d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:01.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f62081a5640 1 -- 172.21.15.40:0/1637064713 shutdown_connections 2024-04-26T21:41:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.958+0000 7f62081a5640 1 -- 172.21.15.40:0/1637064713 wait complete. 2024-04-26T21:41:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.959+0000 7f62081a5640 1 Processor -- start 2024-04-26T21:41:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.959+0000 7f62081a5640 1 -- start start 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f62081a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62000d3bc0 0x7f62000cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f62081a5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f62081a5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62001f1e70 0x7f62001f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f62081a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f62000ca5c0 con 0x7f62000d3bc0 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f62081a5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62000ca440 con 0x7f62001f1e70 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f62081a5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f62000ca740 con 0x7f620015f170 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f6205f41640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62000d3bc0 0x7f62000cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f6205f41640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62000d3bc0 0x7f62000cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47786/0 (socket says 172.21.15.40:47786) 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f6205f41640 1 -- 172.21.15.40:0/878909129 learned_addr learned my addr 172.21.15.40:0/878909129 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f6206742640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62001f1e70 0x7f62001f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f6205740640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.960+0000 7f6205f41640 1 -- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 msgr2=0x7f62000ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f6205f41640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f6205f41640 1 -- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62001f1e70 msgr2=0x7f62001f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f6205f41640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62001f1e70 0x7f62001f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f6205f41640 1 -- 172.21.15.40:0/878909129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62001f5ba0 con 0x7f62000d3bc0 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f6206742640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62001f1e70 0x7f62001f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f6205740640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f6205f41640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62000d3bc0 0x7f62000cc9b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00ebe0 tx=0x7f61fc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f61f67fc640 1 -- 172.21.15.40:0/878909129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc017070 con 0x7f62000d3bc0 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f61f67fc640 1 -- 172.21.15.40:0/878909129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc00cc30 con 0x7f62000d3bc0 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62000cd090 con 0x7f62000d3bc0 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f61f67fc640 1 -- 172.21.15.40:0/878909129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc013470 con 0x7f62000d3bc0 2024-04-26T21:41:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.961+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62000cd650 con 0x7f62000d3bc0 2024-04-26T21:41:01.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.962+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62000694f0 con 0x7f62000d3bc0 2024-04-26T21:41:01.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.962+0000 7f61f67fc640 1 -- 172.21.15.40:0/878909129 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61fc01a070 con 0x7f62000d3bc0 2024-04-26T21:41:01.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.963+0000 7f61f67fc640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61e406c8f0 0x7f61e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:01.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.963+0000 7f61f67fc640 1 -- 172.21.15.40:0/878909129 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7f61fc08b9b0 con 0x7f62000d3bc0 2024-04-26T21:41:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.963+0000 7f6205740640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61e406c8f0 0x7f61e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.964+0000 7f6205740640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61e406c8f0 0x7f61e406edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f62000d1c70 tx=0x7f61f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:01.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:01.967+0000 7f61f67fc640 1 -- 172.21.15.40:0/878909129 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61fc059f30 con 0x7f62000d3bc0 2024-04-26T21:41:02.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.151+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f62000d19c0 con 0x7f62000d3bc0 2024-04-26T21:41:02.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.157+0000 7f61f67fc640 1 -- 172.21.15.40:0/878909129 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f61fc059830 con 0x7f62000d3bc0 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61e406c8f0 msgr2=0x7f61e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61e406c8f0 0x7f61e406edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f62000d1c70 tx=0x7f61f0009040 comp rx=0 tx=0).stop 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62000d3bc0 msgr2=0x7f62000cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62000d3bc0 0x7f62000cc9b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00ebe0 tx=0x7f61fc00c520 comp rx=0 tx=0).stop 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 shutdown_connections 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61e406c8f0 0x7f61e406edb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62001f1e70 0x7f62001f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f620015f170 0x7f62000ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 --2- 172.21.15.40:0/878909129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62000d3bc0 0x7f62000cc9b0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 >> 172.21.15.40:0/878909129 conn(0x7f62000d4f60 msgr2=0x7f62000d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:02.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 shutdown_connections 2024-04-26T21:41:02.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.159+0000 7f62081a5640 1 -- 172.21.15.40:0/878909129 wait complete. 2024-04-26T21:41:02.171 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:41:02.171 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:41:02.207 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:41:02.253 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:41:02.253 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:41:02.253 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:02.269 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:41:02.269 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:41:02.269 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:02.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:41:02.286 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:41:02.286 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:02.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:41:02.325 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:41:02.325 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:02.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:41:02.343 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:41:02.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:02.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.465+0000 7f1ac8d1b640 1 Processor -- start 2024-04-26T21:41:02.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.465+0000 7f1ac8d1b640 1 -- start start 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac8d1b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac8d1b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1ac40d1b30 0x7f1ac40d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac8d1b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac40d10d0 con 0x7f1ac40b2340 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac8d1b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac40d0750 con 0x7f1ac40d1b30 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac8d1b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40d16a0 con 0x7f1ac4162810 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac259c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac2d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1ac40d1b30 0x7f1ac40d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1abbfff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f1ac40b2340 0x7f1ac40cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39830/0 (socket says 172.21.15.40:39830) 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac259c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac4070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:48370/0 (socket says 172.21.15.40:48370) 2024-04-26T21:41:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1abbfff640 1 -- 172.21.15.40:0/2708757797 learned_addr learned my addr 172.21.15.40:0/2708757797 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac259c640 1 -- 172.21.15.40:0/2708757797 >> v2:172.21.15.184:3300/0 conn(0x7f1ac40d1b30 msgr2=0x7f1ac40d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac259c640 1 --2- 172.21.15.40:0/2708757797 >> v2:172.21.15.184:3300/0 conn(0x7f1ac40d1b30 0x7f1ac40d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac259c640 1 -- 172.21.15.40:0/2708757797 >> v1:172.21.15.184:6789/0 conn(0x7f1ac40b2340 legacy=0x7f1ac40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac259c640 1 -- 172.21.15.40:0/2708757797 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac40d1370 con 0x7f1ac4162810 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.466+0000 7f1ac2d9d640 1 --2- 172.21.15.40:0/2708757797 >> v2:172.21.15.184:3300/0 conn(0x7f1ac40d1b30 0x7f1ac40d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.467+0000 7f1ac259c640 1 --2- 172.21.15.40:0/2708757797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac4070350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40097a0 tx=0x7f1ab402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb30107834564020 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.467+0000 7f1ac1d9b640 1 -- 172.21.15.40:0/2708757797 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4037070 con 0x7f1ac4162810 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.467+0000 7f1ac1d9b640 1 -- 172.21.15.40:0/2708757797 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab4032440 con 0x7f1ac4162810 2024-04-26T21:41:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac1d9b640 1 -- 172.21.15.40:0/2708757797 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4032810 con 0x7f1ac4162810 2024-04-26T21:41:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2708757797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 msgr2=0x7f1ac4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2708757797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac4070350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40097a0 tx=0x7f1ab402a180 comp rx=0 tx=0).stop 2024-04-26T21:41:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2708757797 shutdown_connections 2024-04-26T21:41:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2708757797 >> v2:172.21.15.184:3300/0 conn(0x7f1ac40d1b30 0x7f1ac40d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2708757797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac4070350 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2708757797 >> 172.21.15.40:0/2708757797 conn(0x7f1ac40d2da0 msgr2=0x7f1ac40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2708757797 shutdown_connections 2024-04-26T21:41:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.468+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2708757797 wait complete. 2024-04-26T21:41:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.469+0000 7f1ac8d1b640 1 Processor -- start 2024-04-26T21:41:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.469+0000 7f1ac8d1b640 1 -- start start 2024-04-26T21:41:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1ac8d1b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac40b2340 0x7f1ac41542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1ac8d1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac40d1b30 0x7f1ac40c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1ac8d1b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac40c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1ac8d1b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4070890 con 0x7f1ac40d1b30 2024-04-26T21:41:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1ac8d1b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac40d0470 con 0x7f1ac40b2340 2024-04-26T21:41:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1ac8d1b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40cdb00 con 0x7f1ac4162810 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1abbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac40d1b30 0x7f1ac40c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1abbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac40d1b30 0x7f1ac40c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47794/0 (socket says 172.21.15.40:47794) 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1abbfff640 1 -- 172.21.15.40:0/2994537505 learned_addr learned my addr 172.21.15.40:0/2994537505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1ac259c640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac40b2340 0x7f1ac41542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.470+0000 7f1ac2d9d640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac40c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abbfff640 1 -- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 msgr2=0x7f1ac40c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abbfff640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac40c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abbfff640 1 -- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac40b2340 msgr2=0x7f1ac41542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abbfff640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac40b2340 0x7f1ac41542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abbfff640 1 -- 172.21.15.40:0/2994537505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac41581d0 con 0x7f1ac40d1b30 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1ac259c640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac40b2340 0x7f1ac41542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abbfff640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac40d1b30 0x7f1ac40c68b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c400 tx=0x7f1aac00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abaffd640 1 -- 172.21.15.40:0/2994537505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac017070 con 0x7f1ac40d1b30 2024-04-26T21:41:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abaffd640 1 -- 172.21.15.40:0/2994537505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac005ce0 con 0x7f1ac40d1b30 2024-04-26T21:41:02.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac41584b0 con 0x7f1ac40d1b30 2024-04-26T21:41:02.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1abaffd640 1 -- 172.21.15.40:0/2994537505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0072f0 con 0x7f1ac40d1b30 2024-04-26T21:41:02.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.471+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac4154820 con 0x7f1ac40d1b30 2024-04-26T21:41:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.472+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac40694f0 con 0x7f1ac40d1b30 2024-04-26T21:41:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.472+0000 7f1abaffd640 1 -- 172.21.15.40:0/2994537505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1aac01a070 con 0x7f1ac40d1b30 2024-04-26T21:41:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.473+0000 7f1abaffd640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa406c8f0 0x7f1aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.473+0000 7f1abaffd640 1 -- 172.21.15.40:0/2994537505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7f1aac08ad60 con 0x7f1ac40d1b30 2024-04-26T21:41:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.473+0000 7f1ac259c640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa406c8f0 0x7f1aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.474+0000 7f1ac259c640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa406c8f0 0x7f1aa406edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4034000 tx=0x7f1ab402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:02.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.477+0000 7f1abaffd640 1 -- 172.21.15.40:0/2994537505 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1aac0592e0 con 0x7f1ac40d1b30 2024-04-26T21:41:02.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.701+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ac4072180 con 0x7f1ac40d1b30 2024-04-26T21:41:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.702+0000 7f1abaffd640 1 -- 172.21.15.40:0/2994537505 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+3875 (secure 0 0 0) 0x7f1aac058be0 con 0x7f1ac40d1b30 2024-04-26T21:41:02.703 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:02.703 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1024461197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1024461197},{"type":"v1","addr":"172.21.15.184:6836","nonce":1024461197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1422037988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1422037988},{"type":"v1","addr":"172.21.15.184:6837","nonce":1422037988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6975,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3716769386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3716769386},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716769386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3131756044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3131756044},{"type":"v1","addr":"172.21.15.40:6834","nonce":3131756044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[]} 2024-04-26T21:41:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 64 2024-04-26T21:41:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa406c8f0 msgr2=0x7f1aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa406c8f0 0x7f1aa406edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4034000 tx=0x7f1ab402e000 comp rx=0 tx=0).stop 2024-04-26T21:41:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac40d1b30 msgr2=0x7f1ac40c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac40d1b30 0x7f1ac40c68b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c400 tx=0x7f1aac00c800 comp rx=0 tx=0).stop 2024-04-26T21:41:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 shutdown_connections 2024-04-26T21:41:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1aa406c8f0 0x7f1aa406edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ac4162810 0x7f1ac40c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ac40d1b30 0x7f1ac40c68b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 --2- 172.21.15.40:0/2994537505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac40b2340 0x7f1ac41542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 >> 172.21.15.40:0/2994537505 conn(0x7f1ac40d2da0 msgr2=0x7f1ac40cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 shutdown_connections 2024-04-26T21:41:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.704+0000 7f1ac8d1b640 1 -- 172.21.15.40:0/2994537505 wait complete. 2024-04-26T21:41:02.716 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:41:02.716 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:41:02.717 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:41:02.717 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:02.717 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:41:02.717 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:41:02.717 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:41:02.717 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:02.717 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:41:02.717 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:41:02.718 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:41:02.718 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:02.718 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:41:02.718 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:41:02.718 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:41:02.718 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:02.726 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:41:02.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:41:02.727 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:41:02.728 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:41:02.729 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:41:02.729 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:41:02.730 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:41:02.730 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:41:02.769 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:41:02.769 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:02.770 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:41:02.770 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:02.772 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:41:02.772 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:02.773 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:41:02.773 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:02.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:41:02.845 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:41:02.846 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:41:02.852 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:41:02.852 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:41:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.878+0000 7f6a12b9d640 1 Processor -- start 2024-04-26T21:41:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.878+0000 7f6a12b9d640 1 -- start start 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.878+0000 7f6a12b9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c1608f0 0x7f6a0c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.878+0000 7f6a12b9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6a0c0d0a90 0x7f6a0c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.878+0000 7f6a12b9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a0c0cfdd0 0x7f6a0c160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.878+0000 7f6a12b9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0c0cea30 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.878+0000 7f6a12b9d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0c0cde20 con 0x7f6a0c0d0a90 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.878+0000 7f6a12b9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0c0ce100 con 0x7f6a0c1608f0 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a0c0cfdd0 0x7f6a0c160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a0c0cfdd0 0x7f6a0c160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47824/0 (socket says 172.21.15.40:47824) 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 -- 172.21.15.40:0/710736385 learned_addr learned my addr 172.21.15.40:0/710736385 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 -- 172.21.15.40:0/710736385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c1608f0 msgr2=0x7f6a0c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 --2- 172.21.15.40:0/710736385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c1608f0 0x7f6a0c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 -- 172.21.15.40:0/710736385 >> v2:172.21.15.184:3300/0 conn(0x7f6a0c0d0a90 msgr2=0x7f6a0c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 --2- 172.21.15.40:0/710736385 >> v2:172.21.15.184:3300/0 conn(0x7f6a0c0d0a90 0x7f6a0c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 -- 172.21.15.40:0/710736385 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c0ce3e0 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.879+0000 7f6a1113a640 1 --2- 172.21.15.40:0/710736385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c160e10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c073b40 tx=0x7f6a0402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b53a158f1e7fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a0b7fe640 1 -- 172.21.15.40:0/710736385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04036070 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a0b7fe640 1 -- 172.21.15.40:0/710736385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a04031600 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a0b7fe640 1 -- 172.21.15.40:0/710736385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a040319d0 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 -- 172.21.15.40:0/710736385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 msgr2=0x7f6a0c160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/710736385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c160e10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c073b40 tx=0x7f6a0402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 -- 172.21.15.40:0/710736385 shutdown_connections 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/710736385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c160e10 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/710736385 >> v2:172.21.15.184:3300/0 conn(0x7f6a0c0d0a90 0x7f6a0c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/710736385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c1608f0 0x7f6a0c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 -- 172.21.15.40:0/710736385 >> 172.21.15.40:0/710736385 conn(0x7f6a0c0d2ec0 msgr2=0x7f6a0c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 -- 172.21.15.40:0/710736385 shutdown_connections 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.880+0000 7f6a12b9d640 1 -- 172.21.15.40:0/710736385 wait complete. 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.882+0000 7f6a12b9d640 1 Processor -- start 2024-04-26T21:41:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.882+0000 7f6a12b9d640 1 -- start start 2024-04-26T21:41:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.882+0000 7f6a12b9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.882+0000 7f6a12b9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c0d0a90 0x7f6a0c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.882+0000 7f6a12b9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c1608f0 0x7f6a0c14cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.882+0000 7f6a12b9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c0ceec0 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.882+0000 7f6a12b9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c0d0ef0 con 0x7f6a0c1608f0 2024-04-26T21:41:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.882+0000 7f6a12b9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0c0cda40 con 0x7f6a0c0d0a90 2024-04-26T21:41:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c0c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47832/0 (socket says 172.21.15.40:47832) 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 -- 172.21.15.40:0/3447105689 learned_addr learned my addr 172.21.15.40:0/3447105689 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a1113a640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c1608f0 0x7f6a0c14cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a0bfff640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c0d0a90 0x7f6a0c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 -- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c0d0a90 msgr2=0x7f6a0c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c0d0a90 0x7f6a0c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 -- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c1608f0 msgr2=0x7f6a0c14cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c1608f0 0x7f6a0c14cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 -- 172.21.15.40:0/3447105689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c0c8ea0 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a0bfff640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c0d0a90 0x7f6a0c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a10939640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c0c8850 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f69f400b410 tx=0x7f69f400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a097fa640 1 -- 172.21.15.40:0/3447105689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4015020 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a097fa640 1 -- 172.21.15.40:0/3447105689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f4011850 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c0c9180 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a097fa640 1 -- 172.21.15.40:0/3447105689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4011c40 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.883+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c071310 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.884+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c06d350 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.885+0000 7f6a097fa640 1 -- 172.21.15.40:0/3447105689 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69f4018020 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.885+0000 7f6a097fa640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69ec06c8f0 0x7f69ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.885+0000 7f6a097fa640 1 -- 172.21.15.40:0/3447105689 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5507+0+0 (secure 0 0 0) 0x7f69f408b9d0 con 0x7f6a0c0cfdd0 2024-04-26T21:41:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.885+0000 7f6a0bfff640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69ec06c8f0 0x7f69ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.886+0000 7f6a0bfff640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69ec06c8f0 0x7f69ec06edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c160300 tx=0x7f69fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:02.890+0000 7f6a097fa640 1 -- 172.21.15.40:0/3447105689 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f69f401f020 con 0x7f6a0c0cfdd0 2024-04-26T21:41:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.088+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6a0c0694f0 con 0x7f6a0c0cfdd0 2024-04-26T21:41:03.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.848+0000 7f6a097fa640 1 -- 172.21.15.40:0/3447105689 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) ==== 97+0+0 (secure 0 0 0) 0x7f69f4059ec0 con 0x7f6a0c0cfdd0 2024-04-26T21:41:03.849 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69ec06c8f0 msgr2=0x7f69ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69ec06c8f0 0x7f69ec06edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c160300 tx=0x7f69fc008040 comp rx=0 tx=0).stop 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 msgr2=0x7f6a0c0c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c0c8850 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f69f400b410 tx=0x7f69f400b8d0 comp rx=0 tx=0).stop 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 shutdown_connections 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69ec06c8f0 0x7f69ec06edb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a0c1608f0 0x7f6a0c14cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a0c0d0a90 0x7f6a0c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 --2- 172.21.15.40:0/3447105689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a0c0cfdd0 0x7f6a0c0c8850 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:03.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 >> 172.21.15.40:0/3447105689 conn(0x7f6a0c0d2ec0 msgr2=0x7f6a0c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:03.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 shutdown_connections 2024-04-26T21:41:03.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.852+0000 7f6a12b9d640 1 -- 172.21.15.40:0/3447105689 wait complete. 2024-04-26T21:41:03.868 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:41:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.974+0000 7f356fec8640 1 Processor -- start 2024-04-26T21:41:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.974+0000 7f356fec8640 1 -- start start 2024-04-26T21:41:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.975+0000 7f356fec8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35680b0380 0x7f35680ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.975+0000 7f356fec8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35680cfd80 con 0x7f356806c950 2024-04-26T21:41:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.975+0000 7f356fec8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35680d0020 con 0x7f35680b10b0 2024-04-26T21:41:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.975+0000 7f356fec8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f35680af8e0 con 0x7f35680b0380 2024-04-26T21:41:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.975+0000 7f356dc64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35680b0380 0x7f35680ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.975+0000 7f356e465640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f356806c950 0x7f356815e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58694/0 (socket says 172.21.15.40:58694) 2024-04-26T21:41:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.975+0000 7f356e465640 1 -- 172.21.15.40:0/1831251660 learned_addr learned my addr 172.21.15.40:0/1831251660 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.976+0000 7f356dc64640 1 -- 172.21.15.40:0/1831251660 >> v1:172.21.15.184:6789/0 conn(0x7f35680b10b0 legacy=0x7f356806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.976+0000 7f356dc64640 1 -- 172.21.15.40:0/1831251660 >> v1:172.21.15.40:6789/0 conn(0x7f356806c950 legacy=0x7f356815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.976+0000 7f356dc64640 1 -- 172.21.15.40:0/1831251660 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35680d0300 con 0x7f35680b0380 2024-04-26T21:41:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.976+0000 7f356cc62640 1 -- 172.21.15.40:0/1831251660 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458388931 0 0) 0x7f35680cfd80 con 0x7f356806c950 2024-04-26T21:41:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.976+0000 7f356dc64640 1 --2- 172.21.15.40:0/1831251660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f35680ccd50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f35640097a0 tx=0x7f356402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746e49f597bbc123 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356cc62640 1 -- 172.21.15.40:0/1831251660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564037070 con 0x7f35680b0380 2024-04-26T21:41:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356cc62640 1 -- 172.21.15.40:0/1831251660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f356402ae50 con 0x7f35680b0380 2024-04-26T21:41:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356cc62640 1 -- 172.21.15.40:0/1831251660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564032660 con 0x7f35680b0380 2024-04-26T21:41:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356fec8640 1 -- 172.21.15.40:0/1831251660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 msgr2=0x7f35680ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:03.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356fec8640 1 --2- 172.21.15.40:0/1831251660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f35680ccd50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f35640097a0 tx=0x7f356402a140 comp rx=0 tx=0).stop 2024-04-26T21:41:03.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356fec8640 1 -- 172.21.15.40:0/1831251660 shutdown_connections 2024-04-26T21:41:03.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356fec8640 1 --2- 172.21.15.40:0/1831251660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f35680ccd50 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:03.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356fec8640 1 -- 172.21.15.40:0/1831251660 >> 172.21.15.40:0/1831251660 conn(0x7f35680d1cf0 msgr2=0x7f35680d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:03.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356fec8640 1 -- 172.21.15.40:0/1831251660 shutdown_connections 2024-04-26T21:41:03.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.977+0000 7f356fec8640 1 -- 172.21.15.40:0/1831251660 wait complete. 2024-04-26T21:41:03.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.978+0000 7f356fec8640 1 Processor -- start 2024-04-26T21:41:03.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.978+0000 7f356fec8640 1 -- start start 2024-04-26T21:41:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356fec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f356806c950 0x7f3568160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356fec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f356815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356fec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35680b10b0 0x7f356815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356fec8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35680b0760 con 0x7f35680b0380 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356fec8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35680d1980 con 0x7f35680b10b0 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356fec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35680d15c0 con 0x7f356806c950 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356d463640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f356815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356d463640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f356815cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47858/0 (socket says 172.21.15.40:47858) 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.979+0000 7f356d463640 1 -- 172.21.15.40:0/564425055 learned_addr learned my addr 172.21.15.40:0/564425055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356e465640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35680b10b0 0x7f356815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356dc64640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f356806c950 0x7f3568160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356d463640 1 -- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f356806c950 msgr2=0x7f3568160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356d463640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f356806c950 0x7f3568160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356d463640 1 -- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35680b10b0 msgr2=0x7f356815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356d463640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35680b10b0 0x7f356815d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356d463640 1 -- 172.21.15.40:0/564425055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356815edd0 con 0x7f35680b0380 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356e465640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35680b10b0 0x7f356815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356dc64640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f356806c950 0x7f3568160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356d463640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f356815cf60 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f355800c430 tx=0x7f355800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f355e7fc640 1 -- 172.21.15.40:0/564425055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558011790 con 0x7f35680b0380 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f355e7fc640 1 -- 172.21.15.40:0/564425055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3558017070 con 0x7f35680b0380 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35681f4fb0 con 0x7f35680b0380 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f355e7fc640 1 -- 172.21.15.40:0/564425055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558011bb0 con 0x7f35680b0380 2024-04-26T21:41:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.980+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35681f4c80 con 0x7f35680b0380 2024-04-26T21:41:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.981+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35680694f0 con 0x7f35680b0380 2024-04-26T21:41:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.982+0000 7f355e7fc640 1 -- 172.21.15.40:0/564425055 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f355801c460 con 0x7f35680b0380 2024-04-26T21:41:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.982+0000 7f355e7fc640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f354c06c8f0 0x7f354c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.982+0000 7f355e7fc640 1 -- 172.21.15.40:0/564425055 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5249+0+0 (secure 0 0 0) 0x7f355808b830 con 0x7f35680b0380 2024-04-26T21:41:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.982+0000 7f356dc64640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f354c06c8f0 0x7f354c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.983+0000 7f356dc64640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f354c06c8f0 0x7f354c06edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3564034000 tx=0x7f356402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:03.987+0000 7f355e7fc640 1 -- 172.21.15.40:0/564425055 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3558059e20 con 0x7f35680b0380 2024-04-26T21:41:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.244+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f35680d26f0 con 0x7f35680b0380 2024-04-26T21:41:04.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.244+0000 7f355e7fc640 1 -- 172.21.15.40:0/564425055 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f3558059720 con 0x7f35680b0380 2024-04-26T21:41:04.245 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:04.245 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:41:04.245 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmesG+ERAAJS53jYoKUawIOs2zrG5TLg==", 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmq9cvHRAAiYWrMjfAth8wMbs9YlfWgQ==", 2024-04-26T21:41:04.246 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmLRmvGBAA6xAcjLDF7DMjfDqiUCexKA==", 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.247 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmXxkDJBAAbzzf+LUO3Rr8bdzXb+Gj2Q==", 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDYHSxmRkhFBhAAz1UfvJwhA8bolDiRN2eM+Q==", 2024-04-26T21:41:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDbHSxmKAB/BBAAzW5dtz6eczjkDiTQBjfjuA==", 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDeHSxmrienAhAA2JS0l6/Uo9UPc0TeZ1RwRA==", 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDhHSxmv40rARAAN8z+beJe/bTERvO2euMr6w==", 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:41:04.250 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDnHSxmxQ1XCBAAYptVM8WPir+zIHOjbX0tGg==", 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:41:04.251 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDqHSxmv32IBhAAMROkCLjWTOx3ELdkEiVSmw==", 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDtHSxmJdPTBBAAO32qvVLuAzqx1xdajL9CFw==", 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDwHSxmcUMYAxAAIvoAo82ApDXERodD+4+cBQ==", 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw==", 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:41:04.253 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmifWYMBAA6Nskvqt3MZSvFCMHjslELw==", 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.254 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmi6l6LRAAF4EjChVEB8ReULEsl3YppA==", 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmhSMFNhAAjCJvc4FknsSUJQ4NjXRV6Q==", 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:41:04.255 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDTHSxmQk1bMhAAixlszeAmlXc5cAmkFWjhfw==", 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.256 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmYjmPDhAADvvIHrcacI8kOAdeiJGd+Q==", 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmsiwICxAAmKAZ2KS3/dYubutkcGfolA==", 2024-04-26T21:41:04.257 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:41:04.258 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:41:04.258 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:41:04.258 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:41:04.258 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.258 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:04.258 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:41:04.258 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:41:04.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f354c06c8f0 msgr2=0x7f354c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f354c06c8f0 0x7f354c06edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3564034000 tx=0x7f356402e000 comp rx=0 tx=0).stop 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 msgr2=0x7f356815cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f356815cf60 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f355800c430 tx=0x7f355800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 shutdown_connections 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f354c06c8f0 0x7f354c06edb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35680b10b0 0x7f356815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35680b0380 0x7f356815cf60 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 --2- 172.21.15.40:0/564425055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f356806c950 0x7f3568160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 >> 172.21.15.40:0/564425055 conn(0x7f35680d1cf0 msgr2=0x7f35680b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 shutdown_connections 2024-04-26T21:41:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.246+0000 7f356fec8640 1 -- 172.21.15.40:0/564425055 wait complete. 2024-04-26T21:41:04.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.360+0000 7f595d589640 1 Processor -- start 2024-04-26T21:41:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.360+0000 7f595d589640 1 -- start start 2024-04-26T21:41:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f595d589640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59580d06f0 0x7f59580d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f595d589640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f59580b2850 0x7f59580b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f595d589640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59580d0ba0 con 0x7f595815f3a0 2024-04-26T21:41:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f595d589640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f59580d0e20 con 0x7f59580d06f0 2024-04-26T21:41:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f595d589640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f59580b1d70 con 0x7f59580b2850 2024-04-26T21:41:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f59567fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59580d06f0 0x7f59580d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f59567fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59580d06f0 0x7f59580d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47872/0 (socket says 172.21.15.40:47872) 2024-04-26T21:41:04.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f59567fc640 1 -- 172.21.15.40:0/57931607 learned_addr learned my addr 172.21.15.40:0/57931607 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:04.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f59567fc640 1 -- 172.21.15.40:0/57931607 >> v2:172.21.15.184:3300/0 conn(0x7f59580b2850 msgr2=0x7f59580b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:04.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f59567fc640 1 --2- 172.21.15.40:0/57931607 >> v2:172.21.15.184:3300/0 conn(0x7f59580b2850 0x7f59580b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f59567fc640 1 -- 172.21.15.40:0/57931607 >> v1:172.21.15.184:6789/0 conn(0x7f595815f3a0 legacy=0x7f59580d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:04.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.361+0000 7f59567fc640 1 -- 172.21.15.40:0/57931607 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580b2050 con 0x7f59580d06f0 2024-04-26T21:41:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.362+0000 7f59567fc640 1 --2- 172.21.15.40:0/57931607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f59580d1e80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f59580c89f0 tx=0x7f594c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80d28f54fc6f5fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.362+0000 7f5955ffb640 1 -- 172.21.15.40:0/57931607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c037070 con 0x7f59580d06f0 2024-04-26T21:41:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.362+0000 7f5955ffb640 1 -- 172.21.15.40:0/57931607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c032440 con 0x7f59580d06f0 2024-04-26T21:41:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.362+0000 7f5955ffb640 1 -- 172.21.15.40:0/57931607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c032810 con 0x7f59580d06f0 2024-04-26T21:41:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.363+0000 7f595d589640 1 -- 172.21.15.40:0/57931607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 msgr2=0x7f59580d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.363+0000 7f595d589640 1 --2- 172.21.15.40:0/57931607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f59580d1e80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f59580c89f0 tx=0x7f594c02a230 comp rx=0 tx=0).stop 2024-04-26T21:41:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.363+0000 7f595d589640 1 -- 172.21.15.40:0/57931607 shutdown_connections 2024-04-26T21:41:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.363+0000 7f595d589640 1 --2- 172.21.15.40:0/57931607 >> v2:172.21.15.184:3300/0 conn(0x7f59580b2850 0x7f59580b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.363+0000 7f595d589640 1 --2- 172.21.15.40:0/57931607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f59580d1e80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.363+0000 7f595d589640 1 -- 172.21.15.40:0/57931607 >> 172.21.15.40:0/57931607 conn(0x7f59580d4f90 msgr2=0x7f59580d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.363+0000 7f595d589640 1 -- 172.21.15.40:0/57931607 shutdown_connections 2024-04-26T21:41:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.363+0000 7f595d589640 1 -- 172.21.15.40:0/57931607 wait complete. 2024-04-26T21:41:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.364+0000 7f595d589640 1 Processor -- start 2024-04-26T21:41:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.364+0000 7f595d589640 1 -- start start 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f595d589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59580b2850 0x7f595814cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f595d589640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f595806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f595d589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f595815f3a0 0x7f59580cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f595d589640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f595815f900 con 0x7f59580d06f0 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f595d589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f595815f780 con 0x7f595815f3a0 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f595d589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f59580d23c0 con 0x7f59580b2850 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f595806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f595806d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47874/0 (socket says 172.21.15.40:47874) 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 -- 172.21.15.40:0/131641615 learned_addr learned my addr 172.21.15.40:0/131641615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59577fe640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f595815f3a0 0x7f59580cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 -- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59580b2850 msgr2=0x7f595814cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f5956ffd640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59580b2850 0x7f595814cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59580b2850 0x7f595814cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 -- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f595815f3a0 msgr2=0x7f59580cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f595815f3a0 0x7f59580cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 -- 172.21.15.40:0/131641615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595814d000 con 0x7f59580d06f0 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59577fe640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f595815f3a0 0x7f59580cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.365+0000 7f59567fc640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f595806d6b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f594c000c00 tx=0x7f594c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.366+0000 7f593f7fe640 1 -- 172.21.15.40:0/131641615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c041070 con 0x7f59580d06f0 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.366+0000 7f593f7fe640 1 -- 172.21.15.40:0/131641615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c03c870 con 0x7f59580d06f0 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.366+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595806e100 con 0x7f59580d06f0 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.366+0000 7f593f7fe640 1 -- 172.21.15.40:0/131641615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c037040 con 0x7f59580d06f0 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.366+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59580c99e0 con 0x7f59580d06f0 2024-04-26T21:41:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.367+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595806e620 con 0x7f59580d06f0 2024-04-26T21:41:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.367+0000 7f593f7fe640 1 -- 172.21.15.40:0/131641615 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f594c044070 con 0x7f59580d06f0 2024-04-26T21:41:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.368+0000 7f593f7fe640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.368+0000 7f593f7fe640 1 -- 172.21.15.40:0/131641615 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5249+0+0 (secure 0 0 0) 0x7f594c0ac560 con 0x7f59580d06f0 2024-04-26T21:41:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.368+0000 7f5956ffd640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.368+0000 7f5956ffd640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f594800a730 tx=0x7f5948008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.372+0000 7f593f7fe640 1 -- 172.21.15.40:0/131641615 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f59580d0ba0 con 0x7f59580d06f0 2024-04-26T21:41:04.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.560+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59580694f0 con 0x7f59580d06f0 2024-04-26T21:41:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.561+0000 7f593f7fe640 1 -- 172.21.15.40:0/131641615 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+13129 (secure 0 0 0) 0x7f594c07abe0 con 0x7f59580d06f0 2024-04-26T21:41:04.562 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:04.562 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":77,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:03.856744+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:04.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 msgr2=0x7f594406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f594800a730 tx=0x7f5948008040 comp rx=0 tx=0).stop 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 msgr2=0x7f595806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f595806d6b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f594c000c00 tx=0x7f594c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 shutdown_connections 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f595815f3a0 0x7f59580cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59580d06f0 0x7f595806d6b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 --2- 172.21.15.40:0/131641615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59580b2850 0x7f595814cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 >> 172.21.15.40:0/131641615 conn(0x7f59580d4f90 msgr2=0x7f595815e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 shutdown_connections 2024-04-26T21:41:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.563+0000 7f595d589640 1 -- 172.21.15.40:0/131641615 wait complete. 2024-04-26T21:41:04.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:41:04.576 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:41:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.679+0000 7fb03fd97640 1 Processor -- start 2024-04-26T21:41:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.679+0000 7fb03fd97640 1 -- start start 2024-04-26T21:41:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03fd97640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb03815f550 0x7fb0380d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03fd97640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb0380d3bc0 0x7fb0380d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03fd97640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0380d1040 con 0x7fb03815f170 2024-04-26T21:41:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03fd97640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb0380c9430 con 0x7fb03815f550 2024-04-26T21:41:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03fd97640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb0380c9680 con 0x7fb0380d3bc0 2024-04-26T21:41:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03d332640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb03815f550 0x7fb0380d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03db33640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb03815f170 0x7fb0380d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58700/0 (socket says 172.21.15.40:58700) 2024-04-26T21:41:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03db33640 1 -- 172.21.15.40:0/2637392740 learned_addr learned my addr 172.21.15.40:0/2637392740 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03e334640 1 --2- 172.21.15.40:0/2637392740 >> v2:172.21.15.184:3300/0 conn(0x7fb0380d3bc0 0x7fb0380d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03d332640 1 -- 172.21.15.40:0/2637392740 >> v2:172.21.15.184:3300/0 conn(0x7fb0380d3bc0 msgr2=0x7fb0380d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03d332640 1 --2- 172.21.15.40:0/2637392740 >> v2:172.21.15.184:3300/0 conn(0x7fb0380d3bc0 0x7fb0380d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03d332640 1 -- 172.21.15.40:0/2637392740 >> v1:172.21.15.40:6789/0 conn(0x7fb03815f170 legacy=0x7fb0380d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03cb31640 1 -- 172.21.15.40:0/2637392740 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3238870422 0 0) 0x7fb034001110 con 0x7fb03815f170 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03d332640 1 -- 172.21.15.40:0/2637392740 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0380c9960 con 0x7fb03815f550 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.680+0000 7fb03e334640 1 --2- 172.21.15.40:0/2637392740 >> v2:172.21.15.184:3300/0 conn(0x7fb0380d3bc0 0x7fb0380d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.681+0000 7fb03d332640 1 --2- 172.21.15.40:0/2637392740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03815f550 0x7fb0380d5a80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb0280097a0 tx=0x7fb02802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf18301b1d9ecba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.681+0000 7fb03cb31640 1 -- 172.21.15.40:0/2637392740 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028037070 con 0x7fb03815f550 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.681+0000 7fb03cb31640 1 -- 172.21.15.40:0/2637392740 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb028032440 con 0x7fb03815f550 2024-04-26T21:41:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.681+0000 7fb03cb31640 1 -- 172.21.15.40:0/2637392740 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028032810 con 0x7fb03815f550 2024-04-26T21:41:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.682+0000 7fb03fd97640 1 -- 172.21.15.40:0/2637392740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03815f550 msgr2=0x7fb0380d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.682+0000 7fb03fd97640 1 --2- 172.21.15.40:0/2637392740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03815f550 0x7fb0380d5a80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb0280097a0 tx=0x7fb02802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.682+0000 7fb03fd97640 1 -- 172.21.15.40:0/2637392740 shutdown_connections 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.682+0000 7fb03fd97640 1 --2- 172.21.15.40:0/2637392740 >> v2:172.21.15.184:3300/0 conn(0x7fb0380d3bc0 0x7fb0380d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.682+0000 7fb03fd97640 1 --2- 172.21.15.40:0/2637392740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03815f550 0x7fb0380d5a80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.682+0000 7fb03fd97640 1 -- 172.21.15.40:0/2637392740 >> 172.21.15.40:0/2637392740 conn(0x7fb0380d4f60 msgr2=0x7fb0380d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.682+0000 7fb03fd97640 1 -- 172.21.15.40:0/2637392740 shutdown_connections 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.682+0000 7fb03fd97640 1 -- 172.21.15.40:0/2637392740 wait complete. 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.683+0000 7fb03fd97640 1 Processor -- start 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.683+0000 7fb03fd97640 1 -- start start 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03fd97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d3bc0 0x7fb0381f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03fd97640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03815f170 0x7fb0381595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03fd97640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb038159b30 0x7fb0380b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03fd97640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380ca5c0 con 0x7fb0380d3bc0 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03fd97640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380ca440 con 0x7fb03815f170 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03fd97640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0380ca740 con 0x7fb038159b30 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d3bc0 0x7fb0381f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d3bc0 0x7fb0381f79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47892/0 (socket says 172.21.15.40:47892) 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 -- 172.21.15.40:0/1039525951 learned_addr learned my addr 172.21.15.40:0/1039525951 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03d332640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03815f170 0x7fb0381595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03e334640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb038159b30 0x7fb0380b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 -- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb038159b30 msgr2=0x7fb0380b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb038159b30 0x7fb0380b9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 -- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03815f170 msgr2=0x7fb0381595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03815f170 0x7fb0381595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 -- 172.21.15.40:0/1039525951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03815a0b0 con 0x7fb0380d3bc0 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03e334640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb038159b30 0x7fb0380b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03db33640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d3bc0 0x7fb0381f79e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c780 tx=0x7fb03400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb02e7fc640 1 -- 172.21.15.40:0/1039525951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034018070 con 0x7fb0380d3bc0 2024-04-26T21:41:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb02e7fc640 1 -- 172.21.15.40:0/1039525951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0340072d0 con 0x7fb0380d3bc0 2024-04-26T21:41:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03815a390 con 0x7fb0380d3bc0 2024-04-26T21:41:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.684+0000 7fb02e7fc640 1 -- 172.21.15.40:0/1039525951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0340047e0 con 0x7fb0380d3bc0 2024-04-26T21:41:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.685+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03815a950 con 0x7fb0380d3bc0 2024-04-26T21:41:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.686+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0381f4d70 con 0x7fb0380d3bc0 2024-04-26T21:41:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.686+0000 7fb02e7fc640 1 -- 172.21.15.40:0/1039525951 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb03401b070 con 0x7fb0380d3bc0 2024-04-26T21:41:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.687+0000 7fb02e7fc640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01c06c8f0 0x7fb01c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.687+0000 7fb02e7fc640 1 -- 172.21.15.40:0/1039525951 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5249+0+0 (secure 0 0 0) 0x7fb03408ac70 con 0x7fb0380d3bc0 2024-04-26T21:41:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.687+0000 7fb03d332640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01c06c8f0 0x7fb01c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.687+0000 7fb03d332640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01c06c8f0 0x7fb01c06edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb028034000 tx=0x7fb02802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.691+0000 7fb02e7fc640 1 -- 172.21.15.40:0/1039525951 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0380d1040 con 0x7fb0380d3bc0 2024-04-26T21:41:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.876+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0380694f0 con 0x7fb0380d3bc0 2024-04-26T21:41:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.876+0000 7fb02e7fc640 1 -- 172.21.15.40:0/1039525951 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fb0340592f0 con 0x7fb0380d3bc0 2024-04-26T21:41:04.877 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:41:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01c06c8f0 msgr2=0x7fb01c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01c06c8f0 0x7fb01c06edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb028034000 tx=0x7fb02802e000 comp rx=0 tx=0).stop 2024-04-26T21:41:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d3bc0 msgr2=0x7fb0381f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d3bc0 0x7fb0381f79e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c780 tx=0x7fb03400cc40 comp rx=0 tx=0).stop 2024-04-26T21:41:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 shutdown_connections 2024-04-26T21:41:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01c06c8f0 0x7fb01c06edb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb038159b30 0x7fb0380b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03815f170 0x7fb0381595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 --2- 172.21.15.40:0/1039525951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d3bc0 0x7fb0381f79e0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 >> 172.21.15.40:0/1039525951 conn(0x7fb0380d4f60 msgr2=0x7fb0380d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:04.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 shutdown_connections 2024-04-26T21:41:04.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.878+0000 7fb03fd97640 1 -- 172.21.15.40:0/1039525951 wait complete. 2024-04-26T21:41:04.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:41:04.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.989+0000 7face66da640 1 Processor -- start 2024-04-26T21:41:04.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.989+0000 7face66da640 1 -- start start 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7face66da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 0x7face0167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7face66da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face00ce840 con 0x7face015d1e0 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7face66da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face00ceb00 con 0x7face00d1380 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7face66da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7face00cede0 con 0x7face0071af0 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7facdffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7face015d1e0 0x7face00d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58712/0 (socket says 172.21.15.40:58712) 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7facdffff640 1 -- 172.21.15.40:0/2621692720 learned_addr learned my addr 172.21.15.40:0/2621692720 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7face4c77640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 0x7face0167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122740339 0 0) 0x7face00ce840 con 0x7face015d1e0 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.990+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facc8003520 con 0x7face015d1e0 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1879856868 0 0) 0x7facc8003520 con 0x7face015d1e0 2024-04-26T21:41:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 msgr2=0x7face0167bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 0x7face0167bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 >> v1:172.21.15.184:6789/0 conn(0x7face00d1380 legacy=0x7face00d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face00702e0 con 0x7face015d1e0 2024-04-26T21:41:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7facd4001670 con 0x7face015d1e0 2024-04-26T21:41:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 >> v1:172.21.15.40:6789/0 conn(0x7face015d1e0 legacy=0x7face00d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7facc8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 0x7facc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc8004d40 0x7facc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7face00cede0 con 0x7facc80039e0 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facc800baa0 con 0x7facc8004d40 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7facc800bca0 con 0x7face0071af0 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdffff640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7facc8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdf7fe640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 0x7facc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7face4c77640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc8004d40 0x7facc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdffff640 1 -- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 msgr2=0x7facc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdffff640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 0x7facc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdffff640 1 -- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc8004d40 msgr2=0x7facc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdffff640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc8004d40 0x7facc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdffff640 1 -- 172.21.15.40:0/2621692720 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc8003520 con 0x7facc80039e0 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7facdf7fe640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face0071af0 0x7facc8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.991+0000 7face4c77640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc8004d40 0x7facc80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7facdffff640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7facc8003e00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7facd4001eb0 tx=0x7facc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f367556c3cbce98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4007d30 con 0x7facc80039e0 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd40058d0 con 0x7facc80039e0 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7facdeffd640 1 -- 172.21.15.40:0/2621692720 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4005ce0 con 0x7facc80039e0 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7face66da640 1 -- 172.21.15.40:0/2621692720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 msgr2=0x7facc8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7face66da640 1 --2- 172.21.15.40:0/2621692720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7facc8003e00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7facd4001eb0 tx=0x7facc8005830 comp rx=0 tx=0).stop 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7face4c77640 1 -- 172.21.15.40:0/2621692720 reap_dead start 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7face66da640 1 -- 172.21.15.40:0/2621692720 shutdown_connections 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.992+0000 7face66da640 1 -- 172.21.15.40:0/2621692720 >> 172.21.15.40:0/2621692720 conn(0x7face00d2da0 msgr2=0x7face00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.993+0000 7face66da640 1 -- 172.21.15.40:0/2621692720 shutdown_connections 2024-04-26T21:41:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.993+0000 7face66da640 1 -- 172.21.15.40:0/2621692720 wait complete. 2024-04-26T21:41:05.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.994+0000 7face66da640 1 Processor -- start 2024-04-26T21:41:05.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.994+0000 7face66da640 1 -- start start 2024-04-26T21:41:05.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.994+0000 7face66da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7face0159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.994+0000 7face66da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face01581e0 0x7face00ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.994+0000 7face66da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7face00b2a00 0x7face01557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7face66da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7face00ce4c0 con 0x7facc80039e0 2024-04-26T21:41:05.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7face66da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7face0070650 con 0x7face00b2a00 2024-04-26T21:41:05.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7face66da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7face0165bb0 con 0x7face01581e0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7face0159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7face0159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47902/0 (socket says 172.21.15.40:47902) 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 -- 172.21.15.40:0/1795124137 learned_addr learned my addr 172.21.15.40:0/1795124137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7face4c77640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7face00b2a00 0x7face01557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdf7fe640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face01581e0 0x7face00ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 -- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face01581e0 msgr2=0x7face00ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face01581e0 0x7face00ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 -- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7face00b2a00 msgr2=0x7face01557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7face00b2a00 0x7face01557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 -- 172.21.15.40:0/1795124137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face01586e0 con 0x7facc80039e0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdffff640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7face0159780 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7facd40058a0 tx=0x7facd4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdcff9640 1 -- 172.21.15.40:0/1795124137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4042020 con 0x7facc80039e0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdcff9640 1 -- 172.21.15.40:0/1795124137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4007d10 con 0x7facc80039e0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face01589c0 con 0x7facc80039e0 2024-04-26T21:41:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7facdcff9640 1 -- 172.21.15.40:0/1795124137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40074e0 con 0x7facc80039e0 2024-04-26T21:41:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.995+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face0158f80 con 0x7facc80039e0 2024-04-26T21:41:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.996+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face00694f0 con 0x7facc80039e0 2024-04-26T21:41:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.997+0000 7facdcff9640 1 -- 172.21.15.40:0/1795124137 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facd4045020 con 0x7facc80039e0 2024-04-26T21:41:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.997+0000 7facdcff9640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faccc06c920 0x7faccc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.997+0000 7facdcff9640 1 -- 172.21.15.40:0/1795124137 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5249+0+0 (secure 0 0 0) 0x7facd40abc10 con 0x7facc80039e0 2024-04-26T21:41:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.997+0000 7facdf7fe640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faccc06c920 0x7faccc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:04.998+0000 7facdf7fe640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faccc06c920 0x7faccc06ede0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7face00a0db0 tx=0x7facd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.002+0000 7facdcff9640 1 -- 172.21.15.40:0/1795124137 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7facd404c030 con 0x7facc80039e0 2024-04-26T21:41:05.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.189+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7face00afd90 con 0x7facc80039e0 2024-04-26T21:41:05.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.189+0000 7facdcff9640 1 -- 172.21.15.40:0/1795124137 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7facd407a3b0 con 0x7facc80039e0 2024-04-26T21:41:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:05.190 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:41:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:41:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:41:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:41:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:41:05.191 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:41:05.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faccc06c920 msgr2=0x7faccc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faccc06c920 0x7faccc06ede0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7face00a0db0 tx=0x7facd0009040 comp rx=0 tx=0).stop 2024-04-26T21:41:05.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 msgr2=0x7face0159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7face0159780 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7facd40058a0 tx=0x7facd4001a40 comp rx=0 tx=0).stop 2024-04-26T21:41:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 shutdown_connections 2024-04-26T21:41:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faccc06c920 0x7faccc06ede0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7face00b2a00 0x7face01557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7face01581e0 0x7face00ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 --2- 172.21.15.40:0/1795124137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc80039e0 0x7face0159780 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 >> 172.21.15.40:0/1795124137 conn(0x7face00d2da0 msgr2=0x7face00cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 shutdown_connections 2024-04-26T21:41:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.191+0000 7face66da640 1 -- 172.21.15.40:0/1795124137 wait complete. 2024-04-26T21:41:05.203 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:41:05.203 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:41:05.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.306+0000 7f1a1da47640 1 Processor -- start 2024-04-26T21:41:05.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.306+0000 7f1a1da47640 1 -- start start 2024-04-26T21:41:05.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a1da47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a18160d00 0x7f1a18161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a1da47640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1a180b29d0 0x7f1a180b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a1da47640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a18161b30 con 0x7f1a1815d100 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a1da47640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1a181616a0 con 0x7f1a180b29d0 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a1da47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1a180d31e0 con 0x7f1a18160d00 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a16ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1a1815d100 0x7f1a180d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58714/0 (socket says 172.21.15.40:58714) 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a16ffd640 1 -- 172.21.15.40:0/254323893 learned_addr learned my addr 172.21.15.40:0/254323893 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a177fe640 1 --2- 172.21.15.40:0/254323893 >> v2:172.21.15.40:3300/0 conn(0x7f1a180b29d0 0x7f1a180b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.307+0000 7f1a167fc640 1 --2- 172.21.15.40:0/254323893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a18160d00 0x7f1a18161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.308+0000 7f1a177fe640 1 -- 172.21.15.40:0/254323893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a18160d00 msgr2=0x7f1a18161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.308+0000 7f1a177fe640 1 --2- 172.21.15.40:0/254323893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a18160d00 0x7f1a18161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.308+0000 7f1a177fe640 1 -- 172.21.15.40:0/254323893 >> v1:172.21.15.40:6789/0 conn(0x7f1a1815d100 legacy=0x7f1a180d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.308+0000 7f1a15ffb640 1 -- 172.21.15.40:0/254323893 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382893920 0 0) 0x7f1a18161b30 con 0x7f1a1815d100 2024-04-26T21:41:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.308+0000 7f1a177fe640 1 -- 172.21.15.40:0/254323893 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a180d34c0 con 0x7f1a180b29d0 2024-04-26T21:41:05.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.308+0000 7f1a177fe640 1 --2- 172.21.15.40:0/254323893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a180b29d0 0x7f1a180b2e40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009d90 tx=0x7f1a1002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a729556278c866a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.308+0000 7f1a15ffb640 1 -- 172.21.15.40:0/254323893 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10031600 con 0x7f1a180b29d0 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a15ffb640 1 -- 172.21.15.40:0/254323893 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a100317a0 con 0x7f1a180b29d0 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a15ffb640 1 -- 172.21.15.40:0/254323893 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10031b70 con 0x7f1a180b29d0 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a1da47640 1 -- 172.21.15.40:0/254323893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a180b29d0 msgr2=0x7f1a180b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a1da47640 1 --2- 172.21.15.40:0/254323893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a180b29d0 0x7f1a180b2e40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009d90 tx=0x7f1a1002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a1da47640 1 -- 172.21.15.40:0/254323893 shutdown_connections 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a1da47640 1 --2- 172.21.15.40:0/254323893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a180b29d0 0x7f1a180b2e40 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a1da47640 1 --2- 172.21.15.40:0/254323893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a18160d00 0x7f1a18161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a1da47640 1 -- 172.21.15.40:0/254323893 >> 172.21.15.40:0/254323893 conn(0x7f1a1809ba50 msgr2=0x7f1a1809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a1da47640 1 -- 172.21.15.40:0/254323893 shutdown_connections 2024-04-26T21:41:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.309+0000 7f1a1da47640 1 -- 172.21.15.40:0/254323893 wait complete. 2024-04-26T21:41:05.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.310+0000 7f1a1da47640 1 Processor -- start 2024-04-26T21:41:05.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.310+0000 7f1a1da47640 1 -- start start 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a1da47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a180b29d0 0x7f1a180a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a1da47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a1815d100 0x7f1a180b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a1da47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a18160d00 0x7f1a180b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a1da47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1815e4f0 con 0x7f1a18160d00 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a1da47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a180d3a80 con 0x7f1a180b29d0 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a1da47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1a180d2e00 con 0x7f1a1815d100 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a18160d00 0x7f1a180b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a18160d00 0x7f1a180b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47920/0 (socket says 172.21.15.40:47920) 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.311+0000 7f1a177fe640 1 -- 172.21.15.40:0/1242379902 learned_addr learned my addr 172.21.15.40:0/1242379902 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a177fe640 1 -- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a1815d100 msgr2=0x7f1a180b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a167fc640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a1815d100 0x7f1a180b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a16ffd640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a180b29d0 0x7f1a180a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a177fe640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a1815d100 0x7f1a180b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a177fe640 1 -- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a180b29d0 msgr2=0x7f1a180a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a177fe640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a180b29d0 0x7f1a180a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a177fe640 1 -- 172.21.15.40:0/1242379902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a180b6750 con 0x7f1a18160d00 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a16ffd640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a180b29d0 0x7f1a180a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a167fc640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a1815d100 0x7f1a180b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a177fe640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a18160d00 0x7f1a180b6010 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009eb0 tx=0x7f1a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a0ffff640 1 -- 172.21.15.40:0/1242379902 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1003f070 con 0x7f1a18160d00 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a0ffff640 1 -- 172.21.15.40:0/1242379902 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a10030be0 con 0x7f1a18160d00 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a0ffff640 1 -- 172.21.15.40:0/1242379902 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10009450 con 0x7f1a18160d00 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a180b6a30 con 0x7f1a18160d00 2024-04-26T21:41:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.312+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a180b6ff0 con 0x7f1a18160d00 2024-04-26T21:41:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.313+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a18069600 con 0x7f1a18160d00 2024-04-26T21:41:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.314+0000 7f1a0ffff640 1 -- 172.21.15.40:0/1242379902 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a10003990 con 0x7f1a18160d00 2024-04-26T21:41:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.314+0000 7f1a0ffff640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19f806c8f0 0x7f19f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.314+0000 7f1a0ffff640 1 -- 172.21.15.40:0/1242379902 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5249+0+0 (secure 0 0 0) 0x7f1a100ac4e0 con 0x7f1a18160d00 2024-04-26T21:41:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.314+0000 7f1a16ffd640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19f806c8f0 0x7f19f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.315+0000 7f1a16ffd640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19f806c8f0 0x7f19f806edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000f60 tx=0x7f1a00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.318+0000 7f1a0ffff640 1 -- 172.21.15.40:0/1242379902 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a1007aad0 con 0x7f1a18160d00 2024-04-26T21:41:05.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.503+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1a180b2120 con 0x7f1a18160d00 2024-04-26T21:41:05.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.863+0000 7f1a0ffff640 1 -- 172.21.15.40:0/1242379902 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f1a1007a3d0 con 0x7f1a18160d00 2024-04-26T21:41:05.865 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:41:05.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19f806c8f0 msgr2=0x7f19f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19f806c8f0 0x7f19f806edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000f60 tx=0x7f1a00009040 comp rx=0 tx=0).stop 2024-04-26T21:41:05.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a18160d00 msgr2=0x7f1a180b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a18160d00 0x7f1a180b6010 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009eb0 tx=0x7f1a10004a40 comp rx=0 tx=0).stop 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 shutdown_connections 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19f806c8f0 0x7f19f806edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a18160d00 0x7f1a180b6010 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a1815d100 0x7f1a180b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 --2- 172.21.15.40:0/1242379902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a180b29d0 0x7f1a180a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 >> 172.21.15.40:0/1242379902 conn(0x7f1a1809ba50 msgr2=0x7f1a1815ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 shutdown_connections 2024-04-26T21:41:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.866+0000 7f1a1da47640 1 -- 172.21.15.40:0/1242379902 wait complete. 2024-04-26T21:41:05.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:41:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.991+0000 7f80b5971640 1 Processor -- start 2024-04-26T21:41:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.991+0000 7f80b5971640 1 -- start start 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.992+0000 7f80b5971640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f80b00d1240 0x7f80b00d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.992+0000 7f80b5971640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f80b00b4360 0x7f80b00cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.992+0000 7f80b5971640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b00d1b60 con 0x7f80b00d0d60 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.992+0000 7f80b5971640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f80b00cf210 con 0x7f80b00d1240 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.992+0000 7f80b5971640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f80b00ce5e0 con 0x7f80b00b4360 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.992+0000 7f80aeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f80b00d1240 0x7f80b00d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.992+0000 7f80aeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f80b00d1240 0x7f80b00d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47924/0 (socket says 172.21.15.40:47924) 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.992+0000 7f80aeffd640 1 -- 172.21.15.40:0/3100631771 learned_addr learned my addr 172.21.15.40:0/3100631771 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.993+0000 7f80ae7fc640 1 --2- 172.21.15.40:0/3100631771 >> v2:172.21.15.184:3300/0 conn(0x7f80b00b4360 0x7f80b00cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.993+0000 7f80aeffd640 1 -- 172.21.15.40:0/3100631771 >> v2:172.21.15.184:3300/0 conn(0x7f80b00b4360 msgr2=0x7f80b00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:05.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.993+0000 7f80aeffd640 1 --2- 172.21.15.40:0/3100631771 >> v2:172.21.15.184:3300/0 conn(0x7f80b00b4360 0x7f80b00cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.993+0000 7f80aeffd640 1 -- 172.21.15.40:0/3100631771 >> v1:172.21.15.184:6789/0 conn(0x7f80b00d0d60 legacy=0x7f80b00cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:05.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.993+0000 7f80aeffd640 1 -- 172.21.15.40:0/3100631771 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b00ce8c0 con 0x7f80b00d1240 2024-04-26T21:41:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.993+0000 7f80aeffd640 1 --2- 172.21.15.40:0/3100631771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00d1240 0x7f80b00d1620 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f80b00b8f10 tx=0x7f80a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7458b0b304106c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80adffb640 1 -- 172.21.15.40:0/3100631771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0037070 con 0x7f80b00d1240 2024-04-26T21:41:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80adffb640 1 -- 172.21.15.40:0/3100631771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a0032440 con 0x7f80b00d1240 2024-04-26T21:41:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80adffb640 1 -- 172.21.15.40:0/3100631771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0032810 con 0x7f80b00d1240 2024-04-26T21:41:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80b5971640 1 -- 172.21.15.40:0/3100631771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00d1240 msgr2=0x7f80b00d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80b5971640 1 --2- 172.21.15.40:0/3100631771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00d1240 0x7f80b00d1620 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f80b00b8f10 tx=0x7f80a002a270 comp rx=0 tx=0).stop 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80b5971640 1 -- 172.21.15.40:0/3100631771 shutdown_connections 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80b5971640 1 --2- 172.21.15.40:0/3100631771 >> v2:172.21.15.184:3300/0 conn(0x7f80b00b4360 0x7f80b00cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80b5971640 1 --2- 172.21.15.40:0/3100631771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00d1240 0x7f80b00d1620 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80b5971640 1 -- 172.21.15.40:0/3100631771 >> 172.21.15.40:0/3100631771 conn(0x7f80b00d2e60 msgr2=0x7f80b00d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80b5971640 1 -- 172.21.15.40:0/3100631771 shutdown_connections 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.994+0000 7f80b5971640 1 -- 172.21.15.40:0/3100631771 wait complete. 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.995+0000 7f80b5971640 1 Processor -- start 2024-04-26T21:41:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.995+0000 7f80b5971640 1 -- start start 2024-04-26T21:41:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80b5971640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b00d0d60 0x7f80b01f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80b5971640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b00d1240 0x7f80b006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80b5971640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00cb250 0x7f80b0153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80b5971640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f80b00c6610 con 0x7f80b00cb250 2024-04-26T21:41:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80b5971640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80b00cf460 con 0x7f80b00d1240 2024-04-26T21:41:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80b5971640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80b00c6790 con 0x7f80b00d0d60 2024-04-26T21:41:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00cb250 0x7f80b0153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00cb250 0x7f80b0153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47926/0 (socket says 172.21.15.40:47926) 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80af7fe640 1 -- 172.21.15.40:0/3730185024 learned_addr learned my addr 172.21.15.40:0/3730185024 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80aeffd640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b00d0d60 0x7f80b01f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.996+0000 7f80ae7fc640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b00d1240 0x7f80b006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f80af7fe640 1 -- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b00d0d60 msgr2=0x7f80b01f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f80af7fe640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b00d0d60 0x7f80b01f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f80af7fe640 1 -- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b00d1240 msgr2=0x7f80b006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f80af7fe640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b00d1240 0x7f80b006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f80af7fe640 1 -- 172.21.15.40:0/3730185024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b006d890 con 0x7f80b00cb250 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f80af7fe640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00cb250 0x7f80b0153300 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c7a0 tx=0x7f80a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f8097fff640 1 -- 172.21.15.40:0/3730185024 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8003b70 con 0x7f80b00cb250 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f8097fff640 1 -- 172.21.15.40:0/3730185024 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a8003d30 con 0x7f80b00cb250 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b006db30 con 0x7f80b00cb250 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f8097fff640 1 -- 172.21.15.40:0/3730185024 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a800aa90 con 0x7f80b00cb250 2024-04-26T21:41:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.997+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b006e0f0 con 0x7f80b00cb250 2024-04-26T21:41:06.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.998+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b00694f0 con 0x7f80b00cb250 2024-04-26T21:41:06.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.999+0000 7f8097fff640 1 -- 172.21.15.40:0/3730185024 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80a800ac30 con 0x7f80b00cb250 2024-04-26T21:41:06.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.999+0000 7f8097fff640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f809c06c8f0 0x7f809c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:05.999+0000 7f8097fff640 1 -- 172.21.15.40:0/3730185024 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5619+0+0 (secure 0 0 0) 0x7f80a808b9d0 con 0x7f80b00cb250 2024-04-26T21:41:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.000+0000 7f80aeffd640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f809c06c8f0 0x7f809c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.000+0000 7f80aeffd640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f809c06c8f0 0x7f809c06edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f80a0034000 tx=0x7f80a002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.004+0000 7f8097fff640 1 -- 172.21.15.40:0/3730185024 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80a8059ee0 con 0x7f80b00cb250 2024-04-26T21:41:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.193+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f80b0070840 con 0x7f80b00cb250 2024-04-26T21:41:06.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.869+0000 7f8097fff640 1 -- 172.21.15.40:0/3730185024 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f80a80597e0 con 0x7f80b00cb250 2024-04-26T21:41:06.870 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:41:06.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f809c06c8f0 msgr2=0x7f809c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f809c06c8f0 0x7f809c06edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f80a0034000 tx=0x7f80a002e000 comp rx=0 tx=0).stop 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00cb250 msgr2=0x7f80b0153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00cb250 0x7f80b0153300 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c7a0 tx=0x7f80a800cc60 comp rx=0 tx=0).stop 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 shutdown_connections 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f809c06c8f0 0x7f809c06edb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b00cb250 0x7f80b0153300 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b00d1240 0x7f80b006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 --2- 172.21.15.40:0/3730185024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b00d0d60 0x7f80b01f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.873+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 >> 172.21.15.40:0/3730185024 conn(0x7f80b00d2e60 msgr2=0x7f80b015e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.874+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 shutdown_connections 2024-04-26T21:41:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:06.874+0000 7f80b5971640 1 -- 172.21.15.40:0/3730185024 wait complete. 2024-04-26T21:41:06.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:41:07.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.011+0000 7f36ee3b6640 1 Processor -- start 2024-04-26T21:41:07.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.011+0000 7f36ee3b6640 1 -- start start 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.011+0000 7f36ee3b6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36e80d1a00 0x7f36e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.011+0000 7f36ee3b6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e80d0a10 con 0x7f36e80d2320 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.011+0000 7f36ee3b6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e80d0cf0 con 0x7f36e80b40d0 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.011+0000 7f36ee3b6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f36e80d0fd0 con 0x7f36e80d1a00 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.012+0000 7f36e7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36e80d1a00 0x7f36e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.012+0000 7f36e77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f36e80d2320 0x7f36e81625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58724/0 (socket says 172.21.15.40:58724) 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.012+0000 7f36e77fe640 1 -- 172.21.15.40:0/2750985723 learned_addr learned my addr 172.21.15.40:0/2750985723 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.012+0000 7f36e7fff640 1 -- 172.21.15.40:0/2750985723 >> v1:172.21.15.184:6789/0 conn(0x7f36e80b40d0 legacy=0x7f36e80b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.012+0000 7f36e7fff640 1 -- 172.21.15.40:0/2750985723 >> v1:172.21.15.40:6789/0 conn(0x7f36e80d2320 legacy=0x7f36e81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.012+0000 7f36e7fff640 1 -- 172.21.15.40:0/2750985723 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e80d40a0 con 0x7f36e80d1a00 2024-04-26T21:41:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.012+0000 7f36e6ffd640 1 -- 172.21.15.40:0/2750985723 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701096740 0 0) 0x7f36e80d0a10 con 0x7f36e80d2320 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36e7fff640 1 --2- 172.21.15.40:0/2750985723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80d1a00 0x7f36e80d1de0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f36d80097a0 tx=0x7f36d802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad65f1db4a49853 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36e6ffd640 1 -- 172.21.15.40:0/2750985723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8037070 con 0x7f36e80d1a00 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36e6ffd640 1 -- 172.21.15.40:0/2750985723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d8032440 con 0x7f36e80d1a00 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36e6ffd640 1 -- 172.21.15.40:0/2750985723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d802ae60 con 0x7f36e80d1a00 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2750985723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80d1a00 msgr2=0x7f36e80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36ee3b6640 1 --2- 172.21.15.40:0/2750985723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80d1a00 0x7f36e80d1de0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f36d80097a0 tx=0x7f36d802a110 comp rx=0 tx=0).stop 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2750985723 shutdown_connections 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36ee3b6640 1 --2- 172.21.15.40:0/2750985723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80d1a00 0x7f36e80d1de0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2750985723 >> 172.21.15.40:0/2750985723 conn(0x7f36e80d4fd0 msgr2=0x7f36e80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2750985723 shutdown_connections 2024-04-26T21:41:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.013+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2750985723 wait complete. 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.015+0000 7f36ee3b6640 1 Processor -- start 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.015+0000 7f36ee3b6640 1 -- start start 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.015+0000 7f36ee3b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80b40d0 0x7f36e814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.015+0000 7f36ee3b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e80d1a00 0x7f36e81514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.015+0000 7f36ee3b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36e80d2320 0x7f36e8151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.015+0000 7f36ee3b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36e80b5210 con 0x7f36e80b40d0 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.015+0000 7f36ee3b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36e8162f50 con 0x7f36e80d1a00 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.015+0000 7f36ee3b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36e80b5390 con 0x7f36e80d2320 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80b40d0 0x7f36e814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80b40d0 0x7f36e814eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47934/0 (socket says 172.21.15.40:47934) 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 -- 172.21.15.40:0/2678495367 learned_addr learned my addr 172.21.15.40:0/2678495367 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e77fe640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e80d1a00 0x7f36e81514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36ec953640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36e80d2320 0x7f36e8151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 -- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36e80d2320 msgr2=0x7f36e8151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36e80d2320 0x7f36e8151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 -- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e80d1a00 msgr2=0x7f36e81514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e80d1a00 0x7f36e81514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 -- 172.21.15.40:0/2678495367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e80ba1d0 con 0x7f36e80b40d0 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e7fff640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80b40d0 0x7f36e814eb20 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f36d80098c0 tx=0x7f36d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e4ff9640 1 -- 172.21.15.40:0/2678495367 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8037040 con 0x7f36e80b40d0 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e4ff9640 1 -- 172.21.15.40:0/2678495367 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d8042070 con 0x7f36e80b40d0 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e80cbe60 con 0x7f36e80b40d0 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36e4ff9640 1 -- 172.21.15.40:0/2678495367 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8030c70 con 0x7f36e80b40d0 2024-04-26T21:41:07.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.016+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e80cb5c0 con 0x7f36e80b40d0 2024-04-26T21:41:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.017+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e80694f0 con 0x7f36e80b40d0 2024-04-26T21:41:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.018+0000 7f36e4ff9640 1 -- 172.21.15.40:0/2678495367 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36d8047460 con 0x7f36e80b40d0 2024-04-26T21:41:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.018+0000 7f36e4ff9640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36d406c920 0x7f36d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.018+0000 7f36e4ff9640 1 -- 172.21.15.40:0/2678495367 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6001+0+0 (secure 0 0 0) 0x7f36d80ac990 con 0x7f36e80b40d0 2024-04-26T21:41:07.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.019+0000 7f36e77fe640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36d406c920 0x7f36d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:07.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.019+0000 7f36e77fe640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36d406c920 0x7f36d406ede0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f36e8155a80 tx=0x7f36dc007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.023+0000 7f36e4ff9640 1 -- 172.21.15.40:0/2678495367 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36d807abf0 con 0x7f36e80b40d0 2024-04-26T21:41:07.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.247+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f36e80bae30 con 0x7f36e80b40d0 2024-04-26T21:41:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.871+0000 7f36e4ff9640 1 -- 172.21.15.40:0/2678495367 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v67) ==== 174+0+0 (secure 0 0 0) 0x7f36d807a4f0 con 0x7f36e80b40d0 2024-04-26T21:41:07.874 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 11 and data pool 12 2024-04-26T21:41:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36d406c920 msgr2=0x7f36d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36d406c920 0x7f36d406ede0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f36e8155a80 tx=0x7f36dc007d00 comp rx=0 tx=0).stop 2024-04-26T21:41:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80b40d0 msgr2=0x7f36e814eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80b40d0 0x7f36e814eb20 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f36d80098c0 tx=0x7f36d80046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 shutdown_connections 2024-04-26T21:41:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36d406c920 0x7f36d406ede0 secure :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f36e8155a80 tx=0x7f36dc007d00 comp rx=0 tx=0).stop 2024-04-26T21:41:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36e80d2320 0x7f36e8151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:07.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36e80d1a00 0x7f36e81514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 --2- 172.21.15.40:0/2678495367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36e80b40d0 0x7f36e814eb20 secure :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f36d80098c0 tx=0x7f36d80046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:07.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.878+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 >> 172.21.15.40:0/2678495367 conn(0x7f36e80d4fd0 msgr2=0x7f36e80d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:07.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.881+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 shutdown_connections 2024-04-26T21:41:07.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:07.884+0000 7f36ee3b6640 1 -- 172.21.15.40:0/2678495367 wait complete. 2024-04-26T21:41:07.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.015+0000 7f738b6c4640 1 Processor -- start 2024-04-26T21:41:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.015+0000 7f738b6c4640 1 -- start start 2024-04-26T21:41:08.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f738b6c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73840b2340 0x7f73840cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f738b6c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840cd930 0x7f73840d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f738b6c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73840d10d0 con 0x7f7384162810 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f738b6c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f73840d0750 con 0x7f73840b2340 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f738b6c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73840d16a0 con 0x7f73840cd930 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7388c5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73840b2340 0x7f73840cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7389460640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7384162810 0x7f7384070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58726/0 (socket says 172.21.15.40:58726) 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7389460640 1 -- 172.21.15.40:0/1199511229 learned_addr learned my addr 172.21.15.40:0/1199511229 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7389c61640 1 --2- 172.21.15.40:0/1199511229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840cd930 0x7f73840d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7388c5f640 1 -- 172.21.15.40:0/1199511229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840cd930 msgr2=0x7f73840d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7388c5f640 1 --2- 172.21.15.40:0/1199511229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840cd930 0x7f73840d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7388c5f640 1 -- 172.21.15.40:0/1199511229 >> v1:172.21.15.40:6789/0 conn(0x7f7384162810 legacy=0x7f7384070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7388c5f640 1 -- 172.21.15.40:0/1199511229 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73840d1370 con 0x7f73840b2340 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.016+0000 7f7389c61640 1 --2- 172.21.15.40:0/1199511229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840cd930 0x7f73840d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.017+0000 7f7388c5f640 1 --2- 172.21.15.40:0/1199511229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73840b2340 0x7f73840cd3f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f73840d1820 tx=0x7f737402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6329664c595223ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.017+0000 7f737bfff640 1 -- 172.21.15.40:0/1199511229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374037070 con 0x7f73840b2340 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.017+0000 7f737bfff640 1 -- 172.21.15.40:0/1199511229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7374032440 con 0x7f73840b2340 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.017+0000 7f737bfff640 1 -- 172.21.15.40:0/1199511229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374032810 con 0x7f73840b2340 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.018+0000 7f738b6c4640 1 -- 172.21.15.40:0/1199511229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73840b2340 msgr2=0x7f73840cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.018+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1199511229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73840b2340 0x7f73840cd3f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f73840d1820 tx=0x7f737402a180 comp rx=0 tx=0).stop 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.018+0000 7f738b6c4640 1 -- 172.21.15.40:0/1199511229 shutdown_connections 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.018+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1199511229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840cd930 0x7f73840d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.018+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1199511229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73840b2340 0x7f73840cd3f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.018+0000 7f738b6c4640 1 -- 172.21.15.40:0/1199511229 >> 172.21.15.40:0/1199511229 conn(0x7f73840d2da0 msgr2=0x7f73840d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.018+0000 7f738b6c4640 1 -- 172.21.15.40:0/1199511229 shutdown_connections 2024-04-26T21:41:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.018+0000 7f738b6c4640 1 -- 172.21.15.40:0/1199511229 wait complete. 2024-04-26T21:41:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.019+0000 7f738b6c4640 1 Processor -- start 2024-04-26T21:41:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.019+0000 7f738b6c4640 1 -- start start 2024-04-26T21:41:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f738b6c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840b2340 0x7f73840c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f738b6c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73840cd930 0x7f738415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f738b6c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7384162810 0x7f73841540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f738b6c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73840d2070 con 0x7f7384162810 2024-04-26T21:41:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f738b6c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73840d0470 con 0x7f73840cd930 2024-04-26T21:41:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f738b6c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73840d21f0 con 0x7f73840b2340 2024-04-26T21:41:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7384162810 0x7f73841540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7384162810 0x7f73841540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47946/0 (socket says 172.21.15.40:47946) 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 -- 172.21.15.40:0/1953136301 learned_addr learned my addr 172.21.15.40:0/1953136301 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389460640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840b2340 0x7f73840c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7388c5f640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73840cd930 0x7f738415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 -- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840b2340 msgr2=0x7f73840c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840b2340 0x7f73840c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 -- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73840cd930 msgr2=0x7f738415f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73840cd930 0x7f738415f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 -- 172.21.15.40:0/1953136301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738415fb60 con 0x7f7384162810 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7389c61640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7384162810 0x7f73841540b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f737c00ce80 tx=0x7f737c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7379ffb640 1 -- 172.21.15.40:0/1953136301 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c01a070 con 0x7f7384162810 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7379ffb640 1 -- 172.21.15.40:0/1953136301 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737c00d880 con 0x7f7384162810 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73840ca450 con 0x7f7384162810 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.020+0000 7f7379ffb640 1 -- 172.21.15.40:0/1953136301 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c00dc70 con 0x7f7384162810 2024-04-26T21:41:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.021+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73840caae0 con 0x7f7384162810 2024-04-26T21:41:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.021+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73840694f0 con 0x7f7384162810 2024-04-26T21:41:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.022+0000 7f7379ffb640 1 -- 172.21.15.40:0/1953136301 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f737c01d070 con 0x7f7384162810 2024-04-26T21:41:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.022+0000 7f7379ffb640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f736806c8f0 0x7f736806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.022+0000 7f7379ffb640 1 -- 172.21.15.40:0/1953136301 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f737c08c380 con 0x7f7384162810 2024-04-26T21:41:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.023+0000 7f7389460640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f736806c8f0 0x7f736806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.023+0000 7f7389460640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f736806c8f0 0x7f736806edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7380000f60 tx=0x7f7380009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.027+0000 7f7379ffb640 1 -- 172.21.15.40:0/1953136301 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f738415fb60 con 0x7f7384162810 2024-04-26T21:41:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.217+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73840b1d80 con 0x7f7384162810 2024-04-26T21:41:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.218+0000 7f7379ffb640 1 -- 172.21.15.40:0/1953136301 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16922 (secure 0 0 0) 0x7f737c05a060 con 0x7f7384162810 2024-04-26T21:41:08.219 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:08.219 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":80,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:07.249340+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:41:05.508768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T21:41:06.198815+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f736806c8f0 msgr2=0x7f736806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f736806c8f0 0x7f736806edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7380000f60 tx=0x7f7380009040 comp rx=0 tx=0).stop 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7384162810 msgr2=0x7f73841540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7384162810 0x7f73841540b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f737c00ce80 tx=0x7f737c00a710 comp rx=0 tx=0).stop 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 shutdown_connections 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f736806c8f0 0x7f736806edb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7384162810 0x7f73841540b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73840cd930 0x7f738415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 --2- 172.21.15.40:0/1953136301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73840b2340 0x7f73840c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 >> 172.21.15.40:0/1953136301 conn(0x7f73840d2da0 msgr2=0x7f7384070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 shutdown_connections 2024-04-26T21:41:08.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.220+0000 7f738b6c4640 1 -- 172.21.15.40:0/1953136301 wait complete. 2024-04-26T21:41:08.232 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:08.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.332+0000 7fe4c9864640 1 Processor -- start 2024-04-26T21:41:08.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.332+0000 7fe4c9864640 1 -- start start 2024-04-26T21:41:08.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c9864640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe4c40b5f20 0x7fe4c40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c9864640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c40b7e50 con 0x7fe4c40d3bc0 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c9864640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c40d1800 con 0x7fe4c4162880 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c9864640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c40d0580 con 0x7fe4c40b5f20 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c27fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe4c40d3bc0 0x7fe4c40b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58740/0 (socket says 172.21.15.40:58740) 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c27fc640 1 -- 172.21.15.40:0/617983669 learned_addr learned my addr 172.21.15.40:0/617983669 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c37fe640 1 --2- 172.21.15.40:0/617983669 >> v2:172.21.15.184:3300/0 conn(0x7fe4c40b5f20 0x7fe4c40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176086398 0 0) 0x7fe4c40b7e50 con 0x7fe4c40d3bc0 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4ac003520 con 0x7fe4c40d3bc0 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 588251238 0 0) 0x7fe4ac003520 con 0x7fe4c40d3bc0 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 >> v2:172.21.15.184:3300/0 conn(0x7fe4c40b5f20 msgr2=0x7fe4c40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 --2- 172.21.15.40:0/617983669 >> v2:172.21.15.184:3300/0 conn(0x7fe4c40b5f20 0x7fe4c40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 >> v1:172.21.15.184:6789/0 conn(0x7fe4c4162880 legacy=0x7fe4c40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c40d07d0 con 0x7fe4c40d3bc0 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fe4b8001670 con 0x7fe4c40d3bc0 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.333+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 >> v1:172.21.15.40:6789/0 conn(0x7fe4c40d3bc0 legacy=0x7fe4c40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c1ffb640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ac0039b0 0x7fe4ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c1ffb640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c1ffb640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ac0051c0 0x7fe4ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c40d0580 con 0x7fe4ac004800 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac00c060 con 0x7fe4ac0051c0 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe4ac00c260 con 0x7fe4ac0039b0 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c27fc640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c37fe640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ac0051c0 0x7fe4ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c2ffd640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ac0039b0 0x7fe4ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c27fc640 1 -- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ac0039b0 msgr2=0x7fe4ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c27fc640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ac0039b0 0x7fe4ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c27fc640 1 -- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ac0051c0 msgr2=0x7fe4ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c27fc640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ac0051c0 0x7fe4ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c27fc640 1 -- 172.21.15.40:0/617983669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac003520 con 0x7fe4ac004800 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c37fe640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ac0051c0 0x7fe4ac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.334+0000 7fe4c37fe640 1 -- 172.21.15.40:0/617983669 reap_dead start 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c27fc640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c240 tx=0x7fe4ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681ec56d57ed8b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8007d10 con 0x7fe4ac004800 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b80058d0 con 0x7fe4ac004800 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c1ffb640 1 -- 172.21.15.40:0/617983669 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8005d40 con 0x7fe4ac004800 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c9864640 1 -- 172.21.15.40:0/617983669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 msgr2=0x7fe4ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c9864640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c240 tx=0x7fe4ac005830 comp rx=0 tx=0).stop 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c9864640 1 -- 172.21.15.40:0/617983669 shutdown_connections 2024-04-26T21:41:08.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c9864640 1 --2- 172.21.15.40:0/617983669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c9864640 1 -- 172.21.15.40:0/617983669 >> 172.21.15.40:0/617983669 conn(0x7fe4c40d4f60 msgr2=0x7fe4c40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c9864640 1 -- 172.21.15.40:0/617983669 shutdown_connections 2024-04-26T21:41:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.335+0000 7fe4c9864640 1 -- 172.21.15.40:0/617983669 wait complete. 2024-04-26T21:41:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.336+0000 7fe4c9864640 1 Processor -- start 2024-04-26T21:41:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c9864640 1 -- start start 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c9864640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4c41f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c9864640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4c40b1b50 0x7fe4c40ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c9864640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4c40b5220 0x7fe4c4152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c9864640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4162c60 con 0x7fe4ac004800 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c9864640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c40d0d90 con 0x7fe4c40b1b50 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c9864640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c4162de0 con 0x7fe4c40b5220 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4c41f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4c41f55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47968/0 (socket says 172.21.15.40:47968) 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.337+0000 7fe4c2ffd640 1 -- 172.21.15.40:0/3276127765 learned_addr learned my addr 172.21.15.40:0/3276127765 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c37fe640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4c40b5220 0x7fe4c4152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c27fc640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4c40b1b50 0x7fe4c40ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c2ffd640 1 -- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4c40b5220 msgr2=0x7fe4c4152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c2ffd640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4c40b5220 0x7fe4c4152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c2ffd640 1 -- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4c40b1b50 msgr2=0x7fe4c40ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c2ffd640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4c40b1b50 0x7fe4c40ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c2ffd640 1 -- 172.21.15.40:0/3276127765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c40b5ac0 con 0x7fe4ac004800 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c27fc640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4c40b1b50 0x7fe4c40ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c2ffd640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4c41f55a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e4d0 tx=0x7fe4b400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4abfff640 1 -- 172.21.15.40:0/3276127765 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40053a0 con 0x7fe4ac004800 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4abfff640 1 -- 172.21.15.40:0/3276127765 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b4018070 con 0x7fe4ac004800 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c415dc30 con 0x7fe4ac004800 2024-04-26T21:41:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4abfff640 1 -- 172.21.15.40:0/3276127765 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4003810 con 0x7fe4ac004800 2024-04-26T21:41:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.338+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c40cc0b0 con 0x7fe4ac004800 2024-04-26T21:41:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.339+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c40694f0 con 0x7fe4ac004800 2024-04-26T21:41:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.340+0000 7fe4abfff640 1 -- 172.21.15.40:0/3276127765 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4b40039b0 con 0x7fe4ac004800 2024-04-26T21:41:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.340+0000 7fe4abfff640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4b006c920 0x7fe4b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.340+0000 7fe4abfff640 1 -- 172.21.15.40:0/3276127765 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fe4b408b970 con 0x7fe4ac004800 2024-04-26T21:41:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.340+0000 7fe4c27fc640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4b006c920 0x7fe4b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.341+0000 7fe4c27fc640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4b006c920 0x7fe4b006ede0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8033000 tx=0x7fe4b802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.345+0000 7fe4abfff640 1 -- 172.21.15.40:0/3276127765 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4c40d1800 con 0x7fe4ac004800 2024-04-26T21:41:08.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.532+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4c40d3830 con 0x7fe4ac004800 2024-04-26T21:41:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.533+0000 7fe4abfff640 1 -- 172.21.15.40:0/3276127765 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16922 (secure 0 0 0) 0x7fe4b4059ca0 con 0x7fe4ac004800 2024-04-26T21:41:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:08.535 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":80,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:07.249340+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:41:05.508768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T21:41:06.198815+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4b006c920 msgr2=0x7fe4b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4b006c920 0x7fe4b006ede0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8033000 tx=0x7fe4b802e000 comp rx=0 tx=0).stop 2024-04-26T21:41:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 msgr2=0x7fe4c41f55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4c41f55a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e4d0 tx=0x7fe4b400e990 comp rx=0 tx=0).stop 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 shutdown_connections 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe4b006c920 0x7fe4b006ede0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4c40b5220 0x7fe4c4152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4c40b1b50 0x7fe4c40ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 --2- 172.21.15.40:0/3276127765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4ac004800 0x7fe4c41f55a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 >> 172.21.15.40:0/3276127765 conn(0x7fe4c40d4f60 msgr2=0x7fe4c406e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 shutdown_connections 2024-04-26T21:41:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.535+0000 7fe4c9864640 1 -- 172.21.15.40:0/3276127765 wait complete. 2024-04-26T21:41:08.548 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:41:08.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.650+0000 7fe7626dc640 1 Processor -- start 2024-04-26T21:41:08.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.650+0000 7fe7626dc640 1 -- start start 2024-04-26T21:41:08.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe7626dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 0x7fe75c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe7626dc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75c0d1040 con 0x7fe75c15f550 2024-04-26T21:41:08.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe7626dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75c0c9680 con 0x7fe75c15f170 2024-04-26T21:41:08.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe7626dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe75c0c9430 con 0x7fe75c0d3bc0 2024-04-26T21:41:08.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe75b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe75c15f550 0x7fe75c0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46590/0 (socket says 172.21.15.40:46590) 2024-04-26T21:41:08.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe75b7fe640 1 -- 172.21.15.40:0/2200229709 learned_addr learned my addr 172.21.15.40:0/2200229709 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:08.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4254686186 0 0) 0x7fe74c001110 con 0x7fe75c15f550 2024-04-26T21:41:08.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe760c79640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 0x7fe75c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.651+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe748003520 con 0x7fe75c15f550 2024-04-26T21:41:08.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3103129568 0 0) 0x7fe748003520 con 0x7fe75c15f550 2024-04-26T21:41:08.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 msgr2=0x7fe75c0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:08.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 0x7fe75c0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 >> v1:172.21.15.184:6789/0 conn(0x7fe75c15f170 legacy=0x7fe75c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c0c9960 con 0x7fe75c15f550 2024-04-26T21:41:08.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fe74c001840 con 0x7fe75c15f550 2024-04-26T21:41:08.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 >> v1:172.21.15.40:6789/0 conn(0x7fe75c15f550 legacy=0x7fe75c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7480039e0 0x7fe748003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 0x7fe748004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe748004d40 0x7fe7480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c0c9430 con 0x7fe748004d40 2024-04-26T21:41:08.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe74800baa0 con 0x7fe7480039e0 2024-04-26T21:41:08.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe74800bca0 con 0x7fe75c0d3bc0 2024-04-26T21:41:08.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe760c79640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe748004d40 0x7fe7480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75bfff640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7480039e0 0x7fe748003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe760c79640 1 -- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 msgr2=0x7fe748004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:08.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75b7fe640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 0x7fe748004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe760c79640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 0x7fe748004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe760c79640 1 -- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7480039e0 msgr2=0x7fe748003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:08.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe760c79640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7480039e0 0x7fe748003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe760c79640 1 -- 172.21.15.40:0/2200229709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe748003520 con 0x7fe748004d40 2024-04-26T21:41:08.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75bfff640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7480039e0 0x7fe748003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:08.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.652+0000 7fe75b7fe640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe75c0d3bc0 0x7fe748004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:08.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.653+0000 7fe760c79640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe748004d40 0x7fe7480051c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe754009f90 tx=0x7fe748005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8224b8a7e91d67f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.653+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754009c10 con 0x7fe748004d40 2024-04-26T21:41:08.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.653+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe754036070 con 0x7fe748004d40 2024-04-26T21:41:08.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.653+0000 7fe75affd640 1 -- 172.21.15.40:0/2200229709 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754031850 con 0x7fe748004d40 2024-04-26T21:41:08.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.654+0000 7fe7626dc640 1 -- 172.21.15.40:0/2200229709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe748004d40 msgr2=0x7fe7480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.654+0000 7fe7626dc640 1 --2- 172.21.15.40:0/2200229709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe748004d40 0x7fe7480051c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe754009f90 tx=0x7fe748005830 comp rx=0 tx=0).stop 2024-04-26T21:41:08.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.654+0000 7fe760c79640 1 -- 172.21.15.40:0/2200229709 reap_dead start 2024-04-26T21:41:08.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.654+0000 7fe7626dc640 1 -- 172.21.15.40:0/2200229709 shutdown_connections 2024-04-26T21:41:08.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.654+0000 7fe7626dc640 1 -- 172.21.15.40:0/2200229709 >> 172.21.15.40:0/2200229709 conn(0x7fe75c0d4f60 msgr2=0x7fe75c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:08.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.654+0000 7fe7626dc640 1 -- 172.21.15.40:0/2200229709 shutdown_connections 2024-04-26T21:41:08.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.654+0000 7fe7626dc640 1 -- 172.21.15.40:0/2200229709 wait complete. 2024-04-26T21:41:08.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.655+0000 7fe7626dc640 1 Processor -- start 2024-04-26T21:41:08.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.655+0000 7fe7626dc640 1 -- start start 2024-04-26T21:41:08.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe7626dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe748004d40 0x7fe75c0a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe7626dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c1524b0 0x7fe75c09ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe7626dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe75c152bf0 0x7fe75c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe7626dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c0d5c00 con 0x7fe75c1524b0 2024-04-26T21:41:08.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe7626dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c0d5a80 con 0x7fe75c152bf0 2024-04-26T21:41:08.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe7626dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe75c0d5d80 con 0x7fe748004d40 2024-04-26T21:41:08.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c1524b0 0x7fe75c09ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c1524b0 0x7fe75c09ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45102/0 (socket says 172.21.15.40:45102) 2024-04-26T21:41:08.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 -- 172.21.15.40:0/1895781273 learned_addr learned my addr 172.21.15.40:0/1895781273 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:08.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75bfff640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe748004d40 0x7fe75c0a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe760c79640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe75c152bf0 0x7fe75c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 -- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe748004d40 msgr2=0x7fe75c0a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe748004d40 0x7fe75c0a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 -- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe75c152bf0 msgr2=0x7fe75c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:08.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe75c152bf0 0x7fe75c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:08.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 -- 172.21.15.40:0/1895781273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c09f1a0 con 0x7fe75c1524b0 2024-04-26T21:41:08.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe760c79640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe75c152bf0 0x7fe75c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:08.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe75b7fe640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c1524b0 0x7fe75c09ec60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00d8b0 tx=0x7fe74c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe758ff9640 1 -- 172.21.15.40:0/1895781273 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c019070 con 0x7fe75c1524b0 2024-04-26T21:41:08.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.656+0000 7fe758ff9640 1 -- 172.21.15.40:0/1895781273 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c0074e0 con 0x7fe75c1524b0 2024-04-26T21:41:08.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.657+0000 7fe758ff9640 1 -- 172.21.15.40:0/1895781273 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c004a00 con 0x7fe75c1524b0 2024-04-26T21:41:08.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.657+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c09f440 con 0x7fe75c1524b0 2024-04-26T21:41:08.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.657+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c09fa00 con 0x7fe75c1524b0 2024-04-26T21:41:08.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.657+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c06d850 con 0x7fe75c1524b0 2024-04-26T21:41:08.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.658+0000 7fe758ff9640 1 -- 172.21.15.40:0/1895781273 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe74c01c070 con 0x7fe75c1524b0 2024-04-26T21:41:08.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.658+0000 7fe758ff9640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe73c06c920 0x7fe73c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:08.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.658+0000 7fe758ff9640 1 -- 172.21.15.40:0/1895781273 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fe74c08b730 con 0x7fe75c1524b0 2024-04-26T21:41:08.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.659+0000 7fe75bfff640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe73c06c920 0x7fe73c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:08.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.659+0000 7fe75bfff640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe73c06c920 0x7fe73c06ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0c8de0 tx=0x7fe744009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:08.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.663+0000 7fe758ff9640 1 -- 172.21.15.40:0/1895781273 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe75c06d850 con 0x7fe75c1524b0 2024-04-26T21:41:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:08.886+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe75c0694f0 con 0x7fe75c1524b0 2024-04-26T21:41:09.851 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:41:09.850+0000 7fdecb6bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:41:09.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.893+0000 7fe758ff9640 1 -- 172.21.15.40:0/1895781273 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) ==== 118+0+0 (secure 0 0 0) 0x7fe75c0d1040 con 0x7fe75c1524b0 2024-04-26T21:41:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe73c06c920 msgr2=0x7fe73c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe73c06c920 0x7fe73c06ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0c8de0 tx=0x7fe744009040 comp rx=0 tx=0).stop 2024-04-26T21:41:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c1524b0 msgr2=0x7fe75c09ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c1524b0 0x7fe75c09ec60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00d8b0 tx=0x7fe74c00dd70 comp rx=0 tx=0).stop 2024-04-26T21:41:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 shutdown_connections 2024-04-26T21:41:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe73c06c920 0x7fe73c06ede0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe75c152bf0 0x7fe75c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c1524b0 0x7fe75c09ec60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 --2- 172.21.15.40:0/1895781273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe748004d40 0x7fe75c0a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 >> 172.21.15.40:0/1895781273 conn(0x7fe75c0d4f60 msgr2=0x7fe75c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 shutdown_connections 2024-04-26T21:41:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:09.896+0000 7fe7626dc640 1 -- 172.21.15.40:0/1895781273 wait complete. 2024-04-26T21:41:09.910 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:41:09.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:41:10.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.019+0000 7fefce4da640 1 Processor -- start 2024-04-26T21:41:10.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.019+0000 7fefce4da640 1 -- start start 2024-04-26T21:41:10.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefce4da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fefc815f2d0 0x7fefc81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:10.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefce4da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc8072f70 con 0x7fefc8072700 2024-04-26T21:41:10.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefce4da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc8073250 con 0x7fefc806fea0 2024-04-26T21:41:10.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefce4da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fefc8074010 con 0x7fefc815f2d0 2024-04-26T21:41:10.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefc77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fefc8072700 0x7fefc806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46596/0 (socket says 172.21.15.40:46596) 2024-04-26T21:41:10.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefc7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fefc815f2d0 0x7fefc81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:10.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefc77fe640 1 -- 172.21.15.40:0/3090516438 learned_addr learned my addr 172.21.15.40:0/3090516438 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:10.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefc7fff640 1 -- 172.21.15.40:0/3090516438 >> v1:172.21.15.184:6789/0 conn(0x7fefc806fea0 legacy=0x7fefc80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:10.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefc7fff640 1 -- 172.21.15.40:0/3090516438 >> v1:172.21.15.40:6789/0 conn(0x7fefc8072700 legacy=0x7fefc806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:10.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.020+0000 7fefc7fff640 1 -- 172.21.15.40:0/3090516438 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc8073530 con 0x7fefc815f2d0 2024-04-26T21:41:10.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.021+0000 7fefc7fff640 1 --2- 172.21.15.40:0/3090516438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc815f2d0 0x7fefc81604e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fefb80097a0 tx=0x7fefb802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec9c79787778fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:10.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.021+0000 7fefc6ffd640 1 -- 172.21.15.40:0/3090516438 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8037070 con 0x7fefc815f2d0 2024-04-26T21:41:10.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.021+0000 7fefc6ffd640 1 -- 172.21.15.40:0/3090516438 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb8032440 con 0x7fefc815f2d0 2024-04-26T21:41:10.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.021+0000 7fefc6ffd640 1 -- 172.21.15.40:0/3090516438 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8032810 con 0x7fefc815f2d0 2024-04-26T21:41:10.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.022+0000 7fefce4da640 1 -- 172.21.15.40:0/3090516438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc815f2d0 msgr2=0x7fefc81604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:10.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.022+0000 7fefce4da640 1 --2- 172.21.15.40:0/3090516438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc815f2d0 0x7fefc81604e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fefb80097a0 tx=0x7fefb802a230 comp rx=0 tx=0).stop 2024-04-26T21:41:10.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.022+0000 7fefce4da640 1 -- 172.21.15.40:0/3090516438 shutdown_connections 2024-04-26T21:41:10.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.022+0000 7fefce4da640 1 --2- 172.21.15.40:0/3090516438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc815f2d0 0x7fefc81604e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:10.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.022+0000 7fefce4da640 1 -- 172.21.15.40:0/3090516438 >> 172.21.15.40:0/3090516438 conn(0x7fefc80d0be0 msgr2=0x7fefc80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:10.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.022+0000 7fefce4da640 1 -- 172.21.15.40:0/3090516438 shutdown_connections 2024-04-26T21:41:10.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.022+0000 7fefce4da640 1 -- 172.21.15.40:0/3090516438 wait complete. 2024-04-26T21:41:10.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.023+0000 7fefce4da640 1 Processor -- start 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.023+0000 7fefce4da640 1 -- start start 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefce4da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc806fea0 0x7fefc815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefce4da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc8072700 0x7fefc80c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefce4da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefc815f2d0 0x7fefc80c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefce4da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fefc815f6b0 con 0x7fefc806fea0 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefce4da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefc8073af0 con 0x7fefc8072700 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefce4da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fefc8072b90 con 0x7fefc815f2d0 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc806fea0 0x7fefc815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc806fea0 0x7fefc815dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45112/0 (socket says 172.21.15.40:45112) 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc7fff640 1 -- 172.21.15.40:0/1802155083 learned_addr learned my addr 172.21.15.40:0/1802155083 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc77fe640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc8072700 0x7fefc80c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefcca77640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefc815f2d0 0x7fefc80c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc7fff640 1 -- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefc815f2d0 msgr2=0x7fefc80c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc7fff640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefc815f2d0 0x7fefc80c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc7fff640 1 -- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc8072700 msgr2=0x7fefc80c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc7fff640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc8072700 0x7fefc80c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc7fff640 1 -- 172.21.15.40:0/1802155083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc80c7d50 con 0x7fefc806fea0 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.024+0000 7fefc77fe640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc8072700 0x7fefc80c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.025+0000 7fefc7fff640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc806fea0 0x7fefc815dba0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fefb80098c0 tx=0x7fefb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.025+0000 7fefc4ff9640 1 -- 172.21.15.40:0/1802155083 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8037040 con 0x7fefc806fea0 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.025+0000 7fefc4ff9640 1 -- 172.21.15.40:0/1802155083 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb8030770 con 0x7fefc806fea0 2024-04-26T21:41:10.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.025+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc80c8030 con 0x7fefc806fea0 2024-04-26T21:41:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.025+0000 7fefc4ff9640 1 -- 172.21.15.40:0/1802155083 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8030b40 con 0x7fefc806fea0 2024-04-26T21:41:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.025+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc80c85f0 con 0x7fefc806fea0 2024-04-26T21:41:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.026+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc81600c0 con 0x7fefc806fea0 2024-04-26T21:41:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.026+0000 7fefc4ff9640 1 -- 172.21.15.40:0/1802155083 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefb8044020 con 0x7fefc806fea0 2024-04-26T21:41:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.027+0000 7fefc4ff9640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefa806c8f0 0x7fefa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.027+0000 7fefc4ff9640 1 -- 172.21.15.40:0/1802155083 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fefb80ac310 con 0x7fefc806fea0 2024-04-26T21:41:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.027+0000 7fefc77fe640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefa806c8f0 0x7fefa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.028+0000 7fefc77fe640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefa806c8f0 0x7fefa806edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fefc806f6f0 tx=0x7fefb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:10.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.031+0000 7fefc4ff9640 1 -- 172.21.15.40:0/1802155083 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefc81600c0 con 0x7fefc806fea0 2024-04-26T21:41:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.260+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fefc80694f0 con 0x7fefc806fea0 2024-04-26T21:41:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.900+0000 7fefc4ff9640 1 -- 172.21.15.40:0/1802155083 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) ==== 122+0+0 (secure 0 0 0) 0x7fefb8032e20 con 0x7fefc806fea0 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.902+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefa806c8f0 msgr2=0x7fefa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.902+0000 7fefce4da640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefa806c8f0 0x7fefa806edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fefc806f6f0 tx=0x7fefb0009040 comp rx=0 tx=0).stop 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.902+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc806fea0 msgr2=0x7fefc815dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.902+0000 7fefce4da640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc806fea0 0x7fefc815dba0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fefb80098c0 tx=0x7fefb80046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.903+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 shutdown_connections 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.903+0000 7fefce4da640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefa806c8f0 0x7fefa806edb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.903+0000 7fefce4da640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefc815f2d0 0x7fefc80c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.903+0000 7fefce4da640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc8072700 0x7fefc80c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.903+0000 7fefce4da640 1 --2- 172.21.15.40:0/1802155083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc806fea0 0x7fefc815dba0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.903+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 >> 172.21.15.40:0/1802155083 conn(0x7fefc80d0be0 msgr2=0x7fefc8154030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.903+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 shutdown_connections 2024-04-26T21:41:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:10.903+0000 7fefce4da640 1 -- 172.21.15.40:0/1802155083 wait complete. 2024-04-26T21:41:10.916 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:41:10.916 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.023+0000 7fdc83209640 1 Processor -- start 2024-04-26T21:41:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.023+0000 7fdc83209640 1 -- start start 2024-04-26T21:41:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.023+0000 7fdc83209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c0b2340 0x7fdc7c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.023+0000 7fdc83209640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c0d10d0 con 0x7fdc7c0cd930 2024-04-26T21:41:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.023+0000 7fdc83209640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c0d1370 con 0x7fdc7c162810 2024-04-26T21:41:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.023+0000 7fdc83209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c0d0750 con 0x7fdc7c0b2340 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc817a6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdc7c0cd930 0x7fdc7c0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46604/0 (socket says 172.21.15.40:46604) 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc817a6640 1 -- 172.21.15.40:0/1988602424 learned_addr learned my addr 172.21.15.40:0/1988602424 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7bfff640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c0b2340 0x7fdc7c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556095010 0 0) 0x7fdc7c0d10d0 con 0x7fdc7c0cd930 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc68003520 con 0x7fdc7c0cd930 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769325469 0 0) 0x7fdc7c0d1370 con 0x7fdc7c162810 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc7c0d10d0 con 0x7fdc7c162810 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216645627 0 0) 0x7fdc68003520 con 0x7fdc7c0cd930 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c0b2340 msgr2=0x7fdc7c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c0b2340 0x7fdc7c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 >> v1:172.21.15.184:6789/0 conn(0x7fdc7c162810 legacy=0x7fdc7c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c0d01b0 con 0x7fdc7c0cd930 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fdc6c001670 con 0x7fdc7c0cd930 2024-04-26T21:41:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 >> v1:172.21.15.40:6789/0 conn(0x7fdc7c0cd930 legacy=0x7fdc7c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc68003b70 0x7fdc68003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c0b2340 0x7fdc68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc68004d40 0x7fdc680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c0d0750 con 0x7fdc68004d40 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6800baa0 con 0x7fdc68003b70 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6800bca0 con 0x7fdc7c0b2340 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.024+0000 7fdc817a6640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc68004d40 0x7fdc680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.025+0000 7fdc7bfff640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c0b2340 0x7fdc68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.025+0000 7fdc80fa5640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc68003b70 0x7fdc68003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.025+0000 7fdc817a6640 1 -- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c0b2340 msgr2=0x7fdc68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.025+0000 7fdc817a6640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c0b2340 0x7fdc68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.025+0000 7fdc817a6640 1 -- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc68003b70 msgr2=0x7fdc68003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.025+0000 7fdc817a6640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc68003b70 0x7fdc68003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.025+0000 7fdc817a6640 1 -- 172.21.15.40:0/1988602424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc68003520 con 0x7fdc68004d40 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.025+0000 7fdc817a6640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc68004d40 0x7fdc680051c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c076320 tx=0x7fdc68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e12c22d8aebc4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c007be0 con 0x7fdc68004d40 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c005af0 con 0x7fdc68004d40 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc7b7fe640 1 -- 172.21.15.40:0/1988602424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c007070 con 0x7fdc68004d40 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc83209640 1 -- 172.21.15.40:0/1988602424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc68004d40 msgr2=0x7fdc680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc83209640 1 --2- 172.21.15.40:0/1988602424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc68004d40 0x7fdc680051c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c076320 tx=0x7fdc68005830 comp rx=0 tx=0).stop 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc817a6640 1 -- 172.21.15.40:0/1988602424 reap_dead start 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc83209640 1 -- 172.21.15.40:0/1988602424 shutdown_connections 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc83209640 1 -- 172.21.15.40:0/1988602424 >> 172.21.15.40:0/1988602424 conn(0x7fdc7c0d2da0 msgr2=0x7fdc7c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc83209640 1 -- 172.21.15.40:0/1988602424 shutdown_connections 2024-04-26T21:41:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.026+0000 7fdc83209640 1 -- 172.21.15.40:0/1988602424 wait complete. 2024-04-26T21:41:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.027+0000 7fdc83209640 1 Processor -- start 2024-04-26T21:41:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.028+0000 7fdc83209640 1 -- start start 2024-04-26T21:41:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.028+0000 7fdc83209640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc7c0cb5e0 0x7fdc7c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.028+0000 7fdc83209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c14a010 0x7fdc7c14a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.028+0000 7fdc83209640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc7c15d640 0x7fdc7c15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.028+0000 7fdc83209640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c0d2240 con 0x7fdc7c15d640 2024-04-26T21:41:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.028+0000 7fdc83209640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c0d16a0 con 0x7fdc7c0cb5e0 2024-04-26T21:41:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.028+0000 7fdc83209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c0d23c0 con 0x7fdc7c14a010 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc7c15d640 0x7fdc7c15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc7c15d640 0x7fdc7c15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45134/0 (socket says 172.21.15.40:45134) 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 -- 172.21.15.40:0/3277741808 learned_addr learned my addr 172.21.15.40:0/3277741808 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc80fa5640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc7c0cb5e0 0x7fdc7c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc7bfff640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c14a010 0x7fdc7c14a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 -- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c14a010 msgr2=0x7fdc7c14a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c14a010 0x7fdc7c14a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 -- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc7c0cb5e0 msgr2=0x7fdc7c0c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc7c0cb5e0 0x7fdc7c0c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 -- 172.21.15.40:0/3277741808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c14aab0 con 0x7fdc7c15d640 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc817a6640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc7c15d640 0x7fdc7c15daf0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c000c00 tx=0x7fdc7c0d1610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc797fa640 1 -- 172.21.15.40:0/3277741808 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c037040 con 0x7fdc7c15d640 2024-04-26T21:41:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc797fa640 1 -- 172.21.15.40:0/3277741808 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c041070 con 0x7fdc7c15d640 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c14ad90 con 0x7fdc7c15d640 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc797fa640 1 -- 172.21.15.40:0/3277741808 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c007be0 con 0x7fdc7c15d640 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.029+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c158830 con 0x7fdc7c15d640 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.030+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c14af60 con 0x7fdc7c15d640 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.031+0000 7fdc797fa640 1 -- 172.21.15.40:0/3277741808 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc6c007270 con 0x7fdc7c15d640 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.031+0000 7fdc797fa640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc5c06c8f0 0x7fdc5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.031+0000 7fdc797fa640 1 -- 172.21.15.40:0/3277741808 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fdc6c0ab8a0 con 0x7fdc7c15d640 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.032+0000 7fdc80fa5640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc5c06c8f0 0x7fdc5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.032+0000 7fdc80fa5640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc5c06c8f0 0x7fdc5c06edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdc74000f80 tx=0x7fdc74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.036+0000 7fdc797fa640 1 -- 172.21.15.40:0/3277741808 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc7c14af60 con 0x7fdc7c15d640 2024-04-26T21:41:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.263+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc7c0c68d0 con 0x7fdc7c15d640 2024-04-26T21:41:11.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.263+0000 7fdc797fa640 1 -- 172.21.15.40:0/3277741808 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5186 (secure 0 0 0) 0x7fdc6c079c00 con 0x7fdc7c15d640 2024-04-26T21:41:11.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:11.265 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:10.262121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3115383225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3115383225},{"type":"v1","addr":"172.21.15.40:6835","nonce":3115383225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7230,"qdb_cluster":[7230]},"id":5}]} 2024-04-26T21:41:11.265 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 71 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.265+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc5c06c8f0 msgr2=0x7fdc5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.265+0000 7fdc83209640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc5c06c8f0 0x7fdc5c06edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdc74000f80 tx=0x7fdc74009040 comp rx=0 tx=0).stop 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.265+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc7c15d640 msgr2=0x7fdc7c15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.265+0000 7fdc83209640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc7c15d640 0x7fdc7c15daf0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c000c00 tx=0x7fdc7c0d1610 comp rx=0 tx=0).stop 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.265+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 shutdown_connections 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.265+0000 7fdc83209640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdc5c06c8f0 0x7fdc5c06edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.265+0000 7fdc83209640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc7c15d640 0x7fdc7c15daf0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.266+0000 7fdc83209640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc7c14a010 0x7fdc7c14a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.266+0000 7fdc83209640 1 --2- 172.21.15.40:0/3277741808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc7c0cb5e0 0x7fdc7c0c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.266+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 >> 172.21.15.40:0/3277741808 conn(0x7fdc7c0d2da0 msgr2=0x7fdc7c0cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.266+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 shutdown_connections 2024-04-26T21:41:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.266+0000 7fdc83209640 1 -- 172.21.15.40:0/3277741808 wait complete. 2024-04-26T21:41:11.278 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:11.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.378+0000 7fd70d42a640 1 Processor -- start 2024-04-26T21:41:11.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.378+0000 7fd70d42a640 1 -- start start 2024-04-26T21:41:11.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd70d42a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd7080d3bc0 0x7fd7080d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd70d42a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7080b7e50 con 0x7fd7080b5f20 2024-04-26T21:41:11.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd70d42a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7080d1800 con 0x7fd708162880 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd70d42a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd7080d0580 con 0x7fd7080d3bc0 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd7077fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd7080b5f20 0x7fd7080b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46620/0 (socket says 172.21.15.40:46620) 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd7077fe640 1 -- 172.21.15.40:0/915427039 learned_addr learned my addr 172.21.15.40:0/915427039 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1412195580 0 0) 0x7fd7080b7e50 con 0x7fd7080b5f20 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6f0003520 con 0x7fd7080b5f20 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd7067fc640 1 --2- 172.21.15.40:0/915427039 >> v2:172.21.15.184:3300/0 conn(0x7fd7080d3bc0 0x7fd7080d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2896750136 0 0) 0x7fd6f0003520 con 0x7fd7080b5f20 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 >> v2:172.21.15.184:3300/0 conn(0x7fd7080d3bc0 msgr2=0x7fd7080d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 --2- 172.21.15.40:0/915427039 >> v2:172.21.15.184:3300/0 conn(0x7fd7080d3bc0 0x7fd7080d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 >> v1:172.21.15.184:6789/0 conn(0x7fd708162880 legacy=0x7fd7080d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7080d07d0 con 0x7fd7080b5f20 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fd7000019f0 con 0x7fd7080b5f20 2024-04-26T21:41:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 >> v1:172.21.15.40:6789/0 conn(0x7fd7080b5f20 legacy=0x7fd7080b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6f0004800 0x7fd6f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6f00051c0 0x7fd6f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7080d0580 con 0x7fd6f00039b0 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f000c060 con 0x7fd6f00051c0 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.379+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f000c260 con 0x7fd6f0004800 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd706ffd640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd7067fc640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6f0004800 0x7fd6f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd7077fe640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6f00051c0 0x7fd6f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd706ffd640 1 -- 172.21.15.40:0/915427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6f0004800 msgr2=0x7fd6f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd706ffd640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6f0004800 0x7fd6f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd706ffd640 1 -- 172.21.15.40:0/915427039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6f00051c0 msgr2=0x7fd6f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd706ffd640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6f00051c0 0x7fd6f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd706ffd640 1 -- 172.21.15.40:0/915427039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f0003520 con 0x7fd6f00039b0 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.380+0000 7fd7077fe640 1 -- 172.21.15.40:0/915427039 reap_dead start 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd706ffd640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8001750 tx=0x7fd6f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2686af8f0ec59727 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8001370 con 0x7fd6f00039b0 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f80052d0 con 0x7fd6f00039b0 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd705ffb640 1 -- 172.21.15.40:0/915427039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f802abc0 con 0x7fd6f00039b0 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd70d42a640 1 -- 172.21.15.40:0/915427039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 msgr2=0x7fd6f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd70d42a640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8001750 tx=0x7fd6f0005830 comp rx=0 tx=0).stop 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd70d42a640 1 -- 172.21.15.40:0/915427039 shutdown_connections 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd70d42a640 1 --2- 172.21.15.40:0/915427039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd70d42a640 1 -- 172.21.15.40:0/915427039 >> 172.21.15.40:0/915427039 conn(0x7fd7080d4f60 msgr2=0x7fd7080d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd70d42a640 1 -- 172.21.15.40:0/915427039 shutdown_connections 2024-04-26T21:41:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.381+0000 7fd70d42a640 1 -- 172.21.15.40:0/915427039 wait complete. 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd70d42a640 1 Processor -- start 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd70d42a640 1 -- start start 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd70d42a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd7080b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd70d42a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd70815c700 0x7fd7080b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd70d42a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70806de90 0x7fd70806e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd70d42a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd708162c60 con 0x7fd6f00039b0 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd70d42a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd7080d0d90 con 0x7fd70806de90 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd70d42a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd708162de0 con 0x7fd70815c700 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.383+0000 7fd706ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd7080b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd706ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd7080b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45152/0 (socket says 172.21.15.40:45152) 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd706ffd640 1 -- 172.21.15.40:0/2515059278 learned_addr learned my addr 172.21.15.40:0/2515059278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd7077fe640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70806de90 0x7fd70806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd7067fc640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd70815c700 0x7fd7080b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd706ffd640 1 -- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd70815c700 msgr2=0x7fd7080b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd706ffd640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd70815c700 0x7fd7080b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd706ffd640 1 -- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70806de90 msgr2=0x7fd70806e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd706ffd640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70806de90 0x7fd70806e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd706ffd640 1 -- 172.21.15.40:0/2515059278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7080b29b0 con 0x7fd6f00039b0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd706ffd640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd7080b3a50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000c00 tx=0x7fd6f8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd6ef7fe640 1 -- 172.21.15.40:0/2515059278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8040030 con 0x7fd6f00039b0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd6ef7fe640 1 -- 172.21.15.40:0/2515059278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f802be50 con 0x7fd6f00039b0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7080b2c90 con 0x7fd6f00039b0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd6ef7fe640 1 -- 172.21.15.40:0/2515059278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8001370 con 0x7fd6f00039b0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.384+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7080b3250 con 0x7fd6f00039b0 2024-04-26T21:41:11.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.385+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7080694f0 con 0x7fd6f00039b0 2024-04-26T21:41:11.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.386+0000 7fd6ef7fe640 1 -- 172.21.15.40:0/2515059278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6f8043070 con 0x7fd6f00039b0 2024-04-26T21:41:11.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.386+0000 7fd6ef7fe640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f406c8f0 0x7fd6f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.386+0000 7fd6ef7fe640 1 -- 172.21.15.40:0/2515059278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fd6f80ab510 con 0x7fd6f00039b0 2024-04-26T21:41:11.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.386+0000 7fd7067fc640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f406c8f0 0x7fd6f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.387+0000 7fd7067fc640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f406c8f0 0x7fd6f406edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd7080a2da0 tx=0x7fd6fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.390+0000 7fd6ef7fe640 1 -- 172.21.15.40:0/2515059278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6f804b020 con 0x7fd6f00039b0 2024-04-26T21:41:11.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:41:11.482 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:41:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.577+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7080d36c0 con 0x7fd6f00039b0 2024-04-26T21:41:11.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.578+0000 7fd6ef7fe640 1 -- 172.21.15.40:0/2515059278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16909 (secure 0 0 0) 0x7fd7080d1800 con 0x7fd6f00039b0 2024-04-26T21:41:11.579 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:11.579 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:08.879759+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:41:05.508768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T21:41:06.198815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f406c8f0 msgr2=0x7fd6f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f406c8f0 0x7fd6f406edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd7080a2da0 tx=0x7fd6fc008040 comp rx=0 tx=0).stop 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 msgr2=0x7fd7080b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd7080b3a50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000c00 tx=0x7fd6f8001120 comp rx=0 tx=0).stop 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 shutdown_connections 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f406c8f0 0x7fd6f406edb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70806de90 0x7fd70806e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd70815c700 0x7fd7080b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 --2- 172.21.15.40:0/2515059278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f00039b0 0x7fd7080b3a50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 >> 172.21.15.40:0/2515059278 conn(0x7fd7080d4f60 msgr2=0x7fd7080d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:11.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 shutdown_connections 2024-04-26T21:41:11.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.580+0000 7fd70d42a640 1 -- 172.21.15.40:0/2515059278 wait complete. 2024-04-26T21:41:11.592 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:41:11.592 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:41:11.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.732+0000 7f65a2c83640 1 Processor -- start 2024-04-26T21:41:11.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.732+0000 7f65a2c83640 1 -- start start 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f65a2c83640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f659c162880 0x7f659c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f65a2c83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0d3bc0 0x7f659c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f65a2c83640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f659c0b7e50 con 0x7f659c0b5f20 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f65a2c83640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f659c0d0580 con 0x7f659c162880 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f65a2c83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f659c0d1800 con 0x7f659c0d3bc0 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f659bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f659c0b5f20 0x7f659c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46632/0 (socket says 172.21.15.40:46632) 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f659bfff640 1 -- 172.21.15.40:0/4089198306 learned_addr learned my addr 172.21.15.40:0/4089198306 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f65a0a1f640 1 --2- 172.21.15.40:0/4089198306 >> v2:172.21.15.184:3300/0 conn(0x7f659c162880 0x7f659c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708149205 0 0) 0x7f659c0b7e50 con 0x7f659c0b5f20 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f65a1220640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0d3bc0 0x7f659c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.733+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6584003520 con 0x7f659c0b5f20 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1607011264 0 0) 0x7f6584003520 con 0x7f659c0b5f20 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0d3bc0 msgr2=0x7f659c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0d3bc0 0x7f659c0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 >> v2:172.21.15.184:3300/0 conn(0x7f659c162880 msgr2=0x7f659c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 --2- 172.21.15.40:0/4089198306 >> v2:172.21.15.184:3300/0 conn(0x7f659c162880 0x7f659c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c0d07d0 con 0x7f659c0b5f20 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f6590001670 con 0x7f659c0b5f20 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 >> v1:172.21.15.40:6789/0 conn(0x7f659c0b5f20 legacy=0x7f659c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0d3bc0 0x7f65840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6584003f30 0x7f6584004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6584004d40 0x7f65840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0d0580 con 0x7f6584004d40 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0d1800 con 0x7f6584003f30 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f658400b9c0 con 0x7f659c0d3bc0 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f65a1220640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6584004d40 0x7f65840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f65a0a1f640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0d3bc0 0x7f65840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659bfff640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6584003f30 0x7f6584004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f65a1220640 1 -- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0d3bc0 msgr2=0x7f65840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f65a1220640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0d3bc0 0x7f65840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f65a1220640 1 -- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6584003f30 msgr2=0x7f6584004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f65a1220640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6584003f30 0x7f6584004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f65a1220640 1 -- 172.21.15.40:0/4089198306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6584003520 con 0x7f6584004d40 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.734+0000 7f659bfff640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6584003f30 0x7f6584004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.735+0000 7f65a1220640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6584004d40 0x7f65840051c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f659c0b6930 tx=0x7f6584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441b8a5147bebadf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.735+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037070 con 0x7f6584004d40 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.735+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594032410 con 0x7f6584004d40 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.735+0000 7f659b7fe640 1 -- 172.21.15.40:0/4089198306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594009c10 con 0x7f6584004d40 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.735+0000 7f65a2c83640 1 -- 172.21.15.40:0/4089198306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6584004d40 msgr2=0x7f65840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.735+0000 7f65a2c83640 1 --2- 172.21.15.40:0/4089198306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6584004d40 0x7f65840051c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f659c0b6930 tx=0x7f6584005830 comp rx=0 tx=0).stop 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.735+0000 7f65a1220640 1 -- 172.21.15.40:0/4089198306 reap_dead start 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.736+0000 7f65a2c83640 1 -- 172.21.15.40:0/4089198306 shutdown_connections 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.736+0000 7f65a2c83640 1 -- 172.21.15.40:0/4089198306 >> 172.21.15.40:0/4089198306 conn(0x7f659c0d4f60 msgr2=0x7f659c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:11.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.736+0000 7f65a2c83640 1 -- 172.21.15.40:0/4089198306 shutdown_connections 2024-04-26T21:41:11.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.736+0000 7f65a2c83640 1 -- 172.21.15.40:0/4089198306 wait complete. 2024-04-26T21:41:11.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.737+0000 7f65a2c83640 1 Processor -- start 2024-04-26T21:41:11.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.737+0000 7f65a2c83640 1 -- start start 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a2c83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0cba30 0x7f659c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a2c83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f659c0b2030 0x7f659c0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a2c83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659c0ce5d0 0x7f659c1f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a2c83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f659c162c60 con 0x7f659c0ce5d0 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a2c83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0d0d90 con 0x7f659c0b2030 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a2c83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f659c162de0 con 0x7f659c0cba30 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659c0ce5d0 0x7f659c1f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659c0ce5d0 0x7f659c1f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45174/0 (socket says 172.21.15.40:45174) 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 -- 172.21.15.40:0/2556438072 learned_addr learned my addr 172.21.15.40:0/2556438072 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a0a1f640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0cba30 0x7f659c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f659bfff640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f659c0b2030 0x7f659c0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 -- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0cba30 msgr2=0x7f659c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0cba30 0x7f659c0b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 -- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f659c0b2030 msgr2=0x7f659c0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f659c0b2030 0x7f659c0ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 -- 172.21.15.40:0/2556438072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c0ccff0 con 0x7f659c0ce5d0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a1220640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659c0ce5d0 0x7f659c1f4830 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f659400e230 tx=0x7f6594009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65997fa640 1 -- 172.21.15.40:0/2556438072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594042020 con 0x7f659c0ce5d0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65997fa640 1 -- 172.21.15.40:0/2556438072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594003b90 con 0x7f659c0ce5d0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65997fa640 1 -- 172.21.15.40:0/2556438072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037040 con 0x7f659c0ce5d0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c0cd2d0 con 0x7f659c0ce5d0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.738+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c0cd890 con 0x7f659c0ce5d0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.739+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c0694f0 con 0x7f659c0ce5d0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.740+0000 7f65997fa640 1 -- 172.21.15.40:0/2556438072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6594045070 con 0x7f659c0ce5d0 2024-04-26T21:41:11.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.740+0000 7f65997fa640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c920 0x7f658806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:11.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.740+0000 7f65997fa640 1 -- 172.21.15.40:0/2556438072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f65940abab0 con 0x7f659c0ce5d0 2024-04-26T21:41:11.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.741+0000 7f65a0a1f640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c920 0x7f658806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:11.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.741+0000 7f65a0a1f640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c920 0x7f658806ede0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f659c160910 tx=0x7f658c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:11.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.745+0000 7f65997fa640 1 -- 172.21.15.40:0/2556438072 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6594079790 con 0x7f659c0ce5d0 2024-04-26T21:41:11.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.927+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f659c074590 con 0x7f658806c920 2024-04-26T21:41:11.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.934+0000 7f65997fa640 1 -- 172.21.15.40:0/2556438072 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252002 (secure 0 0 0) 0x7f659c074590 con 0x7f658806c920 2024-04-26T21:41:11.938 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:41:11.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c920 msgr2=0x7f658806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c920 0x7f658806ede0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f659c160910 tx=0x7f658c008040 comp rx=0 tx=0).stop 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659c0ce5d0 msgr2=0x7f659c1f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659c0ce5d0 0x7f659c1f4830 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f659400e230 tx=0x7f6594009750 comp rx=0 tx=0).stop 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 shutdown_connections 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c920 0x7f658806ede0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659c0ce5d0 0x7f659c1f4830 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f659c0b2030 0x7f659c0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 --2- 172.21.15.40:0/2556438072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f659c0cba30 0x7f659c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 >> 172.21.15.40:0/2556438072 conn(0x7f659c0d4f60 msgr2=0x7f659c0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 shutdown_connections 2024-04-26T21:41:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:11.937+0000 7f65a2c83640 1 -- 172.21.15.40:0/2556438072 wait complete. 2024-04-26T21:41:11.949 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-04-26T21:41:10.084272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233224,"kb_used_data":8712,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754741496,"statfs":{"total":773094113280,"available":772855291904,"internally_reserved":0,"allocated":8921088,"data_stored":4747200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.216041"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898862+0000","last_change":"2024-04-26T21:41:07.893940+0000","last_active":"2024-04-26T21:41:07.898862+0000","last_peered":"2024-04-26T21:41:07.898862+0000","last_clean":"2024-04-26T21:41:07.898862+0000","last_became_active":"2024-04-26T21:41:07.893626+0000","last_became_peered":"2024-04-26T21:41:07.893626+0000","last_unstale":"2024-04-26T21:41:07.898862+0000","last_undegraded":"2024-04-26T21:41:07.898862+0000","last_fullsized":"2024-04-26T21:41:07.898862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:26.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912765+0000","last_change":"2024-04-26T21:41:07.903280+0000","last_active":"2024-04-26T21:41:07.912765+0000","last_peered":"2024-04-26T21:41:07.912765+0000","last_clean":"2024-04-26T21:41:07.912765+0000","last_became_active":"2024-04-26T21:41:07.902961+0000","last_became_peered":"2024-04-26T21:41:07.902961+0000","last_unstale":"2024-04-26T21:41:07.912765+0000","last_undegraded":"2024-04-26T21:41:07.912765+0000","last_fullsized":"2024-04-26T21:41:07.912765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:34.997572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906367+0000","last_change":"2024-04-26T21:41:07.895896+0000","last_active":"2024-04-26T21:41:07.906367+0000","last_peered":"2024-04-26T21:41:07.906367+0000","last_clean":"2024-04-26T21:41:07.906367+0000","last_became_active":"2024-04-26T21:41:07.895602+0000","last_became_peered":"2024-04-26T21:41:07.895602+0000","last_unstale":"2024-04-26T21:41:07.906367+0000","last_undegraded":"2024-04-26T21:41:07.906367+0000","last_fullsized":"2024-04-26T21:41:07.906367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.467106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.888111+0000","last_change":"2024-04-26T21:41:07.880310+0000","last_active":"2024-04-26T21:41:07.888111+0000","last_peered":"2024-04-26T21:41:07.888111+0000","last_clean":"2024-04-26T21:41:07.888111+0000","last_became_active":"2024-04-26T21:41:07.879984+0000","last_became_peered":"2024-04-26T21:41:07.879984+0000","last_unstale":"2024-04-26T21:41:07.888111+0000","last_undegraded":"2024-04-26T21:41:07.888111+0000","last_fullsized":"2024-04-26T21:41:07.888111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:48:21.988173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.890430+0000","last_change":"2024-04-26T21:41:07.888564+0000","last_active":"2024-04-26T21:41:07.890430+0000","last_peered":"2024-04-26T21:41:07.890430+0000","last_clean":"2024-04-26T21:41:07.890430+0000","last_became_active":"2024-04-26T21:41:07.888283+0000","last_became_peered":"2024-04-26T21:41:07.888283+0000","last_unstale":"2024-04-26T21:41:07.890430+0000","last_undegraded":"2024-04-26T21:41:07.890430+0000","last_fullsized":"2024-04-26T21:41:07.890430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:22.924079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906127+0000","last_change":"2024-04-26T21:41:07.899431+0000","last_active":"2024-04-26T21:41:07.906127+0000","last_peered":"2024-04-26T21:41:07.906127+0000","last_clean":"2024-04-26T21:41:07.906127+0000","last_became_active":"2024-04-26T21:41:07.898768+0000","last_became_peered":"2024-04-26T21:41:07.898768+0000","last_unstale":"2024-04-26T21:41:07.906127+0000","last_undegraded":"2024-04-26T21:41:07.906127+0000","last_fullsized":"2024-04-26T21:41:07.906127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:40.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904749+0000","last_change":"2024-04-26T21:41:07.897460+0000","last_active":"2024-04-26T21:41:07.904749+0000","last_peered":"2024-04-26T21:41:07.904749+0000","last_clean":"2024-04-26T21:41:07.904749+0000","last_became_active":"2024-04-26T21:41:07.897163+0000","last_became_peered":"2024-04-26T21:41:07.897163+0000","last_unstale":"2024-04-26T21:41:07.904749+0000","last_undegraded":"2024-04-26T21:41:07.904749+0000","last_fullsized":"2024-04-26T21:41:07.904749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:36.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905548+0000","last_change":"2024-04-26T21:41:07.895337+0000","last_active":"2024-04-26T21:41:07.905548+0000","last_peered":"2024-04-26T21:41:07.905548+0000","last_clean":"2024-04-26T21:41:07.905548+0000","last_became_active":"2024-04-26T21:41:07.895067+0000","last_became_peered":"2024-04-26T21:41:07.895067+0000","last_unstale":"2024-04-26T21:41:07.905548+0000","last_undegraded":"2024-04-26T21:41:07.905548+0000","last_fullsized":"2024-04-26T21:41:07.905548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:59.859051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.894114+0000","last_change":"2024-04-26T21:41:07.889838+0000","last_active":"2024-04-26T21:41:07.894114+0000","last_peered":"2024-04-26T21:41:07.894114+0000","last_clean":"2024-04-26T21:41:07.894114+0000","last_became_active":"2024-04-26T21:41:07.889513+0000","last_became_peered":"2024-04-26T21:41:07.889513+0000","last_unstale":"2024-04-26T21:41:07.894114+0000","last_undegraded":"2024-04-26T21:41:07.894114+0000","last_fullsized":"2024-04-26T21:41:07.894114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:32.940233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.890580+0000","last_change":"2024-04-26T21:41:07.888360+0000","last_active":"2024-04-26T21:41:07.890580+0000","last_peered":"2024-04-26T21:41:07.890580+0000","last_clean":"2024-04-26T21:41:07.890580+0000","last_became_active":"2024-04-26T21:41:07.888064+0000","last_became_peered":"2024-04-26T21:41:07.888064+0000","last_unstale":"2024-04-26T21:41:07.890580+0000","last_undegraded":"2024-04-26T21:41:07.890580+0000","last_fullsized":"2024-04-26T21:41:07.890580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:07.148493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.897613+0000","last_change":"2024-04-26T21:41:07.893932+0000","last_active":"2024-04-26T21:41:07.897613+0000","last_peered":"2024-04-26T21:41:07.897613+0000","last_clean":"2024-04-26T21:41:07.897613+0000","last_became_active":"2024-04-26T21:41:07.893647+0000","last_became_peered":"2024-04-26T21:41:07.893647+0000","last_unstale":"2024-04-26T21:41:07.897613+0000","last_undegraded":"2024-04-26T21:41:07.897613+0000","last_fullsized":"2024-04-26T21:41:07.897613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:30.597646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.900995+0000","last_change":"2024-04-26T21:41:07.900304+0000","last_active":"2024-04-26T21:41:07.900995+0000","last_peered":"2024-04-26T21:41:07.900995+0000","last_clean":"2024-04-26T21:41:07.900995+0000","last_became_active":"2024-04-26T21:41:07.900022+0000","last_became_peered":"2024-04-26T21:41:07.900022+0000","last_unstale":"2024-04-26T21:41:07.900995+0000","last_undegraded":"2024-04-26T21:41:07.900995+0000","last_fullsized":"2024-04-26T21:41:07.900995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:55.845178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905506+0000","last_change":"2024-04-26T21:41:07.896036+0000","last_active":"2024-04-26T21:41:07.905506+0000","last_peered":"2024-04-26T21:41:07.905506+0000","last_clean":"2024-04-26T21:41:07.905506+0000","last_became_active":"2024-04-26T21:41:07.895762+0000","last_became_peered":"2024-04-26T21:41:07.895762+0000","last_unstale":"2024-04-26T21:41:07.905506+0000","last_undegraded":"2024-04-26T21:41:07.905506+0000","last_fullsized":"2024-04-26T21:41:07.905506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:44.445020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912848+0000","last_change":"2024-04-26T21:41:07.907528+0000","last_active":"2024-04-26T21:41:07.912848+0000","last_peered":"2024-04-26T21:41:07.912848+0000","last_clean":"2024-04-26T21:41:07.912848+0000","last_became_active":"2024-04-26T21:41:07.898594+0000","last_became_peered":"2024-04-26T21:41:07.898594+0000","last_unstale":"2024-04-26T21:41:07.912848+0000","last_undegraded":"2024-04-26T21:41:07.912848+0000","last_fullsized":"2024-04-26T21:41:07.912848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:26.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909843+0000","last_change":"2024-04-26T21:41:07.901427+0000","last_active":"2024-04-26T21:41:07.909843+0000","last_peered":"2024-04-26T21:41:07.909843+0000","last_clean":"2024-04-26T21:41:07.909843+0000","last_became_active":"2024-04-26T21:41:07.901163+0000","last_became_peered":"2024-04-26T21:41:07.901163+0000","last_unstale":"2024-04-26T21:41:07.909843+0000","last_undegraded":"2024-04-26T21:41:07.909843+0000","last_fullsized":"2024-04-26T21:41:07.909843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:23.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.893001+0000","last_change":"2024-04-26T21:41:07.890727+0000","last_active":"2024-04-26T21:41:07.893001+0000","last_peered":"2024-04-26T21:41:07.893001+0000","last_clean":"2024-04-26T21:41:07.893001+0000","last_became_active":"2024-04-26T21:41:07.890444+0000","last_became_peered":"2024-04-26T21:41:07.890444+0000","last_unstale":"2024-04-26T21:41:07.893001+0000","last_undegraded":"2024-04-26T21:41:07.893001+0000","last_fullsized":"2024-04-26T21:41:07.893001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:40.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906534+0000","last_change":"2024-04-26T21:41:07.903999+0000","last_active":"2024-04-26T21:41:07.906534+0000","last_peered":"2024-04-26T21:41:07.906534+0000","last_clean":"2024-04-26T21:41:07.906534+0000","last_became_active":"2024-04-26T21:41:07.903730+0000","last_became_peered":"2024-04-26T21:41:07.903730+0000","last_unstale":"2024-04-26T21:41:07.906534+0000","last_undegraded":"2024-04-26T21:41:07.906534+0000","last_fullsized":"2024-04-26T21:41:07.906534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:14.064616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906231+0000","last_change":"2024-04-26T21:41:07.899122+0000","last_active":"2024-04-26T21:41:07.906231+0000","last_peered":"2024-04-26T21:41:07.906231+0000","last_clean":"2024-04-26T21:41:07.906231+0000","last_became_active":"2024-04-26T21:41:07.898849+0000","last_became_peered":"2024-04-26T21:41:07.898849+0000","last_unstale":"2024-04-26T21:41:07.906231+0000","last_undegraded":"2024-04-26T21:41:07.906231+0000","last_fullsized":"2024-04-26T21:41:07.906231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:54.176753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.896751+0000","last_change":"2024-04-26T21:41:07.891901+0000","last_active":"2024-04-26T21:41:07.896751+0000","last_peered":"2024-04-26T21:41:07.896751+0000","last_clean":"2024-04-26T21:41:07.896751+0000","last_became_active":"2024-04-26T21:41:07.890635+0000","last_became_peered":"2024-04-26T21:41:07.890635+0000","last_unstale":"2024-04-26T21:41:07.896751+0000","last_undegraded":"2024-04-26T21:41:07.896751+0000","last_fullsized":"2024-04-26T21:41:07.896751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:30.597646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.908858+0000","last_change":"2024-04-26T21:41:07.905483+0000","last_active":"2024-04-26T21:41:07.908858+0000","last_peered":"2024-04-26T21:41:07.908858+0000","last_clean":"2024-04-26T21:41:07.908858+0000","last_became_active":"2024-04-26T21:41:07.905185+0000","last_became_peered":"2024-04-26T21:41:07.905185+0000","last_unstale":"2024-04-26T21:41:07.908858+0000","last_undegraded":"2024-04-26T21:41:07.908858+0000","last_fullsized":"2024-04-26T21:41:07.908858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:59.771036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909234+0000","last_change":"2024-04-26T21:41:07.900597+0000","last_active":"2024-04-26T21:41:07.909234+0000","last_peered":"2024-04-26T21:41:07.909234+0000","last_clean":"2024-04-26T21:41:07.909234+0000","last_became_active":"2024-04-26T21:41:07.900305+0000","last_became_peered":"2024-04-26T21:41:07.900305+0000","last_unstale":"2024-04-26T21:41:07.909234+0000","last_undegraded":"2024-04-26T21:41:07.909234+0000","last_fullsized":"2024-04-26T21:41:07.909234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:22.576309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905484+0000","last_change":"2024-04-26T21:41:07.896770+0000","last_active":"2024-04-26T21:41:07.905484+0000","last_peered":"2024-04-26T21:41:07.905484+0000","last_clean":"2024-04-26T21:41:07.905484+0000","last_became_active":"2024-04-26T21:41:07.896494+0000","last_became_peered":"2024-04-26T21:41:07.896494+0000","last_unstale":"2024-04-26T21:41:07.905484+0000","last_undegraded":"2024-04-26T21:41:07.905484+0000","last_fullsized":"2024-04-26T21:41:07.905484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:32.487421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.893680+0000","last_change":"2024-04-26T21:41:07.888426+0000","last_active":"2024-04-26T21:41:07.893680+0000","last_peered":"2024-04-26T21:41:07.893680+0000","last_clean":"2024-04-26T21:41:07.893680+0000","last_became_active":"2024-04-26T21:41:07.887149+0000","last_became_peered":"2024-04-26T21:41:07.887149+0000","last_unstale":"2024-04-26T21:41:07.893680+0000","last_undegraded":"2024-04-26T21:41:07.893680+0000","last_fullsized":"2024-04-26T21:41:07.893680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:52.053762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912862+0000","last_change":"2024-04-26T21:41:07.904976+0000","last_active":"2024-04-26T21:41:07.912862+0000","last_peered":"2024-04-26T21:41:07.912862+0000","last_clean":"2024-04-26T21:41:07.912862+0000","last_became_active":"2024-04-26T21:41:07.904477+0000","last_became_peered":"2024-04-26T21:41:07.904477+0000","last_unstale":"2024-04-26T21:41:07.912862+0000","last_undegraded":"2024-04-26T21:41:07.912862+0000","last_fullsized":"2024-04-26T21:41:07.912862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:41.740959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912886+0000","last_change":"2024-04-26T21:41:07.907095+0000","last_active":"2024-04-26T21:41:07.912886+0000","last_peered":"2024-04-26T21:41:07.912886+0000","last_clean":"2024-04-26T21:41:07.912886+0000","last_became_active":"2024-04-26T21:41:07.906812+0000","last_became_peered":"2024-04-26T21:41:07.906812+0000","last_unstale":"2024-04-26T21:41:07.912886+0000","last_undegraded":"2024-04-26T21:41:07.912886+0000","last_fullsized":"2024-04-26T21:41:07.912886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:30:04.598780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.911946+0000","last_change":"2024-04-26T21:41:07.903788+0000","last_active":"2024-04-26T21:41:07.911946+0000","last_peered":"2024-04-26T21:41:07.911946+0000","last_clean":"2024-04-26T21:41:07.911946+0000","last_became_active":"2024-04-26T21:41:07.903491+0000","last_became_peered":"2024-04-26T21:41:07.903491+0000","last_unstale":"2024-04-26T21:41:07.911946+0000","last_undegraded":"2024-04-26T21:41:07.911946+0000","last_fullsized":"2024-04-26T21:41:07.911946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:36.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.891501+0000","last_change":"2024-04-26T21:41:07.889123+0000","last_active":"2024-04-26T21:41:07.891501+0000","last_peered":"2024-04-26T21:41:07.891501+0000","last_clean":"2024-04-26T21:41:07.891501+0000","last_became_active":"2024-04-26T21:41:07.888852+0000","last_became_peered":"2024-04-26T21:41:07.888852+0000","last_unstale":"2024-04-26T21:41:07.891501+0000","last_undegraded":"2024-04-26T21:41:07.891501+0000","last_fullsized":"2024-04-26T21:41:07.891501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:13.308104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.887858+0000","last_change":"2024-04-26T21:41:07.881706+0000","last_active":"2024-04-26T21:41:07.887858+0000","last_peered":"2024-04-26T21:41:07.887858+0000","last_clean":"2024-04-26T21:41:07.887858+0000","last_became_active":"2024-04-26T21:41:07.881329+0000","last_became_peered":"2024-04-26T21:41:07.881329+0000","last_unstale":"2024-04-26T21:41:07.887858+0000","last_undegraded":"2024-04-26T21:41:07.887858+0000","last_fullsized":"2024-04-26T21:41:07.887858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:36.676505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.891546+0000","last_change":"2024-04-26T21:41:07.889756+0000","last_active":"2024-04-26T21:41:07.891546+0000","last_peered":"2024-04-26T21:41:07.891546+0000","last_clean":"2024-04-26T21:41:07.891546+0000","last_became_active":"2024-04-26T21:41:07.889386+0000","last_became_peered":"2024-04-26T21:41:07.889386+0000","last_unstale":"2024-04-26T21:41:07.891546+0000","last_undegraded":"2024-04-26T21:41:07.891546+0000","last_fullsized":"2024-04-26T21:41:07.891546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:55.845178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898045+0000","last_change":"2024-04-26T21:41:07.893163+0000","last_active":"2024-04-26T21:41:07.898045+0000","last_peered":"2024-04-26T21:41:07.898045+0000","last_clean":"2024-04-26T21:41:07.898045+0000","last_became_active":"2024-04-26T21:41:07.892874+0000","last_became_peered":"2024-04-26T21:41:07.892874+0000","last_unstale":"2024-04-26T21:41:07.898045+0000","last_undegraded":"2024-04-26T21:41:07.898045+0000","last_fullsized":"2024-04-26T21:41:07.898045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.679698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906250+0000","last_change":"2024-04-26T21:41:07.895128+0000","last_active":"2024-04-26T21:41:07.906250+0000","last_peered":"2024-04-26T21:41:07.906250+0000","last_clean":"2024-04-26T21:41:07.906250+0000","last_became_active":"2024-04-26T21:41:07.894830+0000","last_became_peered":"2024-04-26T21:41:07.894830+0000","last_unstale":"2024-04-26T21:41:07.906250+0000","last_undegraded":"2024-04-26T21:41:07.906250+0000","last_fullsized":"2024-04-26T21:41:07.906250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:25.447410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898339+0000","last_change":"2024-04-26T21:41:07.893986+0000","last_active":"2024-04-26T21:41:07.898339+0000","last_peered":"2024-04-26T21:41:07.898339+0000","last_clean":"2024-04-26T21:41:07.898339+0000","last_became_active":"2024-04-26T21:41:07.893685+0000","last_became_peered":"2024-04-26T21:41:07.893685+0000","last_unstale":"2024-04-26T21:41:07.898339+0000","last_undegraded":"2024-04-26T21:41:07.898339+0000","last_fullsized":"2024-04-26T21:41:07.898339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:23.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.902723+0000","last_change":"2024-04-26T21:41:07.894071+0000","last_active":"2024-04-26T21:41:07.902723+0000","last_peered":"2024-04-26T21:41:07.902723+0000","last_clean":"2024-04-26T21:41:07.902723+0000","last_became_active":"2024-04-26T21:41:07.893807+0000","last_became_peered":"2024-04-26T21:41:07.893807+0000","last_unstale":"2024-04-26T21:41:07.902723+0000","last_undegraded":"2024-04-26T21:41:07.902723+0000","last_fullsized":"2024-04-26T21:41:07.902723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:03.104758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.908707+0000","last_change":"2024-04-26T21:41:06.883010+0000","last_active":"2024-04-26T21:41:07.908707+0000","last_peered":"2024-04-26T21:41:07.908707+0000","last_clean":"2024-04-26T21:41:07.908707+0000","last_became_active":"2024-04-26T21:41:06.882711+0000","last_became_peered":"2024-04-26T21:41:06.882711+0000","last_unstale":"2024-04-26T21:41:07.908707+0000","last_undegraded":"2024-04-26T21:41:07.908707+0000","last_fullsized":"2024-04-26T21:41:07.908707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:36.058840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904286+0000","last_change":"2024-04-26T21:41:07.900864+0000","last_active":"2024-04-26T21:41:07.904286+0000","last_peered":"2024-04-26T21:41:07.904286+0000","last_clean":"2024-04-26T21:41:07.904286+0000","last_became_active":"2024-04-26T21:41:07.900588+0000","last_became_peered":"2024-04-26T21:41:07.900588+0000","last_unstale":"2024-04-26T21:41:07.904286+0000","last_undegraded":"2024-04-26T21:41:07.904286+0000","last_fullsized":"2024-04-26T21:41:07.904286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:26.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.894996+0000","last_change":"2024-04-26T21:41:06.889946+0000","last_active":"2024-04-26T21:41:07.894996+0000","last_peered":"2024-04-26T21:41:07.894996+0000","last_clean":"2024-04-26T21:41:07.894996+0000","last_became_active":"2024-04-26T21:41:06.889593+0000","last_became_peered":"2024-04-26T21:41:06.889593+0000","last_unstale":"2024-04-26T21:41:07.894996+0000","last_undegraded":"2024-04-26T21:41:07.894996+0000","last_fullsized":"2024-04-26T21:41:07.894996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:06.146449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898698+0000","last_change":"2024-04-26T21:41:07.894766+0000","last_active":"2024-04-26T21:41:07.898698+0000","last_peered":"2024-04-26T21:41:07.898698+0000","last_clean":"2024-04-26T21:41:07.898698+0000","last_became_active":"2024-04-26T21:41:07.894489+0000","last_became_peered":"2024-04-26T21:41:07.894489+0000","last_unstale":"2024-04-26T21:41:07.898698+0000","last_undegraded":"2024-04-26T21:41:07.898698+0000","last_fullsized":"2024-04-26T21:41:07.898698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:45.308669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895848+0000","last_change":"2024-04-26T21:41:06.889395+0000","last_active":"2024-04-26T21:41:07.895848+0000","last_peered":"2024-04-26T21:41:07.895848+0000","last_clean":"2024-04-26T21:41:07.895848+0000","last_became_active":"2024-04-26T21:41:06.889099+0000","last_became_peered":"2024-04-26T21:41:06.889099+0000","last_unstale":"2024-04-26T21:41:07.895848+0000","last_undegraded":"2024-04-26T21:41:07.895848+0000","last_fullsized":"2024-04-26T21:41:07.895848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:12.480105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909301+0000","last_change":"2024-04-26T21:41:07.900780+0000","last_active":"2024-04-26T21:41:07.909301+0000","last_peered":"2024-04-26T21:41:07.909301+0000","last_clean":"2024-04-26T21:41:07.909301+0000","last_became_active":"2024-04-26T21:41:07.900494+0000","last_became_peered":"2024-04-26T21:41:07.900494+0000","last_unstale":"2024-04-26T21:41:07.909301+0000","last_undegraded":"2024-04-26T21:41:07.909301+0000","last_fullsized":"2024-04-26T21:41:07.909301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:00.438630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895737+0000","last_change":"2024-04-26T21:41:06.888763+0000","last_active":"2024-04-26T21:41:07.895737+0000","last_peered":"2024-04-26T21:41:07.895737+0000","last_clean":"2024-04-26T21:41:07.895737+0000","last_became_active":"2024-04-26T21:41:06.888161+0000","last_became_peered":"2024-04-26T21:41:06.888161+0000","last_unstale":"2024-04-26T21:41:07.895737+0000","last_undegraded":"2024-04-26T21:41:07.895737+0000","last_fullsized":"2024-04-26T21:41:07.895737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:24.383801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905605+0000","last_change":"2024-04-26T21:41:07.900799+0000","last_active":"2024-04-26T21:41:07.905605+0000","last_peered":"2024-04-26T21:41:07.905605+0000","last_clean":"2024-04-26T21:41:07.905605+0000","last_became_active":"2024-04-26T21:41:07.900251+0000","last_became_peered":"2024-04-26T21:41:07.900251+0000","last_unstale":"2024-04-26T21:41:07.905605+0000","last_undegraded":"2024-04-26T21:41:07.905605+0000","last_fullsized":"2024-04-26T21:41:07.905605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.679698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898868+0000","last_change":"2024-04-26T21:41:06.885609+0000","last_active":"2024-04-26T21:41:07.898868+0000","last_peered":"2024-04-26T21:41:07.898868+0000","last_clean":"2024-04-26T21:41:07.898868+0000","last_became_active":"2024-04-26T21:41:06.885339+0000","last_became_peered":"2024-04-26T21:41:06.885339+0000","last_unstale":"2024-04-26T21:41:07.898868+0000","last_undegraded":"2024-04-26T21:41:07.898868+0000","last_fullsized":"2024-04-26T21:41:07.898868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:35.748907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.897638+0000","last_change":"2024-04-26T21:41:07.893359+0000","last_active":"2024-04-26T21:41:07.897638+0000","last_peered":"2024-04-26T21:41:07.897638+0000","last_clean":"2024-04-26T21:41:07.897638+0000","last_became_active":"2024-04-26T21:41:07.893061+0000","last_became_peered":"2024-04-26T21:41:07.893061+0000","last_unstale":"2024-04-26T21:41:07.897638+0000","last_undegraded":"2024-04-26T21:41:07.897638+0000","last_fullsized":"2024-04-26T21:41:07.897638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.159772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.907524+0000","last_change":"2024-04-26T21:41:06.883727+0000","last_active":"2024-04-26T21:41:07.907524+0000","last_peered":"2024-04-26T21:41:07.907524+0000","last_clean":"2024-04-26T21:41:07.907524+0000","last_became_active":"2024-04-26T21:41:06.883476+0000","last_became_peered":"2024-04-26T21:41:06.883476+0000","last_unstale":"2024-04-26T21:41:07.907524+0000","last_undegraded":"2024-04-26T21:41:07.907524+0000","last_fullsized":"2024-04-26T21:41:07.907524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:32.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898860+0000","last_change":"2024-04-26T21:41:07.895759+0000","last_active":"2024-04-26T21:41:07.898860+0000","last_peered":"2024-04-26T21:41:07.898860+0000","last_clean":"2024-04-26T21:41:07.898860+0000","last_became_active":"2024-04-26T21:41:07.894370+0000","last_became_peered":"2024-04-26T21:41:07.894370+0000","last_unstale":"2024-04-26T21:41:07.898860+0000","last_undegraded":"2024-04-26T21:41:07.898860+0000","last_fullsized":"2024-04-26T21:41:07.898860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:12.514077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.887817+0000","last_change":"2024-04-26T21:41:06.886390+0000","last_active":"2024-04-26T21:41:07.887817+0000","last_peered":"2024-04-26T21:41:07.887817+0000","last_clean":"2024-04-26T21:41:07.887817+0000","last_became_active":"2024-04-26T21:41:06.885992+0000","last_became_peered":"2024-04-26T21:41:06.885992+0000","last_unstale":"2024-04-26T21:41:07.887817+0000","last_undegraded":"2024-04-26T21:41:07.887817+0000","last_fullsized":"2024-04-26T21:41:07.887817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:06.146449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904676+0000","last_change":"2024-04-26T21:41:07.897377+0000","last_active":"2024-04-26T21:41:07.904676+0000","last_peered":"2024-04-26T21:41:07.904676+0000","last_clean":"2024-04-26T21:41:07.904676+0000","last_became_active":"2024-04-26T21:41:07.897046+0000","last_became_peered":"2024-04-26T21:41:07.897046+0000","last_unstale":"2024-04-26T21:41:07.904676+0000","last_undegraded":"2024-04-26T21:41:07.904676+0000","last_fullsized":"2024-04-26T21:41:07.904676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:22.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.875479+0000","last_change":"2024-04-26T21:41:06.880632+0000","last_active":"2024-04-26T21:41:07.875479+0000","last_peered":"2024-04-26T21:41:07.875479+0000","last_clean":"2024-04-26T21:41:07.875479+0000","last_became_active":"2024-04-26T21:41:06.880182+0000","last_became_peered":"2024-04-26T21:41:06.880182+0000","last_unstale":"2024-04-26T21:41:07.875479+0000","last_undegraded":"2024-04-26T21:41:07.875479+0000","last_fullsized":"2024-04-26T21:41:07.875479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:43.697586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.897721+0000","last_change":"2024-04-26T21:41:07.895956+0000","last_active":"2024-04-26T21:41:07.897721+0000","last_peered":"2024-04-26T21:41:07.897721+0000","last_clean":"2024-04-26T21:41:07.897721+0000","last_became_active":"2024-04-26T21:41:07.895688+0000","last_became_peered":"2024-04-26T21:41:07.895688+0000","last_unstale":"2024-04-26T21:41:07.897721+0000","last_undegraded":"2024-04-26T21:41:07.897721+0000","last_fullsized":"2024-04-26T21:41:07.897721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:13.308104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.882577+0000","last_change":"2024-04-26T21:41:06.890600+0000","last_active":"2024-04-26T21:41:07.882577+0000","last_peered":"2024-04-26T21:41:07.882577+0000","last_clean":"2024-04-26T21:41:07.882577+0000","last_became_active":"2024-04-26T21:41:06.890215+0000","last_became_peered":"2024-04-26T21:41:06.890215+0000","last_unstale":"2024-04-26T21:41:07.882577+0000","last_undegraded":"2024-04-26T21:41:07.882577+0000","last_fullsized":"2024-04-26T21:41:07.882577+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:54.843135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.888145+0000","last_change":"2024-04-26T21:41:07.881399+0000","last_active":"2024-04-26T21:41:07.888145+0000","last_peered":"2024-04-26T21:41:07.888145+0000","last_clean":"2024-04-26T21:41:07.888145+0000","last_became_active":"2024-04-26T21:41:07.881048+0000","last_became_peered":"2024-04-26T21:41:07.881048+0000","last_unstale":"2024-04-26T21:41:07.888145+0000","last_undegraded":"2024-04-26T21:41:07.888145+0000","last_fullsized":"2024-04-26T21:41:07.888145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.468687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.877723+0000","last_change":"2024-04-26T21:41:06.885244+0000","last_active":"2024-04-26T21:41:07.877723+0000","last_peered":"2024-04-26T21:41:07.877723+0000","last_clean":"2024-04-26T21:41:07.877723+0000","last_became_active":"2024-04-26T21:41:06.884976+0000","last_became_peered":"2024-04-26T21:41:06.884976+0000","last_unstale":"2024-04-26T21:41:07.877723+0000","last_undegraded":"2024-04-26T21:41:07.877723+0000","last_fullsized":"2024-04-26T21:41:07.877723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:14.245124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906079+0000","last_change":"2024-04-26T21:41:07.895869+0000","last_active":"2024-04-26T21:41:07.906079+0000","last_peered":"2024-04-26T21:41:07.906079+0000","last_clean":"2024-04-26T21:41:07.906079+0000","last_became_active":"2024-04-26T21:41:07.895592+0000","last_became_peered":"2024-04-26T21:41:07.895592+0000","last_unstale":"2024-04-26T21:41:07.906079+0000","last_undegraded":"2024-04-26T21:41:07.906079+0000","last_fullsized":"2024-04-26T21:41:07.906079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:13.308104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.910071+0000","last_change":"2024-04-26T21:41:06.892212+0000","last_active":"2024-04-26T21:41:07.910071+0000","last_peered":"2024-04-26T21:41:07.910071+0000","last_clean":"2024-04-26T21:41:07.910071+0000","last_became_active":"2024-04-26T21:41:06.891908+0000","last_became_peered":"2024-04-26T21:41:06.891908+0000","last_unstale":"2024-04-26T21:41:07.910071+0000","last_undegraded":"2024-04-26T21:41:07.910071+0000","last_fullsized":"2024-04-26T21:41:07.910071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:43.131495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906331+0000","last_change":"2024-04-26T21:41:07.896657+0000","last_active":"2024-04-26T21:41:07.906331+0000","last_peered":"2024-04-26T21:41:07.906331+0000","last_clean":"2024-04-26T21:41:07.906331+0000","last_became_active":"2024-04-26T21:41:07.896309+0000","last_became_peered":"2024-04-26T21:41:07.896309+0000","last_unstale":"2024-04-26T21:41:07.906331+0000","last_undegraded":"2024-04-26T21:41:07.906331+0000","last_fullsized":"2024-04-26T21:41:07.906331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:22.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"80'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912244+0000","last_change":"2024-04-26T21:41:06.883956+0000","last_active":"2024-04-26T21:41:07.912244+0000","last_peered":"2024-04-26T21:41:07.912244+0000","last_clean":"2024-04-26T21:41:07.912244+0000","last_became_active":"2024-04-26T21:41:06.883451+0000","last_became_peered":"2024-04-26T21:41:06.883451+0000","last_unstale":"2024-04-26T21:41:07.912244+0000","last_undegraded":"2024-04-26T21:41:07.912244+0000","last_fullsized":"2024-04-26T21:41:07.912244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:19.512895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895442+0000","last_change":"2024-04-26T21:41:07.893429+0000","last_active":"2024-04-26T21:41:07.895442+0000","last_peered":"2024-04-26T21:41:07.895442+0000","last_clean":"2024-04-26T21:41:07.895442+0000","last_became_active":"2024-04-26T21:41:07.893108+0000","last_became_peered":"2024-04-26T21:41:07.893108+0000","last_unstale":"2024-04-26T21:41:07.895442+0000","last_undegraded":"2024-04-26T21:41:07.895442+0000","last_fullsized":"2024-04-26T21:41:07.895442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:13.482148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909520+0000","last_change":"2024-04-26T21:41:06.880133+0000","last_active":"2024-04-26T21:41:07.909520+0000","last_peered":"2024-04-26T21:41:07.909520+0000","last_clean":"2024-04-26T21:41:07.909520+0000","last_became_active":"2024-04-26T21:41:06.879862+0000","last_became_peered":"2024-04-26T21:41:06.879862+0000","last_unstale":"2024-04-26T21:41:07.909520+0000","last_undegraded":"2024-04-26T21:41:07.909520+0000","last_fullsized":"2024-04-26T21:41:07.909520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:38.401980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904760+0000","last_change":"2024-04-26T21:41:07.898592+0000","last_active":"2024-04-26T21:41:07.904760+0000","last_peered":"2024-04-26T21:41:07.904760+0000","last_clean":"2024-04-26T21:41:07.904760+0000","last_became_active":"2024-04-26T21:41:07.898294+0000","last_became_peered":"2024-04-26T21:41:07.898294+0000","last_unstale":"2024-04-26T21:41:07.904760+0000","last_undegraded":"2024-04-26T21:41:07.904760+0000","last_fullsized":"2024-04-26T21:41:07.904760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:54.176753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.882691+0000","last_change":"2024-04-26T21:41:06.892424+0000","last_active":"2024-04-26T21:41:07.882691+0000","last_peered":"2024-04-26T21:41:07.882691+0000","last_clean":"2024-04-26T21:41:07.882691+0000","last_became_active":"2024-04-26T21:41:06.892162+0000","last_became_peered":"2024-04-26T21:41:06.892162+0000","last_unstale":"2024-04-26T21:41:07.882691+0000","last_undegraded":"2024-04-26T21:41:07.882691+0000","last_fullsized":"2024-04-26T21:41:07.882691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:35.626916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904813+0000","last_change":"2024-04-26T21:41:07.899834+0000","last_active":"2024-04-26T21:41:07.904813+0000","last_peered":"2024-04-26T21:41:07.904813+0000","last_clean":"2024-04-26T21:41:07.904813+0000","last_became_active":"2024-04-26T21:41:07.899539+0000","last_became_peered":"2024-04-26T21:41:07.899539+0000","last_unstale":"2024-04-26T21:41:07.904813+0000","last_undegraded":"2024-04-26T21:41:07.904813+0000","last_fullsized":"2024-04-26T21:41:07.904813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:52.530257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.875899+0000","last_change":"2024-04-26T21:41:06.889071+0000","last_active":"2024-04-26T21:41:07.875899+0000","last_peered":"2024-04-26T21:41:07.875899+0000","last_clean":"2024-04-26T21:41:07.875899+0000","last_became_active":"2024-04-26T21:41:06.888585+0000","last_became_peered":"2024-04-26T21:41:06.888585+0000","last_unstale":"2024-04-26T21:41:07.875899+0000","last_undegraded":"2024-04-26T21:41:07.875899+0000","last_fullsized":"2024-04-26T21:41:07.875899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:05.934276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895144+0000","last_change":"2024-04-26T21:41:07.893997+0000","last_active":"2024-04-26T21:41:07.895144+0000","last_peered":"2024-04-26T21:41:07.895144+0000","last_clean":"2024-04-26T21:41:07.895144+0000","last_became_active":"2024-04-26T21:41:07.893712+0000","last_became_peered":"2024-04-26T21:41:07.893712+0000","last_unstale":"2024-04-26T21:41:07.895144+0000","last_undegraded":"2024-04-26T21:41:07.895144+0000","last_fullsized":"2024-04-26T21:41:07.895144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:33.183752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.908457+0000","last_change":"2024-04-26T21:41:06.890341+0000","last_active":"2024-04-26T21:41:07.908457+0000","last_peered":"2024-04-26T21:41:07.908457+0000","last_clean":"2024-04-26T21:41:07.908457+0000","last_became_active":"2024-04-26T21:41:06.889977+0000","last_became_peered":"2024-04-26T21:41:06.889977+0000","last_unstale":"2024-04-26T21:41:07.908457+0000","last_undegraded":"2024-04-26T21:41:07.908457+0000","last_fullsized":"2024-04-26T21:41:07.908457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:12.480105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.907326+0000","last_change":"2024-04-26T21:41:07.902084+0000","last_active":"2024-04-26T21:41:07.907326+0000","last_peered":"2024-04-26T21:41:07.907326+0000","last_clean":"2024-04-26T21:41:07.907326+0000","last_became_active":"2024-04-26T21:41:07.901810+0000","last_became_peered":"2024-04-26T21:41:07.901810+0000","last_unstale":"2024-04-26T21:41:07.907326+0000","last_undegraded":"2024-04-26T21:41:07.907326+0000","last_fullsized":"2024-04-26T21:41:07.907326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.679698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.903754+0000","last_change":"2024-04-26T21:41:06.893855+0000","last_active":"2024-04-26T21:41:07.903754+0000","last_peered":"2024-04-26T21:41:07.903754+0000","last_clean":"2024-04-26T21:41:07.903754+0000","last_became_active":"2024-04-26T21:41:06.893560+0000","last_became_peered":"2024-04-26T21:41:06.893560+0000","last_unstale":"2024-04-26T21:41:07.903754+0000","last_undegraded":"2024-04-26T21:41:07.903754+0000","last_fullsized":"2024-04-26T21:41:07.903754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.595603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906338+0000","last_change":"2024-04-26T21:41:07.899729+0000","last_active":"2024-04-26T21:41:07.906338+0000","last_peered":"2024-04-26T21:41:07.906338+0000","last_clean":"2024-04-26T21:41:07.906338+0000","last_became_active":"2024-04-26T21:41:07.898923+0000","last_became_peered":"2024-04-26T21:41:07.898923+0000","last_unstale":"2024-04-26T21:41:07.906338+0000","last_undegraded":"2024-04-26T21:41:07.906338+0000","last_fullsized":"2024-04-26T21:41:07.906338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:44.804002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906899+0000","last_change":"2024-04-26T21:41:06.893526+0000","last_active":"2024-04-26T21:41:07.906899+0000","last_peered":"2024-04-26T21:41:07.906899+0000","last_clean":"2024-04-26T21:41:07.906899+0000","last_became_active":"2024-04-26T21:41:06.893251+0000","last_became_peered":"2024-04-26T21:41:06.893251+0000","last_unstale":"2024-04-26T21:41:07.906899+0000","last_undegraded":"2024-04-26T21:41:07.906899+0000","last_fullsized":"2024-04-26T21:41:07.906899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:57.452521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.897763+0000","last_change":"2024-04-26T21:41:07.894000+0000","last_active":"2024-04-26T21:41:07.897763+0000","last_peered":"2024-04-26T21:41:07.897763+0000","last_clean":"2024-04-26T21:41:07.897763+0000","last_became_active":"2024-04-26T21:41:07.893723+0000","last_became_peered":"2024-04-26T21:41:07.893723+0000","last_unstale":"2024-04-26T21:41:07.897763+0000","last_undegraded":"2024-04-26T21:41:07.897763+0000","last_fullsized":"2024-04-26T21:41:07.897763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:09.889519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.900587+0000","last_change":"2024-04-26T21:41:06.879741+0000","last_active":"2024-04-26T21:41:07.900587+0000","last_peered":"2024-04-26T21:41:07.900587+0000","last_clean":"2024-04-26T21:41:07.900587+0000","last_became_active":"2024-04-26T21:41:06.879456+0000","last_became_peered":"2024-04-26T21:41:06.879456+0000","last_unstale":"2024-04-26T21:41:07.900587+0000","last_undegraded":"2024-04-26T21:41:07.900587+0000","last_fullsized":"2024-04-26T21:41:07.900587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:55.528540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.896707+0000","last_change":"2024-04-26T21:41:07.892055+0000","last_active":"2024-04-26T21:41:07.896707+0000","last_peered":"2024-04-26T21:41:07.896707+0000","last_clean":"2024-04-26T21:41:07.896707+0000","last_became_active":"2024-04-26T21:41:07.891744+0000","last_became_peered":"2024-04-26T21:41:07.891744+0000","last_unstale":"2024-04-26T21:41:07.896707+0000","last_undegraded":"2024-04-26T21:41:07.896707+0000","last_fullsized":"2024-04-26T21:41:07.896707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.159772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.882664+0000","last_change":"2024-04-26T21:41:06.892445+0000","last_active":"2024-04-26T21:41:07.882664+0000","last_peered":"2024-04-26T21:41:07.882664+0000","last_clean":"2024-04-26T21:41:07.882664+0000","last_became_active":"2024-04-26T21:41:06.892151+0000","last_became_peered":"2024-04-26T21:41:06.892151+0000","last_unstale":"2024-04-26T21:41:07.882664+0000","last_undegraded":"2024-04-26T21:41:07.882664+0000","last_fullsized":"2024-04-26T21:41:07.882664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:32.181709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905582+0000","last_change":"2024-04-26T21:41:07.900570+0000","last_active":"2024-04-26T21:41:07.905582+0000","last_peered":"2024-04-26T21:41:07.905582+0000","last_clean":"2024-04-26T21:41:07.905582+0000","last_became_active":"2024-04-26T21:41:07.899894+0000","last_became_peered":"2024-04-26T21:41:07.899894+0000","last_unstale":"2024-04-26T21:41:07.905582+0000","last_undegraded":"2024-04-26T21:41:07.905582+0000","last_fullsized":"2024-04-26T21:41:07.905582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:40.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.910504+0000","last_change":"2024-04-26T21:41:06.892469+0000","last_active":"2024-04-26T21:41:07.910504+0000","last_peered":"2024-04-26T21:41:07.910504+0000","last_clean":"2024-04-26T21:41:07.910504+0000","last_became_active":"2024-04-26T21:41:06.892155+0000","last_became_peered":"2024-04-26T21:41:06.892155+0000","last_unstale":"2024-04-26T21:41:07.910504+0000","last_undegraded":"2024-04-26T21:41:07.910504+0000","last_fullsized":"2024-04-26T21:41:07.910504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:06.146449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912793+0000","last_change":"2024-04-26T21:41:07.903469+0000","last_active":"2024-04-26T21:41:07.912793+0000","last_peered":"2024-04-26T21:41:07.912793+0000","last_clean":"2024-04-26T21:41:07.912793+0000","last_became_active":"2024-04-26T21:41:07.903179+0000","last_became_peered":"2024-04-26T21:41:07.903179+0000","last_unstale":"2024-04-26T21:41:07.912793+0000","last_undegraded":"2024-04-26T21:41:07.912793+0000","last_fullsized":"2024-04-26T21:41:07.912793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.196025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.883563+0000","last_change":"2024-04-26T21:41:06.892386+0000","last_active":"2024-04-26T21:41:07.883563+0000","last_peered":"2024-04-26T21:41:07.883563+0000","last_clean":"2024-04-26T21:41:07.883563+0000","last_became_active":"2024-04-26T21:41:06.890974+0000","last_became_peered":"2024-04-26T21:41:06.890974+0000","last_unstale":"2024-04-26T21:41:07.883563+0000","last_undegraded":"2024-04-26T21:41:07.883563+0000","last_fullsized":"2024-04-26T21:41:07.883563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:12.480105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.901416+0000","last_change":"2024-04-26T21:41:07.895083+0000","last_active":"2024-04-26T21:41:07.901416+0000","last_peered":"2024-04-26T21:41:07.901416+0000","last_clean":"2024-04-26T21:41:07.901416+0000","last_became_active":"2024-04-26T21:41:07.894807+0000","last_became_peered":"2024-04-26T21:41:07.894807+0000","last_unstale":"2024-04-26T21:41:07.901416+0000","last_undegraded":"2024-04-26T21:41:07.901416+0000","last_fullsized":"2024-04-26T21:41:07.901416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:25.447410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.883657+0000","last_change":"2024-04-26T21:41:06.884947+0000","last_active":"2024-04-26T21:41:07.883657+0000","last_peered":"2024-04-26T21:41:07.883657+0000","last_clean":"2024-04-26T21:41:07.883657+0000","last_became_active":"2024-04-26T21:41:06.884676+0000","last_became_peered":"2024-04-26T21:41:06.884676+0000","last_unstale":"2024-04-26T21:41:07.883657+0000","last_undegraded":"2024-04-26T21:41:07.883657+0000","last_fullsized":"2024-04-26T21:41:07.883657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:54.843135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895307+0000","last_change":"2024-04-26T21:41:07.891525+0000","last_active":"2024-04-26T21:41:07.895307+0000","last_peered":"2024-04-26T21:41:07.895307+0000","last_clean":"2024-04-26T21:41:07.895307+0000","last_became_active":"2024-04-26T21:41:07.891195+0000","last_became_peered":"2024-04-26T21:41:07.891195+0000","last_unstale":"2024-04-26T21:41:07.895307+0000","last_undegraded":"2024-04-26T21:41:07.895307+0000","last_fullsized":"2024-04-26T21:41:07.895307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:12.443656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906814+0000","last_change":"2024-04-26T21:41:06.892246+0000","last_active":"2024-04-26T21:41:07.906814+0000","last_peered":"2024-04-26T21:41:07.906814+0000","last_clean":"2024-04-26T21:41:07.906814+0000","last_became_active":"2024-04-26T21:41:06.891923+0000","last_became_peered":"2024-04-26T21:41:06.891923+0000","last_unstale":"2024-04-26T21:41:07.906814+0000","last_undegraded":"2024-04-26T21:41:07.906814+0000","last_fullsized":"2024-04-26T21:41:07.906814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:15.083326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.894993+0000","last_change":"2024-04-26T21:41:07.892907+0000","last_active":"2024-04-26T21:41:07.894993+0000","last_peered":"2024-04-26T21:41:07.894993+0000","last_clean":"2024-04-26T21:41:07.894993+0000","last_became_active":"2024-04-26T21:41:07.892624+0000","last_became_peered":"2024-04-26T21:41:07.892624+0000","last_unstale":"2024-04-26T21:41:07.894993+0000","last_undegraded":"2024-04-26T21:41:07.894993+0000","last_fullsized":"2024-04-26T21:41:07.894993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:49.407084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.887285+0000","last_change":"2024-04-26T21:41:06.885610+0000","last_active":"2024-04-26T21:41:07.887285+0000","last_peered":"2024-04-26T21:41:07.887285+0000","last_clean":"2024-04-26T21:41:07.887285+0000","last_became_active":"2024-04-26T21:41:06.885296+0000","last_became_peered":"2024-04-26T21:41:06.885296+0000","last_unstale":"2024-04-26T21:41:07.887285+0000","last_undegraded":"2024-04-26T21:41:07.887285+0000","last_fullsized":"2024-04-26T21:41:07.887285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:38.466644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.899056+0000","last_change":"2024-04-26T21:41:07.895738+0000","last_active":"2024-04-26T21:41:07.899056+0000","last_peered":"2024-04-26T21:41:07.899056+0000","last_clean":"2024-04-26T21:41:07.899056+0000","last_became_active":"2024-04-26T21:41:07.895353+0000","last_became_peered":"2024-04-26T21:41:07.895353+0000","last_unstale":"2024-04-26T21:41:07.899056+0000","last_undegraded":"2024-04-26T21:41:07.899056+0000","last_fullsized":"2024-04-26T21:41:07.899056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:10.773944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.885413+0000","last_change":"2024-04-26T21:41:06.886737+0000","last_active":"2024-04-26T21:41:07.885413+0000","last_peered":"2024-04-26T21:41:07.885413+0000","last_clean":"2024-04-26T21:41:07.885413+0000","last_became_active":"2024-04-26T21:41:06.886460+0000","last_became_peered":"2024-04-26T21:41:06.886460+0000","last_unstale":"2024-04-26T21:41:07.885413+0000","last_undegraded":"2024-04-26T21:41:07.885413+0000","last_fullsized":"2024-04-26T21:41:07.885413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:49.157729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.893258+0000","last_change":"2024-04-26T21:41:07.890642+0000","last_active":"2024-04-26T21:41:07.893258+0000","last_peered":"2024-04-26T21:41:07.893258+0000","last_clean":"2024-04-26T21:41:07.893258+0000","last_became_active":"2024-04-26T21:41:07.890306+0000","last_became_peered":"2024-04-26T21:41:07.890306+0000","last_unstale":"2024-04-26T21:41:07.893258+0000","last_undegraded":"2024-04-26T21:41:07.893258+0000","last_fullsized":"2024-04-26T21:41:07.893258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:36.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.880978+0000","last_change":"2024-04-26T21:41:06.889949+0000","last_active":"2024-04-26T21:41:07.880978+0000","last_peered":"2024-04-26T21:41:07.880978+0000","last_clean":"2024-04-26T21:41:07.880978+0000","last_became_active":"2024-04-26T21:41:06.889613+0000","last_became_peered":"2024-04-26T21:41:06.889613+0000","last_unstale":"2024-04-26T21:41:07.880978+0000","last_undegraded":"2024-04-26T21:41:07.880978+0000","last_fullsized":"2024-04-26T21:41:07.880978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:38.466644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.901488+0000","last_change":"2024-04-26T21:41:07.894557+0000","last_active":"2024-04-26T21:41:07.901488+0000","last_peered":"2024-04-26T21:41:07.901488+0000","last_clean":"2024-04-26T21:41:07.901488+0000","last_became_active":"2024-04-26T21:41:07.892591+0000","last_became_peered":"2024-04-26T21:41:07.892591+0000","last_unstale":"2024-04-26T21:41:07.901488+0000","last_undegraded":"2024-04-26T21:41:07.901488+0000","last_fullsized":"2024-04-26T21:41:07.901488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:03.104758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.883428+0000","last_change":"2024-04-26T21:41:06.888469+0000","last_active":"2024-04-26T21:41:07.883428+0000","last_peered":"2024-04-26T21:41:07.883428+0000","last_clean":"2024-04-26T21:41:07.883428+0000","last_became_active":"2024-04-26T21:41:06.887986+0000","last_became_peered":"2024-04-26T21:41:06.887986+0000","last_unstale":"2024-04-26T21:41:07.883428+0000","last_undegraded":"2024-04-26T21:41:07.883428+0000","last_fullsized":"2024-04-26T21:41:07.883428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:42.543019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904639+0000","last_change":"2024-04-26T21:41:07.896999+0000","last_active":"2024-04-26T21:41:07.904639+0000","last_peered":"2024-04-26T21:41:07.904639+0000","last_clean":"2024-04-26T21:41:07.904639+0000","last_became_active":"2024-04-26T21:41:07.896692+0000","last_became_peered":"2024-04-26T21:41:07.896692+0000","last_unstale":"2024-04-26T21:41:07.904639+0000","last_undegraded":"2024-04-26T21:41:07.904639+0000","last_fullsized":"2024-04-26T21:41:07.904639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.467106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.878495+0000","last_change":"2024-04-26T21:41:06.883834+0000","last_active":"2024-04-26T21:41:07.878495+0000","last_peered":"2024-04-26T21:41:07.878495+0000","last_clean":"2024-04-26T21:41:07.878495+0000","last_became_active":"2024-04-26T21:41:06.883555+0000","last_became_peered":"2024-04-26T21:41:06.883555+0000","last_unstale":"2024-04-26T21:41:07.878495+0000","last_undegraded":"2024-04-26T21:41:07.878495+0000","last_fullsized":"2024-04-26T21:41:07.878495+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:38.623818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909898+0000","last_change":"2024-04-26T21:41:07.905376+0000","last_active":"2024-04-26T21:41:07.909898+0000","last_peered":"2024-04-26T21:41:07.909898+0000","last_clean":"2024-04-26T21:41:07.909898+0000","last_became_active":"2024-04-26T21:41:07.904231+0000","last_became_peered":"2024-04-26T21:41:07.904231+0000","last_unstale":"2024-04-26T21:41:07.909898+0000","last_undegraded":"2024-04-26T21:41:07.909898+0000","last_fullsized":"2024-04-26T21:41:07.909898+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:52.530257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.883710+0000","last_change":"2024-04-26T21:41:06.888390+0000","last_active":"2024-04-26T21:41:07.883710+0000","last_peered":"2024-04-26T21:41:07.883710+0000","last_clean":"2024-04-26T21:41:07.883710+0000","last_became_active":"2024-04-26T21:41:06.888006+0000","last_became_peered":"2024-04-26T21:41:06.888006+0000","last_unstale":"2024-04-26T21:41:07.883710+0000","last_undegraded":"2024-04-26T21:41:07.883710+0000","last_fullsized":"2024-04-26T21:41:07.883710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:15.884490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906172+0000","last_change":"2024-04-26T21:41:07.897421+0000","last_active":"2024-04-26T21:41:07.906172+0000","last_peered":"2024-04-26T21:41:07.906172+0000","last_clean":"2024-04-26T21:41:07.906172+0000","last_became_active":"2024-04-26T21:41:07.897147+0000","last_became_peered":"2024-04-26T21:41:07.897147+0000","last_unstale":"2024-04-26T21:41:07.906172+0000","last_undegraded":"2024-04-26T21:41:07.906172+0000","last_fullsized":"2024-04-26T21:41:07.906172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:36.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.880067+0000","last_change":"2024-04-26T21:41:06.895985+0000","last_active":"2024-04-26T21:41:07.880067+0000","last_peered":"2024-04-26T21:41:07.880067+0000","last_clean":"2024-04-26T21:41:07.880067+0000","last_became_active":"2024-04-26T21:41:06.895706+0000","last_became_peered":"2024-04-26T21:41:06.895706+0000","last_unstale":"2024-04-26T21:41:07.880067+0000","last_undegraded":"2024-04-26T21:41:07.880067+0000","last_fullsized":"2024-04-26T21:41:07.880067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:48.405040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904636+0000","last_change":"2024-04-26T21:41:07.904006+0000","last_active":"2024-04-26T21:41:07.904636+0000","last_peered":"2024-04-26T21:41:07.904636+0000","last_clean":"2024-04-26T21:41:07.904636+0000","last_became_active":"2024-04-26T21:41:07.903700+0000","last_became_peered":"2024-04-26T21:41:07.903700+0000","last_unstale":"2024-04-26T21:41:07.904636+0000","last_undegraded":"2024-04-26T21:41:07.904636+0000","last_fullsized":"2024-04-26T21:41:07.904636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:34.997572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.884018+0000","last_change":"2024-04-26T21:41:06.890672+0000","last_active":"2024-04-26T21:41:07.884018+0000","last_peered":"2024-04-26T21:41:07.884018+0000","last_clean":"2024-04-26T21:41:07.884018+0000","last_became_active":"2024-04-26T21:41:06.890385+0000","last_became_peered":"2024-04-26T21:41:06.890385+0000","last_unstale":"2024-04-26T21:41:07.884018+0000","last_undegraded":"2024-04-26T21:41:07.884018+0000","last_fullsized":"2024-04-26T21:41:07.884018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:35.626916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":282,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.885111+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:41:07.885111+0000","last_peered":"2024-04-26T21:41:07.885111+0000","last_clean":"2024-04-26T21:41:07.885111+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:41:07.885111+0000","last_undegraded":"2024-04-26T21:41:07.885111+0000","last_fullsized":"2024-04-26T21:41:07.885111+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":1036176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574926,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":1036176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:41:11.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:41:12.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.090+0000 7f4108237640 1 Processor -- start 2024-04-26T21:41:12.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.090+0000 7f4108237640 1 -- start start 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.091+0000 7f4108237640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015f3a0 0x7f41000d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.091+0000 7f4108237640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41000d0e20 con 0x7f41000d2300 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.091+0000 7f4108237640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41000b1d70 con 0x7f41000d1770 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.091+0000 7f4108237640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41000b2050 con 0x7f410015f3a0 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.091+0000 7f41067d4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f41000d2300 0x7f41000b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46648/0 (socket says 172.21.15.40:46648) 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.091+0000 7f41067d4640 1 -- 172.21.15.40:0/1151908567 learned_addr learned my addr 172.21.15.40:0/1151908567 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4105fd3640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015f3a0 0x7f41000d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263130685 0 0) 0x7f41000d0e20 con 0x7f41000d2300 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40e8003520 con 0x7f41000d2300 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173151996 0 0) 0x7f41000b1d70 con 0x7f41000d1770 2024-04-26T21:41:12.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41000d0e20 con 0x7f41000d1770 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4094441433 0 0) 0x7f40e8003520 con 0x7f41000d2300 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015f3a0 msgr2=0x7f41000d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015f3a0 0x7f41000d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 >> v1:172.21.15.184:6789/0 conn(0x7f41000d1770 legacy=0x7f41000d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41000b2330 con 0x7f41000d2300 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f40f00019f0 con 0x7f41000d2300 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 >> v1:172.21.15.40:6789/0 conn(0x7f41000d2300 legacy=0x7f41000b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40e8003b70 0x7f40e8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015f3a0 0x7f40e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8004d40 0x7f40e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41000b2050 con 0x7f40e8003b70 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40e800ba70 con 0x7f40e8004d40 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40e800bc20 con 0x7f410015f3a0 2024-04-26T21:41:12.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4105fd3640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40e8003b70 0x7f40e8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f41057d2640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015f3a0 0x7f40e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f41067d4640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8004d40 0x7f40e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4105fd3640 1 -- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015f3a0 msgr2=0x7f40e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4105fd3640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015f3a0 0x7f40e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4105fd3640 1 -- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8004d40 msgr2=0x7f40e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4105fd3640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8004d40 0x7f40e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.092+0000 7f4105fd3640 1 -- 172.21.15.40:0/1151908567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e8003520 con 0x7f40e8003b70 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.093+0000 7f4105fd3640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40e8003b70 0x7f40e8003f70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009650 tx=0x7f40e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535b05d2ac4bc272 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.093+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8040070 con 0x7f40e8003b70 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.093+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f80336d0 con 0x7f40e8003b70 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.093+0000 7f4104fd1640 1 -- 172.21.15.40:0/1151908567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8009690 con 0x7f40e8003b70 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.094+0000 7f4108237640 1 -- 172.21.15.40:0/1151908567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40e8003b70 msgr2=0x7f40e8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.094+0000 7f4108237640 1 --2- 172.21.15.40:0/1151908567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40e8003b70 0x7f40e8003f70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009650 tx=0x7f40e8005830 comp rx=0 tx=0).stop 2024-04-26T21:41:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.094+0000 7f41067d4640 1 -- 172.21.15.40:0/1151908567 reap_dead start 2024-04-26T21:41:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.094+0000 7f4108237640 1 -- 172.21.15.40:0/1151908567 shutdown_connections 2024-04-26T21:41:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.094+0000 7f4108237640 1 -- 172.21.15.40:0/1151908567 >> 172.21.15.40:0/1151908567 conn(0x7f41000d4f90 msgr2=0x7f41000d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.094+0000 7f4108237640 1 -- 172.21.15.40:0/1151908567 shutdown_connections 2024-04-26T21:41:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.094+0000 7f4108237640 1 -- 172.21.15.40:0/1151908567 wait complete. 2024-04-26T21:41:12.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.095+0000 7f4108237640 1 Processor -- start 2024-04-26T21:41:12.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.095+0000 7f4108237640 1 -- start start 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f4108237640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8003b70 0x7f4100154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f4108237640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41000cb730 0x7f41001552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f4108237640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015faa0 0x7f410015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f4108237640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41000d0870 con 0x7f41000cb730 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f4108237640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41000d06f0 con 0x7f40e8003b70 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f4108237640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41000d09f0 con 0x7f410015faa0 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41000cb730 0x7f41001552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41000cb730 0x7f41001552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45190/0 (socket says 172.21.15.40:45190) 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 -- 172.21.15.40:0/277513748 learned_addr learned my addr 172.21.15.40:0/277513748 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f4105fd3640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8003b70 0x7f4100154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41067d4640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015faa0 0x7f410015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 -- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015faa0 msgr2=0x7f410015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015faa0 0x7f410015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 -- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8003b70 msgr2=0x7f4100154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8003b70 0x7f4100154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 -- 172.21.15.40:0/277513748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410015ad30 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41067d4640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015faa0 0x7f410015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.096+0000 7f41057d2640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41000cb730 0x7f41001552f0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f40f400e810 tx=0x7f40f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.097+0000 7f40fe7fc640 1 -- 172.21.15.40:0/277513748 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40052e0 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.097+0000 7f40fe7fc640 1 -- 172.21.15.40:0/277513748 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f4018070 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.097+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41001620a0 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.097+0000 7f40fe7fc640 1 -- 172.21.15.40:0/277513748 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f400c900 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.097+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4100162660 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.098+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41000694f0 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.098+0000 7f40fe7fc640 1 -- 172.21.15.40:0/277513748 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40f400caa0 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.099+0000 7f40fe7fc640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40ec06c920 0x7f40ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.099+0000 7f40fe7fc640 1 -- 172.21.15.40:0/277513748 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f40f408ba60 con 0x7f41000cb730 2024-04-26T21:41:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.099+0000 7f4105fd3640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40ec06c920 0x7f40ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.100+0000 7f4105fd3640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40ec06c920 0x7f40ec06ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f40f803d000 tx=0x7f40f8037000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.103+0000 7f40fe7fc640 1 -- 172.21.15.40:0/277513748 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40f4059ce0 con 0x7f41000cb730 2024-04-26T21:41:12.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.285+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41000d3490 con 0x7f40ec06c920 2024-04-26T21:41:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.292+0000 7f40fe7fc640 1 -- 172.21.15.40:0/277513748 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252002 (secure 0 0 0) 0x7f41000d3490 con 0x7f40ec06c920 2024-04-26T21:41:12.296 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40ec06c920 msgr2=0x7f40ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40ec06c920 0x7f40ec06ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f40f803d000 tx=0x7f40f8037000 comp rx=0 tx=0).stop 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41000cb730 msgr2=0x7f41001552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41000cb730 0x7f41001552f0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f40f400e810 tx=0x7f40f400ecd0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 shutdown_connections 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40ec06c920 0x7f40ec06ede0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f410015faa0 0x7f410015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41000cb730 0x7f41001552f0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 --2- 172.21.15.40:0/277513748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40e8003b70 0x7f4100154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 >> 172.21.15.40:0/277513748 conn(0x7f41000d4f90 msgr2=0x7f41000d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 shutdown_connections 2024-04-26T21:41:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.295+0000 7f4108237640 1 -- 172.21.15.40:0/277513748 wait complete. 2024-04-26T21:41:12.307 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-04-26T21:41:12.084721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233224,"kb_used_data":8712,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754741496,"statfs":{"total":773094113280,"available":772855291904,"internally_reserved":0,"allocated":8921088,"data_stored":4747200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001466"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898862+0000","last_change":"2024-04-26T21:41:07.893940+0000","last_active":"2024-04-26T21:41:07.898862+0000","last_peered":"2024-04-26T21:41:07.898862+0000","last_clean":"2024-04-26T21:41:07.898862+0000","last_became_active":"2024-04-26T21:41:07.893626+0000","last_became_peered":"2024-04-26T21:41:07.893626+0000","last_unstale":"2024-04-26T21:41:07.898862+0000","last_undegraded":"2024-04-26T21:41:07.898862+0000","last_fullsized":"2024-04-26T21:41:07.898862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:26.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912765+0000","last_change":"2024-04-26T21:41:07.903280+0000","last_active":"2024-04-26T21:41:07.912765+0000","last_peered":"2024-04-26T21:41:07.912765+0000","last_clean":"2024-04-26T21:41:07.912765+0000","last_became_active":"2024-04-26T21:41:07.902961+0000","last_became_peered":"2024-04-26T21:41:07.902961+0000","last_unstale":"2024-04-26T21:41:07.912765+0000","last_undegraded":"2024-04-26T21:41:07.912765+0000","last_fullsized":"2024-04-26T21:41:07.912765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:34.997572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906367+0000","last_change":"2024-04-26T21:41:07.895896+0000","last_active":"2024-04-26T21:41:07.906367+0000","last_peered":"2024-04-26T21:41:07.906367+0000","last_clean":"2024-04-26T21:41:07.906367+0000","last_became_active":"2024-04-26T21:41:07.895602+0000","last_became_peered":"2024-04-26T21:41:07.895602+0000","last_unstale":"2024-04-26T21:41:07.906367+0000","last_undegraded":"2024-04-26T21:41:07.906367+0000","last_fullsized":"2024-04-26T21:41:07.906367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.467106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.888111+0000","last_change":"2024-04-26T21:41:07.880310+0000","last_active":"2024-04-26T21:41:07.888111+0000","last_peered":"2024-04-26T21:41:07.888111+0000","last_clean":"2024-04-26T21:41:07.888111+0000","last_became_active":"2024-04-26T21:41:07.879984+0000","last_became_peered":"2024-04-26T21:41:07.879984+0000","last_unstale":"2024-04-26T21:41:07.888111+0000","last_undegraded":"2024-04-26T21:41:07.888111+0000","last_fullsized":"2024-04-26T21:41:07.888111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:48:21.988173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.890430+0000","last_change":"2024-04-26T21:41:07.888564+0000","last_active":"2024-04-26T21:41:07.890430+0000","last_peered":"2024-04-26T21:41:07.890430+0000","last_clean":"2024-04-26T21:41:07.890430+0000","last_became_active":"2024-04-26T21:41:07.888283+0000","last_became_peered":"2024-04-26T21:41:07.888283+0000","last_unstale":"2024-04-26T21:41:07.890430+0000","last_undegraded":"2024-04-26T21:41:07.890430+0000","last_fullsized":"2024-04-26T21:41:07.890430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:22.924079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906127+0000","last_change":"2024-04-26T21:41:07.899431+0000","last_active":"2024-04-26T21:41:07.906127+0000","last_peered":"2024-04-26T21:41:07.906127+0000","last_clean":"2024-04-26T21:41:07.906127+0000","last_became_active":"2024-04-26T21:41:07.898768+0000","last_became_peered":"2024-04-26T21:41:07.898768+0000","last_unstale":"2024-04-26T21:41:07.906127+0000","last_undegraded":"2024-04-26T21:41:07.906127+0000","last_fullsized":"2024-04-26T21:41:07.906127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:40.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904749+0000","last_change":"2024-04-26T21:41:07.897460+0000","last_active":"2024-04-26T21:41:07.904749+0000","last_peered":"2024-04-26T21:41:07.904749+0000","last_clean":"2024-04-26T21:41:07.904749+0000","last_became_active":"2024-04-26T21:41:07.897163+0000","last_became_peered":"2024-04-26T21:41:07.897163+0000","last_unstale":"2024-04-26T21:41:07.904749+0000","last_undegraded":"2024-04-26T21:41:07.904749+0000","last_fullsized":"2024-04-26T21:41:07.904749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:36.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905548+0000","last_change":"2024-04-26T21:41:07.895337+0000","last_active":"2024-04-26T21:41:07.905548+0000","last_peered":"2024-04-26T21:41:07.905548+0000","last_clean":"2024-04-26T21:41:07.905548+0000","last_became_active":"2024-04-26T21:41:07.895067+0000","last_became_peered":"2024-04-26T21:41:07.895067+0000","last_unstale":"2024-04-26T21:41:07.905548+0000","last_undegraded":"2024-04-26T21:41:07.905548+0000","last_fullsized":"2024-04-26T21:41:07.905548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:59.859051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.894114+0000","last_change":"2024-04-26T21:41:07.889838+0000","last_active":"2024-04-26T21:41:07.894114+0000","last_peered":"2024-04-26T21:41:07.894114+0000","last_clean":"2024-04-26T21:41:07.894114+0000","last_became_active":"2024-04-26T21:41:07.889513+0000","last_became_peered":"2024-04-26T21:41:07.889513+0000","last_unstale":"2024-04-26T21:41:07.894114+0000","last_undegraded":"2024-04-26T21:41:07.894114+0000","last_fullsized":"2024-04-26T21:41:07.894114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:32.940233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.890580+0000","last_change":"2024-04-26T21:41:07.888360+0000","last_active":"2024-04-26T21:41:07.890580+0000","last_peered":"2024-04-26T21:41:07.890580+0000","last_clean":"2024-04-26T21:41:07.890580+0000","last_became_active":"2024-04-26T21:41:07.888064+0000","last_became_peered":"2024-04-26T21:41:07.888064+0000","last_unstale":"2024-04-26T21:41:07.890580+0000","last_undegraded":"2024-04-26T21:41:07.890580+0000","last_fullsized":"2024-04-26T21:41:07.890580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:07.148493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.897613+0000","last_change":"2024-04-26T21:41:07.893932+0000","last_active":"2024-04-26T21:41:07.897613+0000","last_peered":"2024-04-26T21:41:07.897613+0000","last_clean":"2024-04-26T21:41:07.897613+0000","last_became_active":"2024-04-26T21:41:07.893647+0000","last_became_peered":"2024-04-26T21:41:07.893647+0000","last_unstale":"2024-04-26T21:41:07.897613+0000","last_undegraded":"2024-04-26T21:41:07.897613+0000","last_fullsized":"2024-04-26T21:41:07.897613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:30.597646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.900995+0000","last_change":"2024-04-26T21:41:07.900304+0000","last_active":"2024-04-26T21:41:07.900995+0000","last_peered":"2024-04-26T21:41:07.900995+0000","last_clean":"2024-04-26T21:41:07.900995+0000","last_became_active":"2024-04-26T21:41:07.900022+0000","last_became_peered":"2024-04-26T21:41:07.900022+0000","last_unstale":"2024-04-26T21:41:07.900995+0000","last_undegraded":"2024-04-26T21:41:07.900995+0000","last_fullsized":"2024-04-26T21:41:07.900995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:55.845178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905506+0000","last_change":"2024-04-26T21:41:07.896036+0000","last_active":"2024-04-26T21:41:07.905506+0000","last_peered":"2024-04-26T21:41:07.905506+0000","last_clean":"2024-04-26T21:41:07.905506+0000","last_became_active":"2024-04-26T21:41:07.895762+0000","last_became_peered":"2024-04-26T21:41:07.895762+0000","last_unstale":"2024-04-26T21:41:07.905506+0000","last_undegraded":"2024-04-26T21:41:07.905506+0000","last_fullsized":"2024-04-26T21:41:07.905506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:44.445020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912848+0000","last_change":"2024-04-26T21:41:07.907528+0000","last_active":"2024-04-26T21:41:07.912848+0000","last_peered":"2024-04-26T21:41:07.912848+0000","last_clean":"2024-04-26T21:41:07.912848+0000","last_became_active":"2024-04-26T21:41:07.898594+0000","last_became_peered":"2024-04-26T21:41:07.898594+0000","last_unstale":"2024-04-26T21:41:07.912848+0000","last_undegraded":"2024-04-26T21:41:07.912848+0000","last_fullsized":"2024-04-26T21:41:07.912848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:26.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909843+0000","last_change":"2024-04-26T21:41:07.901427+0000","last_active":"2024-04-26T21:41:07.909843+0000","last_peered":"2024-04-26T21:41:07.909843+0000","last_clean":"2024-04-26T21:41:07.909843+0000","last_became_active":"2024-04-26T21:41:07.901163+0000","last_became_peered":"2024-04-26T21:41:07.901163+0000","last_unstale":"2024-04-26T21:41:07.909843+0000","last_undegraded":"2024-04-26T21:41:07.909843+0000","last_fullsized":"2024-04-26T21:41:07.909843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:23.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.893001+0000","last_change":"2024-04-26T21:41:07.890727+0000","last_active":"2024-04-26T21:41:07.893001+0000","last_peered":"2024-04-26T21:41:07.893001+0000","last_clean":"2024-04-26T21:41:07.893001+0000","last_became_active":"2024-04-26T21:41:07.890444+0000","last_became_peered":"2024-04-26T21:41:07.890444+0000","last_unstale":"2024-04-26T21:41:07.893001+0000","last_undegraded":"2024-04-26T21:41:07.893001+0000","last_fullsized":"2024-04-26T21:41:07.893001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:40.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906534+0000","last_change":"2024-04-26T21:41:07.903999+0000","last_active":"2024-04-26T21:41:07.906534+0000","last_peered":"2024-04-26T21:41:07.906534+0000","last_clean":"2024-04-26T21:41:07.906534+0000","last_became_active":"2024-04-26T21:41:07.903730+0000","last_became_peered":"2024-04-26T21:41:07.903730+0000","last_unstale":"2024-04-26T21:41:07.906534+0000","last_undegraded":"2024-04-26T21:41:07.906534+0000","last_fullsized":"2024-04-26T21:41:07.906534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:14.064616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906231+0000","last_change":"2024-04-26T21:41:07.899122+0000","last_active":"2024-04-26T21:41:07.906231+0000","last_peered":"2024-04-26T21:41:07.906231+0000","last_clean":"2024-04-26T21:41:07.906231+0000","last_became_active":"2024-04-26T21:41:07.898849+0000","last_became_peered":"2024-04-26T21:41:07.898849+0000","last_unstale":"2024-04-26T21:41:07.906231+0000","last_undegraded":"2024-04-26T21:41:07.906231+0000","last_fullsized":"2024-04-26T21:41:07.906231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:54.176753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.896751+0000","last_change":"2024-04-26T21:41:07.891901+0000","last_active":"2024-04-26T21:41:07.896751+0000","last_peered":"2024-04-26T21:41:07.896751+0000","last_clean":"2024-04-26T21:41:07.896751+0000","last_became_active":"2024-04-26T21:41:07.890635+0000","last_became_peered":"2024-04-26T21:41:07.890635+0000","last_unstale":"2024-04-26T21:41:07.896751+0000","last_undegraded":"2024-04-26T21:41:07.896751+0000","last_fullsized":"2024-04-26T21:41:07.896751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:30.597646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.908858+0000","last_change":"2024-04-26T21:41:07.905483+0000","last_active":"2024-04-26T21:41:07.908858+0000","last_peered":"2024-04-26T21:41:07.908858+0000","last_clean":"2024-04-26T21:41:07.908858+0000","last_became_active":"2024-04-26T21:41:07.905185+0000","last_became_peered":"2024-04-26T21:41:07.905185+0000","last_unstale":"2024-04-26T21:41:07.908858+0000","last_undegraded":"2024-04-26T21:41:07.908858+0000","last_fullsized":"2024-04-26T21:41:07.908858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:59.771036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909234+0000","last_change":"2024-04-26T21:41:07.900597+0000","last_active":"2024-04-26T21:41:07.909234+0000","last_peered":"2024-04-26T21:41:07.909234+0000","last_clean":"2024-04-26T21:41:07.909234+0000","last_became_active":"2024-04-26T21:41:07.900305+0000","last_became_peered":"2024-04-26T21:41:07.900305+0000","last_unstale":"2024-04-26T21:41:07.909234+0000","last_undegraded":"2024-04-26T21:41:07.909234+0000","last_fullsized":"2024-04-26T21:41:07.909234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:22.576309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905484+0000","last_change":"2024-04-26T21:41:07.896770+0000","last_active":"2024-04-26T21:41:07.905484+0000","last_peered":"2024-04-26T21:41:07.905484+0000","last_clean":"2024-04-26T21:41:07.905484+0000","last_became_active":"2024-04-26T21:41:07.896494+0000","last_became_peered":"2024-04-26T21:41:07.896494+0000","last_unstale":"2024-04-26T21:41:07.905484+0000","last_undegraded":"2024-04-26T21:41:07.905484+0000","last_fullsized":"2024-04-26T21:41:07.905484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:32.487421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.893680+0000","last_change":"2024-04-26T21:41:07.888426+0000","last_active":"2024-04-26T21:41:07.893680+0000","last_peered":"2024-04-26T21:41:07.893680+0000","last_clean":"2024-04-26T21:41:07.893680+0000","last_became_active":"2024-04-26T21:41:07.887149+0000","last_became_peered":"2024-04-26T21:41:07.887149+0000","last_unstale":"2024-04-26T21:41:07.893680+0000","last_undegraded":"2024-04-26T21:41:07.893680+0000","last_fullsized":"2024-04-26T21:41:07.893680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:52.053762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912862+0000","last_change":"2024-04-26T21:41:07.904976+0000","last_active":"2024-04-26T21:41:07.912862+0000","last_peered":"2024-04-26T21:41:07.912862+0000","last_clean":"2024-04-26T21:41:07.912862+0000","last_became_active":"2024-04-26T21:41:07.904477+0000","last_became_peered":"2024-04-26T21:41:07.904477+0000","last_unstale":"2024-04-26T21:41:07.912862+0000","last_undegraded":"2024-04-26T21:41:07.912862+0000","last_fullsized":"2024-04-26T21:41:07.912862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:41.740959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912886+0000","last_change":"2024-04-26T21:41:07.907095+0000","last_active":"2024-04-26T21:41:07.912886+0000","last_peered":"2024-04-26T21:41:07.912886+0000","last_clean":"2024-04-26T21:41:07.912886+0000","last_became_active":"2024-04-26T21:41:07.906812+0000","last_became_peered":"2024-04-26T21:41:07.906812+0000","last_unstale":"2024-04-26T21:41:07.912886+0000","last_undegraded":"2024-04-26T21:41:07.912886+0000","last_fullsized":"2024-04-26T21:41:07.912886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:30:04.598780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.911946+0000","last_change":"2024-04-26T21:41:07.903788+0000","last_active":"2024-04-26T21:41:07.911946+0000","last_peered":"2024-04-26T21:41:07.911946+0000","last_clean":"2024-04-26T21:41:07.911946+0000","last_became_active":"2024-04-26T21:41:07.903491+0000","last_became_peered":"2024-04-26T21:41:07.903491+0000","last_unstale":"2024-04-26T21:41:07.911946+0000","last_undegraded":"2024-04-26T21:41:07.911946+0000","last_fullsized":"2024-04-26T21:41:07.911946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:36.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.891501+0000","last_change":"2024-04-26T21:41:07.889123+0000","last_active":"2024-04-26T21:41:07.891501+0000","last_peered":"2024-04-26T21:41:07.891501+0000","last_clean":"2024-04-26T21:41:07.891501+0000","last_became_active":"2024-04-26T21:41:07.888852+0000","last_became_peered":"2024-04-26T21:41:07.888852+0000","last_unstale":"2024-04-26T21:41:07.891501+0000","last_undegraded":"2024-04-26T21:41:07.891501+0000","last_fullsized":"2024-04-26T21:41:07.891501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:13.308104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.887858+0000","last_change":"2024-04-26T21:41:07.881706+0000","last_active":"2024-04-26T21:41:07.887858+0000","last_peered":"2024-04-26T21:41:07.887858+0000","last_clean":"2024-04-26T21:41:07.887858+0000","last_became_active":"2024-04-26T21:41:07.881329+0000","last_became_peered":"2024-04-26T21:41:07.881329+0000","last_unstale":"2024-04-26T21:41:07.887858+0000","last_undegraded":"2024-04-26T21:41:07.887858+0000","last_fullsized":"2024-04-26T21:41:07.887858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:36.676505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.891546+0000","last_change":"2024-04-26T21:41:07.889756+0000","last_active":"2024-04-26T21:41:07.891546+0000","last_peered":"2024-04-26T21:41:07.891546+0000","last_clean":"2024-04-26T21:41:07.891546+0000","last_became_active":"2024-04-26T21:41:07.889386+0000","last_became_peered":"2024-04-26T21:41:07.889386+0000","last_unstale":"2024-04-26T21:41:07.891546+0000","last_undegraded":"2024-04-26T21:41:07.891546+0000","last_fullsized":"2024-04-26T21:41:07.891546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:55.845178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898045+0000","last_change":"2024-04-26T21:41:07.893163+0000","last_active":"2024-04-26T21:41:07.898045+0000","last_peered":"2024-04-26T21:41:07.898045+0000","last_clean":"2024-04-26T21:41:07.898045+0000","last_became_active":"2024-04-26T21:41:07.892874+0000","last_became_peered":"2024-04-26T21:41:07.892874+0000","last_unstale":"2024-04-26T21:41:07.898045+0000","last_undegraded":"2024-04-26T21:41:07.898045+0000","last_fullsized":"2024-04-26T21:41:07.898045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.679698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906250+0000","last_change":"2024-04-26T21:41:07.895128+0000","last_active":"2024-04-26T21:41:07.906250+0000","last_peered":"2024-04-26T21:41:07.906250+0000","last_clean":"2024-04-26T21:41:07.906250+0000","last_became_active":"2024-04-26T21:41:07.894830+0000","last_became_peered":"2024-04-26T21:41:07.894830+0000","last_unstale":"2024-04-26T21:41:07.906250+0000","last_undegraded":"2024-04-26T21:41:07.906250+0000","last_fullsized":"2024-04-26T21:41:07.906250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:25.447410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898339+0000","last_change":"2024-04-26T21:41:07.893986+0000","last_active":"2024-04-26T21:41:07.898339+0000","last_peered":"2024-04-26T21:41:07.898339+0000","last_clean":"2024-04-26T21:41:07.898339+0000","last_became_active":"2024-04-26T21:41:07.893685+0000","last_became_peered":"2024-04-26T21:41:07.893685+0000","last_unstale":"2024-04-26T21:41:07.898339+0000","last_undegraded":"2024-04-26T21:41:07.898339+0000","last_fullsized":"2024-04-26T21:41:07.898339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:23.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.902723+0000","last_change":"2024-04-26T21:41:07.894071+0000","last_active":"2024-04-26T21:41:07.902723+0000","last_peered":"2024-04-26T21:41:07.902723+0000","last_clean":"2024-04-26T21:41:07.902723+0000","last_became_active":"2024-04-26T21:41:07.893807+0000","last_became_peered":"2024-04-26T21:41:07.893807+0000","last_unstale":"2024-04-26T21:41:07.902723+0000","last_undegraded":"2024-04-26T21:41:07.902723+0000","last_fullsized":"2024-04-26T21:41:07.902723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:03.104758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.908707+0000","last_change":"2024-04-26T21:41:06.883010+0000","last_active":"2024-04-26T21:41:07.908707+0000","last_peered":"2024-04-26T21:41:07.908707+0000","last_clean":"2024-04-26T21:41:07.908707+0000","last_became_active":"2024-04-26T21:41:06.882711+0000","last_became_peered":"2024-04-26T21:41:06.882711+0000","last_unstale":"2024-04-26T21:41:07.908707+0000","last_undegraded":"2024-04-26T21:41:07.908707+0000","last_fullsized":"2024-04-26T21:41:07.908707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:36.058840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904286+0000","last_change":"2024-04-26T21:41:07.900864+0000","last_active":"2024-04-26T21:41:07.904286+0000","last_peered":"2024-04-26T21:41:07.904286+0000","last_clean":"2024-04-26T21:41:07.904286+0000","last_became_active":"2024-04-26T21:41:07.900588+0000","last_became_peered":"2024-04-26T21:41:07.900588+0000","last_unstale":"2024-04-26T21:41:07.904286+0000","last_undegraded":"2024-04-26T21:41:07.904286+0000","last_fullsized":"2024-04-26T21:41:07.904286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:26.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.894996+0000","last_change":"2024-04-26T21:41:06.889946+0000","last_active":"2024-04-26T21:41:07.894996+0000","last_peered":"2024-04-26T21:41:07.894996+0000","last_clean":"2024-04-26T21:41:07.894996+0000","last_became_active":"2024-04-26T21:41:06.889593+0000","last_became_peered":"2024-04-26T21:41:06.889593+0000","last_unstale":"2024-04-26T21:41:07.894996+0000","last_undegraded":"2024-04-26T21:41:07.894996+0000","last_fullsized":"2024-04-26T21:41:07.894996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:06.146449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898698+0000","last_change":"2024-04-26T21:41:07.894766+0000","last_active":"2024-04-26T21:41:07.898698+0000","last_peered":"2024-04-26T21:41:07.898698+0000","last_clean":"2024-04-26T21:41:07.898698+0000","last_became_active":"2024-04-26T21:41:07.894489+0000","last_became_peered":"2024-04-26T21:41:07.894489+0000","last_unstale":"2024-04-26T21:41:07.898698+0000","last_undegraded":"2024-04-26T21:41:07.898698+0000","last_fullsized":"2024-04-26T21:41:07.898698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:45.308669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895848+0000","last_change":"2024-04-26T21:41:06.889395+0000","last_active":"2024-04-26T21:41:07.895848+0000","last_peered":"2024-04-26T21:41:07.895848+0000","last_clean":"2024-04-26T21:41:07.895848+0000","last_became_active":"2024-04-26T21:41:06.889099+0000","last_became_peered":"2024-04-26T21:41:06.889099+0000","last_unstale":"2024-04-26T21:41:07.895848+0000","last_undegraded":"2024-04-26T21:41:07.895848+0000","last_fullsized":"2024-04-26T21:41:07.895848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:12.480105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909301+0000","last_change":"2024-04-26T21:41:07.900780+0000","last_active":"2024-04-26T21:41:07.909301+0000","last_peered":"2024-04-26T21:41:07.909301+0000","last_clean":"2024-04-26T21:41:07.909301+0000","last_became_active":"2024-04-26T21:41:07.900494+0000","last_became_peered":"2024-04-26T21:41:07.900494+0000","last_unstale":"2024-04-26T21:41:07.909301+0000","last_undegraded":"2024-04-26T21:41:07.909301+0000","last_fullsized":"2024-04-26T21:41:07.909301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:00.438630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895737+0000","last_change":"2024-04-26T21:41:06.888763+0000","last_active":"2024-04-26T21:41:07.895737+0000","last_peered":"2024-04-26T21:41:07.895737+0000","last_clean":"2024-04-26T21:41:07.895737+0000","last_became_active":"2024-04-26T21:41:06.888161+0000","last_became_peered":"2024-04-26T21:41:06.888161+0000","last_unstale":"2024-04-26T21:41:07.895737+0000","last_undegraded":"2024-04-26T21:41:07.895737+0000","last_fullsized":"2024-04-26T21:41:07.895737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:24.383801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905605+0000","last_change":"2024-04-26T21:41:07.900799+0000","last_active":"2024-04-26T21:41:07.905605+0000","last_peered":"2024-04-26T21:41:07.905605+0000","last_clean":"2024-04-26T21:41:07.905605+0000","last_became_active":"2024-04-26T21:41:07.900251+0000","last_became_peered":"2024-04-26T21:41:07.900251+0000","last_unstale":"2024-04-26T21:41:07.905605+0000","last_undegraded":"2024-04-26T21:41:07.905605+0000","last_fullsized":"2024-04-26T21:41:07.905605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.679698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898868+0000","last_change":"2024-04-26T21:41:06.885609+0000","last_active":"2024-04-26T21:41:07.898868+0000","last_peered":"2024-04-26T21:41:07.898868+0000","last_clean":"2024-04-26T21:41:07.898868+0000","last_became_active":"2024-04-26T21:41:06.885339+0000","last_became_peered":"2024-04-26T21:41:06.885339+0000","last_unstale":"2024-04-26T21:41:07.898868+0000","last_undegraded":"2024-04-26T21:41:07.898868+0000","last_fullsized":"2024-04-26T21:41:07.898868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:25:35.748907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.897638+0000","last_change":"2024-04-26T21:41:07.893359+0000","last_active":"2024-04-26T21:41:07.897638+0000","last_peered":"2024-04-26T21:41:07.897638+0000","last_clean":"2024-04-26T21:41:07.897638+0000","last_became_active":"2024-04-26T21:41:07.893061+0000","last_became_peered":"2024-04-26T21:41:07.893061+0000","last_unstale":"2024-04-26T21:41:07.897638+0000","last_undegraded":"2024-04-26T21:41:07.897638+0000","last_fullsized":"2024-04-26T21:41:07.897638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.159772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.907524+0000","last_change":"2024-04-26T21:41:06.883727+0000","last_active":"2024-04-26T21:41:07.907524+0000","last_peered":"2024-04-26T21:41:07.907524+0000","last_clean":"2024-04-26T21:41:07.907524+0000","last_became_active":"2024-04-26T21:41:06.883476+0000","last_became_peered":"2024-04-26T21:41:06.883476+0000","last_unstale":"2024-04-26T21:41:07.907524+0000","last_undegraded":"2024-04-26T21:41:07.907524+0000","last_fullsized":"2024-04-26T21:41:07.907524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:32.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.898860+0000","last_change":"2024-04-26T21:41:07.895759+0000","last_active":"2024-04-26T21:41:07.898860+0000","last_peered":"2024-04-26T21:41:07.898860+0000","last_clean":"2024-04-26T21:41:07.898860+0000","last_became_active":"2024-04-26T21:41:07.894370+0000","last_became_peered":"2024-04-26T21:41:07.894370+0000","last_unstale":"2024-04-26T21:41:07.898860+0000","last_undegraded":"2024-04-26T21:41:07.898860+0000","last_fullsized":"2024-04-26T21:41:07.898860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:12.514077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.887817+0000","last_change":"2024-04-26T21:41:06.886390+0000","last_active":"2024-04-26T21:41:07.887817+0000","last_peered":"2024-04-26T21:41:07.887817+0000","last_clean":"2024-04-26T21:41:07.887817+0000","last_became_active":"2024-04-26T21:41:06.885992+0000","last_became_peered":"2024-04-26T21:41:06.885992+0000","last_unstale":"2024-04-26T21:41:07.887817+0000","last_undegraded":"2024-04-26T21:41:07.887817+0000","last_fullsized":"2024-04-26T21:41:07.887817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:06.146449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904676+0000","last_change":"2024-04-26T21:41:07.897377+0000","last_active":"2024-04-26T21:41:07.904676+0000","last_peered":"2024-04-26T21:41:07.904676+0000","last_clean":"2024-04-26T21:41:07.904676+0000","last_became_active":"2024-04-26T21:41:07.897046+0000","last_became_peered":"2024-04-26T21:41:07.897046+0000","last_unstale":"2024-04-26T21:41:07.904676+0000","last_undegraded":"2024-04-26T21:41:07.904676+0000","last_fullsized":"2024-04-26T21:41:07.904676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:22.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.875479+0000","last_change":"2024-04-26T21:41:06.880632+0000","last_active":"2024-04-26T21:41:07.875479+0000","last_peered":"2024-04-26T21:41:07.875479+0000","last_clean":"2024-04-26T21:41:07.875479+0000","last_became_active":"2024-04-26T21:41:06.880182+0000","last_became_peered":"2024-04-26T21:41:06.880182+0000","last_unstale":"2024-04-26T21:41:07.875479+0000","last_undegraded":"2024-04-26T21:41:07.875479+0000","last_fullsized":"2024-04-26T21:41:07.875479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:43.697586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.897721+0000","last_change":"2024-04-26T21:41:07.895956+0000","last_active":"2024-04-26T21:41:07.897721+0000","last_peered":"2024-04-26T21:41:07.897721+0000","last_clean":"2024-04-26T21:41:07.897721+0000","last_became_active":"2024-04-26T21:41:07.895688+0000","last_became_peered":"2024-04-26T21:41:07.895688+0000","last_unstale":"2024-04-26T21:41:07.897721+0000","last_undegraded":"2024-04-26T21:41:07.897721+0000","last_fullsized":"2024-04-26T21:41:07.897721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:13.308104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.882577+0000","last_change":"2024-04-26T21:41:06.890600+0000","last_active":"2024-04-26T21:41:07.882577+0000","last_peered":"2024-04-26T21:41:07.882577+0000","last_clean":"2024-04-26T21:41:07.882577+0000","last_became_active":"2024-04-26T21:41:06.890215+0000","last_became_peered":"2024-04-26T21:41:06.890215+0000","last_unstale":"2024-04-26T21:41:07.882577+0000","last_undegraded":"2024-04-26T21:41:07.882577+0000","last_fullsized":"2024-04-26T21:41:07.882577+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:54.843135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.888145+0000","last_change":"2024-04-26T21:41:07.881399+0000","last_active":"2024-04-26T21:41:07.888145+0000","last_peered":"2024-04-26T21:41:07.888145+0000","last_clean":"2024-04-26T21:41:07.888145+0000","last_became_active":"2024-04-26T21:41:07.881048+0000","last_became_peered":"2024-04-26T21:41:07.881048+0000","last_unstale":"2024-04-26T21:41:07.888145+0000","last_undegraded":"2024-04-26T21:41:07.888145+0000","last_fullsized":"2024-04-26T21:41:07.888145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.468687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.877723+0000","last_change":"2024-04-26T21:41:06.885244+0000","last_active":"2024-04-26T21:41:07.877723+0000","last_peered":"2024-04-26T21:41:07.877723+0000","last_clean":"2024-04-26T21:41:07.877723+0000","last_became_active":"2024-04-26T21:41:06.884976+0000","last_became_peered":"2024-04-26T21:41:06.884976+0000","last_unstale":"2024-04-26T21:41:07.877723+0000","last_undegraded":"2024-04-26T21:41:07.877723+0000","last_fullsized":"2024-04-26T21:41:07.877723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:14.245124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906079+0000","last_change":"2024-04-26T21:41:07.895869+0000","last_active":"2024-04-26T21:41:07.906079+0000","last_peered":"2024-04-26T21:41:07.906079+0000","last_clean":"2024-04-26T21:41:07.906079+0000","last_became_active":"2024-04-26T21:41:07.895592+0000","last_became_peered":"2024-04-26T21:41:07.895592+0000","last_unstale":"2024-04-26T21:41:07.906079+0000","last_undegraded":"2024-04-26T21:41:07.906079+0000","last_fullsized":"2024-04-26T21:41:07.906079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:13.308104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.910071+0000","last_change":"2024-04-26T21:41:06.892212+0000","last_active":"2024-04-26T21:41:07.910071+0000","last_peered":"2024-04-26T21:41:07.910071+0000","last_clean":"2024-04-26T21:41:07.910071+0000","last_became_active":"2024-04-26T21:41:06.891908+0000","last_became_peered":"2024-04-26T21:41:06.891908+0000","last_unstale":"2024-04-26T21:41:07.910071+0000","last_undegraded":"2024-04-26T21:41:07.910071+0000","last_fullsized":"2024-04-26T21:41:07.910071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:43.131495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906331+0000","last_change":"2024-04-26T21:41:07.896657+0000","last_active":"2024-04-26T21:41:07.906331+0000","last_peered":"2024-04-26T21:41:07.906331+0000","last_clean":"2024-04-26T21:41:07.906331+0000","last_became_active":"2024-04-26T21:41:07.896309+0000","last_became_peered":"2024-04-26T21:41:07.896309+0000","last_unstale":"2024-04-26T21:41:07.906331+0000","last_undegraded":"2024-04-26T21:41:07.906331+0000","last_fullsized":"2024-04-26T21:41:07.906331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:22.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"80'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912244+0000","last_change":"2024-04-26T21:41:06.883956+0000","last_active":"2024-04-26T21:41:07.912244+0000","last_peered":"2024-04-26T21:41:07.912244+0000","last_clean":"2024-04-26T21:41:07.912244+0000","last_became_active":"2024-04-26T21:41:06.883451+0000","last_became_peered":"2024-04-26T21:41:06.883451+0000","last_unstale":"2024-04-26T21:41:07.912244+0000","last_undegraded":"2024-04-26T21:41:07.912244+0000","last_fullsized":"2024-04-26T21:41:07.912244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:19.512895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895442+0000","last_change":"2024-04-26T21:41:07.893429+0000","last_active":"2024-04-26T21:41:07.895442+0000","last_peered":"2024-04-26T21:41:07.895442+0000","last_clean":"2024-04-26T21:41:07.895442+0000","last_became_active":"2024-04-26T21:41:07.893108+0000","last_became_peered":"2024-04-26T21:41:07.893108+0000","last_unstale":"2024-04-26T21:41:07.895442+0000","last_undegraded":"2024-04-26T21:41:07.895442+0000","last_fullsized":"2024-04-26T21:41:07.895442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:13.482148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909520+0000","last_change":"2024-04-26T21:41:06.880133+0000","last_active":"2024-04-26T21:41:07.909520+0000","last_peered":"2024-04-26T21:41:07.909520+0000","last_clean":"2024-04-26T21:41:07.909520+0000","last_became_active":"2024-04-26T21:41:06.879862+0000","last_became_peered":"2024-04-26T21:41:06.879862+0000","last_unstale":"2024-04-26T21:41:07.909520+0000","last_undegraded":"2024-04-26T21:41:07.909520+0000","last_fullsized":"2024-04-26T21:41:07.909520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:38.401980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904760+0000","last_change":"2024-04-26T21:41:07.898592+0000","last_active":"2024-04-26T21:41:07.904760+0000","last_peered":"2024-04-26T21:41:07.904760+0000","last_clean":"2024-04-26T21:41:07.904760+0000","last_became_active":"2024-04-26T21:41:07.898294+0000","last_became_peered":"2024-04-26T21:41:07.898294+0000","last_unstale":"2024-04-26T21:41:07.904760+0000","last_undegraded":"2024-04-26T21:41:07.904760+0000","last_fullsized":"2024-04-26T21:41:07.904760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:54.176753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.882691+0000","last_change":"2024-04-26T21:41:06.892424+0000","last_active":"2024-04-26T21:41:07.882691+0000","last_peered":"2024-04-26T21:41:07.882691+0000","last_clean":"2024-04-26T21:41:07.882691+0000","last_became_active":"2024-04-26T21:41:06.892162+0000","last_became_peered":"2024-04-26T21:41:06.892162+0000","last_unstale":"2024-04-26T21:41:07.882691+0000","last_undegraded":"2024-04-26T21:41:07.882691+0000","last_fullsized":"2024-04-26T21:41:07.882691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:35.626916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904813+0000","last_change":"2024-04-26T21:41:07.899834+0000","last_active":"2024-04-26T21:41:07.904813+0000","last_peered":"2024-04-26T21:41:07.904813+0000","last_clean":"2024-04-26T21:41:07.904813+0000","last_became_active":"2024-04-26T21:41:07.899539+0000","last_became_peered":"2024-04-26T21:41:07.899539+0000","last_unstale":"2024-04-26T21:41:07.904813+0000","last_undegraded":"2024-04-26T21:41:07.904813+0000","last_fullsized":"2024-04-26T21:41:07.904813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:52.530257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.875899+0000","last_change":"2024-04-26T21:41:06.889071+0000","last_active":"2024-04-26T21:41:07.875899+0000","last_peered":"2024-04-26T21:41:07.875899+0000","last_clean":"2024-04-26T21:41:07.875899+0000","last_became_active":"2024-04-26T21:41:06.888585+0000","last_became_peered":"2024-04-26T21:41:06.888585+0000","last_unstale":"2024-04-26T21:41:07.875899+0000","last_undegraded":"2024-04-26T21:41:07.875899+0000","last_fullsized":"2024-04-26T21:41:07.875899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:05.934276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895144+0000","last_change":"2024-04-26T21:41:07.893997+0000","last_active":"2024-04-26T21:41:07.895144+0000","last_peered":"2024-04-26T21:41:07.895144+0000","last_clean":"2024-04-26T21:41:07.895144+0000","last_became_active":"2024-04-26T21:41:07.893712+0000","last_became_peered":"2024-04-26T21:41:07.893712+0000","last_unstale":"2024-04-26T21:41:07.895144+0000","last_undegraded":"2024-04-26T21:41:07.895144+0000","last_fullsized":"2024-04-26T21:41:07.895144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:33.183752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.908457+0000","last_change":"2024-04-26T21:41:06.890341+0000","last_active":"2024-04-26T21:41:07.908457+0000","last_peered":"2024-04-26T21:41:07.908457+0000","last_clean":"2024-04-26T21:41:07.908457+0000","last_became_active":"2024-04-26T21:41:06.889977+0000","last_became_peered":"2024-04-26T21:41:06.889977+0000","last_unstale":"2024-04-26T21:41:07.908457+0000","last_undegraded":"2024-04-26T21:41:07.908457+0000","last_fullsized":"2024-04-26T21:41:07.908457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:12.480105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.907326+0000","last_change":"2024-04-26T21:41:07.902084+0000","last_active":"2024-04-26T21:41:07.907326+0000","last_peered":"2024-04-26T21:41:07.907326+0000","last_clean":"2024-04-26T21:41:07.907326+0000","last_became_active":"2024-04-26T21:41:07.901810+0000","last_became_peered":"2024-04-26T21:41:07.901810+0000","last_unstale":"2024-04-26T21:41:07.907326+0000","last_undegraded":"2024-04-26T21:41:07.907326+0000","last_fullsized":"2024-04-26T21:41:07.907326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:08.679698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.903754+0000","last_change":"2024-04-26T21:41:06.893855+0000","last_active":"2024-04-26T21:41:07.903754+0000","last_peered":"2024-04-26T21:41:07.903754+0000","last_clean":"2024-04-26T21:41:07.903754+0000","last_became_active":"2024-04-26T21:41:06.893560+0000","last_became_peered":"2024-04-26T21:41:06.893560+0000","last_unstale":"2024-04-26T21:41:07.903754+0000","last_undegraded":"2024-04-26T21:41:07.903754+0000","last_fullsized":"2024-04-26T21:41:07.903754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:29.595603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906338+0000","last_change":"2024-04-26T21:41:07.899729+0000","last_active":"2024-04-26T21:41:07.906338+0000","last_peered":"2024-04-26T21:41:07.906338+0000","last_clean":"2024-04-26T21:41:07.906338+0000","last_became_active":"2024-04-26T21:41:07.898923+0000","last_became_peered":"2024-04-26T21:41:07.898923+0000","last_unstale":"2024-04-26T21:41:07.906338+0000","last_undegraded":"2024-04-26T21:41:07.906338+0000","last_fullsized":"2024-04-26T21:41:07.906338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:44.804002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906899+0000","last_change":"2024-04-26T21:41:06.893526+0000","last_active":"2024-04-26T21:41:07.906899+0000","last_peered":"2024-04-26T21:41:07.906899+0000","last_clean":"2024-04-26T21:41:07.906899+0000","last_became_active":"2024-04-26T21:41:06.893251+0000","last_became_peered":"2024-04-26T21:41:06.893251+0000","last_unstale":"2024-04-26T21:41:07.906899+0000","last_undegraded":"2024-04-26T21:41:07.906899+0000","last_fullsized":"2024-04-26T21:41:07.906899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:57.452521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.897763+0000","last_change":"2024-04-26T21:41:07.894000+0000","last_active":"2024-04-26T21:41:07.897763+0000","last_peered":"2024-04-26T21:41:07.897763+0000","last_clean":"2024-04-26T21:41:07.897763+0000","last_became_active":"2024-04-26T21:41:07.893723+0000","last_became_peered":"2024-04-26T21:41:07.893723+0000","last_unstale":"2024-04-26T21:41:07.897763+0000","last_undegraded":"2024-04-26T21:41:07.897763+0000","last_fullsized":"2024-04-26T21:41:07.897763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:09.889519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.900587+0000","last_change":"2024-04-26T21:41:06.879741+0000","last_active":"2024-04-26T21:41:07.900587+0000","last_peered":"2024-04-26T21:41:07.900587+0000","last_clean":"2024-04-26T21:41:07.900587+0000","last_became_active":"2024-04-26T21:41:06.879456+0000","last_became_peered":"2024-04-26T21:41:06.879456+0000","last_unstale":"2024-04-26T21:41:07.900587+0000","last_undegraded":"2024-04-26T21:41:07.900587+0000","last_fullsized":"2024-04-26T21:41:07.900587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:55.528540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.896707+0000","last_change":"2024-04-26T21:41:07.892055+0000","last_active":"2024-04-26T21:41:07.896707+0000","last_peered":"2024-04-26T21:41:07.896707+0000","last_clean":"2024-04-26T21:41:07.896707+0000","last_became_active":"2024-04-26T21:41:07.891744+0000","last_became_peered":"2024-04-26T21:41:07.891744+0000","last_unstale":"2024-04-26T21:41:07.896707+0000","last_undegraded":"2024-04-26T21:41:07.896707+0000","last_fullsized":"2024-04-26T21:41:07.896707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.159772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.882664+0000","last_change":"2024-04-26T21:41:06.892445+0000","last_active":"2024-04-26T21:41:07.882664+0000","last_peered":"2024-04-26T21:41:07.882664+0000","last_clean":"2024-04-26T21:41:07.882664+0000","last_became_active":"2024-04-26T21:41:06.892151+0000","last_became_peered":"2024-04-26T21:41:06.892151+0000","last_unstale":"2024-04-26T21:41:07.882664+0000","last_undegraded":"2024-04-26T21:41:07.882664+0000","last_fullsized":"2024-04-26T21:41:07.882664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:32.181709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.905582+0000","last_change":"2024-04-26T21:41:07.900570+0000","last_active":"2024-04-26T21:41:07.905582+0000","last_peered":"2024-04-26T21:41:07.905582+0000","last_clean":"2024-04-26T21:41:07.905582+0000","last_became_active":"2024-04-26T21:41:07.899894+0000","last_became_peered":"2024-04-26T21:41:07.899894+0000","last_unstale":"2024-04-26T21:41:07.905582+0000","last_undegraded":"2024-04-26T21:41:07.905582+0000","last_fullsized":"2024-04-26T21:41:07.905582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:40.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.910504+0000","last_change":"2024-04-26T21:41:06.892469+0000","last_active":"2024-04-26T21:41:07.910504+0000","last_peered":"2024-04-26T21:41:07.910504+0000","last_clean":"2024-04-26T21:41:07.910504+0000","last_became_active":"2024-04-26T21:41:06.892155+0000","last_became_peered":"2024-04-26T21:41:06.892155+0000","last_unstale":"2024-04-26T21:41:07.910504+0000","last_undegraded":"2024-04-26T21:41:07.910504+0000","last_fullsized":"2024-04-26T21:41:07.910504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:06:06.146449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.912793+0000","last_change":"2024-04-26T21:41:07.903469+0000","last_active":"2024-04-26T21:41:07.912793+0000","last_peered":"2024-04-26T21:41:07.912793+0000","last_clean":"2024-04-26T21:41:07.912793+0000","last_became_active":"2024-04-26T21:41:07.903179+0000","last_became_peered":"2024-04-26T21:41:07.903179+0000","last_unstale":"2024-04-26T21:41:07.912793+0000","last_undegraded":"2024-04-26T21:41:07.912793+0000","last_fullsized":"2024-04-26T21:41:07.912793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.196025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.883563+0000","last_change":"2024-04-26T21:41:06.892386+0000","last_active":"2024-04-26T21:41:07.883563+0000","last_peered":"2024-04-26T21:41:07.883563+0000","last_clean":"2024-04-26T21:41:07.883563+0000","last_became_active":"2024-04-26T21:41:06.890974+0000","last_became_peered":"2024-04-26T21:41:06.890974+0000","last_unstale":"2024-04-26T21:41:07.883563+0000","last_undegraded":"2024-04-26T21:41:07.883563+0000","last_fullsized":"2024-04-26T21:41:07.883563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:12.480105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.901416+0000","last_change":"2024-04-26T21:41:07.895083+0000","last_active":"2024-04-26T21:41:07.901416+0000","last_peered":"2024-04-26T21:41:07.901416+0000","last_clean":"2024-04-26T21:41:07.901416+0000","last_became_active":"2024-04-26T21:41:07.894807+0000","last_became_peered":"2024-04-26T21:41:07.894807+0000","last_unstale":"2024-04-26T21:41:07.901416+0000","last_undegraded":"2024-04-26T21:41:07.901416+0000","last_fullsized":"2024-04-26T21:41:07.901416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:25.447410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.883657+0000","last_change":"2024-04-26T21:41:06.884947+0000","last_active":"2024-04-26T21:41:07.883657+0000","last_peered":"2024-04-26T21:41:07.883657+0000","last_clean":"2024-04-26T21:41:07.883657+0000","last_became_active":"2024-04-26T21:41:06.884676+0000","last_became_peered":"2024-04-26T21:41:06.884676+0000","last_unstale":"2024-04-26T21:41:07.883657+0000","last_undegraded":"2024-04-26T21:41:07.883657+0000","last_fullsized":"2024-04-26T21:41:07.883657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:54.843135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.895307+0000","last_change":"2024-04-26T21:41:07.891525+0000","last_active":"2024-04-26T21:41:07.895307+0000","last_peered":"2024-04-26T21:41:07.895307+0000","last_clean":"2024-04-26T21:41:07.895307+0000","last_became_active":"2024-04-26T21:41:07.891195+0000","last_became_peered":"2024-04-26T21:41:07.891195+0000","last_unstale":"2024-04-26T21:41:07.895307+0000","last_undegraded":"2024-04-26T21:41:07.895307+0000","last_fullsized":"2024-04-26T21:41:07.895307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:12.443656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906814+0000","last_change":"2024-04-26T21:41:06.892246+0000","last_active":"2024-04-26T21:41:07.906814+0000","last_peered":"2024-04-26T21:41:07.906814+0000","last_clean":"2024-04-26T21:41:07.906814+0000","last_became_active":"2024-04-26T21:41:06.891923+0000","last_became_peered":"2024-04-26T21:41:06.891923+0000","last_unstale":"2024-04-26T21:41:07.906814+0000","last_undegraded":"2024-04-26T21:41:07.906814+0000","last_fullsized":"2024-04-26T21:41:07.906814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:15.083326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.894993+0000","last_change":"2024-04-26T21:41:07.892907+0000","last_active":"2024-04-26T21:41:07.894993+0000","last_peered":"2024-04-26T21:41:07.894993+0000","last_clean":"2024-04-26T21:41:07.894993+0000","last_became_active":"2024-04-26T21:41:07.892624+0000","last_became_peered":"2024-04-26T21:41:07.892624+0000","last_unstale":"2024-04-26T21:41:07.894993+0000","last_undegraded":"2024-04-26T21:41:07.894993+0000","last_fullsized":"2024-04-26T21:41:07.894993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:49.407084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.887285+0000","last_change":"2024-04-26T21:41:06.885610+0000","last_active":"2024-04-26T21:41:07.887285+0000","last_peered":"2024-04-26T21:41:07.887285+0000","last_clean":"2024-04-26T21:41:07.887285+0000","last_became_active":"2024-04-26T21:41:06.885296+0000","last_became_peered":"2024-04-26T21:41:06.885296+0000","last_unstale":"2024-04-26T21:41:07.887285+0000","last_undegraded":"2024-04-26T21:41:07.887285+0000","last_fullsized":"2024-04-26T21:41:07.887285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:38.466644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.899056+0000","last_change":"2024-04-26T21:41:07.895738+0000","last_active":"2024-04-26T21:41:07.899056+0000","last_peered":"2024-04-26T21:41:07.899056+0000","last_clean":"2024-04-26T21:41:07.899056+0000","last_became_active":"2024-04-26T21:41:07.895353+0000","last_became_peered":"2024-04-26T21:41:07.895353+0000","last_unstale":"2024-04-26T21:41:07.899056+0000","last_undegraded":"2024-04-26T21:41:07.899056+0000","last_fullsized":"2024-04-26T21:41:07.899056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:10.773944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.885413+0000","last_change":"2024-04-26T21:41:06.886737+0000","last_active":"2024-04-26T21:41:07.885413+0000","last_peered":"2024-04-26T21:41:07.885413+0000","last_clean":"2024-04-26T21:41:07.885413+0000","last_became_active":"2024-04-26T21:41:06.886460+0000","last_became_peered":"2024-04-26T21:41:06.886460+0000","last_unstale":"2024-04-26T21:41:07.885413+0000","last_undegraded":"2024-04-26T21:41:07.885413+0000","last_fullsized":"2024-04-26T21:41:07.885413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:49.157729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.893258+0000","last_change":"2024-04-26T21:41:07.890642+0000","last_active":"2024-04-26T21:41:07.893258+0000","last_peered":"2024-04-26T21:41:07.893258+0000","last_clean":"2024-04-26T21:41:07.893258+0000","last_became_active":"2024-04-26T21:41:07.890306+0000","last_became_peered":"2024-04-26T21:41:07.890306+0000","last_unstale":"2024-04-26T21:41:07.893258+0000","last_undegraded":"2024-04-26T21:41:07.893258+0000","last_fullsized":"2024-04-26T21:41:07.893258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:36.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.880978+0000","last_change":"2024-04-26T21:41:06.889949+0000","last_active":"2024-04-26T21:41:07.880978+0000","last_peered":"2024-04-26T21:41:07.880978+0000","last_clean":"2024-04-26T21:41:07.880978+0000","last_became_active":"2024-04-26T21:41:06.889613+0000","last_became_peered":"2024-04-26T21:41:06.889613+0000","last_unstale":"2024-04-26T21:41:07.880978+0000","last_undegraded":"2024-04-26T21:41:07.880978+0000","last_fullsized":"2024-04-26T21:41:07.880978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:38.466644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.901488+0000","last_change":"2024-04-26T21:41:07.894557+0000","last_active":"2024-04-26T21:41:07.901488+0000","last_peered":"2024-04-26T21:41:07.901488+0000","last_clean":"2024-04-26T21:41:07.901488+0000","last_became_active":"2024-04-26T21:41:07.892591+0000","last_became_peered":"2024-04-26T21:41:07.892591+0000","last_unstale":"2024-04-26T21:41:07.901488+0000","last_undegraded":"2024-04-26T21:41:07.901488+0000","last_fullsized":"2024-04-26T21:41:07.901488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:03.104758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.883428+0000","last_change":"2024-04-26T21:41:06.888469+0000","last_active":"2024-04-26T21:41:07.883428+0000","last_peered":"2024-04-26T21:41:07.883428+0000","last_clean":"2024-04-26T21:41:07.883428+0000","last_became_active":"2024-04-26T21:41:06.887986+0000","last_became_peered":"2024-04-26T21:41:06.887986+0000","last_unstale":"2024-04-26T21:41:07.883428+0000","last_undegraded":"2024-04-26T21:41:07.883428+0000","last_fullsized":"2024-04-26T21:41:07.883428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:42.543019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904639+0000","last_change":"2024-04-26T21:41:07.896999+0000","last_active":"2024-04-26T21:41:07.904639+0000","last_peered":"2024-04-26T21:41:07.904639+0000","last_clean":"2024-04-26T21:41:07.904639+0000","last_became_active":"2024-04-26T21:41:07.896692+0000","last_became_peered":"2024-04-26T21:41:07.896692+0000","last_unstale":"2024-04-26T21:41:07.904639+0000","last_undegraded":"2024-04-26T21:41:07.904639+0000","last_fullsized":"2024-04-26T21:41:07.904639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.467106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.878495+0000","last_change":"2024-04-26T21:41:06.883834+0000","last_active":"2024-04-26T21:41:07.878495+0000","last_peered":"2024-04-26T21:41:07.878495+0000","last_clean":"2024-04-26T21:41:07.878495+0000","last_became_active":"2024-04-26T21:41:06.883555+0000","last_became_peered":"2024-04-26T21:41:06.883555+0000","last_unstale":"2024-04-26T21:41:07.878495+0000","last_undegraded":"2024-04-26T21:41:07.878495+0000","last_fullsized":"2024-04-26T21:41:07.878495+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:38.623818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.909898+0000","last_change":"2024-04-26T21:41:07.905376+0000","last_active":"2024-04-26T21:41:07.909898+0000","last_peered":"2024-04-26T21:41:07.909898+0000","last_clean":"2024-04-26T21:41:07.909898+0000","last_became_active":"2024-04-26T21:41:07.904231+0000","last_became_peered":"2024-04-26T21:41:07.904231+0000","last_unstale":"2024-04-26T21:41:07.909898+0000","last_undegraded":"2024-04-26T21:41:07.909898+0000","last_fullsized":"2024-04-26T21:41:07.909898+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:52.530257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.883710+0000","last_change":"2024-04-26T21:41:06.888390+0000","last_active":"2024-04-26T21:41:07.883710+0000","last_peered":"2024-04-26T21:41:07.883710+0000","last_clean":"2024-04-26T21:41:07.883710+0000","last_became_active":"2024-04-26T21:41:06.888006+0000","last_became_peered":"2024-04-26T21:41:06.888006+0000","last_unstale":"2024-04-26T21:41:07.883710+0000","last_undegraded":"2024-04-26T21:41:07.883710+0000","last_fullsized":"2024-04-26T21:41:07.883710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:15.884490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.906172+0000","last_change":"2024-04-26T21:41:07.897421+0000","last_active":"2024-04-26T21:41:07.906172+0000","last_peered":"2024-04-26T21:41:07.906172+0000","last_clean":"2024-04-26T21:41:07.906172+0000","last_became_active":"2024-04-26T21:41:07.897147+0000","last_became_peered":"2024-04-26T21:41:07.897147+0000","last_unstale":"2024-04-26T21:41:07.906172+0000","last_undegraded":"2024-04-26T21:41:07.906172+0000","last_fullsized":"2024-04-26T21:41:07.906172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:36.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.880067+0000","last_change":"2024-04-26T21:41:06.895985+0000","last_active":"2024-04-26T21:41:07.880067+0000","last_peered":"2024-04-26T21:41:07.880067+0000","last_clean":"2024-04-26T21:41:07.880067+0000","last_became_active":"2024-04-26T21:41:06.895706+0000","last_became_peered":"2024-04-26T21:41:06.895706+0000","last_unstale":"2024-04-26T21:41:07.880067+0000","last_undegraded":"2024-04-26T21:41:07.880067+0000","last_fullsized":"2024-04-26T21:41:07.880067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:48.405040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.904636+0000","last_change":"2024-04-26T21:41:07.904006+0000","last_active":"2024-04-26T21:41:07.904636+0000","last_peered":"2024-04-26T21:41:07.904636+0000","last_clean":"2024-04-26T21:41:07.904636+0000","last_became_active":"2024-04-26T21:41:07.903700+0000","last_became_peered":"2024-04-26T21:41:07.903700+0000","last_unstale":"2024-04-26T21:41:07.904636+0000","last_undegraded":"2024-04-26T21:41:07.904636+0000","last_fullsized":"2024-04-26T21:41:07.904636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:06.865898+0000","last_clean_scrub_stamp":"2024-04-26T21:41:06.865898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:34.997572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.884018+0000","last_change":"2024-04-26T21:41:06.890672+0000","last_active":"2024-04-26T21:41:07.884018+0000","last_peered":"2024-04-26T21:41:07.884018+0000","last_clean":"2024-04-26T21:41:07.884018+0000","last_became_active":"2024-04-26T21:41:06.890385+0000","last_became_peered":"2024-04-26T21:41:06.890385+0000","last_unstale":"2024-04-26T21:41:07.884018+0000","last_undegraded":"2024-04-26T21:41:07.884018+0000","last_fullsized":"2024-04-26T21:41:07.884018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:41:05.863855+0000","last_clean_scrub_stamp":"2024-04-26T21:41:05.863855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:35.626916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":282,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-26T21:41:07.885111+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:41:07.885111+0000","last_peered":"2024-04-26T21:41:07.885111+0000","last_clean":"2024-04-26T21:41:07.885111+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:41:07.885111+0000","last_undegraded":"2024-04-26T21:41:07.885111+0000","last_fullsized":"2024-04-26T21:41:07.885111+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":1036176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574926,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":445808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":1036176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:41:12.311 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:41:12.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:41:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.454+0000 7f85bf0d3640 1 Processor -- start 2024-04-26T21:41:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.454+0000 7f85bf0d3640 1 -- start start 2024-04-26T21:41:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85bf0d3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f85b815f550 0x7f85b80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85bf0d3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b80d1040 con 0x7f85b815f170 2024-04-26T21:41:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85bf0d3640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b80c9680 con 0x7f85b80d3bc0 2024-04-26T21:41:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85bf0d3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f85b80c9430 con 0x7f85b815f550 2024-04-26T21:41:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85b7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f85b815f550 0x7f85b80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85bce6f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f85b815f170 0x7f85b80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46652/0 (socket says 172.21.15.40:46652) 2024-04-26T21:41:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85bce6f640 1 -- 172.21.15.40:0/2429539908 learned_addr learned my addr 172.21.15.40:0/2429539908 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85b7fff640 1 -- 172.21.15.40:0/2429539908 >> v1:172.21.15.184:6789/0 conn(0x7f85b80d3bc0 legacy=0x7f85b80ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85b77fe640 1 -- 172.21.15.40:0/2429539908 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2582037202 0 0) 0x7f85ac001110 con 0x7f85b815f170 2024-04-26T21:41:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85b7fff640 1 -- 172.21.15.40:0/2429539908 >> v1:172.21.15.40:6789/0 conn(0x7f85b815f170 legacy=0x7f85b80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.455+0000 7f85b7fff640 1 -- 172.21.15.40:0/2429539908 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b80c9960 con 0x7f85b815f550 2024-04-26T21:41:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.456+0000 7f85b7fff640 1 --2- 172.21.15.40:0/2429539908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b815f550 0x7f85b80d5a80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f85b80c95b0 tx=0x7f85a402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9011fde2da52ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.456+0000 7f85b77fe640 1 -- 172.21.15.40:0/2429539908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4037070 con 0x7f85b815f550 2024-04-26T21:41:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.456+0000 7f85b77fe640 1 -- 172.21.15.40:0/2429539908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a4032440 con 0x7f85b815f550 2024-04-26T21:41:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.456+0000 7f85b77fe640 1 -- 172.21.15.40:0/2429539908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4032810 con 0x7f85b815f550 2024-04-26T21:41:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.456+0000 7f85bf0d3640 1 -- 172.21.15.40:0/2429539908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b815f550 msgr2=0x7f85b80d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.456+0000 7f85bf0d3640 1 --2- 172.21.15.40:0/2429539908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b815f550 0x7f85b80d5a80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f85b80c95b0 tx=0x7f85a402a250 comp rx=0 tx=0).stop 2024-04-26T21:41:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.457+0000 7f85bf0d3640 1 -- 172.21.15.40:0/2429539908 shutdown_connections 2024-04-26T21:41:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.457+0000 7f85bf0d3640 1 --2- 172.21.15.40:0/2429539908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b815f550 0x7f85b80d5a80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.457+0000 7f85bf0d3640 1 -- 172.21.15.40:0/2429539908 >> 172.21.15.40:0/2429539908 conn(0x7f85b80d4f60 msgr2=0x7f85b80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.457+0000 7f85bf0d3640 1 -- 172.21.15.40:0/2429539908 shutdown_connections 2024-04-26T21:41:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.457+0000 7f85bf0d3640 1 -- 172.21.15.40:0/2429539908 wait complete. 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.458+0000 7f85bf0d3640 1 Processor -- start 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.458+0000 7f85bf0d3640 1 -- start start 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bf0d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85b80d3bc0 0x7f85b8150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bf0d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b815f170 0x7f85b8151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bf0d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b80a2410 0x7f85b8153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bf0d3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85b80d2e30 con 0x7f85b80a2410 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bf0d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85b80d2cb0 con 0x7f85b815f170 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bf0d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85b80d2fb0 con 0x7f85b80d3bc0 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b80a2410 0x7f85b8153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b80a2410 0x7f85b8153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45208/0 (socket says 172.21.15.40:45208) 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 -- 172.21.15.40:0/1413844916 learned_addr learned my addr 172.21.15.40:0/1413844916 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85b7fff640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b815f170 0x7f85b8151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bce6f640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85b80d3bc0 0x7f85b8150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 -- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85b80d3bc0 msgr2=0x7f85b8150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85b80d3bc0 0x7f85b8150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 -- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b815f170 msgr2=0x7f85b8151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b815f170 0x7f85b8151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 -- 172.21.15.40:0/1413844916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b80a2900 con 0x7f85b80a2410 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85bd670640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b80a2410 0x7f85b8153ce0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f85b000c7a0 tx=0x7f85b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.459+0000 7f85b57fa640 1 -- 172.21.15.40:0/1413844916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0003b70 con 0x7f85b80a2410 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.460+0000 7f85b57fa640 1 -- 172.21.15.40:0/1413844916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b0003d30 con 0x7f85b80a2410 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.460+0000 7f85b57fa640 1 -- 172.21.15.40:0/1413844916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b000aa90 con 0x7f85b80a2410 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.460+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b8151b10 con 0x7f85b80a2410 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.460+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81520d0 con 0x7f85b80a2410 2024-04-26T21:41:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.461+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b80694f0 con 0x7f85b80a2410 2024-04-26T21:41:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.461+0000 7f85b57fa640 1 -- 172.21.15.40:0/1413844916 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85b000ac30 con 0x7f85b80a2410 2024-04-26T21:41:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.461+0000 7f85b57fa640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859c06c8f0 0x7f859c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.462+0000 7f85b57fa640 1 -- 172.21.15.40:0/1413844916 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f85b008bbf0 con 0x7f85b80a2410 2024-04-26T21:41:12.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.462+0000 7f85bce6f640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859c06c8f0 0x7f859c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.462+0000 7f85bce6f640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859c06c8f0 0x7f859c06edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000f60 tx=0x7f85ac007970 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.466+0000 7f85b57fa640 1 -- 172.21.15.40:0/1413844916 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85b0059f20 con 0x7f85b80a2410 2024-04-26T21:41:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.722+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f85b80d3650 con 0x7f85b80a2410 2024-04-26T21:41:12.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.724+0000 7f85b57fa640 1 -- 172.21.15.40:0/1413844916 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f85b0059820 con 0x7f85b80a2410 2024-04-26T21:41:12.726 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:41:12.726 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw== 2024-04-26T21:41:12.726 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:41:12.726 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:41:12.726 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:41:12.726 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.726+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859c06c8f0 msgr2=0x7f859c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.726+0000 7f85bf0d3640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859c06c8f0 0x7f859c06edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000f60 tx=0x7f85ac007970 comp rx=0 tx=0).stop 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.726+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b80a2410 msgr2=0x7f85b8153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.726+0000 7f85bf0d3640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b80a2410 0x7f85b8153ce0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f85b000c7a0 tx=0x7f85b000cc60 comp rx=0 tx=0).stop 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.727+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 shutdown_connections 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.727+0000 7f85bf0d3640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f859c06c8f0 0x7f859c06edb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.727+0000 7f85bf0d3640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85b80a2410 0x7f85b8153ce0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.727+0000 7f85bf0d3640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b815f170 0x7f85b8151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.727+0000 7f85bf0d3640 1 --2- 172.21.15.40:0/1413844916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85b80d3bc0 0x7f85b8150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.727+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 >> 172.21.15.40:0/1413844916 conn(0x7f85b80d4f60 msgr2=0x7f85b80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.727+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 shutdown_connections 2024-04-26T21:41:12.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.727+0000 7f85bf0d3640 1 -- 172.21.15.40:0/1413844916 wait complete. 2024-04-26T21:41:12.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:12.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.839+0000 7fca8307e640 1 Processor -- start 2024-04-26T21:41:12.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.839+0000 7fca8307e640 1 -- start start 2024-04-26T21:41:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8307e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca7c162810 0x7fca7c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8307e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 0x7fca7c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8307e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca7c0cd930 0x7fca7c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8307e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fca7c0d0750 con 0x7fca7c0cd930 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8307e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fca7c0d16a0 con 0x7fca7c162810 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8307e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca7c0d10d0 con 0x7fca7c0b2340 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8161b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca7c0cd930 0x7fca7c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8161b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca7c0cd930 0x7fca7c0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45212/0 (socket says 172.21.15.40:45212) 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8161b640 1 -- 172.21.15.40:0/3474680128 learned_addr learned my addr 172.21.15.40:0/3474680128 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca7bfff640 1 --2- 172.21.15.40:0/3474680128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 0x7fca7c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca80e1a640 1 --2- 172.21.15.40:0/3474680128 >> v2:172.21.15.184:3300/0 conn(0x7fca7c162810 0x7fca7c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8161b640 1 -- 172.21.15.40:0/3474680128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 msgr2=0x7fca7c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8161b640 1 --2- 172.21.15.40:0/3474680128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 0x7fca7c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8161b640 1 -- 172.21.15.40:0/3474680128 >> v2:172.21.15.184:3300/0 conn(0x7fca7c162810 msgr2=0x7fca7c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8161b640 1 --2- 172.21.15.40:0/3474680128 >> v2:172.21.15.184:3300/0 conn(0x7fca7c162810 0x7fca7c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.840+0000 7fca8161b640 1 -- 172.21.15.40:0/3474680128 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c0d1370 con 0x7fca7c0cd930 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.841+0000 7fca8161b640 1 --2- 172.21.15.40:0/3474680128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c0cd930 0x7fca7c0d1b30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fca7c073d90 tx=0x7fca7402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c28df0bae21f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.841+0000 7fca7b7fe640 1 -- 172.21.15.40:0/3474680128 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74036070 con 0x7fca7c0cd930 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.841+0000 7fca7b7fe640 1 -- 172.21.15.40:0/3474680128 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74031600 con 0x7fca7c0cd930 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.841+0000 7fca7b7fe640 1 -- 172.21.15.40:0/3474680128 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca740319d0 con 0x7fca7c0cd930 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 -- 172.21.15.40:0/3474680128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c0cd930 msgr2=0x7fca7c0d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 --2- 172.21.15.40:0/3474680128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c0cd930 0x7fca7c0d1b30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fca7c073d90 tx=0x7fca7402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 -- 172.21.15.40:0/3474680128 shutdown_connections 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 --2- 172.21.15.40:0/3474680128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c0cd930 0x7fca7c0d1b30 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 --2- 172.21.15.40:0/3474680128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 0x7fca7c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 --2- 172.21.15.40:0/3474680128 >> v2:172.21.15.184:3300/0 conn(0x7fca7c162810 0x7fca7c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 -- 172.21.15.40:0/3474680128 >> 172.21.15.40:0/3474680128 conn(0x7fca7c0d2da0 msgr2=0x7fca7c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 -- 172.21.15.40:0/3474680128 shutdown_connections 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.842+0000 7fca8307e640 1 -- 172.21.15.40:0/3474680128 wait complete. 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.843+0000 7fca8307e640 1 Processor -- start 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.843+0000 7fca8307e640 1 -- start start 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8307e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 0x7fca7c157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8307e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca7c0cd930 0x7fca7c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8307e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c162810 0x7fca7c06e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8307e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c070890 con 0x7fca7c162810 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8307e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c0d0470 con 0x7fca7c0cd930 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8307e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca7c0d2070 con 0x7fca7c0b2340 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c162810 0x7fca7c06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c162810 0x7fca7c06e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45226/0 (socket says 172.21.15.40:45226) 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 -- 172.21.15.40:0/805957767 learned_addr learned my addr 172.21.15.40:0/805957767 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca80e1a640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 0x7fca7c157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca7bfff640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca7c0cd930 0x7fca7c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 -- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 msgr2=0x7fca7c157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 0x7fca7c157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 -- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca7c0cd930 msgr2=0x7fca7c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca7c0cd930 0x7fca7c06db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 -- 172.21.15.40:0/805957767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c0cb700 con 0x7fca7c162810 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.844+0000 7fca8161b640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c162810 0x7fca7c06e050 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fca74009f40 tx=0x7fca74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.845+0000 7fca797fa640 1 -- 172.21.15.40:0/805957767 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74036050 con 0x7fca7c162810 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.845+0000 7fca797fa640 1 -- 172.21.15.40:0/805957767 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74042070 con 0x7fca7c162810 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.845+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c0c9840 con 0x7fca7c162810 2024-04-26T21:41:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.845+0000 7fca797fa640 1 -- 172.21.15.40:0/805957767 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74009510 con 0x7fca7c162810 2024-04-26T21:41:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.845+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c1f8630 con 0x7fca7c162810 2024-04-26T21:41:12.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.846+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7c0694f0 con 0x7fca7c162810 2024-04-26T21:41:12.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.846+0000 7fca797fa640 1 -- 172.21.15.40:0/805957767 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca74007c50 con 0x7fca7c162810 2024-04-26T21:41:12.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.846+0000 7fca797fa640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca6006c8f0 0x7fca6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:12.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.847+0000 7fca797fa640 1 -- 172.21.15.40:0/805957767 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fca740aca10 con 0x7fca7c162810 2024-04-26T21:41:12.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.847+0000 7fca80e1a640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca6006c8f0 0x7fca6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:12.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.847+0000 7fca80e1a640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca6006c8f0 0x7fca6006edb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fca700079a0 tx=0x7fca70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:12.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:12.851+0000 7fca797fa640 1 -- 172.21.15.40:0/805957767 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca7407a6f0 con 0x7fca7c162810 2024-04-26T21:41:13.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.077+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca7c0d2430 con 0x7fca7c162810 2024-04-26T21:41:13.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.077+0000 7fca797fa640 1 -- 172.21.15.40:0/805957767 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5186 (secure 0 0 0) 0x7fca7407e9f0 con 0x7fca7c162810 2024-04-26T21:41:13.078 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:13.079 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:10.262121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3115383225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3115383225},{"type":"v1","addr":"172.21.15.40:6835","nonce":3115383225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7230,"qdb_cluster":[7230]},"id":5}]} 2024-04-26T21:41:13.079 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 71 2024-04-26T21:41:13.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca6006c8f0 msgr2=0x7fca6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca6006c8f0 0x7fca6006edb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fca700079a0 tx=0x7fca70008040 comp rx=0 tx=0).stop 2024-04-26T21:41:13.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c162810 msgr2=0x7fca7c06e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c162810 0x7fca7c06e050 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fca74009f40 tx=0x7fca74004a40 comp rx=0 tx=0).stop 2024-04-26T21:41:13.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 shutdown_connections 2024-04-26T21:41:13.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca6006c8f0 0x7fca6006edb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca7c162810 0x7fca7c06e050 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca7c0cd930 0x7fca7c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 --2- 172.21.15.40:0/805957767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7c0b2340 0x7fca7c157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 >> 172.21.15.40:0/805957767 conn(0x7fca7c0d2da0 msgr2=0x7fca7c070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 shutdown_connections 2024-04-26T21:41:13.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.079+0000 7fca8307e640 1 -- 172.21.15.40:0/805957767 wait complete. 2024-04-26T21:41:13.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:41:07.249301+0000', 'modified': '2024-04-26T21:41:10.262121+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7230}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7230': {'gid': 7230, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/3115383225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3115383225}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3115383225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7230, 'qdb_cluster': [7230]} 2024-04-26T21:41:13.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:41:13.494 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:41:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.599+0000 7fd6b6420640 1 Processor -- start 2024-04-26T21:41:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.599+0000 7fd6b6420640 1 -- start start 2024-04-26T21:41:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6b6420640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd6b0162810 0x7fd6b0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6b6420640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b00d10d0 con 0x7fd6b00b2340 2024-04-26T21:41:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6b6420640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b00d1370 con 0x7fd6b00d1b30 2024-04-26T21:41:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6b6420640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b00d0750 con 0x7fd6b0162810 2024-04-26T21:41:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6af7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd6b00b2340 0x7fd6b00cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46658/0 (socket says 172.21.15.40:46658) 2024-04-26T21:41:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6af7fe640 1 -- 172.21.15.40:0/873277792 learned_addr learned my addr 172.21.15.40:0/873277792 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6affff640 1 --2- 172.21.15.40:0/873277792 >> v2:172.21.15.184:3300/0 conn(0x7fd6b0162810 0x7fd6b0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2265912757 0 0) 0x7fd6b00d10d0 con 0x7fd6b00b2340 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd69c003520 con 0x7fd6b00b2340 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1307302630 0 0) 0x7fd69c003520 con 0x7fd6b00b2340 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 >> v2:172.21.15.184:3300/0 conn(0x7fd6b0162810 msgr2=0x7fd6b0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.600+0000 7fd6aeffd640 1 --2- 172.21.15.40:0/873277792 >> v2:172.21.15.184:3300/0 conn(0x7fd6b0162810 0x7fd6b0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 >> v1:172.21.15.184:6789/0 conn(0x7fd6b00d1b30 legacy=0x7fd6b0070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b00d01b0 con 0x7fd6b00b2340 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fd6a0001670 con 0x7fd6b00b2340 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 >> v1:172.21.15.40:6789/0 conn(0x7fd6b00b2340 legacy=0x7fd6b00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c004800 0x7fd69c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0051c0 0x7fd69c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b00d0750 con 0x7fd69c0039b0 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c00c060 con 0x7fd69c0051c0 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd69c00c260 con 0x7fd69c004800 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6affff640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0051c0 0x7fd69c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c004800 0x7fd69c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6affff640 1 -- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c004800 msgr2=0x7fd69c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6affff640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c004800 0x7fd69c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6affff640 1 -- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0051c0 msgr2=0x7fd69c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6affff640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0051c0 0x7fd69c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6b49bd640 1 -- 172.21.15.40:0/873277792 reap_dead start 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6affff640 1 -- 172.21.15.40:0/873277792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c003520 con 0x7fd69c0039b0 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.601+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c004800 0x7fd69c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.602+0000 7fd6affff640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd698009ac0 tx=0x7fd69c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d253d37c3176fe13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.602+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698036070 con 0x7fd69c0039b0 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.602+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd698031400 con 0x7fd69c0039b0 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.602+0000 7fd6aeffd640 1 -- 172.21.15.40:0/873277792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698009690 con 0x7fd69c0039b0 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.602+0000 7fd6b6420640 1 -- 172.21.15.40:0/873277792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c0039b0 msgr2=0x7fd69c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.602+0000 7fd6b6420640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd698009ac0 tx=0x7fd69c005830 comp rx=0 tx=0).stop 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.603+0000 7fd6b6420640 1 -- 172.21.15.40:0/873277792 shutdown_connections 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.603+0000 7fd6b6420640 1 --2- 172.21.15.40:0/873277792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.603+0000 7fd6b6420640 1 -- 172.21.15.40:0/873277792 >> 172.21.15.40:0/873277792 conn(0x7fd6b00d2da0 msgr2=0x7fd6b00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.603+0000 7fd6b6420640 1 -- 172.21.15.40:0/873277792 shutdown_connections 2024-04-26T21:41:13.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.603+0000 7fd6b6420640 1 -- 172.21.15.40:0/873277792 wait complete. 2024-04-26T21:41:13.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.604+0000 7fd6b6420640 1 Processor -- start 2024-04-26T21:41:13.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.604+0000 7fd6b6420640 1 -- start start 2024-04-26T21:41:13.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b6420640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0039b0 0x7fd6b00c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b6420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b00c6ce0 0x7fd6b0159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b6420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6b0159f50 0x7fd6b015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b6420640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b00d1fc0 con 0x7fd6b0159f50 2024-04-26T21:41:13.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b6420640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b00d16a0 con 0x7fd69c0039b0 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b6420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b00d2140 con 0x7fd6b00c6ce0 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6b0159f50 0x7fd6b015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6b0159f50 0x7fd6b015a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45242/0 (socket says 172.21.15.40:45242) 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 -- 172.21.15.40:0/1863166699 learned_addr learned my addr 172.21.15.40:0/1863166699 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6affff640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0039b0 0x7fd6b00c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b00c6ce0 0x7fd6b0159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 -- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b00c6ce0 msgr2=0x7fd6b0159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b00c6ce0 0x7fd6b0159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 -- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0039b0 msgr2=0x7fd6b00c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0039b0 0x7fd6b00c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 -- 172.21.15.40:0/1863166699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b015a910 con 0x7fd6b0159f50 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6affff640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0039b0 0x7fd6b00c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6b0159f50 0x7fd6b015a3d0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0073630 tx=0x7fd6a800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6acff9640 1 -- 172.21.15.40:0/1863166699 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a801a070 con 0x7fd6b0159f50 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6acff9640 1 -- 172.21.15.40:0/1863166699 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a8006d60 con 0x7fd6b0159f50 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b6420640 1 -- 172.21.15.40:0/1863166699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b00c8210 con 0x7fd6b0159f50 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6acff9640 1 -- 172.21.15.40:0/1863166699 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8003bb0 con 0x7fd6b0159f50 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.605+0000 7fd6b6420640 1 -- 172.21.15.40:0/1863166699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b01553d0 con 0x7fd6b0159f50 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.607+0000 7fd6acff9640 1 -- 172.21.15.40:0/1863166699 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6a801d070 con 0x7fd6b0159f50 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.607+0000 7fd6acff9640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd69006c8f0 0x7fd69006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.607+0000 7fd6acff9640 1 -- 172.21.15.40:0/1863166699 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fd6a808b2e0 con 0x7fd6b0159f50 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.608+0000 7fd6affff640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd69006c8f0 0x7fd69006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.608+0000 7fd6affff640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd69006c8f0 0x7fd69006edb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd698033000 tx=0x7fd698008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.608+0000 7fd6b9212740 1 Processor -- start 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.608+0000 7fd6b9212740 1 -- start start 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e8735820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e8753230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e870b120 0x55e4e87513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b9212740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e4e866f740 con 0x55e4e870b120 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b9212740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e4e860b3a0 con 0x55e4e8752ce0 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b9212740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e4e866b700 con 0x55e4e8735440 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b49bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e870b120 0x55e4e87513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b49bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e870b120 0x55e4e87513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45246/0 (socket says 172.21.15.40:45246) 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b49bd640 1 -- 172.21.15.40:0/2065381869 learned_addr learned my addr 172.21.15.40:0/2065381869 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6affff640 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e8735820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e8753230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b49bd640 1 -- 172.21.15.40:0/2065381869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 msgr2=0x55e4e8735820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e8735820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b49bd640 1 -- 172.21.15.40:0/2065381869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 msgr2=0x55e4e8753230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e8753230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.609+0000 7fd6b49bd640 1 -- 172.21.15.40:0/2065381869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e8627750 con 0x55e4e870b120 2024-04-26T21:41:13.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e870b120 0x55e4e87513a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8006fd0 tx=0x7fd6a8058a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cf1562fdd02850 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6a5ffb640 1 -- 172.21.15.40:0/2065381869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80572c0 con 0x55e4e870b120 2024-04-26T21:41:13.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6a5ffb640 1 -- 172.21.15.40:0/2065381869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a8055630 con 0x55e4e870b120 2024-04-26T21:41:13.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6a5ffb640 1 -- 172.21.15.40:0/2065381869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8058650 con 0x55e4e870b120 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 -- 172.21.15.40:0/2065381869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e870b120 msgr2=0x55e4e87513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e870b120 0x55e4e87513a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8006fd0 tx=0x7fd6a8058a20 comp rx=0 tx=0).stop 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 -- 172.21.15.40:0/2065381869 shutdown_connections 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e870b120 0x55e4e87513a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e8753230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2065381869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e8735820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 -- 172.21.15.40:0/2065381869 >> 172.21.15.40:0/2065381869 conn(0x55e4e8707a50 msgr2=0x55e4e8750250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 -- 172.21.15.40:0/2065381869 shutdown_connections 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.610+0000 7fd6b9212740 1 -- 172.21.15.40:0/2065381869 wait complete. 2024-04-26T21:41:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.611+0000 7fd6b9212740 1 -- start start 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.611+0000 7fd6b9212740 1 Processor -- start 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8735440 0x55e4e87284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e87485b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 0x55e4e874d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6b9212740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e4e870bec0 con 0x55e4e8735440 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6b9212740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e4e85f6fe0 con 0x55e4e8752ce0 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6b9212740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e4e8705980 con 0x55e4e8748af0 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6b9212740 10 client.0 _ll_register_callbacks cb 0x55e4e860d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8735440 0x55e4e87284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8735440 0x55e4e87284e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45248/0 (socket says 172.21.15.40:45248) 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 -- 172.21.15.40:0/2154034330 learned_addr learned my addr 172.21.15.40:0/2154034330 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e87485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 0x55e4e874d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 -- 172.21.15.40:0/2154034330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 msgr2=0x55e4e874d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 0x55e4e874d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 -- 172.21.15.40:0/2154034330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 msgr2=0x55e4e87485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e87485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 -- 172.21.15.40:0/2154034330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e83812d0 con 0x55e4e8735440 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd6affff640 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8735440 0x55e4e87284e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd69802abb0 tx=0x7fd69802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161964b4e1e013a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.612+0000 7fd68effd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd68effd640 1 -- 172.21.15.40:0/2154034330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69804e060 con 0x55e4e8735440 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd68effd640 1 -- 172.21.15.40:0/2154034330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fd69803c670 con 0x55e4e8735440 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 1 -- 172.21.15.40:0/2154034330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a8055630 con 0x55e4e8735440 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd68effd640 1 -- 172.21.15.40:0/2154034330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fd698054050 con 0x55e4e8735440 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 10 client.7377 fetch_fsmap learned FSMap version 71 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 1 -- 172.21.15.40:0/2154034330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55e4e844b6b0 con 0x55e4e8735440 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd68effd640 1 -- 172.21.15.40:0/2154034330 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a8055630 con 0x55e4e8735440 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd68effd640 1 -- 172.21.15.40:0/2154034330 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7fd698057020 con 0x55e4e8735440 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 10 client.7377 fetch_fsmap finished waiting for FSMap version 71 2024-04-26T21:41:13.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 10 client.7377 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 20 client.7377 populate_metadata read hostname 'smithi040' 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55e4e8763e60 0x55e4e874b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 4 client.7377 mds_command: new command op to 7230 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6b9212740 1 -- 172.21.15.40:0/2154034330 --> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4e867d350 con 0x55e4e8763e60 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.613+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55e4e8763e60 0x55e4e874b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd68effd640 10 client.7377 ms_handle_connect on v2:172.21.15.40:6834/3115383225 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55e4e8763e60 0x55e4e874b710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2f0402b4a82436 server_cookie=6ef72d6b0fb29bff in_seq=0 out_seq=0 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd68effd640 1 -- 172.21.15.40:0/2154034330 <== mds.0 v2:172.21.15.40:6834/3115383225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4e867d350 con 0x55e4e8763e60 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd68effd640 10 client.7377 handle_command_reply: tid=0 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 client.7377 shutdown 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 -- 172.21.15.40:0/2154034330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8735440 msgr2=0x55e4e87284e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8735440 0x55e4e87284e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd69802abb0 tx=0x7fd69802c000 comp rx=0 tx=0).stop 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 -- 172.21.15.40:0/2154034330 shutdown_connections 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55e4e8763e60 0x55e4e874b710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 0x55e4e874d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e87485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd68effd640 0 client.7377 ms_handle_reset on v2:172.21.15.40:6834/3115383225 2024-04-26T21:41:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 --2- 172.21.15.40:0/2154034330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8735440 0x55e4e87284e0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.614+0000 7fd6b9212740 1 -- 172.21.15.40:0/2154034330 >> 172.21.15.40:0/2154034330 conn(0x55e4e870b120 msgr2=0x55e4e8754060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.615+0000 7fd6b9212740 1 -- 172.21.15.40:0/2154034330 shutdown_connections 2024-04-26T21:41:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.615+0000 7fd6b9212740 1 -- 172.21.15.40:0/2154034330 wait complete. 2024-04-26T21:41:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.615+0000 7fd6b9212740 20 client.7377 trim_cache size 0 max 16384 2024-04-26T21:41:13.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.633+0000 7fd6b9212740 1 Processor -- start 2024-04-26T21:41:13.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.633+0000 7fd6b9212740 1 -- start start 2024-04-26T21:41:13.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8735440 0x55e4e87496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 0x55e4e8748230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8752ce0 0x55e4e874e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b9212740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b02015f0 con 0x55e4e8752ce0 2024-04-26T21:41:13.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b9212740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b0200950 con 0x55e4e8735440 2024-04-26T21:41:13.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b9212740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b0200fa0 con 0x55e4e8748af0 2024-04-26T21:41:13.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8752ce0 0x55e4e874e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8752ce0 0x55e4e874e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45254/0 (socket says 172.21.15.40:45254) 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 -- 172.21.15.40:0/3065751700 learned_addr learned my addr 172.21.15.40:0/3065751700 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6affff640 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8735440 0x55e4e87496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 0x55e4e8748230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 -- 172.21.15.40:0/3065751700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 msgr2=0x55e4e8748230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 0x55e4e8748230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 -- 172.21.15.40:0/3065751700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8735440 msgr2=0x55e4e87496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8735440 0x55e4e87496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 -- 172.21.15.40:0/3065751700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e8704880 con 0x55e4e8752ce0 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8752ce0 0x55e4e874e4c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8001550 tx=0x7fd6a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdb4a050c47bc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.634+0000 7fd6a57fa640 1 -- 172.21.15.40:0/3065751700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8088040 con 0x55e4e8752ce0 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6a57fa640 1 -- 172.21.15.40:0/3065751700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a8055630 con 0x55e4e8752ce0 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6a57fa640 1 -- 172.21.15.40:0/3065751700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80782c0 con 0x55e4e8752ce0 2024-04-26T21:41:13.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 -- 172.21.15.40:0/3065751700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8752ce0 msgr2=0x55e4e874e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8752ce0 0x55e4e874e4c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8001550 tx=0x7fd6a8017000 comp rx=0 tx=0).stop 2024-04-26T21:41:13.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 -- 172.21.15.40:0/3065751700 shutdown_connections 2024-04-26T21:41:13.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8752ce0 0x55e4e874e4c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8748af0 0x55e4e8748230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 --2- 172.21.15.40:0/3065751700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8735440 0x55e4e87496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 -- 172.21.15.40:0/3065751700 >> 172.21.15.40:0/3065751700 conn(0x55e4e870b120 msgr2=0x55e4e8704c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 -- 172.21.15.40:0/3065751700 shutdown_connections 2024-04-26T21:41:13.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.635+0000 7fd6b9212740 1 -- 172.21.15.40:0/3065751700 wait complete. 2024-04-26T21:41:13.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.636+0000 7fd6b9212740 1 -- start start 2024-04-26T21:41:13.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.636+0000 7fd6b9212740 1 Processor -- start 2024-04-26T21:41:13.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e87c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8748af0 0x55e4e8708ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b9212740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e8709530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b9212740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e4e8763ce0 con 0x55e4e8748af0 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b9212740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e4e8728a70 con 0x55e4e8752ce0 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b9212740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e4e8764330 con 0x55e4e8735440 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b9212740 10 client.0 _ll_register_callbacks cb 0x7fd69c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8748af0 0x55e4e8708ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8748af0 0x55e4e8708ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45270/0 (socket says 172.21.15.40:45270) 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 -- 172.21.15.40:0/1678591090 learned_addr learned my addr 172.21.15.40:0/1678591090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6affff640 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e87c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b49bd640 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e8709530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 -- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 msgr2=0x55e4e87c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e87c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 -- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 msgr2=0x55e4e8709530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e8709530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 -- 172.21.15.40:0/1678591090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e8774130 con 0x55e4e8748af0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6affff640 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e87c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6af7fe640 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8748af0 0x55e4e8708ff0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd6b006bf00 tx=0x7fd6a0001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369009acfa970e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd68effd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd68effd640 1 -- 172.21.15.40:0/1678591090 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a001fdd0 con 0x55e4e8748af0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd68effd640 1 -- 172.21.15.40:0/1678591090 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fd6a00178b0 con 0x55e4e8748af0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.637+0000 7fd6b9212740 1 -- 172.21.15.40:0/1678591090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a8055630 con 0x55e4e8748af0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd68effd640 1 -- 172.21.15.40:0/1678591090 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fd6a0017ab0 con 0x55e4e8748af0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 10 client.7383 fetch_fsmap learned FSMap version 71 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 1 -- 172.21.15.40:0/1678591090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55e4e8733c60 con 0x55e4e8748af0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd68effd640 1 -- 172.21.15.40:0/1678591090 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a8055630 con 0x55e4e8748af0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd68effd640 1 -- 172.21.15.40:0/1678591090 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7fd6a0032080 con 0x55e4e8748af0 2024-04-26T21:41:13.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 10 client.7383 fetch_fsmap finished waiting for FSMap version 71 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 10 client.7383 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 20 client.7383 populate_metadata read hostname 'smithi040' 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55e4e8745be0 0x55e4e8745fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 4 client.7383 mds_command: new command op to 7230 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 1 -- 172.21.15.40:0/1678591090 --> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] -- command(tid 0: {"prefix": "status"}) -- 0x55e4e86771c0 con 0x55e4e8745be0 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6affff640 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55e4e8745be0 0x55e4e8745fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd68effd640 10 client.7383 ms_handle_connect on v2:172.21.15.40:6834/3115383225 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6affff640 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55e4e8745be0 0x55e4e8745fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a19184362c50c0 server_cookie=7f2a311bd95550c7 in_seq=0 out_seq=0 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd68effd640 1 -- 172.21.15.40:0/1678591090 <== mds.0 v2:172.21.15.40:6834/3115383225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e4e86771c0 con 0x55e4e8745be0 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd68effd640 10 client.7383 handle_command_reply: tid=0 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.638+0000 7fd6b9212740 1 client.7383 shutdown 2024-04-26T21:41:13.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 -- 172.21.15.40:0/1678591090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8748af0 msgr2=0x55e4e8708ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8748af0 0x55e4e8708ff0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd6b006bf00 tx=0x7fd6a0001470 comp rx=0 tx=0).stop 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 -- 172.21.15.40:0/1678591090 shutdown_connections 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55e4e8745be0 0x55e4e8745fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4e8752ce0 0x55e4e8709530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e4e8748af0 0x55e4e8708ff0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd68effd640 0 client.7383 ms_handle_reset on v2:172.21.15.40:6834/3115383225 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 --2- 172.21.15.40:0/1678591090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e4e8735440 0x55e4e87c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 -- 172.21.15.40:0/1678591090 >> 172.21.15.40:0/1678591090 conn(0x55e4e870b120 msgr2=0x55e4e874e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 -- 172.21.15.40:0/1678591090 shutdown_connections 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 1 -- 172.21.15.40:0/1678591090 wait complete. 2024-04-26T21:41:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b9212740 20 client.7383 trim_cache size 0 max 16384 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7230, 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 5.7669435, 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 71, 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 81, 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.790583033000001 2024-04-26T21:41:13.647 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:41:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 -- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd69006c8f0 msgr2=0x7fd69006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd69006c8f0 0x7fd69006edb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd698033000 tx=0x7fd698008030 comp rx=0 tx=0).stop 2024-04-26T21:41:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 -- 172.21.15.40:0/1863166699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6b0159f50 msgr2=0x7fd6b015a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6b0159f50 0x7fd6b015a3d0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0073630 tx=0x7fd6a800fc00 comp rx=0 tx=0).stop 2024-04-26T21:41:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 -- 172.21.15.40:0/1863166699 shutdown_connections 2024-04-26T21:41:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd69006c8f0 0x7fd69006edb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6b0159f50 0x7fd6b015a3d0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b00c6ce0 0x7fd6b0159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 --2- 172.21.15.40:0/1863166699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c0039b0 0x7fd6b00c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.639+0000 7fd6b6420640 1 -- 172.21.15.40:0/1863166699 >> 172.21.15.40:0/1863166699 conn(0x7fd6b00d2da0 msgr2=0x7fd6b00cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.640+0000 7fd6b6420640 1 -- 172.21.15.40:0/1863166699 shutdown_connections 2024-04-26T21:41:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.640+0000 7fd6b6420640 1 -- 172.21.15.40:0/1863166699 wait complete. 2024-04-26T21:41:13.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:41:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.751+0000 7f76fc8b8640 1 Processor -- start 2024-04-26T21:41:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.751+0000 7f76fc8b8640 1 -- start start 2024-04-26T21:41:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76fc8b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76f815f3a0 0x7f76f80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76fc8b8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f76f80d2330 0x7f76f80b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76fc8b8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f80d0e20 con 0x7f76f80d1770 2024-04-26T21:41:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76fc8b8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f76f80b1d70 con 0x7f76f815f3a0 2024-04-26T21:41:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76fc8b8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f76f80b2050 con 0x7f76f80d2330 2024-04-26T21:41:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76f815f3a0 0x7f76f80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f5d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f76f80d1770 0x7f76f80d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46668/0 (socket says 172.21.15.40:46668) 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f5d9b640 1 -- 172.21.15.40:0/2608628394 learned_addr learned my addr 172.21.15.40:0/2608628394 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f6d9d640 1 --2- 172.21.15.40:0/2608628394 >> v2:172.21.15.184:3300/0 conn(0x7f76f80d2330 0x7f76f80b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f559a640 1 -- 172.21.15.40:0/2608628394 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587038859 0 0) 0x7f76f80d0e20 con 0x7f76f80d1770 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f559a640 1 -- 172.21.15.40:0/2608628394 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76e4003520 con 0x7f76f80d1770 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f659c640 1 -- 172.21.15.40:0/2608628394 >> v2:172.21.15.184:3300/0 conn(0x7f76f80d2330 msgr2=0x7f76f80b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f659c640 1 --2- 172.21.15.40:0/2608628394 >> v2:172.21.15.184:3300/0 conn(0x7f76f80d2330 0x7f76f80b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f659c640 1 -- 172.21.15.40:0/2608628394 >> v1:172.21.15.40:6789/0 conn(0x7f76f80d1770 legacy=0x7f76f80d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.752+0000 7f76f659c640 1 -- 172.21.15.40:0/2608628394 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80b2330 con 0x7f76f815f3a0 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.753+0000 7f76f659c640 1 --2- 172.21.15.40:0/2608628394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f815f3a0 0x7f76f80d06f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f76f80c89f0 tx=0x7f76f002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca47a3f721d47ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.753+0000 7f76f559a640 1 -- 172.21.15.40:0/2608628394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0037070 con 0x7f76f815f3a0 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.753+0000 7f76f559a640 1 -- 172.21.15.40:0/2608628394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f0032440 con 0x7f76f815f3a0 2024-04-26T21:41:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.753+0000 7f76f559a640 1 -- 172.21.15.40:0/2608628394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0032810 con 0x7f76f815f3a0 2024-04-26T21:41:13.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.754+0000 7f76fc8b8640 1 -- 172.21.15.40:0/2608628394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f815f3a0 msgr2=0x7f76f80d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.754+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/2608628394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f815f3a0 0x7f76f80d06f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f76f80c89f0 tx=0x7f76f002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.754+0000 7f76fc8b8640 1 -- 172.21.15.40:0/2608628394 shutdown_connections 2024-04-26T21:41:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.754+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/2608628394 >> v2:172.21.15.184:3300/0 conn(0x7f76f80d2330 0x7f76f80b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.754+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/2608628394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f815f3a0 0x7f76f80d06f0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.754+0000 7f76fc8b8640 1 -- 172.21.15.40:0/2608628394 >> 172.21.15.40:0/2608628394 conn(0x7f76f80d4f90 msgr2=0x7f76f80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.754+0000 7f76fc8b8640 1 -- 172.21.15.40:0/2608628394 shutdown_connections 2024-04-26T21:41:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.754+0000 7f76fc8b8640 1 -- 172.21.15.40:0/2608628394 wait complete. 2024-04-26T21:41:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.755+0000 7f76fc8b8640 1 Processor -- start 2024-04-26T21:41:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.755+0000 7f76fc8b8640 1 -- start start 2024-04-26T21:41:13.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76fc8b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f80d1770 0x7f76f81f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76fc8b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f80d2330 0x7f76f80ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76fc8b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f815f3a0 0x7f76f80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76fc8b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b2f10 con 0x7f76f80d2330 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76fc8b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b2d90 con 0x7f76f80d1770 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76fc8b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76f80b3090 con 0x7f76f815f3a0 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76f5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f80d2330 0x7f76f80ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76f5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f80d2330 0x7f76f80ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45278/0 (socket says 172.21.15.40:45278) 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76f5d9b640 1 -- 172.21.15.40:0/1865446281 learned_addr learned my addr 172.21.15.40:0/1865446281 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76f659c640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f80d1770 0x7f76f81f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.756+0000 7f76f6d9d640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f815f3a0 0x7f76f80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76f5d9b640 1 -- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f815f3a0 msgr2=0x7f76f80b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76f5d9b640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f815f3a0 0x7f76f80b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76f5d9b640 1 -- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f80d1770 msgr2=0x7f76f81f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76f5d9b640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f80d1770 0x7f76f81f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76f5d9b640 1 -- 172.21.15.40:0/1865446281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80b7d50 con 0x7f76f80d2330 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76f659c640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f80d1770 0x7f76f81f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76f5d9b640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f80d2330 0x7f76f80ca670 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c4d0 tx=0x7f76e000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76eeffd640 1 -- 172.21.15.40:0/1865446281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0017070 con 0x7f76f80d2330 2024-04-26T21:41:13.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76eeffd640 1 -- 172.21.15.40:0/1865446281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e0005d00 con 0x7f76f80d2330 2024-04-26T21:41:13.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f80b7ff0 con 0x7f76f80d2330 2024-04-26T21:41:13.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76eeffd640 1 -- 172.21.15.40:0/1865446281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e00072b0 con 0x7f76f80d2330 2024-04-26T21:41:13.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.757+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f80b85b0 con 0x7f76f80d2330 2024-04-26T21:41:13.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.758+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f80694f0 con 0x7f76f80d2330 2024-04-26T21:41:13.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.759+0000 7f76eeffd640 1 -- 172.21.15.40:0/1865446281 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76e001a070 con 0x7f76f80d2330 2024-04-26T21:41:13.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.759+0000 7f76eeffd640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76d806c8f0 0x7f76d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:13.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.759+0000 7f76eeffd640 1 -- 172.21.15.40:0/1865446281 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f76e008af50 con 0x7f76f80d2330 2024-04-26T21:41:13.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.759+0000 7f76f659c640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76d806c8f0 0x7f76d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:13.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.760+0000 7f76f659c640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76d806c8f0 0x7f76d806edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f76f0034000 tx=0x7f76f002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:13.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.764+0000 7f76eeffd640 1 -- 172.21.15.40:0/1865446281 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f76e0058c30 con 0x7f76f80d2330 2024-04-26T21:41:13.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.989+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f76f80b6860 con 0x7f76f80d2330 2024-04-26T21:41:13.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.989+0000 7f76eeffd640 1 -- 172.21.15.40:0/1865446281 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f76e005cf30 con 0x7f76f80d2330 2024-04-26T21:41:13.990 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:41:13.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76d806c8f0 msgr2=0x7f76d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76d806c8f0 0x7f76d806edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f76f0034000 tx=0x7f76f002e000 comp rx=0 tx=0).stop 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f80d2330 msgr2=0x7f76f80ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f80d2330 0x7f76f80ca670 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c4d0 tx=0x7f76e000c990 comp rx=0 tx=0).stop 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 shutdown_connections 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76d806c8f0 0x7f76d806edb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f815f3a0 0x7f76f80b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f80d2330 0x7f76f80ca670 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 --2- 172.21.15.40:0/1865446281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f80d1770 0x7f76f81f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 >> 172.21.15.40:0/1865446281 conn(0x7f76f80d4f90 msgr2=0x7f76f80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 shutdown_connections 2024-04-26T21:41:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:13.991+0000 7f76fc8b8640 1 -- 172.21.15.40:0/1865446281 wait complete. 2024-04-26T21:41:14.003 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:41:14.003 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:41:14.003 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:41:14.003 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:41:14.003 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:41:14.004 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:41:14.004 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:41:14.004 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:41:14.004 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:41:14.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:14.004 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:41:14.020 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42374sec preferred_lft 42374sec 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:41:14.021 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:41:14.022 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:41:14.022 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:41:14.022 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:41:14.022 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:41:14.022 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:41:14.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:14.024 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:41:14.083 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:41:14.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:14.085 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:41:14.140 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:41:14.141 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:41:14.141 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:41:14.141 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:41:14.141 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:41:14.141 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:41:14.141 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:41:14.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:41:14.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:41:14.142 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:41:14.142 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:41:14.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:41:14.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:41:14.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:14.266 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:41:14.282 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:41:14.282 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:41:14.282 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:41:14.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:14.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:41:14.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:41:14.346 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:41:14.400 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:41:14.401 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:41:14.401 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:41:14.455 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:41:14.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:14.456 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:41:14.538 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:41:14.538 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:41:14.545 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:41:14.545 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:41:14.545 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:41:14.545 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:41:14.546 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:41:14.546 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:41:14.546 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:41:14.547 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:41:14.547 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:41:14.547 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:41:14.548 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:41:14.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:14.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:41:14.613 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:41:14.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:41:14.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:41:14.715 INFO:teuthology.orchestra.run.smithi040.stdout:client7395 2024-04-26T21:41:14.715 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:41:14.715 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:14.731 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:14.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.870+0000 7f38afbf0640 1 Processor -- start 2024-04-26T21:41:14.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.871+0000 7f38afbf0640 1 -- start start 2024-04-26T21:41:14.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.871+0000 7f38afbf0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38a80b0380 0x7f38a80ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:14.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.871+0000 7f38afbf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b10b0 0x7f38a806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:14.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.871+0000 7f38afbf0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a80cfd80 con 0x7f38a806c780 2024-04-26T21:41:14.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.871+0000 7f38afbf0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f38a80af8e0 con 0x7f38a80b0380 2024-04-26T21:41:14.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.871+0000 7f38afbf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38a80d0020 con 0x7f38a80b10b0 2024-04-26T21:41:14.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.872+0000 7f38ad98c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38a80b0380 0x7f38a80ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:14.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.872+0000 7f38ad98c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38a80b0380 0x7f38a80ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45292/0 (socket says 172.21.15.40:45292) 2024-04-26T21:41:14.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.872+0000 7f38ad98c640 1 -- 172.21.15.40:0/3001490529 learned_addr learned my addr 172.21.15.40:0/3001490529 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:14.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.872+0000 7f38ad18b640 1 --2- 172.21.15.40:0/3001490529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b10b0 0x7f38a806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:14.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.872+0000 7f38ad98c640 1 -- 172.21.15.40:0/3001490529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b10b0 msgr2=0x7f38a806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:14.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.872+0000 7f38ad98c640 1 --2- 172.21.15.40:0/3001490529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b10b0 0x7f38a806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:14.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.872+0000 7f38ad98c640 1 -- 172.21.15.40:0/3001490529 >> v1:172.21.15.184:6789/0 conn(0x7f38a806c780 legacy=0x7f38a8072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:14.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.872+0000 7f38ad98c640 1 -- 172.21.15.40:0/3001490529 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a80d0300 con 0x7f38a80b0380 2024-04-26T21:41:14.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38ad98c640 1 --2- 172.21.15.40:0/3001490529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b0380 0x7f38a80ccd50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009870 tx=0x7f38a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f29e43911e5d727 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:14.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38ac98a640 1 -- 172.21.15.40:0/3001490529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0037070 con 0x7f38a80b0380 2024-04-26T21:41:14.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38ac98a640 1 -- 172.21.15.40:0/3001490529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a0032440 con 0x7f38a80b0380 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38ac98a640 1 -- 172.21.15.40:0/3001490529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0032810 con 0x7f38a80b0380 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38afbf0640 1 -- 172.21.15.40:0/3001490529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b0380 msgr2=0x7f38a80ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38afbf0640 1 --2- 172.21.15.40:0/3001490529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b0380 0x7f38a80ccd50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009870 tx=0x7f38a002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38afbf0640 1 -- 172.21.15.40:0/3001490529 shutdown_connections 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38afbf0640 1 --2- 172.21.15.40:0/3001490529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b10b0 0x7f38a806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38afbf0640 1 --2- 172.21.15.40:0/3001490529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b0380 0x7f38a80ccd50 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38afbf0640 1 -- 172.21.15.40:0/3001490529 >> 172.21.15.40:0/3001490529 conn(0x7f38a80d1cf0 msgr2=0x7f38a80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.873+0000 7f38afbf0640 1 -- 172.21.15.40:0/3001490529 shutdown_connections 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.874+0000 7f38afbf0640 1 -- 172.21.15.40:0/3001490529 wait complete. 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38afbf0640 1 Processor -- start 2024-04-26T21:41:14.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38afbf0640 1 -- start start 2024-04-26T21:41:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38afbf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38a806c780 0x7f38a80b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38afbf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b0380 0x7f38a81e9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38afbf0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b10b0 0x7f38a81e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38afbf0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80b0760 con 0x7f38a80b10b0 2024-04-26T21:41:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38afbf0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80d1980 con 0x7f38a806c780 2024-04-26T21:41:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38afbf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38a80d15c0 con 0x7f38a80b0380 2024-04-26T21:41:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38ae18d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b10b0 0x7f38a81e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.875+0000 7f38ae18d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b10b0 0x7f38a81e79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45306/0 (socket says 172.21.15.40:45306) 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ae18d640 1 -- 172.21.15.40:0/2021100944 learned_addr learned my addr 172.21.15.40:0/2021100944 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ad98c640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38a806c780 0x7f38a80b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ad18b640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b0380 0x7f38a81e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ae18d640 1 -- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b0380 msgr2=0x7f38a81e9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ae18d640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b0380 0x7f38a81e9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ae18d640 1 -- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38a806c780 msgr2=0x7f38a80b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ae18d640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38a806c780 0x7f38a80b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ae18d640 1 -- 172.21.15.40:0/2021100944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a80ca180 con 0x7f38a80b10b0 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38ae18d640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b10b0 0x7f38a81e79b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f389800c710 tx=0x7f389800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38a67fc640 1 -- 172.21.15.40:0/2021100944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898016020 con 0x7f38a80b10b0 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38a67fc640 1 -- 172.21.15.40:0/2021100944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3898006d50 con 0x7f38a80b10b0 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38a67fc640 1 -- 172.21.15.40:0/2021100944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898003bb0 con 0x7f38a80b10b0 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a80ca460 con 0x7f38a80b10b0 2024-04-26T21:41:14.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.876+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a80caa20 con 0x7f38a80b10b0 2024-04-26T21:41:14.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.877+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a80694f0 con 0x7f38a80b10b0 2024-04-26T21:41:14.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.878+0000 7f38a67fc640 1 -- 172.21.15.40:0/2021100944 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f389800a850 con 0x7f38a80b10b0 2024-04-26T21:41:14.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.878+0000 7f38a67fc640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:14.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.878+0000 7f38a67fc640 1 -- 172.21.15.40:0/2021100944 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f389808b7a0 con 0x7f38a80b10b0 2024-04-26T21:41:14.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.878+0000 7f38ad98c640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:14.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.879+0000 7f38ad98c640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f389406c8f0 0x7f389406edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f38a0034000 tx=0x7f38a002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:14.883+0000 7f38a67fc640 1 -- 172.21.15.40:0/2021100944 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3898020020 con 0x7f38a80b10b0 2024-04-26T21:41:15.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.108+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38a80cae20 con 0x7f38a80b10b0 2024-04-26T21:41:15.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.109+0000 7f38a67fc640 1 -- 172.21.15.40:0/2021100944 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5186 (secure 0 0 0) 0x7f389800ce20 con 0x7f38a80b10b0 2024-04-26T21:41:15.110 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:15.110 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:10.262121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3115383225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3115383225},{"type":"v1","addr":"172.21.15.40:6835","nonce":3115383225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7230,"qdb_cluster":[7230]},"id":5}]} 2024-04-26T21:41:15.110 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 71 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f389406c8f0 msgr2=0x7f389406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f389406c8f0 0x7f389406edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f38a0034000 tx=0x7f38a002e000 comp rx=0 tx=0).stop 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b10b0 msgr2=0x7f38a81e79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b10b0 0x7f38a81e79b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f389800c710 tx=0x7f389800cbd0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 shutdown_connections 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a80b10b0 0x7f38a81e79b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38a80b0380 0x7f38a81e9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 --2- 172.21.15.40:0/2021100944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38a806c780 0x7f38a80b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 >> 172.21.15.40:0/2021100944 conn(0x7f38a80d1cf0 msgr2=0x7f38a80b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 shutdown_connections 2024-04-26T21:41:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.111+0000 7f38afbf0640 1 -- 172.21.15.40:0/2021100944 wait complete. 2024-04-26T21:41:15.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 true 2024-04-26T21:41:15.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.224+0000 7f9ca2f78640 1 Processor -- start 2024-04-26T21:41:15.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.224+0000 7f9ca2f78640 1 -- start start 2024-04-26T21:41:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9ca2f78640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c9c0d3bc0 0x7f9c9c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9ca2f78640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9c0b7e50 con 0x7f9c9c162880 2024-04-26T21:41:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9ca2f78640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9c0d1800 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9ca2f78640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9c9c0d0580 con 0x7f9c9c0d3bc0 2024-04-26T21:41:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9ca0d14640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9c9c162880 0x7f9c9c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46670/0 (socket says 172.21.15.40:46670) 2024-04-26T21:41:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9ca0d14640 1 -- 172.21.15.40:0/2661921401 learned_addr learned my addr 172.21.15.40:0/2661921401 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9c9bfff640 1 --2- 172.21.15.40:0/2661921401 >> v2:172.21.15.40:3300/0 conn(0x7f9c9c0d3bc0 0x7f9c9c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9c9bfff640 1 -- 172.21.15.40:0/2661921401 >> v1:172.21.15.184:6789/0 conn(0x7f9c9c0b5f20 legacy=0x7f9c9c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:15.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9c9bfff640 1 -- 172.21.15.40:0/2661921401 >> v1:172.21.15.40:6789/0 conn(0x7f9c9c162880 legacy=0x7f9c9c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:15.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.225+0000 7f9c9bfff640 1 -- 172.21.15.40:0/2661921401 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c0d07d0 con 0x7f9c9c0d3bc0 2024-04-26T21:41:15.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.226+0000 7f9c9bfff640 1 --2- 172.21.15.40:0/2661921401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0d3bc0 0x7f9c9c0d2cb0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9c900097a0 tx=0x7f9c9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf4df352d024bea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:15.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.226+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/2661921401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90037070 con 0x7f9c9c0d3bc0 2024-04-26T21:41:15.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.226+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/2661921401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c90032440 con 0x7f9c9c0d3bc0 2024-04-26T21:41:15.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.226+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/2661921401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90032810 con 0x7f9c9c0d3bc0 2024-04-26T21:41:15.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.227+0000 7f9ca2f78640 1 -- 172.21.15.40:0/2661921401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0d3bc0 msgr2=0x7f9c9c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:15.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.227+0000 7f9ca2f78640 1 --2- 172.21.15.40:0/2661921401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0d3bc0 0x7f9c9c0d2cb0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9c900097a0 tx=0x7f9c9002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.227+0000 7f9ca2f78640 1 -- 172.21.15.40:0/2661921401 shutdown_connections 2024-04-26T21:41:15.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.227+0000 7f9ca2f78640 1 --2- 172.21.15.40:0/2661921401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0d3bc0 0x7f9c9c0d2cb0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.227+0000 7f9ca2f78640 1 -- 172.21.15.40:0/2661921401 >> 172.21.15.40:0/2661921401 conn(0x7f9c9c0d4f60 msgr2=0x7f9c9c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:15.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.227+0000 7f9ca2f78640 1 -- 172.21.15.40:0/2661921401 shutdown_connections 2024-04-26T21:41:15.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.227+0000 7f9ca2f78640 1 -- 172.21.15.40:0/2661921401 wait complete. 2024-04-26T21:41:15.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.228+0000 7f9ca2f78640 1 Processor -- start 2024-04-26T21:41:15.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.228+0000 7f9ca2f78640 1 -- start start 2024-04-26T21:41:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca2f78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0b5f20 0x7f9c9c1f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca2f78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c9c0d3bc0 0x7f9c9c1f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca2f78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c9c162880 0x7f9c9c0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca2f78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c162c60 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca2f78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c0d0d90 con 0x7f9c9c162880 2024-04-26T21:41:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca2f78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9c162de0 con 0x7f9c9c0d3bc0 2024-04-26T21:41:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca0d14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0b5f20 0x7f9c9c1f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca0d14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0b5f20 0x7f9c9c1f49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45328/0 (socket says 172.21.15.40:45328) 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca0d14640 1 -- 172.21.15.40:0/3326024186 learned_addr learned my addr 172.21.15.40:0/3326024186 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9c9bfff640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c9c0d3bc0 0x7f9c9c1f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca1515640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c9c162880 0x7f9c9c0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca0d14640 1 -- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c9c0d3bc0 msgr2=0x7f9c9c1f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca0d14640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c9c0d3bc0 0x7f9c9c1f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca0d14640 1 -- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c9c162880 msgr2=0x7f9c9c0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca0d14640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c9c162880 0x7f9c9c0b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca0d14640 1 -- 172.21.15.40:0/3326024186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c0b30d0 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9ca1515640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c9c162880 0x7f9c9c0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.229+0000 7f9c9bfff640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c9c0d3bc0 0x7f9c9c1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.230+0000 7f9ca0d14640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0b5f20 0x7f9c9c1f49e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c3c0 tx=0x7f9c8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:15.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.230+0000 7f9c997fa640 1 -- 172.21.15.40:0/3326024186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c017070 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.230+0000 7f9c997fa640 1 -- 172.21.15.40:0/3326024186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c005ce0 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.230+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c0b33b0 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.230+0000 7f9c997fa640 1 -- 172.21.15.40:0/3326024186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0072f0 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.230+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c0b3970 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.231+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9c0694f0 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.231+0000 7f9c997fa640 1 -- 172.21.15.40:0/3326024186 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c8c01a070 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.232+0000 7f9c997fa640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c8806c8f0 0x7f9c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:15.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.232+0000 7f9c997fa640 1 -- 172.21.15.40:0/3326024186 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f9c8c08afa0 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.232+0000 7f9c9bfff640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c8806c8f0 0x7f9c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:15.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.233+0000 7f9c9bfff640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c8806c8f0 0x7f9c8806edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9c90034000 tx=0x7f9c9002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:15.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.236+0000 7f9c997fa640 1 -- 172.21.15.40:0/3326024186 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c8c0592d0 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.496+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"} v 0) -- 0x7f9c9c0ce420 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.918+0000 7f9c997fa640 1 -- 172.21.15.40:0/3326024186 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"}]=0 freezing mds.7230 v72) ==== 112+0+0 (secure 0 0 0) 0x7f9c8c058bd0 con 0x7f9c9c0b5f20 2024-04-26T21:41:15.919 INFO:teuthology.orchestra.run.smithi040.stderr:freezing mds.7230 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.920+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c8806c8f0 msgr2=0x7f9c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.920+0000 7f9ca2f78640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c8806c8f0 0x7f9c8806edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9c90034000 tx=0x7f9c9002e000 comp rx=0 tx=0).stop 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.920+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0b5f20 msgr2=0x7f9c9c1f49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.920+0000 7f9ca2f78640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0b5f20 0x7f9c9c1f49e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c3c0 tx=0x7f9c8c00c880 comp rx=0 tx=0).stop 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.921+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 shutdown_connections 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.921+0000 7f9ca2f78640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c8806c8f0 0x7f9c8806edb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.921+0000 7f9ca2f78640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c9c162880 0x7f9c9c0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.921+0000 7f9ca2f78640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c9c0d3bc0 0x7f9c9c1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.921+0000 7f9ca2f78640 1 --2- 172.21.15.40:0/3326024186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c0b5f20 0x7f9c9c1f49e0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.921+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 >> 172.21.15.40:0/3326024186 conn(0x7f9c9c0d4f60 msgr2=0x7f9c9c06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.921+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 shutdown_connections 2024-04-26T21:41:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15.921+0000 7f9ca2f78640 1 -- 172.21.15.40:0/3326024186 wait complete. 2024-04-26T21:41:15.934 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d') 2024-04-26T21:41:15.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:41:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:41:16.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-04-26T21:41:16.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.103+0000 7f26eba45640 1 Processor -- start 2024-04-26T21:41:16.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.103+0000 7f26eba45640 1 -- start start 2024-04-26T21:41:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26eba45640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f26e4162880 0x7f26e40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26eba45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 0x7f26e40d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26eba45640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26e40d39f0 0x7f26e40b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26eba45640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f26e40d0580 con 0x7f26e40d39f0 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26eba45640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f26e40b7e50 con 0x7f26e4162880 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26eba45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26e40d1800 con 0x7f26e40b5f20 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e9fe2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26e40d39f0 0x7f26e40b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e9fe2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26e40d39f0 0x7f26e40b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45334/0 (socket says 172.21.15.40:45334) 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e9fe2640 1 -- 172.21.15.40:0/2529636310 learned_addr learned my addr 172.21.15.40:0/2529636310 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e97e1640 1 --2- 172.21.15.40:0/2529636310 >> v2:172.21.15.184:3300/0 conn(0x7f26e4162880 0x7f26e40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/2529636310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 0x7f26e40d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e9fe2640 1 -- 172.21.15.40:0/2529636310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 msgr2=0x7f26e40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/2529636310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 0x7f26e40d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e9fe2640 1 -- 172.21.15.40:0/2529636310 >> v2:172.21.15.184:3300/0 conn(0x7f26e4162880 msgr2=0x7f26e40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/2529636310 >> v2:172.21.15.184:3300/0 conn(0x7f26e4162880 0x7f26e40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.104+0000 7f26e9fe2640 1 -- 172.21.15.40:0/2529636310 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e40d07d0 con 0x7f26e40d39f0 2024-04-26T21:41:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.105+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/2529636310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e40d39f0 0x7f26e40b69e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f26e408d490 tx=0x7f26dc032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb4c231c9a279e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.105+0000 7f26dbfff640 1 -- 172.21.15.40:0/2529636310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0339f0 con 0x7f26e40d39f0 2024-04-26T21:41:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.105+0000 7f26dbfff640 1 -- 172.21.15.40:0/2529636310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26dc033b90 con 0x7f26e40d39f0 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.105+0000 7f26dbfff640 1 -- 172.21.15.40:0/2529636310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc03b3f0 con 0x7f26e40d39f0 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 -- 172.21.15.40:0/2529636310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e40d39f0 msgr2=0x7f26e40b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 --2- 172.21.15.40:0/2529636310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e40d39f0 0x7f26e40b69e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f26e408d490 tx=0x7f26dc032eb0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 -- 172.21.15.40:0/2529636310 shutdown_connections 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 --2- 172.21.15.40:0/2529636310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e40d39f0 0x7f26e40b69e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 --2- 172.21.15.40:0/2529636310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 0x7f26e40d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 --2- 172.21.15.40:0/2529636310 >> v2:172.21.15.184:3300/0 conn(0x7f26e4162880 0x7f26e40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 -- 172.21.15.40:0/2529636310 >> 172.21.15.40:0/2529636310 conn(0x7f26e40d4f60 msgr2=0x7f26e40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:16.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 -- 172.21.15.40:0/2529636310 shutdown_connections 2024-04-26T21:41:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.106+0000 7f26eba45640 1 -- 172.21.15.40:0/2529636310 wait complete. 2024-04-26T21:41:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.107+0000 7f26eba45640 1 Processor -- start 2024-04-26T21:41:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.107+0000 7f26eba45640 1 -- start start 2024-04-26T21:41:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26eba45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 0x7f26e41f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26eba45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e40d39f0 0x7f26e40b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26eba45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e4162880 0x7f26e40b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26eba45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4162c60 con 0x7f26e4162880 2024-04-26T21:41:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26eba45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26e40d0d90 con 0x7f26e40d39f0 2024-04-26T21:41:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26eba45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26e4162de0 con 0x7f26e40b5f20 2024-04-26T21:41:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e4162880 0x7f26e40b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e4162880 0x7f26e40b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45336/0 (socket says 172.21.15.40:45336) 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 -- 172.21.15.40:0/1068913716 learned_addr learned my addr 172.21.15.40:0/1068913716 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e97e1640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 0x7f26e41f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e40d39f0 0x7f26e40b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 -- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 msgr2=0x7f26e41f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 0x7f26e41f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 -- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e40d39f0 msgr2=0x7f26e40b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e40d39f0 0x7f26e40b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 -- 172.21.15.40:0/1068913716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e41490d0 con 0x7f26e4162880 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.108+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e4162880 0x7f26e40b69e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009f40 tx=0x7f26dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.109+0000 7f26d9ffb640 1 -- 172.21.15.40:0/1068913716 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc04a020 con 0x7f26e4162880 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.109+0000 7f26d9ffb640 1 -- 172.21.15.40:0/1068913716 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26dc045450 con 0x7f26e4162880 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.109+0000 7f26eba45640 1 -- 172.21.15.40:0/1068913716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e415a290 con 0x7f26e4162880 2024-04-26T21:41:16.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.109+0000 7f26d9ffb640 1 -- 172.21.15.40:0/1068913716 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc009510 con 0x7f26e4162880 2024-04-26T21:41:16.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.109+0000 7f26eba45640 1 -- 172.21.15.40:0/1068913716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e415a850 con 0x7f26e4162880 2024-04-26T21:41:16.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.110+0000 7f26d9ffb640 1 -- 172.21.15.40:0/1068913716 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26dc04d020 con 0x7f26e4162880 2024-04-26T21:41:16.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.111+0000 7f26d9ffb640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c806c8f0 0x7f26c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.111+0000 7f26d9ffb640 1 -- 172.21.15.40:0/1068913716 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f26dc0b5430 con 0x7f26e4162880 2024-04-26T21:41:16.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.111+0000 7f26e97e1640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c806c8f0 0x7f26c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.112+0000 7f26e97e1640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c806c8f0 0x7f26c806edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f26e0006fd0 tx=0x7f26e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.112+0000 7f26ee838740 1 Processor -- start 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.112+0000 7f26ee838740 1 -- start start 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26ee838740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc78f80 0x55a6bcc79360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26ee838740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc73e10 0x55a6bcc77b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26ee838740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc780c0 0x55a6bcc78500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26ee838740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a6bcbddd80 con 0x55a6bcc78f80 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26ee838740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a6bcb978b0 con 0x55a6bcc73e10 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26ee838740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a6bcbd9d40 con 0x55a6bcc780c0 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc78f80 0x55a6bcc79360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc78f80 0x55a6bcc79360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45346/0 (socket says 172.21.15.40:45346) 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 -- 172.21.15.40:0/2256692658 learned_addr learned my addr 172.21.15.40:0/2256692658 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc73e10 0x55a6bcc77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc780c0 0x55a6bcc78500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 -- 172.21.15.40:0/2256692658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc780c0 msgr2=0x55a6bcc78500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc780c0 0x55a6bcc78500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 -- 172.21.15.40:0/2256692658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc73e10 msgr2=0x55a6bcc77b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc73e10 0x55a6bcc77b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 -- 172.21.15.40:0/2256692658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6bcb68a40 con 0x55a6bcc78f80 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26e97e1640 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc78f80 0x55a6bcc79360 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007fd0 tx=0x7f26e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca212d6367d9487 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26ceffd640 1 -- 172.21.15.40:0/2256692658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e001c050 con 0x55a6bcc78f80 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.113+0000 7f26ceffd640 1 -- 172.21.15.40:0/2256692658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0026070 con 0x55a6bcc78f80 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ceffd640 1 -- 172.21.15.40:0/2256692658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0021440 con 0x55a6bcc78f80 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 -- 172.21.15.40:0/2256692658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc78f80 msgr2=0x55a6bcc79360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc78f80 0x55a6bcc79360 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007fd0 tx=0x7f26e0013000 comp rx=0 tx=0).stop 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 -- 172.21.15.40:0/2256692658 shutdown_connections 2024-04-26T21:41:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc780c0 0x55a6bcc78500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc73e10 0x55a6bcc77b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 --2- 172.21.15.40:0/2256692658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc78f80 0x55a6bcc79360 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 -- 172.21.15.40:0/2256692658 >> 172.21.15.40:0/2256692658 conn(0x55a6bcc76080 msgr2=0x55a6bccbeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 -- 172.21.15.40:0/2256692658 shutdown_connections 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.114+0000 7f26ee838740 1 -- 172.21.15.40:0/2256692658 wait complete. 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.115+0000 7f26ee838740 1 -- start start 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.115+0000 7f26ee838740 1 Processor -- start 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26ee838740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26ee838740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bccbdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26ee838740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bccc4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26ee838740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a6bcca4b10 con 0x55a6bcc780c0 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26ee838740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a6bcb64d10 con 0x55a6bcc89b70 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26ee838740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a6bcca3f90 con 0x55a6bcc78f80 2024-04-26T21:41:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26ee838740 10 client.0 _ll_register_callbacks cb 0x55a6bcb91210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45356/0 (socket says 172.21.15.40:45356) 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 -- 172.21.15.40:0/879660840 learned_addr learned my addr 172.21.15.40:0/879660840 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bccbdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bccc4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 -- 172.21.15.40:0/879660840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 msgr2=0x55a6bccbdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bccbdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 -- 172.21.15.40:0/879660840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 msgr2=0x55a6bccc4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bccc4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 -- 172.21.15.40:0/879660840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6bc9b97d0 con 0x55a6bcc780c0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26e97e1640 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc5f00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f26e0004310 tx=0x7f26e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf8ca99f99ad2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26c3fff640 1 -- 172.21.15.40:0/879660840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00044f0 con 0x55a6bcc780c0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26c3fff640 1 -- 172.21.15.40:0/879660840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f26e0034c40 con 0x55a6bcc780c0 2024-04-26T21:41:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.116+0000 7f26ee838740 1 -- 172.21.15.40:0/879660840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e0026070 con 0x55a6bcc780c0 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26c3fff640 1 -- 172.21.15.40:0/879660840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f26e001c020 con 0x55a6bcc780c0 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26ee838740 10 client.7422 fetch_fsmap learned FSMap version 72 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26ee838740 1 -- 172.21.15.40:0/879660840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55a6bc8ef3f0 con 0x55a6bcc780c0 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26c3fff640 1 -- 172.21.15.40:0/879660840 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0026070 con 0x55a6bcc780c0 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26c3fff640 1 -- 172.21.15.40:0/879660840 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7f26e0003820 con 0x55a6bcc780c0 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26ee838740 10 client.7422 fetch_fsmap finished waiting for FSMap version 72 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26ee838740 10 client.7422 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26ee838740 20 client.7422 populate_metadata read hostname 'smithi040' 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26ee838740 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55a6bcc89ff0 0x55a6bccc0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26ee838740 4 client.7422 mds_command: new command op to 7230 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26ee838740 1 -- 172.21.15.40:0/879660840 --> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6bcbeb990 con 0x55a6bcc89ff0 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55a6bcc89ff0 0x55a6bccc0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26c3fff640 10 client.7422 ms_handle_connect on v2:172.21.15.40:6834/3115383225 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.117+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55a6bcc89ff0 0x55a6bccc0cc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5937e49b64250990 server_cookie=78121f1efb2b9f5e in_seq=0 out_seq=0 2024-04-26T21:41:16.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26c3fff640 1 -- 172.21.15.40:0/879660840 <== mds.0 v2:172.21.15.40:6834/3115383225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6bcbeb990 con 0x55a6bcc89ff0 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26c3fff640 10 client.7422 handle_command_reply: tid=0 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 client.7422 shutdown 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 -- 172.21.15.40:0/879660840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 msgr2=0x55a6bccc5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc5f00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f26e0004310 tx=0x7f26e0004730 comp rx=0 tx=0).stop 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 -- 172.21.15.40:0/879660840 shutdown_connections 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55a6bcc89ff0 0x55a6bccc0cc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bccc4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bccbdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26c3fff640 0 client.7422 ms_handle_reset on v2:172.21.15.40:6834/3115383225 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 --2- 172.21.15.40:0/879660840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc5f00 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 -- 172.21.15.40:0/879660840 >> 172.21.15.40:0/879660840 conn(0x55a6bcc73e10 msgr2=0x55a6bcc73550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 -- 172.21.15.40:0/879660840 shutdown_connections 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 1 -- 172.21.15.40:0/879660840 wait complete. 2024-04-26T21:41:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.118+0000 7f26ee838740 20 client.7422 trim_cache size 0 max 16384 2024-04-26T21:41:16.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.133+0000 7f26ee838740 1 Processor -- start 2024-04-26T21:41:16.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.133+0000 7f26ee838740 1 -- start start 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26ee838740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bcc9b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26ee838740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc99da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26ee838740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc9a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26ee838740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41fd950 con 0x55a6bcc780c0 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26ee838740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41fc950 con 0x55a6bcc89b70 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26ee838740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26e40cecb0 con 0x55a6bcc78f80 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e97e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bcc9b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e97e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bcc9b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45368/0 (socket says 172.21.15.40:45368) 2024-04-26T21:41:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e97e1640 1 -- 172.21.15.40:0/4025458861 learned_addr learned my addr 172.21.15.40:0/4025458861 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e97e1640 1 -- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 msgr2=0x55a6bcc99da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc9a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc99da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e97e1640 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc99da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e97e1640 1 -- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 msgr2=0x55a6bcc9a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e97e1640 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc9a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e97e1640 1 -- 172.21.15.40:0/4025458861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6bc8af0f0 con 0x55a6bcc780c0 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc9a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.134+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc99da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26e97e1640 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bcc9b380 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f26e0003ae0 tx=0x7f26e002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3e019a1abee3c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ce7fc640 1 -- 172.21.15.40:0/4025458861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0004340 con 0x55a6bcc780c0 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ce7fc640 1 -- 172.21.15.40:0/4025458861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0026040 con 0x55a6bcc780c0 2024-04-26T21:41:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ce7fc640 1 -- 172.21.15.40:0/4025458861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e003d680 con 0x55a6bcc780c0 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 -- 172.21.15.40:0/4025458861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 msgr2=0x55a6bcc9b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bcc9b380 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f26e0003ae0 tx=0x7f26e002c000 comp rx=0 tx=0).stop 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 -- 172.21.15.40:0/4025458861 shutdown_connections 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc9a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc99da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 --2- 172.21.15.40:0/4025458861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bcc9b380 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 -- 172.21.15.40:0/4025458861 >> 172.21.15.40:0/4025458861 conn(0x55a6bcc73e10 msgr2=0x55a6bccc2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 -- 172.21.15.40:0/4025458861 shutdown_connections 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.135+0000 7f26ee838740 1 -- 172.21.15.40:0/4025458861 wait complete. 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.136+0000 7f26ee838740 1 -- start start 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.136+0000 7f26ee838740 1 Processor -- start 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26ee838740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26ee838740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc9ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26ee838740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc99cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26ee838740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a6bccb5130 con 0x55a6bcc780c0 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26ee838740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41fc300 con 0x55a6bcc89b70 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26ee838740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a6bccc6290 con 0x55a6bcc78f80 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26ee838740 10 client.0 _ll_register_callbacks cb 0x7f26d4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45376/0 (socket says 172.21.15.40:45376) 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 -- 172.21.15.40:0/1248745872 learned_addr learned my addr 172.21.15.40:0/1248745872 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc9ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e9fe2640 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc99cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 -- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 msgr2=0x55a6bcc9ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc9ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 -- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 msgr2=0x55a6bcc99cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc99cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 -- 172.21.15.40:0/1248745872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6bcb9e4b0 con 0x55a6bcc780c0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc9ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26e97e1640 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc0ad0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f26e00106c0 tx=0x7f26e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b972ed356acb22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.137+0000 7f26c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26c3fff640 1 -- 172.21.15.40:0/1248745872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0005b90 con 0x55a6bcc780c0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26c3fff640 1 -- 172.21.15.40:0/1248745872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f26e0005d30 con 0x55a6bcc780c0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 1 -- 172.21.15.40:0/1248745872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e0026040 con 0x55a6bcc780c0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26c3fff640 1 -- 172.21.15.40:0/1248745872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f26e004e050 con 0x55a6bcc780c0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 10 client.7428 fetch_fsmap learned FSMap version 72 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 1 -- 172.21.15.40:0/1248745872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55a6bccb4ca0 con 0x55a6bcc780c0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26c3fff640 1 -- 172.21.15.40:0/1248745872 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0026040 con 0x55a6bcc780c0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26c3fff640 1 -- 172.21.15.40:0/1248745872 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7f26e0051020 con 0x55a6bcc780c0 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 10 client.7428 fetch_fsmap finished waiting for FSMap version 72 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 10 client.7428 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-26T21:41:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 20 client.7428 populate_metadata read hostname 'smithi040' 2024-04-26T21:41:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55a6bcc9f2c0 0x55a6bcc9a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 4 client.7428 mds_command: new command op to 7230 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T21:41:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26ee838740 1 -- 172.21.15.40:0/1248745872 --> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6bcbe5800 con 0x55a6bcc9f2c0 2024-04-26T21:41:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.138+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55a6bcc9f2c0 0x55a6bcc9a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.139+0000 7f26c3fff640 10 client.7428 ms_handle_connect on v2:172.21.15.40:6834/3115383225 2024-04-26T21:41:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.139+0000 7f26e8fe0640 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55a6bcc9f2c0 0x55a6bcc9a230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edfb0d5442679c86 server_cookie=5ff348c8ba934e1f in_seq=0 out_seq=0 2024-04-26T21:41:16.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26c3fff640 1 -- 172.21.15.40:0/1248745872 <== mds.0 v2:172.21.15.40:6834/3115383225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6bcbe5800 con 0x55a6bcc9f2c0 2024-04-26T21:41:16.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26c3fff640 10 client.7428 handle_command_reply: tid=0 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 client.7428 shutdown 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 -- 172.21.15.40:0/1248745872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 msgr2=0x55a6bccc0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc0ad0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f26e00106c0 tx=0x7f26e0016000 comp rx=0 tx=0).stop 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 -- 172.21.15.40:0/1248745872 shutdown_connections 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.40:6834/3115383225,v1:172.21.15.40:6835/3115383225] conn(0x55a6bcc9f2c0 0x55a6bcc9a230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bcc89b70 0x55a6bcc99cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a6bcc78f80 0x55a6bcc9ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26c3fff640 0 client.7428 ms_handle_reset on v2:172.21.15.40:6834/3115383225 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 --2- 172.21.15.40:0/1248745872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a6bcc780c0 0x55a6bccc0ad0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 -- 172.21.15.40:0/1248745872 >> 172.21.15.40:0/1248745872 conn(0x55a6bcc73e10 msgr2=0x55a6bcc6f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 -- 172.21.15.40:0/1248745872 shutdown_connections 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 1 -- 172.21.15.40:0/1248745872 wait complete. 2024-04-26T21:41:16.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.153+0000 7f26ee838740 20 client.7428 trim_cache size 0 max 16384 2024-04-26T21:41:16.156 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:41:16.157 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-04-26T21:41:16.157 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0 2024-04-26T21:41:16.157 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 -- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c806c8f0 msgr2=0x7f26c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c806c8f0 0x7f26c806edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f26e0006fd0 tx=0x7f26e0008040 comp rx=0 tx=0).stop 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 -- 172.21.15.40:0/1068913716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e4162880 msgr2=0x7f26e40b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e4162880 0x7f26e40b69e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009f40 tx=0x7f26dc004a40 comp rx=0 tx=0).stop 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 -- 172.21.15.40:0/1068913716 shutdown_connections 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f26c806c8f0 0x7f26c806edb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e4162880 0x7f26e40b69e0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26e40d39f0 0x7f26e40b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 --2- 172.21.15.40:0/1068913716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26e40b5f20 0x7f26e41f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 -- 172.21.15.40:0/1068913716 >> 172.21.15.40:0/1068913716 conn(0x7f26e40d4f60 msgr2=0x7f26e40cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 -- 172.21.15.40:0/1068913716 shutdown_connections 2024-04-26T21:41:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.154+0000 7f26eba45640 1 -- 172.21.15.40:0/1068913716 wait complete. 2024-04-26T21:41:16.165 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-04-26T21:41:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.269+0000 7f0da8722640 1 Processor -- start 2024-04-26T21:41:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.269+0000 7f0da8722640 1 -- start start 2024-04-26T21:41:16.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da8722640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0da015f170 0x7f0da00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da8722640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da015f550 0x7f0da00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da8722640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da00d1040 con 0x7f0da00d34b0 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da8722640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0da00c9430 con 0x7f0da015f170 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da8722640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0da00c9680 con 0x7f0da015f550 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da6cbf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0da00d34b0 0x7f0da00cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46686/0 (socket says 172.21.15.40:46686) 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da6cbf640 1 -- 172.21.15.40:0/1899544724 learned_addr learned my addr 172.21.15.40:0/1899544724 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da64be640 1 --2- 172.21.15.40:0/1899544724 >> v2:172.21.15.184:3300/0 conn(0x7f0da015f170 0x7f0da00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da5cbd640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da015f550 0x7f0da00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3658925552 0 0) 0x7f0d98001490 con 0x7f0da00d34b0 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d8c003520 con 0x7f0da00d34b0 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 905714662 0 0) 0x7f0d8c003520 con 0x7f0da00d34b0 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da015f550 msgr2=0x7f0da00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da015f550 0x7f0da00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 >> v2:172.21.15.184:3300/0 conn(0x7f0da015f170 msgr2=0x7f0da00d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 --2- 172.21.15.40:0/1899544724 >> v2:172.21.15.184:3300/0 conn(0x7f0da015f170 0x7f0da00d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da00c9960 con 0x7f0da00d34b0 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f0d98001bc0 con 0x7f0da00d34b0 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 >> v1:172.21.15.40:6789/0 conn(0x7f0da00d34b0 legacy=0x7f0da00cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0d8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c004800 0x7f0d8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c005180 0x7f0d8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0da00c9430 con 0x7f0d8c0039b0 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0da00c9680 con 0x7f0d8c005180 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.270+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c004600 con 0x7f0d8c004800 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da64be640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0d8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da5cbd640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c004800 0x7f0d8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da6cbf640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c005180 0x7f0d8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da64be640 1 -- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c004800 msgr2=0x7f0d8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da64be640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c004800 0x7f0d8c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da64be640 1 -- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c005180 msgr2=0x7f0d8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da64be640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c005180 0x7f0d8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da64be640 1 -- 172.21.15.40:0/1899544724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c003520 con 0x7f0d8c0039b0 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da6cbf640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c005180 0x7f0d8c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.271+0000 7f0da64be640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0d8c003dd0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009ac0 tx=0x7f0d8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8400d83d16e80b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c036070 con 0x7f0d8c0039b0 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c031400 con 0x7f0d8c0039b0 2024-04-26T21:41:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da54bc640 1 -- 172.21.15.40:0/1899544724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c009690 con 0x7f0d8c0039b0 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da8722640 1 -- 172.21.15.40:0/1899544724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 msgr2=0x7f0d8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da8722640 1 --2- 172.21.15.40:0/1899544724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0d8c003dd0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009ac0 tx=0x7f0d8c005830 comp rx=0 tx=0).stop 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da6cbf640 1 -- 172.21.15.40:0/1899544724 reap_dead start 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da8722640 1 -- 172.21.15.40:0/1899544724 shutdown_connections 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da8722640 1 -- 172.21.15.40:0/1899544724 >> 172.21.15.40:0/1899544724 conn(0x7f0da00d4f60 msgr2=0x7f0da00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da8722640 1 -- 172.21.15.40:0/1899544724 shutdown_connections 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.272+0000 7f0da8722640 1 -- 172.21.15.40:0/1899544724 wait complete. 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.273+0000 7f0da8722640 1 Processor -- start 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.273+0000 7f0da8722640 1 -- start start 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da8722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0da015b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da8722640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da0151250 0x7f0da00b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da8722640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0da01516d0 0x7f0da00ba290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da8722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0da00cb5d0 con 0x7f0d8c0039b0 2024-04-26T21:41:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da8722640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0da00cb450 con 0x7f0da01516d0 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da8722640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0da00cb750 con 0x7f0da0151250 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da64be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0da015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da64be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0da015b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45394/0 (socket says 172.21.15.40:45394) 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.274+0000 7f0da64be640 1 -- 172.21.15.40:0/1338786309 learned_addr learned my addr 172.21.15.40:0/1338786309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da5cbd640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da0151250 0x7f0da00b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da64be640 1 -- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da0151250 msgr2=0x7f0da00b9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da6cbf640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0da01516d0 0x7f0da00ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da64be640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da0151250 0x7f0da00b9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da64be640 1 -- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0da01516d0 msgr2=0x7f0da00ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da64be640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0da01516d0 0x7f0da00ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da64be640 1 -- 172.21.15.40:0/1338786309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da00cd500 con 0x7f0d8c0039b0 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da64be640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0da015b090 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c034040 tx=0x7f0d9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0d96ffd640 1 -- 172.21.15.40:0/1338786309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c040020 con 0x7f0d8c0039b0 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0d96ffd640 1 -- 172.21.15.40:0/1338786309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c003950 con 0x7f0d8c0039b0 2024-04-26T21:41:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da015a2d0 con 0x7f0d8c0039b0 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0d96ffd640 1 -- 172.21.15.40:0/1338786309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c036040 con 0x7f0d8c0039b0 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.275+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da015a890 con 0x7f0d8c0039b0 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.276+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da00694f0 con 0x7f0d8c0039b0 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.277+0000 7f0d96ffd640 1 -- 172.21.15.40:0/1338786309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d9c043020 con 0x7f0d8c0039b0 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.277+0000 7f0d96ffd640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d8406c8f0 0x7f0d8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.277+0000 7f0d96ffd640 1 -- 172.21.15.40:0/1338786309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f0d9c0abe80 con 0x7f0d8c0039b0 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.277+0000 7f0da5cbd640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d8406c8f0 0x7f0d8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.278+0000 7f0da5cbd640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d8406c8f0 0x7f0d8406edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0da00c8f40 tx=0x7f0d90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.281+0000 7f0d96ffd640 1 -- 172.21.15.40:0/1338786309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d9c04a070 con 0x7f0d8c0039b0 2024-04-26T21:41:16.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.467+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7f0da0074590 con 0x7f0d8c0039b0 2024-04-26T21:41:16.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.470+0000 7f0d96ffd640 1 -- 172.21.15.40:0/1338786309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v23)=0 v23) ==== 138+0+0 (secure 0 0 0) 0x7f0da00cd500 con 0x7f0d8c0039b0 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d8406c8f0 msgr2=0x7f0d8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d8406c8f0 0x7f0d8406edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0da00c8f40 tx=0x7f0d90008040 comp rx=0 tx=0).stop 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 msgr2=0x7f0da015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0da015b090 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c034040 tx=0x7f0d9c004730 comp rx=0 tx=0).stop 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 shutdown_connections 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d8406c8f0 0x7f0d8406edb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0da01516d0 0x7f0da00ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0da0151250 0x7f0da00b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 --2- 172.21.15.40:0/1338786309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d8c0039b0 0x7f0da015b090 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 >> 172.21.15.40:0/1338786309 conn(0x7f0da00d4f60 msgr2=0x7f0da00d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 shutdown_connections 2024-04-26T21:41:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:16.472+0000 7f0da8722640 1 -- 172.21.15.40:0/1338786309 wait complete. 2024-04-26T21:41:21.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-26T21:41:21.487 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'timeout 60 mv a/b a/z') 2024-04-26T21:41:21.489 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-26T21:41:21.552 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:41:21.551+0000 7fdecdec4640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7395:47 nref=7 cr=0x55f7e0c2ea80)) (dversion lock w=1 last_client=7395) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55f7e0ef6780] 2024-04-26T21:41:21.552 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:41:21.551+0000 7fdecdec4640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7395:47 nref=7 cr=0x55f7e0c2ea80)) (dversion lock w=1 last_client=7395) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55f7e0ef6780] 2024-04-26T21:41:21.552 INFO:tasks.ceph.mds.c.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fdecdec4640 time 2024-04-26T21:41:21.552588+0000 2024-04-26T21:41:21.552 INFO:tasks.ceph.mds.c.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7fded3b825ed] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55f7dd617c21] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55f7dd83d514] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 4: ceph-mds(+0x1c1a49) [0x55f7dd65ca49] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x55f7dd6e32bf] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2655) [0x55f7dd6e6605] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x55f7dd6ace13] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x55f7dd69b5fc] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55f7dd61f15b] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f7dd61f62c] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x196) [0x55f7dd6030f6] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 12: (DispatchQueue::entry()+0x542) [0x7fded3d81832] 2024-04-26T21:41:21.553 INFO:tasks.ceph.mds.c.smithi040.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x415c71) [0x7fded3e15c71] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 14: /lib64/libc.so.6(+0x89c02) [0x7fded3289c02] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 15: /lib64/libc.so.6(+0x10ec40) [0x7fded330ec40] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:41:21.552+0000 7fdecdec4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fdecdec4640 time 2024-04-26T21:41:21.552588+0000 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7fded3b825ed] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55f7dd617c21] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55f7dd83d514] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 4: ceph-mds(+0x1c1a49) [0x55f7dd65ca49] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x55f7dd6e32bf] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2655) [0x55f7dd6e6605] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x55f7dd6ace13] 2024-04-26T21:41:21.554 INFO:tasks.ceph.mds.c.smithi040.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x55f7dd69b5fc] 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55f7dd61f15b] 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f7dd61f62c] 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x196) [0x55f7dd6030f6] 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: 12: (DispatchQueue::entry()+0x542) [0x7fded3d81832] 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x415c71) [0x7fded3e15c71] 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: 14: /lib64/libc.so.6(+0x89c02) [0x7fded3289c02] 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: 15: /lib64/libc.so.6(+0x10ec40) [0x7fded330ec40] 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr:*** Caught signal (Aborted) ** 2024-04-26T21:41:21.555 INFO:tasks.ceph.mds.c.smithi040.stderr: in thread 7fdecdec4640 thread_name:ms_dispatch 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fded323e6f0] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fded328b94c] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 3: raise() 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 4: abort() 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7fded3b826aa] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55f7dd617c21] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55f7dd83d514] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 8: ceph-mds(+0x1c1a49) [0x55f7dd65ca49] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x55f7dd6e32bf] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2655) [0x55f7dd6e6605] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x55f7dd6ace13] 2024-04-26T21:41:21.556 INFO:tasks.ceph.mds.c.smithi040.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x55f7dd69b5fc] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55f7dd61f15b] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f7dd61f62c] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x196) [0x55f7dd6030f6] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 16: (DispatchQueue::entry()+0x542) [0x7fded3d81832] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x415c71) [0x7fded3e15c71] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 18: /lib64/libc.so.6(+0x89c02) [0x7fded3289c02] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 19: /lib64/libc.so.6(+0x10ec40) [0x7fded330ec40] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:41:21.553+0000 7fdecdec4640 -1 *** Caught signal (Aborted) ** 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: in thread 7fdecdec4640 thread_name:ms_dispatch 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fded323e6f0] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fded328b94c] 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 3: raise() 2024-04-26T21:41:21.557 INFO:tasks.ceph.mds.c.smithi040.stderr: 4: abort() 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7fded3b826aa] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55f7dd617c21] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55f7dd83d514] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 8: ceph-mds(+0x1c1a49) [0x55f7dd65ca49] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x55f7dd6e32bf] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2655) [0x55f7dd6e6605] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x55f7dd6ace13] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x55f7dd69b5fc] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55f7dd61f15b] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f7dd61f62c] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x196) [0x55f7dd6030f6] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 16: (DispatchQueue::entry()+0x542) [0x7fded3d81832] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x415c71) [0x7fded3e15c71] 2024-04-26T21:41:21.558 INFO:tasks.ceph.mds.c.smithi040.stderr: 18: /lib64/libc.so.6(+0x89c02) [0x7fded3289c02] 2024-04-26T21:41:21.559 INFO:tasks.ceph.mds.c.smithi040.stderr: 19: /lib64/libc.so.6(+0x10ec40) [0x7fded330ec40] 2024-04-26T21:41:21.559 INFO:tasks.ceph.mds.c.smithi040.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T21:41:21.559 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.559 INFO:tasks.ceph.mds.c.smithi040.stderr: -4900> 2024-04-26T21:41:09.850+0000 7fdecb6bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr: -8> 2024-04-26T21:41:21.551+0000 7fdecdec4640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7395:47 nref=7 cr=0x55f7e0c2ea80)) (dversion lock w=1 last_client=7395) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55f7e0ef6780] 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr: -6> 2024-04-26T21:41:21.551+0000 7fdecdec4640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7395:47 nref=7 cr=0x55f7e0c2ea80)) (dversion lock w=1 last_client=7395) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55f7e0ef6780] 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr: -1> 2024-04-26T21:41:21.552+0000 7fdecdec4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fdecdec4640 time 2024-04-26T21:41:21.552588+0000 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7fded3b825ed] 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55f7dd617c21] 2024-04-26T21:41:21.564 INFO:tasks.ceph.mds.c.smithi040.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55f7dd83d514] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 4: ceph-mds(+0x1c1a49) [0x55f7dd65ca49] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x55f7dd6e32bf] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2655) [0x55f7dd6e6605] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x55f7dd6ace13] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x55f7dd69b5fc] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55f7dd61f15b] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f7dd61f62c] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x196) [0x55f7dd6030f6] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 12: (DispatchQueue::entry()+0x542) [0x7fded3d81832] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x415c71) [0x7fded3e15c71] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 14: /lib64/libc.so.6(+0x89c02) [0x7fded3289c02] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 15: /lib64/libc.so.6(+0x10ec40) [0x7fded330ec40] 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 0> 2024-04-26T21:41:21.553+0000 7fdecdec4640 -1 *** Caught signal (Aborted) ** 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: in thread 7fdecdec4640 thread_name:ms_dispatch 2024-04-26T21:41:21.565 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fded323e6f0] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fded328b94c] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 3: raise() 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 4: abort() 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7fded3b826aa] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55f7dd617c21] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55f7dd83d514] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 8: ceph-mds(+0x1c1a49) [0x55f7dd65ca49] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x55f7dd6e32bf] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2655) [0x55f7dd6e6605] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x55f7dd6ace13] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x55f7dd69b5fc] 2024-04-26T21:41:21.566 INFO:tasks.ceph.mds.c.smithi040.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55f7dd61f15b] 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f7dd61f62c] 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x196) [0x55f7dd6030f6] 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: 16: (DispatchQueue::entry()+0x542) [0x7fded3d81832] 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x415c71) [0x7fded3e15c71] 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: 18: /lib64/libc.so.6(+0x89c02) [0x7fded3289c02] 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: 19: /lib64/libc.so.6(+0x10ec40) [0x7fded330ec40] 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.567 INFO:tasks.ceph.mds.c.smithi040.stderr: -4900> 2024-04-26T21:41:09.850+0000 7fdecb6bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: -8> 2024-04-26T21:41:21.551+0000 7fdecdec4640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7395:47 nref=7 cr=0x55f7e0c2ea80)) (dversion lock w=1 last_client=7395) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55f7e0ef6780] 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: -6> 2024-04-26T21:41:21.551+0000 7fdecdec4640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7395:47 nref=7 cr=0x55f7e0c2ea80)) (dversion lock w=1 last_client=7395) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55f7e0ef6780] 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: -1> 2024-04-26T21:41:21.552+0000 7fdecdec4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fdecdec4640 time 2024-04-26T21:41:21.552588+0000 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3306-gbc612ffb/rpm/el9/BUILD/ceph-19.0.0-3306-gbc612ffb/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7fded3b825ed] 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55f7dd617c21] 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55f7dd83d514] 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: 4: ceph-mds(+0x1c1a49) [0x55f7dd65ca49] 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x55f7dd6e32bf] 2024-04-26T21:41:21.571 INFO:tasks.ceph.mds.c.smithi040.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2655) [0x55f7dd6e6605] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x55f7dd6ace13] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x55f7dd69b5fc] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55f7dd61f15b] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f7dd61f62c] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x196) [0x55f7dd6030f6] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 12: (DispatchQueue::entry()+0x542) [0x7fded3d81832] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x415c71) [0x7fded3e15c71] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 14: /lib64/libc.so.6(+0x89c02) [0x7fded3289c02] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 15: /lib64/libc.so.6(+0x10ec40) [0x7fded330ec40] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 0> 2024-04-26T21:41:21.553+0000 7fdecdec4640 -1 *** Caught signal (Aborted) ** 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: in thread 7fdecdec4640 thread_name:ms_dispatch 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fded323e6f0] 2024-04-26T21:41:21.572 INFO:tasks.ceph.mds.c.smithi040.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fded328b94c] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 3: raise() 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 4: abort() 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7fded3b826aa] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55f7dd617c21] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55f7dd83d514] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 8: ceph-mds(+0x1c1a49) [0x55f7dd65ca49] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x55f7dd6e32bf] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2655) [0x55f7dd6e6605] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x55f7dd6ace13] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x55f7dd69b5fc] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55f7dd61f15b] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f7dd61f62c] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x196) [0x55f7dd6030f6] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 16: (DispatchQueue::entry()+0x542) [0x7fded3d81832] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x415c71) [0x7fded3e15c71] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 18: /lib64/libc.so.6(+0x89c02) [0x7fded3289c02] 2024-04-26T21:41:21.573 INFO:tasks.ceph.mds.c.smithi040.stderr: 19: /lib64/libc.so.6(+0x10ec40) [0x7fded330ec40] 2024-04-26T21:41:21.574 INFO:tasks.ceph.mds.c.smithi040.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T21:41:21.574 INFO:tasks.ceph.mds.c.smithi040.stderr: 2024-04-26T21:41:21.614 INFO:teuthology.orchestra.run.smithi040.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-26T21:41:21.614 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-26T21:41:21.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.628+0000 7f0e22264640 1 Processor -- start 2024-04-26T21:41:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.629+0000 7f0e22264640 1 -- start start 2024-04-26T21:41:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.629+0000 7f0e22264640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c1602c0 0x7f0e1c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.629+0000 7f0e22264640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0e1c06eff0 0x7f0e1c06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.629+0000 7f0e22264640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c15d810 con 0x7f0e1c0716c0 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.629+0000 7f0e22264640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c15edd0 con 0x7f0e1c06eff0 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.629+0000 7f0e22264640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c15daf0 con 0x7f0e1c1602c0 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e20801640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0e1c0716c0 0x7f0e1c0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50130/0 (socket says 172.21.15.40:50130) 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e20801640 1 -- 172.21.15.40:0/2944118134 learned_addr learned my addr 172.21.15.40:0/2944118134 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1bfff640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c1602c0 0x7f0e1c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/2944118134 >> v2:172.21.15.184:3300/0 conn(0x7f0e1c06eff0 0x7f0e1c06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2532088712 0 0) 0x7f0e1c15d810 con 0x7f0e1c0716c0 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e08003520 con 0x7f0e1c0716c0 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1174145945 0 0) 0x7f0e08003520 con 0x7f0e1c0716c0 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c1602c0 msgr2=0x7f0e1c15e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c1602c0 0x7f0e1c15e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 >> v2:172.21.15.184:3300/0 conn(0x7f0e1c06eff0 msgr2=0x7f0e1c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 --2- 172.21.15.40:0/2944118134 >> v2:172.21.15.184:3300/0 conn(0x7f0e1c06eff0 0x7f0e1c06c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c15ddd0 con 0x7f0e1c0716c0 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f0e140019f0 con 0x7f0e1c0716c0 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/2944118134 >> v2:172.21.15.184:3300/0 conn(0x7f0e1c06eff0 0x7f0e1c06c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 >> v1:172.21.15.40:6789/0 conn(0x7f0e1c0716c0 legacy=0x7f0e1c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e080039b0 0x7f0e08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 0x7f0e08004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e08004ef0 0x7f0e0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15edd0 con 0x7f0e080043a0 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15daf0 con 0x7f0e08004ef0 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0800bf00 con 0x7f0e080039b0 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.630+0000 7f0e1bfff640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c1602c0 0x7f0e1c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.631+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 0x7f0e08004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.631+0000 7f0e20801640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e08004ef0 0x7f0e0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.631+0000 7f0e1bfff640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e080039b0 0x7f0e08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.631+0000 7f0e1b7fe640 1 -- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e080039b0 msgr2=0x7f0e08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.631+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e080039b0 0x7f0e08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.631+0000 7f0e1b7fe640 1 -- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e08004ef0 msgr2=0x7f0e0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.631+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e08004ef0 0x7f0e0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.631+0000 7f0e1b7fe640 1 -- 172.21.15.40:0/2944118134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e08003520 con 0x7f0e080043a0 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.632+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 0x7f0e08004880 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009c90 tx=0x7f0e08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394726617735babc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.632+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04036070 con 0x7f0e080043a0 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.632+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e04031400 con 0x7f0e080043a0 2024-04-26T21:41:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.632+0000 7f0e1affd640 1 -- 172.21.15.40:0/2944118134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04009860 con 0x7f0e080043a0 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.632+0000 7f0e22264640 1 -- 172.21.15.40:0/2944118134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 msgr2=0x7f0e08004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.632+0000 7f0e22264640 1 --2- 172.21.15.40:0/2944118134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 0x7f0e08004880 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009c90 tx=0x7f0e08005830 comp rx=0 tx=0).stop 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.632+0000 7f0e20801640 1 -- 172.21.15.40:0/2944118134 reap_dead start 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.632+0000 7f0e22264640 1 -- 172.21.15.40:0/2944118134 shutdown_connections 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.633+0000 7f0e22264640 1 -- 172.21.15.40:0/2944118134 >> 172.21.15.40:0/2944118134 conn(0x7f0e1c06db40 msgr2=0x7f0e1c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.633+0000 7f0e22264640 1 -- 172.21.15.40:0/2944118134 shutdown_connections 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.633+0000 7f0e22264640 1 -- 172.21.15.40:0/2944118134 wait complete. 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.634+0000 7f0e22264640 1 Processor -- start 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.634+0000 7f0e22264640 1 -- start start 2024-04-26T21:41:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e22264640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 0x7f0e1c152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e22264640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c0ce8e0 0x7f0e1c1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e22264640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c0cff50 0x7f0e1c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e22264640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15e390 con 0x7f0e080043a0 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e22264640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15e0b0 con 0x7f0e1c0ce8e0 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e22264640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c15d470 con 0x7f0e1c0cff50 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 0x7f0e1c152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 0x7f0e1c152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39896/0 (socket says 172.21.15.40:39896) 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 -- 172.21.15.40:0/4197244006 learned_addr learned my addr 172.21.15.40:0/4197244006 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e20801640 1 --2- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c0cff50 0x7f0e1c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c0ce8e0 0x7f0e1c1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 -- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c0cff50 msgr2=0x7f0e1c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 --2- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e1c0cff50 0x7f0e1c0d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 -- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c0ce8e0 msgr2=0x7f0e1c1532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 --2- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c0ce8e0 0x7f0e1c1532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 -- 172.21.15.40:0/4197244006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c152ea0 con 0x7f0e080043a0 2024-04-26T21:41:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.635+0000 7f0e1bfff640 1 --2- 172.21.15.40:0/4197244006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e080043a0 0x7f0e1c152960 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0119d0 tx=0x7f0e0c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.636+0000 7f0e18ff9640 1 -- 172.21.15.40:0/4197244006 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00faf0 con 0x7f0e080043a0 2024-04-26T21:41:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.636+0000 7f0e18ff9640 1 -- 172.21.15.40:0/4197244006 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00fc90 con 0x7f0e080043a0 2024-04-26T21:41:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.636+0000 7f0e18ff9640 1 -- 172.21.15.40:0/4197244006 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c014440 con 0x7f0e080043a0 2024-04-26T21:41:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.636+0000 7f0e22264640 1 -- 172.21.15.40:0/4197244006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c0cee50 con 0x7f0e080043a0 2024-04-26T21:41:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.636+0000 7f0e22264640 1 -- 172.21.15.40:0/4197244006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c0cf410 con 0x7f0e080043a0 2024-04-26T21:41:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.637+0000 7f0e22264640 1 -- 172.21.15.40:0/4197244006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f0e1c0707b0 con 0x7f0e080043a0 2024-04-26T21:41:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.637+0000 7f0e18ff9640 1 -- 172.21.15.40:0/4197244006 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e0c012040 con 0x7f0e080043a0 2024-04-26T21:41:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.638+0000 7f0e18ff9640 1 --2- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.638+0000 7f0e18ff9640 1 -- 172.21.15.40:0/4197244006 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f0e0c08c260 con 0x7f0e080043a0 2024-04-26T21:41:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.638+0000 7f0e18ff9640 1 -- 172.21.15.40:0/4197244006 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7f0e1c0707b0 con 0x7f0e080043a0 2024-04-26T21:41:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.638+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.638+0000 7f0e22264640 1 -- 172.21.15.40:0/4197244006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f0e1c069590 con 0x7f0e080043a0 2024-04-26T21:41:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.639+0000 7f0e1b7fe640 1 --2- 172.21.15.40:0/4197244006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0e04033000 tx=0x7f0e04008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.717+0000 7fa34dcb8640 1 Processor -- start 2024-04-26T21:41:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.717+0000 7fa34dcb8640 1 -- start start 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.717+0000 7fa34dcb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3481608f0 0x7fa3480d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.717+0000 7fa34dcb8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa3480d0a90 0x7fa3480cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.717+0000 7fa34dcb8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3480cde20 con 0x7fa3480cfdd0 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.717+0000 7fa34dcb8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa3480cea30 con 0x7fa3480d0a90 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.717+0000 7fa34dcb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa3480ce100 con 0x7fa3481608f0 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3481608f0 0x7fa3480d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa346ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa3480d0a90 0x7fa3480cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3481608f0 0x7fa3480d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:52770/0 (socket says 172.21.15.40:52770) 2024-04-26T21:41:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa3477fe640 1 -- 172.21.15.40:0/3300106670 learned_addr learned my addr 172.21.15.40:0/3300106670 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa346ffd640 1 -- 172.21.15.40:0/3300106670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3481608f0 msgr2=0x7fa3480d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa346ffd640 1 --2- 172.21.15.40:0/3300106670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3481608f0 0x7fa3480d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa346ffd640 1 -- 172.21.15.40:0/3300106670 >> v1:172.21.15.184:6789/0 conn(0x7fa3480cfdd0 legacy=0x7fa348160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa346ffd640 1 -- 172.21.15.40:0/3300106670 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3480ce3e0 con 0x7fa3480d0a90 2024-04-26T21:41:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.718+0000 7fa3477fe640 1 --2- 172.21.15.40:0/3300106670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3481608f0 0x7fa3480d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.719+0000 7fa346ffd640 1 --2- 172.21.15.40:0/3300106670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 0x7fa3480cf890 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa3300097a0 tx=0x7fa33002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=513971d35fdb7b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.719+0000 7fa3467fc640 1 -- 172.21.15.40:0/3300106670 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330037070 con 0x7fa3480d0a90 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.719+0000 7fa3467fc640 1 -- 172.21.15.40:0/3300106670 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa330032440 con 0x7fa3480d0a90 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.719+0000 7fa3467fc640 1 -- 172.21.15.40:0/3300106670 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330032810 con 0x7fa3480d0a90 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.720+0000 7fa34dcb8640 1 -- 172.21.15.40:0/3300106670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 msgr2=0x7fa3480cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.720+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/3300106670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 0x7fa3480cf890 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa3300097a0 tx=0x7fa33002a180 comp rx=0 tx=0).stop 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.720+0000 7fa34dcb8640 1 -- 172.21.15.40:0/3300106670 shutdown_connections 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.720+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/3300106670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 0x7fa3480cf890 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.720+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/3300106670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3481608f0 0x7fa3480d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.720+0000 7fa34dcb8640 1 -- 172.21.15.40:0/3300106670 >> 172.21.15.40:0/3300106670 conn(0x7fa3480d2ec0 msgr2=0x7fa3480d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.720+0000 7fa34dcb8640 1 -- 172.21.15.40:0/3300106670 shutdown_connections 2024-04-26T21:41:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.720+0000 7fa34dcb8640 1 -- 172.21.15.40:0/3300106670 wait complete. 2024-04-26T21:41:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.721+0000 7fa34dcb8640 1 Processor -- start 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.721+0000 7fa34dcb8640 1 -- start start 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa34dcb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480cfdd0 0x7fa3480c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa34dcb8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 0x7fa3480c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa34dcb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3481608f0 0x7fa34815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa34dcb8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3480b3310 con 0x7fa3481608f0 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa34dcb8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3480d0ef0 con 0x7fa3480d0a90 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa34dcb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa3480ceec0 con 0x7fa3480cfdd0 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3481608f0 0x7fa34815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3481608f0 0x7fa34815a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39910/0 (socket says 172.21.15.40:39910) 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 -- 172.21.15.40:0/707055485 learned_addr learned my addr 172.21.15.40:0/707055485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa346ffd640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 0x7fa3480c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa3477fe640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480cfdd0 0x7fa3480c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 -- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480cfdd0 msgr2=0x7fa3480c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480cfdd0 0x7fa3480c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 -- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 msgr2=0x7fa3480c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 0x7fa3480c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 -- 172.21.15.40:0/707055485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3480cb3a0 con 0x7fa3481608f0 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa347fff640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3481608f0 0x7fa34815a1f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa34000c6b0 tx=0x7fa34000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.722+0000 7fa33ffff640 1 -- 172.21.15.40:0/707055485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340003b70 con 0x7fa3481608f0 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.723+0000 7fa33ffff640 1 -- 172.21.15.40:0/707055485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa340003d30 con 0x7fa3481608f0 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.723+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3480cb680 con 0x7fa3481608f0 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.723+0000 7fa33ffff640 1 -- 172.21.15.40:0/707055485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340005310 con 0x7fa3481608f0 2024-04-26T21:41:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.723+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3480cbf20 con 0x7fa3481608f0 2024-04-26T21:41:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.724+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3480cba70 con 0x7fa3481608f0 2024-04-26T21:41:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.724+0000 7fa33ffff640 1 -- 172.21.15.40:0/707055485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3400054b0 con 0x7fa3481608f0 2024-04-26T21:41:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.724+0000 7fa33ffff640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa32806c8f0 0x7fa32806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.724+0000 7fa33ffff640 1 -- 172.21.15.40:0/707055485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fa34008b950 con 0x7fa3481608f0 2024-04-26T21:41:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.725+0000 7fa3477fe640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa32806c8f0 0x7fa32806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.725+0000 7fa3477fe640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa32806c8f0 0x7fa32806edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa338009630 tx=0x7fa338009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.729+0000 7fa33ffff640 1 -- 172.21.15.40:0/707055485 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa340059630 con 0x7fa3481608f0 2024-04-26T21:41:21.848 INFO:tasks.ceph.mds.c.smithi040.stderr:daemon-helper: command crashed with signal 6 2024-04-26T21:41:21.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.959+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3480694f0 con 0x7fa3481608f0 2024-04-26T21:41:21.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.959+0000 7fa33ffff640 1 -- 172.21.15.40:0/707055485 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5186 (secure 0 0 0) 0x7fa34005d930 con 0x7fa3481608f0 2024-04-26T21:41:21.960 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:21.960 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:15.498605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3115383225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3115383225},{"type":"v1","addr":"172.21.15.40:6835","nonce":3115383225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7230,"qdb_cluster":[7230]},"id":5}]} 2024-04-26T21:41:21.960 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 72 2024-04-26T21:41:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa32806c8f0 msgr2=0x7fa32806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa32806c8f0 0x7fa32806edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa338009630 tx=0x7fa338009240 comp rx=0 tx=0).stop 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3481608f0 msgr2=0x7fa34815a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3481608f0 0x7fa34815a1f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa34000c6b0 tx=0x7fa34000cb70 comp rx=0 tx=0).stop 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 shutdown_connections 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa32806c8f0 0x7fa32806edb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3481608f0 0x7fa34815a1f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3480d0a90 0x7fa3480c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 --2- 172.21.15.40:0/707055485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3480cfdd0 0x7fa3480c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 >> 172.21.15.40:0/707055485 conn(0x7fa3480d2ec0 msgr2=0x7fa34815b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 shutdown_connections 2024-04-26T21:41:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:21.961+0000 7fa34dcb8640 1 -- 172.21.15.40:0/707055485 wait complete. 2024-04-26T21:41:21.973 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T21:41:22.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:22.147+0000 7f0e18ff9640 1 -- 172.21.15.40:0/4197244006 <== mon.0 v2:172.21.15.40:3300/0 7 ==== log(1 entries from seq 1 at 2024-04-26T21:41:21.552560+0000) ==== 468+0+0 (secure 0 0 0) 0x7f0e0c08a710 con 0x7f0e080043a0 2024-04-26T21:41:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: cluster: 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: id: ec445c36-0c6e-44d6-884b-6999dea32f55 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: health: HEALTH_OK 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: services: 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: mon: 3 daemons, quorum a,b,c (age 6m) 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: mgr: x(active, since 6m), standbys: y 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: mds: 1/1 daemons up, 3 standby 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: data: 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: volumes: 1/1 healthy 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: pools: 3 pools, 97 pgs 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: objects: 29 objects, 596 KiB 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: usage: 228 MiB used, 720 GiB / 720 GiB avail 2024-04-26T21:41:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: pgs: 97 active+clean 2024-04-26T21:41:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: io: 2024-04-26T21:41:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: client: 0 B/s rd, 2.9 KiB/s wr, 0 op/s rd, 3 op/s wr 2024-04-26T21:41:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:22.150 INFO:teuthology.orchestra.run.smithi040.stdout:2024-04-26T21:41:21.552560+0000 mds.c [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7395:47 nref=7 cr=0x55f7e0c2ea80)) (dversion lock w=1 last_client=7395) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55f7e0ef6780] 2024-04-26T21:41:23.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-26T21:41:26.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:27.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.088+0000 7f8a280fc640 1 Processor -- start 2024-04-26T21:41:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.088+0000 7f8a280fc640 1 -- start start 2024-04-26T21:41:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a280fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a20162880 0x7f8a200d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a280fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200b5f20 0x7f8a200d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a280fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a200d39f0 0x7f8a200b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a280fc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8a200d0580 con 0x7f8a20162880 2024-04-26T21:41:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a280fc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8a200b7e50 con 0x7f8a200d39f0 2024-04-26T21:41:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a280fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a200d1800 con 0x7f8a200b5f20 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25e98640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a20162880 0x7f8a200d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25e98640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a20162880 0x7f8a200d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39918/0 (socket says 172.21.15.40:39918) 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25e98640 1 -- 172.21.15.40:0/662834133 learned_addr learned my addr 172.21.15.40:0/662834133 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a26699640 1 --2- 172.21.15.40:0/662834133 >> v2:172.21.15.184:3300/0 conn(0x7f8a200d39f0 0x7f8a200b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25697640 1 --2- 172.21.15.40:0/662834133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200b5f20 0x7f8a200d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25e98640 1 -- 172.21.15.40:0/662834133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200b5f20 msgr2=0x7f8a200d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25e98640 1 --2- 172.21.15.40:0/662834133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200b5f20 0x7f8a200d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25e98640 1 -- 172.21.15.40:0/662834133 >> v2:172.21.15.184:3300/0 conn(0x7f8a200d39f0 msgr2=0x7f8a200b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25e98640 1 --2- 172.21.15.40:0/662834133 >> v2:172.21.15.184:3300/0 conn(0x7f8a200d39f0 0x7f8a200b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25e98640 1 -- 172.21.15.40:0/662834133 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a200d07d0 con 0x7f8a20162880 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a25697640 1 --2- 172.21.15.40:0/662834133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200b5f20 0x7f8a200d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.089+0000 7f8a26699640 1 --2- 172.21.15.40:0/662834133 >> v2:172.21.15.184:3300/0 conn(0x7f8a200d39f0 0x7f8a200b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.090+0000 7f8a25e98640 1 --2- 172.21.15.40:0/662834133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a20162880 0x7f8a200d2cb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8a2008f770 tx=0x7f8a1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc50e449d463cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.090+0000 7f8a24e96640 1 -- 172.21.15.40:0/662834133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18037070 con 0x7f8a20162880 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.090+0000 7f8a24e96640 1 -- 172.21.15.40:0/662834133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a18032440 con 0x7f8a20162880 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.090+0000 7f8a24e96640 1 -- 172.21.15.40:0/662834133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18032810 con 0x7f8a20162880 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 -- 172.21.15.40:0/662834133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a20162880 msgr2=0x7f8a200d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 --2- 172.21.15.40:0/662834133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a20162880 0x7f8a200d2cb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8a2008f770 tx=0x7f8a1802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 -- 172.21.15.40:0/662834133 shutdown_connections 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 --2- 172.21.15.40:0/662834133 >> v2:172.21.15.184:3300/0 conn(0x7f8a200d39f0 0x7f8a200b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 --2- 172.21.15.40:0/662834133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a200b5f20 0x7f8a200d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 --2- 172.21.15.40:0/662834133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a20162880 0x7f8a200d2cb0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 -- 172.21.15.40:0/662834133 >> 172.21.15.40:0/662834133 conn(0x7f8a200d4f60 msgr2=0x7f8a200d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 -- 172.21.15.40:0/662834133 shutdown_connections 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.091+0000 7f8a280fc640 1 -- 172.21.15.40:0/662834133 wait complete. 2024-04-26T21:41:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.092+0000 7f8a280fc640 1 Processor -- start 2024-04-26T21:41:27.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.092+0000 7f8a280fc640 1 -- start start 2024-04-26T21:41:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a280fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a200b5f20 0x7f8a2015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a280fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d39f0 0x7f8a200a2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a280fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20162880 0x7f8a200b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a280fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20162c60 con 0x7f8a200b5f20 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a280fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a200d0d90 con 0x7f8a200d39f0 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a280fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a20162de0 con 0x7f8a20162880 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a200b5f20 0x7f8a2015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a200b5f20 0x7f8a2015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39928/0 (socket says 172.21.15.40:39928) 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 -- 172.21.15.40:0/3659949790 learned_addr learned my addr 172.21.15.40:0/3659949790 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25697640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d39f0 0x7f8a200a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a26699640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20162880 0x7f8a200b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 -- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20162880 msgr2=0x7f8a200b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20162880 0x7f8a200b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 -- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d39f0 msgr2=0x7f8a200a2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d39f0 0x7f8a200a2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 -- 172.21.15.40:0/3659949790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a200cef30 con 0x7f8a200b5f20 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25697640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d39f0 0x7f8a200a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.093+0000 7f8a25e98640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a200b5f20 0x7f8a2015b250 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8a18000c00 tx=0x7f8a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.094+0000 7f8a1e7fc640 1 -- 172.21.15.40:0/3659949790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18041070 con 0x7f8a200b5f20 2024-04-26T21:41:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.094+0000 7f8a1e7fc640 1 -- 172.21.15.40:0/3659949790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1803c870 con 0x7f8a200b5f20 2024-04-26T21:41:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.094+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a200cf210 con 0x7f8a200b5f20 2024-04-26T21:41:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.094+0000 7f8a1e7fc640 1 -- 172.21.15.40:0/3659949790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18037040 con 0x7f8a200b5f20 2024-04-26T21:41:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.094+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a200b3fc0 con 0x7f8a200b5f20 2024-04-26T21:41:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.095+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a200694f0 con 0x7f8a200b5f20 2024-04-26T21:41:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.095+0000 7f8a1e7fc640 1 -- 172.21.15.40:0/3659949790 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a18044070 con 0x7f8a200b5f20 2024-04-26T21:41:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.096+0000 7f8a1e7fc640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.096+0000 7f8a1e7fc640 1 -- 172.21.15.40:0/3659949790 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f8a180ac840 con 0x7f8a200b5f20 2024-04-26T21:41:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.096+0000 7f8a25697640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.097+0000 7f8a25697640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8a200ce800 tx=0x7f8a14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:27.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.100+0000 7f8a1e7fc640 1 -- 172.21.15.40:0/3659949790 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a1807ac90 con 0x7f8a200b5f20 2024-04-26T21:41:27.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.325+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a20074590 con 0x7f8a200b5f20 2024-04-26T21:41:27.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.326+0000 7f8a1e7fc640 1 -- 172.21.15.40:0/3659949790 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5186 (secure 0 0 0) 0x7f8a1807a590 con 0x7f8a200b5f20 2024-04-26T21:41:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:27.327 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:15.498605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3115383225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3115383225},{"type":"v1","addr":"172.21.15.40:6835","nonce":3115383225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7230,"qdb_cluster":[7230]},"id":5}]} 2024-04-26T21:41:27.327 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 72 2024-04-26T21:41:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 msgr2=0x7f8a0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8a200ce800 tx=0x7f8a14008040 comp rx=0 tx=0).stop 2024-04-26T21:41:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a200b5f20 msgr2=0x7f8a2015b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a200b5f20 0x7f8a2015b250 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8a18000c00 tx=0x7f8a180046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 shutdown_connections 2024-04-26T21:41:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a0c06c8f0 0x7f8a0c06edb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a20162880 0x7f8a200b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a200d39f0 0x7f8a200a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 --2- 172.21.15.40:0/3659949790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a200b5f20 0x7f8a2015b250 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 >> 172.21.15.40:0/3659949790 conn(0x7f8a200d4f60 msgr2=0x7f8a200ba310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 shutdown_connections 2024-04-26T21:41:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:27.328+0000 7f8a280fc640 1 -- 172.21.15.40:0/3659949790 wait complete. 2024-04-26T21:41:27.340 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T21:41:30.459 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-04-26T21:41:32.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:32.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.454+0000 7fc2440bd640 1 Processor -- start 2024-04-26T21:41:32.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.454+0000 7fc2440bd640 1 -- start start 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc2440bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc23c071e60 0x7fc23c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc2440bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c06fe20 0x7fc23c072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc2440bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc23c15d7c0 con 0x7fc23c072780 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc2440bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c15daa0 con 0x7fc23c071e60 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc2440bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c15dd80 con 0x7fc23c06fe20 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc241e59640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc23c071e60 0x7fc23c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc241e59640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc23c071e60 0x7fc23c06f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42782/0 (socket says 172.21.15.40:42782) 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc241e59640 1 -- 172.21.15.40:0/74114632 learned_addr learned my addr 172.21.15.40:0/74114632 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc24265a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc23c072780 0x7fc23c0cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55510/0 (socket says 172.21.15.40:55510) 2024-04-26T21:41:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.455+0000 7fc241658640 1 --2- 172.21.15.40:0/74114632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c06fe20 0x7fc23c072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:32.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.456+0000 7fc241e59640 1 -- 172.21.15.40:0/74114632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c06fe20 msgr2=0x7fc23c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:32.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.456+0000 7fc241e59640 1 --2- 172.21.15.40:0/74114632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c06fe20 0x7fc23c072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.456+0000 7fc241e59640 1 -- 172.21.15.40:0/74114632 >> v1:172.21.15.40:6789/0 conn(0x7fc23c072780 legacy=0x7fc23c0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:32.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.456+0000 7fc241e59640 1 -- 172.21.15.40:0/74114632 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c15e060 con 0x7fc23c071e60 2024-04-26T21:41:32.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.456+0000 7fc240e57640 1 -- 172.21.15.40:0/74114632 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975829444 0 0) 0x7fc23c15d7c0 con 0x7fc23c072780 2024-04-26T21:41:32.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.456+0000 7fc241e59640 1 --2- 172.21.15.40:0/74114632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c071e60 0x7fc23c06f8e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc238009870 tx=0x7fc23802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43b002e6a28f111 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.456+0000 7fc240e57640 1 -- 172.21.15.40:0/74114632 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238037070 con 0x7fc23c071e60 2024-04-26T21:41:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc240e57640 1 -- 172.21.15.40:0/74114632 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc238032440 con 0x7fc23c071e60 2024-04-26T21:41:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc240e57640 1 -- 172.21.15.40:0/74114632 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238032810 con 0x7fc23c071e60 2024-04-26T21:41:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc2440bd640 1 -- 172.21.15.40:0/74114632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c071e60 msgr2=0x7fc23c06f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc2440bd640 1 --2- 172.21.15.40:0/74114632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c071e60 0x7fc23c06f8e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc238009870 tx=0x7fc23802a270 comp rx=0 tx=0).stop 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc2440bd640 1 -- 172.21.15.40:0/74114632 shutdown_connections 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc2440bd640 1 --2- 172.21.15.40:0/74114632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c06fe20 0x7fc23c072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc2440bd640 1 --2- 172.21.15.40:0/74114632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c071e60 0x7fc23c06f8e0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc2440bd640 1 -- 172.21.15.40:0/74114632 >> 172.21.15.40:0/74114632 conn(0x7fc23c0d0be0 msgr2=0x7fc23c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc2440bd640 1 -- 172.21.15.40:0/74114632 shutdown_connections 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.457+0000 7fc2440bd640 1 -- 172.21.15.40:0/74114632 wait complete. 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.458+0000 7fc2440bd640 1 Processor -- start 2024-04-26T21:41:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.458+0000 7fc2440bd640 1 -- start start 2024-04-26T21:41:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc2440bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c06fe20 0x7fc23c14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc2440bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c071e60 0x7fc23c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc2440bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c072780 0x7fc23c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc2440bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c15d3f0 con 0x7fc23c072780 2024-04-26T21:41:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc2440bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c072c10 con 0x7fc23c06fe20 2024-04-26T21:41:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc2440bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c15d570 con 0x7fc23c071e60 2024-04-26T21:41:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc24265a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c072780 0x7fc23c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc24265a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c072780 0x7fc23c0b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42798/0 (socket says 172.21.15.40:42798) 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc24265a640 1 -- 172.21.15.40:0/1269760743 learned_addr learned my addr 172.21.15.40:0/1269760743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc241658640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c071e60 0x7fc23c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.459+0000 7fc241e59640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c06fe20 0x7fc23c14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc24265a640 1 -- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c071e60 msgr2=0x7fc23c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc24265a640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c071e60 0x7fc23c159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc24265a640 1 -- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c06fe20 msgr2=0x7fc23c14b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc24265a640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c06fe20 0x7fc23c14b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc24265a640 1 -- 172.21.15.40:0/1269760743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c15a410 con 0x7fc23c072780 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc24265a640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c072780 0x7fc23c0b1860 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc23400c700 tx=0x7fc23400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc2327fc640 1 -- 172.21.15.40:0/1269760743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234018070 con 0x7fc23c072780 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc2327fc640 1 -- 172.21.15.40:0/1269760743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23400ce10 con 0x7fc23c072780 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c15a6f0 con 0x7fc23c072780 2024-04-26T21:41:32.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc2327fc640 1 -- 172.21.15.40:0/1269760743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23400a800 con 0x7fc23c072780 2024-04-26T21:41:32.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.460+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c15af90 con 0x7fc23c072780 2024-04-26T21:41:32.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.461+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c15aae0 con 0x7fc23c072780 2024-04-26T21:41:32.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.461+0000 7fc2327fc640 1 -- 172.21.15.40:0/1269760743 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc23401b070 con 0x7fc23c072780 2024-04-26T21:41:32.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.462+0000 7fc2327fc640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc22006c8f0 0x7fc22006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.462+0000 7fc2327fc640 1 -- 172.21.15.40:0/1269760743 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fc23408b180 con 0x7fc23c072780 2024-04-26T21:41:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.462+0000 7fc241e59640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc22006c8f0 0x7fc22006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.463+0000 7fc241e59640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc22006c8f0 0x7fc22006edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc238034000 tx=0x7fc23802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:32.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.467+0000 7fc2327fc640 1 -- 172.21.15.40:0/1269760743 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc234058dd0 con 0x7fc23c072780 2024-04-26T21:41:32.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.690+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc23c0694f0 con 0x7fc23c072780 2024-04-26T21:41:32.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.691+0000 7fc2327fc640 1 -- 172.21.15.40:0/1269760743 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5186 (secure 0 0 0) 0x7fc23405d0d0 con 0x7fc23c072780 2024-04-26T21:41:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:32.692 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:15.498605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3115383225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3115383225},{"type":"v1","addr":"172.21.15.40:6835","nonce":3115383225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7230,"qdb_cluster":[7230]},"id":5}]} 2024-04-26T21:41:32.692 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 72 2024-04-26T21:41:32.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc22006c8f0 msgr2=0x7fc22006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:32.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc22006c8f0 0x7fc22006edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc238034000 tx=0x7fc23802e000 comp rx=0 tx=0).stop 2024-04-26T21:41:32.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c072780 msgr2=0x7fc23c0b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:32.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c072780 0x7fc23c0b1860 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc23400c700 tx=0x7fc23400cbc0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 shutdown_connections 2024-04-26T21:41:32.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc22006c8f0 0x7fc22006edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c072780 0x7fc23c0b1860 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23c071e60 0x7fc23c159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 --2- 172.21.15.40:0/1269760743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c06fe20 0x7fc23c14b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:32.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 >> 172.21.15.40:0/1269760743 conn(0x7fc23c0d0be0 msgr2=0x7fc23c15e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:32.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 shutdown_connections 2024-04-26T21:41:32.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:32.693+0000 7fc2440bd640 1 -- 172.21.15.40:0/1269760743 wait complete. 2024-04-26T21:41:32.705 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-26T21:41:37.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-04-26T21:41:37.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:37.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.819+0000 7fed23e8c640 1 Processor -- start 2024-04-26T21:41:37.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.819+0000 7fed23e8c640 1 -- start start 2024-04-26T21:41:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed23e8c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fed1c15f3a0 0x7fed1c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed23e8c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed1c0d1770 0x7fed1c0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed23e8c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed1c0d2590 con 0x7fed1c0d2160 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed23e8c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fed1c0d0e20 con 0x7fed1c0d1770 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed23e8c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fed1c0b1d70 con 0x7fed1c15f3a0 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed21427640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed1c0d1770 0x7fed1c0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed21427640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed1c0d1770 0x7fed1c0d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42800/0 (socket says 172.21.15.40:42800) 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed21427640 1 -- 172.21.15.40:0/3928478690 learned_addr learned my addr 172.21.15.40:0/3928478690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.820+0000 7fed21c28640 1 --2- 172.21.15.40:0/3928478690 >> v2:172.21.15.184:3300/0 conn(0x7fed1c15f3a0 0x7fed1c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.821+0000 7fed21427640 1 -- 172.21.15.40:0/3928478690 >> v2:172.21.15.184:3300/0 conn(0x7fed1c15f3a0 msgr2=0x7fed1c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.821+0000 7fed21427640 1 --2- 172.21.15.40:0/3928478690 >> v2:172.21.15.184:3300/0 conn(0x7fed1c15f3a0 0x7fed1c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.821+0000 7fed21427640 1 -- 172.21.15.40:0/3928478690 >> v1:172.21.15.184:6789/0 conn(0x7fed1c0d2160 legacy=0x7fed1c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.821+0000 7fed21427640 1 -- 172.21.15.40:0/3928478690 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c0b2050 con 0x7fed1c0d1770 2024-04-26T21:41:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.821+0000 7fed21427640 1 --2- 172.21.15.40:0/3928478690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c0d1770 0x7fed1c0d1c20 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fed1c073b50 tx=0x7fed0c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6cd66c4931dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.821+0000 7fed20c26640 1 -- 172.21.15.40:0/3928478690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c037070 con 0x7fed1c0d1770 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.821+0000 7fed20c26640 1 -- 172.21.15.40:0/3928478690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c032440 con 0x7fed1c0d1770 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.821+0000 7fed20c26640 1 -- 172.21.15.40:0/3928478690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c032810 con 0x7fed1c0d1770 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.822+0000 7fed23e8c640 1 -- 172.21.15.40:0/3928478690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c0d1770 msgr2=0x7fed1c0d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.822+0000 7fed23e8c640 1 --2- 172.21.15.40:0/3928478690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c0d1770 0x7fed1c0d1c20 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fed1c073b50 tx=0x7fed0c02a230 comp rx=0 tx=0).stop 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.822+0000 7fed23e8c640 1 -- 172.21.15.40:0/3928478690 shutdown_connections 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.822+0000 7fed23e8c640 1 --2- 172.21.15.40:0/3928478690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c0d1770 0x7fed1c0d1c20 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.822+0000 7fed23e8c640 1 --2- 172.21.15.40:0/3928478690 >> v2:172.21.15.184:3300/0 conn(0x7fed1c15f3a0 0x7fed1c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.822+0000 7fed23e8c640 1 -- 172.21.15.40:0/3928478690 >> 172.21.15.40:0/3928478690 conn(0x7fed1c0d4f90 msgr2=0x7fed1c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.822+0000 7fed23e8c640 1 -- 172.21.15.40:0/3928478690 shutdown_connections 2024-04-26T21:41:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.822+0000 7fed23e8c640 1 -- 172.21.15.40:0/3928478690 wait complete. 2024-04-26T21:41:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.824+0000 7fed23e8c640 1 Processor -- start 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.824+0000 7fed23e8c640 1 -- start start 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.824+0000 7fed23e8c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c0d1770 0x7fed1c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.824+0000 7fed23e8c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed1c0d2160 0x7fed1c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.824+0000 7fed23e8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c15f3a0 0x7fed1c0b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.824+0000 7fed23e8c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c15f900 con 0x7fed1c15f3a0 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.824+0000 7fed23e8c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c15f780 con 0x7fed1c0d1770 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.824+0000 7fed23e8c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c0b3860 con 0x7fed1c0d2160 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c15f3a0 0x7fed1c0b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c15f3a0 0x7fed1c0b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42808/0 (socket says 172.21.15.40:42808) 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 -- 172.21.15.40:0/2682055637 learned_addr learned my addr 172.21.15.40:0/2682055637 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed21427640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed1c0d2160 0x7fed1c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed21c28640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c0d1770 0x7fed1c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 -- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed1c0d2160 msgr2=0x7fed1c0b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed1c0d2160 0x7fed1c0b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 -- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c0d1770 msgr2=0x7fed1c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c0d1770 0x7fed1c1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 -- 172.21.15.40:0/2682055637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c160970 con 0x7fed1c15f3a0 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed22429640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c15f3a0 0x7fed1c0b5600 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fed1400c780 tx=0x7fed1400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed127fc640 1 -- 172.21.15.40:0/2682055637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14004b60 con 0x7fed1c15f3a0 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed127fc640 1 -- 172.21.15.40:0/2682055637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed14004d00 con 0x7fed1c15f3a0 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c160c50 con 0x7fed1c15f3a0 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed127fc640 1 -- 172.21.15.40:0/2682055637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1400aa90 con 0x7fed1c15f3a0 2024-04-26T21:41:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.825+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c161210 con 0x7fed1c15f3a0 2024-04-26T21:41:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.826+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1c0694f0 con 0x7fed1c15f3a0 2024-04-26T21:41:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.827+0000 7fed127fc640 1 -- 172.21.15.40:0/2682055637 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed1400ac30 con 0x7fed1c15f3a0 2024-04-26T21:41:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.827+0000 7fed127fc640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed0006c8f0 0x7fed0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.828+0000 7fed127fc640 1 -- 172.21.15.40:0/2682055637 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fed1408be80 con 0x7fed1c15f3a0 2024-04-26T21:41:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.828+0000 7fed21c28640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed0006c8f0 0x7fed0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.828+0000 7fed21c28640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed0006c8f0 0x7fed0006edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fed180079a0 tx=0x7fed18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:37.832+0000 7fed127fc640 1 -- 172.21.15.40:0/2682055637 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed14020070 con 0x7fed1c15f3a0 2024-04-26T21:41:38.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.055+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed1c06c900 con 0x7fed1c15f3a0 2024-04-26T21:41:38.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.056+0000 7fed127fc640 1 -- 172.21.15.40:0/2682055637 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5234 (secure 0 0 0) 0x7fed14059ad0 con 0x7fed1c15f3a0 2024-04-26T21:41:38.057 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:38.057 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:35.631361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3115383225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3115383225},{"type":"v1","addr":"172.21.15.40:6835","nonce":3115383225}]},"join_fscid":-1,"laggy_since":"2024-04-26T21:41:35.631361+0000","export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7230,"qdb_cluster":[7230]},"id":5}]} 2024-04-26T21:41:38.057 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 73 2024-04-26T21:41:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed0006c8f0 msgr2=0x7fed0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed0006c8f0 0x7fed0006edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fed180079a0 tx=0x7fed18008040 comp rx=0 tx=0).stop 2024-04-26T21:41:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c15f3a0 msgr2=0x7fed1c0b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c15f3a0 0x7fed1c0b5600 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fed1400c780 tx=0x7fed1400cc40 comp rx=0 tx=0).stop 2024-04-26T21:41:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 shutdown_connections 2024-04-26T21:41:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed0006c8f0 0x7fed0006edb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1c15f3a0 0x7fed1c0b5600 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed1c0d2160 0x7fed1c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 --2- 172.21.15.40:0/2682055637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c0d1770 0x7fed1c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 >> 172.21.15.40:0/2682055637 conn(0x7fed1c0d4f90 msgr2=0x7fed1c0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 shutdown_connections 2024-04-26T21:41:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.058+0000 7fed23e8c640 1 -- 172.21.15.40:0/2682055637 wait complete. 2024-04-26T21:41:38.070 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-04-26T21:41:38.171 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-04-26T21:41:38.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.273+0000 7f476cbe4640 1 Processor -- start 2024-04-26T21:41:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.273+0000 7f476cbe4640 1 -- start start 2024-04-26T21:41:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.274+0000 7f476cbe4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680b2340 0x7f47680cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.274+0000 7f476cbe4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47680d10d0 con 0x7f47680cd930 2024-04-26T21:41:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.274+0000 7f476cbe4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47680d1370 con 0x7f4768162810 2024-04-26T21:41:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.274+0000 7f476cbe4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47680d0750 con 0x7f47680b2340 2024-04-26T21:41:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.274+0000 7f4766d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f47680cd930 0x7f47680d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55522/0 (socket says 172.21.15.40:55522) 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.274+0000 7f4766d9d640 1 -- 172.21.15.40:0/3137668035 learned_addr learned my addr 172.21.15.40:0/3137668035 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f4765d9b640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680b2340 0x7f47680cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1453622610 0 0) 0x7f47680d10d0 con 0x7f47680cd930 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4754003520 con 0x7f47680cd930 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4056976727 0 0) 0x7f4754003520 con 0x7f47680cd930 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680b2340 msgr2=0x7f47680cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680b2340 0x7f47680cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 >> v1:172.21.15.184:6789/0 conn(0x7f4768162810 legacy=0x7f4768070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47680d01b0 con 0x7f47680cd930 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f47580019f0 con 0x7f47680cd930 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 >> v1:172.21.15.40:6789/0 conn(0x7f47680cd930 legacy=0x7f47680d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f4754003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47540043b0 0x7f4754004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680b2340 0x7f47540050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f47680d0750 con 0x7f47540039e0 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f475400b9f0 con 0x7f47540043b0 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f475400bbf0 con 0x7f47680b2340 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f476659c640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f4754003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f4766d9d640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680b2340 0x7f47540050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.275+0000 7f4765d9b640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47540043b0 0x7f4754004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476659c640 1 -- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680b2340 msgr2=0x7f47540050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476659c640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680b2340 0x7f47540050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476659c640 1 -- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47540043b0 msgr2=0x7f4754004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476659c640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47540043b0 0x7f4754004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476659c640 1 -- 172.21.15.40:0/3137668035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4754003520 con 0x7f47540039e0 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476659c640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f4754003de0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4760001910 tx=0x7f4754005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fa86080304a3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760037070 con 0x7f47540039e0 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476002ac60 con 0x7f47540039e0 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.276+0000 7f476559a640 1 -- 172.21.15.40:0/3137668035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600014e0 con 0x7f47540039e0 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.277+0000 7f476cbe4640 1 -- 172.21.15.40:0/3137668035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 msgr2=0x7f4754003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.277+0000 7f476cbe4640 1 --2- 172.21.15.40:0/3137668035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f4754003de0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4760001910 tx=0x7f4754005830 comp rx=0 tx=0).stop 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.277+0000 7f4766d9d640 1 -- 172.21.15.40:0/3137668035 reap_dead start 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.277+0000 7f476cbe4640 1 -- 172.21.15.40:0/3137668035 shutdown_connections 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.277+0000 7f476cbe4640 1 -- 172.21.15.40:0/3137668035 >> 172.21.15.40:0/3137668035 conn(0x7f47680d2da0 msgr2=0x7f47680d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.277+0000 7f476cbe4640 1 -- 172.21.15.40:0/3137668035 shutdown_connections 2024-04-26T21:41:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.277+0000 7f476cbe4640 1 -- 172.21.15.40:0/3137668035 wait complete. 2024-04-26T21:41:38.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.278+0000 7f476cbe4640 1 Processor -- start 2024-04-26T21:41:38.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.278+0000 7f476cbe4640 1 -- start start 2024-04-26T21:41:38.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476cbe4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f47680c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476cbe4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680c97c0 0x7f476814a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476cbe4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476814a710 0x7f476814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476cbe4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f47680d2240 con 0x7f47540039e0 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476cbe4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47680d16a0 con 0x7f476814a710 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476cbe4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47680d23c0 con 0x7f47680c97c0 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f47680c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f47680c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42830/0 (socket says 172.21.15.40:42830) 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476659c640 1 -- 172.21.15.40:0/3004638197 learned_addr learned my addr 172.21.15.40:0/3004638197 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f4766d9d640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476814a710 0x7f476814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f4765d9b640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680c97c0 0x7f476814a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476659c640 1 -- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680c97c0 msgr2=0x7f476814a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476659c640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680c97c0 0x7f476814a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476659c640 1 -- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476814a710 msgr2=0x7f476814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476659c640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476814a710 0x7f476814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.279+0000 7f476659c640 1 -- 172.21.15.40:0/3004638197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476814ce20 con 0x7f47540039e0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.280+0000 7f476659c640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f47680c9280 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4760035040 tx=0x7f47600012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.280+0000 7f475effd640 1 -- 172.21.15.40:0/3004638197 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476003f020 con 0x7f47540039e0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.280+0000 7f475effd640 1 -- 172.21.15.40:0/3004638197 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47600052e0 con 0x7f47540039e0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.280+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476814d100 con 0x7f47540039e0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.280+0000 7f475effd640 1 -- 172.21.15.40:0/3004638197 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760037040 con 0x7f47540039e0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.280+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47680c8a80 con 0x7f47540039e0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.281+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47680694f0 con 0x7f47540039e0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.281+0000 7f475effd640 1 -- 172.21.15.40:0/3004638197 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4760042020 con 0x7f47540039e0 2024-04-26T21:41:38.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.282+0000 7f475effd640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f474806c920 0x7f474806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.282+0000 7f475effd640 1 -- 172.21.15.40:0/3004638197 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f47600abaf0 con 0x7f47540039e0 2024-04-26T21:41:38.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.282+0000 7f4765d9b640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f474806c920 0x7f474806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.283+0000 7f4765d9b640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f474806c920 0x7f474806ede0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f47680d3df0 tx=0x7f4750008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:38.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.286+0000 7f475effd640 1 -- 172.21.15.40:0/3004638197 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4760049030 con 0x7f47540039e0 2024-04-26T21:41:38.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.471+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f476815f1a0 con 0x7f47540039e0 2024-04-26T21:41:38.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.474+0000 7f475effd640 1 -- 172.21.15.40:0/3004638197 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f4760079cf0 con 0x7f47540039e0 2024-04-26T21:41:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f474806c920 msgr2=0x7f474806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f474806c920 0x7f474806ede0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f47680d3df0 tx=0x7f4750008040 comp rx=0 tx=0).stop 2024-04-26T21:41:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 msgr2=0x7f47680c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f47680c9280 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4760035040 tx=0x7f47600012b0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 shutdown_connections 2024-04-26T21:41:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f474806c920 0x7f474806ede0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476814a710 0x7f476814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47680c97c0 0x7f476814a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 --2- 172.21.15.40:0/3004638197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47540039e0 0x7f47680c9280 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 >> 172.21.15.40:0/3004638197 conn(0x7f47680d2da0 msgr2=0x7f4768161880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 shutdown_connections 2024-04-26T21:41:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.476+0000 7f476cbe4640 1 -- 172.21.15.40:0/3004638197 wait complete. 2024-04-26T21:41:38.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 false 2024-04-26T21:41:38.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.589+0000 7fd7a12ea640 1 Processor -- start 2024-04-26T21:41:38.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.589+0000 7fd7a12ea640 1 -- start start 2024-04-26T21:41:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.590+0000 7fd7a12ea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd79c15f5a0 0x7fd79c0d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.590+0000 7fd7a12ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd79c0d2830 0x7fd79c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.590+0000 7fd7a12ea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c0d1a70 con 0x7fd79c0d3c20 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.590+0000 7fd7a12ea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd79c163d30 con 0x7fd79c0d2830 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.590+0000 7fd7a12ea640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd79c0d1d50 con 0x7fd79c15f5a0 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.590+0000 7fd79a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd79c0d2830 0x7fd79c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.590+0000 7fd79b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd79c0d3c20 0x7fd79c15f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55528/0 (socket says 172.21.15.40:55528) 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.590+0000 7fd79b7fe640 1 -- 172.21.15.40:0/466597277 learned_addr learned my addr 172.21.15.40:0/466597277 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd79affd640 1 --2- 172.21.15.40:0/466597277 >> v2:172.21.15.184:3300/0 conn(0x7fd79c15f5a0 0x7fd79c0d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd799ffb640 1 -- 172.21.15.40:0/466597277 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597752389 0 0) 0x7fd79c0d1a70 con 0x7fd79c0d3c20 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd799ffb640 1 -- 172.21.15.40:0/466597277 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd788003520 con 0x7fd79c0d3c20 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd79a7fc640 1 -- 172.21.15.40:0/466597277 >> v2:172.21.15.184:3300/0 conn(0x7fd79c15f5a0 msgr2=0x7fd79c0d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd79a7fc640 1 --2- 172.21.15.40:0/466597277 >> v2:172.21.15.184:3300/0 conn(0x7fd79c15f5a0 0x7fd79c0d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd79a7fc640 1 -- 172.21.15.40:0/466597277 >> v1:172.21.15.40:6789/0 conn(0x7fd79c0d3c20 legacy=0x7fd79c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd79a7fc640 1 -- 172.21.15.40:0/466597277 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c0d2030 con 0x7fd79c0d2830 2024-04-26T21:41:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd79affd640 1 --2- 172.21.15.40:0/466597277 >> v2:172.21.15.184:3300/0 conn(0x7fd79c15f5a0 0x7fd79c0d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.591+0000 7fd79a7fc640 1 --2- 172.21.15.40:0/466597277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d2830 0x7fd79c162f00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd79c08d2f0 tx=0x7fd78402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c5a6c9637eb027 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd799ffb640 1 -- 172.21.15.40:0/466597277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784037070 con 0x7fd79c0d2830 2024-04-26T21:41:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd799ffb640 1 -- 172.21.15.40:0/466597277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd784032440 con 0x7fd79c0d2830 2024-04-26T21:41:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd799ffb640 1 -- 172.21.15.40:0/466597277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78402ae60 con 0x7fd79c0d2830 2024-04-26T21:41:38.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd7a12ea640 1 -- 172.21.15.40:0/466597277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d2830 msgr2=0x7fd79c162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/466597277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d2830 0x7fd79c162f00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd79c08d2f0 tx=0x7fd78402a110 comp rx=0 tx=0).stop 2024-04-26T21:41:38.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd7a12ea640 1 -- 172.21.15.40:0/466597277 shutdown_connections 2024-04-26T21:41:38.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/466597277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d2830 0x7fd79c162f00 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/466597277 >> v2:172.21.15.184:3300/0 conn(0x7fd79c15f5a0 0x7fd79c0d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd7a12ea640 1 -- 172.21.15.40:0/466597277 >> 172.21.15.40:0/466597277 conn(0x7fd79c0d5190 msgr2=0x7fd79c0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd7a12ea640 1 -- 172.21.15.40:0/466597277 shutdown_connections 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.592+0000 7fd7a12ea640 1 -- 172.21.15.40:0/466597277 wait complete. 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd7a12ea640 1 Processor -- start 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd7a12ea640 1 -- start start 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd7a12ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd79c0d2830 0x7fd79c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd7a12ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d3c20 0x7fd79c1ea170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd7a12ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd79c1ea6b0 0x7fd79c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd7a12ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c163440 con 0x7fd79c0d3c20 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd7a12ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c0d4190 con 0x7fd79c0d2830 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd7a12ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd79c1635c0 con 0x7fd79c1ea6b0 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.594+0000 7fd79a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d3c20 0x7fd79c1ea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d3c20 0x7fd79c1ea170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42846/0 (socket says 172.21.15.40:42846) 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79a7fc640 1 -- 172.21.15.40:0/48237356 learned_addr learned my addr 172.21.15.40:0/48237356 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79affd640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd79c0d2830 0x7fd79c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79b7fe640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd79c1ea6b0 0x7fd79c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79a7fc640 1 -- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd79c1ea6b0 msgr2=0x7fd79c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79a7fc640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd79c1ea6b0 0x7fd79c160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79a7fc640 1 -- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd79c0d2830 msgr2=0x7fd79c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79a7fc640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd79c0d2830 0x7fd79c0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79a7fc640 1 -- 172.21.15.40:0/48237356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c162740 con 0x7fd79c0d3c20 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79b7fe640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd79c1ea6b0 0x7fd79c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd79a7fc640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d3c20 0x7fd79c1ea170 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd784000c00 tx=0x7fd7840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd7937fe640 1 -- 172.21.15.40:0/48237356 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784037040 con 0x7fd79c0d3c20 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd7937fe640 1 -- 172.21.15.40:0/48237356 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd784042070 con 0x7fd79c0d3c20 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c1616a0 con 0x7fd79c0d3c20 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd7937fe640 1 -- 172.21.15.40:0/48237356 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784030960 con 0x7fd79c0d3c20 2024-04-26T21:41:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.595+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c161c60 con 0x7fd79c0d3c20 2024-04-26T21:41:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.596+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c1eaac0 con 0x7fd79c0d3c20 2024-04-26T21:41:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.597+0000 7fd7937fe640 1 -- 172.21.15.40:0/48237356 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd784030b00 con 0x7fd79c0d3c20 2024-04-26T21:41:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.597+0000 7fd7937fe640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd77c06c920 0x7fd77c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.597+0000 7fd7937fe640 1 -- 172.21.15.40:0/48237356 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fd7840ac610 con 0x7fd79c0d3c20 2024-04-26T21:41:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.597+0000 7fd79affd640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd77c06c920 0x7fd77c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.598+0000 7fd79affd640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd77c06c920 0x7fd77c06ede0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd794006fd0 tx=0x7fd794008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.602+0000 7fd7937fe640 1 -- 172.21.15.40:0/48237356 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd79c1eaac0 con 0x7fd79c0d3c20 2024-04-26T21:41:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:38.861+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"} v 0) -- 0x7fd79c068d30 con 0x7fd79c0d3c20 2024-04-26T21:41:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.653+0000 7fd7937fe640 1 -- 172.21.15.40:0/48237356 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"}]=0 unfreezing mds.7230 v74) ==== 115+0+0 (secure 0 0 0) 0x7fd784045360 con 0x7fd79c0d3c20 2024-04-26T21:41:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:unfreezing mds.7230 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.655+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd77c06c920 msgr2=0x7fd77c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.655+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd77c06c920 0x7fd77c06ede0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd794006fd0 tx=0x7fd794008040 comp rx=0 tx=0).stop 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.655+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d3c20 msgr2=0x7fd79c1ea170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.655+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d3c20 0x7fd79c1ea170 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd784000c00 tx=0x7fd7840046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.656+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 shutdown_connections 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.656+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd77c06c920 0x7fd77c06ede0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.656+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd79c1ea6b0 0x7fd79c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.656+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd79c0d3c20 0x7fd79c1ea170 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.656+0000 7fd7a12ea640 1 --2- 172.21.15.40:0/48237356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd79c0d2830 0x7fd79c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.656+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 >> 172.21.15.40:0/48237356 conn(0x7fd79c0d5190 msgr2=0x7fd79c15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.656+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 shutdown_connections 2024-04-26T21:41:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.656+0000 7fd7a12ea640 1 -- 172.21.15.40:0/48237356 wait complete. 2024-04-26T21:41:39.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -n kernel.core_pattern 2024-04-26T21:41:39.699 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:41:39.701 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:41:39.701 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-04-26T21:41:39.770 INFO:teuthology.orchestra.run.smithi040.stdout:1714167681.63010.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-04-26T21:41:39.772 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1714167681.63010.core'] 2024-04-26T21:41:39.772 INFO:tasks.cephfs.cephfs_test_case:Found core file 1714167681.63010.core, deleting it 2024-04-26T21:41:39.772 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1714167681.63010.core 2024-04-26T21:41:39.841 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:41:39.841 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:41:39.841 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:41:39.841 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:39.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:41:39.842 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi040 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-04-26T21:41:39.842 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:41:39.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:41:39.884 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:41:39.884 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:39.962 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:41:39.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.989+0000 7f15056bd640 1 Processor -- start 2024-04-26T21:41:39.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.989+0000 7f15056bd640 1 -- start start 2024-04-26T21:41:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.989+0000 7f15056bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1500163610 0x7f15000d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.989+0000 7f15056bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15000d1a70 con 0x7f15000d2830 2024-04-26T21:41:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.989+0000 7f15056bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15000d1d50 con 0x7f150015f5a0 2024-04-26T21:41:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.989+0000 7f15056bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1500163d30 con 0x7f1500163610 2024-04-26T21:41:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14ff7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f15000d2830 0x7f15000cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44556/0 (socket says 172.21.15.40:44556) 2024-04-26T21:41:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14ff7fe640 1 -- 172.21.15.40:0/167923952 learned_addr learned my addr 172.21.15.40:0/167923952 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14f67fc640 1 --2- 172.21.15.40:0/167923952 >> v2:172.21.15.184:3300/0 conn(0x7f1500163610 0x7f15000d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317296619 0 0) 0x7f15000d1a70 con 0x7f15000d2830 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14ec003520 con 0x7f15000d2830 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3101521485 0 0) 0x7f14ec003520 con 0x7f15000d2830 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 >> v2:172.21.15.184:3300/0 conn(0x7f1500163610 msgr2=0x7f15000d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 --2- 172.21.15.40:0/167923952 >> v2:172.21.15.184:3300/0 conn(0x7f1500163610 0x7f15000d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 >> v1:172.21.15.184:6789/0 conn(0x7f150015f5a0 legacy=0x7f1500162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15000d2030 con 0x7f15000d2830 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f14f80019f0 con 0x7f15000d2830 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 >> v1:172.21.15.40:6789/0 conn(0x7f15000d2830 legacy=0x7f15000cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14ec0039b0 0x7f14ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec004800 0x7f14ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14ec0051c0 0x7f14ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec00c130 con 0x7f14ec0051c0 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec00c2e0 con 0x7f14ec004800 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec00c4e0 con 0x7f14ec0039b0 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14ff7fe640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14ec0051c0 0x7f14ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14f67fc640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec004800 0x7f14ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.990+0000 7f14feffd640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14ec0039b0 0x7f14ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.991+0000 7f14ff7fe640 1 -- 172.21.15.40:0/167923952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14ec0039b0 msgr2=0x7f14ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.991+0000 7f14ff7fe640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14ec0039b0 0x7f14ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.991+0000 7f14ff7fe640 1 -- 172.21.15.40:0/167923952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec004800 msgr2=0x7f14ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.991+0000 7f14ff7fe640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec004800 0x7f14ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.991+0000 7f14ff7fe640 1 -- 172.21.15.40:0/167923952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec003520 con 0x7f14ec0051c0 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.991+0000 7f14ff7fe640 1 -- 172.21.15.40:0/167923952 reap_dead start 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.991+0000 7f14ff7fe640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14ec0051c0 0x7f14ec00b9c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f14f800a850 tx=0x7f14ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447e1e94561bdfa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.991+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f800a4a0 con 0x7f14ec0051c0 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f802f070 con 0x7f14ec0051c0 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f14fe7fc640 1 -- 172.21.15.40:0/167923952 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f802bce0 con 0x7f14ec0051c0 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f15056bd640 1 -- 172.21.15.40:0/167923952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14ec0051c0 msgr2=0x7f14ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f15056bd640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14ec0051c0 0x7f14ec00b9c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f14f800a850 tx=0x7f14ec005830 comp rx=0 tx=0).stop 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f15056bd640 1 -- 172.21.15.40:0/167923952 shutdown_connections 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f15056bd640 1 --2- 172.21.15.40:0/167923952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14ec0051c0 0x7f14ec00b9c0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f15056bd640 1 -- 172.21.15.40:0/167923952 >> 172.21.15.40:0/167923952 conn(0x7f15000d5190 msgr2=0x7f15000d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:39.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f15056bd640 1 -- 172.21.15.40:0/167923952 shutdown_connections 2024-04-26T21:41:39.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.992+0000 7f15056bd640 1 -- 172.21.15.40:0/167923952 wait complete. 2024-04-26T21:41:39.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.993+0000 7f15056bd640 1 Processor -- start 2024-04-26T21:41:40.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.993+0000 7f15056bd640 1 -- start start 2024-04-26T21:41:40.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f15056bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec0051c0 0x7f15000ba340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:40.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f15056bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15000b8c10 0x7f15001f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:40.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f15056bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15000cee30 0x7f1500146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:40.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f15056bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15000d3c20 con 0x7f15000b8c10 2024-04-26T21:41:40.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f15056bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15000d4190 con 0x7f14ec0051c0 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f15056bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15000d3da0 con 0x7f15000cee30 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f14f67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15000b8c10 0x7f15001f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f14f67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15000b8c10 0x7f15001f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41352/0 (socket says 172.21.15.40:41352) 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f14f67fc640 1 -- 172.21.15.40:0/213927975 learned_addr learned my addr 172.21.15.40:0/213927975 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f14ff7fe640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15000cee30 0x7f1500146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.994+0000 7f14feffd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec0051c0 0x7f15000ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f67fc640 1 -- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15000cee30 msgr2=0x7f1500146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f67fc640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15000cee30 0x7f1500146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f67fc640 1 -- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec0051c0 msgr2=0x7f15000ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f67fc640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec0051c0 0x7f15000ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f67fc640 1 -- 172.21.15.40:0/213927975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15000b92a0 con 0x7f15000b8c10 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f67fc640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15000b8c10 0x7f15001f51b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f14e80118e0 tx=0x7f14e8011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f7fff640 1 -- 172.21.15.40:0/213927975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800faf0 con 0x7f15000b8c10 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f7fff640 1 -- 172.21.15.40:0/213927975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e800fc90 con 0x7f15000b8c10 2024-04-26T21:41:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15000b9580 con 0x7f15000b8c10 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f14f7fff640 1 -- 172.21.15.40:0/213927975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8014440 con 0x7f15000b8c10 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.995+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15000b9b40 con 0x7f15000b8c10 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.996+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15000694f0 con 0x7f15000b8c10 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.997+0000 7f14f7fff640 1 -- 172.21.15.40:0/213927975 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14e8012040 con 0x7f15000b8c10 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.997+0000 7f14f7fff640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14e006c920 0x7f14e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.997+0000 7f14f7fff640 1 -- 172.21.15.40:0/213927975 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f14e808c3f0 con 0x7f15000b8c10 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.997+0000 7f14feffd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14e006c920 0x7f14e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:39.998+0000 7f14feffd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14e006c920 0x7f14e006ede0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f14f0005590 tx=0x7f14f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.001+0000 7f14f7fff640 1 -- 172.21.15.40:0/213927975 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14e8019030 con 0x7f15000b8c10 2024-04-26T21:41:40.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.226+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1500068d30 con 0x7f15000b8c10 2024-04-26T21:41:40.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.227+0000 7f14f7fff640 1 -- 172.21.15.40:0/213927975 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+4355 (secure 0 0 0) 0x7f14e805a080 con 0x7f15000b8c10 2024-04-26T21:41:40.228 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:40.228 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:39.656540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7224":{"gid":7224,"name":"a","rank":0,"incarnation":75,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-26T21:41:40.228 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 75 2024-04-26T21:41:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14e006c920 msgr2=0x7f14e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14e006c920 0x7f14e006ede0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f14f0005590 tx=0x7f14f0009040 comp rx=0 tx=0).stop 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15000b8c10 msgr2=0x7f15001f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15000b8c10 0x7f15001f51b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f14e80118e0 tx=0x7f14e8011da0 comp rx=0 tx=0).stop 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 shutdown_connections 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f14e006c920 0x7f14e006ede0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15000cee30 0x7f1500146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15000b8c10 0x7f15001f51b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 --2- 172.21.15.40:0/213927975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14ec0051c0 0x7f15000ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.228+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 >> 172.21.15.40:0/213927975 conn(0x7f15000d5190 msgr2=0x7f15000d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.229+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 shutdown_connections 2024-04-26T21:41:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:40.229+0000 7f15056bd640 1 -- 172.21.15.40:0/213927975 wait complete. 2024-04-26T21:41:40.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:41:07.249301+0000', 'modified': '2024-04-26T21:41:39.656540+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7224': {'gid': 7224, 'name': 'a', 'rank': 0, 'incarnation': 75, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/836437049', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 836437049}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 836437049}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:41:40.642 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7224:up:replay 2024-04-26T21:41:41.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:41:41.622 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:41:41.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:41.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.772+0000 7f5cac285640 1 Processor -- start 2024-04-26T21:41:41.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.772+0000 7f5cac285640 1 -- start start 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.773+0000 7f5cac285640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ca40b2340 0x7f5ca40cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.773+0000 7f5cac285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca40d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.773+0000 7f5cac285640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca40d10d0 con 0x7f5ca4162810 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.773+0000 7f5cac285640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca40d0750 con 0x7f5ca40b2340 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.773+0000 7f5cac285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca40d16a0 con 0x7f5ca40cd930 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.773+0000 7f5ca9820640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ca40b2340 0x7f5ca40cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.774+0000 7f5ca9820640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ca40b2340 0x7f5ca40cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41360/0 (socket says 172.21.15.40:41360) 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.774+0000 7f5ca9820640 1 -- 172.21.15.40:0/1828785886 learned_addr learned my addr 172.21.15.40:0/1828785886 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.774+0000 7f5caa822640 1 --2- 172.21.15.40:0/1828785886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca40d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.774+0000 7f5ca9820640 1 -- 172.21.15.40:0/1828785886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 msgr2=0x7f5ca40d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.774+0000 7f5ca9820640 1 --2- 172.21.15.40:0/1828785886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca40d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.774+0000 7f5ca9820640 1 -- 172.21.15.40:0/1828785886 >> v1:172.21.15.184:6789/0 conn(0x7f5ca4162810 legacy=0x7f5ca4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.774+0000 7f5ca9820640 1 -- 172.21.15.40:0/1828785886 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca40d1370 con 0x7f5ca40b2340 2024-04-26T21:41:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.774+0000 7f5ca9820640 1 --2- 172.21.15.40:0/1828785886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40cd3f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40d1820 tx=0x7f5c9402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b41eddd1dd929f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5ca901f640 1 -- 172.21.15.40:0/1828785886 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94037070 con 0x7f5ca40b2340 2024-04-26T21:41:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5ca901f640 1 -- 172.21.15.40:0/1828785886 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c94032440 con 0x7f5ca40b2340 2024-04-26T21:41:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5ca901f640 1 -- 172.21.15.40:0/1828785886 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94032810 con 0x7f5ca40b2340 2024-04-26T21:41:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5cac285640 1 -- 172.21.15.40:0/1828785886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 msgr2=0x7f5ca40cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5cac285640 1 --2- 172.21.15.40:0/1828785886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40cd3f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40d1820 tx=0x7f5c9402a250 comp rx=0 tx=0).stop 2024-04-26T21:41:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5cac285640 1 -- 172.21.15.40:0/1828785886 shutdown_connections 2024-04-26T21:41:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5cac285640 1 --2- 172.21.15.40:0/1828785886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca40d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:41.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5cac285640 1 --2- 172.21.15.40:0/1828785886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40cd3f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:41.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5cac285640 1 -- 172.21.15.40:0/1828785886 >> 172.21.15.40:0/1828785886 conn(0x7f5ca40d2da0 msgr2=0x7f5ca40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:41.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5cac285640 1 -- 172.21.15.40:0/1828785886 shutdown_connections 2024-04-26T21:41:41.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.775+0000 7f5cac285640 1 -- 172.21.15.40:0/1828785886 wait complete. 2024-04-26T21:41:41.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.776+0000 7f5cac285640 1 Processor -- start 2024-04-26T21:41:41.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5cac285640 1 -- start start 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5cac285640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5cac285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5cac285640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca4162810 0x7f5ca40c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5cac285640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40d2070 con 0x7f5ca40b2340 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5cac285640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40d0470 con 0x7f5ca4162810 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5cac285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca40d21f0 con 0x7f5ca40cd930 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5caa021640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5caa021640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41370/0 (socket says 172.21.15.40:41370) 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.777+0000 7f5caa021640 1 -- 172.21.15.40:0/226664029 learned_addr learned my addr 172.21.15.40:0/226664029 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5ca9820640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5caa822640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca4162810 0x7f5ca40c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5caa021640 1 -- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 msgr2=0x7f5ca415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5caa021640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5caa021640 1 -- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca4162810 msgr2=0x7f5ca40c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5caa021640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca4162810 0x7f5ca40c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5caa021640 1 -- 172.21.15.40:0/226664029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca40c6b00 con 0x7f5ca40b2340 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5ca9820640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5caa822640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca4162810 0x7f5ca40c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5caa021640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40b04c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c3f0 tx=0x7f5ca000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5c9affd640 1 -- 172.21.15.40:0/226664029 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca00052d0 con 0x7f5ca40b2340 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5c9affd640 1 -- 172.21.15.40:0/226664029 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca0005490 con 0x7f5ca40b2340 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca40c6de0 con 0x7f5ca40b2340 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5c9affd640 1 -- 172.21.15.40:0/226664029 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0007260 con 0x7f5ca40b2340 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.778+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca40c73a0 con 0x7f5ca40b2340 2024-04-26T21:41:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.779+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca40694f0 con 0x7f5ca40b2340 2024-04-26T21:41:41.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.779+0000 7f5c9affd640 1 -- 172.21.15.40:0/226664029 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ca0007400 con 0x7f5ca40b2340 2024-04-26T21:41:41.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.780+0000 7f5c9affd640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c8806c8f0 0x7f5c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:41.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.780+0000 7f5c9affd640 1 -- 172.21.15.40:0/226664029 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f5ca008b1f0 con 0x7f5ca40b2340 2024-04-26T21:41:41.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.780+0000 7f5ca9820640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c8806c8f0 0x7f5c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:41.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.781+0000 7f5ca9820640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c8806c8f0 0x7f5c8806edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5c94034000 tx=0x7f5c9402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:41.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:41.784+0000 7f5c9affd640 1 -- 172.21.15.40:0/226664029 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ca0059440 con 0x7f5ca40b2340 2024-04-26T21:41:42.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.009+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ca406ce60 con 0x7f5ca40b2340 2024-04-26T21:41:42.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.009+0000 7f5c9affd640 1 -- 172.21.15.40:0/226664029 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5178 (secure 0 0 0) 0x7f5ca0058d40 con 0x7f5ca40b2340 2024-04-26T21:41:42.010 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:42.010 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":77,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7485,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/379824506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":379824506},{"type":"v1","addr":"172.21.15.40:6835","nonce":379824506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:40.669878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7224":{"gid":7224,"name":"a","rank":0,"incarnation":75,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-26T21:41:42.011 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 77 2024-04-26T21:41:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c8806c8f0 msgr2=0x7f5c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c8806c8f0 0x7f5c8806edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5c94034000 tx=0x7f5c9402e000 comp rx=0 tx=0).stop 2024-04-26T21:41:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 msgr2=0x7f5ca40b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40b04c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c3f0 tx=0x7f5ca000c8b0 comp rx=0 tx=0).stop 2024-04-26T21:41:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 shutdown_connections 2024-04-26T21:41:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c8806c8f0 0x7f5c8806edb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca4162810 0x7f5ca40c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca40cd930 0x7f5ca415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 --2- 172.21.15.40:0/226664029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca40b2340 0x7f5ca40b04c0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 >> 172.21.15.40:0/226664029 conn(0x7f5ca40d2da0 msgr2=0x7f5ca40b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 shutdown_connections 2024-04-26T21:41:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42.011+0000 7f5cac285640 1 -- 172.21.15.40:0/226664029 wait complete. 2024-04-26T21:41:42.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:41:07.249301+0000', 'modified': '2024-04-26T21:41:40.669878+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7224': {'gid': 7224, 'name': 'a', 'rank': 0, 'incarnation': 75, 'state': 'up:rejoin', 'state_seq': 12, 'addr': '172.21.15.40:6833/836437049', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 836437049}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 836437049}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:41:42.425 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7224:up:rejoin 2024-04-26T21:41:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:41:43.426 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:43.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.540+0000 7f7594ec5640 1 Processor -- start 2024-04-26T21:41:43.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.540+0000 7f7594ec5640 1 -- start start 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f7594ec5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f7594ec5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75900b5f20 0x7f75900b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f7594ec5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75900b7e50 con 0x7f7590162880 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f7594ec5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f75900d0580 con 0x7f75900b5f20 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f7594ec5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75900d1800 con 0x7f75900d3bc0 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75900b5f20 0x7f75900b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75900b5f20 0x7f75900b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41378/0 (socket says 172.21.15.40:41378) 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758ed9d640 1 -- 172.21.15.40:0/153182946 learned_addr learned my addr 172.21.15.40:0/153182946 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758dd9b640 1 --2- 172.21.15.40:0/153182946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:43.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758ed9d640 1 -- 172.21.15.40:0/153182946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 msgr2=0x7f75900d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:43.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758ed9d640 1 --2- 172.21.15.40:0/153182946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758ed9d640 1 -- 172.21.15.40:0/153182946 >> v1:172.21.15.184:6789/0 conn(0x7f7590162880 legacy=0x7f75900d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:43.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758ed9d640 1 -- 172.21.15.40:0/153182946 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75900d07d0 con 0x7f75900b5f20 2024-04-26T21:41:43.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.541+0000 7f758dd9b640 1 --2- 172.21.15.40:0/153182946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:43.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.542+0000 7f758ed9d640 1 --2- 172.21.15.40:0/153182946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900b5f20 0x7f75900b8300 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7588009e50 tx=0x7f758802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91f092d44ae6f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.542+0000 7f758d59a640 1 -- 172.21.15.40:0/153182946 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588036070 con 0x7f75900b5f20 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.542+0000 7f758d59a640 1 -- 172.21.15.40:0/153182946 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7588031600 con 0x7f75900b5f20 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.542+0000 7f758d59a640 1 -- 172.21.15.40:0/153182946 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880319d0 con 0x7f75900b5f20 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.543+0000 7f7594ec5640 1 -- 172.21.15.40:0/153182946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900b5f20 msgr2=0x7f75900b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.543+0000 7f7594ec5640 1 --2- 172.21.15.40:0/153182946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900b5f20 0x7f75900b8300 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7588009e50 tx=0x7f758802a710 comp rx=0 tx=0).stop 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.543+0000 7f7594ec5640 1 -- 172.21.15.40:0/153182946 shutdown_connections 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.543+0000 7f7594ec5640 1 --2- 172.21.15.40:0/153182946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900b5f20 0x7f75900b8300 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.543+0000 7f7594ec5640 1 --2- 172.21.15.40:0/153182946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.543+0000 7f7594ec5640 1 -- 172.21.15.40:0/153182946 >> 172.21.15.40:0/153182946 conn(0x7f75900d4f60 msgr2=0x7f75900d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.543+0000 7f7594ec5640 1 -- 172.21.15.40:0/153182946 shutdown_connections 2024-04-26T21:41:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.543+0000 7f7594ec5640 1 -- 172.21.15.40:0/153182946 wait complete. 2024-04-26T21:41:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.544+0000 7f7594ec5640 1 Processor -- start 2024-04-26T21:41:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.544+0000 7f7594ec5640 1 -- start start 2024-04-26T21:41:43.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f7594ec5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75900b5f20 0x7f7590153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:43.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f7594ec5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:43.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f7594ec5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7590162880 0x7f75900cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f7594ec5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7590162c60 con 0x7f7590162880 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f7594ec5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75900d0d90 con 0x7f75900b5f20 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f7594ec5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7590162de0 con 0x7f75900d3bc0 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f758ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7590162880 0x7f75900cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f758ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7590162880 0x7f75900cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41390/0 (socket says 172.21.15.40:41390) 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.545+0000 7f758ed9d640 1 -- 172.21.15.40:0/1098457340 learned_addr learned my addr 172.21.15.40:0/1098457340 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758dd9b640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758e59c640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75900b5f20 0x7f7590153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758ed9d640 1 -- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 msgr2=0x7f75900b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758ed9d640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758ed9d640 1 -- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75900b5f20 msgr2=0x7f7590153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758ed9d640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75900b5f20 0x7f7590153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758ed9d640 1 -- 172.21.15.40:0/1098457340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75900ce980 con 0x7f7590162880 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758dd9b640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f758ed9d640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7590162880 0x7f75900cdf30 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7588034040 tx=0x7f7588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f7586ffd640 1 -- 172.21.15.40:0/1098457340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588036050 con 0x7f7590162880 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f7586ffd640 1 -- 172.21.15.40:0/1098457340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7588042070 con 0x7f7590162880 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75900cec60 con 0x7f7590162880 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f7586ffd640 1 -- 172.21.15.40:0/1098457340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588009510 con 0x7f7590162880 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.546+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75900b5610 con 0x7f7590162880 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.547+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75900764d0 con 0x7f7590162880 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.548+0000 7f7586ffd640 1 -- 172.21.15.40:0/1098457340 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7588007c50 con 0x7f7590162880 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.548+0000 7f7586ffd640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f757006c8f0 0x7f757006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.548+0000 7f7586ffd640 1 -- 172.21.15.40:0/1098457340 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f75880acce0 con 0x7f7590162880 2024-04-26T21:41:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.549+0000 7f758e59c640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f757006c8f0 0x7f757006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.549+0000 7f758e59c640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f757006c8f0 0x7f757006edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7580001640 tx=0x7f7580009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.553+0000 7f7586ffd640 1 -- 172.21.15.40:0/1098457340 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75900764d0 con 0x7f7590162880 2024-04-26T21:41:43.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.778+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75900694f0 con 0x7f7590162880 2024-04-26T21:41:43.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.778+0000 7f7586ffd640 1 -- 172.21.15.40:0/1098457340 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5185 (secure 0 0 0) 0x7f758807ae90 con 0x7f7590162880 2024-04-26T21:41:43.778 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:43.778 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7485,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/379824506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":379824506},{"type":"v1","addr":"172.21.15.40:6835","nonce":379824506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:42.673330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7224":{"gid":7224,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7224,"qdb_cluster":[7224]},"id":5}]} 2024-04-26T21:41:43.779 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 78 2024-04-26T21:41:43.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f757006c8f0 msgr2=0x7f757006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:43.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f757006c8f0 0x7f757006edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7580001640 tx=0x7f7580009040 comp rx=0 tx=0).stop 2024-04-26T21:41:43.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7590162880 msgr2=0x7f75900cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:43.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7590162880 0x7f75900cdf30 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7588034040 tx=0x7f7588004a40 comp rx=0 tx=0).stop 2024-04-26T21:41:43.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 shutdown_connections 2024-04-26T21:41:43.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f757006c8f0 0x7f757006edb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7590162880 0x7f75900cdf30 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75900d3bc0 0x7f75900b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 --2- 172.21.15.40:0/1098457340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75900b5f20 0x7f7590153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:43.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 >> 172.21.15.40:0/1098457340 conn(0x7f75900d4f60 msgr2=0x7f75900d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:43.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 shutdown_connections 2024-04-26T21:41:43.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:43.780+0000 7f7594ec5640 1 -- 172.21.15.40:0/1098457340 wait complete. 2024-04-26T21:41:43.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-26T21:41:43.853 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:41:43.854+0000 7f171689a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:41:44.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:41:07.249301+0000', 'modified': '2024-04-26T21:41:42.673330+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7224': {'gid': 7224, 'name': 'a', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.40:6833/836437049', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 836437049}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 836437049}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7224, 'qdb_cluster': [7224]} 2024-04-26T21:41:44.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:41:44.195 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:41:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.301+0000 7f65ab89b640 1 Processor -- start 2024-04-26T21:41:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.301+0000 7f65ab89b640 1 -- start start 2024-04-26T21:41:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65ab89b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f65a40b0380 0x7f65a40ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65ab89b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65a40b10b0 0x7f65a406c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65ab89b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 0x7f65a4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65ab89b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f65a40cfd80 con 0x7f65a40b10b0 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65ab89b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f65a40d0020 con 0x7f65a40b0380 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65ab89b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65a40d0300 con 0x7f65a406c780 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65a8e36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65a40b10b0 0x7f65a406c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65a8e36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65a40b10b0 0x7f65a406c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41406/0 (socket says 172.21.15.40:41406) 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65a8e36640 1 -- 172.21.15.40:0/146065221 learned_addr learned my addr 172.21.15.40:0/146065221 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65a9637640 1 --2- 172.21.15.40:0/146065221 >> v2:172.21.15.184:3300/0 conn(0x7f65a40b0380 0x7f65a40ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.302+0000 7f65a9e38640 1 --2- 172.21.15.40:0/146065221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 0x7f65a4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f65a8e36640 1 -- 172.21.15.40:0/146065221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 msgr2=0x7f65a4072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f65a8e36640 1 --2- 172.21.15.40:0/146065221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 0x7f65a4072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f65a8e36640 1 -- 172.21.15.40:0/146065221 >> v2:172.21.15.184:3300/0 conn(0x7f65a40b0380 msgr2=0x7f65a40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f65a8e36640 1 --2- 172.21.15.40:0/146065221 >> v2:172.21.15.184:3300/0 conn(0x7f65a40b0380 0x7f65a40ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f65a8e36640 1 -- 172.21.15.40:0/146065221 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a40d05e0 con 0x7f65a40b10b0 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f65a8e36640 1 --2- 172.21.15.40:0/146065221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a406c240 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f65a406d290 tx=0x7f659402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32411586d4d504df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f659bfff640 1 -- 172.21.15.40:0/146065221 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037070 con 0x7f65a40b10b0 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f659bfff640 1 -- 172.21.15.40:0/146065221 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594032440 con 0x7f65a40b10b0 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.303+0000 7f659bfff640 1 -- 172.21.15.40:0/146065221 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594032810 con 0x7f65a40b10b0 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 -- 172.21.15.40:0/146065221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 msgr2=0x7f65a406c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 --2- 172.21.15.40:0/146065221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a406c240 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f65a406d290 tx=0x7f659402a140 comp rx=0 tx=0).stop 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 -- 172.21.15.40:0/146065221 shutdown_connections 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 --2- 172.21.15.40:0/146065221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 0x7f65a4072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 --2- 172.21.15.40:0/146065221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a406c240 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 --2- 172.21.15.40:0/146065221 >> v2:172.21.15.184:3300/0 conn(0x7f65a40b0380 0x7f65a40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 -- 172.21.15.40:0/146065221 >> 172.21.15.40:0/146065221 conn(0x7f65a40d1cf0 msgr2=0x7f65a40d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 -- 172.21.15.40:0/146065221 shutdown_connections 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.304+0000 7f65ab89b640 1 -- 172.21.15.40:0/146065221 wait complete. 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.305+0000 7f65ab89b640 1 Processor -- start 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.305+0000 7f65ab89b640 1 -- start start 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65ab89b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 0x7f65a4156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65ab89b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65a40b0380 0x7f65a40cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65ab89b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a40ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65ab89b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40d15c0 con 0x7f65a40b10b0 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65ab89b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40b0760 con 0x7f65a40b0380 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65ab89b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65a40d1740 con 0x7f65a406c780 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65a9e38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a40ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65a9e38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a40ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41414/0 (socket says 172.21.15.40:41414) 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65a9e38640 1 -- 172.21.15.40:0/1029209088 learned_addr learned my addr 172.21.15.40:0/1029209088 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65a9637640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 0x7f65a4156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.306+0000 7f65a8e36640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65a40b0380 0x7f65a40cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65a9e38640 1 -- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 msgr2=0x7f65a4156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65a9e38640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 0x7f65a4156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65a9e38640 1 -- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65a40b0380 msgr2=0x7f65a40cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65a9e38640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65a40b0380 0x7f65a40cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65a9e38640 1 -- 172.21.15.40:0/1029209088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a40ca460 con 0x7f65a40b10b0 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65a8e36640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65a40b0380 0x7f65a40cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65a9e38640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a40ce710 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f659c00eed0 tx=0x7f659c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f6599ffb640 1 -- 172.21.15.40:0/1029209088 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c018020 con 0x7f65a40b10b0 2024-04-26T21:41:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f6599ffb640 1 -- 172.21.15.40:0/1029209088 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c00bd20 con 0x7f65a40b10b0 2024-04-26T21:41:44.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65ab89b640 1 -- 172.21.15.40:0/1029209088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a40ca740 con 0x7f65a40b10b0 2024-04-26T21:41:44.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f6599ffb640 1 -- 172.21.15.40:0/1029209088 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c014640 con 0x7f65a40b10b0 2024-04-26T21:41:44.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.307+0000 7f65ab89b640 1 -- 172.21.15.40:0/1029209088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a40cad00 con 0x7f65a40b10b0 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.309+0000 7f6599ffb640 1 -- 172.21.15.40:0/1029209088 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f659c005d10 con 0x7f65a40b10b0 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.309+0000 7f6599ffb640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c8f0 0x7f658806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.310+0000 7f6599ffb640 1 -- 172.21.15.40:0/1029209088 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f659c08c300 con 0x7f65a40b10b0 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.310+0000 7f65a9637640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c8f0 0x7f658806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.310+0000 7f65ae5b0740 1 Processor -- start 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.310+0000 7f65ae5b0740 1 -- start start 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.310+0000 7f65a9637640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c8f0 0x7f658806edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009b40 tx=0x7f65a0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef44b440 0x5556ef44b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef469230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef421120 0x5556ef4673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5556ef385740 con 0x5556ef421120 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5556ef3213a0 con 0x5556ef44b440 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5556ef381700 con 0x5556ef468ce0 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9e38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef421120 0x5556ef4673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9e38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef421120 0x5556ef4673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41416/0 (socket says 172.21.15.40:41416) 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9e38640 1 -- 172.21.15.40:0/3934884965 learned_addr learned my addr 172.21.15.40:0/3934884965 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9637640 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef44b440 0x5556ef44b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a8e36640 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef469230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9e38640 1 -- 172.21.15.40:0/3934884965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 msgr2=0x5556ef469230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9e38640 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef469230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9e38640 1 -- 172.21.15.40:0/3934884965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef44b440 msgr2=0x5556ef44b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9e38640 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef44b440 0x5556ef44b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.311+0000 7f65a9e38640 1 -- 172.21.15.40:0/3934884965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ef33d750 con 0x5556ef421120 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65a9e38640 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef421120 0x5556ef4673a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f659c062500 tx=0x7f659c059e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c3b58568794128 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f658effd640 1 -- 172.21.15.40:0/3934884965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c088070 con 0x5556ef421120 2024-04-26T21:41:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f658effd640 1 -- 172.21.15.40:0/3934884965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c05c070 con 0x5556ef421120 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f658effd640 1 -- 172.21.15.40:0/3934884965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c072070 con 0x5556ef421120 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3934884965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef421120 msgr2=0x5556ef4673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef421120 0x5556ef4673a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f659c062500 tx=0x7f659c059e50 comp rx=0 tx=0).stop 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3934884965 shutdown_connections 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef421120 0x5556ef4673a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef469230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/3934884965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef44b440 0x5556ef44b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3934884965 >> 172.21.15.40:0/3934884965 conn(0x5556ef41da50 msgr2=0x5556ef466250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3934884965 shutdown_connections 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.312+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3934884965 wait complete. 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.313+0000 7f65ae5b0740 1 -- start start 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.313+0000 7f65ae5b0740 1 Processor -- start 2024-04-26T21:41:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 0x5556ef477270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef468ce0 0x5556ef41aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 0x5556ef44dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5556ef421ec0 con 0x5556ef468ce0 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5556ef30cfe0 con 0x5556ef45ebe0 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5556ef41b980 con 0x5556ef44b440 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65ae5b0740 10 client.0 _ll_register_callbacks cb 0x5556ef323150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef468ce0 0x5556ef41aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef468ce0 0x5556ef41aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41430/0 (socket says 172.21.15.40:41430) 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 -- 172.21.15.40:0/1100372935 learned_addr learned my addr 172.21.15.40:0/1100372935 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a9e38640 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 0x5556ef44dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a9637640 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 0x5556ef477270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 -- 172.21.15.40:0/1100372935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 msgr2=0x5556ef477270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 0x5556ef477270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 -- 172.21.15.40:0/1100372935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 msgr2=0x5556ef44dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 0x5556ef44dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 -- 172.21.15.40:0/1100372935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ef0972d0 con 0x5556ef468ce0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f65a8e36640 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef468ce0 0x5556ef41aa70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6594035040 tx=0x7f6594004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706352134fb76faa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.314+0000 7f6583fff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f6583fff640 1 -- 172.21.15.40:0/1100372935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037040 con 0x5556ef468ce0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f6583fff640 1 -- 172.21.15.40:0/1100372935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f6594032d40 con 0x5556ef468ce0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 1 -- 172.21.15.40:0/1100372935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c05c070 con 0x5556ef468ce0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f6583fff640 1 -- 172.21.15.40:0/1100372935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f6594046410 con 0x5556ef468ce0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 10 client.7515 fetch_fsmap learned FSMap version 78 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 1 -- 172.21.15.40:0/1100372935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x5556ef1616b0 con 0x5556ef468ce0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f6583fff640 1 -- 172.21.15.40:0/1100372935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c05c070 con 0x5556ef468ce0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f6583fff640 1 -- 172.21.15.40:0/1100372935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f65940465b0 con 0x5556ef468ce0 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 10 client.7515 fetch_fsmap finished waiting for FSMap version 78 2024-04-26T21:41:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 10 client.7515 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 20 client.7515 populate_metadata read hostname 'smithi040' 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x5556ef45d440 0x5556ef45d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 4 client.7515 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65ae5b0740 1 -- 172.21.15.40:0/1100372935 --> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556ef393350 con 0x5556ef45d440 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.315+0000 7f65a9637640 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x5556ef45d440 0x5556ef45d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65a9637640 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x5556ef45d440 0x5556ef45d820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b99f3b3fa35bac18 server_cookie=40cca153f1b5a5b3 in_seq=0 out_seq=0 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f6583fff640 10 client.7515 ms_handle_connect on v2:172.21.15.40:6832/836437049 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f6583fff640 1 -- 172.21.15.40:0/1100372935 <== mds.0 v2:172.21.15.40:6832/836437049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556ef393350 con 0x5556ef45d440 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f6583fff640 10 client.7515 handle_command_reply: tid=0 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 client.7515 shutdown 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 -- 172.21.15.40:0/1100372935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef468ce0 msgr2=0x5556ef41aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef468ce0 0x5556ef41aa70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6594035040 tx=0x7f6594004730 comp rx=0 tx=0).stop 2024-04-26T21:41:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 -- 172.21.15.40:0/1100372935 shutdown_connections 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x5556ef45d440 0x5556ef45d820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 0x5556ef44dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef468ce0 0x5556ef41aa70 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f6583fff640 0 client.7515 ms_handle_reset on v2:172.21.15.40:6832/836437049 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/1100372935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 0x5556ef477270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.316+0000 7f65ae5b0740 1 -- 172.21.15.40:0/1100372935 >> 172.21.15.40:0/1100372935 conn(0x5556ef421120 msgr2=0x5556ef4697d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.317+0000 7f65ae5b0740 1 -- 172.21.15.40:0/1100372935 shutdown_connections 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.317+0000 7f65ae5b0740 1 -- 172.21.15.40:0/1100372935 wait complete. 2024-04-26T21:41:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.317+0000 7f65ae5b0740 20 client.7515 trim_cache size 0 max 16384 2024-04-26T21:41:44.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.335+0000 7f65ae5b0740 1 Processor -- start 2024-04-26T21:41:44.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.335+0000 7f65ae5b0740 1 -- start start 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 0x5556ef44ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef45ebe0 0x5556ef4685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef468ce0 0x5556ef421710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4201ed0 con 0x5556ef45ebe0 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4200f90 con 0x5556ef468ce0 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65a40c9eb0 con 0x5556ef44b440 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef45ebe0 0x5556ef4685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef45ebe0 0x5556ef4685f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41442/0 (socket says 172.21.15.40:41442) 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 -- 172.21.15.40:0/3380480102 learned_addr learned my addr 172.21.15.40:0/3380480102 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 -- 172.21.15.40:0/3380480102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 msgr2=0x5556ef44ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 --2- 172.21.15.40:0/3380480102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 0x5556ef44ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 -- 172.21.15.40:0/3380480102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef468ce0 msgr2=0x5556ef421710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 --2- 172.21.15.40:0/3380480102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef468ce0 0x5556ef421710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 -- 172.21.15.40:0/3380480102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ef460140 con 0x5556ef45ebe0 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f65a8e36640 1 --2- 172.21.15.40:0/3380480102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef45ebe0 0x5556ef4685f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6594009960 tx=0x7f659403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9e9be0e39f8fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f658e7fc640 1 -- 172.21.15.40:0/3380480102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037070 con 0x5556ef45ebe0 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f658e7fc640 1 -- 172.21.15.40:0/3380480102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594046ab0 con 0x5556ef45ebe0 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.336+0000 7f658e7fc640 1 -- 172.21.15.40:0/3380480102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594032d80 con 0x5556ef45ebe0 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3380480102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef45ebe0 msgr2=0x5556ef4685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/3380480102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef45ebe0 0x5556ef4685f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6594009960 tx=0x7f659403d000 comp rx=0 tx=0).stop 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3380480102 shutdown_connections 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/3380480102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef468ce0 0x5556ef421710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/3380480102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef45ebe0 0x5556ef4685f0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/3380480102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef44b440 0x5556ef44ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3380480102 >> 172.21.15.40:0/3380480102 conn(0x5556ef421120 msgr2=0x5556ef470460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3380480102 shutdown_connections 2024-04-26T21:41:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 -- 172.21.15.40:0/3380480102 wait complete. 2024-04-26T21:41:44.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 -- start start 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.337+0000 7f65ae5b0740 1 Processor -- start 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef44b440 0x5556ef466f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 0x5556ef4674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65ae5b0740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef45d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5556ef479e10 con 0x5556ef44b440 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4200940 con 0x5556ef45ebe0 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65ae5b0740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5556ef447fe0 con 0x5556ef468ce0 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65ae5b0740 10 client.0 _ll_register_callbacks cb 0x7f65a0009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65a9637640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef44b440 0x5556ef466f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65a9637640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef44b440 0x5556ef466f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41446/0 (socket says 172.21.15.40:41446) 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65a9637640 1 -- 172.21.15.40:0/2291653415 learned_addr learned my addr 172.21.15.40:0/2291653415 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.338+0000 7f65a8e36640 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 0x5556ef4674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65a9e38640 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef45d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65a9637640 1 -- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 msgr2=0x5556ef45d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65a9637640 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef45d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65a9637640 1 -- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 msgr2=0x5556ef4674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65a9637640 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 0x5556ef4674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65a9637640 1 -- 172.21.15.40:0/2291653415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ef466650 con 0x5556ef44b440 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65a9e38640 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef45d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65a9637640 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef44b440 0x5556ef466f60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x5556ef447b10 tx=0x7f65a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e1c6c87dff7842 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f6583fff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f6583fff640 1 -- 172.21.15.40:0/2291653415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a002b340 con 0x5556ef44b440 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f6583fff640 1 -- 172.21.15.40:0/2291653415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f65a002b4e0 con 0x5556ef44b440 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65ae5b0740 1 -- 172.21.15.40:0/2291653415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6594046ab0 con 0x5556ef44b440 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f6583fff640 1 -- 172.21.15.40:0/2291653415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f65a002b6e0 con 0x5556ef44b440 2024-04-26T21:41:44.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65ae5b0740 10 client.7521 fetch_fsmap learned FSMap version 78 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65ae5b0740 1 -- 172.21.15.40:0/2291653415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x5556ef48a720 con 0x5556ef44b440 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f6583fff640 1 -- 172.21.15.40:0/2291653415 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594046ab0 con 0x5556ef44b440 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f6583fff640 1 -- 172.21.15.40:0/2291653415 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f65a002b020 con 0x5556ef44b440 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.339+0000 7f65ae5b0740 10 client.7521 fetch_fsmap finished waiting for FSMap version 78 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 10 client.7521 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 20 client.7521 populate_metadata read hostname 'smithi040' 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x5556ef465610 0x5556ef4659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 4 client.7521 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 -- 172.21.15.40:0/2291653415 --> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] -- command(tid 0: {"prefix": "status"}) -- 0x5556ef38d1c0 con 0x5556ef465610 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65a8e36640 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x5556ef465610 0x5556ef4659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f6583fff640 10 client.7521 ms_handle_connect on v2:172.21.15.40:6832/836437049 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65a8e36640 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x5556ef465610 0x5556ef4659f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813fb4ce75f1fdaa server_cookie=beef288392e01333 in_seq=0 out_seq=0 2024-04-26T21:41:44.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f6583fff640 1 -- 172.21.15.40:0/2291653415 <== mds.0 v2:172.21.15.40:6832/836437049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5556ef38d1c0 con 0x5556ef465610 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f6583fff640 10 client.7521 handle_command_reply: tid=0 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 client.7521 shutdown 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 -- 172.21.15.40:0/2291653415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef44b440 msgr2=0x5556ef466f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef44b440 0x5556ef466f60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x5556ef447b10 tx=0x7f65a00046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 -- 172.21.15.40:0/2291653415 shutdown_connections 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x5556ef465610 0x5556ef4659f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556ef468ce0 0x5556ef45d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556ef45ebe0 0x5556ef4674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 --2- 172.21.15.40:0/2291653415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556ef44b440 0x5556ef466f60 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f6583fff640 0 client.7521 ms_handle_reset on v2:172.21.15.40:6832/836437049 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 -- 172.21.15.40:0/2291653415 >> 172.21.15.40:0/2291653415 conn(0x5556ef421120 msgr2=0x5556ef45f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 -- 172.21.15.40:0/2291653415 shutdown_connections 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.340+0000 7f65ae5b0740 1 -- 172.21.15.40:0/2291653415 wait complete. 2024-04-26T21:41:44.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ae5b0740 20 client.7521 trim_cache size 0 max 16384 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7224, 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 4.6806592680000003, 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 78, 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 82, 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 82, 2024-04-26T21:41:44.347 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 41.492875634999997 2024-04-26T21:41:44.348 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:41:44.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 -- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c8f0 msgr2=0x7f658806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c8f0 0x7f658806edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009b40 tx=0x7f65a0009190 comp rx=0 tx=0).stop 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 -- 172.21.15.40:0/1029209088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 msgr2=0x7f65a40ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a40ce710 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f659c00eed0 tx=0x7f659c00b480 comp rx=0 tx=0).stop 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 -- 172.21.15.40:0/1029209088 shutdown_connections 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f658806c8f0 0x7f658806edb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a40b10b0 0x7f65a40ce710 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65a40b0380 0x7f65a40cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 --2- 172.21.15.40:0/1029209088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65a406c780 0x7f65a4156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 -- 172.21.15.40:0/1029209088 >> 172.21.15.40:0/1029209088 conn(0x7f65a40d1cf0 msgr2=0x7f65a406ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 -- 172.21.15.40:0/1029209088 shutdown_connections 2024-04-26T21:41:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.341+0000 7f65ab89b640 1 -- 172.21.15.40:0/1029209088 wait complete. 2024-04-26T21:41:44.352 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-04-26T21:41:44.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout: File: a/ 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 2bh/43d Inode: 1099511627776 Links: 4 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1316/ ubuntu) 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:41:15.989652136 +0000 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:41:21.550587697 +0000 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:41:21.550587697 +0000 2024-04-26T21:41:44.403 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:41:15.989652136 +0000 2024-04-26T21:41:44.407 INFO:teuthology.orchestra.run.smithi040.stdout:a/ 2024-04-26T21:41:44.407 INFO:teuthology.orchestra.run.smithi040.stdout:a/z 2024-04-26T21:41:44.407 INFO:teuthology.orchestra.run.smithi040.stdout:a/z/d 2024-04-26T21:41:44.407 INFO:teuthology.orchestra.run.smithi040.stdout:a/c 2024-04-26T21:41:44.408 INFO:teuthology.orchestra.run.smithi040.stdout:a/c/d 2024-04-26T21:41:44.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:41:44.414 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-04-26T21:41:44.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.516+0000 7f4ecc10a640 1 Processor -- start 2024-04-26T21:41:44.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.516+0000 7f4ecc10a640 1 -- start start 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ecc10a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4156460 0x7f4ec40cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ecc10a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ec40cba10 0x7f4ec406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ecc10a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4ec406fcd0 0x7f4ec415f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ecc10a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec4156840 con 0x7f4ec40cba10 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ecc10a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec40cbf20 con 0x7f4ec406fcd0 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ecc10a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec40cc200 con 0x7f4ec4156460 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ec96a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ec40cba10 0x7f4ec406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ec96a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ec40cba10 0x7f4ec406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41448/0 (socket says 172.21.15.40:41448) 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ec96a5640 1 -- 172.21.15.40:0/4025575705 learned_addr learned my addr 172.21.15.40:0/4025575705 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/4025575705 >> v2:172.21.15.184:3300/0 conn(0x7f4ec406fcd0 0x7f4ec415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ec96a5640 1 -- 172.21.15.40:0/4025575705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4156460 msgr2=0x7f4ec40cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/4025575705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4156460 0x7f4ec40cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ec96a5640 1 -- 172.21.15.40:0/4025575705 >> v2:172.21.15.184:3300/0 conn(0x7f4ec406fcd0 msgr2=0x7f4ec415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/4025575705 >> v2:172.21.15.184:3300/0 conn(0x7f4ec406fcd0 0x7f4ec415f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4ec96a5640 1 -- 172.21.15.40:0/4025575705 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec40cc4e0 con 0x7f4ec40cba10 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.517+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/4025575705 >> v2:172.21.15.184:3300/0 conn(0x7f4ec406fcd0 0x7f4ec415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.518+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/4025575705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec40cba10 0x7f4ec406f790 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40ce700 tx=0x7f4eb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a8df81c1318218 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.518+0000 7f4ec8ea4640 1 -- 172.21.15.40:0/4025575705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4037070 con 0x7f4ec40cba10 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.518+0000 7f4ec8ea4640 1 -- 172.21.15.40:0/4025575705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4032440 con 0x7f4ec40cba10 2024-04-26T21:41:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.518+0000 7f4ec8ea4640 1 -- 172.21.15.40:0/4025575705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4032810 con 0x7f4ec40cba10 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.518+0000 7f4ecc10a640 1 -- 172.21.15.40:0/4025575705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec40cba10 msgr2=0x7f4ec406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.519+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/4025575705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec40cba10 0x7f4ec406f790 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40ce700 tx=0x7f4eb402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.519+0000 7f4ecc10a640 1 -- 172.21.15.40:0/4025575705 shutdown_connections 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.519+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/4025575705 >> v2:172.21.15.184:3300/0 conn(0x7f4ec406fcd0 0x7f4ec415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.519+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/4025575705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec40cba10 0x7f4ec406f790 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.519+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/4025575705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4156460 0x7f4ec40cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.519+0000 7f4ecc10a640 1 -- 172.21.15.40:0/4025575705 >> 172.21.15.40:0/4025575705 conn(0x7f4ec40d0be0 msgr2=0x7f4ec40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.519+0000 7f4ecc10a640 1 -- 172.21.15.40:0/4025575705 shutdown_connections 2024-04-26T21:41:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.519+0000 7f4ecc10a640 1 -- 172.21.15.40:0/4025575705 wait complete. 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.520+0000 7f4ecc10a640 1 Processor -- start 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.520+0000 7f4ecc10a640 1 -- start start 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.520+0000 7f4ecc10a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec406fcd0 0x7f4ec40c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ecc10a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec40cba10 0x7f4ec41492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ecc10a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4156460 0x7f4ec40c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ecc10a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec406e260 con 0x7f4ec4156460 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ecc10a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4072700 con 0x7f4ec406fcd0 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ecc10a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec406e3e0 con 0x7f4ec40cba10 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4156460 0x7f4ec40c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4156460 0x7f4ec40c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41464/0 (socket says 172.21.15.40:41464) 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 -- 172.21.15.40:0/709114284 learned_addr learned my addr 172.21.15.40:0/709114284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec406fcd0 0x7f4ec40c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec40cba10 0x7f4ec41492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 -- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec40cba10 msgr2=0x7f4ec41492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec40cba10 0x7f4ec41492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 -- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec406fcd0 msgr2=0x7f4ec40c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec406fcd0 0x7f4ec40c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 -- 172.21.15.40:0/709114284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec40c5b40 con 0x7f4ec4156460 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4156460 0x7f4ec40c98b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00cef0 tx=0x7f4ebc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eba7fc640 1 -- 172.21.15.40:0/709114284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc01a070 con 0x7f4ec4156460 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eba7fc640 1 -- 172.21.15.40:0/709114284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc00d860 con 0x7f4ec4156460 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ecc10a640 1 -- 172.21.15.40:0/709114284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec40c6ac0 con 0x7f4ec4156460 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4eba7fc640 1 -- 172.21.15.40:0/709114284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00dc50 con 0x7f4ec4156460 2024-04-26T21:41:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.521+0000 7f4ecc10a640 1 -- 172.21.15.40:0/709114284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec40c6090 con 0x7f4ec4156460 2024-04-26T21:41:44.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.523+0000 7f4eba7fc640 1 -- 172.21.15.40:0/709114284 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ebc01d070 con 0x7f4ec4156460 2024-04-26T21:41:44.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.523+0000 7f4eba7fc640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4ea806c920 0x7f4ea806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.523+0000 7f4eba7fc640 1 -- 172.21.15.40:0/709114284 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f4ebc08c520 con 0x7f4ec4156460 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.524+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4ea806c920 0x7f4ea806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.524+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4ea806c920 0x7f4ea806ede0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098e0 tx=0x7f4ec0009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ecee28740 1 Processor -- start 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ecee28740 1 -- start start 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a16fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a170350 0x55db6a173850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ecee28740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db6a0d9d80 con 0x55db6a19fd30 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ecee28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55db6a0938b0 con 0x55db6a170350 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ecee28740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55db6a0d5d40 con 0x55db6a1bb6f0 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ec9ea6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ec9ea6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41472/0 (socket says 172.21.15.40:41472) 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.525+0000 7f4ec9ea6640 1 -- 172.21.15.40:0/1656488961 learned_addr learned my addr 172.21.15.40:0/1656488961 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a170350 0x55db6a173850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a16fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ec9ea6640 1 -- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 msgr2=0x55db6a16fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a16fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ec9ea6640 1 -- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a170350 msgr2=0x55db6a173850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a170350 0x55db6a173850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ec9ea6640 1 -- 172.21.15.40:0/1656488961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6a064a40 con 0x55db6a19fd30 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a170350 0x55db6a173850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1a0110 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000eb70 tx=0x7f4ec0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09d0cb4eaea1444 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ea77fe640 1 -- 172.21.15.40:0/1656488961 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec001b040 con 0x55db6a19fd30 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ea77fe640 1 -- 172.21.15.40:0/1656488961 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0024070 con 0x55db6a19fd30 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.526+0000 7f4ea77fe640 1 -- 172.21.15.40:0/1656488961 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0015c20 con 0x55db6a19fd30 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 -- 172.21.15.40:0/1656488961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 msgr2=0x55db6a1a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1a0110 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000eb70 tx=0x7f4ec0011000 comp rx=0 tx=0).stop 2024-04-26T21:41:44.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 -- 172.21.15.40:0/1656488961 shutdown_connections 2024-04-26T21:41:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a170350 0x55db6a173850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a16fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 --2- 172.21.15.40:0/1656488961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1a0110 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 -- 172.21.15.40:0/1656488961 >> 172.21.15.40:0/1656488961 conn(0x55db6a172080 msgr2=0x55db6a1bab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 -- 172.21.15.40:0/1656488961 shutdown_connections 2024-04-26T21:41:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 -- 172.21.15.40:0/1656488961 wait complete. 2024-04-26T21:41:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 -- start start 2024-04-26T21:41:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.527+0000 7f4ecee28740 1 Processor -- start 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a1d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a1bb6f0 0x55db6a197440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 0x55db6a1d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ecee28740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db6a175840 con 0x55db6a1bb6f0 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ecee28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55db6a060d10 con 0x55db6a19fd30 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ecee28740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55db6a175ae0 con 0x55db6a19cf30 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ecee28740 10 client.0 _ll_register_callbacks cb 0x55db6a08d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ec96a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a1bb6f0 0x55db6a197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ec96a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a1bb6f0 0x55db6a197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41478/0 (socket says 172.21.15.40:41478) 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ec96a5640 1 -- 172.21.15.40:0/3339957483 learned_addr learned my addr 172.21.15.40:0/3339957483 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.528+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a1d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 0x55db6a1d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ec96a5640 1 -- 172.21.15.40:0/3339957483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 msgr2=0x55db6a1d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 0x55db6a1d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ec96a5640 1 -- 172.21.15.40:0/3339957483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 msgr2=0x55db6a1d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a1d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ec96a5640 1 -- 172.21.15.40:0/3339957483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db69eb57d0 con 0x55db6a1bb6f0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a1bb6f0 0x55db6a197440 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f4eb402a220 tx=0x7f4eb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f013bca7ff4248f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ea4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3339957483 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4037040 con 0x55db6a1bb6f0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3339957483 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f4eb4032df0 con 0x55db6a1bb6f0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ecee28740 1 -- 172.21.15.40:0/3339957483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec0024070 con 0x55db6a1bb6f0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3339957483 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f4eb403c410 con 0x55db6a1bb6f0 2024-04-26T21:41:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ecee28740 10 client.7533 fetch_fsmap learned FSMap version 78 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ecee28740 1 -- 172.21.15.40:0/3339957483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55db69deb3f0 con 0x55db6a1bb6f0 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.529+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3339957483 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0024070 con 0x55db6a1bb6f0 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3339957483 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f4eb403c5b0 con 0x55db6a1bb6f0 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ecee28740 10 client.7533 fetch_fsmap finished waiting for FSMap version 78 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ecee28740 10 client.7533 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ecee28740 20 client.7533 populate_metadata read hostname 'smithi040' 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x55db6a1a2060 0x55db6a1a2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ecee28740 4 client.7533 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ecee28740 1 -- 172.21.15.40:0/3339957483 --> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db6a0e7990 con 0x55db6a1a2060 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x55db6a1a2060 0x55db6a1a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ea4ff9640 10 client.7533 ms_handle_connect on v2:172.21.15.40:6832/836437049 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.530+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x55db6a1a2060 0x55db6a1a2440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9700161cb4fed1 server_cookie=d17215696bf29453 in_seq=0 out_seq=0 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3339957483 <== mds.0 v2:172.21.15.40:6832/836437049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db6a0e7990 con 0x55db6a1a2060 2024-04-26T21:41:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ea4ff9640 10 client.7533 handle_command_reply: tid=0 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 client.7533 shutdown 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 -- 172.21.15.40:0/3339957483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a1bb6f0 msgr2=0x55db6a197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a1bb6f0 0x55db6a197440 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f4eb402a220 tx=0x7f4eb4004730 comp rx=0 tx=0).stop 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 -- 172.21.15.40:0/3339957483 shutdown_connections 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x55db6a1a2060 0x55db6a1a2440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 0x55db6a1d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a1bb6f0 0x55db6a197440 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ea4ff9640 0 client.7533 ms_handle_reset on v2:172.21.15.40:6832/836437049 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3339957483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a1d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 -- 172.21.15.40:0/3339957483 >> 172.21.15.40:0/3339957483 conn(0x55db6a170350 msgr2=0x55db6a16ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 -- 172.21.15.40:0/3339957483 shutdown_connections 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 1 -- 172.21.15.40:0/3339957483 wait complete. 2024-04-26T21:41:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.531+0000 7f4ecee28740 20 client.7533 trim_cache size 0 max 16384 2024-04-26T21:41:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.546+0000 7f4ecee28740 1 Processor -- start 2024-04-26T21:41:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.546+0000 7f4ecee28740 1 -- start start 2024-04-26T21:41:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19cf30 0x55db6a22f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a16b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ecee28740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41fda70 con 0x55db6a19cf30 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ecee28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41682f0 con 0x55db6a19fd30 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ecee28740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec414be40 con 0x55db6a1bb6f0 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19cf30 0x55db6a22f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19cf30 0x55db6a22f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41482/0 (socket says 172.21.15.40:41482) 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 -- 172.21.15.40:0/3754378384 learned_addr learned my addr 172.21.15.40:0/3754378384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 -- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 msgr2=0x55db6a1a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a16b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a1a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 -- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 msgr2=0x55db6a16b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a16b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 -- 172.21.15.40:0/3754378384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db69dab0f0 con 0x55db6a19cf30 2024-04-26T21:41:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a16b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19cf30 0x55db6a22f760 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000eb40 tx=0x7f4ec0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f717409c52d5cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.547+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3754378384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0015c80 con 0x55db6a19cf30 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3754378384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec003c070 con 0x55db6a19cf30 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ea6ffd640 1 -- 172.21.15.40:0/3754378384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00365c0 con 0x55db6a19cf30 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 -- 172.21.15.40:0/3754378384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19cf30 msgr2=0x55db6a22f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19cf30 0x55db6a22f760 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000eb40 tx=0x7f4ec0013000 comp rx=0 tx=0).stop 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 -- 172.21.15.40:0/3754378384 shutdown_connections 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a1bb6f0 0x55db6a1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a19fd30 0x55db6a16b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3754378384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19cf30 0x55db6a22f760 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 -- 172.21.15.40:0/3754378384 >> 172.21.15.40:0/3754378384 conn(0x55db6a170350 msgr2=0x55db6a176e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 -- 172.21.15.40:0/3754378384 shutdown_connections 2024-04-26T21:41:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.548+0000 7f4ecee28740 1 -- 172.21.15.40:0/3754378384 wait complete. 2024-04-26T21:41:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.549+0000 7f4ecee28740 1 -- start start 2024-04-26T21:41:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.549+0000 7f4ecee28740 1 Processor -- start 2024-04-26T21:41:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 0x55db6a22c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ecee28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a1bb6f0 0x55db6a1d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ecee28740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec414c490 con 0x55db6a19fd30 2024-04-26T21:41:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ecee28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55db6a19c790 con 0x55db6a1bb6f0 2024-04-26T21:41:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ecee28740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55db6a16edc0 con 0x55db6a19cf30 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ecee28740 10 client.0 _ll_register_callbacks cb 0x7f4eb40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41488/0 (socket says 172.21.15.40:41488) 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 -- 172.21.15.40:0/3091400644 learned_addr learned my addr 172.21.15.40:0/3091400644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 0x55db6a22c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a1bb6f0 0x55db6a1d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 -- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 msgr2=0x55db6a22c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 0x55db6a22c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 -- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a1bb6f0 msgr2=0x55db6a1d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a1bb6f0 0x55db6a1d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 -- 172.21.15.40:0/3091400644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db6a09a4b0 con 0x55db6a19fd30 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4eca6a7640 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a1bb6f0 0x55db6a1d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ec96a5640 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1d12a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x55db6a19d570 tx=0x7f4eb4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5865a67591919fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ea4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3091400644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4037040 con 0x55db6a19fd30 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ecee28740 1 -- 172.21.15.40:0/3091400644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec003c070 con 0x55db6a19fd30 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.550+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3091400644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f4eb4004230 con 0x55db6a19fd30 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3091400644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f4eb405e050 con 0x55db6a19fd30 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ecee28740 10 client.7539 fetch_fsmap learned FSMap version 78 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ecee28740 1 -- 172.21.15.40:0/3091400644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55db6a196bb0 con 0x55db6a19fd30 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3091400644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec003c070 con 0x55db6a19fd30 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3091400644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f4eb4061020 con 0x55db6a19fd30 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ecee28740 10 client.7539 fetch_fsmap finished waiting for FSMap version 78 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ecee28740 10 client.7539 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ecee28740 20 client.7539 populate_metadata read hostname 'smithi040' 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x55db6a16b8a0 0x55db6a22cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ecee28740 4 client.7539 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T21:41:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ecee28740 1 -- 172.21.15.40:0/3091400644 --> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db6a0e1800 con 0x55db6a16b8a0 2024-04-26T21:41:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x55db6a16b8a0 0x55db6a22cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ea4ff9640 10 client.7539 ms_handle_connect on v2:172.21.15.40:6832/836437049 2024-04-26T21:41:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.551+0000 7f4ec9ea6640 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x55db6a16b8a0 0x55db6a22cc90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e419625c6caf3eb3 server_cookie=d05a04a6ca6a337b in_seq=0 out_seq=0 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ea4ff9640 1 -- 172.21.15.40:0/3091400644 <== mds.0 v2:172.21.15.40:6832/836437049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db6a0e1800 con 0x55db6a16b8a0 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ea4ff9640 10 client.7539 handle_command_reply: tid=0 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 client.7539 shutdown 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 -- 172.21.15.40:0/3091400644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 msgr2=0x55db6a1d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1d12a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x55db6a19d570 tx=0x7f4eb4009240 comp rx=0 tx=0).stop 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 -- 172.21.15.40:0/3091400644 shutdown_connections 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.40:6832/836437049,v1:172.21.15.40:6833/836437049] conn(0x55db6a16b8a0 0x55db6a22cc90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6a1bb6f0 0x55db6a1d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db6a19fd30 0x55db6a1d12a0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 --2- 172.21.15.40:0/3091400644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55db6a19cf30 0x55db6a22c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ea4ff9640 0 client.7539 ms_handle_reset on v2:172.21.15.40:6832/836437049 2024-04-26T21:41:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.563+0000 7f4ecee28740 1 -- 172.21.15.40:0/3091400644 >> 172.21.15.40:0/3091400644 conn(0x55db6a170350 msgr2=0x55db6a197500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecee28740 1 -- 172.21.15.40:0/3091400644 shutdown_connections 2024-04-26T21:41:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecee28740 1 -- 172.21.15.40:0/3091400644 wait complete. 2024-04-26T21:41:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecee28740 20 client.7539 trim_cache size 0 max 16384 2024-04-26T21:41:44.565 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:41:44.565 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-04-26T21:41:44.565 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0 2024-04-26T21:41:44.565 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:41:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 -- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4ea806c920 msgr2=0x7f4ea806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4ea806c920 0x7f4ea806ede0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098e0 tx=0x7f4ec0009060 comp rx=0 tx=0).stop 2024-04-26T21:41:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 -- 172.21.15.40:0/709114284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4156460 msgr2=0x7f4ec40c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4156460 0x7f4ec40c98b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00cef0 tx=0x7f4ebc00a710 comp rx=0 tx=0).stop 2024-04-26T21:41:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 -- 172.21.15.40:0/709114284 shutdown_connections 2024-04-26T21:41:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4ea806c920 0x7f4ea806ede0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4156460 0x7f4ec40c98b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec40cba10 0x7f4ec41492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 --2- 172.21.15.40:0/709114284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec406fcd0 0x7f4ec40c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 -- 172.21.15.40:0/709114284 >> 172.21.15.40:0/709114284 conn(0x7f4ec40d0be0 msgr2=0x7f4ec40b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 -- 172.21.15.40:0/709114284 shutdown_connections 2024-04-26T21:41:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.564+0000 7f4ecc10a640 1 -- 172.21.15.40:0/709114284 wait complete. 2024-04-26T21:41:44.575 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:41:44.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:41:44.621 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:41:44.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:41:44.657 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:41:44.657 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:41:44.657 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:44.673 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:41:44.673 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:44.729 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:44.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.870+0000 7fa7ce4c8640 1 Processor -- start 2024-04-26T21:41:44.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.870+0000 7fa7ce4c8640 1 -- start start 2024-04-26T21:41:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7ce4c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa7c80b3860 0x7fa7c816a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7ce4c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c80d0ba0 con 0x7fa7c815f3a0 2024-04-26T21:41:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7ce4c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c80d2590 con 0x7fa7c80d06f0 2024-04-26T21:41:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7ce4c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c80d0e20 con 0x7fa7c80b3860 2024-04-26T21:41:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7c7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa7c815f3a0 0x7fa7c80d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44568/0 (socket says 172.21.15.40:44568) 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7c7fff640 1 -- 172.21.15.40:0/3539527246 learned_addr learned my addr 172.21.15.40:0/3539527246 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2427129810 0 0) 0x7fa7b8001110 con 0x7fa7c815f3a0 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7b4003520 con 0x7fa7c815f3a0 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.871+0000 7fa7cca65640 1 --2- 172.21.15.40:0/3539527246 >> v2:172.21.15.184:3300/0 conn(0x7fa7c80b3860 0x7fa7c816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1267702570 0 0) 0x7fa7b4003520 con 0x7fa7c815f3a0 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 >> v2:172.21.15.184:3300/0 conn(0x7fa7c80b3860 msgr2=0x7fa7c816a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 --2- 172.21.15.40:0/3539527246 >> v2:172.21.15.184:3300/0 conn(0x7fa7c80b3860 0x7fa7c816a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 >> v1:172.21.15.184:6789/0 conn(0x7fa7c80d06f0 legacy=0x7fa7c80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c80b1d70 con 0x7fa7c815f3a0 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fa7b8001840 con 0x7fa7c815f3a0 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 >> v1:172.21.15.40:6789/0 conn(0x7fa7c815f3a0 legacy=0x7fa7c80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40039b0 0x7fa7b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4004800 0x7fa7b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40052e0 0x7fa7b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c80d0e20 con 0x7fa7b40039b0 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b400c1b0 con 0x7fa7b4004800 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b400c3b0 con 0x7fa7b40052e0 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c7fff640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40039b0 0x7fa7b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7cca65640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40052e0 0x7fa7b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c77fe640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4004800 0x7fa7b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c7fff640 1 -- 172.21.15.40:0/3539527246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40052e0 msgr2=0x7fa7b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c7fff640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40052e0 0x7fa7b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c7fff640 1 -- 172.21.15.40:0/3539527246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4004800 msgr2=0x7fa7b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c7fff640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b4004800 0x7fa7b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7c7fff640 1 -- 172.21.15.40:0/3539527246 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b4003520 con 0x7fa7b40039b0 2024-04-26T21:41:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.872+0000 7fa7cca65640 1 -- 172.21.15.40:0/3539527246 reap_dead start 2024-04-26T21:41:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.873+0000 7fa7c7fff640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40039b0 0x7fa7b4003dd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800c490 tx=0x7fa7b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe8d6a45f25a207 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.873+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800c0e0 con 0x7fa7b40039b0 2024-04-26T21:41:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.873+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b8034070 con 0x7fa7b40039b0 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.873+0000 7fa7c6ffd640 1 -- 172.21.15.40:0/3539527246 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8005d70 con 0x7fa7b40039b0 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.874+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3539527246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40039b0 msgr2=0x7fa7b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.874+0000 7fa7ce4c8640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40039b0 0x7fa7b4003dd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800c490 tx=0x7fa7b4005830 comp rx=0 tx=0).stop 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.874+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3539527246 shutdown_connections 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.874+0000 7fa7ce4c8640 1 --2- 172.21.15.40:0/3539527246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40039b0 0x7fa7b4003dd0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.874+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3539527246 >> 172.21.15.40:0/3539527246 conn(0x7fa7c80d4f90 msgr2=0x7fa7c80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.874+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3539527246 shutdown_connections 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.874+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3539527246 wait complete. 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.875+0000 7fa7ce4c8640 1 Processor -- start 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.875+0000 7fa7ce4c8640 1 -- start start 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7ce4c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40039b0 0x7fa7c81f9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7ce4c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c8152a40 0x7fa7c81520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7ce4c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c80cacf0 0x7fa7c80b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7ce4c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c815f780 con 0x7fa7c80cacf0 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7ce4c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c80b2330 con 0x7fa7c8152a40 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7ce4c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c815f900 con 0x7fa7b40039b0 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c80cacf0 0x7fa7c80b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c80cacf0 0x7fa7c80b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41496/0 (socket says 172.21.15.40:41496) 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 -- 172.21.15.40:0/3860850910 learned_addr learned my addr 172.21.15.40:0/3860850910 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7c7fff640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40039b0 0x7fa7c81f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7c77fe640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c8152a40 0x7fa7c81520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 -- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40039b0 msgr2=0x7fa7c81f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40039b0 0x7fa7c81f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 -- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c8152a40 msgr2=0x7fa7c81520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c8152a40 0x7fa7c81520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 -- 172.21.15.40:0/3860850910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c81f9cb0 con 0x7fa7c80cacf0 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7c77fe640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c8152a40 0x7fa7c81520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7cca65640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c80cacf0 0x7fa7c80b7b80 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa7c808e8d0 tx=0x7fa7c0007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7c4ff9640 1 -- 172.21.15.40:0/3860850910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0007c80 con 0x7fa7c80cacf0 2024-04-26T21:41:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7c4ff9640 1 -- 172.21.15.40:0/3860850910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c001a070 con 0x7fa7c80cacf0 2024-04-26T21:41:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c80c9c50 con 0x7fa7c80cacf0 2024-04-26T21:41:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7c4ff9640 1 -- 172.21.15.40:0/3860850910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0015660 con 0x7fa7c80cacf0 2024-04-26T21:41:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.876+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c80ca210 con 0x7fa7c80cacf0 2024-04-26T21:41:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.877+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c80694f0 con 0x7fa7c80cacf0 2024-04-26T21:41:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.878+0000 7fa7c4ff9640 1 -- 172.21.15.40:0/3860850910 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7c0015800 con 0x7fa7c80cacf0 2024-04-26T21:41:44.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.878+0000 7fa7c4ff9640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa7a806c8f0 0x7fa7a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:44.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.878+0000 7fa7c4ff9640 1 -- 172.21.15.40:0/3860850910 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7fa7c008c1f0 con 0x7fa7c80cacf0 2024-04-26T21:41:44.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.879+0000 7fa7c7fff640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa7a806c8f0 0x7fa7a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:44.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.879+0000 7fa7c7fff640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa7a806c8f0 0x7fa7a806edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4004780 tx=0x7fa7b802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:44.883+0000 7fa7c4ff9640 1 -- 172.21.15.40:0/3860850910 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa7c81f9cb0 con 0x7fa7c80cacf0 2024-04-26T21:41:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.068+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7c80b6860 con 0x7fa7c80cacf0 2024-04-26T21:41:45.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.069+0000 7fa7c4ff9640 1 -- 172.21.15.40:0/3860850910 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+17038 (secure 0 0 0) 0x7fa7c005a4d0 con 0x7fa7c80cacf0 2024-04-26T21:41:45.069 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:45.069 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:39.656336+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:41:05.508768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T21:41:06.198815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3115383225":"2024-04-27T21:41:39.656262+0000","172.21.15.40:6834/3115383225":"2024-04-27T21:41:39.656262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa7a806c8f0 msgr2=0x7fa7a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa7a806c8f0 0x7fa7a806edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4004780 tx=0x7fa7b802e000 comp rx=0 tx=0).stop 2024-04-26T21:41:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c80cacf0 msgr2=0x7fa7c80b7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c80cacf0 0x7fa7c80b7b80 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa7c808e8d0 tx=0x7fa7c0007700 comp rx=0 tx=0).stop 2024-04-26T21:41:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 shutdown_connections 2024-04-26T21:41:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa7a806c8f0 0x7fa7a806edb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c80cacf0 0x7fa7c80b7b80 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c8152a40 0x7fa7c81520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 --2- 172.21.15.40:0/3860850910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40039b0 0x7fa7c81f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 >> 172.21.15.40:0/3860850910 conn(0x7fa7c80d4f90 msgr2=0x7fa7c806dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 shutdown_connections 2024-04-26T21:41:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.071+0000 7fa7ce4c8640 1 -- 172.21.15.40:0/3860850910 wait complete. 2024-04-26T21:41:45.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:41:05.508768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:41:06.198815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:41:45.082 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:41:45.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.186+0000 7f555e425640 1 Processor -- start 2024-04-26T21:41:45.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.186+0000 7f555e425640 1 -- start start 2024-04-26T21:41:45.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f555e425640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f555815f550 0x7f55580d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f555e425640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55580d1040 con 0x7f555815f170 2024-04-26T21:41:45.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f555e425640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55580c9680 con 0x7f55580d3bc0 2024-04-26T21:41:45.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f555e425640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f55580c9430 con 0x7f555815f550 2024-04-26T21:41:45.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f55577fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f555815f550 0x7f55580d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f5557fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f555815f170 0x7f55580d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44572/0 (socket says 172.21.15.40:44572) 2024-04-26T21:41:45.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f55577fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f555815f550 0x7f55580d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41508/0 (socket says 172.21.15.40:41508) 2024-04-26T21:41:45.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f5557fff640 1 -- 172.21.15.40:0/555103533 learned_addr learned my addr 172.21.15.40:0/555103533 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:45.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f55577fe640 1 -- 172.21.15.40:0/555103533 >> v1:172.21.15.184:6789/0 conn(0x7f55580d3bc0 legacy=0x7f55580ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f55577fe640 1 -- 172.21.15.40:0/555103533 >> v1:172.21.15.40:6789/0 conn(0x7f555815f170 legacy=0x7f55580d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f55577fe640 1 -- 172.21.15.40:0/555103533 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55580c9960 con 0x7f555815f550 2024-04-26T21:41:45.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.187+0000 7f5556ffd640 1 -- 172.21.15.40:0/555103533 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1285708436 0 0) 0x7f554c001110 con 0x7f555815f170 2024-04-26T21:41:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.188+0000 7f55577fe640 1 --2- 172.21.15.40:0/555103533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f550 0x7f55580d5a80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f55580c95b0 tx=0x7f554802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbf4fd875299fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.188+0000 7f5556ffd640 1 -- 172.21.15.40:0/555103533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548037070 con 0x7f555815f550 2024-04-26T21:41:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.188+0000 7f5556ffd640 1 -- 172.21.15.40:0/555103533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5548032440 con 0x7f555815f550 2024-04-26T21:41:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.188+0000 7f5556ffd640 1 -- 172.21.15.40:0/555103533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548032810 con 0x7f555815f550 2024-04-26T21:41:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.189+0000 7f555e425640 1 -- 172.21.15.40:0/555103533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f550 msgr2=0x7f55580d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.189+0000 7f555e425640 1 --2- 172.21.15.40:0/555103533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f550 0x7f55580d5a80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f55580c95b0 tx=0x7f554802a250 comp rx=0 tx=0).stop 2024-04-26T21:41:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.189+0000 7f555e425640 1 -- 172.21.15.40:0/555103533 shutdown_connections 2024-04-26T21:41:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.189+0000 7f555e425640 1 --2- 172.21.15.40:0/555103533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f550 0x7f55580d5a80 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.189+0000 7f555e425640 1 -- 172.21.15.40:0/555103533 >> 172.21.15.40:0/555103533 conn(0x7f55580d4f60 msgr2=0x7f55580d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:45.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.189+0000 7f555e425640 1 -- 172.21.15.40:0/555103533 shutdown_connections 2024-04-26T21:41:45.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.189+0000 7f555e425640 1 -- 172.21.15.40:0/555103533 wait complete. 2024-04-26T21:41:45.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.190+0000 7f555e425640 1 Processor -- start 2024-04-26T21:41:45.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.190+0000 7f555e425640 1 -- start start 2024-04-26T21:41:45.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f555e425640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55580d3bc0 0x7f55581f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f555e425640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f170 0x7f5558158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f555e425640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5558159470 0x7f555815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f555e425640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55580d2e30 con 0x7f555815f170 2024-04-26T21:41:45.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f555e425640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55580d2cb0 con 0x7f5558159470 2024-04-26T21:41:45.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f555e425640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55580d2fb0 con 0x7f55580d3bc0 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f55577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f170 0x7f5558158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f55577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f170 0x7f5558158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41522/0 (socket says 172.21.15.40:41522) 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f55577fe640 1 -- 172.21.15.40:0/3626263315 learned_addr learned my addr 172.21.15.40:0/3626263315 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f5557fff640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55580d3bc0 0x7f55581f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f555c9c2640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5558159470 0x7f555815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f55577fe640 1 -- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55580d3bc0 msgr2=0x7f55581f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f55577fe640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55580d3bc0 0x7f55581f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f55577fe640 1 -- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5558159470 msgr2=0x7f555815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f55577fe640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5558159470 0x7f555815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f55577fe640 1 -- 172.21.15.40:0/3626263315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5558159aa0 con 0x7f555815f170 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f555c9c2640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5558159470 0x7f555815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.191+0000 7f5557fff640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55580d3bc0 0x7f55581f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.192+0000 7f55577fe640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f170 0x7f5558158f30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f55480098c0 tx=0x7f55480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.192+0000 7f5554ff9640 1 -- 172.21.15.40:0/3626263315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548037040 con 0x7f555815f170 2024-04-26T21:41:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.192+0000 7f5554ff9640 1 -- 172.21.15.40:0/3626263315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5548042070 con 0x7f555815f170 2024-04-26T21:41:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.192+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5558159d80 con 0x7f555815f170 2024-04-26T21:41:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.192+0000 7f5554ff9640 1 -- 172.21.15.40:0/3626263315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548030a70 con 0x7f555815f170 2024-04-26T21:41:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.192+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555815a340 con 0x7f555815f170 2024-04-26T21:41:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.193+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55580694f0 con 0x7f555815f170 2024-04-26T21:41:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.193+0000 7f5554ff9640 1 -- 172.21.15.40:0/3626263315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5548030c10 con 0x7f555815f170 2024-04-26T21:41:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.194+0000 7f5554ff9640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554406c920 0x7f554406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.194+0000 7f5554ff9640 1 -- 172.21.15.40:0/3626263315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f55480ac700 con 0x7f555815f170 2024-04-26T21:41:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.194+0000 7f5557fff640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554406c920 0x7f554406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.194+0000 7f5557fff640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554406c920 0x7f554406ede0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f554c0013b0 tx=0x7f554c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.198+0000 7f5554ff9640 1 -- 172.21.15.40:0/3626263315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5548045360 con 0x7f555815f170 2024-04-26T21:41:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.381+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f55580c8d40 con 0x7f555815f170 2024-04-26T21:41:45.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.382+0000 7f5554ff9640 1 -- 172.21.15.40:0/3626263315 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f554807a9e0 con 0x7f555815f170 2024-04-26T21:41:45.382 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554406c920 msgr2=0x7f554406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554406c920 0x7f554406ede0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f554c0013b0 tx=0x7f554c009040 comp rx=0 tx=0).stop 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f170 msgr2=0x7f5558158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f170 0x7f5558158f30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f55480098c0 tx=0x7f55480046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 shutdown_connections 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f554406c920 0x7f554406ede0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5558159470 0x7f555815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555815f170 0x7f5558158f30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 --2- 172.21.15.40:0/3626263315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55580d3bc0 0x7f55581f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 >> 172.21.15.40:0/3626263315 conn(0x7f55580d4f60 msgr2=0x7f55580d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 shutdown_connections 2024-04-26T21:41:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.384+0000 7f555e425640 1 -- 172.21.15.40:0/3626263315 wait complete. 2024-04-26T21:41:45.395 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:41:45.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.498+0000 7f2d0eb24640 1 Processor -- start 2024-04-26T21:41:45.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.498+0000 7f2d0eb24640 1 -- start start 2024-04-26T21:41:45.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.499+0000 7f2d0eb24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 0x7f2d080d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.499+0000 7f2d0eb24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d080d3bc0 0x7f2d080d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.499+0000 7f2d0eb24640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d080d1040 con 0x7f2d0815f550 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.499+0000 7f2d0eb24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2d080c9430 con 0x7f2d080d3bc0 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.499+0000 7f2d0eb24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d080c9680 con 0x7f2d0815f170 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.499+0000 7f2d0d0c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d080d3bc0 0x7f2d080d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.499+0000 7f2d07fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2d0815f550 0x7f2d080d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44582/0 (socket says 172.21.15.40:44582) 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.499+0000 7f2d07fff640 1 -- 172.21.15.40:0/774208165 learned_addr learned my addr 172.21.15.40:0/774208165 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d0c8c0640 1 --2- 172.21.15.40:0/774208165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 0x7f2d080d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d0d0c1640 1 -- 172.21.15.40:0/774208165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 msgr2=0x7f2d080d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d0d0c1640 1 --2- 172.21.15.40:0/774208165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 0x7f2d080d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d0d0c1640 1 -- 172.21.15.40:0/774208165 >> v1:172.21.15.40:6789/0 conn(0x7f2d0815f550 legacy=0x7f2d080d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d0d0c1640 1 -- 172.21.15.40:0/774208165 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d080c9960 con 0x7f2d080d3bc0 2024-04-26T21:41:45.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d0d0c1640 1 --2- 172.21.15.40:0/774208165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d080d2cb0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009d90 tx=0x7f2d0002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae2b76fe96805aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d077fe640 1 -- 172.21.15.40:0/774208165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00007740 con 0x7f2d080d3bc0 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d077fe640 1 -- 172.21.15.40:0/774208165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d000078e0 con 0x7f2d080d3bc0 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.500+0000 7f2d077fe640 1 -- 172.21.15.40:0/774208165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00007cb0 con 0x7f2d080d3bc0 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.501+0000 7f2d0eb24640 1 -- 172.21.15.40:0/774208165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 msgr2=0x7f2d080d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.501+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/774208165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d080d2cb0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009d90 tx=0x7f2d0002a660 comp rx=0 tx=0).stop 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.501+0000 7f2d0eb24640 1 -- 172.21.15.40:0/774208165 shutdown_connections 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.501+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/774208165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d080d2cb0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.501+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/774208165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 0x7f2d080d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.501+0000 7f2d0eb24640 1 -- 172.21.15.40:0/774208165 >> 172.21.15.40:0/774208165 conn(0x7f2d080d4f60 msgr2=0x7f2d080d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.501+0000 7f2d0eb24640 1 -- 172.21.15.40:0/774208165 shutdown_connections 2024-04-26T21:41:45.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.501+0000 7f2d0eb24640 1 -- 172.21.15.40:0/774208165 wait complete. 2024-04-26T21:41:45.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.502+0000 7f2d0eb24640 1 Processor -- start 2024-04-26T21:41:45.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.502+0000 7f2d0eb24640 1 -- start start 2024-04-26T21:41:45.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0eb24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d0815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0eb24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 0x7f2d081f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0eb24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d081f8510 0x7f2d081f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0eb24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080ca5c0 con 0x7f2d080d3bc0 2024-04-26T21:41:45.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0eb24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080ca440 con 0x7f2d081f8510 2024-04-26T21:41:45.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0eb24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d080ca740 con 0x7f2d0815f170 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0c8c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d0815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0c8c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d0815ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41536/0 (socket says 172.21.15.40:41536) 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0c8c0640 1 -- 172.21.15.40:0/344461852 learned_addr learned my addr 172.21.15.40:0/344461852 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d07fff640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 0x7f2d081f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.503+0000 7f2d0d0c1640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d081f8510 0x7f2d081f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d0c8c0640 1 -- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 msgr2=0x7f2d081f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d0c8c0640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 0x7f2d081f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d0c8c0640 1 -- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d081f8510 msgr2=0x7f2d081f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d0c8c0640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d081f8510 0x7f2d081f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d0c8c0640 1 -- 172.21.15.40:0/344461852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d080ceed0 con 0x7f2d080d3bc0 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d0c8c0640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d0815ca30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c7f0 tx=0x7f2cf800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d057fa640 1 -- 172.21.15.40:0/344461852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8015070 con 0x7f2d080d3bc0 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d057fa640 1 -- 172.21.15.40:0/344461852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf8004240 con 0x7f2d080d3bc0 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d080ce910 con 0x7f2d080d3bc0 2024-04-26T21:41:45.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d057fa640 1 -- 172.21.15.40:0/344461852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8007620 con 0x7f2d080d3bc0 2024-04-26T21:41:45.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.504+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0809fcf0 con 0x7f2d080d3bc0 2024-04-26T21:41:45.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.505+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d080694f0 con 0x7f2d080d3bc0 2024-04-26T21:41:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.505+0000 7f2d057fa640 1 -- 172.21.15.40:0/344461852 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cf80077c0 con 0x7f2d080d3bc0 2024-04-26T21:41:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.506+0000 7f2d057fa640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cf406c8f0 0x7f2cf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.506+0000 7f2d057fa640 1 -- 172.21.15.40:0/344461852 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f2cf808bcd0 con 0x7f2d080d3bc0 2024-04-26T21:41:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.506+0000 7f2d07fff640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cf406c8f0 0x7f2cf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.507+0000 7f2d07fff640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cf406c8f0 0x7f2cf406edb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2d080a2ea0 tx=0x7f2cfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.510+0000 7f2d057fa640 1 -- 172.21.15.40:0/344461852 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cf8059e80 con 0x7f2d080d3bc0 2024-04-26T21:41:45.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.698+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2d0815cf70 con 0x7f2d080d3bc0 2024-04-26T21:41:45.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.698+0000 7f2d057fa640 1 -- 172.21.15.40:0/344461852 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f2cf8059780 con 0x7f2d080d3bc0 2024-04-26T21:41:45.698 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:41:45.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cf406c8f0 msgr2=0x7f2cf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cf406c8f0 0x7f2cf406edb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2d080a2ea0 tx=0x7f2cfc009040 comp rx=0 tx=0).stop 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 msgr2=0x7f2d0815ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d0815ca30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c7f0 tx=0x7f2cf800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 shutdown_connections 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cf406c8f0 0x7f2cf406edb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d081f8510 0x7f2d081f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0815f170 0x7f2d081f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 --2- 172.21.15.40:0/344461852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d080d3bc0 0x7f2d0815ca30 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.700+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 >> 172.21.15.40:0/344461852 conn(0x7f2d080d4f60 msgr2=0x7f2d0806dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.701+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 shutdown_connections 2024-04-26T21:41:45.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.701+0000 7f2d0eb24640 1 -- 172.21.15.40:0/344461852 wait complete. 2024-04-26T21:41:45.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:41:45.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.815+0000 7fb8491f0640 1 Processor -- start 2024-04-26T21:41:45.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.815+0000 7fb8491f0640 1 -- start start 2024-04-26T21:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.816+0000 7fb8491f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440b4360 0x7fb8440b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.816+0000 7fb8491f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8440d0d60 0x7fb8440b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.816+0000 7fb8491f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8440ce5e0 con 0x7fb8440d1240 2024-04-26T21:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.816+0000 7fb8491f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8440cf210 con 0x7fb8440d0d60 2024-04-26T21:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.816+0000 7fb8491f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8440ce8c0 con 0x7fb8440b4360 2024-04-26T21:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84259c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440b4360 0x7fb8440b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8440d0d60 0x7fb8440b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84259c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440b4360 0x7fb8440b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:36036/0 (socket says 172.21.15.40:36036) 2024-04-26T21:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84259c640 1 -- 172.21.15.40:0/3169453857 learned_addr learned my addr 172.21.15.40:0/3169453857 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb842d9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb8440d1240 0x7fb8440d1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:50344/0 (socket says 172.21.15.40:50344) 2024-04-26T21:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb841d9b640 1 -- 172.21.15.40:0/3169453857 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2977224722 0 0) 0x7fb8440ce5e0 con 0x7fb8440d1240 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb841d9b640 1 -- 172.21.15.40:0/3169453857 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb830003520 con 0x7fb8440d1240 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84259c640 1 -- 172.21.15.40:0/3169453857 >> v2:172.21.15.184:3300/0 conn(0x7fb8440d0d60 msgr2=0x7fb8440b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84259c640 1 --2- 172.21.15.40:0/3169453857 >> v2:172.21.15.184:3300/0 conn(0x7fb8440d0d60 0x7fb8440b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84259c640 1 -- 172.21.15.40:0/3169453857 >> v1:172.21.15.184:6789/0 conn(0x7fb8440d1240 legacy=0x7fb8440d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84259c640 1 -- 172.21.15.40:0/3169453857 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb844160ab0 con 0x7fb8440b4360 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.817+0000 7fb84359e640 1 --2- 172.21.15.40:0/3169453857 >> v2:172.21.15.184:3300/0 conn(0x7fb8440d0d60 0x7fb8440b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.818+0000 7fb84259c640 1 --2- 172.21.15.40:0/3169453857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440b4360 0x7fb8440b2600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009870 tx=0x7fb82c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea98d1e003960d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.818+0000 7fb841d9b640 1 -- 172.21.15.40:0/3169453857 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c037070 con 0x7fb8440b4360 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.818+0000 7fb841d9b640 1 -- 172.21.15.40:0/3169453857 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82c032440 con 0x7fb8440b4360 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.818+0000 7fb841d9b640 1 -- 172.21.15.40:0/3169453857 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c032810 con 0x7fb8440b4360 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.819+0000 7fb8491f0640 1 -- 172.21.15.40:0/3169453857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440b4360 msgr2=0x7fb8440b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.819+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3169453857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440b4360 0x7fb8440b2600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009870 tx=0x7fb82c02a270 comp rx=0 tx=0).stop 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.819+0000 7fb8491f0640 1 -- 172.21.15.40:0/3169453857 shutdown_connections 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.819+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3169453857 >> v2:172.21.15.184:3300/0 conn(0x7fb8440d0d60 0x7fb8440b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.819+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3169453857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440b4360 0x7fb8440b2600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.819+0000 7fb8491f0640 1 -- 172.21.15.40:0/3169453857 >> 172.21.15.40:0/3169453857 conn(0x7fb8440d2e60 msgr2=0x7fb8440d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.819+0000 7fb8491f0640 1 -- 172.21.15.40:0/3169453857 shutdown_connections 2024-04-26T21:41:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.819+0000 7fb8491f0640 1 -- 172.21.15.40:0/3169453857 wait complete. 2024-04-26T21:41:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.820+0000 7fb8491f0640 1 Processor -- start 2024-04-26T21:41:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.820+0000 7fb8491f0640 1 -- start start 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb8491f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8440d0d60 0x7fb8441521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb8491f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440d1240 0x7fb8440cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb8491f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8440cb640 0x7fb8441f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb8491f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8440c6790 con 0x7fb8440cb640 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb8491f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8440c6610 con 0x7fb8440d0d60 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb8491f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8440b3080 con 0x7fb8440d1240 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8440cb640 0x7fb8441f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8440cb640 0x7fb8441f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41546/0 (socket says 172.21.15.40:41546) 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84359e640 1 -- 172.21.15.40:0/3722688464 learned_addr learned my addr 172.21.15.40:0/3722688464 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb842d9d640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8440d0d60 0x7fb8441521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84259c640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440d1240 0x7fb8440cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84359e640 1 -- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440d1240 msgr2=0x7fb8440cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84359e640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440d1240 0x7fb8440cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84359e640 1 -- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8440d0d60 msgr2=0x7fb8441521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84359e640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8440d0d60 0x7fb8441521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.821+0000 7fb84359e640 1 -- 172.21.15.40:0/3722688464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8440ca060 con 0x7fb8440cb640 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.822+0000 7fb84359e640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8440cb640 0x7fb8441f3b70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00efc0 tx=0x7fb83c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.822+0000 7fb83b7fe640 1 -- 172.21.15.40:0/3722688464 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c018030 con 0x7fb8440cb640 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.822+0000 7fb83b7fe640 1 -- 172.21.15.40:0/3722688464 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83c00cde0 con 0x7fb8440cb640 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.822+0000 7fb83b7fe640 1 -- 172.21.15.40:0/3722688464 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c003bb0 con 0x7fb8440cb640 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.822+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8440ca340 con 0x7fb8440cb640 2024-04-26T21:41:45.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.822+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8440ca900 con 0x7fb8440cb640 2024-04-26T21:41:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.823+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8440694f0 con 0x7fb8440cb640 2024-04-26T21:41:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.823+0000 7fb83b7fe640 1 -- 172.21.15.40:0/3722688464 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb83c01b070 con 0x7fb8440cb640 2024-04-26T21:41:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.824+0000 7fb83b7fe640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb82406c920 0x7fb82406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.824+0000 7fb83b7fe640 1 -- 172.21.15.40:0/3722688464 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7fb83c08c5e0 con 0x7fb8440cb640 2024-04-26T21:41:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.824+0000 7fb842d9d640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb82406c920 0x7fb82406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:45.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.825+0000 7fb842d9d640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb82406c920 0x7fb82406ede0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb834000f60 tx=0x7fb834007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:45.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:45.828+0000 7fb83b7fe640 1 -- 172.21.15.40:0/3722688464 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb83c010040 con 0x7fb8440cb640 2024-04-26T21:41:46.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.013+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb8440ce5b0 con 0x7fb8440cb640 2024-04-26T21:41:46.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.013+0000 7fb83b7fe640 1 -- 172.21.15.40:0/3722688464 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7fb83c05a270 con 0x7fb8440cb640 2024-04-26T21:41:46.013 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb82406c920 msgr2=0x7fb82406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb82406c920 0x7fb82406ede0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb834000f60 tx=0x7fb834007cf0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8440cb640 msgr2=0x7fb8441f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8440cb640 0x7fb8441f3b70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00efc0 tx=0x7fb83c009f90 comp rx=0 tx=0).stop 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 shutdown_connections 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb82406c920 0x7fb82406ede0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8440cb640 0x7fb8441f3b70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8440d1240 0x7fb8440cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 --2- 172.21.15.40:0/3722688464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8440d0d60 0x7fb8441521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 >> 172.21.15.40:0/3722688464 conn(0x7fb8440d2e60 msgr2=0x7fb8440b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 shutdown_connections 2024-04-26T21:41:46.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.015+0000 7fb8491f0640 1 -- 172.21.15.40:0/3722688464 wait complete. 2024-04-26T21:41:46.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:41:46.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.129+0000 7f58e6493640 1 Processor -- start 2024-04-26T21:41:46.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.129+0000 7f58e6493640 1 -- start start 2024-04-26T21:41:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e6493640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58e00d3bc0 0x7f58e00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e6493640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58e00b5f20 0x7f58e00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e6493640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e00b7e50 con 0x7f58e0162880 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e6493640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f58e00d0580 con 0x7f58e00b5f20 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e6493640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f58e00d1800 con 0x7f58e00d3bc0 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e4a30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58e00b5f20 0x7f58e00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58dffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f58e0162880 0x7f58e00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44592/0 (socket says 172.21.15.40:44592) 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58dffff640 1 -- 172.21.15.40:0/2118674304 learned_addr learned my addr 172.21.15.40:0/2118674304 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58df7fe640 1 --2- 172.21.15.40:0/2118674304 >> v2:172.21.15.184:3300/0 conn(0x7f58e00d3bc0 0x7f58e00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e4a30640 1 -- 172.21.15.40:0/2118674304 >> v2:172.21.15.184:3300/0 conn(0x7f58e00d3bc0 msgr2=0x7f58e00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e4a30640 1 --2- 172.21.15.40:0/2118674304 >> v2:172.21.15.184:3300/0 conn(0x7f58e00d3bc0 0x7f58e00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e4a30640 1 -- 172.21.15.40:0/2118674304 >> v1:172.21.15.40:6789/0 conn(0x7f58e0162880 legacy=0x7f58e00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58e4a30640 1 -- 172.21.15.40:0/2118674304 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e00d07d0 con 0x7f58e00b5f20 2024-04-26T21:41:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.130+0000 7f58deffd640 1 -- 172.21.15.40:0/2118674304 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2149370626 0 0) 0x7f58e00b7e50 con 0x7f58e0162880 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.131+0000 7f58e4a30640 1 --2- 172.21.15.40:0/2118674304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e00b5f20 0x7f58e00b8300 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f58e008d3d0 tx=0x7f58d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2078761c87435e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.131+0000 7f58deffd640 1 -- 172.21.15.40:0/2118674304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8007740 con 0x7f58e00b5f20 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.131+0000 7f58deffd640 1 -- 172.21.15.40:0/2118674304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d80078e0 con 0x7f58e00b5f20 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.131+0000 7f58deffd640 1 -- 172.21.15.40:0/2118674304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8007cb0 con 0x7f58e00b5f20 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.132+0000 7f58e6493640 1 -- 172.21.15.40:0/2118674304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e00b5f20 msgr2=0x7f58e00b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.132+0000 7f58e6493640 1 --2- 172.21.15.40:0/2118674304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e00b5f20 0x7f58e00b8300 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f58e008d3d0 tx=0x7f58d802a680 comp rx=0 tx=0).stop 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.132+0000 7f58e6493640 1 -- 172.21.15.40:0/2118674304 shutdown_connections 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.132+0000 7f58e6493640 1 --2- 172.21.15.40:0/2118674304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e00b5f20 0x7f58e00b8300 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.132+0000 7f58e6493640 1 --2- 172.21.15.40:0/2118674304 >> v2:172.21.15.184:3300/0 conn(0x7f58e00d3bc0 0x7f58e00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.132+0000 7f58e6493640 1 -- 172.21.15.40:0/2118674304 >> 172.21.15.40:0/2118674304 conn(0x7f58e00d4f60 msgr2=0x7f58e00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.132+0000 7f58e6493640 1 -- 172.21.15.40:0/2118674304 shutdown_connections 2024-04-26T21:41:46.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.132+0000 7f58e6493640 1 -- 172.21.15.40:0/2118674304 wait complete. 2024-04-26T21:41:46.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.133+0000 7f58e6493640 1 Processor -- start 2024-04-26T21:41:46.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.133+0000 7f58e6493640 1 -- start start 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e6493640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58e00b5f20 0x7f58e00b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e6493640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58e00d3bc0 0x7f58e00b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e6493640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e0162880 0x7f58e00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e6493640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0162c60 con 0x7f58e0162880 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e6493640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58e00d0d90 con 0x7f58e00d3bc0 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e6493640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58e0162de0 con 0x7f58e00b5f20 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e0162880 0x7f58e00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e0162880 0x7f58e00b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41554/0 (socket says 172.21.15.40:41554) 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 -- 172.21.15.40:0/124104753 learned_addr learned my addr 172.21.15.40:0/124104753 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58df7fe640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58e00d3bc0 0x7f58e00b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58dffff640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58e00b5f20 0x7f58e00b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 -- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58e00b5f20 msgr2=0x7f58e00b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58e00b5f20 0x7f58e00b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 -- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58e00d3bc0 msgr2=0x7f58e00b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58e00d3bc0 0x7f58e00b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 -- 172.21.15.40:0/124104753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e00b48a0 con 0x7f58e0162880 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58dffff640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58e00b5f20 0x7f58e00b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.134+0000 7f58e4a30640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e0162880 0x7f58e00b9230 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009e50 tx=0x7f58d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.135+0000 7f58dcff9640 1 -- 172.21.15.40:0/124104753 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d803f070 con 0x7f58e0162880 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.135+0000 7f58dcff9640 1 -- 172.21.15.40:0/124104753 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d8031bf0 con 0x7f58e0162880 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.135+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e00b4b80 con 0x7f58e0162880 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.135+0000 7f58dcff9640 1 -- 172.21.15.40:0/124104753 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8044460 con 0x7f58e0162880 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.135+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e00b5140 con 0x7f58e0162880 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.136+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e00cd7f0 con 0x7f58e0162880 2024-04-26T21:41:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.136+0000 7f58dcff9640 1 -- 172.21.15.40:0/124104753 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58d8007790 con 0x7f58e0162880 2024-04-26T21:41:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.136+0000 7f58dcff9640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58c006c8f0 0x7f58c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.137+0000 7f58dcff9640 1 -- 172.21.15.40:0/124104753 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f58d80ad5c0 con 0x7f58e0162880 2024-04-26T21:41:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.137+0000 7f58dffff640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58c006c8f0 0x7f58c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.137+0000 7f58dffff640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58c006c8f0 0x7f58c006edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000f60 tx=0x7f58c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.141+0000 7f58dcff9640 1 -- 172.21.15.40:0/124104753 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f58d807b8a0 con 0x7f58e0162880 2024-04-26T21:41:46.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.339+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58e00b45b0 con 0x7f58e0162880 2024-04-26T21:41:46.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.339+0000 7f58dcff9640 1 -- 172.21.15.40:0/124104753 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v82) ==== 80+0+122 (secure 0 0 0) 0x7f58d807b1a0 con 0x7f58e0162880 2024-04-26T21:41:46.339 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/3115383225 2024-04-27T21:41:39.656262+0000 2024-04-26T21:41:46.339 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/3115383225 2024-04-27T21:41:39.656262+0000 2024-04-26T21:41:46.340 INFO:teuthology.orchestra.run.smithi040.stderr:listed 2 entries 2024-04-26T21:41:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58c006c8f0 msgr2=0x7f58c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58c006c8f0 0x7f58c006edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000f60 tx=0x7f58c8009040 comp rx=0 tx=0).stop 2024-04-26T21:41:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e0162880 msgr2=0x7f58e00b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e0162880 0x7f58e00b9230 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009e50 tx=0x7f58d8004a40 comp rx=0 tx=0).stop 2024-04-26T21:41:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 shutdown_connections 2024-04-26T21:41:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58c006c8f0 0x7f58c006edb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58e0162880 0x7f58e00b9230 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58e00d3bc0 0x7f58e00b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 --2- 172.21.15.40:0/124104753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58e00b5f20 0x7f58e00b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 >> 172.21.15.40:0/124104753 conn(0x7f58e00d4f60 msgr2=0x7f58e00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 shutdown_connections 2024-04-26T21:41:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.341+0000 7f58e6493640 1 -- 172.21.15.40:0/124104753 wait complete. 2024-04-26T21:41:46.352 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:41:46.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:46.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:41:46.379 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:41:46.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:41:46.379 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:41:46.435 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:41:46.435 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:41:46.435 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:46.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:41:46.454 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:41:46.454 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:41:46.454 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:46.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:41:46.491 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:41:46.491 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:41:46.491 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:46.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:41:46.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:46.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.632+0000 7f72e2751640 1 Processor -- start 2024-04-26T21:41:46.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.632+0000 7f72e2751640 1 -- start start 2024-04-26T21:41:46.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.632+0000 7f72e2751640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72dc15d1e0 0x7f72dc0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.632+0000 7f72e2751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 0x7f72dc0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.632+0000 7f72e2751640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72dc0ce840 con 0x7f72dc0d18c0 2024-04-26T21:41:46.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72e2751640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f72dc0cede0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72e2751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc070650 con 0x7f72dc0d0f20 2024-04-26T21:41:46.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72e0cee640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f72dc0d18c0 0x7f72dc0cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44602/0 (socket says 172.21.15.40:44602) 2024-04-26T21:41:46.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72e0cee640 1 -- 172.21.15.40:0/3747080729 learned_addr learned my addr 172.21.15.40:0/3747080729 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72dbfff640 1 --2- 172.21.15.40:0/3747080729 >> v2:172.21.15.184:3300/0 conn(0x7f72dc15d1e0 0x7f72dc0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72db7fe640 1 --2- 172.21.15.40:0/3747080729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 0x7f72dc0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72daffd640 1 -- 172.21.15.40:0/3747080729 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705588568 0 0) 0x7f72dc0ce840 con 0x7f72dc0d18c0 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72daffd640 1 -- 172.21.15.40:0/3747080729 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72c4003520 con 0x7f72dc0d18c0 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72daffd640 1 -- 172.21.15.40:0/3747080729 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1625260901 0 0) 0x7f72c4003520 con 0x7f72dc0d18c0 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72dbfff640 1 -- 172.21.15.40:0/3747080729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 msgr2=0x7f72dc0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72dbfff640 1 --2- 172.21.15.40:0/3747080729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 0x7f72dc0d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72dbfff640 1 -- 172.21.15.40:0/3747080729 >> v1:172.21.15.40:6789/0 conn(0x7f72dc0d18c0 legacy=0x7f72dc0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.633+0000 7f72dbfff640 1 -- 172.21.15.40:0/3747080729 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc0ceb00 con 0x7f72dc15d1e0 2024-04-26T21:41:46.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.634+0000 7f72dbfff640 1 --2- 172.21.15.40:0/3747080729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0d09e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0097a0 tx=0x7f72cc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd58bf31c5974b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.634+0000 7f72daffd640 1 -- 172.21.15.40:0/3747080729 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc037070 con 0x7f72dc15d1e0 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.634+0000 7f72daffd640 1 -- 172.21.15.40:0/3747080729 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc02aba0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.634+0000 7f72daffd640 1 -- 172.21.15.40:0/3747080729 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d40019f0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.635+0000 7f72e2751640 1 -- 172.21.15.40:0/3747080729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc15d1e0 msgr2=0x7f72dc0d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.635+0000 7f72e2751640 1 --2- 172.21.15.40:0/3747080729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0d09e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0097a0 tx=0x7f72cc02a110 comp rx=0 tx=0).stop 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.635+0000 7f72e2751640 1 -- 172.21.15.40:0/3747080729 shutdown_connections 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.635+0000 7f72e2751640 1 --2- 172.21.15.40:0/3747080729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 0x7f72dc0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.635+0000 7f72e2751640 1 --2- 172.21.15.40:0/3747080729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0d09e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.635+0000 7f72e2751640 1 -- 172.21.15.40:0/3747080729 >> 172.21.15.40:0/3747080729 conn(0x7f72dc0d2da0 msgr2=0x7f72dc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.635+0000 7f72e2751640 1 -- 172.21.15.40:0/3747080729 shutdown_connections 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.635+0000 7f72e2751640 1 -- 172.21.15.40:0/3747080729 wait complete. 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.636+0000 7f72e2751640 1 Processor -- start 2024-04-26T21:41:46.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.636+0000 7f72e2751640 1 -- start start 2024-04-26T21:41:46.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e2751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 0x7f72dc14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e2751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d18c0 0x7f72dc0b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e2751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e2751640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc071af0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e2751640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc0ce4c0 con 0x7f72dc0d18c0 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e2751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc071c70 con 0x7f72dc0d0f20 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41562/0 (socket says 172.21.15.40:41562) 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 -- 172.21.15.40:0/986942939 learned_addr learned my addr 172.21.15.40:0/986942939 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72db7fe640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d18c0 0x7f72dc0b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72dbfff640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 0x7f72dc14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 -- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 msgr2=0x7f72dc14f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 0x7f72dc14f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 -- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d18c0 msgr2=0x7f72dc0b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d18c0 0x7f72dc0b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 -- 172.21.15.40:0/986942939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc0c90e0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.637+0000 7f72e0cee640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0b7ac0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f72d400cc00 tx=0x7f72d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.638+0000 7f72d8ff9640 1 -- 172.21.15.40:0/986942939 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4017070 con 0x7f72dc15d1e0 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.638+0000 7f72d8ff9640 1 -- 172.21.15.40:0/986942939 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d400ce10 con 0x7f72dc15d1e0 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.638+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72dc14f9d0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.638+0000 7f72d8ff9640 1 -- 172.21.15.40:0/986942939 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4012600 con 0x7f72dc15d1e0 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.638+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72dc1500f0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.639+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72dc0694f0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.639+0000 7f72d8ff9640 1 -- 172.21.15.40:0/986942939 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72d401a070 con 0x7f72dc15d1e0 2024-04-26T21:41:46.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.639+0000 7f72d8ff9640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.640+0000 7f72d8ff9640 1 -- 172.21.15.40:0/986942939 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f72d408bab0 con 0x7f72dc15d1e0 2024-04-26T21:41:46.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.640+0000 7f72dbfff640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.640+0000 7f72dbfff640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f72cc034000 tx=0x7f72cc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.644+0000 7f72d8ff9640 1 -- 172.21.15.40:0/986942939 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72d4059740 con 0x7f72dc15d1e0 2024-04-26T21:41:46.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.870+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72dc06ce60 con 0x7f72dc15d1e0 2024-04-26T21:41:46.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.871+0000 7f72d8ff9640 1 -- 172.21.15.40:0/986942939 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5185 (secure 0 0 0) 0x7f72d405da40 con 0x7f72dc15d1e0 2024-04-26T21:41:46.871 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:46.871 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7485,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/379824506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":379824506},{"type":"v1","addr":"172.21.15.40:6835","nonce":379824506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:42.673330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7224":{"gid":7224,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7224,"qdb_cluster":[7224]},"id":5}]} 2024-04-26T21:41:46.871 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 78 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.872+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 msgr2=0x7f72c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.872+0000 7f72e2751640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f72cc034000 tx=0x7f72cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.872+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc15d1e0 msgr2=0x7f72dc0b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.872+0000 7f72e2751640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0b7ac0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f72d400cc00 tx=0x7f72d400a430 comp rx=0 tx=0).stop 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.873+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 shutdown_connections 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.873+0000 7f72e2751640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f72c806c8f0 0x7f72c806edb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.873+0000 7f72e2751640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72dc15d1e0 0x7f72dc0b7ac0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.873+0000 7f72e2751640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72dc0d18c0 0x7f72dc0b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.873+0000 7f72e2751640 1 --2- 172.21.15.40:0/986942939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72dc0d0f20 0x7f72dc14f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.873+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 >> 172.21.15.40:0/986942939 conn(0x7f72dc0d2da0 msgr2=0x7f72dc0d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:46.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.873+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 shutdown_connections 2024-04-26T21:41:46.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.873+0000 7f72e2751640 1 -- 172.21.15.40:0/986942939 wait complete. 2024-04-26T21:41:46.884 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:46.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.984+0000 7f7bf0751640 1 Processor -- start 2024-04-26T21:41:46.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.984+0000 7f7bf0751640 1 -- start start 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.985+0000 7f7bf0751640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7be80d06f0 0x7f7be80d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.985+0000 7f7bf0751640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7be80b2850 0x7f7be80b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.985+0000 7f7bf0751640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be80d0ba0 con 0x7f7be815f3a0 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.985+0000 7f7bf0751640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7be80d0e20 con 0x7f7be80b2850 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.985+0000 7f7bf0751640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7be80b1d70 con 0x7f7be80d06f0 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.985+0000 7f7beecee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7be80b2850 0x7f7be80b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.985+0000 7f7beecee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7be80b2850 0x7f7be80b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41578/0 (socket says 172.21.15.40:41578) 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.985+0000 7f7beecee640 1 -- 172.21.15.40:0/3559741095 learned_addr learned my addr 172.21.15.40:0/3559741095 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.986+0000 7f7bedcec640 1 --2- 172.21.15.40:0/3559741095 >> v2:172.21.15.184:3300/0 conn(0x7f7be80d06f0 0x7f7be80d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.986+0000 7f7beecee640 1 -- 172.21.15.40:0/3559741095 >> v2:172.21.15.184:3300/0 conn(0x7f7be80d06f0 msgr2=0x7f7be80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.986+0000 7f7beecee640 1 --2- 172.21.15.40:0/3559741095 >> v2:172.21.15.184:3300/0 conn(0x7f7be80d06f0 0x7f7be80d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.986+0000 7f7beecee640 1 -- 172.21.15.40:0/3559741095 >> v1:172.21.15.184:6789/0 conn(0x7f7be815f3a0 legacy=0x7f7be80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.986+0000 7f7beecee640 1 -- 172.21.15.40:0/3559741095 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be80b2050 con 0x7f7be80b2850 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.986+0000 7f7beecee640 1 --2- 172.21.15.40:0/3559741095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80b2850 0x7f7be80b2c80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7be80b7b80 tx=0x7f7bd802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502bff3330503693 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bed4eb640 1 -- 172.21.15.40:0/3559741095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8036070 con 0x7f7be80b2850 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bed4eb640 1 -- 172.21.15.40:0/3559741095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd8031600 con 0x7f7be80b2850 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bed4eb640 1 -- 172.21.15.40:0/3559741095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80319d0 con 0x7f7be80b2850 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bf0751640 1 -- 172.21.15.40:0/3559741095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80b2850 msgr2=0x7f7be80b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bf0751640 1 --2- 172.21.15.40:0/3559741095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80b2850 0x7f7be80b2c80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7be80b7b80 tx=0x7f7bd802a730 comp rx=0 tx=0).stop 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bf0751640 1 -- 172.21.15.40:0/3559741095 shutdown_connections 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bf0751640 1 --2- 172.21.15.40:0/3559741095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80b2850 0x7f7be80b2c80 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bf0751640 1 --2- 172.21.15.40:0/3559741095 >> v2:172.21.15.184:3300/0 conn(0x7f7be80d06f0 0x7f7be80d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bf0751640 1 -- 172.21.15.40:0/3559741095 >> 172.21.15.40:0/3559741095 conn(0x7f7be80d4f90 msgr2=0x7f7be80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bf0751640 1 -- 172.21.15.40:0/3559741095 shutdown_connections 2024-04-26T21:41:46.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.987+0000 7f7bf0751640 1 -- 172.21.15.40:0/3559741095 wait complete. 2024-04-26T21:41:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.988+0000 7f7bf0751640 1 Processor -- start 2024-04-26T21:41:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bf0751640 1 -- start start 2024-04-26T21:41:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bf0751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7be80b2850 0x7f7be80b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bf0751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80d06f0 0x7f7be8159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bf0751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7be815f3a0 0x7f7be8161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bf0751640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7be815f900 con 0x7f7be80d06f0 2024-04-26T21:41:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bf0751640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7be815f780 con 0x7f7be815f3a0 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bf0751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7be80d23c0 con 0x7f7be80b2850 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bedcec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80d06f0 0x7f7be8159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bedcec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80d06f0 0x7f7be8159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41586/0 (socket says 172.21.15.40:41586) 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.989+0000 7f7bedcec640 1 -- 172.21.15.40:0/765254878 learned_addr learned my addr 172.21.15.40:0/765254878 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7beecee640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7be815f3a0 0x7f7be8161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bee4ed640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7be80b2850 0x7f7be80b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bedcec640 1 -- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7be80b2850 msgr2=0x7f7be80b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bedcec640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7be80b2850 0x7f7be80b8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bedcec640 1 -- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7be815f3a0 msgr2=0x7f7be8161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bedcec640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7be815f3a0 0x7f7be8161bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bedcec640 1 -- 172.21.15.40:0/765254878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be81626b0 con 0x7f7be80d06f0 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7beecee640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7be815f3a0 0x7f7be8161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bedcec640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80d06f0 0x7f7be8159550 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00d830 tx=0x7f7bdc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7be6ffd640 1 -- 172.21.15.40:0/765254878 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00b6c0 con 0x7f7be80d06f0 2024-04-26T21:41:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7be6ffd640 1 -- 172.21.15.40:0/765254878 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bdc019070 con 0x7f7be80d06f0 2024-04-26T21:41:46.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be8162990 con 0x7f7be80d06f0 2024-04-26T21:41:46.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7be6ffd640 1 -- 172.21.15.40:0/765254878 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00bac0 con 0x7f7be80d06f0 2024-04-26T21:41:46.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.990+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be80b7f00 con 0x7f7be80d06f0 2024-04-26T21:41:46.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.991+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be80694f0 con 0x7f7be80d06f0 2024-04-26T21:41:46.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.991+0000 7f7be6ffd640 1 -- 172.21.15.40:0/765254878 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bdc01e460 con 0x7f7be80d06f0 2024-04-26T21:41:46.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.992+0000 7f7be6ffd640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bd406c8f0 0x7f7bd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.992+0000 7f7be6ffd640 1 -- 172.21.15.40:0/765254878 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f7bdc08c4f0 con 0x7f7be80d06f0 2024-04-26T21:41:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.992+0000 7f7bee4ed640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bd406c8f0 0x7f7bd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.993+0000 7f7bee4ed640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bd406c8f0 0x7f7bd406edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7be8160ab0 tx=0x7f7be0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:46.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:46.996+0000 7f7be6ffd640 1 -- 172.21.15.40:0/765254878 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7bdc05a740 con 0x7f7be80d06f0 2024-04-26T21:41:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.182+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7be80b6860 con 0x7f7be80d06f0 2024-04-26T21:41:47.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.184+0000 7f7be6ffd640 1 -- 172.21.15.40:0/765254878 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+17038 (secure 0 0 0) 0x7f7bdc05a040 con 0x7f7be80d06f0 2024-04-26T21:41:47.184 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:47.184 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:39.656336+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:41:05.508768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T21:41:06.198815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3115383225":"2024-04-27T21:41:39.656262+0000","172.21.15.40:6834/3115383225":"2024-04-27T21:41:39.656262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.185+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bd406c8f0 msgr2=0x7f7bd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.185+0000 7f7bf0751640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bd406c8f0 0x7f7bd406edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7be8160ab0 tx=0x7f7be0009040 comp rx=0 tx=0).stop 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80d06f0 msgr2=0x7f7be8159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80d06f0 0x7f7be8159550 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00d830 tx=0x7f7bdc00dcf0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 shutdown_connections 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bd406c8f0 0x7f7bd406edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7be815f3a0 0x7f7be8161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7be80d06f0 0x7f7be8159550 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 --2- 172.21.15.40:0/765254878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7be80b2850 0x7f7be80b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 >> 172.21.15.40:0/765254878 conn(0x7f7be80d4f90 msgr2=0x7f7be80d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:47.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 shutdown_connections 2024-04-26T21:41:47.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.186+0000 7f7bf0751640 1 -- 172.21.15.40:0/765254878 wait complete. 2024-04-26T21:41:47.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:41:05.508768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:41:06.198815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:41:47.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:41:47.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.300+0000 7f5fe38d1640 1 Processor -- start 2024-04-26T21:41:47.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.300+0000 7f5fe38d1640 1 -- start start 2024-04-26T21:41:47.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.301+0000 7f5fe38d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5fdc0d3bc0 0x7f5fdc0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.301+0000 7f5fe38d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fdc0d1040 con 0x7f5fdc15f550 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.301+0000 7f5fe38d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fdc0c9680 con 0x7f5fdc15f170 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.301+0000 7f5fe38d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5fdc0c9430 con 0x7f5fdc0d3bc0 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.301+0000 7f5fe0e6c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5fdc15f550 0x7f5fdc0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44606/0 (socket says 172.21.15.40:44606) 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.301+0000 7f5fe0e6c640 1 -- 172.21.15.40:0/227876976 learned_addr learned my addr 172.21.15.40:0/227876976 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.301+0000 7f5fe1e6e640 1 --2- 172.21.15.40:0/227876976 >> v2:172.21.15.184:3300/0 conn(0x7f5fdc0d3bc0 0x7f5fdc0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.302+0000 7f5fdbfff640 1 -- 172.21.15.40:0/227876976 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 489722135 0 0) 0x7f5fd0001110 con 0x7f5fdc15f550 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.302+0000 7f5fdbfff640 1 -- 172.21.15.40:0/227876976 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fc4003520 con 0x7f5fdc15f550 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.302+0000 7f5fdbfff640 1 -- 172.21.15.40:0/227876976 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2630701242 0 0) 0x7f5fc4003520 con 0x7f5fdc15f550 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.302+0000 7f5fe1e6e640 1 -- 172.21.15.40:0/227876976 >> v1:172.21.15.184:6789/0 conn(0x7f5fdc15f170 legacy=0x7f5fdc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.302+0000 7f5fe1e6e640 1 -- 172.21.15.40:0/227876976 >> v1:172.21.15.40:6789/0 conn(0x7f5fdc15f550 legacy=0x7f5fdc0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.302+0000 7f5fe1e6e640 1 -- 172.21.15.40:0/227876976 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc0c9960 con 0x7f5fdc0d3bc0 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fe1e6e640 1 --2- 172.21.15.40:0/227876976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 0x7f5fdc0d5a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0c95b0 tx=0x7f5fcc02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=111cdbbcb6c94d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fdbfff640 1 -- 172.21.15.40:0/227876976 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0001840 con 0x7f5fdc0d3bc0 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fdbfff640 1 -- 172.21.15.40:0/227876976 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fcc007540 con 0x7f5fdc0d3bc0 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fdbfff640 1 -- 172.21.15.40:0/227876976 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc007710 con 0x7f5fdc0d3bc0 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fe38d1640 1 -- 172.21.15.40:0/227876976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 msgr2=0x7f5fdc0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fe38d1640 1 --2- 172.21.15.40:0/227876976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 0x7f5fdc0d5a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0c95b0 tx=0x7f5fcc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fe38d1640 1 -- 172.21.15.40:0/227876976 shutdown_connections 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fe38d1640 1 --2- 172.21.15.40:0/227876976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 0x7f5fdc0d5a80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fe38d1640 1 -- 172.21.15.40:0/227876976 >> 172.21.15.40:0/227876976 conn(0x7f5fdc0d4f60 msgr2=0x7f5fdc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fe38d1640 1 -- 172.21.15.40:0/227876976 shutdown_connections 2024-04-26T21:41:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.303+0000 7f5fe38d1640 1 -- 172.21.15.40:0/227876976 wait complete. 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.304+0000 7f5fe38d1640 1 Processor -- start 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe38d1640 1 -- start start 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe38d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 0x7f5fdc15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe38d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fdc15f170 0x7f5fdc1f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe38d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fdc0ba8c0 0x7f5fdc1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe38d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc0d2e30 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe38d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc0d2cb0 con 0x7f5fdc0d3bc0 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe38d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5fdc0d2fb0 con 0x7f5fdc15f170 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe1e6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fdc0ba8c0 0x7f5fdc1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe1e6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fdc0ba8c0 0x7f5fdc1525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41596/0 (socket says 172.21.15.40:41596) 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe1e6e640 1 -- 172.21.15.40:0/1949538734 learned_addr learned my addr 172.21.15.40:0/1949538734 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe166d640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 0x7f5fdc15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.305+0000 7f5fe0e6c640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fdc15f170 0x7f5fdc1f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fe1e6e640 1 -- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fdc15f170 msgr2=0x7f5fdc1f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fe1e6e640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fdc15f170 0x7f5fdc1f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fe1e6e640 1 -- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 msgr2=0x7f5fdc15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fe1e6e640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 0x7f5fdc15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fe1e6e640 1 -- 172.21.15.40:0/1949538734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc1f58d0 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fe1e6e640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fdc0ba8c0 0x7f5fdc1525e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009eb0 tx=0x7f5fcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1949538734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc040070 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1949538734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fcc03c830 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1949538734 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc03cbe0 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc1f5bb0 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.306+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1f6170 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.307+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc0d58e0 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.308+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1949538734 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fcc039040 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.308+0000 7f5fd9ffb640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fc806c8f0 0x7f5fc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.308+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1949538734 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f5fcc0ad280 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.309+0000 7f5fe166d640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fc806c8f0 0x7f5fc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.309+0000 7f5fe166d640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fc806c8f0 0x7f5fc806edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4000f60 tx=0x7f5fd4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.313+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1949538734 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fdc0d58e0 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.497+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fdc0694f0 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.498+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1949538734 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f5fcc07afb0 con 0x7f5fdc0ba8c0 2024-04-26T21:41:47.498 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fc806c8f0 msgr2=0x7f5fc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fc806c8f0 0x7f5fc806edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4000f60 tx=0x7f5fd4009040 comp rx=0 tx=0).stop 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fdc0ba8c0 msgr2=0x7f5fdc1525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fdc0ba8c0 0x7f5fdc1525e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009eb0 tx=0x7f5fcc004a40 comp rx=0 tx=0).stop 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 shutdown_connections 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fc806c8f0 0x7f5fc806edb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fdc0ba8c0 0x7f5fdc1525e0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fdc15f170 0x7f5fdc1f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 --2- 172.21.15.40:0/1949538734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fdc0d3bc0 0x7f5fdc15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 >> 172.21.15.40:0/1949538734 conn(0x7f5fdc0d4f60 msgr2=0x7f5fdc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:47.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 shutdown_connections 2024-04-26T21:41:47.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.500+0000 7f5fe38d1640 1 -- 172.21.15.40:0/1949538734 wait complete. 2024-04-26T21:41:47.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:41:47.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.614+0000 7fd6a2869640 1 Processor -- start 2024-04-26T21:41:47.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.614+0000 7fd6a2869640 1 -- start start 2024-04-26T21:41:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd6a2869640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd69c070350 0x7fd69c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd6a2869640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd69c071090 0x7fd69c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd6a2869640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd69c0d2390 con 0x7fd69c0cfde0 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd6a2869640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd69c0afab0 con 0x7fd69c070350 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd6a2869640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd69c06dd20 con 0x7fd69c071090 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd69b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd69c070350 0x7fd69c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd69bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd69c0cfde0 0x7fd69c0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44612/0 (socket says 172.21.15.40:44612) 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd69bfff640 1 -- 172.21.15.40:0/1358766109 learned_addr learned my addr 172.21.15.40:0/1358766109 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd6a0e06640 1 --2- 172.21.15.40:0/1358766109 >> v2:172.21.15.184:3300/0 conn(0x7fd69c071090 0x7fd69c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd69b7fe640 1 -- 172.21.15.40:0/1358766109 >> v2:172.21.15.184:3300/0 conn(0x7fd69c071090 msgr2=0x7fd69c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd69b7fe640 1 --2- 172.21.15.40:0/1358766109 >> v2:172.21.15.184:3300/0 conn(0x7fd69c071090 0x7fd69c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd69b7fe640 1 -- 172.21.15.40:0/1358766109 >> v1:172.21.15.40:6789/0 conn(0x7fd69c0cfde0 legacy=0x7fd69c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd69b7fe640 1 -- 172.21.15.40:0/1358766109 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c06d3f0 con 0x7fd69c070350 2024-04-26T21:41:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.615+0000 7fd69affd640 1 -- 172.21.15.40:0/1358766109 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1892270012 0 0) 0x7fd69c0d2390 con 0x7fd69c0cfde0 2024-04-26T21:41:47.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.616+0000 7fd69b7fe640 1 --2- 172.21.15.40:0/1358766109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c070350 0x7fd69c070b50 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0c6860 tx=0x7fd68c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd72f3cb95feac7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.616+0000 7fd69affd640 1 -- 172.21.15.40:0/1358766109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c037070 con 0x7fd69c070350 2024-04-26T21:41:47.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.616+0000 7fd69affd640 1 -- 172.21.15.40:0/1358766109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c032440 con 0x7fd69c070350 2024-04-26T21:41:47.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.616+0000 7fd69affd640 1 -- 172.21.15.40:0/1358766109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c02ae60 con 0x7fd69c070350 2024-04-26T21:41:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.617+0000 7fd6a2869640 1 -- 172.21.15.40:0/1358766109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c070350 msgr2=0x7fd69c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.617+0000 7fd6a2869640 1 --2- 172.21.15.40:0/1358766109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c070350 0x7fd69c070b50 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0c6860 tx=0x7fd68c02a110 comp rx=0 tx=0).stop 2024-04-26T21:41:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.617+0000 7fd6a2869640 1 -- 172.21.15.40:0/1358766109 shutdown_connections 2024-04-26T21:41:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.617+0000 7fd6a2869640 1 --2- 172.21.15.40:0/1358766109 >> v2:172.21.15.184:3300/0 conn(0x7fd69c071090 0x7fd69c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.617+0000 7fd6a2869640 1 --2- 172.21.15.40:0/1358766109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c070350 0x7fd69c070b50 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.617+0000 7fd6a2869640 1 -- 172.21.15.40:0/1358766109 >> 172.21.15.40:0/1358766109 conn(0x7fd69c0d2d80 msgr2=0x7fd69c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.617+0000 7fd6a2869640 1 -- 172.21.15.40:0/1358766109 shutdown_connections 2024-04-26T21:41:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.617+0000 7fd6a2869640 1 -- 172.21.15.40:0/1358766109 wait complete. 2024-04-26T21:41:47.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.618+0000 7fd6a2869640 1 Processor -- start 2024-04-26T21:41:47.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.618+0000 7fd6a2869640 1 -- start start 2024-04-26T21:41:47.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd6a2869640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c070350 0x7fd69c155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd6a2869640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c071090 0x7fd69c14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd6a2869640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c0cfde0 0x7fd69c158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd6a2869640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c06d9b0 con 0x7fd69c071090 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd6a2869640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c0714c0 con 0x7fd69c070350 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd6a2869640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd69c15ef30 con 0x7fd69c0cfde0 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c071090 0x7fd69c14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c071090 0x7fd69c14c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41610/0 (socket says 172.21.15.40:41610) 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 -- 172.21.15.40:0/2720814746 learned_addr learned my addr 172.21.15.40:0/2720814746 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd6a0e06640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c0cfde0 0x7fd69c158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69bfff640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c070350 0x7fd69c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 -- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c0cfde0 msgr2=0x7fd69c158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c0cfde0 0x7fd69c158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 -- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c070350 msgr2=0x7fd69c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c070350 0x7fd69c155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 -- 172.21.15.40:0/2720814746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c15acf0 con 0x7fd69c071090 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd6a0e06640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c0cfde0 0x7fd69c158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69bfff640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c070350 0x7fd69c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.619+0000 7fd69b7fe640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c071090 0x7fd69c14c220 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009890 tx=0x7fd68c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.620+0000 7fd698ff9640 1 -- 172.21.15.40:0/2720814746 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c037040 con 0x7fd69c071090 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.620+0000 7fd698ff9640 1 -- 172.21.15.40:0/2720814746 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c042070 con 0x7fd69c071090 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.620+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69c15afd0 con 0x7fd69c071090 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.620+0000 7fd698ff9640 1 -- 172.21.15.40:0/2720814746 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c030960 con 0x7fd69c071090 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.620+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69c15b590 con 0x7fd69c071090 2024-04-26T21:41:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.621+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd69c0694f0 con 0x7fd69c071090 2024-04-26T21:41:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.621+0000 7fd698ff9640 1 -- 172.21.15.40:0/2720814746 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd68c030b00 con 0x7fd69c071090 2024-04-26T21:41:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.622+0000 7fd698ff9640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd68806c8f0 0x7fd68806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.622+0000 7fd698ff9640 1 -- 172.21.15.40:0/2720814746 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7fd68c0ac6c0 con 0x7fd69c071090 2024-04-26T21:41:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.622+0000 7fd69bfff640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd68806c8f0 0x7fd68806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.622+0000 7fd69bfff640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd68806c8f0 0x7fd68806edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd690000f60 tx=0x7fd690009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.626+0000 7fd698ff9640 1 -- 172.21.15.40:0/2720814746 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd68c045360 con 0x7fd69c071090 2024-04-26T21:41:47.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.813+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd69c162590 con 0x7fd69c071090 2024-04-26T21:41:47.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.814+0000 7fd698ff9640 1 -- 172.21.15.40:0/2720814746 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fd68c07a910 con 0x7fd69c071090 2024-04-26T21:41:47.814 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd68806c8f0 msgr2=0x7fd68806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd68806c8f0 0x7fd68806edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd690000f60 tx=0x7fd690009040 comp rx=0 tx=0).stop 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c071090 msgr2=0x7fd69c14c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c071090 0x7fd69c14c220 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009890 tx=0x7fd68c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 shutdown_connections 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd68806c8f0 0x7fd68806edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd69c0cfde0 0x7fd69c158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69c071090 0x7fd69c14c220 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 --2- 172.21.15.40:0/2720814746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd69c070350 0x7fd69c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 >> 172.21.15.40:0/2720814746 conn(0x7fd69c0d2d80 msgr2=0x7fd69c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 shutdown_connections 2024-04-26T21:41:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.816+0000 7fd6a2869640 1 -- 172.21.15.40:0/2720814746 wait complete. 2024-04-26T21:41:47.827 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:41:47.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.932+0000 7f91590c8640 1 Processor -- start 2024-04-26T21:41:47.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.932+0000 7f91590c8640 1 -- start start 2024-04-26T21:41:47.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f91590c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 0x7f91540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f91590c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91540b40d0 0x7f91540b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f91590c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91540d0a10 con 0x7f91540d2320 2024-04-26T21:41:47.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f91590c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f91540d0cf0 con 0x7f91540b40d0 2024-04-26T21:41:47.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f91590c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91540d0fd0 con 0x7f91540d1a00 2024-04-26T21:41:47.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f915359e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91540b40d0 0x7f91540b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f915359e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91540b40d0 0x7f91540b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41616/0 (socket says 172.21.15.40:41616) 2024-04-26T21:41:47.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f915359e640 1 -- 172.21.15.40:0/2724139151 learned_addr learned my addr 172.21.15.40:0/2724139151 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:47.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f915359e640 1 -- 172.21.15.40:0/2724139151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 msgr2=0x7f91540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:47.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f915359e640 1 --2- 172.21.15.40:0/2724139151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 0x7f91540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f915359e640 1 -- 172.21.15.40:0/2724139151 >> v1:172.21.15.184:6789/0 conn(0x7f91540d2320 legacy=0x7f91541625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:47.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.933+0000 7f915359e640 1 -- 172.21.15.40:0/2724139151 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91540d40a0 con 0x7f91540b40d0 2024-04-26T21:41:47.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.934+0000 7f915359e640 1 --2- 172.21.15.40:0/2724139151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f91540b4b00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f914c009870 tx=0x7f914c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b20608928ed129 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.934+0000 7f9151d9b640 1 -- 172.21.15.40:0/2724139151 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c037070 con 0x7f91540b40d0 2024-04-26T21:41:47.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.934+0000 7f9151d9b640 1 -- 172.21.15.40:0/2724139151 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f914c032440 con 0x7f91540b40d0 2024-04-26T21:41:47.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.934+0000 7f9151d9b640 1 -- 172.21.15.40:0/2724139151 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c032810 con 0x7f91540b40d0 2024-04-26T21:41:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.935+0000 7f91590c8640 1 -- 172.21.15.40:0/2724139151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 msgr2=0x7f91540b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.935+0000 7f91590c8640 1 --2- 172.21.15.40:0/2724139151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f91540b4b00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f914c009870 tx=0x7f914c02a180 comp rx=0 tx=0).stop 2024-04-26T21:41:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.935+0000 7f91590c8640 1 -- 172.21.15.40:0/2724139151 shutdown_connections 2024-04-26T21:41:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.935+0000 7f91590c8640 1 --2- 172.21.15.40:0/2724139151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f91540b4b00 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.935+0000 7f91590c8640 1 --2- 172.21.15.40:0/2724139151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 0x7f91540d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.935+0000 7f91590c8640 1 -- 172.21.15.40:0/2724139151 >> 172.21.15.40:0/2724139151 conn(0x7f91540d4fd0 msgr2=0x7f91540d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.935+0000 7f91590c8640 1 -- 172.21.15.40:0/2724139151 shutdown_connections 2024-04-26T21:41:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.935+0000 7f91590c8640 1 -- 172.21.15.40:0/2724139151 wait complete. 2024-04-26T21:41:47.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.936+0000 7f91590c8640 1 Processor -- start 2024-04-26T21:41:47.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.936+0000 7f91590c8640 1 -- start start 2024-04-26T21:41:47.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f91590c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f915415f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f91590c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 0x7f91540cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f91590c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91540d2320 0x7f915415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f91590c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91540b5040 con 0x7f91540b40d0 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f91590c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9154162f50 con 0x7f91540d2320 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f91590c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91540b51c0 con 0x7f91540d1a00 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f9152d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f915415f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f9152d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f915415f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41624/0 (socket says 172.21.15.40:41624) 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f9152d9d640 1 -- 172.21.15.40:0/777018295 learned_addr learned my addr 172.21.15.40:0/777018295 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f915359e640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91540d2320 0x7f915415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.937+0000 7f915259c640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 0x7f91540cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f9152d9d640 1 -- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 msgr2=0x7f91540cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f9152d9d640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 0x7f91540cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f9152d9d640 1 -- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91540d2320 msgr2=0x7f915415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f9152d9d640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91540d2320 0x7f915415d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f9152d9d640 1 -- 172.21.15.40:0/777018295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91540b3e70 con 0x7f91540b40d0 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f915359e640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91540d2320 0x7f915415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f9152d9d640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f915415f9c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f914400ba50 tx=0x7f914400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f914b7fe640 1 -- 172.21.15.40:0/777018295 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144019070 con 0x7f91540b40d0 2024-04-26T21:41:47.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f914b7fe640 1 -- 172.21.15.40:0/777018295 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f914400cc00 con 0x7f91540b40d0 2024-04-26T21:41:47.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f914b7fe640 1 -- 172.21.15.40:0/777018295 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144014440 con 0x7f91540b40d0 2024-04-26T21:41:47.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91540ce390 con 0x7f91540b40d0 2024-04-26T21:41:47.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.938+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91540ce950 con 0x7f91540b40d0 2024-04-26T21:41:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.939+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91540694f0 con 0x7f91540b40d0 2024-04-26T21:41:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.940+0000 7f914b7fe640 1 -- 172.21.15.40:0/777018295 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f914401c070 con 0x7f91540b40d0 2024-04-26T21:41:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.940+0000 7f914b7fe640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f913406c920 0x7f913406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.940+0000 7f914b7fe640 1 -- 172.21.15.40:0/777018295 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f914408ba80 con 0x7f91540b40d0 2024-04-26T21:41:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.940+0000 7f915259c640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f913406c920 0x7f913406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.941+0000 7f915259c640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f913406c920 0x7f913406ede0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f913c006fd0 tx=0x7f913c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:47.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:47.945+0000 7f914b7fe640 1 -- 172.21.15.40:0/777018295 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9144059ce0 con 0x7f91540b40d0 2024-04-26T21:41:48.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.130+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f91540d0410 con 0x7f91540b40d0 2024-04-26T21:41:48.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.131+0000 7f914b7fe640 1 -- 172.21.15.40:0/777018295 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f91440595e0 con 0x7f91540b40d0 2024-04-26T21:41:48.131 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f913406c920 msgr2=0x7f913406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f913406c920 0x7f913406ede0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f913c006fd0 tx=0x7f913c008040 comp rx=0 tx=0).stop 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 msgr2=0x7f915415f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f915415f9c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f914400ba50 tx=0x7f914400bf10 comp rx=0 tx=0).stop 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 shutdown_connections 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f913406c920 0x7f913406ede0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91540d2320 0x7f915415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91540d1a00 0x7f91540cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 --2- 172.21.15.40:0/777018295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540b40d0 0x7f915415f9c0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 >> 172.21.15.40:0/777018295 conn(0x7f91540d4fd0 msgr2=0x7f915406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 shutdown_connections 2024-04-26T21:41:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.133+0000 7f91590c8640 1 -- 172.21.15.40:0/777018295 wait complete. 2024-04-26T21:41:48.143 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.245+0000 7fc32f015640 1 Processor -- start 2024-04-26T21:41:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.245+0000 7fc32f015640 1 -- start start 2024-04-26T21:41:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.245+0000 7fc32f015640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3280b4480 0x7fc32806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.245+0000 7fc32f015640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc32806c780 0x7fc3280af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.245+0000 7fc32f015640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3280b49f0 con 0x7fc32815f120 2024-04-26T21:41:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc32f015640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc32815f500 con 0x7fc3280b4480 2024-04-26T21:41:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc32f015640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3280ae8c0 con 0x7fc32806c780 2024-04-26T21:41:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc327fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3280b4480 0x7fc32806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc32cdb1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc32815f120 0x7fc32806f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44622/0 (socket says 172.21.15.40:44622) 2024-04-26T21:41:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc32cdb1640 1 -- 172.21.15.40:0/705653869 learned_addr learned my addr 172.21.15.40:0/705653869 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc32d5b2640 1 --2- 172.21.15.40:0/705653869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc32806c780 0x7fc3280af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc327fff640 1 -- 172.21.15.40:0/705653869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc32806c780 msgr2=0x7fc3280af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc327fff640 1 --2- 172.21.15.40:0/705653869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc32806c780 0x7fc3280af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc327fff640 1 -- 172.21.15.40:0/705653869 >> v1:172.21.15.40:6789/0 conn(0x7fc32815f120 legacy=0x7fc32806f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc327fff640 1 -- 172.21.15.40:0/705653869 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3280b4cd0 con 0x7fc3280b4480 2024-04-26T21:41:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.246+0000 7fc3277fe640 1 -- 172.21.15.40:0/705653869 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2052954785 0 0) 0x7fc3280b49f0 con 0x7fc32815f120 2024-04-26T21:41:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.247+0000 7fc327fff640 1 --2- 172.21.15.40:0/705653869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3280b4480 0x7fc32806c240 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc3140097a0 tx=0x7fc31402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d230be1db3674de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.247+0000 7fc3277fe640 1 -- 172.21.15.40:0/705653869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314037070 con 0x7fc3280b4480 2024-04-26T21:41:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.247+0000 7fc3277fe640 1 -- 172.21.15.40:0/705653869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31402ae50 con 0x7fc3280b4480 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.247+0000 7fc3277fe640 1 -- 172.21.15.40:0/705653869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314032660 con 0x7fc3280b4480 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.247+0000 7fc32f015640 1 -- 172.21.15.40:0/705653869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3280b4480 msgr2=0x7fc32806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.247+0000 7fc32f015640 1 --2- 172.21.15.40:0/705653869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3280b4480 0x7fc32806c240 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc3140097a0 tx=0x7fc31402a100 comp rx=0 tx=0).stop 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.247+0000 7fc32f015640 1 -- 172.21.15.40:0/705653869 shutdown_connections 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.247+0000 7fc32f015640 1 --2- 172.21.15.40:0/705653869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc32806c780 0x7fc3280af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.248+0000 7fc32f015640 1 --2- 172.21.15.40:0/705653869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3280b4480 0x7fc32806c240 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.248+0000 7fc32f015640 1 -- 172.21.15.40:0/705653869 >> 172.21.15.40:0/705653869 conn(0x7fc3280b3180 msgr2=0x7fc3280b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.248+0000 7fc32f015640 1 -- 172.21.15.40:0/705653869 shutdown_connections 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.248+0000 7fc32f015640 1 -- 172.21.15.40:0/705653869 wait complete. 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.249+0000 7fc32f015640 1 Processor -- start 2024-04-26T21:41:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.249+0000 7fc32f015640 1 -- start start 2024-04-26T21:41:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32f015640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32806c780 0x7fc32814b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32f015640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3280b4480 0x7fc3280caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32f015640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc32815f120 0x7fc3280cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32f015640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc3280af570 con 0x7fc32806c780 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32f015640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3280aeba0 con 0x7fc32815f120 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32f015640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3280af6f0 con 0x7fc3280b4480 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32806c780 0x7fc32814b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32806c780 0x7fc32814b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41644/0 (socket says 172.21.15.40:41644) 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 -- 172.21.15.40:0/2718478429 learned_addr learned my addr 172.21.15.40:0/2718478429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32d5b2640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc32815f120 0x7fc3280cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc327fff640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3280b4480 0x7fc3280caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 -- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3280b4480 msgr2=0x7fc3280caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3280b4480 0x7fc3280caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 -- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc32815f120 msgr2=0x7fc3280cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc32815f120 0x7fc3280cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 -- 172.21.15.40:0/2718478429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32814bd30 con 0x7fc32806c780 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32d5b2640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc32815f120 0x7fc3280cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32cdb1640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32806c780 0x7fc32814b190 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00c430 tx=0x7fc31c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc3257fa640 1 -- 172.21.15.40:0/2718478429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0074e0 con 0x7fc32806c780 2024-04-26T21:41:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc3257fa640 1 -- 172.21.15.40:0/2718478429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c0047e0 con 0x7fc32806c780 2024-04-26T21:41:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc3257fa640 1 -- 172.21.15.40:0/2718478429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c004bd0 con 0x7fc32806c780 2024-04-26T21:41:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.250+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32814c010 con 0x7fc32806c780 2024-04-26T21:41:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.251+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32814b6d0 con 0x7fc32806c780 2024-04-26T21:41:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.252+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc328068fc0 con 0x7fc32806c780 2024-04-26T21:41:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.252+0000 7fc3257fa640 1 -- 172.21.15.40:0/2718478429 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc31c011070 con 0x7fc32806c780 2024-04-26T21:41:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.253+0000 7fc3257fa640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc30c06c8f0 0x7fc30c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.253+0000 7fc3257fa640 1 -- 172.21.15.40:0/2718478429 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7fc31c08b980 con 0x7fc32806c780 2024-04-26T21:41:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.253+0000 7fc327fff640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc30c06c8f0 0x7fc30c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.254+0000 7fc327fff640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc30c06c8f0 0x7fc30c06edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc314034000 tx=0x7fc31402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.257+0000 7fc3257fa640 1 -- 172.21.15.40:0/2718478429 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc31c059c60 con 0x7fc32806c780 2024-04-26T21:41:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.483+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc328069cd0 con 0x7fc32806c780 2024-04-26T21:41:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.483+0000 7fc3257fa640 1 -- 172.21.15.40:0/2718478429 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5185 (secure 0 0 0) 0x7fc31c059560 con 0x7fc32806c780 2024-04-26T21:41:48.483 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:48.483 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7485,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/379824506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":379824506},{"type":"v1","addr":"172.21.15.40:6835","nonce":379824506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:42.673330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7224":{"gid":7224,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7224,"qdb_cluster":[7224]},"id":5}]} 2024-04-26T21:41:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 78 2024-04-26T21:41:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc30c06c8f0 msgr2=0x7fc30c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc30c06c8f0 0x7fc30c06edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc314034000 tx=0x7fc31402e000 comp rx=0 tx=0).stop 2024-04-26T21:41:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32806c780 msgr2=0x7fc32814b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32806c780 0x7fc32814b190 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00c430 tx=0x7fc31c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 shutdown_connections 2024-04-26T21:41:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc30c06c8f0 0x7fc30c06edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc32815f120 0x7fc3280cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3280b4480 0x7fc3280caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 --2- 172.21.15.40:0/2718478429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32806c780 0x7fc32814b190 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 >> 172.21.15.40:0/2718478429 conn(0x7fc3280b3180 msgr2=0x7fc3280afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 shutdown_connections 2024-04-26T21:41:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.485+0000 7fc32f015640 1 -- 172.21.15.40:0/2718478429 wait complete. 2024-04-26T21:41:48.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:48.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.597+0000 7fa28834e640 1 Processor -- start 2024-04-26T21:41:48.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.597+0000 7fa28834e640 1 -- start start 2024-04-26T21:41:48.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa28834e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa280162810 0x7fa280070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa28834e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2800b2340 0x7fa2800cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa28834e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2800d10d0 con 0x7fa2800cd930 2024-04-26T21:41:48.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa28834e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa2800d0750 con 0x7fa280162810 2024-04-26T21:41:48.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa28834e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2800d16a0 con 0x7fa2800b2340 2024-04-26T21:41:48.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa2860ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa280162810 0x7fa280070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa2868eb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa2800cd930 0x7fa2800d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44636/0 (socket says 172.21.15.40:44636) 2024-04-26T21:41:48.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa2868eb640 1 -- 172.21.15.40:0/2930509768 learned_addr learned my addr 172.21.15.40:0/2930509768 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:48.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa2858e9640 1 --2- 172.21.15.40:0/2930509768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2800b2340 0x7fa2800cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa2860ea640 1 -- 172.21.15.40:0/2930509768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2800b2340 msgr2=0x7fa2800cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:48.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa2860ea640 1 --2- 172.21.15.40:0/2930509768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2800b2340 0x7fa2800cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa2860ea640 1 -- 172.21.15.40:0/2930509768 >> v1:172.21.15.40:6789/0 conn(0x7fa2800cd930 legacy=0x7fa2800d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.598+0000 7fa2860ea640 1 -- 172.21.15.40:0/2930509768 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2800d1370 con 0x7fa280162810 2024-04-26T21:41:48.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.599+0000 7fa2860ea640 1 --2- 172.21.15.40:0/2930509768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa280162810 0x7fa280070350 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0097a0 tx=0x7fa27c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a92a714d9e4ba0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.599+0000 7fa2850e8640 1 -- 172.21.15.40:0/2930509768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c037070 con 0x7fa280162810 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.599+0000 7fa2850e8640 1 -- 172.21.15.40:0/2930509768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27c032440 con 0x7fa280162810 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.599+0000 7fa2850e8640 1 -- 172.21.15.40:0/2930509768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c032810 con 0x7fa280162810 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.600+0000 7fa28834e640 1 -- 172.21.15.40:0/2930509768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa280162810 msgr2=0x7fa280070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.600+0000 7fa28834e640 1 --2- 172.21.15.40:0/2930509768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa280162810 0x7fa280070350 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0097a0 tx=0x7fa27c02a180 comp rx=0 tx=0).stop 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.600+0000 7fa28834e640 1 -- 172.21.15.40:0/2930509768 shutdown_connections 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.600+0000 7fa28834e640 1 --2- 172.21.15.40:0/2930509768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2800b2340 0x7fa2800cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.600+0000 7fa28834e640 1 --2- 172.21.15.40:0/2930509768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa280162810 0x7fa280070350 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.600+0000 7fa28834e640 1 -- 172.21.15.40:0/2930509768 >> 172.21.15.40:0/2930509768 conn(0x7fa2800d2da0 msgr2=0x7fa2800d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.600+0000 7fa28834e640 1 -- 172.21.15.40:0/2930509768 shutdown_connections 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.600+0000 7fa28834e640 1 -- 172.21.15.40:0/2930509768 wait complete. 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.601+0000 7fa28834e640 1 Processor -- start 2024-04-26T21:41:48.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.601+0000 7fa28834e640 1 -- start start 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa28834e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2800b2340 0x7fa2800cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa28834e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2800cd930 0x7fa2800c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa28834e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa280162810 0x7fa28015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa28834e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa280070890 con 0x7fa2800b2340 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa28834e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2800d0470 con 0x7fa2800cd930 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa28834e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2800d2240 con 0x7fa280162810 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2800b2340 0x7fa2800cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2800b2340 0x7fa2800cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41670/0 (socket says 172.21.15.40:41670) 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 -- 172.21.15.40:0/1861562454 learned_addr learned my addr 172.21.15.40:0/1861562454 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:48.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2868eb640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa280162810 0x7fa28015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2858e9640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2800cd930 0x7fa2800c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 -- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa280162810 msgr2=0x7fa28015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa280162810 0x7fa28015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 -- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2800cd930 msgr2=0x7fa2800c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2800cd930 0x7fa2800c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 -- 172.21.15.40:0/1861562454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2800c6950 con 0x7fa2800b2340 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2858e9640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2800cd930 0x7fa2800c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2868eb640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa280162810 0x7fa28015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa2860ea640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2800b2340 0x7fa2800cf800 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa27c035040 tx=0x7fa27c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa276ffd640 1 -- 172.21.15.40:0/1861562454 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c037040 con 0x7fa2800b2340 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa276ffd640 1 -- 172.21.15.40:0/1861562454 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27c030770 con 0x7fa2800b2340 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2800c6bc0 con 0x7fa2800b2340 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa276ffd640 1 -- 172.21.15.40:0/1861562454 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c030b80 con 0x7fa2800b2340 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.602+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2800c7180 con 0x7fa2800b2340 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.603+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2800694f0 con 0x7fa2800b2340 2024-04-26T21:41:48.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.604+0000 7fa276ffd640 1 -- 172.21.15.40:0/1861562454 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa27c044020 con 0x7fa2800b2340 2024-04-26T21:41:48.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.604+0000 7fa276ffd640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa26406c8f0 0x7fa26406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.604+0000 7fa276ffd640 1 -- 172.21.15.40:0/1861562454 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7fa27c0ac210 con 0x7fa2800b2340 2024-04-26T21:41:48.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.604+0000 7fa2858e9640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa26406c8f0 0x7fa26406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.605+0000 7fa2858e9640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa26406c8f0 0x7fa26406edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa270006fd0 tx=0x7fa270008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.608+0000 7fa276ffd640 1 -- 172.21.15.40:0/1861562454 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa27c032e20 con 0x7fa2800b2340 2024-04-26T21:41:48.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.798+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2800b0ac0 con 0x7fa2800b2340 2024-04-26T21:41:48.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.799+0000 7fa276ffd640 1 -- 172.21.15.40:0/1861562454 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+17038 (secure 0 0 0) 0x7fa27c07a4f0 con 0x7fa2800b2340 2024-04-26T21:41:48.799 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:48.799 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:39.656336+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:41:05.508768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T21:41:06.198815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3115383225":"2024-04-27T21:41:39.656262+0000","172.21.15.40:6834/3115383225":"2024-04-27T21:41:39.656262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa26406c8f0 msgr2=0x7fa26406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa26406c8f0 0x7fa26406edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa270006fd0 tx=0x7fa270008040 comp rx=0 tx=0).stop 2024-04-26T21:41:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2800b2340 msgr2=0x7fa2800cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2800b2340 0x7fa2800cf800 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa27c035040 tx=0x7fa27c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 shutdown_connections 2024-04-26T21:41:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa26406c8f0 0x7fa26406edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa280162810 0x7fa28015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2800cd930 0x7fa2800c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 --2- 172.21.15.40:0/1861562454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2800b2340 0x7fa2800cf800 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 >> 172.21.15.40:0/1861562454 conn(0x7fa2800d2da0 msgr2=0x7fa2801618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:48.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 shutdown_connections 2024-04-26T21:41:48.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.802+0000 7fa28834e640 1 -- 172.21.15.40:0/1861562454 wait complete. 2024-04-26T21:41:48.813 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:41:48.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.914+0000 7f0d7cd78640 1 Processor -- start 2024-04-26T21:41:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.914+0000 7f0d7cd78640 1 -- start start 2024-04-26T21:41:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d7cd78640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d780b2340 0x7f0d780cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d7cd78640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0d780cd930 0x7f0d780d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d7cd78640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d780d10d0 con 0x7f0d78162810 2024-04-26T21:41:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d7cd78640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0d780d0750 con 0x7f0d780b2340 2024-04-26T21:41:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d7cd78640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0d780d16a0 con 0x7f0d780cd930 2024-04-26T21:41:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d75d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d780b2340 0x7f0d780cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d7659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0d78162810 0x7f0d78070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43188/0 (socket says 172.21.15.40:43188) 2024-04-26T21:41:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d7659c640 1 -- 172.21.15.40:0/3219776351 learned_addr learned my addr 172.21.15.40:0/3219776351 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d76d9d640 1 --2- 172.21.15.40:0/3219776351 >> v2:172.21.15.184:3300/0 conn(0x7f0d780cd930 0x7f0d780d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d75d9b640 1 -- 172.21.15.40:0/3219776351 >> v2:172.21.15.184:3300/0 conn(0x7f0d780cd930 msgr2=0x7f0d780d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d75d9b640 1 --2- 172.21.15.40:0/3219776351 >> v2:172.21.15.184:3300/0 conn(0x7f0d780cd930 0x7f0d780d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d75d9b640 1 -- 172.21.15.40:0/3219776351 >> v1:172.21.15.40:6789/0 conn(0x7f0d78162810 legacy=0x7f0d78070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.915+0000 7f0d75d9b640 1 -- 172.21.15.40:0/3219776351 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d780d1370 con 0x7f0d780b2340 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.916+0000 7f0d75d9b640 1 --2- 172.21.15.40:0/3219776351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d780cd3f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0d780b2280 tx=0x7f0d6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b578bef03e9f0a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.916+0000 7f0d7559a640 1 -- 172.21.15.40:0/3219776351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c037070 con 0x7f0d780b2340 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.916+0000 7f0d7559a640 1 -- 172.21.15.40:0/3219776351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c032440 con 0x7f0d780b2340 2024-04-26T21:41:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.916+0000 7f0d7559a640 1 -- 172.21.15.40:0/3219776351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c032810 con 0x7f0d780b2340 2024-04-26T21:41:48.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.917+0000 7f0d7cd78640 1 -- 172.21.15.40:0/3219776351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 msgr2=0x7f0d780cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.917+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/3219776351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d780cd3f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0d780b2280 tx=0x7f0d6c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.917+0000 7f0d7cd78640 1 -- 172.21.15.40:0/3219776351 shutdown_connections 2024-04-26T21:41:48.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.917+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/3219776351 >> v2:172.21.15.184:3300/0 conn(0x7f0d780cd930 0x7f0d780d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.917+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/3219776351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d780cd3f0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.917+0000 7f0d7cd78640 1 -- 172.21.15.40:0/3219776351 >> 172.21.15.40:0/3219776351 conn(0x7f0d780d2da0 msgr2=0x7f0d780d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.917+0000 7f0d7cd78640 1 -- 172.21.15.40:0/3219776351 shutdown_connections 2024-04-26T21:41:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.917+0000 7f0d7cd78640 1 -- 172.21.15.40:0/3219776351 wait complete. 2024-04-26T21:41:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.918+0000 7f0d7cd78640 1 Processor -- start 2024-04-26T21:41:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.918+0000 7f0d7cd78640 1 -- start start 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7cd78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d781584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7cd78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d780cd930 0x7f0d780cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7cd78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d78162810 0x7f0d780ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7cd78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780d2070 con 0x7f0d780b2340 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7cd78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780d0470 con 0x7f0d78162810 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7cd78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d780d21f0 con 0x7f0d780cd930 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d781584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d781584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56428/0 (socket says 172.21.15.40:56428) 2024-04-26T21:41:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7659c640 1 -- 172.21.15.40:0/1250034664 learned_addr learned my addr 172.21.15.40:0/1250034664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d76d9d640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d78162810 0x7f0d780ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7659c640 1 -- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d780cd930 msgr2=0x7f0d780cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7659c640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d780cd930 0x7f0d780cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7659c640 1 -- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d78162810 msgr2=0x7f0d780ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7659c640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d78162810 0x7f0d780ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.919+0000 7f0d7659c640 1 -- 172.21.15.40:0/1250034664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d780c8470 con 0x7f0d780b2340 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.920+0000 7f0d7659c640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d781584b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800c3c0 tx=0x7f0d6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.920+0000 7f0d66ffd640 1 -- 172.21.15.40:0/1250034664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68017070 con 0x7f0d780b2340 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.920+0000 7f0d66ffd640 1 -- 172.21.15.40:0/1250034664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d68005ce0 con 0x7f0d780b2340 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.920+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d780c8750 con 0x7f0d780b2340 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.920+0000 7f0d66ffd640 1 -- 172.21.15.40:0/1250034664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680072f0 con 0x7f0d780b2340 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.920+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d780c8d10 con 0x7f0d780b2340 2024-04-26T21:41:48.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.921+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d780694f0 con 0x7f0d780b2340 2024-04-26T21:41:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.921+0000 7f0d66ffd640 1 -- 172.21.15.40:0/1250034664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d6801a070 con 0x7f0d780b2340 2024-04-26T21:41:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.922+0000 7f0d66ffd640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d5806c8f0 0x7f0d5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.922+0000 7f0d66ffd640 1 -- 172.21.15.40:0/1250034664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f0d6808b120 con 0x7f0d780b2340 2024-04-26T21:41:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.922+0000 7f0d75d9b640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d5806c8f0 0x7f0d5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:48.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.923+0000 7f0d75d9b640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d5806c8f0 0x7f0d5806edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c034000 tx=0x7f0d6c02c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:48.926+0000 7f0d66ffd640 1 -- 172.21.15.40:0/1250034664 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d680592d0 con 0x7f0d780b2340 2024-04-26T21:41:49.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.149+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d780b1f30 con 0x7f0d780b2340 2024-04-26T21:41:49.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.149+0000 7f0d66ffd640 1 -- 172.21.15.40:0/1250034664 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5185 (secure 0 0 0) 0x7f0d68058bd0 con 0x7f0d780b2340 2024-04-26T21:41:49.149 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:49.149 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7485,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/379824506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":379824506},{"type":"v1","addr":"172.21.15.40:6835","nonce":379824506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:42.673330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7224":{"gid":7224,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7224,"qdb_cluster":[7224]},"id":5}]} 2024-04-26T21:41:49.150 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 78 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d5806c8f0 msgr2=0x7f0d5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d5806c8f0 0x7f0d5806edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c034000 tx=0x7f0d6c02c000 comp rx=0 tx=0).stop 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 msgr2=0x7f0d781584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d781584b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800c3c0 tx=0x7f0d6800c880 comp rx=0 tx=0).stop 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 shutdown_connections 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d5806c8f0 0x7f0d5806edb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d78162810 0x7f0d780ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d780cd930 0x7f0d780cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 --2- 172.21.15.40:0/1250034664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d780b2340 0x7f0d781584b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 >> 172.21.15.40:0/1250034664 conn(0x7f0d780d2da0 msgr2=0x7f0d780cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 shutdown_connections 2024-04-26T21:41:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.151+0000 7f0d7cd78640 1 -- 172.21.15.40:0/1250034664 wait complete. 2024-04-26T21:41:49.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.263+0000 7f87afe7e640 1 Processor -- start 2024-04-26T21:41:49.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.263+0000 7f87afe7e640 1 -- start start 2024-04-26T21:41:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.263+0000 7f87afe7e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87a8162880 0x7f87a80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87afe7e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87a80b5f20 0x7f87a80d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87afe7e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 0x7f87a80b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87afe7e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f87a80d0580 con 0x7f87a8162880 2024-04-26T21:41:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87afe7e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f87a80b7e50 con 0x7f87a80b5f20 2024-04-26T21:41:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87afe7e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87a80d1800 con 0x7f87a80d39f0 2024-04-26T21:41:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87adc1a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87a8162880 0x7f87a80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87adc1a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87a8162880 0x7f87a80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56438/0 (socket says 172.21.15.40:56438) 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87adc1a640 1 -- 172.21.15.40:0/3313948616 learned_addr learned my addr 172.21.15.40:0/3313948616 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87ad419640 1 --2- 172.21.15.40:0/3313948616 >> v2:172.21.15.184:3300/0 conn(0x7f87a80b5f20 0x7f87a80d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87ae41b640 1 --2- 172.21.15.40:0/3313948616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 0x7f87a80b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87adc1a640 1 -- 172.21.15.40:0/3313948616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 msgr2=0x7f87a80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87adc1a640 1 --2- 172.21.15.40:0/3313948616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 0x7f87a80b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87adc1a640 1 -- 172.21.15.40:0/3313948616 >> v2:172.21.15.184:3300/0 conn(0x7f87a80b5f20 msgr2=0x7f87a80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87adc1a640 1 --2- 172.21.15.40:0/3313948616 >> v2:172.21.15.184:3300/0 conn(0x7f87a80b5f20 0x7f87a80d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87adc1a640 1 -- 172.21.15.40:0/3313948616 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a80d07d0 con 0x7f87a8162880 2024-04-26T21:41:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.264+0000 7f87ad419640 1 --2- 172.21.15.40:0/3313948616 >> v2:172.21.15.184:3300/0 conn(0x7f87a80b5f20 0x7f87a80d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:49.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87adc1a640 1 --2- 172.21.15.40:0/3313948616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a8162880 0x7f87a80d2cb0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f87a808d3d0 tx=0x7f87a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9d6750715fc66d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87acc18640 1 -- 172.21.15.40:0/3313948616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4037070 con 0x7f87a8162880 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87acc18640 1 -- 172.21.15.40:0/3313948616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a4032440 con 0x7f87a8162880 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87acc18640 1 -- 172.21.15.40:0/3313948616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4032810 con 0x7f87a8162880 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87afe7e640 1 -- 172.21.15.40:0/3313948616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a8162880 msgr2=0x7f87a80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87afe7e640 1 --2- 172.21.15.40:0/3313948616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a8162880 0x7f87a80d2cb0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f87a808d3d0 tx=0x7f87a402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87afe7e640 1 -- 172.21.15.40:0/3313948616 shutdown_connections 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87afe7e640 1 --2- 172.21.15.40:0/3313948616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 0x7f87a80b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87afe7e640 1 --2- 172.21.15.40:0/3313948616 >> v2:172.21.15.184:3300/0 conn(0x7f87a80b5f20 0x7f87a80d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87afe7e640 1 --2- 172.21.15.40:0/3313948616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a8162880 0x7f87a80d2cb0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.265+0000 7f87afe7e640 1 -- 172.21.15.40:0/3313948616 >> 172.21.15.40:0/3313948616 conn(0x7f87a80d4f60 msgr2=0x7f87a80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.266+0000 7f87afe7e640 1 -- 172.21.15.40:0/3313948616 shutdown_connections 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.266+0000 7f87afe7e640 1 -- 172.21.15.40:0/3313948616 wait complete. 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.267+0000 7f87afe7e640 1 Processor -- start 2024-04-26T21:41:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.267+0000 7f87afe7e640 1 -- start start 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.267+0000 7f87afe7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a80b5f20 0x7f87a81514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.267+0000 7f87afe7e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 0x7f87a80b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.267+0000 7f87afe7e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a8162880 0x7f87a80b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.267+0000 7f87afe7e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f87a8162c60 con 0x7f87a80b5f20 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.267+0000 7f87afe7e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87a80d0d90 con 0x7f87a8162880 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.267+0000 7f87afe7e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87a8162de0 con 0x7f87a80d39f0 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a80b5f20 0x7f87a81514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a80b5f20 0x7f87a81514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56442/0 (socket says 172.21.15.40:56442) 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 -- 172.21.15.40:0/2570178263 learned_addr learned my addr 172.21.15.40:0/2570178263 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 -- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 msgr2=0x7f87a80b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87ad419640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 0x7f87a80b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87ae41b640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a8162880 0x7f87a80b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 0x7f87a80b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 -- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a8162880 msgr2=0x7f87a80b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a8162880 0x7f87a80b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 -- 172.21.15.40:0/2570178263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a8151a00 con 0x7f87a80b5f20 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87ae41b640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a8162880 0x7f87a80b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87adc1a640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a80b5f20 0x7f87a81514c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f87a40097c0 tx=0x7f87a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f879e7fc640 1 -- 172.21.15.40:0/2570178263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4037040 con 0x7f87a80b5f20 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f879e7fc640 1 -- 172.21.15.40:0/2570178263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a4042070 con 0x7f87a80b5f20 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a80b31e0 con 0x7f87a80b5f20 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f879e7fc640 1 -- 172.21.15.40:0/2570178263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4030bb0 con 0x7f87a80b5f20 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.268+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a80b37a0 con 0x7f87a80b5f20 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.269+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a80694f0 con 0x7f87a80b5f20 2024-04-26T21:41:49.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.270+0000 7f879e7fc640 1 -- 172.21.15.40:0/2570178263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87a4047460 con 0x7f87a80b5f20 2024-04-26T21:41:49.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.270+0000 7f879e7fc640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.270+0000 7f879e7fc640 1 -- 172.21.15.40:0/2570178263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f87a40ac990 con 0x7f87a80b5f20 2024-04-26T21:41:49.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.271+0000 7f87ad419640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.271+0000 7f87ad419640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f87a80b5480 tx=0x7f8798008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.275+0000 7f879e7fc640 1 -- 172.21.15.40:0/2570178263 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f87a407ab40 con 0x7f87a80b5f20 2024-04-26T21:41:49.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.502+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87a806e760 con 0x7f87a80b5f20 2024-04-26T21:41:49.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.503+0000 7f879e7fc640 1 -- 172.21.15.40:0/2570178263 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5185 (secure 0 0 0) 0x7f87a407a440 con 0x7f87a80b5f20 2024-04-26T21:41:49.503 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:49.503 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7485,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/379824506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":379824506},{"type":"v1","addr":"172.21.15.40:6835","nonce":379824506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:41:07.249301+0000","modified":"2024-04-26T21:41:42.673330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7224":{"gid":7224,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.40:6833/836437049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":836437049},{"type":"v1","addr":"172.21.15.40:6833","nonce":836437049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7224,"qdb_cluster":[7224]},"id":5}]} 2024-04-26T21:41:49.503 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 78 2024-04-26T21:41:49.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 msgr2=0x7f878c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f87a80b5480 tx=0x7f8798008040 comp rx=0 tx=0).stop 2024-04-26T21:41:49.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a80b5f20 msgr2=0x7f87a81514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a80b5f20 0x7f87a81514c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f87a40097c0 tx=0x7f87a40046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 shutdown_connections 2024-04-26T21:41:49.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a8162880 0x7f87a80b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a80d39f0 0x7f87a80b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 --2- 172.21.15.40:0/2570178263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a80b5f20 0x7f87a81514c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 >> 172.21.15.40:0/2570178263 conn(0x7f87a80d4f60 msgr2=0x7f87a8160850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:49.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 shutdown_connections 2024-04-26T21:41:49.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.505+0000 7f87afe7e640 1 -- 172.21.15.40:0/2570178263 wait complete. 2024-04-26T21:41:49.516 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.616+0000 7fef4bdc8640 1 Processor -- start 2024-04-26T21:41:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.616+0000 7fef4bdc8640 1 -- start start 2024-04-26T21:41:49.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.617+0000 7fef4bdc8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef4406d560 0x7fef440d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.617+0000 7fef4bdc8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef440d2420 con 0x7fef440d0070 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.617+0000 7fef4bdc8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef440cd6b0 con 0x7fef4406e270 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.617+0000 7fef4bdc8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fef44162d00 con 0x7fef4406d560 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.617+0000 7fef49b64640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fef440d0070 0x7fef4406db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43192/0 (socket says 172.21.15.40:43192) 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.617+0000 7fef49b64640 1 -- 172.21.15.40:0/2884408128 learned_addr learned my addr 172.21.15.40:0/2884408128 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.617+0000 7fef4a365640 1 --2- 172.21.15.40:0/2884408128 >> v2:172.21.15.40:3300/0 conn(0x7fef4406d560 0x7fef440d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.618+0000 7fef48b62640 1 -- 172.21.15.40:0/2884408128 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1623877172 0 0) 0x7fef440d2420 con 0x7fef440d0070 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.618+0000 7fef48b62640 1 -- 172.21.15.40:0/2884408128 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef2c003520 con 0x7fef440d0070 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.618+0000 7fef4a365640 1 -- 172.21.15.40:0/2884408128 >> v1:172.21.15.184:6789/0 conn(0x7fef4406e270 legacy=0x7fef440ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.618+0000 7fef4a365640 1 -- 172.21.15.40:0/2884408128 >> v1:172.21.15.40:6789/0 conn(0x7fef440d0070 legacy=0x7fef4406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.618+0000 7fef4a365640 1 -- 172.21.15.40:0/2884408128 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef440cd990 con 0x7fef4406d560 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.618+0000 7fef4a365640 1 --2- 172.21.15.40:0/2884408128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406d560 0x7fef440d1d40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0097a0 tx=0x7fef3c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8513e6cba71514c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef48b62640 1 -- 172.21.15.40:0/2884408128 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c037070 con 0x7fef4406d560 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef48b62640 1 -- 172.21.15.40:0/2884408128 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3c032440 con 0x7fef4406d560 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef48b62640 1 -- 172.21.15.40:0/2884408128 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c0327f0 con 0x7fef4406d560 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef4bdc8640 1 -- 172.21.15.40:0/2884408128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406d560 msgr2=0x7fef440d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef4bdc8640 1 --2- 172.21.15.40:0/2884408128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406d560 0x7fef440d1d40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0097a0 tx=0x7fef3c02a230 comp rx=0 tx=0).stop 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef4bdc8640 1 -- 172.21.15.40:0/2884408128 shutdown_connections 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef4bdc8640 1 --2- 172.21.15.40:0/2884408128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406d560 0x7fef440d1d40 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef4bdc8640 1 -- 172.21.15.40:0/2884408128 >> 172.21.15.40:0/2884408128 conn(0x7fef440d2ec0 msgr2=0x7fef440d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef4bdc8640 1 -- 172.21.15.40:0/2884408128 shutdown_connections 2024-04-26T21:41:49.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.619+0000 7fef4bdc8640 1 -- 172.21.15.40:0/2884408128 wait complete. 2024-04-26T21:41:49.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.620+0000 7fef4bdc8640 1 Processor -- start 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef4bdc8640 1 -- start start 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef4bdc8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4406d560 0x7fef4409dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef4bdc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406e270 0x7fef4414f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef4bdc8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef440d0070 0x7fef440701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef4bdc8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef440ce8a0 con 0x7fef4406e270 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef4bdc8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef440d2d10 con 0x7fef440d0070 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef4bdc8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef440cea20 con 0x7fef4406d560 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef49363640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406e270 0x7fef4414f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef49363640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406e270 0x7fef4414f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56468/0 (socket says 172.21.15.40:56468) 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.621+0000 7fef49363640 1 -- 172.21.15.40:0/1262192314 learned_addr learned my addr 172.21.15.40:0/1262192314 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef4a365640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef440d0070 0x7fef440701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef49b64640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4406d560 0x7fef4409dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef49363640 1 -- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4406d560 msgr2=0x7fef4409dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef49363640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4406d560 0x7fef4409dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef49363640 1 -- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef440d0070 msgr2=0x7fef440701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef49363640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef440d0070 0x7fef440701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef49363640 1 -- 172.21.15.40:0/1262192314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef440c81f0 con 0x7fef4406e270 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef4a365640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef440d0070 0x7fef440701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef49363640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406e270 0x7fef4414f2b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fef3800c400 tx=0x7fef3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef427fc640 1 -- 172.21.15.40:0/1262192314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef380052d0 con 0x7fef4406e270 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef427fc640 1 -- 172.21.15.40:0/1262192314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef38005490 con 0x7fef4406e270 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef441f88c0 con 0x7fef4406e270 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef427fc640 1 -- 172.21.15.40:0/1262192314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38007260 con 0x7fef4406e270 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.622+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4414f7f0 con 0x7fef4406e270 2024-04-26T21:41:49.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.623+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef440694f0 con 0x7fef4406e270 2024-04-26T21:41:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.624+0000 7fef427fc640 1 -- 172.21.15.40:0/1262192314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef38007400 con 0x7fef4406e270 2024-04-26T21:41:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.624+0000 7fef427fc640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef3006c920 0x7fef3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.624+0000 7fef427fc640 1 -- 172.21.15.40:0/1262192314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7fef3808b1b0 con 0x7fef4406e270 2024-04-26T21:41:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.624+0000 7fef49b64640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef3006c920 0x7fef3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.625+0000 7fef49b64640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef3006c920 0x7fef3006ede0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fef34001560 tx=0x7fef34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.628+0000 7fef427fc640 1 -- 172.21.15.40:0/1262192314 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef38059490 con 0x7fef4406e270 2024-04-26T21:41:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.814+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef440d27a0 con 0x7fef4406e270 2024-04-26T21:41:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.816+0000 7fef427fc640 1 -- 172.21.15.40:0/1262192314 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+17038 (secure 0 0 0) 0x7fef38058d90 con 0x7fef4406e270 2024-04-26T21:41:49.816 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:49.816 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:39.656336+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:41:05.508768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-26T21:41:06.198815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3115383225":"2024-04-27T21:41:39.656262+0000","172.21.15.40:6834/3115383225":"2024-04-27T21:41:39.656262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.817+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef3006c920 msgr2=0x7fef3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.817+0000 7fef4bdc8640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef3006c920 0x7fef3006ede0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fef34001560 tx=0x7fef34009040 comp rx=0 tx=0).stop 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.817+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406e270 msgr2=0x7fef4414f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.817+0000 7fef4bdc8640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406e270 0x7fef4414f2b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fef3800c400 tx=0x7fef3800c8c0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.818+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 shutdown_connections 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.818+0000 7fef4bdc8640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fef3006c920 0x7fef3006ede0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.818+0000 7fef4bdc8640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef440d0070 0x7fef440701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.818+0000 7fef4bdc8640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef4406e270 0x7fef4414f2b0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.818+0000 7fef4bdc8640 1 --2- 172.21.15.40:0/1262192314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4406d560 0x7fef4409dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.818+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 >> 172.21.15.40:0/1262192314 conn(0x7fef440d2ec0 msgr2=0x7fef44071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.818+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 shutdown_connections 2024-04-26T21:41:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.818+0000 7fef4bdc8640 1 -- 172.21.15.40:0/1262192314 wait complete. 2024-04-26T21:41:49.829 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:41:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.930+0000 7f89b1538640 1 Processor -- start 2024-04-26T21:41:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.930+0000 7f89b1538640 1 -- start start 2024-04-26T21:41:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.931+0000 7f89b1538640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89ac1608f0 0x7f89ac0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.931+0000 7f89b1538640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0d0a90 0x7f89ac0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.931+0000 7f89b1538640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ac0cde20 con 0x7f89ac0cfdd0 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.931+0000 7f89b1538640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f89ac0cea30 con 0x7f89ac1608f0 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.931+0000 7f89b1538640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89ac0ce100 con 0x7f89ac0d0a90 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.931+0000 7f89aaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89ac1608f0 0x7f89ac0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.931+0000 7f89ab7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f89ac0cfdd0 0x7f89ac160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43202/0 (socket says 172.21.15.40:43202) 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.931+0000 7f89ab7fe640 1 -- 172.21.15.40:0/695881010 learned_addr learned my addr 172.21.15.40:0/695881010 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.932+0000 7f89aa7fc640 1 --2- 172.21.15.40:0/695881010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0d0a90 0x7f89ac0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.932+0000 7f89aaffd640 1 -- 172.21.15.40:0/695881010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0d0a90 msgr2=0x7f89ac0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.932+0000 7f89aaffd640 1 --2- 172.21.15.40:0/695881010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0d0a90 0x7f89ac0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.932+0000 7f89aaffd640 1 -- 172.21.15.40:0/695881010 >> v1:172.21.15.40:6789/0 conn(0x7f89ac0cfdd0 legacy=0x7f89ac160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.932+0000 7f89aaffd640 1 -- 172.21.15.40:0/695881010 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac0ce3e0 con 0x7f89ac1608f0 2024-04-26T21:41:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.932+0000 7f89aa7fc640 1 --2- 172.21.15.40:0/695881010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0d0a90 0x7f89ac0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.932+0000 7f89aaffd640 1 --2- 172.21.15.40:0/695881010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac1608f0 0x7f89ac0d1bf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f899c0097a0 tx=0x7f899c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c550bd815ca71d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89a9ffb640 1 -- 172.21.15.40:0/695881010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c037070 con 0x7f89ac1608f0 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89a9ffb640 1 -- 172.21.15.40:0/695881010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899c032440 con 0x7f89ac1608f0 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89a9ffb640 1 -- 172.21.15.40:0/695881010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c032810 con 0x7f89ac1608f0 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89b1538640 1 -- 172.21.15.40:0/695881010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac1608f0 msgr2=0x7f89ac0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89b1538640 1 --2- 172.21.15.40:0/695881010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac1608f0 0x7f89ac0d1bf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f899c0097a0 tx=0x7f899c02a180 comp rx=0 tx=0).stop 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89b1538640 1 -- 172.21.15.40:0/695881010 shutdown_connections 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89b1538640 1 --2- 172.21.15.40:0/695881010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0d0a90 0x7f89ac0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89b1538640 1 --2- 172.21.15.40:0/695881010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac1608f0 0x7f89ac0d1bf0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89b1538640 1 -- 172.21.15.40:0/695881010 >> 172.21.15.40:0/695881010 conn(0x7f89ac0d2ec0 msgr2=0x7f89ac0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89b1538640 1 -- 172.21.15.40:0/695881010 shutdown_connections 2024-04-26T21:41:49.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.933+0000 7f89b1538640 1 -- 172.21.15.40:0/695881010 wait complete. 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.934+0000 7f89b1538640 1 Processor -- start 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.934+0000 7f89b1538640 1 -- start start 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89b1538640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0cfdd0 0x7f89ac15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89b1538640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac0d0a90 0x7f89ac0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89b1538640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ac1608f0 0x7f89ac0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89b1538640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac0b3310 con 0x7f89ac0d0a90 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89b1538640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac0d0ef0 con 0x7f89ac1608f0 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89b1538640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89ac0ceec0 con 0x7f89ac0cfdd0 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac0d0a90 0x7f89ac0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac0d0a90 0x7f89ac0c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56472/0 (socket says 172.21.15.40:56472) 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.935+0000 7f89aa7fc640 1 -- 172.21.15.40:0/2607908274 learned_addr learned my addr 172.21.15.40:0/2607908274 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89aa7fc640 1 -- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0cfdd0 msgr2=0x7f89ac15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89aaffd640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0cfdd0 0x7f89ac15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89ab7fe640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ac1608f0 0x7f89ac0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89aa7fc640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0cfdd0 0x7f89ac15c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89aa7fc640 1 -- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ac1608f0 msgr2=0x7f89ac0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89aa7fc640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ac1608f0 0x7f89ac0c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89aa7fc640 1 -- 172.21.15.40:0/2607908274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac0c79f0 con 0x7f89ac0d0a90 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89aa7fc640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac0d0a90 0x7f89ac0c87b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f89a000c7f0 tx=0x7f89a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89937fe640 1 -- 172.21.15.40:0/2607908274 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0015070 con 0x7f89ac0d0a90 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89937fe640 1 -- 172.21.15.40:0/2607908274 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a0007820 con 0x7f89ac0d0a90 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ac0c7cd0 con 0x7f89ac0d0a90 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89937fe640 1 -- 172.21.15.40:0/2607908274 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0007bd0 con 0x7f89ac0d0a90 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.936+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ac15ce00 con 0x7f89ac0d0a90 2024-04-26T21:41:49.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.937+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ac0694f0 con 0x7f89ac0d0a90 2024-04-26T21:41:49.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.937+0000 7f89937fe640 1 -- 172.21.15.40:0/2607908274 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89a0018070 con 0x7f89ac0d0a90 2024-04-26T21:41:49.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.938+0000 7f89937fe640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f899806c8f0 0x7f899806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:49.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.938+0000 7f89937fe640 1 -- 172.21.15.40:0/2607908274 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f89a008bea0 con 0x7f89ac0d0a90 2024-04-26T21:41:49.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.938+0000 7f89aaffd640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f899806c8f0 0x7f899806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:49.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.939+0000 7f89aaffd640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f899806c8f0 0x7f899806edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f899c034000 tx=0x7f899c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:49.942+0000 7f89937fe640 1 -- 172.21.15.40:0/2607908274 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89a005a0f0 con 0x7f89ac0d0a90 2024-04-26T21:41:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.166+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f89ac0b8d20 con 0x7f89ac0d0a90 2024-04-26T21:41:50.166 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:41:50.167+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:41:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.168+0000 7f89937fe640 1 -- 172.21.15.40:0/2607908274 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) ==== 161+0+0 (secure 0 0 0) 0x7f89a00599f0 con 0x7f89ac0d0a90 2024-04-26T21:41:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:41:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f899806c8f0 msgr2=0x7f899806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f899806c8f0 0x7f899806edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f899c034000 tx=0x7f899c02e000 comp rx=0 tx=0).stop 2024-04-26T21:41:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac0d0a90 msgr2=0x7f89ac0c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac0d0a90 0x7f89ac0c87b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f89a000c7f0 tx=0x7f89a000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 shutdown_connections 2024-04-26T21:41:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f899806c8f0 0x7f899806edb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ac1608f0 0x7f89ac0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ac0d0a90 0x7f89ac0c87b0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 --2- 172.21.15.40:0/2607908274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ac0cfdd0 0x7f89ac15c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 >> 172.21.15.40:0/2607908274 conn(0x7f89ac0d2ec0 msgr2=0x7f89ac071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 shutdown_connections 2024-04-26T21:41:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.171+0000 7f89b1538640 1 -- 172.21.15.40:0/2607908274 wait complete. 2024-04-26T21:41:50.174 INFO:tasks.ceph.mds.a.smithi040.stderr: -1368> 2024-04-26T21:41:43.854+0000 7f171689a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:41:50.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:41:50.201 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:41:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.287+0000 7fa2a630a640 1 Processor -- start 2024-04-26T21:41:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.287+0000 7fa2a630a640 1 -- start start 2024-04-26T21:41:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa2a630a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2a006fc40 0x7fa2a0073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa2a630a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a0156840 con 0x7fa2a0156460 2024-04-26T21:41:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa2a630a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a00cbf20 con 0x7fa2a006f790 2024-04-26T21:41:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa2a630a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a00cc200 con 0x7fa2a006fc40 2024-04-26T21:41:50.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa2a48a7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2a006fc40 0x7fa2a0073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:50.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa29ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa2a0156460 0x7fa2a00cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43224/0 (socket says 172.21.15.40:43224) 2024-04-26T21:41:50.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa29ffff640 1 -- 172.21.15.40:0/1370190265 learned_addr learned my addr 172.21.15.40:0/1370190265 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:50.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa2a48a7640 1 -- 172.21.15.40:0/1370190265 >> v1:172.21.15.184:6789/0 conn(0x7fa2a006f790 legacy=0x7fa2a0072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:50.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa2a48a7640 1 -- 172.21.15.40:0/1370190265 >> v1:172.21.15.40:6789/0 conn(0x7fa2a0156460 legacy=0x7fa2a00cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:50.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa2a48a7640 1 -- 172.21.15.40:0/1370190265 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a00cc4e0 con 0x7fa2a006fc40 2024-04-26T21:41:50.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.288+0000 7fa29effd640 1 -- 172.21.15.40:0/1370190265 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4280274174 0 0) 0x7fa2a0156840 con 0x7fa2a0156460 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.289+0000 7fa2a48a7640 1 --2- 172.21.15.40:0/1370190265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a0073360 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa298009d90 tx=0x7fa29802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7a7ffe2aeb0997 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.289+0000 7fa29effd640 1 -- 172.21.15.40:0/1370190265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298007740 con 0x7fa2a006fc40 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.289+0000 7fa29effd640 1 -- 172.21.15.40:0/1370190265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2980078e0 con 0x7fa2a006fc40 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.289+0000 7fa29effd640 1 -- 172.21.15.40:0/1370190265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298007cb0 con 0x7fa2a006fc40 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.290+0000 7fa2a630a640 1 -- 172.21.15.40:0/1370190265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 msgr2=0x7fa2a0073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.290+0000 7fa2a630a640 1 --2- 172.21.15.40:0/1370190265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a0073360 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa298009d90 tx=0x7fa29802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.290+0000 7fa2a630a640 1 -- 172.21.15.40:0/1370190265 shutdown_connections 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.290+0000 7fa2a630a640 1 --2- 172.21.15.40:0/1370190265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a0073360 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.290+0000 7fa2a630a640 1 -- 172.21.15.40:0/1370190265 >> 172.21.15.40:0/1370190265 conn(0x7fa2a00d0be0 msgr2=0x7fa2a00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.290+0000 7fa2a630a640 1 -- 172.21.15.40:0/1370190265 shutdown_connections 2024-04-26T21:41:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.290+0000 7fa2a630a640 1 -- 172.21.15.40:0/1370190265 wait complete. 2024-04-26T21:41:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.291+0000 7fa2a630a640 1 Processor -- start 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.291+0000 7fa2a630a640 1 -- start start 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa2a630a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a006f790 0x7fa2a0151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa2a630a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a00c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa2a630a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a0156460 0x7fa2a0155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa2a630a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0070050 con 0x7fa2a006fc40 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa2a630a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0072700 con 0x7fa2a006f790 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa2a630a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a00cbbe0 con 0x7fa2a0156460 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa29f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a00c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa29f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a00c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56506/0 (socket says 172.21.15.40:56506) 2024-04-26T21:41:50.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa29f7fe640 1 -- 172.21.15.40:0/2309293214 learned_addr learned my addr 172.21.15.40:0/2309293214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa2a48a7640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a0156460 0x7fa2a0155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.292+0000 7fa29ffff640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a006f790 0x7fa2a0151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29f7fe640 1 -- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a0156460 msgr2=0x7fa2a0155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29f7fe640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a0156460 0x7fa2a0155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29f7fe640 1 -- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a006f790 msgr2=0x7fa2a0151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29f7fe640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a006f790 0x7fa2a0151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29f7fe640 1 -- 172.21.15.40:0/2309293214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a00c9ca0 con 0x7fa2a006fc40 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29f7fe640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a00c5760 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa28800c430 tx=0x7fa28800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29cff9640 1 -- 172.21.15.40:0/2309293214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288011790 con 0x7fa2a006fc40 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29cff9640 1 -- 172.21.15.40:0/2309293214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa288017070 con 0x7fa2a006fc40 2024-04-26T21:41:50.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a00c9f20 con 0x7fa2a006fc40 2024-04-26T21:41:50.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa29cff9640 1 -- 172.21.15.40:0/2309293214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288011bb0 con 0x7fa2a006fc40 2024-04-26T21:41:50.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.293+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a00ca4e0 con 0x7fa2a006fc40 2024-04-26T21:41:50.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.294+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a00694f0 con 0x7fa2a006fc40 2024-04-26T21:41:50.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.294+0000 7fa29cff9640 1 -- 172.21.15.40:0/2309293214 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa28801c460 con 0x7fa2a006fc40 2024-04-26T21:41:50.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.295+0000 7fa29cff9640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28006c8f0 0x7fa28006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:50.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.295+0000 7fa29cff9640 1 -- 172.21.15.40:0/2309293214 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6273+0+0 (secure 0 0 0) 0x7fa28808bc30 con 0x7fa2a006fc40 2024-04-26T21:41:50.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.295+0000 7fa29ffff640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28006c8f0 0x7fa28006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:50.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.296+0000 7fa29ffff640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28006c8f0 0x7fa28006edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa290000f60 tx=0x7fa290007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:50.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.299+0000 7fa29cff9640 1 -- 172.21.15.40:0/2309293214 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2a00c9ca0 con 0x7fa2a006fc40 2024-04-26T21:41:50.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:50.522+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa2a006de80 con 0x7fa2a006fc40 2024-04-26T21:41:51.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.170+0000 7fa29cff9640 1 -- 172.21.15.40:0/2309293214 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) ==== 104+0+0 (secure 0 0 0) 0x7fa288059e20 con 0x7fa2a006fc40 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.172+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28006c8f0 msgr2=0x7fa28006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.172+0000 7fa2a630a640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28006c8f0 0x7fa28006edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa290000f60 tx=0x7fa290007c00 comp rx=0 tx=0).stop 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.172+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 msgr2=0x7fa2a00c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.172+0000 7fa2a630a640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a00c5760 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa28800c430 tx=0x7fa28800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.173+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 shutdown_connections 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.173+0000 7fa2a630a640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28006c8f0 0x7fa28006edb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.173+0000 7fa2a630a640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a0156460 0x7fa2a0155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.173+0000 7fa2a630a640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a006fc40 0x7fa2a00c5760 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.173+0000 7fa2a630a640 1 --2- 172.21.15.40:0/2309293214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a006f790 0x7fa2a0151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.173+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 >> 172.21.15.40:0/2309293214 conn(0x7fa2a00d0be0 msgr2=0x7fa2a00b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.173+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 shutdown_connections 2024-04-26T21:41:51.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.173+0000 7fa2a630a640 1 -- 172.21.15.40:0/2309293214 wait complete. 2024-04-26T21:41:51.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:41:51.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.340+0000 7fb7fa293640 1 Processor -- start 2024-04-26T21:41:51.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.340+0000 7fb7fa293640 1 -- start start 2024-04-26T21:41:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7fa293640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7f41629a0 0x7fb7f40b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7fa293640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7f40d1950 0x7fb7f40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7fa293640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f4162d80 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7fa293640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f40d5f30 con 0x7fb7f41629a0 2024-04-26T21:41:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7fa293640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f40d39d0 con 0x7fb7f40d1950 2024-04-26T21:41:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7f3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7f41629a0 0x7fb7f40b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7f3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7f41629a0 0x7fb7f40b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56508/0 (socket says 172.21.15.40:56508) 2024-04-26T21:41:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7f3fff640 1 -- 172.21.15.40:0/370081877 learned_addr learned my addr 172.21.15.40:0/370081877 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7f3fff640 1 -- 172.21.15.40:0/370081877 >> v2:172.21.15.184:3300/0 conn(0x7fb7f40d1950 msgr2=0x7fb7f40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7f3fff640 1 --2- 172.21.15.40:0/370081877 >> v2:172.21.15.184:3300/0 conn(0x7fb7f40d1950 0x7fb7f40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7f3fff640 1 -- 172.21.15.40:0/370081877 >> v1:172.21.15.184:6789/0 conn(0x7fb7f40b5fd0 legacy=0x7fb7f40d5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.341+0000 7fb7f3fff640 1 -- 172.21.15.40:0/370081877 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f40d3cb0 con 0x7fb7f41629a0 2024-04-26T21:41:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.342+0000 7fb7f3fff640 1 --2- 172.21.15.40:0/370081877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f41629a0 0x7fb7f40b4890 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb7f408f040 tx=0x7fb7dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f678e0ad19ed47f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.342+0000 7fb7f2ffd640 1 -- 172.21.15.40:0/370081877 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc037070 con 0x7fb7f41629a0 2024-04-26T21:41:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.342+0000 7fb7f2ffd640 1 -- 172.21.15.40:0/370081877 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc032440 con 0x7fb7f41629a0 2024-04-26T21:41:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.342+0000 7fb7f2ffd640 1 -- 172.21.15.40:0/370081877 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc032810 con 0x7fb7f41629a0 2024-04-26T21:41:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.343+0000 7fb7fa293640 1 -- 172.21.15.40:0/370081877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f41629a0 msgr2=0x7fb7f40b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.343+0000 7fb7fa293640 1 --2- 172.21.15.40:0/370081877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f41629a0 0x7fb7f40b4890 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb7f408f040 tx=0x7fb7dc02a230 comp rx=0 tx=0).stop 2024-04-26T21:41:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.343+0000 7fb7fa293640 1 -- 172.21.15.40:0/370081877 shutdown_connections 2024-04-26T21:41:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.343+0000 7fb7fa293640 1 --2- 172.21.15.40:0/370081877 >> v2:172.21.15.184:3300/0 conn(0x7fb7f40d1950 0x7fb7f40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.343+0000 7fb7fa293640 1 --2- 172.21.15.40:0/370081877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f41629a0 0x7fb7f40b4890 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.343+0000 7fb7fa293640 1 -- 172.21.15.40:0/370081877 >> 172.21.15.40:0/370081877 conn(0x7fb7f409e3e0 msgr2=0x7fb7f40d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.343+0000 7fb7fa293640 1 -- 172.21.15.40:0/370081877 shutdown_connections 2024-04-26T21:41:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.343+0000 7fb7fa293640 1 -- 172.21.15.40:0/370081877 wait complete. 2024-04-26T21:41:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.344+0000 7fb7fa293640 1 Processor -- start 2024-04-26T21:41:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.344+0000 7fb7fa293640 1 -- start start 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7fa293640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f40b5fd0 0x7fb7f4155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7fa293640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f40d1950 0x7fb7f40cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7fa293640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f41629a0 0x7fb7f40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7fa293640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f40d3f90 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7fa293640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f40d36a0 con 0x7fb7f40d1950 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7fa293640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f40d02c0 con 0x7fb7f41629a0 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f40b5fd0 0x7fb7f4155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f40b5fd0 0x7fb7f4155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56518/0 (socket says 172.21.15.40:56518) 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f3fff640 1 -- 172.21.15.40:0/1487847488 learned_addr learned my addr 172.21.15.40:0/1487847488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f8830640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f41629a0 0x7fb7f40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f37fe640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f40d1950 0x7fb7f40cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f3fff640 1 -- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f41629a0 msgr2=0x7fb7f40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f3fff640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f41629a0 0x7fb7f40cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f3fff640 1 -- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f40d1950 msgr2=0x7fb7f40cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f3fff640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f40d1950 0x7fb7f40cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.345+0000 7fb7f3fff640 1 -- 172.21.15.40:0/1487847488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f4162190 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.346+0000 7fb7f3fff640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f40b5fd0 0x7fb7f4155480 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc02a270 tx=0x7fb7dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.346+0000 7fb7f0ff9640 1 -- 172.21.15.40:0/1487847488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc041070 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.346+0000 7fb7f0ff9640 1 -- 172.21.15.40:0/1487847488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc03c870 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.346+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f4161bd0 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.346+0000 7fb7f0ff9640 1 -- 172.21.15.40:0/1487847488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc037040 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.346+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f40ca590 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.347+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f4069600 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.347+0000 7fb7f0ff9640 1 -- 172.21.15.40:0/1487847488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7dc044070 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.348+0000 7fb7f0ff9640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d406c8f0 0x7fb7d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.348+0000 7fb7f0ff9640 1 -- 172.21.15.40:0/1487847488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6273+0+0 (secure 0 0 0) 0x7fb7dc0aca20 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.348+0000 7fb7f37fe640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d406c8f0 0x7fb7d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.349+0000 7fb7f37fe640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d406c8f0 0x7fb7d406edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4007fd0 tx=0x7fb7e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:51.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.352+0000 7fb7f0ff9640 1 -- 172.21.15.40:0/1487847488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb7dc07adc0 con 0x7fb7f40b5fd0 2024-04-26T21:41:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:51.541+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb7f40d26c0 con 0x7fb7f40b5fd0 2024-04-26T21:41:52.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.173+0000 7fb7f0ff9640 1 -- 172.21.15.40:0/1487847488 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) ==== 181+0+0 (secure 0 0 0) 0x7fb7dc07a6c0 con 0x7fb7f40b5fd0 2024-04-26T21:41:52.174 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:41:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d406c8f0 msgr2=0x7fb7d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d406c8f0 0x7fb7d406edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4007fd0 tx=0x7fb7e4009040 comp rx=0 tx=0).stop 2024-04-26T21:41:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f40b5fd0 msgr2=0x7fb7f4155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f40b5fd0 0x7fb7f4155480 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc02a270 tx=0x7fb7dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 shutdown_connections 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7d406c8f0 0x7fb7d406edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f41629a0 0x7fb7f40cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f40d1950 0x7fb7f40cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 --2- 172.21.15.40:0/1487847488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f40b5fd0 0x7fb7f4155480 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 >> 172.21.15.40:0/1487847488 conn(0x7fb7f409e3e0 msgr2=0x7fb7f40b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.176+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 shutdown_connections 2024-04-26T21:41:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.177+0000 7fb7fa293640 1 -- 172.21.15.40:0/1487847488 wait complete. 2024-04-26T21:41:52.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:41:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.301+0000 7fb8adb6a640 1 Processor -- start 2024-04-26T21:41:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.301+0000 7fb8adb6a640 1 -- start start 2024-04-26T21:41:52.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8adb6a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8a815f460 0x7fb8a80d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:52.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8adb6a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8a80d2150 0x7fb8a80d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:52.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8adb6a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a815f840 con 0x7fb8a80d0760 2024-04-26T21:41:52.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8adb6a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a80d41c0 con 0x7fb8a815f460 2024-04-26T21:41:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8adb6a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a80d37f0 con 0x7fb8a80d2150 2024-04-26T21:41:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8a77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8a815f460 0x7fb8a80d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8a77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8a815f460 0x7fb8a80d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56526/0 (socket says 172.21.15.40:56526) 2024-04-26T21:41:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8a77fe640 1 -- 172.21.15.40:0/2771925933 learned_addr learned my addr 172.21.15.40:0/2771925933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8a77fe640 1 -- 172.21.15.40:0/2771925933 >> v2:172.21.15.184:3300/0 conn(0x7fb8a80d2150 msgr2=0x7fb8a80d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8a77fe640 1 --2- 172.21.15.40:0/2771925933 >> v2:172.21.15.184:3300/0 conn(0x7fb8a80d2150 0x7fb8a80d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8a77fe640 1 -- 172.21.15.40:0/2771925933 >> v1:172.21.15.184:6789/0 conn(0x7fb8a80d0760 legacy=0x7fb8a80d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.302+0000 7fb8a77fe640 1 -- 172.21.15.40:0/2771925933 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a80d2ec0 con 0x7fb8a815f460 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.303+0000 7fb8a77fe640 1 --2- 172.21.15.40:0/2771925933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a815f460 0x7fb8a80d3c80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb8a808ef50 tx=0x7fb89802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99f961bb1f431a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.303+0000 7fb8a67fc640 1 -- 172.21.15.40:0/2771925933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898037070 con 0x7fb8a815f460 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.303+0000 7fb8a67fc640 1 -- 172.21.15.40:0/2771925933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb898032440 con 0x7fb8a815f460 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.303+0000 7fb8a67fc640 1 -- 172.21.15.40:0/2771925933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898032810 con 0x7fb8a815f460 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.304+0000 7fb8adb6a640 1 -- 172.21.15.40:0/2771925933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a815f460 msgr2=0x7fb8a80d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.304+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/2771925933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a815f460 0x7fb8a80d3c80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb8a808ef50 tx=0x7fb89802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.304+0000 7fb8adb6a640 1 -- 172.21.15.40:0/2771925933 shutdown_connections 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.304+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/2771925933 >> v2:172.21.15.184:3300/0 conn(0x7fb8a80d2150 0x7fb8a80d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.304+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/2771925933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a815f460 0x7fb8a80d3c80 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.304+0000 7fb8adb6a640 1 -- 172.21.15.40:0/2771925933 >> 172.21.15.40:0/2771925933 conn(0x7fb8a809e3f0 msgr2=0x7fb8a80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:52.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.304+0000 7fb8adb6a640 1 -- 172.21.15.40:0/2771925933 shutdown_connections 2024-04-26T21:41:52.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.304+0000 7fb8adb6a640 1 -- 172.21.15.40:0/2771925933 wait complete. 2024-04-26T21:41:52.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.305+0000 7fb8adb6a640 1 Processor -- start 2024-04-26T21:41:52.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.305+0000 7fb8adb6a640 1 -- start start 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8adb6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a80d0760 0x7fb8a80b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8adb6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a80d2150 0x7fb8a80b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8adb6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a815f460 0x7fb8a80b8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8adb6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a80d5f30 con 0x7fb8a80d2150 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8adb6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a80d3480 con 0x7fb8a80d0760 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8adb6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a806ce20 con 0x7fb8a815f460 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a80d2150 0x7fb8a80b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a80d2150 0x7fb8a80b7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56532/0 (socket says 172.21.15.40:56532) 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 -- 172.21.15.40:0/320361325 learned_addr learned my addr 172.21.15.40:0/320361325 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a77fe640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a80d0760 0x7fb8a80b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a7fff640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a815f460 0x7fb8a80b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 -- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a815f460 msgr2=0x7fb8a80b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a815f460 0x7fb8a80b8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 -- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a80d0760 msgr2=0x7fb8a80b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a80d0760 0x7fb8a80b9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 -- 172.21.15.40:0/320361325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a80ba1f0 con 0x7fb8a80d2150 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.306+0000 7fb8a6ffd640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a80d2150 0x7fb8a80b7d00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00c410 tx=0x7fb89c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.307+0000 7fb88ffff640 1 -- 172.21.15.40:0/320361325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c017070 con 0x7fb8a80d2150 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.307+0000 7fb88ffff640 1 -- 172.21.15.40:0/320361325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89c0035f0 con 0x7fb8a80d2150 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.307+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a80ba470 con 0x7fb8a80d2150 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.307+0000 7fb88ffff640 1 -- 172.21.15.40:0/320361325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c003a20 con 0x7fb8a80d2150 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.307+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a80baa30 con 0x7fb8a80d2150 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.308+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a8069600 con 0x7fb8a80d2150 2024-04-26T21:41:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.308+0000 7fb88ffff640 1 -- 172.21.15.40:0/320361325 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb89c01a070 con 0x7fb8a80d2150 2024-04-26T21:41:52.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.309+0000 7fb88ffff640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb89406c8f0 0x7fb89406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:52.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.309+0000 7fb88ffff640 1 -- 172.21.15.40:0/320361325 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5835+0+0 (secure 0 0 0) 0x7fb89c08b0e0 con 0x7fb8a80d2150 2024-04-26T21:41:52.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.309+0000 7fb8a77fe640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb89406c8f0 0x7fb89406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:52.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.309+0000 7fb8a77fe640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb89406c8f0 0x7fb89406edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb898034000 tx=0x7fb89802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:52.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.313+0000 7fb88ffff640 1 -- 172.21.15.40:0/320361325 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8a80ba1f0 con 0x7fb8a80d2150 2024-04-26T21:41:52.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:52.498+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb8a80d5af0 con 0x7fb8a80d2150 2024-04-26T21:41:53.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.180+0000 7fb88ffff640 1 -- 172.21.15.40:0/320361325 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) ==== 169+0+0 (secure 0 0 0) 0x7fb89c059490 con 0x7fb8a80d2150 2024-04-26T21:41:53.180 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:41:53.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb89406c8f0 msgr2=0x7fb89406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb89406c8f0 0x7fb89406edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb898034000 tx=0x7fb89802e000 comp rx=0 tx=0).stop 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a80d2150 msgr2=0x7fb8a80b7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a80d2150 0x7fb8a80b7d00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00c410 tx=0x7fb89c00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 shutdown_connections 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb89406c8f0 0x7fb89406edb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a815f460 0x7fb8a80b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a80d2150 0x7fb8a80b7d00 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 --2- 172.21.15.40:0/320361325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a80d0760 0x7fb8a80b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 >> 172.21.15.40:0/320361325 conn(0x7fb8a809e3f0 msgr2=0x7fb8a80b64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 shutdown_connections 2024-04-26T21:41:53.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.182+0000 7fb8adb6a640 1 -- 172.21.15.40:0/320361325 wait complete. 2024-04-26T21:41:53.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:41:53.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.308+0000 7fab62c47640 1 Processor -- start 2024-04-26T21:41:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.308+0000 7fab62c47640 1 -- start start 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab62c47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c162880 0x7fab5c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab62c47640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab5c0b5f20 0x7fab5c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab62c47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5c0b7e50 con 0x7fab5c0d39f0 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab62c47640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fab5c0d0580 con 0x7fab5c0b5f20 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab62c47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c0d1800 con 0x7fab5c162880 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab5bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab5c0b5f20 0x7fab5c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab5bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab5c0b5f20 0x7fab5c0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56544/0 (socket says 172.21.15.40:56544) 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab5bfff640 1 -- 172.21.15.40:0/83073994 learned_addr learned my addr 172.21.15.40:0/83073994 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab5bfff640 1 -- 172.21.15.40:0/83073994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c162880 msgr2=0x7fab5c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab609e3640 1 --2- 172.21.15.40:0/83073994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c162880 0x7fab5c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab5bfff640 1 --2- 172.21.15.40:0/83073994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c162880 0x7fab5c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab5bfff640 1 -- 172.21.15.40:0/83073994 >> v1:172.21.15.184:6789/0 conn(0x7fab5c0d39f0 legacy=0x7fab5c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab5bfff640 1 -- 172.21.15.40:0/83073994 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c0d07d0 con 0x7fab5c0b5f20 2024-04-26T21:41:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.309+0000 7fab609e3640 1 --2- 172.21.15.40:0/83073994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c162880 0x7fab5c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.310+0000 7fab5bfff640 1 --2- 172.21.15.40:0/83073994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c0b5f20 0x7fab5c0d34b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009870 tx=0x7fab4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b612719214207a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.310+0000 7fab5b7fe640 1 -- 172.21.15.40:0/83073994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c037070 con 0x7fab5c0b5f20 2024-04-26T21:41:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.310+0000 7fab5b7fe640 1 -- 172.21.15.40:0/83073994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c032440 con 0x7fab5c0b5f20 2024-04-26T21:41:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.310+0000 7fab5b7fe640 1 -- 172.21.15.40:0/83073994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c032810 con 0x7fab5c0b5f20 2024-04-26T21:41:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.311+0000 7fab62c47640 1 -- 172.21.15.40:0/83073994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c0b5f20 msgr2=0x7fab5c0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.311+0000 7fab62c47640 1 --2- 172.21.15.40:0/83073994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c0b5f20 0x7fab5c0d34b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009870 tx=0x7fab4c02a270 comp rx=0 tx=0).stop 2024-04-26T21:41:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.311+0000 7fab62c47640 1 -- 172.21.15.40:0/83073994 shutdown_connections 2024-04-26T21:41:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.311+0000 7fab62c47640 1 --2- 172.21.15.40:0/83073994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c0b5f20 0x7fab5c0d34b0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.311+0000 7fab62c47640 1 --2- 172.21.15.40:0/83073994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c162880 0x7fab5c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.311+0000 7fab62c47640 1 -- 172.21.15.40:0/83073994 >> 172.21.15.40:0/83073994 conn(0x7fab5c0d4f60 msgr2=0x7fab5c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.311+0000 7fab62c47640 1 -- 172.21.15.40:0/83073994 shutdown_connections 2024-04-26T21:41:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.311+0000 7fab62c47640 1 -- 172.21.15.40:0/83073994 wait complete. 2024-04-26T21:41:53.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.312+0000 7fab62c47640 1 Processor -- start 2024-04-26T21:41:53.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.312+0000 7fab62c47640 1 -- start start 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab62c47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0b5f20 0x7fab5c15fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab62c47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d39f0 0x7fab5c15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab62c47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c162880 0x7fab5c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab62c47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c162c60 con 0x7fab5c162880 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab62c47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c0d0d90 con 0x7fab5c0b5f20 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab62c47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c162de0 con 0x7fab5c0d39f0 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c162880 0x7fab5c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c162880 0x7fab5c15c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56550/0 (socket says 172.21.15.40:56550) 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 -- 172.21.15.40:0/481834451 learned_addr learned my addr 172.21.15.40:0/481834451 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:53.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab5bfff640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d39f0 0x7fab5c15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab609e3640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0b5f20 0x7fab5c15fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 -- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d39f0 msgr2=0x7fab5c15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d39f0 0x7fab5c15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 -- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0b5f20 msgr2=0x7fab5c15fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0b5f20 0x7fab5c15fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 -- 172.21.15.40:0/481834451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c0b5aa0 con 0x7fab5c162880 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab5bfff640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d39f0 0x7fab5c15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.313+0000 7fab611e4640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c162880 0x7fab5c15c8b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c7a0 tx=0x7fab5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.314+0000 7fab597fa640 1 -- 172.21.15.40:0/481834451 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54003b70 con 0x7fab5c162880 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.314+0000 7fab597fa640 1 -- 172.21.15.40:0/481834451 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54003d30 con 0x7fab5c162880 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.314+0000 7fab597fa640 1 -- 172.21.15.40:0/481834451 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5400aa30 con 0x7fab5c162880 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.314+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c0b4ce0 con 0x7fab5c162880 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.314+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c0b52a0 con 0x7fab5c162880 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.315+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c0694f0 con 0x7fab5c162880 2024-04-26T21:41:53.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.315+0000 7fab597fa640 1 -- 172.21.15.40:0/481834451 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab5400abd0 con 0x7fab5c162880 2024-04-26T21:41:53.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.316+0000 7fab597fa640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab3c06c920 0x7fab3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.316+0000 7fab597fa640 1 -- 172.21.15.40:0/481834451 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5421+0+0 (secure 0 0 0) 0x7fab5408b800 con 0x7fab5c162880 2024-04-26T21:41:53.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.316+0000 7fab609e3640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab3c06c920 0x7fab3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.317+0000 7fab609e3640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab3c06c920 0x7fab3c06ede0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fab5c160610 tx=0x7fab44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.320+0000 7fab597fa640 1 -- 172.21.15.40:0/481834451 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab54059d40 con 0x7fab5c162880 2024-04-26T21:41:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.507+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fab5c0cb460 con 0x7fab5c162880 2024-04-26T21:41:53.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.510+0000 7fab597fa640 1 -- 172.21.15.40:0/481834451 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7fab54059640 con 0x7fab5c162880 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab3c06c920 msgr2=0x7fab3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab3c06c920 0x7fab3c06ede0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fab5c160610 tx=0x7fab44008040 comp rx=0 tx=0).stop 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c162880 msgr2=0x7fab5c15c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c162880 0x7fab5c15c8b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c7a0 tx=0x7fab5400cc60 comp rx=0 tx=0).stop 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 shutdown_connections 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab3c06c920 0x7fab3c06ede0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c162880 0x7fab5c15c8b0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d39f0 0x7fab5c15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 --2- 172.21.15.40:0/481834451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0b5f20 0x7fab5c15fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 >> 172.21.15.40:0/481834451 conn(0x7fab5c0d4f60 msgr2=0x7fab5c0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 shutdown_connections 2024-04-26T21:41:53.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.512+0000 7fab62c47640 1 -- 172.21.15.40:0/481834451 wait complete. 2024-04-26T21:41:53.523 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-04-26T21:41:53.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.625+0000 7f06614a2640 1 Processor -- start 2024-04-26T21:41:53.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.625+0000 7f06614a2640 1 -- start start 2024-04-26T21:41:53.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.626+0000 7f06614a2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f065c06e270 0x7f065c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.626+0000 7f06614a2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06d560 0x7f065c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.626+0000 7f06614a2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f065c0d2420 con 0x7f065c0d0070 2024-04-26T21:41:53.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.626+0000 7f06614a2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f065c162d00 con 0x7f065c06e270 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.626+0000 7f06614a2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f065c0d2d10 con 0x7f065c06d560 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.626+0000 7f065a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f065c06e270 0x7f065c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.626+0000 7f065affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f065c0d0070 0x7f065c06db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43228/0 (socket says 172.21.15.40:43228) 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.626+0000 7f065affd640 1 -- 172.21.15.40:0/363723520 learned_addr learned my addr 172.21.15.40:0/363723520 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.627+0000 7f065b7fe640 1 --2- 172.21.15.40:0/363723520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06d560 0x7f065c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.627+0000 7f065a7fc640 1 -- 172.21.15.40:0/363723520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06d560 msgr2=0x7f065c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.627+0000 7f065a7fc640 1 --2- 172.21.15.40:0/363723520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06d560 0x7f065c0ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.627+0000 7f065a7fc640 1 -- 172.21.15.40:0/363723520 >> v1:172.21.15.40:6789/0 conn(0x7f065c0d0070 legacy=0x7f065c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.627+0000 7f0659ffb640 1 -- 172.21.15.40:0/363723520 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899266981 0 0) 0x7f065c0d2420 con 0x7f065c0d0070 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.627+0000 7f065a7fc640 1 -- 172.21.15.40:0/363723520 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c0cd6b0 con 0x7f065c06e270 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.627+0000 7f065b7fe640 1 --2- 172.21.15.40:0/363723520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06d560 0x7f065c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.627+0000 7f065a7fc640 1 --2- 172.21.15.40:0/363723520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c06e270 0x7f065c0d1d40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f06440097a0 tx=0x7f064402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a948bb4726f85a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f0659ffb640 1 -- 172.21.15.40:0/363723520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644037070 con 0x7f065c06e270 2024-04-26T21:41:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f0659ffb640 1 -- 172.21.15.40:0/363723520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0644032440 con 0x7f065c06e270 2024-04-26T21:41:53.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f0659ffb640 1 -- 172.21.15.40:0/363723520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644032810 con 0x7f065c06e270 2024-04-26T21:41:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f06614a2640 1 -- 172.21.15.40:0/363723520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c06e270 msgr2=0x7f065c0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f06614a2640 1 --2- 172.21.15.40:0/363723520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c06e270 0x7f065c0d1d40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f06440097a0 tx=0x7f064402a210 comp rx=0 tx=0).stop 2024-04-26T21:41:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f06614a2640 1 -- 172.21.15.40:0/363723520 shutdown_connections 2024-04-26T21:41:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f06614a2640 1 --2- 172.21.15.40:0/363723520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06d560 0x7f065c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f06614a2640 1 --2- 172.21.15.40:0/363723520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c06e270 0x7f065c0d1d40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f06614a2640 1 -- 172.21.15.40:0/363723520 >> 172.21.15.40:0/363723520 conn(0x7f065c0d2ec0 msgr2=0x7f065c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f06614a2640 1 -- 172.21.15.40:0/363723520 shutdown_connections 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.628+0000 7f06614a2640 1 -- 172.21.15.40:0/363723520 wait complete. 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.629+0000 7f06614a2640 1 Processor -- start 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f06614a2640 1 -- start start 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f06614a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c06d560 0x7f065c1484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f06614a2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06e270 0x7f065c0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f06614a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c0d0070 0x7f065c0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f06614a2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f065c0ce6d0 con 0x7f065c0d0070 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f06614a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f065c0cdc70 con 0x7f065c06d560 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f06614a2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f065c0ce850 con 0x7f065c06e270 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f065b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c0d0070 0x7f065c0c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f065b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c0d0070 0x7f065c0c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56568/0 (socket says 172.21.15.40:56568) 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.630+0000 7f065b7fe640 1 -- 172.21.15.40:0/4216494796 learned_addr learned my addr 172.21.15.40:0/4216494796 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065affd640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c06d560 0x7f065c1484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065a7fc640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06e270 0x7f065c0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065b7fe640 1 -- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06e270 msgr2=0x7f065c0c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065b7fe640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06e270 0x7f065c0c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065b7fe640 1 -- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c06d560 msgr2=0x7f065c1484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065b7fe640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c06d560 0x7f065c1484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065b7fe640 1 -- 172.21.15.40:0/4216494796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c148a10 con 0x7f065c0d0070 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065a7fc640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06e270 0x7f065c0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f065b7fe640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c0d0070 0x7f065c0c7eb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f065400c7f0 tx=0x7f065400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f06537fe640 1 -- 172.21.15.40:0/4216494796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654015070 con 0x7f065c0d0070 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f06537fe640 1 -- 172.21.15.40:0/4216494796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0654007820 con 0x7f065c0d0070 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c0c8b20 con 0x7f065c0d0070 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f06537fe640 1 -- 172.21.15.40:0/4216494796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654007c10 con 0x7f065c0d0070 2024-04-26T21:41:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.631+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c0c90e0 con 0x7f065c0d0070 2024-04-26T21:41:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.632+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c0694f0 con 0x7f065c0d0070 2024-04-26T21:41:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.633+0000 7f06537fe640 1 -- 172.21.15.40:0/4216494796 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0654018070 con 0x7f065c0d0070 2024-04-26T21:41:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.633+0000 7f06537fe640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f063c06c8f0 0x7f063c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.633+0000 7f06537fe640 1 -- 172.21.15.40:0/4216494796 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5421+0+0 (secure 0 0 0) 0x7f065408bb30 con 0x7f065c0d0070 2024-04-26T21:41:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.633+0000 7f065affd640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f063c06c8f0 0x7f063c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.634+0000 7f065affd640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f063c06c8f0 0x7f063c06edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f064c0012e0 tx=0x7f064c001630 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.638+0000 7f06537fe640 1 -- 172.21.15.40:0/4216494796 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0654059a10 con 0x7f065c0d0070 2024-04-26T21:41:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.821+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f065c0cfb60 con 0x7f065c0d0070 2024-04-26T21:41:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.821+0000 7f06537fe640 1 -- 172.21.15.40:0/4216494796 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f065405dd10 con 0x7f065c0d0070 2024-04-26T21:41:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f063c06c8f0 msgr2=0x7f063c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f063c06c8f0 0x7f063c06edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f064c0012e0 tx=0x7f064c001630 comp rx=0 tx=0).stop 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c0d0070 msgr2=0x7f065c0c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c0d0070 0x7f065c0c7eb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f065400c7f0 tx=0x7f065400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 shutdown_connections 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f063c06c8f0 0x7f063c06edb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f065c0d0070 0x7f065c0c7eb0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06e270 0x7f065c0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 --2- 172.21.15.40:0/4216494796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c06d560 0x7f065c1484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 >> 172.21.15.40:0/4216494796 conn(0x7f065c0d2ec0 msgr2=0x7f065c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 shutdown_connections 2024-04-26T21:41:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.823+0000 7f06614a2640 1 -- 172.21.15.40:0/4216494796 wait complete. 2024-04-26T21:41:53.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-04-26T21:41:53.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.935+0000 7fa934511640 1 Processor -- start 2024-04-26T21:41:53.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.935+0000 7fa934511640 1 -- start start 2024-04-26T21:41:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa934511640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa92c162880 0x7fa92c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa934511640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa92c0b7e50 con 0x7fa92c0d3bc0 2024-04-26T21:41:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa934511640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa92c0d1800 con 0x7fa92c0b5f20 2024-04-26T21:41:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa934511640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa92c0d0580 con 0x7fa92c162880 2024-04-26T21:41:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa932aae640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa92c0d3bc0 0x7fa92c0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43230/0 (socket says 172.21.15.40:43230) 2024-04-26T21:41:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa932aae640 1 -- 172.21.15.40:0/1953621933 learned_addr learned my addr 172.21.15.40:0/1953621933 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603871884 0 0) 0x7fa92c0b7e50 con 0x7fa92c0d3bc0 2024-04-26T21:41:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa914003520 con 0x7fa92c0d3bc0 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.936+0000 7fa9322ad640 1 --2- 172.21.15.40:0/1953621933 >> v2:172.21.15.184:3300/0 conn(0x7fa92c162880 0x7fa92c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3934928923 0 0) 0x7fa914003520 con 0x7fa92c0d3bc0 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 >> v2:172.21.15.184:3300/0 conn(0x7fa92c162880 msgr2=0x7fa92c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 --2- 172.21.15.40:0/1953621933 >> v2:172.21.15.184:3300/0 conn(0x7fa92c162880 0x7fa92c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 >> v1:172.21.15.184:6789/0 conn(0x7fa92c0b5f20 legacy=0x7fa92c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c0d07d0 con 0x7fa92c0d3bc0 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fa91c0019f0 con 0x7fa92c0d3bc0 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9322ad640 1 --2- 172.21.15.40:0/1953621933 >> v2:172.21.15.184:3300/0 conn(0x7fa92c162880 0x7fa92c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 >> v1:172.21.15.40:6789/0 conn(0x7fa92c0d3bc0 legacy=0x7fa92c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9140039b0 0x7fa914003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa914004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9140051c0 0x7fa91400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c0d0580 con 0x7fa914004800 2024-04-26T21:41:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa91400c110 con 0x7fa9140051c0 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa91400c310 con 0x7fa9140039b0 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa931aac640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa914004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa932aae640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9140051c0 0x7fa91400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa9322ad640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9140039b0 0x7fa914003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa931aac640 1 -- 172.21.15.40:0/1953621933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9140039b0 msgr2=0x7fa914003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa931aac640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9140039b0 0x7fa914003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa931aac640 1 -- 172.21.15.40:0/1953621933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9140051c0 msgr2=0x7fa91400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa931aac640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9140051c0 0x7fa91400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa932aae640 1 -- 172.21.15.40:0/1953621933 reap_dead start 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.937+0000 7fa931aac640 1 -- 172.21.15.40:0/1953621933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa914003520 con 0x7fa914004800 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.938+0000 7fa931aac640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa914004c80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa920001750 tx=0x7fa914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d286cbe7d8c9dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.938+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920001370 con 0x7fa914004800 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.938+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9200052d0 con 0x7fa914004800 2024-04-26T21:41:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.938+0000 7fa9312ab640 1 -- 172.21.15.40:0/1953621933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92002abc0 con 0x7fa914004800 2024-04-26T21:41:53.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.939+0000 7fa934511640 1 -- 172.21.15.40:0/1953621933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 msgr2=0x7fa914004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.939+0000 7fa934511640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa914004c80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa920001750 tx=0x7fa914005830 comp rx=0 tx=0).stop 2024-04-26T21:41:53.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.939+0000 7fa934511640 1 -- 172.21.15.40:0/1953621933 shutdown_connections 2024-04-26T21:41:53.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.939+0000 7fa934511640 1 --2- 172.21.15.40:0/1953621933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa914004c80 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.939+0000 7fa934511640 1 -- 172.21.15.40:0/1953621933 >> 172.21.15.40:0/1953621933 conn(0x7fa92c0d4f60 msgr2=0x7fa92c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.939+0000 7fa934511640 1 -- 172.21.15.40:0/1953621933 shutdown_connections 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.939+0000 7fa934511640 1 -- 172.21.15.40:0/1953621933 wait complete. 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.940+0000 7fa934511640 1 Processor -- start 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.940+0000 7fa934511640 1 -- start start 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa934511640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa92c0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa934511640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa92c1f44e0 0x7fa92c0b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa934511640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92c0b99f0 0x7fa92c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa934511640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c162c60 con 0x7fa914004800 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa934511640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c0d0d90 con 0x7fa92c0b99f0 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa934511640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c162de0 con 0x7fa92c1f44e0 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa92c0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa92c0b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56594/0 (socket says 172.21.15.40:56594) 2024-04-26T21:41:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 -- 172.21.15.40:0/1856627563 learned_addr learned my addr 172.21.15.40:0/1856627563 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 -- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa92c1f44e0 msgr2=0x7fa92c0b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa931aac640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa92c1f44e0 0x7fa92c0b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa932aae640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92c0b99f0 0x7fa92c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa92c1f44e0 0x7fa92c0b94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 -- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92c0b99f0 msgr2=0x7fa92c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92c0b99f0 0x7fa92c0ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 -- 172.21.15.40:0/1856627563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c0cd1e0 con 0x7fa914004800 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa9322ad640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa92c0b1b50 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa924011600 tx=0x7fa924011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa92affd640 1 -- 172.21.15.40:0/1856627563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924019070 con 0x7fa914004800 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa92affd640 1 -- 172.21.15.40:0/1856627563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92400f9e0 con 0x7fa914004800 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.941+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c0ccf00 con 0x7fa914004800 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.942+0000 7fa92affd640 1 -- 172.21.15.40:0/1856627563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92400f040 con 0x7fa914004800 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.942+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c0cdbf0 con 0x7fa914004800 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.942+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa92c0694f0 con 0x7fa914004800 2024-04-26T21:41:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.943+0000 7fa92affd640 1 -- 172.21.15.40:0/1856627563 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa92401c070 con 0x7fa914004800 2024-04-26T21:41:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.943+0000 7fa92affd640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa91806c920 0x7fa91806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.944+0000 7fa92affd640 1 -- 172.21.15.40:0/1856627563 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5421+0+0 (secure 0 0 0) 0x7fa92408bd70 con 0x7fa914004800 2024-04-26T21:41:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.944+0000 7fa931aac640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa91806c920 0x7fa91806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.944+0000 7fa931aac640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa91806c920 0x7fa91806ede0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa920032000 tx=0x7fa92002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:53.948+0000 7fa92affd640 1 -- 172.21.15.40:0/1856627563 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa92405a220 con 0x7fa914004800 2024-04-26T21:41:54.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:54.216+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7fa92c0d3650 con 0x7fa914004800 2024-04-26T21:41:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.183+0000 7fa92affd640 1 -- 172.21.15.40:0/1856627563 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v340) ==== 141+0+0 (secure 0 0 0) 0x7fa924059b20 con 0x7fa914004800 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.185+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa91806c920 msgr2=0x7fa91806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa91806c920 0x7fa91806ede0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa920032000 tx=0x7fa92002e000 comp rx=0 tx=0).stop 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 msgr2=0x7fa92c0b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa92c0b1b50 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa924011600 tx=0x7fa924011ac0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 shutdown_connections 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa91806c920 0x7fa91806ede0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92c0b99f0 0x7fa92c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa92c1f44e0 0x7fa92c0b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 --2- 172.21.15.40:0/1856627563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa914004800 0x7fa92c0b1b50 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 >> 172.21.15.40:0/1856627563 conn(0x7fa92c0d4f60 msgr2=0x7fa92c15f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 shutdown_connections 2024-04-26T21:41:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.186+0000 7fa934511640 1 -- 172.21.15.40:0/1856627563 wait complete. 2024-04-26T21:41:55.199 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-04-26T21:41:55.200 INFO:tasks.cephfs_test_runner:Starting test: test_object_deletion (tasks.cephfs.test_damage.TestDamage) That the MDS has a clean 'damaged' response to loss of any single metadata object 2024-04-26T21:41:55.200 INFO:tasks.cephfs_test_runner:test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2024-04-26T21:41:55.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:41:55.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.308+0000 7fa5fcb7f640 1 Processor -- start 2024-04-26T21:41:55.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.308+0000 7fa5fcb7f640 1 -- start start 2024-04-26T21:41:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.308+0000 7fa5fcb7f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5f80b2340 0x7fa5f80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.308+0000 7fa5fcb7f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5f80cd930 0x7fa5f80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.308+0000 7fa5fcb7f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5f80d10d0 con 0x7fa5f8162810 2024-04-26T21:41:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.308+0000 7fa5fcb7f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa5f80d0750 con 0x7fa5f80b2340 2024-04-26T21:41:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.308+0000 7fa5fcb7f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa5f80d16a0 con 0x7fa5f80cd930 2024-04-26T21:41:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5effff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5f80b2340 0x7fa5f80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5f659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa5f8162810 0x7fa5f8070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43246/0 (socket says 172.21.15.40:43246) 2024-04-26T21:41:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5f659c640 1 -- 172.21.15.40:0/3269014951 learned_addr learned my addr 172.21.15.40:0/3269014951 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:55.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5f6d9d640 1 --2- 172.21.15.40:0/3269014951 >> v2:172.21.15.184:3300/0 conn(0x7fa5f80cd930 0x7fa5f80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5effff640 1 -- 172.21.15.40:0/3269014951 >> v2:172.21.15.184:3300/0 conn(0x7fa5f80cd930 msgr2=0x7fa5f80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:55.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5effff640 1 --2- 172.21.15.40:0/3269014951 >> v2:172.21.15.184:3300/0 conn(0x7fa5f80cd930 0x7fa5f80d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5effff640 1 -- 172.21.15.40:0/3269014951 >> v1:172.21.15.40:6789/0 conn(0x7fa5f8162810 legacy=0x7fa5f8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5effff640 1 -- 172.21.15.40:0/3269014951 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f80d1370 con 0x7fa5f80b2340 2024-04-26T21:41:55.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5f5d9b640 1 -- 172.21.15.40:0/3269014951 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2770402946 0 0) 0x7fa5f80d10d0 con 0x7fa5f8162810 2024-04-26T21:41:55.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.309+0000 7fa5f6d9d640 1 --2- 172.21.15.40:0/3269014951 >> v2:172.21.15.184:3300/0 conn(0x7fa5f80cd930 0x7fa5f80d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:55.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5effff640 1 --2- 172.21.15.40:0/3269014951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80cd3f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80b2280 tx=0x7fa5e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ab1f04c058e976 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5f5d9b640 1 -- 172.21.15.40:0/3269014951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0037070 con 0x7fa5f80b2340 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5f5d9b640 1 -- 172.21.15.40:0/3269014951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e0032440 con 0x7fa5f80b2340 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5f5d9b640 1 -- 172.21.15.40:0/3269014951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0032810 con 0x7fa5f80b2340 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/3269014951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 msgr2=0x7fa5f80cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/3269014951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80cd3f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80b2280 tx=0x7fa5e002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/3269014951 shutdown_connections 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/3269014951 >> v2:172.21.15.184:3300/0 conn(0x7fa5f80cd930 0x7fa5f80d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/3269014951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80cd3f0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/3269014951 >> 172.21.15.40:0/3269014951 conn(0x7fa5f80d2da0 msgr2=0x7fa5f80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.310+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/3269014951 shutdown_connections 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.311+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/3269014951 wait complete. 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.312+0000 7fa5fcb7f640 1 Processor -- start 2024-04-26T21:41:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.312+0000 7fa5fcb7f640 1 -- start start 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5fcb7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5fcb7f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5f80cd930 0x7fa5f80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5fcb7f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f8162810 0x7fa5f806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5fcb7f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f80d2070 con 0x7fa5f80b2340 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5fcb7f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f80d0470 con 0x7fa5f8162810 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5fcb7f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f80d21f0 con 0x7fa5f80cd930 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56600/0 (socket says 172.21.15.40:56600) 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 -- 172.21.15.40:0/799462598 learned_addr learned my addr 172.21.15.40:0/799462598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f6d9d640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f8162810 0x7fa5f806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5effff640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5f80cd930 0x7fa5f80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 -- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5f80cd930 msgr2=0x7fa5f80c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5f80cd930 0x7fa5f80c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 -- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f8162810 msgr2=0x7fa5f806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f8162810 0x7fa5f806e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 -- 172.21.15.40:0/799462598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f815ed60 con 0x7fa5f80b2340 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5f659c640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80c84c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800c430 tx=0x7fa5e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5eeffd640 1 -- 172.21.15.40:0/799462598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80074e0 con 0x7fa5f80b2340 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5eeffd640 1 -- 172.21.15.40:0/799462598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e80047e0 con 0x7fa5f80b2340 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f80ca5f0 con 0x7fa5f80b2340 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.313+0000 7fa5eeffd640 1 -- 172.21.15.40:0/799462598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8004bd0 con 0x7fa5f80b2340 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.314+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f80cabb0 con 0x7fa5f80b2340 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.315+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f80694f0 con 0x7fa5f80b2340 2024-04-26T21:41:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.315+0000 7fa5eeffd640 1 -- 172.21.15.40:0/799462598 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5e8011070 con 0x7fa5f80b2340 2024-04-26T21:41:55.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.315+0000 7fa5eeffd640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5d806c8f0 0x7fa5d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.316+0000 7fa5eeffd640 1 -- 172.21.15.40:0/799462598 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5421+0+0 (secure 0 0 0) 0x7fa5e808b5a0 con 0x7fa5f80b2340 2024-04-26T21:41:55.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.316+0000 7fa5effff640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5d806c8f0 0x7fa5d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.316+0000 7fa5effff640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5d806c8f0 0x7fa5d806edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0034000 tx=0x7fa5e002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.320+0000 7fa5eeffd640 1 -- 172.21.15.40:0/799462598 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5f815ed60 con 0x7fa5f80b2340 2024-04-26T21:41:55.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.509+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5f815d050 con 0x7fa5f80b2340 2024-04-26T21:41:55.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.510+0000 7fa5eeffd640 1 -- 172.21.15.40:0/799462598 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+13386 (secure 0 0 0) 0x7fa5e8059af0 con 0x7fa5f80b2340 2024-04-26T21:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":85,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:41:53.174185+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/836437049":"2024-04-27T21:41:50.167211+0000","172.21.15.40:6832/836437049":"2024-04-27T21:41:50.167211+0000","172.21.15.40:6835/3115383225":"2024-04-27T21:41:39.656262+0000","172.21.15.40:6834/3115383225":"2024-04-27T21:41:39.656262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5d806c8f0 msgr2=0x7fa5d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5d806c8f0 0x7fa5d806edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0034000 tx=0x7fa5e002e000 comp rx=0 tx=0).stop 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 msgr2=0x7fa5f80c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80c84c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800c430 tx=0x7fa5e800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 shutdown_connections 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5d806c8f0 0x7fa5d806edb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f8162810 0x7fa5f806e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5f80cd930 0x7fa5f80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 --2- 172.21.15.40:0/799462598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5f80b2340 0x7fa5f80c84c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 >> 172.21.15.40:0/799462598 conn(0x7fa5f80d2da0 msgr2=0x7fa5f80cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:55.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 shutdown_connections 2024-04-26T21:41:55.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.512+0000 7fa5fcb7f640 1 -- 172.21.15.40:0/799462598 wait complete. 2024-04-26T21:41:55.523 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:41:55.523 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:41:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.627+0000 7f64bc946640 1 Processor -- start 2024-04-26T21:41:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.627+0000 7f64bc946640 1 -- start start 2024-04-26T21:41:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64bc946640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f64b8070350 0x7f64b8070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64bc946640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64bc946640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b80d2390 con 0x7f64b80cfde0 2024-04-26T21:41:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64bc946640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f64b8071500 con 0x7f64b8070350 2024-04-26T21:41:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64bc946640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64b80afab0 con 0x7f64b8071090 2024-04-26T21:41:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64b5d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f64b8070350 0x7f64b8070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64b6d9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64b659c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f64b80cfde0 0x7f64b80d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:44116/0 (socket says 172.21.15.40:44116) 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64b6d9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b815e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:54664/0 (socket says 172.21.15.40:54664) 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64b5d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f64b8070350 0x7f64b8070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:56504/0 (socket says 172.21.15.40:56504) 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.628+0000 7f64b659c640 1 -- 172.21.15.40:0/2156484556 learned_addr learned my addr 172.21.15.40:0/2156484556 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.629+0000 7f64b559a640 1 -- 172.21.15.40:0/2156484556 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081557364 0 0) 0x7f64b80d2390 con 0x7f64b80cfde0 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.629+0000 7f64b5d9b640 1 -- 172.21.15.40:0/2156484556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 msgr2=0x7f64b815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.629+0000 7f64b5d9b640 1 --2- 172.21.15.40:0/2156484556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.629+0000 7f64b5d9b640 1 -- 172.21.15.40:0/2156484556 >> v1:172.21.15.184:6789/0 conn(0x7f64b80cfde0 legacy=0x7f64b80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.629+0000 7f64b5d9b640 1 -- 172.21.15.40:0/2156484556 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b806d3f0 con 0x7f64b8070350 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.629+0000 7f64b6d9d640 1 --2- 172.21.15.40:0/2156484556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.629+0000 7f64b5d9b640 1 --2- 172.21.15.40:0/2156484556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 0x7f64b8070b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0097a0 tx=0x7f64ac02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e47aac9ed1d1d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64b559a640 1 -- 172.21.15.40:0/2156484556 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac037070 con 0x7f64b8070350 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64b559a640 1 -- 172.21.15.40:0/2156484556 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ac02ae50 con 0x7f64b8070350 2024-04-26T21:41:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64b559a640 1 -- 172.21.15.40:0/2156484556 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac032660 con 0x7f64b8070350 2024-04-26T21:41:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64bc946640 1 -- 172.21.15.40:0/2156484556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 msgr2=0x7f64b8070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64bc946640 1 --2- 172.21.15.40:0/2156484556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 0x7f64b8070b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0097a0 tx=0x7f64ac02a100 comp rx=0 tx=0).stop 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64bc946640 1 -- 172.21.15.40:0/2156484556 shutdown_connections 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64bc946640 1 --2- 172.21.15.40:0/2156484556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64bc946640 1 --2- 172.21.15.40:0/2156484556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 0x7f64b8070b50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64bc946640 1 -- 172.21.15.40:0/2156484556 >> 172.21.15.40:0/2156484556 conn(0x7f64b80d2d80 msgr2=0x7f64b80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64bc946640 1 -- 172.21.15.40:0/2156484556 shutdown_connections 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.630+0000 7f64bc946640 1 -- 172.21.15.40:0/2156484556 wait complete. 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64bc946640 1 Processor -- start 2024-04-26T21:41:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64bc946640 1 -- start start 2024-04-26T21:41:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64bc946640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 0x7f64b80b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64bc946640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b809f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64bc946640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b80cfde0 0x7f64b80b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64bc946640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f64b806d9b0 con 0x7f64b80cfde0 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64bc946640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64b806dd20 con 0x7f64b8070350 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64bc946640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64b815ef30 con 0x7f64b8071090 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.632+0000 7f64b6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b80cfde0 0x7f64b80b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b80cfde0 0x7f64b80b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56610/0 (socket says 172.21.15.40:56610) 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b6d9d640 1 -- 172.21.15.40:0/3126212037 learned_addr learned my addr 172.21.15.40:0/3126212037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b659c640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 0x7f64b80b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b5d9b640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b809f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b6d9d640 1 -- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 msgr2=0x7f64b809f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b6d9d640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b809f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b6d9d640 1 -- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 msgr2=0x7f64b80b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b6d9d640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 0x7f64b80b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b6d9d640 1 -- 172.21.15.40:0/3126212037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b809fcf0 con 0x7f64b80cfde0 2024-04-26T21:41:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64b6d9d640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b80cfde0 0x7f64b80b04d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009f00 tx=0x7f64b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64a6ffd640 1 -- 172.21.15.40:0/3126212037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0018030 con 0x7f64b80cfde0 2024-04-26T21:41:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64a6ffd640 1 -- 172.21.15.40:0/3126212037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b000ce50 con 0x7f64b80cfde0 2024-04-26T21:41:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b809e240 con 0x7f64b80cfde0 2024-04-26T21:41:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64a6ffd640 1 -- 172.21.15.40:0/3126212037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0003bb0 con 0x7f64b80cfde0 2024-04-26T21:41:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.633+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b809e800 con 0x7f64b80cfde0 2024-04-26T21:41:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.634+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b80694f0 con 0x7f64b80cfde0 2024-04-26T21:41:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.635+0000 7f64a6ffd640 1 -- 172.21.15.40:0/3126212037 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64b0005d10 con 0x7f64b80cfde0 2024-04-26T21:41:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.635+0000 7f64a6ffd640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.635+0000 7f64a6ffd640 1 -- 172.21.15.40:0/3126212037 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5421+0+0 (secure 0 0 0) 0x7f64b008c3c0 con 0x7f64b80cfde0 2024-04-26T21:41:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.636+0000 7f64b659c640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.636+0000 7f64b659c640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f649806c8f0 0x7f649806edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f64a8007fd0 tx=0x7f64a8007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.640+0000 7f64a6ffd640 1 -- 172.21.15.40:0/3126212037 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64b0022070 con 0x7f64b80cfde0 2024-04-26T21:41:55.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.828+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64b80c9c10 con 0x7f64b80cfde0 2024-04-26T21:41:55.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.828+0000 7f64a6ffd640 1 -- 172.21.15.40:0/3126212037 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f64b809fcf0 con 0x7f64b80cfde0 2024-04-26T21:41:55.828 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:41:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f649806c8f0 msgr2=0x7f649806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f649806c8f0 0x7f649806edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f64a8007fd0 tx=0x7f64a8007de0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b80cfde0 msgr2=0x7f64b80b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b80cfde0 0x7f64b80b04d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009f00 tx=0x7f64b000c490 comp rx=0 tx=0).stop 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 shutdown_connections 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64b80cfde0 0x7f64b80b04d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64b8071090 0x7f64b809f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 --2- 172.21.15.40:0/3126212037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8070350 0x7f64b80b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 >> 172.21.15.40:0/3126212037 conn(0x7f64b80d2d80 msgr2=0x7f64b81625b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 shutdown_connections 2024-04-26T21:41:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.830+0000 7f64bc946640 1 -- 172.21.15.40:0/3126212037 wait complete. 2024-04-26T21:41:55.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-04-26T21:41:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.942+0000 7fe54b893640 1 Processor -- start 2024-04-26T21:41:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.942+0000 7fe54b893640 1 -- start start 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe54b893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe544162880 0x7fe5440d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe54b893640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe5440d3bc0 0x7fe5440b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe54b893640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5440b7e50 con 0x7fe5440b5f20 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe54b893640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe5440d0580 con 0x7fe5440d3bc0 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe54b893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5440d1800 con 0x7fe544162880 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe549e30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe5440d3bc0 0x7fe5440b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe549e30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe5440d3bc0 0x7fe5440b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56620/0 (socket says 172.21.15.40:56620) 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe549e30640 1 -- 172.21.15.40:0/944227161 learned_addr learned my addr 172.21.15.40:0/944227161 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:55.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.943+0000 7fe54962f640 1 --2- 172.21.15.40:0/944227161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe544162880 0x7fe5440d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.944+0000 7fe549e30640 1 -- 172.21.15.40:0/944227161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe544162880 msgr2=0x7fe5440d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.944+0000 7fe549e30640 1 --2- 172.21.15.40:0/944227161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe544162880 0x7fe5440d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.944+0000 7fe549e30640 1 -- 172.21.15.40:0/944227161 >> v1:172.21.15.184:6789/0 conn(0x7fe5440b5f20 legacy=0x7fe5440d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.944+0000 7fe549e30640 1 -- 172.21.15.40:0/944227161 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5440d07d0 con 0x7fe5440d3bc0 2024-04-26T21:41:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.944+0000 7fe54962f640 1 --2- 172.21.15.40:0/944227161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe544162880 0x7fe5440d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.944+0000 7fe549e30640 1 --2- 172.21.15.40:0/944227161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440d3bc0 0x7fe5440b8300 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe540009e50 tx=0x7fe54002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d322b7cfcc1d852 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe53bfff640 1 -- 172.21.15.40:0/944227161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540036070 con 0x7fe5440d3bc0 2024-04-26T21:41:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe53bfff640 1 -- 172.21.15.40:0/944227161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe540031600 con 0x7fe5440d3bc0 2024-04-26T21:41:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe53bfff640 1 -- 172.21.15.40:0/944227161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5400319d0 con 0x7fe5440d3bc0 2024-04-26T21:41:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe54b893640 1 -- 172.21.15.40:0/944227161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440d3bc0 msgr2=0x7fe5440b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe54b893640 1 --2- 172.21.15.40:0/944227161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440d3bc0 0x7fe5440b8300 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe540009e50 tx=0x7fe54002a710 comp rx=0 tx=0).stop 2024-04-26T21:41:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe54b893640 1 -- 172.21.15.40:0/944227161 shutdown_connections 2024-04-26T21:41:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe54b893640 1 --2- 172.21.15.40:0/944227161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440d3bc0 0x7fe5440b8300 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe54b893640 1 --2- 172.21.15.40:0/944227161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe544162880 0x7fe5440d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe54b893640 1 -- 172.21.15.40:0/944227161 >> 172.21.15.40:0/944227161 conn(0x7fe5440d4f60 msgr2=0x7fe5440d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe54b893640 1 -- 172.21.15.40:0/944227161 shutdown_connections 2024-04-26T21:41:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.945+0000 7fe54b893640 1 -- 172.21.15.40:0/944227161 wait complete. 2024-04-26T21:41:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.946+0000 7fe54b893640 1 Processor -- start 2024-04-26T21:41:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.946+0000 7fe54b893640 1 -- start start 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe54b893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5440b5f20 0x7fe5441f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe54b893640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440d3bc0 0x7fe54415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe54b893640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe544162880 0x7fe5441596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe54b893640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe544162c60 con 0x7fe544162880 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe54b893640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5440d0d90 con 0x7fe5440d3bc0 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe54b893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe544162de0 con 0x7fe5440b5f20 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe549e30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe544162880 0x7fe5441596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe549e30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe544162880 0x7fe5441596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56630/0 (socket says 172.21.15.40:56630) 2024-04-26T21:41:55.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.947+0000 7fe549e30640 1 -- 172.21.15.40:0/3605785208 learned_addr learned my addr 172.21.15.40:0/3605785208 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe54962f640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5440b5f20 0x7fe5441f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe548e2e640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440d3bc0 0x7fe54415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe549e30640 1 -- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5440b5f20 msgr2=0x7fe5441f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe549e30640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5440b5f20 0x7fe5441f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe549e30640 1 -- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440d3bc0 msgr2=0x7fe54415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe549e30640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440d3bc0 0x7fe54415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe549e30640 1 -- 172.21.15.40:0/3605785208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5440cd090 con 0x7fe544162880 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe549e30640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe544162880 0x7fe5441596c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe540034040 tx=0x7fe540004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe539ffb640 1 -- 172.21.15.40:0/3605785208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54003f020 con 0x7fe544162880 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe539ffb640 1 -- 172.21.15.40:0/3605785208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe540037040 con 0x7fe544162880 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5441f89f0 con 0x7fe544162880 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe539ffb640 1 -- 172.21.15.40:0/3605785208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540036070 con 0x7fe544162880 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.948+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5441f8fb0 con 0x7fe544162880 2024-04-26T21:41:55.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.949+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5440694f0 con 0x7fe544162880 2024-04-26T21:41:55.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.950+0000 7fe539ffb640 1 -- 172.21.15.40:0/3605785208 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe540042020 con 0x7fe544162880 2024-04-26T21:41:55.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.950+0000 7fe539ffb640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:55.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.950+0000 7fe539ffb640 1 -- 172.21.15.40:0/3605785208 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5421+0+0 (secure 0 0 0) 0x7fe5400abf90 con 0x7fe544162880 2024-04-26T21:41:55.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.951+0000 7fe54962f640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:55.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.951+0000 7fe54962f640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe53c006fd0 tx=0x7fe53c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:55.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:55.955+0000 7fe539ffb640 1 -- 172.21.15.40:0/3605785208 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe540049030 con 0x7fe544162880 2024-04-26T21:41:56.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:56.211+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7fe544069d60 con 0x7fe544162880 2024-04-26T21:41:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.189+0000 7fe539ffb640 1 -- 172.21.15.40:0/3605785208 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v342) ==== 138+0+0 (secure 0 0 0) 0x7fe54007a440 con 0x7fe544162880 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.191+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 msgr2=0x7fe52806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.191+0000 7fe54b893640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe53c006fd0 tx=0x7fe53c008040 comp rx=0 tx=0).stop 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.191+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe544162880 msgr2=0x7fe5441596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.191+0000 7fe54b893640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe544162880 0x7fe5441596c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe540034040 tx=0x7fe540004a40 comp rx=0 tx=0).stop 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.192+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 shutdown_connections 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.192+0000 7fe54b893640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.192+0000 7fe54b893640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe544162880 0x7fe5441596c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.192+0000 7fe54b893640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5440d3bc0 0x7fe54415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.192+0000 7fe54b893640 1 --2- 172.21.15.40:0/3605785208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5440b5f20 0x7fe5441f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.192+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 >> 172.21.15.40:0/3605785208 conn(0x7fe5440d4f60 msgr2=0x7fe5440d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.192+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 shutdown_connections 2024-04-26T21:41:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.192+0000 7fe54b893640 1 -- 172.21.15.40:0/3605785208 wait complete. 2024-04-26T21:41:57.205 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:41:57.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.314+0000 7f4ed0ebd640 1 Processor -- start 2024-04-26T21:41:57.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.314+0000 7f4ed0ebd640 1 -- start start 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.315+0000 7f4ed0ebd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0d1a00 0x7f4ecc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.315+0000 7f4ed0ebd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d2320 0x7f4ecc1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.315+0000 7f4ed0ebd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ecc0b40d0 0x7f4ecc162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.315+0000 7f4ed0ebd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4ecc163070 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.315+0000 7f4ed0ebd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4ecc0d0a10 con 0x7f4ecc0d1a00 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.315+0000 7f4ed0ebd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc0d0cf0 con 0x7f4ecc0d2320 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ecad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ecc0b40d0 0x7f4ecc162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ecad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ecc0b40d0 0x7f4ecc162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56642/0 (socket says 172.21.15.40:56642) 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ecad9d640 1 -- 172.21.15.40:0/3094808843 learned_addr learned my addr 172.21.15.40:0/3094808843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4eca59c640 1 --2- 172.21.15.40:0/3094808843 >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0d1a00 0x7f4ecc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ec9d9b640 1 --2- 172.21.15.40:0/3094808843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d2320 0x7f4ecc1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ecad9d640 1 -- 172.21.15.40:0/3094808843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d2320 msgr2=0x7f4ecc1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ecad9d640 1 --2- 172.21.15.40:0/3094808843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d2320 0x7f4ecc1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ecad9d640 1 -- 172.21.15.40:0/3094808843 >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0d1a00 msgr2=0x7f4ecc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ecad9d640 1 --2- 172.21.15.40:0/3094808843 >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0d1a00 0x7f4ecc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.316+0000 7f4ecad9d640 1 -- 172.21.15.40:0/3094808843 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc0d0fd0 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ecad9d640 1 --2- 172.21.15.40:0/3094808843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc162b30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc06c650 tx=0x7f4ec402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefb28d71a8293d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ec959a640 1 -- 172.21.15.40:0/3094808843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4037070 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ec959a640 1 -- 172.21.15.40:0/3094808843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec4032440 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ec959a640 1 -- 172.21.15.40:0/3094808843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4032810 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/3094808843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 msgr2=0x7f4ecc162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/3094808843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc162b30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc06c650 tx=0x7f4ec402a140 comp rx=0 tx=0).stop 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/3094808843 shutdown_connections 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/3094808843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc162b30 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/3094808843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d2320 0x7f4ecc1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/3094808843 >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0d1a00 0x7f4ecc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/3094808843 >> 172.21.15.40:0/3094808843 conn(0x7f4ecc0d4fd0 msgr2=0x7f4ecc0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/3094808843 shutdown_connections 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.317+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/3094808843 wait complete. 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.319+0000 7f4ed0ebd640 1 Processor -- start 2024-04-26T21:41:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.319+0000 7f4ed0ebd640 1 -- start start 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.319+0000 7f4ed0ebd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.319+0000 7f4ed0ebd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d1a00 0x7f4ecc0b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.319+0000 7f4ed0ebd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc0d2320 0x7f4ecc1f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.319+0000 7f4ed0ebd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0d3050 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.319+0000 7f4ed0ebd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0d2ed0 con 0x7f4ecc0d2320 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.319+0000 7f4ed0ebd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc0d31d0 con 0x7f4ecc0d1a00 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc0ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56658/0 (socket says 172.21.15.40:56658) 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 -- 172.21.15.40:0/1225061272 learned_addr learned my addr 172.21.15.40:0/1225061272 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4ec9d9b640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d1a00 0x7f4ecc0b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4ecad9d640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc0d2320 0x7f4ecc1f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 -- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d1a00 msgr2=0x7f4ecc0b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d1a00 0x7f4ecc0b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 -- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc0d2320 msgr2=0x7f4ecc1f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc0d2320 0x7f4ecc1f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 -- 172.21.15.40:0/1225061272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc0b8fe0 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4ec9d9b640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d1a00 0x7f4ecc0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4eca59c640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc0ceb30 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00eed0 tx=0x7f4ebc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/1225061272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc018020 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/1225061272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc014400 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/1225061272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0147d0 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc0b8130 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.320+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc1f4e10 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.321+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc0694f0 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.322+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/1225061272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ebc005d10 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.322+0000 7f4ec2ffd640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4eac06c8f0 0x7f4eac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.323+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/1225061272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5421+0+0 (secure 0 0 0) 0x7f4ebc08c040 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.323+0000 7f4ec9d9b640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4eac06c8f0 0x7f4eac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.323+0000 7f4ec9d9b640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4eac06c8f0 0x7f4eac06edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4006fd0 tx=0x7f4eb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.327+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/1225061272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ebc022070 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.511+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4ecc0cbe60 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.514+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/1225061272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f4ebc05a590 con 0x7f4ecc0b40d0 2024-04-26T21:41:57.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4eac06c8f0 msgr2=0x7f4eac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4eac06c8f0 0x7f4eac06edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4006fd0 tx=0x7f4eb4008040 comp rx=0 tx=0).stop 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 msgr2=0x7f4ecc0ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc0ceb30 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00eed0 tx=0x7f4ebc00b480 comp rx=0 tx=0).stop 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 shutdown_connections 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4eac06c8f0 0x7f4eac06edb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc0d2320 0x7f4ecc1f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ecc0d1a00 0x7f4ecc0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 --2- 172.21.15.40:0/1225061272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ecc0b40d0 0x7f4ecc0ceb30 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.516+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 >> 172.21.15.40:0/1225061272 conn(0x7f4ecc0d4fd0 msgr2=0x7f4ecc0b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.517+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 shutdown_connections 2024-04-26T21:41:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.517+0000 7f4ed0ebd640 1 -- 172.21.15.40:0/1225061272 wait complete. 2024-04-26T21:41:57.527 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:41:57.528 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:41:57.566 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:41:57.614 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:41:57.614 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:41:57.615 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:57.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:41:57.630 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:41:57.630 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:57.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:41:57.648 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:41:57.648 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:57.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:41:57.686 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:41:57.686 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:41:57.706 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:41:57.706 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:41:57.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:41:57.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.826+0000 7fee62aa9640 1 Processor -- start 2024-04-26T21:41:57.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.826+0000 7fee62aa9640 1 -- start start 2024-04-26T21:41:57.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee62aa9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee5c0b2340 0x7fee5c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee62aa9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fee5c0cd930 0x7fee5c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee62aa9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5c0d10d0 con 0x7fee5c162810 2024-04-26T21:41:57.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee62aa9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fee5c0d0750 con 0x7fee5c0b2340 2024-04-26T21:41:57.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee62aa9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fee5c0d16a0 con 0x7fee5c0cd930 2024-04-26T21:41:57.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee5bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee5c0b2340 0x7fee5c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee5bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee5c0b2340 0x7fee5c0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56668/0 (socket says 172.21.15.40:56668) 2024-04-26T21:41:57.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee5bfff640 1 -- 172.21.15.40:0/1460798161 learned_addr learned my addr 172.21.15.40:0/1460798161 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:57.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee61046640 1 --2- 172.21.15.40:0/1460798161 >> v2:172.21.15.184:3300/0 conn(0x7fee5c0cd930 0x7fee5c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee5bfff640 1 -- 172.21.15.40:0/1460798161 >> v2:172.21.15.184:3300/0 conn(0x7fee5c0cd930 msgr2=0x7fee5c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:57.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee5bfff640 1 --2- 172.21.15.40:0/1460798161 >> v2:172.21.15.184:3300/0 conn(0x7fee5c0cd930 0x7fee5c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee5bfff640 1 -- 172.21.15.40:0/1460798161 >> v1:172.21.15.184:6789/0 conn(0x7fee5c162810 legacy=0x7fee5c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:41:57.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.827+0000 7fee5bfff640 1 -- 172.21.15.40:0/1460798161 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c0d1370 con 0x7fee5c0b2340 2024-04-26T21:41:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.828+0000 7fee5bfff640 1 --2- 172.21.15.40:0/1460798161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0b2340 0x7fee5c0cd3f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0b2280 tx=0x7fee4802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b482d1d71f3d2b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.828+0000 7fee5b7fe640 1 -- 172.21.15.40:0/1460798161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48037070 con 0x7fee5c0b2340 2024-04-26T21:41:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.828+0000 7fee5b7fe640 1 -- 172.21.15.40:0/1460798161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee48032440 con 0x7fee5c0b2340 2024-04-26T21:41:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.828+0000 7fee5b7fe640 1 -- 172.21.15.40:0/1460798161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48032810 con 0x7fee5c0b2340 2024-04-26T21:41:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.829+0000 7fee62aa9640 1 -- 172.21.15.40:0/1460798161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0b2340 msgr2=0x7fee5c0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.829+0000 7fee62aa9640 1 --2- 172.21.15.40:0/1460798161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0b2340 0x7fee5c0cd3f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0b2280 tx=0x7fee4802a230 comp rx=0 tx=0).stop 2024-04-26T21:41:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.829+0000 7fee62aa9640 1 -- 172.21.15.40:0/1460798161 shutdown_connections 2024-04-26T21:41:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.829+0000 7fee62aa9640 1 --2- 172.21.15.40:0/1460798161 >> v2:172.21.15.184:3300/0 conn(0x7fee5c0cd930 0x7fee5c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.829+0000 7fee62aa9640 1 --2- 172.21.15.40:0/1460798161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0b2340 0x7fee5c0cd3f0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.829+0000 7fee62aa9640 1 -- 172.21.15.40:0/1460798161 >> 172.21.15.40:0/1460798161 conn(0x7fee5c0d2da0 msgr2=0x7fee5c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.829+0000 7fee62aa9640 1 -- 172.21.15.40:0/1460798161 shutdown_connections 2024-04-26T21:41:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.829+0000 7fee62aa9640 1 -- 172.21.15.40:0/1460798161 wait complete. 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.830+0000 7fee62aa9640 1 Processor -- start 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.830+0000 7fee62aa9640 1 -- start start 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee62aa9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee5c0b2340 0x7fee5c0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee62aa9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0cd930 0x7fee5c0c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee62aa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee5c162810 0x7fee5c14e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee62aa9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c0d2070 con 0x7fee5c0cd930 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee62aa9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c0d0470 con 0x7fee5c162810 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee62aa9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c0d21f0 con 0x7fee5c0b2340 2024-04-26T21:41:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0cd930 0x7fee5c0c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0cd930 0x7fee5c0c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56676/0 (socket says 172.21.15.40:56676) 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 -- 172.21.15.40:0/3848951237 learned_addr learned my addr 172.21.15.40:0/3848951237 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee60845640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee5c0b2340 0x7fee5c0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee61046640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee5c162810 0x7fee5c14e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 -- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee5c0b2340 msgr2=0x7fee5c0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee5c0b2340 0x7fee5c0ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 -- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee5c162810 msgr2=0x7fee5c14e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee5c162810 0x7fee5c14e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 -- 172.21.15.40:0/3848951237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c0cb910 con 0x7fee5c0cd930 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee5bfff640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0cd930 0x7fee5c0c68d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fee4802a270 tx=0x7fee480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee597fa640 1 -- 172.21.15.40:0/3848951237 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48041020 con 0x7fee5c0cd930 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee597fa640 1 -- 172.21.15.40:0/3848951237 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee480306c0 con 0x7fee5c0cd930 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c0cbbf0 con 0x7fee5c0cd930 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.831+0000 7fee597fa640 1 -- 172.21.15.40:0/3848951237 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48037040 con 0x7fee5c0cd930 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.832+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c0cc1b0 con 0x7fee5c0cd930 2024-04-26T21:41:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.832+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5c0694f0 con 0x7fee5c0cd930 2024-04-26T21:41:57.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.833+0000 7fee597fa640 1 -- 172.21.15.40:0/3848951237 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee48044020 con 0x7fee5c0cd930 2024-04-26T21:41:57.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.833+0000 7fee597fa640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee4006c8f0 0x7fee4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:41:57.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.833+0000 7fee597fa640 1 -- 172.21.15.40:0/3848951237 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5421+0+0 (secure 0 0 0) 0x7fee480ac0e0 con 0x7fee5c0cd930 2024-04-26T21:41:57.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.834+0000 7fee60845640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee4006c8f0 0x7fee4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:41:57.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.834+0000 7fee60845640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee4006c8f0 0x7fee4006edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fee50006fd0 tx=0x7fee50009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:41:57.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:57.838+0000 7fee597fa640 1 -- 172.21.15.40:0/3848951237 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee4804b030 con 0x7fee5c0cd930 2024-04-26T21:41:58.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.061+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee5c0d3770 con 0x7fee5c0cd930 2024-04-26T21:41:58.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.062+0000 7fee597fa640 1 -- 172.21.15.40:0/3848951237 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+3872 (secure 0 0 0) 0x7fee4807a070 con 0x7fee5c0cd930 2024-04-26T21:41:58.062 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:41:58.062 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2297933854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2297933854},{"type":"v1","addr":"172.21.15.184:6837","nonce":2297933854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1800813039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1800813039},{"type":"v1","addr":"172.21.15.184:6836","nonce":1800813039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7485,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/379824506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":379824506},{"type":"v1","addr":"172.21.15.40:6835","nonce":379824506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7665,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1178023002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1178023002},{"type":"v1","addr":"172.21.15.40:6833","nonce":1178023002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[]} 2024-04-26T21:41:58.062 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 80 2024-04-26T21:41:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee4006c8f0 msgr2=0x7fee4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee4006c8f0 0x7fee4006edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fee50006fd0 tx=0x7fee50009060 comp rx=0 tx=0).stop 2024-04-26T21:41:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0cd930 msgr2=0x7fee5c0c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:41:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0cd930 0x7fee5c0c68d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fee4802a270 tx=0x7fee480046c0 comp rx=0 tx=0).stop 2024-04-26T21:41:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 shutdown_connections 2024-04-26T21:41:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee4006c8f0 0x7fee4006edb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee5c162810 0x7fee5c14e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee5c0cd930 0x7fee5c0c68d0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 --2- 172.21.15.40:0/3848951237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee5c0b2340 0x7fee5c0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:41:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 >> 172.21.15.40:0/3848951237 conn(0x7fee5c0d2da0 msgr2=0x7fee5c0b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:41:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 shutdown_connections 2024-04-26T21:41:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:41:58.064+0000 7fee62aa9640 1 -- 172.21.15.40:0/3848951237 wait complete. 2024-04-26T21:41:58.075 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:41:58.075 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:41:58.075 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:41:58.075 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:58.075 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:41:58.075 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:41:58.076 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:41:58.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:58.076 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:41:58.076 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:41:58.076 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:41:58.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:58.076 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:41:58.076 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:41:58.077 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:41:58.077 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:41:58.088 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:41:58.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:41:58.089 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:41:58.089 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:41:58.090 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:41:58.090 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:41:58.131 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:41:58.131 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:58.132 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:41:58.132 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:58.133 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:41:58.133 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:58.204 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:41:58.213 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:41:58.213 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:42:03.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-26T21:42:04.088 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:42:04.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:42:04.091 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:42:04.092 DEBUG:teuthology.parallel:result is None 2024-04-26T21:42:04.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:42:04.177 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:42:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.206+0000 7f4520817640 1 Processor -- start 2024-04-26T21:42:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.206+0000 7f4520817640 1 -- start start 2024-04-26T21:42:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.207+0000 7f4520817640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f451c15f3a0 0x7f451c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.207+0000 7f4520817640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c0d2330 0x7f451c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f4520817640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451c0d0e20 con 0x7f451c0d1770 2024-04-26T21:42:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f4520817640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f451c0b1d70 con 0x7f451c15f3a0 2024-04-26T21:42:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f4520817640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f451c0b2050 con 0x7f451c0d2330 2024-04-26T21:42:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451ad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c0d2330 0x7f451c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451a59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f451c15f3a0 0x7f451c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451ad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c0d2330 0x7f451c0b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:40440/0 (socket says 172.21.15.40:40440) 2024-04-26T21:42:04.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451ad9d640 1 -- 172.21.15.40:0/647095077 learned_addr learned my addr 172.21.15.40:0/647095077 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:04.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f4519d9b640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f451c0d1770 0x7f451c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:48380/0 (socket says 172.21.15.40:48380) 2024-04-26T21:42:04.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451ad9d640 1 -- 172.21.15.40:0/647095077 >> v2:172.21.15.184:3300/0 conn(0x7f451c15f3a0 msgr2=0x7f451c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:04.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451ad9d640 1 --2- 172.21.15.40:0/647095077 >> v2:172.21.15.184:3300/0 conn(0x7f451c15f3a0 0x7f451c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:04.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451ad9d640 1 -- 172.21.15.40:0/647095077 >> v1:172.21.15.184:6789/0 conn(0x7f451c0d1770 legacy=0x7f451c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:04.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451ad9d640 1 -- 172.21.15.40:0/647095077 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c0b2330 con 0x7f451c0d2330 2024-04-26T21:42:04.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.208+0000 7f451a59c640 1 --2- 172.21.15.40:0/647095077 >> v2:172.21.15.184:3300/0 conn(0x7f451c15f3a0 0x7f451c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:04.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.209+0000 7f451ad9d640 1 --2- 172.21.15.40:0/647095077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c0d2330 0x7f451c0b2850 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4514009e50 tx=0x7f451402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56459e6f1033f82d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:04.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.210+0000 7f451959a640 1 -- 172.21.15.40:0/647095077 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514036070 con 0x7f451c0d2330 2024-04-26T21:42:04.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.210+0000 7f451959a640 1 -- 172.21.15.40:0/647095077 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4514031600 con 0x7f451c0d2330 2024-04-26T21:42:04.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.210+0000 7f451959a640 1 -- 172.21.15.40:0/647095077 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45140319d0 con 0x7f451c0d2330 2024-04-26T21:42:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.210+0000 7f4520817640 1 -- 172.21.15.40:0/647095077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c0d2330 msgr2=0x7f451c0b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.210+0000 7f4520817640 1 --2- 172.21.15.40:0/647095077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c0d2330 0x7f451c0b2850 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4514009e50 tx=0x7f451402a710 comp rx=0 tx=0).stop 2024-04-26T21:42:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.211+0000 7f4520817640 1 -- 172.21.15.40:0/647095077 shutdown_connections 2024-04-26T21:42:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.211+0000 7f4520817640 1 --2- 172.21.15.40:0/647095077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c0d2330 0x7f451c0b2850 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.211+0000 7f4520817640 1 --2- 172.21.15.40:0/647095077 >> v2:172.21.15.184:3300/0 conn(0x7f451c15f3a0 0x7f451c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.211+0000 7f4520817640 1 -- 172.21.15.40:0/647095077 >> 172.21.15.40:0/647095077 conn(0x7f451c0d4f90 msgr2=0x7f451c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.211+0000 7f4520817640 1 -- 172.21.15.40:0/647095077 shutdown_connections 2024-04-26T21:42:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.211+0000 7f4520817640 1 -- 172.21.15.40:0/647095077 wait complete. 2024-04-26T21:42:04.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.212+0000 7f4520817640 1 Processor -- start 2024-04-26T21:42:04.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.212+0000 7f4520817640 1 -- start start 2024-04-26T21:42:04.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f4520817640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451c0d1770 0x7f451c1f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f4520817640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f451c0d2330 0x7f451c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f4520817640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c15f3a0 0x7f451c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f4520817640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0b2f10 con 0x7f451c0d2330 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f4520817640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0b2d90 con 0x7f451c0d1770 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f4520817640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f451c0b3090 con 0x7f451c15f3a0 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f451a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451c0d1770 0x7f451c1f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f451a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451c0d1770 0x7f451c1f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:33494/0 (socket says 172.21.15.40:33494) 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.213+0000 7f451a59c640 1 -- 172.21.15.40:0/3978146327 learned_addr learned my addr 172.21.15.40:0/3978146327 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f4519d9b640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f451c0d2330 0x7f451c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f451ad9d640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c15f3a0 0x7f451c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f451a59c640 1 -- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c15f3a0 msgr2=0x7f451c1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f451a59c640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c15f3a0 0x7f451c1586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f451a59c640 1 -- 172.21.15.40:0/3978146327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f451c0d2330 msgr2=0x7f451c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f451a59c640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f451c0d2330 0x7f451c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f451a59c640 1 -- 172.21.15.40:0/3978146327 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c1f5740 con 0x7f451c0d1770 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f451a59c640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451c0d1770 0x7f451c1f4ea0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e960 tx=0x7f450c00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f450affd640 1 -- 172.21.15.40:0/3978146327 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c017070 con 0x7f451c0d1770 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f450affd640 1 -- 172.21.15.40:0/3978146327 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c00ca00 con 0x7f451c0d1770 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c1f5a20 con 0x7f451c0d1770 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f450affd640 1 -- 172.21.15.40:0/3978146327 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00cdd0 con 0x7f451c0d1770 2024-04-26T21:42:04.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.214+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c1f5fe0 con 0x7f451c0d1770 2024-04-26T21:42:04.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.216+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c06c910 con 0x7f451c0d1770 2024-04-26T21:42:04.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.216+0000 7f450affd640 1 -- 172.21.15.40:0/3978146327 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f450c01a070 con 0x7f451c0d1770 2024-04-26T21:42:04.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.216+0000 7f450affd640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44fc06c920 0x7f44fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:04.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.216+0000 7f450affd640 1 -- 172.21.15.40:0/3978146327 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5421+0+0 (secure 0 0 0) 0x7f450c08b830 con 0x7f451c0d1770 2024-04-26T21:42:04.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.217+0000 7f4519d9b640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44fc06c920 0x7f44fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:04.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.217+0000 7f4519d9b640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44fc06c920 0x7f44fc06ede0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f451c1f8370 tx=0x7f4510007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:04.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.222+0000 7f450affd640 1 -- 172.21.15.40:0/3978146327 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f451c06c910 con 0x7f451c0d1770 2024-04-26T21:42:04.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:04.418+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f451c0694f0 con 0x7f451c0d1770 2024-04-26T21:42:05.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.214+0000 7f450affd640 1 -- 172.21.15.40:0/3978146327 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v90) ==== 97+0+0 (secure 0 0 0) 0x7f450c059730 con 0x7f451c0d1770 2024-04-26T21:42:05.214 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:42:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44fc06c920 msgr2=0x7f44fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44fc06c920 0x7f44fc06ede0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f451c1f8370 tx=0x7f4510007e80 comp rx=0 tx=0).stop 2024-04-26T21:42:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451c0d1770 msgr2=0x7f451c1f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451c0d1770 0x7f451c1f4ea0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e960 tx=0x7f450c00ee20 comp rx=0 tx=0).stop 2024-04-26T21:42:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 shutdown_connections 2024-04-26T21:42:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44fc06c920 0x7f44fc06ede0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451c15f3a0 0x7f451c1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f451c0d2330 0x7f451c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 --2- 172.21.15.40:0/3978146327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451c0d1770 0x7f451c1f4ea0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 >> 172.21.15.40:0/3978146327 conn(0x7f451c0d4f90 msgr2=0x7f451c0d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 shutdown_connections 2024-04-26T21:42:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.217+0000 7f4520817640 1 -- 172.21.15.40:0/3978146327 wait complete. 2024-04-26T21:42:05.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:42:05.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.337+0000 7fd714a1d640 1 Processor -- start 2024-04-26T21:42:05.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.337+0000 7fd714a1d640 1 -- start start 2024-04-26T21:42:05.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.338+0000 7fd714a1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.338+0000 7fd714a1d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd7100d1770 0x7fd7100d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.338+0000 7fd714a1d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7100d25a0 con 0x7fd7100d2130 2024-04-26T21:42:05.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.338+0000 7fd714a1d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd7100d0e20 con 0x7fd7100d1770 2024-04-26T21:42:05.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.338+0000 7fd714a1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd7100b1d70 con 0x7fd71015f3a0 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70dd9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd7100d1770 0x7fd7100d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70e59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70ed9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fd7100d2130 0x7fd7100d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:48396/0 (socket says 172.21.15.40:48396) 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70ed9d640 1 -- 172.21.15.40:0/920244321 learned_addr learned my addr 172.21.15.40:0/920244321 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70d59a640 1 -- 172.21.15.40:0/920244321 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 948710011 0 0) 0x7fd700001490 con 0x7fd7100d2130 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70d59a640 1 -- 172.21.15.40:0/920244321 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6fc003520 con 0x7fd7100d2130 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70e59c640 1 -- 172.21.15.40:0/920244321 >> v2:172.21.15.184:3300/0 conn(0x7fd7100d1770 msgr2=0x7fd7100d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70e59c640 1 --2- 172.21.15.40:0/920244321 >> v2:172.21.15.184:3300/0 conn(0x7fd7100d1770 0x7fd7100d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70e59c640 1 -- 172.21.15.40:0/920244321 >> v1:172.21.15.184:6789/0 conn(0x7fd7100d2130 legacy=0x7fd7100d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70e59c640 1 -- 172.21.15.40:0/920244321 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7100b2050 con 0x7fd71015f3a0 2024-04-26T21:42:05.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.339+0000 7fd70dd9b640 1 --2- 172.21.15.40:0/920244321 >> v2:172.21.15.184:3300/0 conn(0x7fd7100d1770 0x7fd7100d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:05.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.340+0000 7fd70e59c640 1 --2- 172.21.15.40:0/920244321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100d06f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd7080097a0 tx=0x7fd70802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ccaba0e8fd4724 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:05.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.340+0000 7fd70d59a640 1 -- 172.21.15.40:0/920244321 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708037070 con 0x7fd71015f3a0 2024-04-26T21:42:05.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.340+0000 7fd70d59a640 1 -- 172.21.15.40:0/920244321 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd708032440 con 0x7fd71015f3a0 2024-04-26T21:42:05.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.340+0000 7fd70d59a640 1 -- 172.21.15.40:0/920244321 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7080327f0 con 0x7fd71015f3a0 2024-04-26T21:42:05.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.341+0000 7fd714a1d640 1 -- 172.21.15.40:0/920244321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 msgr2=0x7fd7100d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.341+0000 7fd714a1d640 1 --2- 172.21.15.40:0/920244321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100d06f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd7080097a0 tx=0x7fd70802a180 comp rx=0 tx=0).stop 2024-04-26T21:42:05.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.341+0000 7fd714a1d640 1 -- 172.21.15.40:0/920244321 shutdown_connections 2024-04-26T21:42:05.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.341+0000 7fd714a1d640 1 --2- 172.21.15.40:0/920244321 >> v2:172.21.15.184:3300/0 conn(0x7fd7100d1770 0x7fd7100d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.341+0000 7fd714a1d640 1 --2- 172.21.15.40:0/920244321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100d06f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.341+0000 7fd714a1d640 1 -- 172.21.15.40:0/920244321 >> 172.21.15.40:0/920244321 conn(0x7fd7100d4f90 msgr2=0x7fd7100d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:05.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.341+0000 7fd714a1d640 1 -- 172.21.15.40:0/920244321 shutdown_connections 2024-04-26T21:42:05.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.341+0000 7fd714a1d640 1 -- 172.21.15.40:0/920244321 wait complete. 2024-04-26T21:42:05.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.342+0000 7fd714a1d640 1 Processor -- start 2024-04-26T21:42:05.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.342+0000 7fd714a1d640 1 -- start start 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd714a1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7100d1770 0x7fd7100b8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd714a1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7100d2130 0x7fd7101564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd714a1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd714a1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd71015f900 con 0x7fd7100d2130 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd714a1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd71015f780 con 0x7fd7100d1770 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd714a1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd7100b3860 con 0x7fd71015f3a0 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7100d2130 0x7fd7101564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7100d2130 0x7fd7101564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44136/0 (socket says 172.21.15.40:44136) 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70dd9b640 1 -- 172.21.15.40:0/239017856 learned_addr learned my addr 172.21.15.40:0/239017856 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70e59c640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7100d1770 0x7fd7100b8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70ed9d640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70dd9b640 1 -- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 msgr2=0x7fd7100cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70dd9b640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70dd9b640 1 -- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7100d1770 msgr2=0x7fd7100b8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70dd9b640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7100d1770 0x7fd7100b8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70dd9b640 1 -- 172.21.15.40:0/239017856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7101f53a0 con 0x7fd7100d2130 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.343+0000 7fd70e59c640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7100d1770 0x7fd7100b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.344+0000 7fd70dd9b640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7100d2130 0x7fd7101564d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800ebe0 tx=0x7fd6f800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.344+0000 7fd706ffd640 1 -- 172.21.15.40:0/239017856 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8017070 con 0x7fd7100d2130 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.344+0000 7fd706ffd640 1 -- 172.21.15.40:0/239017856 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f800cc30 con 0x7fd7100d2130 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.344+0000 7fd706ffd640 1 -- 172.21.15.40:0/239017856 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8013470 con 0x7fd7100d2130 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.344+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7101f5680 con 0x7fd7100d2130 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.344+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101f5c40 con 0x7fd7100d2130 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.345+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7100694f0 con 0x7fd7100d2130 2024-04-26T21:42:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.345+0000 7fd706ffd640 1 -- 172.21.15.40:0/239017856 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6f801a070 con 0x7fd7100d2130 2024-04-26T21:42:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.346+0000 7fd706ffd640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f006c8f0 0x7fd6f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.346+0000 7fd706ffd640 1 -- 172.21.15.40:0/239017856 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7fd6f808bbf0 con 0x7fd7100d2130 2024-04-26T21:42:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.346+0000 7fd70e59c640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f006c8f0 0x7fd6f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.347+0000 7fd70e59c640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f006c8f0 0x7fd6f006edb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd708034000 tx=0x7fd70802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:05.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.350+0000 7fd706ffd640 1 -- 172.21.15.40:0/239017856 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6f805a270 con 0x7fd7100d2130 2024-04-26T21:42:05.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.606+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd7100b6860 con 0x7fd7100d2130 2024-04-26T21:42:05.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.606+0000 7fd706ffd640 1 -- 172.21.15.40:0/239017856 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fd6f8059b70 con 0x7fd7100d2130 2024-04-26T21:42:05.606 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmesG+ERAAJS53jYoKUawIOs2zrG5TLg==", 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:42:05.607 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmq9cvHRAAiYWrMjfAth8wMbs9YlfWgQ==", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmLRmvGBAA6xAcjLDF7DMjfDqiUCexKA==", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:42:05.608 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmXxkDJBAAbzzf+LUO3Rr8bdzXb+Gj2Q==", 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:42:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDYHSxmRkhFBhAAz1UfvJwhA8bolDiRN2eM+Q==", 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDbHSxmKAB/BBAAzW5dtz6eczjkDiTQBjfjuA==", 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDeHSxmrienAhAA2JS0l6/Uo9UPc0TeZ1RwRA==", 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDhHSxmv40rARAAN8z+beJe/bTERvO2euMr6w==", 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:42:05.611 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDnHSxmxQ1XCBAAYptVM8WPir+zIHOjbX0tGg==", 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDqHSxmv32IBhAAMROkCLjWTOx3ELdkEiVSmw==", 2024-04-26T21:42:05.612 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDtHSxmJdPTBBAAO32qvVLuAzqx1xdajL9CFw==", 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:42:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDwHSxmcUMYAxAAIvoAo82ApDXERodD+4+cBQ==", 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw==", 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:42:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmifWYMBAA6Nskvqt3MZSvFCMHjslELw==", 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.615 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmi6l6LRAAF4EjChVEB8ReULEsl3YppA==", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmhSMFNhAAjCJvc4FknsSUJQ4NjXRV6Q==", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:42:05.616 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDTHSxmQk1bMhAAixlszeAmlXc5cAmkFWjhfw==", 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:42:05.617 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmYjmPDhAADvvIHrcacI8kOAdeiJGd+Q==", 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmsiwICxAAmKAZ2KS3/dYubutkcGfolA==", 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:42:05.618 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:42:05.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:05.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:42:05.619 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:42:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f006c8f0 msgr2=0x7fd6f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f006c8f0 0x7fd6f006edb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd708034000 tx=0x7fd70802e000 comp rx=0 tx=0).stop 2024-04-26T21:42:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7100d2130 msgr2=0x7fd7101564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7100d2130 0x7fd7101564d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800ebe0 tx=0x7fd6f800c520 comp rx=0 tx=0).stop 2024-04-26T21:42:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 shutdown_connections 2024-04-26T21:42:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6f006c8f0 0x7fd6f006edb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd71015f3a0 0x7fd7100cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7100d2130 0x7fd7101564d0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 --2- 172.21.15.40:0/239017856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7100d1770 0x7fd7100b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 >> 172.21.15.40:0/239017856 conn(0x7fd7100d4f90 msgr2=0x7fd710074fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 shutdown_connections 2024-04-26T21:42:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.608+0000 7fd714a1d640 1 -- 172.21.15.40:0/239017856 wait complete. 2024-04-26T21:42:05.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:42:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.723+0000 7fd8fc862640 1 Processor -- start 2024-04-26T21:42:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.723+0000 7fd8fc862640 1 -- start start 2024-04-26T21:42:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8fc862640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f80cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8fc862640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd8f806fea0 0x7fd8f8072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8fc862640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f8156840 con 0x7fd8f80cba10 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8fc862640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f80cbf20 con 0x7fd8f806fea0 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8fc862640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f80cc200 con 0x7fd8f8156460 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f80cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f6d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd8f806fea0 0x7fd8f8072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f80cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:40474/0 (socket says 172.21.15.40:40474) 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f6d9d640 1 --2- 172.21.15.40:0/122393605 >> v2:172.21.15.184:3300/0 conn(0x7fd8f806fea0 0x7fd8f8072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:33520/0 (socket says 172.21.15.40:33520) 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f659c640 1 -- 172.21.15.40:0/122393605 learned_addr learned my addr 172.21.15.40:0/122393605 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f6d9d640 1 -- 172.21.15.40:0/122393605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 msgr2=0x7fd8f80cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f6d9d640 1 --2- 172.21.15.40:0/122393605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f80cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f6d9d640 1 -- 172.21.15.40:0/122393605 >> v1:172.21.15.184:6789/0 conn(0x7fd8f80cba10 legacy=0x7fd8f806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f6d9d640 1 -- 172.21.15.40:0/122393605 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f80cc4e0 con 0x7fd8f806fea0 2024-04-26T21:42:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.724+0000 7fd8f5d9b640 1 -- 172.21.15.40:0/122393605 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1259100342 0 0) 0x7fd8f8156840 con 0x7fd8f80cba10 2024-04-26T21:42:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.725+0000 7fd8f6d9d640 1 --2- 172.21.15.40:0/122393605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f806fea0 0x7fd8f8072c50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009d90 tx=0x7fd8f002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5381e5eb9da3c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.725+0000 7fd8f5d9b640 1 -- 172.21.15.40:0/122393605 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0007740 con 0x7fd8f806fea0 2024-04-26T21:42:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.725+0000 7fd8f5d9b640 1 -- 172.21.15.40:0/122393605 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f00078e0 con 0x7fd8f806fea0 2024-04-26T21:42:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.725+0000 7fd8f5d9b640 1 -- 172.21.15.40:0/122393605 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0007cb0 con 0x7fd8f806fea0 2024-04-26T21:42:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.726+0000 7fd8fc862640 1 -- 172.21.15.40:0/122393605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f806fea0 msgr2=0x7fd8f8072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.726+0000 7fd8fc862640 1 --2- 172.21.15.40:0/122393605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f806fea0 0x7fd8f8072c50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009d90 tx=0x7fd8f002a660 comp rx=0 tx=0).stop 2024-04-26T21:42:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.726+0000 7fd8fc862640 1 -- 172.21.15.40:0/122393605 shutdown_connections 2024-04-26T21:42:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.726+0000 7fd8fc862640 1 --2- 172.21.15.40:0/122393605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f806fea0 0x7fd8f8072c50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.726+0000 7fd8fc862640 1 --2- 172.21.15.40:0/122393605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f80cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.726+0000 7fd8fc862640 1 -- 172.21.15.40:0/122393605 >> 172.21.15.40:0/122393605 conn(0x7fd8f80d0be0 msgr2=0x7fd8f80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.726+0000 7fd8fc862640 1 -- 172.21.15.40:0/122393605 shutdown_connections 2024-04-26T21:42:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.726+0000 7fd8fc862640 1 -- 172.21.15.40:0/122393605 wait complete. 2024-04-26T21:42:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.727+0000 7fd8fc862640 1 Processor -- start 2024-04-26T21:42:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.727+0000 7fd8fc862640 1 -- start start 2024-04-26T21:42:05.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8fc862640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f806fea0 0x7fd8f8168950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8fc862640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f80cba10 0x7fd8f80c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8fc862640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f8167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8fc862640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f806e260 con 0x7fd8f806fea0 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8fc862640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8072700 con 0x7fd8f80cba10 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8fc862640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f806e3e0 con 0x7fd8f8156460 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f806fea0 0x7fd8f8168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f806fea0 0x7fd8f8168950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44148/0 (socket says 172.21.15.40:44148) 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f659c640 1 -- 172.21.15.40:0/1208195818 learned_addr learned my addr 172.21.15.40:0/1208195818 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f6d9d640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f8167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8effff640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f80cba10 0x7fd8f80c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f659c640 1 -- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 msgr2=0x7fd8f8167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f659c640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f8167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f659c640 1 -- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f80cba10 msgr2=0x7fd8f80c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f659c640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f80cba10 0x7fd8f80c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.728+0000 7fd8f659c640 1 -- 172.21.15.40:0/1208195818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f80c40a0 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.729+0000 7fd8f659c640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f806fea0 0x7fd8f8168950 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e840 tx=0x7fd8e800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.729+0000 7fd8eeffd640 1 -- 172.21.15.40:0/1208195818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8017070 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.729+0000 7fd8eeffd640 1 -- 172.21.15.40:0/1208195818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e800c8e0 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.729+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f815ef10 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.729+0000 7fd8eeffd640 1 -- 172.21.15.40:0/1208195818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800ccd0 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.729+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f80af530 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.730+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f80694f0 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.730+0000 7fd8eeffd640 1 -- 172.21.15.40:0/1208195818 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8e801a070 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.731+0000 7fd8eeffd640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8d806c8f0 0x7fd8d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.731+0000 7fd8eeffd640 1 -- 172.21.15.40:0/1208195818 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7fd8e808bc80 con 0x7fd8f806fea0 2024-04-26T21:42:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.731+0000 7fd8effff640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8d806c8f0 0x7fd8d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.731+0000 7fd8effff640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8d806c8f0 0x7fd8d806edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80b0ac0 tx=0x7fd8e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.735+0000 7fd8eeffd640 1 -- 172.21.15.40:0/1208195818 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8e805a1e0 con 0x7fd8f806fea0 2024-04-26T21:42:05.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.926+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8f80cfcd0 con 0x7fd8f806fea0 2024-04-26T21:42:05.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.927+0000 7fd8eeffd640 1 -- 172.21.15.40:0/1208195818 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+13129 (secure 0 0 0) 0x7fd8e8059ae0 con 0x7fd8f806fea0 2024-04-26T21:42:05.926 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:05.927 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:42:05.210863+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.928+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8d806c8f0 msgr2=0x7fd8d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.928+0000 7fd8fc862640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8d806c8f0 0x7fd8d806edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80b0ac0 tx=0x7fd8e0009040 comp rx=0 tx=0).stop 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.928+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f806fea0 msgr2=0x7fd8f8168950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.928+0000 7fd8fc862640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f806fea0 0x7fd8f8168950 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e840 tx=0x7fd8e800ed00 comp rx=0 tx=0).stop 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.929+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 shutdown_connections 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.929+0000 7fd8fc862640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8d806c8f0 0x7fd8d806edb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.929+0000 7fd8fc862640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f8156460 0x7fd8f8167fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.929+0000 7fd8fc862640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f80cba10 0x7fd8f80c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.929+0000 7fd8fc862640 1 --2- 172.21.15.40:0/1208195818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f806fea0 0x7fd8f8168950 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.929+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 >> 172.21.15.40:0/1208195818 conn(0x7fd8f80d0be0 msgr2=0x7fd8f80c9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.929+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 shutdown_connections 2024-04-26T21:42:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:05.929+0000 7fd8fc862640 1 -- 172.21.15.40:0/1208195818 wait complete. 2024-04-26T21:42:05.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:42:05.940 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:42:06.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.048+0000 7f8c18cb0640 1 Processor -- start 2024-04-26T21:42:06.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.048+0000 7f8c18cb0640 1 -- start start 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c18cb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1406fc40 0x7f8c14073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c18cb0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c14156840 con 0x7f8c14156460 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c18cb0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c14070090 con 0x7f8c1406f790 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c18cb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c140cbf20 con 0x7f8c1406fc40 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c1259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8c14156460 0x7f8c140cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51600/0 (socket says 172.21.15.40:51600) 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c1259c640 1 -- 172.21.15.40:0/2017128070 learned_addr learned my addr 172.21.15.40:0/2017128070 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4026709965 0 0) 0x7f8c14156840 con 0x7f8c14156460 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c12d9d640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1406fc40 0x7f8c14073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.049+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bfc003520 con 0x7f8c14156460 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2469143301 0 0) 0x7f8bfc003520 con 0x7f8c14156460 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1406fc40 msgr2=0x7f8c14073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1406fc40 0x7f8c14073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 >> v1:172.21.15.184:6789/0 conn(0x7f8c1406f790 legacy=0x7f8c14072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c140cc200 con 0x7f8c14156460 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f8c0c001670 con 0x7f8c14156460 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 >> v1:172.21.15.40:6789/0 conn(0x7f8c14156460 legacy=0x7f8c140cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bfc0039e0 0x7f8bfc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bfc004800 0x7f8bfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1406fc40 0x7f8bfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c140cbf20 con 0x7f8bfc004800 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc00ba70 con 0x7f8bfc0039e0 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8bfc00bc70 con 0x7f8c1406fc40 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c11d9b640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bfc004800 0x7f8bfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c1259c640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bfc0039e0 0x7f8bfc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c12d9d640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1406fc40 0x7f8bfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c11d9b640 1 -- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1406fc40 msgr2=0x7f8bfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c11d9b640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1406fc40 0x7f8bfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c11d9b640 1 -- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bfc0039e0 msgr2=0x7f8bfc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c11d9b640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bfc0039e0 0x7f8bfc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.050+0000 7f8c11d9b640 1 -- 172.21.15.40:0/2017128070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc003520 con 0x7f8bfc004800 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.051+0000 7f8c11d9b640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bfc004800 0x7f8bfc004c80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8c08001720 tx=0x7f8bfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6d35c218169136 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.051+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08001370 con 0x7f8bfc004800 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.051+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c080052d0 con 0x7f8bfc004800 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.051+0000 7f8c1159a640 1 -- 172.21.15.40:0/2017128070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0802ab90 con 0x7f8bfc004800 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.052+0000 7f8c18cb0640 1 -- 172.21.15.40:0/2017128070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bfc004800 msgr2=0x7f8bfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.052+0000 7f8c18cb0640 1 --2- 172.21.15.40:0/2017128070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bfc004800 0x7f8bfc004c80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8c08001720 tx=0x7f8bfc005830 comp rx=0 tx=0).stop 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.052+0000 7f8c12d9d640 1 -- 172.21.15.40:0/2017128070 reap_dead start 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.052+0000 7f8c18cb0640 1 -- 172.21.15.40:0/2017128070 shutdown_connections 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.052+0000 7f8c18cb0640 1 -- 172.21.15.40:0/2017128070 >> 172.21.15.40:0/2017128070 conn(0x7f8c140d0be0 msgr2=0x7f8c140d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.052+0000 7f8c18cb0640 1 -- 172.21.15.40:0/2017128070 shutdown_connections 2024-04-26T21:42:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.052+0000 7f8c18cb0640 1 -- 172.21.15.40:0/2017128070 wait complete. 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.053+0000 7f8c18cb0640 1 Processor -- start 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.053+0000 7f8c18cb0640 1 -- start start 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c18cb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bfc004800 0x7f8c141479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c18cb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c140b0c50 0x7f8c1414b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c18cb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c140ae8d0 0x7f8c140aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c18cb0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14072700 con 0x7f8c140ae8d0 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c18cb0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c140cc7c0 con 0x7f8c140b0c50 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c18cb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c140cbbe0 con 0x7f8bfc004800 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c12d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c140ae8d0 0x7f8c140aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c12d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c140ae8d0 0x7f8c140aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44166/0 (socket says 172.21.15.40:44166) 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c12d9d640 1 -- 172.21.15.40:0/3570236724 learned_addr learned my addr 172.21.15.40:0/3570236724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c11d9b640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c140b0c50 0x7f8c1414b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c1259c640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bfc004800 0x7f8c141479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c12d9d640 1 -- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bfc004800 msgr2=0x7f8c141479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c12d9d640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bfc004800 0x7f8c141479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c12d9d640 1 -- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c140b0c50 msgr2=0x7f8c1414b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c12d9d640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c140b0c50 0x7f8c1414b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.054+0000 7f8c12d9d640 1 -- 172.21.15.40:0/3570236724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c140c47c0 con 0x7f8c140ae8d0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.055+0000 7f8c12d9d640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c140ae8d0 0x7f8c140aed50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f8c140b6d50 tx=0x7f8c0400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.055+0000 7f8c02ffd640 1 -- 172.21.15.40:0/3570236724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04015430 con 0x7f8c140ae8d0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.055+0000 7f8c02ffd640 1 -- 172.21.15.40:0/3570236724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0401d070 con 0x7f8c140ae8d0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.055+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c140b1460 con 0x7f8c140ae8d0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.055+0000 7f8c02ffd640 1 -- 172.21.15.40:0/3570236724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04015830 con 0x7f8c140ae8d0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.055+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c140c80c0 con 0x7f8c140ae8d0 2024-04-26T21:42:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.056+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c140694f0 con 0x7f8c140ae8d0 2024-04-26T21:42:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.056+0000 7f8c02ffd640 1 -- 172.21.15.40:0/3570236724 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c040159d0 con 0x7f8c140ae8d0 2024-04-26T21:42:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.057+0000 7f8c02ffd640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bf406c8f0 0x7f8bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.057+0000 7f8c02ffd640 1 -- 172.21.15.40:0/3570236724 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7f8c0408c250 con 0x7f8c140ae8d0 2024-04-26T21:42:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.057+0000 7f8c1259c640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bf406c8f0 0x7f8bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.058+0000 7f8c1259c640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bf406c8f0 0x7f8bf406edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c001cd0 tx=0x7f8c0c000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.061+0000 7f8c02ffd640 1 -- 172.21.15.40:0/3570236724 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c140c47c0 con 0x7f8c140ae8d0 2024-04-26T21:42:06.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.246+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c1406de80 con 0x7f8c140ae8d0 2024-04-26T21:42:06.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.247+0000 7f8c02ffd640 1 -- 172.21.15.40:0/3570236724 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f8c04010040 con 0x7f8c140ae8d0 2024-04-26T21:42:06.247 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.248+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bf406c8f0 msgr2=0x7f8bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.248+0000 7f8c18cb0640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bf406c8f0 0x7f8bf406edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c001cd0 tx=0x7f8c0c000ed0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.248+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c140ae8d0 msgr2=0x7f8c140aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.248+0000 7f8c18cb0640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c140ae8d0 0x7f8c140aed50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f8c140b6d50 tx=0x7f8c0400f580 comp rx=0 tx=0).stop 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.249+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 shutdown_connections 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.249+0000 7f8c18cb0640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bf406c8f0 0x7f8bf406edb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.249+0000 7f8c18cb0640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c140ae8d0 0x7f8c140aed50 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.249+0000 7f8c18cb0640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c140b0c50 0x7f8c1414b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.249+0000 7f8c18cb0640 1 --2- 172.21.15.40:0/3570236724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bfc004800 0x7f8c141479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.249+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 >> 172.21.15.40:0/3570236724 conn(0x7f8c140d0be0 msgr2=0x7f8c1406e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.249+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 shutdown_connections 2024-04-26T21:42:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.249+0000 7f8c18cb0640 1 -- 172.21.15.40:0/3570236724 wait complete. 2024-04-26T21:42:06.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:42:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.361+0000 7f7c65407640 1 Processor -- start 2024-04-26T21:42:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.361+0000 7f7c65407640 1 -- start start 2024-04-26T21:42:06.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c65407640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c60162880 0x7f7c600d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c65407640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7c600d3bc0 0x7f7c600b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c65407640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c600b7e50 con 0x7f7c600b5f20 2024-04-26T21:42:06.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c65407640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7c600d0580 con 0x7f7c600d3bc0 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c65407640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7c600d1800 con 0x7f7c60162880 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c5e7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7c600b5f20 0x7f7c600d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51606/0 (socket says 172.21.15.40:51606) 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c5e7fc640 1 -- 172.21.15.40:0/289116269 learned_addr learned my addr 172.21.15.40:0/289116269 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c5f7fe640 1 --2- 172.21.15.40:0/289116269 >> v2:172.21.15.184:3300/0 conn(0x7f7c600d3bc0 0x7f7c600b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c5effd640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c60162880 0x7f7c600d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3319364427 0 0) 0x7f7c600b7e50 con 0x7f7c600b5f20 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.362+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c48003520 con 0x7f7c600b5f20 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3507760214 0 0) 0x7f7c48003520 con 0x7f7c600b5f20 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c60162880 msgr2=0x7f7c600d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c60162880 0x7f7c600d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 >> v2:172.21.15.184:3300/0 conn(0x7f7c600d3bc0 msgr2=0x7f7c600b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 --2- 172.21.15.40:0/289116269 >> v2:172.21.15.184:3300/0 conn(0x7f7c600d3bc0 0x7f7c600b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c600d07d0 con 0x7f7c600b5f20 2024-04-26T21:42:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f7c54001670 con 0x7f7c600b5f20 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 >> v1:172.21.15.40:6789/0 conn(0x7f7c600b5f20 legacy=0x7f7c600d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c60162880 0x7f7c48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c48004d40 0x7f7c480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600d0580 con 0x7f7c480039b0 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600d1800 con 0x7f7c48004d40 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4800b9c0 con 0x7f7c60162880 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5effd640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5e7fc640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c60162880 0x7f7c48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5f7fe640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c48004d40 0x7f7c480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5effd640 1 -- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c60162880 msgr2=0x7f7c48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5effd640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c60162880 0x7f7c48004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5effd640 1 -- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c48004d40 msgr2=0x7f7c480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5effd640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c48004d40 0x7f7c480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.363+0000 7f7c5effd640 1 -- 172.21.15.40:0/289116269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48003520 con 0x7f7c480039b0 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c5effd640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009ac0 tx=0x7f7c48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe5c596c48ae51c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50036070 con 0x7f7c480039b0 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c50031400 con 0x7f7c480039b0 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c5dffb640 1 -- 172.21.15.40:0/289116269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50009690 con 0x7f7c480039b0 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c65407640 1 -- 172.21.15.40:0/289116269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c480039b0 msgr2=0x7f7c48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c65407640 1 --2- 172.21.15.40:0/289116269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009ac0 tx=0x7f7c48005830 comp rx=0 tx=0).stop 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c5f7fe640 1 -- 172.21.15.40:0/289116269 reap_dead start 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c65407640 1 -- 172.21.15.40:0/289116269 shutdown_connections 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c65407640 1 -- 172.21.15.40:0/289116269 >> 172.21.15.40:0/289116269 conn(0x7f7c600d4f60 msgr2=0x7f7c600d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c65407640 1 -- 172.21.15.40:0/289116269 shutdown_connections 2024-04-26T21:42:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.364+0000 7f7c65407640 1 -- 172.21.15.40:0/289116269 wait complete. 2024-04-26T21:42:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.366+0000 7f7c65407640 1 Processor -- start 2024-04-26T21:42:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.366+0000 7f7c65407640 1 -- start start 2024-04-26T21:42:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c65407640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c480039b0 0x7f7c601616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c65407640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c600b3fc0 0x7f7c600b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c65407640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c601f86a0 0x7f7c601f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c65407640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60162c60 con 0x7f7c601f86a0 2024-04-26T21:42:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c65407640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600d0d90 con 0x7f7c600b3fc0 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c65407640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7c60162de0 con 0x7f7c480039b0 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c601f86a0 0x7f7c601f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c601f86a0 0x7f7c601f8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44178/0 (socket says 172.21.15.40:44178) 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 -- 172.21.15.40:0/3793963035 learned_addr learned my addr 172.21.15.40:0/3793963035 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5effd640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c480039b0 0x7f7c601616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5e7fc640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c600b3fc0 0x7f7c600b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 -- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c480039b0 msgr2=0x7f7c601616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c480039b0 0x7f7c601616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 -- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c600b3fc0 msgr2=0x7f7c600b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c600b3fc0 0x7f7c600b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 -- 172.21.15.40:0/3793963035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c601f4e90 con 0x7f7c601f86a0 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c5f7fe640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c601f86a0 0x7f7c601f8b20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7c600b6930 tx=0x7f7c580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c477fe640 1 -- 172.21.15.40:0/3793963035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58011e40 con 0x7f7c601f86a0 2024-04-26T21:42:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c477fe640 1 -- 172.21.15.40:0/3793963035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5801b070 con 0x7f7c601f86a0 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c601f5c10 con 0x7f7c601f86a0 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c477fe640 1 -- 172.21.15.40:0/3793963035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58003990 con 0x7f7c601f86a0 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.367+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c601f5410 con 0x7f7c601f86a0 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.368+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c600694f0 con 0x7f7c601f86a0 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.369+0000 7f7c477fe640 1 -- 172.21.15.40:0/3793963035 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c58005d10 con 0x7f7c601f86a0 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.369+0000 7f7c477fe640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.370+0000 7f7c477fe640 1 -- 172.21.15.40:0/3793963035 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7f7c5808bbc0 con 0x7f7c601f86a0 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.370+0000 7f7c5effd640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.370+0000 7f7c5effd640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7c50033000 tx=0x7f7c50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.374+0000 7f7c477fe640 1 -- 172.21.15.40:0/3793963035 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c58059b60 con 0x7f7c601f86a0 2024-04-26T21:42:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.567+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7c600d3030 con 0x7f7c601f86a0 2024-04-26T21:42:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.567+0000 7f7c477fe640 1 -- 172.21.15.40:0/3793963035 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v90) ==== 84+0+65 (secure 0 0 0) 0x7f7c5805de60 con 0x7f7c601f86a0 2024-04-26T21:42:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:06.567 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:42:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:42:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:42:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:42:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:42:06.568 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7c4c06c8f0 msgr2=0x7f7c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7c50033000 tx=0x7f7c50008030 comp rx=0 tx=0).stop 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c601f86a0 msgr2=0x7f7c601f8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c601f86a0 0x7f7c601f8b20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7c600b6930 tx=0x7f7c580115a0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 shutdown_connections 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c601f86a0 0x7f7c601f8b20 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c600b3fc0 0x7f7c600b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 --2- 172.21.15.40:0/3793963035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c480039b0 0x7f7c601616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.569+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 >> 172.21.15.40:0/3793963035 conn(0x7f7c600d4f60 msgr2=0x7f7c600d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.570+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 shutdown_connections 2024-04-26T21:42:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.570+0000 7f7c65407640 1 -- 172.21.15.40:0/3793963035 wait complete. 2024-04-26T21:42:06.580 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:42:06.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:42:06.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.684+0000 7f92ca778640 1 Processor -- start 2024-04-26T21:42:06.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.684+0000 7f92ca778640 1 -- start start 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.685+0000 7f92ca778640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92c415f7c0 0x7f92c40d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.685+0000 7f92ca778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 0x7f92c41620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.685+0000 7f92ca778640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92c4160930 con 0x7f92c415f3e0 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.685+0000 7f92ca778640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f92c415fd20 con 0x7f92c415f7c0 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.685+0000 7f92ca778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92c4160000 con 0x7f92c40d3bd0 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.685+0000 7f92c37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92c415f7c0 0x7f92c40d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.685+0000 7f92c37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92c415f7c0 0x7f92c40d3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44186/0 (socket says 172.21.15.40:44186) 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.685+0000 7f92c37fe640 1 -- 172.21.15.40:0/1536232813 learned_addr learned my addr 172.21.15.40:0/1536232813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:06.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.686+0000 7f92c8d15640 1 --2- 172.21.15.40:0/1536232813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 0x7f92c41620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.686+0000 7f92c37fe640 1 -- 172.21.15.40:0/1536232813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 msgr2=0x7f92c41620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:06.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.686+0000 7f92c37fe640 1 --2- 172.21.15.40:0/1536232813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 0x7f92c41620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.686+0000 7f92c37fe640 1 -- 172.21.15.40:0/1536232813 >> v1:172.21.15.184:6789/0 conn(0x7f92c415f3e0 legacy=0x7f92c40d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.686+0000 7f92c37fe640 1 -- 172.21.15.40:0/1536232813 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c41602e0 con 0x7f92c415f7c0 2024-04-26T21:42:06.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.686+0000 7f92c37fe640 1 --2- 172.21.15.40:0/1536232813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f7c0 0x7f92c40d3690 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009870 tx=0x7f92b802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f37c806317b9777 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92c2ffd640 1 -- 172.21.15.40:0/1536232813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8037070 con 0x7f92c415f7c0 2024-04-26T21:42:06.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92c2ffd640 1 -- 172.21.15.40:0/1536232813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b8032440 con 0x7f92c415f7c0 2024-04-26T21:42:06.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92c2ffd640 1 -- 172.21.15.40:0/1536232813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8032810 con 0x7f92c415f7c0 2024-04-26T21:42:06.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92ca778640 1 -- 172.21.15.40:0/1536232813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f7c0 msgr2=0x7f92c40d3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92ca778640 1 --2- 172.21.15.40:0/1536232813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f7c0 0x7f92c40d3690 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009870 tx=0x7f92b802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92ca778640 1 -- 172.21.15.40:0/1536232813 shutdown_connections 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92ca778640 1 --2- 172.21.15.40:0/1536232813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 0x7f92c41620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92ca778640 1 --2- 172.21.15.40:0/1536232813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f7c0 0x7f92c40d3690 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92ca778640 1 -- 172.21.15.40:0/1536232813 >> 172.21.15.40:0/1536232813 conn(0x7f92c409e3c0 msgr2=0x7f92c40d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92ca778640 1 -- 172.21.15.40:0/1536232813 shutdown_connections 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.687+0000 7f92ca778640 1 -- 172.21.15.40:0/1536232813 wait complete. 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.688+0000 7f92ca778640 1 Processor -- start 2024-04-26T21:42:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92ca778640 1 -- start start 2024-04-26T21:42:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92ca778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 0x7f92c41f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92ca778640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f3e0 0x7f92c415df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92ca778640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c415f7c0 0x7f92c41e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92ca778640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92c40d33b0 con 0x7f92c415f3e0 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92ca778640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92c4160d40 con 0x7f92c415f7c0 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92ca778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92c41625e0 con 0x7f92c40d3bd0 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92c37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f3e0 0x7f92c415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92c37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f3e0 0x7f92c415df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44202/0 (socket says 172.21.15.40:44202) 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.689+0000 7f92c37fe640 1 -- 172.21.15.40:0/2838519597 learned_addr learned my addr 172.21.15.40:0/2838519597 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c3fff640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 0x7f92c41f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c8d15640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c415f7c0 0x7f92c41e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c37fe640 1 -- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 msgr2=0x7f92c41f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c37fe640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 0x7f92c41f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c37fe640 1 -- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c415f7c0 msgr2=0x7f92c41e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c37fe640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c415f7c0 0x7f92c41e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c37fe640 1 -- 172.21.15.40:0/2838519597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c41f5a90 con 0x7f92c415f3e0 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c8d15640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c415f7c0 0x7f92c41e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c37fe640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f3e0 0x7f92c415df90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f92b802a200 tx=0x7f92b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c0ff9640 1 -- 172.21.15.40:0/2838519597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8041030 con 0x7f92c415f3e0 2024-04-26T21:42:06.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c0ff9640 1 -- 172.21.15.40:0/2838519597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b8030440 con 0x7f92c415f3e0 2024-04-26T21:42:06.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c40ca2c0 con 0x7f92c415f3e0 2024-04-26T21:42:06.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92c0ff9640 1 -- 172.21.15.40:0/2838519597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8037040 con 0x7f92c415f3e0 2024-04-26T21:42:06.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.690+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c40cab60 con 0x7f92c415f3e0 2024-04-26T21:42:06.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.691+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c4069600 con 0x7f92c415f3e0 2024-04-26T21:42:06.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.692+0000 7f92c0ff9640 1 -- 172.21.15.40:0/2838519597 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92b8044070 con 0x7f92c415f3e0 2024-04-26T21:42:06.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.692+0000 7f92c0ff9640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92b006c8f0 0x7f92b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:06.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.692+0000 7f92c0ff9640 1 -- 172.21.15.40:0/2838519597 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7f92b80abfe0 con 0x7f92c415f3e0 2024-04-26T21:42:06.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.692+0000 7f92c3fff640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92b006c8f0 0x7f92b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:06.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.693+0000 7f92c3fff640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92b006c8f0 0x7f92b006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f92b4007fd0 tx=0x7f92b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.697+0000 7f92c0ff9640 1 -- 172.21.15.40:0/2838519597 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92b807a660 con 0x7f92c415f3e0 2024-04-26T21:42:06.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:06.879+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f92c40d57e0 con 0x7f92c415f3e0 2024-04-26T21:42:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.222+0000 7f92c0ff9640 1 -- 172.21.15.40:0/2838519597 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v91) ==== 138+0+0 (secure 0 0 0) 0x7f92b8079f60 con 0x7f92c415f3e0 2024-04-26T21:42:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:42:07.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.224+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92b006c8f0 msgr2=0x7f92b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:07.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.224+0000 7f92ca778640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92b006c8f0 0x7f92b006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f92b4007fd0 tx=0x7f92b4009040 comp rx=0 tx=0).stop 2024-04-26T21:42:07.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.224+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f3e0 msgr2=0x7f92c415df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:07.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.224+0000 7f92ca778640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f3e0 0x7f92c415df90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f92b802a200 tx=0x7f92b80046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.225+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 shutdown_connections 2024-04-26T21:42:07.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.225+0000 7f92ca778640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f92b006c8f0 0x7f92b006edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.225+0000 7f92ca778640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c415f7c0 0x7f92c41e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.225+0000 7f92ca778640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c415f3e0 0x7f92c415df90 secure :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f92b802a200 tx=0x7f92b80046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.225+0000 7f92ca778640 1 --2- 172.21.15.40:0/2838519597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c40d3bd0 0x7f92c41f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.225+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 >> 172.21.15.40:0/2838519597 conn(0x7f92c409e3c0 msgr2=0x7f92c406e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:07.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.225+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 shutdown_connections 2024-04-26T21:42:07.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.226+0000 7f92ca778640 1 -- 172.21.15.40:0/2838519597 wait complete. 2024-04-26T21:42:07.239 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:42:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.348+0000 7f89f251b640 1 Processor -- start 2024-04-26T21:42:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.348+0000 7f89f251b640 1 -- start start 2024-04-26T21:42:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f251b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f89ec15f550 0x7f89ec0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f251b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89ec0d3bc0 0x7f89ec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f251b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ec0d1040 con 0x7f89ec15f170 2024-04-26T21:42:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f251b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec0c9430 con 0x7f89ec0d3bc0 2024-04-26T21:42:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f251b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec0c9680 con 0x7f89ec15f550 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f0ab8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89ec0d3bc0 0x7f89ec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f0ab8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89ec0d3bc0 0x7f89ec0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44214/0 (socket says 172.21.15.40:44214) 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f0ab8640 1 -- 172.21.15.40:0/1557342945 learned_addr learned my addr 172.21.15.40:0/1557342945 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89eb7fe640 1 --2- 172.21.15.40:0/1557342945 >> v2:172.21.15.184:3300/0 conn(0x7f89ec15f550 0x7f89ec0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f0ab8640 1 -- 172.21.15.40:0/1557342945 >> v2:172.21.15.184:3300/0 conn(0x7f89ec15f550 msgr2=0x7f89ec0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f0ab8640 1 --2- 172.21.15.40:0/1557342945 >> v2:172.21.15.184:3300/0 conn(0x7f89ec15f550 0x7f89ec0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f0ab8640 1 -- 172.21.15.40:0/1557342945 >> v1:172.21.15.184:6789/0 conn(0x7f89ec15f170 legacy=0x7f89ec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89f0ab8640 1 -- 172.21.15.40:0/1557342945 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec0c9960 con 0x7f89ec0d3bc0 2024-04-26T21:42:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.349+0000 7f89eb7fe640 1 --2- 172.21.15.40:0/1557342945 >> v2:172.21.15.184:3300/0 conn(0x7f89ec15f550 0x7f89ec0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.350+0000 7f89f0ab8640 1 --2- 172.21.15.40:0/1557342945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec0d3bc0 0x7f89ec0d2cb0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009e50 tx=0x7f89e402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58938036954054b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.350+0000 7f89eaffd640 1 -- 172.21.15.40:0/1557342945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4036070 con 0x7f89ec0d3bc0 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.350+0000 7f89eaffd640 1 -- 172.21.15.40:0/1557342945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e4031600 con 0x7f89ec0d3bc0 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.350+0000 7f89eaffd640 1 -- 172.21.15.40:0/1557342945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40319d0 con 0x7f89ec0d3bc0 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.351+0000 7f89f251b640 1 -- 172.21.15.40:0/1557342945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec0d3bc0 msgr2=0x7f89ec0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.351+0000 7f89f251b640 1 --2- 172.21.15.40:0/1557342945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec0d3bc0 0x7f89ec0d2cb0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009e50 tx=0x7f89e402a710 comp rx=0 tx=0).stop 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.351+0000 7f89f251b640 1 -- 172.21.15.40:0/1557342945 shutdown_connections 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.351+0000 7f89f251b640 1 --2- 172.21.15.40:0/1557342945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec0d3bc0 0x7f89ec0d2cb0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.351+0000 7f89f251b640 1 --2- 172.21.15.40:0/1557342945 >> v2:172.21.15.184:3300/0 conn(0x7f89ec15f550 0x7f89ec0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.351+0000 7f89f251b640 1 -- 172.21.15.40:0/1557342945 >> 172.21.15.40:0/1557342945 conn(0x7f89ec0d4f60 msgr2=0x7f89ec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.351+0000 7f89f251b640 1 -- 172.21.15.40:0/1557342945 shutdown_connections 2024-04-26T21:42:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.351+0000 7f89f251b640 1 -- 172.21.15.40:0/1557342945 wait complete. 2024-04-26T21:42:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.352+0000 7f89f251b640 1 Processor -- start 2024-04-26T21:42:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.352+0000 7f89f251b640 1 -- start start 2024-04-26T21:42:07.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89f251b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0d3bc0 0x7f89ec0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:07.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89f251b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec15f170 0x7f89ec15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89f251b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec0b7e20 0x7f89ec156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89f251b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0ca5c0 con 0x7f89ec15f170 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89f251b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0ca440 con 0x7f89ec0b7e20 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89f251b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89ec0ca740 con 0x7f89ec0d3bc0 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec15f170 0x7f89ec15a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec15f170 0x7f89ec15a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44224/0 (socket says 172.21.15.40:44224) 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89eb7fe640 1 -- 172.21.15.40:0/410716938 learned_addr learned my addr 172.21.15.40:0/410716938 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89ebfff640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0d3bc0 0x7f89ec0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89f0ab8640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec0b7e20 0x7f89ec156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89eb7fe640 1 -- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0d3bc0 msgr2=0x7f89ec0b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89eb7fe640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0d3bc0 0x7f89ec0b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89eb7fe640 1 -- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec0b7e20 msgr2=0x7f89ec156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89eb7fe640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec0b7e20 0x7f89ec156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89eb7fe640 1 -- 172.21.15.40:0/410716938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec156bf0 con 0x7f89ec15f170 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89ebfff640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0d3bc0 0x7f89ec0b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.353+0000 7f89f0ab8640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec0b7e20 0x7f89ec156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.354+0000 7f89eb7fe640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec15f170 0x7f89ec15a2b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c7c0 tx=0x7f89dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.354+0000 7f89e8ff9640 1 -- 172.21.15.40:0/410716938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc015070 con 0x7f89ec15f170 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.354+0000 7f89e8ff9640 1 -- 172.21.15.40:0/410716938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89dc007820 con 0x7f89ec15f170 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.354+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec0b8320 con 0x7f89ec15f170 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.354+0000 7f89e8ff9640 1 -- 172.21.15.40:0/410716938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc007bf0 con 0x7f89ec15f170 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.354+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec0b88e0 con 0x7f89ec15f170 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.355+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec0694f0 con 0x7f89ec15f170 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.355+0000 7f89e8ff9640 1 -- 172.21.15.40:0/410716938 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89dc018070 con 0x7f89ec15f170 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.356+0000 7f89e8ff9640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f89d806c920 0x7f89d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:07.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.356+0000 7f89e8ff9640 1 -- 172.21.15.40:0/410716938 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5619+0+0 (secure 0 0 0) 0x7f89dc08b880 con 0x7f89ec15f170 2024-04-26T21:42:07.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.356+0000 7f89ebfff640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f89d806c920 0x7f89d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:07.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.357+0000 7f89ebfff640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f89d806c920 0x7f89d806ede0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f89e0003eb0 tx=0x7f89e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.361+0000 7f89e8ff9640 1 -- 172.21.15.40:0/410716938 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89dc059d90 con 0x7f89ec15f170 2024-04-26T21:42:07.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:07.547+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f89ec0d1f60 con 0x7f89ec15f170 2024-04-26T21:42:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.228+0000 7f89e8ff9640 1 -- 172.21.15.40:0/410716938 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v92) ==== 170+0+0 (secure 0 0 0) 0x7f89dc059690 con 0x7f89ec15f170 2024-04-26T21:42:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:42:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.232+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f89d806c920 msgr2=0x7f89d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.232+0000 7f89f251b640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f89d806c920 0x7f89d806ede0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f89e0003eb0 tx=0x7f89e0009040 comp rx=0 tx=0).stop 2024-04-26T21:42:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.232+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec15f170 msgr2=0x7f89ec15a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.232+0000 7f89f251b640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec15f170 0x7f89ec15a2b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c7c0 tx=0x7f89dc00cc80 comp rx=0 tx=0).stop 2024-04-26T21:42:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.233+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 shutdown_connections 2024-04-26T21:42:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.233+0000 7f89f251b640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f89d806c920 0x7f89d806ede0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.233+0000 7f89f251b640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec0b7e20 0x7f89ec156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.233+0000 7f89f251b640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89ec15f170 0x7f89ec15a2b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.233+0000 7f89f251b640 1 --2- 172.21.15.40:0/410716938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0d3bc0 0x7f89ec0b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.233+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 >> 172.21.15.40:0/410716938 conn(0x7f89ec0d4f60 msgr2=0x7f89ec0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.235+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 shutdown_connections 2024-04-26T21:42:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.235+0000 7f89f251b640 1 -- 172.21.15.40:0/410716938 wait complete. 2024-04-26T21:42:08.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:42:08.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.370+0000 7fca241af640 1 Processor -- start 2024-04-26T21:42:08.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.370+0000 7fca241af640 1 -- start start 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca241af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca1c070350 0x7fca1c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca241af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca1c071090 0x7fca1c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca241af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca1c0d2390 con 0x7fca1c0cfde0 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca241af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fca1c0afab0 con 0x7fca1c070350 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca241af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fca1c06dd20 con 0x7fca1c071090 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca2174a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca1c070350 0x7fca1c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca2174a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca1c070350 0x7fca1c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44236/0 (socket says 172.21.15.40:44236) 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca2174a640 1 -- 172.21.15.40:0/2228676742 learned_addr learned my addr 172.21.15.40:0/2228676742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca2174a640 1 -- 172.21.15.40:0/2228676742 >> v2:172.21.15.184:3300/0 conn(0x7fca1c071090 msgr2=0x7fca1c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca2174a640 1 --2- 172.21.15.40:0/2228676742 >> v2:172.21.15.184:3300/0 conn(0x7fca1c071090 0x7fca1c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca2174a640 1 -- 172.21.15.40:0/2228676742 >> v1:172.21.15.184:6789/0 conn(0x7fca1c0cfde0 legacy=0x7fca1c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:08.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.371+0000 7fca2174a640 1 -- 172.21.15.40:0/2228676742 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c06d3f0 con 0x7fca1c070350 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.372+0000 7fca2174a640 1 --2- 172.21.15.40:0/2228676742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c070350 0x7fca1c070b50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0c6860 tx=0x7fca0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3557fe85f2aedec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.372+0000 7fca20f49640 1 -- 172.21.15.40:0/2228676742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037070 con 0x7fca1c070350 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.372+0000 7fca20f49640 1 -- 172.21.15.40:0/2228676742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c032440 con 0x7fca1c070350 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.372+0000 7fca20f49640 1 -- 172.21.15.40:0/2228676742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c032810 con 0x7fca1c070350 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.373+0000 7fca241af640 1 -- 172.21.15.40:0/2228676742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c070350 msgr2=0x7fca1c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.373+0000 7fca241af640 1 --2- 172.21.15.40:0/2228676742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c070350 0x7fca1c070b50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0c6860 tx=0x7fca0c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.373+0000 7fca241af640 1 -- 172.21.15.40:0/2228676742 shutdown_connections 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.373+0000 7fca241af640 1 --2- 172.21.15.40:0/2228676742 >> v2:172.21.15.184:3300/0 conn(0x7fca1c071090 0x7fca1c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.373+0000 7fca241af640 1 --2- 172.21.15.40:0/2228676742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c070350 0x7fca1c070b50 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.373+0000 7fca241af640 1 -- 172.21.15.40:0/2228676742 >> 172.21.15.40:0/2228676742 conn(0x7fca1c0d2d80 msgr2=0x7fca1c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.373+0000 7fca241af640 1 -- 172.21.15.40:0/2228676742 shutdown_connections 2024-04-26T21:42:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.373+0000 7fca241af640 1 -- 172.21.15.40:0/2228676742 wait complete. 2024-04-26T21:42:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.374+0000 7fca241af640 1 Processor -- start 2024-04-26T21:42:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.374+0000 7fca241af640 1 -- start start 2024-04-26T21:42:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca241af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca1c070350 0x7fca1c0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca241af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c071090 0x7fca1c0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca241af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca1c0cfde0 0x7fca1c162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca241af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c06d9b0 con 0x7fca1c071090 2024-04-26T21:42:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca241af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c0714c0 con 0x7fca1c0cfde0 2024-04-26T21:42:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca241af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca1c15ef30 con 0x7fca1c070350 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c071090 0x7fca1c0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c071090 0x7fca1c0c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44250/0 (socket says 172.21.15.40:44250) 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 -- 172.21.15.40:0/2568983789 learned_addr learned my addr 172.21.15.40:0/2568983789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2274c640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca1c0cfde0 0x7fca1c162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 -- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca1c070350 msgr2=0x7fca1c0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca21f4b640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca1c070350 0x7fca1c0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca1c070350 0x7fca1c0c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 -- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca1c0cfde0 msgr2=0x7fca1c162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca1c0cfde0 0x7fca1c162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 -- 172.21.15.40:0/2568983789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c15f560 con 0x7fca1c071090 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2274c640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca1c0cfde0 0x7fca1c162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca2174a640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c071090 0x7fca1c0c9350 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fca0c000c00 tx=0x7fca0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca127fc640 1 -- 172.21.15.40:0/2568983789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c041070 con 0x7fca1c071090 2024-04-26T21:42:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca127fc640 1 -- 172.21.15.40:0/2568983789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c03c870 con 0x7fca1c071090 2024-04-26T21:42:08.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c0c82b0 con 0x7fca1c071090 2024-04-26T21:42:08.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca127fc640 1 -- 172.21.15.40:0/2568983789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037040 con 0x7fca1c071090 2024-04-26T21:42:08.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.375+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c0c8870 con 0x7fca1c071090 2024-04-26T21:42:08.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.376+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1c0694f0 con 0x7fca1c071090 2024-04-26T21:42:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.377+0000 7fca127fc640 1 -- 172.21.15.40:0/2568983789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca0c044070 con 0x7fca1c071090 2024-04-26T21:42:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.377+0000 7fca127fc640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0006c8f0 0x7fca0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.377+0000 7fca127fc640 1 -- 172.21.15.40:0/2568983789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6001+0+0 (secure 0 0 0) 0x7fca0c0ac780 con 0x7fca1c071090 2024-04-26T21:42:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.378+0000 7fca21f4b640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0006c8f0 0x7fca0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.378+0000 7fca21f4b640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0006c8f0 0x7fca0006edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fca1800a730 tx=0x7fca18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:08.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.382+0000 7fca127fc640 1 -- 172.21.15.40:0/2568983789 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca0c07ab10 con 0x7fca1c071090 2024-04-26T21:42:08.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:08.606+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fca1c0cf2c0 con 0x7fca1c071090 2024-04-26T21:42:09.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.229+0000 7fca127fc640 1 -- 172.21.15.40:0/2568983789 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v83) ==== 174+0+0 (secure 0 0 0) 0x7fca0c07a410 con 0x7fca1c071090 2024-04-26T21:42:09.230 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 13 and data pool 14 2024-04-26T21:42:09.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0006c8f0 msgr2=0x7fca0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0006c8f0 0x7fca0006edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fca1800a730 tx=0x7fca18008040 comp rx=0 tx=0).stop 2024-04-26T21:42:09.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c071090 msgr2=0x7fca1c0c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c071090 0x7fca1c0c9350 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fca0c000c00 tx=0x7fca0c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 shutdown_connections 2024-04-26T21:42:09.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0006c8f0 0x7fca0006edb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca1c0cfde0 0x7fca1c162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c071090 0x7fca1c0c9350 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 --2- 172.21.15.40:0/2568983789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca1c070350 0x7fca1c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 >> 172.21.15.40:0/2568983789 conn(0x7fca1c0d2d80 msgr2=0x7fca1c0b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:09.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 shutdown_connections 2024-04-26T21:42:09.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.233+0000 7fca241af640 1 -- 172.21.15.40:0/2568983789 wait complete. 2024-04-26T21:42:09.267 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:42:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.373+0000 7fc8e6c52640 1 Processor -- start 2024-04-26T21:42:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.373+0000 7fc8e6c52640 1 -- start start 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e6c52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8e0162810 0x7fc8e0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e6c52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e6c52640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e00d10d0 con 0x7fc8e00b2340 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e6c52640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e00d0750 con 0x7fc8e0162810 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e6c52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e00d16a0 con 0x7fc8e00d1b30 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e49ee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8e0162810 0x7fc8e0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e49ee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8e0162810 0x7fc8e0070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37568/0 (socket says 172.21.15.40:37568) 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e49ee640 1 -- 172.21.15.40:0/3717021902 learned_addr learned my addr 172.21.15.40:0/3717021902 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e51ef640 1 --2- 172.21.15.40:0/3717021902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e49ee640 1 -- 172.21.15.40:0/3717021902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 msgr2=0x7fc8e00d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e49ee640 1 --2- 172.21.15.40:0/3717021902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e49ee640 1 -- 172.21.15.40:0/3717021902 >> v1:172.21.15.184:6789/0 conn(0x7fc8e00b2340 legacy=0x7fc8e00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.374+0000 7fc8e49ee640 1 -- 172.21.15.40:0/3717021902 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e00d1370 con 0x7fc8e0162810 2024-04-26T21:42:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.375+0000 7fc8e49ee640 1 --2- 172.21.15.40:0/3717021902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e0070350 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009870 tx=0x7fc8d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0157a2543f3eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.375+0000 7fc8df7fe640 1 -- 172.21.15.40:0/3717021902 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4037070 con 0x7fc8e0162810 2024-04-26T21:42:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.375+0000 7fc8df7fe640 1 -- 172.21.15.40:0/3717021902 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d4032440 con 0x7fc8e0162810 2024-04-26T21:42:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.375+0000 7fc8df7fe640 1 -- 172.21.15.40:0/3717021902 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4032810 con 0x7fc8e0162810 2024-04-26T21:42:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.376+0000 7fc8e6c52640 1 -- 172.21.15.40:0/3717021902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 msgr2=0x7fc8e0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.376+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/3717021902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e0070350 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009870 tx=0x7fc8d402a270 comp rx=0 tx=0).stop 2024-04-26T21:42:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.376+0000 7fc8e6c52640 1 -- 172.21.15.40:0/3717021902 shutdown_connections 2024-04-26T21:42:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.376+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/3717021902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.376+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/3717021902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e0070350 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.376+0000 7fc8e6c52640 1 -- 172.21.15.40:0/3717021902 >> 172.21.15.40:0/3717021902 conn(0x7fc8e00d2da0 msgr2=0x7fc8e00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.376+0000 7fc8e6c52640 1 -- 172.21.15.40:0/3717021902 shutdown_connections 2024-04-26T21:42:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.376+0000 7fc8e6c52640 1 -- 172.21.15.40:0/3717021902 wait complete. 2024-04-26T21:42:09.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.377+0000 7fc8e6c52640 1 Processor -- start 2024-04-26T21:42:09.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.377+0000 7fc8e6c52640 1 -- start start 2024-04-26T21:42:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e6c52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e00b2340 0x7fc8e014e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e6c52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e6c52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e6c52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0070890 con 0x7fc8e0162810 2024-04-26T21:42:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e6c52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e00d0470 con 0x7fc8e00b2340 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e6c52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e00cdb00 con 0x7fc8e00d1b30 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37570/0 (socket says 172.21.15.40:37570) 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 -- 172.21.15.40:0/1668637895 learned_addr learned my addr 172.21.15.40:0/1668637895 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e49ee640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e00b2340 0x7fc8e014e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8dffff640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 -- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 msgr2=0x7fc8e00cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 -- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e00b2340 msgr2=0x7fc8e014e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e00b2340 0x7fc8e014e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 -- 172.21.15.40:0/1668637895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e00c6bb0 con 0x7fc8e0162810 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8dffff640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8e51ef640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e015d360 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8007e40 tx=0x7fc8d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8dd7fa640 1 -- 172.21.15.40:0/1668637895 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8014620 con 0x7fc8e0162810 2024-04-26T21:42:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.378+0000 7fc8dd7fa640 1 -- 172.21.15.40:0/1668637895 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d8019070 con 0x7fc8e0162810 2024-04-26T21:42:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.379+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e00cc2b0 con 0x7fc8e0162810 2024-04-26T21:42:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.379+0000 7fc8dd7fa640 1 -- 172.21.15.40:0/1668637895 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8014a20 con 0x7fc8e0162810 2024-04-26T21:42:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.379+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e00cc870 con 0x7fc8e0162810 2024-04-26T21:42:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.379+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e00694f0 con 0x7fc8e0162810 2024-04-26T21:42:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.380+0000 7fc8dd7fa640 1 -- 172.21.15.40:0/1668637895 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8d8014bc0 con 0x7fc8e0162810 2024-04-26T21:42:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.380+0000 7fc8dd7fa640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.380+0000 7fc8dd7fa640 1 -- 172.21.15.40:0/1668637895 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fc8d808c4c0 con 0x7fc8e0162810 2024-04-26T21:42:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.381+0000 7fc8e49ee640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.381+0000 7fc8e49ee640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc8c406c8f0 0x7fc8c406edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4034000 tx=0x7fc8d402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:09.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.385+0000 7fc8dd7fa640 1 -- 172.21.15.40:0/1668637895 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8d805a760 con 0x7fc8e0162810 2024-04-26T21:42:09.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.572+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8e00d3760 con 0x7fc8e0162810 2024-04-26T21:42:09.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.573+0000 7fc8dd7fa640 1 -- 172.21.15.40:0/1668637895 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16862 (secure 0 0 0) 0x7fc8d805a060 con 0x7fc8e0162810 2024-04-26T21:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:09.574 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":93,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:42:08.607447+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc8c406c8f0 msgr2=0x7fc8c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc8c406c8f0 0x7fc8c406edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4034000 tx=0x7fc8d402e000 comp rx=0 tx=0).stop 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 msgr2=0x7fc8e015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e015d360 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8007e40 tx=0x7fc8d800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 shutdown_connections 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e0162810 0x7fc8e015d360 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e00d1b30 0x7fc8e00cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 --2- 172.21.15.40:0/1668637895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e00b2340 0x7fc8e014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 >> 172.21.15.40:0/1668637895 conn(0x7fc8e00d2da0 msgr2=0x7fc8e00cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 shutdown_connections 2024-04-26T21:42:09.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.576+0000 7fc8e6c52640 1 -- 172.21.15.40:0/1668637895 wait complete. 2024-04-26T21:42:09.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:42:09.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.688+0000 7efd2c1b6640 1 Processor -- start 2024-04-26T21:42:09.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.688+0000 7efd2c1b6640 1 -- start start 2024-04-26T21:42:09.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd2c1b6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd240b2340 0x7efd240cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd2c1b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd240cd930 0x7efd240d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd2c1b6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd240d10d0 con 0x7efd24162810 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd2c1b6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efd240d0750 con 0x7efd240b2340 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd2c1b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd240d16a0 con 0x7efd240cd930 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd29751640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd240b2340 0x7efd240cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd29751640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd240b2340 0x7efd240cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37586/0 (socket says 172.21.15.40:37586) 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd29f52640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd24162810 0x7efd24070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33008/0 (socket says 172.21.15.40:33008) 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd29751640 1 -- 172.21.15.40:0/1429408015 learned_addr learned my addr 172.21.15.40:0/1429408015 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd2a753640 1 --2- 172.21.15.40:0/1429408015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd240cd930 0x7efd240d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd29751640 1 -- 172.21.15.40:0/1429408015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd240cd930 msgr2=0x7efd240d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd29751640 1 --2- 172.21.15.40:0/1429408015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd240cd930 0x7efd240d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd29751640 1 -- 172.21.15.40:0/1429408015 >> v1:172.21.15.40:6789/0 conn(0x7efd24162810 legacy=0x7efd24070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd28f50640 1 -- 172.21.15.40:0/1429408015 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7110413 0 0) 0x7efd240d10d0 con 0x7efd24162810 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd29751640 1 -- 172.21.15.40:0/1429408015 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd240d1370 con 0x7efd240b2340 2024-04-26T21:42:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.689+0000 7efd2a753640 1 --2- 172.21.15.40:0/1429408015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd240cd930 0x7efd240d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.690+0000 7efd29751640 1 --2- 172.21.15.40:0/1429408015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240b2340 0x7efd240cd3f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7efd240d1820 tx=0x7efd1402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88832ff697a366b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.690+0000 7efd28f50640 1 -- 172.21.15.40:0/1429408015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14037070 con 0x7efd240b2340 2024-04-26T21:42:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.690+0000 7efd28f50640 1 -- 172.21.15.40:0/1429408015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd14032440 con 0x7efd240b2340 2024-04-26T21:42:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.690+0000 7efd28f50640 1 -- 172.21.15.40:0/1429408015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14032810 con 0x7efd240b2340 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.690+0000 7efd2c1b6640 1 -- 172.21.15.40:0/1429408015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240b2340 msgr2=0x7efd240cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.690+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/1429408015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240b2340 0x7efd240cd3f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7efd240d1820 tx=0x7efd1402a250 comp rx=0 tx=0).stop 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.691+0000 7efd2c1b6640 1 -- 172.21.15.40:0/1429408015 shutdown_connections 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.691+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/1429408015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd240cd930 0x7efd240d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.691+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/1429408015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240b2340 0x7efd240cd3f0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.691+0000 7efd2c1b6640 1 -- 172.21.15.40:0/1429408015 >> 172.21.15.40:0/1429408015 conn(0x7efd240d2da0 msgr2=0x7efd240d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.691+0000 7efd2c1b6640 1 -- 172.21.15.40:0/1429408015 shutdown_connections 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.691+0000 7efd2c1b6640 1 -- 172.21.15.40:0/1429408015 wait complete. 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.692+0000 7efd2c1b6640 1 Processor -- start 2024-04-26T21:42:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.692+0000 7efd2c1b6640 1 -- start start 2024-04-26T21:42:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2c1b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd240b2340 0x7efd240c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2c1b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240cd930 0x7efd241f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2c1b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd24162810 0x7efd241f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2c1b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd240d2070 con 0x7efd240cd930 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2c1b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd240d0470 con 0x7efd240b2340 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2c1b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd240d21f0 con 0x7efd24162810 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240cd930 0x7efd241f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240cd930 0x7efd241f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37598/0 (socket says 172.21.15.40:37598) 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 -- 172.21.15.40:0/914662532 learned_addr learned my addr 172.21.15.40:0/914662532 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29f52640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd240b2340 0x7efd240c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2a753640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd24162810 0x7efd241f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 -- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd24162810 msgr2=0x7efd241f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd24162810 0x7efd241f8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 -- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd240b2340 msgr2=0x7efd240c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd240b2340 0x7efd240c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 -- 172.21.15.40:0/914662532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd240ca7e0 con 0x7efd240cd930 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29f52640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd240b2340 0x7efd240c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd29751640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240cd930 0x7efd241f80d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7efd14000c00 tx=0x7efd140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd1a7fc640 1 -- 172.21.15.40:0/914662532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14037040 con 0x7efd240cd930 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd1a7fc640 1 -- 172.21.15.40:0/914662532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd14042070 con 0x7efd240cd930 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd1a7fc640 1 -- 172.21.15.40:0/914662532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14030bb0 con 0x7efd240cd930 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd240caac0 con 0x7efd240cd930 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.693+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd240cb080 con 0x7efd240cd930 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.694+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd240694f0 con 0x7efd240cd930 2024-04-26T21:42:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.695+0000 7efd1a7fc640 1 -- 172.21.15.40:0/914662532 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd14047460 con 0x7efd240cd930 2024-04-26T21:42:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.695+0000 7efd1a7fc640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd0806c8f0 0x7efd0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.695+0000 7efd1a7fc640 1 -- 172.21.15.40:0/914662532 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7efd140ac8c0 con 0x7efd240cd930 2024-04-26T21:42:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.696+0000 7efd29f52640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd0806c8f0 0x7efd0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.696+0000 7efd29f52640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd0806c8f0 0x7efd0806edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efd20000f60 tx=0x7efd20007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.700+0000 7efd1a7fc640 1 -- 172.21.15.40:0/914662532 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd14045360 con 0x7efd240cd930 2024-04-26T21:42:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.892+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd240cd570 con 0x7efd240cd930 2024-04-26T21:42:09.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.893+0000 7efd1a7fc640 1 -- 172.21.15.40:0/914662532 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16862 (secure 0 0 0) 0x7efd1407ab60 con 0x7efd240cd930 2024-04-26T21:42:09.893 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:09.893 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":93,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:42:08.607447+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:42:09.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd0806c8f0 msgr2=0x7efd0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd0806c8f0 0x7efd0806edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efd20000f60 tx=0x7efd20007ca0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240cd930 msgr2=0x7efd241f80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240cd930 0x7efd241f80d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7efd14000c00 tx=0x7efd140046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 shutdown_connections 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efd0806c8f0 0x7efd0806edb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd24162810 0x7efd241f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd240cd930 0x7efd241f80d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 --2- 172.21.15.40:0/914662532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd240b2340 0x7efd240c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 >> 172.21.15.40:0/914662532 conn(0x7efd240d2da0 msgr2=0x7efd240cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 shutdown_connections 2024-04-26T21:42:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:09.895+0000 7efd2c1b6640 1 -- 172.21.15.40:0/914662532 wait complete. 2024-04-26T21:42:09.906 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:42:10.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.010+0000 7f3d99fb4640 1 Processor -- start 2024-04-26T21:42:10.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.010+0000 7f3d99fb4640 1 -- start start 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d99fb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 0x7f3d940d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d99fb4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d940d3bc0 0x7f3d940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d99fb4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d940d1040 con 0x7f3d9415f550 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d99fb4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3d940c9430 con 0x7f3d940d3bc0 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d99fb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d940c9680 con 0x7f3d9415f170 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d93fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d940d3bc0 0x7f3d940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d92ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3d9415f550 0x7f3d940d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33024/0 (socket says 172.21.15.40:33024) 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d92ffd640 1 -- 172.21.15.40:0/3050275424 learned_addr learned my addr 172.21.15.40:0/3050275424 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d93fff640 1 -- 172.21.15.40:0/3050275424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 msgr2=0x7f3d940d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:10.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d93fff640 1 --2- 172.21.15.40:0/3050275424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 0x7f3d940d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:10.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d93fff640 1 -- 172.21.15.40:0/3050275424 >> v1:172.21.15.40:6789/0 conn(0x7f3d9415f550 legacy=0x7f3d940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:10.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d927fc640 1 -- 172.21.15.40:0/3050275424 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333316027 0 0) 0x7f3d7c001110 con 0x7f3d9415f550 2024-04-26T21:42:10.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.011+0000 7f3d93fff640 1 -- 172.21.15.40:0/3050275424 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d940c9960 con 0x7f3d940d3bc0 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.012+0000 7f3d93fff640 1 --2- 172.21.15.40:0/3050275424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d940d2cb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009d90 tx=0x7f3d8c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ae7b4121827f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.012+0000 7f3d927fc640 1 -- 172.21.15.40:0/3050275424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c007740 con 0x7f3d940d3bc0 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.012+0000 7f3d927fc640 1 -- 172.21.15.40:0/3050275424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8c0078e0 con 0x7f3d940d3bc0 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.012+0000 7f3d927fc640 1 -- 172.21.15.40:0/3050275424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c007cb0 con 0x7f3d940d3bc0 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.012+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3050275424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 msgr2=0x7f3d940d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.012+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3050275424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d940d2cb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009d90 tx=0x7f3d8c02a660 comp rx=0 tx=0).stop 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.013+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3050275424 shutdown_connections 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.013+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3050275424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d940d2cb0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:10.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.013+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3050275424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 0x7f3d940d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:10.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.013+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3050275424 >> 172.21.15.40:0/3050275424 conn(0x7f3d940d4f60 msgr2=0x7f3d940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:10.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.013+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3050275424 shutdown_connections 2024-04-26T21:42:10.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.013+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3050275424 wait complete. 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.014+0000 7f3d99fb4640 1 Processor -- start 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.014+0000 7f3d99fb4640 1 -- start start 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d99fb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d9415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d99fb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 0x7f3d94158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d99fb4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d941591b0 0x7f3d940ba460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d99fb4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3d940ca5c0 con 0x7f3d940d3bc0 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d99fb4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d940ca440 con 0x7f3d941591b0 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d99fb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d940ca740 con 0x7f3d9415f170 2024-04-26T21:42:10.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d9415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d9415b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37624/0 (socket says 172.21.15.40:37624) 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 -- 172.21.15.40:0/3200328345 learned_addr learned my addr 172.21.15.40:0/3200328345 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d93fff640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d941591b0 0x7f3d940ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d92ffd640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 0x7f3d94158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 -- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 msgr2=0x7f3d94158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 0x7f3d94158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 -- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d941591b0 msgr2=0x7f3d940ba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d941591b0 0x7f3d940ba460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 -- 172.21.15.40:0/3200328345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9415b5d0 con 0x7f3d940d3bc0 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d937fe640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d9415b090 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400b4a0 tx=0x7f3d8400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d8bfff640 1 -- 172.21.15.40:0/3200328345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84014070 con 0x7f3d940d3bc0 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d8bfff640 1 -- 172.21.15.40:0/3200328345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d84010610 con 0x7f3d940d3bc0 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d941596c0 con 0x7f3d940d3bc0 2024-04-26T21:42:10.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d8bfff640 1 -- 172.21.15.40:0/3200328345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84010a00 con 0x7f3d940d3bc0 2024-04-26T21:42:10.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.015+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d94159c80 con 0x7f3d940d3bc0 2024-04-26T21:42:10.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.016+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d940694f0 con 0x7f3d940d3bc0 2024-04-26T21:42:10.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.017+0000 7f3d8bfff640 1 -- 172.21.15.40:0/3200328345 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d84017070 con 0x7f3d940d3bc0 2024-04-26T21:42:10.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.017+0000 7f3d8bfff640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d7406c8f0 0x7f3d7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:10.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.017+0000 7f3d8bfff640 1 -- 172.21.15.40:0/3200328345 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f3d8408b6a0 con 0x7f3d940d3bc0 2024-04-26T21:42:10.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.017+0000 7f3d92ffd640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d7406c8f0 0x7f3d7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:10.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.018+0000 7f3d92ffd640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d7406c8f0 0x7f3d7406edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3d94152c70 tx=0x7f3d7c007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:10.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.022+0000 7f3d8bfff640 1 -- 172.21.15.40:0/3200328345 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d84059950 con 0x7f3d940d3bc0 2024-04-26T21:42:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:10.247+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3d9406d190 con 0x7f3d940d3bc0 2024-04-26T21:42:10.532 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-26T21:42:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.254+0000 7f3d8bfff640 1 -- 172.21.15.40:0/3200328345 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) ==== 118+0+0 (secure 0 0 0) 0x7f3d84059250 con 0x7f3d940d3bc0 2024-04-26T21:42:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d7406c8f0 msgr2=0x7f3d7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d7406c8f0 0x7f3d7406edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3d94152c70 tx=0x7f3d7c007c00 comp rx=0 tx=0).stop 2024-04-26T21:42:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 msgr2=0x7f3d9415b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d9415b090 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400b4a0 tx=0x7f3d8400b960 comp rx=0 tx=0).stop 2024-04-26T21:42:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 shutdown_connections 2024-04-26T21:42:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d7406c8f0 0x7f3d7406edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d941591b0 0x7f3d940ba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9415f170 0x7f3d94158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 --2- 172.21.15.40:0/3200328345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d940d3bc0 0x7f3d9415b090 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 >> 172.21.15.40:0/3200328345 conn(0x7f3d940d4f60 msgr2=0x7f3d940d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:11.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 shutdown_connections 2024-04-26T21:42:11.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.257+0000 7f3d99fb4640 1 -- 172.21.15.40:0/3200328345 wait complete. 2024-04-26T21:42:11.270 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:42:11.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:42:11.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.380+0000 7f9ec1d38640 1 Processor -- start 2024-04-26T21:42:11.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.381+0000 7f9ec1d38640 1 -- start start 2024-04-26T21:42:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.381+0000 7f9ec1d38640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9ebc0d1a00 0x7f9ebc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.381+0000 7f9ec1d38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ebc0d2320 0x7f9ebc1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.381+0000 7f9ec1d38640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ebc0d0a10 con 0x7f9ebc0b40d0 2024-04-26T21:42:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.381+0000 7f9ec1d38640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9ebc0d0cf0 con 0x7f9ebc0d1a00 2024-04-26T21:42:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.381+0000 7f9ec1d38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9ebc0d0fd0 con 0x7f9ebc0d2320 2024-04-26T21:42:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.381+0000 7f9ebbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9ebc0b40d0 0x7f9ebc0d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33032/0 (socket says 172.21.15.40:33032) 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.381+0000 7f9ebbfff640 1 -- 172.21.15.40:0/1781637862 learned_addr learned my addr 172.21.15.40:0/1781637862 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994986822 0 0) 0x7f9ebc0d0a10 con 0x7f9ebc0b40d0 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9ebb7fe640 1 --2- 172.21.15.40:0/1781637862 >> v2:172.21.15.184:3300/0 conn(0x7f9ebc0d1a00 0x7f9ebc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9ebaffd640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ebc0d2320 0x7f9ebc1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ea4003520 con 0x7f9ebc0b40d0 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 718756880 0 0) 0x7f9ea4003520 con 0x7f9ebc0b40d0 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ebc0d2320 msgr2=0x7f9ebc1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ebc0d2320 0x7f9ebc1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 >> v2:172.21.15.184:3300/0 conn(0x7f9ebc0d1a00 msgr2=0x7f9ebc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 --2- 172.21.15.40:0/1781637862 >> v2:172.21.15.184:3300/0 conn(0x7f9ebc0d1a00 0x7f9ebc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc0d40a0 con 0x7f9ebc0b40d0 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9eb40019f0 con 0x7f9ebc0b40d0 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 >> v1:172.21.15.40:6789/0 conn(0x7f9ebc0b40d0 legacy=0x7f9ebc0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ebc0d2320 0x7f9ea40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4003f30 0x7f9ea4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4004d40 0x7f9ea40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc0d0cf0 con 0x7f9ea4004d40 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc0d0fd0 con 0x7f9ea4003f30 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea400ba10 con 0x7f9ebc0d2320 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9ebbfff640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4004d40 0x7f9ea40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9ebb7fe640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ebc0d2320 0x7f9ea40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.382+0000 7f9ebaffd640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4003f30 0x7f9ea4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9ebbfff640 1 -- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ebc0d2320 msgr2=0x7f9ea40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9ebbfff640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ebc0d2320 0x7f9ea40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9ebbfff640 1 -- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4003f30 msgr2=0x7f9ea4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9ebbfff640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4003f30 0x7f9ea4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9ebbfff640 1 -- 172.21.15.40:0/1781637862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea4003520 con 0x7f9ea4004d40 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9ebbfff640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4004d40 0x7f9ea40051c0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400a890 tx=0x7f9ea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28a86577ecd66b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400a4e0 con 0x7f9ea4004d40 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb402f070 con 0x7f9ea4004d40 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.383+0000 7f9eba7fc640 1 -- 172.21.15.40:0/1781637862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb402bcf0 con 0x7f9ea4004d40 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.384+0000 7f9ec1d38640 1 -- 172.21.15.40:0/1781637862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4004d40 msgr2=0x7f9ea40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.384+0000 7f9ec1d38640 1 --2- 172.21.15.40:0/1781637862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4004d40 0x7f9ea40051c0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400a890 tx=0x7f9ea4005830 comp rx=0 tx=0).stop 2024-04-26T21:42:11.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.384+0000 7f9ebbfff640 1 -- 172.21.15.40:0/1781637862 reap_dead start 2024-04-26T21:42:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.384+0000 7f9ec1d38640 1 -- 172.21.15.40:0/1781637862 shutdown_connections 2024-04-26T21:42:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.384+0000 7f9ec1d38640 1 -- 172.21.15.40:0/1781637862 >> 172.21.15.40:0/1781637862 conn(0x7f9ebc0d4fd0 msgr2=0x7f9ebc0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.384+0000 7f9ec1d38640 1 -- 172.21.15.40:0/1781637862 shutdown_connections 2024-04-26T21:42:11.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.384+0000 7f9ec1d38640 1 -- 172.21.15.40:0/1781637862 wait complete. 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.385+0000 7f9ec1d38640 1 Processor -- start 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.385+0000 7f9ec1d38640 1 -- start start 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ec1d38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ea4004d40 0x7f9ebc0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ec1d38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ebc0a2980 0x7f9ebc15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ec1d38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc1f57b0 0x7f9ebc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ec1d38640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc0d35e0 con 0x7f9ebc0a2980 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ec1d38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc0b5d20 con 0x7f9ebc1f57b0 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ec1d38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9ebc0d3760 con 0x7f9ea4004d40 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ebaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ebc0a2980 0x7f9ebc15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ebaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ebc0a2980 0x7f9ebc15d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37638/0 (socket says 172.21.15.40:37638) 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ebaffd640 1 -- 172.21.15.40:0/3739046303 learned_addr learned my addr 172.21.15.40:0/3739046303 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ebbfff640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc1f57b0 0x7f9ebc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.386+0000 7f9ebb7fe640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ea4004d40 0x7f9ebc0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ebaffd640 1 -- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ea4004d40 msgr2=0x7f9ebc0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ebaffd640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ea4004d40 0x7f9ebc0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ebaffd640 1 -- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc1f57b0 msgr2=0x7f9ebc0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ebaffd640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc1f57b0 0x7f9ebc0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ebaffd640 1 -- 172.21.15.40:0/3739046303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc0a2e00 con 0x7f9ebc0a2980 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ebaffd640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ebc0a2980 0x7f9ebc15d6f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00119a0 tx=0x7f9eb0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ea3fff640 1 -- 172.21.15.40:0/3739046303 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb000faf0 con 0x7f9ebc0a2980 2024-04-26T21:42:11.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ea3fff640 1 -- 172.21.15.40:0/3739046303 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb000fc90 con 0x7f9ebc0a2980 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ea3fff640 1 -- 172.21.15.40:0/3739046303 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0014440 con 0x7f9ebc0a2980 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc1f4fd0 con 0x7f9ebc0a2980 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.387+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc0cba00 con 0x7f9ebc0a2980 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.388+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ebc1620c0 con 0x7f9ebc0a2980 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.388+0000 7f9ea3fff640 1 -- 172.21.15.40:0/3739046303 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9eb0012040 con 0x7f9ebc0a2980 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.389+0000 7f9ea3fff640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ea806c920 0x7f9ea806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.389+0000 7f9ea3fff640 1 -- 172.21.15.40:0/3739046303 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f9eb008c3d0 con 0x7f9ebc0a2980 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.389+0000 7f9ebb7fe640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ea806c920 0x7f9ea806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:11.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.390+0000 7f9ebb7fe640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ea806c920 0x7f9ea806ede0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc1f8520 tx=0x7f9eac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:11.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.393+0000 7f9ea3fff640 1 -- 172.21.15.40:0/3739046303 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ebc1620c0 con 0x7f9ebc0a2980 2024-04-26T21:42:11.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:11.620+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9ebc0694f0 con 0x7f9ebc0a2980 2024-04-26T21:42:11.849 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:42:11.851 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:42:12.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.258+0000 7f9ea3fff640 1 -- 172.21.15.40:0/3739046303 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) ==== 122+0+0 (secure 0 0 0) 0x7f9ebc0a2e00 con 0x7f9ebc0a2980 2024-04-26T21:42:12.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ea806c920 msgr2=0x7f9ea806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ea806c920 0x7f9ea806ede0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc1f8520 tx=0x7f9eac008040 comp rx=0 tx=0).stop 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ebc0a2980 msgr2=0x7f9ebc15d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ebc0a2980 0x7f9ebc15d6f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00119a0 tx=0x7f9eb0011e60 comp rx=0 tx=0).stop 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 shutdown_connections 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ea806c920 0x7f9ea806ede0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc1f57b0 0x7f9ebc0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ebc0a2980 0x7f9ebc15d6f0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 --2- 172.21.15.40:0/3739046303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ea4004d40 0x7f9ebc0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.260+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 >> 172.21.15.40:0/3739046303 conn(0x7f9ebc0d4fd0 msgr2=0x7f9ebc0b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.261+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 shutdown_connections 2024-04-26T21:42:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.261+0000 7f9ec1d38640 1 -- 172.21.15.40:0/3739046303 wait complete. 2024-04-26T21:42:12.275 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:42:12.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:12.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.381+0000 7fd5b7cfc640 1 Processor -- start 2024-04-26T21:42:12.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.381+0000 7fd5b7cfc640 1 -- start start 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b7cfc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5b0162810 0x7fd5b0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b7cfc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00b2340 0x7fd5b00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b7cfc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5b00cd930 0x7fd5b00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b7cfc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b00d0750 con 0x7fd5b00cd930 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b7cfc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b00d16a0 con 0x7fd5b0162810 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b7cfc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b00d10d0 con 0x7fd5b00b2340 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b6299640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5b00cd930 0x7fd5b00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b6299640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5b00cd930 0x7fd5b00d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37648/0 (socket says 172.21.15.40:37648) 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b6299640 1 -- 172.21.15.40:0/1977650277 learned_addr learned my addr 172.21.15.40:0/1977650277 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:12.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b5a98640 1 --2- 172.21.15.40:0/1977650277 >> v2:172.21.15.184:3300/0 conn(0x7fd5b0162810 0x7fd5b0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b5297640 1 --2- 172.21.15.40:0/1977650277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00b2340 0x7fd5b00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b6299640 1 -- 172.21.15.40:0/1977650277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00b2340 msgr2=0x7fd5b00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:12.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b6299640 1 --2- 172.21.15.40:0/1977650277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00b2340 0x7fd5b00cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b6299640 1 -- 172.21.15.40:0/1977650277 >> v2:172.21.15.184:3300/0 conn(0x7fd5b0162810 msgr2=0x7fd5b0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:12.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b6299640 1 --2- 172.21.15.40:0/1977650277 >> v2:172.21.15.184:3300/0 conn(0x7fd5b0162810 0x7fd5b0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.382+0000 7fd5b6299640 1 -- 172.21.15.40:0/1977650277 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b00d1370 con 0x7fd5b00cd930 2024-04-26T21:42:12.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.383+0000 7fd5b6299640 1 --2- 172.21.15.40:0/1977650277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b00cd930 0x7fd5b00d1b30 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0073d90 tx=0x7fd5a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228722d9a1e60616 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:12.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.383+0000 7fd5b4a96640 1 -- 172.21.15.40:0/1977650277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8036070 con 0x7fd5b00cd930 2024-04-26T21:42:12.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.383+0000 7fd5b4a96640 1 -- 172.21.15.40:0/1977650277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a8031600 con 0x7fd5b00cd930 2024-04-26T21:42:12.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.383+0000 7fd5b4a96640 1 -- 172.21.15.40:0/1977650277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a80319d0 con 0x7fd5b00cd930 2024-04-26T21:42:12.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.383+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/1977650277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b00cd930 msgr2=0x7fd5b00d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.383+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/1977650277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b00cd930 0x7fd5b00d1b30 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0073d90 tx=0x7fd5a802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.384+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/1977650277 shutdown_connections 2024-04-26T21:42:12.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.384+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/1977650277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b00cd930 0x7fd5b00d1b30 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.384+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/1977650277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00b2340 0x7fd5b00cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.384+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/1977650277 >> v2:172.21.15.184:3300/0 conn(0x7fd5b0162810 0x7fd5b0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.384+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/1977650277 >> 172.21.15.40:0/1977650277 conn(0x7fd5b00d2da0 msgr2=0x7fd5b00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:12.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.384+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/1977650277 shutdown_connections 2024-04-26T21:42:12.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.384+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/1977650277 wait complete. 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.385+0000 7fd5b7cfc640 1 Processor -- start 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.385+0000 7fd5b7cfc640 1 -- start start 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b7cfc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b00b2340 0x7fd5b014ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b7cfc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00cd930 0x7fd5b00c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b7cfc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b0162810 0x7fd5b00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b7cfc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0070890 con 0x7fd5b0162810 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b7cfc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b00d0470 con 0x7fd5b00b2340 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b7cfc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b00d2070 con 0x7fd5b00cd930 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b0162810 0x7fd5b00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b0162810 0x7fd5b00cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37650/0 (socket says 172.21.15.40:37650) 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 -- 172.21.15.40:0/971617076 learned_addr learned my addr 172.21.15.40:0/971617076 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b5a98640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b00b2340 0x7fd5b014ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b5297640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00cd930 0x7fd5b00c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 -- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00cd930 msgr2=0x7fd5b00c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00cd930 0x7fd5b00c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 -- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b00b2340 msgr2=0x7fd5b014ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b00b2340 0x7fd5b014ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 -- 172.21.15.40:0/971617076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b00c9160 con 0x7fd5b0162810 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b6299640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b0162810 0x7fd5b00cc890 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd5a802ac90 tx=0x7fd5a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5a67fc640 1 -- 172.21.15.40:0/971617076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a803f020 con 0x7fd5b0162810 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5a67fc640 1 -- 172.21.15.40:0/971617076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a8037040 con 0x7fd5b0162810 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.386+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b00c9440 con 0x7fd5b0162810 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.387+0000 7fd5a67fc640 1 -- 172.21.15.40:0/971617076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8036070 con 0x7fd5b0162810 2024-04-26T21:42:12.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.387+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b00ccdd0 con 0x7fd5b0162810 2024-04-26T21:42:12.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.388+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b00694f0 con 0x7fd5b0162810 2024-04-26T21:42:12.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.388+0000 7fd5a67fc640 1 -- 172.21.15.40:0/971617076 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5a8042020 con 0x7fd5b0162810 2024-04-26T21:42:12.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.388+0000 7fd5a67fc640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd59406c8f0 0x7fd59406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.389+0000 7fd5a67fc640 1 -- 172.21.15.40:0/971617076 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fd5a80ac040 con 0x7fd5b0162810 2024-04-26T21:42:12.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.389+0000 7fd5b5a98640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd59406c8f0 0x7fd59406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.389+0000 7fd5b5a98640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd59406c8f0 0x7fd59406edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0079a0 tx=0x7fd5ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:12.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.393+0000 7fd5a67fc640 1 -- 172.21.15.40:0/971617076 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5a8049030 con 0x7fd5b0162810 2024-04-26T21:42:12.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.619+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5b00b1e80 con 0x7fd5b0162810 2024-04-26T21:42:12.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.619+0000 7fd5a67fc640 1 -- 172.21.15.40:0/971617076 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5189 (secure 0 0 0) 0x7fd5a8079c90 con 0x7fd5b0162810 2024-04-26T21:42:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:12.621 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:11.621437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7770":{"gid":7770,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2715490806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2715490806},{"type":"v1","addr":"172.21.15.40:6835","nonce":2715490806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7770,"qdb_cluster":[7770]},"id":6}]} 2024-04-26T21:42:12.621 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 87 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd59406c8f0 msgr2=0x7fd59406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd59406c8f0 0x7fd59406edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0079a0 tx=0x7fd5ac008040 comp rx=0 tx=0).stop 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b0162810 msgr2=0x7fd5b00cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b0162810 0x7fd5b00cc890 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd5a802ac90 tx=0x7fd5a8004a40 comp rx=0 tx=0).stop 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 shutdown_connections 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd59406c8f0 0x7fd59406edb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5b0162810 0x7fd5b00cc890 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b00cd930 0x7fd5b00c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 --2- 172.21.15.40:0/971617076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b00b2340 0x7fd5b014ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.621+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 >> 172.21.15.40:0/971617076 conn(0x7fd5b00d2da0 msgr2=0x7fd5b0070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.622+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 shutdown_connections 2024-04-26T21:42:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.622+0000 7fd5b7cfc640 1 -- 172.21.15.40:0/971617076 wait complete. 2024-04-26T21:42:12.633 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:42:12.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.733+0000 7eff7c1ec640 1 Processor -- start 2024-04-26T21:42:12.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.733+0000 7eff7c1ec640 1 -- start start 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff7c1ec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7eff7415f5a0 0x7eff740d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff7c1ec640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff740d1a70 con 0x7eff74163610 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff7c1ec640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff740d1d50 con 0x7eff740d2830 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff7c1ec640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7eff74163d30 con 0x7eff7415f5a0 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff7a789640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7eff74163610 0x7eff740cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33034/0 (socket says 172.21.15.40:33034) 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff7a789640 1 -- 172.21.15.40:0/2295173549 learned_addr learned my addr 172.21.15.40:0/2295173549 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1932562064 0 0) 0x7eff740d1a70 con 0x7eff74163610 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff79f88640 1 --2- 172.21.15.40:0/2295173549 >> v2:172.21.15.184:3300/0 conn(0x7eff7415f5a0 0x7eff740d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff60003520 con 0x7eff74163610 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2837459931 0 0) 0x7eff60003520 con 0x7eff74163610 2024-04-26T21:42:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 >> v2:172.21.15.184:3300/0 conn(0x7eff7415f5a0 msgr2=0x7eff740d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff78f86640 1 --2- 172.21.15.40:0/2295173549 >> v2:172.21.15.184:3300/0 conn(0x7eff7415f5a0 0x7eff740d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 >> v1:172.21.15.184:6789/0 conn(0x7eff740d2830 legacy=0x7eff74162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff740d2030 con 0x7eff74163610 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.734+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7eff6c0019f0 con 0x7eff74163610 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 >> v1:172.21.15.40:6789/0 conn(0x7eff74163610 legacy=0x7eff740cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff78f86640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff600039b0 0x7eff60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff78f86640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60004800 0x7eff60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff78f86640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600052e0 0x7eff6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff6000c180 con 0x7eff60004800 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff6000c330 con 0x7eff600039b0 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff6000c530 con 0x7eff600052e0 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff79787640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60004800 0x7eff60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff79f88640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff600039b0 0x7eff60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff7a789640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600052e0 0x7eff6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff79787640 1 -- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600052e0 msgr2=0x7eff6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff79787640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600052e0 0x7eff6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff79787640 1 -- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff600039b0 msgr2=0x7eff60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff79787640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff600039b0 0x7eff60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff79787640 1 -- 172.21.15.40:0/2295173549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff60003520 con 0x7eff60004800 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff7a789640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600052e0 0x7eff6000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.735+0000 7eff7a789640 1 -- 172.21.15.40:0/2295173549 reap_dead start 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff79787640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60004800 0x7eff60004c80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7eff64001720 tx=0x7eff60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdea90290d960a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64001370 con 0x7eff60004800 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff640052d0 con 0x7eff60004800 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff78f86640 1 -- 172.21.15.40:0/2295173549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6402ab70 con 0x7eff60004800 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff7c1ec640 1 -- 172.21.15.40:0/2295173549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60004800 msgr2=0x7eff60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff7c1ec640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60004800 0x7eff60004c80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7eff64001720 tx=0x7eff60005830 comp rx=0 tx=0).stop 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff7c1ec640 1 -- 172.21.15.40:0/2295173549 shutdown_connections 2024-04-26T21:42:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff7c1ec640 1 --2- 172.21.15.40:0/2295173549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60004800 0x7eff60004c80 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff7c1ec640 1 -- 172.21.15.40:0/2295173549 >> 172.21.15.40:0/2295173549 conn(0x7eff740d5190 msgr2=0x7eff740d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.736+0000 7eff7c1ec640 1 -- 172.21.15.40:0/2295173549 shutdown_connections 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.737+0000 7eff7c1ec640 1 -- 172.21.15.40:0/2295173549 wait complete. 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.738+0000 7eff7c1ec640 1 Processor -- start 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.738+0000 7eff7c1ec640 1 -- start start 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.738+0000 7eff7c1ec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60004800 0x7eff74158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.738+0000 7eff7c1ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7415b1c0 0x7eff7415b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.738+0000 7eff7c1ec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff740cffc0 0x7eff740d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.738+0000 7eff7c1ec640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff740d3c20 con 0x7eff7415b1c0 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.738+0000 7eff7c1ec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff740d4190 con 0x7eff60004800 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.738+0000 7eff7c1ec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff740d3da0 con 0x7eff740cffc0 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7415b1c0 0x7eff7415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7415b1c0 0x7eff7415b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37670/0 (socket says 172.21.15.40:37670) 2024-04-26T21:42:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 -- 172.21.15.40:0/3070063383 learned_addr learned my addr 172.21.15.40:0/3070063383 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79f88640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60004800 0x7eff74158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff7a789640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff740cffc0 0x7eff740d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 -- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff740cffc0 msgr2=0x7eff740d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff740cffc0 0x7eff740d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 -- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60004800 msgr2=0x7eff74158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60004800 0x7eff74158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 -- 172.21.15.40:0/3070063383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7415c320 con 0x7eff7415b1c0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79f88640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60004800 0x7eff74158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff79787640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7415b1c0 0x7eff7415b640 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7eff64001840 tx=0x7eff64001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff6a7fc640 1 -- 172.21.15.40:0/3070063383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6403f030 con 0x7eff7415b1c0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff6a7fc640 1 -- 172.21.15.40:0/3070063383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff64005480 con 0x7eff7415b1c0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7415bd60 con 0x7eff7415b1c0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff6a7fc640 1 -- 172.21.15.40:0/3070063383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64037040 con 0x7eff7415b1c0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.739+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff740cd010 con 0x7eff7415b1c0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.740+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff740694f0 con 0x7eff7415b1c0 2024-04-26T21:42:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.741+0000 7eff6a7fc640 1 -- 172.21.15.40:0/3070063383 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff64042070 con 0x7eff7415b1c0 2024-04-26T21:42:12.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.741+0000 7eff6a7fc640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff5806c920 0x7eff5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.741+0000 7eff6a7fc640 1 -- 172.21.15.40:0/3070063383 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7eff640ab220 con 0x7eff7415b1c0 2024-04-26T21:42:12.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.741+0000 7eff79f88640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff5806c920 0x7eff5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.742+0000 7eff79f88640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff5806c920 0x7eff5806ede0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7eff70006fd0 tx=0x7eff70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:12.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.745+0000 7eff6a7fc640 1 -- 172.21.15.40:0/3070063383 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff640433b0 con 0x7eff7415b1c0 2024-04-26T21:42:12.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.931+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff74068d30 con 0x7eff7415b1c0 2024-04-26T21:42:12.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.932+0000 7eff6a7fc640 1 -- 172.21.15.40:0/3070063383 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16849 (secure 0 0 0) 0x7eff640793d0 con 0x7eff7415b1c0 2024-04-26T21:42:12.933 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:12.933 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":94,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:42:10.243824+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:42:12.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff5806c920 msgr2=0x7eff5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff5806c920 0x7eff5806ede0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7eff70006fd0 tx=0x7eff70008040 comp rx=0 tx=0).stop 2024-04-26T21:42:12.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7415b1c0 msgr2=0x7eff7415b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7415b1c0 0x7eff7415b640 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7eff64001840 tx=0x7eff64001120 comp rx=0 tx=0).stop 2024-04-26T21:42:12.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 shutdown_connections 2024-04-26T21:42:12.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff5806c920 0x7eff5806ede0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff740cffc0 0x7eff740d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7415b1c0 0x7eff7415b640 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 --2- 172.21.15.40:0/3070063383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60004800 0x7eff74158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 >> 172.21.15.40:0/3070063383 conn(0x7eff740d5190 msgr2=0x7eff740b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:12.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 shutdown_connections 2024-04-26T21:42:12.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:12.934+0000 7eff7c1ec640 1 -- 172.21.15.40:0/3070063383 wait complete. 2024-04-26T21:42:12.947 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:42:12.947 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:42:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.046+0000 7ff6d7339640 1 Processor -- start 2024-04-26T21:42:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.046+0000 7ff6d7339640 1 -- start start 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.047+0000 7ff6d7339640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff6d00d06f0 0x7ff6d00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.047+0000 7ff6d7339640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d00d0ba0 con 0x7ff6d015f3a0 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.047+0000 7ff6d7339640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d00d0e20 con 0x7ff6d00b2850 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.047+0000 7ff6d7339640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d00b1d70 con 0x7ff6d00d06f0 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.047+0000 7ff6d50d5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff6d015f3a0 0x7ff6d00d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33050/0 (socket says 172.21.15.40:33050) 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.047+0000 7ff6d50d5640 1 -- 172.21.15.40:0/831690527 learned_addr learned my addr 172.21.15.40:0/831690527 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6d48d4640 1 --2- 172.21.15.40:0/831690527 >> v2:172.21.15.184:3300/0 conn(0x7ff6d00d06f0 0x7ff6d00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6cffff640 1 -- 172.21.15.40:0/831690527 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4074850649 0 0) 0x7ff6c8001110 con 0x7ff6d015f3a0 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6cffff640 1 -- 172.21.15.40:0/831690527 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6b8003520 con 0x7ff6d015f3a0 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6cffff640 1 -- 172.21.15.40:0/831690527 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059669089 0 0) 0x7ff6d00d0e20 con 0x7ff6d00b2850 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6cffff640 1 -- 172.21.15.40:0/831690527 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6c8001110 con 0x7ff6d00b2850 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6d48d4640 1 -- 172.21.15.40:0/831690527 >> v1:172.21.15.184:6789/0 conn(0x7ff6d00b2850 legacy=0x7ff6d00b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6cffff640 1 -- 172.21.15.40:0/831690527 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1777880324 0 0) 0x7ff6b8003520 con 0x7ff6d015f3a0 2024-04-26T21:42:13.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6d48d4640 1 -- 172.21.15.40:0/831690527 >> v1:172.21.15.40:6789/0 conn(0x7ff6d015f3a0 legacy=0x7ff6d00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.048+0000 7ff6d48d4640 1 -- 172.21.15.40:0/831690527 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d00b2050 con 0x7ff6d00d06f0 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.049+0000 7ff6d48d4640 1 --2- 172.21.15.40:0/831690527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00d06f0 0x7ff6d00d1e80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40097a0 tx=0x7ff6c402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aec6fbbd58597dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.049+0000 7ff6cffff640 1 -- 172.21.15.40:0/831690527 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4037070 con 0x7ff6d00d06f0 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.049+0000 7ff6cffff640 1 -- 172.21.15.40:0/831690527 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c402ac50 con 0x7ff6d00d06f0 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.049+0000 7ff6cffff640 1 -- 172.21.15.40:0/831690527 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8001840 con 0x7ff6d00d06f0 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.050+0000 7ff6d7339640 1 -- 172.21.15.40:0/831690527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00d06f0 msgr2=0x7ff6d00d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.050+0000 7ff6d7339640 1 --2- 172.21.15.40:0/831690527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00d06f0 0x7ff6d00d1e80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40097a0 tx=0x7ff6c402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.050+0000 7ff6d7339640 1 -- 172.21.15.40:0/831690527 shutdown_connections 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.050+0000 7ff6d7339640 1 --2- 172.21.15.40:0/831690527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00d06f0 0x7ff6d00d1e80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.050+0000 7ff6d7339640 1 -- 172.21.15.40:0/831690527 >> 172.21.15.40:0/831690527 conn(0x7ff6d00d4f90 msgr2=0x7ff6d00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.050+0000 7ff6d7339640 1 -- 172.21.15.40:0/831690527 shutdown_connections 2024-04-26T21:42:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.050+0000 7ff6d7339640 1 -- 172.21.15.40:0/831690527 wait complete. 2024-04-26T21:42:13.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.051+0000 7ff6d7339640 1 Processor -- start 2024-04-26T21:42:13.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.051+0000 7ff6d7339640 1 -- start start 2024-04-26T21:42:13.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d7339640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00b2850 0x7ff6d0159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d7339640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d00d06f0 0x7ff6d0161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d7339640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6d015f3a0 0x7ff6d00b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d7339640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d015f900 con 0x7ff6d00d06f0 2024-04-26T21:42:13.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d7339640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d015f780 con 0x7ff6d00b2850 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d7339640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d00d23c0 con 0x7ff6d015f3a0 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d00d06f0 0x7ff6d0161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d00d06f0 0x7ff6d0161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37686/0 (socket says 172.21.15.40:37686) 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 -- 172.21.15.40:0/4120793282 learned_addr learned my addr 172.21.15.40:0/4120793282 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d50d5640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00b2850 0x7ff6d0159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d58d6640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6d015f3a0 0x7ff6d00b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 -- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6d015f3a0 msgr2=0x7ff6d00b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6d015f3a0 0x7ff6d00b83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 -- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00b2850 msgr2=0x7ff6d0159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00b2850 0x7ff6d0159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 -- 172.21.15.40:0/4120793282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d00b7c20 con 0x7ff6d00d06f0 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d50d5640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00b2850 0x7ff6d0159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.052+0000 7ff6d48d4640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d00d06f0 0x7ff6d0161970 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40098c0 tx=0x7ff6c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.053+0000 7ff6cdffb640 1 -- 172.21.15.40:0/4120793282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4037050 con 0x7ff6d00d06f0 2024-04-26T21:42:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.053+0000 7ff6cdffb640 1 -- 172.21.15.40:0/4120793282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c4043070 con 0x7ff6d00d06f0 2024-04-26T21:42:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.053+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d015c940 con 0x7ff6d00d06f0 2024-04-26T21:42:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.053+0000 7ff6cdffb640 1 -- 172.21.15.40:0/4120793282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4006e10 con 0x7ff6d00d06f0 2024-04-26T21:42:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.053+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d015cf00 con 0x7ff6d00d06f0 2024-04-26T21:42:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.054+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d00694f0 con 0x7ff6d00d06f0 2024-04-26T21:42:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.054+0000 7ff6cdffb640 1 -- 172.21.15.40:0/4120793282 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6c403c810 con 0x7ff6d00d06f0 2024-04-26T21:42:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.055+0000 7ff6cdffb640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6bc06c920 0x7ff6bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.055+0000 7ff6cdffb640 1 -- 172.21.15.40:0/4120793282 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7ff6c40ac510 con 0x7ff6d00d06f0 2024-04-26T21:42:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.055+0000 7ff6d50d5640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6bc06c920 0x7ff6bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.055+0000 7ff6d50d5640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6bc06c920 0x7ff6bc06ede0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80013b0 tx=0x7ff6c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.059+0000 7ff6cdffb640 1 -- 172.21.15.40:0/4120793282 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6c407a8d0 con 0x7ff6d00d06f0 2024-04-26T21:42:13.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.239+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6d00d38f0 con 0x7ff6bc06c920 2024-04-26T21:42:13.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.245+0000 7ff6cdffb640 1 -- 172.21.15.40:0/4120793282 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240199 (secure 0 0 0) 0x7ff6d00d38f0 con 0x7ff6bc06c920 2024-04-26T21:42:13.249 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:13.250 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6bc06c920 msgr2=0x7ff6bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6bc06c920 0x7ff6bc06ede0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80013b0 tx=0x7ff6c8009040 comp rx=0 tx=0).stop 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d00d06f0 msgr2=0x7ff6d0161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d00d06f0 0x7ff6d0161970 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40098c0 tx=0x7ff6c40046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 shutdown_connections 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6bc06c920 0x7ff6bc06ede0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6d015f3a0 0x7ff6d00b83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d00d06f0 0x7ff6d0161970 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 --2- 172.21.15.40:0/4120793282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d00b2850 0x7ff6d0159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 >> 172.21.15.40:0/4120793282 conn(0x7ff6d00d4f90 msgr2=0x7ff6d006cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 shutdown_connections 2024-04-26T21:42:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.248+0000 7ff6d7339640 1 -- 172.21.15.40:0/4120793282 wait complete. 2024-04-26T21:42:13.260 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-26T21:42:12.099477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246868,"kb_used_data":9892,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754727852,"statfs":{"total":773094113280,"available":772841320448,"internally_reserved":0,"allocated":10129408,"data_stored":5318149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875469"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.229189+0000","last_change":"2024-04-26T21:42:07.229189+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.229189+0000","last_undegraded":"2024-04-26T21:42:07.229189+0000","last_fullsized":"2024-04-26T21:42:07.229189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231189+0000","last_change":"2024-04-26T21:42:07.231189+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231189+0000","last_undegraded":"2024-04-26T21:42:07.231189+0000","last_fullsized":"2024-04-26T21:42:07.231189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225958+0000","last_change":"2024-04-26T21:42:07.225958+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225958+0000","last_undegraded":"2024-04-26T21:42:07.225958+0000","last_fullsized":"2024-04-26T21:42:07.225958+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230333+0000","last_change":"2024-04-26T21:42:07.230333+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230333+0000","last_undegraded":"2024-04-26T21:42:07.230333+0000","last_fullsized":"2024-04-26T21:42:07.230333+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230505+0000","last_change":"2024-04-26T21:42:07.230505+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230505+0000","last_undegraded":"2024-04-26T21:42:07.230505+0000","last_fullsized":"2024-04-26T21:42:07.230505+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225427+0000","last_change":"2024-04-26T21:42:07.225427+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225427+0000","last_undegraded":"2024-04-26T21:42:07.225427+0000","last_fullsized":"2024-04-26T21:42:07.225427+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.233518+0000","last_change":"2024-04-26T21:42:07.233518+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.233518+0000","last_undegraded":"2024-04-26T21:42:07.233518+0000","last_fullsized":"2024-04-26T21:42:07.233518+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.226561+0000","last_change":"2024-04-26T21:42:07.226561+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.226561+0000","last_undegraded":"2024-04-26T21:42:07.226561+0000","last_fullsized":"2024-04-26T21:42:07.226561+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228634+0000","last_change":"2024-04-26T21:42:07.228634+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228634+0000","last_undegraded":"2024-04-26T21:42:07.228634+0000","last_fullsized":"2024-04-26T21:42:07.228634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.233555+0000","last_change":"2024-04-26T21:42:07.233555+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.233555+0000","last_undegraded":"2024-04-26T21:42:07.233555+0000","last_fullsized":"2024-04-26T21:42:07.233555+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231573+0000","last_change":"2024-04-26T21:42:07.231573+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231573+0000","last_undegraded":"2024-04-26T21:42:07.231573+0000","last_fullsized":"2024-04-26T21:42:07.231573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228262+0000","last_change":"2024-04-26T21:42:07.228262+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228262+0000","last_undegraded":"2024-04-26T21:42:07.228262+0000","last_fullsized":"2024-04-26T21:42:07.228262+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225946+0000","last_change":"2024-04-26T21:42:07.225946+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225946+0000","last_undegraded":"2024-04-26T21:42:07.225946+0000","last_fullsized":"2024-04-26T21:42:07.225946+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228255+0000","last_change":"2024-04-26T21:42:07.228255+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228255+0000","last_undegraded":"2024-04-26T21:42:07.228255+0000","last_fullsized":"2024-04-26T21:42:07.228255+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225464+0000","last_change":"2024-04-26T21:42:07.225464+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225464+0000","last_undegraded":"2024-04-26T21:42:07.225464+0000","last_fullsized":"2024-04-26T21:42:07.225464+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225410+0000","last_change":"2024-04-26T21:42:07.225410+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225410+0000","last_undegraded":"2024-04-26T21:42:07.225410+0000","last_fullsized":"2024-04-26T21:42:07.225410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228035+0000","last_change":"2024-04-26T21:42:07.228035+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228035+0000","last_undegraded":"2024-04-26T21:42:07.228035+0000","last_fullsized":"2024-04-26T21:42:07.228035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225956+0000","last_change":"2024-04-26T21:42:07.225956+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225956+0000","last_undegraded":"2024-04-26T21:42:07.225956+0000","last_fullsized":"2024-04-26T21:42:07.225956+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231058+0000","last_change":"2024-04-26T21:42:07.231058+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231058+0000","last_undegraded":"2024-04-26T21:42:07.231058+0000","last_fullsized":"2024-04-26T21:42:07.231058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228384+0000","last_change":"2024-04-26T21:42:07.228384+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228384+0000","last_undegraded":"2024-04-26T21:42:07.228384+0000","last_fullsized":"2024-04-26T21:42:07.228384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225428+0000","last_change":"2024-04-26T21:42:07.225428+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225428+0000","last_undegraded":"2024-04-26T21:42:07.225428+0000","last_fullsized":"2024-04-26T21:42:07.225428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228361+0000","last_change":"2024-04-26T21:42:07.228361+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228361+0000","last_undegraded":"2024-04-26T21:42:07.228361+0000","last_fullsized":"2024-04-26T21:42:07.228361+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225410+0000","last_change":"2024-04-26T21:42:07.225410+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225410+0000","last_undegraded":"2024-04-26T21:42:07.225410+0000","last_fullsized":"2024-04-26T21:42:07.225410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.226775+0000","last_change":"2024-04-26T21:42:07.226775+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.226775+0000","last_undegraded":"2024-04-26T21:42:07.226775+0000","last_fullsized":"2024-04-26T21:42:07.226775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230579+0000","last_change":"2024-04-26T21:42:07.230579+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230579+0000","last_undegraded":"2024-04-26T21:42:07.230579+0000","last_fullsized":"2024-04-26T21:42:07.230579+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230581+0000","last_change":"2024-04-26T21:42:07.230581+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230581+0000","last_undegraded":"2024-04-26T21:42:07.230581+0000","last_fullsized":"2024-04-26T21:42:07.230581+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228026+0000","last_change":"2024-04-26T21:42:07.228026+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228026+0000","last_undegraded":"2024-04-26T21:42:07.228026+0000","last_fullsized":"2024-04-26T21:42:07.228026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228243+0000","last_change":"2024-04-26T21:42:07.228243+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228243+0000","last_undegraded":"2024-04-26T21:42:07.228243+0000","last_fullsized":"2024-04-26T21:42:07.228243+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228257+0000","last_change":"2024-04-26T21:42:07.228257+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228257+0000","last_undegraded":"2024-04-26T21:42:07.228257+0000","last_fullsized":"2024-04-26T21:42:07.228257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228310+0000","last_change":"2024-04-26T21:42:07.228310+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228310+0000","last_undegraded":"2024-04-26T21:42:07.228310+0000","last_fullsized":"2024-04-26T21:42:07.228310+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225961+0000","last_change":"2024-04-26T21:42:07.225961+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225961+0000","last_undegraded":"2024-04-26T21:42:07.225961+0000","last_fullsized":"2024-04-26T21:42:07.225961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225755+0000","last_change":"2024-04-26T21:42:07.225755+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225755+0000","last_undegraded":"2024-04-26T21:42:07.225755+0000","last_fullsized":"2024-04-26T21:42:07.225755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:42:07.227373+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:42:07.227373+0000","last_peered":"2024-04-26T21:42:07.227373+0000","last_clean":"2024-04-26T21:42:07.227373+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:42:07.227373+0000","last_undegraded":"2024-04-26T21:42:07.227373+0000","last_fullsized":"2024-04-26T21:42:07.227373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":1103664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1110016,"data_stored":513296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1110016,"data_stored":513296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1716224,"data_stored":1109873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:42:13.264 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:42:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.400+0000 7fc716346640 1 Processor -- start 2024-04-26T21:42:13.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc716346640 1 -- start start 2024-04-26T21:42:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc716346640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7100d3bc0 0x7fc7100d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc716346640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100b5f20 0x7fc7100b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc716346640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7100b7e50 con 0x7fc710162880 2024-04-26T21:42:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc716346640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc7100d0580 con 0x7fc7100d3bc0 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc716346640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc7100d1800 con 0x7fc7100b5f20 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc70f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7100d3bc0 0x7fc7100d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc70ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc710162880 0x7fc7100d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33058/0 (socket says 172.21.15.40:33058) 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.401+0000 7fc70ffff640 1 -- 172.21.15.40:0/2810934432 learned_addr learned my addr 172.21.15.40:0/2810934432 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.402+0000 7fc70f7fe640 1 -- 172.21.15.40:0/2810934432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100b5f20 msgr2=0x7fc7100b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.402+0000 7fc70f7fe640 1 --2- 172.21.15.40:0/2810934432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100b5f20 0x7fc7100b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.402+0000 7fc70f7fe640 1 -- 172.21.15.40:0/2810934432 >> v1:172.21.15.40:6789/0 conn(0x7fc710162880 legacy=0x7fc7100d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.402+0000 7fc70f7fe640 1 -- 172.21.15.40:0/2810934432 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7100d07d0 con 0x7fc7100d3bc0 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.402+0000 7fc70effd640 1 -- 172.21.15.40:0/2810934432 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 524194367 0 0) 0x7fc7100b7e50 con 0x7fc710162880 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.402+0000 7fc70f7fe640 1 --2- 172.21.15.40:0/2810934432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7100d3bc0 0x7fc7100d2cb0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc7000097a0 tx=0x7fc70002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcee60ac18640cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc70effd640 1 -- 172.21.15.40:0/2810934432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700037070 con 0x7fc7100d3bc0 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc70effd640 1 -- 172.21.15.40:0/2810934432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc700032440 con 0x7fc7100d3bc0 2024-04-26T21:42:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc70effd640 1 -- 172.21.15.40:0/2810934432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700032810 con 0x7fc7100d3bc0 2024-04-26T21:42:13.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc716346640 1 -- 172.21.15.40:0/2810934432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7100d3bc0 msgr2=0x7fc7100d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc716346640 1 --2- 172.21.15.40:0/2810934432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7100d3bc0 0x7fc7100d2cb0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc7000097a0 tx=0x7fc70002a180 comp rx=0 tx=0).stop 2024-04-26T21:42:13.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc716346640 1 -- 172.21.15.40:0/2810934432 shutdown_connections 2024-04-26T21:42:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc716346640 1 --2- 172.21.15.40:0/2810934432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100b5f20 0x7fc7100b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc716346640 1 --2- 172.21.15.40:0/2810934432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7100d3bc0 0x7fc7100d2cb0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc716346640 1 -- 172.21.15.40:0/2810934432 >> 172.21.15.40:0/2810934432 conn(0x7fc7100d4f60 msgr2=0x7fc7100d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc716346640 1 -- 172.21.15.40:0/2810934432 shutdown_connections 2024-04-26T21:42:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.403+0000 7fc716346640 1 -- 172.21.15.40:0/2810934432 wait complete. 2024-04-26T21:42:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.404+0000 7fc716346640 1 Processor -- start 2024-04-26T21:42:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc716346640 1 -- start start 2024-04-26T21:42:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc716346640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7100b5f20 0x7fc7101532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc716346640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d3bc0 0x7fc7100b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc716346640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc710162880 0x7fc71006df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc716346640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc710162c60 con 0x7fc710162880 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc716346640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100d0d90 con 0x7fc7100b5f20 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc716346640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc710162de0 con 0x7fc7100d3bc0 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc7148e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc710162880 0x7fc71006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc7148e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc710162880 0x7fc71006df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37694/0 (socket says 172.21.15.40:37694) 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.405+0000 7fc7148e3640 1 -- 172.21.15.40:0/2395797493 learned_addr learned my addr 172.21.15.40:0/2395797493 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc70ffff640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7100b5f20 0x7fc7101532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc70f7fe640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d3bc0 0x7fc7100b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc7148e3640 1 -- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d3bc0 msgr2=0x7fc7100b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc7148e3640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d3bc0 0x7fc7100b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc7148e3640 1 -- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7100b5f20 msgr2=0x7fc7101532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc7148e3640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7100b5f20 0x7fc7101532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc7148e3640 1 -- 172.21.15.40:0/2395797493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7100b2f20 con 0x7fc710162880 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc7148e3640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc710162880 0x7fc71006df20 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b970 tx=0x7fc70800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc70cff9640 1 -- 172.21.15.40:0/2395797493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708019070 con 0x7fc710162880 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc70cff9640 1 -- 172.21.15.40:0/2395797493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70800cb00 con 0x7fc710162880 2024-04-26T21:42:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7100b23a0 con 0x7fc710162880 2024-04-26T21:42:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc70cff9640 1 -- 172.21.15.40:0/2395797493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708014440 con 0x7fc710162880 2024-04-26T21:42:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.406+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7100b2960 con 0x7fc710162880 2024-04-26T21:42:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.407+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7100d0ff0 con 0x7fc710162880 2024-04-26T21:42:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.407+0000 7fc70cff9640 1 -- 172.21.15.40:0/2395797493 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc70801c070 con 0x7fc710162880 2024-04-26T21:42:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.408+0000 7fc70cff9640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6f006c8f0 0x7fc6f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.408+0000 7fc70cff9640 1 -- 172.21.15.40:0/2395797493 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fc70808b980 con 0x7fc710162880 2024-04-26T21:42:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.408+0000 7fc70ffff640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6f006c8f0 0x7fc6f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.409+0000 7fc70ffff640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6f006c8f0 0x7fc6f006edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8000f60 tx=0x7fc6f8007b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.412+0000 7fc70cff9640 1 -- 172.21.15.40:0/2395797493 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7100d0ff0 con 0x7fc710162880 2024-04-26T21:42:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.593+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7100d2d50 con 0x7fc6f006c8f0 2024-04-26T21:42:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.599+0000 7fc70cff9640 1 -- 172.21.15.40:0/2395797493 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240199 (secure 0 0 0) 0x7fc7100d2d50 con 0x7fc6f006c8f0 2024-04-26T21:42:13.603 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:42:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6f006c8f0 msgr2=0x7fc6f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6f006c8f0 0x7fc6f006edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8000f60 tx=0x7fc6f8007b50 comp rx=0 tx=0).stop 2024-04-26T21:42:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc710162880 msgr2=0x7fc71006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc710162880 0x7fc71006df20 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b970 tx=0x7fc70800be30 comp rx=0 tx=0).stop 2024-04-26T21:42:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 shutdown_connections 2024-04-26T21:42:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc6f006c8f0 0x7fc6f006edb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc710162880 0x7fc71006df20 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7100d3bc0 0x7fc7100b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 --2- 172.21.15.40:0/2395797493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7100b5f20 0x7fc7101532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 >> 172.21.15.40:0/2395797493 conn(0x7fc7100d4f60 msgr2=0x7fc7100d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 shutdown_connections 2024-04-26T21:42:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.603+0000 7fc716346640 1 -- 172.21.15.40:0/2395797493 wait complete. 2024-04-26T21:42:13.614 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-26T21:42:12.099477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246868,"kb_used_data":9892,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754727852,"statfs":{"total":773094113280,"available":772841320448,"internally_reserved":0,"allocated":10129408,"data_stored":5318149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875469"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.229189+0000","last_change":"2024-04-26T21:42:07.229189+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.229189+0000","last_undegraded":"2024-04-26T21:42:07.229189+0000","last_fullsized":"2024-04-26T21:42:07.229189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231189+0000","last_change":"2024-04-26T21:42:07.231189+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231189+0000","last_undegraded":"2024-04-26T21:42:07.231189+0000","last_fullsized":"2024-04-26T21:42:07.231189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225958+0000","last_change":"2024-04-26T21:42:07.225958+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225958+0000","last_undegraded":"2024-04-26T21:42:07.225958+0000","last_fullsized":"2024-04-26T21:42:07.225958+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230333+0000","last_change":"2024-04-26T21:42:07.230333+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230333+0000","last_undegraded":"2024-04-26T21:42:07.230333+0000","last_fullsized":"2024-04-26T21:42:07.230333+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230505+0000","last_change":"2024-04-26T21:42:07.230505+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230505+0000","last_undegraded":"2024-04-26T21:42:07.230505+0000","last_fullsized":"2024-04-26T21:42:07.230505+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225427+0000","last_change":"2024-04-26T21:42:07.225427+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225427+0000","last_undegraded":"2024-04-26T21:42:07.225427+0000","last_fullsized":"2024-04-26T21:42:07.225427+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.233518+0000","last_change":"2024-04-26T21:42:07.233518+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.233518+0000","last_undegraded":"2024-04-26T21:42:07.233518+0000","last_fullsized":"2024-04-26T21:42:07.233518+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.226561+0000","last_change":"2024-04-26T21:42:07.226561+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.226561+0000","last_undegraded":"2024-04-26T21:42:07.226561+0000","last_fullsized":"2024-04-26T21:42:07.226561+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228634+0000","last_change":"2024-04-26T21:42:07.228634+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228634+0000","last_undegraded":"2024-04-26T21:42:07.228634+0000","last_fullsized":"2024-04-26T21:42:07.228634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.233555+0000","last_change":"2024-04-26T21:42:07.233555+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.233555+0000","last_undegraded":"2024-04-26T21:42:07.233555+0000","last_fullsized":"2024-04-26T21:42:07.233555+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231573+0000","last_change":"2024-04-26T21:42:07.231573+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231573+0000","last_undegraded":"2024-04-26T21:42:07.231573+0000","last_fullsized":"2024-04-26T21:42:07.231573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228262+0000","last_change":"2024-04-26T21:42:07.228262+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228262+0000","last_undegraded":"2024-04-26T21:42:07.228262+0000","last_fullsized":"2024-04-26T21:42:07.228262+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225946+0000","last_change":"2024-04-26T21:42:07.225946+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225946+0000","last_undegraded":"2024-04-26T21:42:07.225946+0000","last_fullsized":"2024-04-26T21:42:07.225946+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228255+0000","last_change":"2024-04-26T21:42:07.228255+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228255+0000","last_undegraded":"2024-04-26T21:42:07.228255+0000","last_fullsized":"2024-04-26T21:42:07.228255+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225464+0000","last_change":"2024-04-26T21:42:07.225464+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225464+0000","last_undegraded":"2024-04-26T21:42:07.225464+0000","last_fullsized":"2024-04-26T21:42:07.225464+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225410+0000","last_change":"2024-04-26T21:42:07.225410+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225410+0000","last_undegraded":"2024-04-26T21:42:07.225410+0000","last_fullsized":"2024-04-26T21:42:07.225410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228035+0000","last_change":"2024-04-26T21:42:07.228035+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228035+0000","last_undegraded":"2024-04-26T21:42:07.228035+0000","last_fullsized":"2024-04-26T21:42:07.228035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225956+0000","last_change":"2024-04-26T21:42:07.225956+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225956+0000","last_undegraded":"2024-04-26T21:42:07.225956+0000","last_fullsized":"2024-04-26T21:42:07.225956+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231058+0000","last_change":"2024-04-26T21:42:07.231058+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231058+0000","last_undegraded":"2024-04-26T21:42:07.231058+0000","last_fullsized":"2024-04-26T21:42:07.231058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228384+0000","last_change":"2024-04-26T21:42:07.228384+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228384+0000","last_undegraded":"2024-04-26T21:42:07.228384+0000","last_fullsized":"2024-04-26T21:42:07.228384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225428+0000","last_change":"2024-04-26T21:42:07.225428+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225428+0000","last_undegraded":"2024-04-26T21:42:07.225428+0000","last_fullsized":"2024-04-26T21:42:07.225428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228361+0000","last_change":"2024-04-26T21:42:07.228361+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228361+0000","last_undegraded":"2024-04-26T21:42:07.228361+0000","last_fullsized":"2024-04-26T21:42:07.228361+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225410+0000","last_change":"2024-04-26T21:42:07.225410+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225410+0000","last_undegraded":"2024-04-26T21:42:07.225410+0000","last_fullsized":"2024-04-26T21:42:07.225410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.226775+0000","last_change":"2024-04-26T21:42:07.226775+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.226775+0000","last_undegraded":"2024-04-26T21:42:07.226775+0000","last_fullsized":"2024-04-26T21:42:07.226775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230579+0000","last_change":"2024-04-26T21:42:07.230579+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230579+0000","last_undegraded":"2024-04-26T21:42:07.230579+0000","last_fullsized":"2024-04-26T21:42:07.230579+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230581+0000","last_change":"2024-04-26T21:42:07.230581+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230581+0000","last_undegraded":"2024-04-26T21:42:07.230581+0000","last_fullsized":"2024-04-26T21:42:07.230581+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228026+0000","last_change":"2024-04-26T21:42:07.228026+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228026+0000","last_undegraded":"2024-04-26T21:42:07.228026+0000","last_fullsized":"2024-04-26T21:42:07.228026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228243+0000","last_change":"2024-04-26T21:42:07.228243+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228243+0000","last_undegraded":"2024-04-26T21:42:07.228243+0000","last_fullsized":"2024-04-26T21:42:07.228243+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228257+0000","last_change":"2024-04-26T21:42:07.228257+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228257+0000","last_undegraded":"2024-04-26T21:42:07.228257+0000","last_fullsized":"2024-04-26T21:42:07.228257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228310+0000","last_change":"2024-04-26T21:42:07.228310+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228310+0000","last_undegraded":"2024-04-26T21:42:07.228310+0000","last_fullsized":"2024-04-26T21:42:07.228310+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225961+0000","last_change":"2024-04-26T21:42:07.225961+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225961+0000","last_undegraded":"2024-04-26T21:42:07.225961+0000","last_fullsized":"2024-04-26T21:42:07.225961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225755+0000","last_change":"2024-04-26T21:42:07.225755+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225755+0000","last_undegraded":"2024-04-26T21:42:07.225755+0000","last_fullsized":"2024-04-26T21:42:07.225755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:42:07.227373+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:42:07.227373+0000","last_peered":"2024-04-26T21:42:07.227373+0000","last_clean":"2024-04-26T21:42:07.227373+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:42:07.227373+0000","last_undegraded":"2024-04-26T21:42:07.227373+0000","last_fullsized":"2024-04-26T21:42:07.227373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":1103664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1110016,"data_stored":513296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1110016,"data_stored":513296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1716224,"data_stored":1109873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:42:13.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:42:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.754+0000 7f0a5aadc640 1 Processor -- start 2024-04-26T21:42:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.754+0000 7f0a5aadc640 1 -- start start 2024-04-26T21:42:13.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a5aadc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a5415f120 0x7f0a5406f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a5aadc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0a540b4480 0x7f0a5406c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a5aadc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a5406c780 0x7f0a540726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a5aadc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0a5415f500 con 0x7f0a5415f120 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a5aadc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0a540ae8c0 con 0x7f0a540b4480 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a5aadc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540aeba0 con 0x7f0a5406c780 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a58878640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a5415f120 0x7f0a5406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a58878640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a5415f120 0x7f0a5406f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37696/0 (socket says 172.21.15.40:37696) 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a58878640 1 -- 172.21.15.40:0/2240597785 learned_addr learned my addr 172.21.15.40:0/2240597785 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a58878640 1 -- 172.21.15.40:0/2240597785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a5406c780 msgr2=0x7f0a540726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a59079640 1 --2- 172.21.15.40:0/2240597785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a5406c780 0x7f0a540726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a4bfff640 1 --2- 172.21.15.40:0/2240597785 >> v2:172.21.15.184:3300/0 conn(0x7f0a540b4480 0x7f0a5406c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a58878640 1 --2- 172.21.15.40:0/2240597785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a5406c780 0x7f0a540726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a58878640 1 -- 172.21.15.40:0/2240597785 >> v2:172.21.15.184:3300/0 conn(0x7f0a540b4480 msgr2=0x7f0a5406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a58878640 1 --2- 172.21.15.40:0/2240597785 >> v2:172.21.15.184:3300/0 conn(0x7f0a540b4480 0x7f0a5406c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a58878640 1 -- 172.21.15.40:0/2240597785 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a540b49f0 con 0x7f0a5415f120 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a59079640 1 --2- 172.21.15.40:0/2240597785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a5406c780 0x7f0a540726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:13.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.755+0000 7f0a4bfff640 1 --2- 172.21.15.40:0/2240597785 >> v2:172.21.15.184:3300/0 conn(0x7f0a540b4480 0x7f0a5406c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:13.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.756+0000 7f0a58878640 1 --2- 172.21.15.40:0/2240597785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5415f120 0x7f0a5406f110 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406d3e0 tx=0x7f0a5002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed44eee35c2f3dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.756+0000 7f0a4b7fe640 1 -- 172.21.15.40:0/2240597785 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50037070 con 0x7f0a5415f120 2024-04-26T21:42:13.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.756+0000 7f0a4b7fe640 1 -- 172.21.15.40:0/2240597785 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a50032440 con 0x7f0a5415f120 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.756+0000 7f0a4b7fe640 1 -- 172.21.15.40:0/2240597785 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50032810 con 0x7f0a5415f120 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 -- 172.21.15.40:0/2240597785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5415f120 msgr2=0x7f0a5406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/2240597785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5415f120 0x7f0a5406f110 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406d3e0 tx=0x7f0a5002a140 comp rx=0 tx=0).stop 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 -- 172.21.15.40:0/2240597785 shutdown_connections 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/2240597785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a5406c780 0x7f0a540726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/2240597785 >> v2:172.21.15.184:3300/0 conn(0x7f0a540b4480 0x7f0a5406c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/2240597785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5415f120 0x7f0a5406f110 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 -- 172.21.15.40:0/2240597785 >> 172.21.15.40:0/2240597785 conn(0x7f0a540b3180 msgr2=0x7f0a540b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 -- 172.21.15.40:0/2240597785 shutdown_connections 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.757+0000 7f0a5aadc640 1 -- 172.21.15.40:0/2240597785 wait complete. 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.758+0000 7f0a5aadc640 1 Processor -- start 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.758+0000 7f0a5aadc640 1 -- start start 2024-04-26T21:42:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a5aadc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5406c780 0x7f0a54157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a5aadc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a540b4480 0x7f0a5415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a5aadc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a5415f120 0x7f0a5415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a5aadc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5406f650 con 0x7f0a5406c780 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a5aadc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a540b4fb0 con 0x7f0a5415f120 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a5aadc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0a54072c20 con 0x7f0a540b4480 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5406c780 0x7f0a54157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5406c780 0x7f0a54157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37712/0 (socket says 172.21.15.40:37712) 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 -- 172.21.15.40:0/1574518139 learned_addr learned my addr 172.21.15.40:0/1574518139 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a59079640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a5415f120 0x7f0a5415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a4bfff640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a540b4480 0x7f0a5415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 -- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a540b4480 msgr2=0x7f0a5415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a540b4480 0x7f0a5415b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 -- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a5415f120 msgr2=0x7f0a5415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a5415f120 0x7f0a5415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 -- 172.21.15.40:0/1574518139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a540cd450 con 0x7f0a5406c780 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a59079640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a5415f120 0x7f0a5415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.759+0000 7f0a58878640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5406c780 0x7f0a54157f10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0a5002a180 tx=0x7f0a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.760+0000 7f0a497fa640 1 -- 172.21.15.40:0/1574518139 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50037040 con 0x7f0a5406c780 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.760+0000 7f0a497fa640 1 -- 172.21.15.40:0/1574518139 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a50042070 con 0x7f0a5406c780 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.760+0000 7f0a497fa640 1 -- 172.21.15.40:0/1574518139 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50030bb0 con 0x7f0a5406c780 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.760+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a540cddf0 con 0x7f0a5406c780 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.760+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a540ce3b0 con 0x7f0a5406c780 2024-04-26T21:42:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.761+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a54068fc0 con 0x7f0a5406c780 2024-04-26T21:42:13.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.761+0000 7f0a497fa640 1 -- 172.21.15.40:0/1574518139 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a50047460 con 0x7f0a5406c780 2024-04-26T21:42:13.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.762+0000 7f0a497fa640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a4006c8f0 0x7f0a4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:13.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.762+0000 7f0a497fa640 1 -- 172.21.15.40:0/1574518139 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f0a500ac8c0 con 0x7f0a5406c780 2024-04-26T21:42:13.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.762+0000 7f0a4bfff640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a4006c8f0 0x7f0a4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:13.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.762+0000 7f0a4bfff640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a4006c8f0 0x7f0a4006edb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0a5415b0f0 tx=0x7f0a44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:13.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.766+0000 7f0a497fa640 1 -- 172.21.15.40:0/1574518139 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a50045360 con 0x7f0a5406c780 2024-04-26T21:42:13.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.946+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a54069cd0 con 0x7f0a4006c8f0 2024-04-26T21:42:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.952+0000 7f0a497fa640 1 -- 172.21.15.40:0/1574518139 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240199 (secure 0 0 0) 0x7f0a54069cd0 con 0x7f0a4006c8f0 2024-04-26T21:42:13.956 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:42:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a4006c8f0 msgr2=0x7f0a4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a4006c8f0 0x7f0a4006edb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0a5415b0f0 tx=0x7f0a44008040 comp rx=0 tx=0).stop 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5406c780 msgr2=0x7f0a54157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5406c780 0x7f0a54157f10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0a5002a180 tx=0x7f0a500046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 shutdown_connections 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0a4006c8f0 0x7f0a4006edb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a5415f120 0x7f0a5415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a540b4480 0x7f0a5415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 --2- 172.21.15.40:0/1574518139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5406c780 0x7f0a54157f10 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 >> 172.21.15.40:0/1574518139 conn(0x7f0a540b3180 msgr2=0x7f0a540b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 shutdown_connections 2024-04-26T21:42:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:13.955+0000 7f0a5aadc640 1 -- 172.21.15.40:0/1574518139 wait complete. 2024-04-26T21:42:13.967 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-26T21:42:12.099477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246868,"kb_used_data":9892,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754727852,"statfs":{"total":773094113280,"available":772841320448,"internally_reserved":0,"allocated":10129408,"data_stored":5318149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875469"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.229189+0000","last_change":"2024-04-26T21:42:07.229189+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.229189+0000","last_undegraded":"2024-04-26T21:42:07.229189+0000","last_fullsized":"2024-04-26T21:42:07.229189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231189+0000","last_change":"2024-04-26T21:42:07.231189+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231189+0000","last_undegraded":"2024-04-26T21:42:07.231189+0000","last_fullsized":"2024-04-26T21:42:07.231189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225958+0000","last_change":"2024-04-26T21:42:07.225958+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225958+0000","last_undegraded":"2024-04-26T21:42:07.225958+0000","last_fullsized":"2024-04-26T21:42:07.225958+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230333+0000","last_change":"2024-04-26T21:42:07.230333+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230333+0000","last_undegraded":"2024-04-26T21:42:07.230333+0000","last_fullsized":"2024-04-26T21:42:07.230333+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230505+0000","last_change":"2024-04-26T21:42:07.230505+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230505+0000","last_undegraded":"2024-04-26T21:42:07.230505+0000","last_fullsized":"2024-04-26T21:42:07.230505+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225427+0000","last_change":"2024-04-26T21:42:07.225427+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225427+0000","last_undegraded":"2024-04-26T21:42:07.225427+0000","last_fullsized":"2024-04-26T21:42:07.225427+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.233518+0000","last_change":"2024-04-26T21:42:07.233518+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.233518+0000","last_undegraded":"2024-04-26T21:42:07.233518+0000","last_fullsized":"2024-04-26T21:42:07.233518+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.226561+0000","last_change":"2024-04-26T21:42:07.226561+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.226561+0000","last_undegraded":"2024-04-26T21:42:07.226561+0000","last_fullsized":"2024-04-26T21:42:07.226561+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228634+0000","last_change":"2024-04-26T21:42:07.228634+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228634+0000","last_undegraded":"2024-04-26T21:42:07.228634+0000","last_fullsized":"2024-04-26T21:42:07.228634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.233555+0000","last_change":"2024-04-26T21:42:07.233555+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.233555+0000","last_undegraded":"2024-04-26T21:42:07.233555+0000","last_fullsized":"2024-04-26T21:42:07.233555+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231573+0000","last_change":"2024-04-26T21:42:07.231573+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231573+0000","last_undegraded":"2024-04-26T21:42:07.231573+0000","last_fullsized":"2024-04-26T21:42:07.231573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228262+0000","last_change":"2024-04-26T21:42:07.228262+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228262+0000","last_undegraded":"2024-04-26T21:42:07.228262+0000","last_fullsized":"2024-04-26T21:42:07.228262+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225946+0000","last_change":"2024-04-26T21:42:07.225946+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225946+0000","last_undegraded":"2024-04-26T21:42:07.225946+0000","last_fullsized":"2024-04-26T21:42:07.225946+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228255+0000","last_change":"2024-04-26T21:42:07.228255+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228255+0000","last_undegraded":"2024-04-26T21:42:07.228255+0000","last_fullsized":"2024-04-26T21:42:07.228255+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225464+0000","last_change":"2024-04-26T21:42:07.225464+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225464+0000","last_undegraded":"2024-04-26T21:42:07.225464+0000","last_fullsized":"2024-04-26T21:42:07.225464+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225410+0000","last_change":"2024-04-26T21:42:07.225410+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225410+0000","last_undegraded":"2024-04-26T21:42:07.225410+0000","last_fullsized":"2024-04-26T21:42:07.225410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228035+0000","last_change":"2024-04-26T21:42:07.228035+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228035+0000","last_undegraded":"2024-04-26T21:42:07.228035+0000","last_fullsized":"2024-04-26T21:42:07.228035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225956+0000","last_change":"2024-04-26T21:42:07.225956+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225956+0000","last_undegraded":"2024-04-26T21:42:07.225956+0000","last_fullsized":"2024-04-26T21:42:07.225956+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.231058+0000","last_change":"2024-04-26T21:42:07.231058+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.231058+0000","last_undegraded":"2024-04-26T21:42:07.231058+0000","last_fullsized":"2024-04-26T21:42:07.231058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228384+0000","last_change":"2024-04-26T21:42:07.228384+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228384+0000","last_undegraded":"2024-04-26T21:42:07.228384+0000","last_fullsized":"2024-04-26T21:42:07.228384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225428+0000","last_change":"2024-04-26T21:42:07.225428+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225428+0000","last_undegraded":"2024-04-26T21:42:07.225428+0000","last_fullsized":"2024-04-26T21:42:07.225428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228361+0000","last_change":"2024-04-26T21:42:07.228361+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228361+0000","last_undegraded":"2024-04-26T21:42:07.228361+0000","last_fullsized":"2024-04-26T21:42:07.228361+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225410+0000","last_change":"2024-04-26T21:42:07.225410+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225410+0000","last_undegraded":"2024-04-26T21:42:07.225410+0000","last_fullsized":"2024-04-26T21:42:07.225410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.226775+0000","last_change":"2024-04-26T21:42:07.226775+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.226775+0000","last_undegraded":"2024-04-26T21:42:07.226775+0000","last_fullsized":"2024-04-26T21:42:07.226775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230579+0000","last_change":"2024-04-26T21:42:07.230579+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230579+0000","last_undegraded":"2024-04-26T21:42:07.230579+0000","last_fullsized":"2024-04-26T21:42:07.230579+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.230581+0000","last_change":"2024-04-26T21:42:07.230581+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.230581+0000","last_undegraded":"2024-04-26T21:42:07.230581+0000","last_fullsized":"2024-04-26T21:42:07.230581+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228026+0000","last_change":"2024-04-26T21:42:07.228026+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228026+0000","last_undegraded":"2024-04-26T21:42:07.228026+0000","last_fullsized":"2024-04-26T21:42:07.228026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228243+0000","last_change":"2024-04-26T21:42:07.228243+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228243+0000","last_undegraded":"2024-04-26T21:42:07.228243+0000","last_fullsized":"2024-04-26T21:42:07.228243+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228257+0000","last_change":"2024-04-26T21:42:07.228257+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228257+0000","last_undegraded":"2024-04-26T21:42:07.228257+0000","last_fullsized":"2024-04-26T21:42:07.228257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.228310+0000","last_change":"2024-04-26T21:42:07.228310+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.228310+0000","last_undegraded":"2024-04-26T21:42:07.228310+0000","last_fullsized":"2024-04-26T21:42:07.228310+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225961+0000","last_change":"2024-04-26T21:42:07.225961+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225961+0000","last_undegraded":"2024-04-26T21:42:07.225961+0000","last_fullsized":"2024-04-26T21:42:07.225961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:42:08.222935+0000","last_change":"2024-04-26T21:42:08.222935+0000","last_active":"2024-04-26T21:42:08.222935+0000","last_peered":"2024-04-26T21:42:08.222935+0000","last_clean":"2024-04-26T21:42:08.222935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:08.222935+0000","last_undegraded":"2024-04-26T21:42:08.222935+0000","last_fullsized":"2024-04-26T21:42:08.222935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-04-26T21:42:07.225755+0000","last_change":"2024-04-26T21:42:07.225755+0000","last_active":"2024-04-26T21:42:07.216858+0000","last_peered":"2024-04-26T21:42:07.216858+0000","last_clean":"2024-04-26T21:42:07.216858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:42:07.225755+0000","last_undegraded":"2024-04-26T21:42:07.225755+0000","last_fullsized":"2024-04-26T21:42:07.225755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:42:07.227373+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:42:07.227373+0000","last_peered":"2024-04-26T21:42:07.227373+0000","last_clean":"2024-04-26T21:42:07.227373+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:42:07.227373+0000","last_undegraded":"2024-04-26T21:42:07.227373+0000","last_fullsized":"2024-04-26T21:42:07.227373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":1103664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1110016,"data_stored":513296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1110016,"data_stored":513296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":519505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1716224,"data_stored":1109873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:42:13.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-04-26T21:42:13.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-04-26T21:42:13.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-04-26T21:42:13.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.229189+0000', 'last_change': '2024-04-26T21:42:07.229189+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.229189+0000', 'last_undegraded': '2024-04-26T21:42:07.229189+0000', 'last_fullsized': '2024-04-26T21:42:07.229189+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.231189+0000', 'last_change': '2024-04-26T21:42:07.231189+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.231189+0000', 'last_undegraded': '2024-04-26T21:42:07.231189+0000', 'last_fullsized': '2024-04-26T21:42:07.231189+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2024-04-26T21:42:13.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225958+0000', 'last_change': '2024-04-26T21:42:07.225958+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225958+0000', 'last_undegraded': '2024-04-26T21:42:07.225958+0000', 'last_fullsized': '2024-04-26T21:42:07.225958+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.230333+0000', 'last_change': '2024-04-26T21:42:07.230333+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.230333+0000', 'last_undegraded': '2024-04-26T21:42:07.230333+0000', 'last_fullsized': '2024-04-26T21:42:07.230333+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.230505+0000', 'last_change': '2024-04-26T21:42:07.230505+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.230505+0000', 'last_undegraded': '2024-04-26T21:42:07.230505+0000', 'last_fullsized': '2024-04-26T21:42:07.230505+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:42:13.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225427+0000', 'last_change': '2024-04-26T21:42:07.225427+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225427+0000', 'last_undegraded': '2024-04-26T21:42:07.225427+0000', 'last_fullsized': '2024-04-26T21:42:07.225427+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.233518+0000', 'last_change': '2024-04-26T21:42:07.233518+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.233518+0000', 'last_undegraded': '2024-04-26T21:42:07.233518+0000', 'last_fullsized': '2024-04-26T21:42:07.233518+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2024-04-26T21:42:13.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.226561+0000', 'last_change': '2024-04-26T21:42:07.226561+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.226561+0000', 'last_undegraded': '2024-04-26T21:42:07.226561+0000', 'last_fullsized': '2024-04-26T21:42:07.226561+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228634+0000', 'last_change': '2024-04-26T21:42:07.228634+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228634+0000', 'last_undegraded': '2024-04-26T21:42:07.228634+0000', 'last_fullsized': '2024-04-26T21:42:07.228634+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.233555+0000', 'last_change': '2024-04-26T21:42:07.233555+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.233555+0000', 'last_undegraded': '2024-04-26T21:42:07.233555+0000', 'last_fullsized': '2024-04-26T21:42:07.233555+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.231573+0000', 'last_change': '2024-04-26T21:42:07.231573+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.231573+0000', 'last_undegraded': '2024-04-26T21:42:07.231573+0000', 'last_fullsized': '2024-04-26T21:42:07.231573+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228262+0000', 'last_change': '2024-04-26T21:42:07.228262+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228262+0000', 'last_undegraded': '2024-04-26T21:42:07.228262+0000', 'last_fullsized': '2024-04-26T21:42:07.228262+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225946+0000', 'last_change': '2024-04-26T21:42:07.225946+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225946+0000', 'last_undegraded': '2024-04-26T21:42:07.225946+0000', 'last_fullsized': '2024-04-26T21:42:07.225946+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-26T21:42:13.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228255+0000', 'last_change': '2024-04-26T21:42:07.228255+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228255+0000', 'last_undegraded': '2024-04-26T21:42:07.228255+0000', 'last_fullsized': '2024-04-26T21:42:07.228255+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225464+0000', 'last_change': '2024-04-26T21:42:07.225464+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225464+0000', 'last_undegraded': '2024-04-26T21:42:07.225464+0000', 'last_fullsized': '2024-04-26T21:42:07.225464+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2024-04-26T21:42:13.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225410+0000', 'last_change': '2024-04-26T21:42:07.225410+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225410+0000', 'last_undegraded': '2024-04-26T21:42:07.225410+0000', 'last_fullsized': '2024-04-26T21:42:07.225410+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228035+0000', 'last_change': '2024-04-26T21:42:07.228035+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228035+0000', 'last_undegraded': '2024-04-26T21:42:07.228035+0000', 'last_fullsized': '2024-04-26T21:42:07.228035+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225956+0000', 'last_change': '2024-04-26T21:42:07.225956+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225956+0000', 'last_undegraded': '2024-04-26T21:42:07.225956+0000', 'last_fullsized': '2024-04-26T21:42:07.225956+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-26T21:42:13.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.231058+0000', 'last_change': '2024-04-26T21:42:07.231058+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.231058+0000', 'last_undegraded': '2024-04-26T21:42:07.231058+0000', 'last_fullsized': '2024-04-26T21:42:07.231058+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228384+0000', 'last_change': '2024-04-26T21:42:07.228384+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228384+0000', 'last_undegraded': '2024-04-26T21:42:07.228384+0000', 'last_fullsized': '2024-04-26T21:42:07.228384+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225428+0000', 'last_change': '2024-04-26T21:42:07.225428+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225428+0000', 'last_undegraded': '2024-04-26T21:42:07.225428+0000', 'last_fullsized': '2024-04-26T21:42:07.225428+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228361+0000', 'last_change': '2024-04-26T21:42:07.228361+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228361+0000', 'last_undegraded': '2024-04-26T21:42:07.228361+0000', 'last_fullsized': '2024-04-26T21:42:07.228361+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-04-26T21:42:13.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225410+0000', 'last_change': '2024-04-26T21:42:07.225410+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225410+0000', 'last_undegraded': '2024-04-26T21:42:07.225410+0000', 'last_fullsized': '2024-04-26T21:42:07.225410+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.226775+0000', 'last_change': '2024-04-26T21:42:07.226775+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.226775+0000', 'last_undegraded': '2024-04-26T21:42:07.226775+0000', 'last_fullsized': '2024-04-26T21:42:07.226775+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.230579+0000', 'last_change': '2024-04-26T21:42:07.230579+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.230579+0000', 'last_undegraded': '2024-04-26T21:42:07.230579+0000', 'last_fullsized': '2024-04-26T21:42:07.230579+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-26T21:42:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.230581+0000', 'last_change': '2024-04-26T21:42:07.230581+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.230581+0000', 'last_undegraded': '2024-04-26T21:42:07.230581+0000', 'last_fullsized': '2024-04-26T21:42:07.230581+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228026+0000', 'last_change': '2024-04-26T21:42:07.228026+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228026+0000', 'last_undegraded': '2024-04-26T21:42:07.228026+0000', 'last_fullsized': '2024-04-26T21:42:07.228026+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2024-04-26T21:42:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228243+0000', 'last_change': '2024-04-26T21:42:07.228243+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228243+0000', 'last_undegraded': '2024-04-26T21:42:07.228243+0000', 'last_fullsized': '2024-04-26T21:42:07.228243+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228257+0000', 'last_change': '2024-04-26T21:42:07.228257+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228257+0000', 'last_undegraded': '2024-04-26T21:42:07.228257+0000', 'last_fullsized': '2024-04-26T21:42:07.228257+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.228310+0000', 'last_change': '2024-04-26T21:42:07.228310+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.228310+0000', 'last_undegraded': '2024-04-26T21:42:07.228310+0000', 'last_fullsized': '2024-04-26T21:42:07.228310+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-26T21:42:13.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2024-04-26T21:42:13.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225961+0000', 'last_change': '2024-04-26T21:42:07.225961+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225961+0000', 'last_undegraded': '2024-04-26T21:42:07.225961+0000', 'last_fullsized': '2024-04-26T21:42:07.225961+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:42:13.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-26T21:42:13.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:42:08.222935+0000', 'last_change': '2024-04-26T21:42:08.222935+0000', 'last_active': '2024-04-26T21:42:08.222935+0000', 'last_peered': '2024-04-26T21:42:08.222935+0000', 'last_clean': '2024-04-26T21:42:08.222935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:08.222935+0000', 'last_undegraded': '2024-04-26T21:42:08.222935+0000', 'last_fullsized': '2024-04-26T21:42:08.222935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:08.222935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:42:13.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2024-04-26T21:42:13.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:42:07.225755+0000', 'last_change': '2024-04-26T21:42:07.225755+0000', 'last_active': '2024-04-26T21:42:07.216858+0000', 'last_peered': '2024-04-26T21:42:07.216858+0000', 'last_clean': '2024-04-26T21:42:07.216858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:42:07.225755+0000', 'last_undegraded': '2024-04-26T21:42:07.225755+0000', 'last_fullsized': '2024-04-26T21:42:07.225755+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'last_clean_scrub_stamp': '2024-04-26T21:42:07.216858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:42:13.990 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:42:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.106+0000 7f0cd7f22640 1 Processor -- start 2024-04-26T21:42:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.106+0000 7f0cd7f22640 1 -- start start 2024-04-26T21:42:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.107+0000 7f0cd7f22640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cd00d1b30 0x7f0cd00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.107+0000 7f0cd7f22640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd00d10d0 con 0x7f0cd00b2340 2024-04-26T21:42:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.107+0000 7f0cd7f22640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd00d1370 con 0x7f0cd0162810 2024-04-26T21:42:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.107+0000 7f0cd7f22640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd00d0750 con 0x7f0cd00d1b30 2024-04-26T21:42:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.107+0000 7f0cd64bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cd00d1b30 0x7f0cd00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.107+0000 7f0cd54bd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0cd00b2340 0x7f0cd00cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33072/0 (socket says 172.21.15.40:33072) 2024-04-26T21:42:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.107+0000 7f0cd54bd640 1 -- 172.21.15.40:0/2928113385 learned_addr learned my addr 172.21.15.40:0/2928113385 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.108+0000 7f0cd64bf640 1 -- 172.21.15.40:0/2928113385 >> v1:172.21.15.184:6789/0 conn(0x7f0cd0162810 legacy=0x7f0cd0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.108+0000 7f0cd64bf640 1 -- 172.21.15.40:0/2928113385 >> v1:172.21.15.40:6789/0 conn(0x7f0cd00b2340 legacy=0x7f0cd00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.108+0000 7f0cd4cbc640 1 -- 172.21.15.40:0/2928113385 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 94655633 0 0) 0x7f0cd00d10d0 con 0x7f0cd00b2340 2024-04-26T21:42:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.108+0000 7f0cd64bf640 1 -- 172.21.15.40:0/2928113385 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd00d01b0 con 0x7f0cd00d1b30 2024-04-26T21:42:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.108+0000 7f0cd64bf640 1 --2- 172.21.15.40:0/2928113385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd00d1fc0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009d90 tx=0x7f0cc002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d221cdb08a0a2e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd4cbc640 1 -- 172.21.15.40:0/2928113385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0007740 con 0x7f0cd00d1b30 2024-04-26T21:42:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd4cbc640 1 -- 172.21.15.40:0/2928113385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc00078e0 con 0x7f0cd00d1b30 2024-04-26T21:42:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd4cbc640 1 -- 172.21.15.40:0/2928113385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0007cb0 con 0x7f0cd00d1b30 2024-04-26T21:42:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd7f22640 1 -- 172.21.15.40:0/2928113385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 msgr2=0x7f0cd00d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd7f22640 1 --2- 172.21.15.40:0/2928113385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd00d1fc0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009d90 tx=0x7f0cc002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd7f22640 1 -- 172.21.15.40:0/2928113385 shutdown_connections 2024-04-26T21:42:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd7f22640 1 --2- 172.21.15.40:0/2928113385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd00d1fc0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd7f22640 1 -- 172.21.15.40:0/2928113385 >> 172.21.15.40:0/2928113385 conn(0x7f0cd00d2da0 msgr2=0x7f0cd00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd7f22640 1 -- 172.21.15.40:0/2928113385 shutdown_connections 2024-04-26T21:42:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.109+0000 7f0cd7f22640 1 -- 172.21.15.40:0/2928113385 wait complete. 2024-04-26T21:42:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.110+0000 7f0cd7f22640 1 Processor -- start 2024-04-26T21:42:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.110+0000 7f0cd7f22640 1 -- start start 2024-04-26T21:42:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd7f22640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd00b2340 0x7f0cd015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd7f22640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd01498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd7f22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd0162810 0x7f0cd0160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd7f22640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd00cdb00 con 0x7f0cd00d1b30 2024-04-26T21:42:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd7f22640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd00d16a0 con 0x7f0cd00b2340 2024-04-26T21:42:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd7f22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd0070b50 con 0x7f0cd0162810 2024-04-26T21:42:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd54bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd01498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd54bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd01498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37730/0 (socket says 172.21.15.40:37730) 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd54bd640 1 -- 172.21.15.40:0/1490155143 learned_addr learned my addr 172.21.15.40:0/1490155143 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd5cbe640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd00b2340 0x7f0cd015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.111+0000 7f0cd64bf640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd0162810 0x7f0cd0160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cd54bd640 1 -- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd0162810 msgr2=0x7f0cd0160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cd54bd640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd0162810 0x7f0cd0160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cd54bd640 1 -- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd00b2340 msgr2=0x7f0cd015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cd54bd640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd00b2340 0x7f0cd015e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cd54bd640 1 -- 172.21.15.40:0/1490155143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd0160130 con 0x7f0cd00d1b30 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cd54bd640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd01498c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c4a0 tx=0x7f0cc400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cce7fc640 1 -- 172.21.15.40:0/1490155143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4018070 con 0x7f0cd00d1b30 2024-04-26T21:42:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cce7fc640 1 -- 172.21.15.40:0/1490155143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc40074e0 con 0x7f0cd00d1b30 2024-04-26T21:42:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd015eb00 con 0x7f0cd00d1b30 2024-04-26T21:42:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cce7fc640 1 -- 172.21.15.40:0/1490155143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4004a20 con 0x7f0cd00d1b30 2024-04-26T21:42:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.112+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd01603d0 con 0x7f0cd00d1b30 2024-04-26T21:42:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.113+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd00694f0 con 0x7f0cd00d1b30 2024-04-26T21:42:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.113+0000 7f0cce7fc640 1 -- 172.21.15.40:0/1490155143 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cc401b070 con 0x7f0cd00d1b30 2024-04-26T21:42:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.114+0000 7f0cce7fc640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c8f0 0x7f0cbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.114+0000 7f0cce7fc640 1 -- 172.21.15.40:0/1490155143 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f0cc408aff0 con 0x7f0cd00d1b30 2024-04-26T21:42:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.114+0000 7f0cd5cbe640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c8f0 0x7f0cbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.115+0000 7f0cd5cbe640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c8f0 0x7f0cbc06edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8000f60 tx=0x7f0cc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:14.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.118+0000 7f0cce7fc640 1 -- 172.21.15.40:0/1490155143 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0cc4059280 con 0x7f0cd00d1b30 2024-04-26T21:42:14.216 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:42:14.215+0000 7f84043d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:42:14.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.376+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f0cd00b2ce0 con 0x7f0cd00d1b30 2024-04-26T21:42:14.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.377+0000 7f0cce7fc640 1 -- 172.21.15.40:0/1490155143 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1433 (secure 0 0 0) 0x7f0cc4058b80 con 0x7f0cd00d1b30 2024-04-26T21:42:14.378 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:14.378 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":448,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":94,"num_osds":8,"num_up_osds":8,"osd_up_since":1714167297,"num_in_osds":8,"osd_in_since":1714167287,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":257454080,"bytes_avail":772836659200,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":87,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7770}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T21:41:40.092364+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T21:42:14.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c8f0 msgr2=0x7f0cbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c8f0 0x7f0cbc06edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8000f60 tx=0x7f0cc8009040 comp rx=0 tx=0).stop 2024-04-26T21:42:14.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 msgr2=0x7f0cd01498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd01498c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c4a0 tx=0x7f0cc400c960 comp rx=0 tx=0).stop 2024-04-26T21:42:14.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 shutdown_connections 2024-04-26T21:42:14.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0cbc06c8f0 0x7f0cbc06edb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cd0162810 0x7f0cd0160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cd00d1b30 0x7f0cd01498c0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 --2- 172.21.15.40:0/1490155143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cd00b2340 0x7f0cd015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 >> 172.21.15.40:0/1490155143 conn(0x7f0cd00d2da0 msgr2=0x7f0cd00b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:14.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 shutdown_connections 2024-04-26T21:42:14.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.379+0000 7f0cd7f22640 1 -- 172.21.15.40:0/1490155143 wait complete. 2024-04-26T21:42:14.391 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:42:14.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:42:14.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.491+0000 7f6109cda640 1 Processor -- start 2024-04-26T21:42:14.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.491+0000 7f6109cda640 1 -- start start 2024-04-26T21:42:14.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6109cda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61040d3bc0 0x7f61040d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6109cda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040b5f20 0x7f61040b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6109cda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61040b7e50 con 0x7f6104162880 2024-04-26T21:42:14.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6109cda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f61040d0580 con 0x7f61040d3bc0 2024-04-26T21:42:14.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6109cda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61040d1800 con 0x7f61040b5f20 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6102ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61040d3bc0 0x7f61040d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6103fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040b5f20 0x7f61040b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6102ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61040d3bc0 0x7f61040d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:43802/0 (socket says 172.21.15.40:43802) 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f61037fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f6104162880 0x7f61040d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:40536/0 (socket says 172.21.15.40:40536) 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6102ffd640 1 -- 172.21.15.40:0/1064749301 learned_addr learned my addr 172.21.15.40:0/1064749301 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6103fff640 1 -- 172.21.15.40:0/1064749301 >> v2:172.21.15.184:3300/0 conn(0x7f61040d3bc0 msgr2=0x7f61040d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6103fff640 1 --2- 172.21.15.40:0/1064749301 >> v2:172.21.15.184:3300/0 conn(0x7f61040d3bc0 0x7f61040d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.492+0000 7f6103fff640 1 -- 172.21.15.40:0/1064749301 >> v1:172.21.15.184:6789/0 conn(0x7f6104162880 legacy=0x7f61040d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.493+0000 7f6103fff640 1 -- 172.21.15.40:0/1064749301 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61040d07d0 con 0x7f61040b5f20 2024-04-26T21:42:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.493+0000 7f6102ffd640 1 --2- 172.21.15.40:0/1064749301 >> v2:172.21.15.184:3300/0 conn(0x7f61040d3bc0 0x7f61040d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.493+0000 7f6103fff640 1 --2- 172.21.15.40:0/1064749301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040b5f20 0x7f61040b8300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009d90 tx=0x7f60fc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3fd43fc811715d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.493+0000 7f61027fc640 1 -- 172.21.15.40:0/1064749301 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc007740 con 0x7f61040b5f20 2024-04-26T21:42:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.493+0000 7f61027fc640 1 -- 172.21.15.40:0/1064749301 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc0078e0 con 0x7f61040b5f20 2024-04-26T21:42:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f61027fc640 1 -- 172.21.15.40:0/1064749301 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc007cb0 con 0x7f61040b5f20 2024-04-26T21:42:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f6109cda640 1 -- 172.21.15.40:0/1064749301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040b5f20 msgr2=0x7f61040b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f6109cda640 1 --2- 172.21.15.40:0/1064749301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040b5f20 0x7f61040b8300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009d90 tx=0x7f60fc02a660 comp rx=0 tx=0).stop 2024-04-26T21:42:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f6109cda640 1 -- 172.21.15.40:0/1064749301 shutdown_connections 2024-04-26T21:42:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f6109cda640 1 --2- 172.21.15.40:0/1064749301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040b5f20 0x7f61040b8300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f6109cda640 1 --2- 172.21.15.40:0/1064749301 >> v2:172.21.15.184:3300/0 conn(0x7f61040d3bc0 0x7f61040d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f6109cda640 1 -- 172.21.15.40:0/1064749301 >> 172.21.15.40:0/1064749301 conn(0x7f61040d4f60 msgr2=0x7f61040d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f6109cda640 1 -- 172.21.15.40:0/1064749301 shutdown_connections 2024-04-26T21:42:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.494+0000 7f6109cda640 1 -- 172.21.15.40:0/1064749301 wait complete. 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.495+0000 7f6109cda640 1 Processor -- start 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.495+0000 7f6109cda640 1 -- start start 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6109cda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61040b5f20 0x7f61040b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6109cda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040d3bc0 0x7f61040b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6109cda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6104162880 0x7f6104158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6109cda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6104162c60 con 0x7f6104162880 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6109cda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61040d0d90 con 0x7f61040b5f20 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6109cda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6104162de0 con 0x7f61040d3bc0 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6103fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6104162880 0x7f6104158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6103fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6104162880 0x7f6104158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37742/0 (socket says 172.21.15.40:37742) 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.496+0000 7f6103fff640 1 -- 172.21.15.40:0/1897164567 learned_addr learned my addr 172.21.15.40:0/1897164567 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f61037fe640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61040b5f20 0x7f61040b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6102ffd640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040d3bc0 0x7f61040b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6103fff640 1 -- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040d3bc0 msgr2=0x7f61040b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6103fff640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040d3bc0 0x7f61040b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6103fff640 1 -- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61040b5f20 msgr2=0x7f61040b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6103fff640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61040b5f20 0x7f61040b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6103fff640 1 -- 172.21.15.40:0/1897164567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6104159800 con 0x7f6104162880 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6103fff640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6104162880 0x7f6104158570 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f60fc02ac50 tx=0x7f60fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f60ebfff640 1 -- 172.21.15.40:0/1897164567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc03f070 con 0x7f6104162880 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f60ebfff640 1 -- 172.21.15.40:0/1897164567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc031c60 con 0x7f6104162880 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6104159ae0 con 0x7f6104162880 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f60ebfff640 1 -- 172.21.15.40:0/1897164567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc009450 con 0x7f6104162880 2024-04-26T21:42:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.497+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610415a0a0 con 0x7f6104162880 2024-04-26T21:42:14.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.498+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610415ec10 con 0x7f6104162880 2024-04-26T21:42:14.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.499+0000 7f60ebfff640 1 -- 172.21.15.40:0/1897164567 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60fc043650 con 0x7f6104162880 2024-04-26T21:42:14.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.499+0000 7f60ebfff640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f006c8f0 0x7f60f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:14.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.499+0000 7f60ebfff640 1 -- 172.21.15.40:0/1897164567 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f60fc0ad040 con 0x7f6104162880 2024-04-26T21:42:14.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.499+0000 7f61037fe640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f006c8f0 0x7f60f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:14.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.500+0000 7f61037fe640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f006c8f0 0x7f60f006edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f60f4000f60 tx=0x7f60f4007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:14.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.503+0000 7f60ebfff640 1 -- 172.21.15.40:0/1897164567 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f610415ec10 con 0x7f6104162880 2024-04-26T21:42:14.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.687+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61040694f0 con 0x7f60f006c8f0 2024-04-26T21:42:14.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.694+0000 7f60ebfff640 1 -- 172.21.15.40:0/1897164567 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252076 (secure 0 0 0) 0x7f61040694f0 con 0x7f60f006c8f0 2024-04-26T21:42:14.698 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:14.699 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:42:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.697+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f006c8f0 msgr2=0x7f60f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.697+0000 7f6109cda640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f006c8f0 0x7f60f006edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f60f4000f60 tx=0x7f60f4007cc0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.697+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6104162880 msgr2=0x7f6104158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.697+0000 7f6109cda640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6104162880 0x7f6104158570 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f60fc02ac50 tx=0x7f60fc004a40 comp rx=0 tx=0).stop 2024-04-26T21:42:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.698+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 shutdown_connections 2024-04-26T21:42:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.698+0000 7f6109cda640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f006c8f0 0x7f60f006edb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.698+0000 7f6109cda640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6104162880 0x7f6104158570 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.698+0000 7f6109cda640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61040d3bc0 0x7f61040b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.698+0000 7f6109cda640 1 --2- 172.21.15.40:0/1897164567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61040b5f20 0x7f61040b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:14.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.698+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 >> 172.21.15.40:0/1897164567 conn(0x7f61040d4f60 msgr2=0x7f6104161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:14.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.698+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 shutdown_connections 2024-04-26T21:42:14.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:14.698+0000 7f6109cda640 1 -- 172.21.15.40:0/1897164567 wait complete. 2024-04-26T21:42:14.709 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-04-26T21:42:14.100081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251420,"kb_used_data":10280,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754723300,"statfs":{"total":773094113280,"available":772836659200,"internally_reserved":0,"allocated":10526720,"data_stored":5505804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001689"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193300+0000","last_change":"2024-04-26T21:42:09.264494+0000","last_active":"2024-04-26T21:42:11.193300+0000","last_peered":"2024-04-26T21:42:11.193300+0000","last_clean":"2024-04-26T21:42:11.193300+0000","last_became_active":"2024-04-26T21:42:09.263985+0000","last_became_peered":"2024-04-26T21:42:09.263985+0000","last_unstale":"2024-04-26T21:42:11.193300+0000","last_undegraded":"2024-04-26T21:42:11.193300+0000","last_fullsized":"2024-04-26T21:42:11.193300+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:37.060054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252392+0000","last_change":"2024-04-26T21:42:09.258182+0000","last_active":"2024-04-26T21:42:10.252392+0000","last_peered":"2024-04-26T21:42:10.252392+0000","last_clean":"2024-04-26T21:42:10.252392+0000","last_became_active":"2024-04-26T21:42:09.257837+0000","last_became_peered":"2024-04-26T21:42:09.257837+0000","last_unstale":"2024-04-26T21:42:10.252392+0000","last_undegraded":"2024-04-26T21:42:10.252392+0000","last_fullsized":"2024-04-26T21:42:10.252392+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:06.225847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193171+0000","last_change":"2024-04-26T21:42:09.257887+0000","last_active":"2024-04-26T21:42:11.193171+0000","last_peered":"2024-04-26T21:42:11.193171+0000","last_clean":"2024-04-26T21:42:11.193171+0000","last_became_active":"2024-04-26T21:42:09.257588+0000","last_became_peered":"2024-04-26T21:42:09.257588+0000","last_unstale":"2024-04-26T21:42:11.193171+0000","last_undegraded":"2024-04-26T21:42:11.193171+0000","last_fullsized":"2024-04-26T21:42:11.193171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:37.741942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029009+0000","last_change":"2024-04-26T21:42:09.267138+0000","last_active":"2024-04-26T21:42:12.029009+0000","last_peered":"2024-04-26T21:42:12.029009+0000","last_clean":"2024-04-26T21:42:12.029009+0000","last_became_active":"2024-04-26T21:42:09.266785+0000","last_became_peered":"2024-04-26T21:42:09.266785+0000","last_unstale":"2024-04-26T21:42:12.029009+0000","last_undegraded":"2024-04-26T21:42:12.029009+0000","last_fullsized":"2024-04-26T21:42:12.029009+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236482+0000","last_change":"2024-04-26T21:42:09.251296+0000","last_active":"2024-04-26T21:42:11.236482+0000","last_peered":"2024-04-26T21:42:11.236482+0000","last_clean":"2024-04-26T21:42:11.236482+0000","last_became_active":"2024-04-26T21:42:09.250106+0000","last_became_peered":"2024-04-26T21:42:09.250106+0000","last_unstale":"2024-04-26T21:42:11.236482+0000","last_undegraded":"2024-04-26T21:42:11.236482+0000","last_fullsized":"2024-04-26T21:42:11.236482+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:37.735401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028327+0000","last_change":"2024-04-26T21:42:09.262844+0000","last_active":"2024-04-26T21:42:12.028327+0000","last_peered":"2024-04-26T21:42:12.028327+0000","last_clean":"2024-04-26T21:42:12.028327+0000","last_became_active":"2024-04-26T21:42:09.260424+0000","last_became_peered":"2024-04-26T21:42:09.260424+0000","last_unstale":"2024-04-26T21:42:12.028327+0000","last_undegraded":"2024-04-26T21:42:12.028327+0000","last_fullsized":"2024-04-26T21:42:12.028327+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:37.051031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028460+0000","last_change":"2024-04-26T21:42:09.258736+0000","last_active":"2024-04-26T21:42:12.028460+0000","last_peered":"2024-04-26T21:42:12.028460+0000","last_clean":"2024-04-26T21:42:12.028460+0000","last_became_active":"2024-04-26T21:42:09.258162+0000","last_became_peered":"2024-04-26T21:42:09.258162+0000","last_unstale":"2024-04-26T21:42:12.028460+0000","last_undegraded":"2024-04-26T21:42:12.028460+0000","last_fullsized":"2024-04-26T21:42:12.028460+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:02.398915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028708+0000","last_change":"2024-04-26T21:42:09.255437+0000","last_active":"2024-04-26T21:42:12.028708+0000","last_peered":"2024-04-26T21:42:12.028708+0000","last_clean":"2024-04-26T21:42:12.028708+0000","last_became_active":"2024-04-26T21:42:09.255146+0000","last_became_peered":"2024-04-26T21:42:09.255146+0000","last_unstale":"2024-04-26T21:42:12.028708+0000","last_undegraded":"2024-04-26T21:42:12.028708+0000","last_fullsized":"2024-04-26T21:42:12.028708+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:24.183661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236546+0000","last_change":"2024-04-26T21:42:09.248867+0000","last_active":"2024-04-26T21:42:11.236546+0000","last_peered":"2024-04-26T21:42:11.236546+0000","last_clean":"2024-04-26T21:42:11.236546+0000","last_became_active":"2024-04-26T21:42:09.248602+0000","last_became_peered":"2024-04-26T21:42:09.248602+0000","last_unstale":"2024-04-26T21:42:11.236546+0000","last_undegraded":"2024-04-26T21:42:11.236546+0000","last_fullsized":"2024-04-26T21:42:11.236546+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:53.382141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.250166+0000","last_change":"2024-04-26T21:42:09.268509+0000","last_active":"2024-04-26T21:42:10.250166+0000","last_peered":"2024-04-26T21:42:10.250166+0000","last_clean":"2024-04-26T21:42:10.250166+0000","last_became_active":"2024-04-26T21:42:09.268187+0000","last_became_peered":"2024-04-26T21:42:09.268187+0000","last_unstale":"2024-04-26T21:42:10.250166+0000","last_undegraded":"2024-04-26T21:42:10.250166+0000","last_fullsized":"2024-04-26T21:42:10.250166+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:23.869675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236564+0000","last_change":"2024-04-26T21:42:09.254222+0000","last_active":"2024-04-26T21:42:11.236564+0000","last_peered":"2024-04-26T21:42:11.236564+0000","last_clean":"2024-04-26T21:42:11.236564+0000","last_became_active":"2024-04-26T21:42:09.253953+0000","last_became_peered":"2024-04-26T21:42:09.253953+0000","last_unstale":"2024-04-26T21:42:11.236564+0000","last_undegraded":"2024-04-26T21:42:11.236564+0000","last_fullsized":"2024-04-26T21:42:11.236564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:33.561796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.250027+0000","last_change":"2024-04-26T21:42:09.258563+0000","last_active":"2024-04-26T21:42:10.250027+0000","last_peered":"2024-04-26T21:42:10.250027+0000","last_clean":"2024-04-26T21:42:10.250027+0000","last_became_active":"2024-04-26T21:42:09.258251+0000","last_became_peered":"2024-04-26T21:42:09.258251+0000","last_unstale":"2024-04-26T21:42:10.250027+0000","last_undegraded":"2024-04-26T21:42:10.250027+0000","last_fullsized":"2024-04-26T21:42:10.250027+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:35.112901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028734+0000","last_change":"2024-04-26T21:42:09.254193+0000","last_active":"2024-04-26T21:42:12.028734+0000","last_peered":"2024-04-26T21:42:12.028734+0000","last_clean":"2024-04-26T21:42:12.028734+0000","last_became_active":"2024-04-26T21:42:09.253750+0000","last_became_peered":"2024-04-26T21:42:09.253750+0000","last_unstale":"2024-04-26T21:42:12.028734+0000","last_undegraded":"2024-04-26T21:42:12.028734+0000","last_fullsized":"2024-04-26T21:42:12.028734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:11.988701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236104+0000","last_change":"2024-04-26T21:42:09.254666+0000","last_active":"2024-04-26T21:42:11.236104+0000","last_peered":"2024-04-26T21:42:11.236104+0000","last_clean":"2024-04-26T21:42:11.236104+0000","last_became_active":"2024-04-26T21:42:09.254360+0000","last_became_peered":"2024-04-26T21:42:09.254360+0000","last_unstale":"2024-04-26T21:42:11.236104+0000","last_undegraded":"2024-04-26T21:42:11.236104+0000","last_fullsized":"2024-04-26T21:42:11.236104+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:53.266651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028749+0000","last_change":"2024-04-26T21:42:09.258644+0000","last_active":"2024-04-26T21:42:12.028749+0000","last_peered":"2024-04-26T21:42:12.028749+0000","last_clean":"2024-04-26T21:42:12.028749+0000","last_became_active":"2024-04-26T21:42:09.258337+0000","last_became_peered":"2024-04-26T21:42:09.258337+0000","last_unstale":"2024-04-26T21:42:12.028749+0000","last_undegraded":"2024-04-26T21:42:12.028749+0000","last_fullsized":"2024-04-26T21:42:12.028749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:48.828972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028312+0000","last_change":"2024-04-26T21:42:09.262980+0000","last_active":"2024-04-26T21:42:12.028312+0000","last_peered":"2024-04-26T21:42:12.028312+0000","last_clean":"2024-04-26T21:42:12.028312+0000","last_became_active":"2024-04-26T21:42:09.262473+0000","last_became_peered":"2024-04-26T21:42:09.262473+0000","last_unstale":"2024-04-26T21:42:12.028312+0000","last_undegraded":"2024-04-26T21:42:12.028312+0000","last_fullsized":"2024-04-26T21:42:12.028312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:24.095510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029673+0000","last_change":"2024-04-26T21:42:09.261833+0000","last_active":"2024-04-26T21:42:12.029673+0000","last_peered":"2024-04-26T21:42:12.029673+0000","last_clean":"2024-04-26T21:42:12.029673+0000","last_became_active":"2024-04-26T21:42:09.261276+0000","last_became_peered":"2024-04-26T21:42:09.261276+0000","last_unstale":"2024-04-26T21:42:12.029673+0000","last_undegraded":"2024-04-26T21:42:12.029673+0000","last_fullsized":"2024-04-26T21:42:12.029673+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:45.799211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193132+0000","last_change":"2024-04-26T21:42:09.256730+0000","last_active":"2024-04-26T21:42:11.193132+0000","last_peered":"2024-04-26T21:42:11.193132+0000","last_clean":"2024-04-26T21:42:11.193132+0000","last_became_active":"2024-04-26T21:42:09.256384+0000","last_became_peered":"2024-04-26T21:42:09.256384+0000","last_unstale":"2024-04-26T21:42:11.193132+0000","last_undegraded":"2024-04-26T21:42:11.193132+0000","last_fullsized":"2024-04-26T21:42:11.193132+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:44.259318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236645+0000","last_change":"2024-04-26T21:42:09.254731+0000","last_active":"2024-04-26T21:42:11.236645+0000","last_peered":"2024-04-26T21:42:11.236645+0000","last_clean":"2024-04-26T21:42:11.236645+0000","last_became_active":"2024-04-26T21:42:09.254458+0000","last_became_peered":"2024-04-26T21:42:09.254458+0000","last_unstale":"2024-04-26T21:42:11.236645+0000","last_undegraded":"2024-04-26T21:42:11.236645+0000","last_fullsized":"2024-04-26T21:42:11.236645+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:02.601613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029155+0000","last_change":"2024-04-26T21:42:09.259722+0000","last_active":"2024-04-26T21:42:12.029155+0000","last_peered":"2024-04-26T21:42:12.029155+0000","last_clean":"2024-04-26T21:42:12.029155+0000","last_became_active":"2024-04-26T21:42:09.259485+0000","last_became_peered":"2024-04-26T21:42:09.259485+0000","last_unstale":"2024-04-26T21:42:12.029155+0000","last_undegraded":"2024-04-26T21:42:12.029155+0000","last_fullsized":"2024-04-26T21:42:12.029155+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:38.522173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236220+0000","last_change":"2024-04-26T21:42:09.246154+0000","last_active":"2024-04-26T21:42:11.236220+0000","last_peered":"2024-04-26T21:42:11.236220+0000","last_clean":"2024-04-26T21:42:11.236220+0000","last_became_active":"2024-04-26T21:42:09.245803+0000","last_became_peered":"2024-04-26T21:42:09.245803+0000","last_unstale":"2024-04-26T21:42:11.236220+0000","last_undegraded":"2024-04-26T21:42:11.236220+0000","last_fullsized":"2024-04-26T21:42:11.236220+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:34.133725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236133+0000","last_change":"2024-04-26T21:42:09.251971+0000","last_active":"2024-04-26T21:42:11.236133+0000","last_peered":"2024-04-26T21:42:11.236133+0000","last_clean":"2024-04-26T21:42:11.236133+0000","last_became_active":"2024-04-26T21:42:09.251677+0000","last_became_peered":"2024-04-26T21:42:09.251677+0000","last_unstale":"2024-04-26T21:42:11.236133+0000","last_undegraded":"2024-04-26T21:42:11.236133+0000","last_fullsized":"2024-04-26T21:42:11.236133+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:25.996727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251603+0000","last_change":"2024-04-26T21:42:09.268630+0000","last_active":"2024-04-26T21:42:10.251603+0000","last_peered":"2024-04-26T21:42:10.251603+0000","last_clean":"2024-04-26T21:42:10.251603+0000","last_became_active":"2024-04-26T21:42:09.268234+0000","last_became_peered":"2024-04-26T21:42:09.268234+0000","last_unstale":"2024-04-26T21:42:10.251603+0000","last_undegraded":"2024-04-26T21:42:10.251603+0000","last_fullsized":"2024-04-26T21:42:10.251603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:11.988701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193125+0000","last_change":"2024-04-26T21:42:09.255874+0000","last_active":"2024-04-26T21:42:11.193125+0000","last_peered":"2024-04-26T21:42:11.193125+0000","last_clean":"2024-04-26T21:42:11.193125+0000","last_became_active":"2024-04-26T21:42:09.255549+0000","last_became_peered":"2024-04-26T21:42:09.255549+0000","last_unstale":"2024-04-26T21:42:11.193125+0000","last_undegraded":"2024-04-26T21:42:11.193125+0000","last_fullsized":"2024-04-26T21:42:11.193125+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:44.735505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251805+0000","last_change":"2024-04-26T21:42:09.250715+0000","last_active":"2024-04-26T21:42:10.251805+0000","last_peered":"2024-04-26T21:42:10.251805+0000","last_clean":"2024-04-26T21:42:10.251805+0000","last_became_active":"2024-04-26T21:42:09.250409+0000","last_became_peered":"2024-04-26T21:42:09.250409+0000","last_unstale":"2024-04-26T21:42:10.251805+0000","last_undegraded":"2024-04-26T21:42:10.251805+0000","last_fullsized":"2024-04-26T21:42:10.251805+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:14.247992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236578+0000","last_change":"2024-04-26T21:42:09.247745+0000","last_active":"2024-04-26T21:42:11.236578+0000","last_peered":"2024-04-26T21:42:11.236578+0000","last_clean":"2024-04-26T21:42:11.236578+0000","last_became_active":"2024-04-26T21:42:09.247421+0000","last_became_peered":"2024-04-26T21:42:09.247421+0000","last_unstale":"2024-04-26T21:42:11.236578+0000","last_undegraded":"2024-04-26T21:42:11.236578+0000","last_fullsized":"2024-04-26T21:42:11.236578+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:03.358699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.250054+0000","last_change":"2024-04-26T21:42:09.250974+0000","last_active":"2024-04-26T21:42:10.250054+0000","last_peered":"2024-04-26T21:42:10.250054+0000","last_clean":"2024-04-26T21:42:10.250054+0000","last_became_active":"2024-04-26T21:42:09.250689+0000","last_became_peered":"2024-04-26T21:42:09.250689+0000","last_unstale":"2024-04-26T21:42:10.250054+0000","last_undegraded":"2024-04-26T21:42:10.250054+0000","last_fullsized":"2024-04-26T21:42:10.250054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:19.164424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029308+0000","last_change":"2024-04-26T21:42:09.259454+0000","last_active":"2024-04-26T21:42:12.029308+0000","last_peered":"2024-04-26T21:42:12.029308+0000","last_clean":"2024-04-26T21:42:12.029308+0000","last_became_active":"2024-04-26T21:42:09.259169+0000","last_became_peered":"2024-04-26T21:42:09.259169+0000","last_unstale":"2024-04-26T21:42:12.029308+0000","last_undegraded":"2024-04-26T21:42:12.029308+0000","last_fullsized":"2024-04-26T21:42:12.029308+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028596+0000","last_change":"2024-04-26T21:42:09.250457+0000","last_active":"2024-04-26T21:42:12.028596+0000","last_peered":"2024-04-26T21:42:12.028596+0000","last_clean":"2024-04-26T21:42:12.028596+0000","last_became_active":"2024-04-26T21:42:09.250185+0000","last_became_peered":"2024-04-26T21:42:09.250185+0000","last_unstale":"2024-04-26T21:42:12.028596+0000","last_undegraded":"2024-04-26T21:42:12.028596+0000","last_fullsized":"2024-04-26T21:42:12.028596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:01.361591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236454+0000","last_change":"2024-04-26T21:42:09.247757+0000","last_active":"2024-04-26T21:42:11.236454+0000","last_peered":"2024-04-26T21:42:11.236454+0000","last_clean":"2024-04-26T21:42:11.236454+0000","last_became_active":"2024-04-26T21:42:09.247514+0000","last_became_peered":"2024-04-26T21:42:09.247514+0000","last_unstale":"2024-04-26T21:42:11.236454+0000","last_undegraded":"2024-04-26T21:42:11.236454+0000","last_fullsized":"2024-04-26T21:42:11.236454+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:25.765330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251325+0000","last_change":"2024-04-26T21:42:09.268310+0000","last_active":"2024-04-26T21:42:10.251325+0000","last_peered":"2024-04-26T21:42:10.251325+0000","last_clean":"2024-04-26T21:42:10.251325+0000","last_became_active":"2024-04-26T21:42:09.268014+0000","last_became_peered":"2024-04-26T21:42:09.268014+0000","last_unstale":"2024-04-26T21:42:10.251325+0000","last_undegraded":"2024-04-26T21:42:10.251325+0000","last_fullsized":"2024-04-26T21:42:10.251325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:44.735505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028320+0000","last_change":"2024-04-26T21:42:09.262763+0000","last_active":"2024-04-26T21:42:12.028320+0000","last_peered":"2024-04-26T21:42:12.028320+0000","last_clean":"2024-04-26T21:42:12.028320+0000","last_became_active":"2024-04-26T21:42:09.262109+0000","last_became_peered":"2024-04-26T21:42:09.262109+0000","last_unstale":"2024-04-26T21:42:12.028320+0000","last_undegraded":"2024-04-26T21:42:12.028320+0000","last_fullsized":"2024-04-26T21:42:12.028320+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:23.458317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029031+0000","last_change":"2024-04-26T21:42:09.261233+0000","last_active":"2024-04-26T21:42:12.029031+0000","last_peered":"2024-04-26T21:42:12.029031+0000","last_clean":"2024-04-26T21:42:12.029031+0000","last_became_active":"2024-04-26T21:42:09.260312+0000","last_became_peered":"2024-04-26T21:42:09.260312+0000","last_unstale":"2024-04-26T21:42:12.029031+0000","last_undegraded":"2024-04-26T21:42:12.029031+0000","last_fullsized":"2024-04-26T21:42:12.029031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:59.637256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"93'4","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252140+0000","last_change":"2024-04-26T21:42:08.248939+0000","last_active":"2024-04-26T21:42:10.252140+0000","last_peered":"2024-04-26T21:42:10.252140+0000","last_clean":"2024-04-26T21:42:10.252140+0000","last_became_active":"2024-04-26T21:42:08.248609+0000","last_became_peered":"2024-04-26T21:42:08.248609+0000","last_unstale":"2024-04-26T21:42:10.252140+0000","last_undegraded":"2024-04-26T21:42:10.252140+0000","last_fullsized":"2024-04-26T21:42:10.252140+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:52.376064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029412+0000","last_change":"2024-04-26T21:42:09.248275+0000","last_active":"2024-04-26T21:42:12.029412+0000","last_peered":"2024-04-26T21:42:12.029412+0000","last_clean":"2024-04-26T21:42:12.029412+0000","last_became_active":"2024-04-26T21:42:09.247956+0000","last_became_peered":"2024-04-26T21:42:09.247956+0000","last_unstale":"2024-04-26T21:42:12.029412+0000","last_undegraded":"2024-04-26T21:42:12.029412+0000","last_fullsized":"2024-04-26T21:42:12.029412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:59.762871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236635+0000","last_change":"2024-04-26T21:42:08.250948+0000","last_active":"2024-04-26T21:42:11.236635+0000","last_peered":"2024-04-26T21:42:11.236635+0000","last_clean":"2024-04-26T21:42:11.236635+0000","last_became_active":"2024-04-26T21:42:08.250565+0000","last_became_peered":"2024-04-26T21:42:08.250565+0000","last_unstale":"2024-04-26T21:42:11.236635+0000","last_undegraded":"2024-04-26T21:42:11.236635+0000","last_fullsized":"2024-04-26T21:42:11.236635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:39.819647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028634+0000","last_change":"2024-04-26T21:42:09.254560+0000","last_active":"2024-04-26T21:42:12.028634+0000","last_peered":"2024-04-26T21:42:12.028634+0000","last_clean":"2024-04-26T21:42:12.028634+0000","last_became_active":"2024-04-26T21:42:09.254271+0000","last_became_peered":"2024-04-26T21:42:09.254271+0000","last_unstale":"2024-04-26T21:42:12.028634+0000","last_undegraded":"2024-04-26T21:42:12.028634+0000","last_fullsized":"2024-04-26T21:42:12.028634+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:53.785856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028604+0000","last_change":"2024-04-26T21:42:08.248518+0000","last_active":"2024-04-26T21:42:12.028604+0000","last_peered":"2024-04-26T21:42:12.028604+0000","last_clean":"2024-04-26T21:42:12.028604+0000","last_became_active":"2024-04-26T21:42:08.248181+0000","last_became_peered":"2024-04-26T21:42:08.248181+0000","last_unstale":"2024-04-26T21:42:12.028604+0000","last_undegraded":"2024-04-26T21:42:12.028604+0000","last_fullsized":"2024-04-26T21:42:12.028604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:19.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029669+0000","last_change":"2024-04-26T21:42:09.262453+0000","last_active":"2024-04-26T21:42:12.029669+0000","last_peered":"2024-04-26T21:42:12.029669+0000","last_clean":"2024-04-26T21:42:12.029669+0000","last_became_active":"2024-04-26T21:42:09.262189+0000","last_became_peered":"2024-04-26T21:42:09.262189+0000","last_unstale":"2024-04-26T21:42:12.029669+0000","last_undegraded":"2024-04-26T21:42:12.029669+0000","last_fullsized":"2024-04-26T21:42:12.029669+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:36.275442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252322+0000","last_change":"2024-04-26T21:42:08.247033+0000","last_active":"2024-04-26T21:42:10.252322+0000","last_peered":"2024-04-26T21:42:10.252322+0000","last_clean":"2024-04-26T21:42:10.252322+0000","last_became_active":"2024-04-26T21:42:08.246730+0000","last_became_peered":"2024-04-26T21:42:08.246730+0000","last_unstale":"2024-04-26T21:42:10.252322+0000","last_undegraded":"2024-04-26T21:42:10.252322+0000","last_fullsized":"2024-04-26T21:42:10.252322+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:01.595536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028614+0000","last_change":"2024-04-26T21:42:09.257950+0000","last_active":"2024-04-26T21:42:12.028614+0000","last_peered":"2024-04-26T21:42:12.028614+0000","last_clean":"2024-04-26T21:42:12.028614+0000","last_became_active":"2024-04-26T21:42:09.257634+0000","last_became_peered":"2024-04-26T21:42:09.257634+0000","last_unstale":"2024-04-26T21:42:12.028614+0000","last_undegraded":"2024-04-26T21:42:12.028614+0000","last_fullsized":"2024-04-26T21:42:12.028614+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:25.097224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236548+0000","last_change":"2024-04-26T21:42:08.251049+0000","last_active":"2024-04-26T21:42:11.236548+0000","last_peered":"2024-04-26T21:42:11.236548+0000","last_clean":"2024-04-26T21:42:11.236548+0000","last_became_active":"2024-04-26T21:42:08.250616+0000","last_became_peered":"2024-04-26T21:42:08.250616+0000","last_unstale":"2024-04-26T21:42:11.236548+0000","last_undegraded":"2024-04-26T21:42:11.236548+0000","last_fullsized":"2024-04-26T21:42:11.236548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:22.339132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251987+0000","last_change":"2024-04-26T21:42:09.253878+0000","last_active":"2024-04-26T21:42:10.251987+0000","last_peered":"2024-04-26T21:42:10.251987+0000","last_clean":"2024-04-26T21:42:10.251987+0000","last_became_active":"2024-04-26T21:42:09.253619+0000","last_became_peered":"2024-04-26T21:42:09.253619+0000","last_unstale":"2024-04-26T21:42:10.251987+0000","last_undegraded":"2024-04-26T21:42:10.251987+0000","last_fullsized":"2024-04-26T21:42:10.251987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:18.430927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029445+0000","last_change":"2024-04-26T21:42:08.247064+0000","last_active":"2024-04-26T21:42:12.029445+0000","last_peered":"2024-04-26T21:42:12.029445+0000","last_clean":"2024-04-26T21:42:12.029445+0000","last_became_active":"2024-04-26T21:42:08.246780+0000","last_became_peered":"2024-04-26T21:42:08.246780+0000","last_unstale":"2024-04-26T21:42:12.029445+0000","last_undegraded":"2024-04-26T21:42:12.029445+0000","last_fullsized":"2024-04-26T21:42:12.029445+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:24.759253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028709+0000","last_change":"2024-04-26T21:42:09.260041+0000","last_active":"2024-04-26T21:42:12.028709+0000","last_peered":"2024-04-26T21:42:12.028709+0000","last_clean":"2024-04-26T21:42:12.028709+0000","last_became_active":"2024-04-26T21:42:09.253986+0000","last_became_peered":"2024-04-26T21:42:09.253986+0000","last_unstale":"2024-04-26T21:42:12.028709+0000","last_undegraded":"2024-04-26T21:42:12.028709+0000","last_fullsized":"2024-04-26T21:42:12.028709+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:54.110266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.192965+0000","last_change":"2024-04-26T21:42:08.239869+0000","last_active":"2024-04-26T21:42:11.192965+0000","last_peered":"2024-04-26T21:42:11.192965+0000","last_clean":"2024-04-26T21:42:11.192965+0000","last_became_active":"2024-04-26T21:42:08.239585+0000","last_became_peered":"2024-04-26T21:42:08.239585+0000","last_unstale":"2024-04-26T21:42:11.192965+0000","last_undegraded":"2024-04-26T21:42:11.192965+0000","last_fullsized":"2024-04-26T21:42:11.192965+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:08.285259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028903+0000","last_change":"2024-04-26T21:42:09.264591+0000","last_active":"2024-04-26T21:42:12.028903+0000","last_peered":"2024-04-26T21:42:12.028903+0000","last_clean":"2024-04-26T21:42:12.028903+0000","last_became_active":"2024-04-26T21:42:09.264287+0000","last_became_peered":"2024-04-26T21:42:09.264287+0000","last_unstale":"2024-04-26T21:42:12.028903+0000","last_undegraded":"2024-04-26T21:42:12.028903+0000","last_fullsized":"2024-04-26T21:42:12.028903+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:18.430927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028384+0000","last_change":"2024-04-26T21:42:08.243726+0000","last_active":"2024-04-26T21:42:12.028384+0000","last_peered":"2024-04-26T21:42:12.028384+0000","last_clean":"2024-04-26T21:42:12.028384+0000","last_became_active":"2024-04-26T21:42:08.243450+0000","last_became_peered":"2024-04-26T21:42:08.243450+0000","last_unstale":"2024-04-26T21:42:12.028384+0000","last_undegraded":"2024-04-26T21:42:12.028384+0000","last_fullsized":"2024-04-26T21:42:12.028384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:12.743868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236495+0000","last_change":"2024-04-26T21:42:09.258072+0000","last_active":"2024-04-26T21:42:11.236495+0000","last_peered":"2024-04-26T21:42:11.236495+0000","last_clean":"2024-04-26T21:42:11.236495+0000","last_became_active":"2024-04-26T21:42:09.257789+0000","last_became_peered":"2024-04-26T21:42:09.257789+0000","last_unstale":"2024-04-26T21:42:11.236495+0000","last_undegraded":"2024-04-26T21:42:11.236495+0000","last_fullsized":"2024-04-26T21:42:11.236495+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:01.023671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252020+0000","last_change":"2024-04-26T21:42:08.249432+0000","last_active":"2024-04-26T21:42:10.252020+0000","last_peered":"2024-04-26T21:42:10.252020+0000","last_clean":"2024-04-26T21:42:10.252020+0000","last_became_active":"2024-04-26T21:42:08.249064+0000","last_became_peered":"2024-04-26T21:42:08.249064+0000","last_unstale":"2024-04-26T21:42:10.252020+0000","last_undegraded":"2024-04-26T21:42:10.252020+0000","last_fullsized":"2024-04-26T21:42:10.252020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:37.516096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028753+0000","last_change":"2024-04-26T21:42:09.255185+0000","last_active":"2024-04-26T21:42:12.028753+0000","last_peered":"2024-04-26T21:42:12.028753+0000","last_clean":"2024-04-26T21:42:12.028753+0000","last_became_active":"2024-04-26T21:42:09.254903+0000","last_became_peered":"2024-04-26T21:42:09.254903+0000","last_unstale":"2024-04-26T21:42:12.028753+0000","last_undegraded":"2024-04-26T21:42:12.028753+0000","last_fullsized":"2024-04-26T21:42:12.028753+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:11.619631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193218+0000","last_change":"2024-04-26T21:42:08.243997+0000","last_active":"2024-04-26T21:42:11.193218+0000","last_peered":"2024-04-26T21:42:11.193218+0000","last_clean":"2024-04-26T21:42:11.193218+0000","last_became_active":"2024-04-26T21:42:08.243708+0000","last_became_peered":"2024-04-26T21:42:08.243708+0000","last_unstale":"2024-04-26T21:42:11.193218+0000","last_undegraded":"2024-04-26T21:42:11.193218+0000","last_fullsized":"2024-04-26T21:42:11.193218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:37.516096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193200+0000","last_change":"2024-04-26T21:42:09.264131+0000","last_active":"2024-04-26T21:42:11.193200+0000","last_peered":"2024-04-26T21:42:11.193200+0000","last_clean":"2024-04-26T21:42:11.193200+0000","last_became_active":"2024-04-26T21:42:09.263307+0000","last_became_peered":"2024-04-26T21:42:09.263307+0000","last_unstale":"2024-04-26T21:42:11.193200+0000","last_undegraded":"2024-04-26T21:42:11.193200+0000","last_fullsized":"2024-04-26T21:42:11.193200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:14.247992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"93'3","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236699+0000","last_change":"2024-04-26T21:42:08.252314+0000","last_active":"2024-04-26T21:42:11.236699+0000","last_peered":"2024-04-26T21:42:11.236699+0000","last_clean":"2024-04-26T21:42:11.236699+0000","last_became_active":"2024-04-26T21:42:08.252062+0000","last_became_peered":"2024-04-26T21:42:08.252062+0000","last_unstale":"2024-04-26T21:42:11.236699+0000","last_undegraded":"2024-04-26T21:42:11.236699+0000","last_fullsized":"2024-04-26T21:42:11.236699+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:13.833108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029148+0000","last_change":"2024-04-26T21:42:09.251068+0000","last_active":"2024-04-26T21:42:12.029148+0000","last_peered":"2024-04-26T21:42:12.029148+0000","last_clean":"2024-04-26T21:42:12.029148+0000","last_became_active":"2024-04-26T21:42:09.250779+0000","last_became_peered":"2024-04-26T21:42:09.250779+0000","last_unstale":"2024-04-26T21:42:12.029148+0000","last_undegraded":"2024-04-26T21:42:12.029148+0000","last_fullsized":"2024-04-26T21:42:12.029148+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:09.291336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"93'6","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029089+0000","last_change":"2024-04-26T21:42:08.248470+0000","last_active":"2024-04-26T21:42:12.029089+0000","last_peered":"2024-04-26T21:42:12.029089+0000","last_clean":"2024-04-26T21:42:12.029089+0000","last_became_active":"2024-04-26T21:42:08.248129+0000","last_became_peered":"2024-04-26T21:42:08.248129+0000","last_unstale":"2024-04-26T21:42:12.029089+0000","last_undegraded":"2024-04-26T21:42:12.029089+0000","last_fullsized":"2024-04-26T21:42:12.029089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:43.253241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028784+0000","last_change":"2024-04-26T21:42:09.254581+0000","last_active":"2024-04-26T21:42:12.028784+0000","last_peered":"2024-04-26T21:42:12.028784+0000","last_clean":"2024-04-26T21:42:12.028784+0000","last_became_active":"2024-04-26T21:42:09.253789+0000","last_became_peered":"2024-04-26T21:42:09.253789+0000","last_unstale":"2024-04-26T21:42:12.028784+0000","last_undegraded":"2024-04-26T21:42:12.028784+0000","last_fullsized":"2024-04-26T21:42:12.028784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:07.477516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028522+0000","last_change":"2024-04-26T21:42:08.250002+0000","last_active":"2024-04-26T21:42:12.028522+0000","last_peered":"2024-04-26T21:42:12.028522+0000","last_clean":"2024-04-26T21:42:12.028522+0000","last_became_active":"2024-04-26T21:42:08.249730+0000","last_became_peered":"2024-04-26T21:42:08.249730+0000","last_unstale":"2024-04-26T21:42:12.028522+0000","last_undegraded":"2024-04-26T21:42:12.028522+0000","last_fullsized":"2024-04-26T21:42:12.028522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:43.729428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236087+0000","last_change":"2024-04-26T21:42:09.252000+0000","last_active":"2024-04-26T21:42:11.236087+0000","last_peered":"2024-04-26T21:42:11.236087+0000","last_clean":"2024-04-26T21:42:11.236087+0000","last_became_active":"2024-04-26T21:42:09.251721+0000","last_became_peered":"2024-04-26T21:42:09.251721+0000","last_unstale":"2024-04-26T21:42:11.236087+0000","last_undegraded":"2024-04-26T21:42:11.236087+0000","last_fullsized":"2024-04-26T21:42:11.236087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:12.821517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028762+0000","last_change":"2024-04-26T21:42:08.251291+0000","last_active":"2024-04-26T21:42:12.028762+0000","last_peered":"2024-04-26T21:42:12.028762+0000","last_clean":"2024-04-26T21:42:12.028762+0000","last_became_active":"2024-04-26T21:42:08.251027+0000","last_became_peered":"2024-04-26T21:42:08.251027+0000","last_unstale":"2024-04-26T21:42:12.028762+0000","last_undegraded":"2024-04-26T21:42:12.028762+0000","last_fullsized":"2024-04-26T21:42:12.028762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:36.735866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.249889+0000","last_change":"2024-04-26T21:42:09.268758+0000","last_active":"2024-04-26T21:42:10.249889+0000","last_peered":"2024-04-26T21:42:10.249889+0000","last_clean":"2024-04-26T21:42:10.249889+0000","last_became_active":"2024-04-26T21:42:09.268502+0000","last_became_peered":"2024-04-26T21:42:09.268502+0000","last_unstale":"2024-04-26T21:42:10.249889+0000","last_undegraded":"2024-04-26T21:42:10.249889+0000","last_fullsized":"2024-04-26T21:42:10.249889+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:19.961609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029379+0000","last_change":"2024-04-26T21:42:08.243376+0000","last_active":"2024-04-26T21:42:12.029379+0000","last_peered":"2024-04-26T21:42:12.029379+0000","last_clean":"2024-04-26T21:42:12.029379+0000","last_became_active":"2024-04-26T21:42:08.243085+0000","last_became_peered":"2024-04-26T21:42:08.243085+0000","last_unstale":"2024-04-26T21:42:12.029379+0000","last_undegraded":"2024-04-26T21:42:12.029379+0000","last_fullsized":"2024-04-26T21:42:12.029379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:33.127648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.250171+0000","last_change":"2024-04-26T21:42:09.248567+0000","last_active":"2024-04-26T21:42:10.250171+0000","last_peered":"2024-04-26T21:42:10.250171+0000","last_clean":"2024-04-26T21:42:10.250171+0000","last_became_active":"2024-04-26T21:42:09.248292+0000","last_became_peered":"2024-04-26T21:42:09.248292+0000","last_unstale":"2024-04-26T21:42:10.250171+0000","last_undegraded":"2024-04-26T21:42:10.250171+0000","last_fullsized":"2024-04-26T21:42:10.250171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:40.761060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"93'8","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028765+0000","last_change":"2024-04-26T21:42:08.249551+0000","last_active":"2024-04-26T21:42:12.028765+0000","last_peered":"2024-04-26T21:42:12.028765+0000","last_clean":"2024-04-26T21:42:12.028765+0000","last_became_active":"2024-04-26T21:42:08.249251+0000","last_became_peered":"2024-04-26T21:42:08.249251+0000","last_unstale":"2024-04-26T21:42:12.028765+0000","last_undegraded":"2024-04-26T21:42:12.028765+0000","last_fullsized":"2024-04-26T21:42:12.028765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:52.260574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029064+0000","last_change":"2024-04-26T21:42:09.253791+0000","last_active":"2024-04-26T21:42:12.029064+0000","last_peered":"2024-04-26T21:42:12.029064+0000","last_clean":"2024-04-26T21:42:12.029064+0000","last_became_active":"2024-04-26T21:42:09.253465+0000","last_became_peered":"2024-04-26T21:42:09.253465+0000","last_unstale":"2024-04-26T21:42:12.029064+0000","last_undegraded":"2024-04-26T21:42:12.029064+0000","last_fullsized":"2024-04-26T21:42:12.029064+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:33.184002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252276+0000","last_change":"2024-04-26T21:42:08.244764+0000","last_active":"2024-04-26T21:42:10.252276+0000","last_peered":"2024-04-26T21:42:10.252276+0000","last_clean":"2024-04-26T21:42:10.252276+0000","last_became_active":"2024-04-26T21:42:08.244222+0000","last_became_peered":"2024-04-26T21:42:08.244222+0000","last_unstale":"2024-04-26T21:42:10.252276+0000","last_undegraded":"2024-04-26T21:42:10.252276+0000","last_fullsized":"2024-04-26T21:42:10.252276+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:32.555719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252044+0000","last_change":"2024-04-26T21:42:09.249727+0000","last_active":"2024-04-26T21:42:10.252044+0000","last_peered":"2024-04-26T21:42:10.252044+0000","last_clean":"2024-04-26T21:42:10.252044+0000","last_became_active":"2024-04-26T21:42:09.249446+0000","last_became_peered":"2024-04-26T21:42:09.249446+0000","last_unstale":"2024-04-26T21:42:10.252044+0000","last_undegraded":"2024-04-26T21:42:10.252044+0000","last_fullsized":"2024-04-26T21:42:10.252044+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028599+0000","last_change":"2024-04-26T21:42:08.243044+0000","last_active":"2024-04-26T21:42:12.028599+0000","last_peered":"2024-04-26T21:42:12.028599+0000","last_clean":"2024-04-26T21:42:12.028599+0000","last_became_active":"2024-04-26T21:42:08.242751+0000","last_became_peered":"2024-04-26T21:42:08.242751+0000","last_unstale":"2024-04-26T21:42:12.028599+0000","last_undegraded":"2024-04-26T21:42:12.028599+0000","last_fullsized":"2024-04-26T21:42:12.028599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:25.955636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236551+0000","last_change":"2024-04-26T21:42:09.248740+0000","last_active":"2024-04-26T21:42:11.236551+0000","last_peered":"2024-04-26T21:42:11.236551+0000","last_clean":"2024-04-26T21:42:11.236551+0000","last_became_active":"2024-04-26T21:42:09.248474+0000","last_became_peered":"2024-04-26T21:42:09.248474+0000","last_unstale":"2024-04-26T21:42:11.236551+0000","last_undegraded":"2024-04-26T21:42:11.236551+0000","last_fullsized":"2024-04-26T21:42:11.236551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:36.706819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236730+0000","last_change":"2024-04-26T21:42:08.246124+0000","last_active":"2024-04-26T21:42:11.236730+0000","last_peered":"2024-04-26T21:42:11.236730+0000","last_clean":"2024-04-26T21:42:11.236730+0000","last_became_active":"2024-04-26T21:42:08.245839+0000","last_became_peered":"2024-04-26T21:42:08.245839+0000","last_unstale":"2024-04-26T21:42:11.236730+0000","last_undegraded":"2024-04-26T21:42:11.236730+0000","last_fullsized":"2024-04-26T21:42:11.236730+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:56.196138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028495+0000","last_change":"2024-04-26T21:42:09.263280+0000","last_active":"2024-04-26T21:42:12.028495+0000","last_peered":"2024-04-26T21:42:12.028495+0000","last_clean":"2024-04-26T21:42:12.028495+0000","last_became_active":"2024-04-26T21:42:09.260279+0000","last_became_peered":"2024-04-26T21:42:09.260279+0000","last_unstale":"2024-04-26T21:42:12.028495+0000","last_undegraded":"2024-04-26T21:42:12.028495+0000","last_fullsized":"2024-04-26T21:42:12.028495+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:54.394728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251034+0000","last_change":"2024-04-26T21:42:08.244889+0000","last_active":"2024-04-26T21:42:10.251034+0000","last_peered":"2024-04-26T21:42:10.251034+0000","last_clean":"2024-04-26T21:42:10.251034+0000","last_became_active":"2024-04-26T21:42:08.243921+0000","last_became_peered":"2024-04-26T21:42:08.243921+0000","last_unstale":"2024-04-26T21:42:10.251034+0000","last_undegraded":"2024-04-26T21:42:10.251034+0000","last_fullsized":"2024-04-26T21:42:10.251034+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:08.285259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251483+0000","last_change":"2024-04-26T21:42:09.258148+0000","last_active":"2024-04-26T21:42:10.251483+0000","last_peered":"2024-04-26T21:42:10.251483+0000","last_clean":"2024-04-26T21:42:10.251483+0000","last_became_active":"2024-04-26T21:42:09.257879+0000","last_became_peered":"2024-04-26T21:42:09.257879+0000","last_unstale":"2024-04-26T21:42:10.251483+0000","last_undegraded":"2024-04-26T21:42:10.251483+0000","last_fullsized":"2024-04-26T21:42:10.251483+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:44.259318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.249851+0000","last_change":"2024-04-26T21:42:08.251739+0000","last_active":"2024-04-26T21:42:10.249851+0000","last_peered":"2024-04-26T21:42:10.249851+0000","last_clean":"2024-04-26T21:42:10.249851+0000","last_became_active":"2024-04-26T21:42:08.251265+0000","last_became_peered":"2024-04-26T21:42:08.251265+0000","last_unstale":"2024-04-26T21:42:10.249851+0000","last_undegraded":"2024-04-26T21:42:10.249851+0000","last_fullsized":"2024-04-26T21:42:10.249851+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:03.455718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251635+0000","last_change":"2024-04-26T21:42:09.268561+0000","last_active":"2024-04-26T21:42:10.251635+0000","last_peered":"2024-04-26T21:42:10.251635+0000","last_clean":"2024-04-26T21:42:10.251635+0000","last_became_active":"2024-04-26T21:42:09.268285+0000","last_became_peered":"2024-04-26T21:42:09.268285+0000","last_unstale":"2024-04-26T21:42:10.251635+0000","last_undegraded":"2024-04-26T21:42:10.251635+0000","last_fullsized":"2024-04-26T21:42:10.251635+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:37.741942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251537+0000","last_change":"2024-04-26T21:42:08.251899+0000","last_active":"2024-04-26T21:42:10.251537+0000","last_peered":"2024-04-26T21:42:10.251537+0000","last_clean":"2024-04-26T21:42:10.251537+0000","last_became_active":"2024-04-26T21:42:08.251624+0000","last_became_peered":"2024-04-26T21:42:08.251624+0000","last_unstale":"2024-04-26T21:42:10.251537+0000","last_undegraded":"2024-04-26T21:42:10.251537+0000","last_fullsized":"2024-04-26T21:42:10.251537+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:37.868807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028726+0000","last_change":"2024-04-26T21:42:09.253482+0000","last_active":"2024-04-26T21:42:12.028726+0000","last_peered":"2024-04-26T21:42:12.028726+0000","last_clean":"2024-04-26T21:42:12.028726+0000","last_became_active":"2024-04-26T21:42:09.253184+0000","last_became_peered":"2024-04-26T21:42:09.253184+0000","last_unstale":"2024-04-26T21:42:12.028726+0000","last_undegraded":"2024-04-26T21:42:12.028726+0000","last_fullsized":"2024-04-26T21:42:12.028726+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.251086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.249795+0000","last_change":"2024-04-26T21:42:08.249157+0000","last_active":"2024-04-26T21:42:10.249795+0000","last_peered":"2024-04-26T21:42:10.249795+0000","last_clean":"2024-04-26T21:42:10.249795+0000","last_became_active":"2024-04-26T21:42:08.248811+0000","last_became_peered":"2024-04-26T21:42:08.248811+0000","last_unstale":"2024-04-26T21:42:10.249795+0000","last_undegraded":"2024-04-26T21:42:10.249795+0000","last_fullsized":"2024-04-26T21:42:10.249795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:25.798370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251949+0000","last_change":"2024-04-26T21:42:09.251492+0000","last_active":"2024-04-26T21:42:10.251949+0000","last_peered":"2024-04-26T21:42:10.251949+0000","last_clean":"2024-04-26T21:42:10.251949+0000","last_became_active":"2024-04-26T21:42:09.251194+0000","last_became_peered":"2024-04-26T21:42:09.251194+0000","last_unstale":"2024-04-26T21:42:10.251949+0000","last_undegraded":"2024-04-26T21:42:10.251949+0000","last_fullsized":"2024-04-26T21:42:10.251949+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:07.477516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028677+0000","last_change":"2024-04-26T21:42:08.246097+0000","last_active":"2024-04-26T21:42:12.028677+0000","last_peered":"2024-04-26T21:42:12.028677+0000","last_clean":"2024-04-26T21:42:12.028677+0000","last_became_active":"2024-04-26T21:42:08.243459+0000","last_became_peered":"2024-04-26T21:42:08.243459+0000","last_unstale":"2024-04-26T21:42:12.028677+0000","last_undegraded":"2024-04-26T21:42:12.028677+0000","last_fullsized":"2024-04-26T21:42:12.028677+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:17.342450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029580+0000","last_change":"2024-04-26T21:42:09.247416+0000","last_active":"2024-04-26T21:42:12.029580+0000","last_peered":"2024-04-26T21:42:12.029580+0000","last_clean":"2024-04-26T21:42:12.029580+0000","last_became_active":"2024-04-26T21:42:09.247141+0000","last_became_peered":"2024-04-26T21:42:09.247141+0000","last_unstale":"2024-04-26T21:42:12.029580+0000","last_undegraded":"2024-04-26T21:42:12.029580+0000","last_fullsized":"2024-04-26T21:42:12.029580+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:09.762828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"94'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236690+0000","last_change":"2024-04-26T21:42:08.251072+0000","last_active":"2024-04-26T21:42:11.236690+0000","last_peered":"2024-04-26T21:42:11.236690+0000","last_clean":"2024-04-26T21:42:11.236690+0000","last_became_active":"2024-04-26T21:42:08.250873+0000","last_became_peered":"2024-04-26T21:42:08.250873+0000","last_unstale":"2024-04-26T21:42:11.236690+0000","last_undegraded":"2024-04-26T21:42:11.236690+0000","last_fullsized":"2024-04-26T21:42:11.236690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:07.499452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029593+0000","last_change":"2024-04-26T21:42:09.251461+0000","last_active":"2024-04-26T21:42:12.029593+0000","last_peered":"2024-04-26T21:42:12.029593+0000","last_clean":"2024-04-26T21:42:12.029593+0000","last_became_active":"2024-04-26T21:42:09.248712+0000","last_became_peered":"2024-04-26T21:42:09.248712+0000","last_unstale":"2024-04-26T21:42:12.029593+0000","last_undegraded":"2024-04-26T21:42:12.029593+0000","last_fullsized":"2024-04-26T21:42:12.029593+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:45.160614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236776+0000","last_change":"2024-04-26T21:42:08.252141+0000","last_active":"2024-04-26T21:42:11.236776+0000","last_peered":"2024-04-26T21:42:11.236776+0000","last_clean":"2024-04-26T21:42:11.236776+0000","last_became_active":"2024-04-26T21:42:08.251718+0000","last_became_peered":"2024-04-26T21:42:08.251718+0000","last_unstale":"2024-04-26T21:42:11.236776+0000","last_undegraded":"2024-04-26T21:42:11.236776+0000","last_fullsized":"2024-04-26T21:42:11.236776+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:36.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251965+0000","last_change":"2024-04-26T21:42:09.255389+0000","last_active":"2024-04-26T21:42:10.251965+0000","last_peered":"2024-04-26T21:42:10.251965+0000","last_clean":"2024-04-26T21:42:10.251965+0000","last_became_active":"2024-04-26T21:42:09.254985+0000","last_became_peered":"2024-04-26T21:42:09.254985+0000","last_unstale":"2024-04-26T21:42:10.251965+0000","last_undegraded":"2024-04-26T21:42:10.251965+0000","last_fullsized":"2024-04-26T21:42:10.251965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:37.060054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252272+0000","last_change":"2024-04-26T21:42:08.249382+0000","last_active":"2024-04-26T21:42:10.252272+0000","last_peered":"2024-04-26T21:42:10.252272+0000","last_clean":"2024-04-26T21:42:10.252272+0000","last_became_active":"2024-04-26T21:42:08.248955+0000","last_became_peered":"2024-04-26T21:42:08.248955+0000","last_unstale":"2024-04-26T21:42:10.252272+0000","last_undegraded":"2024-04-26T21:42:10.252272+0000","last_fullsized":"2024-04-26T21:42:10.252272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:02.352623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.249830+0000","last_change":"2024-04-26T21:42:09.253849+0000","last_active":"2024-04-26T21:42:10.249830+0000","last_peered":"2024-04-26T21:42:10.249830+0000","last_clean":"2024-04-26T21:42:10.249830+0000","last_became_active":"2024-04-26T21:42:09.253586+0000","last_became_peered":"2024-04-26T21:42:09.253586+0000","last_unstale":"2024-04-26T21:42:10.249830+0000","last_undegraded":"2024-04-26T21:42:10.249830+0000","last_fullsized":"2024-04-26T21:42:10.249830+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:40.721633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029130+0000","last_change":"2024-04-26T21:42:08.248552+0000","last_active":"2024-04-26T21:42:12.029130+0000","last_peered":"2024-04-26T21:42:12.029130+0000","last_clean":"2024-04-26T21:42:12.029130+0000","last_became_active":"2024-04-26T21:42:08.248245+0000","last_became_peered":"2024-04-26T21:42:08.248245+0000","last_unstale":"2024-04-26T21:42:12.029130+0000","last_undegraded":"2024-04-26T21:42:12.029130+0000","last_fullsized":"2024-04-26T21:42:12.029130+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:13.241915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193063+0000","last_change":"2024-04-26T21:42:09.261713+0000","last_active":"2024-04-26T21:42:11.193063+0000","last_peered":"2024-04-26T21:42:11.193063+0000","last_clean":"2024-04-26T21:42:11.193063+0000","last_became_active":"2024-04-26T21:42:09.260930+0000","last_became_peered":"2024-04-26T21:42:09.260930+0000","last_unstale":"2024-04-26T21:42:11.193063+0000","last_undegraded":"2024-04-26T21:42:11.193063+0000","last_fullsized":"2024-04-26T21:42:11.193063+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:20.199567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251695+0000","last_change":"2024-04-26T21:42:08.245700+0000","last_active":"2024-04-26T21:42:10.251695+0000","last_peered":"2024-04-26T21:42:10.251695+0000","last_clean":"2024-04-26T21:42:10.251695+0000","last_became_active":"2024-04-26T21:42:08.245432+0000","last_became_peered":"2024-04-26T21:42:08.245432+0000","last_unstale":"2024-04-26T21:42:10.251695+0000","last_undegraded":"2024-04-26T21:42:10.251695+0000","last_fullsized":"2024-04-26T21:42:10.251695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:36.416932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251718+0000","last_change":"2024-04-26T21:42:09.268586+0000","last_active":"2024-04-26T21:42:10.251718+0000","last_peered":"2024-04-26T21:42:10.251718+0000","last_clean":"2024-04-26T21:42:10.251718+0000","last_became_active":"2024-04-26T21:42:09.268274+0000","last_became_peered":"2024-04-26T21:42:09.268274+0000","last_unstale":"2024-04-26T21:42:10.251718+0000","last_undegraded":"2024-04-26T21:42:10.251718+0000","last_fullsized":"2024-04-26T21:42:10.251718+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:48.828972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252100+0000","last_change":"2024-04-26T21:42:08.249281+0000","last_active":"2024-04-26T21:42:10.252100+0000","last_peered":"2024-04-26T21:42:10.252100+0000","last_clean":"2024-04-26T21:42:10.252100+0000","last_became_active":"2024-04-26T21:42:08.249008+0000","last_became_peered":"2024-04-26T21:42:08.249008+0000","last_unstale":"2024-04-26T21:42:10.252100+0000","last_undegraded":"2024-04-26T21:42:10.252100+0000","last_fullsized":"2024-04-26T21:42:10.252100+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:35.700742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029010+0000","last_change":"2024-04-26T21:42:09.261028+0000","last_active":"2024-04-26T21:42:12.029010+0000","last_peered":"2024-04-26T21:42:12.029010+0000","last_clean":"2024-04-26T21:42:12.029010+0000","last_became_active":"2024-04-26T21:42:09.260731+0000","last_became_peered":"2024-04-26T21:42:09.260731+0000","last_unstale":"2024-04-26T21:42:12.029010+0000","last_undegraded":"2024-04-26T21:42:12.029010+0000","last_fullsized":"2024-04-26T21:42:12.029010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:41.024073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028410+0000","last_change":"2024-04-26T21:42:08.246155+0000","last_active":"2024-04-26T21:42:12.028410+0000","last_peered":"2024-04-26T21:42:12.028410+0000","last_clean":"2024-04-26T21:42:12.028410+0000","last_became_active":"2024-04-26T21:42:08.245864+0000","last_became_peered":"2024-04-26T21:42:08.245864+0000","last_unstale":"2024-04-26T21:42:12.028410+0000","last_undegraded":"2024-04-26T21:42:12.028410+0000","last_fullsized":"2024-04-26T21:42:12.028410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:34.149794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028723+0000","last_change":"2024-04-26T21:42:09.258682+0000","last_active":"2024-04-26T21:42:12.028723+0000","last_peered":"2024-04-26T21:42:12.028723+0000","last_clean":"2024-04-26T21:42:12.028723+0000","last_became_active":"2024-04-26T21:42:09.258354+0000","last_became_peered":"2024-04-26T21:42:09.258354+0000","last_unstale":"2024-04-26T21:42:12.028723+0000","last_undegraded":"2024-04-26T21:42:12.028723+0000","last_fullsized":"2024-04-26T21:42:12.028723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:36.119046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029233+0000","last_change":"2024-04-26T21:42:08.244341+0000","last_active":"2024-04-26T21:42:12.029233+0000","last_peered":"2024-04-26T21:42:12.029233+0000","last_clean":"2024-04-26T21:42:12.029233+0000","last_became_active":"2024-04-26T21:42:08.244016+0000","last_became_peered":"2024-04-26T21:42:08.244016+0000","last_unstale":"2024-04-26T21:42:12.029233+0000","last_undegraded":"2024-04-26T21:42:12.029233+0000","last_fullsized":"2024-04-26T21:42:12.029233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:24.990650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":310,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028567+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:42:12.028567+0000","last_peered":"2024-04-26T21:42:12.028567+0000","last_clean":"2024-04-26T21:42:12.028567+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:42:12.028567+0000","last_undegraded":"2024-04-26T21:42:12.028567+0000","last_fullsized":"2024-04-26T21:42:12.028567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":1130455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":542247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1159168,"data_stored":540031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574937,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":540191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":540031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":540031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1163264,"data_stored":540101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1777664,"data_stored":1132717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:42:17.714 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:42:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.827+0000 7f7b449e7640 1 Processor -- start 2024-04-26T21:42:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.827+0000 7f7b449e7640 1 -- start start 2024-04-26T21:42:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b449e7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b400d06f0 0x7f7b400d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b449e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b449e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b400d0ba0 con 0x7f7b4015f3a0 2024-04-26T21:42:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b449e7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7b400d0e20 con 0x7f7b400d06f0 2024-04-26T21:42:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b449e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b400b1d70 con 0x7f7b400b2850 2024-04-26T21:42:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b37fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b400d06f0 0x7f7b400d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b3ed9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b3e59c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f7b4015f3a0 0x7f7b400d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:40546/0 (socket says 172.21.15.40:40546) 2024-04-26T21:42:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b3e59c640 1 -- 172.21.15.40:0/1550439276 learned_addr learned my addr 172.21.15.40:0/1550439276 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b37fff640 1 -- 172.21.15.40:0/1550439276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 msgr2=0x7f7b400b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b37fff640 1 --2- 172.21.15.40:0/1550439276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b37fff640 1 -- 172.21.15.40:0/1550439276 >> v1:172.21.15.184:6789/0 conn(0x7f7b4015f3a0 legacy=0x7f7b400d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b37fff640 1 -- 172.21.15.40:0/1550439276 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b400b2050 con 0x7f7b400d06f0 2024-04-26T21:42:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.828+0000 7f7b3ed9d640 1 --2- 172.21.15.40:0/1550439276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.829+0000 7f7b37fff640 1 --2- 172.21.15.40:0/1550439276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 0x7f7b400d1e80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b280097a0 tx=0x7f7b2802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad84bcef150fa943 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.829+0000 7f7b3dd9b640 1 -- 172.21.15.40:0/1550439276 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28037070 con 0x7f7b400d06f0 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b3dd9b640 1 -- 172.21.15.40:0/1550439276 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b28032440 con 0x7f7b400d06f0 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b3dd9b640 1 -- 172.21.15.40:0/1550439276 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28032810 con 0x7f7b400d06f0 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b449e7640 1 -- 172.21.15.40:0/1550439276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 msgr2=0x7f7b400d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b449e7640 1 --2- 172.21.15.40:0/1550439276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 0x7f7b400d1e80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b280097a0 tx=0x7f7b2802a180 comp rx=0 tx=0).stop 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b449e7640 1 -- 172.21.15.40:0/1550439276 shutdown_connections 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b449e7640 1 --2- 172.21.15.40:0/1550439276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b449e7640 1 --2- 172.21.15.40:0/1550439276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 0x7f7b400d1e80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b449e7640 1 -- 172.21.15.40:0/1550439276 >> 172.21.15.40:0/1550439276 conn(0x7f7b400d4f90 msgr2=0x7f7b400d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b449e7640 1 -- 172.21.15.40:0/1550439276 shutdown_connections 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.830+0000 7f7b449e7640 1 -- 172.21.15.40:0/1550439276 wait complete. 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.832+0000 7f7b449e7640 1 Processor -- start 2024-04-26T21:42:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.832+0000 7f7b449e7640 1 -- start start 2024-04-26T21:42:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.832+0000 7f7b449e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.832+0000 7f7b449e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 0x7f7b400cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.832+0000 7f7b449e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b4015f3a0 0x7f7b400cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.832+0000 7f7b449e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4015f900 con 0x7f7b4015f3a0 2024-04-26T21:42:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.832+0000 7f7b449e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4015f780 con 0x7f7b400d06f0 2024-04-26T21:42:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.832+0000 7f7b449e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b400d23c0 con 0x7f7b400b2850 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 0x7f7b400cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b3ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b4015f3a0 0x7f7b400cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b3e59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 0x7f7b400cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:43840/0 (socket says 172.21.15.40:43840) 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b37fff640 1 -- 172.21.15.40:0/3990744466 learned_addr learned my addr 172.21.15.40:0/3990744466 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b3ed9d640 1 -- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 msgr2=0x7f7b400cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b3ed9d640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b3ed9d640 1 -- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 msgr2=0x7f7b400cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b3ed9d640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 0x7f7b400cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b3ed9d640 1 -- 172.21.15.40:0/3990744466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b400ce9a0 con 0x7f7b4015f3a0 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b3ed9d640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b4015f3a0 0x7f7b400cb3c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800efc0 tx=0x7f7b3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b36ffd640 1 -- 172.21.15.40:0/3990744466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38018030 con 0x7f7b4015f3a0 2024-04-26T21:42:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b36ffd640 1 -- 172.21.15.40:0/3990744466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3800cd30 con 0x7f7b4015f3a0 2024-04-26T21:42:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b400c8f30 con 0x7f7b4015f3a0 2024-04-26T21:42:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b36ffd640 1 -- 172.21.15.40:0/3990744466 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38003bb0 con 0x7f7b4015f3a0 2024-04-26T21:42:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.833+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b400c94f0 con 0x7f7b4015f3a0 2024-04-26T21:42:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.835+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b400694f0 con 0x7f7b4015f3a0 2024-04-26T21:42:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.835+0000 7f7b36ffd640 1 -- 172.21.15.40:0/3990744466 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b38005d10 con 0x7f7b4015f3a0 2024-04-26T21:42:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.835+0000 7f7b36ffd640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2006c8f0 0x7f7b2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.836+0000 7f7b36ffd640 1 -- 172.21.15.40:0/3990744466 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f7b3808c480 con 0x7f7b4015f3a0 2024-04-26T21:42:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.836+0000 7f7b3e59c640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2006c8f0 0x7f7b2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.836+0000 7f7b3e59c640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2006c8f0 0x7f7b2006edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7b30007fd0 tx=0x7f7b30007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:17.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:17.840+0000 7f7b36ffd640 1 -- 172.21.15.40:0/3990744466 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7b38022070 con 0x7f7b4015f3a0 2024-04-26T21:42:18.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.025+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b400b6860 con 0x7f7b2006c8f0 2024-04-26T21:42:18.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.032+0000 7f7b36ffd640 1 -- 172.21.15.40:0/3990744466 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252076 (secure 0 0 0) 0x7f7b400b6860 con 0x7f7b2006c8f0 2024-04-26T21:42:18.036 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:18.037 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:42:18.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2006c8f0 msgr2=0x7f7b2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2006c8f0 0x7f7b2006edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7b30007fd0 tx=0x7f7b30007e80 comp rx=0 tx=0).stop 2024-04-26T21:42:18.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b4015f3a0 msgr2=0x7f7b400cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b4015f3a0 0x7f7b400cb3c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800efc0 tx=0x7f7b3800c490 comp rx=0 tx=0).stop 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 shutdown_connections 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2006c8f0 0x7f7b2006edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b4015f3a0 0x7f7b400cb3c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400d06f0 0x7f7b400cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 --2- 172.21.15.40:0/3990744466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400b2850 0x7f7b400cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 >> 172.21.15.40:0/3990744466 conn(0x7f7b400d4f90 msgr2=0x7f7b400b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 shutdown_connections 2024-04-26T21:42:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.035+0000 7f7b449e7640 1 -- 172.21.15.40:0/3990744466 wait complete. 2024-04-26T21:42:18.047 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-04-26T21:42:16.100717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251420,"kb_used_data":10280,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754723300,"statfs":{"total":773094113280,"available":772836659200,"internally_reserved":0,"allocated":10526720,"data_stored":5505804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.870736"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193300+0000","last_change":"2024-04-26T21:42:09.264494+0000","last_active":"2024-04-26T21:42:11.193300+0000","last_peered":"2024-04-26T21:42:11.193300+0000","last_clean":"2024-04-26T21:42:11.193300+0000","last_became_active":"2024-04-26T21:42:09.263985+0000","last_became_peered":"2024-04-26T21:42:09.263985+0000","last_unstale":"2024-04-26T21:42:11.193300+0000","last_undegraded":"2024-04-26T21:42:11.193300+0000","last_fullsized":"2024-04-26T21:42:11.193300+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:37.060054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252392+0000","last_change":"2024-04-26T21:42:09.258182+0000","last_active":"2024-04-26T21:42:10.252392+0000","last_peered":"2024-04-26T21:42:10.252392+0000","last_clean":"2024-04-26T21:42:10.252392+0000","last_became_active":"2024-04-26T21:42:09.257837+0000","last_became_peered":"2024-04-26T21:42:09.257837+0000","last_unstale":"2024-04-26T21:42:10.252392+0000","last_undegraded":"2024-04-26T21:42:10.252392+0000","last_fullsized":"2024-04-26T21:42:10.252392+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:06.225847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193171+0000","last_change":"2024-04-26T21:42:09.257887+0000","last_active":"2024-04-26T21:42:11.193171+0000","last_peered":"2024-04-26T21:42:11.193171+0000","last_clean":"2024-04-26T21:42:11.193171+0000","last_became_active":"2024-04-26T21:42:09.257588+0000","last_became_peered":"2024-04-26T21:42:09.257588+0000","last_unstale":"2024-04-26T21:42:11.193171+0000","last_undegraded":"2024-04-26T21:42:11.193171+0000","last_fullsized":"2024-04-26T21:42:11.193171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:37.741942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029009+0000","last_change":"2024-04-26T21:42:09.267138+0000","last_active":"2024-04-26T21:42:12.029009+0000","last_peered":"2024-04-26T21:42:12.029009+0000","last_clean":"2024-04-26T21:42:12.029009+0000","last_became_active":"2024-04-26T21:42:09.266785+0000","last_became_peered":"2024-04-26T21:42:09.266785+0000","last_unstale":"2024-04-26T21:42:12.029009+0000","last_undegraded":"2024-04-26T21:42:12.029009+0000","last_fullsized":"2024-04-26T21:42:12.029009+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236482+0000","last_change":"2024-04-26T21:42:09.251296+0000","last_active":"2024-04-26T21:42:11.236482+0000","last_peered":"2024-04-26T21:42:11.236482+0000","last_clean":"2024-04-26T21:42:11.236482+0000","last_became_active":"2024-04-26T21:42:09.250106+0000","last_became_peered":"2024-04-26T21:42:09.250106+0000","last_unstale":"2024-04-26T21:42:11.236482+0000","last_undegraded":"2024-04-26T21:42:11.236482+0000","last_fullsized":"2024-04-26T21:42:11.236482+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:37.735401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028327+0000","last_change":"2024-04-26T21:42:09.262844+0000","last_active":"2024-04-26T21:42:12.028327+0000","last_peered":"2024-04-26T21:42:12.028327+0000","last_clean":"2024-04-26T21:42:12.028327+0000","last_became_active":"2024-04-26T21:42:09.260424+0000","last_became_peered":"2024-04-26T21:42:09.260424+0000","last_unstale":"2024-04-26T21:42:12.028327+0000","last_undegraded":"2024-04-26T21:42:12.028327+0000","last_fullsized":"2024-04-26T21:42:12.028327+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:37.051031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028460+0000","last_change":"2024-04-26T21:42:09.258736+0000","last_active":"2024-04-26T21:42:12.028460+0000","last_peered":"2024-04-26T21:42:12.028460+0000","last_clean":"2024-04-26T21:42:12.028460+0000","last_became_active":"2024-04-26T21:42:09.258162+0000","last_became_peered":"2024-04-26T21:42:09.258162+0000","last_unstale":"2024-04-26T21:42:12.028460+0000","last_undegraded":"2024-04-26T21:42:12.028460+0000","last_fullsized":"2024-04-26T21:42:12.028460+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:02.398915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028708+0000","last_change":"2024-04-26T21:42:09.255437+0000","last_active":"2024-04-26T21:42:12.028708+0000","last_peered":"2024-04-26T21:42:12.028708+0000","last_clean":"2024-04-26T21:42:12.028708+0000","last_became_active":"2024-04-26T21:42:09.255146+0000","last_became_peered":"2024-04-26T21:42:09.255146+0000","last_unstale":"2024-04-26T21:42:12.028708+0000","last_undegraded":"2024-04-26T21:42:12.028708+0000","last_fullsized":"2024-04-26T21:42:12.028708+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:24.183661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236546+0000","last_change":"2024-04-26T21:42:09.248867+0000","last_active":"2024-04-26T21:42:11.236546+0000","last_peered":"2024-04-26T21:42:11.236546+0000","last_clean":"2024-04-26T21:42:11.236546+0000","last_became_active":"2024-04-26T21:42:09.248602+0000","last_became_peered":"2024-04-26T21:42:09.248602+0000","last_unstale":"2024-04-26T21:42:11.236546+0000","last_undegraded":"2024-04-26T21:42:11.236546+0000","last_fullsized":"2024-04-26T21:42:11.236546+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:53.382141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.250166+0000","last_change":"2024-04-26T21:42:09.268509+0000","last_active":"2024-04-26T21:42:10.250166+0000","last_peered":"2024-04-26T21:42:10.250166+0000","last_clean":"2024-04-26T21:42:10.250166+0000","last_became_active":"2024-04-26T21:42:09.268187+0000","last_became_peered":"2024-04-26T21:42:09.268187+0000","last_unstale":"2024-04-26T21:42:10.250166+0000","last_undegraded":"2024-04-26T21:42:10.250166+0000","last_fullsized":"2024-04-26T21:42:10.250166+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:23.869675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236564+0000","last_change":"2024-04-26T21:42:09.254222+0000","last_active":"2024-04-26T21:42:11.236564+0000","last_peered":"2024-04-26T21:42:11.236564+0000","last_clean":"2024-04-26T21:42:11.236564+0000","last_became_active":"2024-04-26T21:42:09.253953+0000","last_became_peered":"2024-04-26T21:42:09.253953+0000","last_unstale":"2024-04-26T21:42:11.236564+0000","last_undegraded":"2024-04-26T21:42:11.236564+0000","last_fullsized":"2024-04-26T21:42:11.236564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:33.561796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.250027+0000","last_change":"2024-04-26T21:42:09.258563+0000","last_active":"2024-04-26T21:42:10.250027+0000","last_peered":"2024-04-26T21:42:10.250027+0000","last_clean":"2024-04-26T21:42:10.250027+0000","last_became_active":"2024-04-26T21:42:09.258251+0000","last_became_peered":"2024-04-26T21:42:09.258251+0000","last_unstale":"2024-04-26T21:42:10.250027+0000","last_undegraded":"2024-04-26T21:42:10.250027+0000","last_fullsized":"2024-04-26T21:42:10.250027+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:35.112901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028734+0000","last_change":"2024-04-26T21:42:09.254193+0000","last_active":"2024-04-26T21:42:12.028734+0000","last_peered":"2024-04-26T21:42:12.028734+0000","last_clean":"2024-04-26T21:42:12.028734+0000","last_became_active":"2024-04-26T21:42:09.253750+0000","last_became_peered":"2024-04-26T21:42:09.253750+0000","last_unstale":"2024-04-26T21:42:12.028734+0000","last_undegraded":"2024-04-26T21:42:12.028734+0000","last_fullsized":"2024-04-26T21:42:12.028734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:11.988701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236104+0000","last_change":"2024-04-26T21:42:09.254666+0000","last_active":"2024-04-26T21:42:11.236104+0000","last_peered":"2024-04-26T21:42:11.236104+0000","last_clean":"2024-04-26T21:42:11.236104+0000","last_became_active":"2024-04-26T21:42:09.254360+0000","last_became_peered":"2024-04-26T21:42:09.254360+0000","last_unstale":"2024-04-26T21:42:11.236104+0000","last_undegraded":"2024-04-26T21:42:11.236104+0000","last_fullsized":"2024-04-26T21:42:11.236104+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:53.266651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028749+0000","last_change":"2024-04-26T21:42:09.258644+0000","last_active":"2024-04-26T21:42:12.028749+0000","last_peered":"2024-04-26T21:42:12.028749+0000","last_clean":"2024-04-26T21:42:12.028749+0000","last_became_active":"2024-04-26T21:42:09.258337+0000","last_became_peered":"2024-04-26T21:42:09.258337+0000","last_unstale":"2024-04-26T21:42:12.028749+0000","last_undegraded":"2024-04-26T21:42:12.028749+0000","last_fullsized":"2024-04-26T21:42:12.028749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:48.828972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028312+0000","last_change":"2024-04-26T21:42:09.262980+0000","last_active":"2024-04-26T21:42:12.028312+0000","last_peered":"2024-04-26T21:42:12.028312+0000","last_clean":"2024-04-26T21:42:12.028312+0000","last_became_active":"2024-04-26T21:42:09.262473+0000","last_became_peered":"2024-04-26T21:42:09.262473+0000","last_unstale":"2024-04-26T21:42:12.028312+0000","last_undegraded":"2024-04-26T21:42:12.028312+0000","last_fullsized":"2024-04-26T21:42:12.028312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:24.095510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029673+0000","last_change":"2024-04-26T21:42:09.261833+0000","last_active":"2024-04-26T21:42:12.029673+0000","last_peered":"2024-04-26T21:42:12.029673+0000","last_clean":"2024-04-26T21:42:12.029673+0000","last_became_active":"2024-04-26T21:42:09.261276+0000","last_became_peered":"2024-04-26T21:42:09.261276+0000","last_unstale":"2024-04-26T21:42:12.029673+0000","last_undegraded":"2024-04-26T21:42:12.029673+0000","last_fullsized":"2024-04-26T21:42:12.029673+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:45.799211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193132+0000","last_change":"2024-04-26T21:42:09.256730+0000","last_active":"2024-04-26T21:42:11.193132+0000","last_peered":"2024-04-26T21:42:11.193132+0000","last_clean":"2024-04-26T21:42:11.193132+0000","last_became_active":"2024-04-26T21:42:09.256384+0000","last_became_peered":"2024-04-26T21:42:09.256384+0000","last_unstale":"2024-04-26T21:42:11.193132+0000","last_undegraded":"2024-04-26T21:42:11.193132+0000","last_fullsized":"2024-04-26T21:42:11.193132+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:44.259318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236645+0000","last_change":"2024-04-26T21:42:09.254731+0000","last_active":"2024-04-26T21:42:11.236645+0000","last_peered":"2024-04-26T21:42:11.236645+0000","last_clean":"2024-04-26T21:42:11.236645+0000","last_became_active":"2024-04-26T21:42:09.254458+0000","last_became_peered":"2024-04-26T21:42:09.254458+0000","last_unstale":"2024-04-26T21:42:11.236645+0000","last_undegraded":"2024-04-26T21:42:11.236645+0000","last_fullsized":"2024-04-26T21:42:11.236645+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:02.601613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029155+0000","last_change":"2024-04-26T21:42:09.259722+0000","last_active":"2024-04-26T21:42:12.029155+0000","last_peered":"2024-04-26T21:42:12.029155+0000","last_clean":"2024-04-26T21:42:12.029155+0000","last_became_active":"2024-04-26T21:42:09.259485+0000","last_became_peered":"2024-04-26T21:42:09.259485+0000","last_unstale":"2024-04-26T21:42:12.029155+0000","last_undegraded":"2024-04-26T21:42:12.029155+0000","last_fullsized":"2024-04-26T21:42:12.029155+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:38.522173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236220+0000","last_change":"2024-04-26T21:42:09.246154+0000","last_active":"2024-04-26T21:42:11.236220+0000","last_peered":"2024-04-26T21:42:11.236220+0000","last_clean":"2024-04-26T21:42:11.236220+0000","last_became_active":"2024-04-26T21:42:09.245803+0000","last_became_peered":"2024-04-26T21:42:09.245803+0000","last_unstale":"2024-04-26T21:42:11.236220+0000","last_undegraded":"2024-04-26T21:42:11.236220+0000","last_fullsized":"2024-04-26T21:42:11.236220+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:34.133725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236133+0000","last_change":"2024-04-26T21:42:09.251971+0000","last_active":"2024-04-26T21:42:11.236133+0000","last_peered":"2024-04-26T21:42:11.236133+0000","last_clean":"2024-04-26T21:42:11.236133+0000","last_became_active":"2024-04-26T21:42:09.251677+0000","last_became_peered":"2024-04-26T21:42:09.251677+0000","last_unstale":"2024-04-26T21:42:11.236133+0000","last_undegraded":"2024-04-26T21:42:11.236133+0000","last_fullsized":"2024-04-26T21:42:11.236133+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:25.996727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251603+0000","last_change":"2024-04-26T21:42:09.268630+0000","last_active":"2024-04-26T21:42:10.251603+0000","last_peered":"2024-04-26T21:42:10.251603+0000","last_clean":"2024-04-26T21:42:10.251603+0000","last_became_active":"2024-04-26T21:42:09.268234+0000","last_became_peered":"2024-04-26T21:42:09.268234+0000","last_unstale":"2024-04-26T21:42:10.251603+0000","last_undegraded":"2024-04-26T21:42:10.251603+0000","last_fullsized":"2024-04-26T21:42:10.251603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:11.988701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193125+0000","last_change":"2024-04-26T21:42:09.255874+0000","last_active":"2024-04-26T21:42:11.193125+0000","last_peered":"2024-04-26T21:42:11.193125+0000","last_clean":"2024-04-26T21:42:11.193125+0000","last_became_active":"2024-04-26T21:42:09.255549+0000","last_became_peered":"2024-04-26T21:42:09.255549+0000","last_unstale":"2024-04-26T21:42:11.193125+0000","last_undegraded":"2024-04-26T21:42:11.193125+0000","last_fullsized":"2024-04-26T21:42:11.193125+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:44.735505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251805+0000","last_change":"2024-04-26T21:42:09.250715+0000","last_active":"2024-04-26T21:42:10.251805+0000","last_peered":"2024-04-26T21:42:10.251805+0000","last_clean":"2024-04-26T21:42:10.251805+0000","last_became_active":"2024-04-26T21:42:09.250409+0000","last_became_peered":"2024-04-26T21:42:09.250409+0000","last_unstale":"2024-04-26T21:42:10.251805+0000","last_undegraded":"2024-04-26T21:42:10.251805+0000","last_fullsized":"2024-04-26T21:42:10.251805+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:14.247992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236578+0000","last_change":"2024-04-26T21:42:09.247745+0000","last_active":"2024-04-26T21:42:11.236578+0000","last_peered":"2024-04-26T21:42:11.236578+0000","last_clean":"2024-04-26T21:42:11.236578+0000","last_became_active":"2024-04-26T21:42:09.247421+0000","last_became_peered":"2024-04-26T21:42:09.247421+0000","last_unstale":"2024-04-26T21:42:11.236578+0000","last_undegraded":"2024-04-26T21:42:11.236578+0000","last_fullsized":"2024-04-26T21:42:11.236578+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:03.358699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.250054+0000","last_change":"2024-04-26T21:42:09.250974+0000","last_active":"2024-04-26T21:42:10.250054+0000","last_peered":"2024-04-26T21:42:10.250054+0000","last_clean":"2024-04-26T21:42:10.250054+0000","last_became_active":"2024-04-26T21:42:09.250689+0000","last_became_peered":"2024-04-26T21:42:09.250689+0000","last_unstale":"2024-04-26T21:42:10.250054+0000","last_undegraded":"2024-04-26T21:42:10.250054+0000","last_fullsized":"2024-04-26T21:42:10.250054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:19.164424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029308+0000","last_change":"2024-04-26T21:42:09.259454+0000","last_active":"2024-04-26T21:42:12.029308+0000","last_peered":"2024-04-26T21:42:12.029308+0000","last_clean":"2024-04-26T21:42:12.029308+0000","last_became_active":"2024-04-26T21:42:09.259169+0000","last_became_peered":"2024-04-26T21:42:09.259169+0000","last_unstale":"2024-04-26T21:42:12.029308+0000","last_undegraded":"2024-04-26T21:42:12.029308+0000","last_fullsized":"2024-04-26T21:42:12.029308+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028596+0000","last_change":"2024-04-26T21:42:09.250457+0000","last_active":"2024-04-26T21:42:12.028596+0000","last_peered":"2024-04-26T21:42:12.028596+0000","last_clean":"2024-04-26T21:42:12.028596+0000","last_became_active":"2024-04-26T21:42:09.250185+0000","last_became_peered":"2024-04-26T21:42:09.250185+0000","last_unstale":"2024-04-26T21:42:12.028596+0000","last_undegraded":"2024-04-26T21:42:12.028596+0000","last_fullsized":"2024-04-26T21:42:12.028596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:01.361591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236454+0000","last_change":"2024-04-26T21:42:09.247757+0000","last_active":"2024-04-26T21:42:11.236454+0000","last_peered":"2024-04-26T21:42:11.236454+0000","last_clean":"2024-04-26T21:42:11.236454+0000","last_became_active":"2024-04-26T21:42:09.247514+0000","last_became_peered":"2024-04-26T21:42:09.247514+0000","last_unstale":"2024-04-26T21:42:11.236454+0000","last_undegraded":"2024-04-26T21:42:11.236454+0000","last_fullsized":"2024-04-26T21:42:11.236454+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:25.765330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251325+0000","last_change":"2024-04-26T21:42:09.268310+0000","last_active":"2024-04-26T21:42:10.251325+0000","last_peered":"2024-04-26T21:42:10.251325+0000","last_clean":"2024-04-26T21:42:10.251325+0000","last_became_active":"2024-04-26T21:42:09.268014+0000","last_became_peered":"2024-04-26T21:42:09.268014+0000","last_unstale":"2024-04-26T21:42:10.251325+0000","last_undegraded":"2024-04-26T21:42:10.251325+0000","last_fullsized":"2024-04-26T21:42:10.251325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:44.735505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028320+0000","last_change":"2024-04-26T21:42:09.262763+0000","last_active":"2024-04-26T21:42:12.028320+0000","last_peered":"2024-04-26T21:42:12.028320+0000","last_clean":"2024-04-26T21:42:12.028320+0000","last_became_active":"2024-04-26T21:42:09.262109+0000","last_became_peered":"2024-04-26T21:42:09.262109+0000","last_unstale":"2024-04-26T21:42:12.028320+0000","last_undegraded":"2024-04-26T21:42:12.028320+0000","last_fullsized":"2024-04-26T21:42:12.028320+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:23.458317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029031+0000","last_change":"2024-04-26T21:42:09.261233+0000","last_active":"2024-04-26T21:42:12.029031+0000","last_peered":"2024-04-26T21:42:12.029031+0000","last_clean":"2024-04-26T21:42:12.029031+0000","last_became_active":"2024-04-26T21:42:09.260312+0000","last_became_peered":"2024-04-26T21:42:09.260312+0000","last_unstale":"2024-04-26T21:42:12.029031+0000","last_undegraded":"2024-04-26T21:42:12.029031+0000","last_fullsized":"2024-04-26T21:42:12.029031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:59.637256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"93'4","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252140+0000","last_change":"2024-04-26T21:42:08.248939+0000","last_active":"2024-04-26T21:42:10.252140+0000","last_peered":"2024-04-26T21:42:10.252140+0000","last_clean":"2024-04-26T21:42:10.252140+0000","last_became_active":"2024-04-26T21:42:08.248609+0000","last_became_peered":"2024-04-26T21:42:08.248609+0000","last_unstale":"2024-04-26T21:42:10.252140+0000","last_undegraded":"2024-04-26T21:42:10.252140+0000","last_fullsized":"2024-04-26T21:42:10.252140+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:52.376064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029412+0000","last_change":"2024-04-26T21:42:09.248275+0000","last_active":"2024-04-26T21:42:12.029412+0000","last_peered":"2024-04-26T21:42:12.029412+0000","last_clean":"2024-04-26T21:42:12.029412+0000","last_became_active":"2024-04-26T21:42:09.247956+0000","last_became_peered":"2024-04-26T21:42:09.247956+0000","last_unstale":"2024-04-26T21:42:12.029412+0000","last_undegraded":"2024-04-26T21:42:12.029412+0000","last_fullsized":"2024-04-26T21:42:12.029412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:59.762871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236635+0000","last_change":"2024-04-26T21:42:08.250948+0000","last_active":"2024-04-26T21:42:11.236635+0000","last_peered":"2024-04-26T21:42:11.236635+0000","last_clean":"2024-04-26T21:42:11.236635+0000","last_became_active":"2024-04-26T21:42:08.250565+0000","last_became_peered":"2024-04-26T21:42:08.250565+0000","last_unstale":"2024-04-26T21:42:11.236635+0000","last_undegraded":"2024-04-26T21:42:11.236635+0000","last_fullsized":"2024-04-26T21:42:11.236635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:39.819647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028634+0000","last_change":"2024-04-26T21:42:09.254560+0000","last_active":"2024-04-26T21:42:12.028634+0000","last_peered":"2024-04-26T21:42:12.028634+0000","last_clean":"2024-04-26T21:42:12.028634+0000","last_became_active":"2024-04-26T21:42:09.254271+0000","last_became_peered":"2024-04-26T21:42:09.254271+0000","last_unstale":"2024-04-26T21:42:12.028634+0000","last_undegraded":"2024-04-26T21:42:12.028634+0000","last_fullsized":"2024-04-26T21:42:12.028634+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:53.785856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028604+0000","last_change":"2024-04-26T21:42:08.248518+0000","last_active":"2024-04-26T21:42:12.028604+0000","last_peered":"2024-04-26T21:42:12.028604+0000","last_clean":"2024-04-26T21:42:12.028604+0000","last_became_active":"2024-04-26T21:42:08.248181+0000","last_became_peered":"2024-04-26T21:42:08.248181+0000","last_unstale":"2024-04-26T21:42:12.028604+0000","last_undegraded":"2024-04-26T21:42:12.028604+0000","last_fullsized":"2024-04-26T21:42:12.028604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:19.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029669+0000","last_change":"2024-04-26T21:42:09.262453+0000","last_active":"2024-04-26T21:42:12.029669+0000","last_peered":"2024-04-26T21:42:12.029669+0000","last_clean":"2024-04-26T21:42:12.029669+0000","last_became_active":"2024-04-26T21:42:09.262189+0000","last_became_peered":"2024-04-26T21:42:09.262189+0000","last_unstale":"2024-04-26T21:42:12.029669+0000","last_undegraded":"2024-04-26T21:42:12.029669+0000","last_fullsized":"2024-04-26T21:42:12.029669+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:36.275442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252322+0000","last_change":"2024-04-26T21:42:08.247033+0000","last_active":"2024-04-26T21:42:10.252322+0000","last_peered":"2024-04-26T21:42:10.252322+0000","last_clean":"2024-04-26T21:42:10.252322+0000","last_became_active":"2024-04-26T21:42:08.246730+0000","last_became_peered":"2024-04-26T21:42:08.246730+0000","last_unstale":"2024-04-26T21:42:10.252322+0000","last_undegraded":"2024-04-26T21:42:10.252322+0000","last_fullsized":"2024-04-26T21:42:10.252322+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:01.595536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028614+0000","last_change":"2024-04-26T21:42:09.257950+0000","last_active":"2024-04-26T21:42:12.028614+0000","last_peered":"2024-04-26T21:42:12.028614+0000","last_clean":"2024-04-26T21:42:12.028614+0000","last_became_active":"2024-04-26T21:42:09.257634+0000","last_became_peered":"2024-04-26T21:42:09.257634+0000","last_unstale":"2024-04-26T21:42:12.028614+0000","last_undegraded":"2024-04-26T21:42:12.028614+0000","last_fullsized":"2024-04-26T21:42:12.028614+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:25.097224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236548+0000","last_change":"2024-04-26T21:42:08.251049+0000","last_active":"2024-04-26T21:42:11.236548+0000","last_peered":"2024-04-26T21:42:11.236548+0000","last_clean":"2024-04-26T21:42:11.236548+0000","last_became_active":"2024-04-26T21:42:08.250616+0000","last_became_peered":"2024-04-26T21:42:08.250616+0000","last_unstale":"2024-04-26T21:42:11.236548+0000","last_undegraded":"2024-04-26T21:42:11.236548+0000","last_fullsized":"2024-04-26T21:42:11.236548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:22.339132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251987+0000","last_change":"2024-04-26T21:42:09.253878+0000","last_active":"2024-04-26T21:42:10.251987+0000","last_peered":"2024-04-26T21:42:10.251987+0000","last_clean":"2024-04-26T21:42:10.251987+0000","last_became_active":"2024-04-26T21:42:09.253619+0000","last_became_peered":"2024-04-26T21:42:09.253619+0000","last_unstale":"2024-04-26T21:42:10.251987+0000","last_undegraded":"2024-04-26T21:42:10.251987+0000","last_fullsized":"2024-04-26T21:42:10.251987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:18.430927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029445+0000","last_change":"2024-04-26T21:42:08.247064+0000","last_active":"2024-04-26T21:42:12.029445+0000","last_peered":"2024-04-26T21:42:12.029445+0000","last_clean":"2024-04-26T21:42:12.029445+0000","last_became_active":"2024-04-26T21:42:08.246780+0000","last_became_peered":"2024-04-26T21:42:08.246780+0000","last_unstale":"2024-04-26T21:42:12.029445+0000","last_undegraded":"2024-04-26T21:42:12.029445+0000","last_fullsized":"2024-04-26T21:42:12.029445+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:24.759253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028709+0000","last_change":"2024-04-26T21:42:09.260041+0000","last_active":"2024-04-26T21:42:12.028709+0000","last_peered":"2024-04-26T21:42:12.028709+0000","last_clean":"2024-04-26T21:42:12.028709+0000","last_became_active":"2024-04-26T21:42:09.253986+0000","last_became_peered":"2024-04-26T21:42:09.253986+0000","last_unstale":"2024-04-26T21:42:12.028709+0000","last_undegraded":"2024-04-26T21:42:12.028709+0000","last_fullsized":"2024-04-26T21:42:12.028709+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:54.110266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.192965+0000","last_change":"2024-04-26T21:42:08.239869+0000","last_active":"2024-04-26T21:42:11.192965+0000","last_peered":"2024-04-26T21:42:11.192965+0000","last_clean":"2024-04-26T21:42:11.192965+0000","last_became_active":"2024-04-26T21:42:08.239585+0000","last_became_peered":"2024-04-26T21:42:08.239585+0000","last_unstale":"2024-04-26T21:42:11.192965+0000","last_undegraded":"2024-04-26T21:42:11.192965+0000","last_fullsized":"2024-04-26T21:42:11.192965+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:08.285259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028903+0000","last_change":"2024-04-26T21:42:09.264591+0000","last_active":"2024-04-26T21:42:12.028903+0000","last_peered":"2024-04-26T21:42:12.028903+0000","last_clean":"2024-04-26T21:42:12.028903+0000","last_became_active":"2024-04-26T21:42:09.264287+0000","last_became_peered":"2024-04-26T21:42:09.264287+0000","last_unstale":"2024-04-26T21:42:12.028903+0000","last_undegraded":"2024-04-26T21:42:12.028903+0000","last_fullsized":"2024-04-26T21:42:12.028903+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:18.430927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028384+0000","last_change":"2024-04-26T21:42:08.243726+0000","last_active":"2024-04-26T21:42:12.028384+0000","last_peered":"2024-04-26T21:42:12.028384+0000","last_clean":"2024-04-26T21:42:12.028384+0000","last_became_active":"2024-04-26T21:42:08.243450+0000","last_became_peered":"2024-04-26T21:42:08.243450+0000","last_unstale":"2024-04-26T21:42:12.028384+0000","last_undegraded":"2024-04-26T21:42:12.028384+0000","last_fullsized":"2024-04-26T21:42:12.028384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:12.743868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236495+0000","last_change":"2024-04-26T21:42:09.258072+0000","last_active":"2024-04-26T21:42:11.236495+0000","last_peered":"2024-04-26T21:42:11.236495+0000","last_clean":"2024-04-26T21:42:11.236495+0000","last_became_active":"2024-04-26T21:42:09.257789+0000","last_became_peered":"2024-04-26T21:42:09.257789+0000","last_unstale":"2024-04-26T21:42:11.236495+0000","last_undegraded":"2024-04-26T21:42:11.236495+0000","last_fullsized":"2024-04-26T21:42:11.236495+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:01.023671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252020+0000","last_change":"2024-04-26T21:42:08.249432+0000","last_active":"2024-04-26T21:42:10.252020+0000","last_peered":"2024-04-26T21:42:10.252020+0000","last_clean":"2024-04-26T21:42:10.252020+0000","last_became_active":"2024-04-26T21:42:08.249064+0000","last_became_peered":"2024-04-26T21:42:08.249064+0000","last_unstale":"2024-04-26T21:42:10.252020+0000","last_undegraded":"2024-04-26T21:42:10.252020+0000","last_fullsized":"2024-04-26T21:42:10.252020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:37.516096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028753+0000","last_change":"2024-04-26T21:42:09.255185+0000","last_active":"2024-04-26T21:42:12.028753+0000","last_peered":"2024-04-26T21:42:12.028753+0000","last_clean":"2024-04-26T21:42:12.028753+0000","last_became_active":"2024-04-26T21:42:09.254903+0000","last_became_peered":"2024-04-26T21:42:09.254903+0000","last_unstale":"2024-04-26T21:42:12.028753+0000","last_undegraded":"2024-04-26T21:42:12.028753+0000","last_fullsized":"2024-04-26T21:42:12.028753+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:11.619631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193218+0000","last_change":"2024-04-26T21:42:08.243997+0000","last_active":"2024-04-26T21:42:11.193218+0000","last_peered":"2024-04-26T21:42:11.193218+0000","last_clean":"2024-04-26T21:42:11.193218+0000","last_became_active":"2024-04-26T21:42:08.243708+0000","last_became_peered":"2024-04-26T21:42:08.243708+0000","last_unstale":"2024-04-26T21:42:11.193218+0000","last_undegraded":"2024-04-26T21:42:11.193218+0000","last_fullsized":"2024-04-26T21:42:11.193218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:37.516096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193200+0000","last_change":"2024-04-26T21:42:09.264131+0000","last_active":"2024-04-26T21:42:11.193200+0000","last_peered":"2024-04-26T21:42:11.193200+0000","last_clean":"2024-04-26T21:42:11.193200+0000","last_became_active":"2024-04-26T21:42:09.263307+0000","last_became_peered":"2024-04-26T21:42:09.263307+0000","last_unstale":"2024-04-26T21:42:11.193200+0000","last_undegraded":"2024-04-26T21:42:11.193200+0000","last_fullsized":"2024-04-26T21:42:11.193200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:14.247992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"93'3","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236699+0000","last_change":"2024-04-26T21:42:08.252314+0000","last_active":"2024-04-26T21:42:11.236699+0000","last_peered":"2024-04-26T21:42:11.236699+0000","last_clean":"2024-04-26T21:42:11.236699+0000","last_became_active":"2024-04-26T21:42:08.252062+0000","last_became_peered":"2024-04-26T21:42:08.252062+0000","last_unstale":"2024-04-26T21:42:11.236699+0000","last_undegraded":"2024-04-26T21:42:11.236699+0000","last_fullsized":"2024-04-26T21:42:11.236699+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:13.833108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029148+0000","last_change":"2024-04-26T21:42:09.251068+0000","last_active":"2024-04-26T21:42:12.029148+0000","last_peered":"2024-04-26T21:42:12.029148+0000","last_clean":"2024-04-26T21:42:12.029148+0000","last_became_active":"2024-04-26T21:42:09.250779+0000","last_became_peered":"2024-04-26T21:42:09.250779+0000","last_unstale":"2024-04-26T21:42:12.029148+0000","last_undegraded":"2024-04-26T21:42:12.029148+0000","last_fullsized":"2024-04-26T21:42:12.029148+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:09.291336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"93'6","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029089+0000","last_change":"2024-04-26T21:42:08.248470+0000","last_active":"2024-04-26T21:42:12.029089+0000","last_peered":"2024-04-26T21:42:12.029089+0000","last_clean":"2024-04-26T21:42:12.029089+0000","last_became_active":"2024-04-26T21:42:08.248129+0000","last_became_peered":"2024-04-26T21:42:08.248129+0000","last_unstale":"2024-04-26T21:42:12.029089+0000","last_undegraded":"2024-04-26T21:42:12.029089+0000","last_fullsized":"2024-04-26T21:42:12.029089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:43.253241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028784+0000","last_change":"2024-04-26T21:42:09.254581+0000","last_active":"2024-04-26T21:42:12.028784+0000","last_peered":"2024-04-26T21:42:12.028784+0000","last_clean":"2024-04-26T21:42:12.028784+0000","last_became_active":"2024-04-26T21:42:09.253789+0000","last_became_peered":"2024-04-26T21:42:09.253789+0000","last_unstale":"2024-04-26T21:42:12.028784+0000","last_undegraded":"2024-04-26T21:42:12.028784+0000","last_fullsized":"2024-04-26T21:42:12.028784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:07.477516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028522+0000","last_change":"2024-04-26T21:42:08.250002+0000","last_active":"2024-04-26T21:42:12.028522+0000","last_peered":"2024-04-26T21:42:12.028522+0000","last_clean":"2024-04-26T21:42:12.028522+0000","last_became_active":"2024-04-26T21:42:08.249730+0000","last_became_peered":"2024-04-26T21:42:08.249730+0000","last_unstale":"2024-04-26T21:42:12.028522+0000","last_undegraded":"2024-04-26T21:42:12.028522+0000","last_fullsized":"2024-04-26T21:42:12.028522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:43.729428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236087+0000","last_change":"2024-04-26T21:42:09.252000+0000","last_active":"2024-04-26T21:42:11.236087+0000","last_peered":"2024-04-26T21:42:11.236087+0000","last_clean":"2024-04-26T21:42:11.236087+0000","last_became_active":"2024-04-26T21:42:09.251721+0000","last_became_peered":"2024-04-26T21:42:09.251721+0000","last_unstale":"2024-04-26T21:42:11.236087+0000","last_undegraded":"2024-04-26T21:42:11.236087+0000","last_fullsized":"2024-04-26T21:42:11.236087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:12.821517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028762+0000","last_change":"2024-04-26T21:42:08.251291+0000","last_active":"2024-04-26T21:42:12.028762+0000","last_peered":"2024-04-26T21:42:12.028762+0000","last_clean":"2024-04-26T21:42:12.028762+0000","last_became_active":"2024-04-26T21:42:08.251027+0000","last_became_peered":"2024-04-26T21:42:08.251027+0000","last_unstale":"2024-04-26T21:42:12.028762+0000","last_undegraded":"2024-04-26T21:42:12.028762+0000","last_fullsized":"2024-04-26T21:42:12.028762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:36.735866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.249889+0000","last_change":"2024-04-26T21:42:09.268758+0000","last_active":"2024-04-26T21:42:10.249889+0000","last_peered":"2024-04-26T21:42:10.249889+0000","last_clean":"2024-04-26T21:42:10.249889+0000","last_became_active":"2024-04-26T21:42:09.268502+0000","last_became_peered":"2024-04-26T21:42:09.268502+0000","last_unstale":"2024-04-26T21:42:10.249889+0000","last_undegraded":"2024-04-26T21:42:10.249889+0000","last_fullsized":"2024-04-26T21:42:10.249889+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:19.961609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029379+0000","last_change":"2024-04-26T21:42:08.243376+0000","last_active":"2024-04-26T21:42:12.029379+0000","last_peered":"2024-04-26T21:42:12.029379+0000","last_clean":"2024-04-26T21:42:12.029379+0000","last_became_active":"2024-04-26T21:42:08.243085+0000","last_became_peered":"2024-04-26T21:42:08.243085+0000","last_unstale":"2024-04-26T21:42:12.029379+0000","last_undegraded":"2024-04-26T21:42:12.029379+0000","last_fullsized":"2024-04-26T21:42:12.029379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:33.127648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.250171+0000","last_change":"2024-04-26T21:42:09.248567+0000","last_active":"2024-04-26T21:42:10.250171+0000","last_peered":"2024-04-26T21:42:10.250171+0000","last_clean":"2024-04-26T21:42:10.250171+0000","last_became_active":"2024-04-26T21:42:09.248292+0000","last_became_peered":"2024-04-26T21:42:09.248292+0000","last_unstale":"2024-04-26T21:42:10.250171+0000","last_undegraded":"2024-04-26T21:42:10.250171+0000","last_fullsized":"2024-04-26T21:42:10.250171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:40.761060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"93'8","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028765+0000","last_change":"2024-04-26T21:42:08.249551+0000","last_active":"2024-04-26T21:42:12.028765+0000","last_peered":"2024-04-26T21:42:12.028765+0000","last_clean":"2024-04-26T21:42:12.028765+0000","last_became_active":"2024-04-26T21:42:08.249251+0000","last_became_peered":"2024-04-26T21:42:08.249251+0000","last_unstale":"2024-04-26T21:42:12.028765+0000","last_undegraded":"2024-04-26T21:42:12.028765+0000","last_fullsized":"2024-04-26T21:42:12.028765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:52.260574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029064+0000","last_change":"2024-04-26T21:42:09.253791+0000","last_active":"2024-04-26T21:42:12.029064+0000","last_peered":"2024-04-26T21:42:12.029064+0000","last_clean":"2024-04-26T21:42:12.029064+0000","last_became_active":"2024-04-26T21:42:09.253465+0000","last_became_peered":"2024-04-26T21:42:09.253465+0000","last_unstale":"2024-04-26T21:42:12.029064+0000","last_undegraded":"2024-04-26T21:42:12.029064+0000","last_fullsized":"2024-04-26T21:42:12.029064+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:33.184002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252276+0000","last_change":"2024-04-26T21:42:08.244764+0000","last_active":"2024-04-26T21:42:10.252276+0000","last_peered":"2024-04-26T21:42:10.252276+0000","last_clean":"2024-04-26T21:42:10.252276+0000","last_became_active":"2024-04-26T21:42:08.244222+0000","last_became_peered":"2024-04-26T21:42:08.244222+0000","last_unstale":"2024-04-26T21:42:10.252276+0000","last_undegraded":"2024-04-26T21:42:10.252276+0000","last_fullsized":"2024-04-26T21:42:10.252276+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:32.555719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252044+0000","last_change":"2024-04-26T21:42:09.249727+0000","last_active":"2024-04-26T21:42:10.252044+0000","last_peered":"2024-04-26T21:42:10.252044+0000","last_clean":"2024-04-26T21:42:10.252044+0000","last_became_active":"2024-04-26T21:42:09.249446+0000","last_became_peered":"2024-04-26T21:42:09.249446+0000","last_unstale":"2024-04-26T21:42:10.252044+0000","last_undegraded":"2024-04-26T21:42:10.252044+0000","last_fullsized":"2024-04-26T21:42:10.252044+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028599+0000","last_change":"2024-04-26T21:42:08.243044+0000","last_active":"2024-04-26T21:42:12.028599+0000","last_peered":"2024-04-26T21:42:12.028599+0000","last_clean":"2024-04-26T21:42:12.028599+0000","last_became_active":"2024-04-26T21:42:08.242751+0000","last_became_peered":"2024-04-26T21:42:08.242751+0000","last_unstale":"2024-04-26T21:42:12.028599+0000","last_undegraded":"2024-04-26T21:42:12.028599+0000","last_fullsized":"2024-04-26T21:42:12.028599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:25.955636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236551+0000","last_change":"2024-04-26T21:42:09.248740+0000","last_active":"2024-04-26T21:42:11.236551+0000","last_peered":"2024-04-26T21:42:11.236551+0000","last_clean":"2024-04-26T21:42:11.236551+0000","last_became_active":"2024-04-26T21:42:09.248474+0000","last_became_peered":"2024-04-26T21:42:09.248474+0000","last_unstale":"2024-04-26T21:42:11.236551+0000","last_undegraded":"2024-04-26T21:42:11.236551+0000","last_fullsized":"2024-04-26T21:42:11.236551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:36.706819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236730+0000","last_change":"2024-04-26T21:42:08.246124+0000","last_active":"2024-04-26T21:42:11.236730+0000","last_peered":"2024-04-26T21:42:11.236730+0000","last_clean":"2024-04-26T21:42:11.236730+0000","last_became_active":"2024-04-26T21:42:08.245839+0000","last_became_peered":"2024-04-26T21:42:08.245839+0000","last_unstale":"2024-04-26T21:42:11.236730+0000","last_undegraded":"2024-04-26T21:42:11.236730+0000","last_fullsized":"2024-04-26T21:42:11.236730+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:56.196138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028495+0000","last_change":"2024-04-26T21:42:09.263280+0000","last_active":"2024-04-26T21:42:12.028495+0000","last_peered":"2024-04-26T21:42:12.028495+0000","last_clean":"2024-04-26T21:42:12.028495+0000","last_became_active":"2024-04-26T21:42:09.260279+0000","last_became_peered":"2024-04-26T21:42:09.260279+0000","last_unstale":"2024-04-26T21:42:12.028495+0000","last_undegraded":"2024-04-26T21:42:12.028495+0000","last_fullsized":"2024-04-26T21:42:12.028495+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:54.394728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251034+0000","last_change":"2024-04-26T21:42:08.244889+0000","last_active":"2024-04-26T21:42:10.251034+0000","last_peered":"2024-04-26T21:42:10.251034+0000","last_clean":"2024-04-26T21:42:10.251034+0000","last_became_active":"2024-04-26T21:42:08.243921+0000","last_became_peered":"2024-04-26T21:42:08.243921+0000","last_unstale":"2024-04-26T21:42:10.251034+0000","last_undegraded":"2024-04-26T21:42:10.251034+0000","last_fullsized":"2024-04-26T21:42:10.251034+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:08.285259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251483+0000","last_change":"2024-04-26T21:42:09.258148+0000","last_active":"2024-04-26T21:42:10.251483+0000","last_peered":"2024-04-26T21:42:10.251483+0000","last_clean":"2024-04-26T21:42:10.251483+0000","last_became_active":"2024-04-26T21:42:09.257879+0000","last_became_peered":"2024-04-26T21:42:09.257879+0000","last_unstale":"2024-04-26T21:42:10.251483+0000","last_undegraded":"2024-04-26T21:42:10.251483+0000","last_fullsized":"2024-04-26T21:42:10.251483+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:44.259318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.249851+0000","last_change":"2024-04-26T21:42:08.251739+0000","last_active":"2024-04-26T21:42:10.249851+0000","last_peered":"2024-04-26T21:42:10.249851+0000","last_clean":"2024-04-26T21:42:10.249851+0000","last_became_active":"2024-04-26T21:42:08.251265+0000","last_became_peered":"2024-04-26T21:42:08.251265+0000","last_unstale":"2024-04-26T21:42:10.249851+0000","last_undegraded":"2024-04-26T21:42:10.249851+0000","last_fullsized":"2024-04-26T21:42:10.249851+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:03.455718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251635+0000","last_change":"2024-04-26T21:42:09.268561+0000","last_active":"2024-04-26T21:42:10.251635+0000","last_peered":"2024-04-26T21:42:10.251635+0000","last_clean":"2024-04-26T21:42:10.251635+0000","last_became_active":"2024-04-26T21:42:09.268285+0000","last_became_peered":"2024-04-26T21:42:09.268285+0000","last_unstale":"2024-04-26T21:42:10.251635+0000","last_undegraded":"2024-04-26T21:42:10.251635+0000","last_fullsized":"2024-04-26T21:42:10.251635+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:37.741942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251537+0000","last_change":"2024-04-26T21:42:08.251899+0000","last_active":"2024-04-26T21:42:10.251537+0000","last_peered":"2024-04-26T21:42:10.251537+0000","last_clean":"2024-04-26T21:42:10.251537+0000","last_became_active":"2024-04-26T21:42:08.251624+0000","last_became_peered":"2024-04-26T21:42:08.251624+0000","last_unstale":"2024-04-26T21:42:10.251537+0000","last_undegraded":"2024-04-26T21:42:10.251537+0000","last_fullsized":"2024-04-26T21:42:10.251537+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:37.868807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028726+0000","last_change":"2024-04-26T21:42:09.253482+0000","last_active":"2024-04-26T21:42:12.028726+0000","last_peered":"2024-04-26T21:42:12.028726+0000","last_clean":"2024-04-26T21:42:12.028726+0000","last_became_active":"2024-04-26T21:42:09.253184+0000","last_became_peered":"2024-04-26T21:42:09.253184+0000","last_unstale":"2024-04-26T21:42:12.028726+0000","last_undegraded":"2024-04-26T21:42:12.028726+0000","last_fullsized":"2024-04-26T21:42:12.028726+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.251086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.249795+0000","last_change":"2024-04-26T21:42:08.249157+0000","last_active":"2024-04-26T21:42:10.249795+0000","last_peered":"2024-04-26T21:42:10.249795+0000","last_clean":"2024-04-26T21:42:10.249795+0000","last_became_active":"2024-04-26T21:42:08.248811+0000","last_became_peered":"2024-04-26T21:42:08.248811+0000","last_unstale":"2024-04-26T21:42:10.249795+0000","last_undegraded":"2024-04-26T21:42:10.249795+0000","last_fullsized":"2024-04-26T21:42:10.249795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:25.798370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251949+0000","last_change":"2024-04-26T21:42:09.251492+0000","last_active":"2024-04-26T21:42:10.251949+0000","last_peered":"2024-04-26T21:42:10.251949+0000","last_clean":"2024-04-26T21:42:10.251949+0000","last_became_active":"2024-04-26T21:42:09.251194+0000","last_became_peered":"2024-04-26T21:42:09.251194+0000","last_unstale":"2024-04-26T21:42:10.251949+0000","last_undegraded":"2024-04-26T21:42:10.251949+0000","last_fullsized":"2024-04-26T21:42:10.251949+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:07.477516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028677+0000","last_change":"2024-04-26T21:42:08.246097+0000","last_active":"2024-04-26T21:42:12.028677+0000","last_peered":"2024-04-26T21:42:12.028677+0000","last_clean":"2024-04-26T21:42:12.028677+0000","last_became_active":"2024-04-26T21:42:08.243459+0000","last_became_peered":"2024-04-26T21:42:08.243459+0000","last_unstale":"2024-04-26T21:42:12.028677+0000","last_undegraded":"2024-04-26T21:42:12.028677+0000","last_fullsized":"2024-04-26T21:42:12.028677+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:17.342450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029580+0000","last_change":"2024-04-26T21:42:09.247416+0000","last_active":"2024-04-26T21:42:12.029580+0000","last_peered":"2024-04-26T21:42:12.029580+0000","last_clean":"2024-04-26T21:42:12.029580+0000","last_became_active":"2024-04-26T21:42:09.247141+0000","last_became_peered":"2024-04-26T21:42:09.247141+0000","last_unstale":"2024-04-26T21:42:12.029580+0000","last_undegraded":"2024-04-26T21:42:12.029580+0000","last_fullsized":"2024-04-26T21:42:12.029580+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:09.762828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"94'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236690+0000","last_change":"2024-04-26T21:42:08.251072+0000","last_active":"2024-04-26T21:42:11.236690+0000","last_peered":"2024-04-26T21:42:11.236690+0000","last_clean":"2024-04-26T21:42:11.236690+0000","last_became_active":"2024-04-26T21:42:08.250873+0000","last_became_peered":"2024-04-26T21:42:08.250873+0000","last_unstale":"2024-04-26T21:42:11.236690+0000","last_undegraded":"2024-04-26T21:42:11.236690+0000","last_fullsized":"2024-04-26T21:42:11.236690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:07.499452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029593+0000","last_change":"2024-04-26T21:42:09.251461+0000","last_active":"2024-04-26T21:42:12.029593+0000","last_peered":"2024-04-26T21:42:12.029593+0000","last_clean":"2024-04-26T21:42:12.029593+0000","last_became_active":"2024-04-26T21:42:09.248712+0000","last_became_peered":"2024-04-26T21:42:09.248712+0000","last_unstale":"2024-04-26T21:42:12.029593+0000","last_undegraded":"2024-04-26T21:42:12.029593+0000","last_fullsized":"2024-04-26T21:42:12.029593+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:45.160614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.236776+0000","last_change":"2024-04-26T21:42:08.252141+0000","last_active":"2024-04-26T21:42:11.236776+0000","last_peered":"2024-04-26T21:42:11.236776+0000","last_clean":"2024-04-26T21:42:11.236776+0000","last_became_active":"2024-04-26T21:42:08.251718+0000","last_became_peered":"2024-04-26T21:42:08.251718+0000","last_unstale":"2024-04-26T21:42:11.236776+0000","last_undegraded":"2024-04-26T21:42:11.236776+0000","last_fullsized":"2024-04-26T21:42:11.236776+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:36.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251965+0000","last_change":"2024-04-26T21:42:09.255389+0000","last_active":"2024-04-26T21:42:10.251965+0000","last_peered":"2024-04-26T21:42:10.251965+0000","last_clean":"2024-04-26T21:42:10.251965+0000","last_became_active":"2024-04-26T21:42:09.254985+0000","last_became_peered":"2024-04-26T21:42:09.254985+0000","last_unstale":"2024-04-26T21:42:10.251965+0000","last_undegraded":"2024-04-26T21:42:10.251965+0000","last_fullsized":"2024-04-26T21:42:10.251965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:37.060054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252272+0000","last_change":"2024-04-26T21:42:08.249382+0000","last_active":"2024-04-26T21:42:10.252272+0000","last_peered":"2024-04-26T21:42:10.252272+0000","last_clean":"2024-04-26T21:42:10.252272+0000","last_became_active":"2024-04-26T21:42:08.248955+0000","last_became_peered":"2024-04-26T21:42:08.248955+0000","last_unstale":"2024-04-26T21:42:10.252272+0000","last_undegraded":"2024-04-26T21:42:10.252272+0000","last_fullsized":"2024-04-26T21:42:10.252272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:02.352623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.249830+0000","last_change":"2024-04-26T21:42:09.253849+0000","last_active":"2024-04-26T21:42:10.249830+0000","last_peered":"2024-04-26T21:42:10.249830+0000","last_clean":"2024-04-26T21:42:10.249830+0000","last_became_active":"2024-04-26T21:42:09.253586+0000","last_became_peered":"2024-04-26T21:42:09.253586+0000","last_unstale":"2024-04-26T21:42:10.249830+0000","last_undegraded":"2024-04-26T21:42:10.249830+0000","last_fullsized":"2024-04-26T21:42:10.249830+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:40.721633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029130+0000","last_change":"2024-04-26T21:42:08.248552+0000","last_active":"2024-04-26T21:42:12.029130+0000","last_peered":"2024-04-26T21:42:12.029130+0000","last_clean":"2024-04-26T21:42:12.029130+0000","last_became_active":"2024-04-26T21:42:08.248245+0000","last_became_peered":"2024-04-26T21:42:08.248245+0000","last_unstale":"2024-04-26T21:42:12.029130+0000","last_undegraded":"2024-04-26T21:42:12.029130+0000","last_fullsized":"2024-04-26T21:42:12.029130+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:13.241915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:11.193063+0000","last_change":"2024-04-26T21:42:09.261713+0000","last_active":"2024-04-26T21:42:11.193063+0000","last_peered":"2024-04-26T21:42:11.193063+0000","last_clean":"2024-04-26T21:42:11.193063+0000","last_became_active":"2024-04-26T21:42:09.260930+0000","last_became_peered":"2024-04-26T21:42:09.260930+0000","last_unstale":"2024-04-26T21:42:11.193063+0000","last_undegraded":"2024-04-26T21:42:11.193063+0000","last_fullsized":"2024-04-26T21:42:11.193063+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:20.199567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251695+0000","last_change":"2024-04-26T21:42:08.245700+0000","last_active":"2024-04-26T21:42:10.251695+0000","last_peered":"2024-04-26T21:42:10.251695+0000","last_clean":"2024-04-26T21:42:10.251695+0000","last_became_active":"2024-04-26T21:42:08.245432+0000","last_became_peered":"2024-04-26T21:42:08.245432+0000","last_unstale":"2024-04-26T21:42:10.251695+0000","last_undegraded":"2024-04-26T21:42:10.251695+0000","last_fullsized":"2024-04-26T21:42:10.251695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:36.416932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.251718+0000","last_change":"2024-04-26T21:42:09.268586+0000","last_active":"2024-04-26T21:42:10.251718+0000","last_peered":"2024-04-26T21:42:10.251718+0000","last_clean":"2024-04-26T21:42:10.251718+0000","last_became_active":"2024-04-26T21:42:09.268274+0000","last_became_peered":"2024-04-26T21:42:09.268274+0000","last_unstale":"2024-04-26T21:42:10.251718+0000","last_undegraded":"2024-04-26T21:42:10.251718+0000","last_fullsized":"2024-04-26T21:42:10.251718+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:48.828972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:10.252100+0000","last_change":"2024-04-26T21:42:08.249281+0000","last_active":"2024-04-26T21:42:10.252100+0000","last_peered":"2024-04-26T21:42:10.252100+0000","last_clean":"2024-04-26T21:42:10.252100+0000","last_became_active":"2024-04-26T21:42:08.249008+0000","last_became_peered":"2024-04-26T21:42:08.249008+0000","last_unstale":"2024-04-26T21:42:10.252100+0000","last_undegraded":"2024-04-26T21:42:10.252100+0000","last_fullsized":"2024-04-26T21:42:10.252100+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:35.700742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029010+0000","last_change":"2024-04-26T21:42:09.261028+0000","last_active":"2024-04-26T21:42:12.029010+0000","last_peered":"2024-04-26T21:42:12.029010+0000","last_clean":"2024-04-26T21:42:12.029010+0000","last_became_active":"2024-04-26T21:42:09.260731+0000","last_became_peered":"2024-04-26T21:42:09.260731+0000","last_unstale":"2024-04-26T21:42:12.029010+0000","last_undegraded":"2024-04-26T21:42:12.029010+0000","last_fullsized":"2024-04-26T21:42:12.029010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:41.024073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028410+0000","last_change":"2024-04-26T21:42:08.246155+0000","last_active":"2024-04-26T21:42:12.028410+0000","last_peered":"2024-04-26T21:42:12.028410+0000","last_clean":"2024-04-26T21:42:12.028410+0000","last_became_active":"2024-04-26T21:42:08.245864+0000","last_became_peered":"2024-04-26T21:42:08.245864+0000","last_unstale":"2024-04-26T21:42:12.028410+0000","last_undegraded":"2024-04-26T21:42:12.028410+0000","last_fullsized":"2024-04-26T21:42:12.028410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:34.149794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028723+0000","last_change":"2024-04-26T21:42:09.258682+0000","last_active":"2024-04-26T21:42:12.028723+0000","last_peered":"2024-04-26T21:42:12.028723+0000","last_clean":"2024-04-26T21:42:12.028723+0000","last_became_active":"2024-04-26T21:42:09.258354+0000","last_became_peered":"2024-04-26T21:42:09.258354+0000","last_unstale":"2024-04-26T21:42:12.028723+0000","last_undegraded":"2024-04-26T21:42:12.028723+0000","last_fullsized":"2024-04-26T21:42:12.028723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:08.222935+0000","last_clean_scrub_stamp":"2024-04-26T21:42:08.222935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:36.119046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.029233+0000","last_change":"2024-04-26T21:42:08.244341+0000","last_active":"2024-04-26T21:42:12.029233+0000","last_peered":"2024-04-26T21:42:12.029233+0000","last_clean":"2024-04-26T21:42:12.029233+0000","last_became_active":"2024-04-26T21:42:08.244016+0000","last_became_peered":"2024-04-26T21:42:08.244016+0000","last_unstale":"2024-04-26T21:42:12.029233+0000","last_undegraded":"2024-04-26T21:42:12.029233+0000","last_fullsized":"2024-04-26T21:42:12.029233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:42:07.216858+0000","last_clean_scrub_stamp":"2024-04-26T21:42:07.216858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:24.990650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":310,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T21:42:12.028567+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:42:12.028567+0000","last_peered":"2024-04-26T21:42:12.028567+0000","last_clean":"2024-04-26T21:42:12.028567+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:42:12.028567+0000","last_undegraded":"2024-04-26T21:42:12.028567+0000","last_fullsized":"2024-04-26T21:42:12.028567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":1130455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":542247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1159168,"data_stored":540031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574937,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":540191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":540031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":540031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1163264,"data_stored":540101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1777664,"data_stored":1132717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:42:18.079 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:42:18.079 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:42:18.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.195+0000 7f4874504640 1 Processor -- start 2024-04-26T21:42:18.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.195+0000 7f4874504640 1 -- start start 2024-04-26T21:42:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4874504640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f486c0cfde0 0x7f486c0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4874504640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f486c070350 0x7f486c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4874504640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c0d2390 con 0x7f486c071090 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4874504640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f486c0afab0 con 0x7f486c070350 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4874504640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f486c06dd20 con 0x7f486c0cfde0 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4871a9f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f486c070350 0x7f486c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4871a9f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f486c070350 0x7f486c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37758/0 (socket says 172.21.15.40:37758) 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4871a9f640 1 -- 172.21.15.40:0/2465111177 learned_addr learned my addr 172.21.15.40:0/2465111177 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.196+0000 7f4871a9f640 1 -- 172.21.15.40:0/2465111177 >> v2:172.21.15.184:3300/0 conn(0x7f486c0cfde0 msgr2=0x7f486c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f48722a0640 1 --2- 172.21.15.40:0/2465111177 >> v2:172.21.15.184:3300/0 conn(0x7f486c0cfde0 0x7f486c0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f4871a9f640 1 --2- 172.21.15.40:0/2465111177 >> v2:172.21.15.184:3300/0 conn(0x7f486c0cfde0 0x7f486c0d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f4871a9f640 1 -- 172.21.15.40:0/2465111177 >> v1:172.21.15.184:6789/0 conn(0x7f486c071090 legacy=0x7f486c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f4871a9f640 1 -- 172.21.15.40:0/2465111177 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c06d3f0 con 0x7f486c070350 2024-04-26T21:42:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f48722a0640 1 --2- 172.21.15.40:0/2465111177 >> v2:172.21.15.184:3300/0 conn(0x7f486c0cfde0 0x7f486c0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f4871a9f640 1 --2- 172.21.15.40:0/2465111177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c070350 0x7f486c070b50 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f486c0762b0 tx=0x7f486002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1289d72464f032b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f487129e640 1 -- 172.21.15.40:0/2465111177 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860037070 con 0x7f486c070350 2024-04-26T21:42:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f487129e640 1 -- 172.21.15.40:0/2465111177 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4860032440 con 0x7f486c070350 2024-04-26T21:42:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.197+0000 7f487129e640 1 -- 172.21.15.40:0/2465111177 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860032810 con 0x7f486c070350 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.198+0000 7f4874504640 1 -- 172.21.15.40:0/2465111177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c070350 msgr2=0x7f486c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.198+0000 7f4874504640 1 --2- 172.21.15.40:0/2465111177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c070350 0x7f486c070b50 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f486c0762b0 tx=0x7f486002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.198+0000 7f4874504640 1 -- 172.21.15.40:0/2465111177 shutdown_connections 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.198+0000 7f4874504640 1 --2- 172.21.15.40:0/2465111177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c070350 0x7f486c070b50 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.198+0000 7f4874504640 1 --2- 172.21.15.40:0/2465111177 >> v2:172.21.15.184:3300/0 conn(0x7f486c0cfde0 0x7f486c0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.198+0000 7f4874504640 1 -- 172.21.15.40:0/2465111177 >> 172.21.15.40:0/2465111177 conn(0x7f486c0d2d80 msgr2=0x7f486c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.198+0000 7f4874504640 1 -- 172.21.15.40:0/2465111177 shutdown_connections 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.198+0000 7f4874504640 1 -- 172.21.15.40:0/2465111177 wait complete. 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.199+0000 7f4874504640 1 Processor -- start 2024-04-26T21:42:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.199+0000 7f4874504640 1 -- start start 2024-04-26T21:42:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4874504640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f486c070350 0x7f486c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4874504640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c071090 0x7f486c09fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4874504640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f486c0cfde0 0x7f486c0b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4874504640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06d9b0 con 0x7f486c071090 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4874504640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0714c0 con 0x7f486c070350 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4874504640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f486c0d20a0 con 0x7f486c0cfde0 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4871a9f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c071090 0x7f486c09fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4871a9f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c071090 0x7f486c09fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37760/0 (socket says 172.21.15.40:37760) 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4871a9f640 1 -- 172.21.15.40:0/3505663845 learned_addr learned my addr 172.21.15.40:0/3505663845 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f48722a0640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f486c070350 0x7f486c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4871a9f640 1 -- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f486c0cfde0 msgr2=0x7f486c0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4871a9f640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f486c0cfde0 0x7f486c0b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4871a9f640 1 -- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f486c070350 msgr2=0x7f486c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4871a9f640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f486c070350 0x7f486c1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f4871a9f640 1 -- 172.21.15.40:0/3505663845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c156720 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.200+0000 7f48722a0640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f486c070350 0x7f486c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.201+0000 7f4871a9f640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c071090 0x7f486c09fb80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4860000c00 tx=0x7f48600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.201+0000 7f486affd640 1 -- 172.21.15.40:0/3505663845 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860041020 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.201+0000 7f486affd640 1 -- 172.21.15.40:0/3505663845 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4860030970 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.201+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c0a0560 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.201+0000 7f486affd640 1 -- 172.21.15.40:0/3505663845 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860037040 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.201+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c09ce10 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.202+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c09cc20 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.202+0000 7f486affd640 1 -- 172.21.15.40:0/3505663845 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4860044020 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.203+0000 7f486affd640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f485806c8f0 0x7f485806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.203+0000 7f486affd640 1 -- 172.21.15.40:0/3505663845 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f48600ac710 con 0x7f486c071090 2024-04-26T21:42:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.203+0000 7f48722a0640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f485806c8f0 0x7f485806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.203+0000 7f48722a0640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f485806c8f0 0x7f485806edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4864006fd0 tx=0x7f4864008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.207+0000 7f486affd640 1 -- 172.21.15.40:0/3505663845 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f486004b030 con 0x7f486c071090 2024-04-26T21:42:18.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.467+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f486c0694f0 con 0x7f486c071090 2024-04-26T21:42:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.472+0000 7f486affd640 1 -- 172.21.15.40:0/3505663845 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f486007a9c0 con 0x7f486c071090 2024-04-26T21:42:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:42:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw== 2024-04-26T21:42:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:42:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:42:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:42:18.473 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:42:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f485806c8f0 msgr2=0x7f485806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f485806c8f0 0x7f485806edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4864006fd0 tx=0x7f4864008040 comp rx=0 tx=0).stop 2024-04-26T21:42:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c071090 msgr2=0x7f486c09fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c071090 0x7f486c09fb80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4860000c00 tx=0x7f48600046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 shutdown_connections 2024-04-26T21:42:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f485806c8f0 0x7f485806edb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f486c0cfde0 0x7f486c0b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f486c071090 0x7f486c09fb80 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 --2- 172.21.15.40:0/3505663845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f486c070350 0x7f486c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 >> 172.21.15.40:0/3505663845 conn(0x7f486c0d2d80 msgr2=0x7f486c0afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:18.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 shutdown_connections 2024-04-26T21:42:18.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.474+0000 7f4874504640 1 -- 172.21.15.40:0/3505663845 wait complete. 2024-04-26T21:42:18.486 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:18.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.586+0000 7f096f825640 1 Processor -- start 2024-04-26T21:42:18.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.586+0000 7f096f825640 1 -- start start 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096f825640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09680cc5b0 0x7f0968072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096f825640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968072780 0x7f0968166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096f825640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096806e240 con 0x7f0968071e60 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096f825640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f09680701f0 con 0x7f09680cc5b0 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096f825640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f096806fc30 con 0x7f0968072780 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096cdc0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09680cc5b0 0x7f0968072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096cdc0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09680cc5b0 0x7f0968072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37768/0 (socket says 172.21.15.40:37768) 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096cdc0640 1 -- 172.21.15.40:0/3870242144 learned_addr learned my addr 172.21.15.40:0/3870242144 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:18.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096ddc2640 1 --2- 172.21.15.40:0/3870242144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968072780 0x7f0968166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096cdc0640 1 -- 172.21.15.40:0/3870242144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968072780 msgr2=0x7f0968166b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:18.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096cdc0640 1 --2- 172.21.15.40:0/3870242144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968072780 0x7f0968166b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096cdc0640 1 -- 172.21.15.40:0/3870242144 >> v1:172.21.15.184:6789/0 conn(0x7f0968071e60 legacy=0x7f096815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096cdc0640 1 -- 172.21.15.40:0/3870242144 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096806ff10 con 0x7f09680cc5b0 2024-04-26T21:42:18.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.587+0000 7f096ddc2640 1 --2- 172.21.15.40:0/3870242144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968072780 0x7f0968166b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:18.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.588+0000 7f096cdc0640 1 --2- 172.21.15.40:0/3870242144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09680cc5b0 0x7f0968072240 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f095c009870 tx=0x7f095c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ee1d0a9db31454 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:18.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.588+0000 7f0967fff640 1 -- 172.21.15.40:0/3870242144 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c037070 con 0x7f09680cc5b0 2024-04-26T21:42:18.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.588+0000 7f0967fff640 1 -- 172.21.15.40:0/3870242144 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c032440 con 0x7f09680cc5b0 2024-04-26T21:42:18.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.588+0000 7f0967fff640 1 -- 172.21.15.40:0/3870242144 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c032810 con 0x7f09680cc5b0 2024-04-26T21:42:18.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.588+0000 7f096f825640 1 -- 172.21.15.40:0/3870242144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09680cc5b0 msgr2=0x7f0968072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.588+0000 7f096f825640 1 --2- 172.21.15.40:0/3870242144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09680cc5b0 0x7f0968072240 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f095c009870 tx=0x7f095c02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.589+0000 7f096f825640 1 -- 172.21.15.40:0/3870242144 shutdown_connections 2024-04-26T21:42:18.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.589+0000 7f096f825640 1 --2- 172.21.15.40:0/3870242144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968072780 0x7f0968166b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.589+0000 7f096f825640 1 --2- 172.21.15.40:0/3870242144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09680cc5b0 0x7f0968072240 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.589+0000 7f096f825640 1 -- 172.21.15.40:0/3870242144 >> 172.21.15.40:0/3870242144 conn(0x7f09680d0be0 msgr2=0x7f09680d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:18.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.589+0000 7f096f825640 1 -- 172.21.15.40:0/3870242144 shutdown_connections 2024-04-26T21:42:18.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.589+0000 7f096f825640 1 -- 172.21.15.40:0/3870242144 wait complete. 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.590+0000 7f096f825640 1 Processor -- start 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.590+0000 7f096f825640 1 -- start start 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096f825640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968071e60 0x7f09680b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096f825640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0968072780 0x7f09680c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096f825640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09680cc5b0 0x7f09681f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096f825640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f096806f8e0 con 0x7f0968072780 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096f825640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0968072c10 con 0x7f09680cc5b0 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096f825640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f096806ddf0 con 0x7f0968071e60 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0968072780 0x7f09680c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0968072780 0x7f09680c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37784/0 (socket says 172.21.15.40:37784) 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 -- 172.21.15.40:0/4081676932 learned_addr learned my addr 172.21.15.40:0/4081676932 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096d5c1640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968071e60 0x7f09680b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096ddc2640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09680cc5b0 0x7f09681f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 -- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968071e60 msgr2=0x7f09680b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968071e60 0x7f09680b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 -- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09680cc5b0 msgr2=0x7f09681f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09680cc5b0 0x7f09681f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 -- 172.21.15.40:0/4081676932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09680b55a0 con 0x7f0968072780 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096cdc0640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0968072780 0x7f09680c7070 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f095c02a200 tx=0x7f095c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f0965ffb640 1 -- 172.21.15.40:0/4081676932 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c041020 con 0x7f0968072780 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f0965ffb640 1 -- 172.21.15.40:0/4081676932 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c0306e0 con 0x7f0968072780 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681f3320 con 0x7f0968072780 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.591+0000 7f0965ffb640 1 -- 172.21.15.40:0/4081676932 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c037040 con 0x7f0968072780 2024-04-26T21:42:18.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.592+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09680b5840 con 0x7f0968072780 2024-04-26T21:42:18.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.593+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09680694f0 con 0x7f0968072780 2024-04-26T21:42:18.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.593+0000 7f0965ffb640 1 -- 172.21.15.40:0/4081676932 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f095c044020 con 0x7f0968072780 2024-04-26T21:42:18.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.593+0000 7f0965ffb640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f095406c920 0x7f095406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:18.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.594+0000 7f0965ffb640 1 -- 172.21.15.40:0/4081676932 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f095c0ac370 con 0x7f0968072780 2024-04-26T21:42:18.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.594+0000 7f096d5c1640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f095406c920 0x7f095406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:18.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.594+0000 7f096d5c1640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f095406c920 0x7f095406ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f09600016e0 tx=0x7f0960009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:18.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.598+0000 7f0965ffb640 1 -- 172.21.15.40:0/4081676932 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f095c04b030 con 0x7f0968072780 2024-04-26T21:42:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.823+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0968158620 con 0x7f0968072780 2024-04-26T21:42:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.823+0000 7f0965ffb640 1 -- 172.21.15.40:0/4081676932 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5189 (secure 0 0 0) 0x7f095c07a050 con 0x7f0968072780 2024-04-26T21:42:18.824 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:18.824 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:11.621437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7770":{"gid":7770,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2715490806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2715490806},{"type":"v1","addr":"172.21.15.40:6835","nonce":2715490806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7770,"qdb_cluster":[7770]},"id":6}]} 2024-04-26T21:42:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 87 2024-04-26T21:42:18.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f095406c920 msgr2=0x7f095406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f095406c920 0x7f095406ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f09600016e0 tx=0x7f0960009040 comp rx=0 tx=0).stop 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0968072780 msgr2=0x7f09680c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0968072780 0x7f09680c7070 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f095c02a200 tx=0x7f095c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 shutdown_connections 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f095406c920 0x7f095406ede0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09680cc5b0 0x7f09681f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0968072780 0x7f09680c7070 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 --2- 172.21.15.40:0/4081676932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0968071e60 0x7f09680b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 >> 172.21.15.40:0/4081676932 conn(0x7f09680d0be0 msgr2=0x7f096815bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 shutdown_connections 2024-04-26T21:42:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:18.825+0000 7f096f825640 1 -- 172.21.15.40:0/4081676932 wait complete. 2024-04-26T21:42:19.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:42:11.621437+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7770}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7770': {'gid': 7770, 'name': 'a', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/2715490806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 2715490806}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 2715490806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7770, 'qdb_cluster': [7770]} 2024-04-26T21:42:19.240 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:42:19.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:42:19.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.348+0000 7f703b423640 1 Processor -- start 2024-04-26T21:42:19.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.348+0000 7f703b423640 1 -- start start 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.349+0000 7f703b423640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f70341608f0 0x7f70340d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.349+0000 7f703b423640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70340cde20 con 0x7f70340cffa0 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.349+0000 7f703b423640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70340ce100 con 0x7f70340d0a90 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.349+0000 7f703b423640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f70340cea30 con 0x7f70341608f0 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.349+0000 7f70399c0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f70340cffa0 0x7f70340b2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37124/0 (socket says 172.21.15.40:37124) 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.349+0000 7f70399c0640 1 -- 172.21.15.40:0/915471822 learned_addr learned my addr 172.21.15.40:0/915471822 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f70391bf640 1 --2- 172.21.15.40:0/915471822 >> v2:172.21.15.184:3300/0 conn(0x7f70341608f0 0x7f70340d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1416923437 0 0) 0x7f70340cde20 con 0x7f70340cffa0 2024-04-26T21:42:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f701c003520 con 0x7f70340cffa0 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2798048671 0 0) 0x7f701c003520 con 0x7f70340cffa0 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 >> v2:172.21.15.184:3300/0 conn(0x7f70341608f0 msgr2=0x7f70340d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 --2- 172.21.15.40:0/915471822 >> v2:172.21.15.184:3300/0 conn(0x7f70341608f0 0x7f70340d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 >> v1:172.21.15.184:6789/0 conn(0x7f70340d0a90 legacy=0x7f70340cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70340ce3e0 con 0x7f70340cffa0 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f70240019f0 con 0x7f70340cffa0 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 >> v1:172.21.15.40:6789/0 conn(0x7f70340cffa0 legacy=0x7f70340b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0039b0 0x7f701c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f701c004800 0x7f701c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f701c0051c0 0x7f701c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70340cea30 con 0x7f701c0051c0 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f701c00c060 con 0x7f701c0039b0 2024-04-26T21:42:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f701c00c260 con 0x7f701c004800 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f70399c0640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f701c0051c0 0x7f701c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f70389be640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f701c004800 0x7f701c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.350+0000 7f70391bf640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0039b0 0x7f701c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f70399c0640 1 -- 172.21.15.40:0/915471822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f701c004800 msgr2=0x7f701c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f70399c0640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f701c004800 0x7f701c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f70399c0640 1 -- 172.21.15.40:0/915471822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0039b0 msgr2=0x7f701c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f70399c0640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0039b0 0x7f701c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f70399c0640 1 -- 172.21.15.40:0/915471822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c003520 con 0x7f701c0051c0 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f70399c0640 1 -- 172.21.15.40:0/915471822 reap_dead start 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f70399c0640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f701c0051c0 0x7f701c00b9c0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f703406fdc0 tx=0x7f701c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809e7272c25ffc12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702400a4a0 con 0x7f701c0051c0 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702402f070 con 0x7f701c0051c0 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.351+0000 7f7033fff640 1 -- 172.21.15.40:0/915471822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702402bcb0 con 0x7f701c0051c0 2024-04-26T21:42:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.352+0000 7f703b423640 1 -- 172.21.15.40:0/915471822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f701c0051c0 msgr2=0x7f701c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.352+0000 7f703b423640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f701c0051c0 0x7f701c00b9c0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f703406fdc0 tx=0x7f701c005830 comp rx=0 tx=0).stop 2024-04-26T21:42:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.352+0000 7f703b423640 1 -- 172.21.15.40:0/915471822 shutdown_connections 2024-04-26T21:42:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.352+0000 7f703b423640 1 --2- 172.21.15.40:0/915471822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f701c0051c0 0x7f701c00b9c0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.352+0000 7f703b423640 1 -- 172.21.15.40:0/915471822 >> 172.21.15.40:0/915471822 conn(0x7f70340d2ec0 msgr2=0x7f70340d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.352+0000 7f703b423640 1 -- 172.21.15.40:0/915471822 shutdown_connections 2024-04-26T21:42:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.352+0000 7f703b423640 1 -- 172.21.15.40:0/915471822 wait complete. 2024-04-26T21:42:19.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.353+0000 7f703b423640 1 Processor -- start 2024-04-26T21:42:19.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.353+0000 7f703b423640 1 -- start start 2024-04-26T21:42:19.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f703b423640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0051c0 0x7f70340ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f703b423640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f703406e190 0x7f7034159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f703b423640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70340b70e0 0x7f70340b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f703b423640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70340ceec0 con 0x7f703406e190 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f703b423640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70340d0ef0 con 0x7f701c0051c0 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f703b423640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70340cda40 con 0x7f70340b70e0 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70389be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f703406e190 0x7f7034159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70389be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f703406e190 0x7f7034159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57140/0 (socket says 172.21.15.40:57140) 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70389be640 1 -- 172.21.15.40:0/2624947033 learned_addr learned my addr 172.21.15.40:0/2624947033 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70391bf640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0051c0 0x7f70340ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70399c0640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70340b70e0 0x7f70340b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70389be640 1 -- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70340b70e0 msgr2=0x7f70340b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70389be640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70340b70e0 0x7f70340b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70389be640 1 -- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0051c0 msgr2=0x7f70340ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70389be640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0051c0 0x7f70340ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.354+0000 7f70389be640 1 -- 172.21.15.40:0/2624947033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703406e970 con 0x7f703406e190 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.355+0000 7f70389be640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f703406e190 0x7f7034159550 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f702800f650 tx=0x7f702800fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.355+0000 7f7031ffb640 1 -- 172.21.15.40:0/2624947033 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280050d0 con 0x7f703406e190 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.355+0000 7f7031ffb640 1 -- 172.21.15.40:0/2624947033 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7028018070 con 0x7f703406e190 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.355+0000 7f703b423640 1 -- 172.21.15.40:0/2624947033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70340c7d90 con 0x7f703406e190 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.355+0000 7f7031ffb640 1 -- 172.21.15.40:0/2624947033 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028005490 con 0x7f703406e190 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.355+0000 7f703b423640 1 -- 172.21.15.40:0/2624947033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70340c8030 con 0x7f703406e190 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.357+0000 7f7031ffb640 1 -- 172.21.15.40:0/2624947033 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f702800d730 con 0x7f703406e190 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.357+0000 7f7031ffb640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f702006c920 0x7f702006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.357+0000 7f7031ffb640 1 -- 172.21.15.40:0/2624947033 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f702808b9a0 con 0x7f703406e190 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.357+0000 7f70391bf640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f702006c920 0x7f702006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.358+0000 7f70391bf640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f702006c920 0x7f702006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f702c009a80 tx=0x7f702c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.358+0000 7f703e21e740 1 Processor -- start 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.358+0000 7f703e21e740 1 -- start start 2024-04-26T21:42:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f703e21e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da60440 0x55d42da60820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f703e21e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f703e21e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da36120 0x55d42da7c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f703e21e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d42d99a740 con 0x55d42da7dce0 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f703e21e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d42d9363a0 con 0x55d42da36120 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f703e21e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d42d996700 con 0x55d42da60440 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57144/0 (socket says 172.21.15.40:57144) 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 -- 172.21.15.40:0/1304175426 learned_addr learned my addr 172.21.15.40:0/1304175426 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70399c0640 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da36120 0x55d42da7c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70391bf640 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da60440 0x55d42da60820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 -- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da60440 msgr2=0x55d42da60820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da60440 0x55d42da60820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 -- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da36120 msgr2=0x55d42da7c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da36120 0x55d42da7c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 -- 172.21.15.40:0/1304175426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42d952750 con 0x55d42da7dce0 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70399c0640 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da36120 0x55d42da7c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f70389be640 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7e230 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7028009010 tx=0x7f7028059cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2619244ee0d95491 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.359+0000 7f701affd640 1 -- 172.21.15.40:0/1304175426 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028071020 con 0x55d42da7dce0 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f701affd640 1 -- 172.21.15.40:0/1304175426 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702805d980 con 0x55d42da7dce0 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f701affd640 1 -- 172.21.15.40:0/1304175426 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280621b0 con 0x55d42da7dce0 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 -- 172.21.15.40:0/1304175426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 msgr2=0x55d42da7e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7e230 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7028009010 tx=0x7f7028059cd0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 -- 172.21.15.40:0/1304175426 shutdown_connections 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da36120 0x55d42da7c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7e230 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 --2- 172.21.15.40:0/1304175426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da60440 0x55d42da60820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 -- 172.21.15.40:0/1304175426 >> 172.21.15.40:0/1304175426 conn(0x55d42da32a50 msgr2=0x55d42da7b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 -- 172.21.15.40:0/1304175426 shutdown_connections 2024-04-26T21:42:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.360+0000 7f703e21e740 1 -- 172.21.15.40:0/1304175426 wait complete. 2024-04-26T21:42:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.361+0000 7f703e21e740 1 -- start start 2024-04-26T21:42:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.361+0000 7f703e21e740 1 Processor -- start 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f703e21e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42da8bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f703e21e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da7dce0 0x55d42da570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f703e21e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da57630 0x55d42da5d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f703e21e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d42da36ec0 con 0x55d42da57630 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f703e21e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d42d921fe0 con 0x55d42da60440 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f703e21e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d42da30980 con 0x55d42da7dce0 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f703e21e740 10 client.0 _ll_register_callbacks cb 0x55d42d938150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da57630 0x55d42da5d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da57630 0x55d42da5d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57152/0 (socket says 172.21.15.40:57152) 2024-04-26T21:42:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 -- 172.21.15.40:0/2970185076 learned_addr learned my addr 172.21.15.40:0/2970185076 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70391bf640 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42da8bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70389be640 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da7dce0 0x55d42da570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 -- 172.21.15.40:0/2970185076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da7dce0 msgr2=0x55d42da570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da7dce0 0x55d42da570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 -- 172.21.15.40:0/2970185076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 msgr2=0x55d42da8bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42da8bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 -- 172.21.15.40:0/2970185076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42d6ac2d0 con 0x55d42da57630 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f70399c0640 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da57630 0x55d42da5d990 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f702402b500 tx=0x7f702402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2f747b6e54435b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f7013fff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f7013fff640 1 -- 172.21.15.40:0/2970185076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024046060 con 0x55d42da57630 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.362+0000 7f7013fff640 1 -- 172.21.15.40:0/2970185076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f702402b850 con 0x55d42da57630 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 1 -- 172.21.15.40:0/2970185076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702805d980 con 0x55d42da57630 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f7013fff640 1 -- 172.21.15.40:0/2970185076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f702404c050 con 0x55d42da57630 2024-04-26T21:42:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 10 client.7911 fetch_fsmap learned FSMap version 87 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 1 -- 172.21.15.40:0/2970185076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55d42d7766b0 con 0x55d42da57630 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f7013fff640 1 -- 172.21.15.40:0/2970185076 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702805d980 con 0x55d42da57630 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f7013fff640 1 -- 172.21.15.40:0/2970185076 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7f702404f020 con 0x55d42da57630 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 10 client.7911 fetch_fsmap finished waiting for FSMap version 87 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 10 client.7911 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 20 client.7911 populate_metadata read hostname 'smithi040' 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] conn(0x55d42da63470 0x55d42da61760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 4 client.7911 mds_command: new command op to 7770 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f703e21e740 1 -- 172.21.15.40:0/2970185076 --> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d42d9a8350 con 0x55d42da63470 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f70391bf640 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] conn(0x55d42da63470 0x55d42da61760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f7013fff640 10 client.7911 ms_handle_connect on v2:172.21.15.40:6834/2715490806 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.363+0000 7f70391bf640 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] conn(0x55d42da63470 0x55d42da61760 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e53f348f9f8c84 server_cookie=6860aaf8a37048a4 in_seq=0 out_seq=0 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f7013fff640 1 -- 172.21.15.40:0/2970185076 <== mds.0 v2:172.21.15.40:6834/2715490806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d42d9a8350 con 0x55d42da63470 2024-04-26T21:42:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f7013fff640 10 client.7911 handle_command_reply: tid=0 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 client.7911 shutdown 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 -- 172.21.15.40:0/2970185076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da57630 msgr2=0x55d42da5d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da57630 0x55d42da5d990 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f702402b500 tx=0x7f702402d070 comp rx=0 tx=0).stop 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 -- 172.21.15.40:0/2970185076 shutdown_connections 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] conn(0x55d42da63470 0x55d42da61760 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da57630 0x55d42da5d990 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da7dce0 0x55d42da570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f7013fff640 0 client.7911 ms_handle_reset on v2:172.21.15.40:6834/2715490806 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 --2- 172.21.15.40:0/2970185076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42da8bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 -- 172.21.15.40:0/2970185076 >> 172.21.15.40:0/2970185076 conn(0x55d42da36120 msgr2=0x55d42da7f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 -- 172.21.15.40:0/2970185076 shutdown_connections 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.364+0000 7f703e21e740 1 -- 172.21.15.40:0/2970185076 wait complete. 2024-04-26T21:42:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.365+0000 7f703e21e740 20 client.7911 trim_cache size 0 max 16384 2024-04-26T21:42:19.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.383+0000 7f703e21e740 1 Processor -- start 2024-04-26T21:42:19.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.383+0000 7f703e21e740 1 -- start start 2024-04-26T21:42:19.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f703e21e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 0x55d42da62920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f703e21e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da60440 0x55d42da56cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f703e21e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da7dce0 0x55d42da8e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f703e21e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70342014f0 con 0x55d42da60440 2024-04-26T21:42:19.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f703e21e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70340b6d80 con 0x55d42da7dce0 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f703e21e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7034200ed0 con 0x55d42da57630 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da60440 0x55d42da56cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da60440 0x55d42da56cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57154/0 (socket says 172.21.15.40:57154) 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 -- 172.21.15.40:0/87235085 learned_addr learned my addr 172.21.15.40:0/87235085 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70399c0640 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da7dce0 0x55d42da8e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70391bf640 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 0x55d42da62920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 -- 172.21.15.40:0/87235085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 msgr2=0x55d42da62920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 0x55d42da62920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 -- 172.21.15.40:0/87235085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da7dce0 msgr2=0x55d42da8e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da7dce0 0x55d42da8e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 -- 172.21.15.40:0/87235085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42da58e20 con 0x55d42da60440 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f70389be640 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da60440 0x55d42da56cd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7028089da0 tx=0x7f7028014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec75c74a1f0a3e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f701a7fc640 1 -- 172.21.15.40:0/87235085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028077070 con 0x55d42da60440 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f701a7fc640 1 -- 172.21.15.40:0/87235085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7028075070 con 0x55d42da60440 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.384+0000 7f701a7fc640 1 -- 172.21.15.40:0/87235085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028071040 con 0x55d42da60440 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 -- 172.21.15.40:0/87235085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da60440 msgr2=0x55d42da56cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da60440 0x55d42da56cd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7028089da0 tx=0x7f7028014000 comp rx=0 tx=0).stop 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 -- 172.21.15.40:0/87235085 shutdown_connections 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da7dce0 0x55d42da8e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da60440 0x55d42da56cd0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 --2- 172.21.15.40:0/87235085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 0x55d42da62920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 -- 172.21.15.40:0/87235085 >> 172.21.15.40:0/87235085 conn(0x55d42da36120 msgr2=0x55d42da31500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 -- 172.21.15.40:0/87235085 shutdown_connections 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 -- 172.21.15.40:0/87235085 wait complete. 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 -- start start 2024-04-26T21:42:19.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.385+0000 7f703e21e740 1 Processor -- start 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f703e21e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 0x55d42da71760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f703e21e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42dad9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f703e21e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f703e21e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d42da632f0 con 0x55d42da7dce0 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f703e21e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d42da57db0 con 0x55d42da60440 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f703e21e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d42da63940 con 0x55d42da57630 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f703e21e740 10 client.0 _ll_register_callbacks cb 0x7f701c004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f70399c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f70399c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57168/0 (socket says 172.21.15.40:57168) 2024-04-26T21:42:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.386+0000 7f70399c0640 1 -- 172.21.15.40:0/3487120270 learned_addr learned my addr 172.21.15.40:0/3487120270 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70389be640 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42dad9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70391bf640 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 0x55d42da71760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70399c0640 1 -- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 msgr2=0x55d42da71760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70399c0640 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 0x55d42da71760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70399c0640 1 -- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 msgr2=0x55d42dad9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70399c0640 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42dad9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70399c0640 1 -- 172.21.15.40:0/3487120270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42dada8e0 con 0x55d42da7dce0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70389be640 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42dad9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f70399c0640 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7cfe0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f702402bf90 tx=0x7f7024038040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7766324e4f4a78e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f7013fff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f7013fff640 1 -- 172.21.15.40:0/3487120270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702402b590 con 0x55d42da7dce0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f7013fff640 1 -- 172.21.15.40:0/3487120270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f7024007110 con 0x55d42da7dce0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f703e21e740 1 -- 172.21.15.40:0/3487120270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7028075070 con 0x55d42da7dce0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f7013fff640 1 -- 172.21.15.40:0/3487120270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f702405a050 con 0x55d42da7dce0 2024-04-26T21:42:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f703e21e740 10 client.7917 fetch_fsmap learned FSMap version 87 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f703e21e740 1 -- 172.21.15.40:0/3487120270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55d42daf5740 con 0x55d42da7dce0 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f7013fff640 1 -- 172.21.15.40:0/3487120270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7028075070 con 0x55d42da7dce0 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.387+0000 7f7013fff640 1 -- 172.21.15.40:0/3487120270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7f702405d020 con 0x55d42da7dce0 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 10 client.7917 fetch_fsmap finished waiting for FSMap version 87 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 10 client.7917 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 20 client.7917 populate_metadata read hostname 'smithi040' 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] conn(0x55d42daf51e0 0x55d42da2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 4 client.7917 mds_command: new command op to 7770 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 -- 172.21.15.40:0/3487120270 --> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] -- command(tid 0: {"prefix": "status"}) -- 0x55d42d9a21c0 con 0x55d42daf51e0 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f70391bf640 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] conn(0x55d42daf51e0 0x55d42da2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f7013fff640 10 client.7917 ms_handle_connect on v2:172.21.15.40:6834/2715490806 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f70391bf640 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] conn(0x55d42daf51e0 0x55d42da2f670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b81ccd13f56820e server_cookie=551cda245df919f4 in_seq=0 out_seq=0 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f7013fff640 1 -- 172.21.15.40:0/3487120270 <== mds.0 v2:172.21.15.40:6834/2715490806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d42d9a21c0 con 0x55d42daf51e0 2024-04-26T21:42:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f7013fff640 10 client.7917 handle_command_reply: tid=0 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 client.7917 shutdown 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 -- 172.21.15.40:0/3487120270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 msgr2=0x55d42da7cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7cfe0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f702402bf90 tx=0x7f7024038040 comp rx=0 tx=0).stop 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 -- 172.21.15.40:0/3487120270 shutdown_connections 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.40:6834/2715490806,v1:172.21.15.40:6835/2715490806] conn(0x55d42daf51e0 0x55d42da2f670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d42da7dce0 0x55d42da7cfe0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d42da60440 0x55d42dad9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f7013fff640 0 client.7917 ms_handle_reset on v2:172.21.15.40:6834/2715490806 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 --2- 172.21.15.40:0/3487120270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d42da57630 0x55d42da71760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 -- 172.21.15.40:0/3487120270 >> 172.21.15.40:0/3487120270 conn(0x55d42da36120 msgr2=0x55d42da56cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 -- 172.21.15.40:0/3487120270 shutdown_connections 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.388+0000 7f703e21e740 1 -- 172.21.15.40:0/3487120270 wait complete. 2024-04-26T21:42:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703e21e740 20 client.7917 trim_cache size 0 max 16384 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7770, 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.159114450000001, 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 87, 2024-04-26T21:42:19.396 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 94, 2024-04-26T21:42:19.397 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 94, 2024-04-26T21:42:19.397 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.210395459000001 2024-04-26T21:42:19.397 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:42:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 -- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f702006c920 msgr2=0x7f702006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f702006c920 0x7f702006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f702c009a80 tx=0x7f702c008040 comp rx=0 tx=0).stop 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 -- 172.21.15.40:0/2624947033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f703406e190 msgr2=0x7f7034159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f703406e190 0x7f7034159550 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f702800f650 tx=0x7f702800fb10 comp rx=0 tx=0).stop 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 -- 172.21.15.40:0/2624947033 shutdown_connections 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f702006c920 0x7f702006ede0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70340b70e0 0x7f70340b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f703406e190 0x7f7034159550 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 --2- 172.21.15.40:0/2624947033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f701c0051c0 0x7f70340ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 -- 172.21.15.40:0/2624947033 >> 172.21.15.40:0/2624947033 conn(0x7f70340d2ec0 msgr2=0x7f70340d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 -- 172.21.15.40:0/2624947033 shutdown_connections 2024-04-26T21:42:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.389+0000 7f703b423640 1 -- 172.21.15.40:0/2624947033 wait complete. 2024-04-26T21:42:19.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:42:19.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.500+0000 7fda2b8d1640 1 Processor -- start 2024-04-26T21:42:19.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.500+0000 7fda2b8d1640 1 -- start start 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda2b8d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24162880 0x7fda240d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda2b8d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda240b7e50 con 0x7fda240b5f20 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda2b8d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda240d1800 con 0x7fda240d3bc0 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda2b8d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda240d0580 con 0x7fda24162880 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda28e6c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda240b5f20 0x7fda240d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37136/0 (socket says 172.21.15.40:37136) 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda28e6c640 1 -- 172.21.15.40:0/681838861 learned_addr learned my addr 172.21.15.40:0/681838861 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 818745982 0 0) 0x7fda240b7e50 con 0x7fda240b5f20 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda2966d640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24162880 0x7fda240d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.501+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda0c003520 con 0x7fda240b5f20 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3335578992 0 0) 0x7fda0c003520 con 0x7fda240b5f20 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24162880 msgr2=0x7fda240d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24162880 0x7fda240d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 >> v1:172.21.15.184:6789/0 conn(0x7fda240d3bc0 legacy=0x7fda240b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda240d07d0 con 0x7fda240b5f20 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fda18001670 con 0x7fda240b5f20 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 >> v1:172.21.15.40:6789/0 conn(0x7fda240b5f20 legacy=0x7fda240d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda0c004800 0x7fda0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24162880 0x7fda0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda240d0580 con 0x7fda0c0039e0 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c00ba70 con 0x7fda0c004800 2024-04-26T21:42:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c00bc70 con 0x7fda24162880 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda2966d640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda28e6c640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda0c004800 0x7fda0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda29e6e640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24162880 0x7fda0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda2966d640 1 -- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24162880 msgr2=0x7fda0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda2966d640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24162880 0x7fda0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda2966d640 1 -- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda0c004800 msgr2=0x7fda0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda2966d640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda0c004800 0x7fda0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.502+0000 7fda2966d640 1 -- 172.21.15.40:0/681838861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c003520 con 0x7fda0c0039e0 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.503+0000 7fda2966d640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda0c003e00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fda14009f90 tx=0x7fda0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8706fb259ab6b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.503+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14009c10 con 0x7fda0c0039e0 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.503+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda14036070 con 0x7fda0c0039e0 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.503+0000 7fda23fff640 1 -- 172.21.15.40:0/681838861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14031830 con 0x7fda0c0039e0 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.504+0000 7fda2b8d1640 1 -- 172.21.15.40:0/681838861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 msgr2=0x7fda0c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.504+0000 7fda2b8d1640 1 --2- 172.21.15.40:0/681838861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda0c003e00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fda14009f90 tx=0x7fda0c005830 comp rx=0 tx=0).stop 2024-04-26T21:42:19.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.504+0000 7fda29e6e640 1 -- 172.21.15.40:0/681838861 reap_dead start 2024-04-26T21:42:19.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.504+0000 7fda2b8d1640 1 -- 172.21.15.40:0/681838861 shutdown_connections 2024-04-26T21:42:19.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.504+0000 7fda2b8d1640 1 -- 172.21.15.40:0/681838861 >> 172.21.15.40:0/681838861 conn(0x7fda240d4f60 msgr2=0x7fda240d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.504+0000 7fda2b8d1640 1 -- 172.21.15.40:0/681838861 shutdown_connections 2024-04-26T21:42:19.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.504+0000 7fda2b8d1640 1 -- 172.21.15.40:0/681838861 wait complete. 2024-04-26T21:42:19.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.505+0000 7fda2b8d1640 1 Processor -- start 2024-04-26T21:42:19.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.505+0000 7fda2b8d1640 1 -- start start 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2b8d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda2415df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2b8d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda240ce090 0x7fda240ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2b8d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2409fc50 0x7fda240a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2b8d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda24162c60 con 0x7fda0c0039e0 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2b8d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda240d0d90 con 0x7fda2409fc50 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2b8d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda24162de0 con 0x7fda240ce090 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda2415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda2415df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57188/0 (socket says 172.21.15.40:57188) 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 -- 172.21.15.40:0/1492385725 learned_addr learned my addr 172.21.15.40:0/1492385725 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda29e6e640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2409fc50 0x7fda240a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda28e6c640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda240ce090 0x7fda240ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 -- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda240ce090 msgr2=0x7fda240ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda240ce090 0x7fda240ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 -- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2409fc50 msgr2=0x7fda240a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2409fc50 0x7fda240a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 -- 172.21.15.40:0/1492385725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda240ccff0 con 0x7fda0c0039e0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda29e6e640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2409fc50 0x7fda240a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda28e6c640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda240ce090 0x7fda240ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.506+0000 7fda2966d640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda2415df90 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fda14009f90 tx=0x7fda14009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.507+0000 7fda21ffb640 1 -- 172.21.15.40:0/1492385725 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14003b90 con 0x7fda0c0039e0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.507+0000 7fda21ffb640 1 -- 172.21.15.40:0/1492385725 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda14003d50 con 0x7fda0c0039e0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.507+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda240cd2d0 con 0x7fda0c0039e0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.507+0000 7fda21ffb640 1 -- 172.21.15.40:0/1492385725 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14004260 con 0x7fda0c0039e0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.507+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda240cd890 con 0x7fda0c0039e0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.508+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda240d1010 con 0x7fda0c0039e0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.508+0000 7fda21ffb640 1 -- 172.21.15.40:0/1492385725 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda14005d10 con 0x7fda0c0039e0 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.509+0000 7fda21ffb640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda1006c8f0 0x7fda1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.509+0000 7fda21ffb640 1 -- 172.21.15.40:0/1492385725 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fda140ace50 con 0x7fda0c0039e0 2024-04-26T21:42:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.509+0000 7fda28e6c640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda1006c8f0 0x7fda1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.509+0000 7fda28e6c640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda1006c8f0 0x7fda1006edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fda18001470 tx=0x7fda18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.513+0000 7fda21ffb640 1 -- 172.21.15.40:0/1492385725 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda240d1010 con 0x7fda0c0039e0 2024-04-26T21:42:19.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.736+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fda240694f0 con 0x7fda0c0039e0 2024-04-26T21:42:19.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.737+0000 7fda21ffb640 1 -- 172.21.15.40:0/1492385725 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) ==== 53+0+73 (secure 0 0 0) 0x7fda1407b180 con 0x7fda0c0039e0 2024-04-26T21:42:19.738 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda1006c8f0 msgr2=0x7fda1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda1006c8f0 0x7fda1006edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fda18001470 tx=0x7fda18009040 comp rx=0 tx=0).stop 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 msgr2=0x7fda2415df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda2415df90 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fda14009f90 tx=0x7fda14009750 comp rx=0 tx=0).stop 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 shutdown_connections 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda1006c8f0 0x7fda1006edb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2409fc50 0x7fda240a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda240ce090 0x7fda240ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 --2- 172.21.15.40:0/1492385725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c0039e0 0x7fda2415df90 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 >> 172.21.15.40:0/1492385725 conn(0x7fda240d4f60 msgr2=0x7fda24160490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 shutdown_connections 2024-04-26T21:42:19.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19.739+0000 7fda2b8d1640 1 -- 172.21.15.40:0/1492385725 wait complete. 2024-04-26T21:42:19.750 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:42:19.750 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:42:19.751 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:42:19.751 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:42:19.751 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:19.751 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:42:19.751 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:42:19.751 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:42:19.751 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:42:19.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:19.751 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:42:19.767 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:42:19.767 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:42:19.767 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:42:19.767 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:19.767 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:42:19.767 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42308sec preferred_lft 42308sec 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:19.768 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:42:19.769 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:42:19.770 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:19.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:19.770 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:42:19.830 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:42:19.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:19.832 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:42:19.889 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:42:19.889 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:42:19.889 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:42:19.889 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:42:19.889 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:42:19.890 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:42:19.890 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:42:19.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:42:19.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:42:19.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:42:19.890 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:42:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:42:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:42:20.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:20.015 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:42:20.031 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:42:20.031 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:42:20.031 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:42:20.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:20.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:42:20.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:42:20.096 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:20.152 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:42:20.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:42:20.152 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:20.208 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:42:20.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:20.208 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:42:20.290 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:42:20.291 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:42:20.298 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:42:20.298 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:42:20.298 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:42:20.298 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:42:20.298 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:42:20.299 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:42:20.299 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:42:20.299 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:42:20.299 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:42:20.299 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:42:20.300 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:42:20.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:20.301 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:20.366 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:42:20.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:42:20.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:42:20.470 INFO:teuthology.orchestra.run.smithi040.stdout:client7926 2024-04-26T21:42:20.471 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:20.471 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:20.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:20.487 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-26T21:42:20.550 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:20.550 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:20.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:20.607 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-26T21:42:20.701 INFO:teuthology.orchestra.run.smithi040.stderr:6+0 records in 2024-04-26T21:42:20.701 INFO:teuthology.orchestra.run.smithi040.stderr:6+0 records out 2024-04-26T21:42:20.701 INFO:teuthology.orchestra.run.smithi040.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0371355 s, 169 MB/s 2024-04-26T21:42:20.701 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:42:20.701 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> import json 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> sys.exit(e.errno) 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> print(json.dumps( 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> indent=2)) 2024-04-26T21:42:20.702 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout: "st_mode": 33188, 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout: "st_ino": 1099511627777, 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout: "st_dev": 43, 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout: "st_nlink": 1, 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout: "st_uid": 1000, 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout: "st_gid": 1316, 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout: "st_size": 6291456, 2024-04-26T21:42:20.771 INFO:teuthology.orchestra.run.smithi040.stdout: "st_atime": 1714167740.6619005, 2024-04-26T21:42:20.772 INFO:teuthology.orchestra.run.smithi040.stdout: "st_mtime": 1714167740.6809003, 2024-04-26T21:42:20.772 INFO:teuthology.orchestra.run.smithi040.stdout: "st_ctime": 1714167740.6809003 2024-04-26T21:42:20.772 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:42:20.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:42:20.781 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:20.781 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:20.838 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:20.838 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:20.893 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:42:21.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.033+0000 7f8d6db19640 1 Processor -- start 2024-04-26T21:42:21.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.033+0000 7f8d6db19640 1 -- start start 2024-04-26T21:42:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d6db19640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d68162880 0x7f8d680d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d6db19640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8d680d3bc0 0x7f8d680b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d6db19640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d680b7e50 con 0x7f8d680b5f20 2024-04-26T21:42:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d6db19640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8d680d0580 con 0x7f8d68162880 2024-04-26T21:42:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d6db19640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8d680d1800 con 0x7f8d680d3bc0 2024-04-26T21:42:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d677fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d68162880 0x7f8d680d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d66ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8d680b5f20 0x7f8d680d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37144/0 (socket says 172.21.15.40:37144) 2024-04-26T21:42:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d66ffd640 1 -- 172.21.15.40:0/2783536960 learned_addr learned my addr 172.21.15.40:0/2783536960 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d67fff640 1 --2- 172.21.15.40:0/2783536960 >> v2:172.21.15.184:3300/0 conn(0x7f8d680d3bc0 0x7f8d680b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d667fc640 1 -- 172.21.15.40:0/2783536960 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733105304 0 0) 0x7f8d680b7e50 con 0x7f8d680b5f20 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d667fc640 1 -- 172.21.15.40:0/2783536960 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d50003520 con 0x7f8d680b5f20 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d677fe640 1 -- 172.21.15.40:0/2783536960 >> v2:172.21.15.184:3300/0 conn(0x7f8d680d3bc0 msgr2=0x7f8d680b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d677fe640 1 --2- 172.21.15.40:0/2783536960 >> v2:172.21.15.184:3300/0 conn(0x7f8d680d3bc0 0x7f8d680b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d677fe640 1 -- 172.21.15.40:0/2783536960 >> v1:172.21.15.40:6789/0 conn(0x7f8d680b5f20 legacy=0x7f8d680d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.034+0000 7f8d677fe640 1 -- 172.21.15.40:0/2783536960 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d680d07d0 con 0x7f8d68162880 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.035+0000 7f8d67fff640 1 --2- 172.21.15.40:0/2783536960 >> v2:172.21.15.184:3300/0 conn(0x7f8d680d3bc0 0x7f8d680b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.035+0000 7f8d677fe640 1 --2- 172.21.15.40:0/2783536960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d68162880 0x7f8d680d2cb0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8d680d11d0 tx=0x7f8d5802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db760556d4efff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.035+0000 7f8d667fc640 1 -- 172.21.15.40:0/2783536960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58037070 con 0x7f8d68162880 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.035+0000 7f8d667fc640 1 -- 172.21.15.40:0/2783536960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d58032440 con 0x7f8d68162880 2024-04-26T21:42:21.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.035+0000 7f8d667fc640 1 -- 172.21.15.40:0/2783536960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58032810 con 0x7f8d68162880 2024-04-26T21:42:21.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.036+0000 7f8d6db19640 1 -- 172.21.15.40:0/2783536960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d68162880 msgr2=0x7f8d680d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.036+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2783536960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d68162880 0x7f8d680d2cb0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8d680d11d0 tx=0x7f8d5802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.036+0000 7f8d6db19640 1 -- 172.21.15.40:0/2783536960 shutdown_connections 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.036+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2783536960 >> v2:172.21.15.184:3300/0 conn(0x7f8d680d3bc0 0x7f8d680b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.036+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2783536960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d68162880 0x7f8d680d2cb0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.036+0000 7f8d6db19640 1 -- 172.21.15.40:0/2783536960 >> 172.21.15.40:0/2783536960 conn(0x7f8d680d4f60 msgr2=0x7f8d680d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.036+0000 7f8d6db19640 1 -- 172.21.15.40:0/2783536960 shutdown_connections 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.036+0000 7f8d6db19640 1 -- 172.21.15.40:0/2783536960 wait complete. 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.037+0000 7f8d6db19640 1 Processor -- start 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.037+0000 7f8d6db19640 1 -- start start 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d6db19640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b5f20 0x7f8d680b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d6db19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680d3bc0 0x7f8d68153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d6db19640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d68162880 0x7f8d680b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d6db19640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68162c60 con 0x7f8d680d3bc0 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d6db19640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680d0d90 con 0x7f8d68162880 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d6db19640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8d68162de0 con 0x7f8d680b5f20 2024-04-26T21:42:21.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d66ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680d3bc0 0x7f8d68153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d66ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680d3bc0 0x7f8d68153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57202/0 (socket says 172.21.15.40:57202) 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d66ffd640 1 -- 172.21.15.40:0/2332586880 learned_addr learned my addr 172.21.15.40:0/2332586880 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d677fe640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b5f20 0x7f8d680b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d67fff640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d68162880 0x7f8d680b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d66ffd640 1 -- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b5f20 msgr2=0x7f8d680b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d66ffd640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b5f20 0x7f8d680b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d66ffd640 1 -- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d68162880 msgr2=0x7f8d680b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d66ffd640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d68162880 0x7f8d680b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d66ffd640 1 -- 172.21.15.40:0/2332586880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d680b2a30 con 0x7f8d680d3bc0 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.038+0000 7f8d67fff640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d68162880 0x7f8d680b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.039+0000 7f8d66ffd640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680d3bc0 0x7f8d68153290 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00c390 tx=0x7f8d5c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.039+0000 7f8d4ffff640 1 -- 172.21.15.40:0/2332586880 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c017070 con 0x7f8d680d3bc0 2024-04-26T21:42:21.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.039+0000 7f8d4ffff640 1 -- 172.21.15.40:0/2332586880 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5c005ce0 con 0x7f8d680d3bc0 2024-04-26T21:42:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.039+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d680b35f0 con 0x7f8d680d3bc0 2024-04-26T21:42:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.039+0000 7f8d4ffff640 1 -- 172.21.15.40:0/2332586880 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0072f0 con 0x7f8d680d3bc0 2024-04-26T21:42:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.039+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d680b1cf0 con 0x7f8d680d3bc0 2024-04-26T21:42:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.040+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d680764d0 con 0x7f8d680d3bc0 2024-04-26T21:42:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.040+0000 7f8d4ffff640 1 -- 172.21.15.40:0/2332586880 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d5c01a070 con 0x7f8d680d3bc0 2024-04-26T21:42:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.041+0000 7f8d4ffff640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.041+0000 7f8d4ffff640 1 -- 172.21.15.40:0/2332586880 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f8d5c08b180 con 0x7f8d680d3bc0 2024-04-26T21:42:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.041+0000 7f8d677fe640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.042+0000 7f8d677fe640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8d58034000 tx=0x7f8d5802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.046+0000 7f8d4ffff640 1 -- 172.21.15.40:0/2332586880 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d680764d0 con 0x7f8d680d3bc0 2024-04-26T21:42:21.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.231+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d680694f0 con 0x7f8d680d3bc0 2024-04-26T21:42:21.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.232+0000 7f8d4ffff640 1 -- 172.21.15.40:0/2332586880 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16849 (secure 0 0 0) 0x7f8d5c0593a0 con 0x7f8d680d3bc0 2024-04-26T21:42:21.233 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:21.234 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":94,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:42:10.243824+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:42:21.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 msgr2=0x7f8d5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8d58034000 tx=0x7f8d5802e000 comp rx=0 tx=0).stop 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680d3bc0 msgr2=0x7f8d68153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680d3bc0 0x7f8d68153290 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00c390 tx=0x7f8d5c00c850 comp rx=0 tx=0).stop 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 shutdown_connections 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d68162880 0x7f8d680b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680d3bc0 0x7f8d68153290 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 --2- 172.21.15.40:0/2332586880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b5f20 0x7f8d680b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 >> 172.21.15.40:0/2332586880 conn(0x7f8d680d4f60 msgr2=0x7f8d680ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 shutdown_connections 2024-04-26T21:42:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.234+0000 7f8d6db19640 1 -- 172.21.15.40:0/2332586880 wait complete. 2024-04-26T21:42:21.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:42:21.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:42:21.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.350+0000 7f5cb6d04640 1 Processor -- start 2024-04-26T21:42:21.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.350+0000 7f5cb6d04640 1 -- start start 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.350+0000 7f5cb6d04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cb00b0380 0x7f5cb00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb6d04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b10b0 0x7f5cb006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb6d04640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb00cfd80 con 0x7f5cb006c780 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb6d04640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb00af8e0 con 0x7f5cb00b0380 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb6d04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb00d0020 con 0x7f5cb00b10b0 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb4aa0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cb00b0380 0x7f5cb00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb4aa0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cb00b0380 0x7f5cb00ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57214/0 (socket says 172.21.15.40:57214) 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb4aa0640 1 -- 172.21.15.40:0/2496542470 learned_addr learned my addr 172.21.15.40:0/2496542470 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5caffff640 1 --2- 172.21.15.40:0/2496542470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b10b0 0x7f5cb006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb4aa0640 1 -- 172.21.15.40:0/2496542470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b10b0 msgr2=0x7f5cb006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb4aa0640 1 --2- 172.21.15.40:0/2496542470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b10b0 0x7f5cb006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb4aa0640 1 -- 172.21.15.40:0/2496542470 >> v1:172.21.15.184:6789/0 conn(0x7f5cb006c780 legacy=0x7f5cb0072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.351+0000 7f5cb4aa0640 1 -- 172.21.15.40:0/2496542470 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb00d0300 con 0x7f5cb00b0380 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.352+0000 7f5cb4aa0640 1 --2- 172.21.15.40:0/2496542470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb00b0380 0x7f5cb00ccd50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009e50 tx=0x7f5ca802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9943291f2b776a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.352+0000 7f5caf7fe640 1 -- 172.21.15.40:0/2496542470 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8036070 con 0x7f5cb00b0380 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.352+0000 7f5caf7fe640 1 -- 172.21.15.40:0/2496542470 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca8031600 con 0x7f5cb00b0380 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.352+0000 7f5caf7fe640 1 -- 172.21.15.40:0/2496542470 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80319d0 con 0x7f5cb00b0380 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.352+0000 7f5cb6d04640 1 -- 172.21.15.40:0/2496542470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb00b0380 msgr2=0x7f5cb00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.352+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/2496542470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb00b0380 0x7f5cb00ccd50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009e50 tx=0x7f5ca802a630 comp rx=0 tx=0).stop 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.353+0000 7f5cb6d04640 1 -- 172.21.15.40:0/2496542470 shutdown_connections 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.353+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/2496542470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b10b0 0x7f5cb006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.353+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/2496542470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb00b0380 0x7f5cb00ccd50 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.353+0000 7f5cb6d04640 1 -- 172.21.15.40:0/2496542470 >> 172.21.15.40:0/2496542470 conn(0x7f5cb00d1cf0 msgr2=0x7f5cb00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:21.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.353+0000 7f5cb6d04640 1 -- 172.21.15.40:0/2496542470 shutdown_connections 2024-04-26T21:42:21.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.353+0000 7f5cb6d04640 1 -- 172.21.15.40:0/2496542470 wait complete. 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.354+0000 7f5cb6d04640 1 Processor -- start 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.354+0000 7f5cb6d04640 1 -- start start 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb6d04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb006c780 0x7f5cb0155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb6d04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b0380 0x7f5cb00c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb6d04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb00b10b0 0x7f5cb01e9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb6d04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00b0760 con 0x7f5cb006c780 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb6d04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00d1980 con 0x7f5cb00b10b0 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb6d04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb00d15c0 con 0x7f5cb00b0380 2024-04-26T21:42:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb006c780 0x7f5cb0155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb006c780 0x7f5cb0155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57228/0 (socket says 172.21.15.40:57228) 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 -- 172.21.15.40:0/324523667 learned_addr learned my addr 172.21.15.40:0/324523667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb52a1640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb00b10b0 0x7f5cb01e9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5caffff640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b0380 0x7f5cb00c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 -- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b0380 msgr2=0x7f5cb00c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b0380 0x7f5cb00c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 -- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb00b10b0 msgr2=0x7f5cb01e9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb00b10b0 0x7f5cb01e9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 -- 172.21.15.40:0/324523667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb00c9d90 con 0x7f5cb006c780 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb52a1640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb00b10b0 0x7f5cb01e9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb4aa0640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb006c780 0x7f5cb0155000 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009f70 tx=0x7f5ca8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cad7fa640 1 -- 172.21.15.40:0/324523667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca803f020 con 0x7f5cb006c780 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cad7fa640 1 -- 172.21.15.40:0/324523667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca8037040 con 0x7f5cb006c780 2024-04-26T21:42:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cad7fa640 1 -- 172.21.15.40:0/324523667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8036070 con 0x7f5cb006c780 2024-04-26T21:42:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb00ca070 con 0x7f5cb006c780 2024-04-26T21:42:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.355+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb00ca630 con 0x7f5cb006c780 2024-04-26T21:42:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.356+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb00694f0 con 0x7f5cb006c780 2024-04-26T21:42:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.357+0000 7f5cad7fa640 1 -- 172.21.15.40:0/324523667 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ca8042020 con 0x7f5cb006c780 2024-04-26T21:42:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.357+0000 7f5cad7fa640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c9006c8f0 0x7f5c9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.357+0000 7f5cad7fa640 1 -- 172.21.15.40:0/324523667 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f5ca80ac2c0 con 0x7f5cb006c780 2024-04-26T21:42:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.357+0000 7f5caffff640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c9006c8f0 0x7f5c9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.358+0000 7f5caffff640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c9006c8f0 0x7f5c9006edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5c98006fd0 tx=0x7f5c98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.362+0000 7f5cad7fa640 1 -- 172.21.15.40:0/324523667 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ca8049030 con 0x7f5cb006c780 2024-04-26T21:42:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.547+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5cb00b2950 con 0x7f5cb006c780 2024-04-26T21:42:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.547+0000 7f5cad7fa640 1 -- 172.21.15.40:0/324523667 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f5ca807a560 con 0x7f5cb006c780 2024-04-26T21:42:21.548 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:42:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c9006c8f0 msgr2=0x7f5c9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c9006c8f0 0x7f5c9006edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5c98006fd0 tx=0x7f5c98008040 comp rx=0 tx=0).stop 2024-04-26T21:42:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb006c780 msgr2=0x7f5cb0155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb006c780 0x7f5cb0155000 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009f70 tx=0x7f5ca8004a40 comp rx=0 tx=0).stop 2024-04-26T21:42:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 shutdown_connections 2024-04-26T21:42:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5c9006c8f0 0x7f5c9006edb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb00b10b0 0x7f5cb01e9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb00b0380 0x7f5cb00c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 --2- 172.21.15.40:0/324523667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cb006c780 0x7f5cb0155000 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 >> 172.21.15.40:0/324523667 conn(0x7f5cb00d1cf0 msgr2=0x7f5cb00b6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 shutdown_connections 2024-04-26T21:42:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.549+0000 7f5cb6d04640 1 -- 172.21.15.40:0/324523667 wait complete. 2024-04-26T21:42:21.561 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:42:21.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.663+0000 7f77e6637640 1 Processor -- start 2024-04-26T21:42:21.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.663+0000 7f77e6637640 1 -- start start 2024-04-26T21:42:21.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.664+0000 7f77e6637640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77e015f170 0x7f77e00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.664+0000 7f77e6637640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e00d1040 con 0x7f77e00d3bc0 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.664+0000 7f77e6637640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e00c9680 con 0x7f77e015f550 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.664+0000 7f77e6637640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f77e00c9430 con 0x7f77e015f170 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.664+0000 7f77e4bd4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f77e00d3bc0 0x7f77e00ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37150/0 (socket says 172.21.15.40:37150) 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.664+0000 7f77dffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77e015f170 0x7f77e00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.664+0000 7f77e4bd4640 1 -- 172.21.15.40:0/1694954413 learned_addr learned my addr 172.21.15.40:0/1694954413 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.665+0000 7f77dffff640 1 -- 172.21.15.40:0/1694954413 >> v1:172.21.15.184:6789/0 conn(0x7f77e015f550 legacy=0x7f77e00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.665+0000 7f77dffff640 1 -- 172.21.15.40:0/1694954413 >> v1:172.21.15.40:6789/0 conn(0x7f77e00d3bc0 legacy=0x7f77e00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.665+0000 7f77deffd640 1 -- 172.21.15.40:0/1694954413 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 628806948 0 0) 0x7f77d8001490 con 0x7f77e00d3bc0 2024-04-26T21:42:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.665+0000 7f77dffff640 1 -- 172.21.15.40:0/1694954413 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e00c9960 con 0x7f77e015f170 2024-04-26T21:42:21.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.665+0000 7f77dffff640 1 --2- 172.21.15.40:0/1694954413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e00d5a80 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f77e00c95b0 tx=0x7f77d002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bcf10725f4e5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.665+0000 7f77deffd640 1 -- 172.21.15.40:0/1694954413 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0037070 con 0x7f77e015f170 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.665+0000 7f77deffd640 1 -- 172.21.15.40:0/1694954413 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d0032440 con 0x7f77e015f170 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.665+0000 7f77deffd640 1 -- 172.21.15.40:0/1694954413 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0032810 con 0x7f77e015f170 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.666+0000 7f77e6637640 1 -- 172.21.15.40:0/1694954413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 msgr2=0x7f77e00d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.666+0000 7f77e6637640 1 --2- 172.21.15.40:0/1694954413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e00d5a80 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f77e00c95b0 tx=0x7f77d002a200 comp rx=0 tx=0).stop 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.666+0000 7f77e6637640 1 -- 172.21.15.40:0/1694954413 shutdown_connections 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.666+0000 7f77e6637640 1 --2- 172.21.15.40:0/1694954413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e00d5a80 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.666+0000 7f77e6637640 1 -- 172.21.15.40:0/1694954413 >> 172.21.15.40:0/1694954413 conn(0x7f77e00d4f60 msgr2=0x7f77e00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.666+0000 7f77e6637640 1 -- 172.21.15.40:0/1694954413 shutdown_connections 2024-04-26T21:42:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.666+0000 7f77e6637640 1 -- 172.21.15.40:0/1694954413 wait complete. 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.667+0000 7f77e6637640 1 Processor -- start 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.667+0000 7f77e6637640 1 -- start start 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77e6637640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e00d3bc0 0x7f77e00cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77e6637640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e0152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77e6637640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e00a0570 0x7f77e00a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77e6637640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77e00d2e30 con 0x7f77e015f170 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77e6637640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77e00d2cb0 con 0x7f77e00a0570 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77e6637640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77e00d2fb0 con 0x7f77e00d3bc0 2024-04-26T21:42:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e0152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e0152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57252/0 (socket says 172.21.15.40:57252) 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.668+0000 7f77df7fe640 1 -- 172.21.15.40:0/3928923935 learned_addr learned my addr 172.21.15.40:0/3928923935 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77e4bd4640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e00a0570 0x7f77e00a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77dffff640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e00d3bc0 0x7f77e00cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77df7fe640 1 -- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e00d3bc0 msgr2=0x7f77e00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77df7fe640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e00d3bc0 0x7f77e00cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77df7fe640 1 -- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e00a0570 msgr2=0x7f77e00a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77df7fe640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e00a0570 0x7f77e00a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77df7fe640 1 -- 172.21.15.40:0/3928923935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e00cd090 con 0x7f77e015f170 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77e4bd4640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e00a0570 0x7f77e00a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77dffff640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e00d3bc0 0x7f77e00cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77df7fe640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e0152340 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009fd0 tx=0x7f77c800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77dcff9640 1 -- 172.21.15.40:0/3928923935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8015020 con 0x7f77e015f170 2024-04-26T21:42:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77dcff9640 1 -- 172.21.15.40:0/3928923935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c8011830 con 0x7f77e015f170 2024-04-26T21:42:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e00cd300 con 0x7f77e015f170 2024-04-26T21:42:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77dcff9640 1 -- 172.21.15.40:0/3928923935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8011c00 con 0x7f77e015f170 2024-04-26T21:42:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.669+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e0152880 con 0x7f77e015f170 2024-04-26T21:42:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.670+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e00694f0 con 0x7f77e015f170 2024-04-26T21:42:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.670+0000 7f77dcff9640 1 -- 172.21.15.40:0/3928923935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77c8018020 con 0x7f77e015f170 2024-04-26T21:42:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.671+0000 7f77dcff9640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f77c006c8f0 0x7f77c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.671+0000 7f77dcff9640 1 -- 172.21.15.40:0/3928923935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f77c808bb20 con 0x7f77e015f170 2024-04-26T21:42:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.671+0000 7f77dffff640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f77c006c8f0 0x7f77c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.672+0000 7f77dffff640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f77c006c8f0 0x7f77c006edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f77d0034000 tx=0x7f77d002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.675+0000 7f77dcff9640 1 -- 172.21.15.40:0/3928923935 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77c801f020 con 0x7f77e015f170 2024-04-26T21:42:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.859+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f77e00c8c80 con 0x7f77e015f170 2024-04-26T21:42:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.859+0000 7f77dcff9640 1 -- 172.21.15.40:0/3928923935 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f77c80172c0 con 0x7f77e015f170 2024-04-26T21:42:21.860 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:42:21.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f77c006c8f0 msgr2=0x7f77c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f77c006c8f0 0x7f77c006edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f77d0034000 tx=0x7f77d002e000 comp rx=0 tx=0).stop 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 msgr2=0x7f77e0152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e0152340 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009fd0 tx=0x7f77c800b870 comp rx=0 tx=0).stop 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 shutdown_connections 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f77c006c8f0 0x7f77c006edb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e00a0570 0x7f77e00a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77e015f170 0x7f77e0152340 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 --2- 172.21.15.40:0/3928923935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e00d3bc0 0x7f77e00cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 >> 172.21.15.40:0/3928923935 conn(0x7f77e00d4f60 msgr2=0x7f77e00d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 shutdown_connections 2024-04-26T21:42:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.861+0000 7f77e6637640 1 -- 172.21.15.40:0/3928923935 wait complete. 2024-04-26T21:42:21.873 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:42:21.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.975+0000 7f9cdfd38640 1 Processor -- start 2024-04-26T21:42:21.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.976+0000 7f9cdfd38640 1 -- start start 2024-04-26T21:42:21.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.976+0000 7f9cdfd38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.976+0000 7f9cdfd38640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cd8162b30 0x7f9cd80b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.976+0000 7f9cdfd38640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd80d0a10 con 0x7f9cd80d1a00 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.976+0000 7f9cdfd38640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd80d0cf0 con 0x7f9cd8162b30 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.976+0000 7f9cdfd38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd80d0fd0 con 0x7f9cd80b40d0 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cde2d5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cd8162b30 0x7f9cd80b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cde2d5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cd8162b30 0x7f9cd80b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57266/0 (socket says 172.21.15.40:57266) 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cde2d5640 1 -- 172.21.15.40:0/2327968926 learned_addr learned my addr 172.21.15.40:0/2327968926 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cdd2d3640 1 --2- 172.21.15.40:0/2327968926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cde2d5640 1 -- 172.21.15.40:0/2327968926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 msgr2=0x7f9cd81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cde2d5640 1 --2- 172.21.15.40:0/2327968926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cde2d5640 1 -- 172.21.15.40:0/2327968926 >> v1:172.21.15.184:6789/0 conn(0x7f9cd80d1a00 legacy=0x7f9cd80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cde2d5640 1 -- 172.21.15.40:0/2327968926 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd8162fa0 con 0x7f9cd8162b30 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.977+0000 7f9cdd2d3640 1 --2- 172.21.15.40:0/2327968926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cde2d5640 1 --2- 172.21.15.40:0/2327968926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80b4b00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009e50 tx=0x7f9cc802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6f65398d02c14c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdcad2640 1 -- 172.21.15.40:0/2327968926 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8036070 con 0x7f9cd8162b30 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdcad2640 1 -- 172.21.15.40:0/2327968926 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc8031600 con 0x7f9cd8162b30 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdcad2640 1 -- 172.21.15.40:0/2327968926 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80319d0 con 0x7f9cd8162b30 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2327968926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 msgr2=0x7f9cd80b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2327968926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80b4b00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009e50 tx=0x7f9cc802a630 comp rx=0 tx=0).stop 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2327968926 shutdown_connections 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2327968926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80b4b00 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2327968926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.978+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2327968926 >> 172.21.15.40:0/2327968926 conn(0x7f9cd80d4fd0 msgr2=0x7f9cd80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.979+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2327968926 shutdown_connections 2024-04-26T21:42:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.979+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2327968926 wait complete. 2024-04-26T21:42:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.980+0000 7f9cdfd38640 1 Processor -- start 2024-04-26T21:42:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.980+0000 7f9cdfd38640 1 -- start start 2024-04-26T21:42:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.980+0000 7f9cdfd38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.980+0000 7f9cdfd38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd80d1a00 0x7f9cd80ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.980+0000 7f9cdfd38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.980+0000 7f9cdfd38640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd80d2670 con 0x7f9cd8162b30 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.980+0000 7f9cdfd38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd80d24f0 con 0x7f9cd80d1a00 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.980+0000 7f9cdfd38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd80b5040 con 0x7f9cd80b40d0 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57280/0 (socket says 172.21.15.40:57280) 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 -- 172.21.15.40:0/2316269255 learned_addr learned my addr 172.21.15.40:0/2316269255 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cddad4640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cdd2d3640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd80d1a00 0x7f9cd80ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 -- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 msgr2=0x7f9cd81f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 -- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd80d1a00 msgr2=0x7f9cd80ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd80d1a00 0x7f9cd80ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 -- 172.21.15.40:0/2316269255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd80b8de0 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cde2d5640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80caa40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8034040 tx=0x7f9cc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cd67fc640 1 -- 172.21.15.40:0/2316269255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8036050 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cd67fc640 1 -- 172.21.15.40:0/2316269255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc8042070 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd80b90c0 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cd67fc640 1 -- 172.21.15.40:0/2316269255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8009510 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.981+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd80b9ba0 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.982+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd80694f0 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.983+0000 7f9cd67fc640 1 -- 172.21.15.40:0/2316269255 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cc8007bf0 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.983+0000 7f9cd67fc640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.984+0000 7f9cd67fc640 1 -- 172.21.15.40:0/2316269255 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f9cc80acb20 con 0x7f9cd8162b30 2024-04-26T21:42:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.984+0000 7f9cddad4640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:21.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.984+0000 7f9cddad4640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9cc406c8f0 0x7f9cc406edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0004f60 tx=0x7f9cd0007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:21.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:21.988+0000 7f9cd67fc640 1 -- 172.21.15.40:0/2316269255 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9cc807a770 con 0x7f9cd8162b30 2024-04-26T21:42:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.173+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9cd80ca2e0 con 0x7f9cd8162b30 2024-04-26T21:42:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.173+0000 7f9cd67fc640 1 -- 172.21.15.40:0/2316269255 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f9cc807ea70 con 0x7f9cd8162b30 2024-04-26T21:42:22.175 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:42:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9cc406c8f0 msgr2=0x7f9cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9cc406c8f0 0x7f9cc406edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0004f60 tx=0x7f9cd0007e30 comp rx=0 tx=0).stop 2024-04-26T21:42:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 msgr2=0x7f9cd80caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80caa40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8034040 tx=0x7f9cc8004a40 comp rx=0 tx=0).stop 2024-04-26T21:42:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 shutdown_connections 2024-04-26T21:42:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8162b30 0x7f9cd80caa40 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd80d1a00 0x7f9cd80ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 --2- 172.21.15.40:0/2316269255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd80b40d0 0x7f9cd81f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 >> 172.21.15.40:0/2316269255 conn(0x7f9cd80d4fd0 msgr2=0x7f9cd80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 shutdown_connections 2024-04-26T21:42:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.175+0000 7f9cdfd38640 1 -- 172.21.15.40:0/2316269255 wait complete. 2024-04-26T21:42:22.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:42:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.289+0000 7f98f4664640 1 Processor -- start 2024-04-26T21:42:22.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.289+0000 7f98f4664640 1 -- start start 2024-04-26T21:42:22.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f4664640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98ec0d06f0 0x7f98ec0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f4664640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec0b2850 0x7f98ec0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f4664640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec0d0ba0 con 0x7f98ec15f3a0 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f4664640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f98ec0d0e20 con 0x7f98ec0d06f0 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f4664640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec0b1d70 con 0x7f98ec0b2850 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f1bff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98ec0d06f0 0x7f98ec0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f1bff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98ec0d06f0 0x7f98ec0d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57292/0 (socket says 172.21.15.40:57292) 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f1bff640 1 -- 172.21.15.40:0/4161241282 learned_addr learned my addr 172.21.15.40:0/4161241282 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.290+0000 7f98f2c01640 1 --2- 172.21.15.40:0/4161241282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec0b2850 0x7f98ec0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.291+0000 7f98f1bff640 1 -- 172.21.15.40:0/4161241282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec0b2850 msgr2=0x7f98ec0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.291+0000 7f98f1bff640 1 --2- 172.21.15.40:0/4161241282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec0b2850 0x7f98ec0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.291+0000 7f98f1bff640 1 -- 172.21.15.40:0/4161241282 >> v1:172.21.15.184:6789/0 conn(0x7f98ec15f3a0 legacy=0x7f98ec0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.291+0000 7f98f1bff640 1 -- 172.21.15.40:0/4161241282 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec0b2050 con 0x7f98ec0d06f0 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.291+0000 7f98f1bff640 1 --2- 172.21.15.40:0/4161241282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec0d1e80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009870 tx=0x7f98dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7727cefa506dc661 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.291+0000 7f98f13fe640 1 -- 172.21.15.40:0/4161241282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc037070 con 0x7f98ec0d06f0 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f13fe640 1 -- 172.21.15.40:0/4161241282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc032440 con 0x7f98ec0d06f0 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f13fe640 1 -- 172.21.15.40:0/4161241282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc032810 con 0x7f98ec0d06f0 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f4664640 1 -- 172.21.15.40:0/4161241282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 msgr2=0x7f98ec0d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f4664640 1 --2- 172.21.15.40:0/4161241282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec0d1e80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009870 tx=0x7f98dc02a270 comp rx=0 tx=0).stop 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f4664640 1 -- 172.21.15.40:0/4161241282 shutdown_connections 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f4664640 1 --2- 172.21.15.40:0/4161241282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec0b2850 0x7f98ec0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f4664640 1 --2- 172.21.15.40:0/4161241282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec0d1e80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f4664640 1 -- 172.21.15.40:0/4161241282 >> 172.21.15.40:0/4161241282 conn(0x7f98ec0d4f90 msgr2=0x7f98ec0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f4664640 1 -- 172.21.15.40:0/4161241282 shutdown_connections 2024-04-26T21:42:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.292+0000 7f98f4664640 1 -- 172.21.15.40:0/4161241282 wait complete. 2024-04-26T21:42:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f4664640 1 Processor -- start 2024-04-26T21:42:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f4664640 1 -- start start 2024-04-26T21:42:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f4664640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98ec0b2850 0x7f98ec0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f4664640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f4664640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec15f3a0 0x7f98ec09ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f4664640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec15f900 con 0x7f98ec0d06f0 2024-04-26T21:42:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f4664640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec15f780 con 0x7f98ec0b2850 2024-04-26T21:42:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f4664640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec0d23c0 con 0x7f98ec15f3a0 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f1bff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f1bff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57308/0 (socket says 172.21.15.40:57308) 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.294+0000 7f98f1bff640 1 -- 172.21.15.40:0/1180298375 learned_addr learned my addr 172.21.15.40:0/1180298375 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f2400640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98ec0b2850 0x7f98ec0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f2c01640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec15f3a0 0x7f98ec09ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f1bff640 1 -- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec15f3a0 msgr2=0x7f98ec09ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f1bff640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec15f3a0 0x7f98ec09ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f1bff640 1 -- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98ec0b2850 msgr2=0x7f98ec0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f1bff640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98ec0b2850 0x7f98ec0cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f1bff640 1 -- 172.21.15.40:0/1180298375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec0cd590 con 0x7f98ec0d06f0 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f1bff640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec158110 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f98dc000c00 tx=0x7f98dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98e2ffd640 1 -- 172.21.15.40:0/1180298375 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc041020 con 0x7f98ec0d06f0 2024-04-26T21:42:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98e2ffd640 1 -- 172.21.15.40:0/1180298375 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc0306c0 con 0x7f98ec0d06f0 2024-04-26T21:42:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec0cd870 con 0x7f98ec0d06f0 2024-04-26T21:42:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98e2ffd640 1 -- 172.21.15.40:0/1180298375 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc037040 con 0x7f98ec0d06f0 2024-04-26T21:42:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.295+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec0cde30 con 0x7f98ec0d06f0 2024-04-26T21:42:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.296+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ec0694f0 con 0x7f98ec0d06f0 2024-04-26T21:42:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.297+0000 7f98e2ffd640 1 -- 172.21.15.40:0/1180298375 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98dc044020 con 0x7f98ec0d06f0 2024-04-26T21:42:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.297+0000 7f98e2ffd640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98d006c8f0 0x7f98d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.297+0000 7f98e2ffd640 1 -- 172.21.15.40:0/1180298375 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f98dc0ac3f0 con 0x7f98ec0d06f0 2024-04-26T21:42:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.297+0000 7f98f2400640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98d006c8f0 0x7f98d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.298+0000 7f98f2400640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98d006c8f0 0x7f98d006edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f98e8001f10 tx=0x7f98e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.302+0000 7f98e2ffd640 1 -- 172.21.15.40:0/1180298375 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f98dc04b030 con 0x7f98ec0d06f0 2024-04-26T21:42:22.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.498+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f98ec0b6860 con 0x7f98ec0d06f0 2024-04-26T21:42:22.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.499+0000 7f98e2ffd640 1 -- 172.21.15.40:0/1180298375 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v94) ==== 80+0+0 (secure 0 0 0) 0x7f98dc07a040 con 0x7f98ec0d06f0 2024-04-26T21:42:22.500 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:42:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98d006c8f0 msgr2=0x7f98d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98d006c8f0 0x7f98d006edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f98e8001f10 tx=0x7f98e8009040 comp rx=0 tx=0).stop 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 msgr2=0x7f98ec158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec158110 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f98dc000c00 tx=0x7f98dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 shutdown_connections 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98d006c8f0 0x7f98d006edb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98ec15f3a0 0x7f98ec09ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec0d06f0 0x7f98ec158110 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 --2- 172.21.15.40:0/1180298375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98ec0b2850 0x7f98ec0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.500+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 >> 172.21.15.40:0/1180298375 conn(0x7f98ec0d4f90 msgr2=0x7f98ec06cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.501+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 shutdown_connections 2024-04-26T21:42:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.501+0000 7f98f4664640 1 -- 172.21.15.40:0/1180298375 wait complete. 2024-04-26T21:42:22.512 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:42:22.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:22.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:22.556 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:42:22.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:22.557 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:42:22.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:22.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.750+0000 7f00de26d640 1 Processor -- start 2024-04-26T21:42:22.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.750+0000 7f00de26d640 1 -- start start 2024-04-26T21:42:22.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.751+0000 7f00de26d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d815f3a0 0x7f00d80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.751+0000 7f00de26d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00d80d2300 0x7f00d80b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.751+0000 7f00de26d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00d80d0e20 con 0x7f00d80d1770 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.751+0000 7f00de26d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f00d80b1d70 con 0x7f00d80d2300 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.751+0000 7f00de26d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00d80b2050 con 0x7f00d815f3a0 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d815f3a0 0x7f00d80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00dc80a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00d80d2300 0x7f00d80b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d815f3a0 0x7f00d80d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:33740/0 (socket says 172.21.15.40:33740) 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00dc80a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00d80d2300 0x7f00d80b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:45202/0 (socket says 172.21.15.40:45202) 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00d7fff640 1 -- 172.21.15.40:0/1322549878 learned_addr learned my addr 172.21.15.40:0/1322549878 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:22.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00d77fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f00d80d1770 0x7f00d80d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:37816/0 (socket says 172.21.15.40:37816) 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00dc80a640 1 -- 172.21.15.40:0/1322549878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d815f3a0 msgr2=0x7f00d80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00dc80a640 1 --2- 172.21.15.40:0/1322549878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d815f3a0 0x7f00d80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00dc80a640 1 -- 172.21.15.40:0/1322549878 >> v1:172.21.15.184:6789/0 conn(0x7f00d80d1770 legacy=0x7f00d80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00dc80a640 1 -- 172.21.15.40:0/1322549878 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d80b2330 con 0x7f00d80d2300 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.752+0000 7f00d7fff640 1 --2- 172.21.15.40:0/1322549878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d815f3a0 0x7f00d80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00dc80a640 1 --2- 172.21.15.40:0/1322549878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d80d2300 0x7f00d80b2850 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009d90 tx=0x7f00d002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9da7b29c64274f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00d6ffd640 1 -- 172.21.15.40:0/1322549878 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0007740 con 0x7f00d80d2300 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00d6ffd640 1 -- 172.21.15.40:0/1322549878 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d00078e0 con 0x7f00d80d2300 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00d6ffd640 1 -- 172.21.15.40:0/1322549878 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0007cb0 con 0x7f00d80d2300 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00de26d640 1 -- 172.21.15.40:0/1322549878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d80d2300 msgr2=0x7f00d80b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00de26d640 1 --2- 172.21.15.40:0/1322549878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d80d2300 0x7f00d80b2850 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009d90 tx=0x7f00d002a660 comp rx=0 tx=0).stop 2024-04-26T21:42:22.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00de26d640 1 -- 172.21.15.40:0/1322549878 shutdown_connections 2024-04-26T21:42:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00de26d640 1 --2- 172.21.15.40:0/1322549878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d80d2300 0x7f00d80b2850 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00de26d640 1 --2- 172.21.15.40:0/1322549878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d815f3a0 0x7f00d80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.753+0000 7f00de26d640 1 -- 172.21.15.40:0/1322549878 >> 172.21.15.40:0/1322549878 conn(0x7f00d80d4f90 msgr2=0x7f00d80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.754+0000 7f00de26d640 1 -- 172.21.15.40:0/1322549878 shutdown_connections 2024-04-26T21:42:22.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.754+0000 7f00de26d640 1 -- 172.21.15.40:0/1322549878 wait complete. 2024-04-26T21:42:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.755+0000 7f00de26d640 1 Processor -- start 2024-04-26T21:42:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.755+0000 7f00de26d640 1 -- start start 2024-04-26T21:42:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00de26d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d80d1770 0x7f00d81512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00de26d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00d80d2300 0x7f00d80ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00de26d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d815f3a0 0x7f00d80cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00de26d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00d80b2f10 con 0x7f00d80d2300 2024-04-26T21:42:22.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00de26d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00d80b2d90 con 0x7f00d815f3a0 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00de26d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00d80b3090 con 0x7f00d80d1770 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00d80d2300 0x7f00d80ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00d80d2300 0x7f00d80ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57320/0 (socket says 172.21.15.40:57320) 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 -- 172.21.15.40:0/229173826 learned_addr learned my addr 172.21.15.40:0/229173826 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00dc80a640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d815f3a0 0x7f00d80cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d7fff640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d80d1770 0x7f00d81512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 -- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d80d1770 msgr2=0x7f00d81512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d80d1770 0x7f00d81512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 -- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d815f3a0 msgr2=0x7f00d80cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d815f3a0 0x7f00d80cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 -- 172.21.15.40:0/229173826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d8160570 con 0x7f00d80d2300 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d77fe640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00d80d2300 0x7f00d80ca8b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00c430 tx=0x7f00cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.756+0000 7f00d4ff9640 1 -- 172.21.15.40:0/229173826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0074e0 con 0x7f00d80d2300 2024-04-26T21:42:22.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.757+0000 7f00d4ff9640 1 -- 172.21.15.40:0/229173826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00cc007680 con 0x7f00d80d2300 2024-04-26T21:42:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.757+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d8160850 con 0x7f00d80d2300 2024-04-26T21:42:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.757+0000 7f00d4ff9640 1 -- 172.21.15.40:0/229173826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc004a00 con 0x7f00d80d2300 2024-04-26T21:42:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.757+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d8160e10 con 0x7f00d80d2300 2024-04-26T21:42:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.757+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d80694f0 con 0x7f00d80d2300 2024-04-26T21:42:22.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.758+0000 7f00d4ff9640 1 -- 172.21.15.40:0/229173826 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00cc01a020 con 0x7f00d80d2300 2024-04-26T21:42:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.758+0000 7f00d4ff9640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00c406c8f0 0x7f00c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.759+0000 7f00d4ff9640 1 -- 172.21.15.40:0/229173826 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f00cc08b340 con 0x7f00d80d2300 2024-04-26T21:42:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.759+0000 7f00d7fff640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00c406c8f0 0x7f00c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:22.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.759+0000 7f00d7fff640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00c406c8f0 0x7f00c406edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f00c8006ca0 tx=0x7f00c8009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:22.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.763+0000 7f00d4ff9640 1 -- 172.21.15.40:0/229173826 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00cc059020 con 0x7f00d80d2300 2024-04-26T21:42:22.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.985+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00d80b6860 con 0x7f00d80d2300 2024-04-26T21:42:22.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.985+0000 7f00d4ff9640 1 -- 172.21.15.40:0/229173826 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5189 (secure 0 0 0) 0x7f00cc05d320 con 0x7f00d80d2300 2024-04-26T21:42:22.987 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:22.987 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:11.621437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7770":{"gid":7770,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2715490806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2715490806},{"type":"v1","addr":"172.21.15.40:6835","nonce":2715490806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7770,"qdb_cluster":[7770]},"id":6}]} 2024-04-26T21:42:22.987 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 87 2024-04-26T21:42:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00c406c8f0 msgr2=0x7f00c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00c406c8f0 0x7f00c406edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f00c8006ca0 tx=0x7f00c8009240 comp rx=0 tx=0).stop 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00d80d2300 msgr2=0x7f00d80ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00d80d2300 0x7f00d80ca8b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00c430 tx=0x7f00cc00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 shutdown_connections 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00c406c8f0 0x7f00c406edb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d815f3a0 0x7f00d80cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00d80d2300 0x7f00d80ca8b0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 --2- 172.21.15.40:0/229173826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00d80d1770 0x7f00d81512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 >> 172.21.15.40:0/229173826 conn(0x7f00d80d4f90 msgr2=0x7f00d806cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 shutdown_connections 2024-04-26T21:42:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:22.987+0000 7f00de26d640 1 -- 172.21.15.40:0/229173826 wait complete. 2024-04-26T21:42:22.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-26T21:42:23.125 INFO:teuthology.orchestra.run.smithi040.stdout:{"message":"","return_code":0} 2024-04-26T21:42:23.125 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:42:23.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.226+0000 7f008dec8640 1 Processor -- start 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.226+0000 7f008dec8640 1 -- start start 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f008dec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0088162880 0x7f00880d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f008dec8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00880b7e50 con 0x7f00880d3bc0 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f008dec8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00880d1800 con 0x7f00880b5f20 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f008dec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00880d0580 con 0x7f0088162880 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f0087fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f00880d3bc0 0x7f00880b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37166/0 (socket says 172.21.15.40:37166) 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f0087fff640 1 -- 172.21.15.40:0/2622565967 learned_addr learned my addr 172.21.15.40:0/2622565967 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f00877fe640 1 --2- 172.21.15.40:0/2622565967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0088162880 0x7f00880d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f00867fc640 1 -- 172.21.15.40:0/2622565967 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1126678611 0 0) 0x7f00880b7e50 con 0x7f00880d3bc0 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.227+0000 7f00867fc640 1 -- 172.21.15.40:0/2622565967 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0070003520 con 0x7f00880d3bc0 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.228+0000 7f00867fc640 1 -- 172.21.15.40:0/2622565967 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 282407200 0 0) 0x7f007c001110 con 0x7f00880b5f20 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.228+0000 7f00867fc640 1 -- 172.21.15.40:0/2622565967 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00880b7e50 con 0x7f00880b5f20 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.228+0000 7f00867fc640 1 -- 172.21.15.40:0/2622565967 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1722200117 0 0) 0x7f0070003520 con 0x7f00880d3bc0 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.228+0000 7f00877fe640 1 -- 172.21.15.40:0/2622565967 >> v1:172.21.15.184:6789/0 conn(0x7f00880b5f20 legacy=0x7f00880d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.228+0000 7f00877fe640 1 -- 172.21.15.40:0/2622565967 >> v1:172.21.15.40:6789/0 conn(0x7f00880d3bc0 legacy=0x7f00880b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.228+0000 7f00877fe640 1 -- 172.21.15.40:0/2622565967 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00880d07d0 con 0x7f0088162880 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.228+0000 7f00877fe640 1 --2- 172.21.15.40:0/2622565967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0088162880 0x7f00880d2cb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00780097d0 tx=0x7f007802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d8009a04bf0ac47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f00867fc640 1 -- 172.21.15.40:0/2622565967 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078037070 con 0x7f0088162880 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f00867fc640 1 -- 172.21.15.40:0/2622565967 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007802ac30 con 0x7f0088162880 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f00867fc640 1 -- 172.21.15.40:0/2622565967 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800019f0 con 0x7f0088162880 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f008dec8640 1 -- 172.21.15.40:0/2622565967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0088162880 msgr2=0x7f00880d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f008dec8640 1 --2- 172.21.15.40:0/2622565967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0088162880 0x7f00880d2cb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00780097d0 tx=0x7f007802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f008dec8640 1 -- 172.21.15.40:0/2622565967 shutdown_connections 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f008dec8640 1 --2- 172.21.15.40:0/2622565967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0088162880 0x7f00880d2cb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f008dec8640 1 -- 172.21.15.40:0/2622565967 >> 172.21.15.40:0/2622565967 conn(0x7f00880d4f60 msgr2=0x7f00880d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f008dec8640 1 -- 172.21.15.40:0/2622565967 shutdown_connections 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.229+0000 7f008dec8640 1 -- 172.21.15.40:0/2622565967 wait complete. 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.230+0000 7f008dec8640 1 Processor -- start 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f008dec8640 1 -- start start 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f008dec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00880b5f20 0x7f0088150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f008dec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00880d3bc0 0x7f00880ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f008dec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0088162880 0x7f00880b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f008dec8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0088162c60 con 0x7f00880b5f20 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f008dec8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00880d0d90 con 0x7f0088162880 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f008dec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0088162de0 con 0x7f00880d3bc0 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f00877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00880b5f20 0x7f0088150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.231+0000 7f00877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00880b5f20 0x7f0088150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57326/0 (socket says 172.21.15.40:57326) 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f00877fe640 1 -- 172.21.15.40:0/2031886852 learned_addr learned my addr 172.21.15.40:0/2031886852 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f0087fff640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0088162880 0x7f00880b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f0086ffd640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00880d3bc0 0x7f00880ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f00877fe640 1 -- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00880d3bc0 msgr2=0x7f00880ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f00877fe640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00880d3bc0 0x7f00880ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f00877fe640 1 -- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0088162880 msgr2=0x7f00880b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f00877fe640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0088162880 0x7f00880b9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f00877fe640 1 -- 172.21.15.40:0/2031886852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00880cedb0 con 0x7f00880b5f20 2024-04-26T21:42:23.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f00877fe640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00880b5f20 0x7f0088150440 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0078035040 tx=0x7f00780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f006ffff640 1 -- 172.21.15.40:0/2031886852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078037050 con 0x7f00880b5f20 2024-04-26T21:42:23.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f006ffff640 1 -- 172.21.15.40:0/2031886852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0078043070 con 0x7f00880b5f20 2024-04-26T21:42:23.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f006ffff640 1 -- 172.21.15.40:0/2031886852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078006e10 con 0x7f00880b5f20 2024-04-26T21:42:23.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00880cc340 con 0x7f00880b5f20 2024-04-26T21:42:23.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.232+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00880cc900 con 0x7f00880b5f20 2024-04-26T21:42:23.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.233+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00880694f0 con 0x7f00880b5f20 2024-04-26T21:42:23.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.234+0000 7f006ffff640 1 -- 172.21.15.40:0/2031886852 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f007803ca30 con 0x7f00880b5f20 2024-04-26T21:42:23.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.234+0000 7f006ffff640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f007406c8f0 0x7f007406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.234+0000 7f006ffff640 1 -- 172.21.15.40:0/2031886852 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f00780ac800 con 0x7f00880b5f20 2024-04-26T21:42:23.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.234+0000 7f0086ffd640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f007406c8f0 0x7f007406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.235+0000 7f0086ffd640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f007406c8f0 0x7f007406edb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f00880bad20 tx=0x7f007c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.239+0000 7f006ffff640 1 -- 172.21.15.40:0/2031886852 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f007807aab0 con 0x7f00880b5f20 2024-04-26T21:42:23.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.475+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f00880cf200 con 0x7f00880b5f20 2024-04-26T21:42:23.477 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:23.476+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:42:23.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.478+0000 7f006ffff640 1 -- 172.21.15.40:0/2031886852 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) ==== 161+0+0 (secure 0 0 0) 0x7f007807a3b0 con 0x7f00880b5f20 2024-04-26T21:42:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.481+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f007406c8f0 msgr2=0x7f007406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.481+0000 7f008dec8640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f007406c8f0 0x7f007406edb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f00880bad20 tx=0x7f007c009040 comp rx=0 tx=0).stop 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.481+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00880b5f20 msgr2=0x7f0088150440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.481+0000 7f008dec8640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00880b5f20 0x7f0088150440 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0078035040 tx=0x7f00780046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.482+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 shutdown_connections 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.482+0000 7f008dec8640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f007406c8f0 0x7f007406edb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.482+0000 7f008dec8640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0088162880 0x7f00880b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.482+0000 7f008dec8640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00880d3bc0 0x7f00880ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.482+0000 7f008dec8640 1 --2- 172.21.15.40:0/2031886852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00880b5f20 0x7f0088150440 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.482+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 >> 172.21.15.40:0/2031886852 conn(0x7f00880d4f60 msgr2=0x7f00880b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.482+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 shutdown_connections 2024-04-26T21:42:23.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.482+0000 7f008dec8640 1 -- 172.21.15.40:0/2031886852 wait complete. 2024-04-26T21:42:23.484 INFO:tasks.ceph.mds.a.smithi040.stderr: -4451> 2024-04-26T21:42:14.215+0000 7f84043d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:42:23.494 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata export - 2024-04-26T21:42:23.514 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:42:23.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb66200bf40 1 Processor -- start 2024-04-26T21:42:23.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb66200bf40 1 -- start start 2024-04-26T21:42:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb66200bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f1a80 0x563a51a0d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb66200bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 0x563a51a0b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb66200bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f4f30 0x563a51935720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb66200bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563a5197e2d0 con 0x563a519f4f30 2024-04-26T21:42:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb66200bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a518dde40 con 0x563a519f1a80 2024-04-26T21:42:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb66200bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563a5197e570 con 0x563a519f1e60 2024-04-26T21:42:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb65f5f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f4f30 0x563a51935720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb65f5f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f4f30 0x563a51935720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57360/0 (socket says 172.21.15.40:57360) 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb65f5f8640 1 -- 172.21.15.40:0/1116452214 learned_addr learned my addr 172.21.15.40:0/1116452214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.536+0000 7fb65e5f6640 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 0x563a51a0b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65edf7640 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f1a80 0x563a51a0d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65f5f8640 1 -- 172.21.15.40:0/1116452214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 msgr2=0x563a51a0b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 0x563a51a0b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65f5f8640 1 -- 172.21.15.40:0/1116452214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f1a80 msgr2=0x563a51a0d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f1a80 0x563a51a0d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65f5f8640 1 -- 172.21.15.40:0/1116452214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a51934880 con 0x563a519f4f30 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f4f30 0x563a51935720 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb658054040 tx=0x7fb658050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c560ac4388a8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65d5f4640 1 -- 172.21.15.40:0/1116452214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658055070 con 0x563a519f4f30 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65d5f4640 1 -- 172.21.15.40:0/1116452214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb658051c50 con 0x563a519f4f30 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb65d5f4640 1 -- 172.21.15.40:0/1116452214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658064440 con 0x563a519f4f30 2024-04-26T21:42:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb66200bf40 1 -- 172.21.15.40:0/1116452214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f4f30 msgr2=0x563a51935720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb66200bf40 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f4f30 0x563a51935720 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb658054040 tx=0x7fb658050000 comp rx=0 tx=0).stop 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb66200bf40 1 -- 172.21.15.40:0/1116452214 shutdown_connections 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb66200bf40 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f4f30 0x563a51935720 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb66200bf40 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 0x563a51a0b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb66200bf40 1 --2- 172.21.15.40:0/1116452214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f1a80 0x563a51a0d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb66200bf40 1 -- 172.21.15.40:0/1116452214 >> 172.21.15.40:0/1116452214 conn(0x563a51934ab0 msgr2=0x563a51a0c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.537+0000 7fb66200bf40 1 -- 172.21.15.40:0/1116452214 shutdown_connections 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.538+0000 7fb66200bf40 1 -- 172.21.15.40:0/1116452214 wait complete. 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.538+0000 7fb66200bf40 1 Processor -- start 2024-04-26T21:42:23.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.538+0000 7fb66200bf40 1 -- start start 2024-04-26T21:42:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb66200bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f1a80 0x563a51935590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb66200bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 0x563a51a09db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb66200bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f4f30 0x563a51a09160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb66200bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563a519f0890 con 0x563a519f1a80 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb66200bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a519f0640 con 0x563a519f4f30 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb66200bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563a519ebc40 con 0x563a519f1e60 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f1a80 0x563a51935590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f1a80 0x563a51935590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57372/0 (socket says 172.21.15.40:57372) 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 learned_addr learned my addr 172.21.15.40:0/111581108 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f4f30 0x563a51a09160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65e5f6640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 0x563a51a09db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 msgr2=0x563a51a09db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a519f1e60 0x563a51a09db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f4f30 msgr2=0x563a51a09160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a519f4f30 0x563a51a09160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a519f46e0 con 0x563a519f1a80 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.539+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f1a80 0x563a51935590 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fb65000dfa0 tx=0x7fb65000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.540+0000 7fb647fff640 1 -- 172.21.15.40:0/111581108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500032d0 con 0x563a519f1a80 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.540+0000 7fb647fff640 1 -- 172.21.15.40:0/111581108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb650016070 con 0x563a519f1a80 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.540+0000 7fb647fff640 1 -- 172.21.15.40:0/111581108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500116a0 con 0x563a519f1a80 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.540+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563a519f49c0 con 0x563a519f1a80 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.540+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a519ecb40 con 0x563a519f1a80 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.541+0000 7fb647fff640 1 -- 172.21.15.40:0/111581108 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb650011840 con 0x563a519f1a80 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.541+0000 7fb647fff640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb64c06b2d0 0x7fb64c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.542+0000 7fb647fff640 1 -- 172.21.15.40:0/111581108 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fb65008a7f0 con 0x563a519f1a80 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stdout:���� 2024-04-26T21:42:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: �� 2024-04-26T21:42:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.542+0000 7fb65e5f6640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb64c06b2d0 0x7fb64c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.542+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563a51935ad0 0x563a519ed850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.542+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.0 13:00000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x563a51935ad0 2024-04-26T21:42:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.542+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563a51935ad0 0x563a519ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.542+0000 7fb65e5f6640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb64c06b2d0 0x7fb64c06d790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x563a51a10210 tx=0x7fb654005ae0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.542+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563a51935ad0 0x563a519ed850 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.545+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.545+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7fb65805a040 0x7fb65805a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.545+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:08000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb65805a040 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.545+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7fb65805a040 0x7fb65805a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.545+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7fb65805a040 0x7fb65805a930 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.546+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v94'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7fb65000e040 con 0x7fb65805a040 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.546+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fb6500586e0 0x7fb650057340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.546+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- osd_op(unknown.0.0:3 13.8 13:10000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb6500586e0 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.546+0000 7fb65e5f6640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fb6500586e0 0x7fb650057340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.546+0000 7fb65e5f6640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fb6500586e0 0x7fb650057340 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.546+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.2 v2:172.21.15.40:6811/3280366756 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb654016840 con 0x7fb6500586e0 2024-04-26T21:42:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.546+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:4 13.18 13:18000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x563a51935ad0 2024-04-26T21:42:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.547+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.547+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb658064b80 0x7fb6580508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.547+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:5 13.4 13:20000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658064b80 2024-04-26T21:42:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.547+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb658064b80 0x7fb6580508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.547+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb658064b80 0x7fb6580508f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.549+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.549+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fb658075920 0x7fb6580743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.550+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:6 13.14 13:28000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658075920 2024-04-26T21:42:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.550+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fb658075920 0x7fb6580743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.550+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fb658075920 0x7fb6580743f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.552+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7fb65000e040 con 0x7fb658075920 2024-04-26T21:42:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.552+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.c 13:30000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb65805a040 2024-04-26T21:42:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.553+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb65000e040 con 0x7fb65805a040 2024-04-26T21:42:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.553+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:8 13.1c 13:38000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658064b80 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.553+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.553+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7fb6580723d0 0x7fb658072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.553+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:9 13.2 13:40000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb6580723d0 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.553+0000 7fb65e5f6640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7fb6580723d0 0x7fb658072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.554+0000 7fb65e5f6640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7fb6580723d0 0x7fb658072810 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.554+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb654016840 con 0x7fb6580723d0 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.554+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:10 13.12 13:48000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658075920 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.554+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb65000e040 con 0x7fb658075920 2024-04-26T21:42:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.554+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- osd_op(unknown.0.0:11 13.a 13:50000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb6500586e0 2024-04-26T21:42:23.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.555+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.2 v2:172.21.15.40:6811/3280366756 2 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb654016840 con 0x7fb6500586e0 2024-04-26T21:42:23.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.555+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.1a 13:58000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb65805a040 2024-04-26T21:42:23.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.555+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.555+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7fb650060b20 0x7fb65005d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.555+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:13 13.6 13:60000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb650060b20 2024-04-26T21:42:23.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.555+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7fb650060b20 0x7fb65005d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.556+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7fb650060b20 0x7fb65005d910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.558+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'3 uv3 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.558+0000 7fb65f5f8640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb658086440 0x7fb6580868a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.558+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- osd_op(unknown.0.0:14 13.16 13:68000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658086440 2024-04-26T21:42:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.558+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb658086440 0x7fb6580868a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.558+0000 7fb65edf7640 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb658086440 0x7fb6580868a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.560+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.3 v2:172.21.15.40:6824/4238362919 1 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb650076040 con 0x7fb658086440 2024-04-26T21:42:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.560+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:15 13.e 13:70000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x563a51935ad0 2024-04-26T21:42:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.560+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.560+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:16 13.1e 13:78000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb650060b20 2024-04-26T21:42:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.561+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.561+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:17 13.1 13:80000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658064b80 2024-04-26T21:42:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.561+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.561+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.11 13:88000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb65805a040 2024-04-26T21:42:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.561+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.561+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:19 13.9 13:90000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb6580723d0 2024-04-26T21:42:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.562+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb654016840 con 0x7fb6580723d0 2024-04-26T21:42:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.562+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- osd_op(unknown.0.0:20 13.19 13:98000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658086440 2024-04-26T21:42:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.562+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.3 v2:172.21.15.40:6824/4238362919 2 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb650076040 con 0x7fb658086440 2024-04-26T21:42:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.562+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:21 13.5 13:a0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb650060b20 2024-04-26T21:42:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.562+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.562+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:22 13.15 13:a8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb650060b20 2024-04-26T21:42:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.563+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v93'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.563+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:23 13.d 13:b0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb650060b20 2024-04-26T21:42:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.563+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v93'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.563+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:24 13.1d 13:b8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x563a51935ad0 2024-04-26T21:42:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.563+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.563+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:25 13.3 13:c0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658075920 2024-04-26T21:42:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.564+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.564+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:26 13.13 13:c8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x563a51935ad0 2024-04-26T21:42:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.564+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.564+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:27 13.b 13:d0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb6580723d0 2024-04-26T21:42:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.565+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7fb654016840 con 0x7fb6580723d0 2024-04-26T21:42:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.565+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:28 13.1b 13:d8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb650060b20 2024-04-26T21:42:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.565+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.565+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:29 13.7 13:e0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x563a51935ad0 2024-04-26T21:42:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.565+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.565+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:30 13.17 13:e8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb6580723d0 2024-04-26T21:42:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.565+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb654016840 con 0x7fb6580723d0 2024-04-26T21:42:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.566+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:31 13.f 13:f0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658064b80 2024-04-26T21:42:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.566+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v93'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.566+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:32 13.1f 13:f8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563a51a0a2f0 con 0x7fb658064b80 2024-04-26T21:42:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.566+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.566+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a519edd90 con 0x7fb65805a040 2024-04-26T21:42:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.567+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(33 601.00000000 [read 0~0] v0'0 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.567+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:34 13.10 13:09401a81:::601.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a519ee130 con 0x7fb65805a040 2024-04-26T21:42:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.567+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(34 601.00000000 [getxattrs out=124b] v0'0 uv4 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.567+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:35 13.10 13:09401a81:::601.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a519f32f0 con 0x7fb65805a040 2024-04-26T21:42:23.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.567+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(35 601.00000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.567+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:36 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a519f3690 con 0x7fb65805a040 2024-04-26T21:42:23.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.567+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(36 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.567+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:37 13.10 13:0bd6d154:::602.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a519f3a80 con 0x7fb65805a040 2024-04-26T21:42:23.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(37 602.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:38 13.10 13:0bd6d154:::602.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a519f3e70 con 0x7fb65805a040 2024-04-26T21:42:23.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(38 602.00000000 [getxattrs out=124b] v0'0 uv6 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:39 13.10 13:0bd6d154:::602.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a519f4260 con 0x7fb65805a040 2024-04-26T21:42:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(39 602.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:40 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf5d0 con 0x7fb65805a040 2024-04-26T21:42:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(40 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:41 13.10 13:0cf6ad0f:::10000000000.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(41 10000000000.00000000 [read 0~0] v0'0 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.568+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:42 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.569+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(42 10000000000.00000000 [getxattrs out=124b] v0'0 uv10 ondisk = 0) ==== 164+0+124 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.569+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:43 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.569+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(43 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.569+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:44 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf5d0 con 0x7fb65805a040 2024-04-26T21:42:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.569+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals out=586b] v0'0 uv10 ondisk = 0) ==== 164+0+586 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: ��*$- 601.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray1 ���� �� ((88���� �� ��*$- 602.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray2 ���� �� ((88���� �� ��2,510000000000.00000000�������������������!�������������������������� ���~_layout��������_parent>8subdir ���� ��  (�,f��s'(�,f��s'8`�,f���(8`�,f���(���� ��OI sixmegs_head1i"��,f���(���$@@`���������,f���(�,f��s'(8`�,f���(8`�,f���( 2024-04-26T21:42:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.569+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:45 13.10 13:0d82a743:::600.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.569+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(45 600.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.569+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:46 13.10 13:0d82a743:::600.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(46 600.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:47 13.10 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(47 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:48 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf5d0 con 0x7fb65805a040 2024-04-26T21:42:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(48 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:49 13.10 13:0d89b25e:::603.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(49 603.00000000 [read 0~0] v0'0 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:50 13.10 13:0d89b25e:::603.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(50 603.00000000 [getxattrs out=124b] v0'0 uv8 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.570+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:51 13.10 13:0d89b25e:::603.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb65805a040 2024-04-26T21:42:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.571+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(51 603.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.571+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:52 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf5d0 con 0x7fb65805a040 2024-04-26T21:42:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.571+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(52 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb650076040 con 0x7fb65805a040 2024-04-26T21:42:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.571+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:53 13.14 13:287034c4:::1.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.571+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(53 1.00000000.inode [read 0~582 out=582b] v0'0 uv15 ondisk = 0) ==== 160+0+582 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.582 INFO:teuthology.orchestra.run.smithi040.stdout:���������,f��s'����security.selinux%unconfined_u:object_r:unlabeled_t:s0������������ �� ��*$- 600.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray0 ���� �� ((88���� �� ��*$- 603.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray3 ���� �� ((88���� �� ��.(11.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)��,f�z �C@@���������,f�z (�,f�z 8`�,f���(8�,f 2024-04-26T21:42:23.582 INFO:teuthology.orchestra.run.smithi040.stdout:h� �,f 2024-04-26T21:42:23.582 INFO:teuthology.orchestra.run.smithi040.stdout:h� ����NH�,f 2024-04-26T21:42:23.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.571+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:54 13.14 13:287034c4:::1.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.572+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(54 1.00000000.inode [getxattrs out=4b] v0'0 uv15 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.583 INFO:teuthology.orchestra.run.smithi040.stdout:h� ������������ �� 2024-04-26T21:42:23.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.572+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:55 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.572+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(55 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.583 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� 2024-04-26T21:42:23.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.572+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:56 13.14 13:287034c4:::1.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf5d0 con 0x7fb658075920 2024-04-26T21:42:23.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.572+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(56 1.00000000.inode [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.572+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:57 13.14 13:292cf221:::200.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.572+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(57 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.573+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:58 13.14 13:292cf221:::200.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.573+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(58 200.00000000 [getxattrs out=4b] v0'0 uv19 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.586 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ��*$- 200.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@�>@�>@�F@@@ ���� �� 2024-04-26T21:42:23.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.573+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:59 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.573+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(59 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.586 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� 2024-04-26T21:42:23.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.573+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:60 13.14 13:292cf221:::200.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf5d0 con 0x7fb658075920 2024-04-26T21:42:23.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.573+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(60 200.00000000 [omap-get-vals out=5b] v0'0 uv19 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.573+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:61 13.14 13:2e2fa760:::200.00000001:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.574+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(61 200.00000001 [read 0~18075 out=18075b] v0'0 uv16 ondisk = 0) ==== 156+0+18075 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.588 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ��*$- 200.00000001�������������������!�������������������������� ���F�F�F�F��XS&YA1e@��XS&YA1h^T�,f �� 2 ((88 2 ( 2024-04-26T21:42:23.588 INFO:teuthology.orchestra.run.smithi040.stdout:( 2024-04-26T21:42:23.588 INFO:teuthology.orchestra.run.smithi040.stdout:8 2024-04-26T21:42:23.588 INFO:teuthology.orchestra.run.smithi040.stdout:�,fg;� 8 2024-04-26T21:42:23.588 INFO:teuthology.orchestra.run.smithi040.stdout:�,fg;� ��������@0@��XS&YA1�� 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:��,f�3��2�FN��� entity_id0hostname smithi040kernel_version5.14.0-437.el9.x86_64root/�{ ����@��XS&YA1����,fkA�n���������@��XS&YA1�,fv�setattr� C����������,f滰�C@@���������,f 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:h� (8�,f滰8�,f 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:h� �,f 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:h� ����NH�,f 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:h� �����������������X@��XS&YA1���,f?�� mkdir�p (�,f�z (�,f�z 8�,f�z 8�,f�z J 8subdir����������,f�z �A�$�����������������,f�z �,f�z (88���������,f�z ����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������2 ((88  C����������,f�z �C@@���������,f�z (�,f�z 8�,f�z 8�,f 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:h� �,f 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:h� ����NH�,f 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:h� ��������������������@��XS&YA1��,f��'openc�p (�,f�z (�,f�z 8�,f��s'8�,f�z J 8subdir����������,f��s'�A�$�����������������,f��s'�,f�z (�,f��s'8�,f��s'8�,f��s'���������,f�z ����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (�,f��s'(�,f��s'8�,f��s'8�,f��s'm [sixmegs����������,f��s'���$@@���������,f��s'�,f��s'�@(88���������,f��s'����security.selinux%unconfined_u:object_r:unlabeled_t:s0����������!!���������@��XS&YA1 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:�,f�( 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:cap update�p (�,f�z (�,f�z 8 �,f� �'8�,f�z J 8subdir����������,f��s'�A�$�����������������,f��s'�,f�z (�,f��s'8 �,f� �'8 �,f� �'���������,f�z ����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (�,f��s'(�,f��s'8 �,f� �'8 �,f� �'m [sixmegs����������,f� �'���$@@ ���������,f� �'�,f��s'��(8 �,f� �'8 �,f� �'���������,f��s'����security.selinux%unconfined_u:object_r:unlabeled_t:s0������������������@��XS&YA1 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:�,fe��( 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:cap update�p  (�,f�z (�,f�z 8P�,fK2w(8�,f�z J 8subdir����������,f��s'�A�$�����������������,f��s'�,f�z (�,f��s'8P�,fK2w(8P�,fK2w(���������,f�z ����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (�,f��s'(�,f��s'8P�,fK2w(8P�,fK2w(m [sixmegs����������,fK2w(���$@@P���������,fK2w(�,f��s'��(8P�,fK2w(8P�,fK2w(���������,f��s'����security.selinux%unconfined_u:object_r:unlabeled_t:s0������������������ @��XS&YA1] 2024-04-26T21:42:23.589 INFO:teuthology.orchestra.run.smithi040.stdout:S 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:I 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:�,fB� 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:cap update 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:p  (�,f�z (�,f�z 8`�,f���(8`�,f���(J 8subdir�������� 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:��,f��s'�A�$�����������������,f��s'�,f�z (�,f��s'8`�,f���(8`�,f���( 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:���������,f�z ����security.selinux%unconfined_u:object_r:unlabeled_t:s0���������  (�,f��s'(�,f��s'8`�,f���(8`�,f���(m [sixmegs����������,f���(���$@@`���������,f���(�,f��s'�(8`�,f���(8`�,f���(���������,f��s'����security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� C����������,f�z �C@@���������,f�z (�,f�z 8`�,f���(8�,f 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:h� �,f 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:h� ����NH�,f 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:h� �����������������)@��XS&YA1�� 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:��,fh=9 �2�FN������3@��XS&YA1 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:23.590 INFO:teuthology.orchestra.run.smithi040.stdout:�,fi�� check_inode_max_size� p  (�,f�z (�,f�z 8`�,f���(8`�,f���(J 8subdir�������� ��,f��s'�A�$�����������������,f��s'�,f�z (�,f��s'8`�,f���(8`�,f���( ���������,f�z ����security.selinux%unconfined_u:object_r:unlabeled_t:s0���������  (�,f��s'(�,f��s'8`�,f���(8`�,f���(� 5sixmegs�������� 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:��,f���(���$@@`���������,f���(�,f��s'(8`�,f���(8`�,f���( 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:���������,f��s'����security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� 5sixmegs�������� 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:��,f���(���$@@`���������,f���(�,f��s'(8`�,f���(8`�,f���( 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:���������,f��s'����security.selinux%unconfined_u:object_r:unlabeled_t:s0����������������34@��XS&YA18.$�a>@��XS&YA1h^T�,fôb2  (�,f�z (�,f�z 8`�,f���(8`�,f���( 2 ( 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:( 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:8 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:�,fg;� 8 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:�,fg;� ��������@�>@��XS&YA1^TJ�,f �scatter_writebehind2 (�,f�z (�,f�z 8`�,f���(8`�,f���(  C����������,f�z �C@@���������,f�z (�,f�z 8`�,f���(8�,f 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:h� �,f 2024-04-26T21:42:23.591 INFO:teuthology.orchestra.run.smithi040.stdout:h� ����NH�,f 2024-04-26T21:42:23.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.574+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:62 13.14 13:2e2fa760:::200.00000001:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.574+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(62 200.00000001 [getxattrs out=4b] v0'0 uv16 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.592 INFO:teuthology.orchestra.run.smithi040.stdout:h� ����������������)B@���� �� 2024-04-26T21:42:23.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.574+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:63 13.14 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.574+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(63 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.593 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� 2024-04-26T21:42:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.574+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:64 13.14 13:2e2fa760:::200.00000001:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf5d0 con 0x7fb658075920 2024-04-26T21:42:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.574+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(64 200.00000001 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.574+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:65 13.14 13:2f55791f:::606.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.575+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(65 606.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.575+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:66 13.14 13:2f55791f:::606.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.575+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(66 606.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.575+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:67 13.14 13:2f55791f:::606.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658075920 2024-04-26T21:42:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.575+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(67 606.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.575+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:68 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf5d0 con 0x7fb658075920 2024-04-26T21:42:23.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.575+0000 7fb65edf7640 1 -- 172.21.15.40:0/111581108 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(68 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb650076040 con 0x7fb658075920 2024-04-26T21:42:23.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.575+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:69 13.1c 13:3ed09add:::607.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae5c0 con 0x7fb658064b80 2024-04-26T21:42:23.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.576+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(69 607.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.576+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:70 13.1c 13:3ed09add:::607.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aae960 con 0x7fb658064b80 2024-04-26T21:42:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.577+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(70 607.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.577+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:71 13.1c 13:3ed09add:::607.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaed00 con 0x7fb658064b80 2024-04-26T21:42:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.577+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(71 607.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.577+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:72 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51ab00b0 con 0x7fb658064b80 2024-04-26T21:42:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.578+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(72 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.578+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.6 13:654134d2:::mds0_openfiles.0:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf0a0 con 0x7fb650060b20 2024-04-26T21:42:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.578+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(73 mds0_openfiles.0 [read 0~0] v0'0 uv3 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.578+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.6 13:654134d2:::mds0_openfiles.0:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.599 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ��*$- 606.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray6 ���� �� ((88���� �� ��*$- 607.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray7 ���� �� ((88���� �� ��.(1mds0_openfiles.0�������������������!�������������������������� �� 2024-04-26T21:42:23.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.578+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(74 mds0_openfiles.0 [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.578+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.579+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(75 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv3 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.579+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.579+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(76 mds0_openfiles.0 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.579+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:77 13.e 13:717a0223:::608.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.579+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(77 608.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.579+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:78 13.e 13:717a0223:::608.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.580+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(78 608.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.580+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:79 13.e 13:717a0223:::608.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.580+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(79 608.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.580+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:80 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.580+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(80 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.580+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.15 13:a93a17c2:::604.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.581+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(81 604.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.581+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.15 13:a93a17c2:::604.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.581+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(82 604.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.581+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.15 13:a93a17c2:::604.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.581+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(83 604.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.581+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.582+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(84 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.582+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:85 13.15 13:aa448500:::500.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.582+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(85 500.00000000 [read 0~90 out=90b] v0'0 uv3 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.582+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:86 13.15 13:aa448500:::500.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.582+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(86 500.00000000 [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.605 INFO:teuthology.orchestra.run.smithi040.stdout:���� ��4.*ceph fs volume v011 ���� �� ��*$- 608.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray8 ���� �� ((88���� �� ��*$- 604.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray4 ���� �� ((88���� �� ��*$- 500.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@@@@@@ ���� �� 2024-04-26T21:42:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.582+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:87 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.606 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� 2024-04-26T21:42:23.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.583+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(87 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.583+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:88 13.15 13:aa448500:::500.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.583+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(88 500.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.583+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:89 13.d 13:b50e409b:::mds_snaptable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.583+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(89 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.583+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:90 13.d 13:b50e409b:::mds_snaptable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.608 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ��+%. mds_snaptable�������������������!�������������������������� ��`ZFF4���� �� 2024-04-26T21:42:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.584+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(90 mds_snaptable [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.584+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:91 13.d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.584+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 21 ==== osd_op_reply(91 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.609 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� 2024-04-26T21:42:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.584+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:92 13.d 13:b50e409b:::mds_snaptable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb650060b20 2024-04-26T21:42:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.584+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.4 v2:172.21.15.184:6800/2829090873 22 ==== osd_op_reply(92 mds_snaptable [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7fb658064440 con 0x7fb650060b20 2024-04-26T21:42:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.584+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.1d 13:b871830b:::605.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.584+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(93 605.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.584+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.1d 13:b871830b:::605.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.585+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(94 605.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.585+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.1d 13:b871830b:::605.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.585+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(95 605.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.585+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.585+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(96 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.585+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.586+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.586+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.613 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ��*$- 605.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray5 ���� �� ((88���� �� ��+%. mds0_inotable�������������������!�������������������������� ��<6""��������� �� 2024-04-26T21:42:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.586+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(98 mds0_inotable [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.586+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.586+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(99 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.614 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� 2024-04-26T21:42:23.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.586+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.587+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(100 mds0_inotable [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.587+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:101 13.13 13:cd5a64a3:::100.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.587+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(101 100.00000000 [read 0~0] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.587+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:102 13.13 13:cd5a64a3:::100.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.587+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(102 100.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.616 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ��*$- 100.00000000�������������������!�������������������������� �� 2024-04-26T21:42:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.587+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.587+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(103 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.617 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ( 2024-04-26T21:42:23.617 INFO:teuthology.orchestra.run.smithi040.stdout:( 2024-04-26T21:42:23.617 INFO:teuthology.orchestra.run.smithi040.stdout:8 2024-04-26T21:42:23.617 INFO:teuthology.orchestra.run.smithi040.stdout:�,fg;� 8 2024-04-26T21:42:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.587+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.588+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(104 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.618 INFO:teuthology.orchestra.run.smithi040.stdout:�,fg;� ���� ��PJ 2024-04-26T21:42:23.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.588+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:105 13.b 13:d0630e4c:::mds0_sessionmap:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf8d0 con 0x7fb6580723d0 2024-04-26T21:42:23.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.588+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.5 v2:172.21.15.184:6808/304953510 5 ==== osd_op_reply(105 mds0_sessionmap [read 0~0] v0'0 uv3 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb654016840 con 0x7fb6580723d0 2024-04-26T21:42:23.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.588+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:106 13.b 13:d0630e4c:::mds0_sessionmap:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aafcc0 con 0x7fb6580723d0 2024-04-26T21:42:23.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.588+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.5 v2:172.21.15.184:6808/304953510 6 ==== osd_op_reply(106 mds0_sessionmap [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 159+0+4 (crc 0 0 0) 0x7fb654016840 con 0x7fb6580723d0 2024-04-26T21:42:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: stray0_head�i����,f<� @A�����������������,f<� (8�,f<� 8�,f<� �,f<� ������������ stray1_head�i����,fd�� @A�����������������,fd�� (8�,fd�� 8�,fd�� �,fd�� ������������ stray2_head�i����,f�t� @A�����������������,f�t� (8�,f�t� 8�,f�t� �,f�t� ������������ stray3_head�i����,fB�� @A�����������������,fB�� (8�,fB�� 8�,fB�� �,fB�� ������������ stray4_head�i����,f��� @A�����������������,f��� (8�,f��� 8�,f��� �,f��� ������������ stray5_head�i����,f�9� @A�����������������,f�9� (8�,f�9� 8�,f�9� �,f�9� ������������ stray6_head�i����,f��� @A�����������������,f��� (8�,f��� 8�,f��� �,f��� ������������ stray7_head�i����,f�E� @A�����������������,f�E� (8�,f�E� 8�,f�E� �,f�E� ������������ stray8_head�i����,f�� @A�����������������,f�� (8�,f�� 8�,f�� �,f�� ������������ stray9_head�i��� �,fg;� @A�����������������,fg;� (8�,fg;� 8�,fg;� �,fg;� ���������������� �� ��-'0mds0_sessionmap�������������������!�������������������������� �� 2024-04-26T21:42:23.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.588+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:107 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51ab0450 con 0x7fb6580723d0 2024-04-26T21:42:23.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.589+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.5 v2:172.21.15.184:6808/304953510 7 ==== osd_op_reply(107 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7fb654016840 con 0x7fb6580723d0 2024-04-26T21:42:23.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.589+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:108 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51ab1800 con 0x7fb6580723d0 2024-04-26T21:42:23.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.589+0000 7fb65e5f6640 1 -- 172.21.15.40:0/111581108 <== osd.5 v2:172.21.15.184:6808/304953510 8 ==== osd_op_reply(108 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7fb654016840 con 0x7fb6580723d0 2024-04-26T21:42:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.589+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.589+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(109 609.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.589+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.589+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(110 609.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.589+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:111 13.7 13:e0b41d06:::609.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.590+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(111 609.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.590+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:112 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x563a51935ad0 2024-04-26T21:42:23.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.590+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(112 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb658064440 con 0x563a51935ad0 2024-04-26T21:42:23.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.590+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.590+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(113 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.590+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.591+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(114 400.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.624 INFO:teuthology.orchestra.run.smithi040.stdout:���� ������ �� ��*$- 609.00000000�������������������!�������������������������� ���~_layout��������_parent>8 stray9 ���� �� ((88���� �� ��*$- 400.00000000�������������������!�������������������������� ��0*���� �� 2024-04-26T21:42:23.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.591+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.591+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(115 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.624 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� 2024-04-26T21:42:23.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.591+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.f 13:f1f69726:::400.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.591+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(116 400.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.591+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.592+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(117 100.00000000.inode [read 0~582 out=582b] v0'0 uv3 ondisk = 0) ==== 162+0+582 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.626 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ��0*3100.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)��,f��� @A�����������������,f��� ( 2024-04-26T21:42:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.592+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.626 INFO:teuthology.orchestra.run.smithi040.stdout:8 �,fg;� 8 �,fg;� �,f��� ����NH������������ �� 2024-04-26T21:42:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.592+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(118 100.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 162+0+4 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.592+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.627 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� 2024-04-26T21:42:23.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.592+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(119 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.592+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.593+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(120 100.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.628 INFO:teuthology.orchestra.run.smithi040.stdout:���� �� ��("+ 2024-04-26T21:42:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.593+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.593+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(121 1.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.593+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.593+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(122 1.00000000 [getxattrs out=92b] v0'0 uv6 ondisk = 0) ==== 154+0+92 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.593+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.593+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(123 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.593+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563a51aaf490 con 0x7fb658064b80 2024-04-26T21:42:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(124 1.00000000 [omap-get-vals out=585b] v0'0 uv6 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x7fb658064440 con 0x7fb658064b80 2024-04-26T21:42:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fb658075920 msgr2=0x7fb6580743f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fb658075920 0x7fb6580743f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7fb65805a040 msgr2=0x7fb65805a930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7fb65805a040 0x7fb65805a930 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fb6500586e0 msgr2=0x7fb650057340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7fb6500586e0 0x7fb650057340 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb658086440 msgr2=0x7fb6580868a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb658086440 0x7fb6580868a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7fb650060b20 msgr2=0x7fb65005d910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7fb650060b20 0x7fb65005d910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 reap_dead start 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7fb6580723d0 msgr2=0x7fb658072810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7fb6580723d0 0x7fb658072810 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb658064b80 msgr2=0x7fb6580508f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb658064b80 0x7fb6580508f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563a51935ad0 msgr2=0x563a519ed850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563a51935ad0 0x563a519ed850 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb64c06b2d0 msgr2=0x7fb64c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb64c06b2d0 0x7fb64c06d790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x563a51a10210 tx=0x7fb654005ae0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f1a80 msgr2=0x563a51935590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb65f5f8640 1 -- 172.21.15.40:0/111581108 reap_dead start 2024-04-26T21:42:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f1a80 0x563a51935590 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fb65000dfa0 tx=0x7fb65000b5b0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 shutdown_connections 2024-04-26T21:42:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 --2- 172.21.15.40:0/111581108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a519f1a80 0x563a51935590 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 >> 172.21.15.40:0/111581108 conn(0x563a51934ab0 msgr2=0x563a519eb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 shutdown_connections 2024-04-26T21:42:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.594+0000 7fb66200bf40 1 -- 172.21.15.40:0/111581108 wait complete. 2024-04-26T21:42:23.634 INFO:teuthology.orchestra.run.smithi040.stdout:1.00000000�������������������!�������������������������� ��d^_layout@@_parent ���� ��  (�,f�z (�,f�z 8`�,f���(8`�,f���(���� ��NH subdir_head1i"��,f��s'�A�$�����������������,f��s'�,f�z (�,f��s'8`�,f���(8`�,f���( ���������,f�z ����security.selinux%unconfined_u:object_r:unlabeled_t:s0������������ �� �� 2024-04-26T21:42:23.634 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata ls 2024-04-26T21:42:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.714+0000 7f4bdef97f40 1 Processor -- start 2024-04-26T21:42:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.714+0000 7f4bdef97f40 1 -- start start 2024-04-26T21:42:23.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdef97f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bc40030f0 0x5621960c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdef97f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 0x5621960c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdef97f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562196028d30 0x5621960d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdef97f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562195fa4e40 con 0x7f4bc40030f0 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdef97f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562196046660 con 0x562196028d30 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdef97f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5621960d2c00 con 0x562195fff670 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bc40030f0 0x5621960c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bc40030f0 0x5621960c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57388/0 (socket says 172.21.15.40:57388) 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 -- 172.21.15.40:0/3702006649 learned_addr learned my addr 172.21.15.40:0/3702006649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdc290640 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 0x5621960c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562196028d30 0x5621960d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 -- 172.21.15.40:0/3702006649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 msgr2=0x5621960c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 0x5621960c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 -- 172.21.15.40:0/3702006649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562196028d30 msgr2=0x5621960d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562196028d30 0x5621960d3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 -- 172.21.15.40:0/3702006649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562195ffe5f0 con 0x7f4bc40030f0 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bc40030f0 0x5621960c5b30 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009980 tx=0x7f4bd4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ff6342eda0bf2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdb28e640 1 -- 172.21.15.40:0/3702006649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4019020 con 0x7f4bc40030f0 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdb28e640 1 -- 172.21.15.40:0/3702006649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd4008a30 con 0x7f4bc40030f0 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.715+0000 7f4bdb28e640 1 -- 172.21.15.40:0/3702006649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4008de0 con 0x7f4bc40030f0 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3702006649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bc40030f0 msgr2=0x5621960c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bc40030f0 0x5621960c5b30 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009980 tx=0x7f4bd4005a00 comp rx=0 tx=0).stop 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3702006649 shutdown_connections 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562196028d30 0x5621960d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 0x5621960c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3702006649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bc40030f0 0x5621960c5b30 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3702006649 >> 172.21.15.40:0/3702006649 conn(0x5621960278a0 msgr2=0x562195fffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3702006649 shutdown_connections 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3702006649 wait complete. 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 Processor -- start 2024-04-26T21:42:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.716+0000 7f4bdef97f40 1 -- start start 2024-04-26T21:42:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdef97f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 0x5621960bf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdef97f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562196028d30 0x5621960d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdef97f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bc40030f0 0x56219602b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdef97f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562195ffe310 con 0x562196028d30 2024-04-26T21:42:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdef97f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562195ffe030 con 0x7f4bc40030f0 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdef97f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5621960d2890 con 0x562195fff670 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdc290640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562196028d30 0x5621960d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdc290640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562196028d30 0x5621960d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57402/0 (socket says 172.21.15.40:57402) 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 learned_addr learned my addr 172.21.15.40:0/3025315695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 0x5621960bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.717+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bc40030f0 0x56219602b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 msgr2=0x5621960bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bdc290640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562195fff670 0x5621960bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bc40030f0 msgr2=0x56219602b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bdc290640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bc40030f0 0x56219602b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621960d4fc0 con 0x562196028d30 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bdc290640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562196028d30 0x5621960d5a60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0057040 tx=0x7f4bd0001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bda28c640 1 -- 172.21.15.40:0/3025315695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0000f70 con 0x562196028d30 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bda28c640 1 -- 172.21.15.40:0/3025315695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd0009040 con 0x562196028d30 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56219602bcb0 con 0x562196028d30 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bda28c640 1 -- 172.21.15.40:0/3025315695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd004dab0 con 0x562196028d30 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.718+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56219602bf90 con 0x562196028d30 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.719+0000 7f4bda28c640 1 -- 172.21.15.40:0/3025315695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bd005e040 con 0x562196028d30 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.720+0000 7f4bda28c640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4bc806b300 0x7f4bc806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.720+0000 7f4bda28c640 1 -- 172.21.15.40:0/3025315695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f4bd00ce960 con 0x562196028d30 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.720+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4bc806b300 0x7f4bc806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.720+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5621960d5fa0 0x5621960c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.720+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960d5fa0 2024-04-26T21:42:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.720+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5621960d5fa0 0x5621960c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.720+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4bc806b300 0x7f4bc806d7c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4005a40 tx=0x7f4bd4015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.721+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5621960d5fa0 0x5621960c27f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.721+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bcc0148a0 con 0x5621960d5fa0 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.721+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f4bcc014040 0x7f4bcc013960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.721+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bcc014040 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.721+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f4bcc014040 0x7f4bcc013960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.721+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f4bcc014040 0x7f4bcc013960 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.722+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v94'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f4bd4015470 con 0x7f4bcc014040 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.722+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f4bd402b040 0x7f4bd402a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.722+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bd402b040 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.722+0000 7f4bdc290640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f4bd402b040 0x7f4bd402a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.722+0000 7f4bdc290640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f4bd402b040 0x7f4bd402a410 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.723+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 <== osd.2 v2:172.21.15.40:6811/3280366756 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd0098d80 con 0x7f4bd402b040 2024-04-26T21:42:23.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.723+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960d5fa0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.723+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bcc0148a0 con 0x5621960d5fa0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.723+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562195ffc1c0 0x7f4bcc0128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.723+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x562195ffc1c0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.723+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562195ffc1c0 0x7f4bcc0128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.724+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562195ffc1c0 0x7f4bcc0128b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.724+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bcc0148a0 con 0x562195ffc1c0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.724+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562196047420 0x7f4bcc025860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.724+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x562196047420 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.724+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562196047420 0x7f4bcc025860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.724+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562196047420 0x7f4bcc025860 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.725+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f4bd4015470 con 0x562196047420 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.725+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bcc014040 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.725+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd4015470 con 0x7f4bcc014040 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.725+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x562195ffc1c0 2024-04-26T21:42:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.725+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4bcc0148a0 con 0x562195ffc1c0 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.725+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5621960cef70 0x7f4bcc024920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.725+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960cef70 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.726+0000 7f4bdc290640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5621960cef70 0x7f4bcc024920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.726+0000 7f4bdc290640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5621960cef70 0x7f4bcc024920 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.726+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd0098d80 con 0x5621960cef70 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.726+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x562196047420 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.726+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd4015470 con 0x562196047420 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.726+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bd402b040 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.727+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 <== osd.2 v2:172.21.15.40:6811/3280366756 2 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd0098d80 con 0x7f4bd402b040 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.727+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bcc014040 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.727+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd4015470 con 0x7f4bcc014040 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.727+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f4bd401e040 0x7f4bd403a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.727+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bd401e040 2024-04-26T21:42:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.727+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f4bd401e040 0x7f4bd403a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.728+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f4bd401e040 0x7f4bd403a910 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.728+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'3 uv3 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f4bcc0148a0 con 0x7f4bd401e040 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.728+0000 7f4bdd292640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7f4bc40022a0 0x7f4bcc036950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.728+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bc40022a0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.728+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7f4bc40022a0 0x7f4bcc036950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.728+0000 7f4bdca91640 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7f4bc40022a0 0x7f4bcc036950 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.729+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.3 v2:172.21.15.40:6824/4238362919 1 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd4015470 con 0x7f4bc40022a0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.729+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960d5fa0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.729+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4bcc0148a0 con 0x5621960d5fa0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.729+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bd401e040 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.729+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bcc0148a0 con 0x7f4bd401e040 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.729+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x562195ffc1c0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.730+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bcc0148a0 con 0x562195ffc1c0 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.730+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bcc014040 2024-04-26T21:42:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.730+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd4015470 con 0x7f4bcc014040 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.730+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960cef70 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.730+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd0098d80 con 0x5621960cef70 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.730+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bc40022a0 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.730+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.3 v2:172.21.15.40:6824/4238362919 2 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd4015470 con 0x7f4bc40022a0 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.730+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bd401e040 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.731+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bcc0148a0 con 0x7f4bd401e040 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.731+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bd401e040 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.731+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v93'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f4bcc0148a0 con 0x7f4bd401e040 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.731+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bd401e040 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.731+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v93'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4bcc0148a0 con 0x7f4bd401e040 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.731+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960d5fa0 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.732+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4bcc0148a0 con 0x5621960d5fa0 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.732+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x562196047420 2024-04-26T21:42:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.732+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd4015470 con 0x562196047420 2024-04-26T21:42:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.732+0000 7f4bdca91640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960d5fa0 2024-04-26T21:42:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.732+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f4bcc0148a0 con 0x5621960d5fa0 2024-04-26T21:42:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.732+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960cef70 2024-04-26T21:42:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.733+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f4bd0098d80 con 0x5621960cef70 2024-04-26T21:42:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.733+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x7f4bd401e040 2024-04-26T21:42:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.733+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bcc0148a0 con 0x7f4bd401e040 2024-04-26T21:42:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.733+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960d5fa0 2024-04-26T21:42:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.733+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4bcc0148a0 con 0x5621960d5fa0 2024-04-26T21:42:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.733+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x5621960cef70 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.734+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4bd0098d80 con 0x5621960cef70 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.734+0000 7f4bdc290640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x562195ffc1c0 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.734+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v93'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4bcc0148a0 con 0x562195ffc1c0 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.734+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x5621960cfff0 con 0x562195ffc1c0 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f4bcc0148a0 con 0x562195ffc1c0 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stdout:601.00000000 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stdout:602.00000000 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stdout:10000000000.00000000 2024-04-26T21:42:23.739 INFO:teuthology.orchestra.run.smithi040.stdout:600.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:603.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:1.00000000.inode 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:200.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:200.00000001 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:606.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:607.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:mds0_openfiles.0 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:608.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:604.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:500.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:mds_snaptable 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:605.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:mds0_inotable 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:100.00000000 2024-04-26T21:42:23.740 INFO:teuthology.orchestra.run.smithi040.stdout:mds0_sessionmap 2024-04-26T21:42:23.741 INFO:teuthology.orchestra.run.smithi040.stdout:609.00000000 2024-04-26T21:42:23.741 INFO:teuthology.orchestra.run.smithi040.stdout:400.00000000 2024-04-26T21:42:23.741 INFO:teuthology.orchestra.run.smithi040.stdout:100.00000000.inode 2024-04-26T21:42:23.741 INFO:teuthology.orchestra.run.smithi040.stdout:1.00000000 2024-04-26T21:42:23.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562196047420 msgr2=0x7f4bcc025860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562196047420 0x7f4bcc025860 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f4bcc014040 msgr2=0x7f4bcc013960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f4bcc014040 0x7f4bcc013960 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f4bd402b040 msgr2=0x7f4bd402a410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f4bd402b040 0x7f4bd402a410 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7f4bc40022a0 msgr2=0x7f4bcc036950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7f4bc40022a0 0x7f4bcc036950 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 reap_dead start 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f4bd401e040 msgr2=0x7f4bd403a910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7f4bd401e040 0x7f4bd403a910 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 reap_dead start 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5621960cef70 msgr2=0x7f4bcc024920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5621960cef70 0x7f4bcc024920 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562195ffc1c0 msgr2=0x7f4bcc0128b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562195ffc1c0 0x7f4bcc0128b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5621960d5fa0 msgr2=0x5621960c27f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5621960d5fa0 0x5621960c27f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4bc806b300 msgr2=0x7f4bc806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4bc806b300 0x7f4bc806d7c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4005a40 tx=0x7f4bd4015000 comp rx=0 tx=0).stop 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562196028d30 msgr2=0x5621960d5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 --2- 172.21.15.40:0/3025315695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562196028d30 0x5621960d5a60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0057040 tx=0x7f4bd0001bc0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdd292640 1 -- 172.21.15.40:0/3025315695 reap_dead start 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 shutdown_connections 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 >> 172.21.15.40:0/3025315695 conn(0x5621960278a0 msgr2=0x562195fffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 shutdown_connections 2024-04-26T21:42:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.735+0000 7f4bdef97f40 1 -- 172.21.15.40:0/3025315695 wait complete. 2024-04-26T21:42:23.745 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 10000000000.00000000 2024-04-26T21:42:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.825+0000 7f3b72023f40 1 Processor -- start 2024-04-26T21:42:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.825+0000 7f3b72023f40 1 -- start start 2024-04-26T21:42:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b72023f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a5930 0x564a2687f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b72023f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a267a8850 0x564a267f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b72023f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a2687c830 0x564a2687e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b72023f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564a2687cc10 con 0x564a267a5930 2024-04-26T21:42:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b72023f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564a2674ee40 con 0x564a267a8850 2024-04-26T21:42:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b72023f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564a2687d0f0 con 0x564a2687c830 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6fb1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a5930 0x564a2687f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6fb1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a5930 0x564a2687f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57418/0 (socket says 172.21.15.40:57418) 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6fb1d640 1 -- 172.21.15.40:0/3570310113 learned_addr learned my addr 172.21.15.40:0/3570310113 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6f31c640 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a267a8850 0x564a267f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b7031e640 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a2687c830 0x564a2687e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6fb1d640 1 -- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a2687c830 msgr2=0x564a2687e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6fb1d640 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a2687c830 0x564a2687e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6fb1d640 1 -- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a267a8850 msgr2=0x564a267f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6fb1d640 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a267a8850 0x564a267f0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6fb1d640 1 -- 172.21.15.40:0/3570310113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2686ff50 con 0x564a267a5930 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b7031e640 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a2687c830 0x564a2687e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.826+0000 7f3b6f31c640 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a267a8850 0x564a267f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b6fb1d640 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a5930 0x564a2687f970 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3b60001620 tx=0x7f3b60001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af7639403783427 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b6e31a640 1 -- 172.21.15.40:0/3570310113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60004520 con 0x564a267a5930 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b6e31a640 1 -- 172.21.15.40:0/3570310113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b600046c0 con 0x564a267a5930 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b6e31a640 1 -- 172.21.15.40:0/3570310113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60004a70 con 0x564a267a5930 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 -- 172.21.15.40:0/3570310113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a5930 msgr2=0x564a2687f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a5930 0x564a2687f970 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3b60001620 tx=0x7f3b60001ae0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 -- 172.21.15.40:0/3570310113 shutdown_connections 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a2687c830 0x564a2687e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a267a8850 0x564a267f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 --2- 172.21.15.40:0/3570310113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a5930 0x564a2687f970 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 -- 172.21.15.40:0/3570310113 >> 172.21.15.40:0/3570310113 conn(0x564a267a6330 msgr2=0x564a267efd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 -- 172.21.15.40:0/3570310113 shutdown_connections 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.827+0000 7f3b72023f40 1 -- 172.21.15.40:0/3570310113 wait complete. 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.828+0000 7f3b72023f40 1 Processor -- start 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.828+0000 7f3b72023f40 1 -- start start 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b72023f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a267a5930 0x564a26914d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b72023f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a8850 0x564a267d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b72023f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a2687c830 0x564a267f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b72023f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564a26870300 con 0x564a267a8850 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b72023f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564a267ee130 con 0x564a2687c830 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b72023f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564a26870550 con 0x564a267a5930 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a8850 0x564a267d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a8850 0x564a267d4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57420/0 (socket says 172.21.15.40:57420) 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 -- 172.21.15.40:0/3631558017 learned_addr learned my addr 172.21.15.40:0/3631558017 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6fb1d640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a267a5930 0x564a26914d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b7031e640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a2687c830 0x564a267f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 -- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a267a5930 msgr2=0x564a26914d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a267a5930 0x564a26914d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 -- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a2687c830 msgr2=0x564a267f0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a2687c830 0x564a267f0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 -- 172.21.15.40:0/3631558017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a26868ec0 con 0x564a267a8850 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6f31c640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a8850 0x564a267d4cd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3b64057040 tx=0x7f3b64002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6d318640 1 -- 172.21.15.40:0/3631558017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64064070 con 0x564a267a8850 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6d318640 1 -- 172.21.15.40:0/3631558017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64054ab0 con 0x564a267a8850 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564a268691a0 con 0x564a267a8850 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.829+0000 7f3b6d318640 1 -- 172.21.15.40:0/3631558017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64059040 con 0x564a267a8850 2024-04-26T21:42:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.830+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a26869760 con 0x564a267a8850 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.831+0000 7f3b6d318640 1 -- 172.21.15.40:0/3631558017 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b64067070 con 0x564a267a8850 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.831+0000 7f3b6d318640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b5c06b300 0x7f3b5c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.831+0000 7f3b6d318640 1 -- 172.21.15.40:0/3631558017 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f3b640ce2f0 con 0x564a267a8850 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.831+0000 7f3b72023f40 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a267d1a20 0x564a26918d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.831+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x564a267d5210 con 0x564a267d1a20 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b6fb1d640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b5c06b300 0x7f3b5c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b7031e640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a267d1a20 0x564a26918d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b7031e640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a267d1a20 0x564a26918d10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b6fb1d640 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b5c06b300 0x7f3b5c06d7c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3b60001620 tx=0x7f3b600021f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b7031e640 1 -- 172.21.15.40:0/3631558017 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7f3b68009a10 con 0x564a267d1a20 2024-04-26T21:42:23.837 INFO:teuthology.orchestra.run.smithi040.stdout:header (274 bytes) : 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000000 04 03 0c 01 00 00 0b 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000020 00 00 00 00 bc 1f 2c 66 d1 cc 73 27 01 00 00 00 |......,f..s'....| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000030 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000050 00 00 bc 1f 2c 66 d1 cc 73 27 01 00 00 00 00 00 |....,f..s'......| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000060 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:000000a0 00 00 00 00 00 00 00 00 bc 1f 2c 66 b4 b6 95 28 |..........,f...(| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-04-26T21:42:23.838 INFO:teuthology.orchestra.run.smithi040.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |`...............| 2024-04-26T21:42:23.839 INFO:teuthology.orchestra.run.smithi040.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.839 INFO:teuthology.orchestra.run.smithi040.stdout:000000e0 00 00 00 00 00 00 bc 1f 2c 66 b4 b6 95 28 00 00 |........,f...(..| 2024-04-26T21:42:23.839 INFO:teuthology.orchestra.run.smithi040.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.839 INFO:teuthology.orchestra.run.smithi040.stdout:* 2024-04-26T21:42:23.839 INFO:teuthology.orchestra.run.smithi040.stdout:00000110 00 00 |..| 2024-04-26T21:42:23.839 INFO:teuthology.orchestra.run.smithi040.stdout:00000112 2024-04-26T21:42:23.839 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:23.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a267d1a20 msgr2=0x564a26918d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a267d1a20 0x564a26918d10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b5c06b300 msgr2=0x7f3b5c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b5c06b300 0x7f3b5c06d7c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3b60001620 tx=0x7f3b600021f0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a8850 msgr2=0x564a267d4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 --2- 172.21.15.40:0/3631558017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a267a8850 0x564a267d4cd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3b64057040 tx=0x7f3b64002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b7031e640 1 -- 172.21.15.40:0/3631558017 reap_dead start 2024-04-26T21:42:23.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 shutdown_connections 2024-04-26T21:42:23.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 >> 172.21.15.40:0/3631558017 conn(0x564a267a6330 msgr2=0x564a267a4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 shutdown_connections 2024-04-26T21:42:23.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.832+0000 7f3b72023f40 1 -- 172.21.15.40:0/3631558017 wait complete. 2024-04-26T21:42:23.841 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 600.00000000 2024-04-26T21:42:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.922+0000 7f51cb725f40 1 Processor -- start 2024-04-26T21:42:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.922+0000 7f51cb725f40 1 -- start start 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51cb725f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba0e1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51cb725f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba09a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51cb725f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 0x556fba161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51cb725f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556fba040e40 con 0x7f51b40022d0 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51cb725f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556fba044e80 con 0x556fba0987c0 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51cb725f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556fba16ec00 con 0x556fba09ad10 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8906640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba0e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8906640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba0e1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57436/0 (socket says 172.21.15.40:57436) 2024-04-26T21:42:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8906640 1 -- 172.21.15.40:0/2473642870 learned_addr learned my addr 172.21.15.40:0/2473642870 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c9107640 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 0x556fba161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8105640 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba09a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8906640 1 -- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 msgr2=0x556fba161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8906640 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 0x556fba161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8906640 1 -- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 msgr2=0x556fba09a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8906640 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba09a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.923+0000 7f51c8906640 1 -- 172.21.15.40:0/2473642870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fba0c51d0 con 0x7f51b40022d0 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51c9107640 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 0x556fba161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51c8105640 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba09a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51c8906640 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba0e1000 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f51b8056040 tx=0x7f51b8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d81277f5ec2c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51c7103640 1 -- 172.21.15.40:0/2473642870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8063070 con 0x7f51b40022d0 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51c7103640 1 -- 172.21.15.40:0/2473642870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b80537c0 con 0x7f51b40022d0 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51c7103640 1 -- 172.21.15.40:0/2473642870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8058040 con 0x7f51b40022d0 2024-04-26T21:42:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 -- 172.21.15.40:0/2473642870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 msgr2=0x556fba0e1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba0e1000 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f51b8056040 tx=0x7f51b8051000 comp rx=0 tx=0).stop 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 -- 172.21.15.40:0/2473642870 shutdown_connections 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 0x556fba161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba09a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 --2- 172.21.15.40:0/2473642870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba0e1000 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 -- 172.21.15.40:0/2473642870 >> 172.21.15.40:0/2473642870 conn(0x556fba0c38a0 msgr2=0x556fba16d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 -- 172.21.15.40:0/2473642870 shutdown_connections 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.924+0000 7f51cb725f40 1 -- 172.21.15.40:0/2473642870 wait complete. 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.925+0000 7f51cb725f40 1 Processor -- start 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.925+0000 7f51cb725f40 1 -- start start 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51cb725f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba200150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51cb725f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 0x556fba16c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51cb725f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba16d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51cb725f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556fba0c5470 con 0x7f51b40022d0 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51cb725f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556fba16d320 con 0x556fba0987c0 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51cb725f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556fba16b610 con 0x556fba09ad10 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba16d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba16d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57444/0 (socket says 172.21.15.40:57444) 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 -- 172.21.15.40:0/961464638 learned_addr learned my addr 172.21.15.40:0/961464638 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c8906640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba200150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c8105640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 0x556fba16c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 -- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 msgr2=0x556fba16c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556fba09ad10 0x556fba16c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 -- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 msgr2=0x556fba200150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba200150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 -- 172.21.15.40:0/961464638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fba16cf40 con 0x7f51b40022d0 2024-04-26T21:42:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c8906640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fba0987c0 0x556fba200150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c9107640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba16d530 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x556fba0c3870 tx=0x7f51c0009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c6101640 1 -- 172.21.15.40:0/961464638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c001a020 con 0x7f51b40022d0 2024-04-26T21:42:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c6101640 1 -- 172.21.15.40:0/961464638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c0016440 con 0x7f51b40022d0 2024-04-26T21:42:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51c6101640 1 -- 172.21.15.40:0/961464638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0016830 con 0x7f51b40022d0 2024-04-26T21:42:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556fba1ff670 con 0x7f51b40022d0 2024-04-26T21:42:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.926+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fba16cc30 con 0x7f51b40022d0 2024-04-26T21:42:23.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.928+0000 7f51c6101640 1 -- 172.21.15.40:0/961464638 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51c001d020 con 0x7f51b40022d0 2024-04-26T21:42:23.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.928+0000 7f51c6101640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51a806b2d0 0x7f51a806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.928+0000 7f51c6101640 1 -- 172.21.15.40:0/961464638 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f51c008abb0 con 0x7f51b40022d0 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.928+0000 7f51cb725f40 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556fba1611c0 0x556fba168b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.928+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556fba1615a0 con 0x556fba1611c0 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.928+0000 7f51c8105640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556fba1611c0 0x556fba168b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.928+0000 7f51c8906640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51a806b2d0 0x7f51a806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51c8105640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556fba1611c0 0x556fba168b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51c8906640 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51a806b2d0 0x7f51a806d790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f51b8065000 tx=0x7f51b8061040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51c8105640 1 -- 172.21.15.40:0/961464638 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f51b0009710 con 0x556fba1611c0 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stdout:header (274 bytes) : 2024-04-26T21:42:23.934 INFO:teuthology.orchestra.run.smithi040.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:* 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:* 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:* 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:* 2024-04-26T21:42:23.935 INFO:teuthology.orchestra.run.smithi040.stdout:00000110 00 00 |..| 2024-04-26T21:42:23.936 INFO:teuthology.orchestra.run.smithi040.stdout:00000112 2024-04-26T21:42:23.936 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556fba1611c0 msgr2=0x556fba168b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556fba1611c0 0x556fba168b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51a806b2d0 msgr2=0x7f51a806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51a806b2d0 0x7f51a806d790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f51b8065000 tx=0x7f51b8061040 comp rx=0 tx=0).stop 2024-04-26T21:42:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 msgr2=0x556fba16d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 --2- 172.21.15.40:0/961464638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b40022d0 0x556fba16d530 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x556fba0c3870 tx=0x7f51c0009de0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51c9107640 1 -- 172.21.15.40:0/961464638 reap_dead start 2024-04-26T21:42:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 shutdown_connections 2024-04-26T21:42:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 >> 172.21.15.40:0/961464638 conn(0x556fba0c38a0 msgr2=0x556fba1657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 shutdown_connections 2024-04-26T21:42:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:23.929+0000 7f51cb725f40 1 -- 172.21.15.40:0/961464638 wait complete. 2024-04-26T21:42:23.938 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000.inode 2024-04-26T21:42:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.019+0000 7fd390c5cf40 1 Processor -- start 2024-04-26T21:42:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.019+0000 7fd390c5cf40 1 -- start start 2024-04-26T21:42:24.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd390c5cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e83220 0x55f947f50c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd390c5cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947e873e0 0x55f947f511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd390c5cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947f5a420 0x55f947f516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd390c5cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f947e2ce40 con 0x55f947f5a420 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd390c5cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f947e30e80 con 0x55f947e83220 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd390c5cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f947e83bb0 con 0x55f947e873e0 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947f5a420 0x55f947f516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947f5a420 0x55f947f516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57470/0 (socket says 172.21.15.40:57470) 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 -- 172.21.15.40:0/3077041855 learned_addr learned my addr 172.21.15.40:0/3077041855 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38df55640 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947e873e0 0x55f947f511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38e756640 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e83220 0x55f947f50c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 -- 172.21.15.40:0/3077041855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947e873e0 msgr2=0x55f947f511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947e873e0 0x55f947f511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 -- 172.21.15.40:0/3077041855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e83220 msgr2=0x55f947f50c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e83220 0x55f947f50c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 -- 172.21.15.40:0/3077041855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f947f58f10 con 0x55f947f5a420 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38ef57640 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947f5a420 0x55f947f516f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd380025b00 tx=0x7fd380051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a086a35b4c96680 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38cf53640 1 -- 172.21.15.40:0/3077041855 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380067070 con 0x55f947f5a420 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38cf53640 1 -- 172.21.15.40:0/3077041855 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd380050b60 con 0x55f947f5a420 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.020+0000 7fd38cf53640 1 -- 172.21.15.40:0/3077041855 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380058400 con 0x55f947f5a420 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 -- 172.21.15.40:0/3077041855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947f5a420 msgr2=0x55f947f516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947f5a420 0x55f947f516f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd380025b00 tx=0x7fd380051000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 -- 172.21.15.40:0/3077041855 shutdown_connections 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947f5a420 0x55f947f516f0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947e873e0 0x55f947f511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 --2- 172.21.15.40:0/3077041855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e83220 0x55f947f50c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 -- 172.21.15.40:0/3077041855 >> 172.21.15.40:0/3077041855 conn(0x55f947e84330 msgr2=0x55f947ecd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 -- 172.21.15.40:0/3077041855 shutdown_connections 2024-04-26T21:42:24.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 -- 172.21.15.40:0/3077041855 wait complete. 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 Processor -- start 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.021+0000 7fd390c5cf40 1 -- start start 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd390c5cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947e83220 0x55f947f52690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd390c5cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e873e0 0x55f947ff48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd390c5cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947f5a420 0x55f947ff4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd390c5cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f947f5bed0 con 0x55f947e83220 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd390c5cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f947f5b5b0 con 0x55f947e873e0 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd390c5cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f947f5b800 con 0x55f947f5a420 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd38e756640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947e83220 0x55f947f52690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd38e756640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947e83220 0x55f947f52690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57484/0 (socket says 172.21.15.40:57484) 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd38e756640 1 -- 172.21.15.40:0/4221180742 learned_addr learned my addr 172.21.15.40:0/4221180742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd38ef57640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947f5a420 0x55f947ff4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.022+0000 7fd38df55640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e873e0 0x55f947ff48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd38e756640 1 -- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947f5a420 msgr2=0x55f947ff4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd38e756640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947f5a420 0x55f947ff4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd38e756640 1 -- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e873e0 msgr2=0x55f947ff48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd38e756640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e873e0 0x55f947ff48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd38e756640 1 -- 172.21.15.40:0/4221180742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f947f4d130 con 0x55f947e83220 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd38ef57640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f947f5a420 0x55f947ff4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd38df55640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f947e873e0 0x55f947ff48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd38e756640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947e83220 0x55f947f52690 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd38800ce20 tx=0x7fd38800a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd3877fe640 1 -- 172.21.15.40:0/4221180742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388008040 con 0x55f947e83220 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd3877fe640 1 -- 172.21.15.40:0/4221180742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd388014020 con 0x55f947e83220 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd3877fe640 1 -- 172.21.15.40:0/4221180742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38800f400 con 0x55f947e83220 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f947f4eb70 con 0x55f947e83220 2024-04-26T21:42:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.023+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f947f53450 con 0x55f947e83220 2024-04-26T21:42:24.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.024+0000 7fd3877fe640 1 -- 172.21.15.40:0/4221180742 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd38800f5a0 con 0x55f947e83220 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.025+0000 7fd3877fe640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd37006b300 0x7fd37006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.025+0000 7fd3877fe640 1 -- 172.21.15.40:0/4221180742 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fd38808a940 con 0x55f947e83220 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.025+0000 7fd390c5cf40 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f947f52bd0 0x55f947f4f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.025+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f947f4db30 con 0x55f947f52bd0 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.025+0000 7fd38df55640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd37006b300 0x7fd37006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.025+0000 7fd38ef57640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f947f52bd0 0x55f947f4f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.025+0000 7fd38ef57640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f947f52bd0 0x55f947f4f880 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd38df55640 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd37006b300 0x7fd37006d7c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55f947e83b50 tx=0x7fd378001a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd38ef57640 1 -- 172.21.15.40:0/4221180742 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd380062610 con 0x55f947f52bd0 2024-04-26T21:42:24.030 INFO:teuthology.orchestra.run.smithi040.stdout:header (0 bytes) : 2024-04-26T21:42:24.031 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f947f52bd0 msgr2=0x55f947f4f880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f947f52bd0 0x55f947f4f880 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd37006b300 msgr2=0x7fd37006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd37006b300 0x7fd37006d7c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55f947e83b50 tx=0x7fd378001a80 comp rx=0 tx=0).stop 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947e83220 msgr2=0x55f947f52690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 --2- 172.21.15.40:0/4221180742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f947e83220 0x55f947f52690 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd38800ce20 tx=0x7fd38800a5d0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd38ef57640 1 -- 172.21.15.40:0/4221180742 reap_dead start 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 shutdown_connections 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 >> 172.21.15.40:0/4221180742 conn(0x55f947e84330 msgr2=0x55f947f5d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 shutdown_connections 2024-04-26T21:42:24.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.026+0000 7fd390c5cf40 1 -- 172.21.15.40:0/4221180742 wait complete. 2024-04-26T21:42:24.033 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000000 2024-04-26T21:42:24.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.114+0000 7f43db51cf40 1 Processor -- start 2024-04-26T21:42:24.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.114+0000 7f43db51cf40 1 -- start start 2024-04-26T21:42:24.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.114+0000 7f43db51cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 0x55b8e8277620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.114+0000 7f43db51cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 0x55b8e8283810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.114+0000 7f43db51cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.114+0000 7f43db51cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b8e81f3c60 con 0x55b8e8281420 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.114+0000 7f43db51cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b8e8153e40 con 0x55b8e81aa220 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.114+0000 7f43db51cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b8e81aab80 con 0x55b8e81ae3e0 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57506/0 (socket says 172.21.15.40:57506) 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2968686310 learned_addr learned my addr 172.21.15.40:0/2968686310 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d7d49640 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 0x55b8e8283810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d854a640 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 0x55b8e8277620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2968686310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 msgr2=0x55b8e8283810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 0x55b8e8283810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2968686310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 msgr2=0x55b8e8277620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 0x55b8e8277620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2968686310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e81f4190 con 0x55b8e8281420 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d8d4b640 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827a2b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f43d000a560 tx=0x7f43d000aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b832786f123cf7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d6d47640 1 -- 172.21.15.40:0/2968686310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0008a40 con 0x55b8e8281420 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d6d47640 1 -- 172.21.15.40:0/2968686310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d0008be0 con 0x55b8e8281420 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.115+0000 7f43d6d47640 1 -- 172.21.15.40:0/2968686310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0013440 con 0x55b8e8281420 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 -- 172.21.15.40:0/2968686310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 msgr2=0x55b8e827a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827a2b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f43d000a560 tx=0x7f43d000aa20 comp rx=0 tx=0).stop 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 -- 172.21.15.40:0/2968686310 shutdown_connections 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827a2b0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 0x55b8e8283810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 --2- 172.21.15.40:0/2968686310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 0x55b8e8277620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 -- 172.21.15.40:0/2968686310 >> 172.21.15.40:0/2968686310 conn(0x55b8e81ab330 msgr2=0x55b8e81adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 -- 172.21.15.40:0/2968686310 shutdown_connections 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 -- 172.21.15.40:0/2968686310 wait complete. 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 Processor -- start 2024-04-26T21:42:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.116+0000 7f43db51cf40 1 -- start start 2024-04-26T21:42:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43db51cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 0x55b8e8311c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43db51cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 0x55b8e827b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43db51cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43db51cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b8e81f4d80 con 0x55b8e8281420 2024-04-26T21:42:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43db51cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b8e81f4aa0 con 0x55b8e81aa220 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43db51cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b8e81f5060 con 0x55b8e81ae3e0 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43d8d4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43d8d4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57512/0 (socket says 172.21.15.40:57512) 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2593045854 learned_addr learned my addr 172.21.15.40:0/2593045854 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43d854a640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 0x55b8e8311c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43d7d49640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 0x55b8e827b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.117+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 msgr2=0x55b8e827b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43d8d4b640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b8e81ae3e0 0x55b8e827b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 msgr2=0x55b8e8311c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43d8d4b640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b8e81aa220 0x55b8e8311c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2593045854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e8276a60 con 0x55b8e8281420 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43d8d4b640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827d870 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x55b8e8279b10 tx=0x7f43d0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43d5d45640 1 -- 172.21.15.40:0/2593045854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0022020 con 0x55b8e8281420 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43d5d45640 1 -- 172.21.15.40:0/2593045854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d0008d90 con 0x55b8e8281420 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b8e83121b0 con 0x55b8e8281420 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43d5d45640 1 -- 172.21.15.40:0/2593045854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0018040 con 0x55b8e8281420 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.118+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e8285ef0 con 0x55b8e8281420 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.119+0000 7f43d5d45640 1 -- 172.21.15.40:0/2593045854 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43d0025020 con 0x55b8e8281420 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.120+0000 7f43d5d45640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f43b806b2d0 0x7f43b806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.120+0000 7f43d5d45640 1 -- 172.21.15.40:0/2593045854 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f43d008c7a0 con 0x55b8e8281420 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.120+0000 7f43d854a640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f43b806b2d0 0x7f43b806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.120+0000 7f43db51cf40 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b8e81f29e0 0x55b8e831bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.120+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55b8e81d7f90 con 0x55b8e81f29e0 2024-04-26T21:42:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.120+0000 7f43d7d49640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b8e81f29e0 0x55b8e831bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.120+0000 7f43d7d49640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b8e81f29e0 0x55b8e831bdc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.120+0000 7f43d854a640 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f43b806b2d0 0x7f43b806d790 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f43c8056000 tx=0x7f43c8007030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43d7d49640 1 -- 172.21.15.40:0/2593045854 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f43c000a440 con 0x55b8e81f29e0 2024-04-26T21:42:24.125 INFO:teuthology.orchestra.run.smithi040.stdout:header (0 bytes) : 2024-04-26T21:42:24.126 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b8e81f29e0 msgr2=0x55b8e831bdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b8e81f29e0 0x55b8e831bdc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f43b806b2d0 msgr2=0x7f43b806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f43b806b2d0 0x7f43b806d790 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f43c8056000 tx=0x7f43c8007030 comp rx=0 tx=0).stop 2024-04-26T21:42:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 msgr2=0x55b8e827d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 --2- 172.21.15.40:0/2593045854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b8e8281420 0x55b8e827d870 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x55b8e8279b10 tx=0x7f43d0002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43d8d4b640 1 -- 172.21.15.40:0/2593045854 reap_dead start 2024-04-26T21:42:24.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 shutdown_connections 2024-04-26T21:42:24.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 >> 172.21.15.40:0/2593045854 conn(0x55b8e81ab330 msgr2=0x55b8e81adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 shutdown_connections 2024-04-26T21:42:24.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.121+0000 7f43db51cf40 1 -- 172.21.15.40:0/2593045854 wait complete. 2024-04-26T21:42:24.128 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000001 2024-04-26T21:42:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.209+0000 7fa5a9fc2f40 1 Processor -- start 2024-04-26T21:42:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.209+0000 7fa5a9fc2f40 1 -- start start 2024-04-26T21:42:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.209+0000 7fa5a9fc2f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc6ed1c0 0x55c0bc719510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.209+0000 7fa5a9fc2f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc738420 0x55c0bc6f0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a9fc2f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc7bff70 0x55c0bc71d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a9fc2f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc695e40 con 0x55c0bc7bff70 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a9fc2f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc699e80 con 0x55c0bc738420 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a9fc2f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c0bc7c4bd0 con 0x55c0bc6ed1c0 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc7bff70 0x55c0bc71d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc7bff70 0x55c0bc71d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57538/0 (socket says 172.21.15.40:57538) 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 -- 172.21.15.40:0/2679037505 learned_addr learned my addr 172.21.15.40:0/2679037505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a72bb640 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc738420 0x55c0bc6f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a7abc640 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc6ed1c0 0x55c0bc719510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 -- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc6ed1c0 msgr2=0x55c0bc719510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc6ed1c0 0x55c0bc719510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 -- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc738420 msgr2=0x55c0bc6f0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc738420 0x55c0bc6f0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 -- 172.21.15.40:0/2679037505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0bc7182f0 con 0x55c0bc7bff70 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a72bb640 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc738420 0x55c0bc6f0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a82bd640 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc7bff70 0x55c0bc71d410 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa5980015c0 tx=0x7fa598001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c44b885f67110c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a62b9640 1 -- 172.21.15.40:0/2679037505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598003650 con 0x55c0bc7bff70 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a62b9640 1 -- 172.21.15.40:0/2679037505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5980037f0 con 0x55c0bc7bff70 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.210+0000 7fa5a62b9640 1 -- 172.21.15.40:0/2679037505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598003c20 con 0x55c0bc7bff70 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/2679037505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc7bff70 msgr2=0x55c0bc71d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc7bff70 0x55c0bc71d410 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa5980015c0 tx=0x7fa598001a80 comp rx=0 tx=0).stop 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/2679037505 shutdown_connections 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc7bff70 0x55c0bc71d410 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc738420 0x55c0bc6f0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 --2- 172.21.15.40:0/2679037505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc6ed1c0 0x55c0bc719510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/2679037505 >> 172.21.15.40:0/2679037505 conn(0x55c0bc7188a0 msgr2=0x55c0bc7c5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/2679037505 shutdown_connections 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/2679037505 wait complete. 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 Processor -- start 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.211+0000 7fa5a9fc2f40 1 -- start start 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a9fc2f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc6ed1c0 0x55c0bc7b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a9fc2f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc738420 0x55c0bc7c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a9fc2f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc7bff70 0x55c0bc85ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a9fc2f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc6efed0 con 0x55c0bc738420 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a9fc2f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc6efbf0 con 0x55c0bc6ed1c0 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a9fc2f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c0bc7c5060 con 0x55c0bc7bff70 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a72bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc738420 0x55c0bc7c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a72bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc738420 0x55c0bc7c07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57544/0 (socket says 172.21.15.40:57544) 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a72bb640 1 -- 172.21.15.40:0/1249942259 learned_addr learned my addr 172.21.15.40:0/1249942259 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a7abc640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc6ed1c0 0x55c0bc7b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.212+0000 7fa5a82bd640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc7bff70 0x55c0bc85ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a72bb640 1 -- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc7bff70 msgr2=0x55c0bc85ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a72bb640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0bc7bff70 0x55c0bc85ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a72bb640 1 -- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc6ed1c0 msgr2=0x55c0bc7b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a72bb640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0bc6ed1c0 0x55c0bc7b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a72bb640 1 -- 172.21.15.40:0/1249942259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0bc7c3250 con 0x55c0bc738420 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a72bb640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc738420 0x55c0bc7c07c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa59000dfc0 tx=0x7fa59000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a52b7640 1 -- 172.21.15.40:0/1249942259 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590015070 con 0x55c0bc738420 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a52b7640 1 -- 172.21.15.40:0/1249942259 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590011640 con 0x55c0bc738420 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a52b7640 1 -- 172.21.15.40:0/1249942259 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590011a10 con 0x55c0bc738420 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c0bc7bf1a0 con 0x55c0bc738420 2024-04-26T21:42:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.213+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0bc7c2ee0 con 0x55c0bc738420 2024-04-26T21:42:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.214+0000 7fa5a52b7640 1 -- 172.21.15.40:0/1249942259 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa590018070 con 0x55c0bc738420 2024-04-26T21:42:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.214+0000 7fa5a52b7640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa58806b2d0 0x7fa58806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.215+0000 7fa5a52b7640 1 -- 172.21.15.40:0/1249942259 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fa59008aa30 con 0x55c0bc738420 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.215+0000 7fa5a9fc2f40 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0bc7c0d00 0x55c0bc731fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.215+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55c0bc7c87e0 con 0x55c0bc7c0d00 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.215+0000 7fa5a7abc640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa58806b2d0 0x7fa58806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.215+0000 7fa5a82bd640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0bc7c0d00 0x55c0bc731fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.215+0000 7fa5a82bd640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0bc7c0d00 0x55c0bc731fb0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.215+0000 7fa5a7abc640 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa58806b2d0 0x7fa58806d790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0057000 tx=0x7fa5a0052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.215+0000 7fa5a82bd640 1 -- 172.21.15.40:0/1249942259 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa59800f030 con 0x55c0bc7c0d00 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stdout:header (0 bytes) : 2024-04-26T21:42:24.220 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0bc7c0d00 msgr2=0x55c0bc731fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0bc7c0d00 0x55c0bc731fb0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa58806b2d0 msgr2=0x7fa58806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa58806b2d0 0x7fa58806d790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0057000 tx=0x7fa5a0052000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc738420 msgr2=0x55c0bc7c07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 --2- 172.21.15.40:0/1249942259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0bc738420 0x55c0bc7c07c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa59000dfc0 tx=0x7fa59000b5b0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a82bd640 1 -- 172.21.15.40:0/1249942259 reap_dead start 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 shutdown_connections 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 >> 172.21.15.40:0/1249942259 conn(0x55c0bc7188a0 msgr2=0x55c0bc7c53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 shutdown_connections 2024-04-26T21:42:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.216+0000 7fa5a9fc2f40 1 -- 172.21.15.40:0/1249942259 wait complete. 2024-04-26T21:42:24.223 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_openfiles.0 2024-04-26T21:42:24.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.303+0000 7fa9afd59f40 1 Processor -- start 2024-04-26T21:42:24.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.303+0000 7fa9afd59f40 1 -- start start 2024-04-26T21:42:24.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9afd59f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716ba01c0 0x55b716be7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9afd59f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716beb420 0x55b716bcbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9afd59f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716c72f70 0x55b716ba3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9afd59f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b716b48e40 con 0x55b716ba01c0 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9afd59f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b716bea660 con 0x55b716c72f70 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9afd59f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b716be7ed0 con 0x55b716beb420 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716ba01c0 0x55b716be7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716ba01c0 0x55b716be7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57552/0 (socket says 172.21.15.40:57552) 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 -- 172.21.15.40:0/1418106095 learned_addr learned my addr 172.21.15.40:0/1418106095 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ae054640 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716c72f70 0x55b716ba3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad052640 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716beb420 0x55b716bcbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 -- 172.21.15.40:0/1418106095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716beb420 msgr2=0x55b716bcbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716beb420 0x55b716bcbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 -- 172.21.15.40:0/1418106095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716c72f70 msgr2=0x55b716ba3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716c72f70 0x55b716ba3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 -- 172.21.15.40:0/1418106095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b716bcb550 con 0x55b716ba01c0 2024-04-26T21:42:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9ad853640 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716ba01c0 0x55b716be7990 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009600 tx=0x7fa9a8009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eafefe0869cf22d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9a7fff640 1 -- 172.21.15.40:0/1418106095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8017070 con 0x55b716ba01c0 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9a7fff640 1 -- 172.21.15.40:0/1418106095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a80066b0 con 0x55b716ba01c0 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.304+0000 7fa9a7fff640 1 -- 172.21.15.40:0/1418106095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8006a60 con 0x55b716ba01c0 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 -- 172.21.15.40:0/1418106095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716ba01c0 msgr2=0x55b716be7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716ba01c0 0x55b716be7990 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009600 tx=0x7fa9a8009ac0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 -- 172.21.15.40:0/1418106095 shutdown_connections 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716c72f70 0x55b716ba3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716beb420 0x55b716bcbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 --2- 172.21.15.40:0/1418106095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716ba01c0 0x55b716be7990 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 -- 172.21.15.40:0/1418106095 >> 172.21.15.40:0/1418106095 conn(0x55b716bcb8a0 msgr2=0x55b716bcc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 -- 172.21.15.40:0/1418106095 shutdown_connections 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 -- 172.21.15.40:0/1418106095 wait complete. 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 Processor -- start 2024-04-26T21:42:24.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.305+0000 7fa9afd59f40 1 -- start start 2024-04-26T21:42:24.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9afd59f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716ba01c0 0x55b716c73350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9afd59f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716beb420 0x55b716c7aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9afd59f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716c72f70 0x55b716c7b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9afd59f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b716bcf2c0 con 0x55b716c72f70 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9afd59f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b716bcb270 con 0x55b716beb420 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9afd59f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b716bcf5a0 con 0x55b716ba01c0 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9ae054640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716c72f70 0x55b716c7b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9ae054640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716c72f70 0x55b716c7b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57556/0 (socket says 172.21.15.40:57556) 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9ae054640 1 -- 172.21.15.40:0/661435185 learned_addr learned my addr 172.21.15.40:0/661435185 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9ad052640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716beb420 0x55b716c7aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.306+0000 7fa9ad853640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716ba01c0 0x55b716c73350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9ae054640 1 -- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716ba01c0 msgr2=0x55b716c73350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9ae054640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b716ba01c0 0x55b716c73350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9ae054640 1 -- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716beb420 msgr2=0x55b716c7aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9ae054640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b716beb420 0x55b716c7aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9ae054640 1 -- 172.21.15.40:0/661435185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b716be5ca0 con 0x55b716c72f70 2024-04-26T21:42:24.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9ae054640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716c72f70 0x55b716c7b030 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0025ae0 tx=0x7fa9a00033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9a6ffd640 1 -- 172.21.15.40:0/661435185 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0068070 con 0x55b716c72f70 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9a6ffd640 1 -- 172.21.15.40:0/661435185 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a00633f0 con 0x55b716c72f70 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b716be5f80 con 0x55b716c72f70 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9a6ffd640 1 -- 172.21.15.40:0/661435185 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0058600 con 0x55b716c72f70 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.307+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b716be5990 con 0x55b716c72f70 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.308+0000 7fa9a6ffd640 1 -- 172.21.15.40:0/661435185 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9a006b070 con 0x55b716c72f70 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.309+0000 7fa9a6ffd640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa99006b2d0 0x7fa99006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.309+0000 7fa9a6ffd640 1 -- 172.21.15.40:0/661435185 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fa9a00ce580 con 0x55b716c72f70 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.309+0000 7fa9afd59f40 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b716be6f70 0x55b716c68a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.309+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55b716be7380 con 0x55b716be6f70 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.309+0000 7fa9ad853640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa99006b2d0 0x7fa99006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.309+0000 7fa9ad052640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b716be6f70 0x55b716c68a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9ad853640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa99006b2d0 0x7fa99006d790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800afd0 tx=0x7fa9a800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9ad052640 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b716be6f70 0x55b716c68a00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9ad052640 1 -- 172.21.15.40:0/661435185 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv3 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7fa998001a70 con 0x55b716be6f70 2024-04-26T21:42:24.314 INFO:teuthology.orchestra.run.smithi040.stdout:header (42 bytes) : 2024-04-26T21:42:24.314 INFO:teuthology.orchestra.run.smithi040.stdout:00000000 13 00 00 00 63 65 70 68 20 66 73 20 76 6f 6c 75 |....ceph fs volu| 2024-04-26T21:42:24.314 INFO:teuthology.orchestra.run.smithi040.stdout:00000010 6d 65 20 76 30 31 31 01 01 0d 00 00 00 03 00 00 |me v011.........| 2024-04-26T21:42:24.314 INFO:teuthology.orchestra.run.smithi040.stdout:00000020 00 00 00 00 00 01 00 00 00 00 |..........| 2024-04-26T21:42:24.314 INFO:teuthology.orchestra.run.smithi040.stdout:0000002a 2024-04-26T21:42:24.314 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b716be6f70 msgr2=0x55b716c68a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b716be6f70 0x55b716c68a00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa99006b2d0 msgr2=0x7fa99006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa99006b2d0 0x7fa99006d790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800afd0 tx=0x7fa9a800f000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716c72f70 msgr2=0x55b716c7b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 --2- 172.21.15.40:0/661435185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b716c72f70 0x55b716c7b030 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0025ae0 tx=0x7fa9a00033a0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9ae054640 1 -- 172.21.15.40:0/661435185 reap_dead start 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 shutdown_connections 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 >> 172.21.15.40:0/661435185 conn(0x55b716bcb8a0 msgr2=0x55b716bd1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 shutdown_connections 2024-04-26T21:42:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.310+0000 7fa9afd59f40 1 -- 172.21.15.40:0/661435185 wait complete. 2024-04-26T21:42:24.317 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 500.00000000 2024-04-26T21:42:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb142b24f40 1 Processor -- start 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb142b24f40 1 -- start start 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb142b24f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 0x55ff26df1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb142b24f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26df3940 0x55ff26e76e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb142b24f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26e7f6b0 0x55ff26da7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb142b24f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ff26e80aa0 con 0x55ff26e7f6b0 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb142b24f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ff26d51e40 con 0x55ff26df3940 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb142b24f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ff26ddba20 con 0x55ff26df2700 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.397+0000 7fb1403ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26e7f6b0 0x55ff26da7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb1403ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26e7f6b0 0x55ff26da7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57566/0 (socket says 172.21.15.40:57566) 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb1403ba640 1 -- 172.21.15.40:0/3915520861 learned_addr learned my addr 172.21.15.40:0/3915520861 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb13f3b8640 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26df3940 0x55ff26e76e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb13fbb9640 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 0x55ff26df1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb1403ba640 1 -- 172.21.15.40:0/3915520861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 msgr2=0x55ff26df1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb1403ba640 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 0x55ff26df1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb1403ba640 1 -- 172.21.15.40:0/3915520861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26df3940 msgr2=0x55ff26e76e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb1403ba640 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26df3940 0x55ff26e76e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb1403ba640 1 -- 172.21.15.40:0/3915520861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff26e821d0 con 0x55ff26e7f6b0 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb1403ba640 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26e7f6b0 0x55ff26da7f80 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb13800a870 tx=0x7fb138005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81885a8b37c19d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb13e3b6640 1 -- 172.21.15.40:0/3915520861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800d9c0 con 0x55ff26e7f6b0 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb13e3b6640 1 -- 172.21.15.40:0/3915520861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13800db60 con 0x55ff26e7f6b0 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.398+0000 7fb13e3b6640 1 -- 172.21.15.40:0/3915520861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138012450 con 0x55ff26e7f6b0 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 -- 172.21.15.40:0/3915520861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26e7f6b0 msgr2=0x55ff26da7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26e7f6b0 0x55ff26da7f80 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb13800a870 tx=0x7fb138005d80 comp rx=0 tx=0).stop 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 -- 172.21.15.40:0/3915520861 shutdown_connections 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26e7f6b0 0x55ff26da7f80 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26df3940 0x55ff26e76e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 --2- 172.21.15.40:0/3915520861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 0x55ff26df1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 -- 172.21.15.40:0/3915520861 >> 172.21.15.40:0/3915520861 conn(0x55ff26dd59b0 msgr2=0x55ff26dd5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 -- 172.21.15.40:0/3915520861 shutdown_connections 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 -- 172.21.15.40:0/3915520861 wait complete. 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 Processor -- start 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.399+0000 7fb142b24f40 1 -- start start 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb142b24f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 0x55ff26e75310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb142b24f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26df3940 0x55ff26e75850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb142b24f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26e7f6b0 0x55ff26e72d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb142b24f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ff26ddaab0 con 0x55ff26df3940 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb142b24f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ff26e824b0 con 0x55ff26e7f6b0 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb142b24f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ff26ddad90 con 0x55ff26df2700 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13f3b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26df3940 0x55ff26e75850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13f3b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26df3940 0x55ff26e75850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57568/0 (socket says 172.21.15.40:57568) 2024-04-26T21:42:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13f3b8640 1 -- 172.21.15.40:0/3902011862 learned_addr learned my addr 172.21.15.40:0/3902011862 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb1403ba640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26e7f6b0 0x55ff26e72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13fbb9640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 0x55ff26e75310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13f3b8640 1 -- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 msgr2=0x55ff26e75310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13f3b8640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 0x55ff26e75310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13f3b8640 1 -- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26e7f6b0 msgr2=0x55ff26e72d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13f3b8640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ff26e7f6b0 0x55ff26e72d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13f3b8640 1 -- 172.21.15.40:0/3902011862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff26da8770 con 0x55ff26df3940 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.400+0000 7fb13fbb9640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ff26df2700 0x55ff26e75310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.401+0000 7fb13f3b8640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26df3940 0x55ff26e75850 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c550 tx=0x7fb13400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.401+0000 7fb13d3b4640 1 -- 172.21.15.40:0/3902011862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1340104c0 con 0x55ff26df3940 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.401+0000 7fb13d3b4640 1 -- 172.21.15.40:0/3902011862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb134015020 con 0x55ff26df3940 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.401+0000 7fb13d3b4640 1 -- 172.21.15.40:0/3902011862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1340108a0 con 0x55ff26df3940 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.401+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff26e7dbf0 con 0x55ff26df3940 2024-04-26T21:42:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.401+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff26f19960 con 0x55ff26df3940 2024-04-26T21:42:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.402+0000 7fb13d3b4640 1 -- 172.21.15.40:0/3902011862 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb134010a40 con 0x55ff26df3940 2024-04-26T21:42:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.402+0000 7fb13d3b4640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb12c06b2d0 0x7fb12c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.403+0000 7fb13d3b4640 1 -- 172.21.15.40:0/3902011862 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fb13408b1d0 con 0x55ff26df3940 2024-04-26T21:42:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.403+0000 7fb13fbb9640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb12c06b2d0 0x7fb12c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.403+0000 7fb142b24f40 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ff26e76730 0x55ff26f103e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.403+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55ff26f10920 con 0x55ff26e76730 2024-04-26T21:42:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.403+0000 7fb1403ba640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ff26e76730 0x55ff26f103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.403+0000 7fb13fbb9640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb12c06b2d0 0x7fb12c06d790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb130057000 tx=0x7fb130052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.403+0000 7fb1403ba640 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ff26e76730 0x55ff26f103e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb1403ba640 1 -- 172.21.15.40:0/3902011862 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb138019020 con 0x55ff26e76730 2024-04-26T21:42:24.409 INFO:teuthology.orchestra.run.smithi040.stdout:header (0 bytes) : 2024-04-26T21:42:24.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ff26e76730 msgr2=0x55ff26f103e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ff26e76730 0x55ff26f103e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb12c06b2d0 msgr2=0x7fb12c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb12c06b2d0 0x7fb12c06d790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb130057000 tx=0x7fb130052000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26df3940 msgr2=0x55ff26e75850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 --2- 172.21.15.40:0/3902011862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff26df3940 0x55ff26e75850 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c550 tx=0x7fb13400ca10 comp rx=0 tx=0).stop 2024-04-26T21:42:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb1403ba640 1 -- 172.21.15.40:0/3902011862 reap_dead start 2024-04-26T21:42:24.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 shutdown_connections 2024-04-26T21:42:24.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 >> 172.21.15.40:0/3902011862 conn(0x55ff26dd59b0 msgr2=0x55ff26e81090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 shutdown_connections 2024-04-26T21:42:24.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.404+0000 7fb142b24f40 1 -- 172.21.15.40:0/3902011862 wait complete. 2024-04-26T21:42:24.412 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds_snaptable 2024-04-26T21:42:24.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.493+0000 7f640cfb3f40 1 Processor -- start 2024-04-26T21:42:24.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.493+0000 7f640cfb3f40 1 -- start start 2024-04-26T21:42:24.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640cfb3f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749d7a650 0x558749e126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640cfb3f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749e1f820 0x558749d75c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640cfb3f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e21950 0x558749d761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640cfb3f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558749cf1e40 con 0x558749d7a650 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640cfb3f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558749cf5e80 con 0x558749e1f820 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640cfb3f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558749e1fc00 con 0x558749e21950 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640aaad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749d7a650 0x558749e126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640aaad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749d7a650 0x558749e126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57572/0 (socket says 172.21.15.40:57572) 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640aaad640 1 -- 172.21.15.40:0/3236267452 learned_addr learned my addr 172.21.15.40:0/3236267452 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640b2ae640 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e21950 0x558749d761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.494+0000 7f640a2ac640 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749e1f820 0x558749d75c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640aaad640 1 -- 172.21.15.40:0/3236267452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e21950 msgr2=0x558749d761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640aaad640 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e21950 0x558749d761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640aaad640 1 -- 172.21.15.40:0/3236267452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749e1f820 msgr2=0x558749d75c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640aaad640 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749e1f820 0x558749d75c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640aaad640 1 -- 172.21.15.40:0/3236267452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558749e12c20 con 0x558749d7a650 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640aaad640 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749d7a650 0x558749e126e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0065e0 tx=0x7f63fc006aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e7c40adc5153b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f64092aa640 1 -- 172.21.15.40:0/3236267452 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc016070 con 0x558749d7a650 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f64092aa640 1 -- 172.21.15.40:0/3236267452 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63fc004540 con 0x558749d7a650 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f64092aa640 1 -- 172.21.15.40:0/3236267452 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc004930 con 0x558749d7a650 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 -- 172.21.15.40:0/3236267452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749d7a650 msgr2=0x558749e126e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749d7a650 0x558749e126e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0065e0 tx=0x7f63fc006aa0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 -- 172.21.15.40:0/3236267452 shutdown_connections 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e21950 0x558749d761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749e1f820 0x558749d75c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 --2- 172.21.15.40:0/3236267452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749d7a650 0x558749e126e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 -- 172.21.15.40:0/3236267452 >> 172.21.15.40:0/3236267452 conn(0x558749d748a0 msgr2=0x558749d74c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 -- 172.21.15.40:0/3236267452 shutdown_connections 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.495+0000 7f640cfb3f40 1 -- 172.21.15.40:0/3236267452 wait complete. 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.496+0000 7f640cfb3f40 1 Processor -- start 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.496+0000 7f640cfb3f40 1 -- start start 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640cfb3f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749d7a650 0x558749e10df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640cfb3f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e1f820 0x558749d914f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640cfb3f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749e21950 0x558749d91a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640cfb3f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558749e16860 con 0x558749e21950 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640cfb3f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558749e16e20 con 0x558749d7a650 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640cfb3f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558749e16b40 con 0x558749e1f820 2024-04-26T21:42:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749e21950 0x558749d91a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749e21950 0x558749d91a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57588/0 (socket says 172.21.15.40:57588) 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 -- 172.21.15.40:0/710083176 learned_addr learned my addr 172.21.15.40:0/710083176 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640a2ac640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e1f820 0x558749d914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640aaad640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749d7a650 0x558749e10df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 -- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e1f820 msgr2=0x558749d914f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558749e1f820 0x558749d914f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 -- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749d7a650 msgr2=0x558749e10df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558749d7a650 0x558749e10df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 -- 172.21.15.40:0/710083176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558749e162a0 con 0x558749e21950 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f640b2ae640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749e21950 0x558749d91a30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f640400a520 tx=0x7f6404002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f63f3fff640 1 -- 172.21.15.40:0/710083176 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404063070 con 0x558749e21950 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.497+0000 7f63f3fff640 1 -- 172.21.15.40:0/710083176 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64040545f0 con 0x558749e21950 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.498+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558749e11870 con 0x558749e21950 2024-04-26T21:42:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.498+0000 7f63f3fff640 1 -- 172.21.15.40:0/710083176 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404059040 con 0x558749e21950 2024-04-26T21:42:24.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.498+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558749e15aa0 con 0x558749e21950 2024-04-26T21:42:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.499+0000 7f63f3fff640 1 -- 172.21.15.40:0/710083176 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6404066070 con 0x558749e21950 2024-04-26T21:42:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.499+0000 7f63f3fff640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63f806b2d0 0x7f63f806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.499+0000 7f63f3fff640 1 -- 172.21.15.40:0/710083176 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f64040ce640 con 0x558749e21950 2024-04-26T21:42:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.499+0000 7f640aaad640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63f806b2d0 0x7f63f806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.499+0000 7f640cfb3f40 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558749e1cd30 0x558749d79aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.499+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x558749e13090 con 0x558749e1cd30 2024-04-26T21:42:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.500+0000 7f640a2ac640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558749e1cd30 0x558749d79aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.500+0000 7f640aaad640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63f806b2d0 0x7f63f806d790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f63fc013000 tx=0x7f63fc009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.500+0000 7f640a2ac640 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558749e1cd30 0x558749d79aa0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.500+0000 7f640a2ac640 1 -- 172.21.15.40:0/710083176 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6400001500 con 0x558749e1cd30 2024-04-26T21:42:24.506 INFO:teuthology.orchestra.run.smithi040.stdout:header (0 bytes) : 2024-04-26T21:42:24.506 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.500+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558749e1cd30 msgr2=0x558749d79aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.500+0000 7f640cfb3f40 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558749e1cd30 0x558749d79aa0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63f806b2d0 msgr2=0x7f63f806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640cfb3f40 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63f806b2d0 0x7f63f806d790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f63fc013000 tx=0x7f63fc009000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749e21950 msgr2=0x558749d91a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640cfb3f40 1 --2- 172.21.15.40:0/710083176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558749e21950 0x558749d91a30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f640400a520 tx=0x7f6404002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640b2ae640 1 -- 172.21.15.40:0/710083176 reap_dead start 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 shutdown_connections 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 >> 172.21.15.40:0/710083176 conn(0x558749d748a0 msgr2=0x558749d47f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 shutdown_connections 2024-04-26T21:42:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.501+0000 7f640cfb3f40 1 -- 172.21.15.40:0/710083176 wait complete. 2024-04-26T21:42:24.509 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_inotable 2024-04-26T21:42:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.590+0000 7f50686cdf40 1 Processor -- start 2024-04-26T21:42:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.590+0000 7f50686cdf40 1 -- start start 2024-04-26T21:42:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50686cdf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb2129cd30 0x55cb21372d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50686cdf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cb212cc6d0 0x55cb213688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50686cdf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb212cd2e0 0x55cb21375070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50686cdf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cb2129bcd0 con 0x55cb2129cd30 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50686cdf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cb21243e40 con 0x55cb212cd2e0 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50686cdf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55cb2129bf70 con 0x55cb212cc6d0 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb2129cd30 0x55cb21372d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb2129cd30 0x55cb21372d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57596/0 (socket says 172.21.15.40:57596) 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 -- 172.21.15.40:0/1277697431 learned_addr learned my addr 172.21.15.40:0/1277697431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50669c8640 1 --2- 172.21.15.40:0/1277697431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb212cd2e0 0x55cb21375070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 -- 172.21.15.40:0/1277697431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cb212cc6d0 msgr2=0x55cb213688b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 --2- 172.21.15.40:0/1277697431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cb212cc6d0 0x55cb213688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 -- 172.21.15.40:0/1277697431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb212cd2e0 msgr2=0x55cb21375070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 --2- 172.21.15.40:0/1277697431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb212cd2e0 0x55cb21375070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 -- 172.21.15.40:0/1277697431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2136cd30 con 0x55cb2129cd30 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50661c7640 1 --2- 172.21.15.40:0/1277697431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb2129cd30 0x55cb21372d60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f506000a520 tx=0x7f506004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65e5fdc6258016f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50649c4640 1 -- 172.21.15.40:0/1277697431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060064070 con 0x55cb2129cd30 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50649c4640 1 -- 172.21.15.40:0/1277697431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506000a240 con 0x55cb2129cd30 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.591+0000 7f50649c4640 1 -- 172.21.15.40:0/1277697431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060059040 con 0x55cb2129cd30 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 -- 172.21.15.40:0/1277697431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb2129cd30 msgr2=0x55cb21372d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 --2- 172.21.15.40:0/1277697431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb2129cd30 0x55cb21372d60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f506000a520 tx=0x7f506004e000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 -- 172.21.15.40:0/1277697431 shutdown_connections 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 --2- 172.21.15.40:0/1277697431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb212cd2e0 0x55cb21375070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 --2- 172.21.15.40:0/1277697431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cb212cc6d0 0x55cb213688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 --2- 172.21.15.40:0/1277697431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb2129cd30 0x55cb21372d60 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 -- 172.21.15.40:0/1277697431 >> 172.21.15.40:0/1277697431 conn(0x55cb212c79b0 msgr2=0x55cb21374220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 -- 172.21.15.40:0/1277697431 shutdown_connections 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 -- 172.21.15.40:0/1277697431 wait complete. 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 Processor -- start 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.592+0000 7f50686cdf40 1 -- start start 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50686cdf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb2129cd30 0x55cb21374220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50686cdf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb212cc6d0 0x55cb21374760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50686cdf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cb212cd2e0 0x55cb2136cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50686cdf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cb213732a0 con 0x55cb212cc6d0 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50686cdf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cb213700d0 con 0x55cb2129cd30 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50686cdf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55cb213693a0 con 0x55cb212cd2e0 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50659c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb212cc6d0 0x55cb21374760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50659c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb212cc6d0 0x55cb21374760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57600/0 (socket says 172.21.15.40:57600) 2024-04-26T21:42:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50659c6640 1 -- 172.21.15.40:0/3655137627 learned_addr learned my addr 172.21.15.40:0/3655137627 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50661c7640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb2129cd30 0x55cb21374220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.593+0000 7f50669c8640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cb212cd2e0 0x55cb2136cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50659c6640 1 -- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cb212cd2e0 msgr2=0x55cb2136cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50659c6640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cb212cd2e0 0x55cb2136cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50659c6640 1 -- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb2129cd30 msgr2=0x55cb21374220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50659c6640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cb2129cd30 0x55cb21374220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50659c6640 1 -- 172.21.15.40:0/3655137627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb21368e30 con 0x55cb212cc6d0 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50659c6640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb212cc6d0 0x55cb21374760 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f505c00bd20 tx=0x7f505c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50577fe640 1 -- 172.21.15.40:0/3655137627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c009e30 con 0x55cb212cc6d0 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50577fe640 1 -- 172.21.15.40:0/3655137627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c010440 con 0x55cb212cc6d0 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb21369110 con 0x55cb212cc6d0 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50577fe640 1 -- 172.21.15.40:0/3655137627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c010850 con 0x55cb212cc6d0 2024-04-26T21:42:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.594+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb21375340 con 0x55cb212cc6d0 2024-04-26T21:42:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.595+0000 7f50577fe640 1 -- 172.21.15.40:0/3655137627 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f505c018020 con 0x55cb212cc6d0 2024-04-26T21:42:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.596+0000 7f50577fe640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f504806b300 0x7f504806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.596+0000 7f50577fe640 1 -- 172.21.15.40:0/3655137627 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f505c08a1c0 con 0x55cb212cc6d0 2024-04-26T21:42:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.596+0000 7f50686cdf40 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55cb212c92e0 0x55cb2136d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.596+0000 7f50661c7640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f504806b300 0x7f504806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.596+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cb2136ab00 con 0x55cb212c92e0 2024-04-26T21:42:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.596+0000 7f50669c8640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55cb212c92e0 0x55cb2136d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.596+0000 7f50661c7640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f504806b300 0x7f504806d7c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5060061000 tx=0x7f5060051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50669c8640 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55cb212c92e0 0x55cb2136d960 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50669c8640 1 -- 172.21.15.40:0/3655137627 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f50580028c0 con 0x55cb212c92e0 2024-04-26T21:42:24.605 INFO:teuthology.orchestra.run.smithi040.stdout:header (0 bytes) : 2024-04-26T21:42:24.605 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55cb212c92e0 msgr2=0x55cb2136d960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55cb212c92e0 0x55cb2136d960 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f504806b300 msgr2=0x7f504806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f504806b300 0x7f504806d7c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5060061000 tx=0x7f5060051000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb212cc6d0 msgr2=0x55cb21374760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 --2- 172.21.15.40:0/3655137627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb212cc6d0 0x55cb21374760 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f505c00bd20 tx=0x7f505c009510 comp rx=0 tx=0).stop 2024-04-26T21:42:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50669c8640 1 -- 172.21.15.40:0/3655137627 reap_dead start 2024-04-26T21:42:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 shutdown_connections 2024-04-26T21:42:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 >> 172.21.15.40:0/3655137627 conn(0x55cb212c79b0 msgr2=0x55cb212c8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 shutdown_connections 2024-04-26T21:42:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.597+0000 7f50686cdf40 1 -- 172.21.15.40:0/3655137627 wait complete. 2024-04-26T21:42:24.608 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000 2024-04-26T21:42:24.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.689+0000 7f981c31af40 1 Processor -- start 2024-04-26T21:42:24.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.689+0000 7f981c31af40 1 -- start start 2024-04-26T21:42:24.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f981c31af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 0x556d7d7b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f981c31af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d801940 0x556d7d7fe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f981c31af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d88d6b0 0x556d7d7e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f981c31af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556d7d88eaa0 con 0x556d7d88d6b0 2024-04-26T21:42:24.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f981c31af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556d7d75fe40 con 0x556d7d801940 2024-04-26T21:42:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f981c31af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556d7d7e9a20 con 0x556d7d800700 2024-04-26T21:42:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f9819bd3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d88d6b0 0x556d7d7e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f9819bd3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d88d6b0 0x556d7d7e71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57614/0 (socket says 172.21.15.40:57614) 2024-04-26T21:42:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f9819bd3640 1 -- 172.21.15.40:0/3509732973 learned_addr learned my addr 172.21.15.40:0/3509732973 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f9818bd1640 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d801940 0x556d7d7fe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.690+0000 7f98193d2640 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 0x556d7d7b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f9819bd3640 1 -- 172.21.15.40:0/3509732973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 msgr2=0x556d7d7b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f9819bd3640 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 0x556d7d7b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f9819bd3640 1 -- 172.21.15.40:0/3509732973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d801940 msgr2=0x556d7d7fe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f9819bd3640 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d801940 0x556d7d7fe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f9819bd3640 1 -- 172.21.15.40:0/3509732973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7d803480 con 0x556d7d88d6b0 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f9819bd3640 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d88d6b0 0x556d7d7e71d0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f980c0259e0 tx=0x7f980c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbb73ced0737978 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f98137fe640 1 -- 172.21.15.40:0/3509732973 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c056040 con 0x556d7d88d6b0 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f98137fe640 1 -- 172.21.15.40:0/3509732973 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980c0529a0 con 0x556d7d88d6b0 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f98137fe640 1 -- 172.21.15.40:0/3509732973 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c052d50 con 0x556d7d88d6b0 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 -- 172.21.15.40:0/3509732973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d88d6b0 msgr2=0x556d7d7e71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d88d6b0 0x556d7d7e71d0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f980c0259e0 tx=0x7f980c051000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 -- 172.21.15.40:0/3509732973 shutdown_connections 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d88d6b0 0x556d7d7e71d0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d801940 0x556d7d7fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 --2- 172.21.15.40:0/3509732973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 0x556d7d7b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 -- 172.21.15.40:0/3509732973 >> 172.21.15.40:0/3509732973 conn(0x556d7d7e39b0 msgr2=0x556d7d7e8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 -- 172.21.15.40:0/3509732973 shutdown_connections 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.691+0000 7f981c31af40 1 -- 172.21.15.40:0/3509732973 wait complete. 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.692+0000 7f981c31af40 1 Processor -- start 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.692+0000 7f981c31af40 1 -- start start 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f981c31af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 0x556d7d7bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f981c31af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d801940 0x556d7d88bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f981c31af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d88d6b0 0x556d7d7ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f981c31af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556d7d88c9c0 con 0x556d7d801940 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f981c31af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556d7d7e8720 con 0x556d7d88d6b0 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f981c31af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556d7d7bae00 con 0x556d7d800700 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d801940 0x556d7d88bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d801940 0x556d7d88bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57618/0 (socket says 172.21.15.40:57618) 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 -- 172.21.15.40:0/2907167738 learned_addr learned my addr 172.21.15.40:0/2907167738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9819bd3640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d88d6b0 0x556d7d7ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f98193d2640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 0x556d7d7bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 -- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 msgr2=0x556d7d7bb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d7d800700 0x556d7d7bb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 -- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d88d6b0 msgr2=0x556d7d7ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d7d88d6b0 0x556d7d7ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 -- 172.21.15.40:0/2907167738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7d7e0b20 con 0x556d7d801940 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.693+0000 7f9818bd1640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d801940 0x556d7d88bba0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9804000f60 tx=0x7f980400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.694+0000 7f98127fc640 1 -- 172.21.15.40:0/2907167738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980400f650 con 0x556d7d801940 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.694+0000 7f98127fc640 1 -- 172.21.15.40:0/2907167738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980400f7f0 con 0x556d7d801940 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.694+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556d7d7e0e00 con 0x556d7d801940 2024-04-26T21:42:24.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.694+0000 7f98127fc640 1 -- 172.21.15.40:0/2907167738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980400fbe0 con 0x556d7d801940 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.694+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7d7e13c0 con 0x556d7d801940 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.695+0000 7f98127fc640 1 -- 172.21.15.40:0/2907167738 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9804017400 con 0x556d7d801940 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.695+0000 7f98127fc640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f97fc06b300 0x7f97fc06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.695+0000 7f98127fc640 1 -- 172.21.15.40:0/2907167738 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f980408b140 con 0x556d7d801940 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.696+0000 7f98193d2640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f97fc06b300 0x7f97fc06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.696+0000 7f981c31af40 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x556d7d7f8cb0 0x556d7d920e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.696+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556d7d8002a0 con 0x556d7d7f8cb0 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.696+0000 7f9819bd3640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x556d7d7f8cb0 0x556d7d920e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.696+0000 7f98193d2640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f97fc06b300 0x7f97fc06d7c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9814005fd0 tx=0x7f981400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.696+0000 7f9819bd3640 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x556d7d7f8cb0 0x556d7d920e40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f9819bd3640 1 -- 172.21.15.40:0/2907167738 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f980c052c30 con 0x556d7d7f8cb0 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:header (274 bytes) : 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000030 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000060 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-04-26T21:42:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:00000090 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:000000a0 00 00 00 00 00 00 00 00 b1 1f 2c 66 67 3b de 0d |..........,fg;..| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 |................| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:000000e0 00 00 00 00 00 00 b1 1f 2c 66 67 3b de 0d 00 00 |........,fg;....| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:* 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:00000110 00 00 |..| 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:00000112 2024-04-26T21:42:24.708 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x556d7d7f8cb0 msgr2=0x556d7d920e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x556d7d7f8cb0 0x556d7d920e40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f97fc06b300 msgr2=0x7f97fc06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f97fc06b300 0x7f97fc06d7c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9814005fd0 tx=0x7f981400c040 comp rx=0 tx=0).stop 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d801940 msgr2=0x556d7d88bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 --2- 172.21.15.40:0/2907167738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d7d801940 0x556d7d88bba0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9804000f60 tx=0x7f980400b6c0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f9819bd3640 1 -- 172.21.15.40:0/2907167738 reap_dead start 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 shutdown_connections 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 >> 172.21.15.40:0/2907167738 conn(0x556d7d7e39b0 msgr2=0x556d7d7e4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 shutdown_connections 2024-04-26T21:42:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.697+0000 7f981c31af40 1 -- 172.21.15.40:0/2907167738 wait complete. 2024-04-26T21:42:24.711 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_sessionmap 2024-04-26T21:42:24.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.791+0000 7fa4b55def40 1 Processor -- start 2024-04-26T21:42:24.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.791+0000 7fa4b55def40 1 -- start start 2024-04-26T21:42:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b55def40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858ce1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b55def40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 0x559858e46cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b55def40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 0x559858e471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b55def40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559858d283f0 con 0x559858d0ebe0 2024-04-26T21:42:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b55def40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559858c87e40 con 0x559858db4440 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b55def40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559858d28690 con 0x559858d0f0e0 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858ce1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858ce1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57644/0 (socket says 172.21.15.40:57644) 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 -- 172.21.15.40:0/2378132456 learned_addr learned my addr 172.21.15.40:0/2378132456 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b38d9640 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 0x559858e471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b28d7640 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 0x559858e46cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 -- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 msgr2=0x559858e46cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 0x559858e46cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 -- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 msgr2=0x559858e471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 0x559858e471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 -- 172.21.15.40:0/2378132456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559858d95be0 con 0x559858d0ebe0 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b38d9640 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 0x559858e471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b30d8640 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858ce1de0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4001680 tx=0x7fa4a4001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31867f92246340e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.792+0000 7fa4b18d5640 1 -- 172.21.15.40:0/2378132456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4003650 con 0x559858d0ebe0 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b18d5640 1 -- 172.21.15.40:0/2378132456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a40037f0 con 0x559858d0ebe0 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b18d5640 1 -- 172.21.15.40:0/2378132456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4003ba0 con 0x559858d0ebe0 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 -- 172.21.15.40:0/2378132456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 msgr2=0x559858ce1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858ce1de0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4001680 tx=0x7fa4a4001b40 comp rx=0 tx=0).stop 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 -- 172.21.15.40:0/2378132456 shutdown_connections 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 0x559858e471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 0x559858e46cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 --2- 172.21.15.40:0/2378132456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858ce1de0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 -- 172.21.15.40:0/2378132456 >> 172.21.15.40:0/2378132456 conn(0x559858cdf330 msgr2=0x559858db1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 -- 172.21.15.40:0/2378132456 shutdown_connections 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.793+0000 7fa4b55def40 1 -- 172.21.15.40:0/2378132456 wait complete. 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b55def40 1 Processor -- start 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b55def40 1 -- start start 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b55def40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858e4da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b55def40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 0x559858cdf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b55def40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 0x559858cdfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b55def40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559858dafc30 con 0x559858d0ebe0 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b55def40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559858d95750 con 0x559858db4440 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b55def40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559858db36f0 con 0x559858d0f0e0 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b30d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858e4da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b30d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858e4da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57654/0 (socket says 172.21.15.40:57654) 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.794+0000 7fa4b30d8640 1 -- 172.21.15.40:0/2235763894 learned_addr learned my addr 172.21.15.40:0/2235763894 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b38d9640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 0x559858cdfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b28d7640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 0x559858cdf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b30d8640 1 -- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 msgr2=0x559858cdf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b30d8640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559858d0f0e0 0x559858cdf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b30d8640 1 -- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 msgr2=0x559858cdfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b30d8640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559858db4440 0x559858cdfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b30d8640 1 -- 172.21.15.40:0/2235763894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559858d25610 con 0x559858d0ebe0 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b30d8640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858e4da40 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4001b80 tx=0x7fa4a4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b08d3640 1 -- 172.21.15.40:0/2235763894 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4023070 con 0x559858d0ebe0 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b08d3640 1 -- 172.21.15.40:0/2235763894 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4013420 con 0x559858d0ebe0 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559858d26330 con 0x559858d0ebe0 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b08d3640 1 -- 172.21.15.40:0/2235763894 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4018040 con 0x559858d0ebe0 2024-04-26T21:42:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.795+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559858d25270 con 0x559858d0ebe0 2024-04-26T21:42:24.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.796+0000 7fa4b08d3640 1 -- 172.21.15.40:0/2235763894 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4a4026070 con 0x559858d0ebe0 2024-04-26T21:42:24.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.797+0000 7fa4b08d3640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4a006b300 0x7fa4a006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.797+0000 7fa4b08d3640 1 -- 172.21.15.40:0/2235763894 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fa4a408b420 con 0x559858d0ebe0 2024-04-26T21:42:24.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.797+0000 7fa4b28d7640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4a006b300 0x7fa4a006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.797+0000 7fa4b55def40 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559858ce2380 0x559858cde730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.797+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559858d2a1d0 con 0x559858ce2380 2024-04-26T21:42:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.797+0000 7fa4b38d9640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559858ce2380 0x559858cde730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b28d7640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4a006b300 0x7fa4a006d7c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x559858e51730 tx=0x7fa4a800c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b38d9640 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559858ce2380 0x559858cde730 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b38d9640 1 -- 172.21.15.40:0/2235763894 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7fa4ac05c040 con 0x559858ce2380 2024-04-26T21:42:24.803 INFO:teuthology.orchestra.run.smithi040.stdout:header (14 bytes) : 2024-04-26T21:42:24.803 INFO:teuthology.orchestra.run.smithi040.stdout:00000000 01 01 08 00 00 00 04 00 00 00 00 00 00 00 |..............| 2024-04-26T21:42:24.803 INFO:teuthology.orchestra.run.smithi040.stdout:0000000e 2024-04-26T21:42:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559858ce2380 msgr2=0x559858cde730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559858ce2380 0x559858cde730 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4a006b300 msgr2=0x7fa4a006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4a006b300 0x7fa4a006d7c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x559858e51730 tx=0x7fa4a800c040 comp rx=0 tx=0).stop 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 msgr2=0x559858e4da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 --2- 172.21.15.40:0/2235763894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559858d0ebe0 0x559858e4da40 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4001b80 tx=0x7fa4a4001250 comp rx=0 tx=0).stop 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b38d9640 1 -- 172.21.15.40:0/2235763894 reap_dead start 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 shutdown_connections 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 >> 172.21.15.40:0/2235763894 conn(0x559858cdf330 msgr2=0x559858db1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 shutdown_connections 2024-04-26T21:42:24.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.798+0000 7fa4b55def40 1 -- 172.21.15.40:0/2235763894 wait complete. 2024-04-26T21:42:24.806 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 400.00000000 2024-04-26T21:42:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.886+0000 7f1c4e463f40 1 Processor -- start 2024-04-26T21:42:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.886+0000 7f1c4e463f40 1 -- start start 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4e463f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 0x5573fd79b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4e463f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd79faf0 0x5573fd7b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4e463f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd848900 0x5573fd840810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4e463f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5573fd718e40 con 0x5573fd848900 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4e463f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5573fd71ce80 con 0x5573fd76f930 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4e463f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5573fd79f010 con 0x5573fd79faf0 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd848900 0x5573fd840810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd848900 0x5573fd840810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57658/0 (socket says 172.21.15.40:57658) 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 -- 172.21.15.40:0/2559250381 learned_addr learned my addr 172.21.15.40:0/2559250381 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4bf5d640 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 0x5573fd79b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4b75c640 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd79faf0 0x5573fd7b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 -- 172.21.15.40:0/2559250381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd79faf0 msgr2=0x5573fd7b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd79faf0 0x5573fd7b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 -- 172.21.15.40:0/2559250381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 msgr2=0x5573fd79b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 0x5573fd79b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 -- 172.21.15.40:0/2559250381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fd846cf0 con 0x5573fd848900 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.887+0000 7f1c4c75e640 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd848900 0x5573fd840810 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400a520 tx=0x7f1c4404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b3b6eab0903c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4a75a640 1 -- 172.21.15.40:0/2559250381 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44059040 con 0x5573fd848900 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4a75a640 1 -- 172.21.15.40:0/2559250381 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c44064070 con 0x5573fd848900 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4a75a640 1 -- 172.21.15.40:0/2559250381 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440547f0 con 0x5573fd848900 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 -- 172.21.15.40:0/2559250381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd848900 msgr2=0x5573fd840810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd848900 0x5573fd840810 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400a520 tx=0x7f1c4404e000 comp rx=0 tx=0).stop 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 -- 172.21.15.40:0/2559250381 shutdown_connections 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd848900 0x5573fd840810 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd79faf0 0x5573fd7b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 --2- 172.21.15.40:0/2559250381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 0x5573fd79b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 -- 172.21.15.40:0/2559250381 >> 172.21.15.40:0/2559250381 conn(0x5573fd770330 msgr2=0x5573fd79ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 -- 172.21.15.40:0/2559250381 shutdown_connections 2024-04-26T21:42:24.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.888+0000 7f1c4e463f40 1 -- 172.21.15.40:0/2559250381 wait complete. 2024-04-26T21:42:24.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.889+0000 7f1c4e463f40 1 Processor -- start 2024-04-26T21:42:24.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.889+0000 7f1c4e463f40 1 -- start start 2024-04-26T21:42:24.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.889+0000 7f1c4e463f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 0x5573fd770e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.889+0000 7f1c4e463f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd79faf0 0x5573fd79cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.889+0000 7f1c4e463f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd848900 0x5573fd79d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.889+0000 7f1c4e463f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5573fd844660 con 0x5573fd79faf0 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4e463f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5573fd7709f0 con 0x5573fd76f930 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4e463f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5573fd844940 con 0x5573fd848900 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd79faf0 0x5573fd79cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd79faf0 0x5573fd79cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57668/0 (socket says 172.21.15.40:57668) 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 -- 172.21.15.40:0/4162262877 learned_addr learned my addr 172.21.15.40:0/4162262877 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4bf5d640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 0x5573fd770e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4c75e640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd848900 0x5573fd79d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 -- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd848900 msgr2=0x5573fd79d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5573fd848900 0x5573fd79d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 -- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 msgr2=0x5573fd770e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5573fd76f930 0x5573fd770e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 -- 172.21.15.40:0/4162262877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fd8df620 con 0x5573fd79faf0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4b75c640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd79faf0 0x5573fd79cc30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f1c34001bb0 tx=0x7f1c3400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c49758640 1 -- 172.21.15.40:0/4162262877 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400f920 con 0x5573fd79faf0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c49758640 1 -- 172.21.15.40:0/4162262877 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34013030 con 0x5573fd79faf0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5573fd8df900 con 0x5573fd79faf0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c49758640 1 -- 172.21.15.40:0/4162262877 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400fcc0 con 0x5573fd79faf0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.890+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573fd8dfec0 con 0x5573fd79faf0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.892+0000 7f1c49758640 1 -- 172.21.15.40:0/4162262877 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c340173d0 con 0x5573fd79faf0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.892+0000 7f1c49758640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c2c06b300 0x7f1c2c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.892+0000 7f1c49758640 1 -- 172.21.15.40:0/4162262877 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f1c3408b4e0 con 0x5573fd79faf0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.892+0000 7f1c4e463f40 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5573fd7713b0 0x5573fd84a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.892+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5573fd84aa80 con 0x5573fd7713b0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.892+0000 7f1c4bf5d640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c2c06b300 0x7f1c2c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4c75e640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5573fd7713b0 0x5573fd84a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4bf5d640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c2c06b300 0x7f1c2c06d7c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5573fd7b5340 tx=0x7f1c3c006930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4c75e640 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5573fd7713b0 0x5573fd84a540 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4c75e640 1 -- 172.21.15.40:0/4162262877 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1c4405c040 con 0x5573fd7713b0 2024-04-26T21:42:24.897 INFO:teuthology.orchestra.run.smithi040.stdout:header (0 bytes) : 2024-04-26T21:42:24.897 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5573fd7713b0 msgr2=0x5573fd84a540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5573fd7713b0 0x5573fd84a540 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c2c06b300 msgr2=0x7f1c2c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c2c06b300 0x7f1c2c06d7c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5573fd7b5340 tx=0x7f1c3c006930 comp rx=0 tx=0).stop 2024-04-26T21:42:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd79faf0 msgr2=0x5573fd79cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 --2- 172.21.15.40:0/4162262877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5573fd79faf0 0x5573fd79cc30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f1c34001bb0 tx=0x7f1c3400ca70 comp rx=0 tx=0).stop 2024-04-26T21:42:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4c75e640 1 -- 172.21.15.40:0/4162262877 reap_dead start 2024-04-26T21:42:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 shutdown_connections 2024-04-26T21:42:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 >> 172.21.15.40:0/4162262877 conn(0x5573fd770330 msgr2=0x5573fd846210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 shutdown_connections 2024-04-26T21:42:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.893+0000 7f1c4e463f40 1 -- 172.21.15.40:0/4162262877 wait complete. 2024-04-26T21:42:24.899 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000.inode 2024-04-26T21:42:24.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.980+0000 7fb1b0e8cf40 1 Processor -- start 2024-04-26T21:42:24.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.980+0000 7fb1b0e8cf40 1 -- start start 2024-04-26T21:42:24.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1b0e8cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623216ea140 0x562321772d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1b0e8cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623217764e0 0x5623216d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1b0e8cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x5623216d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1b0e8cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562321649e40 con 0x5623217768c0 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1b0e8cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56232164de80 con 0x5623217764e0 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1b0e8cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5623216d2690 con 0x5623216ea140 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x5623216d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x5623216d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57672/0 (socket says 172.21.15.40:57672) 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 -- 172.21.15.40:0/1216700243 learned_addr learned my addr 172.21.15.40:0/1216700243 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1ae185640 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623217764e0 0x5623216d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1ae986640 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623216ea140 0x562321772d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 -- 172.21.15.40:0/1216700243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623216ea140 msgr2=0x562321772d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623216ea140 0x562321772d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 -- 172.21.15.40:0/1216700243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623217764e0 msgr2=0x5623216d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623217764e0 0x5623216d16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 -- 172.21.15.40:0/1216700243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623216e7f50 con 0x5623217768c0 2024-04-26T21:42:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.981+0000 7fb1af187640 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x5623216d1c10 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800b370 tx=0x7fb1a800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523a37240f0cb759 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1ad183640 1 -- 172.21.15.40:0/1216700243 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8012020 con 0x5623217768c0 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1ad183640 1 -- 172.21.15.40:0/1216700243 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a800d6b0 con 0x5623217768c0 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1ad183640 1 -- 172.21.15.40:0/1216700243 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800daa0 con 0x5623217768c0 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1216700243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 msgr2=0x5623216d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x5623216d1c10 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800b370 tx=0x7fb1a800b830 comp rx=0 tx=0).stop 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1216700243 shutdown_connections 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x5623216d1c10 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623217764e0 0x5623216d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 --2- 172.21.15.40:0/1216700243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623216ea140 0x562321772d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1216700243 >> 172.21.15.40:0/1216700243 conn(0x5623216cc8a0 msgr2=0x5623216d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1216700243 shutdown_connections 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.982+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1216700243 wait complete. 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.983+0000 7fb1b0e8cf40 1 Processor -- start 2024-04-26T21:42:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.983+0000 7fb1b0e8cf40 1 -- start start 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.983+0000 7fb1b0e8cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623216ea140 0x56232176c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.983+0000 7fb1b0e8cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623217764e0 0x5623216a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.983+0000 7fb1b0e8cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x562321773a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.983+0000 7fb1b0e8cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5623216e8c10 con 0x5623217768c0 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.983+0000 7fb1b0e8cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5623216ec510 con 0x5623216ea140 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.983+0000 7fb1b0e8cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56232177aa20 con 0x5623217764e0 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x562321773a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x562321773a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57688/0 (socket says 172.21.15.40:57688) 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 -- 172.21.15.40:0/1273797737 learned_addr learned my addr 172.21.15.40:0/1273797737 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1ae185640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623217764e0 0x5623216a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1ae986640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623216ea140 0x56232176c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 -- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623217764e0 msgr2=0x5623216a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623217764e0 0x5623216a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 -- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623216ea140 msgr2=0x56232176c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623216ea140 0x56232176c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 -- 172.21.15.40:0/1273797737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232176def0 con 0x5623217768c0 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1af187640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x562321773a40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8000c00 tx=0x7fb1a80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1a7fff640 1 -- 172.21.15.40:0/1273797737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8027070 con 0x5623217768c0 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1a7fff640 1 -- 172.21.15.40:0/1273797737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a800e680 con 0x5623217768c0 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56232176e1d0 con 0x5623217768c0 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1a7fff640 1 -- 172.21.15.40:0/1273797737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800ea90 con 0x5623217768c0 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.984+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232176e510 con 0x5623217768c0 2024-04-26T21:42:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.986+0000 7fb1a7fff640 1 -- 172.21.15.40:0/1273797737 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1a802a070 con 0x5623217768c0 2024-04-26T21:42:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.986+0000 7fb1a7fff640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb19006b300 0x7fb19006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.986+0000 7fb1a7fff640 1 -- 172.21.15.40:0/1273797737 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fb1a808c010 con 0x5623217768c0 2024-04-26T21:42:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.986+0000 7fb1b0e8cf40 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623216a16f0 0x562321774490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.986+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5623218115a0 con 0x5623216a16f0 2024-04-26T21:42:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.986+0000 7fb1ae986640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb19006b300 0x7fb19006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.986+0000 7fb1ae185640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623216a16f0 0x562321774490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1ae986640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb19006b300 0x7fb19006d7c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0000f80 tx=0x7fb1a0006aa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1ae185640 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623216a16f0 0x562321774490 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1ae185640 1 -- 172.21.15.40:0/1273797737 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb19805d040 con 0x5623216a16f0 2024-04-26T21:42:24.992 INFO:teuthology.orchestra.run.smithi040.stdout:header (0 bytes) : 2024-04-26T21:42:24.992 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623216a16f0 msgr2=0x562321774490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623216a16f0 0x562321774490 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb19006b300 msgr2=0x7fb19006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb19006b300 0x7fb19006d7c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0000f80 tx=0x7fb1a0006aa0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 msgr2=0x562321773a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 --2- 172.21.15.40:0/1273797737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623217768c0 0x562321773a40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8000c00 tx=0x7fb1a80040e0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1af187640 1 -- 172.21.15.40:0/1273797737 reap_dead start 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 shutdown_connections 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 >> 172.21.15.40:0/1273797737 conn(0x5623216cc8a0 msgr2=0x562321772d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 shutdown_connections 2024-04-26T21:42:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:24.987+0000 7fb1b0e8cf40 1 -- 172.21.15.40:0/1273797737 wait complete. 2024-04-26T21:42:24.995 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000 2024-04-26T21:42:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51350edf40 1 Processor -- start 2024-04-26T21:42:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51350edf40 1 -- start start 2024-04-26T21:42:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51350edf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa50dabe0 0x562aa5161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51350edf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50db0e0 0x562aa517f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51350edf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa5180440 0x562aa50adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51350edf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562aa50f43f0 con 0x562aa5180440 2024-04-26T21:42:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51350edf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562aa5053e40 con 0x562aa50dabe0 2024-04-26T21:42:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51350edf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562aa50f4690 con 0x562aa50db0e0 2024-04-26T21:42:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.075+0000 7f51333e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa5180440 0x562aa50adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51333e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa5180440 0x562aa50adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57700/0 (socket says 172.21.15.40:57700) 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51333e8640 1 -- 172.21.15.40:0/1649777048 learned_addr learned my addr 172.21.15.40:0/1649777048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f5132be7640 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa50dabe0 0x562aa5161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51323e6640 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50db0e0 0x562aa517f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51333e8640 1 -- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50db0e0 msgr2=0x562aa517f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51333e8640 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50db0e0 0x562aa517f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51333e8640 1 -- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa50dabe0 msgr2=0x562aa5161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51333e8640 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa50dabe0 0x562aa5161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51333e8640 1 -- 172.21.15.40:0/1649777048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aa517b5e0 con 0x562aa5180440 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51323e6640 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50db0e0 0x562aa517f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f5132be7640 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa50dabe0 0x562aa5161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51333e8640 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa5180440 0x562aa50adde0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f512400b620 tx=0x7f512400bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90aef1c158843af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51313e4640 1 -- 172.21.15.40:0/1649777048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124014020 con 0x562aa5180440 2024-04-26T21:42:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51313e4640 1 -- 172.21.15.40:0/1649777048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5124009870 con 0x562aa5180440 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.076+0000 7f51313e4640 1 -- 172.21.15.40:0/1649777048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124009c40 con 0x562aa5180440 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 -- 172.21.15.40:0/1649777048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa5180440 msgr2=0x562aa50adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa5180440 0x562aa50adde0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f512400b620 tx=0x7f512400bae0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 -- 172.21.15.40:0/1649777048 shutdown_connections 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa5180440 0x562aa50adde0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50db0e0 0x562aa517f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 --2- 172.21.15.40:0/1649777048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa50dabe0 0x562aa5161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 -- 172.21.15.40:0/1649777048 >> 172.21.15.40:0/1649777048 conn(0x562aa50ab330 msgr2=0x562aa517da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 -- 172.21.15.40:0/1649777048 shutdown_connections 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 -- 172.21.15.40:0/1649777048 wait complete. 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 Processor -- start 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.077+0000 7f51350edf40 1 -- start start 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51350edf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50dabe0 0x562aa5183620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51350edf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa50db0e0 0x562aa5181870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51350edf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa5180440 0x562aa5175170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51350edf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562aa50aedc0 con 0x562aa50db0e0 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51350edf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562aa50af0a0 con 0x562aa5180440 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51350edf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562aa50ae320 con 0x562aa50dabe0 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51323e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa50db0e0 0x562aa5181870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51323e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa50db0e0 0x562aa5181870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57708/0 (socket says 172.21.15.40:57708) 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51323e6640 1 -- 172.21.15.40:0/1611090291 learned_addr learned my addr 172.21.15.40:0/1611090291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51333e8640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa5180440 0x562aa5175170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f5132be7640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50dabe0 0x562aa5183620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51323e6640 1 -- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50dabe0 msgr2=0x562aa5183620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51323e6640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562aa50dabe0 0x562aa5183620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51323e6640 1 -- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa5180440 msgr2=0x562aa5175170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51323e6640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562aa5180440 0x562aa5175170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.078+0000 7f51323e6640 1 -- 172.21.15.40:0/1611090291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aa5181db0 con 0x562aa50db0e0 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.079+0000 7f51323e6640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa50db0e0 0x562aa5181870 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f511c025ce0 tx=0x7f511c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.079+0000 7f512bfff640 1 -- 172.21.15.40:0/1611090291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c068670 con 0x562aa50db0e0 2024-04-26T21:42:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.079+0000 7f512bfff640 1 -- 172.21.15.40:0/1611090291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c063070 con 0x562aa50db0e0 2024-04-26T21:42:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.079+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562aa51756b0 con 0x562aa50db0e0 2024-04-26T21:42:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.079+0000 7f512bfff640 1 -- 172.21.15.40:0/1611090291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c068a10 con 0x562aa50db0e0 2024-04-26T21:42:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.079+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aa521e240 con 0x562aa50db0e0 2024-04-26T21:42:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.080+0000 7f512bfff640 1 -- 172.21.15.40:0/1611090291 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f511c052070 con 0x562aa50db0e0 2024-04-26T21:42:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.081+0000 7f512bfff640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f511406b2d0 0x7f511406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.081+0000 7f512bfff640 1 -- 172.21.15.40:0/1611090291 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f511c0ced20 con 0x562aa50db0e0 2024-04-26T21:42:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.081+0000 7f51350edf40 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562aa517df30 0x562aa5173b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.081+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x562aa51740c0 con 0x562aa517df30 2024-04-26T21:42:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.081+0000 7f5132be7640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f511406b2d0 0x7f511406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.081+0000 7f51333e8640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562aa517df30 0x562aa5173b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f5132be7640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f511406b2d0 0x7f511406d790 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f512c000c00 tx=0x7f512c0017d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51333e8640 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562aa517df30 0x562aa5173b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51333e8640 1 -- 172.21.15.40:0/1611090291 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f5124009aa0 con 0x562aa517df30 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stdout:header (274 bytes) : 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stdout:00000000 04 03 0c 01 00 00 0d 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stdout:00000020 00 00 00 00 bc 1f 2c 66 e6 8a 7a 20 00 00 00 00 |......,f..z ....| 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stdout:00000030 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-04-26T21:42:25.086 INFO:teuthology.orchestra.run.smithi040.stdout:00000050 00 00 bc 1f 2c 66 e6 8a 7a 20 00 00 00 00 00 00 |....,f..z ......| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:00000060 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:00000090 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:000000a0 00 00 00 00 00 00 00 00 bc 1f 2c 66 b4 b6 95 28 |..........,f...(| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 |`...............| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:000000e0 00 00 00 00 00 00 bc 1f 2c 66 b4 b6 95 28 00 00 |........,f...(..| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:* 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:00000110 00 00 |..| 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout:00000112 2024-04-26T21:42:25.087 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562aa517df30 msgr2=0x562aa5173b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562aa517df30 0x562aa5173b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f511406b2d0 msgr2=0x7f511406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f511406b2d0 0x7f511406d790 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f512c000c00 tx=0x7f512c0017d0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa50db0e0 msgr2=0x562aa5181870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 --2- 172.21.15.40:0/1611090291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562aa50db0e0 0x562aa5181870 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f511c025ce0 tx=0x7f511c002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51333e8640 1 -- 172.21.15.40:0/1611090291 reap_dead start 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 shutdown_connections 2024-04-26T21:42:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 >> 172.21.15.40:0/1611090291 conn(0x562aa50ab330 msgr2=0x562aa515ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 shutdown_connections 2024-04-26T21:42:25.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.082+0000 7f51350edf40 1 -- 172.21.15.40:0/1611090291 wait complete. 2024-04-26T21:42:25.090 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-04-26T21:42:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.171+0000 7f687efdbf40 1 Processor -- start 2024-04-26T21:42:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.171+0000 7f687efdbf40 1 -- start start 2024-04-26T21:42:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687efdbf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c9548e0 0x556b1c8b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687efdbf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c8ba0e0 0x556b1c8b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687efdbf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c95d7d0 0x556b1c88de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687efdbf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556b1c833e40 con 0x556b1c9548e0 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687efdbf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556b1c837e80 con 0x556b1c8ba0e0 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687efdbf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556b1c966150 con 0x556b1c95d7d0 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687cad7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c9548e0 0x556b1c8b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687cad7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c9548e0 0x556b1c8b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57714/0 (socket says 172.21.15.40:57714) 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687cad7640 1 -- 172.21.15.40:0/1709075152 learned_addr learned my addr 172.21.15.40:0/1709075152 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687c2d6640 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c8ba0e0 0x556b1c8b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687d2d8640 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c95d7d0 0x556b1c88de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687cad7640 1 -- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c95d7d0 msgr2=0x556b1c88de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687cad7640 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c95d7d0 0x556b1c88de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687cad7640 1 -- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c8ba0e0 msgr2=0x556b1c8b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687cad7640 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c8ba0e0 0x556b1c8b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687cad7640 1 -- 172.21.15.40:0/1709075152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1c8b9ea0 con 0x556b1c9548e0 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.172+0000 7f687c2d6640 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c8ba0e0 0x556b1c8b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687cad7640 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c9548e0 0x556b1c8b6cb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f68740017e0 tx=0x7f6874009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7d6a3cff9f40aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687b2d4640 1 -- 172.21.15.40:0/1709075152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687400e860 con 0x556b1c9548e0 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687b2d4640 1 -- 172.21.15.40:0/1709075152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6874013030 con 0x556b1c9548e0 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687b2d4640 1 -- 172.21.15.40:0/1709075152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687400ec60 con 0x556b1c9548e0 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 -- 172.21.15.40:0/1709075152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c9548e0 msgr2=0x556b1c8b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c9548e0 0x556b1c8b6cb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f68740017e0 tx=0x7f6874009a90 comp rx=0 tx=0).stop 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 -- 172.21.15.40:0/1709075152 shutdown_connections 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c95d7d0 0x556b1c88de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c8ba0e0 0x556b1c8b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 --2- 172.21.15.40:0/1709075152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c9548e0 0x556b1c8b6cb0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 -- 172.21.15.40:0/1709075152 >> 172.21.15.40:0/1709075152 conn(0x556b1c8b68a0 msgr2=0x556b1c8b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 -- 172.21.15.40:0/1709075152 shutdown_connections 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.173+0000 7f687efdbf40 1 -- 172.21.15.40:0/1709075152 wait complete. 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.174+0000 7f687efdbf40 1 Processor -- start 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.174+0000 7f687efdbf40 1 -- start start 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687efdbf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c9548e0 0x556b1c965630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687efdbf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c95d7d0 0x556b1c94dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687efdbf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c952df0 0x556b1c9532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687efdbf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556b1c955040 con 0x556b1c95d7d0 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687efdbf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556b1c955320 con 0x556b1c9548e0 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687efdbf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556b1c8b9bc0 con 0x556b1c952df0 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c95d7d0 0x556b1c94dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c95d7d0 0x556b1c94dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57720/0 (socket says 172.21.15.40:57720) 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 -- 172.21.15.40:0/797306438 learned_addr learned my addr 172.21.15.40:0/797306438 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687cad7640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c9548e0 0x556b1c965630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687d2d8640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c952df0 0x556b1c9532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 -- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c952df0 msgr2=0x556b1c9532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556b1c952df0 0x556b1c9532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 -- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c9548e0 msgr2=0x556b1c965630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c9548e0 0x556b1c965630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 -- 172.21.15.40:0/797306438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1c961160 con 0x556b1c95d7d0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687cad7640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b1c9548e0 0x556b1c965630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687c2d6640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c95d7d0 0x556b1c94dbf0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f6864057040 tx=0x7f6864003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687a2d2640 1 -- 172.21.15.40:0/797306438 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864065070 con 0x556b1c95d7d0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687a2d2640 1 -- 172.21.15.40:0/797306438 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6864055e20 con 0x556b1c95d7d0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556b1c953d50 con 0x556b1c95d7d0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.175+0000 7f687a2d2640 1 -- 172.21.15.40:0/797306438 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686400b430 con 0x556b1c95d7d0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.176+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1c954310 con 0x556b1c95d7d0 2024-04-26T21:42:25.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.177+0000 7f687a2d2640 1 -- 172.21.15.40:0/797306438 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f686405e040 con 0x556b1c95d7d0 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.177+0000 7f687a2d2640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685c06b300 0x7f685c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.177+0000 7f687a2d2640 1 -- 172.21.15.40:0/797306438 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f68640d0090 con 0x556b1c95d7d0 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556b1c8d17e0 0x556b1c8d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x556b1c8d2170 con 0x556b1c8d17e0 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687cad7640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685c06b300 0x7f685c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687d2d8640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556b1c8d17e0 0x556b1c8d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687d2d8640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556b1c8d17e0 0x556b1c8d1c30 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687cad7640 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685c06b300 0x7f685c06d7c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6874009ad0 tx=0x7f687401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687d2d8640 1 -- 172.21.15.40:0/797306438 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=21b] v0'0 uv10 ondisk = 0) ==== 164+0+21 (crc 0 0 0) 0x7f686c006020 con 0x556b1c8d17e0 2024-04-26T21:42:25.184 INFO:teuthology.orchestra.run.smithi040.stdout:sixmegs_head 2024-04-26T21:42:25.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556b1c8d17e0 msgr2=0x556b1c8d1c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x556b1c8d17e0 0x556b1c8d1c30 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685c06b300 msgr2=0x7f685c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f685c06b300 0x7f685c06d7c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6874009ad0 tx=0x7f687401b040 comp rx=0 tx=0).stop 2024-04-26T21:42:25.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c95d7d0 msgr2=0x556b1c94dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 --2- 172.21.15.40:0/797306438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556b1c95d7d0 0x556b1c94dbf0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f6864057040 tx=0x7f6864003020 comp rx=0 tx=0).stop 2024-04-26T21:42:25.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687d2d8640 1 -- 172.21.15.40:0/797306438 reap_dead start 2024-04-26T21:42:25.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 shutdown_connections 2024-04-26T21:42:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.178+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 >> 172.21.15.40:0/797306438 conn(0x556b1c8b68a0 msgr2=0x556b1c95cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.179+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 shutdown_connections 2024-04-26T21:42:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.179+0000 7f687efdbf40 1 -- 172.21.15.40:0/797306438 wait complete. 2024-04-26T21:42:25.186 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 600.00000000 2024-04-26T21:42:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.269+0000 7f1dd2f1cf40 1 Processor -- start 2024-04-26T21:42:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.269+0000 7f1dd2f1cf40 1 -- start start 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dd2f1cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5b34480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dd2f1cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 0x564ab5b12b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dd2f1cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 0x564ab5b31f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dd2f1cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564ab5b36010 con 0x564ab5b17100 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dd2f1cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564ab5a04e40 con 0x564ab5b350d0 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dd2f1cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564ab5b35520 con 0x564ab5b31b50 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcff4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5b34480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcff4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5b34480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57738/0 (socket says 172.21.15.40:57738) 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcff4a640 1 -- 172.21.15.40:0/2121264598 learned_addr learned my addr 172.21.15.40:0/2121264598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcf749640 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 0x564ab5b12b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcff4a640 1 -- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 msgr2=0x564ab5b12b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dd074b640 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 0x564ab5b31f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcff4a640 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 0x564ab5b12b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcff4a640 1 -- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 msgr2=0x564ab5b31f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcff4a640 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 0x564ab5b31f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcff4a640 1 -- 172.21.15.40:0/2121264598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab5b11f80 con 0x564ab5b17100 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.270+0000 7f1dcf749640 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 0x564ab5b12b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dcff4a640 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5b34480 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00016c0 tx=0x7f1dc0001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73aef03bf582a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dce747640 1 -- 172.21.15.40:0/2121264598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0006220 con 0x564ab5b17100 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dce747640 1 -- 172.21.15.40:0/2121264598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc0010590 con 0x564ab5b17100 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dce747640 1 -- 172.21.15.40:0/2121264598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0010980 con 0x564ab5b17100 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/2121264598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 msgr2=0x564ab5b34480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5b34480 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00016c0 tx=0x7f1dc0001b80 comp rx=0 tx=0).stop 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/2121264598 shutdown_connections 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 0x564ab5b31f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 0x564ab5b12b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 --2- 172.21.15.40:0/2121264598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5b34480 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/2121264598 >> 172.21.15.40:0/2121264598 conn(0x564ab5a5bab0 msgr2=0x564ab5a8b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/2121264598 shutdown_connections 2024-04-26T21:42:25.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.271+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/2121264598 wait complete. 2024-04-26T21:42:25.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.272+0000 7f1dd2f1cf40 1 Processor -- start 2024-04-26T21:42:25.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.272+0000 7f1dd2f1cf40 1 -- start start 2024-04-26T21:42:25.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd2f1cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5bbdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd2f1cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 0x564ab5b309f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd2f1cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 0x564ab5b30f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd2f1cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564ab5b16e60 con 0x564ab5b17100 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd2f1cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564ab5b168a0 con 0x564ab5b350d0 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd2f1cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564ab5b12220 con 0x564ab5b31b50 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5bbdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5bbdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57752/0 (socket says 172.21.15.40:57752) 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 -- 172.21.15.40:0/1071756913 learned_addr learned my addr 172.21.15.40:0/1071756913 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 -- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 msgr2=0x564ab5b309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564ab5b31b50 0x564ab5b309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 -- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 msgr2=0x564ab5b30f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd074b640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 0x564ab5b30f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ab5b350d0 0x564ab5b30f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 -- 172.21.15.40:0/1071756913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab5bc0070 con 0x564ab5b17100 2024-04-26T21:42:25.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcff4a640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5bbdc20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0003000 tx=0x7f1dc0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcd745640 1 -- 172.21.15.40:0/1071756913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0021020 con 0x564ab5b17100 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcd745640 1 -- 172.21.15.40:0/1071756913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc0010bd0 con 0x564ab5b17100 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ab5bc0350 con 0x564ab5b17100 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dcd745640 1 -- 172.21.15.40:0/1071756913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc001d480 con 0x564ab5b17100 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.273+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab5bc0910 con 0x564ab5b17100 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.275+0000 7f1dcd745640 1 -- 172.21.15.40:0/1071756913 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dc0024020 con 0x564ab5b17100 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.275+0000 7f1dcd745640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1db006b2d0 0x7f1db006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.275+0000 7f1dcd745640 1 -- 172.21.15.40:0/1071756913 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f1dc008c0b0 con 0x564ab5b17100 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.275+0000 7f1dcf749640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1db006b2d0 0x7f1db006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.275+0000 7f1dd2f1cf40 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564ab5b13120 0x564ab5bbb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.275+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x564ab5bbba40 con 0x564ab5b13120 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd074b640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564ab5b13120 0x564ab5bbb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd074b640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564ab5b13120 0x564ab5bbb500 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dcf749640 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1db006b2d0 0x7f1db006d790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1db8057000 tx=0x7f1db8051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd074b640 1 -- 172.21.15.40:0/1071756913 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 600.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1dc80096f0 con 0x564ab5b13120 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564ab5b13120 msgr2=0x564ab5bbb500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564ab5b13120 0x564ab5bbb500 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1db006b2d0 msgr2=0x7f1db006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1db006b2d0 0x7f1db006d790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1db8057000 tx=0x7f1db8051000 comp rx=0 tx=0).stop 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 msgr2=0x564ab5bbdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 --2- 172.21.15.40:0/1071756913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ab5b17100 0x564ab5bbdc20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0003000 tx=0x7f1dc0000f40 comp rx=0 tx=0).stop 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd074b640 1 -- 172.21.15.40:0/1071756913 reap_dead start 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 shutdown_connections 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 >> 172.21.15.40:0/1071756913 conn(0x564ab5a5bab0 msgr2=0x564ab5b34480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 shutdown_connections 2024-04-26T21:42:25.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.276+0000 7f1dd2f1cf40 1 -- 172.21.15.40:0/1071756913 wait complete. 2024-04-26T21:42:25.282 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000.inode 2024-04-26T21:42:25.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.323+0000 7f9dee00df40 1 Processor -- start 2024-04-26T21:42:25.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.323+0000 7f9dee00df40 1 -- start start 2024-04-26T21:42:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9dee00df40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645628aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9dee00df40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628cdb50 0x5645628277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9dee00df40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628d10d0 0x564562827ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9dee00df40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5645627a0e40 con 0x5645628224d0 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9dee00df40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5645627a4e80 con 0x5645628cdb50 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9dee00df40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5645628d2010 con 0x5645628d10d0 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645628aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645628aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57762/0 (socket says 172.21.15.40:57762) 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 -- 172.21.15.40:0/4233312944 learned_addr learned my addr 172.21.15.40:0/4233312944 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9dea697640 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628cdb50 0x5645628277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deb699640 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628d10d0 0x564562827ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 -- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628d10d0 msgr2=0x564562827ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628d10d0 0x564562827ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 -- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628cdb50 msgr2=0x5645628277a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628cdb50 0x5645628277a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 -- 172.21.15.40:0/4233312944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645628b2fd0 con 0x5645628224d0 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9dea697640 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628cdb50 0x5645628277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.324+0000 7f9deae98640 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645628aff10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b2e0 tx=0x7f9ddc00b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96899556eb33161b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9de9695640 1 -- 172.21.15.40:0/4233312944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc008070 con 0x5645628224d0 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9de9695640 1 -- 172.21.15.40:0/4233312944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc00d500 con 0x5645628224d0 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9de9695640 1 -- 172.21.15.40:0/4233312944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00d8f0 con 0x5645628224d0 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 -- 172.21.15.40:0/4233312944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 msgr2=0x5645628aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645628aff10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b2e0 tx=0x7f9ddc00b7a0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 -- 172.21.15.40:0/4233312944 shutdown_connections 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628d10d0 0x564562827ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628cdb50 0x5645628277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 --2- 172.21.15.40:0/4233312944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645628aff10 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 -- 172.21.15.40:0/4233312944 >> 172.21.15.40:0/4233312944 conn(0x5645627f7ab0 msgr2=0x5645628246b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 -- 172.21.15.40:0/4233312944 shutdown_connections 2024-04-26T21:42:25.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.325+0000 7f9dee00df40 1 -- 172.21.15.40:0/4233312944 wait complete. 2024-04-26T21:42:25.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.326+0000 7f9dee00df40 1 Processor -- start 2024-04-26T21:42:25.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.326+0000 7f9dee00df40 1 -- start start 2024-04-26T21:42:25.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.326+0000 7f9dee00df40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645627f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.326+0000 7f9dee00df40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628cdb50 0x564562967c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.326+0000 7f9dee00df40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628d10d0 0x564562968190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.326+0000 7f9dee00df40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5645628b0ba0 con 0x5645628224d0 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.326+0000 7f9dee00df40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5645628b35c0 con 0x5645628d10d0 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.326+0000 7f9dee00df40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564562824dc0 con 0x5645628cdb50 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645627f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645627f9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57776/0 (socket says 172.21.15.40:57776) 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 -- 172.21.15.40:0/3040015255 learned_addr learned my addr 172.21.15.40:0/3040015255 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deb699640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628d10d0 0x564562968190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9dea697640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628cdb50 0x564562967c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 -- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628cdb50 msgr2=0x564562967c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645628cdb50 0x564562967c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 -- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628d10d0 msgr2=0x564562968190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645628d10d0 0x564562968190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 -- 172.21.15.40:0/3040015255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645628ce4a0 con 0x5645628224d0 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9deae98640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645627f9f50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b7e0 tx=0x7f9ddc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9dd3fff640 1 -- 172.21.15.40:0/3040015255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc01c650 con 0x5645628224d0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9dd3fff640 1 -- 172.21.15.40:0/3040015255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc021020 con 0x5645628224d0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5645628ce780 con 0x5645628224d0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9dd3fff640 1 -- 172.21.15.40:0/3040015255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc01c850 con 0x5645628224d0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.327+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645628ced40 con 0x5645628224d0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.329+0000 7f9dd3fff640 1 -- 172.21.15.40:0/3040015255 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ddc017070 con 0x5645628224d0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.329+0000 7f9dd3fff640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd806b2d0 0x7f9dd806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.329+0000 7f9dd3fff640 1 -- 172.21.15.40:0/3040015255 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f9ddc08c610 con 0x5645628224d0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.329+0000 7f9dee00df40 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5645627fa490 0x5645628cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.329+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5645628af550 con 0x5645627fa490 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.329+0000 7f9deb699640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5645627fa490 0x5645628cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.329+0000 7f9dea697640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd806b2d0 0x7f9dd806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9deb699640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5645627fa490 0x5645628cc390 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dea697640 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd806b2d0 0x7f9dd806d790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x5645628d0c00 tx=0x7f9de0002850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9deb699640 1 -- 172.21.15.40:0/3040015255 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-keys out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f9de404e040 con 0x5645627fa490 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5645627fa490 msgr2=0x5645628cc390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5645627fa490 0x5645628cc390 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd806b2d0 msgr2=0x7f9dd806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd806b2d0 0x7f9dd806d790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x5645628d0c00 tx=0x7f9de0002850 comp rx=0 tx=0).stop 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 msgr2=0x5645627f9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 --2- 172.21.15.40:0/3040015255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645628224d0 0x5645627f9f50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b7e0 tx=0x7f9ddc000ed0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9deb699640 1 -- 172.21.15.40:0/3040015255 reap_dead start 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 shutdown_connections 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 >> 172.21.15.40:0/3040015255 conn(0x5645627f7ab0 msgr2=0x564562824670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 shutdown_connections 2024-04-26T21:42:25.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.330+0000 7f9dee00df40 1 -- 172.21.15.40:0/3040015255 wait complete. 2024-04-26T21:42:25.336 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000000 2024-04-26T21:42:25.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.418+0000 7f7b4c186f40 1 Processor -- start 2024-04-26T21:42:25.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.418+0000 7f7b4c186f40 1 -- start start 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.418+0000 7f7b4c186f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda242670 0x563dda2b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.418+0000 7f7b4c186f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda225d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.418+0000 7f7b4c186f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda2d6610 0x563dda1fa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.418+0000 7f7b4c186f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563dda1a2e40 con 0x563dda2b8d60 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.418+0000 7f7b4c186f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563dda1a6e80 con 0x563dda2d6610 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.418+0000 7f7b4c186f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563dda2b9140 con 0x563dda242670 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda225d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda225d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57788/0 (socket says 172.21.15.40:57788) 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 -- 172.21.15.40:0/1523687520 learned_addr learned my addr 172.21.15.40:0/1523687520 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b4a483640 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda2d6610 0x563dda1fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49c82640 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda242670 0x563dda2b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 -- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda242670 msgr2=0x563dda2b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda242670 0x563dda2b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 -- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda2d6610 msgr2=0x563dda1fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda2d6610 0x563dda1fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 -- 172.21.15.40:0/1523687520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dda224780 con 0x563dda2b8d60 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b4a483640 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda2d6610 0x563dda1fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b49481640 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda225d50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400a4f0 tx=0x7f7b34051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9860071d2f688edf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b43fff640 1 -- 172.21.15.40:0/1523687520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34056030 con 0x563dda2b8d60 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b43fff640 1 -- 172.21.15.40:0/1523687520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b34052a90 con 0x563dda2b8d60 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.419+0000 7f7b43fff640 1 -- 172.21.15.40:0/1523687520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34052e60 con 0x563dda2b8d60 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 -- 172.21.15.40:0/1523687520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 msgr2=0x563dda225d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda225d50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400a4f0 tx=0x7f7b34051000 comp rx=0 tx=0).stop 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 -- 172.21.15.40:0/1523687520 shutdown_connections 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda2d6610 0x563dda1fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda225d50 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 --2- 172.21.15.40:0/1523687520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda242670 0x563dda2b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 -- 172.21.15.40:0/1523687520 >> 172.21.15.40:0/1523687520 conn(0x563dda1f9ab0 msgr2=0x563dda228f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 -- 172.21.15.40:0/1523687520 shutdown_connections 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 -- 172.21.15.40:0/1523687520 wait complete. 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 Processor -- start 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.420+0000 7f7b4c186f40 1 -- start start 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b4c186f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda242670 0x563dda2cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b4c186f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda2d4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b4c186f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda2d6610 0x563dda2b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b4c186f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563dda229fd0 con 0x563dda2b8d60 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b4c186f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563dda2415e0 con 0x563dda242670 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b4c186f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563dda226290 con 0x563dda2d6610 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49481640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda2d4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49481640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda2d4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57790/0 (socket says 172.21.15.40:57790) 2024-04-26T21:42:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49481640 1 -- 172.21.15.40:0/4070934560 learned_addr learned my addr 172.21.15.40:0/4070934560 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49c82640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda242670 0x563dda2cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b4a483640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda2d6610 0x563dda2b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49481640 1 -- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda2d6610 msgr2=0x563dda2b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49481640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563dda2d6610 0x563dda2b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49481640 1 -- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda242670 msgr2=0x563dda2cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49481640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563dda242670 0x563dda2cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.421+0000 7f7b49481640 1 -- 172.21.15.40:0/4070934560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dda2d5710 con 0x563dda2b8d60 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.422+0000 7f7b49481640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda2d4dc0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7b34045fd0 tx=0x7f7b34002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.422+0000 7f7b42ffd640 1 -- 172.21.15.40:0/4070934560 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34056070 con 0x563dda2b8d60 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.422+0000 7f7b42ffd640 1 -- 172.21.15.40:0/4070934560 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b34066440 con 0x563dda2b8d60 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.422+0000 7f7b42ffd640 1 -- 172.21.15.40:0/4070934560 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3406b040 con 0x563dda2b8d60 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.422+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563dda2d59f0 con 0x563dda2b8d60 2024-04-26T21:42:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.422+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dda2d5fb0 con 0x563dda2b8d60 2024-04-26T21:42:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.423+0000 7f7b42ffd640 1 -- 172.21.15.40:0/4070934560 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b34059020 con 0x563dda2b8d60 2024-04-26T21:42:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.424+0000 7f7b42ffd640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2c06b300 0x7f7b2c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.424+0000 7f7b42ffd640 1 -- 172.21.15.40:0/4070934560 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f7b340d00d0 con 0x563dda2b8d60 2024-04-26T21:42:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.424+0000 7f7b49c82640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2c06b300 0x7f7b2c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.424+0000 7f7b4c186f40 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563dda2a58c0 0x563dda2a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.424+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563dda2a61e0 con 0x563dda2a58c0 2024-04-26T21:42:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.424+0000 7f7b4a483640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563dda2a58c0 0x563dda2a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.424+0000 7f7b4a483640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563dda2a58c0 0x563dda2a5ca0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.424+0000 7f7b49c82640 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2c06b300 0x7f7b2c06d7c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7b44003f10 tx=0x7f7b44002880 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4a483640 1 -- 172.21.15.40:0/4070934560 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000000 [omap-get-keys out=5b] v0'0 uv19 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f7b3c0044c0 con 0x563dda2a58c0 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563dda2a58c0 msgr2=0x563dda2a5ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563dda2a58c0 0x563dda2a5ca0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2c06b300 msgr2=0x7f7b2c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b2c06b300 0x7f7b2c06d7c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7b44003f10 tx=0x7f7b44002880 comp rx=0 tx=0).stop 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 msgr2=0x563dda2d4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 --2- 172.21.15.40:0/4070934560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563dda2b8d60 0x563dda2d4dc0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7b34045fd0 tx=0x7f7b34002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4a483640 1 -- 172.21.15.40:0/4070934560 reap_dead start 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 shutdown_connections 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 >> 172.21.15.40:0/4070934560 conn(0x563dda1f9ab0 msgr2=0x563dda2b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 shutdown_connections 2024-04-26T21:42:25.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.425+0000 7f7b4c186f40 1 -- 172.21.15.40:0/4070934560 wait complete. 2024-04-26T21:42:25.431 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000001 2024-04-26T21:42:25.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.511+0000 7fa42c178f40 1 Processor -- start 2024-04-26T21:42:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa42c178f40 1 -- start start 2024-04-26T21:42:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa42c178f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4140030f0 0x560b5e3d3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa42c178f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b5e3d3640 0x560b5e4a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa42c178f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e499b10 0x560b5e4a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa42c178f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560b5e378e40 con 0x560b5e3d3640 2024-04-26T21:42:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa42c178f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b5e37ce80 con 0x560b5e499b10 2024-04-26T21:42:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa42c178f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560b5e4a6c00 con 0x7fa4140030f0 2024-04-26T21:42:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa429473640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b5e3d3640 0x560b5e4a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa429473640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b5e3d3640 0x560b5e4a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57812/0 (socket says 172.21.15.40:57812) 2024-04-26T21:42:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa429473640 1 -- 172.21.15.40:0/1462601246 learned_addr learned my addr 172.21.15.40:0/1462601246 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa429c74640 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4140030f0 0x560b5e3d3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.512+0000 7fa42a475640 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e499b10 0x560b5e4a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa429473640 1 -- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4140030f0 msgr2=0x560b5e3d3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa429473640 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4140030f0 0x560b5e3d3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa429473640 1 -- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e499b10 msgr2=0x560b5e4a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa429473640 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e499b10 0x560b5e4a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa429473640 1 -- 172.21.15.40:0/1462601246 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5e4010e0 con 0x560b5e3d3640 2024-04-26T21:42:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42a475640 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e499b10 0x560b5e4a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa429473640 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b5e3d3640 0x560b5e4a56a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa420026390 tx=0x7fa42004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5646e36b384fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa41bfff640 1 -- 172.21.15.40:0/1462601246 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420064030 con 0x560b5e3d3640 2024-04-26T21:42:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa41bfff640 1 -- 172.21.15.40:0/1462601246 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4200557e0 con 0x560b5e3d3640 2024-04-26T21:42:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa41bfff640 1 -- 172.21.15.40:0/1462601246 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420055bb0 con 0x560b5e3d3640 2024-04-26T21:42:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42c178f40 1 -- 172.21.15.40:0/1462601246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b5e3d3640 msgr2=0x560b5e4a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42c178f40 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b5e3d3640 0x560b5e4a56a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa420026390 tx=0x7fa42004f000 comp rx=0 tx=0).stop 2024-04-26T21:42:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42c178f40 1 -- 172.21.15.40:0/1462601246 shutdown_connections 2024-04-26T21:42:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42c178f40 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e499b10 0x560b5e4a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42c178f40 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b5e3d3640 0x560b5e4a56a0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42c178f40 1 --2- 172.21.15.40:0/1462601246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4140030f0 0x560b5e3d3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42c178f40 1 -- 172.21.15.40:0/1462601246 >> 172.21.15.40:0/1462601246 conn(0x560b5e3fb8a0 msgr2=0x560b5e4a9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.513+0000 7fa42c178f40 1 -- 172.21.15.40:0/1462601246 shutdown_connections 2024-04-26T21:42:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.514+0000 7fa42c178f40 1 -- 172.21.15.40:0/1462601246 wait complete. 2024-04-26T21:42:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.514+0000 7fa42c178f40 1 Processor -- start 2024-04-26T21:42:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.514+0000 7fa42c178f40 1 -- start start 2024-04-26T21:42:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42c178f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e3d3640 0x560b5e492bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42c178f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b5e499b10 0x560b5e3ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42c178f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4140030f0 0x560b5e400430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42c178f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560b5e49c4a0 con 0x7fa4140030f0 2024-04-26T21:42:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42c178f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b5e4a7af0 con 0x560b5e3d3640 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42c178f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560b5e3ffb50 con 0x560b5e499b10 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4140030f0 0x560b5e400430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4140030f0 0x560b5e400430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57816/0 (socket says 172.21.15.40:57816) 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 -- 172.21.15.40:0/2610215719 learned_addr learned my addr 172.21.15.40:0/2610215719 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa429c74640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e3d3640 0x560b5e492bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa429473640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b5e499b10 0x560b5e3ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 -- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b5e499b10 msgr2=0x560b5e3ffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b5e499b10 0x560b5e3ffef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 -- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e3d3640 msgr2=0x560b5e492bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b5e3d3640 0x560b5e492bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 -- 172.21.15.40:0/2610215719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5e3fef40 con 0x7fa4140030f0 2024-04-26T21:42:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.515+0000 7fa42a475640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4140030f0 0x560b5e400430 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa41c000f80 tx=0x7fa41c00e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.516+0000 7fa41affd640 1 -- 172.21.15.40:0/2610215719 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0123f0 con 0x7fa4140030f0 2024-04-26T21:42:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.516+0000 7fa41affd640 1 -- 172.21.15.40:0/2610215719 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c018070 con 0x7fa4140030f0 2024-04-26T21:42:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.516+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560b5e3ff220 con 0x7fa4140030f0 2024-04-26T21:42:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.516+0000 7fa41affd640 1 -- 172.21.15.40:0/2610215719 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0127b0 con 0x7fa4140030f0 2024-04-26T21:42:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.516+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5e3ff7e0 con 0x7fa4140030f0 2024-04-26T21:42:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.517+0000 7fa41affd640 1 -- 172.21.15.40:0/2610215719 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa41c012950 con 0x7fa4140030f0 2024-04-26T21:42:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.517+0000 7fa41affd640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40c06b2d0 0x7fa40c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.518+0000 7fa41affd640 1 -- 172.21.15.40:0/2610215719 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fa41c08a310 con 0x7fa4140030f0 2024-04-26T21:42:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.518+0000 7fa429c74640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40c06b2d0 0x7fa40c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.518+0000 7fa42c178f40 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560b5e4a2c10 0x560b5e537980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.518+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x560b5e537ec0 con 0x560b5e4a2c10 2024-04-26T21:42:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.518+0000 7fa429473640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560b5e4a2c10 0x560b5e537980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.518+0000 7fa429c74640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40c06b2d0 0x7fa40c06d790 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa424009780 tx=0x7fa424011040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.518+0000 7fa429473640 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560b5e4a2c10 0x560b5e537980 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa429473640 1 -- 172.21.15.40:0/2610215719 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000001 [omap-get-keys out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fa420055b60 con 0x560b5e4a2c10 2024-04-26T21:42:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560b5e4a2c10 msgr2=0x560b5e537980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560b5e4a2c10 0x560b5e537980 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40c06b2d0 msgr2=0x7fa40c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40c06b2d0 0x7fa40c06d790 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa424009780 tx=0x7fa424011040 comp rx=0 tx=0).stop 2024-04-26T21:42:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4140030f0 msgr2=0x560b5e400430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 --2- 172.21.15.40:0/2610215719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4140030f0 0x560b5e400430 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa41c000f80 tx=0x7fa41c00e5a0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42a475640 1 -- 172.21.15.40:0/2610215719 reap_dead start 2024-04-26T21:42:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 shutdown_connections 2024-04-26T21:42:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 >> 172.21.15.40:0/2610215719 conn(0x560b5e3fb8a0 msgr2=0x560b5e535ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 shutdown_connections 2024-04-26T21:42:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.519+0000 7fa42c178f40 1 -- 172.21.15.40:0/2610215719 wait complete. 2024-04-26T21:42:25.536 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_openfiles.0 2024-04-26T21:42:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.606+0000 7f7a3d70ff40 1 Processor -- start 2024-04-26T21:42:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.606+0000 7f7a3d70ff40 1 -- start start 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.606+0000 7f7a3d70ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec3802e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.606+0000 7f7a3d70ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 0x55ec37f57a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.606+0000 7f7a3d70ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 0x55ec3801e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.606+0000 7f7a3d70ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ec37efce40 con 0x55ec37f9f420 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.606+0000 7f7a3d70ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ec37f9e660 con 0x55ec38026f70 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.606+0000 7f7a3d70ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ec3801de50 con 0x55ec3802db70 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec3802e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec3802e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57828/0 (socket says 172.21.15.40:57828) 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 -- 172.21.15.40:0/3922899795 learned_addr learned my addr 172.21.15.40:0/3922899795 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3aa0a640 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 0x55ec37f57a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3ba0c640 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 0x55ec3801e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 -- 172.21.15.40:0/3922899795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 msgr2=0x55ec3801e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 0x55ec3801e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 -- 172.21.15.40:0/3922899795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 msgr2=0x55ec37f57a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 0x55ec37f57a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 -- 172.21.15.40:0/3922899795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec3801e690 con 0x55ec37f9f420 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a3b20b640 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec3802e3d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400a770 tx=0x7f7a3400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0a2ce879ee50ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a39a08640 1 -- 172.21.15.40:0/3922899795 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34019070 con 0x55ec37f9f420 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a39a08640 1 -- 172.21.15.40:0/3922899795 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a34007600 con 0x55ec37f9f420 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.607+0000 7f7a39a08640 1 -- 172.21.15.40:0/3922899795 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a340079d0 con 0x55ec37f9f420 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/3922899795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 msgr2=0x55ec3802e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec3802e3d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400a770 tx=0x7f7a3400a860 comp rx=0 tx=0).stop 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/3922899795 shutdown_connections 2024-04-26T21:42:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 0x55ec3801e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 0x55ec37f57a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 --2- 172.21.15.40:0/3922899795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec3802e3d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/3922899795 >> 172.21.15.40:0/3922899795 conn(0x55ec37f7f8a0 msgr2=0x55ec37f80b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/3922899795 shutdown_connections 2024-04-26T21:42:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/3922899795 wait complete. 2024-04-26T21:42:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 Processor -- start 2024-04-26T21:42:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.608+0000 7f7a3d70ff40 1 -- start start 2024-04-26T21:42:25.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3d70ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec380c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3d70ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 0x55ec3800d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3d70ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 0x55ec38027350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3d70ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ec37f56300 con 0x55ec37f9f420 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3d70ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ec380263a0 con 0x55ec38026f70 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3d70ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ec37f56d40 con 0x55ec3802db70 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3b20b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec380c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3b20b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec380c08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57830/0 (socket says 172.21.15.40:57830) 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3b20b640 1 -- 172.21.15.40:0/4211924719 learned_addr learned my addr 172.21.15.40:0/4211924719 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3aa0a640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 0x55ec3800d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3ba0c640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 0x55ec38027350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3b20b640 1 -- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 msgr2=0x55ec38027350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3b20b640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec3802db70 0x55ec38027350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3b20b640 1 -- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 msgr2=0x55ec3800d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3b20b640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec38026f70 0x55ec3800d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.609+0000 7f7a3b20b640 1 -- 172.21.15.40:0/4211924719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec3802e160 con 0x55ec37f9f420 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.610+0000 7f7a3b20b640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec380c08e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400ad60 tx=0x7f7a34003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.610+0000 7f7a38a06640 1 -- 172.21.15.40:0/4211924719 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34019040 con 0x55ec37f9f420 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.610+0000 7f7a38a06640 1 -- 172.21.15.40:0/4211924719 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a340078f0 con 0x55ec37f9f420 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.610+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec3802e440 con 0x55ec37f9f420 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.610+0000 7f7a38a06640 1 -- 172.21.15.40:0/4211924719 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34007ac0 con 0x55ec37f9f420 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.610+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec3802ea00 con 0x55ec37f9f420 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.611+0000 7f7a38a06640 1 -- 172.21.15.40:0/4211924719 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a34024030 con 0x55ec37f9f420 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.611+0000 7f7a38a06640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a1c06b300 0x7f7a1c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.612+0000 7f7a38a06640 1 -- 172.21.15.40:0/4211924719 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f7a3408ae80 con 0x55ec37f9f420 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.612+0000 7f7a3d70ff40 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ec3802f430 0x55ec380c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.612+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55ec380c3ad0 con 0x55ec3802f430 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.612+0000 7f7a3aa0a640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a1c06b300 0x7f7a1c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.612+0000 7f7a3ba0c640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ec3802f430 0x55ec380c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.612+0000 7f7a3aa0a640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a1c06b300 0x7f7a1c06d7c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7a24057000 tx=0x7f7a24051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.612+0000 7f7a3ba0c640 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ec3802f430 0x55ec380c3590 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3ba0c640 1 -- 172.21.15.40:0/4211924719 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f7a2c010250 con 0x55ec3802f430 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ec3802f430 msgr2=0x55ec380c3590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ec3802f430 0x55ec380c3590 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a1c06b300 msgr2=0x7f7a1c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a1c06b300 0x7f7a1c06d7c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7a24057000 tx=0x7f7a24051000 comp rx=0 tx=0).stop 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 msgr2=0x55ec380c08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 --2- 172.21.15.40:0/4211924719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec37f9f420 0x55ec380c08e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400ad60 tx=0x7f7a34003020 comp rx=0 tx=0).stop 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3ba0c640 1 -- 172.21.15.40:0/4211924719 reap_dead start 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 shutdown_connections 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 >> 172.21.15.40:0/4211924719 conn(0x55ec37f7f8a0 msgr2=0x55ec37f7ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 shutdown_connections 2024-04-26T21:42:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.613+0000 7f7a3d70ff40 1 -- 172.21.15.40:0/4211924719 wait complete. 2024-04-26T21:42:25.618 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 500.00000000 2024-04-26T21:42:25.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.700+0000 7fa87721cf40 1 Processor -- start 2024-04-26T21:42:25.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.700+0000 7fa87721cf40 1 -- start start 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa87721cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 0x56020ffdc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa87721cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ff35af0 0x56020ffdc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa87721cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ffde900 0x56020ffd1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa87721cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56020ff4e9f0 con 0x56020ff35af0 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa87721cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56020feaee40 con 0x56020ff05930 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa87721cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56020ffdece0 con 0x56020ffde900 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8738a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ff35af0 0x56020ffdc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8738a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ff35af0 0x56020ffdc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57846/0 (socket says 172.21.15.40:57846) 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8738a5640 1 -- 172.21.15.40:0/3689853948 learned_addr learned my addr 172.21.15.40:0/3689853948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8740a6640 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 0x56020ffdc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8748a7640 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ffde900 0x56020ffd1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8738a5640 1 -- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ffde900 msgr2=0x56020ffd1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8738a5640 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ffde900 0x56020ffd1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8738a5640 1 -- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 msgr2=0x56020ffdc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8738a5640 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 0x56020ffdc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8738a5640 1 -- 172.21.15.40:0/3689853948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020ff32da0 con 0x56020ff35af0 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.701+0000 7fa8748a7640 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ffde900 0x56020ffd1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa8738a5640 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ff35af0 0x56020ffdc750 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00a4f0 tx=0x7fa85c04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f95af9c7a3fd93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa8728a3640 1 -- 172.21.15.40:0/3689853948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c065070 con 0x56020ff35af0 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa8728a3640 1 -- 172.21.15.40:0/3689853948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa85c0549f0 con 0x56020ff35af0 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa8728a3640 1 -- 172.21.15.40:0/3689853948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c059040 con 0x56020ff35af0 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 -- 172.21.15.40:0/3689853948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ff35af0 msgr2=0x56020ffdc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ff35af0 0x56020ffdc750 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00a4f0 tx=0x7fa85c04e000 comp rx=0 tx=0).stop 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 -- 172.21.15.40:0/3689853948 shutdown_connections 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ffde900 0x56020ffd1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ff35af0 0x56020ffdc750 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 --2- 172.21.15.40:0/3689853948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 0x56020ffdc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 -- 172.21.15.40:0/3689853948 >> 172.21.15.40:0/3689853948 conn(0x56020ff06330 msgr2=0x56020ff04f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 -- 172.21.15.40:0/3689853948 shutdown_connections 2024-04-26T21:42:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.702+0000 7fa87721cf40 1 -- 172.21.15.40:0/3689853948 wait complete. 2024-04-26T21:42:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.703+0000 7fa87721cf40 1 Processor -- start 2024-04-26T21:42:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.703+0000 7fa87721cf40 1 -- start start 2024-04-26T21:42:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa87721cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 0x56020ff08d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa87721cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ff35af0 0x56020ff4d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa87721cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ffde900 0x56020ff4d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa87721cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56020ff33640 con 0x56020ffde900 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa87721cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56020ff33080 con 0x56020ff05930 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa87721cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56020ff33360 con 0x56020ff35af0 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ffde900 0x56020ff4d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ffde900 0x56020ff4d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57860/0 (socket says 172.21.15.40:57860) 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 -- 172.21.15.40:0/2499438001 learned_addr learned my addr 172.21.15.40:0/2499438001 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8740a6640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 0x56020ff08d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8738a5640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ff35af0 0x56020ff4d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 -- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ff35af0 msgr2=0x56020ff4d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56020ff35af0 0x56020ff4d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 -- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 msgr2=0x56020ff08d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56020ff05930 0x56020ff08d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 -- 172.21.15.40:0/2499438001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020ffb9900 con 0x56020ffde900 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8748a7640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ffde900 0x56020ff4d990 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa86400f570 tx=0x7fa86400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8718a1640 1 -- 172.21.15.40:0/2499438001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864019020 con 0x56020ffde900 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8718a1640 1 -- 172.21.15.40:0/2499438001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa864015690 con 0x56020ffde900 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa8718a1640 1 -- 172.21.15.40:0/2499438001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864015a60 con 0x56020ffde900 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56020ffd18d0 con 0x56020ffde900 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.704+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020ff4ded0 con 0x56020ffde900 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.706+0000 7fa8718a1640 1 -- 172.21.15.40:0/2499438001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa86401c020 con 0x56020ffde900 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.706+0000 7fa8718a1640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85406b300 0x7fa85406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.706+0000 7fa8718a1640 1 -- 172.21.15.40:0/2499438001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fa86408af60 con 0x56020ffde900 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.706+0000 7fa8740a6640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85406b300 0x7fa85406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.706+0000 7fa87721cf40 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x56020ff4e520 0x56020ffba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.706+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56020ff092a0 con 0x56020ff4e520 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.707+0000 7fa8738a5640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x56020ff4e520 0x56020ffba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.707+0000 7fa8740a6640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85406b300 0x7fa85406d7c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x5602100709d0 tx=0x7fa86c001720 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.707+0000 7fa8738a5640 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x56020ff4e520 0x56020ffba2f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.707+0000 7fa8738a5640 1 -- 172.21.15.40:0/2499438001 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 500.00000000 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fa85c05c040 con 0x56020ff4e520 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x56020ff4e520 msgr2=0x56020ffba2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x56020ff4e520 0x56020ffba2f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85406b300 msgr2=0x7fa85406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa85406b300 0x7fa85406d7c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x5602100709d0 tx=0x7fa86c001720 comp rx=0 tx=0).stop 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ffde900 msgr2=0x56020ff4d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 --2- 172.21.15.40:0/2499438001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56020ffde900 0x56020ff4d990 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa86400f570 tx=0x7fa86400fa30 comp rx=0 tx=0).stop 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa8748a7640 1 -- 172.21.15.40:0/2499438001 reap_dead start 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 shutdown_connections 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 >> 172.21.15.40:0/2499438001 conn(0x56020ff06330 msgr2=0x56020ff50060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 shutdown_connections 2024-04-26T21:42:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.708+0000 7fa87721cf40 1 -- 172.21.15.40:0/2499438001 wait complete. 2024-04-26T21:42:25.714 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds_snaptable 2024-04-26T21:42:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.795+0000 7f416920ef40 1 Processor -- start 2024-04-26T21:42:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.795+0000 7f416920ef40 1 -- start start 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.795+0000 7f416920ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1a678c0 0x5598b1aed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.795+0000 7f416920ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1ad9a80 0x5598b1aedf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.795+0000 7f416920ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1a1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f416920ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5598b19c5e40 con 0x5598b1ad9e60 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f416920ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5598b19c9e80 con 0x5598b1a678c0 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f416920ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5598b1a66570 con 0x5598b1ad9a80 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1a1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1a1d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57872/0 (socket says 172.21.15.40:57872) 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 -- 172.21.15.40:0/3882306202 learned_addr learned my addr 172.21.15.40:0/3882306202 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41660b8640 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1a678c0 0x5598b1aed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41658b7640 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1ad9a80 0x5598b1aedf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 -- 172.21.15.40:0/3882306202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1ad9a80 msgr2=0x5598b1aedf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1ad9a80 0x5598b1aedf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 -- 172.21.15.40:0/3882306202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1a678c0 msgr2=0x5598b1aed530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1a678c0 0x5598b1aed530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 -- 172.21.15.40:0/3882306202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b1a480b0 con 0x5598b1ad9e60 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41668b9640 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1a1d720 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f416000b210 tx=0x7f416000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a0a8e4c2e118b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41648b5640 1 -- 172.21.15.40:0/3882306202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160008070 con 0x5598b1ad9e60 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41648b5640 1 -- 172.21.15.40:0/3882306202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416000d500 con 0x5598b1ad9e60 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.796+0000 7f41648b5640 1 -- 172.21.15.40:0/3882306202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416000d8d0 con 0x5598b1ad9e60 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 -- 172.21.15.40:0/3882306202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 msgr2=0x5598b1a1d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1a1d720 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f416000b210 tx=0x7f416000b6d0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 -- 172.21.15.40:0/3882306202 shutdown_connections 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1a1d720 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1ad9a80 0x5598b1aedf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 --2- 172.21.15.40:0/3882306202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1a678c0 0x5598b1aed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 -- 172.21.15.40:0/3882306202 >> 172.21.15.40:0/3882306202 conn(0x5598b1a1cab0 msgr2=0x5598b1af79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 -- 172.21.15.40:0/3882306202 shutdown_connections 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 -- 172.21.15.40:0/3882306202 wait complete. 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 Processor -- start 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.797+0000 7f416920ef40 1 -- start start 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f416920ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1a678c0 0x5598b1a5e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f416920ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1ad9a80 0x5598b1a5e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f416920ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1ad7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f416920ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5598b1a48350 con 0x5598b1ad9e60 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f416920ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5598b1a48fd0 con 0x5598b1ad9a80 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f416920ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5598b1a489b0 con 0x5598b1a678c0 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f41668b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1ad7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f41668b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1ad7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57888/0 (socket says 172.21.15.40:57888) 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f41668b9640 1 -- 172.21.15.40:0/1191256191 learned_addr learned my addr 172.21.15.40:0/1191256191 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f41658b7640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1ad9a80 0x5598b1a5e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.798+0000 7f41660b8640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1a678c0 0x5598b1a5e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f41668b9640 1 -- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1a678c0 msgr2=0x5598b1a5e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f41668b9640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5598b1a678c0 0x5598b1a5e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f41668b9640 1 -- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1ad9a80 msgr2=0x5598b1a5e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f41668b9640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5598b1ad9a80 0x5598b1a5e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f41668b9640 1 -- 172.21.15.40:0/1191256191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b1a63d60 con 0x5598b1ad9e60 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f41668b9640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1ad7d00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f416000b710 tx=0x7f4160000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f415f7fe640 1 -- 172.21.15.40:0/1191256191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160021030 con 0x5598b1ad9e60 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f415f7fe640 1 -- 172.21.15.40:0/1191256191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4160001260 con 0x5598b1ad9e60 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f415f7fe640 1 -- 172.21.15.40:0/1191256191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416001c650 con 0x5598b1ad9e60 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598b1af6ae0 con 0x5598b1ad9e60 2024-04-26T21:42:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.799+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b1a64000 con 0x5598b1ad9e60 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.800+0000 7f415f7fe640 1 -- 172.21.15.40:0/1191256191 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4160024070 con 0x5598b1ad9e60 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.801+0000 7f415f7fe640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f414806b300 0x7f414806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.801+0000 7f415f7fe640 1 -- 172.21.15.40:0/1191256191 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f416008bbf0 con 0x5598b1ad9e60 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.801+0000 7f41660b8640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f414806b300 0x7f414806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.801+0000 7f416920ef40 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5598b1a60560 0x5598b1aef930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.801+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5598b1aefe70 con 0x5598b1a60560 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.801+0000 7f41658b7640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5598b1a60560 0x5598b1aef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f41660b8640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f414806b300 0x7f414806d7c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4158057000 tx=0x7f4158007030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f41658b7640 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5598b1a60560 0x5598b1aef930 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f41658b7640 1 -- 172.21.15.40:0/1191256191 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds_snaptable [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f415000e040 con 0x5598b1a60560 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5598b1a60560 msgr2=0x5598b1aef930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5598b1a60560 0x5598b1aef930 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f414806b300 msgr2=0x7f414806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f414806b300 0x7f414806d7c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4158057000 tx=0x7f4158007030 comp rx=0 tx=0).stop 2024-04-26T21:42:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 msgr2=0x5598b1ad7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 --2- 172.21.15.40:0/1191256191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5598b1ad9e60 0x5598b1ad7d00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f416000b710 tx=0x7f4160000ed0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f41668b9640 1 -- 172.21.15.40:0/1191256191 reap_dead start 2024-04-26T21:42:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 shutdown_connections 2024-04-26T21:42:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 >> 172.21.15.40:0/1191256191 conn(0x5598b1a1cab0 msgr2=0x5598b1aeabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 shutdown_connections 2024-04-26T21:42:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.802+0000 7f416920ef40 1 -- 172.21.15.40:0/1191256191 wait complete. 2024-04-26T21:42:25.807 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_inotable 2024-04-26T21:42:25.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.887+0000 7f27fae0bf40 1 Processor -- start 2024-04-26T21:42:25.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.887+0000 7f27fae0bf40 1 -- start start 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27fae0bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 0x55e3cd2ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27fae0bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208790 0x55e3cd2b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27fae0bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208bd0 0x55e3cd2b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27fae0bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e3cd17fe40 con 0x55e3cd208790 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27fae0bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e3cd183e80 con 0x7f27e4003ab0 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27fae0bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e3cd2a1f30 con 0x55e3cd208bd0 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f73f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208790 0x55e3cd2b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f73f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208790 0x55e3cd2b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57906/0 (socket says 172.21.15.40:57906) 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f73f6640 1 -- 172.21.15.40:0/1108113400 learned_addr learned my addr 172.21.15.40:0/1108113400 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f7bf7640 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 0x55e3cd2ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f83f8640 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208bd0 0x55e3cd2b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f73f6640 1 -- 172.21.15.40:0/1108113400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208bd0 msgr2=0x55e3cd2b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f73f6640 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208bd0 0x55e3cd2b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f73f6640 1 -- 172.21.15.40:0/1108113400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 msgr2=0x55e3cd2ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f73f6640 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 0x55e3cd2ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.888+0000 7f27f73f6640 1 -- 172.21.15.40:0/1108113400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3cd2aa510 con 0x55e3cd208790 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27f73f6640 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208790 0x55e3cd2b21b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f27e00018b0 tx=0x7f27e00099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec5d95a436a954a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27f63f4640 1 -- 172.21.15.40:0/1108113400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0010660 con 0x55e3cd208790 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27f63f4640 1 -- 172.21.15.40:0/1108113400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e0016020 con 0x55e3cd208790 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27f63f4640 1 -- 172.21.15.40:0/1108113400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0010a60 con 0x55e3cd208790 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 -- 172.21.15.40:0/1108113400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208790 msgr2=0x55e3cd2b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208790 0x55e3cd2b21b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f27e00018b0 tx=0x7f27e00099a0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 -- 172.21.15.40:0/1108113400 shutdown_connections 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208bd0 0x55e3cd2b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208790 0x55e3cd2b21b0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 --2- 172.21.15.40:0/1108113400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 0x55e3cd2ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 -- 172.21.15.40:0/1108113400 >> 172.21.15.40:0/1108113400 conn(0x55e3cd2028a0 msgr2=0x55e3cd1d98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 -- 172.21.15.40:0/1108113400 shutdown_connections 2024-04-26T21:42:25.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.889+0000 7f27fae0bf40 1 -- 172.21.15.40:0/1108113400 wait complete. 2024-04-26T21:42:25.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.890+0000 7f27fae0bf40 1 Processor -- start 2024-04-26T21:42:25.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.890+0000 7f27fae0bf40 1 -- start start 2024-04-26T21:42:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27fae0bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208790 0x55e3cd2933f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27fae0bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208bd0 0x55e3cd2ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27fae0bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 0x55e3cd2aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27fae0bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e3cd2a9890 con 0x55e3cd208bd0 2024-04-26T21:42:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27fae0bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e3cd2b1e40 con 0x7f27e4003ab0 2024-04-26T21:42:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27fae0bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e3cd1db0e0 con 0x55e3cd208790 2024-04-26T21:42:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208bd0 0x55e3cd2ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208bd0 0x55e3cd2ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57918/0 (socket says 172.21.15.40:57918) 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 -- 172.21.15.40:0/3989180924 learned_addr learned my addr 172.21.15.40:0/3989180924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f7bf7640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208790 0x55e3cd2933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f83f8640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 0x55e3cd2aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 -- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208790 msgr2=0x55e3cd2933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3cd208790 0x55e3cd2933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 -- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 msgr2=0x55e3cd2aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4003ab0 0x55e3cd2aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 -- 172.21.15.40:0/3989180924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3cd293930 con 0x55e3cd208bd0 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.891+0000 7f27f73f6640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208bd0 0x55e3cd2ac970 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f27e00018b0 tx=0x55e3cd221550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.892+0000 7f27f53f2640 1 -- 172.21.15.40:0/3989180924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0029070 con 0x55e3cd208bd0 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.892+0000 7f27f53f2640 1 -- 172.21.15.40:0/3989180924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e001ca80 con 0x55e3cd208bd0 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.892+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e3cd2a2850 con 0x55e3cd208bd0 2024-04-26T21:42:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.892+0000 7f27f53f2640 1 -- 172.21.15.40:0/3989180924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0015050 con 0x55e3cd208bd0 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.892+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3cd2a2e10 con 0x55e3cd208bd0 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.893+0000 7f27f53f2640 1 -- 172.21.15.40:0/3989180924 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27e002c070 con 0x55e3cd208bd0 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.893+0000 7f27f53f2640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f27d806b2d0 0x7f27d806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.893+0000 7f27f53f2640 1 -- 172.21.15.40:0/3989180924 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f27e008c450 con 0x55e3cd208bd0 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.894+0000 7f27f7bf7640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f27d806b2d0 0x7f27d806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.894+0000 7f27fae0bf40 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3cd2a2350 0x55e3cd2a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.894+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55e3cd346dd0 con 0x55e3cd2a2350 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.894+0000 7f27f83f8640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3cd2a2350 0x55e3cd2a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.894+0000 7f27f7bf7640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f27d806b2d0 0x7f27d806d790 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55e3cd2ad530 tx=0x7f27f0001730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.894+0000 7f27f83f8640 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3cd2a2350 0x55e3cd2a3e00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27f83f8640 1 -- 172.21.15.40:0/3989180924 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 mds0_inotable [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f27e804d950 con 0x55e3cd2a2350 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3cd2a2350 msgr2=0x55e3cd2a3e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3cd2a2350 0x55e3cd2a3e00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f27d806b2d0 msgr2=0x7f27d806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f27d806b2d0 0x7f27d806d790 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55e3cd2ad530 tx=0x7f27f0001730 comp rx=0 tx=0).stop 2024-04-26T21:42:25.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208bd0 msgr2=0x55e3cd2ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 --2- 172.21.15.40:0/3989180924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e3cd208bd0 0x55e3cd2ac970 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f27e00018b0 tx=0x55e3cd221550 comp rx=0 tx=0).stop 2024-04-26T21:42:25.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27f83f8640 1 -- 172.21.15.40:0/3989180924 reap_dead start 2024-04-26T21:42:25.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 shutdown_connections 2024-04-26T21:42:25.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 >> 172.21.15.40:0/3989180924 conn(0x55e3cd2028a0 msgr2=0x55e3cd1da4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 shutdown_connections 2024-04-26T21:42:25.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.895+0000 7f27fae0bf40 1 -- 172.21.15.40:0/3989180924 wait complete. 2024-04-26T21:42:25.900 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000 2024-04-26T21:42:25.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.981+0000 7f6aa2017f40 1 Processor -- start 2024-04-26T21:42:25.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6aa2017f40 1 -- start start 2024-04-26T21:42:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6aa2017f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5996201c0 0x55c599623940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6aa2017f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c59966b420 0x55c5996f1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6aa2017f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5996f2f70 0x55c59964ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6aa2017f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c5995c8e40 con 0x55c59966b420 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6aa2017f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c59966a660 con 0x55c5996f2f70 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6aa2017f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c5996ea4d0 con 0x55c5996201c0 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6a9f310640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c59966b420 0x55c5996f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6a9f310640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c59966b420 0x55c5996f1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57944/0 (socket says 172.21.15.40:57944) 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.982+0000 7f6a9f310640 1 -- 172.21.15.40:0/549006090 learned_addr learned my addr 172.21.15.40:0/549006090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa0312640 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5996f2f70 0x55c59964ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9fb11640 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5996201c0 0x55c599623940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9f310640 1 -- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5996201c0 msgr2=0x55c599623940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9f310640 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5996201c0 0x55c599623940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9f310640 1 -- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5996f2f70 msgr2=0x55c59964ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9f310640 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5996f2f70 0x55c59964ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9f310640 1 -- 172.21.15.40:0/549006090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5996f8860 con 0x55c59966b420 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa0312640 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5996f2f70 0x55c59964ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9f310640 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c59966b420 0x55c5996f1290 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6a94002e00 tx=0x7f6a9400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bc5b12630ed9d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9e30e640 1 -- 172.21.15.40:0/549006090 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94013030 con 0x55c59966b420 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9e30e640 1 -- 172.21.15.40:0/549006090 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9400bde0 con 0x55c59966b420 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6a9e30e640 1 -- 172.21.15.40:0/549006090 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400f620 con 0x55c59966b420 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa2017f40 1 -- 172.21.15.40:0/549006090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c59966b420 msgr2=0x55c5996f1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa2017f40 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c59966b420 0x55c5996f1290 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6a94002e00 tx=0x7f6a9400b5b0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa2017f40 1 -- 172.21.15.40:0/549006090 shutdown_connections 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa2017f40 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5996f2f70 0x55c59964ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa2017f40 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c59966b420 0x55c5996f1290 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa2017f40 1 --2- 172.21.15.40:0/549006090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5996201c0 0x55c599623940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa2017f40 1 -- 172.21.15.40:0/549006090 >> 172.21.15.40:0/549006090 conn(0x55c59964b8a0 msgr2=0x55c5996fb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.983+0000 7f6aa2017f40 1 -- 172.21.15.40:0/549006090 shutdown_connections 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.984+0000 7f6aa2017f40 1 -- 172.21.15.40:0/549006090 wait complete. 2024-04-26T21:42:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.984+0000 7f6aa2017f40 1 Processor -- start 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.984+0000 7f6aa2017f40 1 -- start start 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6aa2017f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5996201c0 0x55c5996e9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6aa2017f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c59966b420 0x55c59964c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6aa2017f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5996f2f70 0x55c59964cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6aa2017f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c59964b3f0 con 0x55c5996201c0 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6aa2017f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c59964b6d0 con 0x55c59966b420 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6aa2017f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c599623e80 con 0x55c5996f2f70 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5996201c0 0x55c5996e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5996201c0 0x55c5996e9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57952/0 (socket says 172.21.15.40:57952) 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 -- 172.21.15.40:0/22064896 learned_addr learned my addr 172.21.15.40:0/22064896 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9f310640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c59966b420 0x55c59964c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 -- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5996f2f70 msgr2=0x55c59964cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5996f2f70 0x55c59964cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 -- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c59966b420 msgr2=0x55c59964c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c59966b420 0x55c59964c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 -- 172.21.15.40:0/22064896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5996f2ab0 con 0x55c5996201c0 2024-04-26T21:42:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9fb11640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5996201c0 0x55c5996e9750 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800be40 tx=0x7f6a9800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9d30c640 1 -- 172.21.15.40:0/22064896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98007ad0 con 0x55c5996201c0 2024-04-26T21:42:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.985+0000 7f6a9d30c640 1 -- 172.21.15.40:0/22064896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a98007c70 con 0x55c5996201c0 2024-04-26T21:42:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.986+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c59964f890 con 0x55c5996201c0 2024-04-26T21:42:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.986+0000 7f6a9d30c640 1 -- 172.21.15.40:0/22064896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98015660 con 0x55c5996201c0 2024-04-26T21:42:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.986+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5996fba20 con 0x55c5996201c0 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.987+0000 7f6a9d30c640 1 -- 172.21.15.40:0/22064896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a9801e440 con 0x55c5996201c0 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.987+0000 7f6a9d30c640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06b300 0x7f6a8c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.987+0000 7f6a9d30c640 1 -- 172.21.15.40:0/22064896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f6a98089ff0 con 0x55c5996201c0 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.988+0000 7f6aa2017f40 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c59964fb00 0x55c5996eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.988+0000 7f6a9f310640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06b300 0x7f6a8c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.988+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55c5996f73f0 con 0x55c59964fb00 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.988+0000 7f6aa0312640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c59964fb00 0x55c5996eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.988+0000 7f6a9f310640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06b300 0x7f6a8c06d7c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55c59964d790 tx=0x7f6a94009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.988+0000 7f6aa0312640 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c59964fb00 0x55c5996eb410 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa0312640 1 -- 172.21.15.40:0/22064896 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 100.00000000 [omap-get-keys out=155b] v0'0 uv1 ondisk = 0) ==== 156+0+155 (crc 0 0 0) 0x7f6a90051880 con 0x55c59964fb00 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray0_head 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray1_head 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray2_head 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray3_head 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray4_head 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray5_head 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray6_head 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray7_head 2024-04-26T21:42:25.992 INFO:teuthology.orchestra.run.smithi040.stdout:stray8_head 2024-04-26T21:42:25.993 INFO:teuthology.orchestra.run.smithi040.stdout:stray9_head 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c59964fb00 msgr2=0x55c5996eb410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c59964fb00 0x55c5996eb410 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06b300 msgr2=0x7f6a8c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06b300 0x7f6a8c06d7c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55c59964d790 tx=0x7f6a94009000 comp rx=0 tx=0).stop 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5996201c0 msgr2=0x55c5996e9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 --2- 172.21.15.40:0/22064896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5996201c0 0x55c5996e9750 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800be40 tx=0x7f6a9800bf30 comp rx=0 tx=0).stop 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa0312640 1 -- 172.21.15.40:0/22064896 reap_dead start 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 shutdown_connections 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 >> 172.21.15.40:0/22064896 conn(0x55c59964b8a0 msgr2=0x55c59964eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 shutdown_connections 2024-04-26T21:42:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:25.989+0000 7f6aa2017f40 1 -- 172.21.15.40:0/22064896 wait complete. 2024-04-26T21:42:25.995 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_sessionmap 2024-04-26T21:42:26.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.075+0000 7fcb9851ef40 1 Processor -- start 2024-04-26T21:42:26.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.075+0000 7fcb9851ef40 1 -- start start 2024-04-26T21:42:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb9851ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11908690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb9851ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1190df70 0x564c11912cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb9851ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c11914b70 0x564c1187ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb9851ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564c117e3e40 con 0x564c11914b70 2024-04-26T21:42:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb9851ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564c117e7e80 con 0x564c1190df70 2024-04-26T21:42:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb9851ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564c11916b00 con 0x564c11886420 2024-04-26T21:42:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c11914b70 0x564c1187ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c11914b70 0x564c1187ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57958/0 (socket says 172.21.15.40:57958) 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 -- 172.21.15.40:0/2740957870 learned_addr learned my addr 172.21.15.40:0/2740957870 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb94d49640 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1190df70 0x564c11912cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb9554a640 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11908690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 -- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 msgr2=0x564c11908690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11908690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 -- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1190df70 msgr2=0x564c11912cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1190df70 0x564c11912cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 -- 172.21.15.40:0/2740957870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c11869d80 con 0x564c11914b70 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb9554a640 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11908690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb95d4b640 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c11914b70 0x564c1187ffb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcb90001340 tx=0x7fcb9000b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb114903eb9e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb8f7fe640 1 -- 172.21.15.40:0/2740957870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90012020 con 0x564c11914b70 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb8f7fe640 1 -- 172.21.15.40:0/2740957870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9000d690 con 0x564c11914b70 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.076+0000 7fcb8f7fe640 1 -- 172.21.15.40:0/2740957870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000daa0 con 0x564c11914b70 2024-04-26T21:42:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 -- 172.21.15.40:0/2740957870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c11914b70 msgr2=0x564c1187ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c11914b70 0x564c1187ffb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcb90001340 tx=0x7fcb9000b800 comp rx=0 tx=0).stop 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 -- 172.21.15.40:0/2740957870 shutdown_connections 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c11914b70 0x564c1187ffb0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1190df70 0x564c11912cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 --2- 172.21.15.40:0/2740957870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11908690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 -- 172.21.15.40:0/2740957870 >> 172.21.15.40:0/2740957870 conn(0x564c118668a0 msgr2=0x564c11914fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 -- 172.21.15.40:0/2740957870 shutdown_connections 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.077+0000 7fcb9851ef40 1 -- 172.21.15.40:0/2740957870 wait complete. 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb9851ef40 1 Processor -- start 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb9851ef40 1 -- start start 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb9851ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11910290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb9851ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1190df70 0x564c11904da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb9851ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c11914b70 0x564c11911ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb9851ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564c11916d50 con 0x564c1190df70 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb9851ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564c1186b530 con 0x564c11914b70 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb9851ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564c119141f0 con 0x564c11886420 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb94d49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1190df70 0x564c11904da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb94d49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1190df70 0x564c11904da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57964/0 (socket says 172.21.15.40:57964) 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.078+0000 7fcb94d49640 1 -- 172.21.15.40:0/1146054037 learned_addr learned my addr 172.21.15.40:0/1146054037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb95d4b640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c11914b70 0x564c11911ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb9554a640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11910290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb94d49640 1 -- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 msgr2=0x564c11910290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb94d49640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11910290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb94d49640 1 -- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c11914b70 msgr2=0x564c11911ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb94d49640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c11914b70 0x564c11911ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb94d49640 1 -- 172.21.15.40:0/1146054037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c11910c10 con 0x564c1190df70 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb9554a640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c11886420 0x564c11910290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb94d49640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1190df70 0x564c11904da0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcb80025ed0 tx=0x7fcb80002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb8e7fc640 1 -- 172.21.15.40:0/1146054037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80064030 con 0x564c1190df70 2024-04-26T21:42:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb8e7fc640 1 -- 172.21.15.40:0/1146054037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8005b040 con 0x564c1190df70 2024-04-26T21:42:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564c11910ef0 con 0x564c1190df70 2024-04-26T21:42:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb8e7fc640 1 -- 172.21.15.40:0/1146054037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8005a070 con 0x564c1190df70 2024-04-26T21:42:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.079+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c11880290 con 0x564c1190df70 2024-04-26T21:42:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.081+0000 7fcb8e7fc640 1 -- 172.21.15.40:0/1146054037 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb80067030 con 0x564c1190df70 2024-04-26T21:42:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.081+0000 7fcb8e7fc640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb7806b300 0x7fcb7806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.081+0000 7fcb8e7fc640 1 -- 172.21.15.40:0/1146054037 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fcb800ced00 con 0x564c1190df70 2024-04-26T21:42:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.081+0000 7fcb9554a640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb7806b300 0x7fcb7806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.081+0000 7fcb9851ef40 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564c1190f480 0x564c11880fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.081+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x564c119107d0 con 0x564c1190f480 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.081+0000 7fcb95d4b640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564c1190f480 0x564c11880fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9554a640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb7806b300 0x7fcb7806d7c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000f80 tx=0x7fcb88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb95d4b640 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564c1190f480 0x564c11880fa0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb95d4b640 1 -- 172.21.15.40:0/1146054037 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7fcb9000d840 con 0x564c1190f480 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564c1190f480 msgr2=0x564c11880fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564c1190f480 0x564c11880fa0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb7806b300 msgr2=0x7fcb7806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb7806b300 0x7fcb7806d7c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000f80 tx=0x7fcb88008040 comp rx=0 tx=0).stop 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1190df70 msgr2=0x564c11904da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 --2- 172.21.15.40:0/1146054037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1190df70 0x564c11904da0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcb80025ed0 tx=0x7fcb80002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb95d4b640 1 -- 172.21.15.40:0/1146054037 reap_dead start 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 shutdown_connections 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 >> 172.21.15.40:0/1146054037 conn(0x564c118668a0 msgr2=0x564c11867a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 shutdown_connections 2024-04-26T21:42:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.082+0000 7fcb9851ef40 1 -- 172.21.15.40:0/1146054037 wait complete. 2024-04-26T21:42:26.088 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 400.00000000 2024-04-26T21:42:26.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.168+0000 7fd84513af40 1 Processor -- start 2024-04-26T21:42:26.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.168+0000 7fd84513af40 1 -- start start 2024-04-26T21:42:26.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd84513af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a0f1140 0x562d8a0a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd84513af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 0x562d8a173bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd84513af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a17d8c0 0x562d8a174110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd84513af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562d8a0d9690 con 0x562d8a17d8c0 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd84513af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562d8a0d9c50 con 0x562d8a17d4e0 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd84513af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562d8a0d9970 con 0x562d8a0f1140 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a17d8c0 0x562d8a174110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a17d8c0 0x562d8a174110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57992/0 (socket says 172.21.15.40:57992) 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 -- 172.21.15.40:0/241036945 learned_addr learned my addr 172.21.15.40:0/241036945 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd842433640 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 0x562d8a173bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd842c34640 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a0f1140 0x562d8a0a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 -- 172.21.15.40:0/241036945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a0f1140 msgr2=0x562d8a0a7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a0f1140 0x562d8a0a7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 -- 172.21.15.40:0/241036945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 msgr2=0x562d8a173bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 0x562d8a173bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 -- 172.21.15.40:0/241036945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d89fd3cb0 con 0x562d8a17d8c0 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd843435640 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a17d8c0 0x562d8a174110 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd83400b560 tx=0x7fd83400ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a99e8f572b56197 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.169+0000 7fd841431640 1 -- 172.21.15.40:0/241036945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834014070 con 0x562d8a17d8c0 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd841431640 1 -- 172.21.15.40:0/241036945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd834010800 con 0x562d8a17d8c0 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd841431640 1 -- 172.21.15.40:0/241036945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834010c30 con 0x562d8a17d8c0 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 -- 172.21.15.40:0/241036945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a17d8c0 msgr2=0x562d8a174110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a17d8c0 0x562d8a174110 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd83400b560 tx=0x7fd83400ba20 comp rx=0 tx=0).stop 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 -- 172.21.15.40:0/241036945 shutdown_connections 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a17d8c0 0x562d8a174110 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 0x562d8a173bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 --2- 172.21.15.40:0/241036945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a0f1140 0x562d8a0a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 -- 172.21.15.40:0/241036945 >> 172.21.15.40:0/241036945 conn(0x562d8a0d38a0 msgr2=0x562d8a0ab900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 -- 172.21.15.40:0/241036945 shutdown_connections 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.170+0000 7fd84513af40 1 -- 172.21.15.40:0/241036945 wait complete. 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd84513af40 1 Processor -- start 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd84513af40 1 -- start start 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd84513af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a0f1140 0x562d8a17acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd84513af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 0x562d8a16c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd84513af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a17d8c0 0x562d8a0ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd84513af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562d8a177b00 con 0x562d8a0f1140 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd84513af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562d8a176d80 con 0x562d8a17d4e0 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd84513af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562d8a17f830 con 0x562d8a17d8c0 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd842c34640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a0f1140 0x562d8a17acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd842c34640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a0f1140 0x562d8a17acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58000/0 (socket says 172.21.15.40:58000) 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.171+0000 7fd842c34640 1 -- 172.21.15.40:0/3575049861 learned_addr learned my addr 172.21.15.40:0/3575049861 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd842433640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 0x562d8a16c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd842c34640 1 -- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a17d8c0 msgr2=0x562d8a0ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd843435640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a17d8c0 0x562d8a0ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd842c34640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562d8a17d8c0 0x562d8a0ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd842c34640 1 -- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 msgr2=0x562d8a16c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd842c34640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562d8a17d4e0 0x562d8a16c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd842c34640 1 -- 172.21.15.40:0/3575049861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8a0d7290 con 0x562d8a0f1140 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd842c34640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a0f1140 0x562d8a17acd0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0086f0 tx=0x7fd83c008bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd83bfff640 1 -- 172.21.15.40:0/3575049861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c018070 con 0x562d8a0f1140 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd83bfff640 1 -- 172.21.15.40:0/3575049861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd83c006970 con 0x562d8a0f1140 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d8a0d7570 con 0x562d8a0f1140 2024-04-26T21:42:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd83bfff640 1 -- 172.21.15.40:0/3575049861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c006d60 con 0x562d8a0f1140 2024-04-26T21:42:26.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.172+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8a0d7b30 con 0x562d8a0f1140 2024-04-26T21:42:26.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.173+0000 7fd83bfff640 1 -- 172.21.15.40:0/3575049861 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd83c01b070 con 0x562d8a0f1140 2024-04-26T21:42:26.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.174+0000 7fd83bfff640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd82406b300 0x7fd82406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.174+0000 7fd83bfff640 1 -- 172.21.15.40:0/3575049861 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fd83c08a1f0 con 0x562d8a0f1140 2024-04-26T21:42:26.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.174+0000 7fd84513af40 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562d8a1791c0 0x562d8a1701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.174+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x562d8a1795f0 con 0x562d8a1791c0 2024-04-26T21:42:26.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.174+0000 7fd842433640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd82406b300 0x7fd82406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.174+0000 7fd843435640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562d8a1791c0 0x562d8a1701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd842433640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd82406b300 0x7fd82406d7c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd82c058000 tx=0x7fd82c00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd843435640 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562d8a1791c0 0x562d8a1701e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd843435640 1 -- 172.21.15.40:0/3575049861 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 400.00000000 [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fd834010a30 con 0x562d8a1791c0 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562d8a1791c0 msgr2=0x562d8a1701e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562d8a1791c0 0x562d8a1701e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd82406b300 msgr2=0x7fd82406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd82406b300 0x7fd82406d7c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd82c058000 tx=0x7fd82c00a000 comp rx=0 tx=0).stop 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a0f1140 msgr2=0x562d8a17acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 --2- 172.21.15.40:0/3575049861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d8a0f1140 0x562d8a17acd0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0086f0 tx=0x7fd83c008bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd843435640 1 -- 172.21.15.40:0/3575049861 reap_dead start 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 shutdown_connections 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 >> 172.21.15.40:0/3575049861 conn(0x562d8a0d38a0 msgr2=0x562d8a17e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 shutdown_connections 2024-04-26T21:42:26.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.175+0000 7fd84513af40 1 -- 172.21.15.40:0/3575049861 wait complete. 2024-04-26T21:42:26.182 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000.inode 2024-04-26T21:42:26.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.261+0000 7fd1dba92f40 1 Processor -- start 2024-04-26T21:42:26.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1dba92f40 1 -- start start 2024-04-26T21:42:26.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1dba92f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562490c0420 0x5562490a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1dba92f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556249147f70 0x5562490a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1dba92f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55624914eb70 0x556249077c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1dba92f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55624901de40 con 0x55624914eb70 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1dba92f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5562490bf660 con 0x5562490c0420 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1dba92f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5562490bca70 con 0x556249147f70 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1d9d8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55624914eb70 0x556249077c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1d9d8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55624914eb70 0x556249077c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58016/0 (socket says 172.21.15.40:58016) 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.262+0000 7fd1d9d8d640 1 -- 172.21.15.40:0/2679061816 learned_addr learned my addr 172.21.15.40:0/2679061816 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d8d8b640 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556249147f70 0x5562490a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d958c640 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562490c0420 0x5562490a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d9d8d640 1 -- 172.21.15.40:0/2679061816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556249147f70 msgr2=0x5562490a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d9d8d640 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556249147f70 0x5562490a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d9d8d640 1 -- 172.21.15.40:0/2679061816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562490c0420 msgr2=0x5562490a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d9d8d640 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562490c0420 0x5562490a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d9d8d640 1 -- 172.21.15.40:0/2679061816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556249077130 con 0x55624914eb70 2024-04-26T21:42:26.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d9d8d640 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55624914eb70 0x556249077c10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00b5e0 tx=0x7fd1cc00baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f262a6928bcb103 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d37fe640 1 -- 172.21.15.40:0/2679061816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc014020 con 0x55624914eb70 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d37fe640 1 -- 172.21.15.40:0/2679061816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1cc010820 con 0x55624914eb70 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.263+0000 7fd1d37fe640 1 -- 172.21.15.40:0/2679061816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc010c10 con 0x55624914eb70 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 -- 172.21.15.40:0/2679061816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55624914eb70 msgr2=0x556249077c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55624914eb70 0x556249077c10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00b5e0 tx=0x7fd1cc00baa0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 -- 172.21.15.40:0/2679061816 shutdown_connections 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55624914eb70 0x556249077c10 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556249147f70 0x5562490a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 --2- 172.21.15.40:0/2679061816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562490c0420 0x5562490a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 -- 172.21.15.40:0/2679061816 >> 172.21.15.40:0/2679061816 conn(0x5562490a08a0 msgr2=0x556249074e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 -- 172.21.15.40:0/2679061816 shutdown_connections 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 -- 172.21.15.40:0/2679061816 wait complete. 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 Processor -- start 2024-04-26T21:42:26.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.264+0000 7fd1dba92f40 1 -- start start 2024-04-26T21:42:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1dba92f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562490c0420 0x556249142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1dba92f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556249147f70 0x5562491e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1dba92f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55624914eb70 0x5562490a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1dba92f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556249077410 con 0x556249147f70 2024-04-26T21:42:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1dba92f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5562490a5ba0 con 0x55624914eb70 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1dba92f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5562490b9fb0 con 0x5562490c0420 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d8d8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556249147f70 0x5562491e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d8d8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556249147f70 0x5562491e44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58022/0 (socket says 172.21.15.40:58022) 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d8d8b640 1 -- 172.21.15.40:0/317570281 learned_addr learned my addr 172.21.15.40:0/317570281 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d958c640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562490c0420 0x556249142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d9d8d640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55624914eb70 0x5562490a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d8d8b640 1 -- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562490c0420 msgr2=0x556249142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d8d8b640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562490c0420 0x556249142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d8d8b640 1 -- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55624914eb70 msgr2=0x5562490a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d8d8b640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55624914eb70 0x5562490a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d8d8b640 1 -- 172.21.15.40:0/317570281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562491e4d70 con 0x556249147f70 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.265+0000 7fd1d9d8d640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55624914eb70 0x5562490a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.266+0000 7fd1d8d8b640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556249147f70 0x5562491e44a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4058040 tx=0x7fd1c4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.266+0000 7fd1d27fc640 1 -- 172.21.15.40:0/317570281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4065020 con 0x556249147f70 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.266+0000 7fd1d27fc640 1 -- 172.21.15.40:0/317570281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c405b040 con 0x556249147f70 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.266+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5562491e5050 con 0x556249147f70 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.266+0000 7fd1d27fc640 1 -- 172.21.15.40:0/317570281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c405a070 con 0x556249147f70 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.266+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562491e5690 con 0x556249147f70 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.267+0000 7fd1d27fc640 1 -- 172.21.15.40:0/317570281 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1c4068020 con 0x556249147f70 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.267+0000 7fd1d27fc640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1bc06b300 0x7fd1bc06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.268+0000 7fd1d27fc640 1 -- 172.21.15.40:0/317570281 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fd1c40cf410 con 0x556249147f70 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.268+0000 7fd1dba92f40 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562491e5e90 0x5562491e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.268+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5562491e67e0 con 0x5562491e5e90 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.268+0000 7fd1d958c640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1bc06b300 0x7fd1bc06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.268+0000 7fd1d9d8d640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562491e5e90 0x5562491e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.268+0000 7fd1d958c640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1bc06b300 0x7fd1bc06d7c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4003f10 tx=0x7fd1d4003890 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.268+0000 7fd1d9d8d640 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562491e5e90 0x5562491e62a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1d9d8d640 1 -- 172.21.15.40:0/317570281 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7fd1cc0109d0 con 0x5562491e5e90 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562491e5e90 msgr2=0x5562491e62a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562491e5e90 0x5562491e62a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1bc06b300 msgr2=0x7fd1bc06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd1bc06b300 0x7fd1bc06d7c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4003f10 tx=0x7fd1d4003890 comp rx=0 tx=0).stop 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556249147f70 msgr2=0x5562491e44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 --2- 172.21.15.40:0/317570281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556249147f70 0x5562491e44a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4058040 tx=0x7fd1c4002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1d9d8d640 1 -- 172.21.15.40:0/317570281 reap_dead start 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 shutdown_connections 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 >> 172.21.15.40:0/317570281 conn(0x5562490a08a0 msgr2=0x556249074e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 shutdown_connections 2024-04-26T21:42:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.269+0000 7fd1dba92f40 1 -- 172.21.15.40:0/317570281 wait complete. 2024-04-26T21:42:26.275 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-04-26T21:42:26.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.357+0000 7f087111ff40 1 Processor -- start 2024-04-26T21:42:26.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.357+0000 7f087111ff40 1 -- start start 2024-04-26T21:42:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.357+0000 7f087111ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb562e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.357+0000 7f087111ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb545e60 0x560cdb548550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f087111ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb548f30 0x560cdb53d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f087111ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560cdb4d2570 con 0x560cdb545a80 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f087111ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560cdb4d22d0 con 0x560cdb545e60 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f087111ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560cdb4d2af0 con 0x560cdb548f30 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb562e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb562e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58040/0 (socket says 172.21.15.40:58040) 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 -- 172.21.15.40:0/214062150 learned_addr learned my addr 172.21.15.40:0/214062150 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086d992640 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb545e60 0x560cdb548550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e994640 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb548f30 0x560cdb53d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 -- 172.21.15.40:0/214062150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb548f30 msgr2=0x560cdb53d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb548f30 0x560cdb53d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 -- 172.21.15.40:0/214062150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb545e60 msgr2=0x560cdb548550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb545e60 0x560cdb548550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 -- 172.21.15.40:0/214062150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdb3b4cb0 con 0x560cdb545a80 2024-04-26T21:42:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086e193640 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb562e40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f086800a4f0 tx=0x7f0868051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d507d008950fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086c990640 1 -- 172.21.15.40:0/214062150 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868056030 con 0x560cdb545a80 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086c990640 1 -- 172.21.15.40:0/214062150 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0868052a90 con 0x560cdb545a80 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.358+0000 7f086c990640 1 -- 172.21.15.40:0/214062150 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868052e40 con 0x560cdb545a80 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 -- 172.21.15.40:0/214062150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 msgr2=0x560cdb562e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb562e40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f086800a4f0 tx=0x7f0868051000 comp rx=0 tx=0).stop 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 -- 172.21.15.40:0/214062150 shutdown_connections 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb548f30 0x560cdb53d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb545e60 0x560cdb548550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 --2- 172.21.15.40:0/214062150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb562e40 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 -- 172.21.15.40:0/214062150 >> 172.21.15.40:0/214062150 conn(0x560cdb488ab0 msgr2=0x560cdb489720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 -- 172.21.15.40:0/214062150 shutdown_connections 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 -- 172.21.15.40:0/214062150 wait complete. 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 Processor -- start 2024-04-26T21:42:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.359+0000 7f087111ff40 1 -- start start 2024-04-26T21:42:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f087111ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb5ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f087111ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb545e60 0x560cdb53e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f087111ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb548f30 0x560cdb5558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f087111ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560cdb53feb0 con 0x560cdb545a80 2024-04-26T21:42:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f087111ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560cdb4d2570 con 0x560cdb548f30 2024-04-26T21:42:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f087111ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560cdb545230 con 0x560cdb545e60 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f086e193640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb5ee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f086e193640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb5ee5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58046/0 (socket says 172.21.15.40:58046) 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f086e193640 1 -- 172.21.15.40:0/1189052181 learned_addr learned my addr 172.21.15.40:0/1189052181 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f086e994640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb548f30 0x560cdb5558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.360+0000 7f086d992640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb545e60 0x560cdb53e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f086e193640 1 -- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb545e60 msgr2=0x560cdb53e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f086e193640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560cdb545e60 0x560cdb53e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f086e193640 1 -- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb548f30 msgr2=0x560cdb5558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f086e193640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb548f30 0x560cdb5558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f086e193640 1 -- 172.21.15.40:0/1189052181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdb554e10 con 0x560cdb545a80 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f086e994640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560cdb548f30 0x560cdb5558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f086e193640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb5ee5f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f0868005570 tx=0x7f0868002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f085f7fe640 1 -- 172.21.15.40:0/1189052181 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868056070 con 0x560cdb545a80 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f085f7fe640 1 -- 172.21.15.40:0/1189052181 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0868066440 con 0x560cdb545a80 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560cdb5550f0 con 0x560cdb545a80 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f085f7fe640 1 -- 172.21.15.40:0/1189052181 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086806b040 con 0x560cdb545a80 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.361+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cdb48a010 con 0x560cdb545a80 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.363+0000 7f085f7fe640 1 -- 172.21.15.40:0/1189052181 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0868059020 con 0x560cdb545a80 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.363+0000 7f085f7fe640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f085006b2d0 0x7f085006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.363+0000 7f085f7fe640 1 -- 172.21.15.40:0/1189052181 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f08680cfe90 con 0x560cdb545a80 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.363+0000 7f086d992640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f085006b2d0 0x7f085006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.363+0000 7f087111ff40 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560cdb53ee10 0x560cdb53c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.363+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x560cdb5eeb30 con 0x560cdb53ee10 2024-04-26T21:42:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.363+0000 7f086e994640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560cdb53ee10 0x560cdb53c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f086d992640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f085006b2d0 0x7f085006d790 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x560cdb5612b0 tx=0x7f086400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f086e994640 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560cdb53ee10 0x560cdb53c050 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f086e994640 1 -- 172.21.15.40:0/1189052181 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=20b] v0'0 uv6 ondisk = 0) ==== 154+0+20 (crc 0 0 0) 0x7f086000d770 con 0x560cdb53ee10 2024-04-26T21:42:26.368 INFO:teuthology.orchestra.run.smithi040.stdout:subdir_head 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560cdb53ee10 msgr2=0x560cdb53c050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560cdb53ee10 0x560cdb53c050 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f085006b2d0 msgr2=0x7f085006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f085006b2d0 0x7f085006d790 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x560cdb5612b0 tx=0x7f086400b040 comp rx=0 tx=0).stop 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 msgr2=0x560cdb5ee5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 --2- 172.21.15.40:0/1189052181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560cdb545a80 0x560cdb5ee5f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f0868005570 tx=0x7f0868002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f086e994640 1 -- 172.21.15.40:0/1189052181 reap_dead start 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 shutdown_connections 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 >> 172.21.15.40:0/1189052181 conn(0x560cdb488ab0 msgr2=0x560cdb562d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 shutdown_connections 2024-04-26T21:42:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.364+0000 7f087111ff40 1 -- 172.21.15.40:0/1189052181 wait complete. 2024-04-26T21:42:26.371 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2024-04-26T21:42:26.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.451+0000 7fd36a935f40 1 Processor -- start 2024-04-26T21:42:26.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.451+0000 7fd36a935f40 1 -- start start 2024-04-26T21:42:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd36a935f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873c940 0x5653186f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd36a935f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873e420 0x5653187ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd36a935f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x5653187c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd36a935f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56531869ae40 con 0x5653187c86b0 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd36a935f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56531869ee80 con 0x56531873e420 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd36a935f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56531873e860 con 0x56531873c940 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x5653187c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x5653187c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58068/0 (socket says 172.21.15.40:58068) 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 -- 172.21.15.40:0/3326842916 learned_addr learned my addr 172.21.15.40:0/3326842916 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd367c2e640 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873e420 0x5653187ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd36842f640 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873c940 0x5653186f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 -- 172.21.15.40:0/3326842916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873c940 msgr2=0x5653186f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873c940 0x5653186f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 -- 172.21.15.40:0/3326842916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873e420 msgr2=0x5653187ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873e420 0x5653187ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 -- 172.21.15.40:0/3326842916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653187c0520 con 0x5653187c86b0 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd368c30640 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x5653187c6680 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd360025b00 tx=0x7fd360051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b7ad4178c45941 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd366c2c640 1 -- 172.21.15.40:0/3326842916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360067070 con 0x5653187c86b0 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd366c2c640 1 -- 172.21.15.40:0/3326842916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd360050600 con 0x5653187c86b0 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.452+0000 7fd366c2c640 1 -- 172.21.15.40:0/3326842916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360050da0 con 0x5653187c86b0 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 -- 172.21.15.40:0/3326842916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 msgr2=0x5653187c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x5653187c6680 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd360025b00 tx=0x7fd360051000 comp rx=0 tx=0).stop 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 -- 172.21.15.40:0/3326842916 shutdown_connections 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x5653187c6680 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873e420 0x5653187ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 --2- 172.21.15.40:0/3326842916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873c940 0x5653186f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 -- 172.21.15.40:0/3326842916 >> 172.21.15.40:0/3326842916 conn(0x56531871e9b0 msgr2=0x565318739460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 -- 172.21.15.40:0/3326842916 shutdown_connections 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.453+0000 7fd36a935f40 1 -- 172.21.15.40:0/3326842916 wait complete. 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36a935f40 1 Processor -- start 2024-04-26T21:42:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36a935f40 1 -- start start 2024-04-26T21:42:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36a935f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873c940 0x5653187bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36a935f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873e420 0x5653187c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36a935f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x565318860e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36a935f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5653187cadc0 con 0x5653187c86b0 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36a935f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653187c0790 con 0x56531873c940 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36a935f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x565318720230 con 0x56531873e420 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd368c30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x565318860e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd368c30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x565318860e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58076/0 (socket says 172.21.15.40:58076) 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd368c30640 1 -- 172.21.15.40:0/2940291661 learned_addr learned my addr 172.21.15.40:0/2940291661 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.454+0000 7fd36842f640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873c940 0x5653187bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd367c2e640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873e420 0x5653187c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd368c30640 1 -- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873e420 msgr2=0x5653187c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd368c30640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56531873e420 0x5653187c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd368c30640 1 -- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873c940 msgr2=0x5653187bfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd368c30640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56531873c940 0x5653187bfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd368c30640 1 -- 172.21.15.40:0/2940291661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653187c0da0 con 0x5653187c86b0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd368c30640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x565318860e40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd360065040 tx=0x7fd360003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd365c2a640 1 -- 172.21.15.40:0/2940291661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360071070 con 0x5653187c86b0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd365c2a640 1 -- 172.21.15.40:0/2940291661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd360068040 con 0x5653187c86b0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653187c1080 con 0x5653187c86b0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd365c2a640 1 -- 172.21.15.40:0/2940291661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360067070 con 0x5653187c86b0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.455+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653187c1640 con 0x5653187c86b0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.456+0000 7fd365c2a640 1 -- 172.21.15.40:0/2940291661 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd360074070 con 0x5653187c86b0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.457+0000 7fd365c2a640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd34806b2d0 0x7fd34806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.457+0000 7fd365c2a640 1 -- 172.21.15.40:0/2940291661 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fd3600cf650 con 0x5653187c86b0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.457+0000 7fd36a935f40 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5653187c0910 0x5653187c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.457+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5653187c5af0 con 0x5653187c0910 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.457+0000 7fd367c2e640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5653187c0910 0x5653187c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.457+0000 7fd36842f640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd34806b2d0 0x7fd34806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd367c2e640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5653187c0910 0x5653187c2350 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36842f640 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd34806b2d0 0x7fd34806d790 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x56531873a2c0 tx=0x7fd35000e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd367c2e640 1 -- 172.21.15.40:0/2940291661 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x7fd3580107c0 con 0x5653187c0910 2024-04-26T21:42:26.462 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/10000000000.00000000 mtime 2024-04-26T21:42:23.000000+0000, size 0 2024-04-26T21:42:26.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5653187c0910 msgr2=0x5653187c2350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5653187c0910 0x5653187c2350 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd34806b2d0 msgr2=0x7fd34806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd34806b2d0 0x7fd34806d790 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x56531873a2c0 tx=0x7fd35000e040 comp rx=0 tx=0).stop 2024-04-26T21:42:26.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 msgr2=0x565318860e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 --2- 172.21.15.40:0/2940291661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653187c86b0 0x565318860e40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd360065040 tx=0x7fd360003330 comp rx=0 tx=0).stop 2024-04-26T21:42:26.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd368c30640 1 -- 172.21.15.40:0/2940291661 reap_dead start 2024-04-26T21:42:26.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 shutdown_connections 2024-04-26T21:42:26.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 >> 172.21.15.40:0/2940291661 conn(0x56531871e9b0 msgr2=0x565318723c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 shutdown_connections 2024-04-26T21:42:26.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.458+0000 7fd36a935f40 1 -- 172.21.15.40:0/2940291661 wait complete. 2024-04-26T21:42:26.464 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 600.00000000 2024-04-26T21:42:26.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.545+0000 7f491470ff40 1 Processor -- start 2024-04-26T21:42:26.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.545+0000 7f491470ff40 1 -- start start 2024-04-26T21:42:26.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f491470ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49000022a0 0x5569dfaf5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f491470ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dfaf4030 0x5569dfba1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f491470ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf3590 0x5569dfba1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f491470ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5569dfa6fe40 con 0x5569dfaf3590 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f491470ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5569dfa73e80 con 0x5569dfaf4030 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f491470ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5569dfacad20 con 0x7f49000022a0 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4912a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf3590 0x5569dfba1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4912a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf3590 0x5569dfba1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58100/0 (socket says 172.21.15.40:58100) 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4912a0a640 1 -- 172.21.15.40:0/1735734728 learned_addr learned my addr 172.21.15.40:0/1735734728 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4912a0a640 1 -- 172.21.15.40:0/1735734728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49000022a0 msgr2=0x5569dfaf5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4912a0a640 1 --2- 172.21.15.40:0/1735734728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49000022a0 0x5569dfaf5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4911a08640 1 --2- 172.21.15.40:0/1735734728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dfaf4030 0x5569dfba1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4912a0a640 1 -- 172.21.15.40:0/1735734728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dfaf4030 msgr2=0x5569dfba1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4912a0a640 1 --2- 172.21.15.40:0/1735734728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dfaf4030 0x5569dfba1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.546+0000 7f4912a0a640 1 -- 172.21.15.40:0/1735734728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569dfb836c0 con 0x5569dfaf3590 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f4912a0a640 1 --2- 172.21.15.40:0/1735734728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf3590 0x5569dfba1570 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f490000ab20 tx=0x7f490c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817d78e22322e7b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f4910a06640 1 -- 172.21.15.40:0/1735734728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c00f650 con 0x5569dfaf3590 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f4910a06640 1 -- 172.21.15.40:0/1735734728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c00f810 con 0x5569dfaf3590 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f4910a06640 1 -- 172.21.15.40:0/1735734728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c00fc00 con 0x5569dfaf3590 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 -- 172.21.15.40:0/1735734728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf3590 msgr2=0x5569dfba1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 --2- 172.21.15.40:0/1735734728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf3590 0x5569dfba1570 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f490000ab20 tx=0x7f490c0017a0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 -- 172.21.15.40:0/1735734728 shutdown_connections 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 --2- 172.21.15.40:0/1735734728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf3590 0x5569dfba1570 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 --2- 172.21.15.40:0/1735734728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dfaf4030 0x5569dfba1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 --2- 172.21.15.40:0/1735734728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49000022a0 0x5569dfaf5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 -- 172.21.15.40:0/1735734728 >> 172.21.15.40:0/1735734728 conn(0x5569dfaf28a0 msgr2=0x5569dfaf2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 -- 172.21.15.40:0/1735734728 shutdown_connections 2024-04-26T21:42:26.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.547+0000 7f491470ff40 1 -- 172.21.15.40:0/1735734728 wait complete. 2024-04-26T21:42:26.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.548+0000 7f491470ff40 1 Processor -- start 2024-04-26T21:42:26.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.548+0000 7f491470ff40 1 -- start start 2024-04-26T21:42:26.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.548+0000 7f491470ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5569dfaf3590 0x5569dfb0e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.548+0000 7f491470ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf4030 0x5569dfc39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.548+0000 7f491470ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49000022a0 0x5569dfb9bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.548+0000 7f491470ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5569dfaf8180 con 0x5569dfaf4030 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f491470ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5569dfacaf60 con 0x7f49000022a0 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f491470ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5569dfb83930 con 0x5569dfaf3590 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf4030 0x5569dfc39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf4030 0x5569dfc39ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58108/0 (socket says 172.21.15.40:58108) 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 -- 172.21.15.40:0/180152013 learned_addr learned my addr 172.21.15.40:0/180152013 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4912a0a640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49000022a0 0x5569dfb9bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4912209640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5569dfaf3590 0x5569dfb0e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 -- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5569dfaf3590 msgr2=0x5569dfb0e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5569dfaf3590 0x5569dfb0e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 -- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49000022a0 msgr2=0x5569dfb9bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49000022a0 0x5569dfb9bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 -- 172.21.15.40:0/180152013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569dfb9c4e0 con 0x5569dfaf4030 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4912209640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5569dfaf3590 0x5569dfb0e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4912a0a640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49000022a0 0x5569dfb9bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f4911a08640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf4030 0x5569dfc39ce0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00a6a0 tx=0x7f48fc00ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f490b7fe640 1 -- 172.21.15.40:0/180152013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00ac90 con 0x5569dfaf4030 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f490b7fe640 1 -- 172.21.15.40:0/180152013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48fc00ae30 con 0x5569dfaf4030 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5569dfb9c7c0 con 0x5569dfaf4030 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f490b7fe640 1 -- 172.21.15.40:0/180152013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc008be0 con 0x5569dfaf4030 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.549+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569dfb9cd80 con 0x5569dfaf4030 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.551+0000 7f490b7fe640 1 -- 172.21.15.40:0/180152013 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48fc00b430 con 0x5569dfaf4030 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.551+0000 7f490b7fe640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48f406b2d0 0x7f48f406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.551+0000 7f490b7fe640 1 -- 172.21.15.40:0/180152013 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f48fc08b3d0 con 0x5569dfaf4030 2024-04-26T21:42:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.551+0000 7f491470ff40 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5569dfb9a350 0x5569dfb9a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.551+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5569dfb9ac70 con 0x5569dfb9a350 2024-04-26T21:42:26.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.551+0000 7f4912a0a640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5569dfb9a350 0x5569dfb9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.551+0000 7f4912209640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48f406b2d0 0x7f48f406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f4912a0a640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5569dfb9a350 0x5569dfb9a730 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f4912209640 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48f406b2d0 0x7f48f406d790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4904055000 tx=0x7f4904051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f4912a0a640 1 -- 172.21.15.40:0/180152013 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 600.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f490c00f9c0 con 0x5569dfb9a350 2024-04-26T21:42:26.556 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/600.00000000 mtime 2024-04-26T21:42:09.000000+0000, size 0 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5569dfb9a350 msgr2=0x5569dfb9a730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5569dfb9a350 0x5569dfb9a730 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48f406b2d0 msgr2=0x7f48f406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48f406b2d0 0x7f48f406d790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4904055000 tx=0x7f4904051000 comp rx=0 tx=0).stop 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf4030 msgr2=0x5569dfc39ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 --2- 172.21.15.40:0/180152013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569dfaf4030 0x5569dfc39ce0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00a6a0 tx=0x7f48fc00ab60 comp rx=0 tx=0).stop 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f4912a0a640 1 -- 172.21.15.40:0/180152013 reap_dead start 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 shutdown_connections 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 >> 172.21.15.40:0/180152013 conn(0x5569dfaf28a0 msgr2=0x5569dfaf2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 shutdown_connections 2024-04-26T21:42:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.552+0000 7f491470ff40 1 -- 172.21.15.40:0/180152013 wait complete. 2024-04-26T21:42:26.559 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000.inode 2024-04-26T21:42:26.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668e195f40 1 Processor -- start 2024-04-26T21:42:26.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668e195f40 1 -- start start 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668e195f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dfb327f480 0x55dfb3327420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668e195f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb32986e0 0x55dfb3319740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668e195f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb3324820 0x55dfb3319c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668e195f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dfb31f6e40 con 0x55dfb3324820 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668e195f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dfb31fae80 con 0x55dfb32986e0 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668e195f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55dfb3324c00 con 0x55dfb327f480 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668c490640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb3324820 0x55dfb3319c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668c490640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb3324820 0x55dfb3319c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58118/0 (socket says 172.21.15.40:58118) 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.639+0000 7f668c490640 1 -- 172.21.15.40:0/3919839844 learned_addr learned my addr 172.21.15.40:0/3919839844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668b48e640 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb32986e0 0x55dfb3319740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668bc8f640 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dfb327f480 0x55dfb3327420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668c490640 1 -- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dfb327f480 msgr2=0x55dfb3327420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668c490640 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dfb327f480 0x55dfb3327420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668c490640 1 -- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb32986e0 msgr2=0x55dfb3319740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668c490640 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb32986e0 0x55dfb3319740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668c490640 1 -- 172.21.15.40:0/3919839844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb3320c50 con 0x55dfb3324820 2024-04-26T21:42:26.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668b48e640 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb32986e0 0x55dfb3319740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:26.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668c490640 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb3324820 0x55dfb3319c80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6684001650 tx=0x7f6684001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5f0e1f95b69236 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668a48c640 1 -- 172.21.15.40:0/3919839844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684016070 con 0x55dfb3324820 2024-04-26T21:42:26.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668a48c640 1 -- 172.21.15.40:0/3919839844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668400f710 con 0x55dfb3324820 2024-04-26T21:42:26.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668a48c640 1 -- 172.21.15.40:0/3919839844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668400fb20 con 0x55dfb3324820 2024-04-26T21:42:26.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668e195f40 1 -- 172.21.15.40:0/3919839844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb3324820 msgr2=0x55dfb3319c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668e195f40 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb3324820 0x55dfb3319c80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6684001650 tx=0x7f6684001b10 comp rx=0 tx=0).stop 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668e195f40 1 -- 172.21.15.40:0/3919839844 shutdown_connections 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668e195f40 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb3324820 0x55dfb3319c80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668e195f40 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb32986e0 0x55dfb3319740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668e195f40 1 --2- 172.21.15.40:0/3919839844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dfb327f480 0x55dfb3327420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.640+0000 7f668e195f40 1 -- 172.21.15.40:0/3919839844 >> 172.21.15.40:0/3919839844 conn(0x55dfb32798a0 msgr2=0x55dfb3325940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.641+0000 7f668e195f40 1 -- 172.21.15.40:0/3919839844 shutdown_connections 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.641+0000 7f668e195f40 1 -- 172.21.15.40:0/3919839844 wait complete. 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.641+0000 7f668e195f40 1 Processor -- start 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.641+0000 7f668e195f40 1 -- start start 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668e195f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb327f480 0x55dfb33bcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668e195f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dfb32986e0 0x55dfb33bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668e195f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb3324820 0x55dfb33c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668e195f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dfb3320a00 con 0x55dfb327f480 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668e195f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dfb331a1c0 con 0x55dfb3324820 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668e195f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55dfb3323240 con 0x55dfb32986e0 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb327f480 0x55dfb33bcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb327f480 0x55dfb33bcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58132/0 (socket says 172.21.15.40:58132) 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 -- 172.21.15.40:0/2601214553 learned_addr learned my addr 172.21.15.40:0/2601214553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 -- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dfb32986e0 msgr2=0x55dfb33bd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dfb32986e0 0x55dfb33bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 -- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb3324820 msgr2=0x55dfb33c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb3324820 0x55dfb33c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 -- 172.21.15.40:0/2601214553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb33bda70 con 0x55dfb327f480 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668bc8f640 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb327f480 0x55dfb33bcba0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f667c057040 tx=0x7f667c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668948a640 1 -- 172.21.15.40:0/2601214553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c064020 con 0x55dfb327f480 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.642+0000 7f668948a640 1 -- 172.21.15.40:0/2601214553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f667c05a040 con 0x55dfb327f480 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.643+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dfb3294a80 con 0x55dfb327f480 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.643+0000 7f668948a640 1 -- 172.21.15.40:0/2601214553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c059070 con 0x55dfb327f480 2024-04-26T21:42:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.643+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb3295040 con 0x55dfb327f480 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.644+0000 7f668948a640 1 -- 172.21.15.40:0/2601214553 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f667c067020 con 0x55dfb327f480 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.644+0000 7f668948a640 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f667806b2d0 0x7f667806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.644+0000 7f668948a640 1 -- 172.21.15.40:0/2601214553 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f667c0ce3a0 con 0x55dfb327f480 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.644+0000 7f668e195f40 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dfb33197b0 0x55dfb33c0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55dfb33bd620 con 0x55dfb33197b0 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668b48e640 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f667806b2d0 0x7f667806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668c490640 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dfb33197b0 0x55dfb33c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668c490640 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dfb33197b0 0x55dfb33c0700 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668b48e640 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f667806b2d0 0x7f667806d790 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6680001830 tx=0x7f66800010f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668c490640 1 -- 172.21.15.40:0/2601214553 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv15 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7f668400f070 con 0x55dfb33197b0 2024-04-26T21:42:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/1.00000000.inode mtime 2024-04-26T21:42:23.000000+0000, size 582 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dfb33197b0 msgr2=0x55dfb33c0700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dfb33197b0 0x55dfb33c0700 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f667806b2d0 msgr2=0x7f667806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f667806b2d0 0x7f667806d790 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6680001830 tx=0x7f66800010f0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb327f480 msgr2=0x55dfb33bcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 --2- 172.21.15.40:0/2601214553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfb327f480 0x55dfb33bcba0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f667c057040 tx=0x7f667c002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668c490640 1 -- 172.21.15.40:0/2601214553 reap_dead start 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 shutdown_connections 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 >> 172.21.15.40:0/2601214553 conn(0x55dfb32798a0 msgr2=0x55dfb324dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 shutdown_connections 2024-04-26T21:42:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.645+0000 7f668e195f40 1 -- 172.21.15.40:0/2601214553 wait complete. 2024-04-26T21:42:26.652 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000000 2024-04-26T21:42:26.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.733+0000 7f3eaaa65f40 1 Processor -- start 2024-04-26T21:42:26.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.733+0000 7f3eaaa65f40 1 -- start start 2024-04-26T21:42:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3eaaa65f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa6ea80 0x55dc6fa68950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3eaaa65f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc6fa6ee60 0x55dc6fa6be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3eaaa65f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc6fa71f30 0x55dc6f9dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3eaaa65f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dc6f9fb570 con 0x55dc6fa6ea80 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3eaaa65f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dc6f9fb2d0 con 0x55dc6fa6ee60 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3eaaa65f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55dc6f9fbaf0 con 0x55dc6fa71f30 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa6ea80 0x55dc6fa68950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa6ea80 0x55dc6fa68950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58142/0 (socket says 172.21.15.40:58142) 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 -- 172.21.15.40:0/3166572031 learned_addr learned my addr 172.21.15.40:0/3166572031 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 -- 172.21.15.40:0/3166572031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc6fa71f30 msgr2=0x55dc6f9dd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 --2- 172.21.15.40:0/3166572031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc6fa71f30 0x55dc6f9dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 -- 172.21.15.40:0/3166572031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc6fa6ee60 msgr2=0x55dc6fa6be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 --2- 172.21.15.40:0/3166572031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc6fa6ee60 0x55dc6fa6be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 -- 172.21.15.40:0/3166572031 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6f8ddcb0 con 0x55dc6fa6ea80 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea855f640 1 --2- 172.21.15.40:0/3166572031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa6ea80 0x55dc6fa68950 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800a7a0 tx=0x7f3e9800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9eb4cb3266d07f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea6d5c640 1 -- 172.21.15.40:0/3166572031 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98019070 con 0x55dc6fa6ea80 2024-04-26T21:42:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea6d5c640 1 -- 172.21.15.40:0/3166572031 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e98007600 con 0x55dc6fa6ea80 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.734+0000 7f3ea6d5c640 1 -- 172.21.15.40:0/3166572031 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980079d0 con 0x55dc6fa6ea80 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/3166572031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa6ea80 msgr2=0x55dc6fa68950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 --2- 172.21.15.40:0/3166572031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa6ea80 0x55dc6fa68950 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800a7a0 tx=0x7f3e9800a890 comp rx=0 tx=0).stop 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/3166572031 shutdown_connections 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 --2- 172.21.15.40:0/3166572031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc6fa71f30 0x55dc6f9dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 --2- 172.21.15.40:0/3166572031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc6fa6ee60 0x55dc6fa6be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 --2- 172.21.15.40:0/3166572031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa6ea80 0x55dc6fa68950 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/3166572031 >> 172.21.15.40:0/3166572031 conn(0x55dc6f9b1ab0 msgr2=0x55dc6f9b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/3166572031 shutdown_connections 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/3166572031 wait complete. 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 Processor -- start 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.735+0000 7f3eaaa65f40 1 -- start start 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3eaaa65f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc6fa6ea80 0x55dc6fa88050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3eaaa65f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc6fa6ee60 0x55dc6f9b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3eaaa65f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa71f30 0x55dc6f9b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3eaaa65f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dc6fa67200 con 0x55dc6fa71f30 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3eaaa65f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dc6fa71420 con 0x55dc6fa6ee60 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3eaaa65f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55dc6fa719f0 con 0x55dc6fa6ea80 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3ea8d60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa71f30 0x55dc6f9b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3ea8d60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa71f30 0x55dc6f9b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58156/0 (socket says 172.21.15.40:58156) 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3ea8d60640 1 -- 172.21.15.40:0/1010863027 learned_addr learned my addr 172.21.15.40:0/1010863027 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3ea7d5e640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc6fa6ee60 0x55dc6f9b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.736+0000 7f3ea855f640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc6fa6ea80 0x55dc6fa88050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea8d60640 1 -- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc6fa6ea80 msgr2=0x55dc6fa88050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea8d60640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc6fa6ea80 0x55dc6fa88050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea8d60640 1 -- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc6fa6ee60 msgr2=0x55dc6f9b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea8d60640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc6fa6ee60 0x55dc6f9b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea8d60640 1 -- 172.21.15.40:0/1010863027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6fa88590 con 0x55dc6fa71f30 2024-04-26T21:42:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea8d60640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa71f30 0x55dc6f9b28e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000cfd0 tx=0x7f3ea000a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea5d5a640 1 -- 172.21.15.40:0/1010863027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000ae10 con 0x55dc6fa71f30 2024-04-26T21:42:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea5d5a640 1 -- 172.21.15.40:0/1010863027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea0018070 con 0x55dc6fa71f30 2024-04-26T21:42:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3ea5d5a640 1 -- 172.21.15.40:0/1010863027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0011830 con 0x55dc6fa71f30 2024-04-26T21:42:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc6fa6f2a0 con 0x55dc6fa71f30 2024-04-26T21:42:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.737+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6fa89bc0 con 0x55dc6fa71f30 2024-04-26T21:42:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.738+0000 7f3ea5d5a640 1 -- 172.21.15.40:0/1010863027 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ea00119d0 con 0x55dc6fa71f30 2024-04-26T21:42:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.739+0000 7f3ea5d5a640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e9406b2d0 0x7f3e9406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.739+0000 7f3ea5d5a640 1 -- 172.21.15.40:0/1010863027 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f3ea008ac10 con 0x55dc6fa71f30 2024-04-26T21:42:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.739+0000 7f3ea855f640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e9406b2d0 0x7f3e9406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.739+0000 7f3eaaa65f40 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc6fa887a0 0x55dc6fb23370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.739+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55dc6f9de010 con 0x55dc6fa887a0 2024-04-26T21:42:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.739+0000 7f3ea7d5e640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc6fa887a0 0x55dc6fb23370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.739+0000 7f3ea7d5e640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc6fa887a0 0x55dc6fb23370 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.739+0000 7f3ea855f640 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e9406b2d0 0x7f3e9406d790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800a7a0 tx=0x7f3e9800c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3ea7d5e640 1 -- 172.21.15.40:0/1010863027 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f3e9c0544a0 con 0x55dc6fa887a0 2024-04-26T21:42:26.744 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/200.00000000 mtime 2024-04-26T21:42:23.000000+0000, size 90 2024-04-26T21:42:26.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc6fa887a0 msgr2=0x55dc6fb23370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc6fa887a0 0x55dc6fb23370 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e9406b2d0 msgr2=0x7f3e9406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3e9406b2d0 0x7f3e9406d790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800a7a0 tx=0x7f3e9800c000 comp rx=0 tx=0).stop 2024-04-26T21:42:26.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa71f30 msgr2=0x55dc6f9b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 --2- 172.21.15.40:0/1010863027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc6fa71f30 0x55dc6f9b28e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000cfd0 tx=0x7f3ea000a640 comp rx=0 tx=0).stop 2024-04-26T21:42:26.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3ea8d60640 1 -- 172.21.15.40:0/1010863027 reap_dead start 2024-04-26T21:42:26.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 shutdown_connections 2024-04-26T21:42:26.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 >> 172.21.15.40:0/1010863027 conn(0x55dc6f9b1ab0 msgr2=0x55dc6fa8ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 shutdown_connections 2024-04-26T21:42:26.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.740+0000 7f3eaaa65f40 1 -- 172.21.15.40:0/1010863027 wait complete. 2024-04-26T21:42:26.746 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000001 2024-04-26T21:42:26.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.827+0000 7f3590762f40 1 Processor -- start 2024-04-26T21:42:26.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.827+0000 7f3590762f40 1 -- start start 2024-04-26T21:42:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.827+0000 7f3590762f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de28650 0x55a34de3ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.827+0000 7f3590762f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de41510 0x55a34dec0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.827+0000 7f3590762f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 0x55a34ddf99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.827+0000 7f3590762f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a34decdc00 con 0x55a34de28650 2024-04-26T21:42:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f3590762f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a34dd9fe40 con 0x55a34decd820 2024-04-26T21:42:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f3590762f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a34decde60 con 0x55a34de41510 2024-04-26T21:42:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de28650 0x55a34de3ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de28650 0x55a34de3ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58164/0 (socket says 172.21.15.40:58164) 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 -- 172.21.15.40:0/2431746905 learned_addr learned my addr 172.21.15.40:0/2431746905 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358ea5d640 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 0x55a34ddf99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358da5b640 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de41510 0x55a34dec0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 -- 172.21.15.40:0/2431746905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de41510 msgr2=0x55a34dec0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de41510 0x55a34dec0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 -- 172.21.15.40:0/2431746905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 msgr2=0x55a34ddf99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 0x55a34ddf99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 -- 172.21.15.40:0/2431746905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34dec8a30 con 0x55a34de28650 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358e25c640 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de28650 0x55a34de3ca00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f3580001d10 tx=0x7f358000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5347e517dd04dcf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358ca59640 1 -- 172.21.15.40:0/2431746905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580001e60 con 0x55a34de28650 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358ca59640 1 -- 172.21.15.40:0/2431746905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358000bd80 con 0x55a34de28650 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.828+0000 7f358ca59640 1 -- 172.21.15.40:0/2431746905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358000f640 con 0x55a34de28650 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 -- 172.21.15.40:0/2431746905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de28650 msgr2=0x55a34de3ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de28650 0x55a34de3ca00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f3580001d10 tx=0x7f358000b5b0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 -- 172.21.15.40:0/2431746905 shutdown_connections 2024-04-26T21:42:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 0x55a34ddf99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de41510 0x55a34dec0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 --2- 172.21.15.40:0/2431746905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de28650 0x55a34de3ca00 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 -- 172.21.15.40:0/2431746905 >> 172.21.15.40:0/2431746905 conn(0x55a34de228a0 msgr2=0x55a34decc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 -- 172.21.15.40:0/2431746905 shutdown_connections 2024-04-26T21:42:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 -- 172.21.15.40:0/2431746905 wait complete. 2024-04-26T21:42:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 Processor -- start 2024-04-26T21:42:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.829+0000 7f3590762f40 1 -- start start 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f3590762f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de28650 0x55a34de25d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f3590762f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de41510 0x55a34debd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f3590762f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 0x55a34debdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f3590762f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a34decd000 con 0x55a34de41510 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f3590762f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a34deccdb0 con 0x55a34decd820 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f3590762f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a34dec9080 con 0x55a34de28650 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f358da5b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de41510 0x55a34debd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f358da5b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de41510 0x55a34debd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58170/0 (socket says 172.21.15.40:58170) 2024-04-26T21:42:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f358da5b640 1 -- 172.21.15.40:0/3440404310 learned_addr learned my addr 172.21.15.40:0/3440404310 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.830+0000 7f358ea5d640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 0x55a34debdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f358e25c640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de28650 0x55a34de25d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f358da5b640 1 -- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de28650 msgr2=0x55a34de25d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f358da5b640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de28650 0x55a34de25d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f358da5b640 1 -- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 msgr2=0x55a34debdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f358da5b640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a34decd820 0x55a34debdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f358da5b640 1 -- 172.21.15.40:0/3440404310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34dec9d70 con 0x55a34de41510 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f358e25c640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a34de28650 0x55a34de25d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f358da5b640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de41510 0x55a34debd600 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f358400a740 tx=0x7f358400ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f357f7fe640 1 -- 172.21.15.40:0/3440404310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358401a070 con 0x55a34de41510 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f357f7fe640 1 -- 172.21.15.40:0/3440404310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358400ae50 con 0x55a34de41510 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a34deca010 con 0x55a34de41510 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f357f7fe640 1 -- 172.21.15.40:0/3440404310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584008410 con 0x55a34de41510 2024-04-26T21:42:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.831+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34de27660 con 0x55a34de41510 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.832+0000 7f357f7fe640 1 -- 172.21.15.40:0/3440404310 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f358401d070 con 0x55a34de41510 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.833+0000 7f357f7fe640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f357006b2d0 0x7f357006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.833+0000 7f357f7fe640 1 -- 172.21.15.40:0/3440404310 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f358408b060 con 0x55a34de41510 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.833+0000 7f3590762f40 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a34debe4a0 0x55a34de3ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.833+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55a34de3cf40 con 0x55a34debe4a0 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.833+0000 7f358e25c640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f357006b2d0 0x7f357006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.833+0000 7f358ea5d640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a34debe4a0 0x55a34de3ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.833+0000 7f358ea5d640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a34debe4a0 0x55a34de3ca00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.833+0000 7f358e25c640 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f357006b2d0 0x7f357006d790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3580000fd0 tx=0x7f3580009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f358ea5d640 1 -- 172.21.15.40:0/3440404310 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f3588052930 con 0x55a34debe4a0 2024-04-26T21:42:26.837 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/200.00000001 mtime 2024-04-26T21:42:23.000000+0000, size 18075 2024-04-26T21:42:26.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a34debe4a0 msgr2=0x55a34de3ca00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a34debe4a0 0x55a34de3ca00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f357006b2d0 msgr2=0x7f357006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f357006b2d0 0x7f357006d790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3580000fd0 tx=0x7f3580009000 comp rx=0 tx=0).stop 2024-04-26T21:42:26.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de41510 msgr2=0x55a34debd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 --2- 172.21.15.40:0/3440404310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a34de41510 0x55a34debd600 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f358400a740 tx=0x7f358400ac00 comp rx=0 tx=0).stop 2024-04-26T21:42:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f358ea5d640 1 -- 172.21.15.40:0/3440404310 reap_dead start 2024-04-26T21:42:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 shutdown_connections 2024-04-26T21:42:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 >> 172.21.15.40:0/3440404310 conn(0x55a34de228a0 msgr2=0x55a34decbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 shutdown_connections 2024-04-26T21:42:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.834+0000 7f3590762f40 1 -- 172.21.15.40:0/3440404310 wait complete. 2024-04-26T21:42:26.840 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.0 2024-04-26T21:42:26.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.921+0000 7f91a98f6f40 1 Processor -- start 2024-04-26T21:42:26.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.921+0000 7f91a98f6f40 1 -- start start 2024-04-26T21:42:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a98f6f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91940022d0 0x55999d577d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a98f6f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55999d57ee60 0x55999d580ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a98f6f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d4d8a80 0x55999d4d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a98f6f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55999d454e40 con 0x7f91940022d0 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a98f6f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55999d458e80 con 0x55999d57ee60 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a98f6f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55999d582c00 con 0x55999d4d8a80 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91940022d0 0x55999d577d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91940022d0 0x55999d577d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58188/0 (socket says 172.21.15.40:58188) 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 -- 172.21.15.40:0/1271715243 learned_addr learned my addr 172.21.15.40:0/1271715243 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a6bf1640 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55999d57ee60 0x55999d580ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a7bf3640 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d4d8a80 0x55999d4d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 -- 172.21.15.40:0/1271715243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d4d8a80 msgr2=0x55999d4d8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d4d8a80 0x55999d4d8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 -- 172.21.15.40:0/1271715243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55999d57ee60 msgr2=0x55999d580ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55999d57ee60 0x55999d580ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 -- 172.21.15.40:0/1271715243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999d57d990 con 0x7f91940022d0 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a73f2640 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91940022d0 0x55999d577d70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9198025e10 tx=0x7f919804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436a3d705a275e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a5bef640 1 -- 172.21.15.40:0/1271715243 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198005310 con 0x7f91940022d0 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a5bef640 1 -- 172.21.15.40:0/1271715243 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9198065070 con 0x7f91940022d0 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.922+0000 7f91a5bef640 1 -- 172.21.15.40:0/1271715243 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919804d910 con 0x7f91940022d0 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 -- 172.21.15.40:0/1271715243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91940022d0 msgr2=0x55999d577d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91940022d0 0x55999d577d70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9198025e10 tx=0x7f919804f000 comp rx=0 tx=0).stop 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 -- 172.21.15.40:0/1271715243 shutdown_connections 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d4d8a80 0x55999d4d8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55999d57ee60 0x55999d580ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 --2- 172.21.15.40:0/1271715243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91940022d0 0x55999d577d70 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 -- 172.21.15.40:0/1271715243 >> 172.21.15.40:0/1271715243 conn(0x55999d4d78a0 msgr2=0x55999d4ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 -- 172.21.15.40:0/1271715243 shutdown_connections 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.923+0000 7f91a98f6f40 1 -- 172.21.15.40:0/1271715243 wait complete. 2024-04-26T21:42:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a98f6f40 1 Processor -- start 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a98f6f40 1 -- start start 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a98f6f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55999d4d8a80 0x55999d57c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a98f6f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d57ee60 0x55999d560240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a98f6f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91940022d0 0x55999d576360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a98f6f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55999d57df10 con 0x55999d4d8a80 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a98f6f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55999d5807c0 con 0x7f91940022d0 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a98f6f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55999d4ac280 con 0x55999d57ee60 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a73f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55999d4d8a80 0x55999d57c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a73f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55999d4d8a80 0x55999d57c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58198/0 (socket says 172.21.15.40:58198) 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a73f2640 1 -- 172.21.15.40:0/3136148836 learned_addr learned my addr 172.21.15.40:0/3136148836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.924+0000 7f91a7bf3640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91940022d0 0x55999d576360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a6bf1640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d57ee60 0x55999d560240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a73f2640 1 -- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d57ee60 msgr2=0x55999d560240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a73f2640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55999d57ee60 0x55999d560240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a73f2640 1 -- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91940022d0 msgr2=0x55999d576360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a73f2640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91940022d0 0x55999d576360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a73f2640 1 -- 172.21.15.40:0/3136148836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999d574fc0 con 0x55999d4d8a80 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a73f2640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55999d4d8a80 0x55999d57c0e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f919800a700 tx=0x7f91980033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a4bed640 1 -- 172.21.15.40:0/3136148836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919804db10 con 0x55999d4d8a80 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a4bed640 1 -- 172.21.15.40:0/3136148836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919804dcd0 con 0x55999d4d8a80 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55999d57ad60 con 0x55999d4d8a80 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a4bed640 1 -- 172.21.15.40:0/3136148836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198004840 con 0x55999d4d8a80 2024-04-26T21:42:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.925+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999d57b320 con 0x55999d4d8a80 2024-04-26T21:42:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.926+0000 7f91a4bed640 1 -- 172.21.15.40:0/3136148836 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91980049e0 con 0x55999d4d8a80 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.927+0000 7f91a4bed640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918806b300 0x7f918806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.927+0000 7f91a4bed640 1 -- 172.21.15.40:0/3136148836 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f91980d0140 con 0x55999d4d8a80 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.927+0000 7f91a98f6f40 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55999d561120 0x55999d561550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.927+0000 7f91a6bf1640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918806b300 0x7f918806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.927+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55999d584430 con 0x55999d561120 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.927+0000 7f91a7bf3640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55999d561120 0x55999d561550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a6bf1640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918806b300 0x7f918806d7c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55999d5771b0 tx=0x7f9190001720 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a7bf3640 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55999d561120 0x55999d561550 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a7bf3640 1 -- 172.21.15.40:0/3136148836 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds0_openfiles.0 [stat out=16b] v0'0 uv3 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7f91a0009600 con 0x55999d561120 2024-04-26T21:42:26.932 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/mds0_openfiles.0 mtime 2024-04-26T21:42:23.000000+0000, size 0 2024-04-26T21:42:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55999d561120 msgr2=0x55999d561550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55999d561120 0x55999d561550 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918806b300 msgr2=0x7f918806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f918806b300 0x7f918806d7c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55999d5771b0 tx=0x7f9190001720 comp rx=0 tx=0).stop 2024-04-26T21:42:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55999d4d8a80 msgr2=0x55999d57c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 --2- 172.21.15.40:0/3136148836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55999d4d8a80 0x55999d57c0e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f919800a700 tx=0x7f91980033a0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a7bf3640 1 -- 172.21.15.40:0/3136148836 reap_dead start 2024-04-26T21:42:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 shutdown_connections 2024-04-26T21:42:26.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 >> 172.21.15.40:0/3136148836 conn(0x55999d4d78a0 msgr2=0x55999d57fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 shutdown_connections 2024-04-26T21:42:26.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:26.928+0000 7f91a98f6f40 1 -- 172.21.15.40:0/3136148836 wait complete. 2024-04-26T21:42:26.935 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 500.00000000 2024-04-26T21:42:27.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.015+0000 7f391aa0cf40 1 Processor -- start 2024-04-26T21:42:27.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.015+0000 7f391aa0cf40 1 -- start start 2024-04-26T21:42:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f391aa0cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb507ca1c0 0x55bb5089bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f391aa0cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb50815420 0x55bb507f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f391aa0cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb5089cf70 0x55bb507cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f391aa0cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb50772e40 con 0x55bb5089cf70 2024-04-26T21:42:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f391aa0cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bb50814660 con 0x55bb50815420 2024-04-26T21:42:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f391aa0cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bb5089c060 con 0x55bb507ca1c0 2024-04-26T21:42:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3917f9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb5089cf70 0x55bb507cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3917f9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb5089cf70 0x55bb507cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58208/0 (socket says 172.21.15.40:58208) 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3917f9b640 1 -- 172.21.15.40:0/3511511900 learned_addr learned my addr 172.21.15.40:0/3511511900 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3916f99640 1 --2- 172.21.15.40:0/3511511900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb50815420 0x55bb507f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3917f9b640 1 -- 172.21.15.40:0/3511511900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb507ca1c0 msgr2=0x55bb5089bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3917f9b640 1 --2- 172.21.15.40:0/3511511900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb507ca1c0 0x55bb5089bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3917f9b640 1 -- 172.21.15.40:0/3511511900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb50815420 msgr2=0x55bb507f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3917f9b640 1 --2- 172.21.15.40:0/3511511900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb50815420 0x55bb507f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.016+0000 7f3917f9b640 1 -- 172.21.15.40:0/3511511900 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb507faad0 con 0x55bb5089cf70 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f3917f9b640 1 --2- 172.21.15.40:0/3511511900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb5089cf70 0x55bb507cd880 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f391000a560 tx=0x7f3910050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966b38ec5921dfb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f3915f97640 1 -- 172.21.15.40:0/3511511900 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910055030 con 0x55bb5089cf70 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f3915f97640 1 -- 172.21.15.40:0/3511511900 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3910051ad0 con 0x55bb5089cf70 2024-04-26T21:42:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f3915f97640 1 -- 172.21.15.40:0/3511511900 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910065480 con 0x55bb5089cf70 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 -- 172.21.15.40:0/3511511900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb5089cf70 msgr2=0x55bb507cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 --2- 172.21.15.40:0/3511511900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb5089cf70 0x55bb507cd880 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f391000a560 tx=0x7f3910050000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 -- 172.21.15.40:0/3511511900 shutdown_connections 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 --2- 172.21.15.40:0/3511511900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb5089cf70 0x55bb507cd880 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 --2- 172.21.15.40:0/3511511900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb50815420 0x55bb507f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 --2- 172.21.15.40:0/3511511900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb507ca1c0 0x55bb5089bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 -- 172.21.15.40:0/3511511900 >> 172.21.15.40:0/3511511900 conn(0x55bb507f58a0 msgr2=0x55bb508a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 -- 172.21.15.40:0/3511511900 shutdown_connections 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.017+0000 7f391aa0cf40 1 -- 172.21.15.40:0/3511511900 wait complete. 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.018+0000 7f391aa0cf40 1 Processor -- start 2024-04-26T21:42:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.018+0000 7f391aa0cf40 1 -- start start 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391aa0cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb507ca1c0 0x55bb507cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391aa0cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb50815420 0x55bb5093cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391aa0cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb5089cf70 0x55bb5093d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391aa0cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb507f92a0 con 0x55bb50815420 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391aa0cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bb507f9000 con 0x55bb507ca1c0 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391aa0cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bb507f9580 con 0x55bb5089cf70 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb50815420 0x55bb5093cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb50815420 0x55bb5093cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58222/0 (socket says 172.21.15.40:58222) 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 -- 172.21.15.40:0/171761447 learned_addr learned my addr 172.21.15.40:0/171761447 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391779a640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb507ca1c0 0x55bb507cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3917f9b640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb5089cf70 0x55bb5093d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 -- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb5089cf70 msgr2=0x55bb5093d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb5089cf70 0x55bb5093d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 -- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb507ca1c0 msgr2=0x55bb507cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb507ca1c0 0x55bb507cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 -- 172.21.15.40:0/171761447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb508a5ab0 con 0x55bb50815420 2024-04-26T21:42:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3916f99640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb50815420 0x55bb5093cce0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f390c00cdb0 tx=0x7f390c00a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3914f95640 1 -- 172.21.15.40:0/171761447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c008040 con 0x55bb50815420 2024-04-26T21:42:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3914f95640 1 -- 172.21.15.40:0/171761447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390c014020 con 0x55bb50815420 2024-04-26T21:42:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bb508a5d90 con 0x55bb50815420 2024-04-26T21:42:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f3914f95640 1 -- 172.21.15.40:0/171761447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c00f400 con 0x55bb50815420 2024-04-26T21:42:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.019+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5089f420 con 0x55bb50815420 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.021+0000 7f3914f95640 1 -- 172.21.15.40:0/171761447 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f390c00f5a0 con 0x55bb50815420 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.021+0000 7f3914f95640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406b300 0x7f390406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.021+0000 7f3914f95640 1 -- 172.21.15.40:0/171761447 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f390c08a400 con 0x55bb50815420 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.021+0000 7f391779a640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406b300 0x7f390406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.021+0000 7f391aa0cf40 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55bb507cdfb0 0x55bb5093d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.021+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55bb50899bf0 con 0x55bb507cdfb0 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.022+0000 7f3917f9b640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55bb507cdfb0 0x55bb5093d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.022+0000 7f391779a640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406b300 0x7f390406d7c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x55bb5089b5d0 tx=0x7f390800c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.022+0000 7f3917f9b640 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55bb507cdfb0 0x55bb5093d760 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.022+0000 7f3917f9b640 1 -- 172.21.15.40:0/171761447 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 500.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f3910052d10 con 0x55bb507cdfb0 2024-04-26T21:42:27.028 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/500.00000000 mtime 2024-04-26T21:42:09.000000+0000, size 90 2024-04-26T21:42:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.022+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55bb507cdfb0 msgr2=0x55bb5093d760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.022+0000 7f391aa0cf40 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55bb507cdfb0 0x55bb5093d760 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406b300 msgr2=0x7f390406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f391aa0cf40 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406b300 0x7f390406d7c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x55bb5089b5d0 tx=0x7f390800c040 comp rx=0 tx=0).stop 2024-04-26T21:42:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb50815420 msgr2=0x55bb5093cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f391aa0cf40 1 --2- 172.21.15.40:0/171761447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb50815420 0x55bb5093cce0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f390c00cdb0 tx=0x7f390c00a5d0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f3917f9b640 1 -- 172.21.15.40:0/171761447 reap_dead start 2024-04-26T21:42:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 shutdown_connections 2024-04-26T21:42:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 >> 172.21.15.40:0/171761447 conn(0x55bb507f58a0 msgr2=0x55bb5089a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 shutdown_connections 2024-04-26T21:42:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.023+0000 7f391aa0cf40 1 -- 172.21.15.40:0/171761447 wait complete. 2024-04-26T21:42:27.031 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds_snaptable 2024-04-26T21:42:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.111+0000 7f5b15ab1f40 1 Processor -- start 2024-04-26T21:42:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.111+0000 7f5b15ab1f40 1 -- start start 2024-04-26T21:42:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b15ab1f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b4d4650 0x55f68b56ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b15ab1f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 0x55f68b4a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b15ab1f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b579820 0x55f68b4a5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b15ab1f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f68b44be40 con 0x55f68b579820 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b15ab1f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f68b44fe80 con 0x55f68b4ed510 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b15ab1f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f68b579c00 con 0x55f68b4d4650 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b13dae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b579820 0x55f68b4a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b13dae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b579820 0x55f68b4a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58230/0 (socket says 172.21.15.40:58230) 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b13dae640 1 -- 172.21.15.40:0/194566583 learned_addr learned my addr 172.21.15.40:0/194566583 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b12dac640 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 0x55f68b4a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b135ad640 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b4d4650 0x55f68b56ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b13dae640 1 -- 172.21.15.40:0/194566583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b4d4650 msgr2=0x55f68b56ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b13dae640 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b4d4650 0x55f68b56ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b13dae640 1 -- 172.21.15.40:0/194566583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 msgr2=0x55f68b4a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b13dae640 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 0x55f68b4a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.112+0000 7f5b13dae640 1 -- 172.21.15.40:0/194566583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68b56d710 con 0x55f68b579820 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b13dae640 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b579820 0x55f68b4a5520 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5b04001700 tx=0x7f5b04001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3093216d54a36747 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b11daa640 1 -- 172.21.15.40:0/194566583 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04016070 con 0x55f68b579820 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b11daa640 1 -- 172.21.15.40:0/194566583 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b04004520 con 0x55f68b579820 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b11daa640 1 -- 172.21.15.40:0/194566583 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04004910 con 0x55f68b579820 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/194566583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b579820 msgr2=0x55f68b4a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b579820 0x55f68b4a5520 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5b04001700 tx=0x7f5b04001bc0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/194566583 shutdown_connections 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b579820 0x55f68b4a5520 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 0x55f68b4a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 --2- 172.21.15.40:0/194566583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b4d4650 0x55f68b56ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/194566583 >> 172.21.15.40:0/194566583 conn(0x55f68b4ce8a0 msgr2=0x55f68b4a2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/194566583 shutdown_connections 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.113+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/194566583 wait complete. 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.114+0000 7f5b15ab1f40 1 Processor -- start 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.114+0000 7f5b15ab1f40 1 -- start start 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b15ab1f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b4d4650 0x55f68b578510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b15ab1f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 0x55f68b571890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b15ab1f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b579820 0x55f68b56cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b15ab1f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f68b4a6f40 con 0x55f68b4d4650 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b15ab1f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f68b4eaef0 con 0x55f68b4ed510 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b15ab1f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f68b571310 con 0x55f68b579820 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b4d4650 0x55f68b578510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b4d4650 0x55f68b578510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58246/0 (socket says 172.21.15.40:58246) 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 -- 172.21.15.40:0/3893887123 learned_addr learned my addr 172.21.15.40:0/3893887123 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b12dac640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 0x55f68b571890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b13dae640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b579820 0x55f68b56cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 -- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b579820 msgr2=0x55f68b56cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f68b579820 0x55f68b56cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 -- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 msgr2=0x55f68b571890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f68b4ed510 0x55f68b571890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 -- 172.21.15.40:0/3893887123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68b569790 con 0x55f68b4d4650 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b135ad640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b4d4650 0x55f68b578510 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00dfa0 tx=0x7f5b0c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b10da8640 1 -- 172.21.15.40:0/3893887123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c011480 con 0x55f68b4d4650 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b10da8640 1 -- 172.21.15.40:0/3893887123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c016070 con 0x55f68b4d4650 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f68b5691d0 con 0x55f68b4d4650 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b10da8640 1 -- 172.21.15.40:0/3893887123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c011860 con 0x55f68b4d4650 2024-04-26T21:42:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.115+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68b4eca90 con 0x55f68b4d4650 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.117+0000 7f5b10da8640 1 -- 172.21.15.40:0/3893887123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b0c011a00 con 0x55f68b4d4650 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.117+0000 7f5b10da8640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5af406b2d0 0x7f5af406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.117+0000 7f5b10da8640 1 -- 172.21.15.40:0/3893887123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f5b0c08a820 con 0x55f68b4d4650 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.117+0000 7f5b12dac640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5af406b2d0 0x7f5af406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.117+0000 7f5b15ab1f40 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f68b56d140 0x55f68b57adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.117+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f68b57b300 con 0x55f68b56d140 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.118+0000 7f5b13dae640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f68b56d140 0x55f68b57adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.118+0000 7f5b12dac640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5af406b2d0 0x7f5af406d790 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5afc057000 tx=0x7f5afc051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.118+0000 7f5b13dae640 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f68b56d140 0x55f68b57adc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.118+0000 7f5b13dae640 1 -- 172.21.15.40:0/3893887123 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds_snaptable [stat out=16b] v0'0 uv1 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f5b0400f070 con 0x55f68b56d140 2024-04-26T21:42:27.123 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/mds_snaptable mtime 2024-04-26T21:42:09.000000+0000, size 70 2024-04-26T21:42:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.118+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f68b56d140 msgr2=0x55f68b57adc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.118+0000 7f5b15ab1f40 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f68b56d140 0x55f68b57adc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5af406b2d0 msgr2=0x7f5af406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b15ab1f40 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5af406b2d0 0x7f5af406d790 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5afc057000 tx=0x7f5afc051000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b4d4650 msgr2=0x55f68b578510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b15ab1f40 1 --2- 172.21.15.40:0/3893887123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f68b4d4650 0x55f68b578510 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00dfa0 tx=0x7f5b0c00b5b0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b13dae640 1 -- 172.21.15.40:0/3893887123 reap_dead start 2024-04-26T21:42:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 shutdown_connections 2024-04-26T21:42:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 >> 172.21.15.40:0/3893887123 conn(0x55f68b4ce8a0 msgr2=0x55f68b56bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 shutdown_connections 2024-04-26T21:42:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.119+0000 7f5b15ab1f40 1 -- 172.21.15.40:0/3893887123 wait complete. 2024-04-26T21:42:27.125 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_inotable 2024-04-26T21:42:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.205+0000 7fb795475f40 1 Processor -- start 2024-04-26T21:42:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb795475f40 1 -- start start 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb795475f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a589140 0x55bd3a60c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb795475f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bd3a6154e0 0x55bd3a588580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb795475f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd3a6158c0 0x55bd3a588ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb795475f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bd3a571690 con 0x55bd3a589140 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb795475f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bd3a571c50 con 0x55bd3a6158c0 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb795475f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bd3a571970 con 0x55bd3a6154e0 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb792f6f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a589140 0x55bd3a60c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb792f6f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a589140 0x55bd3a60c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58266/0 (socket says 172.21.15.40:58266) 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb792f6f640 1 -- 172.21.15.40:0/2097780528 learned_addr learned my addr 172.21.15.40:0/2097780528 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb792f6f640 1 -- 172.21.15.40:0/2097780528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bd3a6154e0 msgr2=0x55bd3a588580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb792f6f640 1 --2- 172.21.15.40:0/2097780528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bd3a6154e0 0x55bd3a588580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb792f6f640 1 -- 172.21.15.40:0/2097780528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd3a6158c0 msgr2=0x55bd3a588ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb792f6f640 1 --2- 172.21.15.40:0/2097780528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd3a6158c0 0x55bd3a588ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.206+0000 7fb792f6f640 1 -- 172.21.15.40:0/2097780528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd3a46bcb0 con 0x55bd3a589140 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb792f6f640 1 --2- 172.21.15.40:0/2097780528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a589140 0x55bd3a60c930 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00a530 tx=0x7fb78c00a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbabca87a889aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb79176c640 1 -- 172.21.15.40:0/2097780528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0115f0 con 0x55bd3a589140 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb79176c640 1 -- 172.21.15.40:0/2097780528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c018070 con 0x55bd3a589140 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb79176c640 1 -- 172.21.15.40:0/2097780528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0119b0 con 0x55bd3a589140 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 -- 172.21.15.40:0/2097780528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a589140 msgr2=0x55bd3a60c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 --2- 172.21.15.40:0/2097780528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a589140 0x55bd3a60c930 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00a530 tx=0x7fb78c00a9f0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 -- 172.21.15.40:0/2097780528 shutdown_connections 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 --2- 172.21.15.40:0/2097780528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd3a6158c0 0x55bd3a588ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 --2- 172.21.15.40:0/2097780528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bd3a6154e0 0x55bd3a588580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 --2- 172.21.15.40:0/2097780528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a589140 0x55bd3a60c930 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 -- 172.21.15.40:0/2097780528 >> 172.21.15.40:0/2097780528 conn(0x55bd3a56b8a0 msgr2=0x55bd3a542990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 -- 172.21.15.40:0/2097780528 shutdown_connections 2024-04-26T21:42:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.207+0000 7fb795475f40 1 -- 172.21.15.40:0/2097780528 wait complete. 2024-04-26T21:42:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.208+0000 7fb795475f40 1 Processor -- start 2024-04-26T21:42:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.208+0000 7fb795475f40 1 -- start start 2024-04-26T21:42:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb795475f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd3a589140 0x55bd3a585970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb795475f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a6154e0 0x55bd3a6082f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb795475f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bd3a6158c0 0x55bd3a616540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb795475f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bd3a6143a0 con 0x55bd3a6154e0 2024-04-26T21:42:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb795475f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bd3a618bf0 con 0x55bd3a589140 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb795475f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bd3a60d180 con 0x55bd3a6158c0 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a6154e0 0x55bd3a6082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a6154e0 0x55bd3a6082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58280/0 (socket says 172.21.15.40:58280) 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 -- 172.21.15.40:0/3145354172 learned_addr learned my addr 172.21.15.40:0/3145354172 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb792f6f640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd3a589140 0x55bd3a585970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb793770640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bd3a6158c0 0x55bd3a616540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 -- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bd3a6158c0 msgr2=0x55bd3a616540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bd3a6158c0 0x55bd3a616540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 -- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd3a589140 msgr2=0x55bd3a585970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd3a589140 0x55bd3a585970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 -- 172.21.15.40:0/3145354172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd3a5430b0 con 0x55bd3a6154e0 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb79276e640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a6154e0 0x55bd3a6082f0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00a4e0 tx=0x7fb77c046500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb78bfff640 1 -- 172.21.15.40:0/3145354172 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c065020 con 0x55bd3a6154e0 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb78bfff640 1 -- 172.21.15.40:0/3145354172 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c060670 con 0x55bd3a6154e0 2024-04-26T21:42:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bd3a587560 con 0x55bd3a6154e0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.209+0000 7fb78bfff640 1 -- 172.21.15.40:0/3145354172 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c05a040 con 0x55bd3a6154e0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.210+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd3a542dd0 con 0x55bd3a6154e0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.211+0000 7fb78bfff640 1 -- 172.21.15.40:0/3145354172 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb77c068020 con 0x55bd3a6154e0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.211+0000 7fb78bfff640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb77406b2d0 0x7fb77406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.211+0000 7fb78bfff640 1 -- 172.21.15.40:0/3145354172 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fb77c0cf240 con 0x55bd3a6154e0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.212+0000 7fb792f6f640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb77406b2d0 0x7fb77406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.212+0000 7fb795475f40 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bd3a618060 0x55bd3a6b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.212+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55bd3a6b2ba0 con 0x55bd3a618060 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.212+0000 7fb793770640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bd3a618060 0x55bd3a6b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.212+0000 7fb792f6f640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb77406b2d0 0x7fb77406d790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb78c000f80 tx=0x7fb78c011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.212+0000 7fb793770640 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bd3a618060 0x55bd3a6b2660 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb793770640 1 -- 172.21.15.40:0/3145354172 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 mds0_inotable [stat out=16b] v0'0 uv3 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7fb784009610 con 0x55bd3a618060 2024-04-26T21:42:27.216 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/mds0_inotable mtime 2024-04-26T21:42:23.000000+0000, size 34 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bd3a618060 msgr2=0x55bd3a6b2660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bd3a618060 0x55bd3a6b2660 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb77406b2d0 msgr2=0x7fb77406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb77406b2d0 0x7fb77406d790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb78c000f80 tx=0x7fb78c011000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a6154e0 msgr2=0x55bd3a6082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 --2- 172.21.15.40:0/3145354172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bd3a6154e0 0x55bd3a6082f0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00a4e0 tx=0x7fb77c046500 comp rx=0 tx=0).stop 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb793770640 1 -- 172.21.15.40:0/3145354172 reap_dead start 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 shutdown_connections 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 >> 172.21.15.40:0/3145354172 conn(0x55bd3a56b8a0 msgr2=0x55bd3a60c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 shutdown_connections 2024-04-26T21:42:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.213+0000 7fb795475f40 1 -- 172.21.15.40:0/3145354172 wait complete. 2024-04-26T21:42:27.219 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000 2024-04-26T21:42:27.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3e03e0f40 1 Processor -- start 2024-04-26T21:42:27.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3e03e0f40 1 -- start start 2024-04-26T21:42:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3e03e0f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 0x55f6d18b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3e03e0f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d189ee60 0x55f6d18a1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3e03e0f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3e03e0f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f6d182b570 con 0x55f6d18a1f30 2024-04-26T21:42:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3e03e0f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f6d182b2d0 con 0x55f6d189ee60 2024-04-26T21:42:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3e03e0f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f6d182baf0 con 0x55f6d189ea80 2024-04-26T21:42:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3de6db640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3de6db640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58296/0 (socket says 172.21.15.40:58296) 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.300+0000 7fc3de6db640 1 -- 172.21.15.40:0/1225577875 learned_addr learned my addr 172.21.15.40:0/1225577875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3ddeda640 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 0x55f6d18b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3dd6d9640 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d189ee60 0x55f6d18a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3de6db640 1 -- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 msgr2=0x55f6d18b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3de6db640 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 0x55f6d18b8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3de6db640 1 -- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d189ee60 msgr2=0x55f6d18a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3de6db640 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d189ee60 0x55f6d18a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3de6db640 1 -- 172.21.15.40:0/1225577875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d170dcb0 con 0x55f6d18a1f30 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3ddeda640 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 0x55f6d18b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3de6db640 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb50 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0054040 tx=0x7fc3d0050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e239e4f90502ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3d7fff640 1 -- 172.21.15.40:0/1225577875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0055070 con 0x55f6d18a1f30 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3d7fff640 1 -- 172.21.15.40:0/1225577875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d0051bc0 con 0x55f6d18a1f30 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.301+0000 7fc3d7fff640 1 -- 172.21.15.40:0/1225577875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0064440 con 0x55f6d18a1f30 2024-04-26T21:42:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1225577875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d18a1f30 msgr2=0x55f6d189bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb50 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0054040 tx=0x7fc3d0050000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1225577875 shutdown_connections 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb50 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d189ee60 0x55f6d18a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 --2- 172.21.15.40:0/1225577875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 0x55f6d18b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1225577875 >> 172.21.15.40:0/1225577875 conn(0x55f6d17e1ab0 msgr2=0x55f6d17e2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1225577875 shutdown_connections 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1225577875 wait complete. 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 Processor -- start 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.302+0000 7fc3e03e0f40 1 -- start start 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3e03e0f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 0x55f6d18b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3e03e0f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d189ee60 0x55f6d18108e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3e03e0f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3e03e0f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f6d180e890 con 0x55f6d189ee60 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3e03e0f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f6d1898bf0 con 0x55f6d18a1f30 2024-04-26T21:42:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3e03e0f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f6d18b9310 con 0x55f6d189ea80 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3dd6d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d189ee60 0x55f6d18108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3dd6d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d189ee60 0x55f6d18108e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58304/0 (socket says 172.21.15.40:58304) 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3dd6d9640 1 -- 172.21.15.40:0/1075080635 learned_addr learned my addr 172.21.15.40:0/1075080635 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3ddeda640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 0x55f6d18b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3de6db640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3dd6d9640 1 -- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 msgr2=0x55f6d18b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3dd6d9640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f6d189ea80 0x55f6d18b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3dd6d9640 1 -- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d18a1f30 msgr2=0x55f6d189bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3dd6d9640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6d18a1f30 0x55f6d189bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.303+0000 7fc3dd6d9640 1 -- 172.21.15.40:0/1075080635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d1810e20 con 0x55f6d189ee60 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.304+0000 7fc3dd6d9640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d189ee60 0x55f6d18108e0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009f10 tx=0x7fc3c8005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.304+0000 7fc3d6ffd640 1 -- 172.21.15.40:0/1075080635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80143f0 con 0x55f6d189ee60 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.304+0000 7fc3d6ffd640 1 -- 172.21.15.40:0/1075080635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c8019070 con 0x55f6d189ee60 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.304+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f6d1828650 con 0x55f6d189ee60 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.304+0000 7fc3d6ffd640 1 -- 172.21.15.40:0/1075080635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80147d0 con 0x55f6d189ee60 2024-04-26T21:42:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.304+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d1828c10 con 0x55f6d189ee60 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.305+0000 7fc3d6ffd640 1 -- 172.21.15.40:0/1075080635 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3c8014970 con 0x55f6d189ee60 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.306+0000 7fc3d6ffd640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3c006b2d0 0x7fc3c006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.306+0000 7fc3d6ffd640 1 -- 172.21.15.40:0/1075080635 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fc3c808a6c0 con 0x55f6d189ee60 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.306+0000 7fc3ddeda640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3c006b2d0 0x7fc3c006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.306+0000 7fc3e03e0f40 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f6d18bcf70 0x55f6d18baad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.306+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f6d18bb010 con 0x55f6d18bcf70 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.306+0000 7fc3de6db640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f6d18bcf70 0x55f6d18baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.306+0000 7fc3ddeda640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3c006b2d0 0x7fc3c006d790 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8005fd0 tx=0x7fc3d8005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3de6db640 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f6d18bcf70 0x55f6d18baad0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.310 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/100.00000000 mtime 2024-04-26T21:42:09.000000+0000, size 0 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3de6db640 1 -- 172.21.15.40:0/1075080635 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 100.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fc3d0051d70 con 0x55f6d18bcf70 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f6d18bcf70 msgr2=0x55f6d18baad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f6d18bcf70 0x55f6d18baad0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3c006b2d0 msgr2=0x7fc3c006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3c006b2d0 0x7fc3c006d790 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8005fd0 tx=0x7fc3d8005a10 comp rx=0 tx=0).stop 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d189ee60 msgr2=0x55f6d18108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 --2- 172.21.15.40:0/1075080635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f6d189ee60 0x55f6d18108e0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009f10 tx=0x7fc3c8005f60 comp rx=0 tx=0).stop 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3de6db640 1 -- 172.21.15.40:0/1075080635 reap_dead start 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 shutdown_connections 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 >> 172.21.15.40:0/1075080635 conn(0x55f6d17e1ab0 msgr2=0x55f6d18a1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 shutdown_connections 2024-04-26T21:42:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.307+0000 7fc3e03e0f40 1 -- 172.21.15.40:0/1075080635 wait complete. 2024-04-26T21:42:27.313 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_sessionmap 2024-04-26T21:42:27.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.394+0000 7f18196adf40 1 Processor -- start 2024-04-26T21:42:27.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.394+0000 7f18196adf40 1 -- start start 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.394+0000 7f18196adf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaea80 0x55eaa9ac8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.394+0000 7f18196adf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaee60 0x55eaa99f2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.394+0000 7f18196adf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 0x55eaa99f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.394+0000 7f18196adf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55eaa9a3b2d0 con 0x55eaa9aaee60 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.394+0000 7f18196adf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eaa999ae40 con 0x55eaa9ab1f30 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.394+0000 7f18196adf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55eaa9a3b570 con 0x55eaa9aaea80 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaee60 0x55eaa99f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaee60 0x55eaa99f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58320/0 (socket says 172.21.15.40:58320) 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 -- 172.21.15.40:0/2538142275 learned_addr learned my addr 172.21.15.40:0/2538142275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18171a7640 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaea80 0x55eaa9ac8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18179a8640 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 0x55eaa99f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 -- 172.21.15.40:0/2538142275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaea80 msgr2=0x55eaa9ac8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaea80 0x55eaa9ac8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 -- 172.21.15.40:0/2538142275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 msgr2=0x55eaa99f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 0x55eaa99f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 -- 172.21.15.40:0/2538142275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaa9ac85b0 con 0x55eaa9aaee60 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18169a6640 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaee60 0x55eaa99f2720 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f180000a7a0 tx=0x7f180000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec186e54bf9c639 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18159a4640 1 -- 172.21.15.40:0/2538142275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800019070 con 0x55eaa9aaee60 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18159a4640 1 -- 172.21.15.40:0/2538142275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1800007600 con 0x55eaa9aaee60 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.395+0000 7f18159a4640 1 -- 172.21.15.40:0/2538142275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18000079d0 con 0x55eaa9aaee60 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 -- 172.21.15.40:0/2538142275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaee60 msgr2=0x55eaa99f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaee60 0x55eaa99f2720 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f180000a7a0 tx=0x7f180000a890 comp rx=0 tx=0).stop 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 -- 172.21.15.40:0/2538142275 shutdown_connections 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 0x55eaa99f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaee60 0x55eaa99f2720 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 --2- 172.21.15.40:0/2538142275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaea80 0x55eaa9ac8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 -- 172.21.15.40:0/2538142275 >> 172.21.15.40:0/2538142275 conn(0x55eaa99f1ab0 msgr2=0x55eaa9ac96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 -- 172.21.15.40:0/2538142275 shutdown_connections 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 -- 172.21.15.40:0/2538142275 wait complete. 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 Processor -- start 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.396+0000 7f18196adf40 1 -- start start 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18196adf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaea80 0x55eaa99f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18196adf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaee60 0x55eaa9b62a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18196adf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 0x55eaa9b62fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18196adf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55eaa9acc850 con 0x55eaa9aaea80 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18196adf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eaa9ac9f90 con 0x55eaa9ab1f30 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18196adf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55eaa9aad640 con 0x55eaa9aaee60 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaea80 0x55eaa99f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaea80 0x55eaa99f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58322/0 (socket says 172.21.15.40:58322) 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 -- 172.21.15.40:0/2457000041 learned_addr learned my addr 172.21.15.40:0/2457000041 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18179a8640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 0x55eaa9b62fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18169a6640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaee60 0x55eaa9b62a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 -- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaee60 msgr2=0x55eaa9b62a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaee60 0x55eaa9b62a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 -- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 msgr2=0x55eaa9b62fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eaa9ab1f30 0x55eaa9b62fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 -- 172.21.15.40:0/2457000041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaa9aac400 con 0x55eaa9aaea80 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18169a6640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eaa9aaee60 0x55eaa9b62a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.397+0000 7f18171a7640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaea80 0x55eaa99f2790 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f181000a4f0 tx=0x7f1810002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.398+0000 7f18149a2640 1 -- 172.21.15.40:0/2457000041 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810063030 con 0x55eaa9aaea80 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.398+0000 7f18149a2640 1 -- 172.21.15.40:0/2457000041 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1810054a60 con 0x55eaa9aaea80 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.398+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55eaa9ac8250 con 0x55eaa9aaea80 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.398+0000 7f18149a2640 1 -- 172.21.15.40:0/2457000041 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810059040 con 0x55eaa9aaea80 2024-04-26T21:42:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.398+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaa9ac71c0 con 0x55eaa9aaea80 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.399+0000 7f18149a2640 1 -- 172.21.15.40:0/2457000041 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1810066030 con 0x55eaa9aaea80 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.399+0000 7f18149a2640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17f806b2d0 0x7f17f806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.400+0000 7f18149a2640 1 -- 172.21.15.40:0/2457000041 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f18100ce160 con 0x55eaa9aaea80 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.400+0000 7f18169a6640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17f806b2d0 0x7f17f806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.400+0000 7f18196adf40 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55eaa99f2cd0 0x55eaa9aab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.400+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55eaa9ac35a0 con 0x55eaa99f2cd0 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.400+0000 7f18179a8640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55eaa99f2cd0 0x55eaa9aab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.400+0000 7f18169a6640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17f806b2d0 0x7f17f806d790 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55eaa9aa8bd0 tx=0x7f180000c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.400+0000 7f18179a8640 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55eaa99f2cd0 0x55eaa9aab7d0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18179a8640 1 -- 172.21.15.40:0/2457000041 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(1 mds0_sessionmap [stat out=16b] v0'0 uv3 ondisk = 0) ==== 159+0+16 (crc 0 0 0) 0x7f18080044c0 con 0x55eaa99f2cd0 2024-04-26T21:42:27.405 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/mds0_sessionmap mtime 2024-04-26T21:42:23.000000+0000, size 0 2024-04-26T21:42:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55eaa99f2cd0 msgr2=0x55eaa9aab7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55eaa99f2cd0 0x55eaa9aab7d0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17f806b2d0 msgr2=0x7f17f806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17f806b2d0 0x7f17f806d790 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55eaa9aa8bd0 tx=0x7f180000c000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaea80 msgr2=0x55eaa99f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 --2- 172.21.15.40:0/2457000041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55eaa9aaea80 0x55eaa99f2790 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f181000a4f0 tx=0x7f1810002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18179a8640 1 -- 172.21.15.40:0/2457000041 reap_dead start 2024-04-26T21:42:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 shutdown_connections 2024-04-26T21:42:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 >> 172.21.15.40:0/2457000041 conn(0x55eaa99f1ab0 msgr2=0x55eaa99f1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 shutdown_connections 2024-04-26T21:42:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.401+0000 7f18196adf40 1 -- 172.21.15.40:0/2457000041 wait complete. 2024-04-26T21:42:27.407 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 400.00000000 2024-04-26T21:42:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.490+0000 7f66cf875f40 1 Processor -- start 2024-04-26T21:42:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.490+0000 7f66cf875f40 1 -- start start 2024-04-26T21:42:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66cf875f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d2932420 0x5631d29b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66cf875f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d28ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66cf875f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d29c0b70 0x5631d29bf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66cf875f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5631d288fe40 con 0x5631d29b9f70 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66cf875f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5631d2931660 con 0x5631d2932420 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66cf875f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5631d28e9bf0 con 0x5631d29c0b70 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d28ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d28ea010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58336/0 (socket says 172.21.15.40:58336) 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 -- 172.21.15.40:0/110538802 learned_addr learned my addr 172.21.15.40:0/110538802 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 -- 172.21.15.40:0/110538802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d29c0b70 msgr2=0x5631d29bf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 --2- 172.21.15.40:0/110538802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d29c0b70 0x5631d29bf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 -- 172.21.15.40:0/110538802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d2932420 msgr2=0x5631d29b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 --2- 172.21.15.40:0/110538802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d2932420 0x5631d29b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 -- 172.21.15.40:0/110538802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d29b0e70 con 0x5631d29b9f70 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c7fff640 1 --2- 172.21.15.40:0/110538802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d28ea010 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f66b800a4c0 tx=0x7f66b804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c69cc385a175739 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.491+0000 7f66c77fe640 1 -- 172.21.15.40:0/110538802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b805a030 con 0x5631d29b9f70 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66c77fe640 1 -- 172.21.15.40:0/110538802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b8064070 con 0x5631d29b9f70 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66c77fe640 1 -- 172.21.15.40:0/110538802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b805f440 con 0x5631d29b9f70 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 -- 172.21.15.40:0/110538802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 msgr2=0x5631d28ea010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 --2- 172.21.15.40:0/110538802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d28ea010 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f66b800a4c0 tx=0x7f66b804f000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 -- 172.21.15.40:0/110538802 shutdown_connections 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 --2- 172.21.15.40:0/110538802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d29c0b70 0x5631d29bf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 --2- 172.21.15.40:0/110538802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d28ea010 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 --2- 172.21.15.40:0/110538802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d2932420 0x5631d29b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 -- 172.21.15.40:0/110538802 >> 172.21.15.40:0/110538802 conn(0x5631d29128a0 msgr2=0x5631d2913a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 -- 172.21.15.40:0/110538802 shutdown_connections 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.492+0000 7f66cf875f40 1 -- 172.21.15.40:0/110538802 wait complete. 2024-04-26T21:42:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66cf875f40 1 Processor -- start 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66cf875f40 1 -- start start 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66cf875f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d2932420 0x5631d29ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66cf875f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d292d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66cf875f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d29c0b70 0x5631d292dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66cf875f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5631d2917df0 con 0x5631d29b9f70 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66cf875f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5631d2917c70 con 0x5631d29c0b70 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66cf875f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5631d29beb20 con 0x5631d2932420 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d292d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d292d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58338/0 (socket says 172.21.15.40:58338) 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.493+0000 7f66c7fff640 1 -- 172.21.15.40:0/3792849437 learned_addr learned my addr 172.21.15.40:0/3792849437 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66cd36f640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d2932420 0x5631d29ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66cdb70640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d29c0b70 0x5631d292dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c7fff640 1 -- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d2932420 msgr2=0x5631d29ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c7fff640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d2932420 0x5631d29ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c7fff640 1 -- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d29c0b70 msgr2=0x5631d292dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c7fff640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d29c0b70 0x5631d292dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c7fff640 1 -- 172.21.15.40:0/3792849437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d29b02b0 con 0x5631d29b9f70 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66cd36f640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5631d2932420 0x5631d29ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66cdb70640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5631d29c0b70 0x5631d292dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c7fff640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d292d5a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f66b8061040 tx=0x7f66b8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c67fc640 1 -- 172.21.15.40:0/3792849437 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b805a070 con 0x5631d29b9f70 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c67fc640 1 -- 172.21.15.40:0/3792849437 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b805b040 con 0x5631d29b9f70 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5631d2914200 con 0x5631d29b9f70 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66c67fc640 1 -- 172.21.15.40:0/3792849437 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8055b90 con 0x5631d29b9f70 2024-04-26T21:42:27.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.494+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d29ac770 con 0x5631d29b9f70 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.495+0000 7f66c67fc640 1 -- 172.21.15.40:0/3792849437 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66b805f440 con 0x5631d29b9f70 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.496+0000 7f66c67fc640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b006b2d0 0x7f66b006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.496+0000 7f66c67fc640 1 -- 172.21.15.40:0/3792849437 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f66b80d00f0 con 0x5631d29b9f70 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.496+0000 7f66cf875f40 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5631d29b0660 0x5631d292c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.496+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5631d292ca00 con 0x5631d29b0660 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.496+0000 7f66cd36f640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b006b2d0 0x7f66b006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.496+0000 7f66cdb70640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5631d29b0660 0x5631d292c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cd36f640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b006b2d0 0x7f66b006d790 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x5631d2912280 tx=0x7f66c0001550 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cdb70640 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5631d29b0660 0x5631d292c4c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cdb70640 1 -- 172.21.15.40:0/3792849437 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 400.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f66c800d810 con 0x5631d29b0660 2024-04-26T21:42:27.501 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/400.00000000 mtime 2024-04-26T21:42:09.000000+0000, size 22 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5631d29b0660 msgr2=0x5631d292c4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5631d29b0660 0x5631d292c4c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b006b2d0 msgr2=0x7f66b006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b006b2d0 0x7f66b006d790 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x5631d2912280 tx=0x7f66c0001550 comp rx=0 tx=0).stop 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 msgr2=0x5631d292d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 --2- 172.21.15.40:0/3792849437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631d29b9f70 0x5631d292d5a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f66b8061040 tx=0x7f66b8002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cdb70640 1 -- 172.21.15.40:0/3792849437 reap_dead start 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 shutdown_connections 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 >> 172.21.15.40:0/3792849437 conn(0x5631d29128a0 msgr2=0x5631d29b8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 shutdown_connections 2024-04-26T21:42:27.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.497+0000 7f66cf875f40 1 -- 172.21.15.40:0/3792849437 wait complete. 2024-04-26T21:42:27.504 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000.inode 2024-04-26T21:42:27.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb5bb7f40 1 Processor -- start 2024-04-26T21:42:27.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb5bb7f40 1 -- start start 2024-04-26T21:42:27.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb5bb7f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c9c0022f0 0x56069896d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb5bb7f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560698978060 0x56069896e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb5bb7f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56069896ed80 0x5606988cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb5bb7f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560698848e40 con 0x56069896ed80 2024-04-26T21:42:27.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb5bb7f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56069884ce80 con 0x560698978060 2024-04-26T21:42:27.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb5bb7f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5606988a1250 con 0x7f5c9c0022f0 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb3eb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56069896ed80 0x5606988cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb3eb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56069896ed80 0x5606988cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58366/0 (socket says 172.21.15.40:58366) 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.546+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/1340444816 learned_addr learned my addr 172.21.15.40:0/1340444816 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb2eb2640 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560698978060 0x56069896e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb36b3640 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c9c0022f0 0x56069896d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c9c0022f0 msgr2=0x56069896d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb3eb4640 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c9c0022f0 0x56069896d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560698978060 msgr2=0x56069896e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb3eb4640 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560698978060 0x56069896e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/1340444816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606988eb6b0 con 0x56069896ed80 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb2eb2640 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560698978060 0x56069896e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb3eb4640 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56069896ed80 0x5606988cdb60 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4002230 tx=0x7f5ca4006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db810f5ffe300365 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb1eb0640 1 -- 172.21.15.40:0/1340444816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4018070 con 0x56069896ed80 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb1eb0640 1 -- 172.21.15.40:0/1340444816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca4003450 con 0x56069896ed80 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb1eb0640 1 -- 172.21.15.40:0/1340444816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4003820 con 0x56069896ed80 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/1340444816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56069896ed80 msgr2=0x5606988cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb5bb7f40 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56069896ed80 0x5606988cdb60 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4002230 tx=0x7f5ca4006430 comp rx=0 tx=0).stop 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/1340444816 shutdown_connections 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb5bb7f40 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56069896ed80 0x5606988cdb60 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb5bb7f40 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560698978060 0x56069896e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb5bb7f40 1 --2- 172.21.15.40:0/1340444816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c9c0022f0 0x56069896d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.547+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/1340444816 >> 172.21.15.40:0/1340444816 conn(0x5606988cc9b0 msgr2=0x5606989795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.548+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/1340444816 shutdown_connections 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.548+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/1340444816 wait complete. 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.548+0000 7f5cb5bb7f40 1 Processor -- start 2024-04-26T21:42:27.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.548+0000 7f5cb5bb7f40 1 -- start start 2024-04-26T21:42:27.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb5bb7f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56069896ed80 0x56069896a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb5bb7f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560698978060 0x5606988cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb5bb7f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0022f0 0x5606988ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb5bb7f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56069896e5f0 con 0x7f5c9c0022f0 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb5bb7f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56069896e030 con 0x56069896ed80 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb5bb7f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56069896e310 con 0x560698978060 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0022f0 0x5606988ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0022f0 0x5606988ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58374/0 (socket says 172.21.15.40:58374) 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/3269090329 learned_addr learned my addr 172.21.15.40:0/3269090329 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb2eb2640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560698978060 0x5606988cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb36b3640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56069896ed80 0x56069896a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560698978060 msgr2=0x5606988cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560698978060 0x5606988cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56069896ed80 msgr2=0x56069896a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56069896ed80 0x56069896a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/3269090329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56069896d4d0 con 0x7f5c9c0022f0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.549+0000 7f5cb3eb4640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0022f0 0x5606988ce0a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4016040 tx=0x7f5ca4001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.550+0000 7f5cb0eae640 1 -- 172.21.15.40:0/3269090329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4024070 con 0x7f5c9c0022f0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.550+0000 7f5cb0eae640 1 -- 172.21.15.40:0/3269090329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca4019040 con 0x7f5c9c0022f0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.550+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56069896d1f0 con 0x7f5c9c0022f0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.550+0000 7f5cb0eae640 1 -- 172.21.15.40:0/3269090329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4018070 con 0x7f5c9c0022f0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.550+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56069896c220 con 0x7f5c9c0022f0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.551+0000 7f5cb0eae640 1 -- 172.21.15.40:0/3269090329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ca4012a40 con 0x7f5c9c0022f0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.551+0000 7f5cb0eae640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca006b2d0 0x7f5ca006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.551+0000 7f5cb0eae640 1 -- 172.21.15.40:0/3269090329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f5ca408afc0 con 0x7f5c9c0022f0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.552+0000 7f5cb36b3640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca006b2d0 0x7f5ca006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.552+0000 7f5cb5bb7f40 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x56069896a8c0 0x56069896f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.552+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56069896ca20 con 0x56069896a8c0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.552+0000 7f5cb2eb2640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x56069896a8c0 0x56069896f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.552+0000 7f5cb36b3640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca006b2d0 0x7f5ca006d790 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5cac054000 tx=0x7f5cac050000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.552+0000 7f5cb2eb2640 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x56069896a8c0 0x56069896f6d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.552+0000 7f5cb2eb2640 1 -- 172.21.15.40:0/3269090329 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x7f5ca8009840 con 0x56069896a8c0 2024-04-26T21:42:27.557 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/100.00000000.inode mtime 2024-04-26T21:42:09.000000+0000, size 582 2024-04-26T21:42:27.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x56069896a8c0 msgr2=0x56069896f6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x56069896a8c0 0x56069896f6d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca006b2d0 msgr2=0x7f5ca006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ca006b2d0 0x7f5ca006d790 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5cac054000 tx=0x7f5cac050000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0022f0 msgr2=0x5606988ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 --2- 172.21.15.40:0/3269090329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0022f0 0x5606988ce0a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4016040 tx=0x7f5ca4001cf0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb3eb4640 1 -- 172.21.15.40:0/3269090329 reap_dead start 2024-04-26T21:42:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 shutdown_connections 2024-04-26T21:42:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 >> 172.21.15.40:0/3269090329 conn(0x5606988cc9b0 msgr2=0x5606989765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 shutdown_connections 2024-04-26T21:42:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.553+0000 7f5cb5bb7f40 1 -- 172.21.15.40:0/3269090329 wait complete. 2024-04-26T21:42:27.559 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000 2024-04-26T21:42:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.640+0000 7f9baff84f40 1 Processor -- start 2024-04-26T21:42:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.640+0000 7f9baff84f40 1 -- start start 2024-04-26T21:42:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.640+0000 7f9baff84f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5639a7a42930 0x5639a7a445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.640+0000 7f9baff84f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a72af0 0x5639a7a45570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.640+0000 7f9baff84f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a45ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.640+0000 7f9baff84f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5639a7a8b9f0 con 0x5639a7a72af0 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.640+0000 7f9baff84f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5639a79ebe40 con 0x5639a7b1b900 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.640+0000 7f9baff84f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5639a7b1bce0 con 0x5639a7a42930 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a72af0 0x5639a7a45570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a72af0 0x5639a7a45570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58400/0 (socket says 172.21.15.40:58400) 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 -- 172.21.15.40:0/1570569091 learned_addr learned my addr 172.21.15.40:0/1570569091 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 -- 172.21.15.40:0/1570569091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5639a7a42930 msgr2=0x5639a7a445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5639a7a42930 0x5639a7a445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 -- 172.21.15.40:0/1570569091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 msgr2=0x5639a7a45ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bae27f640 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a45ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a45ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 -- 172.21.15.40:0/1570569091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a7b17ad0 con 0x5639a7a72af0 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bae27f640 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a45ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9bad27d640 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a72af0 0x5639a7a45570 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400a440 tx=0x7f9ba4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7d9dc0980ac450 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9b9ffff640 1 -- 172.21.15.40:0/1570569091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4053af0 con 0x5639a7a72af0 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9b9ffff640 1 -- 172.21.15.40:0/1570569091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba4053c90 con 0x5639a7a72af0 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.641+0000 7f9b9ffff640 1 -- 172.21.15.40:0/1570569091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4067670 con 0x5639a7a72af0 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 -- 172.21.15.40:0/1570569091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a72af0 msgr2=0x5639a7a45570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a72af0 0x5639a7a45570 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400a440 tx=0x7f9ba4051000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 -- 172.21.15.40:0/1570569091 shutdown_connections 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a45ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a72af0 0x5639a7a45570 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 --2- 172.21.15.40:0/1570569091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5639a7a42930 0x5639a7a445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 -- 172.21.15.40:0/1570569091 >> 172.21.15.40:0/1570569091 conn(0x5639a7a43330 msgr2=0x5639a7a6df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 -- 172.21.15.40:0/1570569091 shutdown_connections 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 -- 172.21.15.40:0/1570569091 wait complete. 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 Processor -- start 2024-04-26T21:42:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.642+0000 7f9baff84f40 1 -- start start 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9baff84f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a42930 0x5639a7b162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9baff84f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5639a7a72af0 0x5639a7b16a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9baff84f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a6fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9baff84f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5639a7a42260 con 0x5639a7a42930 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9baff84f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5639a7a440f0 con 0x5639a7b1b900 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9baff84f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5639a7a41f80 con 0x5639a7a72af0 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bada7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a42930 0x5639a7b162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bada7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a42930 0x5639a7b162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58406/0 (socket says 172.21.15.40:58406) 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bada7e640 1 -- 172.21.15.40:0/1600687512 learned_addr learned my addr 172.21.15.40:0/1600687512 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bae27f640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a6fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bada7e640 1 -- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5639a7a72af0 msgr2=0x5639a7b16a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bada7e640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5639a7a72af0 0x5639a7b16a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bada7e640 1 -- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 msgr2=0x5639a7a6fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bada7e640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a6fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bada7e640 1 -- 172.21.15.40:0/1600687512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a7bb6020 con 0x5639a7a42930 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.643+0000 7f9bae27f640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639a7b1b900 0x5639a7a6fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.644+0000 7f9bada7e640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a42930 0x5639a7b162d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000bdb0 tx=0x7f9ba0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.644+0000 7f9b9effd640 1 -- 172.21.15.40:0/1600687512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0008070 con 0x5639a7a42930 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.644+0000 7f9b9effd640 1 -- 172.21.15.40:0/1600687512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba000f650 con 0x5639a7a42930 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.644+0000 7f9b9effd640 1 -- 172.21.15.40:0/1600687512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000fa40 con 0x5639a7a42930 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.644+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639a7bb6300 con 0x5639a7a42930 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.644+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a7b0d1b0 con 0x5639a7a42930 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.645+0000 7f9b9effd640 1 -- 172.21.15.40:0/1600687512 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ba000fbe0 con 0x5639a7a42930 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.646+0000 7f9b9effd640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b9006b2d0 0x7f9b9006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.646+0000 7f9b9effd640 1 -- 172.21.15.40:0/1600687512 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f9ba008ab30 con 0x5639a7a42930 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.646+0000 7f9baff84f40 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5639a7b0d4a0 0x5639a7b0d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.646+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5639a7b0de90 con 0x5639a7b0d4a0 2024-04-26T21:42:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.646+0000 7f9bad27d640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b9006b2d0 0x7f9b9006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.646+0000 7f9bae27f640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5639a7b0d4a0 0x5639a7b0d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9bad27d640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b9006b2d0 0x7f9b9006d790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4064000 tx=0x7f9ba4053000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9bae27f640 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5639a7b0d4a0 0x5639a7b0d950 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.650 INFO:teuthology.orchestra.run.smithi040.stdout:cephfs_metadata/1.00000000 mtime 2024-04-26T21:42:23.000000+0000, size 0 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9bae27f640 1 -- 172.21.15.40:0/1600687512 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 1.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 154+0+16 (crc 0 0 0) 0x7f9ba800b830 con 0x5639a7b0d4a0 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5639a7b0d4a0 msgr2=0x5639a7b0d950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5639a7b0d4a0 0x5639a7b0d950 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b9006b2d0 msgr2=0x7f9b9006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b9006b2d0 0x7f9b9006d790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4064000 tx=0x7f9ba4053000 comp rx=0 tx=0).stop 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a42930 msgr2=0x5639a7b162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 --2- 172.21.15.40:0/1600687512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639a7a42930 0x5639a7b162d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000bdb0 tx=0x7f9ba0009560 comp rx=0 tx=0).stop 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9bae27f640 1 -- 172.21.15.40:0/1600687512 reap_dead start 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 shutdown_connections 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 >> 172.21.15.40:0/1600687512 conn(0x5639a7a43330 msgr2=0x5639a7a44540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 shutdown_connections 2024-04-26T21:42:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.647+0000 7f9baff84f40 1 -- 172.21.15.40:0/1600687512 wait complete. 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '10000000000.00000000' will be 'no damage' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '600.00000000' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000.inode' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000000' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000001' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_openfiles.0' will be 'no damage' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '500.00000000' will be 'no damage' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds_snaptable' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_inotable' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_sessionmap' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '400.00000000' will be 'no damage' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000.inode' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000' will be 'damaged_on_start' 2024-04-26T21:42:27.653 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '10000000000.00000000' will be 'no damage' 2024-04-26T21:42:27.654 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '600.00000000' will be 'damaged_on_start' 2024-04-26T21:42:27.654 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_openfiles.0' will be 'no damage' 2024-04-26T21:42:27.654 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '100.00000000' will be 'damaged_on_start' 2024-04-26T21:42:27.654 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_sessionmap' will be 'damaged_on_start' 2024-04-26T21:42:27.654 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '1.00000000' will be 'damaged_on_start' 2024-04-26T21:42:27.654 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 10000000000.00000000' 2024-04-26T21:42:27.654 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:27.654 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:27.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:42:27.708 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:42:27.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:27.708 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:42:27.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:42:27.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:42:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.904+0000 7fa510b12640 1 Processor -- start 2024-04-26T21:42:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.904+0000 7fa510b12640 1 -- start start 2024-04-26T21:42:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa510b12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c162880 0x7fa50c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa510b12640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa50c0b5f20 0x7fa50c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa510b12640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa50c0d39f0 0x7fa50c0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa510b12640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa50c0d0580 con 0x7fa50c0b5f20 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa510b12640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa50c0b7e50 con 0x7fa50c0d39f0 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa510b12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c0d1800 con 0x7fa50c162880 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa509d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa50c0b5f20 0x7fa50c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa509d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa50c0b5f20 0x7fa50c0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58418/0 (socket says 172.21.15.40:58418) 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa509d9b640 1 -- 172.21.15.40:0/342894062 learned_addr learned my addr 172.21.15.40:0/342894062 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa50ad9d640 1 --2- 172.21.15.40:0/342894062 >> v2:172.21.15.184:3300/0 conn(0x7fa50c0d39f0 0x7fa50c0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa50a59c640 1 --2- 172.21.15.40:0/342894062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c162880 0x7fa50c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa509d9b640 1 -- 172.21.15.40:0/342894062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c162880 msgr2=0x7fa50c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa509d9b640 1 --2- 172.21.15.40:0/342894062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c162880 0x7fa50c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa509d9b640 1 -- 172.21.15.40:0/342894062 >> v2:172.21.15.184:3300/0 conn(0x7fa50c0d39f0 msgr2=0x7fa50c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa509d9b640 1 --2- 172.21.15.40:0/342894062 >> v2:172.21.15.184:3300/0 conn(0x7fa50c0d39f0 0x7fa50c0b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.905+0000 7fa509d9b640 1 -- 172.21.15.40:0/342894062 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c0d07d0 con 0x7fa50c0b5f20 2024-04-26T21:42:27.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.906+0000 7fa509d9b640 1 --2- 172.21.15.40:0/342894062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c0b5f20 0x7fa50c0d34b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa50c08f770 tx=0x7fa4f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f026a84f3359d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.906+0000 7fa50959a640 1 -- 172.21.15.40:0/342894062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4037070 con 0x7fa50c0b5f20 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.906+0000 7fa50959a640 1 -- 172.21.15.40:0/342894062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f4032440 con 0x7fa50c0b5f20 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.906+0000 7fa50959a640 1 -- 172.21.15.40:0/342894062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4032810 con 0x7fa50c0b5f20 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 -- 172.21.15.40:0/342894062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c0b5f20 msgr2=0x7fa50c0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 --2- 172.21.15.40:0/342894062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c0b5f20 0x7fa50c0d34b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa50c08f770 tx=0x7fa4f402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 -- 172.21.15.40:0/342894062 shutdown_connections 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 --2- 172.21.15.40:0/342894062 >> v2:172.21.15.184:3300/0 conn(0x7fa50c0d39f0 0x7fa50c0b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 --2- 172.21.15.40:0/342894062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c0b5f20 0x7fa50c0d34b0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 --2- 172.21.15.40:0/342894062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c162880 0x7fa50c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 -- 172.21.15.40:0/342894062 >> 172.21.15.40:0/342894062 conn(0x7fa50c0d4f60 msgr2=0x7fa50c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 -- 172.21.15.40:0/342894062 shutdown_connections 2024-04-26T21:42:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.907+0000 7fa510b12640 1 -- 172.21.15.40:0/342894062 wait complete. 2024-04-26T21:42:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.908+0000 7fa510b12640 1 Processor -- start 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.908+0000 7fa510b12640 1 -- start start 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa510b12640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50c0b5f20 0x7fa50c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa510b12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c0d39f0 0x7fa50c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa510b12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c162880 0x7fa50c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa510b12640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c162c60 con 0x7fa50c162880 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa510b12640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0d0d90 con 0x7fa50c0b5f20 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa510b12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c162de0 con 0x7fa50c0d39f0 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa50ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c162880 0x7fa50c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa50ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c162880 0x7fa50c06d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58428/0 (socket says 172.21.15.40:58428) 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa50ad9d640 1 -- 172.21.15.40:0/3522346660 learned_addr learned my addr 172.21.15.40:0/3522346660 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa50a59c640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50c0b5f20 0x7fa50c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.909+0000 7fa509d9b640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c0d39f0 0x7fa50c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa50ad9d640 1 -- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c0d39f0 msgr2=0x7fa50c06d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa50ad9d640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c0d39f0 0x7fa50c06d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa50ad9d640 1 -- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50c0b5f20 msgr2=0x7fa50c1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa50ad9d640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50c0b5f20 0x7fa50c1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa50ad9d640 1 -- 172.21.15.40:0/3522346660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c0ced50 con 0x7fa50c162880 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa50a59c640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50c0b5f20 0x7fa50c1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa50ad9d640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c162880 0x7fa50c06d630 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00eed0 tx=0x7fa4fc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa502ffd640 1 -- 172.21.15.40:0/3522346660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc018020 con 0x7fa50c162880 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa502ffd640 1 -- 172.21.15.40:0/3522346660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc014400 con 0x7fa50c162880 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa502ffd640 1 -- 172.21.15.40:0/3522346660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0147f0 con 0x7fa50c162880 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c0ccff0 con 0x7fa50c162880 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.910+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c0cd5b0 con 0x7fa50c162880 2024-04-26T21:42:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.911+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50c0694f0 con 0x7fa50c162880 2024-04-26T21:42:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.911+0000 7fa502ffd640 1 -- 172.21.15.40:0/3522346660 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4fc01b020 con 0x7fa50c162880 2024-04-26T21:42:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.912+0000 7fa502ffd640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ec06c8f0 0x7fa4ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.912+0000 7fa502ffd640 1 -- 172.21.15.40:0/3522346660 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fa4fc08c490 con 0x7fa50c162880 2024-04-26T21:42:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.912+0000 7fa50a59c640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ec06c8f0 0x7fa4ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.913+0000 7fa50a59c640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ec06c8f0 0x7fa4ec06edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa504006fd0 tx=0x7fa504008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:27.916+0000 7fa502ffd640 1 -- 172.21.15.40:0/3522346660 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4fc022030 con 0x7fa50c162880 2024-04-26T21:42:28.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.141+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa50c15fd60 con 0x7fa50c162880 2024-04-26T21:42:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.491+0000 7fa502ffd640 1 -- 172.21.15.40:0/3522346660 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7fa4fc05a6e0 con 0x7fa50c162880 2024-04-26T21:42:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:42:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ec06c8f0 msgr2=0x7fa4ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ec06c8f0 0x7fa4ec06edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa504006fd0 tx=0x7fa504008040 comp rx=0 tx=0).stop 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c162880 msgr2=0x7fa50c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c162880 0x7fa50c06d630 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00eed0 tx=0x7fa4fc00b480 comp rx=0 tx=0).stop 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 shutdown_connections 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ec06c8f0 0x7fa4ec06edb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50c162880 0x7fa50c06d630 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50c0d39f0 0x7fa50c06d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 --2- 172.21.15.40:0/3522346660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50c0b5f20 0x7fa50c1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 >> 172.21.15.40:0/3522346660 conn(0x7fa50c0d4f60 msgr2=0x7fa50c06e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 shutdown_connections 2024-04-26T21:42:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.493+0000 7fa510b12640 1 -- 172.21.15.40:0/3522346660 wait complete. 2024-04-26T21:42:28.506 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:42:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.607+0000 7f940a62a640 1 Processor -- start 2024-04-26T21:42:28.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.607+0000 7f940a62a640 1 -- start start 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.608+0000 7f940a62a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f940415f3a0 0x7f94040d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.608+0000 7f940a62a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040d0e20 con 0x7f94040d1770 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.608+0000 7f940a62a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040b1d70 con 0x7f94040d2330 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.608+0000 7f940a62a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f94040b2050 con 0x7f940415f3a0 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.608+0000 7f9403fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f940415f3a0 0x7f94040d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.608+0000 7f94037fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f94040d1770 0x7f94040d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37458/0 (socket says 172.21.15.40:37458) 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.608+0000 7f94037fe640 1 -- 172.21.15.40:0/28705471 learned_addr learned my addr 172.21.15.40:0/28705471 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.609+0000 7f9403fff640 1 -- 172.21.15.40:0/28705471 >> v1:172.21.15.184:6789/0 conn(0x7f94040d2330 legacy=0x7f94040b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.609+0000 7f9403fff640 1 -- 172.21.15.40:0/28705471 >> v1:172.21.15.40:6789/0 conn(0x7f94040d1770 legacy=0x7f94040d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.609+0000 7f9403fff640 1 -- 172.21.15.40:0/28705471 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94040b2330 con 0x7f940415f3a0 2024-04-26T21:42:28.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.609+0000 7f9403fff640 1 --2- 172.21.15.40:0/28705471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940415f3a0 0x7f94040d06f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f93f40097a0 tx=0x7f93f402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478d5ddf85e7d119 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f9402ffd640 1 -- 172.21.15.40:0/28705471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4037070 con 0x7f940415f3a0 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f9402ffd640 1 -- 172.21.15.40:0/28705471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f402ad60 con 0x7f940415f3a0 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f9402ffd640 1 -- 172.21.15.40:0/28705471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4032660 con 0x7f940415f3a0 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f940a62a640 1 -- 172.21.15.40:0/28705471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940415f3a0 msgr2=0x7f94040d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f940a62a640 1 --2- 172.21.15.40:0/28705471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940415f3a0 0x7f94040d06f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f93f40097a0 tx=0x7f93f402a0d0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f940a62a640 1 -- 172.21.15.40:0/28705471 shutdown_connections 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f940a62a640 1 --2- 172.21.15.40:0/28705471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940415f3a0 0x7f94040d06f0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f940a62a640 1 -- 172.21.15.40:0/28705471 >> 172.21.15.40:0/28705471 conn(0x7f94040d4f90 msgr2=0x7f94040d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f940a62a640 1 -- 172.21.15.40:0/28705471 shutdown_connections 2024-04-26T21:42:28.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.610+0000 7f940a62a640 1 -- 172.21.15.40:0/28705471 wait complete. 2024-04-26T21:42:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.611+0000 7f940a62a640 1 Processor -- start 2024-04-26T21:42:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f940a62a640 1 -- start start 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f940a62a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94040d1770 0x7f94041f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f940a62a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94040d2330 0x7f9404152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f940a62a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f940415f3a0 0x7f94040b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f940a62a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94040b30e0 con 0x7f94040d2330 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f940a62a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94040b2f60 con 0x7f940415f3a0 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f940a62a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94040b3260 con 0x7f94040d1770 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f94037fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94040d2330 0x7f9404152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f94037fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94040d2330 0x7f9404152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58450/0 (socket says 172.21.15.40:58450) 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.612+0000 7f94037fe640 1 -- 172.21.15.40:0/2474984656 learned_addr learned my addr 172.21.15.40:0/2474984656 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f9408bc7640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f940415f3a0 0x7f94040b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f9403fff640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94040d1770 0x7f94041f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f94037fe640 1 -- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94040d1770 msgr2=0x7f94041f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f94037fe640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94040d1770 0x7f94041f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f94037fe640 1 -- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f940415f3a0 msgr2=0x7f94040b9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f94037fe640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f940415f3a0 0x7f94040b9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f94037fe640 1 -- 172.21.15.40:0/2474984656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94041f9260 con 0x7f94040d2330 2024-04-26T21:42:28.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f94037fe640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94040d2330 0x7f9404152d00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f93f800c780 tx=0x7f93f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:28.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f9400ff9640 1 -- 172.21.15.40:0/2474984656 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8017070 con 0x7f94040d2330 2024-04-26T21:42:28.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f9400ff9640 1 -- 172.21.15.40:0/2474984656 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f8005a90 con 0x7f94040d2330 2024-04-26T21:42:28.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94040c8ce0 con 0x7f94040d2330 2024-04-26T21:42:28.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f9400ff9640 1 -- 172.21.15.40:0/2474984656 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8007070 con 0x7f94040d2330 2024-04-26T21:42:28.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.613+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94040c9c20 con 0x7f94040d2330 2024-04-26T21:42:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.614+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94040694f0 con 0x7f94040d2330 2024-04-26T21:42:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.615+0000 7f9400ff9640 1 -- 172.21.15.40:0/2474984656 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93f801a070 con 0x7f94040d2330 2024-04-26T21:42:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.615+0000 7f9400ff9640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f93f006c8f0 0x7f93f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.615+0000 7f9400ff9640 1 -- 172.21.15.40:0/2474984656 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f93f808b270 con 0x7f94040d2330 2024-04-26T21:42:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.615+0000 7f9403fff640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f93f006c8f0 0x7f93f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.616+0000 7f9403fff640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f93f006c8f0 0x7f93f006edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f93f4034000 tx=0x7f93f402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:28.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.619+0000 7f9400ff9640 1 -- 172.21.15.40:0/2474984656 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94041f9260 con 0x7f94040d2330 2024-04-26T21:42:28.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:28.875+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f94040b6860 con 0x7f94040d2330 2024-04-26T21:42:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.498+0000 7f9400ff9640 1 -- 172.21.15.40:0/2474984656 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v91) ==== 107+0+0 (secure 0 0 0) 0x7f93f80594d0 con 0x7f94040d2330 2024-04-26T21:42:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:42:29.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f93f006c8f0 msgr2=0x7f93f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f93f006c8f0 0x7f93f006edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f93f4034000 tx=0x7f93f402e000 comp rx=0 tx=0).stop 2024-04-26T21:42:29.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94040d2330 msgr2=0x7f9404152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94040d2330 0x7f9404152d00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f93f800c780 tx=0x7f93f800cc40 comp rx=0 tx=0).stop 2024-04-26T21:42:29.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 shutdown_connections 2024-04-26T21:42:29.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f93f006c8f0 0x7f93f006edb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f940415f3a0 0x7f94040b9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94040d2330 0x7f9404152d00 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 --2- 172.21.15.40:0/2474984656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94040d1770 0x7f94041f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 >> 172.21.15.40:0/2474984656 conn(0x7f94040d4f90 msgr2=0x7f94040d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:29.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 shutdown_connections 2024-04-26T21:42:29.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.501+0000 7f940a62a640 1 -- 172.21.15.40:0/2474984656 wait complete. 2024-04-26T21:42:29.516 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:42:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.562+0000 7fdbd781ef40 1 Processor -- start 2024-04-26T21:42:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.562+0000 7fdbd781ef40 1 -- start start 2024-04-26T21:42:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.562+0000 7fdbd781ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbbc003ab0 0x55964862fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.562+0000 7fdbd781ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648594b30 0x55964863aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd781ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5596486389a0 0x55964863b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd781ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55964850ce40 con 0x559648594b30 2024-04-26T21:42:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd781ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559648510e80 con 0x7fdbbc003ab0 2024-04-26T21:42:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd781ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559648595690 con 0x5596486389a0 2024-04-26T21:42:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648594b30 0x55964863aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648594b30 0x55964863aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39350/0 (socket says 172.21.15.40:39350) 2024-04-26T21:42:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2013659536 learned_addr learned my addr 172.21.15.40:0/2013659536 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd46a6640 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbbc003ab0 0x55964862fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd4ea7640 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5596486389a0 0x55964863b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2013659536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5596486389a0 msgr2=0x55964863b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5596486389a0 0x55964863b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2013659536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbbc003ab0 msgr2=0x55964862fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbbc003ab0 0x55964862fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2013659536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964863a120 con 0x559648594b30 2024-04-26T21:42:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648594b30 0x55964863aee0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8001910 tx=0x7fdbc80099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db74ad54246f04a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd2ea3640 1 -- 172.21.15.40:0/2013659536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8010660 con 0x559648594b30 2024-04-26T21:42:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd2ea3640 1 -- 172.21.15.40:0/2013659536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc8016020 con 0x559648594b30 2024-04-26T21:42:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.563+0000 7fdbd2ea3640 1 -- 172.21.15.40:0/2013659536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8010a80 con 0x559648594b30 2024-04-26T21:42:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2013659536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648594b30 msgr2=0x55964863aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648594b30 0x55964863aee0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8001910 tx=0x7fdbc80099a0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2013659536 shutdown_connections 2024-04-26T21:42:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5596486389a0 0x55964863b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648594b30 0x55964863aee0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2013659536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbbc003ab0 0x55964862fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2013659536 >> 172.21.15.40:0/2013659536 conn(0x55964858f8a0 msgr2=0x559648563d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2013659536 shutdown_connections 2024-04-26T21:42:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.564+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2013659536 wait complete. 2024-04-26T21:42:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd781ef40 1 Processor -- start 2024-04-26T21:42:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd781ef40 1 -- start start 2024-04-26T21:42:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd781ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596486389a0 0x559648631010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd781ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbbc003ab0 0x55964863ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd781ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648632000 0x55964863b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd781ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55964863a3c0 con 0x559648632000 2024-04-26T21:42:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd781ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55964863aa10 con 0x5596486389a0 2024-04-26T21:42:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd781ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55964863a6a0 con 0x7fdbbc003ab0 2024-04-26T21:42:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd4ea7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648632000 0x55964863b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd4ea7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648632000 0x55964863b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39358/0 (socket says 172.21.15.40:39358) 2024-04-26T21:42:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.565+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 learned_addr learned my addr 172.21.15.40:0/2579436788 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd46a6640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596486389a0 0x559648631010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbbc003ab0 0x55964863ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbbc003ab0 msgr2=0x55964863ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd4ea7640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbbc003ab0 0x55964863ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596486389a0 msgr2=0x559648631010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd4ea7640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596486389a0 0x559648631010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596486324c0 con 0x559648632000 2024-04-26T21:42:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd4ea7640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648632000 0x55964863b2e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00a170 tx=0x7fdbcc0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd1ea1640 1 -- 172.21.15.40:0/2579436788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc064020 con 0x559648632000 2024-04-26T21:42:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd1ea1640 1 -- 172.21.15.40:0/2579436788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc04d8f0 con 0x559648632000 2024-04-26T21:42:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd1ea1640 1 -- 172.21.15.40:0/2579436788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00a3b0 con 0x559648632000 2024-04-26T21:42:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5596486327a0 con 0x559648632000 2024-04-26T21:42:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.566+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559648633c60 con 0x559648632000 2024-04-26T21:42:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.568+0000 7fdbd1ea1640 1 -- 172.21.15.40:0/2579436788 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbcc067020 con 0x559648632000 2024-04-26T21:42:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.568+0000 7fdbd1ea1640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc006b2d0 0x7fdbc006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.568+0000 7fdbd1ea1640 1 -- 172.21.15.40:0/2579436788 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fdbcc0cdf00 con 0x559648632000 2024-04-26T21:42:29.581 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:42:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.568+0000 7fdbd46a6640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc006b2d0 0x7fdbc006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.568+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559648634690 0x559648634a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.568+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x559648634fb0 con 0x559648634690 2024-04-26T21:42:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.569+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559648634690 0x559648634a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.569+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559648634690 0x559648634a70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.569+0000 7fdbd46a6640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc006b2d0 0x7fdbc006d790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4002fd0 tx=0x7fdbc4001730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.570+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v95'11 uv4 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.570+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486dbde0 con 0x559648634690 2024-04-26T21:42:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.571+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v95'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.571+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dc180 con 0x559648634690 2024-04-26T21:42:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.573+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v95'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.586 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:42:29.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.573+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dd530 con 0x559648634690 2024-04-26T21:42:29.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.574+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v95'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.574+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dd8d0 con 0x559648634690 2024-04-26T21:42:29.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.575+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v95'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.575+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dc520 con 0x559648634690 2024-04-26T21:42:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.577+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v95'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.577+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dc930 con 0x559648634690 2024-04-26T21:42:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.578+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v95'17 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.578+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.579+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v95'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.579+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.580+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v95'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.591 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:42:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.580+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.581+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v95'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.581+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.582+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v95'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.582+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.583+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v95'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.583+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.584+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v95'23 uv10 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.584+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.585+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v95'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.585+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.586+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v95'25 uv25 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.597 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:42:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.586+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.587+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v95'26 uv26 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.587+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.588+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v95'27 uv27 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.589+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dccd0 con 0x559648634690 2024-04-26T21:42:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.590+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v95'28 uv28 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.590+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dd0b0 con 0x559648634690 2024-04-26T21:42:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.591+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v95'29 uv29 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.591+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dd0b0 con 0x559648634690 2024-04-26T21:42:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.592+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v95'30 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.592+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486dd0b0 con 0x559648634690 2024-04-26T21:42:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.593+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v95'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.593+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dd0b0 con 0x559648634690 2024-04-26T21:42:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.594+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v95'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.601 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:42:29.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.594+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dd0b0 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.594+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v95'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.594+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dcf80 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.595+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v95'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.596+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dcf80 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.596+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v95'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.596+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dcf80 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.597+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v95'36 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.597+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486dcf80 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.598+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v95'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.598+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dcf80 con 0x559648634690 2024-04-26T21:42:29.605 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:42:29.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.599+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v95'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.599+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dcf80 con 0x559648634690 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.600+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v95'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.600+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dcf80 con 0x559648634690 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.601+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v95'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.601+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486dcf80 con 0x559648634690 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.602+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v95'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x559648634690 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.602+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5596486de180 0x5596486e0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.602+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e0ab0 con 0x5596486de180 2024-04-26T21:42:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.602+0000 7fdbd46a6640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5596486de180 0x5596486e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.603+0000 7fdbd46a6640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5596486de180 0x5596486e0570 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.604+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v95'20 uv15 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.604+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486e0e50 con 0x5596486de180 2024-04-26T21:42:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.606+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v95'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.606+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e11f0 con 0x5596486de180 2024-04-26T21:42:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.607+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v95'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.609 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:42:29.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.607+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e1590 con 0x5596486de180 2024-04-26T21:42:29.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.608+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v95'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.608+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e1930 con 0x5596486de180 2024-04-26T21:42:29.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.609+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v95'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.609+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e1cd0 con 0x5596486de180 2024-04-26T21:42:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.610+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v95'25 uv19 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.611+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.612+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v95'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.612+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.613+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v95'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.614 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:42:29.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.613+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.614+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v95'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.614+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.615+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v95'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.615+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.616+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v95'30 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.616+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.617+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v95'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.617+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.618+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v95'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.619 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:42:29.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.618+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e6730 con 0x5596486de180 2024-04-26T21:42:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.620+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v95'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.620+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e6730 con 0x5596486de180 2024-04-26T21:42:29.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.621+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v95'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.621+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.622+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v95'35 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.622+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.623+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v95'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.623+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.624+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v95'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.625 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:42:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.624+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e3080 con 0x5596486de180 2024-04-26T21:42:29.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.625+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v95'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.625+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e3080 con 0x5596486de180 2024-04-26T21:42:29.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.628+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v95'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.628+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e2070 con 0x5596486de180 2024-04-26T21:42:29.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.631+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v95'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486de180 2024-04-26T21:42:29.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.631+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5596486e26f0 0x5596486e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.631+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e50f0 con 0x5596486e26f0 2024-04-26T21:42:29.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.631+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5596486e26f0 0x5596486e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.631+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5596486e26f0 0x5596486e4ae0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.633+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v95'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x5596486e26f0 2024-04-26T21:42:29.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.634+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486e5510 con 0x5596486e26f0 2024-04-26T21:42:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.636+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v95'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x5596486e26f0 2024-04-26T21:42:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.637+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e5510 con 0x5596486e26f0 2024-04-26T21:42:29.640 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:42:29.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.639+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v95'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x5596486e26f0 2024-04-26T21:42:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.639+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e6520 con 0x5596486e26f0 2024-04-26T21:42:29.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.642+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v95'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x5596486e26f0 2024-04-26T21:42:29.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.642+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e6520 con 0x5596486e26f0 2024-04-26T21:42:29.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.645+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v95'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x5596486e26f0 2024-04-26T21:42:29.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.645+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e5510 con 0x5596486e26f0 2024-04-26T21:42:29.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.648+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v95'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8010860 con 0x5596486e26f0 2024-04-26T21:42:29.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.648+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5596486e5f70 0x5596486e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.648+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e8970 con 0x5596486e5f70 2024-04-26T21:42:29.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.648+0000 7fdbd4ea7640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5596486e5f70 0x5596486e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.649+0000 7fdbd4ea7640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5596486e5f70 0x5596486e63c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.650+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v95'4 uv3 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.650+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486e8d10 con 0x5596486e5f70 2024-04-26T21:42:29.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.651+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v95'5 uv5 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.651+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e90f0 con 0x5596486e5f70 2024-04-26T21:42:29.654 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:42:29.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.653+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v95'6 uv6 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.654+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486e99b0 con 0x5596486e5f70 2024-04-26T21:42:29.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.660+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v95'7 uv7 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.660+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5596486ea260 0x5596486ec650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.660+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486ecb90 con 0x5596486ea260 2024-04-26T21:42:29.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.660+0000 7fdbd46a6640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5596486ea260 0x5596486ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.660+0000 7fdbd46a6640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5596486ea260 0x5596486ec650 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.661+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v95'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.661+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486ecc50 con 0x5596486ea260 2024-04-26T21:42:29.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.664+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v95'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.664+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486ed070 con 0x5596486ea260 2024-04-26T21:42:29.667 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:42:29.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.666+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v95'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.667+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486ee080 con 0x5596486ea260 2024-04-26T21:42:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.669+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v95'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.669+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486ee080 con 0x5596486ea260 2024-04-26T21:42:29.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.672+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v95'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.672+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486ed070 con 0x5596486ea260 2024-04-26T21:42:29.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.674+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v95'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.674+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486ed070 con 0x5596486e5f70 2024-04-26T21:42:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.675+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v95'4 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.676+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486ed410 con 0x5596486e5f70 2024-04-26T21:42:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.678+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v95'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.678+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486ed7b0 con 0x5596486e5f70 2024-04-26T21:42:29.681 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:42:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.681+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v95'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.681+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486eeb60 con 0x5596486e5f70 2024-04-26T21:42:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.683+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v95'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.683+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486eeb60 con 0x5596486e5f70 2024-04-26T21:42:29.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.686+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v95'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.686+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.687+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v95'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.687+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.690+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v95'10 uv3 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.690+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.691+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v95'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.691+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.695 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:42:29.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.694+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v95'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.694+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.697+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v95'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.697+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.700+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v95'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.700+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.701+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v95'2 uv1 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.701+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.702+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v95'3 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.702+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.703 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:42:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.703+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v95'4 uv4 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.703+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.704+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v95'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.704+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486e5f70 2024-04-26T21:42:29.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.705+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v95'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbcc06f070 con 0x5596486e5f70 2024-04-26T21:42:29.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.705+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486ea260 2024-04-26T21:42:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.706+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v95'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.706+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486ea260 2024-04-26T21:42:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.707+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v95'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.707+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486ea260 2024-04-26T21:42:29.708 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:42:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.708+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v95'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.708+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486eef50 con 0x5596486ea260 2024-04-26T21:42:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.709+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v95'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.709+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486eebf0 con 0x5596486ea260 2024-04-26T21:42:29.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.710+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v95'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.710+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edbe0 con 0x5596486ea260 2024-04-26T21:42:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.714+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v95'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.714+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486ea260 2024-04-26T21:42:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.715+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v95'4 uv3 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.715+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486ea260 2024-04-26T21:42:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.716+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v95'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.716+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edb50 con 0x5596486ea260 2024-04-26T21:42:29.720 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:42:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.719+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v95'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.719+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edf90 con 0x5596486ea260 2024-04-26T21:42:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.721+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v95'7 uv7 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.722+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edf90 con 0x5596486ea260 2024-04-26T21:42:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.724+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v95'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.724+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edc30 con 0x5596486ea260 2024-04-26T21:42:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.729+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v95'9 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.729+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486edc30 con 0x5596486ea260 2024-04-26T21:42:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.730+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v95'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.730+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edc10 con 0x5596486ea260 2024-04-26T21:42:29.734 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:42:29.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.733+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v95'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.733+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486edc50 con 0x5596486ea260 2024-04-26T21:42:29.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.736+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v95'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.736+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f0a20 con 0x5596486ea260 2024-04-26T21:42:29.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.737+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v95'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.737+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5596486ee040 0x5596486f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.737+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f10b0 con 0x5596486ee040 2024-04-26T21:42:29.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.737+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5596486ee040 0x5596486f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.738+0000 7fdbd3ea5640 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5596486ee040 0x5596486f0430 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.739+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v95'4 uv3 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486ee040 2024-04-26T21:42:29.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.739+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486f10b0 con 0x5596486ee040 2024-04-26T21:42:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.740+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v95'5 uv5 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486ee040 2024-04-26T21:42:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.740+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f10b0 con 0x5596486ee040 2024-04-26T21:42:29.742 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:42:29.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.741+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v95'6 uv6 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486ee040 2024-04-26T21:42:29.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.741+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1560 con 0x5596486ee040 2024-04-26T21:42:29.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.742+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v95'7 uv7 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486ee040 2024-04-26T21:42:29.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.742+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1560 con 0x5596486ea260 2024-04-26T21:42:29.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.743+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v95'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.743+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486f1560 con 0x5596486ea260 2024-04-26T21:42:29.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.744+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v95'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.744+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486ea260 2024-04-26T21:42:29.746 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:42:29.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.745+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v95'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.745+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f2520 con 0x5596486ea260 2024-04-26T21:42:29.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.746+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v95'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.746+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f2520 con 0x5596486ea260 2024-04-26T21:42:29.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.748+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v95'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.748+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486ea260 2024-04-26T21:42:29.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.749+0000 7fdbd46a6640 1 -- 172.21.15.40:0/2579436788 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v95'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc4011a10 con 0x5596486ea260 2024-04-26T21:42:29.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.749+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.750+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v95'2 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.750+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.751+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v95'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.751+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.753 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:42:29.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.752+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v95'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.753+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.754+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v95'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.754+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.755+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v95'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.755+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.756+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v95'7 uv3 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.756+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.757+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v95'8 uv8 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.757+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.759 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:42:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.758+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v95'9 uv9 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.758+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.759+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v95'10 uv10 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.759+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.760+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v95'11 uv11 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.761+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.762+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v95'12 uv6 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.762+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.763+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v95'13 uv13 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.763+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.764 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:42:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.764+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v95'14 uv14 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.764+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f2520 con 0x5596486e26f0 2024-04-26T21:42:29.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.764+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v95'15 uv15 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.764+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f2520 con 0x5596486e26f0 2024-04-26T21:42:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.765+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v95'16 uv16 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.766+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1510 con 0x5596486e26f0 2024-04-26T21:42:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.766+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v95'17 uv17 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.766+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5596486f1a70 con 0x5596486e26f0 2024-04-26T21:42:29.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd3ea5640 1 -- 172.21.15.40:0/2579436788 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v95'18 uv18 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbc8035040 con 0x5596486e26f0 2024-04-26T21:42:29.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5596486de180 msgr2=0x5596486e0570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5596486de180 0x5596486e0570 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559648634690 msgr2=0x559648634a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559648634690 0x559648634a70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5596486e5f70 msgr2=0x5596486e63c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5596486e5f70 0x5596486e63c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5596486ee040 msgr2=0x5596486f0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5596486ee040 0x5596486f0430 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5596486e26f0 msgr2=0x5596486e4ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5596486e26f0 0x5596486e4ae0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 reap_dead start 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5596486ea260 msgr2=0x5596486ec650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5596486ea260 0x5596486ec650 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 reap_dead start 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd4ea7640 1 -- 172.21.15.40:0/2579436788 reap_dead start 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc006b2d0 msgr2=0x7fdbc006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc006b2d0 0x7fdbc006d790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4002fd0 tx=0x7fdbc4001730 comp rx=0 tx=0).stop 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648632000 msgr2=0x55964863b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648632000 0x55964863b2e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00a170 tx=0x7fdbcc0033a0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 shutdown_connections 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5596486ea260 0x5596486ec650 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc006b2d0 0x7fdbc006d790 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 --2- 172.21.15.40:0/2579436788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559648632000 0x55964863b2e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 >> 172.21.15.40:0/2579436788 conn(0x55964858f8a0 msgr2=0x559648632f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 shutdown_connections 2024-04-26T21:42:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.768+0000 7fdbd781ef40 1 -- 172.21.15.40:0/2579436788 wait complete. 2024-04-26T21:42:29.772 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-26T21:42:29.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.854+0000 7f3cda420f40 1 Processor -- start 2024-04-26T21:42:29.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.854+0000 7f3cda420f40 1 -- start start 2024-04-26T21:42:29.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cda420f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efe7a80 0x557a6ef2b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cda420f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7e60 0x557a6efe8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cda420f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efeaf30 0x557a6f000910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cda420f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557a6eed3e40 con 0x557a6efe7a80 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cda420f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557a6eed7e80 con 0x557a6efeaf30 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cda420f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557a6ef74570 con 0x557a6efe7e60 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efe7a80 0x557a6ef2b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efe7a80 0x557a6ef2b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39372/0 (socket says 172.21.15.40:39372) 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 -- 172.21.15.40:0/2439359378 learned_addr learned my addr 172.21.15.40:0/2439359378 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd6aa5640 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7e60 0x557a6efe8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd7aa7640 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efeaf30 0x557a6f000910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 -- 172.21.15.40:0/2439359378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7e60 msgr2=0x557a6efe8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7e60 0x557a6efe8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 -- 172.21.15.40:0/2439359378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efeaf30 msgr2=0x557a6f000910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efeaf30 0x557a6f000910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 -- 172.21.15.40:0/2439359378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6efea5e0 con 0x557a6efe7a80 2024-04-26T21:42:29.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd72a6640 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efe7a80 0x557a6ef2b9a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800b5e0 tx=0x7f3cc800baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c1cd98563b8416 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd5aa3640 1 -- 172.21.15.40:0/2439359378 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8008070 con 0x557a6efe7a80 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd5aa3640 1 -- 172.21.15.40:0/2439359378 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc80098c0 con 0x557a6efe7a80 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.855+0000 7f3cd5aa3640 1 -- 172.21.15.40:0/2439359378 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8009cb0 con 0x557a6efe7a80 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 -- 172.21.15.40:0/2439359378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efe7a80 msgr2=0x557a6ef2b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efe7a80 0x557a6ef2b9a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800b5e0 tx=0x7f3cc800baa0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 -- 172.21.15.40:0/2439359378 shutdown_connections 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efeaf30 0x557a6f000910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7e60 0x557a6efe8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 --2- 172.21.15.40:0/2439359378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efe7a80 0x557a6ef2b9a0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 -- 172.21.15.40:0/2439359378 >> 172.21.15.40:0/2439359378 conn(0x557a6ef2aab0 msgr2=0x557a6efe6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 -- 172.21.15.40:0/2439359378 shutdown_connections 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 -- 172.21.15.40:0/2439359378 wait complete. 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.856+0000 7f3cda420f40 1 Processor -- start 2024-04-26T21:42:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cda420f40 1 -- start start 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cda420f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7a80 0x557a6ef70810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cda420f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efe7e60 0x557a6ef59640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cda420f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efeaf30 0x557a6effcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cda420f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557a6efe85f0 con 0x557a6efeaf30 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cda420f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557a6ef75e70 con 0x557a6efe7e60 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cda420f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557a6efe82c0 con 0x557a6efe7a80 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cd7aa7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efeaf30 0x557a6effcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cd7aa7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efeaf30 0x557a6effcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39374/0 (socket says 172.21.15.40:39374) 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cd7aa7640 1 -- 172.21.15.40:0/3738220485 learned_addr learned my addr 172.21.15.40:0/3738220485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cd72a6640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7a80 0x557a6ef70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.857+0000 7f3cd6aa5640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efe7e60 0x557a6ef59640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd7aa7640 1 -- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7a80 msgr2=0x557a6ef70810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd7aa7640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557a6efe7a80 0x557a6ef70810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd7aa7640 1 -- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efe7e60 msgr2=0x557a6ef59640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd7aa7640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a6efe7e60 0x557a6ef59640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd7aa7640 1 -- 172.21.15.40:0/3738220485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6ef70d50 con 0x557a6efeaf30 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd7aa7640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efeaf30 0x557a6effcb80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000d3e0 tx=0x7f3cd000d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd4aa1640 1 -- 172.21.15.40:0/3738220485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00125f0 con 0x557a6efeaf30 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd4aa1640 1 -- 172.21.15.40:0/3738220485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd0018070 con 0x557a6efeaf30 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557a6efe15e0 con 0x557a6efeaf30 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cd4aa1640 1 -- 172.21.15.40:0/3738220485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00129b0 con 0x557a6efeaf30 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.858+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6efe6300 con 0x557a6efeaf30 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.859+0000 7f3cd4aa1640 1 -- 172.21.15.40:0/3738220485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cd0012b50 con 0x557a6efeaf30 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.860+0000 7f3cd4aa1640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cc406b2d0 0x7f3cc406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.860+0000 7f3cd4aa1640 1 -- 172.21.15.40:0/3738220485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f3cd008a7e0 con 0x557a6efeaf30 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.860+0000 7f3cd72a6640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cc406b2d0 0x7f3cc406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.860+0000 7f3cda420f40 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557a6efe67b0 0x557a6efe0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.860+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x557a6ef2a6d0 con 0x557a6efe67b0 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.860+0000 7f3cd6aa5640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557a6efe67b0 0x557a6efe0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.860+0000 7f3cd72a6640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cc406b2d0 0x7f3cc406d790 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800b5e0 tx=0x7f3cc800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.860+0000 7f3cd6aa5640 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557a6efe67b0 0x557a6efe0540 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cd6aa5640 1 -- 172.21.15.40:0/3738220485 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v95'42 uv42 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3ccc0545f0 con 0x557a6efe67b0 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557a6efe67b0 msgr2=0x557a6efe0540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557a6efe67b0 0x557a6efe0540 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cc406b2d0 msgr2=0x7f3cc406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cc406b2d0 0x7f3cc406d790 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800b5e0 tx=0x7f3cc800f000 comp rx=0 tx=0).stop 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efeaf30 msgr2=0x557a6effcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 --2- 172.21.15.40:0/3738220485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a6efeaf30 0x557a6effcb80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000d3e0 tx=0x7f3cd000d8a0 comp rx=0 tx=0).stop 2024-04-26T21:42:29.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cd7aa7640 1 -- 172.21.15.40:0/3738220485 reap_dead start 2024-04-26T21:42:29.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 shutdown_connections 2024-04-26T21:42:29.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 >> 172.21.15.40:0/3738220485 conn(0x557a6ef2aab0 msgr2=0x557a6ef2e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:29.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 shutdown_connections 2024-04-26T21:42:29.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:29.862+0000 7f3cda420f40 1 -- 172.21.15.40:0/3738220485 wait complete. 2024-04-26T21:42:29.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:42:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.010+0000 7f265e2bd640 1 Processor -- start 2024-04-26T21:42:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.010+0000 7f265e2bd640 1 -- start start 2024-04-26T21:42:30.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f265e2bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f265815f170 0x7f26580d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f265e2bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f265815f550 0x7f26580d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f265e2bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26580d1040 con 0x7f26580d34b0 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f265e2bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f26580c9430 con 0x7f265815f170 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f265e2bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26580c9680 con 0x7f265815f550 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f2657fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f265815f170 0x7f26580d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f2657fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f265815f170 0x7f26580d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39386/0 (socket says 172.21.15.40:39386) 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f2657fff640 1 -- 172.21.15.40:0/3571653848 learned_addr learned my addr 172.21.15.40:0/3571653848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f26577fe640 1 --2- 172.21.15.40:0/3571653848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f265815f550 0x7f26580d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f2657fff640 1 -- 172.21.15.40:0/3571653848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f265815f550 msgr2=0x7f26580d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f2657fff640 1 --2- 172.21.15.40:0/3571653848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f265815f550 0x7f26580d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f2657fff640 1 -- 172.21.15.40:0/3571653848 >> v1:172.21.15.184:6789/0 conn(0x7f26580d34b0 legacy=0x7f26580cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.011+0000 7f2657fff640 1 -- 172.21.15.40:0/3571653848 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26580c9960 con 0x7f265815f170 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.012+0000 7f2657fff640 1 --2- 172.21.15.40:0/3571653848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f26580d5a80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f264c009870 tx=0x7f264c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980b72434ec9f984 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.012+0000 7f2656ffd640 1 -- 172.21.15.40:0/3571653848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c037070 con 0x7f265815f170 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.012+0000 7f2656ffd640 1 -- 172.21.15.40:0/3571653848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c032440 con 0x7f265815f170 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.012+0000 7f2656ffd640 1 -- 172.21.15.40:0/3571653848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c032810 con 0x7f265815f170 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.013+0000 7f265e2bd640 1 -- 172.21.15.40:0/3571653848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 msgr2=0x7f26580d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.013+0000 7f265e2bd640 1 --2- 172.21.15.40:0/3571653848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f26580d5a80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f264c009870 tx=0x7f264c02a270 comp rx=0 tx=0).stop 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.013+0000 7f265e2bd640 1 -- 172.21.15.40:0/3571653848 shutdown_connections 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.013+0000 7f265e2bd640 1 --2- 172.21.15.40:0/3571653848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f265815f550 0x7f26580d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.013+0000 7f265e2bd640 1 --2- 172.21.15.40:0/3571653848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f26580d5a80 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.013+0000 7f265e2bd640 1 -- 172.21.15.40:0/3571653848 >> 172.21.15.40:0/3571653848 conn(0x7f26580d4f60 msgr2=0x7f26580d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.013+0000 7f265e2bd640 1 -- 172.21.15.40:0/3571653848 shutdown_connections 2024-04-26T21:42:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.013+0000 7f265e2bd640 1 -- 172.21.15.40:0/3571653848 wait complete. 2024-04-26T21:42:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.014+0000 7f265e2bd640 1 Processor -- start 2024-04-26T21:42:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.014+0000 7f265e2bd640 1 -- start start 2024-04-26T21:42:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f265e2bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26580d34b0 0x7f265815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f265e2bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f265815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f265e2bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26581524d0 0x7f265815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f265e2bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26580cb5d0 con 0x7f265815f170 2024-04-26T21:42:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f265e2bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26580cb450 con 0x7f26580d34b0 2024-04-26T21:42:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f265e2bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26580cb750 con 0x7f26581524d0 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f26577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f265815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f26577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f265815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39388/0 (socket says 172.21.15.40:39388) 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f26577fe640 1 -- 172.21.15.40:0/1212789122 learned_addr learned my addr 172.21.15.40:0/1212789122 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f265c85a640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26581524d0 0x7f265815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f2657fff640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26580d34b0 0x7f265815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f26577fe640 1 -- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26581524d0 msgr2=0x7f265815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f26577fe640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26581524d0 0x7f265815b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f26577fe640 1 -- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26580d34b0 msgr2=0x7f265815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f26577fe640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26580d34b0 0x7f265815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f26577fe640 1 -- 172.21.15.40:0/1212789122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2658159f40 con 0x7f265815f170 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.015+0000 7f2657fff640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26580d34b0 0x7f265815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.016+0000 7f26577fe640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f265815b1f0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f264800d620 tx=0x7f264800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:30.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.016+0000 7f2654ff9640 1 -- 172.21.15.40:0/1212789122 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264800b620 con 0x7f265815f170 2024-04-26T21:42:30.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.016+0000 7f2654ff9640 1 -- 172.21.15.40:0/1212789122 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648019070 con 0x7f265815f170 2024-04-26T21:42:30.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.016+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265815be20 con 0x7f265815f170 2024-04-26T21:42:30.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.016+0000 7f2654ff9640 1 -- 172.21.15.40:0/1212789122 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264800ba00 con 0x7f265815f170 2024-04-26T21:42:30.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.016+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265815c100 con 0x7f265815f170 2024-04-26T21:42:30.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.017+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26580694f0 con 0x7f265815f170 2024-04-26T21:42:30.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.017+0000 7f2654ff9640 1 -- 172.21.15.40:0/1212789122 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f264800bba0 con 0x7f265815f170 2024-04-26T21:42:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.018+0000 7f2654ff9640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264406c8f0 0x7f264406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.018+0000 7f2654ff9640 1 -- 172.21.15.40:0/1212789122 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f264808c1b0 con 0x7f265815f170 2024-04-26T21:42:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.018+0000 7f2657fff640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264406c8f0 0x7f264406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.018+0000 7f2657fff640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264406c8f0 0x7f264406edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f264c034000 tx=0x7f264c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.022+0000 7f2654ff9640 1 -- 172.21.15.40:0/1212789122 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f264805a370 con 0x7f265815f170 2024-04-26T21:42:30.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.248+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f26580a2ba0 con 0x7f265815f170 2024-04-26T21:42:30.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.505+0000 7f2654ff9640 1 -- 172.21.15.40:0/1212789122 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v92) ==== 162+0+0 (secure 0 0 0) 0x7f2648059c70 con 0x7f265815f170 2024-04-26T21:42:30.506 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:42:30.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264406c8f0 msgr2=0x7f264406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:30.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264406c8f0 0x7f264406edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f264c034000 tx=0x7f264c02e000 comp rx=0 tx=0).stop 2024-04-26T21:42:30.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 msgr2=0x7f265815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f265815b1f0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f264800d620 tx=0x7f264800dae0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 shutdown_connections 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f264406c8f0 0x7f264406edb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26581524d0 0x7f265815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f265815f170 0x7f265815b1f0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 --2- 172.21.15.40:0/1212789122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26580d34b0 0x7f265815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 >> 172.21.15.40:0/1212789122 conn(0x7f26580d4f60 msgr2=0x7f265806ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 shutdown_connections 2024-04-26T21:42:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:30.507+0000 7f265e2bd640 1 -- 172.21.15.40:0/1212789122 wait complete. 2024-04-26T21:42:30.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:31.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.027+0000 7f95cb450640 1 Processor -- start 2024-04-26T21:42:31.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.028+0000 7f95cb450640 1 -- start start 2024-04-26T21:42:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.028+0000 7f95cb450640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c4162810 0x7f95c4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.028+0000 7f95cb450640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c40d10d0 con 0x7f95c40b2340 2024-04-26T21:42:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.028+0000 7f95cb450640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c40d1370 con 0x7f95c40d1b30 2024-04-26T21:42:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.028+0000 7f95cb450640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40d0750 con 0x7f95c4162810 2024-04-26T21:42:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95c89eb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f95c40b2340 0x7f95c40cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52236/0 (socket says 172.21.15.40:52236) 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95c89eb640 1 -- 172.21.15.40:0/1397495667 learned_addr learned my addr 172.21.15.40:0/1397495667 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95c91ec640 1 --2- 172.21.15.40:0/1397495667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c4162810 0x7f95c4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95bbfff640 1 -- 172.21.15.40:0/1397495667 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2228905272 0 0) 0x7f95c40d10d0 con 0x7f95c40b2340 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95bbfff640 1 -- 172.21.15.40:0/1397495667 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95b0003520 con 0x7f95c40b2340 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95bbfff640 1 -- 172.21.15.40:0/1397495667 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3200988983 0 0) 0x7f95c40d1370 con 0x7f95c40d1b30 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95bbfff640 1 -- 172.21.15.40:0/1397495667 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95c40d10d0 con 0x7f95c40d1b30 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95c91ec640 1 -- 172.21.15.40:0/1397495667 >> v1:172.21.15.184:6789/0 conn(0x7f95c40d1b30 legacy=0x7f95c4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95c91ec640 1 -- 172.21.15.40:0/1397495667 >> v1:172.21.15.40:6789/0 conn(0x7f95c40b2340 legacy=0x7f95c40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95bbfff640 1 -- 172.21.15.40:0/1397495667 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1846506637 0 0) 0x7f95b0003520 con 0x7f95c40b2340 2024-04-26T21:42:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.029+0000 7f95c91ec640 1 -- 172.21.15.40:0/1397495667 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c40d01b0 con 0x7f95c4162810 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.030+0000 7f95c91ec640 1 --2- 172.21.15.40:0/1397495667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c4162810 0x7f95c4070350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f95c00097d0 tx=0x7f95c002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919509fe59347ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.030+0000 7f95bbfff640 1 -- 172.21.15.40:0/1397495667 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0037070 con 0x7f95c4162810 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.030+0000 7f95bbfff640 1 -- 172.21.15.40:0/1397495667 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c002ac30 con 0x7f95c4162810 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.030+0000 7f95bbfff640 1 -- 172.21.15.40:0/1397495667 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4001670 con 0x7f95c4162810 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.031+0000 7f95cb450640 1 -- 172.21.15.40:0/1397495667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c4162810 msgr2=0x7f95c4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.031+0000 7f95cb450640 1 --2- 172.21.15.40:0/1397495667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c4162810 0x7f95c4070350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f95c00097d0 tx=0x7f95c002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.031+0000 7f95cb450640 1 -- 172.21.15.40:0/1397495667 shutdown_connections 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.031+0000 7f95cb450640 1 --2- 172.21.15.40:0/1397495667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c4162810 0x7f95c4070350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.031+0000 7f95cb450640 1 -- 172.21.15.40:0/1397495667 >> 172.21.15.40:0/1397495667 conn(0x7f95c40d2da0 msgr2=0x7f95c40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.031+0000 7f95cb450640 1 -- 172.21.15.40:0/1397495667 shutdown_connections 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.031+0000 7f95cb450640 1 -- 172.21.15.40:0/1397495667 wait complete. 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.032+0000 7f95cb450640 1 Processor -- start 2024-04-26T21:42:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.032+0000 7f95cb450640 1 -- start start 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95cb450640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40b2340 0x7f95c4150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95cb450640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c40d1b30 0x7f95c40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95cb450640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c4162810 0x7f95c40b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95cb450640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4070890 con 0x7f95c4162810 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95cb450640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40d16a0 con 0x7f95c40d1b30 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95cb450640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40cdb00 con 0x7f95c40b2340 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c4162810 0x7f95c40b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c4162810 0x7f95c40b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39392/0 (socket says 172.21.15.40:39392) 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 -- 172.21.15.40:0/994031320 learned_addr learned my addr 172.21.15.40:0/994031320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c89eb640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c40d1b30 0x7f95c40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c91ec640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40b2340 0x7f95c4150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 -- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40b2340 msgr2=0x7f95c4150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40b2340 0x7f95c4150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 -- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c40d1b30 msgr2=0x7f95c40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c40d1b30 0x7f95c40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 -- 172.21.15.40:0/994031320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c40ca050 con 0x7f95c4162810 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c89eb640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c40d1b30 0x7f95c40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.033+0000 7f95c99ed640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c4162810 0x7f95c40b0b00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c790 tx=0x7f95bc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.034+0000 7f95b9ffb640 1 -- 172.21.15.40:0/994031320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc003b70 con 0x7f95c4162810 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.034+0000 7f95b9ffb640 1 -- 172.21.15.40:0/994031320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc003d10 con 0x7f95c4162810 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.034+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c40ca330 con 0x7f95c4162810 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.034+0000 7f95b9ffb640 1 -- 172.21.15.40:0/994031320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00aa50 con 0x7f95c4162810 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.034+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c40ca8f0 con 0x7f95c4162810 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.035+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c40694f0 con 0x7f95c4162810 2024-04-26T21:42:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.035+0000 7f95b9ffb640 1 -- 172.21.15.40:0/994031320 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95bc00abf0 con 0x7f95c4162810 2024-04-26T21:42:31.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.036+0000 7f95b9ffb640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95a806c8f0 0x7f95a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:31.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.036+0000 7f95b9ffb640 1 -- 172.21.15.40:0/994031320 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f95bc08bcb0 con 0x7f95c4162810 2024-04-26T21:42:31.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.036+0000 7f95c91ec640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95a806c8f0 0x7f95a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:31.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.036+0000 7f95c91ec640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95a806c8f0 0x7f95a806edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95c0034000 tx=0x7f95c002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.040+0000 7f95b9ffb640 1 -- 172.21.15.40:0/994031320 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95bc059f00 con 0x7f95c4162810 2024-04-26T21:42:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.268+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95c40cd010 con 0x7f95c4162810 2024-04-26T21:42:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.268+0000 7f95b9ffb640 1 -- 172.21.15.40:0/994031320 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5183 (secure 0 0 0) 0x7f95bc059800 con 0x7f95c4162810 2024-04-26T21:42:31.270 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:31.270 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:30.511087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7974":{"gid":7974,"name":"a","rank":0,"incarnation":93,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6835/2103981319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2103981319},{"type":"v1","addr":"172.21.15.40:6835","nonce":2103981319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:31.270 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 93 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95a806c8f0 msgr2=0x7f95a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95a806c8f0 0x7f95a806edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95c0034000 tx=0x7f95c002e000 comp rx=0 tx=0).stop 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c4162810 msgr2=0x7f95c40b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c4162810 0x7f95c40b0b00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c790 tx=0x7f95bc00cc50 comp rx=0 tx=0).stop 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 shutdown_connections 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95a806c8f0 0x7f95a806edb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c4162810 0x7f95c40b0b00 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c40d1b30 0x7f95c40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 --2- 172.21.15.40:0/994031320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40b2340 0x7f95c4150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.270+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 >> 172.21.15.40:0/994031320 conn(0x7f95c40d2da0 msgr2=0x7f95c40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.271+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 shutdown_connections 2024-04-26T21:42:31.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:31.271+0000 7f95cb450640 1 -- 172.21.15.40:0/994031320 wait complete. 2024-04-26T21:42:31.282 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:42:30.511087+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7974}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7974': {'gid': 7974, 'name': 'a', 'rank': 0, 'incarnation': 93, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6835/2103981319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 2103981319}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 2103981319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:42:31.282 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7974:up:replay 2024-04-26T21:42:31.282 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:42:35.485 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:42:35.484+0000 7f26f1157640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:42:36.686 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:36.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.799+0000 7f19ea22a640 1 Processor -- start 2024-04-26T21:42:36.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.799+0000 7f19ea22a640 1 -- start start 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19ea22a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f19e40cf5e0 0x7f19e40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19ea22a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19e40cbbd0 0x7f19e40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19ea22a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e415b9e0 con 0x7f19e415b600 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19ea22a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f19e40cb9b0 con 0x7f19e40cbbd0 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19ea22a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f19e40d0740 con 0x7f19e40cf5e0 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19e3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19e40cbbd0 0x7f19e40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19e3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19e40cbbd0 0x7f19e40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39394/0 (socket says 172.21.15.40:39394) 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19e3fff640 1 -- 172.21.15.40:0/849744495 learned_addr learned my addr 172.21.15.40:0/849744495 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19e2ffd640 1 --2- 172.21.15.40:0/849744495 >> v2:172.21.15.184:3300/0 conn(0x7f19e40cf5e0 0x7f19e40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19e3fff640 1 -- 172.21.15.40:0/849744495 >> v2:172.21.15.184:3300/0 conn(0x7f19e40cf5e0 msgr2=0x7f19e40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19e3fff640 1 --2- 172.21.15.40:0/849744495 >> v2:172.21.15.184:3300/0 conn(0x7f19e40cf5e0 0x7f19e40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19e3fff640 1 -- 172.21.15.40:0/849744495 >> v1:172.21.15.184:6789/0 conn(0x7f19e415b600 legacy=0x7f19e4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.800+0000 7f19e3fff640 1 -- 172.21.15.40:0/849744495 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e40cb360 con 0x7f19e40cbbd0 2024-04-26T21:42:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.801+0000 7f19e3fff640 1 --2- 172.21.15.40:0/849744495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e40cbbd0 0x7f19e40cef10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f19e415bbb0 tx=0x7f19dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd531850df810bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.801+0000 7f19e27fc640 1 -- 172.21.15.40:0/849744495 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc036070 con 0x7f19e40cbbd0 2024-04-26T21:42:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.801+0000 7f19e27fc640 1 -- 172.21.15.40:0/849744495 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19dc031600 con 0x7f19e40cbbd0 2024-04-26T21:42:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.801+0000 7f19e27fc640 1 -- 172.21.15.40:0/849744495 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0319d0 con 0x7f19e40cbbd0 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.801+0000 7f19ea22a640 1 -- 172.21.15.40:0/849744495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e40cbbd0 msgr2=0x7f19e40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.801+0000 7f19ea22a640 1 --2- 172.21.15.40:0/849744495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e40cbbd0 0x7f19e40cef10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f19e415bbb0 tx=0x7f19dc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.802+0000 7f19ea22a640 1 -- 172.21.15.40:0/849744495 shutdown_connections 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.802+0000 7f19ea22a640 1 --2- 172.21.15.40:0/849744495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e40cbbd0 0x7f19e40cef10 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.802+0000 7f19ea22a640 1 --2- 172.21.15.40:0/849744495 >> v2:172.21.15.184:3300/0 conn(0x7f19e40cf5e0 0x7f19e40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.802+0000 7f19ea22a640 1 -- 172.21.15.40:0/849744495 >> 172.21.15.40:0/849744495 conn(0x7f19e40d0be0 msgr2=0x7f19e40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.802+0000 7f19ea22a640 1 -- 172.21.15.40:0/849744495 shutdown_connections 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.802+0000 7f19ea22a640 1 -- 172.21.15.40:0/849744495 wait complete. 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.803+0000 7f19ea22a640 1 Processor -- start 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.803+0000 7f19ea22a640 1 -- start start 2024-04-26T21:42:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19ea22a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19e40cbbd0 0x7f19e414ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19ea22a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19e40cf5e0 0x7f19e40ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19ea22a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e415b600 0x7f19e414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19ea22a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40d0470 con 0x7f19e415b600 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19ea22a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40d02f0 con 0x7f19e40cbbd0 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19ea22a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19e406ddf0 con 0x7f19e40cf5e0 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e415b600 0x7f19e414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e415b600 0x7f19e414b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39404/0 (socket says 172.21.15.40:39404) 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 -- 172.21.15.40:0/2469400455 learned_addr learned my addr 172.21.15.40:0/2469400455 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e37fe640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19e40cbbd0 0x7f19e414ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e2ffd640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19e40cf5e0 0x7f19e40ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 -- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19e40cf5e0 msgr2=0x7f19e40ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19e40cf5e0 0x7f19e40ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 -- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19e40cbbd0 msgr2=0x7f19e414ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19e40cbbd0 0x7f19e414ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 -- 172.21.15.40:0/2469400455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e40af980 con 0x7f19e415b600 2024-04-26T21:42:36.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.804+0000 7f19e3fff640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e415b600 0x7f19e414b6b0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009f10 tx=0x7f19dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:36.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.805+0000 7f19cbfff640 1 -- 172.21.15.40:0/2469400455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc036050 con 0x7f19e415b600 2024-04-26T21:42:36.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.805+0000 7f19cbfff640 1 -- 172.21.15.40:0/2469400455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19dc042070 con 0x7f19e415b600 2024-04-26T21:42:36.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.805+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e40afc60 con 0x7f19e415b600 2024-04-26T21:42:36.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.805+0000 7f19cbfff640 1 -- 172.21.15.40:0/2469400455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc009510 con 0x7f19e415b600 2024-04-26T21:42:36.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.805+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e4071d60 con 0x7f19e415b600 2024-04-26T21:42:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.806+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e40694f0 con 0x7f19e415b600 2024-04-26T21:42:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.806+0000 7f19cbfff640 1 -- 172.21.15.40:0/2469400455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19dc007bd0 con 0x7f19e415b600 2024-04-26T21:42:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.807+0000 7f19cbfff640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19d006c8f0 0x7f19d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.807+0000 7f19cbfff640 1 -- 172.21.15.40:0/2469400455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f19dc0aca80 con 0x7f19e415b600 2024-04-26T21:42:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.807+0000 7f19e37fe640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19d006c8f0 0x7f19d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.808+0000 7f19e37fe640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19d006c8f0 0x7f19d006edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f19d40016e0 tx=0x7f19d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:36.812+0000 7f19cbfff640 1 -- 172.21.15.40:0/2469400455 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19dc07a680 con 0x7f19e415b600 2024-04-26T21:42:37.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.035+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19e40ce7b0 con 0x7f19e415b600 2024-04-26T21:42:37.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.035+0000 7f19cbfff640 1 -- 172.21.15.40:0/2469400455 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5190 (secure 0 0 0) 0x7f19dc07e980 con 0x7f19e415b600 2024-04-26T21:42:37.036 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:37.036 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":96,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:33.525572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7974":{"gid":7974,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6835/2103981319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2103981319},{"type":"v1","addr":"172.21.15.40:6835","nonce":2103981319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7974,"qdb_cluster":[7974]},"id":6}]} 2024-04-26T21:42:37.037 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 96 2024-04-26T21:42:37.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19d006c8f0 msgr2=0x7f19d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19d006c8f0 0x7f19d006edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f19d40016e0 tx=0x7f19d4009040 comp rx=0 tx=0).stop 2024-04-26T21:42:37.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e415b600 msgr2=0x7f19e414b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e415b600 0x7f19e414b6b0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009f10 tx=0x7f19dc004a40 comp rx=0 tx=0).stop 2024-04-26T21:42:37.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 shutdown_connections 2024-04-26T21:42:37.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f19d006c8f0 0x7f19d006edb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e415b600 0x7f19e414b6b0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19e40cf5e0 0x7f19e40ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 --2- 172.21.15.40:0/2469400455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19e40cbbd0 0x7f19e414ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 >> 172.21.15.40:0/2469400455 conn(0x7f19e40d0be0 msgr2=0x7f19e40cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 shutdown_connections 2024-04-26T21:42:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.037+0000 7f19ea22a640 1 -- 172.21.15.40:0/2469400455 wait complete. 2024-04-26T21:42:37.049 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:42:33.525572+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7974}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7974': {'gid': 7974, 'name': 'a', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.40:6835/2103981319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 2103981319}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 2103981319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7974, 'qdb_cluster': [7974]} 2024-04-26T21:42:37.049 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:42:37.049 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:42:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.150+0000 7f40c05af640 1 Processor -- start 2024-04-26T21:42:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.150+0000 7f40c05af640 1 -- start start 2024-04-26T21:42:37.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40c05af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 0x7f40b8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40c05af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f40b80d1b30 0x7f40b80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40c05af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b80d10d0 con 0x7f40b80b2340 2024-04-26T21:42:37.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40c05af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f40b80d0750 con 0x7f40b80d1b30 2024-04-26T21:42:37.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40c05af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40b80d16a0 con 0x7f40b8162810 2024-04-26T21:42:37.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40beb4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f40b80d1b30 0x7f40b80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40beb4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f40b80d1b30 0x7f40b80d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39406/0 (socket says 172.21.15.40:39406) 2024-04-26T21:42:37.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40beb4c640 1 -- 172.21.15.40:0/2433341270 learned_addr learned my addr 172.21.15.40:0/2433341270 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:37.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40be34b640 1 --2- 172.21.15.40:0/2433341270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 0x7f40b8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40beb4c640 1 -- 172.21.15.40:0/2433341270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 msgr2=0x7f40b8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40beb4c640 1 --2- 172.21.15.40:0/2433341270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 0x7f40b8070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40beb4c640 1 -- 172.21.15.40:0/2433341270 >> v1:172.21.15.184:6789/0 conn(0x7f40b80b2340 legacy=0x7f40b80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.151+0000 7f40beb4c640 1 -- 172.21.15.40:0/2433341270 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b80d1370 con 0x7f40b80d1b30 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.152+0000 7f40beb4c640 1 --2- 172.21.15.40:0/2433341270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80d1b30 0x7f40b80d1fc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009e50 tx=0x7f40a802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43669c0afd8407ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.152+0000 7f40bd349640 1 -- 172.21.15.40:0/2433341270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8036070 con 0x7f40b80d1b30 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.152+0000 7f40bd349640 1 -- 172.21.15.40:0/2433341270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8031600 con 0x7f40b80d1b30 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.152+0000 7f40bd349640 1 -- 172.21.15.40:0/2433341270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80319d0 con 0x7f40b80d1b30 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.153+0000 7f40c05af640 1 -- 172.21.15.40:0/2433341270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80d1b30 msgr2=0x7f40b80d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.153+0000 7f40c05af640 1 --2- 172.21.15.40:0/2433341270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80d1b30 0x7f40b80d1fc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009e50 tx=0x7f40a802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.153+0000 7f40c05af640 1 -- 172.21.15.40:0/2433341270 shutdown_connections 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.153+0000 7f40c05af640 1 --2- 172.21.15.40:0/2433341270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80d1b30 0x7f40b80d1fc0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.153+0000 7f40c05af640 1 --2- 172.21.15.40:0/2433341270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 0x7f40b8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.153+0000 7f40c05af640 1 -- 172.21.15.40:0/2433341270 >> 172.21.15.40:0/2433341270 conn(0x7f40b80d2da0 msgr2=0x7f40b80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.153+0000 7f40c05af640 1 -- 172.21.15.40:0/2433341270 shutdown_connections 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.153+0000 7f40c05af640 1 -- 172.21.15.40:0/2433341270 wait complete. 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.154+0000 7f40c05af640 1 Processor -- start 2024-04-26T21:42:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.154+0000 7f40c05af640 1 -- start start 2024-04-26T21:42:37.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40c05af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80b2340 0x7f40b80ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40c05af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40b80d1b30 0x7f40b80c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40c05af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 0x7f40b80cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40c05af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8070890 con 0x7f40b80b2340 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40c05af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40b80d0470 con 0x7f40b80d1b30 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40c05af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40b80cdb00 con 0x7f40b8162810 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80b2340 0x7f40b80ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80b2340 0x7f40b80ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39414/0 (socket says 172.21.15.40:39414) 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 -- 172.21.15.40:0/1755805602 learned_addr learned my addr 172.21.15.40:0/1755805602 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40b80d1b30 0x7f40b80c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40beb4c640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 0x7f40b80cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 -- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 msgr2=0x7f40b80cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 0x7f40b80cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 -- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40b80d1b30 msgr2=0x7f40b80c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40b80d1b30 0x7f40b80c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 -- 172.21.15.40:0/1755805602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b80ca8c0 con 0x7f40b80b2340 2024-04-26T21:42:37.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40be34b640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80b2340 0x7f40b80ca290 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f40b000c820 tx=0x7f40b000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40b6ffd640 1 -- 172.21.15.40:0/1755805602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0015070 con 0x7f40b80b2340 2024-04-26T21:42:37.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.155+0000 7f40b6ffd640 1 -- 172.21.15.40:0/1755805602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b0004020 con 0x7f40b80b2340 2024-04-26T21:42:37.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.156+0000 7f40c05af640 1 -- 172.21.15.40:0/1755805602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b80c8ff0 con 0x7f40b80b2340 2024-04-26T21:42:37.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.156+0000 7f40b6ffd640 1 -- 172.21.15.40:0/1755805602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b00043f0 con 0x7f40b80b2340 2024-04-26T21:42:37.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.156+0000 7f40c05af640 1 -- 172.21.15.40:0/1755805602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b80c95b0 con 0x7f40b80b2340 2024-04-26T21:42:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.157+0000 7f40b6ffd640 1 -- 172.21.15.40:0/1755805602 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40b0007830 con 0x7f40b80b2340 2024-04-26T21:42:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.157+0000 7f40b6ffd640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40a406c920 0x7f40a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.158+0000 7f40b6ffd640 1 -- 172.21.15.40:0/1755805602 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f40b008baa0 con 0x7f40b80b2340 2024-04-26T21:42:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.158+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40a406c920 0x7f40a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.158+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40a406c920 0x7f40a406ede0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f40b80b3070 tx=0x7f40ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.159+0000 7f40c33a1740 1 Processor -- start 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.159+0000 7f40c33a1740 1 -- start start 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.159+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668caa0440 0x55668caa0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.159+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cabe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.159+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668ca76120 0x55668cabc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40c33a1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55668c9da740 con 0x55668ca76120 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40c33a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55668c9763a0 con 0x55668cabdce0 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40c33a1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55668c9d6700 con 0x55668caa0440 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668ca76120 0x55668cabc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668ca76120 0x55668cabc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39422/0 (socket says 172.21.15.40:39422) 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 -- 172.21.15.40:0/3864932100 learned_addr learned my addr 172.21.15.40:0/3864932100 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cabe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40be34b640 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668caa0440 0x55668caa0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 -- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668caa0440 msgr2=0x55668caa0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668caa0440 0x55668caa0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 -- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 msgr2=0x55668cabe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cabe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 -- 172.21.15.40:0/3864932100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668c992750 con 0x55668ca76120 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cabe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f40beb4c640 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668ca76120 0x55668cabc3a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f40a8034040 tx=0x7f40a802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deadd58e528b1b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f409ffff640 1 -- 172.21.15.40:0/3864932100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8036040 con 0x55668ca76120 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f409ffff640 1 -- 172.21.15.40:0/3864932100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8031780 con 0x55668ca76120 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.160+0000 7f409ffff640 1 -- 172.21.15.40:0/3864932100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8031b30 con 0x55668ca76120 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 -- 172.21.15.40:0/3864932100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668ca76120 msgr2=0x55668cabc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668ca76120 0x55668cabc3a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f40a8034040 tx=0x7f40a802e000 comp rx=0 tx=0).stop 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 -- 172.21.15.40:0/3864932100 shutdown_connections 2024-04-26T21:42:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668ca76120 0x55668cabc3a0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cabe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 --2- 172.21.15.40:0/3864932100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668caa0440 0x55668caa0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 -- 172.21.15.40:0/3864932100 >> 172.21.15.40:0/3864932100 conn(0x55668ca72a50 msgr2=0x55668cabb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 -- 172.21.15.40:0/3864932100 shutdown_connections 2024-04-26T21:42:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.161+0000 7f40c33a1740 1 -- 172.21.15.40:0/3864932100 wait complete. 2024-04-26T21:42:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.162+0000 7f40c33a1740 1 -- start start 2024-04-26T21:42:37.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.162+0000 7f40c33a1740 1 Processor -- start 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cab95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cab3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668cace800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55668ca76ec0 con 0x55668caa0440 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55668c961fe0 con 0x55668cabdce0 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55668ca70980 con 0x55668cab4000 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 10 client.0 _ll_register_callbacks cb 0x55668c978150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cab95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cab95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39434/0 (socket says 172.21.15.40:39434) 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 -- 172.21.15.40:0/382977843 learned_addr learned my addr 172.21.15.40:0/382977843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40beb4c640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668cace800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cab3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 -- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 msgr2=0x55668cace800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668cace800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 -- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 msgr2=0x55668cab3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cab3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 -- 172.21.15.40:0/382977843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668c6ec2d0 con 0x55668caa0440 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40beb4c640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668cace800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cab3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40be34b640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cab95d0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f40b000c7f0 tx=0x7f40b0059db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcbae364b09a4e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f409d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f409d7fa640 1 -- 172.21.15.40:0/382977843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b001e040 con 0x55668caa0440 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f409d7fa640 1 -- 172.21.15.40:0/382977843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f40b005ed30 con 0x55668caa0440 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 1 -- 172.21.15.40:0/382977843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a8031780 con 0x55668caa0440 2024-04-26T21:42:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f409d7fa640 1 -- 172.21.15.40:0/382977843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f40b00621d0 con 0x55668caa0440 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 10 client.8448 fetch_fsmap learned FSMap version 96 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.163+0000 7f40c33a1740 1 -- 172.21.15.40:0/382977843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55668c7b66b0 con 0x55668caa0440 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f409d7fa640 1 -- 172.21.15.40:0/382977843 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8031780 con 0x55668caa0440 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f409d7fa640 1 -- 172.21.15.40:0/382977843 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 96) ==== 3494+0+0 (secure 0 0 0) 0x7f40b0058a40 con 0x55668caa0440 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f40c33a1740 10 client.8448 fetch_fsmap finished waiting for FSMap version 96 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f40c33a1740 10 client.8448 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f40c33a1740 20 client.8448 populate_metadata read hostname 'smithi040' 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f40c33a1740 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] conn(0x55668cab6ad0 0x55668cabb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f40c33a1740 4 client.8448 mds_command: new command op to 7974 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f40c33a1740 1 -- 172.21.15.40:0/382977843 --> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55668c9e8350 con 0x55668cab6ad0 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] conn(0x55668cab6ad0 0x55668cabb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f409d7fa640 10 client.8448 ms_handle_connect on v2:172.21.15.40:6834/2103981319 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.164+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] conn(0x55668cab6ad0 0x55668cabb310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c338bdf4f79ea08f server_cookie=6b19d36eacaa5f8b in_seq=0 out_seq=0 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f409d7fa640 1 -- 172.21.15.40:0/382977843 <== mds.0 v2:172.21.15.40:6834/2103981319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55668c9e8350 con 0x55668cab6ad0 2024-04-26T21:42:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f409d7fa640 10 client.8448 handle_command_reply: tid=0 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 client.8448 shutdown 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 -- 172.21.15.40:0/382977843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 msgr2=0x55668cab95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cab95d0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f40b000c7f0 tx=0x7f40b0059db0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 -- 172.21.15.40:0/382977843 shutdown_connections 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] conn(0x55668cab6ad0 0x55668cabb310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668cace800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f409d7fa640 0 client.8448 ms_handle_reset on v2:172.21.15.40:6834/2103981319 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668cab3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 --2- 172.21.15.40:0/382977843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cab95d0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 -- 172.21.15.40:0/382977843 >> 172.21.15.40:0/382977843 conn(0x55668ca76120 msgr2=0x55668cabf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 -- 172.21.15.40:0/382977843 shutdown_connections 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 1 -- 172.21.15.40:0/382977843 wait complete. 2024-04-26T21:42:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.165+0000 7f40c33a1740 20 client.8448 trim_cache size 0 max 16384 2024-04-26T21:42:37.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40c33a1740 1 Processor -- start 2024-04-26T21:42:37.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40c33a1740 1 -- start start 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cb300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668ca74420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668ca6f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40c33a1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8201000 con 0x55668caa0440 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40c33a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8200360 con 0x55668cabdce0 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40c33a1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40b82009b0 con 0x55668cab4000 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40be34b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cb300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40be34b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cb300c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39440/0 (socket says 172.21.15.40:39440) 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.184+0000 7f40be34b640 1 -- 172.21.15.40:0/4123043878 learned_addr learned my addr 172.21.15.40:0/4123043878 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668ca74420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f40beb4c640 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668ca6f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f40be34b640 1 -- 172.21.15.40:0/4123043878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 msgr2=0x55668ca74420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f40be34b640 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668ca74420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f40be34b640 1 -- 172.21.15.40:0/4123043878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 msgr2=0x55668ca6f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f40be34b640 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668ca6f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f40be34b640 1 -- 172.21.15.40:0/4123043878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668cacadb0 con 0x55668caa0440 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f40be34b640 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cb300c0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f40b0059ea0 tx=0x7f40b005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba812232fdd6c83d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f409f7fe640 1 -- 172.21.15.40:0/4123043878 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b001e040 con 0x55668caa0440 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f409f7fe640 1 -- 172.21.15.40:0/4123043878 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b0007d70 con 0x55668caa0440 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.185+0000 7f409f7fe640 1 -- 172.21.15.40:0/4123043878 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b005ed70 con 0x55668caa0440 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 -- 172.21.15.40:0/4123043878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 msgr2=0x55668cb300c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cb300c0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f40b0059ea0 tx=0x7f40b005b000 comp rx=0 tx=0).stop 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 -- 172.21.15.40:0/4123043878 shutdown_connections 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668cabdce0 0x55668ca6f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cab4000 0x55668ca74420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 --2- 172.21.15.40:0/4123043878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668caa0440 0x55668cb300c0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 -- 172.21.15.40:0/4123043878 >> 172.21.15.40:0/4123043878 conn(0x55668ca76120 msgr2=0x55668caca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 -- 172.21.15.40:0/4123043878 shutdown_connections 2024-04-26T21:42:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 -- 172.21.15.40:0/4123043878 wait complete. 2024-04-26T21:42:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 -- start start 2024-04-26T21:42:37.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.186+0000 7f40c33a1740 1 Processor -- start 2024-04-26T21:42:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668caa0440 0x55668cacde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668cab4000 0x55668ca6f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40c33a1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cabdce0 0x55668ca6fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40c33a1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55668cab6950 con 0x55668cab4000 2024-04-26T21:42:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40c33a1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55668cacf210 con 0x55668caa0440 2024-04-26T21:42:37.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40c33a1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55668cab6f10 con 0x55668cabdce0 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40c33a1740 10 client.0 _ll_register_callbacks cb 0x7f40b0004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40bdb4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668cab4000 0x55668ca6f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40bdb4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668cab4000 0x55668ca6f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39446/0 (socket says 172.21.15.40:39446) 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40bdb4a640 1 -- 172.21.15.40:0/1893750316 learned_addr learned my addr 172.21.15.40:0/1893750316 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40be34b640 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668caa0440 0x55668cacde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.187+0000 7f40beb4c640 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cabdce0 0x55668ca6fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40bdb4a640 1 -- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cabdce0 msgr2=0x55668ca6fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cabdce0 0x55668ca6fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40bdb4a640 1 -- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668caa0440 msgr2=0x55668cacde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668caa0440 0x55668cacde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40bdb4a640 1 -- 172.21.15.40:0/1893750316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668ca9e150 con 0x55668cab4000 2024-04-26T21:42:37.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40beb4c640 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cabdce0 0x55668ca6fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:37.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40bdb4a640 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668cab4000 0x55668ca6f670 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x55668ca9fad0 tx=0x7f40ac0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb62d30e5cb7f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f409d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:42:37.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f409d7fa640 1 -- 172.21.15.40:0/1893750316 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac01e040 con 0x55668cab4000 2024-04-26T21:42:37.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f409d7fa640 1 -- 172.21.15.40:0/1893750316 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f40ac031cd0 con 0x55668cab4000 2024-04-26T21:42:37.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 1 -- 172.21.15.40:0/1893750316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b0007d70 con 0x55668cab4000 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f409d7fa640 1 -- 172.21.15.40:0/1893750316 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f40ac027720 con 0x55668cab4000 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 10 client.8454 fetch_fsmap learned FSMap version 96 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 1 -- 172.21.15.40:0/1893750316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55668cb36e80 con 0x55668cab4000 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f409d7fa640 1 -- 172.21.15.40:0/1893750316 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b0007d70 con 0x55668cab4000 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f409d7fa640 1 -- 172.21.15.40:0/1893750316 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 96) ==== 3494+0+0 (secure 0 0 0) 0x7f40ac02e020 con 0x55668cab4000 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 10 client.8454 fetch_fsmap finished waiting for FSMap version 96 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 10 client.8454 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 20 client.8454 populate_metadata read hostname 'smithi040' 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] conn(0x55668cab0be0 0x55668cab0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 4 client.8454 mds_command: new command op to 7974 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.188+0000 7f40c33a1740 1 -- 172.21.15.40:0/1893750316 --> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] -- command(tid 0: {"prefix": "status"}) -- 0x55668c9e21c0 con 0x55668cab0be0 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40be34b640 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] conn(0x55668cab0be0 0x55668cab0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40be34b640 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] conn(0x55668cab0be0 0x55668cab0fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d268dae2fa54d6ef server_cookie=39a834e93db38670 in_seq=0 out_seq=0 2024-04-26T21:42:37.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f409d7fa640 10 client.8454 ms_handle_connect on v2:172.21.15.40:6834/2103981319 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f409d7fa640 1 -- 172.21.15.40:0/1893750316 <== mds.0 v2:172.21.15.40:6834/2103981319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55668c9e21c0 con 0x55668cab0be0 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f409d7fa640 10 client.8454 handle_command_reply: tid=0 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 client.8454 shutdown 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 -- 172.21.15.40:0/1893750316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668cab4000 msgr2=0x55668ca6f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668cab4000 0x55668ca6f670 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x55668ca9fad0 tx=0x7f40ac0036f0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 -- 172.21.15.40:0/1893750316 shutdown_connections 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.40:6834/2103981319,v1:172.21.15.40:6835/2103981319] conn(0x55668cab0be0 0x55668cab0fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55668cabdce0 0x55668ca6fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55668cab4000 0x55668ca6f670 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 --2- 172.21.15.40:0/1893750316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55668caa0440 0x55668cacde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f409d7fa640 0 client.8454 ms_handle_reset on v2:172.21.15.40:6834/2103981319 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 -- 172.21.15.40:0/1893750316 >> 172.21.15.40:0/1893750316 conn(0x55668ca76120 msgr2=0x55668ca6bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 -- 172.21.15.40:0/1893750316 shutdown_connections 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.189+0000 7f40c33a1740 1 -- 172.21.15.40:0/1893750316 wait complete. 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c33a1740 20 client.8454 trim_cache size 0 max 16384 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7974, 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.6763345090000001, 2024-04-26T21:42:37.198 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 96, 2024-04-26T21:42:37.199 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 95, 2024-04-26T21:42:37.199 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 95, 2024-04-26T21:42:37.199 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 13.673692089999999 2024-04-26T21:42:37.199 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:42:37.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 -- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40a406c920 msgr2=0x7f40a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40a406c920 0x7f40a406ede0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f40b80b3070 tx=0x7f40ac009040 comp rx=0 tx=0).stop 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 -- 172.21.15.40:0/1755805602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80b2340 msgr2=0x7f40b80ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80b2340 0x7f40b80ca290 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f40b000c820 tx=0x7f40b000cce0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 -- 172.21.15.40:0/1755805602 shutdown_connections 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40a406c920 0x7f40a406ede0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40b8162810 0x7f40b80cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40b80d1b30 0x7f40b80c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 --2- 172.21.15.40:0/1755805602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b80b2340 0x7f40b80ca290 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 -- 172.21.15.40:0/1755805602 >> 172.21.15.40:0/1755805602 conn(0x7f40b80d2da0 msgr2=0x7f40b8070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 -- 172.21.15.40:0/1755805602 shutdown_connections 2024-04-26T21:42:37.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.190+0000 7f40c05af640 1 -- 172.21.15.40:0/1755805602 wait complete. 2024-04-26T21:42:37.201 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:42:37.202 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 10000000000.00000000', proceeding to ls 2024-04-26T21:42:37.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:42:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.302+0000 7f51fc196640 1 Processor -- start 2024-04-26T21:42:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.302+0000 7f51fc196640 1 -- start start 2024-04-26T21:42:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51fc196640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 0x7f51f40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51fc196640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f40b7e50 con 0x7f51f4162880 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51fc196640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f40d1800 con 0x7f51f40d3bc0 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51fc196640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51f40d0580 con 0x7f51f40b5f20 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51f9f32640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51f4162880 0x7f51f40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52244/0 (socket says 172.21.15.40:52244) 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51f9f32640 1 -- 172.21.15.40:0/193779841 learned_addr learned my addr 172.21.15.40:0/193779841 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51fa733640 1 --2- 172.21.15.40:0/193779841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 0x7f51f40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51f8f30640 1 -- 172.21.15.40:0/193779841 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625660216 0 0) 0x7f51f40b7e50 con 0x7f51f4162880 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51f8f30640 1 -- 172.21.15.40:0/193779841 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51e0003520 con 0x7f51f4162880 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.303+0000 7f51f8f30640 1 -- 172.21.15.40:0/193779841 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887132288 0 0) 0x7f51e4001110 con 0x7f51f40d3bc0 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.304+0000 7f51f8f30640 1 -- 172.21.15.40:0/193779841 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51f40b7e50 con 0x7f51f40d3bc0 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.304+0000 7f51f8f30640 1 -- 172.21.15.40:0/193779841 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3109774840 0 0) 0x7f51e0003520 con 0x7f51f4162880 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.304+0000 7f51fa733640 1 -- 172.21.15.40:0/193779841 >> v1:172.21.15.184:6789/0 conn(0x7f51f40d3bc0 legacy=0x7f51f40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.304+0000 7f51fa733640 1 -- 172.21.15.40:0/193779841 >> v1:172.21.15.40:6789/0 conn(0x7f51f4162880 legacy=0x7f51f40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.304+0000 7f51fa733640 1 -- 172.21.15.40:0/193779841 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f40d07d0 con 0x7f51f40b5f20 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.304+0000 7f51fa733640 1 --2- 172.21.15.40:0/193779841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 0x7f51f40d2cb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51f40d0700 tx=0x7f51ec02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5edfad09e39561 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51f8f30640 1 -- 172.21.15.40:0/193779841 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0001670 con 0x7f51f40b5f20 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51f8f30640 1 -- 172.21.15.40:0/193779841 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ec007540 con 0x7f51f40b5f20 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51f8f30640 1 -- 172.21.15.40:0/193779841 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0078f0 con 0x7f51f40b5f20 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51fc196640 1 -- 172.21.15.40:0/193779841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 msgr2=0x7f51f40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51fc196640 1 --2- 172.21.15.40:0/193779841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 0x7f51f40d2cb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51f40d0700 tx=0x7f51ec02a660 comp rx=0 tx=0).stop 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51fc196640 1 -- 172.21.15.40:0/193779841 shutdown_connections 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51fc196640 1 --2- 172.21.15.40:0/193779841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 0x7f51f40d2cb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51fc196640 1 -- 172.21.15.40:0/193779841 >> 172.21.15.40:0/193779841 conn(0x7f51f40d4f60 msgr2=0x7f51f40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51fc196640 1 -- 172.21.15.40:0/193779841 shutdown_connections 2024-04-26T21:42:37.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.305+0000 7f51fc196640 1 -- 172.21.15.40:0/193779841 wait complete. 2024-04-26T21:42:37.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.306+0000 7f51fc196640 1 Processor -- start 2024-04-26T21:42:37.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.306+0000 7f51fc196640 1 -- start start 2024-04-26T21:42:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fc196640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 0x7f51f40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fc196640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f40d3bc0 0x7f51f41f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fc196640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f4162880 0x7f51f40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fc196640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4162c60 con 0x7f51f4162880 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fc196640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51f40d0d90 con 0x7f51f40d3bc0 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fc196640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51f4162de0 con 0x7f51f40b5f20 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fa733640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f4162880 0x7f51f40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fa733640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f4162880 0x7f51f40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39452/0 (socket says 172.21.15.40:39452) 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fa733640 1 -- 172.21.15.40:0/796368810 learned_addr learned my addr 172.21.15.40:0/796368810 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fa733640 1 -- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 msgr2=0x7f51f40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fa733640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 0x7f51f40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fa733640 1 -- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f40d3bc0 msgr2=0x7f51f41f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fa733640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f40d3bc0 0x7f51f41f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.307+0000 7f51fa733640 1 -- 172.21.15.40:0/796368810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f40cd5d0 con 0x7f51f4162880 2024-04-26T21:42:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.308+0000 7f51fa733640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f4162880 0x7f51f40ce670 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009eb0 tx=0x7f51ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.308+0000 7f51ea7fc640 1 -- 172.21.15.40:0/796368810 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec03f070 con 0x7f51f4162880 2024-04-26T21:42:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.308+0000 7f51ea7fc640 1 -- 172.21.15.40:0/796368810 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ec031c40 con 0x7f51f4162880 2024-04-26T21:42:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.308+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f40cd8b0 con 0x7f51f4162880 2024-04-26T21:42:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.308+0000 7f51ea7fc640 1 -- 172.21.15.40:0/796368810 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec009450 con 0x7f51f4162880 2024-04-26T21:42:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.308+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f40cde70 con 0x7f51f4162880 2024-04-26T21:42:37.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.309+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f40d4750 con 0x7f51f4162880 2024-04-26T21:42:37.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.309+0000 7f51ea7fc640 1 -- 172.21.15.40:0/796368810 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51ec043410 con 0x7f51f4162880 2024-04-26T21:42:37.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.310+0000 7f51ea7fc640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51d806c8f0 0x7f51d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.310+0000 7f51ea7fc640 1 -- 172.21.15.40:0/796368810 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f51ec0ad070 con 0x7f51f4162880 2024-04-26T21:42:37.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.310+0000 7f51f9f32640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51d806c8f0 0x7f51d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.311+0000 7f51f9f32640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51d806c8f0 0x7f51d806edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f51f0001e10 tx=0x7f51f0000f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.314+0000 7f51ea7fc640 1 -- 172.21.15.40:0/796368810 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51f40d4750 con 0x7f51f4162880 2024-04-26T21:42:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.537+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f51f415f0f0 con 0x7f51f4162880 2024-04-26T21:42:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.538+0000 7f51ea7fc640 1 -- 172.21.15.40:0/796368810 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) ==== 53+0+73 (secure 0 0 0) 0x7f51ec07b2c0 con 0x7f51f4162880 2024-04-26T21:42:37.539 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:42:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51d806c8f0 msgr2=0x7f51d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51d806c8f0 0x7f51d806edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f51f0001e10 tx=0x7f51f0000f90 comp rx=0 tx=0).stop 2024-04-26T21:42:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f4162880 msgr2=0x7f51f40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f4162880 0x7f51f40ce670 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009eb0 tx=0x7f51ec004a40 comp rx=0 tx=0).stop 2024-04-26T21:42:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 shutdown_connections 2024-04-26T21:42:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51d806c8f0 0x7f51d806edb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f4162880 0x7f51f40ce670 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f40d3bc0 0x7f51f41f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 --2- 172.21.15.40:0/796368810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f40b5f20 0x7f51f40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 >> 172.21.15.40:0/796368810 conn(0x7f51f40d4f60 msgr2=0x7f51f40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 shutdown_connections 2024-04-26T21:42:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37.540+0000 7f51fc196640 1 -- 172.21.15.40:0/796368810 wait complete. 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:42:37.552 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:42:37.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:37.553 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:42:37.568 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:42:37.568 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:42:37.568 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:42:37.568 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:37.568 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:42:37.568 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42290sec preferred_lft 42290sec 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:37.569 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:42:37.570 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:42:37.571 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:42:37.571 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:42:37.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:37.571 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:42:37.631 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:42:37.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:37.633 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:42:37.689 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:42:37.689 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:42:37.689 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:42:37.689 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:42:37.689 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:42:37.690 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:42:37.690 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:42:37.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:42:37.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:42:37.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:42:37.690 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:42:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:42:37.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:42:37.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:37.815 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:42:37.830 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:42:37.830 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:42:37.831 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:42:37.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:37.831 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:42:37.895 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:42:37.895 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:37.949 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:42:37.950 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:42:37.950 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:38.004 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:42:38.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:38.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:42:38.087 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:42:38.087 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:42:38.094 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:42:38.095 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:42:38.095 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:42:38.095 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:42:38.095 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:42:38.095 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:42:38.095 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:42:38.095 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:42:38.095 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:42:38.096 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:42:38.097 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:42:38.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:38.097 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:42:38.121 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:42:38.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:42:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:42:38.219 INFO:teuthology.orchestra.run.smithi040.stdout:client6629 2024-04-26T21:42:38.219 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:38.219 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:38.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:38.235 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-26T21:42:38.278 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:42:38.291 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:42:38.291+0000 7f26ed14f640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-04-26T21:42:38.292 INFO:teuthology.orchestra.run.smithi040.stdout:.: 2024-04-26T21:42:38.292 INFO:teuthology.orchestra.run.smithi040.stdout:subdir 2024-04-26T21:42:38.292 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:38.292 INFO:teuthology.orchestra.run.smithi040.stdout:./subdir: 2024-04-26T21:42:38.293 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 10000000000.00000000' did not caused DAMAGED state 2024-04-26T21:42:38.293 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 600.00000000' 2024-04-26T21:42:38.293 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:38.293 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:38.348 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:38.348 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:38.405 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:42:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.544+0000 7f7956207640 1 Processor -- start 2024-04-26T21:42:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.544+0000 7f7956207640 1 -- start start 2024-04-26T21:42:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.545+0000 7f7956207640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f795015f290 0x7f79500cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.545+0000 7f7956207640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f795006f5a0 con 0x7f79500cd060 2024-04-26T21:42:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.545+0000 7f7956207640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f795006f880 con 0x7f79500cd4a0 2024-04-26T21:42:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.545+0000 7f7956207640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f795006f350 con 0x7f795015f290 2024-04-26T21:42:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.545+0000 7f794effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f79500cd060 0x7f7950162e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52254/0 (socket says 172.21.15.40:52254) 2024-04-26T21:42:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.545+0000 7f794effd640 1 -- 172.21.15.40:0/1379520189 learned_addr learned my addr 172.21.15.40:0/1379520189 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.545+0000 7f794f7fe640 1 --2- 172.21.15.40:0/1379520189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f795015f290 0x7f79500cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.546+0000 7f794e7fc640 1 -- 172.21.15.40:0/1379520189 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 161002573 0 0) 0x7f795006f5a0 con 0x7f79500cd060 2024-04-26T21:42:38.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.546+0000 7f794e7fc640 1 -- 172.21.15.40:0/1379520189 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7938003520 con 0x7f79500cd060 2024-04-26T21:42:38.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.546+0000 7f794f7fe640 1 -- 172.21.15.40:0/1379520189 >> v1:172.21.15.184:6789/0 conn(0x7f79500cd4a0 legacy=0x7f79500726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.546+0000 7f794f7fe640 1 -- 172.21.15.40:0/1379520189 >> v1:172.21.15.40:6789/0 conn(0x7f79500cd060 legacy=0x7f7950162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.546+0000 7f794e7fc640 1 -- 172.21.15.40:0/1379520189 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 383201320 0 0) 0x7f7938003520 con 0x7f79500cd060 2024-04-26T21:42:38.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.546+0000 7f794f7fe640 1 -- 172.21.15.40:0/1379520189 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795006fb60 con 0x7f795015f290 2024-04-26T21:42:38.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.547+0000 7f794f7fe640 1 --2- 172.21.15.40:0/1379520189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f795015f290 0x7f79500cb7e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79400097d0 tx=0x7f794002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6f0e36f662fffd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:38.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f794e7fc640 1 -- 172.21.15.40:0/1379520189 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940037070 con 0x7f795015f290 2024-04-26T21:42:38.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f794e7fc640 1 -- 172.21.15.40:0/1379520189 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794002acc0 con 0x7f795015f290 2024-04-26T21:42:38.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f794e7fc640 1 -- 172.21.15.40:0/1379520189 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944001670 con 0x7f795015f290 2024-04-26T21:42:38.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f7956207640 1 -- 172.21.15.40:0/1379520189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f795015f290 msgr2=0x7f79500cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f7956207640 1 --2- 172.21.15.40:0/1379520189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f795015f290 0x7f79500cb7e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79400097d0 tx=0x7f794002a230 comp rx=0 tx=0).stop 2024-04-26T21:42:38.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f7956207640 1 -- 172.21.15.40:0/1379520189 shutdown_connections 2024-04-26T21:42:38.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f7956207640 1 --2- 172.21.15.40:0/1379520189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f795015f290 0x7f79500cb7e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f7956207640 1 -- 172.21.15.40:0/1379520189 >> 172.21.15.40:0/1379520189 conn(0x7f79500d0fd0 msgr2=0x7f79500d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:38.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f7956207640 1 -- 172.21.15.40:0/1379520189 shutdown_connections 2024-04-26T21:42:38.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.552+0000 7f7956207640 1 -- 172.21.15.40:0/1379520189 wait complete. 2024-04-26T21:42:38.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.553+0000 7f7956207640 1 Processor -- start 2024-04-26T21:42:38.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.554+0000 7f7956207640 1 -- start start 2024-04-26T21:42:38.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.554+0000 7f7956207640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500cd060 0x7f7950154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.554+0000 7f7956207640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500cd4a0 0x7f79501615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.554+0000 7f7956207640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795015f290 0x7f7950161b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.554+0000 7f7956207640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7950163710 con 0x7f795015f290 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.554+0000 7f7956207640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7950163590 con 0x7f79500cd4a0 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.554+0000 7f7956207640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7950072dd0 con 0x7f79500cd060 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795015f290 0x7f7950161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795015f290 0x7f7950161b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39462/0 (socket says 172.21.15.40:39462) 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 -- 172.21.15.40:0/752190937 learned_addr learned my addr 172.21.15.40:0/752190937 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794f7fe640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500cd060 0x7f7950154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794effd640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500cd4a0 0x7f79501615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 -- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500cd060 msgr2=0x7f7950154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500cd060 0x7f7950154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 -- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500cd4a0 msgr2=0x7f79501615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500cd4a0 0x7f79501615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 -- 172.21.15.40:0/752190937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79501606b0 con 0x7f795015f290 2024-04-26T21:42:38.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f794ffff640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795015f290 0x7f7950161b30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f794800c7a0 tx=0x7f794800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:38.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f7937fff640 1 -- 172.21.15.40:0/752190937 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948003b70 con 0x7f795015f290 2024-04-26T21:42:38.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f7937fff640 1 -- 172.21.15.40:0/752190937 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7948003d30 con 0x7f795015f290 2024-04-26T21:42:38.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7950160990 con 0x7f795015f290 2024-04-26T21:42:38.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f7937fff640 1 -- 172.21.15.40:0/752190937 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794800aa90 con 0x7f795015f290 2024-04-26T21:42:38.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.555+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7950154950 con 0x7f795015f290 2024-04-26T21:42:38.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.556+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79500b3f30 con 0x7f795015f290 2024-04-26T21:42:38.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.557+0000 7f7937fff640 1 -- 172.21.15.40:0/752190937 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f794800ac30 con 0x7f795015f290 2024-04-26T21:42:38.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.557+0000 7f7937fff640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.557+0000 7f7937fff640 1 -- 172.21.15.40:0/752190937 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f794808bad0 con 0x7f795015f290 2024-04-26T21:42:38.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.557+0000 7f794f7fe640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.558+0000 7f794f7fe640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f793c06c8f0 0x7f793c06edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7940034000 tx=0x7f794002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:38.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.561+0000 7f7937fff640 1 -- 172.21.15.40:0/752190937 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79500b3f30 con 0x7f795015f290 2024-04-26T21:42:38.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.747+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79500694f0 con 0x7f795015f290 2024-04-26T21:42:38.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.749+0000 7f7937fff640 1 -- 172.21.15.40:0/752190937 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16978 (secure 0 0 0) 0x7f79480596e0 con 0x7f795015f290 2024-04-26T21:42:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:38.750 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":95,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:42:23.477686+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:42:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f793c06c8f0 msgr2=0x7f793c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f793c06c8f0 0x7f793c06edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7940034000 tx=0x7f794002e000 comp rx=0 tx=0).stop 2024-04-26T21:42:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795015f290 msgr2=0x7f7950161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795015f290 0x7f7950161b30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f794800c7a0 tx=0x7f794800cc60 comp rx=0 tx=0).stop 2024-04-26T21:42:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 shutdown_connections 2024-04-26T21:42:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795015f290 0x7f7950161b30 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500cd4a0 0x7f79501615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 --2- 172.21.15.40:0/752190937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500cd060 0x7f7950154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 >> 172.21.15.40:0/752190937 conn(0x7f79500d0fd0 msgr2=0x7f79500b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 shutdown_connections 2024-04-26T21:42:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.751+0000 7f7956207640 1 -- 172.21.15.40:0/752190937 wait complete. 2024-04-26T21:42:38.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:42:38.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:42:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.866+0000 7f0356d2b640 1 Processor -- start 2024-04-26T21:42:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.866+0000 7f0356d2b640 1 -- start start 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f0356d2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03501625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f0356d2b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0350162b30 0x7f03500b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f0356d2b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03500d0a10 con 0x7f03500d1a00 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f0356d2b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f03500d0cf0 con 0x7f0350162b30 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f0356d2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03500d0fd0 con 0x7f03500b40d0 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f0354ac7640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f03500d1a00 0x7f03500d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42966/0 (socket says 172.21.15.40:42966) 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f0354ac7640 1 -- 172.21.15.40:0/3546804761 learned_addr learned my addr 172.21.15.40:0/3546804761 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f034ffff640 1 --2- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03501625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.867+0000 7f03552c8640 1 --2- 172.21.15.40:0/3546804761 >> v2:172.21.15.184:3300/0 conn(0x7f0350162b30 0x7f03500b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.868+0000 7f034f7fe640 1 -- 172.21.15.40:0/3546804761 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 252216697 0 0) 0x7f03500d0a10 con 0x7f03500d1a00 2024-04-26T21:42:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.868+0000 7f034f7fe640 1 -- 172.21.15.40:0/3546804761 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0338003520 con 0x7f03500d1a00 2024-04-26T21:42:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.868+0000 7f03552c8640 1 -- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 msgr2=0x7f03501625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.868+0000 7f03552c8640 1 --2- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03501625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.868+0000 7f03552c8640 1 -- 172.21.15.40:0/3546804761 >> v1:172.21.15.40:6789/0 conn(0x7f03500d1a00 legacy=0x7f03500d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.868+0000 7f03552c8640 1 -- 172.21.15.40:0/3546804761 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0350162fa0 con 0x7f0350162b30 2024-04-26T21:42:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.868+0000 7f034ffff640 1 --2- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03501625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:38.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f03552c8640 1 --2- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 0x7f03500b4b00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f03400097a0 tx=0x7f034002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36353563e5b455cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:38.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f034f7fe640 1 -- 172.21.15.40:0/3546804761 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340037070 con 0x7f0350162b30 2024-04-26T21:42:38.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f034f7fe640 1 -- 172.21.15.40:0/3546804761 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034002ae50 con 0x7f0350162b30 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f034f7fe640 1 -- 172.21.15.40:0/3546804761 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340032640 con 0x7f0350162b30 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f0356d2b640 1 -- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 msgr2=0x7f03500b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f0356d2b640 1 --2- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 0x7f03500b4b00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f03400097a0 tx=0x7f034002a100 comp rx=0 tx=0).stop 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f0356d2b640 1 -- 172.21.15.40:0/3546804761 shutdown_connections 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f0356d2b640 1 --2- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 0x7f03500b4b00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f0356d2b640 1 --2- 172.21.15.40:0/3546804761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03501625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f0356d2b640 1 -- 172.21.15.40:0/3546804761 >> 172.21.15.40:0/3546804761 conn(0x7f03500d4fd0 msgr2=0x7f03500d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f0356d2b640 1 -- 172.21.15.40:0/3546804761 shutdown_connections 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.869+0000 7f0356d2b640 1 -- 172.21.15.40:0/3546804761 wait complete. 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.870+0000 7f0356d2b640 1 Processor -- start 2024-04-26T21:42:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.871+0000 7f0356d2b640 1 -- start start 2024-04-26T21:42:38.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.871+0000 7f0356d2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03500cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.871+0000 7f0356d2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03500d1a00 0x7f03500cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.871+0000 7f0356d2b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 0x7f03500ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.871+0000 7f0356d2b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03500d2670 con 0x7f03500d1a00 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.871+0000 7f0356d2b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03500d24f0 con 0x7f0350162b30 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.871+0000 7f0356d2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03500b5040 con 0x7f03500b40d0 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03500d1a00 0x7f03500cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03500d1a00 0x7f03500cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45700/0 (socket says 172.21.15.40:45700) 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 -- 172.21.15.40:0/1379177568 learned_addr learned my addr 172.21.15.40:0/1379177568 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f0354ac7640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03500cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f03552c8640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 0x7f03500ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 -- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 msgr2=0x7f03500cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03500cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 -- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 msgr2=0x7f03500ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 0x7f03500ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 -- 172.21.15.40:0/1379177568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03500ba150 con 0x7f03500d1a00 2024-04-26T21:42:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f03552c8640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 0x7f03500ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034ffff640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03500d1a00 0x7f03500cecd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f034400ec10 tx=0x7f034400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034d7fa640 1 -- 172.21.15.40:0/1379177568 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344017070 con 0x7f03500d1a00 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034d7fa640 1 -- 172.21.15.40:0/1379177568 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034400cc30 con 0x7f03500d1a00 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03500ba430 con 0x7f03500d1a00 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f034d7fa640 1 -- 172.21.15.40:0/1379177568 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344013470 con 0x7f03500d1a00 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.872+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03500cb340 con 0x7f03500d1a00 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.873+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03500694f0 con 0x7f03500d1a00 2024-04-26T21:42:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.874+0000 7f034d7fa640 1 -- 172.21.15.40:0/1379177568 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f034401a070 con 0x7f03500d1a00 2024-04-26T21:42:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.874+0000 7f034d7fa640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f033c06c920 0x7f033c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.874+0000 7f034d7fa640 1 -- 172.21.15.40:0/1379177568 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f034408bf20 con 0x7f03500d1a00 2024-04-26T21:42:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.874+0000 7f0354ac7640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f033c06c920 0x7f033c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.875+0000 7f0354ac7640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f033c06c920 0x7f033c06ede0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0348005c00 tx=0x7f0348009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:38.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:38.879+0000 7f034d7fa640 1 -- 172.21.15.40:0/1379177568 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0344059b20 con 0x7f03500d1a00 2024-04-26T21:42:39.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.061+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03500d53b0 con 0x7f03500d1a00 2024-04-26T21:42:39.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.062+0000 7f034d7fa640 1 -- 172.21.15.40:0/1379177568 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f034405de20 con 0x7f03500d1a00 2024-04-26T21:42:39.063 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:42:39.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f033c06c920 msgr2=0x7f033c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f033c06c920 0x7f033c06ede0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0348005c00 tx=0x7f0348009040 comp rx=0 tx=0).stop 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03500d1a00 msgr2=0x7f03500cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03500d1a00 0x7f03500cecd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f034400ec10 tx=0x7f034400c520 comp rx=0 tx=0).stop 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 shutdown_connections 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f033c06c920 0x7f033c06ede0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0350162b30 0x7f03500ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03500d1a00 0x7f03500cecd0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 --2- 172.21.15.40:0/1379177568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03500b40d0 0x7f03500cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.063+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 >> 172.21.15.40:0/1379177568 conn(0x7f03500d4fd0 msgr2=0x7f03500d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.064+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 shutdown_connections 2024-04-26T21:42:39.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.064+0000 7f0356d2b640 1 -- 172.21.15.40:0/1379177568 wait complete. 2024-04-26T21:42:39.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:42:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.178+0000 7f5fcee37640 1 Processor -- start 2024-04-26T21:42:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.179+0000 7f5fcee37640 1 -- start start 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.179+0000 7f5fcee37640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5fc806c950 0x7f5fc80ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.179+0000 7f5fcee37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc80cfd80 con 0x7f5fc80b0380 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.179+0000 7f5fcee37640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc80d0020 con 0x7f5fc80b10b0 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.179+0000 7f5fcee37640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc80af8e0 con 0x7f5fc806c950 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.179+0000 7f5fccbd3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5fc80b0380 0x7f5fc806c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42978/0 (socket says 172.21.15.40:42978) 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.179+0000 7f5fccbd3640 1 -- 172.21.15.40:0/1486772517 learned_addr learned my addr 172.21.15.40:0/1486772517 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.180+0000 7f5fcd3d4640 1 --2- 172.21.15.40:0/1486772517 >> v2:172.21.15.184:3300/0 conn(0x7f5fc806c950 0x7f5fc80ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.180+0000 7f5fc77fe640 1 -- 172.21.15.40:0/1486772517 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1729041230 0 0) 0x7f5fc80cfd80 con 0x7f5fc80b0380 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.180+0000 7f5fc77fe640 1 -- 172.21.15.40:0/1486772517 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fb8003520 con 0x7f5fc80b0380 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.180+0000 7f5fcd3d4640 1 -- 172.21.15.40:0/1486772517 >> v1:172.21.15.184:6789/0 conn(0x7f5fc80b10b0 legacy=0x7f5fc815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.180+0000 7f5fcd3d4640 1 -- 172.21.15.40:0/1486772517 >> v1:172.21.15.40:6789/0 conn(0x7f5fc80b0380 legacy=0x7f5fc806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.180+0000 7f5fcd3d4640 1 -- 172.21.15.40:0/1486772517 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc80d0300 con 0x7f5fc806c950 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fcd3d4640 1 --2- 172.21.15.40:0/1486772517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc806c950 0x7f5fc80ccd50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009d90 tx=0x7f5fc002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6df370895af69d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fc77fe640 1 -- 172.21.15.40:0/1486772517 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0007740 con 0x7f5fc806c950 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fc77fe640 1 -- 172.21.15.40:0/1486772517 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc00078e0 con 0x7f5fc806c950 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fc77fe640 1 -- 172.21.15.40:0/1486772517 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0007c90 con 0x7f5fc806c950 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fcee37640 1 -- 172.21.15.40:0/1486772517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc806c950 msgr2=0x7f5fc80ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fcee37640 1 --2- 172.21.15.40:0/1486772517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc806c950 0x7f5fc80ccd50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009d90 tx=0x7f5fc002a660 comp rx=0 tx=0).stop 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fcee37640 1 -- 172.21.15.40:0/1486772517 shutdown_connections 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fcee37640 1 --2- 172.21.15.40:0/1486772517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc806c950 0x7f5fc80ccd50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fcee37640 1 -- 172.21.15.40:0/1486772517 >> 172.21.15.40:0/1486772517 conn(0x7f5fc80d1cf0 msgr2=0x7f5fc80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fcee37640 1 -- 172.21.15.40:0/1486772517 shutdown_connections 2024-04-26T21:42:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.181+0000 7f5fcee37640 1 -- 172.21.15.40:0/1486772517 wait complete. 2024-04-26T21:42:39.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.183+0000 7f5fcee37640 1 Processor -- start 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.183+0000 7f5fcee37640 1 -- start start 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fcee37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fc806c950 0x7f5fc8157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fcee37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fc80b0380 0x7f5fc806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fcee37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc80b10b0 0x7f5fc806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fcee37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc80b0760 con 0x7f5fc80b0380 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fcee37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc80d1980 con 0x7f5fc80b10b0 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fcee37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc80d15c0 con 0x7f5fc806c950 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fc80b0380 0x7f5fc806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fc80b0380 0x7f5fc806ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45706/0 (socket says 172.21.15.40:45706) 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 -- 172.21.15.40:0/2327198135 learned_addr learned my addr 172.21.15.40:0/2327198135 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fcd3d4640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc80b10b0 0x7f5fc806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fccbd3640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fc806c950 0x7f5fc8157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 -- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fc806c950 msgr2=0x7f5fc8157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fc806c950 0x7f5fc8157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 -- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc80b10b0 msgr2=0x7f5fc806f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc80b10b0 0x7f5fc806f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 -- 172.21.15.40:0/2327198135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc80b5970 con 0x7f5fc80b0380 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc7fff640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fc80b0380 0x7f5fc806ef90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400c400 tx=0x7f5fb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc57fa640 1 -- 172.21.15.40:0/2327198135 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40052d0 con 0x7f5fc80b0380 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc57fa640 1 -- 172.21.15.40:0/2327198135 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb4005490 con 0x7f5fc80b0380 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc80b5c50 con 0x7f5fc80b0380 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.184+0000 7f5fc57fa640 1 -- 172.21.15.40:0/2327198135 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4007260 con 0x7f5fc80b0380 2024-04-26T21:42:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.185+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc80b6210 con 0x7f5fc80b0380 2024-04-26T21:42:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.186+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc80b6410 con 0x7f5fc80b0380 2024-04-26T21:42:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.186+0000 7f5fc57fa640 1 -- 172.21.15.40:0/2327198135 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fb4007400 con 0x7f5fc80b0380 2024-04-26T21:42:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.186+0000 7f5fc57fa640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fac06c8f0 0x7f5fac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.186+0000 7f5fc57fa640 1 -- 172.21.15.40:0/2327198135 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f5fb408b200 con 0x7f5fc80b0380 2024-04-26T21:42:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.187+0000 7f5fccbd3640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fac06c8f0 0x7f5fac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.187+0000 7f5fccbd3640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fac06c8f0 0x7f5fac06edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005880 tx=0x7f5fbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.191+0000 7f5fc57fa640 1 -- 172.21.15.40:0/2327198135 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fc80b6410 con 0x7f5fc80b0380 2024-04-26T21:42:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.374+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5fc80694f0 con 0x7f5fc80b0380 2024-04-26T21:42:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.375+0000 7f5fc57fa640 1 -- 172.21.15.40:0/2327198135 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f5fb4058e10 con 0x7f5fc80b0380 2024-04-26T21:42:39.376 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:42:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fac06c8f0 msgr2=0x7f5fac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fac06c8f0 0x7f5fac06edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005880 tx=0x7f5fbc009040 comp rx=0 tx=0).stop 2024-04-26T21:42:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fc80b0380 msgr2=0x7f5fc806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fc80b0380 0x7f5fc806ef90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400c400 tx=0x7f5fb400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 shutdown_connections 2024-04-26T21:42:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5fac06c8f0 0x7f5fac06edb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fc80b10b0 0x7f5fc806f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fc80b0380 0x7f5fc806ef90 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 --2- 172.21.15.40:0/2327198135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fc806c950 0x7f5fc8157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 >> 172.21.15.40:0/2327198135 conn(0x7f5fc80d1cf0 msgr2=0x7f5fc80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 shutdown_connections 2024-04-26T21:42:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.377+0000 7f5fcee37640 1 -- 172.21.15.40:0/2327198135 wait complete. 2024-04-26T21:42:39.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:42:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.492+0000 7fca296e1640 1 Processor -- start 2024-04-26T21:42:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.492+0000 7fca296e1640 1 -- start start 2024-04-26T21:42:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca296e1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca24162a20 0x7fca240d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca296e1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca240d1c40 0x7fca240d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca296e1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca240d2ed0 0x7fca240d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca296e1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fca240d07a0 con 0x7fca240d1c40 2024-04-26T21:42:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca296e1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fca240c8bc0 con 0x7fca240d2ed0 2024-04-26T21:42:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca296e1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca240d09f0 con 0x7fca24162a20 2024-04-26T21:42:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca1a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca240d1c40 0x7fca240d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca1a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca240d1c40 0x7fca240d20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45708/0 (socket says 172.21.15.40:45708) 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca1a7fc640 1 -- 172.21.15.40:0/2942721410 learned_addr learned my addr 172.21.15.40:0/2942721410 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca22ffd640 1 --2- 172.21.15.40:0/2942721410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca24162a20 0x7fca240d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca237fe640 1 --2- 172.21.15.40:0/2942721410 >> v2:172.21.15.184:3300/0 conn(0x7fca240d2ed0 0x7fca240d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca1a7fc640 1 -- 172.21.15.40:0/2942721410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca24162a20 msgr2=0x7fca240d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca1a7fc640 1 --2- 172.21.15.40:0/2942721410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca24162a20 0x7fca240d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca1a7fc640 1 -- 172.21.15.40:0/2942721410 >> v2:172.21.15.184:3300/0 conn(0x7fca240d2ed0 msgr2=0x7fca240d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca1a7fc640 1 --2- 172.21.15.40:0/2942721410 >> v2:172.21.15.184:3300/0 conn(0x7fca240d2ed0 0x7fca240d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.493+0000 7fca1a7fc640 1 -- 172.21.15.40:0/2942721410 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca240d0cd0 con 0x7fca240d1c40 2024-04-26T21:42:39.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.494+0000 7fca1a7fc640 1 --2- 172.21.15.40:0/2942721410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca240d20a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fca240d14a0 tx=0x7fca0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4bfe112efd36f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.494+0000 7fca227fc640 1 -- 172.21.15.40:0/2942721410 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037070 con 0x7fca240d1c40 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.494+0000 7fca227fc640 1 -- 172.21.15.40:0/2942721410 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c032440 con 0x7fca240d1c40 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.494+0000 7fca227fc640 1 -- 172.21.15.40:0/2942721410 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c032810 con 0x7fca240d1c40 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 -- 172.21.15.40:0/2942721410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 msgr2=0x7fca240d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 --2- 172.21.15.40:0/2942721410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca240d20a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fca240d14a0 tx=0x7fca0c02a140 comp rx=0 tx=0).stop 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 -- 172.21.15.40:0/2942721410 shutdown_connections 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 --2- 172.21.15.40:0/2942721410 >> v2:172.21.15.184:3300/0 conn(0x7fca240d2ed0 0x7fca240d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 --2- 172.21.15.40:0/2942721410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca240d20a0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 --2- 172.21.15.40:0/2942721410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca24162a20 0x7fca240d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 -- 172.21.15.40:0/2942721410 >> 172.21.15.40:0/2942721410 conn(0x7fca240d4fd0 msgr2=0x7fca240d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 -- 172.21.15.40:0/2942721410 shutdown_connections 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.495+0000 7fca296e1640 1 -- 172.21.15.40:0/2942721410 wait complete. 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.496+0000 7fca296e1640 1 Processor -- start 2024-04-26T21:42:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.496+0000 7fca296e1640 1 -- start start 2024-04-26T21:42:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca296e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca2415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca296e1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca240d2ed0 0x7fca240b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca296e1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24162a20 0x7fca2406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca296e1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca240cbd90 con 0x7fca240d1c40 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca296e1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca240d0fb0 con 0x7fca24162a20 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca296e1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca24162e00 con 0x7fca240d2ed0 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca2415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca2415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45712/0 (socket says 172.21.15.40:45712) 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 -- 172.21.15.40:0/3960035478 learned_addr learned my addr 172.21.15.40:0/3960035478 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca237fe640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24162a20 0x7fca2406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca1a7fc640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca240d2ed0 0x7fca240b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 -- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca240d2ed0 msgr2=0x7fca240b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca240d2ed0 0x7fca240b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 -- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24162a20 msgr2=0x7fca2406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24162a20 0x7fca2406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 -- 172.21.15.40:0/3960035478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2415d980 con 0x7fca240d1c40 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca237fe640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24162a20 0x7fca2406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca1a7fc640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca240d2ed0 0x7fca240b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.497+0000 7fca22ffd640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca2415d440 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fca1400e9b0 tx=0x7fca1400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.498+0000 7fca1bfff640 1 -- 172.21.15.40:0/3960035478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14017070 con 0x7fca240d1c40 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.498+0000 7fca1bfff640 1 -- 172.21.15.40:0/3960035478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1400c9a0 con 0x7fca240d1c40 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.498+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca240b66f0 con 0x7fca240d1c40 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.498+0000 7fca1bfff640 1 -- 172.21.15.40:0/3960035478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1400cd90 con 0x7fca240d1c40 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.498+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca241f93a0 con 0x7fca240d1c40 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.499+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca240763f0 con 0x7fca240d1c40 2024-04-26T21:42:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.499+0000 7fca1bfff640 1 -- 172.21.15.40:0/3960035478 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca1401a070 con 0x7fca240d1c40 2024-04-26T21:42:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.499+0000 7fca1bfff640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0406c8f0 0x7fca0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.500+0000 7fca1bfff640 1 -- 172.21.15.40:0/3960035478 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fca1408bcb0 con 0x7fca240d1c40 2024-04-26T21:42:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.500+0000 7fca1a7fc640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0406c8f0 0x7fca0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.500+0000 7fca1a7fc640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0406c8f0 0x7fca0406edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fca0c034000 tx=0x7fca0c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.504+0000 7fca1bfff640 1 -- 172.21.15.40:0/3960035478 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca240763f0 con 0x7fca240d1c40 2024-04-26T21:42:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.690+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca240694f0 con 0x7fca240d1c40 2024-04-26T21:42:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.691+0000 7fca1bfff640 1 -- 172.21.15.40:0/3960035478 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fca2415d980 con 0x7fca240d1c40 2024-04-26T21:42:39.692 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0406c8f0 msgr2=0x7fca0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0406c8f0 0x7fca0406edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fca0c034000 tx=0x7fca0c02e000 comp rx=0 tx=0).stop 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 msgr2=0x7fca2415d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca2415d440 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fca1400e9b0 tx=0x7fca1400ee70 comp rx=0 tx=0).stop 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 shutdown_connections 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fca0406c8f0 0x7fca0406edb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24162a20 0x7fca2406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca240d2ed0 0x7fca240b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 --2- 172.21.15.40:0/3960035478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca240d1c40 0x7fca2415d440 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 >> 172.21.15.40:0/3960035478 conn(0x7fca240d4fd0 msgr2=0x7fca24160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:39.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 shutdown_connections 2024-04-26T21:42:39.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.693+0000 7fca296e1640 1 -- 172.21.15.40:0/3960035478 wait complete. 2024-04-26T21:42:39.704 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:42:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.805+0000 7f331114e640 1 Processor -- start 2024-04-26T21:42:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.805+0000 7f331114e640 1 -- start start 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.806+0000 7f331114e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f330c0b3860 0x7f330c16a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.806+0000 7f331114e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f330c0d0ba0 con 0x7f330c15f3a0 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.806+0000 7f331114e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f330c0d2590 con 0x7f330c0d06f0 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.806+0000 7f331114e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f330c0d0e20 con 0x7f330c0b3860 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.806+0000 7f330ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f330c15f3a0 0x7f330c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42986/0 (socket says 172.21.15.40:42986) 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.806+0000 7f330ad9d640 1 -- 172.21.15.40:0/260878449 learned_addr learned my addr 172.21.15.40:0/260878449 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.807+0000 7f330b59e640 1 --2- 172.21.15.40:0/260878449 >> v2:172.21.15.184:3300/0 conn(0x7f330c0b3860 0x7f330c16a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.807+0000 7f3309d9b640 1 -- 172.21.15.40:0/260878449 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 411640457 0 0) 0x7f3304001110 con 0x7f330c15f3a0 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.807+0000 7f3309d9b640 1 -- 172.21.15.40:0/260878449 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f8003520 con 0x7f330c15f3a0 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.807+0000 7f3309d9b640 1 -- 172.21.15.40:0/260878449 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2135848975 0 0) 0x7f32f8003520 con 0x7f330c15f3a0 2024-04-26T21:42:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.807+0000 7f330b59e640 1 -- 172.21.15.40:0/260878449 >> v1:172.21.15.184:6789/0 conn(0x7f330c0d06f0 legacy=0x7f330c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.807+0000 7f330b59e640 1 -- 172.21.15.40:0/260878449 >> v1:172.21.15.40:6789/0 conn(0x7f330c15f3a0 legacy=0x7f330c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.807+0000 7f330b59e640 1 -- 172.21.15.40:0/260878449 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c0b1d70 con 0x7f330c0b3860 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f330b59e640 1 --2- 172.21.15.40:0/260878449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0b3860 0x7f330c16a000 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009d90 tx=0x7f32fc02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a9543b350f600 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f3309d9b640 1 -- 172.21.15.40:0/260878449 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304001840 con 0x7f330c0b3860 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f3309d9b640 1 -- 172.21.15.40:0/260878449 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32fc007540 con 0x7f330c0b3860 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f3309d9b640 1 -- 172.21.15.40:0/260878449 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0078f0 con 0x7f330c0b3860 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f331114e640 1 -- 172.21.15.40:0/260878449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0b3860 msgr2=0x7f330c16a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f331114e640 1 --2- 172.21.15.40:0/260878449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0b3860 0x7f330c16a000 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009d90 tx=0x7f32fc02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f331114e640 1 -- 172.21.15.40:0/260878449 shutdown_connections 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f331114e640 1 --2- 172.21.15.40:0/260878449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0b3860 0x7f330c16a000 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.808+0000 7f331114e640 1 -- 172.21.15.40:0/260878449 >> 172.21.15.40:0/260878449 conn(0x7f330c0d4f90 msgr2=0x7f330c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.809+0000 7f331114e640 1 -- 172.21.15.40:0/260878449 shutdown_connections 2024-04-26T21:42:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.809+0000 7f331114e640 1 -- 172.21.15.40:0/260878449 wait complete. 2024-04-26T21:42:39.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.810+0000 7f331114e640 1 Processor -- start 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.810+0000 7f331114e640 1 -- start start 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f331114e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f330c0b3860 0x7f330c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f331114e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0d06f0 0x7f330c0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f331114e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c15f3a0 0x7f330c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f331114e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f330c15f780 con 0x7f330c0b3860 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f331114e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0b2330 con 0x7f330c0d06f0 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f331114e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f330c15f900 con 0x7f330c15f3a0 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f330c0b3860 0x7f330c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f330c0b3860 0x7f330c150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45728/0 (socket says 172.21.15.40:45728) 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 -- 172.21.15.40:0/4245046060 learned_addr learned my addr 172.21.15.40:0/4245046060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330a59c640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0d06f0 0x7f330c0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 -- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c15f3a0 msgr2=0x7f330c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330b59e640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c15f3a0 0x7f330c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c15f3a0 0x7f330c0caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 -- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0d06f0 msgr2=0x7f330c0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0d06f0 0x7f330c0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 -- 172.21.15.40:0/4245046060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c1f8940 con 0x7f330c0b3860 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330a59c640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0d06f0 0x7f330c0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f330ad9d640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f330c0b3860 0x7f330c150530 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f330400c510 tx=0x7f330400c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f33037fe640 1 -- 172.21.15.40:0/4245046060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304017070 con 0x7f330c0b3860 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f33037fe640 1 -- 172.21.15.40:0/4245046060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3304005ce0 con 0x7f330c0b3860 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.811+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c1f8c20 con 0x7f330c0b3860 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.812+0000 7f33037fe640 1 -- 172.21.15.40:0/4245046060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33040072b0 con 0x7f330c0b3860 2024-04-26T21:42:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.812+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c150a70 con 0x7f330c0b3860 2024-04-26T21:42:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.812+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c0694f0 con 0x7f330c0b3860 2024-04-26T21:42:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.813+0000 7f33037fe640 1 -- 172.21.15.40:0/4245046060 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f330401a070 con 0x7f330c0b3860 2024-04-26T21:42:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.813+0000 7f33037fe640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f32ec06c8f0 0x7f32ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.813+0000 7f33037fe640 1 -- 172.21.15.40:0/4245046060 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f330408b210 con 0x7f330c0b3860 2024-04-26T21:42:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.814+0000 7f330a59c640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f32ec06c8f0 0x7f32ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.814+0000 7f330a59c640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f32ec06c8f0 0x7f32ec06edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f32f4000f60 tx=0x7f32f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:39.818+0000 7f33037fe640 1 -- 172.21.15.40:0/4245046060 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f330c0d0ba0 con 0x7f330c0b3860 2024-04-26T21:42:40.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.015+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f330c0b6860 con 0x7f330c0b3860 2024-04-26T21:42:40.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.015+0000 7f33037fe640 1 -- 172.21.15.40:0/4245046060 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v95) ==== 80+0+122 (secure 0 0 0) 0x7f3304059470 con 0x7f330c0b3860 2024-04-26T21:42:40.016 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:42:40.016 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:42:40.017 INFO:teuthology.orchestra.run.smithi040.stderr:listed 2 entries 2024-04-26T21:42:40.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f32ec06c8f0 msgr2=0x7f32ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f32ec06c8f0 0x7f32ec06edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f32f4000f60 tx=0x7f32f4009040 comp rx=0 tx=0).stop 2024-04-26T21:42:40.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f330c0b3860 msgr2=0x7f330c150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f330c0b3860 0x7f330c150530 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f330400c510 tx=0x7f330400c9d0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 shutdown_connections 2024-04-26T21:42:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f32ec06c8f0 0x7f32ec06edb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c15f3a0 0x7f330c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f330c0d06f0 0x7f330c0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 --2- 172.21.15.40:0/4245046060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f330c0b3860 0x7f330c150530 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 >> 172.21.15.40:0/4245046060 conn(0x7f330c0d4f90 msgr2=0x7f330c0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 shutdown_connections 2024-04-26T21:42:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.017+0000 7f331114e640 1 -- 172.21.15.40:0/4245046060 wait complete. 2024-04-26T21:42:40.029 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:42:40.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:40.029 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:42:40.055 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:42:40.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:40.055 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:42:40.116 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:42:40.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.259+0000 7f9260a45640 1 Processor -- start 2024-04-26T21:42:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.259+0000 7f9260a45640 1 -- start start 2024-04-26T21:42:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.259+0000 7f9260a45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.259+0000 7f9260a45640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c0d10d0 con 0x7f925c0d1b30 2024-04-26T21:42:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.259+0000 7f9260a45640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c0d1370 con 0x7f925c0b2340 2024-04-26T21:42:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.259+0000 7f9260a45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f925c0d0750 con 0x7f925c162810 2024-04-26T21:42:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f925ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f925c0d1b30 0x7f925c070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42992/0 (socket says 172.21.15.40:42992) 2024-04-26T21:42:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f925ad9d640 1 -- 172.21.15.40:0/35838043 learned_addr learned my addr 172.21.15.40:0/35838043 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f925a59c640 1 --2- 172.21.15.40:0/35838043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f9259d9b640 1 -- 172.21.15.40:0/35838043 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72326185 0 0) 0x7f925c0d10d0 con 0x7f925c0d1b30 2024-04-26T21:42:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f9259d9b640 1 -- 172.21.15.40:0/35838043 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9248003520 con 0x7f925c0d1b30 2024-04-26T21:42:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f9259d9b640 1 -- 172.21.15.40:0/35838043 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765281 0 0) 0x7f925c0d1370 con 0x7f925c0b2340 2024-04-26T21:42:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f9259d9b640 1 -- 172.21.15.40:0/35838043 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f925c0d10d0 con 0x7f925c0b2340 2024-04-26T21:42:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f925a59c640 1 -- 172.21.15.40:0/35838043 >> v1:172.21.15.184:6789/0 conn(0x7f925c0b2340 legacy=0x7f925c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f9259d9b640 1 -- 172.21.15.40:0/35838043 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3939165048 0 0) 0x7f9248003520 con 0x7f925c0d1b30 2024-04-26T21:42:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f925a59c640 1 -- 172.21.15.40:0/35838043 >> v1:172.21.15.40:6789/0 conn(0x7f925c0d1b30 legacy=0x7f925c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.260+0000 7f925a59c640 1 -- 172.21.15.40:0/35838043 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c0d01b0 con 0x7f925c162810 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.261+0000 7f925a59c640 1 --2- 172.21.15.40:0/35838043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c070350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f924c0097d0 tx=0x7f924c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=820f217b5683ef81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.261+0000 7f9259d9b640 1 -- 172.21.15.40:0/35838043 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c037070 con 0x7f925c162810 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.261+0000 7f9259d9b640 1 -- 172.21.15.40:0/35838043 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924c02ac30 con 0x7f925c162810 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.261+0000 7f9259d9b640 1 -- 172.21.15.40:0/35838043 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540019f0 con 0x7f925c162810 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.262+0000 7f9260a45640 1 -- 172.21.15.40:0/35838043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 msgr2=0x7f925c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.262+0000 7f9260a45640 1 --2- 172.21.15.40:0/35838043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c070350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f924c0097d0 tx=0x7f924c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.262+0000 7f9260a45640 1 -- 172.21.15.40:0/35838043 shutdown_connections 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.262+0000 7f9260a45640 1 --2- 172.21.15.40:0/35838043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c070350 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.262+0000 7f9260a45640 1 -- 172.21.15.40:0/35838043 >> 172.21.15.40:0/35838043 conn(0x7f925c0d2da0 msgr2=0x7f925c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.262+0000 7f9260a45640 1 -- 172.21.15.40:0/35838043 shutdown_connections 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.262+0000 7f9260a45640 1 -- 172.21.15.40:0/35838043 wait complete. 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.263+0000 7f9260a45640 1 Processor -- start 2024-04-26T21:42:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.263+0000 7f9260a45640 1 -- start start 2024-04-26T21:42:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9260a45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c0b2340 0x7f925c151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9260a45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c0d1b30 0x7f925c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9260a45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c15d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9260a45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f925c070890 con 0x7f925c0d1b30 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9260a45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0d16a0 con 0x7f925c0b2340 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9260a45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f925c0cdb00 con 0x7f925c162810 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c0d1b30 0x7f925c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c0d1b30 0x7f925c15d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45734/0 (socket says 172.21.15.40:45734) 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 -- 172.21.15.40:0/3256967798 learned_addr learned my addr 172.21.15.40:0/3256967798 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f925a59c640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c0b2340 0x7f925c151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f925ad9d640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 -- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 msgr2=0x7f925c15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c15d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 -- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c0b2340 msgr2=0x7f925c151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c0b2340 0x7f925c151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 -- 172.21.15.40:0/3256967798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c15e7d0 con 0x7f925c0d1b30 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f925ad9d640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.264+0000 7f9253fff640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c0d1b30 0x7f925c15d0c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f924400c490 tx=0x7f924400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.265+0000 7f9252ffd640 1 -- 172.21.15.40:0/3256967798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92440052d0 con 0x7f925c0d1b30 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.265+0000 7f9252ffd640 1 -- 172.21.15.40:0/3256967798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9244005470 con 0x7f925c0d1b30 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.265+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c15eab0 con 0x7f925c0d1b30 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.265+0000 7f9252ffd640 1 -- 172.21.15.40:0/3256967798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244012870 con 0x7f925c0d1b30 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.265+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c15f070 con 0x7f925c0d1b30 2024-04-26T21:42:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.266+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c0694f0 con 0x7f925c0d1b30 2024-04-26T21:42:40.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.266+0000 7f9252ffd640 1 -- 172.21.15.40:0/3256967798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9244012a10 con 0x7f925c0d1b30 2024-04-26T21:42:40.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.267+0000 7f9252ffd640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f923c06c8f0 0x7f923c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.267+0000 7f9252ffd640 1 -- 172.21.15.40:0/3256967798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f924408ba50 con 0x7f925c0d1b30 2024-04-26T21:42:40.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.267+0000 7f925a59c640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f923c06c8f0 0x7f923c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.267+0000 7f925a59c640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f923c06c8f0 0x7f923c06edb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f924c034000 tx=0x7f924c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:40.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.271+0000 7f9252ffd640 1 -- 172.21.15.40:0/3256967798 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9244059c00 con 0x7f925c0d1b30 2024-04-26T21:42:40.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.493+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f925c0b0110 con 0x7f925c0d1b30 2024-04-26T21:42:40.494 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:40.494+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:42:40.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.495+0000 7f9252ffd640 1 -- 172.21.15.40:0/3256967798 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) ==== 161+0+0 (secure 0 0 0) 0x7f9244059500 con 0x7f925c0d1b30 2024-04-26T21:42:40.496 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:42:40.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f923c06c8f0 msgr2=0x7f923c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f923c06c8f0 0x7f923c06edb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f924c034000 tx=0x7f924c02e000 comp rx=0 tx=0).stop 2024-04-26T21:42:40.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c0d1b30 msgr2=0x7f925c15d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c0d1b30 0x7f925c15d0c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f924400c490 tx=0x7f924400c950 comp rx=0 tx=0).stop 2024-04-26T21:42:40.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 shutdown_connections 2024-04-26T21:42:40.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f923c06c8f0 0x7f923c06edb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c162810 0x7f925c15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c0d1b30 0x7f925c15d0c0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 --2- 172.21.15.40:0/3256967798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c0b2340 0x7f925c151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 >> 172.21.15.40:0/3256967798 conn(0x7f925c0d2da0 msgr2=0x7f925c0b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:40.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 shutdown_connections 2024-04-26T21:42:40.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.498+0000 7f9260a45640 1 -- 172.21.15.40:0/3256967798 wait complete. 2024-04-26T21:42:40.500 INFO:tasks.ceph.mds.a.smithi040.stderr: -655> 2024-04-26T21:42:35.484+0000 7f26f1157640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:42:40.500 INFO:tasks.ceph.mds.a.smithi040.stderr: -189> 2024-04-26T21:42:38.291+0000 7f26ed14f640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-04-26T21:42:40.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:42:40.525 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:42:40.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.613+0000 7f04bab9d640 1 Processor -- start 2024-04-26T21:42:40.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.613+0000 7f04bab9d640 1 -- start start 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04bab9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4071e60 0x7f04b415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04bab9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04b40cc5b0 0x7f04b4166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04bab9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b415e4d0 con 0x7f04b4072240 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04bab9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f04b40701f0 con 0x7f04b40cc5b0 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04bab9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04b406e240 con 0x7f04b4071e60 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04b913a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04b40cc5b0 0x7f04b4166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04b913a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04b40cc5b0 0x7f04b4166b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45756/0 (socket says 172.21.15.40:45756) 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04b913a640 1 -- 172.21.15.40:0/3681417755 learned_addr learned my addr 172.21.15.40:0/3681417755 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04b8939640 1 --2- 172.21.15.40:0/3681417755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4071e60 0x7f04b415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04b913a640 1 -- 172.21.15.40:0/3681417755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4071e60 msgr2=0x7f04b415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04b913a640 1 --2- 172.21.15.40:0/3681417755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4071e60 0x7f04b415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04b913a640 1 -- 172.21.15.40:0/3681417755 >> v1:172.21.15.184:6789/0 conn(0x7f04b4072240 legacy=0x7f04b40726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.614+0000 7f04b913a640 1 -- 172.21.15.40:0/3681417755 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b406fc30 con 0x7f04b40cc5b0 2024-04-26T21:42:40.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.615+0000 7f04b913a640 1 --2- 172.21.15.40:0/3681417755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b40cc5b0 0x7f04b4166b30 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f04b4070370 tx=0x7f04ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53299976d38f9d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:40.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.615+0000 7f04b37fe640 1 -- 172.21.15.40:0/3681417755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac036070 con 0x7f04b40cc5b0 2024-04-26T21:42:40.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.615+0000 7f04b37fe640 1 -- 172.21.15.40:0/3681417755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac031600 con 0x7f04b40cc5b0 2024-04-26T21:42:40.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.615+0000 7f04b37fe640 1 -- 172.21.15.40:0/3681417755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0319d0 con 0x7f04b40cc5b0 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.616+0000 7f04bab9d640 1 -- 172.21.15.40:0/3681417755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b40cc5b0 msgr2=0x7f04b4166b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.616+0000 7f04bab9d640 1 --2- 172.21.15.40:0/3681417755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b40cc5b0 0x7f04b4166b30 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f04b4070370 tx=0x7f04ac02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.616+0000 7f04bab9d640 1 -- 172.21.15.40:0/3681417755 shutdown_connections 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.616+0000 7f04bab9d640 1 --2- 172.21.15.40:0/3681417755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b40cc5b0 0x7f04b4166b30 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.616+0000 7f04bab9d640 1 --2- 172.21.15.40:0/3681417755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4071e60 0x7f04b415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.616+0000 7f04bab9d640 1 -- 172.21.15.40:0/3681417755 >> 172.21.15.40:0/3681417755 conn(0x7f04b40d0be0 msgr2=0x7f04b40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.616+0000 7f04bab9d640 1 -- 172.21.15.40:0/3681417755 shutdown_connections 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.616+0000 7f04bab9d640 1 -- 172.21.15.40:0/3681417755 wait complete. 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.617+0000 7f04bab9d640 1 Processor -- start 2024-04-26T21:42:40.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.617+0000 7f04bab9d640 1 -- start start 2024-04-26T21:42:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04bab9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b4071e60 0x7f04b40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04bab9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4072240 0x7f04b40b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04bab9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04b40cc5b0 0x7f04b40b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04bab9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04b406f8e0 con 0x7f04b4071e60 2024-04-26T21:42:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04bab9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4073fb0 con 0x7f04b40cc5b0 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04bab9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04b406ddf0 con 0x7f04b4072240 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b4071e60 0x7f04b40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b4071e60 0x7f04b40cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45760/0 (socket says 172.21.15.40:45760) 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 -- 172.21.15.40:0/4034841758 learned_addr learned my addr 172.21.15.40:0/4034841758 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b913a640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04b40cc5b0 0x7f04b40b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b3fff640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4072240 0x7f04b40b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 -- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4072240 msgr2=0x7f04b40b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4072240 0x7f04b40b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 -- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04b40cc5b0 msgr2=0x7f04b40b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04b40cc5b0 0x7f04b40b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 -- 172.21.15.40:0/4034841758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b40b2480 con 0x7f04b4071e60 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b3fff640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4072240 0x7f04b40b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.618+0000 7f04b8939640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b4071e60 0x7f04b40cd860 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d620 tx=0x7f04a800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.619+0000 7f04b17fa640 1 -- 172.21.15.40:0/4034841758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a800b620 con 0x7f04b4071e60 2024-04-26T21:42:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.619+0000 7f04b17fa640 1 -- 172.21.15.40:0/4034841758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8019070 con 0x7f04b4071e60 2024-04-26T21:42:40.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.619+0000 7f04b17fa640 1 -- 172.21.15.40:0/4034841758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a800ba00 con 0x7f04b4071e60 2024-04-26T21:42:40.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.619+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b40b6950 con 0x7f04b4071e60 2024-04-26T21:42:40.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.619+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b40b44e0 con 0x7f04b4071e60 2024-04-26T21:42:40.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.620+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b40694f0 con 0x7f04b4071e60 2024-04-26T21:42:40.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.620+0000 7f04b17fa640 1 -- 172.21.15.40:0/4034841758 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04a800bba0 con 0x7f04b4071e60 2024-04-26T21:42:40.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.621+0000 7f04b17fa640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:40.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.621+0000 7f04b17fa640 1 -- 172.21.15.40:0/4034841758 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6273+0+0 (secure 0 0 0) 0x7f04a808c090 con 0x7f04b4071e60 2024-04-26T21:42:40.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.621+0000 7f04b3fff640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:40.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.621+0000 7f04b3fff640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f049806c8f0 0x7f049806edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f04b415f860 tx=0x7f04a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:40.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.625+0000 7f04b17fa640 1 -- 172.21.15.40:0/4034841758 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04b40b2480 con 0x7f04b4071e60 2024-04-26T21:42:40.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:40.887+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f04b40b4be0 con 0x7f04b4071e60 2024-04-26T21:42:41.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.497+0000 7f04b17fa640 1 -- 172.21.15.40:0/4034841758 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v98) ==== 107+0+0 (secure 0 0 0) 0x7f04a805a310 con 0x7f04b4071e60 2024-04-26T21:42:41.498 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:42:41.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f049806c8f0 msgr2=0x7f049806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f049806c8f0 0x7f049806edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f04b415f860 tx=0x7f04a0009040 comp rx=0 tx=0).stop 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b4071e60 msgr2=0x7f04b40cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b4071e60 0x7f04b40cd860 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d620 tx=0x7f04a800dae0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 shutdown_connections 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04b40cc5b0 0x7f04b40b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04b4072240 0x7f04b40b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 --2- 172.21.15.40:0/4034841758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04b4071e60 0x7f04b40cd860 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.499+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 >> 172.21.15.40:0/4034841758 conn(0x7f04b40d0be0 msgr2=0x7f04b415e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.500+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 shutdown_connections 2024-04-26T21:42:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.500+0000 7f04bab9d640 1 -- 172.21.15.40:0/4034841758 wait complete. 2024-04-26T21:42:41.513 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:42:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.598+0000 7fea3d0f2f40 1 Processor -- start 2024-04-26T21:42:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.598+0000 7fea3d0f2f40 1 -- start start 2024-04-26T21:42:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3d0f2f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1130 0x5566e1eae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3d0f2f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1510 0x5566e1ed8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3d0f2f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e1ed9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3d0f2f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5566e1f84b20 con 0x5566e1f68d60 2024-04-26T21:42:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3d0f2f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5566e1e56e40 con 0x5566e1eb1130 2024-04-26T21:42:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3d0f2f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5566e1edc620 con 0x5566e1eb1510 2024-04-26T21:42:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3b3ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e1ed9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3b3ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e1ed9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45786/0 (socket says 172.21.15.40:45786) 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3b3ed640 1 -- 172.21.15.40:0/1647235919 learned_addr learned my addr 172.21.15.40:0/1647235919 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3abec640 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1130 0x5566e1eae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.599+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1510 0x5566e1ed8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea3b3ed640 1 -- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1510 msgr2=0x5566e1ed8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea3b3ed640 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1510 0x5566e1ed8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea3b3ed640 1 -- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1130 msgr2=0x5566e1eae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea3b3ed640 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1130 0x5566e1eae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea3b3ed640 1 -- 172.21.15.40:0/1647235919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e1f86530 con 0x5566e1f68d60 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1510 0x5566e1ed8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea3b3ed640 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e1ed9500 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fea3400a770 tx=0x7fea3400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28130db755bda5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea393e9640 1 -- 172.21.15.40:0/1647235919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34019070 con 0x5566e1f68d60 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea393e9640 1 -- 172.21.15.40:0/1647235919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea34007600 con 0x5566e1f68d60 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.600+0000 7fea393e9640 1 -- 172.21.15.40:0/1647235919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34007a10 con 0x5566e1f68d60 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/1647235919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 msgr2=0x5566e1ed9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e1ed9500 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fea3400a770 tx=0x7fea3400a860 comp rx=0 tx=0).stop 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/1647235919 shutdown_connections 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e1ed9500 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1510 0x5566e1ed8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/1647235919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1130 0x5566e1eae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/1647235919 >> 172.21.15.40:0/1647235919 conn(0x5566e1eadab0 msgr2=0x5566e1eaee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/1647235919 shutdown_connections 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/1647235919 wait complete. 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 Processor -- start 2024-04-26T21:42:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.601+0000 7fea3d0f2f40 1 -- start start 2024-04-26T21:42:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3d0f2f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1130 0x5566e1f65e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3d0f2f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1510 0x5566e2020450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3d0f2f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e2010100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3d0f2f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5566e1f86b80 con 0x5566e1f68d60 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3d0f2f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5566e1edae30 con 0x5566e1eb1510 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3d0f2f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5566e1edab50 con 0x5566e1eb1130 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3b3ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e2010100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3b3ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e2010100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45792/0 (socket says 172.21.15.40:45792) 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 learned_addr learned my addr 172.21.15.40:0/2137966240 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1510 0x5566e2020450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.602+0000 7fea3abec640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1130 0x5566e1f65e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1130 msgr2=0x5566e1f65e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3b3ed640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1130 0x5566e1f65e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1510 msgr2=0x5566e2020450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3b3ed640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1510 0x5566e2020450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e2020ef0 con 0x5566e1f68d60 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3abec640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5566e1eb1130 0x5566e1f65e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566e1eb1510 0x5566e2020450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3b3ed640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e2010100 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fea34013000 tx=0x7fea34003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea33fff640 1 -- 172.21.15.40:0/2137966240 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34023020 con 0x5566e1f68d60 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea33fff640 1 -- 172.21.15.40:0/2137966240 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3401f670 con 0x5566e1f68d60 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5566e20211d0 con 0x5566e1f68d60 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea33fff640 1 -- 172.21.15.40:0/2137966240 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34019040 con 0x5566e1f68d60 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.603+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e1f80850 con 0x5566e1f68d60 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.604+0000 7fea33fff640 1 -- 172.21.15.40:0/2137966240 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea34026020 con 0x5566e1f68d60 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.605+0000 7fea33fff640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea1c06b300 0x7fea1c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.605+0000 7fea33fff640 1 -- 172.21.15.40:0/2137966240 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6273+0+0 (secure 0 0 0) 0x7fea3408bee0 con 0x5566e1f68d60 2024-04-26T21:42:41.609 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.605+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5566e1eef600 0x5566e1f81dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.605+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e1f82300 con 0x5566e1eef600 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.605+0000 7fea3abec640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea1c06b300 0x7fea1c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.605+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5566e1eef600 0x5566e1f81dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.606+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5566e1eef600 0x5566e1f81dc0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.606+0000 7fea3abec640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea1c06b300 0x7fea1c06d7c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fea2c058000 tx=0x7fea2c052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.607+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v96'43 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.607+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e1f826d0 con 0x5566e1eef600 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.608+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v96'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.608+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e20295e0 con 0x5566e1eef600 2024-04-26T21:42:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.611+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v96'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.612 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:42:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.611+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202a990 con 0x5566e1eef600 2024-04-26T21:42:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.612+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v96'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.612+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202ad30 con 0x5566e1eef600 2024-04-26T21:42:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.613+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v96'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.613+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029980 con 0x5566e1eef600 2024-04-26T21:42:41.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.614+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v96'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.614+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.615+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v96'49 uv22 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.615+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.616+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v96'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.617+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.617+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v96'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.618 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:42:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.617+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.618+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v96'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.618+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.619+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v96'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.620+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.620+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v96'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.621+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.621+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v96'55 uv55 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.622 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:42:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.621+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.622+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [setxattr (30)] v96'56 uv56 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.622+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.623+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [setxattr (62)] v96'57 uv57 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.624+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.624+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [omap-set-header] v96'58 uv58 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.624+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.625+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v96'59 uv59 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.625+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.626+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 600.00000000 [create] v96'60 uv35 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.626+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.628+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 600.00000000 [delete] v96'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.628+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.629+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v96'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.629 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:42:41.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.629+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.629+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v96'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.630+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.631+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v96'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.631+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.633+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v96'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.633+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.636+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 603.00000000 [create] v96'66 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.636+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.638+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 603.00000000 [delete] v96'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.638+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.641+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v96'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.642 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:42:41.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.641+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.643+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v96'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.643+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.646+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v96'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.646+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2029d20 con 0x5566e1eef600 2024-04-26T21:42:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.648+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v96'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e1eef600 2024-04-26T21:42:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.648+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5566e202b0d0 0x5566e202d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.648+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202da00 con 0x5566e202b0d0 2024-04-26T21:42:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.648+0000 7fea3abec640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5566e202b0d0 0x5566e202d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.648+0000 7fea3abec640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5566e202b0d0 0x5566e202d4c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.650+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(30 1.00000000.inode [create] v96'45 uv24 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.650+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e202dda0 con 0x5566e202b0d0 2024-04-26T21:42:41.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.651+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(31 1.00000000.inode [delete] v96'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.651+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202e140 con 0x5566e202b0d0 2024-04-26T21:42:41.653 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:42:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.652+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(32 1.00000000.inode [create] v96'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.652+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202e4e0 con 0x5566e202b0d0 2024-04-26T21:42:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.653+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v96'48 uv48 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.653+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202e880 con 0x5566e202b0d0 2024-04-26T21:42:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.655+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v96'49 uv49 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.655+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202ec20 con 0x5566e202b0d0 2024-04-26T21:42:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.656+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(35 200.00000000 [create] v96'50 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.656+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e202efc0 con 0x5566e202b0d0 2024-04-26T21:42:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.657+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(36 200.00000000 [delete] v96'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.657+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.659 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:42:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.658+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(37 200.00000000 [create] v96'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.658+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.660+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v96'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.660+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.661+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v96'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.661+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.662+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(40 200.00000001 [create] v96'55 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.662+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.664+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(41 200.00000001 [delete] v96'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.664+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.665 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:42:41.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.665+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(42 200.00000001 [create] v96'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.665+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2033a20 con 0x5566e202b0d0 2024-04-26T21:42:41.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.666+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(43 200.00000001 [write 0~18075] v96'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.666+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2033a20 con 0x5566e202b0d0 2024-04-26T21:42:41.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.667+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v96'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.668+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.669+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(45 606.00000000 [create] v96'60 uv40 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.669+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.670+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(46 606.00000000 [delete] v96'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.670+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.671+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(47 606.00000000 [create] v96'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.672 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:42:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.671+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2030370 con 0x5566e202b0d0 2024-04-26T21:42:41.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.672+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v96'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.672+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2030370 con 0x5566e202b0d0 2024-04-26T21:42:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.673+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v96'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.673+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e202f360 con 0x5566e202b0d0 2024-04-26T21:42:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.674+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v96'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e202b0d0 2024-04-26T21:42:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.674+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5566e202fc80 0x5566e2030080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.674+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2032790 con 0x5566e202fc80 2024-04-26T21:42:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.675+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5566e202fc80 0x5566e2030080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.675+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5566e202fc80 0x5566e2030080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.676+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(51 607.00000000 [create] v96'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.676+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e2032790 con 0x5566e202fc80 2024-04-26T21:42:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.678+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(52 607.00000000 [delete] v96'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.678+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2032790 con 0x5566e202fc80 2024-04-26T21:42:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.679+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(53 607.00000000 [create] v96'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.679 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:42:41.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.679+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e20337a0 con 0x5566e202fc80 2024-04-26T21:42:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.680+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v96'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.680+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e20337a0 con 0x5566e202fc80 2024-04-26T21:42:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.681+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v96'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.681+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2032790 con 0x5566e202fc80 2024-04-26T21:42:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.683+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.683+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5566e2032e10 0x5566e2033250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.683+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2035bf0 con 0x5566e2032e10 2024-04-26T21:42:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.683+0000 7fea3b3ed640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5566e2032e10 0x5566e2033250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.684+0000 7fea3b3ed640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5566e2032e10 0x5566e2033250 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.685+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v96'10 uv9 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.685+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e2035f90 con 0x5566e2032e10 2024-04-26T21:42:41.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.686+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v96'11 uv11 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.686+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2036330 con 0x5566e2032e10 2024-04-26T21:42:41.688 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:42:41.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.687+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v96'12 uv12 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.688+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2036bd0 con 0x5566e2032e10 2024-04-26T21:42:41.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.689+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v96'13 uv13 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.689+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5566e2037480 0x5566e2039870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.689+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e2039db0 con 0x5566e2037480 2024-04-26T21:42:41.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.689+0000 7fea3abec640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5566e2037480 0x5566e2039870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.690+0000 7fea3abec640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5566e2037480 0x5566e2039870 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.691+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(61 608.00000000 [create] v96'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.691+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e2039e70 con 0x5566e2037480 2024-04-26T21:42:41.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.692+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(62 608.00000000 [delete] v96'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.692+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a260 con 0x5566e2037480 2024-04-26T21:42:41.694 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:42:41.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.694+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(63 608.00000000 [create] v96'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.694+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203b270 con 0x5566e2037480 2024-04-26T21:42:41.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.695+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v96'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.695+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203b270 con 0x5566e2037480 2024-04-26T21:42:41.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.696+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v96'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.696+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a260 con 0x5566e2037480 2024-04-26T21:42:41.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.697+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.697+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a260 con 0x5566e2032e10 2024-04-26T21:42:41.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.699+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(67 604.00000000 [create] v96'15 uv9 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.699+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.700+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(68 604.00000000 [delete] v96'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.700+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.701+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(69 604.00000000 [create] v96'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.702 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:42:41.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.701+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203b610 con 0x5566e2032e10 2024-04-26T21:42:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.702+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v96'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.702+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203b610 con 0x5566e2032e10 2024-04-26T21:42:41.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.705+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v96'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.705+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.706+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v96'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.706+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.709+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(73 500.00000000 [create] v96'21 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.709+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.711+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(74 500.00000000 [delete] v96'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.711+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.713+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(75 500.00000000 [create] v96'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.714 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:42:41.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.713+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.716+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v96'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.716+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.718+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v96'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.718+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.719+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(78 mds_snaptable [create] v96'7 uv6 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.719+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.722+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(79 mds_snaptable [delete] v96'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.722+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.725+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(80 mds_snaptable [create] v96'9 uv9 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.726 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:42:41.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.725+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.726+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(81 mds_snaptable [write 0~70] v96'10 uv10 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.726+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2032e10 2024-04-26T21:42:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.729+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v96'11 uv11 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fea3401c040 con 0x5566e2032e10 2024-04-26T21:42:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.729+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.730+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(83 605.00000000 [create] v96'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.730+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.731+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(84 605.00000000 [delete] v96'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.731+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.732+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(85 605.00000000 [create] v96'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.733 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:42:41.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.732+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.733+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v96'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.733+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.734+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v96'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.735+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.736+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(88 605.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.736+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.738+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(89 mds0_inotable [create] v96'14 uv8 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.738+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.739+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(90 mds0_inotable [delete] v96'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.739+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a600 con 0x5566e2037480 2024-04-26T21:42:41.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.740+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(91 mds0_inotable [create] v96'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.741 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:42:41.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.740+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a9f0 con 0x5566e2037480 2024-04-26T21:42:41.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.741+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(92 mds0_inotable [write 0~34] v96'17 uv17 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.741+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a9f0 con 0x5566e2037480 2024-04-26T21:42:41.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.742+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v96'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.742+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a9f0 con 0x5566e2037480 2024-04-26T21:42:41.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.743+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(94 100.00000000 [create] v96'19 uv13 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.743+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203a9f0 con 0x5566e2037480 2024-04-26T21:42:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.744+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(95 100.00000000 [delete] v96'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.744+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a9f0 con 0x5566e2037480 2024-04-26T21:42:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.745+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(96 100.00000000 [create] v96'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.746 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:42:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.745+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203a9f0 con 0x5566e2037480 2024-04-26T21:42:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.746+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(97 100.00000000 [omap-set-header] v96'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.746+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203ae00 con 0x5566e2037480 2024-04-26T21:42:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.747+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v96'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.747+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5566e203ae00 0x5566e203da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.747+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e203ae00 2024-04-26T21:42:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.747+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5566e203ae00 0x5566e203da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.748+0000 7fea3a3eb640 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5566e203ae00 0x5566e203da30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.749+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(99 mds0_sessionmap [create] v96'9 uv7 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e203ae00 2024-04-26T21:42:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.749+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e203ae00 2024-04-26T21:42:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.750+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v96'10 uv10 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e203ae00 2024-04-26T21:42:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.750+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e203ae00 2024-04-26T21:42:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.750+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(101 mds0_sessionmap [create] v96'11 uv11 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e203ae00 2024-04-26T21:42:41.751 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:42:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.751+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e203ae00 2024-04-26T21:42:41.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.751+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v96'12 uv12 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e203ae00 2024-04-26T21:42:41.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.751+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e2037480 2024-04-26T21:42:41.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.752+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(103 609.00000000 [create] v96'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.753+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e2037480 2024-04-26T21:42:41.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.754+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(104 609.00000000 [delete] v96'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.754+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e2037480 2024-04-26T21:42:41.757 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:42:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.757+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(105 609.00000000 [create] v96'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.757+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203ef80 con 0x5566e2037480 2024-04-26T21:42:41.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.759+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v96'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.759+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203ef80 con 0x5566e2037480 2024-04-26T21:42:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.762+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v96'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.762+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e2037480 2024-04-26T21:42:41.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.763+0000 7fea3abec640 1 -- 172.21.15.40:0/2137966240 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(108 609.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2c069040 con 0x5566e2037480 2024-04-26T21:42:41.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.763+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.766+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v96'7 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.766+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.768+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v96'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.768+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.772 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:42:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.772+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v96'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.772+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.774+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v96'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.774+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.777+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v96'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.777+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.777+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v96'20 uv11 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.777+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.778+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v96'21 uv21 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.778+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.780 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:42:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.779+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v96'22 uv22 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.779+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.782+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v96'23 uv23 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.782+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.783+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v96'24 uv24 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.783+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.786+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v96'25 uv18 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.786+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.789+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v96'26 uv26 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.789+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.792 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:42:41.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.792+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v96'27 uv27 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.792+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203ef80 con 0x5566e202fc80 2024-04-26T21:42:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.794+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v96'28 uv28 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.794+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203ef80 con 0x5566e202fc80 2024-04-26T21:42:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.797+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v96'29 uv29 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.797+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203df70 con 0x5566e202fc80 2024-04-26T21:42:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.798+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v96'30 uv30 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.798+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5566e203e420 con 0x5566e202fc80 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3a3eb640 1 -- 172.21.15.40:0/2137966240 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v96'31 uv31 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fea2400c7c0 con 0x5566e202fc80 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5566e202b0d0 msgr2=0x5566e202d4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5566e202b0d0 0x5566e202d4c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5566e1eef600 msgr2=0x5566e1f81dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5566e1eef600 0x5566e1f81dc0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5566e2032e10 msgr2=0x5566e2033250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5566e2032e10 0x5566e2033250 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5566e203ae00 msgr2=0x5566e203da30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5566e203ae00 0x5566e203da30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5566e202fc80 msgr2=0x5566e2030080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5566e202fc80 0x5566e2030080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 reap_dead start 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5566e2037480 msgr2=0x5566e2039870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5566e2037480 0x5566e2039870 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 reap_dead start 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3b3ed640 1 -- 172.21.15.40:0/2137966240 reap_dead start 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea1c06b300 msgr2=0x7fea1c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea1c06b300 0x7fea1c06d7c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fea2c058000 tx=0x7fea2c052000 comp rx=0 tx=0).stop 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 msgr2=0x5566e2010100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e2010100 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fea34013000 tx=0x7fea34003020 comp rx=0 tx=0).stop 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 shutdown_connections 2024-04-26T21:42:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea1c06b300 0x7fea1c06d7c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 --2- 172.21.15.40:0/2137966240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566e1f68d60 0x5566e2010100 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.801+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 >> 172.21.15.40:0/2137966240 conn(0x5566e1eadab0 msgr2=0x5566e1eda2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.802+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 shutdown_connections 2024-04-26T21:42:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.802+0000 7fea3d0f2f40 1 -- 172.21.15.40:0/2137966240 wait complete. 2024-04-26T21:42:41.808 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 600.00000000 2024-04-26T21:42:41.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d8f20f40 1 Processor -- start 2024-04-26T21:42:41.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d8f20f40 1 -- start start 2024-04-26T21:42:41.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d8f20f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47478c3e0 0x55a4748613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d8f20f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47485f420 0x55a4748618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d8f20f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485fd40 0x55a4747d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d8f20f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a4747d1c60 con 0x55a47485fd40 2024-04-26T21:42:41.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d8f20f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a474731e40 con 0x55a47485f420 2024-04-26T21:42:41.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d8f20f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a4747d3ae0 con 0x55a47478c3e0 2024-04-26T21:42:41.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d67b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485fd40 0x55a4747d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d67b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485fd40 0x55a4747d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45808/0 (socket says 172.21.15.40:45808) 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.889+0000 7fd4d67b3640 1 -- 172.21.15.40:0/2844990921 learned_addr learned my addr 172.21.15.40:0/2844990921 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d57b1640 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47485f420 0x55a4748618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d5fb2640 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47478c3e0 0x55a4748613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d67b3640 1 -- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47478c3e0 msgr2=0x55a4748613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d67b3640 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47478c3e0 0x55a4748613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d67b3640 1 -- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47485f420 msgr2=0x55a4748618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d67b3640 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47485f420 0x55a4748618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d67b3640 1 -- 172.21.15.40:0/2844990921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4747b6270 con 0x55a47485fd40 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d5fb2640 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47478c3e0 0x55a4748613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d67b3640 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485fd40 0x55a4747d20f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800a510 tx=0x7fd4c800a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e58edaa478828 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4cffff640 1 -- 172.21.15.40:0/2844990921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800f7b0 con 0x55a47485fd40 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4cffff640 1 -- 172.21.15.40:0/2844990921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c8015070 con 0x55a47485fd40 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4cffff640 1 -- 172.21.15.40:0/2844990921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800fbb0 con 0x55a47485fd40 2024-04-26T21:42:41.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/2844990921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485fd40 msgr2=0x55a4747d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.890+0000 7fd4d8f20f40 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485fd40 0x55a4747d20f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800a510 tx=0x7fd4c800a9d0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/2844990921 shutdown_connections 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485fd40 0x55a4747d20f0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47485f420 0x55a4748618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 --2- 172.21.15.40:0/2844990921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47478c3e0 0x55a4748613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/2844990921 >> 172.21.15.40:0/2844990921 conn(0x55a474789330 msgr2=0x55a474789fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/2844990921 shutdown_connections 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/2844990921 wait complete. 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 Processor -- start 2024-04-26T21:42:41.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.891+0000 7fd4d8f20f40 1 -- start start 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d8f20f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47478c3e0 0x55a4747cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d8f20f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485f420 0x55a474861590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d8f20f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47485fd40 0x55a4747ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d8f20f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a4748ee440 con 0x55a47485f420 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d8f20f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a474860a70 con 0x55a47478c3e0 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d8f20f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a474860300 con 0x55a47485fd40 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485f420 0x55a474861590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485f420 0x55a474861590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45820/0 (socket says 172.21.15.40:45820) 2024-04-26T21:42:41.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 -- 172.21.15.40:0/4183504533 learned_addr learned my addr 172.21.15.40:0/4183504533 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d5fb2640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47478c3e0 0x55a4747cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d67b3640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47485fd40 0x55a4747ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 -- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47485fd40 msgr2=0x55a4747ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a47485fd40 0x55a4747ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 -- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47478c3e0 msgr2=0x55a4747cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a47478c3e0 0x55a4747cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 -- 172.21.15.40:0/4183504533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4747cb700 con 0x55a47485f420 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.892+0000 7fd4d57b1640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485f420 0x55a474861590 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0025da0 tx=0x7fd4c0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.893+0000 7fd4ceffd640 1 -- 172.21.15.40:0/4183504533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c006b070 con 0x55a47485f420 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.893+0000 7fd4ceffd640 1 -- 172.21.15.40:0/4183504533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0052910 con 0x55a47485f420 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.893+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4747cb9e0 con 0x55a47485f420 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.893+0000 7fd4ceffd640 1 -- 172.21.15.40:0/4183504533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0056070 con 0x55a47485f420 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.893+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4747ce660 con 0x55a47485f420 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.894+0000 7fd4ceffd640 1 -- 172.21.15.40:0/4183504533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4c006e070 con 0x55a47485f420 2024-04-26T21:42:41.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.894+0000 7fd4ceffd640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4b806b300 0x7fd4b806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.894+0000 7fd4ceffd640 1 -- 172.21.15.40:0/4183504533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6273+0+0 (secure 0 0 0) 0x7fd4c00ce7a0 con 0x55a47485f420 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.895+0000 7fd4d5fb2640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4b806b300 0x7fd4b806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.895+0000 7fd4d8f20f40 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a474861ad0 0x55a4748f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.895+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x55a474844eb0 con 0x55a474861ad0 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.895+0000 7fd4d67b3640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a474861ad0 0x55a4748f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.895+0000 7fd4d5fb2640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4b806b300 0x7fd4b806d7c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00077a0 tx=0x7fd4d0005b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.895+0000 7fd4d67b3640 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a474861ad0 0x55a4748f9610 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d67b3640 1 -- 172.21.15.40:0/4183504533 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 600.00000000 [delete] v96'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd4c800e070 con 0x55a474861ad0 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a474861ad0 msgr2=0x55a4748f9610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a474861ad0 0x55a4748f9610 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4b806b300 msgr2=0x7fd4b806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4b806b300 0x7fd4b806d7c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00077a0 tx=0x7fd4d0005b50 comp rx=0 tx=0).stop 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485f420 msgr2=0x55a474861590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 --2- 172.21.15.40:0/4183504533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47485f420 0x55a474861590 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0025da0 tx=0x7fd4c0002fb0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d67b3640 1 -- 172.21.15.40:0/4183504533 reap_dead start 2024-04-26T21:42:41.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 shutdown_connections 2024-04-26T21:42:41.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 >> 172.21.15.40:0/4183504533 conn(0x55a474789330 msgr2=0x55a4747b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:41.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 shutdown_connections 2024-04-26T21:42:41.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:41.896+0000 7fd4d8f20f40 1 -- 172.21.15.40:0/4183504533 wait complete. 2024-04-26T21:42:41.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:42:42.004 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:42:42.006 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:42:42.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.045+0000 7fb80b5e2640 1 Processor -- start 2024-04-26T21:42:42.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.045+0000 7fb80b5e2640 1 -- start start 2024-04-26T21:42:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80b5e2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb804072fb0 0x7fb8040b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80b5e2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb804073390 0x7fb80406f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80b5e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb80406fc80 0x7fb804070110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80b5e2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8040b5c00 con 0x7fb804072fb0 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80b5e2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb80406f2b0 con 0x7fb804073390 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80b5e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8040b5e30 con 0x7fb80406fc80 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80937e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb804072fb0 0x7fb8040b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80937e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb804072fb0 0x7fb8040b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45824/0 (socket says 172.21.15.40:45824) 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80937e640 1 -- 172.21.15.40:0/3865913539 learned_addr learned my addr 172.21.15.40:0/3865913539 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb809b7f640 1 --2- 172.21.15.40:0/3865913539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb80406fc80 0x7fb804070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb808b7d640 1 --2- 172.21.15.40:0/3865913539 >> v2:172.21.15.184:3300/0 conn(0x7fb804073390 0x7fb80406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80937e640 1 -- 172.21.15.40:0/3865913539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb80406fc80 msgr2=0x7fb804070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80937e640 1 --2- 172.21.15.40:0/3865913539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb80406fc80 0x7fb804070110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80937e640 1 -- 172.21.15.40:0/3865913539 >> v2:172.21.15.184:3300/0 conn(0x7fb804073390 msgr2=0x7fb80406f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80937e640 1 --2- 172.21.15.40:0/3865913539 >> v2:172.21.15.184:3300/0 conn(0x7fb804073390 0x7fb80406f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.046+0000 7fb80937e640 1 -- 172.21.15.40:0/3865913539 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8040b6110 con 0x7fb804072fb0 2024-04-26T21:42:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.047+0000 7fb80937e640 1 --2- 172.21.15.40:0/3865913539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb804072fb0 0x7fb8040b56c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb8040a00e0 tx=0x7fb80002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f58ecaf90d4e9df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.047+0000 7fb7fbfff640 1 -- 172.21.15.40:0/3865913539 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800037070 con 0x7fb804072fb0 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.047+0000 7fb7fbfff640 1 -- 172.21.15.40:0/3865913539 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800032440 con 0x7fb804072fb0 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.047+0000 7fb7fbfff640 1 -- 172.21.15.40:0/3865913539 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800032810 con 0x7fb804072fb0 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 -- 172.21.15.40:0/3865913539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb804072fb0 msgr2=0x7fb8040b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/3865913539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb804072fb0 0x7fb8040b56c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb8040a00e0 tx=0x7fb80002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 -- 172.21.15.40:0/3865913539 shutdown_connections 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/3865913539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb80406fc80 0x7fb804070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/3865913539 >> v2:172.21.15.184:3300/0 conn(0x7fb804073390 0x7fb80406f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/3865913539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb804072fb0 0x7fb8040b56c0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 -- 172.21.15.40:0/3865913539 >> 172.21.15.40:0/3865913539 conn(0x7fb8040b39b0 msgr2=0x7fb8040b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 -- 172.21.15.40:0/3865913539 shutdown_connections 2024-04-26T21:42:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.048+0000 7fb80b5e2640 1 -- 172.21.15.40:0/3865913539 wait complete. 2024-04-26T21:42:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.049+0000 7fb80b5e2640 1 Processor -- start 2024-04-26T21:42:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.049+0000 7fb80b5e2640 1 -- start start 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80b5e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb80406fc80 0x7fb804147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80b5e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb804072fb0 0x7fb80414f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80b5e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb804073390 0x7fb80414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80b5e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb80415c160 con 0x7fb80406fc80 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80b5e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8040737f0 con 0x7fb804073390 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80b5e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb80415c2e0 con 0x7fb804072fb0 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb80406fc80 0x7fb804147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb80406fc80 0x7fb804147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45830/0 (socket says 172.21.15.40:45830) 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 -- 172.21.15.40:0/4037604959 learned_addr learned my addr 172.21.15.40:0/4037604959 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb809b7f640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb804073390 0x7fb80414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb808b7d640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb804072fb0 0x7fb80414f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 -- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb804072fb0 msgr2=0x7fb80414f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb804072fb0 0x7fb80414f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 -- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb804073390 msgr2=0x7fb80414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb804073390 0x7fb80414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 -- 172.21.15.40:0/4037604959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8041481b0 con 0x7fb80406fc80 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb809b7f640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb804073390 0x7fb80414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80937e640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb80406fc80 0x7fb804147880 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb800000c00 tx=0x7fb8000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb7f9ffb640 1 -- 172.21.15.40:0/4037604959 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800041020 con 0x7fb80406fc80 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb7f9ffb640 1 -- 172.21.15.40:0/4037604959 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8000306c0 con 0x7fb80406fc80 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb804148490 con 0x7fb80406fc80 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.050+0000 7fb7f9ffb640 1 -- 172.21.15.40:0/4037604959 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800037040 con 0x7fb80406fc80 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.051+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb804148a50 con 0x7fb80406fc80 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.052+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8040694f0 con 0x7fb80406fc80 2024-04-26T21:42:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.052+0000 7fb7f9ffb640 1 -- 172.21.15.40:0/4037604959 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb800044020 con 0x7fb80406fc80 2024-04-26T21:42:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.053+0000 7fb7f9ffb640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7e806c8f0 0x7fb7e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.053+0000 7fb7f9ffb640 1 -- 172.21.15.40:0/4037604959 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6273+0+0 (secure 0 0 0) 0x7fb8000ac680 con 0x7fb80406fc80 2024-04-26T21:42:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.053+0000 7fb808b7d640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7e806c8f0 0x7fb7e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.053+0000 7fb808b7d640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7e806c8f0 0x7fb7e806edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb80414b5b0 tx=0x7fb7f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:42.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.057+0000 7fb7f9ffb640 1 -- 172.21.15.40:0/4037604959 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb80004b030 con 0x7fb80406fc80 2024-04-26T21:42:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.286+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb8040d19e0 con 0x7fb80406fc80 2024-04-26T21:42:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.564+0000 7fb7f9ffb640 1 -- 172.21.15.40:0/4037604959 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v99) ==== 162+0+0 (secure 0 0 0) 0x7fb80007a7d0 con 0x7fb80406fc80 2024-04-26T21:42:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.566+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7e806c8f0 msgr2=0x7fb7e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.566+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7e806c8f0 0x7fb7e806edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb80414b5b0 tx=0x7fb7f4008040 comp rx=0 tx=0).stop 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb80406fc80 msgr2=0x7fb804147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb80406fc80 0x7fb804147880 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb800000c00 tx=0x7fb8000046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 shutdown_connections 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb7e806c8f0 0x7fb7e806edb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb804073390 0x7fb80414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb804072fb0 0x7fb80414f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 --2- 172.21.15.40:0/4037604959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb80406fc80 0x7fb804147880 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 >> 172.21.15.40:0/4037604959 conn(0x7fb8040b39b0 msgr2=0x7fb8040b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 shutdown_connections 2024-04-26T21:42:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.567+0000 7fb80b5e2640 1 -- 172.21.15.40:0/4037604959 wait complete. 2024-04-26T21:42:42.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.680+0000 7f8bc4674640 1 Processor -- start 2024-04-26T21:42:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.681+0000 7f8bc4674640 1 -- start start 2024-04-26T21:42:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.681+0000 7f8bc4674640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bbc15f3a0 0x7f8bbc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.681+0000 7f8bc4674640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8bbc0d1770 0x7f8bbc0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.681+0000 7f8bc4674640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bbc0d2590 con 0x7f8bbc0d2160 2024-04-26T21:42:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.681+0000 7f8bc4674640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8bbc0d0e20 con 0x7f8bbc15f3a0 2024-04-26T21:42:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.681+0000 7f8bc4674640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8bbc0b1d70 con 0x7f8bbc0d1770 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc2410640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bbc15f3a0 0x7f8bbc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc2c11640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8bbc0d2160 0x7f8bbc0d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43004/0 (socket says 172.21.15.40:43004) 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc2c11640 1 -- 172.21.15.40:0/3010032846 learned_addr learned my addr 172.21.15.40:0/3010032846 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc2410640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bbc15f3a0 0x7f8bbc0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45838/0 (socket says 172.21.15.40:45838) 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc1c0f640 1 --2- 172.21.15.40:0/3010032846 >> v2:172.21.15.184:3300/0 conn(0x7f8bbc0d1770 0x7f8bbc0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc2410640 1 -- 172.21.15.40:0/3010032846 >> v2:172.21.15.184:3300/0 conn(0x7f8bbc0d1770 msgr2=0x7f8bbc0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc140e640 1 -- 172.21.15.40:0/3010032846 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3111601053 0 0) 0x7f8bbc0d2590 con 0x7f8bbc0d2160 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc2410640 1 --2- 172.21.15.40:0/3010032846 >> v2:172.21.15.184:3300/0 conn(0x7f8bbc0d1770 0x7f8bbc0d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc2410640 1 -- 172.21.15.40:0/3010032846 >> v1:172.21.15.40:6789/0 conn(0x7f8bbc0d2160 legacy=0x7f8bbc0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc2410640 1 -- 172.21.15.40:0/3010032846 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc0b2050 con 0x7f8bbc15f3a0 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.682+0000 7f8bc1c0f640 1 --2- 172.21.15.40:0/3010032846 >> v2:172.21.15.184:3300/0 conn(0x7f8bbc0d1770 0x7f8bbc0d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc2410640 1 --2- 172.21.15.40:0/3010032846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc15f3a0 0x7f8bbc0d06f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0b7b80 tx=0x7f8bb402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f27075c7a0cd7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc140e640 1 -- 172.21.15.40:0/3010032846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4007740 con 0x7f8bbc15f3a0 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc140e640 1 -- 172.21.15.40:0/3010032846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb40078e0 con 0x7f8bbc15f3a0 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc140e640 1 -- 172.21.15.40:0/3010032846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4007cb0 con 0x7f8bbc15f3a0 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc4674640 1 -- 172.21.15.40:0/3010032846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc15f3a0 msgr2=0x7f8bbc0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc4674640 1 --2- 172.21.15.40:0/3010032846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc15f3a0 0x7f8bbc0d06f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0b7b80 tx=0x7f8bb402a680 comp rx=0 tx=0).stop 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc4674640 1 -- 172.21.15.40:0/3010032846 shutdown_connections 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc4674640 1 --2- 172.21.15.40:0/3010032846 >> v2:172.21.15.184:3300/0 conn(0x7f8bbc0d1770 0x7f8bbc0d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc4674640 1 --2- 172.21.15.40:0/3010032846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc15f3a0 0x7f8bbc0d06f0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc4674640 1 -- 172.21.15.40:0/3010032846 >> 172.21.15.40:0/3010032846 conn(0x7f8bbc0d4f90 msgr2=0x7f8bbc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.683+0000 7f8bc4674640 1 -- 172.21.15.40:0/3010032846 shutdown_connections 2024-04-26T21:42:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.684+0000 7f8bc4674640 1 -- 172.21.15.40:0/3010032846 wait complete. 2024-04-26T21:42:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc4674640 1 Processor -- start 2024-04-26T21:42:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc4674640 1 -- start start 2024-04-26T21:42:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc4674640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0d1770 0x7f8bbc153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc4674640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bbc0d2160 0x7f8bbc0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc4674640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bbc15f3a0 0x7f8bbc0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc4674640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc15f900 con 0x7f8bbc0d1770 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc4674640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc15f780 con 0x7f8bbc15f3a0 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc4674640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8bbc0b3860 con 0x7f8bbc0d2160 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc2410640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0d1770 0x7f8bbc153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc2410640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0d1770 0x7f8bbc153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45854/0 (socket says 172.21.15.40:45854) 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.685+0000 7f8bc2410640 1 -- 172.21.15.40:0/771018780 learned_addr learned my addr 172.21.15.40:0/771018780 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc2c11640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bbc15f3a0 0x7f8bbc0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc1c0f640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bbc0d2160 0x7f8bbc0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc2410640 1 -- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bbc0d2160 msgr2=0x7f8bbc0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc2410640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bbc0d2160 0x7f8bbc0c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc2410640 1 -- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bbc15f3a0 msgr2=0x7f8bbc0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc2410640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bbc15f3a0 0x7f8bbc0b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc2410640 1 -- 172.21.15.40:0/771018780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc153950 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc1c0f640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bbc0d2160 0x7f8bbc0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc2410640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0d1770 0x7f8bbc153410 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009e80 tx=0x7f8bb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bbaffd640 1 -- 172.21.15.40:0/771018780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb403f070 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bbaffd640 1 -- 172.21.15.40:0/771018780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb4031c40 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bbc0cc2b0 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bbaffd640 1 -- 172.21.15.40:0/771018780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4043410 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.686+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bbc0cc870 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.687+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bbc0694f0 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.688+0000 7f8bbaffd640 1 -- 172.21.15.40:0/771018780 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bb40435b0 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.688+0000 7f8bbaffd640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.688+0000 7f8bbaffd640 1 -- 172.21.15.40:0/771018780 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6273+0+0 (secure 0 0 0) 0x7f8bb40ad450 con 0x7f8bbc0d1770 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.688+0000 7f8bc1c0f640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.689+0000 7f8bc1c0f640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ba806c8f0 0x7f8ba806edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0006fd0 tx=0x7f8bb0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.692+0000 7f8bbaffd640 1 -- 172.21.15.40:0/771018780 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8bbc153950 con 0x7f8bbc0d1770 2024-04-26T21:42:42.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.918+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bbc0b6860 con 0x7f8bbc0d1770 2024-04-26T21:42:42.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.918+0000 7f8bbaffd640 1 -- 172.21.15.40:0/771018780 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5186 (secure 0 0 0) 0x7f8bb407b5a0 con 0x7f8bbc0d1770 2024-04-26T21:42:42.919 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:42.919 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:42.567856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8505":{"gid":8505,"name":"a","rank":0,"incarnation":100,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6835/1482743077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1482743077},{"type":"v1","addr":"172.21.15.40:6835","nonce":1482743077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 100 2024-04-26T21:42:42.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ba806c8f0 msgr2=0x7f8ba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ba806c8f0 0x7f8ba806edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0006fd0 tx=0x7f8bb0008040 comp rx=0 tx=0).stop 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0d1770 msgr2=0x7f8bbc153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0d1770 0x7f8bbc153410 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009e80 tx=0x7f8bb4004a40 comp rx=0 tx=0).stop 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 shutdown_connections 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bbc15f3a0 0x7f8bbc0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bbc0d2160 0x7f8bbc0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 --2- 172.21.15.40:0/771018780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0d1770 0x7f8bbc153410 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 >> 172.21.15.40:0/771018780 conn(0x7f8bbc0d4f90 msgr2=0x7f8bbc06e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 shutdown_connections 2024-04-26T21:42:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:42.920+0000 7f8bc4674640 1 -- 172.21.15.40:0/771018780 wait complete. 2024-04-26T21:42:42.932 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:42:45.590 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:42:45.589+0000 7fe7f76ea640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-04-26T21:42:45.590 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:42:45.589+0000 7fe7f76ea640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-04-26T21:42:45.590 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:45.589+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:42:45.591 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:45.589+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:42:45.596 INFO:tasks.ceph.mds.a.smithi040.stderr: -16> 2024-04-26T21:42:45.589+0000 7fe7f76ea640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-04-26T21:42:45.596 INFO:tasks.ceph.mds.a.smithi040.stderr: -14> 2024-04-26T21:42:45.589+0000 7fe7f76ea640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-04-26T21:42:45.624 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:42:47.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:48.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.046+0000 7f6aaf050640 1 Processor -- start 2024-04-26T21:42:48.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.046+0000 7f6aaf050640 1 -- start start 2024-04-26T21:42:48.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aaf050640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6aa8071e60 0x7f6aa806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aaf050640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6aa806fe20 0x7f6aa8072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aaf050640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa8072780 0x7f6aa80cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aaf050640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa815d7c0 con 0x7f6aa806fe20 2024-04-26T21:42:48.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aaf050640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa815daa0 con 0x7f6aa8071e60 2024-04-26T21:42:48.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aaf050640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa815dd80 con 0x7f6aa8072780 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aa7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6aa806fe20 0x7f6aa8072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aa7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6aa806fe20 0x7f6aa8072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45878/0 (socket says 172.21.15.40:45878) 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aa7fff640 1 -- 172.21.15.40:0/2624098896 learned_addr learned my addr 172.21.15.40:0/2624098896 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aacdec640 1 --2- 172.21.15.40:0/2624098896 >> v2:172.21.15.184:3300/0 conn(0x7f6aa8071e60 0x7f6aa806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aad5ed640 1 --2- 172.21.15.40:0/2624098896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa8072780 0x7f6aa80cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aa7fff640 1 -- 172.21.15.40:0/2624098896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa8072780 msgr2=0x7f6aa80cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aa7fff640 1 --2- 172.21.15.40:0/2624098896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa8072780 0x7f6aa80cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aa7fff640 1 -- 172.21.15.40:0/2624098896 >> v2:172.21.15.184:3300/0 conn(0x7f6aa8071e60 msgr2=0x7f6aa806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aa7fff640 1 --2- 172.21.15.40:0/2624098896 >> v2:172.21.15.184:3300/0 conn(0x7f6aa8071e60 0x7f6aa806f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.047+0000 7f6aa7fff640 1 -- 172.21.15.40:0/2624098896 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa815e060 con 0x7f6aa806fe20 2024-04-26T21:42:48.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.048+0000 7f6aa7fff640 1 --2- 172.21.15.40:0/2624098896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa806fe20 0x7f6aa8072240 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80b3700 tx=0x7f6a9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b8c7f163bbb9b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.048+0000 7f6aa77fe640 1 -- 172.21.15.40:0/2624098896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94037070 con 0x7f6aa806fe20 2024-04-26T21:42:48.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.048+0000 7f6aa77fe640 1 -- 172.21.15.40:0/2624098896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a94032440 con 0x7f6aa806fe20 2024-04-26T21:42:48.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.048+0000 7f6aa77fe640 1 -- 172.21.15.40:0/2624098896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94032810 con 0x7f6aa806fe20 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 -- 172.21.15.40:0/2624098896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa806fe20 msgr2=0x7f6aa8072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2624098896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa806fe20 0x7f6aa8072240 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80b3700 tx=0x7f6a9402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 -- 172.21.15.40:0/2624098896 shutdown_connections 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2624098896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa8072780 0x7f6aa80cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2624098896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa806fe20 0x7f6aa8072240 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2624098896 >> v2:172.21.15.184:3300/0 conn(0x7f6aa8071e60 0x7f6aa806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 -- 172.21.15.40:0/2624098896 >> 172.21.15.40:0/2624098896 conn(0x7f6aa80d0be0 msgr2=0x7f6aa80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 -- 172.21.15.40:0/2624098896 shutdown_connections 2024-04-26T21:42:48.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.049+0000 7f6aaf050640 1 -- 172.21.15.40:0/2624098896 wait complete. 2024-04-26T21:42:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.050+0000 7f6aaf050640 1 Processor -- start 2024-04-26T21:42:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.050+0000 7f6aaf050640 1 -- start start 2024-04-26T21:42:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aaf050640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa806fe20 0x7f6aa8149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aaf050640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa8071e60 0x7f6aa80ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aaf050640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6aa8072780 0x7f6aa80ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aaf050640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa815d3f0 con 0x7f6aa8071e60 2024-04-26T21:42:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aaf050640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8072c10 con 0x7f6aa8072780 2024-04-26T21:42:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aaf050640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa815d570 con 0x7f6aa806fe20 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa8071e60 0x7f6aa80ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa8071e60 0x7f6aa80ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45882/0 (socket says 172.21.15.40:45882) 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aa7fff640 1 -- 172.21.15.40:0/2636374980 learned_addr learned my addr 172.21.15.40:0/2636374980 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aacdec640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa806fe20 0x7f6aa8149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.051+0000 7f6aad5ed640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6aa8072780 0x7f6aa80ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa7fff640 1 -- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa806fe20 msgr2=0x7f6aa8149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa7fff640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa806fe20 0x7f6aa8149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa7fff640 1 -- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6aa8072780 msgr2=0x7f6aa80ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa7fff640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6aa8072780 0x7f6aa80ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa7fff640 1 -- 172.21.15.40:0/2636374980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa80c7e30 con 0x7f6aa8071e60 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa7fff640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa8071e60 0x7f6aa80ca130 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f6a9402a220 tx=0x7f6a940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa57fa640 1 -- 172.21.15.40:0/2636374980 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94041020 con 0x7f6aa8071e60 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa57fa640 1 -- 172.21.15.40:0/2636374980 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9403d620 con 0x7f6aa8071e60 2024-04-26T21:42:48.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa80c8110 con 0x7f6aa8071e60 2024-04-26T21:42:48.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aa57fa640 1 -- 172.21.15.40:0/2636374980 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94037040 con 0x7f6aa8071e60 2024-04-26T21:42:48.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.052+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa80b44d0 con 0x7f6aa8071e60 2024-04-26T21:42:48.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.053+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa80694f0 con 0x7f6aa8071e60 2024-04-26T21:42:48.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.054+0000 7f6aa57fa640 1 -- 172.21.15.40:0/2636374980 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a94044020 con 0x7f6aa8071e60 2024-04-26T21:42:48.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.054+0000 7f6aa57fa640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06c8f0 0x7f6a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.054+0000 7f6aa57fa640 1 -- 172.21.15.40:0/2636374980 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6359+0+0 (secure 0 0 0) 0x7f6a940ac3a0 con 0x7f6aa8071e60 2024-04-26T21:42:48.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.055+0000 7f6aacdec640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06c8f0 0x7f6a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.055+0000 7f6aacdec640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06c8f0 0x7f6a8c06edb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6aa81f5d90 tx=0x7f6a9c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.059+0000 7f6aa57fa640 1 -- 172.21.15.40:0/2636374980 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a9404b030 con 0x7f6aa8071e60 2024-04-26T21:42:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.289+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aa806ce60 con 0x7f6aa8071e60 2024-04-26T21:42:48.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.289+0000 7f6aa57fa640 1 -- 172.21.15.40:0/2636374980 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5174 (secure 0 0 0) 0x7f6a9407a5d0 con 0x7f6aa8071e60 2024-04-26T21:42:48.290 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:48.291 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/747650264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":747650264},{"type":"v1","addr":"172.21.15.40:6835","nonce":747650264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:45.591155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:48.291 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 105 2024-04-26T21:42:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06c8f0 msgr2=0x7f6a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06c8f0 0x7f6a8c06edb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6aa81f5d90 tx=0x7f6a9c009190 comp rx=0 tx=0).stop 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa8071e60 msgr2=0x7f6aa80ca130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa8071e60 0x7f6aa80ca130 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f6a9402a220 tx=0x7f6a940046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 shutdown_connections 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a8c06c8f0 0x7f6a8c06edb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6aa8072780 0x7f6aa80ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa8071e60 0x7f6aa80ca130 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.291+0000 7f6aaf050640 1 --2- 172.21.15.40:0/2636374980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa806fe20 0x7f6aa8149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.292+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 >> 172.21.15.40:0/2636374980 conn(0x7f6aa80d0be0 msgr2=0x7f6aa80b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.292+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 shutdown_connections 2024-04-26T21:42:48.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.292+0000 7f6aaf050640 1 -- 172.21.15.40:0/2636374980 wait complete. 2024-04-26T21:42:48.303 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:42:48.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:48.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.403+0000 7f346f82a640 1 Processor -- start 2024-04-26T21:42:48.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.404+0000 7f346f82a640 1 -- start start 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.404+0000 7f346f82a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f34680d3bc0 0x7f34680d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.404+0000 7f346f82a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34680b7e50 con 0x7f3468162880 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.404+0000 7f346f82a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34680d1800 con 0x7f34680b5f20 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.404+0000 7f346f82a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f34680d0580 con 0x7f34680d3bc0 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.404+0000 7f346d5c6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3468162880 0x7f34680d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43036/0 (socket says 172.21.15.40:43036) 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.404+0000 7f346cdc5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f34680d3bc0 0x7f34680d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.405+0000 7f346d5c6640 1 -- 172.21.15.40:0/3994754591 learned_addr learned my addr 172.21.15.40:0/3994754591 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.405+0000 7f345ffff640 1 -- 172.21.15.40:0/3994754591 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717986113 0 0) 0x7f34680b7e50 con 0x7f3468162880 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.405+0000 7f345ffff640 1 -- 172.21.15.40:0/3994754591 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3454003520 con 0x7f3468162880 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.405+0000 7f346cdc5640 1 -- 172.21.15.40:0/3994754591 >> v1:172.21.15.184:6789/0 conn(0x7f34680b5f20 legacy=0x7f34680b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.405+0000 7f346cdc5640 1 -- 172.21.15.40:0/3994754591 >> v1:172.21.15.40:6789/0 conn(0x7f3468162880 legacy=0x7f34680d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.405+0000 7f346cdc5640 1 -- 172.21.15.40:0/3994754591 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34680d07d0 con 0x7f34680d3bc0 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f346cdc5640 1 --2- 172.21.15.40:0/3994754591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680d3bc0 0x7f34680d2cb0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f34580097a0 tx=0x7f345802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105b95aa180f9392 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f345ffff640 1 -- 172.21.15.40:0/3994754591 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3458037070 con 0x7f34680d3bc0 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f345ffff640 1 -- 172.21.15.40:0/3994754591 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3458032440 con 0x7f34680d3bc0 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f345ffff640 1 -- 172.21.15.40:0/3994754591 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34580327f0 con 0x7f34680d3bc0 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f346f82a640 1 -- 172.21.15.40:0/3994754591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680d3bc0 msgr2=0x7f34680d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f346f82a640 1 --2- 172.21.15.40:0/3994754591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680d3bc0 0x7f34680d2cb0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f34580097a0 tx=0x7f345802a230 comp rx=0 tx=0).stop 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f346f82a640 1 -- 172.21.15.40:0/3994754591 shutdown_connections 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f346f82a640 1 --2- 172.21.15.40:0/3994754591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680d3bc0 0x7f34680d2cb0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f346f82a640 1 -- 172.21.15.40:0/3994754591 >> 172.21.15.40:0/3994754591 conn(0x7f34680d4f60 msgr2=0x7f34680d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f346f82a640 1 -- 172.21.15.40:0/3994754591 shutdown_connections 2024-04-26T21:42:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.406+0000 7f346f82a640 1 -- 172.21.15.40:0/3994754591 wait complete. 2024-04-26T21:42:48.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.407+0000 7f346f82a640 1 Processor -- start 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346f82a640 1 -- start start 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346f82a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680b5f20 0x7f346814cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346f82a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34680d3bc0 0x7f346814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346f82a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3468162880 0x7f34680b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346f82a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3468162c60 con 0x7f34680b5f20 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346f82a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34680d0d90 con 0x7f34680d3bc0 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346f82a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3468162de0 con 0x7f3468162880 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346d5c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680b5f20 0x7f346814cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346d5c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680b5f20 0x7f346814cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45904/0 (socket says 172.21.15.40:45904) 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.408+0000 7f346d5c6640 1 -- 172.21.15.40:0/3721499038 learned_addr learned my addr 172.21.15.40:0/3721499038 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346cdc5640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34680d3bc0 0x7f346814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346ddc7640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3468162880 0x7f34680b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346d5c6640 1 -- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3468162880 msgr2=0x7f34680b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346d5c6640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3468162880 0x7f34680b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346d5c6640 1 -- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34680d3bc0 msgr2=0x7f346814c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346d5c6640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34680d3bc0 0x7f346814c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346d5c6640 1 -- 172.21.15.40:0/3721499038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34680b2d20 con 0x7f34680b5f20 2024-04-26T21:42:48.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346ddc7640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3468162880 0x7f34680b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:48.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346d5c6640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680b5f20 0x7f346814cb90 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f346400c410 tx=0x7f346400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f345dffb640 1 -- 172.21.15.40:0/3721499038 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464017070 con 0x7f34680b5f20 2024-04-26T21:42:48.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f345dffb640 1 -- 172.21.15.40:0/3721499038 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3464005ce0 con 0x7f34680b5f20 2024-04-26T21:42:48.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34680b3000 con 0x7f34680b5f20 2024-04-26T21:42:48.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f345dffb640 1 -- 172.21.15.40:0/3721499038 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34640072f0 con 0x7f34680b5f20 2024-04-26T21:42:48.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.409+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34680b35c0 con 0x7f34680b5f20 2024-04-26T21:42:48.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.410+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34680cfb00 con 0x7f34680b5f20 2024-04-26T21:42:48.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.411+0000 7f345dffb640 1 -- 172.21.15.40:0/3721499038 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f346401a070 con 0x7f34680b5f20 2024-04-26T21:42:48.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.411+0000 7f345dffb640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f344c06c8f0 0x7f344c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.411+0000 7f345dffb640 1 -- 172.21.15.40:0/3721499038 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6359+0+0 (secure 0 0 0) 0x7f346408b2f0 con 0x7f34680b5f20 2024-04-26T21:42:48.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.411+0000 7f346cdc5640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f344c06c8f0 0x7f344c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.412+0000 7f346cdc5640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f344c06c8f0 0x7f344c06edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3458034000 tx=0x7f345802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.416+0000 7f345dffb640 1 -- 172.21.15.40:0/3721499038 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34680cfb00 con 0x7f34680b5f20 2024-04-26T21:42:48.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.639+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34680694f0 con 0x7f34680b5f20 2024-04-26T21:42:48.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.639+0000 7f345dffb640 1 -- 172.21.15.40:0/3721499038 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5174 (secure 0 0 0) 0x7f34640594c0 con 0x7f34680b5f20 2024-04-26T21:42:48.640 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:48.641 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/747650264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":747650264},{"type":"v1","addr":"172.21.15.40:6835","nonce":747650264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:45.591155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:48.641 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 105 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f344c06c8f0 msgr2=0x7f344c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f344c06c8f0 0x7f344c06edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3458034000 tx=0x7f345802e000 comp rx=0 tx=0).stop 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680b5f20 msgr2=0x7f346814cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680b5f20 0x7f346814cb90 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f346400c410 tx=0x7f346400c8d0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 shutdown_connections 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f344c06c8f0 0x7f344c06edb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3468162880 0x7f34680b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34680d3bc0 0x7f346814c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 --2- 172.21.15.40:0/3721499038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34680b5f20 0x7f346814cb90 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 >> 172.21.15.40:0/3721499038 conn(0x7f34680d4f60 msgr2=0x7f346815f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 shutdown_connections 2024-04-26T21:42:48.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.642+0000 7f346f82a640 1 -- 172.21.15.40:0/3721499038 wait complete. 2024-04-26T21:42:48.654 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:42:48.654 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:48.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.753+0000 7ff9dd5e2640 1 Processor -- start 2024-04-26T21:42:48.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.753+0000 7ff9dd5e2640 1 -- start start 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9dd5e2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9d815f3a0 0x7ff9d80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9dd5e2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9d80d2330 0x7ff9d80b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9dd5e2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9d80d0e20 con 0x7ff9d80d1770 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9dd5e2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d80b1d70 con 0x7ff9d815f3a0 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9dd5e2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d80b2050 con 0x7ff9d80d2330 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9d6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9d815f3a0 0x7ff9d80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9d6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9d815f3a0 0x7ff9d80d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36982/0 (socket says 172.21.15.40:36982) 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/311111373 learned_addr learned my addr 172.21.15.40:0/311111373 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.754+0000 7ff9d77fe640 1 --2- 172.21.15.40:0/311111373 >> v2:172.21.15.184:3300/0 conn(0x7ff9d80d2330 0x7ff9d80b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.755+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/311111373 >> v2:172.21.15.184:3300/0 conn(0x7ff9d80d2330 msgr2=0x7ff9d80b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.755+0000 7ff9d6ffd640 1 --2- 172.21.15.40:0/311111373 >> v2:172.21.15.184:3300/0 conn(0x7ff9d80d2330 0x7ff9d80b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.755+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/311111373 >> v1:172.21.15.184:6789/0 conn(0x7ff9d80d1770 legacy=0x7ff9d80d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.755+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/311111373 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d80b2330 con 0x7ff9d815f3a0 2024-04-26T21:42:48.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.755+0000 7ff9d6ffd640 1 --2- 172.21.15.40:0/311111373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d815f3a0 0x7ff9d80d06f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80c89f0 tx=0x7ff9c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fe1c3905e6af34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.755+0000 7ff9d5ffb640 1 -- 172.21.15.40:0/311111373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8037070 con 0x7ff9d815f3a0 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.755+0000 7ff9d5ffb640 1 -- 172.21.15.40:0/311111373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c8032440 con 0x7ff9d815f3a0 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.755+0000 7ff9d5ffb640 1 -- 172.21.15.40:0/311111373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8032810 con 0x7ff9d815f3a0 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.756+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/311111373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d815f3a0 msgr2=0x7ff9d80d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.756+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/311111373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d815f3a0 0x7ff9d80d06f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80c89f0 tx=0x7ff9c802a270 comp rx=0 tx=0).stop 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.756+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/311111373 shutdown_connections 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.756+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/311111373 >> v2:172.21.15.184:3300/0 conn(0x7ff9d80d2330 0x7ff9d80b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.756+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/311111373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d815f3a0 0x7ff9d80d06f0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.756+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/311111373 >> 172.21.15.40:0/311111373 conn(0x7ff9d80d4f90 msgr2=0x7ff9d80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:48.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.756+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/311111373 shutdown_connections 2024-04-26T21:42:48.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.756+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/311111373 wait complete. 2024-04-26T21:42:48.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.757+0000 7ff9dd5e2640 1 Processor -- start 2024-04-26T21:42:48.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.757+0000 7ff9dd5e2640 1 -- start start 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9dd5e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d80d1770 0x7ff9d80cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9dd5e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9d80d2330 0x7ff9d81584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9dd5e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9d815f3a0 0x7ff9d80b8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9dd5e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d80b2f10 con 0x7ff9d80d1770 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9dd5e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d80b2d90 con 0x7ff9d815f3a0 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9dd5e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d80b3090 con 0x7ff9d80d2330 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9d6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d80d1770 0x7ff9d80cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9d6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d80d1770 0x7ff9d80cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36984/0 (socket says 172.21.15.40:36984) 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/2128899920 learned_addr learned my addr 172.21.15.40:0/2128899920 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9d77fe640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9d815f3a0 0x7ff9d80b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.758+0000 7ff9d67fc640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9d80d2330 0x7ff9d81584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9d80d2330 msgr2=0x7ff9d81584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9d6ffd640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9d80d2330 0x7ff9d81584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9d815f3a0 msgr2=0x7ff9d80b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9d6ffd640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9d815f3a0 0x7ff9d80b8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/2128899920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d80c8b60 con 0x7ff9d80d1770 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9d6ffd640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d80d1770 0x7ff9d80cb300 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff9c802a2b0 tx=0x7ff9c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9bf7fe640 1 -- 172.21.15.40:0/2128899920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8041020 con 0x7ff9d80d1770 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9bf7fe640 1 -- 172.21.15.40:0/2128899920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c80306c0 con 0x7ff9d80d1770 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d8158a10 con 0x7ff9d80d1770 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9bf7fe640 1 -- 172.21.15.40:0/2128899920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8037040 con 0x7ff9d80d1770 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.759+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d80c90e0 con 0x7ff9d80d1770 2024-04-26T21:42:48.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.760+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d80694f0 con 0x7ff9d80d1770 2024-04-26T21:42:48.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.761+0000 7ff9bf7fe640 1 -- 172.21.15.40:0/2128899920 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9c8044020 con 0x7ff9d80d1770 2024-04-26T21:42:48.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.761+0000 7ff9bf7fe640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9c406c8f0 0x7ff9c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:48.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.761+0000 7ff9bf7fe640 1 -- 172.21.15.40:0/2128899920 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6359+0+0 (secure 0 0 0) 0x7ff9c80ac630 con 0x7ff9d80d1770 2024-04-26T21:42:48.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.761+0000 7ff9d67fc640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9c406c8f0 0x7ff9c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:48.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.762+0000 7ff9d67fc640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9c406c8f0 0x7ff9c406edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000f60 tx=0x7ff9cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.765+0000 7ff9bf7fe640 1 -- 172.21.15.40:0/2128899920 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9c804b030 con 0x7ff9d80d1770 2024-04-26T21:42:48.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.989+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9d80b6860 con 0x7ff9d80d1770 2024-04-26T21:42:48.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.989+0000 7ff9bf7fe640 1 -- 172.21.15.40:0/2128899920 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5174 (secure 0 0 0) 0x7ff9c807a7d0 con 0x7ff9d80d1770 2024-04-26T21:42:48.991 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:48.991 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/747650264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":747650264},{"type":"v1","addr":"172.21.15.40:6835","nonce":747650264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:45.591155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:48.991 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 105 2024-04-26T21:42:48.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9c406c8f0 msgr2=0x7ff9c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9c406c8f0 0x7ff9c406edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000f60 tx=0x7ff9cc009040 comp rx=0 tx=0).stop 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d80d1770 msgr2=0x7ff9d80cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d80d1770 0x7ff9d80cb300 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff9c802a2b0 tx=0x7ff9c80046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 shutdown_connections 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9c406c8f0 0x7ff9c406edb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9d815f3a0 0x7ff9d80b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9d80d2330 0x7ff9d81584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 --2- 172.21.15.40:0/2128899920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d80d1770 0x7ff9d80cb300 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 >> 172.21.15.40:0/2128899920 conn(0x7ff9d80d4f90 msgr2=0x7ff9d8074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 shutdown_connections 2024-04-26T21:42:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:48.991+0000 7ff9dd5e2640 1 -- 172.21.15.40:0/2128899920 wait complete. 2024-04-26T21:42:49.003 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 600.00000000' led to DAMAGED state 2024-04-26T21:42:49.003 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000.inode' 2024-04-26T21:42:49.003 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:49.003 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:49.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:42:49.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:42:49.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:49.019 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:42:49.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:42:49.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:42:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.215+0000 7f491f083640 1 Processor -- start 2024-04-26T21:42:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.215+0000 7f491f083640 1 -- start start 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f491f083640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49180b2340 0x7f49180cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f491f083640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49180d10d0 con 0x7f49180cd930 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f491f083640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49180d1370 con 0x7f4918162810 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f491f083640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f49180d0750 con 0x7f49180b2340 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f4917fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49180b2340 0x7f49180cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f491d620640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f49180cd930 0x7f49180d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35848/0 (socket says 172.21.15.40:35848) 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f491d620640 1 -- 172.21.15.40:0/4016369458 learned_addr learned my addr 172.21.15.40:0/4016369458 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f4917fff640 1 -- 172.21.15.40:0/4016369458 >> v1:172.21.15.184:6789/0 conn(0x7f4918162810 legacy=0x7f4918070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f4917fff640 1 -- 172.21.15.40:0/4016369458 >> v1:172.21.15.40:6789/0 conn(0x7f49180cd930 legacy=0x7f49180d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f4917fff640 1 -- 172.21.15.40:0/4016369458 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49180d01b0 con 0x7f49180b2340 2024-04-26T21:42:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.216+0000 7f49177fe640 1 -- 172.21.15.40:0/4016369458 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551960695 0 0) 0x7f49180d10d0 con 0x7f49180cd930 2024-04-26T21:42:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.217+0000 7f4917fff640 1 --2- 172.21.15.40:0/4016369458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49180b2340 0x7f49180cd3f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f49180d1540 tx=0x7f490402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3cfea09feebea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.217+0000 7f49177fe640 1 -- 172.21.15.40:0/4016369458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904037070 con 0x7f49180b2340 2024-04-26T21:42:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.217+0000 7f49177fe640 1 -- 172.21.15.40:0/4016369458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4904032440 con 0x7f49180b2340 2024-04-26T21:42:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.217+0000 7f49177fe640 1 -- 172.21.15.40:0/4016369458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904032810 con 0x7f49180b2340 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.218+0000 7f491f083640 1 -- 172.21.15.40:0/4016369458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49180b2340 msgr2=0x7f49180cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.218+0000 7f491f083640 1 --2- 172.21.15.40:0/4016369458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49180b2340 0x7f49180cd3f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f49180d1540 tx=0x7f490402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.218+0000 7f491f083640 1 -- 172.21.15.40:0/4016369458 shutdown_connections 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.218+0000 7f491f083640 1 --2- 172.21.15.40:0/4016369458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49180b2340 0x7f49180cd3f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.218+0000 7f491f083640 1 -- 172.21.15.40:0/4016369458 >> 172.21.15.40:0/4016369458 conn(0x7f49180d2da0 msgr2=0x7f49180d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.218+0000 7f491f083640 1 -- 172.21.15.40:0/4016369458 shutdown_connections 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.218+0000 7f491f083640 1 -- 172.21.15.40:0/4016369458 wait complete. 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.219+0000 7f491f083640 1 Processor -- start 2024-04-26T21:42:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.219+0000 7f491f083640 1 -- start start 2024-04-26T21:42:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491f083640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49180b2340 0x7f49180c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491f083640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49180cd930 0x7f491815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491f083640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4918162810 0x7f491815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491f083640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49180d2240 con 0x7f4918162810 2024-04-26T21:42:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491f083640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f49180d16a0 con 0x7f49180b2340 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491f083640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49180d23c0 con 0x7f49180cd930 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4918162810 0x7f491815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4918162810 0x7f491815c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37002/0 (socket says 172.21.15.40:37002) 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 -- 172.21.15.40:0/88037422 learned_addr learned my addr 172.21.15.40:0/88037422 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f4917fff640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49180cd930 0x7f491815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491ce1f640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49180b2340 0x7f49180c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 -- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49180cd930 msgr2=0x7f491815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49180cd930 0x7f491815bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 -- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49180b2340 msgr2=0x7f49180c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49180b2340 0x7f49180c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 -- 172.21.15.40:0/88037422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49180c7270 con 0x7f4918162810 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.220+0000 7f491d620640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4918162810 0x7f491815c0f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f491000c730 tx=0x7f491000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.221+0000 7f49157fa640 1 -- 172.21.15.40:0/88037422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910018070 con 0x7f4918162810 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.221+0000 7f49157fa640 1 -- 172.21.15.40:0/88037422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491000ce40 con 0x7f4918162810 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.221+0000 7f49157fa640 1 -- 172.21.15.40:0/88037422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491000a860 con 0x7f4918162810 2024-04-26T21:42:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.221+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49180c7550 con 0x7f4918162810 2024-04-26T21:42:49.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.221+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49180cb190 con 0x7f4918162810 2024-04-26T21:42:49.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.222+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49180694f0 con 0x7f4918162810 2024-04-26T21:42:49.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.222+0000 7f49157fa640 1 -- 172.21.15.40:0/88037422 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f491001b070 con 0x7f4918162810 2024-04-26T21:42:49.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.222+0000 7f49157fa640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48fc06c920 0x7f48fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.223+0000 7f49157fa640 1 -- 172.21.15.40:0/88037422 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6359+0+0 (secure 0 0 0) 0x7f491008b1f0 con 0x7f4918162810 2024-04-26T21:42:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.223+0000 7f491ce1f640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48fc06c920 0x7f48fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.223+0000 7f491ce1f640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48fc06c920 0x7f48fc06ede0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f490c006fd0 tx=0x7f490c009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.227+0000 7f49157fa640 1 -- 172.21.15.40:0/88037422 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4910058dd0 con 0x7f4918162810 2024-04-26T21:42:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.456+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f491815e0e0 con 0x7f4918162810 2024-04-26T21:42:49.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.609+0000 7f49157fa640 1 -- 172.21.15.40:0/88037422 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7f491005d0d0 con 0x7f4918162810 2024-04-26T21:42:49.610 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48fc06c920 msgr2=0x7f48fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48fc06c920 0x7f48fc06ede0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f490c006fd0 tx=0x7f490c009060 comp rx=0 tx=0).stop 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4918162810 msgr2=0x7f491815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4918162810 0x7f491815c0f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f491000c730 tx=0x7f491000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 shutdown_connections 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f48fc06c920 0x7f48fc06ede0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4918162810 0x7f491815c0f0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49180cd930 0x7f491815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 --2- 172.21.15.40:0/88037422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49180b2340 0x7f49180c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 >> 172.21.15.40:0/88037422 conn(0x7f49180d2da0 msgr2=0x7f49180cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:49.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 shutdown_connections 2024-04-26T21:42:49.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.611+0000 7f491f083640 1 -- 172.21.15.40:0/88037422 wait complete. 2024-04-26T21:42:49.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:42:49.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.724+0000 7fafee609640 1 Processor -- start 2024-04-26T21:42:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.725+0000 7fafee609640 1 -- start start 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.725+0000 7fafee609640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fafe8162810 0x7fafe8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.725+0000 7fafee609640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe80d10d0 con 0x7fafe80d1b30 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.725+0000 7fafee609640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe80d1370 con 0x7fafe80b2340 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.725+0000 7fafee609640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fafe80d0750 con 0x7fafe8162810 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.725+0000 7fafecba6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fafe80d1b30 0x7fafe8070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35852/0 (socket says 172.21.15.40:35852) 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.725+0000 7fafecba6640 1 -- 172.21.15.40:0/3179848149 learned_addr learned my addr 172.21.15.40:0/3179848149 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245392755 0 0) 0x7fafe80d10d0 con 0x7fafe80d1b30 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafd0003520 con 0x7fafe80d1b30 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe7fff640 1 --2- 172.21.15.40:0/3179848149 >> v2:172.21.15.184:3300/0 conn(0x7fafe8162810 0x7fafe8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2588025821 0 0) 0x7fafd0003520 con 0x7fafe80d1b30 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 >> v2:172.21.15.184:3300/0 conn(0x7fafe8162810 msgr2=0x7fafe8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 --2- 172.21.15.40:0/3179848149 >> v2:172.21.15.184:3300/0 conn(0x7fafe8162810 0x7fafe8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 >> v1:172.21.15.184:6789/0 conn(0x7fafe80b2340 legacy=0x7fafe80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe80d01b0 con 0x7fafe80d1b30 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fafe00019f0 con 0x7fafe80d1b30 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 >> v1:172.21.15.40:6789/0 conn(0x7fafe80d1b30 legacy=0x7fafe8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd00039b0 0x7fafd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd0004800 0x7fafd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd00051c0 0x7fafd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafe80d0750 con 0x7fafd00051c0 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafd000c060 con 0x7fafd00039b0 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafd000c260 con 0x7fafd0004800 2024-04-26T21:42:49.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafecba6640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd00051c0 0x7fafd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe7fff640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd00039b0 0x7fafd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.726+0000 7fafe77fe640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd0004800 0x7fafd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafecba6640 1 -- 172.21.15.40:0/3179848149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd0004800 msgr2=0x7fafd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafecba6640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd0004800 0x7fafd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafecba6640 1 -- 172.21.15.40:0/3179848149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd00039b0 msgr2=0x7fafd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafecba6640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd00039b0 0x7fafd0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafecba6640 1 -- 172.21.15.40:0/3179848149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd0003520 con 0x7fafd00051c0 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafecba6640 1 -- 172.21.15.40:0/3179848149 reap_dead start 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafecba6640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd00051c0 0x7fafd000b9c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fafe000a860 tx=0x7fafd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b006c5fed9497a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000a480 con 0x7fafd00051c0 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe002f070 con 0x7fafd00051c0 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.727+0000 7fafe6ffd640 1 -- 172.21.15.40:0/3179848149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe002bca0 con 0x7fafd00051c0 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.728+0000 7fafee609640 1 -- 172.21.15.40:0/3179848149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd00051c0 msgr2=0x7fafd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.728+0000 7fafee609640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd00051c0 0x7fafd000b9c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fafe000a860 tx=0x7fafd0005830 comp rx=0 tx=0).stop 2024-04-26T21:42:49.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.728+0000 7fafee609640 1 -- 172.21.15.40:0/3179848149 shutdown_connections 2024-04-26T21:42:49.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.728+0000 7fafee609640 1 --2- 172.21.15.40:0/3179848149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd00051c0 0x7fafd000b9c0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.728+0000 7fafee609640 1 -- 172.21.15.40:0/3179848149 >> 172.21.15.40:0/3179848149 conn(0x7fafe80d2da0 msgr2=0x7fafe80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:49.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.728+0000 7fafee609640 1 -- 172.21.15.40:0/3179848149 shutdown_connections 2024-04-26T21:42:49.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.728+0000 7fafee609640 1 -- 172.21.15.40:0/3179848149 wait complete. 2024-04-26T21:42:49.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.729+0000 7fafee609640 1 Processor -- start 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.729+0000 7fafee609640 1 -- start start 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafee609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd00051c0 0x7fafe80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafee609640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe80cb7e0 0x7fafe8154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafee609640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe80b0b00 0x7fafe80b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafee609640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafe80d1fc0 con 0x7fafe80b0b00 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafee609640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafe80d16a0 con 0x7fafe80cb7e0 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafee609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafe80d2140 con 0x7fafd00051c0 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafecba6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe80b0b00 0x7fafe80b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafecba6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe80b0b00 0x7fafe80b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37022/0 (socket says 172.21.15.40:37022) 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafecba6640 1 -- 172.21.15.40:0/3533070984 learned_addr learned my addr 172.21.15.40:0/3533070984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafe77fe640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe80cb7e0 0x7fafe8154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafe7fff640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd00051c0 0x7fafe80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafecba6640 1 -- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd00051c0 msgr2=0x7fafe80c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafecba6640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd00051c0 0x7fafe80c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafecba6640 1 -- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe80cb7e0 msgr2=0x7fafe8154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafecba6640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe80cb7e0 0x7fafe8154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafecba6640 1 -- 172.21.15.40:0/3533070984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe80ca450 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.730+0000 7fafe7fff640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd00051c0 0x7fafe80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.731+0000 7fafecba6640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe80b0b00 0x7fafe80b0f80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fafe000a980 tx=0x7fafe002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.731+0000 7fafe4ff9640 1 -- 172.21.15.40:0/3533070984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000a480 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.731+0000 7fafe4ff9640 1 -- 172.21.15.40:0/3533070984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe0034670 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.731+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe80ca730 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.731+0000 7fafe4ff9640 1 -- 172.21.15.40:0/3533070984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0034a40 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.731+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe80cacf0 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.732+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe80694f0 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.732+0000 7fafe4ff9640 1 -- 172.21.15.40:0/3533070984 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafe0034be0 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.733+0000 7fafe4ff9640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafd406c920 0x7fafd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.733+0000 7fafe4ff9640 1 -- 172.21.15.40:0/3533070984 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6359+0+0 (secure 0 0 0) 0x7fafe00abbb0 con 0x7fafe80b0b00 2024-04-26T21:42:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.733+0000 7fafe7fff640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafd406c920 0x7fafd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.734+0000 7fafe7fff640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafd406c920 0x7fafd406ede0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009a40 tx=0x7fafd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:49.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.737+0000 7fafe4ff9640 1 -- 172.21.15.40:0/3533070984 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fafe0046060 con 0x7fafe80b0b00 2024-04-26T21:42:49.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:49.995+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fafe806d940 con 0x7fafe80b0b00 2024-04-26T21:42:50.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.615+0000 7fafe4ff9640 1 -- 172.21.15.40:0/3533070984 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v107) ==== 101+0+0 (secure 0 0 0) 0x7fafe0079d40 con 0x7fafe80b0b00 2024-04-26T21:42:50.616 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.617+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafd406c920 msgr2=0x7fafd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.617+0000 7fafee609640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafd406c920 0x7fafd406ede0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009a40 tx=0x7fafd8008040 comp rx=0 tx=0).stop 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.617+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe80b0b00 msgr2=0x7fafe80b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.617+0000 7fafee609640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe80b0b00 0x7fafe80b0f80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fafe000a980 tx=0x7fafe002d070 comp rx=0 tx=0).stop 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.618+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 shutdown_connections 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.618+0000 7fafee609640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafd406c920 0x7fafd406ede0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.618+0000 7fafee609640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafe80b0b00 0x7fafe80b0f80 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.618+0000 7fafee609640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe80cb7e0 0x7fafe8154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.618+0000 7fafee609640 1 --2- 172.21.15.40:0/3533070984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd00051c0 0x7fafe80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.618+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 >> 172.21.15.40:0/3533070984 conn(0x7fafe80d2da0 msgr2=0x7fafe80cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.618+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 shutdown_connections 2024-04-26T21:42:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.618+0000 7fafee609640 1 -- 172.21.15.40:0/3533070984 wait complete. 2024-04-26T21:42:50.632 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:42:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.677+0000 7f81f6863f40 1 Processor -- start 2024-04-26T21:42:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.677+0000 7f81f6863f40 1 -- start start 2024-04-26T21:42:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f6863f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624dffc2f60 0x5624e0037e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f6863f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624e0039d60 0x5624e0032f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f6863f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 0x5624e0038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f6863f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5624e003a140 con 0x5624dffc2f60 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f6863f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5624dff23e40 con 0x5624e0057610 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f6863f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5624dffc38b0 con 0x5624e0039d60 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624dffc2f60 0x5624e0037e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624dffc2f60 0x5624e0037e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37042/0 (socket says 172.21.15.40:37042) 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 -- 172.21.15.40:0/2953602917 learned_addr learned my addr 172.21.15.40:0/2953602917 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f4b5e640 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 0x5624e0038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 -- 172.21.15.40:0/2953602917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624e0039d60 msgr2=0x5624e0032f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624e0039d60 0x5624e0032f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 -- 172.21.15.40:0/2953602917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 msgr2=0x5624e0038dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 0x5624e0038dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 -- 172.21.15.40:0/2953602917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624e00366b0 con 0x5624dffc2f60 2024-04-26T21:42:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f4b5e640 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 0x5624e0038dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.678+0000 7f81f435d640 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624dffc2f60 0x5624e0037e60 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f81e400a770 tx=0x7f81e400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac080ab3d7e57c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f2b5a640 1 -- 172.21.15.40:0/2953602917 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4018070 con 0x5624dffc2f60 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f2b5a640 1 -- 172.21.15.40:0/2953602917 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e4007600 con 0x5624dffc2f60 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f2b5a640 1 -- 172.21.15.40:0/2953602917 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e40079b0 con 0x5624dffc2f60 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 -- 172.21.15.40:0/2953602917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624dffc2f60 msgr2=0x5624e0037e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624dffc2f60 0x5624e0037e60 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f81e400a770 tx=0x7f81e400a860 comp rx=0 tx=0).stop 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 -- 172.21.15.40:0/2953602917 shutdown_connections 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 0x5624e0038dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624e0039d60 0x5624e0032f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2953602917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624dffc2f60 0x5624e0037e60 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 -- 172.21.15.40:0/2953602917 >> 172.21.15.40:0/2953602917 conn(0x5624dff7aab0 msgr2=0x5624dffaa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 -- 172.21.15.40:0/2953602917 shutdown_connections 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.679+0000 7f81f6863f40 1 -- 172.21.15.40:0/2953602917 wait complete. 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.680+0000 7f81f6863f40 1 Processor -- start 2024-04-26T21:42:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.680+0000 7f81f6863f40 1 -- start start 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f6863f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624dffc2f60 0x5624e004d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f6863f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624e0039d60 0x5624e0029940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f6863f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 0x5624e0029e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f6863f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5624dffa5810 con 0x5624e0039d60 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f6863f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5624e004d750 con 0x5624e0057610 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f6863f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5624e00383a0 con 0x5624dffc2f60 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624e0039d60 0x5624e0029940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624e0039d60 0x5624e0029940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37046/0 (socket says 172.21.15.40:37046) 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 learned_addr learned my addr 172.21.15.40:0/2630161391 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f435d640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624dffc2f60 0x5624e004d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f4b5e640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 0x5624e0029e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624dffc2f60 msgr2=0x5624e004d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624dffc2f60 0x5624e004d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 msgr2=0x5624e0029e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624e0057610 0x5624e0029e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624e002de80 con 0x5624e0039d60 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f435d640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624dffc2f60 0x5624e004d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f3b5c640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624e0039d60 0x5624e0029940 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f81e8025b40 tx=0x7f81e8045840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f1b58640 1 -- 172.21.15.40:0/2630161391 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8059040 con 0x5624e0039d60 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f1b58640 1 -- 172.21.15.40:0/2630161391 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e80547f0 con 0x5624e0039d60 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5624e002e160 con 0x5624e0039d60 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f1b58640 1 -- 172.21.15.40:0/2630161391 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8054be0 con 0x5624e0039d60 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.681+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624e002e720 con 0x5624e0039d60 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.683+0000 7f81f1b58640 1 -- 172.21.15.40:0/2630161391 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81e805f470 con 0x5624e0039d60 2024-04-26T21:42:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.683+0000 7f81f1b58640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f81e006b300 0x7f81e006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.683+0000 7f81f1b58640 1 -- 172.21.15.40:0/2630161391 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6359+0+0 (secure 0 0 0) 0x7f81e80cec10 con 0x5624e0039d60 2024-04-26T21:42:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.684+0000 7f81f435d640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f81e006b300 0x7f81e006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.687 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:42:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.684+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5624e002ef20 0x5624e0030440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.684+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624dff7b5a0 con 0x5624e002ef20 2024-04-26T21:42:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.684+0000 7f81f4b5e640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5624e002ef20 0x5624e0030440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.684+0000 7f81f435d640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f81e006b300 0x7f81e006d7c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f81e4016040 tx=0x7f81e400c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.684+0000 7f81f4b5e640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5624e002ef20 0x5624e0030440 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.685+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v97'73 uv48 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.685+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e0030980 con 0x5624e002ef20 2024-04-26T21:42:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.687+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v97'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.687+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0030d50 con 0x5624e002ef20 2024-04-26T21:42:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.688+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v97'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.689 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:42:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.688+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00310f0 con 0x5624e002ef20 2024-04-26T21:42:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.689+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v97'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.689+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7e20 con 0x5624e002ef20 2024-04-26T21:42:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.690+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v97'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.690+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f72c0 con 0x5624e002ef20 2024-04-26T21:42:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.691+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v97'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.691+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f76b0 con 0x5624e002ef20 2024-04-26T21:42:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.692+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v97'79 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.692+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.694+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v97'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.694+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.695+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v97'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.696 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:42:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.695+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.696+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v97'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.696+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.697+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v97'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.697+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.698+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v97'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.698+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.699+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v97'85 uv59 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.699+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.700+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v97'86 uv86 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.700+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.702+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v97'87 uv87 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.703 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:42:50.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.702+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.703+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v97'88 uv88 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.703+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.704+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v97'89 uv89 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.704+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f7a50 con 0x5624e002ef20 2024-04-26T21:42:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.705+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v97'90 uv90 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.705+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.706+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v97'91 uv91 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.706+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.707+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v97'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.708 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:42:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.707+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f91d0 con 0x5624e002ef20 2024-04-26T21:42:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.708+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v97'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.708+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f91d0 con 0x5624e002ef20 2024-04-26T21:42:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.710+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v97'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.710+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.711+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v97'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.711+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.712+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 603.00000000 [create] v97'96 uv71 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.712+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.713+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 603.00000000 [delete] v97'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.713+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.715 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:42:50.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.714+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v97'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.714+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f91d0 con 0x5624e002ef20 2024-04-26T21:42:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.715+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v97'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.715+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f91d0 con 0x5624e002ef20 2024-04-26T21:42:50.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.716+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v97'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.716+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00f81c0 con 0x5624e002ef20 2024-04-26T21:42:50.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.717+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v97'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e002ef20 2024-04-26T21:42:50.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.717+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5624e00f8840 0x5624e00fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.717+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fb170 con 0x5624e00f8840 2024-04-26T21:42:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.718+0000 7f81f435d640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5624e00f8840 0x5624e00fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.718+0000 7f81f435d640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5624e00f8840 0x5624e00fac30 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.719+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(30 1.00000000.inode [create] v97'67 uv49 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.719+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e00fb510 con 0x5624e00f8840 2024-04-26T21:42:50.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.721+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(31 1.00000000.inode [delete] v97'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.721+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fb8b0 con 0x5624e00f8840 2024-04-26T21:42:50.722 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:42:50.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.722+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(32 1.00000000.inode [create] v97'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.722+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fc100 con 0x5624e00f8840 2024-04-26T21:42:50.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.723+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v97'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.723+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fc4a0 con 0x5624e00f8840 2024-04-26T21:42:50.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.724+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v97'71 uv71 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.724+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fc840 con 0x5624e00f8840 2024-04-26T21:42:50.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.725+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(35 200.00000000 [create] v97'72 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.725+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.726+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(36 200.00000000 [delete] v97'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.726+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.728 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:42:50.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.728+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(37 200.00000000 [create] v97'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.728+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.729+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v97'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.729+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.730+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v97'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.730+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.731+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(40 200.00000001 [create] v97'77 uv59 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.731+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.732+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(41 200.00000001 [delete] v97'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.732+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.734 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:42:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.733+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(42 200.00000001 [create] v97'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.733+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01012a0 con 0x5624e00f8840 2024-04-26T21:42:50.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.734+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(43 200.00000001 [write 0~18075] v97'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.734+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01012a0 con 0x5624e00f8840 2024-04-26T21:42:50.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.735+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v97'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.735+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.736+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(45 606.00000000 [create] v97'82 uv65 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.736+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.737+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(46 606.00000000 [delete] v97'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.737+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.738+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(47 606.00000000 [create] v97'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.739 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:42:50.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.739+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fdbf0 con 0x5624e00f8840 2024-04-26T21:42:50.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.740+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v97'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.740+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fdbf0 con 0x5624e00f8840 2024-04-26T21:42:50.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.743+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v97'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.743+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00fcbe0 con 0x5624e00f8840 2024-04-26T21:42:50.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.744+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v97'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e00f8840 2024-04-26T21:42:50.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.744+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5624e00fd0f0 0x5624e00ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.744+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e00ffaf0 con 0x5624e00fd0f0 2024-04-26T21:42:50.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.744+0000 7f81f3b5c640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5624e00fd0f0 0x5624e00ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.744+0000 7f81f3b5c640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5624e00fd0f0 0x5624e00ff4e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.746+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(51 607.00000000 [create] v97'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.746+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e00fffc0 con 0x5624e00fd0f0 2024-04-26T21:42:50.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.749+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(52 607.00000000 [delete] v97'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.749+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01004a0 con 0x5624e00fd0f0 2024-04-26T21:42:50.753 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:42:50.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.752+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(53 607.00000000 [create] v97'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.752+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0101d50 con 0x5624e00fd0f0 2024-04-26T21:42:50.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.755+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v97'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.755+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01020f0 con 0x5624e00fd0f0 2024-04-26T21:42:50.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.758+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v97'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.758+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0100d40 con 0x5624e00fd0f0 2024-04-26T21:42:50.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.759+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v97'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.759+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5624e0101640 0x5624e01044a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.759+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01049e0 con 0x5624e0101640 2024-04-26T21:42:50.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.759+0000 7f81f4b5e640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5624e0101640 0x5624e01044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.759+0000 7f81f4b5e640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5624e0101640 0x5624e01044a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.760+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v97'14 uv13 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.760+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e01049e0 con 0x5624e0101640 2024-04-26T21:42:50.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.762+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v97'15 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.762+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0104a60 con 0x5624e0101640 2024-04-26T21:42:50.765 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:42:50.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.765+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v97'16 uv16 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.765+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0100990 con 0x5624e0101640 2024-04-26T21:42:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.768+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v97'17 uv17 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.768+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5624e0105650 0x5624e0107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.768+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108050 con 0x5624e0105650 2024-04-26T21:42:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.768+0000 7f81f435d640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5624e0105650 0x5624e0107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.768+0000 7f81f435d640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5624e0105650 0x5624e0107a40 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.770+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(61 608.00000000 [create] v97'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.770+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.773+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(62 608.00000000 [delete] v97'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.773+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.775+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(63 608.00000000 [create] v97'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.775 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:42:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.775+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0109430 con 0x5624e0105650 2024-04-26T21:42:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.776+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v97'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.776+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0109430 con 0x5624e0105650 2024-04-26T21:42:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.779+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v97'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.779+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.781+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v97'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.781+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.782+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(67 604.00000000 [create] v97'26 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.782+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.783+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(68 604.00000000 [delete] v97'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.784+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.785 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:42:50.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.784+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(69 604.00000000 [create] v97'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.784+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0109430 con 0x5624e0101640 2024-04-26T21:42:50.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.785+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v97'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.785+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0109430 con 0x5624e0101640 2024-04-26T21:42:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.786+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v97'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.786+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.787+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v97'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.787+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.788+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(73 500.00000000 [create] v97'32 uv25 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.788+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.789+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(74 500.00000000 [delete] v97'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.789+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.790 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:42:50.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.790+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(75 500.00000000 [create] v97'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.790+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.791+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v97'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.791+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.792+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v97'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.792+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.793+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(78 mds_snaptable [create] v97'12 uv11 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.793+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.794+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(79 mds_snaptable [delete] v97'13 uv13 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.794+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.795 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:42:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.795+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(80 mds_snaptable [create] v97'14 uv14 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.795+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.796+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(81 mds_snaptable [write 0~70] v97'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.796+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0101640 2024-04-26T21:42:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.797+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v97'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f81ec00e5c0 con 0x5624e0101640 2024-04-26T21:42:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.797+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.798+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(83 605.00000000 [create] v97'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.798+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.799+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(84 605.00000000 [delete] v97'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.799+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.802+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(85 605.00000000 [create] v97'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.803 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:42:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.802+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0109430 con 0x5624e0105650 2024-04-26T21:42:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.805+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v97'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.805+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0109430 con 0x5624e0105650 2024-04-26T21:42:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.806+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v97'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.806+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.809+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(88 605.00000000 [omap-set-header] v97'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.809+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.811+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(89 mds0_inotable [create] v97'24 uv18 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.811+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.814+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(90 mds0_inotable [delete] v97'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.814+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108420 con 0x5624e0105650 2024-04-26T21:42:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.817+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(91 mds0_inotable [create] v97'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.818 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:42:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.817+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108830 con 0x5624e0105650 2024-04-26T21:42:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.819+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(92 mds0_inotable [write 0~34] v97'27 uv27 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.819+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01084e0 con 0x5624e0105650 2024-04-26T21:42:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.820+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v97'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.820+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01084c0 con 0x5624e0105650 2024-04-26T21:42:50.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.823+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(94 100.00000000 [create] v97'29 uv23 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.823+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e01084c0 con 0x5624e0105650 2024-04-26T21:42:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.826+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(95 100.00000000 [delete] v97'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.826+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01084c0 con 0x5624e0105650 2024-04-26T21:42:50.828 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:42:50.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.827+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(96 100.00000000 [create] v97'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.827+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e01084e0 con 0x5624e0105650 2024-04-26T21:42:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.828+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(97 100.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.828+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e0108970 con 0x5624e0105650 2024-04-26T21:42:50.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.829+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v97'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.830+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5624e0109010 0x5624e010b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.830+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b940 con 0x5624e0109010 2024-04-26T21:42:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.830+0000 7f81f3b5c640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5624e0109010 0x5624e010b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.830+0000 7f81f3b5c640 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5624e0109010 0x5624e010b400 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.831+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(99 mds0_sessionmap [create] v97'14 uv12 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e0109010 2024-04-26T21:42:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.831+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e010b940 con 0x5624e0109010 2024-04-26T21:42:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.832+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v97'15 uv15 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e0109010 2024-04-26T21:42:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.832+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b940 con 0x5624e0109010 2024-04-26T21:42:50.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.835+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(101 mds0_sessionmap [create] v97'16 uv16 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e0109010 2024-04-26T21:42:50.836 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:42:50.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.835+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e0109010 2024-04-26T21:42:50.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.838+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v97'17 uv17 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e0109010 2024-04-26T21:42:50.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.838+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e0105650 2024-04-26T21:42:50.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.839+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(103 609.00000000 [create] v97'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.839+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e0105650 2024-04-26T21:42:50.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.841+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(104 609.00000000 [delete] v97'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.841+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e0105650 2024-04-26T21:42:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.843+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(105 609.00000000 [create] v97'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.844 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:42:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.843+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e0105650 2024-04-26T21:42:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.844+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v97'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.844+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e0105650 2024-04-26T21:42:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.847+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v97'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.847+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e0105650 2024-04-26T21:42:50.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.848+0000 7f81f435d640 1 -- 172.21.15.40:0/2630161391 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(108 609.00000000 [omap-set-header] v97'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e4007d30 con 0x5624e0105650 2024-04-26T21:42:50.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.848+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e00fd0f0 2024-04-26T21:42:50.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.849+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v97'12 uv11 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.849+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e00fd0f0 2024-04-26T21:42:50.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.850+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v97'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.850+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010b9c0 con 0x5624e00fd0f0 2024-04-26T21:42:50.852 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:42:50.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.851+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v97'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.851+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bdb0 con 0x5624e00fd0f0 2024-04-26T21:42:50.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.852+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v97'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.852+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bdb0 con 0x5624e00fd0f0 2024-04-26T21:42:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.853+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v97'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.854+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bb40 con 0x5624e00fd0f0 2024-04-26T21:42:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.855+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v97'33 uv24 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.855+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e010bb40 con 0x5624e00fd0f0 2024-04-26T21:42:50.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.856+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v97'34 uv34 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.856+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bf30 con 0x5624e00fd0f0 2024-04-26T21:42:50.857 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:42:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.857+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v97'35 uv35 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.857+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bba0 con 0x5624e00fd0f0 2024-04-26T21:42:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.858+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v97'36 uv36 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.858+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bba0 con 0x5624e00fd0f0 2024-04-26T21:42:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.859+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v97'37 uv37 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.859+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bb50 con 0x5624e00fd0f0 2024-04-26T21:42:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.860+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v97'38 uv31 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.860+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5624e010bb50 con 0x5624e00fd0f0 2024-04-26T21:42:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.861+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v97'39 uv39 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.861+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bb50 con 0x5624e00fd0f0 2024-04-26T21:42:50.862 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:42:50.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.861+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v97'40 uv40 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.861+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bb50 con 0x5624e00fd0f0 2024-04-26T21:42:50.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.862+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v97'41 uv41 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.862+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bb30 con 0x5624e00fd0f0 2024-04-26T21:42:50.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.863+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v97'42 uv42 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.863+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bb30 con 0x5624e00fd0f0 2024-04-26T21:42:50.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.864+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v97'43 uv43 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.864+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5624e010bf70 con 0x5624e00fd0f0 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f3b5c640 1 -- 172.21.15.40:0/2630161391 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v97'44 uv44 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f81e8065020 con 0x5624e00fd0f0 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5624e00f8840 msgr2=0x5624e00fac30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5624e00f8840 0x5624e00fac30 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5624e002ef20 msgr2=0x5624e0030440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5624e002ef20 0x5624e0030440 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5624e0101640 msgr2=0x5624e01044a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5624e0101640 0x5624e01044a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5624e0109010 msgr2=0x5624e010b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5624e0109010 0x5624e010b400 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5624e00fd0f0 msgr2=0x5624e00ff4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5624e00fd0f0 0x5624e00ff4e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5624e0105650 msgr2=0x5624e0107a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5624e0105650 0x5624e0107a40 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 reap_dead start 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f4b5e640 1 -- 172.21.15.40:0/2630161391 reap_dead start 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f81e006b300 msgr2=0x7f81e006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f81e006b300 0x7f81e006d7c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f81e4016040 tx=0x7f81e400c000 comp rx=0 tx=0).stop 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624e0039d60 msgr2=0x5624e0029940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624e0039d60 0x5624e0029940 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f81e8025b40 tx=0x7f81e8045840 comp rx=0 tx=0).stop 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 shutdown_connections 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f81e006b300 0x7f81e006d7c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 --2- 172.21.15.40:0/2630161391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5624e0039d60 0x5624e0029940 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.865+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 >> 172.21.15.40:0/2630161391 conn(0x5624dff7aab0 msgr2=0x5624dff7ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:50.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.866+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 shutdown_connections 2024-04-26T21:42:50.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.866+0000 7f81f6863f40 1 -- 172.21.15.40:0/2630161391 wait complete. 2024-04-26T21:42:50.870 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000.inode 2024-04-26T21:42:50.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.951+0000 7f5a7dc43f40 1 Processor -- start 2024-04-26T21:42:50.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.951+0000 7f5a7dc43f40 1 -- start start 2024-04-26T21:42:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.951+0000 7f5a7dc43f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 0x5601354fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.951+0000 7f5a7dc43f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x5601354530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.951+0000 7f5a7dc43f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 0x560135504a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.951+0000 7f5a7dc43f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56013545dc50 con 0x5601355014e0 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.951+0000 7f5a7dc43f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5601353d4e40 con 0x5601355018c0 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.951+0000 7f5a7dc43f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56013545d690 con 0x560135475140 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x5601354530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x5601354530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37052/0 (socket says 172.21.15.40:37052) 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 -- 172.21.15.40:0/3726831066 learned_addr learned my addr 172.21.15.40:0/3726831066 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7b73d640 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 0x5601354fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7bf3e640 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 0x560135504a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 -- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 msgr2=0x5601354fdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 0x5601354fdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 -- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 msgr2=0x560135504a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 0x560135504a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 -- 172.21.15.40:0/3726831066 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601354fec60 con 0x5601355014e0 2024-04-26T21:42:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7b73d640 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 0x5601354fdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7bf3e640 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 0x560135504a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a7af3c640 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x5601354530d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000b950 tx=0x7f5a7000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa9c1bd8537a3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a79f3a640 1 -- 172.21.15.40:0/3726831066 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000acd0 con 0x5601355014e0 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a79f3a640 1 -- 172.21.15.40:0/3726831066 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a70015470 con 0x5601355014e0 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.952+0000 7f5a79f3a640 1 -- 172.21.15.40:0/3726831066 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70015820 con 0x5601355014e0 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/3726831066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 msgr2=0x5601354530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x5601354530d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000b950 tx=0x7f5a7000be10 comp rx=0 tx=0).stop 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/3726831066 shutdown_connections 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 0x560135504a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x5601354530d0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 --2- 172.21.15.40:0/3726831066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 0x5601354fdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/3726831066 >> 172.21.15.40:0/3726831066 conn(0x5601354578a0 msgr2=0x56013542af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/3726831066 shutdown_connections 2024-04-26T21:42:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/3726831066 wait complete. 2024-04-26T21:42:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 Processor -- start 2024-04-26T21:42:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.953+0000 7f5a7dc43f40 1 -- start start 2024-04-26T21:42:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7dc43f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 0x56013542e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7dc43f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x560135456760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7dc43f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 0x5601355031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7dc43f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5601354fe680 con 0x5601355014e0 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7dc43f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560135453610 con 0x5601355018c0 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7dc43f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560135504490 con 0x560135475140 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7af3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x560135456760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7af3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x560135456760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37056/0 (socket says 172.21.15.40:37056) 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7af3c640 1 -- 172.21.15.40:0/1793870234 learned_addr learned my addr 172.21.15.40:0/1793870234 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7b73d640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 0x56013542e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.954+0000 7f5a7bf3e640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 0x5601355031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a7af3c640 1 -- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 msgr2=0x56013542e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a7af3c640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560135475140 0x56013542e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a7af3c640 1 -- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 msgr2=0x5601355031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a7af3c640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601355018c0 0x5601355031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a7af3c640 1 -- 172.21.15.40:0/1793870234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601354f87d0 con 0x5601355014e0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a7af3c640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x560135456760 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f5a70012000 tx=0x7f5a70003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a78f38640 1 -- 172.21.15.40:0/1793870234 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70025070 con 0x5601355014e0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a78f38640 1 -- 172.21.15.40:0/1793870234 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a70005660 con 0x5601355014e0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5601354f83b0 con 0x5601355014e0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a78f38640 1 -- 172.21.15.40:0/1793870234 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000b5a0 con 0x5601355014e0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.955+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56013542ee70 con 0x5601355014e0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.956+0000 7f5a78f38640 1 -- 172.21.15.40:0/1793870234 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a70028070 con 0x5601355014e0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.957+0000 7f5a78f38640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5a6806b300 0x7f5a6806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.957+0000 7f5a78f38640 1 -- 172.21.15.40:0/1793870234 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6359+0+0 (secure 0 0 0) 0x7f5a7008c650 con 0x5601355014e0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.957+0000 7f5a7dc43f40 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560135503700 0x5601354f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.957+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x56013542c570 con 0x560135503700 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.957+0000 7f5a7b73d640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5a6806b300 0x7f5a6806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.957+0000 7f5a7bf3e640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560135503700 0x5601354f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.957+0000 7f5a7bf3e640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560135503700 0x5601354f7a90 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.958+0000 7f5a7b73d640 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5a6806b300 0x7f5a6806d7c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c056000 tx=0x7f5a6c050000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7bf3e640 1 -- 172.21.15.40:0/1793870234 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 1.00000000.inode [delete] v97'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5a74004540 con 0x560135503700 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560135503700 msgr2=0x5601354f7a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560135503700 0x5601354f7a90 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5a6806b300 msgr2=0x7f5a6806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5a6806b300 0x7f5a6806d7c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c056000 tx=0x7f5a6c050000 comp rx=0 tx=0).stop 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 msgr2=0x560135456760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 --2- 172.21.15.40:0/1793870234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5601355014e0 0x560135456760 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f5a70012000 tx=0x7f5a70003020 comp rx=0 tx=0).stop 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7bf3e640 1 -- 172.21.15.40:0/1793870234 reap_dead start 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 shutdown_connections 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 >> 172.21.15.40:0/1793870234 conn(0x5601354578a0 msgr2=0x560135473730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 shutdown_connections 2024-04-26T21:42:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:50.959+0000 7f5a7dc43f40 1 -- 172.21.15.40:0/1793870234 wait complete. 2024-04-26T21:42:50.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:42:51.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.108+0000 7f220f209640 1 Processor -- start 2024-04-26T21:42:51.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.108+0000 7f220f209640 1 -- start start 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220f209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 0x7f22080d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220f209640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2208070350 0x7f2208070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220f209640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2208071090 0x7f220815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220f209640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2208071500 con 0x7f2208070350 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220f209640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f22080afab0 con 0x7f2208071090 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220f209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f220806dd20 con 0x7f22080cfde0 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f21fffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2208070350 0x7f2208070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f21fffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2208070350 0x7f2208070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37060/0 (socket says 172.21.15.40:37060) 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f21fffff640 1 -- 172.21.15.40:0/3809450738 learned_addr learned my addr 172.21.15.40:0/3809450738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220d7a6640 1 --2- 172.21.15.40:0/3809450738 >> v2:172.21.15.184:3300/0 conn(0x7f2208071090 0x7f220815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220cfa5640 1 --2- 172.21.15.40:0/3809450738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 0x7f22080d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f21fffff640 1 -- 172.21.15.40:0/3809450738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 msgr2=0x7f22080d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f21fffff640 1 --2- 172.21.15.40:0/3809450738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 0x7f22080d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f21fffff640 1 -- 172.21.15.40:0/3809450738 >> v2:172.21.15.184:3300/0 conn(0x7f2208071090 msgr2=0x7f220815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f21fffff640 1 --2- 172.21.15.40:0/3809450738 >> v2:172.21.15.184:3300/0 conn(0x7f2208071090 0x7f220815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f21fffff640 1 -- 172.21.15.40:0/3809450738 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22080d2390 con 0x7f2208070350 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.109+0000 7f220d7a6640 1 --2- 172.21.15.40:0/3809450738 >> v2:172.21.15.184:3300/0 conn(0x7f2208071090 0x7f220815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.110+0000 7f21fffff640 1 --2- 172.21.15.40:0/3809450738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f2208070b50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f22080732e0 tx=0x7f21f802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaad617d4f2e93e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.110+0000 7f21ff7fe640 1 -- 172.21.15.40:0/3809450738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8037070 con 0x7f2208070350 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.110+0000 7f21ff7fe640 1 -- 172.21.15.40:0/3809450738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f8032440 con 0x7f2208070350 2024-04-26T21:42:51.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.110+0000 7f21ff7fe640 1 -- 172.21.15.40:0/3809450738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8032810 con 0x7f2208070350 2024-04-26T21:42:51.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 -- 172.21.15.40:0/3809450738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 msgr2=0x7f2208070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 --2- 172.21.15.40:0/3809450738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f2208070b50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f22080732e0 tx=0x7f21f802a140 comp rx=0 tx=0).stop 2024-04-26T21:42:51.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 -- 172.21.15.40:0/3809450738 shutdown_connections 2024-04-26T21:42:51.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 --2- 172.21.15.40:0/3809450738 >> v2:172.21.15.184:3300/0 conn(0x7f2208071090 0x7f220815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 --2- 172.21.15.40:0/3809450738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f2208070b50 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 --2- 172.21.15.40:0/3809450738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 0x7f22080d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 -- 172.21.15.40:0/3809450738 >> 172.21.15.40:0/3809450738 conn(0x7f22080d2d80 msgr2=0x7f22080d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 -- 172.21.15.40:0/3809450738 shutdown_connections 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.111+0000 7f220f209640 1 -- 172.21.15.40:0/3809450738 wait complete. 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.112+0000 7f220f209640 1 Processor -- start 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.112+0000 7f220f209640 1 -- start start 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220f209640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f22080c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220f209640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2208071090 0x7f220815bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220f209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 0x7f220809f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220f209640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f220806d9b0 con 0x7f2208070350 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220f209640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f220806d6d0 con 0x7f2208071090 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220f209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22080d20a0 con 0x7f22080cfde0 2024-04-26T21:42:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220cfa5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f22080c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220cfa5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f22080c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37064/0 (socket says 172.21.15.40:37064) 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220cfa5640 1 -- 172.21.15.40:0/12414907 learned_addr learned my addr 172.21.15.40:0/12414907 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f21fffff640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2208071090 0x7f220815bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220d7a6640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 0x7f220809f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220cfa5640 1 -- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 msgr2=0x7f220809f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220cfa5640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 0x7f220809f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220cfa5640 1 -- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2208071090 msgr2=0x7f220815bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220cfa5640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2208071090 0x7f220815bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.113+0000 7f220cfa5640 1 -- 172.21.15.40:0/12414907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22080c88d0 con 0x7f2208070350 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.114+0000 7f220cfa5640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f22080c82e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f220400e9b0 tx=0x7f220400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.114+0000 7f21fd7fa640 1 -- 172.21.15.40:0/12414907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204017070 con 0x7f2208070350 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.114+0000 7f21fd7fa640 1 -- 172.21.15.40:0/12414907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220400c940 con 0x7f2208070350 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.114+0000 7f21fd7fa640 1 -- 172.21.15.40:0/12414907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220400cd30 con 0x7f2208070350 2024-04-26T21:42:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.114+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22080c8bb0 con 0x7f2208070350 2024-04-26T21:42:51.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.114+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22080c9170 con 0x7f2208070350 2024-04-26T21:42:51.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.115+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22080d2740 con 0x7f2208070350 2024-04-26T21:42:51.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.115+0000 7f21fd7fa640 1 -- 172.21.15.40:0/12414907 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f220401a070 con 0x7f2208070350 2024-04-26T21:42:51.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.116+0000 7f21fd7fa640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f406c8f0 0x7f21f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.116+0000 7f21fd7fa640 1 -- 172.21.15.40:0/12414907 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6359+0+0 (secure 0 0 0) 0x7f220408c030 con 0x7f2208070350 2024-04-26T21:42:51.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.116+0000 7f21fffff640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f406c8f0 0x7f21f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.116+0000 7f21fffff640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f406c8f0 0x7f21f406edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f21f8034000 tx=0x7f21f802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.120+0000 7f21fd7fa640 1 -- 172.21.15.40:0/12414907 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22080d2740 con 0x7f2208070350 2024-04-26T21:42:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.345+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f22080694f0 con 0x7f2208070350 2024-04-26T21:42:51.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.654+0000 7f21fd7fa640 1 -- 172.21.15.40:0/12414907 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) ==== 162+0+0 (secure 0 0 0) 0x7f2204059b80 con 0x7f2208070350 2024-04-26T21:42:51.655 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:42:51.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f406c8f0 msgr2=0x7f21f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f406c8f0 0x7f21f406edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f21f8034000 tx=0x7f21f802e000 comp rx=0 tx=0).stop 2024-04-26T21:42:51.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 msgr2=0x7f22080c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f22080c82e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f220400e9b0 tx=0x7f220400edb0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 shutdown_connections 2024-04-26T21:42:51.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21f406c8f0 0x7f21f406edb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22080cfde0 0x7f220809f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2208071090 0x7f220815bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 --2- 172.21.15.40:0/12414907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208070350 0x7f22080c82e0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 >> 172.21.15.40:0/12414907 conn(0x7f22080d2d80 msgr2=0x7f22080b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:51.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 shutdown_connections 2024-04-26T21:42:51.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.658+0000 7f220f209640 1 -- 172.21.15.40:0/12414907 wait complete. 2024-04-26T21:42:51.668 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:42:51.668+0000 7ff955f10640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-04-26T21:42:51.668 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:42:51.668+0000 7ff955f10640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-26T21:42:51.669 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:51.668+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:42:51.669 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:51.668+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:42:51.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:51.678 INFO:tasks.ceph.mds.a.smithi040.stderr: -16> 2024-04-26T21:42:51.668+0000 7ff955f10640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-04-26T21:42:51.678 INFO:tasks.ceph.mds.a.smithi040.stderr: -14> 2024-04-26T21:42:51.668+0000 7ff955f10640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-26T21:42:51.704 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:42:51.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.778+0000 7f619b807640 1 Processor -- start 2024-04-26T21:42:51.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.778+0000 7f619b807640 1 -- start start 2024-04-26T21:42:51.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f619b807640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f619415f1d0 0x7f619406f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f619b807640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f619406c240 0x7f619406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f619b807640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61940b4480 con 0x7f61940726e0 2024-04-26T21:42:51.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f619b807640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f619406cbe0 con 0x7f619415f1d0 2024-04-26T21:42:51.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f619b807640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f61940ae8c0 con 0x7f619406c240 2024-04-26T21:42:51.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f61995a3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f619415f1d0 0x7f619406f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f61995a3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f619415f1d0 0x7f619406f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37074/0 (socket says 172.21.15.40:37074) 2024-04-26T21:42:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f61995a3640 1 -- 172.21.15.40:0/2692606483 learned_addr learned my addr 172.21.15.40:0/2692606483 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f6198da2640 1 --2- 172.21.15.40:0/2692606483 >> v2:172.21.15.184:3300/0 conn(0x7f619406c240 0x7f619406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f61995a3640 1 -- 172.21.15.40:0/2692606483 >> v2:172.21.15.184:3300/0 conn(0x7f619406c240 msgr2=0x7f619406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f61995a3640 1 --2- 172.21.15.40:0/2692606483 >> v2:172.21.15.184:3300/0 conn(0x7f619406c240 0x7f619406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f61995a3640 1 -- 172.21.15.40:0/2692606483 >> v1:172.21.15.184:6789/0 conn(0x7f61940726e0 legacy=0x7f6194072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.779+0000 7f61995a3640 1 -- 172.21.15.40:0/2692606483 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61940b4720 con 0x7f619415f1d0 2024-04-26T21:42:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.780+0000 7f61995a3640 1 --2- 172.21.15.40:0/2692606483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619415f1d0 0x7f619406f170 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f6194074060 tx=0x7f619002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5286072ff06cff1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.780+0000 7f618bfff640 1 -- 172.21.15.40:0/2692606483 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190037070 con 0x7f619415f1d0 2024-04-26T21:42:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.780+0000 7f618bfff640 1 -- 172.21.15.40:0/2692606483 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6190032440 con 0x7f619415f1d0 2024-04-26T21:42:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.780+0000 7f618bfff640 1 -- 172.21.15.40:0/2692606483 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190032810 con 0x7f619415f1d0 2024-04-26T21:42:51.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.781+0000 7f619b807640 1 -- 172.21.15.40:0/2692606483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619415f1d0 msgr2=0x7f619406f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.781+0000 7f619b807640 1 --2- 172.21.15.40:0/2692606483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619415f1d0 0x7f619406f170 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f6194074060 tx=0x7f619002a270 comp rx=0 tx=0).stop 2024-04-26T21:42:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.781+0000 7f619b807640 1 -- 172.21.15.40:0/2692606483 shutdown_connections 2024-04-26T21:42:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.781+0000 7f619b807640 1 --2- 172.21.15.40:0/2692606483 >> v2:172.21.15.184:3300/0 conn(0x7f619406c240 0x7f619406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.781+0000 7f619b807640 1 --2- 172.21.15.40:0/2692606483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619415f1d0 0x7f619406f170 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.781+0000 7f619b807640 1 -- 172.21.15.40:0/2692606483 >> 172.21.15.40:0/2692606483 conn(0x7f61940b3180 msgr2=0x7f61940b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.781+0000 7f619b807640 1 -- 172.21.15.40:0/2692606483 shutdown_connections 2024-04-26T21:42:51.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.781+0000 7f619b807640 1 -- 172.21.15.40:0/2692606483 wait complete. 2024-04-26T21:42:51.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.782+0000 7f619b807640 1 Processor -- start 2024-04-26T21:42:51.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.782+0000 7f619b807640 1 -- start start 2024-04-26T21:42:51.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f619b807640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619406c240 0x7f61940c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f619b807640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61940726e0 0x7f61940c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f619b807640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619415f1d0 0x7f61940d1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f619b807640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61940b4fc0 con 0x7f619406c240 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f619b807640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61940b4ce0 con 0x7f619415f1d0 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f619b807640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f619406f6b0 con 0x7f61940726e0 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619406c240 0x7f61940c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619406c240 0x7f61940c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37084/0 (socket says 172.21.15.40:37084) 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 -- 172.21.15.40:0/2994662580 learned_addr learned my addr 172.21.15.40:0/2994662580 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f6198da2640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61940726e0 0x7f61940c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f6199da4640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619415f1d0 0x7f61940d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 -- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61940726e0 msgr2=0x7f61940c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61940726e0 0x7f61940c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 -- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619415f1d0 msgr2=0x7f61940d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619415f1d0 0x7f61940d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 -- 172.21.15.40:0/2994662580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61940c6a60 con 0x7f619406c240 2024-04-26T21:42:51.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.783+0000 7f61995a3640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619406c240 0x7f61940c63f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6190000c00 tx=0x7f61900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.784+0000 7f6189ffb640 1 -- 172.21.15.40:0/2994662580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190041070 con 0x7f619406c240 2024-04-26T21:42:51.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.784+0000 7f6189ffb640 1 -- 172.21.15.40:0/2994662580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f619003c870 con 0x7f619406c240 2024-04-26T21:42:51.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.784+0000 7f6189ffb640 1 -- 172.21.15.40:0/2994662580 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190037040 con 0x7f619406c240 2024-04-26T21:42:51.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.784+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61940c6d40 con 0x7f619406c240 2024-04-26T21:42:51.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.784+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61940c7300 con 0x7f619406c240 2024-04-26T21:42:51.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.785+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6194068fc0 con 0x7f619406c240 2024-04-26T21:42:51.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.785+0000 7f6189ffb640 1 -- 172.21.15.40:0/2994662580 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6190044070 con 0x7f619406c240 2024-04-26T21:42:51.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.786+0000 7f6189ffb640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f617806c920 0x7f617806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.786+0000 7f6189ffb640 1 -- 172.21.15.40:0/2994662580 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6561+0+0 (secure 0 0 0) 0x7f61900acaa0 con 0x7f619406c240 2024-04-26T21:42:51.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.786+0000 7f6198da2640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f617806c920 0x7f617806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.786+0000 7f6198da2640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f617806c920 0x7f617806ede0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6194149870 tx=0x7f6184008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:51.790+0000 7f6189ffb640 1 -- 172.21.15.40:0/2994662580 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f619007ac80 con 0x7f619406c240 2024-04-26T21:42:52.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.017+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61941629f0 con 0x7f619406c240 2024-04-26T21:42:52.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.017+0000 7f6189ffb640 1 -- 172.21.15.40:0/2994662580 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+4351 (secure 0 0 0) 0x7f619007a580 con 0x7f619406c240 2024-04-26T21:42:52.018 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:52.018 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":110,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:51.669882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 110 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f617806c920 msgr2=0x7f617806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f617806c920 0x7f617806ede0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6194149870 tx=0x7f6184008040 comp rx=0 tx=0).stop 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619406c240 msgr2=0x7f61940c63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619406c240 0x7f61940c63f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6190000c00 tx=0x7f61900046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 shutdown_connections 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f617806c920 0x7f617806ede0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619415f1d0 0x7f61940d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61940726e0 0x7f61940c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 --2- 172.21.15.40:0/2994662580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619406c240 0x7f61940c63f0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 >> 172.21.15.40:0/2994662580 conn(0x7f61940b3180 msgr2=0x7f6194161950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 shutdown_connections 2024-04-26T21:42:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.019+0000 7f619b807640 1 -- 172.21.15.40:0/2994662580 wait complete. 2024-04-26T21:42:52.031 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:42:52.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:52.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.172+0000 7fa2a25f0640 1 Processor -- start 2024-04-26T21:42:52.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.172+0000 7fa2a25f0640 1 -- start start 2024-04-26T21:42:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.173+0000 7fa2a25f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c06e270 0x7fa29c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.173+0000 7fa2a25f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa29c0d2420 con 0x7fa29c0d0070 2024-04-26T21:42:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.173+0000 7fa2a25f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa29c0cd6b0 con 0x7fa29c06d560 2024-04-26T21:42:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.173+0000 7fa2a25f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa29c162d00 con 0x7fa29c06e270 2024-04-26T21:42:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.173+0000 7fa29bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa29c0d0070 0x7fa29c06db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35872/0 (socket says 172.21.15.40:35872) 2024-04-26T21:42:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.173+0000 7fa29bfff640 1 -- 172.21.15.40:0/2167956822 learned_addr learned my addr 172.21.15.40:0/2167956822 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29b7fe640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c06e270 0x7fa29c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032568755 0 0) 0x7fa29c0d2420 con 0x7fa29c0d0070 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa284003520 con 0x7fa29c0d0070 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2143643365 0 0) 0x7fa284003520 con 0x7fa29c0d0070 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c06e270 msgr2=0x7fa29c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c06e270 0x7fa29c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 >> v1:172.21.15.184:6789/0 conn(0x7fa29c06d560 legacy=0x7fa29c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c0cd990 con 0x7fa29c0d0070 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fa28c001670 con 0x7fa29c0d0070 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 >> v1:172.21.15.40:6789/0 conn(0x7fa29c0d0070 legacy=0x7fa29c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2840039e0 0x7fa284003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c06e270 0x7fa284004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa284004d40 0x7fa2840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c162d00 con 0x7fa284004d40 2024-04-26T21:42:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa28400bb00 con 0x7fa2840039e0 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa28400bd00 con 0x7fa29c06e270 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa2a0b8d640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa284004d40 0x7fa2840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29bfff640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2840039e0 0x7fa284003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa29b7fe640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c06e270 0x7fa284004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa2a0b8d640 1 -- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c06e270 msgr2=0x7fa284004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa2a0b8d640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c06e270 0x7fa284004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa2a0b8d640 1 -- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2840039e0 msgr2=0x7fa284003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa2a0b8d640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2840039e0 0x7fa284003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.174+0000 7fa2a0b8d640 1 -- 172.21.15.40:0/2167956822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa284003520 con 0x7fa284004d40 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.175+0000 7fa2a0b8d640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa284004d40 0x7fa2840051c0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa294001e20 tx=0x7fa284005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdffcaa5f0a6e9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.175+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294001a70 con 0x7fa284004d40 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.175+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29402ae40 con 0x7fa284004d40 2024-04-26T21:42:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.175+0000 7fa29affd640 1 -- 172.21.15.40:0/2167956822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294003bb0 con 0x7fa284004d40 2024-04-26T21:42:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.176+0000 7fa2a25f0640 1 -- 172.21.15.40:0/2167956822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa284004d40 msgr2=0x7fa2840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.176+0000 7fa2a25f0640 1 --2- 172.21.15.40:0/2167956822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa284004d40 0x7fa2840051c0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa294001e20 tx=0x7fa284005830 comp rx=0 tx=0).stop 2024-04-26T21:42:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.176+0000 7fa2a0b8d640 1 -- 172.21.15.40:0/2167956822 reap_dead start 2024-04-26T21:42:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.176+0000 7fa2a25f0640 1 -- 172.21.15.40:0/2167956822 shutdown_connections 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.176+0000 7fa2a25f0640 1 -- 172.21.15.40:0/2167956822 >> 172.21.15.40:0/2167956822 conn(0x7fa29c0d2ec0 msgr2=0x7fa29c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.176+0000 7fa2a25f0640 1 -- 172.21.15.40:0/2167956822 shutdown_connections 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.176+0000 7fa2a25f0640 1 -- 172.21.15.40:0/2167956822 wait complete. 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.177+0000 7fa2a25f0640 1 Processor -- start 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.177+0000 7fa2a25f0640 1 -- start start 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa2a25f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c15fbe0 0x7fa29c160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa2a25f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0719c0 0x7fa29c0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa2a25f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa29c160840 0x7fa29c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa2a25f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c0d1d40 con 0x7fa29c0719c0 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa2a25f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c0d2d10 con 0x7fa29c160840 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa2a25f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa29c0d1ec0 con 0x7fa29c15fbe0 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0719c0 0x7fa29c0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0719c0 0x7fa29c0ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37104/0 (socket says 172.21.15.40:37104) 2024-04-26T21:42:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29b7fe640 1 -- 172.21.15.40:0/888533797 learned_addr learned my addr 172.21.15.40:0/888533797 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29bfff640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c15fbe0 0x7fa29c160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa2a0b8d640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa29c160840 0x7fa29c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29b7fe640 1 -- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c15fbe0 msgr2=0x7fa29c160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29b7fe640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c15fbe0 0x7fa29c160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29b7fe640 1 -- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa29c160840 msgr2=0x7fa29c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29b7fe640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa29c160840 0x7fa29c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.178+0000 7fa29b7fe640 1 -- 172.21.15.40:0/888533797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c071e50 con 0x7fa29c0719c0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.179+0000 7fa29b7fe640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0719c0 0x7fa29c0ceb30 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa2900119a0 tx=0x7fa290011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.179+0000 7fa298ff9640 1 -- 172.21.15.40:0/888533797 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29000faf0 con 0x7fa29c0719c0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.179+0000 7fa298ff9640 1 -- 172.21.15.40:0/888533797 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29000fc90 con 0x7fa29c0719c0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.179+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c15ffc0 con 0x7fa29c0719c0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.179+0000 7fa298ff9640 1 -- 172.21.15.40:0/888533797 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290014440 con 0x7fa29c0719c0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.179+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29c1f5550 con 0x7fa29c0719c0 2024-04-26T21:42:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.180+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa29c0694f0 con 0x7fa29c0719c0 2024-04-26T21:42:52.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.180+0000 7fa298ff9640 1 -- 172.21.15.40:0/888533797 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa290012040 con 0x7fa29c0719c0 2024-04-26T21:42:52.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.181+0000 7fa298ff9640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28806c920 0x7fa28806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.181+0000 7fa298ff9640 1 -- 172.21.15.40:0/888533797 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6561+0+0 (secure 0 0 0) 0x7fa29008c530 con 0x7fa29c0719c0 2024-04-26T21:42:52.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.181+0000 7fa29bfff640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28806c920 0x7fa28806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.181+0000 7fa29bfff640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28806c920 0x7fa28806ede0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa28c001eb0 tx=0x7fa28c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.185+0000 7fa298ff9640 1 -- 172.21.15.40:0/888533797 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa290019030 con 0x7fa29c0719c0 2024-04-26T21:42:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.415+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa29c06c7a0 con 0x7fa29c0719c0 2024-04-26T21:42:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.415+0000 7fa298ff9640 1 -- 172.21.15.40:0/888533797 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+4351 (secure 0 0 0) 0x7fa29005a600 con 0x7fa29c0719c0 2024-04-26T21:42:52.416 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:52.416 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":110,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:51.669882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:52.417 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 110 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28806c920 msgr2=0x7fa28806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28806c920 0x7fa28806ede0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa28c001eb0 tx=0x7fa28c009040 comp rx=0 tx=0).stop 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0719c0 msgr2=0x7fa29c0ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0719c0 0x7fa29c0ceb30 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa2900119a0 tx=0x7fa290011e60 comp rx=0 tx=0).stop 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 shutdown_connections 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa28806c920 0x7fa28806ede0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa29c160840 0x7fa29c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0719c0 0x7fa29c0ceb30 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 --2- 172.21.15.40:0/888533797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa29c15fbe0 0x7fa29c160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 >> 172.21.15.40:0/888533797 conn(0x7fa29c0d2ec0 msgr2=0x7fa29c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 shutdown_connections 2024-04-26T21:42:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.418+0000 7fa2a25f0640 1 -- 172.21.15.40:0/888533797 wait complete. 2024-04-26T21:42:52.429 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:42:52.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.529+0000 7feb6c828640 1 Processor -- start 2024-04-26T21:42:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.529+0000 7feb6c828640 1 -- start start 2024-04-26T21:42:52.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.530+0000 7feb6c828640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feb68162880 0x7feb680d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.530+0000 7feb6c828640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb680b5f20 0x7feb680d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.530+0000 7feb6c828640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb680b7e50 con 0x7feb680d39f0 2024-04-26T21:42:52.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.530+0000 7feb6c828640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7feb680d0580 con 0x7feb68162880 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.530+0000 7feb6c828640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feb680d1800 con 0x7feb680b5f20 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.530+0000 7feb66d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feb680d39f0 0x7feb680b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35878/0 (socket says 172.21.15.40:35878) 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.530+0000 7feb66d9d640 1 -- 172.21.15.40:0/978388922 learned_addr learned my addr 172.21.15.40:0/978388922 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6659c640 1 --2- 172.21.15.40:0/978388922 >> v2:172.21.15.184:3300/0 conn(0x7feb68162880 0x7feb680d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb65d9b640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb680b5f20 0x7feb680d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405114288 0 0) 0x7feb680b7e50 con 0x7feb680d39f0 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb54003520 con 0x7feb680d39f0 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1868513913 0 0) 0x7feb54003520 con 0x7feb680d39f0 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb680b5f20 msgr2=0x7feb680d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb680b5f20 0x7feb680d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 >> v2:172.21.15.184:3300/0 conn(0x7feb68162880 msgr2=0x7feb680d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 --2- 172.21.15.40:0/978388922 >> v2:172.21.15.184:3300/0 conn(0x7feb68162880 0x7feb680d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb680d07d0 con 0x7feb680d39f0 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7feb600019f0 con 0x7feb680d39f0 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 >> v1:172.21.15.40:6789/0 conn(0x7feb680d39f0 legacy=0x7feb680b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb540039b0 0x7feb54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb54004800 0x7feb54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb680b5f20 0x7feb540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feb680d0580 con 0x7feb54004800 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb680d1800 con 0x7feb540039b0 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feb5400b9c0 con 0x7feb680b5f20 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb65d9b640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb54004800 0x7feb54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb6659c640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb540039b0 0x7feb54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb66d9d640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb680b5f20 0x7feb540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb65d9b640 1 -- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb680b5f20 msgr2=0x7feb540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb65d9b640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb680b5f20 0x7feb540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb65d9b640 1 -- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb540039b0 msgr2=0x7feb54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.531+0000 7feb65d9b640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb540039b0 0x7feb54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.532+0000 7feb65d9b640 1 -- 172.21.15.40:0/978388922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb54003520 con 0x7feb54004800 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.532+0000 7feb65d9b640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb54004800 0x7feb54004c80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7feb50009ac0 tx=0x7feb54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df89cc76217c890 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.532+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50036070 con 0x7feb54004800 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.532+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb50031400 con 0x7feb54004800 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.532+0000 7feb6559a640 1 -- 172.21.15.40:0/978388922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50009690 con 0x7feb54004800 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.533+0000 7feb6c828640 1 -- 172.21.15.40:0/978388922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb54004800 msgr2=0x7feb54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.533+0000 7feb6c828640 1 --2- 172.21.15.40:0/978388922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb54004800 0x7feb54004c80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7feb50009ac0 tx=0x7feb54005830 comp rx=0 tx=0).stop 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.533+0000 7feb66d9d640 1 -- 172.21.15.40:0/978388922 reap_dead start 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.533+0000 7feb6c828640 1 -- 172.21.15.40:0/978388922 shutdown_connections 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.533+0000 7feb6c828640 1 -- 172.21.15.40:0/978388922 >> 172.21.15.40:0/978388922 conn(0x7feb680d4f60 msgr2=0x7feb680d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.533+0000 7feb6c828640 1 -- 172.21.15.40:0/978388922 shutdown_connections 2024-04-26T21:42:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.533+0000 7feb6c828640 1 -- 172.21.15.40:0/978388922 wait complete. 2024-04-26T21:42:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.534+0000 7feb6c828640 1 Processor -- start 2024-04-26T21:42:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.534+0000 7feb6c828640 1 -- start start 2024-04-26T21:42:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb6c828640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54004800 0x7feb680b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb6c828640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb6814fc80 0x7feb680b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb6c828640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb680b9b40 0x7feb681f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb6c828640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feb68162c60 con 0x7feb680b9b40 2024-04-26T21:42:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb6c828640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb680d0d90 con 0x7feb54004800 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb6c828640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feb68162de0 con 0x7feb6814fc80 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb66d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb680b9b40 0x7feb681f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb66d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb680b9b40 0x7feb681f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37120/0 (socket says 172.21.15.40:37120) 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb66d9d640 1 -- 172.21.15.40:0/179976334 learned_addr learned my addr 172.21.15.40:0/179976334 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb6659c640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54004800 0x7feb680b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.535+0000 7feb65d9b640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb6814fc80 0x7feb680b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb66d9d640 1 -- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb6814fc80 msgr2=0x7feb680b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb66d9d640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb6814fc80 0x7feb680b9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb66d9d640 1 -- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54004800 msgr2=0x7feb680b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb66d9d640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54004800 0x7feb680b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb66d9d640 1 -- 172.21.15.40:0/179976334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb681f8e80 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb65d9b640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb6814fc80 0x7feb680b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb66d9d640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb680b9b40 0x7feb681f81c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7feb60002230 tx=0x7feb6000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb5effd640 1 -- 172.21.15.40:0/179976334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6000ebb0 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb5effd640 1 -- 172.21.15.40:0/179976334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb60018070 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6814f1a0 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb5effd640 1 -- 172.21.15.40:0/179976334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60013420 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.536+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb680b59c0 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.537+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb680694f0 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.538+0000 7feb5effd640 1 -- 172.21.15.40:0/179976334 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb600135c0 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.538+0000 7feb5effd640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb4806c920 0x7feb4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.538+0000 7feb5effd640 1 -- 172.21.15.40:0/179976334 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6561+0+0 (secure 0 0 0) 0x7feb6008c1c0 con 0x7feb680b9b40 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.538+0000 7feb6659c640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb4806c920 0x7feb4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.539+0000 7feb6659c640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb4806c920 0x7feb4806ede0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feb68162120 tx=0x7feb58009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:52.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.543+0000 7feb5effd640 1 -- 172.21.15.40:0/179976334 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb6005a290 con 0x7feb680b9b40 2024-04-26T21:42:52.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.769+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb6815f5c0 con 0x7feb680b9b40 2024-04-26T21:42:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.769+0000 7feb5effd640 1 -- 172.21.15.40:0/179976334 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5177 (secure 0 0 0) 0x7feb60059b90 con 0x7feb680b9b40 2024-04-26T21:42:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:52.770 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":111,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1182266137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1182266137},{"type":"v1","addr":"172.21.15.184:6837","nonce":1182266137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:51.669882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 111 2024-04-26T21:42:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb4806c920 msgr2=0x7feb4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb4806c920 0x7feb4806ede0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feb68162120 tx=0x7feb58009190 comp rx=0 tx=0).stop 2024-04-26T21:42:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb680b9b40 msgr2=0x7feb681f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb680b9b40 0x7feb681f81c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7feb60002230 tx=0x7feb6000e530 comp rx=0 tx=0).stop 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 shutdown_connections 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb4806c920 0x7feb4806ede0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb680b9b40 0x7feb681f81c0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb6814fc80 0x7feb680b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 --2- 172.21.15.40:0/179976334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54004800 0x7feb680b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 >> 172.21.15.40:0/179976334 conn(0x7feb680d4f60 msgr2=0x7feb68161610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 shutdown_connections 2024-04-26T21:42:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.771+0000 7feb6c828640 1 -- 172.21.15.40:0/179976334 wait complete. 2024-04-26T21:42:52.783 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000.inode' led to DAMAGED state 2024-04-26T21:42:52.783 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000000' 2024-04-26T21:42:52.783 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:52.783 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:52.799 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:42:52.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:42:52.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:52.800 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:42:52.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:52.856 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:42:52.856 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:42:52.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.997+0000 7f9c47140640 1 Processor -- start 2024-04-26T21:42:52.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.997+0000 7f9c47140640 1 -- start start 2024-04-26T21:42:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c47140640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9c40162880 0x7f9c400d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c47140640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 0x7f9c400d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c47140640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c400b7e50 con 0x7f9c400d39f0 2024-04-26T21:42:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c47140640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9c400d0580 con 0x7f9c40162880 2024-04-26T21:42:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c47140640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c400d1800 con 0x7f9c400b5f20 2024-04-26T21:42:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c456dd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9c400d39f0 0x7f9c400b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35880/0 (socket says 172.21.15.40:35880) 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c456dd640 1 -- 172.21.15.40:0/1094348478 learned_addr learned my addr 172.21.15.40:0/1094348478 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936879191 0 0) 0x7f9c400b7e50 con 0x7f9c400d39f0 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c44edc640 1 --2- 172.21.15.40:0/1094348478 >> v2:172.21.15.184:3300/0 conn(0x7f9c40162880 0x7f9c400d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c28003520 con 0x7f9c400d39f0 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.998+0000 7f9c3ffff640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 0x7f9c400d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 467447354 0 0) 0x7f9c28003520 con 0x7f9c400d39f0 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 msgr2=0x7f9c400d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 0x7f9c400d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 >> v2:172.21.15.184:3300/0 conn(0x7f9c40162880 msgr2=0x7f9c400d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 --2- 172.21.15.40:0/1094348478 >> v2:172.21.15.184:3300/0 conn(0x7f9c40162880 0x7f9c400d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c400d07d0 con 0x7f9c400d39f0 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9c380019f0 con 0x7f9c400d39f0 2024-04-26T21:42:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 >> v1:172.21.15.40:6789/0 conn(0x7f9c400d39f0 legacy=0x7f9c400b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c28004800 0x7f9c28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 0x7f9c280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400d0580 con 0x7f9c280039b0 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400d1800 con 0x7f9c28004800 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2800b9c0 con 0x7f9c400b5f20 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c44edc640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c3ffff640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c28004800 0x7f9c28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c456dd640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 0x7f9c280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c44edc640 1 -- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 msgr2=0x7f9c280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c44edc640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 0x7f9c280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c44edc640 1 -- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c28004800 msgr2=0x7f9c28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c44edc640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c28004800 0x7f9c28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c44edc640 1 -- 172.21.15.40:0/1094348478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c28003520 con 0x7f9c280039b0 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:52.999+0000 7f9c456dd640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400b5f20 0x7f9c280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.000+0000 7f9c44edc640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c28003dd0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009ac0 tx=0x7f9c28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abba541bfd0f557 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.000+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30036070 con 0x7f9c280039b0 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.000+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c30031400 con 0x7f9c280039b0 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.000+0000 7f9c3f7fe640 1 -- 172.21.15.40:0/1094348478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30009690 con 0x7f9c280039b0 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.001+0000 7f9c47140640 1 -- 172.21.15.40:0/1094348478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 msgr2=0x7f9c28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.001+0000 7f9c47140640 1 --2- 172.21.15.40:0/1094348478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c28003dd0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009ac0 tx=0x7f9c28005830 comp rx=0 tx=0).stop 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.001+0000 7f9c456dd640 1 -- 172.21.15.40:0/1094348478 reap_dead start 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.001+0000 7f9c47140640 1 -- 172.21.15.40:0/1094348478 shutdown_connections 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.001+0000 7f9c47140640 1 -- 172.21.15.40:0/1094348478 >> 172.21.15.40:0/1094348478 conn(0x7f9c400d4f60 msgr2=0x7f9c400d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.001+0000 7f9c47140640 1 -- 172.21.15.40:0/1094348478 shutdown_connections 2024-04-26T21:42:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.001+0000 7f9c47140640 1 -- 172.21.15.40:0/1094348478 wait complete. 2024-04-26T21:42:53.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.002+0000 7f9c47140640 1 Processor -- start 2024-04-26T21:42:53.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.002+0000 7f9c47140640 1 -- start start 2024-04-26T21:42:53.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c47140640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c40157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c47140640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c400cee70 0x7f9c400cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c47140640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400cd880 0x7f9c400b9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c47140640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40162c60 con 0x7f9c280039b0 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c47140640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400d0d90 con 0x7f9c400cee70 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c47140640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c40162de0 con 0x7f9c400cd880 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c40157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c40157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37128/0 (socket says 172.21.15.40:37128) 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 -- 172.21.15.40:0/1625174852 learned_addr learned my addr 172.21.15.40:0/1625174852 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c3ffff640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c400cee70 0x7f9c400cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c456dd640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400cd880 0x7f9c400b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 -- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400cd880 msgr2=0x7f9c400b9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400cd880 0x7f9c400b9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 -- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c400cee70 msgr2=0x7f9c400cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c400cee70 0x7f9c400cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 -- 172.21.15.40:0/1625174852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c400cddd0 con 0x7f9c280039b0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c3ffff640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c400cee70 0x7f9c400cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c44edc640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c40157490 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009be0 tx=0x7f9c30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.003+0000 7f9c3d7fa640 1 -- 172.21.15.40:0/1625174852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30040020 con 0x7f9c280039b0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.004+0000 7f9c3d7fa640 1 -- 172.21.15.40:0/1625174852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c30003950 con 0x7f9c280039b0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.004+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c400ce0b0 con 0x7f9c280039b0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.004+0000 7f9c3d7fa640 1 -- 172.21.15.40:0/1625174852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30036040 con 0x7f9c280039b0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.004+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c400ce670 con 0x7f9c280039b0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.005+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c400694f0 con 0x7f9c280039b0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.005+0000 7f9c3d7fa640 1 -- 172.21.15.40:0/1625174852 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c30043020 con 0x7f9c280039b0 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.005+0000 7f9c3d7fa640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c2c06c920 0x7f9c2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.006+0000 7f9c3d7fa640 1 -- 172.21.15.40:0/1625174852 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6561+0+0 (secure 0 0 0) 0x7f9c300ab920 con 0x7f9c280039b0 2024-04-26T21:42:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.006+0000 7f9c3ffff640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c2c06c920 0x7f9c2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.006+0000 7f9c3ffff640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c2c06c920 0x7f9c2c06ede0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9c40155bf0 tx=0x7f9c34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.010+0000 7f9c3d7fa640 1 -- 172.21.15.40:0/1625174852 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c30079a80 con 0x7f9c280039b0 2024-04-26T21:42:53.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.233+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9c40161690 con 0x7f9c280039b0 2024-04-26T21:42:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.688+0000 7f9c3d7fa640 1 -- 172.21.15.40:0/1625174852 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) ==== 161+0+0 (secure 0 0 0) 0x7f9c30079380 con 0x7f9c280039b0 2024-04-26T21:42:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c2c06c920 msgr2=0x7f9c2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c2c06c920 0x7f9c2c06ede0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9c40155bf0 tx=0x7f9c34008040 comp rx=0 tx=0).stop 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 msgr2=0x7f9c40157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c40157490 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009be0 tx=0x7f9c30004730 comp rx=0 tx=0).stop 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 shutdown_connections 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c2c06c920 0x7f9c2c06ede0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c400cd880 0x7f9c400b9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c400cee70 0x7f9c400cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 --2- 172.21.15.40:0/1625174852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c280039b0 0x7f9c40157490 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 >> 172.21.15.40:0/1625174852 conn(0x7f9c400d4f60 msgr2=0x7f9c4015fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:53.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 shutdown_connections 2024-04-26T21:42:53.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.691+0000 7f9c47140640 1 -- 172.21.15.40:0/1625174852 wait complete. 2024-04-26T21:42:53.704 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:42:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.807+0000 7f6982504640 1 Processor -- start 2024-04-26T21:42:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.807+0000 7f6982504640 1 -- start start 2024-04-26T21:42:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.808+0000 7f6982504640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f697c162880 0x7f697c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.808+0000 7f6982504640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.808+0000 7f6982504640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697c0b7e50 con 0x7f697c0d39f0 2024-04-26T21:42:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.808+0000 7f6982504640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f697c0d0580 con 0x7f697c162880 2024-04-26T21:42:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.808+0000 7f6982504640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f697c0d1800 con 0x7f697c0b5f20 2024-04-26T21:42:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.808+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.809+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:48008/0 (socket says 172.21.15.40:48008) 2024-04-26T21:42:53.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.809+0000 7f697bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f697c162880 0x7f697c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.809+0000 7f697b7fe640 1 -- 172.21.15.40:0/276136337 learned_addr learned my addr 172.21.15.40:0/276136337 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.809+0000 7f697b7fe640 1 -- 172.21.15.40:0/276136337 >> v2:172.21.15.184:3300/0 conn(0x7f697c162880 msgr2=0x7f697c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.809+0000 7f697b7fe640 1 --2- 172.21.15.40:0/276136337 >> v2:172.21.15.184:3300/0 conn(0x7f697c162880 0x7f697c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.809+0000 7f697b7fe640 1 -- 172.21.15.40:0/276136337 >> v1:172.21.15.184:6789/0 conn(0x7f697c0d39f0 legacy=0x7f697c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.809+0000 7f697b7fe640 1 -- 172.21.15.40:0/276136337 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c0d07d0 con 0x7f697c0b5f20 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.809+0000 7f697bfff640 1 --2- 172.21.15.40:0/276136337 >> v2:172.21.15.184:3300/0 conn(0x7f697c162880 0x7f697c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f697b7fe640 1 --2- 172.21.15.40:0/276136337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0d34b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f696c009870 tx=0x7f696c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79157b18fe68b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f697affd640 1 -- 172.21.15.40:0/276136337 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c037070 con 0x7f697c0b5f20 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f697affd640 1 -- 172.21.15.40:0/276136337 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c032440 con 0x7f697c0b5f20 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f697affd640 1 -- 172.21.15.40:0/276136337 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c032810 con 0x7f697c0b5f20 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f6982504640 1 -- 172.21.15.40:0/276136337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 msgr2=0x7f697c0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f6982504640 1 --2- 172.21.15.40:0/276136337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0d34b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f696c009870 tx=0x7f696c02a270 comp rx=0 tx=0).stop 2024-04-26T21:42:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f6982504640 1 -- 172.21.15.40:0/276136337 shutdown_connections 2024-04-26T21:42:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f6982504640 1 --2- 172.21.15.40:0/276136337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0d34b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f6982504640 1 --2- 172.21.15.40:0/276136337 >> v2:172.21.15.184:3300/0 conn(0x7f697c162880 0x7f697c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f6982504640 1 -- 172.21.15.40:0/276136337 >> 172.21.15.40:0/276136337 conn(0x7f697c0d4f60 msgr2=0x7f697c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.810+0000 7f6982504640 1 -- 172.21.15.40:0/276136337 shutdown_connections 2024-04-26T21:42:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.811+0000 7f6982504640 1 -- 172.21.15.40:0/276136337 wait complete. 2024-04-26T21:42:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.812+0000 7f6982504640 1 Processor -- start 2024-04-26T21:42:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.812+0000 7f6982504640 1 -- start start 2024-04-26T21:42:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.812+0000 7f6982504640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6982504640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697c0d39f0 0x7f697c15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6982504640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697c162880 0x7f697c1f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6982504640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f697c162c60 con 0x7f697c0d39f0 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6982504640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f697c0d0d90 con 0x7f697c162880 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6982504640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f697c162de0 con 0x7f697c0b5f20 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697c0d39f0 0x7f697c15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697c0d39f0 0x7f697c15f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37134/0 (socket says 172.21.15.40:37134) 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 -- 172.21.15.40:0/2909860410 learned_addr learned my addr 172.21.15.40:0/2909860410 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6980aa1640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697c162880 0x7f697c1f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697bfff640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 -- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 msgr2=0x7f697c0cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 -- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697c162880 msgr2=0x7f697c1f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697c162880 0x7f697c1f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 -- 172.21.15.40:0/2909860410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c152ed0 con 0x7f697c0d39f0 2024-04-26T21:42:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f697b7fe640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697c0d39f0 0x7f697c15f6a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6978ff9640 1 -- 172.21.15.40:0/2909860410 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c037040 con 0x7f697c0d39f0 2024-04-26T21:42:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6978ff9640 1 -- 172.21.15.40:0/2909860410 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c042070 con 0x7f697c0d39f0 2024-04-26T21:42:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c1f8dd0 con 0x7f697c0d39f0 2024-04-26T21:42:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.813+0000 7f6978ff9640 1 -- 172.21.15.40:0/2909860410 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c030880 con 0x7f697c0d39f0 2024-04-26T21:42:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.814+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c0cbfa0 con 0x7f697c0d39f0 2024-04-26T21:42:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.815+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c0694f0 con 0x7f697c0d39f0 2024-04-26T21:42:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.815+0000 7f6978ff9640 1 -- 172.21.15.40:0/2909860410 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f696c030a20 con 0x7f697c0d39f0 2024-04-26T21:42:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.815+0000 7f6978ff9640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.816+0000 7f6978ff9640 1 -- 172.21.15.40:0/2909860410 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6561+0+0 (secure 0 0 0) 0x7f696c0ac630 con 0x7f697c0d39f0 2024-04-26T21:42:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.816+0000 7f697bfff640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.816+0000 7f697bfff640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f695c06c8f0 0x7f695c06edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6964009560 tx=0x7f6964009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:53.820+0000 7f6978ff9640 1 -- 172.21.15.40:0/2909860410 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f696c07a0b0 con 0x7f697c0d39f0 2024-04-26T21:42:54.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.080+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f697c0cfe30 con 0x7f697c0d39f0 2024-04-26T21:42:54.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.694+0000 7f6978ff9640 1 -- 172.21.15.40:0/2909860410 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v113) ==== 101+0+0 (secure 0 0 0) 0x7f696c07e3b0 con 0x7f697c0d39f0 2024-04-26T21:42:54.695 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:42:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f695c06c8f0 msgr2=0x7f695c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f695c06c8f0 0x7f695c06edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6964009560 tx=0x7f6964009190 comp rx=0 tx=0).stop 2024-04-26T21:42:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697c0d39f0 msgr2=0x7f697c15f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697c0d39f0 0x7f697c15f6a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 shutdown_connections 2024-04-26T21:42:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697c162880 0x7f697c1f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697c0d39f0 0x7f697c15f6a0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 --2- 172.21.15.40:0/2909860410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697c0b5f20 0x7f697c0cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 >> 172.21.15.40:0/2909860410 conn(0x7f697c0d4f60 msgr2=0x7f697c0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:54.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 shutdown_connections 2024-04-26T21:42:54.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.697+0000 7f6982504640 1 -- 172.21.15.40:0/2909860410 wait complete. 2024-04-26T21:42:54.711 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:42:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.756+0000 7fcfbae0cf40 1 Processor -- start 2024-04-26T21:42:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.756+0000 7fcfbae0cf40 1 -- start start 2024-04-26T21:42:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfbae0cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 0x55e1a869a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfbae0cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 0x55e1a8734a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfbae0cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a8697510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfbae0cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8613e40 con 0x55e1a8744b70 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfbae0cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e1a86b5660 con 0x55e1a86b6420 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfbae0cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e1a866dc10 con 0x55e1a873df70 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a8697510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a8697510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37156/0 (socket says 172.21.15.40:37156) 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 -- 172.21.15.40:0/485517721 learned_addr learned my addr 172.21.15.40:0/485517721 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb7b9a640 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 0x55e1a869a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb7399640 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 0x55e1a8734a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 -- 172.21.15.40:0/485517721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 msgr2=0x55e1a8734a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 0x55e1a8734a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 -- 172.21.15.40:0/485517721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 msgr2=0x55e1a869a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 0x55e1a869a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 -- 172.21.15.40:0/485517721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a866e900 con 0x55e1a8744b70 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb839b640 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a8697510 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00460d0 tx=0x7fcfb0050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f8a8b3243504f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb6397640 1 -- 172.21.15.40:0/485517721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0054040 con 0x55e1a8744b70 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb6397640 1 -- 172.21.15.40:0/485517721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb0050930 con 0x55e1a8744b70 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.757+0000 7fcfb6397640 1 -- 172.21.15.40:0/485517721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0050d20 con 0x55e1a8744b70 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/485517721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 msgr2=0x55e1a8697510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a8697510 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00460d0 tx=0x7fcfb0050000 comp rx=0 tx=0).stop 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/485517721 shutdown_connections 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a8697510 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 0x55e1a8734a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/485517721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 0x55e1a869a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/485517721 >> 172.21.15.40:0/485517721 conn(0x55e1a86968a0 msgr2=0x55e1a866ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/485517721 shutdown_connections 2024-04-26T21:42:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/485517721 wait complete. 2024-04-26T21:42:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.758+0000 7fcfbae0cf40 1 Processor -- start 2024-04-26T21:42:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfbae0cf40 1 -- start start 2024-04-26T21:42:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfbae0cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 0x55e1a8696e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfbae0cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 0x55e1a873f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfbae0cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a86b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfbae0cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e1a866e0a0 con 0x55e1a8744b70 2024-04-26T21:42:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfbae0cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8734f70 con 0x55e1a86b6420 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfbae0cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e1a866d130 con 0x55e1a873df70 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfb839b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a86b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfb839b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a86b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37158/0 (socket says 172.21.15.40:37158) 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 learned_addr learned my addr 172.21.15.40:0/3444885829 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfb7b9a640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 0x55e1a8696e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.759+0000 7fcfb7399640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 0x55e1a873f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 msgr2=0x55e1a873f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb839b640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a873df70 0x55e1a873f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 msgr2=0x55e1a8696e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb839b640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a86b6420 0x55e1a8696e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a87d6740 con 0x55e1a8744b70 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb839b640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a86b2660 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00460d0 tx=0x7fcfb0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb5395640 1 -- 172.21.15.40:0/3444885829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb006d070 con 0x55e1a8744b70 2024-04-26T21:42:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb5395640 1 -- 172.21.15.40:0/3444885829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb0066710 con 0x55e1a8744b70 2024-04-26T21:42:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1a8730100 con 0x55e1a8744b70 2024-04-26T21:42:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfb5395640 1 -- 172.21.15.40:0/3444885829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0054050 con 0x55e1a8744b70 2024-04-26T21:42:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.760+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a87305b0 con 0x55e1a8744b70 2024-04-26T21:42:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.762+0000 7fcfb5395640 1 -- 172.21.15.40:0/3444885829 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfb0070070 con 0x55e1a8744b70 2024-04-26T21:42:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.762+0000 7fcfb5395640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfa406b300 0x7fcfa406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.762+0000 7fcfb5395640 1 -- 172.21.15.40:0/3444885829 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6561+0+0 (secure 0 0 0) 0x7fcfb00cf7e0 con 0x55e1a8744b70 2024-04-26T21:42:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.762+0000 7fcfb7b9a640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfa406b300 0x7fcfa406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.766 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:42:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.763+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e1a866eb70 0x55e1a872f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.763+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a873f8e0 con 0x55e1a866eb70 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.763+0000 7fcfb7399640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e1a866eb70 0x55e1a872f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.763+0000 7fcfb7b9a640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfa406b300 0x7fcfa406d7c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8000c00 tx=0x7fcfa80018b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.763+0000 7fcfb7399640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e1a866eb70 0x55e1a872f510 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.764+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v99'102 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.764+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a86973b0 con 0x55e1a866eb70 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.766+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v99'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.766+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a872fa50 con 0x55e1a866eb70 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.767+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v99'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.768 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:42:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.767+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7b80 con 0x55e1a866eb70 2024-04-26T21:42:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.768+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v99'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.768+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7f20 con 0x55e1a866eb70 2024-04-26T21:42:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.769+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v99'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.769+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6b70 con 0x55e1a866eb70 2024-04-26T21:42:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.771+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v99'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.771+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.772+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v99'108 uv84 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.772+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.773+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v99'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.773+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.774+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v99'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.775 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:42:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.774+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.776+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v99'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.776+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.777+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v99'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.777+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.778+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v99'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.778+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.779+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v99'114 uv91 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.779+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.780+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v99'115 uv115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.780+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.781+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v99'116 uv116 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.782 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:42:54.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.781+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.783+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v99'117 uv117 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.783+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.784+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v99'118 uv118 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.784+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e6f10 con 0x55e1a866eb70 2024-04-26T21:42:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.785+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v99'119 uv119 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.785+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e77a0 con 0x55e1a866eb70 2024-04-26T21:42:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.786+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v99'120 uv120 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.786+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e77a0 con 0x55e1a866eb70 2024-04-26T21:42:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.787+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v99'121 uv95 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.787+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87e77a0 con 0x55e1a866eb70 2024-04-26T21:42:54.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.788+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v99'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.788+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.789 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:42:54.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.789+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v99'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.789+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.790+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v99'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.790+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.791+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v99'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.791+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.792+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v99'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.792+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.793+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v99'127 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.793+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.795+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v99'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.795+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.799 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:42:54.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.798+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v99'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.798+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.801+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v99'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.801+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.803+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v99'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.803+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a866eb70 2024-04-26T21:42:54.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.809+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v99'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a866eb70 2024-04-26T21:42:54.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.809+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a87e87d0 0x55e1a87eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.809+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87e7670 con 0x55e1a87e87d0 2024-04-26T21:42:54.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.809+0000 7fcfb7b9a640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a87e87d0 0x55e1a87eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.809+0000 7fcfb7b9a640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a87e87d0 0x55e1a87eabc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.811 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:42:54.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.811+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v99'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.811+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87eb5b0 con 0x55e1a87e87d0 2024-04-26T21:42:54.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.812+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v99'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.812+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87eb950 con 0x55e1a87e87d0 2024-04-26T21:42:54.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.814+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v99'91 uv91 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.814+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ebcf0 con 0x55e1a87e87d0 2024-04-26T21:42:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.815+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 200.00000000 [create] v99'92 uv76 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.815+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87ec090 con 0x55e1a87e87d0 2024-04-26T21:42:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.816+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 200.00000000 [delete] v99'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.816+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ec430 con 0x55e1a87e87d0 2024-04-26T21:42:54.819 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:42:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.819+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v99'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.819+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ec7d0 con 0x55e1a87e87d0 2024-04-26T21:42:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.822+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v99'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.822+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ecb70 con 0x55e1a87e87d0 2024-04-26T21:42:54.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.824+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v99'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.824+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ecb70 con 0x55e1a87e87d0 2024-04-26T21:42:54.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.827+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000001 [create] v99'97 uv81 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.827+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87ecb70 con 0x55e1a87e87d0 2024-04-26T21:42:54.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.830+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000001 [delete] v99'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.830+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ecb70 con 0x55e1a87e87d0 2024-04-26T21:42:54.831 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:42:54.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.831+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v99'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.831+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f1230 con 0x55e1a87e87d0 2024-04-26T21:42:54.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.834+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [write 0~18075] v99'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.834+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f1230 con 0x55e1a87e87d0 2024-04-26T21:42:54.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.837+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v99'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.837+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ecb70 con 0x55e1a87e87d0 2024-04-26T21:42:54.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.840+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 606.00000000 [create] v99'102 uv87 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.840+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87ecb70 con 0x55e1a87e87d0 2024-04-26T21:42:54.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.841+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 606.00000000 [delete] v99'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.841+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ecb70 con 0x55e1a87e87d0 2024-04-26T21:42:54.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.844+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v99'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.845 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:42:54.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.844+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87edb80 con 0x55e1a87e87d0 2024-04-26T21:42:54.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.847+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v99'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.847+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87edb80 con 0x55e1a87e87d0 2024-04-26T21:42:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.849+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v99'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.849+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ecb70 con 0x55e1a87e87d0 2024-04-26T21:42:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.852+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v99'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87e87d0 2024-04-26T21:42:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.852+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e1a87ed080 0x55e1a87ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.852+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ef9b0 con 0x55e1a87ed080 2024-04-26T21:42:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.852+0000 7fcfb7399640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e1a87ed080 0x55e1a87ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.852+0000 7fcfb7399640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e1a87ed080 0x55e1a87ef470 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.854+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(51 607.00000000 [create] v99'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.854+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87ef9b0 con 0x55e1a87ed080 2024-04-26T21:42:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.855+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(52 607.00000000 [delete] v99'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.855+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87ef9b0 con 0x55e1a87ed080 2024-04-26T21:42:54.859 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:42:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.858+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(53 607.00000000 [create] v99'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.858+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f0db0 con 0x55e1a87ed080 2024-04-26T21:42:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.864+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v99'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.864+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f0db0 con 0x55e1a87ed080 2024-04-26T21:42:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.865+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v99'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.865+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87efda0 con 0x55e1a87ed080 2024-04-26T21:42:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.867+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v99'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.867+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e1a87f02b0 0x55e1a87f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.867+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f2cd0 con 0x55e1a87f02b0 2024-04-26T21:42:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.868+0000 7fcfb839b640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e1a87f02b0 0x55e1a87f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.868+0000 7fcfb839b640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e1a87f02b0 0x55e1a87f26a0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.869+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v99'18 uv17 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.869+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87f31e0 con 0x55e1a87f02b0 2024-04-26T21:42:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.870+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v99'19 uv19 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.870+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f36a0 con 0x55e1a87f02b0 2024-04-26T21:42:54.876 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:42:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.877+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v99'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.877+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f3f90 con 0x55e1a87f02b0 2024-04-26T21:42:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.877+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v99'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.877+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e1a87f4840 0x55e1a87f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.877+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f7170 con 0x55e1a87f4840 2024-04-26T21:42:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.877+0000 7fcfb7b9a640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e1a87f4840 0x55e1a87f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.878+0000 7fcfb7b9a640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e1a87f4840 0x55e1a87f6c30 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.880+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(61 608.00000000 [create] v99'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.880+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87f7230 con 0x55e1a87f4840 2024-04-26T21:42:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.881+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(62 608.00000000 [delete] v99'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.881+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f7650 con 0x55e1a87f4840 2024-04-26T21:42:54.883 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:42:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.883+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(63 608.00000000 [create] v99'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.883+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8660 con 0x55e1a87f4840 2024-04-26T21:42:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.884+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v99'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.884+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8660 con 0x55e1a87f4840 2024-04-26T21:42:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.885+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v99'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.885+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f7650 con 0x55e1a87f4840 2024-04-26T21:42:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.886+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v99'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.886+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f7650 con 0x55e1a87f02b0 2024-04-26T21:42:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.887+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(67 604.00000000 [create] v99'37 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.887+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87f79f0 con 0x55e1a87f02b0 2024-04-26T21:42:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.888+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(68 604.00000000 [delete] v99'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.888+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f7d90 con 0x55e1a87f02b0 2024-04-26T21:42:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.889+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(69 604.00000000 [create] v99'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.890 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:42:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.889+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f9140 con 0x55e1a87f02b0 2024-04-26T21:42:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.892+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v99'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.892+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f9140 con 0x55e1a87f02b0 2024-04-26T21:42:54.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.895+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v99'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.895+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.896+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v99'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.896+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.897+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(73 500.00000000 [create] v99'43 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.897+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.898+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(74 500.00000000 [delete] v99'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.898+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.900 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:42:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.899+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(75 500.00000000 [create] v99'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.899+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.900+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v99'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.900+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.901+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v99'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.902+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.903+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(78 mds_snaptable [create] v99'17 uv16 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.903+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.905+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(79 mds_snaptable [delete] v99'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.905+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.908+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(80 mds_snaptable [create] v99'19 uv19 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.909 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:42:54.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.908+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.909+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(81 mds_snaptable [write 0~70] v99'20 uv20 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.909+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f02b0 2024-04-26T21:42:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.910+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v99'21 uv21 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcfb006f030 con 0x55e1a87f02b0 2024-04-26T21:42:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.910+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.911+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(83 605.00000000 [create] v99'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.911+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.912+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(84 605.00000000 [delete] v99'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.912+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.916 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:42:54.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.915+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(85 605.00000000 [create] v99'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.915+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.918+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v99'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.918+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.923+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v99'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.923+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f84f0 con 0x55e1a87f4840 2024-04-26T21:42:54.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.924+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(88 605.00000000 [omap-set-header] v99'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.924+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.925+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(89 mds0_inotable [create] v99'34 uv28 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.925+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.926+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(90 mds0_inotable [delete] v99'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.926+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.928 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:42:54.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.927+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(91 mds0_inotable [create] v99'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.927+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.928+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(92 mds0_inotable [write 0~34] v99'37 uv37 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.928+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.929+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v99'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.929+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.931+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(94 100.00000000 [create] v99'39 uv33 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.931+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.932+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(95 100.00000000 [delete] v99'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.932+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.933 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:42:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.933+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(96 100.00000000 [create] v99'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.933+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8130 con 0x55e1a87f4840 2024-04-26T21:42:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.934+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(97 100.00000000 [omap-set-header] v99'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.934+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f3ba0 con 0x55e1a87f4840 2024-04-26T21:42:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.935+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v99'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.935+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e1a87f3b60 0x55e1a87fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.935+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87f3b60 2024-04-26T21:42:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.935+0000 7fcfb7399640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e1a87f3b60 0x55e1a87fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.935+0000 7fcfb7399640 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e1a87f3b60 0x55e1a87fb360 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.937+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(99 mds0_sessionmap [create] v99'18 uv17 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87f3b60 2024-04-26T21:42:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.937+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87f3b60 2024-04-26T21:42:54.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.941+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v99'19 uv19 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87f3b60 2024-04-26T21:42:54.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.941+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87f3b60 2024-04-26T21:42:54.942 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:42:54.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.942+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(101 mds0_sessionmap [create] v99'20 uv20 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87f3b60 2024-04-26T21:42:54.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.942+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87f3b60 2024-04-26T21:42:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.945+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v99'21 uv21 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87f3b60 2024-04-26T21:42:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.945+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87f4840 2024-04-26T21:42:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.946+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(103 609.00000000 [create] v99'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.947+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87f4840 2024-04-26T21:42:54.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.948+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(104 609.00000000 [delete] v99'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.948+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87f4840 2024-04-26T21:42:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.949+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(105 609.00000000 [create] v99'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.950 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:42:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.949+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fc8b0 con 0x55e1a87f4840 2024-04-26T21:42:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.950+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v99'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.950+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fc8b0 con 0x55e1a87f4840 2024-04-26T21:42:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.951+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v99'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.951+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87f4840 2024-04-26T21:42:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.952+0000 7fcfb7b9a640 1 -- 172.21.15.40:0/3444885829 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(108 609.00000000 [omap-set-header] v99'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfa800e5c0 con 0x55e1a87f4840 2024-04-26T21:42:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.952+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87ed080 2024-04-26T21:42:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.953+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v99'17 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.953+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87ed080 2024-04-26T21:42:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.954+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.954+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fb8a0 con 0x55e1a87ed080 2024-04-26T21:42:54.956 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:42:54.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.956+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.956+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f86f0 con 0x55e1a87ed080 2024-04-26T21:42:54.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.957+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.957+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8770 con 0x55e1a87ed080 2024-04-26T21:42:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.958+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v99'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.958+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8770 con 0x55e1a87ed080 2024-04-26T21:42:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.959+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v99'45 uv37 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.959+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87fbdd0 con 0x55e1a87ed080 2024-04-26T21:42:54.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.960+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v99'46 uv46 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.960+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fbdd0 con 0x55e1a87ed080 2024-04-26T21:42:54.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.961+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v99'47 uv47 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.962 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:42:54.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.961+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fbd80 con 0x55e1a87ed080 2024-04-26T21:42:54.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.962+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v99'48 uv48 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.962+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fbd80 con 0x55e1a87ed080 2024-04-26T21:42:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.963+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v99'49 uv49 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.963+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fbd80 con 0x55e1a87ed080 2024-04-26T21:42:54.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.964+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v99'50 uv44 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.964+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e1a87fbd80 con 0x55e1a87ed080 2024-04-26T21:42:54.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.965+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v99'51 uv51 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.965+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8770 con 0x55e1a87ed080 2024-04-26T21:42:54.966 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:42:54.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.966+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v99'52 uv52 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.966+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8770 con 0x55e1a87ed080 2024-04-26T21:42:54.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.967+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v99'53 uv53 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.967+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fce10 con 0x55e1a87ed080 2024-04-26T21:42:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.968+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v99'54 uv54 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.968+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87f8770 con 0x55e1a87ed080 2024-04-26T21:42:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.969+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v99'55 uv55 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.969+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e1a87fbd80 con 0x55e1a87ed080 2024-04-26T21:42:54.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfb7399640 1 -- 172.21.15.40:0/3444885829 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v99'56 uv56 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcfac00a420 con 0x55e1a87ed080 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a87e87d0 msgr2=0x55e1a87eabc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a87e87d0 0x55e1a87eabc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e1a866eb70 msgr2=0x55e1a872f510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e1a866eb70 0x55e1a872f510 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e1a87f02b0 msgr2=0x55e1a87f26a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e1a87f02b0 0x55e1a87f26a0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e1a87f3b60 msgr2=0x55e1a87fb360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e1a87f3b60 0x55e1a87fb360 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e1a87ed080 msgr2=0x55e1a87ef470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e1a87ed080 0x55e1a87ef470 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 reap_dead start 2024-04-26T21:42:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e1a87f4840 msgr2=0x55e1a87f6c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e1a87f4840 0x55e1a87f6c30 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 reap_dead start 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfb839b640 1 -- 172.21.15.40:0/3444885829 reap_dead start 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfa406b300 msgr2=0x7fcfa406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfa406b300 0x7fcfa406d7c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8000c00 tx=0x7fcfa80018b0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 msgr2=0x55e1a86b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a86b2660 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00460d0 tx=0x7fcfb0003330 comp rx=0 tx=0).stop 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 shutdown_connections 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfa406b300 0x7fcfa406d7c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 --2- 172.21.15.40:0/3444885829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a8744b70 0x55e1a86b2660 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 >> 172.21.15.40:0/3444885829 conn(0x55e1a86968a0 msgr2=0x55e1a866ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 shutdown_connections 2024-04-26T21:42:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:54.970+0000 7fcfbae0cf40 1 -- 172.21.15.40:0/3444885829 wait complete. 2024-04-26T21:42:54.974 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000000 2024-04-26T21:42:55.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.016+0000 7fbcac0cbf40 1 Processor -- start 2024-04-26T21:42:55.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.016+0000 7fbcac0cbf40 1 -- start start 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbcac0cbf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 0x563b71a65720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbcac0cbf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71b0f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbcac0cbf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 0x563b71b0bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbcac0cbf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563b719dde40 con 0x563b71b07200 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbcac0cbf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563b719e1e80 con 0x563b71b0f5d0 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbcac0cbf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563b71a60680 con 0x7fbc980030f0 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca93c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71b0f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca93c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71b0f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37178/0 (socket says 172.21.15.40:37178) 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca93c4640 1 -- 172.21.15.40:0/3603758316 learned_addr learned my addr 172.21.15.40:0/3603758316 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbcaa3c6640 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 0x563b71b0bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca9bc5640 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 0x563b71a65720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca93c4640 1 -- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 msgr2=0x563b71a65720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca93c4640 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 0x563b71a65720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca93c4640 1 -- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 msgr2=0x563b71b0bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca93c4640 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 0x563b71b0bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca93c4640 1 -- 172.21.15.40:0/3603758316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b71a63b50 con 0x563b71b07200 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.017+0000 7fbca9bc5640 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 0x563b71a65720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbca93c4640 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71b0f090 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbc94026300 tx=0x7fbc9404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6237788ac6b96486 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbca3fff640 1 -- 172.21.15.40:0/3603758316 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400b430 con 0x563b71b07200 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbca3fff640 1 -- 172.21.15.40:0/3603758316 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc9404db90 con 0x563b71b07200 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbca3fff640 1 -- 172.21.15.40:0/3603758316 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94060670 con 0x563b71b07200 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3603758316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 msgr2=0x563b71b0f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71b0f090 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbc94026300 tx=0x7fbc9404f000 comp rx=0 tx=0).stop 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3603758316 shutdown_connections 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 0x563b71b0bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71b0f090 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 --2- 172.21.15.40:0/3603758316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 0x563b71a65720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3603758316 >> 172.21.15.40:0/3603758316 conn(0x563b71a608a0 msgr2=0x563b71b11bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3603758316 shutdown_connections 2024-04-26T21:42:55.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.018+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3603758316 wait complete. 2024-04-26T21:42:55.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.019+0000 7fbcac0cbf40 1 Processor -- start 2024-04-26T21:42:55.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.019+0000 7fbcac0cbf40 1 -- start start 2024-04-26T21:42:55.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcac0cbf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71a7bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcac0cbf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 0x563b71b09dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcac0cbf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 0x563b71b0a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcac0cbf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563b71a63df0 con 0x563b71b07200 2024-04-26T21:42:55.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcac0cbf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563b71a603a0 con 0x563b71b0f5d0 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcac0cbf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563b71b122d0 con 0x7fbc980030f0 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71a7bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71a7bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37184/0 (socket says 172.21.15.40:37184) 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 -- 172.21.15.40:0/3021394612 learned_addr learned my addr 172.21.15.40:0/3021394612 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcaa3c6640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 0x563b71b0a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca93c4640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 0x563b71b09dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 -- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 msgr2=0x563b71b0a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc980030f0 0x563b71b0a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 -- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 msgr2=0x563b71b09dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b71b0f5d0 0x563b71b09dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 -- 172.21.15.40:0/3021394612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b71a7a0e0 con 0x563b71b07200 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca9bc5640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71a7bb50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbca4010510 tx=0x7fbca4010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca2ffd640 1 -- 172.21.15.40:0/3021394612 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400e8b0 con 0x563b71b07200 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca2ffd640 1 -- 172.21.15.40:0/3021394612 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca400ea50 con 0x563b71b07200 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563b71a7b070 con 0x563b71b07200 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbca2ffd640 1 -- 172.21.15.40:0/3021394612 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400ee00 con 0x563b71b07200 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.020+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b71a7b630 con 0x563b71b07200 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.022+0000 7fbca2ffd640 1 -- 172.21.15.40:0/3021394612 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbca40223f0 con 0x563b71b07200 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.022+0000 7fbca2ffd640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc8c06b2d0 0x7fbc8c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.022+0000 7fbca2ffd640 1 -- 172.21.15.40:0/3021394612 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6561+0+0 (secure 0 0 0) 0x7fbca408b790 con 0x563b71b07200 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.022+0000 7fbca93c4640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc8c06b2d0 0x7fbc8c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.022+0000 7fbcac0cbf40 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b71b0b0c0 0x563b71ba50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.022+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x563b71b0a840 con 0x563b71b0b0c0 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.022+0000 7fbcaa3c6640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b71b0b0c0 0x563b71ba50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.023+0000 7fbca93c4640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc8c06b2d0 0x7fbc8c06d790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbc94064000 tx=0x7fbc9406a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.023+0000 7fbcaa3c6640 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b71b0b0c0 0x563b71ba50e0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcaa3c6640 1 -- 172.21.15.40:0/3021394612 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000000 [delete] v99'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbc9c006140 con 0x563b71b0b0c0 2024-04-26T21:42:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b71b0b0c0 msgr2=0x563b71ba50e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b71b0b0c0 0x563b71ba50e0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc8c06b2d0 msgr2=0x7fbc8c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc8c06b2d0 0x7fbc8c06d790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbc94064000 tx=0x7fbc9406a040 comp rx=0 tx=0).stop 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 msgr2=0x563b71a7bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 --2- 172.21.15.40:0/3021394612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b71b07200 0x563b71a7bb50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbca4010510 tx=0x7fbca4010910 comp rx=0 tx=0).stop 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcaa3c6640 1 -- 172.21.15.40:0/3021394612 reap_dead start 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 shutdown_connections 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 >> 172.21.15.40:0/3021394612 conn(0x563b71a608a0 msgr2=0x563b71b11bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 shutdown_connections 2024-04-26T21:42:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.024+0000 7fbcac0cbf40 1 -- 172.21.15.40:0/3021394612 wait complete. 2024-04-26T21:42:55.029 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:42:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.172+0000 7ff54a967640 1 Processor -- start 2024-04-26T21:42:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.172+0000 7ff54a967640 1 -- start start 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff54a967640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff54415f550 0x7ff5440d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff54a967640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5440d3bc0 0x7ff5440d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff54a967640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5440d1040 con 0x7ff54415f170 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff54a967640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff5440c9430 con 0x7ff5440d3bc0 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff54a967640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5440c9680 con 0x7ff54415f550 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff548f04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5440d3bc0 0x7ff5440d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff543fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff54415f170 0x7ff5440d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35910/0 (socket says 172.21.15.40:35910) 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff543fff640 1 -- 172.21.15.40:0/3814884884 learned_addr learned my addr 172.21.15.40:0/3814884884 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff5437fe640 1 --2- 172.21.15.40:0/3814884884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff54415f550 0x7ff5440d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff548f04640 1 -- 172.21.15.40:0/3814884884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff54415f550 msgr2=0x7ff5440d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff548f04640 1 --2- 172.21.15.40:0/3814884884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff54415f550 0x7ff5440d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff548f04640 1 -- 172.21.15.40:0/3814884884 >> v1:172.21.15.40:6789/0 conn(0x7ff54415f170 legacy=0x7ff5440d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff548f04640 1 -- 172.21.15.40:0/3814884884 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5440c9960 con 0x7ff5440d3bc0 2024-04-26T21:42:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.173+0000 7ff542ffd640 1 -- 172.21.15.40:0/3814884884 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483054926 0 0) 0x7ff52c001110 con 0x7ff54415f170 2024-04-26T21:42:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.174+0000 7ff548f04640 1 --2- 172.21.15.40:0/3814884884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440d3bc0 0x7ff5440d2cb0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009d90 tx=0x7ff53c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cce701db70d8af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.174+0000 7ff542ffd640 1 -- 172.21.15.40:0/3814884884 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c007740 con 0x7ff5440d3bc0 2024-04-26T21:42:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.174+0000 7ff542ffd640 1 -- 172.21.15.40:0/3814884884 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53c0078e0 con 0x7ff5440d3bc0 2024-04-26T21:42:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.174+0000 7ff542ffd640 1 -- 172.21.15.40:0/3814884884 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c007cb0 con 0x7ff5440d3bc0 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.175+0000 7ff54a967640 1 -- 172.21.15.40:0/3814884884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440d3bc0 msgr2=0x7ff5440d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.175+0000 7ff54a967640 1 --2- 172.21.15.40:0/3814884884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440d3bc0 0x7ff5440d2cb0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009d90 tx=0x7ff53c02a660 comp rx=0 tx=0).stop 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.175+0000 7ff54a967640 1 -- 172.21.15.40:0/3814884884 shutdown_connections 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.175+0000 7ff54a967640 1 --2- 172.21.15.40:0/3814884884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440d3bc0 0x7ff5440d2cb0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.175+0000 7ff54a967640 1 --2- 172.21.15.40:0/3814884884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff54415f550 0x7ff5440d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.175+0000 7ff54a967640 1 -- 172.21.15.40:0/3814884884 >> 172.21.15.40:0/3814884884 conn(0x7ff5440d4f60 msgr2=0x7ff5440d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.175+0000 7ff54a967640 1 -- 172.21.15.40:0/3814884884 shutdown_connections 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.175+0000 7ff54a967640 1 -- 172.21.15.40:0/3814884884 wait complete. 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.176+0000 7ff54a967640 1 Processor -- start 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.176+0000 7ff54a967640 1 -- start start 2024-04-26T21:42:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff54a967640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5440d3bc0 0x7ff5440b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff54a967640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff54415f170 0x7ff5440ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff54a967640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440ce170 0x7ff5440ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff54a967640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440ca5c0 con 0x7ff5440ce170 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff54a967640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440ca440 con 0x7ff54415f170 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff54a967640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5440ca740 con 0x7ff5440d3bc0 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff548f04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440ce170 0x7ff5440ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff5437fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff54415f170 0x7ff5440ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff548f04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440ce170 0x7ff5440ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37198/0 (socket says 172.21.15.40:37198) 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff548f04640 1 -- 172.21.15.40:0/1336904582 learned_addr learned my addr 172.21.15.40:0/1336904582 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff5437fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff54415f170 0x7ff5440ba0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:37154/0 (socket says 172.21.15.40:37154) 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff543fff640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5440d3bc0 0x7ff5440b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff548f04640 1 -- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5440d3bc0 msgr2=0x7ff5440b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff548f04640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5440d3bc0 0x7ff5440b9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff548f04640 1 -- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff54415f170 msgr2=0x7ff5440ba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff548f04640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff54415f170 0x7ff5440ba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff548f04640 1 -- 172.21.15.40:0/1336904582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5441f8070 con 0x7ff5440ce170 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.177+0000 7ff5437fe640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff54415f170 0x7ff5440ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.178+0000 7ff548f04640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440ce170 0x7ff5440ce620 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009eb0 tx=0x7ff53c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.178+0000 7ff540ff9640 1 -- 172.21.15.40:0/1336904582 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c040070 con 0x7ff5440ce170 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.178+0000 7ff540ff9640 1 -- 172.21.15.40:0/1336904582 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53c031bc0 con 0x7ff5440ce170 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.178+0000 7ff540ff9640 1 -- 172.21.15.40:0/1336904582 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c009450 con 0x7ff5440ce170 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.178+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5441f8350 con 0x7ff5440ce170 2024-04-26T21:42:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.178+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5440ceb60 con 0x7ff5440ce170 2024-04-26T21:42:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.179+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5440694f0 con 0x7ff5440ce170 2024-04-26T21:42:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.179+0000 7ff540ff9640 1 -- 172.21.15.40:0/1336904582 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff53c039040 con 0x7ff5440ce170 2024-04-26T21:42:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.180+0000 7ff540ff9640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff52406c8f0 0x7ff52406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.180+0000 7ff540ff9640 1 -- 172.21.15.40:0/1336904582 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6561+0+0 (secure 0 0 0) 0x7ff53c0ad4b0 con 0x7ff5440ce170 2024-04-26T21:42:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.180+0000 7ff543fff640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff52406c8f0 0x7ff52406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.180+0000 7ff543fff640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff52406c8f0 0x7ff52406edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000f60 tx=0x7ff52c007b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.184+0000 7ff540ff9640 1 -- 172.21.15.40:0/1336904582 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff53c07afc0 con 0x7ff5440ce170 2024-04-26T21:42:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.407+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff544074590 con 0x7ff5440ce170 2024-04-26T21:42:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.644+0000 7ff540ff9640 1 -- 172.21.15.40:0/1336904582 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v114) ==== 162+0+0 (secure 0 0 0) 0x7ff53c07f2c0 con 0x7ff5440ce170 2024-04-26T21:42:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:42:55.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff52406c8f0 msgr2=0x7ff52406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff52406c8f0 0x7ff52406edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000f60 tx=0x7ff52c007b50 comp rx=0 tx=0).stop 2024-04-26T21:42:55.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440ce170 msgr2=0x7ff5440ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440ce170 0x7ff5440ce620 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009eb0 tx=0x7ff53c004a40 comp rx=0 tx=0).stop 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 shutdown_connections 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff52406c8f0 0x7ff52406edb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5440ce170 0x7ff5440ce620 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff54415f170 0x7ff5440ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 --2- 172.21.15.40:0/1336904582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5440d3bc0 0x7ff5440b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 >> 172.21.15.40:0/1336904582 conn(0x7ff5440d4f60 msgr2=0x7ff5440d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 shutdown_connections 2024-04-26T21:42:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.646+0000 7ff54a967640 1 -- 172.21.15.40:0/1336904582 wait complete. 2024-04-26T21:42:55.651 INFO:tasks.ceph.mds.d.smithi184.stderr:2024-04-26T21:42:55.650+0000 7f875ed0f640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-04-26T21:42:55.651 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:55.650+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:42:55.651 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:55.651+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:42:55.657 INFO:tasks.ceph.mds.d.smithi184.stderr: -13> 2024-04-26T21:42:55.650+0000 7f875ed0f640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-04-26T21:42:55.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:55.683 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:42:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.761+0000 7faa53a23640 1 Processor -- start 2024-04-26T21:42:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.762+0000 7faa53a23640 1 -- start start 2024-04-26T21:42:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.762+0000 7faa53a23640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faa4c163610 0x7faa4c0d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.762+0000 7faa53a23640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4c0d1a70 con 0x7faa4c15f5a0 2024-04-26T21:42:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.762+0000 7faa53a23640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4c0d1d50 con 0x7faa4c0d2830 2024-04-26T21:42:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.762+0000 7faa53a23640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faa4c163d30 con 0x7faa4c163610 2024-04-26T21:42:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.762+0000 7faa517bf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa4c15f5a0 0x7faa4c162f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35922/0 (socket says 172.21.15.40:35922) 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.762+0000 7faa517bf640 1 -- 172.21.15.40:0/299986061 learned_addr learned my addr 172.21.15.40:0/299986061 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1254670382 0 0) 0x7faa4c0d1a70 con 0x7faa4c15f5a0 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa34003520 con 0x7faa4c15f5a0 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa50fbe640 1 --2- 172.21.15.40:0/299986061 >> v2:172.21.15.184:3300/0 conn(0x7faa4c163610 0x7faa4c0d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1109992628 0 0) 0x7faa34003520 con 0x7faa4c15f5a0 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 >> v2:172.21.15.184:3300/0 conn(0x7faa4c163610 msgr2=0x7faa4c0d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 --2- 172.21.15.40:0/299986061 >> v2:172.21.15.184:3300/0 conn(0x7faa4c163610 0x7faa4c0d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 >> v1:172.21.15.184:6789/0 conn(0x7faa4c0d2830 legacy=0x7faa4c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c0d2030 con 0x7faa4c15f5a0 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7faa44001670 con 0x7faa4c15f5a0 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 >> v1:172.21.15.40:6789/0 conn(0x7faa4c15f5a0 legacy=0x7faa4c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa34004800 0x7faa34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa340051c0 0x7faa3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa3400c130 con 0x7faa340039b0 2024-04-26T21:42:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa3400c2e0 con 0x7faa34004800 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa3400c4e0 con 0x7faa340051c0 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa517bf640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa50fbe640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa34004800 0x7faa34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa51fc0640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa340051c0 0x7faa3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa517bf640 1 -- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa340051c0 msgr2=0x7faa3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa517bf640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa340051c0 0x7faa3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa517bf640 1 -- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa34004800 msgr2=0x7faa34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa517bf640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa34004800 0x7faa34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa517bf640 1 -- 172.21.15.40:0/299986061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa34003520 con 0x7faa340039b0 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa50fbe640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa34004800 0x7faa34004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.763+0000 7faa51fc0640 1 -- 172.21.15.40:0/299986061 reap_dead start 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.764+0000 7faa517bf640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7faa4400c290 tx=0x7faa34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99f8885aaffd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.764+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44007d30 con 0x7faa340039b0 2024-04-26T21:42:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.764+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa440058d0 con 0x7faa340039b0 2024-04-26T21:42:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.764+0000 7faa4bfff640 1 -- 172.21.15.40:0/299986061 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44005ce0 con 0x7faa340039b0 2024-04-26T21:42:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.765+0000 7faa53a23640 1 -- 172.21.15.40:0/299986061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa340039b0 msgr2=0x7faa34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.765+0000 7faa53a23640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7faa4400c290 tx=0x7faa34005830 comp rx=0 tx=0).stop 2024-04-26T21:42:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.765+0000 7faa53a23640 1 -- 172.21.15.40:0/299986061 shutdown_connections 2024-04-26T21:42:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.765+0000 7faa53a23640 1 --2- 172.21.15.40:0/299986061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.765+0000 7faa53a23640 1 -- 172.21.15.40:0/299986061 >> 172.21.15.40:0/299986061 conn(0x7faa4c0d5190 msgr2=0x7faa4c0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.765+0000 7faa53a23640 1 -- 172.21.15.40:0/299986061 shutdown_connections 2024-04-26T21:42:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.765+0000 7faa53a23640 1 -- 172.21.15.40:0/299986061 wait complete. 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.766+0000 7faa53a23640 1 Processor -- start 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.766+0000 7faa53a23640 1 -- start start 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa53a23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa340039b0 0x7faa4c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa53a23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa4c0b5720 0x7faa4c0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa53a23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4c0cdf30 0x7faa4c0b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa53a23640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c0d3c20 con 0x7faa4c0cdf30 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa53a23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c0d4190 con 0x7faa340039b0 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa53a23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa4c0d3da0 con 0x7faa4c0b5720 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4c0cdf30 0x7faa4c0b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4c0cdf30 0x7faa4c0b94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37220/0 (socket says 172.21.15.40:37220) 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 -- 172.21.15.40:0/2125192598 learned_addr learned my addr 172.21.15.40:0/2125192598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa517bf640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa340039b0 0x7faa4c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa50fbe640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa4c0b5720 0x7faa4c0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 -- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa4c0b5720 msgr2=0x7faa4c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa4c0b5720 0x7faa4c0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 -- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa340039b0 msgr2=0x7faa4c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa340039b0 0x7faa4c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 -- 172.21.15.40:0/2125192598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c0cf300 con 0x7faa4c0cdf30 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.767+0000 7faa51fc0640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4c0cdf30 0x7faa4c0b94b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7faa3c007fd0 tx=0x7faa3c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.768+0000 7faa49ffb640 1 -- 172.21.15.40:0/2125192598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c003970 con 0x7faa4c0cdf30 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.768+0000 7faa49ffb640 1 -- 172.21.15.40:0/2125192598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3c018070 con 0x7faa4c0cdf30 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.768+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c0cd450 con 0x7faa4c0cdf30 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.768+0000 7faa49ffb640 1 -- 172.21.15.40:0/2125192598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c003d70 con 0x7faa4c0cdf30 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.768+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c0cda10 con 0x7faa4c0cdf30 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.769+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4c0694f0 con 0x7faa4c0cdf30 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.769+0000 7faa49ffb640 1 -- 172.21.15.40:0/2125192598 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa3c0050d0 con 0x7faa4c0cdf30 2024-04-26T21:42:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.770+0000 7faa49ffb640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa3806c920 0x7faa3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.770+0000 7faa49ffb640 1 -- 172.21.15.40:0/2125192598 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6647+0+0 (secure 0 0 0) 0x7faa3c08bc60 con 0x7faa4c0cdf30 2024-04-26T21:42:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.770+0000 7faa517bf640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa3806c920 0x7faa3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.770+0000 7faa517bf640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa3806c920 0x7faa3806ede0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa44033000 tx=0x7faa44009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.774+0000 7faa49ffb640 1 -- 172.21.15.40:0/2125192598 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa3c059c40 con 0x7faa4c0cdf30 2024-04-26T21:42:55.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.996+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa4c0cdbe0 con 0x7faa4c0cdf30 2024-04-26T21:42:55.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.997+0000 7faa49ffb640 1 -- 172.21.15.40:0/2125192598 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+4350 (secure 0 0 0) 0x7faa3c059540 con 0x7faa4c0cdf30 2024-04-26T21:42:55.998 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:55.998 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":115,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:55.652134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:55.998 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 115 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa3806c920 msgr2=0x7faa3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa3806c920 0x7faa3806ede0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa44033000 tx=0x7faa44009030 comp rx=0 tx=0).stop 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4c0cdf30 msgr2=0x7faa4c0b94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4c0cdf30 0x7faa4c0b94b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7faa3c007fd0 tx=0x7faa3c00ec10 comp rx=0 tx=0).stop 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 shutdown_connections 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa3806c920 0x7faa3806ede0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4c0cdf30 0x7faa4c0b94b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa4c0b5720 0x7faa4c0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 --2- 172.21.15.40:0/2125192598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa340039b0 0x7faa4c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 >> 172.21.15.40:0/2125192598 conn(0x7faa4c0d5190 msgr2=0x7faa4c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 shutdown_connections 2024-04-26T21:42:56.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:55.999+0000 7faa53a23640 1 -- 172.21.15.40:0/2125192598 wait complete. 2024-04-26T21:42:56.010 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:42:56.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.110+0000 7fb31e9f7640 1 Processor -- start 2024-04-26T21:42:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.110+0000 7fb31e9f7640 1 -- start start 2024-04-26T21:42:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb31e9f7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb318162880 0x7fb3180d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb31e9f7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3180b5f20 0x7fb3180d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb31e9f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180d39f0 0x7fb3180b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb31e9f7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb3180d0580 con 0x7fb318162880 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb31e9f7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb3180b7e50 con 0x7fb3180b5f20 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb31e9f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3180d1800 con 0x7fb3180d39f0 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb317fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb318162880 0x7fb3180d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb317fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb318162880 0x7fb3180d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37236/0 (socket says 172.21.15.40:37236) 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb317fff640 1 -- 172.21.15.40:0/1503454085 learned_addr learned my addr 172.21.15.40:0/1503454085 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb317fff640 1 -- 172.21.15.40:0/1503454085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180d39f0 msgr2=0x7fb3180b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb317fff640 1 --2- 172.21.15.40:0/1503454085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180d39f0 0x7fb3180b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb317fff640 1 -- 172.21.15.40:0/1503454085 >> v2:172.21.15.184:3300/0 conn(0x7fb3180b5f20 msgr2=0x7fb3180d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb317fff640 1 --2- 172.21.15.40:0/1503454085 >> v2:172.21.15.184:3300/0 conn(0x7fb3180b5f20 0x7fb3180d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.111+0000 7fb317fff640 1 -- 172.21.15.40:0/1503454085 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3180d07d0 con 0x7fb318162880 2024-04-26T21:42:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.112+0000 7fb317fff640 1 --2- 172.21.15.40:0/1503454085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb318162880 0x7fb3180d2cb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fb31808d3d0 tx=0x7fb30002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709e3e138783a7e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.112+0000 7fb316ffd640 1 -- 172.21.15.40:0/1503454085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300037070 con 0x7fb318162880 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.112+0000 7fb316ffd640 1 -- 172.21.15.40:0/1503454085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb300032440 con 0x7fb318162880 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.112+0000 7fb316ffd640 1 -- 172.21.15.40:0/1503454085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300032810 con 0x7fb318162880 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 -- 172.21.15.40:0/1503454085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb318162880 msgr2=0x7fb3180d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/1503454085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb318162880 0x7fb3180d2cb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fb31808d3d0 tx=0x7fb30002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 -- 172.21.15.40:0/1503454085 shutdown_connections 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/1503454085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180d39f0 0x7fb3180b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/1503454085 >> v2:172.21.15.184:3300/0 conn(0x7fb3180b5f20 0x7fb3180d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/1503454085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb318162880 0x7fb3180d2cb0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 -- 172.21.15.40:0/1503454085 >> 172.21.15.40:0/1503454085 conn(0x7fb3180d4f60 msgr2=0x7fb3180d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 -- 172.21.15.40:0/1503454085 shutdown_connections 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.113+0000 7fb31e9f7640 1 -- 172.21.15.40:0/1503454085 wait complete. 2024-04-26T21:42:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.114+0000 7fb31e9f7640 1 Processor -- start 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.114+0000 7fb31e9f7640 1 -- start start 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb31e9f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b5f20 0x7fb3180b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb31e9f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3180d39f0 0x7fb31815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb31e9f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb318162880 0x7fb3180b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb31e9f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb318162c60 con 0x7fb3180d39f0 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb31e9f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3180d0d90 con 0x7fb318162880 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb31e9f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb318162de0 con 0x7fb3180b5f20 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb3177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3180d39f0 0x7fb31815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb3177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3180d39f0 0x7fb31815c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37242/0 (socket says 172.21.15.40:37242) 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb3177fe640 1 -- 172.21.15.40:0/75264813 learned_addr learned my addr 172.21.15.40:0/75264813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb317fff640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b5f20 0x7fb3180b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb31cf94640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb318162880 0x7fb3180b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb3177fe640 1 -- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b5f20 msgr2=0x7fb3180b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb3177fe640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b5f20 0x7fb3180b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb3177fe640 1 -- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb318162880 msgr2=0x7fb3180b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb3177fe640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb318162880 0x7fb3180b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb3177fe640 1 -- 172.21.15.40:0/75264813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3181f4d90 con 0x7fb3180d39f0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb317fff640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b5f20 0x7fb3180b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.115+0000 7fb31cf94640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb318162880 0x7fb3180b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.116+0000 7fb3177fe640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3180d39f0 0x7fb31815c020 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fb30800b410 tx=0x7fb30800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.116+0000 7fb314ff9640 1 -- 172.21.15.40:0/75264813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308015020 con 0x7fb3180d39f0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.116+0000 7fb314ff9640 1 -- 172.21.15.40:0/75264813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb308011850 con 0x7fb3180d39f0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.116+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3180b2010 con 0x7fb3180d39f0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.116+0000 7fb314ff9640 1 -- 172.21.15.40:0/75264813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308011c40 con 0x7fb3180d39f0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.116+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3180b25d0 con 0x7fb3180d39f0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.117+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3180694f0 con 0x7fb3180d39f0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.117+0000 7fb314ff9640 1 -- 172.21.15.40:0/75264813 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb308018020 con 0x7fb3180d39f0 2024-04-26T21:42:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.118+0000 7fb314ff9640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb2f806c8f0 0x7fb2f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.118+0000 7fb314ff9640 1 -- 172.21.15.40:0/75264813 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6647+0+0 (secure 0 0 0) 0x7fb30808bd90 con 0x7fb3180d39f0 2024-04-26T21:42:56.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.118+0000 7fb317fff640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb2f806c8f0 0x7fb2f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.119+0000 7fb317fff640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb2f806c8f0 0x7fb2f806edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb300034000 tx=0x7fb30002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.122+0000 7fb314ff9640 1 -- 172.21.15.40:0/75264813 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb30801f020 con 0x7fb3180d39f0 2024-04-26T21:42:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.352+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb318075320 con 0x7fb3180d39f0 2024-04-26T21:42:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.352+0000 7fb314ff9640 1 -- 172.21.15.40:0/75264813 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+4350 (secure 0 0 0) 0x7fb3080172c0 con 0x7fb3180d39f0 2024-04-26T21:42:56.353 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:56.353 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":115,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:55.652134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 115 2024-04-26T21:42:56.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb2f806c8f0 msgr2=0x7fb2f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb2f806c8f0 0x7fb2f806edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb300034000 tx=0x7fb30002e000 comp rx=0 tx=0).stop 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3180d39f0 msgr2=0x7fb31815c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3180d39f0 0x7fb31815c020 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fb30800b410 tx=0x7fb30800b8d0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 shutdown_connections 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb2f806c8f0 0x7fb2f806edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb318162880 0x7fb3180b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3180d39f0 0x7fb31815c020 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 --2- 172.21.15.40:0/75264813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b5f20 0x7fb3180b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 >> 172.21.15.40:0/75264813 conn(0x7fb3180d4f60 msgr2=0x7fb31815f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 shutdown_connections 2024-04-26T21:42:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.354+0000 7fb31e9f7640 1 -- 172.21.15.40:0/75264813 wait complete. 2024-04-26T21:42:56.366 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:42:56.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:56.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.466+0000 7f2d7731b640 1 Processor -- start 2024-04-26T21:42:56.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.466+0000 7f2d7731b640 1 -- start start 2024-04-26T21:42:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.467+0000 7f2d7731b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d700d3bc0 0x7f2d700d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.467+0000 7f2d7731b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 0x7f2d700b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.467+0000 7f2d7731b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d700b7e50 con 0x7f2d70162880 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.467+0000 7f2d7731b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2d700d0580 con 0x7f2d700d3bc0 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.467+0000 7f2d7731b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d700d1800 con 0x7f2d700b5f20 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.467+0000 7f2d748b6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d700d3bc0 0x7f2d700d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.467+0000 7f2d748b6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d700d3bc0 0x7f2d700d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37250/0 (socket says 172.21.15.40:37250) 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.467+0000 7f2d748b6640 1 -- 172.21.15.40:0/2557080406 learned_addr learned my addr 172.21.15.40:0/2557080406 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.468+0000 7f2d758b8640 1 --2- 172.21.15.40:0/2557080406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 0x7f2d700b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.468+0000 7f2d748b6640 1 -- 172.21.15.40:0/2557080406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 msgr2=0x7f2d700b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.468+0000 7f2d748b6640 1 --2- 172.21.15.40:0/2557080406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 0x7f2d700b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.468+0000 7f2d748b6640 1 -- 172.21.15.40:0/2557080406 >> v1:172.21.15.184:6789/0 conn(0x7f2d70162880 legacy=0x7f2d700d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.468+0000 7f2d748b6640 1 -- 172.21.15.40:0/2557080406 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d700d07d0 con 0x7f2d700d3bc0 2024-04-26T21:42:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.468+0000 7f2d748b6640 1 --2- 172.21.15.40:0/2557080406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d700d2cb0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009870 tx=0x7f2d6002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1763fe016de28a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d67fff640 1 -- 172.21.15.40:0/2557080406 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60037070 con 0x7f2d700d3bc0 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d67fff640 1 -- 172.21.15.40:0/2557080406 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d60032440 con 0x7f2d700d3bc0 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d67fff640 1 -- 172.21.15.40:0/2557080406 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60032810 con 0x7f2d700d3bc0 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d7731b640 1 -- 172.21.15.40:0/2557080406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 msgr2=0x7f2d700d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d7731b640 1 --2- 172.21.15.40:0/2557080406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d700d2cb0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009870 tx=0x7f2d6002a210 comp rx=0 tx=0).stop 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d7731b640 1 -- 172.21.15.40:0/2557080406 shutdown_connections 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d7731b640 1 --2- 172.21.15.40:0/2557080406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 0x7f2d700b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d7731b640 1 --2- 172.21.15.40:0/2557080406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d700d2cb0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d7731b640 1 -- 172.21.15.40:0/2557080406 >> 172.21.15.40:0/2557080406 conn(0x7f2d700d4f60 msgr2=0x7f2d700d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d7731b640 1 -- 172.21.15.40:0/2557080406 shutdown_connections 2024-04-26T21:42:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.469+0000 7f2d7731b640 1 -- 172.21.15.40:0/2557080406 wait complete. 2024-04-26T21:42:56.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.470+0000 7f2d7731b640 1 Processor -- start 2024-04-26T21:42:56.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.470+0000 7f2d7731b640 1 -- start start 2024-04-26T21:42:56.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d7731b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 0x7f2d701f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d7731b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d7006de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d7731b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d70162880 0x7f2d7006e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d7731b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70162c60 con 0x7f2d700d3bc0 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d7731b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700d0d90 con 0x7f2d70162880 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d7731b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d70162de0 con 0x7f2d700b5f20 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d748b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d7006de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d748b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d7006de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37256/0 (socket says 172.21.15.40:37256) 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d748b6640 1 -- 172.21.15.40:0/1546524412 learned_addr learned my addr 172.21.15.40:0/1546524412 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d758b8640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d70162880 0x7f2d7006e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.471+0000 7f2d750b7640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 0x7f2d701f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d748b6640 1 -- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 msgr2=0x7f2d701f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d748b6640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 0x7f2d701f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d748b6640 1 -- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d70162880 msgr2=0x7f2d7006e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d748b6640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d70162880 0x7f2d7006e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d748b6640 1 -- 172.21.15.40:0/1546524412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d701f4e50 con 0x7f2d700d3bc0 2024-04-26T21:42:56.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d748b6640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d7006de90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009870 tx=0x7f2d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d65ffb640 1 -- 172.21.15.40:0/1546524412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60041020 con 0x7f2d700d3bc0 2024-04-26T21:42:56.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d65ffb640 1 -- 172.21.15.40:0/1546524412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d600306c0 con 0x7f2d700d3bc0 2024-04-26T21:42:56.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d701f7ea0 con 0x7f2d700d3bc0 2024-04-26T21:42:56.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d65ffb640 1 -- 172.21.15.40:0/1546524412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60037040 con 0x7f2d700d3bc0 2024-04-26T21:42:56.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.472+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7015a350 con 0x7f2d700d3bc0 2024-04-26T21:42:56.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.473+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d700694f0 con 0x7f2d700d3bc0 2024-04-26T21:42:56.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.474+0000 7f2d65ffb640 1 -- 172.21.15.40:0/1546524412 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d60044020 con 0x7f2d700d3bc0 2024-04-26T21:42:56.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.474+0000 7f2d65ffb640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2d5406c8f0 0x7f2d5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.474+0000 7f2d65ffb640 1 -- 172.21.15.40:0/1546524412 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6647+0+0 (secure 0 0 0) 0x7f2d600ac740 con 0x7f2d700d3bc0 2024-04-26T21:42:56.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.474+0000 7f2d750b7640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2d5406c8f0 0x7f2d5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.475+0000 7f2d750b7640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2d5406c8f0 0x7f2d5406edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2d68004ea0 tx=0x7f2d68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.478+0000 7f2d65ffb640 1 -- 172.21.15.40:0/1546524412 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d6004b030 con 0x7f2d700d3bc0 2024-04-26T21:42:56.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.701+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d70074590 con 0x7f2d700d3bc0 2024-04-26T21:42:56.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.703+0000 7f2d65ffb640 1 -- 172.21.15.40:0/1546524412 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5179 (secure 0 0 0) 0x7f2d6007a7b0 con 0x7f2d700d3bc0 2024-04-26T21:42:56.703 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:42:56.704 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":116,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1327671065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1327671065},{"type":"v1","addr":"172.21.15.184:6836","nonce":1327671065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:55.652134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:42:56.704 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 116 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2d5406c8f0 msgr2=0x7f2d5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2d5406c8f0 0x7f2d5406edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2d68004ea0 tx=0x7f2d68009040 comp rx=0 tx=0).stop 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 msgr2=0x7f2d7006de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d7006de90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009870 tx=0x7f2d600046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 shutdown_connections 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2d5406c8f0 0x7f2d5406edb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d70162880 0x7f2d7006e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d700d3bc0 0x7f2d7006de90 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 --2- 172.21.15.40:0/1546524412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d700b5f20 0x7f2d701f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 >> 172.21.15.40:0/1546524412 conn(0x7f2d700d4f60 msgr2=0x7f2d700d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 shutdown_connections 2024-04-26T21:42:56.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.705+0000 7f2d7731b640 1 -- 172.21.15.40:0/1546524412 wait complete. 2024-04-26T21:42:56.716 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000000' led to DAMAGED state 2024-04-26T21:42:56.716 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000001' 2024-04-26T21:42:56.716 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:42:56.716 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:42:56.731 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:42:56.732 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:42:56.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:42:56.732 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:42:56.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:56.786 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:42:56.787 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:42:56.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.927+0000 7fcdd9186640 1 Processor -- start 2024-04-26T21:42:56.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.927+0000 7fcdd9186640 1 -- start start 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd9186640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcdd415f3a0 0x7fcdd40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd9186640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdd40d1770 0x7fcdd40d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd9186640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd40d2160 0x7fcdd40d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd9186640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd40d25d0 con 0x7fcdd40d1770 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd9186640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd40d0e20 con 0x7fcdd415f3a0 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd9186640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd40b1d70 con 0x7fcdd40d2160 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdd40d1770 0x7fcdd40d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdd40d1770 0x7fcdd40d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37262/0 (socket says 172.21.15.40:37262) 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd259c640 1 -- 172.21.15.40:0/3399957643 learned_addr learned my addr 172.21.15.40:0/3399957643 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd2d9d640 1 --2- 172.21.15.40:0/3399957643 >> v2:172.21.15.184:3300/0 conn(0x7fcdd415f3a0 0x7fcdd40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.928+0000 7fcdd359e640 1 --2- 172.21.15.40:0/3399957643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd40d2160 0x7fcdd40d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd259c640 1 -- 172.21.15.40:0/3399957643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd40d2160 msgr2=0x7fcdd40d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd259c640 1 --2- 172.21.15.40:0/3399957643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd40d2160 0x7fcdd40d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd259c640 1 -- 172.21.15.40:0/3399957643 >> v2:172.21.15.184:3300/0 conn(0x7fcdd415f3a0 msgr2=0x7fcdd40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd259c640 1 --2- 172.21.15.40:0/3399957643 >> v2:172.21.15.184:3300/0 conn(0x7fcdd415f3a0 0x7fcdd40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd259c640 1 -- 172.21.15.40:0/3399957643 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd40b2050 con 0x7fcdd40d1770 2024-04-26T21:42:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd359e640 1 --2- 172.21.15.40:0/3399957643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd40d2160 0x7fcdd40d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:42:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd259c640 1 --2- 172.21.15.40:0/3399957643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd40d1c20 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4073b50 tx=0x7fcdbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b445af59c8c373 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd1d9b640 1 -- 172.21.15.40:0/3399957643 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc037070 con 0x7fcdd40d1770 2024-04-26T21:42:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.929+0000 7fcdd1d9b640 1 -- 172.21.15.40:0/3399957643 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdbc032440 con 0x7fcdd40d1770 2024-04-26T21:42:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd1d9b640 1 -- 172.21.15.40:0/3399957643 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc032810 con 0x7fcdd40d1770 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 -- 172.21.15.40:0/3399957643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 msgr2=0x7fcdd40d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 --2- 172.21.15.40:0/3399957643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd40d1c20 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4073b50 tx=0x7fcdbc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 -- 172.21.15.40:0/3399957643 shutdown_connections 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 --2- 172.21.15.40:0/3399957643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd40d2160 0x7fcdd40d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 --2- 172.21.15.40:0/3399957643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd40d1c20 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 --2- 172.21.15.40:0/3399957643 >> v2:172.21.15.184:3300/0 conn(0x7fcdd415f3a0 0x7fcdd40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 -- 172.21.15.40:0/3399957643 >> 172.21.15.40:0/3399957643 conn(0x7fcdd40d4f90 msgr2=0x7fcdd40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 -- 172.21.15.40:0/3399957643 shutdown_connections 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.930+0000 7fcdd9186640 1 -- 172.21.15.40:0/3399957643 wait complete. 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.931+0000 7fcdd9186640 1 Processor -- start 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd9186640 1 -- start start 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd9186640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd9186640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd40d2160 0x7fcdd415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd9186640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd415f3a0 0x7fcdd40b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd9186640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd415f900 con 0x7fcdd40d1770 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd9186640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd415f780 con 0x7fcdd40d2160 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd9186640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd40d3390 con 0x7fcdd415f3a0 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd41532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37272/0 (socket says 172.21.15.40:37272) 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.932+0000 7fcdd2d9d640 1 -- 172.21.15.40:0/2145859090 learned_addr learned my addr 172.21.15.40:0/2145859090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd359e640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd415f3a0 0x7fcdd40b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd259c640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd40d2160 0x7fcdd415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd2d9d640 1 -- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd415f3a0 msgr2=0x7fcdd40b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd2d9d640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd415f3a0 0x7fcdd40b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd2d9d640 1 -- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd40d2160 msgr2=0x7fcdd415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd2d9d640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd40d2160 0x7fcdd415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd2d9d640 1 -- 172.21.15.40:0/2145859090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd41603f0 con 0x7fcdd40d1770 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd2d9d640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd41532a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d620 tx=0x7fcdcc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdcb7fe640 1 -- 172.21.15.40:0/2145859090 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00b620 con 0x7fcdd40d1770 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdcb7fe640 1 -- 172.21.15.40:0/2145859090 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdcc019070 con 0x7fcdd40d1770 2024-04-26T21:42:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd41606d0 con 0x7fcdd40d1770 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdcb7fe640 1 -- 172.21.15.40:0/2145859090 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00ba00 con 0x7fcdd40d1770 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.933+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd41616e0 con 0x7fcdd40d1770 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.934+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd40694f0 con 0x7fcdd40d1770 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.935+0000 7fcdcb7fe640 1 -- 172.21.15.40:0/2145859090 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdcc00bba0 con 0x7fcdd40d1770 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.935+0000 7fcdcb7fe640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdb406c8f0 0x7fcdb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.935+0000 7fcdcb7fe640 1 -- 172.21.15.40:0/2145859090 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6647+0+0 (secure 0 0 0) 0x7fcdcc08c270 con 0x7fcdd40d1770 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.935+0000 7fcdd259c640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdb406c8f0 0x7fcdb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.936+0000 7fcdd259c640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdb406c8f0 0x7fcdb406edb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc034000 tx=0x7fcdbc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:56.939+0000 7fcdcb7fe640 1 -- 172.21.15.40:0/2145859090 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdcc05a380 con 0x7fcdd40d1770 2024-04-26T21:42:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.162+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcdd40b6860 con 0x7fcdd40d1770 2024-04-26T21:42:57.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.709+0000 7fcdcb7fe640 1 -- 172.21.15.40:0/2145859090 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) ==== 161+0+0 (secure 0 0 0) 0x7fcdcc059c80 con 0x7fcdd40d1770 2024-04-26T21:42:57.710 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:42:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdb406c8f0 msgr2=0x7fcdb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdb406c8f0 0x7fcdb406edb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc034000 tx=0x7fcdbc02e000 comp rx=0 tx=0).stop 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 msgr2=0x7fcdd41532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd41532a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d620 tx=0x7fcdcc00dae0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 shutdown_connections 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcdb406c8f0 0x7fcdb406edb0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd415f3a0 0x7fcdd40b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd40d2160 0x7fcdd415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 --2- 172.21.15.40:0/2145859090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40d1770 0x7fcdd41532a0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 >> 172.21.15.40:0/2145859090 conn(0x7fcdd40d4f90 msgr2=0x7fcdd40d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.711+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 shutdown_connections 2024-04-26T21:42:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.712+0000 7fcdd9186640 1 -- 172.21.15.40:0/2145859090 wait complete. 2024-04-26T21:42:57.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:42:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.828+0000 7f8058f33640 1 Processor -- start 2024-04-26T21:42:57.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.828+0000 7f8058f33640 1 -- start start 2024-04-26T21:42:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f8058f33640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f80540d3bc0 0x7f80540d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f8058f33640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f80540b5f20 0x7f80540b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f8058f33640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80540b7e50 con 0x7f8054162880 2024-04-26T21:42:57.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f8058f33640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f80540d0580 con 0x7f80540d3bc0 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f8058f33640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f80540d1800 con 0x7f80540b5f20 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f804bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f80540d3bc0 0x7f80540d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f804bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f80540d3bc0 0x7f80540d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37274/0 (socket says 172.21.15.40:37274) 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f804bfff640 1 -- 172.21.15.40:0/4059246780 learned_addr learned my addr 172.21.15.40:0/4059246780 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f8052d9d640 1 --2- 172.21.15.40:0/4059246780 >> v2:172.21.15.184:3300/0 conn(0x7f80540b5f20 0x7f80540b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f804bfff640 1 -- 172.21.15.40:0/4059246780 >> v2:172.21.15.184:3300/0 conn(0x7f80540b5f20 msgr2=0x7f80540b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f804bfff640 1 --2- 172.21.15.40:0/4059246780 >> v2:172.21.15.184:3300/0 conn(0x7f80540b5f20 0x7f80540b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f804bfff640 1 -- 172.21.15.40:0/4059246780 >> v1:172.21.15.184:6789/0 conn(0x7f8054162880 legacy=0x7f80540d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.829+0000 7f804bfff640 1 -- 172.21.15.40:0/4059246780 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80540d07d0 con 0x7f80540d3bc0 2024-04-26T21:42:57.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.830+0000 7f804bfff640 1 --2- 172.21.15.40:0/4059246780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80540d3bc0 0x7f80540d2cb0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f805408f770 tx=0x7f803c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f92f6a934c920b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.830+0000 7f8051d9b640 1 -- 172.21.15.40:0/4059246780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c037070 con 0x7f80540d3bc0 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.830+0000 7f8051d9b640 1 -- 172.21.15.40:0/4059246780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803c032440 con 0x7f80540d3bc0 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.830+0000 7f8051d9b640 1 -- 172.21.15.40:0/4059246780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c032810 con 0x7f80540d3bc0 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.831+0000 7f8058f33640 1 -- 172.21.15.40:0/4059246780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80540d3bc0 msgr2=0x7f80540d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.831+0000 7f8058f33640 1 --2- 172.21.15.40:0/4059246780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80540d3bc0 0x7f80540d2cb0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f805408f770 tx=0x7f803c02a230 comp rx=0 tx=0).stop 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.831+0000 7f8058f33640 1 -- 172.21.15.40:0/4059246780 shutdown_connections 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.831+0000 7f8058f33640 1 --2- 172.21.15.40:0/4059246780 >> v2:172.21.15.184:3300/0 conn(0x7f80540b5f20 0x7f80540b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.831+0000 7f8058f33640 1 --2- 172.21.15.40:0/4059246780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80540d3bc0 0x7f80540d2cb0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.831+0000 7f8058f33640 1 -- 172.21.15.40:0/4059246780 >> 172.21.15.40:0/4059246780 conn(0x7f80540d4f60 msgr2=0x7f80540d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.831+0000 7f8058f33640 1 -- 172.21.15.40:0/4059246780 shutdown_connections 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.831+0000 7f8058f33640 1 -- 172.21.15.40:0/4059246780 wait complete. 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.832+0000 7f8058f33640 1 Processor -- start 2024-04-26T21:42:57.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.832+0000 7f8058f33640 1 -- start start 2024-04-26T21:42:57.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8058f33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80540b5f20 0x7f80540ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:57.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8058f33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80540d3bc0 0x7f80540b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:57.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8058f33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8054162880 0x7f80541f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:57.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8058f33640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8054162c60 con 0x7f8054162880 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8058f33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80540d0d90 con 0x7f80540d3bc0 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8058f33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8054162de0 con 0x7f80540b5f20 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8054162880 0x7f80541f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8054162880 0x7f80541f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37286/0 (socket says 172.21.15.40:37286) 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 -- 172.21.15.40:0/3730117534 learned_addr learned my addr 172.21.15.40:0/3730117534 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f804bfff640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80540d3bc0 0x7f80540b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f805259c640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80540b5f20 0x7f80540ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 -- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80540b5f20 msgr2=0x7f80540ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80540b5f20 0x7f80540ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 -- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80540d3bc0 msgr2=0x7f80540b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80540d3bc0 0x7f80540b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 -- 172.21.15.40:0/3730117534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80541f5540 con 0x7f8054162880 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f8052d9d640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8054162880 0x7f80541f4dc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f804c007e60 tx=0x7f804c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:57.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f804affd640 1 -- 172.21.15.40:0/3730117534 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c014420 con 0x7f8054162880 2024-04-26T21:42:57.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.833+0000 7f804affd640 1 -- 172.21.15.40:0/3730117534 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804c019070 con 0x7f8054162880 2024-04-26T21:42:57.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.834+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80541f5820 con 0x7f8054162880 2024-04-26T21:42:57.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.834+0000 7f804affd640 1 -- 172.21.15.40:0/3730117534 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c014820 con 0x7f8054162880 2024-04-26T21:42:57.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.834+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80541f6140 con 0x7f8054162880 2024-04-26T21:42:57.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.834+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80540694f0 con 0x7f8054162880 2024-04-26T21:42:57.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.835+0000 7f804affd640 1 -- 172.21.15.40:0/3730117534 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f804c0149c0 con 0x7f8054162880 2024-04-26T21:42:57.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.835+0000 7f804affd640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f803406c8f0 0x7f803406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:57.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.836+0000 7f804affd640 1 -- 172.21.15.40:0/3730117534 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6647+0+0 (secure 0 0 0) 0x7f804c08c6f0 con 0x7f8054162880 2024-04-26T21:42:57.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.836+0000 7f805259c640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f803406c8f0 0x7f803406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:57.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.836+0000 7f805259c640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f803406c8f0 0x7f803406edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8054161f20 tx=0x7f8044009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:57.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:57.840+0000 7f804affd640 1 -- 172.21.15.40:0/3730117534 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f804c05a800 con 0x7f8054162880 2024-04-26T21:42:58.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.101+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f80540cb530 con 0x7f8054162880 2024-04-26T21:42:58.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.715+0000 7f804affd640 1 -- 172.21.15.40:0/3730117534 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v118) ==== 101+0+0 (secure 0 0 0) 0x7f804c05a100 con 0x7f8054162880 2024-04-26T21:42:58.716 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:42:58.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f803406c8f0 msgr2=0x7f803406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f803406c8f0 0x7f803406edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8054161f20 tx=0x7f8044009040 comp rx=0 tx=0).stop 2024-04-26T21:42:58.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8054162880 msgr2=0x7f80541f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8054162880 0x7f80541f4dc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f804c007e60 tx=0x7f804c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 shutdown_connections 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f803406c8f0 0x7f803406edb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8054162880 0x7f80541f4dc0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80540d3bc0 0x7f80540b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 --2- 172.21.15.40:0/3730117534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80540b5f20 0x7f80540ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 >> 172.21.15.40:0/3730117534 conn(0x7f80540d4f60 msgr2=0x7f805415f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 shutdown_connections 2024-04-26T21:42:58.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.717+0000 7f8058f33640 1 -- 172.21.15.40:0/3730117534 wait complete. 2024-04-26T21:42:58.732 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:42:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.776+0000 7f5041e49f40 1 Processor -- start 2024-04-26T21:42:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.776+0000 7f5041e49f40 1 -- start start 2024-04-26T21:42:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5041e49f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280022d0 0x564a5a1e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5041e49f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a5a1a25d0 0x564a5a1d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5041e49f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a5a1ccb40 0x564a5a30cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5041e49f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564a5a14ae40 con 0x564a5a1ccb40 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5041e49f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564a5a14ee80 con 0x7f50280022d0 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5041e49f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564a5a261140 con 0x564a5a1a25d0 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5040144640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a5a1ccb40 0x564a5a30cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5040144640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a5a1ccb40 0x564a5a30cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59456/0 (socket says 172.21.15.40:59456) 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5040144640 1 -- 172.21.15.40:0/1209082795 learned_addr learned my addr 172.21.15.40:0/1209082795 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5040144640 1 -- 172.21.15.40:0/1209082795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a5a1a25d0 msgr2=0x564a5a1d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5040144640 1 --2- 172.21.15.40:0/1209082795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a5a1a25d0 0x564a5a1d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5040144640 1 -- 172.21.15.40:0/1209082795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280022d0 msgr2=0x564a5a1e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5040144640 1 --2- 172.21.15.40:0/1209082795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280022d0 0x564a5a1e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.777+0000 7f5040144640 1 -- 172.21.15.40:0/1209082795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a5a1e6f90 con 0x564a5a1ccb40 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5040144640 1 --2- 172.21.15.40:0/1209082795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a5a1ccb40 0x564a5a30cd40 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5038000f80 tx=0x7f5038006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508b7168a3589aec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f503e140640 1 -- 172.21.15.40:0/1209082795 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003660 con 0x564a5a1ccb40 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f503e140640 1 -- 172.21.15.40:0/1209082795 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038017070 con 0x564a5a1ccb40 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f503e140640 1 -- 172.21.15.40:0/1209082795 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003a00 con 0x564a5a1ccb40 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 -- 172.21.15.40:0/1209082795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a5a1ccb40 msgr2=0x564a5a30cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 --2- 172.21.15.40:0/1209082795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a5a1ccb40 0x564a5a30cd40 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5038000f80 tx=0x7f5038006430 comp rx=0 tx=0).stop 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 -- 172.21.15.40:0/1209082795 shutdown_connections 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 --2- 172.21.15.40:0/1209082795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564a5a1ccb40 0x564a5a30cd40 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 --2- 172.21.15.40:0/1209082795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a5a1a25d0 0x564a5a1d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 --2- 172.21.15.40:0/1209082795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280022d0 0x564a5a1e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 -- 172.21.15.40:0/1209082795 >> 172.21.15.40:0/1209082795 conn(0x564a5a1a1ab0 msgr2=0x564a5a1a1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 -- 172.21.15.40:0/1209082795 shutdown_connections 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.778+0000 7f5041e49f40 1 -- 172.21.15.40:0/1209082795 wait complete. 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.779+0000 7f5041e49f40 1 Processor -- start 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.779+0000 7f5041e49f40 1 -- start start 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5041e49f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5a1a25d0 0x564a5a25cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5041e49f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a5a1ccb40 0x564a5a1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5041e49f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280022d0 0x564a5a1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5041e49f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564a5a25e340 con 0x7f50280022d0 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5041e49f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564a5a1d2200 con 0x564a5a1a25d0 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5041e49f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564a5a1d0720 con 0x564a5a1ccb40 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5040144640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280022d0 0x564a5a1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5040144640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280022d0 0x564a5a1a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59464/0 (socket says 172.21.15.40:59464) 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 learned_addr learned my addr 172.21.15.40:0/784337632 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f503f943640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5a1a25d0 0x564a5a25cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f503f142640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a5a1ccb40 0x564a5a1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a5a1ccb40 msgr2=0x564a5a1a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5040144640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564a5a1ccb40 0x564a5a1a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5a1a25d0 msgr2=0x564a5a25cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5040144640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5a1a25d0 0x564a5a25cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.780+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a5a3099e0 con 0x7f50280022d0 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.781+0000 7f5040144640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280022d0 0x564a5a1a14c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5038014040 tx=0x7f5038001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.781+0000 7f503d13e640 1 -- 172.21.15.40:0/784337632 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038012bb0 con 0x7f50280022d0 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.781+0000 7f503d13e640 1 -- 172.21.15.40:0/784337632 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038020030 con 0x7f50280022d0 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.781+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564a5a309cc0 con 0x7f50280022d0 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.781+0000 7f503d13e640 1 -- 172.21.15.40:0/784337632 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038025420 con 0x7f50280022d0 2024-04-26T21:42:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.781+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a5a30c1c0 con 0x7f50280022d0 2024-04-26T21:42:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.782+0000 7f503d13e640 1 -- 172.21.15.40:0/784337632 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f503801b400 con 0x7f50280022d0 2024-04-26T21:42:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.782+0000 7f503d13e640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f502c06b2d0 0x7f502c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.783+0000 7f503d13e640 1 -- 172.21.15.40:0/784337632 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6647+0+0 (secure 0 0 0) 0x7f503808c710 con 0x7f50280022d0 2024-04-26T21:42:58.788 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.783+0000 7f503f943640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f502c06b2d0 0x7f502c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.783+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a5a25f890 0x564a5a30acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.783+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a30b1f0 con 0x564a5a25f890 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.783+0000 7f503f142640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a5a25f890 0x564a5a30acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.783+0000 7f503f943640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f502c06b2d0 0x7f502c06d790 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5030056000 tx=0x7f5030003040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.783+0000 7f503f142640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a5a25f890 0x564a5a30acb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.785+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v100'133 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.785+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a30b590 con 0x564a5a25f890 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.786+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v100'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.786+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31dff0 con 0x564a5a25f890 2024-04-26T21:42:58.790 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:42:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.787+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v100'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.787+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31f3a0 con 0x564a5a25f890 2024-04-26T21:42:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.788+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v100'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.788+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31f740 con 0x564a5a25f890 2024-04-26T21:42:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.789+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v100'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.789+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31e390 con 0x564a5a25f890 2024-04-26T21:42:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.790+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v100'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.790+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31e730 con 0x564a5a25f890 2024-04-26T21:42:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.791+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v100'139 uv113 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.791+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a31eb20 con 0x564a5a25f890 2024-04-26T21:42:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.793+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v100'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.793+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31eb20 con 0x564a5a25f890 2024-04-26T21:42:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.794+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v100'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.795 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:42:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.794+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.795+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v100'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.795+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.796+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v100'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.796+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.797+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v100'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.797+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.798+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v100'145 uv120 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.798+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.799+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v100'146 uv146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.799+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.800+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v100'147 uv147 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.801 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:42:58.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.800+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.801+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v100'148 uv148 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.801+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.802+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v100'149 uv149 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.802+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31ead0 con 0x564a5a25f890 2024-04-26T21:42:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.803+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v100'150 uv150 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.803+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.804+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v100'151 uv151 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.804+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.806+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v100'152 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.806+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.807+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v100'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.807+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.808+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v100'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.809 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:42:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.808+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a320af0 con 0x564a5a25f890 2024-04-26T21:42:58.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.809+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v100'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.809+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a320af0 con 0x564a5a25f890 2024-04-26T21:42:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.810+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v100'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.810+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.811+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v100'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.811+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.812+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v100'158 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.812+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.813+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v100'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.813+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.814+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v100'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.815 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:42:58.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.814+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a320af0 con 0x564a5a25f890 2024-04-26T21:42:58.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.815+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v100'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.815+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a320af0 con 0x564a5a25f890 2024-04-26T21:42:58.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.816+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v100'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.816+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a31fae0 con 0x564a5a25f890 2024-04-26T21:42:58.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.817+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v100'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a25f890 2024-04-26T21:42:58.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.817+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564a5a31fff0 0x564a5a3223e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.817+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a322920 con 0x564a5a31fff0 2024-04-26T21:42:58.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.818+0000 7f503f943640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564a5a31fff0 0x564a5a3223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.818+0000 7f503f943640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564a5a31fff0 0x564a5a3223e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.819+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v100'109 uv91 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.819+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a322cc0 con 0x564a5a31fff0 2024-04-26T21:42:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.821+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v100'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.821+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a323060 con 0x564a5a31fff0 2024-04-26T21:42:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.822+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v100'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.823 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:42:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.822+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a3238b0 con 0x564a5a31fff0 2024-04-26T21:42:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.823+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v100'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.823+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a323c50 con 0x564a5a31fff0 2024-04-26T21:42:58.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.824+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v100'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.824+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a323ff0 con 0x564a5a31fff0 2024-04-26T21:42:58.826 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:42:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.826+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v100'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.826+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a324390 con 0x564a5a31fff0 2024-04-26T21:42:58.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.827+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v100'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.827+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a324730 con 0x564a5a31fff0 2024-04-26T21:42:58.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.828+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v100'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.828+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a324730 con 0x564a5a31fff0 2024-04-26T21:42:58.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.829+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000001 [create] v100'117 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.829+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a324730 con 0x564a5a31fff0 2024-04-26T21:42:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.830+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000001 [delete] v100'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.830+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a324730 con 0x564a5a31fff0 2024-04-26T21:42:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.831+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v100'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.832 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:42:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.831+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a328df0 con 0x564a5a31fff0 2024-04-26T21:42:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.833+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [write 0~18075] v100'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.833+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a328df0 con 0x564a5a31fff0 2024-04-26T21:42:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.834+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v100'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.834+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a324730 con 0x564a5a31fff0 2024-04-26T21:42:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.835+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 606.00000000 [create] v100'122 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.835+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a324730 con 0x564a5a31fff0 2024-04-26T21:42:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.836+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 606.00000000 [delete] v100'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.836+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a324730 con 0x564a5a31fff0 2024-04-26T21:42:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.838+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v100'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.838 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:42:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.838+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a325740 con 0x564a5a31fff0 2024-04-26T21:42:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.839+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v100'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.839+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a325740 con 0x564a5a31fff0 2024-04-26T21:42:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.840+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v100'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.840+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a324730 con 0x564a5a31fff0 2024-04-26T21:42:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.841+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v100'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a31fff0 2024-04-26T21:42:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.841+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564a5a324c40 0x564a5a327030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.841+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a327570 con 0x564a5a324c40 2024-04-26T21:42:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.841+0000 7f503f142640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564a5a324c40 0x564a5a327030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.842+0000 7f503f142640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564a5a324c40 0x564a5a327030 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.843+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(51 607.00000000 [create] v100'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.843+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a327570 con 0x564a5a324c40 2024-04-26T21:42:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.844+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(52 607.00000000 [delete] v100'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.844+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a327570 con 0x564a5a324c40 2024-04-26T21:42:58.846 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:42:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.845+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(53 607.00000000 [create] v100'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.845+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a328580 con 0x564a5a324c40 2024-04-26T21:42:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.846+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v100'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.846+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a328580 con 0x564a5a324c40 2024-04-26T21:42:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.847+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v100'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.847+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a327570 con 0x564a5a324c40 2024-04-26T21:42:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.848+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v100'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.848+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564a5a327a80 0x564a5a329e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.848+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32a4b0 con 0x564a5a327a80 2024-04-26T21:42:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.849+0000 7f5040144640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564a5a327a80 0x564a5a329e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.849+0000 7f5040144640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564a5a327a80 0x564a5a329e70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.850+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v100'22 uv21 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.850+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a32a9b0 con 0x564a5a327a80 2024-04-26T21:42:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.852+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v100'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.852+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32aeb0 con 0x564a5a327a80 2024-04-26T21:42:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.853+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v100'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.854 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:42:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.853+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32b770 con 0x564a5a327a80 2024-04-26T21:42:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.855+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v100'25 uv25 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.856+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564a5a32c020 0x564a5a32e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.856+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32e990 con 0x564a5a32c020 2024-04-26T21:42:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.856+0000 7f503f943640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564a5a32c020 0x564a5a32e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.856+0000 7f503f943640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564a5a32c020 0x564a5a32e410 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.858+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(61 608.00000000 [create] v100'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.858+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a32ed60 con 0x564a5a32c020 2024-04-26T21:42:58.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.859+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(62 608.00000000 [delete] v100'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.859+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32ed60 con 0x564a5a32c020 2024-04-26T21:42:58.861 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:42:58.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.861+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(63 608.00000000 [create] v100'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.861+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fd70 con 0x564a5a32c020 2024-04-26T21:42:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.862+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v100'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.862+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fd70 con 0x564a5a32c020 2024-04-26T21:42:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.863+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v100'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.863+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32ed60 con 0x564a5a32c020 2024-04-26T21:42:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.865+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v100'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.865+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32ed60 con 0x564a5a327a80 2024-04-26T21:42:58.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.866+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(67 604.00000000 [create] v100'48 uv42 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.866+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a32f100 con 0x564a5a327a80 2024-04-26T21:42:58.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.867+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(68 604.00000000 [delete] v100'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.867+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f4a0 con 0x564a5a327a80 2024-04-26T21:42:58.870 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:42:58.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.869+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(69 604.00000000 [create] v100'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.869+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a3308b0 con 0x564a5a327a80 2024-04-26T21:42:58.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.872+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v100'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.872+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a330870 con 0x564a5a327a80 2024-04-26T21:42:58.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.873+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v100'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.873+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a330870 con 0x564a5a327a80 2024-04-26T21:42:58.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.875+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v100'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.875+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f840 con 0x564a5a327a80 2024-04-26T21:42:58.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.876+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(73 500.00000000 [create] v100'54 uv47 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.876+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a32f880 con 0x564a5a327a80 2024-04-26T21:42:58.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.877+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(74 500.00000000 [delete] v100'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.877+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f840 con 0x564a5a327a80 2024-04-26T21:42:58.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.878+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(75 500.00000000 [create] v100'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.879 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:42:58.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.878+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f840 con 0x564a5a327a80 2024-04-26T21:42:58.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.879+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v100'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.879+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f840 con 0x564a5a327a80 2024-04-26T21:42:58.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.880+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v100'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.880+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f840 con 0x564a5a327a80 2024-04-26T21:42:58.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.881+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(78 mds_snaptable [create] v100'22 uv21 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.881+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a32fc30 con 0x564a5a327a80 2024-04-26T21:42:58.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.882+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(79 mds_snaptable [delete] v100'23 uv23 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.882+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f8d0 con 0x564a5a327a80 2024-04-26T21:42:58.883 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:42:58.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.883+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(80 mds_snaptable [create] v100'24 uv24 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.883+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f8d0 con 0x564a5a327a80 2024-04-26T21:42:58.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.884+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(81 mds_snaptable [write 0~70] v100'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.884+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f880 con 0x564a5a327a80 2024-04-26T21:42:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.885+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v100'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f503802c070 con 0x564a5a327a80 2024-04-26T21:42:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.885+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f880 con 0x564a5a32c020 2024-04-26T21:42:58.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.886+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(83 605.00000000 [create] v100'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.886+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a32f880 con 0x564a5a32c020 2024-04-26T21:42:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.887+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(84 605.00000000 [delete] v100'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.887+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32f880 con 0x564a5a32c020 2024-04-26T21:42:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.890+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(85 605.00000000 [create] v100'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.891 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:42:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.890+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a330d00 con 0x564a5a32c020 2024-04-26T21:42:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.891+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v100'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.891+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a330d00 con 0x564a5a32c020 2024-04-26T21:42:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.892+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v100'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.892+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.893+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(88 605.00000000 [omap-set-header] v100'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.893+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.894+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(89 mds0_inotable [create] v100'44 uv38 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.894+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.895+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(90 mds0_inotable [delete] v100'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.895+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.896+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(91 mds0_inotable [create] v100'46 uv46 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.897 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:42:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.896+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.897+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(92 mds0_inotable [write 0~34] v100'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.897+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.898+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v100'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.898+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.899+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(94 100.00000000 [create] v100'49 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.899+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.900+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(95 100.00000000 [delete] v100'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.900+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.902+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(96 100.00000000 [create] v100'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.903 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:42:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.902+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a32fcf0 con 0x564a5a32c020 2024-04-26T21:42:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.903+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(97 100.00000000 [omap-set-header] v100'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.903+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332b00 con 0x564a5a32c020 2024-04-26T21:42:58.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.904+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v100'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.904+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564a5a3306b0 0x564a5a332f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.904+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a3306b0 2024-04-26T21:42:58.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.904+0000 7f503f142640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564a5a3306b0 0x564a5a332f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.905+0000 7f503f142640 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564a5a3306b0 0x564a5a332f40 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.906+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(99 mds0_sessionmap [create] v100'22 uv21 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a3306b0 2024-04-26T21:42:58.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.906+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a3306b0 2024-04-26T21:42:58.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.907+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v100'23 uv23 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a3306b0 2024-04-26T21:42:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.907+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a3306b0 2024-04-26T21:42:58.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.909+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(101 mds0_sessionmap [create] v100'24 uv24 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a3306b0 2024-04-26T21:42:58.910 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:42:58.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.910+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a3306b0 2024-04-26T21:42:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.912+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v100'25 uv25 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a3306b0 2024-04-26T21:42:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.912+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a32c020 2024-04-26T21:42:58.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.913+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(103 609.00000000 [create] v100'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.913+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a32c020 2024-04-26T21:42:58.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.915+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(104 609.00000000 [delete] v100'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.915+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a32c020 2024-04-26T21:42:58.916 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:42:58.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.916+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(105 609.00000000 [create] v100'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.916+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a32c020 2024-04-26T21:42:58.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.917+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v100'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.917+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a32c020 2024-04-26T21:42:58.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.918+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v100'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.918+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a32c020 2024-04-26T21:42:58.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.919+0000 7f503f943640 1 -- 172.21.15.40:0/784337632 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(108 609.00000000 [omap-set-header] v100'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5030067040 con 0x564a5a32c020 2024-04-26T21:42:58.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.919+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a324c40 2024-04-26T21:42:58.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.920+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v100'22 uv21 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.920+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a324c40 2024-04-26T21:42:58.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.921+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.921+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a332aa0 con 0x564a5a324c40 2024-04-26T21:42:58.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.922+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.923 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:42:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.922+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.923+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.923+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.924+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.924+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.925+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v100'57 uv49 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.925+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.927+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v100'58 uv58 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.927+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.928+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v100'59 uv59 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.929 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:42:58.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.928+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.929+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v100'60 uv60 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.929+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.930+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v100'61 uv61 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.930+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.931+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v100'62 uv56 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.931+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.932+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v100'63 uv63 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.932+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.934 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:42:58.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.933+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v100'64 uv64 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.933+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a334490 con 0x564a5a324c40 2024-04-26T21:42:58.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.934+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v100'65 uv65 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.934+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a334490 con 0x564a5a324c40 2024-04-26T21:42:58.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.935+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v100'66 uv66 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.935+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333480 con 0x564a5a324c40 2024-04-26T21:42:58.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.936+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v100'67 uv67 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.936+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564a5a333930 con 0x564a5a324c40 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f503f142640 1 -- 172.21.15.40:0/784337632 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v100'68 uv68 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5034009770 con 0x564a5a324c40 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564a5a31fff0 msgr2=0x564a5a3223e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564a5a31fff0 0x564a5a3223e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a5a25f890 msgr2=0x564a5a30acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564a5a25f890 0x564a5a30acb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564a5a327a80 msgr2=0x564a5a329e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564a5a327a80 0x564a5a329e70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564a5a3306b0 msgr2=0x564a5a332f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564a5a3306b0 0x564a5a332f40 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564a5a324c40 msgr2=0x564a5a327030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564a5a324c40 0x564a5a327030 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 reap_dead start 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564a5a32c020 msgr2=0x564a5a32e410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564a5a32c020 0x564a5a32e410 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 reap_dead start 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5040144640 1 -- 172.21.15.40:0/784337632 reap_dead start 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f502c06b2d0 msgr2=0x7f502c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f502c06b2d0 0x7f502c06d790 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5030056000 tx=0x7f5030003040 comp rx=0 tx=0).stop 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280022d0 msgr2=0x564a5a1a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280022d0 0x564a5a1a14c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5038014040 tx=0x7f5038001dc0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 shutdown_connections 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f502c06b2d0 0x7f502c06d790 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 --2- 172.21.15.40:0/784337632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280022d0 0x564a5a1a14c0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.937+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 >> 172.21.15.40:0/784337632 conn(0x564a5a1a1ab0 msgr2=0x564a5a1d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.938+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 shutdown_connections 2024-04-26T21:42:58.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.938+0000 7f5041e49f40 1 -- 172.21.15.40:0/784337632 wait complete. 2024-04-26T21:42:58.941 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000001 2024-04-26T21:42:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.982+0000 7f4accb66f40 1 Processor -- start 2024-04-26T21:42:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.982+0000 7f4accb66f40 1 -- start start 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4accb66f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb2ab420 0x5642cb290120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4accb66f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642cb332f70 0x5642cb290660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4accb66f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5642cb339b70 0x5642cb262c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4accb66f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5642cb208e40 con 0x5642cb2ab420 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4accb66f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5642cb2aa660 con 0x5642cb332f70 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4accb66f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5642cb2a7a70 con 0x5642cb339b70 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4aca662640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb2ab420 0x5642cb290120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4aca662640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb2ab420 0x5642cb290120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59482/0 (socket says 172.21.15.40:59482) 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4aca662640 1 -- 172.21.15.40:0/930386964 learned_addr learned my addr 172.21.15.40:0/930386964 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4aca662640 1 -- 172.21.15.40:0/930386964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5642cb339b70 msgr2=0x5642cb262c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4aca662640 1 --2- 172.21.15.40:0/930386964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5642cb339b70 0x5642cb262c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4aca662640 1 -- 172.21.15.40:0/930386964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642cb332f70 msgr2=0x5642cb290660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4aca662640 1 --2- 172.21.15.40:0/930386964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642cb332f70 0x5642cb290660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.983+0000 7f4aca662640 1 -- 172.21.15.40:0/930386964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cb262130 con 0x5642cb2ab420 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4aca662640 1 --2- 172.21.15.40:0/930386964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb2ab420 0x5642cb290120 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4abc002de0 tx=0x7f4abc00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396a155ce1998213 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4ac9660640 1 -- 172.21.15.40:0/930386964 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00f400 con 0x5642cb2ab420 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4ac9660640 1 -- 172.21.15.40:0/930386964 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4abc014070 con 0x5642cb2ab420 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4ac9660640 1 -- 172.21.15.40:0/930386964 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00f7e0 con 0x5642cb2ab420 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 -- 172.21.15.40:0/930386964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb2ab420 msgr2=0x5642cb290120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 --2- 172.21.15.40:0/930386964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb2ab420 0x5642cb290120 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4abc002de0 tx=0x7f4abc00b5b0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 -- 172.21.15.40:0/930386964 shutdown_connections 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 --2- 172.21.15.40:0/930386964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5642cb339b70 0x5642cb262c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 --2- 172.21.15.40:0/930386964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642cb332f70 0x5642cb290660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 --2- 172.21.15.40:0/930386964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb2ab420 0x5642cb290120 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 -- 172.21.15.40:0/930386964 >> 172.21.15.40:0/930386964 conn(0x5642cb28b8a0 msgr2=0x5642cb25fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 -- 172.21.15.40:0/930386964 shutdown_connections 2024-04-26T21:42:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.984+0000 7f4accb66f40 1 -- 172.21.15.40:0/930386964 wait complete. 2024-04-26T21:42:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.985+0000 7f4accb66f40 1 Processor -- start 2024-04-26T21:42:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.985+0000 7f4accb66f40 1 -- start start 2024-04-26T21:42:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.985+0000 7f4accb66f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5642cb2ab420 0x5642cb2a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.985+0000 7f4accb66f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb332f70 0x5642cb33b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.985+0000 7f4accb66f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642cb339b70 0x5642cb32ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.985+0000 7f4accb66f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5642cb262410 con 0x5642cb332f70 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.985+0000 7f4accb66f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5642cb290ba0 con 0x5642cb339b70 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.985+0000 7f4accb66f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5642cb2a4fb0 con 0x5642cb2ab420 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb332f70 0x5642cb33b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb332f70 0x5642cb33b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59490/0 (socket says 172.21.15.40:59490) 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 -- 172.21.15.40:0/4174006974 learned_addr learned my addr 172.21.15.40:0/4174006974 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4acae63640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642cb339b70 0x5642cb32ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4aca662640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5642cb2ab420 0x5642cb2a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 -- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5642cb2ab420 msgr2=0x5642cb2a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5642cb2ab420 0x5642cb2a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 -- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642cb339b70 msgr2=0x5642cb32ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642cb339b70 0x5642cb32ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 -- 172.21.15.40:0/4174006974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cb2a58b0 con 0x5642cb332f70 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac3fff640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb332f70 0x5642cb33b180 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4058040 tx=0x7f4ab4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac37fe640 1 -- 172.21.15.40:0/4174006974 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab405a050 con 0x5642cb332f70 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac37fe640 1 -- 172.21.15.40:0/4174006974 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab405b040 con 0x5642cb332f70 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5642cb2a5ce0 con 0x5642cb332f70 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4ac37fe640 1 -- 172.21.15.40:0/4174006974 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab40618b0 con 0x5642cb332f70 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.986+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cb2903c0 con 0x5642cb332f70 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.988+0000 7f4ac37fe640 1 -- 172.21.15.40:0/4174006974 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ab4061a50 con 0x5642cb332f70 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.988+0000 7f4ac37fe640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4aac06b2d0 0x7f4aac06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.988+0000 7f4ac37fe640 1 -- 172.21.15.40:0/4174006974 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6647+0+0 (secure 0 0 0) 0x7f4ab40cf370 con 0x5642cb332f70 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.988+0000 7f4aca662640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4aac06b2d0 0x7f4aac06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.988+0000 7f4accb66f40 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5642cb3d2d90 0x5642cb3d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.988+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5642cb3d36e0 con 0x5642cb3d2d90 2024-04-26T21:42:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.988+0000 7f4acae63640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5642cb3d2d90 0x5642cb3d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.989+0000 7f4acae63640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5642cb3d2d90 0x5642cb3d31a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.989+0000 7f4aca662640 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4aac06b2d0 0x7f4aac06d790 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00b5f0 tx=0x7f4abc01d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4acae63640 1 -- 172.21.15.40:0/4174006974 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000001 [delete] v100'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ac400c7f0 con 0x5642cb3d2d90 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5642cb3d2d90 msgr2=0x5642cb3d31a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5642cb3d2d90 0x5642cb3d31a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4aac06b2d0 msgr2=0x7f4aac06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4aac06b2d0 0x7f4aac06d790 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00b5f0 tx=0x7f4abc01d040 comp rx=0 tx=0).stop 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb332f70 msgr2=0x5642cb33b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 --2- 172.21.15.40:0/4174006974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5642cb332f70 0x5642cb33b180 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4058040 tx=0x7f4ab4005850 comp rx=0 tx=0).stop 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4acae63640 1 -- 172.21.15.40:0/4174006974 reap_dead start 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 shutdown_connections 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 >> 172.21.15.40:0/4174006974 conn(0x5642cb28b8a0 msgr2=0x5642cb3308c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 shutdown_connections 2024-04-26T21:42:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:58.990+0000 7f4accb66f40 1 -- 172.21.15.40:0/4174006974 wait complete. 2024-04-26T21:42:58.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:42:59.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.138+0000 7f3c9536a640 1 Processor -- start 2024-04-26T21:42:59.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.138+0000 7f3c9536a640 1 -- start start 2024-04-26T21:42:59.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.139+0000 7f3c9536a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c90162cd0 0x7f3c900b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.139+0000 7f3c9536a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c900d0890 con 0x7f3c900d2110 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.139+0000 7f3c9536a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c900d0b70 con 0x7f3c901628f0 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.139+0000 7f3c9536a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3c900d1130 con 0x7f3c90162cd0 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.139+0000 7f3c8e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c90162cd0 0x7f3c900b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.139+0000 7f3c8f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3c900d2110 0x7f3c900b51a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52646/0 (socket says 172.21.15.40:52646) 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.139+0000 7f3c8f7fe640 1 -- 172.21.15.40:0/2506493737 learned_addr learned my addr 172.21.15.40:0/2506493737 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.140+0000 7f3c8e7fc640 1 -- 172.21.15.40:0/2506493737 >> v1:172.21.15.184:6789/0 conn(0x7f3c901628f0 legacy=0x7f3c900d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.140+0000 7f3c8e7fc640 1 -- 172.21.15.40:0/2506493737 >> v1:172.21.15.40:6789/0 conn(0x7f3c900d2110 legacy=0x7f3c900b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.140+0000 7f3c8e7fc640 1 -- 172.21.15.40:0/2506493737 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c900d0e50 con 0x7f3c90162cd0 2024-04-26T21:42:59.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.140+0000 7f3c8dffb640 1 -- 172.21.15.40:0/2506493737 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2649469636 0 0) 0x7f3c900d0890 con 0x7f3c900d2110 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.140+0000 7f3c8e7fc640 1 --2- 172.21.15.40:0/2506493737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c90162cd0 0x7f3c900b4c60 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3c840097a0 tx=0x7f3c8402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625be0f172b270 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c8dffb640 1 -- 172.21.15.40:0/2506493737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84037070 con 0x7f3c90162cd0 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c8dffb640 1 -- 172.21.15.40:0/2506493737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c84032440 con 0x7f3c90162cd0 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c8dffb640 1 -- 172.21.15.40:0/2506493737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84032810 con 0x7f3c90162cd0 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c9536a640 1 -- 172.21.15.40:0/2506493737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c90162cd0 msgr2=0x7f3c900b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c9536a640 1 --2- 172.21.15.40:0/2506493737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c90162cd0 0x7f3c900b4c60 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3c840097a0 tx=0x7f3c8402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c9536a640 1 -- 172.21.15.40:0/2506493737 shutdown_connections 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c9536a640 1 --2- 172.21.15.40:0/2506493737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c90162cd0 0x7f3c900b4c60 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c9536a640 1 -- 172.21.15.40:0/2506493737 >> 172.21.15.40:0/2506493737 conn(0x7f3c900d4ff0 msgr2=0x7f3c900d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c9536a640 1 -- 172.21.15.40:0/2506493737 shutdown_connections 2024-04-26T21:42:59.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.141+0000 7f3c9536a640 1 -- 172.21.15.40:0/2506493737 wait complete. 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.142+0000 7f3c9536a640 1 Processor -- start 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.142+0000 7f3c9536a640 1 -- start start 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c9536a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c900d2110 0x7f3c900cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c9536a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c901628f0 0x7f3c900cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c9536a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c90162cd0 0x7f3c900cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c9536a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c900d16d0 con 0x7f3c901628f0 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c9536a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c900d1550 con 0x7f3c900d2110 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c9536a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3c900b3f50 con 0x7f3c90162cd0 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c901628f0 0x7f3c900cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c901628f0 0x7f3c900cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59506/0 (socket says 172.21.15.40:59506) 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c8e7fc640 1 -- 172.21.15.40:0/3133581074 learned_addr learned my addr 172.21.15.40:0/3133581074 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c8effd640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c900d2110 0x7f3c900cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.143+0000 7f3c8f7fe640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c90162cd0 0x7f3c900cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c8e7fc640 1 -- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c90162cd0 msgr2=0x7f3c900cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c8e7fc640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c90162cd0 0x7f3c900cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c8e7fc640 1 -- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c900d2110 msgr2=0x7f3c900cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c8e7fc640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c900d2110 0x7f3c900cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c8e7fc640 1 -- 172.21.15.40:0/3133581074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c900ba720 con 0x7f3c901628f0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c8e7fc640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c901628f0 0x7f3c900cd650 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3c840098c0 tx=0x7f3c840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c777fe640 1 -- 172.21.15.40:0/3133581074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84037040 con 0x7f3c901628f0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c777fe640 1 -- 172.21.15.40:0/3133581074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c84042070 con 0x7f3c901628f0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c900c9dd0 con 0x7f3c901628f0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c777fe640 1 -- 172.21.15.40:0/3133581074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84030a70 con 0x7f3c901628f0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.144+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c900ba9a0 con 0x7f3c901628f0 2024-04-26T21:42:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.145+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c900694f0 con 0x7f3c901628f0 2024-04-26T21:42:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.145+0000 7f3c777fe640 1 -- 172.21.15.40:0/3133581074 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c84030c10 con 0x7f3c901628f0 2024-04-26T21:42:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.146+0000 7f3c777fe640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c7c06c8f0 0x7f3c7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.146+0000 7f3c777fe640 1 -- 172.21.15.40:0/3133581074 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6647+0+0 (secure 0 0 0) 0x7f3c840ac8b0 con 0x7f3c901628f0 2024-04-26T21:42:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.146+0000 7f3c8effd640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c7c06c8f0 0x7f3c7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.146+0000 7f3c8effd640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c7c06c8f0 0x7f3c7c06edb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3c9015ecf0 tx=0x7f3c88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:59.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.150+0000 7f3c777fe640 1 -- 172.21.15.40:0/3133581074 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c900ba720 con 0x7f3c901628f0 2024-04-26T21:42:59.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.374+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f3c900745d0 con 0x7f3c901628f0 2024-04-26T21:42:59.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.716+0000 7f3c777fe640 1 -- 172.21.15.40:0/3133581074 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v119) ==== 162+0+0 (secure 0 0 0) 0x7f3c8407a2e0 con 0x7f3c901628f0 2024-04-26T21:42:59.717 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c7c06c8f0 msgr2=0x7f3c7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c7c06c8f0 0x7f3c7c06edb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3c9015ecf0 tx=0x7f3c88009040 comp rx=0 tx=0).stop 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c901628f0 msgr2=0x7f3c900cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c901628f0 0x7f3c900cd650 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3c840098c0 tx=0x7f3c840046c0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 shutdown_connections 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3c7c06c8f0 0x7f3c7c06edb0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c90162cd0 0x7f3c900cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c901628f0 0x7f3c900cd650 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 --2- 172.21.15.40:0/3133581074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c900d2110 0x7f3c900cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.718+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 >> 172.21.15.40:0/3133581074 conn(0x7f3c900d4ff0 msgr2=0x7f3c900b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.719+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 shutdown_connections 2024-04-26T21:42:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.719+0000 7f3c9536a640 1 -- 172.21.15.40:0/3133581074 wait complete. 2024-04-26T21:42:59.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:42:59.733 INFO:tasks.ceph.mds.b.smithi184.stderr:2024-04-26T21:42:59.732+0000 7f30d8367640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-04-26T21:42:59.734 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:59.733+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:42:59.734 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:42:59.733+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:42:59.739 INFO:tasks.ceph.mds.b.smithi184.stderr: -14> 2024-04-26T21:42:59.732+0000 7f30d8367640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-04-26T21:42:59.765 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:42:59.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.836+0000 7fa8e66f1640 1 Processor -- start 2024-04-26T21:42:59.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.836+0000 7fa8e66f1640 1 -- start start 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8e66f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8e015f3a0 0x7fa8e00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8e66f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e00d2330 0x7fa8e00b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8e66f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e00d0e20 con 0x7fa8e00d1770 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8e66f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e00b1d70 con 0x7fa8e015f3a0 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8e66f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e00b2050 con 0x7fa8e00d2330 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8dffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8e015f3a0 0x7fa8e00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8dffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8e015f3a0 0x7fa8e00d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59522/0 (socket says 172.21.15.40:59522) 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8dffff640 1 -- 172.21.15.40:0/3333645078 learned_addr learned my addr 172.21.15.40:0/3333645078 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.837+0000 7fa8df7fe640 1 --1- 172.21.15.40:0/3333645078 >> v1:172.21.15.40:6789/0 conn(0x7fa8e00d1770 0x7fa8e00d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52656/0 (socket says 172.21.15.40:52656) 2024-04-26T21:42:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.838+0000 7fa8e4c8e640 1 --2- 172.21.15.40:0/3333645078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e00d2330 0x7fa8e00b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.838+0000 7fa8dffff640 1 -- 172.21.15.40:0/3333645078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e00d2330 msgr2=0x7fa8e00b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:42:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.838+0000 7fa8dffff640 1 --2- 172.21.15.40:0/3333645078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e00d2330 0x7fa8e00b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.838+0000 7fa8dffff640 1 -- 172.21.15.40:0/3333645078 >> v1:172.21.15.40:6789/0 conn(0x7fa8e00d1770 legacy=0x7fa8e00d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.838+0000 7fa8deffd640 1 -- 172.21.15.40:0/3333645078 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 971380437 0 0) 0x7fa8e00d0e20 con 0x7fa8e00d1770 2024-04-26T21:42:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.838+0000 7fa8dffff640 1 -- 172.21.15.40:0/3333645078 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e00b2330 con 0x7fa8e015f3a0 2024-04-26T21:42:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.838+0000 7fa8dffff640 1 --2- 172.21.15.40:0/3333645078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e015f3a0 0x7fa8e00d06f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009870 tx=0x7fa8c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbce74938f588eae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8deffd640 1 -- 172.21.15.40:0/3333645078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8037070 con 0x7fa8e015f3a0 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8deffd640 1 -- 172.21.15.40:0/3333645078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c8032440 con 0x7fa8e015f3a0 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8deffd640 1 -- 172.21.15.40:0/3333645078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8032810 con 0x7fa8e015f3a0 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8e66f1640 1 -- 172.21.15.40:0/3333645078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e015f3a0 msgr2=0x7fa8e00d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/3333645078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e015f3a0 0x7fa8e00d06f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009870 tx=0x7fa8c802a270 comp rx=0 tx=0).stop 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8e66f1640 1 -- 172.21.15.40:0/3333645078 shutdown_connections 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/3333645078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e00d2330 0x7fa8e00b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/3333645078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e015f3a0 0x7fa8e00d06f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8e66f1640 1 -- 172.21.15.40:0/3333645078 >> 172.21.15.40:0/3333645078 conn(0x7fa8e00d4f90 msgr2=0x7fa8e00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8e66f1640 1 -- 172.21.15.40:0/3333645078 shutdown_connections 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.839+0000 7fa8e66f1640 1 -- 172.21.15.40:0/3333645078 wait complete. 2024-04-26T21:42:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.840+0000 7fa8e66f1640 1 Processor -- start 2024-04-26T21:42:59.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.841+0000 7fa8e66f1640 1 -- start start 2024-04-26T21:42:59.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.841+0000 7fa8e66f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e00d1770 0x7fa8e00cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.841+0000 7fa8e66f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e00d2330 0x7fa8e0162790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.841+0000 7fa8e66f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e015f3a0 0x7fa8e015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.841+0000 7fa8e66f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e00b2f10 con 0x7fa8e00d2330 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.841+0000 7fa8e66f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e00b2d90 con 0x7fa8e00d1770 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.841+0000 7fa8e66f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e00b3090 con 0x7fa8e015f3a0 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e00d2330 0x7fa8e0162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e00d2330 0x7fa8e0162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59528/0 (socket says 172.21.15.40:59528) 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 -- 172.21.15.40:0/2006647656 learned_addr learned my addr 172.21.15.40:0/2006647656 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8dffff640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e00d1770 0x7fa8e00cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8e4c8e640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e015f3a0 0x7fa8e015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 -- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e015f3a0 msgr2=0x7fa8e015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e015f3a0 0x7fa8e015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 -- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e00d1770 msgr2=0x7fa8e00cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e00d1770 0x7fa8e00cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 -- 172.21.15.40:0/2006647656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e00cb9d0 con 0x7fa8e00d2330 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8df7fe640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e00d2330 0x7fa8e0162790 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000c4d0 tx=0x7fa8d000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8dcff9640 1 -- 172.21.15.40:0/2006647656 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0018070 con 0x7fa8e00d2330 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8dcff9640 1 -- 172.21.15.40:0/2006647656 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d0007500 con 0x7fa8e00d2330 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e0162cd0 con 0x7fa8e00d2330 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8dcff9640 1 -- 172.21.15.40:0/2006647656 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004a20 con 0x7fa8e00d2330 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.842+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e00ca9b0 con 0x7fa8e00d2330 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.843+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e00694f0 con 0x7fa8e00d2330 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.844+0000 7fa8dcff9640 1 -- 172.21.15.40:0/2006647656 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8d001b070 con 0x7fa8e00d2330 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.844+0000 7fa8dcff9640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa8c006c8f0 0x7fa8c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.844+0000 7fa8dcff9640 1 -- 172.21.15.40:0/2006647656 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7fa8d008b1a0 con 0x7fa8e00d2330 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.844+0000 7fa8dffff640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa8c006c8f0 0x7fa8c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.845+0000 7fa8dffff640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa8c006c8f0 0x7fa8c006edb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8034000 tx=0x7fa8c802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:42:59.849+0000 7fa8dcff9640 1 -- 172.21.15.40:0/2006647656 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa8d0058b80 con 0x7fa8e00d2330 2024-04-26T21:43:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.070+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8e00b6490 con 0x7fa8e00d2330 2024-04-26T21:43:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.070+0000 7fa8dcff9640 1 -- 172.21.15.40:0/2006647656 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+4351 (secure 0 0 0) 0x7fa8d005ce80 con 0x7fa8e00d2330 2024-04-26T21:43:00.072 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:00.072 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:59.734415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 121 2024-04-26T21:43:00.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa8c006c8f0 msgr2=0x7fa8c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa8c006c8f0 0x7fa8c006edb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8034000 tx=0x7fa8c802e000 comp rx=0 tx=0).stop 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e00d2330 msgr2=0x7fa8e0162790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e00d2330 0x7fa8e0162790 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000c4d0 tx=0x7fa8d000c990 comp rx=0 tx=0).stop 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 shutdown_connections 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa8c006c8f0 0x7fa8c006edb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8e015f3a0 0x7fa8e015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e00d2330 0x7fa8e0162790 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 --2- 172.21.15.40:0/2006647656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8e00d1770 0x7fa8e00cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 >> 172.21.15.40:0/2006647656 conn(0x7fa8e00d4f90 msgr2=0x7fa8e00d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 shutdown_connections 2024-04-26T21:43:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.072+0000 7fa8e66f1640 1 -- 172.21.15.40:0/2006647656 wait complete. 2024-04-26T21:43:00.084 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:00.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.184+0000 7f4f59a20640 1 Processor -- start 2024-04-26T21:43:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.184+0000 7f4f59a20640 1 -- start start 2024-04-26T21:43:00.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f59a20640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4f54162880 0x7f4f540d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f59a20640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f540b7e50 con 0x7f4f540b5f20 2024-04-26T21:43:00.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f59a20640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f540d1800 con 0x7f4f540d3bc0 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f59a20640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4f540d0580 con 0x7f4f54162880 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f527fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4f540b5f20 0x7f4f540d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52658/0 (socket says 172.21.15.40:52658) 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f527fc640 1 -- 172.21.15.40:0/889344319 learned_addr learned my addr 172.21.15.40:0/889344319 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/889344319 >> v2:172.21.15.184:3300/0 conn(0x7f4f54162880 0x7f4f540d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3567017406 0 0) 0x7f4f540b7e50 con 0x7f4f540b5f20 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.185+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f40003520 con 0x7f4f540b5f20 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3513379732 0 0) 0x7f4f40003520 con 0x7f4f540b5f20 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 >> v2:172.21.15.184:3300/0 conn(0x7f4f54162880 msgr2=0x7f4f540d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 --2- 172.21.15.40:0/889344319 >> v2:172.21.15.184:3300/0 conn(0x7f4f54162880 0x7f4f540d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 >> v1:172.21.15.184:6789/0 conn(0x7f4f540d3bc0 legacy=0x7f4f540b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f540d07d0 con 0x7f4f540b5f20 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f4f3c001670 con 0x7f4f540b5f20 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 >> v1:172.21.15.40:6789/0 conn(0x7f4f540b5f20 legacy=0x7f4f540d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f400039b0 0x7f4f40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f40004800 0x7f4f40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400051c0 0x7f4f4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f540d0580 con 0x7f4f400039b0 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4000c060 con 0x7f4f400051c0 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4000c260 con 0x7f4f40004800 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f400039b0 0x7f4f40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f527fc640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f40004800 0x7f4f40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f537fe640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400051c0 0x7f4f4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f52ffd640 1 -- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f40004800 msgr2=0x7f4f40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f40004800 0x7f4f40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f52ffd640 1 -- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400051c0 msgr2=0x7f4f4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400051c0 0x7f4f4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f52ffd640 1 -- 172.21.15.40:0/889344319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f40003520 con 0x7f4f400039b0 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f537fe640 1 -- 172.21.15.40:0/889344319 reap_dead start 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.186+0000 7f4f527fc640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f40004800 0x7f4f40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f400039b0 0x7f4f40003dd0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009ac0 tx=0x7f4f40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fbf391687886d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c036070 con 0x7f4f400039b0 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c031400 con 0x7f4f400039b0 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f51ffb640 1 -- 172.21.15.40:0/889344319 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c009690 con 0x7f4f400039b0 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f59a20640 1 -- 172.21.15.40:0/889344319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f400039b0 msgr2=0x7f4f40003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f59a20640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f400039b0 0x7f4f40003dd0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009ac0 tx=0x7f4f40005830 comp rx=0 tx=0).stop 2024-04-26T21:43:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f59a20640 1 -- 172.21.15.40:0/889344319 shutdown_connections 2024-04-26T21:43:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f59a20640 1 --2- 172.21.15.40:0/889344319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f400039b0 0x7f4f40003dd0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.187+0000 7f4f59a20640 1 -- 172.21.15.40:0/889344319 >> 172.21.15.40:0/889344319 conn(0x7f4f540d4f60 msgr2=0x7f4f540d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.188+0000 7f4f59a20640 1 -- 172.21.15.40:0/889344319 shutdown_connections 2024-04-26T21:43:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.188+0000 7f4f59a20640 1 -- 172.21.15.40:0/889344319 wait complete. 2024-04-26T21:43:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.189+0000 7f4f59a20640 1 Processor -- start 2024-04-26T21:43:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.189+0000 7f4f59a20640 1 -- start start 2024-04-26T21:43:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f59a20640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400039b0 0x7f4f540b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f59a20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f540ce3e0 0x7f4f541f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f59a20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540ce860 0x7f4f541f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f59a20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54162c60 con 0x7f4f540ce860 2024-04-26T21:43:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f59a20640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f540d0d90 con 0x7f4f400039b0 2024-04-26T21:43:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f59a20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f54162de0 con 0x7f4f540ce3e0 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540ce860 0x7f4f541f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540ce860 0x7f4f541f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59538/0 (socket says 172.21.15.40:59538) 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 -- 172.21.15.40:0/406103702 learned_addr learned my addr 172.21.15.40:0/406103702 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f527fc640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f540ce3e0 0x7f4f541f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400039b0 0x7f4f540b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 -- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f540ce3e0 msgr2=0x7f4f541f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f540ce3e0 0x7f4f541f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 -- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400039b0 msgr2=0x7f4f540b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400039b0 0x7f4f540b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 -- 172.21.15.40:0/406103702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f541f57f0 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f537fe640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540ce860 0x7f4f541f51b0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4f44007fd0 tx=0x7f4f4400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.190+0000 7f4f4bfff640 1 -- 172.21.15.40:0/406103702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44019070 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.191+0000 7f4f4bfff640 1 -- 172.21.15.40:0/406103702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f44006d60 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.191+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f541f5ad0 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.191+0000 7f4f4bfff640 1 -- 172.21.15.40:0/406103702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44003b90 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.191+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f540b5920 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.191+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f540d2ec0 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.192+0000 7f4f4bfff640 1 -- 172.21.15.40:0/406103702 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f4401c070 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.192+0000 7f4f4bfff640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f3406c8f0 0x7f4f3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.192+0000 7f4f4bfff640 1 -- 172.21.15.40:0/406103702 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f4f4408b5a0 con 0x7f4f540ce860 2024-04-26T21:43:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.193+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f3406c8f0 0x7f4f3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.193+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f3406c8f0 0x7f4f3406edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c033000 tx=0x7f4f4c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.198+0000 7f4f4bfff640 1 -- 172.21.15.40:0/406103702 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f540d2ec0 con 0x7f4f540ce860 2024-04-26T21:43:00.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.424+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f540b49d0 con 0x7f4f540ce860 2024-04-26T21:43:00.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.424+0000 7f4f4bfff640 1 -- 172.21.15.40:0/406103702 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+4351 (secure 0 0 0) 0x7f4f540d1800 con 0x7f4f540ce860 2024-04-26T21:43:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:00.426 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:59.734415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:00.426 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 121 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f3406c8f0 msgr2=0x7f4f3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f3406c8f0 0x7f4f3406edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c033000 tx=0x7f4f4c008030 comp rx=0 tx=0).stop 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540ce860 msgr2=0x7f4f541f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540ce860 0x7f4f541f51b0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4f44007fd0 tx=0x7f4f4400e9e0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 shutdown_connections 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f3406c8f0 0x7f4f3406edb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540ce860 0x7f4f541f51b0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f540ce3e0 0x7f4f541f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 --2- 172.21.15.40:0/406103702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f400039b0 0x7f4f540b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 >> 172.21.15.40:0/406103702 conn(0x7f4f540d4f60 msgr2=0x7f4f5406e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 shutdown_connections 2024-04-26T21:43:00.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.427+0000 7f4f59a20640 1 -- 172.21.15.40:0/406103702 wait complete. 2024-04-26T21:43:00.438 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:00.439 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:00.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.538+0000 7fa4cecda640 1 Processor -- start 2024-04-26T21:43:00.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.538+0000 7fa4cecda640 1 -- start start 2024-04-26T21:43:00.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cecda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4c8162810 0x7fa4c8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cecda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80b2340 0x7fa4c80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cecda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c80d10d0 con 0x7fa4c80cd930 2024-04-26T21:43:00.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cecda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa4c80d0750 con 0x7fa4c8162810 2024-04-26T21:43:00.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cecda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c80d16a0 con 0x7fa4c80b2340 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80b2340 0x7fa4c80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cca76640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4c8162810 0x7fa4c8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cd277640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fa4c80cd930 0x7fa4c80d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:49508/0 (socket says 172.21.15.40:49508) 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cd277640 1 -- 172.21.15.40:0/551320047 learned_addr learned my addr 172.21.15.40:0/551320047 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cca76640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4c8162810 0x7fa4c8070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:44448/0 (socket says 172.21.15.40:44448) 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80b2340 0x7fa4c80cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:51596/0 (socket says 172.21.15.40:51596) 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cca76640 1 -- 172.21.15.40:0/551320047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80b2340 msgr2=0x7fa4c80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cca76640 1 --2- 172.21.15.40:0/551320047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80b2340 0x7fa4c80cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cca76640 1 -- 172.21.15.40:0/551320047 >> v1:172.21.15.184:6789/0 conn(0x7fa4c80cd930 legacy=0x7fa4c80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4c77fe640 1 -- 172.21.15.40:0/551320047 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468949595 0 0) 0x7fa4c80d10d0 con 0x7fa4c80cd930 2024-04-26T21:43:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.539+0000 7fa4cca76640 1 -- 172.21.15.40:0/551320047 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c80d1370 con 0x7fa4c8162810 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.540+0000 7fa4cca76640 1 --2- 172.21.15.40:0/551320047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c8162810 0x7fa4c8070350 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0097a0 tx=0x7fa4bc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45887108eb4c261 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.540+0000 7fa4c77fe640 1 -- 172.21.15.40:0/551320047 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc037070 con 0x7fa4c8162810 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.540+0000 7fa4c77fe640 1 -- 172.21.15.40:0/551320047 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc032440 con 0x7fa4c8162810 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.540+0000 7fa4c77fe640 1 -- 172.21.15.40:0/551320047 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc032810 con 0x7fa4c8162810 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.541+0000 7fa4cecda640 1 -- 172.21.15.40:0/551320047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c8162810 msgr2=0x7fa4c8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.541+0000 7fa4cecda640 1 --2- 172.21.15.40:0/551320047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c8162810 0x7fa4c8070350 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0097a0 tx=0x7fa4bc02a180 comp rx=0 tx=0).stop 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.541+0000 7fa4cecda640 1 -- 172.21.15.40:0/551320047 shutdown_connections 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.541+0000 7fa4cecda640 1 --2- 172.21.15.40:0/551320047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80b2340 0x7fa4c80cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.541+0000 7fa4cecda640 1 --2- 172.21.15.40:0/551320047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c8162810 0x7fa4c8070350 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.541+0000 7fa4cecda640 1 -- 172.21.15.40:0/551320047 >> 172.21.15.40:0/551320047 conn(0x7fa4c80d2da0 msgr2=0x7fa4c80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.541+0000 7fa4cecda640 1 -- 172.21.15.40:0/551320047 shutdown_connections 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.541+0000 7fa4cecda640 1 -- 172.21.15.40:0/551320047 wait complete. 2024-04-26T21:43:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.542+0000 7fa4cecda640 1 Processor -- start 2024-04-26T21:43:00.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.542+0000 7fa4cecda640 1 -- start start 2024-04-26T21:43:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cecda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c80b2340 0x7fa4c80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cecda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80cd930 0x7fa4c81f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cecda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c8162810 0x7fa4c81f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cecda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8070890 con 0x7fa4c80b2340 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cecda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c80d0470 con 0x7fa4c8162810 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cecda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c80d2240 con 0x7fa4c80cd930 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cca76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c80b2340 0x7fa4c80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cca76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c80b2340 0x7fa4c80cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59548/0 (socket says 172.21.15.40:59548) 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cca76640 1 -- 172.21.15.40:0/3382785304 learned_addr learned my addr 172.21.15.40:0/3382785304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cca76640 1 -- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80cd930 msgr2=0x7fa4c81f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cca76640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80cd930 0x7fa4c81f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cca76640 1 -- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c8162810 msgr2=0x7fa4c81f8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cca76640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c8162810 0x7fa4c81f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.543+0000 7fa4cca76640 1 -- 172.21.15.40:0/3382785304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c81f5eb0 con 0x7fa4c80b2340 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.544+0000 7fa4cca76640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c80b2340 0x7fa4c80cc810 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc000c00 tx=0x7fa4bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.544+0000 7fa4c57fa640 1 -- 172.21.15.40:0/3382785304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc037050 con 0x7fa4c80b2340 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.544+0000 7fa4c57fa640 1 -- 172.21.15.40:0/3382785304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc038040 con 0x7fa4c80b2340 2024-04-26T21:43:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.544+0000 7fa4c57fa640 1 -- 172.21.15.40:0/3382785304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc004470 con 0x7fa4c80b2340 2024-04-26T21:43:00.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.544+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c81f84e0 con 0x7fa4c80b2340 2024-04-26T21:43:00.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.544+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c80c9470 con 0x7fa4c80b2340 2024-04-26T21:43:00.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.545+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c80694f0 con 0x7fa4c80b2340 2024-04-26T21:43:00.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.545+0000 7fa4c57fa640 1 -- 172.21.15.40:0/3382785304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4bc03c6d0 con 0x7fa4c80b2340 2024-04-26T21:43:00.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.546+0000 7fa4c57fa640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.546+0000 7fa4c57fa640 1 -- 172.21.15.40:0/3382785304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7fa4bc0aca10 con 0x7fa4c80b2340 2024-04-26T21:43:00.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.546+0000 7fa4c7fff640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.547+0000 7fa4c7fff640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006a90 tx=0x7fa4b4006a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:00.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.550+0000 7fa4c57fa640 1 -- 172.21.15.40:0/3382785304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4c81f5eb0 con 0x7fa4c80b2340 2024-04-26T21:43:00.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.776+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4c80cd160 con 0x7fa4c80b2340 2024-04-26T21:43:00.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.776+0000 7fa4c57fa640 1 -- 172.21.15.40:0/3382785304 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5180 (secure 0 0 0) 0x7fa4bc07a9a0 con 0x7fa4c80b2340 2024-04-26T21:43:00.777 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:00.778 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":122,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:42:59.734415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:00.778 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 122 2024-04-26T21:43:00.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ac06c8f0 msgr2=0x7fa4ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006a90 tx=0x7fa4b4006a20 comp rx=0 tx=0).stop 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c80b2340 msgr2=0x7fa4c80cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c80b2340 0x7fa4c80cc810 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc000c00 tx=0x7fa4bc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 shutdown_connections 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c8162810 0x7fa4c81f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80cd930 0x7fa4c81f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 --2- 172.21.15.40:0/3382785304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c80b2340 0x7fa4c80cc810 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 >> 172.21.15.40:0/3382785304 conn(0x7fa4c80d2da0 msgr2=0x7fa4c80b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 shutdown_connections 2024-04-26T21:43:00.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:00.778+0000 7fa4cecda640 1 -- 172.21.15.40:0/3382785304 wait complete. 2024-04-26T21:43:00.790 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000001' led to DAMAGED state 2024-04-26T21:43:00.790 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_openfiles.0' 2024-04-26T21:43:00.790 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:00.790 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:00.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:43:00.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:43:00.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:00.806 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:43:00.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:43:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:43:00.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:43:01.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.001+0000 7f9530371640 1 Processor -- start 2024-04-26T21:43:01.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.001+0000 7f9530371640 1 -- start start 2024-04-26T21:43:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f9530371640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95280b2340 0x7f95280cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f9530371640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95280cd930 0x7f95280d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f9530371640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95280d10d0 con 0x7f9528162810 2024-04-26T21:43:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f9530371640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95280d0750 con 0x7f95280cd930 2024-04-26T21:43:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f9530371640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95280d16a0 con 0x7f95280b2340 2024-04-26T21:43:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f952e90e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95280cd930 0x7f95280d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f952e90e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95280cd930 0x7f95280d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59562/0 (socket says 172.21.15.40:59562) 2024-04-26T21:43:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f952e90e640 1 -- 172.21.15.40:0/2850507260 learned_addr learned my addr 172.21.15.40:0/2850507260 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f952d90c640 1 --2- 172.21.15.40:0/2850507260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95280b2340 0x7f95280cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f952e90e640 1 -- 172.21.15.40:0/2850507260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95280b2340 msgr2=0x7f95280cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f952e90e640 1 --2- 172.21.15.40:0/2850507260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95280b2340 0x7f95280cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f952e90e640 1 -- 172.21.15.40:0/2850507260 >> v1:172.21.15.184:6789/0 conn(0x7f9528162810 legacy=0x7f9528070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.002+0000 7f952e90e640 1 -- 172.21.15.40:0/2850507260 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95280d1370 con 0x7f95280cd930 2024-04-26T21:43:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.003+0000 7f952e90e640 1 --2- 172.21.15.40:0/2850507260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280cd930 0x7f95280d1b30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9520009e50 tx=0x7f952002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b3f9544a9ca5ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.003+0000 7f952d10b640 1 -- 172.21.15.40:0/2850507260 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520036070 con 0x7f95280cd930 2024-04-26T21:43:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.003+0000 7f952d10b640 1 -- 172.21.15.40:0/2850507260 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9520031600 con 0x7f95280cd930 2024-04-26T21:43:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.003+0000 7f952d10b640 1 -- 172.21.15.40:0/2850507260 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95200319d0 con 0x7f95280cd930 2024-04-26T21:43:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.004+0000 7f9530371640 1 -- 172.21.15.40:0/2850507260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280cd930 msgr2=0x7f95280d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.004+0000 7f9530371640 1 --2- 172.21.15.40:0/2850507260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280cd930 0x7f95280d1b30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9520009e50 tx=0x7f952002a710 comp rx=0 tx=0).stop 2024-04-26T21:43:01.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.004+0000 7f9530371640 1 -- 172.21.15.40:0/2850507260 shutdown_connections 2024-04-26T21:43:01.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.004+0000 7f9530371640 1 --2- 172.21.15.40:0/2850507260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280cd930 0x7f95280d1b30 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.004+0000 7f9530371640 1 --2- 172.21.15.40:0/2850507260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95280b2340 0x7f95280cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.004+0000 7f9530371640 1 -- 172.21.15.40:0/2850507260 >> 172.21.15.40:0/2850507260 conn(0x7f95280d2da0 msgr2=0x7f95280d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:01.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.004+0000 7f9530371640 1 -- 172.21.15.40:0/2850507260 shutdown_connections 2024-04-26T21:43:01.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.004+0000 7f9530371640 1 -- 172.21.15.40:0/2850507260 wait complete. 2024-04-26T21:43:01.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.005+0000 7f9530371640 1 Processor -- start 2024-04-26T21:43:01.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.005+0000 7f9530371640 1 -- start start 2024-04-26T21:43:01.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.005+0000 7f9530371640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280b2340 0x7f952806e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f9530371640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95280cd930 0x7f95280cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f9530371640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9528162810 0x7f95280cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f9530371640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95280d2070 con 0x7f95280b2340 2024-04-26T21:43:01.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f9530371640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95280d0470 con 0x7f95280cd930 2024-04-26T21:43:01.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f9530371640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95280d21f0 con 0x7f9528162810 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280b2340 0x7f952806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280b2340 0x7f952806e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59576/0 (socket says 172.21.15.40:59576) 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 -- 172.21.15.40:0/1149398241 learned_addr learned my addr 172.21.15.40:0/1149398241 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952d90c640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95280cd930 0x7f95280cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e90e640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9528162810 0x7f95280cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 -- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9528162810 msgr2=0x7f95280cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9528162810 0x7f95280cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 -- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95280cd930 msgr2=0x7f95280cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95280cd930 0x7f95280cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 -- 172.21.15.40:0/1149398241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95281f9020 con 0x7f95280b2340 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f952e10d640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280b2340 0x7f952806e740 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f952400c3c0 tx=0x7f952400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f951effd640 1 -- 172.21.15.40:0/1149398241 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240052d0 con 0x7f95280b2340 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f951effd640 1 -- 172.21.15.40:0/1149398241 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9524005490 con 0x7f95280b2340 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95281f4f30 con 0x7f95280b2340 2024-04-26T21:43:01.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.006+0000 7f951effd640 1 -- 172.21.15.40:0/1149398241 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524007260 con 0x7f95280b2340 2024-04-26T21:43:01.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.007+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281f54f0 con 0x7f95280b2340 2024-04-26T21:43:01.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.007+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95280ced70 con 0x7f95280b2340 2024-04-26T21:43:01.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.008+0000 7f951effd640 1 -- 172.21.15.40:0/1149398241 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9524007400 con 0x7f95280b2340 2024-04-26T21:43:01.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.008+0000 7f951effd640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f950c06c8f0 0x7f950c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.008+0000 7f951effd640 1 -- 172.21.15.40:0/1149398241 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f952408b3a0 con 0x7f95280b2340 2024-04-26T21:43:01.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.009+0000 7f952d90c640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f950c06c8f0 0x7f950c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.009+0000 7f952d90c640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f950c06c8f0 0x7f950c06edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f95280d3dd0 tx=0x7f9518008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:01.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.013+0000 7f951effd640 1 -- 172.21.15.40:0/1149398241 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95280ced70 con 0x7f95280b2340 2024-04-26T21:43:01.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.236+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f95280694f0 con 0x7f95280b2340 2024-04-26T21:43:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.748+0000 7f951effd640 1 -- 172.21.15.40:0/1149398241 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v123) ==== 161+0+0 (secure 0 0 0) 0x7f9524058d80 con 0x7f95280b2340 2024-04-26T21:43:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f950c06c8f0 msgr2=0x7f950c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f950c06c8f0 0x7f950c06edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f95280d3dd0 tx=0x7f9518008040 comp rx=0 tx=0).stop 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280b2340 msgr2=0x7f952806e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280b2340 0x7f952806e740 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f952400c3c0 tx=0x7f952400c880 comp rx=0 tx=0).stop 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 shutdown_connections 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f950c06c8f0 0x7f950c06edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9528162810 0x7f95280cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95280cd930 0x7f95280cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 --2- 172.21.15.40:0/1149398241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95280b2340 0x7f952806e740 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 >> 172.21.15.40:0/1149398241 conn(0x7f95280d2da0 msgr2=0x7f9528070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 shutdown_connections 2024-04-26T21:43:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.752+0000 7f9530371640 1 -- 172.21.15.40:0/1149398241 wait complete. 2024-04-26T21:43:01.766 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:43:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.873+0000 7f5ec2d71640 1 Processor -- start 2024-04-26T21:43:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.873+0000 7f5ec2d71640 1 -- start start 2024-04-26T21:43:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec2d71640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ebc162880 0x7f5ebc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec2d71640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5ebc0d3bc0 0x7f5ebc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec2d71640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc0b7e50 con 0x7f5ebc0b5f20 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec2d71640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0d0580 con 0x7f5ebc162880 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec2d71640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0d1800 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec0b0d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ebc162880 0x7f5ebc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec0b0d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ebc162880 0x7f5ebc0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59580/0 (socket says 172.21.15.40:59580) 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec0b0d640 1 -- 172.21.15.40:0/2608902509 learned_addr learned my addr 172.21.15.40:0/2608902509 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec130e640 1 --2- 172.21.15.40:0/2608902509 >> v2:172.21.15.184:3300/0 conn(0x7f5ebc0d3bc0 0x7f5ebc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec0b0d640 1 -- 172.21.15.40:0/2608902509 >> v2:172.21.15.184:3300/0 conn(0x7f5ebc0d3bc0 msgr2=0x7f5ebc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec0b0d640 1 --2- 172.21.15.40:0/2608902509 >> v2:172.21.15.184:3300/0 conn(0x7f5ebc0d3bc0 0x7f5ebc0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec0b0d640 1 -- 172.21.15.40:0/2608902509 >> v1:172.21.15.184:6789/0 conn(0x7f5ebc0b5f20 legacy=0x7f5ebc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec0b0d640 1 -- 172.21.15.40:0/2608902509 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0d07d0 con 0x7f5ebc162880 2024-04-26T21:43:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.874+0000 7f5ec130e640 1 --2- 172.21.15.40:0/2608902509 >> v2:172.21.15.184:3300/0 conn(0x7f5ebc0d3bc0 0x7f5ebc0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.875+0000 7f5ec0b0d640 1 --2- 172.21.15.40:0/2608902509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc162880 0x7f5ebc0d2cb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0d11d0 tx=0x7f5eac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5b53294bf7bc18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.875+0000 7f5ebb7fe640 1 -- 172.21.15.40:0/2608902509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac037070 con 0x7f5ebc162880 2024-04-26T21:43:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.875+0000 7f5ebb7fe640 1 -- 172.21.15.40:0/2608902509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac032440 con 0x7f5ebc162880 2024-04-26T21:43:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.875+0000 7f5ebb7fe640 1 -- 172.21.15.40:0/2608902509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac032810 con 0x7f5ebc162880 2024-04-26T21:43:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.876+0000 7f5ec2d71640 1 -- 172.21.15.40:0/2608902509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc162880 msgr2=0x7f5ebc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.876+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/2608902509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc162880 0x7f5ebc0d2cb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0d11d0 tx=0x7f5eac02a230 comp rx=0 tx=0).stop 2024-04-26T21:43:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.876+0000 7f5ec2d71640 1 -- 172.21.15.40:0/2608902509 shutdown_connections 2024-04-26T21:43:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.876+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/2608902509 >> v2:172.21.15.184:3300/0 conn(0x7f5ebc0d3bc0 0x7f5ebc0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.876+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/2608902509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc162880 0x7f5ebc0d2cb0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.876+0000 7f5ec2d71640 1 -- 172.21.15.40:0/2608902509 >> 172.21.15.40:0/2608902509 conn(0x7f5ebc0d4f60 msgr2=0x7f5ebc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.876+0000 7f5ec2d71640 1 -- 172.21.15.40:0/2608902509 shutdown_connections 2024-04-26T21:43:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.876+0000 7f5ec2d71640 1 -- 172.21.15.40:0/2608902509 wait complete. 2024-04-26T21:43:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.877+0000 7f5ec2d71640 1 Processor -- start 2024-04-26T21:43:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.877+0000 7f5ec2d71640 1 -- start start 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ec2d71640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ebc0b5f20 0x7f5ebc155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ec2d71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0d3bc0 0x7f5ebc15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ec2d71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ebc162880 0x7f5ebc15df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ec2d71640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc162c60 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ec2d71640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d0d90 con 0x7f5ebc0b5f20 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ec2d71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc162de0 con 0x7f5ebc162880 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ebbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0d3bc0 0x7f5ebc15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ebbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0d3bc0 0x7f5ebc15b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59588/0 (socket says 172.21.15.40:59588) 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ebbfff640 1 -- 172.21.15.40:0/665507467 learned_addr learned my addr 172.21.15.40:0/665507467 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ec0b0d640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ebc0b5f20 0x7f5ebc155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ec130e640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ebc162880 0x7f5ebc15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ebbfff640 1 -- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ebc162880 msgr2=0x7f5ebc15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.878+0000 7f5ebbfff640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ebc162880 0x7f5ebc15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5ebbfff640 1 -- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ebc0b5f20 msgr2=0x7f5ebc155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5ebbfff640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ebc0b5f20 0x7f5ebc155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5ebbfff640 1 -- 172.21.15.40:0/665507467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc1f85d0 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5ec0b0d640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ebc0b5f20 0x7f5ebc155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5ec130e640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ebc162880 0x7f5ebc15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5ebbfff640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0d3bc0 0x7f5ebc15b370 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000b3f0 tx=0x7f5eb000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5eb97fa640 1 -- 172.21.15.40:0/665507467 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0015020 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5eb97fa640 1 -- 172.21.15.40:0/665507467 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb0011830 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5eb97fa640 1 -- 172.21.15.40:0/665507467 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0011c20 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc15bca0 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.879+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc1f8b70 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.880+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc08e6c0 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.881+0000 7f5eb97fa640 1 -- 172.21.15.40:0/665507467 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5eb0018020 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.881+0000 7f5eb97fa640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ea806c920 0x7f5ea806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.881+0000 7f5eb97fa640 1 -- 172.21.15.40:0/665507467 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f5eb008bb00 con 0x7f5ebc0d3bc0 2024-04-26T21:43:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.881+0000 7f5ec0b0d640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ea806c920 0x7f5ea806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.882+0000 7f5ec0b0d640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ea806c920 0x7f5ea806ede0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:01.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:01.885+0000 7f5eb97fa640 1 -- 172.21.15.40:0/665507467 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ebc08e6c0 con 0x7f5ebc0d3bc0 2024-04-26T21:43:02.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.143+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5ebc06e380 con 0x7f5ebc0d3bc0 2024-04-26T21:43:02.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.750+0000 7f5eb97fa640 1 -- 172.21.15.40:0/665507467 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v124) ==== 101+0+0 (secure 0 0 0) 0x7f5eb001f030 con 0x7f5ebc0d3bc0 2024-04-26T21:43:02.751 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:43:02.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ea806c920 msgr2=0x7f5ea806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ea806c920 0x7f5ea806ede0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02e000 comp rx=0 tx=0).stop 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0d3bc0 msgr2=0x7f5ebc15b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0d3bc0 0x7f5ebc15b370 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000b3f0 tx=0x7f5eb000b8b0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 shutdown_connections 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5ea806c920 0x7f5ea806ede0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ebc162880 0x7f5ebc15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0d3bc0 0x7f5ebc15b370 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 --2- 172.21.15.40:0/665507467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ebc0b5f20 0x7f5ebc155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 >> 172.21.15.40:0/665507467 conn(0x7f5ebc0d4f60 msgr2=0x7f5ebc15f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 shutdown_connections 2024-04-26T21:43:02.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.753+0000 7f5ec2d71640 1 -- 172.21.15.40:0/665507467 wait complete. 2024-04-26T21:43:02.768 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:43:02.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.812+0000 7fbda2378f40 1 Processor -- start 2024-04-26T21:43:02.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbda2378f40 1 -- start start 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbda2378f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4ed89a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbda2378f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee47d60 0x559c4ee5c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbda2378f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c4ee65610 0x559c4ee5fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbda2378f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559c4ee48140 con 0x559c4edd1670 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbda2378f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559c4ed31e40 con 0x559c4ee65610 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbda2378f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559c4ed8adc0 con 0x559c4ee47d60 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbd9fe74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4ed89a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbd9fe74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4ed89a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59608/0 (socket says 172.21.15.40:59608) 2024-04-26T21:43:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.813+0000 7fbd9fe74640 1 -- 172.21.15.40:0/4226258920 learned_addr learned my addr 172.21.15.40:0/4226258920 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd97fff640 1 --2- 172.21.15.40:0/4226258920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee47d60 0x559c4ee5c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9fe74640 1 -- 172.21.15.40:0/4226258920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee47d60 msgr2=0x559c4ee5c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/4226258920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee47d60 0x559c4ee5c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9fe74640 1 -- 172.21.15.40:0/4226258920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c4ee65610 msgr2=0x559c4ee5fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/4226258920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c4ee65610 0x559c4ee5fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9fe74640 1 -- 172.21.15.40:0/4226258920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4edb4060 con 0x559c4edd1670 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/4226258920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4ed89a30 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fbd90001e30 tx=0x7fbd9000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15276dae4a86574b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9ee72640 1 -- 172.21.15.40:0/4226258920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90012020 con 0x559c4edd1670 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9ee72640 1 -- 172.21.15.40:0/4226258920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9000e670 con 0x559c4edd1670 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.814+0000 7fbd9ee72640 1 -- 172.21.15.40:0/4226258920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000ea80 con 0x559c4edd1670 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 -- 172.21.15.40:0/4226258920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 msgr2=0x559c4ed89a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 --2- 172.21.15.40:0/4226258920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4ed89a30 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fbd90001e30 tx=0x7fbd9000b780 comp rx=0 tx=0).stop 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 -- 172.21.15.40:0/4226258920 shutdown_connections 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 --2- 172.21.15.40:0/4226258920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c4ee65610 0x559c4ee5fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 --2- 172.21.15.40:0/4226258920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee47d60 0x559c4ee5c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 --2- 172.21.15.40:0/4226258920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4ed89a30 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 -- 172.21.15.40:0/4226258920 >> 172.21.15.40:0/4226258920 conn(0x559c4ed88ab0 msgr2=0x559c4ed87f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:02.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 -- 172.21.15.40:0/4226258920 shutdown_connections 2024-04-26T21:43:02.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 -- 172.21.15.40:0/4226258920 wait complete. 2024-04-26T21:43:02.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 Processor -- start 2024-04-26T21:43:02.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.815+0000 7fbda2378f40 1 -- start start 2024-04-26T21:43:02.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbda2378f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4eef8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbda2378f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c4ee47d60 0x559c4edb4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbda2378f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee65610 0x559c4ee44560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbda2378f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559c4edb4340 con 0x559c4edd1670 2024-04-26T21:43:02.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbda2378f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559c4edb59f0 con 0x559c4ee47d60 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbda2378f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559c4ee46f50 con 0x559c4ee65610 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbd9fe74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4eef8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbd9fe74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4eef8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59618/0 (socket says 172.21.15.40:59618) 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 learned_addr learned my addr 172.21.15.40:0/3419123241 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbda0675640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee65610 0x559c4ee44560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.816+0000 7fbd97fff640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c4ee47d60 0x559c4edb4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee65610 msgr2=0x559c4ee44560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559c4ee65610 0x559c4ee44560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c4ee47d60 msgr2=0x559c4edb4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c4ee47d60 0x559c4edb4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4ee64920 con 0x559c4edd1670 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4eef8740 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000b7c0 tx=0x7fbd90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9de70640 1 -- 172.21.15.40:0/3419123241 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000e470 con 0x559c4edd1670 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9de70640 1 -- 172.21.15.40:0/3419123241 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd900197f0 con 0x559c4edd1670 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbd9de70640 1 -- 172.21.15.40:0/3419123241 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90019c00 con 0x559c4edd1670 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559c4ee64ba0 con 0x559c4edd1670 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.817+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4ee65140 con 0x559c4edd1670 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.818+0000 7fbd9de70640 1 -- 172.21.15.40:0/3419123241 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd90015020 con 0x559c4edd1670 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.819+0000 7fbd9de70640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8006b2d0 0x7fbd8006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.819+0000 7fbd9de70640 1 -- 172.21.15.40:0/3419123241 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7fbd9008c7f0 con 0x559c4edd1670 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.819+0000 7fbd97fff640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8006b2d0 0x7fbd8006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.822 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:43:02.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.819+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559c4edb5080 0x559c4ee62de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.819+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ee5efd0 con 0x559c4edb5080 2024-04-26T21:43:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.819+0000 7fbda0675640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559c4edb5080 0x559c4ee62de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.819+0000 7fbda0675640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559c4edb5080 0x559c4ee62de0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.820+0000 7fbd97fff640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8006b2d0 0x7fbd8006d790 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbd88057000 tx=0x7fbd88052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.821+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v101'164 uv138 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.821+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ee615e0 con 0x559c4edb5080 2024-04-26T21:43:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.822+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v101'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.822+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4edb5540 con 0x559c4edb5080 2024-04-26T21:43:02.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.823+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v101'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.825 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:43:02.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.823+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05d10 con 0x559c4edb5080 2024-04-26T21:43:02.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.825+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v101'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.825+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef060b0 con 0x559c4edb5080 2024-04-26T21:43:02.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.826+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v101'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.826+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef04d00 con 0x559c4edb5080 2024-04-26T21:43:02.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.827+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v101'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.827+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef050f0 con 0x559c4edb5080 2024-04-26T21:43:02.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.828+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v101'170 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.828+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef054e0 con 0x559c4edb5080 2024-04-26T21:43:02.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.829+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v101'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.829+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef054e0 con 0x559c4edb5080 2024-04-26T21:43:02.831 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:43:02.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.830+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v101'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.830+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.831+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v101'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.831+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.832+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v101'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.832+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.833+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v101'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.833+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.834+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v101'176 uv151 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.834+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.835+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v101'177 uv177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.835+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.836+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v101'178 uv178 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.836+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.837+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v101'179 uv179 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.837+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.838+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v101'180 uv180 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.838+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05490 con 0x559c4edb5080 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.839+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v101'181 uv181 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.839+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05910 con 0x559c4edb5080 2024-04-26T21:43:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.840+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v101'182 uv182 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.840+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.841+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v101'183 uv157 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.841+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.842+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v101'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.842+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.843+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v101'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.843+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.844+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v101'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.844+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.847+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v101'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.847+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.848+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v101'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.848+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.849+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v101'189 uv163 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.849+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.850+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v101'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.850+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.851+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v101'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.851 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:43:02.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.851+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.852+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v101'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.852+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.853+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v101'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.853+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4edb5080 2024-04-26T21:43:02.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.854+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v101'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4edb5080 2024-04-26T21:43:02.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.854+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559c4ef06960 0x559c4ef08d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.854+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef05740 con 0x559c4ef06960 2024-04-26T21:43:02.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.854+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559c4ef06960 0x559c4ef08d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.854+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559c4ef06960 0x559c4ef08d50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.856+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v101'129 uv113 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.856+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef09290 con 0x559c4ef06960 2024-04-26T21:43:02.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.857+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v101'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.857+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef09630 con 0x559c4ef06960 2024-04-26T21:43:02.859 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:43:02.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.858+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v101'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.858+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef09e80 con 0x559c4ef06960 2024-04-26T21:43:02.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.860+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v101'132 uv132 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.860+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a220 con 0x559c4ef06960 2024-04-26T21:43:02.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.861+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v101'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.861+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a5c0 con 0x559c4ef06960 2024-04-26T21:43:02.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.862+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v101'134 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.862+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.864+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v101'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.864+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.866 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:43:02.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.865+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v101'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.865+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.866+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v101'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.866+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.868+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v101'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.868+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.870 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:43:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.869+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v101'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.869+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0f020 con 0x559c4ef06960 2024-04-26T21:43:02.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.870+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [write 0~18075] v101'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.870+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0f020 con 0x559c4ef06960 2024-04-26T21:43:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.871+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v101'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.871+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.874+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 606.00000000 [create] v101'142 uv127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.874+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.876+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 606.00000000 [delete] v101'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.876+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.877 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:43:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.877+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v101'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.877+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0b970 con 0x559c4ef06960 2024-04-26T21:43:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.878+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v101'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.878+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0b970 con 0x559c4ef06960 2024-04-26T21:43:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.879+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v101'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.879+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0a960 con 0x559c4ef06960 2024-04-26T21:43:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.881+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v101'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef06960 2024-04-26T21:43:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.881+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559c4ef0ae70 0x559c4ef0d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.881+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0d8a0 con 0x559c4ef0ae70 2024-04-26T21:43:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.881+0000 7fbd97fff640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559c4ef0ae70 0x559c4ef0d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.881+0000 7fbd97fff640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559c4ef0ae70 0x559c4ef0d260 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.883+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(51 607.00000000 [create] v101'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.883+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef0dd90 con 0x559c4ef0ae70 2024-04-26T21:43:02.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.884+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(52 607.00000000 [delete] v101'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.884+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0e270 con 0x559c4ef0ae70 2024-04-26T21:43:02.886 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:43:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.885+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(53 607.00000000 [create] v101'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.885+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0fb20 con 0x559c4ef0ae70 2024-04-26T21:43:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.887+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v101'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.887+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0fec0 con 0x559c4ef0ae70 2024-04-26T21:43:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.888+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v101'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.888+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0eb10 con 0x559c4ef0ae70 2024-04-26T21:43:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.889+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v101'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.889+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559c4ef0f430 0x559c4ef12270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.889+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef127b0 con 0x559c4ef0f430 2024-04-26T21:43:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.890+0000 7fbda0675640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559c4ef0f430 0x559c4ef12270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.890+0000 7fbda0675640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559c4ef0f430 0x559c4ef12270 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.891+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v101'26 uv25 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.891+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef127b0 con 0x559c4ef0f430 2024-04-26T21:43:02.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.893+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v101'27 uv27 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.893+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef12830 con 0x559c4ef0f430 2024-04-26T21:43:02.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.894+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v101'28 uv28 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.895 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:43:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.894+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef0e760 con 0x559c4ef0f430 2024-04-26T21:43:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.895+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v101'29 uv29 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.895+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559c4ef13420 0x559c4ef15810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.895+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef15e20 con 0x559c4ef13420 2024-04-26T21:43:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.895+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559c4ef13420 0x559c4ef15810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.895+0000 7fbd9fe74640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559c4ef13420 0x559c4ef15810 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.898+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(61 608.00000000 [create] v101'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.898+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef15ea0 con 0x559c4ef13420 2024-04-26T21:43:02.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.901+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(62 608.00000000 [delete] v101'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.901+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.904 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:43:02.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.903+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(63 608.00000000 [create] v101'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.904+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef172d0 con 0x559c4ef13420 2024-04-26T21:43:02.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.907+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v101'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.907+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef172d0 con 0x559c4ef13420 2024-04-26T21:43:02.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.909+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v101'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.909+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.913+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v101'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.913+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.913+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(67 604.00000000 [create] v101'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.913+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.914+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(68 604.00000000 [delete] v101'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.914+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.916+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(69 604.00000000 [create] v101'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.917 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:43:02.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.916+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef172d0 con 0x559c4ef0f430 2024-04-26T21:43:02.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.918+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v101'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.918+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef172d0 con 0x559c4ef0f430 2024-04-26T21:43:02.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.921+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v101'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.921+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.922+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v101'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.922+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.925+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(73 500.00000000 [create] v101'65 uv58 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.925+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.927+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(74 500.00000000 [delete] v101'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.927+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.932 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:43:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.931+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(75 500.00000000 [create] v101'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.931+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.933+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v101'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.933+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.935+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v101'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.935+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.937+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(78 mds_snaptable [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.937+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.939+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(79 mds_snaptable [delete] v101'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.939+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.943 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:43:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.942+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(80 mds_snaptable [create] v101'29 uv29 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.943+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.944+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(81 mds_snaptable [write 0~70] v101'30 uv30 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.944+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef0f430 2024-04-26T21:43:02.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.946+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v101'31 uv31 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbd9800d6a0 con 0x559c4ef0f430 2024-04-26T21:43:02.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.946+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.948+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(83 605.00000000 [create] v101'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.948+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.949+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(84 605.00000000 [delete] v101'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.949+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.950 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:43:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.950+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(85 605.00000000 [create] v101'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.950+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef172d0 con 0x559c4ef13420 2024-04-26T21:43:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.951+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v101'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.951+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef172d0 con 0x559c4ef13420 2024-04-26T21:43:02.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.952+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v101'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.952+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.953+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(88 605.00000000 [omap-set-header] v101'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.953+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.954+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(89 mds0_inotable [create] v101'54 uv48 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.954+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.955+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(90 mds0_inotable [delete] v101'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.955+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.957 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:43:02.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.957+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(91 mds0_inotable [create] v101'56 uv56 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.957+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef16680 con 0x559c4ef13420 2024-04-26T21:43:02.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.958+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(92 mds0_inotable [write 0~34] v101'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.958+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162e0 con 0x559c4ef13420 2024-04-26T21:43:02.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.959+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v101'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.959+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162e0 con 0x559c4ef13420 2024-04-26T21:43:02.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.960+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(94 100.00000000 [create] v101'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.960+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef162e0 con 0x559c4ef13420 2024-04-26T21:43:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.961+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(95 100.00000000 [delete] v101'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.961+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.963 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:43:02.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.962+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(96 100.00000000 [create] v101'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.962+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef162c0 con 0x559c4ef13420 2024-04-26T21:43:02.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.963+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(97 100.00000000 [omap-set-header] v101'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.963+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef128a0 con 0x559c4ef13420 2024-04-26T21:43:02.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.965+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v101'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.965+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559c4ef170a0 0x559c4ef197f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.965+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef19d30 con 0x559c4ef170a0 2024-04-26T21:43:02.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.965+0000 7fbd97fff640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559c4ef170a0 0x559c4ef197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.965+0000 7fbd97fff640 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559c4ef170a0 0x559c4ef197f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.967+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(99 mds0_sessionmap [create] v101'26 uv25 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef170a0 2024-04-26T21:43:02.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.967+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef170a0 2024-04-26T21:43:02.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.968+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v101'27 uv27 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef170a0 2024-04-26T21:43:02.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.968+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef170a0 2024-04-26T21:43:02.971 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:43:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.971+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(101 mds0_sessionmap [create] v101'28 uv28 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef170a0 2024-04-26T21:43:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.971+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef170a0 2024-04-26T21:43:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.972+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v101'29 uv29 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef170a0 2024-04-26T21:43:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.972+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef13420 2024-04-26T21:43:02.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.973+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(103 609.00000000 [create] v101'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.973+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef13420 2024-04-26T21:43:02.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.975+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(104 609.00000000 [delete] v101'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.975+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef13420 2024-04-26T21:43:02.977 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:43:02.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.976+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(105 609.00000000 [create] v101'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.976+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1b0e0 con 0x559c4ef13420 2024-04-26T21:43:02.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.977+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v101'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.978+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1b0e0 con 0x559c4ef13420 2024-04-26T21:43:02.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.979+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v101'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.979+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef13420 2024-04-26T21:43:02.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.980+0000 7fbd9fe74640 1 -- 172.21.15.40:0/3419123241 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(108 609.00000000 [omap-set-header] v101'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd90029020 con 0x559c4ef13420 2024-04-26T21:43:02.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.980+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.981+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.981+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.982+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.982+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.984 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:43:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.983+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.983+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.984+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.984+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.985+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.985+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.986+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v101'69 uv61 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.986+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.987+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v101'70 uv70 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.987+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.988+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v101'71 uv71 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.989 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:43:02.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.988+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.989+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v101'72 uv72 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.989+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.990+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v101'73 uv73 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.990+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.991+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v101'74 uv68 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.991+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.992+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v101'75 uv75 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.992+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.993+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v101'76 uv76 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.995 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:43:02.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.993+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1b0e0 con 0x559c4ef0ae70 2024-04-26T21:43:02.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.994+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v101'77 uv77 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.994+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1b0e0 con 0x559c4ef0ae70 2024-04-26T21:43:02.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.995+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v101'78 uv78 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.995+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:02.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.996+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v101'79 uv79 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:02.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.996+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559c4ef1a0d0 con 0x559c4ef0ae70 2024-04-26T21:43:03.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbd97fff640 1 -- 172.21.15.40:0/3419123241 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v101'80 uv80 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbd88054b20 con 0x559c4ef0ae70 2024-04-26T21:43:03.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559c4ef06960 msgr2=0x559c4ef08d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559c4ef06960 0x559c4ef08d50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559c4edb5080 msgr2=0x559c4ee62de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559c4edb5080 0x559c4ee62de0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559c4ef0f430 msgr2=0x559c4ef12270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559c4ef0f430 0x559c4ef12270 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559c4ef170a0 msgr2=0x559c4ef197f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559c4ef170a0 0x559c4ef197f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559c4ef0ae70 msgr2=0x559c4ef0d260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559c4ef0ae70 0x559c4ef0d260 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 reap_dead start 2024-04-26T21:43:03.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559c4ef13420 msgr2=0x559c4ef15810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559c4ef13420 0x559c4ef15810 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 reap_dead start 2024-04-26T21:43:03.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.997+0000 7fbda0675640 1 -- 172.21.15.40:0/3419123241 reap_dead start 2024-04-26T21:43:03.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8006b2d0 msgr2=0x7fbd8006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8006b2d0 0x7fbd8006d790 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbd88057000 tx=0x7fbd88052000 comp rx=0 tx=0).stop 2024-04-26T21:43:03.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 msgr2=0x559c4eef8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4eef8740 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000b7c0 tx=0x7fbd90000bc0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 shutdown_connections 2024-04-26T21:43:03.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbd8006b2d0 0x7fbd8006d790 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 --2- 172.21.15.40:0/3419123241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559c4edd1670 0x559c4eef8740 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 >> 172.21.15.40:0/3419123241 conn(0x559c4ed88ab0 msgr2=0x559c4ed87f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:03.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 shutdown_connections 2024-04-26T21:43:03.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:02.998+0000 7fbda2378f40 1 -- 172.21.15.40:0/3419123241 wait complete. 2024-04-26T21:43:03.006 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_openfiles.0 2024-04-26T21:43:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.088+0000 7fcf13677f40 1 Processor -- start 2024-04-26T21:43:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.088+0000 7fcf13677f40 1 -- start start 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf13677f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b04480 0x562c54ad67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf13677f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54b1d6e0 0x562c54ba5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf13677f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54ba9820 0x562c54ba6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf13677f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562c54b048d0 con 0x562c54b1d6e0 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf13677f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562c54a7be40 con 0x562c54b04480 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf13677f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562c54ba9c00 con 0x562c54ba9820 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54b1d6e0 0x562c54ba5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54b1d6e0 0x562c54ba5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59622/0 (socket says 172.21.15.40:59622) 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf0bfff640 1 -- 172.21.15.40:0/2598342612 learned_addr learned my addr 172.21.15.40:0/2598342612 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf0bfff640 1 -- 172.21.15.40:0/2598342612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54ba9820 msgr2=0x562c54ba6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf11972640 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54ba9820 0x562c54ba6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf0bfff640 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54ba9820 0x562c54ba6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf11171640 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b04480 0x562c54ad67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf0bfff640 1 -- 172.21.15.40:0/2598342612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b04480 msgr2=0x562c54ad67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf0bfff640 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b04480 0x562c54ad67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.089+0000 7fcf0bfff640 1 -- 172.21.15.40:0/2598342612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c54b1ae80 con 0x562c54b1d6e0 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf0bfff640 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54b1d6e0 0x562c54ba5ed0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00b2c0 tx=0x7fcefc00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde12297b7cae9f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf0b7fe640 1 -- 172.21.15.40:0/2598342612 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc008070 con 0x562c54b1d6e0 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf0b7fe640 1 -- 172.21.15.40:0/2598342612 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcefc00d500 con 0x562c54b1d6e0 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf0b7fe640 1 -- 172.21.15.40:0/2598342612 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00d8d0 con 0x562c54b1d6e0 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 -- 172.21.15.40:0/2598342612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54b1d6e0 msgr2=0x562c54ba5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54b1d6e0 0x562c54ba5ed0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00b2c0 tx=0x7fcefc00b780 comp rx=0 tx=0).stop 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 -- 172.21.15.40:0/2598342612 shutdown_connections 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54ba9820 0x562c54ba6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54b1d6e0 0x562c54ba5ed0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 --2- 172.21.15.40:0/2598342612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b04480 0x562c54ad67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 -- 172.21.15.40:0/2598342612 >> 172.21.15.40:0/2598342612 conn(0x562c54afe8a0 msgr2=0x562c54b1bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 -- 172.21.15.40:0/2598342612 shutdown_connections 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.090+0000 7fcf13677f40 1 -- 172.21.15.40:0/2598342612 wait complete. 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.091+0000 7fcf13677f40 1 Processor -- start 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.091+0000 7fcf13677f40 1 -- start start 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf13677f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54b04480 0x562c54b99600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf13677f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b1d6e0 0x562c54ba2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf13677f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54ba9820 0x562c54ba2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf13677f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562c54ad54f0 con 0x562c54ba9820 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf13677f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562c54baca60 con 0x562c54b1d6e0 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf13677f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562c54ba6950 con 0x562c54b04480 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54ba9820 0x562c54ba2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11171640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54b04480 0x562c54b99600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b1d6e0 0x562c54ba2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54ba9820 0x562c54ba2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59628/0 (socket says 172.21.15.40:59628) 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 -- 172.21.15.40:0/827039508 learned_addr learned my addr 172.21.15.40:0/827039508 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 -- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54b04480 msgr2=0x562c54b99600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562c54b04480 0x562c54b99600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 -- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b1d6e0 msgr2=0x562c54ba2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c54b1d6e0 0x562c54ba2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 -- 172.21.15.40:0/827039508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c54b9a670 con 0x562c54ba9820 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf11972640 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54ba9820 0x562c54ba2f40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c046100 tx=0x7fcf0c0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf0a7fc640 1 -- 172.21.15.40:0/827039508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c068070 con 0x562c54ba9820 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf0a7fc640 1 -- 172.21.15.40:0/827039508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c058400 con 0x562c54ba9820 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562c54b99ed0 con 0x562c54ba9820 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf0a7fc640 1 -- 172.21.15.40:0/827039508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c050610 con 0x562c54ba9820 2024-04-26T21:43:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.092+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c54b99b40 con 0x562c54ba9820 2024-04-26T21:43:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.094+0000 7fcf0a7fc640 1 -- 172.21.15.40:0/827039508 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf0c06b070 con 0x562c54ba9820 2024-04-26T21:43:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.094+0000 7fcf0a7fc640 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcef406b2d0 0x7fcef406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.094+0000 7fcf0a7fc640 1 -- 172.21.15.40:0/827039508 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7fcf0c0ce750 con 0x562c54ba9820 2024-04-26T21:43:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.094+0000 7fcf11171640 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcef406b2d0 0x7fcef406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.094+0000 7fcf13677f40 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562c54b9a1b0 0x562c54b9bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.094+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x562c54b9c400 con 0x562c54b9a1b0 2024-04-26T21:43:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.095+0000 7fcf0bfff640 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562c54b9a1b0 0x562c54b9bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.095+0000 7fcf11171640 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcef406b2d0 0x7fcef406d790 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x562c54ba0730 tx=0x7fcf04005990 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.095+0000 7fcf0bfff640 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562c54b9a1b0 0x562c54b9bec0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.096+0000 7fcf0bfff640 1 -- 172.21.15.40:0/827039508 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds0_openfiles.0 [delete] v101'30 uv30 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcefc00d730 con 0x562c54b9a1b0 2024-04-26T21:43:03.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.096+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562c54b9a1b0 msgr2=0x562c54b9bec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.096+0000 7fcf13677f40 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562c54b9a1b0 0x562c54b9bec0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcef406b2d0 msgr2=0x7fcef406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf13677f40 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcef406b2d0 0x7fcef406d790 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x562c54ba0730 tx=0x7fcf04005990 comp rx=0 tx=0).stop 2024-04-26T21:43:03.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54ba9820 msgr2=0x562c54ba2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf13677f40 1 --2- 172.21.15.40:0/827039508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c54ba9820 0x562c54ba2f40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c046100 tx=0x7fcf0c0033a0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf11972640 1 -- 172.21.15.40:0/827039508 reap_dead start 2024-04-26T21:43:03.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 shutdown_connections 2024-04-26T21:43:03.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 >> 172.21.15.40:0/827039508 conn(0x562c54afe8a0 msgr2=0x562c54ad67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:03.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 shutdown_connections 2024-04-26T21:43:03.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.097+0000 7fcf13677f40 1 -- 172.21.15.40:0/827039508 wait complete. 2024-04-26T21:43:03.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:43:03.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.245+0000 7f1afe1f0640 1 Processor -- start 2024-04-26T21:43:03.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.245+0000 7f1afe1f0640 1 -- start start 2024-04-26T21:43:03.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1afe1f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1af80cfde0 0x7f1af80d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1afe1f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af8070350 0x7f1af8070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1afe1f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af80d2390 con 0x7f1af8071090 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1afe1f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1af8071500 con 0x7f1af80cfde0 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1afe1f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1af80afab0 con 0x7f1af8070350 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1af8071090 0x7f1af815e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52700/0 (socket says 172.21.15.40:52700) 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af7fff640 1 -- 172.21.15.40:0/3604929310 learned_addr learned my addr 172.21.15.40:0/3604929310 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af6ffd640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af8070350 0x7f1af8070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274370877 0 0) 0x7f1af80d2390 con 0x7f1af8071090 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ae0003520 con 0x7f1af8071090 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af77fe640 1 --2- 172.21.15.40:0/3604929310 >> v2:172.21.15.184:3300/0 conn(0x7f1af80cfde0 0x7f1af80d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 859217123 0 0) 0x7f1ae0003520 con 0x7f1af8071090 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af8070350 msgr2=0x7f1af8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af67fc640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af8070350 0x7f1af8070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 >> v2:172.21.15.184:3300/0 conn(0x7f1af80cfde0 msgr2=0x7f1af80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:03.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.246+0000 7f1af67fc640 1 --2- 172.21.15.40:0/3604929310 >> v2:172.21.15.184:3300/0 conn(0x7f1af80cfde0 0x7f1af80d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af806d3f0 con 0x7f1af8071090 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1af00019f0 con 0x7f1af8071090 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 >> v1:172.21.15.40:6789/0 conn(0x7f1af8071090 legacy=0x7f1af815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af8070350 0x7f1ae00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1ae0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae0004d40 0x7f1ae00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8071500 con 0x7f1ae0003f30 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae000b9c0 con 0x7f1ae0004d40 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae000bb90 con 0x7f1af8070350 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af6ffd640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1ae0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af77fe640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af8070350 0x7f1ae00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af7fff640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae0004d40 0x7f1ae00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af6ffd640 1 -- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af8070350 msgr2=0x7f1ae00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af6ffd640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af8070350 0x7f1ae00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af6ffd640 1 -- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae0004d40 msgr2=0x7f1ae00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af6ffd640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae0004d40 0x7f1ae00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af6ffd640 1 -- 172.21.15.40:0/3604929310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae0003520 con 0x7f1ae0003f30 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.247+0000 7f1af7fff640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae0004d40 0x7f1ae00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1af6ffd640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1ae0004800 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009af0 tx=0x7f1ae0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761d6c58b2f49e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec036070 con 0x7f1ae0003f30 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aec031400 con 0x7f1ae0003f30 2024-04-26T21:43:03.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1af67fc640 1 -- 172.21.15.40:0/3604929310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec009690 con 0x7f1ae0003f30 2024-04-26T21:43:03.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1afe1f0640 1 -- 172.21.15.40:0/3604929310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 msgr2=0x7f1ae0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1afe1f0640 1 --2- 172.21.15.40:0/3604929310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1ae0004800 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009af0 tx=0x7f1ae0005830 comp rx=0 tx=0).stop 2024-04-26T21:43:03.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1af7fff640 1 -- 172.21.15.40:0/3604929310 reap_dead start 2024-04-26T21:43:03.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1afe1f0640 1 -- 172.21.15.40:0/3604929310 shutdown_connections 2024-04-26T21:43:03.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1afe1f0640 1 -- 172.21.15.40:0/3604929310 >> 172.21.15.40:0/3604929310 conn(0x7f1af80d2d80 msgr2=0x7f1af80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:03.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.248+0000 7f1afe1f0640 1 -- 172.21.15.40:0/3604929310 shutdown_connections 2024-04-26T21:43:03.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.249+0000 7f1afe1f0640 1 -- 172.21.15.40:0/3604929310 wait complete. 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.250+0000 7f1afe1f0640 1 Processor -- start 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.250+0000 7f1afe1f0640 1 -- start start 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1afe1f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1af81f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1afe1f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af80b8210 0x7f1af8157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1afe1f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1af809fa20 0x7f1af80b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1afe1f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1af806d9b0 con 0x7f1ae0003f30 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1afe1f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1af806dd20 con 0x7f1af809fa20 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1afe1f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1af80d20a0 con 0x7f1af80b8210 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1af81f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1af81f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59646/0 (socket says 172.21.15.40:59646) 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 -- 172.21.15.40:0/1563424455 learned_addr learned my addr 172.21.15.40:0/1563424455 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af6ffd640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af80b8210 0x7f1af8157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af7fff640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1af809fa20 0x7f1af80b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 -- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af80b8210 msgr2=0x7f1af8157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af80b8210 0x7f1af8157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 -- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1af809fa20 msgr2=0x7f1af80b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1af809fa20 0x7f1af80b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 -- 172.21.15.40:0/1563424455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af8157670 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1af77fe640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1af81f86e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8011aa0 tx=0x7f1ae8011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1adffff640 1 -- 172.21.15.40:0/1563424455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800fac0 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1adffff640 1 -- 172.21.15.40:0/1563424455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae800fc60 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af80a0390 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.251+0000 7f1adffff640 1 -- 172.21.15.40:0/1563424455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8014440 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.252+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af81f7ee0 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.252+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af80694f0 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.253+0000 7f1adffff640 1 -- 172.21.15.40:0/1563424455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ae8012040 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.253+0000 7f1adffff640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1ae406c920 0x7f1ae406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.253+0000 7f1adffff640 1 -- 172.21.15.40:0/1563424455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f1ae808c4b0 con 0x7f1ae0003f30 2024-04-26T21:43:03.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.253+0000 7f1af6ffd640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1ae406c920 0x7f1ae406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:03.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.254+0000 7f1af6ffd640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1ae406c920 0x7f1ae406ede0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1aec033000 tx=0x7f1aec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:03.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.258+0000 7f1adffff640 1 -- 172.21.15.40:0/1563424455 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ae8019030 con 0x7f1ae0003f30 2024-04-26T21:43:03.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.483+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1af81f7c90 con 0x7f1ae0003f30 2024-04-26T21:43:03.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.757+0000 7f1adffff640 1 -- 172.21.15.40:0/1563424455 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v125) ==== 162+0+0 (secure 0 0 0) 0x7f1ae805a570 con 0x7f1ae0003f30 2024-04-26T21:43:03.758 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:43:03.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1ae406c920 msgr2=0x7f1ae406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1ae406c920 0x7f1ae406ede0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1aec033000 tx=0x7f1aec008030 comp rx=0 tx=0).stop 2024-04-26T21:43:03.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 msgr2=0x7f1af81f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:03.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1af81f86e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8011aa0 tx=0x7f1ae8011ea0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 shutdown_connections 2024-04-26T21:43:03.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1ae406c920 0x7f1ae406ede0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1af809fa20 0x7f1af80b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1af80b8210 0x7f1af8157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 --2- 172.21.15.40:0/1563424455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae0003f30 0x7f1af81f86e0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:03.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 >> 172.21.15.40:0/1563424455 conn(0x7f1af80d2d80 msgr2=0x7f1af81624c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:03.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 shutdown_connections 2024-04-26T21:43:03.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:03.759+0000 7f1afe1f0640 1 -- 172.21.15.40:0/1563424455 wait complete. 2024-04-26T21:43:03.769 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:03.768+0000 7f7a3e43b640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-04-26T21:43:04.176 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:04.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.290+0000 7f00b3c83640 1 Processor -- start 2024-04-26T21:43:04.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.290+0000 7f00b3c83640 1 -- start start 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b3c83640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00ac0b2340 0x7f00ac0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b3c83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac0cd930 0x7f00ac0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b3c83640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ac0d10d0 con 0x7f00ac162810 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b3c83640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f00ac0d0750 con 0x7f00ac0b2340 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b3c83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac0d16a0 con 0x7f00ac0cd930 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b2220640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac0cd930 0x7f00ac0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b121e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00ac0b2340 0x7f00ac0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b2220640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac0cd930 0x7f00ac0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:51722/0 (socket says 172.21.15.40:51722) 2024-04-26T21:43:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b1a1f640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f00ac162810 0x7f00ac070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:49540/0 (socket says 172.21.15.40:49540) 2024-04-26T21:43:04.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.291+0000 7f00b2220640 1 -- 172.21.15.40:0/359220528 learned_addr learned my addr 172.21.15.40:0/359220528 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:04.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.292+0000 7f00b2220640 1 -- 172.21.15.40:0/359220528 >> v2:172.21.15.184:3300/0 conn(0x7f00ac0b2340 msgr2=0x7f00ac0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:04.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.292+0000 7f00b2220640 1 --2- 172.21.15.40:0/359220528 >> v2:172.21.15.184:3300/0 conn(0x7f00ac0b2340 0x7f00ac0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.292+0000 7f00b2220640 1 -- 172.21.15.40:0/359220528 >> v1:172.21.15.184:6789/0 conn(0x7f00ac162810 legacy=0x7f00ac070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:04.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.292+0000 7f00b2220640 1 -- 172.21.15.40:0/359220528 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac0d1370 con 0x7f00ac0cd930 2024-04-26T21:43:04.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.292+0000 7f00b121e640 1 --2- 172.21.15.40:0/359220528 >> v2:172.21.15.184:3300/0 conn(0x7f00ac0b2340 0x7f00ac0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:43:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.292+0000 7f00b2220640 1 --2- 172.21.15.40:0/359220528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac0cd930 0x7f00ac0d1b30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0d1820 tx=0x7f00a402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f3596bd31c3291 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b0a1d640 1 -- 172.21.15.40:0/359220528 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4036070 con 0x7f00ac0cd930 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b0a1d640 1 -- 172.21.15.40:0/359220528 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4031600 con 0x7f00ac0cd930 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b0a1d640 1 -- 172.21.15.40:0/359220528 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a40319d0 con 0x7f00ac0cd930 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b3c83640 1 -- 172.21.15.40:0/359220528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac0cd930 msgr2=0x7f00ac0d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b3c83640 1 --2- 172.21.15.40:0/359220528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac0cd930 0x7f00ac0d1b30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0d1820 tx=0x7f00a402a710 comp rx=0 tx=0).stop 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b3c83640 1 -- 172.21.15.40:0/359220528 shutdown_connections 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b3c83640 1 --2- 172.21.15.40:0/359220528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac0cd930 0x7f00ac0d1b30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b3c83640 1 --2- 172.21.15.40:0/359220528 >> v2:172.21.15.184:3300/0 conn(0x7f00ac0b2340 0x7f00ac0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b3c83640 1 -- 172.21.15.40:0/359220528 >> 172.21.15.40:0/359220528 conn(0x7f00ac0d2da0 msgr2=0x7f00ac0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b3c83640 1 -- 172.21.15.40:0/359220528 shutdown_connections 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.293+0000 7f00b3c83640 1 -- 172.21.15.40:0/359220528 wait complete. 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b3c83640 1 Processor -- start 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b3c83640 1 -- start start 2024-04-26T21:43:04.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b3c83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00ac0b2340 0x7f00ac15f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b3c83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ac0cd930 0x7f00ac0ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b3c83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac162810 0x7f00ac153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b3c83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac0d2070 con 0x7f00ac0cd930 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b3c83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac0d0470 con 0x7f00ac0b2340 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b3c83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac0d21f0 con 0x7f00ac162810 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b121e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ac0cd930 0x7f00ac0ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b121e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ac0cd930 0x7f00ac0ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59654/0 (socket says 172.21.15.40:59654) 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.295+0000 7f00b121e640 1 -- 172.21.15.40:0/616970585 learned_addr learned my addr 172.21.15.40:0/616970585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b2220640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac162810 0x7f00ac153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b1a1f640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00ac0b2340 0x7f00ac15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b121e640 1 -- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac162810 msgr2=0x7f00ac153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b121e640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac162810 0x7f00ac153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b121e640 1 -- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00ac0b2340 msgr2=0x7f00ac15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b121e640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00ac0b2340 0x7f00ac15f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b121e640 1 -- 172.21.15.40:0/616970585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac15fb60 con 0x7f00ac0cd930 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b121e640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ac0cd930 0x7f00ac0ce1d0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f009c00ec10 tx=0x7f009c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00a27fc640 1 -- 172.21.15.40:0/616970585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c017070 con 0x7f00ac0cd930 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00a27fc640 1 -- 172.21.15.40:0/616970585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c00cc30 con 0x7f00ac0cd930 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac1f85e0 con 0x7f00ac0cd930 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00a27fc640 1 -- 172.21.15.40:0/616970585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c013470 con 0x7f00ac0cd930 2024-04-26T21:43:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.296+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1f8ba0 con 0x7f00ac0cd930 2024-04-26T21:43:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.297+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ac0c9e50 con 0x7f00ac0cd930 2024-04-26T21:43:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.298+0000 7f00a27fc640 1 -- 172.21.15.40:0/616970585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f009c01a070 con 0x7f00ac0cd930 2024-04-26T21:43:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.298+0000 7f00a27fc640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f009006c920 0x7f009006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.298+0000 7f00a27fc640 1 -- 172.21.15.40:0/616970585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f009c08c120 con 0x7f00ac0cd930 2024-04-26T21:43:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.298+0000 7f00b1a1f640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f009006c920 0x7f009006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.299+0000 7f00b1a1f640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f009006c920 0x7f009006ede0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0a0970 tx=0x7f00a8007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.303+0000 7f00a27fc640 1 -- 172.21.15.40:0/616970585 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00ac0c9e50 con 0x7f00ac0cd930 2024-04-26T21:43:04.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.524+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00ac0694f0 con 0x7f00ac0cd930 2024-04-26T21:43:04.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.525+0000 7f00a27fc640 1 -- 172.21.15.40:0/616970585 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) ==== 95+0+5190 (secure 0 0 0) 0x7f00ac15fb60 con 0x7f00ac0cd930 2024-04-26T21:43:04.526 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":126,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:03.761118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7743":{"gid":7743,"name":"c","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:04.526 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 126 2024-04-26T21:43:04.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f009006c920 msgr2=0x7f009006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:04.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f009006c920 0x7f009006ede0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0a0970 tx=0x7f00a8007e80 comp rx=0 tx=0).stop 2024-04-26T21:43:04.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ac0cd930 msgr2=0x7f00ac0ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ac0cd930 0x7f00ac0ce1d0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f009c00ec10 tx=0x7f009c00c520 comp rx=0 tx=0).stop 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 shutdown_connections 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f009006c920 0x7f009006ede0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00ac162810 0x7f00ac153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00ac0cd930 0x7f00ac0ce1d0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 --2- 172.21.15.40:0/616970585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00ac0b2340 0x7f00ac15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 >> 172.21.15.40:0/616970585 conn(0x7f00ac0d2da0 msgr2=0x7f00ac0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 shutdown_connections 2024-04-26T21:43:04.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:04.527+0000 7f00b3c83640 1 -- 172.21.15.40:0/616970585 wait complete. 2024-04-26T21:43:04.539 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:43:03.761118+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7743': {'gid': 7743, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/3311550037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3311550037}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3311550037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:43:04.539 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7743:up:replay 2024-04-26T21:43:04.539 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:43:09.203 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:09.203+0000 7f7a42443640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:43:09.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:10.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.057+0000 7fb449b2b640 1 Processor -- start 2024-04-26T21:43:10.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.057+0000 7fb449b2b640 1 -- start start 2024-04-26T21:43:10.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb449b2b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb4440b2340 0x7fb4440cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb449b2b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4440cd930 0x7fb4440d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb449b2b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4440d10d0 con 0x7fb444162810 2024-04-26T21:43:10.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb449b2b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb4440d0750 con 0x7fb4440cd930 2024-04-26T21:43:10.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb449b2b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb4440d16a0 con 0x7fb4440b2340 2024-04-26T21:43:10.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb443fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4440cd930 0x7fb4440d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb443fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4440cd930 0x7fb4440d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60244/0 (socket says 172.21.15.40:60244) 2024-04-26T21:43:10.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb443fff640 1 -- 172.21.15.40:0/1329325981 learned_addr learned my addr 172.21.15.40:0/1329325981 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb442ffd640 1 --2- 172.21.15.40:0/1329325981 >> v2:172.21.15.184:3300/0 conn(0x7fb4440b2340 0x7fb4440cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb443fff640 1 -- 172.21.15.40:0/1329325981 >> v2:172.21.15.184:3300/0 conn(0x7fb4440b2340 msgr2=0x7fb4440cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb443fff640 1 --2- 172.21.15.40:0/1329325981 >> v2:172.21.15.184:3300/0 conn(0x7fb4440b2340 0x7fb4440cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb443fff640 1 -- 172.21.15.40:0/1329325981 >> v1:172.21.15.184:6789/0 conn(0x7fb444162810 legacy=0x7fb444070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.058+0000 7fb443fff640 1 -- 172.21.15.40:0/1329325981 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4440d1370 con 0x7fb4440cd930 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.059+0000 7fb443fff640 1 --2- 172.21.15.40:0/1329325981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb4440d1b30 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb4440cd0a0 tx=0x7fb43c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d93d41af245e344 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.059+0000 7fb4427fc640 1 -- 172.21.15.40:0/1329325981 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c036070 con 0x7fb4440cd930 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.059+0000 7fb4427fc640 1 -- 172.21.15.40:0/1329325981 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb43c031600 con 0x7fb4440cd930 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.059+0000 7fb4427fc640 1 -- 172.21.15.40:0/1329325981 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0319d0 con 0x7fb4440cd930 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.060+0000 7fb449b2b640 1 -- 172.21.15.40:0/1329325981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 msgr2=0x7fb4440d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.060+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1329325981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb4440d1b30 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb4440cd0a0 tx=0x7fb43c02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.060+0000 7fb449b2b640 1 -- 172.21.15.40:0/1329325981 shutdown_connections 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.060+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1329325981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb4440d1b30 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.060+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1329325981 >> v2:172.21.15.184:3300/0 conn(0x7fb4440b2340 0x7fb4440cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.060+0000 7fb449b2b640 1 -- 172.21.15.40:0/1329325981 >> 172.21.15.40:0/1329325981 conn(0x7fb4440d2da0 msgr2=0x7fb4440d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.060+0000 7fb449b2b640 1 -- 172.21.15.40:0/1329325981 shutdown_connections 2024-04-26T21:43:10.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.060+0000 7fb449b2b640 1 -- 172.21.15.40:0/1329325981 wait complete. 2024-04-26T21:43:10.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.061+0000 7fb449b2b640 1 Processor -- start 2024-04-26T21:43:10.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.061+0000 7fb449b2b640 1 -- start start 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb449b2b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4440b2340 0x7fb44415c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb449b2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb44414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb449b2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb444162810 0x7fb4440ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb449b2b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb4440d2070 con 0x7fb4440cd930 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb449b2b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4440d0470 con 0x7fb4440b2340 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb449b2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4440d21f0 con 0x7fb444162810 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb442ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb44414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb442ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb44414fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60248/0 (socket says 172.21.15.40:60248) 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb442ffd640 1 -- 172.21.15.40:0/1330753531 learned_addr learned my addr 172.21.15.40:0/1330753531 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb4437fe640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4440b2340 0x7fb44415c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb443fff640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb444162810 0x7fb4440ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb442ffd640 1 -- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb444162810 msgr2=0x7fb4440ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb442ffd640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb444162810 0x7fb4440ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb442ffd640 1 -- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4440b2340 msgr2=0x7fb44415c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb442ffd640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4440b2340 0x7fb44415c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.062+0000 7fb442ffd640 1 -- 172.21.15.40:0/1330753531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44414fb60 con 0x7fb4440cd930 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.063+0000 7fb442ffd640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb44414fdb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb43400d830 tx=0x7fb43400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.063+0000 7fb42bfff640 1 -- 172.21.15.40:0/1330753531 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43400b6c0 con 0x7fb4440cd930 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.063+0000 7fb42bfff640 1 -- 172.21.15.40:0/1330753531 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb434019070 con 0x7fb4440cd930 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.063+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44415cf20 con 0x7fb4440cd930 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.063+0000 7fb42bfff640 1 -- 172.21.15.40:0/1330753531 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43400ba80 con 0x7fb4440cd930 2024-04-26T21:43:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.063+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44415d4e0 con 0x7fb4440cd930 2024-04-26T21:43:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.064+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4440694f0 con 0x7fb4440cd930 2024-04-26T21:43:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.064+0000 7fb42bfff640 1 -- 172.21.15.40:0/1330753531 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb43401e460 con 0x7fb4440cd930 2024-04-26T21:43:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.065+0000 7fb42bfff640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43006c8f0 0x7fb43006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.065+0000 7fb42bfff640 1 -- 172.21.15.40:0/1330753531 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7fb43408c630 con 0x7fb4440cd930 2024-04-26T21:43:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.065+0000 7fb4437fe640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43006c8f0 0x7fb43006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.065+0000 7fb4437fe640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43006c8f0 0x7fb43006edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb438006fd0 tx=0x7fb438008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.069+0000 7fb42bfff640 1 -- 172.21.15.40:0/1330753531 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb44414fb60 con 0x7fb4440cd930 2024-04-26T21:43:10.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.294+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4440cd170 con 0x7fb4440cd930 2024-04-26T21:43:10.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.294+0000 7fb42bfff640 1 -- 172.21.15.40:0/1330753531 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) ==== 95+0+5198 (secure 0 0 0) 0x7fb43405a660 con 0x7fb4440cd930 2024-04-26T21:43:10.294 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:10.294 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":129,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:06.779246+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7743":{"gid":7743,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":20,"addr":"172.21.15.40:6833/3311550037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3311550037},{"type":"v1","addr":"172.21.15.40:6833","nonce":3311550037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7743,"qdb_cluster":[7743]},"id":6}]} 2024-04-26T21:43:10.294 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 129 2024-04-26T21:43:10.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43006c8f0 msgr2=0x7fb43006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43006c8f0 0x7fb43006edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb438006fd0 tx=0x7fb438008040 comp rx=0 tx=0).stop 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 msgr2=0x7fb44414fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb44414fdb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb43400d830 tx=0x7fb43400dcf0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 shutdown_connections 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb43006c8f0 0x7fb43006edb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb444162810 0x7fb4440ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4440cd930 0x7fb44414fdb0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 --2- 172.21.15.40:0/1330753531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4440b2340 0x7fb44415c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.296+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 >> 172.21.15.40:0/1330753531 conn(0x7fb4440d2da0 msgr2=0x7fb4440cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.297+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 shutdown_connections 2024-04-26T21:43:10.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.297+0000 7fb449b2b640 1 -- 172.21.15.40:0/1330753531 wait complete. 2024-04-26T21:43:10.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 129, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:43:06.779246+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7743': {'gid': 7743, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.40:6833/3311550037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3311550037}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3311550037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7743, 'qdb_cluster': [7743]} 2024-04-26T21:43:10.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:43:10.307 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:43:10.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.410+0000 7f54f23f7640 1 Processor -- start 2024-04-26T21:43:10.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.410+0000 7f54f23f7640 1 -- start start 2024-04-26T21:43:10.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.410+0000 7f54f23f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec162880 0x7f54ec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.410+0000 7f54f23f7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54ec0b5f20 0x7f54ec0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.410+0000 7f54f23f7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54ec0b7e50 con 0x7f54ec0d39f0 2024-04-26T21:43:10.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.410+0000 7f54f23f7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f54ec0d0580 con 0x7f54ec0b5f20 2024-04-26T21:43:10.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.410+0000 7f54f23f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f54ec0d1800 con 0x7f54ec162880 2024-04-26T21:43:10.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54eb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54ec0b5f20 0x7f54ec0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54f0994640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f54ec0d39f0 0x7f54ec0b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41958/0 (socket says 172.21.15.40:41958) 2024-04-26T21:43:10.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54eb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54ec0b5f20 0x7f54ec0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60250/0 (socket says 172.21.15.40:60250) 2024-04-26T21:43:10.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54f0994640 1 -- 172.21.15.40:0/876530602 learned_addr learned my addr 172.21.15.40:0/876530602 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:10.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54ebfff640 1 --2- 172.21.15.40:0/876530602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec162880 0x7f54ec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54eb7fe640 1 -- 172.21.15.40:0/876530602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec162880 msgr2=0x7f54ec0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/876530602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec162880 0x7f54ec0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54eb7fe640 1 -- 172.21.15.40:0/876530602 >> v1:172.21.15.40:6789/0 conn(0x7f54ec0d39f0 legacy=0x7f54ec0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.411+0000 7f54eb7fe640 1 -- 172.21.15.40:0/876530602 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec0d07d0 con 0x7f54ec0b5f20 2024-04-26T21:43:10.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/876530602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec0b5f20 0x7f54ec0d34b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f54e00097a0 tx=0x7f54e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aea5fa5d6b2474e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54eaffd640 1 -- 172.21.15.40:0/876530602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0037070 con 0x7f54ec0b5f20 2024-04-26T21:43:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54eaffd640 1 -- 172.21.15.40:0/876530602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e0032440 con 0x7f54ec0b5f20 2024-04-26T21:43:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54eaffd640 1 -- 172.21.15.40:0/876530602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0032810 con 0x7f54ec0b5f20 2024-04-26T21:43:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54f23f7640 1 -- 172.21.15.40:0/876530602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec0b5f20 msgr2=0x7f54ec0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54f23f7640 1 --2- 172.21.15.40:0/876530602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec0b5f20 0x7f54ec0d34b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f54e00097a0 tx=0x7f54e002a180 comp rx=0 tx=0).stop 2024-04-26T21:43:10.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54f23f7640 1 -- 172.21.15.40:0/876530602 shutdown_connections 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54f23f7640 1 --2- 172.21.15.40:0/876530602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec0b5f20 0x7f54ec0d34b0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54f23f7640 1 --2- 172.21.15.40:0/876530602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec162880 0x7f54ec0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54f23f7640 1 -- 172.21.15.40:0/876530602 >> 172.21.15.40:0/876530602 conn(0x7f54ec0d4f60 msgr2=0x7f54ec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54f23f7640 1 -- 172.21.15.40:0/876530602 shutdown_connections 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.412+0000 7f54f23f7640 1 -- 172.21.15.40:0/876530602 wait complete. 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.414+0000 7f54f23f7640 1 Processor -- start 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.414+0000 7f54f23f7640 1 -- start start 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f23f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec0b5f20 0x7f54ec0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f23f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54ec0d39f0 0x7f54ec15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f23f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec162880 0x7f54ec06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f23f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec162c60 con 0x7f54ec162880 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f23f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec0d0d90 con 0x7f54ec0d39f0 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f23f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f54ec162de0 con 0x7f54ec0b5f20 2024-04-26T21:43:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec162880 0x7f54ec06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec162880 0x7f54ec06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60258/0 (socket says 172.21.15.40:60258) 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 -- 172.21.15.40:0/1857800083 learned_addr learned my addr 172.21.15.40:0/1857800083 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54ebfff640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec0b5f20 0x7f54ec0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54ec0d39f0 0x7f54ec15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 -- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec0b5f20 msgr2=0x7f54ec0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec0b5f20 0x7f54ec0cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 -- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54ec0d39f0 msgr2=0x7f54ec15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54ec0d39f0 0x7f54ec15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 -- 172.21.15.40:0/1857800083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec15bc90 con 0x7f54ec162880 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f0994640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec162880 0x7f54ec06df20 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f54e400cbf0 tx=0x7f54e400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54e8ff9640 1 -- 172.21.15.40:0/1857800083 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4017070 con 0x7f54ec162880 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54e8ff9640 1 -- 172.21.15.40:0/1857800083 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e400ace0 con 0x7f54ec162880 2024-04-26T21:43:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f23f7640 1 -- 172.21.15.40:0/1857800083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ec15bf70 con 0x7f54ec162880 2024-04-26T21:43:10.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54e8ff9640 1 -- 172.21.15.40:0/1857800083 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4012600 con 0x7f54ec162880 2024-04-26T21:43:10.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.415+0000 7f54f23f7640 1 -- 172.21.15.40:0/1857800083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ec0cc040 con 0x7f54ec162880 2024-04-26T21:43:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.417+0000 7f54e8ff9640 1 -- 172.21.15.40:0/1857800083 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54e401a070 con 0x7f54ec162880 2024-04-26T21:43:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.417+0000 7f54e8ff9640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54d806c8f0 0x7f54d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.418+0000 7f54e8ff9640 1 -- 172.21.15.40:0/1857800083 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f54e408bae0 con 0x7f54ec162880 2024-04-26T21:43:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.418+0000 7f54ebfff640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54d806c8f0 0x7f54d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.418+0000 7f54ebfff640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54d806c8f0 0x7f54d806edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009b40 tx=0x7f54dc009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.418+0000 7f54f51ef740 1 Processor -- start 2024-04-26T21:43:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54f51ef740 1 -- start start 2024-04-26T21:43:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd0440 0x5594b6cd0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cedce0 0x5594b6cee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6ca6120 0x5594b6cec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54f51ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5594b6c0a740 con 0x5594b6cedce0 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54f51ef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594b6ba63a0 con 0x5594b6cd0440 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54f51ef740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5594b6c06700 con 0x5594b6ca6120 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cedce0 0x5594b6cee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cedce0 0x5594b6cee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60260/0 (socket says 172.21.15.40:60260) 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54eb7fe640 1 -- 172.21.15.40:0/3279056216 learned_addr learned my addr 172.21.15.40:0/3279056216 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.419+0000 7f54f0994640 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6ca6120 0x5594b6cec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54ebfff640 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd0440 0x5594b6cd0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54eb7fe640 1 -- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6ca6120 msgr2=0x5594b6cec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6ca6120 0x5594b6cec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54eb7fe640 1 -- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd0440 msgr2=0x5594b6cd0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd0440 0x5594b6cd0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54eb7fe640 1 -- 172.21.15.40:0/3279056216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b6bc2750 con 0x5594b6cedce0 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54ebfff640 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd0440 0x5594b6cd0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cedce0 0x5594b6cee230 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f54e00097a0 tx=0x7f54e002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4592b2c12de1bcc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54d1ffb640 1 -- 172.21.15.40:0/3279056216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0037040 con 0x5594b6cedce0 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54d1ffb640 1 -- 172.21.15.40:0/3279056216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e00327a0 con 0x5594b6cedce0 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54d1ffb640 1 -- 172.21.15.40:0/3279056216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0032b90 con 0x5594b6cedce0 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54f51ef740 1 -- 172.21.15.40:0/3279056216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cedce0 msgr2=0x5594b6cee230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.420+0000 7f54f51ef740 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cedce0 0x5594b6cee230 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f54e00097a0 tx=0x7f54e002c000 comp rx=0 tx=0).stop 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 -- 172.21.15.40:0/3279056216 shutdown_connections 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6ca6120 0x5594b6cec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cedce0 0x5594b6cee230 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 --2- 172.21.15.40:0/3279056216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd0440 0x5594b6cd0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 -- 172.21.15.40:0/3279056216 >> 172.21.15.40:0/3279056216 conn(0x5594b6ca2a50 msgr2=0x5594b6ceb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 -- 172.21.15.40:0/3279056216 shutdown_connections 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 -- 172.21.15.40:0/3279056216 wait complete. 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 -- start start 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.421+0000 7f54f51ef740 1 Processor -- start 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd0440 0x5594b6cbc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 0x5594b6cd2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd34b0 0x5594b6cd2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f51ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5594b6ca6ec0 con 0x5594b6cd34b0 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f51ef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594b6b91fe0 con 0x5594b6cedce0 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f51ef740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5594b6ca0980 con 0x5594b6cd0440 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f51ef740 10 client.0 _ll_register_callbacks cb 0x5594b6ba8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f0994640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd34b0 0x5594b6cd2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f0994640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd34b0 0x5594b6cd2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60270/0 (socket says 172.21.15.40:60270) 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f0994640 1 -- 172.21.15.40:0/161267067 learned_addr learned my addr 172.21.15.40:0/161267067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54ebfff640 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd0440 0x5594b6cbc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 0x5594b6cd2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f0994640 1 -- 172.21.15.40:0/161267067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd0440 msgr2=0x5594b6cbc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f0994640 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd0440 0x5594b6cbc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f0994640 1 -- 172.21.15.40:0/161267067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 msgr2=0x5594b6cd2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f0994640 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 0x5594b6cd2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.422+0000 7f54f0994640 1 -- 172.21.15.40:0/161267067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b691c2d0 con 0x5594b6cd34b0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f0994640 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd34b0 0x5594b6cd2110 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f54e40594e0 tx=0x7f54e4059610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb579e1ffe437 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54caffd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54caffd640 1 -- 172.21.15.40:0/161267067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4061df0 con 0x5594b6cd34b0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54caffd640 1 -- 172.21.15.40:0/161267067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f54e4004b60 con 0x5594b6cd34b0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 1 -- 172.21.15.40:0/161267067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e00327a0 con 0x5594b6cd34b0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54caffd640 1 -- 172.21.15.40:0/161267067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7f54e40589a0 con 0x5594b6cd34b0 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 10 client.8835 fetch_fsmap learned FSMap version 129 2024-04-26T21:43:10.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 1 -- 172.21.15.40:0/161267067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) -- 0x5594b69e66b0 con 0x5594b6cd34b0 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54caffd640 1 -- 172.21.15.40:0/161267067 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e00327a0 con 0x5594b6cd34b0 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54caffd640 1 -- 172.21.15.40:0/161267067 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 129) ==== 3494+0+0 (secure 0 0 0) 0x7f54e4058b40 con 0x5594b6cd34b0 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 10 client.8835 fetch_fsmap finished waiting for FSMap version 129 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 10 client.8835 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 20 client.8835 populate_metadata read hostname 'smithi040' 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] conn(0x5594b6cf0f80 0x5594b6cf1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 4 client.8835 mds_command: new command op to 7743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54f51ef740 1 -- 172.21.15.40:0/161267067 --> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594b6c18350 con 0x5594b6cf0f80 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.423+0000 7f54ebfff640 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] conn(0x5594b6cf0f80 0x5594b6cf1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.424+0000 7f54caffd640 10 client.8835 ms_handle_connect on v2:172.21.15.40:6832/3311550037 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.424+0000 7f54ebfff640 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] conn(0x5594b6cf0f80 0x5594b6cf1360 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d924778e6dcfa8e server_cookie=f7963867704d1484 in_seq=0 out_seq=0 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.424+0000 7f54caffd640 1 -- 172.21.15.40:0/161267067 <== mds.0 v2:172.21.15.40:6832/3311550037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594b6c18350 con 0x5594b6cf0f80 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.424+0000 7f54caffd640 10 client.8835 handle_command_reply: tid=0 2024-04-26T21:43:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.424+0000 7f54f51ef740 1 client.8835 shutdown 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 -- 172.21.15.40:0/161267067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd34b0 msgr2=0x5594b6cd2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd34b0 0x5594b6cd2110 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f54e40594e0 tx=0x7f54e4059610 comp rx=0 tx=0).stop 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 -- 172.21.15.40:0/161267067 shutdown_connections 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] conn(0x5594b6cf0f80 0x5594b6cf1360 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd34b0 0x5594b6cd2110 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 0x5594b6cd2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 --2- 172.21.15.40:0/161267067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd0440 0x5594b6cbc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54caffd640 0 client.8835 ms_handle_reset on v2:172.21.15.40:6832/3311550037 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 -- 172.21.15.40:0/161267067 >> 172.21.15.40:0/161267067 conn(0x5594b6ca6120 msgr2=0x5594b6cee7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 -- 172.21.15.40:0/161267067 shutdown_connections 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 1 -- 172.21.15.40:0/161267067 wait complete. 2024-04-26T21:43:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.425+0000 7f54f51ef740 20 client.8835 trim_cache size 0 max 16384 2024-04-26T21:43:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.443+0000 7f54f51ef740 1 Processor -- start 2024-04-26T21:43:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.443+0000 7f54f51ef740 1 -- start start 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6ceb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd34b0 0x5594b6ca59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cedce0 0x5594b6cd1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54f51ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec201270 con 0x5594b6cd0440 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54f51ef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec200600 con 0x5594b6cd34b0 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54f51ef740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f54ec200c50 con 0x5594b6cedce0 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6ceb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6ceb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60282/0 (socket says 172.21.15.40:60282) 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 -- 172.21.15.40:0/670899407 learned_addr learned my addr 172.21.15.40:0/670899407 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54f0994640 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cedce0 0x5594b6cd1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 -- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cedce0 msgr2=0x5594b6cd1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cedce0 0x5594b6cd1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 -- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd34b0 msgr2=0x5594b6ca59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd34b0 0x5594b6ca59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd34b0 0x5594b6ca59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 -- 172.21.15.40:0/670899407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b6cef1a0 con 0x5594b6cd0440 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54f0994640 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cedce0 0x5594b6cd1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd34b0 0x5594b6ca59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54ebfff640 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6ceb060 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f54dc011690 tx=0x7f54dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be32f60d0ea6d8b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54d17fa640 1 -- 172.21.15.40:0/670899407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc01e040 con 0x5594b6cd0440 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.444+0000 7f54d17fa640 1 -- 172.21.15.40:0/670899407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54dc01f040 con 0x5594b6cd0440 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54d17fa640 1 -- 172.21.15.40:0/670899407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc003810 con 0x5594b6cd0440 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 -- 172.21.15.40:0/670899407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 msgr2=0x5594b6ceb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6ceb060 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f54dc011690 tx=0x7f54dc014000 comp rx=0 tx=0).stop 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 -- 172.21.15.40:0/670899407 shutdown_connections 2024-04-26T21:43:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cedce0 0x5594b6cd1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cd34b0 0x5594b6ca59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 --2- 172.21.15.40:0/670899407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6ceb060 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 -- 172.21.15.40:0/670899407 >> 172.21.15.40:0/670899407 conn(0x5594b6ca6120 msgr2=0x5594b6c9bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 -- 172.21.15.40:0/670899407 shutdown_connections 2024-04-26T21:43:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.445+0000 7f54f51ef740 1 -- 172.21.15.40:0/670899407 wait complete. 2024-04-26T21:43:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.446+0000 7f54f51ef740 1 -- start start 2024-04-26T21:43:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.446+0000 7f54f51ef740 1 Processor -- start 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.446+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6d4c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.446+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd34b0 0x5594b6ccf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f51ef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 0x5594b6ccf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f51ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5594b6cd2770 con 0x5594b6cd0440 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f51ef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594b6cd19d0 con 0x5594b6cedce0 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f51ef740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5594b6c9f860 con 0x5594b6cd34b0 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f51ef740 10 client.0 _ll_register_callbacks cb 0x7f54e0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6d4c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6d4c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60290/0 (socket says 172.21.15.40:60290) 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 -- 172.21.15.40:0/590629275 learned_addr learned my addr 172.21.15.40:0/590629275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f0994640 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 0x5594b6ccf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd34b0 0x5594b6ccf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 -- 172.21.15.40:0/590629275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd34b0 msgr2=0x5594b6ccf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd34b0 0x5594b6ccf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 -- 172.21.15.40:0/590629275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 msgr2=0x5594b6ccf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 0x5594b6ccf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 -- 172.21.15.40:0/590629275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b6ce0c80 con 0x5594b6cd0440 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54ebfff640 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6d4c2a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f54dc011df0 tx=0x7f54dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a961b50df7a155 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54caffd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54caffd640 1 -- 172.21.15.40:0/590629275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc01e030 con 0x5594b6cd0440 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54caffd640 1 -- 172.21.15.40:0/590629275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f54dc014440 con 0x5594b6cd0440 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f51ef740 1 -- 172.21.15.40:0/590629275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54dc01f040 con 0x5594b6cd0440 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54caffd640 1 -- 172.21.15.40:0/590629275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7f54dc003e50 con 0x5594b6cd0440 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f51ef740 10 client.8841 fetch_fsmap learned FSMap version 129 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.447+0000 7f54f51ef740 1 -- 172.21.15.40:0/590629275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) -- 0x5594b6cecef0 con 0x5594b6cd0440 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54caffd640 1 -- 172.21.15.40:0/590629275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54dc01f040 con 0x5594b6cd0440 2024-04-26T21:43:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54caffd640 1 -- 172.21.15.40:0/590629275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 129) ==== 3494+0+0 (secure 0 0 0) 0x7f54dc042080 con 0x5594b6cd0440 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54f51ef740 10 client.8841 fetch_fsmap finished waiting for FSMap version 129 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54f51ef740 10 client.8841 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54f51ef740 20 client.8841 populate_metadata read hostname 'smithi040' 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54f51ef740 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] conn(0x5594b6cec640 0x5594b6cef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54f51ef740 4 client.8841 mds_command: new command op to 7743 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54f51ef740 1 -- 172.21.15.40:0/590629275 --> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] -- command(tid 0: {"prefix": "status"}) -- 0x5594b6c121c0 con 0x5594b6cec640 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] conn(0x5594b6cec640 0x5594b6cef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54caffd640 10 client.8841 ms_handle_connect on v2:172.21.15.40:6832/3311550037 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54eb7fe640 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] conn(0x5594b6cec640 0x5594b6cef390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76b67075ac715f1 server_cookie=e1638ac396cbe057 in_seq=0 out_seq=0 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54caffd640 1 -- 172.21.15.40:0/590629275 <== mds.0 v2:172.21.15.40:6832/3311550037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5594b6c121c0 con 0x5594b6cec640 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54caffd640 10 client.8841 handle_command_reply: tid=0 2024-04-26T21:43:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.448+0000 7f54f51ef740 1 client.8841 shutdown 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 -- 172.21.15.40:0/590629275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 msgr2=0x5594b6d4c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6d4c2a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f54dc011df0 tx=0x7f54dc004730 comp rx=0 tx=0).stop 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 -- 172.21.15.40:0/590629275 shutdown_connections 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.40:6832/3311550037,v1:172.21.15.40:6833/3311550037] conn(0x5594b6cec640 0x5594b6cef390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594b6cedce0 0x5594b6ccf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594b6cd34b0 0x5594b6ccf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 --2- 172.21.15.40:0/590629275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5594b6cd0440 0x5594b6d4c2a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54caffd640 0 client.8841 ms_handle_reset on v2:172.21.15.40:6832/3311550037 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 -- 172.21.15.40:0/590629275 >> 172.21.15.40:0/590629275 conn(0x5594b6ca6120 msgr2=0x5594b6cd1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 -- 172.21.15.40:0/590629275 shutdown_connections 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 1 -- 172.21.15.40:0/590629275 wait complete. 2024-04-26T21:43:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f51ef740 20 client.8841 trim_cache size 0 max 16384 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7743, 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.6834553860000003, 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 129, 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 101, 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 101, 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 72.242248672000002 2024-04-26T21:43:10.456 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:43:10.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 -- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54d806c8f0 msgr2=0x7f54d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54d806c8f0 0x7f54d806edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009b40 tx=0x7f54dc009190 comp rx=0 tx=0).stop 2024-04-26T21:43:10.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 -- 172.21.15.40:0/1857800083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec162880 msgr2=0x7f54ec06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec162880 0x7f54ec06df20 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f54e400cbf0 tx=0x7f54e400a430 comp rx=0 tx=0).stop 2024-04-26T21:43:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 -- 172.21.15.40:0/1857800083 shutdown_connections 2024-04-26T21:43:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54d806c8f0 0x7f54d806edb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54ec162880 0x7f54ec06df20 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54ec0d39f0 0x7f54ec15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 --2- 172.21.15.40:0/1857800083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54ec0b5f20 0x7f54ec0cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 -- 172.21.15.40:0/1857800083 >> 172.21.15.40:0/1857800083 conn(0x7f54ec0d4f60 msgr2=0x7f54ec0ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 -- 172.21.15.40:0/1857800083 shutdown_connections 2024-04-26T21:43:10.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.449+0000 7f54f23f7640 1 -- 172.21.15.40:0/1857800083 wait complete. 2024-04-26T21:43:10.460 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:43:10.460 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete mds0_openfiles.0', proceeding to ls 2024-04-26T21:43:10.460 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:43:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.560+0000 7fdf6d085640 1 Processor -- start 2024-04-26T21:43:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.561+0000 7fdf6d085640 1 -- start start 2024-04-26T21:43:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.561+0000 7fdf6d085640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf68162880 0x7fdf680d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.561+0000 7fdf6d085640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf680d3bc0 0x7fdf680b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.561+0000 7fdf6d085640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf680b7e50 con 0x7fdf680b5f20 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.561+0000 7fdf6d085640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdf680d0580 con 0x7fdf680d3bc0 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.561+0000 7fdf6d085640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf680d1800 con 0x7fdf68162880 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.561+0000 7fdf6759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf680d3bc0 0x7fdf680b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.562+0000 7fdf6759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf680d3bc0 0x7fdf680b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60292/0 (socket says 172.21.15.40:60292) 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.562+0000 7fdf6759e640 1 -- 172.21.15.40:0/1392376804 learned_addr learned my addr 172.21.15.40:0/1392376804 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.562+0000 7fdf66d9d640 1 --2- 172.21.15.40:0/1392376804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf68162880 0x7fdf680d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.562+0000 7fdf6759e640 1 -- 172.21.15.40:0/1392376804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf68162880 msgr2=0x7fdf680d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.562+0000 7fdf6759e640 1 --2- 172.21.15.40:0/1392376804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf68162880 0x7fdf680d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.562+0000 7fdf6759e640 1 -- 172.21.15.40:0/1392376804 >> v1:172.21.15.184:6789/0 conn(0x7fdf680b5f20 legacy=0x7fdf680d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.562+0000 7fdf6759e640 1 -- 172.21.15.40:0/1392376804 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf680d07d0 con 0x7fdf680d3bc0 2024-04-26T21:43:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.562+0000 7fdf6759e640 1 --2- 172.21.15.40:0/1392376804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf680b8300 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009e50 tx=0x7fdf5802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbe45d736c14d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf65d9b640 1 -- 172.21.15.40:0/1392376804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58036070 con 0x7fdf680d3bc0 2024-04-26T21:43:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf65d9b640 1 -- 172.21.15.40:0/1392376804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf58031600 con 0x7fdf680d3bc0 2024-04-26T21:43:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf65d9b640 1 -- 172.21.15.40:0/1392376804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580319d0 con 0x7fdf680d3bc0 2024-04-26T21:43:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf6d085640 1 -- 172.21.15.40:0/1392376804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 msgr2=0x7fdf680b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf6d085640 1 --2- 172.21.15.40:0/1392376804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf680b8300 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009e50 tx=0x7fdf5802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf6d085640 1 -- 172.21.15.40:0/1392376804 shutdown_connections 2024-04-26T21:43:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf6d085640 1 --2- 172.21.15.40:0/1392376804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf680b8300 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf6d085640 1 --2- 172.21.15.40:0/1392376804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf68162880 0x7fdf680d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf6d085640 1 -- 172.21.15.40:0/1392376804 >> 172.21.15.40:0/1392376804 conn(0x7fdf680d4f60 msgr2=0x7fdf680d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf6d085640 1 -- 172.21.15.40:0/1392376804 shutdown_connections 2024-04-26T21:43:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.563+0000 7fdf6d085640 1 -- 172.21.15.40:0/1392376804 wait complete. 2024-04-26T21:43:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.565+0000 7fdf6d085640 1 Processor -- start 2024-04-26T21:43:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.565+0000 7fdf6d085640 1 -- start start 2024-04-26T21:43:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.565+0000 7fdf6d085640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf680b5f20 0x7fdf681f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.565+0000 7fdf6d085640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf681f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.565+0000 7fdf6d085640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf68162880 0x7fdf6806e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.565+0000 7fdf6d085640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68162c60 con 0x7fdf680d3bc0 2024-04-26T21:43:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.565+0000 7fdf6d085640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf680d0d90 con 0x7fdf68162880 2024-04-26T21:43:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.565+0000 7fdf6d085640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf68162de0 con 0x7fdf680b5f20 2024-04-26T21:43:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf681f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf66d9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf680b5f20 0x7fdf681f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf681f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60302/0 (socket says 172.21.15.40:60302) 2024-04-26T21:43:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 -- 172.21.15.40:0/101484520 learned_addr learned my addr 172.21.15.40:0/101484520 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6759e640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf68162880 0x7fdf6806e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 -- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf680b5f20 msgr2=0x7fdf681f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf680b5f20 0x7fdf681f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 -- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf68162880 msgr2=0x7fdf6806e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf68162880 0x7fdf6806e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 -- 172.21.15.40:0/101484520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf680ba660 con 0x7fdf680d3bc0 2024-04-26T21:43:10.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6659c640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf681f4700 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000c430 tx=0x7fdf5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf5f7fe640 1 -- 172.21.15.40:0/101484520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50011790 con 0x7fdf680d3bc0 2024-04-26T21:43:10.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf5f7fe640 1 -- 172.21.15.40:0/101484520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf50017070 con 0x7fdf680d3bc0 2024-04-26T21:43:10.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf681f4e70 con 0x7fdf680d3bc0 2024-04-26T21:43:10.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf5f7fe640 1 -- 172.21.15.40:0/101484520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50011b70 con 0x7fdf680d3bc0 2024-04-26T21:43:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.566+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf681f5430 con 0x7fdf680d3bc0 2024-04-26T21:43:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.567+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf681f5630 con 0x7fdf680d3bc0 2024-04-26T21:43:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.568+0000 7fdf5f7fe640 1 -- 172.21.15.40:0/101484520 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf50011d10 con 0x7fdf680d3bc0 2024-04-26T21:43:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.568+0000 7fdf5f7fe640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf4806c8f0 0x7fdf4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.568+0000 7fdf5f7fe640 1 -- 172.21.15.40:0/101484520 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7fdf5008bd10 con 0x7fdf680d3bc0 2024-04-26T21:43:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.569+0000 7fdf66d9d640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf4806c8f0 0x7fdf4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.569+0000 7fdf66d9d640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf4806c8f0 0x7fdf4806edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdf600079a0 tx=0x7fdf60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.573+0000 7fdf5f7fe640 1 -- 172.21.15.40:0/101484520 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf50091020 con 0x7fdf680d3bc0 2024-04-26T21:43:10.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.799+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdf680694f0 con 0x7fdf680d3bc0 2024-04-26T21:43:10.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.800+0000 7fdf5f7fe640 1 -- 172.21.15.40:0/101484520 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v129) ==== 53+0+73 (secure 0 0 0) 0x7fdf50059660 con 0x7fdf680d3bc0 2024-04-26T21:43:10.800 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:43:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf4806c8f0 msgr2=0x7fdf4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf4806c8f0 0x7fdf4806edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdf600079a0 tx=0x7fdf60008040 comp rx=0 tx=0).stop 2024-04-26T21:43:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 msgr2=0x7fdf681f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf681f4700 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000c430 tx=0x7fdf5000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 shutdown_connections 2024-04-26T21:43:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf4806c8f0 0x7fdf4806edb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf68162880 0x7fdf6806e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680d3bc0 0x7fdf681f4700 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 --2- 172.21.15.40:0/101484520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf680b5f20 0x7fdf681f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 >> 172.21.15.40:0/101484520 conn(0x7fdf680d4f60 msgr2=0x7fdf680d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 shutdown_connections 2024-04-26T21:43:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:10.802+0000 7fdf6d085640 1 -- 172.21.15.40:0/101484520 wait complete. 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:43:10.813 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:10.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:10.814 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:43:10.829 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:43:10.829 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42257sec preferred_lft 42257sec 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:10.831 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:43:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:43:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:43:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:10.832 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:43:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:43:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:43:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:10.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:10.832 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:43:10.892 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:43:10.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:10.894 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:43:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:43:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:43:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:43:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:43:10.951 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:43:10.951 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:43:10.951 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:43:10.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:43:10.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:43:10.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:43:10.951 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:43:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:43:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:43:11.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:11.076 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:43:11.092 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:43:11.092 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:43:11.092 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:11.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:11.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:43:11.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:43:11.156 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:11.211 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:11.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:43:11.211 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:11.265 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:43:11.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:11.265 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:11.348 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:11.348 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:43:11.356 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:11.356 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:43:11.356 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:43:11.356 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:43:11.356 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:43:11.356 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:43:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:43:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:43:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:43:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:43:11.358 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:43:11.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:11.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:11.423 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:43:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:43:11.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:43:11.520 INFO:teuthology.orchestra.run.smithi040.stdout:client8850 2024-04-26T21:43:11.520 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:11.521 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:11.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:11.536 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-26T21:43:11.579 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:43:11.593 INFO:teuthology.orchestra.run.smithi040.stdout:.: 2024-04-26T21:43:11.594 INFO:teuthology.orchestra.run.smithi040.stdout:subdir 2024-04-26T21:43:11.594 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:11.594 INFO:teuthology.orchestra.run.smithi040.stdout:./subdir: 2024-04-26T21:43:11.594 INFO:teuthology.orchestra.run.smithi040.stdout:sixmegs 2024-04-26T21:43:11.594 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_openfiles.0' did not caused DAMAGED state 2024-04-26T21:43:11.594 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 500.00000000' 2024-04-26T21:43:11.594 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:11.594 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:11.649 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:11.649 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:11.705 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:43:11.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.846+0000 7f80eb699640 1 Processor -- start 2024-04-26T21:43:11.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.846+0000 7f80eb699640 1 -- start start 2024-04-26T21:43:11.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80eb699640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f80e4162810 0x7f80e4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80eb699640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40b2340 0x7f80e40cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80eb699640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e40d10d0 con 0x7f80e40cd930 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80eb699640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f80e40d0750 con 0x7f80e4162810 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80eb699640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80e40d16a0 con 0x7f80e40b2340 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80e9c36640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f80e40cd930 0x7f80e40d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41964/0 (socket says 172.21.15.40:41964) 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80e9c36640 1 -- 172.21.15.40:0/1301932636 learned_addr learned my addr 172.21.15.40:0/1301932636 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80e8c34640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40b2340 0x7f80e40cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.847+0000 7f80e9435640 1 --2- 172.21.15.40:0/1301932636 >> v2:172.21.15.184:3300/0 conn(0x7f80e4162810 0x7f80e4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1107244803 0 0) 0x7f80e40d10d0 con 0x7f80e40cd930 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80d0003520 con 0x7f80e40cd930 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1284274624 0 0) 0x7f80d0003520 con 0x7f80e40cd930 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40b2340 msgr2=0x7f80e40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40b2340 0x7f80e40cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 >> v2:172.21.15.184:3300/0 conn(0x7f80e4162810 msgr2=0x7f80e4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 --2- 172.21.15.40:0/1301932636 >> v2:172.21.15.184:3300/0 conn(0x7f80e4162810 0x7f80e4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e40d1370 con 0x7f80e40cd930 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f80e00019f0 con 0x7f80e40cd930 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 >> v1:172.21.15.40:6789/0 conn(0x7f80e40cd930 legacy=0x7f80e40d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80d00039b0 0x7f80d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40b2340 0x7f80d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d0004d40 0x7f80d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40d0750 con 0x7f80d00039b0 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40d16a0 con 0x7f80d0004d40 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80d000b9c0 con 0x7f80e40b2340 2024-04-26T21:43:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80e9435640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80d00039b0 0x7f80d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80e8c34640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40b2340 0x7f80d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80e9c36640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d0004d40 0x7f80d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80e9435640 1 -- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40b2340 msgr2=0x7f80d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80e9435640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40b2340 0x7f80d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80e9435640 1 -- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d0004d40 msgr2=0x7f80d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80e9435640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d0004d40 0x7f80d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.848+0000 7f80e9435640 1 -- 172.21.15.40:0/1301932636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d0003520 con 0x7f80d00039b0 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.849+0000 7f80e9435640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80d00039b0 0x7f80d0003dd0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009ac0 tx=0x7f80d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd2806caa92b1de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.849+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc036070 con 0x7f80d00039b0 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.849+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80dc031400 con 0x7f80d00039b0 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.849+0000 7f80dbfff640 1 -- 172.21.15.40:0/1301932636 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc009690 con 0x7f80d00039b0 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.850+0000 7f80eb699640 1 -- 172.21.15.40:0/1301932636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80d00039b0 msgr2=0x7f80d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.850+0000 7f80eb699640 1 --2- 172.21.15.40:0/1301932636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80d00039b0 0x7f80d0003dd0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009ac0 tx=0x7f80d0005830 comp rx=0 tx=0).stop 2024-04-26T21:43:11.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.850+0000 7f80e9c36640 1 -- 172.21.15.40:0/1301932636 reap_dead start 2024-04-26T21:43:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.850+0000 7f80eb699640 1 -- 172.21.15.40:0/1301932636 shutdown_connections 2024-04-26T21:43:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.850+0000 7f80eb699640 1 -- 172.21.15.40:0/1301932636 >> 172.21.15.40:0/1301932636 conn(0x7f80e40d2da0 msgr2=0x7f80e40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.850+0000 7f80eb699640 1 -- 172.21.15.40:0/1301932636 shutdown_connections 2024-04-26T21:43:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.850+0000 7f80eb699640 1 -- 172.21.15.40:0/1301932636 wait complete. 2024-04-26T21:43:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.851+0000 7f80eb699640 1 Processor -- start 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.851+0000 7f80eb699640 1 -- start start 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80eb699640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d00039b0 0x7f80e414cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80eb699640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80e415aa60 0x7f80e40c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80eb699640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40c83a0 0x7f80e406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80eb699640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4070890 con 0x7f80e415aa60 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80eb699640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40d0470 con 0x7f80d00039b0 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80eb699640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80e40d2240 con 0x7f80e40c83a0 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80e415aa60 0x7f80e40c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80e415aa60 0x7f80e40c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60310/0 (socket says 172.21.15.40:60310) 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 -- 172.21.15.40:0/2459735488 learned_addr learned my addr 172.21.15.40:0/2459735488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e9c36640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40c83a0 0x7f80e406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e9435640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d00039b0 0x7f80e414cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 -- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40c83a0 msgr2=0x7f80e406e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40c83a0 0x7f80e406e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 -- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d00039b0 msgr2=0x7f80e414cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d00039b0 0x7f80e414cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 -- 172.21.15.40:0/2459735488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e415afd0 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e9435640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d00039b0 0x7f80e414cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80e8c34640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80e415aa60 0x7f80e40c68b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f80d40119d0 tx=0x7f80d4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80d9ffb640 1 -- 172.21.15.40:0/2459735488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d400faf0 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80d9ffb640 1 -- 172.21.15.40:0/2459735488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d400fc90 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80d9ffb640 1 -- 172.21.15.40:0/2459735488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4014440 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.852+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e415b2b0 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.853+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e415b870 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.854+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e40694f0 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.854+0000 7f80d9ffb640 1 -- 172.21.15.40:0/2459735488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80d4012040 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.855+0000 7f80d9ffb640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80c806c8f0 0x7f80c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.855+0000 7f80d9ffb640 1 -- 172.21.15.40:0/2459735488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f80d408c600 con 0x7f80e415aa60 2024-04-26T21:43:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.855+0000 7f80e9435640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80c806c8f0 0x7f80c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.855+0000 7f80e9435640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80c806c8f0 0x7f80c806edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f80dc033000 tx=0x7f80dc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:11.859+0000 7f80d9ffb640 1 -- 172.21.15.40:0/2459735488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80d4019030 con 0x7f80e415aa60 2024-04-26T21:43:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.044+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80e4157df0 con 0x7f80e415aa60 2024-04-26T21:43:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.045+0000 7f80d9ffb640 1 -- 172.21.15.40:0/2459735488 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+17824 (secure 0 0 0) 0x7f80d405a620 con 0x7f80e415aa60 2024-04-26T21:43:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:12.046 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":101,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:42:59.734257+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80c806c8f0 msgr2=0x7f80c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80c806c8f0 0x7f80c806edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f80dc033000 tx=0x7f80dc008030 comp rx=0 tx=0).stop 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80e415aa60 msgr2=0x7f80e40c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80e415aa60 0x7f80e40c68b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f80d40119d0 tx=0x7f80d4011e90 comp rx=0 tx=0).stop 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 shutdown_connections 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f80c806c8f0 0x7f80c806edb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80e40c83a0 0x7f80e406e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80e415aa60 0x7f80e40c68b0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 --2- 172.21.15.40:0/2459735488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d00039b0 0x7f80e414cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.047+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 >> 172.21.15.40:0/2459735488 conn(0x7f80e40d2da0 msgr2=0x7f80e40b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.048+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 shutdown_connections 2024-04-26T21:43:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.048+0000 7f80eb699640 1 -- 172.21.15.40:0/2459735488 wait complete. 2024-04-26T21:43:12.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:43:12.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:43:12.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:43:12.104 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:43:12.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.163+0000 7f1b487f0640 1 Processor -- start 2024-04-26T21:43:12.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.164+0000 7f1b487f0640 1 -- start start 2024-04-26T21:43:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.164+0000 7f1b487f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1b400d1a00 0x7f1b400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.164+0000 7f1b487f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b400d2320 0x7f1b401625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.164+0000 7f1b487f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b400d0a10 con 0x7f1b400b40d0 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.164+0000 7f1b487f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1b400d0cf0 con 0x7f1b400d1a00 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.164+0000 7f1b487f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b400d0fd0 con 0x7f1b400d2320 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b46d8d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1b400b40d0 0x7f1b400d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41974/0 (socket says 172.21.15.40:41974) 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b46d8d640 1 -- 172.21.15.40:0/271569938 learned_addr learned my addr 172.21.15.40:0/271569938 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4658c640 1 --2- 172.21.15.40:0/271569938 >> v2:172.21.15.184:3300/0 conn(0x7f1b400d1a00 0x7f1b400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b45d8b640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b400d2320 0x7f1b401625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2481383327 0 0) 0x7f1b400d0a10 con 0x7f1b400b40d0 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b2c003520 con 0x7f1b400b40d0 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2329198119 0 0) 0x7f1b2c003520 con 0x7f1b400b40d0 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b400d2320 msgr2=0x7f1b401625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b400d2320 0x7f1b401625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 >> v2:172.21.15.184:3300/0 conn(0x7f1b400d1a00 msgr2=0x7f1b400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 --2- 172.21.15.40:0/271569938 >> v2:172.21.15.184:3300/0 conn(0x7f1b400d1a00 0x7f1b400d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b400d40a0 con 0x7f1b400b40d0 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1b380019f0 con 0x7f1b400b40d0 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 >> v1:172.21.15.40:6789/0 conn(0x7f1b400b40d0 legacy=0x7f1b400d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b2c0039b0 0x7f1b2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c004800 0x7f1b2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b400d2320 0x7f1b2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1b400d0cf0 con 0x7f1b2c004800 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b400d0fd0 con 0x7f1b2c0039b0 2024-04-26T21:43:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c00b9f0 con 0x7f1b400d2320 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b45d8b640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c004800 0x7f1b2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.165+0000 7f1b4658c640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b2c0039b0 0x7f1b2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.166+0000 7f1b46d8d640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b400d2320 0x7f1b2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.166+0000 7f1b45d8b640 1 -- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b400d2320 msgr2=0x7f1b2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.166+0000 7f1b45d8b640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b400d2320 0x7f1b2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.166+0000 7f1b45d8b640 1 -- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b2c0039b0 msgr2=0x7f1b2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.166+0000 7f1b45d8b640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b2c0039b0 0x7f1b2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.166+0000 7f1b45d8b640 1 -- 172.21.15.40:0/271569938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c003520 con 0x7f1b2c004800 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.166+0000 7f1b45d8b640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c004800 0x7f1b2c004c80 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009ac0 tx=0x7f1b2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864abeaa912eb214 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30036070 con 0x7f1b2c004800 2024-04-26T21:43:12.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b30031400 con 0x7f1b2c004800 2024-04-26T21:43:12.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b4558a640 1 -- 172.21.15.40:0/271569938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30009690 con 0x7f1b2c004800 2024-04-26T21:43:12.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b487f0640 1 -- 172.21.15.40:0/271569938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c004800 msgr2=0x7f1b2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b487f0640 1 --2- 172.21.15.40:0/271569938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c004800 0x7f1b2c004c80 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009ac0 tx=0x7f1b2c005830 comp rx=0 tx=0).stop 2024-04-26T21:43:12.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b46d8d640 1 -- 172.21.15.40:0/271569938 reap_dead start 2024-04-26T21:43:12.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b487f0640 1 -- 172.21.15.40:0/271569938 shutdown_connections 2024-04-26T21:43:12.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b487f0640 1 -- 172.21.15.40:0/271569938 >> 172.21.15.40:0/271569938 conn(0x7f1b400d4fd0 msgr2=0x7f1b400d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:12.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b487f0640 1 -- 172.21.15.40:0/271569938 shutdown_connections 2024-04-26T21:43:12.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.167+0000 7f1b487f0640 1 -- 172.21.15.40:0/271569938 wait complete. 2024-04-26T21:43:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.168+0000 7f1b487f0640 1 Processor -- start 2024-04-26T21:43:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.168+0000 7f1b487f0640 1 -- start start 2024-04-26T21:43:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b487f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b2c004800 0x7f1b4015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b487f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b400cd420 0x7f1b400ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b487f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b4015f300 0x7f1b4015f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b487f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1b400d35e0 con 0x7f1b400cd420 2024-04-26T21:43:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b487f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b400b5d20 con 0x7f1b4015f300 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b487f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b400d3760 con 0x7f1b2c004800 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b45d8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b400cd420 0x7f1b400ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b45d8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b400cd420 0x7f1b400ba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60334/0 (socket says 172.21.15.40:60334) 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b45d8b640 1 -- 172.21.15.40:0/1517154226 learned_addr learned my addr 172.21.15.40:0/1517154226 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b4658c640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b2c004800 0x7f1b4015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.169+0000 7f1b46d8d640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b4015f300 0x7f1b4015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b45d8b640 1 -- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b2c004800 msgr2=0x7f1b4015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b45d8b640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b2c004800 0x7f1b4015bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b45d8b640 1 -- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b4015f300 msgr2=0x7f1b4015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b45d8b640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b4015f300 0x7f1b4015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b45d8b640 1 -- 172.21.15.40:0/1517154226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b400cc0a0 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b45d8b640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b400cd420 0x7f1b400ba6a0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009be0 tx=0x7f1b30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b36ffd640 1 -- 172.21.15.40:0/1517154226 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30041020 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b36ffd640 1 -- 172.21.15.40:0/1517154226 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3003c470 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b400cc380 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b36ffd640 1 -- 172.21.15.40:0/1517154226 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30036040 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.170+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b400cc940 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.171+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b400694f0 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.171+0000 7f1b36ffd640 1 -- 172.21.15.40:0/1517154226 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b30044020 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.172+0000 7f1b36ffd640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.172+0000 7f1b36ffd640 1 -- 172.21.15.40:0/1517154226 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f1b300abc50 con 0x7f1b400cd420 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.172+0000 7f1b4658c640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.172+0000 7f1b4658c640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b2406c920 0x7f1b2406ede0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ba630 tx=0x7f1b3c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.176+0000 7f1b36ffd640 1 -- 172.21.15.40:0/1517154226 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b300b1020 con 0x7f1b400cd420 2024-04-26T21:43:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.361+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b400d58a0 con 0x7f1b400cd420 2024-04-26T21:43:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.362+0000 7f1b36ffd640 1 -- 172.21.15.40:0/1517154226 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f1b30079d40 con 0x7f1b400cd420 2024-04-26T21:43:12.362 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:43:12.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b2406c920 msgr2=0x7f1b2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b2406c920 0x7f1b2406ede0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ba630 tx=0x7f1b3c009190 comp rx=0 tx=0).stop 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b400cd420 msgr2=0x7f1b400ba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b400cd420 0x7f1b400ba6a0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009be0 tx=0x7f1b30004730 comp rx=0 tx=0).stop 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 shutdown_connections 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b4015f300 0x7f1b4015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b400cd420 0x7f1b400ba6a0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 --2- 172.21.15.40:0/1517154226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b2c004800 0x7f1b4015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.363+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 >> 172.21.15.40:0/1517154226 conn(0x7f1b400d4fd0 msgr2=0x7f1b4006e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.364+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 shutdown_connections 2024-04-26T21:43:12.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.364+0000 7f1b487f0640 1 -- 172.21.15.40:0/1517154226 wait complete. 2024-04-26T21:43:12.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:43:12.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.519+0000 7f0e9da62640 1 Processor -- start 2024-04-26T21:43:12.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.519+0000 7f0e9da62640 1 -- start start 2024-04-26T21:43:12.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e9da62640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9815f2d0 0x7f0e981604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e9da62640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e98072f70 con 0x7f0e9806fea0 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e9da62640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e98073250 con 0x7f0e98072700 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e9da62640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e98074010 con 0x7f0e9815f2d0 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e97fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0e9806fea0 0x7f0e980ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41980/0 (socket says 172.21.15.40:41980) 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e97fff640 1 -- 172.21.15.40:0/2131336030 learned_addr learned my addr 172.21.15.40:0/2131336030 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e977fe640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9815f2d0 0x7f0e981604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136306834 0 0) 0x7f0e98072f70 con 0x7f0e9806fea0 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e80003520 con 0x7f0e9806fea0 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3211647104 0 0) 0x7f0e80003520 con 0x7f0e9806fea0 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9815f2d0 msgr2=0x7f0e981604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9815f2d0 0x7f0e981604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 >> v1:172.21.15.184:6789/0 conn(0x7f0e98072700 legacy=0x7f0e9806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e98073530 con 0x7f0e9806fea0 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f0e900019f0 con 0x7f0e9806fea0 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.520+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 >> v1:172.21.15.40:6789/0 conn(0x7f0e9806fea0 legacy=0x7f0e980ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e967fc640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9815f2d0 0x7f0e800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e967fc640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e80003f60 0x7f0e80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e967fc640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e80004d40 0x7f0e800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98074010 con 0x7f0e80004d40 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8000baa0 con 0x7f0e80003f60 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8000bca0 con 0x7f0e9815f2d0 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e97fff640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e80004d40 0x7f0e800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e977fe640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9815f2d0 0x7f0e800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e96ffd640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e80003f60 0x7f0e80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e97fff640 1 -- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9815f2d0 msgr2=0x7f0e800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e97fff640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9815f2d0 0x7f0e800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e97fff640 1 -- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e80003f60 msgr2=0x7f0e80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e97fff640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e80003f60 0x7f0e80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e97fff640 1 -- 172.21.15.40:0/2131336030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e80003520 con 0x7f0e80004d40 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.521+0000 7f0e96ffd640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e80003f60 0x7f0e80004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e97fff640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e80004d40 0x7f0e800051c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000a860 tx=0x7f0e80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73f767051f8940 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9000a480 con 0x7f0e80004d40 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9002f070 con 0x7f0e80004d40 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e967fc640 1 -- 172.21.15.40:0/2131336030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9002bd40 con 0x7f0e80004d40 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e9da62640 1 -- 172.21.15.40:0/2131336030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e80004d40 msgr2=0x7f0e800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e9da62640 1 --2- 172.21.15.40:0/2131336030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e80004d40 0x7f0e800051c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000a860 tx=0x7f0e80005830 comp rx=0 tx=0).stop 2024-04-26T21:43:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e97fff640 1 -- 172.21.15.40:0/2131336030 reap_dead start 2024-04-26T21:43:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e9da62640 1 -- 172.21.15.40:0/2131336030 shutdown_connections 2024-04-26T21:43:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e9da62640 1 -- 172.21.15.40:0/2131336030 >> 172.21.15.40:0/2131336030 conn(0x7f0e980d0be0 msgr2=0x7f0e980d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e9da62640 1 -- 172.21.15.40:0/2131336030 shutdown_connections 2024-04-26T21:43:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.522+0000 7f0e9da62640 1 -- 172.21.15.40:0/2131336030 wait complete. 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.524+0000 7f0e9da62640 1 Processor -- start 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.524+0000 7f0e9da62640 1 -- start start 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.524+0000 7f0e9da62640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9814d0b0 0x7f0e980ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.524+0000 7f0e9da62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980c4ec0 0x7f0e980c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.524+0000 7f0e9da62640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e980c6a90 0x7f0e981f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e9da62640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9815f6b0 con 0x7f0e980c4ec0 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e9da62640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98073af0 con 0x7f0e980c6a90 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e9da62640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e98072b60 con 0x7f0e9814d0b0 2024-04-26T21:43:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980c4ec0 0x7f0e980c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980c4ec0 0x7f0e980c6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60346/0 (socket says 172.21.15.40:60346) 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 -- 172.21.15.40:0/2665840894 learned_addr learned my addr 172.21.15.40:0/2665840894 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e977fe640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9814d0b0 0x7f0e980ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e97fff640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e980c6a90 0x7f0e981f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 -- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9814d0b0 msgr2=0x7f0e980ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9814d0b0 0x7f0e980ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 -- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e980c6a90 msgr2=0x7f0e981f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e980c6a90 0x7f0e981f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 -- 172.21.15.40:0/2665840894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e980c7cb0 con 0x7f0e980c4ec0 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e96ffd640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980c4ec0 0x7f0e980c6550 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00e5b0 tx=0x7f0e8c00e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e7ffff640 1 -- 172.21.15.40:0/2665840894 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c019070 con 0x7f0e980c4ec0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e7ffff640 1 -- 172.21.15.40:0/2665840894 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e8c0052e0 con 0x7f0e980c4ec0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e980c53f0 con 0x7f0e980c4ec0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e7ffff640 1 -- 172.21.15.40:0/2665840894 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c003830 con 0x7f0e980c4ec0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.525+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e980c59b0 con 0x7f0e980c4ec0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.526+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e980c7e80 con 0x7f0e980c4ec0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.527+0000 7f0e7ffff640 1 -- 172.21.15.40:0/2665840894 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e8c01c070 con 0x7f0e980c4ec0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.527+0000 7f0e7ffff640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e8406c8f0 0x7f0e8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.527+0000 7f0e7ffff640 1 -- 172.21.15.40:0/2665840894 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f0e8c08b370 con 0x7f0e980c4ec0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.528+0000 7f0e977fe640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e8406c8f0 0x7f0e8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.528+0000 7f0e977fe640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e8406c8f0 0x7f0e8406edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009bc0 tx=0x7f0e88009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.532+0000 7f0e7ffff640 1 -- 172.21.15.40:0/2665840894 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e8c058d50 con 0x7f0e980c4ec0 2024-04-26T21:43:12.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.718+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0e980c6e70 con 0x7f0e980c4ec0 2024-04-26T21:43:12.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.718+0000 7f0e7ffff640 1 -- 172.21.15.40:0/2665840894 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f0e8c05d050 con 0x7f0e980c4ec0 2024-04-26T21:43:12.718 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:43:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e8406c8f0 msgr2=0x7f0e8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e8406c8f0 0x7f0e8406edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009bc0 tx=0x7f0e88009190 comp rx=0 tx=0).stop 2024-04-26T21:43:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980c4ec0 msgr2=0x7f0e980c6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980c4ec0 0x7f0e980c6550 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00e5b0 tx=0x7f0e8c00e9b0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 shutdown_connections 2024-04-26T21:43:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0e8406c8f0 0x7f0e8406edb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e980c6a90 0x7f0e981f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980c4ec0 0x7f0e980c6550 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 --2- 172.21.15.40:0/2665840894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e9814d0b0 0x7f0e980ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 >> 172.21.15.40:0/2665840894 conn(0x7f0e980d0be0 msgr2=0x7f0e980b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 shutdown_connections 2024-04-26T21:43:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.720+0000 7f0e9da62640 1 -- 172.21.15.40:0/2665840894 wait complete. 2024-04-26T21:43:12.731 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:43:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.835+0000 7fc2589e2640 1 Processor -- start 2024-04-26T21:43:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.835+0000 7fc2589e2640 1 -- start start 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc2589e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc254070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc2589e2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc254071090 0x7fc25415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc2589e2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2540d2390 con 0x7fc2540cfde0 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc2589e2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc254071500 con 0x7fc254071090 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc2589e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2540afab0 con 0x7fc254070350 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc25259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc2540cfde0 0x7fc2540d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41986/0 (socket says 172.21.15.40:41986) 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc25259c640 1 -- 172.21.15.40:0/1839401679 learned_addr learned my addr 172.21.15.40:0/1839401679 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc252d9d640 1 --2- 172.21.15.40:0/1839401679 >> v2:172.21.15.40:3300/0 conn(0x7fc254071090 0x7fc25415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc24bfff640 1 --2- 172.21.15.40:0/1839401679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc254070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc251d9b640 1 -- 172.21.15.40:0/1839401679 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244864625 0 0) 0x7fc2540d2390 con 0x7fc2540cfde0 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc251d9b640 1 -- 172.21.15.40:0/1839401679 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc240003520 con 0x7fc2540cfde0 2024-04-26T21:43:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc252d9d640 1 -- 172.21.15.40:0/1839401679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 msgr2=0x7fc254070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc252d9d640 1 --2- 172.21.15.40:0/1839401679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc254070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc252d9d640 1 -- 172.21.15.40:0/1839401679 >> v1:172.21.15.40:6789/0 conn(0x7fc2540cfde0 legacy=0x7fc2540d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc252d9d640 1 -- 172.21.15.40:0/1839401679 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25406d3f0 con 0x7fc254071090 2024-04-26T21:43:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.836+0000 7fc24bfff640 1 --2- 172.21.15.40:0/1839401679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc254070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.837+0000 7fc252d9d640 1 --2- 172.21.15.40:0/1839401679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25415e9f0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009d90 tx=0x7fc24c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66fa155dbd773c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.837+0000 7fc251d9b640 1 -- 172.21.15.40:0/1839401679 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c031600 con 0x7fc254071090 2024-04-26T21:43:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.837+0000 7fc251d9b640 1 -- 172.21.15.40:0/1839401679 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc24c0317a0 con 0x7fc254071090 2024-04-26T21:43:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.837+0000 7fc251d9b640 1 -- 172.21.15.40:0/1839401679 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c031b70 con 0x7fc254071090 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.838+0000 7fc2589e2640 1 -- 172.21.15.40:0/1839401679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 msgr2=0x7fc25415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.838+0000 7fc2589e2640 1 --2- 172.21.15.40:0/1839401679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25415e9f0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009d90 tx=0x7fc24c02a5d0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.838+0000 7fc2589e2640 1 -- 172.21.15.40:0/1839401679 shutdown_connections 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.838+0000 7fc2589e2640 1 --2- 172.21.15.40:0/1839401679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25415e9f0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.838+0000 7fc2589e2640 1 --2- 172.21.15.40:0/1839401679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc254070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.838+0000 7fc2589e2640 1 -- 172.21.15.40:0/1839401679 >> 172.21.15.40:0/1839401679 conn(0x7fc2540d2d80 msgr2=0x7fc2540d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.838+0000 7fc2589e2640 1 -- 172.21.15.40:0/1839401679 shutdown_connections 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.838+0000 7fc2589e2640 1 -- 172.21.15.40:0/1839401679 wait complete. 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.839+0000 7fc2589e2640 1 Processor -- start 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.839+0000 7fc2589e2640 1 -- start start 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc2589e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc25409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc2589e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc2589e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2540cfde0 0x7fc25409ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc2589e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc25406d9b0 con 0x7fc254071090 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc2589e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc25406dd20 con 0x7fc2540cfde0 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc2589e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc25415ef30 con 0x7fc254070350 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25414d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60360/0 (socket says 172.21.15.40:60360) 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 -- 172.21.15.40:0/888701687 learned_addr learned my addr 172.21.15.40:0/888701687 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc25259c640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc25409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc252d9d640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2540cfde0 0x7fc25409ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 -- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 msgr2=0x7fc25409d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc25409d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 -- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2540cfde0 msgr2=0x7fc25409ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2540cfde0 0x7fc25409ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 -- 172.21.15.40:0/888701687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2541f4ec0 con 0x7fc254071090 2024-04-26T21:43:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24bfff640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25414d4e0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00e810 tx=0x7fc23c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.840+0000 7fc24affd640 1 -- 172.21.15.40:0/888701687 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00f040 con 0x7fc254071090 2024-04-26T21:43:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.841+0000 7fc24affd640 1 -- 172.21.15.40:0/888701687 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c004030 con 0x7fc254071090 2024-04-26T21:43:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.841+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2541f51a0 con 0x7fc254071090 2024-04-26T21:43:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.841+0000 7fc24affd640 1 -- 172.21.15.40:0/888701687 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c004400 con 0x7fc254071090 2024-04-26T21:43:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.841+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2541f5760 con 0x7fc254071090 2024-04-26T21:43:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.841+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2540694f0 con 0x7fc254071090 2024-04-26T21:43:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.842+0000 7fc24affd640 1 -- 172.21.15.40:0/888701687 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc23c00c6d0 con 0x7fc254071090 2024-04-26T21:43:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.842+0000 7fc24affd640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc23406c8f0 0x7fc23406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.842+0000 7fc24affd640 1 -- 172.21.15.40:0/888701687 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7fc23c08be30 con 0x7fc254071090 2024-04-26T21:43:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.843+0000 7fc25259c640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc23406c8f0 0x7fc23406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.843+0000 7fc25259c640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc23406c8f0 0x7fc23406edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc244007fd0 tx=0x7fc244009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:12.847+0000 7fc24affd640 1 -- 172.21.15.40:0/888701687 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc23c017020 con 0x7fc254071090 2024-04-26T21:43:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.032+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc254069bb0 con 0x7fc254071090 2024-04-26T21:43:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.032+0000 7fc24affd640 1 -- 172.21.15.40:0/888701687 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7fc23c059d40 con 0x7fc254071090 2024-04-26T21:43:13.032 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc23406c8f0 msgr2=0x7fc23406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc23406c8f0 0x7fc23406edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc244007fd0 tx=0x7fc244009040 comp rx=0 tx=0).stop 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 msgr2=0x7fc25414d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25414d4e0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00e810 tx=0x7fc23c00ecd0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 shutdown_connections 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc23406c8f0 0x7fc23406edb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2540cfde0 0x7fc25409ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc254071090 0x7fc25414d4e0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 --2- 172.21.15.40:0/888701687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc254070350 0x7fc25409d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 >> 172.21.15.40:0/888701687 conn(0x7fc2540d2d80 msgr2=0x7fc2540b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 shutdown_connections 2024-04-26T21:43:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.035+0000 7fc2589e2640 1 -- 172.21.15.40:0/888701687 wait complete. 2024-04-26T21:43:13.046 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:43:13.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.148+0000 7f85ecb9d640 1 Processor -- start 2024-04-26T21:43:13.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.148+0000 7f85ecb9d640 1 -- start start 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85ecb9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f85e8162880 0x7f85e80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85ecb9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f85e80b5f20 0x7f85e80d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85ecb9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e80b7e50 con 0x7f85e80d39f0 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85ecb9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f85e80d3e40 con 0x7f85e80b5f20 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85ecb9d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f85e80d0580 con 0x7f85e8162880 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85dffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f85e80b5f20 0x7f85e80d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85dffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f85e80b5f20 0x7f85e80d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60366/0 (socket says 172.21.15.40:60366) 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85dffff640 1 -- 172.21.15.40:0/1947379786 learned_addr learned my addr 172.21.15.40:0/1947379786 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85dffff640 1 -- 172.21.15.40:0/1947379786 >> v2:172.21.15.184:3300/0 conn(0x7f85e8162880 msgr2=0x7f85e80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85e659c640 1 --2- 172.21.15.40:0/1947379786 >> v2:172.21.15.184:3300/0 conn(0x7f85e8162880 0x7f85e80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85dffff640 1 --2- 172.21.15.40:0/1947379786 >> v2:172.21.15.184:3300/0 conn(0x7f85e8162880 0x7f85e80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85dffff640 1 -- 172.21.15.40:0/1947379786 >> v1:172.21.15.184:6789/0 conn(0x7f85e80d39f0 legacy=0x7f85e80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.149+0000 7f85dffff640 1 -- 172.21.15.40:0/1947379786 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e80d1800 con 0x7f85e80b5f20 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.150+0000 7f85dffff640 1 --2- 172.21.15.40:0/1947379786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e80b5f20 0x7f85e80d34b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f85e808d490 tx=0x7f85d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba29e1433e9a135 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.150+0000 7f85e5d9b640 1 -- 172.21.15.40:0/1947379786 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0037070 con 0x7f85e80b5f20 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.150+0000 7f85e5d9b640 1 -- 172.21.15.40:0/1947379786 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d0032440 con 0x7f85e80b5f20 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.150+0000 7f85e5d9b640 1 -- 172.21.15.40:0/1947379786 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0032810 con 0x7f85e80b5f20 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.151+0000 7f85ecb9d640 1 -- 172.21.15.40:0/1947379786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e80b5f20 msgr2=0x7f85e80d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.151+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/1947379786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e80b5f20 0x7f85e80d34b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f85e808d490 tx=0x7f85d002a230 comp rx=0 tx=0).stop 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.151+0000 7f85ecb9d640 1 -- 172.21.15.40:0/1947379786 shutdown_connections 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.151+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/1947379786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e80b5f20 0x7f85e80d34b0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.151+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/1947379786 >> v2:172.21.15.184:3300/0 conn(0x7f85e8162880 0x7f85e80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.151+0000 7f85ecb9d640 1 -- 172.21.15.40:0/1947379786 >> 172.21.15.40:0/1947379786 conn(0x7f85e80d4f60 msgr2=0x7f85e80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.151+0000 7f85ecb9d640 1 -- 172.21.15.40:0/1947379786 shutdown_connections 2024-04-26T21:43:13.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.151+0000 7f85ecb9d640 1 -- 172.21.15.40:0/1947379786 wait complete. 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.152+0000 7f85ecb9d640 1 Processor -- start 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.152+0000 7f85ecb9d640 1 -- start start 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85ecb9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85e80b5f20 0x7f85e81594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85ecb9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85e80d39f0 0x7f85e815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85ecb9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e8162880 0x7f85e806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85ecb9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85e80d0d90 con 0x7f85e8162880 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85ecb9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85e80d0ab0 con 0x7f85e80b5f20 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85ecb9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85e8162c60 con 0x7f85e80d39f0 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e8162880 0x7f85e806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e8162880 0x7f85e806dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60380/0 (socket says 172.21.15.40:60380) 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e6d9d640 1 -- 172.21.15.40:0/4121196530 learned_addr learned my addr 172.21.15.40:0/4121196530 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85dffff640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85e80d39f0 0x7f85e815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e659c640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85e80b5f20 0x7f85e81594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e6d9d640 1 -- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85e80d39f0 msgr2=0x7f85e815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e6d9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85e80d39f0 0x7f85e815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e6d9d640 1 -- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85e80b5f20 msgr2=0x7f85e81594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e6d9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85e80b5f20 0x7f85e81594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e6d9d640 1 -- 172.21.15.40:0/4121196530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e8159a10 con 0x7f85e8162880 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85dffff640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85e80d39f0 0x7f85e815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.153+0000 7f85e659c640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85e80b5f20 0x7f85e81594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.154+0000 7f85e6d9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e8162880 0x7f85e806dee0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f85e000b970 tx=0x7f85e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.154+0000 7f85deffd640 1 -- 172.21.15.40:0/4121196530 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0019070 con 0x7f85e8162880 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.154+0000 7f85deffd640 1 -- 172.21.15.40:0/4121196530 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e000cb40 con 0x7f85e8162880 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.154+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e815b4a0 con 0x7f85e8162880 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.154+0000 7f85deffd640 1 -- 172.21.15.40:0/4121196530 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0014440 con 0x7f85e8162880 2024-04-26T21:43:13.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.154+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e815ba60 con 0x7f85e8162880 2024-04-26T21:43:13.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.155+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e80694f0 con 0x7f85e8162880 2024-04-26T21:43:13.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.155+0000 7f85deffd640 1 -- 172.21.15.40:0/4121196530 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85e001c070 con 0x7f85e8162880 2024-04-26T21:43:13.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.156+0000 7f85deffd640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f85c806c920 0x7f85c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.156+0000 7f85deffd640 1 -- 172.21.15.40:0/4121196530 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f85e008bc40 con 0x7f85e8162880 2024-04-26T21:43:13.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.156+0000 7f85e659c640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f85c806c920 0x7f85c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.156+0000 7f85e659c640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f85c806c920 0x7f85c806ede0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f85d8008890 tx=0x7f85d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.160+0000 7f85deffd640 1 -- 172.21.15.40:0/4121196530 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85e0059b40 con 0x7f85e8162880 2024-04-26T21:43:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.359+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f85e8074590 con 0x7f85e8162880 2024-04-26T21:43:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.359+0000 7f85deffd640 1 -- 172.21.15.40:0/4121196530 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 12 entries v101) ==== 81+0+734 (secure 0 0 0) 0x7f85e0059440 con 0x7f85e8162880 2024-04-26T21:43:13.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:43:13.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:43:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:listed 12 entries 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.361+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f85c806c920 msgr2=0x7f85c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.361+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f85c806c920 0x7f85c806ede0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f85d8008890 tx=0x7f85d8008040 comp rx=0 tx=0).stop 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.361+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e8162880 msgr2=0x7f85e806dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.361+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e8162880 0x7f85e806dee0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f85e000b970 tx=0x7f85e000be30 comp rx=0 tx=0).stop 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.362+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 shutdown_connections 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.362+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f85c806c920 0x7f85c806ede0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.362+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85e8162880 0x7f85e806dee0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.362+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85e80d39f0 0x7f85e815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.362+0000 7f85ecb9d640 1 --2- 172.21.15.40:0/4121196530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85e80b5f20 0x7f85e81594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.362+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 >> 172.21.15.40:0/4121196530 conn(0x7f85e80d4f60 msgr2=0x7f85e8160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.362+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 shutdown_connections 2024-04-26T21:43:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.362+0000 7f85ecb9d640 1 -- 172.21.15.40:0/4121196530 wait complete. 2024-04-26T21:43:13.372 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:43:13.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:13.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:43:13.398 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:43:13.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:13.398 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:43:13.454 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:43:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.597+0000 7f1dc666d640 1 Processor -- start 2024-04-26T21:43:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.597+0000 7f1dc666d640 1 -- start start 2024-04-26T21:43:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dc666d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 0x7f1dc00d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dc666d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1dc00d0a90 0x7f1dc00cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dc666d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc00cde20 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dc666d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc00cea30 con 0x7f1dc00d0a90 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dc666d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc00ce100 con 0x7f1dc01608f0 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dbf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1dc00d0a90 0x7f1dc00cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dc4c0a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1dc00cfdd0 0x7f1dc0160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41994/0 (socket says 172.21.15.40:41994) 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dc4c0a640 1 -- 172.21.15.40:0/1818760484 learned_addr learned my addr 172.21.15.40:0/1818760484 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dbffff640 1 --2- 172.21.15.40:0/1818760484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 0x7f1dc00d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dbf7fe640 1 -- 172.21.15.40:0/1818760484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 msgr2=0x7f1dc00d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dbf7fe640 1 --2- 172.21.15.40:0/1818760484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 0x7f1dc00d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dbf7fe640 1 -- 172.21.15.40:0/1818760484 >> v1:172.21.15.40:6789/0 conn(0x7f1dc00cfdd0 legacy=0x7f1dc0160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dbeffd640 1 -- 172.21.15.40:0/1818760484 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1258121036 0 0) 0x7f1dc00cde20 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.598+0000 7f1dbf7fe640 1 -- 172.21.15.40:0/1818760484 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc00ce3e0 con 0x7f1dc00d0a90 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.599+0000 7f1dbf7fe640 1 --2- 172.21.15.40:0/1818760484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00d0a90 0x7f1dc00cf890 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f1db00097a0 tx=0x7f1db002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b265cb96289d5f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.599+0000 7f1dbeffd640 1 -- 172.21.15.40:0/1818760484 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0037070 con 0x7f1dc00d0a90 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.599+0000 7f1dbeffd640 1 -- 172.21.15.40:0/1818760484 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0032440 con 0x7f1dc00d0a90 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.599+0000 7f1dbeffd640 1 -- 172.21.15.40:0/1818760484 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0032810 con 0x7f1dc00d0a90 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.600+0000 7f1dc666d640 1 -- 172.21.15.40:0/1818760484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00d0a90 msgr2=0x7f1dc00cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.600+0000 7f1dc666d640 1 --2- 172.21.15.40:0/1818760484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00d0a90 0x7f1dc00cf890 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f1db00097a0 tx=0x7f1db002a180 comp rx=0 tx=0).stop 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.600+0000 7f1dc666d640 1 -- 172.21.15.40:0/1818760484 shutdown_connections 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.600+0000 7f1dc666d640 1 --2- 172.21.15.40:0/1818760484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00d0a90 0x7f1dc00cf890 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.600+0000 7f1dc666d640 1 --2- 172.21.15.40:0/1818760484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 0x7f1dc00d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.600+0000 7f1dc666d640 1 -- 172.21.15.40:0/1818760484 >> 172.21.15.40:0/1818760484 conn(0x7f1dc00d2ec0 msgr2=0x7f1dc00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.600+0000 7f1dc666d640 1 -- 172.21.15.40:0/1818760484 shutdown_connections 2024-04-26T21:43:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.600+0000 7f1dc666d640 1 -- 172.21.15.40:0/1818760484 wait complete. 2024-04-26T21:43:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.601+0000 7f1dc666d640 1 Processor -- start 2024-04-26T21:43:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.601+0000 7f1dc666d640 1 -- start start 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dc666d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00cfdd0 0x7f1dc015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dc666d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc00d0a90 0x7f1dc00c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dc666d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 0x7f1dc00cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dc666d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00b3310 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dc666d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00d0ef0 con 0x7f1dc00d0a90 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dc666d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc00ceec0 con 0x7f1dc01608f0 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00cfdd0 0x7f1dc015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00cfdd0 0x7f1dc015a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60406/0 (socket says 172.21.15.40:60406) 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbffff640 1 -- 172.21.15.40:0/4192664090 learned_addr learned my addr 172.21.15.40:0/4192664090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dc4c0a640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 0x7f1dc00cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbf7fe640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc00d0a90 0x7f1dc00c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbffff640 1 -- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 msgr2=0x7f1dc00cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbffff640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 0x7f1dc00cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbffff640 1 -- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc00d0a90 msgr2=0x7f1dc00c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbffff640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc00d0a90 0x7f1dc00c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.602+0000 7f1dbffff640 1 -- 172.21.15.40:0/4192664090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc00c8c40 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.603+0000 7f1dbffff640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00cfdd0 0x7f1dc015a440 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f1da800d600 tx=0x7f1da800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.603+0000 7f1dbcff9640 1 -- 172.21.15.40:0/4192664090 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da80125f0 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.603+0000 7f1dbcff9640 1 -- 172.21.15.40:0/4192664090 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da8019070 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.603+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc00c8f20 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.603+0000 7f1dbcff9640 1 -- 172.21.15.40:0/4192664090 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da80129f0 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.603+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc00b0c10 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.604+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc00694f0 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.604+0000 7f1dbcff9640 1 -- 172.21.15.40:0/4192664090 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1da8012b90 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.605+0000 7f1dbcff9640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1da006c8f0 0x7f1da006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.605+0000 7f1dbcff9640 1 -- 172.21.15.40:0/4192664090 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6733+0+0 (secure 0 0 0) 0x7f1da808c440 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.605+0000 7f1dbf7fe640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1da006c8f0 0x7f1da006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.605+0000 7f1dbf7fe640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1da006c8f0 0x7f1da006edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1db0034000 tx=0x7f1db002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.609+0000 7f1dbcff9640 1 -- 172.21.15.40:0/4192664090 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1da805a3d0 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.832+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1dc00c6600 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.832 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:13.833+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:43:13.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.834+0000 7f1dbcff9640 1 -- 172.21.15.40:0/4192664090 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v130) ==== 161+0+0 (secure 0 0 0) 0x7f1da8059cd0 con 0x7f1dc00cfdd0 2024-04-26T21:43:13.834 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:43:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1da006c8f0 msgr2=0x7f1da006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1da006c8f0 0x7f1da006edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1db0034000 tx=0x7f1db002e000 comp rx=0 tx=0).stop 2024-04-26T21:43:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00cfdd0 msgr2=0x7f1dc015a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00cfdd0 0x7f1dc015a440 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f1da800d600 tx=0x7f1da800dac0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.837 INFO:tasks.ceph.mds.c.smithi040.stderr: -1859> 2024-04-26T21:43:03.768+0000 7f7a3e43b640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-04-26T21:43:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 shutdown_connections 2024-04-26T21:43:13.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1da006c8f0 0x7f1da006edb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc01608f0 0x7f1dc00cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc00d0a90 0x7f1dc00c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 --2- 172.21.15.40:0/4192664090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00cfdd0 0x7f1dc015a440 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 >> 172.21.15.40:0/4192664090 conn(0x7f1dc00d2ec0 msgr2=0x7f1dc00719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:13.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 shutdown_connections 2024-04-26T21:43:13.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.837+0000 7f1dc666d640 1 -- 172.21.15.40:0/4192664090 wait complete. 2024-04-26T21:43:13.838 INFO:tasks.ceph.mds.c.smithi040.stderr: -681> 2024-04-26T21:43:09.203+0000 7f7a42443640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:43:13.849 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:43:13.864 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:43:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.951+0000 7f78c7d04640 1 Processor -- start 2024-04-26T21:43:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.952+0000 7f78c7d04640 1 -- start start 2024-04-26T21:43:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.952+0000 7f78c7d04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c0162810 0x7f78c0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.952+0000 7f78c7d04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78c00d1b30 0x7f78c00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.952+0000 7f78c7d04640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c00d10d0 con 0x7f78c00b2340 2024-04-26T21:43:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.952+0000 7f78c7d04640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f78c00d0750 con 0x7f78c00d1b30 2024-04-26T21:43:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.952+0000 7f78c7d04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00d16a0 con 0x7f78c0162810 2024-04-26T21:43:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c529f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78c00b2340 0x7f78c00cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42020/0 (socket says 172.21.15.40:42020) 2024-04-26T21:43:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c62a1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78c00d1b30 0x7f78c00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c529f640 1 -- 172.21.15.40:0/1181365791 learned_addr learned my addr 172.21.15.40:0/1181365791 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c5aa0640 1 --2- 172.21.15.40:0/1181365791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c0162810 0x7f78c0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c62a1640 1 -- 172.21.15.40:0/1181365791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c0162810 msgr2=0x7f78c0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c62a1640 1 --2- 172.21.15.40:0/1181365791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c0162810 0x7f78c0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c62a1640 1 -- 172.21.15.40:0/1181365791 >> v1:172.21.15.40:6789/0 conn(0x7f78c00b2340 legacy=0x7f78c00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c4a9e640 1 -- 172.21.15.40:0/1181365791 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1018143128 0 0) 0x7f78c00d10d0 con 0x7f78c00b2340 2024-04-26T21:43:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.953+0000 7f78c62a1640 1 -- 172.21.15.40:0/1181365791 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c00d1370 con 0x7f78c00d1b30 2024-04-26T21:43:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c62a1640 1 --2- 172.21.15.40:0/1181365791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c00d1b30 0x7f78c00d1fc0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009d90 tx=0x7f78b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768b49ba0aff3f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c4a9e640 1 -- 172.21.15.40:0/1181365791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8007740 con 0x7f78c00d1b30 2024-04-26T21:43:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c4a9e640 1 -- 172.21.15.40:0/1181365791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b80078e0 con 0x7f78c00d1b30 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c4a9e640 1 -- 172.21.15.40:0/1181365791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8007cb0 con 0x7f78c00d1b30 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c7d04640 1 -- 172.21.15.40:0/1181365791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c00d1b30 msgr2=0x7f78c00d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1181365791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c00d1b30 0x7f78c00d1fc0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009d90 tx=0x7f78b802a660 comp rx=0 tx=0).stop 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c7d04640 1 -- 172.21.15.40:0/1181365791 shutdown_connections 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1181365791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c00d1b30 0x7f78c00d1fc0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1181365791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c0162810 0x7f78c0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c7d04640 1 -- 172.21.15.40:0/1181365791 >> 172.21.15.40:0/1181365791 conn(0x7f78c00d2da0 msgr2=0x7f78c00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.954+0000 7f78c7d04640 1 -- 172.21.15.40:0/1181365791 shutdown_connections 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.955+0000 7f78c7d04640 1 -- 172.21.15.40:0/1181365791 wait complete. 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.956+0000 7f78c7d04640 1 Processor -- start 2024-04-26T21:43:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.956+0000 7f78c7d04640 1 -- start start 2024-04-26T21:43:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.956+0000 7f78c7d04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b2340 0x7f78c01f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.956+0000 7f78c7d04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c00d1b30 0x7f78c00cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.956+0000 7f78c7d04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c0162810 0x7f78c01f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.956+0000 7f78c7d04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0070890 con 0x7f78c0162810 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.956+0000 7f78c7d04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00d0470 con 0x7f78c00d1b30 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.956+0000 7f78c7d04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00cdb00 con 0x7f78c00b2340 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c0162810 0x7f78c01f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c0162810 0x7f78c01f4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60434/0 (socket says 172.21.15.40:60434) 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 -- 172.21.15.40:0/1161463505 learned_addr learned my addr 172.21.15.40:0/1161463505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c5aa0640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b2340 0x7f78c01f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c529f640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c00d1b30 0x7f78c00cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 -- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b2340 msgr2=0x7f78c01f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b2340 0x7f78c01f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 -- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c00d1b30 msgr2=0x7f78c00cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c00d1b30 0x7f78c00cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 -- 172.21.15.40:0/1161463505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c01f5740 con 0x7f78c0162810 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c529f640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c00d1b30 0x7f78c00cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c62a1640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c0162810 0x7f78c01f4df0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009eb0 tx=0x7f78b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78b67fc640 1 -- 172.21.15.40:0/1161463505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8040070 con 0x7f78c0162810 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78b67fc640 1 -- 172.21.15.40:0/1161463505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8031bc0 con 0x7f78c0162810 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78b67fc640 1 -- 172.21.15.40:0/1161463505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8045460 con 0x7f78c0162810 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c01f5a20 con 0x7f78c0162810 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.957+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c00c9f10 con 0x7f78c0162810 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.958+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c00694f0 con 0x7f78c0162810 2024-04-26T21:43:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.959+0000 7f78b67fc640 1 -- 172.21.15.40:0/1161463505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78b8039040 con 0x7f78c0162810 2024-04-26T21:43:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.959+0000 7f78b67fc640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78a406c920 0x7f78a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.959+0000 7f78b67fc640 1 -- 172.21.15.40:0/1161463505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f78b80ad7a0 con 0x7f78c0162810 2024-04-26T21:43:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.959+0000 7f78c5aa0640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78a406c920 0x7f78a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.960+0000 7f78c5aa0640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78a406c920 0x7f78a406ede0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f78b00079a0 tx=0x7f78b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:13.964+0000 7f78b67fc640 1 -- 172.21.15.40:0/1161463505 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78b807b800 con 0x7f78c0162810 2024-04-26T21:43:14.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.223+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f78c00b2b30 con 0x7f78c0162810 2024-04-26T21:43:14.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.836+0000 7f78b67fc640 1 -- 172.21.15.40:0/1161463505 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v131) ==== 107+0+0 (secure 0 0 0) 0x7f78b807b100 con 0x7f78c0162810 2024-04-26T21:43:14.836 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:43:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78a406c920 msgr2=0x7f78a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78a406c920 0x7f78a406ede0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f78b00079a0 tx=0x7f78b0008040 comp rx=0 tx=0).stop 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c0162810 msgr2=0x7f78c01f4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c0162810 0x7f78c01f4df0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009eb0 tx=0x7f78b8004a40 comp rx=0 tx=0).stop 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 shutdown_connections 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78a406c920 0x7f78a406ede0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c0162810 0x7f78c01f4df0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c00d1b30 0x7f78c00cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 --2- 172.21.15.40:0/1161463505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b2340 0x7f78c01f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 >> 172.21.15.40:0/1161463505 conn(0x7f78c00d2da0 msgr2=0x7f78c00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 shutdown_connections 2024-04-26T21:43:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.838+0000 7f78c7d04640 1 -- 172.21.15.40:0/1161463505 wait complete. 2024-04-26T21:43:14.852 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:43:14.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.939+0000 7fe30f5d4f40 1 Processor -- start 2024-04-26T21:43:14.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.939+0000 7fe30f5d4f40 1 -- start start 2024-04-26T21:43:14.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30f5d4f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 0x55f977238580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30f5d4f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f977238ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30f5d4f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 0x55f977199c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30f5d4f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f977115e40 con 0x55f97719caf0 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30f5d4f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f977119e80 con 0x55f97716c930 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30f5d4f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f97719c010 con 0x55f977245900 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f977238ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f977238ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60450/0 (socket says 172.21.15.40:60450) 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 -- 172.21.15.40:0/1834668766 learned_addr learned my addr 172.21.15.40:0/1834668766 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30d8cf640 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 0x55f977199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30d0ce640 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 0x55f977238580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 -- 172.21.15.40:0/1834668766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 msgr2=0x55f977199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 0x55f977199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 -- 172.21.15.40:0/1834668766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 msgr2=0x55f977238580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 0x55f977238580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 -- 172.21.15.40:0/1834668766 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f977240280 con 0x55f97719caf0 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.940+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f977238ac0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8001e30 tx=0x7fe2f800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9d3632b6314a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe3077fe640 1 -- 172.21.15.40:0/1834668766 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8012020 con 0x55f97719caf0 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe3077fe640 1 -- 172.21.15.40:0/1834668766 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f800e670 con 0x55f97719caf0 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe3077fe640 1 -- 172.21.15.40:0/1834668766 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f800eac0 con 0x55f97719caf0 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/1834668766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 msgr2=0x55f977238ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f977238ac0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8001e30 tx=0x7fe2f800b780 comp rx=0 tx=0).stop 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/1834668766 shutdown_connections 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 0x55f977199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f977238ac0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/1834668766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 0x55f977238580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/1834668766 >> 172.21.15.40:0/1834668766 conn(0x55f97716d330 msgr2=0x55f977243210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/1834668766 shutdown_connections 2024-04-26T21:43:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.941+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/1834668766 wait complete. 2024-04-26T21:43:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.942+0000 7fe30f5d4f40 1 Processor -- start 2024-04-26T21:43:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.942+0000 7fe30f5d4f40 1 -- start start 2024-04-26T21:43:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30f5d4f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 0x55f977243400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30f5d4f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f9771b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30f5d4f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 0x55f9771b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30f5d4f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f9772441a0 con 0x55f97719caf0 2024-04-26T21:43:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30f5d4f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f977240030 con 0x55f97716c930 2024-04-26T21:43:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30f5d4f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f9772443f0 con 0x55f977245900 2024-04-26T21:43:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f9771b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f9771b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60464/0 (socket says 172.21.15.40:60464) 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 learned_addr learned my addr 172.21.15.40:0/2067337555 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30d0ce640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 0x55f977243400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30d8cf640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 0x55f9771b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 msgr2=0x55f9771b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 0x55f9771b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 msgr2=0x55f977243400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f97716c930 0x55f977243400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f977238c30 con 0x55f97719caf0 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30d8cf640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f977245900 0x55f9771b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f9771b1f50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x55f977240e70 tx=0x7fe2f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe3067fc640 1 -- 172.21.15.40:0/2067337555 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8027070 con 0x55f97719caf0 2024-04-26T21:43:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.943+0000 7fe3067fc640 1 -- 172.21.15.40:0/2067337555 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f800e470 con 0x55f97719caf0 2024-04-26T21:43:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.944+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f9772d72b0 con 0x55f97719caf0 2024-04-26T21:43:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.944+0000 7fe3067fc640 1 -- 172.21.15.40:0/2067337555 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8019a30 con 0x55f97719caf0 2024-04-26T21:43:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.944+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9772d7590 con 0x55f97719caf0 2024-04-26T21:43:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.945+0000 7fe3067fc640 1 -- 172.21.15.40:0/2067337555 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2f802a070 con 0x55f97719caf0 2024-04-26T21:43:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.945+0000 7fe3067fc640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe2f006b300 0x7fe2f006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.945+0000 7fe3067fc640 1 -- 172.21.15.40:0/2067337555 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7fe2f808c2b0 con 0x55f97719caf0 2024-04-26T21:43:14.948 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:43:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.946+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f977243940 0x55f977199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.946+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9771b29d0 con 0x55f977243940 2024-04-26T21:43:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.946+0000 7fe30d0ce640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe2f006b300 0x7fe2f006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.946+0000 7fe30d8cf640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f977243940 0x55f977199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.946+0000 7fe30d8cf640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f977243940 0x55f977199cf0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.946+0000 7fe30d0ce640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe2f006b300 0x7fe2f006d7c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe300000f80 tx=0x7fe300006a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.949+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v102'195 uv169 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.949+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f97716f6e0 con 0x55f977243940 2024-04-26T21:43:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.950+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v102'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.950+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772d6470 con 0x55f977243940 2024-04-26T21:43:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.951+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v102'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.951 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:43:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.951+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f97719a230 con 0x55f977243940 2024-04-26T21:43:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.952+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v102'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.952+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e9b80 con 0x55f977243940 2024-04-26T21:43:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.953+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v102'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.953+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.954+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v102'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.954+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.955+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v102'201 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.955+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.956+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v102'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.956+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.957 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:43:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.958+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v102'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.958+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.959+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v102'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.959+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.960+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v102'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.960+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.961+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v102'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.961+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.962+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v102'207 uv182 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.962+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.963+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v102'208 uv208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.963+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.964+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v102'209 uv209 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.964 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:43:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.964+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.965+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v102'210 uv210 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.965+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.966+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v102'211 uv211 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.966+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8b70 con 0x55f977243940 2024-04-26T21:43:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.967+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v102'212 uv212 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.967+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.968+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v102'213 uv213 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.968+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.969+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v102'214 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.969+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.970+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v102'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.970+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.970 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:43:14.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.971+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v102'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.971+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.972+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v102'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.972+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.973+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v102'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.973+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.974+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v102'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.974+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.975+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v102'220 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.975+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.976+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v102'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.976+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.976 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:43:14.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.977+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v102'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.977+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.978+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v102'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.978+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.979+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v102'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.979+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772e8f60 con 0x55f977243940 2024-04-26T21:43:14.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.980+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v102'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f977243940 2024-04-26T21:43:14.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.980+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f9772e9f20 0x55f9772ec310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.980+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ec850 con 0x55f9772e9f20 2024-04-26T21:43:14.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.980+0000 7fe30d0ce640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f9772e9f20 0x55f9772ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.980+0000 7fe30d0ce640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f9772e9f20 0x55f9772ec310 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:14.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.982+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v102'152 uv133 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.982+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772ecbf0 con 0x55f9772e9f20 2024-04-26T21:43:14.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.983+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v102'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.983+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ecf90 con 0x55f9772e9f20 2024-04-26T21:43:14.984 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:43:14.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.985+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v102'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.985+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ed330 con 0x55f9772e9f20 2024-04-26T21:43:14.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.986+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v102'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.986+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ed6d0 con 0x55f9772e9f20 2024-04-26T21:43:14.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.987+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v102'156 uv156 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.987+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772eda70 con 0x55f9772e9f20 2024-04-26T21:43:14.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.989+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v102'157 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.989+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772ede10 con 0x55f9772e9f20 2024-04-26T21:43:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.990+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v102'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.990+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:14.991 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:43:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.991+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v102'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.991+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.993+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v102'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.993+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.994+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v102'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.994+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:14.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.995+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v102'162 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.995+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:14.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.996+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v102'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.996+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:14.996 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:43:14.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.997+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v102'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.997+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f2870 con 0x55f9772e9f20 2024-04-26T21:43:14.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.998+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v102'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:14.998+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f2870 con 0x55f9772e9f20 2024-04-26T21:43:14.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.000+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v102'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:14.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.000+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:15.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.001+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v102'167 uv147 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:15.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.001+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:15.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.002+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v102'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.002+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:15.002 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:43:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.003+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v102'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.003+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ef1c0 con 0x55f9772e9f20 2024-04-26T21:43:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.004+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v102'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:15.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.004+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ef1c0 con 0x55f9772e9f20 2024-04-26T21:43:15.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.005+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v102'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:15.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.005+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772ee1b0 con 0x55f9772e9f20 2024-04-26T21:43:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.006+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v102'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772e9f20 2024-04-26T21:43:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.006+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f9772ee710 0x55f9772f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.006+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f1160 con 0x55f9772ee710 2024-04-26T21:43:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.006+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f9772ee710 0x55f9772f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.007+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f9772ee710 0x55f9772f0b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.008+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v102'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.008+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f1620 con 0x55f9772ee710 2024-04-26T21:43:15.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.009+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v102'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.009+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f1b00 con 0x55f9772ee710 2024-04-26T21:43:15.010 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:43:15.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.010+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v102'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.010+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f33b0 con 0x55f9772ee710 2024-04-26T21:43:15.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.011+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v102'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.011+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f3750 con 0x55f9772ee710 2024-04-26T21:43:15.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.012+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v102'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.012+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f23a0 con 0x55f9772ee710 2024-04-26T21:43:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.013+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v102'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.013+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f9772f2ca0 0x55f9772f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.013+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f6040 con 0x55f9772f2ca0 2024-04-26T21:43:15.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.014+0000 7fe30d8cf640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f9772f2ca0 0x55f9772f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.014+0000 7fe30d8cf640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f9772f2ca0 0x55f9772f5b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.017+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v102'34 uv33 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.017+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f6040 con 0x55f9772f2ca0 2024-04-26T21:43:15.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.019+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v102'35 uv35 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.019+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f60c0 con 0x55f9772f2ca0 2024-04-26T21:43:15.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.020+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v102'36 uv36 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.020 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:43:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.020+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f64d0 con 0x55f9772f2ca0 2024-04-26T21:43:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.021+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v102'37 uv37 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.021+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f9772f69e0 0x55f9772f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.021+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9350 con 0x55f9772f69e0 2024-04-26T21:43:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.021+0000 7fe30d0ce640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f9772f69e0 0x55f9772f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.022+0000 7fe30d0ce640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f9772f69e0 0x55f9772f8dd0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.024+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v102'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.025+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.026+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v102'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.026+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.027 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:43:15.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.028+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v102'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.028+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fa730 con 0x55f9772f69e0 2024-04-26T21:43:15.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.029+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v102'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.029+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fa730 con 0x55f9772f69e0 2024-04-26T21:43:15.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.030+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v102'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.030+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.032+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v102'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.032+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.033+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v102'70 uv64 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.033+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.034+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v102'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.034+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.035+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v102'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.035 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:43:15.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.035+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fa730 con 0x55f9772f2ca0 2024-04-26T21:43:15.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.036+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v102'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.036+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fa730 con 0x55f9772f2ca0 2024-04-26T21:43:15.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.037+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v102'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.037+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.038+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v102'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.038+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.039+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v102'76 uv69 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.039+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.040+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v102'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.040+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.041+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v102'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.041 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:43:15.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.041+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.042+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v102'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.042+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.043+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v102'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.043+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.044+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v102'32 uv31 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.044+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.045+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v102'33 uv33 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.045+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.046+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v102'34 uv34 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.046 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:43:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.046+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.047+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v102'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.047+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f2ca0 2024-04-26T21:43:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.050+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v102'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe308053410 con 0x55f9772f2ca0 2024-04-26T21:43:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.050+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.051+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v102'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.051+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.053+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v102'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.053+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.053 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:43:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.054+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v102'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.054+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.055+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v102'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.055+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.056+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v102'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.056+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.056+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v102'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.057+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.058+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v102'64 uv58 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.058+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.059+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v102'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.059+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9720 con 0x55f9772f69e0 2024-04-26T21:43:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.060+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v102'66 uv66 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.059 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:43:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.060+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9b80 con 0x55f9772f69e0 2024-04-26T21:43:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.061+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v102'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.061+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9b30 con 0x55f9772f69e0 2024-04-26T21:43:15.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.062+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v102'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.062+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9b30 con 0x55f9772f69e0 2024-04-26T21:43:15.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.063+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v102'69 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.063+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772f9b30 con 0x55f9772f69e0 2024-04-26T21:43:15.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.064+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v102'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.064+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9b30 con 0x55f9772f69e0 2024-04-26T21:43:15.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.065+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v102'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.064 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:43:15.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.065+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9b30 con 0x55f9772f69e0 2024-04-26T21:43:15.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.066+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v102'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.066+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772f9f40 con 0x55f9772f69e0 2024-04-26T21:43:15.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.067+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v102'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.067+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f9772fa170 0x55f9772fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.067+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fa770 con 0x55f9772fa170 2024-04-26T21:43:15.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.067+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f9772fa170 0x55f9772fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.068+0000 7fe30c8cd640 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f9772fa170 0x55f9772fcb70 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.069+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v102'31 uv29 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772fa170 2024-04-26T21:43:15.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.069+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772fa770 con 0x55f9772fa170 2024-04-26T21:43:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.070+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v102'32 uv32 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772fa170 2024-04-26T21:43:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.070+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fa770 con 0x55f9772fa170 2024-04-26T21:43:15.072 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:43:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.073+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v102'33 uv33 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772fa170 2024-04-26T21:43:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.073+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772fa170 2024-04-26T21:43:15.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.075+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v102'34 uv34 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772fa170 2024-04-26T21:43:15.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.075+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772f69e0 2024-04-26T21:43:15.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.076+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v102'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.076+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772f69e0 2024-04-26T21:43:15.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.077+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v102'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.077+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772f69e0 2024-04-26T21:43:15.078 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:43:15.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.079+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v102'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.079+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fe110 con 0x55f9772f69e0 2024-04-26T21:43:15.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.080+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v102'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.080+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fe110 con 0x55f9772f69e0 2024-04-26T21:43:15.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.081+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v102'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.081+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772f69e0 2024-04-26T21:43:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.083+0000 7fe30d0ce640 1 -- 172.21.15.40:0/2067337555 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v102'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe30001f040 con 0x55f9772f69e0 2024-04-26T21:43:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.083+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.084+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v102'32 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.084+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.085+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v102'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.085+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.086+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.086 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:43:15.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.086+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.087+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.087+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.088+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.088+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.089+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v102'82 uv73 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.089+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.090+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v102'83 uv83 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.090+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.090 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:43:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.091+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v102'84 uv84 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.091+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.092+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v102'85 uv85 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.092+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.093+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v102'86 uv86 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.093+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.094+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v102'87 uv80 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.094+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.095+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v102'88 uv88 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.095+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.096+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v102'89 uv89 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.096 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:43:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.096+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.097+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v102'90 uv90 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.097+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fe110 con 0x55f9772ee710 2024-04-26T21:43:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.098+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v102'91 uv91 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.098+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd100 con 0x55f9772ee710 2024-04-26T21:43:15.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.099+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v102'92 uv92 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.099+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f9772fd4c0 con 0x55f9772ee710 2024-04-26T21:43:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30c8cd640 1 -- 172.21.15.40:0/2067337555 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v102'93 uv93 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe2f8029030 con 0x55f9772ee710 2024-04-26T21:43:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f9772e9f20 msgr2=0x55f9772ec310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f9772e9f20 0x55f9772ec310 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f977243940 msgr2=0x55f977199cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f977243940 0x55f977199cf0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f9772f2ca0 msgr2=0x55f9772f5b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f9772f2ca0 0x55f9772f5b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f9772fa170 msgr2=0x55f9772fcb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f9772fa170 0x55f9772fcb70 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f9772ee710 msgr2=0x55f9772f0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f9772ee710 0x55f9772f0b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f9772f69e0 msgr2=0x55f9772f8dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f9772f69e0 0x55f9772f8dd0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 reap_dead start 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30d8cf640 1 -- 172.21.15.40:0/2067337555 reap_dead start 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe2f006b300 msgr2=0x7fe2f006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe2f006b300 0x7fe2f006d7c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe300000f80 tx=0x7fe300006a20 comp rx=0 tx=0).stop 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 msgr2=0x55f9771b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f9771b1f50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x55f977240e70 tx=0x7fe2f8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 shutdown_connections 2024-04-26T21:43:15.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe2f006b300 0x7fe2f006d7c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 --2- 172.21.15.40:0/2067337555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f97719caf0 0x55f9771b1f50 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.100+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 >> 172.21.15.40:0/2067337555 conn(0x55f97716d330 msgr2=0x55f977198b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.101+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 shutdown_connections 2024-04-26T21:43:15.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.101+0000 7fe30f5d4f40 1 -- 172.21.15.40:0/2067337555 wait complete. 2024-04-26T21:43:15.104 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 500.00000000 2024-04-26T21:43:15.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.146+0000 7fc5ded28f40 1 Processor -- start 2024-04-26T21:43:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.146+0000 7fc5ded28f40 1 -- start start 2024-04-26T21:43:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.146+0000 7fc5ded28f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 0x560afd2f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5ded28f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 0x560afd2e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5ded28f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd25d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5ded28f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560afd2614b0 con 0x560afd2ef6e0 2024-04-26T21:43:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5ded28f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560afd1bfe40 con 0x560afd261940 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5ded28f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560afd2479c0 con 0x560afd216220 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd25d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd25d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60470/0 (socket says 172.21.15.40:60470) 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 -- 172.21.15.40:0/2052246227 learned_addr learned my addr 172.21.15.40:0/2052246227 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dbdbd640 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 0x560afd2f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5db5bc640 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 0x560afd2e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 -- 172.21.15.40:0/2052246227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 msgr2=0x560afd2f07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 0x560afd2f07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 -- 172.21.15.40:0/2052246227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 msgr2=0x560afd2e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 0x560afd2e6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 -- 172.21.15.40:0/2052246227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afd25df80 con 0x560afd2ef6e0 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5dc5be640 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd25d170 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc025ab0 tx=0x7fc5cc050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fded8f99fae350 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5da5ba640 1 -- 172.21.15.40:0/2052246227 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc055030 con 0x560afd2ef6e0 2024-04-26T21:43:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5da5ba640 1 -- 172.21.15.40:0/2052246227 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc051bb0 con 0x560afd2ef6e0 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.147+0000 7fc5da5ba640 1 -- 172.21.15.40:0/2052246227 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc05b430 con 0x560afd2ef6e0 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 -- 172.21.15.40:0/2052246227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 msgr2=0x560afd25d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd25d170 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc025ab0 tx=0x7fc5cc050000 comp rx=0 tx=0).stop 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 -- 172.21.15.40:0/2052246227 shutdown_connections 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd25d170 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 0x560afd2e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 --2- 172.21.15.40:0/2052246227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 0x560afd2f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 -- 172.21.15.40:0/2052246227 >> 172.21.15.40:0/2052246227 conn(0x560afd217330 msgr2=0x560afd25bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 -- 172.21.15.40:0/2052246227 shutdown_connections 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 -- 172.21.15.40:0/2052246227 wait complete. 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.148+0000 7fc5ded28f40 1 Processor -- start 2024-04-26T21:43:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5ded28f40 1 -- start start 2024-04-26T21:43:15.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5ded28f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 0x560afd2e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5ded28f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 0x560afd380950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5ded28f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd389660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5ded28f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560afd2effe0 con 0x560afd2ef6e0 2024-04-26T21:43:15.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5ded28f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560afd25e260 con 0x560afd261940 2024-04-26T21:43:15.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5ded28f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560afd2f02c0 con 0x560afd216220 2024-04-26T21:43:15.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5dc5be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd389660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5dc5be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd389660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60472/0 (socket says 172.21.15.40:60472) 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5dc5be640 1 -- 172.21.15.40:0/4289387683 learned_addr learned my addr 172.21.15.40:0/4289387683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5dbdbd640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 0x560afd2e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.149+0000 7fc5db5bc640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 0x560afd380950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5dc5be640 1 -- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 msgr2=0x560afd2e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5dc5be640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560afd216220 0x560afd2e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5dc5be640 1 -- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 msgr2=0x560afd380950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5dc5be640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560afd261940 0x560afd380950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5dc5be640 1 -- 172.21.15.40:0/4289387683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afd2f1590 con 0x560afd2ef6e0 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5dc5be640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd389660 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0460b0 tx=0x7fc5cc003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5d95b8640 1 -- 172.21.15.40:0/4289387683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc051d60 con 0x560afd2ef6e0 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5d95b8640 1 -- 172.21.15.40:0/4289387683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc067110 con 0x560afd2ef6e0 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5d95b8640 1 -- 172.21.15.40:0/4289387683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc067b70 con 0x560afd2ef6e0 2024-04-26T21:43:15.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560afd244120 con 0x560afd2ef6e0 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.150+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560afd2f1a70 con 0x560afd2ef6e0 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.151+0000 7fc5d95b8640 1 -- 172.21.15.40:0/4289387683 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5cc058020 con 0x560afd2ef6e0 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.152+0000 7fc5d95b8640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5bc06b2d0 0x7fc5bc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.152+0000 7fc5d95b8640 1 -- 172.21.15.40:0/4289387683 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7fc5cc0cfe30 con 0x560afd2ef6e0 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.152+0000 7fc5dbdbd640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5bc06b2d0 0x7fc5bc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.152+0000 7fc5ded28f40 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560afd380e90 0x560afd38a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.152+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x560afd390920 con 0x560afd380e90 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.152+0000 7fc5db5bc640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560afd380e90 0x560afd38a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.153+0000 7fc5dbdbd640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5bc06b2d0 0x7fc5bc06d790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x560afd21b4a0 tx=0x7fc5d400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.153+0000 7fc5db5bc640 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560afd380e90 0x560afd38a0b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5db5bc640 1 -- 172.21.15.40:0/4289387683 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 500.00000000 [delete] v102'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5c4011870 con 0x560afd380e90 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560afd380e90 msgr2=0x560afd38a0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560afd380e90 0x560afd38a0b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5bc06b2d0 msgr2=0x7fc5bc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc5bc06b2d0 0x7fc5bc06d790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x560afd21b4a0 tx=0x7fc5d400c040 comp rx=0 tx=0).stop 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 msgr2=0x560afd389660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 --2- 172.21.15.40:0/4289387683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560afd2ef6e0 0x560afd389660 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0460b0 tx=0x7fc5cc003330 comp rx=0 tx=0).stop 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5dc5be640 1 -- 172.21.15.40:0/4289387683 reap_dead start 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 shutdown_connections 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 >> 172.21.15.40:0/4289387683 conn(0x560afd217330 msgr2=0x560afd25c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 shutdown_connections 2024-04-26T21:43:15.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.154+0000 7fc5ded28f40 1 -- 172.21.15.40:0/4289387683 wait complete. 2024-04-26T21:43:15.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:43:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.303+0000 7f57cf359640 1 Processor -- start 2024-04-26T21:43:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cf359640 1 -- start start 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cf359640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 0x7f57c80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cf359640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57c80d2320 0x7f57c81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cf359640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f57c80b40d0 0x7f57c8162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cf359640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f57c8163070 con 0x7f57c80d2320 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cf359640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f57c80d0a10 con 0x7f57c80b40d0 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cf359640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57c80d0cf0 con 0x7f57c80d1a00 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cc8f4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57c80d2320 0x7f57c81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cc8f4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57c80d2320 0x7f57c81625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60476/0 (socket says 172.21.15.40:60476) 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.304+0000 7f57cc8f4640 1 -- 172.21.15.40:0/3330793294 learned_addr learned my addr 172.21.15.40:0/3330793294 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:15.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.305+0000 7f57cd0f5640 1 --2- 172.21.15.40:0/3330793294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 0x7f57c80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.305+0000 7f57cd8f6640 1 --2- 172.21.15.40:0/3330793294 >> v2:172.21.15.184:3300/0 conn(0x7f57c80b40d0 0x7f57c8162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.305+0000 7f57cc8f4640 1 -- 172.21.15.40:0/3330793294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 msgr2=0x7f57c80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.305+0000 7f57cc8f4640 1 --2- 172.21.15.40:0/3330793294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 0x7f57c80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.305+0000 7f57cc8f4640 1 -- 172.21.15.40:0/3330793294 >> v2:172.21.15.184:3300/0 conn(0x7f57c80b40d0 msgr2=0x7f57c8162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.305+0000 7f57cc8f4640 1 --2- 172.21.15.40:0/3330793294 >> v2:172.21.15.184:3300/0 conn(0x7f57c80b40d0 0x7f57c8162b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.305+0000 7f57cc8f4640 1 -- 172.21.15.40:0/3330793294 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c80d0fd0 con 0x7f57c80d2320 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.305+0000 7f57cc8f4640 1 --2- 172.21.15.40:0/3330793294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c81625f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f57c808e430 tx=0x7f57bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbb01231fa4f4e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57c7fff640 1 -- 172.21.15.40:0/3330793294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc037070 con 0x7f57c80d2320 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57c7fff640 1 -- 172.21.15.40:0/3330793294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57bc032440 con 0x7f57c80d2320 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57c7fff640 1 -- 172.21.15.40:0/3330793294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc032810 con 0x7f57c80d2320 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 -- 172.21.15.40:0/3330793294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 msgr2=0x7f57c81625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 --2- 172.21.15.40:0/3330793294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c81625f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f57c808e430 tx=0x7f57bc02a140 comp rx=0 tx=0).stop 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 -- 172.21.15.40:0/3330793294 shutdown_connections 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 --2- 172.21.15.40:0/3330793294 >> v2:172.21.15.184:3300/0 conn(0x7f57c80b40d0 0x7f57c8162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 --2- 172.21.15.40:0/3330793294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c81625f0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 --2- 172.21.15.40:0/3330793294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 0x7f57c80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 -- 172.21.15.40:0/3330793294 >> 172.21.15.40:0/3330793294 conn(0x7f57c80d4fd0 msgr2=0x7f57c80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 -- 172.21.15.40:0/3330793294 shutdown_connections 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.306+0000 7f57cf359640 1 -- 172.21.15.40:0/3330793294 wait complete. 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.307+0000 7f57cf359640 1 Processor -- start 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cf359640 1 -- start start 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cf359640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57c80b40d0 0x7f57c815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cf359640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 0x7f57c815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cf359640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c80ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cf359640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57c80d3050 con 0x7f57c80d2320 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cf359640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57c80d2ed0 con 0x7f57c80b40d0 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cf359640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57c80d31d0 con 0x7f57c80d1a00 2024-04-26T21:43:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cd8f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c80ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cd8f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c80ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60482/0 (socket says 172.21.15.40:60482) 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.308+0000 7f57cd8f6640 1 -- 172.21.15.40:0/3841179904 learned_addr learned my addr 172.21.15.40:0/3841179904 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cc8f4640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 0x7f57c815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cd8f6640 1 -- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 msgr2=0x7f57c815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cd8f6640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 0x7f57c815d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cd8f6640 1 -- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57c80b40d0 msgr2=0x7f57c815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cd0f5640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57c80b40d0 0x7f57c815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cd8f6640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57c80b40d0 0x7f57c815b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cd8f6640 1 -- 172.21.15.40:0/3841179904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c815be60 con 0x7f57c80d2320 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cd0f5640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57c80b40d0 0x7f57c815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cd8f6640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c80ccab0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f57b8007e60 tx=0x7f57b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57c5ffb640 1 -- 172.21.15.40:0/3841179904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8014620 con 0x7f57c80d2320 2024-04-26T21:43:15.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57c5ffb640 1 -- 172.21.15.40:0/3841179904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b8019070 con 0x7f57c80d2320 2024-04-26T21:43:15.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c80cd490 con 0x7f57c80d2320 2024-04-26T21:43:15.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57c5ffb640 1 -- 172.21.15.40:0/3841179904 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8014a20 con 0x7f57c80d2320 2024-04-26T21:43:15.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.309+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c80b8120 con 0x7f57c80d2320 2024-04-26T21:43:15.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.310+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c8075d40 con 0x7f57c80d2320 2024-04-26T21:43:15.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.311+0000 7f57c5ffb640 1 -- 172.21.15.40:0/3841179904 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57b8014bc0 con 0x7f57c80d2320 2024-04-26T21:43:15.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.311+0000 7f57c5ffb640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57b406c8f0 0x7f57b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.311+0000 7f57c5ffb640 1 -- 172.21.15.40:0/3841179904 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f57b808c770 con 0x7f57c80d2320 2024-04-26T21:43:15.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.311+0000 7f57cd0f5640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57b406c8f0 0x7f57b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.312+0000 7f57cd0f5640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57b406c8f0 0x7f57b406edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f57c0006fd0 tx=0x7f57c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.315+0000 7f57c5ffb640 1 -- 172.21.15.40:0/3841179904 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57c8075d40 con 0x7f57c80d2320 2024-04-26T21:43:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.544+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f57c80694f0 con 0x7f57c80d2320 2024-04-26T21:43:15.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.648+0000 7f57c5ffb640 1 -- 172.21.15.40:0/3841179904 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v132) ==== 162+0+0 (secure 0 0 0) 0x7f57b805a7d0 con 0x7f57c80d2320 2024-04-26T21:43:15.648 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57b406c8f0 msgr2=0x7f57b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57b406c8f0 0x7f57b406edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f57c0006fd0 tx=0x7f57c0008040 comp rx=0 tx=0).stop 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 msgr2=0x7f57c80ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c80ccab0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f57b8007e60 tx=0x7f57b800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 shutdown_connections 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f57b406c8f0 0x7f57b406edb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57c80d2320 0x7f57c80ccab0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57c80d1a00 0x7f57c815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 --2- 172.21.15.40:0/3841179904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57c80b40d0 0x7f57c815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 >> 172.21.15.40:0/3841179904 conn(0x7f57c80d4fd0 msgr2=0x7f57c806d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 shutdown_connections 2024-04-26T21:43:15.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:15.651+0000 7f57cf359640 1 -- 172.21.15.40:0/3841179904 wait complete. 2024-04-26T21:43:16.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.172+0000 7f31a41e6640 1 Processor -- start 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.172+0000 7f31a41e6640 1 -- start start 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a41e6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319c162810 0x7f319c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a41e6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f319c0d1b30 0x7f319c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a41e6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f319c0d10d0 con 0x7f319c0b2340 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a41e6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f319c0d0750 con 0x7f319c0d1b30 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a41e6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f319c0d16a0 con 0x7f319c162810 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a1781640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f319c0b2340 0x7f319c0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42036/0 (socket says 172.21.15.40:42036) 2024-04-26T21:43:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a1781640 1 -- 172.21.15.40:0/577382757 learned_addr learned my addr 172.21.15.40:0/577382757 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a1f82640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319c162810 0x7f319c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a2783640 1 --2- 172.21.15.40:0/577382757 >> v2:172.21.15.184:3300/0 conn(0x7f319c0d1b30 0x7f319c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4240603542 0 0) 0x7f319c0d10d0 con 0x7f319c0b2340 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.173+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3188003520 con 0x7f319c0b2340 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3332375167 0 0) 0x7f3188003520 con 0x7f319c0b2340 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319c162810 msgr2=0x7f319c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319c162810 0x7f319c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 >> v2:172.21.15.184:3300/0 conn(0x7f319c0d1b30 msgr2=0x7f319c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 --2- 172.21.15.40:0/577382757 >> v2:172.21.15.184:3300/0 conn(0x7f319c0d1b30 0x7f319c0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c0d1370 con 0x7f319c0b2340 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a2783640 1 --2- 172.21.15.40:0/577382757 >> v2:172.21.15.184:3300/0 conn(0x7f319c0d1b30 0x7f319c0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f318c001670 con 0x7f319c0b2340 2024-04-26T21:43:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 >> v1:172.21.15.40:6789/0 conn(0x7f319c0b2340 legacy=0x7f319c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31880039b0 0x7f3188003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3188004800 0x7f3188004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319c162810 0x7f31880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f319c0d0750 con 0x7f3188004800 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f319c0d16a0 con 0x7f31880039b0 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f318800b9c0 con 0x7f319c162810 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a1781640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3188004800 0x7f3188004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a1f82640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31880039b0 0x7f3188003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a2783640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319c162810 0x7f31880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a1781640 1 -- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319c162810 msgr2=0x7f31880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a1781640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f319c162810 0x7f31880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a1781640 1 -- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31880039b0 msgr2=0x7f3188003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a1781640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31880039b0 0x7f3188003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.174+0000 7f31a1781640 1 -- 172.21.15.40:0/577382757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3188003520 con 0x7f3188004800 2024-04-26T21:43:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.175+0000 7f31a1781640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3188004800 0x7f3188004c80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f318c007fd0 tx=0x7f3188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21728559458d134 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.175+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c007d50 con 0x7f3188004800 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.175+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c005ad0 con 0x7f3188004800 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.175+0000 7f31a0f80640 1 -- 172.21.15.40:0/577382757 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c007070 con 0x7f3188004800 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.176+0000 7f31a41e6640 1 -- 172.21.15.40:0/577382757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3188004800 msgr2=0x7f3188004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.176+0000 7f31a41e6640 1 --2- 172.21.15.40:0/577382757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3188004800 0x7f3188004c80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f318c007fd0 tx=0x7f3188005830 comp rx=0 tx=0).stop 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.176+0000 7f31a2783640 1 -- 172.21.15.40:0/577382757 reap_dead start 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.176+0000 7f31a41e6640 1 -- 172.21.15.40:0/577382757 shutdown_connections 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.176+0000 7f31a41e6640 1 -- 172.21.15.40:0/577382757 >> 172.21.15.40:0/577382757 conn(0x7f319c0d2da0 msgr2=0x7f319c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.176+0000 7f31a41e6640 1 -- 172.21.15.40:0/577382757 shutdown_connections 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.176+0000 7f31a41e6640 1 -- 172.21.15.40:0/577382757 wait complete. 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.177+0000 7f31a41e6640 1 Processor -- start 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.177+0000 7f31a41e6640 1 -- start start 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a41e6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3188004800 0x7f319c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a41e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c151060 0x7f319c15d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a41e6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c1514c0 0x7f319c15d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a41e6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f319c0cdb00 con 0x7f319c151060 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a41e6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f319c0d0470 con 0x7f319c1514c0 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a41e6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f319c070350 con 0x7f3188004800 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c151060 0x7f319c15d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c151060 0x7f319c15d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60510/0 (socket says 172.21.15.40:60510) 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 -- 172.21.15.40:0/1238643574 learned_addr learned my addr 172.21.15.40:0/1238643574 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1f82640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3188004800 0x7f319c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a2783640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c1514c0 0x7f319c15d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 -- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3188004800 msgr2=0x7f319c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3188004800 0x7f319c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 -- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c1514c0 msgr2=0x7f319c15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c1514c0 0x7f319c15d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 -- 172.21.15.40:0/1238643574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c1f5960 con 0x7f319c151060 2024-04-26T21:43:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a1781640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c151060 0x7f319c15d0e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f318c00c300 tx=0x7f319c0d1610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31927fc640 1 -- 172.21.15.40:0/1238643574 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c042070 con 0x7f319c151060 2024-04-26T21:43:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31927fc640 1 -- 172.21.15.40:0/1238643574 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c007630 con 0x7f319c151060 2024-04-26T21:43:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c0c9fa0 con 0x7f319c151060 2024-04-26T21:43:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.178+0000 7f31927fc640 1 -- 172.21.15.40:0/1238643574 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c007d50 con 0x7f319c151060 2024-04-26T21:43:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.179+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c0cafc0 con 0x7f319c151060 2024-04-26T21:43:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.180+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319c0694f0 con 0x7f319c151060 2024-04-26T21:43:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.180+0000 7f31927fc640 1 -- 172.21.15.40:0/1238643574 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f318c045070 con 0x7f319c151060 2024-04-26T21:43:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.180+0000 7f31927fc640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318006c8f0 0x7f318006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.180+0000 7f31927fc640 1 -- 172.21.15.40:0/1238643574 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f318c0acc30 con 0x7f319c151060 2024-04-26T21:43:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.181+0000 7f31a1f82640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318006c8f0 0x7f318006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.181+0000 7f31a1f82640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318006c8f0 0x7f318006edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3198009a60 tx=0x7f3198009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.185+0000 7f31927fc640 1 -- 172.21.15.40:0/1238643574 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f318c04c030 con 0x7f319c151060 2024-04-26T21:43:16.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.408+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f319c0d3910 con 0x7f319c151060 2024-04-26T21:43:16.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.409+0000 7f31927fc640 1 -- 172.21.15.40:0/1238643574 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) ==== 95+0+5190 (secure 0 0 0) 0x7f318c07adb0 con 0x7f319c151060 2024-04-26T21:43:16.409 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:16.409 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":132,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:15.647749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8898},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8898":{"gid":8898,"name":"c","rank":0,"incarnation":132,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/3114124794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3114124794},{"type":"v1","addr":"172.21.15.40:6833","nonce":3114124794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:16.409 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 132 2024-04-26T21:43:16.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318006c8f0 msgr2=0x7f318006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318006c8f0 0x7f318006edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3198009a60 tx=0x7f3198009190 comp rx=0 tx=0).stop 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c151060 msgr2=0x7f319c15d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c151060 0x7f319c15d0e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f318c00c300 tx=0x7f319c0d1610 comp rx=0 tx=0).stop 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 shutdown_connections 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f318006c8f0 0x7f318006edb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c1514c0 0x7f319c15d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c151060 0x7f319c15d0e0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 --2- 172.21.15.40:0/1238643574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3188004800 0x7f319c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 >> 172.21.15.40:0/1238643574 conn(0x7f319c0d2da0 msgr2=0x7f319c0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 shutdown_connections 2024-04-26T21:43:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:16.411+0000 7f31a41e6640 1 -- 172.21.15.40:0/1238643574 wait complete. 2024-04-26T21:43:16.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:43:15.647749+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8898}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8898': {'gid': 8898, 'name': 'c', 'rank': 0, 'incarnation': 132, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/3114124794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3114124794}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3114124794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:43:16.423 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8898:up:replay 2024-04-26T21:43:16.423 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:43:19.842 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:19.842+0000 7fa24d98b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:43:21.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.941+0000 7fac09b2b640 1 Processor -- start 2024-04-26T21:43:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.941+0000 7fac09b2b640 1 -- start start 2024-04-26T21:43:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac09b2b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fac04162880 0x7fac040d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac09b2b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac040b5f20 0x7fac040d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac09b2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac040d39f0 0x7fac040b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac09b2b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fac040d0580 con 0x7fac040b5f20 2024-04-26T21:43:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac09b2b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fac040b7e50 con 0x7fac04162880 2024-04-26T21:43:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac09b2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac040d1800 con 0x7fac040d39f0 2024-04-26T21:43:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac02ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac040b5f20 0x7fac040d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac02ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac040b5f20 0x7fac040d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52072/0 (socket says 172.21.15.40:52072) 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac02ffd640 1 -- 172.21.15.40:0/2374115266 learned_addr learned my addr 172.21.15.40:0/2374115266 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac037fe640 1 --2- 172.21.15.40:0/2374115266 >> v2:172.21.15.184:3300/0 conn(0x7fac04162880 0x7fac040d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac03fff640 1 --2- 172.21.15.40:0/2374115266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac040d39f0 0x7fac040b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac02ffd640 1 -- 172.21.15.40:0/2374115266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac040d39f0 msgr2=0x7fac040b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac02ffd640 1 --2- 172.21.15.40:0/2374115266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac040d39f0 0x7fac040b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac02ffd640 1 -- 172.21.15.40:0/2374115266 >> v2:172.21.15.184:3300/0 conn(0x7fac04162880 msgr2=0x7fac040d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac02ffd640 1 --2- 172.21.15.40:0/2374115266 >> v2:172.21.15.184:3300/0 conn(0x7fac04162880 0x7fac040d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.942+0000 7fac02ffd640 1 -- 172.21.15.40:0/2374115266 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac040d07d0 con 0x7fac040b5f20 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.943+0000 7fac02ffd640 1 --2- 172.21.15.40:0/2374115266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040b5f20 0x7fac040d34b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fac0408d490 tx=0x7fabf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1aeb22921c64f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.943+0000 7fac027fc640 1 -- 172.21.15.40:0/2374115266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8037070 con 0x7fac040b5f20 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.943+0000 7fac027fc640 1 -- 172.21.15.40:0/2374115266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf8032440 con 0x7fac040b5f20 2024-04-26T21:43:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.943+0000 7fac027fc640 1 -- 172.21.15.40:0/2374115266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8032810 con 0x7fac040b5f20 2024-04-26T21:43:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 -- 172.21.15.40:0/2374115266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040b5f20 msgr2=0x7fac040d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2374115266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040b5f20 0x7fac040d34b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fac0408d490 tx=0x7fabf802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:43:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 -- 172.21.15.40:0/2374115266 shutdown_connections 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2374115266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac040d39f0 0x7fac040b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2374115266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040b5f20 0x7fac040d34b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2374115266 >> v2:172.21.15.184:3300/0 conn(0x7fac04162880 0x7fac040d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 -- 172.21.15.40:0/2374115266 >> 172.21.15.40:0/2374115266 conn(0x7fac040d4f60 msgr2=0x7fac040d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 -- 172.21.15.40:0/2374115266 shutdown_connections 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.944+0000 7fac09b2b640 1 -- 172.21.15.40:0/2374115266 wait complete. 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.945+0000 7fac09b2b640 1 Processor -- start 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.945+0000 7fac09b2b640 1 -- start start 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac09b2b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac040b5f20 0x7fac040a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac09b2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040d39f0 0x7fac04151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac09b2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac04162880 0x7fac040a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac09b2b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fac04162c60 con 0x7fac040d39f0 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac09b2b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac040d0d90 con 0x7fac040b5f20 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac09b2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac04162de0 con 0x7fac04162880 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac02ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040d39f0 0x7fac04151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac02ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040d39f0 0x7fac04151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52084/0 (socket says 172.21.15.40:52084) 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac02ffd640 1 -- 172.21.15.40:0/2635523033 learned_addr learned my addr 172.21.15.40:0/2635523033 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac03fff640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac04162880 0x7fac040a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.946+0000 7fac037fe640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac040b5f20 0x7fac040a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac02ffd640 1 -- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac04162880 msgr2=0x7fac040a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac02ffd640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac04162880 0x7fac040a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac02ffd640 1 -- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac040b5f20 msgr2=0x7fac040a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac02ffd640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac040b5f20 0x7fac040a27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac02ffd640 1 -- 172.21.15.40:0/2635523033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac040a1700 con 0x7fac040d39f0 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac03fff640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac04162880 0x7fac040a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac02ffd640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040d39f0 0x7fac04151190 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fabf80097c0 tx=0x7fabf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fabebfff640 1 -- 172.21.15.40:0/2635523033 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8041020 con 0x7fac040d39f0 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fabebfff640 1 -- 172.21.15.40:0/2635523033 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf80306c0 con 0x7fac040d39f0 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac040a19e0 con 0x7fac040d39f0 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fabebfff640 1 -- 172.21.15.40:0/2635523033 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8037040 con 0x7fac040d39f0 2024-04-26T21:43:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.947+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac040a1fa0 con 0x7fac040d39f0 2024-04-26T21:43:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.948+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac040694f0 con 0x7fac040d39f0 2024-04-26T21:43:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.948+0000 7fabebfff640 1 -- 172.21.15.40:0/2635523033 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabf8044020 con 0x7fac040d39f0 2024-04-26T21:43:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.949+0000 7fabebfff640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fabf006c8f0 0x7fabf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.949+0000 7fabebfff640 1 -- 172.21.15.40:0/2635523033 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7fabf80ac7b0 con 0x7fac040d39f0 2024-04-26T21:43:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.949+0000 7fac037fe640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fabf006c8f0 0x7fabf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.950+0000 7fac037fe640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fabf006c8f0 0x7fabf006edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fabf4006fd0 tx=0x7fabf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:21.953+0000 7fabebfff640 1 -- 172.21.15.40:0/2635523033 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabf804b030 con 0x7fac040d39f0 2024-04-26T21:43:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.178+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac040cffa0 con 0x7fac040d39f0 2024-04-26T21:43:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.179+0000 7fabebfff640 1 -- 172.21.15.40:0/2635523033 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+5197 (secure 0 0 0) 0x7fabf807a780 con 0x7fac040d39f0 2024-04-26T21:43:22.179 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:22.179 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":135,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:18.651869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8898},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8898":{"gid":8898,"name":"c","rank":0,"incarnation":132,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/3114124794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3114124794},{"type":"v1","addr":"172.21.15.40:6833","nonce":3114124794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8898,"qdb_cluster":[8898]},"id":6}]} 2024-04-26T21:43:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 135 2024-04-26T21:43:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fabf006c8f0 msgr2=0x7fabf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fabf006c8f0 0x7fabf006edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fabf4006fd0 tx=0x7fabf4008040 comp rx=0 tx=0).stop 2024-04-26T21:43:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040d39f0 msgr2=0x7fac04151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040d39f0 0x7fac04151190 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fabf80097c0 tx=0x7fabf80046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 shutdown_connections 2024-04-26T21:43:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fabf006c8f0 0x7fabf006edb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac04162880 0x7fac040a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac040d39f0 0x7fac04151190 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 --2- 172.21.15.40:0/2635523033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac040b5f20 0x7fac040a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 >> 172.21.15.40:0/2635523033 conn(0x7fac040d4f60 msgr2=0x7fac0406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 shutdown_connections 2024-04-26T21:43:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.181+0000 7fac09b2b640 1 -- 172.21.15.40:0/2635523033 wait complete. 2024-04-26T21:43:22.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 135, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:43:18.651869+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8898}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8898': {'gid': 8898, 'name': 'c', 'rank': 0, 'incarnation': 132, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6833/3114124794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3114124794}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3114124794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8898, 'qdb_cluster': [8898]} 2024-04-26T21:43:22.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:43:22.192 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:43:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.294+0000 7f7656ed1640 1 Processor -- start 2024-04-26T21:43:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.294+0000 7f7656ed1640 1 -- start start 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.295+0000 7f7656ed1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f76500d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.295+0000 7f7656ed1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76500d2420 con 0x7f765006e270 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.295+0000 7f7656ed1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76500cd6b0 con 0x7f76500d0070 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.295+0000 7f7656ed1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7650162d00 con 0x7f765006d560 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.295+0000 7f764ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f765006e270 0x7f76500ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60140/0 (socket says 172.21.15.40:60140) 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.295+0000 7f764ffff640 1 -- 172.21.15.40:0/541049538 learned_addr learned my addr 172.21.15.40:0/541049538 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.296+0000 7f765546e640 1 --2- 172.21.15.40:0/541049538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f76500d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.296+0000 7f764f7fe640 1 -- 172.21.15.40:0/541049538 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3337098267 0 0) 0x7f76500d2420 con 0x7f765006e270 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.296+0000 7f764f7fe640 1 -- 172.21.15.40:0/541049538 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7638003520 con 0x7f765006e270 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.296+0000 7f765546e640 1 -- 172.21.15.40:0/541049538 >> v1:172.21.15.184:6789/0 conn(0x7f76500d0070 legacy=0x7f765006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.296+0000 7f765546e640 1 -- 172.21.15.40:0/541049538 >> v1:172.21.15.40:6789/0 conn(0x7f765006e270 legacy=0x7f76500ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.296+0000 7f764f7fe640 1 -- 172.21.15.40:0/541049538 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4273340592 0 0) 0x7f7638003520 con 0x7f765006e270 2024-04-26T21:43:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.296+0000 7f765546e640 1 -- 172.21.15.40:0/541049538 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76500cd990 con 0x7f765006d560 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f765546e640 1 --2- 172.21.15.40:0/541049538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f76500d1d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7648009dc0 tx=0x7f764802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5b68d8bf1375ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f764f7fe640 1 -- 172.21.15.40:0/541049538 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644001670 con 0x7f765006d560 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f764f7fe640 1 -- 172.21.15.40:0/541049538 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764802ae50 con 0x7f765006d560 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f764f7fe640 1 -- 172.21.15.40:0/541049538 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648007760 con 0x7f765006d560 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f7656ed1640 1 -- 172.21.15.40:0/541049538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 msgr2=0x7f76500d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f7656ed1640 1 --2- 172.21.15.40:0/541049538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f76500d1d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7648009dc0 tx=0x7f764802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f7656ed1640 1 -- 172.21.15.40:0/541049538 shutdown_connections 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f7656ed1640 1 --2- 172.21.15.40:0/541049538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f76500d1d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f7656ed1640 1 -- 172.21.15.40:0/541049538 >> 172.21.15.40:0/541049538 conn(0x7f76500d2ec0 msgr2=0x7f76500d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.297+0000 7f7656ed1640 1 -- 172.21.15.40:0/541049538 shutdown_connections 2024-04-26T21:43:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.298+0000 7f7656ed1640 1 -- 172.21.15.40:0/541049538 wait complete. 2024-04-26T21:43:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.299+0000 7f7656ed1640 1 Processor -- start 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.299+0000 7f7656ed1640 1 -- start start 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.299+0000 7f7656ed1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f765009cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.299+0000 7f7656ed1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765006e270 0x7f765014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.299+0000 7f7656ed1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76500d0070 0x7f765014e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.299+0000 7f7656ed1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f76500ce8a0 con 0x7f765006e270 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.299+0000 7f7656ed1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76500d2d10 con 0x7f76500d0070 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.299+0000 7f7656ed1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76500cea20 con 0x7f765006d560 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765006e270 0x7f765014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765006e270 0x7f765014f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52096/0 (socket says 172.21.15.40:52096) 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 -- 172.21.15.40:0/3567173679 learned_addr learned my addr 172.21.15.40:0/3567173679 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f7654c6d640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f765009cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f765546e640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76500d0070 0x7f765014e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 -- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 msgr2=0x7f765009cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f765009cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 -- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76500d0070 msgr2=0x7f765014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76500d0070 0x7f765014e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 -- 172.21.15.40:0/3567173679 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765014d340 con 0x7f765006e270 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f765546e640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76500d0070 0x7f765014e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764ffff640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765006e270 0x7f765014f040 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f764400c780 tx=0x7f764400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764d7fa640 1 -- 172.21.15.40:0/3567173679 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644017070 con 0x7f765006e270 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764d7fa640 1 -- 172.21.15.40:0/3567173679 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7644005ad0 con 0x7f765006e270 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f7656ed1640 1 -- 172.21.15.40:0/3567173679 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765014d620 con 0x7f765006e270 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f764d7fa640 1 -- 172.21.15.40:0/3567173679 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644007070 con 0x7f765006e270 2024-04-26T21:43:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.300+0000 7f7656ed1640 1 -- 172.21.15.40:0/3567173679 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765014dbe0 con 0x7f765006e270 2024-04-26T21:43:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.302+0000 7f764d7fa640 1 -- 172.21.15.40:0/3567173679 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f764401a070 con 0x7f765006e270 2024-04-26T21:43:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.302+0000 7f764d7fa640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f763c06c8f0 0x7f763c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.302+0000 7f764d7fa640 1 -- 172.21.15.40:0/3567173679 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f764408b3e0 con 0x7f765006e270 2024-04-26T21:43:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.303+0000 7f7654c6d640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f763c06c8f0 0x7f763c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.303+0000 7f7654c6d640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f763c06c8f0 0x7f763c06edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f76500c97d0 tx=0x7f7640009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.303+0000 7f7659bef740 1 Processor -- start 2024-04-26T21:43:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.303+0000 7f7659bef740 1 -- start start 2024-04-26T21:43:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f7659bef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c930440 0x56135c930820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f7659bef740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c94dce0 0x56135c94e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f7659bef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c906120 0x56135c94c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f7659bef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56135c86a740 con 0x56135c906120 2024-04-26T21:43:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f7659bef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56135c8063a0 con 0x56135c930440 2024-04-26T21:43:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f7659bef740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56135c866700 con 0x56135c94dce0 2024-04-26T21:43:22.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f765546e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c906120 0x56135c94c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f765546e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c906120 0x56135c94c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52112/0 (socket says 172.21.15.40:52112) 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f765546e640 1 -- 172.21.15.40:0/3706725687 learned_addr learned my addr 172.21.15.40:0/3706725687 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f764ffff640 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c94dce0 0x56135c94e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f7654c6d640 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c930440 0x56135c930820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f765546e640 1 -- 172.21.15.40:0/3706725687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c94dce0 msgr2=0x56135c94e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.304+0000 7f765546e640 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c94dce0 0x56135c94e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f765546e640 1 -- 172.21.15.40:0/3706725687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c930440 msgr2=0x56135c930820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f765546e640 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c930440 0x56135c930820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f765546e640 1 -- 172.21.15.40:0/3706725687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135c822750 con 0x56135c906120 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f765546e640 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c906120 0x56135c94c3a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f7648009ee0 tx=0x7f764802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611066c0f353e0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f76367fc640 1 -- 172.21.15.40:0/3706725687 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648007740 con 0x56135c906120 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f76367fc640 1 -- 172.21.15.40:0/3706725687 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76480078e0 con 0x56135c906120 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f76367fc640 1 -- 172.21.15.40:0/3706725687 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648007c90 con 0x56135c906120 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f7659bef740 1 -- 172.21.15.40:0/3706725687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c906120 msgr2=0x56135c94c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f7659bef740 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c906120 0x56135c94c3a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f7648009ee0 tx=0x7f764802e000 comp rx=0 tx=0).stop 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.305+0000 7f7659bef740 1 -- 172.21.15.40:0/3706725687 shutdown_connections 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.306+0000 7f7659bef740 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c906120 0x56135c94c3a0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.306+0000 7f7659bef740 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c94dce0 0x56135c94e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.306+0000 7f7659bef740 1 --2- 172.21.15.40:0/3706725687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c930440 0x56135c930820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.306+0000 7f7659bef740 1 -- 172.21.15.40:0/3706725687 >> 172.21.15.40:0/3706725687 conn(0x56135c902a50 msgr2=0x56135c94b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.306+0000 7f7659bef740 1 -- 172.21.15.40:0/3706725687 shutdown_connections 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.306+0000 7f7659bef740 1 -- 172.21.15.40:0/3706725687 wait complete. 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.306+0000 7f7659bef740 1 -- start start 2024-04-26T21:43:22.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.306+0000 7f7659bef740 1 Processor -- start 2024-04-26T21:43:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f7659bef740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c943920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f7659bef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 0x56135c94c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f7659bef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c94c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f7659bef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56135c906ec0 con 0x56135c91b080 2024-04-26T21:43:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f7659bef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56135c7f1fe0 con 0x56135c94dce0 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f7659bef740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56135c900980 con 0x56135c930440 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f7659bef740 10 client.0 _ll_register_callbacks cb 0x56135c808150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f765546e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c94c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f765546e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c94c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52124/0 (socket says 172.21.15.40:52124) 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f764ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 0x56135c94c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f765546e640 1 -- 172.21.15.40:0/4132951969 learned_addr learned my addr 172.21.15.40:0/4132951969 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.307+0000 7f7654c6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c943920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f765546e640 1 -- 172.21.15.40:0/4132951969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 msgr2=0x56135c943920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f765546e640 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c943920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f765546e640 1 -- 172.21.15.40:0/4132951969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 msgr2=0x56135c94c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f765546e640 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 0x56135c94c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f765546e640 1 -- 172.21.15.40:0/4132951969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135c57c2d0 con 0x56135c91b080 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f765546e640 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c94c710 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f7648009dc0 tx=0x7f7648004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1667d58f9c82e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f762f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:43:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f762f7fe640 1 -- 172.21.15.40:0/4132951969 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648007a90 con 0x56135c91b080 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f762f7fe640 1 -- 172.21.15.40:0/4132951969 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f7648007c30 con 0x56135c91b080 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 1 -- 172.21.15.40:0/4132951969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76480078e0 con 0x56135c91b080 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f762f7fe640 1 -- 172.21.15.40:0/4132951969 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=135) ==== 24+0+0 (secure 0 0 0) 0x7f764805b050 con 0x56135c91b080 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 10 client.8952 fetch_fsmap learned FSMap version 135 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 1 -- 172.21.15.40:0/4132951969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=135,monmap=2+}) -- 0x56135c6466b0 con 0x56135c91b080 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f762f7fe640 1 -- 172.21.15.40:0/4132951969 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76480078e0 con 0x56135c91b080 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f762f7fe640 1 -- 172.21.15.40:0/4132951969 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 135) ==== 3494+0+0 (secure 0 0 0) 0x7f764805e020 con 0x56135c91b080 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 10 client.8952 fetch_fsmap finished waiting for FSMap version 135 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 10 client.8952 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 20 client.8952 populate_metadata read hostname 'smithi040' 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] conn(0x56135c942590 0x56135c942970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 4 client.8952 mds_command: new command op to 8898 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:43:22.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.308+0000 7f7659bef740 1 -- 172.21.15.40:0/4132951969 --> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56135c878350 con 0x56135c942590 2024-04-26T21:43:22.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.309+0000 7f7654c6d640 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] conn(0x56135c942590 0x56135c942970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.309+0000 7f7654c6d640 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] conn(0x56135c942590 0x56135c942970 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29ade72b8990c320 server_cookie=605bb87449e8589d in_seq=0 out_seq=0 2024-04-26T21:43:22.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.309+0000 7f762f7fe640 10 client.8952 ms_handle_connect on v2:172.21.15.40:6832/3114124794 2024-04-26T21:43:22.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f762f7fe640 1 -- 172.21.15.40:0/4132951969 <== mds.0 v2:172.21.15.40:6832/3114124794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56135c878350 con 0x56135c942590 2024-04-26T21:43:22.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f762f7fe640 10 client.8952 handle_command_reply: tid=0 2024-04-26T21:43:22.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 client.8952 shutdown 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 -- 172.21.15.40:0/4132951969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 msgr2=0x56135c94c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c94c710 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f7648009dc0 tx=0x7f7648004ab0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 -- 172.21.15.40:0/4132951969 shutdown_connections 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c94c710 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 0x56135c94c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] conn(0x56135c942590 0x56135c942970 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 --2- 172.21.15.40:0/4132951969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c943920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 -- 172.21.15.40:0/4132951969 >> 172.21.15.40:0/4132951969 conn(0x56135c906120 msgr2=0x56135c94f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f762f7fe640 0 client.8952 ms_handle_reset on v2:172.21.15.40:6832/3114124794 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 -- 172.21.15.40:0/4132951969 shutdown_connections 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 1 -- 172.21.15.40:0/4132951969 wait complete. 2024-04-26T21:43:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.310+0000 7f7659bef740 20 client.8952 trim_cache size 0 max 16384 2024-04-26T21:43:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.328+0000 7f7659bef740 1 Processor -- start 2024-04-26T21:43:22.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.328+0000 7f7659bef740 1 -- start start 2024-04-26T21:43:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7659bef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c944510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7659bef740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c944a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7659bef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 0x56135c9194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7659bef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7650201320 con 0x56135c91b080 2024-04-26T21:43:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7659bef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76500cc130 con 0x56135c94dce0 2024-04-26T21:43:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7659bef740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7650200d00 con 0x56135c930440 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c944510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c944510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52126/0 (socket says 172.21.15.40:52126) 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 -- 172.21.15.40:0/326298512 learned_addr learned my addr 172.21.15.40:0/326298512 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 -- 172.21.15.40:0/326298512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 msgr2=0x56135c944a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 --2- 172.21.15.40:0/326298512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c944a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 -- 172.21.15.40:0/326298512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 msgr2=0x56135c9194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 --2- 172.21.15.40:0/326298512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 0x56135c9194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 -- 172.21.15.40:0/326298512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135c91ae90 con 0x56135c91b080 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7654c6d640 1 --2- 172.21.15.40:0/326298512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c944510 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f764000faa0 tx=0x7f7640012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e3c5b480054e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.329+0000 7f7635ffb640 1 -- 172.21.15.40:0/326298512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400249a0 con 0x56135c91b080 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7635ffb640 1 -- 172.21.15.40:0/326298512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764002f070 con 0x56135c91b080 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7635ffb640 1 -- 172.21.15.40:0/326298512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640024d40 con 0x56135c91b080 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 -- 172.21.15.40:0/326298512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 msgr2=0x56135c944510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 --2- 172.21.15.40:0/326298512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c944510 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f764000faa0 tx=0x7f7640012000 comp rx=0 tx=0).stop 2024-04-26T21:43:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 -- 172.21.15.40:0/326298512 shutdown_connections 2024-04-26T21:43:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 --2- 172.21.15.40:0/326298512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c94dce0 0x56135c9194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 --2- 172.21.15.40:0/326298512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c944a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 --2- 172.21.15.40:0/326298512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c91b080 0x56135c944510 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 -- 172.21.15.40:0/326298512 >> 172.21.15.40:0/326298512 conn(0x56135c906120 msgr2=0x56135c920b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 -- 172.21.15.40:0/326298512 shutdown_connections 2024-04-26T21:43:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.330+0000 7f7659bef740 1 -- 172.21.15.40:0/326298512 wait complete. 2024-04-26T21:43:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.331+0000 7f7659bef740 1 -- start start 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.331+0000 7f7659bef740 1 Processor -- start 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7659bef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c91b080 0x56135c92f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7659bef740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c9c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7659bef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c94dce0 0x56135c944510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7659bef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56135c933b80 con 0x56135c94dce0 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7659bef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56135c931e70 con 0x56135c91b080 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7659bef740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56135c933110 con 0x56135c930440 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7659bef740 10 client.0 _ll_register_callbacks cb 0x7f7644055090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c94dce0 0x56135c944510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c94dce0 0x56135c944510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52130/0 (socket says 172.21.15.40:52130) 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 -- 172.21.15.40:0/3250202129 learned_addr learned my addr 172.21.15.40:0/3250202129 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7654c6d640 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c91b080 0x56135c92f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f764ffff640 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c9c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 -- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 msgr2=0x56135c9c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c9c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 -- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c91b080 msgr2=0x56135c92f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c91b080 0x56135c92f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 -- 172.21.15.40:0/3250202129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135c943810 con 0x56135c94dce0 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7654c6d640 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c91b080 0x56135c92f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f765546e640 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c94dce0 0x56135c944510 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f764800c090 tx=0x7f7648004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e19f945de12370 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f762f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f762f7fe640 1 -- 172.21.15.40:0/3250202129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648063070 con 0x56135c94dce0 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f762f7fe640 1 -- 172.21.15.40:0/3250202129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f7648050b80 con 0x56135c94dce0 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.332+0000 7f7659bef740 1 -- 172.21.15.40:0/3250202129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f764002f070 con 0x56135c94dce0 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f762f7fe640 1 -- 172.21.15.40:0/3250202129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=135) ==== 24+0+0 (secure 0 0 0) 0x7f7648050d80 con 0x56135c94dce0 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7659bef740 10 client.8958 fetch_fsmap learned FSMap version 135 2024-04-26T21:43:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7659bef740 1 -- 172.21.15.40:0/3250202129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=135,monmap=2+}) -- 0x56135c921110 con 0x56135c94dce0 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f762f7fe640 1 -- 172.21.15.40:0/3250202129 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764002f070 con 0x56135c94dce0 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f762f7fe640 1 -- 172.21.15.40:0/3250202129 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 135) ==== 3494+0+0 (secure 0 0 0) 0x7f7648066020 con 0x56135c94dce0 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7659bef740 10 client.8958 fetch_fsmap finished waiting for FSMap version 135 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7659bef740 10 client.8958 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7659bef740 20 client.8958 populate_metadata read hostname 'smithi040' 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7659bef740 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] conn(0x56135c900160 0x56135c96e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7659bef740 4 client.8958 mds_command: new command op to 8898 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7659bef740 1 -- 172.21.15.40:0/3250202129 --> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] -- command(tid 0: {"prefix": "status"}) -- 0x56135c965c60 con 0x56135c900160 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7654c6d640 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] conn(0x56135c900160 0x56135c96e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f762f7fe640 10 client.8958 ms_handle_connect on v2:172.21.15.40:6832/3114124794 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.333+0000 7f7654c6d640 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] conn(0x56135c900160 0x56135c96e140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548232fd8dbbd30f server_cookie=a3338a070c8a2f9e in_seq=0 out_seq=0 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f762f7fe640 1 -- 172.21.15.40:0/3250202129 <== mds.0 v2:172.21.15.40:6832/3114124794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x56135c965c60 con 0x56135c900160 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f762f7fe640 10 client.8958 handle_command_reply: tid=0 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 client.8958 shutdown 2024-04-26T21:43:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 -- 172.21.15.40:0/3250202129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c94dce0 msgr2=0x56135c944510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c94dce0 0x56135c944510 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f764800c090 tx=0x7f7648004a40 comp rx=0 tx=0).stop 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 -- 172.21.15.40:0/3250202129 shutdown_connections 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56135c94dce0 0x56135c944510 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.40:6832/3114124794,v1:172.21.15.40:6833/3114124794] conn(0x56135c900160 0x56135c96e140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56135c930440 0x56135c9c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 --2- 172.21.15.40:0/3250202129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56135c91b080 0x56135c92f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f762f7fe640 0 client.8958 ms_handle_reset on v2:172.21.15.40:6832/3114124794 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 -- 172.21.15.40:0/3250202129 >> 172.21.15.40:0/3250202129 conn(0x56135c906120 msgr2=0x56135c900cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 -- 172.21.15.40:0/3250202129 shutdown_connections 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 1 -- 172.21.15.40:0/3250202129 wait complete. 2024-04-26T21:43:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7659bef740 20 client.8958 trim_cache size 0 max 16384 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 8898, 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.6847873599999996, 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 135, 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 102, 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 102, 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 8.4674364939999993 2024-04-26T21:43:22.340 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:43:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7656ed1640 1 -- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f763c06c8f0 msgr2=0x7f763c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7656ed1640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f763c06c8f0 0x7f763c06edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f76500c97d0 tx=0x7f7640009040 comp rx=0 tx=0).stop 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7656ed1640 1 -- 172.21.15.40:0/3567173679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765006e270 msgr2=0x7f765014f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.334+0000 7f7656ed1640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765006e270 0x7f765014f040 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f764400c780 tx=0x7f764400cc40 comp rx=0 tx=0).stop 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.335+0000 7f7656ed1640 1 -- 172.21.15.40:0/3567173679 shutdown_connections 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.335+0000 7f7656ed1640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f763c06c8f0 0x7f763c06edb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.335+0000 7f7656ed1640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76500d0070 0x7f765014e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.335+0000 7f7656ed1640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765006e270 0x7f765014f040 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.335+0000 7f7656ed1640 1 --2- 172.21.15.40:0/3567173679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f765006d560 0x7f765009cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.335+0000 7f7656ed1640 1 -- 172.21.15.40:0/3567173679 >> 172.21.15.40:0/3567173679 conn(0x7f76500d2ec0 msgr2=0x7f76500d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.335+0000 7f7656ed1640 1 -- 172.21.15.40:0/3567173679 shutdown_connections 2024-04-26T21:43:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.335+0000 7f7656ed1640 1 -- 172.21.15.40:0/3567173679 wait complete. 2024-04-26T21:43:22.345 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:43:22.345 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 500.00000000', proceeding to ls 2024-04-26T21:43:22.345 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:43:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.446+0000 7f51a394d640 1 Processor -- start 2024-04-26T21:43:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.446+0000 7f51a394d640 1 -- start start 2024-04-26T21:43:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a394d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f519c162880 0x7f519c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a394d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f519c0b5f20 0x7f519c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a394d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f519c0b7e50 con 0x7f519c0d39f0 2024-04-26T21:43:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a394d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f519c0d3e40 con 0x7f519c162880 2024-04-26T21:43:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a394d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f519c0d0580 con 0x7f519c0b5f20 2024-04-26T21:43:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a16e9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f519c162880 0x7f519c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a1eea640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f519c0d39f0 0x7f519c0b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60150/0 (socket says 172.21.15.40:60150) 2024-04-26T21:43:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a1eea640 1 -- 172.21.15.40:0/1324775472 learned_addr learned my addr 172.21.15.40:0/1324775472 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a0ee8640 1 --2- 172.21.15.40:0/1324775472 >> v2:172.21.15.184:3300/0 conn(0x7f519c0b5f20 0x7f519c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a16e9640 1 -- 172.21.15.40:0/1324775472 >> v2:172.21.15.184:3300/0 conn(0x7f519c0b5f20 msgr2=0x7f519c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a16e9640 1 --2- 172.21.15.40:0/1324775472 >> v2:172.21.15.184:3300/0 conn(0x7f519c0b5f20 0x7f519c0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a16e9640 1 -- 172.21.15.40:0/1324775472 >> v1:172.21.15.40:6789/0 conn(0x7f519c0d39f0 legacy=0x7f519c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f5193fff640 1 -- 172.21.15.40:0/1324775472 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802509087 0 0) 0x7f519c0b7e50 con 0x7f519c0d39f0 2024-04-26T21:43:22.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.447+0000 7f51a16e9640 1 -- 172.21.15.40:0/1324775472 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c0d1800 con 0x7f519c162880 2024-04-26T21:43:22.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.448+0000 7f51a16e9640 1 --2- 172.21.15.40:0/1324775472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c0d2cb0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f519c08d3d0 tx=0x7f519802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b617851aabec962 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.448+0000 7f5193fff640 1 -- 172.21.15.40:0/1324775472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198037070 con 0x7f519c162880 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.448+0000 7f5193fff640 1 -- 172.21.15.40:0/1324775472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5198032440 con 0x7f519c162880 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.448+0000 7f5193fff640 1 -- 172.21.15.40:0/1324775472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198032810 con 0x7f519c162880 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.449+0000 7f51a394d640 1 -- 172.21.15.40:0/1324775472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 msgr2=0x7f519c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.449+0000 7f51a394d640 1 --2- 172.21.15.40:0/1324775472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c0d2cb0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f519c08d3d0 tx=0x7f519802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.449+0000 7f51a394d640 1 -- 172.21.15.40:0/1324775472 shutdown_connections 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.449+0000 7f51a394d640 1 --2- 172.21.15.40:0/1324775472 >> v2:172.21.15.184:3300/0 conn(0x7f519c0b5f20 0x7f519c0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.449+0000 7f51a394d640 1 --2- 172.21.15.40:0/1324775472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c0d2cb0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.449+0000 7f51a394d640 1 -- 172.21.15.40:0/1324775472 >> 172.21.15.40:0/1324775472 conn(0x7f519c0d4f60 msgr2=0x7f519c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.449+0000 7f51a394d640 1 -- 172.21.15.40:0/1324775472 shutdown_connections 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.449+0000 7f51a394d640 1 -- 172.21.15.40:0/1324775472 wait complete. 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.450+0000 7f51a394d640 1 Processor -- start 2024-04-26T21:43:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.450+0000 7f51a394d640 1 -- start start 2024-04-26T21:43:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a394d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f519c0b5f20 0x7f519c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a394d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519c0d39f0 0x7f519c160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a394d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a394d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f519c0d0d90 con 0x7f519c162880 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a394d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f519c0d0ab0 con 0x7f519c0b5f20 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a394d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f519c162c60 con 0x7f519c0d39f0 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52152/0 (socket says 172.21.15.40:52152) 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 -- 172.21.15.40:0/1671091695 learned_addr learned my addr 172.21.15.40:0/1671091695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a0ee8640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519c0d39f0 0x7f519c160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a16e9640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f519c0b5f20 0x7f519c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 -- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519c0d39f0 msgr2=0x7f519c160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519c0d39f0 0x7f519c160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 -- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f519c0b5f20 msgr2=0x7f519c0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f519c0b5f20 0x7f519c0cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 -- 172.21.15.40:0/1671091695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c1524b0 con 0x7f519c162880 2024-04-26T21:43:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a16e9640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f519c0b5f20 0x7f519c0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a1eea640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c160790 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f519400c700 tx=0x7f519400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f5191ffb640 1 -- 172.21.15.40:0/1671091695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194018070 con 0x7f519c162880 2024-04-26T21:43:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f5191ffb640 1 -- 172.21.15.40:0/1671091695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519400ce10 con 0x7f519c162880 2024-04-26T21:43:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f5191ffb640 1 -- 172.21.15.40:0/1671091695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519400a860 con 0x7f519c162880 2024-04-26T21:43:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.451+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c152790 con 0x7f519c162880 2024-04-26T21:43:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.452+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c152d50 con 0x7f519c162880 2024-04-26T21:43:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.453+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c075d70 con 0x7f519c162880 2024-04-26T21:43:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.453+0000 7f5191ffb640 1 -- 172.21.15.40:0/1671091695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f519401b070 con 0x7f519c162880 2024-04-26T21:43:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.453+0000 7f5191ffb640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518006c8f0 0x7f518006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.453+0000 7f5191ffb640 1 -- 172.21.15.40:0/1671091695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f519408b480 con 0x7f519c162880 2024-04-26T21:43:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.454+0000 7f51a16e9640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518006c8f0 0x7f518006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.454+0000 7f51a16e9640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518006c8f0 0x7f518006edb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5198034000 tx=0x7f519802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.458+0000 7f5191ffb640 1 -- 172.21.15.40:0/1671091695 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f519c075d70 con 0x7f519c162880 2024-04-26T21:43:22.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.684+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f519c0694f0 con 0x7f519c162880 2024-04-26T21:43:22.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.684+0000 7f5191ffb640 1 -- 172.21.15.40:0/1671091695 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v135) ==== 53+0+73 (secure 0 0 0) 0x7f5194058e00 con 0x7f519c162880 2024-04-26T21:43:22.684 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:43:22.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518006c8f0 msgr2=0x7f518006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518006c8f0 0x7f518006edb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5198034000 tx=0x7f519802e000 comp rx=0 tx=0).stop 2024-04-26T21:43:22.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 msgr2=0x7f519c160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c160790 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f519400c700 tx=0x7f519400cbc0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 shutdown_connections 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f518006c8f0 0x7f518006edb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f519c162880 0x7f519c160790 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519c0d39f0 0x7f519c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 --2- 172.21.15.40:0/1671091695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f519c0b5f20 0x7f519c0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 >> 172.21.15.40:0/1671091695 conn(0x7f519c0d4f60 msgr2=0x7f519c161480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 shutdown_connections 2024-04-26T21:43:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22.686+0000 7f51a394d640 1 -- 172.21.15.40:0/1671091695 wait complete. 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:43:22.697 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:22.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:22.697 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:43:22.714 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:43:22.714 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:43:22.714 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:43:22.714 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:22.714 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:43:22.714 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:22.714 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42245sec preferred_lft 42245sec 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:22.715 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:43:22.716 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:43:22.717 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:43:22.717 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:22.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:22.717 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:43:22.778 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:43:22.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:22.780 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:43:22.836 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:43:22.836 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:43:22.836 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:43:22.836 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:43:22.836 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:43:22.836 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:43:22.837 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:43:22.837 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:43:22.837 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:43:22.837 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:43:22.837 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:43:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:43:22.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:43:22.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:22.963 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:43:22.978 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:43:22.979 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:43:22.979 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:22.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:22.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:43:23.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:43:23.043 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:23.099 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:23.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:43:23.099 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:23.153 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:43:23.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:23.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:23.236 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:23.236 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:43:23.243 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:23.244 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:43:23.244 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:43:23.244 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:43:23.244 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:43:23.244 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:43:23.244 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:43:23.244 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:43:23.244 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:43:23.245 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:43:23.246 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:43:23.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:23.246 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:23.308 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:43:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:43:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:43:23.405 INFO:teuthology.orchestra.run.smithi040.stdout:client7795 2024-04-26T21:43:23.406 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:23.406 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:23.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:23.421 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-26T21:43:23.464 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:43:23.479 INFO:teuthology.orchestra.run.smithi040.stdout:.: 2024-04-26T21:43:23.479 INFO:teuthology.orchestra.run.smithi040.stdout:subdir 2024-04-26T21:43:23.479 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:23.479 INFO:teuthology.orchestra.run.smithi040.stdout:./subdir: 2024-04-26T21:43:23.479 INFO:teuthology.orchestra.run.smithi040.stdout:sixmegs 2024-04-26T21:43:23.479 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 500.00000000' did not caused DAMAGED state 2024-04-26T21:43:23.479 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds_snaptable' 2024-04-26T21:43:23.479 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:23.479 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:23.534 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:23.534 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:23.590 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:43:23.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.730+0000 7f5276c6d640 1 Processor -- start 2024-04-26T21:43:23.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.731+0000 7f5276c6d640 1 -- start start 2024-04-26T21:43:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.731+0000 7f5276c6d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f527015f3a0 0x7f52700d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.731+0000 7f5276c6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d2330 0x7f52700b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.731+0000 7f5276c6d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52700d0e20 con 0x7f52700d1770 2024-04-26T21:43:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.731+0000 7f5276c6d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f52700b1d70 con 0x7f527015f3a0 2024-04-26T21:43:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.731+0000 7f5276c6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52700b2050 con 0x7f52700d2330 2024-04-26T21:43:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f5274a09640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f527015f3a0 0x7f52700d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f527520a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d2330 0x7f52700b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f526ffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f52700d1770 0x7f52700d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:37750/0 (socket says 172.21.15.40:37750) 2024-04-26T21:43:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f526ffff640 1 -- 172.21.15.40:0/2013667258 learned_addr learned my addr 172.21.15.40:0/2013667258 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f5274a09640 1 -- 172.21.15.40:0/2013667258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d2330 msgr2=0x7f52700b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f5274a09640 1 --2- 172.21.15.40:0/2013667258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d2330 0x7f52700b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f5274a09640 1 -- 172.21.15.40:0/2013667258 >> v1:172.21.15.184:6789/0 conn(0x7f52700d1770 legacy=0x7f52700d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f5274a09640 1 -- 172.21.15.40:0/2013667258 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52700b2330 con 0x7f527015f3a0 2024-04-26T21:43:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.732+0000 7f527520a640 1 --2- 172.21.15.40:0/2013667258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d2330 0x7f52700b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:43:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.733+0000 7f5274a09640 1 --2- 172.21.15.40:0/2013667258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527015f3a0 0x7f52700d06f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5268009d90 tx=0x7f526802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37ef33c692a4478d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.733+0000 7f526f7fe640 1 -- 172.21.15.40:0/2013667258 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268007740 con 0x7f527015f3a0 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.733+0000 7f526f7fe640 1 -- 172.21.15.40:0/2013667258 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52680078e0 con 0x7f527015f3a0 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.733+0000 7f526f7fe640 1 -- 172.21.15.40:0/2013667258 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268007cb0 con 0x7f527015f3a0 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.734+0000 7f5276c6d640 1 -- 172.21.15.40:0/2013667258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527015f3a0 msgr2=0x7f52700d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.734+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2013667258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527015f3a0 0x7f52700d06f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5268009d90 tx=0x7f526802a660 comp rx=0 tx=0).stop 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.734+0000 7f5276c6d640 1 -- 172.21.15.40:0/2013667258 shutdown_connections 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.734+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2013667258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d2330 0x7f52700b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.734+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2013667258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527015f3a0 0x7f52700d06f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.734+0000 7f5276c6d640 1 -- 172.21.15.40:0/2013667258 >> 172.21.15.40:0/2013667258 conn(0x7f52700d4f90 msgr2=0x7f52700d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:23.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.734+0000 7f5276c6d640 1 -- 172.21.15.40:0/2013667258 shutdown_connections 2024-04-26T21:43:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.734+0000 7f5276c6d640 1 -- 172.21.15.40:0/2013667258 wait complete. 2024-04-26T21:43:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.735+0000 7f5276c6d640 1 Processor -- start 2024-04-26T21:43:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.735+0000 7f5276c6d640 1 -- start start 2024-04-26T21:43:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f5276c6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d1770 0x7f52700cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f5276c6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52700d2330 0x7f5270150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f5276c6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f527015f3a0 0x7f52700ba360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f5276c6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52700b2f10 con 0x7f527015f3a0 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f5276c6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52700b2d90 con 0x7f52700d2330 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f5276c6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52700b3090 con 0x7f52700d1770 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f527520a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f527015f3a0 0x7f52700ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f527520a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f527015f3a0 0x7f52700ba360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52156/0 (socket says 172.21.15.40:52156) 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f527520a640 1 -- 172.21.15.40:0/2164369599 learned_addr learned my addr 172.21.15.40:0/2164369599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f526ffff640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52700d2330 0x7f5270150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f5274a09640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d1770 0x7f52700cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f527520a640 1 -- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d1770 msgr2=0x7f52700cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f527520a640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d1770 0x7f52700cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f527520a640 1 -- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52700d2330 msgr2=0x7f5270150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f527520a640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52700d2330 0x7f5270150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.736+0000 7f527520a640 1 -- 172.21.15.40:0/2164369599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52700b92c0 con 0x7f527015f3a0 2024-04-26T21:43:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.737+0000 7f527520a640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f527015f3a0 0x7f52700ba360 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f526000e900 tx=0x7f526000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.737+0000 7f526d7fa640 1 -- 172.21.15.40:0/2164369599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260017070 con 0x7f527015f3a0 2024-04-26T21:43:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.737+0000 7f526d7fa640 1 -- 172.21.15.40:0/2164369599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f526000ca00 con 0x7f527015f3a0 2024-04-26T21:43:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.737+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52700b95a0 con 0x7f527015f3a0 2024-04-26T21:43:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.737+0000 7f526d7fa640 1 -- 172.21.15.40:0/2164369599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526000cdf0 con 0x7f527015f3a0 2024-04-26T21:43:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.737+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52700b9b60 con 0x7f527015f3a0 2024-04-26T21:43:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.738+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52700694f0 con 0x7f527015f3a0 2024-04-26T21:43:23.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.738+0000 7f526d7fa640 1 -- 172.21.15.40:0/2164369599 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f526001a070 con 0x7f527015f3a0 2024-04-26T21:43:23.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.739+0000 7f526d7fa640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f525c06c8f0 0x7f525c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.739+0000 7f526d7fa640 1 -- 172.21.15.40:0/2164369599 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f526008bd50 con 0x7f527015f3a0 2024-04-26T21:43:23.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.739+0000 7f5274a09640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f525c06c8f0 0x7f525c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.739+0000 7f5274a09640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f525c06c8f0 0x7f525c06edb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5268034000 tx=0x7f526802f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:23.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.743+0000 7f526d7fa640 1 -- 172.21.15.40:0/2164369599 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f526008f030 con 0x7f527015f3a0 2024-04-26T21:43:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.930+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52700b6860 con 0x7f527015f3a0 2024-04-26T21:43:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.931+0000 7f526d7fa640 1 -- 172.21.15.40:0/2164369599 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+17954 (secure 0 0 0) 0x7f5260059760 con 0x7f527015f3a0 2024-04-26T21:43:23.931 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:23.932 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":102,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:43:13.833455+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.933+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f525c06c8f0 msgr2=0x7f525c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.933+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f525c06c8f0 0x7f525c06edb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5268034000 tx=0x7f526802f000 comp rx=0 tx=0).stop 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.933+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f527015f3a0 msgr2=0x7f52700ba360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.933+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f527015f3a0 0x7f52700ba360 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f526000e900 tx=0x7f526000edc0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.934+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 shutdown_connections 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.934+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f525c06c8f0 0x7f525c06edb0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.934+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f527015f3a0 0x7f52700ba360 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.934+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52700d2330 0x7f5270150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.934+0000 7f5276c6d640 1 --2- 172.21.15.40:0/2164369599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52700d1770 0x7f52700cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.934+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 >> 172.21.15.40:0/2164369599 conn(0x7f52700d4f90 msgr2=0x7f527006de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.934+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 shutdown_connections 2024-04-26T21:43:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:23.934+0000 7f5276c6d640 1 -- 172.21.15.40:0/2164369599 wait complete. 2024-04-26T21:43:23.944 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:43:23.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:43:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.049+0000 7fa3fccfc640 1 Processor -- start 2024-04-26T21:43:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.049+0000 7fa3fccfc640 1 -- start start 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3fccfc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa3f815f550 0x7fa3f80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3fccfc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3f80d3bc0 0x7fa3f80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3fccfc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f80d1040 con 0x7fa3f815f170 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3fccfc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f80c9430 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3fccfc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f80c9680 con 0x7fa3f815f550 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3f6d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3f80d3bc0 0x7fa3f80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3f6d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3f80d3bc0 0x7fa3f80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52168/0 (socket says 172.21.15.40:52168) 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3f6d9d640 1 -- 172.21.15.40:0/1199592999 learned_addr learned my addr 172.21.15.40:0/1199592999 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3effff640 1 --2- 172.21.15.40:0/1199592999 >> v2:172.21.15.184:3300/0 conn(0x7fa3f815f550 0x7fa3f80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3f6d9d640 1 -- 172.21.15.40:0/1199592999 >> v2:172.21.15.184:3300/0 conn(0x7fa3f815f550 msgr2=0x7fa3f80d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3f6d9d640 1 --2- 172.21.15.40:0/1199592999 >> v2:172.21.15.184:3300/0 conn(0x7fa3f815f550 0x7fa3f80d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3f6d9d640 1 -- 172.21.15.40:0/1199592999 >> v1:172.21.15.184:6789/0 conn(0x7fa3f815f170 legacy=0x7fa3f80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.050+0000 7fa3f6d9d640 1 -- 172.21.15.40:0/1199592999 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f80c9960 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3f6d9d640 1 --2- 172.21.15.40:0/1199592999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f80d2cb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009e50 tx=0x7fa3f002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe92b44ea8c2983 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3f5d9b640 1 -- 172.21.15.40:0/1199592999 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0036070 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3f5d9b640 1 -- 172.21.15.40:0/1199592999 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f0031600 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3f5d9b640 1 -- 172.21.15.40:0/1199592999 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00319d0 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3fccfc640 1 -- 172.21.15.40:0/1199592999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 msgr2=0x7fa3f80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/1199592999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f80d2cb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009e50 tx=0x7fa3f002a710 comp rx=0 tx=0).stop 2024-04-26T21:43:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3fccfc640 1 -- 172.21.15.40:0/1199592999 shutdown_connections 2024-04-26T21:43:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/1199592999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f80d2cb0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/1199592999 >> v2:172.21.15.184:3300/0 conn(0x7fa3f815f550 0x7fa3f80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.051+0000 7fa3fccfc640 1 -- 172.21.15.40:0/1199592999 >> 172.21.15.40:0/1199592999 conn(0x7fa3f80d4f60 msgr2=0x7fa3f80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.052+0000 7fa3fccfc640 1 -- 172.21.15.40:0/1199592999 shutdown_connections 2024-04-26T21:43:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.052+0000 7fa3fccfc640 1 -- 172.21.15.40:0/1199592999 wait complete. 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.053+0000 7fa3fccfc640 1 Processor -- start 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.053+0000 7fa3fccfc640 1 -- start start 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3fccfc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f81f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3fccfc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3f815f170 0x7fa3f80b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3fccfc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3f81f6460 0x7fa3f81f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3fccfc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80ca5c0 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3fccfc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80ca440 con 0x7fa3f81f6460 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3fccfc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f80ca740 con 0x7fa3f815f170 2024-04-26T21:43:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f81f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f81f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52178/0 (socket says 172.21.15.40:52178) 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 -- 172.21.15.40:0/4249159848 learned_addr learned my addr 172.21.15.40:0/4249159848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3effff640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3f815f170 0x7fa3f80b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f6d9d640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3f81f6460 0x7fa3f81f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 -- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3f815f170 msgr2=0x7fa3f80b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3f815f170 0x7fa3f80b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 -- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3f81f6460 msgr2=0x7fa3f81f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3f81f6460 0x7fa3f81f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 -- 172.21.15.40:0/4249159848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f80cda20 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3f659c640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f81f5cd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800c430 tx=0x7fa3e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3eeffd640 1 -- 172.21.15.40:0/4249159848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8011790 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3eeffd640 1 -- 172.21.15.40:0/4249159848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e8017070 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f80b2090 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.054+0000 7fa3eeffd640 1 -- 172.21.15.40:0/4249159848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8011bb0 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.055+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f80cdfa0 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.056+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f80694f0 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.056+0000 7fa3eeffd640 1 -- 172.21.15.40:0/4249159848 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3e801c460 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.056+0000 7fa3eeffd640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3d806c8f0 0x7fa3d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.057+0000 7fa3eeffd640 1 -- 172.21.15.40:0/4249159848 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7fa3e808bed0 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.057+0000 7fa3effff640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3d806c8f0 0x7fa3d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.057+0000 7fa3effff640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3d806c8f0 0x7fa3d806edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0006fd0 tx=0x7fa3e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.061+0000 7fa3eeffd640 1 -- 172.21.15.40:0/4249159848 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3e8059ea0 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.248+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3f80d0b60 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.249+0000 7fa3eeffd640 1 -- 172.21.15.40:0/4249159848 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7fa3e80597a0 con 0x7fa3f80d3bc0 2024-04-26T21:43:24.249 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3d806c8f0 msgr2=0x7fa3d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3d806c8f0 0x7fa3d806edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0006fd0 tx=0x7fa3e0008040 comp rx=0 tx=0).stop 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 msgr2=0x7fa3f81f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f81f5cd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800c430 tx=0x7fa3e800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 shutdown_connections 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa3d806c8f0 0x7fa3d806edb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3f81f6460 0x7fa3f81f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3f815f170 0x7fa3f80b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 --2- 172.21.15.40:0/4249159848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f80d3bc0 0x7fa3f81f5cd0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 >> 172.21.15.40:0/4249159848 conn(0x7fa3f80d4f60 msgr2=0x7fa3f80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 shutdown_connections 2024-04-26T21:43:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.251+0000 7fa3fccfc640 1 -- 172.21.15.40:0/4249159848 wait complete. 2024-04-26T21:43:24.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:43:24.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.366+0000 7f6d5950a640 1 Processor -- start 2024-04-26T21:43:24.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.366+0000 7f6d5950a640 1 -- start start 2024-04-26T21:43:24.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d5950a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b0380 0x7f6d540ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d5950a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d5406c950 0x7f6d5415e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d5950a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d540cfd80 con 0x7f6d540b10b0 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d5950a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6d540af8e0 con 0x7f6d5406c950 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d5950a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6d540d0020 con 0x7f6d540b0380 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b0380 0x7f6d540ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d537fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d5406c950 0x7f6d5415e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d4bfff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f6d540b10b0 0x7f6d5406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:37758/0 (socket says 172.21.15.40:37758) 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d4bfff640 1 -- 172.21.15.40:0/851629899 learned_addr learned my addr 172.21.15.40:0/851629899 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d527fc640 1 -- 172.21.15.40:0/851629899 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1541856631 0 0) 0x7f6d540cfd80 con 0x7f6d540b10b0 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d527fc640 1 -- 172.21.15.40:0/851629899 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d40003520 con 0x7f6d540b10b0 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d537fe640 1 -- 172.21.15.40:0/851629899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b0380 msgr2=0x7f6d540ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d537fe640 1 --2- 172.21.15.40:0/851629899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b0380 0x7f6d540ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d537fe640 1 -- 172.21.15.40:0/851629899 >> v1:172.21.15.184:6789/0 conn(0x7f6d540b10b0 legacy=0x7f6d5406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d537fe640 1 -- 172.21.15.40:0/851629899 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d540d0300 con 0x7f6d5406c950 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.367+0000 7f6d52ffd640 1 --2- 172.21.15.40:0/851629899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b0380 0x7f6d540ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:43:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.368+0000 7f6d537fe640 1 --2- 172.21.15.40:0/851629899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 0x7f6d5415e970 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d540afa60 tx=0x7f6d4c02a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b32e5741020e40c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.368+0000 7f6d527fc640 1 -- 172.21.15.40:0/851629899 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c007740 con 0x7f6d5406c950 2024-04-26T21:43:24.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.368+0000 7f6d527fc640 1 -- 172.21.15.40:0/851629899 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c0078e0 con 0x7f6d5406c950 2024-04-26T21:43:24.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.368+0000 7f6d527fc640 1 -- 172.21.15.40:0/851629899 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c007cb0 con 0x7f6d5406c950 2024-04-26T21:43:24.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.369+0000 7f6d5950a640 1 -- 172.21.15.40:0/851629899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 msgr2=0x7f6d5415e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.369+0000 7f6d5950a640 1 --2- 172.21.15.40:0/851629899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 0x7f6d5415e970 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d540afa60 tx=0x7f6d4c02a5b0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.369+0000 7f6d5950a640 1 -- 172.21.15.40:0/851629899 shutdown_connections 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.369+0000 7f6d5950a640 1 --2- 172.21.15.40:0/851629899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 0x7f6d5415e970 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.369+0000 7f6d5950a640 1 --2- 172.21.15.40:0/851629899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b0380 0x7f6d540ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.369+0000 7f6d5950a640 1 -- 172.21.15.40:0/851629899 >> 172.21.15.40:0/851629899 conn(0x7f6d540d1cf0 msgr2=0x7f6d540d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.369+0000 7f6d5950a640 1 -- 172.21.15.40:0/851629899 shutdown_connections 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.369+0000 7f6d5950a640 1 -- 172.21.15.40:0/851629899 wait complete. 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.370+0000 7f6d5950a640 1 Processor -- start 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.370+0000 7f6d5950a640 1 -- start start 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d5950a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 0x7f6d54156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d5950a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d540b0380 0x7f6d541fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d5950a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b10b0 0x7f6d540c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d5950a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6d540b0760 con 0x7f6d540b0380 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d5950a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d540d1980 con 0x7f6d5406c950 2024-04-26T21:43:24.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d5950a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6d540d15c0 con 0x7f6d540b10b0 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d540b0380 0x7f6d541fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d540b0380 0x7f6d541fc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52186/0 (socket says 172.21.15.40:52186) 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 -- 172.21.15.40:0/3155111507 learned_addr learned my addr 172.21.15.40:0/3155111507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d537fe640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b10b0 0x7f6d540c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d52ffd640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 0x7f6d54156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 -- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b10b0 msgr2=0x7f6d540c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b10b0 0x7f6d540c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 -- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 msgr2=0x7f6d54156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 0x7f6d54156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 -- 172.21.15.40:0/3155111507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d541fce80 con 0x7f6d540b0380 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.371+0000 7f6d4bfff640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d540b0380 0x7f6d541fc3c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00c390 tx=0x7f6d3c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.372+0000 7f6d4b7fe640 1 -- 172.21.15.40:0/3155111507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c017070 con 0x7f6d540b0380 2024-04-26T21:43:24.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.372+0000 7f6d4b7fe640 1 -- 172.21.15.40:0/3155111507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3c005ce0 con 0x7f6d540b0380 2024-04-26T21:43:24.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.372+0000 7f6d4b7fe640 1 -- 172.21.15.40:0/3155111507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0072f0 con 0x7f6d540b0380 2024-04-26T21:43:24.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.372+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d541fd160 con 0x7f6d540b0380 2024-04-26T21:43:24.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.372+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d540c92f0 con 0x7f6d540b0380 2024-04-26T21:43:24.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.373+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d540694f0 con 0x7f6d540b0380 2024-04-26T21:43:24.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.373+0000 7f6d4b7fe640 1 -- 172.21.15.40:0/3155111507 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d3c01a070 con 0x7f6d540b0380 2024-04-26T21:43:24.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.374+0000 7f6d4b7fe640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d3406c8f0 0x7f6d3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.374+0000 7f6d4b7fe640 1 -- 172.21.15.40:0/3155111507 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f6d3c08b440 con 0x7f6d540b0380 2024-04-26T21:43:24.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.374+0000 7f6d52ffd640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d3406c8f0 0x7f6d3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.374+0000 7f6d52ffd640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d3406c8f0 0x7f6d3406edb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009560 tx=0x7f6d44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.378+0000 7f6d4b7fe640 1 -- 172.21.15.40:0/3155111507 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d3c059370 con 0x7f6d540b0380 2024-04-26T21:43:24.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.566+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d5406c450 con 0x7f6d540b0380 2024-04-26T21:43:24.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.566+0000 7f6d4b7fe640 1 -- 172.21.15.40:0/3155111507 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7f6d3c05f070 con 0x7f6d540b0380 2024-04-26T21:43:24.566 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:43:24.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.568+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d3406c8f0 msgr2=0x7f6d3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.568+0000 7f6d5950a640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d3406c8f0 0x7f6d3406edb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009560 tx=0x7f6d44008040 comp rx=0 tx=0).stop 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.568+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d540b0380 msgr2=0x7f6d541fc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.568+0000 7f6d5950a640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d540b0380 0x7f6d541fc3c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00c390 tx=0x7f6d3c00c850 comp rx=0 tx=0).stop 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.568+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 shutdown_connections 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.568+0000 7f6d5950a640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d3406c8f0 0x7f6d3406edb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.568+0000 7f6d5950a640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b10b0 0x7f6d540c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.568+0000 7f6d5950a640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d540b0380 0x7f6d541fc3c0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.569+0000 7f6d5950a640 1 --2- 172.21.15.40:0/3155111507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d5406c950 0x7f6d54156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.569+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 >> 172.21.15.40:0/3155111507 conn(0x7f6d540d1cf0 msgr2=0x7f6d54070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.569+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 shutdown_connections 2024-04-26T21:43:24.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.569+0000 7f6d5950a640 1 -- 172.21.15.40:0/3155111507 wait complete. 2024-04-26T21:43:24.579 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:43:24.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.683+0000 7f6bf0cb8640 1 Processor -- start 2024-04-26T21:43:24.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.683+0000 7f6bf0cb8640 1 -- start start 2024-04-26T21:43:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.684+0000 7f6bf0cb8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bec0b1fd0 0x7f6bec0d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.684+0000 7f6bf0cb8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6bec0cdb70 0x7f6bec0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.684+0000 7f6bf0cb8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bec06e580 con 0x7f6bec15f770 2024-04-26T21:43:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.684+0000 7f6bf0cb8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6bec06f3e0 con 0x7f6bec0b1fd0 2024-04-26T21:43:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.684+0000 7f6bf0cb8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6bec0cd6e0 con 0x7f6bec0cdb70 2024-04-26T21:43:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.684+0000 7f6be9d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bec0b1fd0 0x7f6bec0d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.684+0000 7f6be9d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bec0b1fd0 0x7f6bec0d1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52188/0 (socket says 172.21.15.40:52188) 2024-04-26T21:43:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.684+0000 7f6be9d9b640 1 -- 172.21.15.40:0/1073929423 learned_addr learned my addr 172.21.15.40:0/1073929423 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.685+0000 7f6bead9d640 1 --2- 172.21.15.40:0/1073929423 >> v2:172.21.15.184:3300/0 conn(0x7f6bec0cdb70 0x7f6bec0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.685+0000 7f6be9d9b640 1 -- 172.21.15.40:0/1073929423 >> v2:172.21.15.184:3300/0 conn(0x7f6bec0cdb70 msgr2=0x7f6bec0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.685+0000 7f6be9d9b640 1 --2- 172.21.15.40:0/1073929423 >> v2:172.21.15.184:3300/0 conn(0x7f6bec0cdb70 0x7f6bec0ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.685+0000 7f6be9d9b640 1 -- 172.21.15.40:0/1073929423 >> v1:172.21.15.184:6789/0 conn(0x7f6bec15f770 legacy=0x7f6bec0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.685+0000 7f6be9d9b640 1 -- 172.21.15.40:0/1073929423 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec06e860 con 0x7f6bec0b1fd0 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.685+0000 7f6bead9d640 1 --2- 172.21.15.40:0/1073929423 >> v2:172.21.15.184:3300/0 conn(0x7f6bec0cdb70 0x7f6bec0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.685+0000 7f6be9d9b640 1 --2- 172.21.15.40:0/1073929423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec0b1fd0 0x7f6bec0d1470 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6bec073c60 tx=0x7f6bd402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3912a48189928fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6be959a640 1 -- 172.21.15.40:0/1073929423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4037070 con 0x7f6bec0b1fd0 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6be959a640 1 -- 172.21.15.40:0/1073929423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd4032440 con 0x7f6bec0b1fd0 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6be959a640 1 -- 172.21.15.40:0/1073929423 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4032810 con 0x7f6bec0b1fd0 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/1073929423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec0b1fd0 msgr2=0x7f6bec0d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/1073929423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec0b1fd0 0x7f6bec0d1470 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6bec073c60 tx=0x7f6bd402a230 comp rx=0 tx=0).stop 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/1073929423 shutdown_connections 2024-04-26T21:43:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/1073929423 >> v2:172.21.15.184:3300/0 conn(0x7f6bec0cdb70 0x7f6bec0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/1073929423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec0b1fd0 0x7f6bec0d1470 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/1073929423 >> 172.21.15.40:0/1073929423 conn(0x7f6bec0d2e60 msgr2=0x7f6bec0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/1073929423 shutdown_connections 2024-04-26T21:43:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.686+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/1073929423 wait complete. 2024-04-26T21:43:24.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.687+0000 7f6bf0cb8640 1 Processor -- start 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.687+0000 7f6bf0cb8640 1 -- start start 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bf0cb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0b1fd0 0x7f6bec1fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bf0cb8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0cdb70 0x7f6bec0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bf0cb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec15f770 0x7f6bec1f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bf0cb8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec0d1b30 con 0x7f6bec15f770 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bf0cb8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec0d19b0 con 0x7f6bec0cdb70 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bf0cb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6bec0ce540 con 0x7f6bec0b1fd0 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bead9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec15f770 0x7f6bec1f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bead9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec15f770 0x7f6bec1f9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52194/0 (socket says 172.21.15.40:52194) 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.688+0000 7f6bead9d640 1 -- 172.21.15.40:0/3036166423 learned_addr learned my addr 172.21.15.40:0/3036166423 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6be9d9b640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0cdb70 0x7f6bec0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bea59c640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0b1fd0 0x7f6bec1fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bead9d640 1 -- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0b1fd0 msgr2=0x7f6bec1fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bead9d640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0b1fd0 0x7f6bec1fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bead9d640 1 -- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0cdb70 msgr2=0x7f6bec0ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bead9d640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0cdb70 0x7f6bec0ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bead9d640 1 -- 172.21.15.40:0/3036166423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec0c6af0 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bead9d640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec15f770 0x7f6bec1f9870 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00cef0 tx=0x7f6bdc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6be2ffd640 1 -- 172.21.15.40:0/3036166423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc01a070 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6be2ffd640 1 -- 172.21.15.40:0/3036166423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bdc00d860 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bec0c6dd0 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6be2ffd640 1 -- 172.21.15.40:0/3036166423 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00dc50 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.689+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bec0cae80 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.690+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bec0694f0 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.690+0000 7f6be2ffd640 1 -- 172.21.15.40:0/3036166423 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bdc01d070 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.691+0000 7f6be2ffd640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6bcc06c8f0 0x7f6bcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.691+0000 7f6be2ffd640 1 -- 172.21.15.40:0/3036166423 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f6bdc08c880 con 0x7f6bec15f770 2024-04-26T21:43:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.691+0000 7f6bea59c640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6bcc06c8f0 0x7f6bcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.692+0000 7f6bea59c640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6bcc06c8f0 0x7f6bcc06edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6be4006fd0 tx=0x7f6be4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.695+0000 7f6be2ffd640 1 -- 172.21.15.40:0/3036166423 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6bdc05a910 con 0x7f6bec15f770 2024-04-26T21:43:24.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.881+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6bec0c9bb0 con 0x7f6bec15f770 2024-04-26T21:43:24.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.881+0000 7f6be2ffd640 1 -- 172.21.15.40:0/3036166423 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f6bdc05a210 con 0x7f6bec15f770 2024-04-26T21:43:24.881 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6bcc06c8f0 msgr2=0x7f6bcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6bcc06c8f0 0x7f6bcc06edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6be4006fd0 tx=0x7f6be4008040 comp rx=0 tx=0).stop 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec15f770 msgr2=0x7f6bec1f9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec15f770 0x7f6bec1f9870 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00cef0 tx=0x7f6bdc00a710 comp rx=0 tx=0).stop 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 shutdown_connections 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6bcc06c8f0 0x7f6bcc06edb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bec15f770 0x7f6bec1f9870 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0cdb70 0x7f6bec0ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 --2- 172.21.15.40:0/3036166423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0b1fd0 0x7f6bec1fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 >> 172.21.15.40:0/3036166423 conn(0x7f6bec0d2e60 msgr2=0x7f6bec0b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 shutdown_connections 2024-04-26T21:43:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.883+0000 7f6bf0cb8640 1 -- 172.21.15.40:0/3036166423 wait complete. 2024-04-26T21:43:24.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:43:24.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.995+0000 7fbccf96a640 1 Processor -- start 2024-04-26T21:43:24.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.995+0000 7fbccf96a640 1 -- start start 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.996+0000 7fbccf96a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcc80d27d0 0x7fbcc80d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.996+0000 7fbccf96a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcc8070790 0x7fbcc806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.996+0000 7fbccf96a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcc806e6b0 con 0x7fbcc806e2a0 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.996+0000 7fbccf96a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc8070c10 con 0x7fbcc8070790 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.996+0000 7fbccf96a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc8070ef0 con 0x7fbcc80d27d0 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.996+0000 7fbcccf05640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcc8070790 0x7fbcc806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.996+0000 7fbccdf07640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbcc806e2a0 0x7fbcc8162fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60154/0 (socket says 172.21.15.40:60154) 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.996+0000 7fbccdf07640 1 -- 172.21.15.40:0/1647829343 learned_addr learned my addr 172.21.15.40:0/1647829343 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.997+0000 7fbcccf05640 1 -- 172.21.15.40:0/1647829343 >> v2:172.21.15.184:3300/0 conn(0x7fbcc80d27d0 msgr2=0x7fbcc80d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.997+0000 7fbccd706640 1 --2- 172.21.15.40:0/1647829343 >> v2:172.21.15.184:3300/0 conn(0x7fbcc80d27d0 0x7fbcc80d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.997+0000 7fbcc7fff640 1 -- 172.21.15.40:0/1647829343 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081227471 0 0) 0x7fbcb8001490 con 0x7fbcc806e2a0 2024-04-26T21:43:24.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.997+0000 7fbcccf05640 1 --2- 172.21.15.40:0/1647829343 >> v2:172.21.15.184:3300/0 conn(0x7fbcc80d27d0 0x7fbcc80d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.997+0000 7fbcccf05640 1 -- 172.21.15.40:0/1647829343 >> v1:172.21.15.40:6789/0 conn(0x7fbcc806e2a0 legacy=0x7fbcc8162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.997+0000 7fbcccf05640 1 -- 172.21.15.40:0/1647829343 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc80711d0 con 0x7fbcc8070790 2024-04-26T21:43:24.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.997+0000 7fbcccf05640 1 --2- 172.21.15.40:0/1647829343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc8070790 0x7fbcc806dd60 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80732c0 tx=0x7fbcbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1afb9bf8905e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbcc7fff640 1 -- 172.21.15.40:0/1647829343 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc037070 con 0x7fbcc8070790 2024-04-26T21:43:24.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbcc7fff640 1 -- 172.21.15.40:0/1647829343 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc032440 con 0x7fbcc8070790 2024-04-26T21:43:24.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbcc7fff640 1 -- 172.21.15.40:0/1647829343 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc032810 con 0x7fbcc8070790 2024-04-26T21:43:24.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbccf96a640 1 -- 172.21.15.40:0/1647829343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc8070790 msgr2=0x7fbcc806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbccf96a640 1 --2- 172.21.15.40:0/1647829343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc8070790 0x7fbcc806dd60 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80732c0 tx=0x7fbcbc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbccf96a640 1 -- 172.21.15.40:0/1647829343 shutdown_connections 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbccf96a640 1 --2- 172.21.15.40:0/1647829343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc8070790 0x7fbcc806dd60 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbccf96a640 1 --2- 172.21.15.40:0/1647829343 >> v2:172.21.15.184:3300/0 conn(0x7fbcc80d27d0 0x7fbcc80d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbccf96a640 1 -- 172.21.15.40:0/1647829343 >> 172.21.15.40:0/1647829343 conn(0x7fbcc80d2e00 msgr2=0x7fbcc80d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbccf96a640 1 -- 172.21.15.40:0/1647829343 shutdown_connections 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.998+0000 7fbccf96a640 1 -- 172.21.15.40:0/1647829343 wait complete. 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.999+0000 7fbccf96a640 1 Processor -- start 2024-04-26T21:43:25.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:24.999+0000 7fbccf96a640 1 -- start start 2024-04-26T21:43:25.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccf96a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc806e2a0 0x7fbcc8159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccf96a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc8070790 0x7fbcc815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccf96a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc80d27d0 0x7fbcc815f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccf96a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc80d2bb0 con 0x7fbcc806e2a0 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccf96a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8071790 con 0x7fbcc80d27d0 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccf96a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc80d06e0 con 0x7fbcc8070790 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccd706640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc806e2a0 0x7fbcc8159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccd706640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc806e2a0 0x7fbcc8159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52218/0 (socket says 172.21.15.40:52218) 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.000+0000 7fbccd706640 1 -- 172.21.15.40:0/2948771988 learned_addr learned my addr 172.21.15.40:0/2948771988 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbccd706640 1 -- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc8070790 msgr2=0x7fbcc815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbccd706640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc8070790 0x7fbcc815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbccd706640 1 -- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc80d27d0 msgr2=0x7fbcc815f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbccd706640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc80d27d0 0x7fbcc815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbccd706640 1 -- 172.21.15.40:0/2948771988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc81f5c00 con 0x7fbcc806e2a0 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbccd706640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc806e2a0 0x7fbcc8159350 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000c820 tx=0x7fbcc000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:25.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbcc5ffb640 1 -- 172.21.15.40:0/2948771988 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0015070 con 0x7fbcc806e2a0 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbcc5ffb640 1 -- 172.21.15.40:0/2948771988 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc0004020 con 0x7fbcc806e2a0 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc815b440 con 0x7fbcc806e2a0 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbcc5ffb640 1 -- 172.21.15.40:0/2948771988 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00043f0 con 0x7fbcc806e2a0 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.001+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc815ba00 con 0x7fbcc806e2a0 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.002+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc80694f0 con 0x7fbcc806e2a0 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.002+0000 7fbcc5ffb640 1 -- 172.21.15.40:0/2948771988 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcc0007830 con 0x7fbcc806e2a0 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.003+0000 7fbcc5ffb640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcb406c8f0 0x7fbcb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.003+0000 7fbcc5ffb640 1 -- 172.21.15.40:0/2948771988 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7fbcc008bdc0 con 0x7fbcc806e2a0 2024-04-26T21:43:25.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.003+0000 7fbcccf05640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcb406c8f0 0x7fbcb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.004+0000 7fbcccf05640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcb406c8f0 0x7fbcb406edb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc034000 tx=0x7fbcbc02c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:25.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.007+0000 7fbcc5ffb640 1 -- 172.21.15.40:0/2948771988 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcc0059d90 con 0x7fbcc806e2a0 2024-04-26T21:43:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.204+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbcc80b8d80 con 0x7fbcc806e2a0 2024-04-26T21:43:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.204+0000 7fbcc5ffb640 1 -- 172.21.15.40:0/2948771988 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v102) ==== 81+0+856 (secure 0 0 0) 0x7fbcc0059690 con 0x7fbcc806e2a0 2024-04-26T21:43:25.204 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:43:25.204 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:43:25.204 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:43:25.205 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:43:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:listed 14 entries 2024-04-26T21:43:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcb406c8f0 msgr2=0x7fbcb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcb406c8f0 0x7fbcb406edb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc034000 tx=0x7fbcbc02c000 comp rx=0 tx=0).stop 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc806e2a0 msgr2=0x7fbcc8159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc806e2a0 0x7fbcc8159350 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000c820 tx=0x7fbcc000cce0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 shutdown_connections 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcb406c8f0 0x7fbcb406edb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc80d27d0 0x7fbcc815f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc8070790 0x7fbcc815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 --2- 172.21.15.40:0/2948771988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc806e2a0 0x7fbcc8159350 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 >> 172.21.15.40:0/2948771988 conn(0x7fbcc80d2e00 msgr2=0x7fbcc80c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 shutdown_connections 2024-04-26T21:43:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.206+0000 7fbccf96a640 1 -- 172.21.15.40:0/2948771988 wait complete. 2024-04-26T21:43:25.217 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:43:25.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:25.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:43:25.242 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:43:25.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:25.243 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:43:25.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:43:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.441+0000 7f8a4ab0a640 1 Processor -- start 2024-04-26T21:43:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.441+0000 7f8a4ab0a640 1 -- start start 2024-04-26T21:43:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a4ab0a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a4415f380 0x7f8a440cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a4ab0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a44073b70 0x7f8a440cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a4ab0a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a440d0620 0x7f8a440cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a4ab0a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8a440729b0 con 0x7f8a4415f380 2024-04-26T21:43:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a4ab0a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8a44072c00 con 0x7f8a440d0620 2024-04-26T21:43:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a4ab0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a44072ee0 con 0x7f8a44073b70 2024-04-26T21:43:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a488a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a4415f380 0x7f8a440cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a488a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a4415f380 0x7f8a440cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52228/0 (socket says 172.21.15.40:52228) 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a488a6640 1 -- 172.21.15.40:0/475244499 learned_addr learned my addr 172.21.15.40:0/475244499 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a43fff640 1 --2- 172.21.15.40:0/475244499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a44073b70 0x7f8a440cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a490a7640 1 --2- 172.21.15.40:0/475244499 >> v2:172.21.15.184:3300/0 conn(0x7f8a440d0620 0x7f8a440cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a488a6640 1 -- 172.21.15.40:0/475244499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a44073b70 msgr2=0x7f8a440cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a488a6640 1 --2- 172.21.15.40:0/475244499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a44073b70 0x7f8a440cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a488a6640 1 -- 172.21.15.40:0/475244499 >> v2:172.21.15.184:3300/0 conn(0x7f8a440d0620 msgr2=0x7f8a440cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a488a6640 1 --2- 172.21.15.40:0/475244499 >> v2:172.21.15.184:3300/0 conn(0x7f8a440d0620 0x7f8a440cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.442+0000 7f8a488a6640 1 -- 172.21.15.40:0/475244499 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a440731c0 con 0x7f8a4415f380 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.443+0000 7f8a488a6640 1 --2- 172.21.15.40:0/475244499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a4415f380 0x7f8a440cb440 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f8a440c4410 tx=0x7f8a3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fec5ac0acc467c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.443+0000 7f8a437fe640 1 -- 172.21.15.40:0/475244499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38037070 con 0x7f8a4415f380 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.443+0000 7f8a437fe640 1 -- 172.21.15.40:0/475244499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a38032440 con 0x7f8a4415f380 2024-04-26T21:43:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.443+0000 7f8a437fe640 1 -- 172.21.15.40:0/475244499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38032810 con 0x7f8a4415f380 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/475244499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a4415f380 msgr2=0x7f8a440cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/475244499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a4415f380 0x7f8a440cb440 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f8a440c4410 tx=0x7f8a3802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/475244499 shutdown_connections 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/475244499 >> v2:172.21.15.184:3300/0 conn(0x7f8a440d0620 0x7f8a440cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/475244499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a44073b70 0x7f8a440cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/475244499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a4415f380 0x7f8a440cb440 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/475244499 >> 172.21.15.40:0/475244499 conn(0x7f8a440d0be0 msgr2=0x7f8a440d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/475244499 shutdown_connections 2024-04-26T21:43:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.444+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/475244499 wait complete. 2024-04-26T21:43:25.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.445+0000 7f8a4ab0a640 1 Processor -- start 2024-04-26T21:43:25.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.445+0000 7f8a4ab0a640 1 -- start start 2024-04-26T21:43:25.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a4ab0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a440d0620 0x7f8a44152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a4ab0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4415f380 0x7f8a440ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a4ab0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a4415cda0 0x7f8a4415d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a4ab0a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a440739b0 con 0x7f8a440d0620 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a4ab0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44073830 con 0x7f8a4415f380 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a4ab0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a440cf320 con 0x7f8a4415cda0 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a440d0620 0x7f8a44152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a440d0620 0x7f8a44152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52232/0 (socket says 172.21.15.40:52232) 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 -- 172.21.15.40:0/970492372 learned_addr learned my addr 172.21.15.40:0/970492372 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a490a7640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a4415cda0 0x7f8a4415d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a43fff640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4415f380 0x7f8a440ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 -- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a4415cda0 msgr2=0x7f8a4415d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a4415cda0 0x7f8a4415d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 -- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4415f380 msgr2=0x7f8a440ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4415f380 0x7f8a440ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 -- 172.21.15.40:0/970492372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a440c9270 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a490a7640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a4415cda0 0x7f8a4415d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.446+0000 7f8a488a6640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a440d0620 0x7f8a44152100 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8a3802a220 tx=0x7f8a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.447+0000 7f8a417fa640 1 -- 172.21.15.40:0/970492372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38041020 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.447+0000 7f8a417fa640 1 -- 172.21.15.40:0/970492372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a380306c0 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.447+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a440c9550 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.447+0000 7f8a417fa640 1 -- 172.21.15.40:0/970492372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38037040 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.447+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a440c9b10 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.448+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a440694f0 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.448+0000 7f8a417fa640 1 -- 172.21.15.40:0/970492372 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a38044020 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.449+0000 7f8a417fa640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a2806c8f0 0x7f8a2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.449+0000 7f8a417fa640 1 -- 172.21.15.40:0/970492372 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6819+0+0 (secure 0 0 0) 0x7f8a380ac780 con 0x7f8a440d0620 2024-04-26T21:43:25.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.449+0000 7f8a43fff640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a2806c8f0 0x7f8a2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.450+0000 7f8a43fff640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a2806c8f0 0x7f8a2806edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8a4415e090 tx=0x7f8a30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:25.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.453+0000 7f8a417fa640 1 -- 172.21.15.40:0/970492372 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a3804b030 con 0x7f8a440d0620 2024-04-26T21:43:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.679+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8a440c7840 con 0x7f8a440d0620 2024-04-26T21:43:25.679 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:25.679+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:43:25.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.681+0000 7f8a417fa640 1 -- 172.21.15.40:0/970492372 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v136) ==== 161+0+0 (secure 0 0 0) 0x7f8a3807a7e0 con 0x7f8a440d0620 2024-04-26T21:43:25.681 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:43:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a2806c8f0 msgr2=0x7f8a2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a2806c8f0 0x7f8a2806edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8a4415e090 tx=0x7f8a30008040 comp rx=0 tx=0).stop 2024-04-26T21:43:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a440d0620 msgr2=0x7f8a44152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a440d0620 0x7f8a44152100 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8a3802a220 tx=0x7f8a380046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 shutdown_connections 2024-04-26T21:43:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a2806c8f0 0x7f8a2806edb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a4415cda0 0x7f8a4415d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4415f380 0x7f8a440ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 --2- 172.21.15.40:0/970492372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a440d0620 0x7f8a44152100 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 >> 172.21.15.40:0/970492372 conn(0x7f8a440d0be0 msgr2=0x7f8a4406e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 shutdown_connections 2024-04-26T21:43:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.684+0000 7f8a4ab0a640 1 -- 172.21.15.40:0/970492372 wait complete. 2024-04-26T21:43:25.685 INFO:tasks.ceph.mds.c.smithi040.stderr: -710> 2024-04-26T21:43:19.842+0000 7fa24d98b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:43:25.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:43:25.713 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:43:25.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.801+0000 7f397dec8640 1 Processor -- start 2024-04-26T21:43:25.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.801+0000 7f397dec8640 1 -- start start 2024-04-26T21:43:25.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f397dec8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3978162880 0x7f39780d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f397dec8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f39780d3bc0 0x7f39780b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f397dec8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39780b7e50 con 0x7f39780b5f20 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f397dec8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f39780d0580 con 0x7f3978162880 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f397dec8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f39780d1800 con 0x7f39780d3bc0 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f39777fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3978162880 0x7f39780d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f3976ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f39780b5f20 0x7f39780d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60180/0 (socket says 172.21.15.40:60180) 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f3976ffd640 1 -- 172.21.15.40:0/3542600255 learned_addr learned my addr 172.21.15.40:0/3542600255 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f3977fff640 1 --2- 172.21.15.40:0/3542600255 >> v2:172.21.15.184:3300/0 conn(0x7f39780d3bc0 0x7f39780b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f39777fe640 1 -- 172.21.15.40:0/3542600255 >> v2:172.21.15.184:3300/0 conn(0x7f39780d3bc0 msgr2=0x7f39780b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f39777fe640 1 --2- 172.21.15.40:0/3542600255 >> v2:172.21.15.184:3300/0 conn(0x7f39780d3bc0 0x7f39780b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f39777fe640 1 -- 172.21.15.40:0/3542600255 >> v1:172.21.15.40:6789/0 conn(0x7f39780b5f20 legacy=0x7f39780d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f39767fc640 1 -- 172.21.15.40:0/3542600255 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3808039474 0 0) 0x7f39780b7e50 con 0x7f39780b5f20 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.802+0000 7f39777fe640 1 -- 172.21.15.40:0/3542600255 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39780d07d0 con 0x7f3978162880 2024-04-26T21:43:25.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.803+0000 7f39777fe640 1 --2- 172.21.15.40:0/3542600255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f39780d2cb0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f39780d11d0 tx=0x7f396802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9f92a60e2f1ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.803+0000 7f39767fc640 1 -- 172.21.15.40:0/3542600255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968037070 con 0x7f3978162880 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.803+0000 7f39767fc640 1 -- 172.21.15.40:0/3542600255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3968032440 con 0x7f3978162880 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.803+0000 7f39767fc640 1 -- 172.21.15.40:0/3542600255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968032810 con 0x7f3978162880 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.804+0000 7f397dec8640 1 -- 172.21.15.40:0/3542600255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 msgr2=0x7f39780d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.804+0000 7f397dec8640 1 --2- 172.21.15.40:0/3542600255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f39780d2cb0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f39780d11d0 tx=0x7f396802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.804+0000 7f397dec8640 1 -- 172.21.15.40:0/3542600255 shutdown_connections 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.804+0000 7f397dec8640 1 --2- 172.21.15.40:0/3542600255 >> v2:172.21.15.184:3300/0 conn(0x7f39780d3bc0 0x7f39780b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.804+0000 7f397dec8640 1 --2- 172.21.15.40:0/3542600255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f39780d2cb0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.804+0000 7f397dec8640 1 -- 172.21.15.40:0/3542600255 >> 172.21.15.40:0/3542600255 conn(0x7f39780d4f60 msgr2=0x7f39780d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.804+0000 7f397dec8640 1 -- 172.21.15.40:0/3542600255 shutdown_connections 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.804+0000 7f397dec8640 1 -- 172.21.15.40:0/3542600255 wait complete. 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.805+0000 7f397dec8640 1 Processor -- start 2024-04-26T21:43:25.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.805+0000 7f397dec8640 1 -- start start 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f397dec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39780b5f20 0x7f397815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f397dec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39780d3bc0 0x7f39780b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f397dec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f397815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f397dec8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3978162c60 con 0x7f3978162880 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f397dec8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39780d0d90 con 0x7f39780d3bc0 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f397dec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3978162de0 con 0x7f39780b5f20 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f397815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f397815f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52250/0 (socket says 172.21.15.40:52250) 2024-04-26T21:43:25.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 -- 172.21.15.40:0/3861700911 learned_addr learned my addr 172.21.15.40:0/3861700911 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f39777fe640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39780b5f20 0x7f397815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3976ffd640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39780d3bc0 0x7f39780b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 -- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39780b5f20 msgr2=0x7f397815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39780b5f20 0x7f397815c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 -- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39780d3bc0 msgr2=0x7f39780b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39780d3bc0 0x7f39780b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 -- 172.21.15.40:0/3861700911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781f4a50 con 0x7f3978162880 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f39777fe640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39780b5f20 0x7f397815c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3976ffd640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39780d3bc0 0x7f39780b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f3977fff640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f397815f550 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f397000cc00 tx=0x7f397000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f395ffff640 1 -- 172.21.15.40:0/3861700911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397000ce10 con 0x7f3978162880 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f395ffff640 1 -- 172.21.15.40:0/3861700911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3970003990 con 0x7f3978162880 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f395ffff640 1 -- 172.21.15.40:0/3861700911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970003d60 con 0x7f3978162880 2024-04-26T21:43:25.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.806+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781f4d30 con 0x7f3978162880 2024-04-26T21:43:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.807+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781f5810 con 0x7f3978162880 2024-04-26T21:43:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.808+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39780694f0 con 0x7f3978162880 2024-04-26T21:43:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.808+0000 7f395ffff640 1 -- 172.21.15.40:0/3861700911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3970003f00 con 0x7f3978162880 2024-04-26T21:43:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.808+0000 7f395ffff640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f396406c920 0x7f396406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.809+0000 7f395ffff640 1 -- 172.21.15.40:0/3861700911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6905+0+0 (secure 0 0 0) 0x7f397008c480 con 0x7f3978162880 2024-04-26T21:43:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.809+0000 7f39777fe640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f396406c920 0x7f396406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.809+0000 7f39777fe640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f396406c920 0x7f396406ede0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3968034000 tx=0x7f396802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:25.813+0000 7f395ffff640 1 -- 172.21.15.40:0/3861700911 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3970014030 con 0x7f3978162880 2024-04-26T21:43:26.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.077+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f39780cb530 con 0x7f3978162880 2024-04-26T21:43:26.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.682+0000 7f395ffff640 1 -- 172.21.15.40:0/3861700911 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v137) ==== 107+0+0 (secure 0 0 0) 0x7f397005a400 con 0x7f3978162880 2024-04-26T21:43:26.682 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f396406c920 msgr2=0x7f396406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f396406c920 0x7f396406ede0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3968034000 tx=0x7f396802e000 comp rx=0 tx=0).stop 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 msgr2=0x7f397815f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f397815f550 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f397000cc00 tx=0x7f397000a5c0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 shutdown_connections 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f396406c920 0x7f396406ede0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3978162880 0x7f397815f550 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39780d3bc0 0x7f39780b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 --2- 172.21.15.40:0/3861700911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39780b5f20 0x7f397815c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 >> 172.21.15.40:0/3861700911 conn(0x7f39780d4f60 msgr2=0x7f39781615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:26.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 shutdown_connections 2024-04-26T21:43:26.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.685+0000 7f397dec8640 1 -- 172.21.15.40:0/3861700911 wait complete. 2024-04-26T21:43:26.698 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:43:26.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.784+0000 7f5bfb867f40 1 Processor -- start 2024-04-26T21:43:26.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.784+0000 7f5bfb867f40 1 -- start start 2024-04-26T21:43:26.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bfb867f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 0x55a2355a7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bfb867f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a2355d6af0 0x55a23567a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bfb867f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a23567f900 0x55a23567e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bfb867f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a23554fe40 con 0x55a2355d6af0 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bfb867f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a235553e80 con 0x55a23567f900 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bfb867f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a2355d6010 con 0x55a2355a6930 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bf8b62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a2355d6af0 0x55a23567a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bf8b62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a2355d6af0 0x55a23567a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52260/0 (socket says 172.21.15.40:52260) 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bf8b62640 1 -- 172.21.15.40:0/839897839 learned_addr learned my addr 172.21.15.40:0/839897839 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bf9363640 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 0x55a2355a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.785+0000 7f5bf9b64640 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a23567f900 0x55a23567e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf8b62640 1 -- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 msgr2=0x55a2355a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 0x55a2355a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf8b62640 1 -- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a23567f900 msgr2=0x55a23567e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a23567f900 0x55a23567e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf8b62640 1 -- 172.21.15.40:0/839897839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2355a7c50 con 0x55a2355d6af0 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf9b64640 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a23567f900 0x55a23567e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a2355d6af0 0x55a23567a660 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f5be400b2f0 tx=0x7f5be400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf92c90cbd7d48f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf37fe640 1 -- 172.21.15.40:0/839897839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4012020 con 0x55a2355d6af0 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf37fe640 1 -- 172.21.15.40:0/839897839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be400e670 con 0x55a2355d6af0 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bf37fe640 1 -- 172.21.15.40:0/839897839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be400ea20 con 0x55a2355d6af0 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.786+0000 7f5bfb867f40 1 -- 172.21.15.40:0/839897839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a2355d6af0 msgr2=0x55a23567a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a2355d6af0 0x55a23567a660 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f5be400b2f0 tx=0x7f5be400b7b0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 -- 172.21.15.40:0/839897839 shutdown_connections 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a23567f900 0x55a23567e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a2355d6af0 0x55a23567a660 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/839897839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 0x55a2355a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 -- 172.21.15.40:0/839897839 >> 172.21.15.40:0/839897839 conn(0x55a2355a7330 msgr2=0x55a2355a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 -- 172.21.15.40:0/839897839 shutdown_connections 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 -- 172.21.15.40:0/839897839 wait complete. 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 Processor -- start 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.787+0000 7f5bfb867f40 1 -- start start 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bfb867f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 0x55a23567d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bfb867f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2355d6af0 0x55a235672500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bfb867f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a23567f900 0x55a2355ebf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bfb867f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a2355d3880 con 0x55a23567f900 2024-04-26T21:43:26.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bfb867f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a2355f1220 con 0x55a2355d6af0 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bfb867f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a2355f1500 con 0x55a2355a6930 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9b64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a23567f900 0x55a2355ebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9b64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a23567f900 0x55a2355ebf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52268/0 (socket says 172.21.15.40:52268) 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 learned_addr learned my addr 172.21.15.40:0/914050554 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9363640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 0x55a23567d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2355d6af0 0x55a235672500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 msgr2=0x55a23567d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9b64640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 0x55a23567d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2355d6af0 msgr2=0x55a235672500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9b64640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2355d6af0 0x55a235672500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.788+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a235711260 con 0x55a23567f900 2024-04-26T21:43:26.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.789+0000 7f5bf9363640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2355a6930 0x55a23567d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:26.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.789+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2355d6af0 0x55a235672500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:26.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.789+0000 7f5bf9b64640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a23567f900 0x55a2355ebf50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5bec025a90 tx=0x7f5bec0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.789+0000 7f5bf27fc640 1 -- 172.21.15.40:0/914050554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec06a070 con 0x55a23567f900 2024-04-26T21:43:26.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.789+0000 7f5bf27fc640 1 -- 172.21.15.40:0/914050554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec05a450 con 0x55a23567f900 2024-04-26T21:43:26.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.789+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a235711540 con 0x55a23567f900 2024-04-26T21:43:26.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.789+0000 7f5bf27fc640 1 -- 172.21.15.40:0/914050554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec055040 con 0x55a23567f900 2024-04-26T21:43:26.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.789+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23567ef60 con 0x55a23567f900 2024-04-26T21:43:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.790+0000 7f5bf27fc640 1 -- 172.21.15.40:0/914050554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bec06d070 con 0x55a23567f900 2024-04-26T21:43:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.791+0000 7f5bf27fc640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bdc06b2d0 0x7f5bdc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.791+0000 7f5bf27fc640 1 -- 172.21.15.40:0/914050554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6905+0+0 (secure 0 0 0) 0x7f5bec0ced80 con 0x55a23567f900 2024-04-26T21:43:26.795 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:43:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.791+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a2355d4130 0x55a2355edff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.791+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2355ec490 con 0x55a2355d4130 2024-04-26T21:43:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.791+0000 7f5bf9363640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bdc06b2d0 0x7f5bdc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.791+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a2355d4130 0x55a2355edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.792+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a2355d4130 0x55a2355edff0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.792+0000 7f5bf9363640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bdc06b2d0 0x7f5bdc06d790 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400f2e0 tx=0x7f5bf40016f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.793+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v103'226 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.793+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235719be0 con 0x55a2355d4130 2024-04-26T21:43:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.794+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v103'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.794+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235719f80 con 0x55a2355d4130 2024-04-26T21:43:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.795+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v103'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.798 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:43:26.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.795+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a23571a320 con 0x55a2355d4130 2024-04-26T21:43:26.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.796+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v103'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.796+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a23571a720 con 0x55a2355d4130 2024-04-26T21:43:26.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.797+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v103'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.797+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235722eb0 con 0x55a2355d4130 2024-04-26T21:43:26.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.798+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v103'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.798+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235723250 con 0x55a2355d4130 2024-04-26T21:43:26.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.799+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v103'232 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.799+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357235f0 con 0x55a2355d4130 2024-04-26T21:43:26.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.801+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v103'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.801+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357235f0 con 0x55a2355d4130 2024-04-26T21:43:26.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.802+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v103'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.801 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:43:26.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.802+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235724600 con 0x55a2355d4130 2024-04-26T21:43:26.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.803+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v103'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.803+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235724600 con 0x55a2355d4130 2024-04-26T21:43:26.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.804+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v103'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.804+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357235f0 con 0x55a2355d4130 2024-04-26T21:43:26.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.805+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v103'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.805+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357235f0 con 0x55a2355d4130 2024-04-26T21:43:26.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.806+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v103'238 uv213 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.806+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357235f0 con 0x55a2355d4130 2024-04-26T21:43:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.807+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v103'239 uv239 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.807+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357235f0 con 0x55a2355d4130 2024-04-26T21:43:26.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.808+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v103'240 uv240 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.808 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:43:26.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.808+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235724600 con 0x55a2355d4130 2024-04-26T21:43:26.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.809+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v103'241 uv241 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.809+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235724600 con 0x55a2355d4130 2024-04-26T21:43:26.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.810+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v103'242 uv242 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.810+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357235f0 con 0x55a2355d4130 2024-04-26T21:43:26.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.811+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v103'243 uv243 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.811+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.812+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v103'244 uv244 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.812+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.813+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v103'245 uv219 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.813+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.814+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v103'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.814+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.815+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v103'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.815 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:43:26.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.815+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235724ab0 con 0x55a2355d4130 2024-04-26T21:43:26.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.816+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v103'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.816+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235724ab0 con 0x55a2355d4130 2024-04-26T21:43:26.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.817+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v103'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.817+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.818+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v103'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.818+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.819+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v103'251 uv225 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.819+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.820+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v103'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.820+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.820 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:43:26.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.821+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v103'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.821+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235724ab0 con 0x55a2355d4130 2024-04-26T21:43:26.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.822+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v103'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.822+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235724ab0 con 0x55a2355d4130 2024-04-26T21:43:26.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.823+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v103'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.823+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235723aa0 con 0x55a2355d4130 2024-04-26T21:43:26.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.824+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v103'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2355d4130 2024-04-26T21:43:26.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.824+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a235723fb0 0x55a2357263a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.824+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357268e0 con 0x55a235723fb0 2024-04-26T21:43:26.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.824+0000 7f5bf9363640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a235723fb0 0x55a2357263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.824+0000 7f5bf9363640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a235723fb0 0x55a2357263a0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.826+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v103'177 uv156 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.826+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235726c80 con 0x55a235723fb0 2024-04-26T21:43:26.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.827+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v103'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.827+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357270e0 con 0x55a235723fb0 2024-04-26T21:43:26.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.828+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v103'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.828 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:43:26.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.828+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357279d0 con 0x55a235723fb0 2024-04-26T21:43:26.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.830+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v103'180 uv180 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.830+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235727d70 con 0x55a235723fb0 2024-04-26T21:43:26.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.831+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v103'181 uv181 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.831+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235728160 con 0x55a235723fb0 2024-04-26T21:43:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.832+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v103'182 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.832+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235728550 con 0x55a235723fb0 2024-04-26T21:43:26.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.833+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v103'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.833+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235728940 con 0x55a235723fb0 2024-04-26T21:43:26.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.834+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v103'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.834 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:43:26.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.834+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235728940 con 0x55a235723fb0 2024-04-26T21:43:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.836+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v103'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.836+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.837+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v103'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.837+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.838+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v103'187 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.838+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.839+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v103'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.839+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.840+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v103'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.840 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:43:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.840+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.842+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v103'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.842+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.843+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v103'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.843+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.844+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v103'192 uv172 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.844+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.845+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v103'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.845+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357288f0 con 0x55a235723fb0 2024-04-26T21:43:26.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.846+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v103'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.846 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:43:26.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.846+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235728930 con 0x55a235723fb0 2024-04-26T21:43:26.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.847+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v103'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.847+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235729dd0 con 0x55a235723fb0 2024-04-26T21:43:26.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.849+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v103'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.849+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235728ce0 con 0x55a235723fb0 2024-04-26T21:43:26.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.850+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v103'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235723fb0 2024-04-26T21:43:26.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.850+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a2357293b0 0x55a23572bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.850+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235729890 con 0x55a2357293b0 2024-04-26T21:43:26.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.850+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a2357293b0 0x55a23572bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.850+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a2357293b0 0x55a23572bd70 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.851+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v103'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.852+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a23572c2b0 con 0x55a2357293b0 2024-04-26T21:43:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.853+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v103'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.853+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a23572c2b0 con 0x55a2357293b0 2024-04-26T21:43:26.853 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:43:26.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.854+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v103'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.854+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a23572d2c0 con 0x55a2357293b0 2024-04-26T21:43:26.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.855+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v103'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.855+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a23572d2c0 con 0x55a2357293b0 2024-04-26T21:43:26.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.856+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v103'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.856+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a23572c2b0 con 0x55a2357293b0 2024-04-26T21:43:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.857+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v103'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.857+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a23572c930 0x55a23572cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.857+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a23572f710 con 0x55a23572c930 2024-04-26T21:43:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.857+0000 7f5bf9b64640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a23572c930 0x55a23572cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.857+0000 7f5bf9b64640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a23572c930 0x55a23572cd30 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.859+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v103'41 uv40 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.859+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a23572fab0 con 0x55a23572c930 2024-04-26T21:43:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.864+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v103'42 uv42 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.864+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a23572fe50 con 0x55a23572c930 2024-04-26T21:43:26.865 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:43:26.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.866+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v103'43 uv43 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.866+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235730710 con 0x55a23572c930 2024-04-26T21:43:26.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.867+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.867+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a235730fc0 0x55a2357333b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.867+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235733940 con 0x55a235730fc0 2024-04-26T21:43:26.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.867+0000 7f5bf9363640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a235730fc0 0x55a2357333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.868+0000 7f5bf9363640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a235730fc0 0x55a2357333b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.869+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v103'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.869+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235733d10 con 0x55a235730fc0 2024-04-26T21:43:26.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.871+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v103'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.871+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235733d10 con 0x55a235730fc0 2024-04-26T21:43:26.872 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:43:26.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.872+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v103'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.872+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734d20 con 0x55a235730fc0 2024-04-26T21:43:26.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.873+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v103'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.874+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734d20 con 0x55a235730fc0 2024-04-26T21:43:26.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.875+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v103'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.875+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235733d10 con 0x55a235730fc0 2024-04-26T21:43:26.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.876+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v103'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.876+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235733d10 con 0x55a23572c930 2024-04-26T21:43:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.877+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v103'83 uv75 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.877+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357340d0 con 0x55a23572c930 2024-04-26T21:43:26.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.879+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v103'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.879+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357344c0 con 0x55a23572c930 2024-04-26T21:43:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.880+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v103'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.880 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:43:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.880+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357348b0 con 0x55a23572c930 2024-04-26T21:43:26.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.883+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v103'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.883+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734d30 con 0x55a23572c930 2024-04-26T21:43:26.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.884+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v103'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.884+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734d30 con 0x55a23572c930 2024-04-26T21:43:26.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.886+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v103'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.887+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357348b0 con 0x55a23572c930 2024-04-26T21:43:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.889+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v103'89 uv82 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.889+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357348b0 con 0x55a23572c930 2024-04-26T21:43:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.892+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v103'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.892+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734d00 con 0x55a23572c930 2024-04-26T21:43:26.895 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:43:26.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.895+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v103'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.895+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a23572c930 2024-04-26T21:43:26.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.898+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v103'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.898+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a23572c930 2024-04-26T21:43:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.899+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v103'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.899+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a23572c930 2024-04-26T21:43:26.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.902+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v103'37 uv36 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.902+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a23572c930 2024-04-26T21:43:26.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.904+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v103'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.904+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a23572c930 2024-04-26T21:43:26.905 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:43:26.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.905+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v103'39 uv39 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.905+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a23572c930 2024-04-26T21:43:26.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.908+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v103'40 uv40 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.908+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a23572c930 2024-04-26T21:43:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.910+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v103'41 uv41 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5bec06c030 con 0x55a23572c930 2024-04-26T21:43:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.910+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a235730fc0 2024-04-26T21:43:26.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.912+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v103'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.912+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a235730fc0 2024-04-26T21:43:26.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.913+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v103'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.913+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734cb0 con 0x55a235730fc0 2024-04-26T21:43:26.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.913+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v103'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.913 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:43:26.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.913+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357348b0 con 0x55a235730fc0 2024-04-26T21:43:26.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.914+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v103'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.914+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.915+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v103'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.915+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.916+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v103'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.916+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.917+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v103'74 uv68 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.917+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.918+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v103'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.918+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.919+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v103'76 uv76 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.919 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:43:26.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.919+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.920+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v103'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.920+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.922+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v103'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.922+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.923+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v103'79 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.923+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.924+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v103'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.924+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.925+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v103'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.925 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:43:26.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.925+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235734860 con 0x55a235730fc0 2024-04-26T21:43:26.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.926+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v103'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.926+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357379f0 con 0x55a235730fc0 2024-04-26T21:43:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.927+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v103'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.927+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a235735710 0x55a235737f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.927+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235737b00 con 0x55a235735710 2024-04-26T21:43:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.927+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a235735710 0x55a235737f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.928+0000 7f5bf8b62640 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a235735710 0x55a235737f70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.929+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v103'36 uv34 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a235735710 2024-04-26T21:43:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.929+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a235737b00 con 0x55a235735710 2024-04-26T21:43:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.930+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v103'37 uv37 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a235735710 2024-04-26T21:43:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.930+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235737b00 con 0x55a235735710 2024-04-26T21:43:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.933+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v103'38 uv38 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a235735710 2024-04-26T21:43:26.933 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:43:26.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.933+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a235735710 2024-04-26T21:43:26.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.935+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v103'39 uv39 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a235735710 2024-04-26T21:43:26.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.935+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a235730fc0 2024-04-26T21:43:26.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.936+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v103'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.936+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a235730fc0 2024-04-26T21:43:26.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.937+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v103'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.938+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a235730fc0 2024-04-26T21:43:26.938 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:43:26.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.938+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v103'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.939+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357394c0 con 0x55a235730fc0 2024-04-26T21:43:26.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.940+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v103'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.940+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357394c0 con 0x55a235730fc0 2024-04-26T21:43:26.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.941+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v103'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.941+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a235730fc0 2024-04-26T21:43:26.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.942+0000 7f5bf9363640 1 -- 172.21.15.40:0/914050554 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v103'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5bf400e040 con 0x55a235730fc0 2024-04-26T21:43:26.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.942+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.943+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v103'37 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.943+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.944+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v103'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.944+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.945 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:43:26.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.945+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v103'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.945+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.947+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.947+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.948+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.948+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.949+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v103'95 uv86 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.949+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.950+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v103'96 uv96 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.950+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.950 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:43:26.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.951+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v103'97 uv97 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.951+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.952+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v103'98 uv98 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.952+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.953+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v103'99 uv99 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.953+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.955+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v103'100 uv93 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.955+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.956+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v103'101 uv101 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.956+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.956 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:43:26.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.957+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v103'102 uv102 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.957+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357394c0 con 0x55a2357293b0 2024-04-26T21:43:26.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.958+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v103'103 uv103 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.958+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357394c0 con 0x55a2357293b0 2024-04-26T21:43:26.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.959+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v103'104 uv104 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.959+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a2357384b0 con 0x55a2357293b0 2024-04-26T21:43:26.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.960+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v103'105 uv105 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.960+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55a235738960 con 0x55a2357293b0 2024-04-26T21:43:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bf8b62640 1 -- 172.21.15.40:0/914050554 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v103'106 uv106 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5be400e820 con 0x55a2357293b0 2024-04-26T21:43:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a235723fb0 msgr2=0x55a2357263a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a235723fb0 0x55a2357263a0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a2355d4130 msgr2=0x55a2355edff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a2355d4130 0x55a2355edff0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a23572c930 msgr2=0x55a23572cd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a23572c930 0x55a23572cd30 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a235735710 msgr2=0x55a235737f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a235735710 0x55a235737f70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a2357293b0 msgr2=0x55a23572bd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 reap_dead start 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a2357293b0 0x55a23572bd70 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a235730fc0 msgr2=0x55a2357333b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a235730fc0 0x55a2357333b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bf9b64640 1 -- 172.21.15.40:0/914050554 reap_dead start 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bdc06b2d0 msgr2=0x7f5bdc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bdc06b2d0 0x7f5bdc06d790 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400f2e0 tx=0x7f5bf40016f0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a23567f900 msgr2=0x55a2355ebf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a23567f900 0x55a2355ebf50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5bec025a90 tx=0x7f5bec0033a0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 shutdown_connections 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bdc06b2d0 0x7f5bdc06d790 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.961+0000 7f5bfb867f40 1 --2- 172.21.15.40:0/914050554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a23567f900 0x55a2355ebf50 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.962+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 >> 172.21.15.40:0/914050554 conn(0x55a2355a7330 msgr2=0x55a2355f1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:26.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.962+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 shutdown_connections 2024-04-26T21:43:26.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:26.962+0000 7f5bfb867f40 1 -- 172.21.15.40:0/914050554 wait complete. 2024-04-26T21:43:26.965 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds_snaptable 2024-04-26T21:43:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.047+0000 7f1dcbeb1f40 1 Processor -- start 2024-04-26T21:43:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.047+0000 7f1dcbeb1f40 1 -- start start 2024-04-26T21:43:27.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.047+0000 7f1dcbeb1f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52baa80 0x55dce52d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.047+0000 7f1dcbeb1f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52bae60 0x55dce522cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.047+0000 7f1dcbeb1f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bdf30 0x55dce52442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.047+0000 7f1dcbeb1f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dce52472d0 con 0x55dce52bae60 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.047+0000 7f1dcbeb1f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dce51a6e40 con 0x55dce52bdf30 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.047+0000 7f1dcbeb1f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55dce5247570 con 0x55dce52baa80 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52bae60 0x55dce522cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52bae60 0x55dce522cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52286/0 (socket says 172.21.15.40:52286) 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 -- 172.21.15.40:0/1589275351 learned_addr learned my addr 172.21.15.40:0/1589275351 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc99ab640 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52baa80 0x55dce52d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dca1ac640 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bdf30 0x55dce52442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 -- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52baa80 msgr2=0x55dce52d4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52baa80 0x55dce52d4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 -- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bdf30 msgr2=0x55dce52442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bdf30 0x55dce52442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 -- 172.21.15.40:0/1589275351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce52bd9c0 con 0x55dce52bae60 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dca1ac640 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bdf30 0x55dce52442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc91aa640 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52bae60 0x55dce522cfe0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1db400a7a0 tx=0x7f1db400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49289abeecedc9da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc3fff640 1 -- 172.21.15.40:0/1589275351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4019070 con 0x55dce52bae60 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc3fff640 1 -- 172.21.15.40:0/1589275351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db4007600 con 0x55dce52bae60 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.048+0000 7f1dc3fff640 1 -- 172.21.15.40:0/1589275351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40079d0 con 0x55dce52bae60 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/1589275351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52bae60 msgr2=0x55dce522cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52bae60 0x55dce522cfe0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1db400a7a0 tx=0x7f1db400a890 comp rx=0 tx=0).stop 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/1589275351 shutdown_connections 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bdf30 0x55dce52442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52bae60 0x55dce522cfe0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 --2- 172.21.15.40:0/1589275351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52baa80 0x55dce52d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/1589275351 >> 172.21.15.40:0/1589275351 conn(0x55dce51fdab0 msgr2=0x55dce51fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/1589275351 shutdown_connections 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/1589275351 wait complete. 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 Processor -- start 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.049+0000 7f1dcbeb1f40 1 -- start start 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dcbeb1f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52baa80 0x55dce52d6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dcbeb1f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bae60 0x55dce5370660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dcbeb1f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52bdf30 0x55dce52b96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dcbeb1f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dce522d930 con 0x55dce52baa80 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dcbeb1f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dce52bdca0 con 0x55dce52bae60 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dcbeb1f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55dce52bcfb0 con 0x55dce52bdf30 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc99ab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52baa80 0x55dce52d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc99ab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52baa80 0x55dce52d6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52302/0 (socket says 172.21.15.40:52302) 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc99ab640 1 -- 172.21.15.40:0/2823725592 learned_addr learned my addr 172.21.15.40:0/2823725592 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc91aa640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bae60 0x55dce5370660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dca1ac640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52bdf30 0x55dce52b96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc99ab640 1 -- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52bdf30 msgr2=0x55dce52b96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc99ab640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dce52bdf30 0x55dce52b96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc99ab640 1 -- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bae60 msgr2=0x55dce5370660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc99ab640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bae60 0x55dce5370660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc99ab640 1 -- 172.21.15.40:0/2823725592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce5220450 con 0x55dce52baa80 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.050+0000 7f1dc91aa640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dce52bae60 0x55dce5370660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.051+0000 7f1dc99ab640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52baa80 0x55dce52d6660 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc045b30 tx=0x7f1dbc003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.051+0000 7f1dc2ffd640 1 -- 172.21.15.40:0/2823725592 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc055040 con 0x55dce52baa80 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.051+0000 7f1dc2ffd640 1 -- 172.21.15.40:0/2823725592 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc065680 con 0x55dce52baa80 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.051+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dce5370ba0 con 0x55dce52baa80 2024-04-26T21:43:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.051+0000 7f1dc2ffd640 1 -- 172.21.15.40:0/2823725592 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc065a70 con 0x55dce52baa80 2024-04-26T21:43:27.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.051+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce5371120 con 0x55dce52baa80 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.052+0000 7f1dc2ffd640 1 -- 172.21.15.40:0/2823725592 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dbc058020 con 0x55dce52baa80 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.053+0000 7f1dc2ffd640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dac06b2d0 0x7f1dac06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.053+0000 7f1dc2ffd640 1 -- 172.21.15.40:0/2823725592 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6905+0+0 (secure 0 0 0) 0x7f1dbc0ced70 con 0x55dce52baa80 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.053+0000 7f1dcbeb1f40 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dce5222430 0x55dce5220c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.053+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55dce5221150 con 0x55dce5222430 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.053+0000 7f1dc91aa640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dac06b2d0 0x7f1dac06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.053+0000 7f1dca1ac640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dce5222430 0x55dce5220c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.053+0000 7f1dc91aa640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dac06b2d0 0x7f1dac06d790 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x55dce52bdf00 tx=0x7f1db400c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.054+0000 7f1dca1ac640 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dce5222430 0x55dce5220c10 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dca1ac640 1 -- 172.21.15.40:0/2823725592 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds_snaptable [delete] v103'42 uv42 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1dc400a440 con 0x55dce5222430 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dce5222430 msgr2=0x55dce5220c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dce5222430 0x55dce5220c10 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dac06b2d0 msgr2=0x7f1dac06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dac06b2d0 0x7f1dac06d790 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x55dce52bdf00 tx=0x7f1db400c000 comp rx=0 tx=0).stop 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52baa80 msgr2=0x55dce52d6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 --2- 172.21.15.40:0/2823725592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dce52baa80 0x55dce52d6660 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc045b30 tx=0x7f1dbc003330 comp rx=0 tx=0).stop 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dca1ac640 1 -- 172.21.15.40:0/2823725592 reap_dead start 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 shutdown_connections 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 >> 172.21.15.40:0/2823725592 conn(0x55dce51fdab0 msgr2=0x55dce522cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 shutdown_connections 2024-04-26T21:43:27.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.055+0000 7f1dcbeb1f40 1 -- 172.21.15.40:0/2823725592 wait complete. 2024-04-26T21:43:27.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:43:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.202+0000 7fbe3fdf0640 1 Processor -- start 2024-04-26T21:43:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.202+0000 7fbe3fdf0640 1 -- start start 2024-04-26T21:43:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3fdf0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbe380d1a00 0x7fbe380d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3fdf0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe380b40d0 0x7fbe380b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3fdf0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe380d0a10 con 0x7fbe380d2320 2024-04-26T21:43:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3fdf0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbe380d0cf0 con 0x7fbe380b40d0 2024-04-26T21:43:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3fdf0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbe380d0fd0 con 0x7fbe380d1a00 2024-04-26T21:43:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3e38d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe380b40d0 0x7fbe380b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3e38d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe380b40d0 0x7fbe380b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52304/0 (socket says 172.21.15.40:52304) 2024-04-26T21:43:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3e38d640 1 -- 172.21.15.40:0/2932631715 learned_addr learned my addr 172.21.15.40:0/2932631715 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.203+0000 7fbe3db8c640 1 --2- 172.21.15.40:0/2932631715 >> v2:172.21.15.184:3300/0 conn(0x7fbe380d1a00 0x7fbe380d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.204+0000 7fbe3e38d640 1 -- 172.21.15.40:0/2932631715 >> v2:172.21.15.184:3300/0 conn(0x7fbe380d1a00 msgr2=0x7fbe380d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.204+0000 7fbe3e38d640 1 --2- 172.21.15.40:0/2932631715 >> v2:172.21.15.184:3300/0 conn(0x7fbe380d1a00 0x7fbe380d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.204+0000 7fbe3e38d640 1 -- 172.21.15.40:0/2932631715 >> v1:172.21.15.184:6789/0 conn(0x7fbe380d2320 legacy=0x7fbe381625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.204+0000 7fbe3e38d640 1 -- 172.21.15.40:0/2932631715 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe380d40a0 con 0x7fbe380b40d0 2024-04-26T21:43:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.204+0000 7fbe3e38d640 1 --2- 172.21.15.40:0/2932631715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380b40d0 0x7fbe380b4b00 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fbe3806c650 tx=0x7fbe2802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31a15a49ea44e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3cb8a640 1 -- 172.21.15.40:0/2932631715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28036070 con 0x7fbe380b40d0 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3cb8a640 1 -- 172.21.15.40:0/2932631715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe28031600 con 0x7fbe380b40d0 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3cb8a640 1 -- 172.21.15.40:0/2932631715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280319d0 con 0x7fbe380b40d0 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2932631715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380b40d0 msgr2=0x7fbe380b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2932631715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380b40d0 0x7fbe380b4b00 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fbe3806c650 tx=0x7fbe2802a610 comp rx=0 tx=0).stop 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2932631715 shutdown_connections 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2932631715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380b40d0 0x7fbe380b4b00 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2932631715 >> v2:172.21.15.184:3300/0 conn(0x7fbe380d1a00 0x7fbe380d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2932631715 >> 172.21.15.40:0/2932631715 conn(0x7fbe380d4fd0 msgr2=0x7fbe380d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2932631715 shutdown_connections 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.205+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2932631715 wait complete. 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.206+0000 7fbe3fdf0640 1 Processor -- start 2024-04-26T21:43:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.206+0000 7fbe3fdf0640 1 -- start start 2024-04-26T21:43:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3fdf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe380b40d0 0x7fbe380cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3fdf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe380d1a00 0x7fbe380cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3fdf0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380d2320 0x7fbe380cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3fdf0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbe380b5040 con 0x7fbe380d2320 2024-04-26T21:43:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3fdf0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38162f50 con 0x7fbe380b40d0 2024-04-26T21:43:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3fdf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe380b51c0 con 0x7fbe380d1a00 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3e38d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380d2320 0x7fbe380cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3e38d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380d2320 0x7fbe380cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52308/0 (socket says 172.21.15.40:52308) 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3e38d640 1 -- 172.21.15.40:0/2820662548 learned_addr learned my addr 172.21.15.40:0/2820662548 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3db8c640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe380b40d0 0x7fbe380cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.207+0000 7fbe3d38b640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe380d1a00 0x7fbe380cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3e38d640 1 -- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe380d1a00 msgr2=0x7fbe380cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3e38d640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe380d1a00 0x7fbe380cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3e38d640 1 -- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe380b40d0 msgr2=0x7fbe380cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3e38d640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe380b40d0 0x7fbe380cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3e38d640 1 -- 172.21.15.40:0/2820662548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe381f5310 con 0x7fbe380d2320 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3d38b640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe380d1a00 0x7fbe380cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3db8c640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe380b40d0 0x7fbe380cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3e38d640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380d2320 0x7fbe380cd510 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009f40 tx=0x7fbe28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe367fc640 1 -- 172.21.15.40:0/2820662548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28036050 con 0x7fbe380d2320 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe367fc640 1 -- 172.21.15.40:0/2820662548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe28042070 con 0x7fbe380d2320 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe381f55f0 con 0x7fbe380d2320 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe367fc640 1 -- 172.21.15.40:0/2820662548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28007c70 con 0x7fbe380d2320 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.208+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe381f5bb0 con 0x7fbe380d2320 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.209+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe380694f0 con 0x7fbe380d2320 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.210+0000 7fbe367fc640 1 -- 172.21.15.40:0/2820662548 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe28047460 con 0x7fbe380d2320 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.210+0000 7fbe367fc640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe2406c8f0 0x7fbe2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.210+0000 7fbe367fc640 1 -- 172.21.15.40:0/2820662548 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6905+0+0 (secure 0 0 0) 0x7fbe280ace80 con 0x7fbe380d2320 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.210+0000 7fbe3db8c640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe2406c8f0 0x7fbe2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.211+0000 7fbe3db8c640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe2406c8f0 0x7fbe2406edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbe380cbe90 tx=0x7fbe30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.214+0000 7fbe367fc640 1 -- 172.21.15.40:0/2820662548 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe28045360 con 0x7fbe380d2320 2024-04-26T21:43:27.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.440+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbe381f5df0 con 0x7fbe380d2320 2024-04-26T21:43:27.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.888+0000 7fbe367fc640 1 -- 172.21.15.40:0/2820662548 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v138) ==== 162+0+0 (secure 0 0 0) 0x7fbe2807ae90 con 0x7fbe380d2320 2024-04-26T21:43:27.888 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe2406c8f0 msgr2=0x7fbe2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe2406c8f0 0x7fbe2406edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbe380cbe90 tx=0x7fbe30008040 comp rx=0 tx=0).stop 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380d2320 msgr2=0x7fbe380cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380d2320 0x7fbe380cd510 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009f40 tx=0x7fbe28004a40 comp rx=0 tx=0).stop 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 shutdown_connections 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbe2406c8f0 0x7fbe2406edb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe380d2320 0x7fbe380cd510 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe380d1a00 0x7fbe380cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 --2- 172.21.15.40:0/2820662548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe380b40d0 0x7fbe380cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 >> 172.21.15.40:0/2820662548 conn(0x7fbe380d4fd0 msgr2=0x7fbe380d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 shutdown_connections 2024-04-26T21:43:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:27.891+0000 7fbe3fdf0640 1 -- 172.21.15.40:0/2820662548 wait complete. 2024-04-26T21:43:27.896 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:27.897+0000 7f03983dc640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-04-26T21:43:27.896 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:27.897+0000 7f03983dc640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-04-26T21:43:27.897 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:27.897+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:43:27.897 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:27.897+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:43:27.903 INFO:tasks.ceph.mds.c.smithi040.stderr: -26> 2024-04-26T21:43:27.897+0000 7f03983dc640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-04-26T21:43:27.903 INFO:tasks.ceph.mds.c.smithi040.stderr: -24> 2024-04-26T21:43:27.897+0000 7f03983dc640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-04-26T21:43:27.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:27.928 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:43:28.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.010+0000 7f239af91640 1 Processor -- start 2024-04-26T21:43:28.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.010+0000 7f239af91640 1 -- start start 2024-04-26T21:43:28.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.010+0000 7f239af91640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f23940b5f20 0x7f23940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.010+0000 7f239af91640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23940b7e50 con 0x7f23940d3bc0 2024-04-26T21:43:28.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.010+0000 7f239af91640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23940d1800 con 0x7f2394162880 2024-04-26T21:43:28.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f239af91640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f23940d0580 con 0x7f23940b5f20 2024-04-26T21:43:28.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f2393fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f23940d3bc0 0x7f23940b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60216/0 (socket says 172.21.15.40:60216) 2024-04-26T21:43:28.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f2393fff640 1 -- 172.21.15.40:0/2929001085 learned_addr learned my addr 172.21.15.40:0/2929001085 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f239952e640 1 --2- 172.21.15.40:0/2929001085 >> v2:172.21.15.184:3300/0 conn(0x7f23940b5f20 0x7f23940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1535750062 0 0) 0x7f23940b7e50 con 0x7f23940d3bc0 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2380003520 con 0x7f23940d3bc0 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 901194030 0 0) 0x7f2380003520 con 0x7f23940d3bc0 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 >> v2:172.21.15.184:3300/0 conn(0x7f23940b5f20 msgr2=0x7f23940d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 --2- 172.21.15.40:0/2929001085 >> v2:172.21.15.184:3300/0 conn(0x7f23940b5f20 0x7f23940d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 >> v1:172.21.15.184:6789/0 conn(0x7f2394162880 legacy=0x7f23940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23940d07d0 con 0x7f23940d3bc0 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f237c001670 con 0x7f23940d3bc0 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 >> v1:172.21.15.40:6789/0 conn(0x7f23940d3bc0 legacy=0x7f23940b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039b0 0x7f2380003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2380004800 0x7f2380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800051c0 0x7f238000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f23940d0580 con 0x7f23800039b0 2024-04-26T21:43:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f238000c060 con 0x7f23800051c0 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f238000c260 con 0x7f2380004800 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.011+0000 7f2398d2d640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039b0 0x7f2380003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.012+0000 7f2393fff640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2380004800 0x7f2380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.012+0000 7f239952e640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800051c0 0x7f238000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.012+0000 7f2398d2d640 1 -- 172.21.15.40:0/2929001085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2380004800 msgr2=0x7f2380004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.012+0000 7f2398d2d640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2380004800 0x7f2380004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.012+0000 7f2398d2d640 1 -- 172.21.15.40:0/2929001085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800051c0 msgr2=0x7f238000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.012+0000 7f2398d2d640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800051c0 0x7f238000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.012+0000 7f239952e640 1 -- 172.21.15.40:0/2929001085 reap_dead start 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.012+0000 7f2398d2d640 1 -- 172.21.15.40:0/2929001085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2380003520 con 0x7f23800039b0 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f2398d2d640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039b0 0x7f2380003dd0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f2384001750 tx=0x7f2380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea47d51c7adf342 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384001370 con 0x7f23800039b0 2024-04-26T21:43:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23840052d0 con 0x7f23800039b0 2024-04-26T21:43:28.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f23937fe640 1 -- 172.21.15.40:0/2929001085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238402abc0 con 0x7f23800039b0 2024-04-26T21:43:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f239af91640 1 -- 172.21.15.40:0/2929001085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039b0 msgr2=0x7f2380003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f239af91640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039b0 0x7f2380003dd0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f2384001750 tx=0x7f2380005830 comp rx=0 tx=0).stop 2024-04-26T21:43:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f239af91640 1 -- 172.21.15.40:0/2929001085 shutdown_connections 2024-04-26T21:43:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f239af91640 1 --2- 172.21.15.40:0/2929001085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23800039b0 0x7f2380003dd0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f239af91640 1 -- 172.21.15.40:0/2929001085 >> 172.21.15.40:0/2929001085 conn(0x7f23940d4f60 msgr2=0x7f23940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f239af91640 1 -- 172.21.15.40:0/2929001085 shutdown_connections 2024-04-26T21:43:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.013+0000 7f239af91640 1 -- 172.21.15.40:0/2929001085 wait complete. 2024-04-26T21:43:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.014+0000 7f239af91640 1 Processor -- start 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.014+0000 7f239af91640 1 -- start start 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f239af91640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039b0 0x7f239415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f239af91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239415c810 0x7f23940b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f239af91640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23940b52b0 0x7f239415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f239af91640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2394162c60 con 0x7f239415c810 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f239af91640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23940d0d90 con 0x7f23800039b0 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f239af91640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2394162de0 con 0x7f23940b52b0 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f2393fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239415c810 0x7f23940b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f2393fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239415c810 0x7f23940b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52346/0 (socket says 172.21.15.40:52346) 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.015+0000 7f2393fff640 1 -- 172.21.15.40:0/647739674 learned_addr learned my addr 172.21.15.40:0/647739674 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f239952e640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23940b52b0 0x7f239415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f2398d2d640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039b0 0x7f239415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f2393fff640 1 -- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23940b52b0 msgr2=0x7f239415d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f2393fff640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23940b52b0 0x7f239415d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f2393fff640 1 -- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039b0 msgr2=0x7f239415b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f2393fff640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039b0 0x7f239415b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f2393fff640 1 -- 172.21.15.40:0/647739674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23940b5770 con 0x7f239415c810 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f2393fff640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239415c810 0x7f23940b4d70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f237c0072f0 tx=0x7f237c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f23917fa640 1 -- 172.21.15.40:0/647739674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c01a070 con 0x7f239415c810 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f23917fa640 1 -- 172.21.15.40:0/647739674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237c00f960 con 0x7f239415c810 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23940b5a50 con 0x7f239415c810 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f23917fa640 1 -- 172.21.15.40:0/647739674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c00fd10 con 0x7f239415c810 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.016+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23940b1ed0 con 0x7f239415c810 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.017+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23940ba7c0 con 0x7f239415c810 2024-04-26T21:43:28.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.018+0000 7f23917fa640 1 -- 172.21.15.40:0/647739674 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f237c01d070 con 0x7f239415c810 2024-04-26T21:43:28.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.018+0000 7f23917fa640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c8f0 0x7f237406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.018+0000 7f23917fa640 1 -- 172.21.15.40:0/647739674 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6991+0+0 (secure 0 0 0) 0x7f237c08b980 con 0x7f239415c810 2024-04-26T21:43:28.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.018+0000 7f2398d2d640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c8f0 0x7f237406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.019+0000 7f2398d2d640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c8f0 0x7f237406edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2384032000 tx=0x7f238402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.022+0000 7f23917fa640 1 -- 172.21.15.40:0/647739674 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f23940ba7c0 con 0x7f239415c810 2024-04-26T21:43:28.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.249+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23940694f0 con 0x7f239415c810 2024-04-26T21:43:28.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.250+0000 7f23917fa640 1 -- 172.21.15.40:0/647739674 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+4355 (secure 0 0 0) 0x7f237c0592f0 con 0x7f239415c810 2024-04-26T21:43:28.251 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:28.251 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":140,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:27.897938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 140 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c8f0 msgr2=0x7f237406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c8f0 0x7f237406edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2384032000 tx=0x7f238402e000 comp rx=0 tx=0).stop 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239415c810 msgr2=0x7f23940b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239415c810 0x7f23940b4d70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f237c0072f0 tx=0x7f237c0077b0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 shutdown_connections 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f237406c8f0 0x7f237406edb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23940b52b0 0x7f239415d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239415c810 0x7f23940b4d70 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 --2- 172.21.15.40:0/647739674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23800039b0 0x7f239415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 >> 172.21.15.40:0/647739674 conn(0x7f23940d4f60 msgr2=0x7f23940d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 shutdown_connections 2024-04-26T21:43:28.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.252+0000 7f239af91640 1 -- 172.21.15.40:0/647739674 wait complete. 2024-04-26T21:43:28.263 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:28.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:28.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.403+0000 7f5d51f59640 1 Processor -- start 2024-04-26T21:43:28.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.403+0000 7f5d51f59640 1 -- start start 2024-04-26T21:43:28.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d51f59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c0b10b0 0x7f5d4c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d51f59640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c0cfd80 con 0x7f5d4c06c950 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d51f59640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c0d0020 con 0x7f5d4c0b0380 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d51f59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c0af8e0 con 0x7f5d4c0b10b0 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5d4c06c950 0x7f5d4c15e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60224/0 (socket says 172.21.15.40:60224) 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4bfff640 1 -- 172.21.15.40:0/66585810 learned_addr learned my addr 172.21.15.40:0/66585810 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4affd640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c0b10b0 0x7f5d4c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399165488 0 0) 0x7f5d4c0cfd80 con 0x7f5d4c06c950 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d34003520 con 0x7f5d4c06c950 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2930932374 0 0) 0x7f5d34003520 con 0x7f5d4c06c950 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c0b10b0 msgr2=0x7f5d4c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c0b10b0 0x7f5d4c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 >> v1:172.21.15.184:6789/0 conn(0x7f5d4c0b0380 legacy=0x7f5d4c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c0d0300 con 0x7f5d4c06c950 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f5d440019f0 con 0x7f5d4c06c950 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 >> v1:172.21.15.40:6789/0 conn(0x7f5d4c06c950 legacy=0x7f5d4c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d340039e0 0x7f5d34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c0b10b0 0x7f5d34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0af8e0 con 0x7f5d34004d40 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3400baf0 con 0x7f5d340039e0 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.404+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3400bcf0 con 0x7f5d4c0b10b0 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.405+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.405+0000 7f5d4b7fe640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d340039e0 0x7f5d34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.405+0000 7f5d4affd640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c0b10b0 0x7f5d34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.405+0000 7f5d4bfff640 1 -- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c0b10b0 msgr2=0x7f5d34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.405+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c0b10b0 0x7f5d34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.405+0000 7f5d4bfff640 1 -- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d340039e0 msgr2=0x7f5d34003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:28.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.405+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d340039e0 0x7f5d34003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.405+0000 7f5d4bfff640 1 -- 172.21.15.40:0/66585810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d34003520 con 0x7f5d34004d40 2024-04-26T21:43:28.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d340051c0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400a810 tx=0x7f5d34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c9833cc2698120 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400a460 con 0x7f5d34004d40 2024-04-26T21:43:28.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4402f070 con 0x7f5d34004d40 2024-04-26T21:43:28.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d4a7fc640 1 -- 172.21.15.40:0/66585810 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4402bc90 con 0x7f5d34004d40 2024-04-26T21:43:28.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d51f59640 1 -- 172.21.15.40:0/66585810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 msgr2=0x7f5d340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d51f59640 1 --2- 172.21.15.40:0/66585810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d340051c0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400a810 tx=0x7f5d34005830 comp rx=0 tx=0).stop 2024-04-26T21:43:28.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d4bfff640 1 -- 172.21.15.40:0/66585810 reap_dead start 2024-04-26T21:43:28.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d51f59640 1 -- 172.21.15.40:0/66585810 shutdown_connections 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d51f59640 1 -- 172.21.15.40:0/66585810 >> 172.21.15.40:0/66585810 conn(0x7f5d4c0d1cf0 msgr2=0x7f5d4c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d51f59640 1 -- 172.21.15.40:0/66585810 shutdown_connections 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.406+0000 7f5d51f59640 1 -- 172.21.15.40:0/66585810 wait complete. 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.407+0000 7f5d51f59640 1 Processor -- start 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d51f59640 1 -- start start 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d51f59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d4c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d51f59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c161850 0x7f5d4c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d51f59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c156150 0x7f5d4c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d51f59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0b0760 con 0x7f5d34004d40 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d51f59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0d1980 con 0x7f5d4c161850 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d51f59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c0d15c0 con 0x7f5d4c156150 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d4b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d4c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d4b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d4c155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52358/0 (socket says 172.21.15.40:52358) 2024-04-26T21:43:28.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.408+0000 7f5d4b7fe640 1 -- 172.21.15.40:0/817580114 learned_addr learned my addr 172.21.15.40:0/817580114 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c156150 0x7f5d4c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4affd640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c161850 0x7f5d4c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4b7fe640 1 -- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c156150 msgr2=0x7f5d4c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4b7fe640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c156150 0x7f5d4c0c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4b7fe640 1 -- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c161850 msgr2=0x7f5d4c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4b7fe640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c161850 0x7f5d4c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4b7fe640 1 -- 172.21.15.40:0/817580114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c0c8dc0 con 0x7f5d34004d40 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c156150 0x7f5d4c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4affd640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c161850 0x7f5d4c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d4b7fe640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d4c155250 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000f820 tx=0x7f5d4000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d33fff640 1 -- 172.21.15.40:0/817580114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d400050d0 con 0x7f5d34004d40 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d33fff640 1 -- 172.21.15.40:0/817580114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d40018070 con 0x7f5d34004d40 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d33fff640 1 -- 172.21.15.40:0/817580114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40005490 con 0x7f5d34004d40 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c0c90a0 con 0x7f5d34004d40 2024-04-26T21:43:28.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.409+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c0c9660 con 0x7f5d34004d40 2024-04-26T21:43:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.410+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c0694f0 con 0x7f5d34004d40 2024-04-26T21:43:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.410+0000 7f5d33fff640 1 -- 172.21.15.40:0/817580114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d4000d730 con 0x7f5d34004d40 2024-04-26T21:43:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.411+0000 7f5d33fff640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3806c8f0 0x7f5d3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.411+0000 7f5d33fff640 1 -- 172.21.15.40:0/817580114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6991+0+0 (secure 0 0 0) 0x7f5d4008bf10 con 0x7f5d34004d40 2024-04-26T21:43:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.411+0000 7f5d4affd640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3806c8f0 0x7f5d3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.412+0000 7f5d4affd640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3806c8f0 0x7f5d3806edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c161c90 tx=0x7f5d3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.415+0000 7f5d33fff640 1 -- 172.21.15.40:0/817580114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d40059e40 con 0x7f5d34004d40 2024-04-26T21:43:28.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.639+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d4c0b28b0 con 0x7f5d34004d40 2024-04-26T21:43:28.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.639+0000 7f5d33fff640 1 -- 172.21.15.40:0/817580114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+4355 (secure 0 0 0) 0x7f5d40059740 con 0x7f5d34004d40 2024-04-26T21:43:28.640 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:28.640 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":140,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:27.897938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:28.640 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 140 2024-04-26T21:43:28.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3806c8f0 msgr2=0x7f5d3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3806c8f0 0x7f5d3806edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c161c90 tx=0x7f5d3c008040 comp rx=0 tx=0).stop 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 msgr2=0x7f5d4c155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d4c155250 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000f820 tx=0x7f5d4000fce0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 shutdown_connections 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d3806c8f0 0x7f5d3806edb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4c156150 0x7f5d4c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c161850 0x7f5d4c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 --2- 172.21.15.40:0/817580114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d34004d40 0x7f5d4c155250 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.641+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 >> 172.21.15.40:0/817580114 conn(0x7f5d4c0d1cf0 msgr2=0x7f5d4c0b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.642+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 shutdown_connections 2024-04-26T21:43:28.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.642+0000 7f5d51f59640 1 -- 172.21.15.40:0/817580114 wait complete. 2024-04-26T21:43:28.652 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:28.652 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:28.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.754+0000 7f4fdc847640 1 Processor -- start 2024-04-26T21:43:28.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.754+0000 7f4fdc847640 1 -- start start 2024-04-26T21:43:28.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fdc847640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4fd8162880 0x7f4fd80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fdc847640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 0x7f4fd80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fdc847640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd80b7e50 con 0x7f4fd80b5f20 2024-04-26T21:43:28.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fdc847640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd80d0580 con 0x7f4fd8162880 2024-04-26T21:43:28.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fdc847640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd80d1800 con 0x7f4fd80d3bc0 2024-04-26T21:43:28.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fd659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4fd8162880 0x7f4fd80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fcffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4fd80b5f20 0x7f4fd80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53416/0 (socket says 172.21.15.40:53416) 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fcffff640 1 -- 172.21.15.40:0/3448827219 learned_addr learned my addr 172.21.15.40:0/3448827219 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fd659c640 1 -- 172.21.15.40:0/3448827219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 msgr2=0x7f4fd80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fd6d9d640 1 --2- 172.21.15.40:0/3448827219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 0x7f4fd80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fd659c640 1 --2- 172.21.15.40:0/3448827219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 0x7f4fd80b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fd659c640 1 -- 172.21.15.40:0/3448827219 >> v1:172.21.15.40:6789/0 conn(0x7f4fd80b5f20 legacy=0x7f4fd80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fd5d9b640 1 -- 172.21.15.40:0/3448827219 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 999692872 0 0) 0x7f4fd80b7e50 con 0x7f4fd80b5f20 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fd659c640 1 -- 172.21.15.40:0/3448827219 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd80d07d0 con 0x7f4fd8162880 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.755+0000 7f4fd6d9d640 1 --2- 172.21.15.40:0/3448827219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 0x7f4fd80b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.756+0000 7f4fd659c640 1 --2- 172.21.15.40:0/3448827219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd8162880 0x7f4fd80d2cb0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80097a0 tx=0x7f4fc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983dc5911df6b69e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.756+0000 7f4fd5d9b640 1 -- 172.21.15.40:0/3448827219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8037070 con 0x7f4fd8162880 2024-04-26T21:43:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.756+0000 7f4fd5d9b640 1 -- 172.21.15.40:0/3448827219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc8032440 con 0x7f4fd8162880 2024-04-26T21:43:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.756+0000 7f4fd5d9b640 1 -- 172.21.15.40:0/3448827219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8032810 con 0x7f4fd8162880 2024-04-26T21:43:28.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.757+0000 7f4fdc847640 1 -- 172.21.15.40:0/3448827219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd8162880 msgr2=0x7f4fd80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.757+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3448827219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd8162880 0x7f4fd80d2cb0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80097a0 tx=0x7f4fc802a180 comp rx=0 tx=0).stop 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.757+0000 7f4fdc847640 1 -- 172.21.15.40:0/3448827219 shutdown_connections 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.757+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3448827219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 0x7f4fd80b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.757+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3448827219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd8162880 0x7f4fd80d2cb0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.757+0000 7f4fdc847640 1 -- 172.21.15.40:0/3448827219 >> 172.21.15.40:0/3448827219 conn(0x7f4fd80d4f60 msgr2=0x7f4fd80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.757+0000 7f4fdc847640 1 -- 172.21.15.40:0/3448827219 shutdown_connections 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.757+0000 7f4fdc847640 1 -- 172.21.15.40:0/3448827219 wait complete. 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.758+0000 7f4fdc847640 1 Processor -- start 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.758+0000 7f4fdc847640 1 -- start start 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fdc847640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd80b5f20 0x7f4fd80b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fdc847640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 0x7f4fd80b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fdc847640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8162880 0x7f4fd80b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fdc847640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8162c60 con 0x7f4fd80b5f20 2024-04-26T21:43:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fdc847640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd80d0d90 con 0x7f4fd8162880 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fdc847640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd8162de0 con 0x7f4fd80d3bc0 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd80b5f20 0x7f4fd80b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd80b5f20 0x7f4fd80b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53858/0 (socket says 172.21.15.40:53858) 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 -- 172.21.15.40:0/3068799045 learned_addr learned my addr 172.21.15.40:0/3068799045 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 -- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 msgr2=0x7f4fd80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 0x7f4fd80b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 -- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8162880 msgr2=0x7f4fd80b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8162880 0x7f4fd80b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 -- 172.21.15.40:0/3068799045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd81f5bf0 con 0x7f4fd80b5f20 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fd659c640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd80b5f20 0x7f4fd80b4af0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8000c00 tx=0x7f4fc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fceffd640 1 -- 172.21.15.40:0/3068799045 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8037050 con 0x7f4fd80b5f20 2024-04-26T21:43:28.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fceffd640 1 -- 172.21.15.40:0/3068799045 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc8038040 con 0x7f4fd80b5f20 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.759+0000 7f4fceffd640 1 -- 172.21.15.40:0/3068799045 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8004470 con 0x7f4fd80b5f20 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.760+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd81f8690 con 0x7f4fd80b5f20 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.760+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81f8c50 con 0x7f4fd80b5f20 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.761+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd80694f0 con 0x7f4fd80b5f20 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.761+0000 7f4fceffd640 1 -- 172.21.15.40:0/3068799045 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fc8030980 con 0x7f4fd80b5f20 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.761+0000 7f4fceffd640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fb806c8f0 0x7f4fb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.762+0000 7f4fceffd640 1 -- 172.21.15.40:0/3068799045 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6991+0+0 (secure 0 0 0) 0x7f4fc80acca0 con 0x7f4fd80b5f20 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.762+0000 7f4fcffff640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fb806c8f0 0x7f4fb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.762+0000 7f4fcffff640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fb806c8f0 0x7f4fb806edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0007fd0 tx=0x7f4fc0007960 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.766+0000 7f4fceffd640 1 -- 172.21.15.40:0/3068799045 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4fc807abd0 con 0x7f4fd80b5f20 2024-04-26T21:43:28.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.993+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fd80cb3a0 con 0x7f4fd80b5f20 2024-04-26T21:43:28.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.994+0000 7f4fceffd640 1 -- 172.21.15.40:0/3068799045 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) ==== 95+0+5181 (secure 0 0 0) 0x7f4fc807a4d0 con 0x7f4fd80b5f20 2024-04-26T21:43:28.994 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:28.994 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1992174713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1992174713},{"type":"v1","addr":"172.21.15.40:6833","nonce":1992174713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:27.897938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:28.994 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 141 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fb806c8f0 msgr2=0x7f4fb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fb806c8f0 0x7f4fb806edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0007fd0 tx=0x7f4fc0007960 comp rx=0 tx=0).stop 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd80b5f20 msgr2=0x7f4fd80b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd80b5f20 0x7f4fd80b4af0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8000c00 tx=0x7f4fc80046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 shutdown_connections 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fb806c8f0 0x7f4fb806edb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8162880 0x7f4fd80b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fd80d3bc0 0x7f4fd80b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 --2- 172.21.15.40:0/3068799045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fd80b5f20 0x7f4fd80b4af0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 >> 172.21.15.40:0/3068799045 conn(0x7f4fd80d4f60 msgr2=0x7f4fd80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 shutdown_connections 2024-04-26T21:43:28.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:28.996+0000 7f4fdc847640 1 -- 172.21.15.40:0/3068799045 wait complete. 2024-04-26T21:43:29.006 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds_snaptable' led to DAMAGED state 2024-04-26T21:43:29.006 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_inotable' 2024-04-26T21:43:29.007 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:29.007 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:29.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:43:29.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:43:29.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:29.023 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:43:29.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:43:29.077 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:43:29.077 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:43:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.219+0000 7fbabc50a640 1 Processor -- start 2024-04-26T21:43:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.219+0000 7fbabc50a640 1 -- start start 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabc50a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbab4162880 0x7fbab40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabc50a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbab40d3bc0 0x7fbab40b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabc50a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab40b7e50 con 0x7fbab40b5f20 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabc50a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbab40d0580 con 0x7fbab40d3bc0 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabc50a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbab40d1800 con 0x7fbab4162880 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabaaa7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbab40d3bc0 0x7fbab40b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbab9aa5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbab40b5f20 0x7fbab40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53430/0 (socket says 172.21.15.40:53430) 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabaaa7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbab40d3bc0 0x7fbab40b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53872/0 (socket says 172.21.15.40:53872) 2024-04-26T21:43:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbab9aa5640 1 -- 172.21.15.40:0/3001248870 learned_addr learned my addr 172.21.15.40:0/3001248870 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbaba2a6640 1 --2- 172.21.15.40:0/3001248870 >> v2:172.21.15.184:3300/0 conn(0x7fbab4162880 0x7fbab40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabaaa7640 1 -- 172.21.15.40:0/3001248870 >> v2:172.21.15.184:3300/0 conn(0x7fbab4162880 msgr2=0x7fbab40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabaaa7640 1 --2- 172.21.15.40:0/3001248870 >> v2:172.21.15.184:3300/0 conn(0x7fbab4162880 0x7fbab40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabaaa7640 1 -- 172.21.15.40:0/3001248870 >> v1:172.21.15.40:6789/0 conn(0x7fbab40b5f20 legacy=0x7fbab40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbabaaa7640 1 -- 172.21.15.40:0/3001248870 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab40d07d0 con 0x7fbab40d3bc0 2024-04-26T21:43:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.220+0000 7fbab92a4640 1 -- 172.21.15.40:0/3001248870 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1535571759 0 0) 0x7fbab40b7e50 con 0x7fbab40b5f20 2024-04-26T21:43:29.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.221+0000 7fbabaaa7640 1 --2- 172.21.15.40:0/3001248870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40d3bc0 0x7fbab40b8300 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fbab408d490 tx=0x7fbaac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e370af04e825a03b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.221+0000 7fbab92a4640 1 -- 172.21.15.40:0/3001248870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac007740 con 0x7fbab40d3bc0 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.221+0000 7fbab92a4640 1 -- 172.21.15.40:0/3001248870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaac0078e0 con 0x7fbab40d3bc0 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.221+0000 7fbab92a4640 1 -- 172.21.15.40:0/3001248870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac007cb0 con 0x7fbab40d3bc0 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.222+0000 7fbabc50a640 1 -- 172.21.15.40:0/3001248870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40d3bc0 msgr2=0x7fbab40b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.222+0000 7fbabc50a640 1 --2- 172.21.15.40:0/3001248870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40d3bc0 0x7fbab40b8300 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fbab408d490 tx=0x7fbaac02a680 comp rx=0 tx=0).stop 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.222+0000 7fbabc50a640 1 -- 172.21.15.40:0/3001248870 shutdown_connections 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.222+0000 7fbabc50a640 1 --2- 172.21.15.40:0/3001248870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40d3bc0 0x7fbab40b8300 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.222+0000 7fbabc50a640 1 --2- 172.21.15.40:0/3001248870 >> v2:172.21.15.184:3300/0 conn(0x7fbab4162880 0x7fbab40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.222+0000 7fbabc50a640 1 -- 172.21.15.40:0/3001248870 >> 172.21.15.40:0/3001248870 conn(0x7fbab40d4f60 msgr2=0x7fbab40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.222+0000 7fbabc50a640 1 -- 172.21.15.40:0/3001248870 shutdown_connections 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.222+0000 7fbabc50a640 1 -- 172.21.15.40:0/3001248870 wait complete. 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.223+0000 7fbabc50a640 1 Processor -- start 2024-04-26T21:43:29.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.223+0000 7fbabc50a640 1 -- start start 2024-04-26T21:43:29.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbabc50a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40b5f20 0x7fbab415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:29.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbabc50a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab40d3bc0 0x7fbab40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbabc50a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab4162880 0x7fbab40b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbabc50a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4162c60 con 0x7fbab40b5f20 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbabc50a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbab40d0d90 con 0x7fbab40d3bc0 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbabc50a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbab4162de0 con 0x7fbab4162880 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40b5f20 0x7fbab415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40b5f20 0x7fbab415d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53886/0 (socket says 172.21.15.40:53886) 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 -- 172.21.15.40:0/1479132565 learned_addr learned my addr 172.21.15.40:0/1479132565 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 -- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab4162880 msgr2=0x7fbab40b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab4162880 0x7fbab40b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 -- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab40d3bc0 msgr2=0x7fbab40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab40d3bc0 0x7fbab40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 -- 172.21.15.40:0/1479132565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab415de80 con 0x7fbab40b5f20 2024-04-26T21:43:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaba2a6640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40b5f20 0x7fbab415d940 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fbab000d800 tx=0x7fbab000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaaaffd640 1 -- 172.21.15.40:0/1479132565 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab000b6c0 con 0x7fbab40b5f20 2024-04-26T21:43:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaaaffd640 1 -- 172.21.15.40:0/1479132565 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab0019070 con 0x7fbab40b5f20 2024-04-26T21:43:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab40b2e90 con 0x7fbab40b5f20 2024-04-26T21:43:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.224+0000 7fbaaaffd640 1 -- 172.21.15.40:0/1479132565 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab000ba80 con 0x7fbab40b5f20 2024-04-26T21:43:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.225+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab40b3450 con 0x7fbab40b5f20 2024-04-26T21:43:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.226+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab40694f0 con 0x7fbab40b5f20 2024-04-26T21:43:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.226+0000 7fbaaaffd640 1 -- 172.21.15.40:0/1479132565 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbab000bc20 con 0x7fbab40b5f20 2024-04-26T21:43:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.226+0000 7fbaaaffd640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba9806c8f0 0x7fba9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.227+0000 7fbaaaffd640 1 -- 172.21.15.40:0/1479132565 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6991+0+0 (secure 0 0 0) 0x7fbab008c540 con 0x7fbab40b5f20 2024-04-26T21:43:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.227+0000 7fbab9aa5640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba9806c8f0 0x7fba9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.227+0000 7fbab9aa5640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba9806c8f0 0x7fba9806edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4007fd0 tx=0x7fbaa4007960 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.231+0000 7fbaaaffd640 1 -- 172.21.15.40:0/1479132565 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbab415de80 con 0x7fbab40b5f20 2024-04-26T21:43:29.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.460+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbab40b4470 con 0x7fbab40b5f20 2024-04-26T21:43:29.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.909+0000 7fbaaaffd640 1 -- 172.21.15.40:0/1479132565 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v142) ==== 161+0+0 (secure 0 0 0) 0x7fbab001c360 con 0x7fbab40b5f20 2024-04-26T21:43:29.909 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba9806c8f0 msgr2=0x7fba9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba9806c8f0 0x7fba9806edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4007fd0 tx=0x7fbaa4007960 comp rx=0 tx=0).stop 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40b5f20 msgr2=0x7fbab415d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40b5f20 0x7fbab415d940 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fbab000d800 tx=0x7fbab000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 shutdown_connections 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba9806c8f0 0x7fba9806edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab4162880 0x7fbab40b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab40d3bc0 0x7fbab40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 --2- 172.21.15.40:0/1479132565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbab40b5f20 0x7fbab415d940 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 >> 172.21.15.40:0/1479132565 conn(0x7fbab40d4f60 msgr2=0x7fbab40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:29.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 shutdown_connections 2024-04-26T21:43:29.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:29.912+0000 7fbabc50a640 1 -- 172.21.15.40:0/1479132565 wait complete. 2024-04-26T21:43:29.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:43:30.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.027+0000 7f9829778640 1 Processor -- start 2024-04-26T21:43:30.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.027+0000 7f9829778640 1 -- start start 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.028+0000 7f9829778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406e270 0x7f98240d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.028+0000 7f9829778640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f982406d560 0x7f98240ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.028+0000 7f9829778640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98240d2420 con 0x7f98240d0070 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.028+0000 7f9829778640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9824162d00 con 0x7f982406d560 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.028+0000 7f9829778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98240d2d10 con 0x7f982406e270 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98227fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406e270 0x7f98240d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98227fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406e270 0x7f98240d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53662/0 (socket says 172.21.15.40:53662) 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98227fc640 1 -- 172.21.15.40:0/3377221326 learned_addr learned my addr 172.21.15.40:0/3377221326 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98237fe640 1 --2- 172.21.15.40:0/3377221326 >> v2:172.21.15.184:3300/0 conn(0x7f982406d560 0x7f98240ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98237fe640 1 -- 172.21.15.40:0/3377221326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406e270 msgr2=0x7f98240d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98237fe640 1 --2- 172.21.15.40:0/3377221326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406e270 0x7f98240d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98237fe640 1 -- 172.21.15.40:0/3377221326 >> v1:172.21.15.184:6789/0 conn(0x7f98240d0070 legacy=0x7f982406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98237fe640 1 -- 172.21.15.40:0/3377221326 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98240cd6b0 con 0x7f982406d560 2024-04-26T21:43:30.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.029+0000 7f98227fc640 1 --2- 172.21.15.40:0/3377221326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406e270 0x7f98240d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.030+0000 7f98237fe640 1 --2- 172.21.15.40:0/3377221326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982406d560 0x7f98240ce190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f981c0097a0 tx=0x7f981c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3584a8465aad17d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.030+0000 7f9821ffb640 1 -- 172.21.15.40:0/3377221326 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c037070 con 0x7f982406d560 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.030+0000 7f9821ffb640 1 -- 172.21.15.40:0/3377221326 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c032440 con 0x7f982406d560 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.030+0000 7f9821ffb640 1 -- 172.21.15.40:0/3377221326 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c032810 con 0x7f982406d560 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.030+0000 7f9829778640 1 -- 172.21.15.40:0/3377221326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982406d560 msgr2=0x7f98240ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.031+0000 7f9829778640 1 --2- 172.21.15.40:0/3377221326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982406d560 0x7f98240ce190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f981c0097a0 tx=0x7f981c02a210 comp rx=0 tx=0).stop 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.031+0000 7f9829778640 1 -- 172.21.15.40:0/3377221326 shutdown_connections 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.031+0000 7f9829778640 1 --2- 172.21.15.40:0/3377221326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982406d560 0x7f98240ce190 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.031+0000 7f9829778640 1 --2- 172.21.15.40:0/3377221326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406e270 0x7f98240d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.031+0000 7f9829778640 1 -- 172.21.15.40:0/3377221326 >> 172.21.15.40:0/3377221326 conn(0x7f98240d2ec0 msgr2=0x7f98240d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.031+0000 7f9829778640 1 -- 172.21.15.40:0/3377221326 shutdown_connections 2024-04-26T21:43:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.031+0000 7f9829778640 1 -- 172.21.15.40:0/3377221326 wait complete. 2024-04-26T21:43:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.032+0000 7f9829778640 1 Processor -- start 2024-04-26T21:43:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.032+0000 7f9829778640 1 -- start start 2024-04-26T21:43:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f9829778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406d560 0x7f982415da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f9829778640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982406e270 0x7f98241f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f9829778640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98240d0070 0x7f9824150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f9829778640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98240ce6d0 con 0x7f982406e270 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f9829778640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98240cdc70 con 0x7f98240d0070 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f9829778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98240ce850 con 0x7f982406d560 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982406e270 0x7f98241f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982406e270 0x7f98241f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53888/0 (socket says 172.21.15.40:53888) 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 -- 172.21.15.40:0/3912903149 learned_addr learned my addr 172.21.15.40:0/3912903149 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98237fe640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98240d0070 0x7f9824150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f9822ffd640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406d560 0x7f982415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 -- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406d560 msgr2=0x7f982415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406d560 0x7f982415da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 -- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98240d0070 msgr2=0x7f9824150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98240d0070 0x7f9824150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 -- 172.21.15.40:0/3912903149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98240c7e00 con 0x7f982406e270 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.033+0000 7f98227fc640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982406e270 0x7f98241f5270 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e9d0 tx=0x7f980c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.034+0000 7f981b7fe640 1 -- 172.21.15.40:0/3912903149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c017070 con 0x7f982406e270 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.034+0000 7f981b7fe640 1 -- 172.21.15.40:0/3912903149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980c00ca60 con 0x7f982406e270 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.034+0000 7f981b7fe640 1 -- 172.21.15.40:0/3912903149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c00ce10 con 0x7f982406e270 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.034+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98240c80e0 con 0x7f982406e270 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.034+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98240c86a0 con 0x7f982406e270 2024-04-26T21:43:30.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.035+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98240694f0 con 0x7f982406e270 2024-04-26T21:43:30.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.035+0000 7f981b7fe640 1 -- 172.21.15.40:0/3912903149 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f980c01a070 con 0x7f982406e270 2024-04-26T21:43:30.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.035+0000 7f981b7fe640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f980406c8f0 0x7f980406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.036+0000 7f981b7fe640 1 -- 172.21.15.40:0/3912903149 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6991+0+0 (secure 0 0 0) 0x7f980c08c2f0 con 0x7f982406e270 2024-04-26T21:43:30.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.036+0000 7f9822ffd640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f980406c8f0 0x7f980406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.036+0000 7f9822ffd640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f980406c8f0 0x7f980406edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9814001e70 tx=0x7f9814009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.040+0000 7f981b7fe640 1 -- 172.21.15.40:0/3912903149 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f980c059bc0 con 0x7f982406e270 2024-04-26T21:43:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.300+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f98240d0470 con 0x7f982406e270 2024-04-26T21:43:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.916+0000 7f981b7fe640 1 -- 172.21.15.40:0/3912903149 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v143) ==== 101+0+0 (secure 0 0 0) 0x7f980c05dec0 con 0x7f982406e270 2024-04-26T21:43:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:43:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f980406c8f0 msgr2=0x7f980406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f980406c8f0 0x7f980406edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9814001e70 tx=0x7f9814009040 comp rx=0 tx=0).stop 2024-04-26T21:43:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982406e270 msgr2=0x7f98241f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982406e270 0x7f98241f5270 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e9d0 tx=0x7f980c00ee90 comp rx=0 tx=0).stop 2024-04-26T21:43:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 shutdown_connections 2024-04-26T21:43:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f980406c8f0 0x7f980406edb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98240d0070 0x7f9824150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982406e270 0x7f98241f5270 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 --2- 172.21.15.40:0/3912903149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982406d560 0x7f982415da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 >> 172.21.15.40:0/3912903149 conn(0x7f98240d2ec0 msgr2=0x7f9824160f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 shutdown_connections 2024-04-26T21:43:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.919+0000 7f9829778640 1 -- 172.21.15.40:0/3912903149 wait complete. 2024-04-26T21:43:30.931 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:43:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.976+0000 7f3505a80f40 1 Processor -- start 2024-04-26T21:43:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.976+0000 7f3505a80f40 1 -- start start 2024-04-26T21:43:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3505a80f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 0x55fc23c49370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3505a80f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c3c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3505a80f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 0x55fc23b6ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3505a80f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fc23bbbbe0 con 0x55fc23c463d0 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3505a80f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc23b19e40 con 0x7f34f0002820 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3505a80f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55fc23bbd860 con 0x55fc23c3f380 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3502d7b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c3c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3502d7b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c3c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53896/0 (socket says 172.21.15.40:53896) 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3502d7b640 1 -- 172.21.15.40:0/52259817 learned_addr learned my addr 172.21.15.40:0/52259817 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3503d7d640 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 0x55fc23b6ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f350357c640 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 0x55fc23c49370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3502d7b640 1 -- 172.21.15.40:0/52259817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 msgr2=0x55fc23b6ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3502d7b640 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 0x55fc23b6ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3502d7b640 1 -- 172.21.15.40:0/52259817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 msgr2=0x55fc23c49370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3502d7b640 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 0x55fc23c49370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.977+0000 7f3502d7b640 1 -- 172.21.15.40:0/52259817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc23ba3380 con 0x55fc23c463d0 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3502d7b640 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c3c6e0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f34ec025e10 tx=0x7f34ec051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb5dd5d9ada2de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3501d79640 1 -- 172.21.15.40:0/52259817 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0508c0 con 0x55fc23c463d0 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3501d79640 1 -- 172.21.15.40:0/52259817 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec050a60 con 0x55fc23c463d0 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3501d79640 1 -- 172.21.15.40:0/52259817 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec050e50 con 0x55fc23c463d0 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 -- 172.21.15.40:0/52259817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 msgr2=0x55fc23c3c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c3c6e0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f34ec025e10 tx=0x7f34ec051000 comp rx=0 tx=0).stop 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 -- 172.21.15.40:0/52259817 shutdown_connections 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 0x55fc23b6ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c3c6e0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 --2- 172.21.15.40:0/52259817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 0x55fc23c49370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 -- 172.21.15.40:0/52259817 >> 172.21.15.40:0/52259817 conn(0x55fc23b9d9b0 msgr2=0x55fc23bba0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 -- 172.21.15.40:0/52259817 shutdown_connections 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.978+0000 7f3505a80f40 1 -- 172.21.15.40:0/52259817 wait complete. 2024-04-26T21:43:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.979+0000 7f3505a80f40 1 Processor -- start 2024-04-26T21:43:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.979+0000 7f3505a80f40 1 -- start start 2024-04-26T21:43:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3505a80f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 0x55fc23c3b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3505a80f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c39d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3505a80f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 0x55fc23b6ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3505a80f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fc23bb9c80 con 0x55fc23c463d0 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3505a80f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc23c44e60 con 0x7f34f0002820 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3505a80f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55fc23c450b0 con 0x55fc23c3f380 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c39d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c39d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53904/0 (socket says 172.21.15.40:53904) 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 learned_addr learned my addr 172.21.15.40:0/753611784 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f350357c640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 0x55fc23c3b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3503d7d640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 0x55fc23b6ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 msgr2=0x55fc23c3b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc23c3f380 0x55fc23c3b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 msgr2=0x55fc23b6ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f0002820 0x55fc23b6ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc23c3ef90 con 0x55fc23c463d0 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3502d7b640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c39d50 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f34ec046140 tx=0x7f34ec003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3500d77640 1 -- 172.21.15.40:0/753611784 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec06b020 con 0x55fc23c463d0 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3500d77640 1 -- 172.21.15.40:0/753611784 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec064460 con 0x55fc23c463d0 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.980+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fc23c3b7f0 con 0x55fc23c463d0 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.981+0000 7f3500d77640 1 -- 172.21.15.40:0/753611784 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0508c0 con 0x55fc23c463d0 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.981+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc23c3ecb0 con 0x55fc23c463d0 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.982+0000 7f3500d77640 1 -- 172.21.15.40:0/753611784 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34ec06e020 con 0x55fc23c463d0 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.982+0000 7f3500d77640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34e406b300 0x7f34e406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.982+0000 7f3500d77640 1 -- 172.21.15.40:0/753611784 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6991+0+0 (secure 0 0 0) 0x7f34ec0ced50 con 0x55fc23c463d0 2024-04-26T21:43:30.985 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:43:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.983+0000 7f350357c640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34e406b300 0x7f34e406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.983+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc23c3a290 0x55fc23ce9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.983+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23c3a6f0 con 0x55fc23c3a290 2024-04-26T21:43:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.983+0000 7f3503d7d640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc23c3a290 0x55fc23ce9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.983+0000 7f3503d7d640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc23c3a290 0x55fc23ce9bf0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.983+0000 7f350357c640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34e406b300 0x7f34e406d7c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009840 tx=0x7f34fc011040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.984+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v104'257 uv231 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.984+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cea130 con 0x55fc23c3a290 2024-04-26T21:43:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.985+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v104'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.986+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cea4d0 con 0x55fc23c3a290 2024-04-26T21:43:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.986+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v104'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.987 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:43:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.986+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb880 con 0x55fc23c3a290 2024-04-26T21:43:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.987+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v104'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.987+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cebc20 con 0x55fc23c3a290 2024-04-26T21:43:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.988+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v104'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.988+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cea870 con 0x55fc23c3a290 2024-04-26T21:43:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.989+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v104'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.989+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceac60 con 0x55fc23c3a290 2024-04-26T21:43:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.991+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v104'263 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.991+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.992+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v104'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.992+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:30.992 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:43:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.993+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v104'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.993+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:30.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.994+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v104'266 uv266 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.994+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.995+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v104'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.995+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.996+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v104'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.996+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.997+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v104'269 uv244 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.997+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.998+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v104'270 uv270 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.998+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:30.999 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:43:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.999+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v104'271 uv271 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:30.999+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.000+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v104'272 uv272 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.000+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:31.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.001+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v104'273 uv273 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.001+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceb000 con 0x55fc23c3a290 2024-04-26T21:43:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.002+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v104'274 uv274 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.002+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.003+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v104'275 uv275 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.003+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.004+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v104'276 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.004+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.005+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v104'277 uv277 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.005+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.011 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.008+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v104'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.008+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cecfd0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.009+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v104'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.009+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cecfd0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.010+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v104'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.010+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.011+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v104'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.011+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.012+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v104'282 uv256 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.012+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.013+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v104'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.013+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.014+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v104'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.014 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:43:31.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.014+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cecfd0 con 0x55fc23c3a290 2024-04-26T21:43:31.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.015+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v104'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.015+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cecfd0 con 0x55fc23c3a290 2024-04-26T21:43:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.016+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v104'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.016+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cebfc0 con 0x55fc23c3a290 2024-04-26T21:43:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.017+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v104'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23c3a290 2024-04-26T21:43:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.017+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc23cec4d0 0x55fc23cee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.017+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23ceee00 con 0x55fc23cec4d0 2024-04-26T21:43:31.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.017+0000 7f350357c640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc23cec4d0 0x55fc23cee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.017+0000 7f350357c640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc23cec4d0 0x55fc23cee8c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.019+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v104'198 uv181 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.019+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cef1a0 con 0x55fc23cec4d0 2024-04-26T21:43:31.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.020+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v104'199 uv199 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.020+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cef540 con 0x55fc23cec4d0 2024-04-26T21:43:31.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.021+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v104'200 uv200 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.021 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:43:31.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.021+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cefd90 con 0x55fc23cec4d0 2024-04-26T21:43:31.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.023+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v104'201 uv201 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.023+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0130 con 0x55fc23cec4d0 2024-04-26T21:43:31.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.024+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v104'202 uv202 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.024+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf04d0 con 0x55fc23cec4d0 2024-04-26T21:43:31.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.025+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v104'203 uv186 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.025+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cf0870 con 0x55fc23cec4d0 2024-04-26T21:43:31.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.026+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v104'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.026+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.027+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v104'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.027 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:43:31.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.027+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.028+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v104'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.028+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.029+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v104'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.029+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.031+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v104'208 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.031+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.032+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v104'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.032+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.033+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v104'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.032 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:43:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.033+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf52d0 con 0x55fc23cec4d0 2024-04-26T21:43:31.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.034+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v104'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.034+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf52d0 con 0x55fc23cec4d0 2024-04-26T21:43:31.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.035+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v104'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.035+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.036+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v104'213 uv197 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.036+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.037+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v104'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.037+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.038+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v104'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.038 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:43:31.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.038+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf1c20 con 0x55fc23cec4d0 2024-04-26T21:43:31.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.039+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v104'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.039+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf1c20 con 0x55fc23cec4d0 2024-04-26T21:43:31.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.040+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v104'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.040+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf0c10 con 0x55fc23cec4d0 2024-04-26T21:43:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.041+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v104'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cec4d0 2024-04-26T21:43:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.041+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc23cf1120 0x55fc23cf3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.041+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf3a50 con 0x55fc23cf1120 2024-04-26T21:43:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.041+0000 7f3502d7b640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc23cf1120 0x55fc23cf3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.042+0000 7f3502d7b640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc23cf1120 0x55fc23cf3510 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.043+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v104'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.043+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cf3df0 con 0x55fc23cf1120 2024-04-26T21:43:31.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.044+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v104'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.044+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf42a0 con 0x55fc23cf1120 2024-04-26T21:43:31.045 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:43:31.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.045+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v104'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.045+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf5b50 con 0x55fc23cf1120 2024-04-26T21:43:31.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.046+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v104'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.046+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf5ef0 con 0x55fc23cf1120 2024-04-26T21:43:31.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.047+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v104'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.047+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf4b40 con 0x55fc23cf1120 2024-04-26T21:43:31.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.048+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v104'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.048+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc23cf5440 0x55fc23cf82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.048+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf87e0 con 0x55fc23cf5440 2024-04-26T21:43:31.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.048+0000 7f3503d7d640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc23cf5440 0x55fc23cf82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.049+0000 7f3503d7d640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc23cf5440 0x55fc23cf82a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.050+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.050+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cf87e0 con 0x55fc23cf5440 2024-04-26T21:43:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.052+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v104'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.052+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf8860 con 0x55fc23cf5440 2024-04-26T21:43:31.054 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:43:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.054+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v104'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.054+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf8c50 con 0x55fc23cf5440 2024-04-26T21:43:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.056+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v104'48 uv48 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.057+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc23cf9160 0x55fc23cfb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.057+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfbb70 con 0x55fc23cf9160 2024-04-26T21:43:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.057+0000 7f350357c640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc23cf9160 0x55fc23cfb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.058+0000 7f350357c640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc23cf9160 0x55fc23cfb550 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.059+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v104'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.059+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cfbc30 con 0x55fc23cf9160 2024-04-26T21:43:31.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.060+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v104'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.060+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.061 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:43:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.062+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v104'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.062+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfd060 con 0x55fc23cf9160 2024-04-26T21:43:31.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.063+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v104'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.063+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfd060 con 0x55fc23cf9160 2024-04-26T21:43:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.064+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v104'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.064+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.067+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v104'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.067+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.068+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v104'94 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.068+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.070+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v104'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.070+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.070 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:43:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.071+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v104'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.071+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfd060 con 0x55fc23cf5440 2024-04-26T21:43:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.072+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v104'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.072+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfd060 con 0x55fc23cf5440 2024-04-26T21:43:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.073+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v104'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.073+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.074+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v104'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.074+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.075+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v104'100 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.075+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.076+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v104'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.076+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.076 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:43:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.077+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v104'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.077+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.078+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v104'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.078+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.079+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v104'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.079+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.079 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:43:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.080+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v104'43 uv43 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.080+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.081+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [write 0~70] v104'44 uv44 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.081+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf5440 2024-04-26T21:43:31.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.082+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v104'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f34f400abf0 con 0x55fc23cf5440 2024-04-26T21:43:31.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.082+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.083+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(83 605.00000000 [create] v104'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.083+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.084+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(84 605.00000000 [delete] v104'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.084+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.085 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:43:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.085+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(85 605.00000000 [create] v104'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.085+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfd060 con 0x55fc23cf9160 2024-04-26T21:43:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.086+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v104'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.086+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfd060 con 0x55fc23cf9160 2024-04-26T21:43:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.087+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v104'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.087+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.088+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(88 605.00000000 [omap-set-header] v104'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.088+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.089+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(89 mds0_inotable [create] v104'84 uv78 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.089+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.090+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(90 mds0_inotable [delete] v104'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.090+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfc050 con 0x55fc23cf9160 2024-04-26T21:43:31.091 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:43:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.091+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(91 mds0_inotable [create] v104'86 uv86 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.091+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf4750 con 0x55fc23cf9160 2024-04-26T21:43:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.092+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(92 mds0_inotable [write 0~34] v104'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.092+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf4750 con 0x55fc23cf9160 2024-04-26T21:43:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.094+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v104'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.094+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf4750 con 0x55fc23cf9160 2024-04-26T21:43:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.095+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(94 100.00000000 [create] v104'89 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.095+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cf4750 con 0x55fc23cf9160 2024-04-26T21:43:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.096+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(95 100.00000000 [delete] v104'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.096+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf4730 con 0x55fc23cf9160 2024-04-26T21:43:31.096 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:43:31.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.097+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(96 100.00000000 [create] v104'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.097+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cf4730 con 0x55fc23cf9160 2024-04-26T21:43:31.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.098+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(97 100.00000000 [omap-set-header] v104'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.098+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cfef30 con 0x55fc23cf9160 2024-04-26T21:43:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.099+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v104'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.099+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc23cfc560 0x55fc23cfe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.099+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cff5f0 con 0x55fc23cfc560 2024-04-26T21:43:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.099+0000 7f3502d7b640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc23cfc560 0x55fc23cfe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.100+0000 7f3502d7b640 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc23cfc560 0x55fc23cfe950 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.101+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(99 mds0_sessionmap [create] v104'40 uv39 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cfc560 2024-04-26T21:43:31.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.101+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cff990 con 0x55fc23cfc560 2024-04-26T21:43:31.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.102+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v104'41 uv41 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cfc560 2024-04-26T21:43:31.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.102+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cff990 con 0x55fc23cfc560 2024-04-26T21:43:31.104 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:43:31.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.105+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(101 mds0_sessionmap [create] v104'42 uv42 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cfc560 2024-04-26T21:43:31.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.105+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cfc560 2024-04-26T21:43:31.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.107+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v104'43 uv43 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cfc560 2024-04-26T21:43:31.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.107+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf9160 2024-04-26T21:43:31.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.108+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(103 609.00000000 [create] v104'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.108+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf9160 2024-04-26T21:43:31.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.110+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(104 609.00000000 [delete] v104'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.110+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf9160 2024-04-26T21:43:31.110 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:43:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.111+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(105 609.00000000 [create] v104'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.111+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23d00e50 con 0x55fc23cf9160 2024-04-26T21:43:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.112+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v104'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.112+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23d00e50 con 0x55fc23cf9160 2024-04-26T21:43:31.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.113+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v104'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.113+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf9160 2024-04-26T21:43:31.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.114+0000 7f350357c640 1 -- 172.21.15.40:0/753611784 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(108 609.00000000 [omap-set-header] v104'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34fc00fd10 con 0x55fc23cf9160 2024-04-26T21:43:31.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.114+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.115+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v104'42 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.115+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.117+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v104'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.117+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.118 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:43:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.118+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v104'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.118+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.119+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v104'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.119+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.120+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.120+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.121+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v104'107 uv99 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.121+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.122+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v104'108 uv108 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.122+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.123 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:43:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.123+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v104'109 uv109 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.123+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.124+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v104'110 uv110 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.124+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.125+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v104'111 uv111 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.125+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.126+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v104'112 uv106 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.126+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.127+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v104'113 uv113 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.128+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.128 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:43:31.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.129+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v104'114 uv114 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.129+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23d00e50 con 0x55fc23cf1120 2024-04-26T21:43:31.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.130+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v104'115 uv115 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.130+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23d00e50 con 0x55fc23cf1120 2024-04-26T21:43:31.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.131+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v104'116 uv116 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.131+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23cffe40 con 0x55fc23cf1120 2024-04-26T21:43:31.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.132+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v104'117 uv117 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.132+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55fc23d000f0 con 0x55fc23cf1120 2024-04-26T21:43:31.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3502d7b640 1 -- 172.21.15.40:0/753611784 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v104'118 uv118 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f34ec098d30 con 0x55fc23cf1120 2024-04-26T21:43:31.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc23cec4d0 msgr2=0x55fc23cee8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc23cec4d0 0x55fc23cee8c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc23c3a290 msgr2=0x55fc23ce9bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc23c3a290 0x55fc23ce9bf0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc23cf5440 msgr2=0x55fc23cf82a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc23cf5440 0x55fc23cf82a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc23cfc560 msgr2=0x55fc23cfe950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc23cfc560 0x55fc23cfe950 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc23cf1120 msgr2=0x55fc23cf3510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc23cf1120 0x55fc23cf3510 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc23cf9160 msgr2=0x55fc23cfb550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc23cf9160 0x55fc23cfb550 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 reap_dead start 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3503d7d640 1 -- 172.21.15.40:0/753611784 reap_dead start 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34e406b300 msgr2=0x7f34e406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34e406b300 0x7f34e406d7c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009840 tx=0x7f34fc011040 comp rx=0 tx=0).stop 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 msgr2=0x55fc23c39d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c39d50 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f34ec046140 tx=0x7f34ec003020 comp rx=0 tx=0).stop 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 shutdown_connections 2024-04-26T21:43:31.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34e406b300 0x7f34e406d7c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 --2- 172.21.15.40:0/753611784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc23c463d0 0x55fc23c39d50 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.133+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 >> 172.21.15.40:0/753611784 conn(0x55fc23b9d9b0 msgr2=0x55fc23bba0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:31.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.134+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 shutdown_connections 2024-04-26T21:43:31.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.134+0000 7f3505a80f40 1 -- 172.21.15.40:0/753611784 wait complete. 2024-04-26T21:43:31.137 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_inotable 2024-04-26T21:43:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77851bf40 1 Processor -- start 2024-04-26T21:43:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77851bf40 1 -- start start 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77851bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 0x561bac7f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77851bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 0x561bac8cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77851bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77851bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561bac8b7140 con 0x561bac8d4610 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77851bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561bac7a0e40 con 0x561bac8b6d60 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77851bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561bac7f9dc0 con 0x561bac840670 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe775da0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe775da0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53928/0 (socket says 172.21.15.40:53928) 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe775da0640 1 -- 172.21.15.40:0/1835987471 learned_addr learned my addr 172.21.15.40:0/1835987471 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe77559f640 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 0x561bac7f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.219+0000 7fe774d9e640 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 0x561bac8cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe775da0640 1 -- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 msgr2=0x561bac7f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe775da0640 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 0x561bac7f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe775da0640 1 -- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 msgr2=0x561bac8cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe775da0640 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 0x561bac8cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe775da0640 1 -- 172.21.15.40:0/1835987471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bac8d0190 con 0x561bac8d4610 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe77559f640 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 0x561bac7f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe775da0640 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cefc0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fe77000a590 tx=0x7fe77000a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0302701f1e3bc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe76f7fe640 1 -- 172.21.15.40:0/1835987471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770019070 con 0x561bac8d4610 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe76f7fe640 1 -- 172.21.15.40:0/1835987471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe770007600 con 0x561bac8d4610 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe76f7fe640 1 -- 172.21.15.40:0/1835987471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7700079d0 con 0x561bac8d4610 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe77851bf40 1 -- 172.21.15.40:0/1835987471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 msgr2=0x561bac8cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe77851bf40 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cefc0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fe77000a590 tx=0x7fe77000a680 comp rx=0 tx=0).stop 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe77851bf40 1 -- 172.21.15.40:0/1835987471 shutdown_connections 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe77851bf40 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cefc0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe77851bf40 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 0x561bac8cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.220+0000 7fe77851bf40 1 --2- 172.21.15.40:0/1835987471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 0x561bac7f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.221+0000 7fe77851bf40 1 -- 172.21.15.40:0/1835987471 >> 172.21.15.40:0/1835987471 conn(0x561bac7f7ab0 msgr2=0x561bac822fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.221+0000 7fe77851bf40 1 -- 172.21.15.40:0/1835987471 shutdown_connections 2024-04-26T21:43:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.221+0000 7fe77851bf40 1 -- 172.21.15.40:0/1835987471 wait complete. 2024-04-26T21:43:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.221+0000 7fe77851bf40 1 Processor -- start 2024-04-26T21:43:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.221+0000 7fe77851bf40 1 -- start start 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe77851bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 0x561bac8c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe77851bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 0x561bac8cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe77851bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe77851bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561bac8b3bd0 con 0x561bac8d4610 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe77851bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561bac8d0430 con 0x561bac8b6d60 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe77851bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561bac8cfce0 con 0x561bac840670 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53940/0 (socket says 172.21.15.40:53940) 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 -- 172.21.15.40:0/2227729299 learned_addr learned my addr 172.21.15.40:0/2227729299 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe774d9e640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 0x561bac8cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe77559f640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 0x561bac8c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 -- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 msgr2=0x561bac8c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561bac840670 0x561bac8c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 -- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 msgr2=0x561bac8cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561bac8b6d60 0x561bac8cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 -- 172.21.15.40:0/2227729299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bac8ac610 con 0x561bac8d4610 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.222+0000 7fe775da0640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cd810 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x561bac828490 tx=0x7fe770003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.223+0000 7fe76e7fc640 1 -- 172.21.15.40:0/2227729299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770023020 con 0x561bac8d4610 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.223+0000 7fe76e7fc640 1 -- 172.21.15.40:0/2227729299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe77001f690 con 0x561bac8d4610 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.223+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561bac8ac8f0 con 0x561bac8d4610 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.223+0000 7fe76e7fc640 1 -- 172.21.15.40:0/2227729299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770019040 con 0x561bac8d4610 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.223+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bac8b3010 con 0x561bac8d4610 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.224+0000 7fe76e7fc640 1 -- 172.21.15.40:0/2227729299 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe770026020 con 0x561bac8d4610 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.224+0000 7fe76e7fc640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75806b2d0 0x7fe75806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.224+0000 7fe76e7fc640 1 -- 172.21.15.40:0/2227729299 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6991+0+0 (secure 0 0 0) 0x7fe77008c140 con 0x561bac8d4610 2024-04-26T21:43:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.225+0000 7fe77851bf40 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561bac8cf890 0x561bac83d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.225+0000 7fe77559f640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75806b2d0 0x7fe75806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.225+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x561bac83dc60 con 0x561bac8cf890 2024-04-26T21:43:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.225+0000 7fe774d9e640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561bac8cf890 0x561bac83d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.225+0000 7fe77559f640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75806b2d0 0x7fe75806d790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe768010580 tx=0x7fe768008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.225+0000 7fe774d9e640 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561bac8cf890 0x561bac83d720 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.226+0000 7fe774d9e640 1 -- 172.21.15.40:0/2227729299 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 mds0_inotable [delete] v104'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe760054870 con 0x561bac8cf890 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.226+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561bac8cf890 msgr2=0x561bac83d720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.226+0000 7fe77851bf40 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561bac8cf890 0x561bac83d720 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75806b2d0 msgr2=0x7fe75806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe77851bf40 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe75806b2d0 0x7fe75806d790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe768010580 tx=0x7fe768008040 comp rx=0 tx=0).stop 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 msgr2=0x561bac8cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe77851bf40 1 --2- 172.21.15.40:0/2227729299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561bac8d4610 0x561bac8cd810 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x561bac828490 tx=0x7fe770003020 comp rx=0 tx=0).stop 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe775da0640 1 -- 172.21.15.40:0/2227729299 reap_dead start 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 shutdown_connections 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 >> 172.21.15.40:0/2227729299 conn(0x561bac7f7ab0 msgr2=0x561bac8ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 shutdown_connections 2024-04-26T21:43:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.227+0000 7fe77851bf40 1 -- 172.21.15.40:0/2227729299 wait complete. 2024-04-26T21:43:31.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:43:31.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.373+0000 7f5d46099640 1 Processor -- start 2024-04-26T21:43:31.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.373+0000 7f5d46099640 1 -- start start 2024-04-26T21:43:31.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d46099640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5d40071e60 0x7f5d4006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d46099640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d400cc480 0x7f5d40166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d46099640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4015d7c0 con 0x7f5d4006fe20 2024-04-26T21:43:31.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d46099640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4015daa0 con 0x7f5d40071e60 2024-04-26T21:43:31.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d46099640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4015dd80 con 0x7f5d400cc480 2024-04-26T21:43:31.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5d4006fe20 0x7f5d40072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53440/0 (socket says 172.21.15.40:53440) 2024-04-26T21:43:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3effd640 1 -- 172.21.15.40:0/4115986067 learned_addr learned my addr 172.21.15.40:0/4115986067 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3ffff640 1 --2- 172.21.15.40:0/4115986067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d400cc480 0x7f5d40166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3f7fe640 1 --2- 172.21.15.40:0/4115986067 >> v2:172.21.15.184:3300/0 conn(0x7f5d40071e60 0x7f5d4006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3e7fc640 1 -- 172.21.15.40:0/4115986067 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2877582162 0 0) 0x7f5d4015d7c0 con 0x7f5d4006fe20 2024-04-26T21:43:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3ffff640 1 -- 172.21.15.40:0/4115986067 >> v2:172.21.15.184:3300/0 conn(0x7f5d40071e60 msgr2=0x7f5d4006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3ffff640 1 --2- 172.21.15.40:0/4115986067 >> v2:172.21.15.184:3300/0 conn(0x7f5d40071e60 0x7f5d4006f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3ffff640 1 -- 172.21.15.40:0/4115986067 >> v1:172.21.15.40:6789/0 conn(0x7f5d4006fe20 legacy=0x7f5d40072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.374+0000 7f5d3ffff640 1 -- 172.21.15.40:0/4115986067 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4015e060 con 0x7f5d400cc480 2024-04-26T21:43:31.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.375+0000 7f5d3ffff640 1 --2- 172.21.15.40:0/4115986067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d400cc480 0x7f5d40166ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009d90 tx=0x7f5d3802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c9cf253356249e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.375+0000 7f5d3e7fc640 1 -- 172.21.15.40:0/4115986067 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38007740 con 0x7f5d400cc480 2024-04-26T21:43:31.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.375+0000 7f5d3e7fc640 1 -- 172.21.15.40:0/4115986067 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d380078e0 con 0x7f5d400cc480 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.375+0000 7f5d3e7fc640 1 -- 172.21.15.40:0/4115986067 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38007cb0 con 0x7f5d400cc480 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.376+0000 7f5d46099640 1 -- 172.21.15.40:0/4115986067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d400cc480 msgr2=0x7f5d40166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.376+0000 7f5d46099640 1 --2- 172.21.15.40:0/4115986067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d400cc480 0x7f5d40166ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009d90 tx=0x7f5d3802a660 comp rx=0 tx=0).stop 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.376+0000 7f5d46099640 1 -- 172.21.15.40:0/4115986067 shutdown_connections 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.376+0000 7f5d46099640 1 --2- 172.21.15.40:0/4115986067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d400cc480 0x7f5d40166ad0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.376+0000 7f5d46099640 1 --2- 172.21.15.40:0/4115986067 >> v2:172.21.15.184:3300/0 conn(0x7f5d40071e60 0x7f5d4006f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.376+0000 7f5d46099640 1 -- 172.21.15.40:0/4115986067 >> 172.21.15.40:0/4115986067 conn(0x7f5d400d0be0 msgr2=0x7f5d400d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.376+0000 7f5d46099640 1 -- 172.21.15.40:0/4115986067 shutdown_connections 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.376+0000 7f5d46099640 1 -- 172.21.15.40:0/4115986067 wait complete. 2024-04-26T21:43:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.377+0000 7f5d46099640 1 Processor -- start 2024-04-26T21:43:31.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.377+0000 7f5d46099640 1 -- start start 2024-04-26T21:43:31.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d46099640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4006fe20 0x7f5d400c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d46099640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d40071e60 0x7f5d4014a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d46099640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d400cc480 0x7f5d400c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d46099640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4015d570 con 0x7f5d40071e60 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d46099640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4015d3f0 con 0x7f5d400cc480 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d46099640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d40072950 con 0x7f5d4006fe20 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d40071e60 0x7f5d4014a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d40071e60 0x7f5d4014a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53946/0 (socket says 172.21.15.40:53946) 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3effd640 1 -- 172.21.15.40:0/24697563 learned_addr learned my addr 172.21.15.40:0/24697563 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3f7fe640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4006fe20 0x7f5d400c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3ffff640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d400cc480 0x7f5d400c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3f7fe640 1 -- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d400cc480 msgr2=0x7f5d400c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3f7fe640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d400cc480 0x7f5d400c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3f7fe640 1 -- 172.21.15.40:0/24697563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d40071e60 msgr2=0x7f5d4014a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3f7fe640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d40071e60 0x7f5d4014a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3f7fe640 1 -- 172.21.15.40:0/24697563 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d401566c0 con 0x7f5d4006fe20 2024-04-26T21:43:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.378+0000 7f5d3effd640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d40071e60 0x7f5d4014a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:43:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.379+0000 7f5d3f7fe640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4006fe20 0x7f5d400c7cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400e880 tx=0x7f5d3400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.379+0000 7f5d27fff640 1 -- 172.21.15.40:0/24697563 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34017070 con 0x7f5d4006fe20 2024-04-26T21:43:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.379+0000 7f5d27fff640 1 -- 172.21.15.40:0/24697563 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3400c940 con 0x7f5d4006fe20 2024-04-26T21:43:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.379+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d400caee0 con 0x7f5d4006fe20 2024-04-26T21:43:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.379+0000 7f5d27fff640 1 -- 172.21.15.40:0/24697563 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400cd30 con 0x7f5d4006fe20 2024-04-26T21:43:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.379+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d400b4e50 con 0x7f5d4006fe20 2024-04-26T21:43:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.380+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d400694f0 con 0x7f5d4006fe20 2024-04-26T21:43:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.380+0000 7f5d27fff640 1 -- 172.21.15.40:0/24697563 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d3401a070 con 0x7f5d4006fe20 2024-04-26T21:43:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.381+0000 7f5d27fff640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d2c06c8f0 0x7f5d2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.381+0000 7f5d27fff640 1 -- 172.21.15.40:0/24697563 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 6991+0+0 (secure 0 0 0) 0x7f5d3408bf00 con 0x7f5d4006fe20 2024-04-26T21:43:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.381+0000 7f5d3effd640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d2c06c8f0 0x7f5d2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.381+0000 7f5d3effd640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d2c06c8f0 0x7f5d2c06edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5d40072870 tx=0x7f5d30007e10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:31.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.386+0000 7f5d27fff640 1 -- 172.21.15.40:0/24697563 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d401566c0 con 0x7f5d4006fe20 2024-04-26T21:43:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.613+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f5d400722e0 con 0x7f5d4006fe20 2024-04-26T21:43:31.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.921+0000 7f5d27fff640 1 -- 172.21.15.40:0/24697563 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v144) ==== 162+0+0 (secure 0 0 0) 0x7f5d340597d0 con 0x7f5d4006fe20 2024-04-26T21:43:31.921 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:43:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d2c06c8f0 msgr2=0x7f5d2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d2c06c8f0 0x7f5d2c06edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5d40072870 tx=0x7f5d30007e10 comp rx=0 tx=0).stop 2024-04-26T21:43:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4006fe20 msgr2=0x7f5d400c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4006fe20 0x7f5d400c7cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400e880 tx=0x7f5d3400ed40 comp rx=0 tx=0).stop 2024-04-26T21:43:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 shutdown_connections 2024-04-26T21:43:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d2c06c8f0 0x7f5d2c06edb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d400cc480 0x7f5d400c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d40071e60 0x7f5d4014a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 --2- 172.21.15.40:0/24697563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d4006fe20 0x7f5d400c7cb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 >> 172.21.15.40:0/24697563 conn(0x7f5d400d0be0 msgr2=0x7f5d40159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 shutdown_connections 2024-04-26T21:43:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:31.923+0000 7f5d46099640 1 -- 172.21.15.40:0/24697563 wait complete. 2024-04-26T21:43:31.926 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:31.926+0000 7efc35ea3640 -1 mds.0.inotable: load_2 could not read table: -2 2024-04-26T21:43:31.926 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:31.927+0000 7efc35ea3640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-04-26T21:43:31.927 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:31.927+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:43:31.927 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:31.927+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:43:31.933 INFO:tasks.ceph.mds.c.smithi040.stderr: -30> 2024-04-26T21:43:31.926+0000 7efc35ea3640 -1 mds.0.inotable: load_2 could not read table: -2 2024-04-26T21:43:31.933 INFO:tasks.ceph.mds.c.smithi040.stderr: -29> 2024-04-26T21:43:31.927+0000 7efc35ea3640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-04-26T21:43:31.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:31.958 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:43:32.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.043+0000 7f8d6ea93640 1 Processor -- start 2024-04-26T21:43:32.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.043+0000 7f8d6ea93640 1 -- start start 2024-04-26T21:43:32.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6ea93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d680b0380 0x7f8d680ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6ea93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8d6806c950 0x7f8d6815e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6ea93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d680cfd80 con 0x7f8d680b10b0 2024-04-26T21:43:32.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6ea93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8d680af8e0 con 0x7f8d680b0380 2024-04-26T21:43:32.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6ea93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8d680d0020 con 0x7f8d6806c950 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6c82f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d680b0380 0x7f8d680ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6c82f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d680b0380 0x7f8d680ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53974/0 (socket says 172.21.15.40:53974) 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6c82f640 1 -- 172.21.15.40:0/2043911581 learned_addr learned my addr 172.21.15.40:0/2043911581 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6d030640 1 --2- 172.21.15.40:0/2043911581 >> v2:172.21.15.184:3300/0 conn(0x7f8d6806c950 0x7f8d6815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6c82f640 1 -- 172.21.15.40:0/2043911581 >> v2:172.21.15.184:3300/0 conn(0x7f8d6806c950 msgr2=0x7f8d6815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6c82f640 1 --2- 172.21.15.40:0/2043911581 >> v2:172.21.15.184:3300/0 conn(0x7f8d6806c950 0x7f8d6815e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6c82f640 1 -- 172.21.15.40:0/2043911581 >> v1:172.21.15.184:6789/0 conn(0x7f8d680b10b0 legacy=0x7f8d6806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.044+0000 7f8d6c82f640 1 -- 172.21.15.40:0/2043911581 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d680d0300 con 0x7f8d680b0380 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.045+0000 7f8d6c82f640 1 --2- 172.21.15.40:0/2043911581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680b0380 0x7f8d680ccd50 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f8d6806d290 tx=0x7f8d6402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c85076ad142d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.045+0000 7f8d5f7fe640 1 -- 172.21.15.40:0/2043911581 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64037070 con 0x7f8d680b0380 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.045+0000 7f8d5f7fe640 1 -- 172.21.15.40:0/2043911581 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d64032440 con 0x7f8d680b0380 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.045+0000 7f8d5f7fe640 1 -- 172.21.15.40:0/2043911581 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64032810 con 0x7f8d680b0380 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.046+0000 7f8d6ea93640 1 -- 172.21.15.40:0/2043911581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680b0380 msgr2=0x7f8d680ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.046+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/2043911581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680b0380 0x7f8d680ccd50 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f8d6806d290 tx=0x7f8d6402a160 comp rx=0 tx=0).stop 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.046+0000 7f8d6ea93640 1 -- 172.21.15.40:0/2043911581 shutdown_connections 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.046+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/2043911581 >> v2:172.21.15.184:3300/0 conn(0x7f8d6806c950 0x7f8d6815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.046+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/2043911581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d680b0380 0x7f8d680ccd50 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.046+0000 7f8d6ea93640 1 -- 172.21.15.40:0/2043911581 >> 172.21.15.40:0/2043911581 conn(0x7f8d680d1cf0 msgr2=0x7f8d680d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.046+0000 7f8d6ea93640 1 -- 172.21.15.40:0/2043911581 shutdown_connections 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.046+0000 7f8d6ea93640 1 -- 172.21.15.40:0/2043911581 wait complete. 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.047+0000 7f8d6ea93640 1 Processor -- start 2024-04-26T21:43:32.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.047+0000 7f8d6ea93640 1 -- start start 2024-04-26T21:43:32.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6ea93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d6806c950 0x7f8d6815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6ea93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b0380 0x7f8d68160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6ea93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d680b10b0 0x7f8d6806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6ea93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680b0760 con 0x7f8d6806c950 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6ea93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680d1980 con 0x7f8d680b10b0 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6ea93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8d680d15c0 con 0x7f8d680b0380 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d6806c950 0x7f8d6815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d6806c950 0x7f8d6815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53990/0 (socket says 172.21.15.40:53990) 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 -- 172.21.15.40:0/881386347 learned_addr learned my addr 172.21.15.40:0/881386347 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6d030640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d680b10b0 0x7f8d6806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d5ffff640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b0380 0x7f8d68160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 -- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b0380 msgr2=0x7f8d68160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b0380 0x7f8d68160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 -- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d680b10b0 msgr2=0x7f8d6806faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d680b10b0 0x7f8d6806faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 -- 172.21.15.40:0/881386347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d680c8f60 con 0x7f8d6806c950 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d6c82f640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d6806c950 0x7f8d6815e0d0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000c00 tx=0x7f8d640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.048+0000 7f8d5d7fa640 1 -- 172.21.15.40:0/881386347 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64041070 con 0x7f8d6806c950 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.049+0000 7f8d5d7fa640 1 -- 172.21.15.40:0/881386347 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6403c870 con 0x7f8d6806c950 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.049+0000 7f8d5d7fa640 1 -- 172.21.15.40:0/881386347 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64037040 con 0x7f8d6806c950 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.049+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d680c9200 con 0x7f8d6806c950 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.049+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d680c9ce0 con 0x7f8d6806c950 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.049+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d680694f0 con 0x7f8d6806c950 2024-04-26T21:43:32.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.050+0000 7f8d5d7fa640 1 -- 172.21.15.40:0/881386347 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d64044070 con 0x7f8d6806c950 2024-04-26T21:43:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.050+0000 7f8d5d7fa640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.050+0000 7f8d5d7fa640 1 -- 172.21.15.40:0/881386347 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7f8d640aca50 con 0x7f8d6806c950 2024-04-26T21:43:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.051+0000 7f8d5ffff640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.051+0000 7f8d5ffff640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8d58006fd0 tx=0x7f8d58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.055+0000 7f8d5d7fa640 1 -- 172.21.15.40:0/881386347 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d6407a9b0 con 0x7f8d6806c950 2024-04-26T21:43:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.278+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d68073da0 con 0x7f8d6806c950 2024-04-26T21:43:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.278+0000 7f8d5d7fa640 1 -- 172.21.15.40:0/881386347 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+4355 (secure 0 0 0) 0x7f8d6407a2b0 con 0x7f8d6806c950 2024-04-26T21:43:32.278 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:32.279 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":146,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:31.927752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:32.279 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 146 2024-04-26T21:43:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 msgr2=0x7f8d5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8d58006fd0 tx=0x7f8d58008040 comp rx=0 tx=0).stop 2024-04-26T21:43:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d6806c950 msgr2=0x7f8d6815e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d6806c950 0x7f8d6815e0d0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000c00 tx=0x7f8d640046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 shutdown_connections 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8d5406c8f0 0x7f8d5406edb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d680b10b0 0x7f8d6806faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d680b0380 0x7f8d68160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.280+0000 7f8d6ea93640 1 --2- 172.21.15.40:0/881386347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d6806c950 0x7f8d6815e0d0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.281+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 >> 172.21.15.40:0/881386347 conn(0x7f8d680d1cf0 msgr2=0x7f8d680b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.281+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 shutdown_connections 2024-04-26T21:43:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.281+0000 7f8d6ea93640 1 -- 172.21.15.40:0/881386347 wait complete. 2024-04-26T21:43:32.291 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:32.291 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:32.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.432+0000 7f87709da640 1 Processor -- start 2024-04-26T21:43:32.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.432+0000 7f87709da640 1 -- start start 2024-04-26T21:43:32.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f87709da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f87709da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f876c072700 0x7f876c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f87709da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f876c0700e0 con 0x7f876c06fcd0 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f87709da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f876c074010 con 0x7f876c072700 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f87709da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f876c072f70 con 0x7f876c15f2d0 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f876ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f876c06fcd0 0x7f876c0cbb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53458/0 (socket says 172.21.15.40:53458) 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f876ad9d640 1 -- 172.21.15.40:0/1909022208 learned_addr learned my addr 172.21.15.40:0/1909022208 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f876a59c640 1 --2- 172.21.15.40:0/1909022208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f8769d9b640 1 --2- 172.21.15.40:0/1909022208 >> v2:172.21.15.184:3300/0 conn(0x7f876c072700 0x7f876c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f876959a640 1 -- 172.21.15.40:0/1909022208 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055768253 0 0) 0x7f8764001490 con 0x7f876c06fcd0 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.433+0000 7f876959a640 1 -- 172.21.15.40:0/1909022208 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8754003520 con 0x7f876c06fcd0 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.434+0000 7f876959a640 1 -- 172.21.15.40:0/1909022208 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2949990823 0 0) 0x7f8754003520 con 0x7f876c06fcd0 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.434+0000 7f876a59c640 1 -- 172.21.15.40:0/1909022208 >> v2:172.21.15.184:3300/0 conn(0x7f876c072700 msgr2=0x7f876c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.434+0000 7f876a59c640 1 --2- 172.21.15.40:0/1909022208 >> v2:172.21.15.184:3300/0 conn(0x7f876c072700 0x7f876c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.434+0000 7f876a59c640 1 -- 172.21.15.40:0/1909022208 >> v1:172.21.15.40:6789/0 conn(0x7f876c06fcd0 legacy=0x7f876c0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.434+0000 7f876a59c640 1 -- 172.21.15.40:0/1909022208 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c073250 con 0x7f876c15f2d0 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.434+0000 7f876a59c640 1 --2- 172.21.15.40:0/1909022208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c1604e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f875c0097a0 tx=0x7f875c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a724358357dc3980 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f876959a640 1 -- 172.21.15.40:0/1909022208 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c037070 con 0x7f876c15f2d0 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f876959a640 1 -- 172.21.15.40:0/1909022208 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f875c02aba0 con 0x7f876c15f2d0 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f876959a640 1 -- 172.21.15.40:0/1909022208 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764001bc0 con 0x7f876c15f2d0 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f87709da640 1 -- 172.21.15.40:0/1909022208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 msgr2=0x7f876c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f87709da640 1 --2- 172.21.15.40:0/1909022208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c1604e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f875c0097a0 tx=0x7f875c02a110 comp rx=0 tx=0).stop 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f87709da640 1 -- 172.21.15.40:0/1909022208 shutdown_connections 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f87709da640 1 --2- 172.21.15.40:0/1909022208 >> v2:172.21.15.184:3300/0 conn(0x7f876c072700 0x7f876c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f87709da640 1 --2- 172.21.15.40:0/1909022208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c1604e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f87709da640 1 -- 172.21.15.40:0/1909022208 >> 172.21.15.40:0/1909022208 conn(0x7f876c0d0be0 msgr2=0x7f876c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:32.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f87709da640 1 -- 172.21.15.40:0/1909022208 shutdown_connections 2024-04-26T21:43:32.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.435+0000 7f87709da640 1 -- 172.21.15.40:0/1909022208 wait complete. 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.436+0000 7f87709da640 1 Processor -- start 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f87709da640 1 -- start start 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f87709da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f876c06fcd0 0x7f876c165fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f87709da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f876c072700 0x7f876c163d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f87709da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f87709da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f876c073af0 con 0x7f876c072700 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f87709da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f876c073810 con 0x7f876c06fcd0 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f87709da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f876c15f6b0 con 0x7f876c15f2d0 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f8769d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f876c072700 0x7f876c163d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f8769d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f876c072700 0x7f876c163d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54006/0 (socket says 172.21.15.40:54006) 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.437+0000 7f8769d9b640 1 -- 172.21.15.40:0/2762732933 learned_addr learned my addr 172.21.15.40:0/2762732933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:32.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f876a59c640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f876c06fcd0 0x7f876c165fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f876ad9d640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f8769d9b640 1 -- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 msgr2=0x7f876c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f8769d9b640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c0ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f8769d9b640 1 -- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f876c06fcd0 msgr2=0x7f876c165fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f8769d9b640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f876c06fcd0 0x7f876c165fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f8769d9b640 1 -- 172.21.15.40:0/2762732933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c1642a0 con 0x7f876c072700 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f8769d9b640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f876c072700 0x7f876c163d60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f876000e930 tx=0x7f876000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f875affd640 1 -- 172.21.15.40:0/2762732933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760017070 con 0x7f876c072700 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f875affd640 1 -- 172.21.15.40:0/2762732933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876000ca00 con 0x7f876c072700 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c0ca9e0 con 0x7f876c072700 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f875affd640 1 -- 172.21.15.40:0/2762732933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876000cdf0 con 0x7f876c072700 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.438+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c164820 con 0x7f876c072700 2024-04-26T21:43:32.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.439+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876c0694f0 con 0x7f876c072700 2024-04-26T21:43:32.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.440+0000 7f875affd640 1 -- 172.21.15.40:0/2762732933 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f876001a070 con 0x7f876c072700 2024-04-26T21:43:32.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.440+0000 7f875affd640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f874c06c8f0 0x7f874c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.440+0000 7f875affd640 1 -- 172.21.15.40:0/2762732933 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7f876008c1a0 con 0x7f876c072700 2024-04-26T21:43:32.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.440+0000 7f876a59c640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f874c06c8f0 0x7f874c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.441+0000 7f876a59c640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f874c06c8f0 0x7f874c06edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f875c034000 tx=0x7f875c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.444+0000 7f875affd640 1 -- 172.21.15.40:0/2762732933 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f876005a0b0 con 0x7f876c072700 2024-04-26T21:43:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.667+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f876c06de80 con 0x7f876c072700 2024-04-26T21:43:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.667+0000 7f875affd640 1 -- 172.21.15.40:0/2762732933 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+4355 (secure 0 0 0) 0x7f87600599b0 con 0x7f876c072700 2024-04-26T21:43:32.667 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:32.667 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":146,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:31.927752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 146 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f874c06c8f0 msgr2=0x7f874c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f874c06c8f0 0x7f874c06edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f875c034000 tx=0x7f875c02e000 comp rx=0 tx=0).stop 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f876c072700 msgr2=0x7f876c163d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f876c072700 0x7f876c163d60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f876000e930 tx=0x7f876000edf0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 shutdown_connections 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f874c06c8f0 0x7f874c06edb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f876c15f2d0 0x7f876c0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f876c072700 0x7f876c163d60 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 --2- 172.21.15.40:0/2762732933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f876c06fcd0 0x7f876c165fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 >> 172.21.15.40:0/2762732933 conn(0x7f876c0d0be0 msgr2=0x7f876c0b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 shutdown_connections 2024-04-26T21:43:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.669+0000 7f87709da640 1 -- 172.21.15.40:0/2762732933 wait complete. 2024-04-26T21:43:32.680 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:32.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.780+0000 7f57929e4640 1 Processor -- start 2024-04-26T21:43:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.780+0000 7f57929e4640 1 -- start start 2024-04-26T21:43:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.781+0000 7f57929e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f578c15f3a0 0x7f578c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.781+0000 7f57929e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578c0d0e20 con 0x7f578c0d1770 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.781+0000 7f57929e4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578c0b1d70 con 0x7f578c0d2330 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.781+0000 7f57929e4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f578c0b2050 con 0x7f578c15f3a0 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.781+0000 7f578bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f578c15f3a0 0x7f578c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.781+0000 7f578b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f578c0d1770 0x7f578c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53464/0 (socket says 172.21.15.40:53464) 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.781+0000 7f578b7fe640 1 -- 172.21.15.40:0/599222618 learned_addr learned my addr 172.21.15.40:0/599222618 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.782+0000 7f578bfff640 1 -- 172.21.15.40:0/599222618 >> v1:172.21.15.184:6789/0 conn(0x7f578c0d2330 legacy=0x7f578c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.782+0000 7f578bfff640 1 -- 172.21.15.40:0/599222618 >> v1:172.21.15.40:6789/0 conn(0x7f578c0d1770 legacy=0x7f578c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.782+0000 7f578bfff640 1 -- 172.21.15.40:0/599222618 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c0b2330 con 0x7f578c15f3a0 2024-04-26T21:43:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.782+0000 7f578affd640 1 -- 172.21.15.40:0/599222618 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902557136 0 0) 0x7f578c0d0e20 con 0x7f578c0d1770 2024-04-26T21:43:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.782+0000 7f578bfff640 1 --2- 172.21.15.40:0/599222618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c0d06f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f577c0097a0 tx=0x7f577c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a922e823c3da4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f578affd640 1 -- 172.21.15.40:0/599222618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c037070 con 0x7f578c15f3a0 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f578affd640 1 -- 172.21.15.40:0/599222618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c032440 con 0x7f578c15f3a0 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f578affd640 1 -- 172.21.15.40:0/599222618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c032810 con 0x7f578c15f3a0 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f57929e4640 1 -- 172.21.15.40:0/599222618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 msgr2=0x7f578c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f57929e4640 1 --2- 172.21.15.40:0/599222618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c0d06f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f577c0097a0 tx=0x7f577c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f57929e4640 1 -- 172.21.15.40:0/599222618 shutdown_connections 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f57929e4640 1 --2- 172.21.15.40:0/599222618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c0d06f0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f57929e4640 1 -- 172.21.15.40:0/599222618 >> 172.21.15.40:0/599222618 conn(0x7f578c0d4f90 msgr2=0x7f578c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f57929e4640 1 -- 172.21.15.40:0/599222618 shutdown_connections 2024-04-26T21:43:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.783+0000 7f57929e4640 1 -- 172.21.15.40:0/599222618 wait complete. 2024-04-26T21:43:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.784+0000 7f57929e4640 1 Processor -- start 2024-04-26T21:43:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f57929e4640 1 -- start start 2024-04-26T21:43:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f57929e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f578c0d1770 0x7f578c0ba850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f57929e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578c0d2330 0x7f578c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f57929e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c15d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f57929e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f578c0b30e0 con 0x7f578c15f3a0 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f57929e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f578c0b2f60 con 0x7f578c0d2330 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f57929e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f578c0b3260 con 0x7f578c0d1770 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f5790f81640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f5790f81640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c15d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54014/0 (socket says 172.21.15.40:54014) 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.785+0000 7f5790f81640 1 -- 172.21.15.40:0/1684460622 learned_addr learned my addr 172.21.15.40:0/1684460622 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f578bfff640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f578c0d1770 0x7f578c0ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f578b7fe640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578c0d2330 0x7f578c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5790f81640 1 -- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f578c0d1770 msgr2=0x7f578c0ba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5790f81640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f578c0d1770 0x7f578c0ba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5790f81640 1 -- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578c0d2330 msgr2=0x7f578c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5790f81640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578c0d2330 0x7f578c15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5790f81640 1 -- 172.21.15.40:0/1684460622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c0ca060 con 0x7f578c15f3a0 2024-04-26T21:43:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5790f81640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c15d910 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f578400c780 tx=0x7f578400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5788ff9640 1 -- 172.21.15.40:0/1684460622 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784004b60 con 0x7f578c15f3a0 2024-04-26T21:43:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5788ff9640 1 -- 172.21.15.40:0/1684460622 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5784004d00 con 0x7f578c15f3a0 2024-04-26T21:43:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578c0ca340 con 0x7f578c15f3a0 2024-04-26T21:43:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f5788ff9640 1 -- 172.21.15.40:0/1684460622 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578400aa90 con 0x7f578c15f3a0 2024-04-26T21:43:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.786+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578c0ca900 con 0x7f578c15f3a0 2024-04-26T21:43:32.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.787+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578c0cab00 con 0x7f578c15f3a0 2024-04-26T21:43:32.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.788+0000 7f5788ff9640 1 -- 172.21.15.40:0/1684460622 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f578400ac30 con 0x7f578c15f3a0 2024-04-26T21:43:32.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.788+0000 7f5788ff9640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f577806c8f0 0x7f577806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.788+0000 7f5788ff9640 1 -- 172.21.15.40:0/1684460622 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7f578408c3b0 con 0x7f578c15f3a0 2024-04-26T21:43:32.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.788+0000 7f578bfff640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f577806c8f0 0x7f577806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.789+0000 7f578bfff640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f577806c8f0 0x7f577806edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f577c034000 tx=0x7f577c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:32.792+0000 7f5788ff9640 1 -- 172.21.15.40:0/1684460622 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f578c0cab00 con 0x7f578c15f3a0 2024-04-26T21:43:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.022+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f578c0694f0 con 0x7f578c15f3a0 2024-04-26T21:43:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.023+0000 7f5788ff9640 1 -- 172.21.15.40:0/1684460622 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 147 v147) ==== 95+0+5181 (secure 0 0 0) 0x7f5784020070 con 0x7f578c15f3a0 2024-04-26T21:43:33.023 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:33.023 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":147,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2962699830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2962699830},{"type":"v1","addr":"172.21.15.40:6833","nonce":2962699830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:31.927752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:33.023 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 147 2024-04-26T21:43:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f577806c8f0 msgr2=0x7f577806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f577806c8f0 0x7f577806edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f577c034000 tx=0x7f577c02e000 comp rx=0 tx=0).stop 2024-04-26T21:43:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 msgr2=0x7f578c15d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c15d910 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f578400c780 tx=0x7f578400cc40 comp rx=0 tx=0).stop 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 shutdown_connections 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f577806c8f0 0x7f577806edb0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f578c15f3a0 0x7f578c15d910 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578c0d2330 0x7f578c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 --2- 172.21.15.40:0/1684460622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f578c0d1770 0x7f578c0ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.024+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 >> 172.21.15.40:0/1684460622 conn(0x7f578c0d4f90 msgr2=0x7f578c0b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.025+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 shutdown_connections 2024-04-26T21:43:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.025+0000 7f57929e4640 1 -- 172.21.15.40:0/1684460622 wait complete. 2024-04-26T21:43:33.035 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_inotable' led to DAMAGED state 2024-04-26T21:43:33.035 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000' 2024-04-26T21:43:33.036 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:33.036 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:33.051 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:43:33.051 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:43:33.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:33.052 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:43:33.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:43:33.107 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:43:33.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:43:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.249+0000 7fefdf407640 1 Processor -- start 2024-04-26T21:43:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.249+0000 7fefdf407640 1 -- start start 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdf407640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd8071e60 0x7fefd806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdf407640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fefd806fe20 0x7fefd8072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdf407640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd815d7c0 con 0x7fefd8072780 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdf407640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fefd815daa0 con 0x7fefd806fe20 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdf407640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fefd815dd80 con 0x7fefd8071e60 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdc9a2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fefd806fe20 0x7fefd8072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdd9a4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fefd8072780 0x7fefd80cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53478/0 (socket says 172.21.15.40:53478) 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdd9a4640 1 -- 172.21.15.40:0/2327612747 learned_addr learned my addr 172.21.15.40:0/2327612747 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.250+0000 7fefdd1a3640 1 --2- 172.21.15.40:0/2327612747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd8071e60 0x7fefd806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.251+0000 7fefdc9a2640 1 -- 172.21.15.40:0/2327612747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd8071e60 msgr2=0x7fefd806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.251+0000 7fefdc9a2640 1 --2- 172.21.15.40:0/2327612747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd8071e60 0x7fefd806f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.251+0000 7fefdc9a2640 1 -- 172.21.15.40:0/2327612747 >> v1:172.21.15.40:6789/0 conn(0x7fefd8072780 legacy=0x7fefd80cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.251+0000 7fefdc9a2640 1 -- 172.21.15.40:0/2327612747 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd815e060 con 0x7fefd806fe20 2024-04-26T21:43:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.251+0000 7fefdc9a2640 1 --2- 172.21.15.40:0/2327612747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd806fe20 0x7fefd8072240 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fefc80097a0 tx=0x7fefc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27a6636b9b37bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.251+0000 7fefcffff640 1 -- 172.21.15.40:0/2327612747 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8037070 con 0x7fefd806fe20 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.251+0000 7fefcffff640 1 -- 172.21.15.40:0/2327612747 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8032440 con 0x7fefd806fe20 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.251+0000 7fefcffff640 1 -- 172.21.15.40:0/2327612747 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8032810 con 0x7fefd806fe20 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.252+0000 7fefdf407640 1 -- 172.21.15.40:0/2327612747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd806fe20 msgr2=0x7fefd8072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.252+0000 7fefdf407640 1 --2- 172.21.15.40:0/2327612747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd806fe20 0x7fefd8072240 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fefc80097a0 tx=0x7fefc802a180 comp rx=0 tx=0).stop 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.252+0000 7fefdf407640 1 -- 172.21.15.40:0/2327612747 shutdown_connections 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.252+0000 7fefdf407640 1 --2- 172.21.15.40:0/2327612747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd806fe20 0x7fefd8072240 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.252+0000 7fefdf407640 1 --2- 172.21.15.40:0/2327612747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd8071e60 0x7fefd806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.252+0000 7fefdf407640 1 -- 172.21.15.40:0/2327612747 >> 172.21.15.40:0/2327612747 conn(0x7fefd80d0be0 msgr2=0x7fefd80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.252+0000 7fefdf407640 1 -- 172.21.15.40:0/2327612747 shutdown_connections 2024-04-26T21:43:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.252+0000 7fefdf407640 1 -- 172.21.15.40:0/2327612747 wait complete. 2024-04-26T21:43:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.253+0000 7fefdf407640 1 Processor -- start 2024-04-26T21:43:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.253+0000 7fefdf407640 1 -- start start 2024-04-26T21:43:33.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdf407640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd806fe20 0x7fefd81f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:33.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdf407640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd8071e60 0x7fefd8148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdf407640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd8072780 0x7fefd8148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdf407640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fefd815d3f0 con 0x7fefd8071e60 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdf407640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8072c10 con 0x7fefd8072780 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdf407640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fefd815d570 con 0x7fefd806fe20 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdc9a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd8071e60 0x7fefd8148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdc9a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd8071e60 0x7fefd8148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54042/0 (socket says 172.21.15.40:54042) 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdc9a2640 1 -- 172.21.15.40:0/2809169595 learned_addr learned my addr 172.21.15.40:0/2809169595 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdd1a3640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd806fe20 0x7fefd81f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdc9a2640 1 -- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd806fe20 msgr2=0x7fefd81f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdd9a4640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd8072780 0x7fefd8148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdc9a2640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd806fe20 0x7fefd81f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdc9a2640 1 -- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd8072780 msgr2=0x7fefd8148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdc9a2640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd8072780 0x7fefd8148cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdc9a2640 1 -- 172.21.15.40:0/2809169595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd815c7d0 con 0x7fefd8071e60 2024-04-26T21:43:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.254+0000 7fefdd9a4640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd8072780 0x7fefd8148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.255+0000 7fefdc9a2640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd8071e60 0x7fefd8148780 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fefc802a1c0 tx=0x7fefc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.255+0000 7fefcdffb640 1 -- 172.21.15.40:0/2809169595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8037040 con 0x7fefd8071e60 2024-04-26T21:43:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.255+0000 7fefcdffb640 1 -- 172.21.15.40:0/2809169595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8042070 con 0x7fefd8071e60 2024-04-26T21:43:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.255+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd80c9920 con 0x7fefd8071e60 2024-04-26T21:43:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.255+0000 7fefcdffb640 1 -- 172.21.15.40:0/2809169595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8030bb0 con 0x7fefd8071e60 2024-04-26T21:43:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.255+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd80c9ee0 con 0x7fefd8071e60 2024-04-26T21:43:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.256+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd80694f0 con 0x7fefd8071e60 2024-04-26T21:43:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.256+0000 7fefcdffb640 1 -- 172.21.15.40:0/2809169595 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefc8047460 con 0x7fefd8071e60 2024-04-26T21:43:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.257+0000 7fefcdffb640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefbc06c8f0 0x7fefbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.257+0000 7fefcdffb640 1 -- 172.21.15.40:0/2809169595 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7fefc80acc60 con 0x7fefd8071e60 2024-04-26T21:43:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.257+0000 7fefdd1a3640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefbc06c8f0 0x7fefbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.257+0000 7fefdd1a3640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefbc06c8f0 0x7fefbc06edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fefd4006fd0 tx=0x7fefd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.261+0000 7fefcdffb640 1 -- 172.21.15.40:0/2809169595 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefc807ab30 con 0x7fefd8071e60 2024-04-26T21:43:33.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.484+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fefd806eec0 con 0x7fefd8071e60 2024-04-26T21:43:33.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.941+0000 7fefcdffb640 1 -- 172.21.15.40:0/2809169595 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v148) ==== 161+0+0 (secure 0 0 0) 0x7fefc807a430 con 0x7fefd8071e60 2024-04-26T21:43:33.941 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:43:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefbc06c8f0 msgr2=0x7fefbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefbc06c8f0 0x7fefbc06edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fefd4006fd0 tx=0x7fefd4008040 comp rx=0 tx=0).stop 2024-04-26T21:43:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd8071e60 msgr2=0x7fefd8148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd8071e60 0x7fefd8148780 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fefc802a1c0 tx=0x7fefc80046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 shutdown_connections 2024-04-26T21:43:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fefbc06c8f0 0x7fefbc06edb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd8072780 0x7fefd8148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd8071e60 0x7fefd8148780 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 --2- 172.21.15.40:0/2809169595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd806fe20 0x7fefd81f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.943+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 >> 172.21.15.40:0/2809169595 conn(0x7fefd80d0be0 msgr2=0x7fefd815e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.944+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 shutdown_connections 2024-04-26T21:43:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:33.944+0000 7fefdf407640 1 -- 172.21.15.40:0/2809169595 wait complete. 2024-04-26T21:43:33.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:43:34.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.061+0000 7f8306cdc640 1 Processor -- start 2024-04-26T21:43:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.061+0000 7f8306cdc640 1 -- start start 2024-04-26T21:43:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8306cdc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8300162880 0x7f83000d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8306cdc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 0x7f83000d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8306cdc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83000d39f0 0x7f83000b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8306cdc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f83000d0580 con 0x7f83000d39f0 2024-04-26T21:43:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8306cdc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f83000b7e50 con 0x7f8300162880 2024-04-26T21:43:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8306cdc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f83000d1800 con 0x7f83000b5f20 2024-04-26T21:43:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8305279640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83000d39f0 0x7f83000b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8305279640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83000d39f0 0x7f83000b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54052/0 (socket says 172.21.15.40:54052) 2024-04-26T21:43:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8305279640 1 -- 172.21.15.40:0/3100776982 learned_addr learned my addr 172.21.15.40:0/3100776982 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8304a78640 1 --2- 172.21.15.40:0/3100776982 >> v2:172.21.15.184:3300/0 conn(0x7f8300162880 0x7f83000d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f82fffff640 1 --2- 172.21.15.40:0/3100776982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 0x7f83000d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8305279640 1 -- 172.21.15.40:0/3100776982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 msgr2=0x7f83000d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8305279640 1 --2- 172.21.15.40:0/3100776982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 0x7f83000d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8305279640 1 -- 172.21.15.40:0/3100776982 >> v2:172.21.15.184:3300/0 conn(0x7f8300162880 msgr2=0x7f83000d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8305279640 1 --2- 172.21.15.40:0/3100776982 >> v2:172.21.15.184:3300/0 conn(0x7f8300162880 0x7f83000d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.062+0000 7f8305279640 1 -- 172.21.15.40:0/3100776982 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83000d07d0 con 0x7f83000d39f0 2024-04-26T21:43:34.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.063+0000 7f8305279640 1 --2- 172.21.15.40:0/3100776982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83000d39f0 0x7f83000b69e0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f830008d490 tx=0x7f82f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54893680a68edff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.063+0000 7f82ff7fe640 1 -- 172.21.15.40:0/3100776982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8036070 con 0x7f83000d39f0 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.063+0000 7f82ff7fe640 1 -- 172.21.15.40:0/3100776982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f8031600 con 0x7f83000d39f0 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.063+0000 7f82ff7fe640 1 -- 172.21.15.40:0/3100776982 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f80319d0 con 0x7f83000d39f0 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 -- 172.21.15.40:0/3100776982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83000d39f0 msgr2=0x7f83000b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 --2- 172.21.15.40:0/3100776982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83000d39f0 0x7f83000b69e0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f830008d490 tx=0x7f82f802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 -- 172.21.15.40:0/3100776982 shutdown_connections 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 --2- 172.21.15.40:0/3100776982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83000d39f0 0x7f83000b69e0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 --2- 172.21.15.40:0/3100776982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 0x7f83000d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 --2- 172.21.15.40:0/3100776982 >> v2:172.21.15.184:3300/0 conn(0x7f8300162880 0x7f83000d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 -- 172.21.15.40:0/3100776982 >> 172.21.15.40:0/3100776982 conn(0x7f83000d4f60 msgr2=0x7f83000d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 -- 172.21.15.40:0/3100776982 shutdown_connections 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.064+0000 7f8306cdc640 1 -- 172.21.15.40:0/3100776982 wait complete. 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.065+0000 7f8306cdc640 1 Processor -- start 2024-04-26T21:43:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.065+0000 7f8306cdc640 1 -- start start 2024-04-26T21:43:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8306cdc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 0x7f83000ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8306cdc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83000d39f0 0x7f83001f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8306cdc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8300162880 0x7f83000b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8306cdc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8300162c60 con 0x7f8300162880 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8306cdc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83000d0d90 con 0x7f83000d39f0 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8306cdc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8300162de0 con 0x7f83000b5f20 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8300162880 0x7f83000b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8300162880 0x7f83000b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54068/0 (socket says 172.21.15.40:54068) 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 -- 172.21.15.40:0/1722309610 learned_addr learned my addr 172.21.15.40:0/1722309610 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8304a78640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 0x7f83000ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f82fffff640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83000d39f0 0x7f83001f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 -- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 msgr2=0x7f83000ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 0x7f83000ba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 -- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83000d39f0 msgr2=0x7f83001f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83000d39f0 0x7f83001f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 -- 172.21.15.40:0/1722309610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83000ce590 con 0x7f8300162880 2024-04-26T21:43:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.066+0000 7f8305279640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8300162880 0x7f83000b5220 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f82f802ac90 tx=0x7f82f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.067+0000 7f82fd7fa640 1 -- 172.21.15.40:0/1722309610 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8041020 con 0x7f8300162880 2024-04-26T21:43:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.067+0000 7f82fd7fa640 1 -- 172.21.15.40:0/1722309610 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f8007780 con 0x7f8300162880 2024-04-26T21:43:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.067+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83000ce870 con 0x7f8300162880 2024-04-26T21:43:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.067+0000 7f82fd7fa640 1 -- 172.21.15.40:0/1722309610 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8036040 con 0x7f8300162880 2024-04-26T21:43:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.067+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83000ce1f0 con 0x7f8300162880 2024-04-26T21:43:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.068+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83000694f0 con 0x7f8300162880 2024-04-26T21:43:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.068+0000 7f82fd7fa640 1 -- 172.21.15.40:0/1722309610 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82f8044020 con 0x7f8300162880 2024-04-26T21:43:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.069+0000 7f82fd7fa640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f82e406c8f0 0x7f82e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.069+0000 7f82fd7fa640 1 -- 172.21.15.40:0/1722309610 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7f82f80acd30 con 0x7f8300162880 2024-04-26T21:43:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.069+0000 7f8304a78640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f82e406c8f0 0x7f82e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.069+0000 7f8304a78640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f82e406c8f0 0x7f82e406edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f82f4006fd0 tx=0x7f82f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:34.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.073+0000 7f82fd7fa640 1 -- 172.21.15.40:0/1722309610 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f82f804b030 con 0x7f8300162880 2024-04-26T21:43:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.337+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f830015fd60 con 0x7f8300162880 2024-04-26T21:43:34.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.948+0000 7f82fd7fa640 1 -- 172.21.15.40:0/1722309610 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v149) ==== 101+0+0 (secure 0 0 0) 0x7f82f807ad50 con 0x7f8300162880 2024-04-26T21:43:34.948 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f82e406c8f0 msgr2=0x7f82e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f82e406c8f0 0x7f82e406edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f82f4006fd0 tx=0x7f82f4008040 comp rx=0 tx=0).stop 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8300162880 msgr2=0x7f83000b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8300162880 0x7f83000b5220 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f82f802ac90 tx=0x7f82f8004a40 comp rx=0 tx=0).stop 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 shutdown_connections 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f82e406c8f0 0x7f82e406edb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8300162880 0x7f83000b5220 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83000d39f0 0x7f83001f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 --2- 172.21.15.40:0/1722309610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83000b5f20 0x7f83000ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 >> 172.21.15.40:0/1722309610 conn(0x7f83000d4f60 msgr2=0x7f8300160370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 shutdown_connections 2024-04-26T21:43:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:34.951+0000 7f8306cdc640 1 -- 172.21.15.40:0/1722309610 wait complete. 2024-04-26T21:43:34.964 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:43:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.011+0000 7f7f5add0f40 1 Processor -- start 2024-04-26T21:43:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.011+0000 7f7f5add0f40 1 -- start start 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f5add0f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c062ef60 0x55f0c06a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f5add0f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 0x55f0c0616350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f5add0f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c06c3610 0x55f0c0612d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f5add0f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f0c062f8b0 con 0x55f0c062ef60 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f5add0f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f0c06a6140 con 0x55f0c06a5d60 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f5add0f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f0c05e8dc0 con 0x55f0c06c3610 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f588ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c062ef60 0x55f0c06a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f588ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c062ef60 0x55f0c06a3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54090/0 (socket says 172.21.15.40:54090) 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f588ca640 1 -- 172.21.15.40:0/342180983 learned_addr learned my addr 172.21.15.40:0/342180983 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f590cb640 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c06c3610 0x55f0c0612d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f580c9640 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 0x55f0c0616350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f588ca640 1 -- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c06c3610 msgr2=0x55f0c0612d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f588ca640 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c06c3610 0x55f0c0612d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f588ca640 1 -- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 msgr2=0x55f0c0616350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.012+0000 7f7f588ca640 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 0x55f0c0616350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f588ca640 1 -- 172.21.15.40:0/342180983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c0512cb0 con 0x55f0c062ef60 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f580c9640 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 0x55f0c0616350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f588ca640 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c062ef60 0x55f0c06a3fe0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000a7a0 tx=0x7f7f5000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b207374b278055d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f570c7640 1 -- 172.21.15.40:0/342180983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50019070 con 0x55f0c062ef60 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f570c7640 1 -- 172.21.15.40:0/342180983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f50007600 con 0x55f0c062ef60 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f570c7640 1 -- 172.21.15.40:0/342180983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500079d0 con 0x55f0c062ef60 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f5add0f40 1 -- 172.21.15.40:0/342180983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c062ef60 msgr2=0x55f0c06a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c062ef60 0x55f0c06a3fe0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000a7a0 tx=0x7f7f5000a890 comp rx=0 tx=0).stop 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f5add0f40 1 -- 172.21.15.40:0/342180983 shutdown_connections 2024-04-26T21:43:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c06c3610 0x55f0c0612d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 0x55f0c0616350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/342180983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c062ef60 0x55f0c06a3fe0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f5add0f40 1 -- 172.21.15.40:0/342180983 >> 172.21.15.40:0/342180983 conn(0x55f0c05e6ab0 msgr2=0x55f0c06a4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.013+0000 7f7f5add0f40 1 -- 172.21.15.40:0/342180983 shutdown_connections 2024-04-26T21:43:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.014+0000 7f7f5add0f40 1 -- 172.21.15.40:0/342180983 wait complete. 2024-04-26T21:43:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.014+0000 7f7f5add0f40 1 Processor -- start 2024-04-26T21:43:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.014+0000 7f7f5add0f40 1 -- start start 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f5add0f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c062ef60 0x55f0c0616350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f5add0f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 0x55f0c06b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f5add0f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c06c3610 0x55f0c0756690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f5add0f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f0c06a1fa0 con 0x55f0c06c3610 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f5add0f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f0c062e5c0 con 0x55f0c06a5d60 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f5add0f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f0c062e070 con 0x55f0c062ef60 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f590cb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c06c3610 0x55f0c0756690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f590cb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c06c3610 0x55f0c0756690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54100/0 (socket says 172.21.15.40:54100) 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 learned_addr learned my addr 172.21.15.40:0/709214371 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f588ca640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c062ef60 0x55f0c0616350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f580c9640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 0x55f0c06b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c062ef60 msgr2=0x55f0c0616350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f590cb640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f0c062ef60 0x55f0c0616350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 msgr2=0x55f0c06b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f590cb640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0c06a5d60 0x55f0c06b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.015+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c07577b0 con 0x55f0c06c3610 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.016+0000 7f7f590cb640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c06c3610 0x55f0c0756690 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c190 tx=0x7f7f4800c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.016+0000 7f7f560c5640 1 -- 172.21.15.40:0/709214371 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48017020 con 0x55f0c06c3610 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.016+0000 7f7f560c5640 1 -- 172.21.15.40:0/709214371 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f48013670 con 0x55f0c06c3610 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.016+0000 7f7f560c5640 1 -- 172.21.15.40:0/709214371 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48013a60 con 0x55f0c06c3610 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.016+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f0c07574d0 con 0x55f0c06c3610 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.016+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c0757e80 con 0x55f0c06c3610 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.017+0000 7f7f560c5640 1 -- 172.21.15.40:0/709214371 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f4801a020 con 0x55f0c06c3610 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.018+0000 7f7f560c5640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f3806b300 0x7f7f3806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.018+0000 7f7f560c5640 1 -- 172.21.15.40:0/709214371 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7f7f4808aff0 con 0x55f0c06c3610 2024-04-26T21:43:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.018+0000 7f7f588ca640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f3806b300 0x7f7f3806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.020 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.018+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f0c0758b90 0x55f0c0758f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.018+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0757080 con 0x55f0c0758b90 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.018+0000 7f7f580c9640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f0c0758b90 0x55f0c0758f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.019+0000 7f7f588ca640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f3806b300 0x7f7f3806d7c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7f50017040 tx=0x7f7f5000c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.019+0000 7f7f580c9640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f0c0758b90 0x55f0c0758f70 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.020+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v105'288 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.020+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c07594b0 con 0x55f0c0758b90 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.022+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v105'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.022+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0762be0 con 0x55f0c0758b90 2024-04-26T21:43:35.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.023+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v105'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.022 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:43:35.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.023+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763f90 con 0x55f0c0758b90 2024-04-26T21:43:35.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.024+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v105'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.024+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0764330 con 0x55f0c0758b90 2024-04-26T21:43:35.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.025+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v105'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.025+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0762f80 con 0x55f0c0758b90 2024-04-26T21:43:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.026+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v105'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.026+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763370 con 0x55f0c0758b90 2024-04-26T21:43:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.027+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v105'294 uv268 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.027+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0763760 con 0x55f0c0758b90 2024-04-26T21:43:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.028+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v105'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.028+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.029+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v105'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.029 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:43:35.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.029+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.030+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v105'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.030+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.031+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v105'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.031+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.032+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v105'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.032+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.033+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v105'300 uv275 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.033+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.034+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v105'301 uv301 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.034+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.035+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v105'302 uv302 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.035 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:43:35.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.035+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.036+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v105'303 uv303 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.036+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.037+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v105'304 uv304 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.037+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.038+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v105'305 uv305 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.038+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.039+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v105'306 uv306 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.039+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.040+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v105'307 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.041+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.042+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v105'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.042+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.042 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:43:35.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.043+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v105'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.043+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.044+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v105'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.044+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.045+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v105'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.045+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.046+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v105'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.046+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.047+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v105'313 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.047+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.048+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v105'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.048+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.049+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v105'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.049 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:43:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.049+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.050+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v105'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.050+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.051+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v105'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.051+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0763b00 con 0x55f0c0758b90 2024-04-26T21:43:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.052+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v105'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0758b90 2024-04-26T21:43:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.052+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f0c0763b00 0x55f0c0766bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.052+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0767130 con 0x55f0c0763b00 2024-04-26T21:43:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.052+0000 7f7f588ca640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f0c0763b00 0x55f0c0766bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.053+0000 7f7f588ca640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f0c0763b00 0x55f0c0766bf0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.054+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v105'219 uv202 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.054+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c07674d0 con 0x55f0c0763b00 2024-04-26T21:43:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.056+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v105'220 uv220 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.056+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0767870 con 0x55f0c0763b00 2024-04-26T21:43:35.056 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:43:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.057+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v105'221 uv221 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.057+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07680c0 con 0x55f0c0763b00 2024-04-26T21:43:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.058+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v105'222 uv222 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.058+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768460 con 0x55f0c0763b00 2024-04-26T21:43:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.060+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v105'223 uv223 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.060+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768800 con 0x55f0c0763b00 2024-04-26T21:43:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.061+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v105'224 uv207 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.061+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.062+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v105'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.062+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.062 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:43:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.063+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v105'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.063+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.064+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v105'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.064+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.065+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v105'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.065+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.066+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v105'229 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.066+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.068+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v105'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.068+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.068 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:43:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.069+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v105'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.069+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076d260 con 0x55f0c0763b00 2024-04-26T21:43:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.070+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v105'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.070+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076d260 con 0x55f0c0763b00 2024-04-26T21:43:35.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.071+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v105'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.071+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.072+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v105'234 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.072+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.073+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v105'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.074+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.074 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:43:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.075+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v105'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.075+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0769bb0 con 0x55f0c0763b00 2024-04-26T21:43:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.076+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v105'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.076+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0769bb0 con 0x55f0c0763b00 2024-04-26T21:43:35.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.077+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v105'238 uv238 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.077+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0768ba0 con 0x55f0c0763b00 2024-04-26T21:43:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.078+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v105'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0763b00 2024-04-26T21:43:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.078+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f0c0769270 0x55f0c076b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.078+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076bcf0 con 0x55f0c0769270 2024-04-26T21:43:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.078+0000 7f7f580c9640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f0c0769270 0x55f0c076b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.078+0000 7f7f580c9640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f0c0769270 0x55f0c076b660 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.080+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v105'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.080+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c076c0c0 con 0x55f0c0769270 2024-04-26T21:43:35.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.081+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v105'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.081+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076c0c0 con 0x55f0c0769270 2024-04-26T21:43:35.082 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:43:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.082+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v105'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.082+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076d0d0 con 0x55f0c0769270 2024-04-26T21:43:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.083+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v105'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.083+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076d0d0 con 0x55f0c0769270 2024-04-26T21:43:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.084+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v105'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.084+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076c0c0 con 0x55f0c0769270 2024-04-26T21:43:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.085+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v105'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.085+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f0c076c9e0 0x55f0c076edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.085+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076f310 con 0x55f0c076c9e0 2024-04-26T21:43:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.085+0000 7f7f590cb640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f0c076c9e0 0x55f0c076edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.086+0000 7f7f590cb640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f0c076c9e0 0x55f0c076edd0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.087+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v105'49 uv48 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.087+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c076f770 con 0x55f0c076c9e0 2024-04-26T21:43:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.088+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v105'50 uv50 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.089+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c076fc20 con 0x55f0c076c9e0 2024-04-26T21:43:35.091 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:43:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.091+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v105'51 uv51 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.091+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0770510 con 0x55f0c076c9e0 2024-04-26T21:43:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.094+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v105'52 uv52 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.094+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f0c0770dc0 0x55f0c07731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.094+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07736f0 con 0x55f0c0770dc0 2024-04-26T21:43:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.094+0000 7f7f588ca640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f0c0770dc0 0x55f0c07731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.095+0000 7f7f588ca640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f0c0770dc0 0x55f0c07731b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.096+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v105'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.096+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0773ac0 con 0x55f0c0770dc0 2024-04-26T21:43:35.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.100+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v105'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.100+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0773ac0 con 0x55f0c0770dc0 2024-04-26T21:43:35.101 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:43:35.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.101+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v105'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.101+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774ad0 con 0x55f0c0770dc0 2024-04-26T21:43:35.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.104+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v105'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.104+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774ad0 con 0x55f0c0770dc0 2024-04-26T21:43:35.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.106+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v105'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.106+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0773ac0 con 0x55f0c0770dc0 2024-04-26T21:43:35.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.109+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v105'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.109+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0773ac0 con 0x55f0c076c9e0 2024-04-26T21:43:35.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.111+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v105'105 uv99 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.111+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0773e80 con 0x55f0c076c9e0 2024-04-26T21:43:35.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.112+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v105'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.112+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774220 con 0x55f0c076c9e0 2024-04-26T21:43:35.114 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:43:35.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.115+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v105'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.115+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0775610 con 0x55f0c076c9e0 2024-04-26T21:43:35.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.116+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v105'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.116+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0775610 con 0x55f0c076c9e0 2024-04-26T21:43:35.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.118+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v105'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.119+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.121+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v105'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.121+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.124+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v105'111 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.124+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.127+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v105'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.127+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.127 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:43:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.128+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v105'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.128+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.129+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v105'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.129+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.130+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v105'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.130+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.131+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v105'46 uv45 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.131+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.132+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v105'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.132+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745c0 con 0x55f0c076c9e0 2024-04-26T21:43:35.132 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:43:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.133+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v105'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.133+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745e0 con 0x55f0c076c9e0 2024-04-26T21:43:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.134+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v105'49 uv49 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.134+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745e0 con 0x55f0c076c9e0 2024-04-26T21:43:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.135+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v105'50 uv50 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7f48023070 con 0x55f0c076c9e0 2024-04-26T21:43:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.135+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745e0 con 0x55f0c0770dc0 2024-04-26T21:43:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.136+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v105'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.136+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c07745e0 con 0x55f0c0770dc0 2024-04-26T21:43:35.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.137+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v105'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.137+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07745e0 con 0x55f0c0770dc0 2024-04-26T21:43:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.138+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v105'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.138 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:43:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.138+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.139+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v105'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.139+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774640 con 0x55f0c0770dc0 2024-04-26T21:43:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.140+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v105'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.140+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.141+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v105'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.141+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.142 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:43:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.142+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v105'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.142+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.143+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [write 0~34] v105'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.143+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.144+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v105'97 uv97 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.144+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.145+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 100.00000000 [create] v105'98 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.145+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.146+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 100.00000000 [delete] v105'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.146+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.146 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:43:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.147+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v105'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.147+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0774a50 con 0x55f0c0770dc0 2024-04-26T21:43:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.148+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [omap-set-header] v105'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.148+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07777b0 con 0x55f0c0770dc0 2024-04-26T21:43:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.149+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v105'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.149+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f0c0775340 0x55f0c0777cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.149+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777730 con 0x55f0c0775340 2024-04-26T21:43:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.149+0000 7f7f580c9640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f0c0775340 0x55f0c0777cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.150+0000 7f7f580c9640 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f0c0775340 0x55f0c0777cd0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.151+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(99 mds0_sessionmap [create] v105'44 uv43 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0775340 2024-04-26T21:43:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.151+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0777730 con 0x55f0c0775340 2024-04-26T21:43:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.152+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v105'45 uv45 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0775340 2024-04-26T21:43:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.152+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c07777b0 con 0x55f0c0775340 2024-04-26T21:43:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.153+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(101 mds0_sessionmap [create] v105'46 uv46 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0775340 2024-04-26T21:43:35.153 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:43:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.153+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0775340 2024-04-26T21:43:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.154+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v105'47 uv47 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0775340 2024-04-26T21:43:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.154+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0770dc0 2024-04-26T21:43:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.155+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(103 609.00000000 [create] v105'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.155+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0770dc0 2024-04-26T21:43:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.156+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(104 609.00000000 [delete] v105'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.156+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0770dc0 2024-04-26T21:43:35.157 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:43:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.157+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v105'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.157+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0770dc0 2024-04-26T21:43:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.158+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v105'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.158+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0770dc0 2024-04-26T21:43:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.159+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v105'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.159+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0770dc0 2024-04-26T21:43:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.160+0000 7f7f588ca640 1 -- 172.21.15.40:0/709214371 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [omap-set-header] v105'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f5001c040 con 0x55f0c0770dc0 2024-04-26T21:43:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.160+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0769270 2024-04-26T21:43:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.161+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v105'47 uv46 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.161+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0769270 2024-04-26T21:43:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.162+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v105'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.162+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0769270 2024-04-26T21:43:35.163 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:43:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.163+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v105'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.163+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0769270 2024-04-26T21:43:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.164+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v105'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.164+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0769270 2024-04-26T21:43:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.165+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v105'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.166+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0777870 con 0x55f0c0769270 2024-04-26T21:43:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.167+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v105'119 uv111 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.167+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.168+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v105'120 uv120 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.168+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.168 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:43:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.169+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v105'121 uv121 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.169+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.170+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v105'122 uv122 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.170+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.171+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v105'123 uv123 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.171+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.172+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v105'124 uv118 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.172+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.173+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v105'125 uv125 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.173+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.174 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:43:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.174+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v105'126 uv126 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.174+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0779220 con 0x55f0c0769270 2024-04-26T21:43:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.175+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v105'127 uv127 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.175+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0779220 con 0x55f0c0769270 2024-04-26T21:43:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.176+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v105'128 uv128 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.176+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.177+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v105'129 uv129 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.177+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x55f0c0778210 con 0x55f0c0769270 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f580c9640 1 -- 172.21.15.40:0/709214371 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v105'130 uv130 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7f40054970 con 0x55f0c0769270 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f0c0763b00 msgr2=0x55f0c0766bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55f0c0763b00 0x55f0c0766bf0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f0c0758b90 msgr2=0x55f0c0758f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55f0c0758b90 0x55f0c0758f70 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f0c076c9e0 msgr2=0x55f0c076edd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f0c076c9e0 0x55f0c076edd0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f0c0775340 msgr2=0x55f0c0777cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55f0c0775340 0x55f0c0777cd0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 reap_dead start 2024-04-26T21:43:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f0c0769270 msgr2=0x55f0c076b660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f0c0769270 0x55f0c076b660 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f590cb640 1 -- 172.21.15.40:0/709214371 reap_dead start 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f0c0770dc0 msgr2=0x55f0c07731b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f0c0770dc0 0x55f0c07731b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f3806b300 msgr2=0x7f7f3806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f3806b300 0x7f7f3806d7c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7f50017040 tx=0x7f7f5000c000 comp rx=0 tx=0).stop 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c06c3610 msgr2=0x55f0c0756690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c06c3610 0x55f0c0756690 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c190 tx=0x7f7f4800c650 comp rx=0 tx=0).stop 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 shutdown_connections 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f0c0770dc0 0x55f0c07731b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f0c0769270 0x55f0c076b660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f3806b300 0x7f7f3806d7c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 --2- 172.21.15.40:0/709214371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0c06c3610 0x55f0c0756690 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 >> 172.21.15.40:0/709214371 conn(0x55f0c05e6ab0 msgr2=0x55f0c06a4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 shutdown_connections 2024-04-26T21:43:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.179+0000 7f7f5add0f40 1 -- 172.21.15.40:0/709214371 wait complete. 2024-04-26T21:43:35.183 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000 2024-04-26T21:43:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.265+0000 7f9ae4472f40 1 Processor -- start 2024-04-26T21:43:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.265+0000 7f9ae4472f40 1 -- start start 2024-04-26T21:43:35.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae4472f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c094bc140 0x564c0953d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae4472f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095484e0 0x564c095475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae4472f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c095488c0 0x564c0949fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae4472f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564c094a4c50 con 0x564c095488c0 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae4472f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564c0941be40 con 0x564c095484e0 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae4472f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564c094a4690 con 0x564c094bc140 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c095488c0 0x564c0949fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c095488c0 0x564c0949fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54114/0 (socket says 172.21.15.40:54114) 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 -- 172.21.15.40:0/3810942837 learned_addr learned my addr 172.21.15.40:0/3810942837 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae1f6c640 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c094bc140 0x564c0953d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae176b640 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095484e0 0x564c095475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 -- 172.21.15.40:0/3810942837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c094bc140 msgr2=0x564c0953d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c094bc140 0x564c0953d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 -- 172.21.15.40:0/3810942837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095484e0 msgr2=0x564c095475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095484e0 0x564c095475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 -- 172.21.15.40:0/3810942837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c09476040 con 0x564c095488c0 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.266+0000 7f9ae276d640 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c095488c0 0x564c0949fa50 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40016e0 tx=0x7f9ad4001ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c538eb83233c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9adbfff640 1 -- 172.21.15.40:0/3810942837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40062e0 con 0x564c095488c0 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9adbfff640 1 -- 172.21.15.40:0/3810942837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4003650 con 0x564c095488c0 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9adbfff640 1 -- 172.21.15.40:0/3810942837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4003a00 con 0x564c095488c0 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 -- 172.21.15.40:0/3810942837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c095488c0 msgr2=0x564c0949fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c095488c0 0x564c0949fa50 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40016e0 tx=0x7f9ad4001ba0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 -- 172.21.15.40:0/3810942837 shutdown_connections 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c095488c0 0x564c0949fa50 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095484e0 0x564c095475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 --2- 172.21.15.40:0/3810942837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c094bc140 0x564c0953d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 -- 172.21.15.40:0/3810942837 >> 172.21.15.40:0/3810942837 conn(0x564c0949e8a0 msgr2=0x564c094bb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 -- 172.21.15.40:0/3810942837 shutdown_connections 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.267+0000 7f9ae4472f40 1 -- 172.21.15.40:0/3810942837 wait complete. 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.268+0000 7f9ae4472f40 1 Processor -- start 2024-04-26T21:43:35.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.268+0000 7f9ae4472f40 1 -- start start 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.268+0000 7f9ae4472f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c094bc140 0x564c0949fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.268+0000 7f9ae4472f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c095484e0 0x564c094b8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.268+0000 7f9ae4472f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095488c0 0x564c09475180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.268+0000 7f9ae4472f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564c094733c0 con 0x564c094bc140 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.268+0000 7f9ae4472f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564c095471f0 con 0x564c095488c0 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.268+0000 7f9ae4472f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564c094bbe70 con 0x564c095484e0 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c094bc140 0x564c0949fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c094bc140 0x564c0949fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54128/0 (socket says 172.21.15.40:54128) 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 -- 172.21.15.40:0/1301123828 learned_addr learned my addr 172.21.15.40:0/1301123828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae276d640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095488c0 0x564c09475180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae176b640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c095484e0 0x564c094b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 -- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c095484e0 msgr2=0x564c094b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c095484e0 0x564c094b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 -- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095488c0 msgr2=0x564c09475180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c095488c0 0x564c09475180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 -- 172.21.15.40:0/1301123828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c09472020 con 0x564c094bc140 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae1f6c640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c094bc140 0x564c0949fff0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f9adc008730 tx=0x7f9adc008820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9adaffd640 1 -- 172.21.15.40:0/1301123828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0147d0 con 0x564c094bc140 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9adaffd640 1 -- 172.21.15.40:0/1301123828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9adc01a070 con 0x564c094bc140 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9adaffd640 1 -- 172.21.15.40:0/1301123828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc014b90 con 0x564c094bc140 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564c09472300 con 0x564c094bc140 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.269+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c094728c0 con 0x564c094bc140 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.271+0000 7f9adaffd640 1 -- 172.21.15.40:0/1301123828 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9adc013030 con 0x564c094bc140 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.271+0000 7f9adaffd640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ac406b2d0 0x7f9ac406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.271+0000 7f9adaffd640 1 -- 172.21.15.40:0/1301123828 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7f9adc08b080 con 0x564c094bc140 2024-04-26T21:43:35.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.271+0000 7f9ae176b640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ac406b2d0 0x7f9ac406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.272+0000 7f9ae4472f40 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564c094756c0 0x564c0954ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.272+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x564c095362e0 con 0x564c094756c0 2024-04-26T21:43:35.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.272+0000 7f9ae276d640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564c094756c0 0x564c0954ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.272+0000 7f9ae176b640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ac406b2d0 0x7f9ac406d790 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9acc058000 tx=0x7f9acc00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.272+0000 7f9ae276d640 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564c094756c0 0x564c0954ac90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.273+0000 7f9ae276d640 1 -- 172.21.15.40:0/1301123828 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 100.00000000 [delete] v105'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9ad400f030 con 0x564c094756c0 2024-04-26T21:43:35.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.273+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564c094756c0 msgr2=0x564c0954ac90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.273+0000 7f9ae4472f40 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564c094756c0 0x564c0954ac90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ac406b2d0 msgr2=0x7f9ac406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae4472f40 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9ac406b2d0 0x7f9ac406d790 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9acc058000 tx=0x7f9acc00a000 comp rx=0 tx=0).stop 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c094bc140 msgr2=0x564c0949fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae4472f40 1 --2- 172.21.15.40:0/1301123828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c094bc140 0x564c0949fff0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f9adc008730 tx=0x7f9adc008820 comp rx=0 tx=0).stop 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae276d640 1 -- 172.21.15.40:0/1301123828 reap_dead start 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 shutdown_connections 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 >> 172.21.15.40:0/1301123828 conn(0x564c0949e8a0 msgr2=0x564c0953a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 shutdown_connections 2024-04-26T21:43:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.274+0000 7f9ae4472f40 1 -- 172.21.15.40:0/1301123828 wait complete. 2024-04-26T21:43:35.276 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:43:35.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.419+0000 7f58f6f39640 1 Processor -- start 2024-04-26T21:43:35.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.419+0000 7f58f6f39640 1 -- start start 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58f6f39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f015f170 0x7f58f00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58f6f39640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58f00d3bc0 0x7f58f00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58f6f39640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f00d1040 con 0x7f58f015f550 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58f6f39640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f58f00c9430 con 0x7f58f00d3bc0 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58f6f39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58f00c9680 con 0x7f58f015f170 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58f54d6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58f00d3bc0 0x7f58f00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58effff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f58f015f550 0x7f58f00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53484/0 (socket says 172.21.15.40:53484) 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58effff640 1 -- 172.21.15.40:0/4213711857 learned_addr learned my addr 172.21.15.40:0/4213711857 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:35.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.420+0000 7f58f4cd5640 1 --2- 172.21.15.40:0/4213711857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f015f170 0x7f58f00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58f54d6640 1 -- 172.21.15.40:0/4213711857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f015f170 msgr2=0x7f58f00d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58f54d6640 1 --2- 172.21.15.40:0/4213711857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f015f170 0x7f58f00d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58f54d6640 1 -- 172.21.15.40:0/4213711857 >> v1:172.21.15.40:6789/0 conn(0x7f58f015f550 legacy=0x7f58f00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58f54d6640 1 -- 172.21.15.40:0/4213711857 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f00c9960 con 0x7f58f00d3bc0 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58ef7fe640 1 -- 172.21.15.40:0/4213711857 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020212876 0 0) 0x7f58e0001110 con 0x7f58f015f550 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58f4cd5640 1 --2- 172.21.15.40:0/4213711857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f015f170 0x7f58f00d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58f54d6640 1 --2- 172.21.15.40:0/4213711857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f00d3bc0 0x7f58f00d2cb0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f58d80097a0 tx=0x7f58d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2f7dc7f11b7b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58ef7fe640 1 -- 172.21.15.40:0/4213711857 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8037070 con 0x7f58f00d3bc0 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58ef7fe640 1 -- 172.21.15.40:0/4213711857 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d8032440 con 0x7f58f00d3bc0 2024-04-26T21:43:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.421+0000 7f58ef7fe640 1 -- 172.21.15.40:0/4213711857 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8032810 con 0x7f58f00d3bc0 2024-04-26T21:43:35.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.422+0000 7f58f6f39640 1 -- 172.21.15.40:0/4213711857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f00d3bc0 msgr2=0x7f58f00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.422+0000 7f58f6f39640 1 --2- 172.21.15.40:0/4213711857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f00d3bc0 0x7f58f00d2cb0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f58d80097a0 tx=0x7f58d802a180 comp rx=0 tx=0).stop 2024-04-26T21:43:35.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.422+0000 7f58f6f39640 1 -- 172.21.15.40:0/4213711857 shutdown_connections 2024-04-26T21:43:35.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.422+0000 7f58f6f39640 1 --2- 172.21.15.40:0/4213711857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f00d3bc0 0x7f58f00d2cb0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.422+0000 7f58f6f39640 1 --2- 172.21.15.40:0/4213711857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f015f170 0x7f58f00d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.422+0000 7f58f6f39640 1 -- 172.21.15.40:0/4213711857 >> 172.21.15.40:0/4213711857 conn(0x7f58f00d4f60 msgr2=0x7f58f00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:35.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.422+0000 7f58f6f39640 1 -- 172.21.15.40:0/4213711857 shutdown_connections 2024-04-26T21:43:35.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.422+0000 7f58f6f39640 1 -- 172.21.15.40:0/4213711857 wait complete. 2024-04-26T21:43:35.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.423+0000 7f58f6f39640 1 Processor -- start 2024-04-26T21:43:35.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.423+0000 7f58f6f39640 1 -- start start 2024-04-26T21:43:35.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58f6f39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f00d3bc0 0x7f58f00cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58f6f39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f015f170 0x7f58f00ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58f6f39640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f01f1e70 0x7f58f01f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58f6f39640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58f00ca5c0 con 0x7f58f015f170 2024-04-26T21:43:35.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58f6f39640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58f00ca440 con 0x7f58f01f1e70 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58f6f39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58f00ca740 con 0x7f58f00d3bc0 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f015f170 0x7f58f00ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f015f170 0x7f58f00ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54140/0 (socket says 172.21.15.40:54140) 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58effff640 1 -- 172.21.15.40:0/693842284 learned_addr learned my addr 172.21.15.40:0/693842284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58f54d6640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f01f1e70 0x7f58f01f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58f4cd5640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f00d3bc0 0x7f58f00cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58effff640 1 -- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f00d3bc0 msgr2=0x7f58f00cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58effff640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f00d3bc0 0x7f58f00cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58effff640 1 -- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f01f1e70 msgr2=0x7f58f01f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.424+0000 7f58effff640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f01f1e70 0x7f58f01f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.425+0000 7f58effff640 1 -- 172.21.15.40:0/693842284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f01f5ba0 con 0x7f58f015f170 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.425+0000 7f58effff640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f015f170 0x7f58f00ce490 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f58e000c430 tx=0x7f58e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.425+0000 7f58ed7fa640 1 -- 172.21.15.40:0/693842284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00074e0 con 0x7f58f015f170 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.425+0000 7f58ed7fa640 1 -- 172.21.15.40:0/693842284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e00047e0 con 0x7f58f015f170 2024-04-26T21:43:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.425+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f00cd090 con 0x7f58f015f170 2024-04-26T21:43:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.425+0000 7f58ed7fa640 1 -- 172.21.15.40:0/693842284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0004bd0 con 0x7f58f015f170 2024-04-26T21:43:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.425+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f00cd650 con 0x7f58f015f170 2024-04-26T21:43:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.426+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f00694f0 con 0x7f58f015f170 2024-04-26T21:43:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.426+0000 7f58ed7fa640 1 -- 172.21.15.40:0/693842284 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58e0011070 con 0x7f58f015f170 2024-04-26T21:43:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.427+0000 7f58ed7fa640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58d006c8f0 0x7f58d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.427+0000 7f58ed7fa640 1 -- 172.21.15.40:0/693842284 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7f58e008bae0 con 0x7f58f015f170 2024-04-26T21:43:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.427+0000 7f58f4cd5640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58d006c8f0 0x7f58d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.428+0000 7f58f4cd5640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58d006c8f0 0x7f58d006edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f58e8003b80 tx=0x7f58e8009890 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.431+0000 7f58ed7fa640 1 -- 172.21.15.40:0/693842284 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f58f00d1040 con 0x7f58f015f170 2024-04-26T21:43:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.657+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f58f00d19c0 con 0x7f58f015f170 2024-04-26T21:43:35.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.950+0000 7f58ed7fa640 1 -- 172.21.15.40:0/693842284 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v150) ==== 162+0+0 (secure 0 0 0) 0x7f58e00593f0 con 0x7f58f015f170 2024-04-26T21:43:35.950 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:43:35.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58d006c8f0 msgr2=0x7f58d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58d006c8f0 0x7f58d006edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f58e8003b80 tx=0x7f58e8009890 comp rx=0 tx=0).stop 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f015f170 msgr2=0x7f58f00ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f015f170 0x7f58f00ce490 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f58e000c430 tx=0x7f58e000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 shutdown_connections 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58d006c8f0 0x7f58d006edb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f01f1e70 0x7f58f01f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f015f170 0x7f58f00ce490 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 --2- 172.21.15.40:0/693842284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f00d3bc0 0x7f58f00cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 >> 172.21.15.40:0/693842284 conn(0x7f58f00d4f60 msgr2=0x7f58f00d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 shutdown_connections 2024-04-26T21:43:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:35.952+0000 7f58f6f39640 1 -- 172.21.15.40:0/693842284 wait complete. 2024-04-26T21:43:35.965 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:36.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.067+0000 7effbf6d1640 1 Processor -- start 2024-04-26T21:43:36.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.067+0000 7effbf6d1640 1 -- start start 2024-04-26T21:43:36.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effbf6d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7effb806c6a0 0x7effb80af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effbf6d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb80b4480 con 0x7effb806c240 2024-04-26T21:43:36.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effbf6d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb80b4720 con 0x7effb815f1d0 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effbf6d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7effb80ae8c0 con 0x7effb806c6a0 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effbcc6c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7effb806c240 0x7effb80aed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53492/0 (socket says 172.21.15.40:53492) 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effbcc6c640 1 -- 172.21.15.40:0/2812189286 learned_addr learned my addr 172.21.15.40:0/2812189286 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effbdc6e640 1 --2- 172.21.15.40:0/2812189286 >> v2:172.21.15.184:3300/0 conn(0x7effb806c6a0 0x7effb80af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614860044 0 0) 0x7effb80b4480 con 0x7effb806c240 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effa4003520 con 0x7effb806c240 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2240845976 0 0) 0x7effa4003520 con 0x7effb806c240 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 >> v2:172.21.15.184:3300/0 conn(0x7effb806c6a0 msgr2=0x7effb80af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effaffff640 1 --2- 172.21.15.40:0/2812189286 >> v2:172.21.15.184:3300/0 conn(0x7effb806c6a0 0x7effb80af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 >> v1:172.21.15.184:6789/0 conn(0x7effb815f1d0 legacy=0x7effb806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb80b4a00 con 0x7effb806c240 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.068+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7effa8001670 con 0x7effb806c240 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 >> v1:172.21.15.40:6789/0 conn(0x7effb806c240 legacy=0x7effb80aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:36.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effaffff640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa40039b0 0x7effa4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effaffff640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4004800 0x7effa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effaffff640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40051c0 0x7effa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7effb80ae8c0 con 0x7effa4004800 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effa400c060 con 0x7effa40039b0 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7effa400c260 con 0x7effa40051c0 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbcc6c640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4004800 0x7effa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbd46d640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa40039b0 0x7effa4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbdc6e640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40051c0 0x7effa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbcc6c640 1 -- 172.21.15.40:0/2812189286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40051c0 msgr2=0x7effa400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbcc6c640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40051c0 0x7effa400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbcc6c640 1 -- 172.21.15.40:0/2812189286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa40039b0 msgr2=0x7effa4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbcc6c640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa40039b0 0x7effa4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbcc6c640 1 -- 172.21.15.40:0/2812189286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa4003520 con 0x7effa4004800 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.069+0000 7effbdc6e640 1 -- 172.21.15.40:0/2812189286 reap_dead start 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effbcc6c640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4004800 0x7effa4004c80 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7effa800c240 tx=0x7effa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e393b7c0624cca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:36.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8007cf0 con 0x7effa4004800 2024-04-26T21:43:36.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa8005af0 con 0x7effa4004800 2024-04-26T21:43:36.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effaffff640 1 -- 172.21.15.40:0/2812189286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8007070 con 0x7effa4004800 2024-04-26T21:43:36.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effbf6d1640 1 -- 172.21.15.40:0/2812189286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4004800 msgr2=0x7effa4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:36.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effbf6d1640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4004800 0x7effa4004c80 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7effa800c240 tx=0x7effa4005830 comp rx=0 tx=0).stop 2024-04-26T21:43:36.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effbf6d1640 1 -- 172.21.15.40:0/2812189286 shutdown_connections 2024-04-26T21:43:36.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effbf6d1640 1 --2- 172.21.15.40:0/2812189286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4004800 0x7effa4004c80 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.070+0000 7effbf6d1640 1 -- 172.21.15.40:0/2812189286 >> 172.21.15.40:0/2812189286 conn(0x7effb80b3180 msgr2=0x7effb80b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.071+0000 7effbf6d1640 1 -- 172.21.15.40:0/2812189286 shutdown_connections 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.071+0000 7effbf6d1640 1 -- 172.21.15.40:0/2812189286 wait complete. 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.072+0000 7effbf6d1640 1 Processor -- start 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.072+0000 7effbf6d1640 1 -- start start 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.072+0000 7effbf6d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4004800 0x7effb81f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.072+0000 7effbf6d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb8159690 0x7effb80af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.072+0000 7effbf6d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb8153ee0 0x7effb8154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.072+0000 7effbf6d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7effb806cb30 con 0x7effb8153ee0 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.072+0000 7effbf6d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effb80b4fc0 con 0x7effa4004800 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.072+0000 7effbf6d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7effb8075ea0 con 0x7effb8159690 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb8153ee0 0x7effb8154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb8153ee0 0x7effb8154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54164/0 (socket says 172.21.15.40:54164) 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 -- 172.21.15.40:0/21989811 learned_addr learned my addr 172.21.15.40:0/21989811 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:36.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbcc6c640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb8159690 0x7effb80af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbd46d640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4004800 0x7effb81f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 -- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb8159690 msgr2=0x7effb80af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb8159690 0x7effb80af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 -- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4004800 msgr2=0x7effb81f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4004800 0x7effb81f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 -- 172.21.15.40:0/21989811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb81f8200 con 0x7effb8153ee0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbcc6c640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb8159690 0x7effb80af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbd46d640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4004800 0x7effb81f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbdc6e640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb8153ee0 0x7effb8154360 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7effb806d3b0 tx=0x7effb000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effadffb640 1 -- 172.21.15.40:0/21989811 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb000fce0 con 0x7effb8153ee0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effadffb640 1 -- 172.21.15.40:0/21989811 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb001d070 con 0x7effb8153ee0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb81f84e0 con 0x7effb8153ee0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effadffb640 1 -- 172.21.15.40:0/21989811 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0015670 con 0x7effb8153ee0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.073+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb81f8aa0 con 0x7effb8153ee0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.074+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb8068fc0 con 0x7effb8153ee0 2024-04-26T21:43:36.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.075+0000 7effadffb640 1 -- 172.21.15.40:0/21989811 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effb0015810 con 0x7effb8153ee0 2024-04-26T21:43:36.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.075+0000 7effadffb640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.075+0000 7effadffb640 1 -- 172.21.15.40:0/21989811 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7077+0+0 (secure 0 0 0) 0x7effb008ca40 con 0x7effb8153ee0 2024-04-26T21:43:36.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.075+0000 7effbd46d640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.076+0000 7effbd46d640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff9c06c8f0 0x7eff9c06edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7effb4005580 tx=0x7effb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:36.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.080+0000 7effadffb640 1 -- 172.21.15.40:0/21989811 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7effb0010040 con 0x7effb8153ee0 2024-04-26T21:43:36.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.307+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effb80694f0 con 0x7effb8153ee0 2024-04-26T21:43:36.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.308+0000 7effadffb640 1 -- 172.21.15.40:0/21989811 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) ==== 95+0+5190 (secure 0 0 0) 0x7effb005a910 con 0x7effb8153ee0 2024-04-26T21:43:36.308 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:36.308 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":151,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:35.952912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"c","rank":0,"incarnation":151,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/2962699830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2962699830},{"type":"v1","addr":"172.21.15.40:6833","nonce":2962699830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:36.309 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 151 2024-04-26T21:43:36.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff9c06c8f0 msgr2=0x7eff9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff9c06c8f0 0x7eff9c06edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7effb4005580 tx=0x7effb4009040 comp rx=0 tx=0).stop 2024-04-26T21:43:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb8153ee0 msgr2=0x7effb8154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb8153ee0 0x7effb8154360 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7effb806d3b0 tx=0x7effb000f1b0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 shutdown_connections 2024-04-26T21:43:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb8153ee0 0x7effb8154360 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb8159690 0x7effb80af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 --2- 172.21.15.40:0/21989811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4004800 0x7effb81f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 >> 172.21.15.40:0/21989811 conn(0x7effb80b3180 msgr2=0x7effb80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:36.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 shutdown_connections 2024-04-26T21:43:36.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:36.310+0000 7effbf6d1640 1 -- 172.21.15.40:0/21989811 wait complete. 2024-04-26T21:43:36.321 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:43:38.975 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:38.974+0000 7f7f0ee34640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-04-26T21:43:38.976 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:38.975+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:43:38.976 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:38.975+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:43:38.984 INFO:tasks.ceph.mds.c.smithi040.stderr: -20> 2024-04-26T21:43:38.974+0000 7f7f0ee34640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-04-26T21:43:39.012 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:43:41.321 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:41.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.434+0000 7f2a10450640 1 Processor -- start 2024-04-26T21:43:41.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.434+0000 7f2a10450640 1 -- start start 2024-04-26T21:43:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.435+0000 7f2a10450640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a0815b600 0x7f2a08071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.435+0000 7f2a10450640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2a080cf5e0 0x7f2a080ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.435+0000 7f2a10450640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a08072450 con 0x7f2a080cbbd0 2024-04-26T21:43:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.435+0000 7f2a10450640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0806e500 con 0x7f2a080cf5e0 2024-04-26T21:43:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.435+0000 7f2a10450640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2a080cb9b0 con 0x7f2a0815b600 2024-04-26T21:43:41.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0e1ec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a0815b600 0x7f2a08071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0d9eb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2a080cf5e0 0x7f2a080ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0d9eb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2a080cf5e0 0x7f2a080ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:51708/0 (socket says 172.21.15.40:51708) 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0d9eb640 1 -- 172.21.15.40:0/3468713553 learned_addr learned my addr 172.21.15.40:0/3468713553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0e9ed640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f2a080cbbd0 0x7f2a0806ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:33808/0 (socket says 172.21.15.40:33808) 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0e1ec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a0815b600 0x7f2a08071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:48842/0 (socket says 172.21.15.40:48842) 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0d9eb640 1 -- 172.21.15.40:0/3468713553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a0815b600 msgr2=0x7f2a08071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0d9eb640 1 --2- 172.21.15.40:0/3468713553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a0815b600 0x7f2a08071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0d9eb640 1 -- 172.21.15.40:0/3468713553 >> v1:172.21.15.184:6789/0 conn(0x7f2a080cbbd0 legacy=0x7f2a0806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0d9eb640 1 -- 172.21.15.40:0/3468713553 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0815b9e0 con 0x7f2a080cf5e0 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0d1ea640 1 -- 172.21.15.40:0/3468713553 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076241612 0 0) 0x7f2a00001490 con 0x7f2a080cbbd0 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.436+0000 7f2a0e1ec640 1 --2- 172.21.15.40:0/3468713553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a0815b600 0x7f2a08071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:43:41.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.437+0000 7f2a0d9eb640 1 --2- 172.21.15.40:0/3468713553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cf5e0 0x7f2a080ce9d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009870 tx=0x7f29f802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a763836e4f0800af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.437+0000 7f2a0d1ea640 1 -- 172.21.15.40:0/3468713553 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8037070 con 0x7f2a080cf5e0 2024-04-26T21:43:41.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.437+0000 7f2a0d1ea640 1 -- 172.21.15.40:0/3468713553 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f8032440 con 0x7f2a080cf5e0 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.437+0000 7f2a0d1ea640 1 -- 172.21.15.40:0/3468713553 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8032810 con 0x7f2a080cf5e0 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.438+0000 7f2a10450640 1 -- 172.21.15.40:0/3468713553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cf5e0 msgr2=0x7f2a080ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.438+0000 7f2a10450640 1 --2- 172.21.15.40:0/3468713553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cf5e0 0x7f2a080ce9d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009870 tx=0x7f29f802a250 comp rx=0 tx=0).stop 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.438+0000 7f2a10450640 1 -- 172.21.15.40:0/3468713553 shutdown_connections 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.438+0000 7f2a10450640 1 --2- 172.21.15.40:0/3468713553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cf5e0 0x7f2a080ce9d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.438+0000 7f2a10450640 1 --2- 172.21.15.40:0/3468713553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a0815b600 0x7f2a08071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.438+0000 7f2a10450640 1 -- 172.21.15.40:0/3468713553 >> 172.21.15.40:0/3468713553 conn(0x7f2a080d0be0 msgr2=0x7f2a080d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.438+0000 7f2a10450640 1 -- 172.21.15.40:0/3468713553 shutdown_connections 2024-04-26T21:43:41.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.438+0000 7f2a10450640 1 -- 172.21.15.40:0/3468713553 wait complete. 2024-04-26T21:43:41.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.439+0000 7f2a10450640 1 Processor -- start 2024-04-26T21:43:41.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.439+0000 7f2a10450640 1 -- start start 2024-04-26T21:43:41.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a10450640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cbbd0 0x7f2a080c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a10450640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a080cf5e0 0x7f2a080c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a10450640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0815b600 0x7f2a08152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a10450640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2a080cb640 con 0x7f2a0815b600 2024-04-26T21:43:41.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a10450640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2a080cb360 con 0x7f2a080cbbd0 2024-04-26T21:43:41.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a10450640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2a080d02f0 con 0x7f2a080cf5e0 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a0e9ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0815b600 0x7f2a08152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a0e9ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0815b600 0x7f2a08152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55702/0 (socket says 172.21.15.40:55702) 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a0e9ed640 1 -- 172.21.15.40:0/1738080061 learned_addr learned my addr 172.21.15.40:0/1738080061 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a0e1ec640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cbbd0 0x7f2a080c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.440+0000 7f2a0d9eb640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a080cf5e0 0x7f2a080c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f2a0e9ed640 1 -- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a080cf5e0 msgr2=0x7f2a080c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f2a0e9ed640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a080cf5e0 0x7f2a080c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f2a0e9ed640 1 -- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cbbd0 msgr2=0x7f2a080c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f2a0e9ed640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cbbd0 0x7f2a080c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f2a0e9ed640 1 -- 172.21.15.40:0/1738080061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0815bc80 con 0x7f2a0815b600 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f2a0e9ed640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0815b600 0x7f2a08152c90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000c7c0 tx=0x7f2a0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f29feffd640 1 -- 172.21.15.40:0/1738080061 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00018070 con 0x7f2a0815b600 2024-04-26T21:43:41.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f29feffd640 1 -- 172.21.15.40:0/1738080061 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0000a5e0 con 0x7f2a0815b600 2024-04-26T21:43:41.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f29feffd640 1 -- 172.21.15.40:0/1738080061 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000a9d0 con 0x7f2a0815b600 2024-04-26T21:43:41.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a080ca220 con 0x7f2a0815b600 2024-04-26T21:43:41.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.441+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a080ca7e0 con 0x7f2a0815b600 2024-04-26T21:43:41.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.442+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a08071f10 con 0x7f2a0815b600 2024-04-26T21:43:41.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.443+0000 7f29feffd640 1 -- 172.21.15.40:0/1738080061 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a0001b070 con 0x7f2a0815b600 2024-04-26T21:43:41.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.443+0000 7f29feffd640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f29ec06c8f0 0x7f29ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.443+0000 7f29feffd640 1 -- 172.21.15.40:0/1738080061 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7163+0+0 (secure 0 0 0) 0x7f2a0008b690 con 0x7f2a0815b600 2024-04-26T21:43:41.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.443+0000 7f2a0e1ec640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f29ec06c8f0 0x7f29ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.444+0000 7f2a0e1ec640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f29ec06c8f0 0x7f29ec06edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009a30 tx=0x7f2a04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.447+0000 7f29feffd640 1 -- 172.21.15.40:0/1738080061 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a08071f10 con 0x7f2a0815b600 2024-04-26T21:43:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.673+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a080ce7b0 con 0x7f2a0815b600 2024-04-26T21:43:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.674+0000 7f29feffd640 1 -- 172.21.15.40:0/1738080061 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) ==== 95+0+5181 (secure 0 0 0) 0x7f2a08072450 con 0x7f2a0815b600 2024-04-26T21:43:41.675 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:41.675 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9183,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2899400637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2899400637},{"type":"v1","addr":"172.21.15.40:6833","nonce":2899400637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:38.977099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 156 2024-04-26T21:43:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f29ec06c8f0 msgr2=0x7f29ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f29ec06c8f0 0x7f29ec06edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009a30 tx=0x7f2a04008040 comp rx=0 tx=0).stop 2024-04-26T21:43:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0815b600 msgr2=0x7f2a08152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0815b600 0x7f2a08152c90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000c7c0 tx=0x7f2a0000cc80 comp rx=0 tx=0).stop 2024-04-26T21:43:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 shutdown_connections 2024-04-26T21:43:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f29ec06c8f0 0x7f29ec06edb0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0815b600 0x7f2a08152c90 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a080cf5e0 0x7f2a080c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 --2- 172.21.15.40:0/1738080061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a080cbbd0 0x7f2a080c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 >> 172.21.15.40:0/1738080061 conn(0x7f2a080d0be0 msgr2=0x7f2a08156060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 shutdown_connections 2024-04-26T21:43:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.676+0000 7f2a10450640 1 -- 172.21.15.40:0/1738080061 wait complete. 2024-04-26T21:43:41.688 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:43:41.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:41.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.788+0000 7f75dd208640 1 Processor -- start 2024-04-26T21:43:41.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.788+0000 7f75dd208640 1 -- start start 2024-04-26T21:43:41.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75dd208640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f75d80d3bc0 0x7f75d80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75dd208640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80b5f20 0x7f75d80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75dd208640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d80b7e50 con 0x7f75d8162880 2024-04-26T21:43:41.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75dd208640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f75d80d0580 con 0x7f75d80d3bc0 2024-04-26T21:43:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75dd208640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75d80d1800 con 0x7f75d80b5f20 2024-04-26T21:43:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75d659c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f75d80d3bc0 0x7f75d80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75d759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80b5f20 0x7f75d80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75d659c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f75d80d3bc0 0x7f75d80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:51734/0 (socket says 172.21.15.40:51734) 2024-04-26T21:43:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.789+0000 7f75d659c640 1 -- 172.21.15.40:0/2162939804 learned_addr learned my addr 172.21.15.40:0/2162939804 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.790+0000 7f75d659c640 1 -- 172.21.15.40:0/2162939804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80b5f20 msgr2=0x7f75d80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.790+0000 7f75d659c640 1 --2- 172.21.15.40:0/2162939804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80b5f20 0x7f75d80b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.790+0000 7f75d659c640 1 -- 172.21.15.40:0/2162939804 >> v1:172.21.15.184:6789/0 conn(0x7f75d8162880 legacy=0x7f75d80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.790+0000 7f75d659c640 1 -- 172.21.15.40:0/2162939804 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d80d07d0 con 0x7f75d80d3bc0 2024-04-26T21:43:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.790+0000 7f75d5d9b640 1 -- 172.21.15.40:0/2162939804 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3900114569 0 0) 0x7f75d80b7e50 con 0x7f75d8162880 2024-04-26T21:43:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.790+0000 7f75d759e640 1 --2- 172.21.15.40:0/2162939804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80b5f20 0x7f75d80b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:43:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.790+0000 7f75d659c640 1 --2- 172.21.15.40:0/2162939804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80d3bc0 0x7f75d80d2cb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009870 tx=0x7f75c002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8410ea4fd4ccbc7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75d5d9b640 1 -- 172.21.15.40:0/2162939804 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0037070 con 0x7f75d80d3bc0 2024-04-26T21:43:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75d5d9b640 1 -- 172.21.15.40:0/2162939804 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c0032440 con 0x7f75d80d3bc0 2024-04-26T21:43:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75d5d9b640 1 -- 172.21.15.40:0/2162939804 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0032810 con 0x7f75d80d3bc0 2024-04-26T21:43:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75dd208640 1 -- 172.21.15.40:0/2162939804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80d3bc0 msgr2=0x7f75d80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75dd208640 1 --2- 172.21.15.40:0/2162939804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80d3bc0 0x7f75d80d2cb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009870 tx=0x7f75c002a270 comp rx=0 tx=0).stop 2024-04-26T21:43:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75dd208640 1 -- 172.21.15.40:0/2162939804 shutdown_connections 2024-04-26T21:43:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75dd208640 1 --2- 172.21.15.40:0/2162939804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80b5f20 0x7f75d80b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75dd208640 1 --2- 172.21.15.40:0/2162939804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80d3bc0 0x7f75d80d2cb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75dd208640 1 -- 172.21.15.40:0/2162939804 >> 172.21.15.40:0/2162939804 conn(0x7f75d80d4f60 msgr2=0x7f75d80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75dd208640 1 -- 172.21.15.40:0/2162939804 shutdown_connections 2024-04-26T21:43:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.791+0000 7f75dd208640 1 -- 172.21.15.40:0/2162939804 wait complete. 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75dd208640 1 Processor -- start 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75dd208640 1 -- start start 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75dd208640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80b5f20 0x7f75d80b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75dd208640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80d3bc0 0x7f75d80b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75dd208640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d8162880 0x7f75d815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75dd208640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8162c60 con 0x7f75d8162880 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75dd208640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75d80d0d90 con 0x7f75d80b5f20 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75dd208640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75d8162de0 con 0x7f75d80d3bc0 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.793+0000 7f75d759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d8162880 0x7f75d815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d8162880 0x7f75d815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55714/0 (socket says 172.21.15.40:55714) 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d759e640 1 -- 172.21.15.40:0/4201015070 learned_addr learned my addr 172.21.15.40:0/4201015070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d659c640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80d3bc0 0x7f75d80b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d6d9d640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80b5f20 0x7f75d80b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d759e640 1 -- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80d3bc0 msgr2=0x7f75d80b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d759e640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80d3bc0 0x7f75d80b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d759e640 1 -- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80b5f20 msgr2=0x7f75d80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d759e640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80b5f20 0x7f75d80b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d759e640 1 -- 172.21.15.40:0/4201015070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d815b3a0 con 0x7f75d8162880 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75d759e640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d8162880 0x7f75d815a790 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f75c800efc0 tx=0x7f75c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75cf7fe640 1 -- 172.21.15.40:0/4201015070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8018030 con 0x7f75d8162880 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75cf7fe640 1 -- 172.21.15.40:0/4201015070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c800cdf0 con 0x7f75d8162880 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d815b680 con 0x7f75d8162880 2024-04-26T21:43:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75cf7fe640 1 -- 172.21.15.40:0/4201015070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8003bb0 con 0x7f75d8162880 2024-04-26T21:43:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.794+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d815bc40 con 0x7f75d8162880 2024-04-26T21:43:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.795+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d80694f0 con 0x7f75d8162880 2024-04-26T21:43:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.796+0000 7f75cf7fe640 1 -- 172.21.15.40:0/4201015070 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75c801b070 con 0x7f75d8162880 2024-04-26T21:43:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.796+0000 7f75cf7fe640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75b806c8f0 0x7f75b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.796+0000 7f75cf7fe640 1 -- 172.21.15.40:0/4201015070 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7163+0+0 (secure 0 0 0) 0x7f75c808ca20 con 0x7f75d8162880 2024-04-26T21:43:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.797+0000 7f75d6d9d640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75b806c8f0 0x7f75b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.797+0000 7f75d6d9d640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75b806c8f0 0x7f75b806edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f75d0000f60 tx=0x7f75d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:41.801+0000 7f75cf7fe640 1 -- 172.21.15.40:0/4201015070 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75c8010040 con 0x7f75d8162880 2024-04-26T21:43:42.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.029+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75d80b78b0 con 0x7f75d8162880 2024-04-26T21:43:42.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.029+0000 7f75cf7fe640 1 -- 172.21.15.40:0/4201015070 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) ==== 95+0+5181 (secure 0 0 0) 0x7f75c805a2e0 con 0x7f75d8162880 2024-04-26T21:43:42.030 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:42.031 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9183,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2899400637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2899400637},{"type":"v1","addr":"172.21.15.40:6833","nonce":2899400637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:38.977099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:42.031 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 156 2024-04-26T21:43:42.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75b806c8f0 msgr2=0x7f75b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75b806c8f0 0x7f75b806edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f75d0000f60 tx=0x7f75d0009040 comp rx=0 tx=0).stop 2024-04-26T21:43:42.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d8162880 msgr2=0x7f75d815a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d8162880 0x7f75d815a790 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f75c800efc0 tx=0x7f75c800c490 comp rx=0 tx=0).stop 2024-04-26T21:43:42.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 shutdown_connections 2024-04-26T21:43:42.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75b806c8f0 0x7f75b806edb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d8162880 0x7f75d815a790 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75d80d3bc0 0x7f75d80b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 --2- 172.21.15.40:0/4201015070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75d80b5f20 0x7f75d80b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 >> 172.21.15.40:0/4201015070 conn(0x7f75d80d4f60 msgr2=0x7f75d80d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:42.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 shutdown_connections 2024-04-26T21:43:42.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.032+0000 7f75dd208640 1 -- 172.21.15.40:0/4201015070 wait complete. 2024-04-26T21:43:42.043 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:42.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:42.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.144+0000 7f822be93640 1 Processor -- start 2024-04-26T21:43:42.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.144+0000 7f822be93640 1 -- start start 2024-04-26T21:43:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822be93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82240d06f0 0x7f82240d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822be93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82240b2850 0x7f82240b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822be93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82240d0b70 con 0x7f822415f3a0 2024-04-26T21:43:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822be93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f82240d0e20 con 0x7f82240b2850 2024-04-26T21:43:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822be93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f82240b1d70 con 0x7f82240d06f0 2024-04-26T21:43:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822a430640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82240b2850 0x7f82240b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822a430640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82240b2850 0x7f82240b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55728/0 (socket says 172.21.15.40:55728) 2024-04-26T21:43:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822a430640 1 -- 172.21.15.40:0/3350685743 learned_addr learned my addr 172.21.15.40:0/3350685743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822942e640 1 --2- 172.21.15.40:0/3350685743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82240d06f0 0x7f82240d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822a430640 1 -- 172.21.15.40:0/3350685743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82240d06f0 msgr2=0x7f82240d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822a430640 1 --2- 172.21.15.40:0/3350685743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82240d06f0 0x7f82240d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822a430640 1 -- 172.21.15.40:0/3350685743 >> v1:172.21.15.184:6789/0 conn(0x7f822415f3a0 legacy=0x7f82240d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:42.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.145+0000 7f822a430640 1 -- 172.21.15.40:0/3350685743 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82240b2050 con 0x7f82240b2850 2024-04-26T21:43:42.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.146+0000 7f822a430640 1 --2- 172.21.15.40:0/3350685743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f82240b2cc0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f821c009e50 tx=0x7f821c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb855b7bf26b3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.146+0000 7f8228c2d640 1 -- 172.21.15.40:0/3350685743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c036070 con 0x7f82240b2850 2024-04-26T21:43:42.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.146+0000 7f8228c2d640 1 -- 172.21.15.40:0/3350685743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821c031600 con 0x7f82240b2850 2024-04-26T21:43:42.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.146+0000 7f8228c2d640 1 -- 172.21.15.40:0/3350685743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0319d0 con 0x7f82240b2850 2024-04-26T21:43:42.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.146+0000 7f822be93640 1 -- 172.21.15.40:0/3350685743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 msgr2=0x7f82240b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.146+0000 7f822be93640 1 --2- 172.21.15.40:0/3350685743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f82240b2cc0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f821c009e50 tx=0x7f821c02a6c0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.147+0000 7f822be93640 1 -- 172.21.15.40:0/3350685743 shutdown_connections 2024-04-26T21:43:42.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.147+0000 7f822be93640 1 --2- 172.21.15.40:0/3350685743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f82240b2cc0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.147+0000 7f822be93640 1 --2- 172.21.15.40:0/3350685743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82240d06f0 0x7f82240d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.147+0000 7f822be93640 1 -- 172.21.15.40:0/3350685743 >> 172.21.15.40:0/3350685743 conn(0x7f82240d4f90 msgr2=0x7f82240d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:42.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.147+0000 7f822be93640 1 -- 172.21.15.40:0/3350685743 shutdown_connections 2024-04-26T21:43:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.147+0000 7f822be93640 1 -- 172.21.15.40:0/3350685743 wait complete. 2024-04-26T21:43:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.148+0000 7f822be93640 1 Processor -- start 2024-04-26T21:43:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.148+0000 7f822be93640 1 -- start start 2024-04-26T21:43:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822be93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f822415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822be93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82240d06f0 0x7f82241585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822be93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822415f3a0 0x7f82240b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822be93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f822415f900 con 0x7f82240b2850 2024-04-26T21:43:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822be93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f822415f780 con 0x7f82240d06f0 2024-04-26T21:43:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822be93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f82240d23c0 con 0x7f822415f3a0 2024-04-26T21:43:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f822415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f822415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55740/0 (socket says 172.21.15.40:55740) 2024-04-26T21:43:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 -- 172.21.15.40:0/2463170915 learned_addr learned my addr 172.21.15.40:0/2463170915 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:42.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822942e640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82240d06f0 0x7f82241585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822a430640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822415f3a0 0x7f82240b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 -- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822415f3a0 msgr2=0x7f82240b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822415f3a0 0x7f82240b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 -- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82240d06f0 msgr2=0x7f82241585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82240d06f0 0x7f82241585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 -- 172.21.15.40:0/2463170915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822415ffc0 con 0x7f82240b2850 2024-04-26T21:43:42.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822942e640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82240d06f0 0x7f82241585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:42.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f8229c2f640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f822415fa80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f822000c430 tx=0x7f822000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f821a7fc640 1 -- 172.21.15.40:0/2463170915 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220011790 con 0x7f82240b2850 2024-04-26T21:43:42.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f821a7fc640 1 -- 172.21.15.40:0/2463170915 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8220017070 con 0x7f82240b2850 2024-04-26T21:43:42.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f821a7fc640 1 -- 172.21.15.40:0/2463170915 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220011bb0 con 0x7f82240b2850 2024-04-26T21:43:42.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82240cc190 con 0x7f82240b2850 2024-04-26T21:43:42.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.149+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82240cc750 con 0x7f82240b2850 2024-04-26T21:43:42.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.150+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82240694f0 con 0x7f82240b2850 2024-04-26T21:43:42.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.151+0000 7f821a7fc640 1 -- 172.21.15.40:0/2463170915 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f822001c460 con 0x7f82240b2850 2024-04-26T21:43:42.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.151+0000 7f821a7fc640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f820806c8f0 0x7f820806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.151+0000 7f821a7fc640 1 -- 172.21.15.40:0/2463170915 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7163+0+0 (secure 0 0 0) 0x7f822008bfe0 con 0x7f82240b2850 2024-04-26T21:43:42.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.152+0000 7f822942e640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f820806c8f0 0x7f820806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.152+0000 7f822942e640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f820806c8f0 0x7f820806edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8214006fd0 tx=0x7f8214008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.156+0000 7f821a7fc640 1 -- 172.21.15.40:0/2463170915 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8220059e50 con 0x7f82240b2850 2024-04-26T21:43:42.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:43:42.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:43:42.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.384+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82240badf0 con 0x7f82240b2850 2024-04-26T21:43:42.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.384+0000 7f821a7fc640 1 -- 172.21.15.40:0/2463170915 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) ==== 95+0+5181 (secure 0 0 0) 0x7f8220059750 con 0x7f82240b2850 2024-04-26T21:43:42.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:42.386 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9183,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2899400637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2899400637},{"type":"v1","addr":"172.21.15.40:6833","nonce":2899400637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:38.977099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:42.386 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 156 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f820806c8f0 msgr2=0x7f820806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f820806c8f0 0x7f820806edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8214006fd0 tx=0x7f8214008040 comp rx=0 tx=0).stop 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 msgr2=0x7f822415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f822415fa80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f822000c430 tx=0x7f822000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 shutdown_connections 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f820806c8f0 0x7f820806edb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f822415f3a0 0x7f82240b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82240d06f0 0x7f82241585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 --2- 172.21.15.40:0/2463170915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82240b2850 0x7f822415fa80 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 >> 172.21.15.40:0/2463170915 conn(0x7f82240d4f90 msgr2=0x7f82240b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:42.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 shutdown_connections 2024-04-26T21:43:42.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.387+0000 7f822be93640 1 -- 172.21.15.40:0/2463170915 wait complete. 2024-04-26T21:43:42.399 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000' led to DAMAGED state 2024-04-26T21:43:42.399 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_sessionmap' 2024-04-26T21:43:42.400 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:42.400 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:42.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:43:42.416 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:43:42.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:42.416 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:43:42.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:43:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:43:42.472 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:43:42.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.612+0000 7fcd75d40640 1 Processor -- start 2024-04-26T21:43:42.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.612+0000 7fcd75d40640 1 -- start start 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.613+0000 7fcd75d40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd70071e60 0x7fcd7006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.613+0000 7fcd75d40640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcd700cc480 0x7fcd70166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.613+0000 7fcd75d40640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7015d7c0 con 0x7fcd7006fe20 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.613+0000 7fcd75d40640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7015daa0 con 0x7fcd70071e60 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.613+0000 7fcd75d40640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7015dd80 con 0x7fcd700cc480 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.613+0000 7fcd6f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd70071e60 0x7fcd7006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.613+0000 7fcd6f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd70071e60 0x7fcd7006f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55750/0 (socket says 172.21.15.40:55750) 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.613+0000 7fcd6f7fe640 1 -- 172.21.15.40:0/2351140670 learned_addr learned my addr 172.21.15.40:0/2351140670 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.614+0000 7fcd6f7fe640 1 -- 172.21.15.40:0/2351140670 >> v2:172.21.15.184:3300/0 conn(0x7fcd700cc480 msgr2=0x7fcd70166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.614+0000 7fcd6f7fe640 1 --2- 172.21.15.40:0/2351140670 >> v2:172.21.15.184:3300/0 conn(0x7fcd700cc480 0x7fcd70166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.614+0000 7fcd6f7fe640 1 -- 172.21.15.40:0/2351140670 >> v1:172.21.15.184:6789/0 conn(0x7fcd7006fe20 legacy=0x7fcd70072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.614+0000 7fcd6f7fe640 1 -- 172.21.15.40:0/2351140670 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7015e060 con 0x7fcd70071e60 2024-04-26T21:43:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.614+0000 7fcd6f7fe640 1 --2- 172.21.15.40:0/2351140670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd70071e60 0x7fcd7006f8e0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fcd700b06b0 tx=0x7fcd6002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb2039c4d50e019 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.614+0000 7fcd6effd640 1 -- 172.21.15.40:0/2351140670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60037070 con 0x7fcd70071e60 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.614+0000 7fcd6effd640 1 -- 172.21.15.40:0/2351140670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd60032440 con 0x7fcd70071e60 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.614+0000 7fcd6effd640 1 -- 172.21.15.40:0/2351140670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60032810 con 0x7fcd70071e60 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.615+0000 7fcd75d40640 1 -- 172.21.15.40:0/2351140670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd70071e60 msgr2=0x7fcd7006f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.615+0000 7fcd75d40640 1 --2- 172.21.15.40:0/2351140670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd70071e60 0x7fcd7006f8e0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fcd700b06b0 tx=0x7fcd6002a230 comp rx=0 tx=0).stop 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.615+0000 7fcd75d40640 1 -- 172.21.15.40:0/2351140670 shutdown_connections 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.615+0000 7fcd75d40640 1 --2- 172.21.15.40:0/2351140670 >> v2:172.21.15.184:3300/0 conn(0x7fcd700cc480 0x7fcd70166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.615+0000 7fcd75d40640 1 --2- 172.21.15.40:0/2351140670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd70071e60 0x7fcd7006f8e0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.615+0000 7fcd75d40640 1 -- 172.21.15.40:0/2351140670 >> 172.21.15.40:0/2351140670 conn(0x7fcd700d0be0 msgr2=0x7fcd700d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.615+0000 7fcd75d40640 1 -- 172.21.15.40:0/2351140670 shutdown_connections 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.615+0000 7fcd75d40640 1 -- 172.21.15.40:0/2351140670 wait complete. 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.616+0000 7fcd75d40640 1 Processor -- start 2024-04-26T21:43:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.616+0000 7fcd75d40640 1 -- start start 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd75d40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd7006fe20 0x7fcd700c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd75d40640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70071e60 0x7fcd700b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd75d40640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd700cc480 0x7fcd700c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd75d40640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7015d570 con 0x7fcd7006fe20 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd75d40640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7015d3f0 con 0x7fcd70071e60 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd75d40640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd70072950 con 0x7fcd700cc480 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd7006fe20 0x7fcd700c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd7006fe20 0x7fcd700c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55752/0 (socket says 172.21.15.40:55752) 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd6f7fe640 1 -- 172.21.15.40:0/4086063927 learned_addr learned my addr 172.21.15.40:0/4086063927 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd6f7fe640 1 -- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd700cc480 msgr2=0x7fcd700c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd6f7fe640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd700cc480 0x7fcd700c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd6f7fe640 1 -- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70071e60 msgr2=0x7fcd700b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd6f7fe640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70071e60 0x7fcd700b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.617+0000 7fcd6f7fe640 1 -- 172.21.15.40:0/4086063927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd700c9900 con 0x7fcd7006fe20 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.618+0000 7fcd6f7fe640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd7006fe20 0x7fcd700c7cb0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fcd6002a270 tx=0x7fcd600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.618+0000 7fcd6cff9640 1 -- 172.21.15.40:0/4086063927 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60041070 con 0x7fcd7006fe20 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.618+0000 7fcd6cff9640 1 -- 172.21.15.40:0/4086063927 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6003c870 con 0x7fcd7006fe20 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.618+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd700c8320 con 0x7fcd7006fe20 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.618+0000 7fcd6cff9640 1 -- 172.21.15.40:0/4086063927 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60037040 con 0x7fcd7006fe20 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.618+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd700c88e0 con 0x7fcd7006fe20 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.619+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd700694f0 con 0x7fcd7006fe20 2024-04-26T21:43:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.619+0000 7fcd6cff9640 1 -- 172.21.15.40:0/4086063927 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd60044070 con 0x7fcd7006fe20 2024-04-26T21:43:42.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.620+0000 7fcd6cff9640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd5006c8f0 0x7fcd5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.620+0000 7fcd6cff9640 1 -- 172.21.15.40:0/4086063927 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7163+0+0 (secure 0 0 0) 0x7fcd600acc00 con 0x7fcd7006fe20 2024-04-26T21:43:42.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.620+0000 7fcd66ffd640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd5006c8f0 0x7fcd5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.620+0000 7fcd66ffd640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd5006c8f0 0x7fcd5006edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcd70158c10 tx=0x7fcd58007990 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.624+0000 7fcd6cff9640 1 -- 172.21.15.40:0/4086063927 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd600b3080 con 0x7fcd7006fe20 2024-04-26T21:43:42.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.846+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcd70069bc0 con 0x7fcd7006fe20 2024-04-26T21:43:42.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.995+0000 7fcd6cff9640 1 -- 172.21.15.40:0/4086063927 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v157) ==== 161+0+0 (secure 0 0 0) 0x7fcd6007ab10 con 0x7fcd7006fe20 2024-04-26T21:43:42.996 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd5006c8f0 msgr2=0x7fcd5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd5006c8f0 0x7fcd5006edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcd70158c10 tx=0x7fcd58007990 comp rx=0 tx=0).stop 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd7006fe20 msgr2=0x7fcd700c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd7006fe20 0x7fcd700c7cb0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fcd6002a270 tx=0x7fcd600046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 shutdown_connections 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcd5006c8f0 0x7fcd5006edb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd700cc480 0x7fcd700c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd70071e60 0x7fcd700b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 --2- 172.21.15.40:0/4086063927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd7006fe20 0x7fcd700c7cb0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 >> 172.21.15.40:0/4086063927 conn(0x7fcd700d0be0 msgr2=0x7fcd7015ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 shutdown_connections 2024-04-26T21:43:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:42.998+0000 7fcd75d40640 1 -- 172.21.15.40:0/4086063927 wait complete. 2024-04-26T21:43:43.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:43:43.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.111+0000 7fe720d9e640 1 Processor -- start 2024-04-26T21:43:43.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.111+0000 7fe720d9e640 1 -- start start 2024-04-26T21:43:43.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe720d9e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe71c06e270 0x7fe71c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe720d9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe71c06d560 0x7fe71c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe720d9e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71c0d2420 con 0x7fe71c0d0070 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe720d9e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe71c162d00 con 0x7fe71c06d560 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe720d9e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe71c0d2d10 con 0x7fe71c06e270 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe71ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe71c06d560 0x7fe71c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe71a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe71c0d0070 0x7fe71c06db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37204/0 (socket says 172.21.15.40:37204) 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe71a59c640 1 -- 172.21.15.40:0/2621275951 learned_addr learned my addr 172.21.15.40:0/2621275951 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.112+0000 7fe711d9b640 1 --2- 172.21.15.40:0/2621275951 >> v2:172.21.15.184:3300/0 conn(0x7fe71c06e270 0x7fe71c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe71ad9d640 1 -- 172.21.15.40:0/2621275951 >> v2:172.21.15.184:3300/0 conn(0x7fe71c06e270 msgr2=0x7fe71c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe71ad9d640 1 --2- 172.21.15.40:0/2621275951 >> v2:172.21.15.184:3300/0 conn(0x7fe71c06e270 0x7fe71c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe71ad9d640 1 -- 172.21.15.40:0/2621275951 >> v1:172.21.15.40:6789/0 conn(0x7fe71c0d0070 legacy=0x7fe71c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe71ad9d640 1 -- 172.21.15.40:0/2621275951 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c0cd6b0 con 0x7fe71c06d560 2024-04-26T21:43:43.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe719d9b640 1 -- 172.21.15.40:0/2621275951 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 445778935 0 0) 0x7fe71c0d2420 con 0x7fe71c0d0070 2024-04-26T21:43:43.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe711d9b640 1 --2- 172.21.15.40:0/2621275951 >> v2:172.21.15.184:3300/0 conn(0x7fe71c06e270 0x7fe71c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe71ad9d640 1 --2- 172.21.15.40:0/2621275951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c0ce190 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe71c073120 tx=0x7fe71402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cfe1717c780875 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe719d9b640 1 -- 172.21.15.40:0/2621275951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714007740 con 0x7fe71c06d560 2024-04-26T21:43:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe719d9b640 1 -- 172.21.15.40:0/2621275951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7140078e0 con 0x7fe71c06d560 2024-04-26T21:43:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.113+0000 7fe719d9b640 1 -- 172.21.15.40:0/2621275951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714007cb0 con 0x7fe71c06d560 2024-04-26T21:43:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.114+0000 7fe720d9e640 1 -- 172.21.15.40:0/2621275951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 msgr2=0x7fe71c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.114+0000 7fe720d9e640 1 --2- 172.21.15.40:0/2621275951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c0ce190 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe71c073120 tx=0x7fe71402a680 comp rx=0 tx=0).stop 2024-04-26T21:43:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.114+0000 7fe720d9e640 1 -- 172.21.15.40:0/2621275951 shutdown_connections 2024-04-26T21:43:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.114+0000 7fe720d9e640 1 --2- 172.21.15.40:0/2621275951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c0ce190 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.114+0000 7fe720d9e640 1 --2- 172.21.15.40:0/2621275951 >> v2:172.21.15.184:3300/0 conn(0x7fe71c06e270 0x7fe71c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.114+0000 7fe720d9e640 1 -- 172.21.15.40:0/2621275951 >> 172.21.15.40:0/2621275951 conn(0x7fe71c0d2ec0 msgr2=0x7fe71c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.114+0000 7fe720d9e640 1 -- 172.21.15.40:0/2621275951 shutdown_connections 2024-04-26T21:43:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.114+0000 7fe720d9e640 1 -- 172.21.15.40:0/2621275951 wait complete. 2024-04-26T21:43:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe720d9e640 1 Processor -- start 2024-04-26T21:43:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe720d9e640 1 -- start start 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe720d9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c09db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe720d9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c06e270 0x7fe71c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe720d9e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0d0070 0x7fe71c0a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe720d9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c0ce6d0 con 0x7fe71c06d560 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe720d9e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c0cdc70 con 0x7fe71c0d0070 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe720d9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c0ce850 con 0x7fe71c06e270 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c09db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.116+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c09db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55766/0 (socket says 172.21.15.40:55766) 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe71a59c640 1 -- 172.21.15.40:0/3677830384 learned_addr learned my addr 172.21.15.40:0/3677830384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe711d9b640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c06e270 0x7fe71c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe71ad9d640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0d0070 0x7fe71c0a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe71a59c640 1 -- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c06e270 msgr2=0x7fe71c0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe71a59c640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c06e270 0x7fe71c0c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe71a59c640 1 -- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0d0070 msgr2=0x7fe71c0a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe71a59c640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0d0070 0x7fe71c0a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe71a59c640 1 -- 172.21.15.40:0/3677830384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c09cdd0 con 0x7fe71c06d560 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe71a59c640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c09db90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00c460 tx=0x7fe70c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe7137fe640 1 -- 172.21.15.40:0/3677830384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c018070 con 0x7fe71c06d560 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe7137fe640 1 -- 172.21.15.40:0/3677830384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70c007500 con 0x7fe71c06d560 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c09d0b0 con 0x7fe71c06d560 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe7137fe640 1 -- 172.21.15.40:0/3677830384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c004a20 con 0x7fe71c06d560 2024-04-26T21:43:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.117+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c09d670 con 0x7fe71c06d560 2024-04-26T21:43:43.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.118+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c0694f0 con 0x7fe71c06d560 2024-04-26T21:43:43.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.119+0000 7fe7137fe640 1 -- 172.21.15.40:0/3677830384 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe70c01b070 con 0x7fe71c06d560 2024-04-26T21:43:43.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.119+0000 7fe7137fe640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:43.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.119+0000 7fe7137fe640 1 -- 172.21.15.40:0/3677830384 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7163+0+0 (secure 0 0 0) 0x7fe70c08b250 con 0x7fe71c06d560 2024-04-26T21:43:43.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.120+0000 7fe711d9b640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:43.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.120+0000 7fe711d9b640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe71c162060 tx=0x7fe704008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.124+0000 7fe7137fe640 1 -- 172.21.15.40:0/3677830384 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe70c058b10 con 0x7fe71c06d560 2024-04-26T21:43:43.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.382+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe71c161ad0 con 0x7fe71c06d560 2024-04-26T21:43:43.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.997+0000 7fe7137fe640 1 -- 172.21.15.40:0/3677830384 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v158) ==== 101+0+0 (secure 0 0 0) 0x7fe70c05ce10 con 0x7fe71c06d560 2024-04-26T21:43:43.998 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:43:44.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6fc06c8f0 msgr2=0x7fe6fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe71c162060 tx=0x7fe704008040 comp rx=0 tx=0).stop 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 msgr2=0x7fe71c09db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c09db90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00c460 tx=0x7fe70c00c920 comp rx=0 tx=0).stop 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 shutdown_connections 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0d0070 0x7fe71c0a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c06e270 0x7fe71c0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 --2- 172.21.15.40:0/3677830384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c06d560 0x7fe71c09db90 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:43.999+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 >> 172.21.15.40:0/3677830384 conn(0x7fe71c0d2ec0 msgr2=0x7fe71c071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.000+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 shutdown_connections 2024-04-26T21:43:44.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.000+0000 7fe720d9e640 1 -- 172.21.15.40:0/3677830384 wait complete. 2024-04-26T21:43:44.013 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:43:44.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.057+0000 7f414871cf40 1 Processor -- start 2024-04-26T21:43:44.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.057+0000 7f414871cf40 1 -- start start 2024-04-26T21:43:44.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f414871cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 0x5608a0400d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f414871cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 0x5608a032a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f414871cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f414871cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5608a0400b20 con 0x5608a03e4d60 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f414871cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5608a03fa080 con 0x5608a032d510 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f414871cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5608a0358620 con 0x5608a032d130 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55790/0 (socket says 172.21.15.40:55790) 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 -- 172.21.15.40:0/260975914 learned_addr learned my addr 172.21.15.40:0/260975914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4144fb1640 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 0x5608a032a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f41457b2640 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 0x5608a0400d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 -- 172.21.15.40:0/260975914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 msgr2=0x5608a0400d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 0x5608a0400d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 -- 172.21.15.40:0/260975914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 msgr2=0x5608a032a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 0x5608a032a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 -- 172.21.15.40:0/260975914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608a0255cb0 con 0x5608a03e4d60 2024-04-26T21:43:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.058+0000 7f4145fb3640 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03728c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f4138005fd0 tx=0x7f4138050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1648c67e2e9c7494 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f413f7fe640 1 -- 172.21.15.40:0/260975914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138055030 con 0x5608a03e4d60 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f413f7fe640 1 -- 172.21.15.40:0/260975914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4138051ba0 con 0x5608a03e4d60 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f413f7fe640 1 -- 172.21.15.40:0/260975914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413805b430 con 0x5608a03e4d60 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 -- 172.21.15.40:0/260975914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 msgr2=0x5608a03728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03728c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f4138005fd0 tx=0x7f4138050000 comp rx=0 tx=0).stop 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 -- 172.21.15.40:0/260975914 shutdown_connections 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03728c0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 0x5608a032a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 --2- 172.21.15.40:0/260975914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 0x5608a0400d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 -- 172.21.15.40:0/260975914 >> 172.21.15.40:0/260975914 conn(0x5608a0329ab0 msgr2=0x5608a03f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 -- 172.21.15.40:0/260975914 shutdown_connections 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.059+0000 7f414871cf40 1 -- 172.21.15.40:0/260975914 wait complete. 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.060+0000 7f414871cf40 1 Processor -- start 2024-04-26T21:43:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.060+0000 7f414871cf40 1 -- start start 2024-04-26T21:43:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.060+0000 7f414871cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 0x5608a03fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.060+0000 7f414871cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 0x5608a03f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.060+0000 7f414871cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.060+0000 7f414871cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5608a04029d0 con 0x5608a03e4d60 2024-04-26T21:43:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.060+0000 7f414871cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5608a03e1c50 con 0x5608a032d510 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.060+0000 7f414871cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5608a0371dd0 con 0x5608a032d130 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03e3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55798/0 (socket says 172.21.15.40:55798) 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 learned_addr learned my addr 172.21.15.40:0/2205594695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f41457b2640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 0x5608a03fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4144fb1640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 0x5608a03f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 msgr2=0x5608a03fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5608a032d130 0x5608a03fe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 msgr2=0x5608a03f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5608a032d510 0x5608a03f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608a03fb250 con 0x5608a03e4d60 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f4145fb3640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03e3e40 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f41380058f0 tx=0x7f4138003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f413e7fc640 1 -- 172.21.15.40:0/2205594695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41380519a0 con 0x5608a03e4d60 2024-04-26T21:43:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f413e7fc640 1 -- 172.21.15.40:0/2205594695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4138051d50 con 0x5608a03e4d60 2024-04-26T21:43:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608a03fee50 con 0x5608a03e4d60 2024-04-26T21:43:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f413e7fc640 1 -- 172.21.15.40:0/2205594695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138067a70 con 0x5608a03e4d60 2024-04-26T21:43:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.061+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608a03fb7f0 con 0x5608a03e4d60 2024-04-26T21:43:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.063+0000 7f413e7fc640 1 -- 172.21.15.40:0/2205594695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f413806f030 con 0x5608a03e4d60 2024-04-26T21:43:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.063+0000 7f413e7fc640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f412806b2d0 0x7f412806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.063+0000 7f413e7fc640 1 -- 172.21.15.40:0/2205594695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7163+0+0 (secure 0 0 0) 0x7f41380cf340 con 0x5608a03e4d60 2024-04-26T21:43:44.067 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.064+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5608a03fdab0 0x5608a03fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.064+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a03fd000 con 0x5608a03fdab0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.064+0000 7f41457b2640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f412806b2d0 0x7f412806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.064+0000 7f4144fb1640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5608a03fdab0 0x5608a03fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.064+0000 7f4144fb1640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5608a03fdab0 0x5608a03fcac0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.064+0000 7f41457b2640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f412806b2d0 0x7f412806d790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4140009660 tx=0x7f414000e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.065+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v106'319 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.065+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a3340 con 0x5608a03fdab0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.066+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v106'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.066+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a36e0 con 0x5608a03fdab0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.067+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v106'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.067+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a4a90 con 0x5608a03fdab0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.068+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v106'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.068+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a4e30 con 0x5608a03fdab0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.069+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v106'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.070+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a3a80 con 0x5608a03fdab0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.071+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v106'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.071+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a3e20 con 0x5608a03fdab0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.072+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v106'325 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.072+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.073+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v106'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.073+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.074 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:43:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.074+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v106'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.074+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.074+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v106'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.074+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.075+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v106'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.075+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.076+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v106'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.076+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.077+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v106'331 uv306 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.077+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.078+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v106'332 uv332 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.078+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.080+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v106'333 uv333 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.081 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:43:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.080+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.081+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v106'334 uv334 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.081+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.082+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v106'335 uv335 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.082+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.083+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v106'336 uv336 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.083+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.084+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v106'337 uv337 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.084+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.085+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v106'338 uv312 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.085+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.086+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v106'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.086+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.087+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v106'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.088 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:43:44.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.087+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.088+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v106'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.088+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.089+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v106'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.089+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.090+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v106'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.090+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.091+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v106'344 uv318 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.091+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.092+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v106'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.092+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.093+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v106'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.093 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:43:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.093+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.094+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v106'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.094+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.095+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v106'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.095+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a03fdab0 2024-04-26T21:43:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.096+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v106'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a03fdab0 2024-04-26T21:43:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.096+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5608a04a51d0 0x5608a04a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.096+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a41c0 con 0x5608a04a51d0 2024-04-26T21:43:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.096+0000 7f41457b2640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5608a04a51d0 0x5608a04a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.096+0000 7f41457b2640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5608a04a51d0 0x5608a04a75c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.098+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v106'241 uv223 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.098+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a7b00 con 0x5608a04a51d0 2024-04-26T21:43:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.099+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v106'242 uv242 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.099+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a7ea0 con 0x5608a04a51d0 2024-04-26T21:43:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.100+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v106'243 uv243 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.101 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:43:44.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.100+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a86f0 con 0x5608a04a51d0 2024-04-26T21:43:44.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.102+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v106'244 uv244 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.102+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a8a90 con 0x5608a04a51d0 2024-04-26T21:43:44.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.103+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v106'245 uv245 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.103+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a8e30 con 0x5608a04a51d0 2024-04-26T21:43:44.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.104+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v106'246 uv228 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.104+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a91d0 con 0x5608a04a51d0 2024-04-26T21:43:44.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.106+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v106'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.106+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.107+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v106'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.108 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:43:44.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.107+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.108+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v106'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.108+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.109+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v106'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.109+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.110+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v106'251 uv233 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.110+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.112+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v106'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.112+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.113+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v106'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.113 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:43:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.113+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04adc30 con 0x5608a04a51d0 2024-04-26T21:43:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.114+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v106'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.114+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04adc30 con 0x5608a04a51d0 2024-04-26T21:43:44.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.115+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v106'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.115+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.116+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v106'256 uv239 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.116+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.117+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v106'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.117+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.118+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v106'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.119 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:43:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.118+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04aa580 con 0x5608a04a51d0 2024-04-26T21:43:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.120+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v106'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.120+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04aa580 con 0x5608a04a51d0 2024-04-26T21:43:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.121+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v106'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.121+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04a9570 con 0x5608a04a51d0 2024-04-26T21:43:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.122+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v106'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04a51d0 2024-04-26T21:43:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.122+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5608a04a9a80 0x5608a04abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.122+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04ac3b0 con 0x5608a04a9a80 2024-04-26T21:43:44.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.122+0000 7f4144fb1640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5608a04a9a80 0x5608a04abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.122+0000 7f4144fb1640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5608a04a9a80 0x5608a04abe70 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.123+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v106'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.123+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04ac3b0 con 0x5608a04a9a80 2024-04-26T21:43:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.125+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v106'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.125+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04ac430 con 0x5608a04a9a80 2024-04-26T21:43:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.126+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v106'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.126 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:43:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.126+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04ad830 con 0x5608a04a9a80 2024-04-26T21:43:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.127+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v106'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.127+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04ad830 con 0x5608a04a9a80 2024-04-26T21:43:44.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.128+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v106'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.128+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04ac820 con 0x5608a04a9a80 2024-04-26T21:43:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.129+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v106'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.129+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5608a04acd30 0x5608a04af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.129+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04af780 con 0x5608a04acd30 2024-04-26T21:43:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.129+0000 7f4145fb3640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5608a04acd30 0x5608a04af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.130+0000 7f4145fb3640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5608a04acd30 0x5608a04af120 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.131+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v106'53 uv52 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.131+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04afc90 con 0x5608a04acd30 2024-04-26T21:43:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.132+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v106'54 uv54 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.132+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b0190 con 0x5608a04acd30 2024-04-26T21:43:44.134 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:43:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.134+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v106'55 uv55 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.134+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b0a50 con 0x5608a04acd30 2024-04-26T21:43:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.139+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v106'56 uv56 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.139+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5608a04b1300 0x5608a04b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.139+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b3c30 con 0x5608a04b1300 2024-04-26T21:43:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.139+0000 7f41457b2640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5608a04b1300 0x5608a04b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.140+0000 7f41457b2640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5608a04b1300 0x5608a04b36f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.141+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v106'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.141+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b3cf0 con 0x5608a04b1300 2024-04-26T21:43:44.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.142+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v106'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.142+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4110 con 0x5608a04b1300 2024-04-26T21:43:44.144 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:43:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.144+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v106'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.144+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b5120 con 0x5608a04b1300 2024-04-26T21:43:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.145+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v106'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.145+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b5120 con 0x5608a04b1300 2024-04-26T21:43:44.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.146+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v106'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.146+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4110 con 0x5608a04b1300 2024-04-26T21:43:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.147+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v106'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.147+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4110 con 0x5608a04acd30 2024-04-26T21:43:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.149+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v106'116 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.149+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b44b0 con 0x5608a04acd30 2024-04-26T21:43:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.150+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v106'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.150+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4850 con 0x5608a04acd30 2024-04-26T21:43:44.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.151+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v106'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.152 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:43:44.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.151+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b5c00 con 0x5608a04acd30 2024-04-26T21:43:44.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.152+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v106'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.152+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b5c00 con 0x5608a04acd30 2024-04-26T21:43:44.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.153+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v106'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.153+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.154+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v106'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.154+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.155+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v106'122 uv115 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.155+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.156+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v106'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.156+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.157+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v106'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.158 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:43:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.157+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.158+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v106'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.158+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.159+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v106'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.159+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.160+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v106'51 uv50 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.160+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.161+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v106'52 uv52 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.161+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.162+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v106'53 uv53 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.163 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:43:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.162+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.163+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v106'54 uv54 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.163+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04acd30 2024-04-26T21:43:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.164+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v106'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f41380cf0b0 con 0x5608a04acd30 2024-04-26T21:43:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.164+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04b1300 2024-04-26T21:43:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.165+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v106'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.165+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04b1300 2024-04-26T21:43:44.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.166+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v106'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.166+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04b1300 2024-04-26T21:43:44.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.167+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v106'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.168 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:43:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.167+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4fb0 con 0x5608a04b1300 2024-04-26T21:43:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.168+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v106'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.168+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4fb0 con 0x5608a04b1300 2024-04-26T21:43:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.169+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v106'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.169+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4fb0 con 0x5608a04b1300 2024-04-26T21:43:44.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.170+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v106'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.170+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4fb0 con 0x5608a04b1300 2024-04-26T21:43:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.171+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v106'104 uv97 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.171+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b4fb0 con 0x5608a04b1300 2024-04-26T21:43:44.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.172+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v106'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.172+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4fb0 con 0x5608a04b1300 2024-04-26T21:43:44.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.173+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v106'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.173 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:43:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.173+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04b1300 2024-04-26T21:43:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.174+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v106'107 uv107 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.174+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04b1300 2024-04-26T21:43:44.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.175+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v106'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.175+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04b1300 2024-04-26T21:43:44.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.176+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v106'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.176 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:43:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.176+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b4bf0 con 0x5608a04b1300 2024-04-26T21:43:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.177+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [omap-set-header] v106'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.177+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b53b0 con 0x5608a04b1300 2024-04-26T21:43:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.178+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v106'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.178+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5608a04b5370 0x5608a04b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.178+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04b5370 2024-04-26T21:43:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.178+0000 7f4144fb1640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5608a04b5370 0x5608a04b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.178+0000 7f4144fb1640 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5608a04b5370 0x5608a04b7fd0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.180+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(99 mds0_sessionmap [create] v106'49 uv47 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04b5370 2024-04-26T21:43:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.180+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04b5370 2024-04-26T21:43:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.181+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v106'50 uv50 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04b5370 2024-04-26T21:43:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.181+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04b5370 2024-04-26T21:43:44.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.182+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(101 mds0_sessionmap [create] v106'51 uv51 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04b5370 2024-04-26T21:43:44.183 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:43:44.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.182+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04b5370 2024-04-26T21:43:44.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.183+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v106'52 uv52 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04b5370 2024-04-26T21:43:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.183+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04b1300 2024-04-26T21:43:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.184+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(103 609.00000000 [create] v106'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.184+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04b1300 2024-04-26T21:43:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.185+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(104 609.00000000 [delete] v106'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.185+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04b1300 2024-04-26T21:43:44.187 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:43:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.186+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v106'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.186+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b9520 con 0x5608a04b1300 2024-04-26T21:43:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.187+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v106'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.187+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b9520 con 0x5608a04b1300 2024-04-26T21:43:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.188+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v106'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.188+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04b1300 2024-04-26T21:43:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.189+0000 7f41457b2640 1 -- 172.21.15.40:0/2205594695 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [omap-set-header] v106'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4140018140 con 0x5608a04b1300 2024-04-26T21:43:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.189+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.192+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v106'52 uv51 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.192+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.193+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v106'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.193+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.195 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:43:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.195+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v106'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.195+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.196+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v106'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.196+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.197+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v106'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.197+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.198+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v106'132 uv123 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.199+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.200+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v106'133 uv133 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.200+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.201 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:43:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.201+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v106'134 uv134 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.201+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.202+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v106'135 uv135 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.202+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.203+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v106'136 uv136 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.203+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.204+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v106'137 uv130 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.204+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.205+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v106'138 uv138 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.205+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.206+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v106'139 uv139 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.207 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:43:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.206+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b9520 con 0x5608a04a9a80 2024-04-26T21:43:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.207+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v106'140 uv140 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.207+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b9520 con 0x5608a04a9a80 2024-04-26T21:43:44.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.208+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v106'141 uv141 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.208+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b8510 con 0x5608a04a9a80 2024-04-26T21:43:44.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.209+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v106'142 uv142 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.209+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5608a04b89c0 con 0x5608a04a9a80 2024-04-26T21:43:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f4144fb1640 1 -- 172.21.15.40:0/2205594695 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v106'143 uv143 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f41300098b0 con 0x5608a04a9a80 2024-04-26T21:43:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5608a04a51d0 msgr2=0x5608a04a75c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5608a04a51d0 0x5608a04a75c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5608a03fdab0 msgr2=0x5608a03fcac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5608a03fdab0 0x5608a03fcac0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5608a04acd30 msgr2=0x5608a04af120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5608a04acd30 0x5608a04af120 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5608a04b5370 msgr2=0x5608a04b7fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5608a04b5370 0x5608a04b7fd0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5608a04a9a80 msgr2=0x5608a04abe70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5608a04a9a80 0x5608a04abe70 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 reap_dead start 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5608a04b1300 msgr2=0x5608a04b36f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5608a04b1300 0x5608a04b36f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 reap_dead start 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.210+0000 7f4145fb3640 1 -- 172.21.15.40:0/2205594695 reap_dead start 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f412806b2d0 msgr2=0x7f412806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f412806b2d0 0x7f412806d790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4140009660 tx=0x7f414000e040 comp rx=0 tx=0).stop 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 msgr2=0x5608a03e3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03e3e40 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f41380058f0 tx=0x7f4138003330 comp rx=0 tx=0).stop 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 shutdown_connections 2024-04-26T21:43:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f412806b2d0 0x7f412806d790 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 --2- 172.21.15.40:0/2205594695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5608a03e4d60 0x5608a03e3e40 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 >> 172.21.15.40:0/2205594695 conn(0x5608a0329ab0 msgr2=0x5608a032aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 shutdown_connections 2024-04-26T21:43:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.211+0000 7f414871cf40 1 -- 172.21.15.40:0/2205594695 wait complete. 2024-04-26T21:43:44.215 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_sessionmap 2024-04-26T21:43:44.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.257+0000 7f3dff51df40 1 Processor -- start 2024-04-26T21:43:44.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.257+0000 7f3dff51df40 1 -- start start 2024-04-26T21:43:44.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.257+0000 7f3dff51df40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 0x55fd47469430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.257+0000 7f3dff51df40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd47481080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.257+0000 7f3dff51df40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 0x55fd47482d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.257+0000 7f3dff51df40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fd47353e40 con 0x55fd474804e0 2024-04-26T21:43:44.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dff51df40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fd47357e80 con 0x55fd474808c0 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dff51df40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55fd473dc690 con 0x55fd473f4140 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd47481080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd47481080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55814/0 (socket says 172.21.15.40:55814) 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 -- 172.21.15.40:0/3912364120 learned_addr learned my addr 172.21.15.40:0/3912364120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfc5d2640 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 0x55fd47469430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfcdd3640 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 0x55fd47482d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 -- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 msgr2=0x55fd47469430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 0x55fd47469430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 -- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 msgr2=0x55fd47482d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 0x55fd47482d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 -- 172.21.15.40:0/3912364120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd474828c0 con 0x55fd474804e0 2024-04-26T21:43:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfcdd3640 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 0x55fd47482d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfbdd1640 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd47481080 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f3df0058040 tx=0x7f3df004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6437a71df29c57b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfadcf640 1 -- 172.21.15.40:0/3912364120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0064070 con 0x55fd474804e0 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfadcf640 1 -- 172.21.15.40:0/3912364120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df005b040 con 0x55fd474804e0 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.258+0000 7f3dfadcf640 1 -- 172.21.15.40:0/3912364120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df005a070 con 0x55fd474804e0 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 -- 172.21.15.40:0/3912364120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 msgr2=0x55fd47481080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd47481080 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f3df0058040 tx=0x7f3df004f000 comp rx=0 tx=0).stop 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 -- 172.21.15.40:0/3912364120 shutdown_connections 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 0x55fd47482d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd47481080 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 --2- 172.21.15.40:0/3912364120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 0x55fd47469430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 -- 172.21.15.40:0/3912364120 >> 172.21.15.40:0/3912364120 conn(0x55fd473d68a0 msgr2=0x55fd473f2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 -- 172.21.15.40:0/3912364120 shutdown_connections 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 -- 172.21.15.40:0/3912364120 wait complete. 2024-04-26T21:43:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 Processor -- start 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.259+0000 7f3dff51df40 1 -- start start 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dff51df40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 0x55fd47484950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dff51df40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd4747c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dff51df40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 0x55fd47480ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dff51df40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fd473d7af0 con 0x55fd474804e0 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dff51df40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fd47482670 con 0x55fd474808c0 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dff51df40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55fd473f0150 con 0x55fd473f4140 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dfbdd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd4747c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dfbdd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd4747c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55824/0 (socket says 172.21.15.40:55824) 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dfbdd1640 1 -- 172.21.15.40:0/2941244667 learned_addr learned my addr 172.21.15.40:0/2941244667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:44.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dfc5d2640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 0x55fd47484950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.260+0000 7f3dfcdd3640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 0x55fd47480ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3dfbdd1640 1 -- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 msgr2=0x55fd47484950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3dfbdd1640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fd473f4140 0x55fd47484950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3dfbdd1640 1 -- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 msgr2=0x55fd47480ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3dfbdd1640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd474808c0 0x55fd47480ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3dfbdd1640 1 -- 172.21.15.40:0/2941244667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd474838b0 con 0x55fd474804e0 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3dfbdd1640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd4747c7f0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3df0026010 tx=0x7f3df0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3df9dcd640 1 -- 172.21.15.40:0/2941244667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df005a070 con 0x55fd474804e0 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3df9dcd640 1 -- 172.21.15.40:0/2941244667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df0055640 con 0x55fd474804e0 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fd47483b90 con 0x55fd474804e0 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3df9dcd640 1 -- 172.21.15.40:0/2941244667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0064040 con 0x55fd474804e0 2024-04-26T21:43:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.261+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd47484150 con 0x55fd474804e0 2024-04-26T21:43:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.262+0000 7f3df9dcd640 1 -- 172.21.15.40:0/2941244667 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3df0055a10 con 0x55fd474804e0 2024-04-26T21:43:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.263+0000 7f3df9dcd640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3de806b2d0 0x7f3de806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.263+0000 7f3df9dcd640 1 -- 172.21.15.40:0/2941244667 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7163+0+0 (secure 0 0 0) 0x7f3df00cfbb0 con 0x55fd474804e0 2024-04-26T21:43:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.263+0000 7f3dff51df40 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fd4747d070 0x55fd47466b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.263+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55fd47467080 con 0x55fd4747d070 2024-04-26T21:43:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.263+0000 7f3dfc5d2640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3de806b2d0 0x7f3de806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.263+0000 7f3dfcdd3640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fd4747d070 0x55fd47466b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.264+0000 7f3dfc5d2640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3de806b2d0 0x7f3de806d790 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x55fd4747f020 tx=0x7f3dec0017b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.264+0000 7f3dfcdd3640 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fd4747d070 0x55fd47466b40 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dfcdd3640 1 -- 172.21.15.40:0/2941244667 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(1 mds0_sessionmap [delete] v106'53 uv53 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3df40018e0 con 0x55fd4747d070 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fd4747d070 msgr2=0x55fd47466b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fd4747d070 0x55fd47466b40 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3de806b2d0 msgr2=0x7f3de806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3de806b2d0 0x7f3de806d790 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x55fd4747f020 tx=0x7f3dec0017b0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 msgr2=0x55fd4747c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 --2- 172.21.15.40:0/2941244667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd474804e0 0x55fd4747c7f0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3df0026010 tx=0x7f3df0002fb0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dfcdd3640 1 -- 172.21.15.40:0/2941244667 reap_dead start 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 shutdown_connections 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 >> 172.21.15.40:0/2941244667 conn(0x55fd473d68a0 msgr2=0x55fd473f2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 shutdown_connections 2024-04-26T21:43:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.265+0000 7f3dff51df40 1 -- 172.21.15.40:0/2941244667 wait complete. 2024-04-26T21:43:44.271 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:43:44.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.413+0000 7fa2cfe09640 1 Processor -- start 2024-04-26T21:43:44.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.413+0000 7fa2cfe09640 1 -- start start 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.414+0000 7fa2cfe09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2c80d1a00 0x7fa2c80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.414+0000 7fa2cfe09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80d2320 0x7fa2c81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.414+0000 7fa2cfe09640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c80d0a10 con 0x7fa2c80b40d0 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.414+0000 7fa2cfe09640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c80d0cf0 con 0x7fa2c80d1a00 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.414+0000 7fa2cfe09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c80d0fd0 con 0x7fa2c80d2320 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.414+0000 7fa2cdba5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2c80d1a00 0x7fa2c80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.415+0000 7fa2cdba5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2c80d1a00 0x7fa2c80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55834/0 (socket says 172.21.15.40:55834) 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.415+0000 7fa2cdba5640 1 -- 172.21.15.40:0/449643330 learned_addr learned my addr 172.21.15.40:0/449643330 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.415+0000 7fa2cd3a4640 1 --2- 172.21.15.40:0/449643330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80d2320 0x7fa2c81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.415+0000 7fa2cdba5640 1 -- 172.21.15.40:0/449643330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80d2320 msgr2=0x7fa2c81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:44.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.415+0000 7fa2cdba5640 1 --2- 172.21.15.40:0/449643330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80d2320 0x7fa2c81625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.415+0000 7fa2cdba5640 1 -- 172.21.15.40:0/449643330 >> v1:172.21.15.184:6789/0 conn(0x7fa2c80b40d0 legacy=0x7fa2c80d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.415+0000 7fa2cdba5640 1 -- 172.21.15.40:0/449643330 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c80d40a0 con 0x7fa2c80d1a00 2024-04-26T21:43:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cdba5640 1 --2- 172.21.15.40:0/449643330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d1a00 0x7fa2c80d1de0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009870 tx=0x7fa2c002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d040679f284eea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2ccba3640 1 -- 172.21.15.40:0/449643330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0037070 con 0x7fa2c80d1a00 2024-04-26T21:43:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2ccba3640 1 -- 172.21.15.40:0/449643330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0032440 con 0x7fa2c80d1a00 2024-04-26T21:43:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2ccba3640 1 -- 172.21.15.40:0/449643330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0032810 con 0x7fa2c80d1a00 2024-04-26T21:43:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cfe09640 1 -- 172.21.15.40:0/449643330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d1a00 msgr2=0x7fa2c80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/449643330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d1a00 0x7fa2c80d1de0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009870 tx=0x7fa2c002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cfe09640 1 -- 172.21.15.40:0/449643330 shutdown_connections 2024-04-26T21:43:44.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/449643330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80d2320 0x7fa2c81625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/449643330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d1a00 0x7fa2c80d1de0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cfe09640 1 -- 172.21.15.40:0/449643330 >> 172.21.15.40:0/449643330 conn(0x7fa2c80d4fd0 msgr2=0x7fa2c80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:44.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cfe09640 1 -- 172.21.15.40:0/449643330 shutdown_connections 2024-04-26T21:43:44.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.416+0000 7fa2cfe09640 1 -- 172.21.15.40:0/449643330 wait complete. 2024-04-26T21:43:44.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.417+0000 7fa2cfe09640 1 Processor -- start 2024-04-26T21:43:44.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2cfe09640 1 -- start start 2024-04-26T21:43:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2cfe09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80b40d0 0x7fa2c814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2cfe09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2c80d1a00 0x7fa2c81513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2cfe09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d2320 0x7fa2c8151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2cfe09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c8162b30 con 0x7fa2c80d2320 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2cfe09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c80b5d60 con 0x7fa2c80d1a00 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2cfe09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c8162cb0 con 0x7fa2c80b40d0 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2ce3a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d2320 0x7fa2c8151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2ce3a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d2320 0x7fa2c8151920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55838/0 (socket says 172.21.15.40:55838) 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.418+0000 7fa2ce3a6640 1 -- 172.21.15.40:0/3086040842 learned_addr learned my addr 172.21.15.40:0/3086040842 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2cdba5640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80b40d0 0x7fa2c814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2cd3a4640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2c80d1a00 0x7fa2c81513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2ce3a6640 1 -- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80b40d0 msgr2=0x7fa2c814eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2ce3a6640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80b40d0 0x7fa2c814eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2ce3a6640 1 -- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2c80d1a00 msgr2=0x7fa2c81513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2ce3a6640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2c80d1a00 0x7fa2c81513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2ce3a6640 1 -- 172.21.15.40:0/3086040842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c814f060 con 0x7fa2c80d2320 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2cd3a4640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2c80d1a00 0x7fa2c81513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2ce3a6640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d2320 0x7fa2c8151920 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c7a0 tx=0x7fa2b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2c67fc640 1 -- 172.21.15.40:0/3086040842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003b70 con 0x7fa2c80d2320 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2c67fc640 1 -- 172.21.15.40:0/3086040842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b8003d30 con 0x7fa2c80d2320 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c80cb3b0 con 0x7fa2c80d2320 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2c67fc640 1 -- 172.21.15.40:0/3086040842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800aa90 con 0x7fa2c80d2320 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.419+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c80cb970 con 0x7fa2c80d2320 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.420+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c80694f0 con 0x7fa2c80d2320 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.421+0000 7fa2c67fc640 1 -- 172.21.15.40:0/3086040842 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2b800ac30 con 0x7fa2c80d2320 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.421+0000 7fa2c67fc640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2b406c8f0 0x7fa2b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.421+0000 7fa2c67fc640 1 -- 172.21.15.40:0/3086040842 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7163+0+0 (secure 0 0 0) 0x7fa2b808bf90 con 0x7fa2c80d2320 2024-04-26T21:43:44.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.421+0000 7fa2cdba5640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2b406c8f0 0x7fa2b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.422+0000 7fa2cdba5640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2b406c8f0 0x7fa2b406edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0034000 tx=0x7fa2c002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.425+0000 7fa2c67fc640 1 -- 172.21.15.40:0/3086040842 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2b8059e10 con 0x7fa2c80d2320 2024-04-26T21:43:44.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:44.651+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa2c80d2f70 con 0x7fa2c80d2320 2024-04-26T21:43:45.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.007+0000 7fa2c67fc640 1 -- 172.21.15.40:0/3086040842 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v159) ==== 162+0+0 (secure 0 0 0) 0x7fa2b8059710 con 0x7fa2c80d2320 2024-04-26T21:43:45.008 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:43:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2b406c8f0 msgr2=0x7fa2b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2b406c8f0 0x7fa2b406edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0034000 tx=0x7fa2c002e000 comp rx=0 tx=0).stop 2024-04-26T21:43:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d2320 msgr2=0x7fa2c8151920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d2320 0x7fa2c8151920 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c7a0 tx=0x7fa2b800cc60 comp rx=0 tx=0).stop 2024-04-26T21:43:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 shutdown_connections 2024-04-26T21:43:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2b406c8f0 0x7fa2b406edb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80d2320 0x7fa2c8151920 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2c80d1a00 0x7fa2c81513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 --2- 172.21.15.40:0/3086040842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2c80b40d0 0x7fa2c814eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 >> 172.21.15.40:0/3086040842 conn(0x7fa2c80d4fd0 msgr2=0x7fa2c81614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 shutdown_connections 2024-04-26T21:43:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.010+0000 7fa2cfe09640 1 -- 172.21.15.40:0/3086040842 wait complete. 2024-04-26T21:43:45.017 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:45.016+0000 7feac6ce1640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-04-26T21:43:45.017 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:45.016+0000 7feac6ce1640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-04-26T21:43:45.017 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:45.017+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:43:45.018 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:45.017+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:43:45.024 INFO:tasks.ceph.mds.c.smithi040.stderr: -27> 2024-04-26T21:43:45.016+0000 7feac6ce1640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-04-26T21:43:45.024 INFO:tasks.ceph.mds.c.smithi040.stderr: -26> 2024-04-26T21:43:45.016+0000 7feac6ce1640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-04-26T21:43:45.025 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:45.048 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:43:45.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.127+0000 7efd0d578640 1 Processor -- start 2024-04-26T21:43:45.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.127+0000 7efd0d578640 1 -- start start 2024-04-26T21:43:45.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.128+0000 7efd0d578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd0815f5a0 0x7efd080d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.128+0000 7efd0d578640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd08163610 0x7efd080cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.128+0000 7efd0d578640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd080d1a70 con 0x7efd080d2830 2024-04-26T21:43:45.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.128+0000 7efd0d578640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efd08163d30 con 0x7efd08163610 2024-04-26T21:43:45.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.128+0000 7efd0d578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd080d1d50 con 0x7efd0815f5a0 2024-04-26T21:43:45.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.128+0000 7efd077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd08163610 0x7efd080cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.128+0000 7efd067fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd080d2830 0x7efd08162f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37218/0 (socket says 172.21.15.40:37218) 2024-04-26T21:43:45.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.128+0000 7efd067fc640 1 -- 172.21.15.40:0/3500842121 learned_addr learned my addr 172.21.15.40:0/3500842121 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.129+0000 7efd06ffd640 1 --2- 172.21.15.40:0/3500842121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd0815f5a0 0x7efd080d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.129+0000 7efd077fe640 1 -- 172.21.15.40:0/3500842121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd0815f5a0 msgr2=0x7efd080d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.129+0000 7efd077fe640 1 --2- 172.21.15.40:0/3500842121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd0815f5a0 0x7efd080d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.129+0000 7efd077fe640 1 -- 172.21.15.40:0/3500842121 >> v1:172.21.15.40:6789/0 conn(0x7efd080d2830 legacy=0x7efd08162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.129+0000 7efd077fe640 1 -- 172.21.15.40:0/3500842121 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd080d2030 con 0x7efd08163610 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.129+0000 7efd077fe640 1 --2- 172.21.15.40:0/3500842121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd08163610 0x7efd080cfda0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009d90 tx=0x7efcf802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51544679226da3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd05ffb640 1 -- 172.21.15.40:0/3500842121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8031600 con 0x7efd08163610 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd05ffb640 1 -- 172.21.15.40:0/3500842121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf80317a0 con 0x7efd08163610 2024-04-26T21:43:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd05ffb640 1 -- 172.21.15.40:0/3500842121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8031b70 con 0x7efd08163610 2024-04-26T21:43:45.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd0d578640 1 -- 172.21.15.40:0/3500842121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd08163610 msgr2=0x7efd080cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd0d578640 1 --2- 172.21.15.40:0/3500842121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd08163610 0x7efd080cfda0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009d90 tx=0x7efcf802a5d0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd0d578640 1 -- 172.21.15.40:0/3500842121 shutdown_connections 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd0d578640 1 --2- 172.21.15.40:0/3500842121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd08163610 0x7efd080cfda0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd0d578640 1 --2- 172.21.15.40:0/3500842121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd0815f5a0 0x7efd080d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd0d578640 1 -- 172.21.15.40:0/3500842121 >> 172.21.15.40:0/3500842121 conn(0x7efd080d5190 msgr2=0x7efd080d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd0d578640 1 -- 172.21.15.40:0/3500842121 shutdown_connections 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.130+0000 7efd0d578640 1 -- 172.21.15.40:0/3500842121 wait complete. 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.131+0000 7efd0d578640 1 Processor -- start 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd0d578640 1 -- start start 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd0d578640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd080d2830 0x7efd0815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd0d578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd08163610 0x7efd08159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd0d578640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd080cea00 0x7efd080b9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd0d578640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd080d3c20 con 0x7efd080cea00 2024-04-26T21:43:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd0d578640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd080d4190 con 0x7efd080d2830 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd0d578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd080d3da0 con 0x7efd08163610 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd080cea00 0x7efd080b9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd080cea00 0x7efd080b9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55876/0 (socket says 172.21.15.40:55876) 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd077fe640 1 -- 172.21.15.40:0/3752310908 learned_addr learned my addr 172.21.15.40:0/3752310908 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd067fc640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd08163610 0x7efd08159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.132+0000 7efd06ffd640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd080d2830 0x7efd0815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd077fe640 1 -- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd08163610 msgr2=0x7efd08159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd077fe640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd08163610 0x7efd08159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd077fe640 1 -- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd080d2830 msgr2=0x7efd0815a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd077fe640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd080d2830 0x7efd0815a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd077fe640 1 -- 172.21.15.40:0/3752310908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd081f5740 con 0x7efd080cea00 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd06ffd640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd080d2830 0x7efd0815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd077fe640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd080cea00 0x7efd080b9f50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009eb0 tx=0x7efcf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efcff7fe640 1 -- 172.21.15.40:0/3752310908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8040070 con 0x7efd080cea00 2024-04-26T21:43:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efcff7fe640 1 -- 172.21.15.40:0/3752310908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf8030b60 con 0x7efd080cea00 2024-04-26T21:43:45.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd080cf250 con 0x7efd080cea00 2024-04-26T21:43:45.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efcff7fe640 1 -- 172.21.15.40:0/3752310908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8009450 con 0x7efd080cea00 2024-04-26T21:43:45.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.133+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd080ca010 con 0x7efd080cea00 2024-04-26T21:43:45.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.134+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd08069a80 con 0x7efd080cea00 2024-04-26T21:43:45.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.135+0000 7efcff7fe640 1 -- 172.21.15.40:0/3752310908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcf8039040 con 0x7efd080cea00 2024-04-26T21:43:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.135+0000 7efcff7fe640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efce806c8f0 0x7efce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.135+0000 7efcff7fe640 1 -- 172.21.15.40:0/3752310908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7efcf80ad800 con 0x7efd080cea00 2024-04-26T21:43:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.135+0000 7efd06ffd640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efce806c8f0 0x7efce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.136+0000 7efd06ffd640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efce806c8f0 0x7efce806edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7efd00006fd0 tx=0x7efd00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.139+0000 7efcff7fe640 1 -- 172.21.15.40:0/3752310908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd08069a80 con 0x7efd080cea00 2024-04-26T21:43:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.365+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd080694f0 con 0x7efd080cea00 2024-04-26T21:43:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.365+0000 7efcff7fe640 1 -- 172.21.15.40:0/3752310908 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) ==== 95+0+4355 (secure 0 0 0) 0x7efcf807b620 con 0x7efd080cea00 2024-04-26T21:43:45.366 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:45.367 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":161,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:45.018290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 161 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.367+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efce806c8f0 msgr2=0x7efce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.367+0000 7efd0d578640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efce806c8f0 0x7efce806edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7efd00006fd0 tx=0x7efd00008040 comp rx=0 tx=0).stop 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.367+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd080cea00 msgr2=0x7efd080b9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.367+0000 7efd0d578640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd080cea00 0x7efd080b9f50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009eb0 tx=0x7efcf8004a40 comp rx=0 tx=0).stop 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.368+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 shutdown_connections 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.368+0000 7efd0d578640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efce806c8f0 0x7efce806edb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.368+0000 7efd0d578640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd080cea00 0x7efd080b9f50 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.368+0000 7efd0d578640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd08163610 0x7efd08159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.368+0000 7efd0d578640 1 --2- 172.21.15.40:0/3752310908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd080d2830 0x7efd0815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.368+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 >> 172.21.15.40:0/3752310908 conn(0x7efd080d5190 msgr2=0x7efd08160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.368+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 shutdown_connections 2024-04-26T21:43:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.368+0000 7efd0d578640 1 -- 172.21.15.40:0/3752310908 wait complete. 2024-04-26T21:43:45.379 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:45.380 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:45.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.519+0000 7f94d1299640 1 Processor -- start 2024-04-26T21:43:45.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.519+0000 7f94d1299640 1 -- start start 2024-04-26T21:43:45.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.520+0000 7f94d1299640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94cc1608f0 0x7f94cc0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.520+0000 7f94d1299640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc0cffa0 0x7f94cc0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.520+0000 7f94d1299640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94cc0cde20 con 0x7f94cc0d0a90 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.520+0000 7f94d1299640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f94cc0cea30 con 0x7f94cc1608f0 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.520+0000 7f94d1299640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94cc0ce100 con 0x7f94cc0cffa0 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.520+0000 7f94caffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94cc1608f0 0x7f94cc0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.520+0000 7f94ca7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f94cc0d0a90 0x7f94cc0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37230/0 (socket says 172.21.15.40:37230) 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.520+0000 7f94ca7fc640 1 -- 172.21.15.40:0/16708032 learned_addr learned my addr 172.21.15.40:0/16708032 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94cb7fe640 1 --2- 172.21.15.40:0/16708032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc0cffa0 0x7f94cc0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94caffd640 1 -- 172.21.15.40:0/16708032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc0cffa0 msgr2=0x7f94cc0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94caffd640 1 --2- 172.21.15.40:0/16708032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc0cffa0 0x7f94cc0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94caffd640 1 -- 172.21.15.40:0/16708032 >> v1:172.21.15.40:6789/0 conn(0x7f94cc0d0a90 legacy=0x7f94cc0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94c9ffb640 1 -- 172.21.15.40:0/16708032 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816759940 0 0) 0x7f94cc0cde20 con 0x7f94cc0d0a90 2024-04-26T21:43:45.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94caffd640 1 -- 172.21.15.40:0/16708032 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc0ce3e0 con 0x7f94cc1608f0 2024-04-26T21:43:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94caffd640 1 --2- 172.21.15.40:0/16708032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc1608f0 0x7f94cc0d1bf0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009d90 tx=0x7f94c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587ea40a628ff78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94c9ffb640 1 -- 172.21.15.40:0/16708032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4007740 con 0x7f94cc1608f0 2024-04-26T21:43:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94c9ffb640 1 -- 172.21.15.40:0/16708032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c40078e0 con 0x7f94cc1608f0 2024-04-26T21:43:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.521+0000 7f94c9ffb640 1 -- 172.21.15.40:0/16708032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4007cb0 con 0x7f94cc1608f0 2024-04-26T21:43:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.522+0000 7f94d1299640 1 -- 172.21.15.40:0/16708032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc1608f0 msgr2=0x7f94cc0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.522+0000 7f94d1299640 1 --2- 172.21.15.40:0/16708032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc1608f0 0x7f94cc0d1bf0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009d90 tx=0x7f94c402a660 comp rx=0 tx=0).stop 2024-04-26T21:43:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.522+0000 7f94d1299640 1 -- 172.21.15.40:0/16708032 shutdown_connections 2024-04-26T21:43:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.522+0000 7f94d1299640 1 --2- 172.21.15.40:0/16708032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc0cffa0 0x7f94cc0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.522+0000 7f94d1299640 1 --2- 172.21.15.40:0/16708032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc1608f0 0x7f94cc0d1bf0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.522+0000 7f94d1299640 1 -- 172.21.15.40:0/16708032 >> 172.21.15.40:0/16708032 conn(0x7f94cc0d2ec0 msgr2=0x7f94cc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.522+0000 7f94d1299640 1 -- 172.21.15.40:0/16708032 shutdown_connections 2024-04-26T21:43:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.522+0000 7f94d1299640 1 -- 172.21.15.40:0/16708032 wait complete. 2024-04-26T21:43:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.523+0000 7f94d1299640 1 Processor -- start 2024-04-26T21:43:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.523+0000 7f94d1299640 1 -- start start 2024-04-26T21:43:45.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94d1299640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc0cffa0 0x7f94cc1f0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94d1299640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94cc0d0a90 0x7f94cc0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94d1299640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc1608f0 0x7f94cc1f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94d1299640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc0ceec0 con 0x7f94cc0cffa0 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94d1299640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc0d0ef0 con 0x7f94cc0d0a90 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94d1299640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94cc0cda40 con 0x7f94cc1608f0 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94caffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc0cffa0 0x7f94cc1f0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94caffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc0cffa0 0x7f94cc1f0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55902/0 (socket says 172.21.15.40:55902) 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94caffd640 1 -- 172.21.15.40:0/2843548414 learned_addr learned my addr 172.21.15.40:0/2843548414 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94cb7fe640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc1608f0 0x7f94cc1f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.524+0000 7f94ca7fc640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94cc0d0a90 0x7f94cc0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94caffd640 1 -- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc1608f0 msgr2=0x7f94cc1f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94caffd640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc1608f0 0x7f94cc1f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94caffd640 1 -- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94cc0d0a90 msgr2=0x7f94cc0c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94caffd640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94cc0d0a90 0x7f94cc0c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94caffd640 1 -- 172.21.15.40:0/2843548414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc0c68c0 con 0x7f94cc0cffa0 2024-04-26T21:43:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94ca7fc640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94cc0d0a90 0x7f94cc0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94caffd640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc0cffa0 0x7f94cc1f0230 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009eb0 tx=0x7f94c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94c37fe640 1 -- 172.21.15.40:0/2843548414 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4040070 con 0x7f94cc0cffa0 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94c37fe640 1 -- 172.21.15.40:0/2843548414 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c4031c40 con 0x7f94cc0cffa0 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94cc1f0770 con 0x7f94cc0cffa0 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94c37fe640 1 -- 172.21.15.40:0/2843548414 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4045460 con 0x7f94cc0cffa0 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.525+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94cc0c6e40 con 0x7f94cc0cffa0 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.526+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94cc0694f0 con 0x7f94cc0cffa0 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.526+0000 7f94c37fe640 1 -- 172.21.15.40:0/2843548414 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94c4039040 con 0x7f94cc0cffa0 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.527+0000 7f94c37fe640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94ac06c8f0 0x7f94ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.527+0000 7f94c37fe640 1 -- 172.21.15.40:0/2843548414 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f94c40adb00 con 0x7f94cc0cffa0 2024-04-26T21:43:45.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.527+0000 7f94ca7fc640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94ac06c8f0 0x7f94ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.528+0000 7f94ca7fc640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94ac06c8f0 0x7f94ac06edb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0a0e20 tx=0x7f94b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.531+0000 7f94c37fe640 1 -- 172.21.15.40:0/2843548414 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94c4043360 con 0x7f94cc0cffa0 2024-04-26T21:43:45.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.754+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94cc0c6600 con 0x7f94cc0cffa0 2024-04-26T21:43:45.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.755+0000 7f94c37fe640 1 -- 172.21.15.40:0/2843548414 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) ==== 95+0+4355 (secure 0 0 0) 0x7f94c407b920 con 0x7f94cc0cffa0 2024-04-26T21:43:45.756 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:45.756 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":161,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:45.018290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:45.756 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 161 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94ac06c8f0 msgr2=0x7f94ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94ac06c8f0 0x7f94ac06edb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0a0e20 tx=0x7f94b4009040 comp rx=0 tx=0).stop 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc0cffa0 msgr2=0x7f94cc1f0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc0cffa0 0x7f94cc1f0230 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009eb0 tx=0x7f94c4004a40 comp rx=0 tx=0).stop 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 shutdown_connections 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94ac06c8f0 0x7f94ac06edb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94cc1608f0 0x7f94cc1f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94cc0d0a90 0x7f94cc0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 --2- 172.21.15.40:0/2843548414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94cc0cffa0 0x7f94cc1f0230 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 >> 172.21.15.40:0/2843548414 conn(0x7f94cc0d2ec0 msgr2=0x7f94cc15b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 shutdown_connections 2024-04-26T21:43:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.757+0000 7f94d1299640 1 -- 172.21.15.40:0/2843548414 wait complete. 2024-04-26T21:43:45.768 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:43:45.769 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.869+0000 7fd8da1e7640 1 Processor -- start 2024-04-26T21:43:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.869+0000 7fd8da1e7640 1 -- start start 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8da1e7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd8d40b2340 0x7fd8d40cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8da1e7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8d40cd930 0x7fd8d40d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8da1e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8d40d10d0 con 0x7fd8d4162810 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8da1e7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd8d40d0750 con 0x7fd8d40cd930 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8da1e7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd8d40d16a0 con 0x7fd8d40b2340 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8d3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8d40cd930 0x7fd8d40d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8d3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8d40cd930 0x7fd8d40d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55914/0 (socket says 172.21.15.40:55914) 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8d3fff640 1 -- 172.21.15.40:0/2121989102 learned_addr learned my addr 172.21.15.40:0/2121989102 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8d2ffd640 1 --2- 172.21.15.40:0/2121989102 >> v2:172.21.15.184:3300/0 conn(0x7fd8d40b2340 0x7fd8d40cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8d3fff640 1 -- 172.21.15.40:0/2121989102 >> v2:172.21.15.184:3300/0 conn(0x7fd8d40b2340 msgr2=0x7fd8d40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8d3fff640 1 --2- 172.21.15.40:0/2121989102 >> v2:172.21.15.184:3300/0 conn(0x7fd8d40b2340 0x7fd8d40cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.870+0000 7fd8d3fff640 1 -- 172.21.15.40:0/2121989102 >> v1:172.21.15.184:6789/0 conn(0x7fd8d4162810 legacy=0x7fd8d4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.871+0000 7fd8d3fff640 1 -- 172.21.15.40:0/2121989102 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d40d1370 con 0x7fd8d40cd930 2024-04-26T21:43:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.871+0000 7fd8d3fff640 1 --2- 172.21.15.40:0/2121989102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d40cd930 0x7fd8d40d1b30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40cd0a0 tx=0x7fd8cc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691b05e635d61a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.871+0000 7fd8d27fc640 1 -- 172.21.15.40:0/2121989102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc036070 con 0x7fd8d40cd930 2024-04-26T21:43:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.871+0000 7fd8d27fc640 1 -- 172.21.15.40:0/2121989102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc031600 con 0x7fd8d40cd930 2024-04-26T21:43:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.871+0000 7fd8d27fc640 1 -- 172.21.15.40:0/2121989102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0319d0 con 0x7fd8d40cd930 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.872+0000 7fd8da1e7640 1 -- 172.21.15.40:0/2121989102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d40cd930 msgr2=0x7fd8d40d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.872+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/2121989102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d40cd930 0x7fd8d40d1b30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40cd0a0 tx=0x7fd8cc02a730 comp rx=0 tx=0).stop 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.872+0000 7fd8da1e7640 1 -- 172.21.15.40:0/2121989102 shutdown_connections 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.872+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/2121989102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d40cd930 0x7fd8d40d1b30 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.872+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/2121989102 >> v2:172.21.15.184:3300/0 conn(0x7fd8d40b2340 0x7fd8d40cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.872+0000 7fd8da1e7640 1 -- 172.21.15.40:0/2121989102 >> 172.21.15.40:0/2121989102 conn(0x7fd8d40d2da0 msgr2=0x7fd8d40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.872+0000 7fd8da1e7640 1 -- 172.21.15.40:0/2121989102 shutdown_connections 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.872+0000 7fd8da1e7640 1 -- 172.21.15.40:0/2121989102 wait complete. 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.873+0000 7fd8da1e7640 1 Processor -- start 2024-04-26T21:43:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.873+0000 7fd8da1e7640 1 -- start start 2024-04-26T21:43:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8da1e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8d40b2340 0x7fd8d41f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8da1e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8d40cd930 0x7fd8d4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8da1e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d4162810 0x7fd8d4157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8da1e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d40d2070 con 0x7fd8d4162810 2024-04-26T21:43:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8da1e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d40d0470 con 0x7fd8d40cd930 2024-04-26T21:43:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8da1e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d40d21f0 con 0x7fd8d40b2340 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d4162810 0x7fd8d4157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d4162810 0x7fd8d4157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55924/0 (socket says 172.21.15.40:55924) 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8d3fff640 1 -- 172.21.15.40:0/3864183544 learned_addr learned my addr 172.21.15.40:0/3864183544 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8d2ffd640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8d40cd930 0x7fd8d4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.874+0000 7fd8d37fe640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8d40b2340 0x7fd8d41f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8d3fff640 1 -- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8d40b2340 msgr2=0x7fd8d41f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8d3fff640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8d40b2340 0x7fd8d41f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8d3fff640 1 -- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8d40cd930 msgr2=0x7fd8d4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8d3fff640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8d40cd930 0x7fd8d4157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8d3fff640 1 -- 172.21.15.40:0/3864183544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d415c560 con 0x7fd8d4162810 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8d2ffd640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8d40cd930 0x7fd8d4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8d3fff640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d4162810 0x7fd8d4157a90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009f40 tx=0x7fd8cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8bbfff640 1 -- 172.21.15.40:0/3864183544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc036050 con 0x7fd8d4162810 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8bbfff640 1 -- 172.21.15.40:0/3864183544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc042070 con 0x7fd8d4162810 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d415ad70 con 0x7fd8d4162810 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8bbfff640 1 -- 172.21.15.40:0/3864183544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc009510 con 0x7fd8d4162810 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.875+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d40c79a0 con 0x7fd8d4162810 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.876+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d40694f0 con 0x7fd8d4162810 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.877+0000 7fd8bbfff640 1 -- 172.21.15.40:0/3864183544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8cc007bf0 con 0x7fd8d4162810 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.877+0000 7fd8bbfff640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8c006c8f0 0x7fd8c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.877+0000 7fd8bbfff640 1 -- 172.21.15.40:0/3864183544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7fd8cc0ace60 con 0x7fd8d4162810 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.877+0000 7fd8d37fe640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8c006c8f0 0x7fd8c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.878+0000 7fd8d37fe640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8c006c8f0 0x7fd8c006edb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4000f60 tx=0x7fd8c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:45.882+0000 7fd8bbfff640 1 -- 172.21.15.40:0/3864183544 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8d415c560 con 0x7fd8d4162810 2024-04-26T21:43:46.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.112+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8d40b4170 con 0x7fd8d4162810 2024-04-26T21:43:46.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.112+0000 7fd8bbfff640 1 -- 172.21.15.40:0/3864183544 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) ==== 95+0+5181 (secure 0 0 0) 0x7fd8cc07ac90 con 0x7fd8d4162810 2024-04-26T21:43:46.113 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:46.114 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":162,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1803384446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1803384446},{"type":"v1","addr":"172.21.15.40:6833","nonce":1803384446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:45.018290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:46.114 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 162 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8c006c8f0 msgr2=0x7fd8c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8c006c8f0 0x7fd8c006edb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4000f60 tx=0x7fd8c4009040 comp rx=0 tx=0).stop 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d4162810 msgr2=0x7fd8d4157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d4162810 0x7fd8d4157a90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009f40 tx=0x7fd8cc004a40 comp rx=0 tx=0).stop 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 shutdown_connections 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8c006c8f0 0x7fd8c006edb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d4162810 0x7fd8d4157a90 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8d40cd930 0x7fd8d4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 --2- 172.21.15.40:0/3864183544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8d40b2340 0x7fd8d41f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 >> 172.21.15.40:0/3864183544 conn(0x7fd8d40d2da0 msgr2=0x7fd8d406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 shutdown_connections 2024-04-26T21:43:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.115+0000 7fd8da1e7640 1 -- 172.21.15.40:0/3864183544 wait complete. 2024-04-26T21:43:46.127 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_sessionmap' led to DAMAGED state 2024-04-26T21:43:46.127 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 400.00000000' 2024-04-26T21:43:46.127 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:46.127 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:46.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:43:46.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:43:46.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:46.144 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:43:46.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:43:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:43:46.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:43:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.340+0000 7f95c9c37640 1 Processor -- start 2024-04-26T21:43:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.340+0000 7f95c9c37640 1 -- start start 2024-04-26T21:43:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.340+0000 7f95c9c37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c415f3a0 0x7f95c40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.340+0000 7f95c9c37640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95c40d1770 0x7f95c40d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.340+0000 7f95c9c37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c40d25a0 con 0x7f95c40d2130 2024-04-26T21:43:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.340+0000 7f95c9c37640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95c40d0e20 con 0x7f95c40d1770 2024-04-26T21:43:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.340+0000 7f95c9c37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40b1d70 con 0x7f95c415f3a0 2024-04-26T21:43:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c2ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95c40d1770 0x7f95c40d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f95c40d2130 0x7f95c40d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37232/0 (socket says 172.21.15.40:37232) 2024-04-26T21:43:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c3fff640 1 -- 172.21.15.40:0/310228729 learned_addr learned my addr 172.21.15.40:0/310228729 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c37fe640 1 --2- 172.21.15.40:0/310228729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c415f3a0 0x7f95c40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c2ffd640 1 -- 172.21.15.40:0/310228729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c415f3a0 msgr2=0x7f95c40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c2ffd640 1 --2- 172.21.15.40:0/310228729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c415f3a0 0x7f95c40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c2ffd640 1 -- 172.21.15.40:0/310228729 >> v1:172.21.15.40:6789/0 conn(0x7f95c40d2130 legacy=0x7f95c40d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c27fc640 1 -- 172.21.15.40:0/310228729 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706061999 0 0) 0x7f95bc001490 con 0x7f95c40d2130 2024-04-26T21:43:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.341+0000 7f95c2ffd640 1 -- 172.21.15.40:0/310228729 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c40b2050 con 0x7f95c40d1770 2024-04-26T21:43:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c2ffd640 1 --2- 172.21.15.40:0/310228729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d1770 0x7f95c40d1bf0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f95b80097a0 tx=0x7f95b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a280dd6dea91be07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c27fc640 1 -- 172.21.15.40:0/310228729 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8037070 con 0x7f95c40d1770 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c27fc640 1 -- 172.21.15.40:0/310228729 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b8032440 con 0x7f95c40d1770 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c27fc640 1 -- 172.21.15.40:0/310228729 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8032810 con 0x7f95c40d1770 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c9c37640 1 -- 172.21.15.40:0/310228729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d1770 msgr2=0x7f95c40d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c9c37640 1 --2- 172.21.15.40:0/310228729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d1770 0x7f95c40d1bf0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f95b80097a0 tx=0x7f95b802a180 comp rx=0 tx=0).stop 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c9c37640 1 -- 172.21.15.40:0/310228729 shutdown_connections 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c9c37640 1 --2- 172.21.15.40:0/310228729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d1770 0x7f95c40d1bf0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c9c37640 1 --2- 172.21.15.40:0/310228729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c415f3a0 0x7f95c40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c9c37640 1 -- 172.21.15.40:0/310228729 >> 172.21.15.40:0/310228729 conn(0x7f95c40d4f90 msgr2=0x7f95c40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c9c37640 1 -- 172.21.15.40:0/310228729 shutdown_connections 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.342+0000 7f95c9c37640 1 -- 172.21.15.40:0/310228729 wait complete. 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.344+0000 7f95c9c37640 1 Processor -- start 2024-04-26T21:43:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.344+0000 7f95c9c37640 1 -- start start 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.344+0000 7f95c9c37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40d1770 0x7f95c4153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.344+0000 7f95c9c37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d2130 0x7f95c40b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.344+0000 7f95c9c37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c415f3a0 0x7f95c40ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.344+0000 7f95c9c37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95c415f900 con 0x7f95c40d2130 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.344+0000 7f95c9c37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95c415f780 con 0x7f95c415f3a0 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.344+0000 7f95c9c37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40b3860 con 0x7f95c40d1770 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d2130 0x7f95c40b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d2130 0x7f95c40b9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55930/0 (socket says 172.21.15.40:55930) 2024-04-26T21:43:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 -- 172.21.15.40:0/4144112482 learned_addr learned my addr 172.21.15.40:0/4144112482 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c37fe640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40d1770 0x7f95c4153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c3fff640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c415f3a0 0x7f95c40ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 -- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40d1770 msgr2=0x7f95c4153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40d1770 0x7f95c4153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 -- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c415f3a0 msgr2=0x7f95c40ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c415f3a0 0x7f95c40ba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 -- 172.21.15.40:0/4144112482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c40c9980 con 0x7f95c40d2130 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c37fe640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40d1770 0x7f95c4153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c3fff640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c415f3a0 0x7f95c40ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c2ffd640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d2130 0x7f95c40b9f90 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000c00 tx=0x7f95b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95abfff640 1 -- 172.21.15.40:0/4144112482 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8037040 con 0x7f95c40d2130 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95abfff640 1 -- 172.21.15.40:0/4144112482 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b8042070 con 0x7f95c40d2130 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c40c9c60 con 0x7f95c40d2130 2024-04-26T21:43:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95abfff640 1 -- 172.21.15.40:0/4144112482 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8030940 con 0x7f95c40d2130 2024-04-26T21:43:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.345+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c40b7dd0 con 0x7f95c40d2130 2024-04-26T21:43:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.346+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c40694f0 con 0x7f95c40d2130 2024-04-26T21:43:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.347+0000 7f95abfff640 1 -- 172.21.15.40:0/4144112482 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95b8030ae0 con 0x7f95c40d2130 2024-04-26T21:43:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.347+0000 7f95abfff640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95b006c8f0 0x7f95b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.347+0000 7f95abfff640 1 -- 172.21.15.40:0/4144112482 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f95b80ac9c0 con 0x7f95c40d2130 2024-04-26T21:43:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.348+0000 7f95c37fe640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95b006c8f0 0x7f95b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.348+0000 7f95c37fe640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95b006c8f0 0x7f95b006edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009af0 tx=0x7f95b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:46.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.352+0000 7f95abfff640 1 -- 172.21.15.40:0/4144112482 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95b80aee20 con 0x7f95c40d2130 2024-04-26T21:43:46.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:46.575+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f95c40b6860 con 0x7f95c40d2130 2024-04-26T21:43:47.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.024+0000 7f95abfff640 1 -- 172.21.15.40:0/4144112482 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v163) ==== 161+0+0 (secure 0 0 0) 0x7f95b8045360 con 0x7f95c40d2130 2024-04-26T21:43:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95b006c8f0 msgr2=0x7f95b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95b006c8f0 0x7f95b006edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009af0 tx=0x7f95b4008040 comp rx=0 tx=0).stop 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d2130 msgr2=0x7f95c40b9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d2130 0x7f95c40b9f90 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000c00 tx=0x7f95b80046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 shutdown_connections 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f95b006c8f0 0x7f95b006edb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c415f3a0 0x7f95c40ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c40d2130 0x7f95c40b9f90 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 --2- 172.21.15.40:0/4144112482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c40d1770 0x7f95c4153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 >> 172.21.15.40:0/4144112482 conn(0x7f95c40d4f90 msgr2=0x7f95c40d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 shutdown_connections 2024-04-26T21:43:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.027+0000 7f95c9c37640 1 -- 172.21.15.40:0/4144112482 wait complete. 2024-04-26T21:43:47.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:43:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.145+0000 7f8f1d452640 1 Processor -- start 2024-04-26T21:43:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.145+0000 7f8f1d452640 1 -- start start 2024-04-26T21:43:47.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f1d452640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f180b2340 0x7f8f180cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f1d452640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8f180cd930 0x7f8f180d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f1d452640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f180d10d0 con 0x7f8f18162810 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f1d452640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8f180d0750 con 0x7f8f180b2340 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f1d452640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8f180d16a0 con 0x7f8f180cd930 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f167fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f180b2340 0x7f8f180cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f167fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f180b2340 0x7f8f180cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55946/0 (socket says 172.21.15.40:55946) 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f167fc640 1 -- 172.21.15.40:0/4099298304 learned_addr learned my addr 172.21.15.40:0/4099298304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.146+0000 7f8f177fe640 1 --2- 172.21.15.40:0/4099298304 >> v2:172.21.15.184:3300/0 conn(0x7f8f180cd930 0x7f8f180d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.147+0000 7f8f167fc640 1 -- 172.21.15.40:0/4099298304 >> v2:172.21.15.184:3300/0 conn(0x7f8f180cd930 msgr2=0x7f8f180d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.147+0000 7f8f167fc640 1 --2- 172.21.15.40:0/4099298304 >> v2:172.21.15.184:3300/0 conn(0x7f8f180cd930 0x7f8f180d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.147+0000 7f8f167fc640 1 -- 172.21.15.40:0/4099298304 >> v1:172.21.15.184:6789/0 conn(0x7f8f18162810 legacy=0x7f8f18070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.147+0000 7f8f167fc640 1 -- 172.21.15.40:0/4099298304 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f180d1370 con 0x7f8f180b2340 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.147+0000 7f8f167fc640 1 --2- 172.21.15.40:0/4099298304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180b2340 0x7f8f180cd3f0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f8f180b2280 tx=0x7f8f0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a5f37bcf912aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.147+0000 7f8f15ffb640 1 -- 172.21.15.40:0/4099298304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c037070 con 0x7f8f180b2340 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f15ffb640 1 -- 172.21.15.40:0/4099298304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c032440 con 0x7f8f180b2340 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f15ffb640 1 -- 172.21.15.40:0/4099298304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c032810 con 0x7f8f180b2340 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f1d452640 1 -- 172.21.15.40:0/4099298304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180b2340 msgr2=0x7f8f180cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f1d452640 1 --2- 172.21.15.40:0/4099298304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180b2340 0x7f8f180cd3f0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f8f180b2280 tx=0x7f8f0c02a270 comp rx=0 tx=0).stop 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f1d452640 1 -- 172.21.15.40:0/4099298304 shutdown_connections 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f1d452640 1 --2- 172.21.15.40:0/4099298304 >> v2:172.21.15.184:3300/0 conn(0x7f8f180cd930 0x7f8f180d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f1d452640 1 --2- 172.21.15.40:0/4099298304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180b2340 0x7f8f180cd3f0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f1d452640 1 -- 172.21.15.40:0/4099298304 >> 172.21.15.40:0/4099298304 conn(0x7f8f180d2da0 msgr2=0x7f8f180d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f1d452640 1 -- 172.21.15.40:0/4099298304 shutdown_connections 2024-04-26T21:43:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.148+0000 7f8f1d452640 1 -- 172.21.15.40:0/4099298304 wait complete. 2024-04-26T21:43:47.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.149+0000 7f8f1d452640 1 Processor -- start 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.149+0000 7f8f1d452640 1 -- start start 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f1d452640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f180b2340 0x7f8f18155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f1d452640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180cd930 0x7f8f18157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f1d452640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f18162810 0x7f8f1815d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f1d452640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d2070 con 0x7f8f180cd930 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f1d452640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d0470 con 0x7f8f180b2340 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f1d452640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180d21f0 con 0x7f8f18162810 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f167fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180cd930 0x7f8f18157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f167fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180cd930 0x7f8f18157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55962/0 (socket says 172.21.15.40:55962) 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f167fc640 1 -- 172.21.15.40:0/3222637020 learned_addr learned my addr 172.21.15.40:0/3222637020 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.150+0000 7f8f177fe640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f18162810 0x7f8f1815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f16ffd640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f180b2340 0x7f8f18155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f167fc640 1 -- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f18162810 msgr2=0x7f8f1815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f167fc640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f18162810 0x7f8f1815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f167fc640 1 -- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f180b2340 msgr2=0x7f8f18155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f167fc640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f180b2340 0x7f8f18155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f167fc640 1 -- 172.21.15.40:0/3222637020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f181562a0 con 0x7f8f180cd930 2024-04-26T21:43:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f16ffd640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f180b2340 0x7f8f18155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f167fc640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180cd930 0x7f8f18157a20 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c02a2b0 tx=0x7f8f0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8eff7fe640 1 -- 172.21.15.40:0/3222637020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c037040 con 0x7f8f180cd930 2024-04-26T21:43:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8eff7fe640 1 -- 172.21.15.40:0/3222637020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c042070 con 0x7f8f180cd930 2024-04-26T21:43:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1815fff0 con 0x7f8f180cd930 2024-04-26T21:43:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8eff7fe640 1 -- 172.21.15.40:0/3222637020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c030880 con 0x7f8f180cd930 2024-04-26T21:43:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.151+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f18160890 con 0x7f8f180cd930 2024-04-26T21:43:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.152+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f180764d0 con 0x7f8f180cd930 2024-04-26T21:43:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.153+0000 7f8eff7fe640 1 -- 172.21.15.40:0/3222637020 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f0c030a20 con 0x7f8f180cd930 2024-04-26T21:43:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.153+0000 7f8eff7fe640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8f0406c8f0 0x7f8f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.153+0000 7f8eff7fe640 1 -- 172.21.15.40:0/3222637020 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f8f0c0aca90 con 0x7f8f180cd930 2024-04-26T21:43:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.153+0000 7f8f16ffd640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8f0406c8f0 0x7f8f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.154+0000 7f8f16ffd640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8f0406c8f0 0x7f8f0406edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8f08001660 tx=0x7f8f08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.157+0000 7f8eff7fe640 1 -- 172.21.15.40:0/3222637020 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f180764d0 con 0x7f8f180cd930 2024-04-26T21:43:47.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:47.418+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8f180694f0 con 0x7f8f180cd930 2024-04-26T21:43:48.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.026+0000 7f8eff7fe640 1 -- 172.21.15.40:0/3222637020 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v164) ==== 101+0+0 (secure 0 0 0) 0x7f8f0c07a8b0 con 0x7f8f180cd930 2024-04-26T21:43:48.027 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:43:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8f0406c8f0 msgr2=0x7f8f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8f0406c8f0 0x7f8f0406edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8f08001660 tx=0x7f8f08009040 comp rx=0 tx=0).stop 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180cd930 msgr2=0x7f8f18157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180cd930 0x7f8f18157a20 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c02a2b0 tx=0x7f8f0c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 shutdown_connections 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8f0406c8f0 0x7f8f0406edb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f18162810 0x7f8f1815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f180cd930 0x7f8f18157a20 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 --2- 172.21.15.40:0/3222637020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f180b2340 0x7f8f18155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 >> 172.21.15.40:0/3222637020 conn(0x7f8f180d2da0 msgr2=0x7f8f180b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 shutdown_connections 2024-04-26T21:43:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.029+0000 7f8f1d452640 1 -- 172.21.15.40:0/3222637020 wait complete. 2024-04-26T21:43:48.043 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:43:48.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.087+0000 7f9487f56f40 1 Processor -- start 2024-04-26T21:43:48.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.087+0000 7f9487f56f40 1 -- start start 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9487f56f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 0x55b380ec5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9487f56f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 0x55b380ec6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9487f56f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380ecb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9487f56f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b380e1a140 con 0x55b380ec69a0 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9487f56f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b380d03e40 con 0x55b380d5b520 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9487f56f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b380d5cdc0 con 0x55b380d5b140 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9486251640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380ecb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9486251640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380ecb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55988/0 (socket says 172.21.15.40:55988) 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9486251640 1 -- 172.21.15.40:0/1603628806 learned_addr learned my addr 172.21.15.40:0/1603628806 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9485a50640 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 0x55b380ec5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f948524f640 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 0x55b380ec6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9486251640 1 -- 172.21.15.40:0/1603628806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 msgr2=0x55b380ec5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9486251640 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 0x55b380ec5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9486251640 1 -- 172.21.15.40:0/1603628806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 msgr2=0x55b380ec6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9486251640 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 0x55b380ec6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.088+0000 7f9486251640 1 -- 172.21.15.40:0/1603628806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b380da0cb0 con 0x55b380ec69a0 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9486251640 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380ecb1d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f9480005fd0 tx=0x7f9480005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554b854acd0b46c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9477fff640 1 -- 172.21.15.40:0/1603628806 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480017030 con 0x55b380ec69a0 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9477fff640 1 -- 172.21.15.40:0/1603628806 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9480012670 con 0x55b380ec69a0 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9477fff640 1 -- 172.21.15.40:0/1603628806 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480012840 con 0x55b380ec69a0 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 -- 172.21.15.40:0/1603628806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 msgr2=0x55b380ecb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380ecb1d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f9480005fd0 tx=0x7f9480005d80 comp rx=0 tx=0).stop 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 -- 172.21.15.40:0/1603628806 shutdown_connections 2024-04-26T21:43:48.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380ecb1d0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 0x55b380ec6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1603628806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 0x55b380ec5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 -- 172.21.15.40:0/1603628806 >> 172.21.15.40:0/1603628806 conn(0x55b380d5aab0 msgr2=0x55b380d89720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 -- 172.21.15.40:0/1603628806 shutdown_connections 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.089+0000 7f9487f56f40 1 -- 172.21.15.40:0/1603628806 wait complete. 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.090+0000 7f9487f56f40 1 Processor -- start 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.090+0000 7f9487f56f40 1 -- start start 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9487f56f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 0x55b380e35930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9487f56f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 0x55b380d8a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9487f56f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380d8a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9487f56f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b380e169f0 con 0x55b380ec69a0 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9487f56f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b380d9f890 con 0x55b380d5b520 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9487f56f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b380e17160 con 0x55b380d5b140 2024-04-26T21:43:48.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380d8a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380d8a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56004/0 (socket says 172.21.15.40:56004) 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 learned_addr learned my addr 172.21.15.40:0/1896788799 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9485a50640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 0x55b380e35930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f948524f640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 0x55b380d8a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 msgr2=0x55b380e35930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b380d5b140 0x55b380e35930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 msgr2=0x55b380d8a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 0x55b380d8a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b380e36180 con 0x55b380ec69a0 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f948524f640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b380d5b520 0x55b380d8a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.091+0000 7f9486251640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380d8a720 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f9480007010 tx=0x7f9480003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.092+0000 7f9476ffd640 1 -- 172.21.15.40:0/1896788799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480013a50 con 0x55b380ec69a0 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.092+0000 7f9476ffd640 1 -- 172.21.15.40:0/1896788799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9480013bf0 con 0x55b380ec69a0 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.092+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b380e36460 con 0x55b380ec69a0 2024-04-26T21:43:48.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.092+0000 7f9476ffd640 1 -- 172.21.15.40:0/1896788799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948001c410 con 0x55b380ec69a0 2024-04-26T21:43:48.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.092+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b380e135e0 con 0x55b380ec69a0 2024-04-26T21:43:48.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.093+0000 7f9476ffd640 1 -- 172.21.15.40:0/1896788799 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f948001a030 con 0x55b380ec69a0 2024-04-26T21:43:48.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.093+0000 7f9476ffd640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f946806b2d0 0x7f946806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.094+0000 7f9476ffd640 1 -- 172.21.15.40:0/1896788799 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f948008c780 con 0x55b380ec69a0 2024-04-26T21:43:48.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.094+0000 7f9485a50640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f946806b2d0 0x7f946806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.098 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:43:48.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.094+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55b380e36a20 0x55b380e117a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.094+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380e11ce0 con 0x55b380e36a20 2024-04-26T21:43:48.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.094+0000 7f948524f640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55b380e36a20 0x55b380e117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.094+0000 7f9485a50640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f946806b2d0 0x7f946806d790 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x55b380ece4c0 tx=0x7f94780092f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.094+0000 7f948524f640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55b380e36a20 0x55b380e117a0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.096+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v107'350 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.096+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380e120c0 con 0x55b380e36a20 2024-04-26T21:43:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.097+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v107'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.097+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380e12490 con 0x55b380e36a20 2024-04-26T21:43:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.098+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v107'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.100 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:43:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.098+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed85c0 con 0x55b380e36a20 2024-04-26T21:43:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.099+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v107'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.099+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed8960 con 0x55b380e36a20 2024-04-26T21:43:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.100+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v107'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.100+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed75b0 con 0x55b380e36a20 2024-04-26T21:43:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.101+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v107'355 uv355 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.102+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed79c0 con 0x55b380e36a20 2024-04-26T21:43:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.103+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v107'356 uv330 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.103+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ed7db0 con 0x55b380e36a20 2024-04-26T21:43:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.104+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v107'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.104+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7db0 con 0x55b380e36a20 2024-04-26T21:43:48.105 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:43:48.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.105+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v107'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.105+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.106+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v107'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.106+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.107+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v107'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.107+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.108+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v107'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.108+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.109+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v107'362 uv337 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.109+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.110+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v107'363 uv363 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.110+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.111 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:43:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.111+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v107'364 uv364 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.111+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.112+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v107'365 uv365 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.112+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.113+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v107'366 uv366 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.113+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed7d60 con 0x55b380e36a20 2024-04-26T21:43:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.114+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v107'367 uv367 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.114+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.115+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v107'368 uv368 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.115+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.118+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v107'369 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.118+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.118+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v107'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.119+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.120 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:43:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.119+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v107'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.119+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed9d10 con 0x55b380e36a20 2024-04-26T21:43:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.120+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v107'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.120+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed9d10 con 0x55b380e36a20 2024-04-26T21:43:48.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.122+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v107'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.122+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.123+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v107'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.123+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.124+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v107'375 uv349 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.124+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.125+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v107'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.125+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.126+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v107'377 uv377 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.127 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:43:48.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.126+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed9d10 con 0x55b380e36a20 2024-04-26T21:43:48.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.127+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v107'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.127+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed9d10 con 0x55b380e36a20 2024-04-26T21:43:48.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.128+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v107'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.128+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ed8d00 con 0x55b380e36a20 2024-04-26T21:43:48.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.129+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v107'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380e36a20 2024-04-26T21:43:48.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.129+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b380ed9210 0x55b380edb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.129+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edbb40 con 0x55b380ed9210 2024-04-26T21:43:48.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.129+0000 7f9485a50640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b380ed9210 0x55b380edb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.129+0000 7f9485a50640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b380ed9210 0x55b380edb600 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.131+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v107'262 uv245 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.131+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380edbee0 con 0x55b380ed9210 2024-04-26T21:43:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.132+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v107'263 uv263 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.132+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edc280 con 0x55b380ed9210 2024-04-26T21:43:48.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.134+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v107'264 uv264 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.135 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:43:48.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.134+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edcad0 con 0x55b380ed9210 2024-04-26T21:43:48.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.135+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v107'265 uv265 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.135+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edce70 con 0x55b380ed9210 2024-04-26T21:43:48.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.136+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v107'266 uv266 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.136+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd210 con 0x55b380ed9210 2024-04-26T21:43:48.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.138+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v107'267 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.138+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380edd5b0 con 0x55b380ed9210 2024-04-26T21:43:48.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.139+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v107'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.139+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.140+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v107'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.141 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:43:48.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.140+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.141+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v107'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.141+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.142+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v107'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.142+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.143+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v107'272 uv255 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.143+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.144+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v107'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.145+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.146+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v107'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.146 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:43:48.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.146+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee2010 con 0x55b380ed9210 2024-04-26T21:43:48.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.147+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v107'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.147+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee2010 con 0x55b380ed9210 2024-04-26T21:43:48.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.148+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v107'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.148+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.149+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v107'277 uv261 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.149+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.150+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v107'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.150+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.151+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v107'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.152 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:43:48.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.151+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ede960 con 0x55b380ed9210 2024-04-26T21:43:48.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.152+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v107'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.152+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ede960 con 0x55b380ed9210 2024-04-26T21:43:48.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.153+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v107'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.153+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380edd950 con 0x55b380ed9210 2024-04-26T21:43:48.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.154+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v107'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ed9210 2024-04-26T21:43:48.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.154+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b380edde60 0x55b380ee0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.154+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee0790 con 0x55b380edde60 2024-04-26T21:43:48.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.155+0000 7f948524f640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b380edde60 0x55b380ee0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.155+0000 7f948524f640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b380edde60 0x55b380ee0250 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.156+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v107'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.156+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee0790 con 0x55b380edde60 2024-04-26T21:43:48.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.157+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v107'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.157+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee0810 con 0x55b380edde60 2024-04-26T21:43:48.159 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:43:48.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.158+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v107'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.158+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee1c40 con 0x55b380edde60 2024-04-26T21:43:48.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.160+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v107'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.160+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee1c40 con 0x55b380edde60 2024-04-26T21:43:48.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.161+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v107'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.161+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee0c30 con 0x55b380edde60 2024-04-26T21:43:48.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.162+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v107'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.162+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b380ee1140 0x55b380ee3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.162+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee3b60 con 0x55b380ee1140 2024-04-26T21:43:48.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.162+0000 7f9486251640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b380ee1140 0x55b380ee3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.163+0000 7f9486251640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b380ee1140 0x55b380ee3530 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.164+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v107'57 uv56 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f94800565a0 con 0x55b380ee1140 2024-04-26T21:43:48.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.164+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee4070 con 0x55b380ee1140 2024-04-26T21:43:48.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.165+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v107'58 uv58 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.165+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee4570 con 0x55b380ee1140 2024-04-26T21:43:48.167 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:43:48.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.166+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v107'59 uv59 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.166+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee4e30 con 0x55b380ee1140 2024-04-26T21:43:48.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.167+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v107'60 uv60 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.167+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b380ee56e0 0x55b380ee7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.167+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8010 con 0x55b380ee56e0 2024-04-26T21:43:48.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.168+0000 7f9485a50640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b380ee56e0 0x55b380ee7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.168+0000 7f9485a50640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b380ee56e0 0x55b380ee7ad0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.169+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v107'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.169+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee80d0 con 0x55b380ee56e0 2024-04-26T21:43:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.171+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v107'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.171+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee84f0 con 0x55b380ee56e0 2024-04-26T21:43:48.173 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:43:48.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.172+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v107'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.172+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee9500 con 0x55b380ee56e0 2024-04-26T21:43:48.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.173+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v107'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.173+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee9500 con 0x55b380ee56e0 2024-04-26T21:43:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.175+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v107'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.175+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee84f0 con 0x55b380ee56e0 2024-04-26T21:43:48.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.176+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v107'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.176+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee84f0 con 0x55b380ee1140 2024-04-26T21:43:48.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.177+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v107'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.177+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee8890 con 0x55b380ee1140 2024-04-26T21:43:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.178+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v107'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.178+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8c30 con 0x55b380ee1140 2024-04-26T21:43:48.180 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:43:48.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.179+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v107'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.179+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee9fe0 con 0x55b380ee1140 2024-04-26T21:43:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.180+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v107'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.180+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee9fe0 con 0x55b380ee1140 2024-04-26T21:43:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.181+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v107'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.181+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.182+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v107'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.182+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.183+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v107'133 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.183+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.184+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v107'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.184+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.186 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:43:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.185+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v107'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.185+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.186+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v107'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.186+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.187+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v107'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.187+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.188+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v107'56 uv55 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.188+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.189+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v107'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.189+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.191 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:43:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.190+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v107'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.190+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.191+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v107'59 uv59 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.191+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee1140 2024-04-26T21:43:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.192+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v107'60 uv60 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f948005c040 con 0x55b380ee1140 2024-04-26T21:43:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.192+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.193+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v107'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.193+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.195+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v107'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.195+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.196 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:43:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.196+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v107'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.196+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.197+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v107'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.197+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.198+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v107'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.198+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.199+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v107'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.199+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.200+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v107'112 uv108 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.200+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.201+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v107'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.201+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee8fd0 con 0x55b380ee56e0 2024-04-26T21:43:48.203 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:43:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.202+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v107'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.202+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee93c0 con 0x55b380ee56e0 2024-04-26T21:43:48.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.204+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v107'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.204+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee93c0 con 0x55b380ee56e0 2024-04-26T21:43:48.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.205+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v107'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.205+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee93c0 con 0x55b380ee56e0 2024-04-26T21:43:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.206+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v107'117 uv111 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.206+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380ee93c0 con 0x55b380ee56e0 2024-04-26T21:43:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.207+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v107'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.207+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee93c0 con 0x55b380ee56e0 2024-04-26T21:43:48.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.208+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v107'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.209 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:43:48.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.208+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee93c0 con 0x55b380ee56e0 2024-04-26T21:43:48.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.209+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v107'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.209+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380ee9870 con 0x55b380ee56e0 2024-04-26T21:43:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.210+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v107'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.210+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b380ee9f30 0x55b380eec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.210+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380ee9f30 2024-04-26T21:43:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.210+0000 7f948524f640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b380ee9f30 0x55b380eec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.211+0000 7f948524f640 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b380ee9f30 0x55b380eec3b0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.212+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v107'54 uv54 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380ee9f30 2024-04-26T21:43:48.213 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:43:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.212+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380ee9f30 2024-04-26T21:43:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.213+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v107'55 uv55 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380ee9f30 2024-04-26T21:43:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.213+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380ee56e0 2024-04-26T21:43:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.214+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(103 609.00000000 [create] v107'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.214+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380ee56e0 2024-04-26T21:43:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.216+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(104 609.00000000 [delete] v107'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.216+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380ee56e0 2024-04-26T21:43:48.217 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:43:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.217+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(105 609.00000000 [create] v107'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.217+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eed900 con 0x55b380ee56e0 2024-04-26T21:43:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.219+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v107'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.219+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eed900 con 0x55b380ee56e0 2024-04-26T21:43:48.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.220+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v107'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.220+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380ee56e0 2024-04-26T21:43:48.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.222+0000 7f9485a50640 1 -- 172.21.15.40:0/1896788799 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(108 609.00000000 [omap-set-header] v107'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9478011c40 con 0x55b380ee56e0 2024-04-26T21:43:48.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.222+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.223+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(109 400.00000000 [create] v107'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.223+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.224+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(110 400.00000000 [delete] v107'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.224+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.226 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:43:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.225+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(111 400.00000000 [create] v107'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.225+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.226+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(112 400.00000000 [write 0~22] v107'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.227+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.228+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(113 400.00000000 [omap-set-header] v107'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.228+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.229+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(114 100.00000000.inode [create] v107'144 uv136 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.229+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.230+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(115 100.00000000.inode [delete] v107'145 uv145 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.230+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.231+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(116 100.00000000.inode [create] v107'146 uv146 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.232 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:43:48.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.231+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.232+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v107'147 uv147 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.232+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.233+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v107'148 uv148 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.233+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.234+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(119 1.00000000 [create] v107'149 uv143 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.234+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.235+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(120 1.00000000 [delete] v107'150 uv150 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.235+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.236+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(121 1.00000000 [create] v107'151 uv151 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.236 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:43:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.236+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eed900 con 0x55b380edde60 2024-04-26T21:43:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.237+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v107'152 uv152 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.237+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eed900 con 0x55b380edde60 2024-04-26T21:43:48.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.238+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v107'153 uv153 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.238+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eec8f0 con 0x55b380edde60 2024-04-26T21:43:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.239+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(124 1.00000000 [omap-set-header] v107'154 uv154 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.239+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b380eecda0 con 0x55b380edde60 2024-04-26T21:43:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f948524f640 1 -- 172.21.15.40:0/1896788799 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v107'155 uv155 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f947c0508c0 con 0x55b380edde60 2024-04-26T21:43:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b380ed9210 msgr2=0x55b380edb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55b380ed9210 0x55b380edb600 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55b380e36a20 msgr2=0x55b380e117a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55b380e36a20 0x55b380e117a0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b380ee1140 msgr2=0x55b380ee3530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b380ee1140 0x55b380ee3530 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b380ee9f30 msgr2=0x55b380eec3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b380ee9f30 0x55b380eec3b0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 reap_dead start 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9486251640 1 -- 172.21.15.40:0/1896788799 reap_dead start 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b380edde60 msgr2=0x55b380ee0250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b380edde60 0x55b380ee0250 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b380ee56e0 msgr2=0x55b380ee7ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b380ee56e0 0x55b380ee7ad0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f946806b2d0 msgr2=0x7f946806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f946806b2d0 0x7f946806d790 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x55b380ece4c0 tx=0x7f94780092f0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 msgr2=0x55b380d8a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380d8a720 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f9480007010 tx=0x7f9480003330 comp rx=0 tx=0).stop 2024-04-26T21:43:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 shutdown_connections 2024-04-26T21:43:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b380ee56e0 0x55b380ee7ad0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b380edde60 0x55b380ee0250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f946806b2d0 0x7f946806d790 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 --2- 172.21.15.40:0/1896788799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b380ec69a0 0x55b380d8a720 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 >> 172.21.15.40:0/1896788799 conn(0x55b380d5aab0 msgr2=0x55b380d59f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 shutdown_connections 2024-04-26T21:43:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.240+0000 7f9487f56f40 1 -- 172.21.15.40:0/1896788799 wait complete. 2024-04-26T21:43:48.244 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 400.00000000 2024-04-26T21:43:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.326+0000 7fe870830f40 1 Processor -- start 2024-04-26T21:43:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.326+0000 7fe870830f40 1 -- start start 2024-04-26T21:43:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.326+0000 7fe870830f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e56fd30 0x55a69e63b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.326+0000 7fe870830f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e59f6d0 0x55a69e572920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.326+0000 7fe870830f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e5a02e0 0x55a69e6494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.326+0000 7fe870830f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a69e516e40 con 0x55a69e59f6d0 2024-04-26T21:43:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.326+0000 7fe870830f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a69e51ae80 con 0x55a69e5a02e0 2024-04-26T21:43:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.326+0000 7fe870830f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a69e56f250 con 0x55a69e56fd30 2024-04-26T21:43:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e59f6d0 0x55a69e572920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e59f6d0 0x55a69e572920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56016/0 (socket says 172.21.15.40:56016) 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 -- 172.21.15.40:0/3686114067 learned_addr learned my addr 172.21.15.40:0/3686114067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe86e32a640 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e56fd30 0x55a69e63b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe86eb2b640 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e5a02e0 0x55a69e6494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 -- 172.21.15.40:0/3686114067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e56fd30 msgr2=0x55a69e63b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e56fd30 0x55a69e63b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 -- 172.21.15.40:0/3686114067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e5a02e0 msgr2=0x55a69e6494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e5a02e0 0x55a69e6494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 -- 172.21.15.40:0/3686114067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69e646100 con 0x55a69e59f6d0 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe867fff640 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e59f6d0 0x55a69e572920 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe858025b00 tx=0x7fe85804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b295c7f044b1dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe86d328640 1 -- 172.21.15.40:0/3686114067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858059040 con 0x55a69e59f6d0 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe86d328640 1 -- 172.21.15.40:0/3686114067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe858064070 con 0x55a69e59f6d0 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.327+0000 7fe86d328640 1 -- 172.21.15.40:0/3686114067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85800a300 con 0x55a69e59f6d0 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 -- 172.21.15.40:0/3686114067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e59f6d0 msgr2=0x55a69e572920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e59f6d0 0x55a69e572920 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe858025b00 tx=0x7fe85804e000 comp rx=0 tx=0).stop 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 -- 172.21.15.40:0/3686114067 shutdown_connections 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e5a02e0 0x55a69e6494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e59f6d0 0x55a69e572920 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 --2- 172.21.15.40:0/3686114067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e56fd30 0x55a69e63b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 -- 172.21.15.40:0/3686114067 >> 172.21.15.40:0/3686114067 conn(0x55a69e59a9b0 msgr2=0x55a69e645950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 -- 172.21.15.40:0/3686114067 shutdown_connections 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 -- 172.21.15.40:0/3686114067 wait complete. 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 Processor -- start 2024-04-26T21:43:48.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.328+0000 7fe870830f40 1 -- start start 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe870830f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e56fd30 0x55a69e63ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe870830f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e59f6d0 0x55a69e637b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe870830f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e5a02e0 0x55a69e6368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe870830f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a69e570390 con 0x55a69e56fd30 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe870830f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a69e63bfb0 con 0x55a69e59f6d0 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe870830f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a69e63bcd0 con 0x55a69e5a02e0 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86e32a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e56fd30 0x55a69e63ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86e32a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e56fd30 0x55a69e63ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56022/0 (socket says 172.21.15.40:56022) 2024-04-26T21:43:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86e32a640 1 -- 172.21.15.40:0/3572175697 learned_addr learned my addr 172.21.15.40:0/3572175697 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe867fff640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e59f6d0 0x55a69e637b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86eb2b640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e5a02e0 0x55a69e6368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86e32a640 1 -- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e5a02e0 msgr2=0x55a69e6368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86e32a640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a69e5a02e0 0x55a69e6368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86e32a640 1 -- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e59f6d0 msgr2=0x55a69e637b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86e32a640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a69e59f6d0 0x55a69e637b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.329+0000 7fe86e32a640 1 -- 172.21.15.40:0/3572175697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69e643450 con 0x55a69e56fd30 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.330+0000 7fe86e32a640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e56fd30 0x55a69e63ca70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fe86000efb0 tx=0x7fe86000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.330+0000 7fe8677fe640 1 -- 172.21.15.40:0/3572175697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860002bf0 con 0x55a69e56fd30 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.330+0000 7fe8677fe640 1 -- 172.21.15.40:0/3572175697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe860002d90 con 0x55a69e56fd30 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.330+0000 7fe8677fe640 1 -- 172.21.15.40:0/3572175697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860001c30 con 0x55a69e56fd30 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.330+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a69e63a750 con 0x55a69e56fd30 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.330+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69e636df0 con 0x55a69e56fd30 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.331+0000 7fe8677fe640 1 -- 172.21.15.40:0/3572175697 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe860002050 con 0x55a69e56fd30 2024-04-26T21:43:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.332+0000 7fe8677fe640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe85006b2d0 0x7fe85006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.332+0000 7fe8677fe640 1 -- 172.21.15.40:0/3572175697 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7fe86008ad00 con 0x55a69e56fd30 2024-04-26T21:43:48.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.332+0000 7fe867fff640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe85006b2d0 0x7fe85006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.332+0000 7fe870830f40 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a69e63b790 0x55a69e648480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.332+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55a69e63ac00 con 0x55a69e63b790 2024-04-26T21:43:48.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.332+0000 7fe86eb2b640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a69e63b790 0x55a69e648480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.332+0000 7fe867fff640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe85006b2d0 0x7fe85006d790 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe858057040 tx=0x7fe858051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.332+0000 7fe86eb2b640 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a69e63b790 0x55a69e648480 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe86eb2b640 1 -- 172.21.15.40:0/3572175697 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 400.00000000 [delete] v107'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8680058d0 con 0x55a69e63b790 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a69e63b790 msgr2=0x55a69e648480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a69e63b790 0x55a69e648480 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe85006b2d0 msgr2=0x7fe85006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe85006b2d0 0x7fe85006d790 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe858057040 tx=0x7fe858051000 comp rx=0 tx=0).stop 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e56fd30 msgr2=0x55a69e63ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 --2- 172.21.15.40:0/3572175697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a69e56fd30 0x55a69e63ca70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fe86000efb0 tx=0x7fe86000c620 comp rx=0 tx=0).stop 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe86eb2b640 1 -- 172.21.15.40:0/3572175697 reap_dead start 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 shutdown_connections 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 >> 172.21.15.40:0/3572175697 conn(0x55a69e59a9b0 msgr2=0x55a69e642a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 shutdown_connections 2024-04-26T21:43:48.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.334+0000 7fe870830f40 1 -- 172.21.15.40:0/3572175697 wait complete. 2024-04-26T21:43:48.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:43:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.481+0000 7f9961b9d640 1 Processor -- start 2024-04-26T21:43:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.481+0000 7f9961b9d640 1 -- start start 2024-04-26T21:43:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f9961b9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f995c0b4360 0x7f995c0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f9961b9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995c0ce5e0 con 0x7f995c0d1240 2024-04-26T21:43:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f9961b9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995c0ce8c0 con 0x7f995c0d0d60 2024-04-26T21:43:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f9961b9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f995c0cf210 con 0x7f995c0b4360 2024-04-26T21:43:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f995affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f995c0b4360 0x7f995c0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f995b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f995c0d1240 0x7f995c0d1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37274/0 (socket says 172.21.15.40:37274) 2024-04-26T21:43:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f995b7fe640 1 -- 172.21.15.40:0/770260218 learned_addr learned my addr 172.21.15.40:0/770260218 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f995a7fc640 1 -- 172.21.15.40:0/770260218 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317532901 0 0) 0x7f995c0ce5e0 con 0x7f995c0d1240 2024-04-26T21:43:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f995a7fc640 1 -- 172.21.15.40:0/770260218 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9944003520 con 0x7f995c0d1240 2024-04-26T21:43:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f995affd640 1 -- 172.21.15.40:0/770260218 >> v1:172.21.15.184:6789/0 conn(0x7f995c0d0d60 legacy=0x7f995c0b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f995affd640 1 -- 172.21.15.40:0/770260218 >> v1:172.21.15.40:6789/0 conn(0x7f995c0d1240 legacy=0x7f995c0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.482+0000 7f995affd640 1 -- 172.21.15.40:0/770260218 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c160ab0 con 0x7f995c0b4360 2024-04-26T21:43:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.483+0000 7f995affd640 1 --2- 172.21.15.40:0/770260218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0b4360 0x7f995c0b2600 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f99500097a0 tx=0x7f995002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9321a2aabdf8e8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.483+0000 7f995a7fc640 1 -- 172.21.15.40:0/770260218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950037070 con 0x7f995c0b4360 2024-04-26T21:43:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.483+0000 7f995a7fc640 1 -- 172.21.15.40:0/770260218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995002ae50 con 0x7f995c0b4360 2024-04-26T21:43:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.483+0000 7f995a7fc640 1 -- 172.21.15.40:0/770260218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950032660 con 0x7f995c0b4360 2024-04-26T21:43:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.484+0000 7f9961b9d640 1 -- 172.21.15.40:0/770260218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0b4360 msgr2=0x7f995c0b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.484+0000 7f9961b9d640 1 --2- 172.21.15.40:0/770260218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0b4360 0x7f995c0b2600 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f99500097a0 tx=0x7f995002a140 comp rx=0 tx=0).stop 2024-04-26T21:43:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.484+0000 7f9961b9d640 1 -- 172.21.15.40:0/770260218 shutdown_connections 2024-04-26T21:43:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.484+0000 7f9961b9d640 1 --2- 172.21.15.40:0/770260218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0b4360 0x7f995c0b2600 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.484+0000 7f9961b9d640 1 -- 172.21.15.40:0/770260218 >> 172.21.15.40:0/770260218 conn(0x7f995c0d2e60 msgr2=0x7f995c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.484+0000 7f9961b9d640 1 -- 172.21.15.40:0/770260218 shutdown_connections 2024-04-26T21:43:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.484+0000 7f9961b9d640 1 -- 172.21.15.40:0/770260218 wait complete. 2024-04-26T21:43:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.485+0000 7f9961b9d640 1 Processor -- start 2024-04-26T21:43:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.485+0000 7f9961b9d640 1 -- start start 2024-04-26T21:43:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f9961b9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0d0d60 0x7f995c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f9961b9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f995c0d1240 0x7f995c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f9961b9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f995c0b8760 0x7f995c0b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f9961b9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f995c0c6790 con 0x7f995c0d0d60 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f9961b9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f995c0c6610 con 0x7f995c0d1240 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f9961b9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f995c0b3250 con 0x7f995c0b8760 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0d0d60 0x7f995c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0d0d60 0x7f995c0c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56048/0 (socket says 172.21.15.40:56048) 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 -- 172.21.15.40:0/1898908105 learned_addr learned my addr 172.21.15.40:0/1898908105 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995affd640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f995c0d1240 0x7f995c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995bfff640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f995c0b8760 0x7f995c0b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 -- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f995c0b8760 msgr2=0x7f995c0b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f995c0b8760 0x7f995c0b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 -- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f995c0d1240 msgr2=0x7f995c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f995c0d1240 0x7f995c06dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 -- 172.21.15.40:0/1898908105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c06e640 con 0x7f995c0d0d60 2024-04-26T21:43:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995affd640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f995c0d1240 0x7f995c06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995bfff640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f995c0b8760 0x7f995c0b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f995b7fe640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0d0d60 0x7f995c0c92d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f994c00c390 tx=0x7f994c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.486+0000 7f9943fff640 1 -- 172.21.15.40:0/1898908105 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c017070 con 0x7f995c0d0d60 2024-04-26T21:43:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.487+0000 7f9943fff640 1 -- 172.21.15.40:0/1898908105 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994c005ce0 con 0x7f995c0d0d60 2024-04-26T21:43:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.487+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c06e920 con 0x7f995c0d0d60 2024-04-26T21:43:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.487+0000 7f9943fff640 1 -- 172.21.15.40:0/1898908105 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c0072f0 con 0x7f995c0d0d60 2024-04-26T21:43:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.487+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c0c7160 con 0x7f995c0d0d60 2024-04-26T21:43:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.487+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995c0694f0 con 0x7f995c0d0d60 2024-04-26T21:43:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.488+0000 7f9943fff640 1 -- 172.21.15.40:0/1898908105 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f994c01a070 con 0x7f995c0d0d60 2024-04-26T21:43:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.488+0000 7f9943fff640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f994806c8f0 0x7f994806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.489+0000 7f9943fff640 1 -- 172.21.15.40:0/1898908105 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f994c08b580 con 0x7f995c0d0d60 2024-04-26T21:43:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.489+0000 7f995affd640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f994806c8f0 0x7f994806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.489+0000 7f995affd640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f994806c8f0 0x7f994806edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9950034000 tx=0x7f995002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.493+0000 7f9943fff640 1 -- 172.21.15.40:0/1898908105 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f994c0593a0 con 0x7f995c0d0d60 2024-04-26T21:43:48.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:48.720+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f995c0ce1e0 con 0x7f995c0d0d60 2024-04-26T21:43:49.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.035+0000 7f9943fff640 1 -- 172.21.15.40:0/1898908105 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v165) ==== 162+0+0 (secure 0 0 0) 0x7f994c058ca0 con 0x7f995c0d0d60 2024-04-26T21:43:49.036 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:43:49.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f994806c8f0 msgr2=0x7f994806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f994806c8f0 0x7f994806edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9950034000 tx=0x7f995002e000 comp rx=0 tx=0).stop 2024-04-26T21:43:49.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0d0d60 msgr2=0x7f995c0c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0d0d60 0x7f995c0c92d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f994c00c390 tx=0x7f994c00c850 comp rx=0 tx=0).stop 2024-04-26T21:43:49.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 shutdown_connections 2024-04-26T21:43:49.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f994806c8f0 0x7f994806edb0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f995c0b8760 0x7f995c0b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f995c0d1240 0x7f995c06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 --2- 172.21.15.40:0/1898908105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c0d0d60 0x7f995c0c92d0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 >> 172.21.15.40:0/1898908105 conn(0x7f995c0d2e60 msgr2=0x7f995c0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:49.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 shutdown_connections 2024-04-26T21:43:49.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.037+0000 7f9961b9d640 1 -- 172.21.15.40:0/1898908105 wait complete. 2024-04-26T21:43:49.453 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.564+0000 7fbaf1d1c640 1 Processor -- start 2024-04-26T21:43:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.564+0000 7fbaf1d1c640 1 -- start start 2024-04-26T21:43:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaf1d1c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0b2340 0x7fbaec0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaf1d1c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaec0d10d0 con 0x7fbaec162810 2024-04-26T21:43:49.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaf1d1c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaec0d1370 con 0x7fbaec0cd930 2024-04-26T21:43:49.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaf1d1c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec0d0750 con 0x7fbaec0b2340 2024-04-26T21:43:49.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaeb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbaec162810 0x7fbaec070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42568/0 (socket says 172.21.15.40:42568) 2024-04-26T21:43:49.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaeb7fe640 1 -- 172.21.15.40:0/2358890065 learned_addr learned my addr 172.21.15.40:0/2358890065 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:49.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaeaffd640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0b2340 0x7fbaec0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796740338 0 0) 0x7fbaec0d10d0 con 0x7fbaec162810 2024-04-26T21:43:49.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.565+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbad4003520 con 0x7fbaec162810 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 968289344 0 0) 0x7fbad4003520 con 0x7fbaec162810 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0b2340 msgr2=0x7fbaec0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0b2340 0x7fbaec0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 >> v1:172.21.15.184:6789/0 conn(0x7fbaec0cd930 legacy=0x7fbaec0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec0d01b0 con 0x7fbaec162810 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fbae0001670 con 0x7fbaec162810 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 >> v1:172.21.15.40:6789/0 conn(0x7fbaec162810 legacy=0x7fbaec070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0b2340 0x7fbad40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad4003f40 0x7fbad4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad4004d40 0x7fbad40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0d0750 con 0x7fbad4004d40 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbad400ba70 con 0x7fbad4003f40 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbad400bc40 con 0x7fbaec0b2340 2024-04-26T21:43:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaebfff640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad4004d40 0x7fbad40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaeb7fe640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0b2340 0x7fbad40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaeaffd640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad4003f40 0x7fbad4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaebfff640 1 -- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0b2340 msgr2=0x7fbad40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaebfff640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0b2340 0x7fbad40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaebfff640 1 -- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad4003f40 msgr2=0x7fbad4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaebfff640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbad4003f40 0x7fbad4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.566+0000 7fbaebfff640 1 -- 172.21.15.40:0/2358890065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad4003520 con 0x7fbad4004d40 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.567+0000 7fbaebfff640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad4004d40 0x7fbad40051c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbae4001f30 tx=0x7fbad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee68ec389b4026b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.567+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4037070 con 0x7fbad4004d40 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.567+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4032440 con 0x7fbad4004d40 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.567+0000 7fbaea7fc640 1 -- 172.21.15.40:0/2358890065 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4001b30 con 0x7fbad4004d40 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.568+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2358890065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad4004d40 msgr2=0x7fbad40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.568+0000 7fbaf1d1c640 1 --2- 172.21.15.40:0/2358890065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad4004d40 0x7fbad40051c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbae4001f30 tx=0x7fbad4005830 comp rx=0 tx=0).stop 2024-04-26T21:43:49.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.568+0000 7fbaebfff640 1 -- 172.21.15.40:0/2358890065 reap_dead start 2024-04-26T21:43:49.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.568+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2358890065 shutdown_connections 2024-04-26T21:43:49.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.568+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2358890065 >> 172.21.15.40:0/2358890065 conn(0x7fbaec0d2da0 msgr2=0x7fbaec0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:49.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.568+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2358890065 shutdown_connections 2024-04-26T21:43:49.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.568+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2358890065 wait complete. 2024-04-26T21:43:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.569+0000 7fbaf1d1c640 1 Processor -- start 2024-04-26T21:43:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.569+0000 7fbaf1d1c640 1 -- start start 2024-04-26T21:43:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaf1d1c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaec06d350 0x7fbaec06d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaf1d1c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0ca880 0x7fbaec06dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaf1d1c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaec06e1b0 0x7fbaec158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaf1d1c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0d2240 con 0x7fbaec06d350 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaf1d1c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0d16a0 con 0x7fbaec06e1b0 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaf1d1c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec0d23c0 con 0x7fbaec0ca880 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaec06d350 0x7fbaec06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaec06d350 0x7fbaec06d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51244/0 (socket says 172.21.15.40:51244) 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 -- 172.21.15.40:0/2786898238 learned_addr learned my addr 172.21.15.40:0/2786898238 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaebfff640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaec06e1b0 0x7fbaec158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeaffd640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0ca880 0x7fbaec06dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 -- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0ca880 msgr2=0x7fbaec06dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0ca880 0x7fbaec06dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 -- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaec06e1b0 msgr2=0x7fbaec158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaec06e1b0 0x7fbaec158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 -- 172.21.15.40:0/2786898238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec06ebb0 con 0x7fbaec06d350 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.570+0000 7fbaeb7fe640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaec06d350 0x7fbaec06d730 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbae0007320 tx=0x7fbae00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.571+0000 7fbad3fff640 1 -- 172.21.15.40:0/2786898238 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae001a070 con 0x7fbaec06d350 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.571+0000 7fbad3fff640 1 -- 172.21.15.40:0/2786898238 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae000f920 con 0x7fbaec06d350 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.571+0000 7fbad3fff640 1 -- 172.21.15.40:0/2786898238 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae000fd10 con 0x7fbaec06d350 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.571+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec1583a0 con 0x7fbaec06d350 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.571+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1548c0 con 0x7fbaec06d350 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.572+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec0694f0 con 0x7fbaec06d350 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.572+0000 7fbad3fff640 1 -- 172.21.15.40:0/2786898238 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbae001d070 con 0x7fbaec06d350 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.572+0000 7fbad3fff640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbad806c8f0 0x7fbad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.573+0000 7fbad3fff640 1 -- 172.21.15.40:0/2786898238 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7fbae008bc50 con 0x7fbaec06d350 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.573+0000 7fbaeaffd640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbad806c8f0 0x7fbad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.573+0000 7fbaeaffd640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbad806c8f0 0x7fbad806edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0b8b60 tx=0x7fbadc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:49.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.577+0000 7fbad3fff640 1 -- 172.21.15.40:0/2786898238 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbae0091020 con 0x7fbaec06d350 2024-04-26T21:43:49.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.803+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaec0b1ce0 con 0x7fbaec06d350 2024-04-26T21:43:49.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.804+0000 7fbad3fff640 1 -- 172.21.15.40:0/2786898238 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) ==== 95+0+5190 (secure 0 0 0) 0x7fbae0059a70 con 0x7fbaec06d350 2024-04-26T21:43:49.805 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:49.805 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":166,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:49.039149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9240":{"gid":9240,"name":"c","rank":0,"incarnation":166,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/1803384446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1803384446},{"type":"v1","addr":"172.21.15.40:6833","nonce":1803384446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:43:49.805 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 166 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbad806c8f0 msgr2=0x7fbad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbad806c8f0 0x7fbad806edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0b8b60 tx=0x7fbadc008040 comp rx=0 tx=0).stop 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaec06d350 msgr2=0x7fbaec06d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaec06d350 0x7fbaec06d730 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbae0007320 tx=0x7fbae00077e0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 shutdown_connections 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbad806c8f0 0x7fbad806edb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaec06e1b0 0x7fbaec158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbaec0ca880 0x7fbaec06dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 --2- 172.21.15.40:0/2786898238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaec06d350 0x7fbaec06d730 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 >> 172.21.15.40:0/2786898238 conn(0x7fbaec0d2da0 msgr2=0x7fbaec0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:49.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 shutdown_connections 2024-04-26T21:43:49.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:49.806+0000 7fbaf1d1c640 1 -- 172.21.15.40:0/2786898238 wait complete. 2024-04-26T21:43:49.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 166, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:43:49.039149+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 107, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9240}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9240': {'gid': 9240, 'name': 'c', 'rank': 0, 'incarnation': 166, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/1803384446', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1803384446}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 1803384446}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:43:49.818 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9240:up:replay 2024-04-26T21:43:49.818 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:43:55.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:43:55.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.335+0000 7f03e9052640 1 Processor -- start 2024-04-26T21:43:55.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.335+0000 7f03e9052640 1 -- start start 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e9052640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e4162810 0x7f03e4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e9052640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e40d10d0 con 0x7f03e40d1b30 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e9052640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e40d1370 con 0x7f03e40b2340 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e9052640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03e40d0750 con 0x7f03e4162810 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e359e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f03e40d1b30 0x7f03e4070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42576/0 (socket says 172.21.15.40:42576) 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e359e640 1 -- 172.21.15.40:0/990611463 learned_addr learned my addr 172.21.15.40:0/990611463 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e2d9d640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e4162810 0x7f03e4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743218374 0 0) 0x7f03e40d10d0 con 0x7f03e40d1b30 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.336+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03cc003520 con 0x7f03e40d1b30 2024-04-26T21:43:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1245000058 0 0) 0x7f03cc003520 con 0x7f03e40d1b30 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e4162810 msgr2=0x7f03e4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e4162810 0x7f03e4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 >> v1:172.21.15.184:6789/0 conn(0x7f03e40b2340 legacy=0x7f03e40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e40d01b0 con 0x7f03e40d1b30 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f03dc0019f0 con 0x7f03e40d1b30 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 >> v1:172.21.15.40:6789/0 conn(0x7f03e40d1b30 legacy=0x7f03e4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc0039e0 0x7f03cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03cc004800 0x7f03cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e4162810 0x7f03cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40d0750 con 0x7f03cc004800 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc00ba70 con 0x7f03cc0039e0 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03cc00bc70 con 0x7f03e4162810 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e259c640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03cc004800 0x7f03cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e2d9d640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc0039e0 0x7f03cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e359e640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e4162810 0x7f03cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e259c640 1 -- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e4162810 msgr2=0x7f03cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e259c640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e4162810 0x7f03cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e259c640 1 -- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc0039e0 msgr2=0x7f03cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e259c640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc0039e0 0x7f03cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.337+0000 7f03e259c640 1 -- 172.21.15.40:0/990611463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc003520 con 0x7f03cc004800 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.338+0000 7f03e259c640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03cc004800 0x7f03cc004c80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f03d8001720 tx=0x7f03cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c6e39c09f84ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.338+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8001370 con 0x7f03cc004800 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.338+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d80052d0 con 0x7f03cc004800 2024-04-26T21:43:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.338+0000 7f03e1d9b640 1 -- 172.21.15.40:0/990611463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d802ab90 con 0x7f03cc004800 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.339+0000 7f03e9052640 1 -- 172.21.15.40:0/990611463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03cc004800 msgr2=0x7f03cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.339+0000 7f03e9052640 1 --2- 172.21.15.40:0/990611463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03cc004800 0x7f03cc004c80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f03d8001720 tx=0x7f03cc005830 comp rx=0 tx=0).stop 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.339+0000 7f03e359e640 1 -- 172.21.15.40:0/990611463 reap_dead start 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.339+0000 7f03e9052640 1 -- 172.21.15.40:0/990611463 shutdown_connections 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.339+0000 7f03e9052640 1 -- 172.21.15.40:0/990611463 >> 172.21.15.40:0/990611463 conn(0x7f03e40d2da0 msgr2=0x7f03e40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.339+0000 7f03e9052640 1 -- 172.21.15.40:0/990611463 shutdown_connections 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.339+0000 7f03e9052640 1 -- 172.21.15.40:0/990611463 wait complete. 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e9052640 1 Processor -- start 2024-04-26T21:43:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e9052640 1 -- start start 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e9052640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc004800 0x7f03e40c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e9052640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e415a340 0x7f03e415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e9052640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e415f290 0x7f03e4161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e9052640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40cdb00 con 0x7f03e415f290 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e9052640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40d16a0 con 0x7f03cc004800 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e9052640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03e4071260 con 0x7f03e415a340 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.341+0000 7f03e359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e415f290 0x7f03e4161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e415f290 0x7f03e4161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51268/0 (socket says 172.21.15.40:51268) 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e359e640 1 -- 172.21.15.40:0/3013761958 learned_addr learned my addr 172.21.15.40:0/3013761958 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e2d9d640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc004800 0x7f03e40c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e259c640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e415a340 0x7f03e415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e359e640 1 -- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e415a340 msgr2=0x7f03e415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e359e640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e415a340 0x7f03e415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e359e640 1 -- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc004800 msgr2=0x7f03e40c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e359e640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc004800 0x7f03e40c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e359e640 1 -- 172.21.15.40:0/3013761958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e4161e20 con 0x7f03e415f290 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e359e640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e415f290 0x7f03e4161060 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f03e4073090 tx=0x7f03dc007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03cb7fe640 1 -- 172.21.15.40:0/3013761958 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00ac20 con 0x7f03e415f290 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03cb7fe640 1 -- 172.21.15.40:0/3013761958 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc018070 con 0x7f03e415f290 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e406d790 con 0x7f03e415f290 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03cb7fe640 1 -- 172.21.15.40:0/3013761958 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0133f0 con 0x7f03e415f290 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.342+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e406dd50 con 0x7f03e415f290 2024-04-26T21:43:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.343+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e40694f0 con 0x7f03e415f290 2024-04-26T21:43:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.344+0000 7f03cb7fe640 1 -- 172.21.15.40:0/3013761958 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03dc013590 con 0x7f03e415f290 2024-04-26T21:43:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.344+0000 7f03cb7fe640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03d006c8f0 0x7f03d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.344+0000 7f03cb7fe640 1 -- 172.21.15.40:0/3013761958 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f03dc08c620 con 0x7f03e415f290 2024-04-26T21:43:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.344+0000 7f03e2d9d640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03d006c8f0 0x7f03d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.345+0000 7f03e2d9d640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03d006c8f0 0x7f03d006edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f03e415ead0 tx=0x7f03d4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.349+0000 7f03cb7fe640 1 -- 172.21.15.40:0/3013761958 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03dc05a4d0 con 0x7f03e415f290 2024-04-26T21:43:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.572+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e406df20 con 0x7f03e415f290 2024-04-26T21:43:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.573+0000 7f03cb7fe640 1 -- 172.21.15.40:0/3013761958 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) ==== 95+0+5197 (secure 0 0 0) 0x7f03dc059dd0 con 0x7f03e415f290 2024-04-26T21:43:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:55.574 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":169,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:43:52.049491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9240":{"gid":9240,"name":"c","rank":0,"incarnation":166,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/1803384446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1803384446},{"type":"v1","addr":"172.21.15.40:6833","nonce":1803384446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9240,"qdb_cluster":[9240]},"id":6}]} 2024-04-26T21:43:55.574 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 169 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03d006c8f0 msgr2=0x7f03d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03d006c8f0 0x7f03d006edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f03e415ead0 tx=0x7f03d4009190 comp rx=0 tx=0).stop 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e415f290 msgr2=0x7f03e4161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e415f290 0x7f03e4161060 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f03e4073090 tx=0x7f03dc007b10 comp rx=0 tx=0).stop 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 shutdown_connections 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f03d006c8f0 0x7f03d006edb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e415f290 0x7f03e4161060 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03e415a340 0x7f03e415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 --2- 172.21.15.40:0/3013761958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03cc004800 0x7f03e40c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 >> 172.21.15.40:0/3013761958 conn(0x7f03e40d2da0 msgr2=0x7f03e40d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 shutdown_connections 2024-04-26T21:43:55.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.575+0000 7f03e9052640 1 -- 172.21.15.40:0/3013761958 wait complete. 2024-04-26T21:43:55.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 169, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:43:52.049491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 107, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9240}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9240': {'gid': 9240, 'name': 'c', 'rank': 0, 'incarnation': 166, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6833/1803384446', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1803384446}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 1803384446}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9240, 'qdb_cluster': [9240]} 2024-04-26T21:43:55.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:43:55.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:43:55.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.687+0000 7ff1e0daf640 1 Processor -- start 2024-04-26T21:43:55.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.687+0000 7ff1e0daf640 1 -- start start 2024-04-26T21:43:55.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1e0daf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc162880 0x7ff1dc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1e0daf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff1dc0b5f20 0x7ff1dc0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1e0daf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1dc0d39f0 0x7ff1dc0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1e0daf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff1dc0d0580 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1e0daf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff1dc0b7e50 con 0x7ff1dc0b5f20 2024-04-26T21:43:55.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1e0daf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc0d1800 con 0x7ff1dc162880 2024-04-26T21:43:55.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1dad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1dc0d39f0 0x7ff1dc0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1dad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1dc0d39f0 0x7ff1dc0b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51276/0 (socket says 172.21.15.40:51276) 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1dad9d640 1 -- 172.21.15.40:0/3341734278 learned_addr learned my addr 172.21.15.40:0/3341734278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1da59c640 1 --2- 172.21.15.40:0/3341734278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc162880 0x7ff1dc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/3341734278 >> v2:172.21.15.184:3300/0 conn(0x7ff1dc0b5f20 0x7ff1dc0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1dad9d640 1 -- 172.21.15.40:0/3341734278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc162880 msgr2=0x7ff1dc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/3341734278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc162880 0x7ff1dc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1dad9d640 1 -- 172.21.15.40:0/3341734278 >> v2:172.21.15.184:3300/0 conn(0x7ff1dc0b5f20 msgr2=0x7ff1dc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/3341734278 >> v2:172.21.15.184:3300/0 conn(0x7ff1dc0b5f20 0x7ff1dc0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.688+0000 7ff1dad9d640 1 -- 172.21.15.40:0/3341734278 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc0d07d0 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.689+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/3341734278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc0b69e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc08e960 tx=0x7ff1d402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4ae9276ef7ff22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.689+0000 7ff1d9d9b640 1 -- 172.21.15.40:0/3341734278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4036070 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.689+0000 7ff1d9d9b640 1 -- 172.21.15.40:0/3341734278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d4031600 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.689+0000 7ff1d9d9b640 1 -- 172.21.15.40:0/3341734278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40319d0 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3341734278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 msgr2=0x7ff1dc0b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3341734278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc0b69e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc08e960 tx=0x7ff1d402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3341734278 shutdown_connections 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3341734278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc0b69e0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3341734278 >> v2:172.21.15.184:3300/0 conn(0x7ff1dc0b5f20 0x7ff1dc0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3341734278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc162880 0x7ff1dc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3341734278 >> 172.21.15.40:0/3341734278 conn(0x7ff1dc0d4f60 msgr2=0x7ff1dc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3341734278 shutdown_connections 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.690+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3341734278 wait complete. 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.691+0000 7ff1e0daf640 1 Processor -- start 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.691+0000 7ff1e0daf640 1 -- start start 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1e0daf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc0b5f20 0x7ff1dc0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1e0daf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc15c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1e0daf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1dc162880 0x7ff1dc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1e0daf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc162c60 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1e0daf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc0d0d90 con 0x7ff1dc162880 2024-04-26T21:43:55.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1e0daf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc162de0 con 0x7ff1dc0b5f20 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc15c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51280/0 (socket says 172.21.15.40:51280) 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 -- 172.21.15.40:0/3235652686 learned_addr learned my addr 172.21.15.40:0/3235652686 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1da59c640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc0b5f20 0x7ff1dc0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1dc162880 0x7ff1dc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 -- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc0b5f20 msgr2=0x7ff1dc0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc0b5f20 0x7ff1dc0b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 -- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1dc162880 msgr2=0x7ff1dc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1dc162880 0x7ff1dc0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 -- 172.21.15.40:0/3235652686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc0cc120 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1da59c640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc0b5f20 0x7ff1dc0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1dc162880 0x7ff1dc0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.692+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc15c770 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400d800 tx=0x7ff1c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.693+0000 7ff1d2ffd640 1 -- 172.21.15.40:0/3235652686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4018030 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.693+0000 7ff1d2ffd640 1 -- 172.21.15.40:0/3235652686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c40096a0 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.693+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3235652686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc0cc400 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.693+0000 7ff1d2ffd640 1 -- 172.21.15.40:0/3235652686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4009870 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.693+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3235652686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc0b2090 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.694+0000 7ff1d2ffd640 1 -- 172.21.15.40:0/3235652686 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1c401b070 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.695+0000 7ff1d2ffd640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff1bc06c920 0x7ff1bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.695+0000 7ff1d2ffd640 1 -- 172.21.15.40:0/3235652686 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7ff1c408c3c0 con 0x7ff1dc0d39f0 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.695+0000 7ff1da59c640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff1bc06c920 0x7ff1bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.695+0000 7ff1da59c640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff1bc06c920 0x7ff1bc06ede0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc006fd0 tx=0x7ff1cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.696+0000 7ff1e3ba6740 1 Processor -- start 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.696+0000 7ff1e3ba6740 1 -- start start 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.696+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0be440 0x55a44c0be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.696+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0dc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.696+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c094120 0x55a44c0da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.696+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a44bff8740 con 0x55a44c0be440 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.696+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a44bf943a0 con 0x55a44c0dbce0 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.696+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a44bff4700 con 0x55a44c094120 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0be440 0x55a44c0be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0be440 0x55a44c0be820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51282/0 (socket says 172.21.15.40:51282) 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 -- 172.21.15.40:0/4066136509 learned_addr learned my addr 172.21.15.40:0/4066136509 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c094120 0x55a44c0da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 -- 172.21.15.40:0/4066136509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c094120 msgr2=0x55a44c0da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c094120 0x55a44c0da3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 -- 172.21.15.40:0/4066136509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 msgr2=0x55a44c0dc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0dc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 -- 172.21.15.40:0/4066136509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44bfb0750 con 0x55a44c0be440 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1da59c640 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0be440 0x55a44c0be820 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00faf0 tx=0x7ff1cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6109f1a6251c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1bbfff640 1 -- 172.21.15.40:0/4066136509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc01b050 con 0x55a44c0be440 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1bbfff640 1 -- 172.21.15.40:0/4066136509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc025070 con 0x55a44c0be440 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.697+0000 7ff1bbfff640 1 -- 172.21.15.40:0/4066136509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc020440 con 0x55a44c0be440 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/4066136509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0be440 msgr2=0x55a44c0be820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0be440 0x55a44c0be820 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00faf0 tx=0x7ff1cc012000 comp rx=0 tx=0).stop 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/4066136509 shutdown_connections 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c094120 0x55a44c0da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0dc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/4066136509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0be440 0x55a44c0be820 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/4066136509 >> 172.21.15.40:0/4066136509 conn(0x55a44c090a50 msgr2=0x55a44c0d9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/4066136509 shutdown_connections 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.698+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/4066136509 wait complete. 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.699+0000 7ff1e3ba6740 1 -- start start 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.699+0000 7ff1e3ba6740 1 Processor -- start 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 0x55a44c0daea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 0x55a44c0d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a44c094ec0 con 0x55a44c0dbce0 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a44bf7ffe0 con 0x55a44c0be440 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a44c08e980 con 0x55a44c0d65c0 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 10 client.0 _ll_register_callbacks cb 0x55a44bf96150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c0d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51288/0 (socket says 172.21.15.40:51288) 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 -- 172.21.15.40:0/1894612014 learned_addr learned my addr 172.21.15.40:0/1894612014 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1da59c640 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 0x55a44c0daea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 -- 172.21.15.40:0/1894612014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 msgr2=0x55a44c0d6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 0x55a44c0d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 -- 172.21.15.40:0/1894612014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 msgr2=0x55a44c0daea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 0x55a44c0daea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 -- 172.21.15.40:0/1894612014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44bd0a2d0 con 0x55a44c0dbce0 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c0d0f40 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff1c408a280 tx=0x7ff1c405a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8994b508ddd656f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:43:55.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1b97fa640 1 -- 172.21.15.40:0/1894612014 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4088060 con 0x55a44c0dbce0 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1b97fa640 1 -- 172.21.15.40:0/1894612014 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7ff1c4089ec0 con 0x55a44c0dbce0 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/1894612014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc025070 con 0x55a44c0dbce0 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1b97fa640 1 -- 172.21.15.40:0/1894612014 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7ff1c4072070 con 0x55a44c0dbce0 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 10 client.9330 fetch_fsmap learned FSMap version 169 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/1894612014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=169,monmap=2+}) -- 0x55a44bdd46b0 con 0x55a44c0dbce0 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.700+0000 7ff1b97fa640 1 -- 172.21.15.40:0/1894612014 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc025070 con 0x55a44c0dbce0 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1b97fa640 1 -- 172.21.15.40:0/1894612014 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 169) ==== 3494+0+0 (secure 0 0 0) 0x7ff1c4025060 con 0x55a44c0dbce0 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1e3ba6740 10 client.9330 fetch_fsmap finished waiting for FSMap version 169 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1e3ba6740 10 client.9330 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1e3ba6740 20 client.9330 populate_metadata read hostname 'smithi040' 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] conn(0x55a44c0ecf40 0x55a44c0ed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1e3ba6740 4 client.9330 mds_command: new command op to 9240 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/1894612014 --> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a44c006350 con 0x55a44c0ecf40 2024-04-26T21:43:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1da59c640 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] conn(0x55a44c0ecf40 0x55a44c0ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1da59c640 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] conn(0x55a44c0ecf40 0x55a44c0ed320 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dec261ebdc5a4c7 server_cookie=ee9478cdf0c01c68 in_seq=0 out_seq=0 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.701+0000 7ff1b97fa640 10 client.9330 ms_handle_connect on v2:172.21.15.40:6832/1803384446 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1b97fa640 1 -- 172.21.15.40:0/1894612014 <== mds.0 v2:172.21.15.40:6832/1803384446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a44c006350 con 0x55a44c0ecf40 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1b97fa640 10 client.9330 handle_command_reply: tid=0 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 client.9330 shutdown 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/1894612014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 msgr2=0x55a44c0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c0d0f40 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff1c408a280 tx=0x7ff1c405a320 comp rx=0 tx=0).stop 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/1894612014 shutdown_connections 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] conn(0x55a44c0ecf40 0x55a44c0ed320 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 0x55a44c0d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c0d0f40 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/1894612014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 0x55a44c0daea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1b97fa640 0 client.9330 ms_handle_reset on v2:172.21.15.40:6832/1803384446 2024-04-26T21:43:55.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/1894612014 >> 172.21.15.40:0/1894612014 conn(0x55a44c094120 msgr2=0x55a44c0dd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/1894612014 shutdown_connections 2024-04-26T21:43:55.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/1894612014 wait complete. 2024-04-26T21:43:55.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.702+0000 7ff1e3ba6740 20 client.9330 trim_cache size 0 max 16384 2024-04-26T21:43:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.721+0000 7ff1e3ba6740 1 Processor -- start 2024-04-26T21:43:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.721+0000 7ff1e3ba6740 1 -- start start 2024-04-26T21:43:55.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0be440 0x55a44c0d5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0d65c0 0x55a44c093440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0e6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc201830 con 0x55a44c0d65c0 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1f8f00 con 0x55a44c0dbce0 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc2011e0 con 0x55a44c0be440 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0d65c0 0x55a44c093440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0d65c0 0x55a44c093440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51294/0 (socket says 172.21.15.40:51294) 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 -- 172.21.15.40:0/13389216 learned_addr learned my addr 172.21.15.40:0/13389216 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1da59c640 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0be440 0x55a44c0d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 -- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0be440 msgr2=0x55a44c0d5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0be440 0x55a44c0d5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 -- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 msgr2=0x55a44c0e6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0e6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 -- 172.21.15.40:0/13389216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44c0d5e70 con 0x55a44c0d65c0 2024-04-26T21:43:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0d65c0 0x55a44c093440 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4000c00 tx=0x7ff1c405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf57cba75c9afa17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1bb7fe640 1 -- 172.21.15.40:0/13389216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4059620 con 0x55a44c0d65c0 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1bb7fe640 1 -- 172.21.15.40:0/13389216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4079070 con 0x55a44c0d65c0 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.722+0000 7ff1bb7fe640 1 -- 172.21.15.40:0/13389216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4058f30 con 0x55a44c0d65c0 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/13389216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0d65c0 msgr2=0x55a44c093440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0d65c0 0x55a44c093440 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4000c00 tx=0x7ff1c405b000 comp rx=0 tx=0).stop 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/13389216 shutdown_connections 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0dbce0 0x55a44c0e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0d65c0 0x55a44c093440 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/13389216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0be440 0x55a44c0d5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/13389216 >> 172.21.15.40:0/13389216 conn(0x55a44c094120 msgr2=0x55a44c094500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/13389216 shutdown_connections 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/13389216 wait complete. 2024-04-26T21:43:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.723+0000 7ff1e3ba6740 1 -- start start 2024-04-26T21:43:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1e3ba6740 1 Processor -- start 2024-04-26T21:43:55.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 0x55a44c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 0x55a44c0bd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1e3ba6740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c156890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a44c0d4e00 con 0x55a44c0dbce0 2024-04-26T21:43:55.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a44c0d23f0 con 0x55a44c0be440 2024-04-26T21:43:55.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1e3ba6740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a44c0b17f0 con 0x55a44c0d65c0 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1e3ba6740 10 client.0 _ll_register_callbacks cb 0x7ff1c400b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.724+0000 7ff1dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51308/0 (socket says 172.21.15.40:51308) 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1dad9d640 1 -- 172.21.15.40:0/3055507146 learned_addr learned my addr 172.21.15.40:0/3055507146 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1d3fff640 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 0x55a44c0bd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1da59c640 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 0x55a44c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1dad9d640 1 -- 172.21.15.40:0/3055507146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 msgr2=0x55a44c0bd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 0x55a44c0bd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1dad9d640 1 -- 172.21.15.40:0/3055507146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 msgr2=0x55a44c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 0x55a44c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1dad9d640 1 -- 172.21.15.40:0/3055507146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44c0bd600 con 0x55a44c0dbce0 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1dad9d640 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c156890 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009f40 tx=0x7ff1d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f1326562833f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1b97fa640 1 -- 172.21.15.40:0/3055507146 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4036040 con 0x55a44c0dbce0 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1b97fa640 1 -- 172.21.15.40:0/3055507146 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7ff1d4031bf0 con 0x55a44c0dbce0 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/3055507146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c4079070 con 0x55a44c0dbce0 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1b97fa640 1 -- 172.21.15.40:0/3055507146 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7ff1d4009510 con 0x55a44c0dbce0 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1e3ba6740 10 client.9336 fetch_fsmap learned FSMap version 169 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.725+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/3055507146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=169,monmap=2+}) -- 0x55a44c1571e0 con 0x55a44c0dbce0 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1b97fa640 1 -- 172.21.15.40:0/3055507146 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4079070 con 0x55a44c0dbce0 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1b97fa640 1 -- 172.21.15.40:0/3055507146 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 169) ==== 3494+0+0 (secure 0 0 0) 0x7ff1d4045430 con 0x55a44c0dbce0 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 10 client.9336 fetch_fsmap finished waiting for FSMap version 169 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 10 client.9336 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 20 client.9336 populate_metadata read hostname 'smithi040' 2024-04-26T21:43:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] conn(0x55a44c0d3d10 0x55a44c0d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 4 client.9336 mds_command: new command op to 9240 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/3055507146 --> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] -- command(tid 0: {"prefix": "status"}) -- 0x55a44c0001c0 con 0x55a44c0d3d10 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1da59c640 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] conn(0x55a44c0d3d10 0x55a44c0d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1b97fa640 10 client.9336 ms_handle_connect on v2:172.21.15.40:6832/1803384446 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1da59c640 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] conn(0x55a44c0d3d10 0x55a44c0d40f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb440238cecf514a server_cookie=607eba553dfd06cb in_seq=0 out_seq=0 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1b97fa640 1 -- 172.21.15.40:0/3055507146 <== mds.0 v2:172.21.15.40:6832/1803384446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a44c0001c0 con 0x55a44c0d3d10 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1b97fa640 10 client.9336 handle_command_reply: tid=0 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 1 client.9336 shutdown 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/3055507146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 msgr2=0x55a44c156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.726+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c156890 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009f40 tx=0x7ff1d4004a40 comp rx=0 tx=0).stop 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/3055507146 shutdown_connections 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.40:6832/1803384446,v1:172.21.15.40:6833/1803384446] conn(0x55a44c0d3d10 0x55a44c0d40f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a44c0dbce0 0x55a44c156890 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a44c0d65c0 0x55a44c0bd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1b97fa640 0 client.9336 ms_handle_reset on v2:172.21.15.40:6832/1803384446 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 1 --2- 172.21.15.40:0/3055507146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44c0be440 0x55a44c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/3055507146 >> 172.21.15.40:0/3055507146 conn(0x55a44c094120 msgr2=0x55a44c089d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/3055507146 shutdown_connections 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 1 -- 172.21.15.40:0/3055507146 wait complete. 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e3ba6740 20 client.9336 trim_cache size 0 max 16384 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 9240, 2024-04-26T21:43:55.734 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:43:55.735 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:43:55.735 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:43:55.735 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.6867675369999997, 2024-04-26T21:43:55.735 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 169, 2024-04-26T21:43:55.735 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 107, 2024-04-26T21:43:55.735 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 107, 2024-04-26T21:43:55.735 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.67719046 2024-04-26T21:43:55.735 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff1bc06c920 msgr2=0x7ff1bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff1bc06c920 0x7ff1bc06ede0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc006fd0 tx=0x7ff1cc008040 comp rx=0 tx=0).stop 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3235652686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 msgr2=0x7ff1dc15c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc15c770 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400d800 tx=0x7ff1c400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3235652686 shutdown_connections 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff1bc06c920 0x7ff1bc06ede0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1dc162880 0x7ff1dc0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1dc0d39f0 0x7ff1dc15c770 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 --2- 172.21.15.40:0/3235652686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1dc0b5f20 0x7ff1dc0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3235652686 >> 172.21.15.40:0/3235652686 conn(0x7ff1dc0d4f60 msgr2=0x7ff1dc0b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3235652686 shutdown_connections 2024-04-26T21:43:55.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.727+0000 7ff1e0daf640 1 -- 172.21.15.40:0/3235652686 wait complete. 2024-04-26T21:43:55.739 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:43:55.739 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 400.00000000', proceeding to ls 2024-04-26T21:43:55.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:43:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.838+0000 7f412b7e7640 1 Processor -- start 2024-04-26T21:43:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.838+0000 7f412b7e7640 1 -- start start 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f412b7e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b0380 0x7f41240ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f412b7e7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41240cfd80 con 0x7f41240b10b0 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f412b7e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41240d0020 con 0x7f412406c950 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f412b7e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41240af8e0 con 0x7f41240b0380 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f4128d82640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f41240b10b0 0x7f412406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42580/0 (socket says 172.21.15.40:42580) 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f4128d82640 1 -- 172.21.15.40:0/298447591 learned_addr learned my addr 172.21.15.40:0/298447591 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f4129583640 1 --2- 172.21.15.40:0/298447591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b0380 0x7f41240ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f411bfff640 1 -- 172.21.15.40:0/298447591 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196080977 0 0) 0x7f41240cfd80 con 0x7f41240b10b0 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.839+0000 7f411bfff640 1 -- 172.21.15.40:0/298447591 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4110003520 con 0x7f41240b10b0 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.840+0000 7f411bfff640 1 -- 172.21.15.40:0/298447591 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810139449 0 0) 0x7f41240d0020 con 0x7f412406c950 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.840+0000 7f4129583640 1 -- 172.21.15.40:0/298447591 >> v1:172.21.15.184:6789/0 conn(0x7f412406c950 legacy=0x7f412415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.840+0000 7f4129583640 1 -- 172.21.15.40:0/298447591 >> v1:172.21.15.40:6789/0 conn(0x7f41240b10b0 legacy=0x7f412406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.840+0000 7f4129583640 1 -- 172.21.15.40:0/298447591 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41240d0300 con 0x7f41240b0380 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.840+0000 7f4129583640 1 --2- 172.21.15.40:0/298447591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b0380 0x7f41240ccd50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41200097d0 tx=0x7f412002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d2615a3ea2e6ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f411bfff640 1 -- 172.21.15.40:0/298447591 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120037070 con 0x7f41240b0380 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f411bfff640 1 -- 172.21.15.40:0/298447591 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4120032440 con 0x7f41240b0380 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f411bfff640 1 -- 172.21.15.40:0/298447591 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41200327f0 con 0x7f41240b0380 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f412b7e7640 1 -- 172.21.15.40:0/298447591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b0380 msgr2=0x7f41240ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f412b7e7640 1 --2- 172.21.15.40:0/298447591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b0380 0x7f41240ccd50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41200097d0 tx=0x7f412002a180 comp rx=0 tx=0).stop 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f412b7e7640 1 -- 172.21.15.40:0/298447591 shutdown_connections 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f412b7e7640 1 --2- 172.21.15.40:0/298447591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b0380 0x7f41240ccd50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f412b7e7640 1 -- 172.21.15.40:0/298447591 >> 172.21.15.40:0/298447591 conn(0x7f41240d1cf0 msgr2=0x7f41240d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f412b7e7640 1 -- 172.21.15.40:0/298447591 shutdown_connections 2024-04-26T21:43:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.841+0000 7f412b7e7640 1 -- 172.21.15.40:0/298447591 wait complete. 2024-04-26T21:43:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.842+0000 7f412b7e7640 1 Processor -- start 2024-04-26T21:43:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.842+0000 7f412b7e7640 1 -- start start 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f412b7e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f412406c950 0x7f41241f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f412b7e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41240b0380 0x7f41240c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f412b7e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b10b0 0x7f41240c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f412b7e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41240b0760 con 0x7f41240b0380 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f412b7e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41240d1980 con 0x7f412406c950 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f412b7e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41240d15c0 con 0x7f41240b10b0 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f4128d82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41240b0380 0x7f41240c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f4128d82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41240b0380 0x7f41240c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51310/0 (socket says 172.21.15.40:51310) 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f4128d82640 1 -- 172.21.15.40:0/763298580 learned_addr learned my addr 172.21.15.40:0/763298580 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:55.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f4128d82640 1 -- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b10b0 msgr2=0x7f41240c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f4129583640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f412406c950 0x7f41241f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.843+0000 7f4129d84640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b10b0 0x7f41240c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4128d82640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b10b0 0x7f41240c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4128d82640 1 -- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f412406c950 msgr2=0x7f41241f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4128d82640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f412406c950 0x7f41241f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4128d82640 1 -- 172.21.15.40:0/763298580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41241f4ff0 con 0x7f41240b0380 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4129583640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f412406c950 0x7f41241f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4129d84640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b10b0 0x7f41240c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4128d82640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41240b0380 0x7f41240c4cd0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f411400c390 tx=0x7f411400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4119ffb640 1 -- 172.21.15.40:0/763298580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114017070 con 0x7f41240b0380 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4119ffb640 1 -- 172.21.15.40:0/763298580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4114005ce0 con 0x7f41240b0380 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41241f52d0 con 0x7f41240b0380 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f4119ffb640 1 -- 172.21.15.40:0/763298580 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140072f0 con 0x7f41240b0380 2024-04-26T21:43:55.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.844+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241f5890 con 0x7f41240b0380 2024-04-26T21:43:55.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.845+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4124070c70 con 0x7f41240b0380 2024-04-26T21:43:55.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.845+0000 7f4119ffb640 1 -- 172.21.15.40:0/763298580 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f411401a070 con 0x7f41240b0380 2024-04-26T21:43:55.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.846+0000 7f4119ffb640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f410806c8f0 0x7f410806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.846+0000 7f4119ffb640 1 -- 172.21.15.40:0/763298580 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f411408b600 con 0x7f41240b0380 2024-04-26T21:43:55.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.846+0000 7f4129583640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f410806c8f0 0x7f410806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.846+0000 7f4129583640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f410806c8f0 0x7f410806edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4120034000 tx=0x7f412002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:55.850+0000 7f4119ffb640 1 -- 172.21.15.40:0/763298580 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4124070c70 con 0x7f41240b0380 2024-04-26T21:43:56.025 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:43:56.025+0000 7fbc3e243640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:43:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.079+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f41240694f0 con 0x7f41240b0380 2024-04-26T21:43:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.080+0000 7f4119ffb640 1 -- 172.21.15.40:0/763298580 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v169) ==== 53+0+73 (secure 0 0 0) 0x7f41140593a0 con 0x7f41240b0380 2024-04-26T21:43:56.082 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:43:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f410806c8f0 msgr2=0x7f410806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f410806c8f0 0x7f410806edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4120034000 tx=0x7f412002e000 comp rx=0 tx=0).stop 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41240b0380 msgr2=0x7f41240c4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41240b0380 0x7f41240c4cd0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f411400c390 tx=0x7f411400c850 comp rx=0 tx=0).stop 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 shutdown_connections 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f410806c8f0 0x7f410806edb0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41240b10b0 0x7f41240c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41240b0380 0x7f41240c4cd0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 --2- 172.21.15.40:0/763298580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f412406c950 0x7f41241f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 >> 172.21.15.40:0/763298580 conn(0x7f41240d1cf0 msgr2=0x7f41240737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 shutdown_connections 2024-04-26T21:43:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56.082+0000 7f412b7e7640 1 -- 172.21.15.40:0/763298580 wait complete. 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:43:56.094 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:56.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:56.095 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:43:56.110 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:43:56.110 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42212sec preferred_lft 42212sec 2024-04-26T21:43:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:43:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:43:56.113 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:43:56.113 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:43:56.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:56.113 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:43:56.173 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:43:56.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:56.175 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:43:56.230 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:43:56.231 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:43:56.231 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:43:56.231 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:43:56.231 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:43:56.231 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:43:56.231 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:43:56.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:43:56.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:43:56.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:43:56.231 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:43:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:43:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:43:56.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:56.357 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:43:56.372 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:43:56.372 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:43:56.372 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:56.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:56.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:43:56.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:43:56.437 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:56.494 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:43:56.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:43:56.495 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:56.549 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:43:56.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:56.550 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:56.631 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:56.631 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:43:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:43:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:43:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:43:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:43:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:43:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:43:56.641 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:43:56.641 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:43:56.641 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:43:56.641 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:43:56.643 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:43:56.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:56.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:43:56.707 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:43:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:43:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:43:56.803 INFO:teuthology.orchestra.run.smithi040.stdout:client7208 2024-04-26T21:43:56.803 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:56.803 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:56.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:56.819 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-26T21:43:56.862 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:43:56.876 INFO:teuthology.orchestra.run.smithi040.stdout:.: 2024-04-26T21:43:56.876 INFO:teuthology.orchestra.run.smithi040.stdout:subdir 2024-04-26T21:43:56.876 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:56.876 INFO:teuthology.orchestra.run.smithi040.stdout:./subdir: 2024-04-26T21:43:56.876 INFO:teuthology.orchestra.run.smithi040.stdout:sixmegs 2024-04-26T21:43:56.876 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 400.00000000' did not caused DAMAGED state 2024-04-26T21:43:56.877 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000.inode' 2024-04-26T21:43:56.877 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:56.877 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:56.932 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:43:56.932 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:43:56.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:43:57.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.128+0000 7f1f5cb38640 1 Processor -- start 2024-04-26T21:43:57.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.128+0000 7f1f5cb38640 1 -- start start 2024-04-26T21:43:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.129+0000 7f1f5cb38640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1f580d06f0 0x7f1f580d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.129+0000 7f1f5cb38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580b2850 0x7f1f580b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.129+0000 7f1f5cb38640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f580d0ba0 con 0x7f1f5815f3a0 2024-04-26T21:43:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.129+0000 7f1f5cb38640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1f580d0e20 con 0x7f1f580d06f0 2024-04-26T21:43:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.129+0000 7f1f5cb38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f580b1d70 con 0x7f1f580b2850 2024-04-26T21:43:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.129+0000 7f1f5659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1f5815f3a0 0x7f1f580d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42596/0 (socket says 172.21.15.40:42596) 2024-04-26T21:43:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5659c640 1 -- 172.21.15.40:0/2829373306 learned_addr learned my addr 172.21.15.40:0/2829373306 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f56d9d640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580b2850 0x7f1f580b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f55d9b640 1 --2- 172.21.15.40:0/2829373306 >> v2:172.21.15.184:3300/0 conn(0x7f1f580d06f0 0x7f1f580d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641587829 0 0) 0x7f1f50001110 con 0x7f1f5815f3a0 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f40003520 con 0x7f1f5815f3a0 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 808643634 0 0) 0x7f1f40003520 con 0x7f1f5815f3a0 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580b2850 msgr2=0x7f1f580b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580b2850 0x7f1f580b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 >> v2:172.21.15.184:3300/0 conn(0x7f1f580d06f0 msgr2=0x7f1f580d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 --2- 172.21.15.40:0/2829373306 >> v2:172.21.15.184:3300/0 conn(0x7f1f580d06f0 0x7f1f580d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f580b2050 con 0x7f1f5815f3a0 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1f50001840 con 0x7f1f5815f3a0 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 >> v1:172.21.15.40:6789/0 conn(0x7f1f5815f3a0 legacy=0x7f1f580d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f400039b0 0x7f1f40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580b2850 0x7f1f40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f40004d40 0x7f1f400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1f580d0e20 con 0x7f1f40004d40 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f580b1d70 con 0x7f1f400039b0 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4000b9c0 con 0x7f1f580b2850 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f56d9d640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f40004d40 0x7f1f400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f5659c640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f400039b0 0x7f1f40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.130+0000 7f1f55d9b640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580b2850 0x7f1f40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f56d9d640 1 -- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580b2850 msgr2=0x7f1f40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f56d9d640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580b2850 0x7f1f40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f56d9d640 1 -- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f400039b0 msgr2=0x7f1f40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f56d9d640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f400039b0 0x7f1f40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f56d9d640 1 -- 172.21.15.40:0/2829373306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f40003520 con 0x7f1f40004d40 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f56d9d640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f40004d40 0x7f1f400051c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009fc0 tx=0x7f1f40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300cccf97097a135 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48009c10 con 0x7f1f40004d40 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f48036070 con 0x7f1f40004d40 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.131+0000 7f1f5559a640 1 -- 172.21.15.40:0/2829373306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48031880 con 0x7f1f40004d40 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.132+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2829373306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f40004d40 msgr2=0x7f1f400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.132+0000 7f1f5cb38640 1 --2- 172.21.15.40:0/2829373306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f40004d40 0x7f1f400051c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009fc0 tx=0x7f1f40005830 comp rx=0 tx=0).stop 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.132+0000 7f1f56d9d640 1 -- 172.21.15.40:0/2829373306 reap_dead start 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.132+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2829373306 shutdown_connections 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.132+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2829373306 >> 172.21.15.40:0/2829373306 conn(0x7f1f580d4f90 msgr2=0x7f1f580d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.132+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2829373306 shutdown_connections 2024-04-26T21:43:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.132+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2829373306 wait complete. 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.133+0000 7f1f5cb38640 1 Processor -- start 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.133+0000 7f1f5cb38640 1 -- start start 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f5cb38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f40004d40 0x7f1f581f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f5cb38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f580cd0d0 0x7f1f580a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f5cb38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580cd550 0x7f1f580a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f5cb38640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5815f900 con 0x7f1f580cd0d0 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f5cb38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5815f780 con 0x7f1f40004d40 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f5cb38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f580d23c0 con 0x7f1f580cd550 2024-04-26T21:43:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f55d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f580cd0d0 0x7f1f580a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f55d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f580cd0d0 0x7f1f580a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51320/0 (socket says 172.21.15.40:51320) 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.134+0000 7f1f55d9b640 1 -- 172.21.15.40:0/2482655394 learned_addr learned my addr 172.21.15.40:0/2482655394 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f55d9b640 1 -- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580cd550 msgr2=0x7f1f580a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f55d9b640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580cd550 0x7f1f580a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f55d9b640 1 -- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f40004d40 msgr2=0x7f1f581f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f5659c640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f40004d40 0x7f1f581f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f55d9b640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f40004d40 0x7f1f581f81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f55d9b640 1 -- 172.21.15.40:0/2482655394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f580cdab0 con 0x7f1f580cd0d0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f55d9b640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f580cd0d0 0x7f1f580a0760 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c011a20 tx=0x7f1f4c011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f46ffd640 1 -- 172.21.15.40:0/2482655394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00fb50 con 0x7f1f580cd0d0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f46ffd640 1 -- 172.21.15.40:0/2482655394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00fcf0 con 0x7f1f580cd0d0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f580cdd90 con 0x7f1f580cd0d0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f46ffd640 1 -- 172.21.15.40:0/2482655394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00fea0 con 0x7f1f580cd0d0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.135+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f580ce350 con 0x7f1f580cd0d0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.136+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f580694f0 con 0x7f1f580cd0d0 2024-04-26T21:43:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.136+0000 7f1f46ffd640 1 -- 172.21.15.40:0/2482655394 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f4c012040 con 0x7f1f580cd0d0 2024-04-26T21:43:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.137+0000 7f1f46ffd640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3806c8f0 0x7f1f3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.137+0000 7f1f46ffd640 1 -- 172.21.15.40:0/2482655394 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f1f4c08c820 con 0x7f1f580cd0d0 2024-04-26T21:43:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.137+0000 7f1f5659c640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3806c8f0 0x7f1f3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.138+0000 7f1f5659c640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3806c8f0 0x7f1f3806edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1f500012f0 tx=0x7f1f50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.141+0000 7f1f46ffd640 1 -- 172.21.15.40:0/2482655394 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f4c019030 con 0x7f1f580cd0d0 2024-04-26T21:43:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.327+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f580b6860 con 0x7f1f580cd0d0 2024-04-26T21:43:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.329+0000 7f1f46ffd640 1 -- 172.21.15.40:0/2482655394 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+18602 (secure 0 0 0) 0x7f1f4c05a5b0 con 0x7f1f580cd0d0 2024-04-26T21:43:57.330 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:43:57.330 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":107,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:43:45.018145+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:43:57.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3806c8f0 msgr2=0x7f1f3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3806c8f0 0x7f1f3806edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1f500012f0 tx=0x7f1f50009040 comp rx=0 tx=0).stop 2024-04-26T21:43:57.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f580cd0d0 msgr2=0x7f1f580a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f580cd0d0 0x7f1f580a0760 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c011a20 tx=0x7f1f4c011ee0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 shutdown_connections 2024-04-26T21:43:57.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f3806c8f0 0x7f1f3806edb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f580cd550 0x7f1f580a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f580cd0d0 0x7f1f580a0760 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 --2- 172.21.15.40:0/2482655394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f40004d40 0x7f1f581f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 >> 172.21.15.40:0/2482655394 conn(0x7f1f580d4f90 msgr2=0x7f1f580d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:57.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 shutdown_connections 2024-04-26T21:43:57.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.331+0000 7f1f5cb38640 1 -- 172.21.15.40:0/2482655394 wait complete. 2024-04-26T21:43:57.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:43:57.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:43:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.446+0000 7f11a1571640 1 Processor -- start 2024-04-26T21:43:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.446+0000 7f11a1571640 1 -- start start 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f11a1571640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f119c070350 0x7f119c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f11a1571640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f119c071090 0x7f119c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f11a1571640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f119c0d2390 con 0x7f119c0cfde0 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f11a1571640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f119c0afab0 con 0x7f119c070350 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f11a1571640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f119c06dd20 con 0x7f119c071090 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f119c070350 0x7f119c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f119c0cfde0 0x7f119c0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42600/0 (socket says 172.21.15.40:42600) 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119affd640 1 -- 172.21.15.40:0/588006019 learned_addr learned my addr 172.21.15.40:0/588006019 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119b7fe640 1 --2- 172.21.15.40:0/588006019 >> v2:172.21.15.184:3300/0 conn(0x7f119c071090 0x7f119c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119a7fc640 1 -- 172.21.15.40:0/588006019 >> v2:172.21.15.184:3300/0 conn(0x7f119c071090 msgr2=0x7f119c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119a7fc640 1 --2- 172.21.15.40:0/588006019 >> v2:172.21.15.184:3300/0 conn(0x7f119c071090 0x7f119c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119a7fc640 1 -- 172.21.15.40:0/588006019 >> v1:172.21.15.40:6789/0 conn(0x7f119c0cfde0 legacy=0x7f119c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f1199ffb640 1 -- 172.21.15.40:0/588006019 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2305848166 0 0) 0x7f119c0d2390 con 0x7f119c0cfde0 2024-04-26T21:43:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119a7fc640 1 -- 172.21.15.40:0/588006019 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c06d3f0 con 0x7f119c070350 2024-04-26T21:43:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.447+0000 7f119b7fe640 1 --2- 172.21.15.40:0/588006019 >> v2:172.21.15.184:3300/0 conn(0x7f119c071090 0x7f119c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.448+0000 7f119a7fc640 1 --2- 172.21.15.40:0/588006019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c070350 0x7f119c070b50 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f119c0c6860 tx=0x7f119002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896555f01246ffda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.448+0000 7f1199ffb640 1 -- 172.21.15.40:0/588006019 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190037070 con 0x7f119c070350 2024-04-26T21:43:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.448+0000 7f1199ffb640 1 -- 172.21.15.40:0/588006019 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1190032440 con 0x7f119c070350 2024-04-26T21:43:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.448+0000 7f1199ffb640 1 -- 172.21.15.40:0/588006019 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119002ae60 con 0x7f119c070350 2024-04-26T21:43:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.449+0000 7f11a1571640 1 -- 172.21.15.40:0/588006019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c070350 msgr2=0x7f119c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.449+0000 7f11a1571640 1 --2- 172.21.15.40:0/588006019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c070350 0x7f119c070b50 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f119c0c6860 tx=0x7f119002a110 comp rx=0 tx=0).stop 2024-04-26T21:43:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.449+0000 7f11a1571640 1 -- 172.21.15.40:0/588006019 shutdown_connections 2024-04-26T21:43:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.449+0000 7f11a1571640 1 --2- 172.21.15.40:0/588006019 >> v2:172.21.15.184:3300/0 conn(0x7f119c071090 0x7f119c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.449+0000 7f11a1571640 1 --2- 172.21.15.40:0/588006019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c070350 0x7f119c070b50 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.449+0000 7f11a1571640 1 -- 172.21.15.40:0/588006019 >> 172.21.15.40:0/588006019 conn(0x7f119c0d2d80 msgr2=0x7f119c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.449+0000 7f11a1571640 1 -- 172.21.15.40:0/588006019 shutdown_connections 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.449+0000 7f11a1571640 1 -- 172.21.15.40:0/588006019 wait complete. 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.450+0000 7f11a1571640 1 Processor -- start 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.450+0000 7f11a1571640 1 -- start start 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f11a1571640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f119c070350 0x7f119c162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f11a1571640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c071090 0x7f119c15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f11a1571640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f119c0cfde0 0x7f119c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f11a1571640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f119c06d9b0 con 0x7f119c071090 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f11a1571640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f119c0714c0 con 0x7f119c0cfde0 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f11a1571640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f119c15ef30 con 0x7f119c070350 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c071090 0x7f119c15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c071090 0x7f119c15ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51344/0 (socket says 172.21.15.40:51344) 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 -- 172.21.15.40:0/4091991215 learned_addr learned my addr 172.21.15.40:0/4091991215 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119affd640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f119c070350 0x7f119c162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119b7fe640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f119c0cfde0 0x7f119c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 -- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f119c070350 msgr2=0x7f119c162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f119c070350 0x7f119c162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 -- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f119c0cfde0 msgr2=0x7f119c0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f119c0cfde0 0x7f119c0c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 -- 172.21.15.40:0/4091991215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c09d050 con 0x7f119c071090 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f119a7fc640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c071090 0x7f119c15ce70 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f1190000c00 tx=0x7f11900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f11837fe640 1 -- 172.21.15.40:0/4091991215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190037040 con 0x7f119c071090 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.451+0000 7f11837fe640 1 -- 172.21.15.40:0/4091991215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1190042070 con 0x7f119c071090 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.452+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c09d330 con 0x7f119c071090 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.452+0000 7f11837fe640 1 -- 172.21.15.40:0/4091991215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190030a50 con 0x7f119c071090 2024-04-26T21:43:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.452+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c09d8f0 con 0x7f119c071090 2024-04-26T21:43:57.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.452+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119c0694f0 con 0x7f119c071090 2024-04-26T21:43:57.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.453+0000 7f11837fe640 1 -- 172.21.15.40:0/4091991215 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1190030bf0 con 0x7f119c071090 2024-04-26T21:43:57.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.453+0000 7f11837fe640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f118806c920 0x7f118806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.454+0000 7f11837fe640 1 -- 172.21.15.40:0/4091991215 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f11900acd10 con 0x7f119c071090 2024-04-26T21:43:57.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.454+0000 7f119affd640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f118806c920 0x7f118806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.454+0000 7f119affd640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f118806c920 0x7f118806ede0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f118c000f60 tx=0x7f118c007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.458+0000 7f11837fe640 1 -- 172.21.15.40:0/4091991215 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11900b2020 con 0x7f119c071090 2024-04-26T21:43:57.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.642+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f119c15ebe0 con 0x7f119c071090 2024-04-26T21:43:57.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.643+0000 7f11837fe640 1 -- 172.21.15.40:0/4091991215 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f1190045360 con 0x7f119c071090 2024-04-26T21:43:57.643 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f118806c920 msgr2=0x7f118806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f118806c920 0x7f118806ede0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f118c000f60 tx=0x7f118c007c00 comp rx=0 tx=0).stop 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c071090 msgr2=0x7f119c15ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c071090 0x7f119c15ce70 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f1190000c00 tx=0x7f11900046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 shutdown_connections 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f118806c920 0x7f118806ede0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f119c0cfde0 0x7f119c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f119c071090 0x7f119c15ce70 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 --2- 172.21.15.40:0/4091991215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f119c070350 0x7f119c162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.644+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 >> 172.21.15.40:0/4091991215 conn(0x7f119c0d2d80 msgr2=0x7f119c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.645+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 shutdown_connections 2024-04-26T21:43:57.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.645+0000 7f11a1571640 1 -- 172.21.15.40:0/4091991215 wait complete. 2024-04-26T21:43:57.656 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:43:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.759+0000 7fb4e2bb7640 1 Processor -- start 2024-04-26T21:43:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.759+0000 7fb4e2bb7640 1 -- start start 2024-04-26T21:43:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e2bb7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4dc15f170 0x7fb4dc0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e2bb7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb4dc15f550 0x7fb4dc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e2bb7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4dc0d1040 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e2bb7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb4dc0c9430 con 0x7fb4dc15f170 2024-04-26T21:43:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e2bb7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb4dc0c9680 con 0x7fb4dc15f550 2024-04-26T21:43:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e0953640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4dc15f170 0x7fb4dc0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e1154640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb4dc0d34b0 0x7fb4dc0cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42604/0 (socket says 172.21.15.40:42604) 2024-04-26T21:43:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e1154640 1 -- 172.21.15.40:0/1800440296 learned_addr learned my addr 172.21.15.40:0/1800440296 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e0953640 1 -- 172.21.15.40:0/1800440296 >> v2:172.21.15.184:3300/0 conn(0x7fb4dc15f550 msgr2=0x7fb4dc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4d3fff640 1 --2- 172.21.15.40:0/1800440296 >> v2:172.21.15.184:3300/0 conn(0x7fb4dc15f550 0x7fb4dc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e0953640 1 --2- 172.21.15.40:0/1800440296 >> v2:172.21.15.184:3300/0 conn(0x7fb4dc15f550 0x7fb4dc0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4d37fe640 1 -- 172.21.15.40:0/1800440296 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279019067 0 0) 0x7fb4d4001490 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e0953640 1 -- 172.21.15.40:0/1800440296 >> v1:172.21.15.40:6789/0 conn(0x7fb4dc0d34b0 legacy=0x7fb4dc0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.760+0000 7fb4e0953640 1 -- 172.21.15.40:0/1800440296 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc0c9960 con 0x7fb4dc15f170 2024-04-26T21:43:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.761+0000 7fb4e0953640 1 --2- 172.21.15.40:0/1800440296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc15f170 0x7fb4dc0d5a80 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80097a0 tx=0x7fb4d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db36f019fac226dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.761+0000 7fb4d37fe640 1 -- 172.21.15.40:0/1800440296 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8037070 con 0x7fb4dc15f170 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.761+0000 7fb4d37fe640 1 -- 172.21.15.40:0/1800440296 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8032440 con 0x7fb4dc15f170 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.761+0000 7fb4d37fe640 1 -- 172.21.15.40:0/1800440296 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8032810 con 0x7fb4dc15f170 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.761+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/1800440296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc15f170 msgr2=0x7fb4dc0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.761+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/1800440296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc15f170 0x7fb4dc0d5a80 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80097a0 tx=0x7fb4d802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.762+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/1800440296 shutdown_connections 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.762+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/1800440296 >> v2:172.21.15.184:3300/0 conn(0x7fb4dc15f550 0x7fb4dc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.762+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/1800440296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc15f170 0x7fb4dc0d5a80 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.762+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/1800440296 >> 172.21.15.40:0/1800440296 conn(0x7fb4dc0d4f60 msgr2=0x7fb4dc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.762+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/1800440296 shutdown_connections 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.762+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/1800440296 wait complete. 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.763+0000 7fb4e2bb7640 1 Processor -- start 2024-04-26T21:43:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.763+0000 7fb4e2bb7640 1 -- start start 2024-04-26T21:43:57.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e2bb7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc0d34b0 0x7fb4dc15bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e2bb7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4dc15f170 0x7fb4dc159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e2bb7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4dc0c7a90 0x7fb4dc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e2bb7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0cb5d0 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e2bb7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0cb450 con 0x7fb4dc0c7a90 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e2bb7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4dc0cb750 con 0x7fb4dc15f170 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc0d34b0 0x7fb4dc15bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc0d34b0 0x7fb4dc15bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51368/0 (socket says 172.21.15.40:51368) 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 -- 172.21.15.40:0/802142022 learned_addr learned my addr 172.21.15.40:0/802142022 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4d3fff640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4dc15f170 0x7fb4dc159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e1154640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4dc0c7a90 0x7fb4dc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 -- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4dc15f170 msgr2=0x7fb4dc159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4dc15f170 0x7fb4dc159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 -- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4dc0c7a90 msgr2=0x7fb4dc06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4dc0c7a90 0x7fb4dc06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 -- 172.21.15.40:0/802142022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc15a340 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e1154640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4dc0c7a90 0x7fb4dc06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4e0953640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc0d34b0 0x7fb4dc15bab0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8000c00 tx=0x7fb4d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4d17fa640 1 -- 172.21.15.40:0/802142022 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8037040 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.764+0000 7fb4d17fa640 1 -- 172.21.15.40:0/802142022 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8042070 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.765+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc15a620 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.765+0000 7fb4d17fa640 1 -- 172.21.15.40:0/802142022 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8030a70 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.765+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc15abe0 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.766+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4dc0694f0 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.766+0000 7fb4d17fa640 1 -- 172.21.15.40:0/802142022 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4d8030c10 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.767+0000 7fb4d17fa640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4c806c8f0 0x7fb4c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.767+0000 7fb4d17fa640 1 -- 172.21.15.40:0/802142022 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7fb4d80acbb0 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.767+0000 7fb4d3fff640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4c806c8f0 0x7fb4c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.767+0000 7fb4d3fff640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4c806c8f0 0x7fb4c806edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006fd0 tx=0x7fb4cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.771+0000 7fb4d17fa640 1 -- 172.21.15.40:0/802142022 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4d8045360 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.955+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4dc0d12c0 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.956+0000 7fb4d17fa640 1 -- 172.21.15.40:0/802142022 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7fb4d807aa60 con 0x7fb4dc0d34b0 2024-04-26T21:43:57.956 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4c806c8f0 msgr2=0x7fb4c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4c806c8f0 0x7fb4c806edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006fd0 tx=0x7fb4cc008040 comp rx=0 tx=0).stop 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc0d34b0 msgr2=0x7fb4dc15bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc0d34b0 0x7fb4dc15bab0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8000c00 tx=0x7fb4d80046c0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 shutdown_connections 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4c806c8f0 0x7fb4c806edb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4dc0c7a90 0x7fb4dc06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4dc15f170 0x7fb4dc159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 --2- 172.21.15.40:0/802142022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4dc0d34b0 0x7fb4dc15bab0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 >> 172.21.15.40:0/802142022 conn(0x7fb4dc0d4f60 msgr2=0x7fb4dc0b9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 shutdown_connections 2024-04-26T21:43:57.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:57.958+0000 7fb4e2bb7640 1 -- 172.21.15.40:0/802142022 wait complete. 2024-04-26T21:43:57.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:43:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.073+0000 7f12f8c64640 1 Processor -- start 2024-04-26T21:43:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.073+0000 7f12f8c64640 1 -- start start 2024-04-26T21:43:58.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f8c64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f40d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f8c64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f12f415f550 0x7f12f40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f8c64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12f40d34b0 0x7f12f40d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f8c64640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f12f40c9430 con 0x7f12f40d34b0 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f8c64640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f12f40d3e20 con 0x7f12f415f550 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f8c64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f12f40d1040 con 0x7f12f415f170 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f2d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12f40d34b0 0x7f12f40d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f2d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12f40d34b0 0x7f12f40d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51378/0 (socket says 172.21.15.40:51378) 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f2d9d640 1 -- 172.21.15.40:0/2579613309 learned_addr learned my addr 172.21.15.40:0/2579613309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f259c640 1 --2- 172.21.15.40:0/2579613309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f40d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12ebfff640 1 --2- 172.21.15.40:0/2579613309 >> v2:172.21.15.184:3300/0 conn(0x7f12f415f550 0x7f12f40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f2d9d640 1 -- 172.21.15.40:0/2579613309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 msgr2=0x7f12f40d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f2d9d640 1 --2- 172.21.15.40:0/2579613309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f40d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f2d9d640 1 -- 172.21.15.40:0/2579613309 >> v2:172.21.15.184:3300/0 conn(0x7f12f415f550 msgr2=0x7f12f40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f2d9d640 1 --2- 172.21.15.40:0/2579613309 >> v2:172.21.15.184:3300/0 conn(0x7f12f415f550 0x7f12f40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12f2d9d640 1 -- 172.21.15.40:0/2579613309 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f40c9680 con 0x7f12f40d34b0 2024-04-26T21:43:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.074+0000 7f12ebfff640 1 --2- 172.21.15.40:0/2579613309 >> v2:172.21.15.184:3300/0 conn(0x7f12f415f550 0x7f12f40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f2d9d640 1 --2- 172.21.15.40:0/2579613309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f40d38e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f12f408e960 tx=0x7f12ec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb6b2645e195fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f1d9b640 1 -- 172.21.15.40:0/2579613309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec036070 con 0x7f12f40d34b0 2024-04-26T21:43:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f1d9b640 1 -- 172.21.15.40:0/2579613309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ec031600 con 0x7f12f40d34b0 2024-04-26T21:43:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f1d9b640 1 -- 172.21.15.40:0/2579613309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0319d0 con 0x7f12f40d34b0 2024-04-26T21:43:58.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f8c64640 1 -- 172.21.15.40:0/2579613309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 msgr2=0x7f12f40d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f8c64640 1 --2- 172.21.15.40:0/2579613309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f40d38e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f12f408e960 tx=0x7f12ec02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f8c64640 1 -- 172.21.15.40:0/2579613309 shutdown_connections 2024-04-26T21:43:58.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f8c64640 1 --2- 172.21.15.40:0/2579613309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f40d38e0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f8c64640 1 --2- 172.21.15.40:0/2579613309 >> v2:172.21.15.184:3300/0 conn(0x7f12f415f550 0x7f12f40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.075+0000 7f12f8c64640 1 --2- 172.21.15.40:0/2579613309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f40d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.076+0000 7f12f8c64640 1 -- 172.21.15.40:0/2579613309 >> 172.21.15.40:0/2579613309 conn(0x7f12f40d4f60 msgr2=0x7f12f40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.076+0000 7f12f8c64640 1 -- 172.21.15.40:0/2579613309 shutdown_connections 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.076+0000 7f12f8c64640 1 -- 172.21.15.40:0/2579613309 wait complete. 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.077+0000 7f12f8c64640 1 Processor -- start 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.077+0000 7f12f8c64640 1 -- start start 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f8c64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f41f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f8c64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f409ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f8c64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12f4157bf0 0x7f12f40ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f8c64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f12f40cfc50 con 0x7f12f40d34b0 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f8c64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12f40c9c40 con 0x7f12f4157bf0 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f8c64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f12f40cfdd0 con 0x7f12f415f170 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f41f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f41f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51392/0 (socket says 172.21.15.40:51392) 2024-04-26T21:43:58.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 -- 172.21.15.40:0/4239841752 learned_addr learned my addr 172.21.15.40:0/4239841752 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12ebfff640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f409ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f2d9d640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12f4157bf0 0x7f12f40ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 -- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 msgr2=0x7f12f409ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f409ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 -- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12f4157bf0 msgr2=0x7f12f40ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12f4157bf0 0x7f12f40ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 -- 172.21.15.40:0/4239841752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f409f5b0 con 0x7f12f40d34b0 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f2d9d640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12f4157bf0 0x7f12f40ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12ebfff640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f409ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12f259c640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f41f5040 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f12e400d800 tx=0x7f12e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.078+0000 7f12eaffd640 1 -- 172.21.15.40:0/4239841752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4018020 con 0x7f12f40d34b0 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.079+0000 7f12eaffd640 1 -- 172.21.15.40:0/4239841752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e4009640 con 0x7f12f40d34b0 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.079+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f409f890 con 0x7f12f40d34b0 2024-04-26T21:43:58.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.079+0000 7f12eaffd640 1 -- 172.21.15.40:0/4239841752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4009a10 con 0x7f12f40d34b0 2024-04-26T21:43:58.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.079+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f4151fc0 con 0x7f12f40d34b0 2024-04-26T21:43:58.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.079+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f40694f0 con 0x7f12f40d34b0 2024-04-26T21:43:58.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.080+0000 7f12eaffd640 1 -- 172.21.15.40:0/4239841752 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12e401b020 con 0x7f12f40d34b0 2024-04-26T21:43:58.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.080+0000 7f12eaffd640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f12d406c8f0 0x7f12d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.081+0000 7f12eaffd640 1 -- 172.21.15.40:0/4239841752 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f12e408c3d0 con 0x7f12f40d34b0 2024-04-26T21:43:58.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.081+0000 7f12ebfff640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f12d406c8f0 0x7f12d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.081+0000 7f12ebfff640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f12d406c8f0 0x7f12d406edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f12f40c8380 tx=0x7f12dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.085+0000 7f12eaffd640 1 -- 172.21.15.40:0/4239841752 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f12e4022030 con 0x7f12f40d34b0 2024-04-26T21:43:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.273+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f12f40d5620 con 0x7f12f40d34b0 2024-04-26T21:43:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.274+0000 7f12eaffd640 1 -- 172.21.15.40:0/4239841752 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f12e405a150 con 0x7f12f40d34b0 2024-04-26T21:43:58.275 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:43:58.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f12d406c8f0 msgr2=0x7f12d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f12d406c8f0 0x7f12d406edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f12f40c8380 tx=0x7f12dc008040 comp rx=0 tx=0).stop 2024-04-26T21:43:58.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 msgr2=0x7f12f41f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f41f5040 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f12e400d800 tx=0x7f12e400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 shutdown_connections 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f12d406c8f0 0x7f12d406edb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12f4157bf0 0x7f12f40ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12f415f170 0x7f12f409ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 --2- 172.21.15.40:0/4239841752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12f40d34b0 0x7f12f41f5040 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 >> 172.21.15.40:0/4239841752 conn(0x7f12f40d4f60 msgr2=0x7f12f40ba5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 shutdown_connections 2024-04-26T21:43:58.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.276+0000 7f12f8c64640 1 -- 172.21.15.40:0/4239841752 wait complete. 2024-04-26T21:43:58.288 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:43:58.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.389+0000 7f30336c8640 1 Processor -- start 2024-04-26T21:43:58.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.389+0000 7f30336c8640 1 -- start start 2024-04-26T21:43:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.390+0000 7f30336c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c162880 0x7f302c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.390+0000 7f30336c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f302c0b5f20 0x7f302c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.390+0000 7f30336c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f302c0b7e50 con 0x7f302c0d39f0 2024-04-26T21:43:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.390+0000 7f30336c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f302c0d0580 con 0x7f302c0b5f20 2024-04-26T21:43:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.390+0000 7f30336c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f302c0d1800 con 0x7f302c162880 2024-04-26T21:43:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.390+0000 7f3031c65640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f302c0d39f0 0x7f302c0b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42606/0 (socket says 172.21.15.40:42606) 2024-04-26T21:43:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.390+0000 7f3031c65640 1 -- 172.21.15.40:0/3186380871 learned_addr learned my addr 172.21.15.40:0/3186380871 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3031464640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c162880 0x7f302c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3030c63640 1 --2- 172.21.15.40:0/3186380871 >> v2:172.21.15.184:3300/0 conn(0x7f302c0b5f20 0x7f302c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 363238001 0 0) 0x7f302c0b7e50 con 0x7f302c0d39f0 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3018003520 con 0x7f302c0d39f0 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1009946315 0 0) 0x7f3018003520 con 0x7f302c0d39f0 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c162880 msgr2=0x7f302c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c162880 0x7f302c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 >> v2:172.21.15.184:3300/0 conn(0x7f302c0b5f20 msgr2=0x7f302c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 --2- 172.21.15.40:0/3186380871 >> v2:172.21.15.184:3300/0 conn(0x7f302c0b5f20 0x7f302c0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c0d07d0 con 0x7f302c0d39f0 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f30280019f0 con 0x7f302c0d39f0 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 >> v1:172.21.15.40:6789/0 conn(0x7f302c0d39f0 legacy=0x7f302c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30180039b0 0x7f3018003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c162880 0x7f3018004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3018004d40 0x7f30180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0d0580 con 0x7f3018004d40 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0d1800 con 0x7f30180039b0 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f301800b9c0 con 0x7f302c162880 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3031c65640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3018004d40 0x7f30180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3031464640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30180039b0 0x7f3018003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3030c63640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c162880 0x7f3018004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3031c65640 1 -- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c162880 msgr2=0x7f3018004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3031c65640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c162880 0x7f3018004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3031c65640 1 -- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30180039b0 msgr2=0x7f3018003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3031c65640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30180039b0 0x7f3018003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.391+0000 7f3031c65640 1 -- 172.21.15.40:0/3186380871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3018003520 con 0x7f3018004d40 2024-04-26T21:43:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.392+0000 7f3031c65640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3018004d40 0x7f30180051c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f302800a8b0 tx=0x7f3018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a0bc24aebdb4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.392+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302800a4d0 con 0x7f3018004d40 2024-04-26T21:43:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.392+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f302802f070 con 0x7f3018004d40 2024-04-26T21:43:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.392+0000 7f3023fff640 1 -- 172.21.15.40:0/3186380871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302802bd10 con 0x7f3018004d40 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.393+0000 7f30336c8640 1 -- 172.21.15.40:0/3186380871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3018004d40 msgr2=0x7f30180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.393+0000 7f30336c8640 1 --2- 172.21.15.40:0/3186380871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3018004d40 0x7f30180051c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f302800a8b0 tx=0x7f3018005830 comp rx=0 tx=0).stop 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.393+0000 7f3031c65640 1 -- 172.21.15.40:0/3186380871 reap_dead start 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.393+0000 7f30336c8640 1 -- 172.21.15.40:0/3186380871 shutdown_connections 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.393+0000 7f30336c8640 1 -- 172.21.15.40:0/3186380871 >> 172.21.15.40:0/3186380871 conn(0x7f302c0d4f60 msgr2=0x7f302c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.393+0000 7f30336c8640 1 -- 172.21.15.40:0/3186380871 shutdown_connections 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.393+0000 7f30336c8640 1 -- 172.21.15.40:0/3186380871 wait complete. 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.394+0000 7f30336c8640 1 Processor -- start 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.394+0000 7f30336c8640 1 -- start start 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f30336c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302c1533e0 0x7f302c1f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f30336c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c0b55b0 0x7f302c0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f30336c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c0b2090 0x7f302c1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f30336c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f302c162c60 con 0x7f302c1533e0 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f30336c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0d0d90 con 0x7f302c0b2090 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f30336c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f302c162de0 con 0x7f302c0b55b0 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302c1533e0 0x7f302c1f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302c1533e0 0x7f302c1f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51418/0 (socket says 172.21.15.40:51418) 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 -- 172.21.15.40:0/2484284177 learned_addr learned my addr 172.21.15.40:0/2484284177 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 -- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c0b55b0 msgr2=0x7f302c0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c0b55b0 0x7f302c0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 -- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c0b2090 msgr2=0x7f302c1f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c0b2090 0x7f302c1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 -- 172.21.15.40:0/2484284177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c1f8b30 con 0x7f302c1533e0 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.395+0000 7f3031464640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302c1533e0 0x7f302c1f84a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f3024011910 tx=0x7f3024011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.396+0000 7f3021ffb640 1 -- 172.21.15.40:0/2484284177 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302400fa10 con 0x7f302c1533e0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.396+0000 7f3021ffb640 1 -- 172.21.15.40:0/2484284177 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f302400fbb0 con 0x7f302c1533e0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.396+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c1f8e10 con 0x7f302c1533e0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.396+0000 7f3021ffb640 1 -- 172.21.15.40:0/2484284177 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024014440 con 0x7f302c1533e0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.396+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c1f5050 con 0x7f302c1533e0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.397+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c1f4e50 con 0x7f302c1533e0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.397+0000 7f3021ffb640 1 -- 172.21.15.40:0/2484284177 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3024012040 con 0x7f302c1533e0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.398+0000 7f3021ffb640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f301006c8f0 0x7f301006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.398+0000 7f3021ffb640 1 -- 172.21.15.40:0/2484284177 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7f302408c6d0 con 0x7f302c1533e0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.398+0000 7f3030c63640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f301006c8f0 0x7f301006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.398+0000 7f3030c63640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f301006c8f0 0x7f301006edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f301c006e80 tx=0x7f301c006df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.402+0000 7f3021ffb640 1 -- 172.21.15.40:0/2484284177 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3024019030 con 0x7f302c1533e0 2024-04-26T21:43:58.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.602+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f302c0694f0 con 0x7f302c1533e0 2024-04-26T21:43:58.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.602+0000 7f3021ffb640 1 -- 172.21.15.40:0/2484284177 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 24 entries v107) ==== 81+0+1464 (secure 0 0 0) 0x7f302405a4e0 con 0x7f302c1533e0 2024-04-26T21:43:58.603 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:43:58.603 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:43:58.603 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:43:58.603 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:43:58.604 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:43:58.605 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:43:58.605 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:43:58.605 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:43:58.605 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:43:58.605 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:43:58.605 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:43:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:listed 24 entries 2024-04-26T21:43:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f301006c8f0 msgr2=0x7f301006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f301006c8f0 0x7f301006edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f301c006e80 tx=0x7f301c006df0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302c1533e0 msgr2=0x7f302c1f84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302c1533e0 0x7f302c1f84a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f3024011910 tx=0x7f3024011dd0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 shutdown_connections 2024-04-26T21:43:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f301006c8f0 0x7f301006edb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c0b2090 0x7f302c1f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f302c0b55b0 0x7f302c0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 --2- 172.21.15.40:0/2484284177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302c1533e0 0x7f302c1f84a0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 >> 172.21.15.40:0/2484284177 conn(0x7f302c0d4f60 msgr2=0x7f302c161560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 shutdown_connections 2024-04-26T21:43:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.604+0000 7f30336c8640 1 -- 172.21.15.40:0/2484284177 wait complete. 2024-04-26T21:43:58.616 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:43:58.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:58.616 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:43:58.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:43:58.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:43:58.641 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:43:58.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:43:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.839+0000 7fc3c71a4640 1 Processor -- start 2024-04-26T21:43:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.839+0000 7fc3c71a4640 1 -- start start 2024-04-26T21:43:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c71a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3c0162880 0x7fc3c00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c71a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc3c00d3bc0 0x7fc3c00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c71a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c00b7e50 con 0x7fc3c00b5f20 2024-04-26T21:43:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c71a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c00d0580 con 0x7fc3c0162880 2024-04-26T21:43:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c71a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c00d1800 con 0x7fc3c00d3bc0 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c4f40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3c0162880 0x7fc3c00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3bffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc3c00b5f20 0x7fc3c00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43040/0 (socket says 172.21.15.40:43040) 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3bffff640 1 -- 172.21.15.40:0/2226252770 learned_addr learned my addr 172.21.15.40:0/2226252770 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c5741640 1 --2- 172.21.15.40:0/2226252770 >> v2:172.21.15.184:3300/0 conn(0x7fc3c00d3bc0 0x7fc3c00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3bf7fe640 1 -- 172.21.15.40:0/2226252770 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597794343 0 0) 0x7fc3c00b7e50 con 0x7fc3c00b5f20 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3bf7fe640 1 -- 172.21.15.40:0/2226252770 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3b0003520 con 0x7fc3c00b5f20 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c4f40640 1 -- 172.21.15.40:0/2226252770 >> v2:172.21.15.184:3300/0 conn(0x7fc3c00d3bc0 msgr2=0x7fc3c00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c4f40640 1 --2- 172.21.15.40:0/2226252770 >> v2:172.21.15.184:3300/0 conn(0x7fc3c00d3bc0 0x7fc3c00b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c4f40640 1 -- 172.21.15.40:0/2226252770 >> v1:172.21.15.40:6789/0 conn(0x7fc3c00b5f20 legacy=0x7fc3c00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.840+0000 7fc3c4f40640 1 -- 172.21.15.40:0/2226252770 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c00d07d0 con 0x7fc3c0162880 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.841+0000 7fc3c4f40640 1 --2- 172.21.15.40:0/2226252770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00d2cb0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00d11d0 tx=0x7fc3b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38dca8395a12d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.841+0000 7fc3bf7fe640 1 -- 172.21.15.40:0/2226252770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4037070 con 0x7fc3c0162880 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.841+0000 7fc3bf7fe640 1 -- 172.21.15.40:0/2226252770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b4032440 con 0x7fc3c0162880 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.841+0000 7fc3bf7fe640 1 -- 172.21.15.40:0/2226252770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4032810 con 0x7fc3c0162880 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.842+0000 7fc3c71a4640 1 -- 172.21.15.40:0/2226252770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 msgr2=0x7fc3c00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.842+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/2226252770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00d2cb0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00d11d0 tx=0x7fc3b402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.842+0000 7fc3c71a4640 1 -- 172.21.15.40:0/2226252770 shutdown_connections 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.842+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/2226252770 >> v2:172.21.15.184:3300/0 conn(0x7fc3c00d3bc0 0x7fc3c00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.842+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/2226252770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00d2cb0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.842+0000 7fc3c71a4640 1 -- 172.21.15.40:0/2226252770 >> 172.21.15.40:0/2226252770 conn(0x7fc3c00d4f60 msgr2=0x7fc3c00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.842+0000 7fc3c71a4640 1 -- 172.21.15.40:0/2226252770 shutdown_connections 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.842+0000 7fc3c71a4640 1 -- 172.21.15.40:0/2226252770 wait complete. 2024-04-26T21:43:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.843+0000 7fc3c71a4640 1 Processor -- start 2024-04-26T21:43:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.843+0000 7fc3c71a4640 1 -- start start 2024-04-26T21:43:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c71a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3c00b5f20 0x7fc3c00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c71a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3c00d3bc0 0x7fc3c014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c71a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00ba160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c71a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0162c60 con 0x7fc3c0162880 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c71a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c00d0d90 con 0x7fc3c00b5f20 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c71a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c0162de0 con 0x7fc3c00d3bc0 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00ba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59690/0 (socket says 172.21.15.40:59690) 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 -- 172.21.15.40:0/4262670628 learned_addr learned my addr 172.21.15.40:0/4262670628 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 -- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3c00d3bc0 msgr2=0x7fc3c014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c4f40640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3c00b5f20 0x7fc3c00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3c00d3bc0 0x7fc3c014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 -- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3c00b5f20 msgr2=0x7fc3c00b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3c00b5f20 0x7fc3c00b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 -- 172.21.15.40:0/4262670628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c014e8b0 con 0x7fc3c0162880 2024-04-26T21:43:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c5741640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00ba160 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009fd0 tx=0x7fc3b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3bd7fa640 1 -- 172.21.15.40:0/4262670628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8014420 con 0x7fc3c0162880 2024-04-26T21:43:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3bd7fa640 1 -- 172.21.15.40:0/4262670628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8019070 con 0x7fc3c0162880 2024-04-26T21:43:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c014eb90 con 0x7fc3c0162880 2024-04-26T21:43:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3bd7fa640 1 -- 172.21.15.40:0/4262670628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8014800 con 0x7fc3c0162880 2024-04-26T21:43:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.844+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c014f150 con 0x7fc3c0162880 2024-04-26T21:43:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.846+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c00694f0 con 0x7fc3c0162880 2024-04-26T21:43:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.846+0000 7fc3bd7fa640 1 -- 172.21.15.40:0/4262670628 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3b80149a0 con 0x7fc3c0162880 2024-04-26T21:43:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.847+0000 7fc3bd7fa640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3a406c8f0 0x7fc3a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.847+0000 7fc3bd7fa640 1 -- 172.21.15.40:0/4262670628 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7249+0+0 (secure 0 0 0) 0x7fc3b808c940 con 0x7fc3c0162880 2024-04-26T21:43:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.847+0000 7fc3c4f40640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3a406c8f0 0x7fc3a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.847+0000 7fc3c4f40640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3a406c8f0 0x7fc3a406edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4034000 tx=0x7fc3b402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:58.851+0000 7fc3bd7fa640 1 -- 172.21.15.40:0/4262670628 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3b805a760 con 0x7fc3c0162880 2024-04-26T21:43:59.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.076+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc3c00d2d50 con 0x7fc3c0162880 2024-04-26T21:43:59.077 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:43:59.076+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:43:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.078+0000 7fc3bd7fa640 1 -- 172.21.15.40:0/4262670628 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v170) ==== 161+0+0 (secure 0 0 0) 0x7fc3b805a060 con 0x7fc3c0162880 2024-04-26T21:43:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.080+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3a406c8f0 msgr2=0x7fc3a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.080+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3a406c8f0 0x7fc3a406edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4034000 tx=0x7fc3b402e000 comp rx=0 tx=0).stop 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.080+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 msgr2=0x7fc3c00ba160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.080+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00ba160 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009fd0 tx=0x7fc3b800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.081+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 shutdown_connections 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.081+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3a406c8f0 0x7fc3a406edb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.081+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c0162880 0x7fc3c00ba160 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.081+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3c00d3bc0 0x7fc3c014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.081+0000 7fc3c71a4640 1 --2- 172.21.15.40:0/4262670628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3c00b5f20 0x7fc3c00b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.081+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 >> 172.21.15.40:0/4262670628 conn(0x7fc3c00d4f60 msgr2=0x7fc3c0160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.081+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 shutdown_connections 2024-04-26T21:43:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.081+0000 7fc3c71a4640 1 -- 172.21.15.40:0/4262670628 wait complete. 2024-04-26T21:43:59.083 INFO:tasks.ceph.mds.c.smithi040.stderr: -617> 2024-04-26T21:43:56.025+0000 7fbc3e243640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:43:59.093 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:43:59.108 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:43:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.194+0000 7f71e19c6640 1 Processor -- start 2024-04-26T21:43:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.194+0000 7f71e19c6640 1 -- start start 2024-04-26T21:43:59.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71e19c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71e19c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71dc0b2340 0x7f71dc0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71e19c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71dc0cd930 0x7f71dc0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71e19c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f71dc0d0750 con 0x7f71dc0b2340 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71e19c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f71dc0d16a0 con 0x7f71dc0cd930 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71e19c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71dc0d10d0 con 0x7f71dc162810 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71da7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71dc0b2340 0x7f71dc0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71da7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71dc0b2340 0x7f71dc0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59724/0 (socket says 172.21.15.40:59724) 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71da7fc640 1 -- 172.21.15.40:0/3063784209 learned_addr learned my addr 172.21.15.40:0/3063784209 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71daffd640 1 --2- 172.21.15.40:0/3063784209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71db7fe640 1 --2- 172.21.15.40:0/3063784209 >> v2:172.21.15.184:3300/0 conn(0x7f71dc0cd930 0x7f71dc0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71da7fc640 1 -- 172.21.15.40:0/3063784209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 msgr2=0x7f71dc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71da7fc640 1 --2- 172.21.15.40:0/3063784209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71da7fc640 1 -- 172.21.15.40:0/3063784209 >> v2:172.21.15.184:3300/0 conn(0x7f71dc0cd930 msgr2=0x7f71dc0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:43:59.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71da7fc640 1 --2- 172.21.15.40:0/3063784209 >> v2:172.21.15.184:3300/0 conn(0x7f71dc0cd930 0x7f71dc0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71da7fc640 1 -- 172.21.15.40:0/3063784209 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc0d1370 con 0x7f71dc0b2340 2024-04-26T21:43:59.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.195+0000 7f71daffd640 1 --2- 172.21.15.40:0/3063784209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:43:59.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.196+0000 7f71da7fc640 1 --2- 172.21.15.40:0/3063784209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0b2340 0x7f71dc0cd3f0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b2250 tx=0x7f71d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4778d9df80a23b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:59.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.198+0000 7f71d9ffb640 1 -- 172.21.15.40:0/3063784209 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0037070 con 0x7f71dc0b2340 2024-04-26T21:43:59.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.198+0000 7f71d9ffb640 1 -- 172.21.15.40:0/3063784209 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d0032440 con 0x7f71dc0b2340 2024-04-26T21:43:59.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.198+0000 7f71d9ffb640 1 -- 172.21.15.40:0/3063784209 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0032810 con 0x7f71dc0b2340 2024-04-26T21:43:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 -- 172.21.15.40:0/3063784209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0b2340 msgr2=0x7f71dc0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 --2- 172.21.15.40:0/3063784209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0b2340 0x7f71dc0cd3f0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b2250 tx=0x7f71d002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 -- 172.21.15.40:0/3063784209 shutdown_connections 2024-04-26T21:43:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 --2- 172.21.15.40:0/3063784209 >> v2:172.21.15.184:3300/0 conn(0x7f71dc0cd930 0x7f71dc0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 --2- 172.21.15.40:0/3063784209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0b2340 0x7f71dc0cd3f0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 --2- 172.21.15.40:0/3063784209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 -- 172.21.15.40:0/3063784209 >> 172.21.15.40:0/3063784209 conn(0x7f71dc0d2da0 msgr2=0x7f71dc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 -- 172.21.15.40:0/3063784209 shutdown_connections 2024-04-26T21:43:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.199+0000 7f71e19c6640 1 -- 172.21.15.40:0/3063784209 wait complete. 2024-04-26T21:43:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.200+0000 7f71e19c6640 1 Processor -- start 2024-04-26T21:43:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71e19c6640 1 -- start start 2024-04-26T21:43:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71e19c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71dc0b2340 0x7f71dc14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71e19c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0cd930 0x7f71dc15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71e19c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc15d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71e19c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc070890 con 0x7f71dc0cd930 2024-04-26T21:43:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71e19c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc0d0470 con 0x7f71dc0b2340 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71e19c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71dc0d2070 con 0x7f71dc162810 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71da7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0cd930 0x7f71dc15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71da7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0cd930 0x7f71dc15d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59736/0 (socket says 172.21.15.40:59736) 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71da7fc640 1 -- 172.21.15.40:0/4007825917 learned_addr learned my addr 172.21.15.40:0/4007825917 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71db7fe640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.201+0000 7f71daffd640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71dc0b2340 0x7f71dc14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71da7fc640 1 -- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 msgr2=0x7f71dc15d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71da7fc640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc15d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71da7fc640 1 -- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71dc0b2340 msgr2=0x7f71dc14d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71da7fc640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71dc0b2340 0x7f71dc14d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71da7fc640 1 -- 172.21.15.40:0/4007825917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc15f1d0 con 0x7f71dc0cd930 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71daffd640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71dc0b2340 0x7f71dc14d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:43:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71da7fc640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0cd930 0x7f71dc15d3d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f71d0000c00 tx=0x7f71d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71c3fff640 1 -- 172.21.15.40:0/4007825917 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0037040 con 0x7f71dc0cd930 2024-04-26T21:43:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71c3fff640 1 -- 172.21.15.40:0/4007825917 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d0042070 con 0x7f71dc0cd930 2024-04-26T21:43:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71c3fff640 1 -- 172.21.15.40:0/4007825917 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0030bb0 con 0x7f71dc0cd930 2024-04-26T21:43:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc15f4b0 con 0x7f71dc0cd930 2024-04-26T21:43:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.202+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc15fa70 con 0x7f71dc0cd930 2024-04-26T21:43:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.203+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc0694f0 con 0x7f71dc0cd930 2024-04-26T21:43:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.204+0000 7f71c3fff640 1 -- 172.21.15.40:0/4007825917 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71d0047460 con 0x7f71dc0cd930 2024-04-26T21:43:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.204+0000 7f71c3fff640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c806c8f0 0x7f71c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.204+0000 7f71c3fff640 1 -- 172.21.15.40:0/4007825917 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7335+0+0 (secure 0 0 0) 0x7f71d00accd0 con 0x7f71dc0cd930 2024-04-26T21:43:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.204+0000 7f71daffd640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c806c8f0 0x7f71c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.205+0000 7f71daffd640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c806c8f0 0x7f71c806edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00a990 tx=0x7f71cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:59.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.208+0000 7f71c3fff640 1 -- 172.21.15.40:0/4007825917 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71d007ab30 con 0x7f71dc0cd930 2024-04-26T21:43:59.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:43:59.472+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f71dc0d2430 con 0x7f71dc0cd930 2024-04-26T21:44:00.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.079+0000 7f71c3fff640 1 -- 172.21.15.40:0/4007825917 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v171) ==== 107+0+0 (secure 0 0 0) 0x7f71d007a430 con 0x7f71dc0cd930 2024-04-26T21:44:00.080 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c806c8f0 msgr2=0x7f71c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c806c8f0 0x7f71c806edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00a990 tx=0x7f71cc008040 comp rx=0 tx=0).stop 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0cd930 msgr2=0x7f71dc15d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0cd930 0x7f71dc15d3d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f71d0000c00 tx=0x7f71d00046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 shutdown_connections 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71c806c8f0 0x7f71c806edb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71dc162810 0x7f71dc15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71dc0cd930 0x7f71dc15d3d0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 --2- 172.21.15.40:0/4007825917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71dc0b2340 0x7f71dc14d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 >> 172.21.15.40:0/4007825917 conn(0x7f71dc0d2da0 msgr2=0x7f71dc06e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:00.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 shutdown_connections 2024-04-26T21:44:00.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.082+0000 7f71e19c6640 1 -- 172.21.15.40:0/4007825917 wait complete. 2024-04-26T21:44:00.096 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:00.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.182+0000 7f2f05b28f40 1 Processor -- start 2024-04-26T21:44:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.182+0000 7f2f05b28f40 1 -- start start 2024-04-26T21:44:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.182+0000 7f2f05b28f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 0x5619fe461190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.182+0000 7f2f05b28f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 0x5619fe52f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.182+0000 7f2f05b28f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.182+0000 7f2f05b28f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5619fe407e40 con 0x5619fe538b70 2024-04-26T21:44:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.182+0000 7f2f05b28f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5619fe4a9660 con 0x5619fe531f70 2024-04-26T21:44:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.182+0000 7f2f05b28f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5619fe531550 con 0x5619fe4aa420 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59750/0 (socket says 172.21.15.40:59750) 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 -- 172.21.15.40:0/434725481 learned_addr learned my addr 172.21.15.40:0/434725481 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03624640 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 0x5619fe461190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2efbfff640 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 0x5619fe52f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 -- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 msgr2=0x5619fe461190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 0x5619fe461190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 -- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 msgr2=0x5619fe52f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 0x5619fe52f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 -- 172.21.15.40:0/434725481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fe4616d0 con 0x5619fe538b70 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2efbfff640 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 0x5619fe52f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f03e25640 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0015c0 tx=0x7f2efc001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184c5d3299c422ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f02622640 1 -- 172.21.15.40:0/434725481 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc003650 con 0x5619fe538b70 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f02622640 1 -- 172.21.15.40:0/434725481 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2efc0037f0 con 0x5619fe538b70 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.183+0000 7f2f02622640 1 -- 172.21.15.40:0/434725481 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc003c60 con 0x5619fe538b70 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 -- 172.21.15.40:0/434725481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 msgr2=0x5619fe48ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0015c0 tx=0x7f2efc001a80 comp rx=0 tx=0).stop 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 -- 172.21.15.40:0/434725481 shutdown_connections 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 0x5619fe52f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/434725481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 0x5619fe461190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 -- 172.21.15.40:0/434725481 >> 172.21.15.40:0/434725481 conn(0x5619fe48a8a0 msgr2=0x5619fe53bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 -- 172.21.15.40:0/434725481 shutdown_connections 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 -- 172.21.15.40:0/434725481 wait complete. 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 Processor -- start 2024-04-26T21:44:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.184+0000 7f2f05b28f40 1 -- start start 2024-04-26T21:44:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f05b28f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 0x5619fe532350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f05b28f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 0x5619fe52b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f05b28f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f05b28f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5619fe5312b0 con 0x5619fe538b70 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f05b28f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5619fe48fa30 con 0x5619fe531f70 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f05b28f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5619fe52fb90 con 0x5619fe4aa420 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f03e25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f03e25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59764/0 (socket says 172.21.15.40:59764) 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 learned_addr learned my addr 172.21.15.40:0/2871073750 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2efbfff640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 0x5619fe52b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.185+0000 7f2f03624640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 0x5619fe532350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 msgr2=0x5619fe532350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f03e25640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619fe4aa420 0x5619fe532350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 msgr2=0x5619fe52b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f03e25640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619fe531f70 0x5619fe52b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fe4a6ef0 con 0x5619fe538b70 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f03e25640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f2efc000f80 tx=0x7f2efc001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f01620640 1 -- 172.21.15.40:0/2871073750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc024070 con 0x5619fe538b70 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f01620640 1 -- 172.21.15.40:0/2871073750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2efc019040 con 0x5619fe538b70 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5619fe52e840 con 0x5619fe538b70 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f01620640 1 -- 172.21.15.40:0/2871073750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc013880 con 0x5619fe538b70 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.186+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fe52bb50 con 0x5619fe538b70 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.187+0000 7f2f01620640 1 -- 172.21.15.40:0/2871073750 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2efc010070 con 0x5619fe538b70 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.188+0000 7f2f01620640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ee406b2d0 0x7f2ee406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.188+0000 7f2f01620640 1 -- 172.21.15.40:0/2871073750 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7335+0+0 (secure 0 0 0) 0x7f2efc08bb20 con 0x5619fe538b70 2024-04-26T21:44:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.188+0000 7f2f03624640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ee406b2d0 0x7f2ee406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.192 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.188+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5619fe48bf90 0x5619fe4a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.188+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe4a4eb0 con 0x5619fe48bf90 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.188+0000 7f2efbfff640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5619fe48bf90 0x5619fe4a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.189+0000 7f2f03624640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ee406b2d0 0x7f2ee406d790 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009740 tx=0x7f2ef4005a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.189+0000 7f2efbfff640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5619fe48bf90 0x5619fe4a4970 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.190+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v108'381 uv355 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.190+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe4a5250 con 0x5619fe48bf90 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.191+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v108'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.191+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5dab40 con 0x5619fe48bf90 2024-04-26T21:44:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.192+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v108'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.194 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.192+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5dbef0 con 0x5619fe48bf90 2024-04-26T21:44:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.193+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v108'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.193+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5dc290 con 0x5619fe48bf90 2024-04-26T21:44:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.194+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v108'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.194+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5daee0 con 0x5619fe48bf90 2024-04-26T21:44:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.196+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v108'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.196+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db2f0 con 0x5619fe48bf90 2024-04-26T21:44:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.197+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v108'387 uv361 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.197+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5db6e0 con 0x5619fe48bf90 2024-04-26T21:44:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.198+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v108'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.198+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db6e0 con 0x5619fe48bf90 2024-04-26T21:44:00.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.200+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v108'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.201 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:00.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.200+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.202+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v108'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.202+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.202+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v108'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.203+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.203+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v108'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.204+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.204+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v108'393 uv368 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.204+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.205+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v108'394 uv394 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.205+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.206+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v108'395 uv395 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.207 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.206+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.207+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v108'396 uv396 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.207+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.208+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v108'397 uv397 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.208+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db690 con 0x5619fe48bf90 2024-04-26T21:44:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.209+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v108'398 uv398 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.209+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5dba90 con 0x5619fe48bf90 2024-04-26T21:44:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.210+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v108'399 uv399 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.210+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.212+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v108'400 uv374 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.212+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.213+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v108'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.213+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.214+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v108'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.215 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:00.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.214+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.215+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v108'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.215+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.216+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v108'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.216+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.217+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v108'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.217+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.218+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v108'406 uv380 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.218+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.219+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v108'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.219+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.220+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v108'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.221 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:00.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.220+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.221+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v108'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.221+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.222+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v108'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.222+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5db940 con 0x5619fe48bf90 2024-04-26T21:44:00.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.223+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v108'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe48bf90 2024-04-26T21:44:00.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.223+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5619fe5dc630 0x5619fe5dea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.223+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5def60 con 0x5619fe5dc630 2024-04-26T21:44:00.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.224+0000 7f2f03624640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5619fe5dc630 0x5619fe5dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.224+0000 7f2f03624640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5619fe5dc630 0x5619fe5dea20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.226+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v108'287 uv266 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.226+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5df300 con 0x5619fe5dc630 2024-04-26T21:44:00.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.227+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v108'288 uv288 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.227+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5df6a0 con 0x5619fe5dc630 2024-04-26T21:44:00.229 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:00.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.228+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v108'289 uv289 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.228+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5dfef0 con 0x5619fe5dc630 2024-04-26T21:44:00.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.229+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v108'290 uv290 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.229+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0290 con 0x5619fe5dc630 2024-04-26T21:44:00.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.231+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v108'291 uv291 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.231+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0630 con 0x5619fe5dc630 2024-04-26T21:44:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.232+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v108'292 uv285 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.232+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5e09d0 con 0x5619fe5dc630 2024-04-26T21:44:00.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.233+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v108'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.233+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.235 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:00.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.234+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v108'294 uv294 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.234+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.236+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v108'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.236+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.237+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v108'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.237+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.238+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v108'297 uv286 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.238+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.240+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v108'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.240+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.241 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:00.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.241+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v108'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.241+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e5430 con 0x5619fe5dc630 2024-04-26T21:44:00.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.242+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v108'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.242+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e5430 con 0x5619fe5dc630 2024-04-26T21:44:00.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.244+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v108'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.244+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.245+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v108'302 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.245+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.246+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v108'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.246+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.247+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v108'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.248 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.247+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e1d80 con 0x5619fe5dc630 2024-04-26T21:44:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.248+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v108'305 uv305 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.248+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e1d80 con 0x5619fe5dc630 2024-04-26T21:44:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.249+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v108'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.250+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e0d70 con 0x5619fe5dc630 2024-04-26T21:44:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.251+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v108'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5dc630 2024-04-26T21:44:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.251+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5619fe5e1280 0x5619fe5e3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.251+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e3bb0 con 0x5619fe5e1280 2024-04-26T21:44:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.251+0000 7f2efbfff640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5619fe5e1280 0x5619fe5e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.251+0000 7f2efbfff640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5619fe5e1280 0x5619fe5e3670 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.253+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v108'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.253+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5e3bb0 con 0x5619fe5e1280 2024-04-26T21:44:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.254+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v108'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.254+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e3c30 con 0x5619fe5e1280 2024-04-26T21:44:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.255+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v108'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.256 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.255+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e5030 con 0x5619fe5e1280 2024-04-26T21:44:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.256+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v108'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.256+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e5030 con 0x5619fe5e1280 2024-04-26T21:44:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.257+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v108'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.257+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e4020 con 0x5619fe5e1280 2024-04-26T21:44:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.258+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v108'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.259+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5619fe5e4530 0x5619fe5e6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.259+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e6f60 con 0x5619fe5e4530 2024-04-26T21:44:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.259+0000 7f2f03e25640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5619fe5e4530 0x5619fe5e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.259+0000 7f2f03e25640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5619fe5e4530 0x5619fe5e6920 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.260+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v108'64 uv63 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.260+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5e7470 con 0x5619fe5e4530 2024-04-26T21:44:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.262+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v108'65 uv65 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.262+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e7970 con 0x5619fe5e4530 2024-04-26T21:44:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.264+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v108'66 uv66 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.265 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.264+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5e8230 con 0x5619fe5e4530 2024-04-26T21:44:00.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.266+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v108'67 uv67 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.266+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5619fe5e8ae0 0x5619fe5eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.266+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5eb450 con 0x5619fe5e8ae0 2024-04-26T21:44:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.266+0000 7f2f03624640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5619fe5e8ae0 0x5619fe5eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.266+0000 7f2f03624640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5619fe5e8ae0 0x5619fe5eaed0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.268+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v108'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.268+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5eb510 con 0x5619fe5e8ae0 2024-04-26T21:44:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.272+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v108'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.272+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5eb930 con 0x5619fe5e8ae0 2024-04-26T21:44:00.275 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:00.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.274+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v108'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.274+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec940 con 0x5619fe5e8ae0 2024-04-26T21:44:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.278+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v108'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.278+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec940 con 0x5619fe5e8ae0 2024-04-26T21:44:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.280+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v108'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.280+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5eb930 con 0x5619fe5e8ae0 2024-04-26T21:44:00.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.283+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v108'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.283+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5eb930 con 0x5619fe5e4530 2024-04-26T21:44:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.284+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v108'138 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.284+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ebcd0 con 0x5619fe5e4530 2024-04-26T21:44:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.285+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v108'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.285+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.286 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.286+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v108'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.286+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ed080 con 0x5619fe5e4530 2024-04-26T21:44:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.287+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v108'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.287+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ed080 con 0x5619fe5e4530 2024-04-26T21:44:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.288+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v108'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.288+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.289+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v108'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.289+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.290+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v108'144 uv137 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.290+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.291+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v108'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.291+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.292 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.292+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v108'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.292+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.293+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v108'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.293+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.294+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v108'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.294+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.295+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v108'61 uv60 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.295+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.297+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v108'62 uv62 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.297+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.298 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.298+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v108'63 uv63 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.298+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.299+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v108'64 uv64 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.299+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e4530 2024-04-26T21:44:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.300+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v108'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2efc00f030 con 0x5619fe5e4530 2024-04-26T21:44:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.300+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e8ae0 2024-04-26T21:44:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.302+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v108'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.302+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e8ae0 2024-04-26T21:44:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.303+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v108'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.303+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec070 con 0x5619fe5e8ae0 2024-04-26T21:44:00.304 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.304+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v108'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.304+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ed470 con 0x5619fe5e8ae0 2024-04-26T21:44:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.305+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v108'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.305+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ed470 con 0x5619fe5e8ae0 2024-04-26T21:44:00.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.306+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v108'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.306+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.307+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v108'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.307+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.308+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v108'122 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.308+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.309+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v108'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.309+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.311 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:00.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.310+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v108'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.310+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.311+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v108'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.311+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.312+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v108'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.312+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.313+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v108'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.313+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.314+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v108'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.314+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.315 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:00.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.315+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v108'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.315+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ec460 con 0x5619fe5e8ae0 2024-04-26T21:44:00.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.316+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v108'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.316+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef270 con 0x5619fe5e8ae0 2024-04-26T21:44:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.317+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v108'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.317+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5619fe5ece20 0x5619fe5ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.317+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5ece20 2024-04-26T21:44:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.317+0000 7f2efbfff640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5619fe5ece20 0x5619fe5ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.318+0000 7f2efbfff640 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5619fe5ece20 0x5619fe5ef6f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.319+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v108'57 uv55 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5ece20 2024-04-26T21:44:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.319+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5ece20 2024-04-26T21:44:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.320+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v108'58 uv58 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5ece20 2024-04-26T21:44:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.320+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5ece20 2024-04-26T21:44:00.323 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.322+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v108'59 uv59 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5ece20 2024-04-26T21:44:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.322+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5ece20 2024-04-26T21:44:00.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.325+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v108'60 uv60 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5ece20 2024-04-26T21:44:00.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.325+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e8ae0 2024-04-26T21:44:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.326+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v108'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.326+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e8ae0 2024-04-26T21:44:00.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.327+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v108'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.327+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e8ae0 2024-04-26T21:44:00.329 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:00.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.328+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v108'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.328+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e8ae0 2024-04-26T21:44:00.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.329+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v108'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.330+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e8ae0 2024-04-26T21:44:00.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.331+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v108'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.331+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e8ae0 2024-04-26T21:44:00.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.332+0000 7f2f03624640 1 -- 172.21.15.40:0/2871073750 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v108'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ef4019040 con 0x5619fe5e8ae0 2024-04-26T21:44:00.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.332+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e1280 2024-04-26T21:44:00.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.333+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v108'64 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.333+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e1280 2024-04-26T21:44:00.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.334+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v108'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.334+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5ef210 con 0x5619fe5e1280 2024-04-26T21:44:00.336 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.336+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v108'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.336+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.337+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v108'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.337+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.338+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v108'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.338+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.339+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v108'157 uv148 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.339+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.340+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v108'158 uv158 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.340+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.341+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v108'159 uv159 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.342 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:00.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.341+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.342+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v108'160 uv160 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.342+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.343+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v108'161 uv161 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.343+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.344+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v108'162 uv155 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.344+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.345+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v108'163 uv163 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.345+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.346+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v108'164 uv164 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.346 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.346+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5f0ce0 con 0x5619fe5e1280 2024-04-26T21:44:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.347+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v108'165 uv165 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.347+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5f0ce0 con 0x5619fe5e1280 2024-04-26T21:44:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.348+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v108'166 uv166 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.348+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efc30 con 0x5619fe5e1280 2024-04-26T21:44:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.348+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v108'167 uv167 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5619fe5efff0 con 0x5619fe5e1280 2024-04-26T21:44:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2efbfff640 1 -- 172.21.15.40:0/2871073750 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v108'168 uv168 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2eec056450 con 0x5619fe5e1280 2024-04-26T21:44:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5619fe5dc630 msgr2=0x5619fe5dea20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5619fe5dc630 0x5619fe5dea20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5619fe48bf90 msgr2=0x5619fe4a4970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5619fe48bf90 0x5619fe4a4970 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5619fe5e4530 msgr2=0x5619fe5e6920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5619fe5e4530 0x5619fe5e6920 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5619fe5ece20 msgr2=0x5619fe5ef6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5619fe5ece20 0x5619fe5ef6f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5619fe5e1280 msgr2=0x5619fe5e3670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5619fe5e1280 0x5619fe5e3670 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5619fe5e8ae0 msgr2=0x5619fe5eaed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5619fe5e8ae0 0x5619fe5eaed0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 reap_dead start 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 reap_dead start 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.349+0000 7f2f03e25640 1 -- 172.21.15.40:0/2871073750 reap_dead start 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ee406b2d0 msgr2=0x7f2ee406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ee406b2d0 0x7f2ee406d790 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009740 tx=0x7f2ef4005a00 comp rx=0 tx=0).stop 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 msgr2=0x5619fe48ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f2efc000f80 tx=0x7f2efc001250 comp rx=0 tx=0).stop 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 shutdown_connections 2024-04-26T21:44:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2ee406b2d0 0x7f2ee406d790 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 --2- 172.21.15.40:0/2871073750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619fe538b70 0x5619fe48ba50 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 >> 172.21.15.40:0/2871073750 conn(0x5619fe48a8a0 msgr2=0x5619fe52f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:00.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 shutdown_connections 2024-04-26T21:44:00.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.350+0000 7f2f05b28f40 1 -- 172.21.15.40:0/2871073750 wait complete. 2024-04-26T21:44:00.354 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000.inode 2024-04-26T21:44:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61faf1af40 1 Processor -- start 2024-04-26T21:44:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61faf1af40 1 -- start start 2024-04-26T21:44:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61faf1af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a336d420 0x55d2a33fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61faf1af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3325280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61faf1af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a33fbb70 0x55d2a33fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61faf1af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d2a32cae40 con 0x55d2a33f4f70 2024-04-26T21:44:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61faf1af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d2a336c660 con 0x55d2a336d420 2024-04-26T21:44:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61faf1af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d2a334e700 con 0x55d2a33fbb70 2024-04-26T21:44:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61f77af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3325280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61f77af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3325280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59782/0 (socket says 172.21.15.40:59782) 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.435+0000 7f61f77af640 1 -- 172.21.15.40:0/986486269 learned_addr learned my addr 172.21.15.40:0/986486269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f87b1640 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a33fbb70 0x55d2a33fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f7fb0640 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a336d420 0x55d2a33fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f77af640 1 -- 172.21.15.40:0/986486269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a33fbb70 msgr2=0x55d2a33fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f77af640 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a33fbb70 0x55d2a33fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f77af640 1 -- 172.21.15.40:0/986486269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a336d420 msgr2=0x55d2a33fd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f77af640 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a336d420 0x55d2a33fd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f77af640 1 -- 172.21.15.40:0/986486269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a33ebdd0 con 0x55d2a33f4f70 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f77af640 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3325280 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c810 tx=0x7f61e000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbbf814944b94e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f67ad640 1 -- 172.21.15.40:0/986486269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e000adb0 con 0x55d2a33f4f70 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f67ad640 1 -- 172.21.15.40:0/986486269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e0008410 con 0x55d2a33f4f70 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.436+0000 7f61f67ad640 1 -- 172.21.15.40:0/986486269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0008800 con 0x55d2a33f4f70 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 -- 172.21.15.40:0/986486269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 msgr2=0x55d2a3325280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3325280 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c810 tx=0x7f61e000ccd0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 -- 172.21.15.40:0/986486269 shutdown_connections 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a33fbb70 0x55d2a33fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3325280 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 --2- 172.21.15.40:0/986486269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a336d420 0x55d2a33fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 -- 172.21.15.40:0/986486269 >> 172.21.15.40:0/986486269 conn(0x55d2a334d8a0 msgr2=0x55d2a334dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 -- 172.21.15.40:0/986486269 shutdown_connections 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 -- 172.21.15.40:0/986486269 wait complete. 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 Processor -- start 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.437+0000 7f61faf1af40 1 -- start start 2024-04-26T21:44:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61faf1af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a336d420 0x55d2a33681c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61faf1af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3368700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61faf1af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a33fbb70 0x55d2a33eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61faf1af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d2a33ec040 con 0x55d2a33f4f70 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61faf1af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d2a3325c70 con 0x55d2a33fbb70 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61faf1af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d2a334ef40 con 0x55d2a336d420 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f77af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3368700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f77af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3368700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59784/0 (socket says 172.21.15.40:59784) 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f77af640 1 -- 172.21.15.40:0/3377278808 learned_addr learned my addr 172.21.15.40:0/3377278808 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f87b1640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a33fbb70 0x55d2a33eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f7fb0640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a336d420 0x55d2a33681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f77af640 1 -- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a336d420 msgr2=0x55d2a33681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f77af640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d2a336d420 0x55d2a33681c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f77af640 1 -- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a33fbb70 msgr2=0x55d2a33eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f77af640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2a33fbb70 0x55d2a33eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.438+0000 7f61f77af640 1 -- 172.21.15.40:0/3377278808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a34895b0 con 0x55d2a33f4f70 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.439+0000 7f61f77af640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3368700 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005000 tx=0x7f61e0003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.439+0000 7f61f57ab640 1 -- 172.21.15.40:0/3377278808 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0026070 con 0x55d2a33f4f70 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.439+0000 7f61f57ab640 1 -- 172.21.15.40:0/3377278808 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e0008cf0 con 0x55d2a33f4f70 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.439+0000 7f61f57ab640 1 -- 172.21.15.40:0/3377278808 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e001b040 con 0x55d2a33f4f70 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.439+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2a33f40a0 con 0x55d2a33f4f70 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.439+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a33eb7f0 con 0x55d2a33f4f70 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.440+0000 7f61f57ab640 1 -- 172.21.15.40:0/3377278808 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61e0029070 con 0x55d2a33f4f70 2024-04-26T21:44:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.441+0000 7f61f57ab640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61d806b2d0 0x7f61d806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.441+0000 7f61f57ab640 1 -- 172.21.15.40:0/3377278808 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7335+0+0 (secure 0 0 0) 0x7f61e008cca0 con 0x55d2a33f4f70 2024-04-26T21:44:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.441+0000 7f61f7fb0640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61d806b2d0 0x7f61d806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.441+0000 7f61faf1af40 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d2a334d070 0x55d2a33674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.441+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x55d2a3367a00 con 0x55d2a334d070 2024-04-26T21:44:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.441+0000 7f61f87b1640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d2a334d070 0x55d2a33674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.441+0000 7f61f7fb0640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61d806b2d0 0x7f61d806d790 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f61f0057000 tx=0x7f61f0062040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.441+0000 7f61f87b1640 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d2a334d070 0x55d2a33674c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61f87b1640 1 -- 172.21.15.40:0/3377278808 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 100.00000000.inode [delete] v108'169 uv169 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f61e80105a0 con 0x55d2a334d070 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d2a334d070 msgr2=0x55d2a33674c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d2a334d070 0x55d2a33674c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61d806b2d0 msgr2=0x7f61d806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61d806b2d0 0x7f61d806d790 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f61f0057000 tx=0x7f61f0062040 comp rx=0 tx=0).stop 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 msgr2=0x55d2a3368700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 --2- 172.21.15.40:0/3377278808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2a33f4f70 0x55d2a3368700 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005000 tx=0x7f61e0003020 comp rx=0 tx=0).stop 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61f87b1640 1 -- 172.21.15.40:0/3377278808 reap_dead start 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 shutdown_connections 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 >> 172.21.15.40:0/3377278808 conn(0x55d2a334d8a0 msgr2=0x55d2a334dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 shutdown_connections 2024-04-26T21:44:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.443+0000 7f61faf1af40 1 -- 172.21.15.40:0/3377278808 wait complete. 2024-04-26T21:44:00.450 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:00.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.591+0000 7f2cc610a640 1 Processor -- start 2024-04-26T21:44:00.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.591+0000 7f2cc610a640 1 -- start start 2024-04-26T21:44:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cc610a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cc015f2d0 0x7f2cc01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cc610a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 0x7f2cc00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cc610a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cc0072f70 con 0x7f2cc0072700 2024-04-26T21:44:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cc610a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc0074010 con 0x7f2cc015f2d0 2024-04-26T21:44:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cc610a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc0073250 con 0x7f2cc006fea0 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cbf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cc015f2d0 0x7f2cc01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cbf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cc015f2d0 0x7f2cc01604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59796/0 (socket says 172.21.15.40:59796) 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cbf7fe640 1 -- 172.21.15.40:0/205447948 learned_addr learned my addr 172.21.15.40:0/205447948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.592+0000 7f2cbffff640 1 --2- 172.21.15.40:0/205447948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 0x7f2cc00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.593+0000 7f2cbf7fe640 1 -- 172.21.15.40:0/205447948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 msgr2=0x7f2cc00ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.593+0000 7f2cbf7fe640 1 --2- 172.21.15.40:0/205447948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 0x7f2cc00ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.593+0000 7f2cbf7fe640 1 -- 172.21.15.40:0/205447948 >> v1:172.21.15.184:6789/0 conn(0x7f2cc0072700 legacy=0x7f2cc006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.593+0000 7f2cbf7fe640 1 -- 172.21.15.40:0/205447948 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc0073530 con 0x7f2cc015f2d0 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.593+0000 7f2cbf7fe640 1 --2- 172.21.15.40:0/205447948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc01604e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000bb10 tx=0x7f2cb002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c034fb152bcfa8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.593+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/205447948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0036020 con 0x7f2cc015f2d0 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/205447948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb0032820 con 0x7f2cc015f2d0 2024-04-26T21:44:00.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cbe7fc640 1 -- 172.21.15.40:0/205447948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0032bf0 con 0x7f2cc015f2d0 2024-04-26T21:44:00.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cc610a640 1 -- 172.21.15.40:0/205447948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 msgr2=0x7f2cc01604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cc610a640 1 --2- 172.21.15.40:0/205447948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc01604e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000bb10 tx=0x7f2cb002c920 comp rx=0 tx=0).stop 2024-04-26T21:44:00.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cc610a640 1 -- 172.21.15.40:0/205447948 shutdown_connections 2024-04-26T21:44:00.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cc610a640 1 --2- 172.21.15.40:0/205447948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 0x7f2cc00ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cc610a640 1 --2- 172.21.15.40:0/205447948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc01604e0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cc610a640 1 -- 172.21.15.40:0/205447948 >> 172.21.15.40:0/205447948 conn(0x7f2cc00d0be0 msgr2=0x7f2cc00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cc610a640 1 -- 172.21.15.40:0/205447948 shutdown_connections 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.594+0000 7f2cc610a640 1 -- 172.21.15.40:0/205447948 wait complete. 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.595+0000 7f2cc610a640 1 Processor -- start 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.595+0000 7f2cc610a640 1 -- start start 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cc610a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 0x7f2cc00c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cc610a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc0072700 0x7f2cc00c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cc610a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cc610a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc015f6b0 con 0x7f2cc015f2d0 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cc610a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0073af0 con 0x7f2cc0072700 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cc610a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc0072b90 con 0x7f2cc006fea0 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cbffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cbffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc015b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59810/0 (socket says 172.21.15.40:59810) 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cbffff640 1 -- 172.21.15.40:0/782487049 learned_addr learned my addr 172.21.15.40:0/782487049 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:00.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cbf7fe640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 0x7f2cc00c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.596+0000 7f2cbeffd640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc0072700 0x7f2cc00c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2cbffff640 1 -- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 msgr2=0x7f2cc00c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2cbffff640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 0x7f2cc00c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2cbffff640 1 -- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc0072700 msgr2=0x7f2cc00c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2cbffff640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc0072700 0x7f2cc00c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2cbffff640 1 -- 172.21.15.40:0/782487049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc015c7a0 con 0x7f2cc015f2d0 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2cbffff640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc015b720 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8007e60 tx=0x7f2cb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2ca7fff640 1 -- 172.21.15.40:0/782487049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8014420 con 0x7f2cc015f2d0 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2ca7fff640 1 -- 172.21.15.40:0/782487049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb8019070 con 0x7f2cc015f2d0 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc015c1e0 con 0x7f2cc015f2d0 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2ca7fff640 1 -- 172.21.15.40:0/782487049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8014820 con 0x7f2cc015f2d0 2024-04-26T21:44:00.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.597+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc015d480 con 0x7f2cc015f2d0 2024-04-26T21:44:00.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.598+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc00694f0 con 0x7f2cc015f2d0 2024-04-26T21:44:00.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.599+0000 7f2ca7fff640 1 -- 172.21.15.40:0/782487049 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cb80149c0 con 0x7f2cc015f2d0 2024-04-26T21:44:00.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.599+0000 7f2ca7fff640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:00.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.599+0000 7f2ca7fff640 1 -- 172.21.15.40:0/782487049 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7335+0+0 (secure 0 0 0) 0x7f2cb808c730 con 0x7f2cc015f2d0 2024-04-26T21:44:00.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.599+0000 7f2cbf7fe640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:00.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.600+0000 7f2cbf7fe640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0037000 tx=0x7f2cb003d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:00.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.603+0000 7f2ca7fff640 1 -- 172.21.15.40:0/782487049 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cb8059f40 con 0x7f2cc015f2d0 2024-04-26T21:44:00.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:00.828+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2cc006de80 con 0x7f2cc015f2d0 2024-04-26T21:44:01.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.202+0000 7f2ca7fff640 1 -- 172.21.15.40:0/782487049 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v172) ==== 162+0+0 (secure 0 0 0) 0x7f2cb805e240 con 0x7f2cc015f2d0 2024-04-26T21:44:01.203 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:01.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 msgr2=0x7f2cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0037000 tx=0x7f2cb003d040 comp rx=0 tx=0).stop 2024-04-26T21:44:01.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 msgr2=0x7f2cc015b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc015b720 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8007e60 tx=0x7f2cb800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 shutdown_connections 2024-04-26T21:44:01.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cc015f2d0 0x7f2cc015b720 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cc0072700 0x7f2cc00c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 --2- 172.21.15.40:0/782487049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cc006fea0 0x7f2cc00c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 >> 172.21.15.40:0/782487049 conn(0x7f2cc00d0be0 msgr2=0x7f2cc0152a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:01.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 shutdown_connections 2024-04-26T21:44:01.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.205+0000 7f2cc610a640 1 -- 172.21.15.40:0/782487049 wait complete. 2024-04-26T21:44:01.213 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:01.212+0000 7f196a290640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-04-26T21:44:01.213 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:01.212+0000 7f196a290640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-26T21:44:01.213 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:01.213+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:01.214 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:01.213+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:01.218 INFO:tasks.ceph.mds.c.smithi040.stderr: -24> 2024-04-26T21:44:01.212+0000 7f196a290640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-04-26T21:44:01.218 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:44:01.212+0000 7f196a290640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-26T21:44:01.219 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:01.244 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:01.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.324+0000 7fe04ac7b640 1 Processor -- start 2024-04-26T21:44:01.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.324+0000 7fe04ac7b640 1 -- start start 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.325+0000 7fe04ac7b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe04415f3a0 0x7fe0440d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.325+0000 7fe04ac7b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe0440d2330 0x7fe0440b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.325+0000 7fe04ac7b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0440d0e20 con 0x7fe0440d1770 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.325+0000 7fe04ac7b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe0440b1d70 con 0x7fe04415f3a0 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.325+0000 7fe04ac7b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe0440b2050 con 0x7fe0440d2330 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.325+0000 7fe048a17640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe04415f3a0 0x7fe0440d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.325+0000 7fe048a17640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe04415f3a0 0x7fe0440d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59852/0 (socket says 172.21.15.40:59852) 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.325+0000 7fe048a17640 1 -- 172.21.15.40:0/2547653950 learned_addr learned my addr 172.21.15.40:0/2547653950 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.326+0000 7fe049218640 1 --2- 172.21.15.40:0/2547653950 >> v2:172.21.15.184:3300/0 conn(0x7fe0440d2330 0x7fe0440b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.326+0000 7fe048a17640 1 -- 172.21.15.40:0/2547653950 >> v2:172.21.15.184:3300/0 conn(0x7fe0440d2330 msgr2=0x7fe0440b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.326+0000 7fe048a17640 1 --2- 172.21.15.40:0/2547653950 >> v2:172.21.15.184:3300/0 conn(0x7fe0440d2330 0x7fe0440b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.326+0000 7fe048a17640 1 -- 172.21.15.40:0/2547653950 >> v1:172.21.15.40:6789/0 conn(0x7fe0440d1770 legacy=0x7fe0440d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.326+0000 7fe048a17640 1 -- 172.21.15.40:0/2547653950 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0440b2330 con 0x7fe04415f3a0 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.326+0000 7fe0437fe640 1 -- 172.21.15.40:0/2547653950 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441518577 0 0) 0x7fe0440d0e20 con 0x7fe0440d1770 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.326+0000 7fe048a17640 1 --2- 172.21.15.40:0/2547653950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f3a0 0x7fe0440d06f0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fe0440c89f0 tx=0x7fe03c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893507384c081678 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe0437fe640 1 -- 172.21.15.40:0/2547653950 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c036070 con 0x7fe04415f3a0 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe0437fe640 1 -- 172.21.15.40:0/2547653950 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03c031600 con 0x7fe04415f3a0 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe0437fe640 1 -- 172.21.15.40:0/2547653950 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0319d0 con 0x7fe04415f3a0 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe04ac7b640 1 -- 172.21.15.40:0/2547653950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f3a0 msgr2=0x7fe0440d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/2547653950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f3a0 0x7fe0440d06f0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fe0440c89f0 tx=0x7fe03c02a730 comp rx=0 tx=0).stop 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe04ac7b640 1 -- 172.21.15.40:0/2547653950 shutdown_connections 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/2547653950 >> v2:172.21.15.184:3300/0 conn(0x7fe0440d2330 0x7fe0440b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/2547653950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f3a0 0x7fe0440d06f0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe04ac7b640 1 -- 172.21.15.40:0/2547653950 >> 172.21.15.40:0/2547653950 conn(0x7fe0440d4f90 msgr2=0x7fe0440d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:01.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe04ac7b640 1 -- 172.21.15.40:0/2547653950 shutdown_connections 2024-04-26T21:44:01.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.327+0000 7fe04ac7b640 1 -- 172.21.15.40:0/2547653950 wait complete. 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.328+0000 7fe04ac7b640 1 Processor -- start 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.328+0000 7fe04ac7b640 1 -- start start 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe04ac7b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1770 0x7fe044159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe04ac7b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440d2330 0x7fe0440b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe04ac7b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe04415f3a0 0x7fe0440b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe04ac7b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440b2f10 con 0x7fe0440d2330 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe04ac7b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440b2d90 con 0x7fe04415f3a0 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe04ac7b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe0440b3090 con 0x7fe0440d1770 2024-04-26T21:44:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe043fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440d2330 0x7fe0440b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe043fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440d2330 0x7fe0440b7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59862/0 (socket says 172.21.15.40:59862) 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.329+0000 7fe043fff640 1 -- 172.21.15.40:0/3144738773 learned_addr learned my addr 172.21.15.40:0/3144738773 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe048a17640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1770 0x7fe044159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe043fff640 1 -- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1770 msgr2=0x7fe044159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe049218640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe04415f3a0 0x7fe0440b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe043fff640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1770 0x7fe044159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe043fff640 1 -- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe04415f3a0 msgr2=0x7fe0440b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe043fff640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe04415f3a0 0x7fe0440b99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe043fff640 1 -- 172.21.15.40:0/3144738773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0440ba530 con 0x7fe0440d2330 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe048a17640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1770 0x7fe044159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe043fff640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440d2330 0x7fe0440b7bb0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fe03400c550 tx=0x7fe03400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe0417fa640 1 -- 172.21.15.40:0/3144738773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0340052d0 con 0x7fe0440d2330 2024-04-26T21:44:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe0417fa640 1 -- 172.21.15.40:0/3144738773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe034005470 con 0x7fe0440d2330 2024-04-26T21:44:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0440ba810 con 0x7fe0440d2330 2024-04-26T21:44:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe0417fa640 1 -- 172.21.15.40:0/3144738773 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034012870 con 0x7fe0440d2330 2024-04-26T21:44:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.330+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0441f8770 con 0x7fe0440d2330 2024-04-26T21:44:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.331+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04415fb40 con 0x7fe0440d2330 2024-04-26T21:44:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.332+0000 7fe0417fa640 1 -- 172.21.15.40:0/3144738773 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe034012a10 con 0x7fe0440d2330 2024-04-26T21:44:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.332+0000 7fe0417fa640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe02406c8f0 0x7fe02406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.332+0000 7fe0417fa640 1 -- 172.21.15.40:0/3144738773 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7fe03408bf20 con 0x7fe0440d2330 2024-04-26T21:44:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.332+0000 7fe048a17640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe02406c8f0 0x7fe02406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.333+0000 7fe048a17640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe02406c8f0 0x7fe02406edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe03c033000 tx=0x7fe03c009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.337+0000 7fe0417fa640 1 -- 172.21.15.40:0/3144738773 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe034059c20 con 0x7fe0440d2330 2024-04-26T21:44:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.567+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0440b6860 con 0x7fe0440d2330 2024-04-26T21:44:01.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.567+0000 7fe0417fa640 1 -- 172.21.15.40:0/3144738773 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+4355 (secure 0 0 0) 0x7fe034059520 con 0x7fe0440d2330 2024-04-26T21:44:01.568 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:01.568 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":174,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:01.214506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:01.568 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 174 2024-04-26T21:44:01.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe02406c8f0 msgr2=0x7fe02406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe02406c8f0 0x7fe02406edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe03c033000 tx=0x7fe03c009000 comp rx=0 tx=0).stop 2024-04-26T21:44:01.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440d2330 msgr2=0x7fe0440b7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440d2330 0x7fe0440b7bb0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fe03400c550 tx=0x7fe03400ca10 comp rx=0 tx=0).stop 2024-04-26T21:44:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 shutdown_connections 2024-04-26T21:44:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe02406c8f0 0x7fe02406edb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe04415f3a0 0x7fe0440b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440d2330 0x7fe0440b7bb0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 --2- 172.21.15.40:0/3144738773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1770 0x7fe044159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 >> 172.21.15.40:0/3144738773 conn(0x7fe0440d4f90 msgr2=0x7fe04406e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 shutdown_connections 2024-04-26T21:44:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.569+0000 7fe04ac7b640 1 -- 172.21.15.40:0/3144738773 wait complete. 2024-04-26T21:44:01.581 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:01.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:01.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.721+0000 7fe2677af640 1 Processor -- start 2024-04-26T21:44:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.721+0000 7fe2677af640 1 -- start start 2024-04-26T21:44:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe2677af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2600ce370 0x7fe260072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe2677af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe260073390 0x7fe2600b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe2677af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26015dd30 con 0x7fe26015a9f0 2024-04-26T21:44:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe2677af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe2600726e0 con 0x7fe260073390 2024-04-26T21:44:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe2677af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe26015e010 con 0x7fe2600ce370 2024-04-26T21:44:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe265d4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe260073390 0x7fe2600b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe26554b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe26015a9f0 0x7fe26015d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43084/0 (socket says 172.21.15.40:43084) 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe26554b640 1 -- 172.21.15.40:0/4167360937 learned_addr learned my addr 172.21.15.40:0/4167360937 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe264d4a640 1 --2- 172.21.15.40:0/4167360937 >> v2:172.21.15.184:3300/0 conn(0x7fe2600ce370 0x7fe260072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe265d4c640 1 -- 172.21.15.40:0/4167360937 >> v2:172.21.15.184:3300/0 conn(0x7fe2600ce370 msgr2=0x7fe260072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe265d4c640 1 --2- 172.21.15.40:0/4167360937 >> v2:172.21.15.184:3300/0 conn(0x7fe2600ce370 0x7fe260072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe257fff640 1 -- 172.21.15.40:0/4167360937 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1378471360 0 0) 0x7fe26015dd30 con 0x7fe26015a9f0 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe265d4c640 1 -- 172.21.15.40:0/4167360937 >> v1:172.21.15.40:6789/0 conn(0x7fe26015a9f0 legacy=0x7fe26015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe265d4c640 1 -- 172.21.15.40:0/4167360937 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26015e2f0 con 0x7fe260073390 2024-04-26T21:44:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.722+0000 7fe264d4a640 1 --2- 172.21.15.40:0/4167360937 >> v2:172.21.15.184:3300/0 conn(0x7fe2600ce370 0x7fe260072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.723+0000 7fe265d4c640 1 --2- 172.21.15.40:0/4167360937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe260073390 0x7fe2600b4790 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fe2600b3f30 tx=0x7fe25802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bbe11ce48295f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.723+0000 7fe257fff640 1 -- 172.21.15.40:0/4167360937 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258007740 con 0x7fe260073390 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.723+0000 7fe257fff640 1 -- 172.21.15.40:0/4167360937 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2580078e0 con 0x7fe260073390 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.723+0000 7fe257fff640 1 -- 172.21.15.40:0/4167360937 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258007cb0 con 0x7fe260073390 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.724+0000 7fe2677af640 1 -- 172.21.15.40:0/4167360937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe260073390 msgr2=0x7fe2600b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.724+0000 7fe2677af640 1 --2- 172.21.15.40:0/4167360937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe260073390 0x7fe2600b4790 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fe2600b3f30 tx=0x7fe25802a680 comp rx=0 tx=0).stop 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.724+0000 7fe2677af640 1 -- 172.21.15.40:0/4167360937 shutdown_connections 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.724+0000 7fe2677af640 1 --2- 172.21.15.40:0/4167360937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe260073390 0x7fe2600b4790 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.724+0000 7fe2677af640 1 --2- 172.21.15.40:0/4167360937 >> v2:172.21.15.184:3300/0 conn(0x7fe2600ce370 0x7fe260072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.724+0000 7fe2677af640 1 -- 172.21.15.40:0/4167360937 >> 172.21.15.40:0/4167360937 conn(0x7fe2600b3180 msgr2=0x7fe2600b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.724+0000 7fe2677af640 1 -- 172.21.15.40:0/4167360937 shutdown_connections 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.724+0000 7fe2677af640 1 -- 172.21.15.40:0/4167360937 wait complete. 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.725+0000 7fe2677af640 1 Processor -- start 2024-04-26T21:44:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.725+0000 7fe2677af640 1 -- start start 2024-04-26T21:44:01.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe2677af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe260073390 0x7fe2601535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe2677af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2600ce370 0x7fe2601415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe2677af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26015a9f0 0x7fe260144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe2677af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe26015af50 con 0x7fe2600ce370 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe2677af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe26015add0 con 0x7fe26015a9f0 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe2677af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2600b4cd0 con 0x7fe260073390 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2600ce370 0x7fe2601415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2600ce370 0x7fe2601415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59884/0 (socket says 172.21.15.40:59884) 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 -- 172.21.15.40:0/366791788 learned_addr learned my addr 172.21.15.40:0/366791788 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe26554b640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe260073390 0x7fe2601535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe265d4c640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26015a9f0 0x7fe260144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 -- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe260073390 msgr2=0x7fe2601535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe260073390 0x7fe2601535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 -- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26015a9f0 msgr2=0x7fe260144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26015a9f0 0x7fe260144020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 -- 172.21.15.40:0/366791788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe260152360 con 0x7fe2600ce370 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe265d4c640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26015a9f0 0x7fe260144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe264d4a640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2600ce370 0x7fe2601415f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c820 tx=0x7fe25000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe255ffb640 1 -- 172.21.15.40:0/366791788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250015070 con 0x7fe2600ce370 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe255ffb640 1 -- 172.21.15.40:0/366791788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe250007a30 con 0x7fe2600ce370 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.726+0000 7fe255ffb640 1 -- 172.21.15.40:0/366791788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250007e00 con 0x7fe2600ce370 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.727+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe260144be0 con 0x7fe2600ce370 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.727+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe260152ac0 con 0x7fe2600ce370 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.728+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2600694f0 con 0x7fe2600ce370 2024-04-26T21:44:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.728+0000 7fe255ffb640 1 -- 172.21.15.40:0/366791788 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe250018070 con 0x7fe2600ce370 2024-04-26T21:44:01.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.728+0000 7fe255ffb640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe24406c8f0 0x7fe24406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.729+0000 7fe255ffb640 1 -- 172.21.15.40:0/366791788 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7fe25008c130 con 0x7fe2600ce370 2024-04-26T21:44:01.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.729+0000 7fe26554b640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe24406c8f0 0x7fe24406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.729+0000 7fe26554b640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe24406c8f0 0x7fe24406edb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe25c007fd0 tx=0x7fe25c007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:01.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.733+0000 7fe255ffb640 1 -- 172.21.15.40:0/366791788 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe260152360 con 0x7fe2600ce370 2024-04-26T21:44:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.957+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2600b0eb0 con 0x7fe2600ce370 2024-04-26T21:44:01.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.958+0000 7fe255ffb640 1 -- 172.21.15.40:0/366791788 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+4355 (secure 0 0 0) 0x7fe2500597d0 con 0x7fe2600ce370 2024-04-26T21:44:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:01.959 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":174,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:01.214506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:01.959 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 174 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe24406c8f0 msgr2=0x7fe24406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe24406c8f0 0x7fe24406edb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe25c007fd0 tx=0x7fe25c007de0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2600ce370 msgr2=0x7fe2601415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2600ce370 0x7fe2601415f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c820 tx=0x7fe25000cc20 comp rx=0 tx=0).stop 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 shutdown_connections 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe24406c8f0 0x7fe24406edb0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26015a9f0 0x7fe260144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2600ce370 0x7fe2601415f0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 --2- 172.21.15.40:0/366791788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe260073390 0x7fe2601535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 >> 172.21.15.40:0/366791788 conn(0x7fe2600b3180 msgr2=0x7fe260158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 shutdown_connections 2024-04-26T21:44:01.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:01.960+0000 7fe2677af640 1 -- 172.21.15.40:0/366791788 wait complete. 2024-04-26T21:44:01.971 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:01.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.072+0000 7f76c3af9640 1 Processor -- start 2024-04-26T21:44:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.072+0000 7f76c3af9640 1 -- start start 2024-04-26T21:44:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c3af9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76bc0d3bc0 0x7f76bc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c3af9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76bc0b7e50 con 0x7f76bc0b5f20 2024-04-26T21:44:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c3af9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76bc0d1800 con 0x7f76bc162880 2024-04-26T21:44:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c3af9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f76bc0d0580 con 0x7f76bc0d3bc0 2024-04-26T21:44:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c1094640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76bc0d3bc0 0x7f76bc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c2096640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f76bc0b5f20 0x7f76bc0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43094/0 (socket says 172.21.15.40:43094) 2024-04-26T21:44:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c2096640 1 -- 172.21.15.40:0/3029213524 learned_addr learned my addr 172.21.15.40:0/3029213524 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c1094640 1 -- 172.21.15.40:0/3029213524 >> v1:172.21.15.184:6789/0 conn(0x7f76bc162880 legacy=0x7f76bc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c1094640 1 -- 172.21.15.40:0/3029213524 >> v1:172.21.15.40:6789/0 conn(0x7f76bc0b5f20 legacy=0x7f76bc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c1094640 1 -- 172.21.15.40:0/3029213524 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc0d07d0 con 0x7f76bc0d3bc0 2024-04-26T21:44:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.073+0000 7f76c0893640 1 -- 172.21.15.40:0/3029213524 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237157232 0 0) 0x7f76bc0b7e50 con 0x7f76bc0b5f20 2024-04-26T21:44:02.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.074+0000 7f76c1094640 1 --2- 172.21.15.40:0/3029213524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc0d3bc0 0x7f76bc0d2cb0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0097a0 tx=0x7f76ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3602de55949abfea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:02.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.074+0000 7f76c0893640 1 -- 172.21.15.40:0/3029213524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac037070 con 0x7f76bc0d3bc0 2024-04-26T21:44:02.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.074+0000 7f76c0893640 1 -- 172.21.15.40:0/3029213524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ac032440 con 0x7f76bc0d3bc0 2024-04-26T21:44:02.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.074+0000 7f76c0893640 1 -- 172.21.15.40:0/3029213524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac032810 con 0x7f76bc0d3bc0 2024-04-26T21:44:02.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.075+0000 7f76c3af9640 1 -- 172.21.15.40:0/3029213524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc0d3bc0 msgr2=0x7f76bc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.075+0000 7f76c3af9640 1 --2- 172.21.15.40:0/3029213524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc0d3bc0 0x7f76bc0d2cb0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0097a0 tx=0x7f76ac02a230 comp rx=0 tx=0).stop 2024-04-26T21:44:02.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.075+0000 7f76c3af9640 1 -- 172.21.15.40:0/3029213524 shutdown_connections 2024-04-26T21:44:02.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.075+0000 7f76c3af9640 1 --2- 172.21.15.40:0/3029213524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc0d3bc0 0x7f76bc0d2cb0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.075+0000 7f76c3af9640 1 -- 172.21.15.40:0/3029213524 >> 172.21.15.40:0/3029213524 conn(0x7f76bc0d4f60 msgr2=0x7f76bc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:02.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.075+0000 7f76c3af9640 1 -- 172.21.15.40:0/3029213524 shutdown_connections 2024-04-26T21:44:02.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.075+0000 7f76c3af9640 1 -- 172.21.15.40:0/3029213524 wait complete. 2024-04-26T21:44:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.076+0000 7f76c3af9640 1 Processor -- start 2024-04-26T21:44:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.076+0000 7f76c3af9640 1 -- start start 2024-04-26T21:44:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c3af9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76bc0b5f20 0x7f76bc15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c3af9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76bc0d3bc0 0x7f76bc0a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c3af9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc162880 0x7f76bc0a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c3af9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc162c60 con 0x7f76bc162880 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c3af9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc0d0d90 con 0x7f76bc0d3bc0 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c3af9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76bc162de0 con 0x7f76bc0b5f20 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc162880 0x7f76bc0a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc162880 0x7f76bc0a28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59912/0 (socket says 172.21.15.40:59912) 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 -- 172.21.15.40:0/827617421 learned_addr learned my addr 172.21.15.40:0/827617421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c1895640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76bc0b5f20 0x7f76bc15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c1094640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76bc0d3bc0 0x7f76bc0a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 -- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76bc0b5f20 msgr2=0x7f76bc15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76bc0b5f20 0x7f76bc15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 -- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76bc0d3bc0 msgr2=0x7f76bc0a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76bc0d3bc0 0x7f76bc0a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 -- 172.21.15.40:0/827617421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc09f040 con 0x7f76bc162880 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c1895640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76bc0b5f20 0x7f76bc15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c1094640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76bc0d3bc0 0x7f76bc0a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c2096640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc162880 0x7f76bc0a28e0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f76b400c7a0 tx=0x7f76b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76b27fc640 1 -- 172.21.15.40:0/827617421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4018070 con 0x7f76bc162880 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76b27fc640 1 -- 172.21.15.40:0/827617421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b400a640 con 0x7f76bc162880 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.077+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc09f320 con 0x7f76bc162880 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.078+0000 7f76b27fc640 1 -- 172.21.15.40:0/827617421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b400aab0 con 0x7f76bc162880 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.078+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc09f8e0 con 0x7f76bc162880 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.079+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76bc0694f0 con 0x7f76bc162880 2024-04-26T21:44:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.079+0000 7f76b27fc640 1 -- 172.21.15.40:0/827617421 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76b401b070 con 0x7f76bc162880 2024-04-26T21:44:02.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.079+0000 7f76b27fc640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76a006c8f0 0x7f76a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.079+0000 7f76b27fc640 1 -- 172.21.15.40:0/827617421 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7f76b408b7a0 con 0x7f76bc162880 2024-04-26T21:44:02.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.080+0000 7f76c1895640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76a006c8f0 0x7f76a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.080+0000 7f76c1895640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76a006c8f0 0x7f76a006edb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f76b8000f60 tx=0x7f76b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:02.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.084+0000 7f76b27fc640 1 -- 172.21.15.40:0/827617421 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f76b40595b0 con 0x7f76bc162880 2024-04-26T21:44:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.312+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76bc0d53e0 con 0x7f76bc162880 2024-04-26T21:44:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.312+0000 7f76b27fc640 1 -- 172.21.15.40:0/827617421 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) ==== 95+0+5181 (secure 0 0 0) 0x7f76b4058eb0 con 0x7f76bc162880 2024-04-26T21:44:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:02.313 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":175,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9432,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3588039422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3588039422},{"type":"v1","addr":"172.21.15.40:6833","nonce":3588039422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:01.214506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 175 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.314+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76a006c8f0 msgr2=0x7f76a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.314+0000 7f76c3af9640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76a006c8f0 0x7f76a006edb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f76b8000f60 tx=0x7f76b8009040 comp rx=0 tx=0).stop 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc162880 msgr2=0x7f76bc0a28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc162880 0x7f76bc0a28e0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f76b400c7a0 tx=0x7f76b400cc60 comp rx=0 tx=0).stop 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 shutdown_connections 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f76a006c8f0 0x7f76a006edb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76bc162880 0x7f76bc0a28e0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76bc0d3bc0 0x7f76bc0a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 --2- 172.21.15.40:0/827617421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76bc0b5f20 0x7f76bc15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 >> 172.21.15.40:0/827617421 conn(0x7f76bc0d4f60 msgr2=0x7f76bc06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:02.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 shutdown_connections 2024-04-26T21:44:02.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.315+0000 7f76c3af9640 1 -- 172.21.15.40:0/827617421 wait complete. 2024-04-26T21:44:02.326 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000.inode' led to DAMAGED state 2024-04-26T21:44:02.326 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000' 2024-04-26T21:44:02.327 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:02.327 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:02.342 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:02.343 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:02.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:02.343 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:02.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:02.398 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:02.398 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:02.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.538+0000 7f3a61009640 1 Processor -- start 2024-04-26T21:44:02.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.538+0000 7f3a61009640 1 -- start start 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a61009640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3a5c0b2340 0x7f3a5c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a61009640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a5c0cd930 0x7f3a5c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a61009640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a5c0d10d0 con 0x7f3a5c162810 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a61009640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5c0d0750 con 0x7f3a5c0cd930 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a61009640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5c0d16a0 con 0x7f3a5c0b2340 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5b59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a5c0cd930 0x7f3a5c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3a5c162810 0x7f3a5c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43100/0 (socket says 172.21.15.40:43100) 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5ad9d640 1 -- 172.21.15.40:0/3528147264 learned_addr learned my addr 172.21.15.40:0/3528147264 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5a59c640 1 -- 172.21.15.40:0/3528147264 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 389371951 0 0) 0x7f3a5c0d10d0 con 0x7f3a5c162810 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5a59c640 1 -- 172.21.15.40:0/3528147264 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a48003520 con 0x7f3a5c162810 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a53fff640 1 --2- 172.21.15.40:0/3528147264 >> v2:172.21.15.184:3300/0 conn(0x7f3a5c0b2340 0x7f3a5c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5b59e640 1 -- 172.21.15.40:0/3528147264 >> v2:172.21.15.184:3300/0 conn(0x7f3a5c0b2340 msgr2=0x7f3a5c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:02.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5b59e640 1 --2- 172.21.15.40:0/3528147264 >> v2:172.21.15.184:3300/0 conn(0x7f3a5c0b2340 0x7f3a5c0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5b59e640 1 -- 172.21.15.40:0/3528147264 >> v1:172.21.15.40:6789/0 conn(0x7f3a5c162810 legacy=0x7f3a5c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.539+0000 7f3a5b59e640 1 -- 172.21.15.40:0/3528147264 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c0d1370 con 0x7f3a5c0cd930 2024-04-26T21:44:02.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.540+0000 7f3a5b59e640 1 --2- 172.21.15.40:0/3528147264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c0cd930 0x7f3a5c0d1b30 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0cd0a0 tx=0x7f3a5402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4b1ebb7c713673 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:02.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.540+0000 7f3a5a59c640 1 -- 172.21.15.40:0/3528147264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54007740 con 0x7f3a5c0cd930 2024-04-26T21:44:02.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.540+0000 7f3a5a59c640 1 -- 172.21.15.40:0/3528147264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a540078e0 con 0x7f3a5c0cd930 2024-04-26T21:44:02.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.540+0000 7f3a5a59c640 1 -- 172.21.15.40:0/3528147264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54007cb0 con 0x7f3a5c0cd930 2024-04-26T21:44:02.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.541+0000 7f3a61009640 1 -- 172.21.15.40:0/3528147264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c0cd930 msgr2=0x7f3a5c0d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.541+0000 7f3a61009640 1 --2- 172.21.15.40:0/3528147264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c0cd930 0x7f3a5c0d1b30 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0cd0a0 tx=0x7f3a5402a680 comp rx=0 tx=0).stop 2024-04-26T21:44:02.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.541+0000 7f3a61009640 1 -- 172.21.15.40:0/3528147264 shutdown_connections 2024-04-26T21:44:02.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.541+0000 7f3a61009640 1 --2- 172.21.15.40:0/3528147264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c0cd930 0x7f3a5c0d1b30 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.541+0000 7f3a61009640 1 --2- 172.21.15.40:0/3528147264 >> v2:172.21.15.184:3300/0 conn(0x7f3a5c0b2340 0x7f3a5c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.541+0000 7f3a61009640 1 -- 172.21.15.40:0/3528147264 >> 172.21.15.40:0/3528147264 conn(0x7f3a5c0d2da0 msgr2=0x7f3a5c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:02.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.541+0000 7f3a61009640 1 -- 172.21.15.40:0/3528147264 shutdown_connections 2024-04-26T21:44:02.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.541+0000 7f3a61009640 1 -- 172.21.15.40:0/3528147264 wait complete. 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.542+0000 7f3a61009640 1 Processor -- start 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.542+0000 7f3a61009640 1 -- start start 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a61009640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a5c0b2340 0x7f3a5c15ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a61009640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a5c0cd930 0x7f3a5c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a61009640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c162810 0x7f3a5c0b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a61009640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d2070 con 0x7f3a5c162810 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a61009640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d0470 con 0x7f3a5c0b2340 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a61009640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d21f0 con 0x7f3a5c0cd930 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c162810 0x7f3a5c0b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c162810 0x7f3a5c0b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59926/0 (socket says 172.21.15.40:59926) 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5b59e640 1 -- 172.21.15.40:0/1787061006 learned_addr learned my addr 172.21.15.40:0/1787061006 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a53fff640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a5c0cd930 0x7f3a5c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5ad9d640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a5c0b2340 0x7f3a5c15ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5b59e640 1 -- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a5c0cd930 msgr2=0x7f3a5c0c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5b59e640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a5c0cd930 0x7f3a5c0c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5b59e640 1 -- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a5c0b2340 msgr2=0x7f3a5c15ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5b59e640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a5c0b2340 0x7f3a5c15ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.543+0000 7f3a5b59e640 1 -- 172.21.15.40:0/1787061006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c160670 con 0x7f3a5c162810 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.544+0000 7f3a5b59e640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c162810 0x7f3a5c0b3c50 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f3a5402ac70 tx=0x7f3a54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.544+0000 7f3a537fe640 1 -- 172.21.15.40:0/1787061006 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54040070 con 0x7f3a5c162810 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.544+0000 7f3a537fe640 1 -- 172.21.15.40:0/1787061006 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54031bc0 con 0x7f3a5c162810 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.544+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c160950 con 0x7f3a5c162810 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.544+0000 7f3a537fe640 1 -- 172.21.15.40:0/1787061006 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54009450 con 0x7f3a5c162810 2024-04-26T21:44:02.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.544+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c1611f0 con 0x7f3a5c162810 2024-04-26T21:44:02.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.545+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c06e570 con 0x7f3a5c162810 2024-04-26T21:44:02.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.545+0000 7f3a537fe640 1 -- 172.21.15.40:0/1787061006 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a54039040 con 0x7f3a5c162810 2024-04-26T21:44:02.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.546+0000 7f3a537fe640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a3c06c8f0 0x7f3a3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.546+0000 7f3a537fe640 1 -- 172.21.15.40:0/1787061006 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7f3a540ad900 con 0x7f3a5c162810 2024-04-26T21:44:02.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.546+0000 7f3a5ad9d640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a3c06c8f0 0x7f3a3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.547+0000 7f3a5ad9d640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a3c06c8f0 0x7f3a3c06edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c007fd0 tx=0x7f3a4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:02.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.550+0000 7f3a537fe640 1 -- 172.21.15.40:0/1787061006 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a5c06e570 con 0x7f3a5c162810 2024-04-26T21:44:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:02.776+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3a5c06f320 con 0x7f3a5c162810 2024-04-26T21:44:03.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.224+0000 7f3a537fe640 1 -- 172.21.15.40:0/1787061006 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v176) ==== 161+0+0 (secure 0 0 0) 0x7f3a5c06f320 con 0x7f3a5c162810 2024-04-26T21:44:03.225 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a3c06c8f0 msgr2=0x7f3a3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a3c06c8f0 0x7f3a3c06edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c007fd0 tx=0x7f3a4c009040 comp rx=0 tx=0).stop 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c162810 msgr2=0x7f3a5c0b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c162810 0x7f3a5c0b3c50 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f3a5402ac70 tx=0x7f3a54004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 shutdown_connections 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3a3c06c8f0 0x7f3a3c06edb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a5c162810 0x7f3a5c0b3c50 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a5c0cd930 0x7f3a5c0c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 --2- 172.21.15.40:0/1787061006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a5c0b2340 0x7f3a5c15ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 >> 172.21.15.40:0/1787061006 conn(0x7f3a5c0d2da0 msgr2=0x7f3a5c06de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 shutdown_connections 2024-04-26T21:44:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.226+0000 7f3a61009640 1 -- 172.21.15.40:0/1787061006 wait complete. 2024-04-26T21:44:03.239 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:03.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.341+0000 7f829dfbf640 1 Processor -- start 2024-04-26T21:44:03.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.341+0000 7f829dfbf640 1 -- start start 2024-04-26T21:44:03.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f829dfbf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8298162880 0x7f82980d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:03.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f829dfbf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82980b5f20 0x7f82980d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:03.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f829dfbf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82980b7e50 con 0x7f82980d39f0 2024-04-26T21:44:03.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f829dfbf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f82980d3e40 con 0x7f82980b5f20 2024-04-26T21:44:03.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f829dfbf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f82980d0580 con 0x7f8298162880 2024-04-26T21:44:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f8296ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82980b5f20 0x7f82980d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f8296ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82980b5f20 0x7f82980d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59932/0 (socket says 172.21.15.40:59932) 2024-04-26T21:44:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f8296ffd640 1 -- 172.21.15.40:0/511988183 learned_addr learned my addr 172.21.15.40:0/511988183 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f82977fe640 1 --2- 172.21.15.40:0/511988183 >> v2:172.21.15.184:3300/0 conn(0x7f8298162880 0x7f82980d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f8296ffd640 1 -- 172.21.15.40:0/511988183 >> v2:172.21.15.184:3300/0 conn(0x7f8298162880 msgr2=0x7f82980d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f8296ffd640 1 --2- 172.21.15.40:0/511988183 >> v2:172.21.15.184:3300/0 conn(0x7f8298162880 0x7f82980d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f8296ffd640 1 -- 172.21.15.40:0/511988183 >> v1:172.21.15.184:6789/0 conn(0x7f82980d39f0 legacy=0x7f82980b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.342+0000 7f8296ffd640 1 -- 172.21.15.40:0/511988183 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82980d1800 con 0x7f82980b5f20 2024-04-26T21:44:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.343+0000 7f8296ffd640 1 --2- 172.21.15.40:0/511988183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f82980d34b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f829808d490 tx=0x7f828c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1729cc9115c703d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.343+0000 7f82967fc640 1 -- 172.21.15.40:0/511988183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c037070 con 0x7f82980b5f20 2024-04-26T21:44:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.343+0000 7f82967fc640 1 -- 172.21.15.40:0/511988183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c032440 con 0x7f82980b5f20 2024-04-26T21:44:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.343+0000 7f82967fc640 1 -- 172.21.15.40:0/511988183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c032810 con 0x7f82980b5f20 2024-04-26T21:44:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.344+0000 7f829dfbf640 1 -- 172.21.15.40:0/511988183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 msgr2=0x7f82980d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.344+0000 7f829dfbf640 1 --2- 172.21.15.40:0/511988183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f82980d34b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f829808d490 tx=0x7f828c02a270 comp rx=0 tx=0).stop 2024-04-26T21:44:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.344+0000 7f829dfbf640 1 -- 172.21.15.40:0/511988183 shutdown_connections 2024-04-26T21:44:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.344+0000 7f829dfbf640 1 --2- 172.21.15.40:0/511988183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f82980d34b0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.344+0000 7f829dfbf640 1 --2- 172.21.15.40:0/511988183 >> v2:172.21.15.184:3300/0 conn(0x7f8298162880 0x7f82980d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.344+0000 7f829dfbf640 1 -- 172.21.15.40:0/511988183 >> 172.21.15.40:0/511988183 conn(0x7f82980d4f60 msgr2=0x7f82980d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:03.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.344+0000 7f829dfbf640 1 -- 172.21.15.40:0/511988183 shutdown_connections 2024-04-26T21:44:03.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.344+0000 7f829dfbf640 1 -- 172.21.15.40:0/511988183 wait complete. 2024-04-26T21:44:03.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.345+0000 7f829dfbf640 1 Processor -- start 2024-04-26T21:44:03.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.345+0000 7f829dfbf640 1 -- start start 2024-04-26T21:44:03.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f829dfbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f8298155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:03.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f829dfbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82980d39f0 0x7f82980cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f829dfbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8298162880 0x7f82980b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f829dfbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f82980d0d90 con 0x7f82980b5f20 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f829dfbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f82980d0ab0 con 0x7f8298162880 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f829dfbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8298162c60 con 0x7f82980d39f0 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f8298155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f8298155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59942/0 (socket says 172.21.15.40:59942) 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 -- 172.21.15.40:0/1969161908 learned_addr learned my addr 172.21.15.40:0/1969161908 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f8296ffd640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82980d39f0 0x7f82980cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f8297fff640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8298162880 0x7f82980b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 -- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82980d39f0 msgr2=0x7f82980cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82980d39f0 0x7f82980cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 -- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8298162880 msgr2=0x7f82980b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8298162880 0x7f82980b93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 -- 172.21.15.40:0/1969161908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82980ccf40 con 0x7f82980b5f20 2024-04-26T21:44:03.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f8296ffd640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82980d39f0 0x7f82980cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:03.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.346+0000 7f82977fe640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f8298155370 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f828800e930 tx=0x7f828800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:03.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.347+0000 7f827ffff640 1 -- 172.21.15.40:0/1969161908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288017070 con 0x7f82980b5f20 2024-04-26T21:44:03.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.347+0000 7f827ffff640 1 -- 172.21.15.40:0/1969161908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828800c9a0 con 0x7f82980b5f20 2024-04-26T21:44:03.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.347+0000 7f827ffff640 1 -- 172.21.15.40:0/1969161908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828800cd90 con 0x7f82980b5f20 2024-04-26T21:44:03.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.347+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82980cc980 con 0x7f82980b5f20 2024-04-26T21:44:03.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.347+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82980cd6b0 con 0x7f82980b5f20 2024-04-26T21:44:03.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.348+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829806eb80 con 0x7f82980b5f20 2024-04-26T21:44:03.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.348+0000 7f827ffff640 1 -- 172.21.15.40:0/1969161908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f828801a070 con 0x7f82980b5f20 2024-04-26T21:44:03.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.349+0000 7f827ffff640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f828406c8f0 0x7f828406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:03.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.349+0000 7f827ffff640 1 -- 172.21.15.40:0/1969161908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7f828808c180 con 0x7f82980b5f20 2024-04-26T21:44:03.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.349+0000 7f8296ffd640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f828406c8f0 0x7f828406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:03.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.349+0000 7f8296ffd640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f828406c8f0 0x7f828406edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f828c034000 tx=0x7f828c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:03.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.353+0000 7f827ffff640 1 -- 172.21.15.40:0/1969161908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f829806eb80 con 0x7f82980b5f20 2024-04-26T21:44:03.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:03.618+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f82980d1470 con 0x7f82980b5f20 2024-04-26T21:44:04.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.230+0000 7f827ffff640 1 -- 172.21.15.40:0/1969161908 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v177) ==== 101+0+0 (secure 0 0 0) 0x7f828805a050 con 0x7f82980b5f20 2024-04-26T21:44:04.231 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f828406c8f0 msgr2=0x7f828406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f828406c8f0 0x7f828406edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f828c034000 tx=0x7f828c02e000 comp rx=0 tx=0).stop 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 msgr2=0x7f8298155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f8298155370 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f828800e930 tx=0x7f828800edf0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 shutdown_connections 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f828406c8f0 0x7f828406edb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8298162880 0x7f82980b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82980d39f0 0x7f82980cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 --2- 172.21.15.40:0/1969161908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82980b5f20 0x7f8298155370 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 >> 172.21.15.40:0/1969161908 conn(0x7f82980d4f60 msgr2=0x7f8298161250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 shutdown_connections 2024-04-26T21:44:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.233+0000 7f829dfbf640 1 -- 172.21.15.40:0/1969161908 wait complete. 2024-04-26T21:44:04.247 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.292+0000 7f38a2b1ef40 1 Processor -- start 2024-04-26T21:44:04.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.292+0000 7f38a2b1ef40 1 -- start start 2024-04-26T21:44:04.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f38a2b1ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319193e0 0x5625319d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f38a2b1ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319ec420 0x562531916710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f38a2b1ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625319ecd40 0x562531916c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f38a2b1ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56253195ec60 con 0x5625319193e0 2024-04-26T21:44:04.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f38a2b1ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5625318bee40 con 0x5625319ecd40 2024-04-26T21:44:04.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f38a2b1ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562531960ae0 con 0x5625319ec420 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319193e0 0x5625319d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319193e0 0x5625319d5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59952/0 (socket says 172.21.15.40:59952) 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 -- 172.21.15.40:0/2359391587 learned_addr learned my addr 172.21.15.40:0/2359391587 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 -- 172.21.15.40:0/2359391587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319ec420 msgr2=0x562531916710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389f349640 1 --2- 172.21.15.40:0/2359391587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319ec420 0x562531916710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 --2- 172.21.15.40:0/2359391587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319ec420 0x562531916710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 -- 172.21.15.40:0/2359391587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625319ecd40 msgr2=0x562531916c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 --2- 172.21.15.40:0/2359391587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625319ecd40 0x562531916c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 -- 172.21.15.40:0/2359391587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562531957c10 con 0x5625319193e0 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.293+0000 7f389fb4a640 1 --2- 172.21.15.40:0/2359391587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319193e0 0x5625319d5000 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f3890005fd0 tx=0x7f3890050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d761df94d7b74d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f389e347640 1 -- 172.21.15.40:0/2359391587 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389004fc50 con 0x5625319193e0 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f389e347640 1 -- 172.21.15.40:0/2359391587 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389004fe10 con 0x5625319193e0 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f389e347640 1 -- 172.21.15.40:0/2359391587 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890057640 con 0x5625319193e0 2024-04-26T21:44:04.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/2359391587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319193e0 msgr2=0x5625319d5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/2359391587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319193e0 0x5625319d5000 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f3890005fd0 tx=0x7f3890050000 comp rx=0 tx=0).stop 2024-04-26T21:44:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/2359391587 shutdown_connections 2024-04-26T21:44:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/2359391587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625319ecd40 0x562531916c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/2359391587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319ec420 0x562531916710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/2359391587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319193e0 0x5625319d5000 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/2359391587 >> 172.21.15.40:0/2359391587 conn(0x562531916330 msgr2=0x562531917710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/2359391587 shutdown_connections 2024-04-26T21:44:04.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.294+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/2359391587 wait complete. 2024-04-26T21:44:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.295+0000 7f38a2b1ef40 1 Processor -- start 2024-04-26T21:44:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.295+0000 7f38a2b1ef40 1 -- start start 2024-04-26T21:44:04.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f38a2b1ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319193e0 0x5625319f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f38a2b1ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319ec420 0x562531a7d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f38a2b1ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625319ecd40 0x562531a7f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f38a2b1ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5625319574a0 con 0x5625319ec420 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f38a2b1ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5625319eb1a0 con 0x5625319ecd40 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f38a2b1ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5625319eaec0 con 0x5625319193e0 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319ec420 0x562531a7d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319ec420 0x562531a7d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59968/0 (socket says 172.21.15.40:59968) 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 learned_addr learned my addr 172.21.15.40:0/26993455 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389fb4a640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319193e0 0x5625319f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f38a034b640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625319ecd40 0x562531a7f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319193e0 msgr2=0x5625319f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625319193e0 0x5625319f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625319ecd40 msgr2=0x562531a7f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625319ecd40 0x562531a7f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625319d3f10 con 0x5625319ec420 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.296+0000 7f389f349640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319ec420 0x562531a7d6f0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f3888007db0 tx=0x7f3888006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.297+0000 7f389d345640 1 -- 172.21.15.40:0/26993455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888006e60 con 0x5625319ec420 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.297+0000 7f389d345640 1 -- 172.21.15.40:0/26993455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3888010710 con 0x5625319ec420 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.297+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5625319d3c70 con 0x5625319ec420 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.297+0000 7f389d345640 1 -- 172.21.15.40:0/26993455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888010ac0 con 0x5625319ec420 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.297+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625319d4670 con 0x5625319ec420 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.298+0000 7f389d345640 1 -- 172.21.15.40:0/26993455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3888010c60 con 0x5625319ec420 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.299+0000 7f389d345640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f388006b2d0 0x7f388006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.299+0000 7f389d345640 1 -- 172.21.15.40:0/26993455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7f388808a4e0 con 0x5625319ec420 2024-04-26T21:44:04.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.299+0000 7f389fb4a640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f388006b2d0 0x7f388006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.302 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.299+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x562531a7dc30 0x562531a88a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.299+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5625319d4120 con 0x562531a7dc30 2024-04-26T21:44:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.299+0000 7f38a034b640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x562531a7dc30 0x562531a88a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.299+0000 7f389fb4a640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f388006b2d0 0x7f388006d790 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3890059000 tx=0x7f3890051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.299+0000 7f38a034b640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x562531a7dc30 0x562531a88a40 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.301+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v109'412 uv386 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.301+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5625319e6470 con 0x562531a7dc30 2024-04-26T21:44:04.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.302+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v109'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.302+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a88f80 con 0x562531a7dc30 2024-04-26T21:44:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.303+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v109'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.304 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.303+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a89350 con 0x562531a7dc30 2024-04-26T21:44:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.304+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v109'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.304+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92d20 con 0x562531a7dc30 2024-04-26T21:44:04.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.305+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v109'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.305+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a91d10 con 0x562531a7dc30 2024-04-26T21:44:04.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.306+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v109'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.306+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a920b0 con 0x562531a7dc30 2024-04-26T21:44:04.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.307+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v109'418 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.307+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a924a0 con 0x562531a7dc30 2024-04-26T21:44:04.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.308+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v109'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.308+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a924a0 con 0x562531a7dc30 2024-04-26T21:44:04.310 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.309+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v109'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.309+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.310+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v109'421 uv421 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.310+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.311+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v109'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.311+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.312+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v109'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.312+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.313+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v109'424 uv399 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.313+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.314+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v109'425 uv425 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.314+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.315+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v109'426 uv426 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.316 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:04.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.315+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.316+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v109'427 uv427 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.316+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.317+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v109'428 uv428 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.317+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92450 con 0x562531a7dc30 2024-04-26T21:44:04.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.318+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v109'429 uv429 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.318+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92810 con 0x562531a7dc30 2024-04-26T21:44:04.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.319+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v109'430 uv430 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.319+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.321+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v109'431 uv405 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.321+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.322+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v109'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.322+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.323 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.323+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v109'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.323+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.324+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v109'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.324+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.325+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v109'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.325+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.327+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v109'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.327+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.328+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v109'437 uv411 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.328+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.329+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v109'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.329+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.331 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.330+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v109'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.330+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.331+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v109'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.331+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.332+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v109'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.332+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a7dc30 2024-04-26T21:44:04.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.334+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v109'442 uv442 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a7dc30 2024-04-26T21:44:04.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.334+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562531a935d0 0x562531a959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.334+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a92700 con 0x562531a935d0 2024-04-26T21:44:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.334+0000 7f389fb4a640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562531a935d0 0x562531a959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.334+0000 7f389fb4a640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562531a935d0 0x562531a959c0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.336+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v109'308 uv291 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.336+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a95f00 con 0x562531a935d0 2024-04-26T21:44:04.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.337+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v109'309 uv309 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.337+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a962a0 con 0x562531a935d0 2024-04-26T21:44:04.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.338+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v109'310 uv310 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.339 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:04.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.338+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a96af0 con 0x562531a935d0 2024-04-26T21:44:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.340+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v109'311 uv311 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.340+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a96e90 con 0x562531a935d0 2024-04-26T21:44:04.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.341+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v109'312 uv312 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.341+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97230 con 0x562531a935d0 2024-04-26T21:44:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.342+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v109'313 uv296 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.342+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a975d0 con 0x562531a935d0 2024-04-26T21:44:04.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.343+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v109'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.343+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97970 con 0x562531a935d0 2024-04-26T21:44:04.345 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.344+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v109'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.344+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97d60 con 0x562531a935d0 2024-04-26T21:44:04.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.346+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v109'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.346+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97d60 con 0x562531a935d0 2024-04-26T21:44:04.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.347+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v109'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.347+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97a00 con 0x562531a935d0 2024-04-26T21:44:04.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.348+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v109'318 uv301 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.348+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a97970 con 0x562531a935d0 2024-04-26T21:44:04.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.349+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v109'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.349+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97970 con 0x562531a935d0 2024-04-26T21:44:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.351+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v109'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.351 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.351+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97970 con 0x562531a935d0 2024-04-26T21:44:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.352+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v109'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.352+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97970 con 0x562531a935d0 2024-04-26T21:44:04.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.353+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v109'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.353+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97970 con 0x562531a935d0 2024-04-26T21:44:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.355+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v109'323 uv307 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.355+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a97970 con 0x562531a935d0 2024-04-26T21:44:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.356+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v109'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.356+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97970 con 0x562531a935d0 2024-04-26T21:44:04.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.357+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v109'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.357 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:04.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.357+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a98e00 con 0x562531a935d0 2024-04-26T21:44:04.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.358+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v109'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.358+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a98d70 con 0x562531a935d0 2024-04-26T21:44:04.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.359+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v109'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.359+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a97a90 con 0x562531a935d0 2024-04-26T21:44:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.360+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v109'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531a935d0 2024-04-26T21:44:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.360+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562531a98110 0x562531a9a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.360+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9ab60 con 0x562531a98110 2024-04-26T21:44:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.360+0000 7f389f349640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562531a98110 0x562531a9a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.361+0000 7f389f349640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562531a98110 0x562531a9a500 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.362+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v109'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.362+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a9b020 con 0x562531a98110 2024-04-26T21:44:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.363+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v109'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.363+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9b500 con 0x562531a98110 2024-04-26T21:44:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.364+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v109'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.365 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.364+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9cdb0 con 0x562531a98110 2024-04-26T21:44:04.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.365+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v109'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.365+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9d150 con 0x562531a98110 2024-04-26T21:44:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.366+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v109'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.366+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9bda0 con 0x562531a98110 2024-04-26T21:44:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.367+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v109'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.367+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562531a9c6a0 0x562531a9f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.367+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9fa40 con 0x562531a9c6a0 2024-04-26T21:44:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.368+0000 7f38a034b640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562531a9c6a0 0x562531a9f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.368+0000 7f38a034b640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562531a9c6a0 0x562531a9f500 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.369+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v109'68 uv67 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.369+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a9fa40 con 0x562531a9c6a0 2024-04-26T21:44:04.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.370+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v109'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.371+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9fac0 con 0x562531a9c6a0 2024-04-26T21:44:04.373 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.373+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v109'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.373+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9feb0 con 0x562531a9c6a0 2024-04-26T21:44:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.376+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v109'71 uv71 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.376+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x562531aa03c0 0x562531aa27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.376+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa2d80 con 0x562531aa03c0 2024-04-26T21:44:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.377+0000 7f389fb4a640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x562531aa03c0 0x562531aa27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.377+0000 7f389fb4a640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x562531aa03c0 0x562531aa27b0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.378+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v109'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.378+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa2e40 con 0x562531aa03c0 2024-04-26T21:44:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.379+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v109'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.379+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.381 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.381+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v109'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.381+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa4240 con 0x562531aa03c0 2024-04-26T21:44:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.382+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v109'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.382+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa4240 con 0x562531aa03c0 2024-04-26T21:44:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.384+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v109'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.384+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.384+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v109'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.384+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.386+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v109'149 uv143 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.386+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.387+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v109'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.387+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.388 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:04.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.388+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v109'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.388+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa4240 con 0x562531a9c6a0 2024-04-26T21:44:04.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.389+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v109'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.389+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa4240 con 0x562531a9c6a0 2024-04-26T21:44:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.390+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v109'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.390+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.391+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v109'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.391+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.392+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v109'155 uv148 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.392+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.393+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v109'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.393+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.395 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:04.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.394+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v109'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.394+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.395+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v109'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.395+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.396+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v109'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.396+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.397+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v109'66 uv65 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.397+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.398+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v109'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.398+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.399+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v109'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.400 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:04.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.399+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.400+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v109'69 uv69 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.400+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531a9c6a0 2024-04-26T21:44:04.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.401+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v109'70 uv70 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38980103b0 con 0x562531a9c6a0 2024-04-26T21:44:04.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.401+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.402+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v109'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.402+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.403+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v109'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.403+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.405 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:04.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.405+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v109'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.405+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa4240 con 0x562531aa03c0 2024-04-26T21:44:04.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.406+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v109'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.406+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa4240 con 0x562531aa03c0 2024-04-26T21:44:04.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.407+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v109'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.407+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.408+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v109'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.408+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.410+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v109'132 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.410+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.411+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v109'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.411+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa3230 con 0x562531aa03c0 2024-04-26T21:44:04.412 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:04.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.412+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v109'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.412+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9b9b0 con 0x562531aa03c0 2024-04-26T21:44:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.413+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v109'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.413+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9b9b0 con 0x562531aa03c0 2024-04-26T21:44:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.413+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v109'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.414+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9b9b0 con 0x562531aa03c0 2024-04-26T21:44:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.414+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v109'137 uv131 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.414+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531a9b9b0 con 0x562531aa03c0 2024-04-26T21:44:04.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.416+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v109'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.416+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9b9b0 con 0x562531aa03c0 2024-04-26T21:44:04.417 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:04.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.417+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v109'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.417+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531a9b9b0 con 0x562531aa03c0 2024-04-26T21:44:04.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.418+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v109'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.418+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6040 con 0x562531aa03c0 2024-04-26T21:44:04.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.419+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v109'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.419+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x562531aa3d60 0x562531aa6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.419+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6700 con 0x562531aa3d60 2024-04-26T21:44:04.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.419+0000 7f389f349640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x562531aa3d60 0x562531aa6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.420+0000 7f389f349640 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x562531aa3d60 0x562531aa6150 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.421+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v109'61 uv60 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531aa3d60 2024-04-26T21:44:04.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.421+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa6aa0 con 0x562531aa3d60 2024-04-26T21:44:04.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.422+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v109'62 uv62 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531aa3d60 2024-04-26T21:44:04.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.422+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6aa0 con 0x562531aa3d60 2024-04-26T21:44:04.425 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.425+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v109'63 uv63 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531aa3d60 2024-04-26T21:44:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.425+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531aa3d60 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.428+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v109'64 uv64 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531aa3d60 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.428+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531aa03c0 2024-04-26T21:44:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.429+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v109'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.429+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531aa03c0 2024-04-26T21:44:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.430+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v109'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.430+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531aa03c0 2024-04-26T21:44:04.432 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.432+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v109'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.432+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa7ea0 con 0x562531aa03c0 2024-04-26T21:44:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.433+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v109'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.433+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa7ea0 con 0x562531aa03c0 2024-04-26T21:44:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.435+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v109'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.435+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531aa03c0 2024-04-26T21:44:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.435+0000 7f389fb4a640 1 -- 172.21.15.40:0/26993455 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v109'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3890060070 con 0x562531aa03c0 2024-04-26T21:44:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.435+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.436+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v109'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.436+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.437+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v109'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.437+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.440 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.439+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v109'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.439+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.440+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v109'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.440+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.442+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v109'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.442+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.443+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v109'170 uv170 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.444 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.443+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.444+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v109'171 uv171 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.444+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.445+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v109'172 uv172 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.445+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.446+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 1.00000000 [create] v109'173 uv168 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.446+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.447+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 1.00000000 [delete] v109'174 uv174 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.447+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.449 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:04.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.448+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v109'175 uv175 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.448+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa7ea0 con 0x562531a98110 2024-04-26T21:44:04.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.449+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v109'176 uv176 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.449+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa7ea0 con 0x562531a98110 2024-04-26T21:44:04.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.450+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v109'177 uv177 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.450+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa6e90 con 0x562531a98110 2024-04-26T21:44:04.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.451+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [omap-set-header] v109'178 uv178 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.451+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x562531aa7340 con 0x562531a98110 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f389f349640 1 -- 172.21.15.40:0/26993455 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v109'179 uv179 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3888010070 con 0x562531a98110 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562531a935d0 msgr2=0x562531a959c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x562531a935d0 0x562531a959c0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x562531a7dc30 msgr2=0x562531a88a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x562531a7dc30 0x562531a88a40 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562531a9c6a0 msgr2=0x562531a9f500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x562531a9c6a0 0x562531a9f500 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x562531aa3d60 msgr2=0x562531aa6150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x562531aa3d60 0x562531aa6150 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562531a98110 msgr2=0x562531a9a500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x562531a98110 0x562531a9a500 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x562531aa03c0 msgr2=0x562531aa27b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x562531aa03c0 0x562531aa27b0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 reap_dead start 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 reap_dead start 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a034b640 1 -- 172.21.15.40:0/26993455 reap_dead start 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f388006b2d0 msgr2=0x7f388006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f388006b2d0 0x7f388006d790 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3890059000 tx=0x7f3890051000 comp rx=0 tx=0).stop 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319ec420 msgr2=0x562531a7d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319ec420 0x562531a7d6f0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f3888007db0 tx=0x7f3888006750 comp rx=0 tx=0).stop 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 shutdown_connections 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f388006b2d0 0x7f388006d790 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 --2- 172.21.15.40:0/26993455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5625319ec420 0x562531a7d6f0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 >> 172.21.15.40:0/26993455 conn(0x562531916330 msgr2=0x5625319e99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 shutdown_connections 2024-04-26T21:44:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.452+0000 7f38a2b1ef40 1 -- 172.21.15.40:0/26993455 wait complete. 2024-04-26T21:44:04.457 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000 2024-04-26T21:44:04.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.497+0000 7f6752d28f40 1 Processor -- start 2024-04-26T21:44:04.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.497+0000 7f6752d28f40 1 -- start start 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f6752d28f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 0x55b79279f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f6752d28f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b792714510 0x55b79279fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f6752d28f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7927a0820 0x55b792794d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f6752d28f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b7927a0c00 con 0x55b7927a0820 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f6752d28f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b792672e40 con 0x55b7926fb650 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f6752d28f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b7927a0e60 con 0x55b792714510 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f67505be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7927a0820 0x55b792794d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f67505be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7927a0820 0x55b792794d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59984/0 (socket says 172.21.15.40:59984) 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f67505be640 1 -- 172.21.15.40:0/1265785608 learned_addr learned my addr 172.21.15.40:0/1265785608 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f67505be640 1 -- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b792714510 msgr2=0x55b79279fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f674fdbd640 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 0x55b79279f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f674f5bc640 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b792714510 0x55b79279fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f67505be640 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b792714510 0x55b79279fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f67505be640 1 -- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 msgr2=0x55b79279f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f67505be640 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 0x55b79279f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f67505be640 1 -- 172.21.15.40:0/1265785608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7926cd6a0 con 0x55b7927a0820 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f674fdbd640 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 0x55b79279f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.498+0000 7f674f5bc640 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b792714510 0x55b79279fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f67505be640 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7927a0820 0x55b792794d90 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f674000a4b0 tx=0x7f6740050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d195ae182f106b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f674e5ba640 1 -- 172.21.15.40:0/1265785608 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740054040 con 0x55b7927a0820 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f674e5ba640 1 -- 172.21.15.40:0/1265785608 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6740050960 con 0x55b7927a0820 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f674e5ba640 1 -- 172.21.15.40:0/1265785608 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740050d50 con 0x55b7927a0820 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 -- 172.21.15.40:0/1265785608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7927a0820 msgr2=0x55b792794d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7927a0820 0x55b792794d90 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f674000a4b0 tx=0x7f6740050000 comp rx=0 tx=0).stop 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 -- 172.21.15.40:0/1265785608 shutdown_connections 2024-04-26T21:44:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7927a0820 0x55b792794d90 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b792714510 0x55b79279fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 --2- 172.21.15.40:0/1265785608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 0x55b79279f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 -- 172.21.15.40:0/1265785608 >> 172.21.15.40:0/1265785608 conn(0x55b7926f58a0 msgr2=0x55b7926cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 -- 172.21.15.40:0/1265785608 shutdown_connections 2024-04-26T21:44:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.499+0000 7f6752d28f40 1 -- 172.21.15.40:0/1265785608 wait complete. 2024-04-26T21:44:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.500+0000 7f6752d28f40 1 Processor -- start 2024-04-26T21:44:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.500+0000 7f6752d28f40 1 -- start start 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f6752d28f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 0x55b792792ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f6752d28f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b792714510 0x55b79282f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f6752d28f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7927a0820 0x55b79282fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f6752d28f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b7926cd980 con 0x55b792714510 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f6752d28f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b79279cd40 con 0x55b7926fb650 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f6752d28f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b7926cdc60 con 0x55b7927a0820 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b792714510 0x55b79282f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b792714510 0x55b79282f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59994/0 (socket says 172.21.15.40:59994) 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 -- 172.21.15.40:0/3450196215 learned_addr learned my addr 172.21.15.40:0/3450196215 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674fdbd640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 0x55b792792ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f67505be640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7927a0820 0x55b79282fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 -- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7927a0820 msgr2=0x55b79282fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7927a0820 0x55b79282fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 -- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 msgr2=0x55b792792ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7926fb650 0x55b792792ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 -- 172.21.15.40:0/3450196215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b792830070 con 0x55b792714510 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674f5bc640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b792714510 0x55b79282f4e0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f6738009d70 tx=0x7f6738006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674d5b8640 1 -- 172.21.15.40:0/3450196215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738006e60 con 0x55b792714510 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674d5b8640 1 -- 172.21.15.40:0/3450196215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6738014430 con 0x55b792714510 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7927a24d0 con 0x55b792714510 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.501+0000 7f674d5b8640 1 -- 172.21.15.40:0/3450196215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738014800 con 0x55b792714510 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.502+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7926f72d0 con 0x55b792714510 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.503+0000 7f674d5b8640 1 -- 172.21.15.40:0/3450196215 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67380149a0 con 0x55b792714510 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.503+0000 7f674d5b8640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f673006b2d0 0x7f673006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.503+0000 7f674d5b8640 1 -- 172.21.15.40:0/3450196215 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7f673808a730 con 0x55b792714510 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.503+0000 7f674fdbd640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f673006b2d0 0x7f673006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.504+0000 7f6752d28f40 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b792798a90 0x55b79283b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.504+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55b7927a2cd0 con 0x55b792798a90 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.504+0000 7f67505be640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b792798a90 0x55b79283b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.504+0000 7f674fdbd640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f673006b2d0 0x7f673006d790 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6748005fd0 tx=0x7f6748005770 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.504+0000 7f67505be640 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b792798a90 0x55b79283b4f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.505+0000 7f67505be640 1 -- 172.21.15.40:0/3450196215 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 1.00000000 [delete] v109'180 uv180 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6740056030 con 0x55b792798a90 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.505+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b792798a90 msgr2=0x55b79283b4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.505+0000 7f6752d28f40 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b792798a90 0x55b79283b4f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.505+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f673006b2d0 msgr2=0x7f673006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.505+0000 7f6752d28f40 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f673006b2d0 0x7f673006d790 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6748005fd0 tx=0x7f6748005770 comp rx=0 tx=0).stop 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.506+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b792714510 msgr2=0x55b79282f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.506+0000 7f6752d28f40 1 --2- 172.21.15.40:0/3450196215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b792714510 0x55b79282f4e0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f6738009d70 tx=0x7f6738006750 comp rx=0 tx=0).stop 2024-04-26T21:44:04.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.506+0000 7f67505be640 1 -- 172.21.15.40:0/3450196215 reap_dead start 2024-04-26T21:44:04.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.506+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 shutdown_connections 2024-04-26T21:44:04.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.506+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 >> 172.21.15.40:0/3450196215 conn(0x55b7926f58a0 msgr2=0x55b7926c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:04.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.506+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 shutdown_connections 2024-04-26T21:44:04.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.506+0000 7f6752d28f40 1 -- 172.21.15.40:0/3450196215 wait complete. 2024-04-26T21:44:04.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:04.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.655+0000 7f22c72d1640 1 Processor -- start 2024-04-26T21:44:04.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.655+0000 7f22c72d1640 1 -- start start 2024-04-26T21:44:04.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c72d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22c00d3bc0 0x7f22c00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c72d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c00d1040 con 0x7f22c015f170 2024-04-26T21:44:04.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c72d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c00c9680 con 0x7f22c015f550 2024-04-26T21:44:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c72d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f22c00c9430 con 0x7f22c00d3bc0 2024-04-26T21:44:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c586e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22c00d3bc0 0x7f22c00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c506d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f22c015f170 0x7f22c00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43102/0 (socket says 172.21.15.40:43102) 2024-04-26T21:44:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c506d640 1 -- 172.21.15.40:0/409960322 learned_addr learned my addr 172.21.15.40:0/409960322 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c586e640 1 -- 172.21.15.40:0/409960322 >> v1:172.21.15.184:6789/0 conn(0x7f22c015f550 legacy=0x7f22c00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c586e640 1 -- 172.21.15.40:0/409960322 >> v1:172.21.15.40:6789/0 conn(0x7f22c015f170 legacy=0x7f22c00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22b7fff640 1 -- 172.21.15.40:0/409960322 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465296460 0 0) 0x7f22bc001110 con 0x7f22c015f170 2024-04-26T21:44:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.656+0000 7f22c586e640 1 -- 172.21.15.40:0/409960322 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c00c9960 con 0x7f22c00d3bc0 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.657+0000 7f22c586e640 1 --2- 172.21.15.40:0/409960322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c00d5a80 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f22c00c95b0 tx=0x7f22b802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f64d7a3cd80fbea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.657+0000 7f22b7fff640 1 -- 172.21.15.40:0/409960322 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8007740 con 0x7f22c00d3bc0 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.657+0000 7f22b7fff640 1 -- 172.21.15.40:0/409960322 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b80078e0 con 0x7f22c00d3bc0 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.657+0000 7f22b7fff640 1 -- 172.21.15.40:0/409960322 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8007cb0 con 0x7f22c00d3bc0 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.658+0000 7f22c72d1640 1 -- 172.21.15.40:0/409960322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 msgr2=0x7f22c00d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.658+0000 7f22c72d1640 1 --2- 172.21.15.40:0/409960322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c00d5a80 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f22c00c95b0 tx=0x7f22b802a710 comp rx=0 tx=0).stop 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.658+0000 7f22c72d1640 1 -- 172.21.15.40:0/409960322 shutdown_connections 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.658+0000 7f22c72d1640 1 --2- 172.21.15.40:0/409960322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c00d5a80 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.658+0000 7f22c72d1640 1 -- 172.21.15.40:0/409960322 >> 172.21.15.40:0/409960322 conn(0x7f22c00d4f60 msgr2=0x7f22c00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.658+0000 7f22c72d1640 1 -- 172.21.15.40:0/409960322 shutdown_connections 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.658+0000 7f22c72d1640 1 -- 172.21.15.40:0/409960322 wait complete. 2024-04-26T21:44:04.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.659+0000 7f22c72d1640 1 Processor -- start 2024-04-26T21:44:04.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.659+0000 7f22c72d1640 1 -- start start 2024-04-26T21:44:04.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c72d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c72d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22c015f170 0x7f22c015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c72d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22c01f4fb0 0x7f22c0158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c72d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f22c00d2e30 con 0x7f22c00d3bc0 2024-04-26T21:44:04.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c72d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22c00d2cb0 con 0x7f22c015f170 2024-04-26T21:44:04.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c72d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22c00d2fb0 con 0x7f22c01f4fb0 2024-04-26T21:44:04.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c506d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c506d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c015ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60008/0 (socket says 172.21.15.40:60008) 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c506d640 1 -- 172.21.15.40:0/3609445269 learned_addr learned my addr 172.21.15.40:0/3609445269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c586e640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22c01f4fb0 0x7f22c0158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c486c640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22c015f170 0x7f22c015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c506d640 1 -- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22c01f4fb0 msgr2=0x7f22c0158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c506d640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22c01f4fb0 0x7f22c0158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c506d640 1 -- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22c015f170 msgr2=0x7f22c015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c506d640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22c015f170 0x7f22c015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.660+0000 7f22c506d640 1 -- 172.21.15.40:0/3609445269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c0159ba0 con 0x7f22c00d3bc0 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.661+0000 7f22c506d640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c015ade0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00c780 tx=0x7f22bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.661+0000 7f22b5ffb640 1 -- 172.21.15.40:0/3609445269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc018070 con 0x7f22c00d3bc0 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.661+0000 7f22b5ffb640 1 -- 172.21.15.40:0/3609445269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22bc0072d0 con 0x7f22c00d3bc0 2024-04-26T21:44:04.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.661+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c0159e80 con 0x7f22c00d3bc0 2024-04-26T21:44:04.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.661+0000 7f22b5ffb640 1 -- 172.21.15.40:0/3609445269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0047e0 con 0x7f22c00d3bc0 2024-04-26T21:44:04.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.661+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c015a440 con 0x7f22c00d3bc0 2024-04-26T21:44:04.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.662+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c00694f0 con 0x7f22c00d3bc0 2024-04-26T21:44:04.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.662+0000 7f22b5ffb640 1 -- 172.21.15.40:0/3609445269 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22bc01b070 con 0x7f22c00d3bc0 2024-04-26T21:44:04.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.663+0000 7f22b5ffb640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f22a406c920 0x7f22a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.663+0000 7f22b5ffb640 1 -- 172.21.15.40:0/3609445269 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7f22bc08b4b0 con 0x7f22c00d3bc0 2024-04-26T21:44:04.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.663+0000 7f22c486c640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f22a406c920 0x7f22a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.664+0000 7f22c486c640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f22a406c920 0x7f22a406ede0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f22c015d730 tx=0x7f22b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.667+0000 7f22b5ffb640 1 -- 172.21.15.40:0/3609445269 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22c00d1040 con 0x7f22c00d3bc0 2024-04-26T21:44:04.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:04.895+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f22c00d5b10 con 0x7f22c00d3bc0 2024-04-26T21:44:05.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.236+0000 7f22b5ffb640 1 -- 172.21.15.40:0/3609445269 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v178) ==== 162+0+0 (secure 0 0 0) 0x7f22bc007d00 con 0x7f22c00d3bc0 2024-04-26T21:44:05.237 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f22a406c920 msgr2=0x7f22a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f22a406c920 0x7f22a406ede0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f22c015d730 tx=0x7f22b0009040 comp rx=0 tx=0).stop 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 msgr2=0x7f22c015ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c015ade0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00c780 tx=0x7f22bc00cc40 comp rx=0 tx=0).stop 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 shutdown_connections 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f22a406c920 0x7f22a406ede0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22c01f4fb0 0x7f22c0158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22c015f170 0x7f22c015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 --2- 172.21.15.40:0/3609445269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22c00d3bc0 0x7f22c015ade0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 >> 172.21.15.40:0/3609445269 conn(0x7f22c00d4f60 msgr2=0x7f22c00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 shutdown_connections 2024-04-26T21:44:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.240+0000 7f22c72d1640 1 -- 172.21.15.40:0/3609445269 wait complete. 2024-04-26T21:44:05.253 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.354+0000 7eff32e09640 1 Processor -- start 2024-04-26T21:44:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.354+0000 7eff32e09640 1 -- start start 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff32e09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff2c0d3bc0 0x7eff2c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff32e09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 0x7eff2c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff32e09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2c0b7e50 con 0x7eff2c162880 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff32e09640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7eff2c0d0580 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff32e09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c0d1800 con 0x7eff2c0b5f20 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff2bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff2c0d3bc0 0x7eff2c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff30ba5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7eff2c162880 0x7eff2c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43104/0 (socket says 172.21.15.40:43104) 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff30ba5640 1 -- 172.21.15.40:0/2459157661 learned_addr learned my addr 172.21.15.40:0/2459157661 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff313a6640 1 --2- 172.21.15.40:0/2459157661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 0x7eff2c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff2bfff640 1 -- 172.21.15.40:0/2459157661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 msgr2=0x7eff2c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff2bfff640 1 --2- 172.21.15.40:0/2459157661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 0x7eff2c0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff2bfff640 1 -- 172.21.15.40:0/2459157661 >> v1:172.21.15.40:6789/0 conn(0x7eff2c162880 legacy=0x7eff2c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:05.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.355+0000 7eff2bfff640 1 -- 172.21.15.40:0/2459157661 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c0d07d0 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.356+0000 7eff2bfff640 1 --2- 172.21.15.40:0/2459157661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c0d2cb0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7eff200097a0 tx=0x7eff2002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba9c98d01392070 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:05.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.356+0000 7eff2b7fe640 1 -- 172.21.15.40:0/2459157661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20037070 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.356+0000 7eff2b7fe640 1 -- 172.21.15.40:0/2459157661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff20032440 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.356+0000 7eff2b7fe640 1 -- 172.21.15.40:0/2459157661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20032810 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.357+0000 7eff32e09640 1 -- 172.21.15.40:0/2459157661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 msgr2=0x7eff2c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:05.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.357+0000 7eff32e09640 1 --2- 172.21.15.40:0/2459157661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c0d2cb0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7eff200097a0 tx=0x7eff2002a180 comp rx=0 tx=0).stop 2024-04-26T21:44:05.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.357+0000 7eff32e09640 1 -- 172.21.15.40:0/2459157661 shutdown_connections 2024-04-26T21:44:05.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.357+0000 7eff32e09640 1 --2- 172.21.15.40:0/2459157661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 0x7eff2c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.357+0000 7eff32e09640 1 --2- 172.21.15.40:0/2459157661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c0d2cb0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.357+0000 7eff32e09640 1 -- 172.21.15.40:0/2459157661 >> 172.21.15.40:0/2459157661 conn(0x7eff2c0d4f60 msgr2=0x7eff2c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.357+0000 7eff32e09640 1 -- 172.21.15.40:0/2459157661 shutdown_connections 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.357+0000 7eff32e09640 1 -- 172.21.15.40:0/2459157661 wait complete. 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.358+0000 7eff32e09640 1 Processor -- start 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.358+0000 7eff32e09640 1 -- start start 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff32e09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 0x7eff2c14a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff32e09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff32e09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff2c162880 0x7eff2c1f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff32e09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c162c60 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff32e09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c0d0d90 con 0x7eff2c162880 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff32e09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c162de0 con 0x7eff2c0b5f20 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c14dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60024/0 (socket says 172.21.15.40:60024) 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 -- 172.21.15.40:0/3749223009 learned_addr learned my addr 172.21.15.40:0/3749223009 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:05.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff30ba5640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 0x7eff2c14a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff313a6640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff2c162880 0x7eff2c1f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 -- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 msgr2=0x7eff2c14a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 0x7eff2c14a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 -- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff2c162880 msgr2=0x7eff2c1f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff2c162880 0x7eff2c1f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 -- 172.21.15.40:0/3749223009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c1f4bc0 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff2bfff640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c14dfc0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7eff20035040 tx=0x7eff200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff297fa640 1 -- 172.21.15.40:0/3749223009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20037040 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff297fa640 1 -- 172.21.15.40:0/3749223009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff20030770 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.359+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c1f8550 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.360+0000 7eff297fa640 1 -- 172.21.15.40:0/3749223009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20030b80 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.360+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c1f8b10 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.360+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c0694f0 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.361+0000 7eff297fa640 1 -- 172.21.15.40:0/3749223009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff20044020 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.361+0000 7eff297fa640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:05.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.361+0000 7eff297fa640 1 -- 172.21.15.40:0/3749223009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7421+0+0 (secure 0 0 0) 0x7eff200ac620 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.362+0000 7eff30ba5640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:05.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.362+0000 7eff30ba5640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff1806c8f0 0x7eff1806edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7eff1c007fd0 tx=0x7eff1c007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:05.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.366+0000 7eff297fa640 1 -- 172.21.15.40:0/3749223009 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff20032e20 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.588+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff2c0cb7d0 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.589+0000 7eff297fa640 1 -- 172.21.15.40:0/3749223009 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) ==== 95+0+5190 (secure 0 0 0) 0x7eff2007a430 con 0x7eff2c0d3bc0 2024-04-26T21:44:05.590 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:05.590 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":179,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:05.240446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"c","rank":0,"incarnation":179,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/3588039422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3588039422},{"type":"v1","addr":"172.21.15.40:6833","nonce":3588039422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:05.590 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 179 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff1806c8f0 msgr2=0x7eff1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff1806c8f0 0x7eff1806edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7eff1c007fd0 tx=0x7eff1c007e80 comp rx=0 tx=0).stop 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 msgr2=0x7eff2c14dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c14dfc0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7eff20035040 tx=0x7eff200046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 shutdown_connections 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff2c162880 0x7eff2c1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c0d3bc0 0x7eff2c14dfc0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 --2- 172.21.15.40:0/3749223009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2c0b5f20 0x7eff2c14a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:05.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 >> 172.21.15.40:0/3749223009 conn(0x7eff2c0d4f60 msgr2=0x7eff2c0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:05.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 shutdown_connections 2024-04-26T21:44:05.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:05.591+0000 7eff32e09640 1 -- 172.21.15.40:0/3749223009 wait complete. 2024-04-26T21:44:05.603 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:44:08.263 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:08.263+0000 7f520e38e640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-04-26T21:44:08.264 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:08.263+0000 7f520e38e640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-04-26T21:44:08.264 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:08.263+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:08.264 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:08.263+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:08.272 INFO:tasks.ceph.mds.c.smithi040.stderr: -16> 2024-04-26T21:44:08.263+0000 7f520e38e640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-04-26T21:44:08.272 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:44:08.263+0000 7f520e38e640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-04-26T21:44:08.299 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:10.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:10.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.716+0000 7f0037cb0640 1 Processor -- start 2024-04-26T21:44:10.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.717+0000 7f0037cb0640 1 -- start start 2024-04-26T21:44:10.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.717+0000 7f0037cb0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00300d0070 0x7f00300d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:10.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.717+0000 7f0037cb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003006d560 0x7f003006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:10.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.717+0000 7f0037cb0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00300d2420 con 0x7f003006e0a0 2024-04-26T21:44:10.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.717+0000 7f0037cb0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0030162d00 con 0x7f00300d0070 2024-04-26T21:44:10.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.717+0000 7f0037cb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00300d2d10 con 0x7f003006d560 2024-04-26T21:44:10.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.717+0000 7f003624d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f003006e0a0 0x7f0030162f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48814/0 (socket says 172.21.15.40:48814) 2024-04-26T21:44:10.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.717+0000 7f003624d640 1 -- 172.21.15.40:0/2339243321 learned_addr learned my addr 172.21.15.40:0/2339243321 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f003524b640 1 --2- 172.21.15.40:0/2339243321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003006d560 0x7f003006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f0035a4c640 1 --2- 172.21.15.40:0/2339243321 >> v2:172.21.15.184:3300/0 conn(0x7f00300d0070 0x7f00300d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f0034a4a640 1 -- 172.21.15.40:0/2339243321 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3210397201 0 0) 0x7f00300d2420 con 0x7f003006e0a0 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f0034a4a640 1 -- 172.21.15.40:0/2339243321 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f001c003520 con 0x7f003006e0a0 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f003524b640 1 -- 172.21.15.40:0/2339243321 >> v2:172.21.15.184:3300/0 conn(0x7f00300d0070 msgr2=0x7f00300d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f003524b640 1 --2- 172.21.15.40:0/2339243321 >> v2:172.21.15.184:3300/0 conn(0x7f00300d0070 0x7f00300d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f003524b640 1 -- 172.21.15.40:0/2339243321 >> v1:172.21.15.40:6789/0 conn(0x7f003006e0a0 legacy=0x7f0030162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f0034a4a640 1 -- 172.21.15.40:0/2339243321 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 210365516 0 0) 0x7f001c003520 con 0x7f003006e0a0 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f003524b640 1 -- 172.21.15.40:0/2339243321 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00300cd6b0 con 0x7f003006d560 2024-04-26T21:44:10.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.718+0000 7f0035a4c640 1 --2- 172.21.15.40:0/2339243321 >> v2:172.21.15.184:3300/0 conn(0x7f00300d0070 0x7f00300d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.719+0000 7f003524b640 1 --2- 172.21.15.40:0/2339243321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003006d560 0x7f003006db60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00200097a0 tx=0x7f002002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa18642ec66d31e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.719+0000 7f0034a4a640 1 -- 172.21.15.40:0/2339243321 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020037070 con 0x7f003006d560 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.719+0000 7f0034a4a640 1 -- 172.21.15.40:0/2339243321 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002002ac30 con 0x7f003006d560 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.719+0000 7f0034a4a640 1 -- 172.21.15.40:0/2339243321 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00280019f0 con 0x7f003006d560 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.720+0000 7f0037cb0640 1 -- 172.21.15.40:0/2339243321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003006d560 msgr2=0x7f003006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.720+0000 7f0037cb0640 1 --2- 172.21.15.40:0/2339243321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003006d560 0x7f003006db60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00200097a0 tx=0x7f002002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.720+0000 7f0037cb0640 1 -- 172.21.15.40:0/2339243321 shutdown_connections 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.720+0000 7f0037cb0640 1 --2- 172.21.15.40:0/2339243321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003006d560 0x7f003006db60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.720+0000 7f0037cb0640 1 --2- 172.21.15.40:0/2339243321 >> v2:172.21.15.184:3300/0 conn(0x7f00300d0070 0x7f00300d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.720+0000 7f0037cb0640 1 -- 172.21.15.40:0/2339243321 >> 172.21.15.40:0/2339243321 conn(0x7f00300d2ec0 msgr2=0x7f00300d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:10.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.720+0000 7f0037cb0640 1 -- 172.21.15.40:0/2339243321 shutdown_connections 2024-04-26T21:44:10.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.720+0000 7f0037cb0640 1 -- 172.21.15.40:0/2339243321 wait complete. 2024-04-26T21:44:10.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.721+0000 7f0037cb0640 1 Processor -- start 2024-04-26T21:44:10.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.721+0000 7f0037cb0640 1 -- start start 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0037cb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003006d560 0x7f00301534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0037cb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003006e0a0 0x7f00301f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0037cb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00300d0070 0x7f00300cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0037cb0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00300cf1a0 con 0x7f003006d560 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0037cb0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00300cdc70 con 0x7f003006e0a0 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0037cb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00300cf320 con 0x7f00300d0070 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003006d560 0x7f00301534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003006d560 0x7f00301534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55092/0 (socket says 172.21.15.40:55092) 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 -- 172.21.15.40:0/1601107754 learned_addr learned my addr 172.21.15.40:0/1601107754 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:10.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f003624d640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00300d0070 0x7f00300cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f003524b640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003006e0a0 0x7f00301f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 -- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00300d0070 msgr2=0x7f00300cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00300d0070 0x7f00300cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 -- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003006e0a0 msgr2=0x7f00301f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003006e0a0 0x7f00301f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 -- 172.21.15.40:0/1601107754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00301f94e0 con 0x7f003006d560 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f003624d640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00300d0070 0x7f00300cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.722+0000 7f0035a4c640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003006d560 0x7f00301534d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f002c00ebe0 tx=0x7f002c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.723+0000 7f00267fc640 1 -- 172.21.15.40:0/1601107754 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c017070 con 0x7f003006d560 2024-04-26T21:44:10.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.723+0000 7f00267fc640 1 -- 172.21.15.40:0/1601107754 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c00cc30 con 0x7f003006d560 2024-04-26T21:44:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.723+0000 7f00267fc640 1 -- 172.21.15.40:0/1601107754 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c013470 con 0x7f003006d560 2024-04-26T21:44:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.723+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00301f4fa0 con 0x7f003006d560 2024-04-26T21:44:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.723+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00301f5520 con 0x7f003006d560 2024-04-26T21:44:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.724+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00300694f0 con 0x7f003006d560 2024-04-26T21:44:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.724+0000 7f00267fc640 1 -- 172.21.15.40:0/1601107754 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f002c01a070 con 0x7f003006d560 2024-04-26T21:44:10.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.725+0000 7f00267fc640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:10.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.725+0000 7f00267fc640 1 -- 172.21.15.40:0/1601107754 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7507+0+0 (secure 0 0 0) 0x7f002c08c2b0 con 0x7f003006d560 2024-04-26T21:44:10.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.725+0000 7f003524b640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:10.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.726+0000 7f003524b640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f001406c8f0 0x7f001406edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0020034000 tx=0x7f002002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:10.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.729+0000 7f00267fc640 1 -- 172.21.15.40:0/1601107754 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00301f94e0 con 0x7f003006d560 2024-04-26T21:44:10.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.954+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00300d0470 con 0x7f003006d560 2024-04-26T21:44:10.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.954+0000 7f00267fc640 1 -- 172.21.15.40:0/1601107754 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) ==== 95+0+5181 (secure 0 0 0) 0x7f002c059f30 con 0x7f003006d560 2024-04-26T21:44:10.955 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:10.955 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":184,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9513,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2324764539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2324764539},{"type":"v1","addr":"172.21.15.40:6833","nonce":2324764539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:08.264886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:10.955 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 184 2024-04-26T21:44:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.956+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f001406c8f0 msgr2=0x7f001406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.956+0000 7f0037cb0640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f001406c8f0 0x7f001406edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0020034000 tx=0x7f002002e000 comp rx=0 tx=0).stop 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.956+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003006d560 msgr2=0x7f00301534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.956+0000 7f0037cb0640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003006d560 0x7f00301534d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f002c00ebe0 tx=0x7f002c00c520 comp rx=0 tx=0).stop 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.956+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 shutdown_connections 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.956+0000 7f0037cb0640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.956+0000 7f0037cb0640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00300d0070 0x7f00300cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.957+0000 7f0037cb0640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003006e0a0 0x7f00301f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.957+0000 7f0037cb0640 1 --2- 172.21.15.40:0/1601107754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003006d560 0x7f00301534d0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.957+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 >> 172.21.15.40:0/1601107754 conn(0x7f00300d2ec0 msgr2=0x7f0030071680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.957+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 shutdown_connections 2024-04-26T21:44:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:10.957+0000 7f0037cb0640 1 -- 172.21.15.40:0/1601107754 wait complete. 2024-04-26T21:44:10.968 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:44:10.968 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:11.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.068+0000 7ff80f272640 1 Processor -- start 2024-04-26T21:44:11.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.068+0000 7ff80f272640 1 -- start start 2024-04-26T21:44:11.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80f272640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff808071e60 0x7ff80815df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80f272640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff808166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80f272640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff80815e4d0 con 0x7ff808072240 2024-04-26T21:44:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80f272640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff8080701f0 con 0x7ff808071e60 2024-04-26T21:44:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80f272640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff80806e240 con 0x7ff8080cc5b0 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80d80f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff808166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80d00e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff808071e60 0x7ff80815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80d80f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff808166b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:34810/0 (socket says 172.21.15.40:34810) 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.069+0000 7ff80d80f640 1 -- 172.21.15.40:0/920897950 learned_addr learned my addr 172.21.15.40:0/920897950 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.070+0000 7ff80d80f640 1 -- 172.21.15.40:0/920897950 >> v2:172.21.15.184:3300/0 conn(0x7ff808071e60 msgr2=0x7ff80815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.070+0000 7ff80d80f640 1 --2- 172.21.15.40:0/920897950 >> v2:172.21.15.184:3300/0 conn(0x7ff808071e60 0x7ff80815df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.070+0000 7ff80d80f640 1 -- 172.21.15.40:0/920897950 >> v1:172.21.15.184:6789/0 conn(0x7ff808072240 legacy=0x7ff8080726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.070+0000 7ff80d80f640 1 -- 172.21.15.40:0/920897950 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80806fc30 con 0x7ff8080cc5b0 2024-04-26T21:44:11.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.070+0000 7ff80d00e640 1 --2- 172.21.15.40:0/920897950 >> v2:172.21.15.184:3300/0 conn(0x7ff808071e60 0x7ff80815df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:11.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.070+0000 7ff80d80f640 1 --2- 172.21.15.40:0/920897950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff808166b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff800009e50 tx=0x7ff80002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbfba5501c7b676 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff7fffff640 1 -- 172.21.15.40:0/920897950 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800036070 con 0x7ff8080cc5b0 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff7fffff640 1 -- 172.21.15.40:0/920897950 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff800031600 con 0x7ff8080cc5b0 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff7fffff640 1 -- 172.21.15.40:0/920897950 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8000319d0 con 0x7ff8080cc5b0 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff80f272640 1 -- 172.21.15.40:0/920897950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 msgr2=0x7ff808166b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff80f272640 1 --2- 172.21.15.40:0/920897950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff808166b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff800009e50 tx=0x7ff80002a680 comp rx=0 tx=0).stop 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff80f272640 1 -- 172.21.15.40:0/920897950 shutdown_connections 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff80f272640 1 --2- 172.21.15.40:0/920897950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff808166b30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff80f272640 1 --2- 172.21.15.40:0/920897950 >> v2:172.21.15.184:3300/0 conn(0x7ff808071e60 0x7ff80815df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff80f272640 1 -- 172.21.15.40:0/920897950 >> 172.21.15.40:0/920897950 conn(0x7ff8080d0be0 msgr2=0x7ff8080d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff80f272640 1 -- 172.21.15.40:0/920897950 shutdown_connections 2024-04-26T21:44:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.071+0000 7ff80f272640 1 -- 172.21.15.40:0/920897950 wait complete. 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.072+0000 7ff80f272640 1 Processor -- start 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.072+0000 7ff80f272640 1 -- start start 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80f272640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff808071e60 0x7ff80814cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80f272640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff808072240 0x7ff8080aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80f272640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff80815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80f272640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff80806f8e0 con 0x7ff808071e60 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80f272640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff808073fb0 con 0x7ff808072240 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80f272640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff80806ddf0 con 0x7ff8080cc5b0 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80d00e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff808071e60 0x7ff80814cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80d00e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff808071e60 0x7ff80814cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55096/0 (socket says 172.21.15.40:55096) 2024-04-26T21:44:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80d00e640 1 -- 172.21.15.40:0/2794496030 learned_addr learned my addr 172.21.15.40:0/2794496030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80d80f640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff80815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.073+0000 7ff80c80d640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff808072240 0x7ff8080aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff80d00e640 1 -- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 msgr2=0x7ff80815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff80d00e640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff80815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff80d00e640 1 -- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff808072240 msgr2=0x7ff8080aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff80d00e640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff808072240 0x7ff8080aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff80d00e640 1 -- 172.21.15.40:0/2794496030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80814d300 con 0x7ff808071e60 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff80d00e640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff808071e60 0x7ff80814cdc0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800e990 tx=0x7ff7f800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff7fdffb640 1 -- 172.21.15.40:0/2794496030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8017070 con 0x7ff808071e60 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff7fdffb640 1 -- 172.21.15.40:0/2794496030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f800c9e0 con 0x7ff808071e60 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80814a6a0 con 0x7ff808071e60 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff7fdffb640 1 -- 172.21.15.40:0/2794496030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800cdb0 con 0x7ff808071e60 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.074+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80814ac60 con 0x7ff808071e60 2024-04-26T21:44:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.075+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8080694f0 con 0x7ff808071e60 2024-04-26T21:44:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.076+0000 7ff7fdffb640 1 -- 172.21.15.40:0/2794496030 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7f801a070 con 0x7ff808071e60 2024-04-26T21:44:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.076+0000 7ff7fdffb640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7ec06c8f0 0x7ff7ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.076+0000 7ff7fdffb640 1 -- 172.21.15.40:0/2794496030 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7507+0+0 (secure 0 0 0) 0x7ff7f808c1c0 con 0x7ff808071e60 2024-04-26T21:44:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.076+0000 7ff80c80d640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7ec06c8f0 0x7ff7ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.077+0000 7ff80c80d640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7ec06c8f0 0x7ff7ec06edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff808159aa0 tx=0x7ff804009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.080+0000 7ff7fdffb640 1 -- 172.21.15.40:0/2794496030 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7f8059e60 con 0x7ff808071e60 2024-04-26T21:44:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.305+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff80815f310 con 0x7ff808071e60 2024-04-26T21:44:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.306+0000 7ff7fdffb640 1 -- 172.21.15.40:0/2794496030 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) ==== 95+0+5181 (secure 0 0 0) 0x7ff7f8059760 con 0x7ff808071e60 2024-04-26T21:44:11.307 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:11.307 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":184,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9513,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2324764539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2324764539},{"type":"v1","addr":"172.21.15.40:6833","nonce":2324764539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:08.264886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 184 2024-04-26T21:44:11.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7ec06c8f0 msgr2=0x7ff7ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7ec06c8f0 0x7ff7ec06edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff808159aa0 tx=0x7ff804009040 comp rx=0 tx=0).stop 2024-04-26T21:44:11.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff808071e60 msgr2=0x7ff80814cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff808071e60 0x7ff80814cdc0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800e990 tx=0x7ff7f800ee50 comp rx=0 tx=0).stop 2024-04-26T21:44:11.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 shutdown_connections 2024-04-26T21:44:11.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff7ec06c8f0 0x7ff7ec06edb0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8080cc5b0 0x7ff80815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff808072240 0x7ff8080aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 --2- 172.21.15.40:0/2794496030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff808071e60 0x7ff80814cdc0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 >> 172.21.15.40:0/2794496030 conn(0x7ff8080d0be0 msgr2=0x7ff80815a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 shutdown_connections 2024-04-26T21:44:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.308+0000 7ff80f272640 1 -- 172.21.15.40:0/2794496030 wait complete. 2024-04-26T21:44:11.320 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:11.320 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:11.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.420+0000 7ff2b06f8640 1 Processor -- start 2024-04-26T21:44:11.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.420+0000 7ff2b06f8640 1 -- start start 2024-04-26T21:44:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.421+0000 7ff2b06f8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2a80b3860 0x7ff2a816a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.421+0000 7ff2b06f8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a80d0ba0 con 0x7ff2a815f3a0 2024-04-26T21:44:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.421+0000 7ff2b06f8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a80d2590 con 0x7ff2a80d06f0 2024-04-26T21:44:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.421+0000 7ff2b06f8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a80d0e20 con 0x7ff2a80b3860 2024-04-26T21:44:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.421+0000 7ff2ae494640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff2a815f3a0 0x7ff2a80d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48818/0 (socket says 172.21.15.40:48818) 2024-04-26T21:44:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.421+0000 7ff2aec95640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2a80b3860 0x7ff2a816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.421+0000 7ff2ae494640 1 -- 172.21.15.40:0/961068601 learned_addr learned my addr 172.21.15.40:0/961068601 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.422+0000 7ff2aec95640 1 -- 172.21.15.40:0/961068601 >> v1:172.21.15.184:6789/0 conn(0x7ff2a80d06f0 legacy=0x7ff2a80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.422+0000 7ff2aec95640 1 -- 172.21.15.40:0/961068601 >> v1:172.21.15.40:6789/0 conn(0x7ff2a815f3a0 legacy=0x7ff2a80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.422+0000 7ff2aec95640 1 -- 172.21.15.40:0/961068601 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a80b1d70 con 0x7ff2a80b3860 2024-04-26T21:44:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.422+0000 7ff2aec95640 1 --2- 172.21.15.40:0/961068601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a816a000 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009d90 tx=0x7ff2a402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deb087acb49c1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2ad492640 1 -- 172.21.15.40:0/961068601 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4031600 con 0x7ff2a80b3860 2024-04-26T21:44:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2ad492640 1 -- 172.21.15.40:0/961068601 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a40317a0 con 0x7ff2a80b3860 2024-04-26T21:44:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2ad492640 1 -- 172.21.15.40:0/961068601 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4031b70 con 0x7ff2a80b3860 2024-04-26T21:44:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2b06f8640 1 -- 172.21.15.40:0/961068601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 msgr2=0x7ff2a816a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2b06f8640 1 --2- 172.21.15.40:0/961068601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a816a000 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009d90 tx=0x7ff2a402a610 comp rx=0 tx=0).stop 2024-04-26T21:44:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2b06f8640 1 -- 172.21.15.40:0/961068601 shutdown_connections 2024-04-26T21:44:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2b06f8640 1 --2- 172.21.15.40:0/961068601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a816a000 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2b06f8640 1 -- 172.21.15.40:0/961068601 >> 172.21.15.40:0/961068601 conn(0x7ff2a80d4f90 msgr2=0x7ff2a80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2b06f8640 1 -- 172.21.15.40:0/961068601 shutdown_connections 2024-04-26T21:44:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.423+0000 7ff2b06f8640 1 -- 172.21.15.40:0/961068601 wait complete. 2024-04-26T21:44:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.424+0000 7ff2b06f8640 1 Processor -- start 2024-04-26T21:44:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.424+0000 7ff2b06f8640 1 -- start start 2024-04-26T21:44:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2b06f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a80c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2b06f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a80d06f0 0x7ff2a81532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2b06f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a815f3a0 0x7ff2a80ba390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2b06f8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a815f780 con 0x7ff2a80b3860 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2b06f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a80b2330 con 0x7ff2a80d06f0 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2b06f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a815f900 con 0x7ff2a815f3a0 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2ae494640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a80c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2ae494640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a80c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55118/0 (socket says 172.21.15.40:55118) 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2ae494640 1 -- 172.21.15.40:0/1381516748 learned_addr learned my addr 172.21.15.40:0/1381516748 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2aec95640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a815f3a0 0x7ff2a80ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2adc93640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a80d06f0 0x7ff2a81532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2ae494640 1 -- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a815f3a0 msgr2=0x7ff2a80ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2ae494640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a815f3a0 0x7ff2a80ba390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2ae494640 1 -- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a80d06f0 msgr2=0x7ff2a81532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2ae494640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a80d06f0 0x7ff2a81532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.425+0000 7ff2ae494640 1 -- 172.21.15.40:0/1381516748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a80b9640 con 0x7ff2a80b3860 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.426+0000 7ff2ae494640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a80c8ac0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c3c0 tx=0x7ff2a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.426+0000 7ff29effd640 1 -- 172.21.15.40:0/1381516748 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0017070 con 0x7ff2a80b3860 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.426+0000 7ff29effd640 1 -- 172.21.15.40:0/1381516748 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a0005ce0 con 0x7ff2a80b3860 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.426+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a80b98b0 con 0x7ff2a80b3860 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.426+0000 7ff29effd640 1 -- 172.21.15.40:0/1381516748 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00072f0 con 0x7ff2a80b3860 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.426+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a80b9e70 con 0x7ff2a80b3860 2024-04-26T21:44:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.427+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a80694f0 con 0x7ff2a80b3860 2024-04-26T21:44:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.427+0000 7ff29effd640 1 -- 172.21.15.40:0/1381516748 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2a001a070 con 0x7ff2a80b3860 2024-04-26T21:44:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.428+0000 7ff29effd640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff28c06c920 0x7ff28c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.428+0000 7ff29effd640 1 -- 172.21.15.40:0/1381516748 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7507+0+0 (secure 0 0 0) 0x7ff2a008b500 con 0x7ff2a80b3860 2024-04-26T21:44:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.428+0000 7ff2adc93640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff28c06c920 0x7ff28c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.428+0000 7ff2adc93640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff28c06c920 0x7ff28c06ede0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8159cf0 tx=0x7ff298009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.432+0000 7ff29effd640 1 -- 172.21.15.40:0/1381516748 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2a80d0ba0 con 0x7ff2a80b3860 2024-04-26T21:44:11.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.660+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2a80ba0b0 con 0x7ff2a80b3860 2024-04-26T21:44:11.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.661+0000 7ff29effd640 1 -- 172.21.15.40:0/1381516748 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) ==== 95+0+5181 (secure 0 0 0) 0x7ff2a80ba0b0 con 0x7ff2a80b3860 2024-04-26T21:44:11.662 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:11.662 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":184,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9513,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2324764539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2324764539},{"type":"v1","addr":"172.21.15.40:6833","nonce":2324764539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:08.264886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 184 2024-04-26T21:44:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff28c06c920 msgr2=0x7ff28c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff28c06c920 0x7ff28c06ede0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8159cf0 tx=0x7ff298009040 comp rx=0 tx=0).stop 2024-04-26T21:44:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 msgr2=0x7ff2a80c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a80c8ac0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c3c0 tx=0x7ff2a000c880 comp rx=0 tx=0).stop 2024-04-26T21:44:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 shutdown_connections 2024-04-26T21:44:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff28c06c920 0x7ff28c06ede0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a815f3a0 0x7ff2a80ba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a80d06f0 0x7ff2a81532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 --2- 172.21.15.40:0/1381516748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a80b3860 0x7ff2a80c8ac0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 >> 172.21.15.40:0/1381516748 conn(0x7ff2a80d4f90 msgr2=0x7ff2a80d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 shutdown_connections 2024-04-26T21:44:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.663+0000 7ff2b06f8640 1 -- 172.21.15.40:0/1381516748 wait complete. 2024-04-26T21:44:11.675 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000' led to DAMAGED state 2024-04-26T21:44:11.675 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 1.00000000.inode' 2024-04-26T21:44:11.675 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:11.675 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:11.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:11.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:11.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:11.691 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:11.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:11.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:11.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.889+0000 7f64190da640 1 Processor -- start 2024-04-26T21:44:11.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.889+0000 7f64190da640 1 -- start start 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.889+0000 7f64190da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6414162810 0x7f6414070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.889+0000 7f64190da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64140b2340 0x7f64140cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.889+0000 7f64190da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64140d10d0 con 0x7f64140cd930 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.889+0000 7f64190da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f64140d0750 con 0x7f6414162810 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.889+0000 7f64190da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64140d16a0 con 0x7f64140b2340 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.889+0000 7f6412d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6414162810 0x7f6414070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.890+0000 7f6412d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6414162810 0x7f6414070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55122/0 (socket says 172.21.15.40:55122) 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.890+0000 7f6412d9d640 1 -- 172.21.15.40:0/1452970193 learned_addr learned my addr 172.21.15.40:0/1452970193 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.890+0000 7f641259c640 1 --2- 172.21.15.40:0/1452970193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64140b2340 0x7f64140cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.890+0000 7f6412d9d640 1 -- 172.21.15.40:0/1452970193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64140b2340 msgr2=0x7f64140cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.890+0000 7f6412d9d640 1 --2- 172.21.15.40:0/1452970193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64140b2340 0x7f64140cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.890+0000 7f6412d9d640 1 -- 172.21.15.40:0/1452970193 >> v1:172.21.15.184:6789/0 conn(0x7f64140cd930 legacy=0x7f64140d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.890+0000 7f6412d9d640 1 -- 172.21.15.40:0/1452970193 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64140d1370 con 0x7f6414162810 2024-04-26T21:44:11.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f6412d9d640 1 --2- 172.21.15.40:0/1452970193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414162810 0x7f6414070350 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f640c009870 tx=0x7f640c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb9b84874fe29b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f6411d9b640 1 -- 172.21.15.40:0/1452970193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c037070 con 0x7f6414162810 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f6411d9b640 1 -- 172.21.15.40:0/1452970193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640c032440 con 0x7f6414162810 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f6411d9b640 1 -- 172.21.15.40:0/1452970193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c032810 con 0x7f6414162810 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f64190da640 1 -- 172.21.15.40:0/1452970193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414162810 msgr2=0x7f6414070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f64190da640 1 --2- 172.21.15.40:0/1452970193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414162810 0x7f6414070350 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f640c009870 tx=0x7f640c02a270 comp rx=0 tx=0).stop 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f64190da640 1 -- 172.21.15.40:0/1452970193 shutdown_connections 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f64190da640 1 --2- 172.21.15.40:0/1452970193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64140b2340 0x7f64140cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f64190da640 1 --2- 172.21.15.40:0/1452970193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414162810 0x7f6414070350 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f64190da640 1 -- 172.21.15.40:0/1452970193 >> 172.21.15.40:0/1452970193 conn(0x7f64140d2da0 msgr2=0x7f64140d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f64190da640 1 -- 172.21.15.40:0/1452970193 shutdown_connections 2024-04-26T21:44:11.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.891+0000 7f64190da640 1 -- 172.21.15.40:0/1452970193 wait complete. 2024-04-26T21:44:11.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.893+0000 7f64190da640 1 Processor -- start 2024-04-26T21:44:11.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.893+0000 7f64190da640 1 -- start start 2024-04-26T21:44:11.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f64190da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64140b2340 0x7f64141500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f64190da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140cd930 0x7f64140b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f64190da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414162810 0x7f64140c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f64190da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6414070890 con 0x7f64140b2340 2024-04-26T21:44:11.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f64190da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64140d0470 con 0x7f64140cd930 2024-04-26T21:44:11.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f64190da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64140d2240 con 0x7f6414162810 2024-04-26T21:44:11.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64140b2340 0x7f64141500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64140b2340 0x7f64141500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55138/0 (socket says 172.21.15.40:55138) 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 -- 172.21.15.40:0/3290605001 learned_addr learned my addr 172.21.15.40:0/3290605001 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f641259c640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140cd930 0x7f64140b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f641359e640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414162810 0x7f64140c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 -- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414162810 msgr2=0x7f64140c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414162810 0x7f64140c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 -- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140cd930 msgr2=0x7f64140b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140cd930 0x7f64140b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 -- 172.21.15.40:0/3290605001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64140c76a0 con 0x7f64140b2340 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f641259c640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140cd930 0x7f64140b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:11.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f6412d9d640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64140b2340 0x7f64141500e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.894+0000 7f63fb7fe640 1 -- 172.21.15.40:0/3290605001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c041020 con 0x7f64140b2340 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.895+0000 7f63fb7fe640 1 -- 172.21.15.40:0/3290605001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640c0306c0 con 0x7f64140b2340 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.895+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6414150620 con 0x7f64140b2340 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.895+0000 7f63fb7fe640 1 -- 172.21.15.40:0/3290605001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c037040 con 0x7f64140b2340 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.895+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64140b14c0 con 0x7f64140b2340 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.895+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64140694f0 con 0x7f64140b2340 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.896+0000 7f63fb7fe640 1 -- 172.21.15.40:0/3290605001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f640c044020 con 0x7f64140b2340 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.896+0000 7f63fb7fe640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f640006c8f0 0x7f640006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.897+0000 7f63fb7fe640 1 -- 172.21.15.40:0/3290605001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7507+0+0 (secure 0 0 0) 0x7f640c0aca50 con 0x7f64140b2340 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.897+0000 7f641259c640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f640006c8f0 0x7f640006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.897+0000 7f641259c640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f640006c8f0 0x7f640006edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6408006fd0 tx=0x7f6408008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:11.901+0000 7f63fb7fe640 1 -- 172.21.15.40:0/3290605001 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f640c04b030 con 0x7f64140b2340 2024-04-26T21:44:12.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.126+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f64140ce9d0 con 0x7f64140b2340 2024-04-26T21:44:12.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:44:12.254 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:44:12.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.283+0000 7f63fb7fe640 1 -- 172.21.15.40:0/3290605001 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v185) ==== 161+0+0 (secure 0 0 0) 0x7f640c07a780 con 0x7f64140b2340 2024-04-26T21:44:12.284 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:12.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f640006c8f0 msgr2=0x7f640006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:12.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f640006c8f0 0x7f640006edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6408006fd0 tx=0x7f6408008040 comp rx=0 tx=0).stop 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64140b2340 msgr2=0x7f64141500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64140b2340 0x7f64141500e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 shutdown_connections 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f640006c8f0 0x7f640006edb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414162810 0x7f64140c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140cd930 0x7f64140b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 --2- 172.21.15.40:0/3290605001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64140b2340 0x7f64141500e0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 >> 172.21.15.40:0/3290605001 conn(0x7f64140d2da0 msgr2=0x7f641406ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 shutdown_connections 2024-04-26T21:44:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.285+0000 7f64190da640 1 -- 172.21.15.40:0/3290605001 wait complete. 2024-04-26T21:44:12.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:12.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.445+0000 7f6134faf640 1 Processor -- start 2024-04-26T21:44:12.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.445+0000 7f6134faf640 1 -- start start 2024-04-26T21:44:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.446+0000 7f6134faf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6130162880 0x7f61300d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.446+0000 7f6134faf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61300b5f20 0x7f61300d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.446+0000 7f6134faf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61300b7e50 con 0x7f61300d39f0 2024-04-26T21:44:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.446+0000 7f6134faf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f61300d3e40 con 0x7f6130162880 2024-04-26T21:44:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.446+0000 7f6134faf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f61300d0580 con 0x7f61300b5f20 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.446+0000 7f612e59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6130162880 0x7f61300d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.446+0000 7f612e59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6130162880 0x7f61300d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55150/0 (socket says 172.21.15.40:55150) 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.446+0000 7f612e59c640 1 -- 172.21.15.40:0/1628097614 learned_addr learned my addr 172.21.15.40:0/1628097614 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.447+0000 7f612e59c640 1 -- 172.21.15.40:0/1628097614 >> v2:172.21.15.184:3300/0 conn(0x7f61300b5f20 msgr2=0x7f61300d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.447+0000 7f612e59c640 1 --2- 172.21.15.40:0/1628097614 >> v2:172.21.15.184:3300/0 conn(0x7f61300b5f20 0x7f61300d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.447+0000 7f612e59c640 1 -- 172.21.15.40:0/1628097614 >> v1:172.21.15.184:6789/0 conn(0x7f61300d39f0 legacy=0x7f61300b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.447+0000 7f612e59c640 1 -- 172.21.15.40:0/1628097614 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61300d1800 con 0x7f6130162880 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.447+0000 7f612e59c640 1 --2- 172.21.15.40:0/1628097614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f61300d2cb0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f613008d3d0 tx=0x7f612802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e300c2250198a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.447+0000 7f612d59a640 1 -- 172.21.15.40:0/1628097614 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128037070 con 0x7f6130162880 2024-04-26T21:44:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.447+0000 7f612d59a640 1 -- 172.21.15.40:0/1628097614 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6128032440 con 0x7f6130162880 2024-04-26T21:44:12.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.447+0000 7f612d59a640 1 -- 172.21.15.40:0/1628097614 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128032810 con 0x7f6130162880 2024-04-26T21:44:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.448+0000 7f6134faf640 1 -- 172.21.15.40:0/1628097614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 msgr2=0x7f61300d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.448+0000 7f6134faf640 1 --2- 172.21.15.40:0/1628097614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f61300d2cb0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f613008d3d0 tx=0x7f612802a230 comp rx=0 tx=0).stop 2024-04-26T21:44:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.448+0000 7f6134faf640 1 -- 172.21.15.40:0/1628097614 shutdown_connections 2024-04-26T21:44:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.448+0000 7f6134faf640 1 --2- 172.21.15.40:0/1628097614 >> v2:172.21.15.184:3300/0 conn(0x7f61300b5f20 0x7f61300d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.448+0000 7f6134faf640 1 --2- 172.21.15.40:0/1628097614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f61300d2cb0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.448+0000 7f6134faf640 1 -- 172.21.15.40:0/1628097614 >> 172.21.15.40:0/1628097614 conn(0x7f61300d4f60 msgr2=0x7f61300d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.448+0000 7f6134faf640 1 -- 172.21.15.40:0/1628097614 shutdown_connections 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.448+0000 7f6134faf640 1 -- 172.21.15.40:0/1628097614 wait complete. 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.449+0000 7f6134faf640 1 Processor -- start 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.449+0000 7f6134faf640 1 -- start start 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f6134faf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300b5f20 0x7f61300b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f6134faf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61300d39f0 0x7f61300cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f6134faf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f613014f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f6134faf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61300d0d90 con 0x7f6130162880 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f6134faf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61300d0ab0 con 0x7f61300b5f20 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f6134faf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6130162c60 con 0x7f61300d39f0 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f612ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f613014f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f612ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f613014f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55158/0 (socket says 172.21.15.40:55158) 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f612ed9d640 1 -- 172.21.15.40:0/403049691 learned_addr learned my addr 172.21.15.40:0/403049691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f612e59c640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300b5f20 0x7f61300b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.450+0000 7f612dd9b640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61300d39f0 0x7f61300cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f612ed9d640 1 -- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61300d39f0 msgr2=0x7f61300cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f612ed9d640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61300d39f0 0x7f61300cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f612ed9d640 1 -- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300b5f20 msgr2=0x7f61300b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f612ed9d640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300b5f20 0x7f61300b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f612ed9d640 1 -- 172.21.15.40:0/403049691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613014f8e0 con 0x7f6130162880 2024-04-26T21:44:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f612dd9b640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61300d39f0 0x7f61300cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f612ed9d640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f613014f3a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f612000b9c0 tx=0x7f612000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f6126ffd640 1 -- 172.21.15.40:0/403049691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120019070 con 0x7f6130162880 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f6126ffd640 1 -- 172.21.15.40:0/403049691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612000cba0 con 0x7f6130162880 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61300ce150 con 0x7f6130162880 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f6126ffd640 1 -- 172.21.15.40:0/403049691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120014440 con 0x7f6130162880 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.451+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61300ce710 con 0x7f6130162880 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.452+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61300694f0 con 0x7f6130162880 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.452+0000 7f6126ffd640 1 -- 172.21.15.40:0/403049691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f612001c070 con 0x7f6130162880 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.453+0000 7f6126ffd640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f611006c8f0 0x7f611006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.453+0000 7f6126ffd640 1 -- 172.21.15.40:0/403049691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7507+0+0 (secure 0 0 0) 0x7f612008bea0 con 0x7f6130162880 2024-04-26T21:44:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.453+0000 7f612e59c640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f611006c8f0 0x7f611006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.454+0000 7f612e59c640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f611006c8f0 0x7f611006edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6128034000 tx=0x7f612802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.457+0000 7f6126ffd640 1 -- 172.21.15.40:0/403049691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f613014f8e0 con 0x7f6130162880 2024-04-26T21:44:12.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:12.712+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f6130162080 con 0x7f6130162880 2024-04-26T21:44:13.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.290+0000 7f6126ffd640 1 -- 172.21.15.40:0/403049691 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v186) ==== 101+0+0 (secure 0 0 0) 0x7f6120059600 con 0x7f6130162880 2024-04-26T21:44:13.291 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f611006c8f0 msgr2=0x7f611006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f611006c8f0 0x7f611006edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6128034000 tx=0x7f612802e000 comp rx=0 tx=0).stop 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 msgr2=0x7f613014f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f613014f3a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f612000b9c0 tx=0x7f612000be80 comp rx=0 tx=0).stop 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 shutdown_connections 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f611006c8f0 0x7f611006edb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6130162880 0x7f613014f3a0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61300d39f0 0x7f61300cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 --2- 172.21.15.40:0/403049691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300b5f20 0x7f61300b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 >> 172.21.15.40:0/403049691 conn(0x7f61300d4f60 msgr2=0x7f6130160ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:13.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 shutdown_connections 2024-04-26T21:44:13.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.293+0000 7f6134faf640 1 -- 172.21.15.40:0/403049691 wait complete. 2024-04-26T21:44:13.306 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.350+0000 7f7b7980ff40 1 Processor -- start 2024-04-26T21:44:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.350+0000 7f7b7980ff40 1 -- start start 2024-04-26T21:44:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b7980ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec008220 0x55c0ec0086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b7980ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec038af0 0x55c0ec0e0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b7980ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 0x55c0ec0e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b7980ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c0ec0519f0 con 0x55c0ec008220 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b7980ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0ebfb1e40 con 0x55c0ec038af0 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b7980ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c0ec0540b0 con 0x55c0ec0e1900 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76641640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec008220 0x55c0ec0086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76e42640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 0x55c0ec0e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76641640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec008220 0x55c0ec0086b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55170/0 (socket says 172.21.15.40:55170) 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76641640 1 -- 172.21.15.40:0/2474344801 learned_addr learned my addr 172.21.15.40:0/2474344801 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec038af0 0x55c0ec0e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76641640 1 -- 172.21.15.40:0/2474344801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 msgr2=0x55c0ec0e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76641640 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 0x55c0ec0e4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76641640 1 -- 172.21.15.40:0/2474344801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec038af0 msgr2=0x55c0ec0e0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76641640 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec038af0 0x55c0ec0e0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b76641640 1 -- 172.21.15.40:0/2474344801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ec036690 con 0x55c0ec008220 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.351+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec038af0 0x55c0ec0e0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b76641640 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec008220 0x55c0ec0086b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7b680099d0 tx=0x7f7b68005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825523efbe0242f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b74e3e640 1 -- 172.21.15.40:0/2474344801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68008a30 con 0x55c0ec008220 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b74e3e640 1 -- 172.21.15.40:0/2474344801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b68018070 con 0x55c0ec008220 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b74e3e640 1 -- 172.21.15.40:0/2474344801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68008dd0 con 0x55c0ec008220 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2474344801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec008220 msgr2=0x55c0ec0086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec008220 0x55c0ec0086b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7b680099d0 tx=0x7f7b68005a00 comp rx=0 tx=0).stop 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2474344801 shutdown_connections 2024-04-26T21:44:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 0x55c0ec0e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec038af0 0x55c0ec0e0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2474344801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec008220 0x55c0ec0086b0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2474344801 >> 172.21.15.40:0/2474344801 conn(0x55c0ec009330 msgr2=0x55c0ec0df990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2474344801 shutdown_connections 2024-04-26T21:44:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.352+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2474344801 wait complete. 2024-04-26T21:44:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.353+0000 7f7b7980ff40 1 Processor -- start 2024-04-26T21:44:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.353+0000 7f7b7980ff40 1 -- start start 2024-04-26T21:44:13.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b7980ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec008220 0x55c0ec16ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b7980ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec038af0 0x55c0ec0d8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b7980ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 0x55c0ec0d9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b7980ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c0ec0529c0 con 0x55c0ec038af0 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b7980ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0ec0360d0 con 0x55c0ec008220 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b7980ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c0ec170570 con 0x55c0ec0e1900 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec038af0 0x55c0ec0d8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec038af0 0x55c0ec0d8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55178/0 (socket says 172.21.15.40:55178) 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 learned_addr learned my addr 172.21.15.40:0/2726341807 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b76641640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec008220 0x55c0ec16ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b76e42640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 0x55c0ec0d9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 msgr2=0x55c0ec0d9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 0x55c0ec0d9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec008220 msgr2=0x55c0ec16ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0ec008220 0x55c0ec16ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ec0e3bc0 con 0x55c0ec038af0 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b76e42640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0ec0e1900 0x55c0ec0d9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec038af0 0x55c0ec0d8870 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00a4f0 tx=0x7f7b6c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.354+0000 7f7b677fe640 1 -- 172.21.15.40:0/2726341807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c064030 con 0x55c0ec038af0 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.355+0000 7f7b677fe640 1 -- 172.21.15.40:0/2726341807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c00a020 con 0x55c0ec038af0 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.355+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c0ec0e3600 con 0x55c0ec038af0 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.355+0000 7f7b677fe640 1 -- 172.21.15.40:0/2726341807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c059040 con 0x55c0ec038af0 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.355+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ec0d8db0 con 0x55c0ec038af0 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.356+0000 7f7b677fe640 1 -- 172.21.15.40:0/2726341807 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b6c054a60 con 0x55c0ec038af0 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.357+0000 7f7b677fe640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b5806b300 0x7f7b5806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.357+0000 7f7b677fe640 1 -- 172.21.15.40:0/2726341807 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7507+0+0 (secure 0 0 0) 0x7f7b6c0cec10 con 0x55c0ec038af0 2024-04-26T21:44:13.361 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.357+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0ec0d9cb0 0x55c0ec0dd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.357+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec00b570 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.357+0000 7f7b76641640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b5806b300 0x7f7b5806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.357+0000 7f7b76e42640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0ec0d9cb0 0x55c0ec0dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.357+0000 7f7b76e42640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0ec0d9cb0 0x55c0ec0dd550 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.358+0000 7f7b76641640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b5806b300 0x7f7b5806d7c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7b68000c00 tx=0x7f7b68012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.359+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v110'443 uv417 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.359+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec00b940 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.360+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v110'444 uv444 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.360+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec00bd10 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.363 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.361+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v110'445 uv445 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.361+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185560 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.362+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v110'446 uv446 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.362+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185900 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.363+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v110'447 uv447 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.363+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec184a00 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.364+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v110'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.364+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec184df0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.366+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v110'449 uv423 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.366+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.367+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v110'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.367+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.368 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.368+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v110'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.368+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.369+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v110'452 uv452 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.369+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.370+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v110'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.370+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.371+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v110'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.371+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.372+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v110'455 uv430 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.372+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.373+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v110'456 uv456 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.373+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.375 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.374+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v110'457 uv457 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.374+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.376+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v110'458 uv458 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.376+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.377+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v110'459 uv459 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.377+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185190 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.378+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v110'460 uv460 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.378+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.379+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v110'461 uv461 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.379+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.381+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v110'462 uv436 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.381+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.382+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v110'463 uv463 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.382+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.383 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.383+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v110'464 uv464 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.383+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec186cb0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.384+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v110'465 uv465 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.384+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec186cb0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.385+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v110'466 uv466 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.385+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.386+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v110'467 uv467 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.386+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.387+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v110'468 uv442 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.387+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.388+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v110'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.388+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.390 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:13.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.389+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v110'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.389+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec186cb0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.390+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v110'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.390+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec186cb0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.391+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v110'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.391+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec185ca0 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.392+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v110'473 uv473 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec0d9cb0 2024-04-26T21:44:13.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.392+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0ec186320 0x55c0ec188710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.392+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec188c50 con 0x55c0ec186320 2024-04-26T21:44:13.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.392+0000 7f7b76641640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0ec186320 0x55c0ec188710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.393+0000 7f7b76641640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0ec186320 0x55c0ec188710 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.394+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v110'330 uv312 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.394+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec188ff0 con 0x55c0ec186320 2024-04-26T21:44:13.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.396+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v110'331 uv331 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.396+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec189390 con 0x55c0ec186320 2024-04-26T21:44:13.398 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:13.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.397+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v110'332 uv332 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.397+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec189be0 con 0x55c0ec186320 2024-04-26T21:44:13.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.399+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v110'333 uv333 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.399+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec189f80 con 0x55c0ec186320 2024-04-26T21:44:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.400+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v110'334 uv334 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.400+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18a320 con 0x55c0ec186320 2024-04-26T21:44:13.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.401+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v110'335 uv317 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.401+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec18a6c0 con 0x55c0ec186320 2024-04-26T21:44:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.402+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v110'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.402+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.404 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.404+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v110'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.404+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.405+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v110'338 uv338 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.405+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.406+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v110'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.406+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.407+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v110'340 uv322 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.407+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.408+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v110'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.408+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.410+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v110'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.410 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.410+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18f120 con 0x55c0ec186320 2024-04-26T21:44:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.411+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v110'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.411+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18f120 con 0x55c0ec186320 2024-04-26T21:44:13.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.412+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v110'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.412+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.413+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v110'345 uv328 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.413+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.414+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v110'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.415+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.416 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:13.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.416+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v110'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.416+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18ba70 con 0x55c0ec186320 2024-04-26T21:44:13.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.417+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v110'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.417+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18ba70 con 0x55c0ec186320 2024-04-26T21:44:13.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.418+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v110'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.418+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18aa60 con 0x55c0ec186320 2024-04-26T21:44:13.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.419+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v110'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec186320 2024-04-26T21:44:13.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.419+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0ec18b360 0x55c0ec18d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.419+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18dc90 con 0x55c0ec18b360 2024-04-26T21:44:13.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.419+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0ec18b360 0x55c0ec18d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.420+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0ec18b360 0x55c0ec18d750 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.421+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v110'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.421+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec18e030 con 0x55c0ec18b360 2024-04-26T21:44:13.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.422+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v110'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.422+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18e510 con 0x55c0ec18b360 2024-04-26T21:44:13.423 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:13.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.423+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v110'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.423+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18fdc0 con 0x55c0ec18b360 2024-04-26T21:44:13.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.424+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v110'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.424+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec190160 con 0x55c0ec18b360 2024-04-26T21:44:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.425+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v110'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.425+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec18edb0 con 0x55c0ec18b360 2024-04-26T21:44:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.426+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v110'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.426+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0ec18f6d0 0x55c0ec192510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.426+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec192a50 con 0x55c0ec18f6d0 2024-04-26T21:44:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.426+0000 7f7b76e42640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0ec18f6d0 0x55c0ec192510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.427+0000 7f7b76e42640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0ec18f6d0 0x55c0ec192510 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.428+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v110'72 uv71 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.428+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec192a50 con 0x55c0ec18f6d0 2024-04-26T21:44:13.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.429+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v110'73 uv73 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.429+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec192ad0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.431 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:13.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.431+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v110'74 uv74 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.431+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec192ec0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.432+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v110'75 uv75 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.432+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0ec1933d0 0x55c0ec1957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.432+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec195d00 con 0x55c0ec1933d0 2024-04-26T21:44:13.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.433+0000 7f7b76641640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0ec1933d0 0x55c0ec1957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.433+0000 7f7b76641640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0ec1933d0 0x55c0ec1957c0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.434+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v110'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.434+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec1933d0 2024-04-26T21:44:13.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.436+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v110'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.436+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec1933d0 2024-04-26T21:44:13.438 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:13.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.438+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v110'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.438+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1970e0 con 0x55c0ec1933d0 2024-04-26T21:44:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.439+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v110'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.439+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1970e0 con 0x55c0ec1933d0 2024-04-26T21:44:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.441+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v110'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.441+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec1933d0 2024-04-26T21:44:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.441+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v110'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.441+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.442+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v110'160 uv154 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.442+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.444+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v110'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.444+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.445 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.445+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v110'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.445+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1970e0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.446+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v110'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.446+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1970e0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.447+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v110'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.447+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.448+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v110'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.448+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.449+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v110'166 uv159 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.449+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.450+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v110'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.450+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.452 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:13.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.451+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v110'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.451+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec196110 con 0x55c0ec18f6d0 2024-04-26T21:44:13.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.452+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v110'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.452+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.453+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v110'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.454+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.455+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v110'71 uv70 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.455+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.456+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v110'72 uv72 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.456+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.458 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:13.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.457+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v110'73 uv73 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.457+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.458+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v110'74 uv74 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.458+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec18f6d0 2024-04-26T21:44:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.459+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v110'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b70003530 con 0x55c0ec18f6d0 2024-04-26T21:44:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.459+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec1933d0 2024-04-26T21:44:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.460+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v110'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.460+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec1933d0 2024-04-26T21:44:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.461+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v110'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.461+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1960d0 con 0x55c0ec1933d0 2024-04-26T21:44:13.463 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:13.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.462+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v110'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.462+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec197500 con 0x55c0ec1933d0 2024-04-26T21:44:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.463+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v110'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.463+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec197500 con 0x55c0ec1933d0 2024-04-26T21:44:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.464+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v110'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.464+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.465+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v110'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.465+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.466+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v110'142 uv136 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.466+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.467+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v110'143 uv143 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.467+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.469 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:13.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.468+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v110'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.468+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.469+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v110'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.470+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.470+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v110'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.470+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.472+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v110'147 uv141 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.472+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.473+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v110'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.473+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.475 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.473+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v110'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.474+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1964f0 con 0x55c0ec1933d0 2024-04-26T21:44:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.474+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v110'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.475+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec199300 con 0x55c0ec1933d0 2024-04-26T21:44:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.476+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v110'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.476+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0ec196eb0 0x55c0ec199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.476+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec196eb0 2024-04-26T21:44:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.476+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0ec196eb0 0x55c0ec199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.476+0000 7f7b75e40640 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0ec196eb0 0x55c0ec199720 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.477+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v110'66 uv64 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec196eb0 2024-04-26T21:44:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.477+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec196eb0 2024-04-26T21:44:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.478+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v110'67 uv67 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec196eb0 2024-04-26T21:44:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.478+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec196eb0 2024-04-26T21:44:13.481 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.481+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v110'68 uv68 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec196eb0 2024-04-26T21:44:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.481+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec196eb0 2024-04-26T21:44:13.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.482+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v110'69 uv69 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec196eb0 2024-04-26T21:44:13.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.482+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec1933d0 2024-04-26T21:44:13.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.483+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v110'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.483+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec1933d0 2024-04-26T21:44:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.484+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v110'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.484+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec1933d0 2024-04-26T21:44:13.486 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.485+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v110'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.486+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec1933d0 2024-04-26T21:44:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.486+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v110'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.486+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec1933d0 2024-04-26T21:44:13.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.487+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v110'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.488+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec1933d0 2024-04-26T21:44:13.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.489+0000 7f7b76641640 1 -- 172.21.15.40:0/2726341807 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v110'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b68026070 con 0x55c0ec1933d0 2024-04-26T21:44:13.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.489+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec18b360 2024-04-26T21:44:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.490+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v110'74 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.490+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec18b360 2024-04-26T21:44:13.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.491+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v110'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.491+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec18b360 2024-04-26T21:44:13.492 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:13.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.492+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v110'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.492+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec18b360 2024-04-26T21:44:13.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.493+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v110'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.493+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec18b360 2024-04-26T21:44:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.494+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v110'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.494+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec18b360 2024-04-26T21:44:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.495+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v110'182 uv172 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.495+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec18b360 2024-04-26T21:44:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.496+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v110'183 uv183 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.496+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec1992a0 con 0x55c0ec18b360 2024-04-26T21:44:13.498 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.497+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v110'184 uv184 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.497+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec199ce0 con 0x55c0ec18b360 2024-04-26T21:44:13.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.498+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v110'185 uv185 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.498+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec199c60 con 0x55c0ec18b360 2024-04-26T21:44:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.499+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v110'186 uv186 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.499+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec199c60 con 0x55c0ec18b360 2024-04-26T21:44:13.501 INFO:teuthology.orchestra.run.smithi040.stdout:Write #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.500+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v110'187 uv187 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.500+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec19ac70 con 0x55c0ec18b360 2024-04-26T21:44:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.501+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v110'188 uv188 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.501+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec19ac70 con 0x55c0ec18b360 2024-04-26T21:44:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.502+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v110'189 uv189 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.502+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec199c60 con 0x55c0ec18b360 2024-04-26T21:44:13.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.503+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [omap-set-header] v110'190 uv190 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.503+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55c0ec199c60 con 0x55c0ec18b360 2024-04-26T21:44:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b75e40640 1 -- 172.21.15.40:0/2726341807 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v110'191 uv191 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7b6c070070 con 0x55c0ec18b360 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0ec186320 msgr2=0x55c0ec188710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0ec186320 0x55c0ec188710 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0ec0d9cb0 msgr2=0x55c0ec0dd550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0ec0d9cb0 0x55c0ec0dd550 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0ec18f6d0 msgr2=0x55c0ec192510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0ec18f6d0 0x55c0ec192510 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0ec196eb0 msgr2=0x55c0ec199720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0ec196eb0 0x55c0ec199720 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0ec18b360 msgr2=0x55c0ec18d750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0ec18b360 0x55c0ec18d750 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0ec1933d0 msgr2=0x55c0ec1957c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0ec1933d0 0x55c0ec1957c0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 reap_dead start 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 reap_dead start 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b76e42640 1 -- 172.21.15.40:0/2726341807 reap_dead start 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b5806b300 msgr2=0x7f7b5806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b5806b300 0x7f7b5806d7c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7b68000c00 tx=0x7f7b68012000 comp rx=0 tx=0).stop 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec038af0 msgr2=0x55c0ec0d8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec038af0 0x55c0ec0d8870 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00a4f0 tx=0x7f7b6c002fb0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 shutdown_connections 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7b5806b300 0x7f7b5806d7c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 --2- 172.21.15.40:0/2726341807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0ec038af0 0x55c0ec0d8870 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 >> 172.21.15.40:0/2726341807 conn(0x55c0ec009330 msgr2=0x55c0ec0e0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 shutdown_connections 2024-04-26T21:44:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.504+0000 7f7b7980ff40 1 -- 172.21.15.40:0/2726341807 wait complete. 2024-04-26T21:44:13.508 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 1.00000000.inode - 2024-04-26T21:44:13.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.590+0000 7f394b314f40 1 Processor -- start 2024-04-26T21:44:13.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.590+0000 7f394b314f40 1 -- start start 2024-04-26T21:44:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394b314f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56463c319930 0x56463c3e5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394b314f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56463c31c850 0x56463c3ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394b314f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c3f0830 0x56463c3ea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394b314f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56463c3f0c10 con 0x56463c3f0830 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394b314f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56463c2c2e40 con 0x56463c31c850 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394b314f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56463c3f10f0 con 0x56463c319930 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c3f0830 0x56463c3ea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c3f0830 0x56463c3ea870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55196/0 (socket says 172.21.15.40:55196) 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 -- 172.21.15.40:0/1798084829 learned_addr learned my addr 172.21.15.40:0/1798084829 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 -- 172.21.15.40:0/1798084829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56463c319930 msgr2=0x56463c3e5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 --2- 172.21.15.40:0/1798084829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56463c319930 0x56463c3e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 -- 172.21.15.40:0/1798084829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56463c31c850 msgr2=0x56463c3ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 --2- 172.21.15.40:0/1798084829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56463c31c850 0x56463c3ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 -- 172.21.15.40:0/1798084829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463c3ed750 con 0x56463c3f0830 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.591+0000 7f394960f640 1 --2- 172.21.15.40:0/1798084829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c3f0830 0x56463c3ea870 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f394400b600 tx=0x7f394400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f68b9b9ab6aa97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f3942ffd640 1 -- 172.21.15.40:0/1798084829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944014030 con 0x56463c3f0830 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f3942ffd640 1 -- 172.21.15.40:0/1798084829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3944010620 con 0x56463c3f0830 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f3942ffd640 1 -- 172.21.15.40:0/1798084829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39440109f0 con 0x56463c3f0830 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 -- 172.21.15.40:0/1798084829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c3f0830 msgr2=0x56463c3ea870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 --2- 172.21.15.40:0/1798084829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c3f0830 0x56463c3ea870 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f394400b600 tx=0x7f394400bac0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 -- 172.21.15.40:0/1798084829 shutdown_connections 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 --2- 172.21.15.40:0/1798084829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c3f0830 0x56463c3ea870 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 --2- 172.21.15.40:0/1798084829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56463c31c850 0x56463c3ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 --2- 172.21.15.40:0/1798084829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56463c319930 0x56463c3e5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 -- 172.21.15.40:0/1798084829 >> 172.21.15.40:0/1798084829 conn(0x56463c31a330 msgr2=0x56463c3eb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 -- 172.21.15.40:0/1798084829 shutdown_connections 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.592+0000 7f394b314f40 1 -- 172.21.15.40:0/1798084829 wait complete. 2024-04-26T21:44:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.593+0000 7f394b314f40 1 Processor -- start 2024-04-26T21:44:13.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.593+0000 7f394b314f40 1 -- start start 2024-04-26T21:44:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.593+0000 7f394b314f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56463c319930 0x56463c3e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.593+0000 7f394b314f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c31c850 0x56463c346650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.593+0000 7f394b314f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56463c3f0830 0x56463c3eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.593+0000 7f394b314f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56463c3e74d0 con 0x56463c31c850 2024-04-26T21:44:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.593+0000 7f394b314f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56463c3e7770 con 0x56463c319930 2024-04-26T21:44:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.593+0000 7f394b314f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56463c3e7000 con 0x56463c3f0830 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c31c850 0x56463c346650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c31c850 0x56463c346650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55202/0 (socket says 172.21.15.40:55202) 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 -- 172.21.15.40:0/3806456896 learned_addr learned my addr 172.21.15.40:0/3806456896 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3948e0e640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56463c319930 0x56463c3e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f394960f640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56463c3f0830 0x56463c3eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 -- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56463c3f0830 msgr2=0x56463c3eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56463c3f0830 0x56463c3eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 -- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56463c319930 msgr2=0x56463c3e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56463c319930 0x56463c3e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 -- 172.21.15.40:0/3806456896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463c3e8d00 con 0x56463c31c850 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3943fff640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c31c850 0x56463c346650 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3934025cc0 tx=0x7f3934055540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3941ffb640 1 -- 172.21.15.40:0/3806456896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934063070 con 0x56463c31c850 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3941ffb640 1 -- 172.21.15.40:0/3806456896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3934055970 con 0x56463c31c850 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56463c3e4870 con 0x56463c31c850 2024-04-26T21:44:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f3941ffb640 1 -- 172.21.15.40:0/3806456896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934055d40 con 0x56463c31c850 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.594+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463c3e83d0 con 0x56463c31c850 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.596+0000 7f3941ffb640 1 -- 172.21.15.40:0/3806456896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f393405c040 con 0x56463c31c850 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.596+0000 7f3941ffb640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f392c06b2d0 0x7f392c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.596+0000 7f3941ffb640 1 -- 172.21.15.40:0/3806456896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7507+0+0 (secure 0 0 0) 0x7f39340cf2a0 con 0x56463c31c850 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.596+0000 7f3948e0e640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f392c06b2d0 0x7f392c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.596+0000 7f394b314f40 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x56463c3e4140 0x56463c3ea670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.596+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x56463c346b90 con 0x56463c3e4140 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.597+0000 7f394960f640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x56463c3e4140 0x56463c3ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.597+0000 7f3948e0e640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f392c06b2d0 0x7f392c06d790 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f393c000f60 tx=0x7f393c0066f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.597+0000 7f394960f640 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x56463c3e4140 0x56463c3ea670 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394960f640 1 -- 172.21.15.40:0/3806456896 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 1.00000000.inode [writefull 0~80] v110'351 uv351 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3944010a50 con 0x56463c3e4140 2024-04-26T21:44:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x56463c3e4140 msgr2=0x56463c3ea670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x56463c3e4140 0x56463c3ea670 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f392c06b2d0 msgr2=0x7f392c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f392c06b2d0 0x7f392c06d790 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f393c000f60 tx=0x7f393c0066f0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c31c850 msgr2=0x56463c346650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 --2- 172.21.15.40:0/3806456896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56463c31c850 0x56463c346650 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3934025cc0 tx=0x7f3934055540 comp rx=0 tx=0).stop 2024-04-26T21:44:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394960f640 1 -- 172.21.15.40:0/3806456896 reap_dead start 2024-04-26T21:44:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 shutdown_connections 2024-04-26T21:44:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 >> 172.21.15.40:0/3806456896 conn(0x56463c31a330 msgr2=0x56463c3eb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 shutdown_connections 2024-04-26T21:44:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.598+0000 7f394b314f40 1 -- 172.21.15.40:0/3806456896 wait complete. 2024-04-26T21:44:13.603 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:13.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.745+0000 7fce536c4640 1 Processor -- start 2024-04-26T21:44:13.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.745+0000 7fce536c4640 1 -- start start 2024-04-26T21:44:13.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.746+0000 7fce536c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fce4c0d1a00 0x7fce4c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.746+0000 7fce536c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce4c0d2320 0x7fce4c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.746+0000 7fce536c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce4c0d0a10 con 0x7fce4c0b40d0 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.746+0000 7fce536c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fce4c0d0cf0 con 0x7fce4c0d2320 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.746+0000 7fce536c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fce4c0d0fd0 con 0x7fce4c0d1a00 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.746+0000 7fce50c5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce4c0d2320 0x7fce4c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.746+0000 7fce50c5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce4c0d2320 0x7fce4c1625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55208/0 (socket says 172.21.15.40:55208) 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.746+0000 7fce50c5f640 1 -- 172.21.15.40:0/2767842016 learned_addr learned my addr 172.21.15.40:0/2767842016 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.747+0000 7fce51460640 1 --2- 172.21.15.40:0/2767842016 >> v2:172.21.15.184:3300/0 conn(0x7fce4c0d1a00 0x7fce4c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.747+0000 7fce50c5f640 1 -- 172.21.15.40:0/2767842016 >> v2:172.21.15.184:3300/0 conn(0x7fce4c0d1a00 msgr2=0x7fce4c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.747+0000 7fce50c5f640 1 --2- 172.21.15.40:0/2767842016 >> v2:172.21.15.184:3300/0 conn(0x7fce4c0d1a00 0x7fce4c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.747+0000 7fce50c5f640 1 -- 172.21.15.40:0/2767842016 >> v1:172.21.15.184:6789/0 conn(0x7fce4c0b40d0 legacy=0x7fce4c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.747+0000 7fce50c5f640 1 -- 172.21.15.40:0/2767842016 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c0d40a0 con 0x7fce4c0d2320 2024-04-26T21:44:13.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.747+0000 7fce50c5f640 1 --2- 172.21.15.40:0/2767842016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c1625f0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fce4c06c650 tx=0x7fce4002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad639530142ba3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce4bfff640 1 -- 172.21.15.40:0/2767842016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40037070 con 0x7fce4c0d2320 2024-04-26T21:44:13.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce4bfff640 1 -- 172.21.15.40:0/2767842016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce40032440 con 0x7fce4c0d2320 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce4bfff640 1 -- 172.21.15.40:0/2767842016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40032810 con 0x7fce4c0d2320 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce536c4640 1 -- 172.21.15.40:0/2767842016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 msgr2=0x7fce4c1625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce536c4640 1 --2- 172.21.15.40:0/2767842016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c1625f0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fce4c06c650 tx=0x7fce4002a150 comp rx=0 tx=0).stop 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce536c4640 1 -- 172.21.15.40:0/2767842016 shutdown_connections 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce536c4640 1 --2- 172.21.15.40:0/2767842016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c1625f0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce536c4640 1 --2- 172.21.15.40:0/2767842016 >> v2:172.21.15.184:3300/0 conn(0x7fce4c0d1a00 0x7fce4c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce536c4640 1 -- 172.21.15.40:0/2767842016 >> 172.21.15.40:0/2767842016 conn(0x7fce4c0d4fd0 msgr2=0x7fce4c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce536c4640 1 -- 172.21.15.40:0/2767842016 shutdown_connections 2024-04-26T21:44:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.748+0000 7fce536c4640 1 -- 172.21.15.40:0/2767842016 wait complete. 2024-04-26T21:44:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.749+0000 7fce536c4640 1 Processor -- start 2024-04-26T21:44:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce536c4640 1 -- start start 2024-04-26T21:44:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce536c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce4c0b40d0 0x7fce4c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce536c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce4c0d1a00 0x7fce4c1503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce536c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c0ba070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce536c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c162b30 con 0x7fce4c0d2320 2024-04-26T21:44:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce536c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c0b5d60 con 0x7fce4c0b40d0 2024-04-26T21:44:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce536c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fce4c162cb0 con 0x7fce4c0d1a00 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce51c61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c0ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce51c61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c0ba070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55218/0 (socket says 172.21.15.40:55218) 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce51c61640 1 -- 172.21.15.40:0/1660906105 learned_addr learned my addr 172.21.15.40:0/1660906105 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce51460640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce4c0b40d0 0x7fce4c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.750+0000 7fce50c5f640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce4c0d1a00 0x7fce4c1503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce51c61640 1 -- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce4c0d1a00 msgr2=0x7fce4c1503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce51c61640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce4c0d1a00 0x7fce4c1503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce51c61640 1 -- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce4c0b40d0 msgr2=0x7fce4c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce51c61640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce4c0b40d0 0x7fce4c15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce51c61640 1 -- 172.21.15.40:0/1660906105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c15dd10 con 0x7fce4c0d2320 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce51c61640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c0ba070 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fce44001eb0 tx=0x7fce4400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce49ffb640 1 -- 172.21.15.40:0/1660906105 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44017070 con 0x7fce4c0d2320 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce49ffb640 1 -- 172.21.15.40:0/1660906105 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce440052d0 con 0x7fce4c0d2320 2024-04-26T21:44:13.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4c15dff0 con 0x7fce4c0d2320 2024-04-26T21:44:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce49ffb640 1 -- 172.21.15.40:0/1660906105 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44003810 con 0x7fce4c0d2320 2024-04-26T21:44:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.751+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4c15e5b0 con 0x7fce4c0d2320 2024-04-26T21:44:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.752+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4c0763f0 con 0x7fce4c0d2320 2024-04-26T21:44:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.753+0000 7fce49ffb640 1 -- 172.21.15.40:0/1660906105 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce4401a070 con 0x7fce4c0d2320 2024-04-26T21:44:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.753+0000 7fce49ffb640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce3806c8f0 0x7fce3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.753+0000 7fce49ffb640 1 -- 172.21.15.40:0/1660906105 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7507+0+0 (secure 0 0 0) 0x7fce4408b540 con 0x7fce4c0d2320 2024-04-26T21:44:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.753+0000 7fce51460640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce3806c8f0 0x7fce3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.754+0000 7fce51460640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce3806c8f0 0x7fce3806edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce3c003b80 tx=0x7fce3c0098d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.757+0000 7fce49ffb640 1 -- 172.21.15.40:0/1660906105 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce4c0763f0 con 0x7fce4c0d2320 2024-04-26T21:44:13.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:13.985+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fce4c161d50 con 0x7fce4c0d2320 2024-04-26T21:44:14.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.301+0000 7fce49ffb640 1 -- 172.21.15.40:0/1660906105 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v187) ==== 162+0+0 (secure 0 0 0) 0x7fce4c161d50 con 0x7fce4c0d2320 2024-04-26T21:44:14.302 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce3806c8f0 msgr2=0x7fce3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce3806c8f0 0x7fce3806edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce3c003b80 tx=0x7fce3c0098d0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 msgr2=0x7fce4c0ba070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c0ba070 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fce44001eb0 tx=0x7fce4400c890 comp rx=0 tx=0).stop 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 shutdown_connections 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce3806c8f0 0x7fce3806edb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce4c0d2320 0x7fce4c0ba070 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce4c0d1a00 0x7fce4c1503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 --2- 172.21.15.40:0/1660906105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce4c0b40d0 0x7fce4c15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 >> 172.21.15.40:0/1660906105 conn(0x7fce4c0d4fd0 msgr2=0x7fce4c06cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:14.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 shutdown_connections 2024-04-26T21:44:14.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.304+0000 7fce536c4640 1 -- 172.21.15.40:0/1660906105 wait complete. 2024-04-26T21:44:14.313 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:14.313+0000 7feab1ae1640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-04-26T21:44:14.313 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:14.313+0000 7feab1ae1640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:44:14.314 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:14.313+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:14.314 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:14.313+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:14.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:14.319 INFO:tasks.ceph.mds.c.smithi040.stderr: -16> 2024-04-26T21:44:14.313+0000 7feab1ae1640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-04-26T21:44:14.319 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:44:14.313+0000 7feab1ae1640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:44:14.344 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:14.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.419+0000 7f4632bf7640 1 Processor -- start 2024-04-26T21:44:14.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.419+0000 7f4632bf7640 1 -- start start 2024-04-26T21:44:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f4632bf7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 0x7f462c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f4632bf7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f462c0b2340 0x7f462c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f4632bf7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f462c0d10d0 con 0x7f462c0cd930 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f4632bf7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f462c0d0750 con 0x7f462c0b2340 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f4632bf7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f462c0d16a0 con 0x7f462c162810 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f4631194640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f462c0cd930 0x7f462c0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48828/0 (socket says 172.21.15.40:48828) 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f4631194640 1 -- 172.21.15.40:0/4028425414 learned_addr learned my addr 172.21.15.40:0/4028425414 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462bfff640 1 --2- 172.21.15.40:0/4028425414 >> v2:172.21.15.184:3300/0 conn(0x7f462c0b2340 0x7f462c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f4630993640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 0x7f462c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3120923031 0 0) 0x7f462c0d10d0 con 0x7f462c0cd930 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f461c003520 con 0x7f462c0cd930 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2449719019 0 0) 0x7f461c003520 con 0x7f462c0cd930 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 msgr2=0x7f462c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 0x7f462c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 >> v2:172.21.15.184:3300/0 conn(0x7f462c0b2340 msgr2=0x7f462c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 --2- 172.21.15.40:0/4028425414 >> v2:172.21.15.184:3300/0 conn(0x7f462c0b2340 0x7f462c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c0d1370 con 0x7f462c0cd930 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f46240019f0 con 0x7f462c0cd930 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 >> v1:172.21.15.40:6789/0 conn(0x7f462c0cd930 legacy=0x7f462c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.420+0000 7f462b7fe640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 0x7f461c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f462b7fe640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f461c003f30 0x7f461c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f462b7fe640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461c004d40 0x7f461c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0d0750 con 0x7f461c004d40 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0d16a0 con 0x7f461c003f30 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f461c00b9c0 con 0x7f462c162810 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f4631194640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461c004d40 0x7f461c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f4630993640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 0x7f461c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f462bfff640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f461c003f30 0x7f461c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f4631194640 1 -- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 msgr2=0x7f461c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f4631194640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 0x7f461c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f4631194640 1 -- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f461c003f30 msgr2=0x7f461c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f4631194640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f461c003f30 0x7f461c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f4631194640 1 -- 172.21.15.40:0/4028425414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c003520 con 0x7f461c004d40 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.421+0000 7f4630993640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c162810 0x7f461c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f4631194640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461c004d40 0x7f461c0051c0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f462c074050 tx=0x7f461c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dbcb02026bfebb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400a4c0 con 0x7f461c004d40 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462402f070 con 0x7f461c004d40 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f462b7fe640 1 -- 172.21.15.40:0/4028425414 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462402bce0 con 0x7f461c004d40 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f4632bf7640 1 -- 172.21.15.40:0/4028425414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461c004d40 msgr2=0x7f461c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f4632bf7640 1 --2- 172.21.15.40:0/4028425414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461c004d40 0x7f461c0051c0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f462c074050 tx=0x7f461c005830 comp rx=0 tx=0).stop 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f4631194640 1 -- 172.21.15.40:0/4028425414 reap_dead start 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f4632bf7640 1 -- 172.21.15.40:0/4028425414 shutdown_connections 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f4632bf7640 1 -- 172.21.15.40:0/4028425414 >> 172.21.15.40:0/4028425414 conn(0x7f462c0d2da0 msgr2=0x7f462c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f4632bf7640 1 -- 172.21.15.40:0/4028425414 shutdown_connections 2024-04-26T21:44:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.422+0000 7f4632bf7640 1 -- 172.21.15.40:0/4028425414 wait complete. 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.424+0000 7f4632bf7640 1 Processor -- start 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.424+0000 7f4632bf7640 1 -- start start 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.424+0000 7f4632bf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462c151320 0x7f462c1f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.424+0000 7f4632bf7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0c68d0 0x7f462c15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.424+0000 7f4632bf7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c15d290 0x7f462c15eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.424+0000 7f4632bf7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0d2240 con 0x7f462c0c68d0 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.424+0000 7f4632bf7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0d0470 con 0x7f462c151320 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.424+0000 7f4632bf7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f462c0d23c0 con 0x7f462c15d290 2024-04-26T21:44:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0c68d0 0x7f462c15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0c68d0 0x7f462c15cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55254/0 (socket says 172.21.15.40:55254) 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 -- 172.21.15.40:0/1802356272 learned_addr learned my addr 172.21.15.40:0/1802356272 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f4630993640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462c151320 0x7f462c1f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f4631194640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c15d290 0x7f462c15eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 -- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c15d290 msgr2=0x7f462c15eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c15d290 0x7f462c15eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 -- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462c151320 msgr2=0x7f462c1f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462c151320 0x7f462c1f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 -- 172.21.15.40:0/1802356272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c151700 con 0x7f462c0c68d0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f462bfff640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0c68d0 0x7f462c15cd50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f4618011940 tx=0x7f4618011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f46297fa640 1 -- 172.21.15.40:0/1802356272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461800f9d0 con 0x7f462c0c68d0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f46297fa640 1 -- 172.21.15.40:0/1802356272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461800fb90 con 0x7f462c0c68d0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f46297fa640 1 -- 172.21.15.40:0/1802356272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618014440 con 0x7f462c0c68d0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c0c6e90 con 0x7f462c0c68d0 2024-04-26T21:44:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.425+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c0c7450 con 0x7f462c0c68d0 2024-04-26T21:44:14.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.426+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c15d6a0 con 0x7f462c0c68d0 2024-04-26T21:44:14.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.427+0000 7f46297fa640 1 -- 172.21.15.40:0/1802356272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4618012040 con 0x7f462c0c68d0 2024-04-26T21:44:14.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.427+0000 7f46297fa640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f461006c920 0x7f461006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.427+0000 7f46297fa640 1 -- 172.21.15.40:0/1802356272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7593+0+0 (secure 0 0 0) 0x7f461808c700 con 0x7f462c0c68d0 2024-04-26T21:44:14.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.427+0000 7f4630993640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f461006c920 0x7f461006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.428+0000 7f4630993640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f461006c920 0x7f461006ede0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f462c0b2870 tx=0x7f4620009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:14.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.432+0000 7f46297fa640 1 -- 172.21.15.40:0/1802356272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f462c15d6a0 con 0x7f462c0c68d0 2024-04-26T21:44:14.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.654+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f462c0694f0 con 0x7f462c0c68d0 2024-04-26T21:44:14.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.654+0000 7f46297fa640 1 -- 172.21.15.40:0/1802356272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) ==== 95+0+4355 (secure 0 0 0) 0x7f4618019030 con 0x7f462c0c68d0 2024-04-26T21:44:14.655 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:14.655 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":189,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:14.315037+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:14.656 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 189 2024-04-26T21:44:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f461006c920 msgr2=0x7f461006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f461006c920 0x7f461006ede0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f462c0b2870 tx=0x7f4620009190 comp rx=0 tx=0).stop 2024-04-26T21:44:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0c68d0 msgr2=0x7f462c15cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0c68d0 0x7f462c15cd50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f4618011940 tx=0x7f4618011d40 comp rx=0 tx=0).stop 2024-04-26T21:44:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 shutdown_connections 2024-04-26T21:44:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f461006c920 0x7f461006ede0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f462c15d290 0x7f462c15eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0c68d0 0x7f462c15cd50 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 --2- 172.21.15.40:0/1802356272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462c151320 0x7f462c1f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 >> 172.21.15.40:0/1802356272 conn(0x7f462c0d2da0 msgr2=0x7f462c0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 shutdown_connections 2024-04-26T21:44:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.656+0000 7f4632bf7640 1 -- 172.21.15.40:0/1802356272 wait complete. 2024-04-26T21:44:14.668 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:14.668 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:14.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.809+0000 7f7473389640 1 Processor -- start 2024-04-26T21:44:14.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.809+0000 7f7473389640 1 -- start start 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f7473389640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f746c162810 0x7f746c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f7473389640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746c0d10d0 con 0x7f746c0d1b30 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f7473389640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746c0d1370 con 0x7f746c0b2340 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f7473389640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f746c0d0750 con 0x7f746c162810 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f7471926640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f746c0d1b30 0x7f746c070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48844/0 (socket says 172.21.15.40:48844) 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f7471926640 1 -- 172.21.15.40:0/3222693777 learned_addr learned my addr 172.21.15.40:0/3222693777 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030541528 0 0) 0x7f746c0d10d0 con 0x7f746c0d1b30 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7454003520 con 0x7f746c0d1b30 2024-04-26T21:44:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.810+0000 7f7471125640 1 --2- 172.21.15.40:0/3222693777 >> v2:172.21.15.184:3300/0 conn(0x7f746c162810 0x7f746c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1356217892 0 0) 0x7f7454003520 con 0x7f746c0d1b30 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 >> v2:172.21.15.184:3300/0 conn(0x7f746c162810 msgr2=0x7f746c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 --2- 172.21.15.40:0/3222693777 >> v2:172.21.15.184:3300/0 conn(0x7f746c162810 0x7f746c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 >> v1:172.21.15.184:6789/0 conn(0x7f746c0b2340 legacy=0x7f746c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c0d01b0 con 0x7f746c0d1b30 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f745c0019f0 con 0x7f746c0d1b30 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 >> v1:172.21.15.40:6789/0 conn(0x7f746c0d1b30 legacy=0x7f746c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f7454003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7454004800 0x7f7454004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74540051c0 0x7f745400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f746c0d0750 con 0x7f74540039b0 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f745400c0f0 con 0x7f7454004800 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f745400c2f0 con 0x7f74540051c0 2024-04-26T21:44:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471125640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f7454003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7470924640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7454004800 0x7f7454004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471926640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74540051c0 0x7f745400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471125640 1 -- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74540051c0 msgr2=0x7f745400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471125640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74540051c0 0x7f745400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471125640 1 -- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7454004800 msgr2=0x7f7454004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471125640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7454004800 0x7f7454004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471125640 1 -- 172.21.15.40:0/3222693777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7454003520 con 0x7f74540039b0 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471926640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74540051c0 0x7f745400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.811+0000 7f7471926640 1 -- 172.21.15.40:0/3222693777 reap_dead start 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.812+0000 7f7471125640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f7454003db0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f746400ba00 tx=0x7f7454005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceda551f04aedca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.812+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464030070 con 0x7f74540039b0 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.812+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746402ad70 con 0x7f74540039b0 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.812+0000 7f746bfff640 1 -- 172.21.15.40:0/3222693777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746400b600 con 0x7f74540039b0 2024-04-26T21:44:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.813+0000 7f7473389640 1 -- 172.21.15.40:0/3222693777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 msgr2=0x7f7454003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.813+0000 7f7473389640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f7454003db0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f746400ba00 tx=0x7f7454005830 comp rx=0 tx=0).stop 2024-04-26T21:44:14.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.813+0000 7f7473389640 1 -- 172.21.15.40:0/3222693777 shutdown_connections 2024-04-26T21:44:14.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.813+0000 7f7473389640 1 --2- 172.21.15.40:0/3222693777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f7454003db0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.813+0000 7f7473389640 1 -- 172.21.15.40:0/3222693777 >> 172.21.15.40:0/3222693777 conn(0x7f746c0d2da0 msgr2=0x7f746c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:14.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.813+0000 7f7473389640 1 -- 172.21.15.40:0/3222693777 shutdown_connections 2024-04-26T21:44:14.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.813+0000 7f7473389640 1 -- 172.21.15.40:0/3222693777 wait complete. 2024-04-26T21:44:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.814+0000 7f7473389640 1 Processor -- start 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.814+0000 7f7473389640 1 -- start start 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7473389640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f746c1f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7473389640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f746c156b10 0x7f746c156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7473389640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f746c0ce490 0x7f746c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7473389640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f746c0d1fc0 con 0x7f74540039b0 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7473389640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f746c0d16a0 con 0x7f746c0ce490 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7473389640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f746c0d2140 con 0x7f746c156b10 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f746c1f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f746c1f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55274/0 (socket says 172.21.15.40:55274) 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 -- 172.21.15.40:0/798721614 learned_addr learned my addr 172.21.15.40:0/798721614 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471926640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f746c0ce490 0x7f746c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 -- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f746c156b10 msgr2=0x7f746c156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7470924640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f746c156b10 0x7f746c156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f746c156b10 0x7f746c156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 -- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f746c0ce490 msgr2=0x7f746c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f746c0ce490 0x7f746c0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 -- 172.21.15.40:0/798721614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c1f55d0 con 0x7f74540039b0 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471926640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f746c0ce490 0x7f746c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7471125640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f746c1f5090 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f746400bb20 tx=0x7f7464004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7469ffb640 1 -- 172.21.15.40:0/798721614 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464043030 con 0x7f74540039b0 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7469ffb640 1 -- 172.21.15.40:0/798721614 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746402ab70 con 0x7f74540039b0 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.815+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c0c9e70 con 0x7f74540039b0 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.816+0000 7f7469ffb640 1 -- 172.21.15.40:0/798721614 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746400b600 con 0x7f74540039b0 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.816+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c0ca430 con 0x7f74540039b0 2024-04-26T21:44:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.817+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746c0694f0 con 0x7f74540039b0 2024-04-26T21:44:14.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.817+0000 7f7469ffb640 1 -- 172.21.15.40:0/798721614 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7464046070 con 0x7f74540039b0 2024-04-26T21:44:14.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.818+0000 7f7469ffb640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f745806c920 0x7f745806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:14.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.818+0000 7f7469ffb640 1 -- 172.21.15.40:0/798721614 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7593+0+0 (secure 0 0 0) 0x7f74640accf0 con 0x7f74540039b0 2024-04-26T21:44:14.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.818+0000 7f7470924640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f745806c920 0x7f745806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:14.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.819+0000 7f7470924640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f745806c920 0x7f745806ede0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f746c162220 tx=0x7f7460009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:14.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:14.822+0000 7f7469ffb640 1 -- 172.21.15.40:0/798721614 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f746407a9a0 con 0x7f74540039b0 2024-04-26T21:44:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.055+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f746c162380 con 0x7f74540039b0 2024-04-26T21:44:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.055+0000 7f7469ffb640 1 -- 172.21.15.40:0/798721614 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) ==== 95+0+4355 (secure 0 0 0) 0x7f746407a2a0 con 0x7f74540039b0 2024-04-26T21:44:15.056 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:15.056 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":189,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:14.315037+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 189 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.057+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f745806c920 msgr2=0x7f745806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.057+0000 7f7473389640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f745806c920 0x7f745806ede0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f746c162220 tx=0x7f7460009040 comp rx=0 tx=0).stop 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.057+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 msgr2=0x7f746c1f5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.057+0000 7f7473389640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f746c1f5090 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f746400bb20 tx=0x7f7464004730 comp rx=0 tx=0).stop 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.058+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 shutdown_connections 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.058+0000 7f7473389640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f745806c920 0x7f745806ede0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.058+0000 7f7473389640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f746c0ce490 0x7f746c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.058+0000 7f7473389640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f746c156b10 0x7f746c156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.058+0000 7f7473389640 1 --2- 172.21.15.40:0/798721614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74540039b0 0x7f746c1f5090 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.058+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 >> 172.21.15.40:0/798721614 conn(0x7f746c0d2da0 msgr2=0x7f746c0b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.058+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 shutdown_connections 2024-04-26T21:44:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.058+0000 7f7473389640 1 -- 172.21.15.40:0/798721614 wait complete. 2024-04-26T21:44:15.069 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:15.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:15.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.169+0000 7f5c1c36a640 1 Processor -- start 2024-04-26T21:44:15.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.169+0000 7f5c1c36a640 1 -- start start 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.170+0000 7f5c1c36a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c14071e60 0x7f5c1415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.170+0000 7f5c1c36a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c140cc5b0 0x7f5c14166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.170+0000 7f5c1c36a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c1415e4d0 con 0x7f5c14072240 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.170+0000 7f5c1c36a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5c140701f0 con 0x7f5c140cc5b0 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.170+0000 7f5c1c36a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1406e240 con 0x7f5c14071e60 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.170+0000 7f5c1a907640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c140cc5b0 0x7f5c14166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.170+0000 7f5c19905640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5c14072240 0x7f5c140726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48852/0 (socket says 172.21.15.40:48852) 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.170+0000 7f5c19905640 1 -- 172.21.15.40:0/3383350671 learned_addr learned my addr 172.21.15.40:0/3383350671 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c1a106640 1 --2- 172.21.15.40:0/3383350671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c14071e60 0x7f5c1415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c1a907640 1 -- 172.21.15.40:0/3383350671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c14071e60 msgr2=0x7f5c1415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c1a907640 1 --2- 172.21.15.40:0/3383350671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c14071e60 0x7f5c1415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c1a907640 1 -- 172.21.15.40:0/3383350671 >> v1:172.21.15.40:6789/0 conn(0x7f5c14072240 legacy=0x7f5c140726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c1a907640 1 -- 172.21.15.40:0/3383350671 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1406fc30 con 0x7f5c140cc5b0 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c19104640 1 -- 172.21.15.40:0/3383350671 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3922762980 0 0) 0x7f5c04001110 con 0x7f5c14072240 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c1a106640 1 --2- 172.21.15.40:0/3383350671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c14071e60 0x7f5c1415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c1a907640 1 --2- 172.21.15.40:0/3383350671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c140cc5b0 0x7f5c14166b30 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f5c14070370 tx=0x7f5c0c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6d53b9a773daa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c19104640 1 -- 172.21.15.40:0/3383350671 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c007740 con 0x7f5c140cc5b0 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c19104640 1 -- 172.21.15.40:0/3383350671 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c0078e0 con 0x7f5c140cc5b0 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.171+0000 7f5c19104640 1 -- 172.21.15.40:0/3383350671 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c007cb0 con 0x7f5c140cc5b0 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.172+0000 7f5c1c36a640 1 -- 172.21.15.40:0/3383350671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c140cc5b0 msgr2=0x7f5c14166b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.172+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/3383350671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c140cc5b0 0x7f5c14166b30 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f5c14070370 tx=0x7f5c0c02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.172+0000 7f5c1c36a640 1 -- 172.21.15.40:0/3383350671 shutdown_connections 2024-04-26T21:44:15.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.172+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/3383350671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c140cc5b0 0x7f5c14166b30 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.172+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/3383350671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c14071e60 0x7f5c1415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.172+0000 7f5c1c36a640 1 -- 172.21.15.40:0/3383350671 >> 172.21.15.40:0/3383350671 conn(0x7f5c140d0be0 msgr2=0x7f5c140d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:15.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.172+0000 7f5c1c36a640 1 -- 172.21.15.40:0/3383350671 shutdown_connections 2024-04-26T21:44:15.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.172+0000 7f5c1c36a640 1 -- 172.21.15.40:0/3383350671 wait complete. 2024-04-26T21:44:15.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.173+0000 7f5c1c36a640 1 Processor -- start 2024-04-26T21:44:15.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.173+0000 7f5c1c36a640 1 -- start start 2024-04-26T21:44:15.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1c36a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c14071e60 0x7f5c140b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1c36a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c14072240 0x7f5c140b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1c36a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c140cc5b0 0x7f5c14148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1c36a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1406f8e0 con 0x7f5c14072240 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1c36a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5c14073fb0 con 0x7f5c14071e60 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1c36a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1406ddf0 con 0x7f5c140cc5b0 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c19905640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c14072240 0x7f5c140b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c19905640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c14072240 0x7f5c140b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55290/0 (socket says 172.21.15.40:55290) 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c19905640 1 -- 172.21.15.40:0/1653673462 learned_addr learned my addr 172.21.15.40:0/1653673462 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1a907640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c140cc5b0 0x7f5c14148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c19905640 1 -- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c140cc5b0 msgr2=0x7f5c14148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c19905640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c140cc5b0 0x7f5c14148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1a106640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c14071e60 0x7f5c140b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c19905640 1 -- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c14071e60 msgr2=0x7f5c140b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c19905640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c14071e60 0x7f5c140b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c19905640 1 -- 172.21.15.40:0/1653673462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c140b4fd0 con 0x7f5c14072240 2024-04-26T21:44:15.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1a907640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c140cc5b0 0x7f5c14148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.174+0000 7f5c1a106640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c14071e60 0x7f5c140b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.175+0000 7f5c19905640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c14072240 0x7f5c140b5950 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400c7b0 tx=0x7f5c0400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.175+0000 7f5c0affd640 1 -- 172.21.15.40:0/1653673462 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04018070 con 0x7f5c14072240 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.175+0000 7f5c0affd640 1 -- 172.21.15.40:0/1653673462 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c040072d0 con 0x7f5c14072240 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.175+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c140c4cb0 con 0x7f5c14072240 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.175+0000 7f5c0affd640 1 -- 172.21.15.40:0/1653673462 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c040047e0 con 0x7f5c14072240 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.175+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c140c5270 con 0x7f5c14072240 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.176+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c140c5a30 con 0x7f5c14072240 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.176+0000 7f5c0affd640 1 -- 172.21.15.40:0/1653673462 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c0401b070 con 0x7f5c14072240 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.177+0000 7f5c0affd640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bf806c8f0 0x7f5bf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.177+0000 7f5c0affd640 1 -- 172.21.15.40:0/1653673462 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7593+0+0 (secure 0 0 0) 0x7f5c0408b590 con 0x7f5c14072240 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.177+0000 7f5c1a106640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bf806c8f0 0x7f5bf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.177+0000 7f5c1a106640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bf806c8f0 0x7f5bf806edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5c10006fd0 tx=0x7f5c10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:15.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.181+0000 7f5c0affd640 1 -- 172.21.15.40:0/1653673462 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c140c5a30 con 0x7f5c14072240 2024-04-26T21:44:15.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.406+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c140694f0 con 0x7f5c14072240 2024-04-26T21:44:15.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.406+0000 7f5c0affd640 1 -- 172.21.15.40:0/1653673462 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+5175 (secure 0 0 0) 0x7f5c1415e4d0 con 0x7f5c14072240 2024-04-26T21:44:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:15.407 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":190,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9573,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/45688484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":45688484},{"type":"v1","addr":"172.21.15.40:6833","nonce":45688484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":190}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:14.315037+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:15.407 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 190 2024-04-26T21:44:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bf806c8f0 msgr2=0x7f5bf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bf806c8f0 0x7f5bf806edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5c10006fd0 tx=0x7f5c10008040 comp rx=0 tx=0).stop 2024-04-26T21:44:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c14072240 msgr2=0x7f5c140b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c14072240 0x7f5c140b5950 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400c7b0 tx=0x7f5c0400cc70 comp rx=0 tx=0).stop 2024-04-26T21:44:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 shutdown_connections 2024-04-26T21:44:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5bf806c8f0 0x7f5bf806edb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c140cc5b0 0x7f5c14148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c14072240 0x7f5c140b5950 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 --2- 172.21.15.40:0/1653673462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c14071e60 0x7f5c140b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 >> 172.21.15.40:0/1653673462 conn(0x7f5c140d0be0 msgr2=0x7f5c1415bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 shutdown_connections 2024-04-26T21:44:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.408+0000 7f5c1c36a640 1 -- 172.21.15.40:0/1653673462 wait complete. 2024-04-26T21:44:15.420 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 1.00000000.inode' led to DAMAGED state 2024-04-26T21:44:15.420 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000000' 2024-04-26T21:44:15.420 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:15.420 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:15.435 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:15.435 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:15.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:15.436 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:15.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:15.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:15.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.632+0000 7f1808b59640 1 Processor -- start 2024-04-26T21:44:15.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.632+0000 7f1808b59640 1 -- start start 2024-04-26T21:44:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1808b59640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f18040b2340 0x7f18040cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1808b59640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f18040cd930 0x7f18040d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1808b59640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18040d10d0 con 0x7f1804162810 2024-04-26T21:44:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1808b59640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f18040d0750 con 0x7f18040b2340 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1808b59640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f18040d16a0 con 0x7f18040cd930 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1801d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f18040b2340 0x7f18040cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f180259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1804162810 0x7f1804070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48858/0 (socket says 172.21.15.40:48858) 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f180259c640 1 -- 172.21.15.40:0/1238876544 learned_addr learned my addr 172.21.15.40:0/1238876544 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1802d9d640 1 --2- 172.21.15.40:0/1238876544 >> v2:172.21.15.184:3300/0 conn(0x7f18040cd930 0x7f18040d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1801d9b640 1 -- 172.21.15.40:0/1238876544 >> v2:172.21.15.184:3300/0 conn(0x7f18040cd930 msgr2=0x7f18040d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1801d9b640 1 --2- 172.21.15.40:0/1238876544 >> v2:172.21.15.184:3300/0 conn(0x7f18040cd930 0x7f18040d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1801d9b640 1 -- 172.21.15.40:0/1238876544 >> v1:172.21.15.40:6789/0 conn(0x7f1804162810 legacy=0x7f1804070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f1801d9b640 1 -- 172.21.15.40:0/1238876544 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18040d1370 con 0x7f18040b2340 2024-04-26T21:44:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.633+0000 7f180159a640 1 -- 172.21.15.40:0/1238876544 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1540571762 0 0) 0x7f18040d10d0 con 0x7f1804162810 2024-04-26T21:44:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.634+0000 7f1801d9b640 1 --2- 172.21.15.40:0/1238876544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18040b2340 0x7f18040cd3f0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f18040b2280 tx=0x7f17f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd13dcbdd1b5d80f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.634+0000 7f180159a640 1 -- 172.21.15.40:0/1238876544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8037070 con 0x7f18040b2340 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.634+0000 7f180159a640 1 -- 172.21.15.40:0/1238876544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f8032440 con 0x7f18040b2340 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.634+0000 7f180159a640 1 -- 172.21.15.40:0/1238876544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8032810 con 0x7f18040b2340 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.635+0000 7f1808b59640 1 -- 172.21.15.40:0/1238876544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18040b2340 msgr2=0x7f18040cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.635+0000 7f1808b59640 1 --2- 172.21.15.40:0/1238876544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18040b2340 0x7f18040cd3f0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f18040b2280 tx=0x7f17f802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.635+0000 7f1808b59640 1 -- 172.21.15.40:0/1238876544 shutdown_connections 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.635+0000 7f1808b59640 1 --2- 172.21.15.40:0/1238876544 >> v2:172.21.15.184:3300/0 conn(0x7f18040cd930 0x7f18040d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.635+0000 7f1808b59640 1 --2- 172.21.15.40:0/1238876544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18040b2340 0x7f18040cd3f0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.635+0000 7f1808b59640 1 -- 172.21.15.40:0/1238876544 >> 172.21.15.40:0/1238876544 conn(0x7f18040d2da0 msgr2=0x7f18040d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.635+0000 7f1808b59640 1 -- 172.21.15.40:0/1238876544 shutdown_connections 2024-04-26T21:44:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.635+0000 7f1808b59640 1 -- 172.21.15.40:0/1238876544 wait complete. 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.636+0000 7f1808b59640 1 Processor -- start 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.636+0000 7f1808b59640 1 -- start start 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1808b59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18040b2340 0x7f1804160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1808b59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18040cd930 0x7f18040c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1808b59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1804162810 0x7f180415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1808b59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f18040d2070 con 0x7f1804162810 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1808b59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18040d0470 con 0x7f18040cd930 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1808b59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f18040d21f0 con 0x7f18040b2340 2024-04-26T21:44:15.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1804162810 0x7f180415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1804162810 0x7f180415bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55306/0 (socket says 172.21.15.40:55306) 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1802d9d640 1 -- 172.21.15.40:0/3506703442 learned_addr learned my addr 172.21.15.40:0/3506703442 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f180259c640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18040b2340 0x7f1804160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1801d9b640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18040cd930 0x7f18040c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1802d9d640 1 -- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18040b2340 msgr2=0x7f1804160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1802d9d640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18040b2340 0x7f1804160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1802d9d640 1 -- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18040cd930 msgr2=0x7f18040c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1802d9d640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18040cd930 0x7f18040c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1802d9d640 1 -- 172.21.15.40:0/3506703442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18040c8580 con 0x7f1804162810 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f180259c640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18040b2340 0x7f1804160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.637+0000 7f1801d9b640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18040cd930 0x7f18040c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.638+0000 7f1802d9d640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1804162810 0x7f180415bb40 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f17f400cdc0 tx=0x7f17f400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.638+0000 7f17f2ffd640 1 -- 172.21.15.40:0/3506703442 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4003b90 con 0x7f1804162810 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.638+0000 7f17f2ffd640 1 -- 172.21.15.40:0/3506703442 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f4003d50 con 0x7f1804162810 2024-04-26T21:44:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.638+0000 7f17f2ffd640 1 -- 172.21.15.40:0/3506703442 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4004260 con 0x7f1804162810 2024-04-26T21:44:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.638+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18040c8820 con 0x7f1804162810 2024-04-26T21:44:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.638+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18040c8de0 con 0x7f1804162810 2024-04-26T21:44:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.639+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18040694f0 con 0x7f1804162810 2024-04-26T21:44:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.639+0000 7f17f2ffd640 1 -- 172.21.15.40:0/3506703442 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17f4004400 con 0x7f1804162810 2024-04-26T21:44:15.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.640+0000 7f17f2ffd640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17e406c8f0 0x7f17e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:15.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.640+0000 7f17f2ffd640 1 -- 172.21.15.40:0/3506703442 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7593+0+0 (secure 0 0 0) 0x7f17f408ce10 con 0x7f1804162810 2024-04-26T21:44:15.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.640+0000 7f180259c640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17e406c8f0 0x7f17e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:15.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.640+0000 7f180259c640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17e406c8f0 0x7f17e406edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f17fc000f60 tx=0x7f17fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:15.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.644+0000 7f17f2ffd640 1 -- 172.21.15.40:0/3506703442 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17f40132d0 con 0x7f1804162810 2024-04-26T21:44:15.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:15.874+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f180406efa0 con 0x7f1804162810 2024-04-26T21:44:16.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.324+0000 7f17f2ffd640 1 -- 172.21.15.40:0/3506703442 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v191) ==== 161+0+0 (secure 0 0 0) 0x7f17f405ab70 con 0x7f1804162810 2024-04-26T21:44:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:16.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17e406c8f0 msgr2=0x7f17e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17e406c8f0 0x7f17e406edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f17fc000f60 tx=0x7f17fc009040 comp rx=0 tx=0).stop 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1804162810 msgr2=0x7f180415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1804162810 0x7f180415bb40 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f17f400cdc0 tx=0x7f17f400a660 comp rx=0 tx=0).stop 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 shutdown_connections 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17e406c8f0 0x7f17e406edb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1804162810 0x7f180415bb40 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18040cd930 0x7f18040c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 --2- 172.21.15.40:0/3506703442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18040b2340 0x7f1804160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 >> 172.21.15.40:0/3506703442 conn(0x7f18040d2da0 msgr2=0x7f18040b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:16.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 shutdown_connections 2024-04-26T21:44:16.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.326+0000 7f1808b59640 1 -- 172.21.15.40:0/3506703442 wait complete. 2024-04-26T21:44:16.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.445+0000 7fd3a82c8640 1 Processor -- start 2024-04-26T21:44:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.445+0000 7fd3a82c8640 1 -- start start 2024-04-26T21:44:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.446+0000 7fd3a82c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3a00d06f0 0x7fd3a00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.446+0000 7fd3a82c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a00d0ba0 con 0x7fd3a00b2850 2024-04-26T21:44:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.446+0000 7fd3a82c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a00d0e20 con 0x7fd3a015f3a0 2024-04-26T21:44:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.446+0000 7fd3a82c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd3a00b1d70 con 0x7fd3a00d06f0 2024-04-26T21:44:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.446+0000 7fd3a6865640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd3a00b2850 0x7fd3a00b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48866/0 (socket says 172.21.15.40:48866) 2024-04-26T21:44:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.446+0000 7fd3a5863640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3a00d06f0 0x7fd3a00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.446+0000 7fd3a6865640 1 -- 172.21.15.40:0/1421612631 learned_addr learned my addr 172.21.15.40:0/1421612631 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.447+0000 7fd3a5062640 1 -- 172.21.15.40:0/1421612631 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4051222043 0 0) 0x7fd390001490 con 0x7fd3a00b2850 2024-04-26T21:44:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.447+0000 7fd3a5062640 1 -- 172.21.15.40:0/1421612631 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd388003520 con 0x7fd3a00b2850 2024-04-26T21:44:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.447+0000 7fd3a5863640 1 -- 172.21.15.40:0/1421612631 >> v1:172.21.15.184:6789/0 conn(0x7fd3a015f3a0 legacy=0x7fd3a00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:16.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.447+0000 7fd3a5863640 1 -- 172.21.15.40:0/1421612631 >> v1:172.21.15.40:6789/0 conn(0x7fd3a00b2850 legacy=0x7fd3a00b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:16.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.447+0000 7fd3a5863640 1 -- 172.21.15.40:0/1421612631 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a00b2050 con 0x7fd3a00d06f0 2024-04-26T21:44:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.447+0000 7fd3a5863640 1 --2- 172.21.15.40:0/1421612631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a00d06f0 0x7fd3a00d1e80 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd3940097a0 tx=0x7fd39402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e96290f8ea1bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.447+0000 7fd3a5062640 1 -- 172.21.15.40:0/1421612631 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394037070 con 0x7fd3a00d06f0 2024-04-26T21:44:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a5062640 1 -- 172.21.15.40:0/1421612631 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd394032440 con 0x7fd3a00d06f0 2024-04-26T21:44:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a5062640 1 -- 172.21.15.40:0/1421612631 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394032810 con 0x7fd3a00d06f0 2024-04-26T21:44:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1421612631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a00d06f0 msgr2=0x7fd3a00d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a82c8640 1 --2- 172.21.15.40:0/1421612631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a00d06f0 0x7fd3a00d1e80 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd3940097a0 tx=0x7fd39402a230 comp rx=0 tx=0).stop 2024-04-26T21:44:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1421612631 shutdown_connections 2024-04-26T21:44:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a82c8640 1 --2- 172.21.15.40:0/1421612631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a00d06f0 0x7fd3a00d1e80 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1421612631 >> 172.21.15.40:0/1421612631 conn(0x7fd3a00d4f90 msgr2=0x7fd3a00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1421612631 shutdown_connections 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.448+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1421612631 wait complete. 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.449+0000 7fd3a82c8640 1 Processor -- start 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.449+0000 7fd3a82c8640 1 -- start start 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a82c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a00b2850 0x7fd3a00b8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a82c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a00d06f0 0x7fd3a0159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a82c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a015f3a0 0x7fd3a00b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a82c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a015f900 con 0x7fd3a015f3a0 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a82c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a015f780 con 0x7fd3a00b2850 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a82c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00d23c0 con 0x7fd3a00d06f0 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a6865640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a015f3a0 0x7fd3a00b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a6865640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a015f3a0 0x7fd3a00b9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55326/0 (socket says 172.21.15.40:55326) 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a6865640 1 -- 172.21.15.40:0/1633039229 learned_addr learned my addr 172.21.15.40:0/1633039229 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a6064640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a00b2850 0x7fd3a00b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.450+0000 7fd3a5863640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a00d06f0 0x7fd3a0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a6865640 1 -- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a00d06f0 msgr2=0x7fd3a0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a6865640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a00d06f0 0x7fd3a0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a6865640 1 -- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a00b2850 msgr2=0x7fd3a00b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a6865640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a00b2850 0x7fd3a00b8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a6865640 1 -- 172.21.15.40:0/1633039229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a00b9800 con 0x7fd3a015f3a0 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a5863640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a00d06f0 0x7fd3a0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a6865640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a015f3a0 0x7fd3a00b9dd0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd39000cb50 tx=0x7fd39000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd39effd640 1 -- 172.21.15.40:0/1633039229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390017070 con 0x7fd3a015f3a0 2024-04-26T21:44:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd39effd640 1 -- 172.21.15.40:0/1633039229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39000ad40 con 0x7fd3a015f3a0 2024-04-26T21:44:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a00b7cb0 con 0x7fd3a015f3a0 2024-04-26T21:44:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd39effd640 1 -- 172.21.15.40:0/1633039229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390012620 con 0x7fd3a015f3a0 2024-04-26T21:44:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.451+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a0159b40 con 0x7fd3a015f3a0 2024-04-26T21:44:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.452+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a00694f0 con 0x7fd3a015f3a0 2024-04-26T21:44:16.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.452+0000 7fd39effd640 1 -- 172.21.15.40:0/1633039229 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd39001a070 con 0x7fd3a015f3a0 2024-04-26T21:44:16.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.453+0000 7fd39effd640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd38c06c8f0 0x7fd38c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:16.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.453+0000 7fd39effd640 1 -- 172.21.15.40:0/1633039229 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7593+0+0 (secure 0 0 0) 0x7fd39008bff0 con 0x7fd3a015f3a0 2024-04-26T21:44:16.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.453+0000 7fd3a6064640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd38c06c8f0 0x7fd38c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:16.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.454+0000 7fd3a6064640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd38c06c8f0 0x7fd38c06edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd3980016e0 tx=0x7fd398009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:16.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.457+0000 7fd39effd640 1 -- 172.21.15.40:0/1633039229 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3a00d0ba0 con 0x7fd3a015f3a0 2024-04-26T21:44:16.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:16.717+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fd3a00b6860 con 0x7fd3a015f3a0 2024-04-26T21:44:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.329+0000 7fd39effd640 1 -- 172.21.15.40:0/1633039229 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v192) ==== 101+0+0 (secure 0 0 0) 0x7fd390059700 con 0x7fd3a015f3a0 2024-04-26T21:44:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd38c06c8f0 msgr2=0x7fd38c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd38c06c8f0 0x7fd38c06edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd3980016e0 tx=0x7fd398009040 comp rx=0 tx=0).stop 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a015f3a0 msgr2=0x7fd3a00b9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a015f3a0 0x7fd3a00b9dd0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd39000cb50 tx=0x7fd39000a430 comp rx=0 tx=0).stop 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 shutdown_connections 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd38c06c8f0 0x7fd38c06edb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3a015f3a0 0x7fd3a00b9dd0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a00d06f0 0x7fd3a0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 --2- 172.21.15.40:0/1633039229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a00b2850 0x7fd3a00b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 >> 172.21.15.40:0/1633039229 conn(0x7fd3a00d4f90 msgr2=0x7fd3a006d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 shutdown_connections 2024-04-26T21:44:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.332+0000 7fd3a82c8640 1 -- 172.21.15.40:0/1633039229 wait complete. 2024-04-26T21:44:17.345 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.387+0000 7f61d2487f40 1 Processor -- start 2024-04-26T21:44:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.387+0000 7f61d2487f40 1 -- start start 2024-04-26T21:44:17.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61d2487f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562a51f9420 0x5562a5277aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61d2487f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a5280f70 0x5562a51daa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61d2487f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562a5287b70 0x5562a51daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61d2487f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5562a5156e40 con 0x5562a5280f70 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61d2487f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5562a51f8660 con 0x5562a5287b70 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61d2487f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5562a51b1520 con 0x5562a51f9420 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a5280f70 0x5562a51daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a5280f70 0x5562a51daa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55348/0 (socket says 172.21.15.40:55348) 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 -- 172.21.15.40:0/2840086905 learned_addr learned my addr 172.21.15.40:0/2840086905 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 -- 172.21.15.40:0/2840086905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562a51f9420 msgr2=0x5562a5277aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 --2- 172.21.15.40:0/2840086905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562a51f9420 0x5562a5277aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 -- 172.21.15.40:0/2840086905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562a5287b70 msgr2=0x5562a51daf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 --2- 172.21.15.40:0/2840086905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562a5287b70 0x5562a51daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 -- 172.21.15.40:0/2840086905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a5289e80 con 0x5562a5280f70 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61cf782640 1 --2- 172.21.15.40:0/2840086905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a5280f70 0x5562a51daa50 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b860 tx=0x7f61b800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e77f989f36189dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61ce780640 1 -- 172.21.15.40:0/2840086905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b800abb0 con 0x5562a5280f70 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61ce780640 1 -- 172.21.15.40:0/2840086905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b800ad50 con 0x5562a5280f70 2024-04-26T21:44:17.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.388+0000 7f61ce780640 1 -- 172.21.15.40:0/2840086905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8008660 con 0x5562a5280f70 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 -- 172.21.15.40:0/2840086905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a5280f70 msgr2=0x5562a51daa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 --2- 172.21.15.40:0/2840086905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a5280f70 0x5562a51daa50 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b860 tx=0x7f61b800bd20 comp rx=0 tx=0).stop 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 -- 172.21.15.40:0/2840086905 shutdown_connections 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 --2- 172.21.15.40:0/2840086905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562a5287b70 0x5562a51daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 --2- 172.21.15.40:0/2840086905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a5280f70 0x5562a51daa50 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 --2- 172.21.15.40:0/2840086905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562a51f9420 0x5562a5277aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 -- 172.21.15.40:0/2840086905 >> 172.21.15.40:0/2840086905 conn(0x5562a51d98a0 msgr2=0x5562a51ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 -- 172.21.15.40:0/2840086905 shutdown_connections 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.389+0000 7f61d2487f40 1 -- 172.21.15.40:0/2840086905 wait complete. 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61d2487f40 1 Processor -- start 2024-04-26T21:44:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61d2487f40 1 -- start start 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61d2487f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a51f9420 0x5562a51f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61d2487f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562a5280f70 0x5562a5277aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61d2487f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562a5287b70 0x5562a5277fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61d2487f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5562a51b1770 con 0x5562a51f9420 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61d2487f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5562a51b00c0 con 0x5562a5280f70 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61d2487f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5562a52893f0 con 0x5562a5287b70 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.390+0000 7f61cff83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a51f9420 0x5562a51f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cff83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a51f9420 0x5562a51f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55364/0 (socket says 172.21.15.40:55364) 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 learned_addr learned my addr 172.21.15.40:0/1915552056 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61d0784640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562a5287b70 0x5562a5277fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cf782640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562a5280f70 0x5562a5277aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562a5287b70 msgr2=0x5562a5277fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cff83640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562a5287b70 0x5562a5277fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562a5280f70 msgr2=0x5562a5277aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cff83640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562a5280f70 0x5562a5277aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a51b10e0 con 0x5562a51f9420 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cff83640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a51f9420 0x5562a51f3c30 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f61c0002190 tx=0x7f61c000b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cd77e640 1 -- 172.21.15.40:0/1915552056 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0006220 con 0x5562a51f9420 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cd77e640 1 -- 172.21.15.40:0/1915552056 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c0005440 con 0x5562a51f9420 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61cd77e640 1 -- 172.21.15.40:0/1915552056 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0005870 con 0x5562a51f9420 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5562a51f3150 con 0x5562a51f9420 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.391+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562a5277580 con 0x5562a51f9420 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.393+0000 7f61cd77e640 1 -- 172.21.15.40:0/1915552056 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61c0005a10 con 0x5562a51f9420 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.393+0000 7f61cd77e640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61b006b2d0 0x7f61b006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.393+0000 7f61cd77e640 1 -- 172.21.15.40:0/1915552056 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7593+0+0 (secure 0 0 0) 0x7f61c008a420 con 0x5562a51f9420 2024-04-26T21:44:17.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.393+0000 7f61cf782640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61b006b2d0 0x7f61b006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.397 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.394+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5562a5274270 0x5562a52818f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.394+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a51f4170 con 0x5562a5274270 2024-04-26T21:44:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.394+0000 7f61d0784640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5562a5274270 0x5562a52818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.394+0000 7f61cf782640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61b006b2d0 0x7f61b006d790 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f61b800bd60 tx=0x7f61b800d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.394+0000 7f61d0784640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5562a5274270 0x5562a52818f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.395+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v111'474 uv448 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.395+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a5281e30 con 0x5562a5274270 2024-04-26T21:44:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.396+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v111'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.396+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a52821d0 con 0x5562a5274270 2024-04-26T21:44:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.397+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v111'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.399 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.398+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532af10 con 0x5562a5274270 2024-04-26T21:44:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.398+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v111'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.398+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532b2b0 con 0x5562a5274270 2024-04-26T21:44:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.399+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v111'478 uv478 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.399+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5329f00 con 0x5562a5274270 2024-04-26T21:44:17.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.400+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v111'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.400+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a310 con 0x5562a5274270 2024-04-26T21:44:17.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.401+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v111'480 uv454 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.401+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a532a700 con 0x5562a5274270 2024-04-26T21:44:17.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.402+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v111'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.402+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a700 con 0x5562a5274270 2024-04-26T21:44:17.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.403+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v111'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.404 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:17.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.403+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.404+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v111'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.404+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.405+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v111'484 uv484 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.405+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.406+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v111'485 uv485 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.406+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.407+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v111'486 uv461 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.407+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.408+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v111'487 uv487 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.408+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.409+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v111'488 uv488 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.409 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.409+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.410+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v111'489 uv489 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.410+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.411+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v111'490 uv490 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.411+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532a6b0 con 0x5562a5274270 2024-04-26T21:44:17.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.411+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v111'491 uv491 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.412+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.412+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v111'492 uv492 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.412+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.413+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v111'493 uv467 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.414+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.414+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v111'494 uv494 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.414+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.415+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v111'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.416 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.415+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532c660 con 0x5562a5274270 2024-04-26T21:44:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.416+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v111'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.416+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532c660 con 0x5562a5274270 2024-04-26T21:44:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.417+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v111'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.417+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.418+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v111'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.418+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.419+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v111'499 uv473 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.419+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.420+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v111'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.420+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.421+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v111'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.422 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:17.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.421+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532c660 con 0x5562a5274270 2024-04-26T21:44:17.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.422+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v111'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.422+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532c660 con 0x5562a5274270 2024-04-26T21:44:17.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.423+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v111'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.423+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532b650 con 0x5562a5274270 2024-04-26T21:44:17.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.424+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v111'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5274270 2024-04-26T21:44:17.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.424+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5562a532bb60 0x5562a532df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.424+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532e490 con 0x5562a532bb60 2024-04-26T21:44:17.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.424+0000 7f61cff83640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5562a532bb60 0x5562a532df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.425+0000 7f61cff83640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5562a532bb60 0x5562a532df50 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.426+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v111'352 uv351 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.426+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a532e830 con 0x5562a532bb60 2024-04-26T21:44:17.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.428+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v111'353 uv353 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.428+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532ebd0 con 0x5562a532bb60 2024-04-26T21:44:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.429+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v111'354 uv354 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.430 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:17.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.429+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532f420 con 0x5562a532bb60 2024-04-26T21:44:17.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.430+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v111'355 uv355 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.430+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532f7c0 con 0x5562a532bb60 2024-04-26T21:44:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.432+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v111'356 uv356 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.432+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a532fb60 con 0x5562a532bb60 2024-04-26T21:44:17.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.433+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v111'357 uv339 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.433+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a532ff00 con 0x5562a532bb60 2024-04-26T21:44:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.434+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v111'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.434+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.435+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v111'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.436 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.435+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.437+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v111'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.437+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.438+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v111'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.438+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.439+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v111'362 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.439+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.441+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v111'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.441+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.442+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v111'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.443 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.443+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5334960 con 0x5562a532bb60 2024-04-26T21:44:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.444+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v111'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.444+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5334960 con 0x5562a532bb60 2024-04-26T21:44:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.445+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v111'366 uv366 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.445+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.447+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v111'367 uv350 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.447+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.448+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v111'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.448+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.449+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v111'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.450 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.449+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53312b0 con 0x5562a532bb60 2024-04-26T21:44:17.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.450+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v111'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.450+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53312b0 con 0x5562a532bb60 2024-04-26T21:44:17.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.451+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v111'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.452+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a53302a0 con 0x5562a532bb60 2024-04-26T21:44:17.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.453+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v111'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a532bb60 2024-04-26T21:44:17.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.453+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562a53307b0 0x5562a5332ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.453+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5333200 con 0x5562a53307b0 2024-04-26T21:44:17.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.453+0000 7f61cf782640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562a53307b0 0x5562a5332ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.453+0000 7f61cf782640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562a53307b0 0x5562a5332ba0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.455+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v111'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.455+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a53336f0 con 0x5562a53307b0 2024-04-26T21:44:17.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.456+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v111'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.456+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5333bd0 con 0x5562a53307b0 2024-04-26T21:44:17.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.457+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v111'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.458 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:17.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.457+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5335480 con 0x5562a53307b0 2024-04-26T21:44:17.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.458+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v111'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.458+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5335820 con 0x5562a53307b0 2024-04-26T21:44:17.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.459+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v111'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.459+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5334470 con 0x5562a53307b0 2024-04-26T21:44:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.460+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v111'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.460+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5562a5334d70 0x5562a5337bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.460+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5338110 con 0x5562a5334d70 2024-04-26T21:44:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.460+0000 7f61d0784640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5562a5334d70 0x5562a5337bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.461+0000 7f61d0784640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5562a5334d70 0x5562a5337bd0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.462+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v111'76 uv75 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.462+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a5338110 con 0x5562a5334d70 2024-04-26T21:44:17.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.463+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v111'77 uv77 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.463+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5338190 con 0x5562a5334d70 2024-04-26T21:44:17.465 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:17.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.464+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v111'78 uv78 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.464+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5338580 con 0x5562a5334d70 2024-04-26T21:44:17.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.465+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v111'79 uv79 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.466+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5562a5338a90 0x5562a533ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.466+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b450 con 0x5562a5338a90 2024-04-26T21:44:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.466+0000 7f61cff83640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5562a5338a90 0x5562a533ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.466+0000 7f61cff83640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5562a5338a90 0x5562a533ae80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.467+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v111'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.467+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533b4d0 con 0x5562a5338a90 2024-04-26T21:44:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.468+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v111'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.469+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.470 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.470+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v111'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.470+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533c900 con 0x5562a5338a90 2024-04-26T21:44:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.472+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v111'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.472+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533c900 con 0x5562a5338a90 2024-04-26T21:44:17.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.473+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v111'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.473+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.473+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v111'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.473+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.475+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v111'171 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.475+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.476+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v111'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.476+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.478+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v111'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.479 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.478+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533c900 con 0x5562a5334d70 2024-04-26T21:44:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.479+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v111'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.479+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533c900 con 0x5562a5334d70 2024-04-26T21:44:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.480+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v111'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.480+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.481+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v111'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.481+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.483+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v111'177 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.483+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.484+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v111'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.484+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.485+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v111'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.486 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.485+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.486+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v111'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.486+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.487+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v111'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.487+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.488+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v111'76 uv75 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.488+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.489+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v111'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.489+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.490 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.490+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v111'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.490+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.491+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v111'79 uv79 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.491+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5334d70 2024-04-26T21:44:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.492+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v111'80 uv80 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f61c8051070 con 0x5562a5334d70 2024-04-26T21:44:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.492+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.493+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v111'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.493+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.494+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v111'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.494+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.495+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v111'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.496 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:17.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.495+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533c900 con 0x5562a5338a90 2024-04-26T21:44:17.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.496+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v111'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.496+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533c900 con 0x5562a5338a90 2024-04-26T21:44:17.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.497+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v111'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.497+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.498+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v111'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.498+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.500+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v111'152 uv146 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.500+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.501+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v111'153 uv153 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.501+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533b8f0 con 0x5562a5338a90 2024-04-26T21:44:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.501+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v111'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.502 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.501+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5334060 con 0x5562a5338a90 2024-04-26T21:44:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.502+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v111'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.502+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5334060 con 0x5562a5338a90 2024-04-26T21:44:17.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.503+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v111'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.503+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5334060 con 0x5562a5338a90 2024-04-26T21:44:17.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.504+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v111'157 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.504+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a5334060 con 0x5562a5338a90 2024-04-26T21:44:17.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.505+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v111'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.505+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5334060 con 0x5562a5338a90 2024-04-26T21:44:17.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.506+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v111'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.507 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:17.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.506+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a5334060 con 0x5562a5338a90 2024-04-26T21:44:17.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.507+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v111'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.507+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533e7d0 con 0x5562a5338a90 2024-04-26T21:44:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.508+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v111'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.508+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5562a533be00 0x5562a533e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.508+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ee90 con 0x5562a533be00 2024-04-26T21:44:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.508+0000 7f61cf782640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5562a533be00 0x5562a533e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.509+0000 7f61cf782640 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5562a533be00 0x5562a533e1f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.510+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v111'70 uv69 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a533be00 2024-04-26T21:44:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.510+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533ee90 con 0x5562a533be00 2024-04-26T21:44:17.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.511+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v111'71 uv71 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a533be00 2024-04-26T21:44:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.511+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ee90 con 0x5562a533be00 2024-04-26T21:44:17.512 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.512+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v111'72 uv72 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a533be00 2024-04-26T21:44:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.512+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a533be00 2024-04-26T21:44:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.513+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v111'73 uv73 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a533be00 2024-04-26T21:44:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.513+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a5338a90 2024-04-26T21:44:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.514+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v111'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.514+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a5338a90 2024-04-26T21:44:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.515+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v111'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.515+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a5338a90 2024-04-26T21:44:17.517 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.516+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v111'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.516+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a5338a90 2024-04-26T21:44:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.517+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v111'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.517+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a5338a90 2024-04-26T21:44:17.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.518+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v111'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.518+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a5338a90 2024-04-26T21:44:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.520+0000 7f61cff83640 1 -- 172.21.15.40:0/1915552056 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v111'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61c000f050 con 0x5562a5338a90 2024-04-26T21:44:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.520+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.521+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v111'79 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.521+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.523+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v111'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.523+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.524 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.523+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v111'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.523+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533f2e0 con 0x5562a53307b0 2024-04-26T21:44:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.524+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v111'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.524+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.525+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v111'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.525+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.526+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v111'192 uv186 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.526+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.527+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v111'193 uv193 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.527+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.528+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v111'194 uv194 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.529 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.528+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.529+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v111'195 uv195 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.529+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.530+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v111'196 uv196 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.530+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.532+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v111'197 uv191 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.532+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.533+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v111'198 uv198 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.533+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef10 con 0x5562a53307b0 2024-04-26T21:44:17.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.533+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v111'199 uv199 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.534 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:17.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.534+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533ef50 con 0x5562a53307b0 2024-04-26T21:44:17.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.534+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v111'200 uv200 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.534+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533f430 con 0x5562a53307b0 2024-04-26T21:44:17.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.535+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v111'201 uv201 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.535+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533f430 con 0x5562a53307b0 2024-04-26T21:44:17.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.536+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v111'202 uv202 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.536+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5562a533f880 con 0x5562a53307b0 2024-04-26T21:44:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.537+0000 7f61cf782640 1 -- 172.21.15.40:0/1915552056 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v111'203 uv203 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f61b8027040 con 0x5562a53307b0 2024-04-26T21:44:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.537+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5562a532bb60 msgr2=0x5562a532df50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.537+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5562a532bb60 0x5562a532df50 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5562a5274270 msgr2=0x5562a52818f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5562a5274270 0x5562a52818f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5562a5334d70 msgr2=0x5562a5337bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5562a5334d70 0x5562a5337bd0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5562a533be00 msgr2=0x5562a533e1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5562a533be00 0x5562a533e1f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562a53307b0 msgr2=0x5562a5332ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5562a53307b0 0x5562a5332ba0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5562a5338a90 msgr2=0x5562a533ae80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5562a5338a90 0x5562a533ae80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 reap_dead start 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 reap_dead start 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d0784640 1 -- 172.21.15.40:0/1915552056 reap_dead start 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61b006b2d0 msgr2=0x7f61b006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61b006b2d0 0x7f61b006d790 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f61b800bd60 tx=0x7f61b800d000 comp rx=0 tx=0).stop 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a51f9420 msgr2=0x5562a51f3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a51f9420 0x5562a51f3c30 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f61c0002190 tx=0x7f61c000b570 comp rx=0 tx=0).stop 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 shutdown_connections 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61b006b2d0 0x7f61b006d790 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 --2- 172.21.15.40:0/1915552056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5562a51f9420 0x5562a51f3c30 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 >> 172.21.15.40:0/1915552056 conn(0x5562a51d98a0 msgr2=0x5562a51ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 shutdown_connections 2024-04-26T21:44:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.538+0000 7f61d2487f40 1 -- 172.21.15.40:0/1915552056 wait complete. 2024-04-26T21:44:17.543 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000000 - 2024-04-26T21:44:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88f128f40 1 Processor -- start 2024-04-26T21:44:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88f128f40 1 -- start start 2024-04-26T21:44:17.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88f128f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a71348f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88f128f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a7133bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88f128f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 0x55a7133bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88f128f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a713492c00 con 0x55a7133ed480 2024-04-26T21:44:17.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88f128f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a713364e40 con 0x55a7134066e0 2024-04-26T21:44:17.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88f128f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a7133ed8a0 con 0x55a713492820 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88cc24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a71348f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88cc24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a71348f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55388/0 (socket says 172.21.15.40:55388) 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.625+0000 7fc88cc24640 1 -- 172.21.15.40:0/3531086989 learned_addr learned my addr 172.21.15.40:0/3531086989 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88d425640 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 0x55a7133bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88c423640 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a7133bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88cc24640 1 -- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 msgr2=0x55a7133bec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88cc24640 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 0x55a7133bec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88cc24640 1 -- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 msgr2=0x55a7133bc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88cc24640 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a7133bc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88cc24640 1 -- 172.21.15.40:0/3531086989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a713485b70 con 0x55a7133ed480 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88c423640 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a7133bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88cc24640 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a71348f4b0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc88400a770 tx=0x7fc88400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957d48b6526ecd23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88b421640 1 -- 172.21.15.40:0/3531086989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884019070 con 0x55a7133ed480 2024-04-26T21:44:17.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88b421640 1 -- 172.21.15.40:0/3531086989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc884007600 con 0x55a7133ed480 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88b421640 1 -- 172.21.15.40:0/3531086989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8840079d0 con 0x55a7133ed480 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88f128f40 1 -- 172.21.15.40:0/3531086989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 msgr2=0x55a71348f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88f128f40 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a71348f4b0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc88400a770 tx=0x7fc88400a860 comp rx=0 tx=0).stop 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88f128f40 1 -- 172.21.15.40:0/3531086989 shutdown_connections 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88f128f40 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 0x55a7133bec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88f128f40 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a7133bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88f128f40 1 --2- 172.21.15.40:0/3531086989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a71348f4b0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.626+0000 7fc88f128f40 1 -- 172.21.15.40:0/3531086989 >> 172.21.15.40:0/3531086989 conn(0x55a7133e78a0 msgr2=0x55a7133be570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.627+0000 7fc88f128f40 1 -- 172.21.15.40:0/3531086989 shutdown_connections 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.627+0000 7fc88f128f40 1 -- 172.21.15.40:0/3531086989 wait complete. 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.627+0000 7fc88f128f40 1 Processor -- start 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.627+0000 7fc88f128f40 1 -- start start 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88f128f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a713485220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88f128f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a713483130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88f128f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 0x55a71352c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88f128f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a713404060 con 0x55a7133ed480 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88f128f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a71348f9f0 con 0x55a7134066e0 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88f128f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a713403b80 con 0x55a713492820 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a713485220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a713485220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55396/0 (socket says 172.21.15.40:55396) 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 -- 172.21.15.40:0/3196320714 learned_addr learned my addr 172.21.15.40:0/3196320714 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88d425640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 0x55a71352c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88c423640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a713483130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 -- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 msgr2=0x55a71352c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a713492820 0x55a71352c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 -- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 msgr2=0x55a713483130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a713483130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 -- 172.21.15.40:0/3196320714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a713481490 con 0x55a7133ed480 2024-04-26T21:44:17.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88c423640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7134066e0 0x55a713483130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:17.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.628+0000 7fc88cc24640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a713485220 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fc88400ad60 tx=0x7fc884003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.629+0000 7fc88a41f640 1 -- 172.21.15.40:0/3196320714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884019040 con 0x55a7133ed480 2024-04-26T21:44:17.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.629+0000 7fc88a41f640 1 -- 172.21.15.40:0/3196320714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc884022070 con 0x55a7133ed480 2024-04-26T21:44:17.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.629+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a713490470 con 0x55a7133ed480 2024-04-26T21:44:17.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.629+0000 7fc88a41f640 1 -- 172.21.15.40:0/3196320714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884014470 con 0x55a7133ed480 2024-04-26T21:44:17.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.629+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a713494fd0 con 0x55a7133ed480 2024-04-26T21:44:17.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.630+0000 7fc88a41f640 1 -- 172.21.15.40:0/3196320714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc884014610 con 0x55a7133ed480 2024-04-26T21:44:17.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.630+0000 7fc88a41f640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc86c06b2d0 0x7fc86c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.630+0000 7fc88a41f640 1 -- 172.21.15.40:0/3196320714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7593+0+0 (secure 0 0 0) 0x7fc88408bb00 con 0x55a7133ed480 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.630+0000 7fc88c423640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc86c06b2d0 0x7fc86c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.631+0000 7fc88f128f40 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a713483670 0x55a7135222b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.631+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55a713481940 con 0x55a713483670 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.631+0000 7fc88d425640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a713483670 0x55a7135222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.631+0000 7fc88c423640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc86c06b2d0 0x7fc86c06d790 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55a71348ea40 tx=0x7fc874003660 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.631+0000 7fc88d425640 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a713483670 0x55a7135222b0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.632+0000 7fc88d425640 1 -- 172.21.15.40:0/3196320714 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000000 [writefull 0~80] v111'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc87c051940 con 0x55a713483670 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a713483670 msgr2=0x55a7135222b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a713483670 0x55a7135222b0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc86c06b2d0 msgr2=0x7fc86c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc86c06b2d0 0x7fc86c06d790 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55a71348ea40 tx=0x7fc874003660 comp rx=0 tx=0).stop 2024-04-26T21:44:17.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 msgr2=0x55a713485220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 --2- 172.21.15.40:0/3196320714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a7133ed480 0x55a713485220 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fc88400ad60 tx=0x7fc884003020 comp rx=0 tx=0).stop 2024-04-26T21:44:17.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88d425640 1 -- 172.21.15.40:0/3196320714 reap_dead start 2024-04-26T21:44:17.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 shutdown_connections 2024-04-26T21:44:17.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 >> 172.21.15.40:0/3196320714 conn(0x55a7133e78a0 msgr2=0x55a713485f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:17.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 shutdown_connections 2024-04-26T21:44:17.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.633+0000 7fc88f128f40 1 -- 172.21.15.40:0/3196320714 wait complete. 2024-04-26T21:44:17.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:17.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.779+0000 7fa092b4d640 1 Processor -- start 2024-04-26T21:44:17.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.779+0000 7fa092b4d640 1 -- start start 2024-04-26T21:44:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa092b4d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa092b4d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa08c15f550 0x7fa08c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa092b4d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa08c0d34b0 0x7fa08c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa092b4d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa08c0c9430 con 0x7fa08c15f550 2024-04-26T21:44:17.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa092b4d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa08c0d3e20 con 0x7fa08c0d34b0 2024-04-26T21:44:17.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa092b4d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c0d1040 con 0x7fa08c15f170 2024-04-26T21:44:17.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa08bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa08c15f550 0x7fa08c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa08bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa08c15f550 0x7fa08c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55410/0 (socket says 172.21.15.40:55410) 2024-04-26T21:44:17.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa08bfff640 1 -- 172.21.15.40:0/4104342906 learned_addr learned my addr 172.21.15.40:0/4104342906 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:17.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa0910ea640 1 --2- 172.21.15.40:0/4104342906 >> v2:172.21.15.184:3300/0 conn(0x7fa08c0d34b0 0x7fa08c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa0908e9640 1 --2- 172.21.15.40:0/4104342906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa08bfff640 1 -- 172.21.15.40:0/4104342906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 msgr2=0x7fa08c0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa08bfff640 1 --2- 172.21.15.40:0/4104342906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa08bfff640 1 -- 172.21.15.40:0/4104342906 >> v2:172.21.15.184:3300/0 conn(0x7fa08c0d34b0 msgr2=0x7fa08c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa08bfff640 1 --2- 172.21.15.40:0/4104342906 >> v2:172.21.15.184:3300/0 conn(0x7fa08c0d34b0 0x7fa08c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa08bfff640 1 -- 172.21.15.40:0/4104342906 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c0c9680 con 0x7fa08c15f550 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.780+0000 7fa0908e9640 1 --2- 172.21.15.40:0/4104342906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.781+0000 7fa08bfff640 1 --2- 172.21.15.40:0/4104342906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c15f550 0x7fa08c0d2cb0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa08c08f770 tx=0x7fa08002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f384dab56446ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.781+0000 7fa08b7fe640 1 -- 172.21.15.40:0/4104342906 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080037070 con 0x7fa08c15f550 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.781+0000 7fa08b7fe640 1 -- 172.21.15.40:0/4104342906 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa080032440 con 0x7fa08c15f550 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.781+0000 7fa08b7fe640 1 -- 172.21.15.40:0/4104342906 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080032810 con 0x7fa08c15f550 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 -- 172.21.15.40:0/4104342906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c15f550 msgr2=0x7fa08c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 --2- 172.21.15.40:0/4104342906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c15f550 0x7fa08c0d2cb0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa08c08f770 tx=0x7fa08002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 -- 172.21.15.40:0/4104342906 shutdown_connections 2024-04-26T21:44:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 --2- 172.21.15.40:0/4104342906 >> v2:172.21.15.184:3300/0 conn(0x7fa08c0d34b0 0x7fa08c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 --2- 172.21.15.40:0/4104342906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c15f550 0x7fa08c0d2cb0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 --2- 172.21.15.40:0/4104342906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 -- 172.21.15.40:0/4104342906 >> 172.21.15.40:0/4104342906 conn(0x7fa08c0d4f60 msgr2=0x7fa08c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 -- 172.21.15.40:0/4104342906 shutdown_connections 2024-04-26T21:44:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.782+0000 7fa092b4d640 1 -- 172.21.15.40:0/4104342906 wait complete. 2024-04-26T21:44:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.783+0000 7fa092b4d640 1 Processor -- start 2024-04-26T21:44:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.783+0000 7fa092b4d640 1 -- start start 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa092b4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c0d34b0 0x7fa08c150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa092b4d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa092b4d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c151720 0x7fa08c154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa092b4d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0cfc50 con 0x7fa08c0d34b0 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa092b4d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0c9c40 con 0x7fa08c151720 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa092b4d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c0cfdd0 con 0x7fa08c15f170 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0908e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c0d34b0 0x7fa08c150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0908e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c0d34b0 0x7fa08c150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55412/0 (socket says 172.21.15.40:55412) 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0908e9640 1 -- 172.21.15.40:0/3479380947 learned_addr learned my addr 172.21.15.40:0/3479380947 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0910ea640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c151720 0x7fa08c154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa08bfff640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0908e9640 1 -- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 msgr2=0x7fa08c1540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0908e9640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c1540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0908e9640 1 -- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c151720 msgr2=0x7fa08c154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0908e9640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c151720 0x7fa08c154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.784+0000 7fa0908e9640 1 -- 172.21.15.40:0/3479380947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c151be0 con 0x7fa08c0d34b0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.785+0000 7fa0908e9640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c0d34b0 0x7fa08c150f70 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fa084007fa0 tx=0x7fa08400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.785+0000 7fa0897fa640 1 -- 172.21.15.40:0/3479380947 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08401a070 con 0x7fa08c0d34b0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.785+0000 7fa0897fa640 1 -- 172.21.15.40:0/3479380947 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa084003b90 con 0x7fa08c0d34b0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.785+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c151ec0 con 0x7fa08c0d34b0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.785+0000 7fa0897fa640 1 -- 172.21.15.40:0/3479380947 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084004030 con 0x7fa08c0d34b0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.785+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c152480 con 0x7fa08c0d34b0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.786+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c0694f0 con 0x7fa08c0d34b0 2024-04-26T21:44:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.786+0000 7fa0897fa640 1 -- 172.21.15.40:0/3479380947 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa08401d070 con 0x7fa08c0d34b0 2024-04-26T21:44:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.786+0000 7fa0897fa640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa07806c8f0 0x7fa07806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.787+0000 7fa0897fa640 1 -- 172.21.15.40:0/3479380947 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7593+0+0 (secure 0 0 0) 0x7fa08408c4e0 con 0x7fa08c0d34b0 2024-04-26T21:44:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.787+0000 7fa08bfff640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa07806c8f0 0x7fa07806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.787+0000 7fa08bfff640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa07806c8f0 0x7fa07806edb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa080034000 tx=0x7fa08002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:17.791+0000 7fa0897fa640 1 -- 172.21.15.40:0/3479380947 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa084010070 con 0x7fa08c0d34b0 2024-04-26T21:44:18.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.017+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa08c0d5a60 con 0x7fa08c0d34b0 2024-04-26T21:44:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.333+0000 7fa0897fa640 1 -- 172.21.15.40:0/3479380947 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v193) ==== 162+0+0 (secure 0 0 0) 0x7fa084059b60 con 0x7fa08c0d34b0 2024-04-26T21:44:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.335+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa07806c8f0 msgr2=0x7fa07806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.335+0000 7fa092b4d640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa07806c8f0 0x7fa07806edb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa080034000 tx=0x7fa08002e000 comp rx=0 tx=0).stop 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.335+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c0d34b0 msgr2=0x7fa08c150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.335+0000 7fa092b4d640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c0d34b0 0x7fa08c150f70 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fa084007fa0 tx=0x7fa08400d560 comp rx=0 tx=0).stop 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.336+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 shutdown_connections 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.336+0000 7fa092b4d640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa07806c8f0 0x7fa07806edb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.336+0000 7fa092b4d640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c151720 0x7fa08c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.336+0000 7fa092b4d640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa08c15f170 0x7fa08c1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.336+0000 7fa092b4d640 1 --2- 172.21.15.40:0/3479380947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c0d34b0 0x7fa08c150f70 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.336+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 >> 172.21.15.40:0/3479380947 conn(0x7fa08c0d4f60 msgr2=0x7fa08c06e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.336+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 shutdown_connections 2024-04-26T21:44:18.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.336+0000 7fa092b4d640 1 -- 172.21.15.40:0/3479380947 wait complete. 2024-04-26T21:44:18.345 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:18.344+0000 7f2f25a00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-04-26T21:44:18.346 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:18.345+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:18.346 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:18.345+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:18.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:18.352 INFO:tasks.ceph.mds.c.smithi040.stderr: -18> 2024-04-26T21:44:18.344+0000 7f2f25a00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-04-26T21:44:18.376 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:18.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.453+0000 7f8ebdd1c640 1 Processor -- start 2024-04-26T21:44:18.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.453+0000 7f8ebdd1c640 1 -- start start 2024-04-26T21:44:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8ebdd1c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8eb8070790 0x7f8eb806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8ebdd1c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb806e6b0 con 0x7f8eb806e2a0 2024-04-26T21:44:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8ebdd1c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb8070c10 con 0x7f8eb80d27d0 2024-04-26T21:44:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8ebdd1c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb8070ef0 con 0x7f8eb8070790 2024-04-26T21:44:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8eb6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8eb8070790 0x7f8eb806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8eb7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8eb806e2a0 0x7f8eb815ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48880/0 (socket says 172.21.15.40:48880) 2024-04-26T21:44:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8eb7fff640 1 -- 172.21.15.40:0/4030577680 learned_addr learned my addr 172.21.15.40:0/4030577680 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8eb6ffd640 1 -- 172.21.15.40:0/4030577680 >> v1:172.21.15.184:6789/0 conn(0x7f8eb80d27d0 legacy=0x7f8eb80d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8eb6ffd640 1 -- 172.21.15.40:0/4030577680 >> v1:172.21.15.40:6789/0 conn(0x7f8eb806e2a0 legacy=0x7f8eb815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.454+0000 7f8eb6ffd640 1 -- 172.21.15.40:0/4030577680 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb80711d0 con 0x7f8eb8070790 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.455+0000 7f8eb6ffd640 1 --2- 172.21.15.40:0/4030577680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb8070790 0x7f8eb806dd60 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0097a0 tx=0x7f8eac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbec71fe50dbd05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.455+0000 7f8eb67fc640 1 -- 172.21.15.40:0/4030577680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac037070 con 0x7f8eb8070790 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.455+0000 7f8eb67fc640 1 -- 172.21.15.40:0/4030577680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac032440 con 0x7f8eb8070790 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.455+0000 7f8eb67fc640 1 -- 172.21.15.40:0/4030577680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac032810 con 0x7f8eb8070790 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.456+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/4030577680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb8070790 msgr2=0x7f8eb806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.456+0000 7f8ebdd1c640 1 --2- 172.21.15.40:0/4030577680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb8070790 0x7f8eb806dd60 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0097a0 tx=0x7f8eac02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.456+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/4030577680 shutdown_connections 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.456+0000 7f8ebdd1c640 1 --2- 172.21.15.40:0/4030577680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb8070790 0x7f8eb806dd60 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.456+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/4030577680 >> 172.21.15.40:0/4030577680 conn(0x7f8eb80d2e00 msgr2=0x7f8eb80d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.456+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/4030577680 shutdown_connections 2024-04-26T21:44:18.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.456+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/4030577680 wait complete. 2024-04-26T21:44:18.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.457+0000 7f8ebdd1c640 1 Processor -- start 2024-04-26T21:44:18.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.457+0000 7f8ebdd1c640 1 -- start start 2024-04-26T21:44:18.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8ebdd1c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb806e2a0 0x7f8eb8151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8ebdd1c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb8070790 0x7f8eb80c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8ebdd1c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb80d27d0 0x7f8eb80c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8ebdd1c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb80d2bb0 con 0x7f8eb806e2a0 2024-04-26T21:44:18.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8ebdd1c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8071790 con 0x7f8eb8070790 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8ebdd1c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb806d790 con 0x7f8eb80d27d0 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb806e2a0 0x7f8eb8151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb806e2a0 0x7f8eb8151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55442/0 (socket says 172.21.15.40:55442) 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 -- 172.21.15.40:0/2709466306 learned_addr learned my addr 172.21.15.40:0/2709466306 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb7fff640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb80d27d0 0x7f8eb80c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb6ffd640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb8070790 0x7f8eb80c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 -- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb80d27d0 msgr2=0x7f8eb80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb80d27d0 0x7f8eb80c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 -- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb8070790 msgr2=0x7f8eb80c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb8070790 0x7f8eb80c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 -- 172.21.15.40:0/2709466306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb80ccfe0 con 0x7f8eb806e2a0 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb6ffd640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb8070790 0x7f8eb80c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:18.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8eb77fe640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb806e2a0 0x7f8eb8151130 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c430 tx=0x7f8ea800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8e9ffff640 1 -- 172.21.15.40:0/2709466306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8011790 con 0x7f8eb806e2a0 2024-04-26T21:44:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8e9ffff640 1 -- 172.21.15.40:0/2709466306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea8017070 con 0x7f8eb806e2a0 2024-04-26T21:44:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.458+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb80c9260 con 0x7f8eb806e2a0 2024-04-26T21:44:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.459+0000 7f8e9ffff640 1 -- 172.21.15.40:0/2709466306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8011bb0 con 0x7f8eb806e2a0 2024-04-26T21:44:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.459+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb80c9820 con 0x7f8eb806e2a0 2024-04-26T21:44:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.459+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb80694f0 con 0x7f8eb806e2a0 2024-04-26T21:44:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.460+0000 7f8e9ffff640 1 -- 172.21.15.40:0/2709466306 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ea801c460 con 0x7f8eb806e2a0 2024-04-26T21:44:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.460+0000 7f8e9ffff640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ea406c8f0 0x7f8ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.460+0000 7f8e9ffff640 1 -- 172.21.15.40:0/2709466306 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7679+0+0 (secure 0 0 0) 0x7f8ea808c1a0 con 0x7f8eb806e2a0 2024-04-26T21:44:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.461+0000 7f8eb6ffd640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ea406c8f0 0x7f8ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.461+0000 7f8eb6ffd640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ea406c8f0 0x7f8ea406edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8eac034000 tx=0x7f8eac02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.465+0000 7f8e9ffff640 1 -- 172.21.15.40:0/2709466306 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ea8059e20 con 0x7f8eb806e2a0 2024-04-26T21:44:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.691+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8eb80c66e0 con 0x7f8eb806e2a0 2024-04-26T21:44:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.691+0000 7f8e9ffff640 1 -- 172.21.15.40:0/2709466306 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) ==== 95+0+4355 (secure 0 0 0) 0x7f8ea8059720 con 0x7f8eb806e2a0 2024-04-26T21:44:18.692 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:18.692 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":195,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:18.346760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 195 2024-04-26T21:44:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ea406c8f0 msgr2=0x7f8ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ea406c8f0 0x7f8ea406edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8eac034000 tx=0x7f8eac02e000 comp rx=0 tx=0).stop 2024-04-26T21:44:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb806e2a0 msgr2=0x7f8eb8151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb806e2a0 0x7f8eb8151130 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c430 tx=0x7f8ea800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 shutdown_connections 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ea406c8f0 0x7f8ea406edb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb80d27d0 0x7f8eb80c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb8070790 0x7f8eb80c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 --2- 172.21.15.40:0/2709466306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8eb806e2a0 0x7f8eb8151130 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 >> 172.21.15.40:0/2709466306 conn(0x7f8eb80d2e00 msgr2=0x7f8eb80cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 shutdown_connections 2024-04-26T21:44:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.693+0000 7f8ebdd1c640 1 -- 172.21.15.40:0/2709466306 wait complete. 2024-04-26T21:44:18.705 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:18.705 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.844+0000 7f4982378640 1 Processor -- start 2024-04-26T21:44:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.844+0000 7f4982378640 1 -- start start 2024-04-26T21:44:18.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.845+0000 7f4982378640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f497c15f3a0 0x7f497c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.845+0000 7f4982378640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c0d0e20 con 0x7f497c0d2300 2024-04-26T21:44:18.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.845+0000 7f4982378640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c0b1d70 con 0x7f497c0d1770 2024-04-26T21:44:18.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.845+0000 7f4982378640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0b2050 con 0x7f497c15f3a0 2024-04-26T21:44:18.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.845+0000 7f4980915640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f497c0d2300 0x7f497c0b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59308/0 (socket says 172.21.15.40:59308) 2024-04-26T21:44:18.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.845+0000 7f4980915640 1 -- 172.21.15.40:0/1871877454 learned_addr learned my addr 172.21.15.40:0/1871877454 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:18.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497bfff640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f497c15f3a0 0x7f497c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 155549627 0 0) 0x7f497c0d0e20 con 0x7f497c0d2300 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4968003520 con 0x7f497c0d2300 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1298566636 0 0) 0x7f4968003520 con 0x7f497c0d2300 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f497c15f3a0 msgr2=0x7f497c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f497c15f3a0 0x7f497c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 >> v1:172.21.15.184:6789/0 conn(0x7f497c0d1770 legacy=0x7f497c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c0b2330 con 0x7f497c0d2300 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f49740019f0 con 0x7f497c0d2300 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 >> v1:172.21.15.40:6789/0 conn(0x7f497c0d2300 legacy=0x7f497c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f497c15f3a0 0x7f49680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4968003f60 0x7f4968004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4968004d40 0x7f49680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f497c0b2050 con 0x7f4968003f60 2024-04-26T21:44:18.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f496800baa0 con 0x7f4968004d40 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f496800bca0 con 0x7f497c15f3a0 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497b7fe640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4968003f60 0x7f4968004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f497bfff640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f497c15f3a0 0x7f49680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.846+0000 7f4980915640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4968004d40 0x7f49680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497b7fe640 1 -- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f497c15f3a0 msgr2=0x7f49680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497b7fe640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f497c15f3a0 0x7f49680039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497b7fe640 1 -- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4968004d40 msgr2=0x7f49680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497b7fe640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4968004d40 0x7f49680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497b7fe640 1 -- 172.21.15.40:0/1871877454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4968003520 con 0x7f4968003f60 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497b7fe640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4968003f60 0x7f4968004800 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f496c001750 tx=0x7f4968005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5075c89b87e76d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c001370 con 0x7f4968003f60 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496c0052d0 con 0x7f4968003f60 2024-04-26T21:44:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.847+0000 7f497affd640 1 -- 172.21.15.40:0/1871877454 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c02abe0 con 0x7f4968003f60 2024-04-26T21:44:18.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.848+0000 7f4982378640 1 -- 172.21.15.40:0/1871877454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4968003f60 msgr2=0x7f4968004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.848+0000 7f4982378640 1 --2- 172.21.15.40:0/1871877454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4968003f60 0x7f4968004800 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f496c001750 tx=0x7f4968005830 comp rx=0 tx=0).stop 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.848+0000 7f4980915640 1 -- 172.21.15.40:0/1871877454 reap_dead start 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.848+0000 7f4982378640 1 -- 172.21.15.40:0/1871877454 shutdown_connections 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.848+0000 7f4982378640 1 -- 172.21.15.40:0/1871877454 >> 172.21.15.40:0/1871877454 conn(0x7f497c0d4f90 msgr2=0x7f497c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.848+0000 7f4982378640 1 -- 172.21.15.40:0/1871877454 shutdown_connections 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.848+0000 7f4982378640 1 -- 172.21.15.40:0/1871877454 wait complete. 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.849+0000 7f4982378640 1 Processor -- start 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.849+0000 7f4982378640 1 -- start start 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4982378640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4968003f60 0x7f497c1f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4982378640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f497c0c9300 0x7f497c06de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4982378640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f497c0c9780 0x7f497c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4982378640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f497c0d0870 con 0x7f497c0c9780 2024-04-26T21:44:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4982378640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f497c0d06f0 con 0x7f497c0c9300 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4982378640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0d09f0 con 0x7f4968003f60 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4980915640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f497c0c9780 0x7f497c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4980915640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f497c0c9780 0x7f497c06e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42918/0 (socket says 172.21.15.40:42918) 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f4980915640 1 -- 172.21.15.40:0/966477673 learned_addr learned my addr 172.21.15.40:0/966477673 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f497bfff640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4968003f60 0x7f497c1f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.850+0000 7f497b7fe640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f497c0c9300 0x7f497c06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4980915640 1 -- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4968003f60 msgr2=0x7f497c1f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4980915640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4968003f60 0x7f497c1f4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4980915640 1 -- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f497c0c9300 msgr2=0x7f497c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4980915640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f497c0c9300 0x7f497c06de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4980915640 1 -- 172.21.15.40:0/966477673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c0c9da0 con 0x7f497c0c9780 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f497bfff640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4968003f60 0x7f497c1f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f497b7fe640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f497c0c9300 0x7f497c06de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4980915640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f497c0c9780 0x7f497c06e340 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f497c0d2ae0 tx=0x7f4974007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4978ff9640 1 -- 172.21.15.40:0/966477673 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497400ab60 con 0x7f497c0c9780 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4978ff9640 1 -- 172.21.15.40:0/966477673 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4974019070 con 0x7f497c0c9780 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497c0ca060 con 0x7f497c0c9780 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4978ff9640 1 -- 172.21.15.40:0/966477673 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740133f0 con 0x7f497c0c9780 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.851+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497c0b86b0 con 0x7f497c0c9780 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.852+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f497c0694f0 con 0x7f497c0c9780 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.852+0000 7f4978ff9640 1 -- 172.21.15.40:0/966477673 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4974013590 con 0x7f497c0c9780 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.853+0000 7f4978ff9640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f495c06c8f0 0x7f495c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.853+0000 7f4978ff9640 1 -- 172.21.15.40:0/966477673 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7679+0+0 (secure 0 0 0) 0x7f497408c8f0 con 0x7f497c0c9780 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.853+0000 7f497bfff640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f495c06c8f0 0x7f495c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.854+0000 7f497bfff640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f495c06c8f0 0x7f495c06edb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4964009bc0 tx=0x7f4964008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:18.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:18.857+0000 7f4978ff9640 1 -- 172.21.15.40:0/966477673 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f497405a570 con 0x7f497c0c9780 2024-04-26T21:44:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.082+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f497c0b6860 con 0x7f497c0c9780 2024-04-26T21:44:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.082+0000 7f4978ff9640 1 -- 172.21.15.40:0/966477673 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) ==== 95+0+4355 (secure 0 0 0) 0x7f4974059e70 con 0x7f497c0c9780 2024-04-26T21:44:19.083 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":195,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:18.346760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:19.084 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 195 2024-04-26T21:44:19.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f495c06c8f0 msgr2=0x7f495c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f495c06c8f0 0x7f495c06edb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4964009bc0 tx=0x7f4964008040 comp rx=0 tx=0).stop 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f497c0c9780 msgr2=0x7f497c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f497c0c9780 0x7f497c06e340 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f497c0d2ae0 tx=0x7f4974007b20 comp rx=0 tx=0).stop 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 shutdown_connections 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f495c06c8f0 0x7f495c06edb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f497c0c9780 0x7f497c06e340 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f497c0c9300 0x7f497c06de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 --2- 172.21.15.40:0/966477673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4968003f60 0x7f497c1f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.084+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 >> 172.21.15.40:0/966477673 conn(0x7f497c0d4f90 msgr2=0x7f497c0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.085+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 shutdown_connections 2024-04-26T21:44:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.085+0000 7f4982378640 1 -- 172.21.15.40:0/966477673 wait complete. 2024-04-26T21:44:19.096 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:19.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.195+0000 7fbb24b24640 1 Processor -- start 2024-04-26T21:44:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.196+0000 7fbb24b24640 1 -- start start 2024-04-26T21:44:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.196+0000 7fbb24b24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.196+0000 7fbb24b24640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbb20072700 0x7fbb2006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.196+0000 7fbb24b24640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb200700e0 con 0x7fbb2006fcd0 2024-04-26T21:44:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb24b24640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbb20074010 con 0x7fbb20072700 2024-04-26T21:44:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb24b24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb20072f70 con 0x7fbb2015f2d0 2024-04-26T21:44:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb1e59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb1e59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:36290/0 (socket says 172.21.15.40:36290) 2024-04-26T21:44:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb1dd9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbb20072700 0x7fbb2006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb1e59c640 1 -- 172.21.15.40:0/4153549052 learned_addr learned my addr 172.21.15.40:0/4153549052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb1e59c640 1 -- 172.21.15.40:0/4153549052 >> v2:172.21.15.184:3300/0 conn(0x7fbb20072700 msgr2=0x7fbb2006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb1e59c640 1 --2- 172.21.15.40:0/4153549052 >> v2:172.21.15.184:3300/0 conn(0x7fbb20072700 0x7fbb2006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb1e59c640 1 -- 172.21.15.40:0/4153549052 >> v1:172.21.15.184:6789/0 conn(0x7fbb2006fcd0 legacy=0x7fbb200cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.197+0000 7fbb1e59c640 1 -- 172.21.15.40:0/4153549052 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb20073250 con 0x7fbb2015f2d0 2024-04-26T21:44:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.198+0000 7fbb1e59c640 1 --2- 172.21.15.40:0/4153549052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201604e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009e50 tx=0x7fbb1002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e162e6d4c5917a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.198+0000 7fbb1d59a640 1 -- 172.21.15.40:0/4153549052 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10036070 con 0x7fbb2015f2d0 2024-04-26T21:44:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.198+0000 7fbb1d59a640 1 -- 172.21.15.40:0/4153549052 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb10031600 con 0x7fbb2015f2d0 2024-04-26T21:44:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.198+0000 7fbb1d59a640 1 -- 172.21.15.40:0/4153549052 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100319d0 con 0x7fbb2015f2d0 2024-04-26T21:44:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.199+0000 7fbb24b24640 1 -- 172.21.15.40:0/4153549052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 msgr2=0x7fbb201604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.199+0000 7fbb24b24640 1 --2- 172.21.15.40:0/4153549052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201604e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009e50 tx=0x7fbb1002a710 comp rx=0 tx=0).stop 2024-04-26T21:44:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.199+0000 7fbb24b24640 1 -- 172.21.15.40:0/4153549052 shutdown_connections 2024-04-26T21:44:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.199+0000 7fbb24b24640 1 --2- 172.21.15.40:0/4153549052 >> v2:172.21.15.184:3300/0 conn(0x7fbb20072700 0x7fbb2006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.199+0000 7fbb24b24640 1 --2- 172.21.15.40:0/4153549052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201604e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.199+0000 7fbb24b24640 1 -- 172.21.15.40:0/4153549052 >> 172.21.15.40:0/4153549052 conn(0x7fbb200d0be0 msgr2=0x7fbb200d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.199+0000 7fbb24b24640 1 -- 172.21.15.40:0/4153549052 shutdown_connections 2024-04-26T21:44:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.199+0000 7fbb24b24640 1 -- 172.21.15.40:0/4153549052 wait complete. 2024-04-26T21:44:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.200+0000 7fbb24b24640 1 Processor -- start 2024-04-26T21:44:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.200+0000 7fbb24b24640 1 -- start start 2024-04-26T21:44:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb24b24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2006fcd0 0x7fbb200cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb24b24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20072700 0x7fbb20154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb24b24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb24b24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20073af0 con 0x7fbb2006fcd0 2024-04-26T21:44:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb24b24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20073810 con 0x7fbb20072700 2024-04-26T21:44:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb24b24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2015f6b0 con 0x7fbb2015f2d0 2024-04-26T21:44:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2006fcd0 0x7fbb200cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2006fcd0 0x7fbb200cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42934/0 (socket says 172.21.15.40:42934) 2024-04-26T21:44:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1e59c640 1 -- 172.21.15.40:0/811053631 learned_addr learned my addr 172.21.15.40:0/811053631 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1dd9b640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20072700 0x7fbb20154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1ed9d640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1e59c640 1 -- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 msgr2=0x7fbb201e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1e59c640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1e59c640 1 -- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20072700 msgr2=0x7fbb20154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1e59c640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20072700 0x7fbb20154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.201+0000 7fbb1e59c640 1 -- 172.21.15.40:0/811053631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb200cf6e0 con 0x7fbb2006fcd0 2024-04-26T21:44:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.202+0000 7fbb1e59c640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2006fcd0 0x7fbb200cf1a0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fbb10034040 tx=0x7fbb10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.202+0000 7fbb0effd640 1 -- 172.21.15.40:0/811053631 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10036050 con 0x7fbb2006fcd0 2024-04-26T21:44:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.202+0000 7fbb0effd640 1 -- 172.21.15.40:0/811053631 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb10042070 con 0x7fbb2006fcd0 2024-04-26T21:44:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.202+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2015d280 con 0x7fbb2006fcd0 2024-04-26T21:44:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.202+0000 7fbb0effd640 1 -- 172.21.15.40:0/811053631 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10009510 con 0x7fbb2006fcd0 2024-04-26T21:44:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.202+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2015d840 con 0x7fbb2006fcd0 2024-04-26T21:44:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.203+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb200694f0 con 0x7fbb2006fcd0 2024-04-26T21:44:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.203+0000 7fbb0effd640 1 -- 172.21.15.40:0/811053631 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb10007c70 con 0x7fbb2006fcd0 2024-04-26T21:44:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.204+0000 7fbb0effd640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb0006c8f0 0x7fbb0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.204+0000 7fbb0effd640 1 -- 172.21.15.40:0/811053631 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7679+0+0 (secure 0 0 0) 0x7fbb100acfc0 con 0x7fbb2006fcd0 2024-04-26T21:44:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.204+0000 7fbb1dd9b640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb0006c8f0 0x7fbb0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.204+0000 7fbb1dd9b640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb0006c8f0 0x7fbb0006edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbb2015e490 tx=0x7fbb14009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:19.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.208+0000 7fbb0effd640 1 -- 172.21.15.40:0/811053631 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb1007a5f0 con 0x7fbb2006fcd0 2024-04-26T21:44:19.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.433+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb200b3d90 con 0x7fbb2006fcd0 2024-04-26T21:44:19.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.434+0000 7fbb0effd640 1 -- 172.21.15.40:0/811053631 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) ==== 95+0+5181 (secure 0 0 0) 0x7fbb1007e8f0 con 0x7fbb2006fcd0 2024-04-26T21:44:19.435 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:19.435 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":196,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2569387163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2569387163},{"type":"v1","addr":"172.21.15.40:6833","nonce":2569387163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:18.346760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:19.435 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 196 2024-04-26T21:44:19.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb0006c8f0 msgr2=0x7fbb0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb0006c8f0 0x7fbb0006edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbb2015e490 tx=0x7fbb14009190 comp rx=0 tx=0).stop 2024-04-26T21:44:19.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2006fcd0 msgr2=0x7fbb200cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2006fcd0 0x7fbb200cf1a0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fbb10034040 tx=0x7fbb10004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 shutdown_connections 2024-04-26T21:44:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb0006c8f0 0x7fbb0006edb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2015f2d0 0x7fbb201e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb20072700 0x7fbb20154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 --2- 172.21.15.40:0/811053631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2006fcd0 0x7fbb200cf1a0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 >> 172.21.15.40:0/811053631 conn(0x7fbb200d0be0 msgr2=0x7fbb200cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 shutdown_connections 2024-04-26T21:44:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.436+0000 7fbb24b24640 1 -- 172.21.15.40:0/811053631 wait complete. 2024-04-26T21:44:19.448 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000000' led to DAMAGED state 2024-04-26T21:44:19.448 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000001' 2024-04-26T21:44:19.448 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:19.448 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:19.465 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:19.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:19.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:19.465 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:19.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:19.520 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:19.520 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:19.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.662+0000 7fd4cab51640 1 Processor -- start 2024-04-26T21:44:19.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.662+0000 7fd4cab51640 1 -- start start 2024-04-26T21:44:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4cab51640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4cab51640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c40d10d0 con 0x7fd4c40d1b30 2024-04-26T21:44:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4cab51640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c40d1370 con 0x7fd4c40b2340 2024-04-26T21:44:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4cab51640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c40d0750 con 0x7fd4c4162810 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c90ee640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd4c40d1b30 0x7fd4c4070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59316/0 (socket says 172.21.15.40:59316) 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c90ee640 1 -- 172.21.15.40:0/763535675 learned_addr learned my addr 172.21.15.40:0/763535675 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c88ed640 1 --2- 172.21.15.40:0/763535675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c37fe640 1 -- 172.21.15.40:0/763535675 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935316760 0 0) 0x7fd4c40d10d0 con 0x7fd4c40d1b30 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c37fe640 1 -- 172.21.15.40:0/763535675 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4b4003520 con 0x7fd4c40d1b30 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c37fe640 1 -- 172.21.15.40:0/763535675 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2572669312 0 0) 0x7fd4c40d1370 con 0x7fd4c40b2340 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c37fe640 1 -- 172.21.15.40:0/763535675 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4c40d10d0 con 0x7fd4c40b2340 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c88ed640 1 -- 172.21.15.40:0/763535675 >> v1:172.21.15.184:6789/0 conn(0x7fd4c40b2340 legacy=0x7fd4c40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c88ed640 1 -- 172.21.15.40:0/763535675 >> v1:172.21.15.40:6789/0 conn(0x7fd4c40d1b30 legacy=0x7fd4c4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.663+0000 7fd4c88ed640 1 -- 172.21.15.40:0/763535675 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c40d01b0 con 0x7fd4c4162810 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.664+0000 7fd4c88ed640 1 --2- 172.21.15.40:0/763535675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c4070350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80097d0 tx=0x7fd4b802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa17ddb2817d12d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.664+0000 7fd4c37fe640 1 -- 172.21.15.40:0/763535675 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8037070 con 0x7fd4c4162810 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.664+0000 7fd4c37fe640 1 -- 172.21.15.40:0/763535675 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b8032440 con 0x7fd4c4162810 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.664+0000 7fd4c37fe640 1 -- 172.21.15.40:0/763535675 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8032810 con 0x7fd4c4162810 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.665+0000 7fd4cab51640 1 -- 172.21.15.40:0/763535675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 msgr2=0x7fd4c4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.665+0000 7fd4cab51640 1 --2- 172.21.15.40:0/763535675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c4070350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80097d0 tx=0x7fd4b802a210 comp rx=0 tx=0).stop 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.665+0000 7fd4cab51640 1 -- 172.21.15.40:0/763535675 shutdown_connections 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.665+0000 7fd4cab51640 1 --2- 172.21.15.40:0/763535675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c4070350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.665+0000 7fd4cab51640 1 -- 172.21.15.40:0/763535675 >> 172.21.15.40:0/763535675 conn(0x7fd4c40d2da0 msgr2=0x7fd4c40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.665+0000 7fd4cab51640 1 -- 172.21.15.40:0/763535675 shutdown_connections 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.665+0000 7fd4cab51640 1 -- 172.21.15.40:0/763535675 wait complete. 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.666+0000 7fd4cab51640 1 Processor -- start 2024-04-26T21:44:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.666+0000 7fd4cab51640 1 -- start start 2024-04-26T21:44:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4cab51640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c40b2340 0x7fd4c40c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4cab51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4c40d1b30 0x7fd4c41f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4cab51640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c41f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4cab51640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c4070890 con 0x7fd4c40d1b30 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4cab51640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c40d16a0 con 0x7fd4c40b2340 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4cab51640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c40cdb00 con 0x7fd4c4162810 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4c40d1b30 0x7fd4c41f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4c40d1b30 0x7fd4c41f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42940/0 (socket says 172.21.15.40:42940) 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c3fff640 1 -- 172.21.15.40:0/2999903251 learned_addr learned my addr 172.21.15.40:0/2999903251 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c88ed640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c40b2340 0x7fd4c40c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c90ee640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c41f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c3fff640 1 -- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 msgr2=0x7fd4c41f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c3fff640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c41f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c3fff640 1 -- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c40b2340 msgr2=0x7fd4c40c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c3fff640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c40b2340 0x7fd4c40c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c3fff640 1 -- 172.21.15.40:0/2999903251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c40ca850 con 0x7fd4c40d1b30 2024-04-26T21:44:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c88ed640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c40b2340 0x7fd4c40c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.667+0000 7fd4c90ee640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c41f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.668+0000 7fd4c3fff640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4c40d1b30 0x7fd4c41f8150 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000c430 tx=0x7fd4b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.668+0000 7fd4c17fa640 1 -- 172.21.15.40:0/2999903251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0017070 con 0x7fd4c40d1b30 2024-04-26T21:44:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.668+0000 7fd4c17fa640 1 -- 172.21.15.40:0/2999903251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b00052f0 con 0x7fd4c40d1b30 2024-04-26T21:44:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.668+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c40caaf0 con 0x7fd4c40d1b30 2024-04-26T21:44:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.668+0000 7fd4c17fa640 1 -- 172.21.15.40:0/2999903251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0003830 con 0x7fd4c40d1b30 2024-04-26T21:44:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.668+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c409fe10 con 0x7fd4c40d1b30 2024-04-26T21:44:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.669+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c409d580 con 0x7fd4c40d1b30 2024-04-26T21:44:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.669+0000 7fd4c17fa640 1 -- 172.21.15.40:0/2999903251 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4b001a070 con 0x7fd4c40d1b30 2024-04-26T21:44:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.670+0000 7fd4c17fa640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4a806c920 0x7fd4a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.670+0000 7fd4c17fa640 1 -- 172.21.15.40:0/2999903251 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7679+0+0 (secure 0 0 0) 0x7fd4b008b6d0 con 0x7fd4c40d1b30 2024-04-26T21:44:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.670+0000 7fd4c88ed640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4a806c920 0x7fd4a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.671+0000 7fd4c88ed640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4a806c920 0x7fd4a806ede0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8034000 tx=0x7fd4b802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.674+0000 7fd4c17fa640 1 -- 172.21.15.40:0/2999903251 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4b0059350 con 0x7fd4c40d1b30 2024-04-26T21:44:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:19.898+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd4c4069a80 con 0x7fd4c40d1b30 2024-04-26T21:44:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.366+0000 7fd4c17fa640 1 -- 172.21.15.40:0/2999903251 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v197) ==== 161+0+0 (secure 0 0 0) 0x7fd4b0058c50 con 0x7fd4c40d1b30 2024-04-26T21:44:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.368+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4a806c920 msgr2=0x7fd4a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.368+0000 7fd4cab51640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4a806c920 0x7fd4a806ede0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8034000 tx=0x7fd4b802e000 comp rx=0 tx=0).stop 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.368+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4c40d1b30 msgr2=0x7fd4c41f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.368+0000 7fd4cab51640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4c40d1b30 0x7fd4c41f8150 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000c430 tx=0x7fd4b000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.369+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 shutdown_connections 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.369+0000 7fd4cab51640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4a806c920 0x7fd4a806ede0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.369+0000 7fd4cab51640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4c4162810 0x7fd4c41f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.369+0000 7fd4cab51640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4c40d1b30 0x7fd4c41f8150 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.369+0000 7fd4cab51640 1 --2- 172.21.15.40:0/2999903251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c40b2340 0x7fd4c40c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.369+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 >> 172.21.15.40:0/2999903251 conn(0x7fd4c40d2da0 msgr2=0x7fd4c40b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.369+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 shutdown_connections 2024-04-26T21:44:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.369+0000 7fd4cab51640 1 -- 172.21.15.40:0/2999903251 wait complete. 2024-04-26T21:44:20.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.484+0000 7f1de054d640 1 Processor -- start 2024-04-26T21:44:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.484+0000 7f1de054d640 1 -- start start 2024-04-26T21:44:20.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.485+0000 7f1de054d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd81608f0 0x7f1dd80d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.485+0000 7f1de054d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1dd80cffa0 0x7f1dd80b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.485+0000 7f1de054d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd80cde20 con 0x7f1dd80d0a90 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.485+0000 7f1de054d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd80cea30 con 0x7f1dd80cffa0 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.485+0000 7f1de054d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd80ce100 con 0x7f1dd81608f0 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.485+0000 7f1dddae8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1dd80d0a90 0x7f1dd80cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59326/0 (socket says 172.21.15.40:59326) 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.485+0000 7f1dddae8640 1 -- 172.21.15.40:0/973977113 learned_addr learned my addr 172.21.15.40:0/973977113 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1dde2e9640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd81608f0 0x7f1dd80d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddeaea640 1 --2- 172.21.15.40:0/973977113 >> v2:172.21.15.184:3300/0 conn(0x7f1dd80cffa0 0x7f1dd80b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1938465447 0 0) 0x7f1dd80cde20 con 0x7f1dd80d0a90 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dc0003520 con 0x7f1dd80d0a90 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3047451772 0 0) 0x7f1dc0003520 con 0x7f1dd80d0a90 2024-04-26T21:44:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd81608f0 msgr2=0x7f1dd80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd81608f0 0x7f1dd80d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 >> v2:172.21.15.184:3300/0 conn(0x7f1dd80cffa0 msgr2=0x7f1dd80b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 --2- 172.21.15.40:0/973977113 >> v2:172.21.15.184:3300/0 conn(0x7f1dd80cffa0 0x7f1dd80b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd80ce3e0 con 0x7f1dd80d0a90 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1dcc001670 con 0x7f1dd80d0a90 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 >> v1:172.21.15.40:6789/0 conn(0x7f1dd80d0a90 legacy=0x7f1dd80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00039b0 0x7f1dc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd81608f0 0x7f1dc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc0004d40 0x7f1dc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80cea30 con 0x7f1dc00039b0 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80ce100 con 0x7f1dc0004d40 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc000b9c0 con 0x7f1dd81608f0 2024-04-26T21:44:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1dde2e9640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00039b0 0x7f1dc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddeaea640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc0004d40 0x7f1dc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1dddae8640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd81608f0 0x7f1dc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1dde2e9640 1 -- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd81608f0 msgr2=0x7f1dc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1dde2e9640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd81608f0 0x7f1dc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1dde2e9640 1 -- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc0004d40 msgr2=0x7f1dc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1dde2e9640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc0004d40 0x7f1dc00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1dde2e9640 1 -- 172.21.15.40:0/973977113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc0003520 con 0x7f1dc00039b0 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.486+0000 7f1ddeaea640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc0004d40 0x7f1dc00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.487+0000 7f1dde2e9640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00039b0 0x7f1dc0003dd0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009ac0 tx=0x7f1dc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a4ab308d349fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.487+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0036070 con 0x7f1dc00039b0 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.487+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd0031400 con 0x7f1dc00039b0 2024-04-26T21:44:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.487+0000 7f1ddd2e7640 1 -- 172.21.15.40:0/973977113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0009690 con 0x7f1dc00039b0 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.488+0000 7f1de054d640 1 -- 172.21.15.40:0/973977113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00039b0 msgr2=0x7f1dc0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.488+0000 7f1de054d640 1 --2- 172.21.15.40:0/973977113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc00039b0 0x7f1dc0003dd0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009ac0 tx=0x7f1dc0005830 comp rx=0 tx=0).stop 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.488+0000 7f1ddeaea640 1 -- 172.21.15.40:0/973977113 reap_dead start 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.488+0000 7f1de054d640 1 -- 172.21.15.40:0/973977113 shutdown_connections 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.488+0000 7f1de054d640 1 -- 172.21.15.40:0/973977113 >> 172.21.15.40:0/973977113 conn(0x7f1dd80d2ec0 msgr2=0x7f1dd80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.488+0000 7f1de054d640 1 -- 172.21.15.40:0/973977113 shutdown_connections 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.488+0000 7f1de054d640 1 -- 172.21.15.40:0/973977113 wait complete. 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.489+0000 7f1de054d640 1 Processor -- start 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.489+0000 7f1de054d640 1 -- start start 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1de054d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc00039b0 0x7f1dd8157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1de054d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd815d900 0x7f1dd815f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1de054d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd815e250 0x7f1dd81f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1de054d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80ceec0 con 0x7f1dd815e250 2024-04-26T21:44:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1de054d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80d0ef0 con 0x7f1dd815d900 2024-04-26T21:44:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1de054d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd80cda40 con 0x7f1dc00039b0 2024-04-26T21:44:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd815e250 0x7f1dd81f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd815e250 0x7f1dd81f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42956/0 (socket says 172.21.15.40:42956) 2024-04-26T21:44:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 -- 172.21.15.40:0/3223215504 learned_addr learned my addr 172.21.15.40:0/3223215504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1dde2e9640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc00039b0 0x7f1dd8157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1dddae8640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd815d900 0x7f1dd815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 -- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc00039b0 msgr2=0x7f1dd8157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc00039b0 0x7f1dd8157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 -- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd815d900 msgr2=0x7f1dd815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd815d900 0x7f1dd815f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 -- 172.21.15.40:0/3223215504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd815c860 con 0x7f1dd815e250 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.490+0000 7f1ddeaea640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd815e250 0x7f1dd81f6e40 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8073530 tx=0x7f1dc8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.491+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/3223215504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc801a030 con 0x7f1dd815e250 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.491+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/3223215504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc8010d80 con 0x7f1dd815e250 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.491+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd815cb40 con 0x7f1dd815e250 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.491+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/3223215504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8003bd0 con 0x7f1dd815e250 2024-04-26T21:44:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.491+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd815d100 con 0x7f1dd815e250 2024-04-26T21:44:20.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.492+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd80694f0 con 0x7f1dd815e250 2024-04-26T21:44:20.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.492+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/3223215504 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dc801d070 con 0x7f1dd815e250 2024-04-26T21:44:20.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.492+0000 7f1dd6ffd640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:20.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.493+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/3223215504 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7679+0+0 (secure 0 0 0) 0x7f1dc808c7f0 con 0x7f1dd815e250 2024-04-26T21:44:20.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.493+0000 7f1dde2e9640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:20.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.493+0000 7f1dde2e9640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0033000 tx=0x7f1dd0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:20.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.497+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/3223215504 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1dc805a500 con 0x7f1dd815e250 2024-04-26T21:44:20.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:20.754+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f1dd80d2330 con 0x7f1dd815e250 2024-04-26T21:44:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.368+0000 7f1dd6ffd640 1 -- 172.21.15.40:0/3223215504 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v198) ==== 101+0+0 (secure 0 0 0) 0x7f1dc8059e00 con 0x7f1dd815e250 2024-04-26T21:44:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.371+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 msgr2=0x7f1dc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.371+0000 7f1de054d640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0033000 tx=0x7f1dd0008030 comp rx=0 tx=0).stop 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.371+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd815e250 msgr2=0x7f1dd81f6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.371+0000 7f1de054d640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd815e250 0x7f1dd81f6e40 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8073530 tx=0x7f1dc8010590 comp rx=0 tx=0).stop 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.372+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 shutdown_connections 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.372+0000 7f1de054d640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1dc406c920 0x7f1dc406ede0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.372+0000 7f1de054d640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd815e250 0x7f1dd81f6e40 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.372+0000 7f1de054d640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd815d900 0x7f1dd815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.372+0000 7f1de054d640 1 --2- 172.21.15.40:0/3223215504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dc00039b0 0x7f1dd8157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.372+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 >> 172.21.15.40:0/3223215504 conn(0x7f1dd80d2ec0 msgr2=0x7f1dd815f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.372+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 shutdown_connections 2024-04-26T21:44:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.372+0000 7f1de054d640 1 -- 172.21.15.40:0/3223215504 wait complete. 2024-04-26T21:44:21.387 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f423468ef40 1 Processor -- start 2024-04-26T21:44:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f423468ef40 1 -- start start 2024-04-26T21:44:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f423468ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6c770 0x55958cffb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f423468ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6e420 0x55958cff01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f423468ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf527b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f423468ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55958cf6e860 con 0x55958cff86b0 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f423468ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55958cecae40 con 0x55958cf6e420 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f423468ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55958cff9aa0 con 0x55958cf6c770 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f4232989640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f4232989640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42966/0 (socket says 172.21.15.40:42966) 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.432+0000 7f4232989640 1 -- 172.21.15.40:0/2564292801 learned_addr learned my addr 172.21.15.40:0/2564292801 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4231987640 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6e420 0x55958cff01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4232188640 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6c770 0x55958cffb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4232989640 1 -- 172.21.15.40:0/2564292801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6c770 msgr2=0x55958cffb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4232989640 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6c770 0x55958cffb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4232989640 1 -- 172.21.15.40:0/2564292801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6e420 msgr2=0x55958cff01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4232989640 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6e420 0x55958cff01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4232989640 1 -- 172.21.15.40:0/2564292801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55958cffbb00 con 0x55958cff86b0 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4232989640 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf527b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f422400c480 tx=0x7f422400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53761e2f91986470 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4230985640 1 -- 172.21.15.40:0/2564292801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42240116b0 con 0x55958cff86b0 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4230985640 1 -- 172.21.15.40:0/2564292801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4224017070 con 0x55958cff86b0 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.433+0000 7f4230985640 1 -- 172.21.15.40:0/2564292801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224011a90 con 0x55958cff86b0 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 -- 172.21.15.40:0/2564292801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 msgr2=0x55958cf527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf527b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f422400c480 tx=0x7f422400c940 comp rx=0 tx=0).stop 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 -- 172.21.15.40:0/2564292801 shutdown_connections 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf527b0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6e420 0x55958cff01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 --2- 172.21.15.40:0/2564292801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6c770 0x55958cffb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 -- 172.21.15.40:0/2564292801 >> 172.21.15.40:0/2564292801 conn(0x55958cf4e9b0 msgr2=0x55958cf69480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 -- 172.21.15.40:0/2564292801 shutdown_connections 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 -- 172.21.15.40:0/2564292801 wait complete. 2024-04-26T21:44:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.434+0000 7f423468ef40 1 Processor -- start 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f423468ef40 1 -- start start 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f423468ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6c770 0x55958d090680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f423468ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6e420 0x55958d090bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f423468ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f423468ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55958cf6b160 con 0x55958cff86b0 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f423468ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55958cf6aed0 con 0x55958cf6c770 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f423468ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55958cf6b440 con 0x55958cf6e420 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f4232989640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f4232989640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf23eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42982/0 (socket says 172.21.15.40:42982) 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.435+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 learned_addr learned my addr 172.21.15.40:0/1664971446 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4231987640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6e420 0x55958d090bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4232188640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6c770 0x55958d090680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6e420 msgr2=0x55958d090bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4232989640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55958cf6e420 0x55958d090bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6c770 msgr2=0x55958d090680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4232989640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6c770 0x55958d090680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55958cfecfc0 con 0x55958cff86b0 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4232188640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958cf6c770 0x55958d090680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f4232989640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf23eb0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f422400c210 tx=0x7f42240033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f42237fe640 1 -- 172.21.15.40:0/1664971446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224012690 con 0x55958cff86b0 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f42237fe640 1 -- 172.21.15.40:0/1664971446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4224012870 con 0x55958cff86b0 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55958cf22e10 con 0x55958cff86b0 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f42237fe640 1 -- 172.21.15.40:0/1664971446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224012c60 con 0x55958cff86b0 2024-04-26T21:44:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.436+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958cf233d0 con 0x55958cff86b0 2024-04-26T21:44:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.438+0000 7f42237fe640 1 -- 172.21.15.40:0/1664971446 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4224024400 con 0x55958cff86b0 2024-04-26T21:44:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.438+0000 7f42237fe640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f421406b2d0 0x7f421406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.438+0000 7f42237fe640 1 -- 172.21.15.40:0/1664971446 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7679+0+0 (secure 0 0 0) 0x7f422408ce10 con 0x55958cff86b0 2024-04-26T21:44:21.442 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.439+0000 7f4232188640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f421406b2d0 0x7f421406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.439+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55958cf24900 0x55958cf686b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.439+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958cf68bf0 con 0x55958cf24900 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.439+0000 7f4231987640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55958cf24900 0x55958cf686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.439+0000 7f4231987640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55958cf24900 0x55958cf686b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.439+0000 7f4232188640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f421406b2d0 0x7f421406d790 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f422c00b680 tx=0x7f422c0058c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.441+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v112'505 uv479 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.441+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958cf68f90 con 0x55958cf24900 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.442+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v112'506 uv506 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.442+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09d4b0 con 0x55958cf24900 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.443+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v112'507 uv507 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.444 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.443+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09e860 con 0x55958cf24900 2024-04-26T21:44:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.444+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v112'508 uv508 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.444+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09ec00 con 0x55958cf24900 2024-04-26T21:44:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.445+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v112'509 uv509 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.445+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09d850 con 0x55958cf24900 2024-04-26T21:44:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.446+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v112'510 uv510 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.446+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09dbf0 con 0x55958cf24900 2024-04-26T21:44:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.447+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v112'511 uv485 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.447+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d09dfe0 con 0x55958cf24900 2024-04-26T21:44:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.449+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v112'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.449+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09dfe0 con 0x55958cf24900 2024-04-26T21:44:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.450+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v112'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.450 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.450+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.451+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v112'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.451+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.452+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v112'515 uv515 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.452+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.453+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v112'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.453+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.454+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v112'517 uv492 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.454+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.455+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v112'518 uv518 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.455+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.456+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v112'519 uv519 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.457 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.456+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.457+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v112'520 uv520 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.457+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.458+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v112'521 uv521 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.458+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.459+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v112'522 uv522 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.460+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.461+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v112'523 uv523 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.461+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.462+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v112'524 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.462+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.463+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v112'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.463+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.464+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v112'526 uv526 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.465 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.464+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.465+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v112'527 uv527 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.465+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.466+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v112'528 uv528 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.466+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.467+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v112'529 uv529 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.467+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.468+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v112'530 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.468+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.469+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v112'531 uv531 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.469+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.470+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v112'532 uv532 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.471 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.470+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.471+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v112'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.471+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.472+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v112'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.472+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d09df90 con 0x55958cf24900 2024-04-26T21:44:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.473+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v112'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958cf24900 2024-04-26T21:44:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.473+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55958cff6770 0x55958d0a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.473+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a1a00 con 0x55958cff6770 2024-04-26T21:44:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.473+0000 7f4232188640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55958cff6770 0x55958d0a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.473+0000 7f4232188640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55958cff6770 0x55958d0a14c0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.475+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v112'374 uv356 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.475+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0a1da0 con 0x55958cff6770 2024-04-26T21:44:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.476+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v112'375 uv375 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.476+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a2140 con 0x55958cff6770 2024-04-26T21:44:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.478+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v112'376 uv376 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.479 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.478+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a2990 con 0x55958cff6770 2024-04-26T21:44:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.479+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v112'377 uv377 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.479+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a2d30 con 0x55958cff6770 2024-04-26T21:44:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.480+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v112'378 uv378 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.480+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a30d0 con 0x55958cff6770 2024-04-26T21:44:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.481+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v112'379 uv373 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.481+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0a3470 con 0x55958cff6770 2024-04-26T21:44:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.483+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v112'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.483+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.484 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.484+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v112'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.484+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.485+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v112'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.485+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.487+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v112'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.487+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.488+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v112'384 uv366 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.488+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.489+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v112'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.490+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.491 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.491+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v112'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.491+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a7ed0 con 0x55958cff6770 2024-04-26T21:44:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.492+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v112'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.492+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a7ed0 con 0x55958cff6770 2024-04-26T21:44:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.494+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v112'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.494+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.495+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v112'389 uv372 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.495+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.497+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v112'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.497+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.499 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.498+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v112'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.498+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a4820 con 0x55958cff6770 2024-04-26T21:44:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.499+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v112'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.499+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a4820 con 0x55958cff6770 2024-04-26T21:44:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.501+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v112'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.501+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a3810 con 0x55958cff6770 2024-04-26T21:44:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.502+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v112'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958cff6770 2024-04-26T21:44:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.502+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55958d0a3d20 0x55958d0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.502+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a67c0 con 0x55958d0a3d20 2024-04-26T21:44:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.502+0000 7f4231987640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55958d0a3d20 0x55958d0a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.503+0000 7f4231987640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55958d0a3d20 0x55958d0a6150 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.504+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v112'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.504+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0a6b90 con 0x55958d0a3d20 2024-04-26T21:44:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.505+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v112'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.505+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a6b90 con 0x55958d0a3d20 2024-04-26T21:44:21.507 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.507+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v112'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.507+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a7ba0 con 0x55958d0a3d20 2024-04-26T21:44:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.508+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v112'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.508+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a7ba0 con 0x55958d0a3d20 2024-04-26T21:44:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.509+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v112'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.509+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a6b90 con 0x55958d0a3d20 2024-04-26T21:44:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.510+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v112'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.510+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55958d0a7460 0x55958d0a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.510+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0a9d90 con 0x55958d0a7460 2024-04-26T21:44:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.510+0000 7f4232989640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55958d0a7460 0x55958d0a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.510+0000 7f4232989640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55958d0a7460 0x55958d0a9850 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.512+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v112'80 uv79 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.512+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0aa240 con 0x55958d0a7460 2024-04-26T21:44:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.513+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v112'81 uv81 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.513+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0aa740 con 0x55958d0a7460 2024-04-26T21:44:21.515 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.514+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v112'82 uv82 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.514+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0ab000 con 0x55958d0a7460 2024-04-26T21:44:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.515+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v112'83 uv83 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.515+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55958d0ab8b0 0x55958d0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.515+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0ae1e0 con 0x55958d0ab8b0 2024-04-26T21:44:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.516+0000 7f4232188640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55958d0ab8b0 0x55958d0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.516+0000 7f4232188640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55958d0ab8b0 0x55958d0adca0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.517+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v112'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.517+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0ae2a0 con 0x55958d0ab8b0 2024-04-26T21:44:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.518+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v112'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.518+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0ae6c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.520 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.520+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v112'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.520+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af6d0 con 0x55958d0ab8b0 2024-04-26T21:44:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.521+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v112'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.521+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af6d0 con 0x55958d0ab8b0 2024-04-26T21:44:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.522+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v112'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.522+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0ae6c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.523+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v112'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.523+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0ae6c0 con 0x55958d0a7460 2024-04-26T21:44:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.524+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v112'182 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.524+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0aea60 con 0x55958d0a7460 2024-04-26T21:44:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.526+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v112'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.526+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0aee00 con 0x55958d0a7460 2024-04-26T21:44:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.527+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v112'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.527 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.527+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b01b0 con 0x55958d0a7460 2024-04-26T21:44:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.528+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v112'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.528+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b01b0 con 0x55958d0a7460 2024-04-26T21:44:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.529+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v112'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.529+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.530+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v112'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.530+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.531+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v112'188 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.531+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.532+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v112'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.532+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.533+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v112'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.534 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.533+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.534+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v112'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.534+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.535+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v112'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.535+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.536+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v112'81 uv80 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.536+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.537+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v112'82 uv82 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.537+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.538+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v112'83 uv83 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.539 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.539+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.540+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v112'84 uv84 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.540+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0a7460 2024-04-26T21:44:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.541+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v112'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4224056b10 con 0x55958d0a7460 2024-04-26T21:44:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.541+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0ab8b0 2024-04-26T21:44:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.542+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v112'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.542+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0ab8b0 2024-04-26T21:44:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.543+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v112'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.543+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af1a0 con 0x55958d0ab8b0 2024-04-26T21:44:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.544+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v112'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.545 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.544+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b05d0 con 0x55958d0ab8b0 2024-04-26T21:44:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.545+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v112'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.545+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b05d0 con 0x55958d0ab8b0 2024-04-26T21:44:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.546+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v112'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.546+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.547+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v112'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.547+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.548+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v112'162 uv156 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.548+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.549+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v112'163 uv163 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.550+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.551 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.551+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v112'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.551+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.552+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v112'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.552+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.553+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v112'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.553+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.554+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v112'167 uv161 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.554+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.555+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v112'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.555+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.557 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.556+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v112'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.556+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0af5c0 con 0x55958d0ab8b0 2024-04-26T21:44:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.557+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v112'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.557+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b23d0 con 0x55958d0ab8b0 2024-04-26T21:44:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.558+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v112'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.558+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55958d0aff80 0x55958d0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.558+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2370 con 0x55958d0aff80 2024-04-26T21:44:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.558+0000 7f4231987640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55958d0aff80 0x55958d0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.559+0000 7f4231987640 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55958d0aff80 0x55958d0b27f0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.560+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v112'74 uv73 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0aff80 2024-04-26T21:44:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.560+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0b2370 con 0x55958d0aff80 2024-04-26T21:44:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.561+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v112'75 uv75 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0aff80 2024-04-26T21:44:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.561+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2370 con 0x55958d0aff80 2024-04-26T21:44:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.562+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v112'76 uv76 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0aff80 2024-04-26T21:44:21.562 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.562+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0aff80 2024-04-26T21:44:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.563+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v112'77 uv77 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0aff80 2024-04-26T21:44:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.563+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0ab8b0 2024-04-26T21:44:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.564+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v112'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.564+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0ab8b0 2024-04-26T21:44:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.565+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v112'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.565+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0ab8b0 2024-04-26T21:44:21.566 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.566+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v112'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.566+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b3d40 con 0x55958d0ab8b0 2024-04-26T21:44:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.567+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v112'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.567+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b3d40 con 0x55958d0ab8b0 2024-04-26T21:44:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.568+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v112'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.568+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0ab8b0 2024-04-26T21:44:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.570+0000 7f4232188640 1 -- 172.21.15.40:0/1664971446 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v112'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f422c00aae0 con 0x55958d0ab8b0 2024-04-26T21:44:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.570+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.571+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v112'84 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.571+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.572+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v112'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.572+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.573+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v112'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.574 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.574+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.575+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v112'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.575+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.576+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v112'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.576+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.577+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v112'204 uv196 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.577+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.578+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v112'205 uv205 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.578+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.580 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.579+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v112'206 uv206 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.579+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.581+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v112'207 uv207 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.581+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.582+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v112'208 uv208 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.582+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.583+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v112'209 uv203 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.583+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.584+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v112'210 uv210 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.584+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.585 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.584+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v112'211 uv211 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.585+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b3d40 con 0x55958d0a3d20 2024-04-26T21:44:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.585+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v112'212 uv212 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.585+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b3d40 con 0x55958d0a3d20 2024-04-26T21:44:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.586+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v112'213 uv213 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.586+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.587+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v112'214 uv214 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.588+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55958d0b2d30 con 0x55958d0a3d20 2024-04-26T21:44:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f4231987640 1 -- 172.21.15.40:0/1664971446 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v112'215 uv215 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f42280546f0 con 0x55958d0a3d20 2024-04-26T21:44:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55958cff6770 msgr2=0x55958d0a14c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55958cff6770 0x55958d0a14c0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55958cf24900 msgr2=0x55958cf686b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55958cf24900 0x55958cf686b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55958d0a7460 msgr2=0x55958d0a9850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55958d0a7460 0x55958d0a9850 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55958d0aff80 msgr2=0x55958d0b27f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55958d0aff80 0x55958d0b27f0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55958d0a3d20 msgr2=0x55958d0a6150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 reap_dead start 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55958d0a3d20 0x55958d0a6150 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55958d0ab8b0 msgr2=0x55958d0adca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f4232989640 1 -- 172.21.15.40:0/1664971446 reap_dead start 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55958d0ab8b0 0x55958d0adca0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f421406b2d0 msgr2=0x7f421406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f421406b2d0 0x7f421406d790 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f422c00b680 tx=0x7f422c0058c0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 msgr2=0x55958cf23eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf23eb0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f422400c210 tx=0x7f42240033a0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 shutdown_connections 2024-04-26T21:44:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55958d0ab8b0 0x55958d0adca0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f421406b2d0 0x7f421406d790 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 --2- 172.21.15.40:0/1664971446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55958cff86b0 0x55958cf23eb0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 >> 172.21.15.40:0/1664971446 conn(0x55958cf4e9b0 msgr2=0x55958cffad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 shutdown_connections 2024-04-26T21:44:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.589+0000 7f423468ef40 1 -- 172.21.15.40:0/1664971446 wait complete. 2024-04-26T21:44:21.593 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000001 - 2024-04-26T21:44:21.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.676+0000 7fb392683f40 1 Processor -- start 2024-04-26T21:44:21.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.676+0000 7fb392683f40 1 -- start start 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb392683f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee5971f480 0x55ee597b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb392683f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee59738510 0x55ee596f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb392683f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 0x55ee596f1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb392683f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ee597c4c00 con 0x55ee5971f480 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb392683f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee59696e40 con 0x55ee59738510 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb392683f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ee5971f8a0 con 0x55ee597c4820 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39017d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee5971f480 0x55ee597b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39017d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee5971f480 0x55ee597b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42992/0 (socket says 172.21.15.40:42992) 2024-04-26T21:44:21.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39017d640 1 -- 172.21.15.40:0/3640723286 learned_addr learned my addr 172.21.15.40:0/3640723286 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb38f97c640 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee59738510 0x55ee596f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39097e640 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 0x55ee596f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39017d640 1 -- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 msgr2=0x55ee596f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39017d640 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 0x55ee596f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39017d640 1 -- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee59738510 msgr2=0x55ee596f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39017d640 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee59738510 0x55ee596f1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39017d640 1 -- 172.21.15.40:0/3640723286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee596f09f0 con 0x55ee5971f480 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb38f97c640 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee59738510 0x55ee596f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.677+0000 7fb39097e640 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 0x55ee596f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb39017d640 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee5971f480 0x55ee597b6eb0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fb3880099a0 tx=0x7fb388005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3118d82de7f849 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb38e97a640 1 -- 172.21.15.40:0/3640723286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388008a30 con 0x55ee5971f480 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb38e97a640 1 -- 172.21.15.40:0/3640723286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb388018070 con 0x55ee5971f480 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb38e97a640 1 -- 172.21.15.40:0/3640723286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388008dd0 con 0x55ee5971f480 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 -- 172.21.15.40:0/3640723286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee5971f480 msgr2=0x55ee597b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee5971f480 0x55ee597b6eb0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fb3880099a0 tx=0x7fb388005a00 comp rx=0 tx=0).stop 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 -- 172.21.15.40:0/3640723286 shutdown_connections 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 0x55ee596f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee59738510 0x55ee596f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 --2- 172.21.15.40:0/3640723286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee5971f480 0x55ee597b6eb0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 -- 172.21.15.40:0/3640723286 >> 172.21.15.40:0/3640723286 conn(0x55ee597198a0 msgr2=0x55ee597c36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 -- 172.21.15.40:0/3640723286 shutdown_connections 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.678+0000 7fb392683f40 1 -- 172.21.15.40:0/3640723286 wait complete. 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.679+0000 7fb392683f40 1 Processor -- start 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.679+0000 7fb392683f40 1 -- start start 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb392683f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee5971f480 0x55ee597b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb392683f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee59738510 0x55ee597bc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb392683f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 0x55ee5971db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb392683f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ee5971adc0 con 0x55ee59738510 2024-04-26T21:44:21.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb392683f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ee597bfa20 con 0x55ee5971f480 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb392683f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ee597b7890 con 0x55ee597c4820 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee59738510 0x55ee597bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee59738510 0x55ee597bc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43002/0 (socket says 172.21.15.40:43002) 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 -- 172.21.15.40:0/2736268137 learned_addr learned my addr 172.21.15.40:0/2736268137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb39017d640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee5971f480 0x55ee597b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb39097e640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 0x55ee5971db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 -- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 msgr2=0x55ee5971db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ee597c4820 0x55ee5971db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 -- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee5971f480 msgr2=0x55ee597b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee5971f480 0x55ee597b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 -- 172.21.15.40:0/2736268137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee597b4130 con 0x55ee59738510 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38f97c640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee59738510 0x55ee597bc310 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fb378025e70 tx=0x7fb378002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38d978640 1 -- 172.21.15.40:0/2736268137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378064020 con 0x55ee59738510 2024-04-26T21:44:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38d978640 1 -- 172.21.15.40:0/2736268137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb378060670 con 0x55ee59738510 2024-04-26T21:44:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.680+0000 7fb38d978640 1 -- 172.21.15.40:0/2736268137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37805a040 con 0x55ee59738510 2024-04-26T21:44:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.681+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ee5971e600 con 0x55ee59738510 2024-04-26T21:44:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.681+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee5971ebc0 con 0x55ee59738510 2024-04-26T21:44:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.682+0000 7fb38d978640 1 -- 172.21.15.40:0/2736268137 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb378067020 con 0x55ee59738510 2024-04-26T21:44:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.682+0000 7fb38d978640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb37006b2d0 0x7fb37006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.682+0000 7fb38d978640 1 -- 172.21.15.40:0/2736268137 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7679+0+0 (secure 0 0 0) 0x7fb3780cf380 con 0x55ee59738510 2024-04-26T21:44:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.683+0000 7fb392683f40 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ee5971a190 0x55ee597b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.683+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55ee597bc850 con 0x55ee5971a190 2024-04-26T21:44:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.683+0000 7fb39017d640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb37006b2d0 0x7fb37006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.683+0000 7fb39097e640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ee5971a190 0x55ee597b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.683+0000 7fb39097e640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ee5971a190 0x55ee597b5b30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.683+0000 7fb39017d640 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb37006b2d0 0x7fb37006d790 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb38800c010 tx=0x7fb388012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb39097e640 1 -- 172.21.15.40:0/2736268137 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000001 [writefull 0~80] v112'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb380001960 con 0x55ee5971a190 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ee5971a190 msgr2=0x55ee597b5b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb392683f40 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ee5971a190 0x55ee597b5b30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb37006b2d0 msgr2=0x7fb37006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb392683f40 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb37006b2d0 0x7fb37006d790 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb38800c010 tx=0x7fb388012000 comp rx=0 tx=0).stop 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee59738510 msgr2=0x55ee597bc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb392683f40 1 --2- 172.21.15.40:0/2736268137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ee59738510 0x55ee597bc310 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fb378025e70 tx=0x7fb378002fb0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb39097e640 1 -- 172.21.15.40:0/2736268137 reap_dead start 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 shutdown_connections 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.684+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 >> 172.21.15.40:0/2736268137 conn(0x55ee597198a0 msgr2=0x55ee596effe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.685+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 shutdown_connections 2024-04-26T21:44:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.685+0000 7fb392683f40 1 -- 172.21.15.40:0/2736268137 wait complete. 2024-04-26T21:44:21.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.831+0000 7f56dac23640 1 Processor -- start 2024-04-26T21:44:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.831+0000 7f56dac23640 1 -- start start 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56dac23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d40b40d0 0x7f56d41625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56dac23640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56d4162b30 0x7f56d40b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56dac23640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d40d0a10 con 0x7f56d40d1a00 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56dac23640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f56d40d0cf0 con 0x7f56d4162b30 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56dac23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56d40d0fd0 con 0x7f56d40b40d0 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56d91c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56d4162b30 0x7f56d40b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56d91c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56d4162b30 0x7f56d40b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43004/0 (socket says 172.21.15.40:43004) 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56d89bf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f56d40d1a00 0x7f56d40d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59344/0 (socket says 172.21.15.40:59344) 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56d91c0640 1 -- 172.21.15.40:0/471909839 learned_addr learned my addr 172.21.15.40:0/471909839 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.832+0000 7f56d3fff640 1 --2- 172.21.15.40:0/471909839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d40b40d0 0x7f56d41625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.833+0000 7f56d91c0640 1 -- 172.21.15.40:0/471909839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d40b40d0 msgr2=0x7f56d41625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.833+0000 7f56d91c0640 1 --2- 172.21.15.40:0/471909839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d40b40d0 0x7f56d41625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.833+0000 7f56d91c0640 1 -- 172.21.15.40:0/471909839 >> v1:172.21.15.40:6789/0 conn(0x7f56d40d1a00 legacy=0x7f56d40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.833+0000 7f56d37fe640 1 -- 172.21.15.40:0/471909839 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142920748 0 0) 0x7f56d40d0a10 con 0x7f56d40d1a00 2024-04-26T21:44:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.833+0000 7f56d91c0640 1 -- 172.21.15.40:0/471909839 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d4162fa0 con 0x7f56d4162b30 2024-04-26T21:44:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.833+0000 7f56d3fff640 1 --2- 172.21.15.40:0/471909839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d40b40d0 0x7f56d41625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.833+0000 7f56d91c0640 1 --2- 172.21.15.40:0/471909839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d4162b30 0x7f56d40b4b00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009870 tx=0x7f56c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48967cfcfd76e930 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56d37fe640 1 -- 172.21.15.40:0/471909839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4037070 con 0x7f56d4162b30 2024-04-26T21:44:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56d37fe640 1 -- 172.21.15.40:0/471909839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c4032440 con 0x7f56d4162b30 2024-04-26T21:44:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56d37fe640 1 -- 172.21.15.40:0/471909839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4032810 con 0x7f56d4162b30 2024-04-26T21:44:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56dac23640 1 -- 172.21.15.40:0/471909839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d4162b30 msgr2=0x7f56d40b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56dac23640 1 --2- 172.21.15.40:0/471909839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d4162b30 0x7f56d40b4b00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009870 tx=0x7f56c402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56dac23640 1 -- 172.21.15.40:0/471909839 shutdown_connections 2024-04-26T21:44:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56dac23640 1 --2- 172.21.15.40:0/471909839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d4162b30 0x7f56d40b4b00 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56dac23640 1 --2- 172.21.15.40:0/471909839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d40b40d0 0x7f56d41625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56dac23640 1 -- 172.21.15.40:0/471909839 >> 172.21.15.40:0/471909839 conn(0x7f56d40d4fd0 msgr2=0x7f56d40d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56dac23640 1 -- 172.21.15.40:0/471909839 shutdown_connections 2024-04-26T21:44:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.834+0000 7f56dac23640 1 -- 172.21.15.40:0/471909839 wait complete. 2024-04-26T21:44:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.835+0000 7f56dac23640 1 Processor -- start 2024-04-26T21:44:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.835+0000 7f56dac23640 1 -- start start 2024-04-26T21:44:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56dac23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56d40b40d0 0x7f56d40ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56dac23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d40d1a00 0x7f56d40ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56dac23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d4162b30 0x7f56d40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56dac23640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40d2670 con 0x7f56d40d1a00 2024-04-26T21:44:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56dac23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40d24f0 con 0x7f56d40b40d0 2024-04-26T21:44:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56dac23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56d40b5040 con 0x7f56d4162b30 2024-04-26T21:44:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d40d1a00 0x7f56d40ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d40d1a00 0x7f56d40ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43008/0 (socket says 172.21.15.40:43008) 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56d3fff640 1 -- 172.21.15.40:0/2150671868 learned_addr learned my addr 172.21.15.40:0/2150671868 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.836+0000 7f56d91c0640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d4162b30 0x7f56d40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d89bf640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56d40b40d0 0x7f56d40ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d3fff640 1 -- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d4162b30 msgr2=0x7f56d40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d3fff640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d4162b30 0x7f56d40cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d3fff640 1 -- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56d40b40d0 msgr2=0x7f56d40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d3fff640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56d40b40d0 0x7f56d40ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d3fff640 1 -- 172.21.15.40:0/2150671868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d40ba6e0 con 0x7f56d40d1a00 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d89bf640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56d40b40d0 0x7f56d40ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d3fff640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d40d1a00 0x7f56d40ce6a0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f56c800ebe0 tx=0x7f56c800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d17fa640 1 -- 172.21.15.40:0/2150671868 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8017070 con 0x7f56d40d1a00 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d17fa640 1 -- 172.21.15.40:0/2150671868 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c800cc30 con 0x7f56d40d1a00 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d40cd150 con 0x7f56d40d1a00 2024-04-26T21:44:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56d17fa640 1 -- 172.21.15.40:0/2150671868 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8013470 con 0x7f56d40d1a00 2024-04-26T21:44:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.837+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d40cd710 con 0x7f56d40d1a00 2024-04-26T21:44:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.838+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d40694f0 con 0x7f56d40d1a00 2024-04-26T21:44:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.839+0000 7f56d17fa640 1 -- 172.21.15.40:0/2150671868 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56c801a070 con 0x7f56d40d1a00 2024-04-26T21:44:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.839+0000 7f56d17fa640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f56c006c920 0x7f56c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.839+0000 7f56d17fa640 1 -- 172.21.15.40:0/2150671868 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7679+0+0 (secure 0 0 0) 0x7f56c808c120 con 0x7f56d40d1a00 2024-04-26T21:44:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.839+0000 7f56d89bf640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f56c006c920 0x7f56c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.840+0000 7f56d89bf640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f56c006c920 0x7f56c006ede0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0012e0 tx=0x7f56cc007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:21.844+0000 7f56d17fa640 1 -- 172.21.15.40:0/2150671868 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f56d40ba6e0 con 0x7f56d40d1a00 2024-04-26T21:44:22.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.070+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f56d40b6480 con 0x7f56d40d1a00 2024-04-26T21:44:22.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.380+0000 7f56d17fa640 1 -- 172.21.15.40:0/2150671868 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v199) ==== 162+0+0 (secure 0 0 0) 0x7f56c80597e0 con 0x7f56d40d1a00 2024-04-26T21:44:22.381 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:22.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f56c006c920 msgr2=0x7f56c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f56c006c920 0x7f56c006ede0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0012e0 tx=0x7f56cc007f90 comp rx=0 tx=0).stop 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d40d1a00 msgr2=0x7f56d40ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d40d1a00 0x7f56d40ce6a0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f56c800ebe0 tx=0x7f56c800c520 comp rx=0 tx=0).stop 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 shutdown_connections 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f56c006c920 0x7f56c006ede0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56d4162b30 0x7f56d40cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56d40d1a00 0x7f56d40ce6a0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 --2- 172.21.15.40:0/2150671868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56d40b40d0 0x7f56d40ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 >> 172.21.15.40:0/2150671868 conn(0x7f56d40d4fd0 msgr2=0x7f56d40d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 shutdown_connections 2024-04-26T21:44:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.382+0000 7f56dac23640 1 -- 172.21.15.40:0/2150671868 wait complete. 2024-04-26T21:44:22.392 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:22.391+0000 7f7bf5595640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:44:22.392 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:22.392+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:22.393 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:22.392+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:22.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:22.399 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:44:22.391+0000 7f7bf5595640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:44:22.423 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:22.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.498+0000 7f58d2787640 1 Processor -- start 2024-04-26T21:44:22.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.498+0000 7f58d2787640 1 -- start start 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d2787640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58cc0b0380 0x7f58cc0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d2787640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58cc06c950 0x7f58cc15e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d2787640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58cc0cfd80 con 0x7f58cc0b10b0 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d2787640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0af8e0 con 0x7f58cc06c950 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d2787640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0d0020 con 0x7f58cc0b0380 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d0d24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58cc06c950 0x7f58cc15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d0d24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58cc06c950 0x7f58cc15e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43028/0 (socket says 172.21.15.40:43028) 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58cb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f58cc0b10b0 0x7f58cc06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59360/0 (socket says 172.21.15.40:59360) 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d0d24640 1 -- 172.21.15.40:0/3762140807 learned_addr learned my addr 172.21.15.40:0/3762140807 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58cbfff640 1 --2- 172.21.15.40:0/3762140807 >> v2:172.21.15.184:3300/0 conn(0x7f58cc0b0380 0x7f58cc0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d0d24640 1 -- 172.21.15.40:0/3762140807 >> v2:172.21.15.184:3300/0 conn(0x7f58cc0b0380 msgr2=0x7f58cc0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d0d24640 1 --2- 172.21.15.40:0/3762140807 >> v2:172.21.15.184:3300/0 conn(0x7f58cc0b0380 0x7f58cc0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d0d24640 1 -- 172.21.15.40:0/3762140807 >> v1:172.21.15.40:6789/0 conn(0x7f58cc0b10b0 legacy=0x7f58cc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58d0d24640 1 -- 172.21.15.40:0/3762140807 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc0d0300 con 0x7f58cc06c950 2024-04-26T21:44:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.499+0000 7f58caffd640 1 -- 172.21.15.40:0/3762140807 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037181030 0 0) 0x7f58cc0cfd80 con 0x7f58cc0b10b0 2024-04-26T21:44:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.500+0000 7f58d0d24640 1 --2- 172.21.15.40:0/3762140807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc06c950 0x7f58cc15e970 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f58cc06d290 tx=0x7f58c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9622ce1935bcd27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.500+0000 7f58caffd640 1 -- 172.21.15.40:0/3762140807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4036070 con 0x7f58cc06c950 2024-04-26T21:44:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.500+0000 7f58caffd640 1 -- 172.21.15.40:0/3762140807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c4031600 con 0x7f58cc06c950 2024-04-26T21:44:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.500+0000 7f58caffd640 1 -- 172.21.15.40:0/3762140807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c40319d0 con 0x7f58cc06c950 2024-04-26T21:44:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.501+0000 7f58d2787640 1 -- 172.21.15.40:0/3762140807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc06c950 msgr2=0x7f58cc15e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.501+0000 7f58d2787640 1 --2- 172.21.15.40:0/3762140807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc06c950 0x7f58cc15e970 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f58cc06d290 tx=0x7f58c402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.501+0000 7f58d2787640 1 -- 172.21.15.40:0/3762140807 shutdown_connections 2024-04-26T21:44:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.501+0000 7f58d2787640 1 --2- 172.21.15.40:0/3762140807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc06c950 0x7f58cc15e970 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.501+0000 7f58d2787640 1 --2- 172.21.15.40:0/3762140807 >> v2:172.21.15.184:3300/0 conn(0x7f58cc0b0380 0x7f58cc0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.501+0000 7f58d2787640 1 -- 172.21.15.40:0/3762140807 >> 172.21.15.40:0/3762140807 conn(0x7f58cc0d1cf0 msgr2=0x7f58cc0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.501+0000 7f58d2787640 1 -- 172.21.15.40:0/3762140807 shutdown_connections 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.501+0000 7f58d2787640 1 -- 172.21.15.40:0/3762140807 wait complete. 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.502+0000 7f58d2787640 1 Processor -- start 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.502+0000 7f58d2787640 1 -- start start 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d2787640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58cc06c950 0x7f58cc0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d2787640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc0b0380 0x7f58cc0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d2787640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc0b10b0 0x7f58cc0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d2787640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0b0760 con 0x7f58cc0b10b0 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d2787640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0d1980 con 0x7f58cc0b0380 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d2787640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc0d15c0 con 0x7f58cc06c950 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc0b10b0 0x7f58cc0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc0b10b0 0x7f58cc0cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43036/0 (socket says 172.21.15.40:43036) 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 -- 172.21.15.40:0/2898032089 learned_addr learned my addr 172.21.15.40:0/2898032089 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58cbfff640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58cc06c950 0x7f58cc0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58cb7fe640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc0b0380 0x7f58cc0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 -- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58cc06c950 msgr2=0x7f58cc0c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58cc06c950 0x7f58cc0c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 -- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc0b0380 msgr2=0x7f58cc0cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc0b0380 0x7f58cc0cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 -- 172.21.15.40:0/2898032089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc0c9d10 con 0x7f58cc0b10b0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58cbfff640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58cc06c950 0x7f58cc0c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58cb7fe640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc0b0380 0x7f58cc0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58d0d24640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc0b10b0 0x7f58cc0cba50 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f58c402ac90 tx=0x7f58c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58c8ff9640 1 -- 172.21.15.40:0/2898032089 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c403f020 con 0x7f58cc0b10b0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.503+0000 7f58c8ff9640 1 -- 172.21.15.40:0/2898032089 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c4037040 con 0x7f58cc0b10b0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.504+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc0cc300 con 0x7f58cc0b10b0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.504+0000 7f58c8ff9640 1 -- 172.21.15.40:0/2898032089 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4036070 con 0x7f58cc0b10b0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.504+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc0c6aa0 con 0x7f58cc0b10b0 2024-04-26T21:44:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.505+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc0694f0 con 0x7f58cc0b10b0 2024-04-26T21:44:22.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.505+0000 7f58c8ff9640 1 -- 172.21.15.40:0/2898032089 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58c4042020 con 0x7f58cc0b10b0 2024-04-26T21:44:22.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.505+0000 7f58c8ff9640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58ac06c8f0 0x7f58ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.506+0000 7f58c8ff9640 1 -- 172.21.15.40:0/2898032089 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f58c40ac7b0 con 0x7f58cc0b10b0 2024-04-26T21:44:22.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.506+0000 7f58cbfff640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58ac06c8f0 0x7f58ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.506+0000 7f58cbfff640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58ac06c8f0 0x7f58ac06edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f58bc006fd0 tx=0x7f58bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:22.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.510+0000 7f58c8ff9640 1 -- 172.21.15.40:0/2898032089 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f58c4049030 con 0x7f58cc0b10b0 2024-04-26T21:44:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.734+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58cc0d2200 con 0x7f58cc0b10b0 2024-04-26T21:44:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.734+0000 7f58c8ff9640 1 -- 172.21.15.40:0/2898032089 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 201 v201) ==== 95+0+4355 (secure 0 0 0) 0x7f58c407a460 con 0x7f58cc0b10b0 2024-04-26T21:44:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:22.736 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":201,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:22.393567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 201 2024-04-26T21:44:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58ac06c8f0 msgr2=0x7f58ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58ac06c8f0 0x7f58ac06edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f58bc006fd0 tx=0x7f58bc008040 comp rx=0 tx=0).stop 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc0b10b0 msgr2=0x7f58cc0cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc0b10b0 0x7f58cc0cba50 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f58c402ac90 tx=0x7f58c4004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 shutdown_connections 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f58ac06c8f0 0x7f58ac06edb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58cc0b10b0 0x7f58cc0cba50 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc0b0380 0x7f58cc0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 --2- 172.21.15.40:0/2898032089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58cc06c950 0x7f58cc0c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 >> 172.21.15.40:0/2898032089 conn(0x7f58cc0d1cf0 msgr2=0x7f58cc0b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.736+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 shutdown_connections 2024-04-26T21:44:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.737+0000 7f58d2787640 1 -- 172.21.15.40:0/2898032089 wait complete. 2024-04-26T21:44:22.748 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:22.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.889+0000 7ff992a15640 1 Processor -- start 2024-04-26T21:44:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.889+0000 7ff992a15640 1 -- start start 2024-04-26T21:44:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff992a15640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0d0070 0x7ff98c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff992a15640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff98c06e270 0x7ff98c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff992a15640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff98c0d2420 con 0x7ff98c06d560 2024-04-26T21:44:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff992a15640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff98c162d00 con 0x7ff98c06e270 2024-04-26T21:44:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff992a15640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff98c0d2d10 con 0x7ff98c0d0070 2024-04-26T21:44:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff990fb2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff98c06e270 0x7ff98c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff990fb2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff98c06e270 0x7ff98c0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43038/0 (socket says 172.21.15.40:43038) 2024-04-26T21:44:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff990fb2640 1 -- 172.21.15.40:0/2192747133 learned_addr learned my addr 172.21.15.40:0/2192747133 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff98bfff640 1 --2- 172.21.15.40:0/2192747133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0d0070 0x7ff98c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff990fb2640 1 -- 172.21.15.40:0/2192747133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0d0070 msgr2=0x7ff98c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff990fb2640 1 --2- 172.21.15.40:0/2192747133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0d0070 0x7ff98c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff990fb2640 1 -- 172.21.15.40:0/2192747133 >> v1:172.21.15.184:6789/0 conn(0x7ff98c06d560 legacy=0x7ff98c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.890+0000 7ff990fb2640 1 -- 172.21.15.40:0/2192747133 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98c0cd6b0 con 0x7ff98c06e270 2024-04-26T21:44:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.891+0000 7ff990fb2640 1 --2- 172.21.15.40:0/2192747133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c06e270 0x7ff98c0ce190 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff984009e50 tx=0x7ff98402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f60e19ae5744c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.891+0000 7ff98affd640 1 -- 172.21.15.40:0/2192747133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984036070 con 0x7ff98c06e270 2024-04-26T21:44:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.891+0000 7ff98affd640 1 -- 172.21.15.40:0/2192747133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff984031600 con 0x7ff98c06e270 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.891+0000 7ff98affd640 1 -- 172.21.15.40:0/2192747133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9840319d0 con 0x7ff98c06e270 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.892+0000 7ff992a15640 1 -- 172.21.15.40:0/2192747133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c06e270 msgr2=0x7ff98c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.892+0000 7ff992a15640 1 --2- 172.21.15.40:0/2192747133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c06e270 0x7ff98c0ce190 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff984009e50 tx=0x7ff98402a7a0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.892+0000 7ff992a15640 1 -- 172.21.15.40:0/2192747133 shutdown_connections 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.892+0000 7ff992a15640 1 --2- 172.21.15.40:0/2192747133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c06e270 0x7ff98c0ce190 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.892+0000 7ff992a15640 1 --2- 172.21.15.40:0/2192747133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0d0070 0x7ff98c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.892+0000 7ff992a15640 1 -- 172.21.15.40:0/2192747133 >> 172.21.15.40:0/2192747133 conn(0x7ff98c0d2ec0 msgr2=0x7ff98c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.892+0000 7ff992a15640 1 -- 172.21.15.40:0/2192747133 shutdown_connections 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.892+0000 7ff992a15640 1 -- 172.21.15.40:0/2192747133 wait complete. 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.893+0000 7ff992a15640 1 Processor -- start 2024-04-26T21:44:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.893+0000 7ff992a15640 1 -- start start 2024-04-26T21:44:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff992a15640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c06d560 0x7ff98c0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff992a15640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06e270 0x7ff98c0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff992a15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c0d0070 0x7ff98c1f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff992a15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c0ce6d0 con 0x7ff98c0d0070 2024-04-26T21:44:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff992a15640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c0cdc70 con 0x7ff98c06e270 2024-04-26T21:44:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff992a15640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff98c0ce850 con 0x7ff98c06d560 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c0d0070 0x7ff98c1f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c0d0070 0x7ff98c1f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43054/0 (socket says 172.21.15.40:43054) 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 -- 172.21.15.40:0/1264052993 learned_addr learned my addr 172.21.15.40:0/1264052993 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff98bfff640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c06d560 0x7ff98c0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff98b7fe640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06e270 0x7ff98c0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 -- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c06d560 msgr2=0x7ff98c0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c06d560 0x7ff98c0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 -- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06e270 msgr2=0x7ff98c0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06e270 0x7ff98c0c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 -- 172.21.15.40:0/1264052993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98c0c7170 con 0x7ff98c0d0070 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff98bfff640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c06d560 0x7ff98c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff98b7fe640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06e270 0x7ff98c0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.894+0000 7ff990fb2640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c0d0070 0x7ff98c1f4d30 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff984009e50 tx=0x7ff984004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.895+0000 7ff988ff9640 1 -- 172.21.15.40:0/1264052993 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984036050 con 0x7ff98c0d0070 2024-04-26T21:44:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.895+0000 7ff988ff9640 1 -- 172.21.15.40:0/1264052993 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff984042070 con 0x7ff98c0d0070 2024-04-26T21:44:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.895+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98c0c7450 con 0x7ff98c0d0070 2024-04-26T21:44:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.895+0000 7ff988ff9640 1 -- 172.21.15.40:0/1264052993 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984009510 con 0x7ff98c0d0070 2024-04-26T21:44:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.895+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98c0c7a10 con 0x7ff98c0d0070 2024-04-26T21:44:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.896+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff98c0694f0 con 0x7ff98c0d0070 2024-04-26T21:44:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.896+0000 7ff988ff9640 1 -- 172.21.15.40:0/1264052993 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff984007b30 con 0x7ff98c0d0070 2024-04-26T21:44:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.897+0000 7ff988ff9640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff97806c8f0 0x7ff97806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.897+0000 7ff988ff9640 1 -- 172.21.15.40:0/1264052993 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7ff9840ad160 con 0x7ff98c0d0070 2024-04-26T21:44:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.897+0000 7ff98bfff640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff97806c8f0 0x7ff97806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.897+0000 7ff98bfff640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff97806c8f0 0x7ff97806edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff97c006fd0 tx=0x7ff97c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:22.901+0000 7ff988ff9640 1 -- 172.21.15.40:0/1264052993 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff984007e20 con 0x7ff98c0d0070 2024-04-26T21:44:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.123+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff98c1626a0 con 0x7ff98c0d0070 2024-04-26T21:44:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.123+0000 7ff988ff9640 1 -- 172.21.15.40:0/1264052993 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 201 v201) ==== 95+0+4355 (secure 0 0 0) 0x7ff98407ae10 con 0x7ff98c0d0070 2024-04-26T21:44:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:23.124 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":201,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:22.393567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 201 2024-04-26T21:44:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff97806c8f0 msgr2=0x7ff97806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff97806c8f0 0x7ff97806edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff97c006fd0 tx=0x7ff97c008040 comp rx=0 tx=0).stop 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c0d0070 msgr2=0x7ff98c1f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c0d0070 0x7ff98c1f4d30 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff984009e50 tx=0x7ff984004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 shutdown_connections 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff97806c8f0 0x7ff97806edb0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff98c0d0070 0x7ff98c1f4d30 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06e270 0x7ff98c0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 --2- 172.21.15.40:0/1264052993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c06d560 0x7ff98c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 >> 172.21.15.40:0/1264052993 conn(0x7ff98c0d2ec0 msgr2=0x7ff98c160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.125+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 shutdown_connections 2024-04-26T21:44:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.126+0000 7ff992a15640 1 -- 172.21.15.40:0/1264052993 wait complete. 2024-04-26T21:44:23.137 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:23.137 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.238+0000 7fce866a0640 1 Processor -- start 2024-04-26T21:44:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.238+0000 7fce866a0640 1 -- start start 2024-04-26T21:44:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce866a0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4250 0x7fce800d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce866a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce800b4e90 0x7fce800b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce866a0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce80163de0 con 0x7fce800c80c0 2024-04-26T21:44:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce866a0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fce800d06c0 con 0x7fce800b4e90 2024-04-26T21:44:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce866a0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fce800d0900 con 0x7fce800b4250 2024-04-26T21:44:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce84c3d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce800b4e90 0x7fce800b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce7f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fce800c80c0 0x7fce800d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59374/0 (socket says 172.21.15.40:59374) 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce7f7fe640 1 -- 172.21.15.40:0/3918727618 learned_addr learned my addr 172.21.15.40:0/3918727618 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce7ffff640 1 --2- 172.21.15.40:0/3918727618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4250 0x7fce800d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce84c3d640 1 -- 172.21.15.40:0/3918727618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4250 msgr2=0x7fce800d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce84c3d640 1 --2- 172.21.15.40:0/3918727618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4250 0x7fce800d1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce84c3d640 1 -- 172.21.15.40:0/3918727618 >> v1:172.21.15.40:6789/0 conn(0x7fce800c80c0 legacy=0x7fce800d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.239+0000 7fce84c3d640 1 -- 172.21.15.40:0/3918727618 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce800d0be0 con 0x7fce800b4e90 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.240+0000 7fce84c3d640 1 --2- 172.21.15.40:0/3918727618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4e90 0x7fce800b5320 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fce78009d90 tx=0x7fce7802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11512a3f0a14c6a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.240+0000 7fce7effd640 1 -- 172.21.15.40:0/3918727618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78031600 con 0x7fce800b4e90 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.240+0000 7fce7effd640 1 -- 172.21.15.40:0/3918727618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce780317a0 con 0x7fce800b4e90 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.240+0000 7fce7effd640 1 -- 172.21.15.40:0/3918727618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78031b70 con 0x7fce800b4e90 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.241+0000 7fce866a0640 1 -- 172.21.15.40:0/3918727618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4e90 msgr2=0x7fce800b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.241+0000 7fce866a0640 1 --2- 172.21.15.40:0/3918727618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4e90 0x7fce800b5320 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fce78009d90 tx=0x7fce7802a5d0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.241+0000 7fce866a0640 1 -- 172.21.15.40:0/3918727618 shutdown_connections 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.241+0000 7fce866a0640 1 --2- 172.21.15.40:0/3918727618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4e90 0x7fce800b5320 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.241+0000 7fce866a0640 1 --2- 172.21.15.40:0/3918727618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4250 0x7fce800d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.241+0000 7fce866a0640 1 -- 172.21.15.40:0/3918727618 >> 172.21.15.40:0/3918727618 conn(0x7fce800d4f90 msgr2=0x7fce800d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.241+0000 7fce866a0640 1 -- 172.21.15.40:0/3918727618 shutdown_connections 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.241+0000 7fce866a0640 1 -- 172.21.15.40:0/3918727618 wait complete. 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.242+0000 7fce866a0640 1 Processor -- start 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.242+0000 7fce866a0640 1 -- start start 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce866a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4250 0x7fce800ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce866a0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4e90 0x7fce800c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce866a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce800c80c0 0x7fce800b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce866a0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fce800b4970 con 0x7fce800b4250 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce866a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce800d11a0 con 0x7fce800c80c0 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce866a0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fce800b4630 con 0x7fce800b4e90 2024-04-26T21:44:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4250 0x7fce800ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4250 0x7fce800ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43084/0 (socket says 172.21.15.40:43084) 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 -- 172.21.15.40:0/2944294153 learned_addr learned my addr 172.21.15.40:0/2944294153 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7f7fe640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4e90 0x7fce800c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 -- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4e90 msgr2=0x7fce800c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4e90 0x7fce800c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce84c3d640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce800c80c0 0x7fce800b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 -- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce800c80c0 msgr2=0x7fce800b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce800c80c0 0x7fce800b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 -- 172.21.15.40:0/2944294153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce800cb600 con 0x7fce800b4250 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.243+0000 7fce7ffff640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4250 0x7fce800ce590 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fce7400c7f0 tx=0x7fce7400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.244+0000 7fce7cff9640 1 -- 172.21.15.40:0/2944294153 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74015070 con 0x7fce800b4250 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.244+0000 7fce7cff9640 1 -- 172.21.15.40:0/2944294153 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce74004220 con 0x7fce800b4250 2024-04-26T21:44:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.244+0000 7fce7cff9640 1 -- 172.21.15.40:0/2944294153 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74007620 con 0x7fce800b4250 2024-04-26T21:44:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.244+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce800cb8e0 con 0x7fce800b4250 2024-04-26T21:44:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.244+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce800cc300 con 0x7fce800b4250 2024-04-26T21:44:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.245+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce800694f0 con 0x7fce800b4250 2024-04-26T21:44:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.245+0000 7fce7cff9640 1 -- 172.21.15.40:0/2944294153 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce740079d0 con 0x7fce800b4250 2024-04-26T21:44:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.246+0000 7fce7cff9640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce6c06c920 0x7fce6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.246+0000 7fce7cff9640 1 -- 172.21.15.40:0/2944294153 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7fce7408c360 con 0x7fce800b4250 2024-04-26T21:44:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.246+0000 7fce7f7fe640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce6c06c920 0x7fce6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.246+0000 7fce7f7fe640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce6c06c920 0x7fce6c06ede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.250+0000 7fce7cff9640 1 -- 172.21.15.40:0/2944294153 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce74059f80 con 0x7fce800b4250 2024-04-26T21:44:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.479+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce800d3a60 con 0x7fce800b4250 2024-04-26T21:44:23.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.479+0000 7fce7cff9640 1 -- 172.21.15.40:0/2944294153 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) ==== 95+0+5181 (secure 0 0 0) 0x7fce74059880 con 0x7fce800b4250 2024-04-26T21:44:23.481 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:23.481 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":202,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9705,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1239220748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1239220748},{"type":"v1","addr":"172.21.15.40:6833","nonce":1239220748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:22.393567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:23.482 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 202 2024-04-26T21:44:23.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce6c06c920 msgr2=0x7fce6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce6c06c920 0x7fce6c06ede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).stop 2024-04-26T21:44:23.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4250 msgr2=0x7fce800ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4250 0x7fce800ce590 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fce7400c7f0 tx=0x7fce7400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 shutdown_connections 2024-04-26T21:44:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fce6c06c920 0x7fce6c06ede0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce800c80c0 0x7fce800b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce800b4e90 0x7fce800c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 --2- 172.21.15.40:0/2944294153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce800b4250 0x7fce800ce590 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 >> 172.21.15.40:0/2944294153 conn(0x7fce800d4f90 msgr2=0x7fce800d1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 shutdown_connections 2024-04-26T21:44:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.482+0000 7fce866a0640 1 -- 172.21.15.40:0/2944294153 wait complete. 2024-04-26T21:44:23.493 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000001' led to DAMAGED state 2024-04-26T21:44:23.494 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 500.00000000' 2024-04-26T21:44:23.494 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:23.494 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:23.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:23.509 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:23.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:23.510 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:23.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:23.565 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:23.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.705+0000 7f33a3e15640 1 Processor -- start 2024-04-26T21:44:23.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.706+0000 7f33a3e15640 1 -- start start 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.706+0000 7f33a3e15640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f339c1608f0 0x7f339c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.706+0000 7f33a3e15640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f339c0cffa0 0x7f339c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.706+0000 7f33a3e15640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f339c0cde20 con 0x7f339c0d0a90 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.706+0000 7f33a3e15640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f339c0cea30 con 0x7f339c1608f0 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.706+0000 7f33a3e15640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f339c0ce100 con 0x7f339c0cffa0 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.707+0000 7f33a1bb1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f339c1608f0 0x7f339c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.707+0000 7f33a13b0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f339c0d0a90 0x7f339c0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59386/0 (socket says 172.21.15.40:59386) 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.707+0000 7f33a13b0640 1 -- 172.21.15.40:0/4261266873 learned_addr learned my addr 172.21.15.40:0/4261266873 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.707+0000 7f33a1bb1640 1 -- 172.21.15.40:0/4261266873 >> v2:172.21.15.184:3300/0 conn(0x7f339c0cffa0 msgr2=0x7f339c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.707+0000 7f33a1bb1640 1 --2- 172.21.15.40:0/4261266873 >> v2:172.21.15.184:3300/0 conn(0x7f339c0cffa0 0x7f339c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.707+0000 7f33a1bb1640 1 -- 172.21.15.40:0/4261266873 >> v1:172.21.15.40:6789/0 conn(0x7f339c0d0a90 legacy=0x7f339c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.707+0000 7f33a0baf640 1 -- 172.21.15.40:0/4261266873 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4252626675 0 0) 0x7f339c0cde20 con 0x7f339c0d0a90 2024-04-26T21:44:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.707+0000 7f33a1bb1640 1 -- 172.21.15.40:0/4261266873 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c0ce3e0 con 0x7f339c1608f0 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a1bb1640 1 --2- 172.21.15.40:0/4261266873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c1608f0 0x7f339c0d1bf0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f339c073370 tx=0x7f339802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb33e995cd8c66a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a0baf640 1 -- 172.21.15.40:0/4261266873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398037070 con 0x7f339c1608f0 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a0baf640 1 -- 172.21.15.40:0/4261266873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3398032440 con 0x7f339c1608f0 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a0baf640 1 -- 172.21.15.40:0/4261266873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398032810 con 0x7f339c1608f0 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a3e15640 1 -- 172.21.15.40:0/4261266873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c1608f0 msgr2=0x7f339c0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a3e15640 1 --2- 172.21.15.40:0/4261266873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c1608f0 0x7f339c0d1bf0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f339c073370 tx=0x7f339802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a3e15640 1 -- 172.21.15.40:0/4261266873 shutdown_connections 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a3e15640 1 --2- 172.21.15.40:0/4261266873 >> v2:172.21.15.184:3300/0 conn(0x7f339c0cffa0 0x7f339c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a3e15640 1 --2- 172.21.15.40:0/4261266873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c1608f0 0x7f339c0d1bf0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a3e15640 1 -- 172.21.15.40:0/4261266873 >> 172.21.15.40:0/4261266873 conn(0x7f339c0d2ec0 msgr2=0x7f339c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a3e15640 1 -- 172.21.15.40:0/4261266873 shutdown_connections 2024-04-26T21:44:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.708+0000 7f33a3e15640 1 -- 172.21.15.40:0/4261266873 wait complete. 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a3e15640 1 Processor -- start 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a3e15640 1 -- start start 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a3e15640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f339c0cffa0 0x7f339c158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a3e15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c0d0a90 0x7f339c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a3e15640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c1608f0 0x7f339c15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a3e15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f339c0ceec0 con 0x7f339c0d0a90 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a3e15640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f339c0d0ef0 con 0x7f339c1608f0 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a3e15640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f339c0cda40 con 0x7f339c0cffa0 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a13b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c0d0a90 0x7f339c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a13b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c0d0a90 0x7f339c15e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43106/0 (socket says 172.21.15.40:43106) 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.710+0000 7f33a13b0640 1 -- 172.21.15.40:0/2717363049 learned_addr learned my addr 172.21.15.40:0/2717363049 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a23b2640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c1608f0 0x7f339c15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a1bb1640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f339c0cffa0 0x7f339c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a13b0640 1 -- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f339c0cffa0 msgr2=0x7f339c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a13b0640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f339c0cffa0 0x7f339c158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a13b0640 1 -- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c1608f0 msgr2=0x7f339c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a13b0640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c1608f0 0x7f339c15eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a13b0640 1 -- 172.21.15.40:0/2717363049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c158980 con 0x7f339c0d0a90 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a23b2640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c1608f0 0x7f339c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a13b0640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c0d0a90 0x7f339c15e650 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f338c00c430 tx=0x7f338c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33927fc640 1 -- 172.21.15.40:0/2717363049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c0074e0 con 0x7f339c0d0a90 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33927fc640 1 -- 172.21.15.40:0/2717363049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f338c0047e0 con 0x7f339c0d0a90 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33927fc640 1 -- 172.21.15.40:0/2717363049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c004bd0 con 0x7f339c0d0a90 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c15f480 con 0x7f339c0d0a90 2024-04-26T21:44:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.711+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c0cc740 con 0x7f339c0d0a90 2024-04-26T21:44:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.712+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c0694f0 con 0x7f339c0d0a90 2024-04-26T21:44:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.712+0000 7f33927fc640 1 -- 172.21.15.40:0/2717363049 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f338c011070 con 0x7f339c0d0a90 2024-04-26T21:44:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.713+0000 7f33927fc640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f338006c8f0 0x7f338006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.713+0000 7f33927fc640 1 -- 172.21.15.40:0/2717363049 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f338c08c040 con 0x7f339c0d0a90 2024-04-26T21:44:23.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.713+0000 7f33a1bb1640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f338006c8f0 0x7f338006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.714+0000 7f33a1bb1640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f338006c8f0 0x7f338006edb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3398034000 tx=0x7f339802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.717+0000 7f33927fc640 1 -- 172.21.15.40:0/2717363049 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f339c158980 con 0x7f339c0d0a90 2024-04-26T21:44:23.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:23.942+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f339c0c6600 con 0x7f339c0d0a90 2024-04-26T21:44:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.404+0000 7f33927fc640 1 -- 172.21.15.40:0/2717363049 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v203) ==== 161+0+0 (secure 0 0 0) 0x7f338c059bd0 con 0x7f339c0d0a90 2024-04-26T21:44:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f338006c8f0 msgr2=0x7f338006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f338006c8f0 0x7f338006edb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3398034000 tx=0x7f339802e000 comp rx=0 tx=0).stop 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c0d0a90 msgr2=0x7f339c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c0d0a90 0x7f339c15e650 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f338c00c430 tx=0x7f338c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 shutdown_connections 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f338006c8f0 0x7f338006edb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c1608f0 0x7f339c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f339c0d0a90 0x7f339c15e650 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 --2- 172.21.15.40:0/2717363049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f339c0cffa0 0x7f339c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 >> 172.21.15.40:0/2717363049 conn(0x7f339c0d2ec0 msgr2=0x7f339c160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:24.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 shutdown_connections 2024-04-26T21:44:24.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.407+0000 7f33a3e15640 1 -- 172.21.15.40:0/2717363049 wait complete. 2024-04-26T21:44:24.419 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:24.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.521+0000 7faaf3196640 1 Processor -- start 2024-04-26T21:44:24.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.522+0000 7faaf3196640 1 -- start start 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.522+0000 7faaf3196640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faaec15f3a0 0x7faaec0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.522+0000 7faaf3196640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d1770 0x7faaec0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.522+0000 7faaf3196640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaec0d25a0 con 0x7faaec0d2130 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.522+0000 7faaf3196640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faaec0d0e20 con 0x7faaec15f3a0 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.522+0000 7faaf3196640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faaec0b1d70 con 0x7faaec0d1770 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaf0f32640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faaec15f3a0 0x7faaec0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaebfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d1770 0x7faaec0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaf0f32640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faaec15f3a0 0x7faaec0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:55450/0 (socket says 172.21.15.40:55450) 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaf0f32640 1 -- 172.21.15.40:0/2897224634 learned_addr learned my addr 172.21.15.40:0/2897224634 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaebfff640 1 --2- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d1770 0x7faaec0d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:36502/0 (socket says 172.21.15.40:36502) 2024-04-26T21:44:24.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaf1733640 1 --1- 172.21.15.40:0/2897224634 >> v1:172.21.15.184:6789/0 conn(0x7faaec0d2130 0x7faaec0d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39466/0 (socket says 172.21.15.40:39466) 2024-04-26T21:44:24.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaf0f32640 1 -- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d1770 msgr2=0x7faaec0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:24.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaf0f32640 1 --2- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d1770 0x7faaec0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaf0f32640 1 -- 172.21.15.40:0/2897224634 >> v1:172.21.15.184:6789/0 conn(0x7faaec0d2130 legacy=0x7faaec0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:24.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaf0f32640 1 -- 172.21.15.40:0/2897224634 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec0b2050 con 0x7faaec15f3a0 2024-04-26T21:44:24.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.523+0000 7faaebfff640 1 --2- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d1770 0x7faaec0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:24.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.524+0000 7faaf0f32640 1 --2- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 0x7faaec0d06f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faae4009870 tx=0x7faae402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afa0a9c17b51d7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.524+0000 7faaeb7fe640 1 -- 172.21.15.40:0/2897224634 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4037070 con 0x7faaec15f3a0 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.524+0000 7faaeb7fe640 1 -- 172.21.15.40:0/2897224634 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae4032440 con 0x7faaec15f3a0 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.524+0000 7faaeb7fe640 1 -- 172.21.15.40:0/2897224634 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4032810 con 0x7faaec15f3a0 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.525+0000 7faaf3196640 1 -- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 msgr2=0x7faaec0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.525+0000 7faaf3196640 1 --2- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 0x7faaec0d06f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faae4009870 tx=0x7faae402a270 comp rx=0 tx=0).stop 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.525+0000 7faaf3196640 1 -- 172.21.15.40:0/2897224634 shutdown_connections 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.525+0000 7faaf3196640 1 --2- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d1770 0x7faaec0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.525+0000 7faaf3196640 1 --2- 172.21.15.40:0/2897224634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 0x7faaec0d06f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.525+0000 7faaf3196640 1 -- 172.21.15.40:0/2897224634 >> 172.21.15.40:0/2897224634 conn(0x7faaec0d4f90 msgr2=0x7faaec0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.525+0000 7faaf3196640 1 -- 172.21.15.40:0/2897224634 shutdown_connections 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.525+0000 7faaf3196640 1 -- 172.21.15.40:0/2897224634 wait complete. 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.526+0000 7faaf3196640 1 Processor -- start 2024-04-26T21:44:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.526+0000 7faaf3196640 1 -- start start 2024-04-26T21:44:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf3196640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0d1770 0x7faaec0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf3196640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d2130 0x7faaec0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf3196640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 0x7faaec06cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf3196640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faaec15f900 con 0x7faaec0d1770 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf3196640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faaec15f780 con 0x7faaec15f3a0 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf3196640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faaec0b3860 con 0x7faaec0d2130 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf0f32640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0d1770 0x7faaec0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf0f32640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0d1770 0x7faaec0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43118/0 (socket says 172.21.15.40:43118) 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf0f32640 1 -- 172.21.15.40:0/3149247644 learned_addr learned my addr 172.21.15.40:0/3149247644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaebfff640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d2130 0x7faaec0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf1733640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 0x7faaec06cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf0f32640 1 -- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d2130 msgr2=0x7faaec0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf0f32640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d2130 0x7faaec0c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf0f32640 1 -- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 msgr2=0x7faaec06cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf0f32640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 0x7faaec06cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.527+0000 7faaf0f32640 1 -- 172.21.15.40:0/3149247644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec0ca1d0 con 0x7faaec0d1770 2024-04-26T21:44:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.528+0000 7faaf1733640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 0x7faaec06cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.528+0000 7faaf0f32640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0d1770 0x7faaec0c9c90 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7faae402a2b0 tx=0x7faae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.528+0000 7faae97fa640 1 -- 172.21.15.40:0/3149247644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4037040 con 0x7faaec0d1770 2024-04-26T21:44:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.528+0000 7faae97fa640 1 -- 172.21.15.40:0/3149247644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae4042070 con 0x7faaec0d1770 2024-04-26T21:44:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.528+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec0cc100 con 0x7faaec0d1770 2024-04-26T21:44:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.528+0000 7faae97fa640 1 -- 172.21.15.40:0/3149247644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4030880 con 0x7faaec0d1770 2024-04-26T21:44:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.528+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec0cc6c0 con 0x7faaec0d1770 2024-04-26T21:44:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.529+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec0694f0 con 0x7faaec0d1770 2024-04-26T21:44:24.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.529+0000 7faae97fa640 1 -- 172.21.15.40:0/3149247644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faae4030a20 con 0x7faaec0d1770 2024-04-26T21:44:24.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.530+0000 7faae97fa640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faacc06c8f0 0x7faacc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:24.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.530+0000 7faae97fa640 1 -- 172.21.15.40:0/3149247644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7faae40acb20 con 0x7faaec0d1770 2024-04-26T21:44:24.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.530+0000 7faaebfff640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faacc06c8f0 0x7faacc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:24.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.531+0000 7faaebfff640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faacc06c8f0 0x7faacc06edb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faadc009bd0 tx=0x7faadc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:24.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.534+0000 7faae97fa640 1 -- 172.21.15.40:0/3149247644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faae407a740 con 0x7faaec0d1770 2024-04-26T21:44:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:24.797+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7faaec0d1c80 con 0x7faaec0d1770 2024-04-26T21:44:25.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.406+0000 7faae97fa640 1 -- 172.21.15.40:0/3149247644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v204) ==== 101+0+0 (secure 0 0 0) 0x7faae407a040 con 0x7faaec0d1770 2024-04-26T21:44:25.407 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faacc06c8f0 msgr2=0x7faacc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faacc06c8f0 0x7faacc06edb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faadc009bd0 tx=0x7faadc00b040 comp rx=0 tx=0).stop 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0d1770 msgr2=0x7faaec0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0d1770 0x7faaec0c9c90 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7faae402a2b0 tx=0x7faae40046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 shutdown_connections 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faacc06c8f0 0x7faacc06edb0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaec15f3a0 0x7faaec06cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaec0d2130 0x7faaec0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 --2- 172.21.15.40:0/3149247644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0d1770 0x7faaec0c9c90 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 >> 172.21.15.40:0/3149247644 conn(0x7faaec0d4f90 msgr2=0x7faaec0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:25.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 shutdown_connections 2024-04-26T21:44:25.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.409+0000 7faaf3196640 1 -- 172.21.15.40:0/3149247644 wait complete. 2024-04-26T21:44:25.424 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:25.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.468+0000 7f7ef040ff40 1 Processor -- start 2024-04-26T21:44:25.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.468+0000 7f7ef040ff40 1 -- start start 2024-04-26T21:44:25.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7ef040ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f09860f60 0x561f09819110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7ef040ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f098d7d60 0x561f0985dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7ef040ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098f5610 0x561f0985e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7ef040ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561f0981adc0 con 0x561f098d7d60 2024-04-26T21:44:25.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7ef040ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561f098d8140 con 0x561f09860f60 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7ef040ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561f0981b060 con 0x561f098f5610 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f098d7d60 0x561f0985dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f098d7d60 0x561f0985dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43146/0 (socket says 172.21.15.40:43146) 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 -- 172.21.15.40:0/878154546 learned_addr learned my addr 172.21.15.40:0/878154546 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eed279640 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f09860f60 0x561f09819110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeda7a640 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098f5610 0x561f0985e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 -- 172.21.15.40:0/878154546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098f5610 msgr2=0x561f0985e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098f5610 0x561f0985e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 -- 172.21.15.40:0/878154546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f09860f60 msgr2=0x561f09819110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f09860f60 0x561f09819110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 -- 172.21.15.40:0/878154546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f09744cb0 con 0x561f098d7d60 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7eeca78640 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f098d7d60 0x561f0985dc00 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009860 tx=0x7f7ee4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab078f964ecfe6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.469+0000 7f7edf7fe640 1 -- 172.21.15.40:0/878154546 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee40088d0 con 0x561f098d7d60 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7edf7fe640 1 -- 172.21.15.40:0/878154546 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee4018070 con 0x561f098d7d60 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7edf7fe640 1 -- 172.21.15.40:0/878154546 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4008cb0 con 0x561f098d7d60 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 -- 172.21.15.40:0/878154546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f098d7d60 msgr2=0x561f0985dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f098d7d60 0x561f0985dc00 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009860 tx=0x7f7ee4005a00 comp rx=0 tx=0).stop 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 -- 172.21.15.40:0/878154546 shutdown_connections 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098f5610 0x561f0985e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f098d7d60 0x561f0985dc00 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/878154546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f09860f60 0x561f09819110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 -- 172.21.15.40:0/878154546 >> 172.21.15.40:0/878154546 conn(0x561f09818ab0 msgr2=0x561f098efd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 -- 172.21.15.40:0/878154546 shutdown_connections 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.470+0000 7f7ef040ff40 1 -- 172.21.15.40:0/878154546 wait complete. 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.471+0000 7f7ef040ff40 1 Processor -- start 2024-04-26T21:44:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.471+0000 7f7ef040ff40 1 -- start start 2024-04-26T21:44:25.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.471+0000 7f7ef040ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f09860f60 0x561f098ef070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.471+0000 7f7ef040ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098d7d60 0x561f098d4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.471+0000 7f7ef040ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f098f5610 0x561f098ec520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7ef040ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561f098493a0 con 0x561f09860f60 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7ef040ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561f098f3cf0 con 0x561f098f5610 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7ef040ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561f098d7230 con 0x561f098d7d60 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f09860f60 0x561f098ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f09860f60 0x561f098ef070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43148/0 (socket says 172.21.15.40:43148) 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 learned_addr learned my addr 172.21.15.40:0/637274914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eeca78640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098d7d60 0x561f098d4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eeda7a640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f098f5610 0x561f098ec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098d7d60 msgr2=0x561f098d4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561f098d7d60 0x561f098d4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f098f5610 msgr2=0x561f098ec520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561f098f5610 0x561f098ec520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f098ecfb0 con 0x561f09860f60 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7eed279640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f09860f60 0x561f098ef070 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0008840 tx=0x7f7ee0008930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7ede7fc640 1 -- 172.21.15.40:0/637274914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00147e0 con 0x561f09860f60 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7ede7fc640 1 -- 172.21.15.40:0/637274914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee001a070 con 0x561f09860f60 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7ede7fc640 1 -- 172.21.15.40:0/637274914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0014bc0 con 0x561f09860f60 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f098ed250 con 0x561f09860f60 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.472+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f098ecc70 con 0x561f09860f60 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.474+0000 7f7ede7fc640 1 -- 172.21.15.40:0/637274914 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ee0015470 con 0x561f09860f60 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.474+0000 7f7ede7fc640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ed006b2d0 0x7f7ed006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.474+0000 7f7ede7fc640 1 -- 172.21.15.40:0/637274914 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f7ee008b8c0 con 0x561f09860f60 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.475+0000 7f7eeca78640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ed006b2d0 0x7f7ed006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.477 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.475+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x561f09817f80 0x561f09818360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.475+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099881b0 con 0x561f09817f80 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.475+0000 7f7eeda7a640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x561f09817f80 0x561f09818360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.475+0000 7f7eeca78640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ed006b2d0 0x7f7ed006d790 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x561f09861850 tx=0x7f7ee4011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.475+0000 7f7eeda7a640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x561f09817f80 0x561f09818360 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.477+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v113'536 uv510 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.477+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099885b0 con 0x561f09817f80 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.478+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v113'537 uv537 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.478+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09988950 con 0x561f09817f80 2024-04-26T21:44:25.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.479+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v113'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.480 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:25.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.479+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099958b0 con 0x561f09817f80 2024-04-26T21:44:25.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.480+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v113'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.480+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995c50 con 0x561f09817f80 2024-04-26T21:44:25.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.481+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v113'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.481+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099948a0 con 0x561f09817f80 2024-04-26T21:44:25.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.482+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v113'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.482+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994c40 con 0x561f09817f80 2024-04-26T21:44:25.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.483+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v113'542 uv516 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.484+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f09995030 con 0x561f09817f80 2024-04-26T21:44:25.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.485+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v113'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.485+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995030 con 0x561f09817f80 2024-04-26T21:44:25.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.486+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v113'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.487 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:25.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.486+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.487+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v113'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.487+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.488+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v113'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.488+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.489+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v113'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.489+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.490+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v113'548 uv523 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.490+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.491+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v113'549 uv549 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.491+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.493 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:25.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.492+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v113'550 uv550 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.492+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.493+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v113'551 uv551 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.493+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.494+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v113'552 uv552 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.494+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09994fe0 con 0x561f09817f80 2024-04-26T21:44:25.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.495+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v113'553 uv553 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.495+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099953c0 con 0x561f09817f80 2024-04-26T21:44:25.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.496+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v113'554 uv554 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.496+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099953a0 con 0x561f09817f80 2024-04-26T21:44:25.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.497+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v113'555 uv529 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.497+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.499+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v113'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.499+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.500 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.500+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v113'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.500+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.501+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v113'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.501+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.502+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v113'559 uv559 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.502+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.503+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v113'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.503+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.504+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v113'561 uv535 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.504+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.505+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v113'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.505+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.506 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.506+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v113'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.506+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.507+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v113'564 uv564 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.507+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.508+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v113'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.508+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09817f80 2024-04-26T21:44:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.509+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v113'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f09817f80 2024-04-26T21:44:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.509+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x561f09996500 0x561f099988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.509+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09995290 con 0x561f09996500 2024-04-26T21:44:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.509+0000 7f7eed279640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x561f09996500 0x561f099988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.510+0000 7f7eed279640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x561f09996500 0x561f099988f0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.511+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v113'396 uv378 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.511+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f09998e30 con 0x561f09996500 2024-04-26T21:44:25.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.513+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v113'397 uv397 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.513+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09999290 con 0x561f09996500 2024-04-26T21:44:25.514 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:25.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.514+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v113'398 uv398 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.514+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09999b50 con 0x561f09996500 2024-04-26T21:44:25.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.515+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v113'399 uv399 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.515+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f09999ef0 con 0x561f09996500 2024-04-26T21:44:25.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.516+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v113'400 uv400 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.516+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999a290 con 0x561f09996500 2024-04-26T21:44:25.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.517+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v113'401 uv383 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.517+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f0999a680 con 0x561f09996500 2024-04-26T21:44:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.518+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v113'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.518+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa70 con 0x561f09996500 2024-04-26T21:44:25.520 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.520+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v113'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.520+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa70 con 0x561f09996500 2024-04-26T21:44:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.521+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v113'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.521+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.522+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v113'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.522+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.523+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v113'406 uv395 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.523+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.524+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v113'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.524+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.526 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.525+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v113'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.525+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.527+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v113'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.527+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.528+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v113'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.528+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.529+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v113'411 uv394 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.529+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.530+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v113'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.530+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa20 con 0x561f09996500 2024-04-26T21:44:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.531+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v113'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.532 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.531+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa40 con 0x561f09996500 2024-04-26T21:44:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.533+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v113'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.533+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa40 con 0x561f09996500 2024-04-26T21:44:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.534+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v113'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.534+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999aa40 con 0x561f09996500 2024-04-26T21:44:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.535+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v113'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f09996500 2024-04-26T21:44:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.535+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x561f0999b3a0 0x561f0999b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.535+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999dfe0 con 0x561f0999b3a0 2024-04-26T21:44:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.535+0000 7f7eeca78640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x561f0999b3a0 0x561f0999b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.536+0000 7f7eeca78640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x561f0999b3a0 0x561f0999b780 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.537+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v113'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.537+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f0999e380 con 0x561f0999b3a0 2024-04-26T21:44:25.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.538+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v113'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.538+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999e720 con 0x561f0999b3a0 2024-04-26T21:44:25.539 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:25.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.539+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v113'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.539+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999fb10 con 0x561f0999b3a0 2024-04-26T21:44:25.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.540+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v113'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.540+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999ff90 con 0x561f0999b3a0 2024-04-26T21:44:25.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.541+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v113'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.541+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f0999eac0 con 0x561f0999b3a0 2024-04-26T21:44:25.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.542+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v113'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.542+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561f0999f580 0x561f099a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.542+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a2930 con 0x561f0999f580 2024-04-26T21:44:25.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.543+0000 7f7eeda7a640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561f0999f580 0x561f099a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.543+0000 7f7eeda7a640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561f0999f580 0x561f099a23f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.544+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v113'84 uv83 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.544+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a2d20 con 0x561f0999f580 2024-04-26T21:44:25.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.546+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v113'85 uv85 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.546+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a2d20 con 0x561f0999f580 2024-04-26T21:44:25.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.547+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v113'86 uv86 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.548 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:25.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.547+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a2d20 con 0x561f0999f580 2024-04-26T21:44:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.548+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v113'87 uv87 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.548+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561f099a3620 0x561f099a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.548+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a5f50 con 0x561f099a3620 2024-04-26T21:44:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.548+0000 7f7eed279640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561f099a3620 0x561f099a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.549+0000 7f7eed279640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561f099a3620 0x561f099a5a10 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.550+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v113'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.550+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a5f50 con 0x561f099a3620 2024-04-26T21:44:25.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.551+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v113'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.551+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a5fd0 con 0x561f099a3620 2024-04-26T21:44:25.552 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:25.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.552+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v113'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.552+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a73d0 con 0x561f099a3620 2024-04-26T21:44:25.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.553+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v113'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.553+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a73d0 con 0x561f099a3620 2024-04-26T21:44:25.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.554+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v113'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.554+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.555+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v113'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.556+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.557+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v113'193 uv187 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.557+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.559+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v113'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.559+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.560 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:25.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.560+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v113'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.560+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a73d0 con 0x561f0999f580 2024-04-26T21:44:25.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.561+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v113'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.561+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a73d0 con 0x561f0999f580 2024-04-26T21:44:25.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.562+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v113'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.562+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.563+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v113'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.563+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.564+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v113'199 uv192 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.564+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.565+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v113'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.565+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.566 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:25.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.566+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v113'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.566+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.567+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v113'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.567+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.568+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v113'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.568+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.569+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v113'86 uv85 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.569+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.570+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v113'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.570+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.572 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:25.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.571+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v113'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.571+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.573+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v113'89 uv89 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.573+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f0999f580 2024-04-26T21:44:25.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.574+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v113'90 uv90 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ee8051860 con 0x561f0999f580 2024-04-26T21:44:25.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.574+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.575+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v113'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.575+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.576+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v113'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.576+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.577 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:25.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.577+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v113'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.577+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a73d0 con 0x561f099a3620 2024-04-26T21:44:25.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.578+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v113'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.578+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a73d0 con 0x561f099a3620 2024-04-26T21:44:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.579+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v113'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.579+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.580+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v113'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.580+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.581+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v113'172 uv166 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.581+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.583+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v113'173 uv173 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.583+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.584+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v113'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.585 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:25.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.584+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.585+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v113'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.585+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.586+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v113'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.586+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.587+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v113'177 uv171 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.587+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.588+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v113'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.588+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.589+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v113'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.590 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:25.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.589+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a63c0 con 0x561f099a3620 2024-04-26T21:44:25.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.590+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v113'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.590+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a91d0 con 0x561f099a3620 2024-04-26T21:44:25.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.592+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v113'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.592+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x561f099a6d80 0x561f099a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.592+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a97d0 con 0x561f099a6d80 2024-04-26T21:44:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.592+0000 7f7eeca78640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x561f099a6d80 0x561f099a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.592+0000 7f7eeca78640 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x561f099a6d80 0x561f099a9170 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.593+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v113'78 uv77 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f099a6d80 2024-04-26T21:44:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.593+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a9b70 con 0x561f099a6d80 2024-04-26T21:44:25.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.594+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v113'79 uv79 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f099a6d80 2024-04-26T21:44:25.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.594+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9b70 con 0x561f099a6d80 2024-04-26T21:44:25.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.596+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v113'80 uv80 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f099a6d80 2024-04-26T21:44:25.596 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:25.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.596+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f099a6d80 2024-04-26T21:44:25.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.597+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v113'81 uv81 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f099a6d80 2024-04-26T21:44:25.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.597+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f099a3620 2024-04-26T21:44:25.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.598+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v113'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.598+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f099a3620 2024-04-26T21:44:25.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.599+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v113'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.599+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f099a3620 2024-04-26T21:44:25.601 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:25.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.600+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v113'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.600+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f099a3620 2024-04-26T21:44:25.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.601+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v113'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.601+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f099a3620 2024-04-26T21:44:25.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.603+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v113'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.603+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f099a3620 2024-04-26T21:44:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.605+0000 7f7eed279640 1 -- 172.21.15.40:0/637274914 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v113'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee0014d60 con 0x561f099a3620 2024-04-26T21:44:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.605+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.605+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v113'89 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.605+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.606+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v113'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.606+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.608+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v113'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.608 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:25.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.608+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.608+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v113'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.609+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.610+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v113'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.610+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.611+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v113'216 uv208 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.611+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.612+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v113'217 uv217 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.612+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9c10 con 0x561f0999b3a0 2024-04-26T21:44:25.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.613+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v113'218 uv218 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.614 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:25.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.613+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9c10 con 0x561f0999b3a0 2024-04-26T21:44:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.615+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v113'219 uv219 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.615+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9c10 con 0x561f0999b3a0 2024-04-26T21:44:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.616+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v113'220 uv220 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.616+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9c10 con 0x561f0999b3a0 2024-04-26T21:44:25.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.617+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v113'221 uv215 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.617+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x561f099a9c10 con 0x561f0999b3a0 2024-04-26T21:44:25.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.618+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v113'222 uv222 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.618+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9c10 con 0x561f0999b3a0 2024-04-26T21:44:25.620 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:25.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.619+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v113'223 uv223 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.619+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9c10 con 0x561f0999b3a0 2024-04-26T21:44:25.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.620+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v113'224 uv224 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.620+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.621+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v113'225 uv225 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.621+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099a9bf0 con 0x561f0999b3a0 2024-04-26T21:44:25.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.622+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v113'226 uv226 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.622+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x561f099aa060 con 0x561f0999b3a0 2024-04-26T21:44:25.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7eeca78640 1 -- 172.21.15.40:0/637274914 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v113'227 uv227 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ee4008bd0 con 0x561f0999b3a0 2024-04-26T21:44:25.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x561f09996500 msgr2=0x561f099988f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x561f09996500 0x561f099988f0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x561f09817f80 msgr2=0x561f09818360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x561f09817f80 0x561f09818360 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561f0999f580 msgr2=0x561f099a23f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561f0999f580 0x561f099a23f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x561f099a6d80 msgr2=0x561f099a9170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x561f099a6d80 0x561f099a9170 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x561f0999b3a0 msgr2=0x561f0999b780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x561f0999b3a0 0x561f0999b780 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561f099a3620 msgr2=0x561f099a5a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x561f099a3620 0x561f099a5a10 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 reap_dead start 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 reap_dead start 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.623+0000 7f7eeda7a640 1 -- 172.21.15.40:0/637274914 reap_dead start 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ed006b2d0 msgr2=0x7f7ed006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ed006b2d0 0x7f7ed006d790 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x561f09861850 tx=0x7f7ee4011000 comp rx=0 tx=0).stop 2024-04-26T21:44:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f09860f60 msgr2=0x561f098ef070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f09860f60 0x561f098ef070 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0008840 tx=0x7f7ee0008930 comp rx=0 tx=0).stop 2024-04-26T21:44:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 shutdown_connections 2024-04-26T21:44:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7ed006b2d0 0x7f7ed006d790 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 --2- 172.21.15.40:0/637274914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f09860f60 0x561f098ef070 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 >> 172.21.15.40:0/637274914 conn(0x561f09818ab0 msgr2=0x561f098f0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 shutdown_connections 2024-04-26T21:44:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.624+0000 7f7ef040ff40 1 -- 172.21.15.40:0/637274914 wait complete. 2024-04-26T21:44:25.628 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 500.00000000 - 2024-04-26T21:44:25.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.671+0000 7f7f23c22f40 1 Processor -- start 2024-04-26T21:44:25.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.671+0000 7f7f23c22f40 1 -- start start 2024-04-26T21:44:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.671+0000 7f7f23c22f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c2b7f40 0x55e85c22cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.671+0000 7f7f23c22f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c22d1a0 0x55e85c2ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.671+0000 7f7f23c22f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.671+0000 7f7f23c22f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e85c1a5e40 con 0x55e85c23f360 2024-04-26T21:44:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.671+0000 7f7f23c22f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e85c1a9e80 con 0x55e85c2b7f40 2024-04-26T21:44:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.671+0000 7f7f23c22f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e85c246570 con 0x55e85c22d1a0 2024-04-26T21:44:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f2171c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c2b7f40 0x55e85c22cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43164/0 (socket says 172.21.15.40:43164) 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 -- 172.21.15.40:0/395899623 learned_addr learned my addr 172.21.15.40:0/395899623 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f2171c640 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c2b7f40 0x55e85c22cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:55500/0 (socket says 172.21.15.40:55500) 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f20f1b640 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c22d1a0 0x55e85c2ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 -- 172.21.15.40:0/395899623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c22d1a0 msgr2=0x55e85c2ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c22d1a0 0x55e85c2ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 -- 172.21.15.40:0/395899623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c2b7f40 msgr2=0x55e85c22cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c2b7f40 0x55e85c22cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 -- 172.21.15.40:0/395899623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85c2b9010 con 0x55e85c23f360 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f21f1d640 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cef00 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c050560 tx=0x7f7f1c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaaa87fa990af9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f1b7fe640 1 -- 172.21.15.40:0/395899623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c050a80 con 0x55e85c23f360 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f1b7fe640 1 -- 172.21.15.40:0/395899623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c050c20 con 0x55e85c23f360 2024-04-26T21:44:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.672+0000 7f7f1b7fe640 1 -- 172.21.15.40:0/395899623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c063440 con 0x55e85c23f360 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 -- 172.21.15.40:0/395899623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 msgr2=0x55e85c2cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cef00 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c050560 tx=0x7f7f1c051000 comp rx=0 tx=0).stop 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 -- 172.21.15.40:0/395899623 shutdown_connections 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cef00 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c22d1a0 0x55e85c2ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 --2- 172.21.15.40:0/395899623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c2b7f40 0x55e85c22cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 -- 172.21.15.40:0/395899623 >> 172.21.15.40:0/395899623 conn(0x55e85c1fcab0 msgr2=0x55e85c1fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 -- 172.21.15.40:0/395899623 shutdown_connections 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 -- 172.21.15.40:0/395899623 wait complete. 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 Processor -- start 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.673+0000 7f7f23c22f40 1 -- start start 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f23c22f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c22d1a0 0x55e85c1fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f23c22f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f23c22f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c2b7f40 0x55e85c2cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f23c22f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e85c23ebd0 con 0x55e85c23f360 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f23c22f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e85c2b8dc0 con 0x55e85c22d1a0 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f23c22f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e85c23eeb0 con 0x55e85c2b7f40 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f20f1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f20f1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43166/0 (socket says 172.21.15.40:43166) 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f20f1b640 1 -- 172.21.15.40:0/1990698502 learned_addr learned my addr 172.21.15.40:0/1990698502 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f21f1d640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c2b7f40 0x55e85c2cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f2171c640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c22d1a0 0x55e85c1fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f20f1b640 1 -- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c2b7f40 msgr2=0x55e85c2cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f20f1b640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e85c2b7f40 0x55e85c2cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f20f1b640 1 -- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c22d1a0 msgr2=0x55e85c1fd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f20f1b640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e85c22d1a0 0x55e85c1fd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.674+0000 7f7f20f1b640 1 -- 172.21.15.40:0/1990698502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85c2d52c0 con 0x55e85c23f360 2024-04-26T21:44:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.675+0000 7f7f20f1b640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cd8b0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009e60 tx=0x7f7f0c00f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.675+0000 7f7f1a7fc640 1 -- 172.21.15.40:0/1990698502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00e660 con 0x55e85c23f360 2024-04-26T21:44:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.675+0000 7f7f1a7fc640 1 -- 172.21.15.40:0/1990698502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0c00e800 con 0x55e85c23f360 2024-04-26T21:44:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.675+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e85c2d55a0 con 0x55e85c23f360 2024-04-26T21:44:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.675+0000 7f7f1a7fc640 1 -- 172.21.15.40:0/1990698502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00ebf0 con 0x55e85c23f360 2024-04-26T21:44:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.675+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85c2d5b60 con 0x55e85c23f360 2024-04-26T21:44:25.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.676+0000 7f7f1a7fc640 1 -- 172.21.15.40:0/1990698502 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f0c01a430 con 0x55e85c23f360 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.677+0000 7f7f1a7fc640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f0406b2d0 0x7f7f0406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.677+0000 7f7f1a7fc640 1 -- 172.21.15.40:0/1990698502 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f7f0c08bf80 con 0x55e85c23f360 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.677+0000 7f7f2171c640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f0406b2d0 0x7f7f0406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.677+0000 7f7f23c22f40 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e85c2b5b60 0x55e85c2d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.677+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55e85c2b4d30 con 0x55e85c2b5b60 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.677+0000 7f7f21f1d640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e85c2b5b60 0x55e85c2d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.677+0000 7f7f2171c640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f0406b2d0 0x7f7f0406d790 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7f14006fd0 tx=0x7f7f140068f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.678+0000 7f7f21f1d640 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e85c2b5b60 0x55e85c2d1c70 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f21f1d640 1 -- 172.21.15.40:0/1990698502 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 500.00000000 [writefull 0~80] v113'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7f1c050880 con 0x55e85c2b5b60 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e85c2b5b60 msgr2=0x55e85c2d1c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e85c2b5b60 0x55e85c2d1c70 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f0406b2d0 msgr2=0x7f7f0406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7f0406b2d0 0x7f7f0406d790 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7f14006fd0 tx=0x7f7f140068f0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 msgr2=0x55e85c2cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 --2- 172.21.15.40:0/1990698502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e85c23f360 0x55e85c2cd8b0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009e60 tx=0x7f7f0c00f820 comp rx=0 tx=0).stop 2024-04-26T21:44:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f21f1d640 1 -- 172.21.15.40:0/1990698502 reap_dead start 2024-04-26T21:44:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 shutdown_connections 2024-04-26T21:44:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 >> 172.21.15.40:0/1990698502 conn(0x55e85c1fcab0 msgr2=0x55e85c2b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 shutdown_connections 2024-04-26T21:44:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.679+0000 7f7f23c22f40 1 -- 172.21.15.40:0/1990698502 wait complete. 2024-04-26T21:44:25.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.825+0000 7f9780e2a640 1 Processor -- start 2024-04-26T21:44:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.825+0000 7f9780e2a640 1 -- start start 2024-04-26T21:44:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9780e2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b0380 0x7f977c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9780e2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f977c0b10b0 0x7f977c06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9780e2a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f977c06c780 0x7f977c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9780e2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f977c0cfd80 con 0x7f977c0b10b0 2024-04-26T21:44:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9780e2a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f977c0d0020 con 0x7f977c06c780 2024-04-26T21:44:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9780e2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f977c0d0300 con 0x7f977c0b0380 2024-04-26T21:44:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9779d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f977c0b10b0 0x7f977c06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9779d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f977c0b10b0 0x7f977c06c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43174/0 (socket says 172.21.15.40:43174) 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9779d9b640 1 -- 172.21.15.40:0/240369647 learned_addr learned my addr 172.21.15.40:0/240369647 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f977a59c640 1 --2- 172.21.15.40:0/240369647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b0380 0x7f977c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f977ad9d640 1 --2- 172.21.15.40:0/240369647 >> v2:172.21.15.184:3300/0 conn(0x7f977c06c780 0x7f977c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9779d9b640 1 -- 172.21.15.40:0/240369647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b0380 msgr2=0x7f977c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9779d9b640 1 --2- 172.21.15.40:0/240369647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b0380 0x7f977c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9779d9b640 1 -- 172.21.15.40:0/240369647 >> v2:172.21.15.184:3300/0 conn(0x7f977c06c780 msgr2=0x7f977c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9779d9b640 1 --2- 172.21.15.40:0/240369647 >> v2:172.21.15.184:3300/0 conn(0x7f977c06c780 0x7f977c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f9779d9b640 1 -- 172.21.15.40:0/240369647 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c0d05e0 con 0x7f977c0b10b0 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.826+0000 7f977ad9d640 1 --2- 172.21.15.40:0/240369647 >> v2:172.21.15.184:3300/0 conn(0x7f977c06c780 0x7f977c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.827+0000 7f9779d9b640 1 --2- 172.21.15.40:0/240369647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c0b10b0 0x7f977c06c240 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f977c0c57c0 tx=0x7f976402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d82df92804ff1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.827+0000 7f977959a640 1 -- 172.21.15.40:0/240369647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764037070 con 0x7f977c0b10b0 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.827+0000 7f977959a640 1 -- 172.21.15.40:0/240369647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9764032440 con 0x7f977c0b10b0 2024-04-26T21:44:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.827+0000 7f977959a640 1 -- 172.21.15.40:0/240369647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764032810 con 0x7f977c0b10b0 2024-04-26T21:44:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 -- 172.21.15.40:0/240369647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c0b10b0 msgr2=0x7f977c06c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 --2- 172.21.15.40:0/240369647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c0b10b0 0x7f977c06c240 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f977c0c57c0 tx=0x7f976402a140 comp rx=0 tx=0).stop 2024-04-26T21:44:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 -- 172.21.15.40:0/240369647 shutdown_connections 2024-04-26T21:44:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 --2- 172.21.15.40:0/240369647 >> v2:172.21.15.184:3300/0 conn(0x7f977c06c780 0x7f977c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 --2- 172.21.15.40:0/240369647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c0b10b0 0x7f977c06c240 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 --2- 172.21.15.40:0/240369647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b0380 0x7f977c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 -- 172.21.15.40:0/240369647 >> 172.21.15.40:0/240369647 conn(0x7f977c0d1cf0 msgr2=0x7f977c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 -- 172.21.15.40:0/240369647 shutdown_connections 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.828+0000 7f9780e2a640 1 -- 172.21.15.40:0/240369647 wait complete. 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.829+0000 7f9780e2a640 1 Processor -- start 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.829+0000 7f9780e2a640 1 -- start start 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f9780e2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c06c780 0x7f977c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f9780e2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f977c0b0380 0x7f977c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f9780e2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b10b0 0x7f977c070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f9780e2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f977c0d15c0 con 0x7f977c06c780 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f9780e2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f977c0b0760 con 0x7f977c0b0380 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f9780e2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f977c0d1740 con 0x7f977c0b10b0 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f977a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c06c780 0x7f977c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f977a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c06c780 0x7f977c15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43184/0 (socket says 172.21.15.40:43184) 2024-04-26T21:44:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f977a59c640 1 -- 172.21.15.40:0/3794374916 learned_addr learned my addr 172.21.15.40:0/3794374916 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f977a59c640 1 -- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b10b0 msgr2=0x7f977c070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f977a59c640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b10b0 0x7f977c070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f977a59c640 1 -- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f977c0b0380 msgr2=0x7f977c1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f977a59c640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f977c0b0380 0x7f977c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.830+0000 7f977a59c640 1 -- 172.21.15.40:0/3794374916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c06fd40 con 0x7f977c06c780 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.831+0000 7f977a59c640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c06c780 0x7f977c15d320 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f977400e9b0 tx=0x7f977400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.831+0000 7f9772ffd640 1 -- 172.21.15.40:0/3794374916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774017070 con 0x7f977c06c780 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.831+0000 7f9772ffd640 1 -- 172.21.15.40:0/3794374916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f977400c9a0 con 0x7f977c06c780 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.831+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c070020 con 0x7f977c06c780 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.831+0000 7f9772ffd640 1 -- 172.21.15.40:0/3794374916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977400cd90 con 0x7f977c06c780 2024-04-26T21:44:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.831+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c0705e0 con 0x7f977c06c780 2024-04-26T21:44:25.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.832+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c0694f0 con 0x7f977c06c780 2024-04-26T21:44:25.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.832+0000 7f9772ffd640 1 -- 172.21.15.40:0/3794374916 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f977401a070 con 0x7f977c06c780 2024-04-26T21:44:25.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.833+0000 7f9772ffd640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.833+0000 7f9772ffd640 1 -- 172.21.15.40:0/3794374916 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f977408c7d0 con 0x7f977c06c780 2024-04-26T21:44:25.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.833+0000 7f9779d9b640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.834+0000 7f9779d9b640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f975c06c8f0 0x7f975c06edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9764034000 tx=0x7f976402c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:25.837+0000 7f9772ffd640 1 -- 172.21.15.40:0/3794374916 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f977405a350 con 0x7f977c06c780 2024-04-26T21:44:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.062+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f977c0aea00 con 0x7f977c06c780 2024-04-26T21:44:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.414+0000 7f9772ffd640 1 -- 172.21.15.40:0/3794374916 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v205) ==== 162+0+0 (secure 0 0 0) 0x7f9774059c50 con 0x7f977c06c780 2024-04-26T21:44:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:26.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f975c06c8f0 msgr2=0x7f975c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f975c06c8f0 0x7f975c06edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9764034000 tx=0x7f976402c000 comp rx=0 tx=0).stop 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c06c780 msgr2=0x7f977c15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c06c780 0x7f977c15d320 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f977400e9b0 tx=0x7f977400ee70 comp rx=0 tx=0).stop 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 shutdown_connections 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f977c0b10b0 0x7f977c070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f977c0b0380 0x7f977c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 --2- 172.21.15.40:0/3794374916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f977c06c780 0x7f977c15d320 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 >> 172.21.15.40:0/3794374916 conn(0x7f977c0d1cf0 msgr2=0x7f977c0cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 shutdown_connections 2024-04-26T21:44:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.416+0000 7f9780e2a640 1 -- 172.21.15.40:0/3794374916 wait complete. 2024-04-26T21:44:26.422 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:26.422+0000 7f6907ec3640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-04-26T21:44:26.422 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:26.422+0000 7f6907ec3640 -1 mds.0.206 unhandled write error (22) Invalid argument, force readonly... 2024-04-26T21:44:26.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:26.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.532+0000 7fde00c12640 1 Processor -- start 2024-04-26T21:44:26.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.532+0000 7fde00c12640 1 -- start start 2024-04-26T21:44:26.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fde00c12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc15f3a0 0x7fddfc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fde00c12640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fddfc0d1770 0x7fddfc0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fde00c12640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddfc0d2130 0x7fddfc0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fde00c12640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fddfc0d25a0 con 0x7fddfc0d2130 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fde00c12640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fddfc0d0e20 con 0x7fddfc0d1770 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fde00c12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc0b1d70 con 0x7fddfc15f3a0 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fddfad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddfc0d2130 0x7fddfc0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fddfad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddfc0d2130 0x7fddfc0d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43198/0 (socket says 172.21.15.40:43198) 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.533+0000 7fddfad9d640 1 -- 172.21.15.40:0/2362254620 learned_addr learned my addr 172.21.15.40:0/2362254620 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.534+0000 7fddf9d9b640 1 --2- 172.21.15.40:0/2362254620 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0d1770 0x7fddfc0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.534+0000 7fddfa59c640 1 --2- 172.21.15.40:0/2362254620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc15f3a0 0x7fddfc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.534+0000 7fddfad9d640 1 -- 172.21.15.40:0/2362254620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc15f3a0 msgr2=0x7fddfc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:26.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.534+0000 7fddfad9d640 1 --2- 172.21.15.40:0/2362254620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc15f3a0 0x7fddfc0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.534+0000 7fddfad9d640 1 -- 172.21.15.40:0/2362254620 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0d1770 msgr2=0x7fddfc0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:26.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.534+0000 7fddfad9d640 1 --2- 172.21.15.40:0/2362254620 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0d1770 0x7fddfc0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.534+0000 7fddfad9d640 1 -- 172.21.15.40:0/2362254620 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc0b2050 con 0x7fddfc0d2130 2024-04-26T21:44:26.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.534+0000 7fddfad9d640 1 --2- 172.21.15.40:0/2362254620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc0d2130 0x7fddfc0d2e50 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fddfc073c50 tx=0x7fddec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be307747c728c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:26.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fddf959a640 1 -- 172.21.15.40:0/2362254620 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec036070 con 0x7fddfc0d2130 2024-04-26T21:44:26.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fddf959a640 1 -- 172.21.15.40:0/2362254620 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec031600 con 0x7fddfc0d2130 2024-04-26T21:44:26.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fddf959a640 1 -- 172.21.15.40:0/2362254620 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0319d0 con 0x7fddfc0d2130 2024-04-26T21:44:26.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 -- 172.21.15.40:0/2362254620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc0d2130 msgr2=0x7fddfc0d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 --2- 172.21.15.40:0/2362254620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc0d2130 0x7fddfc0d2e50 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fddfc073c50 tx=0x7fddec02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 -- 172.21.15.40:0/2362254620 shutdown_connections 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 --2- 172.21.15.40:0/2362254620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc0d2130 0x7fddfc0d2e50 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 --2- 172.21.15.40:0/2362254620 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0d1770 0x7fddfc0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 --2- 172.21.15.40:0/2362254620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc15f3a0 0x7fddfc0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 -- 172.21.15.40:0/2362254620 >> 172.21.15.40:0/2362254620 conn(0x7fddfc0d4f90 msgr2=0x7fddfc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 -- 172.21.15.40:0/2362254620 shutdown_connections 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.535+0000 7fde00c12640 1 -- 172.21.15.40:0/2362254620 wait complete. 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.536+0000 7fde00c12640 1 Processor -- start 2024-04-26T21:44:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fde00c12640 1 -- start start 2024-04-26T21:44:26.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fde00c12640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d1770 0x7fddfc153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:26.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fde00c12640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0d2130 0x7fddfc0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:26.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fde00c12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc15f3a0 0x7fddfc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:26.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fde00c12640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc15f900 con 0x7fddfc15f3a0 2024-04-26T21:44:26.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fde00c12640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc15f780 con 0x7fddfc0d1770 2024-04-26T21:44:26.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fde00c12640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc0d3390 con 0x7fddfc0d2130 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fddfad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc15f3a0 0x7fddfc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fddfad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc15f3a0 0x7fddfc15fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43206/0 (socket says 172.21.15.40:43206) 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.537+0000 7fddfad9d640 1 -- 172.21.15.40:0/4214583832 learned_addr learned my addr 172.21.15.40:0/4214583832 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddf9d9b640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0d2130 0x7fddfc0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddfa59c640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d1770 0x7fddfc153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddfad9d640 1 -- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0d2130 msgr2=0x7fddfc0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddfad9d640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0d2130 0x7fddfc0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddfad9d640 1 -- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d1770 msgr2=0x7fddfc153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddfad9d640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d1770 0x7fddfc153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddfad9d640 1 -- 172.21.15.40:0/4214583832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc0b8020 con 0x7fddfc15f3a0 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddfad9d640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc15f3a0 0x7fddfc15fa80 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fddec009f40 tx=0x7fddec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddf2ffd640 1 -- 172.21.15.40:0/4214583832 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec03f020 con 0x7fddfc15f3a0 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddf2ffd640 1 -- 172.21.15.40:0/4214583832 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec037040 con 0x7fddfc15f3a0 2024-04-26T21:44:26.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc0b8300 con 0x7fddfc15f3a0 2024-04-26T21:44:26.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fddf2ffd640 1 -- 172.21.15.40:0/4214583832 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec036070 con 0x7fddfc15f3a0 2024-04-26T21:44:26.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.538+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc0b88c0 con 0x7fddfc15f3a0 2024-04-26T21:44:26.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.539+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc0694f0 con 0x7fddfc15f3a0 2024-04-26T21:44:26.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.540+0000 7fddf2ffd640 1 -- 172.21.15.40:0/4214583832 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddec042020 con 0x7fddfc15f3a0 2024-04-26T21:44:26.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.540+0000 7fddf2ffd640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:26.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.540+0000 7fddf2ffd640 1 -- 172.21.15.40:0/4214583832 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7fddec0ac860 con 0x7fddfc15f3a0 2024-04-26T21:44:26.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.540+0000 7fddfa59c640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:26.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.541+0000 7fddfa59c640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdddc06c8f0 0x7fdddc06edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fddf4006fd0 tx=0x7fddf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:26.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.545+0000 7fddf2ffd640 1 -- 172.21.15.40:0/4214583832 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddec049030 con 0x7fddfc15f3a0 2024-04-26T21:44:26.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.770+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddfc0b6860 con 0x7fddfc15f3a0 2024-04-26T21:44:26.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.770+0000 7fddf2ffd640 1 -- 172.21.15.40:0/4214583832 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) ==== 95+0+5190 (secure 0 0 0) 0x7fddec07a480 con 0x7fddfc15f3a0 2024-04-26T21:44:26.771 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:26.771 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":206,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:26.418125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9705":{"gid":9705,"name":"c","rank":0,"incarnation":206,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/1239220748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1239220748},{"type":"v1","addr":"172.21.15.40:6833","nonce":1239220748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:26.771 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 206 2024-04-26T21:44:26.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdddc06c8f0 msgr2=0x7fdddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdddc06c8f0 0x7fdddc06edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fddf4006fd0 tx=0x7fddf4008040 comp rx=0 tx=0).stop 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc15f3a0 msgr2=0x7fddfc15fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc15f3a0 0x7fddfc15fa80 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fddec009f40 tx=0x7fddec004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 shutdown_connections 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc15f3a0 0x7fddfc15fa80 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0d2130 0x7fddfc0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 --2- 172.21.15.40:0/4214583832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d1770 0x7fddfc153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 >> 172.21.15.40:0/4214583832 conn(0x7fddfc0d4f90 msgr2=0x7fddfc06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.772+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 shutdown_connections 2024-04-26T21:44:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:26.773+0000 7fde00c12640 1 -- 172.21.15.40:0/4214583832 wait complete. 2024-04-26T21:44:27.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:27.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.291+0000 7f8b20a09640 1 Processor -- start 2024-04-26T21:44:27.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.291+0000 7f8b20a09640 1 -- start start 2024-04-26T21:44:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b20a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c06f790 0x7f8b1c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b20a09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b1c156840 con 0x7f8b1c156460 2024-04-26T21:44:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b20a09640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b1c0cbf20 con 0x7f8b1c072c50 2024-04-26T21:44:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b20a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c0cc200 con 0x7f8b1c06f790 2024-04-26T21:44:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8b1c156460 0x7f8b1c0cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59402/0 (socket says 172.21.15.40:59402) 2024-04-26T21:44:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1a59c640 1 -- 172.21.15.40:0/3867443526 learned_addr learned my addr 172.21.15.40:0/3867443526 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b19d9b640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c06f790 0x7f8b1c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3507635231 0 0) 0x7f8b1c156840 con 0x7f8b1c156460 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b04003520 con 0x7f8b1c156460 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1329734527 0 0) 0x7f8b04003520 con 0x7f8b1c156460 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c06f790 msgr2=0x7f8b1c06fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c06f790 0x7f8b1c06fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 >> v1:172.21.15.184:6789/0 conn(0x7f8b1c072c50 legacy=0x7f8b1c0730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c0cc4e0 con 0x7f8b1c156460 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f8b0c001670 con 0x7f8b1c156460 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 >> v1:172.21.15.40:6789/0 conn(0x7f8b1c156460 legacy=0x7f8b1c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040039e0 0x7f8b04003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c06f790 0x7f8b04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04004d40 0x7f8b040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0cc200 con 0x7f8b04004d40 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0400dcb0 con 0x7f8b040039e0 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.292+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0400deb0 con 0x7f8b1c06f790 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.293+0000 7f8b1ad9d640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04004d40 0x7f8b040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.293+0000 7f8b19d9b640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c06f790 0x7f8b04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.293+0000 7f8b1a59c640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040039e0 0x7f8b04003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.293+0000 7f8b1ad9d640 1 -- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c06f790 msgr2=0x7f8b04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.293+0000 7f8b1ad9d640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c06f790 0x7f8b04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.293+0000 7f8b1ad9d640 1 -- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040039e0 msgr2=0x7f8b04003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.293+0000 7f8b1ad9d640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040039e0 0x7f8b04003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.293+0000 7f8b1ad9d640 1 -- 172.21.15.40:0/3867443526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b04003520 con 0x7f8b04004d40 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b1ad9d640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04004d40 0x7f8b040051c0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001720 tx=0x7f8b04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c15e19450c49e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14001370 con 0x7f8b04004d40 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b140052d0 con 0x7f8b04004d40 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b1959a640 1 -- 172.21.15.40:0/3867443526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1402abc0 con 0x7f8b04004d40 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b20a09640 1 -- 172.21.15.40:0/3867443526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04004d40 msgr2=0x7f8b040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b20a09640 1 --2- 172.21.15.40:0/3867443526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04004d40 0x7f8b040051c0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001720 tx=0x7f8b04005830 comp rx=0 tx=0).stop 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b1ad9d640 1 -- 172.21.15.40:0/3867443526 reap_dead start 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b20a09640 1 -- 172.21.15.40:0/3867443526 shutdown_connections 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b20a09640 1 -- 172.21.15.40:0/3867443526 >> 172.21.15.40:0/3867443526 conn(0x7f8b1c0d0be0 msgr2=0x7f8b1c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b20a09640 1 -- 172.21.15.40:0/3867443526 shutdown_connections 2024-04-26T21:44:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.294+0000 7f8b20a09640 1 -- 172.21.15.40:0/3867443526 wait complete. 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.296+0000 7f8b20a09640 1 Processor -- start 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.296+0000 7f8b20a09640 1 -- start start 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.296+0000 7f8b20a09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b04004d40 0x7f8b1c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.296+0000 7f8b20a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0c3c10 0x7f8b1c0c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.296+0000 7f8b20a09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b1c149160 0x7f8b1c1495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.296+0000 7f8b20a09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0cbbe0 con 0x7f8b1c149160 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.296+0000 7f8b20a09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c072700 con 0x7f8b04004d40 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.296+0000 7f8b20a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c06e260 con 0x7f8b1c0c3c10 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b1c149160 0x7f8b1c1495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b1c149160 0x7f8b1c1495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43228/0 (socket says 172.21.15.40:43228) 2024-04-26T21:44:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 -- 172.21.15.40:0/2577507652 learned_addr learned my addr 172.21.15.40:0/2577507652 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b19d9b640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0c3c10 0x7f8b1c0c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1a59c640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b04004d40 0x7f8b1c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 -- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0c3c10 msgr2=0x7f8b1c0c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0c3c10 0x7f8b1c0c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 -- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b04004d40 msgr2=0x7f8b1c15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b04004d40 0x7f8b1c15c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 -- 172.21.15.40:0/2577507652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c15c940 con 0x7f8b1c149160 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b1ad9d640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b1c149160 0x7f8b1c1495e0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001840 tx=0x7f8b14001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b0affd640 1 -- 172.21.15.40:0/2577507652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14037040 con 0x7f8b1c149160 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b0affd640 1 -- 172.21.15.40:0/2577507652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14040070 con 0x7f8b1c149160 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b0affd640 1 -- 172.21.15.40:0/2577507652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14001370 con 0x7f8b1c149160 2024-04-26T21:44:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c15cca0 con 0x7f8b1c149160 2024-04-26T21:44:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.297+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c15d260 con 0x7f8b1c149160 2024-04-26T21:44:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.298+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c0694f0 con 0x7f8b1c149160 2024-04-26T21:44:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.299+0000 7f8b0affd640 1 -- 172.21.15.40:0/2577507652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b140052d0 con 0x7f8b1c149160 2024-04-26T21:44:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.299+0000 7f8b0affd640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.299+0000 7f8b0affd640 1 -- 172.21.15.40:0/2577507652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f8b140aba80 con 0x7f8b1c149160 2024-04-26T21:44:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.299+0000 7f8b1a59c640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.300+0000 7f8b1a59c640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8afc06c8f0 0x7f8afc06edb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c007fd0 tx=0x7f8b0c0078f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.303+0000 7f8b0affd640 1 -- 172.21.15.40:0/2577507652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b1c15c940 con 0x7f8b1c149160 2024-04-26T21:44:27.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.528+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b1c06de80 con 0x7f8b1c149160 2024-04-26T21:44:27.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.529+0000 7f8b0affd640 1 -- 172.21.15.40:0/2577507652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) ==== 95+0+5193 (secure 0 0 0) 0x7f8b140790e0 con 0x7f8b1c149160 2024-04-26T21:44:27.530 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:27.530 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":207,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":207,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:26.426788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9705":{"gid":9705,"name":"c","rank":0,"incarnation":206,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.40:6833/1239220748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1239220748},{"type":"v1","addr":"172.21.15.40:6833","nonce":1239220748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:27.530 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 207 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8afc06c8f0 msgr2=0x7f8afc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8afc06c8f0 0x7f8afc06edb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c007fd0 tx=0x7f8b0c0078f0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b1c149160 msgr2=0x7f8b1c1495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b1c149160 0x7f8b1c1495e0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001840 tx=0x7f8b14001120 comp rx=0 tx=0).stop 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 shutdown_connections 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b1c149160 0x7f8b1c1495e0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0c3c10 0x7f8b1c0c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 --2- 172.21.15.40:0/2577507652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b04004d40 0x7f8b1c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:27.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 >> 172.21.15.40:0/2577507652 conn(0x7f8b1c0d0be0 msgr2=0x7f8b1c150ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:27.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 shutdown_connections 2024-04-26T21:44:27.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:27.531+0000 7f8b20a09640 1 -- 172.21.15.40:0/2577507652 wait complete. 2024-04-26T21:44:27.542 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 207, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:44:26.426788+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9705}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9705': {'gid': 9705, 'name': 'c', 'rank': 0, 'incarnation': 206, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.40:6833/1239220748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1239220748}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 1239220748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:44:27.543 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9705:up:reconnect 2024-04-26T21:44:27.543 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:44:32.544 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.657+0000 7fdbdfcc8640 1 Processor -- start 2024-04-26T21:44:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.658+0000 7fdbdfcc8640 1 -- start start 2024-04-26T21:44:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.658+0000 7fdbdfcc8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbd815f3a0 0x7fdbd80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.658+0000 7fdbdfcc8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd80d0e20 con 0x7fdbd80d2300 2024-04-26T21:44:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.658+0000 7fdbdfcc8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd80b1d70 con 0x7fdbd80d1770 2024-04-26T21:44:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.658+0000 7fdbdfcc8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd80b2050 con 0x7fdbd815f3a0 2024-04-26T21:44:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbde265640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdbd80d2300 0x7fdbd80b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56450/0 (socket says 172.21.15.40:56450) 2024-04-26T21:44:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbde265640 1 -- 172.21.15.40:0/3881239228 learned_addr learned my addr 172.21.15.40:0/3881239228 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:32.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdda64640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbd815f3a0 0x7fdbd80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:32.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3504419679 0 0) 0x7fdbd80d0e20 con 0x7fdbd80d2300 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbc0003520 con 0x7fdbd80d2300 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3412622914 0 0) 0x7fdbd80b1d70 con 0x7fdbd80d1770 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbd80d0e20 con 0x7fdbd80d1770 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1476218425 0 0) 0x7fdbc0003520 con 0x7fdbd80d2300 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbd815f3a0 msgr2=0x7fdbd80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbd815f3a0 0x7fdbd80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 >> v1:172.21.15.184:6789/0 conn(0x7fdbd80d1770 legacy=0x7fdbd80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd80b2330 con 0x7fdbd80d2300 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fdbc80019f0 con 0x7fdbd80d2300 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 >> v1:172.21.15.40:6789/0 conn(0x7fdbd80d2300 legacy=0x7fdbd80b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbc0003b70 0x7fdbc0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbd815f3a0 0x7fdbc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbc0004d40 0x7fdbc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80b2050 con 0x7fdbc0004d40 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc000ba70 con 0x7fdbc0003b70 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.659+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc000bc70 con 0x7fdbd815f3a0 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.660+0000 7fdbde265640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbc0004d40 0x7fdbc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.660+0000 7fdbdda64640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbc0003b70 0x7fdbc0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.660+0000 7fdbdd263640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbd815f3a0 0x7fdbc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.660+0000 7fdbde265640 1 -- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbd815f3a0 msgr2=0x7fdbc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.660+0000 7fdbde265640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbd815f3a0 0x7fdbc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.660+0000 7fdbde265640 1 -- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbc0003b70 msgr2=0x7fdbc0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.660+0000 7fdbde265640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbc0003b70 0x7fdbc0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.660+0000 7fdbde265640 1 -- 172.21.15.40:0/3881239228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc0003520 con 0x7fdbc0004d40 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbde265640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbc0004d40 0x7fdbc00051c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80d2750 tx=0x7fdbc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a37c90b803bc3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800a370 con 0x7fdbc0004d40 2024-04-26T21:44:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc802f070 con 0x7fdbc0004d40 2024-04-26T21:44:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbdca62640 1 -- 172.21.15.40:0/3881239228 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc802bb50 con 0x7fdbc0004d40 2024-04-26T21:44:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/3881239228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbc0004d40 msgr2=0x7fdbc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbdfcc8640 1 --2- 172.21.15.40:0/3881239228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbc0004d40 0x7fdbc00051c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80d2750 tx=0x7fdbc0005830 comp rx=0 tx=0).stop 2024-04-26T21:44:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbde265640 1 -- 172.21.15.40:0/3881239228 reap_dead start 2024-04-26T21:44:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/3881239228 shutdown_connections 2024-04-26T21:44:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/3881239228 >> 172.21.15.40:0/3881239228 conn(0x7fdbd80d4f90 msgr2=0x7fdbd80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.661+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/3881239228 shutdown_connections 2024-04-26T21:44:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.662+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/3881239228 wait complete. 2024-04-26T21:44:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.663+0000 7fdbdfcc8640 1 Processor -- start 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.663+0000 7fdbdfcc8640 1 -- start start 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.663+0000 7fdbdfcc8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbc0004d40 0x7fdbd81625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.663+0000 7fdbdfcc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd8161500 0x7fdbd8151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdfcc8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd80b5460 0x7fdbd80b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdfcc8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80d0870 con 0x7fdbd8161500 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdfcc8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80d06f0 con 0x7fdbd80b5460 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdfcc8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd80d09f0 con 0x7fdbc0004d40 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd8161500 0x7fdbd8151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd8161500 0x7fdbd8151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34342/0 (socket says 172.21.15.40:34342) 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 -- 172.21.15.40:0/1953411281 learned_addr learned my addr 172.21.15.40:0/1953411281 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdda64640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbc0004d40 0x7fdbd81625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbde265640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd80b5460 0x7fdbd80b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 -- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbc0004d40 msgr2=0x7fdbd81625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbc0004d40 0x7fdbd81625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 -- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd80b5460 msgr2=0x7fdbd80b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd80b5460 0x7fdbd80b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 -- 172.21.15.40:0/1953411281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd80c9c20 con 0x7fdbd8161500 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdda64640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbc0004d40 0x7fdbd81625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdd263640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd8161500 0x7fdbd8151360 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00f840 tx=0x7fdbcc00fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbd67fc640 1 -- 172.21.15.40:0/1953411281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0052e0 con 0x7fdbd8161500 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbd67fc640 1 -- 172.21.15.40:0/1953411281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc018070 con 0x7fdbd8161500 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd80c9f00 con 0x7fdbd8161500 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbd67fc640 1 -- 172.21.15.40:0/1953411281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00d910 con 0x7fdbd8161500 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.664+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd80ca4c0 con 0x7fdbd8161500 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.666+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd80694f0 con 0x7fdbd8161500 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.666+0000 7fdbd67fc640 1 -- 172.21.15.40:0/1953411281 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbcc00dab0 con 0x7fdbd8161500 2024-04-26T21:44:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.666+0000 7fdbd67fc640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc406c8f0 0x7fdbc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.666+0000 7fdbd67fc640 1 -- 172.21.15.40:0/1953411281 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7fdbcc08c2e0 con 0x7fdbd8161500 2024-04-26T21:44:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.667+0000 7fdbdda64640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc406c8f0 0x7fdbc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.667+0000 7fdbdda64640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc406c8f0 0x7fdbc406edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009bc0 tx=0x7fdbd0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.671+0000 7fdbd67fc640 1 -- 172.21.15.40:0/1953411281 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdbcc059f00 con 0x7fdbd8161500 2024-04-26T21:44:32.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.898+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbd815ebc0 con 0x7fdbd8161500 2024-04-26T21:44:32.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.899+0000 7fdbd67fc640 1 -- 172.21.15.40:0/1953411281 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5197 (secure 0 0 0) 0x7fdbcc0600d0 con 0x7fdbd8161500 2024-04-26T21:44:32.899 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:32.899 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":209,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:29.428218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9705":{"gid":9705,"name":"c","rank":0,"incarnation":206,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/1239220748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1239220748},{"type":"v1","addr":"172.21.15.40:6833","nonce":1239220748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9705,"qdb_cluster":[9705]},"id":6}]} 2024-04-26T21:44:32.899 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 209 2024-04-26T21:44:32.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc406c8f0 msgr2=0x7fdbc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:32.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc406c8f0 0x7fdbc406edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009bc0 tx=0x7fdbd0008040 comp rx=0 tx=0).stop 2024-04-26T21:44:32.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd8161500 msgr2=0x7fdbd8151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:32.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd8161500 0x7fdbd8151360 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00f840 tx=0x7fdbcc00fd00 comp rx=0 tx=0).stop 2024-04-26T21:44:32.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 shutdown_connections 2024-04-26T21:44:32.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdbc406c8f0 0x7fdbc406edb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd80b5460 0x7fdbd80b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd8161500 0x7fdbd8151360 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 --2- 172.21.15.40:0/1953411281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbc0004d40 0x7fdbd81625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:32.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 >> 172.21.15.40:0/1953411281 conn(0x7fdbd80d4f90 msgr2=0x7fdbd806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:32.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 shutdown_connections 2024-04-26T21:44:32.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:32.901+0000 7fdbdfcc8640 1 -- 172.21.15.40:0/1953411281 wait complete. 2024-04-26T21:44:33.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:33.402 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:33.403+0000 7f690aec9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:44:33.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.418+0000 7ff3fce52640 1 Processor -- start 2024-04-26T21:44:33.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.418+0000 7ff3fce52640 1 -- start start 2024-04-26T21:44:33.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3fce52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3f815f3a0 0x7ff3f80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3fce52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d1770 0x7ff3f80d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3fce52640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f80d25a0 con 0x7ff3f80d2130 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3fce52640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f80d0e20 con 0x7ff3f815f3a0 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3fce52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f80b1d70 con 0x7ff3f80d1770 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3f815f3a0 0x7ff3f80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3f815f3a0 0x7ff3f80d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34352/0 (socket says 172.21.15.40:34352) 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f659c640 1 -- 172.21.15.40:0/3840379304 learned_addr learned my addr 172.21.15.40:0/3840379304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f6d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff3f80d2130 0x7ff3f80d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56460/0 (socket says 172.21.15.40:56460) 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3effff640 1 --2- 172.21.15.40:0/3840379304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d1770 0x7ff3f80d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f659c640 1 -- 172.21.15.40:0/3840379304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d1770 msgr2=0x7ff3f80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f659c640 1 --2- 172.21.15.40:0/3840379304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d1770 0x7ff3f80d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f659c640 1 -- 172.21.15.40:0/3840379304 >> v1:172.21.15.40:6789/0 conn(0x7ff3f80d2130 legacy=0x7ff3f80d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f659c640 1 -- 172.21.15.40:0/3840379304 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f80b2050 con 0x7ff3f815f3a0 2024-04-26T21:44:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3f5d9b640 1 -- 172.21.15.40:0/3840379304 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2348107776 0 0) 0x7ff3f0001490 con 0x7ff3f80d2130 2024-04-26T21:44:33.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.419+0000 7ff3effff640 1 --2- 172.21.15.40:0/3840379304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d1770 0x7ff3f80d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:33.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.420+0000 7ff3f659c640 1 --2- 172.21.15.40:0/3840379304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f815f3a0 0x7ff3f80d06f0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009870 tx=0x7ff3e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c593eba3d1a301 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.420+0000 7ff3f5d9b640 1 -- 172.21.15.40:0/3840379304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8037070 con 0x7ff3f815f3a0 2024-04-26T21:44:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.420+0000 7ff3f5d9b640 1 -- 172.21.15.40:0/3840379304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e8032440 con 0x7ff3f815f3a0 2024-04-26T21:44:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.420+0000 7ff3f5d9b640 1 -- 172.21.15.40:0/3840379304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8032810 con 0x7ff3f815f3a0 2024-04-26T21:44:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.421+0000 7ff3fce52640 1 -- 172.21.15.40:0/3840379304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f815f3a0 msgr2=0x7ff3f80d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.421+0000 7ff3fce52640 1 --2- 172.21.15.40:0/3840379304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f815f3a0 0x7ff3f80d06f0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009870 tx=0x7ff3e802a270 comp rx=0 tx=0).stop 2024-04-26T21:44:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.421+0000 7ff3fce52640 1 -- 172.21.15.40:0/3840379304 shutdown_connections 2024-04-26T21:44:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.421+0000 7ff3fce52640 1 --2- 172.21.15.40:0/3840379304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d1770 0x7ff3f80d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.421+0000 7ff3fce52640 1 --2- 172.21.15.40:0/3840379304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f815f3a0 0x7ff3f80d06f0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.421+0000 7ff3fce52640 1 -- 172.21.15.40:0/3840379304 >> 172.21.15.40:0/3840379304 conn(0x7ff3f80d4f90 msgr2=0x7ff3f80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.421+0000 7ff3fce52640 1 -- 172.21.15.40:0/3840379304 shutdown_connections 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.421+0000 7ff3fce52640 1 -- 172.21.15.40:0/3840379304 wait complete. 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.422+0000 7ff3fce52640 1 Processor -- start 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.422+0000 7ff3fce52640 1 -- start start 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3fce52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f80d1770 0x7ff3f80cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3fce52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d2130 0x7ff3f8161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3fce52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f815f3a0 0x7ff3f8161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3fce52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f815f900 con 0x7ff3f80d1770 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3fce52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f815f780 con 0x7ff3f815f3a0 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3fce52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f80b3860 con 0x7ff3f80d2130 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f80d1770 0x7ff3f80cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f80d1770 0x7ff3f80cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34356/0 (socket says 172.21.15.40:34356) 2024-04-26T21:44:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f659c640 1 -- 172.21.15.40:0/2229944692 learned_addr learned my addr 172.21.15.40:0/2229944692 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f6d9d640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f815f3a0 0x7ff3f8161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3effff640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d2130 0x7ff3f8161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f659c640 1 -- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d2130 msgr2=0x7ff3f8161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f659c640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d2130 0x7ff3f8161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f659c640 1 -- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f815f3a0 msgr2=0x7ff3f8161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f659c640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f815f3a0 0x7ff3f8161fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.423+0000 7ff3f659c640 1 -- 172.21.15.40:0/2229944692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f80ce910 con 0x7ff3f80d1770 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.424+0000 7ff3f659c640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f80d1770 0x7ff3f80cca50 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8035040 tx=0x7ff3e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.424+0000 7ff3eeffd640 1 -- 172.21.15.40:0/2229944692 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8037040 con 0x7ff3f80d1770 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.424+0000 7ff3eeffd640 1 -- 172.21.15.40:0/2229944692 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e8004230 con 0x7ff3f80d1770 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.424+0000 7ff3eeffd640 1 -- 172.21.15.40:0/2229944692 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e803c670 con 0x7ff3f80d1770 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.424+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f80cebf0 con 0x7ff3f80d1770 2024-04-26T21:44:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.424+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f80c9570 con 0x7ff3f80d1770 2024-04-26T21:44:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.425+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f80694f0 con 0x7ff3f80d1770 2024-04-26T21:44:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.425+0000 7ff3eeffd640 1 -- 172.21.15.40:0/2229944692 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3e803a070 con 0x7ff3f80d1770 2024-04-26T21:44:33.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.426+0000 7ff3eeffd640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3d806c8f0 0x7ff3d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.426+0000 7ff3eeffd640 1 -- 172.21.15.40:0/2229944692 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7ff3e80acb20 con 0x7ff3f80d1770 2024-04-26T21:44:33.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.426+0000 7ff3effff640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3d806c8f0 0x7ff3d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.426+0000 7ff3effff640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3d806c8f0 0x7ff3d806edb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff3f815cf50 tx=0x7ff3e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.430+0000 7ff3eeffd640 1 -- 172.21.15.40:0/2229944692 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3e8041020 con 0x7ff3f80d1770 2024-04-26T21:44:33.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.652+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3f80b6860 con 0x7ff3f80d1770 2024-04-26T21:44:33.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.652+0000 7ff3eeffd640 1 -- 172.21.15.40:0/2229944692 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5197 (secure 0 0 0) 0x7ff3e807a050 con 0x7ff3f80d1770 2024-04-26T21:44:33.652 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:33.652 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":209,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:29.428218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9705":{"gid":9705,"name":"c","rank":0,"incarnation":206,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/1239220748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1239220748},{"type":"v1","addr":"172.21.15.40:6833","nonce":1239220748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9705,"qdb_cluster":[9705]},"id":6}]} 2024-04-26T21:44:33.653 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 209 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.654+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3d806c8f0 msgr2=0x7ff3d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.654+0000 7ff3fce52640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3d806c8f0 0x7ff3d806edb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff3f815cf50 tx=0x7ff3e0008040 comp rx=0 tx=0).stop 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.654+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f80d1770 msgr2=0x7ff3f80cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.654+0000 7ff3fce52640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f80d1770 0x7ff3f80cca50 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8035040 tx=0x7ff3e80046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.655+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 shutdown_connections 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.655+0000 7ff3fce52640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3d806c8f0 0x7ff3d806edb0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.655+0000 7ff3fce52640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f815f3a0 0x7ff3f8161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.655+0000 7ff3fce52640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f80d2130 0x7ff3f8161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.655+0000 7ff3fce52640 1 --2- 172.21.15.40:0/2229944692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f80d1770 0x7ff3f80cca50 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.655+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 >> 172.21.15.40:0/2229944692 conn(0x7ff3f80d4f90 msgr2=0x7ff3f80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.655+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 shutdown_connections 2024-04-26T21:44:33.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.655+0000 7ff3fce52640 1 -- 172.21.15.40:0/2229944692 wait complete. 2024-04-26T21:44:33.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 209, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:44:29.428218+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9705}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9705': {'gid': 9705, 'name': 'c', 'rank': 0, 'incarnation': 206, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6833/1239220748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1239220748}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 1239220748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9705, 'qdb_cluster': [9705]} 2024-04-26T21:44:33.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:44:33.665 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:44:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.768+0000 7fa2e119d640 1 Processor -- start 2024-04-26T21:44:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.768+0000 7fa2e119d640 1 -- start start 2024-04-26T21:44:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.768+0000 7fa2e119d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.768+0000 7fa2e119d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2dc0b5f20 0x7fa2dc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.768+0000 7fa2e119d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2dc0b7e50 con 0x7fa2dc162880 2024-04-26T21:44:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.768+0000 7fa2e119d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa2dc0d0580 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.768+0000 7fa2e119d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc0d1800 con 0x7fa2dc0d3bc0 2024-04-26T21:44:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.769+0000 7fa2db59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2dc0b5f20 0x7fa2dc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.769+0000 7fa2db59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2dc0b5f20 0x7fa2dc0b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34368/0 (socket says 172.21.15.40:34368) 2024-04-26T21:44:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.769+0000 7fa2db59e640 1 -- 172.21.15.40:0/3658362979 learned_addr learned my addr 172.21.15.40:0/3658362979 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.769+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/3658362979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.769+0000 7fa2db59e640 1 -- 172.21.15.40:0/3658362979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 msgr2=0x7fa2dc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.769+0000 7fa2db59e640 1 --2- 172.21.15.40:0/3658362979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.769+0000 7fa2db59e640 1 -- 172.21.15.40:0/3658362979 >> v1:172.21.15.184:6789/0 conn(0x7fa2dc162880 legacy=0x7fa2dc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.769+0000 7fa2db59e640 1 -- 172.21.15.40:0/3658362979 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc0d07d0 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2db59e640 1 --2- 172.21.15.40:0/3658362979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc0b8300 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009e50 tx=0x7fa2d402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5412445e26453a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2da59c640 1 -- 172.21.15.40:0/3658362979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4036070 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2da59c640 1 -- 172.21.15.40:0/3658362979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d4031600 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2da59c640 1 -- 172.21.15.40:0/3658362979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40319d0 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2e119d640 1 -- 172.21.15.40:0/3658362979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 msgr2=0x7fa2dc0b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2e119d640 1 --2- 172.21.15.40:0/3658362979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc0b8300 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009e50 tx=0x7fa2d402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2e119d640 1 -- 172.21.15.40:0/3658362979 shutdown_connections 2024-04-26T21:44:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2e119d640 1 --2- 172.21.15.40:0/3658362979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc0b8300 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2e119d640 1 --2- 172.21.15.40:0/3658362979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2e119d640 1 -- 172.21.15.40:0/3658362979 >> 172.21.15.40:0/3658362979 conn(0x7fa2dc0d4f60 msgr2=0x7fa2dc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2e119d640 1 -- 172.21.15.40:0/3658362979 shutdown_connections 2024-04-26T21:44:33.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.770+0000 7fa2e119d640 1 -- 172.21.15.40:0/3658362979 wait complete. 2024-04-26T21:44:33.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.772+0000 7fa2e119d640 1 Processor -- start 2024-04-26T21:44:33.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.772+0000 7fa2e119d640 1 -- start start 2024-04-26T21:44:33.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.772+0000 7fa2e119d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.772+0000 7fa2e119d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.772+0000 7fa2e119d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2dc162880 0x7fa2dc06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.772+0000 7fa2e119d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc162c60 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.772+0000 7fa2e119d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0d0d90 con 0x7fa2dc162880 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.772+0000 7fa2e119d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc162de0 con 0x7fa2dc0d3bc0 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34378/0 (socket says 172.21.15.40:34378) 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 -- 172.21.15.40:0/2452465062 learned_addr learned my addr 172.21.15.40:0/2452465062 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2db59e640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2dc162880 0x7fa2dc06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 -- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 msgr2=0x7fa2dc0cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 -- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2dc162880 msgr2=0x7fa2dc06dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2dc162880 0x7fa2dc06dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 -- 172.21.15.40:0/2452465062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc0cebb0 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc159660 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00c430 tx=0x7fa2cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2d37fe640 1 -- 172.21.15.40:0/2452465062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc011790 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2d37fe640 1 -- 172.21.15.40:0/2452465062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc017070 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2d37fe640 1 -- 172.21.15.40:0/2452465062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc011b70 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2e119d640 1 -- 172.21.15.40:0/2452465062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc159eb0 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.773+0000 7fa2e119d640 1 -- 172.21.15.40:0/2452465062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc15a150 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.775+0000 7fa2d37fe640 1 -- 172.21.15.40:0/2452465062 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2cc011d10 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.775+0000 7fa2d37fe640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2bc06c8f0 0x7fa2bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.775+0000 7fa2d37fe640 1 -- 172.21.15.40:0/2452465062 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7fa2cc08c1a0 con 0x7fa2dc0b5f20 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.775+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2bc06c8f0 0x7fa2bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.776+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2bc06c8f0 0x7fa2bc06edb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0b7970 tx=0x7fa2c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.776+0000 7fa2e3f8e740 1 Processor -- start 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.776+0000 7fa2e3f8e740 1 -- start start 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b41820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b5f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b17120 0x55f725b5d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f725a7b740 con 0x55f725b41440 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f725a173a0 con 0x55f725b17120 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f725a77700 con 0x55f725b5ece0 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b41820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b41820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34380/0 (socket says 172.21.15.40:34380) 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2dad9d640 1 -- 172.21.15.40:0/2786466131 learned_addr learned my addr 172.21.15.40:0/2786466131 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b5f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.777+0000 7fa2db59e640 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b17120 0x55f725b5d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2dad9d640 1 -- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 msgr2=0x55f725b5f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b5f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2dad9d640 1 -- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b17120 msgr2=0x55f725b5d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b17120 0x55f725b5d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2dad9d640 1 -- 172.21.15.40:0/2786466131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f725a33750 con 0x55f725b41440 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2db59e640 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b17120 0x55f725b5d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b5f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b41820 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc055bb0 tx=0x7fa2cc059f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c0913049f682ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2d0ff9640 1 -- 172.21.15.40:0/2786466131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc062330 con 0x55f725b41440 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2d0ff9640 1 -- 172.21.15.40:0/2786466131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc057a50 con 0x55f725b41440 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2d0ff9640 1 -- 172.21.15.40:0/2786466131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc061ea0 con 0x55f725b41440 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2786466131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 msgr2=0x55f725b41820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.778+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b41820 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc055bb0 tx=0x7fa2cc059f10 comp rx=0 tx=0).stop 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2786466131 shutdown_connections 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b17120 0x55f725b5d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b5f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/2786466131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b41820 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2786466131 >> 172.21.15.40:0/2786466131 conn(0x55f725b13a50 msgr2=0x55f725b5c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2786466131 shutdown_connections 2024-04-26T21:44:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2786466131 wait complete. 2024-04-26T21:44:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 -- start start 2024-04-26T21:44:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.779+0000 7fa2e3f8e740 1 Processor -- start 2024-04-26T21:44:33.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b54920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b15080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b43ed0 0x55f725b155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f725b17ec0 con 0x55f725b41440 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f725a02fe0 con 0x55f725b43ed0 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f725b11980 con 0x55f725b5ece0 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2e3f8e740 10 client.0 _ll_register_callbacks cb 0x55f725a19150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b54920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b54920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34386/0 (socket says 172.21.15.40:34386) 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 -- 172.21.15.40:0/3029166155 learned_addr learned my addr 172.21.15.40:0/3029166155 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 -- 172.21.15.40:0/3029166155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 msgr2=0x55f725b15080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b15080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 -- 172.21.15.40:0/3029166155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b43ed0 msgr2=0x55f725b155c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b43ed0 0x55f725b155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 -- 172.21.15.40:0/3029166155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72578d2d0 con 0x55f725b41440 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b54920 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc055bb0 tx=0x7fa2cc003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424212ca76b816f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.780+0000 7fa2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3029166155 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc076070 con 0x55f725b41440 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3029166155 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7fa2cc0620a0 con 0x55f725b41440 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3029166155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2cc057a50 con 0x55f725b41440 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3029166155 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=209) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc058ba0 con 0x55f725b41440 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 10 client.9804 fetch_fsmap learned FSMap version 209 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3029166155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=209,monmap=2+}) -- 0x55f7258576b0 con 0x55f725b41440 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3029166155 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc057a50 con 0x55f725b41440 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3029166155 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 209) ==== 3494+0+0 (secure 0 0 0) 0x7fa2cc058d40 con 0x55f725b41440 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 10 client.9804 fetch_fsmap finished waiting for FSMap version 209 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 10 client.9804 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:44:33.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 20 client.9804 populate_metadata read hostname 'smithi040' 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] conn(0x55f725b42100 0x55f725b424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 4 client.9804 mds_command: new command op to 9705 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3029166155 --> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f725a89350 con 0x55f725b42100 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.781+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] conn(0x55f725b42100 0x55f725b424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] conn(0x55f725b42100 0x55f725b424e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65b3e564749c548 server_cookie=abd16eb95635ab8e in_seq=0 out_seq=0 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2b9ffb640 10 client.9804 ms_handle_connect on v2:172.21.15.40:6832/1239220748 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3029166155 <== mds.0 v2:172.21.15.40:6832/1239220748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f725a89350 con 0x55f725b42100 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2b9ffb640 10 client.9804 handle_command_reply: tid=0 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 client.9804 shutdown 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3029166155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 msgr2=0x55f725b54920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b54920 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc055bb0 tx=0x7fa2cc003960 comp rx=0 tx=0).stop 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3029166155 shutdown_connections 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] conn(0x55f725b42100 0x55f725b424e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b43ed0 0x55f725b155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2b9ffb640 0 client.9804 ms_handle_reset on v2:172.21.15.40:6832/1239220748 2024-04-26T21:44:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b5ece0 0x55f725b15080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3029166155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b41440 0x55f725b54920 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3029166155 >> 172.21.15.40:0/3029166155 conn(0x55f725b17120 msgr2=0x55f725b5f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3029166155 shutdown_connections 2024-04-26T21:44:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.782+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3029166155 wait complete. 2024-04-26T21:44:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.783+0000 7fa2e3f8e740 20 client.9804 trim_cache size 0 max 16384 2024-04-26T21:44:33.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.801+0000 7fa2e3f8e740 1 Processor -- start 2024-04-26T21:44:33.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.801+0000 7fa2e3f8e740 1 -- start start 2024-04-26T21:44:33.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 0x55f725b549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b42a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b42fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc201e90 con 0x55f725b43ed0 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc2011b0 con 0x55f725b5ece0 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc201800 con 0x55f725b41440 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b42a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b42a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34392/0 (socket says 172.21.15.40:34392) 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 -- 172.21.15.40:0/2861354730 learned_addr learned my addr 172.21.15.40:0/2861354730 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 -- 172.21.15.40:0/2861354730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 msgr2=0x55f725b549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 0x55f725b549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2db59e640 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b42fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 0x55f725b549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 -- 172.21.15.40:0/2861354730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 msgr2=0x55f725b42fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b42fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 -- 172.21.15.40:0/2861354730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f725b457a0 con 0x55f725b43ed0 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b42a80 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4006fd0 tx=0x7fa2c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f415639582cfa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2bbfff640 1 -- 172.21.15.40:0/2861354730 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c402b070 con 0x55f725b43ed0 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2bbfff640 1 -- 172.21.15.40:0/2861354730 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c400ebc0 con 0x55f725b43ed0 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.802+0000 7fa2bbfff640 1 -- 172.21.15.40:0/2861354730 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400f040 con 0x55f725b43ed0 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2861354730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 msgr2=0x55f725b42a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b42a80 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4006fd0 tx=0x7fa2c4014000 comp rx=0 tx=0).stop 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2861354730 shutdown_connections 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b42fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b42a80 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/2861354730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 0x55f725b549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2861354730 >> 172.21.15.40:0/2861354730 conn(0x55f725b17120 msgr2=0x55f725b160f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2861354730 shutdown_connections 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/2861354730 wait complete. 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 -- start start 2024-04-26T21:44:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.803+0000 7fa2e3f8e740 1 Processor -- start 2024-04-26T21:44:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 0x55f725bd5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b3b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2e3f8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b3ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f725b17ec0 con 0x55f725b43ed0 2024-04-26T21:44:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f725a02fe0 con 0x55f725b5ece0 2024-04-26T21:44:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2e3f8e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f725b6c600 con 0x55f725b41440 2024-04-26T21:44:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2e3f8e740 10 client.0 _ll_register_callbacks cb 0x7fa2cc0590b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b3b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b3b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34404/0 (socket says 172.21.15.40:34404) 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2d3fff640 1 -- 172.21.15.40:0/3026054305 learned_addr learned my addr 172.21.15.40:0/3026054305 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2db59e640 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b3ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.804+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 0x55f725bd5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2d3fff640 1 -- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 msgr2=0x55f725bd5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 0x55f725bd5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2d3fff640 1 -- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 msgr2=0x55f725b3ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b3ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2d3fff640 1 -- 172.21.15.40:0/3026054305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f725b5a960 con 0x55f725b43ed0 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2db59e640 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b3ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2d3fff640 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b3b530 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4008010 tx=0x7fa2c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7615e6255ace3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3026054305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4034060 con 0x55f725b43ed0 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3026054305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7fa2c4005b90 con 0x55f725b43ed0 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3026054305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c400ebc0 con 0x55f725b43ed0 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3026054305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=209) ==== 24+0+0 (secure 0 0 0) 0x7fa2c402b040 con 0x55f725b43ed0 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 10 client.9810 fetch_fsmap learned FSMap version 209 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3026054305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=209,monmap=2+}) -- 0x55f725b17bf0 con 0x55f725b43ed0 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3026054305 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c400ebc0 con 0x55f725b43ed0 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3026054305 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 209) ==== 3494+0+0 (secure 0 0 0) 0x7fa2c402e070 con 0x55f725b43ed0 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 10 client.9810 fetch_fsmap finished waiting for FSMap version 209 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 10 client.9810 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 20 client.9810 populate_metadata read hostname 'smithi040' 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] conn(0x55f725b525c0 0x55f725b5f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 4 client.9810 mds_command: new command op to 9705 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:44:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3026054305 --> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] -- command(tid 0: {"prefix": "status"}) -- 0x55f725a831c0 con 0x55f725b525c0 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.805+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] conn(0x55f725b525c0 0x55f725b5f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2dad9d640 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] conn(0x55f725b525c0 0x55f725b5f770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7087a0f1ee9217a0 server_cookie=1a53320faeab0548 in_seq=0 out_seq=0 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2b9ffb640 10 client.9810 ms_handle_connect on v2:172.21.15.40:6832/1239220748 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2b9ffb640 1 -- 172.21.15.40:0/3026054305 <== mds.0 v2:172.21.15.40:6832/1239220748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55f725a831c0 con 0x55f725b525c0 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2b9ffb640 10 client.9810 handle_command_reply: tid=0 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 client.9810 shutdown 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3026054305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 msgr2=0x55f725b3b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b3b530 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4008010 tx=0x7fa2c4004730 comp rx=0 tx=0).stop 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3026054305 shutdown_connections 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.40:6832/1239220748,v1:172.21.15.40:6833/1239220748] conn(0x55f725b525c0 0x55f725b5f770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f725b5ece0 0x55f725b3ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f725b43ed0 0x55f725b3b530 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2b9ffb640 0 client.9810 ms_handle_reset on v2:172.21.15.40:6832/1239220748 2024-04-26T21:44:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 --2- 172.21.15.40:0/3026054305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f725b41440 0x55f725bd5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3026054305 >> 172.21.15.40:0/3026054305 conn(0x55f725b17120 msgr2=0x55f725b6be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3026054305 shutdown_connections 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 1 -- 172.21.15.40:0/3026054305 wait complete. 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.806+0000 7fa2e3f8e740 20 client.9810 trim_cache size 0 max 16384 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 9705, 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:44:33.813 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:44:33.814 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:44:33.814 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 7.3869433390000001, 2024-04-26T21:44:33.814 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 209, 2024-04-26T21:44:33.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 113, 2024-04-26T21:44:33.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 113, 2024-04-26T21:44:33.814 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 11.381227678 2024-04-26T21:44:33.814 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:44:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 -- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2bc06c8f0 msgr2=0x7fa2bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2bc06c8f0 0x7fa2bc06edb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0b7970 tx=0x7fa2c4008040 comp rx=0 tx=0).stop 2024-04-26T21:44:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 -- 172.21.15.40:0/2452465062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 msgr2=0x7fa2dc159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc159660 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00c430 tx=0x7fa2cc00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 -- 172.21.15.40:0/2452465062 shutdown_connections 2024-04-26T21:44:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa2bc06c8f0 0x7fa2bc06edb0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2dc162880 0x7fa2dc06dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2dc0d3bc0 0x7fa2dc0cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 --2- 172.21.15.40:0/2452465062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc0b5f20 0x7fa2dc159660 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 -- 172.21.15.40:0/2452465062 >> 172.21.15.40:0/2452465062 conn(0x7fa2dc0d4f60 msgr2=0x7fa2dc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 -- 172.21.15.40:0/2452465062 shutdown_connections 2024-04-26T21:44:33.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.807+0000 7fa2e119d640 1 -- 172.21.15.40:0/2452465062 wait complete. 2024-04-26T21:44:33.817 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:44:33.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:33.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.918+0000 7f3cd5351640 1 Processor -- start 2024-04-26T21:44:33.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.918+0000 7f3cd5351640 1 -- start start 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3cd5351640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3cd01604e0 0x7f3cd006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3cd5351640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3cd006fcd0 0x7f3cd00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3cd5351640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd0160970 con 0x7f3cd015f2d0 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3cd5351640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd0074010 con 0x7f3cd006fcd0 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3cd5351640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd0070100 con 0x7f3cd01604e0 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3ccf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3cd006fcd0 0x7f3cd00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3ccf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3cd006fcd0 0x7f3cd00ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34406/0 (socket says 172.21.15.40:34406) 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3ccf7fe640 1 -- 172.21.15.40:0/457086599 learned_addr learned my addr 172.21.15.40:0/457086599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3cce7fc640 1 --2- 172.21.15.40:0/457086599 >> v2:172.21.15.184:3300/0 conn(0x7f3cd01604e0 0x7f3cd006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3ccf7fe640 1 -- 172.21.15.40:0/457086599 >> v2:172.21.15.184:3300/0 conn(0x7f3cd01604e0 msgr2=0x7f3cd006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3ccf7fe640 1 --2- 172.21.15.40:0/457086599 >> v2:172.21.15.184:3300/0 conn(0x7f3cd01604e0 0x7f3cd006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3ccf7fe640 1 -- 172.21.15.40:0/457086599 >> v1:172.21.15.184:6789/0 conn(0x7f3cd015f2d0 legacy=0x7f3cd0072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.919+0000 7f3ccf7fe640 1 -- 172.21.15.40:0/457086599 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd0072f70 con 0x7f3cd006fcd0 2024-04-26T21:44:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.920+0000 7f3ccf7fe640 1 --2- 172.21.15.40:0/457086599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd00ce7b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00717a0 tx=0x7f3cc002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14f6373c0d98c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.920+0000 7f3ccdffb640 1 -- 172.21.15.40:0/457086599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0036070 con 0x7f3cd006fcd0 2024-04-26T21:44:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.920+0000 7f3ccdffb640 1 -- 172.21.15.40:0/457086599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc0031600 con 0x7f3cd006fcd0 2024-04-26T21:44:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.920+0000 7f3ccdffb640 1 -- 172.21.15.40:0/457086599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00319d0 con 0x7f3cd006fcd0 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.921+0000 7f3cd5351640 1 -- 172.21.15.40:0/457086599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 msgr2=0x7f3cd00ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.921+0000 7f3cd5351640 1 --2- 172.21.15.40:0/457086599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd00ce7b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00717a0 tx=0x7f3cc002a710 comp rx=0 tx=0).stop 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.921+0000 7f3cd5351640 1 -- 172.21.15.40:0/457086599 shutdown_connections 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.921+0000 7f3cd5351640 1 --2- 172.21.15.40:0/457086599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd00ce7b0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.921+0000 7f3cd5351640 1 --2- 172.21.15.40:0/457086599 >> v2:172.21.15.184:3300/0 conn(0x7f3cd01604e0 0x7f3cd006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.921+0000 7f3cd5351640 1 -- 172.21.15.40:0/457086599 >> 172.21.15.40:0/457086599 conn(0x7f3cd00d0be0 msgr2=0x7f3cd00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.921+0000 7f3cd5351640 1 -- 172.21.15.40:0/457086599 shutdown_connections 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.921+0000 7f3cd5351640 1 -- 172.21.15.40:0/457086599 wait complete. 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.922+0000 7f3cd5351640 1 Processor -- start 2024-04-26T21:44:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.922+0000 7f3cd5351640 1 -- start start 2024-04-26T21:44:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cd5351640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd014e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cd5351640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cd015f2d0 0x7f3cd0156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cd5351640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd01604e0 0x7f3cd00b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cd5351640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0073810 con 0x7f3cd006fcd0 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cd5351640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0073530 con 0x7f3cd01604e0 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cd5351640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd0073af0 con 0x7f3cd015f2d0 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd014e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd014e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34416/0 (socket says 172.21.15.40:34416) 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 -- 172.21.15.40:0/231883251 learned_addr learned my addr 172.21.15.40:0/231883251 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cce7fc640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cd015f2d0 0x7f3cd0156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3ccf7fe640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd01604e0 0x7f3cd00b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 -- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cd015f2d0 msgr2=0x7f3cd0156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cd015f2d0 0x7f3cd0156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 -- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd01604e0 msgr2=0x7f3cd00b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd01604e0 0x7f3cd00b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 -- 172.21.15.40:0/231883251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd015e2d0 con 0x7f3cd006fcd0 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cce7fc640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cd015f2d0 0x7f3cd0156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3ccf7fe640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd01604e0 0x7f3cd00b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.923+0000 7f3cceffd640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd014e2c0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c460 tx=0x7f3cc800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.924+0000 7f3cc77fe640 1 -- 172.21.15.40:0/231883251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8011790 con 0x7f3cd006fcd0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.924+0000 7f3cc77fe640 1 -- 172.21.15.40:0/231883251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc8017070 con 0x7f3cd006fcd0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.924+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd015e5b0 con 0x7f3cd006fcd0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.924+0000 7f3cc77fe640 1 -- 172.21.15.40:0/231883251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8011bb0 con 0x7f3cd006fcd0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.924+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd01528e0 con 0x7f3cd006fcd0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.925+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd00694f0 con 0x7f3cd006fcd0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.925+0000 7f3cc77fe640 1 -- 172.21.15.40:0/231883251 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cc801c460 con 0x7f3cd006fcd0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.925+0000 7f3cc77fe640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cb006c8f0 0x7f3cb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.926+0000 7f3cc77fe640 1 -- 172.21.15.40:0/231883251 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f3cc808c1a0 con 0x7f3cd006fcd0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.926+0000 7f3cce7fc640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cb006c8f0 0x7f3cb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.926+0000 7f3cce7fc640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cb006c8f0 0x7f3cb006edb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8006fd0 tx=0x7f3cb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:33.930+0000 7f3cc77fe640 1 -- 172.21.15.40:0/231883251 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3cc8059dc0 con 0x7f3cd006fcd0 2024-04-26T21:44:34.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.153+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cd00cd630 con 0x7f3cd006fcd0 2024-04-26T21:44:34.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.153+0000 7f3cc77fe640 1 -- 172.21.15.40:0/231883251 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5197 (secure 0 0 0) 0x7f3cc80596c0 con 0x7f3cd006fcd0 2024-04-26T21:44:34.153 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:34.154 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":209,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:29.428218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9705":{"gid":9705,"name":"c","rank":0,"incarnation":206,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/1239220748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1239220748},{"type":"v1","addr":"172.21.15.40:6833","nonce":1239220748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9705,"qdb_cluster":[9705]},"id":6}]} 2024-04-26T21:44:34.154 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 209 2024-04-26T21:44:34.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cb006c8f0 msgr2=0x7f3cb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:34.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cb006c8f0 0x7f3cb006edb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8006fd0 tx=0x7f3cb8008040 comp rx=0 tx=0).stop 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 msgr2=0x7f3cd014e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd014e2c0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c460 tx=0x7f3cc800c920 comp rx=0 tx=0).stop 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 shutdown_connections 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3cb006c8f0 0x7f3cb006edb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cd01604e0 0x7f3cd00b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cd015f2d0 0x7f3cd0156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 --2- 172.21.15.40:0/231883251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd006fcd0 0x7f3cd014e2c0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 >> 172.21.15.40:0/231883251 conn(0x7f3cd00d0be0 msgr2=0x7f3cd014f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 shutdown_connections 2024-04-26T21:44:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.155+0000 7f3cd5351640 1 -- 172.21.15.40:0/231883251 wait complete. 2024-04-26T21:44:34.166 INFO:tasks.cephfs.test_damage:Mutation 'Corrupt 500.00000000' did not prevent MDS startup, attempting ls... 2024-04-26T21:44:34.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:44:34.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.267+0000 7f2f1d447640 1 Processor -- start 2024-04-26T21:44:34.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.267+0000 7f2f1d447640 1 -- start start 2024-04-26T21:44:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f1d447640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2f180d3bc0 0x7f2f180d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f1d447640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2f180b5f20 0x7f2f180b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f1d447640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f180b7e50 con 0x7f2f18162880 2024-04-26T21:44:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f1d447640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2f180d0580 con 0x7f2f180d3bc0 2024-04-26T21:44:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f1d447640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2f180d1800 con 0x7f2f180b5f20 2024-04-26T21:44:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f16ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2f18162880 0x7f2f180d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56470/0 (socket says 172.21.15.40:56470) 2024-04-26T21:44:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f16ffd640 1 -- 172.21.15.40:0/3280512096 learned_addr learned my addr 172.21.15.40:0/3280512096 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f167fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2f180d3bc0 0x7f2f180d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f177fe640 1 --2- 172.21.15.40:0/3280512096 >> v2:172.21.15.184:3300/0 conn(0x7f2f180b5f20 0x7f2f180b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f167fc640 1 -- 172.21.15.40:0/3280512096 >> v2:172.21.15.184:3300/0 conn(0x7f2f180b5f20 msgr2=0x7f2f180b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f167fc640 1 --2- 172.21.15.40:0/3280512096 >> v2:172.21.15.184:3300/0 conn(0x7f2f180b5f20 0x7f2f180b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f167fc640 1 -- 172.21.15.40:0/3280512096 >> v1:172.21.15.40:6789/0 conn(0x7f2f18162880 legacy=0x7f2f180d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.268+0000 7f2f167fc640 1 -- 172.21.15.40:0/3280512096 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f180d07d0 con 0x7f2f180d3bc0 2024-04-26T21:44:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.269+0000 7f2f167fc640 1 --2- 172.21.15.40:0/3280512096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f180d3bc0 0x7f2f180d2cb0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2f1808f770 tx=0x7f2f0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd52d3cb5c55ced6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.269+0000 7f2f15ffb640 1 -- 172.21.15.40:0/3280512096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c037070 con 0x7f2f180d3bc0 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.269+0000 7f2f15ffb640 1 -- 172.21.15.40:0/3280512096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c032440 con 0x7f2f180d3bc0 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.269+0000 7f2f15ffb640 1 -- 172.21.15.40:0/3280512096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c032810 con 0x7f2f180d3bc0 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.270+0000 7f2f1d447640 1 -- 172.21.15.40:0/3280512096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f180d3bc0 msgr2=0x7f2f180d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.270+0000 7f2f1d447640 1 --2- 172.21.15.40:0/3280512096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f180d3bc0 0x7f2f180d2cb0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2f1808f770 tx=0x7f2f0c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.270+0000 7f2f1d447640 1 -- 172.21.15.40:0/3280512096 shutdown_connections 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.270+0000 7f2f1d447640 1 --2- 172.21.15.40:0/3280512096 >> v2:172.21.15.184:3300/0 conn(0x7f2f180b5f20 0x7f2f180b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.270+0000 7f2f1d447640 1 --2- 172.21.15.40:0/3280512096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f180d3bc0 0x7f2f180d2cb0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.270+0000 7f2f1d447640 1 -- 172.21.15.40:0/3280512096 >> 172.21.15.40:0/3280512096 conn(0x7f2f180d4f60 msgr2=0x7f2f180d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.270+0000 7f2f1d447640 1 -- 172.21.15.40:0/3280512096 shutdown_connections 2024-04-26T21:44:34.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.270+0000 7f2f1d447640 1 -- 172.21.15.40:0/3280512096 wait complete. 2024-04-26T21:44:34.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.271+0000 7f2f1d447640 1 Processor -- start 2024-04-26T21:44:34.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.271+0000 7f2f1d447640 1 -- start start 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f1d447640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f180b5f20 0x7f2f1815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f1d447640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f180d3bc0 0x7f2f180a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f1d447640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f18162880 0x7f2f1809ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f1d447640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18162c60 con 0x7f2f18162880 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f1d447640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f180d0d90 con 0x7f2f180b5f20 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f1d447640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2f18162de0 con 0x7f2f180d3bc0 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f18162880 0x7f2f1809ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f18162880 0x7f2f1809ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34438/0 (socket says 172.21.15.40:34438) 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f177fe640 1 -- 172.21.15.40:0/459533922 learned_addr learned my addr 172.21.15.40:0/459533922 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f177fe640 1 -- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f180d3bc0 msgr2=0x7f2f180a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f167fc640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f180d3bc0 0x7f2f180a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f16ffd640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f180b5f20 0x7f2f1815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f177fe640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f180d3bc0 0x7f2f180a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f177fe640 1 -- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f180b5f20 msgr2=0x7f2f1815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f177fe640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f180b5f20 0x7f2f1815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f177fe640 1 -- 172.21.15.40:0/459533922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f180a0b00 con 0x7f2f18162880 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.272+0000 7f2f16ffd640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f180b5f20 0x7f2f1815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.273+0000 7f2f177fe640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f18162880 0x7f2f1809ed80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f2f10007e60 tx=0x7f2f1000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.273+0000 7f2eff7fe640 1 -- 172.21.15.40:0/459533922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10014420 con 0x7f2f18162880 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.273+0000 7f2eff7fe640 1 -- 172.21.15.40:0/459533922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f10019070 con 0x7f2f18162880 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.273+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f180a0de0 con 0x7f2f18162880 2024-04-26T21:44:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.273+0000 7f2eff7fe640 1 -- 172.21.15.40:0/459533922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10014820 con 0x7f2f18162880 2024-04-26T21:44:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.273+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f180a13a0 con 0x7f2f18162880 2024-04-26T21:44:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.274+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f180694f0 con 0x7f2f18162880 2024-04-26T21:44:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.274+0000 7f2eff7fe640 1 -- 172.21.15.40:0/459533922 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f100149c0 con 0x7f2f18162880 2024-04-26T21:44:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.275+0000 7f2eff7fe640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2f0406c8f0 0x7f2f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.275+0000 7f2eff7fe640 1 -- 172.21.15.40:0/459533922 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f2f1008cbe0 con 0x7f2f18162880 2024-04-26T21:44:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.275+0000 7f2f16ffd640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2f0406c8f0 0x7f2f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.276+0000 7f2f16ffd640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2f0406c8f0 0x7f2f0406edb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2f18075e50 tx=0x7f2f08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:34.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.279+0000 7f2eff7fe640 1 -- 172.21.15.40:0/459533922 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f1005a800 con 0x7f2f18162880 2024-04-26T21:44:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.507+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2f180cf370 con 0x7f2f18162880 2024-04-26T21:44:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.507+0000 7f2eff7fe640 1 -- 172.21.15.40:0/459533922 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v209) ==== 53+0+73 (secure 0 0 0) 0x7f2f1005a100 con 0x7f2f18162880 2024-04-26T21:44:34.507 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:44:34.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2f0406c8f0 msgr2=0x7f2f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2f0406c8f0 0x7f2f0406edb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2f18075e50 tx=0x7f2f08009040 comp rx=0 tx=0).stop 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f18162880 msgr2=0x7f2f1809ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f18162880 0x7f2f1809ed80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f2f10007e60 tx=0x7f2f1000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 shutdown_connections 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2f0406c8f0 0x7f2f0406edb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f18162880 0x7f2f1809ed80 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f180d3bc0 0x7f2f180a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 --2- 172.21.15.40:0/459533922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f180b5f20 0x7f2f1815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 >> 172.21.15.40:0/459533922 conn(0x7f2f180d4f60 msgr2=0x7f2f180ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 shutdown_connections 2024-04-26T21:44:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34.509+0000 7f2f1d447640 1 -- 172.21.15.40:0/459533922 wait complete. 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:44:34.520 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:44:34.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:34.521 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:44:34.536 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:44:34.536 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:44:34.536 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:44:34.536 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:44:34.536 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:44:34.536 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:44:34.536 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42173sec preferred_lft 42173sec 2024-04-26T21:44:34.537 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:44:34.538 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:44:34.539 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:44:34.539 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:44:34.539 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:44:34.539 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:44:34.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:34.539 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:44:34.600 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:44:34.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:34.601 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:44:34.658 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:44:34.658 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:44:34.658 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:44:34.658 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:44:34.658 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:44:34.659 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:44:34.659 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:44:34.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:44:34.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:44:34.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:44:34.659 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:44:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:44:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:44:34.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:34.785 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:44:34.800 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:44:34.800 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:44:34.800 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:44:34.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:34.801 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:44:34.864 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:44:34.865 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:44:34.919 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:44:34.920 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:44:34.920 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:44:34.975 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:44:34.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:34.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:44:35.058 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:44:35.058 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:44:35.069 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:44:35.070 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:44:35.070 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:44:35.070 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:44:35.070 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:44:35.072 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:44:35.072 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:44:35.072 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:44:35.072 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:44:35.072 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:44:35.072 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:44:35.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:35.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:44:35.141 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:44:35.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:44:35.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:44:35.238 INFO:teuthology.orchestra.run.smithi040.stdout:client8374 2024-04-26T21:44:35.239 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:35.239 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:35.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:35.254 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-26T21:44:35.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:35.297 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2024-04-26T21:44:35.311 INFO:teuthology.orchestra.run.smithi040.stdout:.: 2024-04-26T21:44:35.311 INFO:teuthology.orchestra.run.smithi040.stdout:subdir 2024-04-26T21:44:35.311 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:35.311 INFO:teuthology.orchestra.run.smithi040.stdout:./subdir: 2024-04-26T21:44:35.311 INFO:teuthology.orchestra.run.smithi040.stdout:sixmegs 2024-04-26T21:44:35.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:35.325 INFO:teuthology.orchestra.run.smithi040.stderr:mkdir: cannot create directory ‘foo’: Read-only file system 2024-04-26T21:44:35.325 INFO:tasks.cephfs.test_damage:mkdir: cannot create directory ‘foo’: Read-only file system 2024-04-26T21:44:35.325 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds_snaptable' 2024-04-26T21:44:35.325 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:35.325 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:35.381 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:35.381 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:35.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:44:35.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.578+0000 7ffad02dc640 1 Processor -- start 2024-04-26T21:44:35.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.578+0000 7ffad02dc640 1 -- start start 2024-04-26T21:44:35.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7ffad02dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffac8162810 0x7ffac8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7ffad02dc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffac80d1b30 0x7ffac80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7ffad02dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac80d10d0 con 0x7ffac80b2340 2024-04-26T21:44:35.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7ffad02dc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffac80d0750 con 0x7ffac8162810 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7ffad02dc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffac80d16a0 con 0x7ffac80d1b30 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface078640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffac8162810 0x7ffac8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface078640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffac8162810 0x7ffac8070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34444/0 (socket says 172.21.15.40:34444) 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface078640 1 -- 172.21.15.40:0/317709320 learned_addr learned my addr 172.21.15.40:0/317709320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface078640 1 -- 172.21.15.40:0/317709320 >> v2:172.21.15.184:3300/0 conn(0x7ffac80d1b30 msgr2=0x7ffac80d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface879640 1 --2- 172.21.15.40:0/317709320 >> v2:172.21.15.184:3300/0 conn(0x7ffac80d1b30 0x7ffac80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface078640 1 --2- 172.21.15.40:0/317709320 >> v2:172.21.15.184:3300/0 conn(0x7ffac80d1b30 0x7ffac80d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface078640 1 -- 172.21.15.40:0/317709320 >> v1:172.21.15.184:6789/0 conn(0x7ffac80b2340 legacy=0x7ffac80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface078640 1 -- 172.21.15.40:0/317709320 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac80d1370 con 0x7ffac8162810 2024-04-26T21:44:35.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.579+0000 7fface879640 1 --2- 172.21.15.40:0/317709320 >> v2:172.21.15.184:3300/0 conn(0x7ffac80d1b30 0x7ffac80d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:35.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.580+0000 7fface078640 1 --2- 172.21.15.40:0/317709320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac8162810 0x7ffac8070350 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7ffac8073330 tx=0x7ffab802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be1f9d3522b288b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:35.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.580+0000 7ffacd076640 1 -- 172.21.15.40:0/317709320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8036070 con 0x7ffac8162810 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.580+0000 7ffacd076640 1 -- 172.21.15.40:0/317709320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab8031600 con 0x7ffac8162810 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.580+0000 7ffacd076640 1 -- 172.21.15.40:0/317709320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab80319d0 con 0x7ffac8162810 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.581+0000 7ffad02dc640 1 -- 172.21.15.40:0/317709320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac8162810 msgr2=0x7ffac8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.581+0000 7ffad02dc640 1 --2- 172.21.15.40:0/317709320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac8162810 0x7ffac8070350 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7ffac8073330 tx=0x7ffab802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.581+0000 7ffad02dc640 1 -- 172.21.15.40:0/317709320 shutdown_connections 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.581+0000 7ffad02dc640 1 --2- 172.21.15.40:0/317709320 >> v2:172.21.15.184:3300/0 conn(0x7ffac80d1b30 0x7ffac80d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.581+0000 7ffad02dc640 1 --2- 172.21.15.40:0/317709320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac8162810 0x7ffac8070350 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.581+0000 7ffad02dc640 1 -- 172.21.15.40:0/317709320 >> 172.21.15.40:0/317709320 conn(0x7ffac80d2da0 msgr2=0x7ffac80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.581+0000 7ffad02dc640 1 -- 172.21.15.40:0/317709320 shutdown_connections 2024-04-26T21:44:35.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.581+0000 7ffad02dc640 1 -- 172.21.15.40:0/317709320 wait complete. 2024-04-26T21:44:35.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.582+0000 7ffad02dc640 1 Processor -- start 2024-04-26T21:44:35.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.582+0000 7ffad02dc640 1 -- start start 2024-04-26T21:44:35.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffad02dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffac80b2340 0x7ffac80c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffad02dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac80d1b30 0x7ffac81e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffad02dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffac8162810 0x7ffac81ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffad02dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8070890 con 0x7ffac80d1b30 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffad02dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffac80d0470 con 0x7ffac80b2340 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffad02dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffac80cdb00 con 0x7ffac8162810 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffacd877640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac80d1b30 0x7ffac81e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffacd877640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac80d1b30 0x7ffac81e9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34448/0 (socket says 172.21.15.40:34448) 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffacd877640 1 -- 172.21.15.40:0/2701013714 learned_addr learned my addr 172.21.15.40:0/2701013714 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7fface078640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffac80b2340 0x7ffac80c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7fface879640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffac8162810 0x7ffac81ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffacd877640 1 -- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffac8162810 msgr2=0x7ffac81ea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffacd877640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffac8162810 0x7ffac81ea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffacd877640 1 -- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffac80b2340 msgr2=0x7ffac80c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffacd877640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffac80b2340 0x7ffac80c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7ffacd877640 1 -- 172.21.15.40:0/2701013714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac80c6be0 con 0x7ffac80d1b30 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.583+0000 7fface078640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffac80b2340 0x7ffac80c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.584+0000 7ffacd877640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac80d1b30 0x7ffac81e9d90 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c430 tx=0x7ffabc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.584+0000 7ffac6ffd640 1 -- 172.21.15.40:0/2701013714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc011790 con 0x7ffac80d1b30 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.584+0000 7ffac6ffd640 1 -- 172.21.15.40:0/2701013714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffabc017070 con 0x7ffac80d1b30 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.584+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac81eb040 con 0x7ffac80d1b30 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.584+0000 7ffac6ffd640 1 -- 172.21.15.40:0/2701013714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc011bb0 con 0x7ffac80d1b30 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.584+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac81eb600 con 0x7ffac80d1b30 2024-04-26T21:44:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.585+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac8162370 con 0x7ffac80d1b30 2024-04-26T21:44:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.585+0000 7ffac6ffd640 1 -- 172.21.15.40:0/2701013714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffabc01c460 con 0x7ffac80d1b30 2024-04-26T21:44:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.586+0000 7ffac6ffd640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffab406c8f0 0x7ffab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.586+0000 7ffac6ffd640 1 -- 172.21.15.40:0/2701013714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7ffabc08c170 con 0x7ffac80d1b30 2024-04-26T21:44:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.586+0000 7fface078640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffab406c8f0 0x7ffab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.586+0000 7fface078640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffab406c8f0 0x7ffab406edb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffab8033000 tx=0x7ffab8009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:35.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.590+0000 7ffac6ffd640 1 -- 172.21.15.40:0/2701013714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffac8162370 con 0x7ffac80d1b30 2024-04-26T21:44:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.775+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffac80694f0 con 0x7ffac80d1b30 2024-04-26T21:44:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.777+0000 7ffac6ffd640 1 -- 172.21.15.40:0/2701013714 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+19374 (secure 0 0 0) 0x7ffabc059e20 con 0x7ffac80d1b30 2024-04-26T21:44:35.777 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:35.777 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":113,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:44:22.393354+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffab406c8f0 msgr2=0x7ffab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffab406c8f0 0x7ffab406edb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffab8033000 tx=0x7ffab8009000 comp rx=0 tx=0).stop 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac80d1b30 msgr2=0x7ffac81e9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac80d1b30 0x7ffac81e9d90 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c430 tx=0x7ffabc00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 shutdown_connections 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffab406c8f0 0x7ffab406edb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffac8162810 0x7ffac81ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffac80d1b30 0x7ffac81e9d90 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 --2- 172.21.15.40:0/2701013714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffac80b2340 0x7ffac80c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 >> 172.21.15.40:0/2701013714 conn(0x7ffac80d2da0 msgr2=0x7ffac80cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 shutdown_connections 2024-04-26T21:44:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.779+0000 7ffad02dc640 1 -- 172.21.15.40:0/2701013714 wait complete. 2024-04-26T21:44:35.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:44:35.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:44:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.894+0000 7f8a32904640 1 Processor -- start 2024-04-26T21:44:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.894+0000 7f8a32904640 1 -- start start 2024-04-26T21:44:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.895+0000 7f8a32904640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a2c0d1a00 0x7f8a2c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.895+0000 7f8a32904640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a2c0d2320 0x7f8a2c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.895+0000 7f8a32904640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a2c0d0a10 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.895+0000 7f8a32904640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8a2c0d0cf0 con 0x7f8a2c0d1a00 2024-04-26T21:44:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.895+0000 7f8a32904640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8a2c0d0fd0 con 0x7f8a2c0d2320 2024-04-26T21:44:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.895+0000 7f8a2bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a2c0d1a00 0x7f8a2c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.895+0000 7f8a30ea1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8a2c0b40d0 0x7f8a2c0d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56486/0 (socket says 172.21.15.40:56486) 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.895+0000 7f8a30ea1640 1 -- 172.21.15.40:0/3295764144 learned_addr learned my addr 172.21.15.40:0/3295764144 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.896+0000 7f8a2b7fe640 1 --2- 172.21.15.40:0/3295764144 >> v2:172.21.15.184:3300/0 conn(0x7f8a2c0d2320 0x7f8a2c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.896+0000 7f8a2bfff640 1 -- 172.21.15.40:0/3295764144 >> v2:172.21.15.184:3300/0 conn(0x7f8a2c0d2320 msgr2=0x7f8a2c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.896+0000 7f8a2bfff640 1 --2- 172.21.15.40:0/3295764144 >> v2:172.21.15.184:3300/0 conn(0x7f8a2c0d2320 0x7f8a2c1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.896+0000 7f8a2bfff640 1 -- 172.21.15.40:0/3295764144 >> v1:172.21.15.40:6789/0 conn(0x7f8a2c0b40d0 legacy=0x7f8a2c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.896+0000 7f8a2bfff640 1 -- 172.21.15.40:0/3295764144 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c0d40a0 con 0x7f8a2c0d1a00 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.896+0000 7f8a2b7fe640 1 --2- 172.21.15.40:0/3295764144 >> v2:172.21.15.184:3300/0 conn(0x7f8a2c0d2320 0x7f8a2c1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.896+0000 7f8a2bfff640 1 --2- 172.21.15.40:0/3295764144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0d1a00 0x7f8a2c0d1de0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c08d380 tx=0x7f8a1c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e116dd3dc3d6254c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.896+0000 7f8a2affd640 1 -- 172.21.15.40:0/3295764144 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c037070 con 0x7f8a2c0d1a00 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a2affd640 1 -- 172.21.15.40:0/3295764144 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c02add0 con 0x7f8a2c0d1a00 2024-04-26T21:44:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a2affd640 1 -- 172.21.15.40:0/3295764144 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c032660 con 0x7f8a2c0d1a00 2024-04-26T21:44:35.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a32904640 1 -- 172.21.15.40:0/3295764144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0d1a00 msgr2=0x7f8a2c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a32904640 1 --2- 172.21.15.40:0/3295764144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0d1a00 0x7f8a2c0d1de0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c08d380 tx=0x7f8a1c02a080 comp rx=0 tx=0).stop 2024-04-26T21:44:35.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a32904640 1 -- 172.21.15.40:0/3295764144 shutdown_connections 2024-04-26T21:44:35.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a32904640 1 --2- 172.21.15.40:0/3295764144 >> v2:172.21.15.184:3300/0 conn(0x7f8a2c0d2320 0x7f8a2c1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a32904640 1 --2- 172.21.15.40:0/3295764144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0d1a00 0x7f8a2c0d1de0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a32904640 1 -- 172.21.15.40:0/3295764144 >> 172.21.15.40:0/3295764144 conn(0x7f8a2c0d4fd0 msgr2=0x7f8a2c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:35.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a32904640 1 -- 172.21.15.40:0/3295764144 shutdown_connections 2024-04-26T21:44:35.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.897+0000 7f8a32904640 1 -- 172.21.15.40:0/3295764144 wait complete. 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.898+0000 7f8a32904640 1 Processor -- start 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.898+0000 7f8a32904640 1 -- start start 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a32904640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0b40d0 0x7f8a2c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a32904640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a2c0d1a00 0x7f8a2c0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a32904640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a2c0d2320 0x7f8a2c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a32904640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c162b30 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a32904640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c0b5d60 con 0x7f8a2c0d2320 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a32904640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c162cb0 con 0x7f8a2c0d1a00 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0b40d0 0x7f8a2c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0b40d0 0x7f8a2c14ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34464/0 (socket says 172.21.15.40:34464) 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2bfff640 1 -- 172.21.15.40:0/3520030480 learned_addr learned my addr 172.21.15.40:0/3520030480 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a30ea1640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a2c0d2320 0x7f8a2c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2b7fe640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a2c0d1a00 0x7f8a2c0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2bfff640 1 -- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a2c0d1a00 msgr2=0x7f8a2c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2bfff640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a2c0d1a00 0x7f8a2c0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2bfff640 1 -- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a2c0d2320 msgr2=0x7f8a2c0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2bfff640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a2c0d2320 0x7f8a2c0c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.899+0000 7f8a2bfff640 1 -- 172.21.15.40:0/3520030480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c0cb510 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.900+0000 7f8a2b7fe640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a2c0d1a00 0x7f8a2c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.900+0000 7f8a30ea1640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a2c0d2320 0x7f8a2c0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.900+0000 7f8a2bfff640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0b40d0 0x7f8a2c14ed90 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c035040 tx=0x7f8a1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.900+0000 7f8a28ff9640 1 -- 172.21.15.40:0/3520030480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c037040 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.900+0000 7f8a28ff9640 1 -- 172.21.15.40:0/3520030480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c03b430 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.900+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2c0cb7f0 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.900+0000 7f8a28ff9640 1 -- 172.21.15.40:0/3520030480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c03b820 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.900+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2c0cbdb0 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.901+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2c0694f0 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.902+0000 7f8a28ff9640 1 -- 172.21.15.40:0/3520030480 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a1c044020 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.902+0000 7f8a28ff9640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a1806c8f0 0x7f8a1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.902+0000 7f8a28ff9640 1 -- 172.21.15.40:0/3520030480 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f8a1c0ac470 con 0x7f8a2c0b40d0 2024-04-26T21:44:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.902+0000 7f8a2b7fe640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a1806c8f0 0x7f8a1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.903+0000 7f8a2b7fe640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a1806c8f0 0x7f8a1806edb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8a20006fd0 tx=0x7f8a20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:35.906+0000 7f8a28ff9640 1 -- 172.21.15.40:0/3520030480 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a1c07a090 con 0x7f8a2c0b40d0 2024-04-26T21:44:36.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.096+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a2c0cf2e0 con 0x7f8a2c0b40d0 2024-04-26T21:44:36.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.097+0000 7f8a28ff9640 1 -- 172.21.15.40:0/3520030480 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f8a1c079990 con 0x7f8a2c0b40d0 2024-04-26T21:44:36.097 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:44:36.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a1806c8f0 msgr2=0x7f8a1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a1806c8f0 0x7f8a1806edb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8a20006fd0 tx=0x7f8a20008040 comp rx=0 tx=0).stop 2024-04-26T21:44:36.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0b40d0 msgr2=0x7f8a2c14ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0b40d0 0x7f8a2c14ed90 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c035040 tx=0x7f8a1c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 shutdown_connections 2024-04-26T21:44:36.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8a1806c8f0 0x7f8a1806edb0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a2c0d2320 0x7f8a2c0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a2c0d1a00 0x7f8a2c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 --2- 172.21.15.40:0/3520030480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a2c0b40d0 0x7f8a2c14ed90 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 >> 172.21.15.40:0/3520030480 conn(0x7f8a2c0d4fd0 msgr2=0x7f8a2c0d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:36.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 shutdown_connections 2024-04-26T21:44:36.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.099+0000 7f8a32904640 1 -- 172.21.15.40:0/3520030480 wait complete. 2024-04-26T21:44:36.110 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:44:36.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.213+0000 7fe0f3589640 1 Processor -- start 2024-04-26T21:44:36.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.213+0000 7fe0f3589640 1 -- start start 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f3589640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0ec15f170 0x7fe0ec0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f3589640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe0ec0d3bc0 0x7fe0ec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f3589640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ec0d1040 con 0x7fe0ec15f550 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f3589640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ec0c9430 con 0x7fe0ec15f170 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f3589640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ec0c9680 con 0x7fe0ec0d3bc0 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f1325640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0ec15f170 0x7fe0ec0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f1325640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0ec15f170 0x7fe0ec0d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34472/0 (socket says 172.21.15.40:34472) 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f1325640 1 -- 172.21.15.40:0/1973348689 learned_addr learned my addr 172.21.15.40:0/1973348689 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:36.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f1b26640 1 --2- 172.21.15.40:0/1973348689 >> v2:172.21.15.184:3300/0 conn(0x7fe0ec0d3bc0 0x7fe0ec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f1325640 1 -- 172.21.15.40:0/1973348689 >> v2:172.21.15.184:3300/0 conn(0x7fe0ec0d3bc0 msgr2=0x7fe0ec0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:36.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f1325640 1 --2- 172.21.15.40:0/1973348689 >> v2:172.21.15.184:3300/0 conn(0x7fe0ec0d3bc0 0x7fe0ec0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f1325640 1 -- 172.21.15.40:0/1973348689 >> v1:172.21.15.184:6789/0 conn(0x7fe0ec15f550 legacy=0x7fe0ec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:36.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.214+0000 7fe0f1325640 1 -- 172.21.15.40:0/1973348689 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec0c9960 con 0x7fe0ec15f170 2024-04-26T21:44:36.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.215+0000 7fe0f1325640 1 --2- 172.21.15.40:0/1973348689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec15f170 0x7fe0ec0d5a80 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009870 tx=0x7fe0e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d266b9276296c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.215+0000 7fe0e3fff640 1 -- 172.21.15.40:0/1973348689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4037070 con 0x7fe0ec15f170 2024-04-26T21:44:36.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.215+0000 7fe0e3fff640 1 -- 172.21.15.40:0/1973348689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e4032440 con 0x7fe0ec15f170 2024-04-26T21:44:36.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.215+0000 7fe0e3fff640 1 -- 172.21.15.40:0/1973348689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4032810 con 0x7fe0ec15f170 2024-04-26T21:44:36.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.216+0000 7fe0f3589640 1 -- 172.21.15.40:0/1973348689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec15f170 msgr2=0x7fe0ec0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.216+0000 7fe0f3589640 1 --2- 172.21.15.40:0/1973348689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec15f170 0x7fe0ec0d5a80 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009870 tx=0x7fe0e402a270 comp rx=0 tx=0).stop 2024-04-26T21:44:36.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.216+0000 7fe0f3589640 1 -- 172.21.15.40:0/1973348689 shutdown_connections 2024-04-26T21:44:36.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.216+0000 7fe0f3589640 1 --2- 172.21.15.40:0/1973348689 >> v2:172.21.15.184:3300/0 conn(0x7fe0ec0d3bc0 0x7fe0ec0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.216+0000 7fe0f3589640 1 --2- 172.21.15.40:0/1973348689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec15f170 0x7fe0ec0d5a80 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.216+0000 7fe0f3589640 1 -- 172.21.15.40:0/1973348689 >> 172.21.15.40:0/1973348689 conn(0x7fe0ec0d4f60 msgr2=0x7fe0ec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:36.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.216+0000 7fe0f3589640 1 -- 172.21.15.40:0/1973348689 shutdown_connections 2024-04-26T21:44:36.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.216+0000 7fe0f3589640 1 -- 172.21.15.40:0/1973348689 wait complete. 2024-04-26T21:44:36.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.217+0000 7fe0f3589640 1 Processor -- start 2024-04-26T21:44:36.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.217+0000 7fe0f3589640 1 -- start start 2024-04-26T21:44:36.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f3589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0ec0d3bc0 0x7fe0ec1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f3589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0ec15f170 0x7fe0ec0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f3589640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0cd370 0x7fe0ec15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f3589640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec0ca5c0 con 0x7fe0ec0cd370 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f3589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec0ca440 con 0x7fe0ec0d3bc0 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f3589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ec0ca740 con 0x7fe0ec15f170 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1b26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0cd370 0x7fe0ec15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1b26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0cd370 0x7fe0ec15b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34478/0 (socket says 172.21.15.40:34478) 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1b26640 1 -- 172.21.15.40:0/3625329198 learned_addr learned my addr 172.21.15.40:0/3625329198 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1325640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0ec0d3bc0 0x7fe0ec1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f0b24640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0ec15f170 0x7fe0ec0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1b26640 1 -- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0ec15f170 msgr2=0x7fe0ec0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1b26640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0ec15f170 0x7fe0ec0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1b26640 1 -- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0ec0d3bc0 msgr2=0x7fe0ec1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1b26640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0ec0d3bc0 0x7fe0ec1513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.218+0000 7fe0f1b26640 1 -- 172.21.15.40:0/3625329198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec151900 con 0x7fe0ec0cd370 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.219+0000 7fe0f1b26640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0cd370 0x7fe0ec15b070 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8007e70 tx=0x7fe0e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.219+0000 7fe0e1ffb640 1 -- 172.21.15.40:0/3625329198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8014620 con 0x7fe0ec0cd370 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.219+0000 7fe0e1ffb640 1 -- 172.21.15.40:0/3625329198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e8019070 con 0x7fe0ec0cd370 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.219+0000 7fe0e1ffb640 1 -- 172.21.15.40:0/3625329198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8014a20 con 0x7fe0ec0cd370 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.219+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec0cd8a0 con 0x7fe0ec0cd370 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.219+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec0cde60 con 0x7fe0ec0cd370 2024-04-26T21:44:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.220+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ec0694f0 con 0x7fe0ec0cd370 2024-04-26T21:44:36.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.220+0000 7fe0e1ffb640 1 -- 172.21.15.40:0/3625329198 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0e8014bc0 con 0x7fe0ec0cd370 2024-04-26T21:44:36.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.221+0000 7fe0e1ffb640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.221+0000 7fe0e1ffb640 1 -- 172.21.15.40:0/3625329198 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7fe0e808cca0 con 0x7fe0ec0cd370 2024-04-26T21:44:36.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.221+0000 7fe0f1325640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.222+0000 7fe0f1325640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0d006c8f0 0x7fe0d006edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4034000 tx=0x7fe0e402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.225+0000 7fe0e1ffb640 1 -- 172.21.15.40:0/3625329198 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0ec151900 con 0x7fe0ec0cd370 2024-04-26T21:44:36.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.410+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0ec074590 con 0x7fe0ec0cd370 2024-04-26T21:44:36.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.411+0000 7fe0e1ffb640 1 -- 172.21.15.40:0/3625329198 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7fe0e805a820 con 0x7fe0ec0cd370 2024-04-26T21:44:36.411 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0d006c8f0 msgr2=0x7fe0d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0d006c8f0 0x7fe0d006edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4034000 tx=0x7fe0e402e000 comp rx=0 tx=0).stop 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0cd370 msgr2=0x7fe0ec15b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0cd370 0x7fe0ec15b070 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8007e70 tx=0x7fe0e800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 shutdown_connections 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0cd370 0x7fe0ec15b070 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0ec15f170 0x7fe0ec0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 --2- 172.21.15.40:0/3625329198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0ec0d3bc0 0x7fe0ec1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 >> 172.21.15.40:0/3625329198 conn(0x7fe0ec0d4f60 msgr2=0x7fe0ec06da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:36.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 shutdown_connections 2024-04-26T21:44:36.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.413+0000 7fe0f3589640 1 -- 172.21.15.40:0/3625329198 wait complete. 2024-04-26T21:44:36.423 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:44:36.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.528+0000 7fb5679da640 1 Processor -- start 2024-04-26T21:44:36.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.528+0000 7fb5679da640 1 -- start start 2024-04-26T21:44:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb5679da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb56015f170 0x7fb5600d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb5679da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5600d1040 con 0x7fb5600d3bc0 2024-04-26T21:44:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb5679da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5600c9680 con 0x7fb56015f550 2024-04-26T21:44:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb5679da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb5600c9430 con 0x7fb56015f170 2024-04-26T21:44:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb565776640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb56015f170 0x7fb5600d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb565f77640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb5600d3bc0 0x7fb5600ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56488/0 (socket says 172.21.15.40:56488) 2024-04-26T21:44:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb565f77640 1 -- 172.21.15.40:0/1663088708 learned_addr learned my addr 172.21.15.40:0/1663088708 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb565776640 1 -- 172.21.15.40:0/1663088708 >> v1:172.21.15.184:6789/0 conn(0x7fb56015f550 legacy=0x7fb5600d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb565776640 1 -- 172.21.15.40:0/1663088708 >> v1:172.21.15.40:6789/0 conn(0x7fb5600d3bc0 legacy=0x7fb5600ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb565776640 1 -- 172.21.15.40:0/1663088708 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5600c9960 con 0x7fb56015f170 2024-04-26T21:44:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.529+0000 7fb557fff640 1 -- 172.21.15.40:0/1663088708 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2033837670 0 0) 0x7fb558001490 con 0x7fb5600d3bc0 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.530+0000 7fb565776640 1 --2- 172.21.15.40:0/1663088708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb5600d5a80 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fb5600c95b0 tx=0x7fb55c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a043301ab86363 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.530+0000 7fb557fff640 1 -- 172.21.15.40:0/1663088708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c037070 con 0x7fb56015f170 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.530+0000 7fb557fff640 1 -- 172.21.15.40:0/1663088708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c032440 con 0x7fb56015f170 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.530+0000 7fb557fff640 1 -- 172.21.15.40:0/1663088708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c032810 con 0x7fb56015f170 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.531+0000 7fb5679da640 1 -- 172.21.15.40:0/1663088708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 msgr2=0x7fb5600d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.531+0000 7fb5679da640 1 --2- 172.21.15.40:0/1663088708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb5600d5a80 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fb5600c95b0 tx=0x7fb55c02a200 comp rx=0 tx=0).stop 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.531+0000 7fb5679da640 1 -- 172.21.15.40:0/1663088708 shutdown_connections 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.531+0000 7fb5679da640 1 --2- 172.21.15.40:0/1663088708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb5600d5a80 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.531+0000 7fb5679da640 1 -- 172.21.15.40:0/1663088708 >> 172.21.15.40:0/1663088708 conn(0x7fb5600d4f60 msgr2=0x7fb5600d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.531+0000 7fb5679da640 1 -- 172.21.15.40:0/1663088708 shutdown_connections 2024-04-26T21:44:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.531+0000 7fb5679da640 1 -- 172.21.15.40:0/1663088708 wait complete. 2024-04-26T21:44:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.532+0000 7fb5679da640 1 Processor -- start 2024-04-26T21:44:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.532+0000 7fb5679da640 1 -- start start 2024-04-26T21:44:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb5679da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5600d3bc0 0x7fb560157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb5679da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb560159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb5679da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56014a6b0 0x7fb5601f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb5679da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5600d2e30 con 0x7fb56015f170 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb5679da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5600d2cb0 con 0x7fb56014a6b0 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb5679da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5600d2fb0 con 0x7fb5600d3bc0 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb560159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb560159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34506/0 (socket says 172.21.15.40:34506) 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 -- 172.21.15.40:0/1054861818 learned_addr learned my addr 172.21.15.40:0/1054861818 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb565f77640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56014a6b0 0x7fb5601f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 -- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5600d3bc0 msgr2=0x7fb560157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb565776640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5600d3bc0 0x7fb560157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5600d3bc0 0x7fb560157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 -- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56014a6b0 msgr2=0x7fb5601f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56014a6b0 0x7fb5601f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 -- 172.21.15.40:0/1054861818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb560159c70 con 0x7fb56015f170 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb565f77640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56014a6b0 0x7fb5601f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb565776640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5600d3bc0 0x7fb560157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb564f75640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb560159520 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fb550009fd0 tx=0x7fb55000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb555ffb640 1 -- 172.21.15.40:0/1054861818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550015020 con 0x7fb56015f170 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb555ffb640 1 -- 172.21.15.40:0/1054861818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb550011830 con 0x7fb56015f170 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb560159f50 con 0x7fb56015f170 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.533+0000 7fb555ffb640 1 -- 172.21.15.40:0/1054861818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550011c00 con 0x7fb56015f170 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.534+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5601f8c50 con 0x7fb56015f170 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.535+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5600694f0 con 0x7fb56015f170 2024-04-26T21:44:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.535+0000 7fb555ffb640 1 -- 172.21.15.40:0/1054861818 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb550018020 con 0x7fb56015f170 2024-04-26T21:44:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.535+0000 7fb555ffb640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb54406c8f0 0x7fb54406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.535+0000 7fb555ffb640 1 -- 172.21.15.40:0/1054861818 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7fb55008c230 con 0x7fb56015f170 2024-04-26T21:44:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.536+0000 7fb565776640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb54406c8f0 0x7fb54406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.536+0000 7fb565776640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb54406c8f0 0x7fb54406edb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb55c034000 tx=0x7fb55c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.540+0000 7fb555ffb640 1 -- 172.21.15.40:0/1054861818 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb55001f020 con 0x7fb56015f170 2024-04-26T21:44:36.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.726+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5600b76e0 con 0x7fb56015f170 2024-04-26T21:44:36.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.726+0000 7fb555ffb640 1 -- 172.21.15.40:0/1054861818 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7fb5500172c0 con 0x7fb56015f170 2024-04-26T21:44:36.726 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:44:36.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb54406c8f0 msgr2=0x7fb54406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb54406c8f0 0x7fb54406edb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb55c034000 tx=0x7fb55c02e000 comp rx=0 tx=0).stop 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 msgr2=0x7fb560159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb560159520 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fb550009fd0 tx=0x7fb55000b870 comp rx=0 tx=0).stop 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 shutdown_connections 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb54406c8f0 0x7fb54406edb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56014a6b0 0x7fb5601f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb56015f170 0x7fb560159520 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 --2- 172.21.15.40:0/1054861818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5600d3bc0 0x7fb560157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 >> 172.21.15.40:0/1054861818 conn(0x7fb5600d4f60 msgr2=0x7fb5600ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 shutdown_connections 2024-04-26T21:44:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.728+0000 7fb5679da640 1 -- 172.21.15.40:0/1054861818 wait complete. 2024-04-26T21:44:36.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:44:36.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.841+0000 7f56aa74d640 1 Processor -- start 2024-04-26T21:44:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.841+0000 7f56aa74d640 1 -- start start 2024-04-26T21:44:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.841+0000 7f56aa74d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56a4071e60 0x7f56a406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.841+0000 7f56aa74d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a406fe20 0x7f56a4072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.841+0000 7f56aa74d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a415d7c0 con 0x7f56a4072780 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.841+0000 7f56aa74d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f56a415daa0 con 0x7f56a4071e60 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.841+0000 7f56aa74d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56a415dd80 con 0x7f56a406fe20 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a3fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56a4071e60 0x7f56a406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a37fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a406fe20 0x7f56a4072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a8cea640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f56a4072780 0x7f56a40cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:60314/0 (socket says 172.21.15.40:60314) 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a8cea640 1 -- 172.21.15.40:0/3689660267 learned_addr learned my addr 172.21.15.40:0/3689660267 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a3fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56a4071e60 0x7f56a406f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:54438/0 (socket says 172.21.15.40:54438) 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a37fe640 1 -- 172.21.15.40:0/3689660267 >> v2:172.21.15.184:3300/0 conn(0x7f56a4071e60 msgr2=0x7f56a406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a37fe640 1 --2- 172.21.15.40:0/3689660267 >> v2:172.21.15.184:3300/0 conn(0x7f56a4071e60 0x7f56a406f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a37fe640 1 -- 172.21.15.40:0/3689660267 >> v1:172.21.15.184:6789/0 conn(0x7f56a4072780 legacy=0x7f56a40cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a37fe640 1 -- 172.21.15.40:0/3689660267 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a415e060 con 0x7f56a406fe20 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a2ffd640 1 -- 172.21.15.40:0/3689660267 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398806949 0 0) 0x7f56a415d7c0 con 0x7f56a4072780 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.842+0000 7f56a3fff640 1 --2- 172.21.15.40:0/3689660267 >> v2:172.21.15.184:3300/0 conn(0x7f56a4071e60 0x7f56a406f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.843+0000 7f56a37fe640 1 --2- 172.21.15.40:0/3689660267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a406fe20 0x7f56a4072240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f56980097a0 tx=0x7f569802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a72fa5abb59ae8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.843+0000 7f56a2ffd640 1 -- 172.21.15.40:0/3689660267 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698037070 con 0x7f56a406fe20 2024-04-26T21:44:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.843+0000 7f56a2ffd640 1 -- 172.21.15.40:0/3689660267 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5698032440 con 0x7f56a406fe20 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.843+0000 7f56a2ffd640 1 -- 172.21.15.40:0/3689660267 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698032810 con 0x7f56a406fe20 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.844+0000 7f56aa74d640 1 -- 172.21.15.40:0/3689660267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a406fe20 msgr2=0x7f56a4072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.844+0000 7f56aa74d640 1 --2- 172.21.15.40:0/3689660267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a406fe20 0x7f56a4072240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f56980097a0 tx=0x7f569802a180 comp rx=0 tx=0).stop 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.844+0000 7f56aa74d640 1 -- 172.21.15.40:0/3689660267 shutdown_connections 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.844+0000 7f56aa74d640 1 --2- 172.21.15.40:0/3689660267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a406fe20 0x7f56a4072240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.844+0000 7f56aa74d640 1 --2- 172.21.15.40:0/3689660267 >> v2:172.21.15.184:3300/0 conn(0x7f56a4071e60 0x7f56a406f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.844+0000 7f56aa74d640 1 -- 172.21.15.40:0/3689660267 >> 172.21.15.40:0/3689660267 conn(0x7f56a40d0be0 msgr2=0x7f56a40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.844+0000 7f56aa74d640 1 -- 172.21.15.40:0/3689660267 shutdown_connections 2024-04-26T21:44:36.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.844+0000 7f56aa74d640 1 -- 172.21.15.40:0/3689660267 wait complete. 2024-04-26T21:44:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.845+0000 7f56aa74d640 1 Processor -- start 2024-04-26T21:44:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.845+0000 7f56aa74d640 1 -- start start 2024-04-26T21:44:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56aa74d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a406fe20 0x7f56a40b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56aa74d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56a4071e60 0x7f56a40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56aa74d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a4072780 0x7f56a41f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56aa74d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56a415d3f0 con 0x7f56a406fe20 2024-04-26T21:44:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56aa74d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4072c10 con 0x7f56a4071e60 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56aa74d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56a415d570 con 0x7f56a4072780 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a406fe20 0x7f56a40b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a406fe20 0x7f56a40b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34520/0 (socket says 172.21.15.40:34520) 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a3fff640 1 -- 172.21.15.40:0/1242527740 learned_addr learned my addr 172.21.15.40:0/1242527740 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a8cea640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a4072780 0x7f56a41f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a37fe640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56a4071e60 0x7f56a40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a3fff640 1 -- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a4072780 msgr2=0x7f56a41f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a3fff640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a4072780 0x7f56a41f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a3fff640 1 -- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56a4071e60 msgr2=0x7f56a40b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a3fff640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56a4071e60 0x7f56a40b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.846+0000 7f56a3fff640 1 -- 172.21.15.40:0/1242527740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a40b3e40 con 0x7f56a406fe20 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.847+0000 7f56a3fff640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a406fe20 0x7f56a40b37b0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f569400ebe0 tx=0x7f569400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.847+0000 7f56a0ff9640 1 -- 172.21.15.40:0/1242527740 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694017070 con 0x7f56a406fe20 2024-04-26T21:44:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.847+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a40b4120 con 0x7f56a406fe20 2024-04-26T21:44:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.847+0000 7f56a0ff9640 1 -- 172.21.15.40:0/1242527740 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569400cc30 con 0x7f56a406fe20 2024-04-26T21:44:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.847+0000 7f56a0ff9640 1 -- 172.21.15.40:0/1242527740 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694013470 con 0x7f56a406fe20 2024-04-26T21:44:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.847+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a40b46e0 con 0x7f56a406fe20 2024-04-26T21:44:36.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.848+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a40694f0 con 0x7f56a406fe20 2024-04-26T21:44:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.848+0000 7f56a0ff9640 1 -- 172.21.15.40:0/1242527740 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f569401a070 con 0x7f56a406fe20 2024-04-26T21:44:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.849+0000 7f56a0ff9640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f569006c8f0 0x7f569006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.849+0000 7f56a0ff9640 1 -- 172.21.15.40:0/1242527740 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f569408c410 con 0x7f56a406fe20 2024-04-26T21:44:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.849+0000 7f56a37fe640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f569006c8f0 0x7f569006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.849+0000 7f56a37fe640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f569006c8f0 0x7f569006edb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5698034000 tx=0x7f569802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:36.853+0000 7f56a0ff9640 1 -- 172.21.15.40:0/1242527740 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5694059f10 con 0x7f56a406fe20 2024-04-26T21:44:37.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.054+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f56a40cc730 con 0x7f56a406fe20 2024-04-26T21:44:37.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.055+0000 7f56a0ff9640 1 -- 172.21.15.40:0/1242527740 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 36 entries v113) ==== 81+0+2188 (secure 0 0 0) 0x7f5694059810 con 0x7f56a406fe20 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:44:37.055 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:44:37.056 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:44:37.057 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:44:37.058 INFO:teuthology.orchestra.run.smithi040.stderr:listed 36 entries 2024-04-26T21:44:37.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f569006c8f0 msgr2=0x7f569006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f569006c8f0 0x7f569006edb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5698034000 tx=0x7f569802e000 comp rx=0 tx=0).stop 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a406fe20 msgr2=0x7f56a40b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a406fe20 0x7f56a40b37b0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f569400ebe0 tx=0x7f569400c520 comp rx=0 tx=0).stop 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 shutdown_connections 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f569006c8f0 0x7f569006edb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56a4072780 0x7f56a41f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56a4071e60 0x7f56a40b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 --2- 172.21.15.40:0/1242527740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a406fe20 0x7f56a40b37b0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 >> 172.21.15.40:0/1242527740 conn(0x7f56a40d0be0 msgr2=0x7f56a40cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 shutdown_connections 2024-04-26T21:44:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.057+0000 7f56aa74d640 1 -- 172.21.15.40:0/1242527740 wait complete. 2024-04-26T21:44:37.067 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:44:37.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:37.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:44:37.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:37.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:37.093 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:37.148 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:37.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.290+0000 7f1cd7df0640 1 Processor -- start 2024-04-26T21:44:37.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.290+0000 7f1cd7df0640 1 -- start start 2024-04-26T21:44:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd7df0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cd00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd7df0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd00b7e50 con 0x7f1cd0162880 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd7df0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd00d1800 con 0x7f1cd00b5f20 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd7df0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd00d0580 con 0x7f1cd00d3bc0 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd5b8c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1cd0162880 0x7f1cd00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56496/0 (socket says 172.21.15.40:56496) 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd5b8c640 1 -- 172.21.15.40:0/3141796060 learned_addr learned my addr 172.21.15.40:0/3141796060 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800534086 0 0) 0x7f1cd00b7e50 con 0x7f1cd0162880 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cb8003520 con 0x7f1cd0162880 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.291+0000 7f1cd538b640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cd00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 885501777 0 0) 0x7f1cb8003520 con 0x7f1cd0162880 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 msgr2=0x7f1cd00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cd00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 >> v1:172.21.15.184:6789/0 conn(0x7f1cd00b5f20 legacy=0x7f1cd00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd00d07d0 con 0x7f1cd0162880 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1cc00019f0 con 0x7f1cd0162880 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 >> v1:172.21.15.40:6789/0 conn(0x7f1cd0162880 legacy=0x7f1cd00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cb80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cb8003f60 0x7f1cb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb8004d40 0x7f1cb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd00d0580 con 0x7f1cb8004d40 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb800baa0 con 0x7f1cb8003f60 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb800bca0 con 0x7f1cd00d3bc0 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd5b8c640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cb80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd638d640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb8004d40 0x7f1cb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd538b640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cb8003f60 0x7f1cb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd5b8c640 1 -- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cb8003f60 msgr2=0x7f1cb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:37.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd5b8c640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cb8003f60 0x7f1cb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd5b8c640 1 -- 172.21.15.40:0/3141796060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb8004d40 msgr2=0x7f1cb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd5b8c640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb8004d40 0x7f1cb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd5b8c640 1 -- 172.21.15.40:0/3141796060 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb8003520 con 0x7f1cd00d3bc0 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.292+0000 7f1cd638d640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb8004d40 0x7f1cb80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.293+0000 7f1cd5b8c640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cb80039e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000a860 tx=0x7f1cb8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c235422b645a03a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.293+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000a480 con 0x7f1cd00d3bc0 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.293+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc002f070 con 0x7f1cd00d3bc0 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.293+0000 7f1cd4b8a640 1 -- 172.21.15.40:0/3141796060 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc002bd40 con 0x7f1cd00d3bc0 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.294+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 msgr2=0x7f1cb80039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.294+0000 7f1cd7df0640 1 --2- 172.21.15.40:0/3141796060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cb80039e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000a860 tx=0x7f1cb8005830 comp rx=0 tx=0).stop 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.294+0000 7f1cd638d640 1 -- 172.21.15.40:0/3141796060 reap_dead start 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.294+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3141796060 shutdown_connections 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.294+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3141796060 >> 172.21.15.40:0/3141796060 conn(0x7f1cd00d4f60 msgr2=0x7f1cd00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.294+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3141796060 shutdown_connections 2024-04-26T21:44:37.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.294+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3141796060 wait complete. 2024-04-26T21:44:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.295+0000 7f1cd7df0640 1 Processor -- start 2024-04-26T21:44:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.295+0000 7f1cd7df0640 1 -- start start 2024-04-26T21:44:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd7df0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cd00cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd7df0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd01f4a50 0x7f1cd015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd7df0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd00b5770 0x7f1cd006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd7df0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0162c60 con 0x7f1cd01f4a50 2024-04-26T21:44:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd7df0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd00d0d90 con 0x7f1cd00b5770 2024-04-26T21:44:37.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd7df0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd0162de0 con 0x7f1cd00d3bc0 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd01f4a50 0x7f1cd015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd01f4a50 0x7f1cd015be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34534/0 (socket says 172.21.15.40:34534) 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 -- 172.21.15.40:0/3599559055 learned_addr learned my addr 172.21.15.40:0/3599559055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd5b8c640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cd00cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 -- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 msgr2=0x7f1cd00cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cd00cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 -- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd00b5770 msgr2=0x7f1cd006cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd00b5770 0x7f1cd006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 -- 172.21.15.40:0/3599559055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd015ca70 con 0x7f1cd01f4a50 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.296+0000 7f1cd538b640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd01f4a50 0x7f1cd015be70 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40119c0 tx=0x7f1cc4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.297+0000 7f1cce7fc640 1 -- 172.21.15.40:0/3599559055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400fa70 con 0x7f1cd01f4a50 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.297+0000 7f1cce7fc640 1 -- 172.21.15.40:0/3599559055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc400fc10 con 0x7f1cd01f4a50 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.297+0000 7f1cce7fc640 1 -- 172.21.15.40:0/3599559055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4014440 con 0x7f1cd01f4a50 2024-04-26T21:44:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.297+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd00cdd40 con 0x7f1cd01f4a50 2024-04-26T21:44:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.297+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd00b1e30 con 0x7f1cd01f4a50 2024-04-26T21:44:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.298+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd00694f0 con 0x7f1cd01f4a50 2024-04-26T21:44:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.298+0000 7f1cce7fc640 1 -- 172.21.15.40:0/3599559055 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cc4012040 con 0x7f1cd01f4a50 2024-04-26T21:44:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.298+0000 7f1cce7fc640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cbc06c8f0 0x7f1cbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.299+0000 7f1cce7fc640 1 -- 172.21.15.40:0/3599559055 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7765+0+0 (secure 0 0 0) 0x7f1cc408c980 con 0x7f1cd01f4a50 2024-04-26T21:44:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.299+0000 7f1cd5b8c640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cbc06c8f0 0x7f1cbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.300+0000 7f1cd5b8c640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cbc06c8f0 0x7f1cbc06edb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0033000 tx=0x7f1cc0009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:37.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.303+0000 7f1cce7fc640 1 -- 172.21.15.40:0/3599559055 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cd015ca70 con 0x7f1cd01f4a50 2024-04-26T21:44:37.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.530+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1cd0074590 con 0x7f1cd01f4a50 2024-04-26T21:44:37.530 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:37.530+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:37.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.532+0000 7f1cce7fc640 1 -- 172.21.15.40:0/3599559055 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v210) ==== 161+0+0 (secure 0 0 0) 0x7f1cc4019030 con 0x7f1cd01f4a50 2024-04-26T21:44:37.531 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:37.534 INFO:tasks.ceph.mds.c.smithi040.stderr: -1840> 2024-04-26T21:44:26.422+0000 7f6907ec3640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-04-26T21:44:37.534 INFO:tasks.ceph.mds.c.smithi040.stderr: -1837> 2024-04-26T21:44:26.422+0000 7f6907ec3640 -1 mds.0.206 unhandled write error (22) Invalid argument, force readonly... 2024-04-26T21:44:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cbc06c8f0 msgr2=0x7f1cbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cbc06c8f0 0x7f1cbc06edb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0033000 tx=0x7f1cc0009030 comp rx=0 tx=0).stop 2024-04-26T21:44:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd01f4a50 msgr2=0x7f1cd015be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd01f4a50 0x7f1cd015be70 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40119c0 tx=0x7f1cc4011e80 comp rx=0 tx=0).stop 2024-04-26T21:44:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 shutdown_connections 2024-04-26T21:44:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1cbc06c8f0 0x7f1cbc06edb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd00b5770 0x7f1cd006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cd01f4a50 0x7f1cd015be70 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 --2- 172.21.15.40:0/3599559055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cd00d3bc0 0x7f1cd00cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 >> 172.21.15.40:0/3599559055 conn(0x7f1cd00d4f60 msgr2=0x7f1cd01619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 shutdown_connections 2024-04-26T21:44:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.534+0000 7f1cd7df0640 1 -- 172.21.15.40:0/3599559055 wait complete. 2024-04-26T21:44:37.539 INFO:tasks.ceph.mds.c.smithi040.stderr: -665> 2024-04-26T21:44:33.403+0000 7f690aec9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:44:37.546 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:37.560 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:37.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.648+0000 7fec26746640 1 Processor -- start 2024-04-26T21:44:37.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.648+0000 7fec26746640 1 -- start start 2024-04-26T21:44:37.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.649+0000 7fec26746640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec2015f3a0 0x7fec200d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.649+0000 7fec26746640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec200d0e20 con 0x7fec200d2330 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.649+0000 7fec26746640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec200b1d70 con 0x7fec200d1770 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.649+0000 7fec26746640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fec200b2050 con 0x7fec2015f3a0 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.649+0000 7fec1ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec2015f3a0 0x7fec200d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.649+0000 7fec24ce3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fec200d2330 0x7fec200b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56510/0 (socket says 172.21.15.40:56510) 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.649+0000 7fec24ce3640 1 -- 172.21.15.40:0/29855140 learned_addr learned my addr 172.21.15.40:0/29855140 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.650+0000 7fec1ffff640 1 -- 172.21.15.40:0/29855140 >> v1:172.21.15.184:6789/0 conn(0x7fec200d1770 legacy=0x7fec200d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.650+0000 7fec1ffff640 1 -- 172.21.15.40:0/29855140 >> v1:172.21.15.40:6789/0 conn(0x7fec200d2330 legacy=0x7fec200b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.650+0000 7fec1effd640 1 -- 172.21.15.40:0/29855140 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179464966 0 0) 0x7fec200d0e20 con 0x7fec200d2330 2024-04-26T21:44:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.650+0000 7fec1ffff640 1 -- 172.21.15.40:0/29855140 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec200b2330 con 0x7fec2015f3a0 2024-04-26T21:44:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.650+0000 7fec1ffff640 1 --2- 172.21.15.40:0/29855140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2015f3a0 0x7fec200d06f0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fec100097a0 tx=0x7fec1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b729f89eec5e0408 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.650+0000 7fec1effd640 1 -- 172.21.15.40:0/29855140 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10037070 con 0x7fec2015f3a0 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec1effd640 1 -- 172.21.15.40:0/29855140 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec10032440 con 0x7fec2015f3a0 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec1effd640 1 -- 172.21.15.40:0/29855140 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10032810 con 0x7fec2015f3a0 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec26746640 1 -- 172.21.15.40:0/29855140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2015f3a0 msgr2=0x7fec200d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec26746640 1 --2- 172.21.15.40:0/29855140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2015f3a0 0x7fec200d06f0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fec100097a0 tx=0x7fec1002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec26746640 1 -- 172.21.15.40:0/29855140 shutdown_connections 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec26746640 1 --2- 172.21.15.40:0/29855140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2015f3a0 0x7fec200d06f0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec26746640 1 -- 172.21.15.40:0/29855140 >> 172.21.15.40:0/29855140 conn(0x7fec200d4f90 msgr2=0x7fec200d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec26746640 1 -- 172.21.15.40:0/29855140 shutdown_connections 2024-04-26T21:44:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.651+0000 7fec26746640 1 -- 172.21.15.40:0/29855140 wait complete. 2024-04-26T21:44:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.652+0000 7fec26746640 1 Processor -- start 2024-04-26T21:44:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.652+0000 7fec26746640 1 -- start start 2024-04-26T21:44:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec26746640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec200d1770 0x7fec201535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec26746640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec200d2330 0x7fec200b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec26746640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2015f3a0 0x7fec200b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec26746640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec200b30e0 con 0x7fec200d1770 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec26746640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec200b2f60 con 0x7fec200d2330 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec26746640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec200b3260 con 0x7fec2015f3a0 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec200d1770 0x7fec201535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec200d1770 0x7fec201535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34568/0 (socket says 172.21.15.40:34568) 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.653+0000 7fec1ffff640 1 -- 172.21.15.40:0/2363098124 learned_addr learned my addr 172.21.15.40:0/2363098124 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1f7fe640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec200d2330 0x7fec200b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1ffff640 1 -- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2015f3a0 msgr2=0x7fec200b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec24ce3640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2015f3a0 0x7fec200b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1ffff640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2015f3a0 0x7fec200b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1ffff640 1 -- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec200d2330 msgr2=0x7fec200b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1ffff640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec200d2330 0x7fec200b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1ffff640 1 -- 172.21.15.40:0/2363098124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec20153ae0 con 0x7fec200d1770 2024-04-26T21:44:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1f7fe640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec200d2330 0x7fec200b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1ffff640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec200d1770 0x7fec201535a0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fec100097a0 tx=0x7fec100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1cff9640 1 -- 172.21.15.40:0/2363098124 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10041070 con 0x7fec200d1770 2024-04-26T21:44:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1cff9640 1 -- 172.21.15.40:0/2363098124 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1003c870 con 0x7fec200d1770 2024-04-26T21:44:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec200ca9b0 con 0x7fec200d1770 2024-04-26T21:44:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec1cff9640 1 -- 172.21.15.40:0/2363098124 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10037040 con 0x7fec200d1770 2024-04-26T21:44:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.654+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec200caf70 con 0x7fec200d1770 2024-04-26T21:44:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.655+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec200694f0 con 0x7fec200d1770 2024-04-26T21:44:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.656+0000 7fec1cff9640 1 -- 172.21.15.40:0/2363098124 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec10044070 con 0x7fec200d1770 2024-04-26T21:44:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.656+0000 7fec1cff9640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.656+0000 7fec1cff9640 1 -- 172.21.15.40:0/2363098124 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7851+0+0 (secure 0 0 0) 0x7fec100acf60 con 0x7fec200d1770 2024-04-26T21:44:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.656+0000 7fec1f7fe640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.657+0000 7fec1f7fe640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fec14007fd0 tx=0x7fec14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.661+0000 7fec1cff9640 1 -- 172.21.15.40:0/2363098124 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec1007ab30 con 0x7fec200d1770 2024-04-26T21:44:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:37.918+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fec200b6860 con 0x7fec200d1770 2024-04-26T21:44:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.537+0000 7fec1cff9640 1 -- 172.21.15.40:0/2363098124 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v211) ==== 107+0+0 (secure 0 0 0) 0x7fec1007a430 con 0x7fec200d1770 2024-04-26T21:44:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:44:38.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 msgr2=0x7fec0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fec14007fd0 tx=0x7fec14009040 comp rx=0 tx=0).stop 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec200d1770 msgr2=0x7fec201535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec200d1770 0x7fec201535a0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fec100097a0 tx=0x7fec100046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 shutdown_connections 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2015f3a0 0x7fec200b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec200d2330 0x7fec200b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 --2- 172.21.15.40:0/2363098124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec200d1770 0x7fec201535a0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.539+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 >> 172.21.15.40:0/2363098124 conn(0x7fec200d4f90 msgr2=0x7fec200d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.540+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 shutdown_connections 2024-04-26T21:44:38.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.540+0000 7fec26746640 1 -- 172.21.15.40:0/2363098124 wait complete. 2024-04-26T21:44:38.551 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:38.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.636+0000 7f1e40ef0f40 1 Processor -- start 2024-04-26T21:44:38.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e40ef0f40 1 -- start start 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e40ef0f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e28002ca0 0x55a998cc8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e40ef0f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 0x55a998cd2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e40ef0f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a998cc9d80 0x55a998cc70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e40ef0f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a998ba2e40 con 0x55a998cc9d80 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e40ef0f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a998ba6e80 con 0x55a998cd3610 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e40ef0f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a998c42c60 con 0x7f1e28002ca0 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e3f1eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a998cc9d80 0x55a998cc70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e3f1eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a998cc9d80 0x55a998cc70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34354/0 (socket says 172.21.15.40:34354) 2024-04-26T21:44:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.637+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/2947035794 learned_addr learned my addr 172.21.15.40:0/2947035794 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 0x55a998cd2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3e9ea640 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e28002ca0 0x55a998cc8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/2947035794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e28002ca0 msgr2=0x55a998cc8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3f1eb640 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e28002ca0 0x55a998cc8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/2947035794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 msgr2=0x55a998cd2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3f1eb640 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 0x55a998cd2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/2947035794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a998c44360 con 0x55a998cc9d80 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3f1eb640 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a998cc9d80 0x55a998cc70f0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800ab20 tx=0x7f1e38003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d5e4b6a5e044e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3d1e7640 1 -- 172.21.15.40:0/2947035794 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38018070 con 0x55a998cc9d80 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3d1e7640 1 -- 172.21.15.40:0/2947035794 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e38003110 con 0x55a998cc9d80 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e3d1e7640 1 -- 172.21.15.40:0/2947035794 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380034e0 con 0x55a998cc9d80 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/2947035794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a998cc9d80 msgr2=0x55a998cc70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a998cc9d80 0x55a998cc70f0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800ab20 tx=0x7f1e38003aa0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/2947035794 shutdown_connections 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a998cc9d80 0x55a998cc70f0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 0x55a998cd2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/2947035794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e28002ca0 0x55a998cc8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.638+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/2947035794 >> 172.21.15.40:0/2947035794 conn(0x55a998bfa330 msgr2=0x55a998ccf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.639+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/2947035794 shutdown_connections 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.639+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/2947035794 wait complete. 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.639+0000 7f1e40ef0f40 1 Processor -- start 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.639+0000 7f1e40ef0f40 1 -- start start 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e40ef0f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a998cc9d80 0x55a998c3dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e40ef0f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 0x55a998cc7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e40ef0f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e28002ca0 0x55a998cc5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e40ef0f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a998cd3a70 con 0x7f1e28002ca0 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e40ef0f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a998cc6f10 con 0x55a998cd3610 2024-04-26T21:44:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e40ef0f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55a998cd2d50 con 0x55a998cc9d80 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e28002ca0 0x55a998cc5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e28002ca0 0x55a998cc5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34362/0 (socket says 172.21.15.40:34362) 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 learned_addr learned my addr 172.21.15.40:0/240590413 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 0x55a998cc7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3e9ea640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a998cc9d80 0x55a998c3dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a998cc9d80 msgr2=0x55a998c3dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a998cc9d80 0x55a998c3dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 msgr2=0x55a998cc7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 0x55a998cc7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a998cc6ca0 con 0x7f1e28002ca0 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a998cd3610 0x55a998cc7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.640+0000 7f1e3f1eb640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e28002ca0 0x55a998cc5620 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x55a998bfef50 tx=0x7f1e380015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.641+0000 7f1e27fff640 1 -- 172.21.15.40:0/240590413 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38025070 con 0x7f1e28002ca0 2024-04-26T21:44:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.641+0000 7f1e27fff640 1 -- 172.21.15.40:0/240590413 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e38013670 con 0x7f1e28002ca0 2024-04-26T21:44:38.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.641+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a998bfd080 con 0x7f1e28002ca0 2024-04-26T21:44:38.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.641+0000 7f1e27fff640 1 -- 172.21.15.40:0/240590413 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38018070 con 0x7f1e28002ca0 2024-04-26T21:44:38.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.641+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a998c3ce00 con 0x7f1e28002ca0 2024-04-26T21:44:38.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.642+0000 7f1e27fff640 1 -- 172.21.15.40:0/240590413 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e38028070 con 0x7f1e28002ca0 2024-04-26T21:44:38.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.642+0000 7f1e27fff640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e2c06b2d0 0x7f1e2c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.643+0000 7f1e27fff640 1 -- 172.21.15.40:0/240590413 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7851+0+0 (secure 0 0 0) 0x7f1e3808bea0 con 0x7f1e28002ca0 2024-04-26T21:44:38.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.643+0000 7f1e3e9ea640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e2c06b2d0 0x7f1e2c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.646 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.643+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a998c3e100 0x55a998c3fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.643+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998c42110 con 0x55a998c3e100 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.643+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a998c3e100 0x55a998c3fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.643+0000 7f1e3e9ea640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e2c06b2d0 0x7f1e2c06d790 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1e30005cf0 tx=0x7f1e3000c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.643+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a998c3e100 0x55a998c3fda0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.645+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v114'567 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.645+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998c27020 con 0x55a998c3e100 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.646+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v114'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.646+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc77c0 con 0x55a998c3e100 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.647+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v114'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.648 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:38.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.647+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d74f50 con 0x55a998c3e100 2024-04-26T21:44:38.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.648+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v114'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.648+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d752f0 con 0x55a998c3e100 2024-04-26T21:44:38.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.649+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v114'571 uv571 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.649+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d73f40 con 0x55a998c3e100 2024-04-26T21:44:38.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.650+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v114'572 uv572 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.650+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d74330 con 0x55a998c3e100 2024-04-26T21:44:38.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.651+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v114'573 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.651+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d74720 con 0x55a998c3e100 2024-04-26T21:44:38.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.652+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v114'574 uv574 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.652+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d74720 con 0x55a998c3e100 2024-04-26T21:44:38.652 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.653+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v114'575 uv575 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.653+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.654+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v114'576 uv576 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.654+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.655+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v114'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.655+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.656+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v114'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.656+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.657+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v114'579 uv554 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.657+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.658+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v114'580 uv580 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.658+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.659+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v114'581 uv581 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.659 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:38.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.659+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.660+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v114'582 uv582 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.660+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.662+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v114'583 uv583 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.662+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d746d0 con 0x55a998c3e100 2024-04-26T21:44:38.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.663+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v114'584 uv584 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.663+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d74ad0 con 0x55a998c3e100 2024-04-26T21:44:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.664+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v114'585 uv585 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.664+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.665+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v114'586 uv560 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.665+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.666+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v114'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.666+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.667+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v114'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.667 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:38.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.667+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.668+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v114'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.668+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.669+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v114'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.669+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.670+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v114'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.670+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.671+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v114'592 uv566 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.671+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.672+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v114'593 uv593 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.672+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.673+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v114'594 uv594 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.673 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:38.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.673+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.674+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v114'595 uv595 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.675+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.676+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v114'596 uv596 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.676+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998cc5b60 con 0x55a998c3e100 2024-04-26T21:44:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.677+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v114'597 uv597 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998c3e100 2024-04-26T21:44:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.677+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a998d75690 0x55a998d77a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.677+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d77fc0 con 0x55a998d75690 2024-04-26T21:44:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.677+0000 7f1e3e9ea640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a998d75690 0x55a998d77a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.677+0000 7f1e3e9ea640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a998d75690 0x55a998d77a80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.679+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v114'421 uv400 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.679+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d78360 con 0x55a998d75690 2024-04-26T21:44:38.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.680+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v114'422 uv422 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.680+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d78700 con 0x55a998d75690 2024-04-26T21:44:38.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.681+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v114'423 uv423 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.681 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:38.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.682+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d78f50 con 0x55a998d75690 2024-04-26T21:44:38.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.683+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v114'424 uv424 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.683+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d792f0 con 0x55a998d75690 2024-04-26T21:44:38.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.684+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v114'425 uv425 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.684+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79690 con 0x55a998d75690 2024-04-26T21:44:38.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.685+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v114'426 uv419 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.685+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d79a30 con 0x55a998d75690 2024-04-26T21:44:38.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.686+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v114'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.686+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.687 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:38.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.687+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v114'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.687+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.689+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v114'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.689+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.690+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v114'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.690+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.691+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v114'431 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.691+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.692+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v114'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.692+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.693 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:38.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.693+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v114'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.693+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7e490 con 0x55a998d75690 2024-04-26T21:44:38.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.694+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v114'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.694+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7e490 con 0x55a998d75690 2024-04-26T21:44:38.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.695+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v114'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.695+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.697+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v114'436 uv416 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.697+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.698+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v114'437 uv437 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.698+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.698 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:38.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.699+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v114'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.699+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7ade0 con 0x55a998d75690 2024-04-26T21:44:38.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.700+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v114'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.700+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7ade0 con 0x55a998d75690 2024-04-26T21:44:38.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.701+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v114'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.701+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d79dd0 con 0x55a998d75690 2024-04-26T21:44:38.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.702+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v114'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d75690 2024-04-26T21:44:38.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.702+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a998d7a2e0 0x55a998d7c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.702+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7cc10 con 0x55a998d7a2e0 2024-04-26T21:44:38.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.702+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a998d7a2e0 0x55a998d7c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.703+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a998d7a2e0 0x55a998d7c6d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.705+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v114'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.705+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d7cc10 con 0x55a998d7a2e0 2024-04-26T21:44:38.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.706+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v114'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.706+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7cc10 con 0x55a998d7a2e0 2024-04-26T21:44:38.706 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:38.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.707+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v114'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.707+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7e010 con 0x55a998d7a2e0 2024-04-26T21:44:38.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.708+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v114'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.708+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7e010 con 0x55a998d7a2e0 2024-04-26T21:44:38.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.709+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v114'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.709+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7d000 con 0x55a998d7a2e0 2024-04-26T21:44:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.710+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v114'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.710+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a998d7d510 0x55a998d7f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.710+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d7ffa0 con 0x55a998d7d510 2024-04-26T21:44:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.711+0000 7f1e3f1eb640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a998d7d510 0x55a998d7f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.711+0000 7f1e3f1eb640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a998d7d510 0x55a998d7f920 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.712+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v114'88 uv87 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.712+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d804b0 con 0x55a998d7d510 2024-04-26T21:44:38.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.713+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v114'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.713+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d809b0 con 0x55a998d7d510 2024-04-26T21:44:38.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.714+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v114'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.714 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:38.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.715+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d81270 con 0x55a998d7d510 2024-04-26T21:44:38.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.716+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v114'91 uv91 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.716+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a998d81b20 0x55a998d83f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.716+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d84450 con 0x55a998d81b20 2024-04-26T21:44:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.716+0000 7f1e3e9ea640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a998d81b20 0x55a998d83f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.716+0000 7f1e3e9ea640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a998d81b20 0x55a998d83f10 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.717+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v114'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.717+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d84530 con 0x55a998d81b20 2024-04-26T21:44:38.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.719+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v114'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.719+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d84950 con 0x55a998d81b20 2024-04-26T21:44:38.719 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:38.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.720+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v114'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.720+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85960 con 0x55a998d81b20 2024-04-26T21:44:38.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.721+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v114'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.721+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85960 con 0x55a998d81b20 2024-04-26T21:44:38.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.722+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v114'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.722+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d84950 con 0x55a998d81b20 2024-04-26T21:44:38.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.724+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v114'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.724+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d84950 con 0x55a998d7d510 2024-04-26T21:44:38.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.725+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v114'205 uv198 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.725+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d84cf0 con 0x55a998d7d510 2024-04-26T21:44:38.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.726+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v114'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.726+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85090 con 0x55a998d7d510 2024-04-26T21:44:38.727 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:38.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.727+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v114'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.727+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d86440 con 0x55a998d7d510 2024-04-26T21:44:38.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.728+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v114'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.728+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d86440 con 0x55a998d7d510 2024-04-26T21:44:38.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.729+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v114'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.729+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.730+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v114'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.730+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.731+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v114'211 uv204 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.732+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.733+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v114'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.733+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.733 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:38.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.734+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v114'213 uv213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.734+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.735+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v114'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.735+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.736+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v114'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.736+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.737+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v114'91 uv90 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.737+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.738+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v114'92 uv92 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.738+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.739+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v114'93 uv93 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.739 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:38.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.739+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.741+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v114'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.741+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d7d510 2024-04-26T21:44:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.741+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v114'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1e38027030 con 0x55a998d7d510 2024-04-26T21:44:38.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.741+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d81b20 2024-04-26T21:44:38.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.742+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v114'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.742+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d81b20 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.743+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v114'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.743+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d81b20 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.744+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v114'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.744+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d86480 con 0x55a998d81b20 2024-04-26T21:44:38.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.745+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v114'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.745+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d86440 con 0x55a998d81b20 2024-04-26T21:44:38.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.746+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v114'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.746+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d81b20 2024-04-26T21:44:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.747+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v114'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.747+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d81b20 2024-04-26T21:44:38.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.748+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v114'182 uv176 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.748+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d81b20 2024-04-26T21:44:38.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.749+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v114'183 uv183 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.749+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d85430 con 0x55a998d81b20 2024-04-26T21:44:38.750 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.750+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v114'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.750+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d854d0 con 0x55a998d81b20 2024-04-26T21:44:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.751+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v114'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.751+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d854d0 con 0x55a998d81b20 2024-04-26T21:44:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.752+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v114'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.752+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d854d0 con 0x55a998d81b20 2024-04-26T21:44:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.753+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v114'187 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.753+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d854d0 con 0x55a998d81b20 2024-04-26T21:44:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.755+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v114'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.755+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d854d0 con 0x55a998d81b20 2024-04-26T21:44:38.755 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.756+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v114'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.756+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d854d0 con 0x55a998d81b20 2024-04-26T21:44:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.757+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v114'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.757+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d80ed0 con 0x55a998d81b20 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.758+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v114'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.758+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a998d80e90 0x55a998d885d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.758+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d80e90 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.758+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a998d80e90 0x55a998d885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.758+0000 7f1e3e1e9640 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a998d80e90 0x55a998d885d0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.759+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v114'83 uv81 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d80e90 2024-04-26T21:44:38.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.759+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d80e90 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.761+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v114'84 uv84 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d80e90 2024-04-26T21:44:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.761+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d80e90 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.761+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v114'85 uv85 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d80e90 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.761+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d80e90 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.764+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v114'86 uv86 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d80e90 2024-04-26T21:44:38.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.764+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d81b20 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.766+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v114'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.766+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d81b20 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.767+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v114'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.767+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d81b20 2024-04-26T21:44:38.767 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.767+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v114'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.768+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d81b20 2024-04-26T21:44:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.769+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v114'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.769+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d81b20 2024-04-26T21:44:38.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.770+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v114'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.770+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d81b20 2024-04-26T21:44:38.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.771+0000 7f1e3e9ea640 1 -- 172.21.15.40:0/240590413 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v114'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e30021040 con 0x55a998d81b20 2024-04-26T21:44:38.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.771+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.772+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v114'94 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.772+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.773+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v114'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.773+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.806 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:38.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.806+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v114'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.806+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.808+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v114'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.808+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.809+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v114'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.809+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.810+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v114'229 uv220 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.810+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.811+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v114'230 uv230 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.811+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.812+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v114'231 uv231 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.812 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:38.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.812+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.813+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v114'232 uv232 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.813+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.814+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v114'233 uv233 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.814+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.815+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v114'234 uv227 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.815+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.816+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v114'235 uv235 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.816+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.817+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v114'236 uv236 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.817 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:38.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.817+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.818+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v114'237 uv237 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.818+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.819+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v114'238 uv238 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.819+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88b10 con 0x55a998d7a2e0 2024-04-26T21:44:38.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.820+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v114'239 uv239 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.820+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a998d88fc0 con 0x55a998d7a2e0 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e3e1e9640 1 -- 172.21.15.40:0/240590413 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v114'240 uv240 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1e34001470 con 0x55a998d7a2e0 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a998d75690 msgr2=0x55a998d77a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55a998d75690 0x55a998d77a80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a998c3e100 msgr2=0x55a998c3fda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55a998c3e100 0x55a998c3fda0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a998d7d510 msgr2=0x55a998d7f920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55a998d7d510 0x55a998d7f920 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a998d80e90 msgr2=0x55a998d885d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55a998d80e90 0x55a998d885d0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a998d7a2e0 msgr2=0x55a998d7c6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 reap_dead start 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55a998d7a2e0 0x55a998d7c6d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a998d81b20 msgr2=0x55a998d83f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55a998d81b20 0x55a998d83f10 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e3f1eb640 1 -- 172.21.15.40:0/240590413 reap_dead start 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e2c06b2d0 msgr2=0x7f1e2c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e2c06b2d0 0x7f1e2c06d790 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1e30005cf0 tx=0x7f1e3000c040 comp rx=0 tx=0).stop 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e28002ca0 msgr2=0x55a998cc5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e28002ca0 0x55a998cc5620 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x55a998bfef50 tx=0x7f1e380015b0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 shutdown_connections 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1e2c06b2d0 0x7f1e2c06d790 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 --2- 172.21.15.40:0/240590413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e28002ca0 0x55a998cc5620 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 >> 172.21.15.40:0/240590413 conn(0x55a998bfa330 msgr2=0x55a998ccf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 shutdown_connections 2024-04-26T21:44:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.821+0000 7f1e40ef0f40 1 -- 172.21.15.40:0/240590413 wait complete. 2024-04-26T21:44:38.826 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds_snaptable - 2024-04-26T21:44:38.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.907+0000 7fbc44d1af40 1 Processor -- start 2024-04-26T21:44:38.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.907+0000 7fbc44d1af40 1 -- start start 2024-04-26T21:44:38.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc44d1af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 0x561ca076a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc44d1af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 0x561ca0839410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc44d1af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca0793c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc44d1af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561ca083dc00 con 0x561ca083d820 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc44d1af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561ca07988d0 con 0x561ca07b1510 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc44d1af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561ca083de60 con 0x561ca0798480 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc425b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca0793c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc425b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca0793c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34382/0 (socket says 172.21.15.40:34382) 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc425b1640 1 -- 172.21.15.40:0/2729072602 learned_addr learned my addr 172.21.15.40:0/2729072602 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc41db0640 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 0x561ca076a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc415af640 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 0x561ca0839410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc425b1640 1 -- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 msgr2=0x561ca076a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc425b1640 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 0x561ca076a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc425b1640 1 -- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 msgr2=0x561ca0839410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc425b1640 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 0x561ca0839410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc425b1640 1 -- 172.21.15.40:0/2729072602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca0692cb0 con 0x561ca083d820 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.908+0000 7fbc415af640 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 0x561ca0839410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc425b1640 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca0793c20 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b5b0 tx=0x7fbc3400ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833311690eae1579 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc3bfff640 1 -- 172.21.15.40:0/2729072602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34014030 con 0x561ca083d820 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc3bfff640 1 -- 172.21.15.40:0/2729072602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc34010600 con 0x561ca083d820 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc3bfff640 1 -- 172.21.15.40:0/2729072602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc340109f0 con 0x561ca083d820 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2729072602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 msgr2=0x561ca0793c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca0793c20 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b5b0 tx=0x7fbc3400ba70 comp rx=0 tx=0).stop 2024-04-26T21:44:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2729072602 shutdown_connections 2024-04-26T21:44:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca0793c20 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 0x561ca0839410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 --2- 172.21.15.40:0/2729072602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 0x561ca076a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2729072602 >> 172.21.15.40:0/2729072602 conn(0x561ca07928a0 msgr2=0x561ca083a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2729072602 shutdown_connections 2024-04-26T21:44:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.909+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2729072602 wait complete. 2024-04-26T21:44:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.910+0000 7fbc44d1af40 1 Processor -- start 2024-04-26T21:44:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.910+0000 7fbc44d1af40 1 -- start start 2024-04-26T21:44:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.910+0000 7fbc44d1af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 0x561ca083e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.910+0000 7fbc44d1af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 0x561ca0829770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc44d1af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca082ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc44d1af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561ca08402f0 con 0x561ca083d820 2024-04-26T21:44:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc44d1af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561ca0829f10 con 0x561ca07b1510 2024-04-26T21:44:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc44d1af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561ca08405d0 con 0x561ca0798480 2024-04-26T21:44:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca082ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca082ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34390/0 (socket says 172.21.15.40:34390) 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 -- 172.21.15.40:0/2516409446 learned_addr learned my addr 172.21.15.40:0/2516409446 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc41db0640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 0x561ca083e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc415af640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 0x561ca0829770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 -- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 msgr2=0x561ca083e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561ca0798480 0x561ca083e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 -- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 msgr2=0x561ca0829770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561ca07b1510 0x561ca0829770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 -- 172.21.15.40:0/2516409446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca076a7a0 con 0x561ca083d820 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc425b1640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca082ff10 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fbc34005780 tx=0x7fbc340033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc3affd640 1 -- 172.21.15.40:0/2516409446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34021070 con 0x561ca083d820 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc3affd640 1 -- 172.21.15.40:0/2516409446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3401abd0 con 0x561ca083d820 2024-04-26T21:44:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561ca082c650 con 0x561ca083d820 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc3affd640 1 -- 172.21.15.40:0/2516409446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34018400 con 0x561ca083d820 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.911+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca082cc10 con 0x561ca083d820 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.913+0000 7fbc3affd640 1 -- 172.21.15.40:0/2516409446 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc34024070 con 0x561ca083d820 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.913+0000 7fbc3affd640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc2406b300 0x7fbc2406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.913+0000 7fbc3affd640 1 -- 172.21.15.40:0/2516409446 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7851+0+0 (secure 0 0 0) 0x7fbc3408c480 con 0x561ca083d820 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.913+0000 7fbc41db0640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc2406b300 0x7fbc2406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.913+0000 7fbc44d1af40 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561ca082d410 0x561ca08d7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.913+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x561ca08d78d0 con 0x561ca082d410 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.914+0000 7fbc415af640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561ca082d410 0x561ca08d7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.914+0000 7fbc41db0640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc2406b300 0x7fbc2406d7c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x561ca08315b0 tx=0x7fbc3c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.914+0000 7fbc415af640 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561ca082d410 0x561ca08d7390 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.915+0000 7fbc415af640 1 -- 172.21.15.40:0/2516409446 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~80] v114'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbc2c04dcf0 con 0x561ca082d410 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.915+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561ca082d410 msgr2=0x561ca08d7390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.915+0000 7fbc44d1af40 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x561ca082d410 0x561ca08d7390 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.915+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc2406b300 msgr2=0x7fbc2406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.915+0000 7fbc44d1af40 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbc2406b300 0x7fbc2406d7c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x561ca08315b0 tx=0x7fbc3c00b040 comp rx=0 tx=0).stop 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.915+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 msgr2=0x561ca082ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.915+0000 7fbc44d1af40 1 --2- 172.21.15.40:0/2516409446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561ca083d820 0x561ca082ff10 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fbc34005780 tx=0x7fbc340033a0 comp rx=0 tx=0).stop 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.915+0000 7fbc425b1640 1 -- 172.21.15.40:0/2516409446 reap_dead start 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.916+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 shutdown_connections 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.916+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 >> 172.21.15.40:0/2516409446 conn(0x561ca07928a0 msgr2=0x561ca083a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.916+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 shutdown_connections 2024-04-26T21:44:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:38.916+0000 7fbc44d1af40 1 -- 172.21.15.40:0/2516409446 wait complete. 2024-04-26T21:44:38.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:39.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.064+0000 7fa571df2640 1 Processor -- start 2024-04-26T21:44:39.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.064+0000 7fa571df2640 1 -- start start 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa571df2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0d1240 0x7fa56c0d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa571df2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa56c0b3610 0x7fa56c0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa571df2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56c0d1b60 con 0x7fa56c0b4360 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa571df2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa56c0cf210 con 0x7fa56c0b3610 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa571df2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c0ce5e0 con 0x7fa56c0d1240 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0d1240 0x7fa56c0d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa56c0b3610 0x7fa56c0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa56c0b3610 0x7fa56c0cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:32772/0 (socket says 172.21.15.40:32772) 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0d1240 0x7fa56c0d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:39990/0 (socket says 172.21.15.40:39990) 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56affd640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fa56c0b4360 0x7fa56c0cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:33084/0 (socket says 172.21.15.40:33084) 2024-04-26T21:44:39.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56bfff640 1 -- 172.21.15.40:0/1443329953 learned_addr learned my addr 172.21.15.40:0/1443329953 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56b7fe640 1 -- 172.21.15.40:0/1443329953 >> v2:172.21.15.184:3300/0 conn(0x7fa56c0b3610 msgr2=0x7fa56c0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56b7fe640 1 --2- 172.21.15.40:0/1443329953 >> v2:172.21.15.184:3300/0 conn(0x7fa56c0b3610 0x7fa56c0cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56b7fe640 1 -- 172.21.15.40:0/1443329953 >> v1:172.21.15.184:6789/0 conn(0x7fa56c0b4360 legacy=0x7fa56c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56a7fc640 1 -- 172.21.15.40:0/1443329953 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444350047 0 0) 0x7fa56c0d1b60 con 0x7fa56c0b4360 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56b7fe640 1 -- 172.21.15.40:0/1443329953 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c0ce8c0 con 0x7fa56c0d1240 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.065+0000 7fa56bfff640 1 --2- 172.21.15.40:0/1443329953 >> v2:172.21.15.184:3300/0 conn(0x7fa56c0b3610 0x7fa56c0cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.066+0000 7fa56b7fe640 1 --2- 172.21.15.40:0/1443329953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0d1240 0x7fa56c0d1620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0097a0 tx=0x7fa55c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde1631358dd01cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.066+0000 7fa56a7fc640 1 -- 172.21.15.40:0/1443329953 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c037070 con 0x7fa56c0d1240 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.066+0000 7fa56a7fc640 1 -- 172.21.15.40:0/1443329953 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c032440 con 0x7fa56c0d1240 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.066+0000 7fa56a7fc640 1 -- 172.21.15.40:0/1443329953 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c032810 con 0x7fa56c0d1240 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.067+0000 7fa571df2640 1 -- 172.21.15.40:0/1443329953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0d1240 msgr2=0x7fa56c0d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.067+0000 7fa571df2640 1 --2- 172.21.15.40:0/1443329953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0d1240 0x7fa56c0d1620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0097a0 tx=0x7fa55c02a180 comp rx=0 tx=0).stop 2024-04-26T21:44:39.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.067+0000 7fa571df2640 1 -- 172.21.15.40:0/1443329953 shutdown_connections 2024-04-26T21:44:39.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.067+0000 7fa571df2640 1 --2- 172.21.15.40:0/1443329953 >> v2:172.21.15.184:3300/0 conn(0x7fa56c0b3610 0x7fa56c0cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.067+0000 7fa571df2640 1 --2- 172.21.15.40:0/1443329953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0d1240 0x7fa56c0d1620 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.067+0000 7fa571df2640 1 -- 172.21.15.40:0/1443329953 >> 172.21.15.40:0/1443329953 conn(0x7fa56c0d2e60 msgr2=0x7fa56c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:39.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.067+0000 7fa571df2640 1 -- 172.21.15.40:0/1443329953 shutdown_connections 2024-04-26T21:44:39.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.067+0000 7fa571df2640 1 -- 172.21.15.40:0/1443329953 wait complete. 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.068+0000 7fa571df2640 1 Processor -- start 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.068+0000 7fa571df2640 1 -- start start 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa571df2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0b3610 0x7fa56c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa571df2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c0d1240 0x7fa56c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa571df2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa56c0c6940 0x7fa56c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa571df2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c0c6610 con 0x7fa56c0d1240 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa571df2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c0cf460 con 0x7fa56c0c6940 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa571df2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c0c6790 con 0x7fa56c0b3610 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c0d1240 0x7fa56c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c0d1240 0x7fa56c1586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34394/0 (socket says 172.21.15.40:34394) 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56affd640 1 -- 172.21.15.40:0/3418434597 learned_addr learned my addr 172.21.15.40:0/3418434597 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56bfff640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa56c0c6940 0x7fa56c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56b7fe640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0b3610 0x7fa56c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56affd640 1 -- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0b3610 msgr2=0x7fa56c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56affd640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0b3610 0x7fa56c0c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56affd640 1 -- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa56c0c6940 msgr2=0x7fa56c0b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56affd640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa56c0c6940 0x7fa56c0b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56affd640 1 -- 172.21.15.40:0/3418434597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c0c6e90 con 0x7fa56c0d1240 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56b7fe640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0b3610 0x7fa56c0c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.069+0000 7fa56bfff640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa56c0c6940 0x7fa56c0b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.070+0000 7fa56affd640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c0d1240 0x7fa56c1586e0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fa56000c4d0 tx=0x7fa56000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.070+0000 7fa553fff640 1 -- 172.21.15.40:0/3418434597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560018070 con 0x7fa56c0d1240 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.070+0000 7fa553fff640 1 -- 172.21.15.40:0/3418434597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa560007500 con 0x7fa56c0d1240 2024-04-26T21:44:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.070+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c0c7170 con 0x7fa56c0d1240 2024-04-26T21:44:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.070+0000 7fa553fff640 1 -- 172.21.15.40:0/3418434597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560004a20 con 0x7fa56c0d1240 2024-04-26T21:44:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.070+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c0c7730 con 0x7fa56c0d1240 2024-04-26T21:44:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.071+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c0694f0 con 0x7fa56c0d1240 2024-04-26T21:44:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.071+0000 7fa553fff640 1 -- 172.21.15.40:0/3418434597 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa56001b070 con 0x7fa56c0d1240 2024-04-26T21:44:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.072+0000 7fa553fff640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa55806c8f0 0x7fa55806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.072+0000 7fa553fff640 1 -- 172.21.15.40:0/3418434597 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7851+0+0 (secure 0 0 0) 0x7fa56008b6d0 con 0x7fa56c0d1240 2024-04-26T21:44:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.072+0000 7fa56b7fe640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa55806c8f0 0x7fa55806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.073+0000 7fa56b7fe640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa55806c8f0 0x7fa55806edb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa55c034000 tx=0x7fa55c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.076+0000 7fa553fff640 1 -- 172.21.15.40:0/3418434597 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5600592a0 con 0x7fa56c0d1240 2024-04-26T21:44:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.303+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa56c15f940 con 0x7fa56c0d1240 2024-04-26T21:44:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.538+0000 7fa553fff640 1 -- 172.21.15.40:0/3418434597 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v212) ==== 162+0+0 (secure 0 0 0) 0x7fa560058ba0 con 0x7fa56c0d1240 2024-04-26T21:44:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa55806c8f0 msgr2=0x7fa55806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa55806c8f0 0x7fa55806edb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa55c034000 tx=0x7fa55c02e000 comp rx=0 tx=0).stop 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c0d1240 msgr2=0x7fa56c1586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c0d1240 0x7fa56c1586e0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fa56000c4d0 tx=0x7fa56000c990 comp rx=0 tx=0).stop 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 shutdown_connections 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa55806c8f0 0x7fa55806edb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa56c0c6940 0x7fa56c0b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c0d1240 0x7fa56c1586e0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 --2- 172.21.15.40:0/3418434597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa56c0b3610 0x7fa56c0c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 >> 172.21.15.40:0/3418434597 conn(0x7fa56c0d2e60 msgr2=0x7fa56c0b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 shutdown_connections 2024-04-26T21:44:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.541+0000 7fa571df2640 1 -- 172.21.15.40:0/3418434597 wait complete. 2024-04-26T21:44:39.546 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:39.546+0000 7f6e79cee640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 101: Malformed input [buffer:3] 2024-04-26T21:44:39.547 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:39.547+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:39.547 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:39.547+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:39.553 INFO:tasks.ceph.mds.c.smithi040.stderr: -21> 2024-04-26T21:44:39.546+0000 7f6e79cee640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 101: Malformed input [buffer:3] 2024-04-26T21:44:39.555 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:39.577 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.658+0000 7f0b2fed9640 1 Processor -- start 2024-04-26T21:44:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.658+0000 7f0b2fed9640 1 -- start start 2024-04-26T21:44:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2fed9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b2815d1e0 0x7f0b280d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2fed9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b280d0f20 0x7f0b280d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2fed9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b280ce840 con 0x7f0b280d18c0 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2fed9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0b280cede0 con 0x7f0b280d0f20 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2fed9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b28070650 con 0x7f0b2815d1e0 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2d474640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b280d0f20 0x7f0b280d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2d474640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b280d0f20 0x7f0b280d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34426/0 (socket says 172.21.15.40:34426) 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2d474640 1 -- 172.21.15.40:0/2164029393 learned_addr learned my addr 172.21.15.40:0/2164029393 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2dc75640 1 --2- 172.21.15.40:0/2164029393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b2815d1e0 0x7f0b280d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2d474640 1 -- 172.21.15.40:0/2164029393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b2815d1e0 msgr2=0x7f0b280d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2d474640 1 --2- 172.21.15.40:0/2164029393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b2815d1e0 0x7f0b280d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.659+0000 7f0b2d474640 1 -- 172.21.15.40:0/2164029393 >> v1:172.21.15.184:6789/0 conn(0x7f0b280d18c0 legacy=0x7f0b280cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.660+0000 7f0b2d474640 1 -- 172.21.15.40:0/2164029393 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b280ceb00 con 0x7f0b280d0f20 2024-04-26T21:44:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.660+0000 7f0b2d474640 1 --2- 172.21.15.40:0/2164029393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280d1380 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009870 tx=0x7f0b1802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a39f37d1c6156c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.660+0000 7f0b2cc73640 1 -- 172.21.15.40:0/2164029393 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18037070 con 0x7f0b280d0f20 2024-04-26T21:44:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.660+0000 7f0b2cc73640 1 -- 172.21.15.40:0/2164029393 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b18032440 con 0x7f0b280d0f20 2024-04-26T21:44:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.660+0000 7f0b2cc73640 1 -- 172.21.15.40:0/2164029393 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18032810 con 0x7f0b280d0f20 2024-04-26T21:44:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.661+0000 7f0b2fed9640 1 -- 172.21.15.40:0/2164029393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 msgr2=0x7f0b280d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.661+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/2164029393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280d1380 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009870 tx=0x7f0b1802a210 comp rx=0 tx=0).stop 2024-04-26T21:44:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.661+0000 7f0b2fed9640 1 -- 172.21.15.40:0/2164029393 shutdown_connections 2024-04-26T21:44:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.661+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/2164029393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280d1380 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.661+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/2164029393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b2815d1e0 0x7f0b280d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.661+0000 7f0b2fed9640 1 -- 172.21.15.40:0/2164029393 >> 172.21.15.40:0/2164029393 conn(0x7f0b280d2da0 msgr2=0x7f0b280d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.661+0000 7f0b2fed9640 1 -- 172.21.15.40:0/2164029393 shutdown_connections 2024-04-26T21:44:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.661+0000 7f0b2fed9640 1 -- 172.21.15.40:0/2164029393 wait complete. 2024-04-26T21:44:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.662+0000 7f0b2fed9640 1 Processor -- start 2024-04-26T21:44:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.662+0000 7f0b2fed9640 1 -- start start 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2fed9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2fed9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b280d18c0 0x7f0b281fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2fed9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2815d1e0 0x7f0b280c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2fed9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0b28071af0 con 0x7f0b280d0f20 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2fed9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b280ce4c0 con 0x7f0b2815d1e0 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2fed9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b28071c70 con 0x7f0b280d18c0 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2dc75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2dc75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34442/0 (socket says 172.21.15.40:34442) 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2dc75640 1 -- 172.21.15.40:0/437346327 learned_addr learned my addr 172.21.15.40:0/437346327 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2d474640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b280d18c0 0x7f0b281fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2dc75640 1 -- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b280d18c0 msgr2=0x7f0b281fcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2dc75640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b280d18c0 0x7f0b281fcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2dc75640 1 -- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2815d1e0 msgr2=0x7f0b280c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2dc75640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2815d1e0 0x7f0b280c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.663+0000 7f0b2dc75640 1 -- 172.21.15.40:0/437346327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b280cc050 con 0x7f0b280d0f20 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.664+0000 7f0b2dc75640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280b8130 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400c7f0 tx=0x7f0b2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.664+0000 7f0b1e7fc640 1 -- 172.21.15.40:0/437346327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24015070 con 0x7f0b280d0f20 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.664+0000 7f0b1e7fc640 1 -- 172.21.15.40:0/437346327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b24004240 con 0x7f0b280d0f20 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.664+0000 7f0b1e7fc640 1 -- 172.21.15.40:0/437346327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24007620 con 0x7f0b280d0f20 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.664+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b280cc330 con 0x7f0b280d0f20 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.664+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b280cc8f0 con 0x7f0b280d0f20 2024-04-26T21:44:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.665+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b280694f0 con 0x7f0b280d0f20 2024-04-26T21:44:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.665+0000 7f0b1e7fc640 1 -- 172.21.15.40:0/437346327 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b240079d0 con 0x7f0b280d0f20 2024-04-26T21:44:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.666+0000 7f0b1e7fc640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b0c06c920 0x7f0b0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.666+0000 7f0b1e7fc640 1 -- 172.21.15.40:0/437346327 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7937+0+0 (secure 0 0 0) 0x7f0b2408c4d0 con 0x7f0b280d0f20 2024-04-26T21:44:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.666+0000 7f0b2d474640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b0c06c920 0x7f0b0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.667+0000 7f0b2d474640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b0c06c920 0x7f0b0c06ede0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.670+0000 7f0b1e7fc640 1 -- 172.21.15.40:0/437346327 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b2405a040 con 0x7f0b280d0f20 2024-04-26T21:44:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.893+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b2806ce60 con 0x7f0b280d0f20 2024-04-26T21:44:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.894+0000 7f0b1e7fc640 1 -- 172.21.15.40:0/437346327 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) ==== 95+0+4355 (secure 0 0 0) 0x7f0b24059940 con 0x7f0b280d0f20 2024-04-26T21:44:39.894 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:39.894 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":214,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:39.547725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 214 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b0c06c920 msgr2=0x7f0b0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b0c06c920 0x7f0b0c06ede0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).stop 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 msgr2=0x7f0b280b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280b8130 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400c7f0 tx=0x7f0b2400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 shutdown_connections 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0b0c06c920 0x7f0b0c06ede0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2815d1e0 0x7f0b280c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b280d18c0 0x7f0b281fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 --2- 172.21.15.40:0/437346327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b280d0f20 0x7f0b280b8130 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 >> 172.21.15.40:0/437346327 conn(0x7f0b280d2da0 msgr2=0x7f0b280b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 shutdown_connections 2024-04-26T21:44:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:39.896+0000 7f0b2fed9640 1 -- 172.21.15.40:0/437346327 wait complete. 2024-04-26T21:44:39.906 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:39.906 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:40.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.047+0000 7f63171b7640 1 Processor -- start 2024-04-26T21:44:40.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f63171b7640 1 -- start start 2024-04-26T21:44:40.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f63171b7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f631015f3a0 0x7f63100d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f63171b7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63100d1770 0x7f63100d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f63171b7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63100d25a0 con 0x7f63100d2130 2024-04-26T21:44:40.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f63171b7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f63100d0e20 con 0x7f63100d1770 2024-04-26T21:44:40.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f63171b7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63100b1d70 con 0x7f631015f3a0 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f630ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63100d1770 0x7f63100d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f6315754640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63100d2130 0x7f63100d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39538/0 (socket says 172.21.15.40:39538) 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.048+0000 7f6315754640 1 -- 172.21.15.40:0/2619295558 learned_addr learned my addr 172.21.15.40:0/2619295558 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.049+0000 7f6314f53640 1 --2- 172.21.15.40:0/2619295558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f631015f3a0 0x7f63100d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.049+0000 7f630ffff640 1 -- 172.21.15.40:0/2619295558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f631015f3a0 msgr2=0x7f63100d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.049+0000 7f630ffff640 1 --2- 172.21.15.40:0/2619295558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f631015f3a0 0x7f63100d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.049+0000 7f630ffff640 1 -- 172.21.15.40:0/2619295558 >> v1:172.21.15.40:6789/0 conn(0x7f63100d2130 legacy=0x7f63100d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.049+0000 7f630ffff640 1 -- 172.21.15.40:0/2619295558 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63100b2050 con 0x7f63100d1770 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.049+0000 7f630ffff640 1 --2- 172.21.15.40:0/2619295558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d1770 0x7f63100d1bf0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0097a0 tx=0x7f62fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a8e8a75de4c2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f630f7fe640 1 -- 172.21.15.40:0/2619295558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc037070 con 0x7f63100d1770 2024-04-26T21:44:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f630f7fe640 1 -- 172.21.15.40:0/2619295558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc032440 con 0x7f63100d1770 2024-04-26T21:44:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f630f7fe640 1 -- 172.21.15.40:0/2619295558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc032810 con 0x7f63100d1770 2024-04-26T21:44:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f63171b7640 1 -- 172.21.15.40:0/2619295558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d1770 msgr2=0x7f63100d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f63171b7640 1 --2- 172.21.15.40:0/2619295558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d1770 0x7f63100d1bf0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0097a0 tx=0x7f62fc02a180 comp rx=0 tx=0).stop 2024-04-26T21:44:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f63171b7640 1 -- 172.21.15.40:0/2619295558 shutdown_connections 2024-04-26T21:44:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f63171b7640 1 --2- 172.21.15.40:0/2619295558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d1770 0x7f63100d1bf0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f63171b7640 1 --2- 172.21.15.40:0/2619295558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f631015f3a0 0x7f63100d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f63171b7640 1 -- 172.21.15.40:0/2619295558 >> 172.21.15.40:0/2619295558 conn(0x7f63100d4f90 msgr2=0x7f63100d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f63171b7640 1 -- 172.21.15.40:0/2619295558 shutdown_connections 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.050+0000 7f63171b7640 1 -- 172.21.15.40:0/2619295558 wait complete. 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.051+0000 7f63171b7640 1 Processor -- start 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.051+0000 7f63171b7640 1 -- start start 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f63171b7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63100d1770 0x7f63100cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f63171b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d2130 0x7f631014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f63171b7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f631015f3a0 0x7f631014f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f63171b7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f631015f900 con 0x7f63100d2130 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f63171b7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f631015f780 con 0x7f631015f3a0 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f63171b7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63100b3860 con 0x7f63100d1770 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f630ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d2130 0x7f631014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f630ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d2130 0x7f631014eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34458/0 (socket says 172.21.15.40:34458) 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.052+0000 7f630ffff640 1 -- 172.21.15.40:0/1487195805 learned_addr learned my addr 172.21.15.40:0/1487195805 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f6314f53640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63100d1770 0x7f63100cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f6315754640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f631015f3a0 0x7f631014f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630ffff640 1 -- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63100d1770 msgr2=0x7f63100cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630ffff640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63100d1770 0x7f63100cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630ffff640 1 -- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f631015f3a0 msgr2=0x7f631014f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630ffff640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f631015f3a0 0x7f631014f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630ffff640 1 -- 172.21.15.40:0/1487195805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63100b97b0 con 0x7f63100d2130 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f6315754640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f631015f3a0 0x7f631014f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630ffff640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d2130 0x7f631014eae0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f62fc035040 tx=0x7f62fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630d7fa640 1 -- 172.21.15.40:0/1487195805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc037040 con 0x7f63100d2130 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630d7fa640 1 -- 172.21.15.40:0/1487195805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc030770 con 0x7f63100d2130 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63100b9a90 con 0x7f63100d2130 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f630d7fa640 1 -- 172.21.15.40:0/1487195805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc030b40 con 0x7f63100d2130 2024-04-26T21:44:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.053+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63100ba570 con 0x7f63100d2130 2024-04-26T21:44:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.054+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63100694f0 con 0x7f63100d2130 2024-04-26T21:44:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.054+0000 7f630d7fa640 1 -- 172.21.15.40:0/1487195805 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62fc044020 con 0x7f63100d2130 2024-04-26T21:44:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.055+0000 7f630d7fa640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f62f406c8f0 0x7f62f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.055+0000 7f630d7fa640 1 -- 172.21.15.40:0/1487195805 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7937+0+0 (secure 0 0 0) 0x7f62fc0ac910 con 0x7f63100d2130 2024-04-26T21:44:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.055+0000 7f6314f53640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f62f406c8f0 0x7f62f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.056+0000 7f6314f53640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f62f406c8f0 0x7f62f406edb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6304006fd0 tx=0x7f6304008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.059+0000 7f630d7fa640 1 -- 172.21.15.40:0/1487195805 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62fc032e20 con 0x7f63100d2130 2024-04-26T21:44:40.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.284+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63100b6860 con 0x7f63100d2130 2024-04-26T21:44:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.284+0000 7f630d7fa640 1 -- 172.21.15.40:0/1487195805 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) ==== 95+0+4355 (secure 0 0 0) 0x7f62fc07a480 con 0x7f63100d2130 2024-04-26T21:44:40.284 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:40.284 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":214,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:39.547725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:40.284 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 214 2024-04-26T21:44:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f62f406c8f0 msgr2=0x7f62f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f62f406c8f0 0x7f62f406edb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6304006fd0 tx=0x7f6304008040 comp rx=0 tx=0).stop 2024-04-26T21:44:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d2130 msgr2=0x7f631014eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d2130 0x7f631014eae0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f62fc035040 tx=0x7f62fc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 shutdown_connections 2024-04-26T21:44:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f62f406c8f0 0x7f62f406edb0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f631015f3a0 0x7f631014f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63100d2130 0x7f631014eae0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 --2- 172.21.15.40:0/1487195805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63100d1770 0x7f63100cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 >> 172.21.15.40:0/1487195805 conn(0x7f63100d4f90 msgr2=0x7f63100d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 shutdown_connections 2024-04-26T21:44:40.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.286+0000 7f63171b7640 1 -- 172.21.15.40:0/1487195805 wait complete. 2024-04-26T21:44:40.297 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:40.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:40.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.397+0000 7f31e21f0640 1 Processor -- start 2024-04-26T21:44:40.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.397+0000 7f31e21f0640 1 -- start start 2024-04-26T21:44:40.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31e21f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31dc0b0380 0x7f31dc0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31e21f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f31dc0b10b0 0x7f31dc06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31e21f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31dc0cfd80 con 0x7f31dc06c780 2024-04-26T21:44:40.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31e21f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f31dc06cbd0 con 0x7f31dc0b0380 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31e21f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f31dc0af8e0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31db7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31dc0b0380 0x7f31dc0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31dbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f31dc06c780 0x7f31dc072f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39544/0 (socket says 172.21.15.40:39544) 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31dbfff640 1 -- 172.21.15.40:0/828836822 learned_addr learned my addr 172.21.15.40:0/828836822 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31daffd640 1 --2- 172.21.15.40:0/828836822 >> v2:172.21.15.184:3300/0 conn(0x7f31dc0b10b0 0x7f31dc06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31db7fe640 1 -- 172.21.15.40:0/828836822 >> v2:172.21.15.184:3300/0 conn(0x7f31dc0b10b0 msgr2=0x7f31dc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31db7fe640 1 --2- 172.21.15.40:0/828836822 >> v2:172.21.15.184:3300/0 conn(0x7f31dc0b10b0 0x7f31dc06c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31db7fe640 1 -- 172.21.15.40:0/828836822 >> v1:172.21.15.40:6789/0 conn(0x7f31dc06c780 legacy=0x7f31dc072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31db7fe640 1 -- 172.21.15.40:0/828836822 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc0d0020 con 0x7f31dc0b0380 2024-04-26T21:44:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.398+0000 7f31da7fc640 1 -- 172.21.15.40:0/828836822 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2233011271 0 0) 0x7f31dc0cfd80 con 0x7f31dc06c780 2024-04-26T21:44:40.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.399+0000 7f31db7fe640 1 --2- 172.21.15.40:0/828836822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b0380 0x7f31dc0ccd50 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0b48c0 tx=0x7f31cc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89dcfbc6eeb9f57b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.399+0000 7f31da7fc640 1 -- 172.21.15.40:0/828836822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc037070 con 0x7f31dc0b0380 2024-04-26T21:44:40.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.399+0000 7f31da7fc640 1 -- 172.21.15.40:0/828836822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31cc032440 con 0x7f31dc0b0380 2024-04-26T21:44:40.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.399+0000 7f31da7fc640 1 -- 172.21.15.40:0/828836822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc02ae60 con 0x7f31dc0b0380 2024-04-26T21:44:40.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.400+0000 7f31e21f0640 1 -- 172.21.15.40:0/828836822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b0380 msgr2=0x7f31dc0ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.400+0000 7f31e21f0640 1 --2- 172.21.15.40:0/828836822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b0380 0x7f31dc0ccd50 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0b48c0 tx=0x7f31cc02a110 comp rx=0 tx=0).stop 2024-04-26T21:44:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.400+0000 7f31e21f0640 1 -- 172.21.15.40:0/828836822 shutdown_connections 2024-04-26T21:44:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.400+0000 7f31e21f0640 1 --2- 172.21.15.40:0/828836822 >> v2:172.21.15.184:3300/0 conn(0x7f31dc0b10b0 0x7f31dc06c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.400+0000 7f31e21f0640 1 --2- 172.21.15.40:0/828836822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b0380 0x7f31dc0ccd50 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.400+0000 7f31e21f0640 1 -- 172.21.15.40:0/828836822 >> 172.21.15.40:0/828836822 conn(0x7f31dc0d1cf0 msgr2=0x7f31dc0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.400+0000 7f31e21f0640 1 -- 172.21.15.40:0/828836822 shutdown_connections 2024-04-26T21:44:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.400+0000 7f31e21f0640 1 -- 172.21.15.40:0/828836822 wait complete. 2024-04-26T21:44:40.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.401+0000 7f31e21f0640 1 Processor -- start 2024-04-26T21:44:40.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.401+0000 7f31e21f0640 1 -- start start 2024-04-26T21:44:40.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31e21f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31dc06c780 0x7f31dc1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31e21f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31dc0b0380 0x7f31dc0b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31e21f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b10b0 0x7f31dc0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31e21f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc0d1980 con 0x7f31dc0b10b0 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31e21f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc0d05e0 con 0x7f31dc06c780 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31e21f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f31dc0b0760 con 0x7f31dc0b0380 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b10b0 0x7f31dc0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b10b0 0x7f31dc0c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34474/0 (socket says 172.21.15.40:34474) 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31dbfff640 1 -- 172.21.15.40:0/2483330329 learned_addr learned my addr 172.21.15.40:0/2483330329 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31daffd640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31dc0b0380 0x7f31dc0b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31db7fe640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31dc06c780 0x7f31dc1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31dbfff640 1 -- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31dc0b0380 msgr2=0x7f31dc0b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31dbfff640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31dc0b0380 0x7f31dc0b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31dbfff640 1 -- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31dc06c780 msgr2=0x7f31dc1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31dbfff640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31dc06c780 0x7f31dc1616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31dbfff640 1 -- 172.21.15.40:0/2483330329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc1608f0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31daffd640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31dc0b0380 0x7f31dc0b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.402+0000 7f31db7fe640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31dc06c780 0x7f31dc1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.403+0000 7f31dbfff640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b10b0 0x7f31dc0c7ff0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f31d400caf0 tx=0x7f31d40021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.403+0000 7f31c3fff640 1 -- 172.21.15.40:0/2483330329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d400ac00 con 0x7f31dc0b10b0 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.403+0000 7f31c3fff640 1 -- 172.21.15.40:0/2483330329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d400ada0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.403+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc160bd0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.403+0000 7f31c3fff640 1 -- 172.21.15.40:0/2483330329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4012640 con 0x7f31dc0b10b0 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.403+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc0b4f40 con 0x7f31dc0b10b0 2024-04-26T21:44:40.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.404+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc0b7bf0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.404+0000 7f31c3fff640 1 -- 172.21.15.40:0/2483330329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31d40127e0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.405+0000 7f31c3fff640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f31c806c8f0 0x7f31c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.405+0000 7f31c3fff640 1 -- 172.21.15.40:0/2483330329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7937+0+0 (secure 0 0 0) 0x7f31d408c9a0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.405+0000 7f31db7fe640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f31c806c8f0 0x7f31c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.405+0000 7f31db7fe640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f31c806c8f0 0x7f31c806edb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f31cc034000 tx=0x7f31cc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.409+0000 7f31c3fff640 1 -- 172.21.15.40:0/2483330329 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31dc0b7bf0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.638+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31dc0694f0 con 0x7f31dc0b10b0 2024-04-26T21:44:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.639+0000 7f31c3fff640 1 -- 172.21.15.40:0/2483330329 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) ==== 95+0+5181 (secure 0 0 0) 0x7f31d405a510 con 0x7f31dc0b10b0 2024-04-26T21:44:40.639 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:40.639 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":215,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1149681708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1149681708},{"type":"v1","addr":"172.21.15.40:6833","nonce":1149681708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":215}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:39.547725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 215 2024-04-26T21:44:40.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f31c806c8f0 msgr2=0x7f31c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f31c806c8f0 0x7f31c806edb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f31cc034000 tx=0x7f31cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b10b0 msgr2=0x7f31dc0c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b10b0 0x7f31dc0c7ff0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f31d400caf0 tx=0x7f31d40021c0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 shutdown_connections 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f31c806c8f0 0x7f31c806edb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0b10b0 0x7f31dc0c7ff0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31dc0b0380 0x7f31dc0b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 --2- 172.21.15.40:0/2483330329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31dc06c780 0x7f31dc1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 >> 172.21.15.40:0/2483330329 conn(0x7f31dc0d1cf0 msgr2=0x7f31dc0b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 shutdown_connections 2024-04-26T21:44:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.641+0000 7f31e21f0640 1 -- 172.21.15.40:0/2483330329 wait complete. 2024-04-26T21:44:40.652 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds_snaptable' led to DAMAGED state 2024-04-26T21:44:40.652 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds0_inotable' 2024-04-26T21:44:40.652 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:40.652 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:40.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:40.668 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:40.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:40.668 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:40.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:40.724 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:40.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:40.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.866+0000 7ff33a007640 1 Processor -- start 2024-04-26T21:44:40.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.866+0000 7ff33a007640 1 -- start start 2024-04-26T21:44:40.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff33a007640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff33415f3a0 0x7ff3340d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff33a007640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff3340d1770 0x7ff3340d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff33a007640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d2160 0x7ff3340d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff33a007640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff3340d25d0 con 0x7ff33415f3a0 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff33a007640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff3340d0e20 con 0x7ff3340d1770 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff33a007640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3340b1d70 con 0x7ff3340d2160 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff3337fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff33415f3a0 0x7ff3340d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff3337fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff33415f3a0 0x7ff3340d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34484/0 (socket says 172.21.15.40:34484) 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff3337fe640 1 -- 172.21.15.40:0/1195581772 learned_addr learned my addr 172.21.15.40:0/1195581772 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.867+0000 7ff333fff640 1 --2- 172.21.15.40:0/1195581772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d2160 0x7ff3340d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3337fe640 1 -- 172.21.15.40:0/1195581772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d2160 msgr2=0x7ff3340d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff332ffd640 1 --2- 172.21.15.40:0/1195581772 >> v2:172.21.15.184:3300/0 conn(0x7ff3340d1770 0x7ff3340d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3337fe640 1 --2- 172.21.15.40:0/1195581772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d2160 0x7ff3340d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3337fe640 1 -- 172.21.15.40:0/1195581772 >> v2:172.21.15.184:3300/0 conn(0x7ff3340d1770 msgr2=0x7ff3340d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3337fe640 1 --2- 172.21.15.40:0/1195581772 >> v2:172.21.15.184:3300/0 conn(0x7ff3340d1770 0x7ff3340d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3337fe640 1 -- 172.21.15.40:0/1195581772 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3340b2050 con 0x7ff33415f3a0 2024-04-26T21:44:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3337fe640 1 --2- 172.21.15.40:0/1195581772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340d06f0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ff3340b7b80 tx=0x7ff32402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae620b31941450df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3327fc640 1 -- 172.21.15.40:0/1195581772 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324037070 con 0x7ff33415f3a0 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3327fc640 1 -- 172.21.15.40:0/1195581772 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff324032440 con 0x7ff33415f3a0 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.868+0000 7ff3327fc640 1 -- 172.21.15.40:0/1195581772 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324032810 con 0x7ff33415f3a0 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 -- 172.21.15.40:0/1195581772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 msgr2=0x7ff3340d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 --2- 172.21.15.40:0/1195581772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340d06f0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ff3340b7b80 tx=0x7ff32402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 -- 172.21.15.40:0/1195581772 shutdown_connections 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 --2- 172.21.15.40:0/1195581772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d2160 0x7ff3340d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 --2- 172.21.15.40:0/1195581772 >> v2:172.21.15.184:3300/0 conn(0x7ff3340d1770 0x7ff3340d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 --2- 172.21.15.40:0/1195581772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340d06f0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 -- 172.21.15.40:0/1195581772 >> 172.21.15.40:0/1195581772 conn(0x7ff3340d4f90 msgr2=0x7ff3340d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 -- 172.21.15.40:0/1195581772 shutdown_connections 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.869+0000 7ff33a007640 1 -- 172.21.15.40:0/1195581772 wait complete. 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.870+0000 7ff33a007640 1 Processor -- start 2024-04-26T21:44:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.870+0000 7ff33a007640 1 -- start start 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff33a007640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d1770 0x7ff33415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff33a007640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3340d2160 0x7ff3340ba510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff33a007640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff33a007640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff33415f900 con 0x7ff33415f3a0 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff33a007640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff33415f780 con 0x7ff3340d2160 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff33a007640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3340d3390 con 0x7ff3340d1770 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff333fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff333fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34500/0 (socket says 172.21.15.40:34500) 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff333fff640 1 -- 172.21.15.40:0/3288566513 learned_addr learned my addr 172.21.15.40:0/3288566513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff332ffd640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3340d2160 0x7ff3340ba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff3337fe640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d1770 0x7ff33415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.871+0000 7ff333fff640 1 -- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d1770 msgr2=0x7ff33415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff333fff640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d1770 0x7ff33415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff333fff640 1 -- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3340d2160 msgr2=0x7ff3340ba510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff333fff640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3340d2160 0x7ff3340ba510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff333fff640 1 -- 172.21.15.40:0/3288566513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3340b7cb0 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff333fff640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340cd7d0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00cd70 tx=0x7ff32c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff31bfff640 1 -- 172.21.15.40:0/3288566513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c003b90 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff31bfff640 1 -- 172.21.15.40:0/3288566513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c003d30 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3340b7f90 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff31bfff640 1 -- 172.21.15.40:0/3288566513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c004240 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.872+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3340b8550 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.873+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3340694f0 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.873+0000 7ff31bfff640 1 -- 172.21.15.40:0/3288566513 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff32c0043e0 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.874+0000 7ff31bfff640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff32006c8f0 0x7ff32006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.874+0000 7ff31bfff640 1 -- 172.21.15.40:0/3288566513 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7937+0+0 (secure 0 0 0) 0x7ff32c08ceb0 con 0x7ff33415f3a0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.874+0000 7ff3337fe640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff32006c8f0 0x7ff32006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.874+0000 7ff3337fe640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff32006c8f0 0x7ff32006edb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff324034000 tx=0x7ff32402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:40.878+0000 7ff31bfff640 1 -- 172.21.15.40:0/3288566513 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff32c01d340 con 0x7ff33415f3a0 2024-04-26T21:44:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.101+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff3340b6860 con 0x7ff33415f3a0 2024-04-26T21:44:41.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.554+0000 7ff31bfff640 1 -- 172.21.15.40:0/3288566513 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v216) ==== 161+0+0 (secure 0 0 0) 0x7ff32c05aab0 con 0x7ff33415f3a0 2024-04-26T21:44:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff32006c8f0 msgr2=0x7ff32006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff32006c8f0 0x7ff32006edb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff324034000 tx=0x7ff32402e000 comp rx=0 tx=0).stop 2024-04-26T21:44:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 msgr2=0x7ff3340cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340cd7d0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00cd70 tx=0x7ff32c00a660 comp rx=0 tx=0).stop 2024-04-26T21:44:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 shutdown_connections 2024-04-26T21:44:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff32006c8f0 0x7ff32006edb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33415f3a0 0x7ff3340cd7d0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3340d2160 0x7ff3340ba510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 --2- 172.21.15.40:0/3288566513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3340d1770 0x7ff33415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 >> 172.21.15.40:0/3288566513 conn(0x7ff3340d4f90 msgr2=0x7ff3340d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 shutdown_connections 2024-04-26T21:44:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.556+0000 7ff33a007640 1 -- 172.21.15.40:0/3288566513 wait complete. 2024-04-26T21:44:41.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.673+0000 7f9e79b1b640 1 Processor -- start 2024-04-26T21:44:41.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.673+0000 7f9e79b1b640 1 -- start start 2024-04-26T21:44:41.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.673+0000 7f9e79b1b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9e740d3bc0 0x7f9e740d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e79b1b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e740b7e50 con 0x7f9e740b5f20 2024-04-26T21:44:41.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e79b1b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e740d1800 con 0x7f9e74162880 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e79b1b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9e740d0580 con 0x7f9e740d3bc0 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e73fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9e740b5f20 0x7f9e740b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39558/0 (socket says 172.21.15.40:39558) 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e73fff640 1 -- 172.21.15.40:0/812790059 learned_addr learned my addr 172.21.15.40:0/812790059 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e72ffd640 1 --2- 172.21.15.40:0/812790059 >> v2:172.21.15.184:3300/0 conn(0x7f9e740d3bc0 0x7f9e740d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3860397973 0 0) 0x7f9e740b7e50 con 0x7f9e740b5f20 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e5c003520 con 0x7f9e740b5f20 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2441091938 0 0) 0x7f9e5c003520 con 0x7f9e740b5f20 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 >> v2:172.21.15.184:3300/0 conn(0x7f9e740d3bc0 msgr2=0x7f9e740d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 --2- 172.21.15.40:0/812790059 >> v2:172.21.15.184:3300/0 conn(0x7f9e740d3bc0 0x7f9e740d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 >> v1:172.21.15.184:6789/0 conn(0x7f9e74162880 legacy=0x7f9e740d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e740d07d0 con 0x7f9e740b5f20 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9e6c0019f0 con 0x7f9e740b5f20 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 >> v1:172.21.15.40:6789/0 conn(0x7f9e740b5f20 legacy=0x7f9e740b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e5c004800 0x7f9e5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e5c0051c0 0x7f9e5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e740d0580 con 0x7f9e5c0039b0 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c00c060 con 0x7f9e5c0051c0 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5c00c260 con 0x7f9e5c004800 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.674+0000 7f9e737fe640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e72ffd640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e5c004800 0x7f9e5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e73fff640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e5c0051c0 0x7f9e5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e737fe640 1 -- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e5c004800 msgr2=0x7f9e5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e737fe640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e5c004800 0x7f9e5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e737fe640 1 -- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e5c0051c0 msgr2=0x7f9e5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e737fe640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e5c0051c0 0x7f9e5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e737fe640 1 -- 172.21.15.40:0/812790059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c003520 con 0x7f9e5c0039b0 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e73fff640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e5c0051c0 0x7f9e5c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e73fff640 1 -- 172.21.15.40:0/812790059 reap_dead start 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e737fe640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e5c003dd0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f9e64001750 tx=0x7f9e5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b367a821e74158 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.675+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64001370 con 0x7f9e5c0039b0 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e640052d0 con 0x7f9e5c0039b0 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e727fc640 1 -- 172.21.15.40:0/812790059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6402abc0 con 0x7f9e5c0039b0 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e79b1b640 1 -- 172.21.15.40:0/812790059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 msgr2=0x7f9e5c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e79b1b640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e5c003dd0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f9e64001750 tx=0x7f9e5c005830 comp rx=0 tx=0).stop 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e79b1b640 1 -- 172.21.15.40:0/812790059 shutdown_connections 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e79b1b640 1 --2- 172.21.15.40:0/812790059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e5c003dd0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e79b1b640 1 -- 172.21.15.40:0/812790059 >> 172.21.15.40:0/812790059 conn(0x7f9e740d4f60 msgr2=0x7f9e740d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:41.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e79b1b640 1 -- 172.21.15.40:0/812790059 shutdown_connections 2024-04-26T21:44:41.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.676+0000 7f9e79b1b640 1 -- 172.21.15.40:0/812790059 wait complete. 2024-04-26T21:44:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.677+0000 7f9e79b1b640 1 Processor -- start 2024-04-26T21:44:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e79b1b640 1 -- start start 2024-04-26T21:44:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e79b1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e7415c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e79b1b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e740b4e60 0x7f9e740b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e79b1b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e740b2a70 0x7f9e740b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e79b1b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74162c60 con 0x7f9e5c0039b0 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e79b1b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e740d0d90 con 0x7f9e740b4e60 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e79b1b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e74162de0 con 0x7f9e740b2a70 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e737fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e7415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e737fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e7415c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34514/0 (socket says 172.21.15.40:34514) 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e737fe640 1 -- 172.21.15.40:0/1925013844 learned_addr learned my addr 172.21.15.40:0/1925013844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e73fff640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e740b2a70 0x7f9e740b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.678+0000 7f9e72ffd640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e740b4e60 0x7f9e740b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e737fe640 1 -- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e740b2a70 msgr2=0x7f9e740b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e737fe640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e740b2a70 0x7f9e740b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e737fe640 1 -- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e740b4e60 msgr2=0x7f9e740b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e737fe640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e740b4e60 0x7f9e740b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e737fe640 1 -- 172.21.15.40:0/1925013844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e740b3760 con 0x7f9e5c0039b0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e73fff640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e740b2a70 0x7f9e740b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e72ffd640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e740b4e60 0x7f9e740b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e737fe640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e7415c890 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000c00 tx=0x7f9e64001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e5bfff640 1 -- 172.21.15.40:0/1925013844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64040030 con 0x7f9e5c0039b0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e5bfff640 1 -- 172.21.15.40:0/1925013844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6402be50 con 0x7f9e5c0039b0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e740cc140 con 0x7f9e5c0039b0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e5bfff640 1 -- 172.21.15.40:0/1925013844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64001370 con 0x7f9e5c0039b0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.679+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e740cc700 con 0x7f9e5c0039b0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.680+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e740694f0 con 0x7f9e5c0039b0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.681+0000 7f9e5bfff640 1 -- 172.21.15.40:0/1925013844 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e64043070 con 0x7f9e5c0039b0 2024-04-26T21:44:41.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.681+0000 7f9e5bfff640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e6006c920 0x7f9e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.681+0000 7f9e5bfff640 1 -- 172.21.15.40:0/1925013844 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7937+0+0 (secure 0 0 0) 0x7f9e640abc40 con 0x7f9e5c0039b0 2024-04-26T21:44:41.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.681+0000 7f9e72ffd640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e6006c920 0x7f9e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.682+0000 7f9e72ffd640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e6006c920 0x7f9e6006ede0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9e740d5840 tx=0x7f9e68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:41.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.685+0000 7f9e5bfff640 1 -- 172.21.15.40:0/1925013844 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e6404b020 con 0x7f9e5c0039b0 2024-04-26T21:44:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:41.943+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f9e740cb530 con 0x7f9e5c0039b0 2024-04-26T21:44:42.382 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:44:42.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:44:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.562+0000 7f9e5bfff640 1 -- 172.21.15.40:0/1925013844 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v217) ==== 101+0+0 (secure 0 0 0) 0x7f9e740d1800 con 0x7f9e5c0039b0 2024-04-26T21:44:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e6006c920 msgr2=0x7f9e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e6006c920 0x7f9e6006ede0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9e740d5840 tx=0x7f9e68008040 comp rx=0 tx=0).stop 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 msgr2=0x7f9e7415c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e7415c890 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000c00 tx=0x7f9e64001120 comp rx=0 tx=0).stop 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 shutdown_connections 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e6006c920 0x7f9e6006ede0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e740b2a70 0x7f9e740b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e740b4e60 0x7f9e740b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 --2- 172.21.15.40:0/1925013844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e5c0039b0 0x7f9e7415c890 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.565+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 >> 172.21.15.40:0/1925013844 conn(0x7f9e740d4f60 msgr2=0x7f9e741603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.566+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 shutdown_connections 2024-04-26T21:44:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.566+0000 7f9e79b1b640 1 -- 172.21.15.40:0/1925013844 wait complete. 2024-04-26T21:44:42.578 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:42.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.623+0000 7fe8f9966f40 1 Processor -- start 2024-04-26T21:44:42.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.623+0000 7fe8f9966f40 1 -- start start 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f9966f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3e67700 0x5623e3eebae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f9966f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e68770 0x5623e3ef6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f9966f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3ef46b0 0x5623e3e22670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f9966f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5623e3e68be0 con 0x5623e3ef46b0 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f9966f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5623e3dc6e40 con 0x5623e3e68770 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f9966f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5623e3e50a20 con 0x5623e3e67700 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f7c63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3ef46b0 0x5623e3e22670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f7c63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3ef46b0 0x5623e3e22670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34532/0 (socket says 172.21.15.40:34532) 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f7c63640 1 -- 172.21.15.40:0/125421746 learned_addr learned my addr 172.21.15.40:0/125421746 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f7462640 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3e67700 0x5623e3eebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e68770 0x5623e3ef6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.624+0000 7fe8f7c63640 1 -- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3e67700 msgr2=0x5623e3eebae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f7c63640 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3e67700 0x5623e3eebae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f7c63640 1 -- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e68770 msgr2=0x5623e3ef6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f7c63640 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e68770 0x5623e3ef6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f7c63640 1 -- 172.21.15.40:0/125421746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e3ef7c70 con 0x5623e3ef46b0 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f7462640 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3e67700 0x5623e3eebae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e68770 0x5623e3ef6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f7c63640 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3ef46b0 0x5623e3e22670 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8001710 tx=0x7fe8e8001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9376f1448070d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f5c5f640 1 -- 172.21.15.40:0/125421746 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80062e0 con 0x5623e3ef46b0 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f5c5f640 1 -- 172.21.15.40:0/125421746 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8003650 con 0x5623e3ef46b0 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f5c5f640 1 -- 172.21.15.40:0/125421746 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8003a20 con 0x5623e3ef46b0 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 -- 172.21.15.40:0/125421746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3ef46b0 msgr2=0x5623e3e22670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3ef46b0 0x5623e3e22670 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8001710 tx=0x7fe8e8001bd0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 -- 172.21.15.40:0/125421746 shutdown_connections 2024-04-26T21:44:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3ef46b0 0x5623e3e22670 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e68770 0x5623e3ef6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/125421746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3e67700 0x5623e3eebae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 -- 172.21.15.40:0/125421746 >> 172.21.15.40:0/125421746 conn(0x5623e3e4a9b0 msgr2=0x5623e3e1cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 -- 172.21.15.40:0/125421746 shutdown_connections 2024-04-26T21:44:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.625+0000 7fe8f9966f40 1 -- 172.21.15.40:0/125421746 wait complete. 2024-04-26T21:44:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.626+0000 7fe8f9966f40 1 Processor -- start 2024-04-26T21:44:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.626+0000 7fe8f9966f40 1 -- start start 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f9966f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e67700 0x5623e3eeafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f9966f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3e68770 0x5623e3ef1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f9966f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3ef46b0 0x5623e3e22670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f9966f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5623e3e4fb20 con 0x5623e3e68770 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f9966f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5623e3eef9f0 con 0x5623e3e67700 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f9966f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5623e3e1d730 con 0x5623e3ef46b0 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3e68770 0x5623e3ef1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3e68770 0x5623e3ef1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34536/0 (socket says 172.21.15.40:34536) 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 learned_addr learned my addr 172.21.15.40:0/4147279188 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3ef46b0 msgr2=0x5623e3e22670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5623e3ef46b0 0x5623e3e22670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f7462640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e67700 0x5623e3eeafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e67700 msgr2=0x5623e3eeafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623e3e67700 0x5623e3eeafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e3ef2930 con 0x5623e3e68770 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3e68770 0x5623e3ef1f00 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec008400 tx=0x7fe8ec0084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.627+0000 7fe8f4c5d640 1 -- 172.21.15.40:0/4147279188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0143f0 con 0x5623e3e68770 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.628+0000 7fe8f4c5d640 1 -- 172.21.15.40:0/4147279188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec014590 con 0x5623e3e68770 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.628+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623e3e641f0 con 0x5623e3e68770 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.628+0000 7fe8f4c5d640 1 -- 172.21.15.40:0/4147279188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec014940 con 0x5623e3e68770 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.628+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e3e41dd0 con 0x5623e3e68770 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.629+0000 7fe8f4c5d640 1 -- 172.21.15.40:0/4147279188 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8ec01b030 con 0x5623e3e68770 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.629+0000 7fe8f4c5d640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe8e406b300 0x7fe8e406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.630+0000 7fe8f4c5d640 1 -- 172.21.15.40:0/4147279188 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7937+0+0 (secure 0 0 0) 0x7fe8ec08ac40 con 0x5623e3e68770 2024-04-26T21:44:42.632 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.630+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5623e3ef2440 0x5623e3e651e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.630+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3e65720 con 0x5623e3ef2440 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.630+0000 7fe8f7462640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe8e406b300 0x7fe8e406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.630+0000 7fe8f7c63640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5623e3ef2440 0x5623e3e651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.630+0000 7fe8f7c63640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5623e3ef2440 0x5623e3e651e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.630+0000 7fe8f7462640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe8e406b300 0x7fe8e406d7c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0056000 tx=0x7fe8f0050000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.632+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v115'598 uv572 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.632+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3e65b20 con 0x5623e3ef2440 2024-04-26T21:44:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.633+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v115'599 uv599 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.633+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99260 con 0x5623e3ef2440 2024-04-26T21:44:42.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.634+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v115'600 uv600 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.634 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:42.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.634+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9a610 con 0x5623e3ef2440 2024-04-26T21:44:42.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.635+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v115'601 uv601 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.635+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9a9b0 con 0x5623e3ef2440 2024-04-26T21:44:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.636+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v115'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.636+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99600 con 0x5623e3ef2440 2024-04-26T21:44:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.638+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v115'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.638+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999c0 con 0x5623e3ef2440 2024-04-26T21:44:42.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.639+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v115'604 uv578 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.639+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3f999c0 con 0x5623e3ef2440 2024-04-26T21:44:42.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.640+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v115'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.640+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.641+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v115'606 uv606 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.641 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:42.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.641+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.642+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v115'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.642+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.643+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v115'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.643+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.644+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v115'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.644+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.645+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v115'610 uv585 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.645+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.646+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v115'611 uv611 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.646+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.647+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v115'612 uv612 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.647 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.647+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.648+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v115'613 uv613 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.648+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.649+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v115'614 uv614 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.650+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f999a0 con 0x5623e3ef2440 2024-04-26T21:44:42.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.651+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v115'615 uv615 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.651+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99d80 con 0x5623e3ef2440 2024-04-26T21:44:42.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.652+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v115'616 uv616 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.652+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99d60 con 0x5623e3ef2440 2024-04-26T21:44:42.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.653+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v115'617 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.653+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3f99d60 con 0x5623e3ef2440 2024-04-26T21:44:42.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.654+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v115'618 uv618 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.654+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.655 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:42.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.655+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v115'619 uv619 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.655+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.656+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v115'620 uv620 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.656+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.657+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v115'621 uv621 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.657+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.658+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v115'622 uv622 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.658+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.659+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v115'623 uv597 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.659+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.660+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v115'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.660+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.661 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:42.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.661+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v115'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.661+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.662+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v115'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.662+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.663+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v115'627 uv627 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.663+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f99c50 con 0x5623e3ef2440 2024-04-26T21:44:42.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.664+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v115'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3ef2440 2024-04-26T21:44:42.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.664+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5623e3f9ad50 0x5623e3f9d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.664+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9d680 con 0x5623e3f9ad50 2024-04-26T21:44:42.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.665+0000 7fe8f7462640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5623e3f9ad50 0x5623e3f9d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.665+0000 7fe8f7462640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5623e3f9ad50 0x5623e3f9d140 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.667+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v115'442 uv425 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe8f000a140 con 0x5623e3f9ad50 2024-04-26T21:44:42.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.667+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3f9da20 con 0x5623e3f9ad50 2024-04-26T21:44:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.668+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v115'443 uv443 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.668+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9ddc0 con 0x5623e3f9ad50 2024-04-26T21:44:42.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.669+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v115'444 uv444 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.669 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:42.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.669+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9e610 con 0x5623e3f9ad50 2024-04-26T21:44:42.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.670+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v115'445 uv445 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.670+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9e9b0 con 0x5623e3f9ad50 2024-04-26T21:44:42.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.672+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v115'446 uv446 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.672+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9ed50 con 0x5623e3f9ad50 2024-04-26T21:44:42.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.673+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v115'447 uv430 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.673+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3f9f0f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.674+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v115'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.674+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.675 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:42.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.676+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v115'449 uv449 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.676+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.677+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v115'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.677+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.678+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v115'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.678+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.679+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v115'452 uv435 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.679+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.681+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v115'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.681+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.681 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.682+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v115'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.682+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa3b50 con 0x5623e3f9ad50 2024-04-26T21:44:42.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.683+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v115'455 uv455 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.683+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa3b50 con 0x5623e3f9ad50 2024-04-26T21:44:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.684+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v115'456 uv456 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.684+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.686+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v115'457 uv441 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.686+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.687+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v115'458 uv458 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.687+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.688 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.688+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v115'459 uv459 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.688+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa04a0 con 0x5623e3f9ad50 2024-04-26T21:44:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.689+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v115'460 uv460 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.689+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa04a0 con 0x5623e3f9ad50 2024-04-26T21:44:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.691+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v115'461 uv461 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.691+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3f9f490 con 0x5623e3f9ad50 2024-04-26T21:44:42.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.692+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v115'462 uv462 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3f9ad50 2024-04-26T21:44:42.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.692+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623e3f9f9a0 0x5623e3fa1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.692+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa22d0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.692+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623e3f9f9a0 0x5623e3fa1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.692+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623e3f9f9a0 0x5623e3fa1d90 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.694+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v115'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.694+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3fa2670 con 0x5623e3f9f9a0 2024-04-26T21:44:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.695+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v115'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.695+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa2ae0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.696+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v115'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.695 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:42.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.696+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa4390 con 0x5623e3f9f9a0 2024-04-26T21:44:42.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.697+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v115'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.697+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa4730 con 0x5623e3f9f9a0 2024-04-26T21:44:42.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.698+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v115'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.698+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa3380 con 0x5623e3f9f9a0 2024-04-26T21:44:42.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.699+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v115'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.699+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5623e3fa3c80 0x5623e3fa6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.699+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa7020 con 0x5623e3fa3c80 2024-04-26T21:44:42.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.700+0000 7fe8f7c63640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5623e3fa3c80 0x5623e3fa6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.700+0000 7fe8f7c63640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5623e3fa3c80 0x5623e3fa6ae0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.701+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v115'92 uv91 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.701+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3fa7020 con 0x5623e3fa3c80 2024-04-26T21:44:42.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.702+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v115'93 uv93 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.703+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa70a0 con 0x5623e3fa3c80 2024-04-26T21:44:42.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.704+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v115'94 uv94 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.704 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:42.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.704+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa7490 con 0x5623e3fa3c80 2024-04-26T21:44:42.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.705+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v115'95 uv95 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.705+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5623e3fa79a0 0x5623e3fa9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.705+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa320 con 0x5623e3fa79a0 2024-04-26T21:44:42.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.705+0000 7fe8f7462640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5623e3fa79a0 0x5623e3fa9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.706+0000 7fe8f7462640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5623e3fa79a0 0x5623e3fa9d90 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.707+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v115'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.707+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3faa3e0 con 0x5623e3fa79a0 2024-04-26T21:44:42.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.708+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v115'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.708+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.709 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:42.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.709+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v115'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.710+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fab810 con 0x5623e3fa79a0 2024-04-26T21:44:42.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.711+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v115'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.711+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fab810 con 0x5623e3fa79a0 2024-04-26T21:44:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.712+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v115'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.712+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.713+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v115'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.713+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.715+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v115'216 uv210 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.715+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.716+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v115'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.716+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.718 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:42.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.717+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v115'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.717+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fab810 con 0x5623e3fa3c80 2024-04-26T21:44:42.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.718+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v115'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.718+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fab810 con 0x5623e3fa3c80 2024-04-26T21:44:42.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.719+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v115'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.719+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.720+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v115'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.720+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.721+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v115'222 uv215 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.721+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.722+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v115'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.722+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.723 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:42.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.723+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v115'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.723+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.724+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v115'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.724+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.725+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v115'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.725+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.726+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v115'97 uv96 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.726+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.727+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v115'98 uv98 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.727+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.727 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:42.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.728+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v115'99 uv99 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.728+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.729+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v115'100 uv100 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.729+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa3c80 2024-04-26T21:44:42.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.730+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v115'101 uv101 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8e800f030 con 0x5623e3fa3c80 2024-04-26T21:44:42.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.730+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.732+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v115'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.732+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.733+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v115'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.733+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.733 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.733+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v115'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.734+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fab810 con 0x5623e3fa79a0 2024-04-26T21:44:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.735+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v115'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.735+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fab810 con 0x5623e3fa79a0 2024-04-26T21:44:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.736+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v115'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.736+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.737+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v115'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.737+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.738+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v115'192 uv186 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.738+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.739+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v115'193 uv193 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.739+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa800 con 0x5623e3fa79a0 2024-04-26T21:44:42.740 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:42.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.740+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v115'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.740+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa8a0 con 0x5623e3fa79a0 2024-04-26T21:44:42.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.741+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v115'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.741+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa8a0 con 0x5623e3fa79a0 2024-04-26T21:44:42.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.742+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v115'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.742+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa8a0 con 0x5623e3fa79a0 2024-04-26T21:44:42.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.744+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v115'197 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.744+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3faa8a0 con 0x5623e3fa79a0 2024-04-26T21:44:42.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.745+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v115'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.745+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa8a0 con 0x5623e3fa79a0 2024-04-26T21:44:42.746 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:42.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.746+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v115'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.746+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faa8a0 con 0x5623e3fa79a0 2024-04-26T21:44:42.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.747+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v115'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.747+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fad630 con 0x5623e3fa79a0 2024-04-26T21:44:42.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.748+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v115'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.748+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5623e3fab310 0x5623e3fad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.748+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fadcf0 con 0x5623e3fab310 2024-04-26T21:44:42.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.749+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5623e3fab310 0x5623e3fad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.749+0000 7fe8f6c61640 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5623e3fab310 0x5623e3fad700 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.750+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v115'87 uv86 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3fab310 2024-04-26T21:44:42.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.750+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3fae090 con 0x5623e3fab310 2024-04-26T21:44:42.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.751+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v115'88 uv88 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3fab310 2024-04-26T21:44:42.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.751+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae110 con 0x5623e3fab310 2024-04-26T21:44:42.752 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:42.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.752+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v115'89 uv89 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3fab310 2024-04-26T21:44:42.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.752+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3fab310 2024-04-26T21:44:42.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.753+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v115'90 uv90 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3fab310 2024-04-26T21:44:42.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.753+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3fa79a0 2024-04-26T21:44:42.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.754+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v115'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.754+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3fa79a0 2024-04-26T21:44:42.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.755+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v115'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.755+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3fa79a0 2024-04-26T21:44:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:42.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.756+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v115'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.756+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3fa79a0 2024-04-26T21:44:42.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.758+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v115'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.758+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3fa79a0 2024-04-26T21:44:42.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.759+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v115'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.759+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3fa79a0 2024-04-26T21:44:42.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.760+0000 7fe8f7462640 1 -- 172.21.15.40:0/4147279188 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v115'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8f00547f0 con 0x5623e3fa79a0 2024-04-26T21:44:42.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.760+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3f9f9a0 2024-04-26T21:44:42.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.761+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v115'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.761+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3f9f9a0 2024-04-26T21:44:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.762+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v115'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.762+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae190 con 0x5623e3f9f9a0 2024-04-26T21:44:42.763 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:42.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.763+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v115'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.763+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa2f60 con 0x5623e3f9f9a0 2024-04-26T21:44:42.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.764+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v115'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.764+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa2f60 con 0x5623e3f9f9a0 2024-04-26T21:44:42.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.766+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v115'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.766+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa2f60 con 0x5623e3f9f9a0 2024-04-26T21:44:42.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.767+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v115'241 uv233 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.767+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3fa2f60 con 0x5623e3f9f9a0 2024-04-26T21:44:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.768+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v115'242 uv242 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.768+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fa2f60 con 0x5623e3f9f9a0 2024-04-26T21:44:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.769+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v115'243 uv243 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.769 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.769+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae640 con 0x5623e3f9f9a0 2024-04-26T21:44:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.770+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v115'244 uv244 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.770+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae640 con 0x5623e3f9f9a0 2024-04-26T21:44:42.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.771+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v115'245 uv245 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.771+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae640 con 0x5623e3f9f9a0 2024-04-26T21:44:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.772+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v115'246 uv240 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.772+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5623e3fae5c0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.773+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v115'247 uv247 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.773+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae5c0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.774 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.774+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v115'248 uv248 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.774+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faf5d0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.775+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v115'249 uv249 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.775+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3faf5d0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.776+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v115'250 uv250 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.776+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae5c0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.777+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v115'251 uv251 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.777+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5623e3fae5c0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f6c61640 1 -- 172.21.15.40:0/4147279188 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v115'252 uv252 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe8ec01ada0 con 0x5623e3f9f9a0 2024-04-26T21:44:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5623e3f9ad50 msgr2=0x5623e3f9d140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5623e3f9ad50 0x5623e3f9d140 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5623e3ef2440 msgr2=0x5623e3e651e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5623e3ef2440 0x5623e3e651e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5623e3fa3c80 msgr2=0x5623e3fa6ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5623e3fa3c80 0x5623e3fa6ae0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5623e3fab310 msgr2=0x5623e3fad700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5623e3fab310 0x5623e3fad700 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623e3f9f9a0 msgr2=0x5623e3fa1d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5623e3f9f9a0 0x5623e3fa1d90 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5623e3fa79a0 msgr2=0x5623e3fa9d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5623e3fa79a0 0x5623e3fa9d90 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 reap_dead start 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f7c63640 1 -- 172.21.15.40:0/4147279188 reap_dead start 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe8e406b300 msgr2=0x7fe8e406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe8e406b300 0x7fe8e406d7c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0056000 tx=0x7fe8f0050000 comp rx=0 tx=0).stop 2024-04-26T21:44:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3e68770 msgr2=0x5623e3ef1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3e68770 0x5623e3ef1f00 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec008400 tx=0x7fe8ec0084f0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 shutdown_connections 2024-04-26T21:44:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe8e406b300 0x7fe8e406d7c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 --2- 172.21.15.40:0/4147279188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5623e3e68770 0x5623e3ef1f00 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 >> 172.21.15.40:0/4147279188 conn(0x5623e3e4a9b0 msgr2=0x5623e3e1cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 shutdown_connections 2024-04-26T21:44:42.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.778+0000 7fe8f9966f40 1 -- 172.21.15.40:0/4147279188 wait complete. 2024-04-26T21:44:42.781 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-26T21:44:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.865+0000 7f033ef01f40 1 Processor -- start 2024-04-26T21:44:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.865+0000 7f033ef01f40 1 -- start start 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.865+0000 7f033ef01f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bfee75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.865+0000 7f033ef01f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef0e60 0x55f5bff0a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.865+0000 7f033ef01f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef3f30 0x55f5bfef32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.865+0000 7f033ef01f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f5bfe7d2d0 con 0x55f5bfef0a80 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.865+0000 7f033ef01f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5bfddce40 con 0x55f5bfef3f30 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.865+0000 7f033ef01f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f5bfe7d570 con 0x55f5bfef0e60 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bfee75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bfee75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34564/0 (socket says 172.21.15.40:34564) 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 -- 172.21.15.40:0/2723215362 learned_addr learned my addr 172.21.15.40:0/2723215362 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c1fc640 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef0e60 0x55f5bff0a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033d1fe640 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef3f30 0x55f5bfef32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 -- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef0e60 msgr2=0x55f5bff0a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef0e60 0x55f5bff0a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 -- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef3f30 msgr2=0x55f5bfef32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef3f30 0x55f5bfef32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 -- 172.21.15.40:0/2723215362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5bfe7ea40 con 0x55f5bfef0a80 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033d1fe640 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef3f30 0x55f5bfef32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033c9fd640 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bfee75e0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f032c00a520 tx=0x7f032c00a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9405607189ea58af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033b1fa640 1 -- 172.21.15.40:0/2723215362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c00f5b0 con 0x55f5bfef0a80 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033b1fa640 1 -- 172.21.15.40:0/2723215362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c00f750 con 0x55f5bfef0a80 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.866+0000 7f033b1fa640 1 -- 172.21.15.40:0/2723215362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c00fb40 con 0x55f5bfef0a80 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 -- 172.21.15.40:0/2723215362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 msgr2=0x55f5bfee75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bfee75e0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f032c00a520 tx=0x7f032c00a9e0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 -- 172.21.15.40:0/2723215362 shutdown_connections 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef3f30 0x55f5bfef32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef0e60 0x55f5bff0a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 --2- 172.21.15.40:0/2723215362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bfee75e0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 -- 172.21.15.40:0/2723215362 >> 172.21.15.40:0/2723215362 conn(0x55f5bfe33ab0 msgr2=0x55f5bfe34720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 -- 172.21.15.40:0/2723215362 shutdown_connections 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 -- 172.21.15.40:0/2723215362 wait complete. 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 Processor -- start 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.867+0000 7f033ef01f40 1 -- start start 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033ef01f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bff0ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033ef01f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef0e60 0x55f5bff0b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033ef01f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef3f30 0x55f5bff072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033ef01f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f5bff0d920 con 0x55f5bfef0a80 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033ef01f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5bff0e060 con 0x55f5bfef0e60 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033ef01f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f5bff0de00 con 0x55f5bfef3f30 2024-04-26T21:44:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c9fd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bff0ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c9fd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bff0ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34576/0 (socket says 172.21.15.40:34576) 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c9fd640 1 -- 172.21.15.40:0/3744477210 learned_addr learned my addr 172.21.15.40:0/3744477210 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c1fc640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef0e60 0x55f5bff0b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033d1fe640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef3f30 0x55f5bff072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c9fd640 1 -- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef3f30 msgr2=0x55f5bff072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c9fd640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef3f30 0x55f5bff072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c9fd640 1 -- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef0e60 msgr2=0x55f5bff0b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c9fd640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5bfef0e60 0x55f5bff0b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033c9fd640 1 -- 172.21.15.40:0/3744477210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5bfeec9b0 con 0x55f5bfef0a80 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.868+0000 7f033d1fe640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5bfef3f30 0x55f5bff072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.869+0000 7f033c9fd640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bff0ad10 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f032c00aa20 tx=0x7f032c003350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.869+0000 7f033a1f8640 1 -- 172.21.15.40:0/3744477210 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c00f5b0 con 0x55f5bfef0a80 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.869+0000 7f033a1f8640 1 -- 172.21.15.40:0/3744477210 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c010a80 con 0x55f5bfef0a80 2024-04-26T21:44:42.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.869+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f5bfeec3f0 con 0x55f5bfef0a80 2024-04-26T21:44:42.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.869+0000 7f033a1f8640 1 -- 172.21.15.40:0/3744477210 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c010e50 con 0x55f5bfef0a80 2024-04-26T21:44:42.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.869+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5bfee9500 con 0x55f5bfef0a80 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.870+0000 7f033a1f8640 1 -- 172.21.15.40:0/3744477210 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f032c017030 con 0x55f5bfef0a80 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.871+0000 7f033a1f8640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f031c06b470 0x7f031c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.871+0000 7f033a1f8640 1 -- 172.21.15.40:0/3744477210 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7937+0+0 (secure 0 0 0) 0x7f032c08cfe0 con 0x55f5bfef0a80 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.871+0000 7f033c1fc640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f031c06b470 0x7f031c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.871+0000 7f033ef01f40 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f5bfeeb2b0 0x55f5bfeeb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.871+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55f5bfeebbd0 con 0x55f5bfeeb2b0 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.871+0000 7f033d1fe640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f5bfeeb2b0 0x55f5bfeeb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.871+0000 7f033c1fc640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f031c06b470 0x7f031c06d930 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x55f5bfe621c0 tx=0x7f032400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.872+0000 7f033d1fe640 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f5bfeeb2b0 0x55f5bfeeb690 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033d1fe640 1 -- 172.21.15.40:0/3744477210 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~80] v115'202 uv202 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0334050050 con 0x55f5bfeeb2b0 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f5bfeeb2b0 msgr2=0x55f5bfeeb690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55f5bfeeb2b0 0x55f5bfeeb690 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f031c06b470 msgr2=0x7f031c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f031c06b470 0x7f031c06d930 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x55f5bfe621c0 tx=0x7f032400c040 comp rx=0 tx=0).stop 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 msgr2=0x55f5bff0ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 --2- 172.21.15.40:0/3744477210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5bfef0a80 0x55f5bff0ad10 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f032c00aa20 tx=0x7f032c003350 comp rx=0 tx=0).stop 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033d1fe640 1 -- 172.21.15.40:0/3744477210 reap_dead start 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 shutdown_connections 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 >> 172.21.15.40:0/3744477210 conn(0x55f5bfe33ab0 msgr2=0x55f5bfeef500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 shutdown_connections 2024-04-26T21:44:42.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:42.873+0000 7f033ef01f40 1 -- 172.21.15.40:0/3744477210 wait complete. 2024-04-26T21:44:42.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:43.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.022+0000 7f98b9a6b640 1 Processor -- start 2024-04-26T21:44:43.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.022+0000 7f98b9a6b640 1 -- start start 2024-04-26T21:44:43.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b9a6b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f98b415f170 0x7f98b40d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b9a6b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b40d1040 con 0x7f98b40d3bc0 2024-04-26T21:44:43.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b9a6b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b40c9680 con 0x7f98b415f550 2024-04-26T21:44:43.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b9a6b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f98b40c9430 con 0x7f98b415f170 2024-04-26T21:44:43.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f98b40d3bc0 0x7f98b40ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39582/0 (socket says 172.21.15.40:39582) 2024-04-26T21:44:43.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b3fff640 1 -- 172.21.15.40:0/2216667501 learned_addr learned my addr 172.21.15.40:0/2216667501 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b37fe640 1 --2- 172.21.15.40:0/2216667501 >> v2:172.21.15.184:3300/0 conn(0x7f98b415f170 0x7f98b40d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b27fc640 1 -- 172.21.15.40:0/2216667501 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1231534782 0 0) 0x7f98ac001490 con 0x7f98b40d3bc0 2024-04-26T21:44:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b27fc640 1 -- 172.21.15.40:0/2216667501 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f989c003520 con 0x7f98b40d3bc0 2024-04-26T21:44:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b37fe640 1 -- 172.21.15.40:0/2216667501 >> v1:172.21.15.184:6789/0 conn(0x7f98b415f550 legacy=0x7f98b40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b27fc640 1 -- 172.21.15.40:0/2216667501 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557778682 0 0) 0x7f98b40c9680 con 0x7f98b415f550 2024-04-26T21:44:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b37fe640 1 -- 172.21.15.40:0/2216667501 >> v1:172.21.15.40:6789/0 conn(0x7f98b40d3bc0 legacy=0x7f98b40ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.023+0000 7f98b37fe640 1 -- 172.21.15.40:0/2216667501 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40c9960 con 0x7f98b415f170 2024-04-26T21:44:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.024+0000 7f98b37fe640 1 --2- 172.21.15.40:0/2216667501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 0x7f98b40d5a80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f98b40c95b0 tx=0x7f98a402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cd33f226e6153e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.024+0000 7f98b27fc640 1 -- 172.21.15.40:0/2216667501 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4037070 con 0x7f98b415f170 2024-04-26T21:44:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.024+0000 7f98b27fc640 1 -- 172.21.15.40:0/2216667501 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a4032440 con 0x7f98b415f170 2024-04-26T21:44:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.024+0000 7f98b27fc640 1 -- 172.21.15.40:0/2216667501 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40327f0 con 0x7f98b415f170 2024-04-26T21:44:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.025+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2216667501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 msgr2=0x7f98b40d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.025+0000 7f98b9a6b640 1 --2- 172.21.15.40:0/2216667501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 0x7f98b40d5a80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f98b40c95b0 tx=0x7f98a402a250 comp rx=0 tx=0).stop 2024-04-26T21:44:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.025+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2216667501 shutdown_connections 2024-04-26T21:44:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.025+0000 7f98b9a6b640 1 --2- 172.21.15.40:0/2216667501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 0x7f98b40d5a80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.025+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2216667501 >> 172.21.15.40:0/2216667501 conn(0x7f98b40d4f60 msgr2=0x7f98b40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.025+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2216667501 shutdown_connections 2024-04-26T21:44:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.025+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2216667501 wait complete. 2024-04-26T21:44:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.026+0000 7f98b9a6b640 1 Processor -- start 2024-04-26T21:44:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.026+0000 7f98b9a6b640 1 -- start start 2024-04-26T21:44:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b9a6b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98b40d3bc0 0x7f98b41f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b9a6b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 0x7f98b40ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b9a6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b41f5740 0x7f98b41f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b9a6b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98b40d2e30 con 0x7f98b41f5740 2024-04-26T21:44:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b9a6b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98b40d2cb0 con 0x7f98b415f170 2024-04-26T21:44:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b9a6b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98b40d2fb0 con 0x7f98b40d3bc0 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b41f5740 0x7f98b41f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b41f5740 0x7f98b41f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34588/0 (socket says 172.21.15.40:34588) 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b3fff640 1 -- 172.21.15.40:0/2202347934 learned_addr learned my addr 172.21.15.40:0/2202347934 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b37fe640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98b40d3bc0 0x7f98b41f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b2ffd640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 0x7f98b40ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b3fff640 1 -- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98b40d3bc0 msgr2=0x7f98b41f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b3fff640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98b40d3bc0 0x7f98b41f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b3fff640 1 -- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 msgr2=0x7f98b40ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b3fff640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 0x7f98b40ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.027+0000 7f98b3fff640 1 -- 172.21.15.40:0/2202347934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b41f6400 con 0x7f98b41f5740 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.028+0000 7f98b2ffd640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 0x7f98b40ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.028+0000 7f98b3fff640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b41f5740 0x7f98b41f5bd0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00cb50 tx=0x7f98ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.028+0000 7f989bfff640 1 -- 172.21.15.40:0/2202347934 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac017070 con 0x7f98b41f5740 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.028+0000 7f989bfff640 1 -- 172.21.15.40:0/2202347934 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac00ad40 con 0x7f98b41f5740 2024-04-26T21:44:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.028+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b41f66e0 con 0x7f98b41f5740 2024-04-26T21:44:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.028+0000 7f989bfff640 1 -- 172.21.15.40:0/2202347934 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac012620 con 0x7f98b41f5740 2024-04-26T21:44:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.028+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b41f6ca0 con 0x7f98b41f5740 2024-04-26T21:44:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.029+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b40694f0 con 0x7f98b41f5740 2024-04-26T21:44:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.029+0000 7f989bfff640 1 -- 172.21.15.40:0/2202347934 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98ac01a070 con 0x7f98b41f5740 2024-04-26T21:44:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.030+0000 7f989bfff640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98a006c8f0 0x7f98a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.030+0000 7f989bfff640 1 -- 172.21.15.40:0/2202347934 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7937+0+0 (secure 0 0 0) 0x7f98ac08c3b0 con 0x7f98b41f5740 2024-04-26T21:44:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.030+0000 7f98b37fe640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98a006c8f0 0x7f98a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.030+0000 7f98b37fe640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98a006c8f0 0x7f98a006edb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f98a4034000 tx=0x7f98a402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.034+0000 7f989bfff640 1 -- 172.21.15.40:0/2202347934 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f98b40d1040 con 0x7f98b41f5740 2024-04-26T21:44:43.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.261+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f98b4074590 con 0x7f98b41f5740 2024-04-26T21:44:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.567+0000 7f989bfff640 1 -- 172.21.15.40:0/2202347934 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v218) ==== 162+0+0 (secure 0 0 0) 0x7f98ac059e00 con 0x7f98b41f5740 2024-04-26T21:44:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98a006c8f0 msgr2=0x7f98a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98a006c8f0 0x7f98a006edb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f98a4034000 tx=0x7f98a402e000 comp rx=0 tx=0).stop 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b41f5740 msgr2=0x7f98b41f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b41f5740 0x7f98b41f5bd0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00cb50 tx=0x7f98ac00a430 comp rx=0 tx=0).stop 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 shutdown_connections 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f98a006c8f0 0x7f98a006edb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b41f5740 0x7f98b41f5bd0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b415f170 0x7f98b40ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 --2- 172.21.15.40:0/2202347934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98b40d3bc0 0x7f98b41f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 >> 172.21.15.40:0/2202347934 conn(0x7f98b40d4f60 msgr2=0x7f98b406d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 shutdown_connections 2024-04-26T21:44:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.570+0000 7f98b9a6b640 1 -- 172.21.15.40:0/2202347934 wait complete. 2024-04-26T21:44:43.573 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:43.574+0000 7f1c12bc4640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 2 < 101: Malformed input [buffer:3] 2024-04-26T21:44:43.574 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:43.574+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:43.574 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:43.574+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:43.580 INFO:tasks.ceph.mds.c.smithi040.stderr: -25> 2024-04-26T21:44:43.574+0000 7f1c12bc4640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 2 < 101: Malformed input [buffer:3] 2024-04-26T21:44:43.584 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:43.605 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:43.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.687+0000 7ff8fc704640 1 Processor -- start 2024-04-26T21:44:43.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.688+0000 7ff8fc704640 1 -- start start 2024-04-26T21:44:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.688+0000 7ff8fc704640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8f40d06f0 0x7ff8f40d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.688+0000 7ff8fc704640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f40d0ba0 con 0x7ff8f40b2850 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.688+0000 7ff8fc704640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f40d0e20 con 0x7ff8f415f3a0 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.688+0000 7ff8fc704640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f40b1d70 con 0x7ff8f40d06f0 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.689+0000 7ff8f9c9f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8f40d06f0 0x7ff8f40d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.689+0000 7ff8faca1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff8f40b2850 0x7ff8f40b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39602/0 (socket says 172.21.15.40:39602) 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.689+0000 7ff8faca1640 1 -- 172.21.15.40:0/2415636248 learned_addr learned my addr 172.21.15.40:0/2415636248 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.689+0000 7ff8f9c9f640 1 -- 172.21.15.40:0/2415636248 >> v1:172.21.15.184:6789/0 conn(0x7ff8f415f3a0 legacy=0x7ff8f40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.689+0000 7ff8f9c9f640 1 -- 172.21.15.40:0/2415636248 >> v1:172.21.15.40:6789/0 conn(0x7ff8f40b2850 legacy=0x7ff8f40b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.689+0000 7ff8f9c9f640 1 -- 172.21.15.40:0/2415636248 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f40b2050 con 0x7ff8f40d06f0 2024-04-26T21:44:43.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8f9c9f640 1 --2- 172.21.15.40:0/2415636248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f40d06f0 0x7ff8f40d1e80 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40097a0 tx=0x7ff8e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7876da7da6fc40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:43.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8f949e640 1 -- 172.21.15.40:0/2415636248 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4037070 con 0x7ff8f40d06f0 2024-04-26T21:44:43.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8f949e640 1 -- 172.21.15.40:0/2415636248 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e4032440 con 0x7ff8f40d06f0 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8f949e640 1 -- 172.21.15.40:0/2415636248 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4032810 con 0x7ff8f40d06f0 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8fc704640 1 -- 172.21.15.40:0/2415636248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f40d06f0 msgr2=0x7ff8f40d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8fc704640 1 --2- 172.21.15.40:0/2415636248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f40d06f0 0x7ff8f40d1e80 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40097a0 tx=0x7ff8e402a230 comp rx=0 tx=0).stop 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8fc704640 1 -- 172.21.15.40:0/2415636248 shutdown_connections 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8fc704640 1 --2- 172.21.15.40:0/2415636248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f40d06f0 0x7ff8f40d1e80 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8fc704640 1 -- 172.21.15.40:0/2415636248 >> 172.21.15.40:0/2415636248 conn(0x7ff8f40d4f90 msgr2=0x7ff8f40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8fc704640 1 -- 172.21.15.40:0/2415636248 shutdown_connections 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.690+0000 7ff8fc704640 1 -- 172.21.15.40:0/2415636248 wait complete. 2024-04-26T21:44:43.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.692+0000 7ff8fc704640 1 Processor -- start 2024-04-26T21:44:43.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.692+0000 7ff8fc704640 1 -- start start 2024-04-26T21:44:43.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8fc704640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8f40b2850 0x7ff8f415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8fc704640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8f40d06f0 0x7ff8f40b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8fc704640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f415f3a0 0x7ff8f40b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8fc704640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f415f900 con 0x7ff8f415f3a0 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8fc704640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f415f780 con 0x7ff8f40b2850 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8fc704640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f40d23c0 con 0x7ff8f40d06f0 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f415f3a0 0x7ff8f40b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f415f3a0 0x7ff8f40b9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34606/0 (socket says 172.21.15.40:34606) 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 -- 172.21.15.40:0/803423491 learned_addr learned my addr 172.21.15.40:0/803423491 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8fa4a0640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8f40b2850 0x7ff8f415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8f9c9f640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8f40d06f0 0x7ff8f40b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 -- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8f40d06f0 msgr2=0x7ff8f40b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8f40d06f0 0x7ff8f40b9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 -- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8f40b2850 msgr2=0x7ff8f415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8f40b2850 0x7ff8f415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 -- 172.21.15.40:0/803423491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f40b8ae0 con 0x7ff8f415f3a0 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8faca1640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f415f3a0 0x7ff8f40b9d90 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00cbc0 tx=0x7ff8ec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8eaffd640 1 -- 172.21.15.40:0/803423491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec017070 con 0x7ff8f415f3a0 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8eaffd640 1 -- 172.21.15.40:0/803423491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ec00ad40 con 0x7ff8f415f3a0 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f40b7d20 con 0x7ff8f415f3a0 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.693+0000 7ff8eaffd640 1 -- 172.21.15.40:0/803423491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec012600 con 0x7ff8f415f3a0 2024-04-26T21:44:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.694+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f40b82e0 con 0x7ff8f415f3a0 2024-04-26T21:44:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.694+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f40694f0 con 0x7ff8f415f3a0 2024-04-26T21:44:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.695+0000 7ff8eaffd640 1 -- 172.21.15.40:0/803423491 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8ec01a070 con 0x7ff8f415f3a0 2024-04-26T21:44:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.695+0000 7ff8eaffd640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8d806c8f0 0x7ff8d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.695+0000 7ff8eaffd640 1 -- 172.21.15.40:0/803423491 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8023+0+0 (secure 0 0 0) 0x7ff8ec08c020 con 0x7ff8f415f3a0 2024-04-26T21:44:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.696+0000 7ff8fa4a0640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8d806c8f0 0x7ff8d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.696+0000 7ff8fa4a0640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8d806c8f0 0x7ff8d806edb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00016e0 tx=0x7ff8f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.700+0000 7ff8eaffd640 1 -- 172.21.15.40:0/803423491 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8f40d0ba0 con 0x7ff8f415f3a0 2024-04-26T21:44:43.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.923+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8f40b6860 con 0x7ff8f415f3a0 2024-04-26T21:44:43.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.923+0000 7ff8eaffd640 1 -- 172.21.15.40:0/803423491 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) ==== 95+0+4355 (secure 0 0 0) 0x7ff8ec059580 con 0x7ff8f415f3a0 2024-04-26T21:44:43.923 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:43.924 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":220,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:43.575125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 220 2024-04-26T21:44:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8d806c8f0 msgr2=0x7ff8d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8d806c8f0 0x7ff8d806edb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00016e0 tx=0x7ff8f0009040 comp rx=0 tx=0).stop 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f415f3a0 msgr2=0x7ff8f40b9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f415f3a0 0x7ff8f40b9d90 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00cbc0 tx=0x7ff8ec00a430 comp rx=0 tx=0).stop 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 shutdown_connections 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8d806c8f0 0x7ff8d806edb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8f415f3a0 0x7ff8f40b9d90 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8f40d06f0 0x7ff8f40b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 --2- 172.21.15.40:0/803423491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8f40b2850 0x7ff8f415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 >> 172.21.15.40:0/803423491 conn(0x7ff8f40d4f90 msgr2=0x7ff8f40b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.925+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 shutdown_connections 2024-04-26T21:44:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:43.926+0000 7ff8fc704640 1 -- 172.21.15.40:0/803423491 wait complete. 2024-04-26T21:44:43.936 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:43.936 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.078+0000 7fed7af89640 1 Processor -- start 2024-04-26T21:44:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.078+0000 7fed7af89640 1 -- start start 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed7af89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74071e60 0x7fed7406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed7af89640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed7406fe20 0x7fed74072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed7af89640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fed74072780 0x7fed740cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed7af89640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fed7415d7c0 con 0x7fed7406fe20 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed7af89640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fed7415daa0 con 0x7fed74072780 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed7af89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fed7415dd80 con 0x7fed74071e60 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed73fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed7406fe20 0x7fed74072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed73fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed7406fe20 0x7fed74072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34622/0 (socket says 172.21.15.40:34622) 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed73fff640 1 -- 172.21.15.40:0/782754984 learned_addr learned my addr 172.21.15.40:0/782754984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed73fff640 1 -- 172.21.15.40:0/782754984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74071e60 msgr2=0x7fed7406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed79526640 1 --2- 172.21.15.40:0/782754984 >> v2:172.21.15.184:3300/0 conn(0x7fed74072780 0x7fed740cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed78d25640 1 --2- 172.21.15.40:0/782754984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74071e60 0x7fed7406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed73fff640 1 --2- 172.21.15.40:0/782754984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74071e60 0x7fed7406f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed73fff640 1 -- 172.21.15.40:0/782754984 >> v2:172.21.15.184:3300/0 conn(0x7fed74072780 msgr2=0x7fed740cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed73fff640 1 --2- 172.21.15.40:0/782754984 >> v2:172.21.15.184:3300/0 conn(0x7fed74072780 0x7fed740cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.079+0000 7fed73fff640 1 -- 172.21.15.40:0/782754984 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7415e060 con 0x7fed7406fe20 2024-04-26T21:44:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.080+0000 7fed73fff640 1 --2- 172.21.15.40:0/782754984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7406fe20 0x7fed74072240 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fed7406e5c0 tx=0x7fed6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce004fc5b520921 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.080+0000 7fed737fe640 1 -- 172.21.15.40:0/782754984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60037070 con 0x7fed7406fe20 2024-04-26T21:44:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.080+0000 7fed737fe640 1 -- 172.21.15.40:0/782754984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed60032440 con 0x7fed7406fe20 2024-04-26T21:44:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.080+0000 7fed737fe640 1 -- 172.21.15.40:0/782754984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60032810 con 0x7fed7406fe20 2024-04-26T21:44:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 -- 172.21.15.40:0/782754984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7406fe20 msgr2=0x7fed74072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 --2- 172.21.15.40:0/782754984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7406fe20 0x7fed74072240 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fed7406e5c0 tx=0x7fed6002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 -- 172.21.15.40:0/782754984 shutdown_connections 2024-04-26T21:44:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 --2- 172.21.15.40:0/782754984 >> v2:172.21.15.184:3300/0 conn(0x7fed74072780 0x7fed740cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 --2- 172.21.15.40:0/782754984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed7406fe20 0x7fed74072240 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 --2- 172.21.15.40:0/782754984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74071e60 0x7fed7406f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 -- 172.21.15.40:0/782754984 >> 172.21.15.40:0/782754984 conn(0x7fed740d0be0 msgr2=0x7fed740d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 -- 172.21.15.40:0/782754984 shutdown_connections 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.081+0000 7fed7af89640 1 -- 172.21.15.40:0/782754984 wait complete. 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.082+0000 7fed7af89640 1 Processor -- start 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.082+0000 7fed7af89640 1 -- start start 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed7af89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7406fe20 0x7fed7414f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed7af89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed74071e60 0x7fed740af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed7af89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74072780 0x7fed740b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed7af89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed7415d3f0 con 0x7fed74071e60 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed7af89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed74072c10 con 0x7fed7406fe20 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed7af89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fed7415d570 con 0x7fed74072780 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed74071e60 0x7fed740af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed74071e60 0x7fed740af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34626/0 (socket says 172.21.15.40:34626) 2024-04-26T21:44:44.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 -- 172.21.15.40:0/2377396404 learned_addr learned my addr 172.21.15.40:0/2377396404 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed79526640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74072780 0x7fed740b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed78d25640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7406fe20 0x7fed7414f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 -- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74072780 msgr2=0x7fed740b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74072780 0x7fed740b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 -- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7406fe20 msgr2=0x7fed7414f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7406fe20 0x7fed7414f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 -- 172.21.15.40:0/2377396404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed740cea00 con 0x7fed74071e60 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed78d25640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7406fe20 0x7fed7414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed79526640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74072780 0x7fed740b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed73fff640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed74071e60 0x7fed740af2e0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fed60000c00 tx=0x7fed600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.083+0000 7fed717fa640 1 -- 172.21.15.40:0/2377396404 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60041070 con 0x7fed74071e60 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.084+0000 7fed717fa640 1 -- 172.21.15.40:0/2377396404 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6003c870 con 0x7fed74071e60 2024-04-26T21:44:44.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.084+0000 7fed717fa640 1 -- 172.21.15.40:0/2377396404 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60037040 con 0x7fed74071e60 2024-04-26T21:44:44.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.084+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed740cece0 con 0x7fed74071e60 2024-04-26T21:44:44.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.084+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7414f680 con 0x7fed74071e60 2024-04-26T21:44:44.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.085+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed740694f0 con 0x7fed74071e60 2024-04-26T21:44:44.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.085+0000 7fed717fa640 1 -- 172.21.15.40:0/2377396404 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed60044070 con 0x7fed74071e60 2024-04-26T21:44:44.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.086+0000 7fed717fa640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed5806c8f0 0x7fed5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.086+0000 7fed717fa640 1 -- 172.21.15.40:0/2377396404 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8023+0+0 (secure 0 0 0) 0x7fed600acf90 con 0x7fed74071e60 2024-04-26T21:44:44.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.086+0000 7fed78d25640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed5806c8f0 0x7fed5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.086+0000 7fed78d25640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed5806c8f0 0x7fed5806edb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fed68006fd0 tx=0x7fed68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.090+0000 7fed717fa640 1 -- 172.21.15.40:0/2377396404 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed6007ab40 con 0x7fed74071e60 2024-04-26T21:44:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.314+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed74160380 con 0x7fed74071e60 2024-04-26T21:44:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.314+0000 7fed717fa640 1 -- 172.21.15.40:0/2377396404 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) ==== 95+0+4355 (secure 0 0 0) 0x7fed6007a440 con 0x7fed74071e60 2024-04-26T21:44:44.314 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:44.314 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":220,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:43.575125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 220 2024-04-26T21:44:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed5806c8f0 msgr2=0x7fed5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed5806c8f0 0x7fed5806edb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fed68006fd0 tx=0x7fed68008040 comp rx=0 tx=0).stop 2024-04-26T21:44:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed74071e60 msgr2=0x7fed740af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed74071e60 0x7fed740af2e0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fed60000c00 tx=0x7fed600046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 shutdown_connections 2024-04-26T21:44:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fed5806c8f0 0x7fed5806edb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed74072780 0x7fed740b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed74071e60 0x7fed740af2e0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 --2- 172.21.15.40:0/2377396404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7406fe20 0x7fed7414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 >> 172.21.15.40:0/2377396404 conn(0x7fed740d0be0 msgr2=0x7fed7415e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 shutdown_connections 2024-04-26T21:44:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.316+0000 7fed7af89640 1 -- 172.21.15.40:0/2377396404 wait complete. 2024-04-26T21:44:44.327 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:44.327 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:44.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.427+0000 7fcaf75c6640 1 Processor -- start 2024-04-26T21:44:44.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.427+0000 7fcaf75c6640 1 -- start start 2024-04-26T21:44:44.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.428+0000 7fcaf75c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 0x7fcaf00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.428+0000 7fcaf75c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcaf00b2850 0x7fcaf00b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.428+0000 7fcaf75c6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf00d0b70 con 0x7fcaf015f3a0 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.428+0000 7fcaf75c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf00d0e20 con 0x7fcaf00b2850 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.428+0000 7fcaf75c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf00b1d70 con 0x7fcaf00d06f0 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.428+0000 7fcaf5b63640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcaf00b2850 0x7fcaf00b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.428+0000 7fcaf5b63640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcaf00b2850 0x7fcaf00b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34640/0 (socket says 172.21.15.40:34640) 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.428+0000 7fcaf5b63640 1 -- 172.21.15.40:0/2894533829 learned_addr learned my addr 172.21.15.40:0/2894533829 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.429+0000 7fcaf4b61640 1 --2- 172.21.15.40:0/2894533829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 0x7fcaf00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.429+0000 7fcaf5b63640 1 -- 172.21.15.40:0/2894533829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 msgr2=0x7fcaf00d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.429+0000 7fcaf5b63640 1 --2- 172.21.15.40:0/2894533829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 0x7fcaf00d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.429+0000 7fcaf5b63640 1 -- 172.21.15.40:0/2894533829 >> v1:172.21.15.184:6789/0 conn(0x7fcaf015f3a0 legacy=0x7fcaf00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.429+0000 7fcaf5b63640 1 -- 172.21.15.40:0/2894533829 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf00b2050 con 0x7fcaf00b2850 2024-04-26T21:44:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf5b63640 1 --2- 172.21.15.40:0/2894533829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf00b2850 0x7fcaf00b2cc0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009e50 tx=0x7fcae802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e854734adf6c7521 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcae7fff640 1 -- 172.21.15.40:0/2894533829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8036070 con 0x7fcaf00b2850 2024-04-26T21:44:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcae7fff640 1 -- 172.21.15.40:0/2894533829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae8031600 con 0x7fcaf00b2850 2024-04-26T21:44:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcae7fff640 1 -- 172.21.15.40:0/2894533829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80319d0 con 0x7fcaf00b2850 2024-04-26T21:44:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf75c6640 1 -- 172.21.15.40:0/2894533829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf00b2850 msgr2=0x7fcaf00b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/2894533829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf00b2850 0x7fcaf00b2cc0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009e50 tx=0x7fcae802a710 comp rx=0 tx=0).stop 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf75c6640 1 -- 172.21.15.40:0/2894533829 shutdown_connections 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/2894533829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf00b2850 0x7fcaf00b2cc0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/2894533829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 0x7fcaf00d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf75c6640 1 -- 172.21.15.40:0/2894533829 >> 172.21.15.40:0/2894533829 conn(0x7fcaf00d4f90 msgr2=0x7fcaf00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf75c6640 1 -- 172.21.15.40:0/2894533829 shutdown_connections 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.430+0000 7fcaf75c6640 1 -- 172.21.15.40:0/2894533829 wait complete. 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf75c6640 1 Processor -- start 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf75c6640 1 -- start start 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf75c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaf00b2850 0x7fcaf015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf75c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 0x7fcaf01502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf75c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf015f3a0 0x7fcaf01f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf75c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf015f900 con 0x7fcaf015f3a0 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf75c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf015f780 con 0x7fcaf00b2850 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf75c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf00d23c0 con 0x7fcaf00d06f0 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf5b63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf015f3a0 0x7fcaf01f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf5b63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf015f3a0 0x7fcaf01f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34646/0 (socket says 172.21.15.40:34646) 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.432+0000 7fcaf5b63640 1 -- 172.21.15.40:0/3051844063 learned_addr learned my addr 172.21.15.40:0/3051844063 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf4b61640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 0x7fcaf01502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf5362640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaf00b2850 0x7fcaf015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf5b63640 1 -- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 msgr2=0x7fcaf01502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf5b63640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 0x7fcaf01502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf5b63640 1 -- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaf00b2850 msgr2=0x7fcaf015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf5b63640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaf00b2850 0x7fcaf015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf5b63640 1 -- 172.21.15.40:0/3051844063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf015ffc0 con 0x7fcaf015f3a0 2024-04-26T21:44:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf5362640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaf00b2850 0x7fcaf015fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf5b63640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf015f3a0 0x7fcaf01f8fa0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009e50 tx=0x7fcae8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcae5ffb640 1 -- 172.21.15.40:0/3051844063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8036050 con 0x7fcaf015f3a0 2024-04-26T21:44:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcae5ffb640 1 -- 172.21.15.40:0/3051844063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae8042070 con 0x7fcaf015f3a0 2024-04-26T21:44:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf01f87e0 con 0x7fcaf015f3a0 2024-04-26T21:44:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcae5ffb640 1 -- 172.21.15.40:0/3051844063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8009510 con 0x7fcaf015f3a0 2024-04-26T21:44:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.433+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf00b9200 con 0x7fcaf015f3a0 2024-04-26T21:44:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.434+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf00694f0 con 0x7fcaf015f3a0 2024-04-26T21:44:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.435+0000 7fcae5ffb640 1 -- 172.21.15.40:0/3051844063 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcae8007bf0 con 0x7fcaf015f3a0 2024-04-26T21:44:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.435+0000 7fcae5ffb640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcad406c8f0 0x7fcad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.435+0000 7fcae5ffb640 1 -- 172.21.15.40:0/3051844063 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8023+0+0 (secure 0 0 0) 0x7fcae80ad1b0 con 0x7fcaf015f3a0 2024-04-26T21:44:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.435+0000 7fcaf5362640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcad406c8f0 0x7fcad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.436+0000 7fcaf5362640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcad406c8f0 0x7fcad406edb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fcaec001700 tx=0x7fcaec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.439+0000 7fcae5ffb640 1 -- 172.21.15.40:0/3051844063 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcae807acc0 con 0x7fcaf015f3a0 2024-04-26T21:44:44.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.666+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaf00b3970 con 0x7fcaf015f3a0 2024-04-26T21:44:44.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.666+0000 7fcae5ffb640 1 -- 172.21.15.40:0/3051844063 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 221 v221) ==== 95+0+5181 (secure 0 0 0) 0x7fcae807a5c0 con 0x7fcaf015f3a0 2024-04-26T21:44:44.667 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:44.667 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":221,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4237566530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4237566530},{"type":"v1","addr":"172.21.15.40:6833","nonce":4237566530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":221}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:43.575125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:44.667 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 221 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.668+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcad406c8f0 msgr2=0x7fcad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.668+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcad406c8f0 0x7fcad406edb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fcaec001700 tx=0x7fcaec009040 comp rx=0 tx=0).stop 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.668+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf015f3a0 msgr2=0x7fcaf01f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.668+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf015f3a0 0x7fcaf01f8fa0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009e50 tx=0x7fcae8004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.669+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 shutdown_connections 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.669+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcad406c8f0 0x7fcad406edb0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.669+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcaf015f3a0 0x7fcaf01f8fa0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.669+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaf00d06f0 0x7fcaf01502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.669+0000 7fcaf75c6640 1 --2- 172.21.15.40:0/3051844063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaf00b2850 0x7fcaf015fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.669+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 >> 172.21.15.40:0/3051844063 conn(0x7fcaf00d4f90 msgr2=0x7fcaf00d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.669+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 shutdown_connections 2024-04-26T21:44:44.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.669+0000 7fcaf75c6640 1 -- 172.21.15.40:0/3051844063 wait complete. 2024-04-26T21:44:44.679 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds0_inotable' led to DAMAGED state 2024-04-26T21:44:44.679 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 400.00000000' 2024-04-26T21:44:44.679 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:44.680 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:44.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:44.695 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:44.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:44.695 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:44.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:44.750 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:44.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.892+0000 7f6a84e7e640 1 Processor -- start 2024-04-26T21:44:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a84e7e640 1 -- start start 2024-04-26T21:44:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a84e7e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a80162810 0x7f6a80070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a84e7e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6a800b2340 0x7f6a800cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a84e7e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a800d10d0 con 0x7f6a800cd930 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a84e7e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6a800d0750 con 0x7f6a80162810 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a84e7e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6a800d16a0 con 0x7f6a800b2340 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a7e59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a80162810 0x7f6a80070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a7e59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a80162810 0x7f6a80070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34658/0 (socket says 172.21.15.40:34658) 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.893+0000 7f6a7e59c640 1 -- 172.21.15.40:0/2797721106 learned_addr learned my addr 172.21.15.40:0/2797721106 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.894+0000 7f6a7e59c640 1 -- 172.21.15.40:0/2797721106 >> v2:172.21.15.184:3300/0 conn(0x7f6a800b2340 msgr2=0x7f6a800cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.894+0000 7f6a7e59c640 1 --2- 172.21.15.40:0/2797721106 >> v2:172.21.15.184:3300/0 conn(0x7f6a800b2340 0x7f6a800cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.894+0000 7f6a7e59c640 1 -- 172.21.15.40:0/2797721106 >> v1:172.21.15.184:6789/0 conn(0x7f6a800cd930 legacy=0x7f6a800d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.894+0000 7f6a7e59c640 1 -- 172.21.15.40:0/2797721106 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a800d1370 con 0x7f6a80162810 2024-04-26T21:44:44.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.894+0000 7f6a7e59c640 1 --2- 172.21.15.40:0/2797721106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a80162810 0x7f6a80070350 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f6a800cd0a0 tx=0x7f6a7002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5b6a482dd35771 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a7dd9b640 1 -- 172.21.15.40:0/2797721106 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70037070 con 0x7f6a80162810 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a7dd9b640 1 -- 172.21.15.40:0/2797721106 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a70032440 con 0x7f6a80162810 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a7dd9b640 1 -- 172.21.15.40:0/2797721106 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70032810 con 0x7f6a80162810 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a84e7e640 1 -- 172.21.15.40:0/2797721106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a80162810 msgr2=0x7f6a80070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/2797721106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a80162810 0x7f6a80070350 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f6a800cd0a0 tx=0x7f6a7002a230 comp rx=0 tx=0).stop 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a84e7e640 1 -- 172.21.15.40:0/2797721106 shutdown_connections 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/2797721106 >> v2:172.21.15.184:3300/0 conn(0x7f6a800b2340 0x7f6a800cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/2797721106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a80162810 0x7f6a80070350 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a84e7e640 1 -- 172.21.15.40:0/2797721106 >> 172.21.15.40:0/2797721106 conn(0x7f6a800d2da0 msgr2=0x7f6a800d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a84e7e640 1 -- 172.21.15.40:0/2797721106 shutdown_connections 2024-04-26T21:44:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.895+0000 7f6a84e7e640 1 -- 172.21.15.40:0/2797721106 wait complete. 2024-04-26T21:44:44.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.896+0000 7f6a84e7e640 1 Processor -- start 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.896+0000 7f6a84e7e640 1 -- start start 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a84e7e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a800b2340 0x7f6a8014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a84e7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800cd930 0x7f6a800c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a84e7e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80162810 0x7f6a800ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a84e7e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80070890 con 0x7f6a800cd930 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a84e7e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a800d0470 con 0x7f6a80162810 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a84e7e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6a800d2240 con 0x7f6a800b2340 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800cd930 0x7f6a800c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800cd930 0x7f6a800c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34662/0 (socket says 172.21.15.40:34662) 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a77fff640 1 -- 172.21.15.40:0/1348199206 learned_addr learned my addr 172.21.15.40:0/1348199206 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a7e59c640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a800b2340 0x7f6a8014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.897+0000 7f6a7ed9d640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80162810 0x7f6a800ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a77fff640 1 -- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a800b2340 msgr2=0x7f6a8014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a77fff640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a800b2340 0x7f6a8014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a77fff640 1 -- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80162810 msgr2=0x7f6a800ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a77fff640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80162810 0x7f6a800ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a77fff640 1 -- 172.21.15.40:0/1348199206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8014fc50 con 0x7f6a800cd930 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a77fff640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800cd930 0x7f6a800c9e70 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800b440 tx=0x7f6a6800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a76ffd640 1 -- 172.21.15.40:0/1348199206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68015020 con 0x7f6a800cd930 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a76ffd640 1 -- 172.21.15.40:0/1348199206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a68011890 con 0x7f6a800cd930 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8014ff30 con 0x7f6a800cd930 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a76ffd640 1 -- 172.21.15.40:0/1348199206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68011c80 con 0x7f6a800cd930 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.898+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801504f0 con 0x7f6a800cd930 2024-04-26T21:44:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.899+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a800694f0 con 0x7f6a800cd930 2024-04-26T21:44:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.900+0000 7f6a76ffd640 1 -- 172.21.15.40:0/1348199206 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a68018020 con 0x7f6a800cd930 2024-04-26T21:44:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.900+0000 7f6a76ffd640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a6006c8f0 0x7f6a6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.900+0000 7f6a76ffd640 1 -- 172.21.15.40:0/1348199206 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8023+0+0 (secure 0 0 0) 0x7f6a6808c550 con 0x7f6a800cd930 2024-04-26T21:44:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.900+0000 7f6a7e59c640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a6006c8f0 0x7f6a6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.901+0000 7f6a7e59c640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a6006c8f0 0x7f6a6006edb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6a70034000 tx=0x7f6a7002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:44.904+0000 7f6a76ffd640 1 -- 172.21.15.40:0/1348199206 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a6801f020 con 0x7f6a800cd930 2024-04-26T21:44:45.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.130+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6a8006d6d0 con 0x7f6a800cd930 2024-04-26T21:44:45.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.583+0000 7f6a76ffd640 1 -- 172.21.15.40:0/1348199206 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v222) ==== 161+0+0 (secure 0 0 0) 0x7f6a6800d040 con 0x7f6a800cd930 2024-04-26T21:44:45.583 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a6006c8f0 msgr2=0x7f6a6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a6006c8f0 0x7f6a6006edb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6a70034000 tx=0x7f6a7002e000 comp rx=0 tx=0).stop 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800cd930 msgr2=0x7f6a800c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800cd930 0x7f6a800c9e70 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800b440 tx=0x7f6a6800b900 comp rx=0 tx=0).stop 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 shutdown_connections 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6a6006c8f0 0x7f6a6006edb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80162810 0x7f6a800ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800cd930 0x7f6a800c9e70 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 --2- 172.21.15.40:0/1348199206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a800b2340 0x7f6a8014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 >> 172.21.15.40:0/1348199206 conn(0x7f6a800d2da0 msgr2=0x7f6a800d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 shutdown_connections 2024-04-26T21:44:45.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.586+0000 7f6a84e7e640 1 -- 172.21.15.40:0/1348199206 wait complete. 2024-04-26T21:44:45.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:45.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.702+0000 7f4518e08640 1 Processor -- start 2024-04-26T21:44:45.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.702+0000 7f4518e08640 1 -- start start 2024-04-26T21:44:45.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.703+0000 7f4518e08640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f451415f3a0 0x7f45140d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:45.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.703+0000 7f4518e08640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45140d1770 0x7f45140d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:45.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.703+0000 7f4518e08640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45140d25a0 con 0x7f45140d2130 2024-04-26T21:44:45.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.703+0000 7f4518e08640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f45140d0e20 con 0x7f451415f3a0 2024-04-26T21:44:45.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.703+0000 7f4518e08640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45140b1d70 con 0x7f45140d1770 2024-04-26T21:44:45.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f451259c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f451415f3a0 0x7f45140d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:45.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f4511d9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45140d1770 0x7f45140d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:45.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f4512d9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f45140d2130 0x7f45140d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:33176/0 (socket says 172.21.15.40:33176) 2024-04-26T21:44:45.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f451259c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f451415f3a0 0x7f45140d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:33034/0 (socket says 172.21.15.40:33034) 2024-04-26T21:44:45.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f4512d9d640 1 -- 172.21.15.40:0/505765055 learned_addr learned my addr 172.21.15.40:0/505765055 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:45.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f451259c640 1 -- 172.21.15.40:0/505765055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45140d1770 msgr2=0x7f45140d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:45.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f451259c640 1 --2- 172.21.15.40:0/505765055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45140d1770 0x7f45140d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f451259c640 1 -- 172.21.15.40:0/505765055 >> v1:172.21.15.184:6789/0 conn(0x7f45140d2130 legacy=0x7f45140d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:45.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f451259c640 1 -- 172.21.15.40:0/505765055 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45140b2050 con 0x7f451415f3a0 2024-04-26T21:44:45.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.704+0000 7f4511d9b640 1 --2- 172.21.15.40:0/505765055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45140d1770 0x7f45140d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:45.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.705+0000 7f451259c640 1 --2- 172.21.15.40:0/505765055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451415f3a0 0x7f45140d06f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f450c0097a0 tx=0x7f450c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90da8e1c35e7c838 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:45.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.705+0000 7f451159a640 1 -- 172.21.15.40:0/505765055 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c037070 con 0x7f451415f3a0 2024-04-26T21:44:45.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.705+0000 7f451159a640 1 -- 172.21.15.40:0/505765055 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c032440 con 0x7f451415f3a0 2024-04-26T21:44:45.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.705+0000 7f451159a640 1 -- 172.21.15.40:0/505765055 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c032810 con 0x7f451415f3a0 2024-04-26T21:44:45.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.706+0000 7f4518e08640 1 -- 172.21.15.40:0/505765055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451415f3a0 msgr2=0x7f45140d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:45.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.706+0000 7f4518e08640 1 --2- 172.21.15.40:0/505765055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451415f3a0 0x7f45140d06f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f450c0097a0 tx=0x7f450c02a180 comp rx=0 tx=0).stop 2024-04-26T21:44:45.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.706+0000 7f4518e08640 1 -- 172.21.15.40:0/505765055 shutdown_connections 2024-04-26T21:44:45.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.706+0000 7f4518e08640 1 --2- 172.21.15.40:0/505765055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45140d1770 0x7f45140d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.706+0000 7f4518e08640 1 --2- 172.21.15.40:0/505765055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f451415f3a0 0x7f45140d06f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.706+0000 7f4518e08640 1 -- 172.21.15.40:0/505765055 >> 172.21.15.40:0/505765055 conn(0x7f45140d4f90 msgr2=0x7f45140d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:45.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.706+0000 7f4518e08640 1 -- 172.21.15.40:0/505765055 shutdown_connections 2024-04-26T21:44:45.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.706+0000 7f4518e08640 1 -- 172.21.15.40:0/505765055 wait complete. 2024-04-26T21:44:45.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.707+0000 7f4518e08640 1 Processor -- start 2024-04-26T21:44:45.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.707+0000 7f4518e08640 1 -- start start 2024-04-26T21:44:45.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4518e08640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45140d1770 0x7f4514153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:45.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4518e08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45140d2130 0x7f45141532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:45.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4518e08640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451415f3a0 0x7f45140ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:45.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4518e08640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f451415f900 con 0x7f45140d2130 2024-04-26T21:44:45.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4518e08640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f451415f780 con 0x7f45140d1770 2024-04-26T21:44:45.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4518e08640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45140b3860 con 0x7f451415f3a0 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4511d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45140d2130 0x7f45141532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4511d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45140d2130 0x7f45141532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34674/0 (socket says 172.21.15.40:34674) 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4511d9b640 1 -- 172.21.15.40:0/3398732994 learned_addr learned my addr 172.21.15.40:0/3398732994 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f451259c640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45140d1770 0x7f4514153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.708+0000 7f4512d9d640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451415f3a0 0x7f45140ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4511d9b640 1 -- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451415f3a0 msgr2=0x7f45140ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4511d9b640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451415f3a0 0x7f45140ba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4511d9b640 1 -- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45140d1770 msgr2=0x7f4514153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4511d9b640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45140d1770 0x7f4514153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4511d9b640 1 -- 172.21.15.40:0/3398732994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45140cd2c0 con 0x7f45140d2130 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4511d9b640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45140d2130 0x7f45141532e0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f450800ebe0 tx=0x7f450800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4502ffd640 1 -- 172.21.15.40:0/3398732994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508017070 con 0x7f45140d2130 2024-04-26T21:44:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4502ffd640 1 -- 172.21.15.40:0/3398732994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450800cc30 con 0x7f45140d2130 2024-04-26T21:44:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45140cd5a0 con 0x7f45140d2130 2024-04-26T21:44:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4502ffd640 1 -- 172.21.15.40:0/3398732994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508013470 con 0x7f45140d2130 2024-04-26T21:44:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.709+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45140cdb60 con 0x7f45140d2130 2024-04-26T21:44:45.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.710+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45140694f0 con 0x7f45140d2130 2024-04-26T21:44:45.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.711+0000 7f4502ffd640 1 -- 172.21.15.40:0/3398732994 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f450801a070 con 0x7f45140d2130 2024-04-26T21:44:45.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.711+0000 7f4502ffd640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44f406c8f0 0x7f44f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:45.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.711+0000 7f4502ffd640 1 -- 172.21.15.40:0/3398732994 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8023+0+0 (secure 0 0 0) 0x7f450808c6b0 con 0x7f45140d2130 2024-04-26T21:44:45.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.711+0000 7f451259c640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44f406c8f0 0x7f44f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:45.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.712+0000 7f451259c640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44f406c8f0 0x7f44f406edb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f450c034000 tx=0x7f450c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:45.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.716+0000 7f4502ffd640 1 -- 172.21.15.40:0/3398732994 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4508059b80 con 0x7f45140d2130 2024-04-26T21:44:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:45.975+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f45140b6860 con 0x7f45140d2130 2024-04-26T21:44:46.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.596+0000 7f4502ffd640 1 -- 172.21.15.40:0/3398732994 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v223) ==== 101+0+0 (secure 0 0 0) 0x7f450805de80 con 0x7f45140d2130 2024-04-26T21:44:46.596 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44f406c8f0 msgr2=0x7f44f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44f406c8f0 0x7f44f406edb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f450c034000 tx=0x7f450c02e000 comp rx=0 tx=0).stop 2024-04-26T21:44:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45140d2130 msgr2=0x7f45141532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45140d2130 0x7f45141532e0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f450800ebe0 tx=0x7f450800c520 comp rx=0 tx=0).stop 2024-04-26T21:44:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 shutdown_connections 2024-04-26T21:44:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f44f406c8f0 0x7f44f406edb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f451415f3a0 0x7f45140ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45140d2130 0x7f45141532e0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 --2- 172.21.15.40:0/3398732994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45140d1770 0x7f4514153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 >> 172.21.15.40:0/3398732994 conn(0x7f45140d4f90 msgr2=0x7f451406cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 shutdown_connections 2024-04-26T21:44:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.599+0000 7f4518e08640 1 -- 172.21.15.40:0/3398732994 wait complete. 2024-04-26T21:44:46.612 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:46.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.658+0000 7f9b37c0cf40 1 Processor -- start 2024-04-26T21:44:46.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.658+0000 7f9b37c0cf40 1 -- start start 2024-04-26T21:44:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b37c0cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e43d670 0x563b9e4b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b37c0cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e4b3d60 0x563b9e4d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b37c0cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4d1610 0x563b9e420050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b37c0cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563b9e39de40 con 0x563b9e4d1610 2024-04-26T21:44:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b37c0cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563b9e3a1e80 con 0x563b9e4b3d60 2024-04-26T21:44:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b37c0cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563b9e4b4140 con 0x563b9e43d670 2024-04-26T21:44:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4d1610 0x563b9e420050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4d1610 0x563b9e420050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34682/0 (socket says 172.21.15.40:34682) 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 -- 172.21.15.40:0/4244643425 learned_addr learned my addr 172.21.15.40:0/4244643425 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b34240640 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e4b3d60 0x563b9e4d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b34a41640 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e43d670 0x563b9e4b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 -- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e43d670 msgr2=0x563b9e4b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e43d670 0x563b9e4b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 -- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e4b3d60 msgr2=0x563b9e4d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e4b3d60 0x563b9e4d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 -- 172.21.15.40:0/4244643425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b9e4d1f80 con 0x563b9e4d1610 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b34a41640 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e43d670 0x563b9e4b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b34240640 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e4b3d60 0x563b9e4d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.659+0000 7f9b35242640 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4d1610 0x563b9e420050 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c001710 tx=0x7f9b2c001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12bd81466425a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b3323e640 1 -- 172.21.15.40:0/4244643425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c016070 con 0x563b9e4d1610 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b3323e640 1 -- 172.21.15.40:0/4244643425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2c00f730 con 0x563b9e4d1610 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b3323e640 1 -- 172.21.15.40:0/4244643425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00fb20 con 0x563b9e4d1610 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/4244643425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4d1610 msgr2=0x563b9e420050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4d1610 0x563b9e420050 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c001710 tx=0x7f9b2c001bd0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/4244643425 shutdown_connections 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4d1610 0x563b9e420050 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e4b3d60 0x563b9e4d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/4244643425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e43d670 0x563b9e4b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/4244643425 >> 172.21.15.40:0/4244643425 conn(0x563b9e3f4ab0 msgr2=0x563b9e421820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/4244643425 shutdown_connections 2024-04-26T21:44:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.660+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/4244643425 wait complete. 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.661+0000 7f9b37c0cf40 1 Processor -- start 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.661+0000 7f9b37c0cf40 1 -- start start 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b37c0cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e43d670 0x563b9e4c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b37c0cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4b3d60 0x563b9e564440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b37c0cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e4d1610 0x563b9e4b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b37c0cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563b9e420590 con 0x563b9e4b3d60 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b37c0cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563b9e4d1250 con 0x563b9e43d670 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b37c0cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563b9e3f5200 con 0x563b9e4d1610 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4b3d60 0x563b9e564440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4b3d60 0x563b9e564440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34698/0 (socket says 172.21.15.40:34698) 2024-04-26T21:44:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 learned_addr learned my addr 172.21.15.40:0/941662590 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34a41640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e43d670 0x563b9e4c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b35242640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e4d1610 0x563b9e4b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e4d1610 msgr2=0x563b9e4b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563b9e4d1610 0x563b9e4b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e43d670 msgr2=0x563b9e4c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b9e43d670 0x563b9e4c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b9e4d04c0 con 0x563b9e4b3d60 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b34240640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4b3d60 0x563b9e564440 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800a4f0 tx=0x7f9b28002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b3223c640 1 -- 172.21.15.40:0/941662590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28059040 con 0x563b9e4b3d60 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b3223c640 1 -- 172.21.15.40:0/941662590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2800a220 con 0x563b9e4b3d60 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.662+0000 7f9b3223c640 1 -- 172.21.15.40:0/941662590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28054c30 con 0x563b9e4b3d60 2024-04-26T21:44:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.663+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563b9e4d07a0 con 0x563b9e4b3d60 2024-04-26T21:44:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.663+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b9e4a8ad0 con 0x563b9e4b3d60 2024-04-26T21:44:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.664+0000 7f9b3223c640 1 -- 172.21.15.40:0/941662590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b2805c050 con 0x563b9e4b3d60 2024-04-26T21:44:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.664+0000 7f9b3223c640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b2006b2d0 0x7f9b2006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.665+0000 7f9b3223c640 1 -- 172.21.15.40:0/941662590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8023+0+0 (secure 0 0 0) 0x7f9b280ceab0 con 0x563b9e4b3d60 2024-04-26T21:44:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.665+0000 7f9b34a41640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b2006b2d0 0x7f9b2006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.667 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.665+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563b9e4c6b70 0x563b9e4a8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.665+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e4d0050 con 0x563b9e4c6b70 2024-04-26T21:44:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.665+0000 7f9b35242640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563b9e4c6b70 0x563b9e4a8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.665+0000 7f9b34a41640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b2006b2d0 0x7f9b2006d790 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9b240011e0 tx=0x7f9b24001e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.665+0000 7f9b35242640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563b9e4c6b70 0x563b9e4a8ff0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.667+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v116'629 uv603 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.667+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e4a9530 con 0x563b9e4c6b70 2024-04-26T21:44:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.668+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v116'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.668+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e4a9900 con 0x563b9e4c6b70 2024-04-26T21:44:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.669+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v116'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.669 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.669+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571a50 con 0x563b9e4c6b70 2024-04-26T21:44:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.670+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v116'632 uv632 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.670+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571df0 con 0x563b9e4c6b70 2024-04-26T21:44:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.671+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v116'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.671+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570a40 con 0x563b9e4c6b70 2024-04-26T21:44:46.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.672+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v116'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.672+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570e00 con 0x563b9e4c6b70 2024-04-26T21:44:46.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.674+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v116'635 uv609 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.674+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e570e00 con 0x563b9e4c6b70 2024-04-26T21:44:46.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.675+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v116'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.675+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.676+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v116'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.675 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:46.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.676+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.677+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v116'638 uv638 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.677+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.678+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v116'639 uv639 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.678+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.679+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v116'640 uv640 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.679+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.680+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v116'641 uv616 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.680+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.681+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v116'642 uv642 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.681+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.682+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v116'643 uv643 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.682 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:46.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.682+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.683+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v116'644 uv644 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.683+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.684+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v116'645 uv645 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.684+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e570de0 con 0x563b9e4c6b70 2024-04-26T21:44:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.685+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v116'646 uv646 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.685+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571650 con 0x563b9e4c6b70 2024-04-26T21:44:46.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.686+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v116'647 uv647 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.686+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571650 con 0x563b9e4c6b70 2024-04-26T21:44:46.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.687+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v116'648 uv622 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.687+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.688+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v116'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.688+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.689+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v116'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.689 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.689+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.690+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v116'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.690+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.691+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v116'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.691+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.692+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v116'653 uv653 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.692+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.693+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v116'654 uv628 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.693+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.694+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v116'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.694+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.695+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v116'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.695+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.696+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v116'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.696+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.697+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v116'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.697+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e4c6b70 2024-04-26T21:44:46.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.698+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v116'659 uv659 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e4c6b70 2024-04-26T21:44:46.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.698+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b9e5726a0 0x563b9e574a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.698+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e571540 con 0x563b9e5726a0 2024-04-26T21:44:46.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.698+0000 7f9b34a41640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b9e5726a0 0x563b9e574a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.698+0000 7f9b34a41640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b9e5726a0 0x563b9e574a90 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.700+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v116'463 uv446 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.700+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e574fd0 con 0x563b9e5726a0 2024-04-26T21:44:46.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.702+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v116'464 uv464 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.702+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e575370 con 0x563b9e5726a0 2024-04-26T21:44:46.702 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:46.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.703+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v116'465 uv465 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.703+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e575bc0 con 0x563b9e5726a0 2024-04-26T21:44:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.704+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v116'466 uv466 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.704+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e575f60 con 0x563b9e5726a0 2024-04-26T21:44:46.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.706+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v116'467 uv467 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.706+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576300 con 0x563b9e5726a0 2024-04-26T21:44:46.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.707+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v116'468 uv451 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.707+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e5766a0 con 0x563b9e5726a0 2024-04-26T21:44:46.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.708+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v116'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.708+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.709 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:46.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.709+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v116'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.709+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.710+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v116'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.710+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.712+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v116'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.712+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.713+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v116'473 uv456 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.713+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.714+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v116'474 uv474 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.714+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.715 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:46.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.715+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v116'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.715+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57b100 con 0x563b9e5726a0 2024-04-26T21:44:46.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.717+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v116'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.717+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57b100 con 0x563b9e5726a0 2024-04-26T21:44:46.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.718+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v116'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.718+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.720+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v116'478 uv462 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.720+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.721+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v116'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.721+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.721 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:46.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.722+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v116'480 uv480 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.722+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e577a50 con 0x563b9e5726a0 2024-04-26T21:44:46.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.723+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v116'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.723+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e577a50 con 0x563b9e5726a0 2024-04-26T21:44:46.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.724+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v116'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.724+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e576a40 con 0x563b9e5726a0 2024-04-26T21:44:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.726+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v116'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e5726a0 2024-04-26T21:44:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.726+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563b9e576f50 0x563b9e579340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.726+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e579880 con 0x563b9e576f50 2024-04-26T21:44:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.726+0000 7f9b34240640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563b9e576f50 0x563b9e579340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.726+0000 7f9b34240640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563b9e576f50 0x563b9e579340 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.728+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v116'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.728+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e579ce0 con 0x563b9e576f50 2024-04-26T21:44:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.729+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v116'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.729+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57a1c0 con 0x563b9e576f50 2024-04-26T21:44:46.729 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.730+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v116'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.730+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57ba70 con 0x563b9e576f50 2024-04-26T21:44:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.731+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v116'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.731+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57be10 con 0x563b9e576f50 2024-04-26T21:44:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.732+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v116'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.732+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57aa60 con 0x563b9e576f50 2024-04-26T21:44:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.733+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v116'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.733+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563b9e57b360 0x563b9e57e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.733+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57e700 con 0x563b9e57b360 2024-04-26T21:44:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.733+0000 7f9b35242640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563b9e57b360 0x563b9e57e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.734+0000 7f9b35242640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563b9e57b360 0x563b9e57e1c0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.735+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v116'96 uv95 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.735+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e57e700 con 0x563b9e57b360 2024-04-26T21:44:46.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.736+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v116'97 uv97 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.736+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57e780 con 0x563b9e57b360 2024-04-26T21:44:46.737 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.737+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v116'98 uv98 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.738+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e57eb70 con 0x563b9e57b360 2024-04-26T21:44:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.739+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v116'99 uv99 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.739+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563b9e57f080 0x563b9e581470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.739+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581a40 con 0x563b9e57f080 2024-04-26T21:44:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.739+0000 7f9b34a41640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563b9e57f080 0x563b9e581470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.739+0000 7f9b34a41640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563b9e57f080 0x563b9e581470 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.741+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v116'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.741+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e581ac0 con 0x563b9e57f080 2024-04-26T21:44:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.742+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v116'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.742+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.743+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v116'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.743 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.743+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e582ec0 con 0x563b9e57f080 2024-04-26T21:44:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.745+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v116'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.745+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e582ec0 con 0x563b9e57f080 2024-04-26T21:44:46.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.746+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v116'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.746+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.747+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v116'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.747+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.749+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v116'227 uv221 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.749+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.750+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v116'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.750+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.751+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v116'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.751 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:46.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.751+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e582ec0 con 0x563b9e57b360 2024-04-26T21:44:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.752+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v116'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.752+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e582ec0 con 0x563b9e57b360 2024-04-26T21:44:46.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.753+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v116'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.753+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.754+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v116'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.755+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.756+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v116'233 uv226 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.756+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.757+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v116'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.757+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.758 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:46.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.758+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v116'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.758+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.759+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v116'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.759+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.761+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v116'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.761+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.762+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v116'102 uv101 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.762+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.763+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v116'103 uv103 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.763+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.764+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v116'104 uv104 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.764 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:46.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.764+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.765+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v116'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.765+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57b360 2024-04-26T21:44:46.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.766+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v116'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b2c00f070 con 0x563b9e57b360 2024-04-26T21:44:46.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.766+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.767+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v116'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.767+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.768+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v116'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.768+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.769 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:46.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.769+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v116'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.769+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e582f00 con 0x563b9e57f080 2024-04-26T21:44:46.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.770+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v116'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.770+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e582ec0 con 0x563b9e57f080 2024-04-26T21:44:46.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.771+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v116'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.771+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.772+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v116'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.772+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.773+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v116'203 uv202 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.773+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.774+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v116'204 uv204 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.774+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.774 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.775+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v116'205 uv205 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.775+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e5822a0 con 0x563b9e57f080 2024-04-26T21:44:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.776+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v116'206 uv206 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.776+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581ed0 con 0x563b9e57f080 2024-04-26T21:44:46.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.777+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v116'207 uv207 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.777+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581ed0 con 0x563b9e57f080 2024-04-26T21:44:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.778+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v116'208 uv201 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.778+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e581ed0 con 0x563b9e57f080 2024-04-26T21:44:46.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.779+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v116'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.779+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581eb0 con 0x563b9e57f080 2024-04-26T21:44:46.779 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:46.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.780+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v116'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.780+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e581ef0 con 0x563b9e57f080 2024-04-26T21:44:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.780+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v116'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.781+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e582340 con 0x563b9e57f080 2024-04-26T21:44:46.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.782+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v116'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.782+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563b9e5829d0 0x563b9e584dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.782+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585300 con 0x563b9e5829d0 2024-04-26T21:44:46.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.782+0000 7f9b34240640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563b9e5829d0 0x563b9e584dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.782+0000 7f9b34240640 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563b9e5829d0 0x563b9e584dc0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.783+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v116'91 uv90 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e5829d0 2024-04-26T21:44:46.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.783+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e5856a0 con 0x563b9e5829d0 2024-04-26T21:44:46.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.784+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v116'92 uv92 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e5829d0 2024-04-26T21:44:46.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.784+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e5856a0 con 0x563b9e5829d0 2024-04-26T21:44:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.785+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v116'93 uv93 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e5829d0 2024-04-26T21:44:46.785 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.785+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585780 con 0x563b9e5829d0 2024-04-26T21:44:46.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.786+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v116'94 uv94 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e5829d0 2024-04-26T21:44:46.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.786+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585780 con 0x563b9e57f080 2024-04-26T21:44:46.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.787+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v116'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.787+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e585740 con 0x563b9e57f080 2024-04-26T21:44:46.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.788+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v116'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.789+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585740 con 0x563b9e57f080 2024-04-26T21:44:46.789 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:46.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.790+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v116'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.790+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585740 con 0x563b9e57f080 2024-04-26T21:44:46.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.791+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v116'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.791+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585740 con 0x563b9e57f080 2024-04-26T21:44:46.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.792+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v116'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.792+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585740 con 0x563b9e57f080 2024-04-26T21:44:46.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.793+0000 7f9b34a41640 1 -- 172.21.15.40:0/941662590 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v116'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b2400ea50 con 0x563b9e57f080 2024-04-26T21:44:46.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.793+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585740 con 0x563b9e576f50 2024-04-26T21:44:46.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.794+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v116'104 uv103 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.794+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e585720 con 0x563b9e576f50 2024-04-26T21:44:46.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.795+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v116'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.795+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e5856a0 con 0x563b9e576f50 2024-04-26T21:44:46.796 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.796+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v116'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.796+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585b90 con 0x563b9e576f50 2024-04-26T21:44:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.797+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v116'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.797+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585b90 con 0x563b9e576f50 2024-04-26T21:44:46.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.798+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v116'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.799+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585b90 con 0x563b9e576f50 2024-04-26T21:44:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.800+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v116'253 uv245 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.800+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e585b90 con 0x563b9e576f50 2024-04-26T21:44:46.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.801+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v116'254 uv254 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.801+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585b90 con 0x563b9e576f50 2024-04-26T21:44:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.802+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v116'255 uv255 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.802 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.802+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585b90 con 0x563b9e576f50 2024-04-26T21:44:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.804+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v116'256 uv256 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.804+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585b90 con 0x563b9e576f50 2024-04-26T21:44:46.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.805+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v116'257 uv257 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.805+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585b90 con 0x563b9e576f50 2024-04-26T21:44:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.806+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v116'258 uv252 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.806+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x563b9e585950 con 0x563b9e576f50 2024-04-26T21:44:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.807+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v116'259 uv259 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.807+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585950 con 0x563b9e576f50 2024-04-26T21:44:46.807 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.808+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v116'260 uv260 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.808+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585950 con 0x563b9e576f50 2024-04-26T21:44:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.809+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v116'261 uv261 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.809+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e5858a0 con 0x563b9e576f50 2024-04-26T21:44:46.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.810+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v116'262 uv262 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.810+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585880 con 0x563b9e576f50 2024-04-26T21:44:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.811+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v116'263 uv263 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.811+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b9e585d20 con 0x563b9e576f50 2024-04-26T21:44:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b34240640 1 -- 172.21.15.40:0/941662590 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v116'264 uv264 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b28098680 con 0x563b9e576f50 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b9e5726a0 msgr2=0x563b9e574a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563b9e5726a0 0x563b9e574a90 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563b9e4c6b70 msgr2=0x563b9e4a8ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563b9e4c6b70 0x563b9e4a8ff0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563b9e57b360 msgr2=0x563b9e57e1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563b9e57b360 0x563b9e57e1c0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563b9e5829d0 msgr2=0x563b9e584dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563b9e5829d0 0x563b9e584dc0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563b9e576f50 msgr2=0x563b9e579340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563b9e576f50 0x563b9e579340 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563b9e57f080 msgr2=0x563b9e581470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563b9e57f080 0x563b9e581470 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 reap_dead start 2024-04-26T21:44:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b35242640 1 -- 172.21.15.40:0/941662590 reap_dead start 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b2006b2d0 msgr2=0x7f9b2006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b2006b2d0 0x7f9b2006d790 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9b240011e0 tx=0x7f9b24001e90 comp rx=0 tx=0).stop 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4b3d60 msgr2=0x563b9e564440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4b3d60 0x563b9e564440 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800a4f0 tx=0x7f9b28002fb0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 shutdown_connections 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b2006b2d0 0x7f9b2006d790 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 --2- 172.21.15.40:0/941662590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563b9e4b3d60 0x563b9e564440 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 >> 172.21.15.40:0/941662590 conn(0x563b9e3f4ab0 msgr2=0x563b9e4b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 shutdown_connections 2024-04-26T21:44:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.812+0000 7f9b37c0cf40 1 -- 172.21.15.40:0/941662590 wait complete. 2024-04-26T21:44:46.818 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 400.00000000 - 2024-04-26T21:44:46.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f651960df40 1 Processor -- start 2024-04-26T21:44:46.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f651960df40 1 -- start start 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f651960df40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a823c1480 0x563a82469560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f651960df40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82455fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f651960df40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a82466820 0x563a82456520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f651960df40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563a82466c00 con 0x563a823da510 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f651960df40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a823c18d0 con 0x563a82466820 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f651960df40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563a82466e60 con 0x563a823c1480 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.901+0000 7f6515bf6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82455fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6515bf6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82455fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34710/0 (socket says 172.21.15.40:34710) 2024-04-26T21:44:46.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6515bf6640 1 -- 172.21.15.40:0/1420368644 learned_addr learned my addr 172.21.15.40:0/1420368644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f65163f7640 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a823c1480 0x563a82469560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6516bf8640 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a82466820 0x563a82456520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6515bf6640 1 -- 172.21.15.40:0/1420368644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a823c1480 msgr2=0x563a82469560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6515bf6640 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a823c1480 0x563a82469560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6515bf6640 1 -- 172.21.15.40:0/1420368644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a82466820 msgr2=0x563a82456520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6515bf6640 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a82466820 0x563a82456520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6515bf6640 1 -- 172.21.15.40:0/1420368644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a822bbcb0 con 0x563a823da510 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6515bf6640 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82455fe0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f650000c810 tx=0x7f650000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d370e0efe1bf3ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6514bf4640 1 -- 172.21.15.40:0/1420368644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500019070 con 0x563a823da510 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6514bf4640 1 -- 172.21.15.40:0/1420368644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650000adb0 con 0x563a823da510 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.902+0000 7f6514bf4640 1 -- 172.21.15.40:0/1420368644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500008670 con 0x563a823da510 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 -- 172.21.15.40:0/1420368644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 msgr2=0x563a82455fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82455fe0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f650000c810 tx=0x7f650000ccd0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 -- 172.21.15.40:0/1420368644 shutdown_connections 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a82466820 0x563a82456520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82455fe0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 --2- 172.21.15.40:0/1420368644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a823c1480 0x563a82469560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 -- 172.21.15.40:0/1420368644 >> 172.21.15.40:0/1420368644 conn(0x563a823bb8a0 msgr2=0x563a82393860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 -- 172.21.15.40:0/1420368644 shutdown_connections 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 -- 172.21.15.40:0/1420368644 wait complete. 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 Processor -- start 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.903+0000 7f651960df40 1 -- start start 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f651960df40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a823c1480 0x563a8245d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f651960df40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82458600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f651960df40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a82466820 0x563a8245a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f651960df40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563a82462720 con 0x563a823da510 2024-04-26T21:44:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f651960df40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a82462a00 con 0x563a823c1480 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f651960df40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563a824683e0 con 0x563a82466820 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f6515bf6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82458600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f65163f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a823c1480 0x563a8245d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f6515bf6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82458600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34722/0 (socket says 172.21.15.40:34722) 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f6515bf6640 1 -- 172.21.15.40:0/134160318 learned_addr learned my addr 172.21.15.40:0/134160318 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f65163f7640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a823c1480 0x563a8245d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:33086/0 (socket says 172.21.15.40:33086) 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.904+0000 7f6516bf8640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a82466820 0x563a8245a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f6515bf6640 1 -- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a82466820 msgr2=0x563a8245a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f6515bf6640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a82466820 0x563a8245a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f6515bf6640 1 -- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a823c1480 msgr2=0x563a8245d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f6515bf6640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a823c1480 0x563a8245d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f6515bf6640 1 -- 172.21.15.40:0/134160318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a823bcbc0 con 0x563a823da510 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f65163f7640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a823c1480 0x563a8245d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:46.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f6515bf6640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82458600 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f6500000c00 tx=0x7f6500003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f650f7fe640 1 -- 172.21.15.40:0/134160318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500019070 con 0x563a823da510 2024-04-26T21:44:46.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f650f7fe640 1 -- 172.21.15.40:0/134160318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6500023070 con 0x563a823da510 2024-04-26T21:44:46.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563a82455d20 con 0x563a823da510 2024-04-26T21:44:46.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f650f7fe640 1 -- 172.21.15.40:0/134160318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000c240 con 0x563a823da510 2024-04-26T21:44:46.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.905+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a82455fc0 con 0x563a823da510 2024-04-26T21:44:46.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.906+0000 7f650f7fe640 1 -- 172.21.15.40:0/134160318 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6500028460 con 0x563a823da510 2024-04-26T21:44:46.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.907+0000 7f650f7fe640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f64f806b300 0x7f64f806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.907+0000 7f650f7fe640 1 -- 172.21.15.40:0/134160318 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8023+0+0 (secure 0 0 0) 0x7f650008cfe0 con 0x563a823da510 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.907+0000 7f651960df40 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a82458b40 0x563a82444600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.907+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563a8245abc0 con 0x563a82458b40 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.907+0000 7f65163f7640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f64f806b300 0x7f64f806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.907+0000 7f6516bf8640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a82458b40 0x563a82444600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.908+0000 7f6516bf8640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a82458b40 0x563a82444600 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.908+0000 7f65163f7640 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f64f806b300 0x7f64f806d7c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6510057000 tx=0x7f6510052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f6516bf8640 1 -- 172.21.15.40:0/134160318 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 400.00000000 [writefull 0~80] v116'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f650800da00 con 0x563a82458b40 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a82458b40 msgr2=0x563a82444600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a82458b40 0x563a82444600 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f64f806b300 msgr2=0x7f64f806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f64f806b300 0x7f64f806d7c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6510057000 tx=0x7f6510052000 comp rx=0 tx=0).stop 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 msgr2=0x563a82458600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 --2- 172.21.15.40:0/134160318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a823da510 0x563a82458600 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f6500000c00 tx=0x7f6500003020 comp rx=0 tx=0).stop 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f6516bf8640 1 -- 172.21.15.40:0/134160318 reap_dead start 2024-04-26T21:44:46.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 shutdown_connections 2024-04-26T21:44:46.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 >> 172.21.15.40:0/134160318 conn(0x563a823bb8a0 msgr2=0x563a82465b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:46.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 shutdown_connections 2024-04-26T21:44:46.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:46.909+0000 7f651960df40 1 -- 172.21.15.40:0/134160318 wait complete. 2024-04-26T21:44:46.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:47.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.055+0000 7f88425af640 1 Processor -- start 2024-04-26T21:44:47.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.055+0000 7f88425af640 1 -- start start 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f88425af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f883c06fc40 0x7f883c073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f88425af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883c156840 con 0x7f883c156460 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f88425af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883c0cbf20 con 0x7f883c06f790 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f88425af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f883c0cc200 con 0x7f883c06fc40 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f883bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f883c156460 0x7f883c0cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39612/0 (socket says 172.21.15.40:39612) 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f883bfff640 1 -- 172.21.15.40:0/3973413628 learned_addr learned my addr 172.21.15.40:0/3973413628 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f8840b4c640 1 --2- 172.21.15.40:0/3973413628 >> v2:172.21.15.184:3300/0 conn(0x7f883c06fc40 0x7f883c073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112079650 0 0) 0x7f883c156840 con 0x7f883c156460 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.056+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8828005730 con 0x7f883c156460 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1384942879 0 0) 0x7f8828005730 con 0x7f883c156460 2024-04-26T21:44:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 >> v2:172.21.15.184:3300/0 conn(0x7f883c06fc40 msgr2=0x7f883c073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 --2- 172.21.15.40:0/3973413628 >> v2:172.21.15.184:3300/0 conn(0x7f883c06fc40 0x7f883c073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 >> v1:172.21.15.184:6789/0 conn(0x7f883c06f790 legacy=0x7f883c072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c0cc4e0 con 0x7f883c156460 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f8824001670 con 0x7f883c156460 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 >> v1:172.21.15.40:6789/0 conn(0x7f883c156460 legacy=0x7f883c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8828005bc0 0x7f8828005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8828006a10 0x7f8828006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88280073d0 0x7f882800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f883c0cc200 con 0x7f88280073d0 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f882800e270 con 0x7f8828006a10 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f882800e470 con 0x7f8828005bc0 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f8840b4c640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88280073d0 0x7f882800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883b7fe640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8828006a10 0x7f8828006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f883bfff640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8828005bc0 0x7f8828005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f8840b4c640 1 -- 172.21.15.40:0/3973413628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8828005bc0 msgr2=0x7f8828005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f8840b4c640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8828005bc0 0x7f8828005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f8840b4c640 1 -- 172.21.15.40:0/3973413628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8828006a10 msgr2=0x7f8828006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f8840b4c640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8828006a10 0x7f8828006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.057+0000 7f8840b4c640 1 -- 172.21.15.40:0/3973413628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8828005730 con 0x7f88280073d0 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.058+0000 7f8840b4c640 1 -- 172.21.15.40:0/3973413628 reap_dead start 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.058+0000 7f8840b4c640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88280073d0 0x7f882800dbd0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f8834009fc0 tx=0x7f8828007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d647d7a0b7673d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.058+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834035020 con 0x7f88280073d0 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.058+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8834003bd0 con 0x7f88280073d0 2024-04-26T21:44:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.058+0000 7f883affd640 1 -- 172.21.15.40:0/3973413628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834009c10 con 0x7f88280073d0 2024-04-26T21:44:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.059+0000 7f88425af640 1 -- 172.21.15.40:0/3973413628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88280073d0 msgr2=0x7f882800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.059+0000 7f88425af640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88280073d0 0x7f882800dbd0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f8834009fc0 tx=0x7f8828007a40 comp rx=0 tx=0).stop 2024-04-26T21:44:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.059+0000 7f88425af640 1 -- 172.21.15.40:0/3973413628 shutdown_connections 2024-04-26T21:44:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.059+0000 7f88425af640 1 --2- 172.21.15.40:0/3973413628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88280073d0 0x7f882800dbd0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.059+0000 7f88425af640 1 -- 172.21.15.40:0/3973413628 >> 172.21.15.40:0/3973413628 conn(0x7f883c0d0be0 msgr2=0x7f883c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.059+0000 7f88425af640 1 -- 172.21.15.40:0/3973413628 shutdown_connections 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.059+0000 7f88425af640 1 -- 172.21.15.40:0/3973413628 wait complete. 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.060+0000 7f88425af640 1 Processor -- start 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.060+0000 7f88425af640 1 -- start start 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f88425af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88280073d0 0x7f883c14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f88425af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c0c9030 0x7f883c153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f88425af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f883c1535d0 0x7f883c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f88425af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f883c070050 con 0x7f883c0c9030 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f88425af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f883c072700 con 0x7f883c1535d0 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f88425af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f883c06e260 con 0x7f88280073d0 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c0c9030 0x7f883c153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c0c9030 0x7f883c153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34730/0 (socket says 172.21.15.40:34730) 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 -- 172.21.15.40:0/2775016565 learned_addr learned my addr 172.21.15.40:0/2775016565 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f8840b4c640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f883c1535d0 0x7f883c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883bfff640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88280073d0 0x7f883c14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 -- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88280073d0 msgr2=0x7f883c14e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88280073d0 0x7f883c14e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 -- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f883c1535d0 msgr2=0x7f883c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f883c1535d0 0x7f883c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 -- 172.21.15.40:0/2775016565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c0c7cb0 con 0x7f883c0c9030 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883bfff640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88280073d0 0x7f883c14e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f883b7fe640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c0c9030 0x7f883c153090 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f882c00f620 tx=0x7f882c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f8838ff9640 1 -- 172.21.15.40:0/2775016565 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c0052e0 con 0x7f883c0c9030 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f8838ff9640 1 -- 172.21.15.40:0/2775016565 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882c018070 con 0x7f883c0c9030 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.061+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c0c7f90 con 0x7f883c0c9030 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.062+0000 7f8838ff9640 1 -- 172.21.15.40:0/2775016565 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c003810 con 0x7f883c0c9030 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.062+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c0c8550 con 0x7f883c0c9030 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.062+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883c0694f0 con 0x7f883c0c9030 2024-04-26T21:44:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.063+0000 7f8838ff9640 1 -- 172.21.15.40:0/2775016565 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f882c0039b0 con 0x7f883c0c9030 2024-04-26T21:44:47.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.063+0000 7f8838ff9640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f881c06c920 0x7f881c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.064+0000 7f8838ff9640 1 -- 172.21.15.40:0/2775016565 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8023+0+0 (secure 0 0 0) 0x7f882c08c040 con 0x7f883c0c9030 2024-04-26T21:44:47.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.064+0000 7f883bfff640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f881c06c920 0x7f881c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.064+0000 7f883bfff640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f881c06c920 0x7f881c06ede0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8824001470 tx=0x7f8824009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.068+0000 7f8838ff9640 1 -- 172.21.15.40:0/2775016565 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f882c059bf0 con 0x7f883c0c9030 2024-04-26T21:44:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.295+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f883c152180 con 0x7f883c0c9030 2024-04-26T21:44:47.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.598+0000 7f8838ff9640 1 -- 172.21.15.40:0/2775016565 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v224) ==== 162+0+0 (secure 0 0 0) 0x7f882c0594f0 con 0x7f883c0c9030 2024-04-26T21:44:47.598 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f881c06c920 msgr2=0x7f881c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f881c06c920 0x7f881c06ede0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8824001470 tx=0x7f8824009040 comp rx=0 tx=0).stop 2024-04-26T21:44:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c0c9030 msgr2=0x7f883c153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c0c9030 0x7f883c153090 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f882c00f620 tx=0x7f882c00fae0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 shutdown_connections 2024-04-26T21:44:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f881c06c920 0x7f881c06ede0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f883c1535d0 0x7f883c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c0c9030 0x7f883c153090 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 --2- 172.21.15.40:0/2775016565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88280073d0 0x7f883c14e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 >> 172.21.15.40:0/2775016565 conn(0x7f883c0d0be0 msgr2=0x7f883c15e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 shutdown_connections 2024-04-26T21:44:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.600+0000 7f88425af640 1 -- 172.21.15.40:0/2775016565 wait complete. 2024-04-26T21:44:47.607 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:47.607+0000 7f589adb6640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-04-26T21:44:47.607 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:47.608+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:47.607 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:47.608+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:47.613 INFO:tasks.ceph.mds.c.smithi040.stderr: -28> 2024-04-26T21:44:47.607+0000 7f589adb6640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-04-26T21:44:47.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:47.638 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:47.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.717+0000 7ff903780640 1 Processor -- start 2024-04-26T21:44:47.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff903780640 1 -- start start 2024-04-26T21:44:47.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff903780640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc162810 0x7ff8fc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff903780640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8fc0d1b30 0x7ff8fc0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff903780640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8fc0d10d0 con 0x7ff8fc0b2340 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff903780640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff8fc0d0750 con 0x7ff8fc0d1b30 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff903780640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff8fc0d16a0 con 0x7ff8fc162810 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff901d1d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8fc0d1b30 0x7ff8fc0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff900d1b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff8fc0b2340 0x7ff8fc0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39628/0 (socket says 172.21.15.40:39628) 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.718+0000 7ff900d1b640 1 -- 172.21.15.40:0/2815571215 learned_addr learned my addr 172.21.15.40:0/2815571215 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.719+0000 7ff90151c640 1 --2- 172.21.15.40:0/2815571215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc162810 0x7ff8fc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.719+0000 7ff901d1d640 1 -- 172.21.15.40:0/2815571215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc162810 msgr2=0x7ff8fc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.719+0000 7ff901d1d640 1 --2- 172.21.15.40:0/2815571215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc162810 0x7ff8fc070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.719+0000 7ff901d1d640 1 -- 172.21.15.40:0/2815571215 >> v1:172.21.15.40:6789/0 conn(0x7ff8fc0b2340 legacy=0x7ff8fc0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.719+0000 7ff901d1d640 1 -- 172.21.15.40:0/2815571215 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc0d1370 con 0x7ff8fc0d1b30 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.719+0000 7ff8f3fff640 1 -- 172.21.15.40:0/2815571215 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 251285384 0 0) 0x7ff8fc0d10d0 con 0x7ff8fc0b2340 2024-04-26T21:44:47.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff901d1d640 1 --2- 172.21.15.40:0/2815571215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc0d1b30 0x7ff8fc0d1fc0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009d90 tx=0x7ff8f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ec42b5a8708c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff8f3fff640 1 -- 172.21.15.40:0/2815571215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4007740 con 0x7ff8fc0d1b30 2024-04-26T21:44:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff8f3fff640 1 -- 172.21.15.40:0/2815571215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f40078e0 con 0x7ff8fc0d1b30 2024-04-26T21:44:47.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff8f3fff640 1 -- 172.21.15.40:0/2815571215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4007cb0 con 0x7ff8fc0d1b30 2024-04-26T21:44:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff903780640 1 -- 172.21.15.40:0/2815571215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc0d1b30 msgr2=0x7ff8fc0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff903780640 1 --2- 172.21.15.40:0/2815571215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc0d1b30 0x7ff8fc0d1fc0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009d90 tx=0x7ff8f402a660 comp rx=0 tx=0).stop 2024-04-26T21:44:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff903780640 1 -- 172.21.15.40:0/2815571215 shutdown_connections 2024-04-26T21:44:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff903780640 1 --2- 172.21.15.40:0/2815571215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc0d1b30 0x7ff8fc0d1fc0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff903780640 1 --2- 172.21.15.40:0/2815571215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc162810 0x7ff8fc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff903780640 1 -- 172.21.15.40:0/2815571215 >> 172.21.15.40:0/2815571215 conn(0x7ff8fc0d2da0 msgr2=0x7ff8fc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff903780640 1 -- 172.21.15.40:0/2815571215 shutdown_connections 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.720+0000 7ff903780640 1 -- 172.21.15.40:0/2815571215 wait complete. 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff903780640 1 Processor -- start 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff903780640 1 -- start start 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff903780640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2340 0x7ff8fc160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff903780640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0d1b30 0x7ff8fc0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff903780640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc162810 0x7ff8fc0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff903780640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc070890 con 0x7ff8fc162810 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff903780640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc0d0470 con 0x7ff8fc0b2340 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff903780640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff8fc0cdb00 con 0x7ff8fc0d1b30 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff901d1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc162810 0x7ff8fc0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff901d1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc162810 0x7ff8fc0afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34758/0 (socket says 172.21.15.40:34758) 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.722+0000 7ff901d1d640 1 -- 172.21.15.40:0/2160094327 learned_addr learned my addr 172.21.15.40:0/2160094327 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff90151c640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2340 0x7ff8fc160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff900d1b640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0d1b30 0x7ff8fc0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff901d1d640 1 -- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0d1b30 msgr2=0x7ff8fc0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff901d1d640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0d1b30 0x7ff8fc0cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff901d1d640 1 -- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2340 msgr2=0x7ff8fc160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff901d1d640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2340 0x7ff8fc160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff901d1d640 1 -- 172.21.15.40:0/2160094327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc1613e0 con 0x7ff8fc162810 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff901d1d640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc162810 0x7ff8fc0afab0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009d90 tx=0x7ff8f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff8f1ffb640 1 -- 172.21.15.40:0/2160094327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4040070 con 0x7ff8fc162810 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff8f1ffb640 1 -- 172.21.15.40:0/2160094327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f4031c40 con 0x7ff8fc162810 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff8f1ffb640 1 -- 172.21.15.40:0/2160094327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4009450 con 0x7ff8fc162810 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc1616c0 con 0x7ff8fc162810 2024-04-26T21:44:47.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.723+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1610f0 con 0x7ff8fc162810 2024-04-26T21:44:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.724+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc0c9e70 con 0x7ff8fc162810 2024-04-26T21:44:47.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.725+0000 7ff8f1ffb640 1 -- 172.21.15.40:0/2160094327 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8f4039040 con 0x7ff8fc162810 2024-04-26T21:44:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.725+0000 7ff8f1ffb640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8e006c8f0 0x7ff8e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.725+0000 7ff8f1ffb640 1 -- 172.21.15.40:0/2160094327 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8109+0+0 (secure 0 0 0) 0x7ff8f40ad2f0 con 0x7ff8fc162810 2024-04-26T21:44:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.725+0000 7ff90151c640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8e006c8f0 0x7ff8e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.726+0000 7ff90151c640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8e006c8f0 0x7ff8e006edb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009970 tx=0x7ff8f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:47.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.730+0000 7ff8f1ffb640 1 -- 172.21.15.40:0/2160094327 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8f407ae50 con 0x7ff8fc162810 2024-04-26T21:44:47.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.954+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8fc0694f0 con 0x7ff8fc162810 2024-04-26T21:44:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.955+0000 7ff8f1ffb640 1 -- 172.21.15.40:0/2160094327 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) ==== 95+0+4355 (secure 0 0 0) 0x7ff8f407a750 con 0x7ff8fc162810 2024-04-26T21:44:47.955 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:47.955 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":226,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:47.608405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 226 2024-04-26T21:44:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8e006c8f0 msgr2=0x7ff8e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8e006c8f0 0x7ff8e006edb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009970 tx=0x7ff8f8008040 comp rx=0 tx=0).stop 2024-04-26T21:44:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc162810 msgr2=0x7ff8fc0afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc162810 0x7ff8fc0afab0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009d90 tx=0x7ff8f4004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 shutdown_connections 2024-04-26T21:44:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8e006c8f0 0x7ff8e006edb0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc162810 0x7ff8fc0afab0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0d1b30 0x7ff8fc0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 --2- 172.21.15.40:0/2160094327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2340 0x7ff8fc160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 >> 172.21.15.40:0/2160094327 conn(0x7ff8fc0d2da0 msgr2=0x7ff8fc070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 shutdown_connections 2024-04-26T21:44:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:47.957+0000 7ff903780640 1 -- 172.21.15.40:0/2160094327 wait complete. 2024-04-26T21:44:47.968 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:47.968 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.109+0000 7f9506719640 1 Processor -- start 2024-04-26T21:44:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.109+0000 7f9506719640 1 -- start start 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9506719640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b2340 0x7f95000cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9506719640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95000cd930 0x7f95000d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9506719640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95000d10d0 con 0x7f9500162810 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9506719640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95000d0750 con 0x7f95000cd930 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9506719640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95000d16a0 con 0x7f95000b2340 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9504cb6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95000cd930 0x7f95000d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f94fffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9500162810 0x7f9500070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39636/0 (socket says 172.21.15.40:39636) 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f94fffff640 1 -- 172.21.15.40:0/875752535 learned_addr learned my addr 172.21.15.40:0/875752535 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f94ff7fe640 1 --2- 172.21.15.40:0/875752535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b2340 0x7f95000cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9504cb6640 1 -- 172.21.15.40:0/875752535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b2340 msgr2=0x7f95000cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9504cb6640 1 --2- 172.21.15.40:0/875752535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b2340 0x7f95000cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9504cb6640 1 -- 172.21.15.40:0/875752535 >> v1:172.21.15.40:6789/0 conn(0x7f9500162810 legacy=0x7f9500070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f94feffd640 1 -- 172.21.15.40:0/875752535 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 89756010 0 0) 0x7f95000d10d0 con 0x7f9500162810 2024-04-26T21:44:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f9504cb6640 1 -- 172.21.15.40:0/875752535 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95000d1370 con 0x7f95000cd930 2024-04-26T21:44:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.110+0000 7f94ff7fe640 1 --2- 172.21.15.40:0/875752535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b2340 0x7f95000cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f9504cb6640 1 --2- 172.21.15.40:0/875752535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f95000d1b30 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009d90 tx=0x7f94f802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879c476cabbb8182 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f94feffd640 1 -- 172.21.15.40:0/875752535 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8007740 con 0x7f95000cd930 2024-04-26T21:44:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f94feffd640 1 -- 172.21.15.40:0/875752535 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f80078e0 con 0x7f95000cd930 2024-04-26T21:44:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f94feffd640 1 -- 172.21.15.40:0/875752535 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8007cb0 con 0x7f95000cd930 2024-04-26T21:44:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f9506719640 1 -- 172.21.15.40:0/875752535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 msgr2=0x7f95000d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f9506719640 1 --2- 172.21.15.40:0/875752535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f95000d1b30 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009d90 tx=0x7f94f802a660 comp rx=0 tx=0).stop 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f9506719640 1 -- 172.21.15.40:0/875752535 shutdown_connections 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f9506719640 1 --2- 172.21.15.40:0/875752535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f95000d1b30 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f9506719640 1 --2- 172.21.15.40:0/875752535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b2340 0x7f95000cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.111+0000 7f9506719640 1 -- 172.21.15.40:0/875752535 >> 172.21.15.40:0/875752535 conn(0x7f95000d2da0 msgr2=0x7f95000d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.112+0000 7f9506719640 1 -- 172.21.15.40:0/875752535 shutdown_connections 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.112+0000 7f9506719640 1 -- 172.21.15.40:0/875752535 wait complete. 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.113+0000 7f9506719640 1 Processor -- start 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.113+0000 7f9506719640 1 -- start start 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.113+0000 7f9506719640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000b2340 0x7f9500159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f9506719640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f950006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f9506719640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9500162810 0x7f950006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f9506719640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95000d2070 con 0x7f95000cd930 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f9506719640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95000d0470 con 0x7f95000b2340 2024-04-26T21:44:48.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f9506719640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95000d21f0 con 0x7f9500162810 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f950006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f950006d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34766/0 (socket says 172.21.15.40:34766) 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 -- 172.21.15.40:0/1799935286 learned_addr learned my addr 172.21.15.40:0/1799935286 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f9504cb6640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9500162810 0x7f950006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94fffff640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000b2340 0x7f9500159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 -- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9500162810 msgr2=0x7f950006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9500162810 0x7f950006d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 -- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000b2340 msgr2=0x7f9500159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000b2340 0x7f9500159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 -- 172.21.15.40:0/1799935286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95001f4850 con 0x7f95000cd930 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94ff7fe640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f950006d350 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f94f000c7f0 tx=0x7f94f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94fcff9640 1 -- 172.21.15.40:0/1799935286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0015070 con 0x7f95000cd930 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94fcff9640 1 -- 172.21.15.40:0/1799935286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f0007a30 con 0x7f95000cd930 2024-04-26T21:44:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95000c7900 con 0x7f95000cd930 2024-04-26T21:44:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f94fcff9640 1 -- 172.21.15.40:0/1799935286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0007e00 con 0x7f95000cd930 2024-04-26T21:44:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.114+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95000c7ec0 con 0x7f95000cd930 2024-04-26T21:44:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.115+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95000694f0 con 0x7f95000cd930 2024-04-26T21:44:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.116+0000 7f94fcff9640 1 -- 172.21.15.40:0/1799935286 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94f0018070 con 0x7f95000cd930 2024-04-26T21:44:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.116+0000 7f94fcff9640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.116+0000 7f94fcff9640 1 -- 172.21.15.40:0/1799935286 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8109+0+0 (secure 0 0 0) 0x7f94f008c510 con 0x7f95000cd930 2024-04-26T21:44:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.116+0000 7f94fffff640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.117+0000 7f94fffff640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94e006c8f0 0x7f94e006edb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000f60 tx=0x7f94e8007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.121+0000 7f94fcff9640 1 -- 172.21.15.40:0/1799935286 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94f0059f40 con 0x7f95000cd930 2024-04-26T21:44:48.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.348+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9500069cd0 con 0x7f95000cd930 2024-04-26T21:44:48.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.348+0000 7f94fcff9640 1 -- 172.21.15.40:0/1799935286 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) ==== 95+0+4355 (secure 0 0 0) 0x7f94f0059840 con 0x7f95000cd930 2024-04-26T21:44:48.348 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:48.348 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":226,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:47.608405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:48.348 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 226 2024-04-26T21:44:48.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94e006c8f0 msgr2=0x7f94e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94e006c8f0 0x7f94e006edb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000f60 tx=0x7f94e8007c00 comp rx=0 tx=0).stop 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 msgr2=0x7f950006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f950006d350 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f94f000c7f0 tx=0x7f94f000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 shutdown_connections 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9500162810 0x7f950006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000cd930 0x7f950006d350 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 --2- 172.21.15.40:0/1799935286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000b2340 0x7f9500159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 >> 172.21.15.40:0/1799935286 conn(0x7f95000d2da0 msgr2=0x7f950015ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.350+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 shutdown_connections 2024-04-26T21:44:48.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.351+0000 7f9506719640 1 -- 172.21.15.40:0/1799935286 wait complete. 2024-04-26T21:44:48.361 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:48.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:48.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.462+0000 7f46dd27e640 1 Processor -- start 2024-04-26T21:44:48.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.462+0000 7f46dd27e640 1 -- start start 2024-04-26T21:44:48.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.462+0000 7f46dd27e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.462+0000 7f46dd27e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d815af40 con 0x7f46d81617f0 2024-04-26T21:44:48.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.462+0000 7f46dd27e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d815e320 con 0x7f46d815ab60 2024-04-26T21:44:48.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.462+0000 7f46dd27e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46d8160d10 con 0x7f46d806c950 2024-04-26T21:44:48.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46d77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f46d81617f0 0x7f46d815db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39638/0 (socket says 172.21.15.40:39638) 2024-04-26T21:44:48.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46d77fe640 1 -- 172.21.15.40:0/2584065065 learned_addr learned my addr 172.21.15.40:0/2584065065 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46ce7fc640 1 --2- 172.21.15.40:0/2584065065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46d67fc640 1 -- 172.21.15.40:0/2584065065 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2483262275 0 0) 0x7f46d815af40 con 0x7f46d81617f0 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46d67fc640 1 -- 172.21.15.40:0/2584065065 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46c4003520 con 0x7f46d81617f0 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46d67fc640 1 -- 172.21.15.40:0/2584065065 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979760866 0 0) 0x7f46d815e320 con 0x7f46d815ab60 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46d67fc640 1 -- 172.21.15.40:0/2584065065 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46d815af40 con 0x7f46d815ab60 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46d67fc640 1 -- 172.21.15.40:0/2584065065 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1581295595 0 0) 0x7f46c4003520 con 0x7f46d81617f0 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46ce7fc640 1 -- 172.21.15.40:0/2584065065 >> v1:172.21.15.184:6789/0 conn(0x7f46d815ab60 legacy=0x7f46d806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46ce7fc640 1 -- 172.21.15.40:0/2584065065 >> v1:172.21.15.40:6789/0 conn(0x7f46d81617f0 legacy=0x7f46d815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.463+0000 7f46ce7fc640 1 -- 172.21.15.40:0/2584065065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d8160ff0 con 0x7f46d806c950 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.464+0000 7f46ce7fc640 1 --2- 172.21.15.40:0/2584065065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815d5f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f46c00097d0 tx=0x7f46c002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc3d0defa7d3e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.464+0000 7f46d67fc640 1 -- 172.21.15.40:0/2584065065 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0037070 con 0x7f46d806c950 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.464+0000 7f46d67fc640 1 -- 172.21.15.40:0/2584065065 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c002ac30 con 0x7f46d806c950 2024-04-26T21:44:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.464+0000 7f46d67fc640 1 -- 172.21.15.40:0/2584065065 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d00019f0 con 0x7f46d806c950 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.465+0000 7f46dd27e640 1 -- 172.21.15.40:0/2584065065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 msgr2=0x7f46d815d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.465+0000 7f46dd27e640 1 --2- 172.21.15.40:0/2584065065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815d5f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f46c00097d0 tx=0x7f46c002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.465+0000 7f46dd27e640 1 -- 172.21.15.40:0/2584065065 shutdown_connections 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.465+0000 7f46dd27e640 1 --2- 172.21.15.40:0/2584065065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815d5f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.465+0000 7f46dd27e640 1 -- 172.21.15.40:0/2584065065 >> 172.21.15.40:0/2584065065 conn(0x7f46d80d1cf0 msgr2=0x7f46d80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.465+0000 7f46dd27e640 1 -- 172.21.15.40:0/2584065065 shutdown_connections 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.465+0000 7f46dd27e640 1 -- 172.21.15.40:0/2584065065 wait complete. 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.466+0000 7f46dd27e640 1 Processor -- start 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.466+0000 7f46dd27e640 1 -- start start 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46dd27e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46dd27e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d815ab60 0x7f46d80ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46dd27e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46d81617f0 0x7f46d80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46dd27e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8168010 con 0x7f46d815ab60 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46dd27e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8167e90 con 0x7f46d81617f0 2024-04-26T21:44:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46dd27e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46d8168190 con 0x7f46d806c950 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d815ab60 0x7f46d80ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d815ab60 0x7f46d80ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34780/0 (socket says 172.21.15.40:34780) 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 -- 172.21.15.40:0/3603852625 learned_addr learned my addr 172.21.15.40:0/3603852625 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46d77fe640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46d81617f0 0x7f46d80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46d6ffd640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 -- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 msgr2=0x7f46d815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 -- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46d81617f0 msgr2=0x7f46d80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46d81617f0 0x7f46d80ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 -- 172.21.15.40:0/3603852625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d815f8f0 con 0x7f46d815ab60 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46d6ffd640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46d77fe640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46d81617f0 0x7f46d80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.467+0000 7f46ce7fc640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d815ab60 0x7f46d80ca270 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f46c0035040 tx=0x7f46c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.468+0000 7f46cffff640 1 -- 172.21.15.40:0/3603852625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0041020 con 0x7f46d815ab60 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.468+0000 7f46cffff640 1 -- 172.21.15.40:0/3603852625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c003d640 con 0x7f46d815ab60 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.468+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d815fbd0 con 0x7f46d815ab60 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.468+0000 7f46cffff640 1 -- 172.21.15.40:0/3603852625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0037040 con 0x7f46d815ab60 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.468+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d8160190 con 0x7f46d815ab60 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.469+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d80694f0 con 0x7f46d815ab60 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.469+0000 7f46cffff640 1 -- 172.21.15.40:0/3603852625 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46c0044020 con 0x7f46d815ab60 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.470+0000 7f46cffff640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f46b806c8f0 0x7f46b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.470+0000 7f46cffff640 1 -- 172.21.15.40:0/3603852625 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8109+0+0 (secure 0 0 0) 0x7f46c00aca40 con 0x7f46d815ab60 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.470+0000 7f46d6ffd640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f46b806c8f0 0x7f46b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.471+0000 7f46d6ffd640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f46b806c8f0 0x7f46b806edb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f46c80011e0 tx=0x7f46c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.474+0000 7f46cffff640 1 -- 172.21.15.40:0/3603852625 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46c004b030 con 0x7f46d815ab60 2024-04-26T21:44:48.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.697+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46d80ce7c0 con 0x7f46d815ab60 2024-04-26T21:44:48.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.698+0000 7f46cffff640 1 -- 172.21.15.40:0/3603852625 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 227 v227) ==== 95+0+5182 (secure 0 0 0) 0x7f46c00ace20 con 0x7f46d815ab60 2024-04-26T21:44:48.697 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:48.698 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":227,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2224283166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2224283166},{"type":"v1","addr":"172.21.15.40:6833","nonce":2224283166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":227}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:47.608405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:48.698 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 227 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.699+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f46b806c8f0 msgr2=0x7f46b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.699+0000 7f46dd27e640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f46b806c8f0 0x7f46b806edb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f46c80011e0 tx=0x7f46c8009040 comp rx=0 tx=0).stop 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d815ab60 msgr2=0x7f46d80ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d815ab60 0x7f46d80ca270 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f46c0035040 tx=0x7f46c00046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 shutdown_connections 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f46b806c8f0 0x7f46b806edb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46d81617f0 0x7f46d80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d815ab60 0x7f46d80ca270 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 --2- 172.21.15.40:0/3603852625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46d806c950 0x7f46d815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 >> 172.21.15.40:0/3603852625 conn(0x7f46d80d1cf0 msgr2=0x7f46d815b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 shutdown_connections 2024-04-26T21:44:48.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.700+0000 7f46dd27e640 1 -- 172.21.15.40:0/3603852625 wait complete. 2024-04-26T21:44:48.710 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 400.00000000' led to DAMAGED state 2024-04-26T21:44:48.710 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 100.00000000.inode' 2024-04-26T21:44:48.711 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:48.711 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:48.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:48.728 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:48.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:48.728 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:48.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:48.782 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:48.782 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.924+0000 7f461fc7b640 1 Processor -- start 2024-04-26T21:44:48.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.924+0000 7f461fc7b640 1 -- start start 2024-04-26T21:44:48.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461fc7b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46180b0380 0x7f46180ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461fc7b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f46180b10b0 0x7f461806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461fc7b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46180cfd80 con 0x7f461806c780 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461fc7b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f461806cbd0 con 0x7f46180b0380 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461fc7b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f46180af8e0 con 0x7f46180b10b0 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461da17640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46180b0380 0x7f46180ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461e218640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f461806c780 0x7f4618072f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32888/0 (socket says 172.21.15.40:32888) 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461e218640 1 -- 172.21.15.40:0/3606431286 learned_addr learned my addr 172.21.15.40:0/3606431286 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461da17640 1 -- 172.21.15.40:0/3606431286 >> v2:172.21.15.184:3300/0 conn(0x7f46180b10b0 msgr2=0x7f461806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.925+0000 7f461da17640 1 --2- 172.21.15.40:0/3606431286 >> v2:172.21.15.184:3300/0 conn(0x7f46180b10b0 0x7f461806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.926+0000 7f461da17640 1 -- 172.21.15.40:0/3606431286 >> v1:172.21.15.40:6789/0 conn(0x7f461806c780 legacy=0x7f4618072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.926+0000 7f461da17640 1 -- 172.21.15.40:0/3606431286 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46180d0020 con 0x7f46180b0380 2024-04-26T21:44:48.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.926+0000 7f461ca15640 1 -- 172.21.15.40:0/3606431286 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1395629586 0 0) 0x7f46180cfd80 con 0x7f461806c780 2024-04-26T21:44:48.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.926+0000 7f461da17640 1 --2- 172.21.15.40:0/3606431286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46180b0380 0x7f46180ccd50 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f46180b48c0 tx=0x7f461002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ebe4c2b4435964 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.926+0000 7f461ca15640 1 -- 172.21.15.40:0/3606431286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610037070 con 0x7f46180b0380 2024-04-26T21:44:48.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.926+0000 7f461ca15640 1 -- 172.21.15.40:0/3606431286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4610032440 con 0x7f46180b0380 2024-04-26T21:44:48.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.926+0000 7f461ca15640 1 -- 172.21.15.40:0/3606431286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461002ae60 con 0x7f46180b0380 2024-04-26T21:44:48.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.927+0000 7f461fc7b640 1 -- 172.21.15.40:0/3606431286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46180b0380 msgr2=0x7f46180ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.927+0000 7f461fc7b640 1 --2- 172.21.15.40:0/3606431286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46180b0380 0x7f46180ccd50 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f46180b48c0 tx=0x7f461002a110 comp rx=0 tx=0).stop 2024-04-26T21:44:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.927+0000 7f461fc7b640 1 -- 172.21.15.40:0/3606431286 shutdown_connections 2024-04-26T21:44:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.927+0000 7f461fc7b640 1 --2- 172.21.15.40:0/3606431286 >> v2:172.21.15.184:3300/0 conn(0x7f46180b10b0 0x7f461806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.927+0000 7f461fc7b640 1 --2- 172.21.15.40:0/3606431286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46180b0380 0x7f46180ccd50 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.927+0000 7f461fc7b640 1 -- 172.21.15.40:0/3606431286 >> 172.21.15.40:0/3606431286 conn(0x7f46180d1cf0 msgr2=0x7f46180d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.927+0000 7f461fc7b640 1 -- 172.21.15.40:0/3606431286 shutdown_connections 2024-04-26T21:44:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.927+0000 7f461fc7b640 1 -- 172.21.15.40:0/3606431286 wait complete. 2024-04-26T21:44:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.928+0000 7f461fc7b640 1 Processor -- start 2024-04-26T21:44:48.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.928+0000 7f461fc7b640 1 -- start start 2024-04-26T21:44:48.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461fc7b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461806c780 0x7f46180cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461fc7b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46180b0380 0x7f461815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461fc7b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46180b10b0 0x7f46181f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461fc7b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f46180d1980 con 0x7f461806c780 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461fc7b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46180d05e0 con 0x7f46180b0380 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461fc7b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46180b0760 con 0x7f46180b10b0 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461da17640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461806c780 0x7f46180cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461da17640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461806c780 0x7f46180cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59048/0 (socket says 172.21.15.40:59048) 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461da17640 1 -- 172.21.15.40:0/1069872479 learned_addr learned my addr 172.21.15.40:0/1069872479 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461e218640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46180b10b0 0x7f46181f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461d216640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46180b0380 0x7f461815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461da17640 1 -- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46180b10b0 msgr2=0x7f46181f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461da17640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46180b10b0 0x7f46181f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461da17640 1 -- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46180b0380 msgr2=0x7f461815a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461da17640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46180b0380 0x7f461815a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461da17640 1 -- 172.21.15.40:0/1069872479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461815e470 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461e218640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46180b10b0 0x7f46181f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.929+0000 7f461d216640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46180b0380 0x7f461815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.930+0000 7f461da17640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461806c780 0x7f46180cb250 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f4610035040 tx=0x7f46100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.930+0000 7f46167fc640 1 -- 172.21.15.40:0/1069872479 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610037040 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.930+0000 7f46167fc640 1 -- 172.21.15.40:0/1069872479 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4610004230 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.930+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46181f8900 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.930+0000 7f46167fc640 1 -- 172.21.15.40:0/1069872479 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610032e20 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.930+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181f8ec0 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.931+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46180694f0 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.931+0000 7f46167fc640 1 -- 172.21.15.40:0/1069872479 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4610030770 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.932+0000 7f46167fc640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f460406c920 0x7f460406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.932+0000 7f46167fc640 1 -- 172.21.15.40:0/1069872479 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8109+0+0 (secure 0 0 0) 0x7f46100ad7a0 con 0x7f461806c780 2024-04-26T21:44:48.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.932+0000 7f461d216640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f460406c920 0x7f460406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.933+0000 7f461d216640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f460406c920 0x7f460406ede0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f460c006fd0 tx=0x7f460c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:48.936+0000 7f46167fc640 1 -- 172.21.15.40:0/1069872479 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f461007b270 con 0x7f461806c780 2024-04-26T21:44:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.160+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f461806f030 con 0x7f461806c780 2024-04-26T21:44:49.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.619+0000 7f46167fc640 1 -- 172.21.15.40:0/1069872479 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v228) ==== 161+0+0 (secure 0 0 0) 0x7f461007ab70 con 0x7f461806c780 2024-04-26T21:44:49.619 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:49.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f460406c920 msgr2=0x7f460406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:49.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f460406c920 0x7f460406ede0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f460c006fd0 tx=0x7f460c008040 comp rx=0 tx=0).stop 2024-04-26T21:44:49.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461806c780 msgr2=0x7f46180cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:49.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461806c780 0x7f46180cb250 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f4610035040 tx=0x7f46100046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 shutdown_connections 2024-04-26T21:44:49.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f460406c920 0x7f460406ede0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46180b10b0 0x7f46181f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46180b0380 0x7f461815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 --2- 172.21.15.40:0/1069872479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f461806c780 0x7f46180cb250 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 >> 172.21.15.40:0/1069872479 conn(0x7f46180d1cf0 msgr2=0x7f46180b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:49.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 shutdown_connections 2024-04-26T21:44:49.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.621+0000 7f461fc7b640 1 -- 172.21.15.40:0/1069872479 wait complete. 2024-04-26T21:44:49.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.735+0000 7fd9036c8640 1 Processor -- start 2024-04-26T21:44:49.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.735+0000 7fd9036c8640 1 -- start start 2024-04-26T21:44:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd9036c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8fc0b0380 0x7fd8fc0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd9036c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd8fc0b10b0 0x7fd8fc06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd9036c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8fc0cfd80 con 0x7fd8fc06c780 2024-04-26T21:44:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd9036c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd8fc06cbd0 con 0x7fd8fc0b0380 2024-04-26T21:44:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd9036c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd8fc0af8e0 con 0x7fd8fc0b10b0 2024-04-26T21:44:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd901464640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8fc0b0380 0x7fd8fc0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd901464640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8fc0b0380 0x7fd8fc0ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59058/0 (socket says 172.21.15.40:59058) 2024-04-26T21:44:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd901464640 1 -- 172.21.15.40:0/1964408814 learned_addr learned my addr 172.21.15.40:0/1964408814 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd900c63640 1 --2- 172.21.15.40:0/1964408814 >> v2:172.21.15.184:3300/0 conn(0x7fd8fc0b10b0 0x7fd8fc06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd901464640 1 -- 172.21.15.40:0/1964408814 >> v2:172.21.15.184:3300/0 conn(0x7fd8fc0b10b0 msgr2=0x7fd8fc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd901464640 1 --2- 172.21.15.40:0/1964408814 >> v2:172.21.15.184:3300/0 conn(0x7fd8fc0b10b0 0x7fd8fc06c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd901464640 1 -- 172.21.15.40:0/1964408814 >> v1:172.21.15.184:6789/0 conn(0x7fd8fc06c780 legacy=0x7fd8fc072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:49.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.736+0000 7fd901464640 1 -- 172.21.15.40:0/1964408814 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc0d0020 con 0x7fd8fc0b0380 2024-04-26T21:44:49.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.737+0000 7fd901464640 1 --2- 172.21.15.40:0/1964408814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc0b0380 0x7fd8fc0ccd50 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0b48c0 tx=0x7fd8f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5acf864871e0218 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.737+0000 7fd8f3fff640 1 -- 172.21.15.40:0/1964408814 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8037070 con 0x7fd8fc0b0380 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.737+0000 7fd8f3fff640 1 -- 172.21.15.40:0/1964408814 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f8032440 con 0x7fd8fc0b0380 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.737+0000 7fd8f3fff640 1 -- 172.21.15.40:0/1964408814 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8032810 con 0x7fd8fc0b0380 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.738+0000 7fd9036c8640 1 -- 172.21.15.40:0/1964408814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc0b0380 msgr2=0x7fd8fc0ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.738+0000 7fd9036c8640 1 --2- 172.21.15.40:0/1964408814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc0b0380 0x7fd8fc0ccd50 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0b48c0 tx=0x7fd8f802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.738+0000 7fd9036c8640 1 -- 172.21.15.40:0/1964408814 shutdown_connections 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.738+0000 7fd9036c8640 1 --2- 172.21.15.40:0/1964408814 >> v2:172.21.15.184:3300/0 conn(0x7fd8fc0b10b0 0x7fd8fc06c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.738+0000 7fd9036c8640 1 --2- 172.21.15.40:0/1964408814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc0b0380 0x7fd8fc0ccd50 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.738+0000 7fd9036c8640 1 -- 172.21.15.40:0/1964408814 >> 172.21.15.40:0/1964408814 conn(0x7fd8fc0d1cf0 msgr2=0x7fd8fc0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.738+0000 7fd9036c8640 1 -- 172.21.15.40:0/1964408814 shutdown_connections 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.738+0000 7fd9036c8640 1 -- 172.21.15.40:0/1964408814 wait complete. 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.739+0000 7fd9036c8640 1 Processor -- start 2024-04-26T21:44:49.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.739+0000 7fd9036c8640 1 -- start start 2024-04-26T21:44:49.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd9036c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc06c780 0x7fd8fc1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:49.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd9036c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8fc0b0380 0x7fd8fc0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd9036c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8fc0b10b0 0x7fd8fc0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd9036c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0d1980 con 0x7fd8fc06c780 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd9036c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0d05e0 con 0x7fd8fc0b10b0 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd9036c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc0b0760 con 0x7fd8fc0b0380 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc06c780 0x7fd8fc1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc06c780 0x7fd8fc1542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59060/0 (socket says 172.21.15.40:59060) 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 -- 172.21.15.40:0/3836033603 learned_addr learned my addr 172.21.15.40:0/3836033603 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd900c63640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8fc0b0380 0x7fd8fc0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 -- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8fc0b0380 msgr2=0x7fd8fc0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8fc0b0380 0x7fd8fc0c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 -- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8fc0b10b0 msgr2=0x7fd8fc0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901c65640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8fc0b10b0 0x7fd8fc0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8fc0b10b0 0x7fd8fc0cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 -- 172.21.15.40:0/3836033603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc1f50e0 con 0x7fd8fc06c780 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd901464640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc06c780 0x7fd8fc1542b0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fd8f802a220 tx=0x7fd8f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd8f1ffb640 1 -- 172.21.15.40:0/3836033603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8041020 con 0x7fd8fc06c780 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd8f1ffb640 1 -- 172.21.15.40:0/3836033603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f80306c0 con 0x7fd8fc06c780 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc1f53c0 con 0x7fd8fc06c780 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.740+0000 7fd8f1ffb640 1 -- 172.21.15.40:0/3836033603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8037040 con 0x7fd8fc06c780 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.741+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc1f5980 con 0x7fd8fc06c780 2024-04-26T21:44:49.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.742+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8fc0694f0 con 0x7fd8fc06c780 2024-04-26T21:44:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.742+0000 7fd8f1ffb640 1 -- 172.21.15.40:0/3836033603 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8f8044020 con 0x7fd8fc06c780 2024-04-26T21:44:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.743+0000 7fd8f1ffb640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8e006c8f0 0x7fd8e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.743+0000 7fd8f1ffb640 1 -- 172.21.15.40:0/3836033603 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8109+0+0 (secure 0 0 0) 0x7fd8f80acbe0 con 0x7fd8fc06c780 2024-04-26T21:44:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.743+0000 7fd900c63640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8e006c8f0 0x7fd8e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.743+0000 7fd900c63640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8e006c8f0 0x7fd8e006edb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc071490 tx=0x7fd8ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:49.747+0000 7fd8f1ffb640 1 -- 172.21.15.40:0/3836033603 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8f804b030 con 0x7fd8fc06c780 2024-04-26T21:44:50.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.006+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fd8fc0ceb20 con 0x7fd8fc06c780 2024-04-26T21:44:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.624+0000 7fd8f1ffb640 1 -- 172.21.15.40:0/3836033603 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v229) ==== 101+0+0 (secure 0 0 0) 0x7fd8f807a740 con 0x7fd8fc06c780 2024-04-26T21:44:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8e006c8f0 msgr2=0x7fd8e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8e006c8f0 0x7fd8e006edb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc071490 tx=0x7fd8ec008040 comp rx=0 tx=0).stop 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc06c780 msgr2=0x7fd8fc1542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc06c780 0x7fd8fc1542b0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fd8f802a220 tx=0x7fd8f80046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 shutdown_connections 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd8e006c8f0 0x7fd8e006edb0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8fc0b10b0 0x7fd8fc0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8fc0b0380 0x7fd8fc0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 --2- 172.21.15.40:0/3836033603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8fc06c780 0x7fd8fc1542b0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 >> 172.21.15.40:0/3836033603 conn(0x7fd8fc0d1cf0 msgr2=0x7fd8fc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 shutdown_connections 2024-04-26T21:44:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.627+0000 7fd9036c8640 1 -- 172.21.15.40:0/3836033603 wait complete. 2024-04-26T21:44:50.640 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.686+0000 7f313ab95f40 1 Processor -- start 2024-04-26T21:44:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.686+0000 7f313ab95f40 1 -- start start 2024-04-26T21:44:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.686+0000 7f313ab95f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b21ed3e0 0x5635b222fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.686+0000 7f313ab95f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b22c0420 0x5635b222d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.686+0000 7f313ab95f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0d40 0x5635b2230830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.686+0000 7f313ab95f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5635b2232c60 con 0x5635b22c0420 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.686+0000 7f313ab95f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5635b2192e40 con 0x5635b22c0d40 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.686+0000 7f313ab95f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5635b2234ae0 con 0x5635b21ed3e0 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b22c0420 0x5635b222d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b22c0420 0x5635b222d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59076/0 (socket says 172.21.15.40:59076) 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f313868f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b21ed3e0 0x5635b222fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 -- 172.21.15.40:0/2546930931 learned_addr learned my addr 172.21.15.40:0/2546930931 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f3138e90640 1 --2- 172.21.15.40:0/2546930931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0d40 0x5635b2230830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 -- 172.21.15.40:0/2546930931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b21ed3e0 msgr2=0x5635b222fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 --2- 172.21.15.40:0/2546930931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b21ed3e0 0x5635b222fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 -- 172.21.15.40:0/2546930931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0d40 msgr2=0x5635b2230830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 --2- 172.21.15.40:0/2546930931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0d40 0x5635b2230830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 -- 172.21.15.40:0/2546930931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635b22bf130 con 0x5635b22c0420 2024-04-26T21:44:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f312ffff640 1 --2- 172.21.15.40:0/2546930931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b22c0420 0x5635b222d3d0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f312000a770 tx=0x7f312000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e47d3c74223115a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f313768d640 1 -- 172.21.15.40:0/2546930931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120018070 con 0x5635b22c0420 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f313768d640 1 -- 172.21.15.40:0/2546930931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3120007600 con 0x5635b22c0420 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.687+0000 7f313768d640 1 -- 172.21.15.40:0/2546930931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31200079b0 con 0x5635b22c0420 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 -- 172.21.15.40:0/2546930931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b22c0420 msgr2=0x5635b222d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 --2- 172.21.15.40:0/2546930931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b22c0420 0x5635b222d3d0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f312000a770 tx=0x7f312000a860 comp rx=0 tx=0).stop 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 -- 172.21.15.40:0/2546930931 shutdown_connections 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 --2- 172.21.15.40:0/2546930931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0d40 0x5635b2230830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 --2- 172.21.15.40:0/2546930931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b22c0420 0x5635b222d3d0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 --2- 172.21.15.40:0/2546930931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b21ed3e0 0x5635b222fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 -- 172.21.15.40:0/2546930931 >> 172.21.15.40:0/2546930931 conn(0x5635b21ea330 msgr2=0x5635b21eafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 -- 172.21.15.40:0/2546930931 shutdown_connections 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.688+0000 7f313ab95f40 1 -- 172.21.15.40:0/2546930931 wait complete. 2024-04-26T21:44:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313ab95f40 1 Processor -- start 2024-04-26T21:44:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313ab95f40 1 -- start start 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313ab95f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b21ed3e0 0x5635b22c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313ab95f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0420 0x5635b22b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313ab95f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b22c0d40 0x5635b22b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313ab95f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5635b22346c0 con 0x5635b21ed3e0 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313ab95f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5635b2234230 con 0x5635b22c0420 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313ab95f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5635b22c4d60 con 0x5635b22c0d40 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.689+0000 7f313868f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b21ed3e0 0x5635b22c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313868f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b21ed3e0 0x5635b22c2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59080/0 (socket says 172.21.15.40:59080) 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 learned_addr learned my addr 172.21.15.40:0/556420585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f312ffff640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0420 0x5635b22b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f3138e90640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b22c0d40 0x5635b22b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b22c0d40 msgr2=0x5635b22b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313868f640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5635b22c0d40 0x5635b22b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0420 msgr2=0x5635b22b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313868f640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635b22c0420 0x5635b22b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635b22be4f0 con 0x5635b21ed3e0 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313868f640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b21ed3e0 0x5635b22c2cd0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f3128045ef0 tx=0x7f3128002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313668b640 1 -- 172.21.15.40:0/556420585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128054b70 con 0x5635b21ed3e0 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313668b640 1 -- 172.21.15.40:0/556420585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3128054d10 con 0x5635b21ed3e0 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5635b22b7e00 con 0x5635b21ed3e0 2024-04-26T21:44:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313668b640 1 -- 172.21.15.40:0/556420585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312805f660 con 0x5635b21ed3e0 2024-04-26T21:44:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.690+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635b21e99b0 con 0x5635b21ed3e0 2024-04-26T21:44:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.692+0000 7f313668b640 1 -- 172.21.15.40:0/556420585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f312805c040 con 0x5635b21ed3e0 2024-04-26T21:44:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.692+0000 7f313668b640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f311806b300 0x7f311806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.692+0000 7f313668b640 1 -- 172.21.15.40:0/556420585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8109+0+0 (secure 0 0 0) 0x7f31280cf300 con 0x5635b21ed3e0 2024-04-26T21:44:50.695 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.692+0000 7f312ffff640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f311806b300 0x7f311806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.692+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5635b22b8100 0x5635b22bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.692+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b222dbb0 con 0x5635b22b8100 2024-04-26T21:44:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.693+0000 7f3138e90640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5635b22b8100 0x5635b22bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.693+0000 7f312ffff640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f311806b300 0x7f311806d7c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x5635b2217100 tx=0x7f312000c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.693+0000 7f3138e90640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5635b22b8100 0x5635b22bc2a0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.694+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v117'660 uv634 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.695+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b22bc7e0 con 0x5635b22b8100 2024-04-26T21:44:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.696+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v117'661 uv661 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.696+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b22bcbb0 con 0x5635b22b8100 2024-04-26T21:44:50.697 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.697+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v117'662 uv662 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.697+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364830 con 0x5635b22b8100 2024-04-26T21:44:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.698+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v117'663 uv663 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.698+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364bd0 con 0x5635b22b8100 2024-04-26T21:44:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.699+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v117'664 uv664 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.699+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2363820 con 0x5635b22b8100 2024-04-26T21:44:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.700+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v117'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.700+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2363c10 con 0x5635b22b8100 2024-04-26T21:44:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.701+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v117'666 uv640 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.701+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2364000 con 0x5635b22b8100 2024-04-26T21:44:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.703+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v117'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.703+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643f0 con 0x5635b22b8100 2024-04-26T21:44:50.703 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.704+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v117'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.704+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643f0 con 0x5635b22b8100 2024-04-26T21:44:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.705+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v117'669 uv669 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.705+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643a0 con 0x5635b22b8100 2024-04-26T21:44:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.706+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v117'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.706+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643a0 con 0x5635b22b8100 2024-04-26T21:44:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.707+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v117'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.707+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643a0 con 0x5635b22b8100 2024-04-26T21:44:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.708+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v117'672 uv647 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.708+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23643a0 con 0x5635b22b8100 2024-04-26T21:44:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.709+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v117'673 uv673 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.709+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643a0 con 0x5635b22b8100 2024-04-26T21:44:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.710+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v117'674 uv674 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.710 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.710+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643a0 con 0x5635b22b8100 2024-04-26T21:44:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.711+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v117'675 uv675 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.711+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643a0 con 0x5635b22b8100 2024-04-26T21:44:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.712+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v117'676 uv676 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.712+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23643a0 con 0x5635b22b8100 2024-04-26T21:44:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.713+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v117'677 uv677 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.714+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.715+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v117'678 uv678 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.715+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.716+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v117'679 uv653 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.716+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.717+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v117'680 uv680 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.717+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.718 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:50.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.718+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v117'681 uv681 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.718+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2365f80 con 0x5635b22b8100 2024-04-26T21:44:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.719+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v117'682 uv682 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.719+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2365f80 con 0x5635b22b8100 2024-04-26T21:44:50.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.720+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v117'683 uv683 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.720+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.721+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v117'684 uv684 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.721+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.723+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v117'685 uv659 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.723+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.724+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v117'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.724+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.725 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:50.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.725+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v117'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.725+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2365f80 con 0x5635b22b8100 2024-04-26T21:44:50.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.726+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v117'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.726+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2365f80 con 0x5635b22b8100 2024-04-26T21:44:50.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.727+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v117'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.727+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2364f70 con 0x5635b22b8100 2024-04-26T21:44:50.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.728+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v117'690 uv690 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b22b8100 2024-04-26T21:44:50.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.729+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5635b23655f0 0x5635b23679e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.729+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2367f20 con 0x5635b23655f0 2024-04-26T21:44:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.729+0000 7f313868f640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5635b23655f0 0x5635b23679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.729+0000 7f313868f640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5635b23655f0 0x5635b23679e0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.731+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v117'484 uv467 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.731+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23682c0 con 0x5635b23655f0 2024-04-26T21:44:50.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.732+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v117'485 uv485 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.732+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2368660 con 0x5635b23655f0 2024-04-26T21:44:50.733 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:50.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.733+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v117'486 uv486 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.733+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2368eb0 con 0x5635b23655f0 2024-04-26T21:44:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.735+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v117'487 uv487 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.735+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369250 con 0x5635b23655f0 2024-04-26T21:44:50.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.736+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v117'488 uv488 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.736+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23695f0 con 0x5635b23655f0 2024-04-26T21:44:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.737+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v117'489 uv472 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.737+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2369990 con 0x5635b23655f0 2024-04-26T21:44:50.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.738+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v117'490 uv490 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.738+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.739 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:50.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.740+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v117'491 uv491 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.740+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.741+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v117'492 uv492 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.741+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.742+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v117'493 uv493 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.742+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.743+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v117'494 uv477 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.743+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.745+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v117'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.745+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.745 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:50.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.746+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v117'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.746+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236e3f0 con 0x5635b23655f0 2024-04-26T21:44:50.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.747+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v117'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.747+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236e3f0 con 0x5635b23655f0 2024-04-26T21:44:50.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.749+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v117'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.749+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.750+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v117'499 uv483 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.750+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.751+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v117'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.751+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.752 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:50.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.752+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v117'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.752+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236ad40 con 0x5635b23655f0 2024-04-26T21:44:50.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.754+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v117'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.754+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236ad40 con 0x5635b23655f0 2024-04-26T21:44:50.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.755+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v117'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.755+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2369d30 con 0x5635b23655f0 2024-04-26T21:44:50.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.756+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v117'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b23655f0 2024-04-26T21:44:50.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.756+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5635b236a3b0 0x5635b236c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.756+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236cce0 con 0x5635b236a3b0 2024-04-26T21:44:50.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.756+0000 7f312ffff640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5635b236a3b0 0x5635b236c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.757+0000 7f312ffff640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5635b236a3b0 0x5635b236c7a0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.758+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v117'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.758+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b236d140 con 0x5635b236a3b0 2024-04-26T21:44:50.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.759+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v117'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.759+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236d620 con 0x5635b236a3b0 2024-04-26T21:44:50.760 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.761+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v117'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.761+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236eed0 con 0x5635b236a3b0 2024-04-26T21:44:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.762+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v117'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.762+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236f270 con 0x5635b236a3b0 2024-04-26T21:44:50.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.763+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v117'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.763+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b236dec0 con 0x5635b236a3b0 2024-04-26T21:44:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.764+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v117'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.764+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5635b236e7e0 0x5635b2371620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.764+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2371b60 con 0x5635b236e7e0 2024-04-26T21:44:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.764+0000 7f3138e90640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5635b236e7e0 0x5635b2371620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.765+0000 7f3138e90640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5635b236e7e0 0x5635b2371620 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.766+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v117'100 uv99 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.766+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2371b60 con 0x5635b236e7e0 2024-04-26T21:44:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.767+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v117'101 uv101 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.767+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2371f50 con 0x5635b236e7e0 2024-04-26T21:44:50.768 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.768+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v117'102 uv102 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.768+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2371f50 con 0x5635b236e7e0 2024-04-26T21:44:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.770+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v117'103 uv103 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.770+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5635b2372460 0x5635b2374850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.770+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2374e20 con 0x5635b2372460 2024-04-26T21:44:50.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.770+0000 7f313868f640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5635b2372460 0x5635b2374850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.770+0000 7f313868f640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5635b2372460 0x5635b2374850 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.772+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v117'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.772+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b2372460 2024-04-26T21:44:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.773+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v117'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.773+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b2372460 2024-04-26T21:44:50.773 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.774+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v117'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.774+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2376200 con 0x5635b2372460 2024-04-26T21:44:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.775+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v117'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.775+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2376200 con 0x5635b2372460 2024-04-26T21:44:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.776+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v117'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.776+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b2372460 2024-04-26T21:44:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.777+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v117'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.777+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.779+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v117'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.779+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.780+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v117'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.780+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.780 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.781+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v117'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.781+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2376200 con 0x5635b236e7e0 2024-04-26T21:44:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.782+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v117'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.782+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2376200 con 0x5635b236e7e0 2024-04-26T21:44:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.783+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v117'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.783+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.784+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v117'243 uv243 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.784+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.785+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v117'244 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.785+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.786+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v117'245 uv245 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.786+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.787 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.787+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v117'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.787+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.788+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v117'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.788+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.789+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v117'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.789+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.793+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v117'107 uv106 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.793+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.794+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v117'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.794+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.795 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.795+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v117'109 uv109 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.795+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.796+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v117'110 uv110 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.796+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b236e7e0 2024-04-26T21:44:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.797+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v117'111 uv111 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f31300026a0 con 0x5635b236e7e0 2024-04-26T21:44:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.797+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b2372460 2024-04-26T21:44:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.799+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v117'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.799+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b2372460 2024-04-26T21:44:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.800+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v117'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.800+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23751f0 con 0x5635b2372460 2024-04-26T21:44:50.801 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.801+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v117'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.801+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23765f0 con 0x5635b2372460 2024-04-26T21:44:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.802+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v117'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.802+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23765f0 con 0x5635b2372460 2024-04-26T21:44:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.803+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v117'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.803+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.804+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v117'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.804+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.806+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v117'213 uv207 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.806+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.807+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v117'214 uv214 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.807+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.807 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.808+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v117'215 uv215 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.808+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.809+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v117'216 uv216 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.809+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.810+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v117'217 uv217 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.810+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.811+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v117'218 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.811+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.812+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v117'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.812+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.813 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.813+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v117'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.813+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23755e0 con 0x5635b2372460 2024-04-26T21:44:50.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.814+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v117'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.814+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b23783f0 con 0x5635b2372460 2024-04-26T21:44:50.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.815+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v117'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.815+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5635b2375fa0 0x5635b2378810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.815+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2375fa0 2024-04-26T21:44:50.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.816+0000 7f312ffff640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5635b2375fa0 0x5635b2378810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.816+0000 7f312ffff640 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5635b2375fa0 0x5635b2378810 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.817+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v117'95 uv94 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b2375fa0 2024-04-26T21:44:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.817+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2375fa0 2024-04-26T21:44:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.818+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v117'96 uv96 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b2375fa0 2024-04-26T21:44:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.818+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2375fa0 2024-04-26T21:44:50.819 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.819+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v117'97 uv97 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b2375fa0 2024-04-26T21:44:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.819+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2375fa0 2024-04-26T21:44:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.820+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v117'98 uv98 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b2375fa0 2024-04-26T21:44:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.820+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2372460 2024-04-26T21:44:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.821+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v117'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.821+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2372460 2024-04-26T21:44:50.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.823+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v117'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.823+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2372460 2024-04-26T21:44:50.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.824+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v117'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.823 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:50.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.824+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2372460 2024-04-26T21:44:50.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.825+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v117'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.825+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378390 con 0x5635b2372460 2024-04-26T21:44:50.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.825+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v117'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.825+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b2372460 2024-04-26T21:44:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.827+0000 7f313868f640 1 -- 172.21.15.40:0/556420585 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v117'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3128070070 con 0x5635b2372460 2024-04-26T21:44:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.827+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.828+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v117'110 uv109 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.828+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.829+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v117'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.829+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.829 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:50.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.830+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v117'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.830+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.831+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v117'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.831+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.832+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v117'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.832+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.833+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v117'265 uv257 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.833+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.834+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v117'266 uv266 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.834+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.835+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v117'267 uv267 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.835 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:50.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.835+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.836+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v117'268 uv268 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.836+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.837+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v117'269 uv269 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.837+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.838+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v117'270 uv264 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.838+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.839+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v117'271 uv271 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.839+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.840 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:50.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.840+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v117'272 uv272 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.840+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2379d60 con 0x5635b236a3b0 2024-04-26T21:44:50.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.841+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v117'273 uv273 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.841+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2379d60 con 0x5635b236a3b0 2024-04-26T21:44:50.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.842+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v117'274 uv274 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.842+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2378d50 con 0x5635b236a3b0 2024-04-26T21:44:50.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.843+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v117'275 uv275 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.843+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5635b2379200 con 0x5635b236a3b0 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f312ffff640 1 -- 172.21.15.40:0/556420585 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v117'276 uv276 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f312001b040 con 0x5635b236a3b0 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5635b23655f0 msgr2=0x5635b23679e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5635b23655f0 0x5635b23679e0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5635b22b8100 msgr2=0x5635b22bc2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5635b22b8100 0x5635b22bc2a0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5635b236e7e0 msgr2=0x5635b2371620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x5635b236e7e0 0x5635b2371620 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5635b2375fa0 msgr2=0x5635b2378810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x5635b2375fa0 0x5635b2378810 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5635b236a3b0 msgr2=0x5635b236c7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x5635b236a3b0 0x5635b236c7a0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5635b2372460 msgr2=0x5635b2374850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5635b2372460 0x5635b2374850 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 reap_dead start 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 reap_dead start 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f3138e90640 1 -- 172.21.15.40:0/556420585 reap_dead start 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f311806b300 msgr2=0x7f311806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f311806b300 0x7f311806d7c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x5635b2217100 tx=0x7f312000c000 comp rx=0 tx=0).stop 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b21ed3e0 msgr2=0x5635b22c2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b21ed3e0 0x5635b22c2cd0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f3128045ef0 tx=0x7f3128002fb0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 shutdown_connections 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f311806b300 0x7f311806d7c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 --2- 172.21.15.40:0/556420585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635b21ed3e0 0x5635b22c2cd0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 >> 172.21.15.40:0/556420585 conn(0x5635b21ea330 msgr2=0x5635b222f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 shutdown_connections 2024-04-26T21:44:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.844+0000 7f313ab95f40 1 -- 172.21.15.40:0/556420585 wait complete. 2024-04-26T21:44:50.848 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 100.00000000.inode - 2024-04-26T21:44:50.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.930+0000 7f39f7c87f40 1 Processor -- start 2024-04-26T21:44:50.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f7c87f40 1 -- start start 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f7c87f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 0x55864f674240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f7c87f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 0x55864f704d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f7c87f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f708c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f7c87f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55864f6772d0 con 0x55864f6eae60 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f7c87f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55864f5d6e40 con 0x55864f6788c0 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f7c87f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55864f677570 con 0x55864f6eaa80 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f5f84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f708c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f5f84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f708c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59106/0 (socket says 172.21.15.40:59106) 2024-04-26T21:44:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.931+0000 7f39f5f84640 1 -- 172.21.15.40:0/2857686503 learned_addr learned my addr 172.21.15.40:0/2857686503 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f5f84640 1 -- 172.21.15.40:0/2857686503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 msgr2=0x55864f704d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f5783640 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 0x55864f674240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f4f82640 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 0x55864f704d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f5f84640 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 0x55864f704d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f5f84640 1 -- 172.21.15.40:0/2857686503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 msgr2=0x55864f674240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f5f84640 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 0x55864f674240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f5f84640 1 -- 172.21.15.40:0/2857686503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864f6585d0 con 0x55864f6eae60 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f5f84640 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f708c10 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f39e800b5d0 tx=0x7f39e800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a84a3c2ca47bef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39e77fe640 1 -- 172.21.15.40:0/2857686503 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8014030 con 0x55864f6eae60 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39e77fe640 1 -- 172.21.15.40:0/2857686503 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e8010620 con 0x55864f6eae60 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39e77fe640 1 -- 172.21.15.40:0/2857686503 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80109f0 con 0x55864f6eae60 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f7c87f40 1 -- 172.21.15.40:0/2857686503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 msgr2=0x55864f708c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f7c87f40 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f708c10 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f39e800b5d0 tx=0x7f39e800ba90 comp rx=0 tx=0).stop 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f7c87f40 1 -- 172.21.15.40:0/2857686503 shutdown_connections 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f7c87f40 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f708c10 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f7c87f40 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 0x55864f704d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f7c87f40 1 --2- 172.21.15.40:0/2857686503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 0x55864f674240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.932+0000 7f39f7c87f40 1 -- 172.21.15.40:0/2857686503 >> 172.21.15.40:0/2857686503 conn(0x55864f62dab0 msgr2=0x55864f62e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.933+0000 7f39f7c87f40 1 -- 172.21.15.40:0/2857686503 shutdown_connections 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.933+0000 7f39f7c87f40 1 -- 172.21.15.40:0/2857686503 wait complete. 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.933+0000 7f39f7c87f40 1 Processor -- start 2024-04-26T21:44:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.933+0000 7f39f7c87f40 1 -- start start 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f7c87f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 0x55864f6ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f7c87f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 0x55864f6e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f7c87f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f79d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f7c87f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55864f658900 con 0x55864f6eae60 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f7c87f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55864f709150 con 0x55864f6788c0 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f7c87f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55864f6edf30 con 0x55864f6eaa80 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f79d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f79d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59112/0 (socket says 172.21.15.40:59112) 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 -- 172.21.15.40:0/1068658640 learned_addr learned my addr 172.21.15.40:0/1068658640 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5783640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 0x55864f6ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f4f82640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 0x55864f6e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 -- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 msgr2=0x55864f6e7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55864f6eaa80 0x55864f6e7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 -- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 msgr2=0x55864f6ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55864f6788c0 0x55864f6ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 -- 172.21.15.40:0/1068658640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864f794bf0 con 0x55864f6eae60 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.934+0000 7f39f5f84640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f79d020 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f39e8007010 tx=0x7f39e80033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.935+0000 7f39e67fc640 1 -- 172.21.15.40:0/1068658640 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e801a9b0 con 0x55864f6eae60 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.935+0000 7f39e67fc640 1 -- 172.21.15.40:0/1068658640 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e801ab50 con 0x55864f6eae60 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.935+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55864f794eb0 con 0x55864f6eae60 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.935+0000 7f39e67fc640 1 -- 172.21.15.40:0/1068658640 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80243f0 con 0x55864f6eae60 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.935+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864f65cbb0 con 0x55864f6eae60 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.936+0000 7f39e67fc640 1 -- 172.21.15.40:0/1068658640 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39e8017070 con 0x55864f6eae60 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.936+0000 7f39e67fc640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39d806b2d0 0x7f39d806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.937+0000 7f39e67fc640 1 -- 172.21.15.40:0/1068658640 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8109+0+0 (secure 0 0 0) 0x7f39e808ca00 con 0x55864f6eae60 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.937+0000 7f39f5783640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39d806b2d0 0x7f39d806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.937+0000 7f39f7c87f40 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55864f79d560 0x55864f6594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.937+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55864f65d0d0 con 0x55864f79d560 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.937+0000 7f39f4f82640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55864f79d560 0x55864f6594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.937+0000 7f39f5783640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39d806b2d0 0x7f39d806d790 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f39f0057000 tx=0x7f39f0060040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.937+0000 7f39f4f82640 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55864f79d560 0x55864f6594d0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f4f82640 1 -- 172.21.15.40:0/1068658640 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 100.00000000.inode [writefull 0~80] v117'277 uv277 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f39ec00f610 con 0x55864f79d560 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55864f79d560 msgr2=0x55864f6594d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55864f79d560 0x55864f6594d0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39d806b2d0 msgr2=0x7f39d806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f39d806b2d0 0x7f39d806d790 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f39f0057000 tx=0x7f39f0060040 comp rx=0 tx=0).stop 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 msgr2=0x55864f79d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 --2- 172.21.15.40:0/1068658640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55864f6eae60 0x55864f79d020 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f39e8007010 tx=0x7f39e80033a0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f5f84640 1 -- 172.21.15.40:0/1068658640 reap_dead start 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 shutdown_connections 2024-04-26T21:44:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 >> 172.21.15.40:0/1068658640 conn(0x55864f62dab0 msgr2=0x55864f6ed510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:50.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 shutdown_connections 2024-04-26T21:44:50.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:50.939+0000 7f39f7c87f40 1 -- 172.21.15.40:0/1068658640 wait complete. 2024-04-26T21:44:50.943 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:51.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.087+0000 7f71f1519640 1 Processor -- start 2024-04-26T21:44:51.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.087+0000 7f71f1519640 1 -- start start 2024-04-26T21:44:51.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71f1519640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71f1519640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71ec0d2390 con 0x7f71ec071090 2024-04-26T21:44:51.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71f1519640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71ec06d3f0 con 0x7f71ec0cfde0 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71f1519640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec071500 con 0x7f71ec070350 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71eb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f71ec071090 0x7f71ec15e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32902/0 (socket says 172.21.15.40:32902) 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71eb7fe640 1 -- 172.21.15.40:0/914310985 learned_addr learned my addr 172.21.15.40:0/914310985 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71ea7fc640 1 --2- 172.21.15.40:0/914310985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71e9ffb640 1 -- 172.21.15.40:0/914310985 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407799985 0 0) 0x7f71ec0d2390 con 0x7f71ec071090 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71e9ffb640 1 -- 172.21.15.40:0/914310985 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71d8003520 con 0x7f71ec071090 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71e9ffb640 1 -- 172.21.15.40:0/914310985 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723467876 0 0) 0x7f71ec06d3f0 con 0x7f71ec0cfde0 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71ea7fc640 1 -- 172.21.15.40:0/914310985 >> v1:172.21.15.184:6789/0 conn(0x7f71ec0cfde0 legacy=0x7f71ec0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71ea7fc640 1 -- 172.21.15.40:0/914310985 >> v1:172.21.15.40:6789/0 conn(0x7f71ec071090 legacy=0x7f71ec15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.088+0000 7f71ea7fc640 1 -- 172.21.15.40:0/914310985 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec06d6d0 con 0x7f71ec070350 2024-04-26T21:44:51.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.089+0000 7f71ea7fc640 1 --2- 172.21.15.40:0/914310985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec070b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f71d40097d0 tx=0x7f71d402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800b5c0fd366d9dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.089+0000 7f71e9ffb640 1 -- 172.21.15.40:0/914310985 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4037070 con 0x7f71ec070350 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.089+0000 7f71e9ffb640 1 -- 172.21.15.40:0/914310985 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d4032440 con 0x7f71ec070350 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.089+0000 7f71e9ffb640 1 -- 172.21.15.40:0/914310985 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4032810 con 0x7f71ec070350 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.090+0000 7f71f1519640 1 -- 172.21.15.40:0/914310985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 msgr2=0x7f71ec070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.090+0000 7f71f1519640 1 --2- 172.21.15.40:0/914310985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec070b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f71d40097d0 tx=0x7f71d402a180 comp rx=0 tx=0).stop 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.090+0000 7f71f1519640 1 -- 172.21.15.40:0/914310985 shutdown_connections 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.090+0000 7f71f1519640 1 --2- 172.21.15.40:0/914310985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec070b50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.090+0000 7f71f1519640 1 -- 172.21.15.40:0/914310985 >> 172.21.15.40:0/914310985 conn(0x7f71ec0d2d80 msgr2=0x7f71ec0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.090+0000 7f71f1519640 1 -- 172.21.15.40:0/914310985 shutdown_connections 2024-04-26T21:44:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.090+0000 7f71f1519640 1 -- 172.21.15.40:0/914310985 wait complete. 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.091+0000 7f71f1519640 1 Processor -- start 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.091+0000 7f71f1519640 1 -- start start 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71f1519640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec1583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71f1519640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ec071090 0x7f71ec0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71f1519640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71ec0cfde0 0x7f71ec15b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71f1519640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec06dd20 con 0x7f71ec0cfde0 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71f1519640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec06f1c0 con 0x7f71ec071090 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71f1519640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec06f340 con 0x7f71ec070350 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71ec0cfde0 0x7f71ec15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71ec0cfde0 0x7f71ec15b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59116/0 (socket says 172.21.15.40:59116) 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71eb7fe640 1 -- 172.21.15.40:0/3402717925 learned_addr learned my addr 172.21.15.40:0/3402717925 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71eaffd640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec1583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.092+0000 7f71ea7fc640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ec071090 0x7f71ec0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71eb7fe640 1 -- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 msgr2=0x7f71ec1583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71eb7fe640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec1583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71eb7fe640 1 -- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ec071090 msgr2=0x7f71ec0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71eb7fe640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ec071090 0x7f71ec0c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71eb7fe640 1 -- 172.21.15.40:0/3402717925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec0c9bd0 con 0x7f71ec0cfde0 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71eb7fe640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71ec0cfde0 0x7f71ec15b470 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00cad0 tx=0x7f71dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71e37fe640 1 -- 172.21.15.40:0/3402717925 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc017070 con 0x7f71ec0cfde0 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71e37fe640 1 -- 172.21.15.40:0/3402717925 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc00ac30 con 0x7f71ec0cfde0 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec1f6c50 con 0x7f71ec0cfde0 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71e37fe640 1 -- 172.21.15.40:0/3402717925 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc012400 con 0x7f71ec0cfde0 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.093+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec1f7210 con 0x7f71ec0cfde0 2024-04-26T21:44:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.094+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec0694f0 con 0x7f71ec0cfde0 2024-04-26T21:44:51.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.095+0000 7f71e37fe640 1 -- 172.21.15.40:0/3402717925 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71dc01a070 con 0x7f71ec0cfde0 2024-04-26T21:44:51.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.095+0000 7f71e37fe640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71cc06c8f0 0x7f71cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.095+0000 7f71e37fe640 1 -- 172.21.15.40:0/3402717925 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8109+0+0 (secure 0 0 0) 0x7f71dc08c1c0 con 0x7f71ec0cfde0 2024-04-26T21:44:51.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.095+0000 7f71eaffd640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71cc06c8f0 0x7f71cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.096+0000 7f71eaffd640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71cc06c8f0 0x7f71cc06edb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f71e4000f60 tx=0x7f71e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.100+0000 7f71e37fe640 1 -- 172.21.15.40:0/3402717925 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71dc059640 con 0x7f71ec0cfde0 2024-04-26T21:44:51.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.326+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f71ec0b2190 con 0x7f71ec0cfde0 2024-04-26T21:44:51.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.626+0000 7f71e37fe640 1 -- 172.21.15.40:0/3402717925 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v230) ==== 162+0+0 (secure 0 0 0) 0x7f71dc08b0b0 con 0x7f71ec0cfde0 2024-04-26T21:44:51.626 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71cc06c8f0 msgr2=0x7f71cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71cc06c8f0 0x7f71cc06edb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f71e4000f60 tx=0x7f71e4009040 comp rx=0 tx=0).stop 2024-04-26T21:44:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71ec0cfde0 msgr2=0x7f71ec15b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71ec0cfde0 0x7f71ec15b470 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00cad0 tx=0x7f71dc00cf90 comp rx=0 tx=0).stop 2024-04-26T21:44:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 shutdown_connections 2024-04-26T21:44:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f71cc06c8f0 0x7f71cc06edb0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71ec0cfde0 0x7f71ec15b470 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ec071090 0x7f71ec0c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 --2- 172.21.15.40:0/3402717925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ec070350 0x7f71ec1583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 >> 172.21.15.40:0/3402717925 conn(0x7f71ec0d2d80 msgr2=0x7f71ec0afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:51.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 shutdown_connections 2024-04-26T21:44:51.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.628+0000 7f71f1519640 1 -- 172.21.15.40:0/3402717925 wait complete. 2024-04-26T21:44:51.637 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:51.637+0000 7f934454f640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-04-26T21:44:51.637 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:51.637+0000 7f934454f640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:44:51.637 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:51.638+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:51.637 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:51.638+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:51.642 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:51.643 INFO:tasks.ceph.mds.c.smithi040.stderr: -27> 2024-04-26T21:44:51.637+0000 7f934454f640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-04-26T21:44:51.643 INFO:tasks.ceph.mds.c.smithi040.stderr: -17> 2024-04-26T21:44:51.637+0000 7f934454f640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:44:51.668 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:51.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.744+0000 7f6722a37640 1 Processor -- start 2024-04-26T21:44:51.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.744+0000 7f6722a37640 1 -- start start 2024-04-26T21:44:51.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.745+0000 7f6722a37640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f671c0d06f0 0x7f671c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.745+0000 7f6722a37640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f671c0b2850 0x7f671c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.745+0000 7f6722a37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671c0d0ba0 con 0x7f671c15f3a0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.745+0000 7f6722a37640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f671c0d0e20 con 0x7f671c0d06f0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.745+0000 7f6722a37640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f671c0b1d70 con 0x7f671c0b2850 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.745+0000 7f671b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f671c0d06f0 0x7f671c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.745+0000 7f671bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f671c15f3a0 0x7f671c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32916/0 (socket says 172.21.15.40:32916) 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.745+0000 7f671bfff640 1 -- 172.21.15.40:0/632411846 learned_addr learned my addr 172.21.15.40:0/632411846 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.746+0000 7f6720fd4640 1 --2- 172.21.15.40:0/632411846 >> v2:172.21.15.184:3300/0 conn(0x7f671c0b2850 0x7f671c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.746+0000 7f671affd640 1 -- 172.21.15.40:0/632411846 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1842933254 0 0) 0x7f670c001110 con 0x7f671c15f3a0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.746+0000 7f671affd640 1 -- 172.21.15.40:0/632411846 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6704003520 con 0x7f671c15f3a0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.746+0000 7f671b7fe640 1 -- 172.21.15.40:0/632411846 >> v2:172.21.15.184:3300/0 conn(0x7f671c0b2850 msgr2=0x7f671c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.746+0000 7f671b7fe640 1 --2- 172.21.15.40:0/632411846 >> v2:172.21.15.184:3300/0 conn(0x7f671c0b2850 0x7f671c0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.746+0000 7f671b7fe640 1 -- 172.21.15.40:0/632411846 >> v1:172.21.15.40:6789/0 conn(0x7f671c15f3a0 legacy=0x7f671c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.746+0000 7f671b7fe640 1 -- 172.21.15.40:0/632411846 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c0b2050 con 0x7f671c0d06f0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.746+0000 7f671b7fe640 1 --2- 172.21.15.40:0/632411846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c0d06f0 0x7f671c0d1e80 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f671c0c89f0 tx=0x7f671002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9234a7d3354ddf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f671affd640 1 -- 172.21.15.40:0/632411846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710037070 con 0x7f671c0d06f0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f671affd640 1 -- 172.21.15.40:0/632411846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6710032440 con 0x7f671c0d06f0 2024-04-26T21:44:51.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f671affd640 1 -- 172.21.15.40:0/632411846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710032810 con 0x7f671c0d06f0 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f6722a37640 1 -- 172.21.15.40:0/632411846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c0d06f0 msgr2=0x7f671c0d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f6722a37640 1 --2- 172.21.15.40:0/632411846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c0d06f0 0x7f671c0d1e80 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f671c0c89f0 tx=0x7f671002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f6722a37640 1 -- 172.21.15.40:0/632411846 shutdown_connections 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f6722a37640 1 --2- 172.21.15.40:0/632411846 >> v2:172.21.15.184:3300/0 conn(0x7f671c0b2850 0x7f671c0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f6722a37640 1 --2- 172.21.15.40:0/632411846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c0d06f0 0x7f671c0d1e80 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f6722a37640 1 -- 172.21.15.40:0/632411846 >> 172.21.15.40:0/632411846 conn(0x7f671c0d4f90 msgr2=0x7f671c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f6722a37640 1 -- 172.21.15.40:0/632411846 shutdown_connections 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.747+0000 7f6722a37640 1 -- 172.21.15.40:0/632411846 wait complete. 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.748+0000 7f6722a37640 1 Processor -- start 2024-04-26T21:44:51.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6722a37640 1 -- start start 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6722a37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c0b2850 0x7f671c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6722a37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f671c0d06f0 0x7f671c0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6722a37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15f3a0 0x7f671c0cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6722a37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f671c15f900 con 0x7f671c15f3a0 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6722a37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f671c15f780 con 0x7f671c0b2850 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6722a37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f671c0d23c0 con 0x7f671c0d06f0 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6720fd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15f3a0 0x7f671c0cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6720fd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15f3a0 0x7f671c0cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59140/0 (socket says 172.21.15.40:59140) 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.749+0000 7f6720fd4640 1 -- 172.21.15.40:0/2363805087 learned_addr learned my addr 172.21.15.40:0/2363805087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f671b7fe640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f671c0d06f0 0x7f671c0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f671bfff640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c0b2850 0x7f671c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6720fd4640 1 -- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f671c0d06f0 msgr2=0x7f671c0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6720fd4640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f671c0d06f0 0x7f671c0cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6720fd4640 1 -- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c0b2850 msgr2=0x7f671c1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6720fd4640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c0b2850 0x7f671c1585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6720fd4640 1 -- 172.21.15.40:0/2363805087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c158b30 con 0x7f671c15f3a0 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f671b7fe640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f671c0d06f0 0x7f671c0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6720fd4640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15f3a0 0x7f671c0cefc0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f6714007e40 tx=0x7f671400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6718ff9640 1 -- 172.21.15.40:0/2363805087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714014620 con 0x7f671c15f3a0 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6718ff9640 1 -- 172.21.15.40:0/2363805087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6714019070 con 0x7f671c15f3a0 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6718ff9640 1 -- 172.21.15.40:0/2363805087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714014a20 con 0x7f671c15f3a0 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c0cd120 con 0x7f671c15f3a0 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.750+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c0cd6e0 con 0x7f671c15f3a0 2024-04-26T21:44:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.751+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c0694f0 con 0x7f671c15f3a0 2024-04-26T21:44:51.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.752+0000 7f6718ff9640 1 -- 172.21.15.40:0/2363805087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6714014bc0 con 0x7f671c15f3a0 2024-04-26T21:44:51.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.752+0000 7f6718ff9640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670806c8f0 0x7f670806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.752+0000 7f6718ff9640 1 -- 172.21.15.40:0/2363805087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8195+0+0 (secure 0 0 0) 0x7f671408ce70 con 0x7f671c15f3a0 2024-04-26T21:44:51.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.752+0000 7f671bfff640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670806c8f0 0x7f670806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.753+0000 7f671bfff640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670806c8f0 0x7f670806edb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f670c0012b0 tx=0x7f670c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:51.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.756+0000 7f6718ff9640 1 -- 172.21.15.40:0/2363805087 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f671c158b30 con 0x7f671c15f3a0 2024-04-26T21:44:51.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.981+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f671c0b6860 con 0x7f671c15f3a0 2024-04-26T21:44:51.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.982+0000 7f6718ff9640 1 -- 172.21.15.40:0/2363805087 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 232 v232) ==== 95+0+4355 (secure 0 0 0) 0x7f671405a890 con 0x7f671c15f3a0 2024-04-26T21:44:51.982 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:51.982 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":232,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":232,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:51.638455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:51.982 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 232 2024-04-26T21:44:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670806c8f0 msgr2=0x7f670806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670806c8f0 0x7f670806edb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f670c0012b0 tx=0x7f670c009040 comp rx=0 tx=0).stop 2024-04-26T21:44:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15f3a0 msgr2=0x7f671c0cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15f3a0 0x7f671c0cefc0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f6714007e40 tx=0x7f671400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 shutdown_connections 2024-04-26T21:44:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670806c8f0 0x7f670806edb0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15f3a0 0x7f671c0cefc0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f671c0d06f0 0x7f671c0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 --2- 172.21.15.40:0/2363805087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c0b2850 0x7f671c1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 >> 172.21.15.40:0/2363805087 conn(0x7f671c0d4f90 msgr2=0x7f671c06e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 shutdown_connections 2024-04-26T21:44:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:51.984+0000 7f6722a37640 1 -- 172.21.15.40:0/2363805087 wait complete. 2024-04-26T21:44:51.994 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:51.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:52.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.136+0000 7fc99a1b4640 1 Processor -- start 2024-04-26T21:44:52.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.136+0000 7fc99a1b4640 1 -- start start 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc99a1b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940d3bc0 0x7fc9940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc99a1b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9940b5f20 0x7fc9940b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc99a1b4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9940b7e50 con 0x7fc994162880 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc99a1b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc9940d0580 con 0x7fc9940b5f20 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc99a1b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940d1800 con 0x7fc9940d3bc0 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc993fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9940b5f20 0x7fc9940b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc993fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9940b5f20 0x7fc9940b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59154/0 (socket says 172.21.15.40:59154) 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc993fff640 1 -- 172.21.15.40:0/212022244 learned_addr learned my addr 172.21.15.40:0/212022244 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc992ffd640 1 --2- 172.21.15.40:0/212022244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940d3bc0 0x7fc9940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc993fff640 1 -- 172.21.15.40:0/212022244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940d3bc0 msgr2=0x7fc9940d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:52.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc993fff640 1 --2- 172.21.15.40:0/212022244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940d3bc0 0x7fc9940d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc993fff640 1 -- 172.21.15.40:0/212022244 >> v1:172.21.15.184:6789/0 conn(0x7fc994162880 legacy=0x7fc9940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:52.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.137+0000 7fc993fff640 1 -- 172.21.15.40:0/212022244 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9940d07d0 con 0x7fc9940b5f20 2024-04-26T21:44:52.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.138+0000 7fc993fff640 1 --2- 172.21.15.40:0/212022244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9940b5f20 0x7fc9940b8300 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009e50 tx=0x7fc98c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1183290eadec9d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.138+0000 7fc9927fc640 1 -- 172.21.15.40:0/212022244 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c036070 con 0x7fc9940b5f20 2024-04-26T21:44:52.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.138+0000 7fc9927fc640 1 -- 172.21.15.40:0/212022244 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c031600 con 0x7fc9940b5f20 2024-04-26T21:44:52.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.138+0000 7fc9927fc640 1 -- 172.21.15.40:0/212022244 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0319d0 con 0x7fc9940b5f20 2024-04-26T21:44:52.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.139+0000 7fc99a1b4640 1 -- 172.21.15.40:0/212022244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9940b5f20 msgr2=0x7fc9940b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.139+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/212022244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9940b5f20 0x7fc9940b8300 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009e50 tx=0x7fc98c02a710 comp rx=0 tx=0).stop 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.139+0000 7fc99a1b4640 1 -- 172.21.15.40:0/212022244 shutdown_connections 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.139+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/212022244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9940b5f20 0x7fc9940b8300 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.139+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/212022244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940d3bc0 0x7fc9940d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.139+0000 7fc99a1b4640 1 -- 172.21.15.40:0/212022244 >> 172.21.15.40:0/212022244 conn(0x7fc9940d4f60 msgr2=0x7fc9940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.139+0000 7fc99a1b4640 1 -- 172.21.15.40:0/212022244 shutdown_connections 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.139+0000 7fc99a1b4640 1 -- 172.21.15.40:0/212022244 wait complete. 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.140+0000 7fc99a1b4640 1 Processor -- start 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.140+0000 7fc99a1b4640 1 -- start start 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc99a1b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940b5f20 0x7fc9941ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc99a1b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9940d3bc0 0x7fc9940cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc99a1b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc994162880 0x7fc9940ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc99a1b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc994162c60 con 0x7fc994162880 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc99a1b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9940d0d90 con 0x7fc9940d3bc0 2024-04-26T21:44:52.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc99a1b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc994162de0 con 0x7fc9940b5f20 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc994162880 0x7fc9940ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc994162880 0x7fc9940ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59170/0 (socket says 172.21.15.40:59170) 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 -- 172.21.15.40:0/3175963622 learned_addr learned my addr 172.21.15.40:0/3175963622 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc9937fe640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940b5f20 0x7fc9941ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 -- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940b5f20 msgr2=0x7fc9941ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940b5f20 0x7fc9941ea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 -- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9940d3bc0 msgr2=0x7fc9940cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9940d3bc0 0x7fc9940cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 -- 172.21.15.40:0/3175963622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9940ce930 con 0x7fc994162880 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.141+0000 7fc993fff640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc994162880 0x7fc9940ce0d0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009e50 tx=0x7fc98c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.142+0000 7fc97bfff640 1 -- 172.21.15.40:0/3175963622 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c036050 con 0x7fc994162880 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.142+0000 7fc97bfff640 1 -- 172.21.15.40:0/3175963622 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c042070 con 0x7fc994162880 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.142+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9941e7eb0 con 0x7fc994162880 2024-04-26T21:44:52.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.142+0000 7fc97bfff640 1 -- 172.21.15.40:0/3175963622 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c009510 con 0x7fc994162880 2024-04-26T21:44:52.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.142+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941e8470 con 0x7fc994162880 2024-04-26T21:44:52.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.143+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9940694f0 con 0x7fc994162880 2024-04-26T21:44:52.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.143+0000 7fc97bfff640 1 -- 172.21.15.40:0/3175963622 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc98c007bd0 con 0x7fc994162880 2024-04-26T21:44:52.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.144+0000 7fc97bfff640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc98006c8f0 0x7fc98006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.144+0000 7fc97bfff640 1 -- 172.21.15.40:0/3175963622 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8195+0+0 (secure 0 0 0) 0x7fc98c0ad360 con 0x7fc994162880 2024-04-26T21:44:52.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.144+0000 7fc9937fe640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc98006c8f0 0x7fc98006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.144+0000 7fc9937fe640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc98006c8f0 0x7fc98006edb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc984001780 tx=0x7fc984009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.148+0000 7fc97bfff640 1 -- 172.21.15.40:0/3175963622 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc98c07ad80 con 0x7fc994162880 2024-04-26T21:44:52.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.371+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc994069b20 con 0x7fc994162880 2024-04-26T21:44:52.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.371+0000 7fc97bfff640 1 -- 172.21.15.40:0/3175963622 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 232 v232) ==== 95+0+4355 (secure 0 0 0) 0x7fc98c07a680 con 0x7fc994162880 2024-04-26T21:44:52.371 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:52.372 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":232,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":232,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:51.638455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:52.372 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 232 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc98006c8f0 msgr2=0x7fc98006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc98006c8f0 0x7fc98006edb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc984001780 tx=0x7fc984009040 comp rx=0 tx=0).stop 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc994162880 msgr2=0x7fc9940ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc994162880 0x7fc9940ce0d0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009e50 tx=0x7fc98c004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 shutdown_connections 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc98006c8f0 0x7fc98006edb0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc994162880 0x7fc9940ce0d0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9940d3bc0 0x7fc9940cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 --2- 172.21.15.40:0/3175963622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940b5f20 0x7fc9941ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.373+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 >> 172.21.15.40:0/3175963622 conn(0x7fc9940d4f60 msgr2=0x7fc9940d21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.374+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 shutdown_connections 2024-04-26T21:44:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.374+0000 7fc99a1b4640 1 -- 172.21.15.40:0/3175963622 wait complete. 2024-04-26T21:44:52.384 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:52.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:52.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.486+0000 7f53fd540640 1 Processor -- start 2024-04-26T21:44:52.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.486+0000 7f53fd540640 1 -- start start 2024-04-26T21:44:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.486+0000 7f53fd540640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 0x7f53f80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.486+0000 7f53fd540640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f53f80d3bc0 0x7f53f80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.486+0000 7f53fd540640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f80b7e50 con 0x7f53f80b5f20 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.486+0000 7f53fd540640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f53f80d0580 con 0x7f53f80d3bc0 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.486+0000 7f53fd540640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f53f80d1800 con 0x7f53f8162880 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.487+0000 7f53f77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f53f80d3bc0 0x7f53f80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.487+0000 7f53f77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f53f80d3bc0 0x7f53f80b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59174/0 (socket says 172.21.15.40:59174) 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.487+0000 7f53f77fe640 1 -- 172.21.15.40:0/3299717382 learned_addr learned my addr 172.21.15.40:0/3299717382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.487+0000 7f53f6ffd640 1 --2- 172.21.15.40:0/3299717382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 0x7f53f80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.487+0000 7f53f77fe640 1 -- 172.21.15.40:0/3299717382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 msgr2=0x7f53f80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.487+0000 7f53f77fe640 1 --2- 172.21.15.40:0/3299717382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 0x7f53f80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.487+0000 7f53f77fe640 1 -- 172.21.15.40:0/3299717382 >> v1:172.21.15.184:6789/0 conn(0x7f53f80b5f20 legacy=0x7f53f80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:52.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.487+0000 7f53f77fe640 1 -- 172.21.15.40:0/3299717382 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f80d07d0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53f77fe640 1 --2- 172.21.15.40:0/3299717382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80b8300 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009e50 tx=0x7f53f002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10828e009467a03b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53f67fc640 1 -- 172.21.15.40:0/3299717382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0036070 con 0x7f53f80d3bc0 2024-04-26T21:44:52.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53f67fc640 1 -- 172.21.15.40:0/3299717382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f0031600 con 0x7f53f80d3bc0 2024-04-26T21:44:52.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53f67fc640 1 -- 172.21.15.40:0/3299717382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f00319d0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53fd540640 1 -- 172.21.15.40:0/3299717382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 msgr2=0x7f53f80b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53fd540640 1 --2- 172.21.15.40:0/3299717382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80b8300 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009e50 tx=0x7f53f002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53fd540640 1 -- 172.21.15.40:0/3299717382 shutdown_connections 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53fd540640 1 --2- 172.21.15.40:0/3299717382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80b8300 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53fd540640 1 --2- 172.21.15.40:0/3299717382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 0x7f53f80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.488+0000 7f53fd540640 1 -- 172.21.15.40:0/3299717382 >> 172.21.15.40:0/3299717382 conn(0x7f53f80d4f60 msgr2=0x7f53f80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.489+0000 7f53fd540640 1 -- 172.21.15.40:0/3299717382 shutdown_connections 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.489+0000 7f53fd540640 1 -- 172.21.15.40:0/3299717382 wait complete. 2024-04-26T21:44:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.490+0000 7f53fd540640 1 Processor -- start 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.490+0000 7f53fd540640 1 -- start start 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.490+0000 7f53fd540640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f80b5f20 0x7f53f8159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.490+0000 7f53fd540640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.490+0000 7f53fd540640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 0x7f53f806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.490+0000 7f53fd540640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8162c60 con 0x7f53f80d3bc0 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.490+0000 7f53fd540640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f53f80d0d90 con 0x7f53f80b5f20 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.490+0000 7f53fd540640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f53f8162de0 con 0x7f53f8162880 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59180/0 (socket says 172.21.15.40:59180) 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 -- 172.21.15.40:0/1098218130 learned_addr learned my addr 172.21.15.40:0/1098218130 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53f6ffd640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f80b5f20 0x7f53f8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53f77fe640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 0x7f53f806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 -- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 msgr2=0x7f53f806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 0x7f53f806cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 -- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f80b5f20 msgr2=0x7f53f8159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f80b5f20 0x7f53f8159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 -- 172.21.15.40:0/1098218130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f80ceba0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53effff640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80cd9c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c430 tx=0x7f53e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53ef7fe640 1 -- 172.21.15.40:0/1098218130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0011790 con 0x7f53f80d3bc0 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53ef7fe640 1 -- 172.21.15.40:0/1098218130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e0017070 con 0x7f53f80d3bc0 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f8159650 con 0x7f53f80d3bc0 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53ef7fe640 1 -- 172.21.15.40:0/1098218130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0011bb0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.491+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f80ce8c0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.492+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f80694f0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.493+0000 7f53ef7fe640 1 -- 172.21.15.40:0/1098218130 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53e001c460 con 0x7f53f80d3bc0 2024-04-26T21:44:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.493+0000 7f53ef7fe640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53d806c8f0 0x7f53d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.493+0000 7f53ef7fe640 1 -- 172.21.15.40:0/1098218130 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8195+0+0 (secure 0 0 0) 0x7f53e008c3f0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.493+0000 7f53f6ffd640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53d806c8f0 0x7f53d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.494+0000 7f53f6ffd640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53d806c8f0 0x7f53d806edb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f53e8006fd0 tx=0x7f53e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.498+0000 7f53ef7fe640 1 -- 172.21.15.40:0/1098218130 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53e0059ea0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.723+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53f80cbaf0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.724+0000 7f53ef7fe640 1 -- 172.21.15.40:0/1098218130 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 233 v233) ==== 95+0+5182 (secure 0 0 0) 0x7f53e00597a0 con 0x7f53f80d3bc0 2024-04-26T21:44:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:52.724 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":233,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10107,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4016455310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4016455310},{"type":"v1","addr":"172.21.15.40:6833","nonce":4016455310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":233}],"filesystems":[{"mdsmap":{"epoch":232,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:51.638455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:52.724 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 233 2024-04-26T21:44:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53d806c8f0 msgr2=0x7f53d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53d806c8f0 0x7f53d806edb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f53e8006fd0 tx=0x7f53e8008040 comp rx=0 tx=0).stop 2024-04-26T21:44:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 msgr2=0x7f53f80cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80cd9c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c430 tx=0x7f53e000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 shutdown_connections 2024-04-26T21:44:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f53d806c8f0 0x7f53d806edb0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53f8162880 0x7f53f806cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80d3bc0 0x7f53f80cd9c0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 --2- 172.21.15.40:0/1098218130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f80b5f20 0x7f53f8159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 >> 172.21.15.40:0/1098218130 conn(0x7f53f80d4f60 msgr2=0x7f53f80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 shutdown_connections 2024-04-26T21:44:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.726+0000 7f53fd540640 1 -- 172.21.15.40:0/1098218130 wait complete. 2024-04-26T21:44:52.737 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 100.00000000.inode' led to DAMAGED state 2024-04-26T21:44:52.737 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 1.00000000.inode' 2024-04-26T21:44:52.737 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:52.737 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:52.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:52.754 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:52.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:52.755 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:52.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:52.810 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:52.810 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:52.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.952+0000 7f34cc659640 1 Processor -- start 2024-04-26T21:44:52.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.952+0000 7f34cc659640 1 -- start start 2024-04-26T21:44:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34cc659640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f34c415f3a0 0x7f34c40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34cc659640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c40d0e20 con 0x7f34c40d1770 2024-04-26T21:44:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34cc659640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c40b1d70 con 0x7f34c40d2330 2024-04-26T21:44:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34cc659640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f34c40b2050 con 0x7f34c415f3a0 2024-04-26T21:44:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c9bf4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f34c40d1770 0x7f34c40d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32930/0 (socket says 172.21.15.40:32930) 2024-04-26T21:44:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c9bf4640 1 -- 172.21.15.40:0/4075722057 learned_addr learned my addr 172.21.15.40:0/4075722057 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860185163 0 0) 0x7f34c40d0e20 con 0x7f34c40d1770 2024-04-26T21:44:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b0003520 con 0x7f34c40d1770 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34ca3f5640 1 --2- 172.21.15.40:0/4075722057 >> v2:172.21.15.184:3300/0 conn(0x7f34c415f3a0 0x7f34c40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3863345006 0 0) 0x7f34b0003520 con 0x7f34c40d1770 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 >> v2:172.21.15.184:3300/0 conn(0x7f34c415f3a0 msgr2=0x7f34c40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 --2- 172.21.15.40:0/4075722057 >> v2:172.21.15.184:3300/0 conn(0x7f34c415f3a0 0x7f34c40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 >> v1:172.21.15.184:6789/0 conn(0x7f34c40d2330 legacy=0x7f34c40b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c40b2330 con 0x7f34c40d1770 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f34b4001690 con 0x7f34c40d1770 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 >> v1:172.21.15.40:6789/0 conn(0x7f34c40d1770 legacy=0x7f34c40d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34b0003a00 0x7f34b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.953+0000 7f34c93f3640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34b0004800 0x7f34b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34c93f3640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40b2050 con 0x7f34b00051c0 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34b000c0c0 con 0x7f34b0003a00 2024-04-26T21:44:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34b000c2c0 con 0x7f34b0004800 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34cabf6640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34ca3f5640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34b0003a00 0x7f34b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34c9bf4640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34b0004800 0x7f34b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34cabf6640 1 -- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34b0004800 msgr2=0x7f34b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34cabf6640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34b0004800 0x7f34b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34cabf6640 1 -- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34b0003a00 msgr2=0x7f34b0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34cabf6640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34b0003a00 0x7f34b0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34cabf6640 1 -- 172.21.15.40:0/4075722057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b0003520 con 0x7f34b00051c0 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.954+0000 7f34ca3f5640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34b0003a00 0x7f34b0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34cabf6640 1 -- 172.21.15.40:0/4075722057 reap_dead start 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34cabf6640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34b000b9c0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f34bc001e50 tx=0x7f34b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f035e58e040fcd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0009690 con 0x7f34b00051c0 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc003970 con 0x7f34b00051c0 2024-04-26T21:44:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34c93f3640 1 -- 172.21.15.40:0/4075722057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc001a70 con 0x7f34b00051c0 2024-04-26T21:44:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34cc659640 1 -- 172.21.15.40:0/4075722057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 msgr2=0x7f34b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34cc659640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34b000b9c0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f34bc001e50 tx=0x7f34b0005830 comp rx=0 tx=0).stop 2024-04-26T21:44:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34cc659640 1 -- 172.21.15.40:0/4075722057 shutdown_connections 2024-04-26T21:44:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34cc659640 1 --2- 172.21.15.40:0/4075722057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34b000b9c0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.955+0000 7f34cc659640 1 -- 172.21.15.40:0/4075722057 >> 172.21.15.40:0/4075722057 conn(0x7f34c40d4f90 msgr2=0x7f34c40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.956+0000 7f34cc659640 1 -- 172.21.15.40:0/4075722057 shutdown_connections 2024-04-26T21:44:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.956+0000 7f34cc659640 1 -- 172.21.15.40:0/4075722057 wait complete. 2024-04-26T21:44:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.957+0000 7f34cc659640 1 Processor -- start 2024-04-26T21:44:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.957+0000 7f34cc659640 1 -- start start 2024-04-26T21:44:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.957+0000 7f34cc659640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34c40b9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.957+0000 7f34cc659640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c415dbd0 0x7f34c41f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.957+0000 7f34cc659640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c41f89b0 0x7f34c41f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.957+0000 7f34cc659640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40d0870 con 0x7f34b00051c0 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.957+0000 7f34cc659640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40d06f0 con 0x7f34c415dbd0 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.957+0000 7f34cc659640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34c40d09f0 con 0x7f34c41f89b0 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34c40b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34c40b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59202/0 (socket says 172.21.15.40:59202) 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 -- 172.21.15.40:0/3796002329 learned_addr learned my addr 172.21.15.40:0/3796002329 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34c9bf4640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c415dbd0 0x7f34c41f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34cabf6640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c41f89b0 0x7f34c41f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 -- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c41f89b0 msgr2=0x7f34c41f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c41f89b0 0x7f34c41f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 -- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c415dbd0 msgr2=0x7f34c41f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c415dbd0 0x7f34c41f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 -- 172.21.15.40:0/3796002329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c40b9cf0 con 0x7f34b00051c0 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34ca3f5640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34c40b9580 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f34c0011930 tx=0x7f34c0011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34baffd640 1 -- 172.21.15.40:0/3796002329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c000fa90 con 0x7f34b00051c0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34baffd640 1 -- 172.21.15.40:0/3796002329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c000fc30 con 0x7f34b00051c0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c40b9fd0 con 0x7f34b00051c0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34baffd640 1 -- 172.21.15.40:0/3796002329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0014440 con 0x7f34b00051c0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.958+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c40ba590 con 0x7f34b00051c0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.959+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c40694f0 con 0x7f34b00051c0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.960+0000 7f34baffd640 1 -- 172.21.15.40:0/3796002329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34c0012040 con 0x7f34b00051c0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.960+0000 7f34baffd640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34a806c920 0x7f34a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.960+0000 7f34baffd640 1 -- 172.21.15.40:0/3796002329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8195+0+0 (secure 0 0 0) 0x7f34c008ca00 con 0x7f34b00051c0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.960+0000 7f34c9bf4640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34a806c920 0x7f34a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.961+0000 7f34c9bf4640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34a806c920 0x7f34a806ede0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f34b4000c00 tx=0x7f34b4000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:52.965+0000 7f34baffd640 1 -- 172.21.15.40:0/3796002329 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34c0019030 con 0x7f34b00051c0 2024-04-26T21:44:53.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.189+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f34c41f8d90 con 0x7f34b00051c0 2024-04-26T21:44:53.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.654+0000 7f34baffd640 1 -- 172.21.15.40:0/3796002329 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v234) ==== 161+0+0 (secure 0 0 0) 0x7f34c005a4b0 con 0x7f34b00051c0 2024-04-26T21:44:53.655 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:53.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34a806c920 msgr2=0x7f34a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:53.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34a806c920 0x7f34a806ede0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f34b4000c00 tx=0x7f34b4000f80 comp rx=0 tx=0).stop 2024-04-26T21:44:53.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 msgr2=0x7f34c40b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:53.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34c40b9580 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f34c0011930 tx=0x7f34c0011df0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 shutdown_connections 2024-04-26T21:44:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f34a806c920 0x7f34a806ede0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c41f89b0 0x7f34c41f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c415dbd0 0x7f34c41f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 --2- 172.21.15.40:0/3796002329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34b00051c0 0x7f34c40b9580 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 >> 172.21.15.40:0/3796002329 conn(0x7f34c40d4f90 msgr2=0x7f34c40d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 shutdown_connections 2024-04-26T21:44:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.657+0000 7f34cc659640 1 -- 172.21.15.40:0/3796002329 wait complete. 2024-04-26T21:44:53.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.775+0000 7efdc202a640 1 Processor -- start 2024-04-26T21:44:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.775+0000 7efdc202a640 1 -- start start 2024-04-26T21:44:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdc202a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efdbc0d3bc0 0x7efdbc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdc202a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0b5f20 0x7efdbc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdc202a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdbc0b7e50 con 0x7efdbc162880 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdc202a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efdbc0d0580 con 0x7efdbc0d3bc0 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdc202a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc0d1800 con 0x7efdbc0b5f20 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdbb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efdbc162880 0x7efdbc0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32938/0 (socket says 172.21.15.40:32938) 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdbb7fe640 1 -- 172.21.15.40:0/113937737 learned_addr learned my addr 172.21.15.40:0/113937737 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdbaffd640 1 --2- 172.21.15.40:0/113937737 >> v2:172.21.15.184:3300/0 conn(0x7efdbc0d3bc0 0x7efdbc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdbbfff640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0b5f20 0x7efdbc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1021173949 0 0) 0x7efdbc0b7e50 con 0x7efdbc162880 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efda4003520 con 0x7efdbc162880 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 130977040 0 0) 0x7efda4003520 con 0x7efdbc162880 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0b5f20 msgr2=0x7efdbc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0b5f20 0x7efdbc0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 >> v2:172.21.15.184:3300/0 conn(0x7efdbc0d3bc0 msgr2=0x7efdbc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 --2- 172.21.15.40:0/113937737 >> v2:172.21.15.184:3300/0 conn(0x7efdbc0d3bc0 0x7efdbc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc0d07d0 con 0x7efdbc162880 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7efdac001670 con 0x7efdbc162880 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 >> v1:172.21.15.40:6789/0 conn(0x7efdbc162880 legacy=0x7efdbc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.776+0000 7efdba7fc640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdba7fc640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0b5f20 0x7efda4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdba7fc640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4004d40 0x7efda40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0d0580 con 0x7efda4004d40 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0d1800 con 0x7efda40039b0 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efda400b9c0 con 0x7efdbc0b5f20 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdbbfff640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4004d40 0x7efda40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdbb7fe640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdbaffd640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0b5f20 0x7efda4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdbbfff640 1 -- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0b5f20 msgr2=0x7efda4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdbbfff640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0b5f20 0x7efda4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdbbfff640 1 -- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40039b0 msgr2=0x7efda4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdbbfff640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.777+0000 7efdbbfff640 1 -- 172.21.15.40:0/113937737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda4003520 con 0x7efda4004d40 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdbbfff640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4004d40 0x7efda40051c0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009fc0 tx=0x7efda4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcd91e8c317c0cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4009c10 con 0x7efda4004d40 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb4036070 con 0x7efda4004d40 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdba7fc640 1 -- 172.21.15.40:0/113937737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4031880 con 0x7efda4004d40 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdc202a640 1 -- 172.21.15.40:0/113937737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4004d40 msgr2=0x7efda40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdc202a640 1 --2- 172.21.15.40:0/113937737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda4004d40 0x7efda40051c0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009fc0 tx=0x7efda4005830 comp rx=0 tx=0).stop 2024-04-26T21:44:53.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdbbfff640 1 -- 172.21.15.40:0/113937737 reap_dead start 2024-04-26T21:44:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdc202a640 1 -- 172.21.15.40:0/113937737 shutdown_connections 2024-04-26T21:44:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdc202a640 1 -- 172.21.15.40:0/113937737 >> 172.21.15.40:0/113937737 conn(0x7efdbc0d4f60 msgr2=0x7efdbc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdc202a640 1 -- 172.21.15.40:0/113937737 shutdown_connections 2024-04-26T21:44:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.778+0000 7efdc202a640 1 -- 172.21.15.40:0/113937737 wait complete. 2024-04-26T21:44:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.780+0000 7efdc202a640 1 Processor -- start 2024-04-26T21:44:53.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.780+0000 7efdc202a640 1 -- start start 2024-04-26T21:44:53.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.780+0000 7efdc202a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc1f4610 0x7efdbc0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.780+0000 7efdc202a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0ceba0 0x7efdbc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.780+0000 7efdc202a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdbc0b5280 0x7efdbc1f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.780+0000 7efdc202a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc162c60 con 0x7efdbc1f4610 2024-04-26T21:44:53.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.780+0000 7efdc202a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0d0d90 con 0x7efdbc0b5280 2024-04-26T21:44:53.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.780+0000 7efdc202a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc162de0 con 0x7efdbc0ceba0 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc1f4610 0x7efdbc0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc1f4610 0x7efdbc0b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59212/0 (socket says 172.21.15.40:59212) 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1927545790 learned_addr learned my addr 172.21.15.40:0/1927545790 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbaffd640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0ceba0 0x7efdbc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbbfff640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdbc0b5280 0x7efdbc1f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0ceba0 msgr2=0x7efdbc0b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0ceba0 0x7efdbc0b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdbc0b5280 msgr2=0x7efdbc1f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdbc0b5280 0x7efdbc1f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1927545790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc15ca10 con 0x7efdbc1f4610 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdbb7fe640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc1f4610 0x7efdbc0b2560 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7efdac0017e0 tx=0x7efdac007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efda3fff640 1 -- 172.21.15.40:0/1927545790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac00f9c0 con 0x7efdbc1f4610 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efda3fff640 1 -- 172.21.15.40:0/1927545790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac018070 con 0x7efdbc1f4610 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc0b2c60 con 0x7efdbc1f4610 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efda3fff640 1 -- 172.21.15.40:0/1927545790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac00fde0 con 0x7efdbc1f4610 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.781+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc0b3220 con 0x7efdbc1f4610 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.782+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdbc0694f0 con 0x7efdbc1f4610 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.783+0000 7efda3fff640 1 -- 172.21.15.40:0/1927545790 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdac01c410 con 0x7efdbc1f4610 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.783+0000 7efda3fff640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda806c920 0x7efda806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.783+0000 7efda3fff640 1 -- 172.21.15.40:0/1927545790 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8195+0+0 (secure 0 0 0) 0x7efdac08c1b0 con 0x7efdbc1f4610 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.783+0000 7efdbaffd640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda806c920 0x7efda806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.784+0000 7efdbaffd640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda806c920 0x7efda806ede0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0ce3d0 tx=0x7efdb0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:53.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:53.787+0000 7efda3fff640 1 -- 172.21.15.40:0/1927545790 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdac059c50 con 0x7efdbc1f4610 2024-04-26T21:44:54.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.045+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7efdbc06da00 con 0x7efdbc1f4610 2024-04-26T21:44:54.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.663+0000 7efda3fff640 1 -- 172.21.15.40:0/1927545790 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v235) ==== 101+0+0 (secure 0 0 0) 0x7efdac059550 con 0x7efdbc1f4610 2024-04-26T21:44:54.663 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:54.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda806c920 msgr2=0x7efda806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda806c920 0x7efda806ede0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0ce3d0 tx=0x7efdb0009190 comp rx=0 tx=0).stop 2024-04-26T21:44:54.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc1f4610 msgr2=0x7efdbc0b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc1f4610 0x7efdbc0b2560 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7efdac0017e0 tx=0x7efdac007660 comp rx=0 tx=0).stop 2024-04-26T21:44:54.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 shutdown_connections 2024-04-26T21:44:54.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efda806c920 0x7efda806ede0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdbc0b5280 0x7efdbc1f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdbc0ceba0 0x7efdbc0b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 --2- 172.21.15.40:0/1927545790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc1f4610 0x7efdbc0b2560 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 >> 172.21.15.40:0/1927545790 conn(0x7efdbc0d4f60 msgr2=0x7efdbc1610a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:54.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 shutdown_connections 2024-04-26T21:44:54.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.665+0000 7efdc202a640 1 -- 172.21.15.40:0/1927545790 wait complete. 2024-04-26T21:44:54.678 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:54.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.723+0000 7f3d877e6f40 1 Processor -- start 2024-04-26T21:44:54.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.723+0000 7f3d877e6f40 1 -- start start 2024-04-26T21:44:54.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d877e6f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d740030f0 0x558829d3aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d877e6f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558829d47900 0x558829d48ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d877e6f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d490e0 0x558829d44210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d877e6f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558829c16e40 con 0x558829d490e0 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d877e6f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558829c1ae80 con 0x7f3d740030f0 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d877e6f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558829c9d010 con 0x558829d47900 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d490e0 0x558829d44210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d490e0 0x558829d44210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59224/0 (socket says 172.21.15.40:59224) 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 -- 172.21.15.40:0/3419607539 learned_addr learned my addr 172.21.15.40:0/3419607539 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 -- 172.21.15.40:0/3419607539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558829d47900 msgr2=0x558829d48ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 --2- 172.21.15.40:0/3419607539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558829d47900 0x558829d48ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 -- 172.21.15.40:0/3419607539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d740030f0 msgr2=0x558829d3aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 --2- 172.21.15.40:0/3419607539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d740030f0 0x558829d3aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 -- 172.21.15.40:0/3419607539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558829d49870 con 0x558829d490e0 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.724+0000 7f3d85ae3640 1 --2- 172.21.15.40:0/3419607539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d490e0 0x558829d44210 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x558829c737b0 tx=0x7f3d78005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348506c8c0ed8a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d7f7fe640 1 -- 172.21.15.40:0/3419607539 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78017030 con 0x558829d490e0 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d7f7fe640 1 -- 172.21.15.40:0/3419607539 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d78012670 con 0x558829d490e0 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d7f7fe640 1 -- 172.21.15.40:0/3419607539 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78012820 con 0x558829d490e0 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 -- 172.21.15.40:0/3419607539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d490e0 msgr2=0x558829d44210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/3419607539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d490e0 0x558829d44210 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x558829c737b0 tx=0x7f3d78005d80 comp rx=0 tx=0).stop 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 -- 172.21.15.40:0/3419607539 shutdown_connections 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/3419607539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d490e0 0x558829d44210 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/3419607539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558829d47900 0x558829d48ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/3419607539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d740030f0 0x558829d3aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 -- 172.21.15.40:0/3419607539 >> 172.21.15.40:0/3419607539 conn(0x558829c6e330 msgr2=0x558829c6e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 -- 172.21.15.40:0/3419607539 shutdown_connections 2024-04-26T21:44:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.725+0000 7f3d877e6f40 1 -- 172.21.15.40:0/3419607539 wait complete. 2024-04-26T21:44:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.726+0000 7f3d877e6f40 1 Processor -- start 2024-04-26T21:44:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.726+0000 7f3d877e6f40 1 -- start start 2024-04-26T21:44:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.726+0000 7f3d877e6f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d47900 0x558829d42ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.726+0000 7f3d877e6f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558829d490e0 0x558829d36b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.726+0000 7f3d877e6f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740030f0 0x558829d37080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.726+0000 7f3d877e6f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558829c6efc0 con 0x558829d47900 2024-04-26T21:44:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.726+0000 7f3d877e6f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558829d49b50 con 0x558829d490e0 2024-04-26T21:44:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.726+0000 7f3d877e6f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558829d49570 con 0x7f3d740030f0 2024-04-26T21:44:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d47900 0x558829d42ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d47900 0x558829d42ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59228/0 (socket says 172.21.15.40:59228) 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 learned_addr learned my addr 172.21.15.40:0/2141948293 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d85ae3640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740030f0 0x558829d37080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d84ae1640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558829d490e0 0x558829d36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740030f0 msgr2=0x558829d37080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740030f0 0x558829d37080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558829d490e0 msgr2=0x558829d36b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558829d490e0 0x558829d36b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558829d44bf0 con 0x558829d47900 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d84ae1640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558829d490e0 0x558829d36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d852e2640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d47900 0x558829d42ef0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f3d800011d0 tx=0x7f3d8000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d7e7fc640 1 -- 172.21.15.40:0/2141948293 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80015020 con 0x558829d47900 2024-04-26T21:44:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d7e7fc640 1 -- 172.21.15.40:0/2141948293 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d80002dd0 con 0x558829d47900 2024-04-26T21:44:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558829d44ec0 con 0x558829d47900 2024-04-26T21:44:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d7e7fc640 1 -- 172.21.15.40:0/2141948293 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80011670 con 0x558829d47900 2024-04-26T21:44:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.727+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558829d45480 con 0x558829d47900 2024-04-26T21:44:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.729+0000 7f3d7e7fc640 1 -- 172.21.15.40:0/2141948293 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d80018020 con 0x558829d47900 2024-04-26T21:44:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.729+0000 7f3d7e7fc640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d6806b300 0x7f3d6806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.729+0000 7f3d7e7fc640 1 -- 172.21.15.40:0/2141948293 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8195+0+0 (secure 0 0 0) 0x7f3d8008b1e0 con 0x558829d47900 2024-04-26T21:44:54.732 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.729+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x558829d43430 0x558829cb4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.729+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829cb45d0 con 0x558829d43430 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.729+0000 7f3d85ae3640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x558829d43430 0x558829cb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.730+0000 7f3d84ae1640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d6806b300 0x7f3d6806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.730+0000 7f3d85ae3640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x558829d43430 0x558829cb4090 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.730+0000 7f3d84ae1640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d6806b300 0x7f3d6806d7c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3d70056000 tx=0x7f3d70050000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.731+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v118'691 uv665 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.731+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829cb4990 con 0x558829d43430 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.733+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v118'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.733+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829cb4d60 con 0x558829d43430 2024-04-26T21:44:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.734+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v118'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.735 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.734+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829cb5100 con 0x558829d43430 2024-04-26T21:44:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.735+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v118'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.735+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deb210 con 0x558829d43430 2024-04-26T21:44:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.736+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v118'695 uv695 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.736+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea200 con 0x558829d43430 2024-04-26T21:44:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.737+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v118'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.737+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea5c0 con 0x558829d43430 2024-04-26T21:44:54.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.738+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v118'697 uv671 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.738+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.739+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v118'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.739+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.740 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:54.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.740+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v118'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.740+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.741+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v118'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.741+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.742+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v118'701 uv701 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.742+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.743+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v118'702 uv702 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.743+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.744+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v118'703 uv678 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.744+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.745+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v118'704 uv704 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.745+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.746+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v118'705 uv705 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.746 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.746+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.747+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v118'706 uv706 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.747+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.748+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v118'707 uv707 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.748+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dea960 con 0x558829d43430 2024-04-26T21:44:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.749+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v118'708 uv708 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.749+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.750+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v118'709 uv709 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.751+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.752+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v118'710 uv684 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.752+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.753+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v118'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.753+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.754+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v118'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.754 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:54.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.754+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dec5c0 con 0x558829d43430 2024-04-26T21:44:54.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.755+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v118'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.755+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dec5c0 con 0x558829d43430 2024-04-26T21:44:54.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.756+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v118'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.756+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.757+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v118'715 uv715 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.757+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.758+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v118'716 uv690 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.758+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.759+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v118'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.759+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.759 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.760+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v118'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.760+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dec5c0 con 0x558829d43430 2024-04-26T21:44:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.761+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v118'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.761+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dec5c0 con 0x558829d43430 2024-04-26T21:44:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.762+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v118'720 uv720 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.762+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deb5b0 con 0x558829d43430 2024-04-26T21:44:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.763+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v118'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829d43430 2024-04-26T21:44:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.763+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x558829debac0 0x558829dedeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.763+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dee3f0 con 0x558829debac0 2024-04-26T21:44:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.763+0000 7f3d852e2640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x558829debac0 0x558829dedeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.763+0000 7f3d852e2640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x558829debac0 0x558829dedeb0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.765+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v118'505 uv488 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.765+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dee790 con 0x558829debac0 2024-04-26T21:44:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.767+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v118'506 uv506 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.767+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829deeb30 con 0x558829debac0 2024-04-26T21:44:54.767 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.768+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v118'507 uv507 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.768+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829def380 con 0x558829debac0 2024-04-26T21:44:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.769+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v118'508 uv508 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.769+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829def720 con 0x558829debac0 2024-04-26T21:44:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.771+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v118'509 uv509 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.771+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829defac0 con 0x558829debac0 2024-04-26T21:44:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.772+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v118'510 uv493 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.772+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829defe60 con 0x558829debac0 2024-04-26T21:44:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.773+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v118'511 uv511 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.773+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.774 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.774+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v118'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.774+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.776+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v118'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.776+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.777+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v118'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.777+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.778+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v118'515 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.778+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.780+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v118'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.780+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.780 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.781+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v118'517 uv517 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.781+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df48c0 con 0x558829debac0 2024-04-26T21:44:54.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.782+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v118'518 uv518 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.782+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df48c0 con 0x558829debac0 2024-04-26T21:44:54.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.784+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v118'519 uv519 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.784+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.785+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v118'520 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.785+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.786+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v118'521 uv521 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.786+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.787 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.787+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v118'522 uv522 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.787+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df1210 con 0x558829debac0 2024-04-26T21:44:54.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.789+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v118'523 uv523 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.789+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df1210 con 0x558829debac0 2024-04-26T21:44:54.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.790+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v118'524 uv524 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.790+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df0200 con 0x558829debac0 2024-04-26T21:44:54.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.791+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v118'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829debac0 2024-04-26T21:44:54.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.791+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x558829df0710 0x558829df2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.791+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df30d0 con 0x558829df0710 2024-04-26T21:44:54.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.791+0000 7f3d84ae1640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x558829df0710 0x558829df2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.792+0000 7f3d84ae1640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x558829df0710 0x558829df2b00 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.793+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v118'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.793+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829df3590 con 0x558829df0710 2024-04-26T21:44:54.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.794+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v118'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.794+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df3a70 con 0x558829df0710 2024-04-26T21:44:54.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.795+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v118'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.795 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:54.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.795+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df5320 con 0x558829df0710 2024-04-26T21:44:54.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.796+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v118'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.796+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df56c0 con 0x558829df0710 2024-04-26T21:44:54.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.797+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v118'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.797+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df4310 con 0x558829df0710 2024-04-26T21:44:54.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.800+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v118'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.800+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558829df4c10 0x558829df7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.800+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df7fb0 con 0x558829df4c10 2024-04-26T21:44:54.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.800+0000 7f3d85ae3640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558829df4c10 0x558829df7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.801+0000 7f3d85ae3640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558829df4c10 0x558829df7a70 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.802+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v118'104 uv103 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.802+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829df7fb0 con 0x558829df4c10 2024-04-26T21:44:54.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.804+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v118'105 uv105 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.804+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df7fb0 con 0x558829df4c10 2024-04-26T21:44:54.804 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:54.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.805+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v118'106 uv106 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.805+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829df83d0 con 0x558829df4c10 2024-04-26T21:44:54.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.806+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v118'107 uv107 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.806+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x558829df88e0 0x558829dfacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.806+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb250 con 0x558829df88e0 2024-04-26T21:44:54.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.806+0000 7f3d852e2640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x558829df88e0 0x558829dfacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.807+0000 7f3d852e2640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x558829df88e0 0x558829dfacd0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.808+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v118'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d80021070 con 0x558829df88e0 2024-04-26T21:44:54.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.808+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dfb310 con 0x558829df88e0 2024-04-26T21:44:54.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.809+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v118'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.809+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.810 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:54.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.810+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v118'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.810+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfc740 con 0x558829df88e0 2024-04-26T21:44:54.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.812+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v118'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.812+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfc740 con 0x558829df88e0 2024-04-26T21:44:54.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.813+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v118'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.813+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.814+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v118'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.814+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.815+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v118'249 uv243 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.815+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.817+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v118'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.817+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.817 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:54.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.818+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v118'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.818+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfc740 con 0x558829df4c10 2024-04-26T21:44:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.819+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v118'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.819+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfc740 con 0x558829df4c10 2024-04-26T21:44:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.820+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v118'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.820+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.821+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v118'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.821+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.823+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v118'255 uv248 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.823+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.824+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v118'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.824+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.825 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:54.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.825+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v118'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.825+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.827+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v118'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.827+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.828+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v118'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.828+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.829+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v118'112 uv111 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.829+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.830+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v118'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.830+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.830 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:54.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.831+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v118'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.831+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.832+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v118'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.832+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df4c10 2024-04-26T21:44:54.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.833+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v118'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d78012ba0 con 0x558829df4c10 2024-04-26T21:44:54.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.833+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.834+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v118'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.834+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.835+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v118'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.835+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.836 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:54.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.836+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v118'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.836+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.837+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v118'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.837+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.838+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v118'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.838+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.839+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v118'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.839+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.840+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v118'223 uv217 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.840+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.842+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v118'224 uv224 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.842+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb730 con 0x558829df88e0 2024-04-26T21:44:54.842 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:54.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.843+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v118'225 uv225 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.843+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb830 con 0x558829df88e0 2024-04-26T21:44:54.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.844+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v118'226 uv226 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.844+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb7f0 con 0x558829df88e0 2024-04-26T21:44:54.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.846+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v118'227 uv227 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.846+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb7d0 con 0x558829df88e0 2024-04-26T21:44:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.847+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v118'228 uv222 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.847+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dfb7d0 con 0x558829df88e0 2024-04-26T21:44:54.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.848+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v118'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.848+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb7d0 con 0x558829df88e0 2024-04-26T21:44:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.849+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v118'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.849 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.849+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfb7d0 con 0x558829df88e0 2024-04-26T21:44:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.850+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v118'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.850+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfe610 con 0x558829df88e0 2024-04-26T21:44:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.852+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v118'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.852+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x558829dfbc10 0x558829dfe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.852+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dfecd0 con 0x558829dfbc10 2024-04-26T21:44:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.852+0000 7f3d84ae1640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x558829dfbc10 0x558829dfe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.853+0000 7f3d84ae1640 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x558829dfbc10 0x558829dfe000 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.854+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v118'99 uv98 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829dfbc10 2024-04-26T21:44:54.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.854+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dff070 con 0x558829dfbc10 2024-04-26T21:44:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.855+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v118'100 uv100 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829dfbc10 2024-04-26T21:44:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.855+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff0f0 con 0x558829dfbc10 2024-04-26T21:44:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.856+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v118'101 uv101 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829dfbc10 2024-04-26T21:44:54.856 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.856+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829dfbc10 2024-04-26T21:44:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.857+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v118'102 uv102 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829dfbc10 2024-04-26T21:44:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.857+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df88e0 2024-04-26T21:44:54.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.858+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v118'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.858+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df88e0 2024-04-26T21:44:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.859+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v118'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.859+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df88e0 2024-04-26T21:44:54.860 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.861+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v118'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.861+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829e004f0 con 0x558829df88e0 2024-04-26T21:44:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.862+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v118'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.862+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829e004f0 con 0x558829df88e0 2024-04-26T21:44:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.863+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v118'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.863+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df88e0 2024-04-26T21:44:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.864+0000 7f3d852e2640 1 -- 172.21.15.40:0/2141948293 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v118'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d80070d70 con 0x558829df88e0 2024-04-26T21:44:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.864+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.865+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v118'115 uv114 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.866+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.867+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v118'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.867+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.868+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v118'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.867 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.868+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.869+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v118'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.869+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.870+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v118'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.870+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.871+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v118'278 uv277 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.871+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.874+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v118'279 uv279 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.874+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.875+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v118'280 uv280 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.875 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.875+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.876+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v118'281 uv281 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.876+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.877+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v118'282 uv282 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.877+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.878+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v118'283 uv276 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.878+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.879+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v118'284 uv284 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.879+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.880+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v118'285 uv285 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.880 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.880+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829e004f0 con 0x558829df0710 2024-04-26T21:44:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.881+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v118'286 uv286 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.881+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829e004f0 con 0x558829df0710 2024-04-26T21:44:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.882+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v118'287 uv287 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.882+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff4e0 con 0x558829df0710 2024-04-26T21:44:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.883+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v118'288 uv288 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.883+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x558829dff990 con 0x558829df0710 2024-04-26T21:44:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d84ae1640 1 -- 172.21.15.40:0/2141948293 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v118'289 uv289 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d70065070 con 0x558829df0710 2024-04-26T21:44:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x558829debac0 msgr2=0x558829dedeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x558829debac0 0x558829dedeb0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x558829d43430 msgr2=0x558829cb4090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x558829d43430 0x558829cb4090 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558829df4c10 msgr2=0x558829df7a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x558829df4c10 0x558829df7a70 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x558829dfbc10 msgr2=0x558829dfe000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x558829dfbc10 0x558829dfe000 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x558829df0710 msgr2=0x558829df2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x558829df0710 0x558829df2b00 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x558829df88e0 msgr2=0x558829dfacd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x558829df88e0 0x558829dfacd0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 reap_dead start 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 reap_dead start 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d85ae3640 1 -- 172.21.15.40:0/2141948293 reap_dead start 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d6806b300 msgr2=0x7f3d6806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d6806b300 0x7f3d6806d7c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3d70056000 tx=0x7f3d70050000 comp rx=0 tx=0).stop 2024-04-26T21:44:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d47900 msgr2=0x558829d42ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d47900 0x558829d42ef0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f3d800011d0 tx=0x7f3d8000c620 comp rx=0 tx=0).stop 2024-04-26T21:44:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 shutdown_connections 2024-04-26T21:44:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3d6806b300 0x7f3d6806d7c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 --2- 172.21.15.40:0/2141948293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558829d47900 0x558829d42ef0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 >> 172.21.15.40:0/2141948293 conn(0x558829c6e330 msgr2=0x558829d438b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 shutdown_connections 2024-04-26T21:44:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.884+0000 7f3d877e6f40 1 -- 172.21.15.40:0/2141948293 wait complete. 2024-04-26T21:44:54.887 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 1.00000000.inode 0 2024-04-26T21:44:54.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.930+0000 7fbc02610f40 1 Processor -- start 2024-04-26T21:44:54.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.930+0000 7fbc02610f40 1 -- start start 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbc02610f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f6bf420 0x55ed5f675f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbc02610f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f746f70 0x55ed5f6769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbc02610f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f74db70 0x55ed5f676ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbc02610f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ed5f61ce40 con 0x55ed5f74db70 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbc02610f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ed5f6be660 con 0x55ed5f746f70 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbc02610f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ed5f6a0390 con 0x55ed5f6bf420 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f74db70 0x55ed5f676ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f74db70 0x55ed5f676ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59240/0 (socket says 172.21.15.40:59240) 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 -- 172.21.15.40:0/2601647718 learned_addr learned my addr 172.21.15.40:0/2601647718 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbfeb99640 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f746f70 0x55ed5f6769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbff39a640 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f6bf420 0x55ed5f675f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 -- 172.21.15.40:0/2601647718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f6bf420 msgr2=0x55ed5f675f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f6bf420 0x55ed5f675f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 -- 172.21.15.40:0/2601647718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f746f70 msgr2=0x55ed5f6769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f746f70 0x55ed5f6769a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 -- 172.21.15.40:0/2601647718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5f6a05e0 con 0x55ed5f74db70 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbffb9b640 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f74db70 0x55ed5f676ee0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8001e90 tx=0x7fbbf800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f95327599f4712a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbfdb97640 1 -- 172.21.15.40:0/2601647718 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8008070 con 0x55ed5f74db70 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbfdb97640 1 -- 172.21.15.40:0/2601647718 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf800d500 con 0x55ed5f74db70 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.931+0000 7fbbfdb97640 1 -- 172.21.15.40:0/2601647718 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800d8d0 con 0x55ed5f74db70 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 -- 172.21.15.40:0/2601647718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f74db70 msgr2=0x55ed5f676ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f74db70 0x55ed5f676ee0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8001e90 tx=0x7fbbf800b6a0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 -- 172.21.15.40:0/2601647718 shutdown_connections 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f74db70 0x55ed5f676ee0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f746f70 0x55ed5f6769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 --2- 172.21.15.40:0/2601647718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f6bf420 0x55ed5f675f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 -- 172.21.15.40:0/2601647718 >> 172.21.15.40:0/2601647718 conn(0x55ed5f69f8a0 msgr2=0x55ed5f69fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 -- 172.21.15.40:0/2601647718 shutdown_connections 2024-04-26T21:44:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.932+0000 7fbc02610f40 1 -- 172.21.15.40:0/2601647718 wait complete. 2024-04-26T21:44:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbc02610f40 1 Processor -- start 2024-04-26T21:44:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbc02610f40 1 -- start start 2024-04-26T21:44:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbc02610f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f6bf420 0x55ed5f7445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbc02610f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f746f70 0x55ed5f6a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbc02610f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f74db70 0x55ed5f7e2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbc02610f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ed5f6a49c0 con 0x55ed5f746f70 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbc02610f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ed5f74fc00 con 0x55ed5f6bf420 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbc02610f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ed5f746a80 con 0x55ed5f74db70 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbbfeb99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f746f70 0x55ed5f6a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbbfeb99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f746f70 0x55ed5f6a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59256/0 (socket says 172.21.15.40:59256) 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbbfeb99640 1 -- 172.21.15.40:0/801010987 learned_addr learned my addr 172.21.15.40:0/801010987 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbbff39a640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f6bf420 0x55ed5f7445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.933+0000 7fbbffb9b640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f74db70 0x55ed5f7e2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfeb99640 1 -- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f74db70 msgr2=0x55ed5f7e2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfeb99640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ed5f74db70 0x55ed5f7e2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfeb99640 1 -- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f6bf420 msgr2=0x55ed5f7445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfeb99640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed5f6bf420 0x55ed5f7445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfeb99640 1 -- 172.21.15.40:0/801010987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5f676970 con 0x55ed5f746f70 2024-04-26T21:44:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfeb99640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f746f70 0x55ed5f6a07b0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8058040 tx=0x7fbbe8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfcb95640 1 -- 172.21.15.40:0/801010987 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8065020 con 0x55ed5f746f70 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfcb95640 1 -- 172.21.15.40:0/801010987 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe805b040 con 0x55ed5f746f70 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbbfcb95640 1 -- 172.21.15.40:0/801010987 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe805a070 con 0x55ed5f746f70 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ed5f676c50 con 0x55ed5f746f70 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.934+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5f6a0cf0 con 0x55ed5f746f70 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.935+0000 7fbbfcb95640 1 -- 172.21.15.40:0/801010987 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbe8068020 con 0x55ed5f746f70 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.936+0000 7fbbfcb95640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe006b2d0 0x7fbbe006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.936+0000 7fbbfcb95640 1 -- 172.21.15.40:0/801010987 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8195+0+0 (secure 0 0 0) 0x7fbbe80cfc50 con 0x55ed5f746f70 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.936+0000 7fbbff39a640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe006b2d0 0x7fbbe006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.936+0000 7fbc02610f40 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ed5f6ba000 0x55ed5f6ba410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.936+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55ed5f6ba950 con 0x55ed5f6ba000 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.936+0000 7fbbffb9b640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ed5f6ba000 0x55ed5f6ba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.936+0000 7fbbff39a640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe006b2d0 0x7fbbe006d790 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000d820 tx=0x7fbbf0007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.936+0000 7fbbffb9b640 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ed5f6ba000 0x55ed5f6ba410 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbbffb9b640 1 -- 172.21.15.40:0/801010987 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 1.00000000.inode [truncate 0] v118'526 uv526 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbbf800d6b0 con 0x55ed5f6ba000 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ed5f6ba000 msgr2=0x55ed5f6ba410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ed5f6ba000 0x55ed5f6ba410 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe006b2d0 msgr2=0x7fbbe006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbbe006b2d0 0x7fbbe006d790 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000d820 tx=0x7fbbf0007040 comp rx=0 tx=0).stop 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f746f70 msgr2=0x55ed5f6a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 --2- 172.21.15.40:0/801010987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed5f746f70 0x55ed5f6a07b0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8058040 tx=0x7fbbe8002fb0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbbffb9b640 1 -- 172.21.15.40:0/801010987 reap_dead start 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 shutdown_connections 2024-04-26T21:44:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 >> 172.21.15.40:0/801010987 conn(0x55ed5f69f8a0 msgr2=0x55ed5f675f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:54.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 shutdown_connections 2024-04-26T21:44:54.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:54.938+0000 7fbc02610f40 1 -- 172.21.15.40:0/801010987 wait complete. 2024-04-26T21:44:54.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.084+0000 7f35edf9d640 1 Processor -- start 2024-04-26T21:44:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.084+0000 7f35edf9d640 1 -- start start 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35edf9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d1a00 0x7f35e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35edf9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35e80d2320 0x7f35e81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35edf9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e80d0a10 con 0x7f35e80b40d0 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35edf9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f35e80d0cf0 con 0x7f35e80d2320 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35edf9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35e80d0fd0 con 0x7f35e80d1a00 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35e6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35e80d2320 0x7f35e81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35e6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35e80d2320 0x7f35e81625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59272/0 (socket says 172.21.15.40:59272) 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35e6ffd640 1 -- 172.21.15.40:0/3915302005 learned_addr learned my addr 172.21.15.40:0/3915302005 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.085+0000 7f35e77fe640 1 --2- 172.21.15.40:0/3915302005 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d1a00 0x7f35e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.086+0000 7f35e6ffd640 1 -- 172.21.15.40:0/3915302005 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d1a00 msgr2=0x7f35e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.086+0000 7f35e6ffd640 1 --2- 172.21.15.40:0/3915302005 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d1a00 0x7f35e80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.086+0000 7f35e6ffd640 1 -- 172.21.15.40:0/3915302005 >> v1:172.21.15.184:6789/0 conn(0x7f35e80b40d0 legacy=0x7f35e80d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.086+0000 7f35e6ffd640 1 -- 172.21.15.40:0/3915302005 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e80d40a0 con 0x7f35e80d2320 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.086+0000 7f35e6ffd640 1 --2- 172.21.15.40:0/3915302005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d2320 0x7f35e81625f0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009870 tx=0x7f35dc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f068898b7c9ded8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.086+0000 7f35e67fc640 1 -- 172.21.15.40:0/3915302005 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc037070 con 0x7f35e80d2320 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35e67fc640 1 -- 172.21.15.40:0/3915302005 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35dc032440 con 0x7f35e80d2320 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35e67fc640 1 -- 172.21.15.40:0/3915302005 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc032810 con 0x7f35e80d2320 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35edf9d640 1 -- 172.21.15.40:0/3915302005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d2320 msgr2=0x7f35e81625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35edf9d640 1 --2- 172.21.15.40:0/3915302005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d2320 0x7f35e81625f0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009870 tx=0x7f35dc02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35edf9d640 1 -- 172.21.15.40:0/3915302005 shutdown_connections 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35edf9d640 1 --2- 172.21.15.40:0/3915302005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d2320 0x7f35e81625f0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35edf9d640 1 --2- 172.21.15.40:0/3915302005 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d1a00 0x7f35e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35edf9d640 1 -- 172.21.15.40:0/3915302005 >> 172.21.15.40:0/3915302005 conn(0x7f35e80d4fd0 msgr2=0x7f35e80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35edf9d640 1 -- 172.21.15.40:0/3915302005 shutdown_connections 2024-04-26T21:44:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.087+0000 7f35edf9d640 1 -- 172.21.15.40:0/3915302005 wait complete. 2024-04-26T21:44:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.088+0000 7f35edf9d640 1 Processor -- start 2024-04-26T21:44:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.088+0000 7f35edf9d640 1 -- start start 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35edf9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35e80b40d0 0x7f35e80c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35edf9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d1a00 0x7f35e81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35edf9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d2320 0x7f35e80cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35edf9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8162b30 con 0x7f35e80d1a00 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35edf9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35e80b5d60 con 0x7f35e80b40d0 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35edf9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35e8162cb0 con 0x7f35e80d2320 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d1a00 0x7f35e81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d1a00 0x7f35e81543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59282/0 (socket says 172.21.15.40:59282) 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35e6ffd640 1 -- 172.21.15.40:0/1481006400 learned_addr learned my addr 172.21.15.40:0/1481006400 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35e77fe640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35e80b40d0 0x7f35e80c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.089+0000 7f35e7fff640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d2320 0x7f35e80cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35e6ffd640 1 -- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d2320 msgr2=0x7f35e80cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35e6ffd640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d2320 0x7f35e80cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35e6ffd640 1 -- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35e80b40d0 msgr2=0x7f35e80c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35e6ffd640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35e80b40d0 0x7f35e80c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35e6ffd640 1 -- 172.21.15.40:0/1481006400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e80c8d60 con 0x7f35e80d1a00 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35e7fff640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d2320 0x7f35e80cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35e6ffd640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d1a00 0x7f35e81543d0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02a200 tx=0x7f35dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35cffff640 1 -- 172.21.15.40:0/1481006400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc041030 con 0x7f35e80d1a00 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35cffff640 1 -- 172.21.15.40:0/1481006400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35dc030440 con 0x7f35e80d1a00 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35cffff640 1 -- 172.21.15.40:0/1481006400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc037040 con 0x7f35e80d1a00 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e80c9040 con 0x7f35e80d1a00 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.090+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e80c9600 con 0x7f35e80d1a00 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.091+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e80694f0 con 0x7f35e80d1a00 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.092+0000 7f35cffff640 1 -- 172.21.15.40:0/1481006400 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35dc044070 con 0x7f35e80d1a00 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.092+0000 7f35cffff640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f35d406c920 0x7f35d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.092+0000 7f35cffff640 1 -- 172.21.15.40:0/1481006400 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8195+0+0 (secure 0 0 0) 0x7f35dc0ac860 con 0x7f35e80d1a00 2024-04-26T21:44:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.092+0000 7f35e77fe640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f35d406c920 0x7f35d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.093+0000 7f35e77fe640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f35d406c920 0x7f35d406ede0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f35e81620c0 tx=0x7f35d8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.096+0000 7f35cffff640 1 -- 172.21.15.40:0/1481006400 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35dc04b070 con 0x7f35e80d1a00 2024-04-26T21:44:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.322+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f35e806d850 con 0x7f35e80d1a00 2024-04-26T21:44:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.664+0000 7f35cffff640 1 -- 172.21.15.40:0/1481006400 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v236) ==== 162+0+0 (secure 0 0 0) 0x7f35dc07a310 con 0x7f35e80d1a00 2024-04-26T21:44:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f35d406c920 msgr2=0x7f35d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f35d406c920 0x7f35d406ede0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f35e81620c0 tx=0x7f35d8009190 comp rx=0 tx=0).stop 2024-04-26T21:44:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d1a00 msgr2=0x7f35e81543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d1a00 0x7f35e81543d0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02a200 tx=0x7f35dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 shutdown_connections 2024-04-26T21:44:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f35d406c920 0x7f35d406ede0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35e80d2320 0x7f35e80cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e80d1a00 0x7f35e81543d0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 --2- 172.21.15.40:0/1481006400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35e80b40d0 0x7f35e80c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 >> 172.21.15.40:0/1481006400 conn(0x7f35e80d4fd0 msgr2=0x7f35e80cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 shutdown_connections 2024-04-26T21:44:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.666+0000 7f35edf9d640 1 -- 172.21.15.40:0/1481006400 wait complete. 2024-04-26T21:44:55.672 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:55.673+0000 7f6cfef19640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-04-26T21:44:55.672 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:55.673+0000 7f6cfef19640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-26T21:44:55.673 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:55.673+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:55.673 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:55.673+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:55.678 INFO:tasks.ceph.mds.c.smithi040.stderr: -16> 2024-04-26T21:44:55.673+0000 7f6cfef19640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-04-26T21:44:55.678 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:44:55.673+0000 7f6cfef19640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-26T21:44:55.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:55.702 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:55.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.782+0000 7f311d869640 1 Processor -- start 2024-04-26T21:44:55.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.782+0000 7f311d869640 1 -- start start 2024-04-26T21:44:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f311d869640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3118162880 0x7f31180d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f311d869640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31180d3bc0 0x7f31180b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f311d869640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31180b7e50 con 0x7f31180b5f20 2024-04-26T21:44:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f311d869640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f31180d0580 con 0x7f31180d3bc0 2024-04-26T21:44:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f311d869640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f31180d1800 con 0x7f3118162880 2024-04-26T21:44:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f31177fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31180d3bc0 0x7f31180b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f31177fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31180d3bc0 0x7f31180b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59306/0 (socket says 172.21.15.40:59306) 2024-04-26T21:44:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f31177fe640 1 -- 172.21.15.40:0/457908627 learned_addr learned my addr 172.21.15.40:0/457908627 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f3116ffd640 1 --2- 172.21.15.40:0/457908627 >> v2:172.21.15.184:3300/0 conn(0x7f3118162880 0x7f31180d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f31177fe640 1 -- 172.21.15.40:0/457908627 >> v2:172.21.15.184:3300/0 conn(0x7f3118162880 msgr2=0x7f31180d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.783+0000 7f31177fe640 1 --2- 172.21.15.40:0/457908627 >> v2:172.21.15.184:3300/0 conn(0x7f3118162880 0x7f31180d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.784+0000 7f31177fe640 1 -- 172.21.15.40:0/457908627 >> v1:172.21.15.184:6789/0 conn(0x7f31180b5f20 legacy=0x7f31180d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.784+0000 7f31177fe640 1 -- 172.21.15.40:0/457908627 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31180d07d0 con 0x7f31180d3bc0 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.784+0000 7f3116ffd640 1 --2- 172.21.15.40:0/457908627 >> v2:172.21.15.184:3300/0 conn(0x7f3118162880 0x7f31180d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.784+0000 7f31177fe640 1 --2- 172.21.15.40:0/457908627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f31180b8300 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f311808d490 tx=0x7f311002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f18c6fbcb04190 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.784+0000 7f31167fc640 1 -- 172.21.15.40:0/457908627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110036070 con 0x7f31180d3bc0 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.784+0000 7f31167fc640 1 -- 172.21.15.40:0/457908627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3110031600 con 0x7f31180d3bc0 2024-04-26T21:44:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.784+0000 7f31167fc640 1 -- 172.21.15.40:0/457908627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100319d0 con 0x7f31180d3bc0 2024-04-26T21:44:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.785+0000 7f311d869640 1 -- 172.21.15.40:0/457908627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 msgr2=0x7f31180b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.785+0000 7f311d869640 1 --2- 172.21.15.40:0/457908627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f31180b8300 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f311808d490 tx=0x7f311002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.785+0000 7f311d869640 1 -- 172.21.15.40:0/457908627 shutdown_connections 2024-04-26T21:44:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.785+0000 7f311d869640 1 --2- 172.21.15.40:0/457908627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f31180b8300 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.785+0000 7f311d869640 1 --2- 172.21.15.40:0/457908627 >> v2:172.21.15.184:3300/0 conn(0x7f3118162880 0x7f31180d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.785+0000 7f311d869640 1 -- 172.21.15.40:0/457908627 >> 172.21.15.40:0/457908627 conn(0x7f31180d4f60 msgr2=0x7f31180d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.785+0000 7f311d869640 1 -- 172.21.15.40:0/457908627 shutdown_connections 2024-04-26T21:44:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.785+0000 7f311d869640 1 -- 172.21.15.40:0/457908627 wait complete. 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.786+0000 7f311d869640 1 Processor -- start 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.786+0000 7f311d869640 1 -- start start 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f311d869640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31180b5f20 0x7f31181f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f311d869640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f3118153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f311d869640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3118162880 0x7f31180b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f311d869640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3118162c60 con 0x7f31180d3bc0 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f311d869640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31180d0d90 con 0x7f3118162880 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f311d869640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3118162de0 con 0x7f31180b5f20 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f310ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f3118153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f310ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f3118153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59316/0 (socket says 172.21.15.40:59316) 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f310ffff640 1 -- 172.21.15.40:0/575558597 learned_addr learned my addr 172.21.15.40:0/575558597 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f31177fe640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3118162880 0x7f31180b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f3116ffd640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31180b5f20 0x7f31181f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f310ffff640 1 -- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31180b5f20 msgr2=0x7f31181f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f310ffff640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31180b5f20 0x7f31181f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f310ffff640 1 -- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3118162880 msgr2=0x7f31180b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f310ffff640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3118162880 0x7f31180b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f310ffff640 1 -- 172.21.15.40:0/575558597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31180b1b50 con 0x7f31180d3bc0 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.787+0000 7f3116ffd640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31180b5f20 0x7f31181f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.788+0000 7f310ffff640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f3118153220 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f310000b3f0 tx=0x7f310000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.788+0000 7f3114ff9640 1 -- 172.21.15.40:0/575558597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100015020 con 0x7f31180d3bc0 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.788+0000 7f3114ff9640 1 -- 172.21.15.40:0/575558597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3100011830 con 0x7f31180d3bc0 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.788+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31181f56a0 con 0x7f31180d3bc0 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.788+0000 7f3114ff9640 1 -- 172.21.15.40:0/575558597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100011c20 con 0x7f31180d3bc0 2024-04-26T21:44:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.788+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31180b1df0 con 0x7f31180d3bc0 2024-04-26T21:44:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.789+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31180694f0 con 0x7f31180d3bc0 2024-04-26T21:44:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.789+0000 7f3114ff9640 1 -- 172.21.15.40:0/575558597 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3100018020 con 0x7f31180d3bc0 2024-04-26T21:44:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.790+0000 7f3114ff9640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f30f806c8f0 0x7f30f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.790+0000 7f3114ff9640 1 -- 172.21.15.40:0/575558597 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8281+0+0 (secure 0 0 0) 0x7f310008c190 con 0x7f31180d3bc0 2024-04-26T21:44:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.790+0000 7f3116ffd640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f30f806c8f0 0x7f30f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.790+0000 7f3116ffd640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f30f806c8f0 0x7f30f806edb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3108006fd0 tx=0x7f3108008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:55.794+0000 7f3114ff9640 1 -- 172.21.15.40:0/575558597 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f310001f030 con 0x7f31180d3bc0 2024-04-26T21:44:56.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.018+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3118161c90 con 0x7f31180d3bc0 2024-04-26T21:44:56.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.018+0000 7f3114ff9640 1 -- 172.21.15.40:0/575558597 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 237 v237) ==== 95+0+4355 (secure 0 0 0) 0x7f31000172c0 con 0x7f31180d3bc0 2024-04-26T21:44:56.018 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:56.018 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":237,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":237,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:55.673812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:56.019 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 237 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.020+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f30f806c8f0 msgr2=0x7f30f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.020+0000 7f311d869640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f30f806c8f0 0x7f30f806edb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3108006fd0 tx=0x7f3108008040 comp rx=0 tx=0).stop 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.020+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 msgr2=0x7f3118153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.020+0000 7f311d869640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f3118153220 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f310000b3f0 tx=0x7f310000b8b0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.021+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 shutdown_connections 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.021+0000 7f311d869640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f30f806c8f0 0x7f30f806edb0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.021+0000 7f311d869640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3118162880 0x7f31180b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.021+0000 7f311d869640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180d3bc0 0x7f3118153220 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.021+0000 7f311d869640 1 --2- 172.21.15.40:0/575558597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31180b5f20 0x7f31181f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.021+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 >> 172.21.15.40:0/575558597 conn(0x7f31180d4f60 msgr2=0x7f31180d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.021+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 shutdown_connections 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.021+0000 7f311d869640 1 -- 172.21.15.40:0/575558597 wait complete. 2024-04-26T21:44:56.031 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:56.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.173+0000 7f2abab40640 1 Processor -- start 2024-04-26T21:44:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.173+0000 7f2abab40640 1 -- start start 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2abab40640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab40d1b30 0x7f2ab40d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2abab40640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab40d10d0 con 0x7f2ab40b2340 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2abab40640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab40d1370 con 0x7f2ab4162810 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2abab40640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab40d0750 con 0x7f2ab40d1b30 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2ab40b2340 0x7f2ab40cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32966/0 (socket says 172.21.15.40:32966) 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab3fff640 1 -- 172.21.15.40:0/3673911292 learned_addr learned my addr 172.21.15.40:0/3673911292 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab90dd640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab40d1b30 0x7f2ab40d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940152210 0 0) 0x7f2ab40d10d0 con 0x7f2ab40b2340 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a9c003520 con 0x7f2ab40b2340 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1163427877 0 0) 0x7f2a9c003520 con 0x7f2ab40b2340 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab40d1b30 msgr2=0x7f2ab40d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab37fe640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab40d1b30 0x7f2ab40d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.174+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 >> v1:172.21.15.184:6789/0 conn(0x7f2ab4162810 legacy=0x7f2ab4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab40d01b0 con 0x7f2ab40b2340 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f2aa8001670 con 0x7f2ab40b2340 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 >> v1:172.21.15.40:6789/0 conn(0x7f2ab40b2340 legacy=0x7f2ab40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab40d1b30 0x7f2a9c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c003f60 0x7f2a9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c004d40 0x7f2a9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab40d0750 con 0x7f2a9c003f60 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c00baa0 con 0x7f2a9c004d40 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c00bca0 con 0x7f2ab40d1b30 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab3fff640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c003f60 0x7f2a9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab88dc640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab40d1b30 0x7f2a9c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab90dd640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c004d40 0x7f2a9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab3fff640 1 -- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab40d1b30 msgr2=0x7f2a9c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab3fff640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab40d1b30 0x7f2a9c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab3fff640 1 -- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c004d40 msgr2=0x7f2a9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab3fff640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c004d40 0x7f2a9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.175+0000 7f2ab3fff640 1 -- 172.21.15.40:0/3673911292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c003520 con 0x7f2a9c003f60 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2ab3fff640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c003f60 0x7f2a9c004800 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c210 tx=0x7f2a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62905552598d19c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8007d30 con 0x7f2a9c003f60 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa8005ad0 con 0x7f2a9c003f60 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2ab37fe640 1 -- 172.21.15.40:0/3673911292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8007070 con 0x7f2a9c003f60 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2abab40640 1 -- 172.21.15.40:0/3673911292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c003f60 msgr2=0x7f2a9c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2abab40640 1 --2- 172.21.15.40:0/3673911292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c003f60 0x7f2a9c004800 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c210 tx=0x7f2a9c005830 comp rx=0 tx=0).stop 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2ab90dd640 1 -- 172.21.15.40:0/3673911292 reap_dead start 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2abab40640 1 -- 172.21.15.40:0/3673911292 shutdown_connections 2024-04-26T21:44:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2abab40640 1 -- 172.21.15.40:0/3673911292 >> 172.21.15.40:0/3673911292 conn(0x7f2ab40d2da0 msgr2=0x7f2ab40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:56.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2abab40640 1 -- 172.21.15.40:0/3673911292 shutdown_connections 2024-04-26T21:44:56.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.176+0000 7f2abab40640 1 -- 172.21.15.40:0/3673911292 wait complete. 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.178+0000 7f2abab40640 1 Processor -- start 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.178+0000 7f2abab40640 1 -- start start 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.178+0000 7f2abab40640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c003f60 0x7f2ab40cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.178+0000 7f2abab40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab40c9470 0x7f2ab40c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.178+0000 7f2abab40640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab414f170 0x7f2ab4070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.178+0000 7f2abab40640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab40cdb00 con 0x7f2ab40c9470 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2abab40640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab40d16a0 con 0x7f2a9c003f60 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2abab40640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab40d1fc0 con 0x7f2ab414f170 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab40c9470 0x7f2ab40c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab40c9470 0x7f2ab40c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59332/0 (socket says 172.21.15.40:59332) 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 -- 172.21.15.40:0/2568330875 learned_addr learned my addr 172.21.15.40:0/2568330875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab88dc640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c003f60 0x7f2ab40cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab90dd640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab414f170 0x7f2ab4070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 -- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab414f170 msgr2=0x7f2ab4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab414f170 0x7f2ab4070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 -- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c003f60 msgr2=0x7f2ab40cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c003f60 0x7f2ab40cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 -- 172.21.15.40:0/2568330875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab40ca3b0 con 0x7f2ab40c9470 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab3fff640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab40c9470 0x7f2ab40c9e70 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c330 tx=0x7f2aa8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab17fa640 1 -- 172.21.15.40:0/2568330875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8042020 con 0x7f2ab40c9470 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab17fa640 1 -- 172.21.15.40:0/2568330875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa8004bf0 con 0x7f2ab40c9470 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab40ca680 con 0x7f2ab40c9470 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2ab17fa640 1 -- 172.21.15.40:0/2568330875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8007d10 con 0x7f2ab40c9470 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.179+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab40cac40 con 0x7f2ab40c9470 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.180+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab40694f0 con 0x7f2ab40c9470 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.181+0000 7f2ab17fa640 1 -- 172.21.15.40:0/2568330875 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2aa8045020 con 0x7f2ab40c9470 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.181+0000 7f2ab17fa640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2aa006c8f0 0x7f2aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.181+0000 7f2ab17fa640 1 -- 172.21.15.40:0/2568330875 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8281+0+0 (secure 0 0 0) 0x7f2aa80ac500 con 0x7f2ab40c9470 2024-04-26T21:44:56.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.181+0000 7f2ab88dc640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2aa006c8f0 0x7f2aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.182+0000 7f2ab88dc640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2aa006c8f0 0x7f2aa006edb0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40017e0 tx=0x7f2aa4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:56.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.185+0000 7f2ab17fa640 1 -- 172.21.15.40:0/2568330875 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2aa804c020 con 0x7f2ab40c9470 2024-04-26T21:44:56.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.412+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ab415d450 con 0x7f2ab40c9470 2024-04-26T21:44:56.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.413+0000 7f2ab17fa640 1 -- 172.21.15.40:0/2568330875 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 237 v237) ==== 95+0+4355 (secure 0 0 0) 0x7f2aa8079910 con 0x7f2ab40c9470 2024-04-26T21:44:56.413 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:56.413 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":237,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":237,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:55.673812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:56.413 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 237 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2aa006c8f0 msgr2=0x7f2aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2aa006c8f0 0x7f2aa006edb0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40017e0 tx=0x7f2aa4009040 comp rx=0 tx=0).stop 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab40c9470 msgr2=0x7f2ab40c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab40c9470 0x7f2ab40c9e70 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c330 tx=0x7f2aa8001a40 comp rx=0 tx=0).stop 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 shutdown_connections 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2aa006c8f0 0x7f2aa006edb0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ab414f170 0x7f2ab4070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab40c9470 0x7f2ab40c9e70 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 --2- 172.21.15.40:0/2568330875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a9c003f60 0x7f2ab40cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 >> 172.21.15.40:0/2568330875 conn(0x7f2ab40d2da0 msgr2=0x7f2ab40d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:56.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 shutdown_connections 2024-04-26T21:44:56.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.415+0000 7f2abab40640 1 -- 172.21.15.40:0/2568330875 wait complete. 2024-04-26T21:44:56.427 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:44:56.427 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.528+0000 7fe24c3c6640 1 Processor -- start 2024-04-26T21:44:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.528+0000 7fe24c3c6640 1 -- start start 2024-04-26T21:44:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24c3c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe244162810 0x7fe244070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24c3c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2440b2340 0x7fe2440cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24c3c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 0x7fe2440d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24c3c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe2440d0750 con 0x7fe244162810 2024-04-26T21:44:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24c3c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe2440d16a0 con 0x7fe2440b2340 2024-04-26T21:44:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24c3c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2440d10d0 con 0x7fe2440cd930 2024-04-26T21:44:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a162640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe244162810 0x7fe244070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a162640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe244162810 0x7fe244070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59336/0 (socket says 172.21.15.40:59336) 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a162640 1 -- 172.21.15.40:0/4108015025 learned_addr learned my addr 172.21.15.40:0/4108015025 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a963640 1 --2- 172.21.15.40:0/4108015025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 0x7fe2440d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a162640 1 -- 172.21.15.40:0/4108015025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 msgr2=0x7fe2440d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a162640 1 --2- 172.21.15.40:0/4108015025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 0x7fe2440d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a162640 1 -- 172.21.15.40:0/4108015025 >> v2:172.21.15.184:3300/0 conn(0x7fe2440b2340 msgr2=0x7fe2440cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe249961640 1 --2- 172.21.15.40:0/4108015025 >> v2:172.21.15.184:3300/0 conn(0x7fe2440b2340 0x7fe2440cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a162640 1 --2- 172.21.15.40:0/4108015025 >> v2:172.21.15.184:3300/0 conn(0x7fe2440b2340 0x7fe2440cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.529+0000 7fe24a162640 1 -- 172.21.15.40:0/4108015025 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2440d1370 con 0x7fe244162810 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.530+0000 7fe24a162640 1 --2- 172.21.15.40:0/4108015025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe244162810 0x7fe244070350 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fe2440cd0a0 tx=0x7fe24002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfd03124fb61bae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.530+0000 7fe249160640 1 -- 172.21.15.40:0/4108015025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240037070 con 0x7fe244162810 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.530+0000 7fe249160640 1 -- 172.21.15.40:0/4108015025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe240032440 con 0x7fe244162810 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.530+0000 7fe249160640 1 -- 172.21.15.40:0/4108015025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240032810 con 0x7fe244162810 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.530+0000 7fe24c3c6640 1 -- 172.21.15.40:0/4108015025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe244162810 msgr2=0x7fe244070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.530+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/4108015025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe244162810 0x7fe244070350 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fe2440cd0a0 tx=0x7fe24002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.531+0000 7fe24c3c6640 1 -- 172.21.15.40:0/4108015025 shutdown_connections 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.531+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/4108015025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 0x7fe2440d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.531+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/4108015025 >> v2:172.21.15.184:3300/0 conn(0x7fe2440b2340 0x7fe2440cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.531+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/4108015025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe244162810 0x7fe244070350 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.531+0000 7fe24c3c6640 1 -- 172.21.15.40:0/4108015025 >> 172.21.15.40:0/4108015025 conn(0x7fe2440d2da0 msgr2=0x7fe2440d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.531+0000 7fe24c3c6640 1 -- 172.21.15.40:0/4108015025 shutdown_connections 2024-04-26T21:44:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.531+0000 7fe24c3c6640 1 -- 172.21.15.40:0/4108015025 wait complete. 2024-04-26T21:44:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.532+0000 7fe24c3c6640 1 Processor -- start 2024-04-26T21:44:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.532+0000 7fe24c3c6640 1 -- start start 2024-04-26T21:44:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.532+0000 7fe24c3c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2440b2340 0x7fe2440cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.532+0000 7fe24c3c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 0x7fe24414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.532+0000 7fe24c3c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe244162810 0x7fe2440c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.532+0000 7fe24c3c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe244070890 con 0x7fe2440b2340 2024-04-26T21:44:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.532+0000 7fe24c3c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2440d0470 con 0x7fe244162810 2024-04-26T21:44:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.532+0000 7fe24c3c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2440d2070 con 0x7fe2440cd930 2024-04-26T21:44:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2440b2340 0x7fe2440cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2440b2340 0x7fe2440cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59350/0 (socket says 172.21.15.40:59350) 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 -- 172.21.15.40:0/3178058070 learned_addr learned my addr 172.21.15.40:0/3178058070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe249961640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 0x7fe24414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a963640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe244162810 0x7fe2440c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 -- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 msgr2=0x7fe24414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 0x7fe24414f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 -- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe244162810 msgr2=0x7fe2440c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe244162810 0x7fe2440c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 -- 172.21.15.40:0/3178058070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2440ccd20 con 0x7fe2440b2340 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24a162640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2440b2340 0x7fe2440cc7e0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fe24002a220 tx=0x7fe2400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe23affd640 1 -- 172.21.15.40:0/3178058070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240037040 con 0x7fe2440b2340 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe23affd640 1 -- 172.21.15.40:0/3178058070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe240042070 con 0x7fe2440b2340 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2440cb740 con 0x7fe2440b2340 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe23affd640 1 -- 172.21.15.40:0/3178058070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240030bb0 con 0x7fe2440b2340 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.533+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2440cbfe0 con 0x7fe2440b2340 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.534+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2440694f0 con 0x7fe2440b2340 2024-04-26T21:44:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.535+0000 7fe23affd640 1 -- 172.21.15.40:0/3178058070 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe240047460 con 0x7fe2440b2340 2024-04-26T21:44:56.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.535+0000 7fe23affd640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe22806c8f0 0x7fe22806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.535+0000 7fe23affd640 1 -- 172.21.15.40:0/3178058070 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8281+0+0 (secure 0 0 0) 0x7fe2400ad190 con 0x7fe2440b2340 2024-04-26T21:44:56.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.535+0000 7fe249961640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe22806c8f0 0x7fe22806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.536+0000 7fe249961640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe22806c8f0 0x7fe22806edb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe244152d00 tx=0x7fe234008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:56.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.539+0000 7fe23affd640 1 -- 172.21.15.40:0/3178058070 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe240045360 con 0x7fe2440b2340 2024-04-26T21:44:56.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.764+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2440d1cb0 con 0x7fe2440b2340 2024-04-26T21:44:56.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.764+0000 7fe23affd640 1 -- 172.21.15.40:0/3178058070 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 238 v238) ==== 95+0+5179 (secure 0 0 0) 0x7fe24007ab60 con 0x7fe2440b2340 2024-04-26T21:44:56.764 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:44:56.764 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":238,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/752925509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":752925509},{"type":"v1","addr":"172.21.15.40:6833","nonce":752925509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":238}],"filesystems":[{"mdsmap":{"epoch":237,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:55.673812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:44:56.765 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 238 2024-04-26T21:44:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe22806c8f0 msgr2=0x7fe22806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe22806c8f0 0x7fe22806edb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe244152d00 tx=0x7fe234008040 comp rx=0 tx=0).stop 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2440b2340 msgr2=0x7fe2440cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2440b2340 0x7fe2440cc7e0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fe24002a220 tx=0x7fe2400046c0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 shutdown_connections 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe22806c8f0 0x7fe22806edb0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe244162810 0x7fe2440c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2440cd930 0x7fe24414f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 --2- 172.21.15.40:0/3178058070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2440b2340 0x7fe2440cc7e0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 >> 172.21.15.40:0/3178058070 conn(0x7fe2440d2da0 msgr2=0x7fe2440b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 shutdown_connections 2024-04-26T21:44:56.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.766+0000 7fe24c3c6640 1 -- 172.21.15.40:0/3178058070 wait complete. 2024-04-26T21:44:56.777 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 1.00000000.inode' led to DAMAGED state 2024-04-26T21:44:56.777 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000000' 2024-04-26T21:44:56.777 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:44:56.777 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:44:56.793 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:44:56.793 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:44:56.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:44:56.793 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:44:56.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:44:56.848 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:44:56.849 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:44:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.990+0000 7fbb188ea640 1 Processor -- start 2024-04-26T21:44:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.990+0000 7fbb188ea640 1 -- start start 2024-04-26T21:44:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb188ea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbb14162810 0x7fbb14070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb188ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb140d1b30 0x7fbb140d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb188ea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb140d10d0 con 0x7fbb140b2340 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb188ea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbb140d0750 con 0x7fbb140d1b30 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb188ea640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbb140d16a0 con 0x7fbb14162810 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb12d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb140d1b30 0x7fbb140d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb11d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbb140b2340 0x7fbb140cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32980/0 (socket says 172.21.15.40:32980) 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb11d9b640 1 -- 172.21.15.40:0/2625760621 learned_addr learned my addr 172.21.15.40:0/2625760621 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.991+0000 7fbb1259c640 1 --2- 172.21.15.40:0/2625760621 >> v2:172.21.15.184:3300/0 conn(0x7fbb14162810 0x7fbb14070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb12d9d640 1 -- 172.21.15.40:0/2625760621 >> v2:172.21.15.184:3300/0 conn(0x7fbb14162810 msgr2=0x7fbb14070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb12d9d640 1 --2- 172.21.15.40:0/2625760621 >> v2:172.21.15.184:3300/0 conn(0x7fbb14162810 0x7fbb14070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb12d9d640 1 -- 172.21.15.40:0/2625760621 >> v1:172.21.15.40:6789/0 conn(0x7fbb140b2340 legacy=0x7fbb140cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb1159a640 1 -- 172.21.15.40:0/2625760621 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794665465 0 0) 0x7fbb140d10d0 con 0x7fbb140b2340 2024-04-26T21:44:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb12d9d640 1 -- 172.21.15.40:0/2625760621 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb140d1370 con 0x7fbb140d1b30 2024-04-26T21:44:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb12d9d640 1 --2- 172.21.15.40:0/2625760621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140d1fc0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fbb14073d90 tx=0x7fbb0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a332e45437a53b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb1159a640 1 -- 172.21.15.40:0/2625760621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04007740 con 0x7fbb140d1b30 2024-04-26T21:44:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb1159a640 1 -- 172.21.15.40:0/2625760621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb040078e0 con 0x7fbb140d1b30 2024-04-26T21:44:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.992+0000 7fbb1159a640 1 -- 172.21.15.40:0/2625760621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04007cb0 con 0x7fbb140d1b30 2024-04-26T21:44:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.993+0000 7fbb188ea640 1 -- 172.21.15.40:0/2625760621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 msgr2=0x7fbb140d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.993+0000 7fbb188ea640 1 --2- 172.21.15.40:0/2625760621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140d1fc0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fbb14073d90 tx=0x7fbb0402a680 comp rx=0 tx=0).stop 2024-04-26T21:44:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.993+0000 7fbb188ea640 1 -- 172.21.15.40:0/2625760621 shutdown_connections 2024-04-26T21:44:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.993+0000 7fbb188ea640 1 --2- 172.21.15.40:0/2625760621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140d1fc0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.993+0000 7fbb188ea640 1 --2- 172.21.15.40:0/2625760621 >> v2:172.21.15.184:3300/0 conn(0x7fbb14162810 0x7fbb14070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.993+0000 7fbb188ea640 1 -- 172.21.15.40:0/2625760621 >> 172.21.15.40:0/2625760621 conn(0x7fbb140d2da0 msgr2=0x7fbb140d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.993+0000 7fbb188ea640 1 -- 172.21.15.40:0/2625760621 shutdown_connections 2024-04-26T21:44:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.993+0000 7fbb188ea640 1 -- 172.21.15.40:0/2625760621 wait complete. 2024-04-26T21:44:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.994+0000 7fbb188ea640 1 Processor -- start 2024-04-26T21:44:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.994+0000 7fbb188ea640 1 -- start start 2024-04-26T21:44:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb188ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140b2340 0x7fbb141619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb188ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb188ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14162810 0x7fbb1414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb188ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14070890 con 0x7fbb140d1b30 2024-04-26T21:44:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb188ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb140d0470 con 0x7fbb14162810 2024-04-26T21:44:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb188ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb140cdb00 con 0x7fbb140b2340 2024-04-26T21:44:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb11d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb11d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59368/0 (socket says 172.21.15.40:59368) 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.995+0000 7fbb11d9b640 1 -- 172.21.15.40:0/3658572760 learned_addr learned my addr 172.21.15.40:0/3658572760 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb12d9d640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14162810 0x7fbb1414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb1259c640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140b2340 0x7fbb141619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb11d9b640 1 -- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140b2340 msgr2=0x7fbb141619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb11d9b640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140b2340 0x7fbb141619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb11d9b640 1 -- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14162810 msgr2=0x7fbb1414d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb11d9b640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14162810 0x7fbb1414d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb11d9b640 1 -- 172.21.15.40:0/3658572760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1414c4c0 con 0x7fbb140d1b30 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb12d9d640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14162810 0x7fbb1414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb1259c640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140b2340 0x7fbb141619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb11d9b640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140cc800 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800c4d0 tx=0x7fbb0800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3658572760 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08018070 con 0x7fbb140d1b30 2024-04-26T21:44:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3658572760 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb08007500 con 0x7fbb140d1b30 2024-04-26T21:44:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1414c7a0 con 0x7fbb140d1b30 2024-04-26T21:44:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3658572760 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08004a20 con 0x7fbb140d1b30 2024-04-26T21:44:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.996+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1414cd60 con 0x7fbb140d1b30 2024-04-26T21:44:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.997+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb140694f0 con 0x7fbb140d1b30 2024-04-26T21:44:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.998+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3658572760 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb0801b070 con 0x7fbb140d1b30 2024-04-26T21:44:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.998+0000 7fbb02ffd640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaf406c8f0 0x7fbaf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.998+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3658572760 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8281+0+0 (secure 0 0 0) 0x7fbb0808b960 con 0x7fbb140d1b30 2024-04-26T21:44:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.998+0000 7fbb1259c640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaf406c8f0 0x7fbaf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:56.999+0000 7fbb1259c640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaf406c8f0 0x7fbaf406edb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006fd0 tx=0x7fbb0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.002+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3658572760 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb08059330 con 0x7fbb140d1b30 2024-04-26T21:44:57.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.228+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbb140b3cc0 con 0x7fbb140d1b30 2024-04-26T21:44:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.681+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3658572760 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v239) ==== 161+0+0 (secure 0 0 0) 0x7fbb08058c30 con 0x7fbb140d1b30 2024-04-26T21:44:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:44:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.683+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaf406c8f0 msgr2=0x7fbaf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.683+0000 7fbb188ea640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaf406c8f0 0x7fbaf406edb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006fd0 tx=0x7fbb0c008040 comp rx=0 tx=0).stop 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 msgr2=0x7fbb140cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140cc800 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800c4d0 tx=0x7fbb0800c990 comp rx=0 tx=0).stop 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 shutdown_connections 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaf406c8f0 0x7fbaf406edb0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14162810 0x7fbb1414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb140d1b30 0x7fbb140cc800 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 --2- 172.21.15.40:0/3658572760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140b2340 0x7fbb141619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 >> 172.21.15.40:0/3658572760 conn(0x7fbb140d2da0 msgr2=0x7fbb140b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 shutdown_connections 2024-04-26T21:44:57.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.684+0000 7fbb188ea640 1 -- 172.21.15.40:0/3658572760 wait complete. 2024-04-26T21:44:57.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:44:57.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.801+0000 7f1c81b46640 1 Processor -- start 2024-04-26T21:44:57.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.801+0000 7f1c81b46640 1 -- start start 2024-04-26T21:44:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c81b46640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1c7c0b2340 0x7f1c7c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c81b46640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c7c0cd930 0x7f1c7c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c81b46640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c7c0d10d0 con 0x7f1c7c162810 2024-04-26T21:44:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c81b46640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1c7c0d0750 con 0x7f1c7c0cd930 2024-04-26T21:44:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c81b46640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1c7c0d16a0 con 0x7f1c7c0b2340 2024-04-26T21:44:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c7bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c7c0cd930 0x7f1c7c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c7bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c7c0cd930 0x7f1c7c0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59376/0 (socket says 172.21.15.40:59376) 2024-04-26T21:44:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c7bfff640 1 -- 172.21.15.40:0/2895308873 learned_addr learned my addr 172.21.15.40:0/2895308873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:57.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c7affd640 1 --2- 172.21.15.40:0/2895308873 >> v2:172.21.15.184:3300/0 conn(0x7f1c7c0b2340 0x7f1c7c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:57.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c7bfff640 1 -- 172.21.15.40:0/2895308873 >> v2:172.21.15.184:3300/0 conn(0x7f1c7c0b2340 msgr2=0x7f1c7c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:57.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c7bfff640 1 --2- 172.21.15.40:0/2895308873 >> v2:172.21.15.184:3300/0 conn(0x7f1c7c0b2340 0x7f1c7c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c7bfff640 1 -- 172.21.15.40:0/2895308873 >> v1:172.21.15.184:6789/0 conn(0x7f1c7c162810 legacy=0x7f1c7c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:57.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.802+0000 7f1c7bfff640 1 -- 172.21.15.40:0/2895308873 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c0d1370 con 0x7f1c7c0cd930 2024-04-26T21:44:57.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.803+0000 7f1c7bfff640 1 --2- 172.21.15.40:0/2895308873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c0cd930 0x7f1c7c0d1b30 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0cd0a0 tx=0x7f1c7402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed9fee45874f9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:57.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.803+0000 7f1c7a7fc640 1 -- 172.21.15.40:0/2895308873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74036070 con 0x7f1c7c0cd930 2024-04-26T21:44:57.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.803+0000 7f1c7a7fc640 1 -- 172.21.15.40:0/2895308873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c74031600 con 0x7f1c7c0cd930 2024-04-26T21:44:57.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.803+0000 7f1c7a7fc640 1 -- 172.21.15.40:0/2895308873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c740319d0 con 0x7f1c7c0cd930 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.804+0000 7f1c81b46640 1 -- 172.21.15.40:0/2895308873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c0cd930 msgr2=0x7f1c7c0d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.804+0000 7f1c81b46640 1 --2- 172.21.15.40:0/2895308873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c0cd930 0x7f1c7c0d1b30 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0cd0a0 tx=0x7f1c7402a730 comp rx=0 tx=0).stop 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.804+0000 7f1c81b46640 1 -- 172.21.15.40:0/2895308873 shutdown_connections 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.804+0000 7f1c81b46640 1 --2- 172.21.15.40:0/2895308873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c0cd930 0x7f1c7c0d1b30 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.804+0000 7f1c81b46640 1 --2- 172.21.15.40:0/2895308873 >> v2:172.21.15.184:3300/0 conn(0x7f1c7c0b2340 0x7f1c7c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.804+0000 7f1c81b46640 1 -- 172.21.15.40:0/2895308873 >> 172.21.15.40:0/2895308873 conn(0x7f1c7c0d2da0 msgr2=0x7f1c7c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.804+0000 7f1c81b46640 1 -- 172.21.15.40:0/2895308873 shutdown_connections 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.804+0000 7f1c81b46640 1 -- 172.21.15.40:0/2895308873 wait complete. 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.805+0000 7f1c81b46640 1 Processor -- start 2024-04-26T21:44:57.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.805+0000 7f1c81b46640 1 -- start start 2024-04-26T21:44:57.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c81b46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c7c0b2340 0x7f1c7c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c81b46640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c7c0cd930 0x7f1c7c15e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c81b46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c162810 0x7f1c7c159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c81b46640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c0d2070 con 0x7f1c7c162810 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c81b46640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c0d0470 con 0x7f1c7c0cd930 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c81b46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7c0d21f0 con 0x7f1c7c0b2340 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c162810 0x7f1c7c159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c162810 0x7f1c7c159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59388/0 (socket says 172.21.15.40:59388) 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 -- 172.21.15.40:0/700279172 learned_addr learned my addr 172.21.15.40:0/700279172 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7b7fe640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c7c0b2340 0x7f1c7c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7affd640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c7c0cd930 0x7f1c7c15e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 -- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c7c0b2340 msgr2=0x7f1c7c1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c7c0b2340 0x7f1c7c1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 -- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c7c0cd930 msgr2=0x7f1c7c15e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c7c0cd930 0x7f1c7c15e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 -- 172.21.15.40:0/700279172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c158230 con 0x7f1c7c162810 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7b7fe640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c7c0b2340 0x7f1c7c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.806+0000 7f1c7bfff640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c162810 0x7f1c7c159fa0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f1c7402ad20 tx=0x7f1c74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.807+0000 7f1c63fff640 1 -- 172.21.15.40:0/700279172 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7403f020 con 0x7f1c7c162810 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.807+0000 7f1c63fff640 1 -- 172.21.15.40:0/700279172 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c74037040 con 0x7f1c7c162810 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.807+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c158510 con 0x7f1c7c162810 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.807+0000 7f1c63fff640 1 -- 172.21.15.40:0/700279172 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74036070 con 0x7f1c7c162810 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.807+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c158ad0 con 0x7f1c7c162810 2024-04-26T21:44:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.808+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7c0694f0 con 0x7f1c7c162810 2024-04-26T21:44:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.808+0000 7f1c63fff640 1 -- 172.21.15.40:0/700279172 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c74042020 con 0x7f1c7c162810 2024-04-26T21:44:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.809+0000 7f1c63fff640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c6806c8f0 0x7f1c6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.809+0000 7f1c63fff640 1 -- 172.21.15.40:0/700279172 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8281+0+0 (secure 0 0 0) 0x7f1c740aca70 con 0x7f1c7c162810 2024-04-26T21:44:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.809+0000 7f1c7b7fe640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c6806c8f0 0x7f1c6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.810+0000 7f1c7b7fe640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c6806c8f0 0x7f1c6806edb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1c700015e0 tx=0x7f1c70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:57.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:57.813+0000 7f1c63fff640 1 -- 172.21.15.40:0/700279172 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c74049030 con 0x7f1c7c162810 2024-04-26T21:44:58.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.071+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f1c7c0cd710 con 0x7f1c7c162810 2024-04-26T21:44:58.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.686+0000 7f1c63fff640 1 -- 172.21.15.40:0/700279172 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v240) ==== 101+0+0 (secure 0 0 0) 0x7f1c7407a440 con 0x7f1c7c162810 2024-04-26T21:44:58.686 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.688+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c6806c8f0 msgr2=0x7f1c6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.688+0000 7f1c81b46640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c6806c8f0 0x7f1c6806edb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1c700015e0 tx=0x7f1c70009040 comp rx=0 tx=0).stop 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.688+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c162810 msgr2=0x7f1c7c159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.688+0000 7f1c81b46640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c162810 0x7f1c7c159fa0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f1c7402ad20 tx=0x7f1c74004a40 comp rx=0 tx=0).stop 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.689+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 shutdown_connections 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.689+0000 7f1c81b46640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1c6806c8f0 0x7f1c6806edb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.689+0000 7f1c81b46640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c7c162810 0x7f1c7c159fa0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.689+0000 7f1c81b46640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c7c0cd930 0x7f1c7c15e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.689+0000 7f1c81b46640 1 --2- 172.21.15.40:0/700279172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c7c0b2340 0x7f1c7c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.689+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 >> 172.21.15.40:0/700279172 conn(0x7f1c7c0d2da0 msgr2=0x7f1c7c06d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:58.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.689+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 shutdown_connections 2024-04-26T21:44:58.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.689+0000 7f1c81b46640 1 -- 172.21.15.40:0/700279172 wait complete. 2024-04-26T21:44:58.705 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:44:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.755+0000 7f6dbb080f40 1 Processor -- start 2024-04-26T21:44:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.755+0000 7f6dbb080f40 1 -- start start 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6dbb080f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7882c8c0 0x55dc788b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6dbb080f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7889ea80 0x55dc78947290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6dbb080f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ee60 0x55dc7894a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6dbb080f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dc7882b2d0 con 0x55dc7889ea80 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6dbb080f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dc7878ae40 con 0x55dc7889ee60 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6dbb080f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55dc7882b570 con 0x55dc7882c8c0 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db8379640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7889ea80 0x55dc78947290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db8379640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7889ea80 0x55dc78947290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50476/0 (socket says 172.21.15.40:50476) 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db8379640 1 -- 172.21.15.40:0/411752663 learned_addr learned my addr 172.21.15.40:0/411752663 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db8b7a640 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7882c8c0 0x55dc788b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db937b640 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ee60 0x55dc7894a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db8379640 1 -- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7882c8c0 msgr2=0x55dc788b9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db8379640 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7882c8c0 0x55dc788b9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db8379640 1 -- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ee60 msgr2=0x55dc7894a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.756+0000 7f6db8379640 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ee60 0x55dc7894a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.757+0000 7f6db8379640 1 -- 172.21.15.40:0/411752663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7889d3a0 con 0x55dc7889ea80 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.757+0000 7f6db937b640 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ee60 0x55dc7894a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.757+0000 7f6db8379640 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7889ea80 0x55dc78947290 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f6da0001e30 tx=0x7f6da000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfe44193e5b8aec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.757+0000 7f6db7377640 1 -- 172.21.15.40:0/411752663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0012020 con 0x55dc7889ea80 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.757+0000 7f6db7377640 1 -- 172.21.15.40:0/411752663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da000e670 con 0x55dc7889ea80 2024-04-26T21:44:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.757+0000 7f6db7377640 1 -- 172.21.15.40:0/411752663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da000eac0 con 0x55dc7889ea80 2024-04-26T21:44:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.757+0000 7f6dbb080f40 1 -- 172.21.15.40:0/411752663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7889ea80 msgr2=0x55dc78947290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.757+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7889ea80 0x55dc78947290 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f6da0001e30 tx=0x7f6da000b780 comp rx=0 tx=0).stop 2024-04-26T21:44:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 -- 172.21.15.40:0/411752663 shutdown_connections 2024-04-26T21:44:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ee60 0x55dc7894a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7889ea80 0x55dc78947290 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/411752663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7882c8c0 0x55dc788b9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 -- 172.21.15.40:0/411752663 >> 172.21.15.40:0/411752663 conn(0x55dc787e1ab0 msgr2=0x55dc787e2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 -- 172.21.15.40:0/411752663 shutdown_connections 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 -- 172.21.15.40:0/411752663 wait complete. 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 Processor -- start 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.758+0000 7f6dbb080f40 1 -- start start 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6dbb080f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7882c8c0 0x55dc7894fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6dbb080f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ea80 0x55dc7894f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6dbb080f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7889ee60 0x55dc78825020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6dbb080f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dc788b8e60 con 0x55dc7882c8c0 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6dbb080f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dc788a0890 con 0x55dc7889ea80 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6dbb080f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55dc7889dc90 con 0x55dc7889ee60 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8b7a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7882c8c0 0x55dc7894fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8b7a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7882c8c0 0x55dc7894fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50480/0 (socket says 172.21.15.40:50480) 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 learned_addr learned my addr 172.21.15.40:0/1697220833 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8379640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ea80 0x55dc7894f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db937b640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7889ee60 0x55dc78825020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7889ee60 msgr2=0x55dc78825020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8b7a640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7889ee60 0x55dc78825020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ea80 msgr2=0x55dc7894f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8b7a640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ea80 0x55dc7894f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc78950520 con 0x55dc7882c8c0 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db8379640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc7889ea80 0x55dc7894f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.759+0000 7f6db937b640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55dc7889ee60 0x55dc78825020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.760+0000 7f6db8b7a640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7882c8c0 0x55dc7894fdf0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f6da80259e0 tx=0x7f6da8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.760+0000 7f6db6375640 1 -- 172.21.15.40:0/1697220833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8056040 con 0x55dc7882c8c0 2024-04-26T21:44:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.760+0000 7f6db6375640 1 -- 172.21.15.40:0/1697220833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da8065640 con 0x55dc7882c8c0 2024-04-26T21:44:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.760+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc78950800 con 0x55dc7882c8c0 2024-04-26T21:44:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.760+0000 7f6db6375640 1 -- 172.21.15.40:0/1697220833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8005a80 con 0x55dc7882c8c0 2024-04-26T21:44:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.760+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc78950dc0 con 0x55dc7882c8c0 2024-04-26T21:44:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.761+0000 7f6db6375640 1 -- 172.21.15.40:0/1697220833 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6da8059020 con 0x55dc7882c8c0 2024-04-26T21:44:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.762+0000 7f6db6375640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d9806b2d0 0x7f6d9806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.762+0000 7f6db6375640 1 -- 172.21.15.40:0/1697220833 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8281+0+0 (secure 0 0 0) 0x7f6da80cef00 con 0x55dc7882c8c0 2024-04-26T21:44:58.768 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:44:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.762+0000 7f6db8379640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d9806b2d0 0x7f6d9806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.762+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55dc788b9f40 0x55dc789515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.762+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78951b00 con 0x55dc788b9f40 2024-04-26T21:44:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.762+0000 7f6db937b640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55dc788b9f40 0x55dc789515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.763+0000 7f6db937b640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55dc788b9f40 0x55dc789515c0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.763+0000 7f6db8379640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d9806b2d0 0x7f6d9806d790 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6da0001e30 tx=0x7f6da0009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.764+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v119'722 uv696 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.764+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7895b270 con 0x55dc788b9f40 2024-04-26T21:44:58.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.765+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v119'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.765+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895b610 con 0x55dc788b9f40 2024-04-26T21:44:58.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.766+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v119'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.771 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:44:58.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.767+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c9c0 con 0x55dc788b9f40 2024-04-26T21:44:58.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.768+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v119'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.768+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895cd60 con 0x55dc788b9f40 2024-04-26T21:44:58.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.769+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v119'726 uv726 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.769+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895b9b0 con 0x55dc788b9f40 2024-04-26T21:44:58.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.770+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v119'727 uv727 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.770+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895bda0 con 0x55dc788b9f40 2024-04-26T21:44:58.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.771+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v119'728 uv702 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.771+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7895c190 con 0x55dc788b9f40 2024-04-26T21:44:58.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.772+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v119'729 uv729 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.772+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c190 con 0x55dc788b9f40 2024-04-26T21:44:58.775 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:44:58.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.773+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v119'730 uv730 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.773+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.774+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v119'731 uv731 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.774+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.775+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v119'732 uv732 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.775+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.776+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v119'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.776+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.777+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v119'734 uv709 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.777+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.779+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v119'735 uv735 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.779+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.780+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v119'736 uv736 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.780 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:44:58.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.780+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.781+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v119'737 uv737 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.781+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.782+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v119'738 uv738 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.782+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c140 con 0x55dc788b9f40 2024-04-26T21:44:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.783+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v119'739 uv739 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.783+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c550 con 0x55dc788b9f40 2024-04-26T21:44:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.784+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v119'740 uv740 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.784+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c520 con 0x55dc788b9f40 2024-04-26T21:44:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.786+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v119'741 uv715 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.786+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7895c520 con 0x55dc788b9f40 2024-04-26T21:44:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.787+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v119'742 uv742 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.787+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c520 con 0x55dc788b9f40 2024-04-26T21:44:58.787 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:44:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.788+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v119'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.788+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.789+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v119'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.789+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.790+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v119'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.790+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.791+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v119'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.791+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.792+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v119'747 uv721 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.792+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.793+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v119'748 uv748 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.793+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.794+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v119'749 uv749 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.794 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:44:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.794+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.796+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v119'750 uv750 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.796+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.797+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v119'751 uv751 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.797+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc788b9f40 2024-04-26T21:44:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.798+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v119'752 uv752 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc788b9f40 2024-04-26T21:44:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.798+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc7895d610 0x55dc7895fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.798+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7895c3f0 con 0x55dc7895d610 2024-04-26T21:44:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.798+0000 7f6db8b7a640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc7895d610 0x55dc7895fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.798+0000 7f6db8b7a640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc7895d610 0x55dc7895fa00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.800+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v119'527 uv526 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.800+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7895ff40 con 0x55dc7895d610 2024-04-26T21:44:58.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.802+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v119'528 uv528 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.802+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789602e0 con 0x55dc7895d610 2024-04-26T21:44:58.802 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:44:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.803+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v119'529 uv529 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.803+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78960b30 con 0x55dc7895d610 2024-04-26T21:44:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.804+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v119'530 uv530 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.804+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78960ed0 con 0x55dc7895d610 2024-04-26T21:44:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.805+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v119'531 uv531 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.805+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78961270 con 0x55dc7895d610 2024-04-26T21:44:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.806+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v119'532 uv514 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.806+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc78961610 con 0x55dc7895d610 2024-04-26T21:44:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.808+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v119'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.808+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.808 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:44:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.809+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v119'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.809+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.810+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v119'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.810+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.811+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v119'536 uv536 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.811+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.812+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v119'537 uv519 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.812+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.813+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v119'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.813+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.814 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:44:58.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.815+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v119'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.815+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78966070 con 0x55dc7895d610 2024-04-26T21:44:58.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.816+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v119'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.816+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78966070 con 0x55dc7895d610 2024-04-26T21:44:58.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.817+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v119'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.817+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.820+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v119'542 uv525 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.820+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.822+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v119'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.822+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.823 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:44:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.823+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v119'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.823+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789629c0 con 0x55dc7895d610 2024-04-26T21:44:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.825+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v119'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.825+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789629c0 con 0x55dc7895d610 2024-04-26T21:44:58.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.826+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v119'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.826+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789619b0 con 0x55dc7895d610 2024-04-26T21:44:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.827+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v119'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7895d610 2024-04-26T21:44:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.827+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55dc78962030 0x55dc78964420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.827+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78964960 con 0x55dc78962030 2024-04-26T21:44:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.827+0000 7f6db8379640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55dc78962030 0x55dc78964420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.827+0000 7f6db8379640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55dc78962030 0x55dc78964420 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.829+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v119'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.829+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc78964dd0 con 0x55dc78962030 2024-04-26T21:44:58.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.830+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v119'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.830+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789652b0 con 0x55dc78962030 2024-04-26T21:44:58.830 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:44:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.831+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v119'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.831+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78966b60 con 0x55dc78962030 2024-04-26T21:44:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.832+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v119'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.832+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78966f00 con 0x55dc78962030 2024-04-26T21:44:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.833+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v119'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.833+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78965b50 con 0x55dc78962030 2024-04-26T21:44:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.834+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v119'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.834+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dc78966450 0x55dc789692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.834+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789697f0 con 0x55dc78966450 2024-04-26T21:44:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.834+0000 7f6db937b640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dc78966450 0x55dc789692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.835+0000 7f6db937b640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dc78966450 0x55dc789692b0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.836+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v119'108 uv107 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.836+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc789697f0 con 0x55dc78966450 2024-04-26T21:44:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.837+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v119'109 uv109 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.837+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78969810 con 0x55dc78966450 2024-04-26T21:44:58.837 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:44:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.838+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v119'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.838+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78969c50 con 0x55dc78966450 2024-04-26T21:44:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.839+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v119'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.839+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55dc7896a160 0x55dc7896c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.839+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896ca90 con 0x55dc7896a160 2024-04-26T21:44:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.839+0000 7f6db8b7a640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55dc7896a160 0x55dc7896c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.840+0000 7f6db8b7a640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55dc7896a160 0x55dc7896c550 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.841+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v119'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.841+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7896cb70 con 0x55dc7896a160 2024-04-26T21:44:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.842+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v119'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.842+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.843 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:44:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.844+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v119'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.844+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896dfa0 con 0x55dc7896a160 2024-04-26T21:44:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.845+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v119'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.845+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896dfa0 con 0x55dc7896a160 2024-04-26T21:44:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.846+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v119'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.846+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.847+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v119'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.847+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.849+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v119'260 uv254 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.849+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.850+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v119'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.850+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.851 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:44:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.851+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v119'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.851+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896dfa0 con 0x55dc78966450 2024-04-26T21:44:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.852+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v119'263 uv263 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.852+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896dfa0 con 0x55dc78966450 2024-04-26T21:44:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.853+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v119'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.854+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.855+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v119'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.855+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.856+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v119'266 uv259 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.856+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.857+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v119'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.857+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.858 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:44:58.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.858+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v119'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.858+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.859+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v119'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.859+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.860+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v119'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.860+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.861+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v119'117 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.861+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.863+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v119'118 uv118 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.863+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.864+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v119'119 uv119 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.864 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:44:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.864+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.865+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v119'120 uv120 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.865+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc78966450 2024-04-26T21:44:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.866+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v119'121 uv121 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6db0009890 con 0x55dc78966450 2024-04-26T21:44:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.866+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.867+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v119'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.867+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.868+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v119'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.868+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.868 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:44:58.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.869+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v119'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.869+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896dfa0 con 0x55dc7896a160 2024-04-26T21:44:58.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.870+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v119'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.870+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896dfa0 con 0x55dc7896a160 2024-04-26T21:44:58.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.871+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v119'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.871+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.872+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v119'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.872+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.873+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v119'233 uv227 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.873+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.874+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v119'234 uv234 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.874+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896cf90 con 0x55dc7896a160 2024-04-26T21:44:58.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.875+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v119'235 uv235 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.875 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:44:58.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.875+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896d350 con 0x55dc7896a160 2024-04-26T21:44:58.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.876+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v119'236 uv236 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.876+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896d070 con 0x55dc7896a160 2024-04-26T21:44:58.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.877+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v119'237 uv237 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.877+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896d070 con 0x55dc7896a160 2024-04-26T21:44:58.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.878+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v119'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.878+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc7896d070 con 0x55dc7896a160 2024-04-26T21:44:58.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.880+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v119'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.880+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896d050 con 0x55dc7896a160 2024-04-26T21:44:58.881 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:44:58.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.881+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v119'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.881+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896d050 con 0x55dc7896a160 2024-04-26T21:44:58.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.882+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v119'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.882+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896fe60 con 0x55dc7896a160 2024-04-26T21:44:58.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.883+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v119'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.883+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55dc7896db60 0x55dc789704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:58.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.883+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc7896ff50 con 0x55dc7896db60 2024-04-26T21:44:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.884+0000 7f6db8379640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55dc7896db60 0x55dc789704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.884+0000 7f6db8379640 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55dc7896db60 0x55dc789704a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.885+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v119'103 uv102 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc7896db60 2024-04-26T21:44:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.885+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc7896db60 2024-04-26T21:44:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.886+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v119'104 uv104 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc7896db60 2024-04-26T21:44:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.886+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc7896db60 2024-04-26T21:44:58.887 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:44:58.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.887+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v119'105 uv105 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc7896db60 2024-04-26T21:44:58.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.887+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc7896db60 2024-04-26T21:44:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.888+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v119'106 uv106 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc7896db60 2024-04-26T21:44:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.888+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc7896a160 2024-04-26T21:44:58.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.889+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v119'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.889+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc7896a160 2024-04-26T21:44:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.891+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v119'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.891+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc7896a160 2024-04-26T21:44:58.891 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:44:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.892+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v119'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.892+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789719f0 con 0x55dc7896a160 2024-04-26T21:44:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.893+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v119'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.893+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789719f0 con 0x55dc7896a160 2024-04-26T21:44:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.894+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v119'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.894+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc7896a160 2024-04-26T21:44:58.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.895+0000 7f6db8b7a640 1 -- 172.21.15.40:0/1697220833 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v119'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da806b020 con 0x55dc7896a160 2024-04-26T21:44:58.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.895+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.896+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v119'120 uv119 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.896+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.897+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v119'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.897+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.897 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:44:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.898+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v119'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.898+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.898+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v119'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.898+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.899+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v119'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.899+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.901+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v119'290 uv282 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.901+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.902+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v119'291 uv291 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.902+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.902+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v119'292 uv292 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.902 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:44:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.903+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.903+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v119'293 uv293 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.904+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.905+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v119'294 uv294 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.905+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.905+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v119'295 uv289 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.906+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.906+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v119'296 uv296 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.906+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.907+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v119'297 uv297 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.907 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:44:58.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.907+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.908+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v119'298 uv298 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.908+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.909+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v119'299 uv299 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.909+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc789709e0 con 0x55dc78962030 2024-04-26T21:44:58.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.910+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v119'300 uv300 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.910+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55dc78970e30 con 0x55dc78962030 2024-04-26T21:44:58.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6db8379640 1 -- 172.21.15.40:0/1697220833 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v119'301 uv301 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6da0020030 con 0x55dc78962030 2024-04-26T21:44:58.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc7895d610 msgr2=0x55dc7895fa00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55dc7895d610 0x55dc7895fa00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55dc788b9f40 msgr2=0x55dc789515c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55dc788b9f40 0x55dc789515c0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dc78966450 msgr2=0x55dc789692b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55dc78966450 0x55dc789692b0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55dc7896db60 msgr2=0x55dc789704a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55dc7896db60 0x55dc789704a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55dc78962030 msgr2=0x55dc78964420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55dc78962030 0x55dc78964420 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55dc7896a160 msgr2=0x55dc7896c550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55dc7896a160 0x55dc7896c550 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 reap_dead start 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 reap_dead start 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6db937b640 1 -- 172.21.15.40:0/1697220833 reap_dead start 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d9806b2d0 msgr2=0x7f6d9806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.911+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d9806b2d0 0x7f6d9806d790 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6da0001e30 tx=0x7f6da0009000 comp rx=0 tx=0).stop 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.912+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7882c8c0 msgr2=0x55dc7894fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:58.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.912+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7882c8c0 0x55dc7894fdf0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f6da80259e0 tx=0x7f6da8003330 comp rx=0 tx=0).stop 2024-04-26T21:44:58.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.912+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 shutdown_connections 2024-04-26T21:44:58.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.912+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d9806b2d0 0x7f6d9806d790 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.912+0000 7f6dbb080f40 1 --2- 172.21.15.40:0/1697220833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dc7882c8c0 0x55dc7894fdf0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:58.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.912+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 >> 172.21.15.40:0/1697220833 conn(0x55dc787e1ab0 msgr2=0x55dc788a00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:58.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.912+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 shutdown_connections 2024-04-26T21:44:58.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.912+0000 7f6dbb080f40 1 -- 172.21.15.40:0/1697220833 wait complete. 2024-04-26T21:44:58.915 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000000 0 2024-04-26T21:44:58.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.999+0000 7fabbfc65f40 1 Processor -- start 2024-04-26T21:44:58.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:58.999+0000 7fabbfc65f40 1 -- start start 2024-04-26T21:44:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbfc65f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a91d9d30 0x55e1a92a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbfc65f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a92096d0 0x55e1a9222210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbfc65f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a920a2e0 0x55e1a9222750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbfc65f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e1a91d8cd0 con 0x55e1a920a2e0 2024-04-26T21:44:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbfc65f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e1a9180e40 con 0x55e1a91d9d30 2024-04-26T21:44:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbfc65f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e1a91d8f70 con 0x55e1a92096d0 2024-04-26T21:44:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a920a2e0 0x55e1a9222750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a920a2e0 0x55e1a9222750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50498/0 (socket says 172.21.15.40:50498) 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 -- 172.21.15.40:0/2052634034 learned_addr learned my addr 172.21.15.40:0/2052634034 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbcf5e640 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a92096d0 0x55e1a9222210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbd75f640 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a91d9d30 0x55e1a92a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 -- 172.21.15.40:0/2052634034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a92096d0 msgr2=0x55e1a9222210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a92096d0 0x55e1a9222210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 -- 172.21.15.40:0/2052634034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a91d9d30 msgr2=0x55e1a92a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a91d9d30 0x55e1a92a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 -- 172.21.15.40:0/2052634034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a92b1940 con 0x55e1a920a2e0 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabbdf60640 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a920a2e0 0x55e1a9222750 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c4a0 tx=0x7fabb000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721b6f6885f26aff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabb77fe640 1 -- 172.21.15.40:0/2052634034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00115f0 con 0x55e1a920a2e0 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabb77fe640 1 -- 172.21.15.40:0/2052634034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb0011790 con 0x55e1a920a2e0 2024-04-26T21:44:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.000+0000 7fabb77fe640 1 -- 172.21.15.40:0/2052634034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0011b60 con 0x55e1a920a2e0 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 -- 172.21.15.40:0/2052634034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a920a2e0 msgr2=0x55e1a9222750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a920a2e0 0x55e1a9222750 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c4a0 tx=0x7fabb000c960 comp rx=0 tx=0).stop 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 -- 172.21.15.40:0/2052634034 shutdown_connections 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a920a2e0 0x55e1a9222750 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a92096d0 0x55e1a9222210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 --2- 172.21.15.40:0/2052634034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a91d9d30 0x55e1a92a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 -- 172.21.15.40:0/2052634034 >> 172.21.15.40:0/2052634034 conn(0x55e1a92049b0 msgr2=0x55e1a9205b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 -- 172.21.15.40:0/2052634034 shutdown_connections 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 -- 172.21.15.40:0/2052634034 wait complete. 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 Processor -- start 2024-04-26T21:44:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.001+0000 7fabbfc65f40 1 -- start start 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbfc65f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a91d9d30 0x55e1a929f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbfc65f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a92096d0 0x55e1a92b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbfc65f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a920a2e0 0x55e1a92b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbfc65f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e1a92a6230 con 0x55e1a92096d0 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbfc65f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e1a92b1be0 con 0x55e1a920a2e0 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbfc65f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e1a92a6510 con 0x55e1a91d9d30 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbcf5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a92096d0 0x55e1a92b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbcf5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a92096d0 0x55e1a92b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50514/0 (socket says 172.21.15.40:50514) 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbcf5e640 1 -- 172.21.15.40:0/1784632315 learned_addr learned my addr 172.21.15.40:0/1784632315 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbd75f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a91d9d30 0x55e1a929f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.002+0000 7fabbdf60640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a920a2e0 0x55e1a92b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabbcf5e640 1 -- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a91d9d30 msgr2=0x55e1a929f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabbcf5e640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e1a91d9d30 0x55e1a929f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabbcf5e640 1 -- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a920a2e0 msgr2=0x55e1a92b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabbcf5e640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e1a920a2e0 0x55e1a92b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabbcf5e640 1 -- 172.21.15.40:0/1784632315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a92ad130 con 0x55e1a92096d0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabbcf5e640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a92096d0 0x55e1a92b2e60 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7faba800eda0 tx=0x7faba800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabb67fc640 1 -- 172.21.15.40:0/1784632315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8015020 con 0x55e1a92096d0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabb67fc640 1 -- 172.21.15.40:0/1784632315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba8011450 con 0x55e1a92096d0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1a92a0050 con 0x55e1a92096d0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabb67fc640 1 -- 172.21.15.40:0/1784632315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8011820 con 0x55e1a92096d0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.003+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a929fc50 con 0x55e1a92096d0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.004+0000 7fabb67fc640 1 -- 172.21.15.40:0/1784632315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faba8018020 con 0x55e1a92096d0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.005+0000 7fabb67fc640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faba006b2d0 0x7faba006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.005+0000 7fabb67fc640 1 -- 172.21.15.40:0/1784632315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8281+0+0 (secure 0 0 0) 0x7faba808b350 con 0x55e1a92096d0 2024-04-26T21:44:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.005+0000 7fabbfc65f40 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a92afa80 0x55e1a9223730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.005+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55e1a9223c70 con 0x55e1a92afa80 2024-04-26T21:44:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.005+0000 7fabbd75f640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faba006b2d0 0x7faba006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.005+0000 7fabbdf60640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a92afa80 0x55e1a9223730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.005+0000 7fabbdf60640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a92afa80 0x55e1a9223730 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.006+0000 7fabbd75f640 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faba006b2d0 0x7faba006d790 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fabb8057000 tx=0x7fabb8062040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbdf60640 1 -- 172.21.15.40:0/1784632315 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000000 [truncate 0] v119'548 uv548 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fabb0011940 con 0x55e1a92afa80 2024-04-26T21:44:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a92afa80 msgr2=0x55e1a9223730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e1a92afa80 0x55e1a9223730 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faba006b2d0 msgr2=0x7faba006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faba006b2d0 0x7faba006d790 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fabb8057000 tx=0x7fabb8062040 comp rx=0 tx=0).stop 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a92096d0 msgr2=0x55e1a92b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 --2- 172.21.15.40:0/1784632315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e1a92096d0 0x55e1a92b2e60 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7faba800eda0 tx=0x7faba800c620 comp rx=0 tx=0).stop 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbdf60640 1 -- 172.21.15.40:0/1784632315 reap_dead start 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 shutdown_connections 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 >> 172.21.15.40:0/1784632315 conn(0x55e1a92049b0 msgr2=0x55e1a92b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 shutdown_connections 2024-04-26T21:44:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.007+0000 7fabbfc65f40 1 -- 172.21.15.40:0/1784632315 wait complete. 2024-04-26T21:44:59.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:44:59.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.153+0000 7f062a524640 1 Processor -- start 2024-04-26T21:44:59.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.153+0000 7f062a524640 1 -- start start 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f062a524640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0624070350 0x7f0624070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f062a524640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624071090 0x7f062415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f062a524640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06240d2390 con 0x7f06240cfde0 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f062a524640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0624071500 con 0x7f0624070350 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f062a524640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06240afab0 con 0x7f0624071090 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f06237fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0624070350 0x7f0624070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f0623fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f06240cfde0 0x7f06240d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52698/0 (socket says 172.21.15.40:52698) 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f06237fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0624070350 0x7f0624070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50530/0 (socket says 172.21.15.40:50530) 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f0623fff640 1 -- 172.21.15.40:0/3500409481 learned_addr learned my addr 172.21.15.40:0/3500409481 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f0628ac1640 1 --2- 172.21.15.40:0/3500409481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624071090 0x7f062415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f06237fe640 1 -- 172.21.15.40:0/3500409481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624071090 msgr2=0x7f062415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:44:59.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f06237fe640 1 --2- 172.21.15.40:0/3500409481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624071090 0x7f062415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f06237fe640 1 -- 172.21.15.40:0/3500409481 >> v1:172.21.15.40:6789/0 conn(0x7f06240cfde0 legacy=0x7f06240d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f0622ffd640 1 -- 172.21.15.40:0/3500409481 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 578748239 0 0) 0x7f06240d2390 con 0x7f06240cfde0 2024-04-26T21:44:59.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.154+0000 7f06237fe640 1 -- 172.21.15.40:0/3500409481 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062406d3f0 con 0x7f0624070350 2024-04-26T21:44:59.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.155+0000 7f06237fe640 1 --2- 172.21.15.40:0/3500409481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0624070350 0x7f0624070b50 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f060c0097a0 tx=0x7f060c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a714b716e5b06a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.155+0000 7f0622ffd640 1 -- 172.21.15.40:0/3500409481 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c037070 con 0x7f0624070350 2024-04-26T21:44:59.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.155+0000 7f0622ffd640 1 -- 172.21.15.40:0/3500409481 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f060c02ae50 con 0x7f0624070350 2024-04-26T21:44:59.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.155+0000 7f0622ffd640 1 -- 172.21.15.40:0/3500409481 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c032660 con 0x7f0624070350 2024-04-26T21:44:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.156+0000 7f062a524640 1 -- 172.21.15.40:0/3500409481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0624070350 msgr2=0x7f0624070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.156+0000 7f062a524640 1 --2- 172.21.15.40:0/3500409481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0624070350 0x7f0624070b50 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f060c0097a0 tx=0x7f060c02a100 comp rx=0 tx=0).stop 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.156+0000 7f062a524640 1 -- 172.21.15.40:0/3500409481 shutdown_connections 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.156+0000 7f062a524640 1 --2- 172.21.15.40:0/3500409481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624071090 0x7f062415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.156+0000 7f062a524640 1 --2- 172.21.15.40:0/3500409481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0624070350 0x7f0624070b50 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.156+0000 7f062a524640 1 -- 172.21.15.40:0/3500409481 >> 172.21.15.40:0/3500409481 conn(0x7f06240d2d80 msgr2=0x7f06240d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.156+0000 7f062a524640 1 -- 172.21.15.40:0/3500409481 shutdown_connections 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.156+0000 7f062a524640 1 -- 172.21.15.40:0/3500409481 wait complete. 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.157+0000 7f062a524640 1 Processor -- start 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.157+0000 7f062a524640 1 -- start start 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f062a524640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624070350 0x7f06240c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f062a524640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0624071090 0x7f062415b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f062a524640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06240cfde0 0x7f0624159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f062a524640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f062406d9b0 con 0x7f06240cfde0 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f062a524640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f062406dd20 con 0x7f0624071090 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f062a524640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f062415ef30 con 0x7f0624070350 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0628ac1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06240cfde0 0x7f0624159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0628ac1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06240cfde0 0x7f0624159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50546/0 (socket says 172.21.15.40:50546) 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0628ac1640 1 -- 172.21.15.40:0/3470052017 learned_addr learned my addr 172.21.15.40:0/3470052017 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f06237fe640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0624071090 0x7f062415b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0623fff640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624070350 0x7f06240c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0628ac1640 1 -- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624070350 msgr2=0x7f06240c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0628ac1640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624070350 0x7f06240c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0628ac1640 1 -- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0624071090 msgr2=0x7f062415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0628ac1640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0624071090 0x7f062415b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.158+0000 7f0628ac1640 1 -- 172.21.15.40:0/3470052017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062415c360 con 0x7f06240cfde0 2024-04-26T21:44:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.159+0000 7f0628ac1640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06240cfde0 0x7f0624159a20 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f061c007e70 tx=0x7f061c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.159+0000 7f0620ff9640 1 -- 172.21.15.40:0/3470052017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c014620 con 0x7f06240cfde0 2024-04-26T21:44:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.159+0000 7f0620ff9640 1 -- 172.21.15.40:0/3470052017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c019070 con 0x7f06240cfde0 2024-04-26T21:44:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.159+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06241f4bd0 con 0x7f06240cfde0 2024-04-26T21:44:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.159+0000 7f0620ff9640 1 -- 172.21.15.40:0/3470052017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c014a20 con 0x7f06240cfde0 2024-04-26T21:44:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.159+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06241f5190 con 0x7f06240cfde0 2024-04-26T21:44:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.160+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06240694f0 con 0x7f06240cfde0 2024-04-26T21:44:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.160+0000 7f0620ff9640 1 -- 172.21.15.40:0/3470052017 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f061c014bc0 con 0x7f06240cfde0 2024-04-26T21:44:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.161+0000 7f0620ff9640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f060406c8f0 0x7f060406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.161+0000 7f0620ff9640 1 -- 172.21.15.40:0/3470052017 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8281+0+0 (secure 0 0 0) 0x7f061c08cd70 con 0x7f06240cfde0 2024-04-26T21:44:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.161+0000 7f0623fff640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f060406c8f0 0x7f060406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.161+0000 7f0623fff640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f060406c8f0 0x7f060406edb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0614007fd0 tx=0x7f0614007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.165+0000 7f0620ff9640 1 -- 172.21.15.40:0/3470052017 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f061c05a0f0 con 0x7f06240cfde0 2024-04-26T21:44:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.393+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f062406ce60 con 0x7f06240cfde0 2024-04-26T21:44:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.692+0000 7f0620ff9640 1 -- 172.21.15.40:0/3470052017 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v241) ==== 162+0+0 (secure 0 0 0) 0x7f061c05e3f0 con 0x7f06240cfde0 2024-04-26T21:44:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:44:59.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f060406c8f0 msgr2=0x7f060406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f060406c8f0 0x7f060406edb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0614007fd0 tx=0x7f0614007de0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06240cfde0 msgr2=0x7f0624159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06240cfde0 0x7f0624159a20 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f061c007e70 tx=0x7f061c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 shutdown_connections 2024-04-26T21:44:59.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f060406c8f0 0x7f060406edb0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06240cfde0 0x7f0624159a20 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0624071090 0x7f062415b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 --2- 172.21.15.40:0/3470052017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0624070350 0x7f06240c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 >> 172.21.15.40:0/3470052017 conn(0x7f06240d2d80 msgr2=0x7f06241625b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:59.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 shutdown_connections 2024-04-26T21:44:59.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.694+0000 7f062a524640 1 -- 172.21.15.40:0/3470052017 wait complete. 2024-04-26T21:44:59.700 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:44:59.700+0000 7f01949fb640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:44:59.701 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:59.701+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:44:59.701 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:44:59.701+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:44:59.706 INFO:tasks.ceph.mds.c.smithi040.stderr: -17> 2024-04-26T21:44:59.700+0000 7f01949fb640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:44:59.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:44:59.731 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:44:59.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.812+0000 7f34a90c4640 1 Processor -- start 2024-04-26T21:44:59.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.812+0000 7f34a90c4640 1 -- start start 2024-04-26T21:44:59.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a90c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a4162810 0x7f34a4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a90c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a40d10d0 con 0x7f34a40b2340 2024-04-26T21:44:59.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a90c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a40d1370 con 0x7f34a40d1b30 2024-04-26T21:44:59.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a90c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34a40d0750 con 0x7f34a4162810 2024-04-26T21:44:59.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f34a40b2340 0x7f34a40cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52710/0 (socket says 172.21.15.40:52710) 2024-04-26T21:44:59.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a259c640 1 -- 172.21.15.40:0/2225891620 learned_addr learned my addr 172.21.15.40:0/2225891620 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a2d9d640 1 --2- 172.21.15.40:0/2225891620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a4162810 0x7f34a4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a1d9b640 1 -- 172.21.15.40:0/2225891620 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785006932 0 0) 0x7f34a40d10d0 con 0x7f34a40b2340 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a1d9b640 1 -- 172.21.15.40:0/2225891620 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3490003520 con 0x7f34a40b2340 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a1d9b640 1 -- 172.21.15.40:0/2225891620 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423879441 0 0) 0x7f34a40d1370 con 0x7f34a40d1b30 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a1d9b640 1 -- 172.21.15.40:0/2225891620 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34a40d10d0 con 0x7f34a40d1b30 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a2d9d640 1 -- 172.21.15.40:0/2225891620 >> v1:172.21.15.184:6789/0 conn(0x7f34a40d1b30 legacy=0x7f34a4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a2d9d640 1 -- 172.21.15.40:0/2225891620 >> v1:172.21.15.40:6789/0 conn(0x7f34a40b2340 legacy=0x7f34a40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a1d9b640 1 -- 172.21.15.40:0/2225891620 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4052202831 0 0) 0x7f3490003520 con 0x7f34a40b2340 2024-04-26T21:44:59.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.813+0000 7f34a2d9d640 1 -- 172.21.15.40:0/2225891620 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a40d01b0 con 0x7f34a4162810 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.814+0000 7f34a2d9d640 1 --2- 172.21.15.40:0/2225891620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a4162810 0x7f34a4070350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f349c0097d0 tx=0x7f349c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18714b770a9ad6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.814+0000 7f34a1d9b640 1 -- 172.21.15.40:0/2225891620 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c037070 con 0x7f34a4162810 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.814+0000 7f34a1d9b640 1 -- 172.21.15.40:0/2225891620 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c02ac30 con 0x7f34a4162810 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.814+0000 7f34a1d9b640 1 -- 172.21.15.40:0/2225891620 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c001670 con 0x7f34a4162810 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.815+0000 7f34a90c4640 1 -- 172.21.15.40:0/2225891620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a4162810 msgr2=0x7f34a4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.815+0000 7f34a90c4640 1 --2- 172.21.15.40:0/2225891620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a4162810 0x7f34a4070350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f349c0097d0 tx=0x7f349c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.815+0000 7f34a90c4640 1 -- 172.21.15.40:0/2225891620 shutdown_connections 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.815+0000 7f34a90c4640 1 --2- 172.21.15.40:0/2225891620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a4162810 0x7f34a4070350 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.815+0000 7f34a90c4640 1 -- 172.21.15.40:0/2225891620 >> 172.21.15.40:0/2225891620 conn(0x7f34a40d2da0 msgr2=0x7f34a40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.815+0000 7f34a90c4640 1 -- 172.21.15.40:0/2225891620 shutdown_connections 2024-04-26T21:44:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.815+0000 7f34a90c4640 1 -- 172.21.15.40:0/2225891620 wait complete. 2024-04-26T21:44:59.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.816+0000 7f34a90c4640 1 Processor -- start 2024-04-26T21:44:59.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.816+0000 7f34a90c4640 1 -- start start 2024-04-26T21:44:59.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a90c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34a40b2340 0x7f34a41f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a90c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a40d1b30 0x7f34a40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a90c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4162810 0x7f34a414c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a90c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4070890 con 0x7f34a4162810 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a90c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34a40d16a0 con 0x7f34a40b2340 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a90c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34a40cdb00 con 0x7f34a40d1b30 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4162810 0x7f34a414c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4162810 0x7f34a414c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50570/0 (socket says 172.21.15.40:50570) 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a359e640 1 -- 172.21.15.40:0/2459451278 learned_addr learned my addr 172.21.15.40:0/2459451278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a259c640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a40d1b30 0x7f34a40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.817+0000 7f34a2d9d640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34a40b2340 0x7f34a41f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a359e640 1 -- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a40d1b30 msgr2=0x7f34a40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a359e640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a40d1b30 0x7f34a40ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a359e640 1 -- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34a40b2340 msgr2=0x7f34a41f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a359e640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34a40b2340 0x7f34a41f8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a359e640 1 -- 172.21.15.40:0/2459451278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a40cd1d0 con 0x7f34a4162810 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a259c640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a40d1b30 0x7f34a40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a359e640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4162810 0x7f34a414c7d0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f349400c7c0 tx=0x7f349400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f349b7fe640 1 -- 172.21.15.40:0/2459451278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494003b70 con 0x7f34a4162810 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f349b7fe640 1 -- 172.21.15.40:0/2459451278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3494003d10 con 0x7f34a4162810 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a41611d0 con 0x7f34a4162810 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f349b7fe640 1 -- 172.21.15.40:0/2459451278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349400a9f0 con 0x7f34a4162810 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.818+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a414bfd0 con 0x7f34a4162810 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.819+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a41616f0 con 0x7f34a4162810 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.819+0000 7f349b7fe640 1 -- 172.21.15.40:0/2459451278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f349400ab90 con 0x7f34a4162810 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.820+0000 7f349b7fe640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f348406c8f0 0x7f348406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.820+0000 7f349b7fe640 1 -- 172.21.15.40:0/2459451278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8367+0+0 (secure 0 0 0) 0x7f349408c5d0 con 0x7f34a4162810 2024-04-26T21:44:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.820+0000 7f34a2d9d640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f348406c8f0 0x7f348406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.821+0000 7f34a2d9d640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f348406c8f0 0x7f348406edb0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f349c034000 tx=0x7f349c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:44:59.824+0000 7f349b7fe640 1 -- 172.21.15.40:0/2459451278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3494059ef0 con 0x7f34a4162810 2024-04-26T21:45:00.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.046+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34a40694f0 con 0x7f34a4162810 2024-04-26T21:45:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.047+0000 7f349b7fe640 1 -- 172.21.15.40:0/2459451278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 243 v243) ==== 95+0+4355 (secure 0 0 0) 0x7f34940597f0 con 0x7f34a4162810 2024-04-26T21:45:00.047 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:00.047 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":243,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":243,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:59.701685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 243 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f348406c8f0 msgr2=0x7f348406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f348406c8f0 0x7f348406edb0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f349c034000 tx=0x7f349c02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4162810 msgr2=0x7f34a414c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4162810 0x7f34a414c7d0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f349400c7c0 tx=0x7f349400cc80 comp rx=0 tx=0).stop 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 shutdown_connections 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f348406c8f0 0x7f348406edb0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4162810 0x7f34a414c7d0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34a40d1b30 0x7f34a40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 --2- 172.21.15.40:0/2459451278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34a40b2340 0x7f34a41f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 >> 172.21.15.40:0/2459451278 conn(0x7f34a40d2da0 msgr2=0x7f34a40cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:00.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 shutdown_connections 2024-04-26T21:45:00.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.049+0000 7f34a90c4640 1 -- 172.21.15.40:0/2459451278 wait complete. 2024-04-26T21:45:00.060 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:00.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:00.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.200+0000 7fb034919640 1 Processor -- start 2024-04-26T21:45:00.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.200+0000 7fb034919640 1 -- start start 2024-04-26T21:45:00.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.201+0000 7fb034919640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03015f3a0 0x7fb0300d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.201+0000 7fb034919640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb0300d1770 0x7fb0300d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.201+0000 7fb034919640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0300d25a0 con 0x7fb0300d2130 2024-04-26T21:45:00.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.201+0000 7fb034919640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb0300d0e20 con 0x7fb0300d1770 2024-04-26T21:45:00.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.201+0000 7fb034919640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0300b1d70 con 0x7fb03015f3a0 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.201+0000 7fb02ed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb0300d2130 0x7fb0300d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52726/0 (socket says 172.21.15.40:52726) 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.201+0000 7fb02ed9d640 1 -- 172.21.15.40:0/4015743777 learned_addr learned my addr 172.21.15.40:0/4015743777 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02e59c640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03015f3a0 0x7fb0300d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426877612 0 0) 0x7fb028001110 con 0x7fb0300d2130 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/4015743777 >> v2:172.21.15.184:3300/0 conn(0x7fb0300d1770 0x7fb0300d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb01c003520 con 0x7fb0300d2130 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1296385697 0 0) 0x7fb01c003520 con 0x7fb0300d2130 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03015f3a0 msgr2=0x7fb0300d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03015f3a0 0x7fb0300d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 >> v2:172.21.15.184:3300/0 conn(0x7fb0300d1770 msgr2=0x7fb0300d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 --2- 172.21.15.40:0/4015743777 >> v2:172.21.15.184:3300/0 conn(0x7fb0300d1770 0x7fb0300d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0300b2050 con 0x7fb0300d2130 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb028001840 con 0x7fb0300d2130 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 >> v1:172.21.15.40:6789/0 conn(0x7fb0300d2130 legacy=0x7fb0300d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03015f3a0 0x7fb01c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c003f30 0x7fb01c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb01c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300d0e20 con 0x7fb01c004d40 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300b1d70 con 0x7fb01c003f30 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb01c00b9c0 con 0x7fb03015f3a0 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb01c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c003f30 0x7fb01c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.202+0000 7fb02e59c640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03015f3a0 0x7fb01c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02ed9d640 1 -- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03015f3a0 msgr2=0x7fb01c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03015f3a0 0x7fb01c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02ed9d640 1 -- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c003f30 msgr2=0x7fb01c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c003f30 0x7fb01c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02ed9d640 1 -- 172.21.15.40:0/4015743777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c003520 con 0x7fb01c004d40 2024-04-26T21:45:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb01c0051c0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fb02800c490 tx=0x7fb01c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1bc4157f5f0468 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:00.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028032020 con 0x7fb01c004d40 2024-04-26T21:45:00.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb028005ad0 con 0x7fb01c004d40 2024-04-26T21:45:00.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.203+0000 7fb02d59a640 1 -- 172.21.15.40:0/4015743777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02800c0e0 con 0x7fb01c004d40 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.204+0000 7fb034919640 1 -- 172.21.15.40:0/4015743777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 msgr2=0x7fb01c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.204+0000 7fb034919640 1 --2- 172.21.15.40:0/4015743777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb01c0051c0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fb02800c490 tx=0x7fb01c005830 comp rx=0 tx=0).stop 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.204+0000 7fb02ed9d640 1 -- 172.21.15.40:0/4015743777 reap_dead start 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.204+0000 7fb034919640 1 -- 172.21.15.40:0/4015743777 shutdown_connections 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.204+0000 7fb034919640 1 -- 172.21.15.40:0/4015743777 >> 172.21.15.40:0/4015743777 conn(0x7fb0300d4f90 msgr2=0x7fb0300d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.204+0000 7fb034919640 1 -- 172.21.15.40:0/4015743777 shutdown_connections 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.204+0000 7fb034919640 1 -- 172.21.15.40:0/4015743777 wait complete. 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.205+0000 7fb034919640 1 Processor -- start 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.205+0000 7fb034919640 1 -- start start 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb034919640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb03015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb034919640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300b8270 0x7fb0300b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb034919640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb030161790 0x7fb030161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb034919640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb03015f900 con 0x7fb01c004d40 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb034919640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb03015f780 con 0x7fb0300b8270 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb034919640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0300b3860 con 0x7fb030161790 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb02e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb03015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb02e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb03015b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50592/0 (socket says 172.21.15.40:50592) 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb02e59c640 1 -- 172.21.15.40:0/2541648705 learned_addr learned my addr 172.21.15.40:0/2541648705 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300b8270 0x7fb0300b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.206+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb030161790 0x7fb030161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb02e59c640 1 -- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb030161790 msgr2=0x7fb030161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb02e59c640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb030161790 0x7fb030161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb02e59c640 1 -- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300b8270 msgr2=0x7fb0300b86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb02e59c640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300b8270 0x7fb0300b86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb02e59c640 1 -- 172.21.15.40:0/2541648705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0301609d0 con 0x7fb01c004d40 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb030161790 0x7fb030161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb02e59c640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb03015b360 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fb020011e30 tx=0x7fb02000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb026ffd640 1 -- 172.21.15.40:0/2541648705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02000fea0 con 0x7fb01c004d40 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb026ffd640 1 -- 172.21.15.40:0/2541648705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb020015630 con 0x7fb01c004d40 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb026ffd640 1 -- 172.21.15.40:0/2541648705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020015a00 con 0x7fb01c004d40 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb030160cb0 con 0x7fb01c004d40 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.207+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb030161270 con 0x7fb01c004d40 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.208+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0300694f0 con 0x7fb01c004d40 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.209+0000 7fb026ffd640 1 -- 172.21.15.40:0/2541648705 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb020015ba0 con 0x7fb01c004d40 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.209+0000 7fb026ffd640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01006c920 0x7fb01006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.209+0000 7fb026ffd640 1 -- 172.21.15.40:0/2541648705 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8367+0+0 (secure 0 0 0) 0x7fb02008d690 con 0x7fb01c004d40 2024-04-26T21:45:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.209+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01006c920 0x7fb01006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.210+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01006c920 0x7fb01006ede0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb03015e520 tx=0x7fb018009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:00.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.213+0000 7fb026ffd640 1 -- 172.21.15.40:0/2541648705 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb020019030 con 0x7fb01c004d40 2024-04-26T21:45:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.445+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0300b6860 con 0x7fb01c004d40 2024-04-26T21:45:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.445+0000 7fb026ffd640 1 -- 172.21.15.40:0/2541648705 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 243 v243) ==== 95+0+4355 (secure 0 0 0) 0x7fb020010040 con 0x7fb01c004d40 2024-04-26T21:45:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:00.445 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":243,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":243,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:59.701685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 243 2024-04-26T21:45:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01006c920 msgr2=0x7fb01006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01006c920 0x7fb01006ede0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb03015e520 tx=0x7fb018009190 comp rx=0 tx=0).stop 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 msgr2=0x7fb03015b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb03015b360 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fb020011e30 tx=0x7fb02000f580 comp rx=0 tx=0).stop 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 shutdown_connections 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb01006c920 0x7fb01006ede0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb030161790 0x7fb030161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300b8270 0x7fb0300b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 --2- 172.21.15.40:0/2541648705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c004d40 0x7fb03015b360 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.447+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 >> 172.21.15.40:0/2541648705 conn(0x7fb0300d4f90 msgr2=0x7fb03006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.448+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 shutdown_connections 2024-04-26T21:45:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.448+0000 7fb034919640 1 -- 172.21.15.40:0/2541648705 wait complete. 2024-04-26T21:45:00.458 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:00.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:00.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.559+0000 7f6151446640 1 Processor -- start 2024-04-26T21:45:00.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.559+0000 7f6151446640 1 -- start start 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f6151446640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f614c06e270 0x7f614c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f6151446640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c06d560 0x7f614c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f6151446640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614c0d2420 con 0x7f614c0d0070 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f6151446640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f614c162d00 con 0x7f614c06e270 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f6151446640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f614c0d2d10 con 0x7f614c06d560 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f614affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f614c0d0070 0x7f614c06db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52740/0 (socket says 172.21.15.40:52740) 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f614affd640 1 -- 172.21.15.40:0/2464724154 learned_addr learned my addr 172.21.15.40:0/2464724154 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f614b7fe640 1 --2- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c06d560 0x7f614c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.560+0000 7f614a7fc640 1 --2- 172.21.15.40:0/2464724154 >> v2:172.21.15.184:3300/0 conn(0x7f614c06e270 0x7f614c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.561+0000 7f6149ffb640 1 -- 172.21.15.40:0/2464724154 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 727028532 0 0) 0x7f614c0d2420 con 0x7f614c0d0070 2024-04-26T21:45:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.561+0000 7f6149ffb640 1 -- 172.21.15.40:0/2464724154 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6138003520 con 0x7f614c0d0070 2024-04-26T21:45:00.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.561+0000 7f6149ffb640 1 -- 172.21.15.40:0/2464724154 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2765318822 0 0) 0x7f6138003520 con 0x7f614c0d0070 2024-04-26T21:45:00.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.561+0000 7f614a7fc640 1 -- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c06d560 msgr2=0x7f614c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:00.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.561+0000 7f614a7fc640 1 --2- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c06d560 0x7f614c0ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.561+0000 7f614a7fc640 1 -- 172.21.15.40:0/2464724154 >> v1:172.21.15.40:6789/0 conn(0x7f614c0d0070 legacy=0x7f614c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.561+0000 7f614a7fc640 1 -- 172.21.15.40:0/2464724154 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0cd6b0 con 0x7f614c06e270 2024-04-26T21:45:00.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.561+0000 7f614b7fe640 1 --2- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c06d560 0x7f614c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f614a7fc640 1 --2- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 0x7f614c0d1d40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f61340097a0 tx=0x7f613402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ec1fd27e8cb432 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6149ffb640 1 -- 172.21.15.40:0/2464724154 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134037070 con 0x7f614c06e270 2024-04-26T21:45:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6149ffb640 1 -- 172.21.15.40:0/2464724154 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613402ac30 con 0x7f614c06e270 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6149ffb640 1 -- 172.21.15.40:0/2464724154 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144001670 con 0x7f614c06e270 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6151446640 1 -- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 msgr2=0x7f614c0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6151446640 1 --2- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 0x7f614c0d1d40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f61340097a0 tx=0x7f613402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6151446640 1 -- 172.21.15.40:0/2464724154 shutdown_connections 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6151446640 1 --2- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c06d560 0x7f614c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6151446640 1 --2- 172.21.15.40:0/2464724154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 0x7f614c0d1d40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6151446640 1 -- 172.21.15.40:0/2464724154 >> 172.21.15.40:0/2464724154 conn(0x7f614c0d2ec0 msgr2=0x7f614c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6151446640 1 -- 172.21.15.40:0/2464724154 shutdown_connections 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.562+0000 7f6151446640 1 -- 172.21.15.40:0/2464724154 wait complete. 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.564+0000 7f6151446640 1 Processor -- start 2024-04-26T21:45:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.564+0000 7f6151446640 1 -- start start 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f6151446640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614c06d560 0x7f614c1f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f6151446640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 0x7f614c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f6151446640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c0d0070 0x7f614c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f6151446640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f614c0ce6d0 con 0x7f614c06d560 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f6151446640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f614c0cdc70 con 0x7f614c06e270 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f6151446640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f614c0ce850 con 0x7f614c0d0070 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614c06d560 0x7f614c1f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614c06d560 0x7f614c1f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50600/0 (socket says 172.21.15.40:50600) 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 -- 172.21.15.40:0/2134255261 learned_addr learned my addr 172.21.15.40:0/2134255261 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614a7fc640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 0x7f614c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 -- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c0d0070 msgr2=0x7f614c1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c0d0070 0x7f614c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 -- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 msgr2=0x7f614c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 0x7f614c15b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 -- 172.21.15.40:0/2134255261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c1f5fe0 con 0x7f614c06d560 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f614affd640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614c06d560 0x7f614c1f8fa0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f614400c680 tx=0x7f614400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f61437fe640 1 -- 172.21.15.40:0/2134255261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144017070 con 0x7f614c06d560 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f61437fe640 1 -- 172.21.15.40:0/2134255261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6144005ad0 con 0x7f614c06d560 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c1f94e0 con 0x7f614c06d560 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f61437fe640 1 -- 172.21.15.40:0/2134255261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144007070 con 0x7f614c06d560 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.565+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c1f5d00 con 0x7f614c06d560 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.566+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c0694f0 con 0x7f614c06d560 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.567+0000 7f61437fe640 1 -- 172.21.15.40:0/2134255261 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f614401a070 con 0x7f614c06d560 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.567+0000 7f61437fe640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.567+0000 7f61437fe640 1 -- 172.21.15.40:0/2134255261 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8367+0+0 (secure 0 0 0) 0x7f614408b8f0 con 0x7f614c06d560 2024-04-26T21:45:00.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.567+0000 7f614a7fc640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:00.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.568+0000 7f614a7fc640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f612c06c8f0 0x7f612c06edb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6134034000 tx=0x7f613402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.571+0000 7f61437fe640 1 -- 172.21.15.40:0/2134255261 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6144058cb0 con 0x7f614c06d560 2024-04-26T21:45:00.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.796+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f614c0d0470 con 0x7f614c06d560 2024-04-26T21:45:00.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.797+0000 7f61437fe640 1 -- 172.21.15.40:0/2134255261 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 244 v244) ==== 95+0+5182 (secure 0 0 0) 0x7f614405cfb0 con 0x7f614c06d560 2024-04-26T21:45:00.797 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:00.797 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":244,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1838768570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1838768570},{"type":"v1","addr":"172.21.15.40:6833","nonce":1838768570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":244}],"filesystems":[{"mdsmap":{"epoch":243,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:44:59.701685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 244 2024-04-26T21:45:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f612c06c8f0 msgr2=0x7f612c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f612c06c8f0 0x7f612c06edb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6134034000 tx=0x7f613402e000 comp rx=0 tx=0).stop 2024-04-26T21:45:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614c06d560 msgr2=0x7f614c1f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614c06d560 0x7f614c1f8fa0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f614400c680 tx=0x7f614400cb40 comp rx=0 tx=0).stop 2024-04-26T21:45:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 shutdown_connections 2024-04-26T21:45:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614c0d0070 0x7f614c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614c06e270 0x7f614c15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 --2- 172.21.15.40:0/2134255261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614c06d560 0x7f614c1f8fa0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 >> 172.21.15.40:0/2134255261 conn(0x7f614c0d2ec0 msgr2=0x7f614c06db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 shutdown_connections 2024-04-26T21:45:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:00.799+0000 7f6151446640 1 -- 172.21.15.40:0/2134255261 wait complete. 2024-04-26T21:45:00.810 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000000' led to DAMAGED state 2024-04-26T21:45:00.810 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000001' 2024-04-26T21:45:00.810 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:00.810 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:00.825 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:45:00.826 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:00.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:00.826 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:00.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:45:00.880 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:45:00.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:01.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.022+0000 7ff704fe4640 1 Processor -- start 2024-04-26T21:45:01.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.022+0000 7ff704fe4640 1 -- start start 2024-04-26T21:45:01.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff704fe4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7000d3bc0 0x7ff7000d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff704fe4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7000b5f20 0x7ff7000b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff704fe4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7000b7e50 con 0x7ff700162880 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff704fe4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff7000d0580 con 0x7ff7000d3bc0 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff704fe4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff7000d1800 con 0x7ff7000b5f20 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff6fdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7000d3bc0 0x7ff7000d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff6fe59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff700162880 0x7ff7000d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52752/0 (socket says 172.21.15.40:52752) 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff6fdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7000d3bc0 0x7ff7000d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50608/0 (socket says 172.21.15.40:50608) 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff6fe59c640 1 -- 172.21.15.40:0/2748847717 learned_addr learned my addr 172.21.15.40:0/2748847717 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.023+0000 7ff6fed9d640 1 --2- 172.21.15.40:0/2748847717 >> v2:172.21.15.184:3300/0 conn(0x7ff7000b5f20 0x7ff7000b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fdd9b640 1 -- 172.21.15.40:0/2748847717 >> v2:172.21.15.184:3300/0 conn(0x7ff7000b5f20 msgr2=0x7ff7000b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fdd9b640 1 --2- 172.21.15.40:0/2748847717 >> v2:172.21.15.184:3300/0 conn(0x7ff7000b5f20 0x7ff7000b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fdd9b640 1 -- 172.21.15.40:0/2748847717 >> v1:172.21.15.40:6789/0 conn(0x7ff700162880 legacy=0x7ff7000d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fdd9b640 1 -- 172.21.15.40:0/2748847717 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7000d07d0 con 0x7ff7000d3bc0 2024-04-26T21:45:01.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fd59a640 1 -- 172.21.15.40:0/2748847717 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508440572 0 0) 0x7ff7000b7e50 con 0x7ff700162880 2024-04-26T21:45:01.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fdd9b640 1 --2- 172.21.15.40:0/2748847717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000d3bc0 0x7ff7000d2cb0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7ff70008f770 tx=0x7ff6e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d21b2370a250e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fd59a640 1 -- 172.21.15.40:0/2748847717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8037070 con 0x7ff7000d3bc0 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fd59a640 1 -- 172.21.15.40:0/2748847717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e8032440 con 0x7ff7000d3bc0 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.024+0000 7ff6fd59a640 1 -- 172.21.15.40:0/2748847717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8032810 con 0x7ff7000d3bc0 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.025+0000 7ff704fe4640 1 -- 172.21.15.40:0/2748847717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000d3bc0 msgr2=0x7ff7000d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.025+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2748847717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000d3bc0 0x7ff7000d2cb0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7ff70008f770 tx=0x7ff6e802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.025+0000 7ff704fe4640 1 -- 172.21.15.40:0/2748847717 shutdown_connections 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.025+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2748847717 >> v2:172.21.15.184:3300/0 conn(0x7ff7000b5f20 0x7ff7000b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.025+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2748847717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000d3bc0 0x7ff7000d2cb0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.025+0000 7ff704fe4640 1 -- 172.21.15.40:0/2748847717 >> 172.21.15.40:0/2748847717 conn(0x7ff7000d4f60 msgr2=0x7ff7000d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.025+0000 7ff704fe4640 1 -- 172.21.15.40:0/2748847717 shutdown_connections 2024-04-26T21:45:01.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.025+0000 7ff704fe4640 1 -- 172.21.15.40:0/2748847717 wait complete. 2024-04-26T21:45:01.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.026+0000 7ff704fe4640 1 Processor -- start 2024-04-26T21:45:01.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.026+0000 7ff704fe4640 1 -- start start 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff704fe4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000b5f20 0x7ff7001f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff704fe4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7000d3bc0 0x7ff7000ba480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff704fe4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff700162880 0x7ff7000a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff704fe4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff700162c60 con 0x7ff7000b5f20 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff704fe4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7000d0d90 con 0x7ff700162880 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff704fe4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff700162de0 con 0x7ff7000d3bc0 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff6fe59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000b5f20 0x7ff7001f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff6fe59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000b5f20 0x7ff7001f80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50624/0 (socket says 172.21.15.40:50624) 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff6fe59c640 1 -- 172.21.15.40:0/2915543787 learned_addr learned my addr 172.21.15.40:0/2915543787 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff6fed9d640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff700162880 0x7ff7000a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.027+0000 7ff6fdd9b640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7000d3bc0 0x7ff7000ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6fe59c640 1 -- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7000d3bc0 msgr2=0x7ff7000ba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6fe59c640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7000d3bc0 0x7ff7000ba480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6fe59c640 1 -- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff700162880 msgr2=0x7ff7000a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6fe59c640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff700162880 0x7ff7000a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6fe59c640 1 -- 172.21.15.40:0/2915543787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff700154a50 con 0x7ff7000b5f20 2024-04-26T21:45:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6fe59c640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000b5f20 0x7ff7001f80e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c4d0 tx=0x7ff6f800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6f6ffd640 1 -- 172.21.15.40:0/2915543787 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8018070 con 0x7ff7000b5f20 2024-04-26T21:45:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6f6ffd640 1 -- 172.21.15.40:0/2915543787 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f8007500 con 0x7ff7000b5f20 2024-04-26T21:45:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7001f87e0 con 0x7ff7000b5f20 2024-04-26T21:45:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff6f6ffd640 1 -- 172.21.15.40:0/2915543787 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8004a20 con 0x7ff7000b5f20 2024-04-26T21:45:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.028+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7001f8da0 con 0x7ff7000b5f20 2024-04-26T21:45:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.029+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7001f8fe0 con 0x7ff7000b5f20 2024-04-26T21:45:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.029+0000 7ff6f6ffd640 1 -- 172.21.15.40:0/2915543787 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6f801b070 con 0x7ff7000b5f20 2024-04-26T21:45:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.030+0000 7ff6f6ffd640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.030+0000 7ff6f6ffd640 1 -- 172.21.15.40:0/2915543787 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8367+0+0 (secure 0 0 0) 0x7ff6f808b900 con 0x7ff7000b5f20 2024-04-26T21:45:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.030+0000 7ff6fdd9b640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.031+0000 7ff6fdd9b640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6e006c8f0 0x7ff6e006edb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8034000 tx=0x7ff6e802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.034+0000 7ff6f6ffd640 1 -- 172.21.15.40:0/2915543787 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7001f8fe0 con 0x7ff7000b5f20 2024-04-26T21:45:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.258+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff7000694f0 con 0x7ff7000b5f20 2024-04-26T21:45:01.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.710+0000 7ff6f6ffd640 1 -- 172.21.15.40:0/2915543787 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v245) ==== 161+0+0 (secure 0 0 0) 0x7ff700154a50 con 0x7ff7000b5f20 2024-04-26T21:45:01.711 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6e006c8f0 msgr2=0x7ff6e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6e006c8f0 0x7ff6e006edb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8034000 tx=0x7ff6e802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000b5f20 msgr2=0x7ff7001f80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000b5f20 0x7ff7001f80e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c4d0 tx=0x7ff6f800c990 comp rx=0 tx=0).stop 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 shutdown_connections 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff700162880 0x7ff7000a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7000d3bc0 0x7ff7000ba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 --2- 172.21.15.40:0/2915543787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7000b5f20 0x7ff7001f80e0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 >> 172.21.15.40:0/2915543787 conn(0x7ff7000d4f60 msgr2=0x7ff7000d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:01.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 shutdown_connections 2024-04-26T21:45:01.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.713+0000 7ff704fe4640 1 -- 172.21.15.40:0/2915543787 wait complete. 2024-04-26T21:45:01.725 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:01.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.828+0000 7f4e7917f640 1 Processor -- start 2024-04-26T21:45:01.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.828+0000 7f4e7917f640 1 -- start start 2024-04-26T21:45:01.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7917f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e74071e60 0x7f4e7406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7917f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4e740cc480 0x7f4e74166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7917f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7415d7c0 con 0x7f4e7406fe20 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7917f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7415daa0 con 0x7f4e740cc480 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7917f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7415dd80 con 0x7f4e74071e60 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e6bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4e7406fe20 0x7f4e74072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52766/0 (socket says 172.21.15.40:52766) 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e6bfff640 1 -- 172.21.15.40:0/1276823378 learned_addr learned my addr 172.21.15.40:0/1276823378 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e72d9d640 1 --2- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e74071e60 0x7f4e7406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7359e640 1 --2- 172.21.15.40:0/1276823378 >> v2:172.21.15.184:3300/0 conn(0x7f4e740cc480 0x7f4e74166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7259c640 1 -- 172.21.15.40:0/1276823378 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604422099 0 0) 0x7f4e7415d7c0 con 0x7f4e7406fe20 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7259c640 1 -- 172.21.15.40:0/1276823378 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e60003520 con 0x7f4e7406fe20 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7359e640 1 -- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e74071e60 msgr2=0x7f4e7406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7259c640 1 -- 172.21.15.40:0/1276823378 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1715123534 0 0) 0x7f4e60003520 con 0x7f4e7406fe20 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7359e640 1 --2- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e74071e60 0x7f4e7406f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7359e640 1 -- 172.21.15.40:0/1276823378 >> v1:172.21.15.40:6789/0 conn(0x7f4e7406fe20 legacy=0x7f4e74072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.829+0000 7f4e7359e640 1 -- 172.21.15.40:0/1276823378 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7415e060 con 0x7f4e740cc480 2024-04-26T21:45:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.830+0000 7f4e72d9d640 1 --2- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e74071e60 0x7f4e7406f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.830+0000 7f4e7359e640 1 --2- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 0x7f4e74166ad0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009d90 tx=0x7f4e6c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20391b75898fb4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.830+0000 7f4e7259c640 1 -- 172.21.15.40:0/1276823378 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c001670 con 0x7f4e740cc480 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.830+0000 7f4e7259c640 1 -- 172.21.15.40:0/1276823378 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6c007540 con 0x7f4e740cc480 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.830+0000 7f4e7259c640 1 -- 172.21.15.40:0/1276823378 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0078f0 con 0x7f4e740cc480 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.831+0000 7f4e7917f640 1 -- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 msgr2=0x7f4e74166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.831+0000 7f4e7917f640 1 --2- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 0x7f4e74166ad0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009d90 tx=0x7f4e6c02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.831+0000 7f4e7917f640 1 -- 172.21.15.40:0/1276823378 shutdown_connections 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.831+0000 7f4e7917f640 1 --2- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 0x7f4e74166ad0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.831+0000 7f4e7917f640 1 --2- 172.21.15.40:0/1276823378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e74071e60 0x7f4e7406f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.831+0000 7f4e7917f640 1 -- 172.21.15.40:0/1276823378 >> 172.21.15.40:0/1276823378 conn(0x7f4e740d0be0 msgr2=0x7f4e740d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.831+0000 7f4e7917f640 1 -- 172.21.15.40:0/1276823378 shutdown_connections 2024-04-26T21:45:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.831+0000 7f4e7917f640 1 -- 172.21.15.40:0/1276823378 wait complete. 2024-04-26T21:45:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.832+0000 7f4e7917f640 1 Processor -- start 2024-04-26T21:45:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.832+0000 7f4e7917f640 1 -- start start 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e7917f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e7406fe20 0x7f4e740b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e7917f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74071e60 0x7f4e741f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e7917f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 0x7f4e741f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e7917f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7415d570 con 0x7f4e74071e60 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e7917f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7415d3f0 con 0x7f4e740cc480 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e7917f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4e74072950 con 0x7f4e7406fe20 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e6bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74071e60 0x7f4e741f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e6bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74071e60 0x7f4e741f8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50640/0 (socket says 172.21.15.40:50640) 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e6bfff640 1 -- 172.21.15.40:0/66493024 learned_addr learned my addr 172.21.15.40:0/66493024 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e7359e640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 0x7f4e741f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.833+0000 7f4e72d9d640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e7406fe20 0x7f4e740b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6bfff640 1 -- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e7406fe20 msgr2=0x7f4e740b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6bfff640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e7406fe20 0x7f4e740b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6bfff640 1 -- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 msgr2=0x7f4e741f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6bfff640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 0x7f4e741f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6bfff640 1 -- 172.21.15.40:0/66493024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e740c8550 con 0x7f4e74071e60 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e72d9d640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e7406fe20 0x7f4e740b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6bfff640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74071e60 0x7f4e741f8200 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c780 tx=0x7f4e5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6b7fe640 1 -- 172.21.15.40:0/66493024 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c017070 con 0x7f4e74071e60 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6b7fe640 1 -- 172.21.15.40:0/66493024 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c005ad0 con 0x7f4e74071e60 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e740af770 con 0x7f4e74071e60 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e6b7fe640 1 -- 172.21.15.40:0/66493024 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c007070 con 0x7f4e74071e60 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.834+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e740b0010 con 0x7f4e74071e60 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.835+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e740694f0 con 0x7f4e74071e60 2024-04-26T21:45:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.835+0000 7f4e6b7fe640 1 -- 172.21.15.40:0/66493024 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e5c01a070 con 0x7f4e74071e60 2024-04-26T21:45:01.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.836+0000 7f4e6b7fe640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e5406c8f0 0x7f4e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:01.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.836+0000 7f4e6b7fe640 1 -- 172.21.15.40:0/66493024 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8367+0+0 (secure 0 0 0) 0x7f4e5c08bbc0 con 0x7f4e74071e60 2024-04-26T21:45:01.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.836+0000 7f4e72d9d640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e5406c8f0 0x7f4e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:01.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.837+0000 7f4e72d9d640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e5406c8f0 0x7f4e5406edb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009ba0 tx=0x7f4e64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:01.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:01.840+0000 7f4e6b7fe640 1 -- 172.21.15.40:0/66493024 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e740c8550 con 0x7f4e74071e60 2024-04-26T21:45:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.099+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f4e74073b20 con 0x7f4e74071e60 2024-04-26T21:45:02.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.719+0000 7f4e6b7fe640 1 -- 172.21.15.40:0/66493024 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v246) ==== 101+0+0 (secure 0 0 0) 0x7f4e5c0594e0 con 0x7f4e74071e60 2024-04-26T21:45:02.719 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.721+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e5406c8f0 msgr2=0x7f4e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e5406c8f0 0x7f4e5406edb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009ba0 tx=0x7f4e64008040 comp rx=0 tx=0).stop 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74071e60 msgr2=0x7f4e741f8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74071e60 0x7f4e741f8200 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c780 tx=0x7f4e5c00cc40 comp rx=0 tx=0).stop 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 shutdown_connections 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4e5406c8f0 0x7f4e5406edb0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e740cc480 0x7f4e741f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74071e60 0x7f4e741f8200 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 --2- 172.21.15.40:0/66493024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e7406fe20 0x7f4e740b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 >> 172.21.15.40:0/66493024 conn(0x7f4e740d0be0 msgr2=0x7f4e74072cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 shutdown_connections 2024-04-26T21:45:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.722+0000 7f4e7917f640 1 -- 172.21.15.40:0/66493024 wait complete. 2024-04-26T21:45:02.736 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.780+0000 7fde525fef40 1 Processor -- start 2024-04-26T21:45:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.780+0000 7fde525fef40 1 -- start start 2024-04-26T21:45:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde525fef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae41fd7670 0x55ae42067ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde525fef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae4204dd60 0x55ae42065d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde525fef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae41fbaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde525fef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ae41f37e40 con 0x55ae4206b610 2024-04-26T21:45:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde525fef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ae41f3be80 con 0x55ae4204dd60 2024-04-26T21:45:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde525fef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ae4204e140 con 0x55ae41fd7670 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae41fbaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae41fbaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50668/0 (socket says 172.21.15.40:50668) 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 -- 172.21.15.40:0/1171289996 learned_addr learned my addr 172.21.15.40:0/1171289996 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde500fa640 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae41fd7670 0x55ae42067ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde4f8f9640 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae4204dd60 0x55ae42065d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 -- 172.21.15.40:0/1171289996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae41fd7670 msgr2=0x55ae42067ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae41fd7670 0x55ae42067ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 -- 172.21.15.40:0/1171289996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae4204dd60 msgr2=0x55ae42065d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae4204dd60 0x55ae42065d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 -- 172.21.15.40:0/1171289996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae42066de0 con 0x55ae4206b610 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.781+0000 7fde508fb640 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae41fbaf90 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fde4800a3b0 tx=0x7fde4800a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cdbcbb530f9d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde4e8f7640 1 -- 172.21.15.40:0/1171289996 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4800b8c0 con 0x55ae4206b610 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde4e8f7640 1 -- 172.21.15.40:0/1171289996 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4800f030 con 0x55ae4206b610 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde4e8f7640 1 -- 172.21.15.40:0/1171289996 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4800bd00 con 0x55ae4206b610 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 -- 172.21.15.40:0/1171289996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 msgr2=0x55ae41fbaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae41fbaf90 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fde4800a3b0 tx=0x7fde4800a870 comp rx=0 tx=0).stop 2024-04-26T21:45:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 -- 172.21.15.40:0/1171289996 shutdown_connections 2024-04-26T21:45:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae41fbaf90 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae4204dd60 0x55ae42065d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 --2- 172.21.15.40:0/1171289996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae41fd7670 0x55ae42067ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 -- 172.21.15.40:0/1171289996 >> 172.21.15.40:0/1171289996 conn(0x55ae41f8eab0 msgr2=0x55ae420673a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 -- 172.21.15.40:0/1171289996 shutdown_connections 2024-04-26T21:45:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.782+0000 7fde525fef40 1 -- 172.21.15.40:0/1171289996 wait complete. 2024-04-26T21:45:02.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.783+0000 7fde525fef40 1 Processor -- start 2024-04-26T21:45:02.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.783+0000 7fde525fef40 1 -- start start 2024-04-26T21:45:02.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde525fef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae41fd7670 0x55ae41f8e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde525fef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae4204dd60 0x55ae42062eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde525fef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae4204b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde525fef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ae41fd7240 con 0x55ae4206b610 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde525fef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ae42067080 con 0x55ae41fd7670 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde525fef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ae41fd6f60 con 0x55ae4204dd60 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae4204b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae4204b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50682/0 (socket says 172.21.15.40:50682) 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 learned_addr learned my addr 172.21.15.40:0/596500625 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde500fa640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae41fd7670 0x55ae41f8e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde4f8f9640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae4204dd60 0x55ae42062eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae4204dd60 msgr2=0x55ae42062eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ae4204dd60 0x55ae42062eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae41fd7670 msgr2=0x55ae41f8e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae41fd7670 0x55ae41f8e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae42063fe0 con 0x55ae4206b610 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde508fb640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae4204b340 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fde4800a8b0 tx=0x7fde48002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde4d8f5640 1 -- 172.21.15.40:0/596500625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48024070 con 0x55ae4206b610 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde4d8f5640 1 -- 172.21.15.40:0/596500625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde48014420 con 0x55ae4206b610 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ae420642c0 con 0x55ae4206b610 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.784+0000 7fde4d8f5640 1 -- 172.21.15.40:0/596500625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48014890 con 0x55ae4206b610 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.785+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae42064880 con 0x55ae4206b610 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.786+0000 7fde4d8f5640 1 -- 172.21.15.40:0/596500625 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde48027070 con 0x55ae4206b610 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.786+0000 7fde4d8f5640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde3006b2d0 0x7fde3006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.786+0000 7fde4d8f5640 1 -- 172.21.15.40:0/596500625 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8367+0+0 (secure 0 0 0) 0x7fde4808c050 con 0x55ae4206b610 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.787+0000 7fde500fa640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde3006b2d0 0x7fde3006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.789 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.787+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ae41f8e6b0 0x55ae41fd48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.787+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae420633f0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.787+0000 7fde4f8f9640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ae41f8e6b0 0x55ae41fd48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.787+0000 7fde500fa640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde3006b2d0 0x7fde3006d790 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fde40000f80 tx=0x7fde40006370 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.787+0000 7fde4f8f9640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ae41f8e6b0 0x55ae41fd48f0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.789+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v120'753 uv727 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.789+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae41fd4e30 con 0x55ae41f8e6b0 2024-04-26T21:45:02.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.790+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v120'754 uv754 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.790+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae41fd5200 con 0x55ae41f8e6b0 2024-04-26T21:45:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.791+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v120'755 uv755 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.792 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.791+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae41fd55a0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.792+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v120'756 uv756 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.792+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae41fd59a0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.794+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v120'757 uv757 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.794+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210b1b0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.795+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v120'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.795+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210b550 con 0x55ae41f8e6b0 2024-04-26T21:45:02.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.796+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v120'759 uv733 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.796+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4210b8f0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.797+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v120'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.797+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210b8f0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.798+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v120'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.799 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:02.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.798+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210c900 con 0x55ae41f8e6b0 2024-04-26T21:45:02.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.799+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v120'762 uv762 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.799+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210c900 con 0x55ae41f8e6b0 2024-04-26T21:45:02.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.800+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v120'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.801+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210b8f0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.802+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v120'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.802+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210b8f0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.803+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v120'765 uv740 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.803+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4210b8f0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.804+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v120'766 uv766 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.804+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210b8f0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.805+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v120'767 uv767 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.806 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:02.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.805+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210c900 con 0x55ae41f8e6b0 2024-04-26T21:45:02.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.806+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v120'768 uv768 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.806+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210c900 con 0x55ae41f8e6b0 2024-04-26T21:45:02.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.807+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v120'769 uv769 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.807+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210b8f0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.808+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v120'770 uv770 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.808+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.810+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v120'771 uv771 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.810+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.811+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v120'772 uv746 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.811+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.812+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v120'773 uv773 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.812+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.814 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:02.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.813+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v120'774 uv774 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.813+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210cdb0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.814+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v120'775 uv775 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.814+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210cdb0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.815+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v120'776 uv776 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.815+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.816+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v120'777 uv777 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.816+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.817+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v120'778 uv752 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.817+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.818+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v120'779 uv779 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.819+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.820+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v120'780 uv780 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.821 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.820+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210cdb0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.821+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v120'781 uv781 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.821+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210cdb0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.822+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v120'782 uv782 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.822+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210bda0 con 0x55ae41f8e6b0 2024-04-26T21:45:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.823+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v120'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae41f8e6b0 2024-04-26T21:45:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.823+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ae4210c2b0 0x55ae4210e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.823+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210ebe0 con 0x55ae4210c2b0 2024-04-26T21:45:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.823+0000 7fde500fa640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ae4210c2b0 0x55ae4210e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.823+0000 7fde500fa640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ae4210c2b0 0x55ae4210e6a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.825+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v120'549 uv531 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.825+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4210ef80 con 0x55ae4210c2b0 2024-04-26T21:45:02.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.827+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v120'550 uv550 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.827+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210f320 con 0x55ae4210c2b0 2024-04-26T21:45:02.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.828+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v120'551 uv551 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.829 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:02.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.828+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210fc10 con 0x55ae4210c2b0 2024-04-26T21:45:02.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.829+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v120'552 uv552 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.829+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4210ffb0 con 0x55ae4210c2b0 2024-04-26T21:45:02.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.831+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v120'553 uv553 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.831+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110350 con 0x55ae4210c2b0 2024-04-26T21:45:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.832+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v120'554 uv548 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.832+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae421106f0 con 0x55ae4210c2b0 2024-04-26T21:45:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.833+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v120'555 uv555 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.833+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110ae0 con 0x55ae4210c2b0 2024-04-26T21:45:02.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.835+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v120'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.835 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:02.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.835+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110ae0 con 0x55ae4210c2b0 2024-04-26T21:45:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.836+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v120'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.836+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.837+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v120'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.837+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.838+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v120'559 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.838+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.840+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v120'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.840+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.841+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v120'561 uv561 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.842 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:02.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.841+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.842+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v120'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.842+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.844+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v120'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.844+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.845+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v120'564 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.845+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.846+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v120'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.846+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110a90 con 0x55ae4210c2b0 2024-04-26T21:45:02.847 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:02.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.847+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v120'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.847+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42111e90 con 0x55ae4210c2b0 2024-04-26T21:45:02.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.848+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v120'567 uv567 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.848+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42111b20 con 0x55ae4210c2b0 2024-04-26T21:45:02.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.849+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v120'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.849+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42110b10 con 0x55ae4210c2b0 2024-04-26T21:45:02.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.850+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v120'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae4210c2b0 2024-04-26T21:45:02.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.850+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ae42111410 0x55ae421117f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.850+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42113f20 con 0x55ae42111410 2024-04-26T21:45:02.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.850+0000 7fde4f8f9640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ae42111410 0x55ae421117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.851+0000 7fde4f8f9640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ae42111410 0x55ae421117f0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.852+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v120'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.852+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae42113f20 con 0x55ae42111410 2024-04-26T21:45:02.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.853+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v120'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.853+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42113f20 con 0x55ae42111410 2024-04-26T21:45:02.856 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:02.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.855+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v120'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.855+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42114fd0 con 0x55ae42111410 2024-04-26T21:45:02.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.856+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v120'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.856+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42114f90 con 0x55ae42111410 2024-04-26T21:45:02.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.857+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v120'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.857+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42114f90 con 0x55ae42111410 2024-04-26T21:45:02.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.858+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v120'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.858+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ae42114430 0x55ae421148c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.858+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42117410 con 0x55ae42114430 2024-04-26T21:45:02.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.859+0000 7fde508fb640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ae42114430 0x55ae421148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.859+0000 7fde508fb640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ae42114430 0x55ae421148c0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.860+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v120'112 uv111 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.860+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae421177b0 con 0x55ae42114430 2024-04-26T21:45:02.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.862+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v120'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.862+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42117c20 con 0x55ae42114430 2024-04-26T21:45:02.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.863+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v120'114 uv114 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.864 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:02.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.863+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae421184c0 con 0x55ae42114430 2024-04-26T21:45:02.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.864+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v120'115 uv115 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.864+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ae42118d70 0x55ae4211b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.864+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211b6a0 con 0x55ae42118d70 2024-04-26T21:45:02.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.864+0000 7fde500fa640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ae42118d70 0x55ae4211b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.865+0000 7fde500fa640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ae42118d70 0x55ae4211b160 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.866+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v120'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.866+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211b760 con 0x55ae42118d70 2024-04-26T21:45:02.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.867+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v120'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.867+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211bb80 con 0x55ae42118d70 2024-04-26T21:45:02.869 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:02.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.868+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v120'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.868+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211cb90 con 0x55ae42118d70 2024-04-26T21:45:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.869+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v120'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.869+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211cb90 con 0x55ae42118d70 2024-04-26T21:45:02.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.871+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v120'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.871+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211bb80 con 0x55ae42118d70 2024-04-26T21:45:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.872+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v120'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.872+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211bb80 con 0x55ae42114430 2024-04-26T21:45:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.873+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v120'271 uv265 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.873+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211bf20 con 0x55ae42114430 2024-04-26T21:45:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.874+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v120'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.874+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c2c0 con 0x55ae42114430 2024-04-26T21:45:02.876 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.875+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v120'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.875+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211d670 con 0x55ae42114430 2024-04-26T21:45:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.877+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v120'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.877+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211d670 con 0x55ae42114430 2024-04-26T21:45:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.878+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v120'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.878+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.879+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v120'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.879+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.880+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v120'277 uv270 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.880+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.881+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v120'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.881+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.882+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v120'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.883 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.883+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.884+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v120'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.884+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.885+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v120'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.885+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.886+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v120'122 uv121 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.886+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.887+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v120'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.887+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.889 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.888+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v120'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.888+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.889+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v120'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.889+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42114430 2024-04-26T21:45:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.890+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v120'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde48026030 con 0x55ae42114430 2024-04-26T21:45:02.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.890+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.892+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v120'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.892+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.893+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v120'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.893+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.894 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:02.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.894+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v120'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.894+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211d6b0 con 0x55ae42118d70 2024-04-26T21:45:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.895+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v120'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.895+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211d670 con 0x55ae42118d70 2024-04-26T21:45:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.896+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v120'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.896+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.897+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v120'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.897+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.898+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v120'243 uv237 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.898+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.899+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v120'244 uv244 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.899+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.900 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:02.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.900+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v120'245 uv245 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.900+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211ca50 con 0x55ae42118d70 2024-04-26T21:45:02.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.901+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v120'246 uv246 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.901+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211ca50 con 0x55ae42118d70 2024-04-26T21:45:02.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.902+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v120'247 uv247 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.902+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.904+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v120'248 uv242 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.904+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211ca50 con 0x55ae42118d70 2024-04-26T21:45:02.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.905+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v120'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.905+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.907 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:02.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.906+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v120'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.906+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211c660 con 0x55ae42118d70 2024-04-26T21:45:02.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.907+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v120'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.907+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211ce10 con 0x55ae42118d70 2024-04-26T21:45:02.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.909+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v120'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.909+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ae4211d4d0 0x55ae4211f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.909+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae4211d4d0 2024-04-26T21:45:02.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.909+0000 7fde4f8f9640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ae4211d4d0 0x55ae4211f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.909+0000 7fde4f8f9640 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ae4211d4d0 0x55ae4211f8c0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.910+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v120'107 uv106 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae4211d4d0 2024-04-26T21:45:02.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.910+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae4211d4d0 2024-04-26T21:45:02.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.911+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v120'108 uv108 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae4211d4d0 2024-04-26T21:45:02.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.911+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae4211d4d0 2024-04-26T21:45:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.914+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v120'109 uv109 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae4211d4d0 2024-04-26T21:45:02.915 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.914+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae4211d4d0 2024-04-26T21:45:02.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.916+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v120'110 uv110 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae4211d4d0 2024-04-26T21:45:02.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.917+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42118d70 2024-04-26T21:45:02.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.918+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v120'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.918+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42118d70 2024-04-26T21:45:02.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.919+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v120'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.919+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42118d70 2024-04-26T21:45:02.920 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:02.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.920+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v120'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.920+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42120e10 con 0x55ae42118d70 2024-04-26T21:45:02.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.921+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v120'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.921+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42120e10 con 0x55ae42118d70 2024-04-26T21:45:02.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.922+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v120'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.922+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42118d70 2024-04-26T21:45:02.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.923+0000 7fde500fa640 1 -- 172.21.15.40:0/596500625 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v120'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde40014730 con 0x55ae42118d70 2024-04-26T21:45:02.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.923+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.924+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v120'125 uv124 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.924+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.925+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v120'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.925+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.927 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:02.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.927+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v120'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.927+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.928+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v120'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.928+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.929+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v120'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.929+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.930+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v120'302 uv294 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.930+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.931+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v120'303 uv303 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.931+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.932+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v120'304 uv304 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.933 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.932+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.933+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v120'305 uv305 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.933+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.934+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v120'306 uv306 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.934+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.935+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v120'307 uv301 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.935+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.936+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v120'308 uv308 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.936+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.937+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v120'309 uv309 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.938 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.937+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42120e10 con 0x55ae42111410 2024-04-26T21:45:02.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.939+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v120'310 uv310 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.939+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae42120e10 con 0x55ae42111410 2024-04-26T21:45:02.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.940+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v120'311 uv311 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.940+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae4211fe00 con 0x55ae42111410 2024-04-26T21:45:02.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.941+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v120'312 uv312 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.942+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ae421202b0 con 0x55ae42111410 2024-04-26T21:45:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.942+0000 7fde4f8f9640 1 -- 172.21.15.40:0/596500625 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v120'313 uv313 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde38055940 con 0x55ae42111410 2024-04-26T21:45:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.942+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ae4210c2b0 msgr2=0x55ae4210e6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.942+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ae4210c2b0 0x55ae4210e6a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.942+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ae41f8e6b0 msgr2=0x55ae41fd48f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.942+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ae41f8e6b0 0x55ae41fd48f0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.942+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ae42114430 msgr2=0x55ae421148c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ae42114430 0x55ae421148c0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ae4211d4d0 msgr2=0x55ae4211f8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ae4211d4d0 0x55ae4211f8c0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ae42111410 msgr2=0x55ae421117f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ae42111410 0x55ae421117f0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ae42118d70 msgr2=0x55ae4211b160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ae42118d70 0x55ae4211b160 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 reap_dead start 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 reap_dead start 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde508fb640 1 -- 172.21.15.40:0/596500625 reap_dead start 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde3006b2d0 msgr2=0x7fde3006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde3006b2d0 0x7fde3006d790 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fde40000f80 tx=0x7fde40006370 comp rx=0 tx=0).stop 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 msgr2=0x55ae4204b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae4204b340 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fde4800a8b0 tx=0x7fde48002f70 comp rx=0 tx=0).stop 2024-04-26T21:45:02.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 shutdown_connections 2024-04-26T21:45:02.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde3006b2d0 0x7fde3006d790 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 --2- 172.21.15.40:0/596500625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae4206b610 0x55ae4204b340 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 >> 172.21.15.40:0/596500625 conn(0x55ae41f8eab0 msgr2=0x55ae42067380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:02.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 shutdown_connections 2024-04-26T21:45:02.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:02.943+0000 7fde525fef40 1 -- 172.21.15.40:0/596500625 wait complete. 2024-04-26T21:45:02.947 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000001 0 2024-04-26T21:45:03.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.030+0000 7fa940720f40 1 Processor -- start 2024-04-26T21:45:03.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.030+0000 7fa940720f40 1 -- start start 2024-04-26T21:45:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa940720f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132bb6420 0x557132c42650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa940720f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132c3df70 0x557132b9ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa940720f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c44b70 0x557132bb29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa940720f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557132bb5660 con 0x557132bb6420 2024-04-26T21:45:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa940720f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557132bb5940 con 0x557132c3df70 2024-04-26T21:45:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa940720f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557132bb5c20 con 0x557132c44b70 2024-04-26T21:45:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132bb6420 0x557132c42650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132bb6420 0x557132c42650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50698/0 (socket says 172.21.15.40:50698) 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 -- 172.21.15.40:0/1715120446 learned_addr learned my addr 172.21.15.40:0/1715120446 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93e100640 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c44b70 0x557132bb29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 -- 172.21.15.40:0/1715120446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c44b70 msgr2=0x557132bb29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c44b70 0x557132bb29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 -- 172.21.15.40:0/1715120446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132c3df70 msgr2=0x557132b9ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d0fe640 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132c3df70 0x557132b9ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132c3df70 0x557132b9ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 -- 172.21.15.40:0/1715120446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557132a96cb0 con 0x557132bb6420 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa93d8ff640 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132bb6420 0x557132c42650 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c4a0 tx=0x7fa93000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143e222ba572797d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa937fff640 1 -- 172.21.15.40:0/1715120446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9300115f0 con 0x557132bb6420 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa937fff640 1 -- 172.21.15.40:0/1715120446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa930017070 con 0x557132bb6420 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.031+0000 7fa937fff640 1 -- 172.21.15.40:0/1715120446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9300119d0 con 0x557132bb6420 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 -- 172.21.15.40:0/1715120446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132bb6420 msgr2=0x557132c42650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132bb6420 0x557132c42650 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c4a0 tx=0x7fa93000c960 comp rx=0 tx=0).stop 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 -- 172.21.15.40:0/1715120446 shutdown_connections 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c44b70 0x557132bb29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132c3df70 0x557132b9ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 --2- 172.21.15.40:0/1715120446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132bb6420 0x557132c42650 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 -- 172.21.15.40:0/1715120446 >> 172.21.15.40:0/1715120446 conn(0x557132b968a0 msgr2=0x557132b96170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 -- 172.21.15.40:0/1715120446 shutdown_connections 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 -- 172.21.15.40:0/1715120446 wait complete. 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.032+0000 7fa940720f40 1 Processor -- start 2024-04-26T21:45:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa940720f40 1 -- start start 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa940720f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132bb6420 0x557132c346e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa940720f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c3df70 0x557132cdaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa940720f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132c44b70 0x557132cdb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa940720f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557132c35660 con 0x557132c44b70 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa940720f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557132b97f60 con 0x557132bb6420 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa940720f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557132c35940 con 0x557132c3df70 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa93e100640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132c44b70 0x557132cdb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa93e100640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132c44b70 0x557132cdb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50710/0 (socket says 172.21.15.40:50710) 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa93e100640 1 -- 172.21.15.40:0/1389563380 learned_addr learned my addr 172.21.15.40:0/1389563380 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa93d8ff640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132bb6420 0x557132c346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.033+0000 7fa93d0fe640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c3df70 0x557132cdaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa93e100640 1 -- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c3df70 msgr2=0x557132cdaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa93e100640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557132c3df70 0x557132cdaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa93e100640 1 -- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132bb6420 msgr2=0x557132c346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa93e100640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557132bb6420 0x557132c346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa93e100640 1 -- 172.21.15.40:0/1389563380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557132cdb440 con 0x557132c44b70 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa93e100640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132c44b70 0x557132cdb640 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fa93800bdf0 tx=0x7fa93800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa936ffd640 1 -- 172.21.15.40:0/1389563380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938007a90 con 0x557132c44b70 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa936ffd640 1 -- 172.21.15.40:0/1389563380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938007c50 con 0x557132c44b70 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa936ffd640 1 -- 172.21.15.40:0/1389563380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938015420 con 0x557132c44b70 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557132c3e680 con 0x557132c44b70 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.034+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557132c3ec40 con 0x557132c44b70 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.035+0000 7fa936ffd640 1 -- 172.21.15.40:0/1389563380 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa93801e440 con 0x557132c44b70 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.036+0000 7fa936ffd640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa92006b2d0 0x7fa92006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.036+0000 7fa936ffd640 1 -- 172.21.15.40:0/1389563380 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8367+0+0 (secure 0 0 0) 0x7fa93808a850 con 0x557132c44b70 2024-04-26T21:45:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.036+0000 7fa93d8ff640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa92006b2d0 0x7fa92006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.036+0000 7fa940720f40 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x557132bb2600 0x557132c3f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.036+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557132c3fc60 con 0x557132bb2600 2024-04-26T21:45:03.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.036+0000 7fa93d0fe640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x557132bb2600 0x557132c3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.037+0000 7fa93d8ff640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa92006b2d0 0x7fa92006d790 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa930000f80 tx=0x7fa93000e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.037+0000 7fa93d0fe640 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x557132bb2600 0x557132c3f720 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa93d0fe640 1 -- 172.21.15.40:0/1389563380 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(1 200.00000001 [truncate 0] v120'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa92805d040 con 0x557132bb2600 2024-04-26T21:45:03.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x557132bb2600 msgr2=0x557132c3f720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x557132bb2600 0x557132c3f720 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa92006b2d0 msgr2=0x7fa92006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa92006b2d0 0x7fa92006d790 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa930000f80 tx=0x7fa93000e040 comp rx=0 tx=0).stop 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132c44b70 msgr2=0x557132cdb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 --2- 172.21.15.40:0/1389563380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557132c44b70 0x557132cdb640 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fa93800bdf0 tx=0x7fa93800bee0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa93e100640 1 -- 172.21.15.40:0/1389563380 reap_dead start 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 shutdown_connections 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 >> 172.21.15.40:0/1389563380 conn(0x557132b968a0 msgr2=0x557132c46a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 shutdown_connections 2024-04-26T21:45:03.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.038+0000 7fa940720f40 1 -- 172.21.15.40:0/1389563380 wait complete. 2024-04-26T21:45:03.042 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:03.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.184+0000 7fad3d87b640 1 Processor -- start 2024-04-26T21:45:03.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.184+0000 7fad3d87b640 1 -- start start 2024-04-26T21:45:03.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.185+0000 7fad3d87b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.185+0000 7fad3d87b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fad38162cd0 0x7fad380d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.185+0000 7fad3d87b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad380d0890 con 0x7fad380d1550 2024-04-26T21:45:03.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.185+0000 7fad3d87b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fad380d1130 con 0x7fad38162cd0 2024-04-26T21:45:03.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.185+0000 7fad3d87b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fad380d0b70 con 0x7fad381628f0 2024-04-26T21:45:03.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad36ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad367fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fad38162cd0 0x7fad380d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad377fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fad380d1550 0x7fad380d2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39792/0 (socket says 172.21.15.40:39792) 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad36ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:50344/0 (socket says 172.21.15.40:50344) 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad377fe640 1 -- 172.21.15.40:0/3556009188 learned_addr learned my addr 172.21.15.40:0/3556009188 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad36ffd640 1 -- 172.21.15.40:0/3556009188 >> v2:172.21.15.184:3300/0 conn(0x7fad38162cd0 msgr2=0x7fad380d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad36ffd640 1 --2- 172.21.15.40:0/3556009188 >> v2:172.21.15.184:3300/0 conn(0x7fad38162cd0 0x7fad380d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad36ffd640 1 -- 172.21.15.40:0/3556009188 >> v1:172.21.15.184:6789/0 conn(0x7fad380d1550 legacy=0x7fad380d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.186+0000 7fad36ffd640 1 -- 172.21.15.40:0/3556009188 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad380d0e50 con 0x7fad381628f0 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.187+0000 7fad36ffd640 1 --2- 172.21.15.40:0/3556009188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380d1a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fad300097a0 tx=0x7fad3002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=243e288c48b804f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.187+0000 7fad35ffb640 1 -- 172.21.15.40:0/3556009188 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30037070 con 0x7fad381628f0 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.187+0000 7fad35ffb640 1 -- 172.21.15.40:0/3556009188 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3002ae50 con 0x7fad381628f0 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.187+0000 7fad35ffb640 1 -- 172.21.15.40:0/3556009188 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30032660 con 0x7fad381628f0 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.187+0000 7fad3d87b640 1 -- 172.21.15.40:0/3556009188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 msgr2=0x7fad380d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.187+0000 7fad3d87b640 1 --2- 172.21.15.40:0/3556009188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380d1a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fad300097a0 tx=0x7fad3002a100 comp rx=0 tx=0).stop 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.188+0000 7fad3d87b640 1 -- 172.21.15.40:0/3556009188 shutdown_connections 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.188+0000 7fad3d87b640 1 --2- 172.21.15.40:0/3556009188 >> v2:172.21.15.184:3300/0 conn(0x7fad38162cd0 0x7fad380d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.188+0000 7fad3d87b640 1 --2- 172.21.15.40:0/3556009188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380d1a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.188+0000 7fad3d87b640 1 -- 172.21.15.40:0/3556009188 >> 172.21.15.40:0/3556009188 conn(0x7fad380d4ff0 msgr2=0x7fad380d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:03.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.188+0000 7fad3d87b640 1 -- 172.21.15.40:0/3556009188 shutdown_connections 2024-04-26T21:45:03.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.188+0000 7fad3d87b640 1 -- 172.21.15.40:0/3556009188 wait complete. 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.189+0000 7fad3d87b640 1 Processor -- start 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.189+0000 7fad3d87b640 1 -- start start 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad3d87b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380d1550 0x7fad38151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad3d87b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad3d87b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad38162cd0 0x7fad3815e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad3d87b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad380b5df0 con 0x7fad380d1550 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad3d87b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad380b5c70 con 0x7fad38162cd0 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad3d87b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fad380b5f70 con 0x7fad381628f0 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380d1550 0x7fad38151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380d1550 0x7fad38151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50716/0 (socket says 172.21.15.40:50716) 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 -- 172.21.15.40:0/669032491 learned_addr learned my addr 172.21.15.40:0/669032491 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad367fc640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad377fe640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad38162cd0 0x7fad3815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 -- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 msgr2=0x7fad380ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 -- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad38162cd0 msgr2=0x7fad3815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad38162cd0 0x7fad3815e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 -- 172.21.15.40:0/669032491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad380cde50 con 0x7fad380d1550 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad377fe640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad38162cd0 0x7fad3815e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad36ffd640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380d1550 0x7fad38151320 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fad300098c0 tx=0x7fad300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.190+0000 7fad2ffff640 1 -- 172.21.15.40:0/669032491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30041030 con 0x7fad380d1550 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.191+0000 7fad2ffff640 1 -- 172.21.15.40:0/669032491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3002ac50 con 0x7fad380d1550 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.191+0000 7fad2ffff640 1 -- 172.21.15.40:0/669032491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30037040 con 0x7fad380d1550 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.191+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad380ccad0 con 0x7fad380d1550 2024-04-26T21:45:03.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.191+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad380cd370 con 0x7fad380d1550 2024-04-26T21:45:03.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.192+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad380694f0 con 0x7fad380d1550 2024-04-26T21:45:03.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.192+0000 7fad2ffff640 1 -- 172.21.15.40:0/669032491 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad30044070 con 0x7fad380d1550 2024-04-26T21:45:03.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.192+0000 7fad2ffff640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1806c920 0x7fad1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.193+0000 7fad2ffff640 1 -- 172.21.15.40:0/669032491 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8367+0+0 (secure 0 0 0) 0x7fad300acbb0 con 0x7fad380d1550 2024-04-26T21:45:03.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.193+0000 7fad367fc640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1806c920 0x7fad1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.193+0000 7fad367fc640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1806c920 0x7fad1806ede0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fad381f8960 tx=0x7fad20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.197+0000 7fad2ffff640 1 -- 172.21.15.40:0/669032491 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad380cde50 con 0x7fad380d1550 2024-04-26T21:45:03.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.422+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fad3806d620 con 0x7fad380d1550 2024-04-26T21:45:03.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.724+0000 7fad2ffff640 1 -- 172.21.15.40:0/669032491 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v247) ==== 162+0+0 (secure 0 0 0) 0x7fad3007a650 con 0x7fad380d1550 2024-04-26T21:45:03.725 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1806c920 msgr2=0x7fad1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1806c920 0x7fad1806ede0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fad381f8960 tx=0x7fad20008040 comp rx=0 tx=0).stop 2024-04-26T21:45:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380d1550 msgr2=0x7fad38151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380d1550 0x7fad38151320 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fad300098c0 tx=0x7fad300046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 shutdown_connections 2024-04-26T21:45:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1806c920 0x7fad1806ede0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad38162cd0 0x7fad3815e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad381628f0 0x7fad380ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 --2- 172.21.15.40:0/669032491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380d1550 0x7fad38151320 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 >> 172.21.15.40:0/669032491 conn(0x7fad380d4ff0 msgr2=0x7fad380b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 shutdown_connections 2024-04-26T21:45:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.726+0000 7fad3d87b640 1 -- 172.21.15.40:0/669032491 wait complete. 2024-04-26T21:45:03.735 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:03.735+0000 7f0d27a94640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:45:03.736 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:03.735+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:45:03.736 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:03.735+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:45:03.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:03.742 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:45:03.735+0000 7f0d27a94640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-26T21:45:03.767 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:45:03.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.841+0000 7f2669462640 1 Processor -- start 2024-04-26T21:45:03.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.842+0000 7f2669462640 1 -- start start 2024-04-26T21:45:03.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.842+0000 7f2669462640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f266415f290 0x7f26640cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.842+0000 7f2669462640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2664163590 0x7f26640726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.842+0000 7f2669462640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266406f5a0 con 0x7f26640cd060 2024-04-26T21:45:03.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.842+0000 7f2669462640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f266406f350 con 0x7f266415f290 2024-04-26T21:45:03.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.842+0000 7f2669462640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f266406f880 con 0x7f2664163590 2024-04-26T21:45:03.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.843+0000 7f2662ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f266415f290 0x7f26640cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.843+0000 7f26627fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f26640cd060 0x7f2664162e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52780/0 (socket says 172.21.15.40:52780) 2024-04-26T21:45:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.843+0000 7f2662ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f266415f290 0x7f26640cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50728/0 (socket says 172.21.15.40:50728) 2024-04-26T21:45:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.843+0000 7f26627fc640 1 -- 172.21.15.40:0/2571987063 learned_addr learned my addr 172.21.15.40:0/2571987063 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.843+0000 7f2662ffd640 1 -- 172.21.15.40:0/2571987063 >> v2:172.21.15.184:3300/0 conn(0x7f2664163590 msgr2=0x7f26640726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.843+0000 7f2662ffd640 1 --2- 172.21.15.40:0/2571987063 >> v2:172.21.15.184:3300/0 conn(0x7f2664163590 0x7f26640726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.843+0000 7f2662ffd640 1 -- 172.21.15.40:0/2571987063 >> v1:172.21.15.40:6789/0 conn(0x7f26640cd060 legacy=0x7f2664162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.843+0000 7f2662ffd640 1 -- 172.21.15.40:0/2571987063 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266406fb60 con 0x7f266415f290 2024-04-26T21:45:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2662ffd640 1 --2- 172.21.15.40:0/2571987063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640cb7e0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f2664070350 tx=0x7f265402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7154dfef3156569 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2661ffb640 1 -- 172.21.15.40:0/2571987063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654037070 con 0x7f266415f290 2024-04-26T21:45:03.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2661ffb640 1 -- 172.21.15.40:0/2571987063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2654032440 con 0x7f266415f290 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2661ffb640 1 -- 172.21.15.40:0/2571987063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654032810 con 0x7f266415f290 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2669462640 1 -- 172.21.15.40:0/2571987063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 msgr2=0x7f26640cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2669462640 1 --2- 172.21.15.40:0/2571987063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640cb7e0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f2664070350 tx=0x7f265402a230 comp rx=0 tx=0).stop 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2669462640 1 -- 172.21.15.40:0/2571987063 shutdown_connections 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2669462640 1 --2- 172.21.15.40:0/2571987063 >> v2:172.21.15.184:3300/0 conn(0x7f2664163590 0x7f26640726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2669462640 1 --2- 172.21.15.40:0/2571987063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640cb7e0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2669462640 1 -- 172.21.15.40:0/2571987063 >> 172.21.15.40:0/2571987063 conn(0x7f26640d0fd0 msgr2=0x7f26640d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2669462640 1 -- 172.21.15.40:0/2571987063 shutdown_connections 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.844+0000 7f2669462640 1 -- 172.21.15.40:0/2571987063 wait complete. 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.846+0000 7f2669462640 1 Processor -- start 2024-04-26T21:45:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.846+0000 7f2669462640 1 -- start start 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.846+0000 7f2669462640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26640cd060 0x7f26641f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.846+0000 7f2669462640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.846+0000 7f2669462640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2664163590 0x7f26640b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.846+0000 7f2669462640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26640cd650 con 0x7f266415f290 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.846+0000 7f2669462640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26640cd4d0 con 0x7f26640cd060 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.846+0000 7f2669462640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2664072c00 con 0x7f2664163590 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50730/0 (socket says 172.21.15.40:50730) 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 -- 172.21.15.40:0/279892214 learned_addr learned my addr 172.21.15.40:0/279892214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f2662ffd640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26640cd060 0x7f26641f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26637fe640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2664163590 0x7f26640b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 -- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2664163590 msgr2=0x7f26640b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2664163590 0x7f26640b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 -- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26640cd060 msgr2=0x7f26641f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26640cd060 0x7f26641f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 -- 172.21.15.40:0/279892214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26641f58b0 con 0x7f266415f290 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f26627fc640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640b17c0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f265800c400 tx=0x7f265800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f264b7fe640 1 -- 172.21.15.40:0/279892214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658017070 con 0x7f266415f290 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f264b7fe640 1 -- 172.21.15.40:0/279892214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658005ce0 con 0x7f266415f290 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26641604d0 con 0x7f266415f290 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f264b7fe640 1 -- 172.21.15.40:0/279892214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580072f0 con 0x7f266415f290 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.847+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26640b45a0 con 0x7f266415f290 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.848+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26640694f0 con 0x7f266415f290 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.849+0000 7f264b7fe640 1 -- 172.21.15.40:0/279892214 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f265801a070 con 0x7f266415f290 2024-04-26T21:45:03.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.849+0000 7f264b7fe640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f265006c8f0 0x7f265006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:03.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.849+0000 7f264b7fe640 1 -- 172.21.15.40:0/279892214 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f265808b900 con 0x7f266415f290 2024-04-26T21:45:03.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.849+0000 7f2662ffd640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f265006c8f0 0x7f265006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:03.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.850+0000 7f2662ffd640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f265006c8f0 0x7f265006edb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2654034000 tx=0x7f265402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:03.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:03.853+0000 7f264b7fe640 1 -- 172.21.15.40:0/279892214 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26641f58b0 con 0x7f266415f290 2024-04-26T21:45:04.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.084+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26640c7a60 con 0x7f266415f290 2024-04-26T21:45:04.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.084+0000 7f264b7fe640 1 -- 172.21.15.40:0/279892214 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 249 v249) ==== 95+0+4355 (secure 0 0 0) 0x7f2658058c60 con 0x7f266415f290 2024-04-26T21:45:04.085 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:04.085 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":249,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":249,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:03.737132+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:04.086 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 249 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f265006c8f0 msgr2=0x7f265006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f265006c8f0 0x7f265006edb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2654034000 tx=0x7f265402e000 comp rx=0 tx=0).stop 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 msgr2=0x7f26640b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640b17c0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f265800c400 tx=0x7f265800c8c0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 shutdown_connections 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f265006c8f0 0x7f265006edb0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2664163590 0x7f26640b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f266415f290 0x7f26640b17c0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 --2- 172.21.15.40:0/279892214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26640cd060 0x7f26641f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 >> 172.21.15.40:0/279892214 conn(0x7f26640d0fd0 msgr2=0x7f26640d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.086+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 shutdown_connections 2024-04-26T21:45:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.087+0000 7f2669462640 1 -- 172.21.15.40:0/279892214 wait complete. 2024-04-26T21:45:04.098 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:04.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:04.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.239+0000 7f9241247640 1 Processor -- start 2024-04-26T21:45:04.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.239+0000 7f9241247640 1 -- start start 2024-04-26T21:45:04.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f9241247640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f923c162880 0x7f923c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f9241247640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f923c0b5f20 0x7f923c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f9241247640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f923c0b7e50 con 0x7f923c0d39f0 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f9241247640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f923c0d3e40 con 0x7f923c162880 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f9241247640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f923c0d0580 con 0x7f923c0b5f20 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f923ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f923c162880 0x7f923c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f923ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f923c162880 0x7f923c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50746/0 (socket says 172.21.15.40:50746) 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f923ad9d640 1 -- 172.21.15.40:0/871418916 learned_addr learned my addr 172.21.15.40:0/871418916 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f9233fff640 1 --2- 172.21.15.40:0/871418916 >> v2:172.21.15.184:3300/0 conn(0x7f923c0b5f20 0x7f923c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f923ad9d640 1 -- 172.21.15.40:0/871418916 >> v2:172.21.15.184:3300/0 conn(0x7f923c0b5f20 msgr2=0x7f923c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f923ad9d640 1 --2- 172.21.15.40:0/871418916 >> v2:172.21.15.184:3300/0 conn(0x7f923c0b5f20 0x7f923c0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f923ad9d640 1 -- 172.21.15.40:0/871418916 >> v1:172.21.15.184:6789/0 conn(0x7f923c0d39f0 legacy=0x7f923c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.240+0000 7f923ad9d640 1 -- 172.21.15.40:0/871418916 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c0d1800 con 0x7f923c162880 2024-04-26T21:45:04.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.241+0000 7f923ad9d640 1 --2- 172.21.15.40:0/871418916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c162880 0x7f923c0d2cb0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f923c08d3d0 tx=0x7f922c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12816fc6959d3783 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:04.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.241+0000 7f923a59c640 1 -- 172.21.15.40:0/871418916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c037070 con 0x7f923c162880 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.241+0000 7f923a59c640 1 -- 172.21.15.40:0/871418916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922c032440 con 0x7f923c162880 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.241+0000 7f923a59c640 1 -- 172.21.15.40:0/871418916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c032810 con 0x7f923c162880 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.242+0000 7f9241247640 1 -- 172.21.15.40:0/871418916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c162880 msgr2=0x7f923c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.242+0000 7f9241247640 1 --2- 172.21.15.40:0/871418916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c162880 0x7f923c0d2cb0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f923c08d3d0 tx=0x7f922c02a230 comp rx=0 tx=0).stop 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.242+0000 7f9241247640 1 -- 172.21.15.40:0/871418916 shutdown_connections 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.242+0000 7f9241247640 1 --2- 172.21.15.40:0/871418916 >> v2:172.21.15.184:3300/0 conn(0x7f923c0b5f20 0x7f923c0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.242+0000 7f9241247640 1 --2- 172.21.15.40:0/871418916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c162880 0x7f923c0d2cb0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.242+0000 7f9241247640 1 -- 172.21.15.40:0/871418916 >> 172.21.15.40:0/871418916 conn(0x7f923c0d4f60 msgr2=0x7f923c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.242+0000 7f9241247640 1 -- 172.21.15.40:0/871418916 shutdown_connections 2024-04-26T21:45:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.242+0000 7f9241247640 1 -- 172.21.15.40:0/871418916 wait complete. 2024-04-26T21:45:04.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.243+0000 7f9241247640 1 Processor -- start 2024-04-26T21:45:04.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.243+0000 7f9241247640 1 -- start start 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9241247640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f923c0b5f20 0x7f923c14d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9241247640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c0d39f0 0x7f923c0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9241247640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f923c162880 0x7f923c160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9241247640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f923c0d0d90 con 0x7f923c0d39f0 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9241247640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f923c0d0ab0 con 0x7f923c0b5f20 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9241247640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f923c162c60 con 0x7f923c162880 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9233fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c0d39f0 0x7f923c0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9233fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c0d39f0 0x7f923c0ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50758/0 (socket says 172.21.15.40:50758) 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9233fff640 1 -- 172.21.15.40:0/76066619 learned_addr learned my addr 172.21.15.40:0/76066619 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9233fff640 1 -- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f923c162880 msgr2=0x7f923c160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f923b59e640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f923c162880 0x7f923c160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9233fff640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f923c162880 0x7f923c160420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9233fff640 1 -- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f923c0b5f20 msgr2=0x7f923c14d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9233fff640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f923c0b5f20 0x7f923c14d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.244+0000 7f9233fff640 1 -- 172.21.15.40:0/76066619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c159a10 con 0x7f923c0d39f0 2024-04-26T21:45:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.245+0000 7f9233fff640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c0d39f0 0x7f923c0ce5e0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f922400d620 tx=0x7f922400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.245+0000 7f92337fe640 1 -- 172.21.15.40:0/76066619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922400b620 con 0x7f923c0d39f0 2024-04-26T21:45:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.245+0000 7f92337fe640 1 -- 172.21.15.40:0/76066619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9224019070 con 0x7f923c0d39f0 2024-04-26T21:45:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.245+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923c15a0d0 con 0x7f923c0d39f0 2024-04-26T21:45:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.245+0000 7f92337fe640 1 -- 172.21.15.40:0/76066619 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922400ba00 con 0x7f923c0d39f0 2024-04-26T21:45:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.245+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923c15a690 con 0x7f923c0d39f0 2024-04-26T21:45:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.246+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923c0694f0 con 0x7f923c0d39f0 2024-04-26T21:45:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.246+0000 7f92337fe640 1 -- 172.21.15.40:0/76066619 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f922400bba0 con 0x7f923c0d39f0 2024-04-26T21:45:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.247+0000 7f92337fe640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f921c06c8f0 0x7f921c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.247+0000 7f92337fe640 1 -- 172.21.15.40:0/76066619 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f922408ca90 con 0x7f923c0d39f0 2024-04-26T21:45:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.247+0000 7f923ad9d640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f921c06c8f0 0x7f921c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.248+0000 7f923ad9d640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f921c06c8f0 0x7f921c06edb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f922c034000 tx=0x7f922c02c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.252+0000 7f92337fe640 1 -- 172.21.15.40:0/76066619 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f923c159a10 con 0x7f923c0d39f0 2024-04-26T21:45:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.476+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f923c075220 con 0x7f923c0d39f0 2024-04-26T21:45:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.477+0000 7f92337fe640 1 -- 172.21.15.40:0/76066619 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 249 v249) ==== 95+0+4355 (secure 0 0 0) 0x7f922405a3b0 con 0x7f923c0d39f0 2024-04-26T21:45:04.478 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:04.478 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":249,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":249,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:03.737132+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 249 2024-04-26T21:45:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f921c06c8f0 msgr2=0x7f921c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f921c06c8f0 0x7f921c06edb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f922c034000 tx=0x7f922c02c000 comp rx=0 tx=0).stop 2024-04-26T21:45:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c0d39f0 msgr2=0x7f923c0ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c0d39f0 0x7f923c0ce5e0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f922400d620 tx=0x7f922400dae0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 shutdown_connections 2024-04-26T21:45:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f921c06c8f0 0x7f921c06edb0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f923c162880 0x7f923c160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923c0d39f0 0x7f923c0ce5e0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 --2- 172.21.15.40:0/76066619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f923c0b5f20 0x7f923c14d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 >> 172.21.15.40:0/76066619 conn(0x7f923c0d4f60 msgr2=0x7f923c0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 shutdown_connections 2024-04-26T21:45:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.479+0000 7f9241247640 1 -- 172.21.15.40:0/76066619 wait complete. 2024-04-26T21:45:04.491 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:04.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:04.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.591+0000 7ff40832b640 1 Processor -- start 2024-04-26T21:45:04.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.591+0000 7ff40832b640 1 -- start start 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff40832b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff400162880 0x7ff4000d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff40832b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff4000b5f20 0x7ff4000d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff40832b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4000b7e50 con 0x7ff4000d39f0 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff40832b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff4000d0580 con 0x7ff4000b5f20 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff40832b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4000d1800 con 0x7ff400162880 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff4068c8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff4000d39f0 0x7ff4000b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52786/0 (socket says 172.21.15.40:52786) 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff4068c8640 1 -- 172.21.15.40:0/3709674805 learned_addr learned my addr 172.21.15.40:0/3709674805 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff4060c7640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff400162880 0x7ff4000d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff4058c6640 1 --2- 172.21.15.40:0/3709674805 >> v2:172.21.15.184:3300/0 conn(0x7ff4000b5f20 0x7ff4000d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914063550 0 0) 0x7ff4000b7e50 con 0x7ff4000d39f0 2024-04-26T21:45:04.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.592+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3ec003520 con 0x7ff4000d39f0 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 685082623 0 0) 0x7ff3ec003520 con 0x7ff4000d39f0 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff400162880 msgr2=0x7ff4000d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff400162880 0x7ff4000d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 >> v2:172.21.15.184:3300/0 conn(0x7ff4000b5f20 msgr2=0x7ff4000d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 --2- 172.21.15.40:0/3709674805 >> v2:172.21.15.184:3300/0 conn(0x7ff4000b5f20 0x7ff4000d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4000d07d0 con 0x7ff4000d39f0 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7ff3f80019f0 con 0x7ff4000d39f0 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 >> v1:172.21.15.40:6789/0 conn(0x7ff4000d39f0 legacy=0x7ff4000b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff400162880 0x7ff3ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4000d0580 con 0x7ff3ec004800 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4000d1800 con 0x7ff3ec0039b0 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec00b9c0 con 0x7ff400162880 2024-04-26T21:45:04.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4058c6640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4060c7640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4068c8640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff400162880 0x7ff3ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4058c6640 1 -- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff400162880 msgr2=0x7ff3ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4058c6640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff400162880 0x7ff3ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4058c6640 1 -- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec0039b0 msgr2=0x7ff3ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4058c6640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.593+0000 7ff4058c6640 1 -- 172.21.15.40:0/3709674805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec003520 con 0x7ff3ec004800 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.594+0000 7ff4058c6640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009ac0 tx=0x7ff3ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab81961f634a40b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.594+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0036070 con 0x7ff3ec004800 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.594+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f0031400 con 0x7ff3ec004800 2024-04-26T21:45:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.594+0000 7ff4050c5640 1 -- 172.21.15.40:0/3709674805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0009690 con 0x7ff3ec004800 2024-04-26T21:45:04.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.595+0000 7ff40832b640 1 -- 172.21.15.40:0/3709674805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ec004800 msgr2=0x7ff3ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.595+0000 7ff40832b640 1 --2- 172.21.15.40:0/3709674805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009ac0 tx=0x7ff3ec005830 comp rx=0 tx=0).stop 2024-04-26T21:45:04.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.595+0000 7ff4068c8640 1 -- 172.21.15.40:0/3709674805 reap_dead start 2024-04-26T21:45:04.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.595+0000 7ff40832b640 1 -- 172.21.15.40:0/3709674805 shutdown_connections 2024-04-26T21:45:04.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.595+0000 7ff40832b640 1 -- 172.21.15.40:0/3709674805 >> 172.21.15.40:0/3709674805 conn(0x7ff4000d4f60 msgr2=0x7ff4000d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.595+0000 7ff40832b640 1 -- 172.21.15.40:0/3709674805 shutdown_connections 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.595+0000 7ff40832b640 1 -- 172.21.15.40:0/3709674805 wait complete. 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.596+0000 7ff40832b640 1 Processor -- start 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.596+0000 7ff40832b640 1 -- start start 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff40832b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec004800 0x7ff4000a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff40832b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4000b2a70 0x7ff4001f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff40832b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4001f8730 0x7ff40015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff40832b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff400162c60 con 0x7ff4001f8730 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff40832b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4000d0d90 con 0x7ff3ec004800 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff40832b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff400162de0 con 0x7ff4000b2a70 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4001f8730 0x7ff40015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4001f8730 0x7ff40015f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50774/0 (socket says 172.21.15.40:50774) 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 -- 172.21.15.40:0/8603027 learned_addr learned my addr 172.21.15.40:0/8603027 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4058c6640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4000b2a70 0x7ff4001f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4060c7640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec004800 0x7ff4000a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 -- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4000b2a70 msgr2=0x7ff4001f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4000b2a70 0x7ff4001f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 -- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec004800 msgr2=0x7ff4000a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec004800 0x7ff4000a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 -- 172.21.15.40:0/8603027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4001f8c40 con 0x7ff4001f8730 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4060c7640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec004800 0x7ff4000a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff4068c8640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4001f8730 0x7ff40015f5e0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7ff4000b7b30 tx=0x7ff3f8007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff3f6ffd640 1 -- 172.21.15.40:0/8603027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800ad00 con 0x7ff4001f8730 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff3f6ffd640 1 -- 172.21.15.40:0/8603027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f8018070 con 0x7ff4001f8730 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4001f8f20 con 0x7ff4001f8730 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.597+0000 7ff3f6ffd640 1 -- 172.21.15.40:0/8603027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8013630 con 0x7ff4001f8730 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.598+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4000a2be0 con 0x7ff4001f8730 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.598+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4000694f0 con 0x7ff4001f8730 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.599+0000 7ff3f6ffd640 1 -- 172.21.15.40:0/8603027 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3f80137d0 con 0x7ff4001f8730 2024-04-26T21:45:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.599+0000 7ff3f6ffd640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3e406c8f0 0x7ff3e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:04.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.599+0000 7ff3f6ffd640 1 -- 172.21.15.40:0/8603027 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7ff3f808ccf0 con 0x7ff4001f8730 2024-04-26T21:45:04.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.599+0000 7ff4060c7640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3e406c8f0 0x7ff3e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:04.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.600+0000 7ff4060c7640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3e406c8f0 0x7ff3e406edb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009a60 tx=0x7ff3fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:04.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.603+0000 7ff3f6ffd640 1 -- 172.21.15.40:0/8603027 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3f805a640 con 0x7ff4001f8730 2024-04-26T21:45:04.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.829+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff40006e8d0 con 0x7ff4001f8730 2024-04-26T21:45:04.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.830+0000 7ff3f6ffd640 1 -- 172.21.15.40:0/8603027 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 250 v250) ==== 95+0+5182 (secure 0 0 0) 0x7ff3f8059f40 con 0x7ff4001f8730 2024-04-26T21:45:04.831 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:04.831 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":250,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10308,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3538213517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3538213517},{"type":"v1","addr":"172.21.15.40:6833","nonce":3538213517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":250}],"filesystems":[{"mdsmap":{"epoch":249,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:03.737132+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:04.831 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 250 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3e406c8f0 msgr2=0x7ff3e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3e406c8f0 0x7ff3e406edb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009a60 tx=0x7ff3fc008040 comp rx=0 tx=0).stop 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4001f8730 msgr2=0x7ff40015f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4001f8730 0x7ff40015f5e0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7ff4000b7b30 tx=0x7ff3f8007bd0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 shutdown_connections 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff3e406c8f0 0x7ff3e406edb0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4001f8730 0x7ff40015f5e0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4000b2a70 0x7ff4001f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 --2- 172.21.15.40:0/8603027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3ec004800 0x7ff4000a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 >> 172.21.15.40:0/8603027 conn(0x7ff4000d4f60 msgr2=0x7ff4000ba010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:04.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 shutdown_connections 2024-04-26T21:45:04.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:04.832+0000 7ff40832b640 1 -- 172.21.15.40:0/8603027 wait complete. 2024-04-26T21:45:04.843 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000001' led to DAMAGED state 2024-04-26T21:45:04.843 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 500.00000000' 2024-04-26T21:45:04.844 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:04.844 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:04.859 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:45:04.859 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:04.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:04.860 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:04.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:45:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:45:04.915 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:05.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.056+0000 7fde04378640 1 Processor -- start 2024-04-26T21:45:05.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.056+0000 7fde04378640 1 -- start start 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde04378640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddfc162880 0x7fddfc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde04378640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fddfc0b5f20 0x7fddfc0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde04378640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddfc0b7e50 con 0x7fddfc0d39f0 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde04378640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fddfc0d3e40 con 0x7fddfc162880 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde04378640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fddfc0d0580 con 0x7fddfc0b5f20 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde02114640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddfc162880 0x7fddfc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde02915640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fddfc0d39f0 0x7fddfc0b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52798/0 (socket says 172.21.15.40:52798) 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde02915640 1 -- 172.21.15.40:0/2083898661 learned_addr learned my addr 172.21.15.40:0/2083898661 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde01913640 1 --2- 172.21.15.40:0/2083898661 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0b5f20 0x7fddfc0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde02114640 1 -- 172.21.15.40:0/2083898661 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0b5f20 msgr2=0x7fddfc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde02114640 1 --2- 172.21.15.40:0/2083898661 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0b5f20 0x7fddfc0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde02114640 1 -- 172.21.15.40:0/2083898661 >> v1:172.21.15.40:6789/0 conn(0x7fddfc0d39f0 legacy=0x7fddfc0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde02114640 1 -- 172.21.15.40:0/2083898661 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc0d1800 con 0x7fddfc162880 2024-04-26T21:45:05.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde01112640 1 -- 172.21.15.40:0/2083898661 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1016807437 0 0) 0x7fddfc0b7e50 con 0x7fddfc0d39f0 2024-04-26T21:45:05.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.057+0000 7fde01913640 1 --2- 172.21.15.40:0/2083898661 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0b5f20 0x7fddfc0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:05.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.058+0000 7fde02114640 1 --2- 172.21.15.40:0/2083898661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc0d2cb0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fddfc08d3d0 tx=0x7fddf802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b777942f396a3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:05.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.058+0000 7fde01112640 1 -- 172.21.15.40:0/2083898661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8037070 con 0x7fddfc162880 2024-04-26T21:45:05.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.058+0000 7fde01112640 1 -- 172.21.15.40:0/2083898661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf8032440 con 0x7fddfc162880 2024-04-26T21:45:05.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.058+0000 7fde01112640 1 -- 172.21.15.40:0/2083898661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8032810 con 0x7fddfc162880 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.059+0000 7fde04378640 1 -- 172.21.15.40:0/2083898661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 msgr2=0x7fddfc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.059+0000 7fde04378640 1 --2- 172.21.15.40:0/2083898661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc0d2cb0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fddfc08d3d0 tx=0x7fddf802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.059+0000 7fde04378640 1 -- 172.21.15.40:0/2083898661 shutdown_connections 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.059+0000 7fde04378640 1 --2- 172.21.15.40:0/2083898661 >> v2:172.21.15.184:3300/0 conn(0x7fddfc0b5f20 0x7fddfc0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.059+0000 7fde04378640 1 --2- 172.21.15.40:0/2083898661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc0d2cb0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.059+0000 7fde04378640 1 -- 172.21.15.40:0/2083898661 >> 172.21.15.40:0/2083898661 conn(0x7fddfc0d4f60 msgr2=0x7fddfc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.059+0000 7fde04378640 1 -- 172.21.15.40:0/2083898661 shutdown_connections 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.059+0000 7fde04378640 1 -- 172.21.15.40:0/2083898661 wait complete. 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.060+0000 7fde04378640 1 Processor -- start 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.060+0000 7fde04378640 1 -- start start 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde04378640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0b5f20 0x7fddfc1f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde04378640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d39f0 0x7fddfc15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde04378640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde04378640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc0d0d90 con 0x7fddfc162880 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde04378640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc0d0ab0 con 0x7fddfc0d39f0 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde04378640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc162c60 con 0x7fddfc0b5f20 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50780/0 (socket says 172.21.15.40:50780) 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 -- 172.21.15.40:0/1836194690 learned_addr learned my addr 172.21.15.40:0/1836194690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02114640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0b5f20 0x7fddfc1f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde01913640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d39f0 0x7fddfc15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 -- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0b5f20 msgr2=0x7fddfc1f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0b5f20 0x7fddfc1f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 -- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d39f0 msgr2=0x7fddfc15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d39f0 0x7fddfc15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 -- 172.21.15.40:0/1836194690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc1f6310 con 0x7fddfc162880 2024-04-26T21:45:05.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02114640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0b5f20 0x7fddfc1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:05.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.061+0000 7fde02915640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc161480 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fddf400c700 tx=0x7fddf400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:05.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.062+0000 7fddf2ffd640 1 -- 172.21.15.40:0/1836194690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4018070 con 0x7fddfc162880 2024-04-26T21:45:05.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.062+0000 7fddf2ffd640 1 -- 172.21.15.40:0/1836194690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf400ce10 con 0x7fddfc162880 2024-04-26T21:45:05.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.062+0000 7fddf2ffd640 1 -- 172.21.15.40:0/1836194690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf400a800 con 0x7fddfc162880 2024-04-26T21:45:05.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.062+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc15cb40 con 0x7fddfc162880 2024-04-26T21:45:05.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.062+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc0b4090 con 0x7fddfc162880 2024-04-26T21:45:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.063+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc075d70 con 0x7fddfc162880 2024-04-26T21:45:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.063+0000 7fddf2ffd640 1 -- 172.21.15.40:0/1836194690 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddf401b070 con 0x7fddfc162880 2024-04-26T21:45:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.064+0000 7fddf2ffd640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdde006c920 0x7fdde006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.064+0000 7fddf2ffd640 1 -- 172.21.15.40:0/1836194690 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7fddf408ba90 con 0x7fddfc162880 2024-04-26T21:45:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.064+0000 7fde02114640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdde006c920 0x7fdde006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.064+0000 7fde02114640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdde006c920 0x7fdde006ede0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.068+0000 7fddf2ffd640 1 -- 172.21.15.40:0/1836194690 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddfc075d70 con 0x7fddfc162880 2024-04-26T21:45:05.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.297+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fddfc0694f0 con 0x7fddfc162880 2024-04-26T21:45:05.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.747+0000 7fddf2ffd640 1 -- 172.21.15.40:0/1836194690 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v251) ==== 161+0+0 (secure 0 0 0) 0x7fddf4059440 con 0x7fddfc162880 2024-04-26T21:45:05.748 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.749+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdde006c920 msgr2=0x7fdde006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.749+0000 7fde04378640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdde006c920 0x7fdde006ede0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.749+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 msgr2=0x7fddfc161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.749+0000 7fde04378640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc161480 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fddf400c700 tx=0x7fddf400cbc0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.750+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 shutdown_connections 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.750+0000 7fde04378640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdde006c920 0x7fdde006ede0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.750+0000 7fde04378640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddfc162880 0x7fddfc161480 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.750+0000 7fde04378640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddfc0d39f0 0x7fddfc15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.750+0000 7fde04378640 1 --2- 172.21.15.40:0/1836194690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddfc0b5f20 0x7fddfc1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.750+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 >> 172.21.15.40:0/1836194690 conn(0x7fddfc0d4f60 msgr2=0x7fddfc0ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.750+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 shutdown_connections 2024-04-26T21:45:05.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.750+0000 7fde04378640 1 -- 172.21.15.40:0/1836194690 wait complete. 2024-04-26T21:45:05.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.866+0000 7fb24174e640 1 Processor -- start 2024-04-26T21:45:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.866+0000 7fb24174e640 1 -- start start 2024-04-26T21:45:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.866+0000 7fb24174e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb23c15f1d0 0x7fb23c06f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.866+0000 7fb24174e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb23c073500 con 0x7fb23c0afd60 2024-04-26T21:45:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.866+0000 7fb24174e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb23c0b4480 con 0x7fb23c06c240 2024-04-26T21:45:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.866+0000 7fb24174e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb23c0ae8c0 con 0x7fb23c15f1d0 2024-04-26T21:45:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb23b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb23c0afd60 0x7fb23c072df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52814/0 (socket says 172.21.15.40:52814) 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb23b7fe640 1 -- 172.21.15.40:0/3258390758 learned_addr learned my addr 172.21.15.40:0/3258390758 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb23affd640 1 --2- 172.21.15.40:0/3258390758 >> v2:172.21.15.184:3300/0 conn(0x7fb23c15f1d0 0x7fb23c06f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4213866832 0 0) 0x7fb23c073500 con 0x7fb23c0afd60 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb228003520 con 0x7fb23c0afd60 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1109371240 0 0) 0x7fb228003520 con 0x7fb23c0afd60 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 >> v2:172.21.15.184:3300/0 conn(0x7fb23c15f1d0 msgr2=0x7fb23c06f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 --2- 172.21.15.40:0/3258390758 >> v2:172.21.15.184:3300/0 conn(0x7fb23c15f1d0 0x7fb23c06f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 >> v1:172.21.15.184:6789/0 conn(0x7fb23c06c240 legacy=0x7fb23c0726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c0b4720 con 0x7fb23c0afd60 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb22c0019f0 con 0x7fb23c0afd60 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 >> v1:172.21.15.40:6789/0 conn(0x7fb23c0afd60 legacy=0x7fb23c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2280039b0 0x7fb228003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb228004800 0x7fb228004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280051c0 0x7fb22800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c0ae8c0 con 0x7fb2280039b0 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb22800c060 con 0x7fb228004800 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb22800c260 con 0x7fb2280051c0 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb23affd640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2280039b0 0x7fb228003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb23b7fe640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280051c0 0x7fb22800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.867+0000 7fb23a7fc640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb228004800 0x7fb228004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb23affd640 1 -- 172.21.15.40:0/3258390758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280051c0 msgr2=0x7fb22800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb23affd640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280051c0 0x7fb22800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb23affd640 1 -- 172.21.15.40:0/3258390758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb228004800 msgr2=0x7fb228004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb23affd640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb228004800 0x7fb228004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb23affd640 1 -- 172.21.15.40:0/3258390758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb228003520 con 0x7fb2280039b0 2024-04-26T21:45:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb23b7fe640 1 -- 172.21.15.40:0/3258390758 reap_dead start 2024-04-26T21:45:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb23affd640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2280039b0 0x7fb228003dd0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fb234009ac0 tx=0x7fb228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5274d967a1082998 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234036070 con 0x7fb2280039b0 2024-04-26T21:45:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb234031400 con 0x7fb2280039b0 2024-04-26T21:45:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.868+0000 7fb239ffb640 1 -- 172.21.15.40:0/3258390758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234009690 con 0x7fb2280039b0 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.869+0000 7fb24174e640 1 -- 172.21.15.40:0/3258390758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2280039b0 msgr2=0x7fb228003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.869+0000 7fb24174e640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2280039b0 0x7fb228003dd0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fb234009ac0 tx=0x7fb228005830 comp rx=0 tx=0).stop 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.869+0000 7fb24174e640 1 -- 172.21.15.40:0/3258390758 shutdown_connections 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.869+0000 7fb24174e640 1 --2- 172.21.15.40:0/3258390758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2280039b0 0x7fb228003dd0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.869+0000 7fb24174e640 1 -- 172.21.15.40:0/3258390758 >> 172.21.15.40:0/3258390758 conn(0x7fb23c0b3180 msgr2=0x7fb23c0b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.869+0000 7fb24174e640 1 -- 172.21.15.40:0/3258390758 shutdown_connections 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.869+0000 7fb24174e640 1 -- 172.21.15.40:0/3258390758 wait complete. 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.870+0000 7fb24174e640 1 Processor -- start 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.870+0000 7fb24174e640 1 -- start start 2024-04-26T21:45:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb24174e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280039b0 0x7fb23c156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb24174e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb23c158150 0x7fb23c139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb24174e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb23c15bc30 0x7fb23c15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb24174e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c0b4fc0 con 0x7fb23c15bc30 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb24174e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c0b4ce0 con 0x7fb23c158150 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb24174e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb23c075ea0 con 0x7fb2280039b0 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb23b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb23c15bc30 0x7fb23c15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb23b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb23c15bc30 0x7fb23c15c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50796/0 (socket says 172.21.15.40:50796) 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb23b7fe640 1 -- 172.21.15.40:0/3202039788 learned_addr learned my addr 172.21.15.40:0/3202039788 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb23affd640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280039b0 0x7fb23c156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.871+0000 7fb23a7fc640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb23c158150 0x7fb23c139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb23b7fe640 1 -- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280039b0 msgr2=0x7fb23c156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb23b7fe640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280039b0 0x7fb23c156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb23b7fe640 1 -- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb23c158150 msgr2=0x7fb23c139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb23b7fe640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb23c158150 0x7fb23c139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb23b7fe640 1 -- 172.21.15.40:0/3202039788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c156f10 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb23a7fc640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb23c158150 0x7fb23c139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb23b7fe640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb23c15bc30 0x7fb23c15c0b0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fb23c06e4f0 tx=0x7fb22c007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb2337fe640 1 -- 172.21.15.40:0/3202039788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00eaa0 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb2337fe640 1 -- 172.21.15.40:0/3202039788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c00ec40 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c1571f0 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb2337fe640 1 -- 172.21.15.40:0/3202039788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c014430 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.872+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1585d0 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.873+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c068fc0 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.874+0000 7fb2337fe640 1 -- 172.21.15.40:0/3202039788 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb22c0145f0 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.874+0000 7fb2337fe640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb21c06c920 0x7fb21c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.874+0000 7fb2337fe640 1 -- 172.21.15.40:0/3202039788 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7fb22c08c260 con 0x7fb23c15bc30 2024-04-26T21:45:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.874+0000 7fb23affd640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb21c06c920 0x7fb21c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.875+0000 7fb23affd640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb21c06c920 0x7fb21c06ede0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb234033000 tx=0x7fb234008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:05.878+0000 7fb2337fe640 1 -- 172.21.15.40:0/3202039788 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb22c059530 con 0x7fb23c15bc30 2024-04-26T21:45:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.137+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb23c159380 con 0x7fb23c15bc30 2024-04-26T21:45:06.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.752+0000 7fb2337fe640 1 -- 172.21.15.40:0/3202039788 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v252) ==== 101+0+0 (secure 0 0 0) 0x7fb22c05d830 con 0x7fb23c15bc30 2024-04-26T21:45:06.753 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.754+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb21c06c920 msgr2=0x7fb21c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.754+0000 7fb24174e640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb21c06c920 0x7fb21c06ede0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb234033000 tx=0x7fb234008030 comp rx=0 tx=0).stop 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.754+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb23c15bc30 msgr2=0x7fb23c15c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.754+0000 7fb24174e640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb23c15bc30 0x7fb23c15c0b0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fb23c06e4f0 tx=0x7fb22c007b70 comp rx=0 tx=0).stop 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.755+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 shutdown_connections 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.755+0000 7fb24174e640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb21c06c920 0x7fb21c06ede0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.755+0000 7fb24174e640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb23c15bc30 0x7fb23c15c0b0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.755+0000 7fb24174e640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb23c158150 0x7fb23c139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.755+0000 7fb24174e640 1 --2- 172.21.15.40:0/3202039788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2280039b0 0x7fb23c156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.755+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 >> 172.21.15.40:0/3202039788 conn(0x7fb23c0b3180 msgr2=0x7fb23c0b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.755+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 shutdown_connections 2024-04-26T21:45:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.755+0000 7fb24174e640 1 -- 172.21.15.40:0/3202039788 wait complete. 2024-04-26T21:45:06.769 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:06.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.813+0000 7f7a3f222f40 1 Processor -- start 2024-04-26T21:45:06.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.813+0000 7f7a3f222f40 1 -- start start 2024-04-26T21:45:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3f222f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49aa80 0x564bdd40d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3f222f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49ae60 0x564bdd40c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3f222f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49df30 0x564bdd40c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3f222f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564bdd4272d0 con 0x564bdd49df30 2024-04-26T21:45:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3f222f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564bdd386e40 con 0x564bdd49ae60 2024-04-26T21:45:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3f222f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564bdd427570 con 0x564bdd49aa80 2024-04-26T21:45:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3d51f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49df30 0x564bdd40c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3d51f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49df30 0x564bdd40c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50820/0 (socket says 172.21.15.40:50820) 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3d51f640 1 -- 172.21.15.40:0/2508583233 learned_addr learned my addr 172.21.15.40:0/2508583233 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.814+0000 7f7a3cd1e640 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49aa80 0x564bdd40d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a37fff640 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49ae60 0x564bdd40c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a3d51f640 1 -- 172.21.15.40:0/2508583233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49aa80 msgr2=0x564bdd40d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a3d51f640 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49aa80 0x564bdd40d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a3d51f640 1 -- 172.21.15.40:0/2508583233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49ae60 msgr2=0x564bdd40c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a3d51f640 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49ae60 0x564bdd40c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a3d51f640 1 -- 172.21.15.40:0/2508583233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bdd4b8ea0 con 0x564bdd49df30 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a3d51f640 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49df30 0x564bdd40c990 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800a4b0 tx=0x7f7a3800a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8214647209865c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a36ffd640 1 -- 172.21.15.40:0/2508583233 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38015070 con 0x564bdd49df30 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a36ffd640 1 -- 172.21.15.40:0/2508583233 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3800e730 con 0x564bdd49df30 2024-04-26T21:45:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a36ffd640 1 -- 172.21.15.40:0/2508583233 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800eb20 con 0x564bdd49df30 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a3f222f40 1 -- 172.21.15.40:0/2508583233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49df30 msgr2=0x564bdd40c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.815+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49df30 0x564bdd40c990 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800a4b0 tx=0x7f7a3800a970 comp rx=0 tx=0).stop 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 -- 172.21.15.40:0/2508583233 shutdown_connections 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49df30 0x564bdd40c990 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49ae60 0x564bdd40c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/2508583233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49aa80 0x564bdd40d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 -- 172.21.15.40:0/2508583233 >> 172.21.15.40:0/2508583233 conn(0x564bdd3ddab0 msgr2=0x564bdd3de720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 -- 172.21.15.40:0/2508583233 shutdown_connections 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 -- 172.21.15.40:0/2508583233 wait complete. 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 Processor -- start 2024-04-26T21:45:06.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.816+0000 7f7a3f222f40 1 -- start start 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a3f222f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49aa80 0x564bdd4b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a3f222f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49ae60 0x564bdd494860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a3f222f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49df30 0x564bdd498700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a3f222f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564bdd4b5160 con 0x564bdd49ae60 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a3f222f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564bdd4b4e80 con 0x564bdd49aa80 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a3f222f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564bdd499750 con 0x564bdd49df30 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49ae60 0x564bdd494860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49ae60 0x564bdd494860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50832/0 (socket says 172.21.15.40:50832) 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 learned_addr learned my addr 172.21.15.40:0/1529791270 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a3cd1e640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49aa80 0x564bdd4b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a3d51f640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49df30 0x564bdd498700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49df30 msgr2=0x564bdd498700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a37fff640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564bdd49df30 0x564bdd498700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49aa80 msgr2=0x564bdd4b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a37fff640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bdd49aa80 0x564bdd4b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.817+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bdd4b47d0 con 0x564bdd49ae60 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.818+0000 7f7a37fff640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49ae60 0x564bdd494860 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f7a28007f20 tx=0x7f7a28006810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.818+0000 7f7a35ffb640 1 -- 172.21.15.40:0/1529791270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28018070 con 0x564bdd49ae60 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.818+0000 7f7a35ffb640 1 -- 172.21.15.40:0/1529791270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a28011700 con 0x564bdd49ae60 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.818+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564bdd4b5a70 con 0x564bdd49ae60 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.818+0000 7f7a35ffb640 1 -- 172.21.15.40:0/1529791270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28011af0 con 0x564bdd49ae60 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.818+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bdd4b6030 con 0x564bdd49ae60 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.819+0000 7f7a35ffb640 1 -- 172.21.15.40:0/1529791270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a2801b070 con 0x564bdd49ae60 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.820+0000 7f7a35ffb640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2006b2d0 0x7f7a2006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.820+0000 7f7a35ffb640 1 -- 172.21.15.40:0/1529791270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f7a2808a680 con 0x564bdd49ae60 2024-04-26T21:45:06.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.820+0000 7f7a3cd1e640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2006b2d0 0x7f7a2006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.824 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.820+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564bdd4b55b0 0x564bdd54d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.820+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd54db80 con 0x564bdd4b55b0 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.820+0000 7f7a3d51f640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564bdd4b55b0 0x564bdd54d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.821+0000 7f7a3cd1e640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2006b2d0 0x7f7a2006d790 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7a30057000 tx=0x7f7a30052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.821+0000 7f7a3d51f640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564bdd4b55b0 0x564bdd54d640 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.822+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v121'784 uv758 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.822+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd498c40 con 0x564bdd4b55b0 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.823+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v121'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.823+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd559780 con 0x564bdd4b55b0 2024-04-26T21:45:06.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.824+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v121'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.826 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:06.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.824+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55ab30 con 0x564bdd4b55b0 2024-04-26T21:45:06.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.826+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v121'787 uv787 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.826+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55aed0 con 0x564bdd4b55b0 2024-04-26T21:45:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.827+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v121'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.827+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd559b20 con 0x564bdd4b55b0 2024-04-26T21:45:06.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.828+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v121'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.828+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd559f30 con 0x564bdd4b55b0 2024-04-26T21:45:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.829+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v121'790 uv764 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.829+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.830+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v121'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.830+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.831+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v121'792 uv792 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.832 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:06.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.831+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.832+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v121'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.832+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.833+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v121'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.833+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.834+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v121'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.835+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.836+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v121'796 uv771 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.836+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.837+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v121'797 uv797 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.837+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.838+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v121'798 uv798 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.839 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:06.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.838+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.839+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v121'799 uv799 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.839+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.840+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v121'800 uv800 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.840+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.841+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v121'801 uv801 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.841+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.843+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v121'802 uv802 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.843+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.844+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v121'803 uv777 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.844+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.845+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v121'804 uv804 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.845+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.846+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v121'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.847 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:06.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.846+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.847+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v121'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.847+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.848+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v121'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.848+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.849+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v121'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.849+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.850+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v121'809 uv783 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.850+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.852+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v121'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.852+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.852+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v121'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.853 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.853+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.854+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v121'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.854+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.855+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v121'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.855+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55a2d0 con 0x564bdd4b55b0 2024-04-26T21:45:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.856+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v121'814 uv814 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd4b55b0 2024-04-26T21:45:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.856+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564bdd55a6d0 0x564bdd55d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.856+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55dcd0 con 0x564bdd55a6d0 2024-04-26T21:45:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.856+0000 7f7a3cd1e640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564bdd55a6d0 0x564bdd55d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.856+0000 7f7a3cd1e640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564bdd55a6d0 0x564bdd55d790 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.858+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v121'571 uv553 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.858+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd55e070 con 0x564bdd55a6d0 2024-04-26T21:45:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.859+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v121'572 uv572 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.859+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55e410 con 0x564bdd55a6d0 2024-04-26T21:45:06.861 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.861+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v121'573 uv573 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.861+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55ec60 con 0x564bdd55a6d0 2024-04-26T21:45:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.862+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v121'574 uv574 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.862+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f000 con 0x564bdd55a6d0 2024-04-26T21:45:06.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.863+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v121'575 uv575 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.863+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f3a0 con 0x564bdd55a6d0 2024-04-26T21:45:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.865+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v121'576 uv558 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.865+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.866+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v121'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.866+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.868 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:06.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.867+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v121'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.867+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.869+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v121'579 uv579 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.869+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.870+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v121'580 uv580 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.870+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.871+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v121'581 uv570 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.872+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.873+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v121'582 uv582 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.873+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.875 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.874+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v121'583 uv583 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.874+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd563e00 con 0x564bdd55a6d0 2024-04-26T21:45:06.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.876+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v121'584 uv584 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.876+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd563e00 con 0x564bdd55a6d0 2024-04-26T21:45:06.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.877+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v121'585 uv585 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.877+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.879+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v121'586 uv569 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.879+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.880+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v121'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.880+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.881+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v121'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.882 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:06.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.881+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd560750 con 0x564bdd55a6d0 2024-04-26T21:45:06.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.882+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v121'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.882+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd560750 con 0x564bdd55a6d0 2024-04-26T21:45:06.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.883+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v121'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.883+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd55f740 con 0x564bdd55a6d0 2024-04-26T21:45:06.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.884+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v121'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd55a6d0 2024-04-26T21:45:06.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.884+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564bdd55fc50 0x564bdd562040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.884+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd562580 con 0x564bdd55fc50 2024-04-26T21:45:06.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.884+0000 7f7a37fff640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564bdd55fc50 0x564bdd562040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.885+0000 7f7a37fff640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564bdd55fc50 0x564bdd562040 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.886+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v121'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.886+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd562a20 con 0x564bdd55fc50 2024-04-26T21:45:06.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.887+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v121'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.887+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd562f00 con 0x564bdd55fc50 2024-04-26T21:45:06.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.888+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v121'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.889 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:06.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.888+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd5647b0 con 0x564bdd55fc50 2024-04-26T21:45:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.889+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v121'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.889+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd564b50 con 0x564bdd55fc50 2024-04-26T21:45:06.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.890+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v121'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.890+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd5637a0 con 0x564bdd55fc50 2024-04-26T21:45:06.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.891+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v121'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.891+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564bdd5640a0 0x564bdd566f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.891+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd567440 con 0x564bdd5640a0 2024-04-26T21:45:06.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.891+0000 7f7a3d51f640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564bdd5640a0 0x564bdd566f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.892+0000 7f7a3d51f640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564bdd5640a0 0x564bdd566f00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.893+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v121'116 uv115 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.893+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd567440 con 0x564bdd5640a0 2024-04-26T21:45:06.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.894+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v121'117 uv117 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.894+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd567460 con 0x564bdd5640a0 2024-04-26T21:45:06.896 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.895+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v121'118 uv118 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.896+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd5678a0 con 0x564bdd5640a0 2024-04-26T21:45:06.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.897+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v121'119 uv119 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.897+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564bdd567db0 0x564bdd56a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.897+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56a730 con 0x564bdd567db0 2024-04-26T21:45:06.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.897+0000 7f7a3cd1e640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564bdd567db0 0x564bdd56a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.897+0000 7f7a3cd1e640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564bdd567db0 0x564bdd56a1a0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.899+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v121'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.899+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56a7f0 con 0x564bdd567db0 2024-04-26T21:45:06.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.900+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v121'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.900+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.901+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v121'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.902 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:06.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.901+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56bbf0 con 0x564bdd567db0 2024-04-26T21:45:06.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.902+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v121'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.902+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56bbf0 con 0x564bdd567db0 2024-04-26T21:45:06.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.903+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v121'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.903+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.904+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v121'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.905+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.906+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v121'282 uv276 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.906+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.907+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v121'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.907+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.909 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:06.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.908+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v121'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.908+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56bbf0 con 0x564bdd5640a0 2024-04-26T21:45:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.909+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v121'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.909+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56bbf0 con 0x564bdd5640a0 2024-04-26T21:45:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.910+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v121'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.910+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.911+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v121'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.911+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.912+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v121'288 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.912+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.913+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v121'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.913+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.914+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v121'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.915 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:06.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.914+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.915+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v121'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.915+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.917+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v121'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.917+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.918+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v121'127 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.918+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.919+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v121'128 uv128 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.919+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.920+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v121'129 uv129 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.921 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:06.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.920+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.921+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v121'130 uv130 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.921+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd5640a0 2024-04-26T21:45:06.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.922+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v121'131 uv131 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7a38018040 con 0x564bdd5640a0 2024-04-26T21:45:06.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.922+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.923+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v121'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.924+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.925+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v121'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.925+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.926+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v121'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.927 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:06.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.926+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.927+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v121'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.927+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.928+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v121'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.928+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.929+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v121'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.929+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.931+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v121'253 uv247 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.931+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.932+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v121'254 uv254 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.932+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56abe0 con 0x564bdd567db0 2024-04-26T21:45:06.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.933+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v121'255 uv255 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.934 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:06.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.933+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56afd0 con 0x564bdd567db0 2024-04-26T21:45:06.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.934+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v121'256 uv256 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.934+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56afd0 con 0x564bdd567db0 2024-04-26T21:45:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.935+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v121'257 uv257 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.935+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56afd0 con 0x564bdd567db0 2024-04-26T21:45:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.936+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v121'258 uv252 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.936+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56afd0 con 0x564bdd567db0 2024-04-26T21:45:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.938+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v121'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.938+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56afd0 con 0x564bdd567db0 2024-04-26T21:45:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.939+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v121'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.940 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:06.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.939+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56afd0 con 0x564bdd567db0 2024-04-26T21:45:06.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.940+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v121'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.940+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56b390 con 0x564bdd567db0 2024-04-26T21:45:06.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.941+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v121'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.941+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564bdd56b5f0 0x564bdd56e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:06.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.941+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56bbc0 con 0x564bdd56b5f0 2024-04-26T21:45:06.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.941+0000 7f7a37fff640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564bdd56b5f0 0x564bdd56e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:06.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.942+0000 7f7a37fff640 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564bdd56b5f0 0x564bdd56e030 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.943+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v121'111 uv110 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd56b5f0 2024-04-26T21:45:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.943+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56bbc0 con 0x564bdd56b5f0 2024-04-26T21:45:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.944+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v121'112 uv112 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd56b5f0 2024-04-26T21:45:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.944+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56bc40 con 0x564bdd56b5f0 2024-04-26T21:45:06.946 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:06.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.945+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v121'113 uv113 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd56b5f0 2024-04-26T21:45:06.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.945+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd56b5f0 2024-04-26T21:45:06.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.946+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v121'114 uv114 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd56b5f0 2024-04-26T21:45:06.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.946+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd567db0 2024-04-26T21:45:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.947+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v121'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.947+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd567db0 2024-04-26T21:45:06.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.948+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v121'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.949+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd567db0 2024-04-26T21:45:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.950+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v121'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.951 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.950+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56f580 con 0x564bdd567db0 2024-04-26T21:45:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.951+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v121'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.951+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56f580 con 0x564bdd567db0 2024-04-26T21:45:06.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.952+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v121'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.952+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd567db0 2024-04-26T21:45:06.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.953+0000 7f7a3cd1e640 1 -- 172.21.15.40:0/1529791270 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v121'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a30050050 con 0x564bdd567db0 2024-04-26T21:45:06.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.953+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.954+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v121'130 uv129 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.954+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.956+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v121'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.956+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.958 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:06.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.957+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v121'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.957+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.958+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v121'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.958+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.959+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v121'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.959+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.960+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v121'314 uv306 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.960+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.961+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v121'315 uv315 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.961+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.963 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:06.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.962+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v121'316 uv316 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.962+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.963+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v121'317 uv317 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.963+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.964+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v121'318 uv318 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.964+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.965+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v121'319 uv313 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.965+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.966+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v121'320 uv320 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.966+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.967+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v121'321 uv321 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.968 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:06.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.967+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56f580 con 0x564bdd55fc50 2024-04-26T21:45:06.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.968+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v121'322 uv322 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.968+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56f580 con 0x564bdd55fc50 2024-04-26T21:45:06.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.969+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v121'323 uv323 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.969+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56e570 con 0x564bdd55fc50 2024-04-26T21:45:06.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.970+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v121'324 uv324 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.970+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564bdd56ea20 con 0x564bdd55fc50 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a37fff640 1 -- 172.21.15.40:0/1529791270 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v121'325 uv325 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7a2801a030 con 0x564bdd55fc50 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564bdd55a6d0 msgr2=0x564bdd55d790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x564bdd55a6d0 0x564bdd55d790 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564bdd4b55b0 msgr2=0x564bdd54d640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x564bdd4b55b0 0x564bdd54d640 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564bdd5640a0 msgr2=0x564bdd566f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x564bdd5640a0 0x564bdd566f00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564bdd56b5f0 msgr2=0x564bdd56e030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x564bdd56b5f0 0x564bdd56e030 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564bdd55fc50 msgr2=0x564bdd562040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x564bdd55fc50 0x564bdd562040 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564bdd567db0 msgr2=0x564bdd56a1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x564bdd567db0 0x564bdd56a1a0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 reap_dead start 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3d51f640 1 -- 172.21.15.40:0/1529791270 reap_dead start 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2006b2d0 msgr2=0x7f7a2006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2006b2d0 0x7f7a2006d790 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7a30057000 tx=0x7f7a30052000 comp rx=0 tx=0).stop 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49ae60 msgr2=0x564bdd494860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49ae60 0x564bdd494860 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f7a28007f20 tx=0x7f7a28006810 comp rx=0 tx=0).stop 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 shutdown_connections 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2006b2d0 0x7f7a2006d790 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 --2- 172.21.15.40:0/1529791270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564bdd49ae60 0x564bdd494860 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 >> 172.21.15.40:0/1529791270 conn(0x564bdd3ddab0 msgr2=0x564bdd40d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.971+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 shutdown_connections 2024-04-26T21:45:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:06.972+0000 7f7a3f222f40 1 -- 172.21.15.40:0/1529791270 wait complete. 2024-04-26T21:45:06.975 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 500.00000000 0 2024-04-26T21:45:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.058+0000 7f25acb1df40 1 Processor -- start 2024-04-26T21:45:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.058+0000 7f25acb1df40 1 -- start start 2024-04-26T21:45:07.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25acb1df40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 0x555b2304b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25acb1df40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5a80 0x555b230e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25acb1df40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5e60 0x555b230e3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25acb1df40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555b230522d0 con 0x555b230c5e60 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25acb1df40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555b22fb1e40 con 0x555b230538c0 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25acb1df40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555b23052570 con 0x555b230c5a80 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5e60 0x555b230e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5e60 0x555b230e3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50860/0 (socket says 172.21.15.40:50860) 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 -- 172.21.15.40:0/2286589158 learned_addr learned my addr 172.21.15.40:0/2286589158 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25a93a0640 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5a80 0x555b230e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25a9ba1640 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 0x555b2304b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 -- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5a80 msgr2=0x555b230e32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5a80 0x555b230e32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 -- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 msgr2=0x555b2304b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 0x555b2304b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 -- 172.21.15.40:0/2286589158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b230dd450 con 0x555b230c5e60 2024-04-26T21:45:07.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25a9ba1640 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 0x555b2304b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25aa3a2640 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5e60 0x555b230e3820 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f259c00a4a0 tx=0x7f259c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e8ed9824ea7542 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25a3fff640 1 -- 172.21.15.40:0/2286589158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c056040 con 0x555b230c5e60 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25a3fff640 1 -- 172.21.15.40:0/2286589158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c0528a0 con 0x555b230c5e60 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.059+0000 7f25a3fff640 1 -- 172.21.15.40:0/2286589158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c052c90 con 0x555b230c5e60 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 -- 172.21.15.40:0/2286589158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5e60 msgr2=0x555b230e3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5e60 0x555b230e3820 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f259c00a4a0 tx=0x7f259c051000 comp rx=0 tx=0).stop 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 -- 172.21.15.40:0/2286589158 shutdown_connections 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5e60 0x555b230e3820 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5a80 0x555b230e32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 --2- 172.21.15.40:0/2286589158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 0x555b2304b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 -- 172.21.15.40:0/2286589158 >> 172.21.15.40:0/2286589158 conn(0x555b23008ab0 msgr2=0x555b23009720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 -- 172.21.15.40:0/2286589158 shutdown_connections 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.060+0000 7f25acb1df40 1 -- 172.21.15.40:0/2286589158 wait complete. 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25acb1df40 1 Processor -- start 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25acb1df40 1 -- start start 2024-04-26T21:45:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25acb1df40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 0x555b230d9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25acb1df40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5a80 0x555b23038cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25acb1df40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5e60 0x555b23177d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25acb1df40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555b230e2850 con 0x555b230c5a80 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25acb1df40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555b230e25b0 con 0x555b230538c0 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25acb1df40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555b2304ab40 con 0x555b230c5e60 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25a93a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5a80 0x555b23038cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25a93a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5a80 0x555b23038cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50862/0 (socket says 172.21.15.40:50862) 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25a93a0640 1 -- 172.21.15.40:0/160862545 learned_addr learned my addr 172.21.15.40:0/160862545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25a9ba1640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 0x555b230d9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.061+0000 7f25aa3a2640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5e60 0x555b23177d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a93a0640 1 -- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5e60 msgr2=0x555b23177d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a93a0640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b230c5e60 0x555b23177d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a93a0640 1 -- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 msgr2=0x555b230d9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a93a0640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b230538c0 0x555b230d9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a93a0640 1 -- 172.21.15.40:0/160862545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b230d87d0 con 0x555b230c5a80 2024-04-26T21:45:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a93a0640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5a80 0x555b23038cc0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f259400c560 tx=0x7f259400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a2ffd640 1 -- 172.21.15.40:0/160862545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594008070 con 0x555b230c5a80 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a2ffd640 1 -- 172.21.15.40:0/160862545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2594016070 con 0x555b230c5a80 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25a2ffd640 1 -- 172.21.15.40:0/160862545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594010920 con 0x555b230c5a80 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b230e1bf0 con 0x555b230c5a80 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.062+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b230381e0 con 0x555b230c5a80 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.064+0000 7f25a2ffd640 1 -- 172.21.15.40:0/160862545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2594010ac0 con 0x555b230c5a80 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.064+0000 7f25a2ffd640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f258c06b2d0 0x7f258c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.064+0000 7f25a2ffd640 1 -- 172.21.15.40:0/160862545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f259408b850 con 0x555b230c5a80 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.064+0000 7f25a9ba1640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f258c06b2d0 0x7f258c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.064+0000 7f25acb1df40 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b23039200 0x555b230d7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.064+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x555b230d7940 con 0x555b23039200 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.064+0000 7f25aa3a2640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b23039200 0x555b230d7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.065+0000 7f25a9ba1640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f258c06b2d0 0x7f258c06d790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009860 tx=0x7f25a400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.065+0000 7f25aa3a2640 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b23039200 0x555b230d7400 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25aa3a2640 1 -- 172.21.15.40:0/160862545 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 500.00000000 [truncate 0] v121'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f259c052a50 con 0x555b23039200 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b23039200 msgr2=0x555b230d7400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b23039200 0x555b230d7400 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f258c06b2d0 msgr2=0x7f258c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f258c06b2d0 0x7f258c06d790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009860 tx=0x7f25a400c040 comp rx=0 tx=0).stop 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5a80 msgr2=0x555b23038cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 --2- 172.21.15.40:0/160862545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b230c5a80 0x555b23038cc0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f259400c560 tx=0x7f259400c960 comp rx=0 tx=0).stop 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25aa3a2640 1 -- 172.21.15.40:0/160862545 reap_dead start 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 shutdown_connections 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 >> 172.21.15.40:0/160862545 conn(0x555b23008ab0 msgr2=0x555b230dd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 shutdown_connections 2024-04-26T21:45:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.066+0000 7f25acb1df40 1 -- 172.21.15.40:0/160862545 wait complete. 2024-04-26T21:45:07.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:07.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.214+0000 7f36c4eb8640 1 Processor -- start 2024-04-26T21:45:07.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.214+0000 7f36c4eb8640 1 -- start start 2024-04-26T21:45:07.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36c4eb8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36c00cfde0 0x7f36c00d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36c4eb8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36c0070350 0x7f36c0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36c4eb8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c00d2390 con 0x7f36c0071090 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36c4eb8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f36c00afab0 con 0x7f36c0070350 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36c4eb8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f36c006dd20 con 0x7f36c00cfde0 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36bdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36c0070350 0x7f36c0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36bdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36c0070350 0x7f36c0070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50866/0 (socket says 172.21.15.40:50866) 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36bdd9b640 1 -- 172.21.15.40:0/1151625949 learned_addr learned my addr 172.21.15.40:0/1151625949 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36be59c640 1 --2- 172.21.15.40:0/1151625949 >> v2:172.21.15.184:3300/0 conn(0x7f36c00cfde0 0x7f36c00d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36bdd9b640 1 -- 172.21.15.40:0/1151625949 >> v2:172.21.15.184:3300/0 conn(0x7f36c00cfde0 msgr2=0x7f36c00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.215+0000 7f36bdd9b640 1 --2- 172.21.15.40:0/1151625949 >> v2:172.21.15.184:3300/0 conn(0x7f36c00cfde0 0x7f36c00d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.216+0000 7f36bdd9b640 1 -- 172.21.15.40:0/1151625949 >> v1:172.21.15.184:6789/0 conn(0x7f36c0071090 legacy=0x7f36c015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.216+0000 7f36bdd9b640 1 -- 172.21.15.40:0/1151625949 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c006d3f0 con 0x7f36c0070350 2024-04-26T21:45:07.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.216+0000 7f36bdd9b640 1 --2- 172.21.15.40:0/1151625949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c0070350 0x7f36c0070b50 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f36c00762b0 tx=0x7f36a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3723796817620beb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.216+0000 7f36bd59a640 1 -- 172.21.15.40:0/1151625949 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8037070 con 0x7f36c0070350 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.216+0000 7f36bd59a640 1 -- 172.21.15.40:0/1151625949 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a8032440 con 0x7f36c0070350 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.216+0000 7f36bd59a640 1 -- 172.21.15.40:0/1151625949 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8032810 con 0x7f36c0070350 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.217+0000 7f36c4eb8640 1 -- 172.21.15.40:0/1151625949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c0070350 msgr2=0x7f36c0070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.217+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/1151625949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c0070350 0x7f36c0070b50 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f36c00762b0 tx=0x7f36a802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.217+0000 7f36c4eb8640 1 -- 172.21.15.40:0/1151625949 shutdown_connections 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.217+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/1151625949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c0070350 0x7f36c0070b50 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.217+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/1151625949 >> v2:172.21.15.184:3300/0 conn(0x7f36c00cfde0 0x7f36c00d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.217+0000 7f36c4eb8640 1 -- 172.21.15.40:0/1151625949 >> 172.21.15.40:0/1151625949 conn(0x7f36c00d2d80 msgr2=0x7f36c00d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.217+0000 7f36c4eb8640 1 -- 172.21.15.40:0/1151625949 shutdown_connections 2024-04-26T21:45:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.217+0000 7f36c4eb8640 1 -- 172.21.15.40:0/1151625949 wait complete. 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.218+0000 7f36c4eb8640 1 Processor -- start 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.218+0000 7f36c4eb8640 1 -- start start 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36c4eb8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c0070350 0x7f36c015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36c4eb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c0071090 0x7f36c01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36c4eb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c00cfde0 0x7f36c015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36c4eb8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36c006d9b0 con 0x7f36c00cfde0 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36c4eb8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36c00714c0 con 0x7f36c0070350 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36c4eb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36c00d20a0 con 0x7f36c0071090 2024-04-26T21:45:07.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c00cfde0 0x7f36c015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c00cfde0 0x7f36c015b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50882/0 (socket says 172.21.15.40:50882) 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36bed9d640 1 -- 172.21.15.40:0/2094848693 learned_addr learned my addr 172.21.15.40:0/2094848693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36bdd9b640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c0071090 0x7f36c01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.219+0000 7f36be59c640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c0070350 0x7f36c015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36bed9d640 1 -- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c0071090 msgr2=0x7f36c01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36bed9d640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c0071090 0x7f36c01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36bed9d640 1 -- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c0070350 msgr2=0x7f36c015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36bed9d640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c0070350 0x7f36c015fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36bed9d640 1 -- 172.21.15.40:0/2094848693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c01f5cb0 con 0x7f36c00cfde0 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36bed9d640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c00cfde0 0x7f36c015b110 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f36b800c3f0 tx=0x7f36b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36b6ffd640 1 -- 172.21.15.40:0/2094848693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80052d0 con 0x7f36c00cfde0 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36b6ffd640 1 -- 172.21.15.40:0/2094848693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b8005490 con 0x7f36c00cfde0 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36b6ffd640 1 -- 172.21.15.40:0/2094848693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8007260 con 0x7f36c00cfde0 2024-04-26T21:45:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c0159830 con 0x7f36c00cfde0 2024-04-26T21:45:07.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.220+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c01f5410 con 0x7f36c00cfde0 2024-04-26T21:45:07.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.221+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c00694f0 con 0x7f36c00cfde0 2024-04-26T21:45:07.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.222+0000 7f36b6ffd640 1 -- 172.21.15.40:0/2094848693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36b8007400 con 0x7f36c00cfde0 2024-04-26T21:45:07.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.222+0000 7f36b6ffd640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36a006c8f0 0x7f36a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:07.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.222+0000 7f36b6ffd640 1 -- 172.21.15.40:0/2094848693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f36b808b950 con 0x7f36c00cfde0 2024-04-26T21:45:07.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.222+0000 7f36be59c640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36a006c8f0 0x7f36a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:07.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.223+0000 7f36be59c640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36a006c8f0 0x7f36a006edb0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f36c015cf60 tx=0x7f36b00098d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:07.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.226+0000 7f36b6ffd640 1 -- 172.21.15.40:0/2094848693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36c0159830 con 0x7f36c00cfde0 2024-04-26T21:45:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.450+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f36c00b0880 con 0x7f36c00cfde0 2024-04-26T21:45:07.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.753+0000 7f36b6ffd640 1 -- 172.21.15.40:0/2094848693 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v253) ==== 162+0+0 (secure 0 0 0) 0x7f36b8058cb0 con 0x7f36c00cfde0 2024-04-26T21:45:07.754 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:07.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.755+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36a006c8f0 msgr2=0x7f36a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36a006c8f0 0x7f36a006edb0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f36c015cf60 tx=0x7f36b00098d0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c00cfde0 msgr2=0x7f36c015b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c00cfde0 0x7f36c015b110 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f36b800c3f0 tx=0x7f36b800c8b0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 shutdown_connections 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36a006c8f0 0x7f36a006edb0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c00cfde0 0x7f36c015b110 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c0071090 0x7f36c01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 --2- 172.21.15.40:0/2094848693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c0070350 0x7f36c015fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 >> 172.21.15.40:0/2094848693 conn(0x7f36c00d2d80 msgr2=0x7f36c00b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 shutdown_connections 2024-04-26T21:45:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:07.756+0000 7f36c4eb8640 1 -- 172.21.15.40:0/2094848693 wait complete. 2024-04-26T21:45:08.172 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:08.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.285+0000 7f13744da640 1 Processor -- start 2024-04-26T21:45:08.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.285+0000 7f13744da640 1 -- start start 2024-04-26T21:45:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.286+0000 7f13744da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c0d06f0 0x7f136c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.286+0000 7f13744da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f136c0b2850 0x7f136c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.286+0000 7f13744da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f136c0d0b70 con 0x7f136c15f3a0 2024-04-26T21:45:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.286+0000 7f13744da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f136c0d0e20 con 0x7f136c0b2850 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.286+0000 7f13744da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f136c0b1d70 con 0x7f136c0d06f0 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.286+0000 7f1372276640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f136c15f3a0 0x7f136c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52834/0 (socket says 172.21.15.40:52834) 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1372276640 1 -- 172.21.15.40:0/1547364682 learned_addr learned my addr 172.21.15.40:0/1547364682 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371a75640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c0d06f0 0x7f136c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1372a77640 1 --2- 172.21.15.40:0/1547364682 >> v2:172.21.15.184:3300/0 conn(0x7f136c0b2850 0x7f136c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1327804198 0 0) 0x7f136c0d0b70 con 0x7f136c15f3a0 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1358003520 con 0x7f136c15f3a0 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 645435086 0 0) 0x7f1358003520 con 0x7f136c15f3a0 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c0d06f0 msgr2=0x7f136c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c0d06f0 0x7f136c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 >> v2:172.21.15.184:3300/0 conn(0x7f136c0b2850 msgr2=0x7f136c0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 --2- 172.21.15.40:0/1547364682 >> v2:172.21.15.184:3300/0 conn(0x7f136c0b2850 0x7f136c0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c0b2050 con 0x7f136c15f3a0 2024-04-26T21:45:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1368001670 con 0x7f136c15f3a0 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 >> v1:172.21.15.40:6789/0 conn(0x7f136c15f3a0 legacy=0x7f136c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13580039b0 0x7f1358003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f1358004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c0d06f0 0x7f13580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f136c0d0e20 con 0x7f1358004800 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f136c0b1d70 con 0x7f13580039b0 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f135800b9c0 con 0x7f136c0d06f0 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1371a75640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f1358004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.287+0000 7f1372276640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13580039b0 0x7f1358003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.288+0000 7f1372a77640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c0d06f0 0x7f13580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.288+0000 7f1371a75640 1 -- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c0d06f0 msgr2=0x7f13580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.288+0000 7f1371a75640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c0d06f0 0x7f13580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.288+0000 7f1371a75640 1 -- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13580039b0 msgr2=0x7f1358003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.288+0000 7f1371a75640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13580039b0 0x7f1358003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.288+0000 7f1371a75640 1 -- 172.21.15.40:0/1547364682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1358003520 con 0x7f1358004800 2024-04-26T21:45:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.288+0000 7f1371a75640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f1358004c80 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f135c009ac0 tx=0x7f1358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b8b3e00971f689 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:08.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c036070 con 0x7f1358004800 2024-04-26T21:45:08.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c031400 con 0x7f1358004800 2024-04-26T21:45:08.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f1371274640 1 -- 172.21.15.40:0/1547364682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c009690 con 0x7f1358004800 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f13744da640 1 -- 172.21.15.40:0/1547364682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 msgr2=0x7f1358004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f13744da640 1 --2- 172.21.15.40:0/1547364682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f1358004c80 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f135c009ac0 tx=0x7f1358005830 comp rx=0 tx=0).stop 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f1372a77640 1 -- 172.21.15.40:0/1547364682 reap_dead start 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f13744da640 1 -- 172.21.15.40:0/1547364682 shutdown_connections 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f13744da640 1 -- 172.21.15.40:0/1547364682 >> 172.21.15.40:0/1547364682 conn(0x7f136c0d4f90 msgr2=0x7f136c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f13744da640 1 -- 172.21.15.40:0/1547364682 shutdown_connections 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.289+0000 7f13744da640 1 -- 172.21.15.40:0/1547364682 wait complete. 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.290+0000 7f13744da640 1 Processor -- start 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f13744da640 1 -- start start 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f13744da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f136c161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f13744da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c14bdc0 0x7f136c1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f13744da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136c15c200 0x7f136c15c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f13744da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f136c15f900 con 0x7f1358004800 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f13744da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f136c15f780 con 0x7f136c15c200 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f13744da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f136c0d1e80 con 0x7f136c14bdc0 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f1372276640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f136c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f1372276640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f136c161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50900/0 (socket says 172.21.15.40:50900) 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.291+0000 7f1372276640 1 -- 172.21.15.40:0/3402933714 learned_addr learned my addr 172.21.15.40:0/3402933714 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1371a75640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c14bdc0 0x7f136c1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1372a77640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136c15c200 0x7f136c15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1372276640 1 -- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c14bdc0 msgr2=0x7f136c1f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1372276640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c14bdc0 0x7f136c1f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1372276640 1 -- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136c15c200 msgr2=0x7f136c15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1372276640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136c15c200 0x7f136c15c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1372276640 1 -- 172.21.15.40:0/3402933714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c161fd0 con 0x7f1358004800 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1371a75640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c14bdc0 0x7f136c1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1372a77640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136c15c200 0x7f136c15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1372276640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f136c161680 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f1368007340 tx=0x7f1368007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1362ffd640 1 -- 172.21.15.40:0/3402933714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136801a070 con 0x7f1358004800 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1362ffd640 1 -- 172.21.15.40:0/3402933714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136800f8e0 con 0x7f1358004800 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f1362ffd640 1 -- 172.21.15.40:0/3402933714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136800fcb0 con 0x7f1358004800 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136c1622b0 con 0x7f1358004800 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.292+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136c162870 con 0x7f1358004800 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.293+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136c0694f0 con 0x7f1358004800 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.294+0000 7f1362ffd640 1 -- 172.21.15.40:0/3402933714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f136801d360 con 0x7f1358004800 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.294+0000 7f1362ffd640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f135006c8f0 0x7f135006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.294+0000 7f1362ffd640 1 -- 172.21.15.40:0/3402933714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f136808c010 con 0x7f1358004800 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.294+0000 7f1371a75640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f135006c8f0 0x7f135006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.295+0000 7f1371a75640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f135006c8f0 0x7f135006edb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f135c033000 tx=0x7f135c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:08.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.299+0000 7f1362ffd640 1 -- 172.21.15.40:0/3402933714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1368059930 con 0x7f1358004800 2024-04-26T21:45:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.522+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f136c0b6860 con 0x7f1358004800 2024-04-26T21:45:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.523+0000 7f1362ffd640 1 -- 172.21.15.40:0/3402933714 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 254 v254) ==== 95+0+5193 (secure 0 0 0) 0x7f1368059230 con 0x7f1358004800 2024-04-26T21:45:08.524 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:08.524 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":254,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":254,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:07.757235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10308":{"gid":10308,"name":"c","rank":0,"incarnation":254,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/3538213517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3538213517},{"type":"v1","addr":"172.21.15.40:6833","nonce":3538213517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 254 2024-04-26T21:45:08.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f135006c8f0 msgr2=0x7f135006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:08.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f135006c8f0 0x7f135006edb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f135c033000 tx=0x7f135c008030 comp rx=0 tx=0).stop 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 msgr2=0x7f136c161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f136c161680 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f1368007340 tx=0x7f1368007800 comp rx=0 tx=0).stop 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 shutdown_connections 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f135006c8f0 0x7f135006edb0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136c15c200 0x7f136c15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136c14bdc0 0x7f136c1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 --2- 172.21.15.40:0/3402933714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358004800 0x7f136c161680 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 >> 172.21.15.40:0/3402933714 conn(0x7f136c0d4f90 msgr2=0x7f136c06e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 shutdown_connections 2024-04-26T21:45:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:08.525+0000 7f13744da640 1 -- 172.21.15.40:0/3402933714 wait complete. 2024-04-26T21:45:08.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 254, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:45:07.757235+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10308}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10308': {'gid': 10308, 'name': 'c', 'rank': 0, 'incarnation': 254, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/3538213517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3538213517}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3538213517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:45:08.536 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10308:up:replay 2024-04-26T21:45:08.536 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:45:12.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:45:12.448 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:45:13.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:14.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.052+0000 7f4757a37640 1 Processor -- start 2024-04-26T21:45:14.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.052+0000 7f4757a37640 1 -- start start 2024-04-26T21:45:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.053+0000 7f4757a37640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f475006f790 0x7f475006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.053+0000 7f4757a37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4750156840 con 0x7f4750072c50 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.053+0000 7f4757a37640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47500cbf20 con 0x7f4750156460 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.053+0000 7f4757a37640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f47500cc200 con 0x7f475006f790 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.053+0000 7f4755fd4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4750072c50 0x7f4750073060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34258/0 (socket says 172.21.15.40:34258) 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.053+0000 7f4755fd4640 1 -- 172.21.15.40:0/4231148897 learned_addr learned my addr 172.21.15.40:0/4231148897 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.053+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 885123668 0 0) 0x7f4750156840 con 0x7f4750072c50 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.053+0000 7f4754fd2640 1 --2- 172.21.15.40:0/4231148897 >> v2:172.21.15.184:3300/0 conn(0x7f475006f790 0x7f475006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f473c003520 con 0x7f4750072c50 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4141173329 0 0) 0x7f473c003520 con 0x7f4750072c50 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 >> v2:172.21.15.184:3300/0 conn(0x7f475006f790 msgr2=0x7f475006fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 --2- 172.21.15.40:0/4231148897 >> v2:172.21.15.184:3300/0 conn(0x7f475006f790 0x7f475006fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 >> v1:172.21.15.184:6789/0 conn(0x7f4750156460 legacy=0x7f47500cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47500cc4e0 con 0x7f4750072c50 2024-04-26T21:45:14.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f47480019f0 con 0x7f4750072c50 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 >> v1:172.21.15.40:6789/0 conn(0x7f4750072c50 legacy=0x7f4750073060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f473c0039b0 0x7f473c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f473c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f473c0051c0 0x7f473c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f47500cc200 con 0x7f473c004800 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f473c00c060 con 0x7f473c0039b0 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f473c00c260 con 0x7f473c0051c0 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4754fd2640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f473c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f47557d3640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f473c0039b0 0x7f473c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4755fd4640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f473c0051c0 0x7f473c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4754fd2640 1 -- 172.21.15.40:0/4231148897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f473c0051c0 msgr2=0x7f473c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4754fd2640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f473c0051c0 0x7f473c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4754fd2640 1 -- 172.21.15.40:0/4231148897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f473c0039b0 msgr2=0x7f473c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4754fd2640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f473c0039b0 0x7f473c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4755fd4640 1 -- 172.21.15.40:0/4231148897 reap_dead start 2024-04-26T21:45:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.054+0000 7f4754fd2640 1 -- 172.21.15.40:0/4231148897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c003520 con 0x7f473c004800 2024-04-26T21:45:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.055+0000 7f4754fd2640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f473c004c80 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f474000bfc0 tx=0x7f473c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2de20c93f113e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.055+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740038070 con 0x7f473c004800 2024-04-26T21:45:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.055+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4740033600 con 0x7f473c004800 2024-04-26T21:45:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.055+0000 7f4747fff640 1 -- 172.21.15.40:0/4231148897 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474000bc10 con 0x7f473c004800 2024-04-26T21:45:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.056+0000 7f4757a37640 1 -- 172.21.15.40:0/4231148897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 msgr2=0x7f473c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.056+0000 7f4757a37640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f473c004c80 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f474000bfc0 tx=0x7f473c005830 comp rx=0 tx=0).stop 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.056+0000 7f4757a37640 1 -- 172.21.15.40:0/4231148897 shutdown_connections 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.056+0000 7f4757a37640 1 --2- 172.21.15.40:0/4231148897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f473c004c80 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.056+0000 7f4757a37640 1 -- 172.21.15.40:0/4231148897 >> 172.21.15.40:0/4231148897 conn(0x7f47500d0be0 msgr2=0x7f47500d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.056+0000 7f4757a37640 1 -- 172.21.15.40:0/4231148897 shutdown_connections 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.056+0000 7f4757a37640 1 -- 172.21.15.40:0/4231148897 wait complete. 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.057+0000 7f4757a37640 1 Processor -- start 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.057+0000 7f4757a37640 1 -- start start 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f4757a37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f475014dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f4757a37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47500b5d20 0x7f47501524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f4757a37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4750152a10 0x7f47501f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f4757a37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f475006e260 con 0x7f473c004800 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f4757a37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4750072700 con 0x7f47500b5d20 2024-04-26T21:45:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f4757a37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f475006e3e0 con 0x7f4750152a10 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f47557d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f475014dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f47557d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f475014dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38644/0 (socket says 172.21.15.40:38644) 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f47557d3640 1 -- 172.21.15.40:0/3329508417 learned_addr learned my addr 172.21.15.40:0/3329508417 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f47557d3640 1 -- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4750152a10 msgr2=0x7f47501f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f47557d3640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4750152a10 0x7f47501f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f47557d3640 1 -- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47500b5d20 msgr2=0x7f47501524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f47557d3640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47500b5d20 0x7f47501524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.058+0000 7f47557d3640 1 -- 172.21.15.40:0/3329508417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47500b3990 con 0x7f473c004800 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.059+0000 7f47557d3640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f475014dd90 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f474c00f620 tx=0x7f474c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.059+0000 7f4745ffb640 1 -- 172.21.15.40:0/3329508417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0052e0 con 0x7f473c004800 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.059+0000 7f4745ffb640 1 -- 172.21.15.40:0/3329508417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c018070 con 0x7f473c004800 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.059+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4750154790 con 0x7f473c004800 2024-04-26T21:45:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.059+0000 7f4745ffb640 1 -- 172.21.15.40:0/3329508417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c003810 con 0x7f473c004800 2024-04-26T21:45:14.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.059+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47500b3c30 con 0x7f473c004800 2024-04-26T21:45:14.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.060+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4750152e30 con 0x7f473c004800 2024-04-26T21:45:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.060+0000 7f4745ffb640 1 -- 172.21.15.40:0/3329508417 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f474c0039b0 con 0x7f473c004800 2024-04-26T21:45:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.061+0000 7f4745ffb640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.061+0000 7f4745ffb640 1 -- 172.21.15.40:0/3329508417 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f474c08c310 con 0x7f473c004800 2024-04-26T21:45:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.061+0000 7f4754fd2640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.062+0000 7f4754fd2640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f473406c8f0 0x7f473406edb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4740035000 tx=0x7f474002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.065+0000 7f4745ffb640 1 -- 172.21.15.40:0/3329508417 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f474c059cc0 con 0x7f473c004800 2024-04-26T21:45:14.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.290+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47500694f0 con 0x7f473c004800 2024-04-26T21:45:14.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.290+0000 7f4745ffb640 1 -- 172.21.15.40:0/3329508417 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 257 v257) ==== 95+0+5202 (secure 0 0 0) 0x7f474c0595c0 con 0x7f473c004800 2024-04-26T21:45:14.291 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:14.291 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":257,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":257,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:10.771347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10308":{"gid":10308,"name":"c","rank":0,"incarnation":254,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/3538213517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3538213517},{"type":"v1","addr":"172.21.15.40:6833","nonce":3538213517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10308,"qdb_cluster":[10308]},"id":6}]} 2024-04-26T21:45:14.291 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 257 2024-04-26T21:45:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f473406c8f0 msgr2=0x7f473406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f473406c8f0 0x7f473406edb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4740035000 tx=0x7f474002e000 comp rx=0 tx=0).stop 2024-04-26T21:45:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 msgr2=0x7f475014dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f475014dd90 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f474c00f620 tx=0x7f474c00fae0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 shutdown_connections 2024-04-26T21:45:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4750152a10 0x7f47501f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47500b5d20 0x7f47501524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 --2- 172.21.15.40:0/3329508417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f473c004800 0x7f475014dd90 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 >> 172.21.15.40:0/3329508417 conn(0x7f47500d0be0 msgr2=0x7f47500cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 shutdown_connections 2024-04-26T21:45:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.292+0000 7f4757a37640 1 -- 172.21.15.40:0/3329508417 wait complete. 2024-04-26T21:45:14.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 257, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:45:10.771347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10308}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10308': {'gid': 10308, 'name': 'c', 'rank': 0, 'incarnation': 254, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6833/3538213517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3538213517}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3538213517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10308, 'qdb_cluster': [10308]} 2024-04-26T21:45:14.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:45:14.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:45:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.406+0000 7f73aca07640 1 Processor -- start 2024-04-26T21:45:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.406+0000 7f73aca07640 1 -- start start 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73aca07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a8162810 0x7f73a8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73aca07640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73a80b2340 0x7f73a80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73aca07640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73a80cd930 0x7f73a80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73aca07640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f73a80d0750 con 0x7f73a80b2340 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73aca07640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f73a80d16a0 con 0x7f73a80cd930 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73aca07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73a80d10d0 con 0x7f73a8162810 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f739ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73a80b2340 0x7f73a80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f739ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73a80b2340 0x7f73a80cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38646/0 (socket says 172.21.15.40:38646) 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f739ffff640 1 -- 172.21.15.40:0/2266493066 learned_addr learned my addr 172.21.15.40:0/2266493066 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/2266493066 >> v2:172.21.15.184:3300/0 conn(0x7f73a80cd930 0x7f73a80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73a659c640 1 --2- 172.21.15.40:0/2266493066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a8162810 0x7f73a8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f739ffff640 1 -- 172.21.15.40:0/2266493066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a8162810 msgr2=0x7f73a8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f739ffff640 1 --2- 172.21.15.40:0/2266493066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a8162810 0x7f73a8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f739ffff640 1 -- 172.21.15.40:0/2266493066 >> v2:172.21.15.184:3300/0 conn(0x7f73a80cd930 msgr2=0x7f73a80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f739ffff640 1 --2- 172.21.15.40:0/2266493066 >> v2:172.21.15.184:3300/0 conn(0x7f73a80cd930 0x7f73a80d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f739ffff640 1 -- 172.21.15.40:0/2266493066 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a80d1370 con 0x7f73a80b2340 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.407+0000 7f73a659c640 1 --2- 172.21.15.40:0/2266493066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a8162810 0x7f73a8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.408+0000 7f739ffff640 1 --2- 172.21.15.40:0/2266493066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a80b2340 0x7f73a80cd3f0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f73a80b2250 tx=0x7f739002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914fc64c538ad836 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.408+0000 7f73a5d9b640 1 -- 172.21.15.40:0/2266493066 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390037070 con 0x7f73a80b2340 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.408+0000 7f73a5d9b640 1 -- 172.21.15.40:0/2266493066 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7390032440 con 0x7f73a80b2340 2024-04-26T21:45:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.408+0000 7f73a5d9b640 1 -- 172.21.15.40:0/2266493066 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390032810 con 0x7f73a80b2340 2024-04-26T21:45:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.408+0000 7f73aca07640 1 -- 172.21.15.40:0/2266493066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a80b2340 msgr2=0x7f73a80cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.409+0000 7f73aca07640 1 --2- 172.21.15.40:0/2266493066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a80b2340 0x7f73a80cd3f0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f73a80b2250 tx=0x7f739002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.409+0000 7f73aca07640 1 -- 172.21.15.40:0/2266493066 shutdown_connections 2024-04-26T21:45:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.409+0000 7f73aca07640 1 --2- 172.21.15.40:0/2266493066 >> v2:172.21.15.184:3300/0 conn(0x7f73a80cd930 0x7f73a80d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.409+0000 7f73aca07640 1 --2- 172.21.15.40:0/2266493066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a80b2340 0x7f73a80cd3f0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.409+0000 7f73aca07640 1 --2- 172.21.15.40:0/2266493066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a8162810 0x7f73a8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.409+0000 7f73aca07640 1 -- 172.21.15.40:0/2266493066 >> 172.21.15.40:0/2266493066 conn(0x7f73a80d2da0 msgr2=0x7f73a80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.409+0000 7f73aca07640 1 -- 172.21.15.40:0/2266493066 shutdown_connections 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.409+0000 7f73aca07640 1 -- 172.21.15.40:0/2266493066 wait complete. 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.410+0000 7f73aca07640 1 Processor -- start 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.410+0000 7f73aca07640 1 -- start start 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73aca07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80b2340 0x7f73a80ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73aca07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80cd930 0x7f73a80ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73aca07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8162810 0x7f73a806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73aca07640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8070890 con 0x7f73a8162810 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73aca07640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a80d0470 con 0x7f73a80b2340 2024-04-26T21:45:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73aca07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73a80d2070 con 0x7f73a80cd930 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8162810 0x7f73a806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8162810 0x7f73a806dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38662/0 (socket says 172.21.15.40:38662) 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 -- 172.21.15.40:0/3981663989 learned_addr learned my addr 172.21.15.40:0/3981663989 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f739ffff640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80cd930 0x7f73a80ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a659c640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80b2340 0x7f73a80ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 -- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80cd930 msgr2=0x7f73a80ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80cd930 0x7f73a80ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 -- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80b2340 msgr2=0x7f73a80ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80b2340 0x7f73a80ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 -- 172.21.15.40:0/3981663989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a80c6a00 con 0x7f73a8162810 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a659c640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80b2340 0x7f73a80ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8162810 0x7f73a806dd60 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f73a000eea0 tx=0x7f73a000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f739effd640 1 -- 172.21.15.40:0/3981663989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0018020 con 0x7f73a8162810 2024-04-26T21:45:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f739effd640 1 -- 172.21.15.40:0/3981663989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0014400 con 0x7f73a8162810 2024-04-26T21:45:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.411+0000 7f73aca07640 1 -- 172.21.15.40:0/3981663989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a80c6ce0 con 0x7f73a8162810 2024-04-26T21:45:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.412+0000 7f739effd640 1 -- 172.21.15.40:0/3981663989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a00147f0 con 0x7f73a8162810 2024-04-26T21:45:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.412+0000 7f73aca07640 1 -- 172.21.15.40:0/3981663989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a806e2a0 con 0x7f73a8162810 2024-04-26T21:45:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.413+0000 7f739effd640 1 -- 172.21.15.40:0/3981663989 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73a0005d10 con 0x7f73a8162810 2024-04-26T21:45:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.413+0000 7f739effd640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f738806c8f0 0x7f738806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.414+0000 7f739effd640 1 -- 172.21.15.40:0/3981663989 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f73a008d060 con 0x7f73a8162810 2024-04-26T21:45:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.414+0000 7f73a659c640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f738806c8f0 0x7f738806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.414+0000 7f73a659c640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f738806c8f0 0x7f738806edb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7398006fd0 tx=0x7f7398008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.414+0000 7f73af7fa740 1 Processor -- start 2024-04-26T21:45:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.415+0000 7f73af7fa740 1 -- start start 2024-04-26T21:45:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.415+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4bd440 0x55ab2a4bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.415+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 0x55ab2a4db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.415+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a493120 0x55ab2a4d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.415+0000 7f73af7fa740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a3f7740 con 0x55ab2a493120 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.415+0000 7f73af7fa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a3933a0 con 0x55ab2a4dace0 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.415+0000 7f73af7fa740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ab2a3f3700 con 0x55ab2a4bd440 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a493120 0x55ab2a4d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a493120 0x55ab2a4d93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38678/0 (socket says 172.21.15.40:38678) 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 -- 172.21.15.40:0/3923890756 learned_addr learned my addr 172.21.15.40:0/3923890756 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a659c640 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4bd440 0x55ab2a4bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f739ffff640 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 0x55ab2a4db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 -- 172.21.15.40:0/3923890756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4bd440 msgr2=0x55ab2a4bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4bd440 0x55ab2a4bd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 -- 172.21.15.40:0/3923890756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 msgr2=0x55ab2a4db230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 0x55ab2a4db230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 -- 172.21.15.40:0/3923890756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2a3af750 con 0x55ab2a493120 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a493120 0x55ab2a4d93a0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f73a005a2a0 tx=0x7f73a005aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f7aedc20147210 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f7387fff640 1 -- 172.21.15.40:0/3923890756 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0062ef0 con 0x55ab2a493120 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f7387fff640 1 -- 172.21.15.40:0/3923890756 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0058610 con 0x55ab2a493120 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.416+0000 7f7387fff640 1 -- 172.21.15.40:0/3923890756 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0059760 con 0x55ab2a493120 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 -- 172.21.15.40:0/3923890756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a493120 msgr2=0x55ab2a4d93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a493120 0x55ab2a4d93a0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f73a005a2a0 tx=0x7f73a005aa90 comp rx=0 tx=0).stop 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 -- 172.21.15.40:0/3923890756 shutdown_connections 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a493120 0x55ab2a4d93a0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 0x55ab2a4db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 --2- 172.21.15.40:0/3923890756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4bd440 0x55ab2a4bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 -- 172.21.15.40:0/3923890756 >> 172.21.15.40:0/3923890756 conn(0x55ab2a48fa50 msgr2=0x55ab2a4d8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 -- 172.21.15.40:0/3923890756 shutdown_connections 2024-04-26T21:45:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 -- 172.21.15.40:0/3923890756 wait complete. 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.417+0000 7f73af7fa740 1 -- start start 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.418+0000 7f73af7fa740 1 Processor -- start 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.418+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a4c0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.418+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a4c0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.418+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a4d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73af7fa740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a493ec0 con 0x55ab2a4dace0 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73af7fa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a37efe0 con 0x55ab2a4bd440 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73af7fa740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ab2a48d980 con 0x55ab2a4cfcb0 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73af7fa740 10 client.0 _ll_register_callbacks cb 0x55ab2a395150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a4c0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a4c0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38694/0 (socket says 172.21.15.40:38694) 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 -- 172.21.15.40:0/1934131003 learned_addr learned my addr 172.21.15.40:0/1934131003 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73a659c640 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a4c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a4d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 -- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 msgr2=0x55ab2a4d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a4d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 -- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 msgr2=0x55ab2a4c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a4c0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 -- 172.21.15.40:0/1934131003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2a1092d0 con 0x55ab2a4dace0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73a659c640 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a4c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f739ffff640 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a4c0a90 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f7390009960 tx=0x7f7390004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f761ccf80e2bf793 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73857fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73857fa640 1 -- 172.21.15.40:0/1934131003 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390037040 con 0x55ab2a4dace0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73857fa640 1 -- 172.21.15.40:0/1934131003 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f739003c410 con 0x55ab2a4dace0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73af7fa740 1 -- 172.21.15.40:0/1934131003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a0058610 con 0x55ab2a4dace0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73857fa640 1 -- 172.21.15.40:0/1934131003 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7f7390046630 con 0x55ab2a4dace0 2024-04-26T21:45:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.419+0000 7f73af7fa740 10 client.10395 fetch_fsmap learned FSMap version 257 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73af7fa740 1 -- 172.21.15.40:0/1934131003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=2+}) -- 0x55ab2a1d36b0 con 0x55ab2a4dace0 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73857fa640 1 -- 172.21.15.40:0/1934131003 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0058610 con 0x55ab2a4dace0 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73857fa640 1 -- 172.21.15.40:0/1934131003 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 257) ==== 3494+0+0 (secure 0 0 0) 0x7f73900467d0 con 0x55ab2a4dace0 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73af7fa740 10 client.10395 fetch_fsmap finished waiting for FSMap version 257 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73af7fa740 10 client.10395 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73af7fa740 20 client.10395 populate_metadata read hostname 'smithi040' 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] conn(0x55ab2a4acd30 0x55ab2a4ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73af7fa740 4 client.10395 mds_command: new command op to 10308 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73af7fa740 1 -- 172.21.15.40:0/1934131003 --> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab2a405350 con 0x55ab2a4acd30 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73a659c640 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] conn(0x55ab2a4acd30 0x55ab2a4ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73857fa640 10 client.10395 ms_handle_connect on v2:172.21.15.40:6832/3538213517 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.420+0000 7f73a659c640 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] conn(0x55ab2a4acd30 0x55ab2a4ad110 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6929b7c4c08f3e server_cookie=247857f11fca675f in_seq=0 out_seq=0 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73857fa640 1 -- 172.21.15.40:0/1934131003 <== mds.0 v2:172.21.15.40:6832/3538213517 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab2a405350 con 0x55ab2a4acd30 2024-04-26T21:45:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73857fa640 10 client.10395 handle_command_reply: tid=0 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 client.10395 shutdown 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 -- 172.21.15.40:0/1934131003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 msgr2=0x55ab2a4c0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a4c0a90 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f7390009960 tx=0x7f7390004730 comp rx=0 tx=0).stop 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 -- 172.21.15.40:0/1934131003 shutdown_connections 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a4d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a4c0a90 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] conn(0x55ab2a4acd30 0x55ab2a4ad110 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1934131003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a4c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 -- 172.21.15.40:0/1934131003 >> 172.21.15.40:0/1934131003 conn(0x55ab2a493120 msgr2=0x55ab2a4dc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73857fa640 0 client.10395 ms_handle_reset on v2:172.21.15.40:6832/3538213517 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 -- 172.21.15.40:0/1934131003 shutdown_connections 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 1 -- 172.21.15.40:0/1934131003 wait complete. 2024-04-26T21:45:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.421+0000 7f73af7fa740 20 client.10395 trim_cache size 0 max 16384 2024-04-26T21:45:14.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.440+0000 7f73af7fa740 1 Processor -- start 2024-04-26T21:45:14.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.440+0000 7f73af7fa740 1 -- start start 2024-04-26T21:45:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.440+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4bd440 0x55ab2a4b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.440+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a4d9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.440+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 0x55ab2a4eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.440+0000 7f73af7fa740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8203420 con 0x55ab2a4bd440 2024-04-26T21:45:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.440+0000 7f73af7fa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8202400 con 0x55ab2a4dace0 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.440+0000 7f73af7fa740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73a80c8790 con 0x55ab2a4cfcb0 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4bd440 0x55ab2a4b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4bd440 0x55ab2a4b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38698/0 (socket says 172.21.15.40:38698) 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 -- 172.21.15.40:0/3571772699 learned_addr learned my addr 172.21.15.40:0/3571772699 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f739ffff640 1 --2- 172.21.15.40:0/3571772699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a4d9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 -- 172.21.15.40:0/3571772699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 msgr2=0x55ab2a4d9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 --2- 172.21.15.40:0/3571772699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a4d9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 -- 172.21.15.40:0/3571772699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 msgr2=0x55ab2a4eaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 --2- 172.21.15.40:0/3571772699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 0x55ab2a4eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 -- 172.21.15.40:0/3571772699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2a492700 con 0x55ab2a4bd440 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73a659c640 1 --2- 172.21.15.40:0/3571772699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4bd440 0x55ab2a4b0510 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f739800eb20 tx=0x7f7398014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b703f161ebc86de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73877fe640 1 -- 172.21.15.40:0/3571772699 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739800f040 con 0x55ab2a4bd440 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73877fe640 1 -- 172.21.15.40:0/3571772699 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7398004030 con 0x55ab2a4bd440 2024-04-26T21:45:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.441+0000 7f73877fe640 1 -- 172.21.15.40:0/3571772699 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980041e0 con 0x55ab2a4bd440 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 -- 172.21.15.40:0/3571772699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4bd440 msgr2=0x55ab2a4b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 --2- 172.21.15.40:0/3571772699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4bd440 0x55ab2a4b0510 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f739800eb20 tx=0x7f7398014000 comp rx=0 tx=0).stop 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 -- 172.21.15.40:0/3571772699 shutdown_connections 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 --2- 172.21.15.40:0/3571772699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4dace0 0x55ab2a4eaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 --2- 172.21.15.40:0/3571772699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a4d9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 --2- 172.21.15.40:0/3571772699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4bd440 0x55ab2a4b0510 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 -- 172.21.15.40:0/3571772699 >> 172.21.15.40:0/3571772699 conn(0x55ab2a493120 msgr2=0x55ab2a4d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 -- 172.21.15.40:0/3571772699 shutdown_connections 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 -- 172.21.15.40:0/3571772699 wait complete. 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 -- start start 2024-04-26T21:45:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.442+0000 7f73af7fa740 1 Processor -- start 2024-04-26T21:45:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a549bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a48d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73af7fa740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a491360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73af7fa740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a4d2e70 con 0x55ab2a4dace0 2024-04-26T21:45:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73af7fa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8201d70 con 0x55ab2a4bd440 2024-04-26T21:45:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73af7fa740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ab2a4abfb0 con 0x55ab2a4cfcb0 2024-04-26T21:45:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73af7fa740 10 client.0 _ll_register_callbacks cb 0x7f73a00095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73a6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a491360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73a6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a491360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38706/0 (socket says 172.21.15.40:38706) 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73a6d9d640 1 -- 172.21.15.40:0/1114391279 learned_addr learned my addr 172.21.15.40:0/1114391279 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f73a659c640 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a549bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.443+0000 7f739ffff640 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a48d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73a6d9d640 1 -- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 msgr2=0x55ab2a48d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a48d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73a6d9d640 1 -- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 msgr2=0x55ab2a549bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a549bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73a6d9d640 1 -- 172.21.15.40:0/1114391279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2a4b0820 con 0x55ab2a4dace0 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f739ffff640 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a48d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73a6d9d640 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a491360 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b910 tx=0x7f73a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb212a00104a3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73857fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:45:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73857fa640 1 -- 172.21.15.40:0/1114391279 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a007e060 con 0x55ab2a4dace0 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73857fa640 1 -- 172.21.15.40:0/1114391279 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f73a0059760 con 0x55ab2a4dace0 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 1 -- 172.21.15.40:0/1114391279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7398004030 con 0x55ab2a4dace0 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73857fa640 1 -- 172.21.15.40:0/1114391279 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7f73a0079070 con 0x55ab2a4dace0 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 10 client.10401 fetch_fsmap learned FSMap version 257 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 1 -- 172.21.15.40:0/1114391279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=2+}) -- 0x55ab2a48c870 con 0x55ab2a4dace0 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73857fa640 1 -- 172.21.15.40:0/1114391279 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7398004030 con 0x55ab2a4dace0 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73857fa640 1 -- 172.21.15.40:0/1114391279 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 257) ==== 3494+0+0 (secure 0 0 0) 0x7f73a005e170 con 0x55ab2a4dace0 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 10 client.10401 fetch_fsmap finished waiting for FSMap version 257 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 10 client.10401 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 20 client.10401 populate_metadata read hostname 'smithi040' 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] conn(0x55ab2a4e7e70 0x55ab2a4cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 4 client.10401 mds_command: new command op to 10308 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.444+0000 7f73af7fa740 1 -- 172.21.15.40:0/1114391279 --> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] -- command(tid 0: {"prefix": "status"}) -- 0x55ab2a3ff1c0 con 0x55ab2a4e7e70 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73a659c640 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] conn(0x55ab2a4e7e70 0x55ab2a4cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73857fa640 10 client.10401 ms_handle_connect on v2:172.21.15.40:6832/3538213517 2024-04-26T21:45:14.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73a659c640 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] conn(0x55ab2a4e7e70 0x55ab2a4cdbe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9709bf8fa5da7619 server_cookie=3ea5bb636accd8d2 in_seq=0 out_seq=0 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73857fa640 1 -- 172.21.15.40:0/1114391279 <== mds.0 v2:172.21.15.40:6832/3538213517 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ab2a3ff1c0 con 0x55ab2a4e7e70 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73857fa640 10 client.10401 handle_command_reply: tid=0 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 client.10401 shutdown 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 -- 172.21.15.40:0/1114391279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 msgr2=0x55ab2a491360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a491360 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b910 tx=0x7f73a0004a40 comp rx=0 tx=0).stop 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 -- 172.21.15.40:0/1114391279 shutdown_connections 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab2a4dace0 0x55ab2a491360 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab2a4cfcb0 0x55ab2a48d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.40:6832/3538213517,v1:172.21.15.40:6833/3538213517] conn(0x55ab2a4e7e70 0x55ab2a4cdbe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 --2- 172.21.15.40:0/1114391279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab2a4bd440 0x55ab2a549bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 -- 172.21.15.40:0/1114391279 >> 172.21.15.40:0/1114391279 conn(0x55ab2a493120 msgr2=0x55ab2a4c0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73857fa640 0 client.10401 ms_handle_reset on v2:172.21.15.40:6832/3538213517 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 -- 172.21.15.40:0/1114391279 shutdown_connections 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.445+0000 7f73af7fa740 1 -- 172.21.15.40:0/1114391279 wait complete. 2024-04-26T21:45:14.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73af7fa740 20 client.10401 trim_cache size 0 max 16384 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 10308, 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.687848174, 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 257, 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 121, 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 121, 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.677570783 2024-04-26T21:45:14.453 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:45:14.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 -- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f738806c8f0 msgr2=0x7f738806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f738806c8f0 0x7f738806edb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7398006fd0 tx=0x7f7398008040 comp rx=0 tx=0).stop 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 -- 172.21.15.40:0/3981663989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8162810 msgr2=0x7f73a806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8162810 0x7f73a806dd60 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f73a000eea0 tx=0x7f73a000ef90 comp rx=0 tx=0).stop 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 -- 172.21.15.40:0/3981663989 shutdown_connections 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f738806c8f0 0x7f738806edb0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8162810 0x7f73a806dd60 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80cd930 0x7f73a80ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 --2- 172.21.15.40:0/3981663989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80b2340 0x7f73a80ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 -- 172.21.15.40:0/3981663989 >> 172.21.15.40:0/3981663989 conn(0x7f73a80d2da0 msgr2=0x7f73a8070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 -- 172.21.15.40:0/3981663989 shutdown_connections 2024-04-26T21:45:14.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.446+0000 7f73aca07640 1 -- 172.21.15.40:0/3981663989 wait complete. 2024-04-26T21:45:14.458 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:45:14.458 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Truncate 500.00000000', proceeding to ls 2024-04-26T21:45:14.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:45:14.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.557+0000 7f1fb53da640 1 Processor -- start 2024-04-26T21:45:14.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.558+0000 7f1fb53da640 1 -- start start 2024-04-26T21:45:14.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.558+0000 7f1fb53da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1fb00d06f0 0x7f1fb00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.558+0000 7f1fb53da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb00b2850 0x7f1fb00b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.558+0000 7f1fb53da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb00d0ba0 con 0x7f1fb015f3a0 2024-04-26T21:45:14.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.558+0000 7f1fb53da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb00d0e20 con 0x7f1fb00d06f0 2024-04-26T21:45:14.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.558+0000 7f1fb53da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb00b1d70 con 0x7f1fb00b2850 2024-04-26T21:45:14.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.558+0000 7f1faeffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1fb015f3a0 0x7f1fb00d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34260/0 (socket says 172.21.15.40:34260) 2024-04-26T21:45:14.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.558+0000 7f1faeffd640 1 -- 172.21.15.40:0/2668654214 learned_addr learned my addr 172.21.15.40:0/2668654214 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1faf7fe640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb00b2850 0x7f1fb00b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fa7fff640 1 --2- 172.21.15.40:0/2668654214 >> v2:172.21.15.184:3300/0 conn(0x7f1fb00d06f0 0x7f1fb00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991037170 0 0) 0x7f1fa0001110 con 0x7f1fb015f3a0 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f9c003520 con 0x7f1fb015f3a0 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2647923131 0 0) 0x7f1f9c003520 con 0x7f1fb015f3a0 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb00b2850 msgr2=0x7f1fb00b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb00b2850 0x7f1fb00b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 >> v2:172.21.15.184:3300/0 conn(0x7f1fb00d06f0 msgr2=0x7f1fb00d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 --2- 172.21.15.40:0/2668654214 >> v2:172.21.15.184:3300/0 conn(0x7f1fb00d06f0 0x7f1fb00d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb00b2050 con 0x7f1fb015f3a0 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1fa0001840 con 0x7f1fb015f3a0 2024-04-26T21:45:14.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 >> v1:172.21.15.40:6789/0 conn(0x7f1fb015f3a0 legacy=0x7f1fb00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb00b2850 0x7f1f9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c003f30 0x7f1f9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c004d40 0x7f1f9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00d0e20 con 0x7f1f9c004d40 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00b1d70 con 0x7f1f9c003f30 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c00b9c0 con 0x7f1fb00b2850 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1faf7fe640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c004d40 0x7f1f9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1faeffd640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb00b2850 0x7f1f9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.559+0000 7f1fa7fff640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c003f30 0x7f1f9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1faf7fe640 1 -- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb00b2850 msgr2=0x7f1f9c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1faf7fe640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb00b2850 0x7f1f9c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1faf7fe640 1 -- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c003f30 msgr2=0x7f1f9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1faf7fe640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c003f30 0x7f1f9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1faf7fe640 1 -- 172.21.15.40:0/2668654214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c003520 con 0x7f1f9c004d40 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1faf7fe640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c004d40 0x7f1f9c0051c0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009fc0 tx=0x7f1f9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f887427b117e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8009c10 con 0x7f1f9c004d40 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa8036070 con 0x7f1f9c004d40 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.560+0000 7f1fae7fc640 1 -- 172.21.15.40:0/2668654214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8031670 con 0x7f1f9c004d40 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.561+0000 7f1fb53da640 1 -- 172.21.15.40:0/2668654214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c004d40 msgr2=0x7f1f9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.561+0000 7f1fb53da640 1 --2- 172.21.15.40:0/2668654214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c004d40 0x7f1f9c0051c0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009fc0 tx=0x7f1f9c005830 comp rx=0 tx=0).stop 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.561+0000 7f1faf7fe640 1 -- 172.21.15.40:0/2668654214 reap_dead start 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.561+0000 7f1fb53da640 1 -- 172.21.15.40:0/2668654214 shutdown_connections 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.561+0000 7f1fb53da640 1 -- 172.21.15.40:0/2668654214 >> 172.21.15.40:0/2668654214 conn(0x7f1fb00d4f90 msgr2=0x7f1fb00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.561+0000 7f1fb53da640 1 -- 172.21.15.40:0/2668654214 shutdown_connections 2024-04-26T21:45:14.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.561+0000 7f1fb53da640 1 -- 172.21.15.40:0/2668654214 wait complete. 2024-04-26T21:45:14.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.562+0000 7f1fb53da640 1 Processor -- start 2024-04-26T21:45:14.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.562+0000 7f1fb53da640 1 -- start start 2024-04-26T21:45:14.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fb53da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb0153630 0x7f1fb00ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fb53da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb00cb540 0x7f1fb00ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fb53da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb015fa80 0x7f1fb015ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fb53da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb015f900 con 0x7f1fb00cb540 2024-04-26T21:45:14.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fb53da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb015f780 con 0x7f1fb015fa80 2024-04-26T21:45:14.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fb53da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb00d23c0 con 0x7f1fb0153630 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fa7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb00cb540 0x7f1fb00ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fa7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb00cb540 0x7f1fb00ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38720/0 (socket says 172.21.15.40:38720) 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fa7fff640 1 -- 172.21.15.40:0/3619352852 learned_addr learned my addr 172.21.15.40:0/3619352852 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1faeffd640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb0153630 0x7f1fb00ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1faf7fe640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb015fa80 0x7f1fb015ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fa7fff640 1 -- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb0153630 msgr2=0x7f1fb00ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fa7fff640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb0153630 0x7f1fb00ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fa7fff640 1 -- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb015fa80 msgr2=0x7f1fb015ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fa7fff640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb015fa80 0x7f1fb015ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.563+0000 7f1fa7fff640 1 -- 172.21.15.40:0/3619352852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb00ca4a0 con 0x7f1fb00cb540 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.564+0000 7f1faeffd640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb0153630 0x7f1fb00ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.564+0000 7f1fa7fff640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb00cb540 0x7f1fb00ce5b0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f1f98011960 tx=0x7f1f98011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.564+0000 7f1fa77fe640 1 -- 172.21.15.40:0/3619352852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9800faf0 con 0x7f1fb00cb540 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.564+0000 7f1fa77fe640 1 -- 172.21.15.40:0/3619352852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9800fc90 con 0x7f1fb00cb540 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.564+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb00ca780 con 0x7f1fb00cb540 2024-04-26T21:45:14.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.564+0000 7f1fa77fe640 1 -- 172.21.15.40:0/3619352852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98014440 con 0x7f1fb00cb540 2024-04-26T21:45:14.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.564+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb00cad40 con 0x7f1fb00cb540 2024-04-26T21:45:14.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.565+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb00694f0 con 0x7f1fb00cb540 2024-04-26T21:45:14.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.565+0000 7f1fa77fe640 1 -- 172.21.15.40:0/3619352852 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f98012040 con 0x7f1fb00cb540 2024-04-26T21:45:14.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.566+0000 7f1fa77fe640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f9006c8f0 0x7f1f9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:14.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.566+0000 7f1fa77fe640 1 -- 172.21.15.40:0/3619352852 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f1f9808cde0 con 0x7f1fb00cb540 2024-04-26T21:45:14.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.566+0000 7f1faeffd640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f9006c8f0 0x7f1f9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:14.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.566+0000 7f1faeffd640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f9006c8f0 0x7f1f9006edb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0000fb0 tx=0x7f1fa0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:14.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.570+0000 7f1fa77fe640 1 -- 172.21.15.40:0/3619352852 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f98019030 con 0x7f1fb00cb540 2024-04-26T21:45:14.748 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:14.747+0000 7fb9c4ed5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:45:14.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.794+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1fb00b6860 con 0x7f1fb00cb540 2024-04-26T21:45:14.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.794+0000 7f1fa77fe640 1 -- 172.21.15.40:0/3619352852 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v257) ==== 53+0+73 (secure 0 0 0) 0x7f1f9805a6a0 con 0x7f1fb00cb540 2024-04-26T21:45:14.795 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:45:14.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f9006c8f0 msgr2=0x7f1f9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f9006c8f0 0x7f1f9006edb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0000fb0 tx=0x7f1fa0009040 comp rx=0 tx=0).stop 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb00cb540 msgr2=0x7f1fb00ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb00cb540 0x7f1fb00ce5b0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f1f98011960 tx=0x7f1f98011e20 comp rx=0 tx=0).stop 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 shutdown_connections 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f9006c8f0 0x7f1f9006edb0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb015fa80 0x7f1fb015ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb00cb540 0x7f1fb00ce5b0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 --2- 172.21.15.40:0/3619352852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb0153630 0x7f1fb00ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.796+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 >> 172.21.15.40:0/3619352852 conn(0x7f1fb00d4f90 msgr2=0x7f1fb00d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.797+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 shutdown_connections 2024-04-26T21:45:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:14.797+0000 7f1fb53da640 1 -- 172.21.15.40:0/3619352852 wait complete. 2024-04-26T21:45:14.808 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:45:14.808 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:45:14.808 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:45:14.808 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:45:14.808 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:45:14.808 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:45:14.809 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:45:14.809 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:45:14.809 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:45:14.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:14.809 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:45:14.824 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:45:14.825 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42133sec preferred_lft 42133sec 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:14.826 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:14.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:14.827 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:45:14.887 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:45:14.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:14.889 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:45:14.945 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:45:14.945 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:45:14.946 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:45:14.946 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:45:14.946 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:45:14.946 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:45:14.946 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:45:14.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:45:14.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:45:14.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:45:14.946 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:45:15.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:45:15.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:45:15.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:15.071 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:45:15.087 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:45:15.087 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:45:15.088 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:45:15.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:15.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:45:15.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:45:15.152 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:45:15.207 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:45:15.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:45:15.208 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:45:15.262 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:45:15.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:15.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:45:15.345 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:45:15.345 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:45:15.353 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:45:15.353 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:45:15.353 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:45:15.353 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:45:15.353 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:45:15.354 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:45:15.354 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:45:15.354 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:45:15.354 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:45:15.354 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:45:15.356 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:45:15.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:15.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:45:15.420 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:45:15.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:45:15.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:45:15.517 INFO:teuthology.orchestra.run.smithi040.stdout:client8770 2024-04-26T21:45:15.517 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:15.517 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:15.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:15.533 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-26T21:45:15.576 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:45:15.590 INFO:teuthology.orchestra.run.smithi040.stdout:.: 2024-04-26T21:45:15.590 INFO:teuthology.orchestra.run.smithi040.stdout:subdir 2024-04-26T21:45:15.591 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:15.591 INFO:teuthology.orchestra.run.smithi040.stdout:./subdir: 2024-04-26T21:45:15.591 INFO:teuthology.orchestra.run.smithi040.stdout:sixmegs 2024-04-26T21:45:15.591 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 500.00000000' did not caused DAMAGED state 2024-04-26T21:45:15.591 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds_snaptable' 2024-04-26T21:45:15.591 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:15.591 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:15.646 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:15.646 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:15.702 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:15.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.843+0000 7fde46779640 1 Processor -- start 2024-04-26T21:45:15.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.843+0000 7fde46779640 1 -- start start 2024-04-26T21:45:15.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde46779640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fde4006f790 0x7fde4006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:15.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde46779640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde40156840 con 0x7fde40156460 2024-04-26T21:45:15.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde46779640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde400cbf20 con 0x7fde40072c50 2024-04-26T21:45:15.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde46779640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fde400cc200 con 0x7fde4006f790 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fde40156460 0x7fde400cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34266/0 (socket says 172.21.15.40:34266) 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3ffff640 1 -- 172.21.15.40:0/346612768 learned_addr learned my addr 172.21.15.40:0/346612768 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3f7fe640 1 --2- 172.21.15.40:0/346612768 >> v2:172.21.15.184:3300/0 conn(0x7fde4006f790 0x7fde4006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3634257681 0 0) 0x7fde40156840 con 0x7fde40156460 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde28003520 con 0x7fde40156460 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 783167737 0 0) 0x7fde28003520 con 0x7fde40156460 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 >> v2:172.21.15.184:3300/0 conn(0x7fde4006f790 msgr2=0x7fde4006fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 --2- 172.21.15.40:0/346612768 >> v2:172.21.15.184:3300/0 conn(0x7fde4006f790 0x7fde4006fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 >> v1:172.21.15.184:6789/0 conn(0x7fde40072c50 legacy=0x7fde40073060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde400cc4e0 con 0x7fde40156460 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fde30001670 con 0x7fde40156460 2024-04-26T21:45:15.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 >> v1:172.21.15.40:6789/0 conn(0x7fde40156460 legacy=0x7fde400cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280039b0 0x7fde28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde28004800 0x7fde28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fde400cc200 con 0x7fde280051c0 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fde2800c060 con 0x7fde28004800 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.844+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fde2800c260 con 0x7fde280039b0 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde44d16640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde3ffff640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280039b0 0x7fde28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde3f7fe640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde28004800 0x7fde28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde44d16640 1 -- 172.21.15.40:0/346612768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280039b0 msgr2=0x7fde28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde44d16640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280039b0 0x7fde28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde44d16640 1 -- 172.21.15.40:0/346612768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde28004800 msgr2=0x7fde28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde44d16640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde28004800 0x7fde28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:15.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde44d16640 1 -- 172.21.15.40:0/346612768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde28003520 con 0x7fde280051c0 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.845+0000 7fde44d16640 1 -- 172.21.15.40:0/346612768 reap_dead start 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde44d16640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fde38001ee0 tx=0x7fde28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bbff7b9a17dd79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38035070 con 0x7fde280051c0 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde38003bb0 con 0x7fde280051c0 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde3effd640 1 -- 172.21.15.40:0/346612768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38001b00 con 0x7fde280051c0 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde46779640 1 -- 172.21.15.40:0/346612768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde280051c0 msgr2=0x7fde2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde46779640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fde38001ee0 tx=0x7fde28005830 comp rx=0 tx=0).stop 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde46779640 1 -- 172.21.15.40:0/346612768 shutdown_connections 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde46779640 1 --2- 172.21.15.40:0/346612768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde46779640 1 -- 172.21.15.40:0/346612768 >> 172.21.15.40:0/346612768 conn(0x7fde400d0be0 msgr2=0x7fde400d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde46779640 1 -- 172.21.15.40:0/346612768 shutdown_connections 2024-04-26T21:45:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.846+0000 7fde46779640 1 -- 172.21.15.40:0/346612768 wait complete. 2024-04-26T21:45:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.847+0000 7fde46779640 1 Processor -- start 2024-04-26T21:45:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde46779640 1 -- start start 2024-04-26T21:45:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde46779640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280051c0 0x7fde400c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde46779640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde40152340 0x7fde4014e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde46779640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde400c73a0 0x7fde40073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde46779640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fde4006e260 con 0x7fde40152340 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde46779640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fde40072700 con 0x7fde400c73a0 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde46779640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fde4006e3e0 con 0x7fde280051c0 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde3f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde40152340 0x7fde4014e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde3f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde40152340 0x7fde4014e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38736/0 (socket says 172.21.15.40:38736) 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.848+0000 7fde3f7fe640 1 -- 172.21.15.40:0/4010718998 learned_addr learned my addr 172.21.15.40:0/4010718998 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde44d16640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde400c73a0 0x7fde40073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3ffff640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280051c0 0x7fde400c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3f7fe640 1 -- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280051c0 msgr2=0x7fde400c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3f7fe640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280051c0 0x7fde400c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3f7fe640 1 -- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde400c73a0 msgr2=0x7fde40073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3f7fe640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde400c73a0 0x7fde40073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3f7fe640 1 -- 172.21.15.40:0/4010718998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde40152820 con 0x7fde40152340 2024-04-26T21:45:15.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde44d16640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde400c73a0 0x7fde40073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3f7fe640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde40152340 0x7fde4014e050 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fde340118e0 tx=0x7fde34011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3cff9640 1 -- 172.21.15.40:0/4010718998 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3400fa90 con 0x7fde40152340 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3cff9640 1 -- 172.21.15.40:0/4010718998 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3400fc30 con 0x7fde40152340 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde40152b00 con 0x7fde40152340 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde3cff9640 1 -- 172.21.15.40:0/4010718998 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34014440 con 0x7fde40152340 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.849+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde401530c0 con 0x7fde40152340 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.850+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde400694f0 con 0x7fde40152340 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.850+0000 7fde3cff9640 1 -- 172.21.15.40:0/4010718998 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde34012040 con 0x7fde40152340 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.851+0000 7fde3cff9640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde2c06c920 0x7fde2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.851+0000 7fde3cff9640 1 -- 172.21.15.40:0/4010718998 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7fde3408cc50 con 0x7fde40152340 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.851+0000 7fde3ffff640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde2c06c920 0x7fde2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:15.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.852+0000 7fde3ffff640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde2c06c920 0x7fde2c06ede0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fde30001790 tx=0x7fde30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:15.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:15.855+0000 7fde3cff9640 1 -- 172.21.15.40:0/4010718998 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde34092020 con 0x7fde40152340 2024-04-26T21:45:16.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.041+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde4006de80 con 0x7fde40152340 2024-04-26T21:45:16.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.042+0000 7fde3cff9640 1 -- 172.21.15.40:0/4010718998 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+20412 (secure 0 0 0) 0x7fde34019030 con 0x7fde40152340 2024-04-26T21:45:16.043 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:16.043 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":121,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:45:03.736965+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde2c06c920 msgr2=0x7fde2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde2c06c920 0x7fde2c06ede0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fde30001790 tx=0x7fde30009040 comp rx=0 tx=0).stop 2024-04-26T21:45:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde40152340 msgr2=0x7fde4014e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde40152340 0x7fde4014e050 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fde340118e0 tx=0x7fde34011da0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 shutdown_connections 2024-04-26T21:45:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fde2c06c920 0x7fde2c06ede0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde400c73a0 0x7fde40073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde40152340 0x7fde4014e050 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 --2- 172.21.15.40:0/4010718998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde280051c0 0x7fde400c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 >> 172.21.15.40:0/4010718998 conn(0x7fde400d0be0 msgr2=0x7fde4015e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:16.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 shutdown_connections 2024-04-26T21:45:16.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.044+0000 7fde46779640 1 -- 172.21.15.40:0/4010718998 wait complete. 2024-04-26T21:45:16.055 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:45:16.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:45:16.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.159+0000 7f25fef5f640 1 Processor -- start 2024-04-26T21:45:16.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.159+0000 7f25fef5f640 1 -- start start 2024-04-26T21:45:16.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.159+0000 7f25fef5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f815f2d0 0x7f25f81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fef5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25f806fea0 0x7f25f80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fef5f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f8072f70 con 0x7f25f8072700 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fef5f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f25f8074010 con 0x7f25f806fea0 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fef5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25f8073250 con 0x7f25f815f2d0 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fd4fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25f806fea0 0x7f25f80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fd4fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25f806fea0 0x7f25f80ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38750/0 (socket says 172.21.15.40:38750) 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fd4fc640 1 -- 172.21.15.40:0/189881141 learned_addr learned my addr 172.21.15.40:0/189881141 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fccfb640 1 --2- 172.21.15.40:0/189881141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f815f2d0 0x7f25f81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fd4fc640 1 -- 172.21.15.40:0/189881141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f815f2d0 msgr2=0x7f25f81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fd4fc640 1 --2- 172.21.15.40:0/189881141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f815f2d0 0x7f25f81604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fd4fc640 1 -- 172.21.15.40:0/189881141 >> v1:172.21.15.184:6789/0 conn(0x7f25f8072700 legacy=0x7f25f806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.160+0000 7f25fd4fc640 1 -- 172.21.15.40:0/189881141 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f8073530 con 0x7f25f806fea0 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25fd4fc640 1 --2- 172.21.15.40:0/189881141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f80ce7b0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009e50 tx=0x7f25f002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967ef4b677ee862b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25f77fe640 1 -- 172.21.15.40:0/189881141 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0036070 con 0x7f25f806fea0 2024-04-26T21:45:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25f77fe640 1 -- 172.21.15.40:0/189881141 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f0031600 con 0x7f25f806fea0 2024-04-26T21:45:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25f77fe640 1 -- 172.21.15.40:0/189881141 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00319d0 con 0x7f25f806fea0 2024-04-26T21:45:16.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25fef5f640 1 -- 172.21.15.40:0/189881141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 msgr2=0x7f25f80ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25fef5f640 1 --2- 172.21.15.40:0/189881141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f80ce7b0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009e50 tx=0x7f25f002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25fef5f640 1 -- 172.21.15.40:0/189881141 shutdown_connections 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25fef5f640 1 --2- 172.21.15.40:0/189881141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f80ce7b0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25fef5f640 1 --2- 172.21.15.40:0/189881141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f815f2d0 0x7f25f81604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25fef5f640 1 -- 172.21.15.40:0/189881141 >> 172.21.15.40:0/189881141 conn(0x7f25f80d0be0 msgr2=0x7f25f80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.161+0000 7f25fef5f640 1 -- 172.21.15.40:0/189881141 shutdown_connections 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.162+0000 7f25fef5f640 1 -- 172.21.15.40:0/189881141 wait complete. 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.163+0000 7f25fef5f640 1 Processor -- start 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.163+0000 7f25fef5f640 1 -- start start 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.163+0000 7f25fef5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f81543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.163+0000 7f25fef5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f8072700 0x7f25f8154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.163+0000 7f25fef5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25f815f2d0 0x7f25f81f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fef5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25f815f6b0 con 0x7f25f806fea0 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fef5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8073af0 con 0x7f25f815f2d0 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fef5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25f8072b60 con 0x7f25f8072700 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f81543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f81543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38760/0 (socket says 172.21.15.40:38760) 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 -- 172.21.15.40:0/2031603131 learned_addr learned my addr 172.21.15.40:0/2031603131 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25f7fff640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f8072700 0x7f25f8154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fd4fc640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25f815f2d0 0x7f25f81f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 -- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f8072700 msgr2=0x7f25f8154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f8072700 0x7f25f8154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 -- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25f815f2d0 msgr2=0x7f25f81f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25f815f2d0 0x7f25f81f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 -- 172.21.15.40:0/2031603131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f8154e40 con 0x7f25f806fea0 2024-04-26T21:45:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25f7fff640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f8072700 0x7f25f8154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:16.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fd4fc640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25f815f2d0 0x7f25f81f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:16.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fccfb640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f81543c0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00f9f0 tx=0x7f25ec00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25f57fa640 1 -- 172.21.15.40:0/2031603131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec007c30 con 0x7f25f806fea0 2024-04-26T21:45:16.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25f57fa640 1 -- 172.21.15.40:0/2031603131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec007dd0 con 0x7f25f806fea0 2024-04-26T21:45:16.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25f57fa640 1 -- 172.21.15.40:0/2031603131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec011690 con 0x7f25f806fea0 2024-04-26T21:45:16.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f80cb640 con 0x7f25f806fea0 2024-04-26T21:45:16.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.164+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f81553f0 con 0x7f25f806fea0 2024-04-26T21:45:16.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.165+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f80694f0 con 0x7f25f806fea0 2024-04-26T21:45:16.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.166+0000 7f25f57fa640 1 -- 172.21.15.40:0/2031603131 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25ec019020 con 0x7f25f806fea0 2024-04-26T21:45:16.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.166+0000 7f25f57fa640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f25dc06c8f0 0x7f25dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.166+0000 7f25f57fa640 1 -- 172.21.15.40:0/2031603131 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f25ec08e790 con 0x7f25f806fea0 2024-04-26T21:45:16.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.166+0000 7f25f7fff640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f25dc06c8f0 0x7f25dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.167+0000 7f25f7fff640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f25dc06c8f0 0x7f25dc06edb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f25f80b2660 tx=0x7f25e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.171+0000 7f25f57fa640 1 -- 172.21.15.40:0/2031603131 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25ec05c140 con 0x7f25f806fea0 2024-04-26T21:45:16.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.354+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25f806de80 con 0x7f25f806fea0 2024-04-26T21:45:16.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.354+0000 7f25f57fa640 1 -- 172.21.15.40:0/2031603131 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f25ec05ba40 con 0x7f25f806fea0 2024-04-26T21:45:16.355 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:45:16.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f25dc06c8f0 msgr2=0x7f25dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f25dc06c8f0 0x7f25dc06edb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f25f80b2660 tx=0x7f25e4009040 comp rx=0 tx=0).stop 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 msgr2=0x7f25f81543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f81543c0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00f9f0 tx=0x7f25ec00feb0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 shutdown_connections 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f25dc06c8f0 0x7f25dc06edb0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25f815f2d0 0x7f25f81f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25f8072700 0x7f25f8154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 --2- 172.21.15.40:0/2031603131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f806fea0 0x7f25f81543c0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 >> 172.21.15.40:0/2031603131 conn(0x7f25f80d0be0 msgr2=0x7f25f815d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 shutdown_connections 2024-04-26T21:45:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.356+0000 7f25fef5f640 1 -- 172.21.15.40:0/2031603131 wait complete. 2024-04-26T21:45:16.368 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:45:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.471+0000 7f9b7f0bd640 1 Processor -- start 2024-04-26T21:45:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.471+0000 7f9b7f0bd640 1 -- start start 2024-04-26T21:45:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b7f0bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7815f170 0x7f9b780d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b7f0bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b780d1040 con 0x7f9b7815f550 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b7f0bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b780c9680 con 0x7f9b780d3bc0 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b7f0bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b780c9430 con 0x7f9b7815f170 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b77fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9b7815f550 0x7f9b780d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34268/0 (socket says 172.21.15.40:34268) 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b77fff640 1 -- 172.21.15.40:0/3333066888 learned_addr learned my addr 172.21.15.40:0/3333066888 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b7ce59640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7815f170 0x7f9b780d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441882388 0 0) 0x7f9b6c001110 con 0x7f9b7815f550 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.472+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b60003520 con 0x7f9b7815f550 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1603684345 0 0) 0x7f9b60003520 con 0x7f9b7815f550 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7815f170 msgr2=0x7f9b780d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7815f170 0x7f9b780d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 >> v1:172.21.15.184:6789/0 conn(0x7f9b780d3bc0 legacy=0x7f9b780ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b780c9960 con 0x7f9b7815f550 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9b6c001840 con 0x7f9b7815f550 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 >> v1:172.21.15.40:6789/0 conn(0x7f9b7815f550 legacy=0x7f9b780d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b600039e0 0x7f9b60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7815f170 0x7f9b60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9b780c9430 con 0x7f9b60004d40 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6000b9c0 con 0x7f9b600039e0 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6000bbc0 con 0x7f9b7815f170 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b7d65a640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b77fff640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7815f170 0x7f9b60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b7ce59640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b600039e0 0x7f9b60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b7d65a640 1 -- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7815f170 msgr2=0x7f9b60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b7d65a640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7815f170 0x7f9b60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b7d65a640 1 -- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b600039e0 msgr2=0x7f9b60003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b7d65a640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b600039e0 0x7f9b60003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.473+0000 7f9b7d65a640 1 -- 172.21.15.40:0/3333066888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b60003520 con 0x7f9b60004d40 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.474+0000 7f9b7d65a640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b600051c0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f9b68001860 tx=0x7f9b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00c33f62220a8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.474+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68037070 con 0x7f9b60004d40 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.474+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6802abf0 con 0x7f9b60004d40 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.474+0000 7f9b777fe640 1 -- 172.21.15.40:0/3333066888 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68001430 con 0x7f9b60004d40 2024-04-26T21:45:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.474+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/3333066888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 msgr2=0x7f9b600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.474+0000 7f9b7f0bd640 1 --2- 172.21.15.40:0/3333066888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b600051c0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f9b68001860 tx=0x7f9b60005830 comp rx=0 tx=0).stop 2024-04-26T21:45:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.475+0000 7f9b7d65a640 1 -- 172.21.15.40:0/3333066888 reap_dead start 2024-04-26T21:45:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.475+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/3333066888 shutdown_connections 2024-04-26T21:45:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.475+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/3333066888 >> 172.21.15.40:0/3333066888 conn(0x7f9b780d4f60 msgr2=0x7f9b780d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.475+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/3333066888 shutdown_connections 2024-04-26T21:45:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.475+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/3333066888 wait complete. 2024-04-26T21:45:16.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.476+0000 7f9b7f0bd640 1 Processor -- start 2024-04-26T21:45:16.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.476+0000 7f9b7f0bd640 1 -- start start 2024-04-26T21:45:16.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7f0bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b780b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7f0bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b78157f90 0x7f9b781583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7f0bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7809ec60 0x7f9b7809f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7f0bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9b780d5c00 con 0x7f9b60004d40 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7f0bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b780d5a80 con 0x7f9b78157f90 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7f0bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b780d5d80 con 0x7f9b7809ec60 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b780b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b780b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38782/0 (socket says 172.21.15.40:38782) 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 -- 172.21.15.40:0/2681742171 learned_addr learned my addr 172.21.15.40:0/2681742171 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7d65a640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7809ec60 0x7f9b7809f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b77fff640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b78157f90 0x7f9b781583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 -- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7809ec60 msgr2=0x7f9b7809f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7809ec60 0x7f9b7809f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 -- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b78157f90 msgr2=0x7f9b781583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b78157f90 0x7f9b781583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 -- 172.21.15.40:0/2681742171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7809f900 con 0x7f9b60004d40 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b77fff640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b78157f90 0x7f9b781583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7ce59640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b780b2b20 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009f90 tx=0x7f9b7000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b757fa640 1 -- 172.21.15.40:0/2681742171 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7000fea0 con 0x7f9b60004d40 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b757fa640 1 -- 172.21.15.40:0/2681742171 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b70015630 con 0x7f9b60004d40 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7809fbe0 con 0x7f9b60004d40 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b757fa640 1 -- 172.21.15.40:0/2681742171 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70015a20 con 0x7f9b60004d40 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.477+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b780a01a0 con 0x7f9b60004d40 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.478+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b780a05d0 con 0x7f9b60004d40 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.479+0000 7f9b757fa640 1 -- 172.21.15.40:0/2681742171 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b70015bc0 con 0x7f9b60004d40 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.479+0000 7f9b757fa640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b6406c920 0x7f9b6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.479+0000 7f9b757fa640 1 -- 172.21.15.40:0/2681742171 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f9b7008d7c0 con 0x7f9b60004d40 2024-04-26T21:45:16.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.480+0000 7f9b77fff640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b6406c920 0x7f9b6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.480+0000 7f9b77fff640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b6406c920 0x7f9b6406ede0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9b780b97a0 tx=0x7f9b6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.484+0000 7f9b757fa640 1 -- 172.21.15.40:0/2681742171 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b70019030 con 0x7f9b60004d40 2024-04-26T21:45:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.668+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9b780694f0 con 0x7f9b60004d40 2024-04-26T21:45:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.668+0000 7f9b757fa640 1 -- 172.21.15.40:0/2681742171 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7f9b70010040 con 0x7f9b60004d40 2024-04-26T21:45:16.669 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:45:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b6406c920 msgr2=0x7f9b6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b6406c920 0x7f9b6406ede0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9b780b97a0 tx=0x7f9b6c009040 comp rx=0 tx=0).stop 2024-04-26T21:45:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 msgr2=0x7f9b780b2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b780b2b20 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009f90 tx=0x7f9b7000f580 comp rx=0 tx=0).stop 2024-04-26T21:45:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 shutdown_connections 2024-04-26T21:45:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b6406c920 0x7f9b6406ede0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7809ec60 0x7f9b7809f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b78157f90 0x7f9b781583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 --2- 172.21.15.40:0/2681742171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b60004d40 0x7f9b780b2b20 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 >> 172.21.15.40:0/2681742171 conn(0x7f9b780d4f60 msgr2=0x7f9b780cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 shutdown_connections 2024-04-26T21:45:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.670+0000 7f9b7f0bd640 1 -- 172.21.15.40:0/2681742171 wait complete. 2024-04-26T21:45:16.682 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:45:16.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.785+0000 7f0df2209640 1 Processor -- start 2024-04-26T21:45:16.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.785+0000 7f0df2209640 1 -- start start 2024-04-26T21:45:16.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0df2209640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0dec0b40d0 0x7f0dec1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0df2209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec162b30 0x7f0dec0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0df2209640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dec0d0a10 con 0x7f0dec0d1a00 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0df2209640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0dec0d0cf0 con 0x7f0dec0b40d0 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0df2209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0dec0d0fd0 con 0x7f0dec162b30 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0deaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0dec0b40d0 0x7f0dec1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0deb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0dec0d1a00 0x7f0dec0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34276/0 (socket says 172.21.15.40:34276) 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0deb7fe640 1 -- 172.21.15.40:0/1647875080 learned_addr learned my addr 172.21.15.40:0/1647875080 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0deaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0dec0b40d0 0x7f0dec1625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38784/0 (socket says 172.21.15.40:38784) 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0debfff640 1 --2- 172.21.15.40:0/1647875080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec162b30 0x7f0dec0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0deaffd640 1 -- 172.21.15.40:0/1647875080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec162b30 msgr2=0x7f0dec0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0deaffd640 1 --2- 172.21.15.40:0/1647875080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec162b30 0x7f0dec0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0deaffd640 1 -- 172.21.15.40:0/1647875080 >> v1:172.21.15.40:6789/0 conn(0x7f0dec0d1a00 legacy=0x7f0dec0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0dea7fc640 1 -- 172.21.15.40:0/1647875080 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1167722216 0 0) 0x7f0dec0d0a10 con 0x7f0dec0d1a00 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.786+0000 7f0deaffd640 1 -- 172.21.15.40:0/1647875080 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec162f80 con 0x7f0dec0b40d0 2024-04-26T21:45:16.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.787+0000 7f0deaffd640 1 --2- 172.21.15.40:0/1647875080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec0b40d0 0x7f0dec1625f0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f0de00097a0 tx=0x7f0de002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ffa2812e0f4e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.791+0000 7f0dea7fc640 1 -- 172.21.15.40:0/1647875080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0037070 con 0x7f0dec0b40d0 2024-04-26T21:45:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.791+0000 7f0dea7fc640 1 -- 172.21.15.40:0/1647875080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de002ae50 con 0x7f0dec0b40d0 2024-04-26T21:45:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.791+0000 7f0dea7fc640 1 -- 172.21.15.40:0/1647875080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0032660 con 0x7f0dec0b40d0 2024-04-26T21:45:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.792+0000 7f0df2209640 1 -- 172.21.15.40:0/1647875080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec0b40d0 msgr2=0x7f0dec1625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.792+0000 7f0df2209640 1 --2- 172.21.15.40:0/1647875080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec0b40d0 0x7f0dec1625f0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f0de00097a0 tx=0x7f0de002a100 comp rx=0 tx=0).stop 2024-04-26T21:45:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.792+0000 7f0df2209640 1 -- 172.21.15.40:0/1647875080 shutdown_connections 2024-04-26T21:45:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.792+0000 7f0df2209640 1 --2- 172.21.15.40:0/1647875080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec162b30 0x7f0dec0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.792+0000 7f0df2209640 1 --2- 172.21.15.40:0/1647875080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec0b40d0 0x7f0dec1625f0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.792+0000 7f0df2209640 1 -- 172.21.15.40:0/1647875080 >> 172.21.15.40:0/1647875080 conn(0x7f0dec0d4fd0 msgr2=0x7f0dec0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.792+0000 7f0df2209640 1 -- 172.21.15.40:0/1647875080 shutdown_connections 2024-04-26T21:45:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.792+0000 7f0df2209640 1 -- 172.21.15.40:0/1647875080 wait complete. 2024-04-26T21:45:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.793+0000 7f0df2209640 1 Processor -- start 2024-04-26T21:45:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.793+0000 7f0df2209640 1 -- start start 2024-04-26T21:45:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0df2209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec0b40d0 0x7f0dec151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0df2209640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dec0d1a00 0x7f0dec0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0df2209640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec162b30 0x7f0dec0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0df2209640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec0d2670 con 0x7f0dec162b30 2024-04-26T21:45:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0df2209640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec0d24f0 con 0x7f0dec0d1a00 2024-04-26T21:45:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0df2209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0dec0b5040 con 0x7f0dec0b40d0 2024-04-26T21:45:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0debfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec162b30 0x7f0dec0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0debfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec162b30 0x7f0dec0b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38788/0 (socket says 172.21.15.40:38788) 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0debfff640 1 -- 172.21.15.40:0/391038842 learned_addr learned my addr 172.21.15.40:0/391038842 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0deaffd640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dec0d1a00 0x7f0dec0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.794+0000 7f0deb7fe640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec0b40d0 0x7f0dec151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0debfff640 1 -- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec0b40d0 msgr2=0x7f0dec151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0debfff640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec0b40d0 0x7f0dec151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0debfff640 1 -- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dec0d1a00 msgr2=0x7f0dec0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0debfff640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dec0d1a00 0x7f0dec0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0debfff640 1 -- 172.21.15.40:0/391038842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec0c9730 con 0x7f0dec162b30 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0debfff640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec162b30 0x7f0dec0b58d0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009d30 tx=0x7f0de400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0dd3fff640 1 -- 172.21.15.40:0/391038842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4018020 con 0x7f0dec162b30 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0dd3fff640 1 -- 172.21.15.40:0/391038842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de4014480 con 0x7f0dec162b30 2024-04-26T21:45:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dec0cc2f0 con 0x7f0dec162b30 2024-04-26T21:45:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0dd3fff640 1 -- 172.21.15.40:0/391038842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4014870 con 0x7f0dec162b30 2024-04-26T21:45:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.795+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dec0cc8b0 con 0x7f0dec162b30 2024-04-26T21:45:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.796+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dec0694f0 con 0x7f0dec162b30 2024-04-26T21:45:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.796+0000 7f0dd3fff640 1 -- 172.21.15.40:0/391038842 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0de4005d10 con 0x7f0dec162b30 2024-04-26T21:45:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.797+0000 7f0dd3fff640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dd806c8f0 0x7f0dd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.797+0000 7f0dd3fff640 1 -- 172.21.15.40:0/391038842 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f0de408d060 con 0x7f0dec162b30 2024-04-26T21:45:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.797+0000 7f0deb7fe640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dd806c8f0 0x7f0dd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.798+0000 7f0deb7fe640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dd806c8f0 0x7f0dd806edb0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc000f60 tx=0x7f0ddc007b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.801+0000 7f0dd3fff640 1 -- 172.21.15.40:0/391038842 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0de4022070 con 0x7f0dec162b30 2024-04-26T21:45:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.983+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0dec0b5f60 con 0x7f0dec162b30 2024-04-26T21:45:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.984+0000 7f0dd3fff640 1 -- 172.21.15.40:0/391038842 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7f0dec0c9730 con 0x7f0dec162b30 2024-04-26T21:45:16.985 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:45:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dd806c8f0 msgr2=0x7f0dd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dd806c8f0 0x7f0dd806edb0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc000f60 tx=0x7f0ddc007b50 comp rx=0 tx=0).stop 2024-04-26T21:45:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec162b30 msgr2=0x7f0dec0b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec162b30 0x7f0dec0b58d0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009d30 tx=0x7f0de400c490 comp rx=0 tx=0).stop 2024-04-26T21:45:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 shutdown_connections 2024-04-26T21:45:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0dd806c8f0 0x7f0dd806edb0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec162b30 0x7f0dec0b58d0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dec0d1a00 0x7f0dec0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 --2- 172.21.15.40:0/391038842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dec0b40d0 0x7f0dec151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:16.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 >> 172.21.15.40:0/391038842 conn(0x7f0dec0d4fd0 msgr2=0x7f0dec0d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:16.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 shutdown_connections 2024-04-26T21:45:16.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:16.986+0000 7f0df2209640 1 -- 172.21.15.40:0/391038842 wait complete. 2024-04-26T21:45:16.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:45:17.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.098+0000 7f3928af8640 1 Processor -- start 2024-04-26T21:45:17.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.098+0000 7f3928af8640 1 -- start start 2024-04-26T21:45:17.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.099+0000 7f3928af8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f392415f3a0 0x7f39240d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.099+0000 7f3928af8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39240d0e20 con 0x7f39240d2330 2024-04-26T21:45:17.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.099+0000 7f3928af8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39240b1d70 con 0x7f39240d1770 2024-04-26T21:45:17.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.099+0000 7f3928af8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f39240b2050 con 0x7f392415f3a0 2024-04-26T21:45:17.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.099+0000 7f3922d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f39240d2330 0x7f39240b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34286/0 (socket says 172.21.15.40:34286) 2024-04-26T21:45:17.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.099+0000 7f3922d9d640 1 -- 172.21.15.40:0/2508239429 learned_addr learned my addr 172.21.15.40:0/2508239429 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:17.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392259c640 1 --2- 172.21.15.40:0/2508239429 >> v2:172.21.15.184:3300/0 conn(0x7f392415f3a0 0x7f39240d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 578846591 0 0) 0x7f39240d0e20 con 0x7f39240d2330 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f390c003520 con 0x7f39240d2330 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4226033435 0 0) 0x7f390c003520 con 0x7f39240d2330 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 >> v2:172.21.15.184:3300/0 conn(0x7f392415f3a0 msgr2=0x7f39240d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 --2- 172.21.15.40:0/2508239429 >> v2:172.21.15.184:3300/0 conn(0x7f392415f3a0 0x7f39240d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 >> v1:172.21.15.184:6789/0 conn(0x7f39240d1770 legacy=0x7f39240d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240b2330 con 0x7f39240d2330 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f3914001a10 con 0x7f39240d2330 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 >> v1:172.21.15.40:6789/0 conn(0x7f39240d2330 legacy=0x7f39240b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c0039b0 0x7f390c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390c004800 0x7f390c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f390c0051c0 0x7f390c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39240b2050 con 0x7f390c004800 2024-04-26T21:45:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f390c00c060 con 0x7f390c0039b0 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f390c00c260 con 0x7f390c0051c0 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3921d9b640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390c004800 0x7f390c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f392259c640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c0039b0 0x7f390c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3922d9d640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f390c0051c0 0x7f390c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3921d9b640 1 -- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f390c0051c0 msgr2=0x7f390c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3921d9b640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f390c0051c0 0x7f390c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3921d9b640 1 -- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c0039b0 msgr2=0x7f390c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3921d9b640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c0039b0 0x7f390c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3921d9b640 1 -- 172.21.15.40:0/2508239429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c003520 con 0x7f390c004800 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3922d9d640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f390c0051c0 0x7f390c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.100+0000 7f3922d9d640 1 -- 172.21.15.40:0/2508239429 reap_dead start 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.101+0000 7f3921d9b640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390c004800 0x7f390c004c80 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f3918001750 tx=0x7f390c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ef3c48ad033ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.101+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918001370 con 0x7f390c004800 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.101+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39180052d0 con 0x7f390c004800 2024-04-26T21:45:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.101+0000 7f392159a640 1 -- 172.21.15.40:0/2508239429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391802abc0 con 0x7f390c004800 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.102+0000 7f3928af8640 1 -- 172.21.15.40:0/2508239429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390c004800 msgr2=0x7f390c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.102+0000 7f3928af8640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390c004800 0x7f390c004c80 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f3918001750 tx=0x7f390c005830 comp rx=0 tx=0).stop 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.102+0000 7f3928af8640 1 -- 172.21.15.40:0/2508239429 shutdown_connections 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.102+0000 7f3928af8640 1 --2- 172.21.15.40:0/2508239429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390c004800 0x7f390c004c80 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.102+0000 7f3928af8640 1 -- 172.21.15.40:0/2508239429 >> 172.21.15.40:0/2508239429 conn(0x7f39240d4f90 msgr2=0x7f39240d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.102+0000 7f3928af8640 1 -- 172.21.15.40:0/2508239429 shutdown_connections 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.102+0000 7f3928af8640 1 -- 172.21.15.40:0/2508239429 wait complete. 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.103+0000 7f3928af8640 1 Processor -- start 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.103+0000 7f3928af8640 1 -- start start 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3928af8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c004800 0x7f3924159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3928af8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39240cdd60 0x7f39241f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3928af8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39241f8510 0x7f39240b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3928af8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39240d0870 con 0x7f39240cdd60 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3928af8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39240d06f0 con 0x7f390c004800 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3928af8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39240d09f0 con 0x7f39241f8510 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3921d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39240cdd60 0x7f39241f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3921d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39240cdd60 0x7f39241f58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38814/0 (socket says 172.21.15.40:38814) 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3921d9b640 1 -- 172.21.15.40:0/2820411957 learned_addr learned my addr 172.21.15.40:0/2820411957 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3922d9d640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39241f8510 0x7f39240b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f392259c640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c004800 0x7f3924159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3921d9b640 1 -- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39241f8510 msgr2=0x7f39240b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3921d9b640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39241f8510 0x7f39240b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3921d9b640 1 -- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c004800 msgr2=0x7f3924159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3921d9b640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c004800 0x7f3924159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.104+0000 7f3921d9b640 1 -- 172.21.15.40:0/2820411957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39241f8e80 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.105+0000 7f3921d9b640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39240cdd60 0x7f39241f58c0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f3918000c00 tx=0x7f3918001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.105+0000 7f3912ffd640 1 -- 172.21.15.40:0/2820411957 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391803f030 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.105+0000 7f3912ffd640 1 -- 172.21.15.40:0/2820411957 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3918005480 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.105+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39241f9160 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.105+0000 7f3912ffd640 1 -- 172.21.15.40:0/2820411957 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918037040 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.105+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39241f53a0 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.106+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39241f4ec0 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.106+0000 7f3912ffd640 1 -- 172.21.15.40:0/2820411957 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3918042070 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.107+0000 7f3912ffd640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406c8f0 0x7f390406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.107+0000 7f3912ffd640 1 -- 172.21.15.40:0/2820411957 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7f39180abc90 con 0x7f39240cdd60 2024-04-26T21:45:17.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.107+0000 7f392259c640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406c8f0 0x7f390406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.108+0000 7f392259c640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406c8f0 0x7f390406edb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f391c009b80 tx=0x7f391c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.111+0000 7f3912ffd640 1 -- 172.21.15.40:0/2820411957 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3918079520 con 0x7f39240cdd60 2024-04-26T21:45:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.309+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f39240694f0 con 0x7f39240cdd60 2024-04-26T21:45:17.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.309+0000 7f3912ffd640 1 -- 172.21.15.40:0/2820411957 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 52 entries v121) ==== 81+0+3162 (secure 0 0 0) 0x7f3918078e20 con 0x7f39240cdd60 2024-04-26T21:45:17.310 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:45:17.310 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:45:17.310 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:45:17.310 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:45:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:45:17.312 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:45:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:45:17.314 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:45:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:listed 52 entries 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406c8f0 msgr2=0x7f390406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406c8f0 0x7f390406edb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f391c009b80 tx=0x7f391c008040 comp rx=0 tx=0).stop 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39240cdd60 msgr2=0x7f39241f58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39240cdd60 0x7f39241f58c0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f3918000c00 tx=0x7f3918001120 comp rx=0 tx=0).stop 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 shutdown_connections 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f390406c8f0 0x7f390406edb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39241f8510 0x7f39240b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39240cdd60 0x7f39241f58c0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 --2- 172.21.15.40:0/2820411957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f390c004800 0x7f3924159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 >> 172.21.15.40:0/2820411957 conn(0x7f39240d4f90 msgr2=0x7f39240b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 shutdown_connections 2024-04-26T21:45:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.311+0000 7f3928af8640 1 -- 172.21.15.40:0/2820411957 wait complete. 2024-04-26T21:45:17.323 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:45:17.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:17.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:45:17.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:17.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:17.356 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:17.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.552+0000 7fdf31020640 1 Processor -- start 2024-04-26T21:45:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.552+0000 7fdf31020640 1 -- start start 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf31020640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdf2c162810 0x7fdf2c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf31020640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf2c0b2340 0x7fdf2c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf31020640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c0cd930 0x7fdf2c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf31020640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdf2c0d0750 con 0x7fdf2c0b2340 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf31020640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdf2c0d16a0 con 0x7fdf2c162810 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf31020640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2c0d10d0 con 0x7fdf2c0cd930 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf23fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf2c0b2340 0x7fdf2c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf23fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf2c0b2340 0x7fdf2c0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38820/0 (socket says 172.21.15.40:38820) 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf23fff640 1 -- 172.21.15.40:0/4246980781 learned_addr learned my addr 172.21.15.40:0/4246980781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf2b59e640 1 --2- 172.21.15.40:0/4246980781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c0cd930 0x7fdf2c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf23fff640 1 -- 172.21.15.40:0/4246980781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c0cd930 msgr2=0x7fdf2c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf23fff640 1 --2- 172.21.15.40:0/4246980781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c0cd930 0x7fdf2c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf2ad9d640 1 --2- 172.21.15.40:0/4246980781 >> v2:172.21.15.184:3300/0 conn(0x7fdf2c162810 0x7fdf2c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf23fff640 1 -- 172.21.15.40:0/4246980781 >> v2:172.21.15.184:3300/0 conn(0x7fdf2c162810 msgr2=0x7fdf2c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf23fff640 1 --2- 172.21.15.40:0/4246980781 >> v2:172.21.15.184:3300/0 conn(0x7fdf2c162810 0x7fdf2c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf23fff640 1 -- 172.21.15.40:0/4246980781 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c0d1370 con 0x7fdf2c0b2340 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.553+0000 7fdf2b59e640 1 --2- 172.21.15.40:0/4246980781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c0cd930 0x7fdf2c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.554+0000 7fdf23fff640 1 --2- 172.21.15.40:0/4246980781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0cd3f0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c073d90 tx=0x7fdf1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9556e5b2a2ab21d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.554+0000 7fdf2a59c640 1 -- 172.21.15.40:0/4246980781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14037070 con 0x7fdf2c0b2340 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.554+0000 7fdf2a59c640 1 -- 172.21.15.40:0/4246980781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf14032440 con 0x7fdf2c0b2340 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.554+0000 7fdf2a59c640 1 -- 172.21.15.40:0/4246980781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14032810 con 0x7fdf2c0b2340 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.554+0000 7fdf31020640 1 -- 172.21.15.40:0/4246980781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0b2340 msgr2=0x7fdf2c0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.554+0000 7fdf31020640 1 --2- 172.21.15.40:0/4246980781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0cd3f0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c073d90 tx=0x7fdf1402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.555+0000 7fdf31020640 1 -- 172.21.15.40:0/4246980781 shutdown_connections 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.555+0000 7fdf31020640 1 --2- 172.21.15.40:0/4246980781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c0cd930 0x7fdf2c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.555+0000 7fdf31020640 1 --2- 172.21.15.40:0/4246980781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0cd3f0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.555+0000 7fdf31020640 1 --2- 172.21.15.40:0/4246980781 >> v2:172.21.15.184:3300/0 conn(0x7fdf2c162810 0x7fdf2c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.555+0000 7fdf31020640 1 -- 172.21.15.40:0/4246980781 >> 172.21.15.40:0/4246980781 conn(0x7fdf2c0d2da0 msgr2=0x7fdf2c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:17.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.555+0000 7fdf31020640 1 -- 172.21.15.40:0/4246980781 shutdown_connections 2024-04-26T21:45:17.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.555+0000 7fdf31020640 1 -- 172.21.15.40:0/4246980781 wait complete. 2024-04-26T21:45:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.556+0000 7fdf31020640 1 Processor -- start 2024-04-26T21:45:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.556+0000 7fdf31020640 1 -- start start 2024-04-26T21:45:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf31020640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c14d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf31020640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0cd930 0x7fdf2c0c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf31020640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c162810 0x7fdf2c0c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf31020640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c070890 con 0x7fdf2c0cd930 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf31020640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c0d0470 con 0x7fdf2c0b2340 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf31020640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2c0d2070 con 0x7fdf2c162810 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0cd930 0x7fdf2c0c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0cd930 0x7fdf2c0c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38826/0 (socket says 172.21.15.40:38826) 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 -- 172.21.15.40:0/2778701463 learned_addr learned my addr 172.21.15.40:0/2778701463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf2b59e640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c162810 0x7fdf2c0c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf2ad9d640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c14d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 -- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c162810 msgr2=0x7fdf2c0c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c162810 0x7fdf2c0c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 -- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf2c0b2340 msgr2=0x7fdf2c14d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c14d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 -- 172.21.15.40:0/2778701463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c1f8900 con 0x7fdf2c0cd930 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf2ad9d640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c14d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf23fff640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0cd930 0x7fdf2c0c8520 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fdf14000c00 tx=0x7fdf140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf237fe640 1 -- 172.21.15.40:0/2778701463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14037040 con 0x7fdf2c0cd930 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf237fe640 1 -- 172.21.15.40:0/2778701463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf14042070 con 0x7fdf2c0cd930 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf237fe640 1 -- 172.21.15.40:0/2778701463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14030bb0 con 0x7fdf2c0cd930 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.557+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c1f8be0 con 0x7fdf2c0cd930 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.558+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c1f91a0 con 0x7fdf2c0cd930 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.559+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2c0694f0 con 0x7fdf2c0cd930 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.559+0000 7fdf237fe640 1 -- 172.21.15.40:0/2778701463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf14047460 con 0x7fdf2c0cd930 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.560+0000 7fdf237fe640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf0c06c8f0 0x7fdf0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.560+0000 7fdf237fe640 1 -- 172.21.15.40:0/2778701463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8453+0+0 (secure 0 0 0) 0x7fdf140ad3a0 con 0x7fdf2c0cd930 2024-04-26T21:45:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.560+0000 7fdf2ad9d640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf0c06c8f0 0x7fdf0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.560+0000 7fdf2ad9d640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf0c06c8f0 0x7fdf0c06edb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00b820 tx=0x7fdf1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.564+0000 7fdf237fe640 1 -- 172.21.15.40:0/2778701463 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf1407abd0 con 0x7fdf2c0cd930 2024-04-26T21:45:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.787+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdf2c06f340 con 0x7fdf2c0cd930 2024-04-26T21:45:17.788 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:17.787+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:45:17.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.789+0000 7fdf237fe640 1 -- 172.21.15.40:0/2778701463 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v258) ==== 161+0+0 (secure 0 0 0) 0x7fdf1407a4d0 con 0x7fdf2c0cd930 2024-04-26T21:45:17.789 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.791+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf0c06c8f0 msgr2=0x7fdf0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.791+0000 7fdf31020640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf0c06c8f0 0x7fdf0c06edb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00b820 tx=0x7fdf1c008040 comp rx=0 tx=0).stop 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.791+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0cd930 msgr2=0x7fdf2c0c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.791+0000 7fdf31020640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0cd930 0x7fdf2c0c8520 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fdf14000c00 tx=0x7fdf140046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.792+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 shutdown_connections 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.792+0000 7fdf31020640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdf0c06c8f0 0x7fdf0c06edb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.792+0000 7fdf31020640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf2c162810 0x7fdf2c0c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.792+0000 7fdf31020640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0cd930 0x7fdf2c0c8520 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.792+0000 7fdf31020640 1 --2- 172.21.15.40:0/2778701463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c14d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.792+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 >> 172.21.15.40:0/2778701463 conn(0x7fdf2c0d2da0 msgr2=0x7fdf2c070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.792+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 shutdown_connections 2024-04-26T21:45:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.792+0000 7fdf31020640 1 -- 172.21.15.40:0/2778701463 wait complete. 2024-04-26T21:45:17.794 INFO:tasks.ceph.mds.c.smithi040.stderr: -604> 2024-04-26T21:45:14.747+0000 7fb9c4ed5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:45:17.805 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:17.820 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:45:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.907+0000 7faa83178640 1 Processor -- start 2024-04-26T21:45:17.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.908+0000 7faa83178640 1 -- start start 2024-04-26T21:45:17.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.908+0000 7faa83178640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c15f3a0 0x7faa7c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.908+0000 7faa83178640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7c0d0e20 con 0x7faa7c0d2300 2024-04-26T21:45:17.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.908+0000 7faa83178640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7c0b1d70 con 0x7faa7c0d1770 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.908+0000 7faa83178640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c0b2050 con 0x7faa7c15f3a0 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa81715640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa7c0d2300 0x7faa7c0b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34314/0 (socket says 172.21.15.40:34314) 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa81715640 1 -- 172.21.15.40:0/2974431003 learned_addr learned my addr 172.21.15.40:0/2974431003 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa80f14640 1 --2- 172.21.15.40:0/2974431003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c15f3a0 0x7faa7c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa7b7fe640 1 -- 172.21.15.40:0/2974431003 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107103647 0 0) 0x7faa7c0d0e20 con 0x7faa7c0d2300 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa7b7fe640 1 -- 172.21.15.40:0/2974431003 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa68003520 con 0x7faa7c0d2300 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa7b7fe640 1 -- 172.21.15.40:0/2974431003 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463508343 0 0) 0x7faa7c0b1d70 con 0x7faa7c0d1770 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa7b7fe640 1 -- 172.21.15.40:0/2974431003 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa7c0d0e20 con 0x7faa7c0d1770 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa7b7fe640 1 -- 172.21.15.40:0/2974431003 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2591133584 0 0) 0x7faa68003520 con 0x7faa7c0d2300 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa80f14640 1 -- 172.21.15.40:0/2974431003 >> v1:172.21.15.184:6789/0 conn(0x7faa7c0d1770 legacy=0x7faa7c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa80f14640 1 -- 172.21.15.40:0/2974431003 >> v1:172.21.15.40:6789/0 conn(0x7faa7c0d2300 legacy=0x7faa7c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.909+0000 7faa80f14640 1 -- 172.21.15.40:0/2974431003 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c0b2330 con 0x7faa7c15f3a0 2024-04-26T21:45:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.910+0000 7faa80f14640 1 --2- 172.21.15.40:0/2974431003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c15f3a0 0x7faa7c0d06f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa74009dc0 tx=0x7faa7402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f15dec3ad50c43c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.910+0000 7faa7b7fe640 1 -- 172.21.15.40:0/2974431003 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64001670 con 0x7faa7c15f3a0 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.910+0000 7faa7b7fe640 1 -- 172.21.15.40:0/2974431003 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa74007540 con 0x7faa7c15f3a0 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.910+0000 7faa7b7fe640 1 -- 172.21.15.40:0/2974431003 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74007910 con 0x7faa7c15f3a0 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.911+0000 7faa83178640 1 -- 172.21.15.40:0/2974431003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c15f3a0 msgr2=0x7faa7c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.911+0000 7faa83178640 1 --2- 172.21.15.40:0/2974431003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c15f3a0 0x7faa7c0d06f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa74009dc0 tx=0x7faa7402a660 comp rx=0 tx=0).stop 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.911+0000 7faa83178640 1 -- 172.21.15.40:0/2974431003 shutdown_connections 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.911+0000 7faa83178640 1 --2- 172.21.15.40:0/2974431003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c15f3a0 0x7faa7c0d06f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.911+0000 7faa83178640 1 -- 172.21.15.40:0/2974431003 >> 172.21.15.40:0/2974431003 conn(0x7faa7c0d4f90 msgr2=0x7faa7c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.911+0000 7faa83178640 1 -- 172.21.15.40:0/2974431003 shutdown_connections 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.911+0000 7faa83178640 1 -- 172.21.15.40:0/2974431003 wait complete. 2024-04-26T21:45:17.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.912+0000 7faa83178640 1 Processor -- start 2024-04-26T21:45:17.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.912+0000 7faa83178640 1 -- start start 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa83178640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c0d1770 0x7faa7c1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa83178640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7c0d2300 0x7faa7c0cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa83178640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa7c15f3a0 0x7faa7c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa83178640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c0b30e0 con 0x7faa7c0d2300 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa83178640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c0b2f60 con 0x7faa7c15f3a0 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa83178640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c0b3260 con 0x7faa7c0d1770 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7c0d2300 0x7faa7c0cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7c0d2300 0x7faa7c0cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38854/0 (socket says 172.21.15.40:38854) 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 -- 172.21.15.40:0/122518890 learned_addr learned my addr 172.21.15.40:0/122518890 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa80f14640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c0d1770 0x7faa7c1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa81715640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa7c15f3a0 0x7faa7c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 -- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c0d1770 msgr2=0x7faa7c1532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c0d1770 0x7faa7c1532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 -- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa7c15f3a0 msgr2=0x7faa7c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa7c15f3a0 0x7faa7c0caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 -- 172.21.15.40:0/122518890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c0cc9d0 con 0x7faa7c0d2300 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa81715640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa7c15f3a0 0x7faa7c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.913+0000 7faa7bfff640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7c0d2300 0x7faa7c0cc060 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c490 tx=0x7faa6c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.914+0000 7faa797fa640 1 -- 172.21.15.40:0/122518890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0052d0 con 0x7faa7c0d2300 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.914+0000 7faa797fa640 1 -- 172.21.15.40:0/122518890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6c005470 con 0x7faa7c0d2300 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.914+0000 7faa797fa640 1 -- 172.21.15.40:0/122518890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c012850 con 0x7faa7c0d2300 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.914+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c0cccb0 con 0x7faa7c0d2300 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.914+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c0cd270 con 0x7faa7c0d2300 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.915+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7c0694f0 con 0x7faa7c0d2300 2024-04-26T21:45:17.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.915+0000 7faa797fa640 1 -- 172.21.15.40:0/122518890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa6c0129f0 con 0x7faa7c0d2300 2024-04-26T21:45:17.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.916+0000 7faa797fa640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa5c06c8f0 0x7faa5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:17.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.916+0000 7faa797fa640 1 -- 172.21.15.40:0/122518890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8539+0+0 (secure 0 0 0) 0x7faa6c08c2e0 con 0x7faa7c0d2300 2024-04-26T21:45:17.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.916+0000 7faa80f14640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa5c06c8f0 0x7faa5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:17.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.917+0000 7faa80f14640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa5c06c8f0 0x7faa5c06edb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faa74034000 tx=0x7faa7402f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:17.920+0000 7faa797fa640 1 -- 172.21.15.40:0/122518890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa6c059bb0 con 0x7faa7c0d2300 2024-04-26T21:45:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.179+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7faa7c0b38f0 con 0x7faa7c0d2300 2024-04-26T21:45:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.799+0000 7faa797fa640 1 -- 172.21.15.40:0/122518890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v259) ==== 107+0+0 (secure 0 0 0) 0x7faa6c0594b0 con 0x7faa7c0d2300 2024-04-26T21:45:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa5c06c8f0 msgr2=0x7faa5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa5c06c8f0 0x7faa5c06edb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faa74034000 tx=0x7faa7402f000 comp rx=0 tx=0).stop 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7c0d2300 msgr2=0x7faa7c0cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7c0d2300 0x7faa7c0cc060 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c490 tx=0x7faa6c00c950 comp rx=0 tx=0).stop 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 shutdown_connections 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa5c06c8f0 0x7faa5c06edb0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa7c15f3a0 0x7faa7c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7c0d2300 0x7faa7c0cc060 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 --2- 172.21.15.40:0/122518890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa7c0d1770 0x7faa7c1532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 >> 172.21.15.40:0/122518890 conn(0x7faa7c0d4f90 msgr2=0x7faa7c0b60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 shutdown_connections 2024-04-26T21:45:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.802+0000 7faa83178640 1 -- 172.21.15.40:0/122518890 wait complete. 2024-04-26T21:45:18.816 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.902+0000 7f9365a0bf40 1 Processor -- start 2024-04-26T21:45:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.902+0000 7f9365a0bf40 1 -- start start 2024-04-26T21:45:18.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9365a0bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a4b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9365a0bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 0x55e3e2980d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9365a0bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 0x55e3e2981290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9365a0bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e3e2929e40 con 0x7f934c0044e0 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9365a0bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e3e292de80 con 0x55e3e2a55ac0 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9365a0bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e3e29b2690 con 0x55e3e2a4bd90 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a4b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a4b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45244/0 (socket says 172.21.15.40:45244) 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 -- 172.21.15.40:0/2086305789 learned_addr learned my addr 172.21.15.40:0/2086305789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f936307a640 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 0x55e3e2981290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362078640 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 0x55e3e2980d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 -- 172.21.15.40:0/2086305789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 msgr2=0x55e3e2981290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 0x55e3e2981290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 -- 172.21.15.40:0/2086305789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 msgr2=0x55e3e2980d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:18.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 0x55e3e2980d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 -- 172.21.15.40:0/2086305789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e2a57a40 con 0x7f934c0044e0 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9362879640 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a4b4d0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f9354025840 tx=0x7f9354051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70064e911c08a411 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9361076640 1 -- 172.21.15.40:0/2086305789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354058040 con 0x7f934c0044e0 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9361076640 1 -- 172.21.15.40:0/2086305789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9354006020 con 0x7f934c0044e0 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.903+0000 7f9361076640 1 -- 172.21.15.40:0/2086305789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354053720 con 0x7f934c0044e0 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 -- 172.21.15.40:0/2086305789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 msgr2=0x55e3e2a4b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a4b4d0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f9354025840 tx=0x7f9354051000 comp rx=0 tx=0).stop 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 -- 172.21.15.40:0/2086305789 shutdown_connections 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 0x55e3e2981290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 0x55e3e2980d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/2086305789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a4b4d0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 -- 172.21.15.40:0/2086305789 >> 172.21.15.40:0/2086305789 conn(0x55e3e29ac8a0 msgr2=0x55e3e2a56ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 -- 172.21.15.40:0/2086305789 shutdown_connections 2024-04-26T21:45:18.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.904+0000 7f9365a0bf40 1 -- 172.21.15.40:0/2086305789 wait complete. 2024-04-26T21:45:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f9365a0bf40 1 Processor -- start 2024-04-26T21:45:18.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f9365a0bf40 1 -- start start 2024-04-26T21:45:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f9365a0bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 0x55e3e2a592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f9365a0bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 0x55e3e2aed070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f9365a0bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a46eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f9365a0bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e3e2a57d20 con 0x7f934c0044e0 2024-04-26T21:45:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f9365a0bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e3e2a54ed0 con 0x55e3e2a55ac0 2024-04-26T21:45:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f9365a0bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e3e2a58000 con 0x55e3e2a4bd90 2024-04-26T21:45:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.905+0000 7f936307a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a46eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f936307a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a46eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45246/0 (socket says 172.21.15.40:45246) 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 learned_addr learned my addr 172.21.15.40:0/3351423491 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f9362078640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 0x55e3e2aed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f9362879640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 0x55e3e2a592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 msgr2=0x55e3e2a592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f936307a640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e3e2a4bd90 0x55e3e2a592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 msgr2=0x55e3e2aed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f936307a640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 0x55e3e2aed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e2aec2d0 con 0x7f934c0044e0 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f9362078640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3e2a55ac0 0x55e3e2aed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f936307a640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a46eb0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x55e3e29ac870 tx=0x7f935c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f934bfff640 1 -- 172.21.15.40:0/3351423491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c00b790 con 0x7f934c0044e0 2024-04-26T21:45:18.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f934bfff640 1 -- 172.21.15.40:0/3351423491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935c00b950 con 0x7f934c0044e0 2024-04-26T21:45:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f934bfff640 1 -- 172.21.15.40:0/3351423491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c00bd40 con 0x7f934c0044e0 2024-04-26T21:45:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e3e29c7e70 con 0x7f934c0044e0 2024-04-26T21:45:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.906+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e2aec870 con 0x7f934c0044e0 2024-04-26T21:45:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.908+0000 7f934bfff640 1 -- 172.21.15.40:0/3351423491 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f935c009030 con 0x7f934c0044e0 2024-04-26T21:45:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.908+0000 7f934bfff640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f935006b2d0 0x7f935006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.908+0000 7f934bfff640 1 -- 172.21.15.40:0/3351423491 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8539+0+0 (secure 0 0 0) 0x7f935c08b320 con 0x7f934c0044e0 2024-04-26T21:45:18.912 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:18.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.908+0000 7f9362879640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f935006b2d0 0x7f935006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.908+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e3e2a53bc0 0x55e3e2aed7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.908+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2a4ae80 con 0x55e3e2a53bc0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.909+0000 7f9362078640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e3e2a53bc0 0x55e3e2aed7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.909+0000 7f9362078640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e3e2a53bc0 0x55e3e2aed7c0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.909+0000 7f9362879640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f935006b2d0 0x7f935006d790 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9354062000 tx=0x7f9354052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.911+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v122'815 uv789 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.911+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2aedd00 con 0x55e3e2a53bc0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.912+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v122'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.912+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2aee0d0 con 0x55e3e2a53bc0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.913+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v122'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.914 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.913+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9a40 con 0x55e3e2a53bc0 2024-04-26T21:45:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.914+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v122'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.914+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9de0 con 0x55e3e2a53bc0 2024-04-26T21:45:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.915+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v122'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.915+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af8a30 con 0x55e3e2a53bc0 2024-04-26T21:45:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.916+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v122'820 uv820 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.916+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af8dd0 con 0x55e3e2a53bc0 2024-04-26T21:45:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.919+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v122'821 uv795 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.919+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.920+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v122'822 uv822 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.920+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.921 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.921+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v122'823 uv823 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.921+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.922+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v122'824 uv824 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.922+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.923+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v122'825 uv825 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.923+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.924+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v122'826 uv826 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.924+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.925+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v122'827 uv802 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.925+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.927+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v122'828 uv828 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.927+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.928+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v122'829 uv829 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.930 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:18.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.928+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.929+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v122'830 uv830 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.929+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.930+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v122'831 uv831 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.930+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2af9170 con 0x55e3e2a53bc0 2024-04-26T21:45:18.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.931+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v122'832 uv832 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.931+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.932+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v122'833 uv833 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.932+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.933+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v122'834 uv808 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.933+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.935+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v122'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.935+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.936 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:18.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.936+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v122'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.936+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afb190 con 0x55e3e2a53bc0 2024-04-26T21:45:18.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.937+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v122'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.937+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afb190 con 0x55e3e2a53bc0 2024-04-26T21:45:18.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.938+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v122'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.938+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.939+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v122'839 uv839 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.939+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.940+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v122'840 uv814 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.940+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.941+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v122'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.941+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.943 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:18.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.942+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v122'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.942+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afb190 con 0x55e3e2a53bc0 2024-04-26T21:45:18.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.943+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v122'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.943+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afb190 con 0x55e3e2a53bc0 2024-04-26T21:45:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.944+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v122'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.944+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afa180 con 0x55e3e2a53bc0 2024-04-26T21:45:18.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.945+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v122'845 uv845 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2a53bc0 2024-04-26T21:45:18.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.945+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e3e2afa690 0x55e3e2afca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.945+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afcfc0 con 0x55e3e2afa690 2024-04-26T21:45:18.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.946+0000 7f9362879640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e3e2afa690 0x55e3e2afca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.946+0000 7f9362879640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e3e2afa690 0x55e3e2afca80 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.948+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v122'596 uv575 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.948+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2afd360 con 0x55e3e2afa690 2024-04-26T21:45:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.949+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v122'597 uv597 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.949+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afd700 con 0x55e3e2afa690 2024-04-26T21:45:18.951 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:18.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.950+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v122'598 uv598 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.950+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afdf50 con 0x55e3e2afa690 2024-04-26T21:45:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.952+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v122'599 uv599 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.952+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afe2f0 con 0x55e3e2afa690 2024-04-26T21:45:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.953+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v122'600 uv600 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.953+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afe690 con 0x55e3e2afa690 2024-04-26T21:45:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.954+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v122'601 uv594 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.954+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2afea30 con 0x55e3e2afa690 2024-04-26T21:45:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.955+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v122'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.955+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.956 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:18.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.956+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v122'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.956+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.957+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v122'604 uv604 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.957+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.958+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v122'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.958+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.960+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v122'606 uv595 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.960+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.961+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v122'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.961+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.963 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.962+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v122'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.962+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b03490 con 0x55e3e2afa690 2024-04-26T21:45:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.964+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v122'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.964+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b03490 con 0x55e3e2afa690 2024-04-26T21:45:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.965+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v122'610 uv610 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.965+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.966+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v122'611 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.966+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.967+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v122'612 uv612 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.967+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.968+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v122'613 uv613 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.969 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.968+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2affde0 con 0x55e3e2afa690 2024-04-26T21:45:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.969+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v122'614 uv614 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.969+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2affde0 con 0x55e3e2afa690 2024-04-26T21:45:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.970+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v122'615 uv615 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.970+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2afedd0 con 0x55e3e2afa690 2024-04-26T21:45:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.971+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v122'616 uv616 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2afa690 2024-04-26T21:45:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.972+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e3e2aff2e0 0x55e3e2b016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.972+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b01c10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.972+0000 7f9362078640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e3e2aff2e0 0x55e3e2b016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.972+0000 7f9362078640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e3e2aff2e0 0x55e3e2b016d0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.973+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v122'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.973+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b01c10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.974+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v122'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.974+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b01c10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.976 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.975+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v122'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.975+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b03010 con 0x55e3e2aff2e0 2024-04-26T21:45:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.976+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v122'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.976+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b03010 con 0x55e3e2aff2e0 2024-04-26T21:45:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.978+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v122'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.978+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b02000 con 0x55e3e2aff2e0 2024-04-26T21:45:18.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.979+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v122'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935800ba10 con 0x55e3e2aff2e0 2024-04-26T21:45:18.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.979+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e3e2b02510 0x55e3e2b04900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.979+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b04f30 con 0x55e3e2b02510 2024-04-26T21:45:18.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.979+0000 7f936307a640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e3e2b02510 0x55e3e2b04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.979+0000 7f936307a640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e3e2b02510 0x55e3e2b04900 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.981+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v122'123 uv122 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.981+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b05440 con 0x55e3e2b02510 2024-04-26T21:45:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.982+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v122'124 uv124 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.982+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05940 con 0x55e3e2b02510 2024-04-26T21:45:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.983+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v122'125 uv125 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.984 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.983+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b06200 con 0x55e3e2b02510 2024-04-26T21:45:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.984+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v122'126 uv126 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.984+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3e2b06ab0 0x55e3e2b08ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.984+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b093e0 con 0x55e3e2b06ab0 2024-04-26T21:45:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.984+0000 7f9362879640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3e2b06ab0 0x55e3e2b08ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.985+0000 7f9362879640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3e2b06ab0 0x55e3e2b08ea0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:18.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.986+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v122'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:18.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.986+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b094a0 con 0x55e3e2b06ab0 2024-04-26T21:45:18.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.987+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v122'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:18.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.987+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b098c0 con 0x55e3e2b06ab0 2024-04-26T21:45:18.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.988+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v122'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:18.989 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:18.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.988+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a8d0 con 0x55e3e2b06ab0 2024-04-26T21:45:18.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.989+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v122'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:18.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.989+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a8d0 con 0x55e3e2b06ab0 2024-04-26T21:45:18.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.990+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v122'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:18.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.990+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b098c0 con 0x55e3e2b06ab0 2024-04-26T21:45:18.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.992+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v122'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:18.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.992+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b098c0 con 0x55e3e2b02510 2024-04-26T21:45:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.993+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v122'294 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.993+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b09c60 con 0x55e3e2b02510 2024-04-26T21:45:18.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.994+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v122'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.994+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:18.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.995+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v122'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.996 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:18.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.995+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0b010 con 0x55e3e2b02510 2024-04-26T21:45:18.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.996+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v122'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.996+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0b010 con 0x55e3e2b02510 2024-04-26T21:45:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.997+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v122'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:18.997+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.000+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v122'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.000+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.001+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v122'300 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.001+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.002+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v122'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.002+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.003+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v122'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.004 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:19.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.003+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.004+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v122'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.004+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.005+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v122'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.005+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.006+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v122'132 uv131 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.006+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.007+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v122'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.007+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.009 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:19.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.008+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v122'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.008+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.009+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v122'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.009+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b02510 2024-04-26T21:45:19.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.010+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v122'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f935c01f070 con 0x55e3e2b02510 2024-04-26T21:45:19.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.010+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.011+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v122'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.011+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.012+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v122'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.012+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.014 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.013+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v122'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.013+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.016+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v122'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.016+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.017+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v122'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.017+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.018+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v122'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.018+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.019+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v122'263 uv257 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.019+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.021+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v122'264 uv264 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.021+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a000 con 0x55e3e2b06ab0 2024-04-26T21:45:19.022 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.022+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v122'265 uv265 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.022+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a560 con 0x55e3e2b06ab0 2024-04-26T21:45:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.023+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v122'266 uv266 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.023+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a560 con 0x55e3e2b06ab0 2024-04-26T21:45:19.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.024+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v122'267 uv267 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.024+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a490 con 0x55e3e2b06ab0 2024-04-26T21:45:19.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.025+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v122'268 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.025+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b0a490 con 0x55e3e2b06ab0 2024-04-26T21:45:19.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.026+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v122'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.026+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a470 con 0x55e3e2b06ab0 2024-04-26T21:45:19.028 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:19.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.027+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v122'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.027+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a470 con 0x55e3e2b06ab0 2024-04-26T21:45:19.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.028+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v122'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.028+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0a860 con 0x55e3e2b06ab0 2024-04-26T21:45:19.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.029+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v122'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.029+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e3e2b0aef0 0x55e3e2b0d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.029+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2b0aef0 2024-04-26T21:45:19.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.029+0000 7f9362078640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e3e2b0aef0 0x55e3e2b0d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.030+0000 7f9362078640 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e3e2b0aef0 0x55e3e2b0d2e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.031+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v122'116 uv114 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2b0aef0 2024-04-26T21:45:19.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.031+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2b0aef0 2024-04-26T21:45:19.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.032+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v122'117 uv117 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2b0aef0 2024-04-26T21:45:19.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.032+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2b0aef0 2024-04-26T21:45:19.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.033+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v122'118 uv118 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2b0aef0 2024-04-26T21:45:19.034 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:19.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.033+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2b0aef0 2024-04-26T21:45:19.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.034+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v122'119 uv119 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2b0aef0 2024-04-26T21:45:19.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.034+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2b06ab0 2024-04-26T21:45:19.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.035+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v122'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.035+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2b06ab0 2024-04-26T21:45:19.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.038+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v122'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.038+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2b06ab0 2024-04-26T21:45:19.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.039+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v122'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.040 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:19.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.039+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0e830 con 0x55e3e2b06ab0 2024-04-26T21:45:19.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.040+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v122'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.040+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0e830 con 0x55e3e2b06ab0 2024-04-26T21:45:19.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.041+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v122'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.041+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2b06ab0 2024-04-26T21:45:19.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.042+0000 7f9362879640 1 -- 172.21.15.40:0/3351423491 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v122'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9354003030 con 0x55e3e2b06ab0 2024-04-26T21:45:19.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.042+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2aff2e0 2024-04-26T21:45:19.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.043+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v122'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.043+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2aff2e0 2024-04-26T21:45:19.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.045+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v122'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.045+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0d820 con 0x55e3e2aff2e0 2024-04-26T21:45:19.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.046+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v122'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.047 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:19.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.046+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.047+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v122'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.047+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.048+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v122'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.048+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.049+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v122'327 uv318 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.049+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.050+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v122'328 uv328 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.050+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.051+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v122'329 uv329 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.052 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:19.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.051+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.053+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v122'330 uv330 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.053+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.054+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v122'331 uv331 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.054+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.055+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v122'332 uv325 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.055+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.056+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v122'333 uv333 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.056+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.057+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v122'334 uv334 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.058 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:19.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.057+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.058+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v122'335 uv335 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.058+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.059+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v122'336 uv336 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.059+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b05de0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.060+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v122'337 uv337 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.060+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e3e2b0dcd0 con 0x55e3e2aff2e0 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9362078640 1 -- 172.21.15.40:0/3351423491 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v122'338 uv338 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9358024040 con 0x55e3e2aff2e0 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e3e2afa690 msgr2=0x55e3e2afca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55e3e2afa690 0x55e3e2afca80 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e3e2a53bc0 msgr2=0x55e3e2aed7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55e3e2a53bc0 0x55e3e2aed7c0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e3e2b02510 msgr2=0x55e3e2b04900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e3e2b02510 0x55e3e2b04900 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e3e2b0aef0 msgr2=0x55e3e2b0d2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55e3e2b0aef0 0x55e3e2b0d2e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e3e2aff2e0 msgr2=0x55e3e2b016d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e3e2aff2e0 0x55e3e2b016d0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3e2b06ab0 msgr2=0x55e3e2b08ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55e3e2b06ab0 0x55e3e2b08ea0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 reap_dead start 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 reap_dead start 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f936307a640 1 -- 172.21.15.40:0/3351423491 reap_dead start 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f935006b2d0 msgr2=0x7f935006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f935006b2d0 0x7f935006d790 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9354062000 tx=0x7f9354052000 comp rx=0 tx=0).stop 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 msgr2=0x55e3e2a46eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a46eb0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x55e3e29ac870 tx=0x7f935c00da30 comp rx=0 tx=0).stop 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 shutdown_connections 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f935006b2d0 0x7f935006d790 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 --2- 172.21.15.40:0/3351423491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f934c0044e0 0x55e3e2a46eb0 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.061+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 >> 172.21.15.40:0/3351423491 conn(0x55e3e29ac8a0 msgr2=0x55e3e29adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.062+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 shutdown_connections 2024-04-26T21:45:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.062+0000 7f9365a0bf40 1 -- 172.21.15.40:0/3351423491 wait complete. 2024-04-26T21:45:19.066 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds_snaptable 0 2024-04-26T21:45:19.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.148+0000 7fd245849f40 1 Processor -- start 2024-04-26T21:45:19.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.148+0000 7fd245849f40 1 -- start start 2024-04-26T21:45:19.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd245849f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 0x55e91c06a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd245849f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0f8820 0x55e91c06aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd245849f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0fa950 0x55e91c0eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd245849f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e91bfcae40 con 0x55e91c0f8820 2024-04-26T21:45:19.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd245849f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e91bfcee80 con 0x55e91c053650 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd245849f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e91c0f8c00 con 0x55e91c0fa950 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd242b44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0f8820 0x55e91c06aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd243345640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 0x55e91c06a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd242b44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0f8820 0x55e91c06aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45256/0 (socket says 172.21.15.40:45256) 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd242b44640 1 -- 172.21.15.40:0/3063337842 learned_addr learned my addr 172.21.15.40:0/3063337842 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd243b46640 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0fa950 0x55e91c0eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd242b44640 1 -- 172.21.15.40:0/3063337842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0fa950 msgr2=0x55e91c0eb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd242b44640 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0fa950 0x55e91c0eb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd242b44640 1 -- 172.21.15.40:0/3063337842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 msgr2=0x55e91c06a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd242b44640 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 0x55e91c06a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd242b44640 1 -- 172.21.15.40:0/3063337842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91c06c930 con 0x55e91c0f8820 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.149+0000 7fd243345640 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 0x55e91c06a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd242b44640 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0f8820 0x55e91c06aa30 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fd23800b230 tx=0x7fd23800b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841b0b088678dd02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd241b42640 1 -- 172.21.15.40:0/3063337842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23800f660 con 0x55e91c0f8820 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd241b42640 1 -- 172.21.15.40:0/3063337842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23800f820 con 0x55e91c0f8820 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd241b42640 1 -- 172.21.15.40:0/3063337842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23800fc10 con 0x55e91c0f8820 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 -- 172.21.15.40:0/3063337842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0f8820 msgr2=0x55e91c06aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0f8820 0x55e91c06aa30 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fd23800b230 tx=0x7fd23800b6f0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 -- 172.21.15.40:0/3063337842 shutdown_connections 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0fa950 0x55e91c0eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0f8820 0x55e91c06aa30 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 --2- 172.21.15.40:0/3063337842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 0x55e91c06a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 -- 172.21.15.40:0/3063337842 >> 172.21.15.40:0/3063337842 conn(0x55e91c04d8a0 msgr2=0x55e91c04dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 -- 172.21.15.40:0/3063337842 shutdown_connections 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.150+0000 7fd245849f40 1 -- 172.21.15.40:0/3063337842 wait complete. 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.151+0000 7fd245849f40 1 Processor -- start 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.151+0000 7fd245849f40 1 -- start start 2024-04-26T21:45:19.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.151+0000 7fd245849f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 0x55e91c069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.151+0000 7fd245849f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0f8820 0x55e91c190830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.151+0000 7fd245849f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0fa950 0x55e91c067a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.151+0000 7fd245849f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e91c0f6730 con 0x55e91c0fa950 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.151+0000 7fd245849f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e91c0f6a10 con 0x55e91c053650 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.151+0000 7fd245849f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e91c04e8e0 con 0x55e91c0f8820 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0fa950 0x55e91c067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0fa950 0x55e91c067a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45270/0 (socket says 172.21.15.40:45270) 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 -- 172.21.15.40:0/3356551001 learned_addr learned my addr 172.21.15.40:0/3356551001 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd242b44640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0f8820 0x55e91c190830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243345640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 0x55e91c069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 -- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0f8820 msgr2=0x55e91c190830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e91c0f8820 0x55e91c190830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 -- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 msgr2=0x55e91c069730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e91c053650 0x55e91c069730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 -- 172.21.15.40:0/3356551001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91c0523a0 con 0x55e91c0fa950 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd243b46640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0fa950 0x55e91c067a00 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd23400a4a0 tx=0x7fd234003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd240b40640 1 -- 172.21.15.40:0/3356551001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234055040 con 0x55e91c0fa950 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd240b40640 1 -- 172.21.15.40:0/3356551001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd234069070 con 0x55e91c0fa950 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd240b40640 1 -- 172.21.15.40:0/3356551001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234005a80 con 0x55e91c0fa950 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e91c052680 con 0x55e91c0fa950 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.152+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e91c052c40 con 0x55e91c0fa950 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.154+0000 7fd240b40640 1 -- 172.21.15.40:0/3356551001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd234063070 con 0x55e91c0fa950 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.154+0000 7fd240b40640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd23006b2d0 0x7fd23006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.154+0000 7fd240b40640 1 -- 172.21.15.40:0/3356551001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8539+0+0 (secure 0 0 0) 0x7fd2340cf460 con 0x55e91c0fa950 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.154+0000 7fd243345640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd23006b2d0 0x7fd23006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.154+0000 7fd245849f40 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e91c0f6180 0x55e91c194c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.154+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55e91c190d70 con 0x55e91c0f6180 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.155+0000 7fd242b44640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e91c0f6180 0x55e91c194c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.155+0000 7fd243345640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd23006b2d0 0x7fd23006d790 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd23c000c00 tx=0x7fd23c001750 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.155+0000 7fd242b44640 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e91c0f6180 0x55e91c194c10 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd242b44640 1 -- 172.21.15.40:0/3356551001 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds_snaptable [truncate 0] v122'137 uv137 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd23800f9d0 con 0x55e91c0f6180 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e91c0f6180 msgr2=0x55e91c194c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55e91c0f6180 0x55e91c194c10 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd23006b2d0 msgr2=0x7fd23006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd23006b2d0 0x7fd23006d790 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd23c000c00 tx=0x7fd23c001750 comp rx=0 tx=0).stop 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0fa950 msgr2=0x55e91c067a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 --2- 172.21.15.40:0/3356551001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e91c0fa950 0x55e91c067a00 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd23400a4a0 tx=0x7fd234003330 comp rx=0 tx=0).stop 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd243b46640 1 -- 172.21.15.40:0/3356551001 reap_dead start 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 shutdown_connections 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 >> 172.21.15.40:0/3356551001 conn(0x55e91c04d8a0 msgr2=0x55e91c0f3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 shutdown_connections 2024-04-26T21:45:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.156+0000 7fd245849f40 1 -- 172.21.15.40:0/3356551001 wait complete. 2024-04-26T21:45:19.254 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.392+0000 7f2407420640 1 Processor -- start 2024-04-26T21:45:19.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.392+0000 7f2407420640 1 -- start start 2024-04-26T21:45:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.393+0000 7f2407420640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24000b40d0 0x7f24001625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.393+0000 7f2407420640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24000d0a10 con 0x7f24000d1a00 2024-04-26T21:45:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.393+0000 7f2407420640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24000d0cf0 con 0x7f2400162b30 2024-04-26T21:45:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.393+0000 7f2407420640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f24000d0fd0 con 0x7f24000b40d0 2024-04-26T21:45:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.393+0000 7f24051bc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f24000d1a00 0x7f24000d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44572/0 (socket says 172.21.15.40:44572) 2024-04-26T21:45:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.393+0000 7f24051bc640 1 -- 172.21.15.40:0/1049457970 learned_addr learned my addr 172.21.15.40:0/1049457970 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24049bb640 1 --2- 172.21.15.40:0/1049457970 >> v2:172.21.15.184:3300/0 conn(0x7f24000b40d0 0x7f24001625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1932691489 0 0) 0x7f24000d0a10 con 0x7f24000d1a00 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23e8003520 con 0x7f24000d1a00 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2725086191 0 0) 0x7f23e8003520 con 0x7f24000d1a00 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 >> v2:172.21.15.184:3300/0 conn(0x7f24000b40d0 msgr2=0x7f24001625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 --2- 172.21.15.40:0/1049457970 >> v2:172.21.15.184:3300/0 conn(0x7f24000b40d0 0x7f24001625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 >> v1:172.21.15.184:6789/0 conn(0x7f2400162b30 legacy=0x7f24000b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2400162f80 con 0x7f24000d1a00 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f23f00019f0 con 0x7f24000d1a00 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 >> v1:172.21.15.40:6789/0 conn(0x7f24000d1a00 legacy=0x7f24000d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e80039b0 0x7f23e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80051c0 0x7f23e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f24000d0fd0 con 0x7f23e80039b0 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23e800c160 con 0x7f23e8004800 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f23e800c360 con 0x7f23e80051c0 2024-04-26T21:45:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24051bc640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e80039b0 0x7f23e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24049bb640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24059bd640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80051c0 0x7f23e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24051bc640 1 -- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80051c0 msgr2=0x7f23e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24051bc640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80051c0 0x7f23e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24051bc640 1 -- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8004800 msgr2=0x7f23e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24051bc640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24051bc640 1 -- 172.21.15.40:0/1049457970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e8003520 con 0x7f23e80039b0 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24059bd640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80051c0 0x7f23e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.394+0000 7f24059bd640 1 -- 172.21.15.40:0/1049457970 reap_dead start 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.395+0000 7f24051bc640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e80039b0 0x7f23e8003dd0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f23f000a860 tx=0x7f23e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becdd9e6fe0502ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.395+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000a480 con 0x7f23e80039b0 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.395+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f002f070 con 0x7f23e80039b0 2024-04-26T21:45:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.395+0000 7f23fffff640 1 -- 172.21.15.40:0/1049457970 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f002bd40 con 0x7f23e80039b0 2024-04-26T21:45:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.396+0000 7f2407420640 1 -- 172.21.15.40:0/1049457970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e80039b0 msgr2=0x7f23e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.396+0000 7f2407420640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e80039b0 0x7f23e8003dd0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f23f000a860 tx=0x7f23e8005830 comp rx=0 tx=0).stop 2024-04-26T21:45:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.396+0000 7f2407420640 1 -- 172.21.15.40:0/1049457970 shutdown_connections 2024-04-26T21:45:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.396+0000 7f2407420640 1 --2- 172.21.15.40:0/1049457970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e80039b0 0x7f23e8003dd0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.396+0000 7f2407420640 1 -- 172.21.15.40:0/1049457970 >> 172.21.15.40:0/1049457970 conn(0x7f24000d4fd0 msgr2=0x7f24000d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.396+0000 7f2407420640 1 -- 172.21.15.40:0/1049457970 shutdown_connections 2024-04-26T21:45:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.396+0000 7f2407420640 1 -- 172.21.15.40:0/1049457970 wait complete. 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.397+0000 7f2407420640 1 Processor -- start 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.397+0000 7f2407420640 1 -- start start 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f2407420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80039b0 0x7f24000c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f2407420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24000ceb80 0x7f24000cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f2407420640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24000cced0 0x7f2400154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f2407420640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2400162770 con 0x7f24000ceb80 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f2407420640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24001625f0 con 0x7f24000cced0 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f2407420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24001628f0 con 0x7f23e80039b0 2024-04-26T21:45:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f24049bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24000ceb80 0x7f24000cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f24049bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24000ceb80 0x7f24000cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45280/0 (socket says 172.21.15.40:45280) 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f24049bb640 1 -- 172.21.15.40:0/87584855 learned_addr learned my addr 172.21.15.40:0/87584855 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f24059bd640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24000cced0 0x7f2400154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.398+0000 7f24051bc640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80039b0 0x7f24000c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f24049bb640 1 -- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80039b0 msgr2=0x7f24000c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f24049bb640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80039b0 0x7f24000c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f24049bb640 1 -- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24000cced0 msgr2=0x7f2400154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f24049bb640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24000cced0 0x7f2400154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f24049bb640 1 -- 172.21.15.40:0/87584855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24000cd530 con 0x7f24000ceb80 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f24049bb640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24000ceb80 0x7f24000cc990 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f23f4011a70 tx=0x7f23f4011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f23fdffb640 1 -- 172.21.15.40:0/87584855 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400fa90 con 0x7f24000ceb80 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f23fdffb640 1 -- 172.21.15.40:0/87584855 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f400fc30 con 0x7f24000ceb80 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24000cd810 con 0x7f24000ceb80 2024-04-26T21:45:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f23fdffb640 1 -- 172.21.15.40:0/87584855 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4014440 con 0x7f24000ceb80 2024-04-26T21:45:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.399+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24000cf000 con 0x7f24000ceb80 2024-04-26T21:45:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.400+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24000cd2b0 con 0x7f24000ceb80 2024-04-26T21:45:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.401+0000 7f23fdffb640 1 -- 172.21.15.40:0/87584855 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23f4012040 con 0x7f24000ceb80 2024-04-26T21:45:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.401+0000 7f23fdffb640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.401+0000 7f23fdffb640 1 -- 172.21.15.40:0/87584855 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8539+0+0 (secure 0 0 0) 0x7f23f408cbb0 con 0x7f24000ceb80 2024-04-26T21:45:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.401+0000 7f24051bc640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.402+0000 7f24051bc640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23ec06c8f0 0x7f23ec06edb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f23f0033000 tx=0x7f23f0009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.405+0000 7f23fdffb640 1 -- 172.21.15.40:0/87584855 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24000cd2b0 con 0x7f24000ceb80 2024-04-26T21:45:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.632+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f24000694f0 con 0x7f24000ceb80 2024-04-26T21:45:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.805+0000 7f23fdffb640 1 -- 172.21.15.40:0/87584855 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v260) ==== 162+0+0 (secure 0 0 0) 0x7f23f4019030 con 0x7f24000ceb80 2024-04-26T21:45:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23ec06c8f0 msgr2=0x7f23ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23ec06c8f0 0x7f23ec06edb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f23f0033000 tx=0x7f23f0009030 comp rx=0 tx=0).stop 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24000ceb80 msgr2=0x7f24000cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24000ceb80 0x7f24000cc990 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f23f4011a70 tx=0x7f23f4011e70 comp rx=0 tx=0).stop 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 shutdown_connections 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24000cced0 0x7f2400154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24000ceb80 0x7f24000cc990 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 --2- 172.21.15.40:0/87584855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23e80039b0 0x7f24000c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 >> 172.21.15.40:0/87584855 conn(0x7f24000d4fd0 msgr2=0x7f2400161540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 shutdown_connections 2024-04-26T21:45:19.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.808+0000 7f2407420640 1 -- 172.21.15.40:0/87584855 wait complete. 2024-04-26T21:45:19.813 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:19.812+0000 7f2e7bcdc640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-04-26T21:45:19.814 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:19.813+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:45:19.814 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:19.813+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:45:19.819 INFO:tasks.ceph.mds.c.smithi040.stderr: -23> 2024-04-26T21:45:19.812+0000 7f2e7bcdc640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-04-26T21:45:19.822 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:19.843 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:45:19.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.922+0000 7fb5ee2d1640 1 Processor -- start 2024-04-26T21:45:19.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.922+0000 7fb5ee2d1640 1 -- start start 2024-04-26T21:45:19.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.923+0000 7fb5ee2d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5e8157660 0x7fb5e80b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.923+0000 7fb5ee2d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb5e80cfb50 0x7fb5e80b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.923+0000 7fb5ee2d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e815d490 con 0x7fb5e806c240 2024-04-26T21:45:19.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.923+0000 7fb5ee2d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e815e080 con 0x7fb5e8157660 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.923+0000 7fb5ee2d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e815d750 con 0x7fb5e80cfb50 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.923+0000 7fb5e7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5e8157660 0x7fb5e80b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.923+0000 7fb5e7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5e8157660 0x7fb5e80b5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45312/0 (socket says 172.21.15.40:45312) 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.923+0000 7fb5e7fff640 1 -- 172.21.15.40:0/3783260609 learned_addr learned my addr 172.21.15.40:0/3783260609 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.924+0000 7fb5e77fe640 1 --2- 172.21.15.40:0/3783260609 >> v2:172.21.15.184:3300/0 conn(0x7fb5e80cfb50 0x7fb5e80b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.924+0000 7fb5e7fff640 1 -- 172.21.15.40:0/3783260609 >> v2:172.21.15.184:3300/0 conn(0x7fb5e80cfb50 msgr2=0x7fb5e80b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.924+0000 7fb5e7fff640 1 --2- 172.21.15.40:0/3783260609 >> v2:172.21.15.184:3300/0 conn(0x7fb5e80cfb50 0x7fb5e80b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.924+0000 7fb5e7fff640 1 -- 172.21.15.40:0/3783260609 >> v1:172.21.15.184:6789/0 conn(0x7fb5e806c240 legacy=0x7fb5e806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.924+0000 7fb5e7fff640 1 -- 172.21.15.40:0/3783260609 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e815da30 con 0x7fb5e8157660 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.924+0000 7fb5e7fff640 1 --2- 172.21.15.40:0/3783260609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8157660 0x7fb5e80b5fd0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fb5e806ce90 tx=0x7fb5dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f444b308c37d284c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5e6ffd640 1 -- 172.21.15.40:0/3783260609 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc037070 con 0x7fb5e8157660 2024-04-26T21:45:19.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5e6ffd640 1 -- 172.21.15.40:0/3783260609 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc032440 con 0x7fb5e8157660 2024-04-26T21:45:19.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5e6ffd640 1 -- 172.21.15.40:0/3783260609 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc032810 con 0x7fb5e8157660 2024-04-26T21:45:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/3783260609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8157660 msgr2=0x7fb5e80b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/3783260609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8157660 0x7fb5e80b5fd0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fb5e806ce90 tx=0x7fb5dc02a270 comp rx=0 tx=0).stop 2024-04-26T21:45:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/3783260609 shutdown_connections 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/3783260609 >> v2:172.21.15.184:3300/0 conn(0x7fb5e80cfb50 0x7fb5e80b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/3783260609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8157660 0x7fb5e80b5fd0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/3783260609 >> 172.21.15.40:0/3783260609 conn(0x7fb5e80b3180 msgr2=0x7fb5e80b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/3783260609 shutdown_connections 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.925+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/3783260609 wait complete. 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.926+0000 7fb5ee2d1640 1 Processor -- start 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ee2d1640 1 -- start start 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ee2d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e806c240 0x7fb5e8154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ee2d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e8157660 0x7fb5e814cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ee2d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8156110 0x7fb5e81565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ee2d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8072860 con 0x7fb5e8156110 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ee2d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80726e0 con 0x7fb5e806c240 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ee2d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e80729e0 con 0x7fb5e8157660 2024-04-26T21:45:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ec86e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8156110 0x7fb5e81565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ec86e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8156110 0x7fb5e81565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45328/0 (socket says 172.21.15.40:45328) 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.927+0000 7fb5ec86e640 1 -- 172.21.15.40:0/165662884 learned_addr learned my addr 172.21.15.40:0/165662884 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5e7fff640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e806c240 0x7fb5e8154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5e77fe640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e8157660 0x7fb5e814cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5ec86e640 1 -- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e8157660 msgr2=0x7fb5e814cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5ec86e640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e8157660 0x7fb5e814cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5ec86e640 1 -- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e806c240 msgr2=0x7fb5e8154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5ec86e640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e806c240 0x7fb5e8154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5ec86e640 1 -- 172.21.15.40:0/165662884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e81539f0 con 0x7fb5e8156110 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5ec86e640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8156110 0x7fb5e81565c0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000c7a0 tx=0x7fb5e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5e4ff9640 1 -- 172.21.15.40:0/165662884 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0003b70 con 0x7fb5e8156110 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5e4ff9640 1 -- 172.21.15.40:0/165662884 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e0003d30 con 0x7fb5e8156110 2024-04-26T21:45:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e8153cd0 con 0x7fb5e8156110 2024-04-26T21:45:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5e4ff9640 1 -- 172.21.15.40:0/165662884 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000aa90 con 0x7fb5e8156110 2024-04-26T21:45:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.928+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e8154290 con 0x7fb5e8156110 2024-04-26T21:45:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.929+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e8068fc0 con 0x7fb5e8156110 2024-04-26T21:45:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.930+0000 7fb5e4ff9640 1 -- 172.21.15.40:0/165662884 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5e000ac30 con 0x7fb5e8156110 2024-04-26T21:45:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.930+0000 7fb5e4ff9640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5d406c8f0 0x7fb5d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.930+0000 7fb5e4ff9640 1 -- 172.21.15.40:0/165662884 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8625+0+0 (secure 0 0 0) 0x7fb5e008c490 con 0x7fb5e8156110 2024-04-26T21:45:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.930+0000 7fb5e7fff640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5d406c8f0 0x7fb5d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.931+0000 7fb5e7fff640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5d406c8f0 0x7fb5d406edb0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc034000 tx=0x7fb5dc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:19.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:19.935+0000 7fb5e4ff9640 1 -- 172.21.15.40:0/165662884 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5e0059740 con 0x7fb5e8156110 2024-04-26T21:45:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.158+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5e80b5620 con 0x7fb5e8156110 2024-04-26T21:45:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.159+0000 7fb5e4ff9640 1 -- 172.21.15.40:0/165662884 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 262 v262) ==== 95+0+4355 (secure 0 0 0) 0x7fb5e005da40 con 0x7fb5e8156110 2024-04-26T21:45:20.160 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:20.160 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":262,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":262,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:19.814679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 262 2024-04-26T21:45:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.160+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5d406c8f0 msgr2=0x7fb5d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.160+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5d406c8f0 0x7fb5d406edb0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc034000 tx=0x7fb5dc02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.160+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8156110 msgr2=0x7fb5e81565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.160+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8156110 0x7fb5e81565c0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000c7a0 tx=0x7fb5e000cc60 comp rx=0 tx=0).stop 2024-04-26T21:45:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.161+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 shutdown_connections 2024-04-26T21:45:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.161+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb5d406c8f0 0x7fb5d406edb0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.161+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e8156110 0x7fb5e81565c0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.161+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5e8157660 0x7fb5e814cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.161+0000 7fb5ee2d1640 1 --2- 172.21.15.40:0/165662884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5e806c240 0x7fb5e8154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.161+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 >> 172.21.15.40:0/165662884 conn(0x7fb5e80b3180 msgr2=0x7fb5e815e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.161+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 shutdown_connections 2024-04-26T21:45:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.161+0000 7fb5ee2d1640 1 -- 172.21.15.40:0/165662884 wait complete. 2024-04-26T21:45:20.172 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:20.172 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.314+0000 7f5f37133640 1 Processor -- start 2024-04-26T21:45:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.314+0000 7f5f37133640 1 -- start start 2024-04-26T21:45:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f37133640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5f300b5f20 0x7f5f300d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f37133640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f300b7e50 con 0x7f5f30162880 2024-04-26T21:45:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f37133640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f300d1800 con 0x7f5f300d3bc0 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f37133640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5f300d0580 con 0x7f5f300b5f20 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f34ecf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5f30162880 0x7f5f300d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44582/0 (socket says 172.21.15.40:44582) 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f34ecf640 1 -- 172.21.15.40:0/1219525199 learned_addr learned my addr 172.21.15.40:0/1219525199 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4129377557 0 0) 0x7f5f300b7e50 con 0x7f5f30162880 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f356d0640 1 --2- 172.21.15.40:0/1219525199 >> v2:172.21.15.184:3300/0 conn(0x7f5f300b5f20 0x7f5f300d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f20003520 con 0x7f5f30162880 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3673234689 0 0) 0x7f5f20003520 con 0x7f5f30162880 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 >> v2:172.21.15.184:3300/0 conn(0x7f5f300b5f20 msgr2=0x7f5f300d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 --2- 172.21.15.40:0/1219525199 >> v2:172.21.15.184:3300/0 conn(0x7f5f300b5f20 0x7f5f300d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 >> v1:172.21.15.184:6789/0 conn(0x7f5f300d3bc0 legacy=0x7f5f300b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f300d07d0 con 0x7f5f30162880 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f5f24001670 con 0x7f5f30162880 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 >> v1:172.21.15.40:6789/0 conn(0x7f5f30162880 legacy=0x7f5f300d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f200039b0 0x7f5f20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f20004800 0x7f5f20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.315+0000 7f5f2f7fe640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200051c0 0x7f5f2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5f300d0580 con 0x7f5f200039b0 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2000c060 con 0x7f5f20004800 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2000c260 con 0x7f5f200051c0 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f34ecf640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f200039b0 0x7f5f20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f2ffff640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f20004800 0x7f5f20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f356d0640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200051c0 0x7f5f2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f34ecf640 1 -- 172.21.15.40:0/1219525199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200051c0 msgr2=0x7f5f2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f34ecf640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200051c0 0x7f5f2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f34ecf640 1 -- 172.21.15.40:0/1219525199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f20004800 msgr2=0x7f5f20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f34ecf640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f20004800 0x7f5f20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f34ecf640 1 -- 172.21.15.40:0/1219525199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f20003520 con 0x7f5f200039b0 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.316+0000 7f5f356d0640 1 -- 172.21.15.40:0/1219525199 reap_dead start 2024-04-26T21:45:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f34ecf640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f200039b0 0x7f5f20003dd0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c240 tx=0x7f5f20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c1c9a55e3670ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24007d10 con 0x7f5f200039b0 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f24005ad0 con 0x7f5f200039b0 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f2f7fe640 1 -- 172.21.15.40:0/1219525199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24007070 con 0x7f5f200039b0 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f37133640 1 -- 172.21.15.40:0/1219525199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f200039b0 msgr2=0x7f5f20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f37133640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f200039b0 0x7f5f20003dd0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c240 tx=0x7f5f20005830 comp rx=0 tx=0).stop 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f37133640 1 -- 172.21.15.40:0/1219525199 shutdown_connections 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f37133640 1 --2- 172.21.15.40:0/1219525199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f200039b0 0x7f5f20003dd0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f37133640 1 -- 172.21.15.40:0/1219525199 >> 172.21.15.40:0/1219525199 conn(0x7f5f300d4f60 msgr2=0x7f5f300d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f37133640 1 -- 172.21.15.40:0/1219525199 shutdown_connections 2024-04-26T21:45:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.317+0000 7f5f37133640 1 -- 172.21.15.40:0/1219525199 wait complete. 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.318+0000 7f5f37133640 1 Processor -- start 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.318+0000 7f5f37133640 1 -- start start 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f37133640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200039b0 0x7f5f300cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f37133640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f300ce130 0x7f5f3015c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f37133640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f300b4db0 0x7f5f300b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f37133640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5f30162c60 con 0x7f5f300b4db0 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f37133640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f300d0d90 con 0x7f5f300ce130 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f37133640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5f30162de0 con 0x7f5f200039b0 2024-04-26T21:45:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f356d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f300b4db0 0x7f5f300b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f356d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f300b4db0 0x7f5f300b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45340/0 (socket says 172.21.15.40:45340) 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.319+0000 7f5f356d0640 1 -- 172.21.15.40:0/723449670 learned_addr learned my addr 172.21.15.40:0/723449670 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f34ecf640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200039b0 0x7f5f300cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f2ffff640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f300ce130 0x7f5f3015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f356d0640 1 -- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200039b0 msgr2=0x7f5f300cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f356d0640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200039b0 0x7f5f300cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f356d0640 1 -- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f300ce130 msgr2=0x7f5f3015c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f356d0640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f300ce130 0x7f5f3015c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f356d0640 1 -- 172.21.15.40:0/723449670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3015cbd0 con 0x7f5f300b4db0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f356d0640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f300b4db0 0x7f5f300b5260 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f5f300d11d0 tx=0x7f5f280115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/723449670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28011db0 con 0x7f5f300b4db0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/723449670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2801b070 con 0x7f5f300b4db0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f300ccf90 con 0x7f5f300b4db0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/723449670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28003990 con 0x7f5f300b4db0 2024-04-26T21:45:20.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.320+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f300cd550 con 0x7f5f300b4db0 2024-04-26T21:45:20.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.321+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f300694f0 con 0x7f5f300b4db0 2024-04-26T21:45:20.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.322+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/723449670 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f28005d10 con 0x7f5f300b4db0 2024-04-26T21:45:20.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.322+0000 7f5f2d7fa640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f1406c920 0x7f5f1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.322+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/723449670 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8625+0+0 (secure 0 0 0) 0x7f5f2808cac0 con 0x7f5f300b4db0 2024-04-26T21:45:20.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.322+0000 7f5f34ecf640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f1406c920 0x7f5f1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.323+0000 7f5f34ecf640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f1406c920 0x7f5f1406ede0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5f24032000 tx=0x7f5f2402d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:20.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.327+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/723449670 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5f28025020 con 0x7f5f300b4db0 2024-04-26T21:45:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.551+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f300bab80 con 0x7f5f300b4db0 2024-04-26T21:45:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.552+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/723449670 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 262 v262) ==== 95+0+4355 (secure 0 0 0) 0x7f5f2805a3c0 con 0x7f5f300b4db0 2024-04-26T21:45:20.553 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:20.554 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":262,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":262,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:19.814679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:20.556 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 262 2024-04-26T21:45:20.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f1406c920 msgr2=0x7f5f1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f1406c920 0x7f5f1406ede0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5f24032000 tx=0x7f5f2402d000 comp rx=0 tx=0).stop 2024-04-26T21:45:20.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f300b4db0 msgr2=0x7f5f300b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f300b4db0 0x7f5f300b5260 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f5f300d11d0 tx=0x7f5f280115a0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 shutdown_connections 2024-04-26T21:45:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5f1406c920 0x7f5f1406ede0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f300b4db0 0x7f5f300b5260 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f300ce130 0x7f5f3015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 --2- 172.21.15.40:0/723449670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f200039b0 0x7f5f300cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 >> 172.21.15.40:0/723449670 conn(0x7f5f300d4f60 msgr2=0x7f5f3006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 shutdown_connections 2024-04-26T21:45:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.554+0000 7f5f37133640 1 -- 172.21.15.40:0/723449670 wait complete. 2024-04-26T21:45:20.566 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:20.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:20.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.666+0000 7f5236104640 1 Processor -- start 2024-04-26T21:45:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.667+0000 7f5236104640 1 -- start start 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.667+0000 7f5236104640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f523006e270 0x7f52300d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.667+0000 7f5236104640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52300d2420 con 0x7f523006d560 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.667+0000 7f5236104640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52300cd6b0 con 0x7f52300d0070 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.667+0000 7f5236104640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5230162d00 con 0x7f523006e270 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.667+0000 7f522ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f523006d560 0x7f52300ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44588/0 (socket says 172.21.15.40:44588) 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.667+0000 7f522ffff640 1 -- 172.21.15.40:0/1417988237 learned_addr learned my addr 172.21.15.40:0/1417988237 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522effd640 1 --2- 172.21.15.40:0/1417988237 >> v2:172.21.15.184:3300/0 conn(0x7f523006e270 0x7f52300d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 417707044 0 0) 0x7f52300d2420 con 0x7f523006d560 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5218003520 con 0x7f523006d560 2024-04-26T21:45:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2491025206 0 0) 0x7f5218003520 con 0x7f523006d560 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 >> v2:172.21.15.184:3300/0 conn(0x7f523006e270 msgr2=0x7f52300d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 --2- 172.21.15.40:0/1417988237 >> v2:172.21.15.184:3300/0 conn(0x7f523006e270 0x7f52300d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 >> v1:172.21.15.184:6789/0 conn(0x7f52300d0070 legacy=0x7f523006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52300cd990 con 0x7f523006d560 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f52280019f0 con 0x7f523006d560 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522effd640 1 --2- 172.21.15.40:0/1417988237 >> v2:172.21.15.184:3300/0 conn(0x7f523006e270 0x7f52300d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 >> v1:172.21.15.40:6789/0 conn(0x7f523006d560 legacy=0x7f52300ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52180039b0 0x7f5218003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5218004800 0x7f5218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52180051c0 0x7f521800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5230162d00 con 0x7f52180051c0 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f521800c060 con 0x7f5218004800 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f521800c260 con 0x7f52180039b0 2024-04-26T21:45:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522ffff640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52180051c0 0x7f521800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522effd640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5218004800 0x7f5218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522f7fe640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52180039b0 0x7f5218003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522ffff640 1 -- 172.21.15.40:0/1417988237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52180039b0 msgr2=0x7f5218003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522ffff640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52180039b0 0x7f5218003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522ffff640 1 -- 172.21.15.40:0/1417988237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5218004800 msgr2=0x7f5218004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522ffff640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5218004800 0x7f5218004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.668+0000 7f522ffff640 1 -- 172.21.15.40:0/1417988237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5218003520 con 0x7f52180051c0 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.669+0000 7f522ffff640 1 -- 172.21.15.40:0/1417988237 reap_dead start 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.669+0000 7f522ffff640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52180051c0 0x7f521800b9c0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f522800a850 tx=0x7f5218005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0393552170dbd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.669+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522800a4a0 con 0x7f52180051c0 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.669+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522802f070 con 0x7f52180051c0 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.669+0000 7f522e7fc640 1 -- 172.21.15.40:0/1417988237 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522802bce0 con 0x7f52180051c0 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.670+0000 7f5236104640 1 -- 172.21.15.40:0/1417988237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52180051c0 msgr2=0x7f521800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.670+0000 7f5236104640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52180051c0 0x7f521800b9c0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f522800a850 tx=0x7f5218005830 comp rx=0 tx=0).stop 2024-04-26T21:45:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.670+0000 7f5236104640 1 -- 172.21.15.40:0/1417988237 shutdown_connections 2024-04-26T21:45:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.670+0000 7f5236104640 1 --2- 172.21.15.40:0/1417988237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52180051c0 0x7f521800b9c0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.670+0000 7f5236104640 1 -- 172.21.15.40:0/1417988237 >> 172.21.15.40:0/1417988237 conn(0x7f52300d2ec0 msgr2=0x7f52300d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.670+0000 7f5236104640 1 -- 172.21.15.40:0/1417988237 shutdown_connections 2024-04-26T21:45:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.670+0000 7f5236104640 1 -- 172.21.15.40:0/1417988237 wait complete. 2024-04-26T21:45:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.671+0000 7f5236104640 1 Processor -- start 2024-04-26T21:45:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.671+0000 7f5236104640 1 -- start start 2024-04-26T21:45:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f5236104640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52180051c0 0x7f523015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f5236104640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523009d7a0 0x7f523009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f5236104640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523009fe60 0x7f52300a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f5236104640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52300d1d40 con 0x7f523009fe60 2024-04-26T21:45:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f5236104640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52300d2d10 con 0x7f52180051c0 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f5236104640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52300d1ec0 con 0x7f523009d7a0 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523009fe60 0x7f52300a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523009fe60 0x7f52300a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45354/0 (socket says 172.21.15.40:45354) 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522ffff640 1 -- 172.21.15.40:0/2525315130 learned_addr learned my addr 172.21.15.40:0/2525315130 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522effd640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523009d7a0 0x7f523009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522f7fe640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52180051c0 0x7f523015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522ffff640 1 -- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523009d7a0 msgr2=0x7f523009dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522ffff640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523009d7a0 0x7f523009dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522ffff640 1 -- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52180051c0 msgr2=0x7f523015fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522ffff640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52180051c0 0x7f523015fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522ffff640 1 -- 172.21.15.40:0/2525315130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523009e310 con 0x7f523009fe60 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.672+0000 7f522f7fe640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52180051c0 0x7f523015fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.673+0000 7f522ffff640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523009fe60 0x7f52300a02e0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f522802b530 tx=0x7f522802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.673+0000 7f5217fff640 1 -- 172.21.15.40:0/2525315130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228034880 con 0x7f523009fe60 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.673+0000 7f5217fff640 1 -- 172.21.15.40:0/2525315130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5228034a40 con 0x7f523009fe60 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.673+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523009e5f0 con 0x7f523009fe60 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.673+0000 7f5217fff640 1 -- 172.21.15.40:0/2525315130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228034e30 con 0x7f523009fe60 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.673+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523009ebb0 con 0x7f523009fe60 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.674+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52300694f0 con 0x7f523009fe60 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.674+0000 7f5217fff640 1 -- 172.21.15.40:0/2525315130 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5228040430 con 0x7f523009fe60 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.675+0000 7f5217fff640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f521c06c920 0x7f521c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.675+0000 7f5217fff640 1 -- 172.21.15.40:0/2525315130 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8625+0+0 (secure 0 0 0) 0x7f52280accb0 con 0x7f523009fe60 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.675+0000 7f522f7fe640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f521c06c920 0x7f521c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.676+0000 7f522f7fe640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f521c06c920 0x7f521c06ede0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5220000c00 tx=0x7f5220009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.679+0000 7f5217fff640 1 -- 172.21.15.40:0/2525315130 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5228079ed0 con 0x7f523009fe60 2024-04-26T21:45:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.902+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52300cea00 con 0x7f523009fe60 2024-04-26T21:45:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.902+0000 7f5217fff640 1 -- 172.21.15.40:0/2525315130 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 263 v263) ==== 95+0+5182 (secure 0 0 0) 0x7f522807e1d0 con 0x7f523009fe60 2024-04-26T21:45:20.903 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:20.903 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":263,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3791539151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3791539151},{"type":"v1","addr":"172.21.15.40:6833","nonce":3791539151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":263}],"filesystems":[{"mdsmap":{"epoch":262,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:19.814679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 263 2024-04-26T21:45:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f521c06c920 msgr2=0x7f521c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f521c06c920 0x7f521c06ede0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5220000c00 tx=0x7f5220009040 comp rx=0 tx=0).stop 2024-04-26T21:45:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523009fe60 msgr2=0x7f52300a02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523009fe60 0x7f52300a02e0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f522802b530 tx=0x7f522802d070 comp rx=0 tx=0).stop 2024-04-26T21:45:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 shutdown_connections 2024-04-26T21:45:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f521c06c920 0x7f521c06ede0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523009fe60 0x7f52300a02e0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523009d7a0 0x7f523009dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 --2- 172.21.15.40:0/2525315130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52180051c0 0x7f523015fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 >> 172.21.15.40:0/2525315130 conn(0x7f52300d2ec0 msgr2=0x7f5230071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 shutdown_connections 2024-04-26T21:45:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:20.904+0000 7f5236104640 1 -- 172.21.15.40:0/2525315130 wait complete. 2024-04-26T21:45:20.916 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds_snaptable' led to DAMAGED state 2024-04-26T21:45:20.916 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds0_inotable' 2024-04-26T21:45:20.916 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:20.916 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:20.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:45:20.932 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:20.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:20.932 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:20.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:45:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:45:20.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:21.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.130+0000 7f9aa656a640 1 Processor -- start 2024-04-26T21:45:21.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.130+0000 7f9aa656a640 1 -- start start 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.131+0000 7f9aa656a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9aa0071e60 0x7f9aa006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.131+0000 7f9aa656a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa0166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.131+0000 7f9aa656a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa015d7c0 con 0x7f9aa006fe20 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.131+0000 7f9aa656a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa015daa0 con 0x7f9aa0071e60 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.131+0000 7f9aa656a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa015dd80 con 0x7f9aa00cc480 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9aa4b07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa0166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9a9ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9aa0071e60 0x7f9aa006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9aa4b07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa0166ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:35242/0 (socket says 172.21.15.40:35242) 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9a9ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9aa0071e60 0x7f9aa006f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:44102/0 (socket says 172.21.15.40:44102) 2024-04-26T21:45:21.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9aa4b07640 1 -- 172.21.15.40:0/1625670388 learned_addr learned my addr 172.21.15.40:0/1625670388 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:21.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9a9f7fe640 1 --1- 172.21.15.40:0/1625670388 >> v1:172.21.15.184:6789/0 conn(0x7f9aa006fe20 0x7f9aa0072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:60302/0 (socket says 172.21.15.40:60302) 2024-04-26T21:45:21.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9a9ffff640 1 -- 172.21.15.40:0/1625670388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 msgr2=0x7f9aa0166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:21.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9a9ffff640 1 --2- 172.21.15.40:0/1625670388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa0166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9a9ffff640 1 -- 172.21.15.40:0/1625670388 >> v1:172.21.15.184:6789/0 conn(0x7f9aa006fe20 legacy=0x7f9aa0072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9a9ffff640 1 -- 172.21.15.40:0/1625670388 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa015e060 con 0x7f9aa0071e60 2024-04-26T21:45:21.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9a9effd640 1 -- 172.21.15.40:0/1625670388 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 69469257 0 0) 0x7f9aa015d7c0 con 0x7f9aa006fe20 2024-04-26T21:45:21.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.132+0000 7f9aa4b07640 1 --2- 172.21.15.40:0/1625670388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa0166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:21.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.133+0000 7f9a9ffff640 1 --2- 172.21.15.40:0/1625670388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 0x7f9aa006f8e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9a900097a0 tx=0x7f9a9002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba6a80ba62cd09d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:21.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.133+0000 7f9a9effd640 1 -- 172.21.15.40:0/1625670388 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90037070 con 0x7f9aa0071e60 2024-04-26T21:45:21.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.133+0000 7f9a9effd640 1 -- 172.21.15.40:0/1625670388 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a90032440 con 0x7f9aa0071e60 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.133+0000 7f9a9effd640 1 -- 172.21.15.40:0/1625670388 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90032810 con 0x7f9aa0071e60 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.133+0000 7f9aa656a640 1 -- 172.21.15.40:0/1625670388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 msgr2=0x7f9aa006f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.133+0000 7f9aa656a640 1 --2- 172.21.15.40:0/1625670388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 0x7f9aa006f8e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9a900097a0 tx=0x7f9a9002a180 comp rx=0 tx=0).stop 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.134+0000 7f9aa656a640 1 -- 172.21.15.40:0/1625670388 shutdown_connections 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.134+0000 7f9aa656a640 1 --2- 172.21.15.40:0/1625670388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa0166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.134+0000 7f9aa656a640 1 --2- 172.21.15.40:0/1625670388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 0x7f9aa006f8e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.134+0000 7f9aa656a640 1 -- 172.21.15.40:0/1625670388 >> 172.21.15.40:0/1625670388 conn(0x7f9aa00d0be0 msgr2=0x7f9aa00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.134+0000 7f9aa656a640 1 -- 172.21.15.40:0/1625670388 shutdown_connections 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.134+0000 7f9aa656a640 1 -- 172.21.15.40:0/1625670388 wait complete. 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.135+0000 7f9aa656a640 1 Processor -- start 2024-04-26T21:45:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.135+0000 7f9aa656a640 1 -- start start 2024-04-26T21:45:21.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9aa656a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9aa006fe20 0x7f9aa01561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9aa656a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 0x7f9aa00b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9aa656a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa00b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9aa656a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa015d570 con 0x7f9aa006fe20 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9aa656a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa015d3f0 con 0x7f9aa0071e60 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9aa656a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa0072950 con 0x7f9aa00cc480 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9aa006fe20 0x7f9aa01561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9aa006fe20 0x7f9aa01561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45370/0 (socket says 172.21.15.40:45370) 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 -- 172.21.15.40:0/3333022101 learned_addr learned my addr 172.21.15.40:0/3333022101 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9aa4b07640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa00b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9f7fe640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 0x7f9aa00b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 -- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 msgr2=0x7f9aa00b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa00b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 -- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 msgr2=0x7f9aa00b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 0x7f9aa00b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 -- 172.21.15.40:0/3333022101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa00aa6f0 con 0x7f9aa006fe20 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9f7fe640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 0x7f9aa00b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9ffff640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9aa006fe20 0x7f9aa01561b0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f9a90035040 tx=0x7f9a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9cff9640 1 -- 172.21.15.40:0/3333022101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90037050 con 0x7f9aa006fe20 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9a9cff9640 1 -- 172.21.15.40:0/3333022101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a90038040 con 0x7f9aa006fe20 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.136+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa01f7010 con 0x7f9aa006fe20 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.137+0000 7f9a9cff9640 1 -- 172.21.15.40:0/3333022101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90004490 con 0x7f9aa006fe20 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.137+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa01f6d30 con 0x7f9aa006fe20 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.137+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa00694f0 con 0x7f9aa006fe20 2024-04-26T21:45:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.138+0000 7f9a9cff9640 1 -- 172.21.15.40:0/3333022101 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a9003c6d0 con 0x7f9aa006fe20 2024-04-26T21:45:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.138+0000 7f9a9cff9640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.138+0000 7f9a9cff9640 1 -- 172.21.15.40:0/3333022101 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8625+0+0 (secure 0 0 0) 0x7f9a900ad0f0 con 0x7f9aa006fe20 2024-04-26T21:45:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.139+0000 7f9a9f7fe640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.139+0000 7f9a9f7fe640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00ac800 tx=0x7f9a94007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:21.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.143+0000 7f9a9cff9640 1 -- 172.21.15.40:0/3333022101 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a9007a970 con 0x7f9aa006fe20 2024-04-26T21:45:21.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.370+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9aa00b16d0 con 0x7f9aa006fe20 2024-04-26T21:45:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.824+0000 7f9a9cff9640 1 -- 172.21.15.40:0/3333022101 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v264) ==== 161+0+0 (secure 0 0 0) 0x7f9a9007a270 con 0x7f9aa006fe20 2024-04-26T21:45:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a8c06c8f0 msgr2=0x7f9a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00ac800 tx=0x7f9a94007c10 comp rx=0 tx=0).stop 2024-04-26T21:45:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9aa006fe20 msgr2=0x7f9aa01561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9aa006fe20 0x7f9aa01561b0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f9a90035040 tx=0x7f9a900046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 shutdown_connections 2024-04-26T21:45:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aa00cc480 0x7f9aa00b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa0071e60 0x7f9aa00b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 --2- 172.21.15.40:0/3333022101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9aa006fe20 0x7f9aa01561b0 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 >> 172.21.15.40:0/3333022101 conn(0x7f9aa00d0be0 msgr2=0x7f9aa015e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 shutdown_connections 2024-04-26T21:45:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.827+0000 7f9aa656a640 1 -- 172.21.15.40:0/3333022101 wait complete. 2024-04-26T21:45:21.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.942+0000 7ffa2996a640 1 Processor -- start 2024-04-26T21:45:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.942+0000 7ffa2996a640 1 -- start start 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.943+0000 7ffa2996a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffa2415f3a0 0x7ffa240d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.943+0000 7ffa2996a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa240d1770 0x7ffa240d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.943+0000 7ffa2996a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa240d25a0 con 0x7ffa240d2130 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.943+0000 7ffa2996a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffa240d0e20 con 0x7ffa2415f3a0 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.943+0000 7ffa2996a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa240b1d70 con 0x7ffa240d1770 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.943+0000 7ffa237fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffa240d2130 0x7ffa240d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44590/0 (socket says 172.21.15.40:44590) 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.943+0000 7ffa237fe640 1 -- 172.21.15.40:0/2853778287 learned_addr learned my addr 172.21.15.40:0/2853778287 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316504597 0 0) 0x7ffa14001490 con 0x7ffa240d2130 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa0c003520 con 0x7ffa240d2130 2024-04-26T21:45:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa227fc640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa240d1770 0x7ffa240d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa22ffd640 1 --2- 172.21.15.40:0/2853778287 >> v2:172.21.15.184:3300/0 conn(0x7ffa2415f3a0 0x7ffa240d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 859672484 0 0) 0x7ffa0c003520 con 0x7ffa240d2130 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa240d1770 msgr2=0x7ffa240d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa240d1770 0x7ffa240d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 >> v2:172.21.15.184:3300/0 conn(0x7ffa2415f3a0 msgr2=0x7ffa240d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 --2- 172.21.15.40:0/2853778287 >> v2:172.21.15.184:3300/0 conn(0x7ffa2415f3a0 0x7ffa240d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa240b2050 con 0x7ffa240d2130 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7ffa14001bc0 con 0x7ffa240d2130 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 >> v1:172.21.15.40:6789/0 conn(0x7ffa240d2130 legacy=0x7ffa240d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c0039b0 0x7ffa0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c004800 0x7ffa0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa240d1770 0x7ffa0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa240d0e20 con 0x7ffa0c0039b0 2024-04-26T21:45:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa240b1d70 con 0x7ffa0c004800 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c00b9c0 con 0x7ffa240d1770 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa22ffd640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c0039b0 0x7ffa0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa237fe640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa240d1770 0x7ffa0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa227fc640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c004800 0x7ffa0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa22ffd640 1 -- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa240d1770 msgr2=0x7ffa0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa22ffd640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa240d1770 0x7ffa0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa22ffd640 1 -- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c004800 msgr2=0x7ffa0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa22ffd640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c004800 0x7ffa0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.944+0000 7ffa22ffd640 1 -- 172.21.15.40:0/2853778287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c003520 con 0x7ffa0c0039b0 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.945+0000 7ffa22ffd640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c0039b0 0x7ffa0c003dd0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009ac0 tx=0x7ffa0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbcc127db6c289 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.945+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c036070 con 0x7ffa0c0039b0 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.945+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c031400 con 0x7ffa0c0039b0 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.945+0000 7ffa21ffb640 1 -- 172.21.15.40:0/2853778287 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c009690 con 0x7ffa0c0039b0 2024-04-26T21:45:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.946+0000 7ffa2996a640 1 -- 172.21.15.40:0/2853778287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c0039b0 msgr2=0x7ffa0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.946+0000 7ffa2996a640 1 --2- 172.21.15.40:0/2853778287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c0039b0 0x7ffa0c003dd0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009ac0 tx=0x7ffa0c005830 comp rx=0 tx=0).stop 2024-04-26T21:45:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.946+0000 7ffa237fe640 1 -- 172.21.15.40:0/2853778287 reap_dead start 2024-04-26T21:45:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.946+0000 7ffa2996a640 1 -- 172.21.15.40:0/2853778287 shutdown_connections 2024-04-26T21:45:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.946+0000 7ffa2996a640 1 -- 172.21.15.40:0/2853778287 >> 172.21.15.40:0/2853778287 conn(0x7ffa240d4f90 msgr2=0x7ffa240d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.946+0000 7ffa2996a640 1 -- 172.21.15.40:0/2853778287 shutdown_connections 2024-04-26T21:45:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.946+0000 7ffa2996a640 1 -- 172.21.15.40:0/2853778287 wait complete. 2024-04-26T21:45:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.947+0000 7ffa2996a640 1 Processor -- start 2024-04-26T21:45:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.947+0000 7ffa2996a640 1 -- start start 2024-04-26T21:45:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa2996a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c0039b0 0x7ffa240cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa2996a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa24156450 0x7ffa241558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa2996a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa240b8320 0x7ffa240b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa2996a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2415f900 con 0x7ffa240b8320 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa2996a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2415f780 con 0x7ffa0c0039b0 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa2996a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa240b3860 con 0x7ffa24156450 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa240b8320 0x7ffa240b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa240b8320 0x7ffa240b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45386/0 (socket says 172.21.15.40:45386) 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa237fe640 1 -- 172.21.15.40:0/3720703859 learned_addr learned my addr 172.21.15.40:0/3720703859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa227fc640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa24156450 0x7ffa241558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa22ffd640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c0039b0 0x7ffa240cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa237fe640 1 -- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa24156450 msgr2=0x7ffa241558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa237fe640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa24156450 0x7ffa241558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa237fe640 1 -- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c0039b0 msgr2=0x7ffa240cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa237fe640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c0039b0 0x7ffa240cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa237fe640 1 -- 172.21.15.40:0/3720703859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa240cc760 con 0x7ffa240b8320 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.948+0000 7ffa227fc640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa24156450 0x7ffa241558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.949+0000 7ffa237fe640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa240b8320 0x7ffa240b5b60 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7ffa240d2ae0 tx=0x7ffa14007060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.949+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3720703859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14019070 con 0x7ffa240b8320 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.949+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3720703859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa14007930 con 0x7ffa240b8320 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.949+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa240cca40 con 0x7ffa240b8320 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.949+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3720703859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa140135f0 con 0x7ffa240b8320 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.949+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa240ca870 con 0x7ffa240b8320 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.950+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa240694f0 con 0x7ffa240b8320 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.950+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3720703859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa1401c070 con 0x7ffa240b8320 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.951+0000 7ffa0bfff640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffa1006c920 0x7ffa1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.951+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3720703859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8625+0+0 (secure 0 0 0) 0x7ffa1408c300 con 0x7ffa240b8320 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.951+0000 7ffa22ffd640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffa1006c920 0x7ffa1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.952+0000 7ffa22ffd640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffa1006c920 0x7ffa1006ede0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c033000 tx=0x7ffa1c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:21.955+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3720703859 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa240d25a0 con 0x7ffa240b8320 2024-04-26T21:45:22.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.213+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ffa240caa40 con 0x7ffa240b8320 2024-04-26T21:45:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.829+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3720703859 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v265) ==== 101+0+0 (secure 0 0 0) 0x7ffa14059490 con 0x7ffa240b8320 2024-04-26T21:45:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:45:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffa1006c920 msgr2=0x7ffa1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffa1006c920 0x7ffa1006ede0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c033000 tx=0x7ffa1c008030 comp rx=0 tx=0).stop 2024-04-26T21:45:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa240b8320 msgr2=0x7ffa240b5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa240b8320 0x7ffa240b5b60 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7ffa240d2ae0 tx=0x7ffa14007060 comp rx=0 tx=0).stop 2024-04-26T21:45:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 shutdown_connections 2024-04-26T21:45:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffa1006c920 0x7ffa1006ede0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa240b8320 0x7ffa240b5b60 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa24156450 0x7ffa241558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 --2- 172.21.15.40:0/3720703859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c0039b0 0x7ffa240cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 >> 172.21.15.40:0/3720703859 conn(0x7ffa240d4f90 msgr2=0x7ffa240d3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 shutdown_connections 2024-04-26T21:45:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.832+0000 7ffa2996a640 1 -- 172.21.15.40:0/3720703859 wait complete. 2024-04-26T21:45:22.846 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.890+0000 7f9ddd921f40 1 Processor -- start 2024-04-26T21:45:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.890+0000 7f9ddd921f40 1 -- start start 2024-04-26T21:45:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddd921f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c90bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddd921f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c8423e0 0x55778c8805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddd921f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c915420 0x55778c86dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddd921f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55778c887c60 con 0x55778c83e220 2024-04-26T21:45:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddd921f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55778c7e7e40 con 0x55778c915420 2024-04-26T21:45:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddd921f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55778c83eb80 con 0x55778c8423e0 2024-04-26T21:45:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddb41d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c90bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddb41d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c90bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45410/0 (socket says 172.21.15.40:45410) 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddb41d640 1 -- 172.21.15.40:0/1824494199 learned_addr learned my addr 172.21.15.40:0/1824494199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.891+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c8423e0 0x55778c8805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddbc1e640 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c915420 0x55778c86dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddb41d640 1 -- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c8423e0 msgr2=0x55778c8805e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c8423e0 0x55778c8805e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddb41d640 1 -- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c915420 msgr2=0x55778c86dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c915420 0x55778c86dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddb41d640 1 -- 172.21.15.40:0/1824494199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778c86f680 con 0x55778c83e220 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddbc1e640 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c915420 0x55778c86dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c90bae0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400a440 tx=0x7f9dd4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8d24d530a85151 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9dd9c1a640 1 -- 172.21.15.40:0/1824494199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4056930 con 0x55778c83e220 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9dd9c1a640 1 -- 172.21.15.40:0/1824494199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd4056ad0 con 0x55778c83e220 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9dd9c1a640 1 -- 172.21.15.40:0/1824494199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4054440 con 0x55778c83e220 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddd921f40 1 -- 172.21.15.40:0/1824494199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 msgr2=0x55778c90bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.892+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c90bae0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400a440 tx=0x7f9dd4051000 comp rx=0 tx=0).stop 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 -- 172.21.15.40:0/1824494199 shutdown_connections 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c915420 0x55778c86dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c8423e0 0x55778c8805e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/1824494199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c90bae0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 -- 172.21.15.40:0/1824494199 >> 172.21.15.40:0/1824494199 conn(0x55778c83f330 msgr2=0x55778c9179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 -- 172.21.15.40:0/1824494199 shutdown_connections 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 -- 172.21.15.40:0/1824494199 wait complete. 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 Processor -- start 2024-04-26T21:45:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.893+0000 7f9ddd921f40 1 -- start start 2024-04-26T21:45:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddd921f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c913080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddd921f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c8423e0 0x55778c90f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddd921f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c915420 0x55778c86dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddd921f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55778c86f920 con 0x55778c83e220 2024-04-26T21:45:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddd921f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55778c888e70 con 0x55778c8423e0 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddd921f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55778c86fc00 con 0x55778c915420 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddb41d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c913080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddb41d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c913080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45412/0 (socket says 172.21.15.40:45412) 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 learned_addr learned my addr 172.21.15.40:0/2891345903 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddbc1e640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c915420 0x55778c86dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c8423e0 0x55778c90f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c915420 msgr2=0x55778c86dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55778c915420 0x55778c86dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c8423e0 msgr2=0x55778c90f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c8423e0 0x55778c90f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778c917fb0 con 0x55778c83e220 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.894+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55778c8423e0 0x55778c90f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.895+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c913080 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4025cc0 tx=0x7f9dd4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.895+0000 7f9dd8c18640 1 -- 172.21.15.40:0/2891345903 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4056c80 con 0x55778c83e220 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.895+0000 7f9dd8c18640 1 -- 172.21.15.40:0/2891345903 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd406c070 con 0x55778c83e220 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.895+0000 7f9dd8c18640 1 -- 172.21.15.40:0/2891345903 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4054440 con 0x55778c83e220 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.895+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55778c918290 con 0x55778c83e220 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.895+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778c9135c0 con 0x55778c83e220 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.896+0000 7f9dd8c18640 1 -- 172.21.15.40:0/2891345903 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dd406e030 con 0x55778c83e220 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.897+0000 7f9dd8c18640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dbc06b2d0 0x7f9dbc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.897+0000 7f9dd8c18640 1 -- 172.21.15.40:0/2891345903 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8625+0+0 (secure 0 0 0) 0x7f9dd40d0730 con 0x55778c83e220 2024-04-26T21:45:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.897+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dbc06b2d0 0x7f9dbc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.902 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.897+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55778c86e3e0 0x55778c90f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.897+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c90fda0 con 0x55778c86e3e0 2024-04-26T21:45:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.897+0000 7f9ddbc1e640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55778c86e3e0 0x55778c90f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.898+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dbc06b2d0 0x7f9dbc06d790 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55778c914a90 tx=0x7f9dc4001700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.898+0000 7f9ddbc1e640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55778c86e3e0 0x55778c90f0e0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.899+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v123'846 uv820 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.899+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c90ca60 con 0x55778c86e3e0 2024-04-26T21:45:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.900+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v123'847 uv847 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.900+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c90ce00 con 0x55778c86e3e0 2024-04-26T21:45:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.902+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v123'848 uv848 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.904 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.902+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c90d1d0 con 0x55778c86e3e0 2024-04-26T21:45:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.903+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v123'849 uv849 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.903+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c90d600 con 0x55778c86e3e0 2024-04-26T21:45:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.904+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v123'850 uv850 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.904+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9baa00 con 0x55778c86e3e0 2024-04-26T21:45:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.905+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v123'851 uv851 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.905+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bada0 con 0x55778c86e3e0 2024-04-26T21:45:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.906+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v123'852 uv826 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.906+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9bb140 con 0x55778c86e3e0 2024-04-26T21:45:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.907+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v123'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.907+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb140 con 0x55778c86e3e0 2024-04-26T21:45:22.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.908+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v123'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.909 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:22.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.908+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bc150 con 0x55778c86e3e0 2024-04-26T21:45:22.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.909+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v123'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.909+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bc150 con 0x55778c86e3e0 2024-04-26T21:45:22.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.910+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v123'856 uv856 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.910+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb140 con 0x55778c86e3e0 2024-04-26T21:45:22.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.911+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v123'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.911+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb140 con 0x55778c86e3e0 2024-04-26T21:45:22.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.912+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v123'858 uv833 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.912+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9bb140 con 0x55778c86e3e0 2024-04-26T21:45:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.913+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v123'859 uv859 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.913+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb140 con 0x55778c86e3e0 2024-04-26T21:45:22.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.914+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v123'860 uv860 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.915 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.915+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bc150 con 0x55778c86e3e0 2024-04-26T21:45:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.916+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v123'861 uv861 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.916+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bc150 con 0x55778c86e3e0 2024-04-26T21:45:22.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.916+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v123'862 uv862 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.917+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb140 con 0x55778c86e3e0 2024-04-26T21:45:22.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.918+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v123'863 uv863 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.918+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.919+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v123'864 uv864 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.919+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.920+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v123'865 uv839 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.920+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.921+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v123'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.921+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.923 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:22.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.922+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v123'867 uv867 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.922+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bc8b0 con 0x55778c86e3e0 2024-04-26T21:45:22.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.923+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v123'868 uv868 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.923+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bc8b0 con 0x55778c86e3e0 2024-04-26T21:45:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.924+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v123'869 uv869 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.924+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.925+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v123'870 uv870 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.925+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.926+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v123'871 uv845 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.927+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.928+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v123'872 uv872 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.928+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.929 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.929+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v123'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.929+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bc8b0 con 0x55778c86e3e0 2024-04-26T21:45:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.930+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v123'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.930+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bc8b0 con 0x55778c86e3e0 2024-04-26T21:45:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.931+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v123'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.931+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bb8a0 con 0x55778c86e3e0 2024-04-26T21:45:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.932+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v123'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c86e3e0 2024-04-26T21:45:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.932+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55778c9bbdb0 0x55778c9be1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.932+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9be6e0 con 0x55778c9bbdb0 2024-04-26T21:45:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.932+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55778c9bbdb0 0x55778c9be1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.932+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55778c9bbdb0 0x55778c9be1a0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.934+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v123'617 uv600 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.934+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9bea80 con 0x55778c9bbdb0 2024-04-26T21:45:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.936+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v123'618 uv618 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.936+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bee20 con 0x55778c9bbdb0 2024-04-26T21:45:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.937+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v123'619 uv619 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.938 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.937+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bf670 con 0x55778c9bbdb0 2024-04-26T21:45:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.938+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v123'620 uv620 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.939+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bfa10 con 0x55778c9bbdb0 2024-04-26T21:45:22.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.940+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v123'621 uv621 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.940+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9bfdb0 con 0x55778c9bbdb0 2024-04-26T21:45:22.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.942+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v123'622 uv605 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.942+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9c0150 con 0x55778c9bbdb0 2024-04-26T21:45:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.943+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v123'623 uv623 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.943+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.945 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:22.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.945+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v123'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.945+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.946+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v123'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.946+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.948+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v123'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.948+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.949+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v123'627 uv610 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.949+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.951+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v123'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.951+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.953 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:22.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.952+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v123'629 uv629 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.952+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c4bb0 con 0x55778c9bbdb0 2024-04-26T21:45:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.954+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v123'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.954+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c4bb0 con 0x55778c9bbdb0 2024-04-26T21:45:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.955+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v123'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.955+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.957+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v123'632 uv616 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.957+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.958+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v123'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.958+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.960 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:22.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.959+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v123'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.959+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c1500 con 0x55778c9bbdb0 2024-04-26T21:45:22.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.961+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v123'635 uv635 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.961+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c1500 con 0x55778c9bbdb0 2024-04-26T21:45:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.962+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v123'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.962+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c04f0 con 0x55778c9bbdb0 2024-04-26T21:45:22.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.964+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v123'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9bbdb0 2024-04-26T21:45:22.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.964+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55778c9c0b70 0x55778c9c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.964+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c3580 con 0x55778c9c0b70 2024-04-26T21:45:22.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.964+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55778c9c0b70 0x55778c9c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.965+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55778c9c0b70 0x55778c9c2f60 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:22.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.966+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v123'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:22.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.966+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9c3a40 con 0x55778c9c0b70 2024-04-26T21:45:22.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.967+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v123'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:22.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.967+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c3f20 con 0x55778c9c0b70 2024-04-26T21:45:22.969 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:22.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.968+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v123'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:22.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.968+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c5800 con 0x55778c9c0b70 2024-04-26T21:45:22.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.969+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v123'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:22.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.969+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c5ba0 con 0x55778c9c0b70 2024-04-26T21:45:22.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.970+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v123'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:22.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.970+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c47f0 con 0x55778c9c0b70 2024-04-26T21:45:22.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.971+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v123'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:22.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.972+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55778c9c50f0 0x55778c9c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.972+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c8490 con 0x55778c9c50f0 2024-04-26T21:45:22.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.972+0000 7f9ddbc1e640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55778c9c50f0 0x55778c9c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.972+0000 7f9ddbc1e640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55778c9c50f0 0x55778c9c7f50 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:22.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.973+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v123'127 uv126 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.973+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9c8490 con 0x55778c9c50f0 2024-04-26T21:45:22.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.975+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v123'128 uv128 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.975+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c84b0 con 0x55778c9c50f0 2024-04-26T21:45:22.976 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:22.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.976+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v123'129 uv129 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.976+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9c88f0 con 0x55778c9c50f0 2024-04-26T21:45:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.977+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v123'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.977+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55778c9c8e00 0x55778c9cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.977+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cb730 con 0x55778c9c8e00 2024-04-26T21:45:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.977+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55778c9c8e00 0x55778c9cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.978+0000 7f9ddb41d640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55778c9c8e00 0x55778c9cb1f0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.979+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v123'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.979+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cb7f0 con 0x55778c9c8e00 2024-04-26T21:45:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.980+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v123'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.980+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.981+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v123'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:22.982 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.981+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9ccc20 con 0x55778c9c8e00 2024-04-26T21:45:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.982+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v123'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.982+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9ccc20 con 0x55778c9c8e00 2024-04-26T21:45:22.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.983+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v123'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:22.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.984+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:22.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.985+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v123'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:22.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.985+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.986+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v123'305 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.986+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.987+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v123'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.987+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.989 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.988+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v123'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.988+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9ccc20 con 0x55778c9c50f0 2024-04-26T21:45:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.989+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v123'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.989+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9ccc20 con 0x55778c9c50f0 2024-04-26T21:45:22.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.990+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v123'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.990+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.991+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v123'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.991+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.992+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v123'311 uv304 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.992+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.993+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v123'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.993+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.995 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:22.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.994+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v123'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.994+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.995+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v123'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.996+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.996+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v123'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.997+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.998+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v123'138 uv137 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:22.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.998+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:22.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.999+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v123'139 uv139 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:22.999+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.000+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v123'140 uv140 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:23.001 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:23.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.000+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.001+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v123'141 uv141 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.001+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c50f0 2024-04-26T21:45:23.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.002+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v123'142 uv142 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dcc00d860 con 0x55778c9c50f0 2024-04-26T21:45:23.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.002+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:23.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.004+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v123'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.004+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:23.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.005+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v123'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.005+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:23.006 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.006+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v123'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.006+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9ccc20 con 0x55778c9c8e00 2024-04-26T21:45:23.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.007+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v123'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.007+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9ccc20 con 0x55778c9c8e00 2024-04-26T21:45:23.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.008+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v123'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.008+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:23.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.009+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v123'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.009+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:23.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.010+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v123'273 uv267 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.010+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:23.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.011+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v123'274 uv274 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.011+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbc10 con 0x55778c9c8e00 2024-04-26T21:45:23.013 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:23.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.012+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v123'275 uv275 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.012+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbfd0 con 0x55778c9c8e00 2024-04-26T21:45:23.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.013+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v123'276 uv276 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.013+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbfd0 con 0x55778c9c8e00 2024-04-26T21:45:23.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.014+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v123'277 uv277 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.014+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbfd0 con 0x55778c9c8e00 2024-04-26T21:45:23.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.015+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v123'278 uv272 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.016+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cbfd0 con 0x55778c9c8e00 2024-04-26T21:45:23.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.016+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v123'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.017+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbfd0 con 0x55778c9c8e00 2024-04-26T21:45:23.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.017+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v123'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.018 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:23.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.017+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cbfd0 con 0x55778c9c8e00 2024-04-26T21:45:23.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.018+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v123'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.019+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cede0 con 0x55778c9c8e00 2024-04-26T21:45:23.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.020+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v123'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.020+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55778c9cc990 0x55778c9ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.020+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf2c0 con 0x55778c9cc990 2024-04-26T21:45:23.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.020+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55778c9cc990 0x55778c9ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.020+0000 7f9ddac1c640 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55778c9cc990 0x55778c9ced80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.022+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v123'120 uv119 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9cc990 2024-04-26T21:45:23.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.022+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9cc990 2024-04-26T21:45:23.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.023+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v123'121 uv121 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9cc990 2024-04-26T21:45:23.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.023+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9cc990 2024-04-26T21:45:23.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.024+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v123'122 uv122 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9cc990 2024-04-26T21:45:23.026 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:23.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.024+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9cc990 2024-04-26T21:45:23.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.025+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v123'123 uv123 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9cc990 2024-04-26T21:45:23.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.026+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c8e00 2024-04-26T21:45:23.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.027+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v123'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.027+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c8e00 2024-04-26T21:45:23.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.028+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v123'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.028+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c8e00 2024-04-26T21:45:23.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.029+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v123'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.030 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:23.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.029+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9d0670 con 0x55778c9c8e00 2024-04-26T21:45:23.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.030+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v123'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.030+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9d0670 con 0x55778c9c8e00 2024-04-26T21:45:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.031+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v123'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.031+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c8e00 2024-04-26T21:45:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.032+0000 7f9ddb41d640 1 -- 172.21.15.40:0/2891345903 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v123'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd40a4050 con 0x55778c9c8e00 2024-04-26T21:45:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.032+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.033+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v123'140 uv139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.033+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.035+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v123'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.035+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.036 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.036+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v123'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.036+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.037+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v123'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.037+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.038+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v123'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.038+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.040+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v123'339 uv331 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.040+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.041+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v123'340 uv340 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.041+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.042+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v123'341 uv341 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.042 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:23.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.042+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.043+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v123'342 uv342 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.043+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.044+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v123'343 uv343 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.044+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.045+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v123'344 uv338 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.045+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.046+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v123'345 uv345 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.046+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.047 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:23.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.047+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v123'346 uv346 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.047+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.048+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v123'347 uv347 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.048+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9d0670 con 0x55778c9c0b70 2024-04-26T21:45:23.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.049+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v123'348 uv348 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.049+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cf660 con 0x55778c9c0b70 2024-04-26T21:45:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.049+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v123'349 uv349 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.050+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55778c9cfa50 con 0x55778c9c0b70 2024-04-26T21:45:23.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddac1c640 1 -- 172.21.15.40:0/2891345903 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v123'350 uv350 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc401a040 con 0x55778c9c0b70 2024-04-26T21:45:23.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55778c9bbdb0 msgr2=0x55778c9be1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55778c9bbdb0 0x55778c9be1a0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55778c86e3e0 msgr2=0x55778c90f0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55778c86e3e0 0x55778c90f0e0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55778c9c50f0 msgr2=0x55778c9c7f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55778c9c50f0 0x55778c9c7f50 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55778c9cc990 msgr2=0x55778c9ced80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55778c9cc990 0x55778c9ced80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55778c9c0b70 msgr2=0x55778c9c2f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55778c9c0b70 0x55778c9c2f60 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 reap_dead start 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55778c9c8e00 msgr2=0x55778c9cb1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55778c9c8e00 0x55778c9cb1f0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 reap_dead start 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddbc1e640 1 -- 172.21.15.40:0/2891345903 reap_dead start 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dbc06b2d0 msgr2=0x7f9dbc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dbc06b2d0 0x7f9dbc06d790 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55778c914a90 tx=0x7f9dc4001700 comp rx=0 tx=0).stop 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 msgr2=0x55778c913080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c913080 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4025cc0 tx=0x7f9dd4002fb0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 shutdown_connections 2024-04-26T21:45:23.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dbc06b2d0 0x7f9dbc06d790 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 --2- 172.21.15.40:0/2891345903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55778c83e220 0x55778c913080 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 >> 172.21.15.40:0/2891345903 conn(0x55778c83f330 msgr2=0x55778c913ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:23.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 shutdown_connections 2024-04-26T21:45:23.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.051+0000 7f9ddd921f40 1 -- 172.21.15.40:0/2891345903 wait complete. 2024-04-26T21:45:23.055 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds0_inotable 0 2024-04-26T21:45:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.138+0000 7f4150520f40 1 Processor -- start 2024-04-26T21:45:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.138+0000 7f4150520f40 1 -- start start 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f4150520f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41380030f0 0x560343117510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f4150520f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 0x560343117a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f4150520f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5603431b9350 0x56034324d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f4150520f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56034308ee40 con 0x7f41380030f0 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f4150520f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560343092e80 con 0x5603430e98b0 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f4150520f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5603430e9e80 con 0x5603431b9350 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f414d6ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41380030f0 0x560343117510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f414d6ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41380030f0 0x560343117510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45436/0 (socket says 172.21.15.40:45436) 2024-04-26T21:45:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f414d6ff640 1 -- 172.21.15.40:0/710657449 learned_addr learned my addr 172.21.15.40:0/710657449 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f414cefe640 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 0x560343117a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.139+0000 7f414df00640 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5603431b9350 0x56034324d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f414d6ff640 1 -- 172.21.15.40:0/710657449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5603431b9350 msgr2=0x56034324d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f414d6ff640 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5603431b9350 0x56034324d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f414d6ff640 1 -- 172.21.15.40:0/710657449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 msgr2=0x560343117a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f414d6ff640 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 0x560343117a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f414d6ff640 1 -- 172.21.15.40:0/710657449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603431b13f0 con 0x7f41380030f0 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f414d6ff640 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41380030f0 0x560343117510 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f4148009600 tx=0x7f4148009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef64fb59dfd37db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f413f7fe640 1 -- 172.21.15.40:0/710657449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414800e830 con 0x7f41380030f0 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f413f7fe640 1 -- 172.21.15.40:0/710657449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414800e9f0 con 0x7f41380030f0 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f413f7fe640 1 -- 172.21.15.40:0/710657449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414800ede0 con 0x7f41380030f0 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 -- 172.21.15.40:0/710657449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41380030f0 msgr2=0x560343117510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41380030f0 0x560343117510 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f4148009600 tx=0x7f4148009ac0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 -- 172.21.15.40:0/710657449 shutdown_connections 2024-04-26T21:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5603431b9350 0x56034324d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 0x560343117a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 --2- 172.21.15.40:0/710657449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41380030f0 0x560343117510 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 -- 172.21.15.40:0/710657449 >> 172.21.15.40:0/710657449 conn(0x5603431118a0 msgr2=0x5603430e8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 -- 172.21.15.40:0/710657449 shutdown_connections 2024-04-26T21:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.140+0000 7f4150520f40 1 -- 172.21.15.40:0/710657449 wait complete. 2024-04-26T21:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.141+0000 7f4150520f40 1 Processor -- start 2024-04-26T21:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.141+0000 7f4150520f40 1 -- start start 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f4150520f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 0x5603431c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f4150520f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603431b9350 0x5603431c0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f4150520f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41380030f0 0x5603430e8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f4150520f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560343251870 con 0x5603431b9350 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f4150520f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5603432515b0 con 0x5603430e98b0 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f4150520f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5603431b8210 con 0x7f41380030f0 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603431b9350 0x5603431c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603431b9350 0x5603431c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45452/0 (socket says 172.21.15.40:45452) 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 -- 172.21.15.40:0/3274416935 learned_addr learned my addr 172.21.15.40:0/3274416935 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414d6ff640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 0x5603431c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414df00640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41380030f0 0x5603430e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 -- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41380030f0 msgr2=0x5603430e8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41380030f0 0x5603430e8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 -- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 msgr2=0x5603431c0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603430e98b0 0x5603431c0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 -- 172.21.15.40:0/3274416935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603431bf990 con 0x5603431b9350 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f414cefe640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603431b9350 0x5603431c0430 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f4144025770 tx=0x7f4144001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.142+0000 7f413e7fc640 1 -- 172.21.15.40:0/3274416935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144063020 con 0x5603431b9350 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.143+0000 7f413e7fc640 1 -- 172.21.15.40:0/3274416935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41440559d0 con 0x5603431b9350 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.143+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603431c1660 con 0x5603431b9350 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.143+0000 7f413e7fc640 1 -- 172.21.15.40:0/3274416935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41440014d0 con 0x5603431b9350 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.143+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603431c1940 con 0x5603431b9350 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.144+0000 7f413e7fc640 1 -- 172.21.15.40:0/3274416935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4144066020 con 0x5603431b9350 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.144+0000 7f413e7fc640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f413006b2d0 0x7f413006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.144+0000 7f413e7fc640 1 -- 172.21.15.40:0/3274416935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8625+0+0 (secure 0 0 0) 0x7f41440ce970 con 0x5603431b9350 2024-04-26T21:45:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.145+0000 7f414d6ff640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f413006b2d0 0x7f413006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.145+0000 7f4150520f40 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5603431c0970 0x5603431b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.145+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5603430e8c50 con 0x5603431c0970 2024-04-26T21:45:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.145+0000 7f414df00640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5603431c0970 0x5603431b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.145+0000 7f414d6ff640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f413006b2d0 0x7f413006d790 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4148009b00 tx=0x7f414800b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.145+0000 7f414df00640 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5603431c0970 0x5603431b2960 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.146+0000 7f414df00640 1 -- 172.21.15.40:0/3274416935 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 mds0_inotable [truncate 0] v123'283 uv283 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4140012900 con 0x5603431c0970 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5603431c0970 msgr2=0x5603431b2960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x5603431c0970 0x5603431b2960 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f413006b2d0 msgr2=0x7f413006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f413006b2d0 0x7f413006d790 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4148009b00 tx=0x7f414800b000 comp rx=0 tx=0).stop 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603431b9350 msgr2=0x5603431c0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 --2- 172.21.15.40:0/3274416935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5603431b9350 0x5603431c0430 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f4144025770 tx=0x7f4144001170 comp rx=0 tx=0).stop 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f414df00640 1 -- 172.21.15.40:0/3274416935 reap_dead start 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 shutdown_connections 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 >> 172.21.15.40:0/3274416935 conn(0x5603431118a0 msgr2=0x5603431ba0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 shutdown_connections 2024-04-26T21:45:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.147+0000 7f4150520f40 1 -- 172.21.15.40:0/3274416935 wait complete. 2024-04-26T21:45:23.151 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.254+0000 7fc17edf0640 1 Processor -- start 2024-04-26T21:45:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.254+0000 7fc17edf0640 1 -- start start 2024-04-26T21:45:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17edf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1780d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17edf0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1780b40d0 0x7fc1780b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17edf0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1780d0a10 con 0x7fc1780d2320 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17edf0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc1780d0cf0 con 0x7fc1780b40d0 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17edf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1780d0fd0 con 0x7fc1780d1a00 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17cb8c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1780d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17d38d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1780b40d0 0x7fc1780b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17cb8c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1780d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:35322/0 (socket says 172.21.15.40:35322) 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc177fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fc1780d2320 0x7fc1781625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:60320/0 (socket says 172.21.15.40:60320) 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17d38d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1780b40d0 0x7fc1780b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:44190/0 (socket says 172.21.15.40:44190) 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17cb8c640 1 -- 172.21.15.40:0/3475756557 learned_addr learned my addr 172.21.15.40:0/3475756557 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17cb8c640 1 -- 172.21.15.40:0/3475756557 >> v2:172.21.15.184:3300/0 conn(0x7fc1780b40d0 msgr2=0x7fc1780b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17cb8c640 1 --2- 172.21.15.40:0/3475756557 >> v2:172.21.15.184:3300/0 conn(0x7fc1780b40d0 0x7fc1780b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17cb8c640 1 -- 172.21.15.40:0/3475756557 >> v1:172.21.15.184:6789/0 conn(0x7fc1780d2320 legacy=0x7fc1781625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc1777fe640 1 -- 172.21.15.40:0/3475756557 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3889538103 0 0) 0x7fc1780d0a10 con 0x7fc1780d2320 2024-04-26T21:45:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17cb8c640 1 -- 172.21.15.40:0/3475756557 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1780d40a0 con 0x7fc1780d1a00 2024-04-26T21:45:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.255+0000 7fc17d38d640 1 --2- 172.21.15.40:0/3475756557 >> v2:172.21.15.184:3300/0 conn(0x7fc1780b40d0 0x7fc1780b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.256+0000 7fc17cb8c640 1 --2- 172.21.15.40:0/3475756557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1780d1de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009870 tx=0x7fc16c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b8cade8d3ad6b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.256+0000 7fc1777fe640 1 -- 172.21.15.40:0/3475756557 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c037070 con 0x7fc1780d1a00 2024-04-26T21:45:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.256+0000 7fc1777fe640 1 -- 172.21.15.40:0/3475756557 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc16c032440 con 0x7fc1780d1a00 2024-04-26T21:45:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.256+0000 7fc1777fe640 1 -- 172.21.15.40:0/3475756557 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c032810 con 0x7fc1780d1a00 2024-04-26T21:45:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.257+0000 7fc17edf0640 1 -- 172.21.15.40:0/3475756557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 msgr2=0x7fc1780d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.257+0000 7fc17edf0640 1 --2- 172.21.15.40:0/3475756557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1780d1de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009870 tx=0x7fc16c02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.257+0000 7fc17edf0640 1 -- 172.21.15.40:0/3475756557 shutdown_connections 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.257+0000 7fc17edf0640 1 --2- 172.21.15.40:0/3475756557 >> v2:172.21.15.184:3300/0 conn(0x7fc1780b40d0 0x7fc1780b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.257+0000 7fc17edf0640 1 --2- 172.21.15.40:0/3475756557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1780d1de0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.257+0000 7fc17edf0640 1 -- 172.21.15.40:0/3475756557 >> 172.21.15.40:0/3475756557 conn(0x7fc1780d4fd0 msgr2=0x7fc1780d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.257+0000 7fc17edf0640 1 -- 172.21.15.40:0/3475756557 shutdown_connections 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.257+0000 7fc17edf0640 1 -- 172.21.15.40:0/3475756557 wait complete. 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.258+0000 7fc17edf0640 1 Processor -- start 2024-04-26T21:45:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.258+0000 7fc17edf0640 1 -- start start 2024-04-26T21:45:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17edf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1780b40d0 0x7fc17814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17edf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1781512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17edf0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1780d2320 0x7fc178151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17edf0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780b5040 con 0x7fc1780d2320 2024-04-26T21:45:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17edf0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc178162f50 con 0x7fc1780b40d0 2024-04-26T21:45:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17edf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1780b51c0 con 0x7fc1780d1a00 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17d38d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1780d2320 0x7fc178151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17d38d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1780d2320 0x7fc178151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45464/0 (socket says 172.21.15.40:45464) 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17d38d640 1 -- 172.21.15.40:0/217031325 learned_addr learned my addr 172.21.15.40:0/217031325 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17cb8c640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1780b40d0 0x7fc17814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc177fff640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1781512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17d38d640 1 -- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 msgr2=0x7fc1781512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17d38d640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1781512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17d38d640 1 -- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1780b40d0 msgr2=0x7fc17814f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17d38d640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1780b40d0 0x7fc17814f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17d38d640 1 -- 172.21.15.40:0/217031325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1780c8bb0 con 0x7fc1780d2320 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc177fff640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1781512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.259+0000 7fc17cb8c640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1780b40d0 0x7fc17814f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.260+0000 7fc17d38d640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1780d2320 0x7fc178151800 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fc17000efc0 tx=0x7fc170009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.260+0000 7fc1757fa640 1 -- 172.21.15.40:0/217031325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170018030 con 0x7fc1780d2320 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.260+0000 7fc1757fa640 1 -- 172.21.15.40:0/217031325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc17000cde0 con 0x7fc1780d2320 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.260+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1780ca5c0 con 0x7fc1780d2320 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.260+0000 7fc1757fa640 1 -- 172.21.15.40:0/217031325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170003bb0 con 0x7fc1780d2320 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.260+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17815e8c0 con 0x7fc1780d2320 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.261+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1780ca790 con 0x7fc1780d2320 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.261+0000 7fc1757fa640 1 -- 172.21.15.40:0/217031325 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc17001b070 con 0x7fc1780d2320 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.262+0000 7fc1757fa640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc15c06c8f0 0x7fc15c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.262+0000 7fc1757fa640 1 -- 172.21.15.40:0/217031325 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8625+0+0 (secure 0 0 0) 0x7fc17008d1b0 con 0x7fc1780d2320 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.262+0000 7fc17cb8c640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc15c06c8f0 0x7fc15c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.262+0000 7fc17cb8c640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc15c06c8f0 0x7fc15c06edb0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc16c034000 tx=0x7fc16c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.266+0000 7fc1757fa640 1 -- 172.21.15.40:0/217031325 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc170010040 con 0x7fc1780d2320 2024-04-26T21:45:23.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.498+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc1780694f0 con 0x7fc1780d2320 2024-04-26T21:45:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.831+0000 7fc1757fa640 1 -- 172.21.15.40:0/217031325 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v266) ==== 162+0+0 (secure 0 0 0) 0x7fc17005aa20 con 0x7fc1780d2320 2024-04-26T21:45:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc15c06c8f0 msgr2=0x7fc15c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc15c06c8f0 0x7fc15c06edb0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc16c034000 tx=0x7fc16c02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1780d2320 msgr2=0x7fc178151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1780d2320 0x7fc178151800 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fc17000efc0 tx=0x7fc170009f90 comp rx=0 tx=0).stop 2024-04-26T21:45:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 shutdown_connections 2024-04-26T21:45:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc15c06c8f0 0x7fc15c06edb0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1780d2320 0x7fc178151800 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1780d1a00 0x7fc1781512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 --2- 172.21.15.40:0/217031325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1780b40d0 0x7fc17814f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 >> 172.21.15.40:0/217031325 conn(0x7fc1780d4fd0 msgr2=0x7fc1780b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 shutdown_connections 2024-04-26T21:45:23.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.833+0000 7fc17edf0640 1 -- 172.21.15.40:0/217031325 wait complete. 2024-04-26T21:45:23.842 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:23.842+0000 7fa49b8ab640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-04-26T21:45:23.843 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:23.842+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:45:23.843 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:23.842+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:45:23.847 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:23.848 INFO:tasks.ceph.mds.c.smithi040.stderr: -29> 2024-04-26T21:45:23.842+0000 7fa49b8ab640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-04-26T21:45:23.872 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:45:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.948+0000 7fa452f79640 1 Processor -- start 2024-04-26T21:45:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.948+0000 7fa452f79640 1 -- start start 2024-04-26T21:45:23.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa452f79640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa44c0d0070 0x7fa44c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa452f79640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa452f79640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44c0d2420 con 0x7fa44c06e0a0 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa452f79640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa44c162d00 con 0x7fa44c0d0070 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa452f79640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c0d2d10 con 0x7fa44c06d560 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa450d15640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa44c0d0070 0x7fa44c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa451516640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa44c06e0a0 0x7fa44c162f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44648/0 (socket says 172.21.15.40:44648) 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa451516640 1 -- 172.21.15.40:0/734372076 learned_addr learned my addr 172.21.15.40:0/734372076 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.949+0000 7fa44bfff640 1 --2- 172.21.15.40:0/734372076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.950+0000 7fa450d15640 1 -- 172.21.15.40:0/734372076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 msgr2=0x7fa44c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.950+0000 7fa450d15640 1 --2- 172.21.15.40:0/734372076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.950+0000 7fa450d15640 1 -- 172.21.15.40:0/734372076 >> v1:172.21.15.40:6789/0 conn(0x7fa44c06e0a0 legacy=0x7fa44c162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.950+0000 7fa450d15640 1 -- 172.21.15.40:0/734372076 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c0cd6b0 con 0x7fa44c0d0070 2024-04-26T21:45:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.950+0000 7fa44b7fe640 1 -- 172.21.15.40:0/734372076 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217555250 0 0) 0x7fa44c0d2420 con 0x7fa44c06e0a0 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.950+0000 7fa450d15640 1 --2- 172.21.15.40:0/734372076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c0d1d40 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fa444009d90 tx=0x7fa44402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f0819b4bd8c100 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa44b7fe640 1 -- 172.21.15.40:0/734372076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444007740 con 0x7fa44c0d0070 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa44b7fe640 1 -- 172.21.15.40:0/734372076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4440078e0 con 0x7fa44c0d0070 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa44b7fe640 1 -- 172.21.15.40:0/734372076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444007cb0 con 0x7fa44c0d0070 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa452f79640 1 -- 172.21.15.40:0/734372076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 msgr2=0x7fa44c0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa452f79640 1 --2- 172.21.15.40:0/734372076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c0d1d40 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fa444009d90 tx=0x7fa44402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa452f79640 1 -- 172.21.15.40:0/734372076 shutdown_connections 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa452f79640 1 --2- 172.21.15.40:0/734372076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa452f79640 1 --2- 172.21.15.40:0/734372076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c0d1d40 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa452f79640 1 -- 172.21.15.40:0/734372076 >> 172.21.15.40:0/734372076 conn(0x7fa44c0d2ec0 msgr2=0x7fa44c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:23.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa452f79640 1 -- 172.21.15.40:0/734372076 shutdown_connections 2024-04-26T21:45:23.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.951+0000 7fa452f79640 1 -- 172.21.15.40:0/734372076 wait complete. 2024-04-26T21:45:23.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.952+0000 7fa452f79640 1 Processor -- start 2024-04-26T21:45:23.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.952+0000 7fa452f79640 1 -- start start 2024-04-26T21:45:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa452f79640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c1f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa452f79640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c06e0a0 0x7fa44c160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa452f79640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa452f79640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c0cf1a0 con 0x7fa44c0d0070 2024-04-26T21:45:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa452f79640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c0cdc70 con 0x7fa44c06e0a0 2024-04-26T21:45:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa452f79640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c0cf320 con 0x7fa44c06d560 2024-04-26T21:45:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa451516640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa451516640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c161430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45506/0 (socket says 172.21.15.40:45506) 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.953+0000 7fa451516640 1 -- 172.21.15.40:0/2516101413 learned_addr learned my addr 172.21.15.40:0/2516101413 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa450d15640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c1f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa44bfff640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c06e0a0 0x7fa44c160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa451516640 1 -- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 msgr2=0x7fa44c1f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa451516640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c1f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa451516640 1 -- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c06e0a0 msgr2=0x7fa44c160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa451516640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c06e0a0 0x7fa44c160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa451516640 1 -- 172.21.15.40:0/2516101413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c1f6280 con 0x7fa44c0d0070 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa450d15640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c1f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa44bfff640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c06e0a0 0x7fa44c160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa451516640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c161430 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00c410 tx=0x7fa43c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa4497fa640 1 -- 172.21.15.40:0/2516101413 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0074e0 con 0x7fa44c0d0070 2024-04-26T21:45:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa4497fa640 1 -- 172.21.15.40:0/2516101413 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c0047e0 con 0x7fa44c0d0070 2024-04-26T21:45:23.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c0c6c90 con 0x7fa44c0d0070 2024-04-26T21:45:23.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa4497fa640 1 -- 172.21.15.40:0/2516101413 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c004bd0 con 0x7fa44c0d0070 2024-04-26T21:45:23.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.954+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c0c7250 con 0x7fa44c0d0070 2024-04-26T21:45:23.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.955+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44c0694f0 con 0x7fa44c0d0070 2024-04-26T21:45:23.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.956+0000 7fa4497fa640 1 -- 172.21.15.40:0/2516101413 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa43c011070 con 0x7fa44c0d0070 2024-04-26T21:45:23.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.956+0000 7fa4497fa640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806c920 0x7fa43806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:23.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.956+0000 7fa4497fa640 1 -- 172.21.15.40:0/2516101413 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8711+0+0 (secure 0 0 0) 0x7fa43c08c300 con 0x7fa44c0d0070 2024-04-26T21:45:23.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.957+0000 7fa450d15640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806c920 0x7fa43806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:23.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.957+0000 7fa450d15640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806c920 0x7fa43806ede0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa444034000 tx=0x7fa44402f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:23.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:23.961+0000 7fa4497fa640 1 -- 172.21.15.40:0/2516101413 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa43c059b20 con 0x7fa44c0d0070 2024-04-26T21:45:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.191+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa44c0c7be0 con 0x7fa44c0d0070 2024-04-26T21:45:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.191+0000 7fa4497fa640 1 -- 172.21.15.40:0/2516101413 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 268 v268) ==== 95+0+4355 (secure 0 0 0) 0x7fa43c059420 con 0x7fa44c0d0070 2024-04-26T21:45:24.192 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:24.192 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":268,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":268,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:23.843970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 268 2024-04-26T21:45:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806c920 msgr2=0x7fa43806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806c920 0x7fa43806ede0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa444034000 tx=0x7fa44402f000 comp rx=0 tx=0).stop 2024-04-26T21:45:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 msgr2=0x7fa44c161430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c161430 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00c410 tx=0x7fa43c00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 shutdown_connections 2024-04-26T21:45:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806c920 0x7fa43806ede0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa44c0d0070 0x7fa44c161430 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c06e0a0 0x7fa44c160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 --2- 172.21.15.40:0/2516101413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c06d560 0x7fa44c1f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 >> 172.21.15.40:0/2516101413 conn(0x7fa44c0d2ec0 msgr2=0x7fa44c0b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 shutdown_connections 2024-04-26T21:45:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.193+0000 7fa452f79640 1 -- 172.21.15.40:0/2516101413 wait complete. 2024-04-26T21:45:24.205 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:24.205 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:24.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.345+0000 7f5e3447b640 1 Processor -- start 2024-04-26T21:45:24.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.345+0000 7f5e3447b640 1 -- start start 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e3447b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e2c0b2340 0x7f5e2c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e3447b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2c0d10d0 con 0x7f5e2c162810 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e3447b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2c0d1370 con 0x7f5e2c0cd930 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e3447b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5e2c0d0750 con 0x7f5e2c0b2340 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e31a16640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e2c0b2340 0x7f5e2c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e32217640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5e2c162810 0x7f5e2c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44664/0 (socket says 172.21.15.40:44664) 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e32217640 1 -- 172.21.15.40:0/3963068098 learned_addr learned my addr 172.21.15.40:0/3963068098 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e31a16640 1 -- 172.21.15.40:0/3963068098 >> v1:172.21.15.184:6789/0 conn(0x7f5e2c0cd930 legacy=0x7f5e2c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e31a16640 1 -- 172.21.15.40:0/3963068098 >> v1:172.21.15.40:6789/0 conn(0x7f5e2c162810 legacy=0x7f5e2c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.346+0000 7f5e31a16640 1 -- 172.21.15.40:0/3963068098 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c0d01b0 con 0x7f5e2c0b2340 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e31a16640 1 --2- 172.21.15.40:0/3963068098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0cd3f0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0d1540 tx=0x7f5e2002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7955ff70176bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e31215640 1 -- 172.21.15.40:0/3963068098 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20037070 con 0x7f5e2c0b2340 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e31215640 1 -- 172.21.15.40:0/3963068098 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e20032440 con 0x7f5e2c0b2340 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e31215640 1 -- 172.21.15.40:0/3963068098 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20032810 con 0x7f5e2c0b2340 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e3447b640 1 -- 172.21.15.40:0/3963068098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 msgr2=0x7f5e2c0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e3447b640 1 --2- 172.21.15.40:0/3963068098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0cd3f0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0d1540 tx=0x7f5e2002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e3447b640 1 -- 172.21.15.40:0/3963068098 shutdown_connections 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e3447b640 1 --2- 172.21.15.40:0/3963068098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0cd3f0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.347+0000 7f5e3447b640 1 -- 172.21.15.40:0/3963068098 >> 172.21.15.40:0/3963068098 conn(0x7f5e2c0d2da0 msgr2=0x7f5e2c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.348+0000 7f5e3447b640 1 -- 172.21.15.40:0/3963068098 shutdown_connections 2024-04-26T21:45:24.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.348+0000 7f5e3447b640 1 -- 172.21.15.40:0/3963068098 wait complete. 2024-04-26T21:45:24.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e3447b640 1 Processor -- start 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e3447b640 1 -- start start 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e3447b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e3447b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2c0cd930 0x7f5e2c14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e3447b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2c162810 0x7f5e2c15d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e3447b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c0d2240 con 0x7f5e2c0b2340 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e3447b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c0d16a0 con 0x7f5e2c162810 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e3447b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c0d23c0 con 0x7f5e2c0cd930 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e32217640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e32217640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45526/0 (socket says 172.21.15.40:45526) 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.349+0000 7f5e32217640 1 -- 172.21.15.40:0/4253010547 learned_addr learned my addr 172.21.15.40:0/4253010547 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e31a16640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2c0cd930 0x7f5e2c14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e32a18640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2c162810 0x7f5e2c15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e32217640 1 -- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2c0cd930 msgr2=0x7f5e2c14e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e32217640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2c0cd930 0x7f5e2c14e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e32217640 1 -- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2c162810 msgr2=0x7f5e2c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e32217640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2c162810 0x7f5e2c15d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e32217640 1 -- 172.21.15.40:0/4253010547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c1588e0 con 0x7f5e2c0b2340 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e31a16640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2c0cd930 0x7f5e2c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e32217640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0c7bb0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400c3f0 tx=0x7f5e2400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e2affd640 1 -- 172.21.15.40:0/4253010547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24017070 con 0x7f5e2c0b2340 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e2affd640 1 -- 172.21.15.40:0/4253010547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e24005ce0 con 0x7f5e2c0b2340 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c14e730 con 0x7f5e2c0b2340 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e2affd640 1 -- 172.21.15.40:0/4253010547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e240072f0 con 0x7f5e2c0b2340 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.350+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c0c8990 con 0x7f5e2c0b2340 2024-04-26T21:45:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.351+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e2c0694f0 con 0x7f5e2c0b2340 2024-04-26T21:45:24.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.352+0000 7f5e2affd640 1 -- 172.21.15.40:0/4253010547 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e2401a070 con 0x7f5e2c0b2340 2024-04-26T21:45:24.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.352+0000 7f5e2affd640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5e1806c8f0 0x7f5e1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.352+0000 7f5e2affd640 1 -- 172.21.15.40:0/4253010547 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8711+0+0 (secure 0 0 0) 0x7f5e2408ba50 con 0x7f5e2c0b2340 2024-04-26T21:45:24.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.352+0000 7f5e31a16640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5e1806c8f0 0x7f5e1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.353+0000 7f5e31a16640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5e1806c8f0 0x7f5e1806edb0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5e20034000 tx=0x7f5e2002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:24.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.356+0000 7f5e2affd640 1 -- 172.21.15.40:0/4253010547 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e2c14e730 con 0x7f5e2c0b2340 2024-04-26T21:45:24.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.582+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e2c0cd710 con 0x7f5e2c0b2340 2024-04-26T21:45:24.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.582+0000 7f5e2affd640 1 -- 172.21.15.40:0/4253010547 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 268 v268) ==== 95+0+4355 (secure 0 0 0) 0x7f5e24059300 con 0x7f5e2c0b2340 2024-04-26T21:45:24.583 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:24.583 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":268,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":268,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:23.843970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:24.584 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 268 2024-04-26T21:45:24.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5e1806c8f0 msgr2=0x7f5e1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5e1806c8f0 0x7f5e1806edb0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5e20034000 tx=0x7f5e2002e000 comp rx=0 tx=0).stop 2024-04-26T21:45:24.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 msgr2=0x7f5e2c0c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0c7bb0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400c3f0 tx=0x7f5e2400c8b0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 shutdown_connections 2024-04-26T21:45:24.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5e1806c8f0 0x7f5e1806edb0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e2c162810 0x7f5e2c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2c0cd930 0x7f5e2c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 --2- 172.21.15.40:0/4253010547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c0b2340 0x7f5e2c0c7bb0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 >> 172.21.15.40:0/4253010547 conn(0x7f5e2c0d2da0 msgr2=0x7f5e2c0d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:24.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 shutdown_connections 2024-04-26T21:45:24.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.584+0000 7f5e3447b640 1 -- 172.21.15.40:0/4253010547 wait complete. 2024-04-26T21:45:24.596 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:24.597 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:24.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.697+0000 7f47c3cd1640 1 Processor -- start 2024-04-26T21:45:24.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.697+0000 7f47c3cd1640 1 -- start start 2024-04-26T21:45:24.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c3cd1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47bc162880 0x7f47bc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c3cd1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f47bc0b5f20 0x7f47bc0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c3cd1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47bc0b7e50 con 0x7f47bc0d39f0 2024-04-26T21:45:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c3cd1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f47bc0d3e40 con 0x7f47bc162880 2024-04-26T21:45:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c3cd1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f47bc0d0580 con 0x7f47bc0b5f20 2024-04-26T21:45:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c1a6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47bc162880 0x7f47bc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c1a6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47bc162880 0x7f47bc0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45540/0 (socket says 172.21.15.40:45540) 2024-04-26T21:45:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c1a6d640 1 -- 172.21.15.40:0/376408879 learned_addr learned my addr 172.21.15.40:0/376408879 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c126c640 1 --2- 172.21.15.40:0/376408879 >> v2:172.21.15.184:3300/0 conn(0x7f47bc0b5f20 0x7f47bc0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c1a6d640 1 -- 172.21.15.40:0/376408879 >> v2:172.21.15.184:3300/0 conn(0x7f47bc0b5f20 msgr2=0x7f47bc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c1a6d640 1 --2- 172.21.15.40:0/376408879 >> v2:172.21.15.184:3300/0 conn(0x7f47bc0b5f20 0x7f47bc0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c1a6d640 1 -- 172.21.15.40:0/376408879 >> v1:172.21.15.184:6789/0 conn(0x7f47bc0d39f0 legacy=0x7f47bc0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c1a6d640 1 -- 172.21.15.40:0/376408879 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc0d1800 con 0x7f47bc162880 2024-04-26T21:45:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.698+0000 7f47c126c640 1 --2- 172.21.15.40:0/376408879 >> v2:172.21.15.184:3300/0 conn(0x7f47bc0b5f20 0x7f47bc0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.699+0000 7f47c1a6d640 1 --2- 172.21.15.40:0/376408879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0d2cb0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f47bc08d3d0 tx=0x7f47b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80b0f963c2efceb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.699+0000 7f47c0a6b640 1 -- 172.21.15.40:0/376408879 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8037070 con 0x7f47bc162880 2024-04-26T21:45:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.699+0000 7f47c0a6b640 1 -- 172.21.15.40:0/376408879 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b8032440 con 0x7f47bc162880 2024-04-26T21:45:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.699+0000 7f47c0a6b640 1 -- 172.21.15.40:0/376408879 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8032810 con 0x7f47bc162880 2024-04-26T21:45:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.700+0000 7f47c3cd1640 1 -- 172.21.15.40:0/376408879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 msgr2=0x7f47bc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.700+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/376408879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0d2cb0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f47bc08d3d0 tx=0x7f47b802a230 comp rx=0 tx=0).stop 2024-04-26T21:45:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.700+0000 7f47c3cd1640 1 -- 172.21.15.40:0/376408879 shutdown_connections 2024-04-26T21:45:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.700+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/376408879 >> v2:172.21.15.184:3300/0 conn(0x7f47bc0b5f20 0x7f47bc0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.700+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/376408879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0d2cb0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.700+0000 7f47c3cd1640 1 -- 172.21.15.40:0/376408879 >> 172.21.15.40:0/376408879 conn(0x7f47bc0d4f60 msgr2=0x7f47bc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.700+0000 7f47c3cd1640 1 -- 172.21.15.40:0/376408879 shutdown_connections 2024-04-26T21:45:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.700+0000 7f47c3cd1640 1 -- 172.21.15.40:0/376408879 wait complete. 2024-04-26T21:45:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.701+0000 7f47c3cd1640 1 Processor -- start 2024-04-26T21:45:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.701+0000 7f47c3cd1640 1 -- start start 2024-04-26T21:45:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c3cd1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47bc0b5f20 0x7f47bc153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c3cd1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47bc0d39f0 0x7f47bc0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c3cd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c3cd1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc0d0d90 con 0x7f47bc162880 2024-04-26T21:45:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c3cd1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc0d0ab0 con 0x7f47bc0d39f0 2024-04-26T21:45:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c3cd1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc162c60 con 0x7f47bc0b5f20 2024-04-26T21:45:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45546/0 (socket says 172.21.15.40:45546) 2024-04-26T21:45:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 -- 172.21.15.40:0/2015210021 learned_addr learned my addr 172.21.15.40:0/2015210021 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c126c640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47bc0d39f0 0x7f47bc0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c1a6d640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47bc0b5f20 0x7f47bc153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 -- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47bc0b5f20 msgr2=0x7f47bc153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47bc0b5f20 0x7f47bc153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 -- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47bc0d39f0 msgr2=0x7f47bc0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47bc0d39f0 0x7f47bc0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 -- 172.21.15.40:0/2015210021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc0b26e0 con 0x7f47bc162880 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c126c640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47bc0d39f0 0x7f47bc0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c226e640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0b5280 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f47b400b990 tx=0x7f47b400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47b27fc640 1 -- 172.21.15.40:0/2015210021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4019070 con 0x7f47bc162880 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47b27fc640 1 -- 172.21.15.40:0/2015210021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b400cba0 con 0x7f47bc162880 2024-04-26T21:45:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47b27fc640 1 -- 172.21.15.40:0/2015210021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4014440 con 0x7f47bc162880 2024-04-26T21:45:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc0b29c0 con 0x7f47bc162880 2024-04-26T21:45:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.702+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc0b2f80 con 0x7f47bc162880 2024-04-26T21:45:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.703+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47bc0694f0 con 0x7f47bc162880 2024-04-26T21:45:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.704+0000 7f47b27fc640 1 -- 172.21.15.40:0/2015210021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47b401c070 con 0x7f47bc162880 2024-04-26T21:45:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.704+0000 7f47b27fc640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47a006c8f0 0x7f47a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.704+0000 7f47b27fc640 1 -- 172.21.15.40:0/2015210021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8711+0+0 (secure 0 0 0) 0x7f47b408c410 con 0x7f47bc162880 2024-04-26T21:45:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.704+0000 7f47c1a6d640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47a006c8f0 0x7f47a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.705+0000 7f47c1a6d640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47a006c8f0 0x7f47a006edb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f47b8034000 tx=0x7f47b802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:24.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.708+0000 7f47b27fc640 1 -- 172.21.15.40:0/2015210021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47b4059cc0 con 0x7f47bc162880 2024-04-26T21:45:24.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.934+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47bc0d1540 con 0x7f47bc162880 2024-04-26T21:45:24.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.934+0000 7f47b27fc640 1 -- 172.21.15.40:0/2015210021 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 269 v269) ==== 95+0+5182 (secure 0 0 0) 0x7f47b40595c0 con 0x7f47bc162880 2024-04-26T21:45:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:24.935 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":269,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4173136579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4173136579},{"type":"v1","addr":"172.21.15.40:6833","nonce":4173136579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":269}],"filesystems":[{"mdsmap":{"epoch":268,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:23.843970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:24.936 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 269 2024-04-26T21:45:24.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47a006c8f0 msgr2=0x7f47a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47a006c8f0 0x7f47a006edb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f47b8034000 tx=0x7f47b802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:24.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 msgr2=0x7f47bc0b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:24.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0b5280 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f47b400b990 tx=0x7f47b400be50 comp rx=0 tx=0).stop 2024-04-26T21:45:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 shutdown_connections 2024-04-26T21:45:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47a006c8f0 0x7f47a006edb0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47bc162880 0x7f47bc0b5280 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47bc0d39f0 0x7f47bc0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 --2- 172.21.15.40:0/2015210021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47bc0b5f20 0x7f47bc153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 >> 172.21.15.40:0/2015210021 conn(0x7f47bc0d4f60 msgr2=0x7f47bc0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 shutdown_connections 2024-04-26T21:45:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:24.936+0000 7f47c3cd1640 1 -- 172.21.15.40:0/2015210021 wait complete. 2024-04-26T21:45:24.948 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds0_inotable' led to DAMAGED state 2024-04-26T21:45:24.948 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 400.00000000' 2024-04-26T21:45:24.948 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:24.948 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:24.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:45:24.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:24.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:24.965 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:25.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:45:25.020 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:45:25.020 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:25.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.161+0000 7f04eed9e640 1 Processor -- start 2024-04-26T21:45:25.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.161+0000 7f04eed9e640 1 -- start start 2024-04-26T21:45:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04eed9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04e80d17d0 0x7f04e80b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04eed9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d2400 0x7f04e80d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04eed9e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e80b66c0 con 0x7f04e80d0c80 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04eed9e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f04e80d37b0 con 0x7f04e80d17d0 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04eed9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04e80d3d70 con 0x7f04e80d2400 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04edd9c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04e80d17d0 0x7f04e80b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04edd9c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04e80d17d0 0x7f04e80b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45560/0 (socket says 172.21.15.40:45560) 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04edd9c640 1 -- 172.21.15.40:0/3345358500 learned_addr learned my addr 172.21.15.40:0/3345358500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04ed59b640 1 --2- 172.21.15.40:0/3345358500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d2400 0x7f04e80d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04edd9c640 1 -- 172.21.15.40:0/3345358500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d2400 msgr2=0x7f04e80d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04edd9c640 1 --2- 172.21.15.40:0/3345358500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d2400 0x7f04e80d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.162+0000 7f04edd9c640 1 -- 172.21.15.40:0/3345358500 >> v1:172.21.15.184:6789/0 conn(0x7f04e80d0c80 legacy=0x7f04e80b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.163+0000 7f04edd9c640 1 -- 172.21.15.40:0/3345358500 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e80d20f0 con 0x7f04e80d17d0 2024-04-26T21:45:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.163+0000 7f04edd9c640 1 --2- 172.21.15.40:0/3345358500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d17d0 0x7f04e80b6180 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009870 tx=0x7f04e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bff9f168689f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.163+0000 7f04ecd9a640 1 -- 172.21.15.40:0/3345358500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4037070 con 0x7f04e80d17d0 2024-04-26T21:45:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.163+0000 7f04ecd9a640 1 -- 172.21.15.40:0/3345358500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e4032440 con 0x7f04e80d17d0 2024-04-26T21:45:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.163+0000 7f04ecd9a640 1 -- 172.21.15.40:0/3345358500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4032810 con 0x7f04e80d17d0 2024-04-26T21:45:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.164+0000 7f04eed9e640 1 -- 172.21.15.40:0/3345358500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d17d0 msgr2=0x7f04e80b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.164+0000 7f04eed9e640 1 --2- 172.21.15.40:0/3345358500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d17d0 0x7f04e80b6180 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009870 tx=0x7f04e402a180 comp rx=0 tx=0).stop 2024-04-26T21:45:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.164+0000 7f04eed9e640 1 -- 172.21.15.40:0/3345358500 shutdown_connections 2024-04-26T21:45:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.164+0000 7f04eed9e640 1 --2- 172.21.15.40:0/3345358500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d2400 0x7f04e80d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.164+0000 7f04eed9e640 1 --2- 172.21.15.40:0/3345358500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d17d0 0x7f04e80b6180 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.164+0000 7f04eed9e640 1 -- 172.21.15.40:0/3345358500 >> 172.21.15.40:0/3345358500 conn(0x7f04e809e210 msgr2=0x7f04e80d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:25.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.164+0000 7f04eed9e640 1 -- 172.21.15.40:0/3345358500 shutdown_connections 2024-04-26T21:45:25.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.164+0000 7f04eed9e640 1 -- 172.21.15.40:0/3345358500 wait complete. 2024-04-26T21:45:25.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.165+0000 7f04eed9e640 1 Processor -- start 2024-04-26T21:45:25.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.165+0000 7f04eed9e640 1 -- start start 2024-04-26T21:45:25.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04eed9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d0c80 0x7f04e8157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04eed9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d17d0 0x7f04e80ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04eed9e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e80d2400 0x7f04e80ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04eed9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04e80d5a10 con 0x7f04e80d0c80 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04eed9e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04e80d5730 con 0x7f04e80d2400 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04eed9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04e80d3a00 con 0x7f04e80d17d0 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04edd9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d0c80 0x7f04e8157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04edd9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d0c80 0x7f04e8157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45566/0 (socket says 172.21.15.40:45566) 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04edd9c640 1 -- 172.21.15.40:0/2608931092 learned_addr learned my addr 172.21.15.40:0/2608931092 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04ed59b640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d17d0 0x7f04e80ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04ee59d640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e80d2400 0x7f04e80ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04edd9c640 1 -- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d17d0 msgr2=0x7f04e80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04edd9c640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d17d0 0x7f04e80ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04edd9c640 1 -- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e80d2400 msgr2=0x7f04e80ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.166+0000 7f04edd9c640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e80d2400 0x7f04e80ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04edd9c640 1 -- 172.21.15.40:0/2608931092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e80cbf70 con 0x7f04e80d0c80 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04ee59d640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e80d2400 0x7f04e80ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04ed59b640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d17d0 0x7f04e80ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04edd9c640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d0c80 0x7f04e8157030 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f04e4000c00 tx=0x7f04e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04de7fc640 1 -- 172.21.15.40:0/2608931092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4041020 con 0x7f04e80d0c80 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04de7fc640 1 -- 172.21.15.40:0/2608931092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e40306c0 con 0x7f04e80d0c80 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e80ceda0 con 0x7f04e80d0c80 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04de7fc640 1 -- 172.21.15.40:0/2608931092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4037040 con 0x7f04e80d0c80 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.167+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e80cc510 con 0x7f04e80d0c80 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.168+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e8069370 con 0x7f04e80d0c80 2024-04-26T21:45:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.168+0000 7f04de7fc640 1 -- 172.21.15.40:0/2608931092 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04e4044020 con 0x7f04e80d0c80 2024-04-26T21:45:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.169+0000 7f04de7fc640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04cc06c8f0 0x7f04cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.169+0000 7f04de7fc640 1 -- 172.21.15.40:0/2608931092 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8711+0+0 (secure 0 0 0) 0x7f04e40acf70 con 0x7f04e80d0c80 2024-04-26T21:45:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.169+0000 7f04ed59b640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04cc06c8f0 0x7f04cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.170+0000 7f04ed59b640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04cc06c8f0 0x7f04cc06edb0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f04e80cb460 tx=0x7f04d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.173+0000 7f04de7fc640 1 -- 172.21.15.40:0/2608931092 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04e404b030 con 0x7f04e80d0c80 2024-04-26T21:45:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.401+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f04e8157b60 con 0x7f04e80d0c80 2024-04-26T21:45:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.852+0000 7f04de7fc640 1 -- 172.21.15.40:0/2608931092 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v270) ==== 161+0+0 (secure 0 0 0) 0x7f04e80b66c0 con 0x7f04e80d0c80 2024-04-26T21:45:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:25.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04cc06c8f0 msgr2=0x7f04cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04cc06c8f0 0x7f04cc06edb0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f04e80cb460 tx=0x7f04d8008040 comp rx=0 tx=0).stop 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d0c80 msgr2=0x7f04e8157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d0c80 0x7f04e8157030 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f04e4000c00 tx=0x7f04e40046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 shutdown_connections 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04cc06c8f0 0x7f04cc06edb0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e80d2400 0x7f04e80ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e80d17d0 0x7f04e80ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 --2- 172.21.15.40:0/2608931092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e80d0c80 0x7f04e8157030 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 >> 172.21.15.40:0/2608931092 conn(0x7f04e809e210 msgr2=0x7f04e80d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 shutdown_connections 2024-04-26T21:45:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.854+0000 7f04eed9e640 1 -- 172.21.15.40:0/2608931092 wait complete. 2024-04-26T21:45:25.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:25.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.971+0000 7fc7a56cf640 1 Processor -- start 2024-04-26T21:45:25.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.971+0000 7fc7a56cf640 1 -- start start 2024-04-26T21:45:25.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.972+0000 7fc7a56cf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc7a015f3a0 0x7fc7a00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.972+0000 7fc7a56cf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7a00d1770 0x7fc7a00d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.972+0000 7fc7a56cf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a00d2590 con 0x7fc7a00d2160 2024-04-26T21:45:25.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.972+0000 7fc7a56cf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a00d0e20 con 0x7fc7a00d1770 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.972+0000 7fc7a56cf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a00b1d70 con 0x7fc7a015f3a0 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.972+0000 7fc79e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7a00d1770 0x7fc7a00d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.972+0000 7fc79e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7a00d1770 0x7fc7a00d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45570/0 (socket says 172.21.15.40:45570) 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.972+0000 7fc79e7fc640 1 -- 172.21.15.40:0/1798068707 learned_addr learned my addr 172.21.15.40:0/1798068707 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.973+0000 7fc79effd640 1 --2- 172.21.15.40:0/1798068707 >> v2:172.21.15.184:3300/0 conn(0x7fc7a015f3a0 0x7fc7a00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.973+0000 7fc79e7fc640 1 -- 172.21.15.40:0/1798068707 >> v2:172.21.15.184:3300/0 conn(0x7fc7a015f3a0 msgr2=0x7fc7a00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.973+0000 7fc79e7fc640 1 --2- 172.21.15.40:0/1798068707 >> v2:172.21.15.184:3300/0 conn(0x7fc7a015f3a0 0x7fc7a00d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.973+0000 7fc79e7fc640 1 -- 172.21.15.40:0/1798068707 >> v1:172.21.15.184:6789/0 conn(0x7fc7a00d2160 legacy=0x7fc7a00d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.973+0000 7fc79e7fc640 1 -- 172.21.15.40:0/1798068707 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a00b2050 con 0x7fc7a00d1770 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.973+0000 7fc79effd640 1 --2- 172.21.15.40:0/1798068707 >> v2:172.21.15.184:3300/0 conn(0x7fc7a015f3a0 0x7fc7a00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc79e7fc640 1 --2- 172.21.15.40:0/1798068707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a00d1c20 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0073b50 tx=0x7fc79402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78eb75af7174a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc79dffb640 1 -- 172.21.15.40:0/1798068707 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794037070 con 0x7fc7a00d1770 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc79dffb640 1 -- 172.21.15.40:0/1798068707 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc794032440 con 0x7fc7a00d1770 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc79dffb640 1 -- 172.21.15.40:0/1798068707 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794032810 con 0x7fc7a00d1770 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc7a56cf640 1 -- 172.21.15.40:0/1798068707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 msgr2=0x7fc7a00d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/1798068707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a00d1c20 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0073b50 tx=0x7fc79402a270 comp rx=0 tx=0).stop 2024-04-26T21:45:25.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc7a56cf640 1 -- 172.21.15.40:0/1798068707 shutdown_connections 2024-04-26T21:45:25.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/1798068707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a00d1c20 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/1798068707 >> v2:172.21.15.184:3300/0 conn(0x7fc7a015f3a0 0x7fc7a00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc7a56cf640 1 -- 172.21.15.40:0/1798068707 >> 172.21.15.40:0/1798068707 conn(0x7fc7a00d4f90 msgr2=0x7fc7a00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:25.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc7a56cf640 1 -- 172.21.15.40:0/1798068707 shutdown_connections 2024-04-26T21:45:25.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.974+0000 7fc7a56cf640 1 -- 172.21.15.40:0/1798068707 wait complete. 2024-04-26T21:45:25.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc7a56cf640 1 Processor -- start 2024-04-26T21:45:25.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc7a56cf640 1 -- start start 2024-04-26T21:45:25.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc7a56cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a01538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc7a56cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7a00d2160 0x7fc7a00b9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc7a56cf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7a015f3a0 0x7fc7a00d3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc7a56cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a015f900 con 0x7fc7a00d1770 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc7a56cf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a015f780 con 0x7fc7a015f3a0 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc7a56cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a00b3860 con 0x7fc7a00d2160 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.976+0000 7fc79effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a01538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a01538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45578/0 (socket says 172.21.15.40:45578) 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79effd640 1 -- 172.21.15.40:0/221335324 learned_addr learned my addr 172.21.15.40:0/221335324 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79e7fc640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7a00d2160 0x7fc7a00b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79f7fe640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7a015f3a0 0x7fc7a00d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79effd640 1 -- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7a00d2160 msgr2=0x7fc7a00b9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79effd640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7a00d2160 0x7fc7a00b9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79effd640 1 -- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7a015f3a0 msgr2=0x7fc7a00d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79effd640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7a015f3a0 0x7fc7a00d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79effd640 1 -- 172.21.15.40:0/221335324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a0159960 con 0x7fc7a00d1770 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79f7fe640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7a015f3a0 0x7fc7a00d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc79effd640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a01538c0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fc79800d800 tx=0x7fc79800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc7877fe640 1 -- 172.21.15.40:0/221335324 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798004030 con 0x7fc7a00d1770 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc7877fe640 1 -- 172.21.15.40:0/221335324 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7980041d0 con 0x7fc7a00d1770 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc7877fe640 1 -- 172.21.15.40:0/221335324 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7980096a0 con 0x7fc7a00d1770 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a01f88e0 con 0x7fc7a00d1770 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.977+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a01f8ea0 con 0x7fc7a00d1770 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.978+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a00694f0 con 0x7fc7a00d1770 2024-04-26T21:45:25.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.979+0000 7fc7877fe640 1 -- 172.21.15.40:0/221335324 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc798009840 con 0x7fc7a00d1770 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.979+0000 7fc7877fe640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc78c06c920 0x7fc78c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.979+0000 7fc7877fe640 1 -- 172.21.15.40:0/221335324 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8711+0+0 (secure 0 0 0) 0x7fc79808ca00 con 0x7fc7a00d1770 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.979+0000 7fc79e7fc640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc78c06c920 0x7fc78c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.980+0000 7fc79e7fc640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc78c06c920 0x7fc78c06ede0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc794034000 tx=0x7fc79402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:25.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:25.984+0000 7fc7877fe640 1 -- 172.21.15.40:0/221335324 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc79805a220 con 0x7fc7a00d1770 2024-04-26T21:45:26.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.247+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc7a00b6860 con 0x7fc7a00d1770 2024-04-26T21:45:26.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.860+0000 7fc7877fe640 1 -- 172.21.15.40:0/221335324 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v271) ==== 101+0+0 (secure 0 0 0) 0x7fc798059b20 con 0x7fc7a00d1770 2024-04-26T21:45:26.861 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc78c06c920 msgr2=0x7fc78c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc78c06c920 0x7fc78c06ede0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc794034000 tx=0x7fc79402e000 comp rx=0 tx=0).stop 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 msgr2=0x7fc7a01538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a01538c0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fc79800d800 tx=0x7fc79800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 shutdown_connections 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc78c06c920 0x7fc78c06ede0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7a015f3a0 0x7fc7a00d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7a00d2160 0x7fc7a00b9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 --2- 172.21.15.40:0/221335324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7a00d1770 0x7fc7a01538c0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 >> 172.21.15.40:0/221335324 conn(0x7fc7a00d4f90 msgr2=0x7fc7a00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 shutdown_connections 2024-04-26T21:45:26.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.863+0000 7fc7a56cf640 1 -- 172.21.15.40:0/221335324 wait complete. 2024-04-26T21:45:26.877 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:26.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.922+0000 7f0d5d71ff40 1 Processor -- start 2024-04-26T21:45:26.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.922+0000 7f0d5d71ff40 1 -- start start 2024-04-26T21:45:26.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5d71ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d03930 0x55d9b6d30850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5d71ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6d33af0 0x55d9b6ddb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5d71ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6ddc900 0x55d9b6d04710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5d71ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d9b6cace40 con 0x55d9b6ddc900 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5d71ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d9b6cb0e80 con 0x55d9b6d03930 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5d71ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d9b6d33010 con 0x55d9b6d33af0 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5af94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6ddc900 0x55d9b6d04710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5af94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6ddc900 0x55d9b6d04710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45590/0 (socket says 172.21.15.40:45590) 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5af94640 1 -- 172.21.15.40:0/535381220 learned_addr learned my addr 172.21.15.40:0/535381220 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d59f92640 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6d33af0 0x55d9b6ddb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5a793640 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d03930 0x55d9b6d30850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.923+0000 7f0d5af94640 1 -- 172.21.15.40:0/535381220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6d33af0 msgr2=0x55d9b6ddb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d5af94640 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6d33af0 0x55d9b6ddb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d5af94640 1 -- 172.21.15.40:0/535381220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d03930 msgr2=0x55d9b6d30850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d5af94640 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d03930 0x55d9b6d30850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d5af94640 1 -- 172.21.15.40:0/535381220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b6dd0470 con 0x55d9b6ddc900 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d5af94640 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6ddc900 0x55d9b6d04710 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f0d54006560 tx=0x7f0d54006a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4de07e5c63c75bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d58f90640 1 -- 172.21.15.40:0/535381220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54016070 con 0x55d9b6ddc900 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d58f90640 1 -- 172.21.15.40:0/535381220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d54004540 con 0x55d9b6ddc900 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d58f90640 1 -- 172.21.15.40:0/535381220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54004930 con 0x55d9b6ddc900 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/535381220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6ddc900 msgr2=0x55d9b6d04710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.924+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6ddc900 0x55d9b6d04710 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f0d54006560 tx=0x7f0d54006a20 comp rx=0 tx=0).stop 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/535381220 shutdown_connections 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6ddc900 0x55d9b6d04710 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6d33af0 0x55d9b6ddb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/535381220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d03930 0x55d9b6d30850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/535381220 >> 172.21.15.40:0/535381220 conn(0x55d9b6d04330 msgr2=0x55d9b6d31010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/535381220 shutdown_connections 2024-04-26T21:45:26.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/535381220 wait complete. 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 Processor -- start 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.925+0000 7f0d5d71ff40 1 -- start start 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5d71ff40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6d03930 0x55d9b6d4aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5d71ff40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d33af0 0x55d9b6e69520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5d71ff40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6ddc900 0x55d9b6d30850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5d71ff40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d9b6dd0710 con 0x55d9b6d03930 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5d71ff40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d9b6d30370 con 0x55d9b6d33af0 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5d71ff40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55d9b6d4eb50 con 0x55d9b6ddc900 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5a793640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6d03930 0x55d9b6d4aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5a793640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6d03930 0x55d9b6d4aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45600/0 (socket says 172.21.15.40:45600) 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 learned_addr learned my addr 172.21.15.40:0/933773293 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5af94640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6ddc900 0x55d9b6d30850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d59f92640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d33af0 0x55d9b6e69520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6ddc900 msgr2=0x55d9b6d30850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5a793640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d9b6ddc900 0x55d9b6d30850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d33af0 msgr2=0x55d9b6e69520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5a793640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d33af0 0x55d9b6e69520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b6d4b020 con 0x55d9b6d03930 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.926+0000 7f0d59f92640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b6d33af0 0x55d9b6e69520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.927+0000 7f0d5a793640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6d03930 0x55d9b6d4aae0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00efb0 tx=0x7f0d4c00c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.927+0000 7f0d4b7fe640 1 -- 172.21.15.40:0/933773293 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00cd30 con 0x55d9b6d03930 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.927+0000 7f0d4b7fe640 1 -- 172.21.15.40:0/933773293 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c001990 con 0x55d9b6d03930 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.927+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d9b6e666d0 con 0x55d9b6d03930 2024-04-26T21:45:26.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.927+0000 7f0d4b7fe640 1 -- 172.21.15.40:0/933773293 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c001d80 con 0x55d9b6d03930 2024-04-26T21:45:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.927+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b6e66c90 con 0x55d9b6d03930 2024-04-26T21:45:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.928+0000 7f0d4b7fe640 1 -- 172.21.15.40:0/933773293 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d4c002050 con 0x55d9b6d03930 2024-04-26T21:45:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.929+0000 7f0d4b7fe640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d3c06b2d0 0x7f0d3c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.929+0000 7f0d4b7fe640 1 -- 172.21.15.40:0/933773293 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8711+0+0 (secure 0 0 0) 0x7f0d4c08b2f0 con 0x55d9b6d03930 2024-04-26T21:45:26.932 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:26.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.929+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55d9b6dd0000 0x55d9b6e679a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.929+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d4c730 con 0x55d9b6dd0000 2024-04-26T21:45:26.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.929+0000 7f0d59f92640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d3c06b2d0 0x7f0d3c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.929+0000 7f0d5af94640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55d9b6dd0000 0x55d9b6e679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.930+0000 7f0d5af94640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55d9b6dd0000 0x55d9b6e679a0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.930+0000 7f0d59f92640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d3c06b2d0 0x7f0d3c06d790 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0d50056000 tx=0x7f0d50050000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.931+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v124'877 uv851 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.931+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e7fa30 con 0x55d9b6dd0000 2024-04-26T21:45:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.933+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v124'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.933+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e7fdd0 con 0x55d9b6dd0000 2024-04-26T21:45:26.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.934+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v124'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.935 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:26.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.934+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e81180 con 0x55d9b6dd0000 2024-04-26T21:45:26.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.935+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v124'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.935+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e81520 con 0x55d9b6dd0000 2024-04-26T21:45:26.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.936+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v124'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.936+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e80170 con 0x55d9b6dd0000 2024-04-26T21:45:26.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.937+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v124'882 uv882 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.937+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e80510 con 0x55d9b6dd0000 2024-04-26T21:45:26.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.938+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v124'883 uv857 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.938+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.939+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v124'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.939+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.940+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v124'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.941 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:26.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.940+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.941+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v124'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.941+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.942+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v124'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.942+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.944+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v124'888 uv888 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.944+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.945+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v124'889 uv864 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.945+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.946+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v124'890 uv890 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.946+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.947+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v124'891 uv891 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.948 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:26.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.947+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.948+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v124'892 uv892 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.948+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.949+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v124'893 uv893 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.949+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e808b0 con 0x55d9b6dd0000 2024-04-26T21:45:26.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.950+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v124'894 uv894 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.950+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.951+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v124'895 uv895 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.951+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.953+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v124'896 uv870 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.953+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.954+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v124'897 uv897 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.954+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.955+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v124'898 uv898 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.956 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:26.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.955+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.956+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v124'899 uv899 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.956+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.958+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v124'900 uv900 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.958+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.959+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v124'901 uv901 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.959+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.960+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v124'902 uv876 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.960+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.961+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v124'903 uv903 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.961+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.962+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v124'904 uv904 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.963 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:26.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.962+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6d30d90 con 0x55d9b6dd0000 2024-04-26T21:45:26.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.963+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v124'905 uv905 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.963+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e80b60 con 0x55d9b6dd0000 2024-04-26T21:45:26.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.964+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v124'906 uv906 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.964+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e80b60 con 0x55d9b6dd0000 2024-04-26T21:45:26.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.965+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v124'907 uv907 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6dd0000 2024-04-26T21:45:26.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.965+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55d9b6e81dd0 0x55d9b6e841c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.965+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e80b60 con 0x55d9b6e81dd0 2024-04-26T21:45:26.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.966+0000 7f0d5a793640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55d9b6e81dd0 0x55d9b6e841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.966+0000 7f0d5a793640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55d9b6e81dd0 0x55d9b6e841c0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.968+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v124'638 uv621 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.968+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e84700 con 0x55d9b6e81dd0 2024-04-26T21:45:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.970+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v124'639 uv639 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.970+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e84aa0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.971+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v124'640 uv640 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.972 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:26.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.971+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e852f0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.972+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v124'641 uv641 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.972+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85690 con 0x55d9b6e81dd0 2024-04-26T21:45:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.974+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v124'642 uv642 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.974+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85a30 con 0x55d9b6e81dd0 2024-04-26T21:45:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.975+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v124'643 uv626 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.975+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.976+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v124'644 uv644 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.977+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.980 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:26.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.979+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v124'645 uv645 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.980+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.981+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v124'646 uv646 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.981+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.982+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v124'647 uv647 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.982+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.984+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v124'648 uv631 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.984+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.985+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v124'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.985+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.987 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:26.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.986+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v124'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.986+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e8a490 con 0x55d9b6e81dd0 2024-04-26T21:45:26.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.988+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v124'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.988+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e8a490 con 0x55d9b6e81dd0 2024-04-26T21:45:26.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.989+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v124'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.989+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.990+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v124'653 uv637 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.990+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.992+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v124'654 uv654 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.992+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.994 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:26.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.993+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v124'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.993+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e86de0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.994+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v124'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.995+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e86de0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.996+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v124'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.996+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e85dd0 con 0x55d9b6e81dd0 2024-04-26T21:45:26.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.997+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v124'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e81dd0 2024-04-26T21:45:26.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.997+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d9b6e86450 0x55d9b6e88840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.997+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e88dc0 con 0x55d9b6e86450 2024-04-26T21:45:26.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.997+0000 7f0d59f92640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d9b6e86450 0x55d9b6e88840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.998+0000 7f0d59f92640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d9b6e86450 0x55d9b6e88840 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.999+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v124'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:26.999+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e89280 con 0x55d9b6e86450 2024-04-26T21:45:27.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.000+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v124'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.000+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e89760 con 0x55d9b6e86450 2024-04-26T21:45:27.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.001+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v124'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.002 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:27.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.001+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e84f20 con 0x55d9b6e86450 2024-04-26T21:45:27.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.002+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v124'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.002+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e8b0d0 con 0x55d9b6e86450 2024-04-26T21:45:27.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.003+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v124'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.003+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e8a000 con 0x55d9b6e86450 2024-04-26T21:45:27.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.004+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v124'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.004+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55d9b6e8aa90 0x55d9b6e8d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.004+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e8da50 con 0x55d9b6e8aa90 2024-04-26T21:45:27.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.004+0000 7f0d5af94640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55d9b6e8aa90 0x55d9b6e8d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.005+0000 7f0d5af94640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55d9b6e8aa90 0x55d9b6e8d4d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.006+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v124'131 uv130 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.006+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e8de70 con 0x55d9b6e8aa90 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.007+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v124'132 uv132 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.007+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e8de70 con 0x55d9b6e8aa90 2024-04-26T21:45:27.009 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:27.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.009+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v124'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.009+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e8de90 con 0x55d9b6e8aa90 2024-04-26T21:45:27.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.010+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v124'134 uv134 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.010+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55d9b6e8e800 0x55d9b6e90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.010+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91130 con 0x55d9b6e8e800 2024-04-26T21:45:27.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.010+0000 7f0d5a793640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55d9b6e8e800 0x55d9b6e90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.011+0000 7f0d5a793640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55d9b6e8e800 0x55d9b6e90bf0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.012+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v124'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.012+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e91130 con 0x55d9b6e8e800 2024-04-26T21:45:27.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.013+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v124'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.013+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.015+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v124'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.015 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:27.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.015+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e92560 con 0x55d9b6e8e800 2024-04-26T21:45:27.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.016+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v124'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.016+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e92560 con 0x55d9b6e8e800 2024-04-26T21:45:27.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.017+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v124'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.017+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.018+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v124'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.018+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.020+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v124'316 uv310 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.020+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.021+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v124'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.021+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.023 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.022+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v124'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.022+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e92560 con 0x55d9b6e8aa90 2024-04-26T21:45:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.023+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v124'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.023+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e92560 con 0x55d9b6e8aa90 2024-04-26T21:45:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.025+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v124'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.025+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.026+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v124'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.026+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.027+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v124'322 uv315 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.027+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.028+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v124'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.028+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.030 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:27.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.030+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v124'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.030+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.031+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v124'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.031+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.032+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v124'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.032+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.033+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v124'143 uv142 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.033+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.034+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v124'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.034+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.035+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v124'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.036 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:27.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.035+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.036+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v124'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.036+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8aa90 2024-04-26T21:45:27.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.037+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v124'147 uv147 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d5400f070 con 0x55d9b6e8aa90 2024-04-26T21:45:27.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.037+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.038+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v124'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.038+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.039+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v124'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.039+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.040+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v124'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.041 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:27.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.040+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e92560 con 0x55d9b6e8e800 2024-04-26T21:45:27.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.041+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v124'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.041+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e92560 con 0x55d9b6e8e800 2024-04-26T21:45:27.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.042+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v124'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.042+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.043+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v124'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.043+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.044+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v124'284 uv283 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.044+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.045+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v124'285 uv285 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.046+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91550 con 0x55d9b6e8e800 2024-04-26T21:45:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.047+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v124'286 uv286 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.047 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:27.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.047+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91910 con 0x55d9b6e8e800 2024-04-26T21:45:27.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.048+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v124'287 uv287 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.048+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91910 con 0x55d9b6e8e800 2024-04-26T21:45:27.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.049+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v124'288 uv288 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.049+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91910 con 0x55d9b6e8e800 2024-04-26T21:45:27.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.050+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v124'289 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.050+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e91910 con 0x55d9b6e8e800 2024-04-26T21:45:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.051+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v124'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.051+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91910 con 0x55d9b6e8e800 2024-04-26T21:45:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.052+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v124'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.053 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.052+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e91910 con 0x55d9b6e8e800 2024-04-26T21:45:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.053+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v124'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.053+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94720 con 0x55d9b6e8e800 2024-04-26T21:45:27.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.055+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v124'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.055+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55d9b6e922d0 0x55d9b6e946c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.055+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94c00 con 0x55d9b6e922d0 2024-04-26T21:45:27.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.055+0000 7f0d59f92640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55d9b6e922d0 0x55d9b6e946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.055+0000 7f0d59f92640 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55d9b6e922d0 0x55d9b6e946c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.057+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v124'124 uv123 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e922d0 2024-04-26T21:45:27.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.057+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e922d0 2024-04-26T21:45:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.058+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v124'125 uv125 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e922d0 2024-04-26T21:45:27.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.058+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e922d0 2024-04-26T21:45:27.059 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:27.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.059+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v124'126 uv126 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e922d0 2024-04-26T21:45:27.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.059+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e922d0 2024-04-26T21:45:27.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.060+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v124'127 uv127 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e922d0 2024-04-26T21:45:27.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.060+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e8e800 2024-04-26T21:45:27.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.061+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v124'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.062+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e8e800 2024-04-26T21:45:27.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.063+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v124'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.063+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e8e800 2024-04-26T21:45:27.064 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:27.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.064+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v124'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.064+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e95fb0 con 0x55d9b6e8e800 2024-04-26T21:45:27.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.065+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v124'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.065+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e95fb0 con 0x55d9b6e8e800 2024-04-26T21:45:27.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.066+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v124'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.066+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e8e800 2024-04-26T21:45:27.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.067+0000 7f0d5a793640 1 -- 172.21.15.40:0/933773293 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v124'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d4c054b10 con 0x55d9b6e8e800 2024-04-26T21:45:27.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.067+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.068+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v124'145 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.068+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.069+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v124'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.069+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.071 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:27.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.070+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v124'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.070+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.072+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v124'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.072+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.073+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v124'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.073+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.074+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v124'351 uv343 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.074+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.077+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v124'352 uv352 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.077+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.078+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v124'353 uv353 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.079 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:27.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.078+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fc0 con 0x55d9b6e86450 2024-04-26T21:45:27.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.079+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v124'354 uv354 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.079+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fc0 con 0x55d9b6e86450 2024-04-26T21:45:27.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.080+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v124'355 uv355 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.080+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fc0 con 0x55d9b6e86450 2024-04-26T21:45:27.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.081+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v124'356 uv350 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.081+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55d9b6e953b0 con 0x55d9b6e86450 2024-04-26T21:45:27.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.082+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v124'357 uv357 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.082+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.083 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:27.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.083+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v124'358 uv358 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.083+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e95fb0 con 0x55d9b6e86450 2024-04-26T21:45:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.084+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v124'359 uv359 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.084+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e95fb0 con 0x55d9b6e86450 2024-04-26T21:45:27.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.085+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v124'360 uv360 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.085+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e94fa0 con 0x55d9b6e86450 2024-04-26T21:45:27.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.086+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v124'361 uv361 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.086+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d9b6e95390 con 0x55d9b6e86450 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d59f92640 1 -- 172.21.15.40:0/933773293 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v124'362 uv362 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d5006a040 con 0x55d9b6e86450 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55d9b6e81dd0 msgr2=0x55d9b6e841c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55d9b6e81dd0 0x55d9b6e841c0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55d9b6dd0000 msgr2=0x55d9b6e679a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55d9b6dd0000 0x55d9b6e679a0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55d9b6e8aa90 msgr2=0x55d9b6e8d4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55d9b6e8aa90 0x55d9b6e8d4d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55d9b6e922d0 msgr2=0x55d9b6e946c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55d9b6e922d0 0x55d9b6e946c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d9b6e86450 msgr2=0x55d9b6e88840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55d9b6e86450 0x55d9b6e88840 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55d9b6e8e800 msgr2=0x55d9b6e90bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55d9b6e8e800 0x55d9b6e90bf0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 reap_dead start 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5af94640 1 -- 172.21.15.40:0/933773293 reap_dead start 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d3c06b2d0 msgr2=0x7f0d3c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d3c06b2d0 0x7f0d3c06d790 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0d50056000 tx=0x7f0d50050000 comp rx=0 tx=0).stop 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6d03930 msgr2=0x55d9b6d4aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6d03930 0x55d9b6d4aae0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00efb0 tx=0x7f0d4c00c620 comp rx=0 tx=0).stop 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 shutdown_connections 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d3c06b2d0 0x7f0d3c06d790 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 --2- 172.21.15.40:0/933773293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d9b6d03930 0x55d9b6d4aae0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 >> 172.21.15.40:0/933773293 conn(0x55d9b6d04330 msgr2=0x55d9b6ddbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 shutdown_connections 2024-04-26T21:45:27.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.087+0000 7f0d5d71ff40 1 -- 172.21.15.40:0/933773293 wait complete. 2024-04-26T21:45:27.091 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 400.00000000 0 2024-04-26T21:45:27.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.173+0000 7f5333574f40 1 Processor -- start 2024-04-26T21:45:27.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f5333574f40 1 -- start start 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f5333574f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a1cd30 0x55e7e5aee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f5333574f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5a48de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f5333574f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4d2e0 0x55e7e5af0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f5333574f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5a1bf70 con 0x55e7e5a4c6d0 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f5333574f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5a1bcd0 con 0x55e7e5a4d2e0 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f5333574f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e7e5a1c250 con 0x55e7e5a1cd30 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f533086d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5a48de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f533086d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5a48de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45614/0 (socket says 172.21.15.40:45614) 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.174+0000 7f533086d640 1 -- 172.21.15.40:0/1069396178 learned_addr learned my addr 172.21.15.40:0/1069396178 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533186f640 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4d2e0 0x55e7e5af0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533106e640 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a1cd30 0x55e7e5aee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533086d640 1 -- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a1cd30 msgr2=0x55e7e5aee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533086d640 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a1cd30 0x55e7e5aee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533086d640 1 -- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4d2e0 msgr2=0x55e7e5af0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533086d640 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4d2e0 0x55e7e5af0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533086d640 1 -- 172.21.15.40:0/1069396178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e5946cb0 con 0x55e7e5a4c6d0 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533106e640 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a1cd30 0x55e7e5aee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533186f640 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4d2e0 0x55e7e5af0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f533086d640 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5a48de0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f531c009760 tx=0x7f531c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5095e295fe7df82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f532b7fe640 1 -- 172.21.15.40:0/1069396178 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c017070 con 0x55e7e5a4c6d0 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f532b7fe640 1 -- 172.21.15.40:0/1069396178 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c0090d0 con 0x55e7e5a4c6d0 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f532b7fe640 1 -- 172.21.15.40:0/1069396178 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c006b60 con 0x55e7e5a4c6d0 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f5333574f40 1 -- 172.21.15.40:0/1069396178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a4c6d0 msgr2=0x55e7e5a48de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f5333574f40 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5a48de0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f531c009760 tx=0x7f531c009c20 comp rx=0 tx=0).stop 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f5333574f40 1 -- 172.21.15.40:0/1069396178 shutdown_connections 2024-04-26T21:45:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f5333574f40 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4d2e0 0x55e7e5af0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f5333574f40 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5a48de0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f5333574f40 1 --2- 172.21.15.40:0/1069396178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a1cd30 0x55e7e5aee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f5333574f40 1 -- 172.21.15.40:0/1069396178 >> 172.21.15.40:0/1069396178 conn(0x55e7e5a479b0 msgr2=0x55e7e5a1f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.175+0000 7f5333574f40 1 -- 172.21.15.40:0/1069396178 shutdown_connections 2024-04-26T21:45:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.176+0000 7f5333574f40 1 -- 172.21.15.40:0/1069396178 wait complete. 2024-04-26T21:45:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.176+0000 7f5333574f40 1 Processor -- start 2024-04-26T21:45:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.176+0000 7f5333574f40 1 -- start start 2024-04-26T21:45:27.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f5333574f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a1cd30 0x55e7e5af2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f5333574f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5ae6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f5333574f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a4d2e0 0x55e7e5aef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f5333574f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5aeb6b0 con 0x55e7e5a1cd30 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f5333574f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5a64300 con 0x55e7e5a4c6d0 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f5333574f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e7e5aeb950 con 0x55e7e5a4d2e0 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a1cd30 0x55e7e5af2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a1cd30 0x55e7e5af2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45616/0 (socket says 172.21.15.40:45616) 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 -- 172.21.15.40:0/639407396 learned_addr learned my addr 172.21.15.40:0/639407396 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533086d640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5ae6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533186f640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a4d2e0 0x55e7e5aef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 -- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a4d2e0 msgr2=0x55e7e5aef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e7e5a4d2e0 0x55e7e5aef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 -- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4c6d0 msgr2=0x55e7e5ae6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7e5a4c6d0 0x55e7e5ae6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 -- 172.21.15.40:0/639407396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e5a48de0 con 0x55e7e5a1cd30 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f533106e640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a1cd30 0x55e7e5af2ed0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f532c005930 tx=0x7f532c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.177+0000 7f532a7fc640 1 -- 172.21.15.40:0/639407396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c053b10 con 0x55e7e5a1cd30 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.178+0000 7f532a7fc640 1 -- 172.21.15.40:0/639407396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532c053cb0 con 0x55e7e5a1cd30 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.178+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e7e5a490c0 con 0x55e7e5a1cd30 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.178+0000 7f532a7fc640 1 -- 172.21.15.40:0/639407396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c053e60 con 0x55e7e5a1cd30 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.178+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e5ae8f90 con 0x55e7e5a1cd30 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.179+0000 7f532a7fc640 1 -- 172.21.15.40:0/639407396 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f532c05e410 con 0x55e7e5a1cd30 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.179+0000 7f532a7fc640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f531406b300 0x7f531406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.179+0000 7f532a7fc640 1 -- 172.21.15.40:0/639407396 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8711+0+0 (secure 0 0 0) 0x7f532c0cf710 con 0x55e7e5a1cd30 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.179+0000 7f533086d640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f531406b300 0x7f531406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.180+0000 7f5333574f40 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e7e5a4be10 0x55e7e5ae9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.180+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55e7e5ae9fb0 con 0x55e7e5a4be10 2024-04-26T21:45:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.180+0000 7f533186f640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e7e5a4be10 0x55e7e5ae9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.180+0000 7f533086d640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f531406b300 0x7f531406d7c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x55e7e5a201d0 tx=0x7f531c00f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.180+0000 7f533186f640 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e7e5a4be10 0x55e7e5ae9a70 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f533186f640 1 -- 172.21.15.40:0/639407396 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 400.00000000 [truncate 0] v124'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f532400e710 con 0x55e7e5a4be10 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e7e5a4be10 msgr2=0x55e7e5ae9a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55e7e5a4be10 0x55e7e5ae9a70 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f531406b300 msgr2=0x7f531406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f531406b300 0x7f531406d7c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x55e7e5a201d0 tx=0x7f531c00f000 comp rx=0 tx=0).stop 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a1cd30 msgr2=0x55e7e5af2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 --2- 172.21.15.40:0/639407396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e7e5a1cd30 0x55e7e5af2ed0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f532c005930 tx=0x7f532c002fb0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f533186f640 1 -- 172.21.15.40:0/639407396 reap_dead start 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 shutdown_connections 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 >> 172.21.15.40:0/639407396 conn(0x55e7e5a479b0 msgr2=0x55e7e5af4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 shutdown_connections 2024-04-26T21:45:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.181+0000 7f5333574f40 1 -- 172.21.15.40:0/639407396 wait complete. 2024-04-26T21:45:27.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.329+0000 7faf43683640 1 Processor -- start 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.329+0000 7faf43683640 1 -- start start 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf43683640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faf3c0d1a00 0x7faf3c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf43683640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d2320 0x7faf3c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf43683640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf3c0b40d0 0x7faf3c162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf43683640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7faf3c163070 con 0x7faf3c0b40d0 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf43683640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faf3c0d0a10 con 0x7faf3c0d1a00 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf43683640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faf3c0d0cf0 con 0x7faf3c0d2320 2024-04-26T21:45:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf41c20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf3c0b40d0 0x7faf3c162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf41c20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf3c0b40d0 0x7faf3c162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45630/0 (socket says 172.21.15.40:45630) 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.330+0000 7faf41c20640 1 -- 172.21.15.40:0/2206765943 learned_addr learned my addr 172.21.15.40:0/2206765943 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf40c1e640 1 --2- 172.21.15.40:0/2206765943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d2320 0x7faf3c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf4141f640 1 --2- 172.21.15.40:0/2206765943 >> v2:172.21.15.184:3300/0 conn(0x7faf3c0d1a00 0x7faf3c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf41c20640 1 -- 172.21.15.40:0/2206765943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d2320 msgr2=0x7faf3c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf41c20640 1 --2- 172.21.15.40:0/2206765943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d2320 0x7faf3c1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf41c20640 1 -- 172.21.15.40:0/2206765943 >> v2:172.21.15.184:3300/0 conn(0x7faf3c0d1a00 msgr2=0x7faf3c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf41c20640 1 --2- 172.21.15.40:0/2206765943 >> v2:172.21.15.184:3300/0 conn(0x7faf3c0d1a00 0x7faf3c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf41c20640 1 -- 172.21.15.40:0/2206765943 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c0d0fd0 con 0x7faf3c0b40d0 2024-04-26T21:45:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf4141f640 1 --2- 172.21.15.40:0/2206765943 >> v2:172.21.15.184:3300/0 conn(0x7faf3c0d1a00 0x7faf3c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.331+0000 7faf41c20640 1 --2- 172.21.15.40:0/2206765943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0b40d0 0x7faf3c162b30 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7faf3c06c650 tx=0x7faf3402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6de7fc8b7b89a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf3bfff640 1 -- 172.21.15.40:0/2206765943 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34037070 con 0x7faf3c0b40d0 2024-04-26T21:45:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf3bfff640 1 -- 172.21.15.40:0/2206765943 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf34032440 con 0x7faf3c0b40d0 2024-04-26T21:45:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf3bfff640 1 -- 172.21.15.40:0/2206765943 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34032810 con 0x7faf3c0b40d0 2024-04-26T21:45:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 -- 172.21.15.40:0/2206765943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0b40d0 msgr2=0x7faf3c162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 --2- 172.21.15.40:0/2206765943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0b40d0 0x7faf3c162b30 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7faf3c06c650 tx=0x7faf3402a140 comp rx=0 tx=0).stop 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 -- 172.21.15.40:0/2206765943 shutdown_connections 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 --2- 172.21.15.40:0/2206765943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0b40d0 0x7faf3c162b30 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 --2- 172.21.15.40:0/2206765943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d2320 0x7faf3c1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 --2- 172.21.15.40:0/2206765943 >> v2:172.21.15.184:3300/0 conn(0x7faf3c0d1a00 0x7faf3c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 -- 172.21.15.40:0/2206765943 >> 172.21.15.40:0/2206765943 conn(0x7faf3c0d4fd0 msgr2=0x7faf3c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 -- 172.21.15.40:0/2206765943 shutdown_connections 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.332+0000 7faf43683640 1 -- 172.21.15.40:0/2206765943 wait complete. 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.333+0000 7faf43683640 1 Processor -- start 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.333+0000 7faf43683640 1 -- start start 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf43683640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf3c0b40d0 0x7faf3c154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf43683640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d1a00 0x7faf3c0cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf43683640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0d2320 0x7faf3c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf43683640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c0d3050 con 0x7faf3c0d2320 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf43683640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c0d2ed0 con 0x7faf3c0b40d0 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf43683640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faf3c0d31d0 con 0x7faf3c0d1a00 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf41c20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0d2320 0x7faf3c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf41c20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0d2320 0x7faf3c0ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45636/0 (socket says 172.21.15.40:45636) 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf41c20640 1 -- 172.21.15.40:0/3395957355 learned_addr learned my addr 172.21.15.40:0/3395957355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf40c1e640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d1a00 0x7faf3c0cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.334+0000 7faf4141f640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf3c0b40d0 0x7faf3c154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf41c20640 1 -- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d1a00 msgr2=0x7faf3c0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf41c20640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d1a00 0x7faf3c0cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf41c20640 1 -- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf3c0b40d0 msgr2=0x7faf3c154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf41c20640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf3c0b40d0 0x7faf3c154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf41c20640 1 -- 172.21.15.40:0/3395957355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c0ce710 con 0x7faf3c0d2320 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf41c20640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0d2320 0x7faf3c0ca6e0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7faf340090b0 tx=0x7faf340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf39ffb640 1 -- 172.21.15.40:0/3395957355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34037040 con 0x7faf3c0d2320 2024-04-26T21:45:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf39ffb640 1 -- 172.21.15.40:0/3395957355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf34042070 con 0x7faf3c0d2320 2024-04-26T21:45:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf39ffb640 1 -- 172.21.15.40:0/3395957355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34030920 con 0x7faf3c0d2320 2024-04-26T21:45:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3c0cdb90 con 0x7faf3c0d2320 2024-04-26T21:45:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.335+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3c0ce430 con 0x7faf3c0d2320 2024-04-26T21:45:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.336+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3c0694f0 con 0x7faf3c0d2320 2024-04-26T21:45:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.337+0000 7faf39ffb640 1 -- 172.21.15.40:0/3395957355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf34030ac0 con 0x7faf3c0d2320 2024-04-26T21:45:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.337+0000 7faf39ffb640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.337+0000 7faf39ffb640 1 -- 172.21.15.40:0/3395957355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8711+0+0 (secure 0 0 0) 0x7faf340ad020 con 0x7faf3c0d2320 2024-04-26T21:45:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.337+0000 7faf4141f640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.338+0000 7faf4141f640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2806c8f0 0x7faf2806edb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faf2c003fd0 tx=0x7faf2c003990 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.341+0000 7faf39ffb640 1 -- 172.21.15.40:0/3395957355 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf3407a280 con 0x7faf3c0d2320 2024-04-26T21:45:27.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.574+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7faf3c0cbe60 con 0x7faf3c0d2320 2024-04-26T21:45:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.861+0000 7faf39ffb640 1 -- 172.21.15.40:0/3395957355 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v272) ==== 162+0+0 (secure 0 0 0) 0x7faf3407e580 con 0x7faf3c0d2320 2024-04-26T21:45:27.863 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2806c8f0 msgr2=0x7faf2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2806c8f0 0x7faf2806edb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faf2c003fd0 tx=0x7faf2c003990 comp rx=0 tx=0).stop 2024-04-26T21:45:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0d2320 msgr2=0x7faf3c0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0d2320 0x7faf3c0ca6e0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7faf340090b0 tx=0x7faf340046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 shutdown_connections 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf3c0d2320 0x7faf3c0ca6e0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf3c0d1a00 0x7faf3c0cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 --2- 172.21.15.40:0/3395957355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf3c0b40d0 0x7faf3c154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 >> 172.21.15.40:0/3395957355 conn(0x7faf3c0d4fd0 msgr2=0x7faf3c0b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 shutdown_connections 2024-04-26T21:45:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.864+0000 7faf43683640 1 -- 172.21.15.40:0/3395957355 wait complete. 2024-04-26T21:45:27.870 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:27.870+0000 7f536d3ae640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-04-26T21:45:27.871 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:27.870+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:45:27.871 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:27.870+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:45:27.876 INFO:tasks.ceph.mds.c.smithi040.stderr: -30> 2024-04-26T21:45:27.870+0000 7f536d3ae640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-04-26T21:45:27.879 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:27.902 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:45:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.981+0000 7fab63607640 1 Processor -- start 2024-04-26T21:45:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.981+0000 7fab63607640 1 -- start start 2024-04-26T21:45:27.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab63607640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab5c15d1e0 0x7fab5c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab63607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 0x7fab5c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab63607640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5c0ce840 con 0x7fab5c0d18c0 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab63607640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fab5c0cede0 con 0x7fab5c15d1e0 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab63607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c070650 con 0x7fab5c0d0f20 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab613a3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab5c15d1e0 0x7fab5c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab61ba4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fab5c0d18c0 0x7fab5c0cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44684/0 (socket says 172.21.15.40:44684) 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab61ba4640 1 -- 172.21.15.40:0/243491737 learned_addr learned my addr 172.21.15.40:0/243491737 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab613a3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab5c15d1e0 0x7fab5c0d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45656/0 (socket says 172.21.15.40:45656) 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab60ba2640 1 --2- 172.21.15.40:0/243491737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 0x7fab5c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab613a3640 1 -- 172.21.15.40:0/243491737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 msgr2=0x7fab5c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab613a3640 1 --2- 172.21.15.40:0/243491737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 0x7fab5c0d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab613a3640 1 -- 172.21.15.40:0/243491737 >> v1:172.21.15.40:6789/0 conn(0x7fab5c0d18c0 legacy=0x7fab5c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab613a3640 1 -- 172.21.15.40:0/243491737 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c0ceb00 con 0x7fab5c15d1e0 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.982+0000 7fab53fff640 1 -- 172.21.15.40:0/243491737 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3212002341 0 0) 0x7fab5c0ce840 con 0x7fab5c0d18c0 2024-04-26T21:45:27.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.983+0000 7fab613a3640 1 --2- 172.21.15.40:0/243491737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c0d09e0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fab580097a0 tx=0x7fab5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd54103fbea041e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.983+0000 7fab53fff640 1 -- 172.21.15.40:0/243491737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58037070 con 0x7fab5c15d1e0 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.983+0000 7fab53fff640 1 -- 172.21.15.40:0/243491737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab58032440 con 0x7fab5c15d1e0 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.983+0000 7fab53fff640 1 -- 172.21.15.40:0/243491737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58032810 con 0x7fab5c15d1e0 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.983+0000 7fab63607640 1 -- 172.21.15.40:0/243491737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 msgr2=0x7fab5c0d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.983+0000 7fab63607640 1 --2- 172.21.15.40:0/243491737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c0d09e0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fab580097a0 tx=0x7fab5802a180 comp rx=0 tx=0).stop 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.983+0000 7fab63607640 1 -- 172.21.15.40:0/243491737 shutdown_connections 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.984+0000 7fab63607640 1 --2- 172.21.15.40:0/243491737 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 0x7fab5c0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.984+0000 7fab63607640 1 --2- 172.21.15.40:0/243491737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c0d09e0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.984+0000 7fab63607640 1 -- 172.21.15.40:0/243491737 >> 172.21.15.40:0/243491737 conn(0x7fab5c0d2da0 msgr2=0x7fab5c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.984+0000 7fab63607640 1 -- 172.21.15.40:0/243491737 shutdown_connections 2024-04-26T21:45:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.984+0000 7fab63607640 1 -- 172.21.15.40:0/243491737 wait complete. 2024-04-26T21:45:27.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.985+0000 7fab63607640 1 Processor -- start 2024-04-26T21:45:27.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.985+0000 7fab63607640 1 -- start start 2024-04-26T21:45:27.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.985+0000 7fab63607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 0x7fab5c1e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.985+0000 7fab63607640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0d18c0 0x7fab5c0b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.985+0000 7fab63607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c14dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.985+0000 7fab63607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c071af0 con 0x7fab5c15d1e0 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.985+0000 7fab63607640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c0ce4c0 con 0x7fab5c0d18c0 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.985+0000 7fab63607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c071c70 con 0x7fab5c0d0f20 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c14dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c14dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45668/0 (socket says 172.21.15.40:45668) 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 -- 172.21.15.40:0/1261142839 learned_addr learned my addr 172.21.15.40:0/1261142839 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab613a3640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 0x7fab5c1e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab60ba2640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0d18c0 0x7fab5c0b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 -- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 msgr2=0x7fab5c1e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 0x7fab5c1e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 -- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0d18c0 msgr2=0x7fab5c0b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0d18c0 0x7fab5c0b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 -- 172.21.15.40:0/1261142839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c1e7510 con 0x7fab5c15d1e0 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab60ba2640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0d18c0 0x7fab5c0b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab61ba4640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c14dde0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c7d0 tx=0x7fab5400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab51ffb640 1 -- 172.21.15.40:0/1261142839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54018070 con 0x7fab5c15d1e0 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab51ffb640 1 -- 172.21.15.40:0/1261142839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5400a640 con 0x7fab5c15d1e0 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c1e77f0 con 0x7fab5c15d1e0 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab51ffb640 1 -- 172.21.15.40:0/1261142839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5400aa70 con 0x7fab5c15d1e0 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.986+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c1e8090 con 0x7fab5c15d1e0 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.987+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c1e8290 con 0x7fab5c15d1e0 2024-04-26T21:45:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.988+0000 7fab51ffb640 1 -- 172.21.15.40:0/1261142839 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab5401b070 con 0x7fab5c15d1e0 2024-04-26T21:45:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.988+0000 7fab51ffb640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab4006c920 0x7fab4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.988+0000 7fab51ffb640 1 -- 172.21.15.40:0/1261142839 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8797+0+0 (secure 0 0 0) 0x7fab5408bd90 con 0x7fab5c15d1e0 2024-04-26T21:45:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.988+0000 7fab613a3640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab4006c920 0x7fab4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.989+0000 7fab613a3640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab4006c920 0x7fab4006ede0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fab58034000 tx=0x7fab5802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:27.992+0000 7fab51ffb640 1 -- 172.21.15.40:0/1261142839 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab5c1e8290 con 0x7fab5c15d1e0 2024-04-26T21:45:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.215+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab5c0694f0 con 0x7fab5c15d1e0 2024-04-26T21:45:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.215+0000 7fab51ffb640 1 -- 172.21.15.40:0/1261142839 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 274 v274) ==== 95+0+4355 (secure 0 0 0) 0x7fab54058e80 con 0x7fab5c15d1e0 2024-04-26T21:45:28.216 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:28.216 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":274,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":274,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:27.871871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 274 2024-04-26T21:45:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab4006c920 msgr2=0x7fab4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab4006c920 0x7fab4006ede0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fab58034000 tx=0x7fab5802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 msgr2=0x7fab5c14dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c14dde0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c7d0 tx=0x7fab5400cc90 comp rx=0 tx=0).stop 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 shutdown_connections 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab4006c920 0x7fab4006ede0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab5c15d1e0 0x7fab5c14dde0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab5c0d18c0 0x7fab5c0b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 --2- 172.21.15.40:0/1261142839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab5c0d0f20 0x7fab5c1e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 >> 172.21.15.40:0/1261142839 conn(0x7fab5c0d2da0 msgr2=0x7fab5c0b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 shutdown_connections 2024-04-26T21:45:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.217+0000 7fab63607640 1 -- 172.21.15.40:0/1261142839 wait complete. 2024-04-26T21:45:28.229 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:28.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.369+0000 7f2e6fb7f640 1 Processor -- start 2024-04-26T21:45:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.369+0000 7f2e6fb7f640 1 -- start start 2024-04-26T21:45:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.370+0000 7f2e6fb7f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e6815f3a0 0x7f2e680d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.370+0000 7f2e6fb7f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e680d2330 0x7f2e680b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.370+0000 7f2e6fb7f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e680d0e20 con 0x7f2e680d1770 2024-04-26T21:45:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.370+0000 7f2e6fb7f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2e680b1d70 con 0x7f2e6815f3a0 2024-04-26T21:45:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.370+0000 7f2e6fb7f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2e680b2050 con 0x7f2e680d2330 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.370+0000 7f2e6d91b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e6815f3a0 0x7f2e680d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.370+0000 7f2e6d91b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e6815f3a0 0x7f2e680d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45682/0 (socket says 172.21.15.40:45682) 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.370+0000 7f2e6d91b640 1 -- 172.21.15.40:0/1571581523 learned_addr learned my addr 172.21.15.40:0/1571581523 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.371+0000 7f2e6d91b640 1 -- 172.21.15.40:0/1571581523 >> v2:172.21.15.184:3300/0 conn(0x7f2e680d2330 msgr2=0x7f2e680b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.371+0000 7f2e6d91b640 1 --2- 172.21.15.40:0/1571581523 >> v2:172.21.15.184:3300/0 conn(0x7f2e680d2330 0x7f2e680b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.371+0000 7f2e6d91b640 1 -- 172.21.15.40:0/1571581523 >> v1:172.21.15.184:6789/0 conn(0x7f2e680d1770 legacy=0x7f2e680d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.371+0000 7f2e6d91b640 1 -- 172.21.15.40:0/1571581523 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e680b2330 con 0x7f2e6815f3a0 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.371+0000 7f2e6d91b640 1 --2- 172.21.15.40:0/1571581523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e680d06f0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f2e68073b50 tx=0x7f2e6002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc47ae191b09d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6c919640 1 -- 172.21.15.40:0/1571581523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60037070 con 0x7f2e6815f3a0 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6c919640 1 -- 172.21.15.40:0/1571581523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e60032440 con 0x7f2e6815f3a0 2024-04-26T21:45:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6c919640 1 -- 172.21.15.40:0/1571581523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60032810 con 0x7f2e6815f3a0 2024-04-26T21:45:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/1571581523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 msgr2=0x7f2e680d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/1571581523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e680d06f0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f2e68073b50 tx=0x7f2e6002a210 comp rx=0 tx=0).stop 2024-04-26T21:45:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/1571581523 shutdown_connections 2024-04-26T21:45:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/1571581523 >> v2:172.21.15.184:3300/0 conn(0x7f2e680d2330 0x7f2e680b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/1571581523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e680d06f0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/1571581523 >> 172.21.15.40:0/1571581523 conn(0x7f2e680d4f90 msgr2=0x7f2e680d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/1571581523 shutdown_connections 2024-04-26T21:45:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.372+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/1571581523 wait complete. 2024-04-26T21:45:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.373+0000 7f2e6fb7f640 1 Processor -- start 2024-04-26T21:45:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.373+0000 7f2e6fb7f640 1 -- start start 2024-04-26T21:45:28.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6fb7f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e680d1770 0x7f2e680c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6fb7f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e680d2330 0x7f2e680ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6fb7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e68152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6fb7f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e680b2f10 con 0x7f2e6815f3a0 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6fb7f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e680b2d90 con 0x7f2e680d1770 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6fb7f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e680b3090 con 0x7f2e680d2330 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6e11c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e68152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6e11c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e68152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45690/0 (socket says 172.21.15.40:45690) 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6e11c640 1 -- 172.21.15.40:0/2762889545 learned_addr learned my addr 172.21.15.40:0/2762889545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6d11a640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e680d2330 0x7f2e680ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6d91b640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e680d1770 0x7f2e680c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6e11c640 1 -- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e680d2330 msgr2=0x7f2e680ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6e11c640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e680d2330 0x7f2e680ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6e11c640 1 -- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e680d1770 msgr2=0x7f2e680c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6e11c640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e680d1770 0x7f2e680c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6e11c640 1 -- 172.21.15.40:0/2762889545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e680b8980 con 0x7f2e6815f3a0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6d11a640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e680d2330 0x7f2e680ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.374+0000 7f2e6d91b640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e680d1770 0x7f2e680c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.375+0000 7f2e6e11c640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e68152b20 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800b970 tx=0x7f2e5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.375+0000 7f2e667fc640 1 -- 172.21.15.40:0/2762889545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58019070 con 0x7f2e6815f3a0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.375+0000 7f2e667fc640 1 -- 172.21.15.40:0/2762889545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5800cb60 con 0x7f2e6815f3a0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.375+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e680c90c0 con 0x7f2e6815f3a0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.375+0000 7f2e667fc640 1 -- 172.21.15.40:0/2762889545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58014440 con 0x7f2e6815f3a0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.375+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e680c9680 con 0x7f2e6815f3a0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.376+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e680c9e40 con 0x7f2e6815f3a0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.376+0000 7f2e667fc640 1 -- 172.21.15.40:0/2762889545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e5801c070 con 0x7f2e6815f3a0 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.377+0000 7f2e667fc640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e5406c8f0 0x7f2e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.377+0000 7f2e667fc640 1 -- 172.21.15.40:0/2762889545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8797+0+0 (secure 0 0 0) 0x7f2e5808c5e0 con 0x7f2e6815f3a0 2024-04-26T21:45:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.377+0000 7f2e6d91b640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e5406c8f0 0x7f2e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.377+0000 7f2e6d91b640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e5406c8f0 0x7f2e5406edb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2e60034000 tx=0x7f2e6002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.381+0000 7f2e667fc640 1 -- 172.21.15.40:0/2762889545 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e680c9e40 con 0x7f2e6815f3a0 2024-04-26T21:45:28.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.603+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e680694f0 con 0x7f2e6815f3a0 2024-04-26T21:45:28.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.604+0000 7f2e667fc640 1 -- 172.21.15.40:0/2762889545 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 274 v274) ==== 95+0+4355 (secure 0 0 0) 0x7f2e58059d20 con 0x7f2e6815f3a0 2024-04-26T21:45:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:28.605 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":274,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":274,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:27.871871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:28.605 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 274 2024-04-26T21:45:28.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e5406c8f0 msgr2=0x7f2e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e5406c8f0 0x7f2e5406edb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2e60034000 tx=0x7f2e6002e000 comp rx=0 tx=0).stop 2024-04-26T21:45:28.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 msgr2=0x7f2e68152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e68152b20 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800b970 tx=0x7f2e5800be30 comp rx=0 tx=0).stop 2024-04-26T21:45:28.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 shutdown_connections 2024-04-26T21:45:28.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e5406c8f0 0x7f2e5406edb0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6815f3a0 0x7f2e68152b20 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e680d2330 0x7f2e680ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 --2- 172.21.15.40:0/2762889545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e680d1770 0x7f2e680c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 >> 172.21.15.40:0/2762889545 conn(0x7f2e680d4f90 msgr2=0x7f2e6815e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 shutdown_connections 2024-04-26T21:45:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.606+0000 7f2e6fb7f640 1 -- 172.21.15.40:0/2762889545 wait complete. 2024-04-26T21:45:28.617 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:28.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:28.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.717+0000 7f07b3483640 1 Processor -- start 2024-04-26T21:45:28.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.717+0000 7f07b3483640 1 -- start start 2024-04-26T21:45:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07b3483640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f07ac0b2340 0x7f07ac0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07b3483640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07ac0d10d0 con 0x7f07ac0cd930 2024-04-26T21:45:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07b3483640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07ac0d1370 con 0x7f07ac162810 2024-04-26T21:45:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07b3483640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f07ac0d0750 con 0x7f07ac0b2340 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07b1a20640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f07ac0cd930 0x7f07ac0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58284/0 (socket says 172.21.15.40:58284) 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07b1a20640 1 -- 172.21.15.40:0/1732724849 learned_addr learned my addr 172.21.15.40:0/1732724849 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07b0a1e640 1 --2- 172.21.15.40:0/1732724849 >> v2:172.21.15.184:3300/0 conn(0x7f07ac0b2340 0x7f07ac0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3702840326 0 0) 0x7f07ac0d10d0 con 0x7f07ac0cd930 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.718+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0794003520 con 0x7f07ac0cd930 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 158704970 0 0) 0x7f0794003520 con 0x7f07ac0cd930 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 >> v2:172.21.15.184:3300/0 conn(0x7f07ac0b2340 msgr2=0x7f07ac0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 --2- 172.21.15.40:0/1732724849 >> v2:172.21.15.184:3300/0 conn(0x7f07ac0b2340 0x7f07ac0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 >> v1:172.21.15.184:6789/0 conn(0x7f07ac162810 legacy=0x7f07ac070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac0d01b0 con 0x7f07ac0cd930 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f07a4001670 con 0x7f07ac0cd930 2024-04-26T21:45:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 >> v1:172.21.15.40:6789/0 conn(0x7f07ac0cd930 legacy=0x7f07ac0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07940039b0 0x7f0794003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0794004800 0x7f0794004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07940051c0 0x7f079400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac0d0750 con 0x7f0794004800 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f079400c090 con 0x7f07940051c0 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f079400c290 con 0x7f07940039b0 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b0a1e640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0794004800 0x7f0794004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b1a20640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07940051c0 0x7f079400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b121f640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07940039b0 0x7f0794003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b0a1e640 1 -- 172.21.15.40:0/1732724849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07940039b0 msgr2=0x7f0794003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b0a1e640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07940039b0 0x7f0794003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b0a1e640 1 -- 172.21.15.40:0/1732724849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07940051c0 msgr2=0x7f079400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:28.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b0a1e640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07940051c0 0x7f079400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b1a20640 1 -- 172.21.15.40:0/1732724849 reap_dead start 2024-04-26T21:45:28.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.719+0000 7f07b0a1e640 1 -- 172.21.15.40:0/1732724849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0794003520 con 0x7f0794004800 2024-04-26T21:45:28.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.720+0000 7f07b0a1e640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0794004800 0x7f0794004c80 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009ac0 tx=0x7f0794005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b62c5fb75df583 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.720+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0036070 con 0x7f0794004800 2024-04-26T21:45:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.720+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a0031400 con 0x7f0794004800 2024-04-26T21:45:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.720+0000 7f07abfff640 1 -- 172.21.15.40:0/1732724849 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0009690 con 0x7f0794004800 2024-04-26T21:45:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.721+0000 7f07b3483640 1 -- 172.21.15.40:0/1732724849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0794004800 msgr2=0x7f0794004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.721+0000 7f07b3483640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0794004800 0x7f0794004c80 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009ac0 tx=0x7f0794005830 comp rx=0 tx=0).stop 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.721+0000 7f07b3483640 1 -- 172.21.15.40:0/1732724849 shutdown_connections 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.721+0000 7f07b3483640 1 --2- 172.21.15.40:0/1732724849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0794004800 0x7f0794004c80 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.721+0000 7f07b3483640 1 -- 172.21.15.40:0/1732724849 >> 172.21.15.40:0/1732724849 conn(0x7f07ac0d2da0 msgr2=0x7f07ac0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.721+0000 7f07b3483640 1 -- 172.21.15.40:0/1732724849 shutdown_connections 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.721+0000 7f07b3483640 1 -- 172.21.15.40:0/1732724849 wait complete. 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.722+0000 7f07b3483640 1 Processor -- start 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.722+0000 7f07b3483640 1 -- start start 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b3483640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0794004800 0x7f07ac156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b3483640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ac161d10 0x7f07ac1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b3483640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07ac0ca4f0 0x7f07ac0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b3483640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac0cd3f0 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b3483640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac0d16a0 con 0x7f07ac161d10 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b3483640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f07ac0cd570 con 0x7f0794004800 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07ac0ca4f0 0x7f07ac0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07ac0ca4f0 0x7f07ac0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41258/0 (socket says 172.21.15.40:41258) 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 -- 172.21.15.40:0/3204747309 learned_addr learned my addr 172.21.15.40:0/3204747309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b121f640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0794004800 0x7f07ac156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b0a1e640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ac161d10 0x7f07ac1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 -- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0794004800 msgr2=0x7f07ac156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0794004800 0x7f07ac156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 -- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ac161d10 msgr2=0x7f07ac1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ac161d10 0x7f07ac1599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 -- 172.21.15.40:0/3204747309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac0cc8b0 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b1a20640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07ac0ca4f0 0x7f07ac0ca980 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f07ac076320 tx=0x7f07a4007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07a9ffb640 1 -- 172.21.15.40:0/3204747309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a401a070 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07a9ffb640 1 -- 172.21.15.40:0/3204747309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a400f6a0 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac0ccb90 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.723+0000 7f07a9ffb640 1 -- 172.21.15.40:0/3204747309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a400fa90 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.724+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac0cc560 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.725+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ac0694f0 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.725+0000 7f07a9ffb640 1 -- 172.21.15.40:0/3204747309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07a401d070 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.725+0000 7f07a9ffb640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f079806c8f0 0x7f079806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.726+0000 7f07a9ffb640 1 -- 172.21.15.40:0/3204747309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8797+0+0 (secure 0 0 0) 0x7f07a408bc80 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.726+0000 7f07b121f640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f079806c8f0 0x7f079806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.726+0000 7f07b121f640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f079806c8f0 0x7f079806edb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f079c000f80 tx=0x7f079c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:28.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.730+0000 7f07a9ffb640 1 -- 172.21.15.40:0/3204747309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07a4058e30 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.959+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07ac0b28e0 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.959+0000 7f07a9ffb640 1 -- 172.21.15.40:0/3204747309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 275 v275) ==== 95+0+5182 (secure 0 0 0) 0x7f07a405d130 con 0x7f07ac0ca4f0 2024-04-26T21:45:28.960 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:28.961 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":275,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10614,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2896027915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2896027915},{"type":"v1","addr":"172.21.15.40:6833","nonce":2896027915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":275}],"filesystems":[{"mdsmap":{"epoch":274,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:27.871871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:28.961 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 275 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f079806c8f0 msgr2=0x7f079806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f079806c8f0 0x7f079806edb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f079c000f80 tx=0x7f079c009040 comp rx=0 tx=0).stop 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07ac0ca4f0 msgr2=0x7f07ac0ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07ac0ca4f0 0x7f07ac0ca980 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f07ac076320 tx=0x7f07a4007430 comp rx=0 tx=0).stop 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 shutdown_connections 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f079806c8f0 0x7f079806edb0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07ac0ca4f0 0x7f07ac0ca980 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ac161d10 0x7f07ac1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 --2- 172.21.15.40:0/3204747309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0794004800 0x7f07ac156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 >> 172.21.15.40:0/3204747309 conn(0x7f07ac0d2da0 msgr2=0x7f07ac06d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 shutdown_connections 2024-04-26T21:45:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:28.962+0000 7f07b3483640 1 -- 172.21.15.40:0/3204747309 wait complete. 2024-04-26T21:45:28.973 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 400.00000000' led to DAMAGED state 2024-04-26T21:45:28.974 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 100.00000000.inode' 2024-04-26T21:45:28.974 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:28.974 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:28.989 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:45:28.989 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:28.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:28.989 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:29.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:45:29.044 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:45:29.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:29.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.185+0000 7f63b04fc640 1 Processor -- start 2024-04-26T21:45:29.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.185+0000 7f63b04fc640 1 -- start start 2024-04-26T21:45:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.186+0000 7f63b04fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a815f3a0 0x7f63a80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.186+0000 7f63b04fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a80d0e20 con 0x7f63a80d1770 2024-04-26T21:45:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.186+0000 7f63b04fc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a80b1d70 con 0x7f63a80d2300 2024-04-26T21:45:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.186+0000 7f63b04fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63a80b2050 con 0x7f63a815f3a0 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.186+0000 7f63ada97640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63a80d1770 0x7f63a80d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58296/0 (socket says 172.21.15.40:58296) 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.186+0000 7f63ada97640 1 -- 172.21.15.40:0/4036882193 learned_addr learned my addr 172.21.15.40:0/4036882193 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2846478376 0 0) 0x7f63a80d0e20 con 0x7f63a80d1770 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ae298640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a815f3a0 0x7f63a80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6394003520 con 0x7f63a80d1770 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2738088533 0 0) 0x7f6394003520 con 0x7f63a80d1770 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a815f3a0 msgr2=0x7f63a80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a815f3a0 0x7f63a80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 >> v1:172.21.15.184:6789/0 conn(0x7f63a80d2300 legacy=0x7f63a80b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a80b2330 con 0x7f63a80d1770 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f6398001670 con 0x7f63a80d1770 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 >> v1:172.21.15.40:6789/0 conn(0x7f63a80d1770 legacy=0x7f63a80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63940039e0 0x7f6394003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a815f3a0 0x7f6394004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6394004d40 0x7f63940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f63a80b2050 con 0x7f6394004d40 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f639400ba70 con 0x7f63940039e0 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f639400bc70 con 0x7f63a815f3a0 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63aea99640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6394004d40 0x7f63940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ae298640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63940039e0 0x7f6394003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.187+0000 7f63ada97640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a815f3a0 0x7f6394004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63aea99640 1 -- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a815f3a0 msgr2=0x7f6394004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63aea99640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63a815f3a0 0x7f6394004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63aea99640 1 -- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63940039e0 msgr2=0x7f6394003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63aea99640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63940039e0 0x7f6394003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63aea99640 1 -- 172.21.15.40:0/4036882193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6394003520 con 0x7f6394004d40 2024-04-26T21:45:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63aea99640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6394004d40 0x7f63940051c0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f63a80d2750 tx=0x7f6394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc05b147435fa339 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0001a70 con 0x7f6394004d40 2024-04-26T21:45:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a002ae10 con 0x7f6394004d40 2024-04-26T21:45:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.188+0000 7f63ad296640 1 -- 172.21.15.40:0/4036882193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0003bb0 con 0x7f6394004d40 2024-04-26T21:45:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.189+0000 7f63b04fc640 1 -- 172.21.15.40:0/4036882193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6394004d40 msgr2=0x7f63940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.189+0000 7f63b04fc640 1 --2- 172.21.15.40:0/4036882193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6394004d40 0x7f63940051c0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f63a80d2750 tx=0x7f6394005830 comp rx=0 tx=0).stop 2024-04-26T21:45:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.189+0000 7f63aea99640 1 -- 172.21.15.40:0/4036882193 reap_dead start 2024-04-26T21:45:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.189+0000 7f63b04fc640 1 -- 172.21.15.40:0/4036882193 shutdown_connections 2024-04-26T21:45:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.189+0000 7f63b04fc640 1 -- 172.21.15.40:0/4036882193 >> 172.21.15.40:0/4036882193 conn(0x7f63a80d4f90 msgr2=0x7f63a80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.189+0000 7f63b04fc640 1 -- 172.21.15.40:0/4036882193 shutdown_connections 2024-04-26T21:45:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.189+0000 7f63b04fc640 1 -- 172.21.15.40:0/4036882193 wait complete. 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.190+0000 7f63b04fc640 1 Processor -- start 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.190+0000 7f63b04fc640 1 -- start start 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63b04fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6394004d40 0x7f63a815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63b04fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a8161920 0x7f63a80c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63b04fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a80c93d0 0x7f63a80c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63b04fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f63a80d0870 con 0x7f63a8161920 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63b04fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63a80d06f0 con 0x7f63a80c93d0 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63b04fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63a80d09f0 con 0x7f6394004d40 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ada97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a8161920 0x7f63a80c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ada97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a8161920 0x7f63a80c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41288/0 (socket says 172.21.15.40:41288) 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ada97640 1 -- 172.21.15.40:0/3851509269 learned_addr learned my addr 172.21.15.40:0/3851509269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ae298640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6394004d40 0x7f63a815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63aea99640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a80c93d0 0x7f63a80c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ada97640 1 -- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6394004d40 msgr2=0x7f63a815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ada97640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6394004d40 0x7f63a815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ada97640 1 -- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a80c93d0 msgr2=0x7f63a80c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ada97640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a80c93d0 0x7f63a80c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63ada97640 1 -- 172.21.15.40:0/3851509269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a80cc060 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.191+0000 7f63aea99640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a80c93d0 0x7f63a80c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.192+0000 7f63ada97640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a8161920 0x7f63a80c8e90 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f63980072c0 tx=0x7f6398007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.192+0000 7f639effd640 1 -- 172.21.15.40:0/3851509269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639800f960 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.192+0000 7f639effd640 1 -- 172.21.15.40:0/3851509269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6398018070 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.192+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a8161e20 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.192+0000 7f639effd640 1 -- 172.21.15.40:0/3851509269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639800fd60 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.192+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81623e0 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.193+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a80694f0 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.193+0000 7f639effd640 1 -- 172.21.15.40:0/3851509269 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f639801d460 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.193+0000 7f639effd640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f638c06c920 0x7f638c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.194+0000 7f639effd640 1 -- 172.21.15.40:0/3851509269 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8797+0+0 (secure 0 0 0) 0x7f639808c8f0 con 0x7f63a8161920 2024-04-26T21:45:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.194+0000 7f63ae298640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f638c06c920 0x7f638c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:29.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.194+0000 7f63ae298640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f638c06c920 0x7f638c06ede0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f63a80ca240 tx=0x7f63a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:29.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.198+0000 7f639effd640 1 -- 172.21.15.40:0/3851509269 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f639805a0c0 con 0x7f63a8161920 2024-04-26T21:45:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.426+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f63a80babb0 con 0x7f63a8161920 2024-04-26T21:45:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.886+0000 7f639effd640 1 -- 172.21.15.40:0/3851509269 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v276) ==== 161+0+0 (secure 0 0 0) 0x7f63980599c0 con 0x7f63a8161920 2024-04-26T21:45:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f638c06c920 msgr2=0x7f638c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f638c06c920 0x7f638c06ede0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f63a80ca240 tx=0x7f63a4008040 comp rx=0 tx=0).stop 2024-04-26T21:45:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a8161920 msgr2=0x7f63a80c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a8161920 0x7f63a80c8e90 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f63980072c0 tx=0x7f6398007780 comp rx=0 tx=0).stop 2024-04-26T21:45:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 shutdown_connections 2024-04-26T21:45:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f638c06c920 0x7f638c06ede0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63a80c93d0 0x7f63a80c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a8161920 0x7f63a80c8e90 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 --2- 172.21.15.40:0/3851509269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6394004d40 0x7f63a815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 >> 172.21.15.40:0/3851509269 conn(0x7f63a80d4f90 msgr2=0x7f63a80d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 shutdown_connections 2024-04-26T21:45:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:29.888+0000 7f63b04fc640 1 -- 172.21.15.40:0/3851509269 wait complete. 2024-04-26T21:45:29.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.004+0000 7fcf3c223640 1 Processor -- start 2024-04-26T21:45:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.005+0000 7fcf3c223640 1 -- start start 2024-04-26T21:45:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.005+0000 7fcf3c223640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf340b5f20 0x7fcf340d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.005+0000 7fcf3c223640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf340b7e50 con 0x7fcf34162880 2024-04-26T21:45:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf3c223640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf340d1800 con 0x7fcf340d3bc0 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf3c223640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf340d0580 con 0x7fcf340b5f20 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf39fbf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcf34162880 0x7fcf340d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58312/0 (socket says 172.21.15.40:58312) 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf39fbf640 1 -- 172.21.15.40:0/2206395338 learned_addr learned my addr 172.21.15.40:0/2206395338 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967884170 0 0) 0x7fcf340b7e50 con 0x7fcf34162880 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf3a7c0640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf340b5f20 0x7fcf340d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf1c003520 con 0x7fcf34162880 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4146928682 0 0) 0x7fcf1c003520 con 0x7fcf34162880 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf340b5f20 msgr2=0x7fcf340d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf340b5f20 0x7fcf340d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 >> v1:172.21.15.184:6789/0 conn(0x7fcf340d3bc0 legacy=0x7fcf340b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340d07d0 con 0x7fcf34162880 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fcf2c001670 con 0x7fcf34162880 2024-04-26T21:45:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 >> v1:172.21.15.40:6789/0 conn(0x7fcf34162880 legacy=0x7fcf340d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1c0039e0 0x7fcf1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf1c004800 0x7fcf1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf340b5f20 0x7fcf1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf340d0580 con 0x7fcf1c0039e0 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c00ba70 con 0x7fcf1c004800 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c00bc70 con 0x7fcf340b5f20 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.006+0000 7fcf39fbf640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1c0039e0 0x7fcf1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.007+0000 7fcf3a7c0640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf340b5f20 0x7fcf1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.007+0000 7fcf397be640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf1c004800 0x7fcf1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.007+0000 7fcf39fbf640 1 -- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf340b5f20 msgr2=0x7fcf1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.007+0000 7fcf39fbf640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf340b5f20 0x7fcf1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.007+0000 7fcf39fbf640 1 -- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf1c004800 msgr2=0x7fcf1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.007+0000 7fcf39fbf640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf1c004800 0x7fcf1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.007+0000 7fcf39fbf640 1 -- 172.21.15.40:0/2206395338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c003520 con 0x7fcf1c0039e0 2024-04-26T21:45:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.007+0000 7fcf39fbf640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1c0039e0 0x7fcf1c003e20 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c210 tx=0x7fcf1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523a4a4818ddabc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c007d10 con 0x7fcf1c0039e0 2024-04-26T21:45:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c005ad0 con 0x7fcf1c0039e0 2024-04-26T21:45:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf38fbd640 1 -- 172.21.15.40:0/2206395338 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c007070 con 0x7fcf1c0039e0 2024-04-26T21:45:30.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf3c223640 1 -- 172.21.15.40:0/2206395338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1c0039e0 msgr2=0x7fcf1c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf3c223640 1 --2- 172.21.15.40:0/2206395338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1c0039e0 0x7fcf1c003e20 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c210 tx=0x7fcf1c005830 comp rx=0 tx=0).stop 2024-04-26T21:45:30.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf3a7c0640 1 -- 172.21.15.40:0/2206395338 reap_dead start 2024-04-26T21:45:30.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf3c223640 1 -- 172.21.15.40:0/2206395338 shutdown_connections 2024-04-26T21:45:30.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf3c223640 1 -- 172.21.15.40:0/2206395338 >> 172.21.15.40:0/2206395338 conn(0x7fcf340d4f60 msgr2=0x7fcf340d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:30.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf3c223640 1 -- 172.21.15.40:0/2206395338 shutdown_connections 2024-04-26T21:45:30.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.008+0000 7fcf3c223640 1 -- 172.21.15.40:0/2206395338 wait complete. 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.009+0000 7fcf3c223640 1 Processor -- start 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.009+0000 7fcf3c223640 1 -- start start 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf3c223640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf1c0039e0 0x7fcf3414daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf3c223640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf340b4350 0x7fcf340b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf3c223640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf340ccff0 0x7fcf340cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf3c223640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34162c60 con 0x7fcf340b4350 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf3c223640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf340d0d90 con 0x7fcf340ccff0 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf3c223640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf34162de0 con 0x7fcf1c0039e0 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf397be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf340b4350 0x7fcf340b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf397be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf340b4350 0x7fcf340b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41310/0 (socket says 172.21.15.40:41310) 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf397be640 1 -- 172.21.15.40:0/1557950664 learned_addr learned my addr 172.21.15.40:0/1557950664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf3a7c0640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf340ccff0 0x7fcf340cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.010+0000 7fcf39fbf640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf1c0039e0 0x7fcf3414daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf397be640 1 -- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf1c0039e0 msgr2=0x7fcf3414daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf397be640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf1c0039e0 0x7fcf3414daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf397be640 1 -- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf340ccff0 msgr2=0x7fcf340cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf397be640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf340ccff0 0x7fcf340cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf397be640 1 -- 172.21.15.40:0/1557950664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340cdbd0 con 0x7fcf340b4350 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf397be640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf340b4350 0x7fcf340b47d0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800f5f0 tx=0x7fcf2800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf327fc640 1 -- 172.21.15.40:0/1557950664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280052e0 con 0x7fcf340b4350 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf327fc640 1 -- 172.21.15.40:0/1557950664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf28018070 con 0x7fcf340b4350 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf340cdeb0 con 0x7fcf340b4350 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf327fc640 1 -- 172.21.15.40:0/1557950664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28003810 con 0x7fcf340b4350 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.011+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf340ce470 con 0x7fcf340b4350 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.012+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf340694f0 con 0x7fcf340b4350 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.012+0000 7fcf327fc640 1 -- 172.21.15.40:0/1557950664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf280039b0 con 0x7fcf340b4350 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.013+0000 7fcf327fc640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcf2006c8f0 0x7fcf2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.013+0000 7fcf327fc640 1 -- 172.21.15.40:0/1557950664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8797+0+0 (secure 0 0 0) 0x7fcf2808c560 con 0x7fcf340b4350 2024-04-26T21:45:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.013+0000 7fcf39fbf640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcf2006c8f0 0x7fcf2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.014+0000 7fcf39fbf640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcf2006c8f0 0x7fcf2006edb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c032000 tx=0x7fcf2c02d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.017+0000 7fcf327fc640 1 -- 172.21.15.40:0/1557950664 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf340d1800 con 0x7fcf340b4350 2024-04-26T21:45:30.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.274+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fcf340d3030 con 0x7fcf340b4350 2024-04-26T21:45:30.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.891+0000 7fcf327fc640 1 -- 172.21.15.40:0/1557950664 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v277) ==== 101+0+0 (secure 0 0 0) 0x7fcf28059d30 con 0x7fcf340b4350 2024-04-26T21:45:30.892 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:45:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcf2006c8f0 msgr2=0x7fcf2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcf2006c8f0 0x7fcf2006edb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c032000 tx=0x7fcf2c02d000 comp rx=0 tx=0).stop 2024-04-26T21:45:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf340b4350 msgr2=0x7fcf340b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf340b4350 0x7fcf340b47d0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800f5f0 tx=0x7fcf2800fab0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 shutdown_connections 2024-04-26T21:45:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcf2006c8f0 0x7fcf2006edb0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf340ccff0 0x7fcf340cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf340b4350 0x7fcf340b47d0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 --2- 172.21.15.40:0/1557950664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf1c0039e0 0x7fcf3414daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 >> 172.21.15.40:0/1557950664 conn(0x7fcf340d4f60 msgr2=0x7fcf340ba210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 shutdown_connections 2024-04-26T21:45:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.893+0000 7fcf3c223640 1 -- 172.21.15.40:0/1557950664 wait complete. 2024-04-26T21:45:30.907 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:30.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.950+0000 7fde092cbf40 1 Processor -- start 2024-04-26T21:45:30.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.950+0000 7fde092cbf40 1 -- start start 2024-04-26T21:45:30.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde092cbf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde092cbf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 0x55de2d9fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde092cbf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 0x55de2daa9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde092cbf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55de2da19660 con 0x55de2da1a420 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde092cbf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55de2da19940 con 0x55de2daa1f70 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde092cbf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55de2da19c20 con 0x55de2daa8b70 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde06dc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde06dc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41326/0 (socket says 172.21.15.40:41326) 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde06dc5640 1 -- 172.21.15.40:0/1334578650 learned_addr learned my addr 172.21.15.40:0/1334578650 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde065c4640 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 0x55de2d9fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde075c6640 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 0x55de2daa9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde06dc5640 1 -- 172.21.15.40:0/1334578650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 msgr2=0x55de2daa9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde06dc5640 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 0x55de2daa9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde06dc5640 1 -- 172.21.15.40:0/1334578650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 msgr2=0x55de2d9fbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde06dc5640 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 0x55de2d9fbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.951+0000 7fde06dc5640 1 -- 172.21.15.40:0/1334578650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2d8facb0 con 0x55de2da1a420 2024-04-26T21:45:30.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde06dc5640 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d15f0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fde0000a7a0 tx=0x7fde0000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a5aec5a2575f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:30.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde055c2640 1 -- 172.21.15.40:0/1334578650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00018070 con 0x55de2da1a420 2024-04-26T21:45:30.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde055c2640 1 -- 172.21.15.40:0/1334578650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde00007600 con 0x55de2da1a420 2024-04-26T21:45:30.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde055c2640 1 -- 172.21.15.40:0/1334578650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde000079b0 con 0x55de2da1a420 2024-04-26T21:45:30.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde092cbf40 1 -- 172.21.15.40:0/1334578650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 msgr2=0x55de2d9d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde092cbf40 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d15f0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fde0000a7a0 tx=0x7fde0000a890 comp rx=0 tx=0).stop 2024-04-26T21:45:30.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde092cbf40 1 -- 172.21.15.40:0/1334578650 shutdown_connections 2024-04-26T21:45:30.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde092cbf40 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 0x55de2daa9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde092cbf40 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 0x55de2d9fbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde092cbf40 1 --2- 172.21.15.40:0/1334578650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d15f0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde092cbf40 1 -- 172.21.15.40:0/1334578650 >> 172.21.15.40:0/1334578650 conn(0x55de2d9fa8a0 msgr2=0x55de2daaa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.952+0000 7fde092cbf40 1 -- 172.21.15.40:0/1334578650 shutdown_connections 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.953+0000 7fde092cbf40 1 -- 172.21.15.40:0/1334578650 wait complete. 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.953+0000 7fde092cbf40 1 Processor -- start 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.953+0000 7fde092cbf40 1 -- start start 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde092cbf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde092cbf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 0x55de2da98630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde092cbf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 0x55de2daa6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde092cbf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55de2da00070 con 0x55de2da1a420 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde092cbf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55de2da995c0 con 0x55de2daa1f70 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde092cbf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55de2daaa7d0 con 0x55de2daa8b70 2024-04-26T21:45:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41330/0 (socket says 172.21.15.40:41330) 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 learned_addr learned my addr 172.21.15.40:0/2731189601 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde075c6640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 0x55de2daa6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde065c4640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 0x55de2da98630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 msgr2=0x55de2daa6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de2daa8b70 0x55de2daa6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 msgr2=0x55de2da98630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de2daa1f70 0x55de2da98630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2daaafe0 con 0x55de2da1a420 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.954+0000 7fde06dc5640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d1330 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fde00000c00 tx=0x7fde00003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.955+0000 7fddeffff640 1 -- 172.21.15.40:0/2731189601 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00018070 con 0x55de2da1a420 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.955+0000 7fddeffff640 1 -- 172.21.15.40:0/2731189601 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde00019040 con 0x55de2da1a420 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.955+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de2da98b70 con 0x55de2da1a420 2024-04-26T21:45:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.955+0000 7fddeffff640 1 -- 172.21.15.40:0/2731189601 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00007c00 con 0x55de2da1a420 2024-04-26T21:45:30.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.955+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de2d9d19c0 con 0x55de2da1a420 2024-04-26T21:45:30.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.956+0000 7fddeffff640 1 -- 172.21.15.40:0/2731189601 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde00013410 con 0x55de2da1a420 2024-04-26T21:45:30.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.956+0000 7fddeffff640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddf406b2d0 0x7fddf406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.957+0000 7fddeffff640 1 -- 172.21.15.40:0/2731189601 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8797+0+0 (secure 0 0 0) 0x7fde0008b900 con 0x55de2da1a420 2024-04-26T21:45:30.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.957+0000 7fde065c4640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddf406b2d0 0x7fddf406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.960 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.957+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55de2db3e800 0x55de2db3ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.957+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db3f120 con 0x55de2db3e800 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.957+0000 7fde075c6640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55de2db3e800 0x55de2db3ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.957+0000 7fde065c4640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddf406b2d0 0x7fddf406d790 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fddfc057000 tx=0x7fddfc051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.957+0000 7fde075c6640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55de2db3e800 0x55de2db3ebe0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.959+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v125'908 uv882 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.959+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db3f500 con 0x55de2db3e800 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.960+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v125'909 uv909 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.960+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db3f8d0 con 0x55de2db3e800 2024-04-26T21:45:30.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.961+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v125'910 uv910 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.962 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:30.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.961+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4ba30 con 0x55de2db3e800 2024-04-26T21:45:30.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.962+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v125'911 uv911 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.962+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4bdd0 con 0x55de2db3e800 2024-04-26T21:45:30.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.964+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v125'912 uv912 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.964+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4aa20 con 0x55de2db3e800 2024-04-26T21:45:30.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.965+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v125'913 uv913 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.965+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4adc0 con 0x55de2db3e800 2024-04-26T21:45:30.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.966+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v125'914 uv888 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.966+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db4b1b0 con 0x55de2db3e800 2024-04-26T21:45:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.967+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v125'915 uv915 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.967+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b1b0 con 0x55de2db3e800 2024-04-26T21:45:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.968+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v125'916 uv916 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.969 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.968+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.969+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v125'917 uv917 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.969+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.971+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v125'918 uv918 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.971+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.972+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v125'919 uv919 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.972+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.973+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v125'920 uv895 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.973+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.974+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v125'921 uv921 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.974+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.976 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.975+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v125'922 uv922 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.975+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.976+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v125'923 uv923 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.976+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.977+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v125'924 uv924 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.977+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4b160 con 0x55de2db3e800 2024-04-26T21:45:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.978+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v125'925 uv925 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.978+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.979+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v125'926 uv926 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.979+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.981+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v125'927 uv901 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.981+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.982+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v125'928 uv928 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.982+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.983+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v125'929 uv929 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.983 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.983+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4d180 con 0x55de2db3e800 2024-04-26T21:45:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.984+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v125'930 uv930 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.984+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4d180 con 0x55de2db3e800 2024-04-26T21:45:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.985+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v125'931 uv931 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.985+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.986+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v125'932 uv932 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.986+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.987+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v125'933 uv907 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.987+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.988+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v125'934 uv934 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.988+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.989 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.989+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v125'935 uv935 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.989+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4d180 con 0x55de2db3e800 2024-04-26T21:45:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.990+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v125'936 uv936 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.990+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4d180 con 0x55de2db3e800 2024-04-26T21:45:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.991+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v125'937 uv937 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.991+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4c170 con 0x55de2db3e800 2024-04-26T21:45:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.992+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v125'938 uv938 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db3e800 2024-04-26T21:45:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.992+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55de2db4c680 0x55de2db4ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.992+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4efb0 con 0x55de2db4c680 2024-04-26T21:45:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.992+0000 7fde06dc5640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55de2db4c680 0x55de2db4ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.993+0000 7fde06dc5640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55de2db4c680 0x55de2db4ea70 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.994+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v125'659 uv642 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.995+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db4f350 con 0x55de2db4c680 2024-04-26T21:45:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.996+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v125'660 uv660 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.996+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4f6f0 con 0x55de2db4c680 2024-04-26T21:45:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.997+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v125'661 uv661 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:30.998 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.997+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db4ff40 con 0x55de2db4c680 2024-04-26T21:45:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.999+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v125'662 uv662 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:30.999+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db502e0 con 0x55de2db4c680 2024-04-26T21:45:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.000+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v125'663 uv663 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.000+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50680 con 0x55de2db4c680 2024-04-26T21:45:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.001+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v125'664 uv647 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.001+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.003+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v125'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.003+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.004+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v125'666 uv666 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.006 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.004+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.005+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v125'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.005+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.006+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v125'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.006+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.008+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v125'669 uv652 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.008+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.009+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v125'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.009+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.011+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v125'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.011 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.011+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db550e0 con 0x55de2db4c680 2024-04-26T21:45:31.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.012+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v125'672 uv672 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.012+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db550e0 con 0x55de2db4c680 2024-04-26T21:45:31.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.013+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v125'673 uv673 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.013+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.015+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v125'674 uv658 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.015+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.016+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v125'675 uv675 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.016+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.018 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:31.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.018+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v125'676 uv676 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.018+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db51a30 con 0x55de2db4c680 2024-04-26T21:45:31.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.019+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v125'677 uv677 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.019+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db51a30 con 0x55de2db4c680 2024-04-26T21:45:31.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.020+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v125'678 uv678 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.020+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db50a20 con 0x55de2db4c680 2024-04-26T21:45:31.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.021+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v125'679 uv679 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db4c680 2024-04-26T21:45:31.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.021+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55de2db50f30 0x55de2db53320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.021+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db53940 con 0x55de2db50f30 2024-04-26T21:45:31.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.021+0000 7fde065c4640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55de2db50f30 0x55de2db53320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.022+0000 7fde065c4640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55de2db50f30 0x55de2db53320 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.023+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v125'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.023+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db53e30 con 0x55de2db50f30 2024-04-26T21:45:31.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.024+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v125'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.024+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db54310 con 0x55de2db50f30 2024-04-26T21:45:31.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.025+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v125'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.026 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:31.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.025+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db55bc0 con 0x55de2db50f30 2024-04-26T21:45:31.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.027+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v125'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.027+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db55f60 con 0x55de2db50f30 2024-04-26T21:45:31.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.028+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v125'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.028+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db54bb0 con 0x55de2db50f30 2024-04-26T21:45:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.029+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v125'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.029+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55de2db554b0 0x55de2db58310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.029+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db58850 con 0x55de2db554b0 2024-04-26T21:45:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.029+0000 7fde075c6640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55de2db554b0 0x55de2db58310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.030+0000 7fde075c6640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55de2db554b0 0x55de2db58310 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.031+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v125'135 uv134 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.031+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db58850 con 0x55de2db554b0 2024-04-26T21:45:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.032+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v125'136 uv136 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.032+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db588d0 con 0x55de2db554b0 2024-04-26T21:45:31.034 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:31.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.033+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v125'137 uv137 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.033+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db58cc0 con 0x55de2db554b0 2024-04-26T21:45:31.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.035+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v125'138 uv138 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.035+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55de2db591d0 0x55de2db5b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.035+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bb40 con 0x55de2db591d0 2024-04-26T21:45:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.035+0000 7fde06dc5640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55de2db591d0 0x55de2db5b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.035+0000 7fde06dc5640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55de2db591d0 0x55de2db5b5c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.037+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v125'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.037+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5bc00 con 0x55de2db591d0 2024-04-26T21:45:31.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.038+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v125'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.038+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.040 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:31.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.039+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v125'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.039+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5d000 con 0x55de2db591d0 2024-04-26T21:45:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.040+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v125'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.040+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5d000 con 0x55de2db591d0 2024-04-26T21:45:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.041+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v125'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.041+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.043+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v125'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.043+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.044+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v125'327 uv321 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.044+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.045+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v125'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.045+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.047 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:31.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.046+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v125'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.046+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5d000 con 0x55de2db554b0 2024-04-26T21:45:31.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.047+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v125'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.047+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5d000 con 0x55de2db554b0 2024-04-26T21:45:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.048+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v125'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.048+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.049+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v125'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.050+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.051+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v125'333 uv326 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.051+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.052+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v125'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.052+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.053 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.053+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v125'335 uv335 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.053+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.054+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v125'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.054+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.055+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v125'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.055+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.056+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v125'148 uv147 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.056+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.057+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v125'149 uv149 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.057+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.058+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v125'150 uv150 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.059 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:31.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.058+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.059+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v125'151 uv151 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.060+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db554b0 2024-04-26T21:45:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.061+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v125'152 uv152 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf800d950 con 0x55de2db554b0 2024-04-26T21:45:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.061+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.062+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v125'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.062+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.063+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v125'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.063+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.064 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.064+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v125'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.064+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5d000 con 0x55de2db591d0 2024-04-26T21:45:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.065+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v125'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.065+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5d000 con 0x55de2db591d0 2024-04-26T21:45:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.066+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v125'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.066+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.067+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v125'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.067+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.068+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v125'294 uv288 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.068+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.069+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v125'295 uv295 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.069+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5bff0 con 0x55de2db591d0 2024-04-26T21:45:31.071 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.070+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v125'296 uv296 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.070+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5c3b0 con 0x55de2db591d0 2024-04-26T21:45:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.071+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v125'297 uv297 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.071+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5c3b0 con 0x55de2db591d0 2024-04-26T21:45:31.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.072+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v125'298 uv298 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.072+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5c3b0 con 0x55de2db591d0 2024-04-26T21:45:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.073+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v125'299 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.073+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5c3b0 con 0x55de2db591d0 2024-04-26T21:45:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.074+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v125'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.075+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5c3b0 con 0x55de2db591d0 2024-04-26T21:45:31.076 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:31.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.075+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v125'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.075+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5c3b0 con 0x55de2db591d0 2024-04-26T21:45:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.076+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v125'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.076+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5f1c0 con 0x55de2db591d0 2024-04-26T21:45:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.078+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v125'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.078+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55de2db5cd70 0x55de2db5f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.078+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5f6a0 con 0x55de2db5cd70 2024-04-26T21:45:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.078+0000 7fde065c4640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55de2db5cd70 0x55de2db5f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.078+0000 7fde065c4640 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55de2db5cd70 0x55de2db5f160 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.079+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v125'128 uv127 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db5cd70 2024-04-26T21:45:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.079+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db5cd70 2024-04-26T21:45:31.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.081+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v125'129 uv129 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db5cd70 2024-04-26T21:45:31.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.081+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db5cd70 2024-04-26T21:45:31.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.082+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v125'130 uv130 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db5cd70 2024-04-26T21:45:31.082 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.082+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db5cd70 2024-04-26T21:45:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.083+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v125'131 uv131 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db5cd70 2024-04-26T21:45:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.083+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db591d0 2024-04-26T21:45:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.084+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v125'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.084+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db591d0 2024-04-26T21:45:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.085+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v125'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.085+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db591d0 2024-04-26T21:45:31.087 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.086+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v125'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.086+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db60a50 con 0x55de2db591d0 2024-04-26T21:45:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.087+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v125'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.087+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db60a50 con 0x55de2db591d0 2024-04-26T21:45:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.088+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v125'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.089+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db591d0 2024-04-26T21:45:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.090+0000 7fde06dc5640 1 -- 172.21.15.40:0/2731189601 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v125'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0001b070 con 0x55de2db591d0 2024-04-26T21:45:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.090+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.091+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v125'151 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.091+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.092+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v125'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.092+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.094 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.093+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v125'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.093+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.095+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v125'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.095+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.096+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v125'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.096+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.097+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v125'363 uv355 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.097+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.098+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v125'364 uv364 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.099+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.100 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:31.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.100+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v125'365 uv365 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.100+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.101+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v125'366 uv366 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.101+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.102+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v125'367 uv367 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.102+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.103+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v125'368 uv362 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.103+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.104+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v125'369 uv369 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.104+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.105 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:31.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.105+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v125'370 uv370 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.105+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db60a50 con 0x55de2db50f30 2024-04-26T21:45:31.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.106+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v125'371 uv371 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.106+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db60a50 con 0x55de2db50f30 2024-04-26T21:45:31.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.107+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v125'372 uv372 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.107+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fa40 con 0x55de2db50f30 2024-04-26T21:45:31.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.108+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v125'373 uv373 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.108+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55de2db5fef0 con 0x55de2db50f30 2024-04-26T21:45:31.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde065c4640 1 -- 172.21.15.40:0/2731189601 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v125'374 uv374 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc06c040 con 0x55de2db50f30 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55de2db4c680 msgr2=0x55de2db4ea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55de2db4c680 0x55de2db4ea70 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55de2db3e800 msgr2=0x55de2db3ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55de2db3e800 0x55de2db3ebe0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55de2db554b0 msgr2=0x55de2db58310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55de2db554b0 0x55de2db58310 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55de2db5cd70 msgr2=0x55de2db5f160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55de2db5cd70 0x55de2db5f160 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55de2db50f30 msgr2=0x55de2db53320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55de2db50f30 0x55de2db53320 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55de2db591d0 msgr2=0x55de2db5b5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55de2db591d0 0x55de2db5b5c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 reap_dead start 2024-04-26T21:45:31.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 reap_dead start 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde075c6640 1 -- 172.21.15.40:0/2731189601 reap_dead start 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddf406b2d0 msgr2=0x7fddf406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddf406b2d0 0x7fddf406d790 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fddfc057000 tx=0x7fddfc051000 comp rx=0 tx=0).stop 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 msgr2=0x55de2d9d1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d1330 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fde00000c00 tx=0x7fde00003020 comp rx=0 tx=0).stop 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 shutdown_connections 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddf406b2d0 0x7fddf406d790 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 --2- 172.21.15.40:0/2731189601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55de2da1a420 0x55de2d9d1330 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 >> 172.21.15.40:0/2731189601 conn(0x55de2d9fa8a0 msgr2=0x55de2daa57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 shutdown_connections 2024-04-26T21:45:31.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.109+0000 7fde092cbf40 1 -- 172.21.15.40:0/2731189601 wait complete. 2024-04-26T21:45:31.113 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 100.00000000.inode 0 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.195+0000 7efdde63bf40 1 Processor -- start 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.195+0000 7efdde63bf40 1 -- start start 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efdde63bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 0x55f5722f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efdde63bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 0x55f5722dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efdde63bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f5722d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efdde63bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f572267570 con 0x55f5722ddf30 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efdde63bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5722672d0 con 0x55f5722daa80 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efdde63bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f572267af0 con 0x55f5722dae60 2024-04-26T21:45:31.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efddc936640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f5722d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efddc936640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f5722d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41344/0 (socket says 172.21.15.40:41344) 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efddc936640 1 -- 172.21.15.40:0/663442297 learned_addr learned my addr 172.21.15.40:0/663442297 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efddc936640 1 -- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 msgr2=0x55f5722dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.196+0000 7efddb934640 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 0x55f5722dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efddc135640 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 0x55f5722f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efddc936640 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 0x55f5722dd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efddc936640 1 -- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 msgr2=0x55f5722f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efddc936640 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 0x55f5722f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efddc936640 1 -- 172.21.15.40:0/663442297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f572149cb0 con 0x55f5722ddf30 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efddb934640 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 0x55f5722dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efddc135640 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 0x55f5722f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efddc936640 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f5722d4770 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c440 tx=0x7efdcc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e79fb5e9ae66281 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdda932640 1 -- 172.21.15.40:0/663442297 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0116b0 con 0x55f5722ddf30 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdda932640 1 -- 172.21.15.40:0/663442297 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdcc017070 con 0x55f5722ddf30 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdda932640 1 -- 172.21.15.40:0/663442297 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc011ab0 con 0x55f5722ddf30 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 -- 172.21.15.40:0/663442297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 msgr2=0x55f5722d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f5722d4770 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c440 tx=0x7efdcc00c900 comp rx=0 tx=0).stop 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 -- 172.21.15.40:0/663442297 shutdown_connections 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f5722d4770 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 0x55f5722dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 --2- 172.21.15.40:0/663442297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 0x55f5722f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 -- 172.21.15.40:0/663442297 >> 172.21.15.40:0/663442297 conn(0x55f57221dab0 msgr2=0x55f57221e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 -- 172.21.15.40:0/663442297 shutdown_connections 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.197+0000 7efdde63bf40 1 -- 172.21.15.40:0/663442297 wait complete. 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.198+0000 7efdde63bf40 1 Processor -- start 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.198+0000 7efdde63bf40 1 -- start start 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efdde63bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 0x55f5722dd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efdde63bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 0x55f5722f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efdde63bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f57221de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efdde63bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f5722f87e0 con 0x55f5722ddf30 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efdde63bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5722f8ae0 con 0x55f5722daa80 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efdde63bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f57221eed0 con 0x55f5722dae60 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f57221de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f57221de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41348/0 (socket says 172.21.15.40:41348) 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 -- 172.21.15.40:0/3923222000 learned_addr learned my addr 172.21.15.40:0/3923222000 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddb934640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 0x55f5722f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc135640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 0x55f5722dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 -- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 msgr2=0x55f5722f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f5722dae60 0x55f5722f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 -- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 msgr2=0x55f5722dd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5722daa80 0x55f5722dd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 -- 172.21.15.40:0/3923222000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5722d5d80 con 0x55f5722ddf30 2024-04-26T21:45:31.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.199+0000 7efddc936640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f57221de90 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00ce00 tx=0x7efdcc0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.200+0000 7efdd9930640 1 -- 172.21.15.40:0/3923222000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc020070 con 0x55f5722ddf30 2024-04-26T21:45:31.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.200+0000 7efdd9930640 1 -- 172.21.15.40:0/3923222000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdcc012c70 con 0x55f5722ddf30 2024-04-26T21:45:31.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.200+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f57224e050 con 0x55f5722ddf30 2024-04-26T21:45:31.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.200+0000 7efdd9930640 1 -- 172.21.15.40:0/3923222000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0256a0 con 0x55f5722ddf30 2024-04-26T21:45:31.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.200+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5722638a0 con 0x55f5722ddf30 2024-04-26T21:45:31.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.201+0000 7efdd9930640 1 -- 172.21.15.40:0/3923222000 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdcc025840 con 0x55f5722ddf30 2024-04-26T21:45:31.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.202+0000 7efdd9930640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efdbc06b300 0x7efdbc06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.202+0000 7efdd9930640 1 -- 172.21.15.40:0/3923222000 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8797+0+0 (secure 0 0 0) 0x7efdcc08da50 con 0x55f5722ddf30 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.202+0000 7efddc135640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efdbc06b300 0x7efdbc06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.202+0000 7efdde63bf40 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f572264380 0x55f572264780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.202+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f572264cc0 con 0x55f572264380 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.202+0000 7efddb934640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f572264380 0x55f572264780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.202+0000 7efddc135640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efdbc06b300 0x7efdbc06d7c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x55f572268bd0 tx=0x7efdd400d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.203+0000 7efddb934640 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f572264380 0x55f572264780 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efddb934640 1 -- 172.21.15.40:0/3923222000 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 100.00000000.inode [truncate 0] v125'375 uv375 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efdc4054970 con 0x55f572264380 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f572264380 msgr2=0x55f572264780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55f572264380 0x55f572264780 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efdbc06b300 msgr2=0x7efdbc06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efdbc06b300 0x7efdbc06d7c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x55f572268bd0 tx=0x7efdd400d040 comp rx=0 tx=0).stop 2024-04-26T21:45:31.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 msgr2=0x55f57221de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 --2- 172.21.15.40:0/3923222000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f5722ddf30 0x55f57221de90 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00ce00 tx=0x7efdcc0033a0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efddc936640 1 -- 172.21.15.40:0/3923222000 reap_dead start 2024-04-26T21:45:31.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 shutdown_connections 2024-04-26T21:45:31.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 >> 172.21.15.40:0/3923222000 conn(0x55f57221dab0 msgr2=0x55f57224a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:31.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 shutdown_connections 2024-04-26T21:45:31.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.204+0000 7efdde63bf40 1 -- 172.21.15.40:0/3923222000 wait complete. 2024-04-26T21:45:31.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:31.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.352+0000 7fa8a05c6640 1 Processor -- start 2024-04-26T21:45:31.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.352+0000 7fa8a05c6640 1 -- start start 2024-04-26T21:45:31.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.353+0000 7fa8a05c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa898070350 0x7fa898070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.353+0000 7fa8a05c6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8980d2390 con 0x7fa8980cfde0 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.353+0000 7fa8a05c6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89806d3f0 con 0x7fa898071090 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.353+0000 7fa8a05c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa8980afab0 con 0x7fa898070350 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.353+0000 7fa89e362640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa8980cfde0 0x7fa8980d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58314/0 (socket says 172.21.15.40:58314) 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.353+0000 7fa89e362640 1 -- 172.21.15.40:0/1970437836 learned_addr learned my addr 172.21.15.40:0/1970437836 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.353+0000 7fa89db61640 1 --2- 172.21.15.40:0/1970437836 >> v2:172.21.15.184:3300/0 conn(0x7fa898070350 0x7fa898070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.354+0000 7fa89d360640 1 -- 172.21.15.40:0/1970437836 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591482914 0 0) 0x7fa8980d2390 con 0x7fa8980cfde0 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.354+0000 7fa89d360640 1 -- 172.21.15.40:0/1970437836 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa884003520 con 0x7fa8980cfde0 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.354+0000 7fa89db61640 1 -- 172.21.15.40:0/1970437836 >> v1:172.21.15.184:6789/0 conn(0x7fa898071090 legacy=0x7fa89815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.354+0000 7fa89db61640 1 -- 172.21.15.40:0/1970437836 >> v1:172.21.15.40:6789/0 conn(0x7fa8980cfde0 legacy=0x7fa8980d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.354+0000 7fa89d360640 1 -- 172.21.15.40:0/1970437836 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1378332814 0 0) 0x7fa89806d3f0 con 0x7fa898071090 2024-04-26T21:45:31.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.354+0000 7fa89db61640 1 -- 172.21.15.40:0/1970437836 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89806d6d0 con 0x7fa898070350 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.354+0000 7fa89db61640 1 --2- 172.21.15.40:0/1970437836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898070350 0x7fa898070b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa8880097a0 tx=0x7fa88802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d678e5c9dc92e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa89d360640 1 -- 172.21.15.40:0/1970437836 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888037070 con 0x7fa898070350 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa89d360640 1 -- 172.21.15.40:0/1970437836 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa888032440 con 0x7fa898070350 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa89d360640 1 -- 172.21.15.40:0/1970437836 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888032810 con 0x7fa898070350 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1970437836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898070350 msgr2=0x7fa898070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa8a05c6640 1 --2- 172.21.15.40:0/1970437836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898070350 0x7fa898070b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa8880097a0 tx=0x7fa88802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1970437836 shutdown_connections 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa8a05c6640 1 --2- 172.21.15.40:0/1970437836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898070350 0x7fa898070b50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1970437836 >> 172.21.15.40:0/1970437836 conn(0x7fa8980d2d80 msgr2=0x7fa8980d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1970437836 shutdown_connections 2024-04-26T21:45:31.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.355+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1970437836 wait complete. 2024-04-26T21:45:31.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.356+0000 7fa8a05c6640 1 Processor -- start 2024-04-26T21:45:31.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.356+0000 7fa8a05c6640 1 -- start start 2024-04-26T21:45:31.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa8a05c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa898070350 0x7fa8980b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa8a05c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898071090 0x7fa8980b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa8a05c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8980cfde0 0x7fa8980b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa8a05c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8980714c0 con 0x7fa898070350 2024-04-26T21:45:31.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa8a05c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa89806dd20 con 0x7fa898071090 2024-04-26T21:45:31.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa8a05c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa89806f1c0 con 0x7fa8980cfde0 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa89e362640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa898070350 0x7fa8980b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa89e362640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa898070350 0x7fa8980b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41356/0 (socket says 172.21.15.40:41356) 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa89e362640 1 -- 172.21.15.40:0/1340900598 learned_addr learned my addr 172.21.15.40:0/1340900598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa89db61640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898071090 0x7fa8980b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.357+0000 7fa89eb63640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8980cfde0 0x7fa8980b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa89e362640 1 -- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8980cfde0 msgr2=0x7fa8980b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa89e362640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8980cfde0 0x7fa8980b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa89e362640 1 -- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898071090 msgr2=0x7fa8980b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa89e362640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898071090 0x7fa8980b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa89e362640 1 -- 172.21.15.40:0/1340900598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa898157c00 con 0x7fa898070350 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa89e362640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa898070350 0x7fa8980b4850 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c390 tx=0x7fa89000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa88effd640 1 -- 172.21.15.40:0/1340900598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890017070 con 0x7fa898070350 2024-04-26T21:45:31.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa88effd640 1 -- 172.21.15.40:0/1340900598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa890005ce0 con 0x7fa898070350 2024-04-26T21:45:31.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa88effd640 1 -- 172.21.15.40:0/1340900598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900072f0 con 0x7fa898070350 2024-04-26T21:45:31.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8980b4d90 con 0x7fa898070350 2024-04-26T21:45:31.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.358+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8981f8640 con 0x7fa898070350 2024-04-26T21:45:31.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.359+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8980694f0 con 0x7fa898070350 2024-04-26T21:45:31.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.359+0000 7fa88effd640 1 -- 172.21.15.40:0/1340900598 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa89001a070 con 0x7fa898070350 2024-04-26T21:45:31.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.360+0000 7fa88effd640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa87c06c8f0 0x7fa87c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:31.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.360+0000 7fa88effd640 1 -- 172.21.15.40:0/1340900598 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8797+0+0 (secure 0 0 0) 0x7fa89008bb10 con 0x7fa898070350 2024-04-26T21:45:31.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.360+0000 7fa89db61640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa87c06c8f0 0x7fa87c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:31.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.360+0000 7fa89db61640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa87c06c8f0 0x7fa87c06edb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa888034000 tx=0x7fa88802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:31.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.364+0000 7fa88effd640 1 -- 172.21.15.40:0/1340900598 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa890058ca0 con 0x7fa898070350 2024-04-26T21:45:31.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.587+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa89806ce60 con 0x7fa898070350 2024-04-26T21:45:31.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.896+0000 7fa88effd640 1 -- 172.21.15.40:0/1340900598 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v278) ==== 162+0+0 (secure 0 0 0) 0x7fa89005cfa0 con 0x7fa898070350 2024-04-26T21:45:31.897 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:31.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa87c06c8f0 msgr2=0x7fa87c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa87c06c8f0 0x7fa87c06edb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa888034000 tx=0x7fa88802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:31.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa898070350 msgr2=0x7fa8980b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:31.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa898070350 0x7fa8980b4850 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c390 tx=0x7fa89000c850 comp rx=0 tx=0).stop 2024-04-26T21:45:31.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 shutdown_connections 2024-04-26T21:45:31.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa87c06c8f0 0x7fa87c06edb0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8980cfde0 0x7fa8980b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa898071090 0x7fa8980b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 --2- 172.21.15.40:0/1340900598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa898070350 0x7fa8980b4850 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 >> 172.21.15.40:0/1340900598 conn(0x7fa8980d2d80 msgr2=0x7fa8980d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 shutdown_connections 2024-04-26T21:45:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:31.898+0000 7fa8a05c6640 1 -- 172.21.15.40:0/1340900598 wait complete. 2024-04-26T21:45:31.906 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:31.906+0000 7f47200fe640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-04-26T21:45:31.907 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:31.906+0000 7f47200fe640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-26T21:45:31.907 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:31.906+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:45:31.907 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:31.906+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:45:31.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:31.913 INFO:tasks.ceph.mds.c.smithi040.stderr: -24> 2024-04-26T21:45:31.906+0000 7f47200fe640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-04-26T21:45:31.914 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:45:31.906+0000 7f47200fe640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-26T21:45:31.938 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:45:32.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.017+0000 7fa1aba62640 1 Processor -- start 2024-04-26T21:45:32.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.017+0000 7fa1aba62640 1 -- start start 2024-04-26T21:45:32.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.018+0000 7fa1aba62640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4071e60 0x7fa1a415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.018+0000 7fa1aba62640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1a4072240 0x7fa1a40726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.018+0000 7fa1aba62640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a415e4d0 con 0x7fa1a40cb5a0 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.018+0000 7fa1aba62640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a40701f0 con 0x7fa1a4072240 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.018+0000 7fa1aba62640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a406e240 con 0x7fa1a4071e60 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.018+0000 7fa1a97fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4071e60 0x7fa1a415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.019+0000 7fa1a97fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4071e60 0x7fa1a415df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:37594/0 (socket says 172.21.15.40:37594) 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.019+0000 7fa1a8ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1a4072240 0x7fa1a40726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.019+0000 7fa1a97fe640 1 -- 172.21.15.40:0/2333035297 learned_addr learned my addr 172.21.15.40:0/2333035297 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.019+0000 7fa1a97fe640 1 -- 172.21.15.40:0/2333035297 >> v2:172.21.15.184:3300/0 conn(0x7fa1a4072240 msgr2=0x7fa1a40726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.019+0000 7fa1a97fe640 1 --2- 172.21.15.40:0/2333035297 >> v2:172.21.15.184:3300/0 conn(0x7fa1a4072240 0x7fa1a40726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.019+0000 7fa1a97fe640 1 -- 172.21.15.40:0/2333035297 >> v1:172.21.15.184:6789/0 conn(0x7fa1a40cb5a0 legacy=0x7fa1a40cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.019+0000 7fa1a97fe640 1 -- 172.21.15.40:0/2333035297 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a406fc30 con 0x7fa1a4071e60 2024-04-26T21:45:32.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1a97fe640 1 --2- 172.21.15.40:0/2333035297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4071e60 0x7fa1a415df90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009870 tx=0x7fa1a002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da1b3f12fd4b37c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa19bfff640 1 -- 172.21.15.40:0/2333035297 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0037070 con 0x7fa1a4071e60 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa19bfff640 1 -- 172.21.15.40:0/2333035297 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a0032440 con 0x7fa1a4071e60 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa19bfff640 1 -- 172.21.15.40:0/2333035297 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0032810 con 0x7fa1a4071e60 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1aba62640 1 -- 172.21.15.40:0/2333035297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4071e60 msgr2=0x7fa1a415df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1aba62640 1 --2- 172.21.15.40:0/2333035297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4071e60 0x7fa1a415df90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009870 tx=0x7fa1a002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1aba62640 1 -- 172.21.15.40:0/2333035297 shutdown_connections 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1aba62640 1 --2- 172.21.15.40:0/2333035297 >> v2:172.21.15.184:3300/0 conn(0x7fa1a4072240 0x7fa1a40726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1aba62640 1 --2- 172.21.15.40:0/2333035297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4071e60 0x7fa1a415df90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1aba62640 1 -- 172.21.15.40:0/2333035297 >> 172.21.15.40:0/2333035297 conn(0x7fa1a40d0be0 msgr2=0x7fa1a40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1aba62640 1 -- 172.21.15.40:0/2333035297 shutdown_connections 2024-04-26T21:45:32.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.020+0000 7fa1aba62640 1 -- 172.21.15.40:0/2333035297 wait complete. 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1aba62640 1 Processor -- start 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1aba62640 1 -- start start 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1aba62640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a4071e60 0x7fa1a4149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1aba62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1a4072240 0x7fa1a40c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1aba62640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a40cb5a0 0x7fa1a40add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1aba62640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4072be0 con 0x7fa1a4072240 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1aba62640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4073fb0 con 0x7fa1a4071e60 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1aba62640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a406f8e0 con 0x7fa1a40cb5a0 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1a8ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1a4072240 0x7fa1a40c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1a9fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a40cb5a0 0x7fa1a40add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1a8ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1a4072240 0x7fa1a40c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41390/0 (socket says 172.21.15.40:41390) 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.022+0000 7fa1a97fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a4071e60 0x7fa1a4149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1a8ffd640 1 -- 172.21.15.40:0/1655407943 learned_addr learned my addr 172.21.15.40:0/1655407943 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1a8ffd640 1 -- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a40cb5a0 msgr2=0x7fa1a40add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1a8ffd640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a40cb5a0 0x7fa1a40add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1a8ffd640 1 -- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a4071e60 msgr2=0x7fa1a4149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1a8ffd640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a4071e60 0x7fa1a4149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1a8ffd640 1 -- 172.21.15.40:0/1655407943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40c9bd0 con 0x7fa1a4072240 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1a97fe640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a4071e60 0x7fa1a4149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1a8ffd640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1a4072240 0x7fa1a40c9180 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fa19400e9b0 tx=0x7fa19400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa199ffb640 1 -- 172.21.15.40:0/1655407943 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194017070 con 0x7fa1a4072240 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa199ffb640 1 -- 172.21.15.40:0/1655407943 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19400c9a0 con 0x7fa1a4072240 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a40c9eb0 con 0x7fa1a4072240 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa199ffb640 1 -- 172.21.15.40:0/1655407943 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19400cd90 con 0x7fa1a4072240 2024-04-26T21:45:32.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.023+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a40ca470 con 0x7fa1a4072240 2024-04-26T21:45:32.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.024+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a40694f0 con 0x7fa1a4072240 2024-04-26T21:45:32.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.025+0000 7fa199ffb640 1 -- 172.21.15.40:0/1655407943 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa19401a070 con 0x7fa1a4072240 2024-04-26T21:45:32.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.025+0000 7fa199ffb640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa18806c8f0 0x7fa18806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.025+0000 7fa199ffb640 1 -- 172.21.15.40:0/1655407943 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fa19408caf0 con 0x7fa1a4072240 2024-04-26T21:45:32.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.026+0000 7fa1a97fe640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa18806c8f0 0x7fa18806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.026+0000 7fa1a97fe640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa18806c8f0 0x7fa18806edb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0034000 tx=0x7fa1a002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.030+0000 7fa199ffb640 1 -- 172.21.15.40:0/1655407943 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa194059c10 con 0x7fa1a4072240 2024-04-26T21:45:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.252+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1a415dff0 con 0x7fa1a4072240 2024-04-26T21:45:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.252+0000 7fa199ffb640 1 -- 172.21.15.40:0/1655407943 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 280 v280) ==== 95+0+4355 (secure 0 0 0) 0x7fa19405df10 con 0x7fa1a4072240 2024-04-26T21:45:32.253 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:32.253 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":280,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":280,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:31.908115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 280 2024-04-26T21:45:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa18806c8f0 msgr2=0x7fa18806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa18806c8f0 0x7fa18806edb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0034000 tx=0x7fa1a002e000 comp rx=0 tx=0).stop 2024-04-26T21:45:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1a4072240 msgr2=0x7fa1a40c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1a4072240 0x7fa1a40c9180 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fa19400e9b0 tx=0x7fa19400ee70 comp rx=0 tx=0).stop 2024-04-26T21:45:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 shutdown_connections 2024-04-26T21:45:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa18806c8f0 0x7fa18806edb0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a40cb5a0 0x7fa1a40add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1a4072240 0x7fa1a40c9180 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 --2- 172.21.15.40:0/1655407943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a4071e60 0x7fa1a4149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 >> 172.21.15.40:0/1655407943 conn(0x7fa1a40d0be0 msgr2=0x7fa1a415a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 shutdown_connections 2024-04-26T21:45:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.254+0000 7fa1aba62640 1 -- 172.21.15.40:0/1655407943 wait complete. 2024-04-26T21:45:32.266 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:32.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:32.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.407+0000 7fe335864640 1 Processor -- start 2024-04-26T21:45:32.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.407+0000 7fe335864640 1 -- start start 2024-04-26T21:45:32.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.407+0000 7fe335864640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3300d06f0 0x7fe3300d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.407+0000 7fe335864640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3300d0ba0 con 0x7fe3300b2850 2024-04-26T21:45:32.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe335864640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3300d0e20 con 0x7fe33015f3a0 2024-04-26T21:45:32.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe335864640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe3300b1d70 con 0x7fe3300d06f0 2024-04-26T21:45:32.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe32e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3300d06f0 0x7fe3300d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe32f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe3300b2850 0x7fe3300b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58336/0 (socket says 172.21.15.40:58336) 2024-04-26T21:45:32.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe32e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3300d06f0 0x7fe3300d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41398/0 (socket says 172.21.15.40:41398) 2024-04-26T21:45:32.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe32f7fe640 1 -- 172.21.15.40:0/1782556086 learned_addr learned my addr 172.21.15.40:0/1782556086 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:32.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe32e7fc640 1 -- 172.21.15.40:0/1782556086 >> v1:172.21.15.184:6789/0 conn(0x7fe33015f3a0 legacy=0x7fe3300d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:32.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe32e7fc640 1 -- 172.21.15.40:0/1782556086 >> v1:172.21.15.40:6789/0 conn(0x7fe3300b2850 legacy=0x7fe3300b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe32dffb640 1 -- 172.21.15.40:0/1782556086 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204559659 0 0) 0x7fe328001490 con 0x7fe3300b2850 2024-04-26T21:45:32.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.408+0000 7fe32e7fc640 1 -- 172.21.15.40:0/1782556086 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3300b2050 con 0x7fe3300d06f0 2024-04-26T21:45:32.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe32e7fc640 1 --2- 172.21.15.40:0/1782556086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3300d06f0 0x7fe3300d1e80 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fe3240097a0 tx=0x7fe32402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24709c8328a4ef8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe32dffb640 1 -- 172.21.15.40:0/1782556086 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324037070 con 0x7fe3300d06f0 2024-04-26T21:45:32.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe32dffb640 1 -- 172.21.15.40:0/1782556086 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe324032440 con 0x7fe3300d06f0 2024-04-26T21:45:32.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe32dffb640 1 -- 172.21.15.40:0/1782556086 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324032810 con 0x7fe3300d06f0 2024-04-26T21:45:32.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe335864640 1 -- 172.21.15.40:0/1782556086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3300d06f0 msgr2=0x7fe3300d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe335864640 1 --2- 172.21.15.40:0/1782556086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3300d06f0 0x7fe3300d1e80 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fe3240097a0 tx=0x7fe32402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe335864640 1 -- 172.21.15.40:0/1782556086 shutdown_connections 2024-04-26T21:45:32.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe335864640 1 --2- 172.21.15.40:0/1782556086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3300d06f0 0x7fe3300d1e80 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe335864640 1 -- 172.21.15.40:0/1782556086 >> 172.21.15.40:0/1782556086 conn(0x7fe3300d4f90 msgr2=0x7fe3300d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:32.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe335864640 1 -- 172.21.15.40:0/1782556086 shutdown_connections 2024-04-26T21:45:32.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.409+0000 7fe335864640 1 -- 172.21.15.40:0/1782556086 wait complete. 2024-04-26T21:45:32.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.411+0000 7fe335864640 1 Processor -- start 2024-04-26T21:45:32.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.411+0000 7fe335864640 1 -- start start 2024-04-26T21:45:32.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.411+0000 7fe335864640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3300b2850 0x7fe3300ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.411+0000 7fe335864640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3300d06f0 0x7fe3300cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.411+0000 7fe335864640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe33015f3a0 0x7fe3300b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.411+0000 7fe335864640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe33015f900 con 0x7fe33015f3a0 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.411+0000 7fe335864640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe33015f780 con 0x7fe3300d06f0 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.411+0000 7fe335864640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe3300d23c0 con 0x7fe3300b2850 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe33015f3a0 0x7fe3300b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe33015f3a0 0x7fe3300b8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41406/0 (socket says 172.21.15.40:41406) 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 -- 172.21.15.40:0/770232651 learned_addr learned my addr 172.21.15.40:0/770232651 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32effd640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3300b2850 0x7fe3300ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32e7fc640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3300d06f0 0x7fe3300cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 -- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3300b2850 msgr2=0x7fe3300ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3300b2850 0x7fe3300ba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 -- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3300d06f0 msgr2=0x7fe3300cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3300d06f0 0x7fe3300cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 -- 172.21.15.40:0/770232651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3300baa30 con 0x7fe33015f3a0 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32effd640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3300b2850 0x7fe3300ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32e7fc640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3300d06f0 0x7fe3300cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe32f7fe640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe33015f3a0 0x7fe3300b8700 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fe32800c7f0 tx=0x7fe32800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe317fff640 1 -- 172.21.15.40:0/770232651 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328018070 con 0x7fe33015f3a0 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe317fff640 1 -- 172.21.15.40:0/770232651 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32800a5e0 con 0x7fe33015f3a0 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3300cc8b0 con 0x7fe33015f3a0 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe317fff640 1 -- 172.21.15.40:0/770232651 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32800aa10 con 0x7fe33015f3a0 2024-04-26T21:45:32.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.412+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3300b7f00 con 0x7fe33015f3a0 2024-04-26T21:45:32.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.413+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3300694f0 con 0x7fe33015f3a0 2024-04-26T21:45:32.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.414+0000 7fe317fff640 1 -- 172.21.15.40:0/770232651 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe32801b070 con 0x7fe33015f3a0 2024-04-26T21:45:32.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.414+0000 7fe317fff640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe31c06c8f0 0x7fe31c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.414+0000 7fe317fff640 1 -- 172.21.15.40:0/770232651 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fe32808bd50 con 0x7fe33015f3a0 2024-04-26T21:45:32.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.415+0000 7fe32effd640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe31c06c8f0 0x7fe31c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.415+0000 7fe32effd640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe31c06c8f0 0x7fe31c06edb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe32000a830 tx=0x7fe320008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.419+0000 7fe317fff640 1 -- 172.21.15.40:0/770232651 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe32808f030 con 0x7fe33015f3a0 2024-04-26T21:45:32.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.646+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3300bacb0 con 0x7fe33015f3a0 2024-04-26T21:45:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.646+0000 7fe317fff640 1 -- 172.21.15.40:0/770232651 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 280 v280) ==== 95+0+4355 (secure 0 0 0) 0x7fe3300d0ba0 con 0x7fe33015f3a0 2024-04-26T21:45:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:32.647 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":280,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":280,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:31.908115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 280 2024-04-26T21:45:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe31c06c8f0 msgr2=0x7fe31c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe31c06c8f0 0x7fe31c06edb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe32000a830 tx=0x7fe320008040 comp rx=0 tx=0).stop 2024-04-26T21:45:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe33015f3a0 msgr2=0x7fe3300b8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe33015f3a0 0x7fe3300b8700 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fe32800c7f0 tx=0x7fe32800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 shutdown_connections 2024-04-26T21:45:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe31c06c8f0 0x7fe31c06edb0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe33015f3a0 0x7fe3300b8700 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3300d06f0 0x7fe3300cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 --2- 172.21.15.40:0/770232651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3300b2850 0x7fe3300ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 >> 172.21.15.40:0/770232651 conn(0x7fe3300d4f90 msgr2=0x7fe3300b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 shutdown_connections 2024-04-26T21:45:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.648+0000 7fe335864640 1 -- 172.21.15.40:0/770232651 wait complete. 2024-04-26T21:45:32.660 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:32.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:32.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.759+0000 7f6118fda640 1 Processor -- start 2024-04-26T21:45:32.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.759+0000 7f6118fda640 1 -- start start 2024-04-26T21:45:32.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.760+0000 7f6118fda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61140b3860 0x7f611416a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.760+0000 7f6118fda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61140d0ba0 con 0x7f61140d06f0 2024-04-26T21:45:32.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.760+0000 7f6118fda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61140d2590 con 0x7f611415f3a0 2024-04-26T21:45:32.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.760+0000 7f6118fda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61140d0e20 con 0x7f61140b3860 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.760+0000 7f6111d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f61140d06f0 0x7f61140d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58346/0 (socket says 172.21.15.40:58346) 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.760+0000 7f6111d9b640 1 -- 172.21.15.40:0/1099935514 learned_addr learned my addr 172.21.15.40:0/1099935514 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710353468 0 0) 0x7f60fc001110 con 0x7f61140d06f0 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6100003520 con 0x7f61140d06f0 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f6112d9d640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61140b3860 0x7f611416a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 229547015 0 0) 0x7f6100003520 con 0x7f61140d06f0 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61140b3860 msgr2=0x7f611416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61140b3860 0x7f611416a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 >> v1:172.21.15.184:6789/0 conn(0x7f611415f3a0 legacy=0x7f61140d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61140b1d70 con 0x7f61140d06f0 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f60fc001840 con 0x7f61140d06f0 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 >> v1:172.21.15.40:6789/0 conn(0x7f61140d06f0 legacy=0x7f61140d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61000039e0 0x7f6100003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6100004800 0x7f6100004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61140b3860 0x7f61000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61140d0e20 con 0x7f61000039e0 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f610000ba70 con 0x7f6100004800 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f610000bc70 con 0x7f61140b3860 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611259c640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61000039e0 0x7f6100003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f6111d9b640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6100004800 0x7f6100004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f6112d9d640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61140b3860 0x7f61000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611259c640 1 -- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61140b3860 msgr2=0x7f61000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611259c640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61140b3860 0x7f61000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611259c640 1 -- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6100004800 msgr2=0x7f6100004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.761+0000 7f611259c640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6100004800 0x7f6100004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.762+0000 7f611259c640 1 -- 172.21.15.40:0/1099935514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6100003520 con 0x7f61000039e0 2024-04-26T21:45:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.762+0000 7f611259c640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61000039e0 0x7f6100003e00 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f610c001720 tx=0x7f6100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbf255bff62431a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.762+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c001370 con 0x7f61000039e0 2024-04-26T21:45:32.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.762+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f610c0052d0 con 0x7f61000039e0 2024-04-26T21:45:32.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.762+0000 7f611159a640 1 -- 172.21.15.40:0/1099935514 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c02ab90 con 0x7f61000039e0 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.763+0000 7f6118fda640 1 -- 172.21.15.40:0/1099935514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61000039e0 msgr2=0x7f6100003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.763+0000 7f6118fda640 1 --2- 172.21.15.40:0/1099935514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61000039e0 0x7f6100003e00 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f610c001720 tx=0x7f6100005830 comp rx=0 tx=0).stop 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.763+0000 7f6112d9d640 1 -- 172.21.15.40:0/1099935514 reap_dead start 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.763+0000 7f6118fda640 1 -- 172.21.15.40:0/1099935514 shutdown_connections 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.763+0000 7f6118fda640 1 -- 172.21.15.40:0/1099935514 >> 172.21.15.40:0/1099935514 conn(0x7f61140d4f90 msgr2=0x7f61140d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.763+0000 7f6118fda640 1 -- 172.21.15.40:0/1099935514 shutdown_connections 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.763+0000 7f6118fda640 1 -- 172.21.15.40:0/1099935514 wait complete. 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.764+0000 7f6118fda640 1 Processor -- start 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6118fda640 1 -- start start 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6118fda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000039e0 0x7f611414bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6118fda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61140c9a90 0x7f61140b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6118fda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61140c9f10 0x7f61140b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6118fda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f611415f780 con 0x7f61140c9a90 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6118fda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61140b2330 con 0x7f61140c9f10 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6118fda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f611415f900 con 0x7f61000039e0 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6111d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61140c9a90 0x7f61140b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6111d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61140c9a90 0x7f61140b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41424/0 (socket says 172.21.15.40:41424) 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6111d9b640 1 -- 172.21.15.40:0/2575563685 learned_addr learned my addr 172.21.15.40:0/2575563685 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f6112d9d640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61140c9f10 0x7f61140b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.765+0000 7f611259c640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000039e0 0x7f611414bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f6111d9b640 1 -- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000039e0 msgr2=0x7f611414bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f6111d9b640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000039e0 0x7f611414bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f6111d9b640 1 -- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61140c9f10 msgr2=0x7f61140b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f6111d9b640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61140c9f10 0x7f61140b80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f6111d9b640 1 -- 172.21.15.40:0/2575563685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61140ca450 con 0x7f61140c9a90 2024-04-26T21:45:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f6111d9b640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61140c9a90 0x7f61140b7b80 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f60fc007310 tx=0x7f60fc0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f610affd640 1 -- 172.21.15.40:0/2575563685 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc01a070 con 0x7f61140c9a90 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f610affd640 1 -- 172.21.15.40:0/2575563685 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc001230 con 0x7f61140c9a90 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61140ca730 con 0x7f61140c9a90 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f610affd640 1 -- 172.21.15.40:0/2575563685 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc00fb80 con 0x7f61140c9a90 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.766+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61140cacf0 con 0x7f61140c9a90 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.767+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61140694f0 con 0x7f61140c9a90 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.768+0000 7f610affd640 1 -- 172.21.15.40:0/2575563685 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60fc01d070 con 0x7f61140c9a90 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.768+0000 7f610affd640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f406c920 0x7f60f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.768+0000 7f610affd640 1 -- 172.21.15.40:0/2575563685 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f60fc08c140 con 0x7f61140c9a90 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.768+0000 7f611259c640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f406c920 0x7f60f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.769+0000 7f611259c640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f406c920 0x7f60f406ede0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f610c032000 tx=0x7f610c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.772+0000 7f610affd640 1 -- 172.21.15.40:0/2575563685 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61140d0ba0 con 0x7f61140c9a90 2024-04-26T21:45:32.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.997+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61140b6860 con 0x7f61140c9a90 2024-04-26T21:45:32.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:32.998+0000 7f610affd640 1 -- 172.21.15.40:0/2575563685 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+5182 (secure 0 0 0) 0x7f60fc0598b0 con 0x7f61140c9a90 2024-04-26T21:45:32.999 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:32.999 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":281,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4214786907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4214786907},{"type":"v1","addr":"172.21.15.40:6833","nonce":4214786907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":281}],"filesystems":[{"mdsmap":{"epoch":280,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:31.908115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:32.999 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 281 2024-04-26T21:45:33.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f406c920 msgr2=0x7f60f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f406c920 0x7f60f406ede0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f610c032000 tx=0x7f610c02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:33.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61140c9a90 msgr2=0x7f61140b7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61140c9a90 0x7f61140b7b80 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f60fc007310 tx=0x7f60fc0077d0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 shutdown_connections 2024-04-26T21:45:33.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f60f406c920 0x7f60f406ede0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61140c9f10 0x7f61140b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61140c9a90 0x7f61140b7b80 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 --2- 172.21.15.40:0/2575563685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000039e0 0x7f611414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 >> 172.21.15.40:0/2575563685 conn(0x7f61140d4f90 msgr2=0x7f61140b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 shutdown_connections 2024-04-26T21:45:33.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.000+0000 7f6118fda640 1 -- 172.21.15.40:0/2575563685 wait complete. 2024-04-26T21:45:33.011 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 100.00000000.inode' led to DAMAGED state 2024-04-26T21:45:33.011 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head' 2024-04-26T21:45:33.012 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:33.012 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:33.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:45:33.027 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:33.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:33.027 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:33.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:45:33.083 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:45:33.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:33.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.224+0000 7f3fc7522640 1 Processor -- start 2024-04-26T21:45:33.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.225+0000 7f3fc7522640 1 -- start start 2024-04-26T21:45:33.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.225+0000 7f3fc7522640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3fc00b3860 0x7f3fc016a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.225+0000 7f3fc7522640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fc00d0ba0 con 0x7f3fc00d06f0 2024-04-26T21:45:33.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.225+0000 7f3fc7522640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fc00d2590 con 0x7f3fc015f3a0 2024-04-26T21:45:33.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.225+0000 7f3fc7522640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc00d0e20 con 0x7f3fc00b3860 2024-04-26T21:45:33.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.225+0000 7f3fc4abd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3fc00d06f0 0x7f3fc00d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58354/0 (socket says 172.21.15.40:58354) 2024-04-26T21:45:33.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.225+0000 7f3fc4abd640 1 -- 172.21.15.40:0/2205226650 learned_addr learned my addr 172.21.15.40:0/2205226650 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472399961 0 0) 0x7f3fb4001110 con 0x7f3fc00d06f0 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc5abf640 1 --2- 172.21.15.40:0/2205226650 >> v2:172.21.15.184:3300/0 conn(0x7f3fc00b3860 0x7f3fc016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fa8003520 con 0x7f3fc00d06f0 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3060652364 0 0) 0x7f3fa8003520 con 0x7f3fc00d06f0 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 >> v2:172.21.15.184:3300/0 conn(0x7f3fc00b3860 msgr2=0x7f3fc016a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 --2- 172.21.15.40:0/2205226650 >> v2:172.21.15.184:3300/0 conn(0x7f3fc00b3860 0x7f3fc016a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 >> v1:172.21.15.184:6789/0 conn(0x7f3fc015f3a0 legacy=0x7f3fc00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc00b1d70 con 0x7f3fc00d06f0 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f3fb4001840 con 0x7f3fc00d06f0 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 >> v1:172.21.15.40:6789/0 conn(0x7f3fc00d06f0 legacy=0x7f3fc00d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fa80039b0 0x7f3fa8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa8004800 0x7f3fa8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa80051c0 0x7f3fa800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc00d0e20 con 0x7f3fa8004800 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa800c160 con 0x7f3fa80039b0 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa800c360 con 0x7f3fa80051c0 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc4abd640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa8004800 0x7f3fa8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc52be640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fa80039b0 0x7f3fa8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc5abf640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa80051c0 0x7f3fa800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc4abd640 1 -- 172.21.15.40:0/2205226650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa80051c0 msgr2=0x7f3fa800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc4abd640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa80051c0 0x7f3fa800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc4abd640 1 -- 172.21.15.40:0/2205226650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fa80039b0 msgr2=0x7f3fa8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc4abd640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fa80039b0 0x7f3fa8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc4abd640 1 -- 172.21.15.40:0/2205226650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa8003520 con 0x7f3fa8004800 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.226+0000 7f3fc5abf640 1 -- 172.21.15.40:0/2205226650 reap_dead start 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.227+0000 7f3fc4abd640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa8004800 0x7f3fa8004c80 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400c4c0 tx=0x7f3fa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1620f98cc53ba21c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.227+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400c0e0 con 0x7f3fa8004800 2024-04-26T21:45:33.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.227+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb4034070 con 0x7f3fa8004800 2024-04-26T21:45:33.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.227+0000 7f3fbffff640 1 -- 172.21.15.40:0/2205226650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4005d70 con 0x7f3fa8004800 2024-04-26T21:45:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.228+0000 7f3fc7522640 1 -- 172.21.15.40:0/2205226650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa8004800 msgr2=0x7f3fa8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.228+0000 7f3fc7522640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa8004800 0x7f3fa8004c80 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400c4c0 tx=0x7f3fa8005830 comp rx=0 tx=0).stop 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.228+0000 7f3fc7522640 1 -- 172.21.15.40:0/2205226650 shutdown_connections 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.228+0000 7f3fc7522640 1 --2- 172.21.15.40:0/2205226650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa8004800 0x7f3fa8004c80 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.228+0000 7f3fc7522640 1 -- 172.21.15.40:0/2205226650 >> 172.21.15.40:0/2205226650 conn(0x7f3fc00d4f90 msgr2=0x7f3fc00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.228+0000 7f3fc7522640 1 -- 172.21.15.40:0/2205226650 shutdown_connections 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.228+0000 7f3fc7522640 1 -- 172.21.15.40:0/2205226650 wait complete. 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.229+0000 7f3fc7522640 1 Processor -- start 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.229+0000 7f3fc7522640 1 -- start start 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc7522640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa8004800 0x7f3fc015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc7522640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc00cca30 0x7f3fc00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc7522640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc0160c50 0x7f3fc01610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc7522640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc015f780 con 0x7f3fc0160c50 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc7522640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc00b2330 con 0x7f3fc00cca30 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc7522640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc015f900 con 0x7f3fa8004800 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc0160c50 0x7f3fc01610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc0160c50 0x7f3fc01610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41428/0 (socket says 172.21.15.40:41428) 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 -- 172.21.15.40:0/1734843076 learned_addr learned my addr 172.21.15.40:0/1734843076 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc52be640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa8004800 0x7f3fc015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc4abd640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc00cca30 0x7f3fc00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 -- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa8004800 msgr2=0x7f3fc015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa8004800 0x7f3fc015c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 -- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc00cca30 msgr2=0x7f3fc00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc00cca30 0x7f3fc00cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 -- 172.21.15.40:0/1734843076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc015cc30 con 0x7f3fc0160c50 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc52be640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa8004800 0x7f3fc015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.230+0000 7f3fc5abf640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc0160c50 0x7f3fc01610d0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f3fc008e8d0 tx=0x7f3fb00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.231+0000 7f3fbdffb640 1 -- 172.21.15.40:0/1734843076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0011db0 con 0x7f3fc0160c50 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.231+0000 7f3fbdffb640 1 -- 172.21.15.40:0/1734843076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb001b070 con 0x7f3fc0160c50 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.231+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc015fbb0 con 0x7f3fc0160c50 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.231+0000 7f3fbdffb640 1 -- 172.21.15.40:0/1734843076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0003990 con 0x7f3fc0160c50 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.231+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc0160170 con 0x7f3fc0160c50 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.232+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc00694f0 con 0x7f3fc0160c50 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.232+0000 7f3fbdffb640 1 -- 172.21.15.40:0/1734843076 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fb0003b30 con 0x7f3fc0160c50 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.233+0000 7f3fbdffb640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fac06c920 0x7f3fac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.233+0000 7f3fbdffb640 1 -- 172.21.15.40:0/1734843076 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f3fb008d0b0 con 0x7f3fc0160c50 2024-04-26T21:45:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.233+0000 7f3fc52be640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fac06c920 0x7f3fac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.233+0000 7f3fc52be640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fac06c920 0x7f3fac06ede0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000c00 tx=0x7f3fb8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.237+0000 7f3fbdffb640 1 -- 172.21.15.40:0/1734843076 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3fb0003e20 con 0x7f3fc0160c50 2024-04-26T21:45:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.459+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3fc00cf4e0 con 0x7f3fc0160c50 2024-04-26T21:45:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.919+0000 7f3fbdffb640 1 -- 172.21.15.40:0/1734843076 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v282) ==== 161+0+0 (secure 0 0 0) 0x7f3fb005a850 con 0x7f3fc0160c50 2024-04-26T21:45:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fac06c920 msgr2=0x7f3fac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fac06c920 0x7f3fac06ede0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000c00 tx=0x7f3fb8009040 comp rx=0 tx=0).stop 2024-04-26T21:45:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc0160c50 msgr2=0x7f3fc01610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc0160c50 0x7f3fc01610d0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f3fc008e8d0 tx=0x7f3fb00115a0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 shutdown_connections 2024-04-26T21:45:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3fac06c920 0x7f3fac06ede0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc0160c50 0x7f3fc01610d0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc00cca30 0x7f3fc00cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 --2- 172.21.15.40:0/1734843076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fa8004800 0x7f3fc015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 >> 172.21.15.40:0/1734843076 conn(0x7f3fc00d4f90 msgr2=0x7f3fc00d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 shutdown_connections 2024-04-26T21:45:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:33.921+0000 7f3fc7522640 1 -- 172.21.15.40:0/1734843076 wait complete. 2024-04-26T21:45:33.934 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:34.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.036+0000 7fb9274b8640 1 Processor -- start 2024-04-26T21:45:34.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.037+0000 7fb9274b8640 1 -- start start 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.037+0000 7fb9274b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb92015f3a0 0x7fb9200d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.037+0000 7fb9274b8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb9200d1770 0x7fb9200d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.037+0000 7fb9274b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9200d2160 0x7fb9200d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.037+0000 7fb9274b8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb9200d25d0 con 0x7fb92015f3a0 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.037+0000 7fb9274b8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb9200d0e20 con 0x7fb9200d1770 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.037+0000 7fb9274b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9200b1d70 con 0x7fb9200d2160 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925254640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb92015f3a0 0x7fb9200d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925254640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb92015f3a0 0x7fb9200d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41434/0 (socket says 172.21.15.40:41434) 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925254640 1 -- 172.21.15.40:0/3416649171 learned_addr learned my addr 172.21.15.40:0/3416649171 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925a55640 1 --2- 172.21.15.40:0/3416649171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9200d2160 0x7fb9200d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb924a53640 1 --2- 172.21.15.40:0/3416649171 >> v2:172.21.15.184:3300/0 conn(0x7fb9200d1770 0x7fb9200d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925254640 1 -- 172.21.15.40:0/3416649171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9200d2160 msgr2=0x7fb9200d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925254640 1 --2- 172.21.15.40:0/3416649171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9200d2160 0x7fb9200d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925254640 1 -- 172.21.15.40:0/3416649171 >> v2:172.21.15.184:3300/0 conn(0x7fb9200d1770 msgr2=0x7fb9200d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925254640 1 --2- 172.21.15.40:0/3416649171 >> v2:172.21.15.184:3300/0 conn(0x7fb9200d1770 0x7fb9200d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.038+0000 7fb925254640 1 -- 172.21.15.40:0/3416649171 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9200b2050 con 0x7fb92015f3a0 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb925254640 1 --2- 172.21.15.40:0/3416649171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb92015f3a0 0x7fb9200d06f0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fb9200b7b80 tx=0x7fb91c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccad958ffdb94e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb917fff640 1 -- 172.21.15.40:0/3416649171 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c037070 con 0x7fb92015f3a0 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb917fff640 1 -- 172.21.15.40:0/3416649171 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c032440 con 0x7fb92015f3a0 2024-04-26T21:45:34.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb917fff640 1 -- 172.21.15.40:0/3416649171 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c032810 con 0x7fb92015f3a0 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 -- 172.21.15.40:0/3416649171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb92015f3a0 msgr2=0x7fb9200d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 --2- 172.21.15.40:0/3416649171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb92015f3a0 0x7fb9200d06f0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fb9200b7b80 tx=0x7fb91c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 -- 172.21.15.40:0/3416649171 shutdown_connections 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 --2- 172.21.15.40:0/3416649171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9200d2160 0x7fb9200d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 --2- 172.21.15.40:0/3416649171 >> v2:172.21.15.184:3300/0 conn(0x7fb9200d1770 0x7fb9200d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 --2- 172.21.15.40:0/3416649171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb92015f3a0 0x7fb9200d06f0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 -- 172.21.15.40:0/3416649171 >> 172.21.15.40:0/3416649171 conn(0x7fb9200d4f90 msgr2=0x7fb9200d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 -- 172.21.15.40:0/3416649171 shutdown_connections 2024-04-26T21:45:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.039+0000 7fb9274b8640 1 -- 172.21.15.40:0/3416649171 wait complete. 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb9274b8640 1 Processor -- start 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb9274b8640 1 -- start start 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb9274b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9200d1770 0x7fb92015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb9274b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9200d2160 0x7fb9200cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb9274b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92015f3a0 0x7fb9201f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb9274b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb92015f900 con 0x7fb9200d1770 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb9274b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb92015f780 con 0x7fb9200d2160 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb9274b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9200d3390 con 0x7fb92015f3a0 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb925254640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9200d1770 0x7fb92015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb925254640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9200d1770 0x7fb92015bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41436/0 (socket says 172.21.15.40:41436) 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.041+0000 7fb925254640 1 -- 172.21.15.40:0/2940312354 learned_addr learned my addr 172.21.15.40:0/2940312354 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb925a55640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92015f3a0 0x7fb9201f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb924a53640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9200d2160 0x7fb9200cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb925254640 1 -- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92015f3a0 msgr2=0x7fb9201f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb925254640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92015f3a0 0x7fb9201f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb925254640 1 -- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9200d2160 msgr2=0x7fb9200cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb925254640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9200d2160 0x7fb9200cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb925254640 1 -- 172.21.15.40:0/2940312354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9200c9030 con 0x7fb9200d1770 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb925254640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9200d1770 0x7fb92015bbb0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fb91c02a220 tx=0x7fb91c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb915ffb640 1 -- 172.21.15.40:0/2940312354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c041020 con 0x7fb9200d1770 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb915ffb640 1 -- 172.21.15.40:0/2940312354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c0306c0 con 0x7fb9200d1770 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9200c9310 con 0x7fb9200d1770 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb915ffb640 1 -- 172.21.15.40:0/2940312354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c037040 con 0x7fb9200d1770 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.042+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9200c98d0 con 0x7fb9200d1770 2024-04-26T21:45:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.043+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9200694f0 con 0x7fb9200d1770 2024-04-26T21:45:34.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.044+0000 7fb915ffb640 1 -- 172.21.15.40:0/2940312354 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb91c044020 con 0x7fb9200d1770 2024-04-26T21:45:34.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.044+0000 7fb915ffb640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb90406c8f0 0x7fb90406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.044+0000 7fb915ffb640 1 -- 172.21.15.40:0/2940312354 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fb91c0acf20 con 0x7fb9200d1770 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.045+0000 7fb924a53640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb90406c8f0 0x7fb90406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.045+0000 7fb924a53640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb90406c8f0 0x7fb90406edb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb9200cd7e0 tx=0x7fb910008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.049+0000 7fb915ffb640 1 -- 172.21.15.40:0/2940312354 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb91c04b030 con 0x7fb9200d1770 2024-04-26T21:45:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.305+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb92015ebc0 con 0x7fb9200d1770 2024-04-26T21:45:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.924+0000 7fb915ffb640 1 -- 172.21.15.40:0/2940312354 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v283) ==== 101+0+0 (secure 0 0 0) 0x7fb91c07a070 con 0x7fb9200d1770 2024-04-26T21:45:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb90406c8f0 msgr2=0x7fb90406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb90406c8f0 0x7fb90406edb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb9200cd7e0 tx=0x7fb910008040 comp rx=0 tx=0).stop 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9200d1770 msgr2=0x7fb92015bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9200d1770 0x7fb92015bbb0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fb91c02a220 tx=0x7fb91c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 shutdown_connections 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb90406c8f0 0x7fb90406edb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92015f3a0 0x7fb9201f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9200d2160 0x7fb9200cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 --2- 172.21.15.40:0/2940312354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9200d1770 0x7fb92015bbb0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 >> 172.21.15.40:0/2940312354 conn(0x7fb9200d4f90 msgr2=0x7fb9200d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 shutdown_connections 2024-04-26T21:45:34.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.927+0000 7fb9274b8640 1 -- 172.21.15.40:0/2940312354 wait complete. 2024-04-26T21:45:34.941 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.985+0000 7f5a06528f40 1 Processor -- start 2024-04-26T21:45:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.985+0000 7f5a06528f40 1 -- start start 2024-04-26T21:45:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a06528f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf8700 0x55fc44c7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a06528f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44bf9770 0x55fc44c87f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a06528f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44c856b0 0x55fc44c88490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a06528f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fc44be1a20 con 0x55fc44bf9770 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a06528f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc44bf9bc0 con 0x55fc44bf8700 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a06528f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55fc44be1d00 con 0x55fc44c856b0 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a03821640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44bf9770 0x55fc44c87f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a03821640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44bf9770 0x55fc44c87f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41458/0 (socket says 172.21.15.40:41458) 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a03821640 1 -- 172.21.15.40:0/465051994 learned_addr learned my addr 172.21.15.40:0/465051994 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a04823640 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44c856b0 0x55fc44c88490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a04022640 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf8700 0x55fc44c7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a03821640 1 -- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44c856b0 msgr2=0x55fc44c88490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a03821640 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44c856b0 0x55fc44c88490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a03821640 1 -- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf8700 msgr2=0x55fc44c7d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a03821640 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf8700 0x55fc44c7d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a03821640 1 -- 172.21.15.40:0/465051994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc44adacb0 con 0x55fc44bf9770 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.986+0000 7f5a04022640 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf8700 0x55fc44c7d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a03821640 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44bf9770 0x55fc44c87f50 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00a4f0 tx=0x7f59ec051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c68c28df1d89e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a0281f640 1 -- 172.21.15.40:0/465051994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec063070 con 0x55fc44bf9770 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a0281f640 1 -- 172.21.15.40:0/465051994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec054970 con 0x55fc44bf9770 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a0281f640 1 -- 172.21.15.40:0/465051994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec054d20 con 0x55fc44bf9770 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 -- 172.21.15.40:0/465051994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44bf9770 msgr2=0x55fc44c87f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44bf9770 0x55fc44c87f50 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00a4f0 tx=0x7f59ec051000 comp rx=0 tx=0).stop 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 -- 172.21.15.40:0/465051994 shutdown_connections 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44c856b0 0x55fc44c88490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44bf9770 0x55fc44c87f50 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 --2- 172.21.15.40:0/465051994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf8700 0x55fc44c7d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 -- 172.21.15.40:0/465051994 >> 172.21.15.40:0/465051994 conn(0x55fc44bdb9b0 msgr2=0x55fc44be0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 -- 172.21.15.40:0/465051994 shutdown_connections 2024-04-26T21:45:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.987+0000 7f5a06528f40 1 -- 172.21.15.40:0/465051994 wait complete. 2024-04-26T21:45:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.988+0000 7f5a06528f40 1 Processor -- start 2024-04-26T21:45:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.988+0000 7f5a06528f40 1 -- start start 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a06528f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44bf8700 0x55fc44c7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a06528f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf9770 0x55fc44c7fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a06528f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44c856b0 0x55fc44d176f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a06528f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fc44be1290 con 0x55fc44c856b0 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a06528f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc44be1570 con 0x55fc44bf9770 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a06528f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55fc44bb27d0 con 0x55fc44bf8700 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44c856b0 0x55fc44d176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44c856b0 0x55fc44d176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41460/0 (socket says 172.21.15.40:41460) 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 learned_addr learned my addr 172.21.15.40:0/494309355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04022640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44bf8700 0x55fc44c7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a03821640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf9770 0x55fc44c7fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44bf8700 msgr2=0x55fc44c7c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fc44bf8700 0x55fc44c7c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf9770 msgr2=0x55fc44c7fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc44bf9770 0x55fc44c7fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc44bdb3b0 con 0x55fc44c856b0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.989+0000 7f5a04823640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44c856b0 0x55fc44d176f0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009e80 tx=0x7f59fc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.990+0000 7f5a0181d640 1 -- 172.21.15.40:0/494309355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc0143f0 con 0x55fc44c856b0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.990+0000 7f5a0181d640 1 -- 172.21.15.40:0/494309355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59fc019070 con 0x55fc44c856b0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.990+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fc44d194c0 con 0x55fc44c856b0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.990+0000 7f5a0181d640 1 -- 172.21.15.40:0/494309355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc0147b0 con 0x55fc44c856b0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.990+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc44c805c0 con 0x55fc44c856b0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.991+0000 7f5a0181d640 1 -- 172.21.15.40:0/494309355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59fc014950 con 0x55fc44c856b0 2024-04-26T21:45:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.991+0000 7f5a0181d640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f59e406b2d0 0x7f59e406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.992+0000 7f5a0181d640 1 -- 172.21.15.40:0/494309355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f59fc08b090 con 0x55fc44c856b0 2024-04-26T21:45:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.992+0000 7f5a04022640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f59e406b2d0 0x7f59e406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.995 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.992+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc44be0690 0x55fc44d219f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.992+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bb21e0 con 0x55fc44be0690 2024-04-26T21:45:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.992+0000 7f5a03821640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc44be0690 0x55fc44d219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.992+0000 7f5a04022640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f59e406b2d0 0x7f59e406d790 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f59f400d760 tx=0x7f59f4007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.992+0000 7f5a03821640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc44be0690 0x55fc44d219f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.994+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v126'939 uv913 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.994+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d17c30 con 0x55fc44be0690 2024-04-26T21:45:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.995+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v126'940 uv940 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.995+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d21f30 con 0x55fc44be0690 2024-04-26T21:45:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.996+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v126'941 uv941 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:34.998 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.996+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d222d0 con 0x55fc44be0690 2024-04-26T21:45:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.997+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v126'942 uv942 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.997+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2bcd0 con 0x55fc44be0690 2024-04-26T21:45:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.998+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v126'943 uv943 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:34.999+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2acc0 con 0x55fc44be0690 2024-04-26T21:45:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.000+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v126'944 uv944 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.000+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b0b0 con 0x55fc44be0690 2024-04-26T21:45:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.001+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v126'945 uv919 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.001+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d2b0b0 con 0x55fc44be0690 2024-04-26T21:45:35.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.002+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v126'946 uv946 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.002+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.003 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.003+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v126'947 uv947 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.003+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.004+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v126'948 uv948 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.004+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.005+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v126'949 uv949 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.005+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.006+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v126'950 uv950 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.006+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.007+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v126'951 uv926 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.007+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.009+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v126'952 uv952 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.009+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.010 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.010+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v126'953 uv953 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.010+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.011+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v126'954 uv954 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.011+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.012+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v126'955 uv955 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.012+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b060 con 0x55fc44be0690 2024-04-26T21:45:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.013+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v126'956 uv956 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.013+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b490 con 0x55fc44be0690 2024-04-26T21:45:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.014+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v126'957 uv957 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.014+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b900 con 0x55fc44be0690 2024-04-26T21:45:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.015+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v126'958 uv932 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.015+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d2b900 con 0x55fc44be0690 2024-04-26T21:45:35.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.016+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v126'959 uv959 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.016+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2b420 con 0x55fc44be0690 2024-04-26T21:45:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.017+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v126'960 uv960 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.018 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.017+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.019+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v126'961 uv961 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.019+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.020+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v126'962 uv962 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.020+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.021+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v126'963 uv963 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.021+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.022+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v126'964 uv938 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.022+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.023+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v126'965 uv965 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.023+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.024 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.024+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v126'966 uv966 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.024+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.025+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v126'967 uv967 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.025+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.026+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v126'968 uv968 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.026+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44be0690 2024-04-26T21:45:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.027+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v126'969 uv969 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44be0690 2024-04-26T21:45:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.027+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc44d2c070 0x55fc44d2e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.027+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44bf50c0 con 0x55fc44d2c070 2024-04-26T21:45:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.027+0000 7f5a04022640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc44d2c070 0x55fc44d2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.028+0000 7f5a04022640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc44d2c070 0x55fc44d2e460 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.030+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v126'680 uv663 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.030+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d2e9a0 con 0x55fc44d2c070 2024-04-26T21:45:35.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.032+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v126'681 uv681 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.032+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2ed40 con 0x55fc44d2c070 2024-04-26T21:45:35.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.034+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v126'682 uv682 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.035 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:35.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.034+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2f590 con 0x55fc44d2c070 2024-04-26T21:45:35.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.035+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v126'683 uv683 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.035+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2f930 con 0x55fc44d2c070 2024-04-26T21:45:35.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.037+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v126'684 uv684 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.037+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d2fcd0 con 0x55fc44d2c070 2024-04-26T21:45:35.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.038+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v126'685 uv668 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.038+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.040+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v126'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.040+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.042 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.042+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v126'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.042+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.043+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v126'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.043+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.045+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v126'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.045+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.046+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v126'690 uv673 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.046+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.048+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v126'691 uv691 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.048+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.049+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v126'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.049+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d34730 con 0x55fc44d2c070 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.051+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v126'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.051+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d34730 con 0x55fc44d2c070 2024-04-26T21:45:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.052+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v126'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.052+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.054+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v126'695 uv679 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.054+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.055+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v126'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.055+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.057 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.056+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v126'697 uv697 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.056+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d31080 con 0x55fc44d2c070 2024-04-26T21:45:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.058+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v126'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.058+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d31080 con 0x55fc44d2c070 2024-04-26T21:45:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.059+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v126'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.059+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d30070 con 0x55fc44d2c070 2024-04-26T21:45:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.060+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v126'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d2c070 2024-04-26T21:45:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.060+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc44d306f0 0x55fc44d32ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.060+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d33070 con 0x55fc44d306f0 2024-04-26T21:45:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.060+0000 7f5a03821640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc44d306f0 0x55fc44d32ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.061+0000 7f5a03821640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc44d306f0 0x55fc44d32ae0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.062+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v126'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.062+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d33440 con 0x55fc44d306f0 2024-04-26T21:45:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.063+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v126'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.064+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d33440 con 0x55fc44d306f0 2024-04-26T21:45:35.065 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.065+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v126'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.065+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d34450 con 0x55fc44d306f0 2024-04-26T21:45:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.066+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v126'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.066+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d34450 con 0x55fc44d306f0 2024-04-26T21:45:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.067+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v126'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.067+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d33440 con 0x55fc44d306f0 2024-04-26T21:45:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.068+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v126'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.068+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc44d33ac0 0x55fc44d33f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.068+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d368a0 con 0x55fc44d33ac0 2024-04-26T21:45:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.068+0000 7f5a04823640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc44d33ac0 0x55fc44d33f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.068+0000 7f5a04823640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc44d33ac0 0x55fc44d33f30 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.070+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v126'139 uv138 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.070+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d36c40 con 0x55fc44d33ac0 2024-04-26T21:45:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.071+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v126'140 uv140 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.071+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d36fe0 con 0x55fc44d33ac0 2024-04-26T21:45:35.073 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:35.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.072+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v126'141 uv141 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.073+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d378a0 con 0x55fc44d33ac0 2024-04-26T21:45:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.075+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v126'142 uv142 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.075+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc44d38150 0x55fc44d3a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.075+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3aa80 con 0x55fc44d38150 2024-04-26T21:45:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.075+0000 7f5a04022640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc44d38150 0x55fc44d3a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.075+0000 7f5a04022640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc44d38150 0x55fc44d3a540 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.076+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v126'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.076+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3ab40 con 0x55fc44d38150 2024-04-26T21:45:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.077+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v126'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.077+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3af60 con 0x55fc44d38150 2024-04-26T21:45:35.079 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:35.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.078+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v126'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.079+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3bf70 con 0x55fc44d38150 2024-04-26T21:45:35.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.080+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v126'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.080+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3bf70 con 0x55fc44d38150 2024-04-26T21:45:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.081+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v126'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.081+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3af60 con 0x55fc44d38150 2024-04-26T21:45:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.082+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v126'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.082+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3af60 con 0x55fc44d33ac0 2024-04-26T21:45:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.084+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v126'338 uv332 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.084+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3b300 con 0x55fc44d33ac0 2024-04-26T21:45:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.085+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v126'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.085+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3b6a0 con 0x55fc44d33ac0 2024-04-26T21:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.086+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v126'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.086+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ca50 con 0x55fc44d33ac0 2024-04-26T21:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.087+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v126'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.087+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ca50 con 0x55fc44d33ac0 2024-04-26T21:45:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.088+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v126'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.088+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.089+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v126'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.089+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.090+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v126'344 uv337 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.091+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.091+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v126'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.092+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.094 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.093+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v126'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.093+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.095+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v126'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.095+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.095+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v126'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.096+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.097+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v126'153 uv152 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.097+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.098+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v126'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.098+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.099 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.099+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v126'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.099+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba80 con 0x55fc44d33ac0 2024-04-26T21:45:35.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.100+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v126'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.100+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d33ac0 2024-04-26T21:45:35.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.101+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v126'157 uv157 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f59fc0548b0 con 0x55fc44d33ac0 2024-04-26T21:45:35.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.101+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d38150 2024-04-26T21:45:35.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.102+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v126'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.102+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d38150 2024-04-26T21:45:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.103+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v126'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.103+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d38150 2024-04-26T21:45:35.105 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.104+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v126'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.104+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba60 con 0x55fc44d38150 2024-04-26T21:45:35.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.105+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v126'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.105+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba60 con 0x55fc44d38150 2024-04-26T21:45:35.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.106+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v126'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.106+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba60 con 0x55fc44d38150 2024-04-26T21:45:35.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.107+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v126'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.107+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d38150 2024-04-26T21:45:35.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.108+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v126'304 uv298 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.108+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d38150 2024-04-26T21:45:35.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.110+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v126'305 uv305 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.110+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ba40 con 0x55fc44d38150 2024-04-26T21:45:35.111 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:35.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.111+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v126'306 uv306 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.111+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3bf10 con 0x55fc44d38150 2024-04-26T21:45:35.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.112+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v126'307 uv307 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.112+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3bf10 con 0x55fc44d38150 2024-04-26T21:45:35.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.113+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v126'308 uv308 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.113+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3bec0 con 0x55fc44d38150 2024-04-26T21:45:35.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.115+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v126'309 uv303 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.115+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3bec0 con 0x55fc44d38150 2024-04-26T21:45:35.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.116+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v126'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.116+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3bec0 con 0x55fc44d38150 2024-04-26T21:45:35.118 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:35.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.118+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v126'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.118+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3bec0 con 0x55fc44d38150 2024-04-26T21:45:35.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.119+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v126'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.119+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ecd0 con 0x55fc44d38150 2024-04-26T21:45:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.120+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v126'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.120+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc44d3c880 0x55fc44d3f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.120+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d3c880 2024-04-26T21:45:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.120+0000 7f5a03821640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc44d3c880 0x55fc44d3f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.121+0000 7f5a03821640 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc44d3c880 0x55fc44d3f110 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.122+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v126'132 uv131 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d3c880 2024-04-26T21:45:35.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.122+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d3c880 2024-04-26T21:45:35.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.123+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v126'133 uv133 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d3c880 2024-04-26T21:45:35.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.123+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d3c880 2024-04-26T21:45:35.124 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:35.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.124+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v126'134 uv134 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d3c880 2024-04-26T21:45:35.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.124+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d3c880 2024-04-26T21:45:35.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.125+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v126'135 uv135 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d3c880 2024-04-26T21:45:35.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.125+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d38150 2024-04-26T21:45:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.126+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v126'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.126+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d38150 2024-04-26T21:45:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.127+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v126'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.127+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d38150 2024-04-26T21:45:35.129 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:35.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.128+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v126'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.128+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d38150 2024-04-26T21:45:35.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.129+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v126'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.129+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d38150 2024-04-26T21:45:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.131+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v126'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.131+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d38150 2024-04-26T21:45:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.132+0000 7f5a04022640 1 -- 172.21.15.40:0/494309355 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v126'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59f400fa20 con 0x55fc44d38150 2024-04-26T21:45:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.132+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d306f0 2024-04-26T21:45:35.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.133+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v126'156 uv155 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.133+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d306f0 2024-04-26T21:45:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.134+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v126'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.134+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3ec70 con 0x55fc44d306f0 2024-04-26T21:45:35.136 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.135+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v126'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.135+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.136+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v126'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.136+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.137+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v126'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.138+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.139+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v126'376 uv375 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.139+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.140+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v126'377 uv377 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.140+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.141 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.141+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v126'378 uv378 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.141+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.142+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v126'379 uv379 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.142+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.143+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v126'380 uv380 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.143+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.144+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v126'381 uv374 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.144+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.145+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v126'382 uv382 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.145+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.147 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.146+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v126'383 uv383 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.146+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d40660 con 0x55fc44d306f0 2024-04-26T21:45:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.147+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v126'384 uv384 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.147+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d40660 con 0x55fc44d306f0 2024-04-26T21:45:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.148+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v126'385 uv385 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.148+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.149+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v126'386 uv386 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.149+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55fc44d3f650 con 0x55fc44d306f0 2024-04-26T21:45:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a03821640 1 -- 172.21.15.40:0/494309355 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v126'387 uv387 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f59ec054c00 con 0x55fc44d306f0 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc44d2c070 msgr2=0x55fc44d2e460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55fc44d2c070 0x55fc44d2e460 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc44be0690 msgr2=0x55fc44d219f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55fc44be0690 0x55fc44d219f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc44d33ac0 msgr2=0x55fc44d33f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55fc44d33ac0 0x55fc44d33f30 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc44d3c880 msgr2=0x55fc44d3f110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55fc44d3c880 0x55fc44d3f110 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc44d306f0 msgr2=0x55fc44d32ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55fc44d306f0 0x55fc44d32ae0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc44d38150 msgr2=0x55fc44d3a540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 reap_dead start 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55fc44d38150 0x55fc44d3a540 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 reap_dead start 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a04823640 1 -- 172.21.15.40:0/494309355 reap_dead start 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f59e406b2d0 msgr2=0x7f59e406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f59e406b2d0 0x7f59e406d790 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f59f400d760 tx=0x7f59f4007040 comp rx=0 tx=0).stop 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44c856b0 msgr2=0x55fc44d176f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44c856b0 0x55fc44d176f0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009e80 tx=0x7f59fc009f70 comp rx=0 tx=0).stop 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 shutdown_connections 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f59e406b2d0 0x7f59e406d790 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 --2- 172.21.15.40:0/494309355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fc44c856b0 0x55fc44d176f0 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 >> 172.21.15.40:0/494309355 conn(0x55fc44bdb9b0 msgr2=0x55fc44be0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 shutdown_connections 2024-04-26T21:45:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.150+0000 7f5a06528f40 1 -- 172.21.15.40:0/494309355 wait complete. 2024-04-26T21:45:35.155 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 sixmegs_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:45:35.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.238+0000 7fd6fe725f40 1 Processor -- start 2024-04-26T21:45:35.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.238+0000 7fd6fe725f40 1 -- start start 2024-04-26T21:45:35.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fe725f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f224baaa0 0x560f2255f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fe725f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 0x560f225617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fe725f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f225629e0 0x560f2255b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fe725f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560f224ba0d0 con 0x560f225629e0 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fe725f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560f22432e40 con 0x560f224baaa0 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fe725f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560f224b97a0 con 0x560f224d54f0 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f225629e0 0x560f2255b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f225629e0 0x560f2255b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41480/0 (socket says 172.21.15.40:41480) 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 -- 172.21.15.40:0/3914729341 learned_addr learned my addr 172.21.15.40:0/3914729341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6faf54640 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 0x560f225617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fb755640 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f224baaa0 0x560f2255f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 -- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 msgr2=0x560f225617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 0x560f225617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 -- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f224baaa0 msgr2=0x560f2255f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f224baaa0 0x560f2255f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 -- 172.21.15.40:0/3914729341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2255e9d0 con 0x560f225629e0 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fb755640 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f224baaa0 0x560f2255f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:35.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6fbf56640 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f225629e0 0x560f2255b430 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400a4e0 tx=0x7fd6f400a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fd8d9c46c2bd5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6f9f52640 1 -- 172.21.15.40:0/3914729341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400f7b0 con 0x560f225629e0 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6f9f52640 1 -- 172.21.15.40:0/3914729341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f4015070 con 0x560f225629e0 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.239+0000 7fd6f9f52640 1 -- 172.21.15.40:0/3914729341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400fb70 con 0x560f225629e0 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 -- 172.21.15.40:0/3914729341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f225629e0 msgr2=0x560f2255b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f225629e0 0x560f2255b430 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400a4e0 tx=0x7fd6f400a9a0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 -- 172.21.15.40:0/3914729341 shutdown_connections 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f225629e0 0x560f2255b430 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 0x560f225617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 --2- 172.21.15.40:0/3914729341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f224baaa0 0x560f2255f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 -- 172.21.15.40:0/3914729341 >> 172.21.15.40:0/3914729341 conn(0x560f224b58b0 msgr2=0x560f224b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 -- 172.21.15.40:0/3914729341 shutdown_connections 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 -- 172.21.15.40:0/3914729341 wait complete. 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.240+0000 7fd6fe725f40 1 Processor -- start 2024-04-26T21:45:35.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fe725f40 1 -- start start 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fe725f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f224baaa0 0x560f2255fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fe725f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 0x560f2255d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fe725f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f225629e0 0x560f22559660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fe725f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560f2255ec90 con 0x560f224baaa0 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fe725f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560f22489920 con 0x560f225629e0 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fe725f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560f224d1460 con 0x560f224d54f0 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fb755640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f224baaa0 0x560f2255fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fb755640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f224baaa0 0x560f2255fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41488/0 (socket says 172.21.15.40:41488) 2024-04-26T21:45:35.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fb755640 1 -- 172.21.15.40:0/4225845217 learned_addr learned my addr 172.21.15.40:0/4225845217 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6fbf56640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f225629e0 0x560f22559660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.241+0000 7fd6faf54640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 0x560f2255d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6fb755640 1 -- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 msgr2=0x560f2255d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6fb755640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f224d54f0 0x560f2255d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6fb755640 1 -- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f225629e0 msgr2=0x560f22559660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6fb755640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f225629e0 0x560f22559660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6fb755640 1 -- 172.21.15.40:0/4225845217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f22560220 con 0x560f224baaa0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6fb755640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f224baaa0 0x560f2255fc90 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00cef0 tx=0x7fd6ec00a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6f8f50640 1 -- 172.21.15.40:0/4225845217 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec008040 con 0x560f224baaa0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6f8f50640 1 -- 172.21.15.40:0/4225845217 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ec014020 con 0x560f224baaa0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f224accd0 con 0x560f224baaa0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6f8f50640 1 -- 172.21.15.40:0/4225845217 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00f400 con 0x560f224baaa0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.242+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f225607c0 con 0x560f224baaa0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.243+0000 7fd6f8f50640 1 -- 172.21.15.40:0/4225845217 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6ec00f5a0 con 0x560f224baaa0 2024-04-26T21:45:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.244+0000 7fd6f8f50640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6e806b300 0x7fd6e806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.244+0000 7fd6f8f50640 1 -- 172.21.15.40:0/4225845217 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fd6ec08b040 con 0x560f224baaa0 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.244+0000 7fd6faf54640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6e806b300 0x7fd6e806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.244+0000 7fd6fe725f40 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560f2255a3a0 0x560f225f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.244+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=104b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x560f224ad1b0 con 0x560f2255a3a0 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.244+0000 7fd6fbf56640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560f2255a3a0 0x560f225f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.244+0000 7fd6faf54640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6e806b300 0x7fd6e806d7c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0057000 tx=0x7fd6f0051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.244+0000 7fd6fbf56640 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560f2255a3a0 0x560f225f39c0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fbf56640 1 -- 172.21.15.40:0/4225845217 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v126'970 uv970 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd6f400e070 con 0x560f2255a3a0 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560f2255a3a0 msgr2=0x560f225f39c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560f2255a3a0 0x560f225f39c0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6e806b300 msgr2=0x7fd6e806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd6e806b300 0x7fd6e806d7c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0057000 tx=0x7fd6f0051000 comp rx=0 tx=0).stop 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f224baaa0 msgr2=0x560f2255fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 --2- 172.21.15.40:0/4225845217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f224baaa0 0x560f2255fc90 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00cef0 tx=0x7fd6ec00a5d0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fbf56640 1 -- 172.21.15.40:0/4225845217 reap_dead start 2024-04-26T21:45:35.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 shutdown_connections 2024-04-26T21:45:35.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 >> 172.21.15.40:0/4225845217 conn(0x560f224b58b0 msgr2=0x560f2255f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 shutdown_connections 2024-04-26T21:45:35.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.246+0000 7fd6fe725f40 1 -- 172.21.15.40:0/4225845217 wait complete. 2024-04-26T21:45:35.251 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.353+0000 7ff795512640 1 Processor -- start 2024-04-26T21:45:35.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.353+0000 7ff795512640 1 -- start start 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.354+0000 7ff795512640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff7900b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.354+0000 7ff795512640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7900d0d60 0x7ff7900d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.354+0000 7ff795512640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79006e580 con 0x7ff7900d1720 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.354+0000 7ff795512640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff79006f3e0 con 0x7ff7900d0d60 2024-04-26T21:45:35.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.354+0000 7ff795512640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7900cd6e0 con 0x7ff79015f770 2024-04-26T21:45:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78effd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff7900b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78e7fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7900d0d60 0x7ff7900d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78effd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff7900b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:37724/0 (socket says 172.21.15.40:37724) 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78effd640 1 -- 172.21.15.40:0/1772807574 learned_addr learned my addr 172.21.15.40:0/1772807574 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78e7fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7900d0d60 0x7ff7900d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:41440/0 (socket says 172.21.15.40:41440) 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78effd640 1 -- 172.21.15.40:0/1772807574 >> v2:172.21.15.184:3300/0 conn(0x7ff7900d0d60 msgr2=0x7ff7900d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78effd640 1 --2- 172.21.15.40:0/1772807574 >> v2:172.21.15.184:3300/0 conn(0x7ff7900d0d60 0x7ff7900d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78effd640 1 -- 172.21.15.40:0/1772807574 >> v1:172.21.15.184:6789/0 conn(0x7ff7900d1720 legacy=0x7ff7900cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.355+0000 7ff78effd640 1 -- 172.21.15.40:0/1772807574 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79006e860 con 0x7ff79015f770 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff78effd640 1 --2- 172.21.15.40:0/1772807574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff7900b1fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff7880098a0 tx=0x7ff78802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f011736a5324cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff78dffb640 1 -- 172.21.15.40:0/1772807574 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff79015f770 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff78dffb640 1 -- 172.21.15.40:0/1772807574 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788032440 con 0x7ff79015f770 2024-04-26T21:45:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff78dffb640 1 -- 172.21.15.40:0/1772807574 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788032810 con 0x7ff79015f770 2024-04-26T21:45:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff795512640 1 -- 172.21.15.40:0/1772807574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 msgr2=0x7ff7900b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff795512640 1 --2- 172.21.15.40:0/1772807574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff7900b1fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff7880098a0 tx=0x7ff78802a290 comp rx=0 tx=0).stop 2024-04-26T21:45:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff795512640 1 -- 172.21.15.40:0/1772807574 shutdown_connections 2024-04-26T21:45:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff795512640 1 --2- 172.21.15.40:0/1772807574 >> v2:172.21.15.184:3300/0 conn(0x7ff7900d0d60 0x7ff7900d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff795512640 1 --2- 172.21.15.40:0/1772807574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff7900b1fd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff795512640 1 -- 172.21.15.40:0/1772807574 >> 172.21.15.40:0/1772807574 conn(0x7ff7900d2e60 msgr2=0x7ff7900d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff795512640 1 -- 172.21.15.40:0/1772807574 shutdown_connections 2024-04-26T21:45:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.356+0000 7ff795512640 1 -- 172.21.15.40:0/1772807574 wait complete. 2024-04-26T21:45:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.358+0000 7ff795512640 1 Processor -- start 2024-04-26T21:45:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.358+0000 7ff795512640 1 -- start start 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.358+0000 7ff795512640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7900d0d60 0x7ff7901f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.358+0000 7ff795512640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7900d1720 0x7ff79014da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff795512640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff79014df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff795512640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7900ceb80 con 0x7ff7900d0d60 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff795512640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7900d1bb0 con 0x7ff7900d1720 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff795512640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7900ced00 con 0x7ff79015f770 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7900d0d60 0x7ff7901f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7900d0d60 0x7ff7901f4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41496/0 (socket says 172.21.15.40:41496) 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 -- 172.21.15.40:0/2929250596 learned_addr learned my addr 172.21.15.40:0/2929250596 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78e7fc640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7900d1720 0x7ff79014da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78f7fe640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff79014df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 -- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 msgr2=0x7ff79014df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff79014df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 -- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7900d1720 msgr2=0x7ff79014da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7900d1720 0x7ff79014da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 -- 172.21.15.40:0/2929250596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79009e570 con 0x7ff7900d0d60 2024-04-26T21:45:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff78effd640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7900d0d60 0x7ff7901f4fb0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7ff788000c00 tx=0x7ff7880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff7877fe640 1 -- 172.21.15.40:0/2929250596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037040 con 0x7ff7900d0d60 2024-04-26T21:45:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff7877fe640 1 -- 172.21.15.40:0/2929250596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788030440 con 0x7ff7900d0d60 2024-04-26T21:45:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7900cc7f0 con 0x7ff7900d0d60 2024-04-26T21:45:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff7877fe640 1 -- 172.21.15.40:0/2929250596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788030810 con 0x7ff7900d0d60 2024-04-26T21:45:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.359+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79009eaf0 con 0x7ff7900d0d60 2024-04-26T21:45:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.360+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7900d11e0 con 0x7ff7900d0d60 2024-04-26T21:45:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.361+0000 7ff7877fe640 1 -- 172.21.15.40:0/2929250596 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff788044030 con 0x7ff7900d0d60 2024-04-26T21:45:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.361+0000 7ff7877fe640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff77006c920 0x7ff77006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.361+0000 7ff7877fe640 1 -- 172.21.15.40:0/2929250596 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7ff7880ac3e0 con 0x7ff7900d0d60 2024-04-26T21:45:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.362+0000 7ff78e7fc640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff77006c920 0x7ff77006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.362+0000 7ff78e7fc640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff77006c920 0x7ff77006ede0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff7900d38e0 tx=0x7ff778009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:35.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.366+0000 7ff7877fe640 1 -- 172.21.15.40:0/2929250596 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7900d11e0 con 0x7ff7900d0d60 2024-04-26T21:45:35.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.590+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff7900694f0 con 0x7ff7900d0d60 2024-04-26T21:45:35.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.669+0000 7ff7877fe640 1 -- 172.21.15.40:0/2929250596 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v284) ==== 162+0+0 (secure 0 0 0) 0x7ff788079530 con 0x7ff7900d0d60 2024-04-26T21:45:35.669 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff77006c920 msgr2=0x7ff77006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff77006c920 0x7ff77006ede0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff7900d38e0 tx=0x7ff778009190 comp rx=0 tx=0).stop 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7900d0d60 msgr2=0x7ff7901f4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7900d0d60 0x7ff7901f4fb0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7ff788000c00 tx=0x7ff7880046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 shutdown_connections 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff77006c920 0x7ff77006ede0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff79015f770 0x7ff79014df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7900d1720 0x7ff79014da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 --2- 172.21.15.40:0/2929250596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7900d0d60 0x7ff7901f4fb0 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 >> 172.21.15.40:0/2929250596 conn(0x7ff7900d2e60 msgr2=0x7ff7900717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 shutdown_connections 2024-04-26T21:45:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:35.672+0000 7ff795512640 1 -- 172.21.15.40:0/2929250596 wait complete. 2024-04-26T21:45:36.087 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:36.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.192+0000 7fd9fbd87640 1 Processor -- start 2024-04-26T21:45:36.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.193+0000 7fd9fbd87640 1 -- start start 2024-04-26T21:45:36.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.193+0000 7fd9fbd87640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd9f415f3a0 0x7fd9f40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.193+0000 7fd9fbd87640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9f40d2330 0x7fd9f40b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.193+0000 7fd9fbd87640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f40d0e20 con 0x7fd9f40d1770 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.193+0000 7fd9fbd87640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f40b1d70 con 0x7fd9f40d2330 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.193+0000 7fd9fbd87640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f40b2050 con 0x7fd9f415f3a0 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.194+0000 7fd9fa324640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9f40d2330 0x7fd9f40b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.194+0000 7fd9fa324640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9f40d2330 0x7fd9f40b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41504/0 (socket says 172.21.15.40:41504) 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.194+0000 7fd9fa324640 1 -- 172.21.15.40:0/3577628500 learned_addr learned my addr 172.21.15.40:0/3577628500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.194+0000 7fd9f9b23640 1 --2- 172.21.15.40:0/3577628500 >> v2:172.21.15.184:3300/0 conn(0x7fd9f415f3a0 0x7fd9f40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.194+0000 7fd9fa324640 1 -- 172.21.15.40:0/3577628500 >> v2:172.21.15.184:3300/0 conn(0x7fd9f415f3a0 msgr2=0x7fd9f40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.194+0000 7fd9fa324640 1 --2- 172.21.15.40:0/3577628500 >> v2:172.21.15.184:3300/0 conn(0x7fd9f415f3a0 0x7fd9f40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.194+0000 7fd9fa324640 1 -- 172.21.15.40:0/3577628500 >> v1:172.21.15.184:6789/0 conn(0x7fd9f40d1770 legacy=0x7fd9f40d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.194+0000 7fd9fa324640 1 -- 172.21.15.40:0/3577628500 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f40b2330 con 0x7fd9f40d2330 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fa324640 1 --2- 172.21.15.40:0/3577628500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d2330 0x7fd9f40b2850 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40c89f0 tx=0x7fd9f002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372700db66b74dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9f8b21640 1 -- 172.21.15.40:0/3577628500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0036070 con 0x7fd9f40d2330 2024-04-26T21:45:36.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9f8b21640 1 -- 172.21.15.40:0/3577628500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f0031600 con 0x7fd9f40d2330 2024-04-26T21:45:36.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9f8b21640 1 -- 172.21.15.40:0/3577628500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00319d0 con 0x7fd9f40d2330 2024-04-26T21:45:36.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3577628500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d2330 msgr2=0x7fd9f40b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3577628500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d2330 0x7fd9f40b2850 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40c89f0 tx=0x7fd9f002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3577628500 shutdown_connections 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3577628500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d2330 0x7fd9f40b2850 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3577628500 >> v2:172.21.15.184:3300/0 conn(0x7fd9f415f3a0 0x7fd9f40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3577628500 >> 172.21.15.40:0/3577628500 conn(0x7fd9f40d4f90 msgr2=0x7fd9f40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3577628500 shutdown_connections 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.195+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3577628500 wait complete. 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9fbd87640 1 Processor -- start 2024-04-26T21:45:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9fbd87640 1 -- start start 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9fbd87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d1770 0x7fd9f40b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9fbd87640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40d2330 0x7fd9f40b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9fbd87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f415f3a0 0x7fd9f4161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9fbd87640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f40b2f10 con 0x7fd9f40d1770 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9fbd87640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f40b2d90 con 0x7fd9f415f3a0 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9fbd87640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f40b3090 con 0x7fd9f40d2330 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.197+0000 7fd9f9b23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d1770 0x7fd9f40b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9b23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d1770 0x7fd9f40b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41510/0 (socket says 172.21.15.40:41510) 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9b23640 1 -- 172.21.15.40:0/3093523840 learned_addr learned my addr 172.21.15.40:0/3093523840 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9fa324640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f415f3a0 0x7fd9f4161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9322640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40d2330 0x7fd9f40b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9b23640 1 -- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40d2330 msgr2=0x7fd9f40b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9b23640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40d2330 0x7fd9f40b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9b23640 1 -- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f415f3a0 msgr2=0x7fd9f4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9b23640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f415f3a0 0x7fd9f4161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9b23640 1 -- 172.21.15.40:0/3093523840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f4162720 con 0x7fd9f40d1770 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9322640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40d2330 0x7fd9f40b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9f9b23640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d1770 0x7fd9f40b5750 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00d620 tx=0x7fd9ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9ea7fc640 1 -- 172.21.15.40:0/3093523840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00b620 con 0x7fd9f40d1770 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9ea7fc640 1 -- 172.21.15.40:0/3093523840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec019070 con 0x7fd9f40d1770 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f415fa30 con 0x7fd9f40d1770 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9ea7fc640 1 -- 172.21.15.40:0/3093523840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00ba00 con 0x7fd9f40d1770 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.198+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f415fff0 con 0x7fd9f40d1770 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.199+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f40694f0 con 0x7fd9f40d1770 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.200+0000 7fd9ea7fc640 1 -- 172.21.15.40:0/3093523840 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9ec00bba0 con 0x7fd9f40d1770 2024-04-26T21:45:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.200+0000 7fd9ea7fc640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9d806c920 0x7fd9d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.200+0000 7fd9ea7fc640 1 -- 172.21.15.40:0/3093523840 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fd9ec08cab0 con 0x7fd9f40d1770 2024-04-26T21:45:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.200+0000 7fd9f9322640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9d806c920 0x7fd9d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.201+0000 7fd9f9322640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9d806c920 0x7fd9d806ede0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4161510 tx=0x7fd9e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.204+0000 7fd9ea7fc640 1 -- 172.21.15.40:0/3093523840 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd9ec05a2b0 con 0x7fd9f40d1770 2024-04-26T21:45:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.431+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9f40b9320 con 0x7fd9f40d1770 2024-04-26T21:45:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.432+0000 7fd9ea7fc640 1 -- 172.21.15.40:0/3093523840 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 284 v284) ==== 95+0+5193 (secure 0 0 0) 0x7fd9ec059bb0 con 0x7fd9f40d1770 2024-04-26T21:45:36.433 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:36.433 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":284,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":284,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:35.668882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"c","rank":0,"incarnation":284,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/4214786907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4214786907},{"type":"v1","addr":"172.21.15.40:6833","nonce":4214786907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 284 2024-04-26T21:45:36.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9d806c920 msgr2=0x7fd9d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9d806c920 0x7fd9d806ede0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4161510 tx=0x7fd9e4009040 comp rx=0 tx=0).stop 2024-04-26T21:45:36.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d1770 msgr2=0x7fd9f40b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:36.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d1770 0x7fd9f40b5750 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00d620 tx=0x7fd9ec00dae0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 shutdown_connections 2024-04-26T21:45:36.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9d806c920 0x7fd9d806ede0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f415f3a0 0x7fd9f4161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40d2330 0x7fd9f40b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 --2- 172.21.15.40:0/3093523840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40d1770 0x7fd9f40b5750 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 >> 172.21.15.40:0/3093523840 conn(0x7fd9f40d4f90 msgr2=0x7fd9f40b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 shutdown_connections 2024-04-26T21:45:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:36.434+0000 7fd9fbd87640 1 -- 172.21.15.40:0/3093523840 wait complete. 2024-04-26T21:45:36.445 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 284, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:45:35.668882+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10677}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10677': {'gid': 10677, 'name': 'c', 'rank': 0, 'incarnation': 284, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/4214786907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 4214786907}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 4214786907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:45:36.446 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10677:up:replay 2024-04-26T21:45:36.446 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:45:41.850 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:41.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.963+0000 7f8bb37a5640 1 Processor -- start 2024-04-26T21:45:41.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.964+0000 7f8bb37a5640 1 -- start start 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.964+0000 7f8bb37a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bac162810 0x7f8bac070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.964+0000 7f8bb37a5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8bac0d1b30 0x7f8bac0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.964+0000 7f8bb37a5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bac0d10d0 con 0x7f8bac0b2340 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.964+0000 7f8bb37a5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8bac0d0750 con 0x7f8bac162810 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.964+0000 7f8bb37a5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8bac0d16a0 con 0x7f8bac0d1b30 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb1541640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bac162810 0x7f8bac070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb0d40640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8bac0b2340 0x7f8bac0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37024/0 (socket says 172.21.15.40:37024) 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb0d40640 1 -- 172.21.15.40:0/3887785851 learned_addr learned my addr 172.21.15.40:0/3887785851 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb1d42640 1 --2- 172.21.15.40:0/3887785851 >> v2:172.21.15.184:3300/0 conn(0x7f8bac0d1b30 0x7f8bac0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:41.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb1541640 1 -- 172.21.15.40:0/3887785851 >> v2:172.21.15.184:3300/0 conn(0x7f8bac0d1b30 msgr2=0x7f8bac0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb1541640 1 --2- 172.21.15.40:0/3887785851 >> v2:172.21.15.184:3300/0 conn(0x7f8bac0d1b30 0x7f8bac0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb1541640 1 -- 172.21.15.40:0/3887785851 >> v1:172.21.15.40:6789/0 conn(0x7f8bac0b2340 legacy=0x7f8bac0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8babfff640 1 -- 172.21.15.40:0/3887785851 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156153690 0 0) 0x7f8bac0d10d0 con 0x7f8bac0b2340 2024-04-26T21:45:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb1541640 1 -- 172.21.15.40:0/3887785851 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac0d1370 con 0x7f8bac162810 2024-04-26T21:45:41.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.965+0000 7f8bb1d42640 1 --2- 172.21.15.40:0/3887785851 >> v2:172.21.15.184:3300/0 conn(0x7f8bac0d1b30 0x7f8bac0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.966+0000 7f8bb1541640 1 --2- 172.21.15.40:0/3887785851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac162810 0x7f8bac070350 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f8bac073330 tx=0x7f8ba402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850f3438be2055dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.966+0000 7f8babfff640 1 -- 172.21.15.40:0/3887785851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4007740 con 0x7f8bac162810 2024-04-26T21:45:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.966+0000 7f8babfff640 1 -- 172.21.15.40:0/3887785851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba40078e0 con 0x7f8bac162810 2024-04-26T21:45:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.966+0000 7f8babfff640 1 -- 172.21.15.40:0/3887785851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4007cb0 con 0x7f8bac162810 2024-04-26T21:45:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.966+0000 7f8bb37a5640 1 -- 172.21.15.40:0/3887785851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac162810 msgr2=0x7f8bac070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.966+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/3887785851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac162810 0x7f8bac070350 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f8bac073330 tx=0x7f8ba402a680 comp rx=0 tx=0).stop 2024-04-26T21:45:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.967+0000 7f8bb37a5640 1 -- 172.21.15.40:0/3887785851 shutdown_connections 2024-04-26T21:45:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.967+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/3887785851 >> v2:172.21.15.184:3300/0 conn(0x7f8bac0d1b30 0x7f8bac0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.967+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/3887785851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac162810 0x7f8bac070350 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.967+0000 7f8bb37a5640 1 -- 172.21.15.40:0/3887785851 >> 172.21.15.40:0/3887785851 conn(0x7f8bac0d2da0 msgr2=0x7f8bac0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.967+0000 7f8bb37a5640 1 -- 172.21.15.40:0/3887785851 shutdown_connections 2024-04-26T21:45:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.967+0000 7f8bb37a5640 1 -- 172.21.15.40:0/3887785851 wait complete. 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.968+0000 7f8bb37a5640 1 Processor -- start 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.968+0000 7f8bb37a5640 1 -- start start 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb37a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0b2340 0x7f8bac153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb37a5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bac0d1b30 0x7f8bac0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb37a5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bac162810 0x7f8bac0c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb37a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac070890 con 0x7f8bac0b2340 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb37a5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac0d0470 con 0x7f8bac162810 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb37a5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8bac0cdb00 con 0x7f8bac0d1b30 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0b2340 0x7f8bac153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0b2340 0x7f8bac153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48520/0 (socket says 172.21.15.40:48520) 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 -- 172.21.15.40:0/2750731538 learned_addr learned my addr 172.21.15.40:0/2750731538 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb0d40640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bac0d1b30 0x7f8bac0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1d42640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bac162810 0x7f8bac0c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 -- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bac0d1b30 msgr2=0x7f8bac0c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bac0d1b30 0x7f8bac0c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 -- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bac162810 msgr2=0x7f8bac0c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bac162810 0x7f8bac0c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 -- 172.21.15.40:0/2750731538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac0cae80 con 0x7f8bac0b2340 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb1541640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0b2340 0x7f8bac153f30 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4000f80 tx=0x7f8ba4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8ba9ffb640 1 -- 172.21.15.40:0/2750731538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba403f070 con 0x7f8bac0b2340 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8ba9ffb640 1 -- 172.21.15.40:0/2750731538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba4031c40 con 0x7f8bac0b2340 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.969+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bac0c6a70 con 0x7f8bac0b2340 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.970+0000 7f8ba9ffb640 1 -- 172.21.15.40:0/2750731538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4003990 con 0x7f8bac0b2340 2024-04-26T21:45:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.970+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bac152450 con 0x7f8bac0b2340 2024-04-26T21:45:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.971+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bac0694f0 con 0x7f8bac0b2340 2024-04-26T21:45:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.971+0000 7f8ba9ffb640 1 -- 172.21.15.40:0/2750731538 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ba4007790 con 0x7f8bac0b2340 2024-04-26T21:45:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.972+0000 7f8ba9ffb640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b9806c920 0x7f8b9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.972+0000 7f8ba9ffb640 1 -- 172.21.15.40:0/2750731538 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f8ba40adaa0 con 0x7f8bac0b2340 2024-04-26T21:45:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.972+0000 7f8bb0d40640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b9806c920 0x7f8b9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.973+0000 7f8bb0d40640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b9806c920 0x7f8b9806ede0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8bac069fd0 tx=0x7f8ba0007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:41.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:41.977+0000 7f8ba9ffb640 1 -- 172.21.15.40:0/2750731538 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ba407b210 con 0x7f8bac0b2340 2024-04-26T21:45:42.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.207+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bac0b2ca0 con 0x7f8bac0b2340 2024-04-26T21:45:42.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.207+0000 7f8ba9ffb640 1 -- 172.21.15.40:0/2750731538 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 287 v287) ==== 95+0+5202 (secure 0 0 0) 0x7f8ba407ab10 con 0x7f8bac0b2340 2024-04-26T21:45:42.208 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:42.208 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":287,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":287,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:38.674239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"c","rank":0,"incarnation":284,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/4214786907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4214786907},{"type":"v1","addr":"172.21.15.40:6833","nonce":4214786907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10677,"qdb_cluster":[10677]},"id":6}]} 2024-04-26T21:45:42.209 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 287 2024-04-26T21:45:42.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b9806c920 msgr2=0x7f8b9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b9806c920 0x7f8b9806ede0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8bac069fd0 tx=0x7f8ba0007b90 comp rx=0 tx=0).stop 2024-04-26T21:45:42.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0b2340 msgr2=0x7f8bac153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0b2340 0x7f8bac153f30 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4000f80 tx=0x7f8ba4004a40 comp rx=0 tx=0).stop 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 shutdown_connections 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8b9806c920 0x7f8b9806ede0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bac162810 0x7f8bac0c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bac0d1b30 0x7f8bac0c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 --2- 172.21.15.40:0/2750731538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0b2340 0x7f8bac153f30 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 >> 172.21.15.40:0/2750731538 conn(0x7f8bac0d2da0 msgr2=0x7f8bac0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 shutdown_connections 2024-04-26T21:45:42.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.209+0000 7f8bb37a5640 1 -- 172.21.15.40:0/2750731538 wait complete. 2024-04-26T21:45:42.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 287, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:45:38.674239+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10677}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10677': {'gid': 10677, 'name': 'c', 'rank': 0, 'incarnation': 284, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6833/4214786907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 4214786907}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 4214786907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10677, 'qdb_cluster': [10677]} 2024-04-26T21:45:42.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:45:42.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:45:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.322+0000 7fc73990a640 1 Processor -- start 2024-04-26T21:45:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.322+0000 7fc73990a640 1 -- start start 2024-04-26T21:45:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.323+0000 7fc73990a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc7340d1bf0 0x7fc7340d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.323+0000 7fc73990a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7340cde20 con 0x7fc7340cffa0 2024-04-26T21:45:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.323+0000 7fc73990a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7340ce100 con 0x7fc7341608f0 2024-04-26T21:45:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.323+0000 7fc73990a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc7340cea30 con 0x7fc7340d1bf0 2024-04-26T21:45:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.323+0000 7fc7327fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc7340cffa0 0x7fc7340b2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37034/0 (socket says 172.21.15.40:37034) 2024-04-26T21:45:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.323+0000 7fc7327fc640 1 -- 172.21.15.40:0/4264725915 learned_addr learned my addr 172.21.15.40:0/4264725915 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc7337fe640 1 --2- 172.21.15.40:0/4264725915 >> v2:172.21.15.184:3300/0 conn(0x7fc7340d1bf0 0x7fc7340d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc731ffb640 1 -- 172.21.15.40:0/4264725915 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303050538 0 0) 0x7fc7340cde20 con 0x7fc7340cffa0 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc731ffb640 1 -- 172.21.15.40:0/4264725915 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc720003520 con 0x7fc7340cffa0 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc731ffb640 1 -- 172.21.15.40:0/4264725915 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2693636714 0 0) 0x7fc7340ce100 con 0x7fc7341608f0 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc731ffb640 1 -- 172.21.15.40:0/4264725915 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7340cde20 con 0x7fc7341608f0 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc731ffb640 1 -- 172.21.15.40:0/4264725915 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2754894001 0 0) 0x7fc720003520 con 0x7fc7340cffa0 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc7337fe640 1 -- 172.21.15.40:0/4264725915 >> v1:172.21.15.184:6789/0 conn(0x7fc7341608f0 legacy=0x7fc7340cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc7337fe640 1 -- 172.21.15.40:0/4264725915 >> v1:172.21.15.40:6789/0 conn(0x7fc7340cffa0 legacy=0x7fc7340b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.324+0000 7fc7337fe640 1 -- 172.21.15.40:0/4264725915 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7340ce3e0 con 0x7fc7340d1bf0 2024-04-26T21:45:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc7337fe640 1 --2- 172.21.15.40:0/4264725915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340d1bf0 0x7fc7340d0a90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc724009d90 tx=0x7fc72402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3744787a053d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc731ffb640 1 -- 172.21.15.40:0/4264725915 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0015f0 con 0x7fc7340d1bf0 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc731ffb640 1 -- 172.21.15.40:0/4264725915 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc724007540 con 0x7fc7340d1bf0 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc731ffb640 1 -- 172.21.15.40:0/4264725915 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724007910 con 0x7fc7340d1bf0 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc73990a640 1 -- 172.21.15.40:0/4264725915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340d1bf0 msgr2=0x7fc7340d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc73990a640 1 --2- 172.21.15.40:0/4264725915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340d1bf0 0x7fc7340d0a90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc724009d90 tx=0x7fc72402a680 comp rx=0 tx=0).stop 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc73990a640 1 -- 172.21.15.40:0/4264725915 shutdown_connections 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc73990a640 1 --2- 172.21.15.40:0/4264725915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340d1bf0 0x7fc7340d0a90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc73990a640 1 -- 172.21.15.40:0/4264725915 >> 172.21.15.40:0/4264725915 conn(0x7fc7340d2ec0 msgr2=0x7fc7340d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc73990a640 1 -- 172.21.15.40:0/4264725915 shutdown_connections 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.325+0000 7fc73990a640 1 -- 172.21.15.40:0/4264725915 wait complete. 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.327+0000 7fc73990a640 1 Processor -- start 2024-04-26T21:45:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.327+0000 7fc73990a640 1 -- start start 2024-04-26T21:45:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.327+0000 7fc73990a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340cffa0 0x7fc7340ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.327+0000 7fc73990a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7340d1bf0 0x7fc7340c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc73990a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7341608f0 0x7fc7341e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc73990a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc7340cda40 con 0x7fc7340d1bf0 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc73990a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7340ceec0 con 0x7fc7340cffa0 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc73990a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc7340cdbc0 con 0x7fc7341608f0 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7340d1bf0 0x7fc7340c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7340d1bf0 0x7fc7340c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48524/0 (socket says 172.21.15.40:48524) 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 -- 172.21.15.40:0/69235553 learned_addr learned my addr 172.21.15.40:0/69235553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7337fe640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7341608f0 0x7fc7341e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc732ffd640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340cffa0 0x7fc7340ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 -- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7341608f0 msgr2=0x7fc7341e6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7341608f0 0x7fc7341e6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 -- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340cffa0 msgr2=0x7fc7340ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340cffa0 0x7fc7340ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 -- 172.21.15.40:0/69235553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7340cad60 con 0x7fc7340d1bf0 2024-04-26T21:45:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc7327fc640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7340d1bf0 0x7fc7340c8bc0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c4e0 tx=0x7fc71c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc72bfff640 1 -- 172.21.15.40:0/69235553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c005ce0 con 0x7fc7340d1bf0 2024-04-26T21:45:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc72bfff640 1 -- 172.21.15.40:0/69235553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c007070 con 0x7fc7340d1bf0 2024-04-26T21:45:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc73990a640 1 -- 172.21.15.40:0/69235553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7340cb040 con 0x7fc7340d1bf0 2024-04-26T21:45:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc72bfff640 1 -- 172.21.15.40:0/69235553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0074a0 con 0x7fc7340d1bf0 2024-04-26T21:45:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.328+0000 7fc73990a640 1 -- 172.21.15.40:0/69235553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7340cb600 con 0x7fc7340d1bf0 2024-04-26T21:45:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.330+0000 7fc72bfff640 1 -- 172.21.15.40:0/69235553 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc71c01b400 con 0x7fc7340d1bf0 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.330+0000 7fc72bfff640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71406c920 0x7fc71406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.330+0000 7fc72bfff640 1 -- 172.21.15.40:0/69235553 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fc71c08c5f0 con 0x7fc7340d1bf0 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.331+0000 7fc732ffd640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71406c920 0x7fc71406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.331+0000 7fc732ffd640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71406c920 0x7fc71406ede0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0010e0 tx=0x7fc72c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.331+0000 7fc73c629740 1 Processor -- start 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.331+0000 7fc73c629740 1 -- start start 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc73c629740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0279f0 0x5654bd000050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc73c629740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd02a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc73c629740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcffdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc73c629740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5654bcf64740 con 0x5654bd0279f0 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc73c629740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5654bcf003a0 con 0x5654bd027dd0 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc73c629740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5654bcf60700 con 0x5654bd02a790 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc732ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0279f0 0x5654bd000050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc732ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0279f0 0x5654bd000050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48528/0 (socket says 172.21.15.40:48528) 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc732ffd640 1 -- 172.21.15.40:0/4153002788 learned_addr learned my addr 172.21.15.40:0/4153002788 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc7337fe640 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcffdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc7327fc640 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd02a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc732ffd640 1 -- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 msgr2=0x5654bcffdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc732ffd640 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcffdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc732ffd640 1 -- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 msgr2=0x5654bd02a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc732ffd640 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd02a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc732ffd640 1 -- 172.21.15.40:0/4153002788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bcf1c750 con 0x5654bd0279f0 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc7327fc640 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd02a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.332+0000 7fc7337fe640 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcffdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.333+0000 7fc732ffd640 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0279f0 0x5654bd000050 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fc72c001e80 tx=0x7fc72c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c29bb509eff080 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.333+0000 7fc7297fa640 1 -- 172.21.15.40:0/4153002788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0035f0 con 0x5654bd0279f0 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.333+0000 7fc7297fa640 1 -- 172.21.15.40:0/4153002788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c003790 con 0x5654bd0279f0 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.333+0000 7fc7297fa640 1 -- 172.21.15.40:0/4153002788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c003b40 con 0x5654bd0279f0 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.333+0000 7fc73c629740 1 -- 172.21.15.40:0/4153002788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0279f0 msgr2=0x5654bd000050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.333+0000 7fc73c629740 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0279f0 0x5654bd000050 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fc72c001e80 tx=0x7fc72c011000 comp rx=0 tx=0).stop 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 -- 172.21.15.40:0/4153002788 shutdown_connections 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcffdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd02a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 --2- 172.21.15.40:0/4153002788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0279f0 0x5654bd000050 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 -- 172.21.15.40:0/4153002788 >> 172.21.15.40:0/4153002788 conn(0x5654bcffca50 msgr2=0x5654bd045250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 -- 172.21.15.40:0/4153002788 shutdown_connections 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 -- 172.21.15.40:0/4153002788 wait complete. 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 -- start start 2024-04-26T21:45:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.334+0000 7fc73c629740 1 Processor -- start 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc73c629740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd03c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc73c629740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcff9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc73c629740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bcffa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc73c629740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5654bceebfe0 con 0x5654bd0150c0 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc73c629740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5654bd042f50 con 0x5654bd027dd0 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc73c629740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5654bd015920 con 0x5654bd02a790 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc73c629740 10 client.0 _ll_register_callbacks cb 0x5654bcf02150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bcffa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bcffa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48536/0 (socket says 172.21.15.40:48536) 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7337fe640 1 -- 172.21.15.40:0/938010779 learned_addr learned my addr 172.21.15.40:0/938010779 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc732ffd640 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd03c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7327fc640 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcff9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7337fe640 1 -- 172.21.15.40:0/938010779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 msgr2=0x5654bcff9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7337fe640 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcff9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7337fe640 1 -- 172.21.15.40:0/938010779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 msgr2=0x5654bd03c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7337fe640 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd03c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.335+0000 7fc7337fe640 1 -- 172.21.15.40:0/938010779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bcc762d0 con 0x5654bd0150c0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc7337fe640 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bcffa190 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fc72402ac70 tx=0x7fc724004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcb09ab62a5eb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc7127fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc7127fc640 1 -- 172.21.15.40:0/938010779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724009450 con 0x5654bd0150c0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc7127fc640 1 -- 172.21.15.40:0/938010779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fc72403b410 con 0x5654bd0150c0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 1 -- 172.21.15.40:0/938010779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c003790 con 0x5654bd0150c0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc7127fc640 1 -- 172.21.15.40:0/938010779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=287) ==== 24+0+0 (secure 0 0 0) 0x7fc724045640 con 0x5654bd0150c0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 10 client.10758 fetch_fsmap learned FSMap version 287 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 1 -- 172.21.15.40:0/938010779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=287,monmap=2+}) -- 0x5654bcd406b0 con 0x5654bd0150c0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc7127fc640 1 -- 172.21.15.40:0/938010779 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c003790 con 0x5654bd0150c0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc7127fc640 1 -- 172.21.15.40:0/938010779 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 287) ==== 3494+0+0 (secure 0 0 0) 0x7fc724043070 con 0x5654bd0150c0 2024-04-26T21:45:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 10 client.10758 fetch_fsmap finished waiting for FSMap version 287 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 10 client.10758 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 20 client.10758 populate_metadata read hostname 'smithi040' 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x5654bd0294b0 0x5654bd029890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 4 client.10758 mds_command: new command op to 10677 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.336+0000 7fc73c629740 1 -- 172.21.15.40:0/938010779 --> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654bcf72350 con 0x5654bd0294b0 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.337+0000 7fc732ffd640 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x5654bd0294b0 0x5654bd029890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.337+0000 7fc7127fc640 10 client.10758 ms_handle_connect on v2:172.21.15.40:6832/4214786907 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.337+0000 7fc732ffd640 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x5654bd0294b0 0x5654bd029890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba6bd9141344c831 server_cookie=510d2cb92bb582c in_seq=0 out_seq=0 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.337+0000 7fc7127fc640 1 -- 172.21.15.40:0/938010779 <== mds.0 v2:172.21.15.40:6832/4214786907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654bcf72350 con 0x5654bd0294b0 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.337+0000 7fc7127fc640 10 client.10758 handle_command_reply: tid=0 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.337+0000 7fc73c629740 1 client.10758 shutdown 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 -- 172.21.15.40:0/938010779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 msgr2=0x5654bcffa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bcffa190 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fc72402ac70 tx=0x7fc724004ab0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 -- 172.21.15.40:0/938010779 shutdown_connections 2024-04-26T21:45:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x5654bd0294b0 0x5654bd029890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bcffa190 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bcff9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc7127fc640 0 client.10758 ms_handle_reset on v2:172.21.15.40:6832/4214786907 2024-04-26T21:45:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 --2- 172.21.15.40:0/938010779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd03c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 -- 172.21.15.40:0/938010779 >> 172.21.15.40:0/938010779 conn(0x5654bd0279f0 msgr2=0x5654bd0459e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 -- 172.21.15.40:0/938010779 shutdown_connections 2024-04-26T21:45:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 1 -- 172.21.15.40:0/938010779 wait complete. 2024-04-26T21:45:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.338+0000 7fc73c629740 20 client.10758 trim_cache size 0 max 16384 2024-04-26T21:45:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.356+0000 7fc73c629740 1 Processor -- start 2024-04-26T21:45:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.356+0000 7fc73c629740 1 -- start start 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc73c629740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd0150c0 0x5654bd014750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc73c629740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd027dd0 0x5654bd02ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc73c629740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bd02bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc73c629740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341fe700 con 0x5654bd027dd0 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc73c629740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc73406eec0 con 0x5654bd0150c0 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc73c629740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc7341fe190 con 0x5654bd02a790 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd027dd0 0x5654bd02ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd027dd0 0x5654bd02ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48542/0 (socket says 172.21.15.40:48542) 2024-04-26T21:45:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 -- 172.21.15.40:0/3482989008 learned_addr learned my addr 172.21.15.40:0/3482989008 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 -- 172.21.15.40:0/3482989008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 msgr2=0x5654bd02bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 --2- 172.21.15.40:0/3482989008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bd02bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 -- 172.21.15.40:0/3482989008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd0150c0 msgr2=0x5654bd014750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 --2- 172.21.15.40:0/3482989008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd0150c0 0x5654bd014750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 -- 172.21.15.40:0/3482989008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bcffdb90 con 0x5654bd027dd0 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.357+0000 7fc7327fc640 1 --2- 172.21.15.40:0/3482989008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd027dd0 0x5654bd02ba60 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fc71c055a30 tx=0x7fc71c059e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7317e5567f7a207 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc728ff9640 1 -- 172.21.15.40:0/3482989008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c088070 con 0x5654bd027dd0 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc728ff9640 1 -- 172.21.15.40:0/3482989008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c062240 con 0x5654bd027dd0 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc728ff9640 1 -- 172.21.15.40:0/3482989008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0578d0 con 0x5654bd027dd0 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 -- 172.21.15.40:0/3482989008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd027dd0 msgr2=0x5654bd02ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 --2- 172.21.15.40:0/3482989008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd027dd0 0x5654bd02ba60 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fc71c055a30 tx=0x7fc71c059e10 comp rx=0 tx=0).stop 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 -- 172.21.15.40:0/3482989008 shutdown_connections 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 --2- 172.21.15.40:0/3482989008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bd02bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 --2- 172.21.15.40:0/3482989008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd027dd0 0x5654bd02ba60 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 --2- 172.21.15.40:0/3482989008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd0150c0 0x5654bd014750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 -- 172.21.15.40:0/3482989008 >> 172.21.15.40:0/3482989008 conn(0x5654bd0279f0 msgr2=0x5654bd02ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 -- 172.21.15.40:0/3482989008 shutdown_connections 2024-04-26T21:45:42.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.358+0000 7fc73c629740 1 -- 172.21.15.40:0/3482989008 wait complete. 2024-04-26T21:45:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.359+0000 7fc73c629740 1 -- start start 2024-04-26T21:45:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.359+0000 7fc73c629740 1 Processor -- start 2024-04-26T21:45:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bd0b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd024970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bd045580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5654bd03c800 con 0x5654bd0150c0 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5654bd02d3a0 con 0x5654bd027dd0 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5654bd02dde0 con 0x5654bd02a790 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 10 client.0 _ll_register_callbacks cb 0x7fc7200036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bd0b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bd0b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48548/0 (socket says 172.21.15.40:48548) 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 -- 172.21.15.40:0/601167913 learned_addr learned my addr 172.21.15.40:0/601167913 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 -- 172.21.15.40:0/601167913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 msgr2=0x5654bd045580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bd045580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 -- 172.21.15.40:0/601167913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 msgr2=0x5654bd024970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd024970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 -- 172.21.15.40:0/601167913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bcffdff0 con 0x5654bd0150c0 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc732ffd640 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bd0b7300 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fc72c014000 tx=0x7fc72c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a819debfd48cda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc7127fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:45:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc7127fc640 1 -- 172.21.15.40:0/601167913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c003c40 con 0x5654bd0150c0 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc7127fc640 1 -- 172.21.15.40:0/601167913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fc72c003de0 con 0x5654bd0150c0 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 1 -- 172.21.15.40:0/601167913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc71c062240 con 0x5654bd0150c0 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc7127fc640 1 -- 172.21.15.40:0/601167913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=287) ==== 24+0+0 (secure 0 0 0) 0x7fc72c03f050 con 0x5654bd0150c0 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 10 client.10764 fetch_fsmap learned FSMap version 287 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.360+0000 7fc73c629740 1 -- 172.21.15.40:0/601167913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=287,monmap=2+}) -- 0x5654bcffed70 con 0x5654bd0150c0 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc7127fc640 1 -- 172.21.15.40:0/601167913 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c062240 con 0x5654bd0150c0 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc7127fc640 1 -- 172.21.15.40:0/601167913 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 287) ==== 3494+0+0 (secure 0 0 0) 0x7fc72c030170 con 0x5654bd0150c0 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 10 client.10764 fetch_fsmap finished waiting for FSMap version 287 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 10 client.10764 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 20 client.10764 populate_metadata read hostname 'smithi040' 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x5654bd026520 0x5654bd026900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 4 client.10764 mds_command: new command op to 10677 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 -- 172.21.15.40:0/601167913 --> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] -- command(tid 0: {"prefix": "status"}) -- 0x5654bcf6c1c0 con 0x5654bd026520 2024-04-26T21:45:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc7327fc640 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x5654bd026520 0x5654bd026900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc7127fc640 10 client.10764 ms_handle_connect on v2:172.21.15.40:6832/4214786907 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc7327fc640 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x5654bd026520 0x5654bd026900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30d5d351891d52c server_cookie=c8ba1e9143756df in_seq=0 out_seq=0 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc7127fc640 1 -- 172.21.15.40:0/601167913 <== mds.0 v2:172.21.15.40:6832/4214786907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x5654bcf6c1c0 con 0x5654bd026520 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc7127fc640 10 client.10764 handle_command_reply: tid=0 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 client.10764 shutdown 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 -- 172.21.15.40:0/601167913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 msgr2=0x5654bd0b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bd0b7300 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fc72c014000 tx=0x7fc72c000ed0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 -- 172.21.15.40:0/601167913 shutdown_connections 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x5654bd026520 0x5654bd026900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5654bd02a790 0x5654bd045580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc7127fc640 0 client.10764 ms_handle_reset on v2:172.21.15.40:6832/4214786907 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5654bd027dd0 0x5654bd024970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 --2- 172.21.15.40:0/601167913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5654bd0150c0 0x5654bd0b7300 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.361+0000 7fc73c629740 1 -- 172.21.15.40:0/601167913 >> 172.21.15.40:0/601167913 conn(0x5654bd0279f0 msgr2=0x5654bd029890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73c629740 1 -- 172.21.15.40:0/601167913 shutdown_connections 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73c629740 1 -- 172.21.15.40:0/601167913 wait complete. 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73c629740 20 client.10764 trim_cache size 0 max 16384 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 10677, 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:45:42.368 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.6922336309999997, 2024-04-26T21:45:42.369 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 287, 2024-04-26T21:45:42.369 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 126, 2024-04-26T21:45:42.369 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 126, 2024-04-26T21:45:42.369 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.422360204 2024-04-26T21:45:42.369 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 -- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71406c920 msgr2=0x7fc71406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71406c920 0x7fc71406ede0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0010e0 tx=0x7fc72c009040 comp rx=0 tx=0).stop 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 -- 172.21.15.40:0/69235553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7340d1bf0 msgr2=0x7fc7340c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7340d1bf0 0x7fc7340c8bc0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c4e0 tx=0x7fc71c00c9a0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 -- 172.21.15.40:0/69235553 shutdown_connections 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71406c920 0x7fc71406ede0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7341608f0 0x7fc7341e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7340d1bf0 0x7fc7340c8bc0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 --2- 172.21.15.40:0/69235553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7340cffa0 0x7fc7340ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 -- 172.21.15.40:0/69235553 >> 172.21.15.40:0/69235553 conn(0x7fc7340d2ec0 msgr2=0x7fc7340b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 -- 172.21.15.40:0/69235553 shutdown_connections 2024-04-26T21:45:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.362+0000 7fc73990a640 1 -- 172.21.15.40:0/69235553 wait complete. 2024-04-26T21:45:42.374 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:45:42.374 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head', proceeding to ls 2024-04-26T21:45:42.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:45:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.473+0000 7f1529045640 1 Processor -- start 2024-04-26T21:45:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.473+0000 7f1529045640 1 -- start start 2024-04-26T21:45:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1529045640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1524071e60 0x7f152406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1529045640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f152406fe20 0x7f1524072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1529045640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f152415d7c0 con 0x7f1524072780 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1529045640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f152415daa0 con 0x7f1524071e60 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1529045640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f152415dd80 con 0x7f152406fe20 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1522d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1524071e60 0x7f152406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f152359e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1524072780 0x7f15240cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37050/0 (socket says 172.21.15.40:37050) 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f152359e640 1 -- 172.21.15.40:0/87370780 learned_addr learned my addr 172.21.15.40:0/87370780 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f151bfff640 1 --2- 172.21.15.40:0/87370780 >> v2:172.21.15.184:3300/0 conn(0x7f152406fe20 0x7f1524072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1522d9d640 1 -- 172.21.15.40:0/87370780 >> v2:172.21.15.184:3300/0 conn(0x7f152406fe20 msgr2=0x7f1524072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1522d9d640 1 --2- 172.21.15.40:0/87370780 >> v2:172.21.15.184:3300/0 conn(0x7f152406fe20 0x7f1524072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1522d9d640 1 -- 172.21.15.40:0/87370780 >> v1:172.21.15.40:6789/0 conn(0x7f1524072780 legacy=0x7f15240cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f1522d9d640 1 -- 172.21.15.40:0/87370780 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152415e060 con 0x7f1524071e60 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.474+0000 7f152259c640 1 -- 172.21.15.40:0/87370780 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244245885 0 0) 0x7f152415d7c0 con 0x7f1524072780 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.475+0000 7f1522d9d640 1 --2- 172.21.15.40:0/87370780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1524071e60 0x7f152406f8e0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f15240b3820 tx=0x7f151402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12379d97b9bc831 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.475+0000 7f152259c640 1 -- 172.21.15.40:0/87370780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514037070 con 0x7f1524071e60 2024-04-26T21:45:42.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.475+0000 7f152259c640 1 -- 172.21.15.40:0/87370780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1514032440 con 0x7f1524071e60 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.475+0000 7f152259c640 1 -- 172.21.15.40:0/87370780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514032810 con 0x7f1524071e60 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.475+0000 7f1529045640 1 -- 172.21.15.40:0/87370780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1524071e60 msgr2=0x7f152406f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.475+0000 7f1529045640 1 --2- 172.21.15.40:0/87370780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1524071e60 0x7f152406f8e0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f15240b3820 tx=0x7f151402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.476+0000 7f1529045640 1 -- 172.21.15.40:0/87370780 shutdown_connections 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.476+0000 7f1529045640 1 --2- 172.21.15.40:0/87370780 >> v2:172.21.15.184:3300/0 conn(0x7f152406fe20 0x7f1524072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.476+0000 7f1529045640 1 --2- 172.21.15.40:0/87370780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1524071e60 0x7f152406f8e0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.476+0000 7f1529045640 1 -- 172.21.15.40:0/87370780 >> 172.21.15.40:0/87370780 conn(0x7f15240d0be0 msgr2=0x7f15240d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.476+0000 7f1529045640 1 -- 172.21.15.40:0/87370780 shutdown_connections 2024-04-26T21:45:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.476+0000 7f1529045640 1 -- 172.21.15.40:0/87370780 wait complete. 2024-04-26T21:45:42.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.477+0000 7f1529045640 1 Processor -- start 2024-04-26T21:45:42.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.477+0000 7f1529045640 1 -- start start 2024-04-26T21:45:42.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1529045640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f152406fe20 0x7f1524151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1529045640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1524071e60 0x7f15240b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1529045640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1524072780 0x7f15240cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1529045640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f152415d3f0 con 0x7f152406fe20 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1529045640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1524072bd0 con 0x7f1524072780 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1529045640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f152415d570 con 0x7f1524071e60 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f152406fe20 0x7f1524151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f152406fe20 0x7f1524151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48562/0 (socket says 172.21.15.40:48562) 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 -- 172.21.15.40:0/3375594040 learned_addr learned my addr 172.21.15.40:0/3375594040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f151bfff640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1524071e60 0x7f15240b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f152359e640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1524072780 0x7f15240cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 -- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1524071e60 msgr2=0x7f15240b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1524071e60 0x7f15240b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 -- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1524072780 msgr2=0x7f15240cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1524072780 0x7f15240cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 -- 172.21.15.40:0/3375594040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15240b4be0 con 0x7f152406fe20 2024-04-26T21:45:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1522d9d640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f152406fe20 0x7f1524151160 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f1514000c00 tx=0x7f15140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f151b7fe640 1 -- 172.21.15.40:0/3375594040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514037040 con 0x7f152406fe20 2024-04-26T21:45:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f151b7fe640 1 -- 172.21.15.40:0/3375594040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1514042070 con 0x7f152406fe20 2024-04-26T21:45:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.478+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15240b4ec0 con 0x7f152406fe20 2024-04-26T21:45:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.479+0000 7f151b7fe640 1 -- 172.21.15.40:0/3375594040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514030bd0 con 0x7f152406fe20 2024-04-26T21:45:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.479+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15240b5760 con 0x7f152406fe20 2024-04-26T21:45:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.479+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15240b52b0 con 0x7f152406fe20 2024-04-26T21:45:42.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.480+0000 7f151b7fe640 1 -- 172.21.15.40:0/3375594040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1514047460 con 0x7f152406fe20 2024-04-26T21:45:42.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.480+0000 7f151b7fe640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f150406c8f0 0x7f150406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:42.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.481+0000 7f151b7fe640 1 -- 172.21.15.40:0/3375594040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f15140ad3e0 con 0x7f152406fe20 2024-04-26T21:45:42.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.481+0000 7f151bfff640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f150406c8f0 0x7f150406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:42.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.481+0000 7f151bfff640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f150406c8f0 0x7f150406edb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f15240b2530 tx=0x7f150c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:42.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.485+0000 7f151b7fe640 1 -- 172.21.15.40:0/3375594040 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1514032e20 con 0x7f152406fe20 2024-04-26T21:45:42.592 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:45:42.595 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:45:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.711+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f15240694f0 con 0x7f152406fe20 2024-04-26T21:45:42.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.712+0000 7f151b7fe640 1 -- 172.21.15.40:0/3375594040 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v287) ==== 53+0+73 (secure 0 0 0) 0x7f151407ab50 con 0x7f152406fe20 2024-04-26T21:45:42.713 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f150406c8f0 msgr2=0x7f150406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f150406c8f0 0x7f150406edb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f15240b2530 tx=0x7f150c008040 comp rx=0 tx=0).stop 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f152406fe20 msgr2=0x7f1524151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f152406fe20 0x7f1524151160 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f1514000c00 tx=0x7f15140046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 shutdown_connections 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f150406c8f0 0x7f150406edb0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1524072780 0x7f15240cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1524071e60 0x7f15240b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 --2- 172.21.15.40:0/3375594040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f152406fe20 0x7f1524151160 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 >> 172.21.15.40:0/3375594040 conn(0x7f15240d0be0 msgr2=0x7f15240b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:42.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 shutdown_connections 2024-04-26T21:45:42.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42.714+0000 7f1529045640 1 -- 172.21.15.40:0/3375594040 wait complete. 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:45:42.726 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:45:42.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:42.726 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:45:42.743 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42105sec preferred_lft 42105sec 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:45:42.744 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:45:42.745 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:45:42.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:42.746 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:45:42.806 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:45:42.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:42.808 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:45:42.864 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:45:42.864 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:45:42.864 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:45:42.864 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:45:42.864 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:45:42.865 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:45:42.865 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:45:42.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:45:42.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:45:42.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:45:42.865 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:45:42.918 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:42.918+0000 7f3961e8c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:45:42.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:45:42.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:45:42.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:42.991 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:45:43.047 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:45:43.047 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:45:43.047 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:45:43.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:43.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:45:43.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:45:43.112 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:45:43.167 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:45:43.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:45:43.167 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:45:43.222 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:45:43.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:43.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:45:43.305 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:45:43.306 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:45:43.313 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:45:43.313 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:45:43.313 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:45:43.314 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:45:43.314 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:45:43.314 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:45:43.314 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:45:43.314 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:45:43.314 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:45:43.314 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:45:43.316 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:45:43.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:43.316 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:45:43.381 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:45:43.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:45:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:45:43.477 INFO:teuthology.orchestra.run.smithi040.stdout:client10776 2024-04-26T21:45:43.478 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:43.478 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:43.493 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:45:43.493 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:45:43.493 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:45:43.493 DEBUG:teuthology.orchestra.run.smithi040:> import json 2024-04-26T21:45:43.493 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:45:43.493 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir/sixmegs'"'"') 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> sys.exit(e.errno) 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> print(json.dumps( 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> indent=2)) 2024-04-26T21:45:43.494 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:45:43.536 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:45:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-04-26T21:45:43.612 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-04-26T21:45:43.613 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-04-26T21:45:43.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:43.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.753+0000 7f66d81da640 1 Processor -- start 2024-04-26T21:45:43.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.753+0000 7f66d81da640 1 -- start start 2024-04-26T21:45:43.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d81da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66d01604e0 0x7f66d006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d81da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d0160970 con 0x7f66d015f2d0 2024-04-26T21:45:43.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d81da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d0072f70 con 0x7f66d006fcd0 2024-04-26T21:45:43.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d81da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f66d0074010 con 0x7f66d01604e0 2024-04-26T21:45:43.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d5f76640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f66d015f2d0 0x7f66d0072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37058/0 (socket says 172.21.15.40:37058) 2024-04-26T21:45:43.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d5f76640 1 -- 172.21.15.40:0/1943683124 learned_addr learned my addr 172.21.15.40:0/1943683124 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:43.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d5775640 1 --2- 172.21.15.40:0/1943683124 >> v2:172.21.15.40:3300/0 conn(0x7f66d01604e0 0x7f66d006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d4f74640 1 -- 172.21.15.40:0/1943683124 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397901947 0 0) 0x7f66d0160970 con 0x7f66d015f2d0 2024-04-26T21:45:43.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d4f74640 1 -- 172.21.15.40:0/1943683124 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66bc003520 con 0x7f66d015f2d0 2024-04-26T21:45:43.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d5775640 1 -- 172.21.15.40:0/1943683124 >> v1:172.21.15.184:6789/0 conn(0x7f66d006fcd0 legacy=0x7f66d00ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d5775640 1 -- 172.21.15.40:0/1943683124 >> v1:172.21.15.40:6789/0 conn(0x7f66d015f2d0 legacy=0x7f66d0072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.754+0000 7f66d5775640 1 -- 172.21.15.40:0/1943683124 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d0073250 con 0x7f66d01604e0 2024-04-26T21:45:43.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.755+0000 7f66d5775640 1 --2- 172.21.15.40:0/1943683124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d006f790 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f66c00097a0 tx=0x7f66c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23fd1aaaec0a568 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.755+0000 7f66d4f74640 1 -- 172.21.15.40:0/1943683124 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0037070 con 0x7f66d01604e0 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.755+0000 7f66d4f74640 1 -- 172.21.15.40:0/1943683124 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c0032440 con 0x7f66d01604e0 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.755+0000 7f66d4f74640 1 -- 172.21.15.40:0/1943683124 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00327f0 con 0x7f66d01604e0 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.756+0000 7f66d81da640 1 -- 172.21.15.40:0/1943683124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 msgr2=0x7f66d006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.756+0000 7f66d81da640 1 --2- 172.21.15.40:0/1943683124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d006f790 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f66c00097a0 tx=0x7f66c002a230 comp rx=0 tx=0).stop 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.756+0000 7f66d81da640 1 -- 172.21.15.40:0/1943683124 shutdown_connections 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.756+0000 7f66d81da640 1 --2- 172.21.15.40:0/1943683124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d006f790 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.756+0000 7f66d81da640 1 -- 172.21.15.40:0/1943683124 >> 172.21.15.40:0/1943683124 conn(0x7f66d00d0be0 msgr2=0x7f66d00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.756+0000 7f66d81da640 1 -- 172.21.15.40:0/1943683124 shutdown_connections 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.756+0000 7f66d81da640 1 -- 172.21.15.40:0/1943683124 wait complete. 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.757+0000 7f66d81da640 1 Processor -- start 2024-04-26T21:45:43.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.757+0000 7f66d81da640 1 -- start start 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d81da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66d006fcd0 0x7f66d014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d81da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66d015f2d0 0x7f66d00caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d81da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d0157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d81da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0073810 con 0x7f66d01604e0 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d81da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0070100 con 0x7f66d015f2d0 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d81da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66d0073af0 con 0x7f66d006fcd0 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d0157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d0157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48582/0 (socket says 172.21.15.40:48582) 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 -- 172.21.15.40:0/2579040557 learned_addr learned my addr 172.21.15.40:0/2579040557 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d5f76640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66d006fcd0 0x7f66d014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 -- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66d006fcd0 msgr2=0x7f66d014a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66d006fcd0 0x7f66d014a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 -- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66d015f2d0 msgr2=0x7f66d00caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66d015f2d0 0x7f66d00caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 -- 172.21.15.40:0/2579040557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d01547c0 con 0x7f66d01604e0 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66d6777640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d0157fd0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f66c800c7d0 tx=0x7f66c800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.758+0000 7f66c67fc640 1 -- 172.21.15.40:0/2579040557 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8003b70 con 0x7f66d01604e0 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.759+0000 7f66c67fc640 1 -- 172.21.15.40:0/2579040557 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c8003d10 con 0x7f66d01604e0 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.759+0000 7f66c67fc640 1 -- 172.21.15.40:0/2579040557 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c800aa90 con 0x7f66d01604e0 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.759+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d0154aa0 con 0x7f66d01604e0 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.759+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d00cb430 con 0x7f66d01604e0 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.759+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d00694f0 con 0x7f66d01604e0 2024-04-26T21:45:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.760+0000 7f66c67fc640 1 -- 172.21.15.40:0/2579040557 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66c800ac30 con 0x7f66d01604e0 2024-04-26T21:45:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.760+0000 7f66c67fc640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b406c8f0 0x7f66b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.761+0000 7f66c67fc640 1 -- 172.21.15.40:0/2579040557 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f66c808c740 con 0x7f66d01604e0 2024-04-26T21:45:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.761+0000 7f66d5f76640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b406c8f0 0x7f66b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.761+0000 7f66d5f76640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b406c8f0 0x7f66b406edb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0011e0 tx=0x7f66cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.765+0000 7f66c67fc640 1 -- 172.21.15.40:0/2579040557 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66c8059ee0 con 0x7f66d01604e0 2024-04-26T21:45:43.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.991+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66d00c7a70 con 0x7f66d01604e0 2024-04-26T21:45:43.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.991+0000 7f66c67fc640 1 -- 172.21.15.40:0/2579040557 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 287 v287) ==== 95+0+5202 (secure 0 0 0) 0x7f66c80597e0 con 0x7f66d01604e0 2024-04-26T21:45:43.992 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:43.992 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":287,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":287,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:38.674239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"c","rank":0,"incarnation":284,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/4214786907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4214786907},{"type":"v1","addr":"172.21.15.40:6833","nonce":4214786907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10677,"qdb_cluster":[10677]},"id":6}]} 2024-04-26T21:45:43.993 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 287 2024-04-26T21:45:43.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b406c8f0 msgr2=0x7f66b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b406c8f0 0x7f66b406edb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0011e0 tx=0x7f66cc009040 comp rx=0 tx=0).stop 2024-04-26T21:45:43.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 msgr2=0x7f66d0157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:43.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d0157fd0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f66c800c7d0 tx=0x7f66c800cc90 comp rx=0 tx=0).stop 2024-04-26T21:45:43.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 shutdown_connections 2024-04-26T21:45:43.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f66b406c8f0 0x7f66b406edb0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d01604e0 0x7f66d0157fd0 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66d015f2d0 0x7f66d00caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 --2- 172.21.15.40:0/2579040557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66d006fcd0 0x7f66d014a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:43.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 >> 172.21.15.40:0/2579040557 conn(0x7f66d00d0be0 msgr2=0x7f66d0150ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:43.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 shutdown_connections 2024-04-26T21:45:43.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:43.993+0000 7f66d81da640 1 -- 172.21.15.40:0/2579040557 wait complete. 2024-04-26T21:45:44.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-26T21:45:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.106+0000 7f8599609640 1 Processor -- start 2024-04-26T21:45:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.106+0000 7f8599609640 1 -- start start 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8599609640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8594162880 0x7f85940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8599609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d3bc0 0x7f85940b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8599609640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85940b7e50 con 0x7f85940b5f20 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8599609640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f85940d0580 con 0x7f8594162880 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8599609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85940d1800 con 0x7f85940d3bc0 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8592ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8594162880 0x7f85940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f85927fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f85940b5f20 0x7f85940d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37064/0 (socket says 172.21.15.40:37064) 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f85927fc640 1 -- 172.21.15.40:0/245240327 learned_addr learned my addr 172.21.15.40:0/245240327 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f85937fe640 1 --2- 172.21.15.40:0/245240327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d3bc0 0x7f85940b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8592ffd640 1 -- 172.21.15.40:0/245240327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d3bc0 msgr2=0x7f85940b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8592ffd640 1 --2- 172.21.15.40:0/245240327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d3bc0 0x7f85940b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8592ffd640 1 -- 172.21.15.40:0/245240327 >> v1:172.21.15.40:6789/0 conn(0x7f85940b5f20 legacy=0x7f85940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8592ffd640 1 -- 172.21.15.40:0/245240327 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940d07d0 con 0x7f8594162880 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f8591ffb640 1 -- 172.21.15.40:0/245240327 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1232478698 0 0) 0x7f85940b7e50 con 0x7f85940b5f20 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.107+0000 7f85937fe640 1 --2- 172.21.15.40:0/245240327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d3bc0 0x7f85940b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.108+0000 7f8592ffd640 1 --2- 172.21.15.40:0/245240327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8594162880 0x7f85940d2cb0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f858c0097a0 tx=0x7f858c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421d6041a4c0a1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.108+0000 7f8591ffb640 1 -- 172.21.15.40:0/245240327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c037070 con 0x7f8594162880 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.108+0000 7f8591ffb640 1 -- 172.21.15.40:0/245240327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858c032440 con 0x7f8594162880 2024-04-26T21:45:44.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.108+0000 7f8591ffb640 1 -- 172.21.15.40:0/245240327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c032810 con 0x7f8594162880 2024-04-26T21:45:44.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.109+0000 7f8599609640 1 -- 172.21.15.40:0/245240327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8594162880 msgr2=0x7f85940d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.109+0000 7f8599609640 1 --2- 172.21.15.40:0/245240327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8594162880 0x7f85940d2cb0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f858c0097a0 tx=0x7f858c02a180 comp rx=0 tx=0).stop 2024-04-26T21:45:44.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.109+0000 7f8599609640 1 -- 172.21.15.40:0/245240327 shutdown_connections 2024-04-26T21:45:44.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.109+0000 7f8599609640 1 --2- 172.21.15.40:0/245240327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d3bc0 0x7f85940b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.109+0000 7f8599609640 1 --2- 172.21.15.40:0/245240327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8594162880 0x7f85940d2cb0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.109+0000 7f8599609640 1 -- 172.21.15.40:0/245240327 >> 172.21.15.40:0/245240327 conn(0x7f85940d4f60 msgr2=0x7f85940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.109+0000 7f8599609640 1 -- 172.21.15.40:0/245240327 shutdown_connections 2024-04-26T21:45:44.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.109+0000 7f8599609640 1 -- 172.21.15.40:0/245240327 wait complete. 2024-04-26T21:45:44.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.110+0000 7f8599609640 1 Processor -- start 2024-04-26T21:45:44.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.110+0000 7f8599609640 1 -- start start 2024-04-26T21:45:44.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f8599609640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85940b5f20 0x7f859409f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f8599609640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85940d3bc0 0x7f85940a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f8599609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8594162880 0x7f85941f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f8599609640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8594162c60 con 0x7f85940d3bc0 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f8599609640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85940d0d90 con 0x7f85940b5f20 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f8599609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8594162de0 con 0x7f8594162880 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85927fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85940d3bc0 0x7f85940a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85927fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85940d3bc0 0x7f85940a2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48598/0 (socket says 172.21.15.40:48598) 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85927fc640 1 -- 172.21.15.40:0/3167927338 learned_addr learned my addr 172.21.15.40:0/3167927338 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85937fe640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8594162880 0x7f85941f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f8592ffd640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85940b5f20 0x7f859409f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85927fc640 1 -- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8594162880 msgr2=0x7f85941f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85927fc640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8594162880 0x7f85941f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85927fc640 1 -- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85940b5f20 msgr2=0x7f859409f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85927fc640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85940b5f20 0x7f859409f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f85927fc640 1 -- 172.21.15.40:0/3167927338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940a29c0 con 0x7f85940d3bc0 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.111+0000 7f8592ffd640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85940b5f20 0x7f859409f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.112+0000 7f85927fc640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85940d3bc0 0x7f85940a2480 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f858800c430 tx=0x7f858800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.112+0000 7f857b7fe640 1 -- 172.21.15.40:0/3167927338 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85880074e0 con 0x7f85940d3bc0 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.112+0000 7f857b7fe640 1 -- 172.21.15.40:0/3167927338 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85880047e0 con 0x7f85940d3bc0 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.112+0000 7f8599609640 1 -- 172.21.15.40:0/3167927338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859409fb80 con 0x7f85940d3bc0 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.112+0000 7f857b7fe640 1 -- 172.21.15.40:0/3167927338 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588004bd0 con 0x7f85940d3bc0 2024-04-26T21:45:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.112+0000 7f8599609640 1 -- 172.21.15.40:0/3167927338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85940a0420 con 0x7f85940d3bc0 2024-04-26T21:45:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.113+0000 7f857b7fe640 1 -- 172.21.15.40:0/3167927338 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8588011070 con 0x7f85940d3bc0 2024-04-26T21:45:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.114+0000 7f857b7fe640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f858006c8f0 0x7f858006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.114+0000 7f857b7fe640 1 -- 172.21.15.40:0/3167927338 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f858808c4f0 con 0x7f85940d3bc0 2024-04-26T21:45:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.114+0000 7f8592ffd640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f858006c8f0 0x7f858006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.115+0000 7f859c402740 1 Processor -- start 2024-04-26T21:45:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.115+0000 7f8592ffd640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f858006c8f0 0x7f858006edb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f858c034000 tx=0x7f858c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.115+0000 7f859c402740 1 -- start start 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.115+0000 7f859c402740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e549535cf0 0x55e549537e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.115+0000 7f859c402740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e5495360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.115+0000 7f859c402740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e5495607b0 0x55e549560c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.115+0000 7f859c402740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e54949ab80 con 0x55e549535cf0 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.115+0000 7f859c402740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e5494364d0 con 0x55e5495723c0 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f859c402740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e549496b40 con 0x55e5495607b0 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e549535cf0 0x55e549537e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e549535cf0 0x55e549537e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48604/0 (socket says 172.21.15.40:48604) 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 -- 172.21.15.40:0/2781107513 learned_addr learned my addr 172.21.15.40:0/2781107513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f85937fe640 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e5495607b0 0x55e549560c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f85927fc640 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e5495360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 -- 172.21.15.40:0/2781107513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e5495607b0 msgr2=0x55e549560c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e5495607b0 0x55e549560c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 -- 172.21.15.40:0/2781107513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 msgr2=0x55e5495360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e5495360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 -- 172.21.15.40:0/2781107513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e549463770 con 0x55e549535cf0 2024-04-26T21:45:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8592ffd640 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e549535cf0 0x55e549537e50 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f858c009890 tx=0x7f858c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfba729a74a361e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8578ff9640 1 -- 172.21.15.40:0/2781107513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c04d070 con 0x55e549535cf0 2024-04-26T21:45:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8578ff9640 1 -- 172.21.15.40:0/2781107513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858c048470 con 0x55e549535cf0 2024-04-26T21:45:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.116+0000 7f8578ff9640 1 -- 172.21.15.40:0/2781107513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c037040 con 0x55e549535cf0 2024-04-26T21:45:44.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 -- 172.21.15.40:0/2781107513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e549535cf0 msgr2=0x55e549537e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e549535cf0 0x55e549537e50 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f858c009890 tx=0x7f858c02c000 comp rx=0 tx=0).stop 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 -- 172.21.15.40:0/2781107513 shutdown_connections 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e5495607b0 0x55e549560c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e5495360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 --2- 172.21.15.40:0/2781107513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e549535cf0 0x55e549537e50 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 -- 172.21.15.40:0/2781107513 >> 172.21.15.40:0/2781107513 conn(0x55e5494cae90 msgr2=0x55e54957b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 -- 172.21.15.40:0/2781107513 shutdown_connections 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.117+0000 7f859c402740 1 -- 172.21.15.40:0/2781107513 wait complete. 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 1 -- start start 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 1 Processor -- start 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e549552190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54957e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549575cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e549433e70 con 0x55e5495607b0 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e54955e300 con 0x55e5495723c0 2024-04-26T21:45:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e54958c300 con 0x55e549535cf0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.118+0000 7f859c402740 10 client.0 _ll_register_callbacks cb 0x55e549444ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54957e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54957e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48608/0 (socket says 172.21.15.40:48608) 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 -- 172.21.15.40:0/1854007120 learned_addr learned my addr 172.21.15.40:0/1854007120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f8592ffd640 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e549552190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85937fe640 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549575cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 -- 172.21.15.40:0/1854007120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 msgr2=0x55e549552190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e549552190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 -- 172.21.15.40:0/1854007120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 msgr2=0x55e549575cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549575cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 -- 172.21.15.40:0/1854007120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5492787c0 con 0x55e5495607b0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f85927fc640 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54957e870 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f85880559c0 tx=0x7f8588062140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df94275170f64ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f8571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f8571ffb640 1 -- 172.21.15.40:0/1854007120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588071020 con 0x55e5495607b0 2024-04-26T21:45:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f8571ffb640 1 -- 172.21.15.40:0/1854007120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f8588055bd0 con 0x55e5495607b0 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f859c402740 1 -- 172.21.15.40:0/1854007120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f858c048470 con 0x55e5495607b0 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f8571ffb640 1 -- 172.21.15.40:0/1854007120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=287) ==== 24+0+0 (secure 0 0 0) 0x7f85880589b0 con 0x55e5495607b0 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f859c402740 10 client.10800 fetch_fsmap learned FSMap version 287 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f859c402740 1 -- 172.21.15.40:0/1854007120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=287,monmap=2+}) -- 0x55e5492d8f50 con 0x55e5495607b0 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.119+0000 7f8571ffb640 1 -- 172.21.15.40:0/1854007120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858c048470 con 0x55e5495607b0 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f8571ffb640 1 -- 172.21.15.40:0/1854007120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 287) ==== 3494+0+0 (secure 0 0 0) 0x7f8588058b50 con 0x55e5495607b0 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f859c402740 10 client.10800 fetch_fsmap finished waiting for FSMap version 287 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f859c402740 10 client.10800 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f859c402740 20 client.10800 populate_metadata read hostname 'smithi040' 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f859c402740 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x55e549596d90 0x55e549599180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f859c402740 4 client.10800 mds_command: new command op to 10677 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f859c402740 1 -- 172.21.15.40:0/1854007120 --> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e54944e240 con 0x55e549596d90 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f8592ffd640 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x55e549596d90 0x55e549599180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f8592ffd640 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x55e549596d90 0x55e549599180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0bdf704ec7a40a server_cookie=a8c8a18774ac10e in_seq=0 out_seq=0 2024-04-26T21:45:44.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.120+0000 7f8571ffb640 10 client.10800 ms_handle_connect on v2:172.21.15.40:6832/4214786907 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f8571ffb640 1 -- 172.21.15.40:0/1854007120 <== mds.0 v2:172.21.15.40:6832/4214786907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e54944e240 con 0x55e549596d90 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f8571ffb640 10 client.10800 handle_command_reply: tid=0 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 client.10800 shutdown 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 -- 172.21.15.40:0/1854007120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 msgr2=0x55e54957e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54957e870 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f85880559c0 tx=0x7f8588062140 comp rx=0 tx=0).stop 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 -- 172.21.15.40:0/1854007120 shutdown_connections 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x55e549596d90 0x55e549599180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549575cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54957e870 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 --2- 172.21.15.40:0/1854007120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e549552190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 -- 172.21.15.40:0/1854007120 >> 172.21.15.40:0/1854007120 conn(0x55e5494cae90 msgr2=0x55e54955d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f8571ffb640 0 client.10800 ms_handle_reset on v2:172.21.15.40:6832/4214786907 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 -- 172.21.15.40:0/1854007120 shutdown_connections 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 1 -- 172.21.15.40:0/1854007120 wait complete. 2024-04-26T21:45:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.121+0000 7f859c402740 20 client.10800 trim_cache size 0 max 16384 2024-04-26T21:45:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.134+0000 7f859c402740 1 Processor -- start 2024-04-26T21:45:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.134+0000 7f859c402740 1 -- start start 2024-04-26T21:45:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f859c402740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e54959ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f859c402740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54955cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f859c402740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549571d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f859c402740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8594202860 con 0x55e5495607b0 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f859c402740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8594201bf0 con 0x55e5495723c0 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f859c402740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8594202210 con 0x55e549535cf0 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85927fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54955cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85927fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54955cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48624/0 (socket says 172.21.15.40:48624) 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85927fc640 1 -- 172.21.15.40:0/4182926834 learned_addr learned my addr 172.21.15.40:0/4182926834 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f8592ffd640 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e54959ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85937fe640 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549571d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85927fc640 1 -- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 msgr2=0x55e54959ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85927fc640 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e54959ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85927fc640 1 -- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 msgr2=0x55e549571d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85927fc640 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549571d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85927fc640 1 -- 172.21.15.40:0/4182926834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e54916c0b0 con 0x55e5495607b0 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.135+0000 7f85937fe640 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549571d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f85927fc640 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54955cd70 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f8588059260 tx=0x7f858805d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963e0bbf2df4c983 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f8573fff640 1 -- 172.21.15.40:0/4182926834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588071070 con 0x55e5495607b0 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f8573fff640 1 -- 172.21.15.40:0/4182926834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8588058b50 con 0x55e5495607b0 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f8573fff640 1 -- 172.21.15.40:0/4182926834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588055590 con 0x55e5495607b0 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 -- 172.21.15.40:0/4182926834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 msgr2=0x55e54955cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54955cd70 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f8588059260 tx=0x7f858805d000 comp rx=0 tx=0).stop 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 -- 172.21.15.40:0/4182926834 shutdown_connections 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e549571d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e54955cd70 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 --2- 172.21.15.40:0/4182926834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e54959ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 -- 172.21.15.40:0/4182926834 >> 172.21.15.40:0/4182926834 conn(0x55e5494cae90 msgr2=0x55e54957c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 -- 172.21.15.40:0/4182926834 shutdown_connections 2024-04-26T21:45:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.136+0000 7f859c402740 1 -- 172.21.15.40:0/4182926834 wait complete. 2024-04-26T21:45:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.137+0000 7f859c402740 1 -- start start 2024-04-26T21:45:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.137+0000 7f859c402740 1 Processor -- start 2024-04-26T21:45:44.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e5495f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e549589c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e54958a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e54955d3c0 con 0x55e5495607b0 2024-04-26T21:45:44.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e54958c040 con 0x55e5495723c0 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e54958bdb0 con 0x55e549535cf0 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 10 client.0 _ll_register_callbacks cb 0x7f8588058ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e549589c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e549589c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48628/0 (socket says 172.21.15.40:48628) 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 -- 172.21.15.40:0/2218640765 learned_addr learned my addr 172.21.15.40:0/2218640765 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f8592ffd640 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e5495f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85937fe640 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e54958a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 -- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 msgr2=0x55e5495f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e5495f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 -- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 msgr2=0x55e54958a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e54958a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 -- 172.21.15.40:0/2218640765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e549551b60 con 0x55e5495607b0 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85937fe640 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e54958a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:44.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f85927fc640 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e549589c50 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f85880559c0 tx=0x7f8588089c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f4dc9e3dad97dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f8571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f8571ffb640 1 -- 172.21.15.40:0/2218640765 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588071040 con 0x55e5495607b0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f8571ffb640 1 -- 172.21.15.40:0/2218640765 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f858805eaa0 con 0x55e5495607b0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 1 -- 172.21.15.40:0/2218640765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8588058b50 con 0x55e5495607b0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f8571ffb640 1 -- 172.21.15.40:0/2218640765 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=287) ==== 24+0+0 (secure 0 0 0) 0x7f858805eca0 con 0x55e5495607b0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 10 client.10806 fetch_fsmap learned FSMap version 287 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.138+0000 7f859c402740 1 -- 172.21.15.40:0/2218640765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=287,monmap=2+}) -- 0x55e5495f5ce0 con 0x55e5495607b0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f8571ffb640 1 -- 172.21.15.40:0/2218640765 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8588058b50 con 0x55e5495607b0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f8571ffb640 1 -- 172.21.15.40:0/2218640765 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 287) ==== 3494+0+0 (secure 0 0 0) 0x7f858805a890 con 0x55e5495607b0 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 10 client.10806 fetch_fsmap finished waiting for FSMap version 287 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 10 client.10806 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 20 client.10806 populate_metadata read hostname 'smithi040' 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x55e549575170 0x55e5495344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 4 client.10806 mds_command: new command op to 10677 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 -- 172.21.15.40:0/2218640765 --> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55e549577490 con 0x55e549575170 2024-04-26T21:45:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f8592ffd640 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x55e549575170 0x55e5495344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f8592ffd640 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x55e549575170 0x55e5495344b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f32a205ad9291d server_cookie=3dc165ee51114516 in_seq=0 out_seq=0 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f8571ffb640 10 client.10806 ms_handle_connect on v2:172.21.15.40:6832/4214786907 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f8571ffb640 1 -- 172.21.15.40:0/2218640765 <== mds.0 v2:172.21.15.40:6832/4214786907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+215 (crc 0 0 0) 0x55e549577490 con 0x55e549575170 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f8571ffb640 10 client.10806 handle_command_reply: tid=0 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 client.10806 shutdown 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 -- 172.21.15.40:0/2218640765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 msgr2=0x55e549589c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e549589c50 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f85880559c0 tx=0x7f8588089c30 comp rx=0 tx=0).stop 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 -- 172.21.15.40:0/2218640765 shutdown_connections 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.40:6832/4214786907,v1:172.21.15.40:6833/4214786907] conn(0x55e549575170 0x55e5495344b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5495723c0 0x55e54958a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5495607b0 0x55e549589c50 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f8571ffb640 0 client.10806 ms_handle_reset on v2:172.21.15.40:6832/4214786907 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 --2- 172.21.15.40:0/2218640765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e549535cf0 0x55e5495f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 -- 172.21.15.40:0/2218640765 >> 172.21.15.40:0/2218640765 conn(0x55e5494cae90 msgr2=0x55e54955f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.139+0000 7f859c402740 1 -- 172.21.15.40:0/2218640765 shutdown_connections 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f859c402740 1 -- 172.21.15.40:0/2218640765 wait complete. 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f859c402740 20 client.10806 trim_cache size 0 max 16384 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "dentry", 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3951947306, 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627776, 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "frag": "*", 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "sixmegs", 2024-04-26T21:45:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_id": "head", 2024-04-26T21:45:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/subdir/sixmegs" 2024-04-26T21:45:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:45:44.148 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:45:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 -- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f858006c8f0 msgr2=0x7f858006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f858006c8f0 0x7f858006edb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f858c034000 tx=0x7f858c02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 -- 172.21.15.40:0/3167927338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85940d3bc0 msgr2=0x7f85940a2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85940d3bc0 0x7f85940a2480 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f858800c430 tx=0x7f858800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 -- 172.21.15.40:0/3167927338 shutdown_connections 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f858006c8f0 0x7f858006edb0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8594162880 0x7f85941f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85940d3bc0 0x7f85940a2480 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 --2- 172.21.15.40:0/3167927338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85940b5f20 0x7f859409f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 -- 172.21.15.40:0/3167927338 >> 172.21.15.40:0/3167927338 conn(0x7f85940d4f60 msgr2=0x7f85940ba310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 -- 172.21.15.40:0/3167927338 shutdown_connections 2024-04-26T21:45:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.140+0000 7f8599609640 1 -- 172.21.15.40:0/3167927338 wait complete. 2024-04-26T21:45:44.151 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 100.00000000:stray0_head' 2024-04-26T21:45:44.152 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:44.152 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:44.167 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:44.167 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:44.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:45:44.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.363+0000 7f11121c9640 1 Processor -- start 2024-04-26T21:45:44.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.363+0000 7f11121c9640 1 -- start start 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.364+0000 7f11121c9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f110c15f3a0 0x7f110c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.364+0000 7f11121c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c0d2330 0x7f110c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.364+0000 7f11121c9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f110c0d0e20 con 0x7f110c0d1770 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.364+0000 7f11121c9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f110c0b1d70 con 0x7f110c15f3a0 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.364+0000 7f11121c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f110c0b2050 con 0x7f110c0d2330 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.364+0000 7f110b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f110c15f3a0 0x7f110c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.364+0000 7f110affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f110c0d1770 0x7f110c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37074/0 (socket says 172.21.15.40:37074) 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.364+0000 7f110affd640 1 -- 172.21.15.40:0/2722904794 learned_addr learned my addr 172.21.15.40:0/2722904794 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.365+0000 7f110bfff640 1 --2- 172.21.15.40:0/2722904794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c0d2330 0x7f110c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.365+0000 7f110b7fe640 1 -- 172.21.15.40:0/2722904794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c0d2330 msgr2=0x7f110c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.365+0000 7f110b7fe640 1 --2- 172.21.15.40:0/2722904794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c0d2330 0x7f110c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.365+0000 7f110b7fe640 1 -- 172.21.15.40:0/2722904794 >> v1:172.21.15.40:6789/0 conn(0x7f110c0d1770 legacy=0x7f110c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.365+0000 7f110b7fe640 1 -- 172.21.15.40:0/2722904794 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c0b2330 con 0x7f110c15f3a0 2024-04-26T21:45:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.365+0000 7f110b7fe640 1 --2- 172.21.15.40:0/2722904794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c15f3a0 0x7f110c0d06f0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097a0 tx=0x7f10fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800c342742395729 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f110a7fc640 1 -- 172.21.15.40:0/2722904794 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc037070 con 0x7f110c15f3a0 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f110a7fc640 1 -- 172.21.15.40:0/2722904794 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10fc032440 con 0x7f110c15f3a0 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f110a7fc640 1 -- 172.21.15.40:0/2722904794 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc032810 con 0x7f110c15f3a0 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f11121c9640 1 -- 172.21.15.40:0/2722904794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c15f3a0 msgr2=0x7f110c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f11121c9640 1 --2- 172.21.15.40:0/2722904794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c15f3a0 0x7f110c0d06f0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097a0 tx=0x7f10fc02a180 comp rx=0 tx=0).stop 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f11121c9640 1 -- 172.21.15.40:0/2722904794 shutdown_connections 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f11121c9640 1 --2- 172.21.15.40:0/2722904794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c0d2330 0x7f110c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f11121c9640 1 --2- 172.21.15.40:0/2722904794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c15f3a0 0x7f110c0d06f0 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f11121c9640 1 -- 172.21.15.40:0/2722904794 >> 172.21.15.40:0/2722904794 conn(0x7f110c0d4f90 msgr2=0x7f110c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f11121c9640 1 -- 172.21.15.40:0/2722904794 shutdown_connections 2024-04-26T21:45:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.366+0000 7f11121c9640 1 -- 172.21.15.40:0/2722904794 wait complete. 2024-04-26T21:45:44.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.367+0000 7f11121c9640 1 Processor -- start 2024-04-26T21:45:44.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f11121c9640 1 -- start start 2024-04-26T21:45:44.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f11121c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110c0d1770 0x7f110c1f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f11121c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c0d2330 0x7f110c0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f11121c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c15f3a0 0x7f110c1f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f11121c9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f110c0b2f10 con 0x7f110c0d2330 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f11121c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f110c0b2d90 con 0x7f110c0d1770 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f11121c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f110c0b3090 con 0x7f110c15f3a0 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f110affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c0d2330 0x7f110c0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f110affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c0d2330 0x7f110c0b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48648/0 (socket says 172.21.15.40:48648) 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.368+0000 7f110affd640 1 -- 172.21.15.40:0/4005297606 learned_addr learned my addr 172.21.15.40:0/4005297606 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f110bfff640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c15f3a0 0x7f110c1f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f110b7fe640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110c0d1770 0x7f110c1f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f110affd640 1 -- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c15f3a0 msgr2=0x7f110c1f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f110affd640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c15f3a0 0x7f110c1f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f110affd640 1 -- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110c0d1770 msgr2=0x7f110c1f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f110affd640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110c0d1770 0x7f110c1f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f110affd640 1 -- 172.21.15.40:0/4005297606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c0b8af0 con 0x7f110c0d2330 2024-04-26T21:45:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f110affd640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c0d2330 0x7f110c0b5c00 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f110000c430 tx=0x7f110000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f10f3fff640 1 -- 172.21.15.40:0/4005297606 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100017070 con 0x7f110c0d2330 2024-04-26T21:45:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f10f3fff640 1 -- 172.21.15.40:0/4005297606 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1100005ce0 con 0x7f110c0d2330 2024-04-26T21:45:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110c0b8dd0 con 0x7f110c0d2330 2024-04-26T21:45:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f10f3fff640 1 -- 172.21.15.40:0/4005297606 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11000072f0 con 0x7f110c0d2330 2024-04-26T21:45:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.369+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110c0b9390 con 0x7f110c0d2330 2024-04-26T21:45:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.370+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110c0694f0 con 0x7f110c0d2330 2024-04-26T21:45:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.371+0000 7f10f3fff640 1 -- 172.21.15.40:0/4005297606 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f110001a070 con 0x7f110c0d2330 2024-04-26T21:45:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.371+0000 7f10f3fff640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f10f806c8f0 0x7f10f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.371+0000 7f10f3fff640 1 -- 172.21.15.40:0/4005297606 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f110008bba0 con 0x7f110c0d2330 2024-04-26T21:45:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.371+0000 7f110b7fe640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f10f806c8f0 0x7f10f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.372+0000 7f110b7fe640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f10f806c8f0 0x7f10f806edb0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f10fc034000 tx=0x7f10fc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.376+0000 7f10f3fff640 1 -- 172.21.15.40:0/4005297606 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11000593a0 con 0x7f110c0d2330 2024-04-26T21:45:44.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.562+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f110c0b6860 con 0x7f110c0d2330 2024-04-26T21:45:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.564+0000 7f10f3fff640 1 -- 172.21.15.40:0/4005297606 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+21062 (secure 0 0 0) 0x7f1100058ca0 con 0x7f110c0d2330 2024-04-26T21:45:44.565 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:44.565 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":126,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:45:31.907943+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6832/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6833/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6833/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3538213517":"2024-04-27T21:45:17.788819+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/3538213517":"2024-04-27T21:45:17.788819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:45:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f10f806c8f0 msgr2=0x7f10f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f10f806c8f0 0x7f10f806edb0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f10fc034000 tx=0x7f10fc02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c0d2330 msgr2=0x7f110c0b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c0d2330 0x7f110c0b5c00 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f110000c430 tx=0x7f110000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 shutdown_connections 2024-04-26T21:45:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f10f806c8f0 0x7f10f806edb0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110c15f3a0 0x7f110c1f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f110c0d2330 0x7f110c0b5c00 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 --2- 172.21.15.40:0/4005297606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110c0d1770 0x7f110c1f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 >> 172.21.15.40:0/4005297606 conn(0x7f110c0d4f90 msgr2=0x7f110c0d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 shutdown_connections 2024-04-26T21:45:44.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.566+0000 7f11121c9640 1 -- 172.21.15.40:0/4005297606 wait complete. 2024-04-26T21:45:44.578 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:45:44.578 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:45:44.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.681+0000 7f0f98ac8640 1 Processor -- start 2024-04-26T21:45:44.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.681+0000 7f0f98ac8640 1 -- start start 2024-04-26T21:45:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f98ac8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f9415f170 0x7f0f940d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f98ac8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f940d3bc0 0x7f0f940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f98ac8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f940d1040 con 0x7f0f9415f550 2024-04-26T21:45:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f98ac8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0f940c9430 con 0x7f0f940d3bc0 2024-04-26T21:45:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f98ac8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0f940c9680 con 0x7f0f9415f170 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f92d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f940d3bc0 0x7f0f940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f8bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0f9415f550 0x7f0f940d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37090/0 (socket says 172.21.15.40:37090) 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f8bfff640 1 -- 172.21.15.40:0/1784627894 learned_addr learned my addr 172.21.15.40:0/1784627894 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f9259c640 1 --2- 172.21.15.40:0/1784627894 >> v2:172.21.15.184:3300/0 conn(0x7f0f9415f170 0x7f0f940d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f92d9d640 1 -- 172.21.15.40:0/1784627894 >> v2:172.21.15.184:3300/0 conn(0x7f0f9415f170 msgr2=0x7f0f940d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f92d9d640 1 --2- 172.21.15.40:0/1784627894 >> v2:172.21.15.184:3300/0 conn(0x7f0f9415f170 0x7f0f940d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f92d9d640 1 -- 172.21.15.40:0/1784627894 >> v1:172.21.15.40:6789/0 conn(0x7f0f9415f550 legacy=0x7f0f940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f92d9d640 1 -- 172.21.15.40:0/1784627894 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f940c9960 con 0x7f0f940d3bc0 2024-04-26T21:45:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.682+0000 7f0f91d9b640 1 -- 172.21.15.40:0/1784627894 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1208070736 0 0) 0x7f0f7c001110 con 0x7f0f9415f550 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.683+0000 7f0f92d9d640 1 --2- 172.21.15.40:0/1784627894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f940d3bc0 0x7f0f940d2cb0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009d90 tx=0x7f0f8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c0777ca6c88da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.683+0000 7f0f91d9b640 1 -- 172.21.15.40:0/1784627894 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c007740 con 0x7f0f940d3bc0 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.683+0000 7f0f91d9b640 1 -- 172.21.15.40:0/1784627894 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8c0078e0 con 0x7f0f940d3bc0 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.683+0000 7f0f91d9b640 1 -- 172.21.15.40:0/1784627894 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c007cb0 con 0x7f0f940d3bc0 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.684+0000 7f0f98ac8640 1 -- 172.21.15.40:0/1784627894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f940d3bc0 msgr2=0x7f0f940d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.684+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/1784627894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f940d3bc0 0x7f0f940d2cb0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009d90 tx=0x7f0f8c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.684+0000 7f0f98ac8640 1 -- 172.21.15.40:0/1784627894 shutdown_connections 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.684+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/1784627894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f940d3bc0 0x7f0f940d2cb0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.684+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/1784627894 >> v2:172.21.15.184:3300/0 conn(0x7f0f9415f170 0x7f0f940d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.684+0000 7f0f98ac8640 1 -- 172.21.15.40:0/1784627894 >> 172.21.15.40:0/1784627894 conn(0x7f0f940d4f60 msgr2=0x7f0f940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.684+0000 7f0f98ac8640 1 -- 172.21.15.40:0/1784627894 shutdown_connections 2024-04-26T21:45:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.684+0000 7f0f98ac8640 1 -- 172.21.15.40:0/1784627894 wait complete. 2024-04-26T21:45:44.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.685+0000 7f0f98ac8640 1 Processor -- start 2024-04-26T21:45:44.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.685+0000 7f0f98ac8640 1 -- start start 2024-04-26T21:45:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f98ac8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f940d3bc0 0x7f0f94152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f98ac8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9415f170 0x7f0f940b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f98ac8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f940ba0d0 0x7f0f9415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f98ac8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0f940ca5c0 con 0x7f0f9415f170 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f98ac8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f940ca440 con 0x7f0f940ba0d0 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f98ac8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f940ca740 con 0x7f0f940d3bc0 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9415f170 0x7f0f940b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9415f170 0x7f0f940b9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48658/0 (socket says 172.21.15.40:48658) 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f8bfff640 1 -- 172.21.15.40:0/2644070741 learned_addr learned my addr 172.21.15.40:0/2644070741 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f9259c640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f940d3bc0 0x7f0f94152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f92d9d640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f940ba0d0 0x7f0f9415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f8bfff640 1 -- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f940d3bc0 msgr2=0x7f0f94152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f8bfff640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f940d3bc0 0x7f0f94152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f8bfff640 1 -- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f940ba0d0 msgr2=0x7f0f9415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f8bfff640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f940ba0d0 0x7f0f9415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f8bfff640 1 -- 172.21.15.40:0/2644070741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f941f4ee0 con 0x7f0f9415f170 2024-04-26T21:45:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f9259c640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f940d3bc0 0x7f0f94152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.686+0000 7f0f92d9d640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f940ba0d0 0x7f0f9415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.687+0000 7f0f8bfff640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9415f170 0x7f0f940b9b90 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00c3e0 tx=0x7f0f7c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.687+0000 7f0f8affd640 1 -- 172.21.15.40:0/2644070741 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0074e0 con 0x7f0f9415f170 2024-04-26T21:45:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.687+0000 7f0f8affd640 1 -- 172.21.15.40:0/2644070741 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7c0047e0 con 0x7f0f9415f170 2024-04-26T21:45:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.687+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f941f51c0 con 0x7f0f9415f170 2024-04-26T21:45:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.687+0000 7f0f8affd640 1 -- 172.21.15.40:0/2644070741 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c004bd0 con 0x7f0f9415f170 2024-04-26T21:45:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.687+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f941f5780 con 0x7f0f9415f170 2024-04-26T21:45:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.688+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f940ba4e0 con 0x7f0f9415f170 2024-04-26T21:45:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.688+0000 7f0f8affd640 1 -- 172.21.15.40:0/2644070741 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f7c011070 con 0x7f0f9415f170 2024-04-26T21:45:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.689+0000 7f0f8affd640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f7406c8f0 0x7f0f7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.689+0000 7f0f8affd640 1 -- 172.21.15.40:0/2644070741 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f0f7c08c380 con 0x7f0f9415f170 2024-04-26T21:45:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.689+0000 7f0f9259c640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f7406c8f0 0x7f0f7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.689+0000 7f0f9259c640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f7406c8f0 0x7f0f7406edb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f0f84006fd0 tx=0x7f0f84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.693+0000 7f0f8affd640 1 -- 172.21.15.40:0/2644070741 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f940ba4e0 con 0x7f0f9415f170 2024-04-26T21:45:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.878+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f940694f0 con 0x7f0f9415f170 2024-04-26T21:45:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.879+0000 7f0f8affd640 1 -- 172.21.15.40:0/2644070741 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f0f940d1040 con 0x7f0f9415f170 2024-04-26T21:45:44.880 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f7406c8f0 msgr2=0x7f0f7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f7406c8f0 0x7f0f7406edb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f0f84006fd0 tx=0x7f0f84008040 comp rx=0 tx=0).stop 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9415f170 msgr2=0x7f0f940b9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9415f170 0x7f0f940b9b90 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00c3e0 tx=0x7f0f7c00c8a0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 shutdown_connections 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0f7406c8f0 0x7f0f7406edb0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f940ba0d0 0x7f0f9415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9415f170 0x7f0f940b9b90 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 --2- 172.21.15.40:0/2644070741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f940d3bc0 0x7f0f94152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 >> 172.21.15.40:0/2644070741 conn(0x7f0f940d4f60 msgr2=0x7f0f940d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 shutdown_connections 2024-04-26T21:45:44.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.881+0000 7f0f98ac8640 1 -- 172.21.15.40:0/2644070741 wait complete. 2024-04-26T21:45:44.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:45:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.995+0000 7fdddde23640 1 Processor -- start 2024-04-26T21:45:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.995+0000 7fdddde23640 1 -- start start 2024-04-26T21:45:44.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fdddde23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd815f550 0x7fddd80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fdddde23640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddd80d3bc0 0x7fddd80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:44.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fdddde23640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd80d1040 con 0x7fddd815f170 2024-04-26T21:45:44.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fdddde23640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fddd80c9430 con 0x7fddd80d3bc0 2024-04-26T21:45:44.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fdddde23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fddd80c9680 con 0x7fddd815f550 2024-04-26T21:45:44.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddd80d3bc0 0x7fddd80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fddd815f170 0x7fddd80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37094/0 (socket says 172.21.15.40:37094) 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd77fe640 1 -- 172.21.15.40:0/1853728307 learned_addr learned my addr 172.21.15.40:0/1853728307 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd6ffd640 1 --2- 172.21.15.40:0/1853728307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd815f550 0x7fddd80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd67fc640 1 -- 172.21.15.40:0/1853728307 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965940929 0 0) 0x7fddc0001110 con 0x7fddd815f170 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd67fc640 1 -- 172.21.15.40:0/1853728307 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc4003520 con 0x7fddd815f170 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd7fff640 1 -- 172.21.15.40:0/1853728307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd815f550 msgr2=0x7fddd80d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd7fff640 1 --2- 172.21.15.40:0/1853728307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd815f550 0x7fddd80d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd7fff640 1 -- 172.21.15.40:0/1853728307 >> v1:172.21.15.40:6789/0 conn(0x7fddd815f170 legacy=0x7fddd80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.996+0000 7fddd7fff640 1 -- 172.21.15.40:0/1853728307 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd80c9960 con 0x7fddd80d3bc0 2024-04-26T21:45:44.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.997+0000 7fddd7fff640 1 --2- 172.21.15.40:0/1853728307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd80d3bc0 0x7fddd80d2cb0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009d90 tx=0x7fddd002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40a41e90ab4b257 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:44.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.997+0000 7fddd67fc640 1 -- 172.21.15.40:0/1853728307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0007740 con 0x7fddd80d3bc0 2024-04-26T21:45:44.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.997+0000 7fddd67fc640 1 -- 172.21.15.40:0/1853728307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd00078e0 con 0x7fddd80d3bc0 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.997+0000 7fddd67fc640 1 -- 172.21.15.40:0/1853728307 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0007c90 con 0x7fddd80d3bc0 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.998+0000 7fdddde23640 1 -- 172.21.15.40:0/1853728307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd80d3bc0 msgr2=0x7fddd80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.998+0000 7fdddde23640 1 --2- 172.21.15.40:0/1853728307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd80d3bc0 0x7fddd80d2cb0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009d90 tx=0x7fddd002a660 comp rx=0 tx=0).stop 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.998+0000 7fdddde23640 1 -- 172.21.15.40:0/1853728307 shutdown_connections 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.998+0000 7fdddde23640 1 --2- 172.21.15.40:0/1853728307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd80d3bc0 0x7fddd80d2cb0 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.998+0000 7fdddde23640 1 --2- 172.21.15.40:0/1853728307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd815f550 0x7fddd80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.998+0000 7fdddde23640 1 -- 172.21.15.40:0/1853728307 >> 172.21.15.40:0/1853728307 conn(0x7fddd80d4f60 msgr2=0x7fddd80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.998+0000 7fdddde23640 1 -- 172.21.15.40:0/1853728307 shutdown_connections 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.998+0000 7fdddde23640 1 -- 172.21.15.40:0/1853728307 wait complete. 2024-04-26T21:45:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.999+0000 7fdddde23640 1 Processor -- start 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:44.999+0000 7fdddde23640 1 -- start start 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fdddde23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd80d3bc0 0x7fddd8157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fdddde23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd815f170 0x7fddd81f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fdddde23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddd81595f0 0x7fddd8159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fdddde23640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80ca5c0 con 0x7fddd815f170 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fdddde23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80ca440 con 0x7fddd81595f0 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fdddde23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fddd80ca740 con 0x7fddd80d3bc0 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd815f170 0x7fddd81f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd815f170 0x7fddd81f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48674/0 (socket says 172.21.15.40:48674) 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 -- 172.21.15.40:0/2446320040 learned_addr learned my addr 172.21.15.40:0/2446320040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd77fe640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd80d3bc0 0x7fddd8157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 -- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd80d3bc0 msgr2=0x7fddd8157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd80d3bc0 0x7fddd8157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 -- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddd81595f0 msgr2=0x7fddd8159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddd81595f0 0x7fddd8159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 -- 172.21.15.40:0/2446320040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd80cbfa0 con 0x7fddd815f170 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd77fe640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd80d3bc0 0x7fddd8157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.000+0000 7fddd6ffd640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd815f170 0x7fddd81f4b40 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fddc800e870 tx=0x7fddc800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.001+0000 7fddcffff640 1 -- 172.21.15.40:0/2446320040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8017070 con 0x7fddd815f170 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.001+0000 7fddcffff640 1 -- 172.21.15.40:0/2446320040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc800c940 con 0x7fddd815f170 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.001+0000 7fddcffff640 1 -- 172.21.15.40:0/2446320040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc800cd30 con 0x7fddd815f170 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.001+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd80cc280 con 0x7fddd815f170 2024-04-26T21:45:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.001+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd80cc840 con 0x7fddd815f170 2024-04-26T21:45:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.002+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd80694f0 con 0x7fddd815f170 2024-04-26T21:45:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.002+0000 7fddcffff640 1 -- 172.21.15.40:0/2446320040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddc801a070 con 0x7fddd815f170 2024-04-26T21:45:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.003+0000 7fddcffff640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddb806c8f0 0x7fddb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.003+0000 7fddcffff640 1 -- 172.21.15.40:0/2446320040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fddc808ca30 con 0x7fddd815f170 2024-04-26T21:45:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.003+0000 7fddd77fe640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddb806c8f0 0x7fddb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.003+0000 7fddd77fe640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddb806c8f0 0x7fddb806edb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fddc00012b0 tx=0x7fddc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:45.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.007+0000 7fddcffff640 1 -- 172.21.15.40:0/2446320040 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddc805a1d0 con 0x7fddd815f170 2024-04-26T21:45:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.196+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fddd815dc10 con 0x7fddd815f170 2024-04-26T21:45:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.196+0000 7fddcffff640 1 -- 172.21.15.40:0/2446320040 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) ==== 104+0+11 (secure 0 0 0) 0x7fddc8059ad0 con 0x7fddd815f170 2024-04-26T21:45:45.197 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:45:45.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddb806c8f0 msgr2=0x7fddb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddb806c8f0 0x7fddb806edb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fddc00012b0 tx=0x7fddc0009040 comp rx=0 tx=0).stop 2024-04-26T21:45:45.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd815f170 msgr2=0x7fddd81f4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd815f170 0x7fddd81f4b40 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fddc800e870 tx=0x7fddc800ed30 comp rx=0 tx=0).stop 2024-04-26T21:45:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 shutdown_connections 2024-04-26T21:45:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fddb806c8f0 0x7fddb806edb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddd81595f0 0x7fddd8159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd815f170 0x7fddd81f4b40 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 --2- 172.21.15.40:0/2446320040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddd80d3bc0 0x7fddd8157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 >> 172.21.15.40:0/2446320040 conn(0x7fddd80d4f60 msgr2=0x7fddd80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 shutdown_connections 2024-04-26T21:45:45.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.198+0000 7fdddde23640 1 -- 172.21.15.40:0/2446320040 wait complete. 2024-04-26T21:45:45.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:45:45.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.314+0000 7ff7afbb4640 1 Processor -- start 2024-04-26T21:45:45.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.314+0000 7ff7afbb4640 1 -- start start 2024-04-26T21:45:45.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.315+0000 7ff7afbb4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7a80b40d0 0x7ff7a81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.315+0000 7ff7afbb4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7a8162b30 0x7ff7a80b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.315+0000 7ff7afbb4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a80d0a10 con 0x7ff7a80d1a00 2024-04-26T21:45:45.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.315+0000 7ff7afbb4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a80d0cf0 con 0x7ff7a80b40d0 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.315+0000 7ff7afbb4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a80d0fd0 con 0x7ff7a8162b30 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.315+0000 7ff7ad14f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7a80b40d0 0x7ff7a81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.315+0000 7ff7ad14f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7a80b40d0 0x7ff7a81625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48684/0 (socket says 172.21.15.40:48684) 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.315+0000 7ff7ad14f640 1 -- 172.21.15.40:0/2714340627 learned_addr learned my addr 172.21.15.40:0/2714340627 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.316+0000 7ff7ae151640 1 --2- 172.21.15.40:0/2714340627 >> v2:172.21.15.184:3300/0 conn(0x7ff7a8162b30 0x7ff7a80b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.316+0000 7ff7ad14f640 1 -- 172.21.15.40:0/2714340627 >> v2:172.21.15.184:3300/0 conn(0x7ff7a8162b30 msgr2=0x7ff7a80b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.316+0000 7ff7ad14f640 1 --2- 172.21.15.40:0/2714340627 >> v2:172.21.15.184:3300/0 conn(0x7ff7a8162b30 0x7ff7a80b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.316+0000 7ff7ad14f640 1 -- 172.21.15.40:0/2714340627 >> v1:172.21.15.184:6789/0 conn(0x7ff7a80d1a00 legacy=0x7ff7a80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.316+0000 7ff7ad14f640 1 -- 172.21.15.40:0/2714340627 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a8162f80 con 0x7ff7a80b40d0 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.316+0000 7ff7ae151640 1 --2- 172.21.15.40:0/2714340627 >> v2:172.21.15.184:3300/0 conn(0x7ff7a8162b30 0x7ff7a80b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.316+0000 7ff7ad14f640 1 --2- 172.21.15.40:0/2714340627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a81625f0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80b69a0 tx=0x7ff79c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85df3407ea29c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7ac94e640 1 -- 172.21.15.40:0/2714340627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c037070 con 0x7ff7a80b40d0 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7ac94e640 1 -- 172.21.15.40:0/2714340627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c032440 con 0x7ff7a80b40d0 2024-04-26T21:45:45.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7ac94e640 1 -- 172.21.15.40:0/2714340627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c032810 con 0x7ff7a80b40d0 2024-04-26T21:45:45.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7afbb4640 1 -- 172.21.15.40:0/2714340627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 msgr2=0x7ff7a81625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/2714340627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a81625f0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80b69a0 tx=0x7ff79c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7afbb4640 1 -- 172.21.15.40:0/2714340627 shutdown_connections 2024-04-26T21:45:45.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/2714340627 >> v2:172.21.15.184:3300/0 conn(0x7ff7a8162b30 0x7ff7a80b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/2714340627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a81625f0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7afbb4640 1 -- 172.21.15.40:0/2714340627 >> 172.21.15.40:0/2714340627 conn(0x7ff7a80d4fd0 msgr2=0x7ff7a80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7afbb4640 1 -- 172.21.15.40:0/2714340627 shutdown_connections 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.317+0000 7ff7afbb4640 1 -- 172.21.15.40:0/2714340627 wait complete. 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.318+0000 7ff7afbb4640 1 Processor -- start 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.318+0000 7ff7afbb4640 1 -- start start 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7afbb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a80b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7afbb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7a80d1a00 0x7ff7a80ba800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7afbb4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7a8162b30 0x7ff7a814f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7afbb4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a80d2670 con 0x7ff7a80b40d0 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7afbb4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a80d24f0 con 0x7ff7a8162b30 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7afbb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a80b5040 con 0x7ff7a80d1a00 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad950640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a80b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad950640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a80b7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48686/0 (socket says 172.21.15.40:48686) 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad950640 1 -- 172.21.15.40:0/1454541517 learned_addr learned my addr 172.21.15.40:0/1454541517 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad14f640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7a80d1a00 0x7ff7a80ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ae151640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7a8162b30 0x7ff7a814f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad950640 1 -- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7a80d1a00 msgr2=0x7ff7a80ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad950640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7a80d1a00 0x7ff7a80ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad950640 1 -- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7a8162b30 msgr2=0x7ff7a814f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad950640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7a8162b30 0x7ff7a814f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.319+0000 7ff7ad950640 1 -- 172.21.15.40:0/1454541517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a80bad40 con 0x7ff7a80b40d0 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.320+0000 7ff7ad14f640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7a80d1a00 0x7ff7a80ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.320+0000 7ff7ae151640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7a8162b30 0x7ff7a814f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.320+0000 7ff7ad950640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a80b7d80 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000c460 tx=0x7ff7a000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.320+0000 7ff7a67fc640 1 -- 172.21.15.40:0/1454541517 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0016020 con 0x7ff7a80b40d0 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.320+0000 7ff7a67fc640 1 -- 172.21.15.40:0/1454541517 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a0012870 con 0x7ff7a80b40d0 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.320+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a80cd290 con 0x7ff7a80b40d0 2024-04-26T21:45:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.320+0000 7ff7a67fc640 1 -- 172.21.15.40:0/1454541517 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0012c60 con 0x7ff7a80b40d0 2024-04-26T21:45:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.320+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a80cd850 con 0x7ff7a80b40d0 2024-04-26T21:45:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.321+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a80694f0 con 0x7ff7a80b40d0 2024-04-26T21:45:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.321+0000 7ff7a67fc640 1 -- 172.21.15.40:0/1454541517 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7a0019020 con 0x7ff7a80b40d0 2024-04-26T21:45:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.322+0000 7ff7a67fc640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff79406c8f0 0x7ff79406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.322+0000 7ff7a67fc640 1 -- 172.21.15.40:0/1454541517 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7ff7a008c530 con 0x7ff7a80b40d0 2024-04-26T21:45:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.322+0000 7ff7ad14f640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff79406c8f0 0x7ff79406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.322+0000 7ff7ad14f640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff79406c8f0 0x7ff79406edb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff79c034000 tx=0x7ff79c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:45.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.326+0000 7ff7a67fc640 1 -- 172.21.15.40:0/1454541517 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7a0020030 con 0x7ff7a80b40d0 2024-04-26T21:45:45.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.512+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7a814fb20 con 0x7ff7a80b40d0 2024-04-26T21:45:45.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.512+0000 7ff7a67fc640 1 -- 172.21.15.40:0/1454541517 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) ==== 100+0+11 (secure 0 0 0) 0x7ff7a0059ca0 con 0x7ff7a80b40d0 2024-04-26T21:45:45.513 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:45:45.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff79406c8f0 msgr2=0x7ff79406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff79406c8f0 0x7ff79406edb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff79c034000 tx=0x7ff79c02e000 comp rx=0 tx=0).stop 2024-04-26T21:45:45.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 msgr2=0x7ff7a80b7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a80b7d80 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000c460 tx=0x7ff7a000c920 comp rx=0 tx=0).stop 2024-04-26T21:45:45.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 shutdown_connections 2024-04-26T21:45:45.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff79406c8f0 0x7ff79406edb0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7a8162b30 0x7ff7a814f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7a80d1a00 0x7ff7a80ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 --2- 172.21.15.40:0/1454541517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7a80b40d0 0x7ff7a80b7d80 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 >> 172.21.15.40:0/1454541517 conn(0x7ff7a80d4fd0 msgr2=0x7ff7a80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:45.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 shutdown_connections 2024-04-26T21:45:45.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.514+0000 7ff7afbb4640 1 -- 172.21.15.40:0/1454541517 wait complete. 2024-04-26T21:45:45.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:45:45.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.626+0000 7f4746551640 1 Processor -- start 2024-04-26T21:45:45.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.626+0000 7f4746551640 1 -- start start 2024-04-26T21:45:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.627+0000 7f4746551640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4740162880 0x7f47400d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.627+0000 7f4746551640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f47400d3bc0 0x7f47400b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.627+0000 7f4746551640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47400b7e50 con 0x7f47400b5f20 2024-04-26T21:45:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.627+0000 7f4746551640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f47400d0580 con 0x7f4740162880 2024-04-26T21:45:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.627+0000 7f4746551640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f47400d1800 con 0x7f47400d3bc0 2024-04-26T21:45:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.627+0000 7f473ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4740162880 0x7f47400d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.627+0000 7f473f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f47400b5f20 0x7f47400d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37096/0 (socket says 172.21.15.40:37096) 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.627+0000 7f473f7fe640 1 -- 172.21.15.40:0/232913560 learned_addr learned my addr 172.21.15.40:0/232913560 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.628+0000 7f4744aee640 1 --2- 172.21.15.40:0/232913560 >> v2:172.21.15.184:3300/0 conn(0x7f47400d3bc0 0x7f47400b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.628+0000 7f473ffff640 1 -- 172.21.15.40:0/232913560 >> v2:172.21.15.184:3300/0 conn(0x7f47400d3bc0 msgr2=0x7f47400b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.628+0000 7f473ffff640 1 --2- 172.21.15.40:0/232913560 >> v2:172.21.15.184:3300/0 conn(0x7f47400d3bc0 0x7f47400b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.628+0000 7f473ffff640 1 -- 172.21.15.40:0/232913560 >> v1:172.21.15.40:6789/0 conn(0x7f47400b5f20 legacy=0x7f47400d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.628+0000 7f473ffff640 1 -- 172.21.15.40:0/232913560 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47400d07d0 con 0x7f4740162880 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.628+0000 7f473effd640 1 -- 172.21.15.40:0/232913560 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2736733721 0 0) 0x7f47400b7e50 con 0x7f47400b5f20 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.628+0000 7f4744aee640 1 --2- 172.21.15.40:0/232913560 >> v2:172.21.15.184:3300/0 conn(0x7f47400d3bc0 0x7f47400b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.628+0000 7f473ffff640 1 --2- 172.21.15.40:0/232913560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4740162880 0x7f47400d2cb0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f47400d11d0 tx=0x7f472802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f511ef2b04ac406 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f473effd640 1 -- 172.21.15.40:0/232913560 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728037070 con 0x7f4740162880 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f473effd640 1 -- 172.21.15.40:0/232913560 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4728032440 con 0x7f4740162880 2024-04-26T21:45:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f473effd640 1 -- 172.21.15.40:0/232913560 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728032810 con 0x7f4740162880 2024-04-26T21:45:45.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f4746551640 1 -- 172.21.15.40:0/232913560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4740162880 msgr2=0x7f47400d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f4746551640 1 --2- 172.21.15.40:0/232913560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4740162880 0x7f47400d2cb0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f47400d11d0 tx=0x7f472802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f4746551640 1 -- 172.21.15.40:0/232913560 shutdown_connections 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f4746551640 1 --2- 172.21.15.40:0/232913560 >> v2:172.21.15.184:3300/0 conn(0x7f47400d3bc0 0x7f47400b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f4746551640 1 --2- 172.21.15.40:0/232913560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4740162880 0x7f47400d2cb0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f4746551640 1 -- 172.21.15.40:0/232913560 >> 172.21.15.40:0/232913560 conn(0x7f47400d4f60 msgr2=0x7f47400d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f4746551640 1 -- 172.21.15.40:0/232913560 shutdown_connections 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.629+0000 7f4746551640 1 -- 172.21.15.40:0/232913560 wait complete. 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.630+0000 7f4746551640 1 Processor -- start 2024-04-26T21:45:45.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f4746551640 1 -- start start 2024-04-26T21:45:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f4746551640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47400b5f20 0x7f47400ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f4746551640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47400d3bc0 0x7f47400ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f4746551640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4740162880 0x7f47400b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f4746551640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4740162c60 con 0x7f47400d3bc0 2024-04-26T21:45:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f4746551640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47400d0d90 con 0x7f47400b5f20 2024-04-26T21:45:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f4746551640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4740162de0 con 0x7f4740162880 2024-04-26T21:45:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f473f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47400d3bc0 0x7f47400ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f473f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47400d3bc0 0x7f47400ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48710/0 (socket says 172.21.15.40:48710) 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.631+0000 7f473f7fe640 1 -- 172.21.15.40:0/77699315 learned_addr learned my addr 172.21.15.40:0/77699315 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473ffff640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47400b5f20 0x7f47400ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f4744aee640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4740162880 0x7f47400b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473f7fe640 1 -- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4740162880 msgr2=0x7f47400b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473f7fe640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4740162880 0x7f47400b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473f7fe640 1 -- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47400b5f20 msgr2=0x7f47400ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473f7fe640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47400b5f20 0x7f47400ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473f7fe640 1 -- 172.21.15.40:0/77699315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4740154940 con 0x7f47400d3bc0 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473ffff640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47400b5f20 0x7f47400ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473f7fe640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47400d3bc0 0x7f47400ce6b0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f473000c420 tx=0x7f473000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473cff9640 1 -- 172.21.15.40:0/77699315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47300074e0 con 0x7f47400d3bc0 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473cff9640 1 -- 172.21.15.40:0/77699315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47300047e0 con 0x7f47400d3bc0 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47400b4840 con 0x7f47400d3bc0 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f473cff9640 1 -- 172.21.15.40:0/77699315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730004c10 con 0x7f47400d3bc0 2024-04-26T21:45:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.632+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47400b3090 con 0x7f47400d3bc0 2024-04-26T21:45:45.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.633+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47400694f0 con 0x7f47400d3bc0 2024-04-26T21:45:45.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.634+0000 7f473cff9640 1 -- 172.21.15.40:0/77699315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4730011070 con 0x7f47400d3bc0 2024-04-26T21:45:45.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.634+0000 7f473cff9640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472006c8f0 0x7f472006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:45.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.634+0000 7f473cff9640 1 -- 172.21.15.40:0/77699315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7f473008c350 con 0x7f47400d3bc0 2024-04-26T21:45:45.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.634+0000 7f473ffff640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472006c8f0 0x7f472006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:45.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.635+0000 7f473ffff640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472006c8f0 0x7f472006edb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f4728034000 tx=0x7f472802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:45.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.639+0000 7f473cff9640 1 -- 172.21.15.40:0/77699315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f473000e280 con 0x7f47400d3bc0 2024-04-26T21:45:45.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.837+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4740074590 con 0x7f47400d3bc0 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.838+0000 7f473cff9640 1 -- 172.21.15.40:0/77699315 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 62 entries v126) ==== 81+0+3772 (secure 0 0 0) 0x7f4730059b50 con 0x7f47400d3bc0 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2896027915 2024-04-27T21:45:31.907924+0000 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2896027915 2024-04-27T21:45:31.907924+0000 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4173136579 2024-04-27T21:45:27.871638+0000 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:45:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3791539151 2024-04-27T21:45:23.843779+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4192917874 2024-04-27T21:45:19.814510+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:45:45.840 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:45:45.841 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:45:45.841 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:45:45.841 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3791539151 2024-04-27T21:45:23.843779+0000 2024-04-26T21:45:45.841 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:45:45.841 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:45:45.841 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:45:45.841 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3538213517 2024-04-27T21:45:17.788819+0000 2024-04-26T21:45:45.841 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4192917874 2024-04-27T21:45:19.814510+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:45:45.842 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4173136579 2024-04-27T21:45:27.871638+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:45:45.843 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:45:45.844 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:45:45.844 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:45:45.844 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:45:45.844 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:45:45.844 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3538213517 2024-04-27T21:45:17.788819+0000 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:listed 62 entries 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472006c8f0 msgr2=0x7f472006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472006c8f0 0x7f472006edb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f4728034000 tx=0x7f472802e000 comp rx=0 tx=0).stop 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47400d3bc0 msgr2=0x7f47400ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47400d3bc0 0x7f47400ce6b0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f473000c420 tx=0x7f473000c8e0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 shutdown_connections 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f472006c8f0 0x7f472006edb0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4740162880 0x7f47400b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47400d3bc0 0x7f47400ce6b0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 --2- 172.21.15.40:0/77699315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47400b5f20 0x7f47400ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 >> 172.21.15.40:0/77699315 conn(0x7f47400d4f60 msgr2=0x7f47400d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:45.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 shutdown_connections 2024-04-26T21:45:45.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:45.840+0000 7f4746551640 1 -- 172.21.15.40:0/77699315 wait complete. 2024-04-26T21:45:45.851 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:45:45.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:45.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:45:45.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:45.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:45.877 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:45.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:46.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.074+0000 7fdec7738640 1 Processor -- start 2024-04-26T21:45:46.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.074+0000 7fdec7738640 1 -- start start 2024-04-26T21:45:46.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec7738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec0162810 0x7fdec0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec7738640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdec00b2340 0x7fdec00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec7738640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec00d10d0 con 0x7fdec00cd930 2024-04-26T21:45:46.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec7738640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdec00d0750 con 0x7fdec00b2340 2024-04-26T21:45:46.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec7738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdec00d16a0 con 0x7fdec0162810 2024-04-26T21:45:46.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec4cd3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdec00b2340 0x7fdec00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec5cd5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdec00cd930 0x7fdec00d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37098/0 (socket says 172.21.15.40:37098) 2024-04-26T21:45:46.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec5cd5640 1 -- 172.21.15.40:0/3714149428 learned_addr learned my addr 172.21.15.40:0/3714149428 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec4cd3640 1 --2- 172.21.15.40:0/3714149428 >> v2:172.21.15.40:3300/0 conn(0x7fdec00b2340 0x7fdec00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48720/0 (socket says 172.21.15.40:48720) 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec4cd3640 1 -- 172.21.15.40:0/3714149428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec0162810 msgr2=0x7fdec0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec54d4640 1 --2- 172.21.15.40:0/3714149428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec0162810 0x7fdec0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdebffff640 1 -- 172.21.15.40:0/3714149428 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 53307700 0 0) 0x7fdec00d10d0 con 0x7fdec00cd930 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec4cd3640 1 --2- 172.21.15.40:0/3714149428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec0162810 0x7fdec0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec4cd3640 1 -- 172.21.15.40:0/3714149428 >> v1:172.21.15.40:6789/0 conn(0x7fdec00cd930 legacy=0x7fdec00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.075+0000 7fdec4cd3640 1 -- 172.21.15.40:0/3714149428 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec00d1370 con 0x7fdec00b2340 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.076+0000 7fdec54d4640 1 --2- 172.21.15.40:0/3714149428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec0162810 0x7fdec0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.076+0000 7fdec4cd3640 1 --2- 172.21.15.40:0/3714149428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec00b2340 0x7fdec00cd3f0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40097a0 tx=0x7fdeb402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18bbfe4159a8b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.076+0000 7fdebffff640 1 -- 172.21.15.40:0/3714149428 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4037070 con 0x7fdec00b2340 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.076+0000 7fdebffff640 1 -- 172.21.15.40:0/3714149428 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4032440 con 0x7fdec00b2340 2024-04-26T21:45:46.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.076+0000 7fdebffff640 1 -- 172.21.15.40:0/3714149428 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4032810 con 0x7fdec00b2340 2024-04-26T21:45:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.077+0000 7fdec7738640 1 -- 172.21.15.40:0/3714149428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec00b2340 msgr2=0x7fdec00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.077+0000 7fdec7738640 1 --2- 172.21.15.40:0/3714149428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec00b2340 0x7fdec00cd3f0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40097a0 tx=0x7fdeb402a180 comp rx=0 tx=0).stop 2024-04-26T21:45:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.077+0000 7fdec7738640 1 -- 172.21.15.40:0/3714149428 shutdown_connections 2024-04-26T21:45:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.077+0000 7fdec7738640 1 --2- 172.21.15.40:0/3714149428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec00b2340 0x7fdec00cd3f0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.077+0000 7fdec7738640 1 --2- 172.21.15.40:0/3714149428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec0162810 0x7fdec0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.077+0000 7fdec7738640 1 -- 172.21.15.40:0/3714149428 >> 172.21.15.40:0/3714149428 conn(0x7fdec00d2da0 msgr2=0x7fdec00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.077+0000 7fdec7738640 1 -- 172.21.15.40:0/3714149428 shutdown_connections 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.077+0000 7fdec7738640 1 -- 172.21.15.40:0/3714149428 wait complete. 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.078+0000 7fdec7738640 1 Processor -- start 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.078+0000 7fdec7738640 1 -- start start 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec7738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec00b2340 0x7fdec015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec7738640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec00cd930 0x7fdec015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec7738640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec0162810 0x7fdec00c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec7738640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0070890 con 0x7fdec0162810 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec7738640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdec00d0470 con 0x7fdec00cd930 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec7738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdec00d2240 con 0x7fdec00b2340 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec0162810 0x7fdec00c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec0162810 0x7fdec00c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48732/0 (socket says 172.21.15.40:48732) 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 -- 172.21.15.40:0/2056478448 learned_addr learned my addr 172.21.15.40:0/2056478448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 -- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec00b2340 msgr2=0x7fdec015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec00b2340 0x7fdec015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 -- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec00cd930 msgr2=0x7fdec015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec00cd930 0x7fdec015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 -- 172.21.15.40:0/2056478448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec0160990 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.079+0000 7fdec5cd5640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec0162810 0x7fdec00c8ca0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c760 tx=0x7fdeb000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.080+0000 7fdebdffb640 1 -- 172.21.15.40:0/2056478448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0018070 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.080+0000 7fdebdffb640 1 -- 172.21.15.40:0/2056478448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb000a640 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.080+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec0160c70 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.080+0000 7fdebdffb640 1 -- 172.21.15.40:0/2056478448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000aa30 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.080+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec0161230 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.081+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec014ceb0 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.081+0000 7fdebdffb640 1 -- 172.21.15.40:0/2056478448 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdeb001b070 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.082+0000 7fdebdffb640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.082+0000 7fdebdffb640 1 -- 172.21.15.40:0/2056478448 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8883+0+0 (secure 0 0 0) 0x7fdeb008bd70 con 0x7fdec0162810 2024-04-26T21:45:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.082+0000 7fdec54d4640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.083+0000 7fdec54d4640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdeac06c8f0 0x7fdeac06edb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8006fd0 tx=0x7fdeb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:46.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.086+0000 7fdebdffb640 1 -- 172.21.15.40:0/2056478448 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdec014ceb0 con 0x7fdec0162810 2024-04-26T21:45:46.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.310+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdec00694f0 con 0x7fdec0162810 2024-04-26T21:45:46.311 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:46.311+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:45:46.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.312+0000 7fdebdffb640 1 -- 172.21.15.40:0/2056478448 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v288) ==== 161+0+0 (secure 0 0 0) 0x7fdeb00594e0 con 0x7fdec0162810 2024-04-26T21:45:46.313 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdeac06c8f0 msgr2=0x7fdeac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdeac06c8f0 0x7fdeac06edb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8006fd0 tx=0x7fdeb8008040 comp rx=0 tx=0).stop 2024-04-26T21:45:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec0162810 msgr2=0x7fdec00c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec0162810 0x7fdec00c8ca0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c760 tx=0x7fdeb000cc20 comp rx=0 tx=0).stop 2024-04-26T21:45:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 shutdown_connections 2024-04-26T21:45:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec0162810 0x7fdec00c8ca0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec00cd930 0x7fdec015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 --2- 172.21.15.40:0/2056478448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdec00b2340 0x7fdec015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 >> 172.21.15.40:0/2056478448 conn(0x7fdec00d2da0 msgr2=0x7fdec00b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:46.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 shutdown_connections 2024-04-26T21:45:46.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.316+0000 7fdec7738640 1 -- 172.21.15.40:0/2056478448 wait complete. 2024-04-26T21:45:46.318 INFO:tasks.ceph.mds.c.smithi040.stderr: -570> 2024-04-26T21:45:42.918+0000 7f3961e8c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:45:46.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:46.342 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:45:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.429+0000 7fc731af1640 1 Processor -- start 2024-04-26T21:45:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.429+0000 7fc731af1640 1 -- start start 2024-04-26T21:45:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.430+0000 7fc731af1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc72c0b5f20 0x7fc72c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.430+0000 7fc731af1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72c0b7e50 con 0x7fc72c0d3bc0 2024-04-26T21:45:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.430+0000 7fc731af1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72c0d1800 con 0x7fc72c162880 2024-04-26T21:45:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.430+0000 7fc731af1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc72c0d0580 con 0x7fc72c0b5f20 2024-04-26T21:45:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.430+0000 7fc72bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc72c0b5f20 0x7fc72c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.430+0000 7fc72affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc72c0d3bc0 0x7fc72c0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37108/0 (socket says 172.21.15.40:37108) 2024-04-26T21:45:46.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.430+0000 7fc72bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc72c0b5f20 0x7fc72c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48758/0 (socket says 172.21.15.40:48758) 2024-04-26T21:45:46.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.430+0000 7fc72affd640 1 -- 172.21.15.40:0/1666616780 learned_addr learned my addr 172.21.15.40:0/1666616780 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:46.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.431+0000 7fc72bfff640 1 -- 172.21.15.40:0/1666616780 >> v1:172.21.15.184:6789/0 conn(0x7fc72c162880 legacy=0x7fc72c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.431+0000 7fc72bfff640 1 -- 172.21.15.40:0/1666616780 >> v1:172.21.15.40:6789/0 conn(0x7fc72c0d3bc0 legacy=0x7fc72c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.431+0000 7fc72bfff640 1 -- 172.21.15.40:0/1666616780 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c0d07d0 con 0x7fc72c0b5f20 2024-04-26T21:45:46.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.431+0000 7fc72a7fc640 1 -- 172.21.15.40:0/1666616780 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3687705480 0 0) 0x7fc72c0b7e50 con 0x7fc72c0d3bc0 2024-04-26T21:45:46.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.431+0000 7fc72bfff640 1 --2- 172.21.15.40:0/1666616780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0d2cb0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fc724009d90 tx=0x7fc72402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfe0e1c02ab26bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:46.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.431+0000 7fc72a7fc640 1 -- 172.21.15.40:0/1666616780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724007740 con 0x7fc72c0b5f20 2024-04-26T21:45:46.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.431+0000 7fc72a7fc640 1 -- 172.21.15.40:0/1666616780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7240078e0 con 0x7fc72c0b5f20 2024-04-26T21:45:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.431+0000 7fc72a7fc640 1 -- 172.21.15.40:0/1666616780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724007cb0 con 0x7fc72c0b5f20 2024-04-26T21:45:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.432+0000 7fc731af1640 1 -- 172.21.15.40:0/1666616780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 msgr2=0x7fc72c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.432+0000 7fc731af1640 1 --2- 172.21.15.40:0/1666616780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0d2cb0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fc724009d90 tx=0x7fc72402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.432+0000 7fc731af1640 1 -- 172.21.15.40:0/1666616780 shutdown_connections 2024-04-26T21:45:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.432+0000 7fc731af1640 1 --2- 172.21.15.40:0/1666616780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0d2cb0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.432+0000 7fc731af1640 1 -- 172.21.15.40:0/1666616780 >> 172.21.15.40:0/1666616780 conn(0x7fc72c0d4f60 msgr2=0x7fc72c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.432+0000 7fc731af1640 1 -- 172.21.15.40:0/1666616780 shutdown_connections 2024-04-26T21:45:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.432+0000 7fc731af1640 1 -- 172.21.15.40:0/1666616780 wait complete. 2024-04-26T21:45:46.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.433+0000 7fc731af1640 1 Processor -- start 2024-04-26T21:45:46.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.433+0000 7fc731af1640 1 -- start start 2024-04-26T21:45:46.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc731af1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc731af1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc72c0d3bc0 0x7fc72c1f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc731af1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc72c162880 0x7fc72c1f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc731af1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c162c60 con 0x7fc72c0b5f20 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc731af1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c0d0d90 con 0x7fc72c162880 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc731af1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc72c162de0 con 0x7fc72c0d3bc0 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc72b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc72b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48762/0 (socket says 172.21.15.40:48762) 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc72b7fe640 1 -- 172.21.15.40:0/2377400320 learned_addr learned my addr 172.21.15.40:0/2377400320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc72affd640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc72c0d3bc0 0x7fc72c1f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc72bfff640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc72c162880 0x7fc72c1f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc72b7fe640 1 -- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc72c0d3bc0 msgr2=0x7fc72c1f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc72b7fe640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc72c0d3bc0 0x7fc72c1f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.434+0000 7fc72b7fe640 1 -- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc72c162880 msgr2=0x7fc72c1f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.435+0000 7fc72b7fe640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc72c162880 0x7fc72c1f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.435+0000 7fc72b7fe640 1 -- 172.21.15.40:0/2377400320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c1f4b80 con 0x7fc72c0b5f20 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.435+0000 7fc72b7fe640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0b2560 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c430 tx=0x7fc71c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.435+0000 7fc713fff640 1 -- 172.21.15.40:0/2377400320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c017070 con 0x7fc72c0b5f20 2024-04-26T21:45:46.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.435+0000 7fc713fff640 1 -- 172.21.15.40:0/2377400320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c0052d0 con 0x7fc72c0b5f20 2024-04-26T21:45:46.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.435+0000 7fc713fff640 1 -- 172.21.15.40:0/2377400320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c003830 con 0x7fc72c0b5f20 2024-04-26T21:45:46.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.435+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c1f8c40 con 0x7fc72c0b5f20 2024-04-26T21:45:46.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.435+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c1f4df0 con 0x7fc72c0b5f20 2024-04-26T21:45:46.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.436+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72c0694f0 con 0x7fc72c0b5f20 2024-04-26T21:45:46.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.436+0000 7fc713fff640 1 -- 172.21.15.40:0/2377400320 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc71c01a070 con 0x7fc72c0b5f20 2024-04-26T21:45:46.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.437+0000 7fc713fff640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:46.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.437+0000 7fc713fff640 1 -- 172.21.15.40:0/2377400320 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8969+0+0 (secure 0 0 0) 0x7fc71c08bb70 con 0x7fc72c0b5f20 2024-04-26T21:45:46.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.437+0000 7fc72affd640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:46.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.437+0000 7fc72affd640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71806c8f0 0x7fc71806edb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0b5cb0 tx=0x7fc720007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:46.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.441+0000 7fc713fff640 1 -- 172.21.15.40:0/2377400320 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc72c0d1800 con 0x7fc72c0b5f20 2024-04-26T21:45:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:46.704+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc72c0cf390 con 0x7fc72c0b5f20 2024-04-26T21:45:47.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.314+0000 7fc713fff640 1 -- 172.21.15.40:0/2377400320 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v289) ==== 107+0+0 (secure 0 0 0) 0x7fc71c0592c0 con 0x7fc72c0b5f20 2024-04-26T21:45:47.315 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:45:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71806c8f0 msgr2=0x7fc71806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71806c8f0 0x7fc71806edb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0b5cb0 tx=0x7fc720007b90 comp rx=0 tx=0).stop 2024-04-26T21:45:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 msgr2=0x7fc72c0b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0b2560 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c430 tx=0x7fc71c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 shutdown_connections 2024-04-26T21:45:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc72c162880 0x7fc72c1f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc72c0d3bc0 0x7fc72c1f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 --2- 172.21.15.40:0/2377400320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc72c0b5f20 0x7fc72c0b2560 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 >> 172.21.15.40:0/2377400320 conn(0x7fc72c0d4f60 msgr2=0x7fc72c0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 shutdown_connections 2024-04-26T21:45:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.317+0000 7fc731af1640 1 -- 172.21.15.40:0/2377400320 wait complete. 2024-04-26T21:45:47.331 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:47.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.416+0000 7f7bd8b1cf40 1 Processor -- start 2024-04-26T21:45:47.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd8b1cf40 1 -- start start 2024-04-26T21:45:47.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd8b1cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466d14640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd8b1cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466ceb850 0x559466db2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd8b1cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466dbf830 0x559466dbde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd8b1cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559466dbfc10 con 0x559466ce8930 2024-04-26T21:45:47.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd8b1cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559466c91e40 con 0x559466ceb850 2024-04-26T21:45:47.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd8b1cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559466dc00f0 con 0x559466dbf830 2024-04-26T21:45:47.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd5b4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466d14640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd5b4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466d14640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48786/0 (socket says 172.21.15.40:48786) 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.417+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/512948170 learned_addr learned my addr 172.21.15.40:0/512948170 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd5349640 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466ceb850 0x559466db2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd634b640 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466dbf830 0x559466dbde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466dbf830 msgr2=0x559466dbde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466dbf830 0x559466dbde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466ceb850 msgr2=0x559466db2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466ceb850 0x559466db2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/512948170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559466d30d40 con 0x559466ce8930 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd5349640 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466ceb850 0x559466db2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466d14640 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8001ce0 tx=0x7f7bc800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28faa4c71ac623db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bc7fff640 1 -- 172.21.15.40:0/512948170 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800bd80 con 0x559466ce8930 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bc7fff640 1 -- 172.21.15.40:0/512948170 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc800f400 con 0x559466ce8930 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.418+0000 7f7bc7fff640 1 -- 172.21.15.40:0/512948170 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800f830 con 0x559466ce8930 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/512948170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 msgr2=0x559466d14640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466d14640 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8001ce0 tx=0x7f7bc800b5b0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/512948170 shutdown_connections 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466dbf830 0x559466dbde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466ceb850 0x559466db2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/512948170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466d14640 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/512948170 >> 172.21.15.40:0/512948170 conn(0x559466ce9330 msgr2=0x559466dbe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/512948170 shutdown_connections 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/512948170 wait complete. 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 Processor -- start 2024-04-26T21:45:47.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.419+0000 7f7bd8b1cf40 1 -- start start 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd8b1cf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466db22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd8b1cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466ceb850 0x559466daf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd8b1cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466dbf830 0x559466e55e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd8b1cf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559466d33b90 con 0x559466ce8930 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd8b1cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559466d30aa0 con 0x559466dbf830 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd8b1cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559466d33e30 con 0x559466ceb850 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5b4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466db22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5b4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466db22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48792/0 (socket says 172.21.15.40:48792) 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 learned_addr learned my addr 172.21.15.40:0/3090945136 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd634b640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466dbf830 0x559466e55e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5349640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466ceb850 0x559466daf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466ceb850 msgr2=0x559466daf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466ceb850 0x559466daf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466dbf830 msgr2=0x559466e55e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559466dbf830 0x559466e55e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559466db3e40 con 0x559466ce8930 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.420+0000 7f7bd5349640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559466ceb850 0x559466daf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.421+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466db22b0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x559466d14410 tx=0x7f7bc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.421+0000 7f7bc6ffd640 1 -- 172.21.15.40:0/3090945136 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8020070 con 0x559466ce8930 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.421+0000 7f7bc6ffd640 1 -- 172.21.15.40:0/3090945136 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc801c440 con 0x559466ce8930 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.421+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559466db40e0 con 0x559466ce8930 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.421+0000 7f7bc6ffd640 1 -- 172.21.15.40:0/3090945136 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc801c850 con 0x559466ce8930 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.421+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559466db0000 con 0x559466ce8930 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.422+0000 7f7bc6ffd640 1 -- 172.21.15.40:0/3090945136 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bc8023070 con 0x559466ce8930 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.423+0000 7f7bc6ffd640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bb806b2d0 0x7f7bb806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.423+0000 7f7bc6ffd640 1 -- 172.21.15.40:0/3090945136 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8969+0+0 (secure 0 0 0) 0x7f7bc808ccf0 con 0x559466ce8930 2024-04-26T21:45:47.427 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.423+0000 7f7bd5349640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bb806b2d0 0x7f7bb806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.423+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559466e56360 0x559466db0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.423+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466db0a60 con 0x559466e56360 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.423+0000 7f7bd634b640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559466e56360 0x559466db0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.424+0000 7f7bd5349640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bb806b2d0 0x7f7bb806d790 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc057000 tx=0x7f7bcc052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.424+0000 7f7bd634b640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559466e56360 0x559466db0520 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.426+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v127'971 uv944 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.426+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e62460 con 0x559466e56360 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.427+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v127'972 uv972 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.427+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e62800 con 0x559466e56360 2024-04-26T21:45:47.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.428+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v127'973 uv973 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.429 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:47.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.428+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e63bb0 con 0x559466e56360 2024-04-26T21:45:47.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.429+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v127'974 uv974 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.429+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e63f50 con 0x559466e56360 2024-04-26T21:45:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.430+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v127'975 uv975 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.430+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e62ba0 con 0x559466e56360 2024-04-26T21:45:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.431+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v127'976 uv976 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.431+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e62f40 con 0x559466e56360 2024-04-26T21:45:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.432+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v127'977 uv950 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.433+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.434+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v127'978 uv978 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.434+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.435 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:47.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.435+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v127'979 uv979 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.435+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.436+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v127'980 uv980 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.436+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.437+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v127'981 uv981 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.437+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.438+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v127'982 uv982 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.438+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.440+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v127'983 uv970 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.440+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.441+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v127'984 uv984 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.441+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.442+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v127'985 uv985 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.443 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.442+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.443+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v127'986 uv986 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.443+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.444+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v127'987 uv987 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.444+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.445+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v127'988 uv988 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.445+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.447+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v127'989 uv989 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.447+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.448+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v127'990 uv963 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.448+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.449+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v127'991 uv991 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.449+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.451 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:47.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.450+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v127'992 uv992 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.450+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.451+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v127'993 uv993 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.451+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.452+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v127'994 uv994 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.452+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.453+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v127'995 uv995 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.453+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.455+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v127'996 uv969 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.455+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.456+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v127'997 uv997 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.456+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.457 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:47.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.457+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v127'998 uv998 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.457+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.458+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v127'999 uv999 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.458+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.459+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v127'1000 uv1000 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.459+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e56360 2024-04-26T21:45:47.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.460+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v127'1001 uv1001 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e56360 2024-04-26T21:45:47.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.460+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559466e636c0 0x559466e66810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.460+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e632e0 con 0x559466e636c0 2024-04-26T21:45:47.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.460+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559466e636c0 0x559466e66810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.461+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559466e636c0 0x559466e66810 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.463+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v127'705 uv684 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.463+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e66d50 con 0x559466e636c0 2024-04-26T21:45:47.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.464+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v127'706 uv706 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.464+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e670f0 con 0x559466e636c0 2024-04-26T21:45:47.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.465+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v127'707 uv707 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.466 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:47.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.465+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e67940 con 0x559466e636c0 2024-04-26T21:45:47.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.467+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v127'708 uv708 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.467+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e67ce0 con 0x559466e636c0 2024-04-26T21:45:47.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.468+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v127'709 uv709 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.468+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e68080 con 0x559466e636c0 2024-04-26T21:45:47.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.470+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v127'710 uv703 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.470+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e68420 con 0x559466e636c0 2024-04-26T21:45:47.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.471+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v127'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.471+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.473 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:47.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.472+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v127'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.472+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.474+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v127'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.474+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.475+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v127'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.475+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.476+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v127'715 uv704 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.476+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.477+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v127'716 uv716 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.477+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.479 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:47.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.479+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v127'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.479+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6ce80 con 0x559466e636c0 2024-04-26T21:45:47.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.480+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v127'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.480+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6ce80 con 0x559466e636c0 2024-04-26T21:45:47.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.481+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v127'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.481+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.483+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v127'720 uv700 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.483+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.484+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v127'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.484+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.486 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:47.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.485+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v127'722 uv722 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.485+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e697d0 con 0x559466e636c0 2024-04-26T21:45:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.486+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v127'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.486+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e697d0 con 0x559466e636c0 2024-04-26T21:45:47.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.488+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v127'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.488+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e687c0 con 0x559466e636c0 2024-04-26T21:45:47.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.489+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v127'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e636c0 2024-04-26T21:45:47.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.489+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559466e68e40 0x559466e6b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.489+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6b7b0 con 0x559466e68e40 2024-04-26T21:45:47.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.489+0000 7f7bd5349640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559466e68e40 0x559466e6b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.490+0000 7f7bd5349640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559466e68e40 0x559466e6b230 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.491+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v127'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.491+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e6bca0 con 0x559466e68e40 2024-04-26T21:45:47.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.492+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v127'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.492+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6c180 con 0x559466e68e40 2024-04-26T21:45:47.494 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.493+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v127'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.493+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6da30 con 0x559466e68e40 2024-04-26T21:45:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.494+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v127'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.494+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6ddd0 con 0x559466e68e40 2024-04-26T21:45:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.495+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v127'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.496+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6ca20 con 0x559466e68e40 2024-04-26T21:45:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.497+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v127'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.497+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559466e6d320 0x559466e70180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.497+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e706c0 con 0x559466e6d320 2024-04-26T21:45:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.497+0000 7f7bd634b640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559466e6d320 0x559466e70180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.497+0000 7f7bd634b640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559466e6d320 0x559466e70180 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.499+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v127'145 uv144 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.499+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e706c0 con 0x559466e6d320 2024-04-26T21:45:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.500+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v127'146 uv146 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.500+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e70740 con 0x559466e6d320 2024-04-26T21:45:47.501 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.501+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v127'147 uv147 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.501+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e70b30 con 0x559466e6d320 2024-04-26T21:45:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.502+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v127'148 uv148 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.502+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559466e71040 0x559466e73430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.502+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e739b0 con 0x559466e71040 2024-04-26T21:45:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.502+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559466e71040 0x559466e73430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.503+0000 7f7bd5b4a640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559466e71040 0x559466e73430 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.504+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v127'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.504+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e73a70 con 0x559466e71040 2024-04-26T21:45:47.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.505+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v127'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.505+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.507+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v127'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.508 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:47.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.507+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e74ea0 con 0x559466e71040 2024-04-26T21:45:47.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.508+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v127'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.508+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e74ea0 con 0x559466e71040 2024-04-26T21:45:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.509+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v127'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.509+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.510+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v127'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.510+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.512+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v127'349 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.512+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.513+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v127'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.513+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.514+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v127'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.515 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:47.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.514+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e74ea0 con 0x559466e6d320 2024-04-26T21:45:47.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.515+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v127'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.515+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e74ea0 con 0x559466e6d320 2024-04-26T21:45:47.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.516+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v127'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.516+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.517+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v127'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.517+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.518+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v127'355 uv348 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.518+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.519+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v127'356 uv356 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.519+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.521+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v127'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.522 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:47.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.521+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.522+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v127'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.522+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.523+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v127'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.523+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.524+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v127'158 uv157 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.524+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.525+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v127'159 uv159 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.525+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.526+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v127'160 uv160 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.528 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:47.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.526+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.527+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v127'161 uv161 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.527+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e6d320 2024-04-26T21:45:47.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.528+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v127'162 uv162 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7bd000e960 con 0x559466e6d320 2024-04-26T21:45:47.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.528+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.529+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v127'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.529+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.530+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v127'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.530+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.532 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:47.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.531+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v127'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.531+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e74ea0 con 0x559466e71040 2024-04-26T21:45:47.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.532+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v127'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.532+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e74ea0 con 0x559466e71040 2024-04-26T21:45:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.533+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v127'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.533+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.534+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v127'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.534+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.535+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v127'314 uv308 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.535+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.536+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v127'315 uv315 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.536+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e73e90 con 0x559466e71040 2024-04-26T21:45:47.538 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:47.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.537+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v127'316 uv316 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.537+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6c610 con 0x559466e71040 2024-04-26T21:45:47.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.538+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v127'317 uv317 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.538+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6c610 con 0x559466e71040 2024-04-26T21:45:47.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.539+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v127'318 uv318 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.539+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6c610 con 0x559466e71040 2024-04-26T21:45:47.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.540+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v127'319 uv313 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.540+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e6c610 con 0x559466e71040 2024-04-26T21:45:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.541+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v127'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.541+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6c610 con 0x559466e71040 2024-04-26T21:45:47.543 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:47.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.542+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v127'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.542+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e6c610 con 0x559466e71040 2024-04-26T21:45:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.543+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v127'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.543+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e76ce0 con 0x559466e71040 2024-04-26T21:45:47.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.545+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v127'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.545+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559466e749c0 0x559466e76db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.545+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e773a0 con 0x559466e749c0 2024-04-26T21:45:47.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.545+0000 7f7bd5349640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559466e749c0 0x559466e76db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.546+0000 7f7bd5349640 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559466e749c0 0x559466e76db0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.547+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v127'137 uv135 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e749c0 2024-04-26T21:45:47.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.547+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e77740 con 0x559466e749c0 2024-04-26T21:45:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.548+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v127'138 uv138 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e749c0 2024-04-26T21:45:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.548+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77740 con 0x559466e749c0 2024-04-26T21:45:47.552 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:47.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.551+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v127'139 uv139 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e749c0 2024-04-26T21:45:47.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.551+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e749c0 2024-04-26T21:45:47.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.552+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v127'140 uv140 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e749c0 2024-04-26T21:45:47.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.552+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e71040 2024-04-26T21:45:47.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.554+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v127'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.554+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e71040 2024-04-26T21:45:47.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.555+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v127'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.555+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e71040 2024-04-26T21:45:47.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.556+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v127'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.557 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:47.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.556+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e78b40 con 0x559466e71040 2024-04-26T21:45:47.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.557+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v127'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.557+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e78b40 con 0x559466e71040 2024-04-26T21:45:47.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.558+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v127'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.559+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e71040 2024-04-26T21:45:47.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.560+0000 7f7bd5b4a640 1 -- 172.21.15.40:0/3090945136 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v127'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bc802c070 con 0x559466e71040 2024-04-26T21:45:47.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.560+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.562+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v127'161 uv160 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.562+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.562+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v127'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.562+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.564 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:47.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.563+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v127'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.563+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.564+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v127'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.564+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.565+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v127'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.566+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.567+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v127'389 uv380 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.567+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.568+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v127'390 uv390 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.568+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.569+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v127'391 uv391 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.570 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:47.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.569+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.570+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v127'392 uv392 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.570+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.571+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v127'393 uv393 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.571+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.572+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v127'394 uv387 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.572+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.573+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v127'395 uv395 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.573+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.574+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v127'396 uv396 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.575 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:47.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.574+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e78b40 con 0x559466e68e40 2024-04-26T21:45:47.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.575+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v127'397 uv397 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.575+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e78b40 con 0x559466e68e40 2024-04-26T21:45:47.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.576+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v127'398 uv398 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.576+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77b30 con 0x559466e68e40 2024-04-26T21:45:47.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.577+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v127'399 uv399 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.577+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x559466e77fe0 con 0x559466e68e40 2024-04-26T21:45:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd5349640 1 -- 172.21.15.40:0/3090945136 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v127'400 uv400 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7bcc053ac0 con 0x559466e68e40 2024-04-26T21:45:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559466e636c0 msgr2=0x559466e66810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x559466e636c0 0x559466e66810 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559466e56360 msgr2=0x559466db0520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559466e56360 0x559466db0520 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559466e6d320 msgr2=0x559466e70180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x559466e6d320 0x559466e70180 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559466e749c0 msgr2=0x559466e76db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x559466e749c0 0x559466e76db0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559466e68e40 msgr2=0x559466e6b230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x559466e68e40 0x559466e6b230 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559466e71040 msgr2=0x559466e73430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x559466e71040 0x559466e73430 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 reap_dead start 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.578+0000 7f7bd634b640 1 -- 172.21.15.40:0/3090945136 reap_dead start 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bb806b2d0 msgr2=0x7f7bb806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bb806b2d0 0x7f7bb806d790 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc057000 tx=0x7f7bcc052000 comp rx=0 tx=0).stop 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 msgr2=0x559466db22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466db22b0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x559466d14410 tx=0x7f7bc8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 shutdown_connections 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7bb806b2d0 0x7f7bb806d790 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 --2- 172.21.15.40:0/3090945136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559466ce8930 0x559466db22b0 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 >> 172.21.15.40:0/3090945136 conn(0x559466ce9330 msgr2=0x559466d14c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 shutdown_connections 2024-04-26T21:45:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.579+0000 7f7bd8b1cf40 1 -- 172.21.15.40:0/3090945136 wait complete. 2024-04-26T21:45:47.583 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 100.00000000 stray0_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:45:47.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.627+0000 7f3b29c0ef40 1 Processor -- start 2024-04-26T21:45:47.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.627+0000 7f3b29c0ef40 1 -- start start 2024-04-26T21:45:47.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.627+0000 7f3b29c0ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb954d0c80 0x55bb9550e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b29c0ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 0x55bb955a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b29c0ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb9559ee70 0x55bb955b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b29c0ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb954e5c40 con 0x55bb9559ee70 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b29c0ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bb9548ae40 con 0x55bb95511f00 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b29c0ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bb9552b990 con 0x55bb954d0c80 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27f09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb9559ee70 0x55bb955b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27708640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb954d0c80 0x55bb9550e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b26f07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 0x55bb955a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27708640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb954d0c80 0x55bb9550e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:34830/0 (socket says 172.21.15.40:34830) 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27f09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb9559ee70 0x55bb955b7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48804/0 (socket says 172.21.15.40:48804) 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27708640 1 -- 172.21.15.40:0/2984766629 learned_addr learned my addr 172.21.15.40:0/2984766629 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27f09640 1 -- 172.21.15.40:0/2984766629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb954d0c80 msgr2=0x55bb9550e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27f09640 1 --2- 172.21.15.40:0/2984766629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb954d0c80 0x55bb9550e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27f09640 1 -- 172.21.15.40:0/2984766629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 msgr2=0x55bb955a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27f09640 1 --2- 172.21.15.40:0/2984766629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 0x55bb955a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27f09640 1 -- 172.21.15.40:0/2984766629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9550eb80 con 0x55bb9559ee70 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b27f09640 1 --2- 172.21.15.40:0/2984766629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb9559ee70 0x55bb955b7e70 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f3b20053000 tx=0x7f3b2004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efb1522b066cf9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b25f05640 1 -- 172.21.15.40:0/2984766629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20065070 con 0x55bb9559ee70 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b25f05640 1 -- 172.21.15.40:0/2984766629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b200548b0 con 0x55bb9559ee70 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.628+0000 7f3b25f05640 1 -- 172.21.15.40:0/2984766629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20059040 con 0x55bb9559ee70 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/2984766629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb9559ee70 msgr2=0x55bb955b7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 --2- 172.21.15.40:0/2984766629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb9559ee70 0x55bb955b7e70 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f3b20053000 tx=0x7f3b2004e000 comp rx=0 tx=0).stop 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/2984766629 shutdown_connections 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 --2- 172.21.15.40:0/2984766629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb9559ee70 0x55bb955b7e70 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 --2- 172.21.15.40:0/2984766629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 0x55bb955a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 --2- 172.21.15.40:0/2984766629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb954d0c80 0x55bb9550e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/2984766629 >> 172.21.15.40:0/2984766629 conn(0x55bb954e2860 msgr2=0x55bb955a1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/2984766629 shutdown_connections 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/2984766629 wait complete. 2024-04-26T21:45:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.629+0000 7f3b29c0ef40 1 Processor -- start 2024-04-26T21:45:47.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b29c0ef40 1 -- start start 2024-04-26T21:45:47.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b29c0ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb954d0c80 0x55bb955b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b29c0ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 0x55bb955a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b29c0ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb9559ee70 0x55bb955b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b29c0ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bb9559b5e0 con 0x55bb954d0c80 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b29c0ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bb9550e1b0 con 0x55bb95511f00 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b29c0ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55bb9559e4c0 con 0x55bb9559ee70 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b27708640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb954d0c80 0x55bb955b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b27708640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb954d0c80 0x55bb955b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48818/0 (socket says 172.21.15.40:48818) 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b27708640 1 -- 172.21.15.40:0/841649099 learned_addr learned my addr 172.21.15.40:0/841649099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b27f09640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb9559ee70 0x55bb955b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.630+0000 7f3b26f07640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 0x55bb955a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b27708640 1 -- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb9559ee70 msgr2=0x55bb955b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b27708640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55bb9559ee70 0x55bb955b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b27708640 1 -- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 msgr2=0x55bb955a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b27708640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb95511f00 0x55bb955a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b27708640 1 -- 172.21.15.40:0/841649099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb954e3de0 con 0x55bb954d0c80 2024-04-26T21:45:47.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b27708640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb954d0c80 0x55bb955b51c0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f3b18010930 tx=0x7f3b18010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b24f03640 1 -- 172.21.15.40:0/841649099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18017030 con 0x55bb954d0c80 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b24f03640 1 -- 172.21.15.40:0/841649099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b18002a70 con 0x55bb954d0c80 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b24f03640 1 -- 172.21.15.40:0/841649099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18002c20 con 0x55bb954d0c80 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bb954e3b00 con 0x55bb954d0c80 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.631+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb954e45c0 con 0x55bb954d0c80 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.632+0000 7f3b24f03640 1 -- 172.21.15.40:0/841649099 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b1801a030 con 0x55bb954d0c80 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.633+0000 7f3b24f03640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1406b2d0 0x7f3b1406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.633+0000 7f3b24f03640 1 -- 172.21.15.40:0/841649099 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8969+0+0 (secure 0 0 0) 0x7f3b1808ba00 con 0x55bb954d0c80 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.633+0000 7f3b26f07640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1406b2d0 0x7f3b1406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.633+0000 7f3b29c0ef40 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bb955b5760 0x55bb955b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.633+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55bb955b60e0 con 0x55bb955b5760 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.633+0000 7f3b27f09640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bb955b5760 0x55bb955b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.633+0000 7f3b26f07640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1406b2d0 0x7f3b1406d790 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x55bb95522e20 tx=0x7f3b1c001570 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.634+0000 7f3b27f09640 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bb955b5760 0x55bb955b5ba0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b27f09640 1 -- 172.21.15.40:0/841649099 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 100.00000000 [omap-set-vals] v127'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3b20069040 con 0x55bb955b5760 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bb955b5760 msgr2=0x55bb955b5ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55bb955b5760 0x55bb955b5ba0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1406b2d0 msgr2=0x7f3b1406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3b1406b2d0 0x7f3b1406d790 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x55bb95522e20 tx=0x7f3b1c001570 comp rx=0 tx=0).stop 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb954d0c80 msgr2=0x55bb955b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 --2- 172.21.15.40:0/841649099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bb954d0c80 0x55bb955b51c0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f3b18010930 tx=0x7f3b18010df0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b27f09640 1 -- 172.21.15.40:0/841649099 reap_dead start 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 shutdown_connections 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 >> 172.21.15.40:0/841649099 conn(0x55bb954e2860 msgr2=0x55bb955117f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 shutdown_connections 2024-04-26T21:45:47.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.635+0000 7f3b29c0ef40 1 -- 172.21.15.40:0/841649099 wait complete. 2024-04-26T21:45:47.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.743+0000 7fc06436a640 1 Processor -- start 2024-04-26T21:45:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.743+0000 7fc06436a640 1 -- start start 2024-04-26T21:45:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.744+0000 7fc06436a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc05c15f170 0x7fc05c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.744+0000 7fc06436a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05c0d1040 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.744+0000 7fc06436a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05c0c9680 con 0x7fc05c15f550 2024-04-26T21:45:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.744+0000 7fc06436a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc05c0c9430 con 0x7fc05c15f170 2024-04-26T21:45:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.744+0000 7fc062106640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc05c15f170 0x7fc05c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.744+0000 7fc062907640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc05c0d3bc0 0x7fc05c0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37136/0 (socket says 172.21.15.40:37136) 2024-04-26T21:45:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.744+0000 7fc062907640 1 -- 172.21.15.40:0/1352510332 learned_addr learned my addr 172.21.15.40:0/1352510332 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.745+0000 7fc062106640 1 -- 172.21.15.40:0/1352510332 >> v1:172.21.15.184:6789/0 conn(0x7fc05c15f550 legacy=0x7fc05c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.745+0000 7fc062106640 1 -- 172.21.15.40:0/1352510332 >> v1:172.21.15.40:6789/0 conn(0x7fc05c0d3bc0 legacy=0x7fc05c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.745+0000 7fc062106640 1 -- 172.21.15.40:0/1352510332 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c0c9960 con 0x7fc05c15f170 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.745+0000 7fc061104640 1 -- 172.21.15.40:0/1352510332 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235080608 0 0) 0x7fc054001110 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.745+0000 7fc062106640 1 --2- 172.21.15.40:0/1352510332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c15f170 0x7fc05c0d5a80 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0c95b0 tx=0x7fc05002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5e5e583b91b2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.745+0000 7fc061104640 1 -- 172.21.15.40:0/1352510332 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050007740 con 0x7fc05c15f170 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.745+0000 7fc061104640 1 -- 172.21.15.40:0/1352510332 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0500078e0 con 0x7fc05c15f170 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.745+0000 7fc061104640 1 -- 172.21.15.40:0/1352510332 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050007cb0 con 0x7fc05c15f170 2024-04-26T21:45:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.746+0000 7fc06436a640 1 -- 172.21.15.40:0/1352510332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c15f170 msgr2=0x7fc05c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.746+0000 7fc06436a640 1 --2- 172.21.15.40:0/1352510332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c15f170 0x7fc05c0d5a80 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0c95b0 tx=0x7fc05002a710 comp rx=0 tx=0).stop 2024-04-26T21:45:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.746+0000 7fc06436a640 1 -- 172.21.15.40:0/1352510332 shutdown_connections 2024-04-26T21:45:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.746+0000 7fc06436a640 1 --2- 172.21.15.40:0/1352510332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c15f170 0x7fc05c0d5a80 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.746+0000 7fc06436a640 1 -- 172.21.15.40:0/1352510332 >> 172.21.15.40:0/1352510332 conn(0x7fc05c0d4f60 msgr2=0x7fc05c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.746+0000 7fc06436a640 1 -- 172.21.15.40:0/1352510332 shutdown_connections 2024-04-26T21:45:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.746+0000 7fc06436a640 1 -- 172.21.15.40:0/1352510332 wait complete. 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.747+0000 7fc06436a640 1 Processor -- start 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.747+0000 7fc06436a640 1 -- start start 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc06436a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c0d3bc0 0x7fc05c14e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc06436a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc05c15f170 0x7fc05c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc06436a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc05c0cdf70 0x7fc05c0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc06436a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c0d2e30 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc06436a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c0d2cb0 con 0x7fc05c15f170 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc06436a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc05c0d2fb0 con 0x7fc05c0cdf70 2024-04-26T21:45:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062106640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c0d3bc0 0x7fc05c14e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062106640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c0d3bc0 0x7fc05c14e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48832/0 (socket says 172.21.15.40:48832) 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062106640 1 -- 172.21.15.40:0/1346296449 learned_addr learned my addr 172.21.15.40:0/1346296449 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062907640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc05c0cdf70 0x7fc05c0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc061905640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc05c15f170 0x7fc05c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062106640 1 -- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc05c0cdf70 msgr2=0x7fc05c0cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062106640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc05c0cdf70 0x7fc05c0cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062106640 1 -- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc05c15f170 msgr2=0x7fc05c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062106640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc05c15f170 0x7fc05c0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.748+0000 7fc062106640 1 -- 172.21.15.40:0/1346296449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c0ccdb0 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.749+0000 7fc062106640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c0d3bc0 0x7fc05c14e420 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fc050000f80 tx=0x7fc050004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.749+0000 7fc04effd640 1 -- 172.21.15.40:0/1346296449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050040070 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.749+0000 7fc04effd640 1 -- 172.21.15.40:0/1346296449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc050031d00 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.749+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05c0cd090 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.749+0000 7fc04effd640 1 -- 172.21.15.40:0/1346296449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050003b90 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.749+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05c14e960 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.750+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05c0694f0 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.750+0000 7fc04effd640 1 -- 172.21.15.40:0/1346296449 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc050039040 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.751+0000 7fc04effd640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc04006c8f0 0x7fc04006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.751+0000 7fc04effd640 1 -- 172.21.15.40:0/1346296449 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8969+0+0 (secure 0 0 0) 0x7fc0500adb50 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.751+0000 7fc061905640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc04006c8f0 0x7fc04006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.751+0000 7fc061905640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc04006c8f0 0x7fc04006edb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0a2de0 tx=0x7fc058009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.755+0000 7fc04effd640 1 -- 172.21.15.40:0/1346296449 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc05007ac20 con 0x7fc05c0d3bc0 2024-04-26T21:45:47.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:47.980+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc05c0ce350 con 0x7fc05c0d3bc0 2024-04-26T21:45:48.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.710+0000 7fc04effd640 1 -- 172.21.15.40:0/1346296449 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v290) ==== 162+0+0 (secure 0 0 0) 0x7fc05007ef20 con 0x7fc05c0d3bc0 2024-04-26T21:45:48.711 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.712+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc04006c8f0 msgr2=0x7fc04006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc04006c8f0 0x7fc04006edb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0a2de0 tx=0x7fc058009040 comp rx=0 tx=0).stop 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c0d3bc0 msgr2=0x7fc05c14e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c0d3bc0 0x7fc05c14e420 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fc050000f80 tx=0x7fc050004a40 comp rx=0 tx=0).stop 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 shutdown_connections 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc04006c8f0 0x7fc04006edb0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc05c0cdf70 0x7fc05c0cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc05c15f170 0x7fc05c0cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 --2- 172.21.15.40:0/1346296449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc05c0d3bc0 0x7fc05c14e420 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 >> 172.21.15.40:0/1346296449 conn(0x7fc05c0d4f60 msgr2=0x7fc05c0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 shutdown_connections 2024-04-26T21:45:48.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.713+0000 7fc06436a640 1 -- 172.21.15.40:0/1346296449 wait complete. 2024-04-26T21:45:48.729 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:48.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.835+0000 7fd28f771640 1 Processor -- start 2024-04-26T21:45:48.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.836+0000 7fd28f771640 1 -- start start 2024-04-26T21:45:48.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.836+0000 7fd28f771640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd28815f3a0 0x7fd2880d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:48.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.836+0000 7fd28f771640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2880d0e20 con 0x7fd2880d2300 2024-04-26T21:45:48.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.836+0000 7fd28f771640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2880b1d70 con 0x7fd2880d1770 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.836+0000 7fd28f771640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2880b2050 con 0x7fd28815f3a0 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd28dd0e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd2880d2300 0x7fd2880b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42208/0 (socket says 172.21.15.40:42208) 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd28dd0e640 1 -- 172.21.15.40:0/593197504 learned_addr learned my addr 172.21.15.40:0/593197504 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd28d50d640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd28815f3a0 0x7fd2880d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622272358 0 0) 0x7fd2880d0e20 con 0x7fd2880d2300 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd274003520 con 0x7fd2880d2300 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4229491682 0 0) 0x7fd274003520 con 0x7fd2880d2300 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd28815f3a0 msgr2=0x7fd2880d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd28815f3a0 0x7fd2880d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 >> v1:172.21.15.184:6789/0 conn(0x7fd2880d1770 legacy=0x7fd2880d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2880b2330 con 0x7fd2880d2300 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fd2800019f0 con 0x7fd2880d2300 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 >> v1:172.21.15.40:6789/0 conn(0x7fd2880d2300 legacy=0x7fd2880b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2740039e0 0x7fd274003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd28815f3a0 0x7fd274004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd274004d40 0x7fd2740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd2880b2050 con 0x7fd2740039e0 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd27400ba70 con 0x7fd274004d40 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd27400bc70 con 0x7fd28815f3a0 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.837+0000 7fd28d50d640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2740039e0 0x7fd274003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.838+0000 7fd28cd0c640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd28815f3a0 0x7fd274004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.838+0000 7fd28dd0e640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd274004d40 0x7fd2740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.838+0000 7fd28d50d640 1 -- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd28815f3a0 msgr2=0x7fd274004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.838+0000 7fd28d50d640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd28815f3a0 0x7fd274004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.838+0000 7fd28d50d640 1 -- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd274004d40 msgr2=0x7fd2740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.838+0000 7fd28d50d640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd274004d40 0x7fd2740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.838+0000 7fd28d50d640 1 -- 172.21.15.40:0/593197504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd274003520 con 0x7fd2740039e0 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd28d50d640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2740039e0 0x7fd274003e00 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fd284009a90 tx=0x7fd274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd9c717e7d89a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284036070 con 0x7fd2740039e0 2024-04-26T21:45:48.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd284031400 con 0x7fd2740039e0 2024-04-26T21:45:48.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd27ffff640 1 -- 172.21.15.40:0/593197504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284009690 con 0x7fd2740039e0 2024-04-26T21:45:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd28f771640 1 -- 172.21.15.40:0/593197504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2740039e0 msgr2=0x7fd274003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd28f771640 1 --2- 172.21.15.40:0/593197504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2740039e0 0x7fd274003e00 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fd284009a90 tx=0x7fd274005830 comp rx=0 tx=0).stop 2024-04-26T21:45:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd28dd0e640 1 -- 172.21.15.40:0/593197504 reap_dead start 2024-04-26T21:45:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd28f771640 1 -- 172.21.15.40:0/593197504 shutdown_connections 2024-04-26T21:45:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd28f771640 1 -- 172.21.15.40:0/593197504 >> 172.21.15.40:0/593197504 conn(0x7fd2880d4f90 msgr2=0x7fd2880d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd28f771640 1 -- 172.21.15.40:0/593197504 shutdown_connections 2024-04-26T21:45:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.839+0000 7fd28f771640 1 -- 172.21.15.40:0/593197504 wait complete. 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28f771640 1 Processor -- start 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28f771640 1 -- start start 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28f771640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2740039e0 0x7fd28815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28f771640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2880b7b80 0x7fd2880c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28f771640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28815ffd0 0x7fd2880b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28f771640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd2880d0870 con 0x7fd28815ffd0 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28f771640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2880d06f0 con 0x7fd2880b7b80 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28f771640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2880d09f0 con 0x7fd2740039e0 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.841+0000 7fd28dd0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28815ffd0 0x7fd2880b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28dd0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28815ffd0 0x7fd2880b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55460/0 (socket says 172.21.15.40:55460) 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28d50d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2740039e0 0x7fd28815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28dd0e640 1 -- 172.21.15.40:0/767044451 learned_addr learned my addr 172.21.15.40:0/767044451 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28cd0c640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2880b7b80 0x7fd2880c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28dd0e640 1 -- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2740039e0 msgr2=0x7fd28815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28dd0e640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2740039e0 0x7fd28815c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28dd0e640 1 -- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2880b7b80 msgr2=0x7fd2880c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28dd0e640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2880b7b80 0x7fd2880c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28dd0e640 1 -- 172.21.15.40:0/767044451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2880c9340 con 0x7fd28815ffd0 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28dd0e640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28815ffd0 0x7fd2880b4a80 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fd280002230 tx=0x7fd28000a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd27dffb640 1 -- 172.21.15.40:0/767044451 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28000acb0 con 0x7fd28815ffd0 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd27dffb640 1 -- 172.21.15.40:0/767044451 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd280018070 con 0x7fd28815ffd0 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2880c9620 con 0x7fd28815ffd0 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd27dffb640 1 -- 172.21.15.40:0/767044451 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280013630 con 0x7fd28815ffd0 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.842+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2880ca0c0 con 0x7fd28815ffd0 2024-04-26T21:45:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.844+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2880694f0 con 0x7fd28815ffd0 2024-04-26T21:45:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.844+0000 7fd27dffb640 1 -- 172.21.15.40:0/767044451 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2800137d0 con 0x7fd28815ffd0 2024-04-26T21:45:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.844+0000 7fd27dffb640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd26c06c8f0 0x7fd26c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.844+0000 7fd27dffb640 1 -- 172.21.15.40:0/767044451 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8969+0+0 (secure 0 0 0) 0x7fd28008cde0 con 0x7fd28815ffd0 2024-04-26T21:45:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.845+0000 7fd28d50d640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd26c06c8f0 0x7fd26c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.845+0000 7fd28d50d640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd26c06c8f0 0x7fd26c06edb0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd284033000 tx=0x7fd284008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:48.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:48.849+0000 7fd27dffb640 1 -- 172.21.15.40:0/767044451 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd280059eb0 con 0x7fd28815ffd0 2024-04-26T21:45:49.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.072+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2880b6860 con 0x7fd28815ffd0 2024-04-26T21:45:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.073+0000 7fd27dffb640 1 -- 172.21.15.40:0/767044451 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5193 (secure 0 0 0) 0x7fd28005e1b0 con 0x7fd28815ffd0 2024-04-26T21:45:49.074 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:49.074 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":291,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:48.713505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10866":{"gid":10866,"name":"c","rank":0,"incarnation":291,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/1660522931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1660522931},{"type":"v1","addr":"172.21.15.40:6833","nonce":1660522931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 291 2024-04-26T21:45:49.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd26c06c8f0 msgr2=0x7fd26c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:49.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd26c06c8f0 0x7fd26c06edb0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd284033000 tx=0x7fd284008030 comp rx=0 tx=0).stop 2024-04-26T21:45:49.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28815ffd0 msgr2=0x7fd2880b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:49.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28815ffd0 0x7fd2880b4a80 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fd280002230 tx=0x7fd28000a730 comp rx=0 tx=0).stop 2024-04-26T21:45:49.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 shutdown_connections 2024-04-26T21:45:49.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd26c06c8f0 0x7fd26c06edb0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:49.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28815ffd0 0x7fd2880b4a80 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2880b7b80 0x7fd2880c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 --2- 172.21.15.40:0/767044451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2740039e0 0x7fd28815c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 >> 172.21.15.40:0/767044451 conn(0x7fd2880d4f90 msgr2=0x7fd28806cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 shutdown_connections 2024-04-26T21:45:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:49.075+0000 7fd28f771640 1 -- 172.21.15.40:0/767044451 wait complete. 2024-04-26T21:45:49.087 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:45:51.731 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:45:51.731+0000 7f72c01fc640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-04-26T21:45:51.732 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:51.732+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:45:51.732 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:45:51.732+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:45:51.741 INFO:tasks.ceph.mds.c.smithi040.stderr: -17> 2024-04-26T21:45:51.731+0000 7f72c01fc640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-04-26T21:45:51.768 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:45:54.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:54.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.201+0000 7fab7e4f1640 1 Processor -- start 2024-04-26T21:45:54.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.201+0000 7fab7e4f1640 1 -- start start 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.202+0000 7fab7e4f1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fab780b2340 0x7fab780cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.202+0000 7fab7e4f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab780cd930 0x7fab780d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.202+0000 7fab7e4f1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab780d10d0 con 0x7fab78162810 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.202+0000 7fab7e4f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fab780d0750 con 0x7fab780cd930 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.202+0000 7fab7e4f1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fab780d16a0 con 0x7fab780b2340 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.202+0000 7fab7ca8e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab780cd930 0x7fab780d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.202+0000 7fab7ca8e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab780cd930 0x7fab780d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55482/0 (socket says 172.21.15.40:55482) 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.202+0000 7fab7ca8e640 1 -- 172.21.15.40:0/3694004571 learned_addr learned my addr 172.21.15.40:0/3694004571 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab7ca8e640 1 -- 172.21.15.40:0/3694004571 >> v2:172.21.15.184:3300/0 conn(0x7fab780b2340 msgr2=0x7fab780cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:54.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab777fe640 1 --2- 172.21.15.40:0/3694004571 >> v2:172.21.15.184:3300/0 conn(0x7fab780b2340 0x7fab780cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab7ca8e640 1 --2- 172.21.15.40:0/3694004571 >> v2:172.21.15.184:3300/0 conn(0x7fab780b2340 0x7fab780cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab7ca8e640 1 -- 172.21.15.40:0/3694004571 >> v1:172.21.15.184:6789/0 conn(0x7fab78162810 legacy=0x7fab78070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:54.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab7ca8e640 1 -- 172.21.15.40:0/3694004571 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab780d1370 con 0x7fab780cd930 2024-04-26T21:45:54.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab7ca8e640 1 --2- 172.21.15.40:0/3694004571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab780d1b30 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fab780cd0a0 tx=0x7fab7002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fda20eed3f33b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab76ffd640 1 -- 172.21.15.40:0/3694004571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70036070 con 0x7fab780cd930 2024-04-26T21:45:54.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab76ffd640 1 -- 172.21.15.40:0/3694004571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab70031600 con 0x7fab780cd930 2024-04-26T21:45:54.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.203+0000 7fab76ffd640 1 -- 172.21.15.40:0/3694004571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab700319d0 con 0x7fab780cd930 2024-04-26T21:45:54.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.204+0000 7fab7e4f1640 1 -- 172.21.15.40:0/3694004571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 msgr2=0x7fab780d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.204+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/3694004571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab780d1b30 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fab780cd0a0 tx=0x7fab7002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.204+0000 7fab7e4f1640 1 -- 172.21.15.40:0/3694004571 shutdown_connections 2024-04-26T21:45:54.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.204+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/3694004571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab780d1b30 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.204+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/3694004571 >> v2:172.21.15.184:3300/0 conn(0x7fab780b2340 0x7fab780cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.204+0000 7fab7e4f1640 1 -- 172.21.15.40:0/3694004571 >> 172.21.15.40:0/3694004571 conn(0x7fab780d2da0 msgr2=0x7fab780d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:54.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.204+0000 7fab7e4f1640 1 -- 172.21.15.40:0/3694004571 shutdown_connections 2024-04-26T21:45:54.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.204+0000 7fab7e4f1640 1 -- 172.21.15.40:0/3694004571 wait complete. 2024-04-26T21:45:54.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.205+0000 7fab7e4f1640 1 Processor -- start 2024-04-26T21:45:54.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab7e4f1640 1 -- start start 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab7e4f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab780b2340 0x7fab7815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab7e4f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab7814fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab7e4f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab78162810 0x7fab780ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab7e4f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab780d2070 con 0x7fab780cd930 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab7e4f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab780d0470 con 0x7fab78162810 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab7e4f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab780d21f0 con 0x7fab780b2340 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab7814fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab7814fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55496/0 (socket says 172.21.15.40:55496) 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab777fe640 1 -- 172.21.15.40:0/2084552374 learned_addr learned my addr 172.21.15.40:0/2084552374 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:54.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab7ca8e640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab78162810 0x7fab780ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.206+0000 7fab77fff640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab780b2340 0x7fab7815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab777fe640 1 -- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab780b2340 msgr2=0x7fab7815c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab777fe640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab780b2340 0x7fab7815c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab777fe640 1 -- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab78162810 msgr2=0x7fab780ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab777fe640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab78162810 0x7fab780ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab777fe640 1 -- 172.21.15.40:0/2084552374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7814fb60 con 0x7fab780cd930 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab777fe640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab7814fdb0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fab6800c820 tx=0x7fab6800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab74ff9640 1 -- 172.21.15.40:0/2084552374 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68015070 con 0x7fab780cd930 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab74ff9640 1 -- 172.21.15.40:0/2084552374 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab68004240 con 0x7fab780cd930 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7815cf20 con 0x7fab780cd930 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab74ff9640 1 -- 172.21.15.40:0/2084552374 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6800ce10 con 0x7fab780cd930 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.207+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7815d4e0 con 0x7fab780cd930 2024-04-26T21:45:54.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.208+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab780694f0 con 0x7fab780cd930 2024-04-26T21:45:54.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.209+0000 7fab74ff9640 1 -- 172.21.15.40:0/2084552374 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab68007830 con 0x7fab780cd930 2024-04-26T21:45:54.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.209+0000 7fab74ff9640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab5806c8f0 0x7fab5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.209+0000 7fab74ff9640 1 -- 172.21.15.40:0/2084552374 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fab6808c6e0 con 0x7fab780cd930 2024-04-26T21:45:54.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.209+0000 7fab77fff640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab5806c8f0 0x7fab5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.210+0000 7fab77fff640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab5806c8f0 0x7fab5806edb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fab60006fd0 tx=0x7fab60009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.214+0000 7fab74ff9640 1 -- 172.21.15.40:0/2084552374 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab7814fb60 con 0x7fab780cd930 2024-04-26T21:45:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.437+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab780cd170 con 0x7fab780cd930 2024-04-26T21:45:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.438+0000 7fab74ff9640 1 -- 172.21.15.40:0/2084552374 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 296 v296) ==== 95+0+5182 (secure 0 0 0) 0x7fab68059760 con 0x7fab780cd930 2024-04-26T21:45:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:54.439 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":296,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10914,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1753847972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1753847972},{"type":"v1","addr":"172.21.15.40:6833","nonce":1753847972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":296}],"filesystems":[{"mdsmap":{"epoch":295,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:51.733522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:54.439 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 296 2024-04-26T21:45:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab5806c8f0 msgr2=0x7fab5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab5806c8f0 0x7fab5806edb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fab60006fd0 tx=0x7fab60009060 comp rx=0 tx=0).stop 2024-04-26T21:45:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 msgr2=0x7fab7814fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab7814fdb0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fab6800c820 tx=0x7fab6800cce0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 shutdown_connections 2024-04-26T21:45:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fab5806c8f0 0x7fab5806edb0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab78162810 0x7fab780ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab780cd930 0x7fab7814fdb0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 --2- 172.21.15.40:0/2084552374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab780b2340 0x7fab7815c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 >> 172.21.15.40:0/2084552374 conn(0x7fab780d2da0 msgr2=0x7fab780cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:54.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 shutdown_connections 2024-04-26T21:45:54.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.440+0000 7fab7e4f1640 1 -- 172.21.15.40:0/2084552374 wait complete. 2024-04-26T21:45:54.452 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:45:54.452 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.553+0000 7f23fbd04640 1 Processor -- start 2024-04-26T21:45:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.553+0000 7f23fbd04640 1 -- start start 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23fbd04640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f23f4162810 0x7f23f4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23fbd04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f23f40b2340 0x7f23f40cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23fbd04640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f40d10d0 con 0x7f23f40cd930 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23fbd04640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f23f40d0750 con 0x7f23f40b2340 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23fbd04640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f23f40d16a0 con 0x7f23f4162810 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23f929f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f23f40b2340 0x7f23f40cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23fa2a1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f23f40cd930 0x7f23f40d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42218/0 (socket says 172.21.15.40:42218) 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23fa2a1640 1 -- 172.21.15.40:0/2891951065 learned_addr learned my addr 172.21.15.40:0/2891951065 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23f929f640 1 -- 172.21.15.40:0/2891951065 >> v2:172.21.15.184:3300/0 conn(0x7f23f4162810 msgr2=0x7f23f4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23f9aa0640 1 --2- 172.21.15.40:0/2891951065 >> v2:172.21.15.184:3300/0 conn(0x7f23f4162810 0x7f23f4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23f929f640 1 --2- 172.21.15.40:0/2891951065 >> v2:172.21.15.184:3300/0 conn(0x7f23f4162810 0x7f23f4070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23f8a9e640 1 -- 172.21.15.40:0/2891951065 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877602251 0 0) 0x7f23f40d10d0 con 0x7f23f40cd930 2024-04-26T21:45:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23f929f640 1 -- 172.21.15.40:0/2891951065 >> v1:172.21.15.40:6789/0 conn(0x7f23f40cd930 legacy=0x7f23f40d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.554+0000 7f23f929f640 1 -- 172.21.15.40:0/2891951065 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f40d1370 con 0x7f23f40b2340 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23f929f640 1 --2- 172.21.15.40:0/2891951065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40b2340 0x7f23f40cd3f0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f23f4073d90 tx=0x7f23e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945ac8f6d716e01e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23f8a9e640 1 -- 172.21.15.40:0/2891951065 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4037070 con 0x7f23f40b2340 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23f8a9e640 1 -- 172.21.15.40:0/2891951065 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e4032440 con 0x7f23f40b2340 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23f8a9e640 1 -- 172.21.15.40:0/2891951065 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4032810 con 0x7f23f40b2340 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23fbd04640 1 -- 172.21.15.40:0/2891951065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40b2340 msgr2=0x7f23f40cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23fbd04640 1 --2- 172.21.15.40:0/2891951065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40b2340 0x7f23f40cd3f0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f23f4073d90 tx=0x7f23e402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23fbd04640 1 -- 172.21.15.40:0/2891951065 shutdown_connections 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23fbd04640 1 --2- 172.21.15.40:0/2891951065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40b2340 0x7f23f40cd3f0 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23fbd04640 1 --2- 172.21.15.40:0/2891951065 >> v2:172.21.15.184:3300/0 conn(0x7f23f4162810 0x7f23f4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.555+0000 7f23fbd04640 1 -- 172.21.15.40:0/2891951065 >> 172.21.15.40:0/2891951065 conn(0x7f23f40d2da0 msgr2=0x7f23f40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.556+0000 7f23fbd04640 1 -- 172.21.15.40:0/2891951065 shutdown_connections 2024-04-26T21:45:54.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.556+0000 7f23fbd04640 1 -- 172.21.15.40:0/2891951065 wait complete. 2024-04-26T21:45:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.557+0000 7f23fbd04640 1 Processor -- start 2024-04-26T21:45:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.557+0000 7f23fbd04640 1 -- start start 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.557+0000 7f23fbd04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23f40b2340 0x7f23f415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.557+0000 7f23fbd04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23f40cd930 0x7f23f40c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fbd04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f4162810 0x7f23f40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fbd04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4070890 con 0x7f23f4162810 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fbd04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40d0470 con 0x7f23f40cd930 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fbd04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f23f40d2240 con 0x7f23f40b2340 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f4162810 0x7f23f40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f4162810 0x7f23f40cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55522/0 (socket says 172.21.15.40:55522) 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 -- 172.21.15.40:0/1948872877 learned_addr learned my addr 172.21.15.40:0/1948872877 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23f9aa0640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23f40b2340 0x7f23f415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23f929f640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23f40cd930 0x7f23f40c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 -- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23f40b2340 msgr2=0x7f23f415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23f40b2340 0x7f23f415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 -- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23f40cd930 msgr2=0x7f23f40c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23f40cd930 0x7f23f40c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 -- 172.21.15.40:0/1948872877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f40cb5e0 con 0x7f23f4162810 2024-04-26T21:45:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fa2a1640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f4162810 0x7f23f40cc680 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00c760 tx=0x7f23ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23ea7fc640 1 -- 172.21.15.40:0/1948872877 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec018070 con 0x7f23f4162810 2024-04-26T21:45:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23ea7fc640 1 -- 172.21.15.40:0/1948872877 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ec00a640 con 0x7f23f4162810 2024-04-26T21:45:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23ea7fc640 1 -- 172.21.15.40:0/1948872877 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec00aa30 con 0x7f23f4162810 2024-04-26T21:45:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f40cb8c0 con 0x7f23f4162810 2024-04-26T21:45:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.558+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f40cbe80 con 0x7f23f4162810 2024-04-26T21:45:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.559+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f40694f0 con 0x7f23f4162810 2024-04-26T21:45:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.560+0000 7f23ea7fc640 1 -- 172.21.15.40:0/1948872877 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23ec01b070 con 0x7f23f4162810 2024-04-26T21:45:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.560+0000 7f23ea7fc640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23d806c8f0 0x7f23d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.560+0000 7f23ea7fc640 1 -- 172.21.15.40:0/1948872877 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f23ec08bdd0 con 0x7f23f4162810 2024-04-26T21:45:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.561+0000 7f23f9aa0640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23d806c8f0 0x7f23d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.561+0000 7f23f9aa0640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23d806c8f0 0x7f23d806edb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f23f000aa20 tx=0x7f23f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.565+0000 7f23ea7fc640 1 -- 172.21.15.40:0/1948872877 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f23ec058e50 con 0x7f23f4162810 2024-04-26T21:45:54.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.788+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23f406ce60 con 0x7f23f4162810 2024-04-26T21:45:54.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.789+0000 7f23ea7fc640 1 -- 172.21.15.40:0/1948872877 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 296 v296) ==== 95+0+5182 (secure 0 0 0) 0x7f23ec08d090 con 0x7f23f4162810 2024-04-26T21:45:54.789 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:54.790 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":296,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10914,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1753847972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1753847972},{"type":"v1","addr":"172.21.15.40:6833","nonce":1753847972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":296}],"filesystems":[{"mdsmap":{"epoch":295,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:51.733522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:54.790 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 296 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.790+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23d806c8f0 msgr2=0x7f23d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.790+0000 7f23fbd04640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23d806c8f0 0x7f23d806edb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f23f000aa20 tx=0x7f23f0008040 comp rx=0 tx=0).stop 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f4162810 msgr2=0x7f23f40cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f4162810 0x7f23f40cc680 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00c760 tx=0x7f23ec00cc20 comp rx=0 tx=0).stop 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 shutdown_connections 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f23d806c8f0 0x7f23d806edb0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f4162810 0x7f23f40cc680 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23f40cd930 0x7f23f40c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 --2- 172.21.15.40:0/1948872877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23f40b2340 0x7f23f415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 >> 172.21.15.40:0/1948872877 conn(0x7f23f40d2da0 msgr2=0x7f23f40b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 shutdown_connections 2024-04-26T21:45:54.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.791+0000 7f23fbd04640 1 -- 172.21.15.40:0/1948872877 wait complete. 2024-04-26T21:45:54.802 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:45:54.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.902+0000 7f043eff7640 1 Processor -- start 2024-04-26T21:45:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.902+0000 7f043eff7640 1 -- start start 2024-04-26T21:45:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f043eff7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0438162810 0x7f0438070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f043eff7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f04380b2340 0x7f04380cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f043eff7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04380d10d0 con 0x7f04380cd930 2024-04-26T21:45:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f043eff7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f04380d0750 con 0x7f04380b2340 2024-04-26T21:45:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f043eff7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04380d16a0 con 0x7f0438162810 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f043cd93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0438162810 0x7f0438070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f0437fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f04380b2340 0x7f04380cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f0437fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f04380b2340 0x7f04380cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:51278/0 (socket says 172.21.15.40:51278) 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.903+0000 7f0437fff640 1 -- 172.21.15.40:0/2416726743 learned_addr learned my addr 172.21.15.40:0/2416726743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.904+0000 7f0437fff640 1 -- 172.21.15.40:0/2416726743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0438162810 msgr2=0x7f0438070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.904+0000 7f0437fff640 1 --2- 172.21.15.40:0/2416726743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0438162810 0x7f0438070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.904+0000 7f0437fff640 1 -- 172.21.15.40:0/2416726743 >> v1:172.21.15.184:6789/0 conn(0x7f04380cd930 legacy=0x7f04380d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.904+0000 7f04377fe640 1 -- 172.21.15.40:0/2416726743 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3047717377 0 0) 0x7f04380d10d0 con 0x7f04380cd930 2024-04-26T21:45:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.904+0000 7f0437fff640 1 -- 172.21.15.40:0/2416726743 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04380d1370 con 0x7f04380b2340 2024-04-26T21:45:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f0437fff640 1 --2- 172.21.15.40:0/2416726743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04380b2340 0x7f04380cd3f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0428009870 tx=0x7f042802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39bcef0b5ef0ba59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f04377fe640 1 -- 172.21.15.40:0/2416726743 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428037070 con 0x7f04380b2340 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f04377fe640 1 -- 172.21.15.40:0/2416726743 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0428032440 con 0x7f04380b2340 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f04377fe640 1 -- 172.21.15.40:0/2416726743 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428032810 con 0x7f04380b2340 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f043eff7640 1 -- 172.21.15.40:0/2416726743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04380b2340 msgr2=0x7f04380cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f043eff7640 1 --2- 172.21.15.40:0/2416726743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04380b2340 0x7f04380cd3f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0428009870 tx=0x7f042802a270 comp rx=0 tx=0).stop 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f043eff7640 1 -- 172.21.15.40:0/2416726743 shutdown_connections 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f043eff7640 1 --2- 172.21.15.40:0/2416726743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04380b2340 0x7f04380cd3f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f043eff7640 1 --2- 172.21.15.40:0/2416726743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0438162810 0x7f0438070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f043eff7640 1 -- 172.21.15.40:0/2416726743 >> 172.21.15.40:0/2416726743 conn(0x7f04380d2da0 msgr2=0x7f04380d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f043eff7640 1 -- 172.21.15.40:0/2416726743 shutdown_connections 2024-04-26T21:45:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.905+0000 7f043eff7640 1 -- 172.21.15.40:0/2416726743 wait complete. 2024-04-26T21:45:54.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.907+0000 7f043eff7640 1 Processor -- start 2024-04-26T21:45:54.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.907+0000 7f043eff7640 1 -- start start 2024-04-26T21:45:54.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.907+0000 7f043eff7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04380b2340 0x7f0438161b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.907+0000 7f043eff7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04380cd930 0x7f043806e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.907+0000 7f043eff7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0438162810 0x7f04380ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.907+0000 7f043eff7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0438070890 con 0x7f04380cd930 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.907+0000 7f043eff7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04380d0470 con 0x7f0438162810 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.907+0000 7f043eff7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04380d2240 con 0x7f04380b2340 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04380cd930 0x7f043806e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04380cd930 0x7f043806e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55536/0 (socket says 172.21.15.40:55536) 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 -- 172.21.15.40:0/1601625604 learned_addr learned my addr 172.21.15.40:0/1601625604 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f043cd93640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04380b2340 0x7f0438161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f043d594640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0438162810 0x7f04380ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 -- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04380b2340 msgr2=0x7f0438161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04380b2340 0x7f0438161b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 -- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0438162810 msgr2=0x7f04380ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0438162810 0x7f04380ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 -- 172.21.15.40:0/1601625604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0438162040 con 0x7f04380cd930 2024-04-26T21:45:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f0437fff640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04380cd930 0x7f043806e880 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f0428000c00 tx=0x7f04280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f04357fa640 1 -- 172.21.15.40:0/1601625604 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428037040 con 0x7f04380cd930 2024-04-26T21:45:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f04357fa640 1 -- 172.21.15.40:0/1601625604 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0428042070 con 0x7f04380cd930 2024-04-26T21:45:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043806d7e0 con 0x7f04380cd930 2024-04-26T21:45:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f04357fa640 1 -- 172.21.15.40:0/1601625604 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428030880 con 0x7f04380cd930 2024-04-26T21:45:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.908+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f043806dda0 con 0x7f04380cd930 2024-04-26T21:45:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.909+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04380694f0 con 0x7f04380cd930 2024-04-26T21:45:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.910+0000 7f04357fa640 1 -- 172.21.15.40:0/1601625604 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0428030a20 con 0x7f04380cd930 2024-04-26T21:45:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.910+0000 7f04357fa640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f041806c8f0 0x7f041806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.910+0000 7f04357fa640 1 -- 172.21.15.40:0/1601625604 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f04280acff0 con 0x7f04380cd930 2024-04-26T21:45:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.911+0000 7f043cd93640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f041806c8f0 0x7f041806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.911+0000 7f043cd93640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f041806c8f0 0x7f041806edb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0420009b40 tx=0x7f0420009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:54.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:54.915+0000 7f04357fa640 1 -- 172.21.15.40:0/1601625604 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f042807a050 con 0x7f04380cd930 2024-04-26T21:45:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.139+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04380c9be0 con 0x7f04380cd930 2024-04-26T21:45:55.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.139+0000 7f04357fa640 1 -- 172.21.15.40:0/1601625604 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 296 v296) ==== 95+0+5182 (secure 0 0 0) 0x7f042807e350 con 0x7f04380cd930 2024-04-26T21:45:55.140 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:55.140 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":296,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":10914,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1753847972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1753847972},{"type":"v1","addr":"172.21.15.40:6833","nonce":1753847972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":296}],"filesystems":[{"mdsmap":{"epoch":295,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:51.733522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:55.140 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 296 2024-04-26T21:45:55.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f041806c8f0 msgr2=0x7f041806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f041806c8f0 0x7f041806edb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0420009b40 tx=0x7f0420009190 comp rx=0 tx=0).stop 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04380cd930 msgr2=0x7f043806e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04380cd930 0x7f043806e880 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f0428000c00 tx=0x7f04280046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 shutdown_connections 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f041806c8f0 0x7f041806edb0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0438162810 0x7f04380ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04380cd930 0x7f043806e880 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 --2- 172.21.15.40:0/1601625604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04380b2340 0x7f0438161b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 >> 172.21.15.40:0/1601625604 conn(0x7f04380d2da0 msgr2=0x7f04380b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 shutdown_connections 2024-04-26T21:45:55.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.141+0000 7f043eff7640 1 -- 172.21.15.40:0/1601625604 wait complete. 2024-04-26T21:45:55.153 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap key 100.00000000:stray0_head' led to DAMAGED state 2024-04-26T21:45:55.153 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 1.00000000:subdir_head' 2024-04-26T21:45:55.153 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:45:55.153 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:45:55.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:45:55.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:45:55.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:45:55.169 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:45:55.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:45:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:45:55.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.365+0000 7f93605ec640 1 Processor -- start 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.365+0000 7f93605ec640 1 -- start start 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f93605ec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93580d3bc0 0x7f93580d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f93605ec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93580b5f20 0x7f93580b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f93605ec640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93580b7e50 con 0x7f9358162880 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f93605ec640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f93580d0580 con 0x7f93580b5f20 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f93605ec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f93580d1800 con 0x7f93580d3bc0 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935e388640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9358162880 0x7f93580d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42220/0 (socket says 172.21.15.40:42220) 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935e388640 1 -- 172.21.15.40:0/3633055661 learned_addr learned my addr 172.21.15.40:0/3633055661 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4179524391 0 0) 0x7f93580b7e50 con 0x7f9358162880 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935db87640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93580d3bc0 0x7f93580d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9344003520 con 0x7f9358162880 2024-04-26T21:45:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935eb89640 1 --2- 172.21.15.40:0/3633055661 >> v2:172.21.15.184:3300/0 conn(0x7f93580b5f20 0x7f93580b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1799542944 0 0) 0x7f9344003520 con 0x7f9358162880 2024-04-26T21:45:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93580d3bc0 msgr2=0x7f93580d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93580d3bc0 0x7f93580d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 >> v2:172.21.15.184:3300/0 conn(0x7f93580b5f20 msgr2=0x7f93580b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 --2- 172.21.15.40:0/3633055661 >> v2:172.21.15.184:3300/0 conn(0x7f93580b5f20 0x7f93580b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93580d07d0 con 0x7f9358162880 2024-04-26T21:45:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9354001670 con 0x7f9358162880 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 >> v1:172.21.15.40:6789/0 conn(0x7f9358162880 legacy=0x7f93580d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93580d3bc0 0x7f93440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f9344004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9344004d40 0x7f93440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93580d0580 con 0x7f9344003f30 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93580d1800 con 0x7f9344004d40 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f934400ba10 con 0x7f93580d3bc0 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935eb89640 1 --2- 172.21.15.40:0/3633055661 >> v2:172.21.15.184:3300/0 conn(0x7f93580b5f20 0x7f93580b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935db87640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f9344004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935e388640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93580d3bc0 0x7f93440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935eb89640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9344004d40 0x7f93440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935db87640 1 -- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93580d3bc0 msgr2=0x7f93440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935db87640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93580d3bc0 0x7f93440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935db87640 1 -- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9344004d40 msgr2=0x7f93440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935db87640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9344004d40 0x7f93440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.366+0000 7f935db87640 1 -- 172.21.15.40:0/3633055661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9344003520 con 0x7f9344003f30 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f935db87640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f9344004800 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f9348009ac0 tx=0x7f9344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba8345ef16d14aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348036070 con 0x7f9344003f30 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9348031400 con 0x7f9344003f30 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f935d386640 1 -- 172.21.15.40:0/3633055661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348009690 con 0x7f9344003f30 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f93605ec640 1 -- 172.21.15.40:0/3633055661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 msgr2=0x7f9344004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f93605ec640 1 --2- 172.21.15.40:0/3633055661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f9344004800 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f9348009ac0 tx=0x7f9344005830 comp rx=0 tx=0).stop 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f935eb89640 1 -- 172.21.15.40:0/3633055661 reap_dead start 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f93605ec640 1 -- 172.21.15.40:0/3633055661 shutdown_connections 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f93605ec640 1 -- 172.21.15.40:0/3633055661 >> 172.21.15.40:0/3633055661 conn(0x7f93580d4f60 msgr2=0x7f93580d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f93605ec640 1 -- 172.21.15.40:0/3633055661 shutdown_connections 2024-04-26T21:45:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.367+0000 7f93605ec640 1 -- 172.21.15.40:0/3633055661 wait complete. 2024-04-26T21:45:55.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.369+0000 7f93605ec640 1 Processor -- start 2024-04-26T21:45:55.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.369+0000 7f93605ec640 1 -- start start 2024-04-26T21:45:55.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.369+0000 7f93605ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f935814f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.369+0000 7f93605ec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93581e9ea0 0x7f93581ea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.369+0000 7f93605ec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93580b4680 0x7f93580b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.369+0000 7f93605ec640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9358162c60 con 0x7f9344003f30 2024-04-26T21:45:55.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.369+0000 7f93605ec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93580d0d90 con 0x7f93580b4680 2024-04-26T21:45:55.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.369+0000 7f93605ec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9358162de0 con 0x7f93581e9ea0 2024-04-26T21:45:55.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f935814f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f935814f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55564/0 (socket says 172.21.15.40:55564) 2024-04-26T21:45:55.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 -- 172.21.15.40:0/2609472979 learned_addr learned my addr 172.21.15.40:0/2609472979 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:55.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935eb89640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93580b4680 0x7f93580b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935db87640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93581e9ea0 0x7f93581ea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 -- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93581e9ea0 msgr2=0x7f93581ea2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93581e9ea0 0x7f93581ea2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 -- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93580b4680 msgr2=0x7f93580b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93580b4680 0x7f93580b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 -- 172.21.15.40:0/2609472979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93580cd2f0 con 0x7f9344003f30 2024-04-26T21:45:55.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f935e388640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f935814f270 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f93540073a0 tx=0x7f9354007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f934effd640 1 -- 172.21.15.40:0/2609472979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935401a070 con 0x7f9344003f30 2024-04-26T21:45:55.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f934effd640 1 -- 172.21.15.40:0/2609472979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935400f940 con 0x7f9344003f30 2024-04-26T21:45:55.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93580b53b0 con 0x7f9344003f30 2024-04-26T21:45:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f934effd640 1 -- 172.21.15.40:0/2609472979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935400fd10 con 0x7f9344003f30 2024-04-26T21:45:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.370+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93580b5970 con 0x7f9344003f30 2024-04-26T21:45:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.371+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93580764d0 con 0x7f9344003f30 2024-04-26T21:45:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.372+0000 7f934effd640 1 -- 172.21.15.40:0/2609472979 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f935401d070 con 0x7f9344003f30 2024-04-26T21:45:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.372+0000 7f934effd640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f933c06c8f0 0x7f933c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.372+0000 7f934effd640 1 -- 172.21.15.40:0/2609472979 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f935408c310 con 0x7f9344003f30 2024-04-26T21:45:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.372+0000 7f935db87640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f933c06c8f0 0x7f933c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.373+0000 7f935db87640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f933c06c8f0 0x7f933c06edb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9348033000 tx=0x7f9348008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.377+0000 7f934effd640 1 -- 172.21.15.40:0/2609472979 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93580764d0 con 0x7f9344003f30 2024-04-26T21:45:55.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.602+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f93580694f0 con 0x7f9344003f30 2024-04-26T21:45:55.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.754+0000 7f934effd640 1 -- 172.21.15.40:0/2609472979 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v297) ==== 161+0+0 (secure 0 0 0) 0x7f93540599f0 con 0x7f9344003f30 2024-04-26T21:45:55.755 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:45:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f933c06c8f0 msgr2=0x7f933c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f933c06c8f0 0x7f933c06edb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9348033000 tx=0x7f9348008030 comp rx=0 tx=0).stop 2024-04-26T21:45:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 msgr2=0x7f935814f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f935814f270 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f93540073a0 tx=0x7f9354007860 comp rx=0 tx=0).stop 2024-04-26T21:45:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 shutdown_connections 2024-04-26T21:45:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f933c06c8f0 0x7f933c06edb0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93580b4680 0x7f93580b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93581e9ea0 0x7f93581ea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 --2- 172.21.15.40:0/2609472979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9344003f30 0x7f935814f270 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 >> 172.21.15.40:0/2609472979 conn(0x7f93580d4f60 msgr2=0x7f935815fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 shutdown_connections 2024-04-26T21:45:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.756+0000 7f93605ec640 1 -- 172.21.15.40:0/2609472979 wait complete. 2024-04-26T21:45:55.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:45:55.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.869+0000 7f1bbb8cf640 1 Processor -- start 2024-04-26T21:45:55.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.869+0000 7f1bbb8cf640 1 -- start start 2024-04-26T21:45:55.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bbb8cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bbb8cf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1bb40b5f20 0x7f1bb40d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bbb8cf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1bb40d39f0 0x7f1bb40b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bbb8cf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1bb40d0580 con 0x7f1bb40d39f0 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bbb8cf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1bb40b7e50 con 0x7f1bb40b5f20 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bbb8cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb40d1800 con 0x7f1bb4162880 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1bb40d39f0 0x7f1bb40b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1bb40d39f0 0x7f1bb40b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55580/0 (socket says 172.21.15.40:55580) 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 -- 172.21.15.40:0/4038318166 learned_addr learned my addr 172.21.15.40:0/4038318166 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb966b640 1 --2- 172.21.15.40:0/4038318166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb8e6a640 1 --2- 172.21.15.40:0/4038318166 >> v2:172.21.15.184:3300/0 conn(0x7f1bb40b5f20 0x7f1bb40d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 -- 172.21.15.40:0/4038318166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 msgr2=0x7f1bb40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 --2- 172.21.15.40:0/4038318166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 -- 172.21.15.40:0/4038318166 >> v2:172.21.15.184:3300/0 conn(0x7f1bb40b5f20 msgr2=0x7f1bb40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 --2- 172.21.15.40:0/4038318166 >> v2:172.21.15.184:3300/0 conn(0x7f1bb40b5f20 0x7f1bb40d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 -- 172.21.15.40:0/4038318166 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb40d07d0 con 0x7f1bb40d39f0 2024-04-26T21:45:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1bb9e6c640 1 --2- 172.21.15.40:0/4038318166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb40b69e0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f1bb408e960 tx=0x7f1bb002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfea505900ed9aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.871+0000 7f1babfff640 1 -- 172.21.15.40:0/4038318166 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0036070 con 0x7f1bb40d39f0 2024-04-26T21:45:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1babfff640 1 -- 172.21.15.40:0/4038318166 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb0031600 con 0x7f1bb40d39f0 2024-04-26T21:45:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1babfff640 1 -- 172.21.15.40:0/4038318166 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00319d0 con 0x7f1bb40d39f0 2024-04-26T21:45:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/4038318166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 msgr2=0x7f1bb40b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/4038318166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb40b69e0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f1bb408e960 tx=0x7f1bb002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/4038318166 shutdown_connections 2024-04-26T21:45:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/4038318166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb40b69e0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/4038318166 >> v2:172.21.15.184:3300/0 conn(0x7f1bb40b5f20 0x7f1bb40d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/4038318166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/4038318166 >> 172.21.15.40:0/4038318166 conn(0x7f1bb40d4f60 msgr2=0x7f1bb40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/4038318166 shutdown_connections 2024-04-26T21:45:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.872+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/4038318166 wait complete. 2024-04-26T21:45:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.873+0000 7f1bbb8cf640 1 Processor -- start 2024-04-26T21:45:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.873+0000 7f1bbb8cf640 1 -- start start 2024-04-26T21:45:55.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bbb8cf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bb40b5f20 0x7f1bb40b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bbb8cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bbb8cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bbb8cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb4162c60 con 0x7f1bb40d39f0 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bbb8cf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb40d0d90 con 0x7f1bb40b5f20 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bbb8cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb4162de0 con 0x7f1bb4162880 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bb8e6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bb8e6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb415c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55582/0 (socket says 172.21.15.40:55582) 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bb8e6a640 1 -- 172.21.15.40:0/1058486275 learned_addr learned my addr 172.21.15.40:0/1058486275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bb9e6c640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.874+0000 7f1bb966b640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bb40b5f20 0x7f1bb40b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bb8e6a640 1 -- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 msgr2=0x7f1bb40cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bb8e6a640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bb8e6a640 1 -- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bb40b5f20 msgr2=0x7f1bb40b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bb8e6a640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bb40b5f20 0x7f1bb40b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bb8e6a640 1 -- 172.21.15.40:0/1058486275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb40cc120 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bb9e6c640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bb966b640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bb40b5f20 0x7f1bb40b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bb8e6a640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb415c770 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400d800 tx=0x7f1ba400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1ba9ffb640 1 -- 172.21.15.40:0/1058486275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4018030 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1ba9ffb640 1 -- 172.21.15.40:0/1058486275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba40096a0 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb40cc400 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1ba9ffb640 1 -- 172.21.15.40:0/1058486275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4009870 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.875+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb40b2090 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.876+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb40694f0 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.876+0000 7f1ba9ffb640 1 -- 172.21.15.40:0/1058486275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ba401b070 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.877+0000 7f1ba9ffb640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b9806c920 0x7f1b9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.877+0000 7f1ba9ffb640 1 -- 172.21.15.40:0/1058486275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f1ba408cb10 con 0x7f1bb40d39f0 2024-04-26T21:45:55.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.877+0000 7f1bb966b640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b9806c920 0x7f1b9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:55.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.878+0000 7f1bb966b640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b9806c920 0x7f1b9806ede0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006fd0 tx=0x7f1bac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:55.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:55.881+0000 7f1ba9ffb640 1 -- 172.21.15.40:0/1058486275 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ba405a1e0 con 0x7f1bb40d39f0 2024-04-26T21:45:56.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.142+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f1bb4069cd0 con 0x7f1bb40d39f0 2024-04-26T21:45:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.763+0000 7f1ba9ffb640 1 -- 172.21.15.40:0/1058486275 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v298) ==== 101+0+0 (secure 0 0 0) 0x7f1ba4059ae0 con 0x7f1bb40d39f0 2024-04-26T21:45:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:45:56.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b9806c920 msgr2=0x7f1b9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b9806c920 0x7f1b9806ede0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006fd0 tx=0x7f1bac008040 comp rx=0 tx=0).stop 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 msgr2=0x7f1bb415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb415c770 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400d800 tx=0x7f1ba400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 shutdown_connections 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1b9806c920 0x7f1b9806ede0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb4162880 0x7f1bb40cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40d39f0 0x7f1bb415c770 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.765+0000 7f1bbb8cf640 1 --2- 172.21.15.40:0/1058486275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bb40b5f20 0x7f1bb40b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.766+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 >> 172.21.15.40:0/1058486275 conn(0x7f1bb40d4f60 msgr2=0x7f1bb40b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.766+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 shutdown_connections 2024-04-26T21:45:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.766+0000 7f1bbb8cf640 1 -- 172.21.15.40:0/1058486275 wait complete. 2024-04-26T21:45:56.779 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:45:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.824+0000 7fa45acd1f40 1 Processor -- start 2024-04-26T21:45:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.824+0000 7fa45acd1f40 1 -- start start 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa45acd1f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 0x5578917bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa45acd1f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 0x5578916e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa45acd1f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x55789184be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa45acd1f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55789172d2d0 con 0x5578917a3f30 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa45acd1f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55789168ce40 con 0x5578917a0e60 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa45acd1f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55789172d570 con 0x5578917a0a80 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa458fce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x55789184be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa458fce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x55789184be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55586/0 (socket says 172.21.15.40:55586) 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa458fce640 1 -- 172.21.15.40:0/3697087017 learned_addr learned my addr 172.21.15.40:0/3697087017 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa457fcc640 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 0x5578916e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa4587cd640 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 0x5578917bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa458fce640 1 -- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 msgr2=0x5578917bd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa458fce640 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 0x5578917bd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa458fce640 1 -- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 msgr2=0x5578916e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa458fce640 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 0x5578916e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa458fce640 1 -- 172.21.15.40:0/3697087017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789184a910 con 0x5578917a3f30 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa4587cd640 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 0x5578917bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.825+0000 7fa457fcc640 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 0x5578916e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa458fce640 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x55789184be00 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fa45000a430 tx=0x7fa450005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd42bba4fb530fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa456fca640 1 -- 172.21.15.40:0/3697087017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45000b8c0 con 0x5578917a3f30 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa456fca640 1 -- 172.21.15.40:0/3697087017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45000f030 con 0x5578917a3f30 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa456fca640 1 -- 172.21.15.40:0/3697087017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45000bcc0 con 0x5578917a3f30 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 -- 172.21.15.40:0/3697087017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 msgr2=0x55789184be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x55789184be00 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fa45000a430 tx=0x7fa450005f90 comp rx=0 tx=0).stop 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 -- 172.21.15.40:0/3697087017 shutdown_connections 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x55789184be00 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 0x5578916e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/3697087017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 0x5578917bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 -- 172.21.15.40:0/3697087017 >> 172.21.15.40:0/3697087017 conn(0x5578916e3ab0 msgr2=0x5578917a32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 -- 172.21.15.40:0/3697087017 shutdown_connections 2024-04-26T21:45:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.826+0000 7fa45acd1f40 1 -- 172.21.15.40:0/3697087017 wait complete. 2024-04-26T21:45:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.827+0000 7fa45acd1f40 1 Processor -- start 2024-04-26T21:45:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.827+0000 7fa45acd1f40 1 -- start start 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa45acd1f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 0x557891725d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa45acd1f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 0x5578918486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa45acd1f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x5578917b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa45acd1f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5578917bd2d0 con 0x5578917a3f30 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa45acd1f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5578917be120 con 0x5578917a0e60 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa45acd1f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5578917bde50 con 0x5578917a0a80 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x5578917b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x5578917b7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55596/0 (socket says 172.21.15.40:55596) 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 learned_addr learned my addr 172.21.15.40:0/1376403875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa457fcc640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 0x5578918486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa4587cd640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 0x557891725d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 msgr2=0x557891725d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578917a0a80 0x557891725d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 msgr2=0x5578918486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578917a0e60 0x5578918486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557891713600 con 0x5578917a3f30 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.828+0000 7fa458fce640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x5578917b7ff0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b910 tx=0x7fa450002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.829+0000 7fa455fc8640 1 -- 172.21.15.40:0/1376403875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450024070 con 0x5578917a3f30 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.829+0000 7fa455fc8640 1 -- 172.21.15.40:0/1376403875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45000bc80 con 0x5578917a3f30 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.829+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5578917138e0 con 0x5578917a3f30 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.829+0000 7fa455fc8640 1 -- 172.21.15.40:0/1376403875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450014660 con 0x5578917a3f30 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.829+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557891713ea0 con 0x5578917a3f30 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.830+0000 7fa455fc8640 1 -- 172.21.15.40:0/1376403875 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa450027070 con 0x5578917a3f30 2024-04-26T21:45:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.831+0000 7fa455fc8640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806b2d0 0x7fa43806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.831+0000 7fa455fc8640 1 -- 172.21.15.40:0/1376403875 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fa45008c140 con 0x5578917a3f30 2024-04-26T21:45:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.831+0000 7fa4587cd640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806b2d0 0x7fa43806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.833 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:45:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.831+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557891848c10 0x5578918531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.831+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5578918536e0 con 0x557891848c10 2024-04-26T21:45:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.831+0000 7fa457fcc640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557891848c10 0x5578918531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.831+0000 7fa4587cd640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806b2d0 0x7fa43806d790 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa44800e870 tx=0x7fa448007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.832+0000 7fa457fcc640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557891848c10 0x5578918531a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.833+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v128'1002 uv976 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.833+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891853a80 con 0x557891848c10 2024-04-26T21:45:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.835+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v128'1003 uv1003 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.835+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891853e20 con 0x557891848c10 2024-04-26T21:45:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.836+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v128'1004 uv1004 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.836 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:45:56.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.836+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860770 con 0x557891848c10 2024-04-26T21:45:56.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.837+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v128'1005 uv1005 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.837+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860b10 con 0x557891848c10 2024-04-26T21:45:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.838+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v128'1006 uv1006 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.838+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185f760 con 0x557891848c10 2024-04-26T21:45:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.839+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v128'1007 uv1007 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.839+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fb50 con 0x557891848c10 2024-04-26T21:45:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.840+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v128'1008 uv982 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.840+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55789185ff40 con 0x557891848c10 2024-04-26T21:45:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.842+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v128'1009 uv1009 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.842+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185ff40 con 0x557891848c10 2024-04-26T21:45:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.843+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v128'1010 uv1010 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.843 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:45:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.843+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.844+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v128'1011 uv1011 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.844+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.845+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v128'1012 uv1012 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.845+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.846+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v128'1013 uv1013 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.846+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.847+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v128'1014 uv989 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.847+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.848+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v128'1015 uv1015 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.848+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.850+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v128'1016 uv1016 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.849 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:45:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.850+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.851+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v128'1017 uv1017 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.851+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.852+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v128'1018 uv1018 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.852+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789185fef0 con 0x557891848c10 2024-04-26T21:45:56.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.853+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v128'1019 uv1019 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.853+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.854+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v128'1020 uv1020 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.854+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.855+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v128'1021 uv995 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.855+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.857+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v128'1022 uv1022 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.857+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.858+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v128'1023 uv1023 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.858 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:45:56.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.858+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891861ec0 con 0x557891848c10 2024-04-26T21:45:56.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.859+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v128'1024 uv1024 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.859+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891861ec0 con 0x557891848c10 2024-04-26T21:45:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.860+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v128'1025 uv1025 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.860+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.861+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v128'1026 uv1026 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.861+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.862+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v128'1027 uv1001 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.862+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.863+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v128'1028 uv1028 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.863+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.864 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:45:56.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.865+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v128'1029 uv1029 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.865+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891861ec0 con 0x557891848c10 2024-04-26T21:45:56.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.866+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v128'1030 uv1030 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.866+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891861ec0 con 0x557891848c10 2024-04-26T21:45:56.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.868+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v128'1031 uv1031 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.868+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891860eb0 con 0x557891848c10 2024-04-26T21:45:56.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.869+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v128'1032 uv1032 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891848c10 2024-04-26T21:45:56.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.869+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5578918613c0 0x5578918637b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.869+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891863cf0 con 0x5578918613c0 2024-04-26T21:45:56.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.870+0000 7fa4587cd640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5578918613c0 0x5578918637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.870+0000 7fa4587cd640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5578918613c0 0x5578918637b0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.872+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v128'727 uv709 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.872+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891864090 con 0x5578918613c0 2024-04-26T21:45:56.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.873+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v128'728 uv728 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.873+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891864430 con 0x5578918613c0 2024-04-26T21:45:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.875+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v128'729 uv729 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.875 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:45:56.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.875+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891864c80 con 0x5578918613c0 2024-04-26T21:45:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.876+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v128'730 uv730 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.876+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865020 con 0x5578918613c0 2024-04-26T21:45:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.877+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v128'731 uv731 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.877+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5578918653c0 con 0x5578918613c0 2024-04-26T21:45:56.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.879+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v128'732 uv714 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.879+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.880+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v128'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.880+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.881+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v128'734 uv734 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.881 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:45:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.881+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.883+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v128'735 uv735 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.883+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.884+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v128'736 uv736 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.884+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.885+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v128'737 uv719 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.886+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.887+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v128'738 uv738 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.887+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.888+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v128'739 uv739 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.888 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:45:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.888+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891869e20 con 0x5578918613c0 2024-04-26T21:45:56.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.890+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v128'740 uv740 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.890+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891869e20 con 0x5578918613c0 2024-04-26T21:45:56.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.891+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v128'741 uv741 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.891+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.892+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v128'742 uv725 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.892+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.893+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v128'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.894+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.894 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:45:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.895+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v128'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.895+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891866770 con 0x5578918613c0 2024-04-26T21:45:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.896+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v128'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.896+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891866770 con 0x5578918613c0 2024-04-26T21:45:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.897+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v128'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.897+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891865760 con 0x5578918613c0 2024-04-26T21:45:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.899+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v128'747 uv747 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x5578918613c0 2024-04-26T21:45:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.899+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x557891865c70 0x557891868060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.899+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5578918685a0 con 0x557891865c70 2024-04-26T21:45:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.899+0000 7fa457fcc640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x557891865c70 0x557891868060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.899+0000 7fa457fcc640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x557891865c70 0x557891868060 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.901+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v128'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.901+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891868970 con 0x557891865c70 2024-04-26T21:45:56.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.902+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v128'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.902+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891868970 con 0x557891865c70 2024-04-26T21:45:56.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.903+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v128'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.903 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:45:56.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.903+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891869980 con 0x557891865c70 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.904+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v128'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.904+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891869980 con 0x557891865c70 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.905+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v128'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.905+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891868970 con 0x557891865c70 2024-04-26T21:45:56.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.906+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v128'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.907+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x557891869040 0x55789186b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.907+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789186bb70 con 0x557891869040 2024-04-26T21:45:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.907+0000 7fa458fce640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x557891869040 0x55789186b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.907+0000 7fa458fce640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x557891869040 0x55789186b4a0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.908+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v128'149 uv148 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.909+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55789186c080 con 0x557891869040 2024-04-26T21:45:56.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.910+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v128'150 uv150 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.910+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789186c580 con 0x557891869040 2024-04-26T21:45:56.911 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:45:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.911+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v128'151 uv151 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.911+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789186ce40 con 0x557891869040 2024-04-26T21:45:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.913+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v128'152 uv152 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.913+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55789186d6f0 0x55789186fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.913+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870020 con 0x55789186d6f0 2024-04-26T21:45:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.913+0000 7fa4587cd640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55789186d6f0 0x55789186fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.913+0000 7fa4587cd640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55789186d6f0 0x55789186fae0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.914+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v128'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.915+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x5578918700e0 con 0x55789186d6f0 2024-04-26T21:45:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.916+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v128'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.916+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870500 con 0x55789186d6f0 2024-04-26T21:45:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.917+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v128'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.917 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:45:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.917+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871510 con 0x55789186d6f0 2024-04-26T21:45:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.918+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v128'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.918+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871510 con 0x55789186d6f0 2024-04-26T21:45:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.919+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v128'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.919+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870500 con 0x55789186d6f0 2024-04-26T21:45:56.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.920+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v128'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.921+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870500 con 0x557891869040 2024-04-26T21:45:56.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.922+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v128'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.922+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x5578918708a0 con 0x557891869040 2024-04-26T21:45:56.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.923+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v128'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.924+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.925+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v128'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.924 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:45:56.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.925+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871c50 con 0x557891869040 2024-04-26T21:45:56.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.926+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v128'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.926+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871c50 con 0x557891869040 2024-04-26T21:45:56.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.927+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v128'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.927+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.928+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v128'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.928+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.929+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v128'366 uv359 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.929+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.930+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v128'367 uv367 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.930+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.931 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:45:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.931+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v128'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.931+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.933+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v128'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.933+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.934+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v128'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.934+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.935+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v128'163 uv162 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.935+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.936+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v128'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.936+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.937+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v128'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.937 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:45:56.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.937+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.938+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v128'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.938+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x557891869040 2024-04-26T21:45:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.939+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v128'167 uv167 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa450089d70 con 0x557891869040 2024-04-26T21:45:56.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.939+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x55789186d6f0 2024-04-26T21:45:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.941+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v128'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.941+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891870c40 con 0x55789186d6f0 2024-04-26T21:45:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.942+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v128'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.942+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x55789186d6f0 2024-04-26T21:45:56.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.943+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v128'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.942 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:45:56.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.943+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871c90 con 0x55789186d6f0 2024-04-26T21:45:56.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.944+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v128'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.944+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871c50 con 0x55789186d6f0 2024-04-26T21:45:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.945+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v128'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.945+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x55789186d6f0 2024-04-26T21:45:56.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.946+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v128'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.946+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x55789186d6f0 2024-04-26T21:45:56.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.947+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v128'325 uv318 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.947+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891870c40 con 0x55789186d6f0 2024-04-26T21:45:56.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.948+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v128'326 uv326 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.948+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870c40 con 0x55789186d6f0 2024-04-26T21:45:56.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.949+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v128'327 uv327 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.949 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:45:56.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.950+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871030 con 0x55789186d6f0 2024-04-26T21:45:56.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.951+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v128'328 uv328 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.951+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870d00 con 0x55789186d6f0 2024-04-26T21:45:56.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.952+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v128'329 uv329 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.952+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870d00 con 0x55789186d6f0 2024-04-26T21:45:56.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.953+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v128'330 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.953+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891870d00 con 0x55789186d6f0 2024-04-26T21:45:56.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.954+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v128'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.954+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870ce0 con 0x55789186d6f0 2024-04-26T21:45:56.955 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:45:56.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.955+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v128'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.955+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891870ce0 con 0x55789186d6f0 2024-04-26T21:45:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.956+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v128'333 uv333 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.956+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55789186ca60 con 0x55789186d6f0 2024-04-26T21:45:56.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.958+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v128'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.958+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55789186ca20 0x557891873dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.958+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x55789186ca20 2024-04-26T21:45:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.958+0000 7fa457fcc640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55789186ca20 0x557891873dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.958+0000 7fa457fcc640 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55789186ca20 0x557891873dd0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:56.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.959+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v128'142 uv140 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x55789186ca20 2024-04-26T21:45:56.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.959+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891874310 con 0x55789186ca20 2024-04-26T21:45:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.960+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v128'143 uv143 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x55789186ca20 2024-04-26T21:45:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.960+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x55789186ca20 2024-04-26T21:45:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.961+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v128'144 uv144 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x55789186ca20 2024-04-26T21:45:56.961 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:45:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.961+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x55789186ca20 2024-04-26T21:45:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.962+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v128'145 uv145 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x55789186ca20 2024-04-26T21:45:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.962+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x55789186d6f0 2024-04-26T21:45:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.964+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v128'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.964+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891874310 con 0x55789186d6f0 2024-04-26T21:45:56.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.965+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v128'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.965+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x55789186d6f0 2024-04-26T21:45:56.966 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:45:56.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.966+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v128'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.966+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x55789186d6f0 2024-04-26T21:45:56.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.967+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v128'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.967+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x55789186d6f0 2024-04-26T21:45:56.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.969+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v128'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.969+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x55789186d6f0 2024-04-26T21:45:56.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.970+0000 7fa4587cd640 1 -- 172.21.15.40:0/1376403875 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v128'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa448011b40 con 0x55789186d6f0 2024-04-26T21:45:56.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.970+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x557891865c70 2024-04-26T21:45:56.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.971+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v128'166 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.971+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891874310 con 0x557891865c70 2024-04-26T21:45:56.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.972+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v128'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.972+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874310 con 0x557891865c70 2024-04-26T21:45:56.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.973+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v128'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.973 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:45:56.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.973+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5578918711a0 con 0x557891865c70 2024-04-26T21:45:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.974+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v128'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.974+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5578918711a0 con 0x557891865c70 2024-04-26T21:45:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.975+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v128'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.975+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871180 con 0x557891865c70 2024-04-26T21:45:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.977+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v128'402 uv393 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.977+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891871180 con 0x557891865c70 2024-04-26T21:45:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.978+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v128'403 uv403 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.978+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871180 con 0x557891865c70 2024-04-26T21:45:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.979+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v128'404 uv404 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.979 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:45:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.979+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871180 con 0x557891865c70 2024-04-26T21:45:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.980+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v128'405 uv405 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.980+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871180 con 0x557891865c70 2024-04-26T21:45:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.981+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v128'406 uv406 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.981+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871180 con 0x557891865c70 2024-04-26T21:45:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.983+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v128'407 uv400 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.983+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x557891871180 con 0x557891865c70 2024-04-26T21:45:56.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.984+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v128'408 uv408 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.984+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871180 con 0x557891865c70 2024-04-26T21:45:56.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.985+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v128'409 uv409 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.985 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:45:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.985+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5578918711c0 con 0x557891865c70 2024-04-26T21:45:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.986+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v128'410 uv410 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.986+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871160 con 0x557891865c70 2024-04-26T21:45:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.987+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v128'411 uv411 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.987+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891871160 con 0x557891865c70 2024-04-26T21:45:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.988+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v128'412 uv412 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.988+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x557891874860 con 0x557891865c70 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa457fcc640 1 -- 172.21.15.40:0/1376403875 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v128'413 uv413 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa4400545f0 con 0x557891865c70 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5578918613c0 msgr2=0x5578918637b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x5578918613c0 0x5578918637b0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557891848c10 msgr2=0x5578918531a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x557891848c10 0x5578918531a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x557891869040 msgr2=0x55789186b4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x557891869040 0x55789186b4a0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55789186ca20 msgr2=0x557891873dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55789186ca20 0x557891873dd0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 reap_dead start 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x557891865c70 msgr2=0x557891868060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x557891865c70 0x557891868060 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa458fce640 1 -- 172.21.15.40:0/1376403875 reap_dead start 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55789186d6f0 msgr2=0x55789186fae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55789186d6f0 0x55789186fae0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806b2d0 msgr2=0x7fa43806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806b2d0 0x7fa43806d790 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa44800e870 tx=0x7fa448007040 comp rx=0 tx=0).stop 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 msgr2=0x5578917b7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x5578917b7ff0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b910 tx=0x7fa450002f70 comp rx=0 tx=0).stop 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 shutdown_connections 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55789186d6f0 0x55789186fae0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa43806b2d0 0x7fa43806d790 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 --2- 172.21.15.40:0/1376403875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5578917a3f30 0x5578917b7ff0 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.989+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 >> 172.21.15.40:0/1376403875 conn(0x5578916e3ab0 msgr2=0x5578917a32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.990+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 shutdown_connections 2024-04-26T21:45:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:56.990+0000 7fa45acd1f40 1 -- 172.21.15.40:0/1376403875 wait complete. 2024-04-26T21:45:56.993 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 1.00000000 subdir_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:45:57.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.079+0000 7f371118af40 1 Processor -- start 2024-04-26T21:45:57.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.079+0000 7f371118af40 1 -- start start 2024-04-26T21:45:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.079+0000 7f371118af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c56357e710 0x55c56359a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.079+0000 7f371118af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 0x55c563599490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.079+0000 7f371118af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5636267a0 0x55c563629390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.079+0000 7f371118af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c5634f6e40 con 0x55c56357e710 2024-04-26T21:45:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.079+0000 7f371118af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c56357ee70 con 0x55c5636267a0 2024-04-26T21:45:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.079+0000 7f371118af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c5636148e0 con 0x55c56357fd10 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c56357e710 0x55c56359a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c56357e710 0x55c56359a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55602/0 (socket says 172.21.15.40:55602) 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 -- 172.21.15.40:0/1539589179 learned_addr learned my addr 172.21.15.40:0/1539589179 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370f485640 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5636267a0 0x55c563629390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370e483640 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 0x55c563599490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 -- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 msgr2=0x55c563599490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 0x55c563599490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 -- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5636267a0 msgr2=0x55c563629390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5636267a0 0x55c563629390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 -- 172.21.15.40:0/1539589179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c563627690 con 0x55c56357e710 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370e483640 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 0x55c563599490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370ec84640 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c56357e710 0x55c56359a0c0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f370800a550 tx=0x7f370804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a579a12312bc696 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370d481640 1 -- 172.21.15.40:0/1539589179 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708059040 con 0x55c56357e710 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370d481640 1 -- 172.21.15.40:0/1539589179 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3708064070 con 0x55c56357e710 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.080+0000 7f370d481640 1 -- 172.21.15.40:0/1539589179 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37080547f0 con 0x55c56357e710 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 -- 172.21.15.40:0/1539589179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c56357e710 msgr2=0x55c56359a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c56357e710 0x55c56359a0c0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f370800a550 tx=0x7f370804e000 comp rx=0 tx=0).stop 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 -- 172.21.15.40:0/1539589179 shutdown_connections 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5636267a0 0x55c563629390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 0x55c563599490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 --2- 172.21.15.40:0/1539589179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c56357e710 0x55c56359a0c0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 -- 172.21.15.40:0/1539589179 >> 172.21.15.40:0/1539589179 conn(0x55c56357a9c0 msgr2=0x55c56357a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 -- 172.21.15.40:0/1539589179 shutdown_connections 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 -- 172.21.15.40:0/1539589179 wait complete. 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 Processor -- start 2024-04-26T21:45:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.081+0000 7f371118af40 1 -- start start 2024-04-26T21:45:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f371118af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c56357e710 0x55c563597070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f371118af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 0x55c563625780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f371118af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5636267a0 0x55c563623560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f371118af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c563599de0 con 0x55c5636267a0 2024-04-26T21:45:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f371118af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c563627910 con 0x55c56357e710 2024-04-26T21:45:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f371118af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c56359a600 con 0x55c56357fd10 2024-04-26T21:45:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370f485640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5636267a0 0x55c563623560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370f485640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5636267a0 0x55c563623560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55608/0 (socket says 172.21.15.40:55608) 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370f485640 1 -- 172.21.15.40:0/670448384 learned_addr learned my addr 172.21.15.40:0/670448384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370ec84640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c56357e710 0x55c563597070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370e483640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 0x55c563625780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370f485640 1 -- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 msgr2=0x55c563625780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370f485640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c56357fd10 0x55c563625780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370f485640 1 -- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c56357e710 msgr2=0x55c563597070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370f485640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c56357e710 0x55c563597070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.082+0000 7f370f485640 1 -- 172.21.15.40:0/670448384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5635975b0 con 0x55c5636267a0 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.083+0000 7f370f485640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5636267a0 0x55c563623560 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f370000e7c0 tx=0x7f370000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.083+0000 7f3707fff640 1 -- 172.21.15.40:0/670448384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700016020 con 0x55c5636267a0 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.083+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c56359a3a0 con 0x55c5636267a0 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.083+0000 7f3707fff640 1 -- 172.21.15.40:0/670448384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3700005e20 con 0x55c5636267a0 2024-04-26T21:45:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.083+0000 7f3707fff640 1 -- 172.21.15.40:0/670448384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370000c8b0 con 0x55c5636267a0 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.083+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c563626e20 con 0x55c5636267a0 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.084+0000 7f3707fff640 1 -- 172.21.15.40:0/670448384 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3700019020 con 0x55c5636267a0 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.085+0000 7f3707fff640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36f006b2d0 0x7f36f006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.085+0000 7f3707fff640 1 -- 172.21.15.40:0/670448384 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f370008b690 con 0x55c5636267a0 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.085+0000 7f370ec84640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36f006b2d0 0x7f36f006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.085+0000 7f371118af40 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c563625cc0 0x55c563595bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.085+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55c563625160 con 0x55c563625cc0 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.085+0000 7f370e483640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c563625cc0 0x55c563595bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.086+0000 7f370ec84640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36f006b2d0 0x7f36f006d790 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3708000c00 tx=0x7f3708051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.086+0000 7f370e483640 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c563625cc0 0x55c563595bc0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f370e483640 1 -- 172.21.15.40:0/670448384 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 1.00000000 [omap-set-vals] v128'414 uv414 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f36f80098b0 con 0x55c563625cc0 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c563625cc0 msgr2=0x55c563595bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c563625cc0 0x55c563595bc0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36f006b2d0 msgr2=0x7f36f006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f36f006b2d0 0x7f36f006d790 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3708000c00 tx=0x7f3708051000 comp rx=0 tx=0).stop 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5636267a0 msgr2=0x55c563623560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 --2- 172.21.15.40:0/670448384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5636267a0 0x55c563623560 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f370000e7c0 tx=0x7f370000ec80 comp rx=0 tx=0).stop 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f370f485640 1 -- 172.21.15.40:0/670448384 reap_dead start 2024-04-26T21:45:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 shutdown_connections 2024-04-26T21:45:57.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 >> 172.21.15.40:0/670448384 conn(0x55c56357a9c0 msgr2=0x55c563629350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:57.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 shutdown_connections 2024-04-26T21:45:57.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.087+0000 7f371118af40 1 -- 172.21.15.40:0/670448384 wait complete. 2024-04-26T21:45:57.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:45:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.195+0000 7fe65c67e640 1 Processor -- start 2024-04-26T21:45:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.195+0000 7fe65c67e640 1 -- start start 2024-04-26T21:45:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.195+0000 7fe65c67e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe6540b0380 0x7fe6540ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.195+0000 7fe65c67e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 0x7fe65406c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.195+0000 7fe65c67e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe65406c780 0x7fe654072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.195+0000 7fe65c67e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe6540cfd80 con 0x7fe6540b0380 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.195+0000 7fe65c67e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe6540d0020 con 0x7fe65406c780 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.195+0000 7fe65c67e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6540d0300 con 0x7fe6540b10b0 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65a41a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe6540b0380 0x7fe6540ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65a41a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe6540b0380 0x7fe6540ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55624/0 (socket says 172.21.15.40:55624) 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65a41a640 1 -- 172.21.15.40:0/1751183948 learned_addr learned my addr 172.21.15.40:0/1751183948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe659c19640 1 --2- 172.21.15.40:0/1751183948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 0x7fe65406c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65ac1b640 1 --2- 172.21.15.40:0/1751183948 >> v2:172.21.15.184:3300/0 conn(0x7fe65406c780 0x7fe654072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65a41a640 1 -- 172.21.15.40:0/1751183948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 msgr2=0x7fe65406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65a41a640 1 --2- 172.21.15.40:0/1751183948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 0x7fe65406c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65a41a640 1 -- 172.21.15.40:0/1751183948 >> v2:172.21.15.184:3300/0 conn(0x7fe65406c780 msgr2=0x7fe654072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65a41a640 1 --2- 172.21.15.40:0/1751183948 >> v2:172.21.15.184:3300/0 conn(0x7fe65406c780 0x7fe654072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65a41a640 1 -- 172.21.15.40:0/1751183948 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6540d05e0 con 0x7fe6540b0380 2024-04-26T21:45:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.196+0000 7fe65ac1b640 1 --2- 172.21.15.40:0/1751183948 >> v2:172.21.15.184:3300/0 conn(0x7fe65406c780 0x7fe654072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe65a41a640 1 --2- 172.21.15.40:0/1751183948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6540b0380 0x7fe6540ccd50 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fe6540c57c0 tx=0x7fe65002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27d4570b2c1684b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe659418640 1 -- 172.21.15.40:0/1751183948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650037070 con 0x7fe6540b0380 2024-04-26T21:45:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe659418640 1 -- 172.21.15.40:0/1751183948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe650032440 con 0x7fe6540b0380 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe659418640 1 -- 172.21.15.40:0/1751183948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650032810 con 0x7fe6540b0380 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe65c67e640 1 -- 172.21.15.40:0/1751183948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6540b0380 msgr2=0x7fe6540ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe65c67e640 1 --2- 172.21.15.40:0/1751183948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6540b0380 0x7fe6540ccd50 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fe6540c57c0 tx=0x7fe65002a140 comp rx=0 tx=0).stop 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe65c67e640 1 -- 172.21.15.40:0/1751183948 shutdown_connections 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe65c67e640 1 --2- 172.21.15.40:0/1751183948 >> v2:172.21.15.184:3300/0 conn(0x7fe65406c780 0x7fe654072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe65c67e640 1 --2- 172.21.15.40:0/1751183948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 0x7fe65406c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe65c67e640 1 --2- 172.21.15.40:0/1751183948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6540b0380 0x7fe6540ccd50 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.197+0000 7fe65c67e640 1 -- 172.21.15.40:0/1751183948 >> 172.21.15.40:0/1751183948 conn(0x7fe6540d1cf0 msgr2=0x7fe6540d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.198+0000 7fe65c67e640 1 -- 172.21.15.40:0/1751183948 shutdown_connections 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.198+0000 7fe65c67e640 1 -- 172.21.15.40:0/1751183948 wait complete. 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65c67e640 1 Processor -- start 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65c67e640 1 -- start start 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65c67e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65406c780 0x7fe6540c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65c67e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6540b0380 0x7fe6540cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65c67e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 0x7fe6540c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65c67e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe6540d15c0 con 0x7fe65406c780 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65c67e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6540b0760 con 0x7fe6540b0380 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65c67e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6540d1740 con 0x7fe6540b10b0 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.199+0000 7fe65a41a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65406c780 0x7fe6540c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65a41a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65406c780 0x7fe6540c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55632/0 (socket says 172.21.15.40:55632) 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe659c19640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6540b0380 0x7fe6540cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65a41a640 1 -- 172.21.15.40:0/2544740730 learned_addr learned my addr 172.21.15.40:0/2544740730 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65ac1b640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 0x7fe6540c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65a41a640 1 -- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 msgr2=0x7fe6540c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65a41a640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 0x7fe6540c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65a41a640 1 -- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6540b0380 msgr2=0x7fe6540cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65a41a640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6540b0380 0x7fe6540cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65a41a640 1 -- 172.21.15.40:0/2544740730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6540cbcf0 con 0x7fe65406c780 2024-04-26T21:45:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65a41a640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65406c780 0x7fe6540c6230 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fe65002a180 tx=0x7fe6500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe64affd640 1 -- 172.21.15.40:0/2544740730 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650041070 con 0x7fe65406c780 2024-04-26T21:45:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe64affd640 1 -- 172.21.15.40:0/2544740730 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65003c870 con 0x7fe65406c780 2024-04-26T21:45:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6540c6880 con 0x7fe65406c780 2024-04-26T21:45:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe64affd640 1 -- 172.21.15.40:0/2544740730 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650037040 con 0x7fe65406c780 2024-04-26T21:45:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.200+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6540c8e60 con 0x7fe65406c780 2024-04-26T21:45:57.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.201+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6540d0b00 con 0x7fe65406c780 2024-04-26T21:45:57.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.202+0000 7fe64affd640 1 -- 172.21.15.40:0/2544740730 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe650044070 con 0x7fe65406c780 2024-04-26T21:45:57.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.202+0000 7fe64affd640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe63806c920 0x7fe63806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:57.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.202+0000 7fe64affd640 1 -- 172.21.15.40:0/2544740730 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fe6500ad660 con 0x7fe65406c780 2024-04-26T21:45:57.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.202+0000 7fe659c19640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe63806c920 0x7fe63806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:57.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.203+0000 7fe659c19640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe63806c920 0x7fe63806ede0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe6541f82e0 tx=0x7fe644008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.206+0000 7fe64affd640 1 -- 172.21.15.40:0/2544740730 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe6540d0b00 con 0x7fe65406c780 2024-04-26T21:45:57.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.436+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe6540d2960 con 0x7fe65406c780 2024-04-26T21:45:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.764+0000 7fe64affd640 1 -- 172.21.15.40:0/2544740730 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v299) ==== 162+0+0 (secure 0 0 0) 0x7fe6540d2960 con 0x7fe65406c780 2024-04-26T21:45:57.765 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:45:57.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe63806c920 msgr2=0x7fe63806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe63806c920 0x7fe63806ede0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe6541f82e0 tx=0x7fe644008040 comp rx=0 tx=0).stop 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65406c780 msgr2=0x7fe6540c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65406c780 0x7fe6540c6230 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fe65002a180 tx=0x7fe6500046c0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 shutdown_connections 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe63806c920 0x7fe63806ede0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6540b10b0 0x7fe6540c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6540b0380 0x7fe6540cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 --2- 172.21.15.40:0/2544740730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65406c780 0x7fe6540c6230 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 >> 172.21.15.40:0/2544740730 conn(0x7fe6540d1cf0 msgr2=0x7fe6540cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 shutdown_connections 2024-04-26T21:45:57.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:57.767+0000 7fe65c67e640 1 -- 172.21.15.40:0/2544740730 wait complete. 2024-04-26T21:45:58.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:45:58.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.297+0000 7f4043420640 1 Processor -- start 2024-04-26T21:45:58.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.297+0000 7f4043420640 1 -- start start 2024-04-26T21:45:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.298+0000 7f4043420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 0x7f403c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.298+0000 7f4043420640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f403c0d1b30 0x7f403c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.298+0000 7f4043420640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c0d10d0 con 0x7f403c0b2340 2024-04-26T21:45:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.298+0000 7f4043420640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f403c0d0750 con 0x7f403c0d1b30 2024-04-26T21:45:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.298+0000 7f4043420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f403c0d16a0 con 0x7f403c162810 2024-04-26T21:45:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.298+0000 7f40409bb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f403c0b2340 0x7f403c0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42232/0 (socket says 172.21.15.40:42232) 2024-04-26T21:45:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.298+0000 7f40409bb640 1 -- 172.21.15.40:0/1053994049 learned_addr learned my addr 172.21.15.40:0/1053994049 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:45:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1618634689 0 0) 0x7f403c0d10d0 con 0x7f403c0b2340 2024-04-26T21:45:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4024003520 con 0x7f403c0b2340 2024-04-26T21:45:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f40411bc640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 0x7f403c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f40419bd640 1 --2- 172.21.15.40:0/1053994049 >> v2:172.21.15.184:3300/0 conn(0x7f403c0d1b30 0x7f403c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3955371115 0 0) 0x7f4024003520 con 0x7f403c0b2340 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 msgr2=0x7f403c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 0x7f403c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 >> v2:172.21.15.184:3300/0 conn(0x7f403c0d1b30 msgr2=0x7f403c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 --2- 172.21.15.40:0/1053994049 >> v2:172.21.15.184:3300/0 conn(0x7f403c0d1b30 0x7f403c0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0d1370 con 0x7f403c0b2340 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f4030001670 con 0x7f403c0b2340 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f40411bc640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 0x7f403c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 >> v1:172.21.15.40:6789/0 conn(0x7f403c0b2340 legacy=0x7f403c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 0x7f40240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024003fa0 0x7f4024004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4024004d40 0x7f40240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0d0750 con 0x7f4024004d40 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0d16a0 con 0x7f4024003fa0 2024-04-26T21:45:58.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f402400baa0 con 0x7f403c162810 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f40419bd640 1 --2- 172.21.15.40:0/1053994049 >> v2:172.21.15.184:3300/0 conn(0x7f403c0d1b30 0x7f403c0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f40411bc640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 0x7f40240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f40419bd640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4024004d40 0x7f40240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.299+0000 7f40409bb640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024003fa0 0x7f4024004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f40419bd640 1 -- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 msgr2=0x7f40240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f40419bd640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 0x7f40240039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f40419bd640 1 -- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024003fa0 msgr2=0x7f4024004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f40419bd640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024003fa0 0x7f4024004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f40419bd640 1 -- 172.21.15.40:0/1053994049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4024003520 con 0x7f4024004d40 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f40411bc640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c162810 0x7f40240039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f40419bd640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4024004d40 0x7f40240051c0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f40340098a0 tx=0x7f4024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde2a7429f1f1675 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034037070 con 0x7f4024004d40 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403402ad20 con 0x7f4024004d40 2024-04-26T21:45:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.300+0000 7f403bfff640 1 -- 172.21.15.40:0/1053994049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403400f040 con 0x7f4024004d40 2024-04-26T21:45:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.301+0000 7f4043420640 1 -- 172.21.15.40:0/1053994049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4024004d40 msgr2=0x7f40240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.301+0000 7f4043420640 1 --2- 172.21.15.40:0/1053994049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4024004d40 0x7f40240051c0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f40340098a0 tx=0x7f4024005830 comp rx=0 tx=0).stop 2024-04-26T21:45:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.301+0000 7f40419bd640 1 -- 172.21.15.40:0/1053994049 reap_dead start 2024-04-26T21:45:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.301+0000 7f4043420640 1 -- 172.21.15.40:0/1053994049 shutdown_connections 2024-04-26T21:45:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.301+0000 7f4043420640 1 -- 172.21.15.40:0/1053994049 >> 172.21.15.40:0/1053994049 conn(0x7f403c0d2da0 msgr2=0x7f403c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.301+0000 7f4043420640 1 -- 172.21.15.40:0/1053994049 shutdown_connections 2024-04-26T21:45:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.301+0000 7f4043420640 1 -- 172.21.15.40:0/1053994049 wait complete. 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.302+0000 7f4043420640 1 Processor -- start 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.302+0000 7f4043420640 1 -- start start 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f4043420640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024004d40 0x7f403c1f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f4043420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0b2cc0 0x7f403c0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f4043420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403c0ca910 0x7f403c0cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f4043420640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0cdb00 con 0x7f403c0ca910 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f4043420640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0d0470 con 0x7f4024004d40 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f4043420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f403c070350 con 0x7f403c0b2cc0 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f40419bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403c0ca910 0x7f403c0cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f40419bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403c0ca910 0x7f403c0cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55658/0 (socket says 172.21.15.40:55658) 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f40419bd640 1 -- 172.21.15.40:0/350949196 learned_addr learned my addr 172.21.15.40:0/350949196 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.303+0000 7f40411bc640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024004d40 0x7f403c1f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f40409bb640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0b2cc0 0x7f403c0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f40419bd640 1 -- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0b2cc0 msgr2=0x7f403c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f40419bd640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0b2cc0 0x7f403c0b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f40419bd640 1 -- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024004d40 msgr2=0x7f403c1f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f40419bd640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024004d40 0x7f403c1f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f40419bd640 1 -- 172.21.15.40:0/350949196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0b3680 con 0x7f403c0ca910 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f40411bc640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024004d40 0x7f403c1f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f40419bd640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403c0ca910 0x7f403c0cad90 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f40340099c0 tx=0x7f4034004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f4039ffb640 1 -- 172.21.15.40:0/350949196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034037030 con 0x7f403c0ca910 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f4039ffb640 1 -- 172.21.15.40:0/350949196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4034041070 con 0x7f403c0ca910 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c0cb2d0 con 0x7f403c0ca910 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f4039ffb640 1 -- 172.21.15.40:0/350949196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c009c80 con 0x7f403c0ca910 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.304+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c0cbae0 con 0x7f403c0ca910 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.305+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c0694f0 con 0x7f403c0ca910 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.306+0000 7f4039ffb640 1 -- 172.21.15.40:0/350949196 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f403403b7f0 con 0x7f403c0ca910 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.306+0000 7f4039ffb640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f402806c8f0 0x7f402806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.306+0000 7f4039ffb640 1 -- 172.21.15.40:0/350949196 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f40340ac990 con 0x7f403c0ca910 2024-04-26T21:45:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.307+0000 7f40411bc640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f402806c8f0 0x7f402806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.307+0000 7f40411bc640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f402806c8f0 0x7f402806edb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f402c0095c0 tx=0x7f402c009820 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.311+0000 7f4039ffb640 1 -- 172.21.15.40:0/350949196 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f403400f000 con 0x7f403c0ca910 2024-04-26T21:45:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.535+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f403c1f4750 con 0x7f403c0ca910 2024-04-26T21:45:58.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.536+0000 7f4039ffb640 1 -- 172.21.15.40:0/350949196 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 300 v300) ==== 95+0+5193 (secure 0 0 0) 0x7f4034079fd0 con 0x7f403c0ca910 2024-04-26T21:45:58.536 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:45:58.536 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":300,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":300,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:45:57.767406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10914":{"gid":10914,"name":"c","rank":0,"incarnation":300,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/1753847972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1753847972},{"type":"v1","addr":"172.21.15.40:6833","nonce":1753847972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:45:58.536 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 300 2024-04-26T21:45:58.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f402806c8f0 msgr2=0x7f402806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f402806c8f0 0x7f402806edb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f402c0095c0 tx=0x7f402c009820 comp rx=0 tx=0).stop 2024-04-26T21:45:58.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403c0ca910 msgr2=0x7f403c0cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:58.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403c0ca910 0x7f403c0cad90 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f40340099c0 tx=0x7f4034004730 comp rx=0 tx=0).stop 2024-04-26T21:45:58.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 shutdown_connections 2024-04-26T21:45:58.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f402806c8f0 0x7f402806edb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403c0ca910 0x7f403c0cad90 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0b2cc0 0x7f403c0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 --2- 172.21.15.40:0/350949196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4024004d40 0x7f403c1f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 >> 172.21.15.40:0/350949196 conn(0x7f403c0d2da0 msgr2=0x7f403c0d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 shutdown_connections 2024-04-26T21:45:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:45:58.538+0000 7f4043420640 1 -- 172.21.15.40:0/350949196 wait complete. 2024-04-26T21:45:58.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 300, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:45:57.767406+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10914}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10914': {'gid': 10914, 'name': 'c', 'rank': 0, 'incarnation': 300, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/1753847972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1753847972}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 1753847972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:45:58.549 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10914:up:replay 2024-04-26T21:45:58.549 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:46:02.746 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:02.746+0000 7faee4b21640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:46:03.785 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:03.785+0000 7f86209e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-26T21:46:03.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:04.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.067+0000 7f661dc83640 1 Processor -- start 2024-04-26T21:46:04.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.067+0000 7f661dc83640 1 -- start start 2024-04-26T21:46:04.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.067+0000 7f661dc83640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f661815f3a0 0x7f66180d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.067+0000 7f661dc83640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66180d0e20 con 0x7f66180d1770 2024-04-26T21:46:04.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.067+0000 7f661dc83640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66180b1d70 con 0x7f66180d2330 2024-04-26T21:46:04.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.067+0000 7f661dc83640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f66180b2050 con 0x7f661815f3a0 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f660effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f66180d1770 0x7f66180d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44500/0 (socket says 172.21.15.40:44500) 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f660effd640 1 -- 172.21.15.40:0/404210935 learned_addr learned my addr 172.21.15.40:0/404210935 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f66177fe640 1 --2- 172.21.15.40:0/404210935 >> v2:172.21.15.184:3300/0 conn(0x7f661815f3a0 0x7f66180d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f6616ffd640 1 -- 172.21.15.40:0/404210935 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709156103 0 0) 0x7f66180d0e20 con 0x7f66180d1770 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f6616ffd640 1 -- 172.21.15.40:0/404210935 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6604003520 con 0x7f66180d1770 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f66177fe640 1 -- 172.21.15.40:0/404210935 >> v1:172.21.15.184:6789/0 conn(0x7f66180d2330 legacy=0x7f66180b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f6616ffd640 1 -- 172.21.15.40:0/404210935 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834427987 0 0) 0x7f66180b1d70 con 0x7f66180d2330 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f66177fe640 1 -- 172.21.15.40:0/404210935 >> v1:172.21.15.40:6789/0 conn(0x7f66180d1770 legacy=0x7f66180d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.068+0000 7f66177fe640 1 -- 172.21.15.40:0/404210935 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66180b2330 con 0x7f661815f3a0 2024-04-26T21:46:04.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.069+0000 7f66177fe640 1 --2- 172.21.15.40:0/404210935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f661815f3a0 0x7f66180d06f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f66080097a0 tx=0x7f660802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd3079867e35c3a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.069+0000 7f6616ffd640 1 -- 172.21.15.40:0/404210935 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608037070 con 0x7f661815f3a0 2024-04-26T21:46:04.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.069+0000 7f6616ffd640 1 -- 172.21.15.40:0/404210935 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660802ae50 con 0x7f661815f3a0 2024-04-26T21:46:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.069+0000 7f6616ffd640 1 -- 172.21.15.40:0/404210935 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608032660 con 0x7f661815f3a0 2024-04-26T21:46:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.070+0000 7f661dc83640 1 -- 172.21.15.40:0/404210935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f661815f3a0 msgr2=0x7f66180d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.070+0000 7f661dc83640 1 --2- 172.21.15.40:0/404210935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f661815f3a0 0x7f66180d06f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f66080097a0 tx=0x7f660802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.070+0000 7f661dc83640 1 -- 172.21.15.40:0/404210935 shutdown_connections 2024-04-26T21:46:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.070+0000 7f661dc83640 1 --2- 172.21.15.40:0/404210935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f661815f3a0 0x7f66180d06f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.070+0000 7f661dc83640 1 -- 172.21.15.40:0/404210935 >> 172.21.15.40:0/404210935 conn(0x7f66180d4f90 msgr2=0x7f66180d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.070+0000 7f661dc83640 1 -- 172.21.15.40:0/404210935 shutdown_connections 2024-04-26T21:46:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.070+0000 7f661dc83640 1 -- 172.21.15.40:0/404210935 wait complete. 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.071+0000 7f661dc83640 1 Processor -- start 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.071+0000 7f661dc83640 1 -- start start 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f661dc83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66180d1770 0x7f6618154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f661dc83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66180d2330 0x7f66180cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f661dc83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f661815f3a0 0x7f66180b9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f661dc83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f66180b30e0 con 0x7f66180d2330 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f661dc83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66180b2f60 con 0x7f66180d1770 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f661dc83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66180b3260 con 0x7f661815f3a0 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f660effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66180d2330 0x7f66180cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f660effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66180d2330 0x7f66180cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39824/0 (socket says 172.21.15.40:39824) 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f660effd640 1 -- 172.21.15.40:0/471109409 learned_addr learned my addr 172.21.15.40:0/471109409 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.072+0000 7f66177fe640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66180d1770 0x7f6618154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f6617fff640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f661815f3a0 0x7f66180b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f660effd640 1 -- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f661815f3a0 msgr2=0x7f66180b9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f660effd640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f661815f3a0 0x7f66180b9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f660effd640 1 -- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66180d1770 msgr2=0x7f6618154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f660effd640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66180d1770 0x7f6618154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f660effd640 1 -- 172.21.15.40:0/471109409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66181604a0 con 0x7f66180d2330 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f6617fff640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f661815f3a0 0x7f66180b9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f660effd640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66180d2330 0x7f66180cb560 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f660000c7b0 tx=0x7f660000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f6614ff9640 1 -- 172.21.15.40:0/471109409 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600018070 con 0x7f66180d2330 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f6614ff9640 1 -- 172.21.15.40:0/471109409 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66000074e0 con 0x7f66180d2330 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f6614ff9640 1 -- 172.21.15.40:0/471109409 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660000ce60 con 0x7f66180d2330 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6618154940 con 0x7f66180d2330 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.073+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66180c9c40 con 0x7f66180d2330 2024-04-26T21:46:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.074+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66180694f0 con 0x7f66180d2330 2024-04-26T21:46:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.075+0000 7f6614ff9640 1 -- 172.21.15.40:0/471109409 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f660001b070 con 0x7f66180d2330 2024-04-26T21:46:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.075+0000 7f6614ff9640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f65f806c8f0 0x7f65f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.075+0000 7f6614ff9640 1 -- 172.21.15.40:0/471109409 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f660008be40 con 0x7f66180d2330 2024-04-26T21:46:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.075+0000 7f66177fe640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f65f806c8f0 0x7f65f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.076+0000 7f66177fe640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f65f806c8f0 0x7f65f806edb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6608034000 tx=0x7f660802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.080+0000 7f6614ff9640 1 -- 172.21.15.40:0/471109409 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66181604a0 con 0x7f66180d2330 2024-04-26T21:46:04.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.307+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66180b4140 con 0x7f66180d2330 2024-04-26T21:46:04.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.307+0000 7f6614ff9640 1 -- 172.21.15.40:0/471109409 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 304 v304) ==== 95+0+5202 (secure 0 0 0) 0x7f6600059510 con 0x7f66180d2330 2024-04-26T21:46:04.307 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:04.307 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":304,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":304,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:03.782753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10914":{"gid":10914,"name":"c","rank":0,"incarnation":300,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6833/1753847972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1753847972},{"type":"v1","addr":"172.21.15.40:6833","nonce":1753847972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10914,"qdb_cluster":[10914]},"id":6}]} 2024-04-26T21:46:04.308 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 304 2024-04-26T21:46:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f65f806c8f0 msgr2=0x7f65f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f65f806c8f0 0x7f65f806edb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6608034000 tx=0x7f660802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66180d2330 msgr2=0x7f66180cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66180d2330 0x7f66180cb560 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f660000c7b0 tx=0x7f660000cc70 comp rx=0 tx=0).stop 2024-04-26T21:46:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 shutdown_connections 2024-04-26T21:46:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f65f806c8f0 0x7f65f806edb0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f661815f3a0 0x7f66180b9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66180d2330 0x7f66180cb560 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 --2- 172.21.15.40:0/471109409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66180d1770 0x7f6618154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 >> 172.21.15.40:0/471109409 conn(0x7f66180d4f90 msgr2=0x7f66180b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 shutdown_connections 2024-04-26T21:46:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.310+0000 7f661dc83640 1 -- 172.21.15.40:0/471109409 wait complete. 2024-04-26T21:46:04.321 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 304, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:46:03.782753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10914}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10914': {'gid': 10914, 'name': 'c', 'rank': 0, 'incarnation': 300, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.40:6833/1753847972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1753847972}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 1753847972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10914, 'qdb_cluster': [10914]} 2024-04-26T21:46:04.321 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:46:04.321 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:46:04.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.422+0000 7fafdd3bf640 1 Processor -- start 2024-04-26T21:46:04.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.422+0000 7fafdd3bf640 1 -- start start 2024-04-26T21:46:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.423+0000 7fafdd3bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fafd80d3bc0 0x7fafd80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.423+0000 7fafdd3bf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd80b7e50 con 0x7fafd80b5f20 2024-04-26T21:46:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.423+0000 7fafdd3bf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd80d1800 con 0x7fafd8162880 2024-04-26T21:46:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.423+0000 7fafdd3bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fafd80d0580 con 0x7fafd80d3bc0 2024-04-26T21:46:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.423+0000 7fafd77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fafd80b5f20 0x7fafd80b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44510/0 (socket says 172.21.15.40:44510) 2024-04-26T21:46:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.423+0000 7fafd77fe640 1 -- 172.21.15.40:0/1465898845 learned_addr learned my addr 172.21.15.40:0/1465898845 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.423+0000 7fafd67fc640 1 --2- 172.21.15.40:0/1465898845 >> v2:172.21.15.184:3300/0 conn(0x7fafd80d3bc0 0x7fafd80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.423+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2891343861 0 0) 0x7fafd80b7e50 con 0x7fafd80b5f20 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc4003520 con 0x7fafd80b5f20 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 841663220 0 0) 0x7fafd0001110 con 0x7fafd8162880 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafd80b7e50 con 0x7fafd8162880 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 436839822 0 0) 0x7fafc4003520 con 0x7fafd80b5f20 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 >> v2:172.21.15.184:3300/0 conn(0x7fafd80d3bc0 msgr2=0x7fafd80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 --2- 172.21.15.40:0/1465898845 >> v2:172.21.15.184:3300/0 conn(0x7fafd80d3bc0 0x7fafd80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 >> v1:172.21.15.184:6789/0 conn(0x7fafd8162880 legacy=0x7fafd80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd80d07d0 con 0x7fafd80b5f20 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fafc80019f0 con 0x7fafd80b5f20 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 >> v1:172.21.15.40:6789/0 conn(0x7fafd80b5f20 legacy=0x7fafd80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc4003b90 0x7fafc4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc4004800 0x7fafc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafd80d0580 con 0x7fafc40051c0 2024-04-26T21:46:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafc400c090 con 0x7fafc4003b90 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafc400c290 con 0x7fafc4004800 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc4003b90 0x7fafc4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd77fe640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd67fc640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc4004800 0x7fafc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd77fe640 1 -- 172.21.15.40:0/1465898845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc4004800 msgr2=0x7fafc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd77fe640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc4004800 0x7fafc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd77fe640 1 -- 172.21.15.40:0/1465898845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc4003b90 msgr2=0x7fafc4003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd77fe640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc4003b90 0x7fafc4003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.424+0000 7fafd77fe640 1 -- 172.21.15.40:0/1465898845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc4003520 con 0x7fafc40051c0 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.425+0000 7fafd77fe640 1 -- 172.21.15.40:0/1465898845 reap_dead start 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.425+0000 7fafd77fe640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafc400b9c0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fafc800a730 tx=0x7fafc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd9e00834a13655 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.425+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc800a350 con 0x7fafc40051c0 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.425+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc802f070 con 0x7fafc40051c0 2024-04-26T21:46:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.425+0000 7fafd5ffb640 1 -- 172.21.15.40:0/1465898845 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc802bb70 con 0x7fafc40051c0 2024-04-26T21:46:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.426+0000 7fafdd3bf640 1 -- 172.21.15.40:0/1465898845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 msgr2=0x7fafc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.426+0000 7fafdd3bf640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafc400b9c0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fafc800a730 tx=0x7fafc4005830 comp rx=0 tx=0).stop 2024-04-26T21:46:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.426+0000 7fafdd3bf640 1 -- 172.21.15.40:0/1465898845 shutdown_connections 2024-04-26T21:46:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.426+0000 7fafdd3bf640 1 --2- 172.21.15.40:0/1465898845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafc400b9c0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.426+0000 7fafdd3bf640 1 -- 172.21.15.40:0/1465898845 >> 172.21.15.40:0/1465898845 conn(0x7fafd80d4f60 msgr2=0x7fafd80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.426+0000 7fafdd3bf640 1 -- 172.21.15.40:0/1465898845 shutdown_connections 2024-04-26T21:46:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.426+0000 7fafdd3bf640 1 -- 172.21.15.40:0/1465898845 wait complete. 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.427+0000 7fafdd3bf640 1 Processor -- start 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.427+0000 7fafdd3bf640 1 -- start start 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafdd3bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafd81f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafdd3bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd81618f0 0x7fafd8153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafdd3bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd8161d70 0x7fafd8154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafdd3bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8162c60 con 0x7fafc40051c0 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafdd3bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafd80d0d90 con 0x7fafd81618f0 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafdd3bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafd8162de0 con 0x7fafd8161d70 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafd81f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafd81f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39848/0 (socket says 172.21.15.40:39848) 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd6ffd640 1 -- 172.21.15.40:0/2799299622 learned_addr learned my addr 172.21.15.40:0/2799299622 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd77fe640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd8161d70 0x7fafd8154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd67fc640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd81618f0 0x7fafd8153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd6ffd640 1 -- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd8161d70 msgr2=0x7fafd8154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd8161d70 0x7fafd8154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd6ffd640 1 -- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd81618f0 msgr2=0x7fafd8153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd81618f0 0x7fafd8153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd6ffd640 1 -- 172.21.15.40:0/2799299622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd81f75d0 con 0x7fafc40051c0 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.428+0000 7fafd67fc640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd81618f0 0x7fafd8153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.429+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafd81f7090 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fafd000f800 tx=0x7fafd000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.429+0000 7fafcf7fe640 1 -- 172.21.15.40:0/2799299622 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd001a070 con 0x7fafc40051c0 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.429+0000 7fafcf7fe640 1 -- 172.21.15.40:0/2799299622 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd0005320 con 0x7fafc40051c0 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.429+0000 7fafdd3bf640 1 -- 172.21.15.40:0/2799299622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd81f78a0 con 0x7fafc40051c0 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.429+0000 7fafcf7fe640 1 -- 172.21.15.40:0/2799299622 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0007270 con 0x7fafc40051c0 2024-04-26T21:46:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.429+0000 7fafdd3bf640 1 -- 172.21.15.40:0/2799299622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd81f7e60 con 0x7fafc40051c0 2024-04-26T21:46:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.430+0000 7fafcf7fe640 1 -- 172.21.15.40:0/2799299622 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafd0007410 con 0x7fafc40051c0 2024-04-26T21:46:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.431+0000 7fafcf7fe640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafb806c920 0x7fafb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.431+0000 7fafcf7fe640 1 -- 172.21.15.40:0/2799299622 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fafd008c190 con 0x7fafc40051c0 2024-04-26T21:46:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.431+0000 7fafd67fc640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafb806c920 0x7fafb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.432+0000 7fafd67fc640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafb806c920 0x7fafb806ede0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fafc00093d0 tx=0x7fafc0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.432+0000 7fafe01bc740 1 Processor -- start 2024-04-26T21:46:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.432+0000 7fafe01bc740 1 -- start start 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ef7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f15230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ecd120 0x55e035f133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafe01bc740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e035e31740 con 0x55e035ef7440 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafe01bc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e035dcd3a0 con 0x55e035ecd120 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafe01bc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e035e2d700 con 0x55e035f14ce0 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ef7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ef7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39850/0 (socket says 172.21.15.40:39850) 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd6ffd640 1 -- 172.21.15.40:0/582796231 learned_addr learned my addr 172.21.15.40:0/582796231 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd67fc640 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f15230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd77fe640 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ecd120 0x55e035f133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd6ffd640 1 -- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 msgr2=0x55e035f15230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f15230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd6ffd640 1 -- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ecd120 msgr2=0x55e035f133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ecd120 0x55e035f133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd6ffd640 1 -- 172.21.15.40:0/582796231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e035de9750 con 0x55e035ef7440 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.433+0000 7fafd77fe640 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ecd120 0x55e035f133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ef7820 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fafd00555c0 tx=0x7fafd001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfe7daf3be3a039 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafccff9640 1 -- 172.21.15.40:0/582796231 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0059160 con 0x55e035ef7440 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafccff9640 1 -- 172.21.15.40:0/582796231 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd005d510 con 0x55e035ef7440 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafccff9640 1 -- 172.21.15.40:0/582796231 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0061d40 con 0x55e035ef7440 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafe01bc740 1 -- 172.21.15.40:0/582796231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 msgr2=0x55e035ef7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafe01bc740 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ef7820 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fafd00555c0 tx=0x7fafd001c000 comp rx=0 tx=0).stop 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafe01bc740 1 -- 172.21.15.40:0/582796231 shutdown_connections 2024-04-26T21:46:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafe01bc740 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ecd120 0x55e035f133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafe01bc740 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f15230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafe01bc740 1 --2- 172.21.15.40:0/582796231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ef7820 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.434+0000 7fafe01bc740 1 -- 172.21.15.40:0/582796231 >> 172.21.15.40:0/582796231 conn(0x55e035ec9a50 msgr2=0x55e035f12250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.435+0000 7fafe01bc740 1 -- 172.21.15.40:0/582796231 shutdown_connections 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.435+0000 7fafe01bc740 1 -- 172.21.15.40:0/582796231 wait complete. 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.435+0000 7fafe01bc740 1 -- start start 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.435+0000 7fafe01bc740 1 Processor -- start 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035efb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f26460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f0a640 0x55e035ef83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafe01bc740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e035ecdec0 con 0x55e035f0a640 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafe01bc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e035db8fe0 con 0x55e035ef7440 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafe01bc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e035ec7980 con 0x55e035f14ce0 2024-04-26T21:46:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafe01bc740 10 client.0 _ll_register_callbacks cb 0x55e035dcf150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f0a640 0x55e035ef83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f0a640 0x55e035ef83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39866/0 (socket says 172.21.15.40:39866) 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 -- 172.21.15.40:0/211222447 learned_addr learned my addr 172.21.15.40:0/211222447 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd67fc640 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f26460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035efb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 -- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 msgr2=0x55e035f26460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f26460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 -- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 msgr2=0x55e035efb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035efb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 -- 172.21.15.40:0/211222447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e035b432d0 con 0x55e035f0a640 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035efb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafd77fe640 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f0a640 0x55e035ef83f0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fafc800a820 tx=0x7fafc802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341d07f2eb765dff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.436+0000 7fafb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:46:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafb5ffb640 1 -- 172.21.15.40:0/211222447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8001ea0 con 0x55e035f0a640 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafb5ffb640 1 -- 172.21.15.40:0/211222447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fafc8004d60 con 0x55e035f0a640 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 1 -- 172.21.15.40:0/211222447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd005d510 con 0x55e035f0a640 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafb5ffb640 1 -- 172.21.15.40:0/211222447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=304) ==== 24+0+0 (secure 0 0 0) 0x7fafc802b940 con 0x55e035f0a640 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 10 client.10992 fetch_fsmap learned FSMap version 304 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 1 -- 172.21.15.40:0/211222447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=304,monmap=2+}) -- 0x55e035c0d6b0 con 0x55e035f0a640 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafb5ffb640 1 -- 172.21.15.40:0/211222447 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd005d510 con 0x55e035f0a640 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafb5ffb640 1 -- 172.21.15.40:0/211222447 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 304) ==== 3494+0+0 (secure 0 0 0) 0x7fafc802bae0 con 0x55e035f0a640 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 10 client.10992 fetch_fsmap finished waiting for FSMap version 304 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 10 client.10992 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 20 client.10992 populate_metadata read hostname 'smithi040' 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x55e035f229b0 0x55e035f22d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 4 client.10992 mds_command: new command op to 10914 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafe01bc740 1 -- 172.21.15.40:0/211222447 --> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e035e3f350 con 0x55e035f229b0 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.437+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x55e035f229b0 0x55e035f22d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x55e035f229b0 0x55e035f22d90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd8ec7021b4577a server_cookie=faee9c01a2d1f21c in_seq=0 out_seq=0 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafb5ffb640 10 client.10992 ms_handle_connect on v2:172.21.15.40:6832/1753847972 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafb5ffb640 1 -- 172.21.15.40:0/211222447 <== mds.0 v2:172.21.15.40:6832/1753847972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e035e3f350 con 0x55e035f229b0 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafb5ffb640 10 client.10992 handle_command_reply: tid=0 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 client.10992 shutdown 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 -- 172.21.15.40:0/211222447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f0a640 msgr2=0x55e035ef83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f0a640 0x55e035ef83f0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fafc800a820 tx=0x7fafc802d070 comp rx=0 tx=0).stop 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 -- 172.21.15.40:0/211222447 shutdown_connections 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x55e035f229b0 0x55e035f22d90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f0a640 0x55e035ef83f0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f26460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafb5ffb640 0 client.10992 ms_handle_reset on v2:172.21.15.40:6832/1753847972 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 --2- 172.21.15.40:0/211222447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035efb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 -- 172.21.15.40:0/211222447 >> 172.21.15.40:0/211222447 conn(0x55e035ecd120 msgr2=0x55e035f157d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 -- 172.21.15.40:0/211222447 shutdown_connections 2024-04-26T21:46:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.438+0000 7fafe01bc740 1 -- 172.21.15.40:0/211222447 wait complete. 2024-04-26T21:46:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.439+0000 7fafe01bc740 20 client.10992 trim_cache size 0 max 16384 2024-04-26T21:46:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.457+0000 7fafe01bc740 1 Processor -- start 2024-04-26T21:46:04.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.457+0000 7fafe01bc740 1 -- start start 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.457+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ecd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035f0a640 0x55e035f207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f20ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafe01bc740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8203340 con 0x55e035ef7440 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafe01bc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8202640 con 0x55e035f0a640 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafe01bc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafd8202c90 con 0x55e035f14ce0 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ecd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ecd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39876/0 (socket says 172.21.15.40:39876) 2024-04-26T21:46:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 -- 172.21.15.40:0/2662785150 learned_addr learned my addr 172.21.15.40:0/2662785150 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd67fc640 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035f0a640 0x55e035f207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd77fe640 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f20ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 -- 172.21.15.40:0/2662785150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 msgr2=0x55e035f20ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f20ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 -- 172.21.15.40:0/2662785150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035f0a640 msgr2=0x55e035f207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035f0a640 0x55e035f207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 -- 172.21.15.40:0/2662785150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e035ec8170 con 0x55e035ef7440 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ecd6f0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fafd0058e60 tx=0x7fafd005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a476e05b5f2e2131 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafb7fff640 1 -- 172.21.15.40:0/2662785150 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0057460 con 0x55e035ef7440 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafb7fff640 1 -- 172.21.15.40:0/2662785150 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd005d1b0 con 0x55e035ef7440 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.458+0000 7fafb7fff640 1 -- 172.21.15.40:0/2662785150 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd006db70 con 0x55e035ef7440 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 -- 172.21.15.40:0/2662785150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 msgr2=0x55e035ecd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ecd6f0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fafd0058e60 tx=0x7fafd005b000 comp rx=0 tx=0).stop 2024-04-26T21:46:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 -- 172.21.15.40:0/2662785150 shutdown_connections 2024-04-26T21:46:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f14ce0 0x55e035f20ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035f0a640 0x55e035f207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 --2- 172.21.15.40:0/2662785150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035ef7440 0x55e035ecd6f0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 -- 172.21.15.40:0/2662785150 >> 172.21.15.40:0/2662785150 conn(0x55e035ecd120 msgr2=0x55e035efae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 -- 172.21.15.40:0/2662785150 shutdown_connections 2024-04-26T21:46:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 -- 172.21.15.40:0/2662785150 wait complete. 2024-04-26T21:46:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 -- start start 2024-04-26T21:46:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.459+0000 7fafe01bc740 1 Processor -- start 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035f8c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f0a640 0x55e035f07be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafe01bc740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f14ce0 0x55e035f08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafe01bc740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e035f269a0 con 0x55e035f14ce0 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafe01bc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e035efb650 con 0x55e035ef7440 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafe01bc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e035f0b6d0 con 0x55e035f0a640 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafe01bc740 10 client.0 _ll_register_callbacks cb 0x7fafc40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f14ce0 0x55e035f08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f14ce0 0x55e035f08120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39878/0 (socket says 172.21.15.40:39878) 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd77fe640 1 -- 172.21.15.40:0/1911778724 learned_addr learned my addr 172.21.15.40:0/1911778724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035f8c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd67fc640 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f0a640 0x55e035f07be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd77fe640 1 -- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f0a640 msgr2=0x55e035f07be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd77fe640 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f0a640 0x55e035f07be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd77fe640 1 -- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 msgr2=0x55e035f8c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd77fe640 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035f8c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd77fe640 1 -- 172.21.15.40:0/1911778724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e035f22310 con 0x55e035f14ce0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.460+0000 7fafd67fc640 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f0a640 0x55e035f07be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafd77fe640 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f14ce0 0x55e035f08120 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fafc8000f80 tx=0x7fafc802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9522c5ffeacfef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafb5ffb640 1 -- 172.21.15.40:0/1911778724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8001ea0 con 0x55e035f14ce0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 1 -- 172.21.15.40:0/1911778724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd005d1b0 con 0x55e035f14ce0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafb5ffb640 1 -- 172.21.15.40:0/1911778724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fafc8034880 con 0x55e035f14ce0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafb5ffb640 1 -- 172.21.15.40:0/1911778724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=304) ==== 24+0+0 (secure 0 0 0) 0x7fafc8034a80 con 0x55e035f14ce0 2024-04-26T21:46:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 10 client.10998 fetch_fsmap learned FSMap version 304 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 1 -- 172.21.15.40:0/1911778724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=304,monmap=2+}) -- 0x55e035f22310 con 0x55e035f14ce0 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafb5ffb640 1 -- 172.21.15.40:0/1911778724 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd005d1b0 con 0x55e035f14ce0 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafb5ffb640 1 -- 172.21.15.40:0/1911778724 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 304) ==== 3494+0+0 (secure 0 0 0) 0x7fafc8034c20 con 0x55e035f14ce0 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 10 client.10998 fetch_fsmap finished waiting for FSMap version 304 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 10 client.10998 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 20 client.10998 populate_metadata read hostname 'smithi040' 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x55e035f210f0 0x55e035f8d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 4 client.10998 mds_command: new command op to 10914 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafe01bc740 1 -- 172.21.15.40:0/1911778724 --> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] -- command(tid 0: {"prefix": "status"}) -- 0x55e035e391c0 con 0x55e035f210f0 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.461+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x55e035f210f0 0x55e035f8d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafb5ffb640 10 client.10998 ms_handle_connect on v2:172.21.15.40:6832/1753847972 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x55e035f210f0 0x55e035f8d1f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22cfffaa7ff71cb server_cookie=5c0ed55cb379a8e9 in_seq=0 out_seq=0 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafb5ffb640 1 -- 172.21.15.40:0/1911778724 <== mds.0 v2:172.21.15.40:6832/1753847972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55e035e391c0 con 0x55e035f210f0 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafb5ffb640 10 client.10998 handle_command_reply: tid=0 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 client.10998 shutdown 2024-04-26T21:46:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 -- 172.21.15.40:0/1911778724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f14ce0 msgr2=0x55e035f08120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f14ce0 0x55e035f08120 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fafc8000f80 tx=0x7fafc802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 -- 172.21.15.40:0/1911778724 shutdown_connections 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x55e035f210f0 0x55e035f8d1f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e035f14ce0 0x55e035f08120 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafb5ffb640 0 client.10998 ms_handle_reset on v2:172.21.15.40:6832/1753847972 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e035f0a640 0x55e035f07be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 --2- 172.21.15.40:0/1911778724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e035ef7440 0x55e035f8c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 -- 172.21.15.40:0/1911778724 >> 172.21.15.40:0/1911778724 conn(0x55e035ecd120 msgr2=0x55e035f219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 -- 172.21.15.40:0/1911778724 shutdown_connections 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 1 -- 172.21.15.40:0/1911778724 wait complete. 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.462+0000 7fafe01bc740 20 client.10998 trim_cache size 0 max 16384 2024-04-26T21:46:04.468 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 10914, 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.6930729549999999, 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 304, 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 128, 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 128, 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 12.691226632999999 2024-04-26T21:46:04.469 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:46:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 -- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafb806c920 msgr2=0x7fafb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafb806c920 0x7fafb806ede0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fafc00093d0 tx=0x7fafc0008040 comp rx=0 tx=0).stop 2024-04-26T21:46:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 -- 172.21.15.40:0/2799299622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 msgr2=0x7fafd81f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafd81f7090 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fafd000f800 tx=0x7fafd000fcc0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 -- 172.21.15.40:0/2799299622 shutdown_connections 2024-04-26T21:46:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fafb806c920 0x7fafb806ede0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafd8161d70 0x7fafd8154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd81618f0 0x7fafd8153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 --2- 172.21.15.40:0/2799299622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc40051c0 0x7fafd81f7090 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 -- 172.21.15.40:0/2799299622 >> 172.21.15.40:0/2799299622 conn(0x7fafd80d4f60 msgr2=0x7fafd80d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 -- 172.21.15.40:0/2799299622 shutdown_connections 2024-04-26T21:46:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.463+0000 7fafdd3bf640 1 -- 172.21.15.40:0/2799299622 wait complete. 2024-04-26T21:46:04.473 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:46:04.473 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 1.00000000:subdir_head', proceeding to ls 2024-04-26T21:46:04.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:46:04.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.573+0000 7fc6377f0640 1 Processor -- start 2024-04-26T21:46:04.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.574+0000 7fc6377f0640 1 -- start start 2024-04-26T21:46:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.574+0000 7fc6377f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc6300b3860 0x7fc63016a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.574+0000 7fc6377f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6300d0ba0 con 0x7fc6300d06f0 2024-04-26T21:46:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.574+0000 7fc6377f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6300d2590 con 0x7fc63015f3a0 2024-04-26T21:46:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.574+0000 7fc6377f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc6300d0e20 con 0x7fc6300b3860 2024-04-26T21:46:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc634d8b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc6300d06f0 0x7fc6300d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44526/0 (socket says 172.21.15.40:44526) 2024-04-26T21:46:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc634d8b640 1 -- 172.21.15.40:0/1769700798 learned_addr learned my addr 172.21.15.40:0/1769700798 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc635d8d640 1 --2- 172.21.15.40:0/1769700798 >> v2:172.21.15.184:3300/0 conn(0x7fc6300b3860 0x7fc63016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc62ffff640 1 -- 172.21.15.40:0/1769700798 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3055982498 0 0) 0x7fc624001110 con 0x7fc6300d06f0 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc62ffff640 1 -- 172.21.15.40:0/1769700798 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc618003520 con 0x7fc6300d06f0 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc62ffff640 1 -- 172.21.15.40:0/1769700798 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2096429216 0 0) 0x7fc618003520 con 0x7fc6300d06f0 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc635d8d640 1 -- 172.21.15.40:0/1769700798 >> v1:172.21.15.184:6789/0 conn(0x7fc63015f3a0 legacy=0x7fc6300d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc635d8d640 1 -- 172.21.15.40:0/1769700798 >> v1:172.21.15.40:6789/0 conn(0x7fc6300d06f0 legacy=0x7fc6300d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.575+0000 7fc635d8d640 1 -- 172.21.15.40:0/1769700798 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6300b1d70 con 0x7fc6300b3860 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.576+0000 7fc635d8d640 1 --2- 172.21.15.40:0/1769700798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6300b3860 0x7fc63016a000 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc6280097a0 tx=0x7fc62802a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2563974a4535430a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.576+0000 7fc62ffff640 1 -- 172.21.15.40:0/1769700798 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628037070 con 0x7fc6300b3860 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.576+0000 7fc62ffff640 1 -- 172.21.15.40:0/1769700798 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62802aad0 con 0x7fc6300b3860 2024-04-26T21:46:04.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.576+0000 7fc62ffff640 1 -- 172.21.15.40:0/1769700798 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624001840 con 0x7fc6300b3860 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.577+0000 7fc6377f0640 1 -- 172.21.15.40:0/1769700798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6300b3860 msgr2=0x7fc63016a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.577+0000 7fc6377f0640 1 --2- 172.21.15.40:0/1769700798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6300b3860 0x7fc63016a000 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc6280097a0 tx=0x7fc62802a040 comp rx=0 tx=0).stop 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.577+0000 7fc6377f0640 1 -- 172.21.15.40:0/1769700798 shutdown_connections 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.577+0000 7fc6377f0640 1 --2- 172.21.15.40:0/1769700798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6300b3860 0x7fc63016a000 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.577+0000 7fc6377f0640 1 -- 172.21.15.40:0/1769700798 >> 172.21.15.40:0/1769700798 conn(0x7fc6300d4f90 msgr2=0x7fc6300d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.577+0000 7fc6377f0640 1 -- 172.21.15.40:0/1769700798 shutdown_connections 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.577+0000 7fc6377f0640 1 -- 172.21.15.40:0/1769700798 wait complete. 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.578+0000 7fc6377f0640 1 Processor -- start 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.578+0000 7fc6377f0640 1 -- start start 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc6377f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6300b3860 0x7fc6300cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc6377f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6300d06f0 0x7fc6300ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc6377f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63015f3a0 0x7fc6300b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc6377f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc63015f780 con 0x7fc63015f3a0 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc6377f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6300b2330 con 0x7fc6300d06f0 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc6377f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc63015f900 con 0x7fc6300b3860 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63015f3a0 0x7fc6300b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63015f3a0 0x7fc6300b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39886/0 (socket says 172.21.15.40:39886) 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 -- 172.21.15.40:0/1801155150 learned_addr learned my addr 172.21.15.40:0/1801155150 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 -- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6300b3860 msgr2=0x7fc6300cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6300b3860 0x7fc6300cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 -- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6300d06f0 msgr2=0x7fc6300ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6300d06f0 0x7fc6300ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 -- 172.21.15.40:0/1801155150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6300c9ff0 con 0x7fc63015f3a0 2024-04-26T21:46:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc635d8d640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63015f3a0 0x7fc6300b47a0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098c0 tx=0x7fc6280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc62dffb640 1 -- 172.21.15.40:0/1801155150 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628037040 con 0x7fc63015f3a0 2024-04-26T21:46:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc62dffb640 1 -- 172.21.15.40:0/1801155150 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc628004030 con 0x7fc63015f3a0 2024-04-26T21:46:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6300ca2d0 con 0x7fc63015f3a0 2024-04-26T21:46:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.579+0000 7fc62dffb640 1 -- 172.21.15.40:0/1801155150 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628006e10 con 0x7fc63015f3a0 2024-04-26T21:46:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.580+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6300ca890 con 0x7fc63015f3a0 2024-04-26T21:46:04.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.581+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6300cc110 con 0x7fc63015f3a0 2024-04-26T21:46:04.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.581+0000 7fc62dffb640 1 -- 172.21.15.40:0/1801155150 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc62803a040 con 0x7fc63015f3a0 2024-04-26T21:46:04.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.581+0000 7fc62dffb640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc61c06c8f0 0x7fc61c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:04.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.582+0000 7fc62dffb640 1 -- 172.21.15.40:0/1801155150 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fc6280acbb0 con 0x7fc63015f3a0 2024-04-26T21:46:04.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.582+0000 7fc63558c640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc61c06c8f0 0x7fc61c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:04.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.582+0000 7fc63558c640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc61c06c8f0 0x7fc61c06edb0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc620001d10 tx=0x7fc620007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:04.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.586+0000 7fc62dffb640 1 -- 172.21.15.40:0/1801155150 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc628042020 con 0x7fc63015f3a0 2024-04-26T21:46:04.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.812+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc6300694f0 con 0x7fc63015f3a0 2024-04-26T21:46:04.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.812+0000 7fc62dffb640 1 -- 172.21.15.40:0/1801155150 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v304) ==== 53+0+73 (secure 0 0 0) 0x7fc62807a430 con 0x7fc63015f3a0 2024-04-26T21:46:04.812 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:46:04.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc61c06c8f0 msgr2=0x7fc61c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc61c06c8f0 0x7fc61c06edb0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc620001d10 tx=0x7fc620007cd0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63015f3a0 msgr2=0x7fc6300b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63015f3a0 0x7fc6300b47a0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098c0 tx=0x7fc6280046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 shutdown_connections 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc61c06c8f0 0x7fc61c06edb0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63015f3a0 0x7fc6300b47a0 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6300d06f0 0x7fc6300ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 --2- 172.21.15.40:0/1801155150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6300b3860 0x7fc6300cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 >> 172.21.15.40:0/1801155150 conn(0x7fc6300d4f90 msgr2=0x7fc63006e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 shutdown_connections 2024-04-26T21:46:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:04.814+0000 7fc6377f0640 1 -- 172.21.15.40:0/1801155150 wait complete. 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:46:04.825 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:04.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:04.826 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:46:04.841 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:04.842 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42083sec preferred_lft 42083sec 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:04.843 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:46:04.844 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:04.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:04.845 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:46:04.904 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:46:04.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:04.906 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:46:04.962 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:46:04.962 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:46:04.962 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:46:04.962 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:46:04.962 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:46:04.963 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:04.963 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:46:04.963 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:46:04.963 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:46:04.963 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:46:04.963 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:46:05.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:05.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:05.088 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:46:05.104 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:46:05.104 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:46:05.104 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:05.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:05.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:46:05.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:05.168 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:05.224 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:05.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:05.224 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:05.279 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:46:05.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:05.280 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:05.362 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:05.362 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:46:05.371 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:05.371 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:46:05.371 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:46:05.372 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:46:05.372 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:46:05.372 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:46:05.372 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:46:05.372 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:46:05.372 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:46:05.372 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:46:05.373 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:46:05.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:05.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:05.445 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:46:05.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:46:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:05.542 INFO:teuthology.orchestra.run.smithi040.stdout:client9205 2024-04-26T21:46:05.543 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:05.543 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> import json 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir'"'"') 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> sys.exit(e.errno) 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> print(json.dumps( 2024-04-26T21:46:05.559 DEBUG:teuthology.orchestra.run.smithi040:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-26T21:46:05.560 DEBUG:teuthology.orchestra.run.smithi040:> indent=2)) 2024-04-26T21:46:05.560 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:46:05.601 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:46:05.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-04-26T21:46:05.675 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-04-26T21:46:05.675 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-04-26T21:46:05.676 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.817+0000 7f1f6f093640 1 Processor -- start 2024-04-26T21:46:05.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.817+0000 7f1f6f093640 1 -- start start 2024-04-26T21:46:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.817+0000 7f1f6f093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f680b0380 0x7f1f680ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.817+0000 7f1f6f093640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f680b10b0 0x7f1f6806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.817+0000 7f1f6f093640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1f6806c780 0x7f1f68072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.817+0000 7f1f6f093640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1f680cfd80 con 0x7f1f680b10b0 2024-04-26T21:46:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.817+0000 7f1f6f093640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1f680d0020 con 0x7f1f6806c780 2024-04-26T21:46:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.817+0000 7f1f6f093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f680d0300 con 0x7f1f680b0380 2024-04-26T21:46:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f67fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f680b10b0 0x7f1f6806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f67fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f680b10b0 0x7f1f6806c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39890/0 (socket says 172.21.15.40:39890) 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f67fff640 1 -- 172.21.15.40:0/2978428478 learned_addr learned my addr 172.21.15.40:0/2978428478 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f6ce2f640 1 --2- 172.21.15.40:0/2978428478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f680b0380 0x7f1f680ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f6d630640 1 --2- 172.21.15.40:0/2978428478 >> v2:172.21.15.184:3300/0 conn(0x7f1f6806c780 0x7f1f68072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f67fff640 1 -- 172.21.15.40:0/2978428478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f680b0380 msgr2=0x7f1f680ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f67fff640 1 --2- 172.21.15.40:0/2978428478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f680b0380 0x7f1f680ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f67fff640 1 -- 172.21.15.40:0/2978428478 >> v2:172.21.15.184:3300/0 conn(0x7f1f6806c780 msgr2=0x7f1f68072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f67fff640 1 --2- 172.21.15.40:0/2978428478 >> v2:172.21.15.184:3300/0 conn(0x7f1f6806c780 0x7f1f68072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f67fff640 1 -- 172.21.15.40:0/2978428478 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f680d05e0 con 0x7f1f680b10b0 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.818+0000 7f1f6d630640 1 --2- 172.21.15.40:0/2978428478 >> v2:172.21.15.184:3300/0 conn(0x7f1f6806c780 0x7f1f68072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f67fff640 1 --2- 172.21.15.40:0/2978428478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6806c240 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f1f680c57c0 tx=0x7f1f5c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5fad8c3d797e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f677fe640 1 -- 172.21.15.40:0/2978428478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c037070 con 0x7f1f680b10b0 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f677fe640 1 -- 172.21.15.40:0/2978428478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c032440 con 0x7f1f680b10b0 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f677fe640 1 -- 172.21.15.40:0/2978428478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c032810 con 0x7f1f680b10b0 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 -- 172.21.15.40:0/2978428478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 msgr2=0x7f1f6806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 --2- 172.21.15.40:0/2978428478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6806c240 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f1f680c57c0 tx=0x7f1f5c02a140 comp rx=0 tx=0).stop 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 -- 172.21.15.40:0/2978428478 shutdown_connections 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 --2- 172.21.15.40:0/2978428478 >> v2:172.21.15.184:3300/0 conn(0x7f1f6806c780 0x7f1f68072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 --2- 172.21.15.40:0/2978428478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6806c240 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 --2- 172.21.15.40:0/2978428478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f680b0380 0x7f1f680ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 -- 172.21.15.40:0/2978428478 >> 172.21.15.40:0/2978428478 conn(0x7f1f680d1cf0 msgr2=0x7f1f680d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 -- 172.21.15.40:0/2978428478 shutdown_connections 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.819+0000 7f1f6f093640 1 -- 172.21.15.40:0/2978428478 wait complete. 2024-04-26T21:46:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.821+0000 7f1f6f093640 1 Processor -- start 2024-04-26T21:46:05.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.821+0000 7f1f6f093640 1 -- start start 2024-04-26T21:46:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.821+0000 7f1f6f093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f6806c780 0x7f1f6815e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.821+0000 7f1f6f093640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f680b0380 0x7f1f681f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6f093640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6f093640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1f680d15c0 con 0x7f1f680b10b0 2024-04-26T21:46:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6f093640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f680b0760 con 0x7f1f680b0380 2024-04-26T21:46:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6f093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f680d1740 con 0x7f1f6806c780 2024-04-26T21:46:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6815dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39892/0 (socket says 172.21.15.40:39892) 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 -- 172.21.15.40:0/3193815997 learned_addr learned my addr 172.21.15.40:0/3193815997 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6ce2f640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f6806c780 0x7f1f6815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f67fff640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f680b0380 0x7f1f681f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 -- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f6806c780 msgr2=0x7f1f6815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f6806c780 0x7f1f6815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 -- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f680b0380 msgr2=0x7f1f681f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f680b0380 0x7f1f681f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 -- 172.21.15.40:0/3193815997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681f4ce0 con 0x7f1f680b10b0 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6d630640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6815dd60 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f1f60007e10 tx=0x7f1f6000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f657fa640 1 -- 172.21.15.40:0/3193815997 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6001a070 con 0x7f1f680b10b0 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f657fa640 1 -- 172.21.15.40:0/3193815997 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f60003b90 con 0x7f1f680b10b0 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6815ec70 con 0x7f1f680b10b0 2024-04-26T21:46:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f657fa640 1 -- 172.21.15.40:0/3193815997 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60004030 con 0x7f1f680b10b0 2024-04-26T21:46:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.822+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f681f4fa0 con 0x7f1f680b10b0 2024-04-26T21:46:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.823+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f680694f0 con 0x7f1f680b10b0 2024-04-26T21:46:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.824+0000 7f1f657fa640 1 -- 172.21.15.40:0/3193815997 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f6001d070 con 0x7f1f680b10b0 2024-04-26T21:46:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.824+0000 7f1f657fa640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f5406c8f0 0x7f1f5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.824+0000 7f1f657fa640 1 -- 172.21.15.40:0/3193815997 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f1f6008cac0 con 0x7f1f680b10b0 2024-04-26T21:46:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.825+0000 7f1f6ce2f640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f5406c8f0 0x7f1f5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.825+0000 7f1f6ce2f640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f5406c8f0 0x7f1f5406edb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1f58006fd0 tx=0x7f1f58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:05.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:05.828+0000 7f1f657fa640 1 -- 172.21.15.40:0/3193815997 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f60010070 con 0x7f1f680b10b0 2024-04-26T21:46:06.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.050+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f680b22c0 con 0x7f1f680b10b0 2024-04-26T21:46:06.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.050+0000 7f1f657fa640 1 -- 172.21.15.40:0/3193815997 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 304 v304) ==== 95+0+5202 (secure 0 0 0) 0x7f1f6005a220 con 0x7f1f680b10b0 2024-04-26T21:46:06.050 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:06.051 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":304,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":304,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:03.782753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10914":{"gid":10914,"name":"c","rank":0,"incarnation":300,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6833/1753847972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1753847972},{"type":"v1","addr":"172.21.15.40:6833","nonce":1753847972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10914,"qdb_cluster":[10914]},"id":6}]} 2024-04-26T21:46:06.051 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 304 2024-04-26T21:46:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.052+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f5406c8f0 msgr2=0x7f1f5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.052+0000 7f1f6f093640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f5406c8f0 0x7f1f5406edb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1f58006fd0 tx=0x7f1f58008040 comp rx=0 tx=0).stop 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.052+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 msgr2=0x7f1f6815dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.052+0000 7f1f6f093640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6815dd60 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f1f60007e10 tx=0x7f1f6000d4a0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.052+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 shutdown_connections 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.053+0000 7f1f6f093640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1f5406c8f0 0x7f1f5406edb0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.053+0000 7f1f6f093640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f680b10b0 0x7f1f6815dd60 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.053+0000 7f1f6f093640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f680b0380 0x7f1f681f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.053+0000 7f1f6f093640 1 --2- 172.21.15.40:0/3193815997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f6806c780 0x7f1f6815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.053+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 >> 172.21.15.40:0/3193815997 conn(0x7f1f680d1cf0 msgr2=0x7f1f6806ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.053+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 shutdown_connections 2024-04-26T21:46:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.053+0000 7f1f6f093640 1 -- 172.21.15.40:0/3193815997 wait complete. 2024-04-26T21:46:06.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-26T21:46:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.166+0000 7f40f562a640 1 Processor -- start 2024-04-26T21:46:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.166+0000 7f40f562a640 1 -- start start 2024-04-26T21:46:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.167+0000 7f40f562a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f40f006d560 0x7f40f00d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.167+0000 7f40f562a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f00d2420 con 0x7f40f006e270 2024-04-26T21:46:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.167+0000 7f40f562a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f00cd6b0 con 0x7f40f00d0070 2024-04-26T21:46:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.167+0000 7f40f562a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f40f0162d00 con 0x7f40f006d560 2024-04-26T21:46:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.167+0000 7f40ee7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f40f006e270 0x7f40f00ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44536/0 (socket says 172.21.15.40:44536) 2024-04-26T21:46:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.167+0000 7f40ee7fc640 1 -- 172.21.15.40:0/232359193 learned_addr learned my addr 172.21.15.40:0/232359193 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1384025917 0 0) 0x7f40f00d2420 con 0x7f40f006e270 2024-04-26T21:46:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/232359193 >> v2:172.21.15.184:3300/0 conn(0x7f40f006d560 0x7f40f00d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40d8003520 con 0x7f40f006e270 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173285913 0 0) 0x7f40d8003520 con 0x7f40f006e270 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 >> v2:172.21.15.184:3300/0 conn(0x7f40f006d560 msgr2=0x7f40f00d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 --2- 172.21.15.40:0/232359193 >> v2:172.21.15.184:3300/0 conn(0x7f40f006d560 0x7f40f00d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 >> v1:172.21.15.184:6789/0 conn(0x7f40f00d0070 legacy=0x7f40f006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f00cd990 con 0x7f40f006e270 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f40e4001670 con 0x7f40f006e270 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 >> v1:172.21.15.40:6789/0 conn(0x7f40f006e270 legacy=0x7f40f00ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d80039b0 0x7f40d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40d8004800 0x7f40d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80051c0 0x7f40d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0162d00 con 0x7f40d80039b0 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40d800c060 con 0x7f40d8004800 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40d800c260 con 0x7f40d80051c0 2024-04-26T21:46:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40eeffd640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d80039b0 0x7f40d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40d8004800 0x7f40d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80051c0 0x7f40d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40eeffd640 1 -- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80051c0 msgr2=0x7f40d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40eeffd640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80051c0 0x7f40d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40eeffd640 1 -- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40d8004800 msgr2=0x7f40d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40eeffd640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40d8004800 0x7f40d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40eeffd640 1 -- 172.21.15.40:0/232359193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d8003520 con 0x7f40d80039b0 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40ef7fe640 1 -- 172.21.15.40:0/232359193 reap_dead start 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.168+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40d8004800 0x7f40d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.169+0000 7f40eeffd640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d80039b0 0x7f40d8003dd0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f40e8001750 tx=0x7f40d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a2f2eaf16b3b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.169+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8001370 con 0x7f40d80039b0 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.169+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e80052d0 con 0x7f40d80039b0 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.169+0000 7f40edffb640 1 -- 172.21.15.40:0/232359193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e802abe0 con 0x7f40d80039b0 2024-04-26T21:46:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.170+0000 7f40f562a640 1 -- 172.21.15.40:0/232359193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d80039b0 msgr2=0x7f40d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.170+0000 7f40f562a640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d80039b0 0x7f40d8003dd0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f40e8001750 tx=0x7f40d8005830 comp rx=0 tx=0).stop 2024-04-26T21:46:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.170+0000 7f40f562a640 1 -- 172.21.15.40:0/232359193 shutdown_connections 2024-04-26T21:46:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.170+0000 7f40f562a640 1 --2- 172.21.15.40:0/232359193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d80039b0 0x7f40d8003dd0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.170+0000 7f40f562a640 1 -- 172.21.15.40:0/232359193 >> 172.21.15.40:0/232359193 conn(0x7f40f00d2ec0 msgr2=0x7f40f00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.170+0000 7f40f562a640 1 -- 172.21.15.40:0/232359193 shutdown_connections 2024-04-26T21:46:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.170+0000 7f40f562a640 1 -- 172.21.15.40:0/232359193 wait complete. 2024-04-26T21:46:06.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.171+0000 7f40f562a640 1 Processor -- start 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.171+0000 7f40f562a640 1 -- start start 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40f562a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80039b0 0x7f40f009f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40f562a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f009cca0 0x7f40f009dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40f562a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f009d0e0 0x7f40f00c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40f562a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40f00d1d40 con 0x7f40f009d0e0 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40f562a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40f00d2d10 con 0x7f40f009cca0 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40f562a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40f00d1ec0 con 0x7f40d80039b0 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f009d0e0 0x7f40f00c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f009d0e0 0x7f40f00c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39908/0 (socket says 172.21.15.40:39908) 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 -- 172.21.15.40:0/720888123 learned_addr learned my addr 172.21.15.40:0/720888123 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40eeffd640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80039b0 0x7f40f009f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f009cca0 0x7f40f009dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 -- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80039b0 msgr2=0x7f40f009f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80039b0 0x7f40f009f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 -- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f009cca0 msgr2=0x7f40f009dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f009cca0 0x7f40f009dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 -- 172.21.15.40:0/720888123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f009fd10 con 0x7f40f009d0e0 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40eeffd640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80039b0 0x7f40f009f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f009cca0 0x7f40f009dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.172+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f009d0e0 0x7f40f00c82e0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f40f00d3fc0 tx=0x7f40e00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.173+0000 7f40d77fe640 1 -- 172.21.15.40:0/720888123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0011e10 con 0x7f40f009d0e0 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.173+0000 7f40d77fe640 1 -- 172.21.15.40:0/720888123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e001b070 con 0x7f40f009d0e0 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.173+0000 7f40f562a640 1 -- 172.21.15.40:0/720888123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f009fff0 con 0x7f40f009d0e0 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.173+0000 7f40d77fe640 1 -- 172.21.15.40:0/720888123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0003990 con 0x7f40f009d0e0 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.173+0000 7f40f562a640 1 -- 172.21.15.40:0/720888123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f00a0890 con 0x7f40f009d0e0 2024-04-26T21:46:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.174+0000 7f40d77fe640 1 -- 172.21.15.40:0/720888123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40e0005d10 con 0x7f40f009d0e0 2024-04-26T21:46:06.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.174+0000 7f40d77fe640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40dc06c8f0 0x7f40dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.175+0000 7f40d77fe640 1 -- 172.21.15.40:0/720888123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f40e008ce40 con 0x7f40f009d0e0 2024-04-26T21:46:06.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.175+0000 7f40eeffd640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40dc06c8f0 0x7f40dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.175+0000 7f40eeffd640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40dc06c8f0 0x7f40dc06edb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f40e8032000 tx=0x7f40e802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.176+0000 7f40f8424740 1 Processor -- start 2024-04-26T21:46:06.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.176+0000 7f40f8424740 1 -- start start 2024-04-26T21:46:06.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40f8424740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x562847110e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40f8424740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56284714b3c0 0x56284710f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40f8424740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5628471397b0 0x562847139c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40f8424740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562847073b80 con 0x56284714b3c0 2024-04-26T21:46:06.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40f8424740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56284700f4d0 con 0x56284710ecf0 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40f8424740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56284706fb40 con 0x5628471397b0 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56284714b3c0 0x56284710f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56284714b3c0 0x56284710f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39914/0 (socket says 172.21.15.40:39914) 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 -- 172.21.15.40:0/270181906 learned_addr learned my addr 172.21.15.40:0/270181906 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40eeffd640 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x562847110e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5628471397b0 0x562847139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 -- 172.21.15.40:0/270181906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5628471397b0 msgr2=0x562847139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5628471397b0 0x562847139c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 -- 172.21.15.40:0/270181906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 msgr2=0x562847110e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x562847110e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 -- 172.21.15.40:0/270181906 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56284703c770 con 0x56284714b3c0 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56284714b3c0 0x56284710f0d0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f40e4005ac0 tx=0x7f40e4005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b5184991c85251 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40d4ff9640 1 -- 172.21.15.40:0/270181906 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4019070 con 0x56284714b3c0 2024-04-26T21:46:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40d4ff9640 1 -- 172.21.15.40:0/270181906 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e400fa10 con 0x56284714b3c0 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.177+0000 7f40d4ff9640 1 -- 172.21.15.40:0/270181906 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4010040 con 0x56284714b3c0 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 -- 172.21.15.40:0/270181906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56284714b3c0 msgr2=0x56284710f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56284714b3c0 0x56284710f0d0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f40e4005ac0 tx=0x7f40e4005f80 comp rx=0 tx=0).stop 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 -- 172.21.15.40:0/270181906 shutdown_connections 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5628471397b0 0x562847139c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56284714b3c0 0x56284710f0d0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 --2- 172.21.15.40:0/270181906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x562847110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 -- 172.21.15.40:0/270181906 >> 172.21.15.40:0/270181906 conn(0x5628470a3e90 msgr2=0x5628471547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 -- 172.21.15.40:0/270181906 shutdown_connections 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.178+0000 7f40f8424740 1 -- 172.21.15.40:0/270181906 wait complete. 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 1 -- start start 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 1 Processor -- start 2024-04-26T21:46:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284710ecf0 0x56284713a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284713a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284714b3c0 0x562847138750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56284700ce70 con 0x5628471397b0 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562847137300 con 0x56284714b3c0 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562847166340 con 0x56284710ecf0 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.179+0000 7f40f8424740 10 client.0 _ll_register_callbacks cb 0x56284701dea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284713a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284713a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39930/0 (socket says 172.21.15.40:39930) 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 -- 172.21.15.40:0/428708745 learned_addr learned my addr 172.21.15.40:0/428708745 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40eeffd640 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284710ecf0 0x56284713a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284714b3c0 0x562847138750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 -- 172.21.15.40:0/428708745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284710ecf0 msgr2=0x56284713a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284710ecf0 0x56284713a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 -- 172.21.15.40:0/428708745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284714b3c0 msgr2=0x562847138750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284714b3c0 0x562847138750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 -- 172.21.15.40:0/428708745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562846e517c0 con 0x5628471397b0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284713a740 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f40e4017040 tx=0x7f40e4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e438c2fe043413a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40cdffb640 1 -- 172.21.15.40:0/428708745 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4024060 con 0x5628471397b0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40cdffb640 1 -- 172.21.15.40:0/428708745 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f40e40047e0 con 0x5628471397b0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40f8424740 1 -- 172.21.15.40:0/428708745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e400fa10 con 0x5628471397b0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40cdffb640 1 -- 172.21.15.40:0/428708745 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=304) ==== 24+0+0 (secure 0 0 0) 0x7f40e402b050 con 0x5628471397b0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40f8424740 10 client.11022 fetch_fsmap learned FSMap version 304 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.180+0000 7f40f8424740 1 -- 172.21.15.40:0/428708745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=304,monmap=2+}) -- 0x562846eb1f50 con 0x5628471397b0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40cdffb640 1 -- 172.21.15.40:0/428708745 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e400fa10 con 0x5628471397b0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40cdffb640 1 -- 172.21.15.40:0/428708745 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 304) ==== 3494+0+0 (secure 0 0 0) 0x7f40e4013070 con 0x5628471397b0 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40f8424740 10 client.11022 fetch_fsmap finished waiting for FSMap version 304 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40f8424740 10 client.11022 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:46:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40f8424740 20 client.11022 populate_metadata read hostname 'smithi040' 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40f8424740 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x562847167ec0 0x5628471682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40f8424740 4 client.11022 mds_command: new command op to 10914 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40f8424740 1 -- 172.21.15.40:0/428708745 --> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562847027240 con 0x562847167ec0 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40eeffd640 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x562847167ec0 0x5628471682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40eeffd640 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x562847167ec0 0x5628471682a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e746b67a09e076d server_cookie=e8cbef66a678680e in_seq=0 out_seq=0 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.181+0000 7f40cdffb640 10 client.11022 ms_handle_connect on v2:172.21.15.40:6832/1753847972 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40cdffb640 1 -- 172.21.15.40:0/428708745 <== mds.0 v2:172.21.15.40:6832/1753847972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562847027240 con 0x562847167ec0 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40cdffb640 10 client.11022 handle_command_reply: tid=0 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 client.11022 shutdown 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 -- 172.21.15.40:0/428708745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 msgr2=0x56284713a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284713a740 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f40e4017040 tx=0x7f40e4001a40 comp rx=0 tx=0).stop 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 -- 172.21.15.40:0/428708745 shutdown_connections 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x562847167ec0 0x5628471682a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284714b3c0 0x562847138750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284713a740 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 --2- 172.21.15.40:0/428708745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284710ecf0 0x56284713a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40cdffb640 0 client.11022 ms_handle_reset on v2:172.21.15.40:6832/1753847972 2024-04-26T21:46:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 -- 172.21.15.40:0/428708745 >> 172.21.15.40:0/428708745 conn(0x5628470a3e90 msgr2=0x5628471348b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 -- 172.21.15.40:0/428708745 shutdown_connections 2024-04-26T21:46:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 1 -- 172.21.15.40:0/428708745 wait complete. 2024-04-26T21:46:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.182+0000 7f40f8424740 20 client.11022 trim_cache size 0 max 16384 2024-04-26T21:46:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40f8424740 1 Processor -- start 2024-04-26T21:46:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40f8424740 1 -- start start 2024-04-26T21:46:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40f8424740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x56284716e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40f8424740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284716ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40f8424740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 0x5628471578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40f8424740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0203420 con 0x5628471397b0 2024-04-26T21:46:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40f8424740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0202720 con 0x56284710ecf0 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40f8424740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40f0202d70 con 0x56284714b3c0 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284716ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284716ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39938/0 (socket says 172.21.15.40:39938) 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.196+0000 7f40ee7fc640 1 -- 172.21.15.40:0/3469317235 learned_addr learned my addr 172.21.15.40:0/3469317235 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40eeffd640 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x56284716e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 0x5628471578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40ee7fc640 1 -- 172.21.15.40:0/3469317235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 msgr2=0x5628471578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 0x5628471578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40ee7fc640 1 -- 172.21.15.40:0/3469317235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 msgr2=0x56284716e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x56284716e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40ee7fc640 1 -- 172.21.15.40:0/3469317235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562846d450b0 con 0x5628471397b0 2024-04-26T21:46:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284716ea00 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f40e4001970 tx=0x7f40e401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2debc044ea161d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40cffff640 1 -- 172.21.15.40:0/3469317235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4024040 con 0x5628471397b0 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40cffff640 1 -- 172.21.15.40:0/3469317235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e40047e0 con 0x5628471397b0 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.197+0000 7f40cffff640 1 -- 172.21.15.40:0/3469317235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4019040 con 0x5628471397b0 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 -- 172.21.15.40:0/3469317235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 msgr2=0x56284716ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284716ea00 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f40e4001970 tx=0x7f40e401d000 comp rx=0 tx=0).stop 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 -- 172.21.15.40:0/3469317235 shutdown_connections 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 0x5628471578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x56284716ea00 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 --2- 172.21.15.40:0/3469317235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x56284716e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 -- 172.21.15.40:0/3469317235 >> 172.21.15.40:0/3469317235 conn(0x5628470a3e90 msgr2=0x562847167790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 -- 172.21.15.40:0/3469317235 shutdown_connections 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 -- 172.21.15.40:0/3469317235 wait complete. 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 -- start start 2024-04-26T21:46:06.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.198+0000 7f40f8424740 1 Processor -- start 2024-04-26T21:46:06.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40f8424740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x5628471ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40f8424740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x562847137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40f8424740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 0x56284710d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40f8424740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562847138d30 con 0x5628471397b0 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40f8424740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56284713c730 con 0x56284710ecf0 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40f8424740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562847164f50 con 0x56284714b3c0 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40f8424740 10 client.0 _ll_register_callbacks cb 0x7f40e4017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x562847137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x562847137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39944/0 (socket says 172.21.15.40:39944) 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40ee7fc640 1 -- 172.21.15.40:0/857004387 learned_addr learned my addr 172.21.15.40:0/857004387 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40ef7fe640 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 0x56284710d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40eeffd640 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x5628471ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.199+0000 7f40ee7fc640 1 -- 172.21.15.40:0/857004387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 msgr2=0x56284710d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 0x56284710d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40ee7fc640 1 -- 172.21.15.40:0/857004387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 msgr2=0x5628471ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x5628471ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40ee7fc640 1 -- 172.21.15.40:0/857004387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56284710b9c0 con 0x5628471397b0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40ee7fc640 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x562847137ca0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f40e4031040 tx=0x7f40e4001450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7c9a7a4b136edc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40cdffb640 1 -- 172.21.15.40:0/857004387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e403b030 con 0x5628471397b0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40cdffb640 1 -- 172.21.15.40:0/857004387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f40e4015820 con 0x5628471397b0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 1 -- 172.21.15.40:0/857004387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e40047e0 con 0x5628471397b0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40cdffb640 1 -- 172.21.15.40:0/857004387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=304) ==== 24+0+0 (secure 0 0 0) 0x7f40e4045050 con 0x5628471397b0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 10 client.11028 fetch_fsmap learned FSMap version 304 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 1 -- 172.21.15.40:0/857004387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=304,monmap=2+}) -- 0x56284710dd80 con 0x5628471397b0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40cdffb640 1 -- 172.21.15.40:0/857004387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e40047e0 con 0x5628471397b0 2024-04-26T21:46:06.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40cdffb640 1 -- 172.21.15.40:0/857004387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 304) ==== 3494+0+0 (secure 0 0 0) 0x7f40e4013030 con 0x5628471397b0 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 10 client.11028 fetch_fsmap finished waiting for FSMap version 304 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 10 client.11028 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 20 client.11028 populate_metadata read hostname 'smithi040' 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x56284710d5f0 0x56284713a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 4 client.11028 mds_command: new command op to 10914 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.200+0000 7f40f8424740 1 -- 172.21.15.40:0/857004387 --> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x562847081790 con 0x56284710d5f0 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40eeffd640 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x56284710d5f0 0x56284713a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40cdffb640 10 client.11028 ms_handle_connect on v2:172.21.15.40:6832/1753847972 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40eeffd640 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x56284710d5f0 0x56284713a010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4843b6718b5c89 server_cookie=afd584061649fd49 in_seq=0 out_seq=0 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40cdffb640 1 -- 172.21.15.40:0/857004387 <== mds.0 v2:172.21.15.40:6832/1753847972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+194 (crc 0 0 0) 0x562847081790 con 0x56284710d5f0 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40cdffb640 10 client.11028 handle_command_reply: tid=0 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 client.11028 shutdown 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 -- 172.21.15.40:0/857004387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 msgr2=0x562847137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x562847137ca0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f40e4031040 tx=0x7f40e4001450 comp rx=0 tx=0).stop 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 -- 172.21.15.40:0/857004387 shutdown_connections 2024-04-26T21:46:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.40:6832/1753847972,v1:172.21.15.40:6833/1753847972] conn(0x56284710d5f0 0x56284713a010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56284714b3c0 0x56284710d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5628471397b0 0x562847137ca0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 --2- 172.21.15.40:0/857004387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56284710ecf0 0x5628471ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 -- 172.21.15.40:0/857004387 >> 172.21.15.40:0/857004387 conn(0x5628470a3e90 msgr2=0x56284716e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40cdffb640 0 client.11028 ms_handle_reset on v2:172.21.15.40:6832/1753847972 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 -- 172.21.15.40:0/857004387 shutdown_connections 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 1 -- 172.21.15.40:0/857004387 wait complete. 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.201+0000 7f40f8424740 20 client.11028 trim_cache size 0 max 16384 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:46:06.208 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "dentry", 2024-04-26T21:46:06.209 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1729025209, 2024-04-26T21:46:06.209 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1, 2024-04-26T21:46:06.209 INFO:teuthology.orchestra.run.smithi040.stdout: "frag": "*", 2024-04-26T21:46:06.209 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "subdir", 2024-04-26T21:46:06.209 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_id": "head", 2024-04-26T21:46:06.209 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/subdir" 2024-04-26T21:46:06.209 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:46:06.209 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 -- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40dc06c8f0 msgr2=0x7f40dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40dc06c8f0 0x7f40dc06edb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f40e8032000 tx=0x7f40e802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 -- 172.21.15.40:0/720888123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f009d0e0 msgr2=0x7f40f00c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f009d0e0 0x7f40f00c82e0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f40f00d3fc0 tx=0x7f40e00115a0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 -- 172.21.15.40:0/720888123 shutdown_connections 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f40dc06c8f0 0x7f40dc06edb0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f009d0e0 0x7f40f00c82e0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f009cca0 0x7f40f009dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 --2- 172.21.15.40:0/720888123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40d80039b0 0x7f40f009f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 -- 172.21.15.40:0/720888123 >> 172.21.15.40:0/720888123 conn(0x7f40f00d2ec0 msgr2=0x7f40f00ca170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 -- 172.21.15.40:0/720888123 shutdown_connections 2024-04-26T21:46:06.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.202+0000 7f40f562a640 1 -- 172.21.15.40:0/720888123 wait complete. 2024-04-26T21:46:06.212 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 10000000000.00000000' 2024-04-26T21:46:06.213 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:06.213 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:06.228 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:06.228 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:06.284 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:06.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.424+0000 7fb825bf9640 1 Processor -- start 2024-04-26T21:46:06.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.424+0000 7fb825bf9640 1 -- start start 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.425+0000 7fb825bf9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8200d06f0 0x7fb8200d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.425+0000 7fb825bf9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8200b2850 0x7fb8200b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.425+0000 7fb825bf9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8200d0ba0 con 0x7fb82015f3a0 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.425+0000 7fb825bf9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8200d0e20 con 0x7fb8200d06f0 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.425+0000 7fb825bf9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8200b1d70 con 0x7fb8200b2850 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.425+0000 7fb81f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb82015f3a0 0x7fb8200d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44544/0 (socket says 172.21.15.40:44544) 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.425+0000 7fb81f7fe640 1 -- 172.21.15.40:0/4217546493 learned_addr learned my addr 172.21.15.40:0/4217546493 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81effd640 1 --2- 172.21.15.40:0/4217546493 >> v2:172.21.15.184:3300/0 conn(0x7fb8200d06f0 0x7fb8200d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81ffff640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8200b2850 0x7fb8200b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3198669189 0 0) 0x7fb810001110 con 0x7fb82015f3a0 2024-04-26T21:46:06.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb808003520 con 0x7fb82015f3a0 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1601732100 0 0) 0x7fb808003520 con 0x7fb82015f3a0 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8200b2850 msgr2=0x7fb8200b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8200b2850 0x7fb8200b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 >> v2:172.21.15.184:3300/0 conn(0x7fb8200d06f0 msgr2=0x7fb8200d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 --2- 172.21.15.40:0/4217546493 >> v2:172.21.15.184:3300/0 conn(0x7fb8200d06f0 0x7fb8200d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8200b2050 con 0x7fb82015f3a0 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb810001840 con 0x7fb82015f3a0 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 >> v1:172.21.15.40:6789/0 conn(0x7fb82015f3a0 legacy=0x7fb8200d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8200b2850 0x7fb8080039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb808003f30 0x7fb808004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb808004d40 0x7fb8080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200d0e20 con 0x7fb808004d40 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200b1d70 con 0x7fb808003f30 2024-04-26T21:46:06.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb80800b9c0 con 0x7fb8200b2850 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81ffff640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb808004d40 0x7fb8080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81f7fe640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8200b2850 0x7fb8080039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81effd640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb808003f30 0x7fb808004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81ffff640 1 -- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8200b2850 msgr2=0x7fb8080039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81ffff640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8200b2850 0x7fb8080039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81ffff640 1 -- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb808003f30 msgr2=0x7fb808004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81ffff640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb808003f30 0x7fb808004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.426+0000 7fb81ffff640 1 -- 172.21.15.40:0/4217546493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb808003520 con 0x7fb808004d40 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.427+0000 7fb81ffff640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb808004d40 0x7fb8080051c0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fb818009fc0 tx=0x7fb808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7847f7459b3539 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.427+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818009c10 con 0x7fb808004d40 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.427+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb818036070 con 0x7fb808004d40 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.427+0000 7fb81e7fc640 1 -- 172.21.15.40:0/4217546493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818031670 con 0x7fb808004d40 2024-04-26T21:46:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.428+0000 7fb825bf9640 1 -- 172.21.15.40:0/4217546493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb808004d40 msgr2=0x7fb8080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.428+0000 7fb825bf9640 1 --2- 172.21.15.40:0/4217546493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb808004d40 0x7fb8080051c0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fb818009fc0 tx=0x7fb808005830 comp rx=0 tx=0).stop 2024-04-26T21:46:06.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.428+0000 7fb81ffff640 1 -- 172.21.15.40:0/4217546493 reap_dead start 2024-04-26T21:46:06.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.428+0000 7fb825bf9640 1 -- 172.21.15.40:0/4217546493 shutdown_connections 2024-04-26T21:46:06.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.428+0000 7fb825bf9640 1 -- 172.21.15.40:0/4217546493 >> 172.21.15.40:0/4217546493 conn(0x7fb8200d4f90 msgr2=0x7fb8200d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.428+0000 7fb825bf9640 1 -- 172.21.15.40:0/4217546493 shutdown_connections 2024-04-26T21:46:06.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.428+0000 7fb825bf9640 1 -- 172.21.15.40:0/4217546493 wait complete. 2024-04-26T21:46:06.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.429+0000 7fb825bf9640 1 Processor -- start 2024-04-26T21:46:06.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.429+0000 7fb825bf9640 1 -- start start 2024-04-26T21:46:06.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb825bf9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb820153630 0x7fb8200ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb825bf9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8200cb540 0x7fb8200ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb825bf9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb82015fa80 0x7fb82015ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb825bf9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb82015f900 con 0x7fb820153630 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb825bf9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb82015f780 con 0x7fb8200cb540 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb825bf9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8200d23c0 con 0x7fb82015fa80 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb820153630 0x7fb8200ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb820153630 0x7fb8200ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39968/0 (socket says 172.21.15.40:39968) 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 -- 172.21.15.40:0/3739808696 learned_addr learned my addr 172.21.15.40:0/3739808696 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 -- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb82015fa80 msgr2=0x7fb82015ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb82015fa80 0x7fb82015ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 -- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8200cb540 msgr2=0x7fb8200ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8200cb540 0x7fb8200ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 -- 172.21.15.40:0/3739808696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8200ca4a0 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.430+0000 7fb81f7fe640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb820153630 0x7fb8200ce070 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fb8100019b0 tx=0x7fb8100077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.431+0000 7fb807fff640 1 -- 172.21.15.40:0/3739808696 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81000fa00 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.431+0000 7fb807fff640 1 -- 172.21.15.40:0/3739808696 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb810018070 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.431+0000 7fb807fff640 1 -- 172.21.15.40:0/3739808696 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81000fde0 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.431+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8200ca780 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.431+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8200cad40 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.432+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8200694f0 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.432+0000 7fb807fff640 1 -- 172.21.15.40:0/3739808696 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb81001d460 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.433+0000 7fb807fff640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb80c06c8f0 0x7fb80c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.433+0000 7fb807fff640 1 -- 172.21.15.40:0/3739808696 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fb81008cb40 con 0x7fb820153630 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.433+0000 7fb81effd640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb80c06c8f0 0x7fb80c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.433+0000 7fb81effd640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb80c06c8f0 0x7fb80c06edb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb814006b10 tx=0x7fb814006aa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.437+0000 7fb807fff640 1 -- 172.21.15.40:0/3739808696 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8200d0ba0 con 0x7fb820153630 2024-04-26T21:46:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.621+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8200b6860 con 0x7fb820153630 2024-04-26T21:46:06.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.622+0000 7fb807fff640 1 -- 172.21.15.40:0/3739808696 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+21322 (secure 0 0 0) 0x7fb81005a210 con 0x7fb820153630 2024-04-26T21:46:06.623 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:06.623 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":128,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:45:51.733276+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6833/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6832/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6833/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6833/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6832/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3538213517":"2024-04-27T21:45:17.788819+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/3538213517":"2024-04-27T21:45:17.788819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb80c06c8f0 msgr2=0x7fb80c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb80c06c8f0 0x7fb80c06edb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb814006b10 tx=0x7fb814006aa0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb820153630 msgr2=0x7fb8200ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb820153630 0x7fb8200ce070 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fb8100019b0 tx=0x7fb8100077a0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 shutdown_connections 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb80c06c8f0 0x7fb80c06edb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb82015fa80 0x7fb82015ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8200cb540 0x7fb8200ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 --2- 172.21.15.40:0/3739808696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb820153630 0x7fb8200ce070 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 >> 172.21.15.40:0/3739808696 conn(0x7fb8200d4f90 msgr2=0x7fb8200d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 shutdown_connections 2024-04-26T21:46:06.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.624+0000 7fb825bf9640 1 -- 172.21.15.40:0/3739808696 wait complete. 2024-04-26T21:46:06.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:06.636 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:06.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.740+0000 7f7a45f11640 1 Processor -- start 2024-04-26T21:46:06.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.740+0000 7f7a45f11640 1 -- start start 2024-04-26T21:46:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a45f11640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a4015d250 0x7f7a400d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a45f11640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7a400d39f0 0x7f7a400d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a45f11640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a400701e0 con 0x7f7a400d1550 2024-04-26T21:46:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a45f11640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7a400d19c0 con 0x7f7a4015d250 2024-04-26T21:46:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a45f11640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7a4015d630 con 0x7f7a400d39f0 2024-04-26T21:46:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a3f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a4015d250 0x7f7a400d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a3ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7a400d1550 0x7f7a400cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44550/0 (socket says 172.21.15.40:44550) 2024-04-26T21:46:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a3ffff640 1 -- 172.21.15.40:0/3120058641 learned_addr learned my addr 172.21.15.40:0/3120058641 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.741+0000 7f7a3effd640 1 --2- 172.21.15.40:0/3120058641 >> v2:172.21.15.184:3300/0 conn(0x7f7a400d39f0 0x7f7a400d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3f7fe640 1 -- 172.21.15.40:0/3120058641 >> v2:172.21.15.184:3300/0 conn(0x7f7a400d39f0 msgr2=0x7f7a400d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3f7fe640 1 --2- 172.21.15.40:0/3120058641 >> v2:172.21.15.184:3300/0 conn(0x7f7a400d39f0 0x7f7a400d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3f7fe640 1 -- 172.21.15.40:0/3120058641 >> v1:172.21.15.40:6789/0 conn(0x7f7a400d1550 legacy=0x7f7a400cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3e7fc640 1 -- 172.21.15.40:0/3120058641 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 351704529 0 0) 0x7f7a400701e0 con 0x7f7a400d1550 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3f7fe640 1 -- 172.21.15.40:0/3120058641 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a400ce370 con 0x7f7a4015d250 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3f7fe640 1 --2- 172.21.15.40:0/3120058641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a4015d250 0x7f7a400d0950 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f7a40074050 tx=0x7f7a3002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8226ea9d897c535 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3e7fc640 1 -- 172.21.15.40:0/3120058641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30037070 con 0x7f7a4015d250 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3e7fc640 1 -- 172.21.15.40:0/3120058641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a30032440 con 0x7f7a4015d250 2024-04-26T21:46:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.742+0000 7f7a3e7fc640 1 -- 172.21.15.40:0/3120058641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3002ae60 con 0x7f7a4015d250 2024-04-26T21:46:06.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.743+0000 7f7a45f11640 1 -- 172.21.15.40:0/3120058641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a4015d250 msgr2=0x7f7a400d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.743+0000 7f7a45f11640 1 --2- 172.21.15.40:0/3120058641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a4015d250 0x7f7a400d0950 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f7a40074050 tx=0x7f7a3002a110 comp rx=0 tx=0).stop 2024-04-26T21:46:06.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.743+0000 7f7a45f11640 1 -- 172.21.15.40:0/3120058641 shutdown_connections 2024-04-26T21:46:06.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.743+0000 7f7a45f11640 1 --2- 172.21.15.40:0/3120058641 >> v2:172.21.15.184:3300/0 conn(0x7f7a400d39f0 0x7f7a400d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.743+0000 7f7a45f11640 1 --2- 172.21.15.40:0/3120058641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a4015d250 0x7f7a400d0950 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.743+0000 7f7a45f11640 1 -- 172.21.15.40:0/3120058641 >> 172.21.15.40:0/3120058641 conn(0x7f7a400d2d80 msgr2=0x7f7a400d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.743+0000 7f7a45f11640 1 -- 172.21.15.40:0/3120058641 shutdown_connections 2024-04-26T21:46:06.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.743+0000 7f7a45f11640 1 -- 172.21.15.40:0/3120058641 wait complete. 2024-04-26T21:46:06.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.744+0000 7f7a45f11640 1 Processor -- start 2024-04-26T21:46:06.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.744+0000 7f7a45f11640 1 -- start start 2024-04-26T21:46:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a45f11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a400d1550 0x7f7a401eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a45f11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a400d39f0 0x7f7a400c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a45f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4015d250 0x7f7a400c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a45f11640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400ce8d0 con 0x7f7a400d39f0 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a45f11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400704e0 con 0x7f7a400d1550 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a45f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a400cebb0 con 0x7f7a4015d250 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a3effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a400d39f0 0x7f7a400c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a3effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a400d39f0 0x7f7a400c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39984/0 (socket says 172.21.15.40:39984) 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a3effd640 1 -- 172.21.15.40:0/2588278854 learned_addr learned my addr 172.21.15.40:0/2588278854 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a3ffff640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4015d250 0x7f7a400c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.745+0000 7f7a3f7fe640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a400d1550 0x7f7a401eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a3effd640 1 -- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4015d250 msgr2=0x7f7a400c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a3effd640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4015d250 0x7f7a400c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a3effd640 1 -- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a400d1550 msgr2=0x7f7a401eed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a3effd640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a400d1550 0x7f7a401eed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a3effd640 1 -- 172.21.15.40:0/2588278854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a401ef290 con 0x7f7a400d39f0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a3ffff640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4015d250 0x7f7a400c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a3f7fe640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a400d1550 0x7f7a401eed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a3effd640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a400d39f0 0x7f7a400c8e20 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400e9b0 tx=0x7f7a3400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a27fff640 1 -- 172.21.15.40:0/2588278854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34017070 con 0x7f7a400d39f0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a27fff640 1 -- 172.21.15.40:0/2588278854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3400c9a0 con 0x7f7a400d39f0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a400cc2a0 con 0x7f7a400d39f0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a27fff640 1 -- 172.21.15.40:0/2588278854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400cd90 con 0x7f7a400d39f0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.746+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a400cc820 con 0x7f7a400d39f0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.747+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a400694f0 con 0x7f7a400d39f0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.747+0000 7f7a27fff640 1 -- 172.21.15.40:0/2588278854 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a3401a070 con 0x7f7a400d39f0 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.748+0000 7f7a27fff640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.748+0000 7f7a27fff640 1 -- 172.21.15.40:0/2588278854 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f7a3408c8c0 con 0x7f7a400d39f0 2024-04-26T21:46:06.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.748+0000 7f7a3f7fe640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:06.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.749+0000 7f7a3f7fe640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7a30034000 tx=0x7f7a3002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:06.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.752+0000 7f7a27fff640 1 -- 172.21.15.40:0/2588278854 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a34059f90 con 0x7f7a400d39f0 2024-04-26T21:46:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.936+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a40160670 con 0x7f7a400d39f0 2024-04-26T21:46:06.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.936+0000 7f7a27fff640 1 -- 172.21.15.40:0/2588278854 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7f7a34059890 con 0x7f7a400d39f0 2024-04-26T21:46:06.936 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:46:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2c06c8f0 msgr2=0x7f7a2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7a30034000 tx=0x7f7a3002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:06.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a400d39f0 msgr2=0x7f7a400c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a400d39f0 0x7f7a400c8e20 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400e9b0 tx=0x7f7a3400ee70 comp rx=0 tx=0).stop 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 shutdown_connections 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7a2c06c8f0 0x7f7a2c06edb0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4015d250 0x7f7a400c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a400d39f0 0x7f7a400c8e20 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 --2- 172.21.15.40:0/2588278854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a400d1550 0x7f7a401eed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 >> 172.21.15.40:0/2588278854 conn(0x7f7a400d2d80 msgr2=0x7f7a400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 shutdown_connections 2024-04-26T21:46:06.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:06.938+0000 7f7a45f11640 1 -- 172.21.15.40:0/2588278854 wait complete. 2024-04-26T21:46:06.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.053+0000 7f11ea9e7640 1 Processor -- start 2024-04-26T21:46:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.053+0000 7f11ea9e7640 1 -- start start 2024-04-26T21:46:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.053+0000 7f11ea9e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e415f170 0x7f11e40d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.053+0000 7f11ea9e7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f11e415f550 0x7f11e40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.053+0000 7f11ea9e7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11e40d1040 con 0x7f11e40d34b0 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11ea9e7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f11e40c9430 con 0x7f11e415f550 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11ea9e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f11e40c9680 con 0x7f11e415f170 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f11e415f550 0x7f11e40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e8f84640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f11e40d34b0 0x7f11e40cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44562/0 (socket says 172.21.15.40:44562) 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e8f84640 1 -- 172.21.15.40:0/1795671184 learned_addr learned my addr 172.21.15.40:0/1795671184 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e3fff640 1 --2- 172.21.15.40:0/1795671184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e415f170 0x7f11e40d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e37fe640 1 -- 172.21.15.40:0/1795671184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e415f170 msgr2=0x7f11e40d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e37fe640 1 --2- 172.21.15.40:0/1795671184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e415f170 0x7f11e40d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e37fe640 1 -- 172.21.15.40:0/1795671184 >> v1:172.21.15.40:6789/0 conn(0x7f11e40d34b0 legacy=0x7f11e40cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e37fe640 1 -- 172.21.15.40:0/1795671184 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e40c9960 con 0x7f11e415f550 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e2ffd640 1 -- 172.21.15.40:0/1795671184 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 927924957 0 0) 0x7f11dc001490 con 0x7f11e40d34b0 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.054+0000 7f11e3fff640 1 --2- 172.21.15.40:0/1795671184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e415f170 0x7f11e40d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11e37fe640 1 --2- 172.21.15.40:0/1795671184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e415f550 0x7f11e40d2cb0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f11d80097a0 tx=0x7f11d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e4889f0dfb966b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11e2ffd640 1 -- 172.21.15.40:0/1795671184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8037070 con 0x7f11e415f550 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11e2ffd640 1 -- 172.21.15.40:0/1795671184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d8032440 con 0x7f11e415f550 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11e2ffd640 1 -- 172.21.15.40:0/1795671184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8032810 con 0x7f11e415f550 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11ea9e7640 1 -- 172.21.15.40:0/1795671184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e415f550 msgr2=0x7f11e40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/1795671184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e415f550 0x7f11e40d2cb0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f11d80097a0 tx=0x7f11d802a180 comp rx=0 tx=0).stop 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11ea9e7640 1 -- 172.21.15.40:0/1795671184 shutdown_connections 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/1795671184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e415f550 0x7f11e40d2cb0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/1795671184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e415f170 0x7f11e40d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11ea9e7640 1 -- 172.21.15.40:0/1795671184 >> 172.21.15.40:0/1795671184 conn(0x7f11e40d4f60 msgr2=0x7f11e40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11ea9e7640 1 -- 172.21.15.40:0/1795671184 shutdown_connections 2024-04-26T21:46:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.055+0000 7f11ea9e7640 1 -- 172.21.15.40:0/1795671184 wait complete. 2024-04-26T21:46:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.057+0000 7f11ea9e7640 1 Processor -- start 2024-04-26T21:46:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.057+0000 7f11ea9e7640 1 -- start start 2024-04-26T21:46:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.057+0000 7f11ea9e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e40d34b0 0x7f11e415d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.057+0000 7f11ea9e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11e415f170 0x7f11e40a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.057+0000 7f11ea9e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e40a20f0 0x7f11e40cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.057+0000 7f11ea9e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f11e40cb5d0 con 0x7f11e40a20f0 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.057+0000 7f11ea9e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11e40cb450 con 0x7f11e415f170 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.057+0000 7f11ea9e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f11e40cb750 con 0x7f11e40d34b0 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e40a20f0 0x7f11e40cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e40a20f0 0x7f11e40cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39996/0 (socket says 172.21.15.40:39996) 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 -- 172.21.15.40:0/2643548618 learned_addr learned my addr 172.21.15.40:0/2643548618 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e3fff640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e40d34b0 0x7f11e415d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e37fe640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11e415f170 0x7f11e40a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 -- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e40d34b0 msgr2=0x7f11e415d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e40d34b0 0x7f11e415d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 -- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11e415f170 msgr2=0x7f11e40a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11e415f170 0x7f11e40a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 -- 172.21.15.40:0/2643548618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e40cdd30 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e3fff640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e40d34b0 0x7f11e415d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e37fe640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11e415f170 0x7f11e40a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e8f84640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e40a20f0 0x7f11e40cd120 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00c820 tx=0x7f11dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e0ff9640 1 -- 172.21.15.40:0/2643548618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc018070 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e0ff9640 1 -- 172.21.15.40:0/2643548618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11dc00a5e0 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e40ce010 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11e0ff9640 1 -- 172.21.15.40:0/2643548618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00a9d0 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.058+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e40ce5d0 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.059+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e40694f0 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.060+0000 7f11e0ff9640 1 -- 172.21.15.40:0/2643548618 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11dc01b070 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.060+0000 7f11e0ff9640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f11d006c8f0 0x7f11d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.060+0000 7f11e0ff9640 1 -- 172.21.15.40:0/2643548618 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f11dc08bf60 con 0x7f11e40a20f0 2024-04-26T21:46:07.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.061+0000 7f11e3fff640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f11d006c8f0 0x7f11d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.061+0000 7f11e3fff640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f11d006c8f0 0x7f11d006edb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f11e40b6be0 tx=0x7f11d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.065+0000 7f11e0ff9640 1 -- 172.21.15.40:0/2643548618 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11e40d1040 con 0x7f11e40a20f0 2024-04-26T21:46:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.253+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f11e40cf370 con 0x7f11e40a20f0 2024-04-26T21:46:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.253+0000 7f11e0ff9640 1 -- 172.21.15.40:0/2643548618 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7f11dc0595d0 con 0x7f11e40a20f0 2024-04-26T21:46:07.253 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:46:07.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f11d006c8f0 msgr2=0x7f11d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f11d006c8f0 0x7f11d006edb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f11e40b6be0 tx=0x7f11d4008040 comp rx=0 tx=0).stop 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e40a20f0 msgr2=0x7f11e40cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e40a20f0 0x7f11e40cd120 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00c820 tx=0x7f11dc00cce0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 shutdown_connections 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f11d006c8f0 0x7f11d006edb0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11e40a20f0 0x7f11e40cd120 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11e415f170 0x7f11e40a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 --2- 172.21.15.40:0/2643548618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11e40d34b0 0x7f11e415d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 >> 172.21.15.40:0/2643548618 conn(0x7f11e40d4f60 msgr2=0x7f11e406d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 shutdown_connections 2024-04-26T21:46:07.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.255+0000 7f11ea9e7640 1 -- 172.21.15.40:0/2643548618 wait complete. 2024-04-26T21:46:07.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.371+0000 7f0349a2a640 1 Processor -- start 2024-04-26T21:46:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.371+0000 7f0349a2a640 1 -- start start 2024-04-26T21:46:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f0349a2a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f034415f170 0x7f03440d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f0349a2a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03440d1040 con 0x7f03440d3bc0 2024-04-26T21:46:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f0349a2a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03440c9680 con 0x7f034415f550 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f0349a2a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f03440c9430 con 0x7f034415f170 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03437fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f03440d3bc0 0x7f03440ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44574/0 (socket says 172.21.15.40:44574) 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03437fe640 1 -- 172.21.15.40:0/675097672 learned_addr learned my addr 172.21.15.40:0/675097672 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f0342ffd640 1 --2- 172.21.15.40:0/675097672 >> v2:172.21.15.184:3300/0 conn(0x7f034415f170 0x7f03440d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4004924191 0 0) 0x7f033c001490 con 0x7f03440d3bc0 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0330003520 con 0x7f03440d3bc0 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3864444574 0 0) 0x7f0330003520 con 0x7f03440d3bc0 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 >> v2:172.21.15.184:3300/0 conn(0x7f034415f170 msgr2=0x7f03440d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 --2- 172.21.15.40:0/675097672 >> v2:172.21.15.184:3300/0 conn(0x7f034415f170 0x7f03440d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 >> v1:172.21.15.184:6789/0 conn(0x7f034415f550 legacy=0x7f03440d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03440c9960 con 0x7f03440d3bc0 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f033c001bc0 con 0x7f03440d3bc0 2024-04-26T21:46:07.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.372+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 >> v1:172.21.15.40:6789/0 conn(0x7f03440d3bc0 legacy=0x7f03440ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f03427fc640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f0330003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f03427fc640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0330004800 0x7f0330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f03427fc640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03300051c0 0x7f033000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03440c9430 con 0x7f03300039b0 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f033000c090 con 0x7f03300051c0 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f033000c290 con 0x7f0330004800 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f0342ffd640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f0330003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f033bfff640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0330004800 0x7f0330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f03437fe640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03300051c0 0x7f033000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f0342ffd640 1 -- 172.21.15.40:0/675097672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0330004800 msgr2=0x7f0330004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f0342ffd640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0330004800 0x7f0330004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f0342ffd640 1 -- 172.21.15.40:0/675097672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03300051c0 msgr2=0x7f033000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f0342ffd640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03300051c0 0x7f033000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f0342ffd640 1 -- 172.21.15.40:0/675097672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0330003520 con 0x7f03300039b0 2024-04-26T21:46:07.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.373+0000 7f03437fe640 1 -- 172.21.15.40:0/675097672 reap_dead start 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f0342ffd640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f0330003df0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f0334009ac0 tx=0x7f0330005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84decfff6cda3231 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334036070 con 0x7f03300039b0 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0334031400 con 0x7f03300039b0 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f03427fc640 1 -- 172.21.15.40:0/675097672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334009690 con 0x7f03300039b0 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f0349a2a640 1 -- 172.21.15.40:0/675097672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 msgr2=0x7f0330003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f0349a2a640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f0330003df0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f0334009ac0 tx=0x7f0330005830 comp rx=0 tx=0).stop 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f0349a2a640 1 -- 172.21.15.40:0/675097672 shutdown_connections 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f0349a2a640 1 --2- 172.21.15.40:0/675097672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f0330003df0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f0349a2a640 1 -- 172.21.15.40:0/675097672 >> 172.21.15.40:0/675097672 conn(0x7f03440d4f60 msgr2=0x7f03440d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f0349a2a640 1 -- 172.21.15.40:0/675097672 shutdown_connections 2024-04-26T21:46:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.374+0000 7f0349a2a640 1 -- 172.21.15.40:0/675097672 wait complete. 2024-04-26T21:46:07.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0349a2a640 1 Processor -- start 2024-04-26T21:46:07.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0349a2a640 1 -- start start 2024-04-26T21:46:07.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0349a2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f03441584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0349a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03440a1440 0x7f03440a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0349a2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034406dc50 0x7f034406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0349a2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03440d5c00 con 0x7f03300039b0 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0349a2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03440d5a80 con 0x7f034406dc50 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0349a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03440d5d80 con 0x7f03440a1440 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0342ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f03441584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0342ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f03441584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40004/0 (socket says 172.21.15.40:40004) 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.376+0000 7f0342ffd640 1 -- 172.21.15.40:0/3646128510 learned_addr learned my addr 172.21.15.40:0/3646128510 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f033bfff640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03440a1440 0x7f03440a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f03437fe640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034406dc50 0x7f034406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0342ffd640 1 -- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03440a1440 msgr2=0x7f03440a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0342ffd640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03440a1440 0x7f03440a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0342ffd640 1 -- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034406dc50 msgr2=0x7f034406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0342ffd640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034406dc50 0x7f034406e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0342ffd640 1 -- 172.21.15.40:0/3646128510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03440a1f00 con 0x7f03300039b0 2024-04-26T21:46:07.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f033bfff640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03440a1440 0x7f03440a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f03437fe640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034406dc50 0x7f034406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0342ffd640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f03441584c0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f0334009be0 tx=0x7f0334004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0340ff9640 1 -- 172.21.15.40:0/3646128510 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334036050 con 0x7f03300039b0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0340ff9640 1 -- 172.21.15.40:0/3646128510 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0334037040 con 0x7f03300039b0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03440a21e0 con 0x7f03300039b0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0340ff9640 1 -- 172.21.15.40:0/3646128510 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340044e0 con 0x7f03300039b0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.377+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03440a27a0 con 0x7f03300039b0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.378+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03440694f0 con 0x7f03300039b0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.379+0000 7f0340ff9640 1 -- 172.21.15.40:0/3646128510 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0334044020 con 0x7f03300039b0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.379+0000 7f0340ff9640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f032406c8f0 0x7f032406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.379+0000 7f0340ff9640 1 -- 172.21.15.40:0/3646128510 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f03340aca80 con 0x7f03300039b0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.379+0000 7f033bfff640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f032406c8f0 0x7f032406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.380+0000 7f033bfff640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f032406c8f0 0x7f032406edb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f034415ecf0 tx=0x7f032c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.383+0000 7f0340ff9640 1 -- 172.21.15.40:0/3646128510 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f033407a150 con 0x7f03300039b0 2024-04-26T21:46:07.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.572+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f034406d300 con 0x7f03300039b0 2024-04-26T21:46:07.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.573+0000 7f0340ff9640 1 -- 172.21.15.40:0/3646128510 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7f0334079a50 con 0x7f03300039b0 2024-04-26T21:46:07.573 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:46:07.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f032406c8f0 msgr2=0x7f032406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f032406c8f0 0x7f032406edb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f034415ecf0 tx=0x7f032c009040 comp rx=0 tx=0).stop 2024-04-26T21:46:07.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 msgr2=0x7f03441584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f03441584c0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f0334009be0 tx=0x7f0334004730 comp rx=0 tx=0).stop 2024-04-26T21:46:07.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 shutdown_connections 2024-04-26T21:46:07.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f032406c8f0 0x7f032406edb0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034406dc50 0x7f034406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03440a1440 0x7f03440a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 --2- 172.21.15.40:0/3646128510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03300039b0 0x7f03441584c0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 >> 172.21.15.40:0/3646128510 conn(0x7f03440d4f60 msgr2=0x7f03440cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 shutdown_connections 2024-04-26T21:46:07.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.575+0000 7f0349a2a640 1 -- 172.21.15.40:0/3646128510 wait complete. 2024-04-26T21:46:07.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.688+0000 7fbb57d71640 1 Processor -- start 2024-04-26T21:46:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.688+0000 7fbb57d71640 1 -- start start 2024-04-26T21:46:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb57d71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50162810 0x7fbb50070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb57d71640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb500b2340 0x7fbb500cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb57d71640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb500d10d0 con 0x7fbb500cd930 2024-04-26T21:46:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb57d71640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbb500d0750 con 0x7fbb500b2340 2024-04-26T21:46:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb57d71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb500d16a0 con 0x7fbb50162810 2024-04-26T21:46:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb5530c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb500b2340 0x7fbb500cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb5630e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbb500cd930 0x7fbb500d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44578/0 (socket says 172.21.15.40:44578) 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb5630e640 1 -- 172.21.15.40:0/456094664 learned_addr learned my addr 172.21.15.40:0/456094664 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb55b0d640 1 --2- 172.21.15.40:0/456094664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50162810 0x7fbb50070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb5530c640 1 -- 172.21.15.40:0/456094664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50162810 msgr2=0x7fbb50070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb5530c640 1 --2- 172.21.15.40:0/456094664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50162810 0x7fbb50070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb5530c640 1 -- 172.21.15.40:0/456094664 >> v1:172.21.15.40:6789/0 conn(0x7fbb500cd930 legacy=0x7fbb500d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb5530c640 1 -- 172.21.15.40:0/456094664 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb500d1370 con 0x7fbb500b2340 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb54b0b640 1 -- 172.21.15.40:0/456094664 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083135006 0 0) 0x7fbb500d10d0 con 0x7fbb500cd930 2024-04-26T21:46:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.689+0000 7fbb55b0d640 1 --2- 172.21.15.40:0/456094664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50162810 0x7fbb50070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.690+0000 7fbb5530c640 1 --2- 172.21.15.40:0/456094664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb500b2340 0x7fbb500cd3f0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fbb400097a0 tx=0x7fbb4002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8141ee2c1b80f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.690+0000 7fbb54b0b640 1 -- 172.21.15.40:0/456094664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40037070 con 0x7fbb500b2340 2024-04-26T21:46:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.690+0000 7fbb54b0b640 1 -- 172.21.15.40:0/456094664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb40032440 con 0x7fbb500b2340 2024-04-26T21:46:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.690+0000 7fbb54b0b640 1 -- 172.21.15.40:0/456094664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40032810 con 0x7fbb500b2340 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.691+0000 7fbb57d71640 1 -- 172.21.15.40:0/456094664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb500b2340 msgr2=0x7fbb500cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.691+0000 7fbb57d71640 1 --2- 172.21.15.40:0/456094664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb500b2340 0x7fbb500cd3f0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fbb400097a0 tx=0x7fbb4002a180 comp rx=0 tx=0).stop 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.691+0000 7fbb57d71640 1 -- 172.21.15.40:0/456094664 shutdown_connections 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.691+0000 7fbb57d71640 1 --2- 172.21.15.40:0/456094664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb500b2340 0x7fbb500cd3f0 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.691+0000 7fbb57d71640 1 --2- 172.21.15.40:0/456094664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50162810 0x7fbb50070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.691+0000 7fbb57d71640 1 -- 172.21.15.40:0/456094664 >> 172.21.15.40:0/456094664 conn(0x7fbb500d2da0 msgr2=0x7fbb500d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.691+0000 7fbb57d71640 1 -- 172.21.15.40:0/456094664 shutdown_connections 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.691+0000 7fbb57d71640 1 -- 172.21.15.40:0/456094664 wait complete. 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.692+0000 7fbb57d71640 1 Processor -- start 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.692+0000 7fbb57d71640 1 -- start start 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb57d71640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb500b2340 0x7fbb5015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb57d71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb500cd930 0x7fbb5015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb57d71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb50162810 0x7fbb500c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb57d71640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb50070890 con 0x7fbb50162810 2024-04-26T21:46:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb57d71640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb500d0470 con 0x7fbb500b2340 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb57d71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb500d2240 con 0x7fbb500cd930 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb50162810 0x7fbb500c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb50162810 0x7fbb500c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40014/0 (socket says 172.21.15.40:40014) 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 -- 172.21.15.40:0/4219120144 learned_addr learned my addr 172.21.15.40:0/4219120144 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 -- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb500cd930 msgr2=0x7fbb5015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5530c640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb500cd930 0x7fbb5015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb55b0d640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb500b2340 0x7fbb5015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb500cd930 0x7fbb5015ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 -- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb500b2340 msgr2=0x7fbb5015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb500b2340 0x7fbb5015e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 -- 172.21.15.40:0/4219120144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb50160990 con 0x7fbb50162810 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5530c640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb500cd930 0x7fbb5015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb55b0d640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb500b2340 0x7fbb5015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb5630e640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb50162810 0x7fbb500c8ca0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c760 tx=0x7fbb4800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb467fc640 1 -- 172.21.15.40:0/4219120144 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48018070 con 0x7fbb50162810 2024-04-26T21:46:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb467fc640 1 -- 172.21.15.40:0/4219120144 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4800a640 con 0x7fbb50162810 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb50160c70 con 0x7fbb50162810 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb467fc640 1 -- 172.21.15.40:0/4219120144 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800aa30 con 0x7fbb50162810 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.693+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb50161230 con 0x7fbb50162810 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.694+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5014ceb0 con 0x7fbb50162810 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.695+0000 7fbb467fc640 1 -- 172.21.15.40:0/4219120144 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb4801b070 con 0x7fbb50162810 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.695+0000 7fbb467fc640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb3406c8f0 0x7fbb3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.696+0000 7fbb467fc640 1 -- 172.21.15.40:0/4219120144 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7fbb4808be70 con 0x7fbb50162810 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.696+0000 7fbb55b0d640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb3406c8f0 0x7fbb3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.696+0000 7fbb55b0d640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb3406c8f0 0x7fbb3406edb0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c006fd0 tx=0x7fbb4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:07.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.700+0000 7fbb467fc640 1 -- 172.21.15.40:0/4219120144 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb5014ceb0 con 0x7fbb50162810 2024-04-26T21:46:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.897+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb500694f0 con 0x7fbb50162810 2024-04-26T21:46:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.897+0000 7fbb467fc640 1 -- 172.21.15.40:0/4219120144 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 66 entries v128) ==== 81+0+4016 (secure 0 0 0) 0x7fbb480594e0 con 0x7fbb50162810 2024-04-26T21:46:07.897 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1660522931 2024-04-27T21:45:51.733253+0000 2024-04-26T21:46:07.897 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4214786907 2024-04-27T21:45:46.312381+0000 2024-04-26T21:46:07.897 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2896027915 2024-04-27T21:45:31.907924+0000 2024-04-26T21:46:07.897 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2896027915 2024-04-27T21:45:31.907924+0000 2024-04-26T21:46:07.897 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4173136579 2024-04-27T21:45:27.871638+0000 2024-04-26T21:46:07.897 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3791539151 2024-04-27T21:45:23.843779+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:46:07.898 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4192917874 2024-04-27T21:45:19.814510+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4214786907 2024-04-27T21:45:46.312381+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3791539151 2024-04-27T21:45:23.843779+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1660522931 2024-04-27T21:45:51.733253+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3538213517 2024-04-27T21:45:17.788819+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:46:07.899 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4192917874 2024-04-27T21:45:19.814510+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:46:07.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4173136579 2024-04-27T21:45:27.871638+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:46:07.901 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:46:07.902 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:46:07.902 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3538213517 2024-04-27T21:45:17.788819+0000 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:listed 66 entries 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb3406c8f0 msgr2=0x7fbb3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb3406c8f0 0x7fbb3406edb0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c006fd0 tx=0x7fbb4c008040 comp rx=0 tx=0).stop 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb50162810 msgr2=0x7fbb500c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb50162810 0x7fbb500c8ca0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c760 tx=0x7fbb4800cc20 comp rx=0 tx=0).stop 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 shutdown_connections 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb3406c8f0 0x7fbb3406edb0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb50162810 0x7fbb500c8ca0 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb500cd930 0x7fbb5015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 --2- 172.21.15.40:0/4219120144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb500b2340 0x7fbb5015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 >> 172.21.15.40:0/4219120144 conn(0x7fbb500d2da0 msgr2=0x7fbb500b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 shutdown_connections 2024-04-26T21:46:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:07.899+0000 7fbb57d71640 1 -- 172.21.15.40:0/4219120144 wait complete. 2024-04-26T21:46:07.910 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:46:07.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:07.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:46:07.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:46:07.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:07.936 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:46:07.991 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:46:08.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.133+0000 7f69fefda640 1 Processor -- start 2024-04-26T21:46:08.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.134+0000 7f69fefda640 1 -- start start 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.134+0000 7f69fefda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f69f815b600 0x7f69f8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.134+0000 7f69fefda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69f80cf5e0 0x7f69f80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.134+0000 7f69fefda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69f8072450 con 0x7f69f80cbbd0 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.134+0000 7f69fefda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f69f80cb9b0 con 0x7f69f80cf5e0 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.134+0000 7f69fefda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f69f815b9e0 con 0x7f69f815b600 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.134+0000 7f69f7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69f80cf5e0 0x7f69f80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.135+0000 7f69f7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69f80cf5e0 0x7f69f80ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40018/0 (socket says 172.21.15.40:40018) 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.135+0000 7f69f7fff640 1 -- 172.21.15.40:0/552126170 learned_addr learned my addr 172.21.15.40:0/552126170 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.135+0000 7f69fcd76640 1 --2- 172.21.15.40:0/552126170 >> v2:172.21.15.184:3300/0 conn(0x7f69f815b600 0x7f69f8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.135+0000 7f69f7fff640 1 -- 172.21.15.40:0/552126170 >> v2:172.21.15.184:3300/0 conn(0x7f69f815b600 msgr2=0x7f69f8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.135+0000 7f69f7fff640 1 --2- 172.21.15.40:0/552126170 >> v2:172.21.15.184:3300/0 conn(0x7f69f815b600 0x7f69f8071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.135+0000 7f69f7fff640 1 -- 172.21.15.40:0/552126170 >> v1:172.21.15.184:6789/0 conn(0x7f69f80cbbd0 legacy=0x7f69f806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.135+0000 7f69f7fff640 1 -- 172.21.15.40:0/552126170 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f80d0740 con 0x7f69f80cf5e0 2024-04-26T21:46:08.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.135+0000 7f69f7fff640 1 --2- 172.21.15.40:0/552126170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f80cf5e0 0x7f69f80ce9d0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f69f80cbb30 tx=0x7f69e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0517e794b7c12e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69f77fe640 1 -- 172.21.15.40:0/552126170 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4037070 con 0x7f69f80cf5e0 2024-04-26T21:46:08.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69f77fe640 1 -- 172.21.15.40:0/552126170 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e4032440 con 0x7f69f80cf5e0 2024-04-26T21:46:08.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69f77fe640 1 -- 172.21.15.40:0/552126170 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4032810 con 0x7f69f80cf5e0 2024-04-26T21:46:08.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69fefda640 1 -- 172.21.15.40:0/552126170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f80cf5e0 msgr2=0x7f69f80ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69fefda640 1 --2- 172.21.15.40:0/552126170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f80cf5e0 0x7f69f80ce9d0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f69f80cbb30 tx=0x7f69e402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69fefda640 1 -- 172.21.15.40:0/552126170 shutdown_connections 2024-04-26T21:46:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69fefda640 1 --2- 172.21.15.40:0/552126170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f80cf5e0 0x7f69f80ce9d0 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69fefda640 1 --2- 172.21.15.40:0/552126170 >> v2:172.21.15.184:3300/0 conn(0x7f69f815b600 0x7f69f8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69fefda640 1 -- 172.21.15.40:0/552126170 >> 172.21.15.40:0/552126170 conn(0x7f69f80d0be0 msgr2=0x7f69f80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69fefda640 1 -- 172.21.15.40:0/552126170 shutdown_connections 2024-04-26T21:46:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.136+0000 7f69fefda640 1 -- 172.21.15.40:0/552126170 wait complete. 2024-04-26T21:46:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.137+0000 7f69fefda640 1 Processor -- start 2024-04-26T21:46:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.137+0000 7f69fefda640 1 -- start start 2024-04-26T21:46:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.138+0000 7f69fefda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f80cbbd0 0x7f69f80b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.138+0000 7f69fefda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f80cf5e0 0x7f69f8153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.138+0000 7f69fefda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f815b600 0x7f69f80ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.138+0000 7f69fefda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f69f80d02f0 con 0x7f69f815b600 2024-04-26T21:46:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.138+0000 7f69fefda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69f80cb640 con 0x7f69f80cf5e0 2024-04-26T21:46:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.138+0000 7f69fefda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69f80d0470 con 0x7f69f80cbbd0 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.138+0000 7f69fd577640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f815b600 0x7f69f80ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fd577640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f815b600 0x7f69f80ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40034/0 (socket says 172.21.15.40:40034) 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fd577640 1 -- 172.21.15.40:0/208860569 learned_addr learned my addr 172.21.15.40:0/208860569 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fcd76640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f80cbbd0 0x7f69f80b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69f7fff640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f80cf5e0 0x7f69f8153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fd577640 1 -- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f80cbbd0 msgr2=0x7f69f80b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fd577640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f80cbbd0 0x7f69f80b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fd577640 1 -- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f80cf5e0 msgr2=0x7f69f8153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fd577640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f80cf5e0 0x7f69f8153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fd577640 1 -- 172.21.15.40:0/208860569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f8149900 con 0x7f69f815b600 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fd577640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f815b600 0x7f69f80ca1b0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f69f000c7d0 tx=0x7f69f000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69f57fa640 1 -- 172.21.15.40:0/208860569 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0003b70 con 0x7f69f815b600 2024-04-26T21:46:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69f57fa640 1 -- 172.21.15.40:0/208860569 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f0003d30 con 0x7f69f815b600 2024-04-26T21:46:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69f57fa640 1 -- 172.21.15.40:0/208860569 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f000aa90 con 0x7f69f815b600 2024-04-26T21:46:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f80b5b90 con 0x7f69f815b600 2024-04-26T21:46:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.139+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f80ca6f0 con 0x7f69f815b600 2024-04-26T21:46:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.140+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f8073750 con 0x7f69f815b600 2024-04-26T21:46:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.141+0000 7f69f57fa640 1 -- 172.21.15.40:0/208860569 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69f000ac30 con 0x7f69f815b600 2024-04-26T21:46:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.141+0000 7f69f57fa640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69dc06c8f0 0x7f69dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.141+0000 7f69f57fa640 1 -- 172.21.15.40:0/208860569 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9055+0+0 (secure 0 0 0) 0x7f69f008c680 con 0x7f69f815b600 2024-04-26T21:46:08.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.141+0000 7f69fcd76640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69dc06c8f0 0x7f69dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.142+0000 7f69fcd76640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69dc06c8f0 0x7f69dc06edb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f69ec006fd0 tx=0x7f69ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.145+0000 7f69f57fa640 1 -- 172.21.15.40:0/208860569 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f69f8149900 con 0x7f69f815b600 2024-04-26T21:46:08.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.369+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f69f80ce7b0 con 0x7f69f815b600 2024-04-26T21:46:08.370 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:08.370+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:46:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.372+0000 7f69f57fa640 1 -- 172.21.15.40:0/208860569 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v305) ==== 161+0+0 (secure 0 0 0) 0x7f69f8072450 con 0x7f69f815b600 2024-04-26T21:46:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69dc06c8f0 msgr2=0x7f69dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69dc06c8f0 0x7f69dc06edb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f69ec006fd0 tx=0x7f69ec008040 comp rx=0 tx=0).stop 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f815b600 msgr2=0x7f69f80ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f815b600 0x7f69f80ca1b0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f69f000c7d0 tx=0x7f69f000cc90 comp rx=0 tx=0).stop 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 shutdown_connections 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f69dc06c8f0 0x7f69dc06edb0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69f815b600 0x7f69f80ca1b0 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f80cf5e0 0x7f69f8153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 --2- 172.21.15.40:0/208860569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f80cbbd0 0x7f69f80b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 >> 172.21.15.40:0/208860569 conn(0x7f69f80d0be0 msgr2=0x7f69f80b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 shutdown_connections 2024-04-26T21:46:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.374+0000 7f69fefda640 1 -- 172.21.15.40:0/208860569 wait complete. 2024-04-26T21:46:08.375 INFO:tasks.ceph.mds.c.smithi040.stderr: -575> 2024-04-26T21:46:02.746+0000 7faee4b21640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:46:08.386 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:46:08.401 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:46:08.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.489+0000 7fc4ced24640 1 Processor -- start 2024-04-26T21:46:08.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.489+0000 7fc4ced24640 1 -- start start 2024-04-26T21:46:08.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.490+0000 7fc4ced24640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc4c80d06f0 0x7fc4c80d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.490+0000 7fc4ced24640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c80d0ba0 con 0x7fc4c80b2850 2024-04-26T21:46:08.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.490+0000 7fc4ced24640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c80d0e20 con 0x7fc4c815f3a0 2024-04-26T21:46:08.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.490+0000 7fc4ced24640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c80b1d70 con 0x7fc4c80d06f0 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.490+0000 7fc4cd2c1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc4c80b2850 0x7fc4c80b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44592/0 (socket says 172.21.15.40:44592) 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.490+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/3207071208 learned_addr learned my addr 172.21.15.40:0/3207071208 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c7fff640 1 --2- 172.21.15.40:0/3207071208 >> v2:172.21.15.184:3300/0 conn(0x7fc4c80d06f0 0x7fc4c80d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990303345 0 0) 0x7fc4b8001110 con 0x7fc4c80b2850 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4b0003520 con 0x7fc4c80b2850 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3934043111 0 0) 0x7fc4b0003520 con 0x7fc4c80b2850 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 >> v2:172.21.15.184:3300/0 conn(0x7fc4c80d06f0 msgr2=0x7fc4c80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 --2- 172.21.15.40:0/3207071208 >> v2:172.21.15.184:3300/0 conn(0x7fc4c80d06f0 0x7fc4c80d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 >> v1:172.21.15.184:6789/0 conn(0x7fc4c815f3a0 legacy=0x7fc4c80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c80b2050 con 0x7fc4c80b2850 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fc4b8001840 con 0x7fc4c80b2850 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 >> v1:172.21.15.40:6789/0 conn(0x7fc4c80b2850 legacy=0x7fc4c80b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b00039b0 0x7fc4b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b0004800 0x7fc4b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b00051c0 0x7fc4b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c80b1d70 con 0x7fc4b00051c0 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b000c060 con 0x7fc4b00039b0 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b000c260 con 0x7fc4b0004800 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4cd2c1640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b00051c0 0x7fc4b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4ccac0640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b00039b0 0x7fc4b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4c7fff640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b0004800 0x7fc4b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/3207071208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b0004800 msgr2=0x7fc4b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4cd2c1640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b0004800 0x7fc4b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/3207071208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b00039b0 msgr2=0x7fc4b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4cd2c1640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b00039b0 0x7fc4b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.491+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/3207071208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b0003520 con 0x7fc4b00051c0 2024-04-26T21:46:08.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.492+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/3207071208 reap_dead start 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.492+0000 7fc4cd2c1640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b00051c0 0x7fc4b000b9c0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800c4c0 tx=0x7fc4b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0f3024160708f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.492+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800c0e0 con 0x7fc4b00051c0 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.492+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b80058d0 con 0x7fc4b00051c0 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.492+0000 7fc4c77fe640 1 -- 172.21.15.40:0/3207071208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8005ce0 con 0x7fc4b00051c0 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.493+0000 7fc4ced24640 1 -- 172.21.15.40:0/3207071208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b00051c0 msgr2=0x7fc4b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.493+0000 7fc4ced24640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b00051c0 0x7fc4b000b9c0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800c4c0 tx=0x7fc4b0005830 comp rx=0 tx=0).stop 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.493+0000 7fc4ced24640 1 -- 172.21.15.40:0/3207071208 shutdown_connections 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.493+0000 7fc4ced24640 1 --2- 172.21.15.40:0/3207071208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b00051c0 0x7fc4b000b9c0 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.493+0000 7fc4ced24640 1 -- 172.21.15.40:0/3207071208 >> 172.21.15.40:0/3207071208 conn(0x7fc4c80d4f90 msgr2=0x7fc4c80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.493+0000 7fc4ced24640 1 -- 172.21.15.40:0/3207071208 shutdown_connections 2024-04-26T21:46:08.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.493+0000 7fc4ced24640 1 -- 172.21.15.40:0/3207071208 wait complete. 2024-04-26T21:46:08.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.494+0000 7fc4ced24640 1 Processor -- start 2024-04-26T21:46:08.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.494+0000 7fc4ced24640 1 -- start start 2024-04-26T21:46:08.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4ced24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b00051c0 0x7fc4c80cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4ced24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c80b2c30 0x7fc4c80b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4ced24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c814ec90 0x7fc4c814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4ced24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c815f900 con 0x7fc4c814ec90 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4ced24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c815f780 con 0x7fc4c80b2c30 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4ced24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c80d1770 con 0x7fc4b00051c0 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c814ec90 0x7fc4c814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c814ec90 0x7fc4c814f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40066/0 (socket says 172.21.15.40:40066) 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/2483432681 learned_addr learned my addr 172.21.15.40:0/2483432681 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4ccac0640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b00051c0 0x7fc4c80cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4c7fff640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c80b2c30 0x7fc4c80b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b00051c0 msgr2=0x7fc4c80cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b00051c0 0x7fc4c80cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c80b2c30 msgr2=0x7fc4c80b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c80b2c30 0x7fc4c80b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 -- 172.21.15.40:0/2483432681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c80cf500 con 0x7fc4c814ec90 2024-04-26T21:46:08.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4ccac0640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b00051c0 0x7fc4c80cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4c7fff640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c80b2c30 0x7fc4c80b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.495+0000 7fc4cd2c1640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c814ec90 0x7fc4c814f140 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80013d0 tx=0x7fc4b8001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.496+0000 7fc4c57fa640 1 -- 172.21.15.40:0/2483432681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b803e030 con 0x7fc4c814ec90 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.496+0000 7fc4c57fa640 1 -- 172.21.15.40:0/2483432681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b8001110 con 0x7fc4c814ec90 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.496+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c814f9f0 con 0x7fc4c814ec90 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.496+0000 7fc4c57fa640 1 -- 172.21.15.40:0/2483432681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80049f0 con 0x7fc4c814ec90 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.496+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c814ffb0 con 0x7fc4c814ec90 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.497+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c80694f0 con 0x7fc4c814ec90 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.497+0000 7fc4c57fa640 1 -- 172.21.15.40:0/2483432681 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4b8041030 con 0x7fc4c814ec90 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.498+0000 7fc4c57fa640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.498+0000 7fc4c57fa640 1 -- 172.21.15.40:0/2483432681 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7fc4b80ac600 con 0x7fc4c814ec90 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.498+0000 7fc4ccac0640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:08.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.498+0000 7fc4ccac0640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc4b406c8f0 0x7fc4b406edb0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000f80 tx=0x7fc4c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:08.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.502+0000 7fc4c57fa640 1 -- 172.21.15.40:0/2483432681 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4c80d0ba0 con 0x7fc4c814ec90 2024-04-26T21:46:08.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:08.764+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc4c80b6860 con 0x7fc4c814ec90 2024-04-26T21:46:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.373+0000 7fc4c57fa640 1 -- 172.21.15.40:0/2483432681 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v306) ==== 107+0+0 (secure 0 0 0) 0x7fc4b80796b0 con 0x7fc4c814ec90 2024-04-26T21:46:09.374 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:46:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc4b406c8f0 msgr2=0x7fc4b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc4b406c8f0 0x7fc4b406edb0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000f80 tx=0x7fc4c0009040 comp rx=0 tx=0).stop 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c814ec90 msgr2=0x7fc4c814f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c814ec90 0x7fc4c814f140 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80013d0 tx=0x7fc4b8001c10 comp rx=0 tx=0).stop 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 shutdown_connections 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c814ec90 0x7fc4c814f140 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c80b2c30 0x7fc4c80b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 --2- 172.21.15.40:0/2483432681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b00051c0 0x7fc4c80cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 >> 172.21.15.40:0/2483432681 conn(0x7fc4c80d4f90 msgr2=0x7fc4c80d33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 shutdown_connections 2024-04-26T21:46:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.376+0000 7fc4ced24640 1 -- 172.21.15.40:0/2483432681 wait complete. 2024-04-26T21:46:09.389 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:46:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.473+0000 7efcef67af40 1 Processor -- start 2024-04-26T21:46:09.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.473+0000 7efcef67af40 1 -- start start 2024-04-26T21:46:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.473+0000 7efcef67af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab043cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcef67af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7a80 0x55ab043b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcef67af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7e60 0x55ab0433c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcef67af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ab042a3e40 con 0x55ab043458c0 2024-04-26T21:46:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcef67af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ab042a7e80 con 0x55ab043b7a80 2024-04-26T21:46:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcef67af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ab04344570 con 0x55ab043b7e60 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab043cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab043cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35946/0 (socket says 172.21.15.40:35946) 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 -- 172.21.15.40:0/1569907274 learned_addr learned my addr 172.21.15.40:0/1569907274 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcec973640 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7a80 0x55ab043b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced975640 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7e60 0x55ab0433c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 -- 172.21.15.40:0/1569907274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7e60 msgr2=0x55ab0433c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7e60 0x55ab0433c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 -- 172.21.15.40:0/1569907274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7a80 msgr2=0x55ab043b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7a80 0x55ab043b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 -- 172.21.15.40:0/1569907274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab043d17b0 con 0x55ab043458c0 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efced174640 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab043cb590 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7efce0001e00 tx=0x7efce000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bb2b92a36a3f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcdf7fe640 1 -- 172.21.15.40:0/1569907274 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce000be40 con 0x55ab043458c0 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcdf7fe640 1 -- 172.21.15.40:0/1569907274 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce000f400 con 0x55ab043458c0 2024-04-26T21:46:09.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.474+0000 7efcdf7fe640 1 -- 172.21.15.40:0/1569907274 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce000f7d0 con 0x55ab043458c0 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 -- 172.21.15.40:0/1569907274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 msgr2=0x55ab043cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab043cb590 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7efce0001e00 tx=0x7efce000b670 comp rx=0 tx=0).stop 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 -- 172.21.15.40:0/1569907274 shutdown_connections 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7e60 0x55ab0433c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7a80 0x55ab043b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 --2- 172.21.15.40:0/1569907274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab043cb590 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 -- 172.21.15.40:0/1569907274 >> 172.21.15.40:0/1569907274 conn(0x55ab042faab0 msgr2=0x55ab042fb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 -- 172.21.15.40:0/1569907274 shutdown_connections 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 -- 172.21.15.40:0/1569907274 wait complete. 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.475+0000 7efcef67af40 1 Processor -- start 2024-04-26T21:46:09.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efcef67af40 1 -- start start 2024-04-26T21:46:09.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efcef67af40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab0433c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efcef67af40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7a80 0x55ab0446b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efcef67af40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7e60 0x55ab0446bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efcef67af40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ab043d0dd0 con 0x55ab043458c0 2024-04-26T21:46:09.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efcef67af40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ab043d0780 con 0x55ab043b7e60 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efcef67af40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ab043b6680 con 0x55ab043b7a80 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efced174640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab0433c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efced174640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab0433c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35956/0 (socket says 172.21.15.40:35956) 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 learned_addr learned my addr 172.21.15.40:0/2690832401 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efcec973640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7a80 0x55ab0446b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.476+0000 7efced975640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7e60 0x55ab0446bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7a80 msgr2=0x55ab0446b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efced174640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ab043b7a80 0x55ab0446b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7e60 msgr2=0x55ab0446bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efced174640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7e60 0x55ab0446bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab043415f0 con 0x55ab043458c0 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efced975640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab043b7e60 0x55ab0446bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efced174640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab0433c960 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7efce0000c00 tx=0x7efce0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efcde7fc640 1 -- 172.21.15.40:0/2690832401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce001c440 con 0x55ab043458c0 2024-04-26T21:46:09.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efcde7fc640 1 -- 172.21.15.40:0/2690832401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce0021020 con 0x55ab043458c0 2024-04-26T21:46:09.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ab043b5b10 con 0x55ab043458c0 2024-04-26T21:46:09.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efcde7fc640 1 -- 172.21.15.40:0/2690832401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce001c6a0 con 0x55ab043458c0 2024-04-26T21:46:09.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.477+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab04341fb0 con 0x55ab043458c0 2024-04-26T21:46:09.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.478+0000 7efcde7fc640 1 -- 172.21.15.40:0/2690832401 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efce0017070 con 0x55ab043458c0 2024-04-26T21:46:09.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.479+0000 7efcde7fc640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcd006b2d0 0x7efcd006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.479+0000 7efcde7fc640 1 -- 172.21.15.40:0/2690832401 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7efce008cbc0 con 0x55ab043458c0 2024-04-26T21:46:09.481 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:46:09.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.479+0000 7efcec973640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcd006b2d0 0x7efcd006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.479+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ab0433cea0 0x55ab04340050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.479+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04341a60 con 0x55ab0433cea0 2024-04-26T21:46:09.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.479+0000 7efced975640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ab0433cea0 0x55ab04340050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.480+0000 7efcec973640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcd006b2d0 0x7efcd006d790 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efce4001ae0 tx=0x7efce4001a70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.480+0000 7efced975640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ab0433cea0 0x55ab04340050 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.481+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v129'1033 uv1007 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.482+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04340590 con 0x55ab0433cea0 2024-04-26T21:46:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.483+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v129'1034 uv1034 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.483+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0446c250 con 0x55ab0433cea0 2024-04-26T21:46:09.483 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:46:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.484+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v129'1035 uv1035 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.484+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475ab0 con 0x55ab0433cea0 2024-04-26T21:46:09.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.485+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v129'1036 uv1036 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.485+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475e50 con 0x55ab0433cea0 2024-04-26T21:46:09.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.486+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v129'1037 uv1037 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.486+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04474aa0 con 0x55ab0433cea0 2024-04-26T21:46:09.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.487+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v129'1038 uv1038 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.487+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04474e60 con 0x55ab0433cea0 2024-04-26T21:46:09.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.488+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v129'1039 uv1013 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.488+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.490+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v129'1040 uv1040 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.490+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.490 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:46:09.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.491+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v129'1041 uv1041 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.491+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.492+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v129'1042 uv1042 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.492+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.493+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v129'1043 uv1043 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.493+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.494+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v129'1044 uv1044 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.494+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.495+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v129'1045 uv1020 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.495+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.496+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v129'1046 uv1046 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.496+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.497+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v129'1047 uv1047 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.497 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:46:09.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.497+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.498+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v129'1048 uv1048 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.498+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.500+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v129'1049 uv1049 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.500+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04475200 con 0x55ab0433cea0 2024-04-26T21:46:09.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.501+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v129'1050 uv1050 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.501+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.502+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v129'1051 uv1051 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.502+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.503+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v129'1052 uv1026 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.503+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.504+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v129'1053 uv1053 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.504+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.505 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:46:09.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.505+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v129'1054 uv1054 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.505+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04477200 con 0x55ab0433cea0 2024-04-26T21:46:09.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.506+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v129'1055 uv1055 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.506+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04477200 con 0x55ab0433cea0 2024-04-26T21:46:09.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.507+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v129'1056 uv1056 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.507+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.508+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v129'1057 uv1057 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.508+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.509+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v129'1058 uv1032 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.509+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.510+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v129'1059 uv1059 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.510+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.511 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:46:09.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.512+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v129'1060 uv1060 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.512+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04477200 con 0x55ab0433cea0 2024-04-26T21:46:09.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.513+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v129'1061 uv1061 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.513+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04477200 con 0x55ab0433cea0 2024-04-26T21:46:09.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.514+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v129'1062 uv1062 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.514+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab044761f0 con 0x55ab0433cea0 2024-04-26T21:46:09.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.515+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v129'1063 uv1063 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0433cea0 2024-04-26T21:46:09.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.515+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ab04476700 0x55ab04478af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.515+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04479030 con 0x55ab04476700 2024-04-26T21:46:09.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.515+0000 7efced174640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ab04476700 0x55ab04478af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.515+0000 7efced174640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ab04476700 0x55ab04478af0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.517+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v129'752 uv731 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.517+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab044793d0 con 0x55ab04476700 2024-04-26T21:46:09.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.519+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v129'753 uv753 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.519+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04479770 con 0x55ab04476700 2024-04-26T21:46:09.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.520+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v129'754 uv754 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.520 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:46:09.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.520+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04479fc0 con 0x55ab04476700 2024-04-26T21:46:09.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.522+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v129'755 uv755 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.522+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447a360 con 0x55ab04476700 2024-04-26T21:46:09.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.523+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v129'756 uv756 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.523+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447a700 con 0x55ab04476700 2024-04-26T21:46:09.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.524+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v129'757 uv750 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.524+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab0447aaa0 con 0x55ab04476700 2024-04-26T21:46:09.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.526+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v129'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.526+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.527+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v129'759 uv759 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.527 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:46:09.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.527+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.528+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v129'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.528+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.530+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v129'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.530+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.531+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v129'762 uv751 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.531+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.532+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v129'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.532+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.533+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v129'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.533 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:46:09.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.533+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447f500 con 0x55ab04476700 2024-04-26T21:46:09.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.535+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v129'765 uv765 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.535+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447f500 con 0x55ab04476700 2024-04-26T21:46:09.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.536+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v129'766 uv766 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.536+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.537+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v129'767 uv747 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.537+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.538+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v129'768 uv768 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.538+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.539 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:46:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.539+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v129'769 uv769 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.539+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447be50 con 0x55ab04476700 2024-04-26T21:46:09.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.541+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v129'770 uv770 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.541+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447be50 con 0x55ab04476700 2024-04-26T21:46:09.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.542+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v129'771 uv771 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.542+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447ae40 con 0x55ab04476700 2024-04-26T21:46:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.543+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v129'772 uv772 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04476700 2024-04-26T21:46:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.543+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ab0447b350 0x55ab0447d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.543+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447dc80 con 0x55ab0447b350 2024-04-26T21:46:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.543+0000 7efcec973640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ab0447b350 0x55ab0447d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.544+0000 7efcec973640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ab0447b350 0x55ab0447d740 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.545+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v129'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.545+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab0447e020 con 0x55ab0447b350 2024-04-26T21:46:09.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.546+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v129'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.546+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447e3c0 con 0x55ab0447b350 2024-04-26T21:46:09.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.547+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v129'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.547 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:46:09.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.547+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447f770 con 0x55ab0447b350 2024-04-26T21:46:09.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.548+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v129'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.548+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447fb10 con 0x55ab0447b350 2024-04-26T21:46:09.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.549+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v129'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.549+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0447e760 con 0x55ab0447b350 2024-04-26T21:46:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.550+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v129'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.550+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ab0447f220 0x55ab04481ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.550+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04482470 con 0x55ab0447f220 2024-04-26T21:46:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.550+0000 7efced975640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ab0447f220 0x55ab04481ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.551+0000 7efced975640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ab0447f220 0x55ab04481ec0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.552+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v129'154 uv153 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.552+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04482860 con 0x55ab0447f220 2024-04-26T21:46:09.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.553+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v129'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.554+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04482860 con 0x55ab0447f220 2024-04-26T21:46:09.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.555+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v129'156 uv156 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.555 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:46:09.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.555+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04482860 con 0x55ab0447f220 2024-04-26T21:46:09.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.557+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v129'157 uv157 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.557+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ab04483190 0x55ab04485580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.557+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485ac0 con 0x55ab04483190 2024-04-26T21:46:09.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.558+0000 7efced174640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ab04483190 0x55ab04485580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.558+0000 7efced174640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ab04483190 0x55ab04485580 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.559+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v129'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.559+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04485ac0 con 0x55ab04483190 2024-04-26T21:46:09.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.560+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v129'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.560+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485b00 con 0x55ab04483190 2024-04-26T21:46:09.561 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:46:09.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.562+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v129'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.562+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04486f30 con 0x55ab04483190 2024-04-26T21:46:09.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.563+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v129'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.563+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04486f30 con 0x55ab04483190 2024-04-26T21:46:09.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.564+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v129'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.564+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.565+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v129'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.565+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.566+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v129'371 uv365 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.566+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.568+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v129'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.568+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.568 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:46:09.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.569+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v129'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.569+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04486f30 con 0x55ab0447f220 2024-04-26T21:46:09.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.570+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v129'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.570+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04486f30 con 0x55ab0447f220 2024-04-26T21:46:09.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.571+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v129'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.571+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.572+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v129'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.572+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.573+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v129'377 uv370 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.573+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.574+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v129'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.574+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.575 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:46:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.575+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v129'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.576+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.577+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v129'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.577+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.578+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v129'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.578+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.579+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v129'168 uv167 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.579+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.580+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v129'169 uv169 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.580+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.580 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:46:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.581+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v129'170 uv170 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.581+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.581+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v129'171 uv171 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.581+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab0447f220 2024-04-26T21:46:09.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.582+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v129'172 uv172 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efce804f970 con 0x55ab0447f220 2024-04-26T21:46:09.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.582+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.584+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v129'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.584+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.585+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v129'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.585+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.586+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v129'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.585 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:46:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.586+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04486f30 con 0x55ab04483190 2024-04-26T21:46:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.587+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v129'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.587+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04486f30 con 0x55ab04483190 2024-04-26T21:46:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.588+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v129'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.588+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.589+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v129'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.589+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.590+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v129'335 uv329 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.590+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.592+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v129'336 uv336 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.592+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.592 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:46:09.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.593+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v129'337 uv337 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.593+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.594+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v129'338 uv338 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.594+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.595+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v129'339 uv339 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.595+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.596+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v129'340 uv334 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.596+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.597+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v129'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.597+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.598 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:46:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.598+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v129'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.598+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04485f20 con 0x55ab04483190 2024-04-26T21:46:09.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.599+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v129'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.599+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04488d30 con 0x55ab04483190 2024-04-26T21:46:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.601+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v129'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.601+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ab04486a50 0x55ab044892f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.601+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04488e40 con 0x55ab04486a50 2024-04-26T21:46:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.601+0000 7efcec973640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ab04486a50 0x55ab044892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.602+0000 7efcec973640 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ab04486a50 0x55ab044892f0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.603+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v129'147 uv145 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab04486a50 2024-04-26T21:46:09.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.603+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab04486a50 2024-04-26T21:46:09.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.604+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v129'148 uv148 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab04486a50 2024-04-26T21:46:09.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.604+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab04486a50 2024-04-26T21:46:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.605+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v129'149 uv149 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab04486a50 2024-04-26T21:46:09.605 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:46:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.605+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab04486a50 2024-04-26T21:46:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.606+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v129'150 uv150 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab04486a50 2024-04-26T21:46:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.606+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab04483190 2024-04-26T21:46:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.607+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v129'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.608+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab04483190 2024-04-26T21:46:09.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.609+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v129'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.609+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab04483190 2024-04-26T21:46:09.609 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:46:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.610+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v129'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.610+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0448a840 con 0x55ab04483190 2024-04-26T21:46:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.611+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v129'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.611+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0448a840 con 0x55ab04483190 2024-04-26T21:46:09.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.612+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v129'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.612+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab04483190 2024-04-26T21:46:09.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.613+0000 7efced174640 1 -- 172.21.15.40:0/2690832401 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v129'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce0056230 con 0x55ab04483190 2024-04-26T21:46:09.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.613+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.615+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v129'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.615+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.616+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v129'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.616+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.617 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:46:09.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.617+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v129'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.617+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.619+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v129'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.619+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.620+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v129'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.620+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.621+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v129'416 uv406 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.622+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.623+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v129'417 uv417 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.623+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.624+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v129'418 uv418 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.624 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:46:09.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.624+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.625+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v129'419 uv419 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.625+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.626+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v129'420 uv420 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.626+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.627+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v129'421 uv414 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.627+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.628+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v129'422 uv422 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.628+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489830 con 0x55ab0447b350 2024-04-26T21:46:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.629+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v129'423 uv423 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.629 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:46:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.629+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0448ac30 con 0x55ab0447b350 2024-04-26T21:46:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.630+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v129'424 uv424 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.631+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab0448ac30 con 0x55ab0447b350 2024-04-26T21:46:09.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.631+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v129'425 uv425 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.632+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489970 con 0x55ab0447b350 2024-04-26T21:46:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.633+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v129'426 uv426 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.633+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ab04489e20 con 0x55ab0447b350 2024-04-26T21:46:09.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcec973640 1 -- 172.21.15.40:0/2690832401 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v129'427 uv427 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efce4011970 con 0x55ab0447b350 2024-04-26T21:46:09.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ab04476700 msgr2=0x55ab04478af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55ab04476700 0x55ab04478af0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ab0433cea0 msgr2=0x55ab04340050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55ab0433cea0 0x55ab04340050 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ab0447f220 msgr2=0x55ab04481ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55ab0447f220 0x55ab04481ec0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ab04486a50 msgr2=0x55ab044892f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ab04486a50 0x55ab044892f0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ab0447b350 msgr2=0x55ab0447d740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55ab0447b350 0x55ab0447d740 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ab04483190 msgr2=0x55ab04485580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55ab04483190 0x55ab04485580 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 reap_dead start 2024-04-26T21:46:09.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efced975640 1 -- 172.21.15.40:0/2690832401 reap_dead start 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcd006b2d0 msgr2=0x7efcd006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcd006b2d0 0x7efcd006d790 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efce4001ae0 tx=0x7efce4001a70 comp rx=0 tx=0).stop 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 msgr2=0x55ab0433c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab0433c960 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7efce0000c00 tx=0x7efce0000ed0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 shutdown_connections 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efcd006b2d0 0x7efcd006d790 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 --2- 172.21.15.40:0/2690832401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ab043458c0 0x55ab0433c960 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 >> 172.21.15.40:0/2690832401 conn(0x55ab042faab0 msgr2=0x55ab043b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 shutdown_connections 2024-04-26T21:46:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.634+0000 7efcef67af40 1 -- 172.21.15.40:0/2690832401 wait complete. 2024-04-26T21:46:09.637 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 10000000000.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:46:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.721+0000 7f2041a83f40 1 Processor -- start 2024-04-26T21:46:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.721+0000 7f2041a83f40 1 -- start start 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f2041a83f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c58351c0 0x5581c5865120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f2041a83f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c5880420 0x5581c590e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f2041a83f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5907f70 0x5581c590ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f2041a83f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5581c57dde40 con 0x5581c58351c0 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f2041a83f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5581c587f660 con 0x5581c5907f70 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f2041a83f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5581c58ff120 con 0x5581c5880420 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f203f57d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c58351c0 0x5581c5865120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f203f57d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c58351c0 0x5581c5865120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35962/0 (socket says 172.21.15.40:35962) 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f203f57d640 1 -- 172.21.15.40:0/1064946815 learned_addr learned my addr 172.21.15.40:0/1064946815 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f203ed7c640 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c5880420 0x5581c590e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.722+0000 7f203fd7e640 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5907f70 0x5581c590ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203f57d640 1 -- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c5880420 msgr2=0x5581c590e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203f57d640 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c5880420 0x5581c590e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203f57d640 1 -- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5907f70 msgr2=0x5581c590ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203f57d640 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5907f70 0x5581c590ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203f57d640 1 -- 172.21.15.40:0/1064946815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c587cc10 con 0x5581c58351c0 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203ed7c640 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c5880420 0x5581c590e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203f57d640 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c58351c0 0x5581c5865120 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f2038009790 tx=0x7f2038005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948f3abfbc799f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203dd7a640 1 -- 172.21.15.40:0/1064946815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038008870 con 0x5581c58351c0 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203dd7a640 1 -- 172.21.15.40:0/1064946815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2038018070 con 0x5581c58351c0 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f203dd7a640 1 -- 172.21.15.40:0/1064946815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038008c50 con 0x5581c58351c0 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f2041a83f40 1 -- 172.21.15.40:0/1064946815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c58351c0 msgr2=0x5581c5865120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f2041a83f40 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c58351c0 0x5581c5865120 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f2038009790 tx=0x7f2038005a00 comp rx=0 tx=0).stop 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f2041a83f40 1 -- 172.21.15.40:0/1064946815 shutdown_connections 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f2041a83f40 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5907f70 0x5581c590ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f2041a83f40 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c5880420 0x5581c590e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f2041a83f40 1 --2- 172.21.15.40:0/1064946815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c58351c0 0x5581c5865120 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f2041a83f40 1 -- 172.21.15.40:0/1064946815 >> 172.21.15.40:0/1064946815 conn(0x5581c58608a0 msgr2=0x5581c5861510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.723+0000 7f2041a83f40 1 -- 172.21.15.40:0/1064946815 shutdown_connections 2024-04-26T21:46:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.724+0000 7f2041a83f40 1 -- 172.21.15.40:0/1064946815 wait complete. 2024-04-26T21:46:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.724+0000 7f2041a83f40 1 Processor -- start 2024-04-26T21:46:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.724+0000 7f2041a83f40 1 -- start start 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f2041a83f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c58351c0 0x5581c5909f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f2041a83f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5880420 0x5581c58fba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f2041a83f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c5907f70 0x5581c58fa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f2041a83f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5581c5861dc0 con 0x5581c5907f70 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f2041a83f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5581c5864c90 con 0x5581c5880420 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f2041a83f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5581c590d9d0 con 0x5581c58351c0 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c5907f70 0x5581c58fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c5907f70 0x5581c58fa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35976/0 (socket says 172.21.15.40:35976) 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 -- 172.21.15.40:0/3092731911 learned_addr learned my addr 172.21.15.40:0/3092731911 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203f57d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c58351c0 0x5581c5909f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203ed7c640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5880420 0x5581c58fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 -- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c58351c0 msgr2=0x5581c5909f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581c58351c0 0x5581c5909f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 -- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5880420 msgr2=0x5581c58fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581c5880420 0x5581c58fba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 -- 172.21.15.40:0/3092731911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c590a4c0 con 0x5581c5907f70 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203fd7e640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c5907f70 0x5581c58fa640 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f2030025ab0 tx=0x7f2030003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.725+0000 7f203cd78640 1 -- 172.21.15.40:0/3092731911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203006a070 con 0x5581c5907f70 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.726+0000 7f203cd78640 1 -- 172.21.15.40:0/3092731911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2030051540 con 0x5581c5907f70 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.726+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5581c587b0a0 con 0x5581c5907f70 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.726+0000 7f203cd78640 1 -- 172.21.15.40:0/3092731911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030055040 con 0x5581c5907f70 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.726+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c590aa70 con 0x5581c5907f70 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.727+0000 7f203cd78640 1 -- 172.21.15.40:0/3092731911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f203006d070 con 0x5581c5907f70 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.727+0000 7f203cd78640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f202006b300 0x7f202006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.727+0000 7f203cd78640 1 -- 172.21.15.40:0/3092731911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7f20300cf0a0 con 0x5581c5907f70 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.728+0000 7f203f57d640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f202006b300 0x7f202006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.728+0000 7f2041a83f40 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5581c58fab80 0x5581c5863fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.728+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5581c587bb80 con 0x5581c58fab80 2024-04-26T21:46:09.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.728+0000 7f203ed7c640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5581c58fab80 0x5581c5863fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.728+0000 7f203f57d640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f202006b300 0x7f202006d7c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2038000c00 tx=0x7f2038012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.728+0000 7f203ed7c640 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5581c58fab80 0x5581c5863fd0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f203ed7c640 1 -- 172.21.15.40:0/3092731911 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-header] v129'1064 uv1064 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2028011890 con 0x5581c58fab80 2024-04-26T21:46:09.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5581c58fab80 msgr2=0x5581c5863fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x5581c58fab80 0x5581c5863fd0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f202006b300 msgr2=0x7f202006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f202006b300 0x7f202006d7c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2038000c00 tx=0x7f2038012000 comp rx=0 tx=0).stop 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c5907f70 msgr2=0x5581c58fa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 --2- 172.21.15.40:0/3092731911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581c5907f70 0x5581c58fa640 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f2030025ab0 tx=0x7f2030003330 comp rx=0 tx=0).stop 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f203fd7e640 1 -- 172.21.15.40:0/3092731911 reap_dead start 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 shutdown_connections 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 >> 172.21.15.40:0/3092731911 conn(0x5581c58608a0 msgr2=0x5581c58614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 shutdown_connections 2024-04-26T21:46:09.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.730+0000 7f2041a83f40 1 -- 172.21.15.40:0/3092731911 wait complete. 2024-04-26T21:46:09.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:46:09.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.876+0000 7f262fcc5640 1 Processor -- start 2024-04-26T21:46:09.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.877+0000 7f262fcc5640 1 -- start start 2024-04-26T21:46:09.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.877+0000 7f262fcc5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815f550 0x7f26280d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.877+0000 7f262fcc5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280d1040 con 0x7f26280d3bc0 2024-04-26T21:46:09.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.877+0000 7f262fcc5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280c9680 con 0x7f262815f170 2024-04-26T21:46:09.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.877+0000 7f262fcc5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26280c9430 con 0x7f262815f550 2024-04-26T21:46:09.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262e262640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f26280d3bc0 0x7f26280ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53496/0 (socket says 172.21.15.40:53496) 2024-04-26T21:46:09.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262e262640 1 -- 172.21.15.40:0/1875588923 learned_addr learned my addr 172.21.15.40:0/1875588923 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:09.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262d260640 1 --2- 172.21.15.40:0/1875588923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815f550 0x7f26280d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262ca5f640 1 -- 172.21.15.40:0/1875588923 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159342394 0 0) 0x7f2618001490 con 0x7f26280d3bc0 2024-04-26T21:46:09.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262ca5f640 1 -- 172.21.15.40:0/1875588923 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2610003520 con 0x7f26280d3bc0 2024-04-26T21:46:09.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262ca5f640 1 -- 172.21.15.40:0/1875588923 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977093326 0 0) 0x7f26280c9680 con 0x7f262815f170 2024-04-26T21:46:09.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262ca5f640 1 -- 172.21.15.40:0/1875588923 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2618001490 con 0x7f262815f170 2024-04-26T21:46:09.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262ca5f640 1 -- 172.21.15.40:0/1875588923 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2307845224 0 0) 0x7f2610003520 con 0x7f26280d3bc0 2024-04-26T21:46:09.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262d260640 1 -- 172.21.15.40:0/1875588923 >> v1:172.21.15.184:6789/0 conn(0x7f262815f170 legacy=0x7f26280d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262d260640 1 -- 172.21.15.40:0/1875588923 >> v1:172.21.15.40:6789/0 conn(0x7f26280d3bc0 legacy=0x7f26280ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.878+0000 7f262d260640 1 -- 172.21.15.40:0/1875588923 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26280c9960 con 0x7f262815f550 2024-04-26T21:46:09.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.879+0000 7f262d260640 1 --2- 172.21.15.40:0/1875588923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815f550 0x7f26280d5a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f261c0097d0 tx=0x7f261c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbc92f4bfb5f01e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.879+0000 7f262ca5f640 1 -- 172.21.15.40:0/1875588923 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c037070 con 0x7f262815f550 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.879+0000 7f262ca5f640 1 -- 172.21.15.40:0/1875588923 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f261c02ac30 con 0x7f262815f550 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.879+0000 7f262ca5f640 1 -- 172.21.15.40:0/1875588923 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618001bc0 con 0x7f262815f550 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.880+0000 7f262fcc5640 1 -- 172.21.15.40:0/1875588923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815f550 msgr2=0x7f26280d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.880+0000 7f262fcc5640 1 --2- 172.21.15.40:0/1875588923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815f550 0x7f26280d5a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f261c0097d0 tx=0x7f261c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.880+0000 7f262fcc5640 1 -- 172.21.15.40:0/1875588923 shutdown_connections 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.880+0000 7f262fcc5640 1 --2- 172.21.15.40:0/1875588923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815f550 0x7f26280d5a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.880+0000 7f262fcc5640 1 -- 172.21.15.40:0/1875588923 >> 172.21.15.40:0/1875588923 conn(0x7f26280d4f60 msgr2=0x7f26280d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.880+0000 7f262fcc5640 1 -- 172.21.15.40:0/1875588923 shutdown_connections 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.880+0000 7f262fcc5640 1 -- 172.21.15.40:0/1875588923 wait complete. 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.881+0000 7f262fcc5640 1 Processor -- start 2024-04-26T21:46:09.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.881+0000 7f262fcc5640 1 -- start start 2024-04-26T21:46:09.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262fcc5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26280d3bc0 0x7f26281e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262fcc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f262815f170 0x7f262815c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262fcc5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815c860 0x7f262815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262fcc5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26280d2e30 con 0x7f262815f170 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262fcc5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26280d2cb0 con 0x7f26280d3bc0 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262fcc5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26280d2fb0 con 0x7f262815c860 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f262815f170 0x7f262815c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f262815f170 0x7f262815c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35980/0 (socket says 172.21.15.40:35980) 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 -- 172.21.15.40:0/3369046341 learned_addr learned my addr 172.21.15.40:0/3369046341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262e262640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815c860 0x7f262815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262da61640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26280d3bc0 0x7f26281e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 -- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815c860 msgr2=0x7f262815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815c860 0x7f262815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 -- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26280d3bc0 msgr2=0x7f26281e51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26280d3bc0 0x7f26281e51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 -- 172.21.15.40:0/3369046341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26280b94e0 con 0x7f262815f170 2024-04-26T21:46:09.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262d260640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f262815f170 0x7f262815c320 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f261c035040 tx=0x7f261c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f26267fc640 1 -- 172.21.15.40:0/3369046341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c041020 con 0x7f262815f170 2024-04-26T21:46:09.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f26267fc640 1 -- 172.21.15.40:0/3369046341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f261c03d640 con 0x7f262815f170 2024-04-26T21:46:09.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26281e58a0 con 0x7f262815f170 2024-04-26T21:46:09.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.882+0000 7f26267fc640 1 -- 172.21.15.40:0/3369046341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c037040 con 0x7f262815f170 2024-04-26T21:46:09.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.883+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281e5e60 con 0x7f262815f170 2024-04-26T21:46:09.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.883+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262815cc70 con 0x7f262815f170 2024-04-26T21:46:09.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.884+0000 7f26267fc640 1 -- 172.21.15.40:0/3369046341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f261c044020 con 0x7f262815f170 2024-04-26T21:46:09.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.884+0000 7f26267fc640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f261406c8f0 0x7f261406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:09.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.885+0000 7f26267fc640 1 -- 172.21.15.40:0/3369046341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7f261c0acf20 con 0x7f262815f170 2024-04-26T21:46:09.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.885+0000 7f262da61640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f261406c8f0 0x7f261406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:09.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.885+0000 7f262da61640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f261406c8f0 0x7f261406edb0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f26200011e0 tx=0x7f2620009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:09.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:09.889+0000 7f26267fc640 1 -- 172.21.15.40:0/3369046341 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f262815cc70 con 0x7f262815f170 2024-04-26T21:46:10.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.117+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f26280694f0 con 0x7f262815f170 2024-04-26T21:46:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.675+0000 7f26267fc640 1 -- 172.21.15.40:0/3369046341 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v307) ==== 162+0+0 (secure 0 0 0) 0x7f261c04b030 con 0x7f262815f170 2024-04-26T21:46:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.678+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f261406c8f0 msgr2=0x7f261406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.678+0000 7f262fcc5640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f261406c8f0 0x7f261406edb0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f26200011e0 tx=0x7f2620009040 comp rx=0 tx=0).stop 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.678+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f262815f170 msgr2=0x7f262815c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.678+0000 7f262fcc5640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f262815f170 0x7f262815c320 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f261c035040 tx=0x7f261c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.679+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 shutdown_connections 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.679+0000 7f262fcc5640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f261406c8f0 0x7f261406edb0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.679+0000 7f262fcc5640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262815c860 0x7f262815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.679+0000 7f262fcc5640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f262815f170 0x7f262815c320 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.679+0000 7f262fcc5640 1 --2- 172.21.15.40:0/3369046341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26280d3bc0 0x7f26281e51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.679+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 >> 172.21.15.40:0/3369046341 conn(0x7f26280d4f60 msgr2=0x7f262815e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.679+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 shutdown_connections 2024-04-26T21:46:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:10.679+0000 7f262fcc5640 1 -- 172.21.15.40:0/3369046341 wait complete. 2024-04-26T21:46:11.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.210+0000 7f78ff522640 1 Processor -- start 2024-04-26T21:46:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.210+0000 7f78ff522640 1 -- start start 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78ff522640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8162810 0x7f78f8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78ff522640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f80d10d0 con 0x7f78f80d1b30 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78ff522640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f80d1370 con 0x7f78f80b2340 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78ff522640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78f80d0750 con 0x7f78f8162810 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78fdabf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78f80d1b30 0x7f78f8070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53500/0 (socket says 172.21.15.40:53500) 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78fdabf640 1 -- 172.21.15.40:0/3013836976 learned_addr learned my addr 172.21.15.40:0/3013836976 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78fd2be640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8162810 0x7f78f8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329057290 0 0) 0x7f78f80d10d0 con 0x7f78f80d1b30 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78e0003520 con 0x7f78f80d1b30 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1891317232 0 0) 0x7f78e0003520 con 0x7f78f80d1b30 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8162810 msgr2=0x7f78f8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8162810 0x7f78f8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 >> v1:172.21.15.184:6789/0 conn(0x7f78f80b2340 legacy=0x7f78f80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f80d01b0 con 0x7f78f80d1b30 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f78e80019f0 con 0x7f78f80d1b30 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 >> v1:172.21.15.40:6789/0 conn(0x7f78f80d1b30 legacy=0x7f78f8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78e00039e0 0x7f78e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78e0004390 0x7f78e0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8162810 0x7f78e00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.211+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78f80d0750 con 0x7f78e0004390 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78e000b9c0 con 0x7f78e00039e0 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78e000bbc0 con 0x7f78f8162810 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fcabd640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78e0004390 0x7f78e0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fd2be640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78e00039e0 0x7f78e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fdabf640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8162810 0x7f78e00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fd2be640 1 -- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8162810 msgr2=0x7f78e00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fd2be640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f8162810 0x7f78e00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fd2be640 1 -- 172.21.15.40:0/3013836976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78e0004390 msgr2=0x7f78e0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fd2be640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78e0004390 0x7f78e0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fd2be640 1 -- 172.21.15.40:0/3013836976 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e0003520 con 0x7f78e00039e0 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.212+0000 7f78fcabd640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78e0004390 0x7f78e0004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78fd2be640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78e00039e0 0x7f78e0003dc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009af0 tx=0x7f78e0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87737c093fdccc15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0036070 con 0x7f78e00039e0 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f0031400 con 0x7f78e00039e0 2024-04-26T21:46:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78f7fff640 1 -- 172.21.15.40:0/3013836976 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0009690 con 0x7f78e00039e0 2024-04-26T21:46:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78ff522640 1 -- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78e00039e0 msgr2=0x7f78e0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78ff522640 1 --2- 172.21.15.40:0/3013836976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78e00039e0 0x7f78e0003dc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009af0 tx=0x7f78e0005830 comp rx=0 tx=0).stop 2024-04-26T21:46:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78fdabf640 1 -- 172.21.15.40:0/3013836976 reap_dead start 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78ff522640 1 -- 172.21.15.40:0/3013836976 shutdown_connections 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.213+0000 7f78ff522640 1 -- 172.21.15.40:0/3013836976 >> 172.21.15.40:0/3013836976 conn(0x7f78f80d2da0 msgr2=0x7f78f80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.214+0000 7f78ff522640 1 -- 172.21.15.40:0/3013836976 shutdown_connections 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.214+0000 7f78ff522640 1 -- 172.21.15.40:0/3013836976 wait complete. 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.215+0000 7f78ff522640 1 Processor -- start 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.215+0000 7f78ff522640 1 -- start start 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.215+0000 7f78ff522640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78e00039e0 0x7f78f81521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.215+0000 7f78ff522640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f81603e0 0x7f78f80c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.215+0000 7f78ff522640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f815f4f0 0x7f78f80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78ff522640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78f80cdb00 con 0x7f78f815f4f0 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78ff522640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78f80d16a0 con 0x7f78f81603e0 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78ff522640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78f8071260 con 0x7f78e00039e0 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f815f4f0 0x7f78f80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f815f4f0 0x7f78f80c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36000/0 (socket says 172.21.15.40:36000) 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 -- 172.21.15.40:0/1584266123 learned_addr learned my addr 172.21.15.40:0/1584266123 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fd2be640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78e00039e0 0x7f78f81521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fcabd640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f81603e0 0x7f78f80c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 -- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78e00039e0 msgr2=0x7f78f81521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78e00039e0 0x7f78f81521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 -- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f81603e0 msgr2=0x7f78f80c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f81603e0 0x7f78f80c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 -- 172.21.15.40:0/1584266123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f8160910 con 0x7f78f815f4f0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78fdabf640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f815f4f0 0x7f78f80c8a20 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f78f80cf3c0 tx=0x7f78e800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78f5ffb640 1 -- 172.21.15.40:0/1584266123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec001430 con 0x7f78f815f4f0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78f5ffb640 1 -- 172.21.15.40:0/1584266123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e800ec40 con 0x7f78f815f4f0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f8160bf0 con 0x7f78f815f4f0 2024-04-26T21:46:11.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.216+0000 7f78f5ffb640 1 -- 172.21.15.40:0/1584266123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8014430 con 0x7f78f815f4f0 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.217+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f81611b0 con 0x7f78f815f4f0 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.218+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f80694f0 con 0x7f78f815f4f0 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.218+0000 7f78f5ffb640 1 -- 172.21.15.40:0/1584266123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78e80145d0 con 0x7f78f815f4f0 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.218+0000 7f78f5ffb640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e406c8f0 0x7f78e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.219+0000 7f78f5ffb640 1 -- 172.21.15.40:0/1584266123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7f78e808c4f0 con 0x7f78f815f4f0 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.219+0000 7f78fd2be640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e406c8f0 0x7f78e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:11.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.219+0000 7f78fd2be640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e406c8f0 0x7f78e406edb0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f78f0033000 tx=0x7f78f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:11.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.223+0000 7f78f5ffb640 1 -- 172.21.15.40:0/1584266123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78e80594b0 con 0x7f78f815f4f0 2024-04-26T21:46:11.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.447+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78f80b1ce0 con 0x7f78f815f4f0 2024-04-26T21:46:11.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.447+0000 7f78f5ffb640 1 -- 172.21.15.40:0/1584266123 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 307 v307) ==== 95+0+5190 (secure 0 0 0) 0x7f78e805d7b0 con 0x7f78f815f4f0 2024-04-26T21:46:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:11.447 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":307,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":307,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:10.674020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11079":{"gid":11079,"name":"c","rank":0,"incarnation":307,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/771026484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":771026484},{"type":"v1","addr":"172.21.15.40:6833","nonce":771026484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:11.448 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 307 2024-04-26T21:46:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e406c8f0 msgr2=0x7f78e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e406c8f0 0x7f78e406edb0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f78f0033000 tx=0x7f78f0008030 comp rx=0 tx=0).stop 2024-04-26T21:46:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f815f4f0 msgr2=0x7f78f80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f815f4f0 0x7f78f80c8a20 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f78f80cf3c0 tx=0x7f78e800e530 comp rx=0 tx=0).stop 2024-04-26T21:46:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 shutdown_connections 2024-04-26T21:46:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f78e406c8f0 0x7f78e406edb0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78f815f4f0 0x7f78f80c8a20 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f81603e0 0x7f78f80c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 --2- 172.21.15.40:0/1584266123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78e00039e0 0x7f78f81521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 >> 172.21.15.40:0/1584266123 conn(0x7f78f80d2da0 msgr2=0x7f78f80d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 shutdown_connections 2024-04-26T21:46:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:11.449+0000 7f78ff522640 1 -- 172.21.15.40:0/1584266123 wait complete. 2024-04-26T21:46:11.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 307, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:46:10.674020+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11079}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11079': {'gid': 11079, 'name': 'c', 'rank': 0, 'incarnation': 307, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/771026484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 771026484}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 771026484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:46:11.460 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11079:up:replay 2024-04-26T21:46:11.460 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:46:12.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:46:12.750 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:46:14.379 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:14.380+0000 7f1f51249640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:46:16.863 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:16.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.977+0000 7f5146693640 1 Processor -- start 2024-04-26T21:46:16.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.977+0000 7f5146693640 1 -- start start 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.978+0000 7f5146693640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5140071e60 0x7f514006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.978+0000 7f5146693640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51400cc480 0x7f5140166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.978+0000 7f5146693640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f514015d7c0 con 0x7f514006fe20 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.978+0000 7f5146693640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f514015daa0 con 0x7f51400cc480 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.978+0000 7f5146693640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f514015dd80 con 0x7f5140071e60 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.979+0000 7f5144c30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51400cc480 0x7f5140166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.979+0000 7f5144c30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51400cc480 0x7f5140166ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36010/0 (socket says 172.21.15.40:36010) 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.979+0000 7f5144c30640 1 -- 172.21.15.40:0/2568394532 learned_addr learned my addr 172.21.15.40:0/2568394532 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.979+0000 7f513ffff640 1 --2- 172.21.15.40:0/2568394532 >> v2:172.21.15.184:3300/0 conn(0x7f5140071e60 0x7f514006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.979+0000 7f5144c30640 1 -- 172.21.15.40:0/2568394532 >> v2:172.21.15.184:3300/0 conn(0x7f5140071e60 msgr2=0x7f514006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.979+0000 7f5144c30640 1 --2- 172.21.15.40:0/2568394532 >> v2:172.21.15.184:3300/0 conn(0x7f5140071e60 0x7f514006f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.979+0000 7f5144c30640 1 -- 172.21.15.40:0/2568394532 >> v1:172.21.15.184:6789/0 conn(0x7f514006fe20 legacy=0x7f5140072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.979+0000 7f5144c30640 1 -- 172.21.15.40:0/2568394532 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514015e060 con 0x7f51400cc480 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f5144c30640 1 --2- 172.21.15.40:0/2568394532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51400cc480 0x7f5140166ad0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f51400b3700 tx=0x7f513802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a33478c70718b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f513effd640 1 -- 172.21.15.40:0/2568394532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138036070 con 0x7f51400cc480 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f513effd640 1 -- 172.21.15.40:0/2568394532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5138031600 con 0x7f51400cc480 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f513effd640 1 -- 172.21.15.40:0/2568394532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380319d0 con 0x7f51400cc480 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f5146693640 1 -- 172.21.15.40:0/2568394532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51400cc480 msgr2=0x7f5140166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f5146693640 1 --2- 172.21.15.40:0/2568394532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51400cc480 0x7f5140166ad0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f51400b3700 tx=0x7f513802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f5146693640 1 -- 172.21.15.40:0/2568394532 shutdown_connections 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f5146693640 1 --2- 172.21.15.40:0/2568394532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51400cc480 0x7f5140166ad0 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f5146693640 1 --2- 172.21.15.40:0/2568394532 >> v2:172.21.15.184:3300/0 conn(0x7f5140071e60 0x7f514006f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f5146693640 1 -- 172.21.15.40:0/2568394532 >> 172.21.15.40:0/2568394532 conn(0x7f51400d0be0 msgr2=0x7f51400d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.980+0000 7f5146693640 1 -- 172.21.15.40:0/2568394532 shutdown_connections 2024-04-26T21:46:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.981+0000 7f5146693640 1 -- 172.21.15.40:0/2568394532 wait complete. 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.982+0000 7f5146693640 1 Processor -- start 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.982+0000 7f5146693640 1 -- start start 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5146693640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f514006fe20 0x7f514014b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5146693640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5140071e60 0x7f51401f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5146693640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51400cc480 0x7f51400ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5146693640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f514015d570 con 0x7f514006fe20 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5146693640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f514015d3f0 con 0x7f5140071e60 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5146693640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5140072950 con 0x7f51400cc480 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f514006fe20 0x7f514014b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f514006fe20 0x7f514014b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36022/0 (socket says 172.21.15.40:36022) 2024-04-26T21:46:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 -- 172.21.15.40:0/3455313254 learned_addr learned my addr 172.21.15.40:0/3455313254 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5144c30640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51400cc480 0x7f51400ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513f7fe640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5140071e60 0x7f51401f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 -- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51400cc480 msgr2=0x7f51400ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51400cc480 0x7f51400ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 -- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5140071e60 msgr2=0x7f51401f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5140071e60 0x7f51401f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 -- 172.21.15.40:0/3455313254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51400b4d60 con 0x7f514006fe20 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5144c30640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51400cc480 0x7f51400ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513f7fe640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5140071e60 0x7f51401f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513ffff640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f514006fe20 0x7f514014b120 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f513000d800 tx=0x7f513000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513cff9640 1 -- 172.21.15.40:0/3455313254 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513000b6c0 con 0x7f514006fe20 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513cff9640 1 -- 172.21.15.40:0/3455313254 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5130019070 con 0x7f514006fe20 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f513cff9640 1 -- 172.21.15.40:0/3455313254 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513000ba80 con 0x7f514006fe20 2024-04-26T21:46:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.983+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51400b5040 con 0x7f514006fe20 2024-04-26T21:46:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.984+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51400b5600 con 0x7f514006fe20 2024-04-26T21:46:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.985+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51400694f0 con 0x7f514006fe20 2024-04-26T21:46:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.985+0000 7f513cff9640 1 -- 172.21.15.40:0/3455313254 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f513000bc20 con 0x7f514006fe20 2024-04-26T21:46:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.985+0000 7f513cff9640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512006c8f0 0x7f512006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.986+0000 7f513cff9640 1 -- 172.21.15.40:0/3455313254 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7f513008ce00 con 0x7f514006fe20 2024-04-26T21:46:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.986+0000 7f513f7fe640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512006c8f0 0x7f512006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.986+0000 7f513f7fe640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512006c8f0 0x7f512006edb0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f514015ad70 tx=0x7f5128009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:16.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:16.990+0000 7f513cff9640 1 -- 172.21.15.40:0/3455313254 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f513001c360 con 0x7f514006fe20 2024-04-26T21:46:17.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.216+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51401585f0 con 0x7f514006fe20 2024-04-26T21:46:17.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.217+0000 7f513cff9640 1 -- 172.21.15.40:0/3455313254 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5199 (secure 0 0 0) 0x7f513005a470 con 0x7f514006fe20 2024-04-26T21:46:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:17.217 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":310,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:13.679543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11079":{"gid":11079,"name":"c","rank":0,"incarnation":307,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6833/771026484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":771026484},{"type":"v1","addr":"172.21.15.40:6833","nonce":771026484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11079,"qdb_cluster":[11079]},"id":6}]} 2024-04-26T21:46:17.217 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 310 2024-04-26T21:46:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512006c8f0 msgr2=0x7f512006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512006c8f0 0x7f512006edb0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f514015ad70 tx=0x7f5128009040 comp rx=0 tx=0).stop 2024-04-26T21:46:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f514006fe20 msgr2=0x7f514014b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f514006fe20 0x7f514014b120 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f513000d800 tx=0x7f513000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 shutdown_connections 2024-04-26T21:46:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f512006c8f0 0x7f512006edb0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51400cc480 0x7f51400ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5140071e60 0x7f51401f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 --2- 172.21.15.40:0/3455313254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f514006fe20 0x7f514014b120 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 >> 172.21.15.40:0/3455313254 conn(0x7f51400d0be0 msgr2=0x7f5140072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 shutdown_connections 2024-04-26T21:46:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.219+0000 7f5146693640 1 -- 172.21.15.40:0/3455313254 wait complete. 2024-04-26T21:46:17.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 310, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:46:13.679543+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11079}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11079': {'gid': 11079, 'name': 'c', 'rank': 0, 'incarnation': 307, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6833/771026484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 771026484}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 771026484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11079, 'qdb_cluster': [11079]} 2024-04-26T21:46:17.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:46:17.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:46:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.334+0000 7fd008abd640 1 Processor -- start 2024-04-26T21:46:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.334+0000 7fd008abd640 1 -- start start 2024-04-26T21:46:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fd008abd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0040d1380 0x7fd0040d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fd008abd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0040d1ce0 con 0x7fd004070ae0 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fd008abd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0040ce840 con 0x7fd00415d1e0 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fd008abd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0040cede0 con 0x7fd0040d1380 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fcffbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0040d1380 0x7fd0040d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fd002d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd004070ae0 0x7fd004070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53504/0 (socket says 172.21.15.40:53504) 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fd002d9d640 1 -- 172.21.15.40:0/2939977370 learned_addr learned my addr 172.21.15.40:0/2939977370 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fcffbfff640 1 -- 172.21.15.40:0/2939977370 >> v1:172.21.15.184:6789/0 conn(0x7fd00415d1e0 legacy=0x7fd0040d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fcffbfff640 1 -- 172.21.15.40:0/2939977370 >> v1:172.21.15.40:6789/0 conn(0x7fd004070ae0 legacy=0x7fd004070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fcffbfff640 1 -- 172.21.15.40:0/2939977370 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0040ceb00 con 0x7fd0040d1380 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.335+0000 7fd001d9b640 1 -- 172.21.15.40:0/2939977370 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349885084 0 0) 0x7fd0040d1ce0 con 0x7fd004070ae0 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.336+0000 7fcffbfff640 1 --2- 172.21.15.40:0/2939977370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0040d1380 0x7fd0040d17a0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0097a0 tx=0x7fcfec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74ee2f03af228c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.336+0000 7fd001d9b640 1 -- 172.21.15.40:0/2939977370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec037070 con 0x7fd0040d1380 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.336+0000 7fd001d9b640 1 -- 172.21.15.40:0/2939977370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec032440 con 0x7fd0040d1380 2024-04-26T21:46:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.336+0000 7fd001d9b640 1 -- 172.21.15.40:0/2939977370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec032810 con 0x7fd0040d1380 2024-04-26T21:46:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.336+0000 7fd008abd640 1 -- 172.21.15.40:0/2939977370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0040d1380 msgr2=0x7fd0040d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.336+0000 7fd008abd640 1 --2- 172.21.15.40:0/2939977370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0040d1380 0x7fd0040d17a0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0097a0 tx=0x7fcfec02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.337+0000 7fd008abd640 1 -- 172.21.15.40:0/2939977370 shutdown_connections 2024-04-26T21:46:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.337+0000 7fd008abd640 1 --2- 172.21.15.40:0/2939977370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0040d1380 0x7fd0040d17a0 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.337+0000 7fd008abd640 1 -- 172.21.15.40:0/2939977370 >> 172.21.15.40:0/2939977370 conn(0x7fd0040d2da0 msgr2=0x7fd0040d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.337+0000 7fd008abd640 1 -- 172.21.15.40:0/2939977370 shutdown_connections 2024-04-26T21:46:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.337+0000 7fd008abd640 1 -- 172.21.15.40:0/2939977370 wait complete. 2024-04-26T21:46:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.338+0000 7fd008abd640 1 Processor -- start 2024-04-26T21:46:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.338+0000 7fd008abd640 1 -- start start 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd008abd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd004070ae0 0x7fd004150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd008abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040d1380 0x7fd00415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd008abd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415d1e0 0x7fd00415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd008abd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0040d10f0 con 0x7fd00415d1e0 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd008abd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd004070650 con 0x7fd004070ae0 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd008abd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0040ce4c0 con 0x7fd0040d1380 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415d1e0 0x7fd00415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415d1e0 0x7fd00415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36038/0 (socket says 172.21.15.40:36038) 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 -- 172.21.15.40:0/701114554 learned_addr learned my addr 172.21.15.40:0/701114554 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:17.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fcffbfff640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040d1380 0x7fd00415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd00259c640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd004070ae0 0x7fd004150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 -- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040d1380 msgr2=0x7fd00415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040d1380 0x7fd00415e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 -- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd004070ae0 msgr2=0x7fd004150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd004070ae0 0x7fd004150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 -- 172.21.15.40:0/701114554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0040cb940 con 0x7fd00415d1e0 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd00259c640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd004070ae0 0x7fd004150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fd002d9d640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415d1e0 0x7fd00415ebe0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c7d0 tx=0x7fcffc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.339+0000 7fcffaffd640 1 -- 172.21.15.40:0/701114554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc018070 con 0x7fd00415d1e0 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.340+0000 7fcffaffd640 1 -- 172.21.15.40:0/701114554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc00a640 con 0x7fd00415d1e0 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.340+0000 7fcffaffd640 1 -- 172.21.15.40:0/701114554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc00aa70 con 0x7fd00415d1e0 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.340+0000 7fd008abd640 1 -- 172.21.15.40:0/701114554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0040cbc20 con 0x7fd00415d1e0 2024-04-26T21:46:17.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.340+0000 7fd008abd640 1 -- 172.21.15.40:0/701114554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0040cc1e0 con 0x7fd00415d1e0 2024-04-26T21:46:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.341+0000 7fcffaffd640 1 -- 172.21.15.40:0/701114554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcffc01b070 con 0x7fd00415d1e0 2024-04-26T21:46:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.342+0000 7fcffaffd640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfe406c8f0 0x7fcfe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.342+0000 7fcffaffd640 1 -- 172.21.15.40:0/701114554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7fcffc08be30 con 0x7fd00415d1e0 2024-04-26T21:46:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.342+0000 7fd00259c640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfe406c8f0 0x7fcfe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.342+0000 7fd00259c640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfe406c8f0 0x7fcfe406edb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcff400a810 tx=0x7fcff4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.343+0000 7fd00b7cf740 1 Processor -- start 2024-04-26T21:46:17.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.343+0000 7fd00b7cf740 1 -- start start 2024-04-26T21:46:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.343+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae639f0 0x564b5ae3c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.343+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae66250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.343+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae39ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.343+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564b5ada0740 con 0x564b5ae63dd0 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.343+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b5ad3c3a0 con 0x564b5ae639f0 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.343+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564b5ad9c700 con 0x564b5ae66790 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae66250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae66250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36052/0 (socket says 172.21.15.40:36052) 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 -- 172.21.15.40:0/1716016057 learned_addr learned my addr 172.21.15.40:0/1716016057 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fd00259c640 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae639f0 0x564b5ae3c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fd002d9d640 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae39ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 -- 172.21.15.40:0/1716016057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 msgr2=0x564b5ae39ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae39ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 -- 172.21.15.40:0/1716016057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae639f0 msgr2=0x564b5ae3c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae639f0 0x564b5ae3c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 -- 172.21.15.40:0/1716016057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b5ad58750 con 0x564b5ae63dd0 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcffbfff640 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae66250 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0098c0 tx=0x7fcfec02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e236bc8d6a01d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcfe3fff640 1 -- 172.21.15.40:0/1716016057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec037040 con 0x564b5ae63dd0 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcfe3fff640 1 -- 172.21.15.40:0/1716016057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec032740 con 0x564b5ae63dd0 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.344+0000 7fcfe3fff640 1 -- 172.21.15.40:0/1716016057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec032af0 con 0x564b5ae63dd0 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1716016057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 msgr2=0x564b5ae66250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae66250 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0098c0 tx=0x7fcfec02c000 comp rx=0 tx=0).stop 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1716016057 shutdown_connections 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae39ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae66250 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1716016057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae639f0 0x564b5ae3c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1716016057 >> 172.21.15.40:0/1716016057 conn(0x564b5ae38a50 msgr2=0x564b5ae81250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1716016057 shutdown_connections 2024-04-26T21:46:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1716016057 wait complete. 2024-04-26T21:46:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.345+0000 7fd00b7cf740 1 -- start start 2024-04-26T21:46:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.346+0000 7fd00b7cf740 1 Processor -- start 2024-04-26T21:46:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.346+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae7ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.346+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.346+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae80df0 0x564b5ae78400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.346+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564b5ad27fe0 con 0x564b5ae63dd0 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.346+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b5ae7ef50 con 0x564b5ae80df0 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.346+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564b5ae81420 con 0x564b5ae66790 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00b7cf740 10 client.0 _ll_register_callbacks cb 0x564b5ad3e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae7ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae7ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36056/0 (socket says 172.21.15.40:36056) 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 -- 172.21.15.40:0/804274590 learned_addr learned my addr 172.21.15.40:0/804274590 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd002d9d640 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae80df0 0x564b5ae78400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fcffbfff640 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 -- 172.21.15.40:0/804274590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 msgr2=0x564b5ae808b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae808b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 -- 172.21.15.40:0/804274590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae80df0 msgr2=0x564b5ae78400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae80df0 0x564b5ae78400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 -- 172.21.15.40:0/804274590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5aab22d0 con 0x564b5ae63dd0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00259c640 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae7ddb0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fcff400cf80 tx=0x7fcff40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7238f1ae2859bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fcfe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fcfe17fa640 1 -- 172.21.15.40:0/804274590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4023020 con 0x564b5ae63dd0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fcfe17fa640 1 -- 172.21.15.40:0/804274590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7fcff4013610 con 0x564b5ae63dd0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00b7cf740 1 -- 172.21.15.40:0/804274590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfec032740 con 0x564b5ae63dd0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fcfe17fa640 1 -- 172.21.15.40:0/804274590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=310) ==== 24+0+0 (secure 0 0 0) 0x7fcff402e050 con 0x564b5ae63dd0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00b7cf740 10 client.11136 fetch_fsmap learned FSMap version 310 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.347+0000 7fd00b7cf740 1 -- 172.21.15.40:0/804274590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=310,monmap=2+}) -- 0x564b5ab7c6b0 con 0x564b5ae63dd0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fcfe17fa640 1 -- 172.21.15.40:0/804274590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec032740 con 0x564b5ae63dd0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fcfe17fa640 1 -- 172.21.15.40:0/804274590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 310) ==== 3494+0+0 (secure 0 0 0) 0x7fcff4013810 con 0x564b5ae63dd0 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fd00b7cf740 10 client.11136 fetch_fsmap finished waiting for FSMap version 310 2024-04-26T21:46:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fd00b7cf740 10 client.11136 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fd00b7cf740 20 client.11136 populate_metadata read hostname 'smithi040' 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] conn(0x564b5ae50a00 0x564b5ae4f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fd00b7cf740 4 client.11136 mds_command: new command op to 11079 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fd00b7cf740 1 -- 172.21.15.40:0/804274590 --> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b5adae350 con 0x564b5ae50a00 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fcffbfff640 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] conn(0x564b5ae50a00 0x564b5ae4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fcfe17fa640 10 client.11136 ms_handle_connect on v2:172.21.15.40:6832/771026484 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.348+0000 7fcffbfff640 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] conn(0x564b5ae50a00 0x564b5ae4f860 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5d90373dda3172 server_cookie=ddbbd3a4b59c10a1 in_seq=0 out_seq=0 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fcfe17fa640 1 -- 172.21.15.40:0/804274590 <== mds.0 v2:172.21.15.40:6832/771026484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b5adae350 con 0x564b5ae50a00 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fcfe17fa640 10 client.11136 handle_command_reply: tid=0 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 client.11136 shutdown 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 -- 172.21.15.40:0/804274590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 msgr2=0x564b5ae7ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae7ddb0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fcff400cf80 tx=0x7fcff40046b0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 -- 172.21.15.40:0/804274590 shutdown_connections 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] conn(0x564b5ae50a00 0x564b5ae4f860 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae80df0 0x564b5ae78400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae808b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fcfe17fa640 0 client.11136 ms_handle_reset on v2:172.21.15.40:6832/771026484 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/804274590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae63dd0 0x564b5ae7ddb0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 -- 172.21.15.40:0/804274590 >> 172.21.15.40:0/804274590 conn(0x564b5ae639f0 msgr2=0x564b5ae39ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 -- 172.21.15.40:0/804274590 shutdown_connections 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 1 -- 172.21.15.40:0/804274590 wait complete. 2024-04-26T21:46:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.349+0000 7fd00b7cf740 20 client.11136 trim_cache size 0 max 16384 2024-04-26T21:46:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.368+0000 7fd00b7cf740 1 Processor -- start 2024-04-26T21:46:17.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.368+0000 7fd00b7cf740 1 -- start start 2024-04-26T21:46:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae3cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae66790 0x564b5ae4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae80df0 0x564b5ae4fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0041ff780 con 0x564b5ae66790 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0041fe790 con 0x564b5ae63dd0 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd00406e960 con 0x564b5ae80df0 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae66790 0x564b5ae4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae66790 0x564b5ae4f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36068/0 (socket says 172.21.15.40:36068) 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 -- 172.21.15.40:0/1953738437 learned_addr learned my addr 172.21.15.40:0/1953738437 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fd00259c640 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae3cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 -- 172.21.15.40:0/1953738437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae80df0 msgr2=0x564b5ae4fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae80df0 0x564b5ae4fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 -- 172.21.15.40:0/1953738437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 msgr2=0x564b5ae3cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae3cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 -- 172.21.15.40:0/1953738437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b5ae7e450 con 0x564b5ae66790 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fd00259c640 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae3cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcffbfff640 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae66790 0x564b5ae4f830 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fcfec02a7d0 tx=0x7fcfec008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843d883fb684340b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcfe37fe640 1 -- 172.21.15.40:0/1953738437 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec037040 con 0x564b5ae66790 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcfe37fe640 1 -- 172.21.15.40:0/1953738437 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec004480 con 0x564b5ae66790 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.369+0000 7fcfe37fe640 1 -- 172.21.15.40:0/1953738437 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec051df0 con 0x564b5ae66790 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1953738437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae66790 msgr2=0x564b5ae4f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae66790 0x564b5ae4f830 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fcfec02a7d0 tx=0x7fcfec008030 comp rx=0 tx=0).stop 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1953738437 shutdown_connections 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae80df0 0x564b5ae4fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae66790 0x564b5ae4f830 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1953738437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae3cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1953738437 >> 172.21.15.40:0/1953738437 conn(0x564b5ae639f0 msgr2=0x564b5ae31cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1953738437 shutdown_connections 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.370+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1953738437 wait complete. 2024-04-26T21:46:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.371+0000 7fd00b7cf740 1 -- start start 2024-04-26T21:46:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.371+0000 7fd00b7cf740 1 Processor -- start 2024-04-26T21:46:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae4f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae4fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae80df0 0x564b5af03840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564b5ae50880 con 0x564b5ae80df0 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd00406efb0 con 0x564b5ae63dd0 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564b5ae50ea0 con 0x564b5ae66790 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 10 client.0 _ll_register_callbacks cb 0x7fcff400af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae80df0 0x564b5af03840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae80df0 0x564b5af03840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36076/0 (socket says 172.21.15.40:36076) 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 -- 172.21.15.40:0/1535345284 learned_addr learned my addr 172.21.15.40:0/1535345284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00259c640 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae4f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fcffbfff640 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae4fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 -- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 msgr2=0x564b5ae4fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae4fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 -- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 msgr2=0x564b5ae4f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae4f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 -- 172.21.15.40:0/1535345284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5ae3c9f0 con 0x564b5ae80df0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00259c640 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae4f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd002d9d640 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae80df0 0x564b5af03840 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fcffc058b30 tx=0x7fcffc058f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc595b04aba0c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fcfe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fcfe17fa640 1 -- 172.21.15.40:0/1535345284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc087060 con 0x564b5ae80df0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fcfe17fa640 1 -- 172.21.15.40:0/1535345284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7fcffc061580 con 0x564b5ae80df0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1535345284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfec004480 con 0x564b5ae80df0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fcfe17fa640 1 -- 172.21.15.40:0/1535345284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=310) ==== 24+0+0 (secure 0 0 0) 0x7fcffc05c040 con 0x564b5ae80df0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 10 client.11142 fetch_fsmap learned FSMap version 310 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.372+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1535345284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=310,monmap=2+}) -- 0x564b5ae3a360 con 0x564b5ae80df0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fcfe17fa640 1 -- 172.21.15.40:0/1535345284 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec004480 con 0x564b5ae80df0 2024-04-26T21:46:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fcfe17fa640 1 -- 172.21.15.40:0/1535345284 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 310) ==== 3494+0+0 (secure 0 0 0) 0x7fcffc078240 con 0x564b5ae80df0 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00b7cf740 10 client.11142 fetch_fsmap finished waiting for FSMap version 310 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00b7cf740 10 client.11142 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00b7cf740 20 client.11142 populate_metadata read hostname 'smithi040' 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] conn(0x564b5af04170 0x564b5ae35670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00b7cf740 4 client.11142 mds_command: new command op to 11079 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1535345284 --> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] -- command(tid 0: {"prefix": "status"}) -- 0x564b5ae81cc0 con 0x564b5af04170 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00259c640 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] conn(0x564b5af04170 0x564b5ae35670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fcfe17fa640 10 client.11142 ms_handle_connect on v2:172.21.15.40:6832/771026484 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00259c640 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] conn(0x564b5af04170 0x564b5ae35670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73bd34d0c6b9f661 server_cookie=eea3521d8e3e9bf2 in_seq=0 out_seq=0 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fcfe17fa640 1 -- 172.21.15.40:0/1535345284 <== mds.0 v2:172.21.15.40:6832/771026484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x564b5ae81cc0 con 0x564b5af04170 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fcfe17fa640 10 client.11142 handle_command_reply: tid=0 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.373+0000 7fd00b7cf740 1 client.11142 shutdown 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1535345284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae80df0 msgr2=0x564b5af03840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae80df0 0x564b5af03840 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fcffc058b30 tx=0x7fcffc058f40 comp rx=0 tx=0).stop 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1535345284 shutdown_connections 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.40:6832/771026484,v1:172.21.15.40:6833/771026484] conn(0x564b5af04170 0x564b5ae35670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b5ae80df0 0x564b5af03840 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b5ae66790 0x564b5ae4fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 --2- 172.21.15.40:0/1535345284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b5ae63dd0 0x564b5ae4f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fcfe17fa640 0 client.11142 ms_handle_reset on v2:172.21.15.40:6832/771026484 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1535345284 >> 172.21.15.40:0/1535345284 conn(0x564b5ae639f0 msgr2=0x564b5ae4e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1535345284 shutdown_connections 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 1 -- 172.21.15.40:0/1535345284 wait complete. 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd00b7cf740 20 client.11142 trim_cache size 0 max 16384 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:46:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 11079, 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.6977548340000004, 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 310, 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 129, 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 129, 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 8.9703910180000008 2024-04-26T21:46:17.381 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:46:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 -- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfe406c8f0 msgr2=0x7fcfe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfe406c8f0 0x7fcfe406edb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcff400a810 tx=0x7fcff4008040 comp rx=0 tx=0).stop 2024-04-26T21:46:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 -- 172.21.15.40:0/701114554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415d1e0 msgr2=0x7fd00415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415d1e0 0x7fd00415ebe0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c7d0 tx=0x7fcffc00cc90 comp rx=0 tx=0).stop 2024-04-26T21:46:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 -- 172.21.15.40:0/701114554 shutdown_connections 2024-04-26T21:46:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcfe406c8f0 0x7fcfe406edb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415d1e0 0x7fd00415ebe0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0040d1380 0x7fd00415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 --2- 172.21.15.40:0/701114554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd004070ae0 0x7fd004150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 -- 172.21.15.40:0/701114554 >> 172.21.15.40:0/701114554 conn(0x7fd0040d2da0 msgr2=0x7fd004070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 -- 172.21.15.40:0/701114554 shutdown_connections 2024-04-26T21:46:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.374+0000 7fd008abd640 1 -- 172.21.15.40:0/701114554 wait complete. 2024-04-26T21:46:17.385 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:46:17.385 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on 10000000000.00000000', proceeding to ls 2024-04-26T21:46:17.385 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:46:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.486+0000 7fe21f4d1640 1 Processor -- start 2024-04-26T21:46:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.486+0000 7fe21f4d1640 1 -- start start 2024-04-26T21:46:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21f4d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2181608f0 0x7fe2180d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21f4d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180d0a90 0x7fe2180cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21f4d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2180cde20 con 0x7fe2180cfdd0 2024-04-26T21:46:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21f4d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe2180cea30 con 0x7fe2181608f0 2024-04-26T21:46:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21f4d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2180ce100 con 0x7fe2180d0a90 2024-04-26T21:46:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21d26d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2181608f0 0x7fe2180d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21d26d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2181608f0 0x7fe2180d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36084/0 (socket says 172.21.15.40:36084) 2024-04-26T21:46:17.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21d26d640 1 -- 172.21.15.40:0/2638904113 learned_addr learned my addr 172.21.15.40:0/2638904113 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21ca6c640 1 --2- 172.21.15.40:0/2638904113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180d0a90 0x7fe2180cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21d26d640 1 -- 172.21.15.40:0/2638904113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180d0a90 msgr2=0x7fe2180cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21d26d640 1 --2- 172.21.15.40:0/2638904113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180d0a90 0x7fe2180cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21d26d640 1 -- 172.21.15.40:0/2638904113 >> v1:172.21.15.184:6789/0 conn(0x7fe2180cfdd0 legacy=0x7fe218160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.487+0000 7fe21d26d640 1 -- 172.21.15.40:0/2638904113 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2180ce3e0 con 0x7fe2181608f0 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.488+0000 7fe21d26d640 1 --2- 172.21.15.40:0/2638904113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2181608f0 0x7fe2180d1bf0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fe208009870 tx=0x7fe20802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dbe06d4b67a1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.488+0000 7fe20ffff640 1 -- 172.21.15.40:0/2638904113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208037070 con 0x7fe2181608f0 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.488+0000 7fe20ffff640 1 -- 172.21.15.40:0/2638904113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe208032440 con 0x7fe2181608f0 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.488+0000 7fe20ffff640 1 -- 172.21.15.40:0/2638904113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208032810 con 0x7fe2181608f0 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.488+0000 7fe21f4d1640 1 -- 172.21.15.40:0/2638904113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2181608f0 msgr2=0x7fe2180d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.488+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/2638904113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2181608f0 0x7fe2180d1bf0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fe208009870 tx=0x7fe20802a210 comp rx=0 tx=0).stop 2024-04-26T21:46:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.489+0000 7fe21f4d1640 1 -- 172.21.15.40:0/2638904113 shutdown_connections 2024-04-26T21:46:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.489+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/2638904113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180d0a90 0x7fe2180cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.489+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/2638904113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2181608f0 0x7fe2180d1bf0 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.489+0000 7fe21f4d1640 1 -- 172.21.15.40:0/2638904113 >> 172.21.15.40:0/2638904113 conn(0x7fe2180d2ec0 msgr2=0x7fe2180d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.489+0000 7fe21f4d1640 1 -- 172.21.15.40:0/2638904113 shutdown_connections 2024-04-26T21:46:17.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.489+0000 7fe21f4d1640 1 -- 172.21.15.40:0/2638904113 wait complete. 2024-04-26T21:46:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.490+0000 7fe21f4d1640 1 Processor -- start 2024-04-26T21:46:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.490+0000 7fe21f4d1640 1 -- start start 2024-04-26T21:46:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.490+0000 7fe21f4d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180cfdd0 0x7fe2180c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.490+0000 7fe21f4d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2180d0a90 0x7fe2180c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21f4d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2181608f0 0x7fe2181f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21f4d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe2180b3310 con 0x7fe2180d0a90 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21f4d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2180d0ef0 con 0x7fe2181608f0 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21f4d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2180ceec0 con 0x7fe2180cfdd0 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2180d0a90 0x7fe2180c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2180d0a90 0x7fe2180c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36100/0 (socket says 172.21.15.40:36100) 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 -- 172.21.15.40:0/1026281378 learned_addr learned my addr 172.21.15.40:0/1026281378 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21d26d640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180cfdd0 0x7fe2180c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21da6e640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2181608f0 0x7fe2181f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 -- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180cfdd0 msgr2=0x7fe2180c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180cfdd0 0x7fe2180c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 -- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2181608f0 msgr2=0x7fe2181f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2181608f0 0x7fe2181f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 -- 172.21.15.40:0/1026281378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2180c76e0 con 0x7fe2180d0a90 2024-04-26T21:46:17.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21da6e640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2181608f0 0x7fe2181f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21d26d640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180cfdd0 0x7fe2180c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21ca6c640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2180d0a90 0x7fe2180c6eb0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fe21400d5f0 tx=0x7fe21400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe20dffb640 1 -- 172.21.15.40:0/1026281378 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140125f0 con 0x7fe2180d0a90 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe20dffb640 1 -- 172.21.15.40:0/1026281378 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe214019070 con 0x7fe2180d0a90 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2180c79c0 con 0x7fe2180d0a90 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.491+0000 7fe20dffb640 1 -- 172.21.15.40:0/1026281378 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140129d0 con 0x7fe2180d0a90 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.492+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2180c7f80 con 0x7fe2180d0a90 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.492+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2180694f0 con 0x7fe2180d0a90 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.493+0000 7fe20dffb640 1 -- 172.21.15.40:0/1026281378 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe214012b70 con 0x7fe2180d0a90 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.493+0000 7fe20dffb640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.493+0000 7fe20dffb640 1 -- 172.21.15.40:0/1026281378 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7fe21408cd90 con 0x7fe2180d0a90 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.494+0000 7fe21d26d640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.494+0000 7fe21d26d640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe208034000 tx=0x7fe20802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:17.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.498+0000 7fe20dffb640 1 -- 172.21.15.40:0/1026281378 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe21405a400 con 0x7fe2180d0a90 2024-04-26T21:46:17.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.720+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe2181e6be0 con 0x7fe2180d0a90 2024-04-26T21:46:17.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.720+0000 7fe20dffb640 1 -- 172.21.15.40:0/1026281378 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v310) ==== 53+0+73 (secure 0 0 0) 0x7fe214059d00 con 0x7fe2180d0a90 2024-04-26T21:46:17.720 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:46:17.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.722+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1fc06c8f0 msgr2=0x7fe1fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.722+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe208034000 tx=0x7fe20802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.722+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2180d0a90 msgr2=0x7fe2180c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.722+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2180d0a90 0x7fe2180c6eb0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fe21400d5f0 tx=0x7fe21400dab0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.722+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 shutdown_connections 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.723+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.723+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2181608f0 0x7fe2181f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.723+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2180d0a90 0x7fe2180c6eb0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.723+0000 7fe21f4d1640 1 --2- 172.21.15.40:0/1026281378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2180cfdd0 0x7fe2180c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.723+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 >> 172.21.15.40:0/1026281378 conn(0x7fe2180d2ec0 msgr2=0x7fe218070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.723+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 shutdown_connections 2024-04-26T21:46:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17.723+0000 7fe21f4d1640 1 -- 172.21.15.40:0/1026281378 wait complete. 2024-04-26T21:46:17.733 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:46:17.733 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:46:17.733 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:46:17.733 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:46:17.733 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:17.733 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:46:17.733 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:46:17.733 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:46:17.734 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:17.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:17.734 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:46:17.749 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:46:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42070sec preferred_lft 42070sec 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:17.751 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:46:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:46:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:17.752 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:46:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:46:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:46:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:17.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:17.752 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:46:17.812 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:46:17.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:17.814 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:46:17.871 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:46:17.871 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:46:17.871 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:46:17.871 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:46:17.871 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:46:17.871 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:17.871 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:46:17.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:46:17.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:46:17.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:46:17.872 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:46:17.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:17.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:17.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:17.997 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:46:18.012 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:46:18.013 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:46:18.013 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:18.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:18.013 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:46:18.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:18.077 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:18.132 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:18.133 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:18.133 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:18.187 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:46:18.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:18.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:18.271 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:18.271 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:46:18.279 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:18.279 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:46:18.279 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:46:18.279 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:46:18.279 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:46:18.281 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:46:18.281 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:46:18.281 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:46:18.281 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:46:18.281 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:46:18.282 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:46:18.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:18.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:18.347 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:46:18.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:46:18.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:18.444 INFO:teuthology.orchestra.run.smithi040.stdout:client9313 2024-04-26T21:46:18.444 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:18.444 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:18.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:18.460 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-26T21:46:18.502 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:46:18.516 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:18.516+0000 7f1f4d241640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-26T21:46:18.517 INFO:teuthology.orchestra.run.smithi040.stdout:.: 2024-04-26T21:46:18.517 INFO:teuthology.orchestra.run.smithi040.stdout:subdir 2024-04-26T21:46:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:18.517 INFO:teuthology.orchestra.run.smithi040.stdout:./subdir: 2024-04-26T21:46:18.517 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 10000000000.00000000' did not caused DAMAGED state 2024-04-26T21:46:18.517 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 600.00000000' 2024-04-26T21:46:18.517 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:18.517 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:18.573 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:18.573 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:18.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:18.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.769+0000 7f8ccb9f0640 1 Processor -- start 2024-04-26T21:46:18.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.770+0000 7f8ccb9f0640 1 -- start start 2024-04-26T21:46:18.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.770+0000 7f8ccb9f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8cc406fc40 0x7f8cc4073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:18.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.770+0000 7f8ccb9f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cc4156840 con 0x7f8cc406f790 2024-04-26T21:46:18.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.770+0000 7f8ccb9f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cc40cbf20 con 0x7f8cc4156460 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.770+0000 7f8ccb9f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8cc40cc200 con 0x7f8cc406fc40 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cc8f8b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8cc406f790 0x7f8cc4072c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46084/0 (socket says 172.21.15.40:46084) 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cc8f8b640 1 -- 172.21.15.40:0/1777946934 learned_addr learned my addr 172.21.15.40:0/1777946934 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 823386397 0 0) 0x7f8cc4156840 con 0x7f8cc406f790 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cc9f8d640 1 --2- 172.21.15.40:0/1777946934 >> v2:172.21.15.184:3300/0 conn(0x7f8cc406fc40 0x7f8cc4073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cb0003520 con 0x7f8cc406f790 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1724679900 0 0) 0x7f8cb0003520 con 0x7f8cc406f790 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 >> v2:172.21.15.184:3300/0 conn(0x7f8cc406fc40 msgr2=0x7f8cc4073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 --2- 172.21.15.40:0/1777946934 >> v2:172.21.15.184:3300/0 conn(0x7f8cc406fc40 0x7f8cc4073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 >> v1:172.21.15.184:6789/0 conn(0x7f8cc4156460 legacy=0x7f8cc40cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc40cc4e0 con 0x7f8cc406f790 2024-04-26T21:46:18.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f8cb4001670 con 0x7f8cc406f790 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 >> v1:172.21.15.40:6789/0 conn(0x7f8cc406f790 legacy=0x7f8cc4072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00039b0 0x7f8cb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cb0004800 0x7f8cb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cb00051c0 0x7f8cb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc40cc200 con 0x7f8cb00051c0 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb000c060 con 0x7f8cb00039b0 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb000c260 con 0x7f8cb0004800 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cc9f8d640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cb00051c0 0x7f8cb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.771+0000 7f8cc978c640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00039b0 0x7f8cb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.772+0000 7f8cc8f8b640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cb0004800 0x7f8cb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.772+0000 7f8cc9f8d640 1 -- 172.21.15.40:0/1777946934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cb0004800 msgr2=0x7f8cb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.772+0000 7f8cc9f8d640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cb0004800 0x7f8cb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.772+0000 7f8cc9f8d640 1 -- 172.21.15.40:0/1777946934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00039b0 msgr2=0x7f8cb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:18.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.772+0000 7f8cc9f8d640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00039b0 0x7f8cb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.772+0000 7f8cc9f8d640 1 -- 172.21.15.40:0/1777946934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb0003520 con 0x7f8cb00051c0 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.772+0000 7f8cc9f8d640 1 -- 172.21.15.40:0/1777946934 reap_dead start 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.772+0000 7f8cc9f8d640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cb00051c0 0x7f8cb000b9c0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009fc0 tx=0x7f8cb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3901b42e59e93de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc035020 con 0x7f8cb00051c0 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cbc003bd0 con 0x7f8cb00051c0 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8cbbfff640 1 -- 172.21.15.40:0/1777946934 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc009c10 con 0x7f8cb00051c0 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1777946934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cb00051c0 msgr2=0x7f8cb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8ccb9f0640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cb00051c0 0x7f8cb000b9c0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009fc0 tx=0x7f8cb0005830 comp rx=0 tx=0).stop 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1777946934 shutdown_connections 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8ccb9f0640 1 --2- 172.21.15.40:0/1777946934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cb00051c0 0x7f8cb000b9c0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1777946934 >> 172.21.15.40:0/1777946934 conn(0x7f8cc40d0be0 msgr2=0x7f8cc40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1777946934 shutdown_connections 2024-04-26T21:46:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.773+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1777946934 wait complete. 2024-04-26T21:46:18.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.774+0000 7f8ccb9f0640 1 Processor -- start 2024-04-26T21:46:18.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8ccb9f0640 1 -- start start 2024-04-26T21:46:18.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8ccb9f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00051c0 0x7f8cc41f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:18.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8ccb9f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cc40b50d0 0x7f8cc40c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:18.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8ccb9f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cc41f4590 0x7f8cc41f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:18.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8ccb9f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc4070050 con 0x7f8cc40b50d0 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8ccb9f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc4072700 con 0x7f8cb00051c0 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8ccb9f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc406e260 con 0x7f8cc41f4590 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8cc8f8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cc40b50d0 0x7f8cc40c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8cc8f8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cc40b50d0 0x7f8cc40c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44230/0 (socket says 172.21.15.40:44230) 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.775+0000 7f8cc8f8b640 1 -- 172.21.15.40:0/1209560941 learned_addr learned my addr 172.21.15.40:0/1209560941 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc978c640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00051c0 0x7f8cc41f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc9f8d640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cc41f4590 0x7f8cc41f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc8f8b640 1 -- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cc41f4590 msgr2=0x7f8cc41f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc8f8b640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cc41f4590 0x7f8cc41f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc8f8b640 1 -- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00051c0 msgr2=0x7f8cc41f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc8f8b640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00051c0 0x7f8cc41f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc8f8b640 1 -- 172.21.15.40:0/1209560941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc40b5550 con 0x7f8cc40b50d0 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc978c640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00051c0 0x7f8cc41f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cc8f8b640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cc40b50d0 0x7f8cc40c6410 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4007340 tx=0x7f8cb4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:18.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cb9ffb640 1 -- 172.21.15.40:0/1209560941 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb400f8c0 con 0x7f8cc40b50d0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cb9ffb640 1 -- 172.21.15.40:0/1209560941 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb4018070 con 0x7f8cc40b50d0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8cb9ffb640 1 -- 172.21.15.40:0/1209560941 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb400fca0 con 0x7f8cc40b50d0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc40b3900 con 0x7f8cc40b50d0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.776+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc40b3ec0 con 0x7f8cc40b50d0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.777+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc40694f0 con 0x7f8cc40b50d0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.778+0000 7f8cb9ffb640 1 -- 172.21.15.40:0/1209560941 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cb401d460 con 0x7f8cc40b50d0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.778+0000 7f8cb9ffb640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ca806c8f0 0x7f8ca806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.778+0000 7f8cb9ffb640 1 -- 172.21.15.40:0/1209560941 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7f8cb408c970 con 0x7f8cc40b50d0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.778+0000 7f8cc978c640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ca806c8f0 0x7f8ca806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:18.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.779+0000 7f8cc978c640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ca806c8f0 0x7f8ca806edb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0005580 tx=0x7f8cc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:18.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.782+0000 7f8cb9ffb640 1 -- 172.21.15.40:0/1209560941 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8cc40b5550 con 0x7f8cc40b50d0 2024-04-26T21:46:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.969+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cc40c7740 con 0x7f8cc40b50d0 2024-04-26T21:46:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.971+0000 7f8cb9ffb640 1 -- 172.21.15.40:0/1209560941 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+21452 (secure 0 0 0) 0x7f8cb405ee20 con 0x7f8cc40b50d0 2024-04-26T21:46:18.971 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:18.971 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":129,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:46:08.370627+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6832/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6833/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6833/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6832/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6833/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6833/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6832/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3538213517":"2024-04-27T21:45:17.788819+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/3538213517":"2024-04-27T21:45:17.788819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ca806c8f0 msgr2=0x7f8ca806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ca806c8f0 0x7f8ca806edb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0005580 tx=0x7f8cc0009040 comp rx=0 tx=0).stop 2024-04-26T21:46:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cc40b50d0 msgr2=0x7f8cc40c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cc40b50d0 0x7f8cc40c6410 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4007340 tx=0x7f8cb4007800 comp rx=0 tx=0).stop 2024-04-26T21:46:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 shutdown_connections 2024-04-26T21:46:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8ca806c8f0 0x7f8ca806edb0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cc41f4590 0x7f8cc41f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cc40b50d0 0x7f8cc40c6410 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 --2- 172.21.15.40:0/1209560941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb00051c0 0x7f8cc41f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 >> 172.21.15.40:0/1209560941 conn(0x7f8cc40d0be0 msgr2=0x7f8cc40d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 shutdown_connections 2024-04-26T21:46:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:18.973+0000 7f8ccb9f0640 1 -- 172.21.15.40:0/1209560941 wait complete. 2024-04-26T21:46:18.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:18.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.088+0000 7f17d0215640 1 Processor -- start 2024-04-26T21:46:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.088+0000 7f17d0215640 1 -- start start 2024-04-26T21:46:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17d0215640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17d0215640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17c8072c50 0x7f17c80730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17d0215640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c8156840 con 0x7f17c8156460 2024-04-26T21:46:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17d0215640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f17c80cbf20 con 0x7f17c8072c50 2024-04-26T21:46:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17d0215640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17c80cc200 con 0x7f17c806f790 2024-04-26T21:46:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17ce7b2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17c8072c50 0x7f17c80730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17ce7b2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17c8072c50 0x7f17c80730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44242/0 (socket says 172.21.15.40:44242) 2024-04-26T21:46:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17ce7b2640 1 -- 172.21.15.40:0/1030416264 learned_addr learned my addr 172.21.15.40:0/1030416264 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17ce7b2640 1 -- 172.21.15.40:0/1030416264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 msgr2=0x7f17c806fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17cd7b0640 1 --2- 172.21.15.40:0/1030416264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17ce7b2640 1 --2- 172.21.15.40:0/1030416264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c806fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17ce7b2640 1 -- 172.21.15.40:0/1030416264 >> v1:172.21.15.184:6789/0 conn(0x7f17c8156460 legacy=0x7f17c80cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17ce7b2640 1 -- 172.21.15.40:0/1030416264 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c80cc4e0 con 0x7f17c8072c50 2024-04-26T21:46:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.089+0000 7f17cd7b0640 1 --2- 172.21.15.40:0/1030416264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c806fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.090+0000 7f17ce7b2640 1 --2- 172.21.15.40:0/1030416264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c80730c0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e50 tx=0x7f17c402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22619795e0c7f847 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.090+0000 7f17ccfaf640 1 -- 172.21.15.40:0/1030416264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4036070 con 0x7f17c8072c50 2024-04-26T21:46:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.090+0000 7f17ccfaf640 1 -- 172.21.15.40:0/1030416264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4031600 con 0x7f17c8072c50 2024-04-26T21:46:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.090+0000 7f17ccfaf640 1 -- 172.21.15.40:0/1030416264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c40319d0 con 0x7f17c8072c50 2024-04-26T21:46:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.091+0000 7f17d0215640 1 -- 172.21.15.40:0/1030416264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 msgr2=0x7f17c80730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.091+0000 7f17d0215640 1 --2- 172.21.15.40:0/1030416264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c80730c0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e50 tx=0x7f17c402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.091+0000 7f17d0215640 1 -- 172.21.15.40:0/1030416264 shutdown_connections 2024-04-26T21:46:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.091+0000 7f17d0215640 1 --2- 172.21.15.40:0/1030416264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c80730c0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.091+0000 7f17d0215640 1 --2- 172.21.15.40:0/1030416264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c806fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.091+0000 7f17d0215640 1 -- 172.21.15.40:0/1030416264 >> 172.21.15.40:0/1030416264 conn(0x7f17c80d0be0 msgr2=0x7f17c80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.091+0000 7f17d0215640 1 -- 172.21.15.40:0/1030416264 shutdown_connections 2024-04-26T21:46:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.091+0000 7f17d0215640 1 -- 172.21.15.40:0/1030416264 wait complete. 2024-04-26T21:46:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.092+0000 7f17d0215640 1 Processor -- start 2024-04-26T21:46:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.092+0000 7f17d0215640 1 -- start start 2024-04-26T21:46:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17d0215640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17d0215640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c81e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17d0215640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17c8156460 0x7f17c81e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17d0215640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f17c80cbbe0 con 0x7f17c8072c50 2024-04-26T21:46:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17d0215640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8072700 con 0x7f17c8156460 2024-04-26T21:46:19.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17d0215640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17c806e260 con 0x7f17c806f790 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c81e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c81e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44246/0 (socket says 172.21.15.40:44246) 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 -- 172.21.15.40:0/4006098887 learned_addr learned my addr 172.21.15.40:0/4006098887 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cdfb1640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17ce7b2640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17c8156460 0x7f17c81e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 -- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 msgr2=0x7f17c80b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c80b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 -- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17c8156460 msgr2=0x7f17c81e8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17c8156460 0x7f17c81e8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 -- 172.21.15.40:0/4006098887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c8153680 con 0x7f17c8072c50 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17ce7b2640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17c8156460 0x7f17c81e8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.093+0000 7f17cd7b0640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c81e88a0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f17b800c820 tx=0x7f17b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.094+0000 7f17be7fc640 1 -- 172.21.15.40:0/4006098887 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8015070 con 0x7f17c8072c50 2024-04-26T21:46:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.094+0000 7f17be7fc640 1 -- 172.21.15.40:0/4006098887 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b8007a30 con 0x7f17c8072c50 2024-04-26T21:46:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.094+0000 7f17be7fc640 1 -- 172.21.15.40:0/4006098887 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8007e00 con 0x7f17c8072c50 2024-04-26T21:46:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.094+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c8153960 con 0x7f17c8072c50 2024-04-26T21:46:19.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.094+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c8153f20 con 0x7f17c8072c50 2024-04-26T21:46:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.095+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c80694f0 con 0x7f17c8072c50 2024-04-26T21:46:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.095+0000 7f17be7fc640 1 -- 172.21.15.40:0/4006098887 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17b8018070 con 0x7f17c8072c50 2024-04-26T21:46:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.096+0000 7f17be7fc640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17ac06c8f0 0x7f17ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.096+0000 7f17be7fc640 1 -- 172.21.15.40:0/4006098887 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7f17b808c850 con 0x7f17c8072c50 2024-04-26T21:46:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.096+0000 7f17cdfb1640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17ac06c8f0 0x7f17ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.096+0000 7f17cdfb1640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17ac06c8f0 0x7f17ac06edb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f17c0006fd0 tx=0x7f17c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.100+0000 7f17be7fc640 1 -- 172.21.15.40:0/4006098887 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17b8059f50 con 0x7f17c8072c50 2024-04-26T21:46:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.282+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f17c806de80 con 0x7f17c8072c50 2024-04-26T21:46:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.282+0000 7f17be7fc640 1 -- 172.21.15.40:0/4006098887 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) ==== 93+0+10 (secure 0 0 0) 0x7f17b8059850 con 0x7f17c8072c50 2024-04-26T21:46:19.283 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:46:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17ac06c8f0 msgr2=0x7f17ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17ac06c8f0 0x7f17ac06edb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f17c0006fd0 tx=0x7f17c0008040 comp rx=0 tx=0).stop 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 msgr2=0x7f17c81e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c81e88a0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f17b800c820 tx=0x7f17b800cc20 comp rx=0 tx=0).stop 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 shutdown_connections 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f17ac06c8f0 0x7f17ac06edb0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17c8156460 0x7f17c81e8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17c8072c50 0x7f17c81e88a0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 --2- 172.21.15.40:0/4006098887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17c806f790 0x7f17c80b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 >> 172.21.15.40:0/4006098887 conn(0x7f17c80d0be0 msgr2=0x7f17c8150c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 shutdown_connections 2024-04-26T21:46:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.284+0000 7f17d0215640 1 -- 172.21.15.40:0/4006098887 wait complete. 2024-04-26T21:46:19.295 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.399+0000 7fae74cf8640 1 Processor -- start 2024-04-26T21:46:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.399+0000 7fae74cf8640 1 -- start start 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae74cf8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fae700cc480 0x7fae70072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae74cf8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70072780 0x7fae70166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae74cf8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7015d7c0 con 0x7fae70071e60 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae74cf8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fae7015daa0 con 0x7fae700cc480 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae74cf8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae7015dd80 con 0x7fae70072780 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fae70071e60 0x7fae7006f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46086/0 (socket says 172.21.15.40:46086) 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6e59c640 1 -- 172.21.15.40:0/3892646502 learned_addr learned my addr 172.21.15.40:0/3892646502 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6dd9b640 1 --2- 172.21.15.40:0/3892646502 >> v2:172.21.15.184:3300/0 conn(0x7fae700cc480 0x7fae70072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6ed9d640 1 --2- 172.21.15.40:0/3892646502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70072780 0x7fae70166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6d59a640 1 -- 172.21.15.40:0/3892646502 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2512203447 0 0) 0x7fae7015d7c0 con 0x7fae70071e60 2024-04-26T21:46:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6d59a640 1 -- 172.21.15.40:0/3892646502 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae5c003520 con 0x7fae70071e60 2024-04-26T21:46:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6dd9b640 1 -- 172.21.15.40:0/3892646502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70072780 msgr2=0x7fae70166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6dd9b640 1 --2- 172.21.15.40:0/3892646502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70072780 0x7fae70166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6dd9b640 1 -- 172.21.15.40:0/3892646502 >> v1:172.21.15.40:6789/0 conn(0x7fae70071e60 legacy=0x7fae7006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.400+0000 7fae6dd9b640 1 -- 172.21.15.40:0/3892646502 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7015e060 con 0x7fae700cc480 2024-04-26T21:46:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.401+0000 7fae6dd9b640 1 --2- 172.21.15.40:0/3892646502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae700cc480 0x7fae70072240 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fae580097a0 tx=0x7fae5802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3cfae45e05df5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.401+0000 7fae6d59a640 1 -- 172.21.15.40:0/3892646502 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58037070 con 0x7fae700cc480 2024-04-26T21:46:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.401+0000 7fae6d59a640 1 -- 172.21.15.40:0/3892646502 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae58032440 con 0x7fae700cc480 2024-04-26T21:46:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.401+0000 7fae6d59a640 1 -- 172.21.15.40:0/3892646502 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580327f0 con 0x7fae700cc480 2024-04-26T21:46:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.402+0000 7fae74cf8640 1 -- 172.21.15.40:0/3892646502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae700cc480 msgr2=0x7fae70072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.402+0000 7fae74cf8640 1 --2- 172.21.15.40:0/3892646502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae700cc480 0x7fae70072240 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fae580097a0 tx=0x7fae5802a180 comp rx=0 tx=0).stop 2024-04-26T21:46:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.402+0000 7fae74cf8640 1 -- 172.21.15.40:0/3892646502 shutdown_connections 2024-04-26T21:46:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.402+0000 7fae74cf8640 1 --2- 172.21.15.40:0/3892646502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70072780 0x7fae70166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.402+0000 7fae74cf8640 1 --2- 172.21.15.40:0/3892646502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae700cc480 0x7fae70072240 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.402+0000 7fae74cf8640 1 -- 172.21.15.40:0/3892646502 >> 172.21.15.40:0/3892646502 conn(0x7fae700d0be0 msgr2=0x7fae700d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.402+0000 7fae74cf8640 1 -- 172.21.15.40:0/3892646502 shutdown_connections 2024-04-26T21:46:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.402+0000 7fae74cf8640 1 -- 172.21.15.40:0/3892646502 wait complete. 2024-04-26T21:46:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.403+0000 7fae74cf8640 1 Processor -- start 2024-04-26T21:46:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.403+0000 7fae74cf8640 1 -- start start 2024-04-26T21:46:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae74cf8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70071e60 0x7fae700afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae74cf8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae70072780 0x7fae700b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae74cf8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700cc480 0x7fae700c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae74cf8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae7015d3f0 con 0x7fae700cc480 2024-04-26T21:46:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae74cf8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae70072c10 con 0x7fae70072780 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae74cf8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae7015d570 con 0x7fae70071e60 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700cc480 0x7fae700c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700cc480 0x7fae700c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44262/0 (socket says 172.21.15.40:44262) 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6ed9d640 1 -- 172.21.15.40:0/50295163 learned_addr learned my addr 172.21.15.40:0/50295163 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6e59c640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70071e60 0x7fae700afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6dd9b640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae70072780 0x7fae700b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6ed9d640 1 -- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70071e60 msgr2=0x7fae700afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6ed9d640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70071e60 0x7fae700afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6ed9d640 1 -- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae70072780 msgr2=0x7fae700b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6ed9d640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae70072780 0x7fae700b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6ed9d640 1 -- 172.21.15.40:0/50295163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae700b54f0 con 0x7fae700cc480 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6dd9b640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae70072780 0x7fae700b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.404+0000 7fae6e59c640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70071e60 0x7fae700afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.405+0000 7fae6ed9d640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700cc480 0x7fae700c6210 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fae60009dd0 tx=0x7fae6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.405+0000 7fae66ffd640 1 -- 172.21.15.40:0/50295163 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60018020 con 0x7fae700cc480 2024-04-26T21:46:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.405+0000 7fae66ffd640 1 -- 172.21.15.40:0/50295163 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae60014480 con 0x7fae700cc480 2024-04-26T21:46:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.405+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae70148430 con 0x7fae700cc480 2024-04-26T21:46:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.405+0000 7fae66ffd640 1 -- 172.21.15.40:0/50295163 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60014870 con 0x7fae700cc480 2024-04-26T21:46:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.405+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae701489f0 con 0x7fae700cc480 2024-04-26T21:46:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.406+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae700d11f0 con 0x7fae700cc480 2024-04-26T21:46:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.406+0000 7fae66ffd640 1 -- 172.21.15.40:0/50295163 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae60005d10 con 0x7fae700cc480 2024-04-26T21:46:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.407+0000 7fae66ffd640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5006c8f0 0x7fae5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.407+0000 7fae66ffd640 1 -- 172.21.15.40:0/50295163 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7fae6008d360 con 0x7fae700cc480 2024-04-26T21:46:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.407+0000 7fae6e59c640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5006c8f0 0x7fae5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.408+0000 7fae6e59c640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5006c8f0 0x7fae5006edb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fae68007fd0 tx=0x7fae68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.411+0000 7fae66ffd640 1 -- 172.21.15.40:0/50295163 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae700d11f0 con 0x7fae700cc480 2024-04-26T21:46:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.594+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae700694f0 con 0x7fae700cc480 2024-04-26T21:46:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.595+0000 7fae66ffd640 1 -- 172.21.15.40:0/50295163 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) ==== 104+0+11 (secure 0 0 0) 0x7fae60022070 con 0x7fae700cc480 2024-04-26T21:46:19.595 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:46:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5006c8f0 msgr2=0x7fae5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5006c8f0 0x7fae5006edb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fae68007fd0 tx=0x7fae68009040 comp rx=0 tx=0).stop 2024-04-26T21:46:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700cc480 msgr2=0x7fae700c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700cc480 0x7fae700c6210 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fae60009dd0 tx=0x7fae6000c490 comp rx=0 tx=0).stop 2024-04-26T21:46:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 shutdown_connections 2024-04-26T21:46:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fae5006c8f0 0x7fae5006edb0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700cc480 0x7fae700c6210 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae70072780 0x7fae700b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 --2- 172.21.15.40:0/50295163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae70071e60 0x7fae700afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 >> 172.21.15.40:0/50295163 conn(0x7fae700d0be0 msgr2=0x7fae7015a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 shutdown_connections 2024-04-26T21:46:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.597+0000 7fae74cf8640 1 -- 172.21.15.40:0/50295163 wait complete. 2024-04-26T21:46:19.607 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.712+0000 7f06b26da640 1 Processor -- start 2024-04-26T21:46:19.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.713+0000 7f06b26da640 1 -- start start 2024-04-26T21:46:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.713+0000 7f06b26da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac0d0d60 0x7f06ac0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.713+0000 7f06b26da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06ac0ce5e0 con 0x7f06ac0d1240 2024-04-26T21:46:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.713+0000 7f06b26da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06ac0ce8c0 con 0x7f06ac0b4360 2024-04-26T21:46:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.713+0000 7f06b26da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac0cf210 con 0x7f06ac0d0d60 2024-04-26T21:46:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.713+0000 7f06abfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f06ac0d1240 0x7f06ac0d1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46092/0 (socket says 172.21.15.40:46092) 2024-04-26T21:46:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.713+0000 7f06abfff640 1 -- 172.21.15.40:0/844458291 learned_addr learned my addr 172.21.15.40:0/844458291 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:19.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06b0c77640 1 --2- 172.21.15.40:0/844458291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac0d0d60 0x7f06ac0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06aaffd640 1 -- 172.21.15.40:0/844458291 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4223978463 0 0) 0x7f06ac0ce5e0 con 0x7f06ac0d1240 2024-04-26T21:46:19.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06aaffd640 1 -- 172.21.15.40:0/844458291 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0694003520 con 0x7f06ac0d1240 2024-04-26T21:46:19.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06aaffd640 1 -- 172.21.15.40:0/844458291 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3267066311 0 0) 0x7f06ac0ce8c0 con 0x7f06ac0b4360 2024-04-26T21:46:19.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06aaffd640 1 -- 172.21.15.40:0/844458291 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06ac0ce5e0 con 0x7f06ac0b4360 2024-04-26T21:46:19.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06aaffd640 1 -- 172.21.15.40:0/844458291 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3833544312 0 0) 0x7f0694003520 con 0x7f06ac0d1240 2024-04-26T21:46:19.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06b0c77640 1 -- 172.21.15.40:0/844458291 >> v1:172.21.15.184:6789/0 conn(0x7f06ac0b4360 legacy=0x7f06ac0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06b0c77640 1 -- 172.21.15.40:0/844458291 >> v1:172.21.15.40:6789/0 conn(0x7f06ac0d1240 legacy=0x7f06ac0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.714+0000 7f06b0c77640 1 -- 172.21.15.40:0/844458291 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac160ab0 con 0x7f06ac0d0d60 2024-04-26T21:46:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.715+0000 7f06b0c77640 1 --2- 172.21.15.40:0/844458291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac0d0d60 0x7f06ac0b2d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009dc0 tx=0x7f06a402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d5f1928078300f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.715+0000 7f06aaffd640 1 -- 172.21.15.40:0/844458291 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c001670 con 0x7f06ac0d0d60 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.715+0000 7f06aaffd640 1 -- 172.21.15.40:0/844458291 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a4007540 con 0x7f06ac0d0d60 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.715+0000 7f06aaffd640 1 -- 172.21.15.40:0/844458291 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4007910 con 0x7f06ac0d0d60 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.715+0000 7f06b26da640 1 -- 172.21.15.40:0/844458291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac0d0d60 msgr2=0x7f06ac0b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.715+0000 7f06b26da640 1 --2- 172.21.15.40:0/844458291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac0d0d60 0x7f06ac0b2d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009dc0 tx=0x7f06a402a660 comp rx=0 tx=0).stop 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.716+0000 7f06b26da640 1 -- 172.21.15.40:0/844458291 shutdown_connections 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.716+0000 7f06b26da640 1 --2- 172.21.15.40:0/844458291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac0d0d60 0x7f06ac0b2d10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.716+0000 7f06b26da640 1 -- 172.21.15.40:0/844458291 >> 172.21.15.40:0/844458291 conn(0x7f06ac0d2e60 msgr2=0x7f06ac0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.716+0000 7f06b26da640 1 -- 172.21.15.40:0/844458291 shutdown_connections 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.716+0000 7f06b26da640 1 -- 172.21.15.40:0/844458291 wait complete. 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.717+0000 7f06b26da640 1 Processor -- start 2024-04-26T21:46:19.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.717+0000 7f06b26da640 1 -- start start 2024-04-26T21:46:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06b26da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d0d60 0x7f06ac1f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06b26da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06ac0d1240 0x7f06ac1f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06b26da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac1f50b0 0x7f06ac0b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06b26da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac0c6790 con 0x7f06ac0d0d60 2024-04-26T21:46:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06b26da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac0c6610 con 0x7f06ac0d1240 2024-04-26T21:46:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06b26da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac0b3250 con 0x7f06ac1f50b0 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d0d60 0x7f06ac1f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d0d60 0x7f06ac1f2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44270/0 (socket says 172.21.15.40:44270) 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 -- 172.21.15.40:0/1159880583 learned_addr learned my addr 172.21.15.40:0/1159880583 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 -- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac1f50b0 msgr2=0x7f06ac0b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac1f50b0 0x7f06ac0b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 -- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06ac0d1240 msgr2=0x7f06ac1f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06ac0d1240 0x7f06ac1f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 -- 172.21.15.40:0/1159880583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac1f5720 con 0x7f06ac0d0d60 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06abfff640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d0d60 0x7f06ac1f2260 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c5a0 tx=0x7f069c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06a8ff9640 1 -- 172.21.15.40:0/1159880583 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c017070 con 0x7f06ac0d0d60 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06a8ff9640 1 -- 172.21.15.40:0/1159880583 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c005ce0 con 0x7f06ac0d0d60 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.718+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac1f5a00 con 0x7f06ac0d0d60 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.719+0000 7f06a8ff9640 1 -- 172.21.15.40:0/1159880583 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0072b0 con 0x7f06ac0d0d60 2024-04-26T21:46:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.719+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac0b5ca0 con 0x7f06ac0d0d60 2024-04-26T21:46:19.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.720+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ac0caf80 con 0x7f06ac0d0d60 2024-04-26T21:46:19.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.720+0000 7f06a8ff9640 1 -- 172.21.15.40:0/1159880583 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f069c01a070 con 0x7f06ac0d0d60 2024-04-26T21:46:19.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.720+0000 7f06a8ff9640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069806c920 0x7f069806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:19.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.720+0000 7f06a8ff9640 1 -- 172.21.15.40:0/1159880583 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7f069c08bdb0 con 0x7f06ac0d0d60 2024-04-26T21:46:19.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.721+0000 7f06ab7fe640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069806c920 0x7f069806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:19.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.721+0000 7f06ab7fe640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069806c920 0x7f069806ede0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f06a00010e0 tx=0x7f06a0007950 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.725+0000 7f06a8ff9640 1 -- 172.21.15.40:0/1159880583 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f069c059420 con 0x7f06ac0d0d60 2024-04-26T21:46:19.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.908+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f06ac0694f0 con 0x7f06ac0d0d60 2024-04-26T21:46:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.909+0000 7f06a8ff9640 1 -- 172.21.15.40:0/1159880583 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) ==== 100+0+11 (secure 0 0 0) 0x7f069c058d20 con 0x7f06ac0d0d60 2024-04-26T21:46:19.909 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069806c920 msgr2=0x7f069806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069806c920 0x7f069806ede0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f06a00010e0 tx=0x7f06a0007950 comp rx=0 tx=0).stop 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d0d60 msgr2=0x7f06ac1f2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d0d60 0x7f06ac1f2260 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c5a0 tx=0x7f069c00ca60 comp rx=0 tx=0).stop 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 shutdown_connections 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f069806c920 0x7f069806ede0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06ac1f50b0 0x7f06ac0b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06ac0d1240 0x7f06ac1f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 --2- 172.21.15.40:0/1159880583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06ac0d0d60 0x7f06ac1f2260 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 >> 172.21.15.40:0/1159880583 conn(0x7f06ac0d2e60 msgr2=0x7f06ac0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 shutdown_connections 2024-04-26T21:46:19.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:19.911+0000 7f06b26da640 1 -- 172.21.15.40:0/1159880583 wait complete. 2024-04-26T21:46:19.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:20.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.024+0000 7f430a815640 1 Processor -- start 2024-04-26T21:46:20.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.024+0000 7f430a815640 1 -- start start 2024-04-26T21:46:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f430a815640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43040b2340 0x7f43040cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f430a815640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43040d10d0 con 0x7f43040cd930 2024-04-26T21:46:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f430a815640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43040d1370 con 0x7f4304162810 2024-04-26T21:46:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f430a815640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f43040d0750 con 0x7f43040b2340 2024-04-26T21:46:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f43037fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43040b2340 0x7f43040cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f4308db2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f43040cd930 0x7f43040d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46096/0 (socket says 172.21.15.40:46096) 2024-04-26T21:46:20.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f4308db2640 1 -- 172.21.15.40:0/2138385548 learned_addr learned my addr 172.21.15.40:0/2138385548 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:20.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f43037fe640 1 -- 172.21.15.40:0/2138385548 >> v1:172.21.15.184:6789/0 conn(0x7f4304162810 legacy=0x7f4304070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f43037fe640 1 -- 172.21.15.40:0/2138385548 >> v1:172.21.15.40:6789/0 conn(0x7f43040cd930 legacy=0x7f43040d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.025+0000 7f43037fe640 1 -- 172.21.15.40:0/2138385548 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43040d01b0 con 0x7f43040b2340 2024-04-26T21:46:20.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.026+0000 7f43037fe640 1 --2- 172.21.15.40:0/2138385548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43040b2340 0x7f43040cd3f0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f43040d1540 tx=0x7f42f402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e8935c7aa55079 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.026+0000 7f4302ffd640 1 -- 172.21.15.40:0/2138385548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4037070 con 0x7f43040b2340 2024-04-26T21:46:20.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.026+0000 7f4302ffd640 1 -- 172.21.15.40:0/2138385548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f402ada0 con 0x7f43040b2340 2024-04-26T21:46:20.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.026+0000 7f4302ffd640 1 -- 172.21.15.40:0/2138385548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4032660 con 0x7f43040b2340 2024-04-26T21:46:20.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.027+0000 7f430a815640 1 -- 172.21.15.40:0/2138385548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43040b2340 msgr2=0x7f43040cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.027+0000 7f430a815640 1 --2- 172.21.15.40:0/2138385548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43040b2340 0x7f43040cd3f0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f43040d1540 tx=0x7f42f402a110 comp rx=0 tx=0).stop 2024-04-26T21:46:20.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.027+0000 7f430a815640 1 -- 172.21.15.40:0/2138385548 shutdown_connections 2024-04-26T21:46:20.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.027+0000 7f430a815640 1 --2- 172.21.15.40:0/2138385548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43040b2340 0x7f43040cd3f0 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.027+0000 7f430a815640 1 -- 172.21.15.40:0/2138385548 >> 172.21.15.40:0/2138385548 conn(0x7f43040d2da0 msgr2=0x7f43040d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:20.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.027+0000 7f430a815640 1 -- 172.21.15.40:0/2138385548 shutdown_connections 2024-04-26T21:46:20.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.027+0000 7f430a815640 1 -- 172.21.15.40:0/2138385548 wait complete. 2024-04-26T21:46:20.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.028+0000 7f430a815640 1 Processor -- start 2024-04-26T21:46:20.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.028+0000 7f430a815640 1 -- start start 2024-04-26T21:46:20.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f430a815640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43040b2340 0x7f43041f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f430a815640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43040cd930 0x7f430415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f430a815640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4304162810 0x7f430415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f430a815640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43040d2240 con 0x7f4304162810 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f430a815640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43040d16a0 con 0x7f43040b2340 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f430a815640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43040d23c0 con 0x7f43040cd930 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4304162810 0x7f430415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4304162810 0x7f430415d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44284/0 (socket says 172.21.15.40:44284) 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 -- 172.21.15.40:0/2996315605 learned_addr learned my addr 172.21.15.40:0/2996315605 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f43037fe640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43040cd930 0x7f430415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4303fff640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43040b2340 0x7f43041f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 -- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43040cd930 msgr2=0x7f430415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43040cd930 0x7f430415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 -- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43040b2340 msgr2=0x7f43041f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43040b2340 0x7f43041f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 -- 172.21.15.40:0/2996315605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430415ec10 con 0x7f4304162810 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4303fff640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43040b2340 0x7f43041f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f43037fe640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43040cd930 0x7f430415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4308db2640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4304162810 0x7f430415d6e0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00caf0 tx=0x7f42fc002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.029+0000 7f4300ff9640 1 -- 172.21.15.40:0/2996315605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc018070 con 0x7f4304162810 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.030+0000 7f4300ff9640 1 -- 172.21.15.40:0/2996315605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42fc00a700 con 0x7f4304162810 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.030+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43041e97d0 con 0x7f4304162810 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.030+0000 7f4300ff9640 1 -- 172.21.15.40:0/2996315605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc00aab0 con 0x7f4304162810 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.030+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430415f190 con 0x7f4304162810 2024-04-26T21:46:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.031+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43040694f0 con 0x7f4304162810 2024-04-26T21:46:20.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.031+0000 7f4300ff9640 1 -- 172.21.15.40:0/2996315605 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42fc01b070 con 0x7f4304162810 2024-04-26T21:46:20.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.032+0000 7f4300ff9640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42f006c920 0x7f42f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.032+0000 7f4300ff9640 1 -- 172.21.15.40:0/2996315605 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7f42fc08c190 con 0x7f4304162810 2024-04-26T21:46:20.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.032+0000 7f4303fff640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42f006c920 0x7f42f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.033+0000 7f4303fff640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42f006c920 0x7f42f006ede0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f42f80015e0 tx=0x7f42f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.036+0000 7f4300ff9640 1 -- 172.21.15.40:0/2996315605 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42fc059770 con 0x7f4304162810 2024-04-26T21:46:20.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.234+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43040b45e0 con 0x7f4304162810 2024-04-26T21:46:20.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.234+0000 7f4300ff9640 1 -- 172.21.15.40:0/2996315605 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 68 entries v129) ==== 81+0+4138 (secure 0 0 0) 0x7f42fc059070 con 0x7f4304162810 2024-04-26T21:46:20.234 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1753847972 2024-04-27T21:46:08.370602+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1753847972 2024-04-27T21:46:08.370602+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1660522931 2024-04-27T21:45:51.733253+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4214786907 2024-04-27T21:45:46.312381+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2896027915 2024-04-27T21:45:31.907924+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2896027915 2024-04-27T21:45:31.907924+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4173136579 2024-04-27T21:45:27.871638+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:46:20.235 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3791539151 2024-04-27T21:45:23.843779+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4192917874 2024-04-27T21:45:19.814510+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4214786907 2024-04-27T21:45:46.312381+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:46:20.236 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3791539151 2024-04-27T21:45:23.843779+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1660522931 2024-04-27T21:45:51.733253+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3538213517 2024-04-27T21:45:17.788819+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:46:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4192917874 2024-04-27T21:45:19.814510+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4173136579 2024-04-27T21:45:27.871638+0000 2024-04-26T21:46:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:46:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3538213517 2024-04-27T21:45:17.788819+0000 2024-04-26T21:46:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:listed 68 entries 2024-04-26T21:46:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42f006c920 msgr2=0x7f42f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42f006c920 0x7f42f006ede0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f42f80015e0 tx=0x7f42f8009040 comp rx=0 tx=0).stop 2024-04-26T21:46:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4304162810 msgr2=0x7f430415d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4304162810 0x7f430415d6e0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00caf0 tx=0x7f42fc002080 comp rx=0 tx=0).stop 2024-04-26T21:46:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 shutdown_connections 2024-04-26T21:46:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42f006c920 0x7f42f006ede0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4304162810 0x7f430415d6e0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43040cd930 0x7f430415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 --2- 172.21.15.40:0/2996315605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43040b2340 0x7f43041f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 >> 172.21.15.40:0/2996315605 conn(0x7f43040d2da0 msgr2=0x7f43040d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 shutdown_connections 2024-04-26T21:46:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.236+0000 7f430a815640 1 -- 172.21.15.40:0/2996315605 wait complete. 2024-04-26T21:46:20.247 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:46:20.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:20.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:46:20.279 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:46:20.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:20.280 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:46:20.335 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:46:20.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.478+0000 7fc2d2be6640 1 Processor -- start 2024-04-26T21:46:20.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.478+0000 7fc2d2be6640 1 -- start start 2024-04-26T21:46:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2d2be6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc2cc162810 0x7fc2cc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2d2be6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc0d10d0 con 0x7fc2cc0b2340 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2d2be6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc0d1370 con 0x7fc2cc0d1b30 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2d2be6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc2cc0d0750 con 0x7fc2cc162810 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2cbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc2cc0b2340 0x7fc2cc0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46108/0 (socket says 172.21.15.40:46108) 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2cbfff640 1 -- 172.21.15.40:0/600643163 learned_addr learned my addr 172.21.15.40:0/600643163 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2d0982640 1 --2- 172.21.15.40:0/600643163 >> v2:172.21.15.184:3300/0 conn(0x7fc2cc162810 0x7fc2cc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/600643163 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481810887 0 0) 0x7fc2cc0d10d0 con 0x7fc2cc0b2340 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.479+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/600643163 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2b4003520 con 0x7fc2cc0b2340 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.480+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/600643163 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1664953813 0 0) 0x7fc2cc0d1370 con 0x7fc2cc0d1b30 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.480+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/600643163 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2cc0d10d0 con 0x7fc2cc0d1b30 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.480+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/600643163 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1144902892 0 0) 0x7fc2b4003520 con 0x7fc2cc0b2340 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.480+0000 7fc2d0982640 1 -- 172.21.15.40:0/600643163 >> v1:172.21.15.184:6789/0 conn(0x7fc2cc0d1b30 legacy=0x7fc2cc070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.480+0000 7fc2d0982640 1 -- 172.21.15.40:0/600643163 >> v1:172.21.15.40:6789/0 conn(0x7fc2cc0b2340 legacy=0x7fc2cc0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.480+0000 7fc2d0982640 1 -- 172.21.15.40:0/600643163 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc0d01b0 con 0x7fc2cc162810 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.480+0000 7fc2d0982640 1 --2- 172.21.15.40:0/600643163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc162810 0x7fc2cc070350 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0097a0 tx=0x7fc2bc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72005c8ea37969df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/600643163 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc037070 con 0x7fc2cc162810 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/600643163 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2bc02ac50 con 0x7fc2cc162810 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/600643163 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0001670 con 0x7fc2cc162810 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2d2be6640 1 -- 172.21.15.40:0/600643163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc162810 msgr2=0x7fc2cc070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2d2be6640 1 --2- 172.21.15.40:0/600643163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc162810 0x7fc2cc070350 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0097a0 tx=0x7fc2bc02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2d2be6640 1 -- 172.21.15.40:0/600643163 shutdown_connections 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2d2be6640 1 --2- 172.21.15.40:0/600643163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc162810 0x7fc2cc070350 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2d2be6640 1 -- 172.21.15.40:0/600643163 >> 172.21.15.40:0/600643163 conn(0x7fc2cc0d2da0 msgr2=0x7fc2cc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2d2be6640 1 -- 172.21.15.40:0/600643163 shutdown_connections 2024-04-26T21:46:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.481+0000 7fc2d2be6640 1 -- 172.21.15.40:0/600643163 wait complete. 2024-04-26T21:46:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.482+0000 7fc2d2be6640 1 Processor -- start 2024-04-26T21:46:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.482+0000 7fc2d2be6640 1 -- start start 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d2be6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc0b2340 0x7fc2cc0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d2be6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc0d1b30 0x7fc2cc1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d2be6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2cc162810 0x7fc2cc0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d2be6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc070890 con 0x7fc2cc0b2340 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d2be6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0d16a0 con 0x7fc2cc0d1b30 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d2be6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2cc0cdb00 con 0x7fc2cc162810 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d0982640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc0b2340 0x7fc2cc0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d0982640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc0b2340 0x7fc2cc0afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44296/0 (socket says 172.21.15.40:44296) 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.483+0000 7fc2d0982640 1 -- 172.21.15.40:0/267113363 learned_addr learned my addr 172.21.15.40:0/267113363 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d1183640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2cc162810 0x7fc2cc0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2cbfff640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc0d1b30 0x7fc2cc1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d0982640 1 -- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2cc162810 msgr2=0x7fc2cc0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d0982640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2cc162810 0x7fc2cc0c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d0982640 1 -- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc0d1b30 msgr2=0x7fc2cc1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d0982640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc0d1b30 0x7fc2cc1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d0982640 1 -- 172.21.15.40:0/267113363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc14ffe0 con 0x7fc2cc0b2340 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d0982640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc0b2340 0x7fc2cc0afab0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc035040 tx=0x7fc2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2c97fa640 1 -- 172.21.15.40:0/267113363 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc037050 con 0x7fc2cc0b2340 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2c97fa640 1 -- 172.21.15.40:0/267113363 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2bc043070 con 0x7fc2cc0b2340 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2c97fa640 1 -- 172.21.15.40:0/267113363 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc006e10 con 0x7fc2cc0b2340 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc0c6950 con 0x7fc2cc0b2340 2024-04-26T21:46:20.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.484+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc149a00 con 0x7fc2cc0b2340 2024-04-26T21:46:20.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.485+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2cc0694f0 con 0x7fc2cc0b2340 2024-04-26T21:46:20.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.486+0000 7fc2c97fa640 1 -- 172.21.15.40:0/267113363 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2bc03c830 con 0x7fc2cc0b2340 2024-04-26T21:46:20.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.486+0000 7fc2c97fa640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc2b806c8f0 0x7fc2b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.486+0000 7fc2c97fa640 1 -- 172.21.15.40:0/267113363 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9141+0+0 (secure 0 0 0) 0x7fc2bc0ad0c0 con 0x7fc2cc0b2340 2024-04-26T21:46:20.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.486+0000 7fc2cbfff640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc2b806c8f0 0x7fc2b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.487+0000 7fc2cbfff640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc2b806c8f0 0x7fc2b806edb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc15cba0 tx=0x7fc2c0001670 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.491+0000 7fc2c97fa640 1 -- 172.21.15.40:0/267113363 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2bc07a0e0 con 0x7fc2cc0b2340 2024-04-26T21:46:20.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.721+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc2cc15cfe0 con 0x7fc2cc0b2340 2024-04-26T21:46:20.721 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:20.722+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:46:20.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.725+0000 7fc2c97fa640 1 -- 172.21.15.40:0/267113363 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v311) ==== 161+0+0 (secure 0 0 0) 0x7fc2bc07e3e0 con 0x7fc2cc0b2340 2024-04-26T21:46:20.724 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:46:20.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc2b806c8f0 msgr2=0x7fc2b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc2b806c8f0 0x7fc2b806edb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc15cba0 tx=0x7fc2c0001670 comp rx=0 tx=0).stop 2024-04-26T21:46:20.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc0b2340 msgr2=0x7fc2cc0afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc0b2340 0x7fc2cc0afab0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc035040 tx=0x7fc2bc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 shutdown_connections 2024-04-26T21:46:20.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc2b806c8f0 0x7fc2b806edb0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2cc162810 0x7fc2cc0c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc0d1b30 0x7fc2cc1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 --2- 172.21.15.40:0/267113363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc0b2340 0x7fc2cc0afab0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 >> 172.21.15.40:0/267113363 conn(0x7fc2cc0d2da0 msgr2=0x7fc2cc0b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:20.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 shutdown_connections 2024-04-26T21:46:20.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.727+0000 7fc2d2be6640 1 -- 172.21.15.40:0/267113363 wait complete. 2024-04-26T21:46:20.728 INFO:tasks.ceph.mds.c.smithi040.stderr: -725> 2024-04-26T21:46:14.380+0000 7f1f51249640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:46:20.729 INFO:tasks.ceph.mds.c.smithi040.stderr: -213> 2024-04-26T21:46:18.516+0000 7f1f4d241640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-26T21:46:20.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:46:20.754 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:46:20.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.843+0000 7f8ed5299640 1 Processor -- start 2024-04-26T21:46:20.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.843+0000 7f8ed5299640 1 -- start start 2024-04-26T21:46:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ed5299640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8ed00b0380 0x7f8ed00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ed5299640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 0x7f8ed006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ed5299640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed00cfd80 con 0x7f8ed006c780 2024-04-26T21:46:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ed5299640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed00af8e0 con 0x7f8ed00b0380 2024-04-26T21:46:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ed5299640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed00d0020 con 0x7f8ed00b10b0 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8eceffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8ed00b0380 0x7f8ed00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ecf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8ed006c780 0x7f8ed0072f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46126/0 (socket says 172.21.15.40:46126) 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ecf7fe640 1 -- 172.21.15.40:0/4009677623 learned_addr learned my addr 172.21.15.40:0/4009677623 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ec7fff640 1 --2- 172.21.15.40:0/4009677623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 0x7f8ed006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ece7fc640 1 -- 172.21.15.40:0/4009677623 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808803296 0 0) 0x7f8ed00cfd80 con 0x7f8ed006c780 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8ece7fc640 1 -- 172.21.15.40:0/4009677623 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ebc003520 con 0x7f8ed006c780 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8eceffd640 1 -- 172.21.15.40:0/4009677623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 msgr2=0x7f8ed006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8eceffd640 1 --2- 172.21.15.40:0/4009677623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 0x7f8ed006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8eceffd640 1 -- 172.21.15.40:0/4009677623 >> v1:172.21.15.40:6789/0 conn(0x7f8ed006c780 legacy=0x7f8ed0072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.844+0000 7f8eceffd640 1 -- 172.21.15.40:0/4009677623 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed00d0300 con 0x7f8ed00b0380 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.845+0000 7f8eceffd640 1 --2- 172.21.15.40:0/4009677623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed00ccd50 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00097a0 tx=0x7f8ec002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f4c8b8c57ac4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.845+0000 7f8ece7fc640 1 -- 172.21.15.40:0/4009677623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0037070 con 0x7f8ed00b0380 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.845+0000 7f8ece7fc640 1 -- 172.21.15.40:0/4009677623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec002ae50 con 0x7f8ed00b0380 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.845+0000 7f8ece7fc640 1 -- 172.21.15.40:0/4009677623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0032660 con 0x7f8ed00b0380 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.846+0000 7f8ed5299640 1 -- 172.21.15.40:0/4009677623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 msgr2=0x7f8ed00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.846+0000 7f8ed5299640 1 --2- 172.21.15.40:0/4009677623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed00ccd50 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00097a0 tx=0x7f8ec002a100 comp rx=0 tx=0).stop 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.846+0000 7f8ed5299640 1 -- 172.21.15.40:0/4009677623 shutdown_connections 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.846+0000 7f8ed5299640 1 --2- 172.21.15.40:0/4009677623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 0x7f8ed006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.846+0000 7f8ed5299640 1 --2- 172.21.15.40:0/4009677623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed00ccd50 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.846+0000 7f8ed5299640 1 -- 172.21.15.40:0/4009677623 >> 172.21.15.40:0/4009677623 conn(0x7f8ed00d1cf0 msgr2=0x7f8ed00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.846+0000 7f8ed5299640 1 -- 172.21.15.40:0/4009677623 shutdown_connections 2024-04-26T21:46:20.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.846+0000 7f8ed5299640 1 -- 172.21.15.40:0/4009677623 wait complete. 2024-04-26T21:46:20.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.847+0000 7f8ed5299640 1 Processor -- start 2024-04-26T21:46:20.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.847+0000 7f8ed5299640 1 -- start start 2024-04-26T21:46:20.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ed5299640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ed006c780 0x7f8ed015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ed5299640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed015e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ed5299640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 0x7f8ed0070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ed5299640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed00b0760 con 0x7f8ed00b0380 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ed5299640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed00d1980 con 0x7f8ed006c780 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ed5299640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed00d15c0 con 0x7f8ed00b10b0 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44326/0 (socket says 172.21.15.40:44326) 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 -- 172.21.15.40:0/2009814317 learned_addr learned my addr 172.21.15.40:0/2009814317 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ecf7fe640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 0x7f8ed0070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8eceffd640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ed006c780 0x7f8ed015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 -- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 msgr2=0x7f8ed0070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 0x7f8ed0070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 -- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ed006c780 msgr2=0x7f8ed015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ed006c780 0x7f8ed015cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 -- 172.21.15.40:0/2009814317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed015dd40 con 0x7f8ed00b0380 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec7fff640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed015e480 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800d620 tx=0x7f8eb800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec77fe640 1 -- 172.21.15.40:0/2009814317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800b620 con 0x7f8ed00b0380 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec77fe640 1 -- 172.21.15.40:0/2009814317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb8019070 con 0x7f8ed00b0380 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ec77fe640 1 -- 172.21.15.40:0/2009814317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800ba00 con 0x7f8ed00b0380 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed015e020 con 0x7f8ed00b0380 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.848+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed00b58e0 con 0x7f8ed00b0380 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.850+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed00694f0 con 0x7f8ed00b0380 2024-04-26T21:46:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.850+0000 7f8ec77fe640 1 -- 172.21.15.40:0/2009814317 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8eb800bba0 con 0x7f8ed00b0380 2024-04-26T21:46:20.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.850+0000 7f8ec77fe640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8eb006c920 0x7f8eb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:20.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.850+0000 7f8ec77fe640 1 -- 172.21.15.40:0/2009814317 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9227+0+0 (secure 0 0 0) 0x7f8eb808ce20 con 0x7f8ed00b0380 2024-04-26T21:46:20.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.851+0000 7f8eceffd640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8eb006c920 0x7f8eb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:20.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.851+0000 7f8eceffd640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8eb006c920 0x7f8eb006ede0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0034000 tx=0x7f8ec002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:20.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:20.855+0000 7f8ec77fe640 1 -- 172.21.15.40:0/2009814317 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8eb805a330 con 0x7f8ed00b0380 2024-04-26T21:46:21.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.114+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8ed015d2a0 con 0x7f8ed00b0380 2024-04-26T21:46:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.728+0000 7f8ec77fe640 1 -- 172.21.15.40:0/2009814317 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v312) ==== 107+0+0 (secure 0 0 0) 0x7f8eb8059c30 con 0x7f8ed00b0380 2024-04-26T21:46:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:46:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8eb006c920 msgr2=0x7f8eb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8eb006c920 0x7f8eb006ede0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0034000 tx=0x7f8ec002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 msgr2=0x7f8ed015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed015e480 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800d620 tx=0x7f8eb800dae0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 shutdown_connections 2024-04-26T21:46:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8eb006c920 0x7f8eb006ede0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ed00b10b0 0x7f8ed0070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed00b0380 0x7f8ed015e480 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 --2- 172.21.15.40:0/2009814317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ed006c780 0x7f8ed015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 >> 172.21.15.40:0/2009814317 conn(0x7f8ed00d1cf0 msgr2=0x7f8ed00cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 shutdown_connections 2024-04-26T21:46:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.731+0000 7f8ed5299640 1 -- 172.21.15.40:0/2009814317 wait complete. 2024-04-26T21:46:21.745 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:46:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.832+0000 7feb7c362f40 1 Processor -- start 2024-04-26T21:46:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.832+0000 7feb7c362f40 1 -- start start 2024-04-26T21:46:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7c362f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f3729e1c0 0x563f3736fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7c362f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f37379660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7c362f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f37370f70 0x563f372a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7c362f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563f37246e40 con 0x563f372e9420 2024-04-26T21:46:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7c362f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563f3724ae80 con 0x563f3729e1c0 2024-04-26T21:46:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7c362f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563f373707e0 con 0x563f37370f70 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7965b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f37379660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7965b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f37379660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44328/0 (socket says 172.21.15.40:44328) 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7965b640 1 -- 172.21.15.40:0/3864494452 learned_addr learned my addr 172.21.15.40:0/3864494452 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7a65d640 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f37370f70 0x563f372a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f3729e1c0 0x563f3736fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7965b640 1 -- 172.21.15.40:0/3864494452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f37370f70 msgr2=0x563f372a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7965b640 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f37370f70 0x563f372a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7965b640 1 -- 172.21.15.40:0/3864494452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f3729e1c0 msgr2=0x563f3736fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7965b640 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f3729e1c0 0x563f3736fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.833+0000 7feb7965b640 1 -- 172.21.15.40:0/3864494452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f372a0550 con 0x563f372e9420 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7965b640 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f37379660 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7feb70001e80 tx=0x7feb7000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234a3c7eb06adf90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb6bfff640 1 -- 172.21.15.40:0/3864494452 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70016070 con 0x563f372e9420 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb6bfff640 1 -- 172.21.15.40:0/3864494452 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7000be50 con 0x563f372e9420 2024-04-26T21:46:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb6bfff640 1 -- 172.21.15.40:0/3864494452 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000d910 con 0x563f372e9420 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 -- 172.21.15.40:0/3864494452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 msgr2=0x563f37379660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f37379660 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7feb70001e80 tx=0x7feb7000b650 comp rx=0 tx=0).stop 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 -- 172.21.15.40:0/3864494452 shutdown_connections 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f37370f70 0x563f372a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f37379660 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3864494452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f3729e1c0 0x563f3736fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 -- 172.21.15.40:0/3864494452 >> 172.21.15.40:0/3864494452 conn(0x563f372c98a0 msgr2=0x563f372ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 -- 172.21.15.40:0/3864494452 shutdown_connections 2024-04-26T21:46:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.834+0000 7feb7c362f40 1 -- 172.21.15.40:0/3864494452 wait complete. 2024-04-26T21:46:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.835+0000 7feb7c362f40 1 Processor -- start 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.835+0000 7feb7c362f40 1 -- start start 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7c362f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f3729e1c0 0x563f3736e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7c362f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f372cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7c362f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f37370f70 0x563f372cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7c362f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563f37377870 con 0x563f372e9420 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7c362f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563f372a0270 con 0x563f37370f70 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7c362f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563f37377b50 con 0x563f3729e1c0 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f372cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f372cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44332/0 (socket says 172.21.15.40:44332) 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 learned_addr learned my addr 172.21.15.40:0/3177278922 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f3729e1c0 0x563f3736e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7a65d640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f37370f70 0x563f372cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f3729e1c0 msgr2=0x563f3736e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f3729e1c0 0x563f3736e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f37370f70 msgr2=0x563f372cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f37370f70 0x563f372cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3740d460 con 0x563f372e9420 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563f3729e1c0 0x563f3736e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7a65d640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f37370f70 0x563f372cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.836+0000 7feb7965b640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f372cda70 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7feb7000f000 tx=0x7feb70000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.837+0000 7feb6affd640 1 -- 172.21.15.40:0/3177278922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70016070 con 0x563f372e9420 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.837+0000 7feb6affd640 1 -- 172.21.15.40:0/3177278922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb70010440 con 0x563f372e9420 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.837+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563f3740d740 con 0x563f372e9420 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.837+0000 7feb6affd640 1 -- 172.21.15.40:0/3177278922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70010810 con 0x563f372e9420 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.837+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3740dd00 con 0x563f372e9420 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.838+0000 7feb6affd640 1 -- 172.21.15.40:0/3177278922 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb700109b0 con 0x563f372e9420 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.839+0000 7feb6affd640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb5c06b2d0 0x7feb5c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.839+0000 7feb6affd640 1 -- 172.21.15.40:0/3177278922 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9227+0+0 (secure 0 0 0) 0x7feb7008ca90 con 0x563f372e9420 2024-04-26T21:46:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.839+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb5c06b2d0 0x7feb5c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.841 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:46:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.839+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563f3736ece0 0x563f37371450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.839+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37371990 con 0x563f3736ece0 2024-04-26T21:46:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.839+0000 7feb7a65d640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563f3736ece0 0x563f37371450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.839+0000 7feb7a65d640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563f3736ece0 0x563f37371450 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.839+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb5c06b2d0 0x7feb5c06d790 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00a520 tx=0x7feb6c050000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.843+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v130'1065 uv1038 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.843+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f37371d60 con 0x563f3736ece0 2024-04-26T21:46:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.845+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v130'1066 uv1066 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.845+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37372130 con 0x563f3736ece0 2024-04-26T21:46:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.846+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v130'1067 uv1067 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.846 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:46:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.846+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f373724d0 con 0x563f3736ece0 2024-04-26T21:46:21.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.847+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v130'1068 uv1068 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.848+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a8c0 con 0x563f3736ece0 2024-04-26T21:46:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.849+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v130'1069 uv1069 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.849+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f374198b0 con 0x563f3736ece0 2024-04-26T21:46:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.850+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v130'1070 uv1070 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.850+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37419ca0 con 0x563f3736ece0 2024-04-26T21:46:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.851+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v130'1071 uv1044 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.851+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.852+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v130'1072 uv1072 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.852+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.854+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v130'1073 uv1073 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.853 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:46:21.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.854+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.855+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v130'1074 uv1074 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.855+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.856+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v130'1075 uv1075 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.856+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.857+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v130'1076 uv1076 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.857+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.858+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v130'1077 uv1064 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.858+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.859+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v130'1078 uv1078 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.860+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.861+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v130'1079 uv1079 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.861 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:46:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.861+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.862+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v130'1080 uv1080 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.862+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.863+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v130'1081 uv1081 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.863+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741a040 con 0x563f3736ece0 2024-04-26T21:46:21.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.864+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v130'1082 uv1082 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.864+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.865+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v130'1083 uv1083 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.865+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.867+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v130'1084 uv1057 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.867+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.868+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v130'1085 uv1085 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.868+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.868 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:46:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.869+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v130'1086 uv1086 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.869+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741bc70 con 0x563f3736ece0 2024-04-26T21:46:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.870+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v130'1087 uv1087 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.870+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741bc70 con 0x563f3736ece0 2024-04-26T21:46:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.871+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v130'1088 uv1088 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.871+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.872+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v130'1089 uv1089 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.872+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.874+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v130'1090 uv1063 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.874+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.875+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v130'1091 uv1091 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.875+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.875 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:46:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.876+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v130'1092 uv1092 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.876+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741bc70 con 0x563f3736ece0 2024-04-26T21:46:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.877+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v130'1093 uv1093 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.877+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741bc70 con 0x563f3736ece0 2024-04-26T21:46:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.878+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v130'1094 uv1094 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.878+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741ac60 con 0x563f3736ece0 2024-04-26T21:46:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.879+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v130'1095 uv1095 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f3736ece0 2024-04-26T21:46:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.879+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563f3741b170 0x563f3741d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.879+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741daa0 con 0x563f3741b170 2024-04-26T21:46:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.880+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563f3741b170 0x563f3741d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.880+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563f3741b170 0x563f3741d560 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.884+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v130'777 uv756 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.884+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3741de40 con 0x563f3741b170 2024-04-26T21:46:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.886+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v130'778 uv778 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.886+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741e260 con 0x563f3741b170 2024-04-26T21:46:21.887 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:46:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.887+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v130'779 uv779 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.887+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741eb20 con 0x563f3741b170 2024-04-26T21:46:21.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.889+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v130'780 uv780 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.889+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741eec0 con 0x563f3741b170 2024-04-26T21:46:21.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.890+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v130'781 uv781 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.890+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741f260 con 0x563f3741b170 2024-04-26T21:46:21.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.891+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v130'782 uv775 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.891+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3741f650 con 0x563f3741b170 2024-04-26T21:46:21.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.893+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v130'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.893+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741fa40 con 0x563f3741b170 2024-04-26T21:46:21.894 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:46:21.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.895+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v130'784 uv784 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.895+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741fa40 con 0x563f3741b170 2024-04-26T21:46:21.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.896+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v130'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.896+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741f9f0 con 0x563f3741b170 2024-04-26T21:46:21.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.898+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v130'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.898+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741f9f0 con 0x563f3741b170 2024-04-26T21:46:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.899+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v130'787 uv776 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.899+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3741f9f0 con 0x563f3741b170 2024-04-26T21:46:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.900+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v130'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.900+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741f9f0 con 0x563f3741b170 2024-04-26T21:46:21.901 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:46:21.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.902+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v130'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.902+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f374240b0 con 0x563f3741b170 2024-04-26T21:46:21.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.904+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v130'790 uv790 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.904+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f374240b0 con 0x563f3741b170 2024-04-26T21:46:21.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.905+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v130'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.905+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741f9f0 con 0x563f3741b170 2024-04-26T21:46:21.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.907+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v130'792 uv772 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.907+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3741f9f0 con 0x563f3741b170 2024-04-26T21:46:21.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.908+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v130'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.908+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741f9f0 con 0x563f3741b170 2024-04-26T21:46:21.909 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:46:21.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.909+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v130'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.909+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37420a00 con 0x563f3741b170 2024-04-26T21:46:21.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.911+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v130'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.911+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37420a00 con 0x563f3741b170 2024-04-26T21:46:21.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.912+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v130'796 uv796 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.912+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3741f9f0 con 0x563f3741b170 2024-04-26T21:46:21.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.913+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v130'797 uv797 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f3741b170 2024-04-26T21:46:21.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.913+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563f374202c0 0x563f374226b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.913+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37422bf0 con 0x563f374202c0 2024-04-26T21:46:21.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.914+0000 7feb7965b640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563f374202c0 0x563f374226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.914+0000 7feb7965b640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563f374202c0 0x563f374226b0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.915+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v130'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.916+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f37422fd0 con 0x563f374202c0 2024-04-26T21:46:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.917+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v130'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.917+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f374234b0 con 0x563f374202c0 2024-04-26T21:46:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.918+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v130'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.918 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:46:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.918+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37424d60 con 0x563f374202c0 2024-04-26T21:46:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.919+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v130'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.919+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37425100 con 0x563f374202c0 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.920+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v130'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.920+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423d50 con 0x563f374202c0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.922+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v130'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.922+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563f37424650 0x563f374274b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.922+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f374279f0 con 0x563f37424650 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.922+0000 7feb7a65d640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563f37424650 0x563f374274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.923+0000 7feb7a65d640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563f37424650 0x563f374274b0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.924+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v130'161 uv160 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.924+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f374279f0 con 0x563f37424650 2024-04-26T21:46:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.925+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v130'162 uv162 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.925+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37427e10 con 0x563f37424650 2024-04-26T21:46:21.926 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:46:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.926+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v130'163 uv163 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.926+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37427e10 con 0x563f37424650 2024-04-26T21:46:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.928+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v130'164 uv164 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.928+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563f37428490 0x563f3742a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.928+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742adc0 con 0x563f37428490 2024-04-26T21:46:21.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.928+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563f37428490 0x563f3742a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.928+0000 7feb79e5c640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563f37428490 0x563f3742a880 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.931+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v130'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.931+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742ae40 con 0x563f37428490 2024-04-26T21:46:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.933+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v130'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.933+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.935 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:46:21.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.935+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v130'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.935+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742c270 con 0x563f37428490 2024-04-26T21:46:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.937+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v130'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.937+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742c270 con 0x563f37428490 2024-04-26T21:46:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.938+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v130'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.938+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.939+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v130'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.939+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.941+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v130'382 uv376 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.941+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.942+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v130'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.942+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.942 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:46:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.943+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v130'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.943+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742c270 con 0x563f37424650 2024-04-26T21:46:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.944+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v130'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.944+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742c270 con 0x563f37424650 2024-04-26T21:46:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.945+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v130'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.945+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.946+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v130'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.946+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.947+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v130'388 uv381 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.947+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.948+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v130'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.948+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.948 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:46:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.949+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v130'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.949+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.950+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v130'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.950+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.951+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v130'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.951+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.952+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v130'173 uv172 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.952+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.953+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v130'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.953+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.954 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:46:21.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.954+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v130'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.954+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.956+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v130'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.956+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37424650 2024-04-26T21:46:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.957+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v130'177 uv177 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb74004520 con 0x563f37424650 2024-04-26T21:46:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.957+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.958+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v130'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.958+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.959+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v130'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.959+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.960+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v130'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.960 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:46:21.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.960+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742c270 con 0x563f37428490 2024-04-26T21:46:21.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.962+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v130'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.962+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742c270 con 0x563f37428490 2024-04-26T21:46:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.963+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v130'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.963+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.964+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v130'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.964+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.965+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v130'345 uv339 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.965+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.967+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v130'346 uv346 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.967+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.967+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v130'347 uv347 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.967 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:46:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.968+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.969+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v130'348 uv348 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.969+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.970+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v130'349 uv349 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.970+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.971+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v130'350 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.971+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.972+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v130'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.972+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.973+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v130'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.973 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:46:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.973+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742b260 con 0x563f37428490 2024-04-26T21:46:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.974+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v130'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.974+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742e050 con 0x563f37428490 2024-04-26T21:46:21.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.975+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v130'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.975+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563f3742b650 0x563f3742da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:21.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.975+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742e710 con 0x563f3742b650 2024-04-26T21:46:21.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.976+0000 7feb7965b640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563f3742b650 0x563f3742da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:21.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.976+0000 7feb7965b640 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563f3742b650 0x563f3742da40 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:21.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.977+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v130'152 uv150 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f3742b650 2024-04-26T21:46:21.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.977+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742eab0 con 0x563f3742b650 2024-04-26T21:46:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.980+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v130'153 uv153 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f3742b650 2024-04-26T21:46:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.980+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742eaf0 con 0x563f3742b650 2024-04-26T21:46:21.981 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:46:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.981+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v130'154 uv154 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f3742b650 2024-04-26T21:46:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.981+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742eee0 con 0x563f3742b650 2024-04-26T21:46:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.982+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v130'155 uv155 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f3742b650 2024-04-26T21:46:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.982+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742eee0 con 0x563f37428490 2024-04-26T21:46:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.983+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v130'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.983+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742eee0 con 0x563f37428490 2024-04-26T21:46:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.984+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v130'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.985+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742eee0 con 0x563f37428490 2024-04-26T21:46:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.985+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v130'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.985 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:46:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.986+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742fef0 con 0x563f37428490 2024-04-26T21:46:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.987+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v130'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.987+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742fef0 con 0x563f37428490 2024-04-26T21:46:21.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.988+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v130'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.988+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742eee0 con 0x563f37428490 2024-04-26T21:46:21.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.989+0000 7feb79e5c640 1 -- 172.21.15.40:0/3177278922 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v130'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb6c05c040 con 0x563f37428490 2024-04-26T21:46:21.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.989+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742eee0 con 0x563f374202c0 2024-04-26T21:46:21.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.991+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v130'176 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.991+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f3742ebd0 con 0x563f374202c0 2024-04-26T21:46:21.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.992+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v130'177 uv177 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.992+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742ebd0 con 0x563f374202c0 2024-04-26T21:46:21.993 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:46:21.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.993+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v130'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.993+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:21.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.994+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v130'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.994+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:21.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.995+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v130'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.995+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:21.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.997+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v130'429 uv420 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.997+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:21.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.998+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v130'430 uv430 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.998+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:21.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.999+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v130'431 uv431 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:21.999 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:46:21.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:21.999+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:21.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.000+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v130'432 uv432 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.000+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:22.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.001+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v130'433 uv433 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.001+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:22.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.002+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v130'434 uv427 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.002+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:22.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.003+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v130'435 uv435 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.003+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.004+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v130'436 uv436 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.004 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:46:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.004+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.005+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v130'437 uv437 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.005+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.006+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v130'438 uv438 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.006+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f37423930 con 0x563f374202c0 2024-04-26T21:46:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.007+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v130'439 uv439 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.007+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x563f3742f0b0 con 0x563f374202c0 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7965b640 1 -- 172.21.15.40:0/3177278922 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v130'440 uv440 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb70056050 con 0x563f374202c0 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563f3741b170 msgr2=0x563f3741d560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x563f3741b170 0x563f3741d560 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563f3736ece0 msgr2=0x563f37371450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x563f3736ece0 0x563f37371450 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563f37424650 msgr2=0x563f374274b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x563f37424650 0x563f374274b0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563f3742b650 msgr2=0x563f3742da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x563f3742b650 0x563f3742da40 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563f374202c0 msgr2=0x563f374226b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563f374202c0 0x563f374226b0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563f37428490 msgr2=0x563f3742a880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563f37428490 0x563f3742a880 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 reap_dead start 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7a65d640 1 -- 172.21.15.40:0/3177278922 reap_dead start 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb5c06b2d0 msgr2=0x7feb5c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb5c06b2d0 0x7feb5c06d790 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00a520 tx=0x7feb6c050000 comp rx=0 tx=0).stop 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 msgr2=0x563f372cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f372cda70 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7feb7000f000 tx=0x7feb70000bc0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 shutdown_connections 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.008+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7feb5c06b2d0 0x7feb5c06d790 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.009+0000 7feb7c362f40 1 --2- 172.21.15.40:0/3177278922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563f372e9420 0x563f372cda70 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.009+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 >> 172.21.15.40:0/3177278922 conn(0x563f372c98a0 msgr2=0x563f3736f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.009+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 shutdown_connections 2024-04-26T21:46:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.009+0000 7feb7c362f40 1 -- 172.21.15.40:0/3177278922 wait complete. 2024-04-26T21:46:22.012 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 600.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f6058a1ef40 1 Processor -- start 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f6058a1ef40 1 -- start start 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f6058a1ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 0x559aee5cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f6058a1ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 0x559aee51fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f6058a1ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee537fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f6058a1ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559aee49be40 con 0x559aee5ccb70 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f6058a1ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559aee53d660 con 0x559aee53e420 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f6058a1ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559aee51e360 con 0x559aee5c5f70 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f60560a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee537fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f60560a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee537fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44350/0 (socket says 172.21.15.40:44350) 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.095+0000 7f60560a7640 1 -- 172.21.15.40:0/3843986836 learned_addr learned my addr 172.21.15.40:0/3843986836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f60550a5640 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 0x559aee51fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f60558a6640 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 0x559aee5cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f60560a7640 1 -- 172.21.15.40:0/3843986836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 msgr2=0x559aee51fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f60560a7640 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 0x559aee51fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f60560a7640 1 -- 172.21.15.40:0/3843986836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 msgr2=0x559aee5cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f60560a7640 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 0x559aee5cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f60560a7640 1 -- 172.21.15.40:0/3843986836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aee51e080 con 0x559aee5ccb70 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f60560a7640 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee537fb0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f604800a4b0 tx=0x7f604800a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4ae73bd5fe6252 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f604ffff640 1 -- 172.21.15.40:0/3843986836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048015070 con 0x559aee5ccb70 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f604ffff640 1 -- 172.21.15.40:0/3843986836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604800e730 con 0x559aee5ccb70 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f604ffff640 1 -- 172.21.15.40:0/3843986836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604800eb20 con 0x559aee5ccb70 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3843986836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 msgr2=0x559aee537fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f6058a1ef40 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee537fb0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f604800a4b0 tx=0x7f604800a970 comp rx=0 tx=0).stop 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3843986836 shutdown_connections 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f6058a1ef40 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee537fb0 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.096+0000 7f6058a1ef40 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 0x559aee51fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.097+0000 7f6058a1ef40 1 --2- 172.21.15.40:0/3843986836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 0x559aee5cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.097+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3843986836 >> 172.21.15.40:0/3843986836 conn(0x559aee51e8a0 msgr2=0x559aee5cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.097+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3843986836 shutdown_connections 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.097+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3843986836 wait complete. 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.097+0000 7f6058a1ef40 1 Processor -- start 2024-04-26T21:46:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.097+0000 7f6058a1ef40 1 -- start start 2024-04-26T21:46:22.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f6058a1ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 0x559aee4f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f6058a1ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 0x559aee4f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f6058a1ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee539000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f6058a1ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559aee4f6f00 con 0x559aee5ccb70 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f6058a1ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559aee4f71e0 con 0x559aee53e420 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f6058a1ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x559aee5c56c0 con 0x559aee5c5f70 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60560a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee539000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60560a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee539000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44356/0 (socket says 172.21.15.40:44356) 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60560a7640 1 -- 172.21.15.40:0/3665378923 learned_addr learned my addr 172.21.15.40:0/3665378923 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60550a5640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 0x559aee4f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60558a6640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 0x559aee4f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60560a7640 1 -- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 msgr2=0x559aee4f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60560a7640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559aee5c5f70 0x559aee4f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60560a7640 1 -- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 msgr2=0x559aee4f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60560a7640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559aee53e420 0x559aee4f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.098+0000 7f60560a7640 1 -- 172.21.15.40:0/3665378923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aee5c7410 con 0x559aee5ccb70 2024-04-26T21:46:22.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.099+0000 7f60560a7640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee539000 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f6048010000 tx=0x7f60480032f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.099+0000 7f604effd640 1 -- 172.21.15.40:0/3665378923 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604801f020 con 0x559aee5ccb70 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.099+0000 7f604effd640 1 -- 172.21.15.40:0/3665378923 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6048016040 con 0x559aee5ccb70 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.099+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559aee5c76f0 con 0x559aee5ccb70 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.099+0000 7f604effd640 1 -- 172.21.15.40:0/3665378923 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048015070 con 0x559aee5ccb70 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.099+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aee5c7cb0 con 0x559aee5ccb70 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.100+0000 7f604effd640 1 -- 172.21.15.40:0/3665378923 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6048022020 con 0x559aee5ccb70 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.100+0000 7f604effd640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f603806b2d0 0x7f603806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.101+0000 7f604effd640 1 -- 172.21.15.40:0/3665378923 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9227+0+0 (secure 0 0 0) 0x7f604808c750 con 0x559aee5ccb70 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.101+0000 7f60558a6640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f603806b2d0 0x7f603806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.101+0000 7f6058a1ef40 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559aee5c81d0 0x559aee539a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.101+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x559aee5cb520 con 0x559aee5c81d0 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.101+0000 7f60550a5640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559aee5c81d0 0x559aee539a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.101+0000 7f60558a6640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f603806b2d0 0x7f603806d790 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f60500097c0 tx=0x7f6050005b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.101+0000 7f60550a5640 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559aee5c81d0 0x559aee539a50 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f60550a5640 1 -- 172.21.15.40:0/3665378923 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 600.00000000 [omap-set-header] v130'1096 uv1096 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f604004db40 con 0x559aee5c81d0 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559aee5c81d0 msgr2=0x559aee539a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x559aee5c81d0 0x559aee539a50 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f603806b2d0 msgr2=0x7f603806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f603806b2d0 0x7f603806d790 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f60500097c0 tx=0x7f6050005b30 comp rx=0 tx=0).stop 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 msgr2=0x559aee539000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 --2- 172.21.15.40:0/3665378923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559aee5ccb70 0x559aee539000 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f6048010000 tx=0x7f60480032f0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f60560a7640 1 -- 172.21.15.40:0/3665378923 reap_dead start 2024-04-26T21:46:22.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 shutdown_connections 2024-04-26T21:46:22.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 >> 172.21.15.40:0/3665378923 conn(0x559aee51e8a0 msgr2=0x559aee4f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:22.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 shutdown_connections 2024-04-26T21:46:22.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.103+0000 7f6058a1ef40 1 -- 172.21.15.40:0/3665378923 wait complete. 2024-04-26T21:46:22.106 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:46:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.250+0000 7ff1836d9640 1 Processor -- start 2024-04-26T21:46:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.250+0000 7ff1836d9640 1 -- start start 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff1836d9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff17c15f550 0x7ff17c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff1836d9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff17c0d3bc0 0x7ff17c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff1836d9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff17c0d1040 con 0x7ff17c15f170 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff1836d9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff17c0c9430 con 0x7ff17c0d3bc0 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff1836d9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff17c0c9680 con 0x7ff17c15f550 2024-04-26T21:46:22.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff181c76640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff17c0d3bc0 0x7ff17c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff181c76640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff17c0d3bc0 0x7ff17c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44366/0 (socket says 172.21.15.40:44366) 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff181c76640 1 -- 172.21.15.40:0/3444058915 learned_addr learned my addr 172.21.15.40:0/3444058915 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff180c74640 1 --2- 172.21.15.40:0/3444058915 >> v2:172.21.15.184:3300/0 conn(0x7ff17c15f550 0x7ff17c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff181c76640 1 -- 172.21.15.40:0/3444058915 >> v2:172.21.15.184:3300/0 conn(0x7ff17c15f550 msgr2=0x7ff17c0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff181c76640 1 --2- 172.21.15.40:0/3444058915 >> v2:172.21.15.184:3300/0 conn(0x7ff17c15f550 0x7ff17c0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff181c76640 1 -- 172.21.15.40:0/3444058915 >> v1:172.21.15.184:6789/0 conn(0x7ff17c15f170 legacy=0x7ff17c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.251+0000 7ff181c76640 1 -- 172.21.15.40:0/3444058915 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c0c9960 con 0x7ff17c0d3bc0 2024-04-26T21:46:22.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.252+0000 7ff181c76640 1 --2- 172.21.15.40:0/3444058915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c0d3bc0 0x7ff17c0d2cb0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7ff174009e50 tx=0x7ff17402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac3b7cae3a26bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.252+0000 7ff173fff640 1 -- 172.21.15.40:0/3444058915 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174036070 con 0x7ff17c0d3bc0 2024-04-26T21:46:22.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.252+0000 7ff173fff640 1 -- 172.21.15.40:0/3444058915 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff174031600 con 0x7ff17c0d3bc0 2024-04-26T21:46:22.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.252+0000 7ff173fff640 1 -- 172.21.15.40:0/3444058915 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1740319d0 con 0x7ff17c0d3bc0 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.252+0000 7ff1836d9640 1 -- 172.21.15.40:0/3444058915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c0d3bc0 msgr2=0x7ff17c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.252+0000 7ff1836d9640 1 --2- 172.21.15.40:0/3444058915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c0d3bc0 0x7ff17c0d2cb0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7ff174009e50 tx=0x7ff17402a760 comp rx=0 tx=0).stop 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.253+0000 7ff1836d9640 1 -- 172.21.15.40:0/3444058915 shutdown_connections 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.253+0000 7ff1836d9640 1 --2- 172.21.15.40:0/3444058915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c0d3bc0 0x7ff17c0d2cb0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.253+0000 7ff1836d9640 1 --2- 172.21.15.40:0/3444058915 >> v2:172.21.15.184:3300/0 conn(0x7ff17c15f550 0x7ff17c0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.253+0000 7ff1836d9640 1 -- 172.21.15.40:0/3444058915 >> 172.21.15.40:0/3444058915 conn(0x7ff17c0d4f60 msgr2=0x7ff17c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.253+0000 7ff1836d9640 1 -- 172.21.15.40:0/3444058915 shutdown_connections 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.253+0000 7ff1836d9640 1 -- 172.21.15.40:0/3444058915 wait complete. 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.254+0000 7ff1836d9640 1 Processor -- start 2024-04-26T21:46:22.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.254+0000 7ff1836d9640 1 -- start start 2024-04-26T21:46:22.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.254+0000 7ff1836d9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff17c0d3bc0 0x7ff17c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.254+0000 7ff1836d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c15f170 0x7ff17c0a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff1836d9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff17c0a2620 0x7ff17c0b99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff1836d9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c0ca5c0 con 0x7ff17c15f170 2024-04-26T21:46:22.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff1836d9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c0ca440 con 0x7ff17c0d3bc0 2024-04-26T21:46:22.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff1836d9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff17c0ca740 con 0x7ff17c0a2620 2024-04-26T21:46:22.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c15f170 0x7ff17c0a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c15f170 0x7ff17c0a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44368/0 (socket says 172.21.15.40:44368) 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 -- 172.21.15.40:0/1847771361 learned_addr learned my addr 172.21.15.40:0/1847771361 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff181c76640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff17c0a2620 0x7ff17c0b99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff181475640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff17c0d3bc0 0x7ff17c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 -- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff17c0a2620 msgr2=0x7ff17c0b99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff17c0a2620 0x7ff17c0b99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 -- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff17c0d3bc0 msgr2=0x7ff17c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff17c0d3bc0 0x7ff17c150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 -- 172.21.15.40:0/1847771361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c0ba750 con 0x7ff17c15f170 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff181475640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff17c0d3bc0 0x7ff17c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff180c74640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c15f170 0x7ff17c0a20e0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e870 tx=0x7ff16c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff171ffb640 1 -- 172.21.15.40:0/1847771361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c017070 con 0x7ff17c15f170 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff171ffb640 1 -- 172.21.15.40:0/1847771361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c00ee60 con 0x7ff17c15f170 2024-04-26T21:46:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff171ffb640 1 -- 172.21.15.40:0/1847771361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00cb00 con 0x7ff17c15f170 2024-04-26T21:46:22.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.255+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c151470 con 0x7ff17c15f170 2024-04-26T21:46:22.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.256+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c151a30 con 0x7ff17c15f170 2024-04-26T21:46:22.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.256+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c0a2a30 con 0x7ff17c15f170 2024-04-26T21:46:22.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.257+0000 7ff171ffb640 1 -- 172.21.15.40:0/1847771361 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff16c01a070 con 0x7ff17c15f170 2024-04-26T21:46:22.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.257+0000 7ff171ffb640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff16006c920 0x7ff16006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.257+0000 7ff171ffb640 1 -- 172.21.15.40:0/1847771361 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9227+0+0 (secure 0 0 0) 0x7ff16c08cdf0 con 0x7ff17c15f170 2024-04-26T21:46:22.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.258+0000 7ff181475640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff16006c920 0x7ff16006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.258+0000 7ff181475640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff16006c920 0x7ff16006ede0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff178001680 tx=0x7ff178009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.262+0000 7ff171ffb640 1 -- 172.21.15.40:0/1847771361 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff17c0a2a30 con 0x7ff17c15f170 2024-04-26T21:46:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.486+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff17c069c90 con 0x7ff17c15f170 2024-04-26T21:46:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.735+0000 7ff171ffb640 1 -- 172.21.15.40:0/1847771361 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v313) ==== 162+0+0 (secure 0 0 0) 0x7ff16c05a410 con 0x7ff17c15f170 2024-04-26T21:46:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:46:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff16006c920 msgr2=0x7ff16006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff16006c920 0x7ff16006ede0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff178001680 tx=0x7ff178009040 comp rx=0 tx=0).stop 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c15f170 msgr2=0x7ff17c0a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c15f170 0x7ff17c0a20e0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e870 tx=0x7ff16c00ed30 comp rx=0 tx=0).stop 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 shutdown_connections 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff16006c920 0x7ff16006ede0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff17c0a2620 0x7ff17c0b99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff17c15f170 0x7ff17c0a20e0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 --2- 172.21.15.40:0/1847771361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff17c0d3bc0 0x7ff17c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 >> 172.21.15.40:0/1847771361 conn(0x7ff17c0d4f60 msgr2=0x7ff17c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 shutdown_connections 2024-04-26T21:46:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.737+0000 7ff1836d9640 1 -- 172.21.15.40:0/1847771361 wait complete. 2024-04-26T21:46:22.749 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:22.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.850+0000 7f50093e7640 1 Processor -- start 2024-04-26T21:46:22.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.850+0000 7f50093e7640 1 -- start start 2024-04-26T21:46:22.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.851+0000 7f50093e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f500415f3a0 0x7f50040d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.851+0000 7f50093e7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f50040d1770 0x7f50040d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.851+0000 7f50093e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50040d25a0 con 0x7f50040d2130 2024-04-26T21:46:22.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.851+0000 7f50093e7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f50040d0e20 con 0x7f50040d1770 2024-04-26T21:46:22.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.851+0000 7f50093e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50040b1d70 con 0x7f500415f3a0 2024-04-26T21:46:22.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f50027fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f50040d1770 0x7f50040d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f500415f3a0 0x7f50040d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f50027fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f50040d1770 0x7f50040d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:38390/0 (socket says 172.21.15.40:38390) 2024-04-26T21:46:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f50027fc640 1 -- 172.21.15.40:0/3151843928 learned_addr learned my addr 172.21.15.40:0/3151843928 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f50027fc640 1 -- 172.21.15.40:0/3151843928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f500415f3a0 msgr2=0x7f50040d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f50027fc640 1 --2- 172.21.15.40:0/3151843928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f500415f3a0 0x7f50040d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f50027fc640 1 -- 172.21.15.40:0/3151843928 >> v1:172.21.15.184:6789/0 conn(0x7f50040d2130 legacy=0x7f50040d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f5001ffb640 1 -- 172.21.15.40:0/3151843928 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104103215 0 0) 0x7f4ff4001110 con 0x7f50040d2130 2024-04-26T21:46:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f50027fc640 1 -- 172.21.15.40:0/3151843928 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50040b2050 con 0x7f50040d1770 2024-04-26T21:46:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.852+0000 7f5002ffd640 1 --2- 172.21.15.40:0/3151843928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f500415f3a0 0x7f50040d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f50027fc640 1 --2- 172.21.15.40:0/3151843928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50040d1770 0x7f50040d1bf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009870 tx=0x7f4fec02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8ee76d8dbdb41c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f5001ffb640 1 -- 172.21.15.40:0/3151843928 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec037070 con 0x7f50040d1770 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f5001ffb640 1 -- 172.21.15.40:0/3151843928 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fec032440 con 0x7f50040d1770 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f5001ffb640 1 -- 172.21.15.40:0/3151843928 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec032810 con 0x7f50040d1770 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f50093e7640 1 -- 172.21.15.40:0/3151843928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50040d1770 msgr2=0x7f50040d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f50093e7640 1 --2- 172.21.15.40:0/3151843928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50040d1770 0x7f50040d1bf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009870 tx=0x7f4fec02a250 comp rx=0 tx=0).stop 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f50093e7640 1 -- 172.21.15.40:0/3151843928 shutdown_connections 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f50093e7640 1 --2- 172.21.15.40:0/3151843928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50040d1770 0x7f50040d1bf0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f50093e7640 1 --2- 172.21.15.40:0/3151843928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f500415f3a0 0x7f50040d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.853+0000 7f50093e7640 1 -- 172.21.15.40:0/3151843928 >> 172.21.15.40:0/3151843928 conn(0x7f50040d4f90 msgr2=0x7f50040d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.854+0000 7f50093e7640 1 -- 172.21.15.40:0/3151843928 shutdown_connections 2024-04-26T21:46:22.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.854+0000 7f50093e7640 1 -- 172.21.15.40:0/3151843928 wait complete. 2024-04-26T21:46:22.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.855+0000 7f50093e7640 1 Processor -- start 2024-04-26T21:46:22.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.855+0000 7f50093e7640 1 -- start start 2024-04-26T21:46:22.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50093e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50040d1770 0x7f5004157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50093e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50040d2130 0x7f5004158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50093e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500415f3a0 0x7f50040b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50093e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f500415f900 con 0x7f50040d2130 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50093e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f500415f780 con 0x7f500415f3a0 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50093e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50040b3860 con 0x7f50040d1770 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50040d2130 0x7f5004158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50040d2130 0x7f5004158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44374/0 (socket says 172.21.15.40:44374) 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 -- 172.21.15.40:0/3034911017 learned_addr learned my addr 172.21.15.40:0/3034911017 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f5002ffd640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50040d1770 0x7f5004157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50037fe640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500415f3a0 0x7f50040b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 -- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50040d1770 msgr2=0x7f5004157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50040d1770 0x7f5004157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 -- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500415f3a0 msgr2=0x7f50040b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500415f3a0 0x7f50040b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 -- 172.21.15.40:0/3034911017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5004158ad0 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f5002ffd640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50040d1770 0x7f5004157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50027fc640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50040d2130 0x7f5004158080 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f4fec035040 tx=0x7f4fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f4ffb7fe640 1 -- 172.21.15.40:0/3034911017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec037040 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f4ffb7fe640 1 -- 172.21.15.40:0/3034911017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fec004230 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5004158db0 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.856+0000 7f4ffb7fe640 1 -- 172.21.15.40:0/3034911017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec003610 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.857+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5004159370 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.857+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f500406e970 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.858+0000 7f4ffb7fe640 1 -- 172.21.15.40:0/3034911017 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fec0037b0 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.858+0000 7f4ffb7fe640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fe406c8f0 0x7f4fe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.858+0000 7f4ffb7fe640 1 -- 172.21.15.40:0/3034911017 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9227+0+0 (secure 0 0 0) 0x7f4fec0ad670 con 0x7f50040d2130 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.859+0000 7f5002ffd640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fe406c8f0 0x7f4fe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.859+0000 7f5002ffd640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fe406c8f0 0x7f4fe406edb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f50040d0170 tx=0x7f4ffc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:22.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:22.863+0000 7f4ffb7fe640 1 -- 172.21.15.40:0/3034911017 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f500406e970 con 0x7f50040d2130 2024-04-26T21:46:23.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.089+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50040b6860 con 0x7f50040d2130 2024-04-26T21:46:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.089+0000 7f4ffb7fe640 1 -- 172.21.15.40:0/3034911017 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 314 v314) ==== 95+0+5193 (secure 0 0 0) 0x7f4fec07ac90 con 0x7f50040d2130 2024-04-26T21:46:23.089 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:23.089 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":314,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":314,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:22.738113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11196":{"gid":11196,"name":"c","rank":0,"incarnation":314,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/1812066725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1812066725},{"type":"v1","addr":"172.21.15.40:6833","nonce":1812066725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 314 2024-04-26T21:46:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fe406c8f0 msgr2=0x7f4fe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fe406c8f0 0x7f4fe406edb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f50040d0170 tx=0x7f4ffc00b040 comp rx=0 tx=0).stop 2024-04-26T21:46:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50040d2130 msgr2=0x7f5004158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50040d2130 0x7f5004158080 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f4fec035040 tx=0x7f4fec0046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 shutdown_connections 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4fe406c8f0 0x7f4fe406edb0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500415f3a0 0x7f50040b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50040d2130 0x7f5004158080 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 --2- 172.21.15.40:0/3034911017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50040d1770 0x7f5004157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 >> 172.21.15.40:0/3034911017 conn(0x7f50040d4f90 msgr2=0x7f50040d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 shutdown_connections 2024-04-26T21:46:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:23.091+0000 7f50093e7640 1 -- 172.21.15.40:0/3034911017 wait complete. 2024-04-26T21:46:23.102 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:46:25.764 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:25.764+0000 7fee41f19640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-26T21:46:25.810 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:25.764+0000 7fee41f19640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-04-26T21:46:25.811 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:25.764+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:46:25.811 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:25.764+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:46:25.811 INFO:tasks.ceph.mds.c.smithi040.stderr: -17> 2024-04-26T21:46:25.764+0000 7fee41f19640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-26T21:46:25.811 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:46:25.764+0000 7fee41f19640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-04-26T21:46:25.811 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:46:28.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.216+0000 7fda56c2a640 1 Processor -- start 2024-04-26T21:46:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.216+0000 7fda56c2a640 1 -- start start 2024-04-26T21:46:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda56c2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015ab60 0x7fda5006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda56c2a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fda5015dd00 0x7fda5006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda56c2a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5015af40 con 0x7fda501617f0 2024-04-26T21:46:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda56c2a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fda5015e320 con 0x7fda5015dd00 2024-04-26T21:46:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda56c2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda50160d10 con 0x7fda5015ab60 2024-04-26T21:46:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda549c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015ab60 0x7fda5006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda551c7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fda5015dd00 0x7fda5006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda549c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015ab60 0x7fda5006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:45650/0 (socket says 172.21.15.40:45650) 2024-04-26T21:46:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda4ffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fda501617f0 0x7fda5015d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:38504/0 (socket says 172.21.15.40:38504) 2024-04-26T21:46:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.217+0000 7fda549c6640 1 -- 172.21.15.40:0/3292509237 learned_addr learned my addr 172.21.15.40:0/3292509237 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.218+0000 7fda551c7640 1 -- 172.21.15.40:0/3292509237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015ab60 msgr2=0x7fda5006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.218+0000 7fda551c7640 1 --2- 172.21.15.40:0/3292509237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015ab60 0x7fda5006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.218+0000 7fda551c7640 1 -- 172.21.15.40:0/3292509237 >> v1:172.21.15.184:6789/0 conn(0x7fda501617f0 legacy=0x7fda5015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.218+0000 7fda551c7640 1 -- 172.21.15.40:0/3292509237 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda50160ff0 con 0x7fda5015dd00 2024-04-26T21:46:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.218+0000 7fda549c6640 1 --2- 172.21.15.40:0/3292509237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015ab60 0x7fda5006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:28.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.218+0000 7fda551c7640 1 --2- 172.21.15.40:0/3292509237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5015dd00 0x7fda5006c780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fda48009d90 tx=0x7fda4802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2690f935559bf55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda4f7fe640 1 -- 172.21.15.40:0/3292509237 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48007740 con 0x7fda5015dd00 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda4f7fe640 1 -- 172.21.15.40:0/3292509237 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda480078e0 con 0x7fda5015dd00 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda4f7fe640 1 -- 172.21.15.40:0/3292509237 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48007cb0 con 0x7fda5015dd00 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda56c2a640 1 -- 172.21.15.40:0/3292509237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5015dd00 msgr2=0x7fda5006c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda56c2a640 1 --2- 172.21.15.40:0/3292509237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5015dd00 0x7fda5006c780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fda48009d90 tx=0x7fda4802a660 comp rx=0 tx=0).stop 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda56c2a640 1 -- 172.21.15.40:0/3292509237 shutdown_connections 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda56c2a640 1 --2- 172.21.15.40:0/3292509237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5015dd00 0x7fda5006c780 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda56c2a640 1 --2- 172.21.15.40:0/3292509237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015ab60 0x7fda5006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda56c2a640 1 -- 172.21.15.40:0/3292509237 >> 172.21.15.40:0/3292509237 conn(0x7fda500d1cf0 msgr2=0x7fda500d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda56c2a640 1 -- 172.21.15.40:0/3292509237 shutdown_connections 2024-04-26T21:46:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.219+0000 7fda56c2a640 1 -- 172.21.15.40:0/3292509237 wait complete. 2024-04-26T21:46:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.221+0000 7fda56c2a640 1 Processor -- start 2024-04-26T21:46:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.221+0000 7fda56c2a640 1 -- start start 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.221+0000 7fda56c2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5015ab60 0x7fda500d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.221+0000 7fda56c2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015dd00 0x7fda500b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.221+0000 7fda56c2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda501617f0 0x7fda501f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.221+0000 7fda56c2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda500cc4c0 con 0x7fda5015ab60 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.221+0000 7fda56c2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda500cc340 con 0x7fda501617f0 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.221+0000 7fda56c2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda500cc640 con 0x7fda5015dd00 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5015ab60 0x7fda500d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5015ab60 0x7fda500d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44382/0 (socket says 172.21.15.40:44382) 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 -- 172.21.15.40:0/4223819847 learned_addr learned my addr 172.21.15.40:0/4223819847 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda4ffff640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015dd00 0x7fda500b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda551c7640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda501617f0 0x7fda501f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 -- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015dd00 msgr2=0x7fda500b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015dd00 0x7fda500b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 -- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda501617f0 msgr2=0x7fda501f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda501617f0 0x7fda501f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 -- 172.21.15.40:0/4223819847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda500d09b0 con 0x7fda5015ab60 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda549c6640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5015ab60 0x7fda500d02d0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9d0 tx=0x7fda4000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda4d7fa640 1 -- 172.21.15.40:0/4223819847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40017070 con 0x7fda5015ab60 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda4d7fa640 1 -- 172.21.15.40:0/4223819847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4000ca60 con 0x7fda5015ab60 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda500d0c90 con 0x7fda5015ab60 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.222+0000 7fda4d7fa640 1 -- 172.21.15.40:0/4223819847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4000ce10 con 0x7fda5015ab60 2024-04-26T21:46:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.223+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda500d1250 con 0x7fda5015ab60 2024-04-26T21:46:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.223+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda500694f0 con 0x7fda5015ab60 2024-04-26T21:46:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.224+0000 7fda4d7fa640 1 -- 172.21.15.40:0/4223819847 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda4001a070 con 0x7fda5015ab60 2024-04-26T21:46:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.224+0000 7fda4d7fa640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda3c06c8f0 0x7fda3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.224+0000 7fda4d7fa640 1 -- 172.21.15.40:0/4223819847 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7fda4008cc50 con 0x7fda5015ab60 2024-04-26T21:46:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.225+0000 7fda4ffff640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda3c06c8f0 0x7fda3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.225+0000 7fda4ffff640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda3c06c8f0 0x7fda3c06edb0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fda500b1ea0 tx=0x7fda44007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.229+0000 7fda4d7fa640 1 -- 172.21.15.40:0/4223819847 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda40092020 con 0x7fda5015ab60 2024-04-26T21:46:28.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.454+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda50073c00 con 0x7fda5015ab60 2024-04-26T21:46:28.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.454+0000 7fda4d7fa640 1 -- 172.21.15.40:0/4223819847 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 319 v319) ==== 95+0+5182 (secure 0 0 0) 0x7fda40059bc0 con 0x7fda5015ab60 2024-04-26T21:46:28.456 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:28.456 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":319,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1769689018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1769689018},{"type":"v1","addr":"172.21.15.40:6833","nonce":1769689018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":319}],"filesystems":[{"mdsmap":{"epoch":318,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:25.766122+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:28.458 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 319 2024-04-26T21:46:28.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda3c06c8f0 msgr2=0x7fda3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda3c06c8f0 0x7fda3c06edb0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fda500b1ea0 tx=0x7fda44007e80 comp rx=0 tx=0).stop 2024-04-26T21:46:28.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5015ab60 msgr2=0x7fda500d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5015ab60 0x7fda500d02d0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9d0 tx=0x7fda4000ee90 comp rx=0 tx=0).stop 2024-04-26T21:46:28.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 shutdown_connections 2024-04-26T21:46:28.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fda3c06c8f0 0x7fda3c06edb0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda501617f0 0x7fda501f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda5015dd00 0x7fda500b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 --2- 172.21.15.40:0/4223819847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5015ab60 0x7fda500d02d0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 >> 172.21.15.40:0/4223819847 conn(0x7fda500d1cf0 msgr2=0x7fda500b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:28.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 shutdown_connections 2024-04-26T21:46:28.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.456+0000 7fda56c2a640 1 -- 172.21.15.40:0/4223819847 wait complete. 2024-04-26T21:46:28.468 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:46:28.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:28.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.568+0000 7fc42c3bf640 1 Processor -- start 2024-04-26T21:46:28.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.568+0000 7fc42c3bf640 1 -- start start 2024-04-26T21:46:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42c3bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc424162810 0x7fc424070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42c3bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4240b2340 0x7fc4240cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42c3bf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4240d10d0 con 0x7fc4240cd930 2024-04-26T21:46:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42c3bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc4240d0750 con 0x7fc424162810 2024-04-26T21:46:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42c3bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4240d16a0 con 0x7fc4240b2340 2024-04-26T21:46:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42a95c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc4240cd930 0x7fc4240d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46144/0 (socket says 172.21.15.40:46144) 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42a95c640 1 -- 172.21.15.40:0/3841967596 learned_addr learned my addr 172.21.15.40:0/3841967596 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42995a640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4240b2340 0x7fc4240cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 724444233 0 0) 0x7fc4240d10d0 con 0x7fc4240cd930 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc42a15b640 1 --2- 172.21.15.40:0/3841967596 >> v2:172.21.15.184:3300/0 conn(0x7fc424162810 0x7fc424070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc410003520 con 0x7fc4240cd930 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 893170991 0 0) 0x7fc410003520 con 0x7fc4240cd930 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4240b2340 msgr2=0x7fc4240cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4240b2340 0x7fc4240cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 >> v2:172.21.15.184:3300/0 conn(0x7fc424162810 msgr2=0x7fc424070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 --2- 172.21.15.40:0/3841967596 >> v2:172.21.15.184:3300/0 conn(0x7fc424162810 0x7fc424070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4240d1370 con 0x7fc4240cd930 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fc41c0019f0 con 0x7fc4240cd930 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 >> v1:172.21.15.40:6789/0 conn(0x7fc4240cd930 legacy=0x7fc4240d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4100039b0 0x7fc410003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4240b2340 0x7fc410004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc4100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240d0750 con 0x7fc410004d40 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240d16a0 con 0x7fc4100039b0 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.569+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc41000b9c0 con 0x7fc4240b2340 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.570+0000 7fc42a95c640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc4100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.570+0000 7fc42a15b640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4100039b0 0x7fc410003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.570+0000 7fc42995a640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4240b2340 0x7fc410004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.570+0000 7fc42a95c640 1 -- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4240b2340 msgr2=0x7fc410004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.570+0000 7fc42a95c640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4240b2340 0x7fc410004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.570+0000 7fc42a95c640 1 -- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4100039b0 msgr2=0x7fc410003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.570+0000 7fc42a95c640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4100039b0 0x7fc410003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.570+0000 7fc42a95c640 1 -- 172.21.15.40:0/3841967596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc410003520 con 0x7fc410004d40 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc42a95c640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc4100051c0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fc424076320 tx=0x7fc410005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60deff861d38d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00a4c0 con 0x7fc410004d40 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41c02f070 con 0x7fc410004d40 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc429159640 1 -- 172.21.15.40:0/3841967596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c02bc70 con 0x7fc410004d40 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc42c3bf640 1 -- 172.21.15.40:0/3841967596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 msgr2=0x7fc4100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc42c3bf640 1 --2- 172.21.15.40:0/3841967596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc4100051c0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fc424076320 tx=0x7fc410005830 comp rx=0 tx=0).stop 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc42a95c640 1 -- 172.21.15.40:0/3841967596 reap_dead start 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc42c3bf640 1 -- 172.21.15.40:0/3841967596 shutdown_connections 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc42c3bf640 1 -- 172.21.15.40:0/3841967596 >> 172.21.15.40:0/3841967596 conn(0x7fc4240d2da0 msgr2=0x7fc4240d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc42c3bf640 1 -- 172.21.15.40:0/3841967596 shutdown_connections 2024-04-26T21:46:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.571+0000 7fc42c3bf640 1 -- 172.21.15.40:0/3841967596 wait complete. 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42c3bf640 1 Processor -- start 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42c3bf640 1 -- start start 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42c3bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc424161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42c3bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc424161840 0x7fc42415d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42c3bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42415d750 0x7fc424155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42c3bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc424070890 con 0x7fc410004d40 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42c3bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240d0470 con 0x7fc42415d750 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42c3bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4240d2240 con 0x7fc424161840 2024-04-26T21:46:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42a15b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc424161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42a15b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc424161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44392/0 (socket says 172.21.15.40:44392) 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.573+0000 7fc42a15b640 1 -- 172.21.15.40:0/4089619370 learned_addr learned my addr 172.21.15.40:0/4089619370 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42995a640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc424161840 0x7fc42415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42a95c640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42415d750 0x7fc424155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42a15b640 1 -- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc424161840 msgr2=0x7fc42415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42a15b640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc424161840 0x7fc42415d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42a15b640 1 -- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42415d750 msgr2=0x7fc424155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42a15b640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42415d750 0x7fc424155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42a15b640 1 -- 172.21.15.40:0/4089619370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc424160260 con 0x7fc410004d40 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42a95c640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42415d750 0x7fc424155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42a15b640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc424161300 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fc420011960 tx=0x7fc420011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc41affd640 1 -- 172.21.15.40:0/4089619370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42000f9d0 con 0x7fc410004d40 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc41affd640 1 -- 172.21.15.40:0/4089619370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42000fb70 con 0x7fc410004d40 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc424160540 con 0x7fc410004d40 2024-04-26T21:46:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc41affd640 1 -- 172.21.15.40:0/4089619370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420014440 con 0x7fc410004d40 2024-04-26T21:46:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.574+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc424160b00 con 0x7fc410004d40 2024-04-26T21:46:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.575+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42415db60 con 0x7fc410004d40 2024-04-26T21:46:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.576+0000 7fc41affd640 1 -- 172.21.15.40:0/4089619370 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc420012040 con 0x7fc410004d40 2024-04-26T21:46:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.576+0000 7fc41affd640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc40806c920 0x7fc40806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.576+0000 7fc41affd640 1 -- 172.21.15.40:0/4089619370 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7fc42008cfc0 con 0x7fc410004d40 2024-04-26T21:46:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.576+0000 7fc42995a640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc40806c920 0x7fc40806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.577+0000 7fc42995a640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc40806c920 0x7fc40806ede0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc42406e8c0 tx=0x7fc414009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.580+0000 7fc41affd640 1 -- 172.21.15.40:0/4089619370 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc42415db60 con 0x7fc410004d40 2024-04-26T21:46:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.809+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4240694f0 con 0x7fc410004d40 2024-04-26T21:46:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.810+0000 7fc41affd640 1 -- 172.21.15.40:0/4089619370 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 319 v319) ==== 95+0+5182 (secure 0 0 0) 0x7fc420019030 con 0x7fc410004d40 2024-04-26T21:46:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:28.811 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":319,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1769689018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1769689018},{"type":"v1","addr":"172.21.15.40:6833","nonce":1769689018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":319}],"filesystems":[{"mdsmap":{"epoch":318,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:25.766122+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 319 2024-04-26T21:46:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc40806c920 msgr2=0x7fc40806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc40806c920 0x7fc40806ede0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc42406e8c0 tx=0x7fc414009190 comp rx=0 tx=0).stop 2024-04-26T21:46:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 msgr2=0x7fc424161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc424161300 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fc420011960 tx=0x7fc420011d60 comp rx=0 tx=0).stop 2024-04-26T21:46:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 shutdown_connections 2024-04-26T21:46:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc40806c920 0x7fc40806ede0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42415d750 0x7fc424155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc424161840 0x7fc42415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 --2- 172.21.15.40:0/4089619370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc410004d40 0x7fc424161300 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 >> 172.21.15.40:0/4089619370 conn(0x7fc4240d2da0 msgr2=0x7fc424074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 shutdown_connections 2024-04-26T21:46:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.812+0000 7fc42c3bf640 1 -- 172.21.15.40:0/4089619370 wait complete. 2024-04-26T21:46:28.823 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:46:28.824 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.924+0000 7fbb109e7640 1 Processor -- start 2024-04-26T21:46:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.924+0000 7fbb109e7640 1 -- start start 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.924+0000 7fbb109e7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbb0c06e270 0x7fbb0c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.924+0000 7fbb109e7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb0c06d560 0x7fbb0c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.924+0000 7fbb109e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0c0d2420 con 0x7fbb0c0d0070 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.924+0000 7fbb109e7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0c162d00 con 0x7fbb0c06d560 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.924+0000 7fbb109e7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0c0d2d10 con 0x7fbb0c06e270 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb0ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb0c06d560 0x7fbb0c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb0ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb0c06d560 0x7fbb0c0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40114/0 (socket says 172.21.15.40:40114) 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb0ad9d640 1 -- 172.21.15.40:0/2224308406 learned_addr learned my addr 172.21.15.40:0/2224308406 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb03fff640 1 --2- 172.21.15.40:0/2224308406 >> v2:172.21.15.184:3300/0 conn(0x7fbb0c06e270 0x7fbb0c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb0ad9d640 1 -- 172.21.15.40:0/2224308406 >> v2:172.21.15.184:3300/0 conn(0x7fbb0c06e270 msgr2=0x7fbb0c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb0ad9d640 1 --2- 172.21.15.40:0/2224308406 >> v2:172.21.15.184:3300/0 conn(0x7fbb0c06e270 0x7fbb0c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb0ad9d640 1 -- 172.21.15.40:0/2224308406 >> v1:172.21.15.184:6789/0 conn(0x7fbb0c0d0070 legacy=0x7fbb0c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:28.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb0ad9d640 1 -- 172.21.15.40:0/2224308406 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c0cd6b0 con 0x7fbb0c06d560 2024-04-26T21:46:28.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.925+0000 7fbb03fff640 1 --2- 172.21.15.40:0/2224308406 >> v2:172.21.15.184:3300/0 conn(0x7fbb0c06e270 0x7fbb0c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:28.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb0ad9d640 1 --2- 172.21.15.40:0/2224308406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c0ce190 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c073120 tx=0x7fbb0402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5eaba82934e6f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb09d9b640 1 -- 172.21.15.40:0/2224308406 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04036070 con 0x7fbb0c06d560 2024-04-26T21:46:28.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb09d9b640 1 -- 172.21.15.40:0/2224308406 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb04031600 con 0x7fbb0c06d560 2024-04-26T21:46:28.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb09d9b640 1 -- 172.21.15.40:0/2224308406 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040319d0 con 0x7fbb0c06d560 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb109e7640 1 -- 172.21.15.40:0/2224308406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 msgr2=0x7fbb0c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb109e7640 1 --2- 172.21.15.40:0/2224308406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c0ce190 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c073120 tx=0x7fbb0402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb109e7640 1 -- 172.21.15.40:0/2224308406 shutdown_connections 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb109e7640 1 --2- 172.21.15.40:0/2224308406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c0ce190 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb109e7640 1 --2- 172.21.15.40:0/2224308406 >> v2:172.21.15.184:3300/0 conn(0x7fbb0c06e270 0x7fbb0c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb109e7640 1 -- 172.21.15.40:0/2224308406 >> 172.21.15.40:0/2224308406 conn(0x7fbb0c0d2ec0 msgr2=0x7fbb0c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb109e7640 1 -- 172.21.15.40:0/2224308406 shutdown_connections 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.926+0000 7fbb109e7640 1 -- 172.21.15.40:0/2224308406 wait complete. 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.928+0000 7fbb109e7640 1 Processor -- start 2024-04-26T21:46:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.928+0000 7fbb109e7640 1 -- start start 2024-04-26T21:46:28.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.928+0000 7fbb109e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.928+0000 7fbb109e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0c06e270 0x7fbb0c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.928+0000 7fbb109e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb0c0d0070 0x7fbb0c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.928+0000 7fbb109e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c0ce6d0 con 0x7fbb0c06d560 2024-04-26T21:46:28.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.928+0000 7fbb109e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c0cdc70 con 0x7fbb0c06e270 2024-04-26T21:46:28.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.928+0000 7fbb109e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0c0ce850 con 0x7fbb0c0d0070 2024-04-26T21:46:28.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c14e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40128/0 (socket says 172.21.15.40:40128) 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 -- 172.21.15.40:0/3146261631 learned_addr learned my addr 172.21.15.40:0/3146261631 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0ad9d640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb0c0d0070 0x7fbb0c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb03fff640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0c06e270 0x7fbb0c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 -- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb0c0d0070 msgr2=0x7fbb0c15e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb0c0d0070 0x7fbb0c15e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 -- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0c06e270 msgr2=0x7fbb0c0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0c06e270 0x7fbb0c0c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 -- 172.21.15.40:0/3146261631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c15cfe0 con 0x7fbb0c06d560 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb03fff640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0c06e270 0x7fbb0c0c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb0a59c640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c14e3a0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00b420 tx=0x7fbafc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3146261631 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc011830 con 0x7fbb0c06d560 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3146261631 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc016070 con 0x7fbb0c06d560 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3146261631 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc011c10 con 0x7fbb0c06d560 2024-04-26T21:46:28.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c15d280 con 0x7fbb0c06d560 2024-04-26T21:46:28.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.929+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c15d840 con 0x7fbb0c06d560 2024-04-26T21:46:28.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.930+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c076180 con 0x7fbb0c06d560 2024-04-26T21:46:28.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.931+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3146261631 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbafc01b460 con 0x7fbb0c06d560 2024-04-26T21:46:28.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.931+0000 7fbb02ffd640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaec06c8f0 0x7fbaec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:28.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.931+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3146261631 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7fbafc08c450 con 0x7fbb0c06d560 2024-04-26T21:46:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.932+0000 7fbb03fff640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaec06c8f0 0x7fbaec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.932+0000 7fbb03fff640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaec06c8f0 0x7fbaec06edb0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0b3e80 tx=0x7fbaf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:28.936+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3146261631 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb0c076180 con 0x7fbb0c06d560 2024-04-26T21:46:29.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.159+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb0c0d0470 con 0x7fbb0c06d560 2024-04-26T21:46:29.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.160+0000 7fbb02ffd640 1 -- 172.21.15.40:0/3146261631 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 319 v319) ==== 95+0+5182 (secure 0 0 0) 0x7fbafc020020 con 0x7fbb0c06d560 2024-04-26T21:46:29.161 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:29.161 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":319,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1769689018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1769689018},{"type":"v1","addr":"172.21.15.40:6833","nonce":1769689018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":319}],"filesystems":[{"mdsmap":{"epoch":318,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:25.766122+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:29.161 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 319 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaec06c8f0 msgr2=0x7fbaec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaec06c8f0 0x7fbaec06edb0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0b3e80 tx=0x7fbaf4008040 comp rx=0 tx=0).stop 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 msgr2=0x7fbb0c14e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c14e3a0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00b420 tx=0x7fbafc00b8e0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 shutdown_connections 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbaec06c8f0 0x7fbaec06edb0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb0c0d0070 0x7fbb0c15e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0c06e270 0x7fbb0c0c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 --2- 172.21.15.40:0/3146261631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb0c06d560 0x7fbb0c14e3a0 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 >> 172.21.15.40:0/3146261631 conn(0x7fbb0c0d2ec0 msgr2=0x7fbb0c0cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 shutdown_connections 2024-04-26T21:46:29.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.162+0000 7fbb109e7640 1 -- 172.21.15.40:0/3146261631 wait complete. 2024-04-26T21:46:29.174 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 600.00000000' led to DAMAGED state 2024-04-26T21:46:29.174 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_openfiles.0' 2024-04-26T21:46:29.174 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:29.174 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:29.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:46:29.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:46:29.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:29.190 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:46:29.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:46:29.244 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:46:29.245 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:46:29.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.385+0000 7fa9f0e64640 1 Processor -- start 2024-04-26T21:46:29.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.385+0000 7fa9f0e64640 1 -- start start 2024-04-26T21:46:29.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9f0e64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9ec073b70 0x7fa9ec0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9f0e64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa9ec0cbb50 0x7fa9ec0af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9f0e64640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ec072c00 con 0x7fa9ec15f380 2024-04-26T21:46:29.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9f0e64640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ec0729b0 con 0x7fa9ec073b70 2024-04-26T21:46:29.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9f0e64640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ec072ee0 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9e3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9ec073b70 0x7fa9ec0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9ea59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa9ec15f380 0x7fa9ec0cb440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39748/0 (socket says 172.21.15.40:39748) 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9ea59c640 1 -- 172.21.15.40:0/548831384 learned_addr learned my addr 172.21.15.40:0/548831384 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9e3fff640 1 -- 172.21.15.40:0/548831384 >> v2:172.21.15.184:3300/0 conn(0x7fa9ec0cbb50 msgr2=0x7fa9ec0af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9ead9d640 1 --2- 172.21.15.40:0/548831384 >> v2:172.21.15.184:3300/0 conn(0x7fa9ec0cbb50 0x7fa9ec0af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9e3fff640 1 --2- 172.21.15.40:0/548831384 >> v2:172.21.15.184:3300/0 conn(0x7fa9ec0cbb50 0x7fa9ec0af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9e3fff640 1 -- 172.21.15.40:0/548831384 >> v1:172.21.15.40:6789/0 conn(0x7fa9ec15f380 legacy=0x7fa9ec0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9e9d9b640 1 -- 172.21.15.40:0/548831384 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1038220989 0 0) 0x7fa9ec072c00 con 0x7fa9ec15f380 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9e3fff640 1 -- 172.21.15.40:0/548831384 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec0731c0 con 0x7fa9ec073b70 2024-04-26T21:46:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.386+0000 7fa9ead9d640 1 --2- 172.21.15.40:0/548831384 >> v2:172.21.15.184:3300/0 conn(0x7fa9ec0cbb50 0x7fa9ec0af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.387+0000 7fa9e3fff640 1 --2- 172.21.15.40:0/548831384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec073b70 0x7fa9ec0aecb0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0735d0 tx=0x7fa9d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2be0fb78d31d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.387+0000 7fa9e9d9b640 1 -- 172.21.15.40:0/548831384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4037070 con 0x7fa9ec073b70 2024-04-26T21:46:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.387+0000 7fa9e9d9b640 1 -- 172.21.15.40:0/548831384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d4032440 con 0x7fa9ec073b70 2024-04-26T21:46:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.387+0000 7fa9e9d9b640 1 -- 172.21.15.40:0/548831384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4032810 con 0x7fa9ec073b70 2024-04-26T21:46:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.388+0000 7fa9f0e64640 1 -- 172.21.15.40:0/548831384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec073b70 msgr2=0x7fa9ec0aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.388+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/548831384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec073b70 0x7fa9ec0aecb0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0735d0 tx=0x7fa9d402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.388+0000 7fa9f0e64640 1 -- 172.21.15.40:0/548831384 shutdown_connections 2024-04-26T21:46:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.388+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/548831384 >> v2:172.21.15.184:3300/0 conn(0x7fa9ec0cbb50 0x7fa9ec0af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.388+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/548831384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec073b70 0x7fa9ec0aecb0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.388+0000 7fa9f0e64640 1 -- 172.21.15.40:0/548831384 >> 172.21.15.40:0/548831384 conn(0x7fa9ec0d0be0 msgr2=0x7fa9ec0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.388+0000 7fa9f0e64640 1 -- 172.21.15.40:0/548831384 shutdown_connections 2024-04-26T21:46:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.388+0000 7fa9f0e64640 1 -- 172.21.15.40:0/548831384 wait complete. 2024-04-26T21:46:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.389+0000 7fa9f0e64640 1 Processor -- start 2024-04-26T21:46:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.389+0000 7fa9f0e64640 1 -- start start 2024-04-26T21:46:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9f0e64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec0cbb50 0x7fa9ec0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9f0e64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9ec15f380 0x7fa9ec1f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9f0e64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9ec0c6a10 0x7fa9ec06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9f0e64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec073620 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9f0e64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec0d08b0 con 0x7fa9ec15f380 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9f0e64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec0737a0 con 0x7fa9ec0c6a10 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec0cbb50 0x7fa9ec0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec0cbb50 0x7fa9ec0c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40160/0 (socket says 172.21.15.40:40160) 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 -- 172.21.15.40:0/2634651837 learned_addr learned my addr 172.21.15.40:0/2634651837 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 -- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9ec0c6a10 msgr2=0x7fa9ec06e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9ec0c6a10 0x7fa9ec06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 -- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9ec15f380 msgr2=0x7fa9ec1f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9e3fff640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9ec15f380 0x7fa9ec1f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9ec15f380 0x7fa9ec1f3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 -- 172.21.15.40:0/2634651837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec0c6fa0 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.390+0000 7fa9ea59c640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec0cbb50 0x7fa9ec0c5120 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c430 tx=0x7fa9dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.391+0000 7fa9e2ffd640 1 -- 172.21.15.40:0/2634651837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0074e0 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.391+0000 7fa9e2ffd640 1 -- 172.21.15.40:0/2634651837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc0047e0 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.391+0000 7fa9e2ffd640 1 -- 172.21.15.40:0/2634651837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004bd0 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.391+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec0c7280 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.391+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec0c7840 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.392+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ec0c7a80 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.392+0000 7fa9e2ffd640 1 -- 172.21.15.40:0/2634651837 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9dc011070 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.392+0000 7fa9e2ffd640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.392+0000 7fa9e2ffd640 1 -- 172.21.15.40:0/2634651837 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7fa9dc08c610 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.393+0000 7fa9e3fff640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.393+0000 7fa9e3fff640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4034000 tx=0x7fa9d402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.397+0000 7fa9e2ffd640 1 -- 172.21.15.40:0/2634651837 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9ec0c7a80 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.619+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa9ec159e00 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.784+0000 7fa9e2ffd640 1 -- 172.21.15.40:0/2634651837 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v320) ==== 161+0+0 (secure 0 0 0) 0x7fa9dc059c60 con 0x7fa9ec0cbb50 2024-04-26T21:46:29.785 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9cc06c8f0 msgr2=0x7fa9cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4034000 tx=0x7fa9d402e000 comp rx=0 tx=0).stop 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec0cbb50 msgr2=0x7fa9ec0c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec0cbb50 0x7fa9ec0c5120 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c430 tx=0x7fa9dc00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 shutdown_connections 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9cc06c8f0 0x7fa9cc06edb0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9ec0c6a10 0x7fa9ec06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9ec15f380 0x7fa9ec1f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 --2- 172.21.15.40:0/2634651837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec0cbb50 0x7fa9ec0c5120 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 >> 172.21.15.40:0/2634651837 conn(0x7fa9ec0d0be0 msgr2=0x7fa9ec0afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 shutdown_connections 2024-04-26T21:46:29.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.786+0000 7fa9f0e64640 1 -- 172.21.15.40:0/2634651837 wait complete. 2024-04-26T21:46:29.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:46:29.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.900+0000 7fa129b03640 1 Processor -- start 2024-04-26T21:46:29.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.900+0000 7fa129b03640 1 -- start start 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa129b03640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1240d1b30 0x7fa1240d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa129b03640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1240d10d0 con 0x7fa124162810 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa129b03640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1240d1370 con 0x7fa1240b2340 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa129b03640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa1240d0750 con 0x7fa1240d1b30 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa123fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1240d1b30 0x7fa1240d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa1237fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa124162810 0x7fa124070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39758/0 (socket says 172.21.15.40:39758) 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa1237fe640 1 -- 172.21.15.40:0/2508744926 learned_addr learned my addr 172.21.15.40:0/2508744926 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa123fff640 1 -- 172.21.15.40:0/2508744926 >> v1:172.21.15.184:6789/0 conn(0x7fa1240b2340 legacy=0x7fa1240cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa123fff640 1 -- 172.21.15.40:0/2508744926 >> v1:172.21.15.40:6789/0 conn(0x7fa124162810 legacy=0x7fa124070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa123fff640 1 -- 172.21.15.40:0/2508744926 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1240d01b0 con 0x7fa1240d1b30 2024-04-26T21:46:29.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.901+0000 7fa1227fc640 1 -- 172.21.15.40:0/2508744926 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2319656294 0 0) 0x7fa1240d10d0 con 0x7fa124162810 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa123fff640 1 --2- 172.21.15.40:0/2508744926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240d1b30 0x7fa1240d1fc0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009d90 tx=0x7fa11c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6123ef92f7fce3bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa1227fc640 1 -- 172.21.15.40:0/2508744926 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c007740 con 0x7fa1240d1b30 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa1227fc640 1 -- 172.21.15.40:0/2508744926 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c0078e0 con 0x7fa1240d1b30 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa1227fc640 1 -- 172.21.15.40:0/2508744926 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c007cb0 con 0x7fa1240d1b30 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa129b03640 1 -- 172.21.15.40:0/2508744926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240d1b30 msgr2=0x7fa1240d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa129b03640 1 --2- 172.21.15.40:0/2508744926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240d1b30 0x7fa1240d1fc0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009d90 tx=0x7fa11c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa129b03640 1 -- 172.21.15.40:0/2508744926 shutdown_connections 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa129b03640 1 --2- 172.21.15.40:0/2508744926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240d1b30 0x7fa1240d1fc0 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.902+0000 7fa129b03640 1 -- 172.21.15.40:0/2508744926 >> 172.21.15.40:0/2508744926 conn(0x7fa1240d2da0 msgr2=0x7fa1240d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.903+0000 7fa129b03640 1 -- 172.21.15.40:0/2508744926 shutdown_connections 2024-04-26T21:46:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.903+0000 7fa129b03640 1 -- 172.21.15.40:0/2508744926 wait complete. 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.904+0000 7fa129b03640 1 Processor -- start 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.904+0000 7fa129b03640 1 -- start start 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa129b03640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240b2340 0x7fa1241f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa129b03640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1240d1b30 0x7fa124160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa129b03640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa124162810 0x7fa12415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa129b03640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1240cdb00 con 0x7fa1240b2340 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa129b03640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1240d16a0 con 0x7fa1240d1b30 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa129b03640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa124070b50 con 0x7fa124162810 2024-04-26T21:46:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240b2340 0x7fa1241f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240b2340 0x7fa1241f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40190/0 (socket says 172.21.15.40:40190) 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 -- 172.21.15.40:0/859451902 learned_addr learned my addr 172.21.15.40:0/859451902 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa122ffd640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1240d1b30 0x7fa124160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 -- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa124162810 msgr2=0x7fa12415bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa123fff640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa124162810 0x7fa12415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa124162810 0x7fa12415bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 -- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1240d1b30 msgr2=0x7fa124160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1240d1b30 0x7fa124160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 -- 172.21.15.40:0/859451902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa124161800 con 0x7fa1240b2340 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa122ffd640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1240d1b30 0x7fa124160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa123fff640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa124162810 0x7fa12415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa1237fe640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240b2340 0x7fa1241f8530 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c460 tx=0x7fa11400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa10bfff640 1 -- 172.21.15.40:0/859451902 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114018070 con 0x7fa1240b2340 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa10bfff640 1 -- 172.21.15.40:0/859451902 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa114007500 con 0x7fa1240b2340 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa124161ae0 con 0x7fa1240b2340 2024-04-26T21:46:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa10bfff640 1 -- 172.21.15.40:0/859451902 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114004a20 con 0x7fa1240b2340 2024-04-26T21:46:29.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.905+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1240c7d30 con 0x7fa1240b2340 2024-04-26T21:46:29.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.906+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1240cfdb0 con 0x7fa1240b2340 2024-04-26T21:46:29.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.907+0000 7fa10bfff640 1 -- 172.21.15.40:0/859451902 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa11401b070 con 0x7fa1240b2340 2024-04-26T21:46:29.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.907+0000 7fa10bfff640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa11006c8f0 0x7fa11006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:29.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.908+0000 7fa10bfff640 1 -- 172.21.15.40:0/859451902 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7fa11408bce0 con 0x7fa1240b2340 2024-04-26T21:46:29.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.908+0000 7fa122ffd640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa11006c8f0 0x7fa11006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:29.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.908+0000 7fa122ffd640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa11006c8f0 0x7fa11006edb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa124160ab0 tx=0x7fa118009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:29.912+0000 7fa10bfff640 1 -- 172.21.15.40:0/859451902 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa1240cfdb0 con 0x7fa1240b2340 2024-04-26T21:46:30.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.170+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa1240694f0 con 0x7fa1240b2340 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.789+0000 7fa10bfff640 1 -- 172.21.15.40:0/859451902 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v321) ==== 101+0+0 (secure 0 0 0) 0x7fa114059210 con 0x7fa1240b2340 2024-04-26T21:46:30.790 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa11006c8f0 msgr2=0x7fa11006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa11006c8f0 0x7fa11006edb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa124160ab0 tx=0x7fa118009040 comp rx=0 tx=0).stop 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240b2340 msgr2=0x7fa1241f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240b2340 0x7fa1241f8530 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c460 tx=0x7fa11400c920 comp rx=0 tx=0).stop 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 shutdown_connections 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa11006c8f0 0x7fa11006edb0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa124162810 0x7fa12415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1240d1b30 0x7fa124160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 --2- 172.21.15.40:0/859451902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1240b2340 0x7fa1241f8530 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 >> 172.21.15.40:0/859451902 conn(0x7fa1240d2da0 msgr2=0x7fa1240ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 shutdown_connections 2024-04-26T21:46:30.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.792+0000 7fa129b03640 1 -- 172.21.15.40:0/859451902 wait complete. 2024-04-26T21:46:30.806 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:46:30.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.851+0000 7f33afb59f40 1 Processor -- start 2024-04-26T21:46:30.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.851+0000 7f33afb59f40 1 -- start start 2024-04-26T21:46:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33afb59f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b652f930 0x55c0b652f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33afb59f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b655faf0 0x55c0b65327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33afb59f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b6608900 0x55c0b655cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33afb59f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c0b64d8e40 con 0x55c0b6608900 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33afb59f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0b64dce80 con 0x55c0b655faf0 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33afb59f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c0b655f010 con 0x55c0b652f930 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b6608900 0x55c0b655cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b6608900 0x55c0b655cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40216/0 (socket says 172.21.15.40:40216) 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 -- 172.21.15.40:0/2413899078 learned_addr learned my addr 172.21.15.40:0/2413899078 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ace52640 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b655faf0 0x55c0b65327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ad653640 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b652f930 0x55c0b652f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 -- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b652f930 msgr2=0x55c0b652f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b652f930 0x55c0b652f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 -- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b655faf0 msgr2=0x55c0b65327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b655faf0 0x55c0b65327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 -- 172.21.15.40:0/2413899078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b655c640 con 0x55c0b6608900 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ace52640 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b655faf0 0x55c0b65327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f33ade54640 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b6608900 0x55c0b655cfa0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f33a80016b0 tx=0x7f33a8006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cf12de70850eda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f339f7fe640 1 -- 172.21.15.40:0/2413899078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8003650 con 0x55c0b6608900 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f339f7fe640 1 -- 172.21.15.40:0/2413899078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a80037f0 con 0x55c0b6608900 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.852+0000 7f339f7fe640 1 -- 172.21.15.40:0/2413899078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8003c00 con 0x55c0b6608900 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 -- 172.21.15.40:0/2413899078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b6608900 msgr2=0x55c0b655cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b6608900 0x55c0b655cfa0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f33a80016b0 tx=0x7f33a8006f90 comp rx=0 tx=0).stop 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 -- 172.21.15.40:0/2413899078 shutdown_connections 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b6608900 0x55c0b655cfa0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b655faf0 0x55c0b65327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2413899078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b652f930 0x55c0b652f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 -- 172.21.15.40:0/2413899078 >> 172.21.15.40:0/2413899078 conn(0x55c0b6530330 msgr2=0x55c0b6606a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 -- 172.21.15.40:0/2413899078 shutdown_connections 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.853+0000 7f33afb59f40 1 -- 172.21.15.40:0/2413899078 wait complete. 2024-04-26T21:46:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.854+0000 7f33afb59f40 1 Processor -- start 2024-04-26T21:46:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.854+0000 7f33afb59f40 1 -- start start 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.854+0000 7f33afb59f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b652f930 0x55c0b655cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.854+0000 7f33afb59f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b655faf0 0x55c0b6530710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.854+0000 7f33afb59f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b6608900 0x55c0b6530c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.854+0000 7f33afb59f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c0b65fc800 con 0x55c0b652f930 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.854+0000 7f33afb59f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0b655af60 con 0x55c0b6608900 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.854+0000 7f33afb59f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c0b6531a30 con 0x55c0b655faf0 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b652f930 0x55c0b655cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b652f930 0x55c0b655cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40228/0 (socket says 172.21.15.40:40228) 2024-04-26T21:46:30.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 learned_addr learned my addr 172.21.15.40:0/2388663055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ace52640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b655faf0 0x55c0b6530710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ade54640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b6608900 0x55c0b6530c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b655faf0 msgr2=0x55c0b6530710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c0b655faf0 0x55c0b6530710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b6608900 msgr2=0x55c0b6530c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c0b6608900 0x55c0b6530c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b6603760 con 0x55c0b652f930 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33ad653640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b652f930 0x55c0b655cc50 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f33a0002c80 tx=0x7f33a000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f339e7fc640 1 -- 172.21.15.40:0/2388663055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0015070 con 0x55c0b652f930 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f339e7fc640 1 -- 172.21.15.40:0/2388663055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a0011640 con 0x55c0b652f930 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f339e7fc640 1 -- 172.21.15.40:0/2388663055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0011a30 con 0x55c0b652f930 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c0b6603a40 con 0x55c0b652f930 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.855+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b6609290 con 0x55c0b652f930 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.857+0000 7f339e7fc640 1 -- 172.21.15.40:0/2388663055 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33a0018070 con 0x55c0b652f930 2024-04-26T21:46:30.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.857+0000 7f339e7fc640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f339006b2d0 0x7f339006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.857+0000 7f339e7fc640 1 -- 172.21.15.40:0/2388663055 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f33a008b690 con 0x55c0b652f930 2024-04-26T21:46:30.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.858+0000 7f33ace52640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f339006b2d0 0x7f339006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.861 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:46:30.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.858+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0b655d190 0x55c0b6531190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.858+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b65fb7b0 con 0x55c0b655d190 2024-04-26T21:46:30.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.858+0000 7f33ade54640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0b655d190 0x55c0b6531190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.858+0000 7f33ade54640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0b655d190 0x55c0b6531190 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.858+0000 7f33ace52640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f339006b2d0 0x7f339006d790 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f33a4057000 tx=0x7f33a4052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.860+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v131'1097 uv1070 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.860+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b6698b30 con 0x55c0b655d190 2024-04-26T21:46:30.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.861+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v131'1098 uv1098 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.861+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b6698f20 con 0x55c0b655d190 2024-04-26T21:46:30.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.863+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v131'1099 uv1099 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.863 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:46:30.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.863+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b6699310 con 0x55c0b655d190 2024-04-26T21:46:30.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.864+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v131'1100 uv1100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.864+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b6699740 con 0x55c0b655d190 2024-04-26T21:46:30.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.865+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v131'1101 uv1101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.865+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ac190 con 0x55c0b655d190 2024-04-26T21:46:30.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.866+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v131'1102 uv1102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.866+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ac530 con 0x55c0b655d190 2024-04-26T21:46:30.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.867+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v131'1103 uv1076 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.867+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66ac530 con 0x55c0b655d190 2024-04-26T21:46:30.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.868+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v131'1104 uv1104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.868+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ac530 con 0x55c0b655d190 2024-04-26T21:46:30.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.870+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v131'1105 uv1105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.870 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:46:30.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.870+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ad540 con 0x55c0b655d190 2024-04-26T21:46:30.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.871+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v131'1106 uv1106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.871+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ad540 con 0x55c0b655d190 2024-04-26T21:46:30.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.872+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v131'1107 uv1107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.872+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ac530 con 0x55c0b655d190 2024-04-26T21:46:30.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.873+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v131'1108 uv1108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.873+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ac530 con 0x55c0b655d190 2024-04-26T21:46:30.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.874+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v131'1109 uv1083 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.874+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66ac530 con 0x55c0b655d190 2024-04-26T21:46:30.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.875+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v131'1110 uv1110 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.875+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ac530 con 0x55c0b655d190 2024-04-26T21:46:30.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.876+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v131'1111 uv1111 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.877 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:46:30.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.876+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ad540 con 0x55c0b655d190 2024-04-26T21:46:30.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.877+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v131'1112 uv1112 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.877+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ad540 con 0x55c0b655d190 2024-04-26T21:46:30.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.880+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v131'1113 uv1113 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.880+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66ac530 con 0x55c0b655d190 2024-04-26T21:46:30.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.881+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v131'1114 uv1114 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.881+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.882+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v131'1115 uv1115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.882+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.883+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v131'1116 uv1096 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.884+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.885+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v131'1117 uv1117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.885+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.886+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v131'1118 uv1118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.886 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:46:30.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.886+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66adca0 con 0x55c0b655d190 2024-04-26T21:46:30.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.887+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v131'1119 uv1119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.887+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66adca0 con 0x55c0b655d190 2024-04-26T21:46:30.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.888+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v131'1120 uv1120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.888+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.889+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v131'1121 uv1121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.889+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.891+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v131'1122 uv1095 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.891+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.893+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v131'1123 uv1123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.893+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.894 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:46:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.893+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v131'1124 uv1124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.894+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66adca0 con 0x55c0b655d190 2024-04-26T21:46:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.895+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v131'1125 uv1125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.895+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66adca0 con 0x55c0b655d190 2024-04-26T21:46:30.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.896+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v131'1126 uv1126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.896+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66acc90 con 0x55c0b655d190 2024-04-26T21:46:30.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.897+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v131'1127 uv1127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b655d190 2024-04-26T21:46:30.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.897+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0b66ad1a0 0x55c0b66af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.897+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66afad0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.897+0000 7f33ad653640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0b66ad1a0 0x55c0b66af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.897+0000 7f33ad653640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0b66ad1a0 0x55c0b66af590 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.899+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v131'799 uv781 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.899+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66afe70 con 0x55c0b66ad1a0 2024-04-26T21:46:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.901+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v131'800 uv800 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.901+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b02b0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.903 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:46:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.902+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v131'801 uv801 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.902+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b0b70 con 0x55c0b66ad1a0 2024-04-26T21:46:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.904+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v131'802 uv802 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.904+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b0f10 con 0x55c0b66ad1a0 2024-04-26T21:46:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.905+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v131'803 uv803 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.905+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b12b0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.906+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v131'804 uv786 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.907+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66b1650 con 0x55c0b66ad1a0 2024-04-26T21:46:30.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.908+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v131'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.908+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b1a40 con 0x55c0b66ad1a0 2024-04-26T21:46:30.910 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:46:30.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.909+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v131'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.909+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b1a40 con 0x55c0b66ad1a0 2024-04-26T21:46:30.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.911+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v131'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.911+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.912+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v131'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.912+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.913+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v131'809 uv791 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.913+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.915+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v131'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.915+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.916 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:46:30.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.916+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v131'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.916+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.917+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v131'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.917+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.919+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v131'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.919+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.920+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v131'814 uv797 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.920+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.921+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v131'815 uv815 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.921+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.923 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:46:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.923+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v131'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.923+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b2a00 con 0x55c0b66ad1a0 2024-04-26T21:46:30.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.924+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v131'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.924+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b2a00 con 0x55c0b66ad1a0 2024-04-26T21:46:30.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.925+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v131'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.925+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b19f0 con 0x55c0b66ad1a0 2024-04-26T21:46:30.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.926+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v131'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ad1a0 2024-04-26T21:46:30.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.927+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0b66b22c0 0x55c0b66b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.927+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b4bf0 con 0x55c0b66b22c0 2024-04-26T21:46:30.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.927+0000 7f33ace52640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0b66b22c0 0x55c0b66b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.927+0000 7f33ace52640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0b66b22c0 0x55c0b66b46b0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.929+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v131'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.929+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66b4ff0 con 0x55c0b66b22c0 2024-04-26T21:46:30.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.930+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v131'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.930+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b54d0 con 0x55c0b66b22c0 2024-04-26T21:46:30.931 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:46:30.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.931+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v131'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.931+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b6d80 con 0x55c0b66b22c0 2024-04-26T21:46:30.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.932+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v131'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.932+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b7120 con 0x55c0b66b22c0 2024-04-26T21:46:30.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.933+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v131'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.933+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b5d70 con 0x55c0b66b22c0 2024-04-26T21:46:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.934+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v131'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.934+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0b66b6670 0x55c0b66b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.934+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b9a10 con 0x55c0b66b6670 2024-04-26T21:46:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.934+0000 7f33ade54640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0b66b6670 0x55c0b66b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.935+0000 7f33ade54640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0b66b6670 0x55c0b66b94d0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.936+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v131'165 uv164 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.936+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66b9a10 con 0x55c0b66b6670 2024-04-26T21:46:30.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.937+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v131'166 uv166 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.937+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b9a90 con 0x55c0b66b6670 2024-04-26T21:46:30.939 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:46:30.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.938+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v131'167 uv167 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.939+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66b9eb0 con 0x55c0b66b6670 2024-04-26T21:46:30.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.940+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v131'168 uv168 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.940+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0b66ba3c0 0x55c0b66bc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.940+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bcd30 con 0x55c0b66ba3c0 2024-04-26T21:46:30.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.940+0000 7f33ad653640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0b66ba3c0 0x55c0b66bc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.940+0000 7f33ad653640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0b66ba3c0 0x55c0b66bc7b0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.942+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v131'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.942+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66bcdb0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.943+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v131'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.943+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.945 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:46:30.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.944+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v131'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.944+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66be1e0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.945+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v131'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.946+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66be1e0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.947+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v131'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.947+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.948+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v131'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.948+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.950+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v131'393 uv387 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.950+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.951+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v131'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.951+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.953 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:46:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.952+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v131'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.952+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66be1e0 con 0x55c0b66b6670 2024-04-26T21:46:30.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.953+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v131'396 uv396 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.953+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66be1e0 con 0x55c0b66b6670 2024-04-26T21:46:30.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.954+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v131'397 uv397 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.954+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.955+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v131'398 uv398 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.955+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.957+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v131'399 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.957+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.958+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v131'400 uv400 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.958+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.959 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:46:30.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.959+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v131'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.959+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.960+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v131'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.960+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.961+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v131'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.961+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.962+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v131'178 uv177 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.962+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.963+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v131'179 uv179 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.963+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.965 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:46:30.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.964+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v131'180 uv180 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.964+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.965+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v131'181 uv181 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.965+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66b6670 2024-04-26T21:46:30.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.966+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v131'182 uv182 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f33a800f030 con 0x55c0b66b6670 2024-04-26T21:46:30.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.966+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.967+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v131'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.967+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.968+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v131'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.968+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.969+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v131'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.970 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:46:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.969+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66be1e0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.970+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v131'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.970+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66be1e0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.971+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v131'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.971+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.972+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v131'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.972+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.973+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v131'355 uv349 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.973+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.974+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v131'356 uv356 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.975+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd1d0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.975+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v131'357 uv357 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.976 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:46:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.976+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd590 con 0x55c0b66ba3c0 2024-04-26T21:46:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.976+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v131'358 uv358 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.977+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd590 con 0x55c0b66ba3c0 2024-04-26T21:46:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.977+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v131'359 uv359 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.978+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd590 con 0x55c0b66ba3c0 2024-04-26T21:46:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.979+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v131'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.979+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66bd590 con 0x55c0b66ba3c0 2024-04-26T21:46:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.980+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v131'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.980+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd590 con 0x55c0b66ba3c0 2024-04-26T21:46:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.981+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v131'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.981 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:46:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.981+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66bd590 con 0x55c0b66ba3c0 2024-04-26T21:46:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.982+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v131'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.982+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c03a0 con 0x55c0b66ba3c0 2024-04-26T21:46:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.983+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v131'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.983+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0b66bdf50 0x55c0b66c0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.983+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66bdf50 2024-04-26T21:46:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.983+0000 7f33ace52640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0b66bdf50 0x55c0b66c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.983+0000 7f33ace52640 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0b66bdf50 0x55c0b66c0790 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.984+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v131'157 uv155 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66bdf50 2024-04-26T21:46:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.985+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66bdf50 2024-04-26T21:46:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.986+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v131'158 uv158 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66bdf50 2024-04-26T21:46:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.986+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66bdf50 2024-04-26T21:46:30.987 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:46:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.986+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v131'159 uv159 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66bdf50 2024-04-26T21:46:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.987+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66bdf50 2024-04-26T21:46:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.987+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v131'160 uv160 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66bdf50 2024-04-26T21:46:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.987+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66ba3c0 2024-04-26T21:46:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.989+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v131'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.989+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66ba3c0 2024-04-26T21:46:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.990+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v131'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.990+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66ba3c0 2024-04-26T21:46:30.992 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:46:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.991+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v131'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.991+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66ba3c0 2024-04-26T21:46:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.992+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v131'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.992+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66ba3c0 2024-04-26T21:46:30.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.993+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v131'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.993+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66ba3c0 2024-04-26T21:46:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.994+0000 7f33ad653640 1 -- 172.21.15.40:0/2388663055 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v131'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a0021070 con 0x55c0b66ba3c0 2024-04-26T21:46:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.994+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.995+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v131'181 uv180 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.995+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.996+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v131'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.996+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:30.998 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:46:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.997+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v131'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.998+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.999+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v131'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:30.999+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.000+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v131'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.000+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.001+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v131'442 uv433 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.001+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.002+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v131'443 uv443 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.002+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:46:31.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.003+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v131'444 uv444 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.003+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.004+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v131'445 uv445 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.004+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.005+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v131'446 uv446 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.005+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.006+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v131'447 uv440 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.007+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.007+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v131'448 uv448 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.008+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.009 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:46:31.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.008+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v131'449 uv449 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.008+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.010+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v131'450 uv450 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.010+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.010+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v131'451 uv451 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.011+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0340 con 0x55c0b66b22c0 2024-04-26T21:46:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.011+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v131'452 uv452 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.011+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55c0b66c0e10 con 0x55c0b66b22c0 2024-04-26T21:46:31.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33ace52640 1 -- 172.21.15.40:0/2388663055 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v131'453 uv453 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f33a4061be0 con 0x55c0b66b22c0 2024-04-26T21:46:31.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0b66ad1a0 msgr2=0x55c0b66af590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x55c0b66ad1a0 0x55c0b66af590 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0b655d190 msgr2=0x55c0b6531190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x55c0b655d190 0x55c0b6531190 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0b66b6670 msgr2=0x55c0b66b94d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55c0b66b6670 0x55c0b66b94d0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0b66bdf50 msgr2=0x55c0b66c0790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55c0b66bdf50 0x55c0b66c0790 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0b66b22c0 msgr2=0x55c0b66b46b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55c0b66b22c0 0x55c0b66b46b0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0b66ba3c0 msgr2=0x55c0b66bc7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55c0b66ba3c0 0x55c0b66bc7b0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 reap_dead start 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33ade54640 1 -- 172.21.15.40:0/2388663055 reap_dead start 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f339006b2d0 msgr2=0x7f339006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f339006b2d0 0x7f339006d790 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f33a4057000 tx=0x7f33a4052000 comp rx=0 tx=0).stop 2024-04-26T21:46:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b652f930 msgr2=0x55c0b655cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b652f930 0x55c0b655cc50 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f33a0002c80 tx=0x7f33a000b5b0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 shutdown_connections 2024-04-26T21:46:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f339006b2d0 0x7f339006d790 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 --2- 172.21.15.40:0/2388663055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0b652f930 0x55c0b655cc50 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 >> 172.21.15.40:0/2388663055 conn(0x55c0b6530330 msgr2=0x55c0b66065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 shutdown_connections 2024-04-26T21:46:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.013+0000 7f33afb59f40 1 -- 172.21.15.40:0/2388663055 wait complete. 2024-04-26T21:46:31.017 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_openfiles.0 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.059+0000 7f2c04f91f40 1 Processor -- start 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.059+0000 7f2c04f91f40 1 -- start start 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.059+0000 7f2c04f91f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01fa14700 0x55f01fa96a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.059+0000 7f2c04f91f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa15770 0x55f01faa4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.059+0000 7f2c04f91f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01faa16b0 0x55f01faa47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.059+0000 7f2c04f91f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f01f9fda20 con 0x55f01faa16b0 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.059+0000 7f2c04f91f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f01fa15be0 con 0x55f01fa15770 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.059+0000 7f2c04f91f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f01f9fdd00 con 0x55f01fa14700 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01faa16b0 0x55f01faa47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c02a8b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01fa14700 0x55f01fa96a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01faa16b0 0x55f01faa47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40234/0 (socket says 172.21.15.40:40234) 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 -- 172.21.15.40:0/1932987465 learned_addr learned my addr 172.21.15.40:0/1932987465 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0228a640 1 --2- 172.21.15.40:0/1932987465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa15770 0x55f01faa4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 -- 172.21.15.40:0/1932987465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01fa14700 msgr2=0x55f01fa96a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 --2- 172.21.15.40:0/1932987465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01fa14700 0x55f01fa96a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 -- 172.21.15.40:0/1932987465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa15770 msgr2=0x55f01faa4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 --2- 172.21.15.40:0/1932987465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa15770 0x55f01faa4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 -- 172.21.15.40:0/1932987465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f01f8f6cb0 con 0x55f01faa16b0 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c0328c640 1 --2- 172.21.15.40:0/1932987465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01faa16b0 0x55f01faa47a0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00a7a0 tx=0x7f2bfc00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9aec15af9793724 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c01288640 1 -- 172.21.15.40:0/1932987465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc019070 con 0x55f01faa16b0 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c01288640 1 -- 172.21.15.40:0/1932987465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc007600 con 0x55f01faa16b0 2024-04-26T21:46:31.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.060+0000 7f2c01288640 1 -- 172.21.15.40:0/1932987465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0079d0 con 0x55f01faa16b0 2024-04-26T21:46:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1932987465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01faa16b0 msgr2=0x55f01faa47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 --2- 172.21.15.40:0/1932987465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01faa16b0 0x55f01faa47a0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00a7a0 tx=0x7f2bfc00a890 comp rx=0 tx=0).stop 2024-04-26T21:46:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1932987465 shutdown_connections 2024-04-26T21:46:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 --2- 172.21.15.40:0/1932987465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01faa16b0 0x55f01faa47a0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 --2- 172.21.15.40:0/1932987465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa15770 0x55f01faa4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 --2- 172.21.15.40:0/1932987465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01fa14700 0x55f01fa96a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1932987465 >> 172.21.15.40:0/1932987465 conn(0x55f01f9f79b0 msgr2=0x55f01f9f7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1932987465 shutdown_connections 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1932987465 wait complete. 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 Processor -- start 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.061+0000 7f2c04f91f40 1 -- start start 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c04f91f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa14700 0x55f01fa93300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c04f91f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01fa15770 0x55f01fa92710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c04f91f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01faa16b0 0x55f01fa950d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c04f91f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f01fa9ffa0 con 0x55f01fa15770 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c04f91f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f01faa2850 con 0x55f01fa14700 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c04f91f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f01faa0280 con 0x55f01faa16b0 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0228a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01fa15770 0x55f01fa92710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0228a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01fa15770 0x55f01fa92710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40250/0 (socket says 172.21.15.40:40250) 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0228a640 1 -- 172.21.15.40:0/1528602502 learned_addr learned my addr 172.21.15.40:0/1528602502 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:31.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c02a8b640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa14700 0x55f01fa93300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0328c640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01faa16b0 0x55f01fa950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0228a640 1 -- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01faa16b0 msgr2=0x55f01fa950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0228a640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f01faa16b0 0x55f01fa950d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0228a640 1 -- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa14700 msgr2=0x55f01fa93300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0228a640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa14700 0x55f01fa93300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c0228a640 1 -- 172.21.15.40:0/1528602502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f01faa4800 con 0x55f01fa15770 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.062+0000 7f2c02a8b640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f01fa14700 0x55f01fa93300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.063+0000 7f2c0228a640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01fa15770 0x55f01fa92710 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009f40 tx=0x7f2bf80084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.063+0000 7f2bebfff640 1 -- 172.21.15.40:0/1528602502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8014600 con 0x55f01fa15770 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.063+0000 7f2bebfff640 1 -- 172.21.15.40:0/1528602502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf8019070 con 0x55f01fa15770 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.063+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f01fa9f7d0 con 0x55f01fa15770 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.063+0000 7f2bebfff640 1 -- 172.21.15.40:0/1528602502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80149a0 con 0x55f01fa15770 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.063+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01faa4ac0 con 0x55f01fa15770 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.064+0000 7f2bebfff640 1 -- 172.21.15.40:0/1528602502 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bf8013070 con 0x55f01fa15770 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.065+0000 7f2bebfff640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf006b2d0 0x7f2bf006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.065+0000 7f2bebfff640 1 -- 172.21.15.40:0/1528602502 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f2bf808b360 con 0x55f01fa15770 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.065+0000 7f2c02a8b640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf006b2d0 0x7f2bf006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.065+0000 7f2c04f91f40 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f01fa9edb0 0x55f01fa979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.065+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55f01fa92e20 con 0x55f01fa9edb0 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.065+0000 7f2c0328c640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f01fa9edb0 0x55f01fa979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.065+0000 7f2c02a8b640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf006b2d0 0x7f2bf006d790 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4056000 tx=0x7f2bf4007030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.065+0000 7f2c0328c640 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f01fa9edb0 0x55f01fa979b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.066+0000 7f2c0328c640 1 -- 172.21.15.40:0/1528602502 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-set-header] v131'169 uv169 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2bfc0079d0 con 0x55f01fa9edb0 2024-04-26T21:46:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f01fa9edb0 msgr2=0x55f01fa979b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55f01fa9edb0 0x55f01fa979b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf006b2d0 msgr2=0x7f2bf006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2bf006b2d0 0x7f2bf006d790 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4056000 tx=0x7f2bf4007030 comp rx=0 tx=0).stop 2024-04-26T21:46:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01fa15770 msgr2=0x55f01fa92710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 --2- 172.21.15.40:0/1528602502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f01fa15770 0x55f01fa92710 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009f40 tx=0x7f2bf80084c0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c0328c640 1 -- 172.21.15.40:0/1528602502 reap_dead start 2024-04-26T21:46:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 shutdown_connections 2024-04-26T21:46:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 >> 172.21.15.40:0/1528602502 conn(0x55f01f9f79b0 msgr2=0x55f01faa34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 shutdown_connections 2024-04-26T21:46:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.067+0000 7f2c04f91f40 1 -- 172.21.15.40:0/1528602502 wait complete. 2024-04-26T21:46:31.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:46:31.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.215+0000 7f74b99f7640 1 Processor -- start 2024-04-26T21:46:31.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.215+0000 7f74b99f7640 1 -- start start 2024-04-26T21:46:31.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b99f7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f74b40d1720 0x7f74b40d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b99f7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74b40d39f0 0x7f74b4070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b99f7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74b40701e0 con 0x7f74b415d250 2024-04-26T21:46:31.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b99f7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f74b415d630 con 0x7f74b40d39f0 2024-04-26T21:46:31.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b99f7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f74b40704e0 con 0x7f74b40d1720 2024-04-26T21:46:31.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f74b415d250 0x7f74b40d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39782/0 (socket says 172.21.15.40:39782) 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b2ffd640 1 -- 172.21.15.40:0/367170095 learned_addr learned my addr 172.21.15.40:0/367170095 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74b40d39f0 0x7f74b4070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74abfff640 1 --2- 172.21.15.40:0/367170095 >> v2:172.21.15.184:3300/0 conn(0x7f74b40d1720 0x7f74b40d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b37fe640 1 -- 172.21.15.40:0/367170095 >> v2:172.21.15.184:3300/0 conn(0x7f74b40d1720 msgr2=0x7f74b40d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b37fe640 1 --2- 172.21.15.40:0/367170095 >> v2:172.21.15.184:3300/0 conn(0x7f74b40d1720 0x7f74b40d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b37fe640 1 -- 172.21.15.40:0/367170095 >> v1:172.21.15.40:6789/0 conn(0x7f74b415d250 legacy=0x7f74b40d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b37fe640 1 -- 172.21.15.40:0/367170095 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b40ce370 con 0x7f74b40d39f0 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74b27fc640 1 -- 172.21.15.40:0/367170095 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930861907 0 0) 0x7f74b40701e0 con 0x7f74b415d250 2024-04-26T21:46:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.216+0000 7f74abfff640 1 --2- 172.21.15.40:0/367170095 >> v2:172.21.15.184:3300/0 conn(0x7f74b40d1720 0x7f74b40d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:31.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.217+0000 7f74b37fe640 1 --2- 172.21.15.40:0/367170095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b40d39f0 0x7f74b4070970 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f74b4074050 tx=0x7f74ac02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb11a2fe19aef97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.217+0000 7f74b27fc640 1 -- 172.21.15.40:0/367170095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac007740 con 0x7f74b40d39f0 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.217+0000 7f74b27fc640 1 -- 172.21.15.40:0/367170095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ac0078e0 con 0x7f74b40d39f0 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.217+0000 7f74b27fc640 1 -- 172.21.15.40:0/367170095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac007cb0 con 0x7f74b40d39f0 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.218+0000 7f74b99f7640 1 -- 172.21.15.40:0/367170095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b40d39f0 msgr2=0x7f74b4070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.218+0000 7f74b99f7640 1 --2- 172.21.15.40:0/367170095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b40d39f0 0x7f74b4070970 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f74b4074050 tx=0x7f74ac02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.218+0000 7f74b99f7640 1 -- 172.21.15.40:0/367170095 shutdown_connections 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.218+0000 7f74b99f7640 1 --2- 172.21.15.40:0/367170095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b40d39f0 0x7f74b4070970 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.218+0000 7f74b99f7640 1 --2- 172.21.15.40:0/367170095 >> v2:172.21.15.184:3300/0 conn(0x7f74b40d1720 0x7f74b40d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.218+0000 7f74b99f7640 1 -- 172.21.15.40:0/367170095 >> 172.21.15.40:0/367170095 conn(0x7f74b40d2d80 msgr2=0x7f74b40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.218+0000 7f74b99f7640 1 -- 172.21.15.40:0/367170095 shutdown_connections 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.218+0000 7f74b99f7640 1 -- 172.21.15.40:0/367170095 wait complete. 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.219+0000 7f74b99f7640 1 Processor -- start 2024-04-26T21:46:31.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.219+0000 7f74b99f7640 1 -- start start 2024-04-26T21:46:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b99f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b40d1720 0x7f74b41573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b99f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74b40d39f0 0x7f74b4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b99f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b415d250 0x7f74b40ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b99f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74b40cebb0 con 0x7f74b415d250 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b99f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74b40ce8d0 con 0x7f74b40d1720 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b99f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74b4070eb0 con 0x7f74b40d39f0 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b415d250 0x7f74b40ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b415d250 0x7f74b40ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40272/0 (socket says 172.21.15.40:40272) 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 -- 172.21.15.40:0/2908512021 learned_addr learned my addr 172.21.15.40:0/2908512021 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74abfff640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74b40d39f0 0x7f74b4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b2ffd640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b40d1720 0x7f74b41573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 -- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74b40d39f0 msgr2=0x7f74b4158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74b40d39f0 0x7f74b4158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 -- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b40d1720 msgr2=0x7f74b41573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b40d1720 0x7f74b41573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 -- 172.21.15.40:0/2908512021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b40c6950 con 0x7f74b415d250 2024-04-26T21:46:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74abfff640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74b40d39f0 0x7f74b4158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b37fe640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b415d250 0x7f74b40ca860 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009e80 tx=0x7f74ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b0ff9640 1 -- 172.21.15.40:0/2908512021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac040070 con 0x7f74b415d250 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b0ff9640 1 -- 172.21.15.40:0/2908512021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ac031c40 con 0x7f74b415d250 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b40c6c30 con 0x7f74b415d250 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b0ff9640 1 -- 172.21.15.40:0/2908512021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac009450 con 0x7f74b415d250 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.220+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b40c71f0 con 0x7f74b415d250 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.222+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b40694f0 con 0x7f74b415d250 2024-04-26T21:46:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.222+0000 7f74b0ff9640 1 -- 172.21.15.40:0/2908512021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74ac039040 con 0x7f74b415d250 2024-04-26T21:46:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.222+0000 7f74b0ff9640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f749406c920 0x7f749406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.222+0000 7f74b0ff9640 1 -- 172.21.15.40:0/2908512021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f74ac0adc80 con 0x7f74b415d250 2024-04-26T21:46:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.222+0000 7f74b2ffd640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f749406c920 0x7f749406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.223+0000 7f74b2ffd640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f749406c920 0x7f749406ede0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f74a4000f60 tx=0x7f74a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.227+0000 7f74b0ff9640 1 -- 172.21.15.40:0/2908512021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74b40ce370 con 0x7f74b415d250 2024-04-26T21:46:31.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.453+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f74b40d0fb0 con 0x7f74b415d250 2024-04-26T21:46:31.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.791+0000 7f74b0ff9640 1 -- 172.21.15.40:0/2908512021 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v322) ==== 162+0+0 (secure 0 0 0) 0x7f74ac07b1e0 con 0x7f74b415d250 2024-04-26T21:46:31.792 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:46:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f749406c920 msgr2=0x7f749406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f749406c920 0x7f749406ede0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f74a4000f60 tx=0x7f74a4009040 comp rx=0 tx=0).stop 2024-04-26T21:46:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b415d250 msgr2=0x7f74b40ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b415d250 0x7f74b40ca860 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009e80 tx=0x7f74ac004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 shutdown_connections 2024-04-26T21:46:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f749406c920 0x7f749406ede0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b415d250 0x7f74b40ca860 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74b40d39f0 0x7f74b4158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 --2- 172.21.15.40:0/2908512021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b40d1720 0x7f74b41573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:31.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 >> 172.21.15.40:0/2908512021 conn(0x7f74b40d2d80 msgr2=0x7f74b415f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:31.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 shutdown_connections 2024-04-26T21:46:31.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:31.794+0000 7f74b99f7640 1 -- 172.21.15.40:0/2908512021 wait complete. 2024-04-26T21:46:31.803 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:31.802+0000 7f482ecea640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-04-26T21:46:32.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.325+0000 7f1787589640 1 Processor -- start 2024-04-26T21:46:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.326+0000 7f1787589640 1 -- start start 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.326+0000 7f1787589640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f178015f3a0 0x7f17800d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.326+0000 7f1787589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d1770 0x7f17800d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.326+0000 7f1787589640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17800d2130 0x7f17800d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.326+0000 7f1787589640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f17800d25a0 con 0x7f17800d2130 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.326+0000 7f1787589640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f17800d0e20 con 0x7f178015f3a0 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.326+0000 7f1787589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17800b1d70 con 0x7f17800d1770 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785b26640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17800d2130 0x7f17800d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785b26640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17800d2130 0x7f17800d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40280/0 (socket says 172.21.15.40:40280) 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785b26640 1 -- 172.21.15.40:0/2512126344 learned_addr learned my addr 172.21.15.40:0/2512126344 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1784b24640 1 --2- 172.21.15.40:0/2512126344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d1770 0x7f17800d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785325640 1 --2- 172.21.15.40:0/2512126344 >> v2:172.21.15.184:3300/0 conn(0x7f178015f3a0 0x7f17800d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785b26640 1 -- 172.21.15.40:0/2512126344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d1770 msgr2=0x7f17800d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785b26640 1 --2- 172.21.15.40:0/2512126344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d1770 0x7f17800d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785b26640 1 -- 172.21.15.40:0/2512126344 >> v2:172.21.15.184:3300/0 conn(0x7f178015f3a0 msgr2=0x7f17800d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785b26640 1 --2- 172.21.15.40:0/2512126344 >> v2:172.21.15.184:3300/0 conn(0x7f178015f3a0 0x7f17800d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.327+0000 7f1785b26640 1 -- 172.21.15.40:0/2512126344 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17800b2050 con 0x7f17800d2130 2024-04-26T21:46:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.328+0000 7f1785b26640 1 --2- 172.21.15.40:0/2512126344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800d2130 0x7f17800d2e50 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f1780073b50 tx=0x7f177802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499974dccd29ef54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.328+0000 7f1777fff640 1 -- 172.21.15.40:0/2512126344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778036070 con 0x7f17800d2130 2024-04-26T21:46:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.328+0000 7f1777fff640 1 -- 172.21.15.40:0/2512126344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778031600 con 0x7f17800d2130 2024-04-26T21:46:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.328+0000 7f1777fff640 1 -- 172.21.15.40:0/2512126344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17780319d0 con 0x7f17800d2130 2024-04-26T21:46:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.328+0000 7f1787589640 1 -- 172.21.15.40:0/2512126344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800d2130 msgr2=0x7f17800d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.328+0000 7f1787589640 1 --2- 172.21.15.40:0/2512126344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800d2130 0x7f17800d2e50 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f1780073b50 tx=0x7f177802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.329+0000 7f1787589640 1 -- 172.21.15.40:0/2512126344 shutdown_connections 2024-04-26T21:46:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.329+0000 7f1787589640 1 --2- 172.21.15.40:0/2512126344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800d2130 0x7f17800d2e50 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.329+0000 7f1787589640 1 --2- 172.21.15.40:0/2512126344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d1770 0x7f17800d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.329+0000 7f1787589640 1 --2- 172.21.15.40:0/2512126344 >> v2:172.21.15.184:3300/0 conn(0x7f178015f3a0 0x7f17800d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.329+0000 7f1787589640 1 -- 172.21.15.40:0/2512126344 >> 172.21.15.40:0/2512126344 conn(0x7f17800d4f90 msgr2=0x7f17800d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.329+0000 7f1787589640 1 -- 172.21.15.40:0/2512126344 shutdown_connections 2024-04-26T21:46:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.329+0000 7f1787589640 1 -- 172.21.15.40:0/2512126344 wait complete. 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.330+0000 7f1787589640 1 Processor -- start 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.330+0000 7f1787589640 1 -- start start 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1787589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17800d1770 0x7f178015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1787589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d2130 0x7f1780159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1787589640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178015f3a0 0x7f17800b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1787589640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f178015f900 con 0x7f178015f3a0 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1787589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f178015f780 con 0x7f17800d1770 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1787589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17800d3390 con 0x7f17800d2130 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178015f3a0 0x7f17800b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178015f3a0 0x7f17800b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40290/0 (socket says 172.21.15.40:40290) 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 -- 172.21.15.40:0/41057626 learned_addr learned my addr 172.21.15.40:0/41057626 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785325640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17800d1770 0x7f178015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1784b24640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d2130 0x7f1780159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 -- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d2130 msgr2=0x7f1780159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d2130 0x7f1780159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 -- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17800d1770 msgr2=0x7f178015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17800d1770 0x7f178015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 -- 172.21.15.40:0/41057626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178015fc20 con 0x7f178015f3a0 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1784b24640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d2130 0x7f1780159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1785b26640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178015f3a0 0x7f17800b54d0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f1778034040 tx=0x7f1778004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1775ffb640 1 -- 172.21.15.40:0/41057626 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778036050 con 0x7f178015f3a0 2024-04-26T21:46:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.331+0000 7f1775ffb640 1 -- 172.21.15.40:0/41057626 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778042070 con 0x7f178015f3a0 2024-04-26T21:46:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.332+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178015ff00 con 0x7f178015f3a0 2024-04-26T21:46:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.332+0000 7f1775ffb640 1 -- 172.21.15.40:0/41057626 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778009510 con 0x7f178015f3a0 2024-04-26T21:46:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.332+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17801604c0 con 0x7f178015f3a0 2024-04-26T21:46:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.333+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1780162d20 con 0x7f178015f3a0 2024-04-26T21:46:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.333+0000 7f1775ffb640 1 -- 172.21.15.40:0/41057626 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1778007bf0 con 0x7f178015f3a0 2024-04-26T21:46:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.334+0000 7f1775ffb640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.334+0000 7f1775ffb640 1 -- 172.21.15.40:0/41057626 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f17780ad660 con 0x7f178015f3a0 2024-04-26T21:46:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.334+0000 7f1785325640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.335+0000 7f1785325640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f176406c8f0 0x7f176406edb0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f178015d1e0 tx=0x7f177c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.338+0000 7f1775ffb640 1 -- 172.21.15.40:0/41057626 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f177807ac50 con 0x7f178015f3a0 2024-04-26T21:46:32.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.561+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17800694f0 con 0x7f178015f3a0 2024-04-26T21:46:32.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.561+0000 7f1775ffb640 1 -- 172.21.15.40:0/41057626 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 323 v323) ==== 95+0+5193 (secure 0 0 0) 0x7f177807a550 con 0x7f178015f3a0 2024-04-26T21:46:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:32.563 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":323,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":323,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:31.794920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11238":{"gid":11238,"name":"c","rank":0,"incarnation":323,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/1769689018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1769689018},{"type":"v1","addr":"172.21.15.40:6833","nonce":1769689018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:32.563 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 323 2024-04-26T21:46:32.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.563+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f176406c8f0 msgr2=0x7f176406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.563+0000 7f1787589640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f176406c8f0 0x7f176406edb0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f178015d1e0 tx=0x7f177c009190 comp rx=0 tx=0).stop 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.563+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178015f3a0 msgr2=0x7f17800b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.563+0000 7f1787589640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178015f3a0 0x7f17800b54d0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f1778034040 tx=0x7f1778004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.564+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 shutdown_connections 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.564+0000 7f1787589640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.564+0000 7f1787589640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178015f3a0 0x7f17800b54d0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.564+0000 7f1787589640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17800d2130 0x7f1780159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.564+0000 7f1787589640 1 --2- 172.21.15.40:0/41057626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17800d1770 0x7f178015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.564+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 >> 172.21.15.40:0/41057626 conn(0x7f17800d4f90 msgr2=0x7f17800d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.564+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 shutdown_connections 2024-04-26T21:46:32.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:32.564+0000 7f1787589640 1 -- 172.21.15.40:0/41057626 wait complete. 2024-04-26T21:46:32.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 323, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:46:31.794920+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 131, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11238}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11238': {'gid': 11238, 'name': 'c', 'rank': 0, 'incarnation': 323, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6833/1769689018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1769689018}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 1769689018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:46:32.575 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11238:up:replay 2024-04-26T21:46:32.575 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:46:36.778 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:36.777+0000 7f4832cf2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:46:37.980 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.094+0000 7f8c4ab5f640 1 Processor -- start 2024-04-26T21:46:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.094+0000 7f8c4ab5f640 1 -- start start 2024-04-26T21:46:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c4ab5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c4415d1e0 0x7f8c440d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c4ab5f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8c440d0f20 0x7f8c440d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c4ab5f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c440ce840 con 0x7f8c440d18c0 2024-04-26T21:46:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c4ab5f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8c440cede0 con 0x7f8c4415d1e0 2024-04-26T21:46:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c4ab5f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8c44070650 con 0x7f8c440d0f20 2024-04-26T21:46:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c488fb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c4415d1e0 0x7f8c440d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c490fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8c440d18c0 0x7f8c440cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39784/0 (socket says 172.21.15.40:39784) 2024-04-26T21:46:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c488fb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c4415d1e0 0x7f8c440d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40294/0 (socket says 172.21.15.40:40294) 2024-04-26T21:46:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c490fc640 1 -- 172.21.15.40:0/1369112606 learned_addr learned my addr 172.21.15.40:0/1369112606 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c43fff640 1 --2- 172.21.15.40:0/1369112606 >> v2:172.21.15.184:3300/0 conn(0x7f8c440d0f20 0x7f8c440d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c488fb640 1 -- 172.21.15.40:0/1369112606 >> v2:172.21.15.184:3300/0 conn(0x7f8c440d0f20 msgr2=0x7f8c440d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c488fb640 1 --2- 172.21.15.40:0/1369112606 >> v2:172.21.15.184:3300/0 conn(0x7f8c440d0f20 0x7f8c440d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c488fb640 1 -- 172.21.15.40:0/1369112606 >> v1:172.21.15.40:6789/0 conn(0x7f8c440d18c0 legacy=0x7f8c440cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.095+0000 7f8c488fb640 1 -- 172.21.15.40:0/1369112606 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c440ceb00 con 0x7f8c4415d1e0 2024-04-26T21:46:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.096+0000 7f8c488fb640 1 --2- 172.21.15.40:0/1369112606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c440d09e0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f8c440c64b0 tx=0x7f8c3802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245b816a55ad4f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.096+0000 7f8c437fe640 1 -- 172.21.15.40:0/1369112606 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38037070 con 0x7f8c4415d1e0 2024-04-26T21:46:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.096+0000 7f8c437fe640 1 -- 172.21.15.40:0/1369112606 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c38032440 con 0x7f8c4415d1e0 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.096+0000 7f8c437fe640 1 -- 172.21.15.40:0/1369112606 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38032810 con 0x7f8c4415d1e0 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.097+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/1369112606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 msgr2=0x7f8c440d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.097+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/1369112606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c440d09e0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f8c440c64b0 tx=0x7f8c3802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.097+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/1369112606 shutdown_connections 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.097+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/1369112606 >> v2:172.21.15.184:3300/0 conn(0x7f8c440d0f20 0x7f8c440d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.097+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/1369112606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c440d09e0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.097+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/1369112606 >> 172.21.15.40:0/1369112606 conn(0x7f8c440d2da0 msgr2=0x7f8c440d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.097+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/1369112606 shutdown_connections 2024-04-26T21:46:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.097+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/1369112606 wait complete. 2024-04-26T21:46:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.098+0000 7f8c4ab5f640 1 Processor -- start 2024-04-26T21:46:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.098+0000 7f8c4ab5f640 1 -- start start 2024-04-26T21:46:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c4ab5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c440d0f20 0x7f8c440c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c4ab5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c440d18c0 0x7f8c44157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c4ab5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c441f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c4ab5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44071af0 con 0x7f8c4415d1e0 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c4ab5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c440ce4c0 con 0x7f8c440d18c0 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c4ab5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c44071c70 con 0x7f8c440d0f20 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c490fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c441f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c490fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c441f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40306/0 (socket says 172.21.15.40:40306) 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c490fc640 1 -- 172.21.15.40:0/428287851 learned_addr learned my addr 172.21.15.40:0/428287851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c488fb640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c440d0f20 0x7f8c440c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.099+0000 7f8c43fff640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c440d18c0 0x7f8c44157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c490fc640 1 -- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c440d0f20 msgr2=0x7f8c440c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c490fc640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c440d0f20 0x7f8c440c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c490fc640 1 -- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c440d18c0 msgr2=0x7f8c44157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c490fc640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c440d18c0 0x7f8c44157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c490fc640 1 -- 172.21.15.40:0/428287851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c441f5a60 con 0x7f8c4415d1e0 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c43fff640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c440d18c0 0x7f8c44157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c490fc640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c441f4e90 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00cbc0 tx=0x7f8c3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c417fa640 1 -- 172.21.15.40:0/428287851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c017070 con 0x7f8c4415d1e0 2024-04-26T21:46:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c417fa640 1 -- 172.21.15.40:0/428287851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00ad40 con 0x7f8c4415d1e0 2024-04-26T21:46:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c441f5d40 con 0x7f8c4415d1e0 2024-04-26T21:46:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c417fa640 1 -- 172.21.15.40:0/428287851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c012600 con 0x7f8c4415d1e0 2024-04-26T21:46:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.100+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c440c9bd0 con 0x7f8c4415d1e0 2024-04-26T21:46:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.101+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c440694f0 con 0x7f8c4415d1e0 2024-04-26T21:46:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.101+0000 7f8c417fa640 1 -- 172.21.15.40:0/428287851 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c3c01a070 con 0x7f8c4415d1e0 2024-04-26T21:46:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.102+0000 7f8c417fa640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8c2806c8f0 0x7f8c2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.102+0000 7f8c417fa640 1 -- 172.21.15.40:0/428287851 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f8c3c08c710 con 0x7f8c4415d1e0 2024-04-26T21:46:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.102+0000 7f8c488fb640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8c2806c8f0 0x7f8c2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.103+0000 7f8c488fb640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8c2806c8f0 0x7f8c2806edb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8c38034000 tx=0x7f8c3802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.107+0000 7f8c417fa640 1 -- 172.21.15.40:0/428287851 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c3c059600 con 0x7f8c4415d1e0 2024-04-26T21:46:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.333+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c440afd00 con 0x7f8c4415d1e0 2024-04-26T21:46:38.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.333+0000 7f8c417fa640 1 -- 172.21.15.40:0/428287851 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 326 v326) ==== 95+0+5202 (secure 0 0 0) 0x7f8c3c05d900 con 0x7f8c4415d1e0 2024-04-26T21:46:38.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:38.334 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":326,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":326,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:34.809681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11238":{"gid":11238,"name":"c","rank":0,"incarnation":323,"state":"up:active","state_seq":5,"addr":"172.21.15.40:6833/1769689018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1769689018},{"type":"v1","addr":"172.21.15.40:6833","nonce":1769689018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11238,"qdb_cluster":[11238]},"id":6}]} 2024-04-26T21:46:38.334 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 326 2024-04-26T21:46:38.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8c2806c8f0 msgr2=0x7f8c2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8c2806c8f0 0x7f8c2806edb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8c38034000 tx=0x7f8c3802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 msgr2=0x7f8c441f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c441f4e90 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00cbc0 tx=0x7f8c3c00a430 comp rx=0 tx=0).stop 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 shutdown_connections 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8c2806c8f0 0x7f8c2806edb0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4415d1e0 0x7f8c441f4e90 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c440d18c0 0x7f8c44157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 --2- 172.21.15.40:0/428287851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c440d0f20 0x7f8c440c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.335+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 >> 172.21.15.40:0/428287851 conn(0x7f8c440d2da0 msgr2=0x7f8c440b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.336+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 shutdown_connections 2024-04-26T21:46:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.336+0000 7f8c4ab5f640 1 -- 172.21.15.40:0/428287851 wait complete. 2024-04-26T21:46:38.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 326, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:42:08.607413+0000', 'modified': '2024-04-26T21:46:34.809681+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 131, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11238}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11238': {'gid': 11238, 'name': 'c', 'rank': 0, 'incarnation': 323, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.40:6833/1769689018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1769689018}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 1769689018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11238, 'qdb_cluster': [11238]} 2024-04-26T21:46:38.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:46:38.347 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:46:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.448+0000 7fe19c5f0640 1 Processor -- start 2024-04-26T21:46:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.448+0000 7fe19c5f0640 1 -- start start 2024-04-26T21:46:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.449+0000 7fe19c5f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe19415f3a0 0x7fe1940d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.449+0000 7fe19c5f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe1940d1770 0x7fe1940d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.449+0000 7fe19c5f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d2160 0x7fe1940d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.449+0000 7fe19c5f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe1940d25d0 con 0x7fe19415f3a0 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.449+0000 7fe19c5f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe1940d0e20 con 0x7fe1940d1770 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.449+0000 7fe19c5f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1940b1d70 con 0x7fe1940d2160 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.449+0000 7fe19a38c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe19415f3a0 0x7fe1940d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19a38c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe19415f3a0 0x7fe1940d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40322/0 (socket says 172.21.15.40:40322) 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19a38c640 1 -- 172.21.15.40:0/1062302571 learned_addr learned my addr 172.21.15.40:0/1062302571 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe199b8b640 1 --2- 172.21.15.40:0/1062302571 >> v2:172.21.15.184:3300/0 conn(0x7fe1940d1770 0x7fe1940d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/1062302571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d2160 0x7fe1940d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19a38c640 1 -- 172.21.15.40:0/1062302571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d2160 msgr2=0x7fe1940d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1062302571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d2160 0x7fe1940d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19a38c640 1 -- 172.21.15.40:0/1062302571 >> v2:172.21.15.184:3300/0 conn(0x7fe1940d1770 msgr2=0x7fe1940d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1062302571 >> v2:172.21.15.184:3300/0 conn(0x7fe1940d1770 0x7fe1940d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19a38c640 1 -- 172.21.15.40:0/1062302571 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1940b2050 con 0x7fe19415f3a0 2024-04-26T21:46:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe199b8b640 1 --2- 172.21.15.40:0/1062302571 >> v2:172.21.15.184:3300/0 conn(0x7fe1940d1770 0x7fe1940d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.450+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/1062302571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d2160 0x7fe1940d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1062302571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19415f3a0 0x7fe1940d06f0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fe1940b7b80 tx=0x7fe18c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7874c55f92c2afe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19938a640 1 -- 172.21.15.40:0/1062302571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c037070 con 0x7fe19415f3a0 2024-04-26T21:46:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19938a640 1 -- 172.21.15.40:0/1062302571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe18c032440 con 0x7fe19415f3a0 2024-04-26T21:46:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19938a640 1 -- 172.21.15.40:0/1062302571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c032810 con 0x7fe19415f3a0 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1062302571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19415f3a0 msgr2=0x7fe1940d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1062302571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19415f3a0 0x7fe1940d06f0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fe1940b7b80 tx=0x7fe18c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1062302571 shutdown_connections 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1062302571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d2160 0x7fe1940d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1062302571 >> v2:172.21.15.184:3300/0 conn(0x7fe1940d1770 0x7fe1940d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1062302571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19415f3a0 0x7fe1940d06f0 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1062302571 >> 172.21.15.40:0/1062302571 conn(0x7fe1940d4f90 msgr2=0x7fe1940d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1062302571 shutdown_connections 2024-04-26T21:46:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.451+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1062302571 wait complete. 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe19c5f0640 1 Processor -- start 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe19c5f0640 1 -- start start 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe19c5f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d1770 0x7fe1941502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe19c5f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1940d2160 0x7fe19415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe19c5f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe19415f3a0 0x7fe19415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe19c5f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe19415f900 con 0x7fe1940d2160 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe19c5f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe19415f780 con 0x7fe19415f3a0 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe19c5f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1940d3390 con 0x7fe1940d1770 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe199b8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1940d2160 0x7fe19415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe199b8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1940d2160 0x7fe19415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40336/0 (socket says 172.21.15.40:40336) 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.453+0000 7fe199b8b640 1 -- 172.21.15.40:0/1349949593 learned_addr learned my addr 172.21.15.40:0/1349949593 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d1770 0x7fe1941502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe19415f3a0 0x7fe19415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe199b8b640 1 -- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d1770 msgr2=0x7fe1941502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe199b8b640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d1770 0x7fe1941502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe199b8b640 1 -- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe19415f3a0 msgr2=0x7fe19415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe199b8b640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe19415f3a0 0x7fe19415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe199b8b640 1 -- 172.21.15.40:0/1349949593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1940b87d0 con 0x7fe1940d2160 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d1770 0x7fe1941502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe19415f3a0 0x7fe19415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe199b8b640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1940d2160 0x7fe19415d860 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fe18800e810 tx=0x7fe18800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe192ffd640 1 -- 172.21.15.40:0/1349949593 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18800f040 con 0x7fe1940d2160 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe192ffd640 1 -- 172.21.15.40:0/1349949593 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188004030 con 0x7fe1940d2160 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe192ffd640 1 -- 172.21.15.40:0/1349949593 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188004400 con 0x7fe1940d2160 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1349949593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1940b8ab0 con 0x7fe1940d2160 2024-04-26T21:46:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.454+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1349949593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1940b9070 con 0x7fe1940d2160 2024-04-26T21:46:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.456+0000 7fe192ffd640 1 -- 172.21.15.40:0/1349949593 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe18800c6d0 con 0x7fe1940d2160 2024-04-26T21:46:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.456+0000 7fe192ffd640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe18006c8f0 0x7fe18006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.456+0000 7fe192ffd640 1 -- 172.21.15.40:0/1349949593 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7fe18808c630 con 0x7fe1940d2160 2024-04-26T21:46:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.456+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe18006c8f0 0x7fe18006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.457+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe18006c8f0 0x7fe18006edb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe18c034000 tx=0x7fe18c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.457+0000 7fe19f3e9740 1 Processor -- start 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.457+0000 7fe19f3e9740 1 -- start start 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f875820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f893230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f84b120 0x56425f8913a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56425f7af740 con 0x56425f892ce0 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56425f74b3a0 con 0x56425f84b120 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56425f7ab700 con 0x56425f875440 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe199b8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f893230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe199b8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f893230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40350/0 (socket says 172.21.15.40:40350) 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe199b8b640 1 -- 172.21.15.40:0/559371144 learned_addr learned my addr 172.21.15.40:0/559371144 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe19a38c640 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f875820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f84b120 0x56425f8913a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe199b8b640 1 -- 172.21.15.40:0/559371144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 msgr2=0x56425f875820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe199b8b640 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f875820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe199b8b640 1 -- 172.21.15.40:0/559371144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f84b120 msgr2=0x56425f8913a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe199b8b640 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f84b120 0x56425f8913a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.458+0000 7fe199b8b640 1 -- 172.21.15.40:0/559371144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425f767750 con 0x56425f892ce0 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe199b8b640 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f893230 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fe188009fd0 tx=0x7fe1880596c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5400e50b3d787497 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe17bfff640 1 -- 172.21.15.40:0/559371144 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188020040 con 0x56425f892ce0 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe17bfff640 1 -- 172.21.15.40:0/559371144 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188057880 con 0x56425f892ce0 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe17bfff640 1 -- 172.21.15.40:0/559371144 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188061cd0 con 0x56425f892ce0 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe19f3e9740 1 -- 172.21.15.40:0/559371144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 msgr2=0x56425f893230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f893230 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fe188009fd0 tx=0x7fe1880596c0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe19f3e9740 1 -- 172.21.15.40:0/559371144 shutdown_connections 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f84b120 0x56425f8913a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f893230 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/559371144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f875820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.459+0000 7fe19f3e9740 1 -- 172.21.15.40:0/559371144 >> 172.21.15.40:0/559371144 conn(0x56425f847a50 msgr2=0x56425f890250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.460+0000 7fe19f3e9740 1 -- 172.21.15.40:0/559371144 shutdown_connections 2024-04-26T21:46:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.460+0000 7fe19f3e9740 1 -- 172.21.15.40:0/559371144 wait complete. 2024-04-26T21:46:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.460+0000 7fe19f3e9740 1 -- start start 2024-04-26T21:46:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.460+0000 7fe19f3e9740 1 Processor -- start 2024-04-26T21:46:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f875440 0x56425f8a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f892ce0 0x56425f888160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 0x56425f888c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56425f84bec0 con 0x56425f875440 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56425f736fe0 con 0x56425f878510 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56425f845980 con 0x56425f892ce0 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19f3e9740 10 client.0 _ll_register_callbacks cb 0x56425f74d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19a38c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f875440 0x56425f8a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19a38c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f875440 0x56425f8a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40352/0 (socket says 172.21.15.40:40352) 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19a38c640 1 -- 172.21.15.40:0/1574361077 learned_addr learned my addr 172.21.15.40:0/1574361077 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe199b8b640 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f892ce0 0x56425f888160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 0x56425f888c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19a38c640 1 -- 172.21.15.40:0/1574361077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f892ce0 msgr2=0x56425f888160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f892ce0 0x56425f888160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19a38c640 1 -- 172.21.15.40:0/1574361077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 msgr2=0x56425f888c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 0x56425f888c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.461+0000 7fe19a38c640 1 -- 172.21.15.40:0/1574361077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425f4c12d0 con 0x56425f875440 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19a38c640 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f875440 0x56425f8a10a0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fe18c02a7d0 tx=0x7fe18c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3c48276c5aca36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe1797fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:46:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe1797fa640 1 -- 172.21.15.40:0/1574361077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c037040 con 0x56425f875440 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe1797fa640 1 -- 172.21.15.40:0/1574361077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7fe18c03caf0 con 0x56425f875440 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 1 -- 172.21.15.40:0/1574361077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe188057880 con 0x56425f875440 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe1797fa640 1 -- 172.21.15.40:0/1574361077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=326) ==== 24+0+0 (secure 0 0 0) 0x7fe18c056050 con 0x56425f875440 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 10 client.11328 fetch_fsmap learned FSMap version 326 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 1 -- 172.21.15.40:0/1574361077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=326,monmap=2+}) -- 0x56425f58b6b0 con 0x56425f875440 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe1797fa640 1 -- 172.21.15.40:0/1574361077 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188057880 con 0x56425f875440 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe1797fa640 1 -- 172.21.15.40:0/1574361077 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 326) ==== 3494+0+0 (secure 0 0 0) 0x7fe18c03ccf0 con 0x56425f875440 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 10 client.11328 fetch_fsmap finished waiting for FSMap version 326 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 10 client.11328 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 20 client.11328 populate_metadata read hostname 'smithi040' 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] conn(0x56425f8a3dc0 0x56425f8a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 4 client.11328 mds_command: new command op to 11238 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe19f3e9740 1 -- 172.21.15.40:0/1574361077 --> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425f7bd350 con 0x56425f8a3dc0 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.462+0000 7fe199b8b640 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] conn(0x56425f8a3dc0 0x56425f8a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.463+0000 7fe1797fa640 10 client.11328 ms_handle_connect on v2:172.21.15.40:6832/1769689018 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.463+0000 7fe199b8b640 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] conn(0x56425f8a3dc0 0x56425f8a41a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce762340223354c4 server_cookie=5272257abe2e21b9 in_seq=0 out_seq=0 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.463+0000 7fe1797fa640 1 -- 172.21.15.40:0/1574361077 <== mds.0 v2:172.21.15.40:6832/1769689018 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56425f7bd350 con 0x56425f8a3dc0 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.463+0000 7fe1797fa640 10 client.11328 handle_command_reply: tid=0 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.463+0000 7fe19f3e9740 1 client.11328 shutdown 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 -- 172.21.15.40:0/1574361077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f875440 msgr2=0x56425f8a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f875440 0x56425f8a10a0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fe18c02a7d0 tx=0x7fe18c02c070 comp rx=0 tx=0).stop 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 -- 172.21.15.40:0/1574361077 shutdown_connections 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] conn(0x56425f8a3dc0 0x56425f8a41a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 0x56425f888c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f892ce0 0x56425f888160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe1797fa640 0 client.11328 ms_handle_reset on v2:172.21.15.40:6832/1769689018 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/1574361077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f875440 0x56425f8a10a0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 -- 172.21.15.40:0/1574361077 >> 172.21.15.40:0/1574361077 conn(0x56425f84b120 msgr2=0x56425f894060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 -- 172.21.15.40:0/1574361077 shutdown_connections 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 1 -- 172.21.15.40:0/1574361077 wait complete. 2024-04-26T21:46:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.464+0000 7fe19f3e9740 20 client.11328 trim_cache size 0 max 16384 2024-04-26T21:46:38.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.482+0000 7fe19f3e9740 1 Processor -- start 2024-04-26T21:46:38.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.482+0000 7fe19f3e9740 1 -- start start 2024-04-26T21:46:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f8b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f878510 0x56425f844670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f892ce0 0x56425f88ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe194202d40 con 0x56425f878510 2024-04-26T21:46:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe194202080 con 0x56425f892ce0 2024-04-26T21:46:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1942026d0 con 0x56425f875440 2024-04-26T21:46:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f878510 0x56425f844670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f878510 0x56425f844670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40368/0 (socket says 172.21.15.40:40368) 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 -- 172.21.15.40:0/3472487411 learned_addr learned my addr 172.21.15.40:0/3472487411 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 -- 172.21.15.40:0/3472487411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 msgr2=0x56425f8b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 --2- 172.21.15.40:0/3472487411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f8b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 -- 172.21.15.40:0/3472487411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f892ce0 msgr2=0x56425f88ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 --2- 172.21.15.40:0/3472487411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f892ce0 0x56425f88ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 -- 172.21.15.40:0/3472487411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425f875b90 con 0x56425f878510 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe199b8b640 1 --2- 172.21.15.40:0/3472487411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f878510 0x56425f844670 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fe188059510 tx=0x7fe18801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7327360959fe399c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe17b7fe640 1 -- 172.21.15.40:0/3472487411 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188020040 con 0x56425f878510 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe17b7fe640 1 -- 172.21.15.40:0/3472487411 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188055bf0 con 0x56425f878510 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.483+0000 7fe17b7fe640 1 -- 172.21.15.40:0/3472487411 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188061ed0 con 0x56425f878510 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 -- 172.21.15.40:0/3472487411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f878510 msgr2=0x56425f844670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/3472487411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f878510 0x56425f844670 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fe188059510 tx=0x7fe18801d000 comp rx=0 tx=0).stop 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 -- 172.21.15.40:0/3472487411 shutdown_connections 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/3472487411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f892ce0 0x56425f88ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/3472487411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f878510 0x56425f844670 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/3472487411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f8b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 -- 172.21.15.40:0/3472487411 >> 172.21.15.40:0/3472487411 conn(0x56425f84b120 msgr2=0x56425f84b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 -- 172.21.15.40:0/3472487411 shutdown_connections 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 -- 172.21.15.40:0/3472487411 wait complete. 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.484+0000 7fe19f3e9740 1 -- start start 2024-04-26T21:46:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.485+0000 7fe19f3e9740 1 Processor -- start 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.485+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f8a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.485+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 0x56425f891fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.485+0000 7fe19f3e9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f90ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.485+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56425f8765d0 con 0x56425f892ce0 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.485+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56425f887c40 con 0x56425f878510 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.485+0000 7fe19f3e9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56425f870d90 con 0x56425f875440 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.485+0000 7fe19f3e9740 10 client.0 _ll_register_callbacks cb 0x7fe18c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f90ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f90ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40382/0 (socket says 172.21.15.40:40382) 2024-04-26T21:46:38.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 -- 172.21.15.40:0/919933160 learned_addr learned my addr 172.21.15.40:0/919933160 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19a38c640 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f8a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe199b8b640 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 0x56425f891fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 -- 172.21.15.40:0/919933160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 msgr2=0x56425f8a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f8a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 -- 172.21.15.40:0/919933160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 msgr2=0x56425f891fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 0x56425f891fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 -- 172.21.15.40:0/919933160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425f90a050 con 0x56425f892ce0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19ab8d640 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f90ab10 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fe184009540 tx=0x7fe184011600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33decb5c8077d6be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe1797fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe1797fa640 1 -- 172.21.15.40:0/919933160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18401c070 con 0x56425f892ce0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe1797fa640 1 -- 172.21.15.40:0/919933160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7fe184011de0 con 0x56425f892ce0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19f3e9740 1 -- 172.21.15.40:0/919933160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe188055bf0 con 0x56425f892ce0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe1797fa640 1 -- 172.21.15.40:0/919933160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=326) ==== 24+0+0 (secure 0 0 0) 0x7fe184003990 con 0x56425f892ce0 2024-04-26T21:46:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19f3e9740 10 client.11334 fetch_fsmap learned FSMap version 326 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.486+0000 7fe19f3e9740 1 -- 172.21.15.40:0/919933160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=326,monmap=2+}) -- 0x56425f876f00 con 0x56425f892ce0 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe1797fa640 1 -- 172.21.15.40:0/919933160 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188055bf0 con 0x56425f892ce0 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe1797fa640 1 -- 172.21.15.40:0/919933160 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 326) ==== 3494+0+0 (secure 0 0 0) 0x7fe184003b30 con 0x56425f892ce0 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 10 client.11334 fetch_fsmap finished waiting for FSMap version 326 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 10 client.11334 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 20 client.11334 populate_metadata read hostname 'smithi040' 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] conn(0x56425f8f2850 0x56425f8f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 4 client.11334 mds_command: new command op to 11238 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 1 -- 172.21.15.40:0/919933160 --> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] -- command(tid 0: {"prefix": "status"}) -- 0x56425f7b71c0 con 0x56425f8f2850 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19a38c640 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] conn(0x56425f8f2850 0x56425f8f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19a38c640 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] conn(0x56425f8f2850 0x56425f8f2c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7344609e4f36c7cc server_cookie=fe98ad9238f7fb23 in_seq=0 out_seq=0 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe1797fa640 10 client.11334 ms_handle_connect on v2:172.21.15.40:6832/1769689018 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe1797fa640 1 -- 172.21.15.40:0/919933160 <== mds.0 v2:172.21.15.40:6832/1769689018 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x56425f7b71c0 con 0x56425f8f2850 2024-04-26T21:46:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe1797fa640 10 client.11334 handle_command_reply: tid=0 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 1 client.11334 shutdown 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 1 -- 172.21.15.40:0/919933160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 msgr2=0x56425f90ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.487+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f90ab10 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fe184009540 tx=0x7fe184011600 comp rx=0 tx=0).stop 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 1 -- 172.21.15.40:0/919933160 shutdown_connections 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.40:6832/1769689018,v1:172.21.15.40:6833/1769689018] conn(0x56425f8f2850 0x56425f8f2c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56425f892ce0 0x56425f90ab10 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56425f878510 0x56425f891fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 1 --2- 172.21.15.40:0/919933160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56425f875440 0x56425f8a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe1797fa640 0 client.11334 ms_handle_reset on v2:172.21.15.40:6832/1769689018 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 1 -- 172.21.15.40:0/919933160 >> 172.21.15.40:0/919933160 conn(0x56425f84b120 msgr2=0x56425f888a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 1 -- 172.21.15.40:0/919933160 shutdown_connections 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 1 -- 172.21.15.40:0/919933160 wait complete. 2024-04-26T21:46:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19f3e9740 20 client.11334 trim_cache size 0 max 16384 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 11238, 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.6896748150000001, 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 326, 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 131, 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 131, 2024-04-26T21:46:38.495 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 12.687246909000001 2024-04-26T21:46:38.496 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe18006c8f0 msgr2=0x7fe18006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe18006c8f0 0x7fe18006edb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe18c034000 tx=0x7fe18c02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1349949593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1940d2160 msgr2=0x7fe19415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1940d2160 0x7fe19415d860 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fe18800e810 tx=0x7fe18800ecd0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1349949593 shutdown_connections 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe18006c8f0 0x7fe18006edb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe19415f3a0 0x7fe19415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1940d2160 0x7fe19415d860 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 --2- 172.21.15.40:0/1349949593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1940d1770 0x7fe1941502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1349949593 >> 172.21.15.40:0/1349949593 conn(0x7fe1940d4f90 msgr2=0x7fe1940d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1349949593 shutdown_connections 2024-04-26T21:46:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.488+0000 7fe19c5f0640 1 -- 172.21.15.40:0/1349949593 wait complete. 2024-04-26T21:46:38.499 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:46:38.500 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on mds0_openfiles.0', proceeding to ls 2024-04-26T21:46:38.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:46:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.599+0000 7f9a547c1640 1 Processor -- start 2024-04-26T21:46:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.599+0000 7f9a547c1640 1 -- start start 2024-04-26T21:46:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.600+0000 7f9a547c1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9a4c162810 0x7f9a4c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.600+0000 7f9a547c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.600+0000 7f9a547c1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4c0d10d0 con 0x7f9a4c0cd930 2024-04-26T21:46:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.600+0000 7f9a547c1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9a4c0d0750 con 0x7f9a4c162810 2024-04-26T21:46:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.600+0000 7f9a547c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4c0d16a0 con 0x7f9a4c0b2340 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a51d5c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a5255d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9a4c162810 0x7f9a4c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a51d5c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:38842/0 (socket says 172.21.15.40:38842) 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a5255d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9a4c162810 0x7f9a4c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:53744/0 (socket says 172.21.15.40:53744) 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a51d5c640 1 -- 172.21.15.40:0/2990784042 learned_addr learned my addr 172.21.15.40:0/2990784042 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a52d5e640 1 --1- 172.21.15.40:0/2990784042 >> v1:172.21.15.184:6789/0 conn(0x7f9a4c0cd930 0x7f9a4c0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:33058/0 (socket says 172.21.15.40:33058) 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a51d5c640 1 -- 172.21.15.40:0/2990784042 >> v2:172.21.15.184:3300/0 conn(0x7f9a4c162810 msgr2=0x7f9a4c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a51d5c640 1 --2- 172.21.15.40:0/2990784042 >> v2:172.21.15.184:3300/0 conn(0x7f9a4c162810 0x7f9a4c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a51d5c640 1 -- 172.21.15.40:0/2990784042 >> v1:172.21.15.184:6789/0 conn(0x7f9a4c0cd930 legacy=0x7f9a4c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.601+0000 7f9a51d5c640 1 -- 172.21.15.40:0/2990784042 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c0d1370 con 0x7f9a4c0b2340 2024-04-26T21:46:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a51d5c640 1 --2- 172.21.15.40:0/2990784042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cd3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009870 tx=0x7f9a4002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef363657e3913d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a5155b640 1 -- 172.21.15.40:0/2990784042 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40037070 con 0x7f9a4c0b2340 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a5155b640 1 -- 172.21.15.40:0/2990784042 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a40032440 con 0x7f9a4c0b2340 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a5155b640 1 -- 172.21.15.40:0/2990784042 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40032810 con 0x7f9a4c0b2340 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a547c1640 1 -- 172.21.15.40:0/2990784042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 msgr2=0x7f9a4c0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2990784042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cd3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009870 tx=0x7f9a4002a270 comp rx=0 tx=0).stop 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a547c1640 1 -- 172.21.15.40:0/2990784042 shutdown_connections 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2990784042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cd3f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2990784042 >> v2:172.21.15.184:3300/0 conn(0x7f9a4c162810 0x7f9a4c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.602+0000 7f9a547c1640 1 -- 172.21.15.40:0/2990784042 >> 172.21.15.40:0/2990784042 conn(0x7f9a4c0d2da0 msgr2=0x7f9a4c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.603+0000 7f9a547c1640 1 -- 172.21.15.40:0/2990784042 shutdown_connections 2024-04-26T21:46:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.603+0000 7f9a547c1640 1 -- 172.21.15.40:0/2990784042 wait complete. 2024-04-26T21:46:38.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.604+0000 7f9a547c1640 1 Processor -- start 2024-04-26T21:46:38.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.604+0000 7f9a547c1640 1 -- start start 2024-04-26T21:46:38.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.604+0000 7f9a547c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.604+0000 7f9a547c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c0cd930 0x7f9a4c0cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.604+0000 7f9a547c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c162810 0x7f9a4c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.604+0000 7f9a547c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c070890 con 0x7f9a4c0cd930 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.604+0000 7f9a547c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c0d0470 con 0x7f9a4c162810 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.604+0000 7f9a547c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4c0d2240 con 0x7f9a4c0b2340 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c0cd930 0x7f9a4c0cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a5255d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c0cd930 0x7f9a4c0cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40398/0 (socket says 172.21.15.40:40398) 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a52d5e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c162810 0x7f9a4c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a5255d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:38848/0 (socket says 172.21.15.40:38848) 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 -- 172.21.15.40:0/2919536756 learned_addr learned my addr 172.21.15.40:0/2919536756 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 -- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 msgr2=0x7f9a4c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 -- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c162810 msgr2=0x7f9a4c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c162810 0x7f9a4c0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 -- 172.21.15.40:0/2919536756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c0ca880 con 0x7f9a4c0cd930 2024-04-26T21:46:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a5255d640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:38.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a51d5c640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c0cd930 0x7f9a4c0cb8d0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f9a4002a2b0 tx=0x7f9a400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a4affd640 1 -- 172.21.15.40:0/2919536756 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40037040 con 0x7f9a4c0cd930 2024-04-26T21:46:38.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a4affd640 1 -- 172.21.15.40:0/2919536756 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a40042070 con 0x7f9a4c0cd930 2024-04-26T21:46:38.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c0cab60 con 0x7f9a4c0cd930 2024-04-26T21:46:38.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a4affd640 1 -- 172.21.15.40:0/2919536756 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40030860 con 0x7f9a4c0cd930 2024-04-26T21:46:38.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.605+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c0cb120 con 0x7f9a4c0cd930 2024-04-26T21:46:38.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.606+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c0694f0 con 0x7f9a4c0cd930 2024-04-26T21:46:38.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.607+0000 7f9a4affd640 1 -- 172.21.15.40:0/2919536756 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a40030a00 con 0x7f9a4c0cd930 2024-04-26T21:46:38.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.607+0000 7f9a4affd640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a3806c8f0 0x7f9a3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:38.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.608+0000 7f9a5255d640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a3806c8f0 0x7f9a3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:38.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.608+0000 7f9a4affd640 1 -- 172.21.15.40:0/2919536756 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f9a400ad170 con 0x7f9a4c0cd930 2024-04-26T21:46:38.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.608+0000 7f9a5255d640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a3806c8f0 0x7f9a3806edb0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0b17e0 tx=0x7f9a44009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.612+0000 7f9a4affd640 1 -- 172.21.15.40:0/2919536756 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a4007a110 con 0x7f9a4c0cd930 2024-04-26T21:46:38.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.837+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9a4c0cc470 con 0x7f9a4c0cd930 2024-04-26T21:46:38.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.838+0000 7f9a4affd640 1 -- 172.21.15.40:0/2919536756 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v326) ==== 53+0+73 (secure 0 0 0) 0x7f9a4007e410 con 0x7f9a4c0cd930 2024-04-26T21:46:38.839 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:46:38.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a3806c8f0 msgr2=0x7f9a3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a3806c8f0 0x7f9a3806edb0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0b17e0 tx=0x7f9a44009190 comp rx=0 tx=0).stop 2024-04-26T21:46:38.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c0cd930 msgr2=0x7f9a4c0cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:38.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c0cd930 0x7f9a4c0cb8d0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f9a4002a2b0 tx=0x7f9a400046c0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 shutdown_connections 2024-04-26T21:46:38.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9a3806c8f0 0x7f9a3806edb0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c162810 0x7f9a4c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c0cd930 0x7f9a4c0cb8d0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 --2- 172.21.15.40:0/2919536756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c0b2340 0x7f9a4c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:38.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 >> 172.21.15.40:0/2919536756 conn(0x7f9a4c0d2da0 msgr2=0x7f9a4c0d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:38.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 shutdown_connections 2024-04-26T21:46:38.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:38.840+0000 7f9a547c1640 1 -- 172.21.15.40:0/2919536756 wait complete. 2024-04-26T21:46:38.851 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:46:38.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:46:38.851 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:46:38.851 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:46:38.852 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:38.852 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:46:38.852 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:46:38.852 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:46:38.852 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:38.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:38.852 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:46:38.867 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:46:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42049sec preferred_lft 42049sec 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:46:38.869 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:46:38.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:38.870 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:46:38.931 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:46:38.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:38.932 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:46:38.988 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:46:38.988 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:46:38.988 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:46:38.989 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:46:38.989 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:46:38.989 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:46:38.989 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:46:38.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:46:38.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:46:38.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:46:38.989 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:46:39.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:46:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:39.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:39.114 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:46:39.129 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:46:39.130 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:46:39.130 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:39.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:39.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:46:39.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:39.194 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:39.250 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:46:39.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:46:39.250 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:39.305 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:46:39.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:39.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:39.388 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:39.388 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:46:39.397 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:46:39.397 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:46:39.397 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:46:39.397 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:46:39.397 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:46:39.398 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:46:39.398 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:46:39.398 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:46:39.398 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:46:39.398 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:46:39.400 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:46:39.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:39.400 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:46:39.463 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:46:39.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:46:39.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:46:39.559 INFO:teuthology.orchestra.run.smithi040.stdout:client9448 2024-04-26T21:46:39.560 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:39.560 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:39.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:39.575 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-26T21:46:39.618 INFO:teuthology.orchestra.run:waiting for 20 2024-04-26T21:46:39.632 INFO:teuthology.orchestra.run.smithi040.stdout:.: 2024-04-26T21:46:39.632 INFO:teuthology.orchestra.run.smithi040.stdout:subdir 2024-04-26T21:46:39.632 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:39.632 INFO:teuthology.orchestra.run.smithi040.stdout:./subdir: 2024-04-26T21:46:39.633 INFO:teuthology.orchestra.run.smithi040.stdout:sixmegs 2024-04-26T21:46:39.633 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_openfiles.0' did not caused DAMAGED state 2024-04-26T21:46:39.633 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 100.00000000' 2024-04-26T21:46:39.633 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:39.633 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:39.688 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:39.688 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:39.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:46:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.883+0000 7f9df2e74640 1 Processor -- start 2024-04-26T21:46:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.883+0000 7f9df2e74640 1 -- start start 2024-04-26T21:46:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9df2e74640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9dec15f3a0 0x7f9dec0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9df2e74640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dec0d0e20 con 0x7f9dec0d2330 2024-04-26T21:46:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9df2e74640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dec0b1d70 con 0x7f9dec0d1770 2024-04-26T21:46:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9df2e74640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9dec0b2050 con 0x7f9dec15f3a0 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9df1411640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9dec0d2330 0x7f9dec0b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40370/0 (socket says 172.21.15.40:40370) 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9df1411640 1 -- 172.21.15.40:0/3623393971 learned_addr learned my addr 172.21.15.40:0/3623393971 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9df0c10640 1 --2- 172.21.15.40:0/3623393971 >> v2:172.21.15.184:3300/0 conn(0x7f9dec15f3a0 0x7f9dec0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 486875301 0 0) 0x7f9dec0d0e20 con 0x7f9dec0d2330 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ddc003520 con 0x7f9dec0d2330 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3063634754 0 0) 0x7f9ddc003520 con 0x7f9dec0d2330 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 >> v2:172.21.15.184:3300/0 conn(0x7f9dec15f3a0 msgr2=0x7f9dec0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 --2- 172.21.15.40:0/3623393971 >> v2:172.21.15.184:3300/0 conn(0x7f9dec15f3a0 0x7f9dec0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 >> v1:172.21.15.184:6789/0 conn(0x7f9dec0d1770 legacy=0x7f9dec0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0b2330 con 0x7f9dec0d2330 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9de4001a10 con 0x7f9dec0d2330 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 >> v1:172.21.15.40:6789/0 conn(0x7f9dec0d2330 legacy=0x7f9dec0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ddc0039b0 0x7f9ddc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc004800 0x7f9ddc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc0051c0 0x7f9ddc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0b2050 con 0x7f9ddc0051c0 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc00c060 con 0x7f9ddc004800 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.884+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9ddc00c260 con 0x7f9ddc0039b0 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.885+0000 7f9df1411640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc0051c0 0x7f9ddc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.885+0000 7f9df0c10640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ddc0039b0 0x7f9ddc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.885+0000 7f9debfff640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc004800 0x7f9ddc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.885+0000 7f9df1411640 1 -- 172.21.15.40:0/3623393971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ddc0039b0 msgr2=0x7f9ddc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.885+0000 7f9df1411640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ddc0039b0 0x7f9ddc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.885+0000 7f9df1411640 1 -- 172.21.15.40:0/3623393971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc004800 msgr2=0x7f9ddc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.885+0000 7f9df1411640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc004800 0x7f9ddc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.885+0000 7f9df1411640 1 -- 172.21.15.40:0/3623393971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc003520 con 0x7f9ddc0051c0 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df1411640 1 -- 172.21.15.40:0/3623393971 reap_dead start 2024-04-26T21:46:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df1411640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc0051c0 0x7f9ddc00b9c0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f9de400a8f0 tx=0x7f9ddc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e9b338f846aecd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de400a4c0 con 0x7f9ddc0051c0 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de402f070 con 0x7f9ddc0051c0 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9deb7fe640 1 -- 172.21.15.40:0/3623393971 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de402bcb0 con 0x7f9ddc0051c0 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df2e74640 1 -- 172.21.15.40:0/3623393971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc0051c0 msgr2=0x7f9ddc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df2e74640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc0051c0 0x7f9ddc00b9c0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f9de400a8f0 tx=0x7f9ddc005830 comp rx=0 tx=0).stop 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df2e74640 1 -- 172.21.15.40:0/3623393971 shutdown_connections 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df2e74640 1 --2- 172.21.15.40:0/3623393971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc0051c0 0x7f9ddc00b9c0 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df2e74640 1 -- 172.21.15.40:0/3623393971 >> 172.21.15.40:0/3623393971 conn(0x7f9dec0d4f90 msgr2=0x7f9dec0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df2e74640 1 -- 172.21.15.40:0/3623393971 shutdown_connections 2024-04-26T21:46:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.886+0000 7f9df2e74640 1 -- 172.21.15.40:0/3623393971 wait complete. 2024-04-26T21:46:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.888+0000 7f9df2e74640 1 Processor -- start 2024-04-26T21:46:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.888+0000 7f9df2e74640 1 -- start start 2024-04-26T21:46:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.888+0000 7f9df2e74640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc0051c0 0x7f9dec1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.888+0000 7f9df2e74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec0cc2d0 0x7f9dec14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.888+0000 7f9df2e74640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dec0cc750 0x7f9dec14f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9df2e74640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0d0870 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9df2e74640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0d06f0 con 0x7f9ddc0051c0 2024-04-26T21:46:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9df2e74640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9dec0d09f0 con 0x7f9dec0cc750 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec0cc2d0 0x7f9dec14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec0cc2d0 0x7f9dec14f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53904/0 (socket says 172.21.15.40:53904) 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 -- 172.21.15.40:0/1878134643 learned_addr learned my addr 172.21.15.40:0/1878134643 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9df0c10640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc0051c0 0x7f9dec1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9df1411640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dec0cc750 0x7f9dec14f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 -- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dec0cc750 msgr2=0x7f9dec14f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dec0cc750 0x7f9dec14f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 -- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc0051c0 msgr2=0x7f9dec1609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc0051c0 0x7f9dec1609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 -- 172.21.15.40:0/1878134643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec161070 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9df0c10640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc0051c0 0x7f9dec1609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9debfff640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec0cc2d0 0x7f9dec14f1c0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800f620 tx=0x7f9dd800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9de97fa640 1 -- 172.21.15.40:0/1878134643 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80052e0 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9de97fa640 1 -- 172.21.15.40:0/1878134643 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd8018070 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9de97fa640 1 -- 172.21.15.40:0/1878134643 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8003810 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec161350 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.889+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec161910 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.890+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dec0694f0 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.891+0000 7f9de97fa640 1 -- 172.21.15.40:0/1878134643 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dd80039b0 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.891+0000 7f9de97fa640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd006c920 0x7f9dd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.891+0000 7f9de97fa640 1 -- 172.21.15.40:0/1878134643 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f9dd808c6d0 con 0x7f9dec0cc2d0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.892+0000 7f9df0c10640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd006c920 0x7f9dd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.892+0000 7f9df0c10640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd006c920 0x7f9dd006ede0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009b40 tx=0x7f9de0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:39.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:39.895+0000 7f9de97fa640 1 -- 172.21.15.40:0/1878134643 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9dd8059cc0 con 0x7f9dec0cc2d0 2024-04-26T21:46:40.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.080+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dec162d10 con 0x7f9dec0cc2d0 2024-04-26T21:46:40.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.082+0000 7f9de97fa640 1 -- 172.21.15.40:0/1878134643 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+21710 (secure 0 0 0) 0x7f9dd80595c0 con 0x7f9dec0cc2d0 2024-04-26T21:46:40.083 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:40.083 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":131,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:46:25.765815+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6832/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6833/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6832/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6833/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6833/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6832/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6833/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6833/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6832/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3538213517":"2024-04-27T21:45:17.788819+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/3538213517":"2024-04-27T21:45:17.788819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd006c920 msgr2=0x7f9dd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd006c920 0x7f9dd006ede0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009b40 tx=0x7f9de0008040 comp rx=0 tx=0).stop 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec0cc2d0 msgr2=0x7f9dec14f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec0cc2d0 0x7f9dec14f1c0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800f620 tx=0x7f9dd800fae0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 shutdown_connections 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9dd006c920 0x7f9dd006ede0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dec0cc750 0x7f9dec14f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec0cc2d0 0x7f9dec14f1c0 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 --2- 172.21.15.40:0/1878134643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ddc0051c0 0x7f9dec1609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 >> 172.21.15.40:0/1878134643 conn(0x7f9dec0d4f90 msgr2=0x7f9dec0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 shutdown_connections 2024-04-26T21:46:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.084+0000 7f9df2e74640 1 -- 172.21.15.40:0/1878134643 wait complete. 2024-04-26T21:46:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:46:40.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:46:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.199+0000 7f51ed55f640 1 Processor -- start 2024-04-26T21:46:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.199+0000 7f51ed55f640 1 -- start start 2024-04-26T21:46:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51ed55f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80d1240 0x7f51e80d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51ed55f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e80d0690 con 0x7f51e80b4360 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51ed55f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e80d1b60 con 0x7f51e80b3610 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51ed55f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51e80cf210 con 0x7f51e80d1240 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51dffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51e80b4360 0x7f51e80cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40384/0 (socket says 172.21.15.40:40384) 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51dffff640 1 -- 172.21.15.40:0/1921897612 learned_addr learned my addr 172.21.15.40:0/1921897612 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e6ffd640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80d1240 0x7f51e80d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2646089007 0 0) 0x7f51e80d0690 con 0x7f51e80b4360 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51d4003520 con 0x7f51e80b4360 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 671787249 0 0) 0x7f51d4003520 con 0x7f51e80b4360 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80d1240 msgr2=0x7f51e80d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80d1240 0x7f51e80d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 >> v1:172.21.15.184:6789/0 conn(0x7f51e80b3610 legacy=0x7f51e80cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e80ce5e0 con 0x7f51e80b4360 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f51d0001670 con 0x7f51e80b4360 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 >> v1:172.21.15.40:6789/0 conn(0x7f51e80b4360 legacy=0x7f51e80cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.200+0000 7f51e67fc640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e67fc640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51d4004390 0x7f51d4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e67fc640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80d1240 0x7f51d40050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51e80cf210 con 0x7f51d40039e0 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51d400b9c0 con 0x7f51d4004390 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51d400bbc0 con 0x7f51e80d1240 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e6ffd640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51dffff640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51d4004390 0x7f51d4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e77fe640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80d1240 0x7f51d40050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e6ffd640 1 -- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80d1240 msgr2=0x7f51d40050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e6ffd640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80d1240 0x7f51d40050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e6ffd640 1 -- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51d4004390 msgr2=0x7f51d4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e6ffd640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51d4004390 0x7f51d4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.201+0000 7f51e6ffd640 1 -- 172.21.15.40:0/1921897612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d4003520 con 0x7f51d40039e0 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51e6ffd640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51d4003dc0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009ac0 tx=0x7f51d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce56098fabcf8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8036070 con 0x7f51d40039e0 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d8031400 con 0x7f51d40039e0 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51e67fc640 1 -- 172.21.15.40:0/1921897612 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8009690 con 0x7f51d40039e0 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51ed55f640 1 -- 172.21.15.40:0/1921897612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 msgr2=0x7f51d4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51ed55f640 1 --2- 172.21.15.40:0/1921897612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51d4003dc0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009ac0 tx=0x7f51d4005830 comp rx=0 tx=0).stop 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51e77fe640 1 -- 172.21.15.40:0/1921897612 reap_dead start 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51ed55f640 1 -- 172.21.15.40:0/1921897612 shutdown_connections 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51ed55f640 1 -- 172.21.15.40:0/1921897612 >> 172.21.15.40:0/1921897612 conn(0x7f51e80d2e60 msgr2=0x7f51e80d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51ed55f640 1 -- 172.21.15.40:0/1921897612 shutdown_connections 2024-04-26T21:46:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.202+0000 7f51ed55f640 1 -- 172.21.15.40:0/1921897612 wait complete. 2024-04-26T21:46:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51ed55f640 1 Processor -- start 2024-04-26T21:46:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51ed55f640 1 -- start start 2024-04-26T21:46:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51ed55f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51e8154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51ed55f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80c6960 0x7f51e80b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51ed55f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80c6de0 0x7f51e80b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51ed55f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51e80cf460 con 0x7f51d40039e0 2024-04-26T21:46:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51ed55f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51e8160ab0 con 0x7f51e80c6de0 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51ed55f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51e80c6610 con 0x7f51e80c6960 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51e8154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51e8154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53930/0 (socket says 172.21.15.40:53930) 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.204+0000 7f51e6ffd640 1 -- 172.21.15.40:0/2258911650 learned_addr learned my addr 172.21.15.40:0/2258911650 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51dffff640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80c6960 0x7f51e80b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51e77fe640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80c6de0 0x7f51e80b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51e6ffd640 1 -- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80c6960 msgr2=0x7f51e80b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51e6ffd640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80c6960 0x7f51e80b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51e6ffd640 1 -- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80c6de0 msgr2=0x7f51e80b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51e6ffd640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80c6de0 0x7f51e80b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51e6ffd640 1 -- 172.21.15.40:0/2258911650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e8154950 con 0x7f51d40039e0 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51dffff640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80c6960 0x7f51e80b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:40.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51e77fe640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80c6de0 0x7f51e80b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51e6ffd640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51e8154410 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009be0 tx=0x7f51d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51df7fe640 1 -- 172.21.15.40:0/2258911650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8040020 con 0x7f51d40039e0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51df7fe640 1 -- 172.21.15.40:0/2258911650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d8003c40 con 0x7f51d40039e0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e80c73b0 con 0x7f51d40039e0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51df7fe640 1 -- 172.21.15.40:0/2258911650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8036040 con 0x7f51d40039e0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.205+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e80c7ce0 con 0x7f51d40039e0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.206+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e80c78d0 con 0x7f51d40039e0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.207+0000 7f51df7fe640 1 -- 172.21.15.40:0/2258911650 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51d8043020 con 0x7f51d40039e0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.207+0000 7f51df7fe640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51c806c8f0 0x7f51c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.207+0000 7f51df7fe640 1 -- 172.21.15.40:0/2258911650 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f51d80ad190 con 0x7f51d40039e0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.207+0000 7f51dffff640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51c806c8f0 0x7f51c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.208+0000 7f51dffff640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51c806c8f0 0x7f51c806edb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f51e81f7d80 tx=0x7f51d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.211+0000 7f51df7fe640 1 -- 172.21.15.40:0/2258911650 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51e80c78d0 con 0x7f51d40039e0 2024-04-26T21:46:40.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.397+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51e80694f0 con 0x7f51d40039e0 2024-04-26T21:46:40.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.397+0000 7f51df7fe640 1 -- 172.21.15.40:0/2258911650 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) ==== 93+0+10 (secure 0 0 0) 0x7f51d804a020 con 0x7f51d40039e0 2024-04-26T21:46:40.399 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:46:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51c806c8f0 msgr2=0x7f51c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51c806c8f0 0x7f51c806edb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f51e81f7d80 tx=0x7f51d0009040 comp rx=0 tx=0).stop 2024-04-26T21:46:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 msgr2=0x7f51e8154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51e8154410 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009be0 tx=0x7f51d8004730 comp rx=0 tx=0).stop 2024-04-26T21:46:40.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 shutdown_connections 2024-04-26T21:46:40.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f51c806c8f0 0x7f51c806edb0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80c6de0 0x7f51e80b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80c6960 0x7f51e80b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 --2- 172.21.15.40:0/2258911650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51d40039e0 0x7f51e8154410 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 >> 172.21.15.40:0/2258911650 conn(0x7f51e80d2e60 msgr2=0x7f51e806e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 shutdown_connections 2024-04-26T21:46:40.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.399+0000 7f51ed55f640 1 -- 172.21.15.40:0/2258911650 wait complete. 2024-04-26T21:46:40.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:46:40.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.514+0000 7f4177578640 1 Processor -- start 2024-04-26T21:46:40.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.514+0000 7f4177578640 1 -- start start 2024-04-26T21:46:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4177578640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41700b0380 0x7f41700ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4177578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41700b10b0 0x7f417006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4177578640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41700cfd80 con 0x7f417006c780 2024-04-26T21:46:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4177578640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f41700af8e0 con 0x7f41700b0380 2024-04-26T21:46:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4177578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41700d0020 con 0x7f41700b10b0 2024-04-26T21:46:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4175314640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41700b0380 0x7f41700ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4175314640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41700b0380 0x7f41700ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53932/0 (socket says 172.21.15.40:53932) 2024-04-26T21:46:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4175314640 1 -- 172.21.15.40:0/1071055046 learned_addr learned my addr 172.21.15.40:0/1071055046 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4174b13640 1 --2- 172.21.15.40:0/1071055046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41700b10b0 0x7f417006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4175314640 1 -- 172.21.15.40:0/1071055046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41700b10b0 msgr2=0x7f417006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4175314640 1 --2- 172.21.15.40:0/1071055046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41700b10b0 0x7f417006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4175314640 1 -- 172.21.15.40:0/1071055046 >> v1:172.21.15.184:6789/0 conn(0x7f417006c780 legacy=0x7f4170072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.515+0000 7f4175314640 1 -- 172.21.15.40:0/1071055046 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41700d0300 con 0x7f41700b0380 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f4175314640 1 --2- 172.21.15.40:0/1071055046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f41700ccd50 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f4168009e50 tx=0x7f416802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dea26b6d7858f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f416ffff640 1 -- 172.21.15.40:0/1071055046 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168036070 con 0x7f41700b0380 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f416ffff640 1 -- 172.21.15.40:0/1071055046 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4168031600 con 0x7f41700b0380 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f416ffff640 1 -- 172.21.15.40:0/1071055046 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41680319d0 con 0x7f41700b0380 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f4177578640 1 -- 172.21.15.40:0/1071055046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 msgr2=0x7f41700ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f4177578640 1 --2- 172.21.15.40:0/1071055046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f41700ccd50 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f4168009e50 tx=0x7f416802a680 comp rx=0 tx=0).stop 2024-04-26T21:46:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f4177578640 1 -- 172.21.15.40:0/1071055046 shutdown_connections 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f4177578640 1 --2- 172.21.15.40:0/1071055046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41700b10b0 0x7f417006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f4177578640 1 --2- 172.21.15.40:0/1071055046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f41700ccd50 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.516+0000 7f4177578640 1 -- 172.21.15.40:0/1071055046 >> 172.21.15.40:0/1071055046 conn(0x7f41700d1cf0 msgr2=0x7f41700d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.517+0000 7f4177578640 1 -- 172.21.15.40:0/1071055046 shutdown_connections 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.517+0000 7f4177578640 1 -- 172.21.15.40:0/1071055046 wait complete. 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.518+0000 7f4177578640 1 Processor -- start 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.518+0000 7f4177578640 1 -- start start 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.518+0000 7f4177578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417006c780 0x7f4170150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.518+0000 7f4177578640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f417015e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4177578640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41700b10b0 0x7f417015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4177578640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41700b0760 con 0x7f41700b0380 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4177578640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41700d1980 con 0x7f41700b10b0 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4177578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41700d15c0 con 0x7f417006c780 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f417015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f417015e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53934/0 (socket says 172.21.15.40:53934) 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 -- 172.21.15.40:0/1310470738 learned_addr learned my addr 172.21.15.40:0/1310470738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4175314640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417006c780 0x7f4170150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4175b15640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41700b10b0 0x7f417015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 -- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417006c780 msgr2=0x7f4170150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417006c780 0x7f4170150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 -- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41700b10b0 msgr2=0x7f417015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41700b10b0 0x7f417015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 -- 172.21.15.40:0/1310470738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41701507d0 con 0x7f41700b0380 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4174b13640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f417015e850 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f416400d5d0 tx=0x7f416400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f416dffb640 1 -- 172.21.15.40:0/1310470738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640125f0 con 0x7f41700b0380 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f416dffb640 1 -- 172.21.15.40:0/1310470738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4164019070 con 0x7f41700b0380 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41701f4ff0 con 0x7f41700b0380 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.519+0000 7f416dffb640 1 -- 172.21.15.40:0/1310470738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640129f0 con 0x7f41700b0380 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.520+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41701f55b0 con 0x7f41700b0380 2024-04-26T21:46:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.520+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41700694f0 con 0x7f41700b0380 2024-04-26T21:46:40.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.521+0000 7f416dffb640 1 -- 172.21.15.40:0/1310470738 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4164012b90 con 0x7f41700b0380 2024-04-26T21:46:40.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.521+0000 7f416dffb640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f415c06c8f0 0x7f415c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.521+0000 7f416dffb640 1 -- 172.21.15.40:0/1310470738 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f416408cd40 con 0x7f41700b0380 2024-04-26T21:46:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.521+0000 7f4175314640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f415c06c8f0 0x7f415c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.522+0000 7f4175314640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f415c06c8f0 0x7f415c06edb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4168033000 tx=0x7f4168009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.525+0000 7f416dffb640 1 -- 172.21.15.40:0/1310470738 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41701507d0 con 0x7f41700b0380 2024-04-26T21:46:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.711+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41700b2470 con 0x7f41700b0380 2024-04-26T21:46:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.711+0000 7f416dffb640 1 -- 172.21.15.40:0/1310470738 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) ==== 104+0+11 (secure 0 0 0) 0x7f416405a300 con 0x7f41700b0380 2024-04-26T21:46:40.712 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:46:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f415c06c8f0 msgr2=0x7f415c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f415c06c8f0 0x7f415c06edb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4168033000 tx=0x7f4168009000 comp rx=0 tx=0).stop 2024-04-26T21:46:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 msgr2=0x7f417015e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f417015e850 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f416400d5d0 tx=0x7f416400da90 comp rx=0 tx=0).stop 2024-04-26T21:46:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 shutdown_connections 2024-04-26T21:46:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f415c06c8f0 0x7f415c06edb0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41700b10b0 0x7f417015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700b0380 0x7f417015e850 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 --2- 172.21.15.40:0/1310470738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417006c780 0x7f4170150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 >> 172.21.15.40:0/1310470738 conn(0x7f41700d1cf0 msgr2=0x7f417006f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 shutdown_connections 2024-04-26T21:46:40.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.714+0000 7f4177578640 1 -- 172.21.15.40:0/1310470738 wait complete. 2024-04-26T21:46:40.725 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:46:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.829+0000 7f68b0cbb640 1 Processor -- start 2024-04-26T21:46:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.829+0000 7f68b0cbb640 1 -- start start 2024-04-26T21:46:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68b0cbb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0b5f30 0x7f68ac0d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68b0cbb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f68ac0cff60 0x7f68ac0d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68b0cbb640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68ac0d5b90 con 0x7f68ac160750 2024-04-26T21:46:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68b0cbb640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f68ac160b30 con 0x7f68ac0cff60 2024-04-26T21:46:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68b0cbb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f68ac0d5e70 con 0x7f68ac0b5f30 2024-04-26T21:46:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0b5f30 0x7f68ac0d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68aad9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f68ac0cff60 0x7f68ac0d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0b5f30 0x7f68ac0d2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:46030/0 (socket says 172.21.15.40:46030) 2024-04-26T21:46:40.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68a3fff640 1 -- 172.21.15.40:0/2044096250 learned_addr learned my addr 172.21.15.40:0/2044096250 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:40.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68a3fff640 1 -- 172.21.15.40:0/2044096250 >> v2:172.21.15.184:3300/0 conn(0x7f68ac0cff60 msgr2=0x7f68ac0d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:40.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68a3fff640 1 --2- 172.21.15.40:0/2044096250 >> v2:172.21.15.184:3300/0 conn(0x7f68ac0cff60 0x7f68ac0d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68a3fff640 1 -- 172.21.15.40:0/2044096250 >> v1:172.21.15.184:6789/0 conn(0x7f68ac160750 legacy=0x7f68ac0d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.830+0000 7f68a3fff640 1 -- 172.21.15.40:0/2044096250 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac0d3930 con 0x7f68ac0b5f30 2024-04-26T21:46:40.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.831+0000 7f68a3fff640 1 --2- 172.21.15.40:0/2044096250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0b5f30 0x7f68ac0d2150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6894009870 tx=0x7f689402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcea7442954736b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.832+0000 7f68a9d9b640 1 -- 172.21.15.40:0/2044096250 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894037070 con 0x7f68ac0b5f30 2024-04-26T21:46:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.832+0000 7f68a9d9b640 1 -- 172.21.15.40:0/2044096250 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6894032440 con 0x7f68ac0b5f30 2024-04-26T21:46:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.832+0000 7f68a9d9b640 1 -- 172.21.15.40:0/2044096250 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894032810 con 0x7f68ac0b5f30 2024-04-26T21:46:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.833+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2044096250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0b5f30 msgr2=0x7f68ac0d2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.833+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2044096250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0b5f30 0x7f68ac0d2150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6894009870 tx=0x7f689402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.833+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2044096250 shutdown_connections 2024-04-26T21:46:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.833+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2044096250 >> v2:172.21.15.184:3300/0 conn(0x7f68ac0cff60 0x7f68ac0d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.833+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2044096250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0b5f30 0x7f68ac0d2150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.833+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2044096250 >> 172.21.15.40:0/2044096250 conn(0x7f68ac0d5070 msgr2=0x7f68ac0d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.833+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2044096250 shutdown_connections 2024-04-26T21:46:40.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.833+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2044096250 wait complete. 2024-04-26T21:46:40.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.834+0000 7f68b0cbb640 1 Processor -- start 2024-04-26T21:46:40.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.834+0000 7f68b0cbb640 1 -- start start 2024-04-26T21:46:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68b0cbb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68ac0b5f30 0x7f68ac151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68b0cbb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0cff60 0x7f68ac0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68b0cbb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac160750 0x7f68ac14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68b0cbb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac0d3ef0 con 0x7f68ac160750 2024-04-26T21:46:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68b0cbb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac0d3620 con 0x7f68ac0b5f30 2024-04-26T21:46:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68b0cbb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f68ac160ef0 con 0x7f68ac0cff60 2024-04-26T21:46:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68aad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac160750 0x7f68ac14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68aad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac160750 0x7f68ac14fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53942/0 (socket says 172.21.15.40:53942) 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68aad9d640 1 -- 172.21.15.40:0/2673250695 learned_addr learned my addr 172.21.15.40:0/2673250695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68a3fff640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0cff60 0x7f68ac0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.835+0000 7f68aa59c640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68ac0b5f30 0x7f68ac151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68aad9d640 1 -- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0cff60 msgr2=0x7f68ac0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68aad9d640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0cff60 0x7f68ac0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68aad9d640 1 -- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68ac0b5f30 msgr2=0x7f68ac151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68aad9d640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68ac0b5f30 0x7f68ac151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68aad9d640 1 -- 172.21.15.40:0/2673250695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac0cd6d0 con 0x7f68ac160750 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68aad9d640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac160750 0x7f68ac14fef0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f68a400efc0 tx=0x7f68a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68a2ffd640 1 -- 172.21.15.40:0/2673250695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4018020 con 0x7f68ac160750 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68a2ffd640 1 -- 172.21.15.40:0/2673250695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a4014480 con 0x7f68ac160750 2024-04-26T21:46:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ac0cd9b0 con 0x7f68ac160750 2024-04-26T21:46:40.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68a2ffd640 1 -- 172.21.15.40:0/2673250695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4014870 con 0x7f68ac160750 2024-04-26T21:46:40.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.836+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ac0cdf70 con 0x7f68ac160750 2024-04-26T21:46:40.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.837+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ac0694f0 con 0x7f68ac160750 2024-04-26T21:46:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.838+0000 7f68a2ffd640 1 -- 172.21.15.40:0/2673250695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68a4005d10 con 0x7f68ac160750 2024-04-26T21:46:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.838+0000 7f68a2ffd640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f688c06c8f0 0x7f688c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.838+0000 7f68a2ffd640 1 -- 172.21.15.40:0/2673250695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f68a408d400 con 0x7f68ac160750 2024-04-26T21:46:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.838+0000 7f68aa59c640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f688c06c8f0 0x7f688c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.839+0000 7f68aa59c640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f688c06c8f0 0x7f688c06edb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f689c0016e0 tx=0x7f689c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:40.842+0000 7f68a2ffd640 1 -- 172.21.15.40:0/2673250695 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68a4022070 con 0x7f68ac160750 2024-04-26T21:46:41.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.027+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f68ac0b6900 con 0x7f68ac160750 2024-04-26T21:46:41.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.027+0000 7f68a2ffd640 1 -- 172.21.15.40:0/2673250695 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) ==== 100+0+11 (secure 0 0 0) 0x7f68a4010040 con 0x7f68ac160750 2024-04-26T21:46:41.028 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f688c06c8f0 msgr2=0x7f688c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f688c06c8f0 0x7f688c06edb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f689c0016e0 tx=0x7f689c009040 comp rx=0 tx=0).stop 2024-04-26T21:46:41.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac160750 msgr2=0x7f68ac14fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac160750 0x7f68ac14fef0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f68a400efc0 tx=0x7f68a400c490 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 shutdown_connections 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f688c06c8f0 0x7f688c06edb0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac160750 0x7f68ac14fef0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68ac0cff60 0x7f68ac0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 --2- 172.21.15.40:0/2673250695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68ac0b5f30 0x7f68ac151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 >> 172.21.15.40:0/2673250695 conn(0x7f68ac0d5070 msgr2=0x7f68ac15c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 shutdown_connections 2024-04-26T21:46:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.029+0000 7f68b0cbb640 1 -- 172.21.15.40:0/2673250695 wait complete. 2024-04-26T21:46:41.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:46:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.141+0000 7f72ad7b6640 1 Processor -- start 2024-04-26T21:46:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.141+0000 7f72ad7b6640 1 -- start start 2024-04-26T21:46:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72ad7b6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72a8156460 0x7f72a80cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72ad7b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a806fea0 0x7f72a8072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72ad7b6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a8156840 con 0x7f72a80cba10 2024-04-26T21:46:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72ad7b6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f72a80cbf20 con 0x7f72a8156460 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72ad7b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72a80cc200 con 0x7f72a806fea0 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a806fea0 0x7f72a8072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a6ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72a8156460 0x7f72a80cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a6ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72a8156460 0x7f72a80cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:43394/0 (socket says 172.21.15.40:43394) 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a806fea0 0x7f72a8072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:46050/0 (socket says 172.21.15.40:46050) 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a67fc640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f72a80cba10 0x7f72a806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:60920/0 (socket says 172.21.15.40:60920) 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a77fe640 1 -- 172.21.15.40:0/365116141 learned_addr learned my addr 172.21.15.40:0/365116141 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a77fe640 1 -- 172.21.15.40:0/365116141 >> v2:172.21.15.184:3300/0 conn(0x7f72a8156460 msgr2=0x7f72a80cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a77fe640 1 --2- 172.21.15.40:0/365116141 >> v2:172.21.15.184:3300/0 conn(0x7f72a8156460 0x7f72a80cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.142+0000 7f72a77fe640 1 -- 172.21.15.40:0/365116141 >> v1:172.21.15.184:6789/0 conn(0x7f72a80cba10 legacy=0x7f72a806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.143+0000 7f72a77fe640 1 -- 172.21.15.40:0/365116141 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a80cc4e0 con 0x7f72a806fea0 2024-04-26T21:46:41.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.143+0000 7f72a77fe640 1 --2- 172.21.15.40:0/365116141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a806fea0 0x7f72a8072c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009e50 tx=0x7f72a002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b08722cdade1fa2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.143+0000 7f72a5ffb640 1 -- 172.21.15.40:0/365116141 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0036070 con 0x7f72a806fea0 2024-04-26T21:46:41.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72a5ffb640 1 -- 172.21.15.40:0/365116141 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a0031600 con 0x7f72a806fea0 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72a5ffb640 1 -- 172.21.15.40:0/365116141 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a00319d0 con 0x7f72a806fea0 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72ad7b6640 1 -- 172.21.15.40:0/365116141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a806fea0 msgr2=0x7f72a8072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/365116141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a806fea0 0x7f72a8072c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009e50 tx=0x7f72a002a710 comp rx=0 tx=0).stop 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72ad7b6640 1 -- 172.21.15.40:0/365116141 shutdown_connections 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/365116141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a806fea0 0x7f72a8072c50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/365116141 >> v2:172.21.15.184:3300/0 conn(0x7f72a8156460 0x7f72a80cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72ad7b6640 1 -- 172.21.15.40:0/365116141 >> 172.21.15.40:0/365116141 conn(0x7f72a80d0be0 msgr2=0x7f72a80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72ad7b6640 1 -- 172.21.15.40:0/365116141 shutdown_connections 2024-04-26T21:46:41.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.144+0000 7f72ad7b6640 1 -- 172.21.15.40:0/365116141 wait complete. 2024-04-26T21:46:41.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.145+0000 7f72ad7b6640 1 Processor -- start 2024-04-26T21:46:41.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.145+0000 7f72ad7b6640 1 -- start start 2024-04-26T21:46:41.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72ad7b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72a806fea0 0x7f72a814f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72ad7b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a80cba10 0x7f72a80cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72ad7b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72a8156460 0x7f72a80cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72ad7b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f72a806e260 con 0x7f72a8156460 2024-04-26T21:46:41.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72ad7b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8072700 con 0x7f72a806fea0 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72ad7b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72a806e3e0 con 0x7f72a80cba10 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72a8156460 0x7f72a80cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72a8156460 0x7f72a80cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53950/0 (socket says 172.21.15.40:53950) 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72a77fe640 1 -- 172.21.15.40:0/3823460793 learned_addr learned my addr 172.21.15.40:0/3823460793 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72a6ffd640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72a806fea0 0x7f72a814f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.146+0000 7f72a67fc640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a80cba10 0x7f72a80cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f72a77fe640 1 -- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a80cba10 msgr2=0x7f72a80cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f72a77fe640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a80cba10 0x7f72a80cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f72a77fe640 1 -- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72a806fea0 msgr2=0x7f72a814f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f72a77fe640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72a806fea0 0x7f72a814f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f72a77fe640 1 -- 172.21.15.40:0/3823460793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a814fcc0 con 0x7f72a8156460 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f72a77fe640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72a8156460 0x7f72a80cfc70 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f72a0034040 tx=0x7f72a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f728f7fe640 1 -- 172.21.15.40:0/3823460793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0036050 con 0x7f72a8156460 2024-04-26T21:46:41.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f728f7fe640 1 -- 172.21.15.40:0/3823460793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a0042070 con 0x7f72a8156460 2024-04-26T21:46:41.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a81f6a00 con 0x7f72a8156460 2024-04-26T21:46:41.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f728f7fe640 1 -- 172.21.15.40:0/3823460793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0009510 con 0x7f72a8156460 2024-04-26T21:46:41.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.147+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a81f6fc0 con 0x7f72a8156460 2024-04-26T21:46:41.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.148+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a81f7680 con 0x7f72a8156460 2024-04-26T21:46:41.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.149+0000 7f728f7fe640 1 -- 172.21.15.40:0/3823460793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72a0007bf0 con 0x7f72a8156460 2024-04-26T21:46:41.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.149+0000 7f728f7fe640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f729406c8f0 0x7f729406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.149+0000 7f728f7fe640 1 -- 172.21.15.40:0/3823460793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f72a00ad610 con 0x7f72a8156460 2024-04-26T21:46:41.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.149+0000 7f72a6ffd640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f729406c8f0 0x7f729406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.150+0000 7f72a6ffd640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f729406c8f0 0x7f729406edb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7298009aa0 tx=0x7f7298009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.153+0000 7f728f7fe640 1 -- 172.21.15.40:0/3823460793 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72a81f7680 con 0x7f72a8156460 2024-04-26T21:46:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.354+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f72a81513d0 con 0x7f72a8156460 2024-04-26T21:46:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.354+0000 7f728f7fe640 1 -- 172.21.15.40:0/3823460793 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 72 entries v131) ==== 81+0+4380 (secure 0 0 0) 0x7f72a81513d0 con 0x7f72a8156460 2024-04-26T21:46:41.355 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/771026484 2024-04-27T21:46:20.722276+0000 2024-04-26T21:46:41.355 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/771026484 2024-04-27T21:46:20.722276+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1753847972 2024-04-27T21:46:08.370602+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1753847972 2024-04-27T21:46:08.370602+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1660522931 2024-04-27T21:45:51.733253+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4214786907 2024-04-27T21:45:46.312381+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2896027915 2024-04-27T21:45:31.907924+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2896027915 2024-04-27T21:45:31.907924+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4173136579 2024-04-27T21:45:27.871638+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6837/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1992174713 2024-04-27T21:43:31.927565+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:46:41.356 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3791539151 2024-04-27T21:45:23.843779+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4192917874 2024-04-27T21:45:19.814510+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/1482743077 2024-04-27T21:42:45.590944+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4214786907 2024-04-27T21:45:46.312381+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:46:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3791539151 2024-04-27T21:45:23.843779+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1812066725 2024-04-27T21:46:25.765788+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1660522931 2024-04-27T21:45:51.733253+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3311550037 2024-04-27T21:43:13.833433+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2899400637 2024-04-27T21:43:45.018129+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3538213517 2024-04-27T21:45:17.788819+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2103981319 2024-04-27T21:42:40.495461+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6835/1182266137 2024-04-27T21:42:55.651896+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/68430788 2024-04-27T21:44:01.214317+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2962699830 2024-04-27T21:43:38.976843+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4016455310 2024-04-27T21:44:55.673612+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:46:41.358 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4192917874 2024-04-27T21:45:19.814510+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6836/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/747650264 2024-04-27T21:42:51.669693+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/3588039422 2024-04-27T21:44:08.264618+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2324764539 2024-04-27T21:44:14.314860+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2224283166 2024-04-27T21:44:51.638256+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/45688484 2024-04-27T21:44:18.346545+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1812066725 2024-04-27T21:46:25.765788+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/4173136579 2024-04-27T21:45:27.871638+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/343744670 2024-04-27T21:43:27.897726+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/2569387163 2024-04-27T21:44:22.393324+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1803384446 2024-04-27T21:43:59.077926+0000 2024-04-26T21:46:41.359 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1239220748 2024-04-27T21:44:37.530725+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3114124794 2024-04-27T21:43:25.679833+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/4237566530 2024-04-27T21:44:47.608188+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1051970054 2024-04-27T21:44:39.547503+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/2715490806 2024-04-27T21:42:23.477662+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/1149681708 2024-04-27T21:44:43.574926+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/752925509 2024-04-27T21:44:59.701505+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6833/1838768570 2024-04-27T21:45:03.736947+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.184:6834/1327671065 2024-04-27T21:42:59.734236+0000 2024-04-26T21:46:41.360 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6832/3538213517 2024-04-27T21:45:17.788819+0000 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:listed 72 entries 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f729406c8f0 msgr2=0x7f729406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f729406c8f0 0x7f729406edb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7298009aa0 tx=0x7f7298009190 comp rx=0 tx=0).stop 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72a8156460 msgr2=0x7f72a80cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72a8156460 0x7f72a80cfc70 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f72a0034040 tx=0x7f72a0004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 shutdown_connections 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f729406c8f0 0x7f729406edb0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72a8156460 0x7f72a80cfc70 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72a80cba10 0x7f72a80cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 --2- 172.21.15.40:0/3823460793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72a806fea0 0x7f72a814f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 >> 172.21.15.40:0/3823460793 conn(0x7f72a80d0be0 msgr2=0x7f72a8150cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 shutdown_connections 2024-04-26T21:46:41.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.357+0000 7f72ad7b6640 1 -- 172.21.15.40:0/3823460793 wait complete. 2024-04-26T21:46:41.368 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:46:41.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:41.368 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:46:41.394 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:46:41.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:41.394 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:46:41.450 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:46:41.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.591+0000 7f854412c640 1 Processor -- start 2024-04-26T21:46:41.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.591+0000 7f854412c640 1 -- start start 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.592+0000 7f854412c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f853c15f3a0 0x7f853c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.592+0000 7f854412c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f853c0d2330 0x7f853c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.592+0000 7f854412c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f853c0d0e20 con 0x7f853c0d1770 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.592+0000 7f854412c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f853c0b1d70 con 0x7f853c0d2330 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.592+0000 7f854412c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f853c0b2050 con 0x7f853c15f3a0 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.592+0000 7f85426c9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f853c0d2330 0x7f853c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.592+0000 7f85416c7640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f853c0d1770 0x7f853c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40398/0 (socket says 172.21.15.40:40398) 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.592+0000 7f85416c7640 1 -- 172.21.15.40:0/4104550887 learned_addr learned my addr 172.21.15.40:0/4104550887 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.593+0000 7f8541ec8640 1 --2- 172.21.15.40:0/4104550887 >> v2:172.21.15.184:3300/0 conn(0x7f853c15f3a0 0x7f853c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.593+0000 7f85426c9640 1 -- 172.21.15.40:0/4104550887 >> v2:172.21.15.184:3300/0 conn(0x7f853c15f3a0 msgr2=0x7f853c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.593+0000 7f8540ec6640 1 -- 172.21.15.40:0/4104550887 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1249264031 0 0) 0x7f853c0d0e20 con 0x7f853c0d1770 2024-04-26T21:46:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.593+0000 7f85426c9640 1 --2- 172.21.15.40:0/4104550887 >> v2:172.21.15.184:3300/0 conn(0x7f853c15f3a0 0x7f853c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.593+0000 7f85426c9640 1 -- 172.21.15.40:0/4104550887 >> v1:172.21.15.40:6789/0 conn(0x7f853c0d1770 legacy=0x7f853c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.593+0000 7f85426c9640 1 -- 172.21.15.40:0/4104550887 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c0b2330 con 0x7f853c0d2330 2024-04-26T21:46:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.593+0000 7f85426c9640 1 --2- 172.21.15.40:0/4104550887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b2850 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f853c073b50 tx=0x7f852c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d0ccdf9f3615e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f8540ec6640 1 -- 172.21.15.40:0/4104550887 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c007740 con 0x7f853c0d2330 2024-04-26T21:46:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f8540ec6640 1 -- 172.21.15.40:0/4104550887 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c0078e0 con 0x7f853c0d2330 2024-04-26T21:46:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f8540ec6640 1 -- 172.21.15.40:0/4104550887 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c007cb0 con 0x7f853c0d2330 2024-04-26T21:46:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f854412c640 1 -- 172.21.15.40:0/4104550887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 msgr2=0x7f853c0b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f854412c640 1 --2- 172.21.15.40:0/4104550887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b2850 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f853c073b50 tx=0x7f852c02a680 comp rx=0 tx=0).stop 2024-04-26T21:46:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f854412c640 1 -- 172.21.15.40:0/4104550887 shutdown_connections 2024-04-26T21:46:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f854412c640 1 --2- 172.21.15.40:0/4104550887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b2850 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f854412c640 1 --2- 172.21.15.40:0/4104550887 >> v2:172.21.15.184:3300/0 conn(0x7f853c15f3a0 0x7f853c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f854412c640 1 -- 172.21.15.40:0/4104550887 >> 172.21.15.40:0/4104550887 conn(0x7f853c0d4f90 msgr2=0x7f853c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f854412c640 1 -- 172.21.15.40:0/4104550887 shutdown_connections 2024-04-26T21:46:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.594+0000 7f854412c640 1 -- 172.21.15.40:0/4104550887 wait complete. 2024-04-26T21:46:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.595+0000 7f854412c640 1 Processor -- start 2024-04-26T21:46:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.595+0000 7f854412c640 1 -- start start 2024-04-26T21:46:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f854412c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c0d1770 0x7f853c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f854412c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f854412c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c15f3a0 0x7f853c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f854412c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f853c0b2f10 con 0x7f853c0d2330 2024-04-26T21:46:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f854412c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f853c0b2d90 con 0x7f853c15f3a0 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f854412c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f853c0b3090 con 0x7f853c0d1770 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f85416c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f85416c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53970/0 (socket says 172.21.15.40:53970) 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.596+0000 7f85416c7640 1 -- 172.21.15.40:0/366636566 learned_addr learned my addr 172.21.15.40:0/366636566 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f85426c9640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c15f3a0 0x7f853c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f8541ec8640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c0d1770 0x7f853c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f85416c7640 1 -- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c0d1770 msgr2=0x7f853c15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f85416c7640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c0d1770 0x7f853c15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f85416c7640 1 -- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c15f3a0 msgr2=0x7f853c0b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f85416c7640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c15f3a0 0x7f853c0b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f85416c7640 1 -- 172.21.15.40:0/366636566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c160570 con 0x7f853c0d2330 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f8541ec8640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c0d1770 0x7f853c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f85426c9640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c15f3a0 0x7f853c0b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f85416c7640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b85a0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f853000c4a0 tx=0x7f853000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f853a7fc640 1 -- 172.21.15.40:0/366636566 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530017070 con 0x7f853c0d2330 2024-04-26T21:46:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f853a7fc640 1 -- 172.21.15.40:0/366636566 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8530005ce0 con 0x7f853c0d2330 2024-04-26T21:46:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f853a7fc640 1 -- 172.21.15.40:0/366636566 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85300072b0 con 0x7f853c0d2330 2024-04-26T21:46:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c160850 con 0x7f853c0d2330 2024-04-26T21:46:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.597+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c160e10 con 0x7f853c0d2330 2024-04-26T21:46:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.598+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853c0694f0 con 0x7f853c0d2330 2024-04-26T21:46:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.599+0000 7f853a7fc640 1 -- 172.21.15.40:0/366636566 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f853001a070 con 0x7f853c0d2330 2024-04-26T21:46:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.599+0000 7f853a7fc640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f852806c8f0 0x7f852806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.599+0000 7f853a7fc640 1 -- 172.21.15.40:0/366636566 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f853008bd80 con 0x7f853c0d2330 2024-04-26T21:46:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.599+0000 7f8541ec8640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f852806c8f0 0x7f852806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.600+0000 7f8541ec8640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f852806c8f0 0x7f852806edb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f853c06de10 tx=0x7f8534008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.603+0000 7f853a7fc640 1 -- 172.21.15.40:0/366636566 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85300593d0 con 0x7f853c0d2330 2024-04-26T21:46:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.827+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f853c0b6860 con 0x7f853c0d2330 2024-04-26T21:46:41.828 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:41.827+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:46:41.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.833+0000 7f853a7fc640 1 -- 172.21.15.40:0/366636566 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v327) ==== 161+0+0 (secure 0 0 0) 0x7f8530058cd0 con 0x7f853c0d2330 2024-04-26T21:46:41.834 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:46:41.836 INFO:tasks.ceph.mds.c.smithi040.stderr: -1859> 2024-04-26T21:46:31.802+0000 7f482ecea640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-04-26T21:46:41.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.836+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f852806c8f0 msgr2=0x7f852806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.836+0000 7f854412c640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f852806c8f0 0x7f852806edb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f853c06de10 tx=0x7f8534008040 comp rx=0 tx=0).stop 2024-04-26T21:46:41.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.836+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 msgr2=0x7f853c0b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.836+0000 7f854412c640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b85a0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f853000c4a0 tx=0x7f853000c960 comp rx=0 tx=0).stop 2024-04-26T21:46:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.837+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 shutdown_connections 2024-04-26T21:46:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.837+0000 7f854412c640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f852806c8f0 0x7f852806edb0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.837+0000 7f854412c640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c15f3a0 0x7f853c0b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.837+0000 7f854412c640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853c0d2330 0x7f853c0b85a0 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.837+0000 7f854412c640 1 --2- 172.21.15.40:0/366636566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c0d1770 0x7f853c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.837+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 >> 172.21.15.40:0/366636566 conn(0x7f853c0d4f90 msgr2=0x7f853c0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.837+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 shutdown_connections 2024-04-26T21:46:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.837+0000 7f854412c640 1 -- 172.21.15.40:0/366636566 wait complete. 2024-04-26T21:46:41.839 INFO:tasks.ceph.mds.c.smithi040.stderr: -710> 2024-04-26T21:46:36.777+0000 7f4832cf2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:46:41.850 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:46:41.865 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:46:41.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.952+0000 7fc577c47640 1 Processor -- start 2024-04-26T21:46:41.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.953+0000 7fc577c47640 1 -- start start 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.953+0000 7fc577c47640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc570163610 0x7fc5700d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.953+0000 7fc577c47640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5700d1a70 con 0x7fc5700d2830 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.953+0000 7fc577c47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5700d1d50 con 0x7fc57015f5a0 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.953+0000 7fc577c47640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc570163d30 con 0x7fc570163610 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.953+0000 7fc5761e4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc5700d2830 0x7fc5700cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40416/0 (socket says 172.21.15.40:40416) 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.954+0000 7fc5761e4640 1 -- 172.21.15.40:0/3596576044 learned_addr learned my addr 172.21.15.40:0/3596576044 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.954+0000 7fc5751e2640 1 --2- 172.21.15.40:0/3596576044 >> v2:172.21.15.184:3300/0 conn(0x7fc570163610 0x7fc5700d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.954+0000 7fc5749e1640 1 -- 172.21.15.40:0/3596576044 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1905806151 0 0) 0x7fc5700d1a70 con 0x7fc5700d2830 2024-04-26T21:46:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.954+0000 7fc5749e1640 1 -- 172.21.15.40:0/3596576044 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc55c003520 con 0x7fc5700d2830 2024-04-26T21:46:41.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.954+0000 7fc5751e2640 1 -- 172.21.15.40:0/3596576044 >> v1:172.21.15.184:6789/0 conn(0x7fc57015f5a0 legacy=0x7fc570162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.954+0000 7fc5749e1640 1 -- 172.21.15.40:0/3596576044 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935835916 0 0) 0x7fc5700d1d50 con 0x7fc57015f5a0 2024-04-26T21:46:41.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.954+0000 7fc5751e2640 1 -- 172.21.15.40:0/3596576044 >> v1:172.21.15.40:6789/0 conn(0x7fc5700d2830 legacy=0x7fc5700cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.954+0000 7fc5751e2640 1 -- 172.21.15.40:0/3596576044 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5700d2030 con 0x7fc570163610 2024-04-26T21:46:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.955+0000 7fc5751e2640 1 --2- 172.21.15.40:0/3596576044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570163610 0x7fc5700d5cb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc5600097a0 tx=0x7fc56002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5b62d04ffb5fe26 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.955+0000 7fc5749e1640 1 -- 172.21.15.40:0/3596576044 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560037070 con 0x7fc570163610 2024-04-26T21:46:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.955+0000 7fc5749e1640 1 -- 172.21.15.40:0/3596576044 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc560032440 con 0x7fc570163610 2024-04-26T21:46:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.955+0000 7fc5749e1640 1 -- 172.21.15.40:0/3596576044 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5600327f0 con 0x7fc570163610 2024-04-26T21:46:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.956+0000 7fc577c47640 1 -- 172.21.15.40:0/3596576044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570163610 msgr2=0x7fc5700d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.956+0000 7fc577c47640 1 --2- 172.21.15.40:0/3596576044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570163610 0x7fc5700d5cb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc5600097a0 tx=0x7fc56002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.956+0000 7fc577c47640 1 -- 172.21.15.40:0/3596576044 shutdown_connections 2024-04-26T21:46:41.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.956+0000 7fc577c47640 1 --2- 172.21.15.40:0/3596576044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570163610 0x7fc5700d5cb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.956+0000 7fc577c47640 1 -- 172.21.15.40:0/3596576044 >> 172.21.15.40:0/3596576044 conn(0x7fc5700d5190 msgr2=0x7fc5700d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:41.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.956+0000 7fc577c47640 1 -- 172.21.15.40:0/3596576044 shutdown_connections 2024-04-26T21:46:41.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.956+0000 7fc577c47640 1 -- 172.21.15.40:0/3596576044 wait complete. 2024-04-26T21:46:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.957+0000 7fc577c47640 1 Processor -- start 2024-04-26T21:46:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.957+0000 7fc577c47640 1 -- start start 2024-04-26T21:46:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc577c47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5700d2830 0x7fc5700b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc577c47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc570163610 0x7fc57006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc577c47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc57006e310 0x7fc5700cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc577c47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc5700d3c20 con 0x7fc5700d2830 2024-04-26T21:46:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc577c47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5700d4190 con 0x7fc57006e310 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc577c47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc5700d3da0 con 0x7fc570163610 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5700d2830 0x7fc5700b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5700d2830 0x7fc5700b8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54010/0 (socket says 172.21.15.40:54010) 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 -- 172.21.15.40:0/984179165 learned_addr learned my addr 172.21.15.40:0/984179165 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5751e2640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc570163610 0x7fc57006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5761e4640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc57006e310 0x7fc5700cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 -- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc570163610 msgr2=0x7fc57006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc570163610 0x7fc57006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 -- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc57006e310 msgr2=0x7fc5700cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc57006e310 0x7fc5700cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 -- 172.21.15.40:0/984179165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5701614e0 con 0x7fc5700d2830 2024-04-26T21:46:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5751e2640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc570163610 0x7fc57006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5759e3640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5700d2830 0x7fc5700b8ae0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00c4e0 tx=0x7fc56c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5667fc640 1 -- 172.21.15.40:0/984179165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c017070 con 0x7fc5700d2830 2024-04-26T21:46:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5667fc640 1 -- 172.21.15.40:0/984179165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c0035f0 con 0x7fc5700d2830 2024-04-26T21:46:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5700c91f0 con 0x7fc5700d2830 2024-04-26T21:46:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc5667fc640 1 -- 172.21.15.40:0/984179165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0039e0 con 0x7fc5700d2830 2024-04-26T21:46:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.958+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc570161a60 con 0x7fc5700d2830 2024-04-26T21:46:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.959+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5700694f0 con 0x7fc5700d2830 2024-04-26T21:46:41.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.960+0000 7fc5667fc640 1 -- 172.21.15.40:0/984179165 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc56c01a070 con 0x7fc5700d2830 2024-04-26T21:46:41.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.960+0000 7fc5667fc640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc55406c8f0 0x7fc55406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:41.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.960+0000 7fc5667fc640 1 -- 172.21.15.40:0/984179165 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9399+0+0 (secure 0 0 0) 0x7fc56c08bca0 con 0x7fc5700d2830 2024-04-26T21:46:41.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.960+0000 7fc5751e2640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc55406c8f0 0x7fc55406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:41.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.961+0000 7fc5751e2640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc55406c8f0 0x7fc55406edb0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc560034000 tx=0x7fc56002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:41.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:41.965+0000 7fc5667fc640 1 -- 172.21.15.40:0/984179165 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc56c058bc0 con 0x7fc5700d2830 2024-04-26T21:46:42.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.223+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc5700bb110 con 0x7fc5700d2830 2024-04-26T21:46:42.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:46:42.820 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:46:42.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.843+0000 7fc5667fc640 1 -- 172.21.15.40:0/984179165 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v328) ==== 107+0+0 (secure 0 0 0) 0x7fc56c05cec0 con 0x7fc5700d2830 2024-04-26T21:46:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:nothing to do: rank is not damaged 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc55406c8f0 msgr2=0x7fc55406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc55406c8f0 0x7fc55406edb0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc560034000 tx=0x7fc56002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5700d2830 msgr2=0x7fc5700b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5700d2830 0x7fc5700b8ae0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00c4e0 tx=0x7fc56c00c9a0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 shutdown_connections 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc55406c8f0 0x7fc55406edb0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc57006e310 0x7fc5700cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc570163610 0x7fc57006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 --2- 172.21.15.40:0/984179165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5700d2830 0x7fc5700b8ae0 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 >> 172.21.15.40:0/984179165 conn(0x7fc5700d5190 msgr2=0x7fc5700b7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:42.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 shutdown_connections 2024-04-26T21:46:42.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.846+0000 7fc577c47640 1 -- 172.21.15.40:0/984179165 wait complete. 2024-04-26T21:46:42.860 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:46:42.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.958+0000 7f0d86b1bf40 1 Processor -- start 2024-04-26T21:46:42.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.958+0000 7f0d86b1bf40 1 -- start start 2024-04-26T21:46:42.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d86b1bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d67700 0x560ec1dec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d86b1bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d68940 0x560ec1d4f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d86b1bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 0x560ec1d1cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d86b1bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560ec1cc6e40 con 0x560ec1d68940 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d86b1bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560ec1ccae80 con 0x560ec1df46b0 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d86b1bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560ec1df5aa0 con 0x560ec1d67700 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d83e14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d68940 0x560ec1d4f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d83e14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d68940 0x560ec1d4f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54026/0 (socket says 172.21.15.40:54026) 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d83e14640 1 -- 172.21.15.40:0/879596290 learned_addr learned my addr 172.21.15.40:0/879596290 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d84615640 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d67700 0x560ec1dec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.959+0000 7f0d84e16640 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 0x560ec1d1cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d83e14640 1 -- 172.21.15.40:0/879596290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d67700 msgr2=0x560ec1dec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:42.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d83e14640 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d67700 0x560ec1dec020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d83e14640 1 -- 172.21.15.40:0/879596290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 msgr2=0x560ec1d1cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d83e14640 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 0x560ec1d1cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d83e14640 1 -- 172.21.15.40:0/879596290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ec1df6820 con 0x560ec1d68940 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d83e14640 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d68940 0x560ec1d4f140 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f0d78001e10 tx=0x7f0d7800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a77dab81b92100c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d82e12640 1 -- 172.21.15.40:0/879596290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800be40 con 0x560ec1d68940 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d82e12640 1 -- 172.21.15.40:0/879596290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7800f400 con 0x560ec1d68940 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d82e12640 1 -- 172.21.15.40:0/879596290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800f7f0 con 0x560ec1d68940 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/879596290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d68940 msgr2=0x560ec1d4f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d68940 0x560ec1d4f140 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f0d78001e10 tx=0x7f0d7800b670 comp rx=0 tx=0).stop 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/879596290 shutdown_connections 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 0x560ec1d1cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d68940 0x560ec1d4f140 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/879596290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d67700 0x560ec1dec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/879596290 >> 172.21.15.40:0/879596290 conn(0x560ec1d4a9b0 msgr2=0x560ec1d4fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:42.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/879596290 shutdown_connections 2024-04-26T21:46:42.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.960+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/879596290 wait complete. 2024-04-26T21:46:42.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.961+0000 7f0d86b1bf40 1 Processor -- start 2024-04-26T21:46:42.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.961+0000 7f0d86b1bf40 1 -- start start 2024-04-26T21:46:42.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d86b1bf40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d67700 0x560ec1e8d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d86b1bf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d68940 0x560ec1deb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d86b1bf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 0x560ec1d1cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d86b1bf40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560ec1df6120 con 0x560ec1d67700 2024-04-26T21:46:42.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d86b1bf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560ec1df6b00 con 0x560ec1df46b0 2024-04-26T21:46:42.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d86b1bf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560ec1dec560 con 0x560ec1d68940 2024-04-26T21:46:42.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d67700 0x560ec1e8d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d67700 0x560ec1e8d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54032/0 (socket says 172.21.15.40:54032) 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 learned_addr learned my addr 172.21.15.40:0/3282496130 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d83e14640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d68940 0x560ec1deb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84e16640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 0x560ec1d1cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d68940 msgr2=0x560ec1deb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560ec1d68940 0x560ec1deb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 msgr2=0x560ec1d1cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ec1df46b0 0x560ec1d1cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ec1d4f270 con 0x560ec1d67700 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.962+0000 7f0d84615640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d67700 0x560ec1e8d060 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c058040 tx=0x7f0d7c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.963+0000 7f0d81e10640 1 -- 172.21.15.40:0/3282496130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c065070 con 0x560ec1d67700 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.963+0000 7f0d81e10640 1 -- 172.21.15.40:0/3282496130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7c055460 con 0x560ec1d67700 2024-04-26T21:46:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.963+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ec1dee6e0 con 0x560ec1d67700 2024-04-26T21:46:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.963+0000 7f0d81e10640 1 -- 172.21.15.40:0/3282496130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c05a040 con 0x560ec1d67700 2024-04-26T21:46:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.963+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ec1deeca0 con 0x560ec1d67700 2024-04-26T21:46:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.964+0000 7f0d81e10640 1 -- 172.21.15.40:0/3282496130 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d7c068070 con 0x560ec1d67700 2024-04-26T21:46:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.965+0000 7f0d81e10640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d7006b300 0x7f0d7006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.965+0000 7f0d81e10640 1 -- 172.21.15.40:0/3282496130 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9399+0+0 (secure 0 0 0) 0x7f0d7c0cef70 con 0x560ec1d67700 2024-04-26T21:46:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.965+0000 7f0d83e14640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d7006b300 0x7f0d7006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.968 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:46:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.965+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560ec1def1c0 0x560ec1def5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.965+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1df3560 con 0x560ec1def1c0 2024-04-26T21:46:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.965+0000 7f0d84e16640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560ec1def1c0 0x560ec1def5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.965+0000 7f0d83e14640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d7006b300 0x7f0d7006d7c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x560ec1d67070 tx=0x7f0d78009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.965+0000 7f0d84e16640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560ec1def1c0 0x560ec1def5a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.967+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v132'1128 uv1102 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.967+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1df3090 con 0x560ec1def1c0 2024-04-26T21:46:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.968+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v132'1129 uv1129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.969+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1defae0 con 0x560ec1def1c0 2024-04-26T21:46:42.970 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:46:42.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.970+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v132'1130 uv1130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.970+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9a690 con 0x560ec1def1c0 2024-04-26T21:46:42.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.971+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v132'1131 uv1131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.971+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9aa30 con 0x560ec1def1c0 2024-04-26T21:46:42.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.972+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v132'1132 uv1132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.972+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99680 con 0x560ec1def1c0 2024-04-26T21:46:42.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.973+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v132'1133 uv1133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.974+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99a70 con 0x560ec1def1c0 2024-04-26T21:46:42.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.975+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v132'1134 uv1108 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.975+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.976+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v132'1135 uv1135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.976+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.977+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v132'1136 uv1136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.978 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:46:42.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.977+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.978+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v132'1137 uv1137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.978+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.979+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v132'1138 uv1138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.979+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.980+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v132'1139 uv1139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.980+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.982+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v132'1140 uv1115 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.982+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.983+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v132'1141 uv1141 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.983+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.984+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v132'1142 uv1142 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.985 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:46:42.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.984+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.985+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v132'1143 uv1143 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.985+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.986+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v132'1144 uv1144 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.986+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e99e10 con 0x560ec1def1c0 2024-04-26T21:46:42.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.987+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v132'1145 uv1145 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.987+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:42.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.988+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v132'1146 uv1146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.988+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:42.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.990+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v132'1147 uv1121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.990+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:42.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.991+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v132'1148 uv1148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.991+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:42.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.992+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v132'1149 uv1149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.993 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:46:42.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.992+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9bde0 con 0x560ec1def1c0 2024-04-26T21:46:42.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.993+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v132'1150 uv1150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.993+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9bde0 con 0x560ec1def1c0 2024-04-26T21:46:42.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.994+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v132'1151 uv1151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.994+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:42.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.995+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v132'1152 uv1152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.995+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:42.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.996+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v132'1153 uv1127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.997+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:42.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.998+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v132'1154 uv1154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.998+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.999+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v132'1155 uv1155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:42.999 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:46:43.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:42.999+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9bde0 con 0x560ec1def1c0 2024-04-26T21:46:43.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.000+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v132'1156 uv1156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:43.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.000+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9bde0 con 0x560ec1def1c0 2024-04-26T21:46:43.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.001+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v132'1157 uv1157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:43.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.001+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9add0 con 0x560ec1def1c0 2024-04-26T21:46:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.002+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v132'1158 uv1158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1def1c0 2024-04-26T21:46:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.002+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560ec1e9b2e0 0x560ec1e9d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.002+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9dc10 con 0x560ec1e9b2e0 2024-04-26T21:46:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.002+0000 7f0d84615640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560ec1e9b2e0 0x560ec1e9d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.002+0000 7f0d84615640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560ec1e9b2e0 0x560ec1e9d6d0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.005+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v132'824 uv803 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.005+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1e9dfb0 con 0x560ec1e9b2e0 2024-04-26T21:46:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.006+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v132'825 uv825 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.006+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9e350 con 0x560ec1e9b2e0 2024-04-26T21:46:43.008 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:46:43.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.007+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v132'826 uv826 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.007+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9eba0 con 0x560ec1e9b2e0 2024-04-26T21:46:43.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.009+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v132'827 uv827 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.009+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9ef40 con 0x560ec1e9b2e0 2024-04-26T21:46:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.010+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v132'828 uv828 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.010+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9f2e0 con 0x560ec1e9b2e0 2024-04-26T21:46:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.012+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v132'829 uv822 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.012+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1e9f680 con 0x560ec1e9b2e0 2024-04-26T21:46:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.013+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v132'830 uv830 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.013+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.015 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:46:43.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.015+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v132'831 uv831 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.015+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.016+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v132'832 uv832 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.016+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.018+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v132'833 uv833 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.018+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.019+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v132'834 uv823 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.019+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.020+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v132'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.020+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.022 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:46:43.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.022+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v132'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.022+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea40e0 con 0x560ec1e9b2e0 2024-04-26T21:46:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.023+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v132'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.023+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea40e0 con 0x560ec1e9b2e0 2024-04-26T21:46:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.024+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v132'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.024+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.025+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v132'839 uv819 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.026+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.027+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v132'840 uv840 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.027+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.029 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:46:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.028+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v132'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.028+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea0a30 con 0x560ec1e9b2e0 2024-04-26T21:46:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.029+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v132'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.029+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea0a30 con 0x560ec1e9b2e0 2024-04-26T21:46:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.031+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v132'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.031+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1e9fa20 con 0x560ec1e9b2e0 2024-04-26T21:46:43.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.032+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v132'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1e9b2e0 2024-04-26T21:46:43.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.032+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560ec1ea00a0 0x560ec1ea2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.032+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea29d0 con 0x560ec1ea00a0 2024-04-26T21:46:43.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.032+0000 7f0d83e14640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560ec1ea00a0 0x560ec1ea2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.033+0000 7f0d83e14640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560ec1ea00a0 0x560ec1ea2490 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.034+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v132'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.034+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1ea2e70 con 0x560ec1ea00a0 2024-04-26T21:46:43.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.035+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v132'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.035+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea3350 con 0x560ec1ea00a0 2024-04-26T21:46:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.036+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v132'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.037 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:46:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.036+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea4c00 con 0x560ec1ea00a0 2024-04-26T21:46:43.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.038+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v132'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.038+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea4fa0 con 0x560ec1ea00a0 2024-04-26T21:46:43.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.039+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v132'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.039+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea3bf0 con 0x560ec1ea00a0 2024-04-26T21:46:43.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.040+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v132'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.040+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560ec1ea44f0 0x560ec1ea7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.040+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea7890 con 0x560ec1ea44f0 2024-04-26T21:46:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.040+0000 7f0d84e16640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560ec1ea44f0 0x560ec1ea7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.040+0000 7f0d84e16640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560ec1ea44f0 0x560ec1ea7350 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.042+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v132'173 uv172 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.042+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1ea7890 con 0x560ec1ea44f0 2024-04-26T21:46:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.043+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v132'174 uv174 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.043+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea7890 con 0x560ec1ea44f0 2024-04-26T21:46:43.045 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:46:43.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.044+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v132'175 uv175 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.044+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea7ca0 con 0x560ec1ea44f0 2024-04-26T21:46:43.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.046+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v132'176 uv176 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.046+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560ec1ea81b0 0x560ec1eaa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.046+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaab70 con 0x560ec1ea81b0 2024-04-26T21:46:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.046+0000 7f0d84615640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560ec1ea81b0 0x560ec1eaa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.046+0000 7f0d84615640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560ec1ea81b0 0x560ec1eaa5a0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.048+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v132'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.048+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea81b0 2024-04-26T21:46:43.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.049+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v132'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.049+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea81b0 2024-04-26T21:46:43.051 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:46:43.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.050+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v132'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.050+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eabf50 con 0x560ec1ea81b0 2024-04-26T21:46:43.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.051+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v132'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.051+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eabf50 con 0x560ec1ea81b0 2024-04-26T21:46:43.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.053+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v132'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.053+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea81b0 2024-04-26T21:46:43.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.054+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v132'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.054+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.056+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v132'404 uv398 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.056+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.057+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v132'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.057+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.058+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v132'406 uv406 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.059 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:46:43.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.058+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eabf50 con 0x560ec1ea44f0 2024-04-26T21:46:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.059+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v132'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.059+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eabf50 con 0x560ec1ea44f0 2024-04-26T21:46:43.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.060+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v132'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.060+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.061+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v132'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.061+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.063+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v132'410 uv403 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.063+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.064+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v132'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.064+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.065+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v132'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.066 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:46:43.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.065+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.066+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v132'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.066+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.067+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v132'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.067+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.069+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v132'183 uv182 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.069+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.070+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v132'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.070+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.071 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:46:43.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.071+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v132'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.071+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.072+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v132'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.072+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea44f0 2024-04-26T21:46:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.073+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v132'187 uv187 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d74001920 con 0x560ec1ea44f0 2024-04-26T21:46:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.073+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea81b0 2024-04-26T21:46:43.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.074+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v132'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.074+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea81b0 2024-04-26T21:46:43.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.076+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v132'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.076+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea81b0 2024-04-26T21:46:43.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.077+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v132'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.077 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:46:43.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.077+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea81b0 2024-04-26T21:46:43.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.077+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v132'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.077+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf40 con 0x560ec1ea81b0 2024-04-26T21:46:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.079+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v132'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.079+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaaf80 con 0x560ec1ea81b0 2024-04-26T21:46:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.079+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v132'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.079+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eab380 con 0x560ec1ea81b0 2024-04-26T21:46:43.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.081+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v132'365 uv359 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.081+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eab380 con 0x560ec1ea81b0 2024-04-26T21:46:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.082+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v132'366 uv366 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.082+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eab380 con 0x560ec1ea81b0 2024-04-26T21:46:43.083 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:46:43.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.083+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v132'367 uv367 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.083+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea3820 con 0x560ec1ea81b0 2024-04-26T21:46:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.085+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v132'368 uv368 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.085+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea3820 con 0x560ec1ea81b0 2024-04-26T21:46:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.086+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v132'369 uv369 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.086+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea3820 con 0x560ec1ea81b0 2024-04-26T21:46:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.087+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v132'370 uv364 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.087+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1ea3820 con 0x560ec1ea81b0 2024-04-26T21:46:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.088+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v132'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.088+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea3800 con 0x560ec1ea81b0 2024-04-26T21:46:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:46:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.089+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v132'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.089+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1ea3840 con 0x560ec1ea81b0 2024-04-26T21:46:43.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.090+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v132'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.090+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eae660 con 0x560ec1ea81b0 2024-04-26T21:46:43.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.091+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v132'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.091+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560ec1eac360 0x560ec1eaecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.091+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eae750 con 0x560ec1eac360 2024-04-26T21:46:43.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.092+0000 7f0d83e14640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560ec1eac360 0x560ec1eaecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.092+0000 7f0d83e14640 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560ec1eac360 0x560ec1eaecc0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.094+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v132'162 uv160 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1eac360 2024-04-26T21:46:43.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.094+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1eac360 2024-04-26T21:46:43.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.095+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v132'163 uv163 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1eac360 2024-04-26T21:46:43.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.095+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1eac360 2024-04-26T21:46:43.097 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:46:43.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.096+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v132'164 uv164 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1eac360 2024-04-26T21:46:43.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.096+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1eac360 2024-04-26T21:46:43.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.098+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v132'165 uv165 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1eac360 2024-04-26T21:46:43.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.098+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea81b0 2024-04-26T21:46:43.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.099+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v132'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.099+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea81b0 2024-04-26T21:46:43.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.100+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v132'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.100+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea81b0 2024-04-26T21:46:43.101 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:46:43.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.101+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v132'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.101+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea81b0 2024-04-26T21:46:43.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.102+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v132'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.102+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea81b0 2024-04-26T21:46:43.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.103+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v132'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.103+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea81b0 2024-04-26T21:46:43.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.104+0000 7f0d84615640 1 -- 172.21.15.40:0/3282496130 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v132'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7c071070 con 0x560ec1ea81b0 2024-04-26T21:46:43.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.104+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea00a0 2024-04-26T21:46:43.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.106+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v132'186 uv185 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.106+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea00a0 2024-04-26T21:46:43.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.107+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v132'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.107+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf200 con 0x560ec1ea00a0 2024-04-26T21:46:43.108 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:46:43.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.108+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v132'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.108+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf690 con 0x560ec1ea00a0 2024-04-26T21:46:43.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.109+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v132'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.109+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf640 con 0x560ec1ea00a0 2024-04-26T21:46:43.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.110+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v132'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.110+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf640 con 0x560ec1ea00a0 2024-04-26T21:46:43.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.111+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v132'455 uv446 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.111+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaf3a0 con 0x560ec1ea00a0 2024-04-26T21:46:43.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.112+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v132'456 uv456 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.112+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf3a0 con 0x560ec1ea00a0 2024-04-26T21:46:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.113+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v132'457 uv457 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.114 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:46:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.113+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf3a0 con 0x560ec1ea00a0 2024-04-26T21:46:43.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.114+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v132'458 uv458 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.114+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf360 con 0x560ec1ea00a0 2024-04-26T21:46:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.115+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v132'459 uv459 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.115+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf360 con 0x560ec1ea00a0 2024-04-26T21:46:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.117+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v132'460 uv453 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.117+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x560ec1eaf360 con 0x560ec1ea00a0 2024-04-26T21:46:43.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.118+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v132'461 uv461 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.118+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf360 con 0x560ec1ea00a0 2024-04-26T21:46:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.119+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v132'462 uv462 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.119 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:46:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.119+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eb0760 con 0x560ec1ea00a0 2024-04-26T21:46:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.120+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v132'463 uv463 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.120+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eb0760 con 0x560ec1ea00a0 2024-04-26T21:46:43.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.121+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v132'464 uv464 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.121+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf480 con 0x560ec1ea00a0 2024-04-26T21:46:43.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.122+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v132'465 uv465 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.122+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x560ec1eaf930 con 0x560ec1ea00a0 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d83e14640 1 -- 172.21.15.40:0/3282496130 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v132'466 uv466 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0d7800f910 con 0x560ec1ea00a0 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560ec1e9b2e0 msgr2=0x560ec1e9d6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560ec1e9b2e0 0x560ec1e9d6d0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560ec1def1c0 msgr2=0x560ec1def5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560ec1def1c0 0x560ec1def5a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560ec1ea44f0 msgr2=0x560ec1ea7350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560ec1ea44f0 0x560ec1ea7350 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560ec1eac360 msgr2=0x560ec1eaecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560ec1eac360 0x560ec1eaecc0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 reap_dead start 2024-04-26T21:46:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d84e16640 1 -- 172.21.15.40:0/3282496130 reap_dead start 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560ec1ea00a0 msgr2=0x560ec1ea2490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560ec1ea00a0 0x560ec1ea2490 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560ec1ea81b0 msgr2=0x560ec1eaa5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560ec1ea81b0 0x560ec1eaa5a0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d7006b300 msgr2=0x7f0d7006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d7006b300 0x7f0d7006d7c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x560ec1d67070 tx=0x7f0d78009000 comp rx=0 tx=0).stop 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d67700 msgr2=0x560ec1e8d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d67700 0x560ec1e8d060 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c058040 tx=0x7f0d7c002fb0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 shutdown_connections 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560ec1ea81b0 0x560ec1eaa5a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560ec1ea00a0 0x560ec1ea2490 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0d7006b300 0x7f0d7006d7c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 --2- 172.21.15.40:0/3282496130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560ec1d67700 0x560ec1e8d060 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 >> 172.21.15.40:0/3282496130 conn(0x560ec1d4a9b0 msgr2=0x560ec1d4fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 shutdown_connections 2024-04-26T21:46:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.123+0000 7f0d86b1bf40 1 -- 172.21.15.40:0/3282496130 wait complete. 2024-04-26T21:46:43.128 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 100.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:46:43.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.210+0000 7f6d8efb3f40 1 Processor -- start 2024-04-26T21:46:43.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.210+0000 7f6d8efb3f40 1 -- start start 2024-04-26T21:46:43.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.210+0000 7f6d8efb3f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c1058bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.210+0000 7f6d8efb3f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10605d60 0x563c10572dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.210+0000 7f6d8efb3f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10623610 0x563c10573310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.210+0000 7f6d8efb3f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563c10606140 con 0x563c1058ef60 2024-04-26T21:46:43.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.210+0000 7f6d8efb3f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c104efe40 con 0x563c10605d60 2024-04-26T21:46:43.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.210+0000 7f6d8efb3f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563c1058f8b0 con 0x563c10623610 2024-04-26T21:46:43.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c1058bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c1058bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54046/0 (socket says 172.21.15.40:54046) 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 -- 172.21.15.40:0/376518213 learned_addr learned my addr 172.21.15.40:0/376518213 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8c2ac640 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10605d60 0x563c10572dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8d2ae640 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10623610 0x563c10573310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 -- 172.21.15.40:0/376518213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10623610 msgr2=0x563c10573310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10623610 0x563c10573310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 -- 172.21.15.40:0/376518213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10605d60 msgr2=0x563c10572dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10605d60 0x563c10572dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 -- 172.21.15.40:0/376518213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c10573850 con 0x563c1058ef60 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8caad640 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c1058bd90 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98ca321fa254f87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8b2aa640 1 -- 172.21.15.40:0/376518213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c055030 con 0x563c1058ef60 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8b2aa640 1 -- 172.21.15.40:0/376518213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7c051b40 con 0x563c1058ef60 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.211+0000 7f6d8b2aa640 1 -- 172.21.15.40:0/376518213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c05b430 con 0x563c1058ef60 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/376518213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 msgr2=0x563c1058bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c1058bd90 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c050000 comp rx=0 tx=0).stop 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/376518213 shutdown_connections 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10623610 0x563c10573310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10605d60 0x563c10572dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 --2- 172.21.15.40:0/376518213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c1058bd90 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/376518213 >> 172.21.15.40:0/376518213 conn(0x563c10546ab0 msgr2=0x563c10571fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/376518213 shutdown_connections 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/376518213 wait complete. 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 Processor -- start 2024-04-26T21:46:43.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.212+0000 7f6d8efb3f40 1 -- start start 2024-04-26T21:46:43.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8efb3f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c10614c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8efb3f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10605d60 0x563c10603700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8efb3f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10623610 0x563c105fc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8efb3f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563c10573e40 con 0x563c1058ef60 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8efb3f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c10573bf0 con 0x563c10623610 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8efb3f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563c1058c2d0 con 0x563c10605d60 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8caad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c10614c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8caad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c10614c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54058/0 (socket says 172.21.15.40:54058) 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8caad640 1 -- 172.21.15.40:0/670193817 learned_addr learned my addr 172.21.15.40:0/670193817 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8d2ae640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10623610 0x563c105fc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8c2ac640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10605d60 0x563c10603700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8caad640 1 -- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10605d60 msgr2=0x563c10603700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.213+0000 7f6d8caad640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563c10605d60 0x563c10603700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8caad640 1 -- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10623610 msgr2=0x563c105fc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8caad640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10623610 0x563c105fc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8caad640 1 -- 172.21.15.40:0/670193817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c106007c0 con 0x563c1058ef60 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8d2ae640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c10623610 0x563c105fc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8caad640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c10614c80 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c025a20 tx=0x7f6d7c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8a2a8640 1 -- 172.21.15.40:0/670193817 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c055040 con 0x563c1058ef60 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8a2a8640 1 -- 172.21.15.40:0/670193817 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7c051920 con 0x563c1058ef60 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563c10600aa0 con 0x563c1058ef60 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8a2a8640 1 -- 172.21.15.40:0/670193817 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c066a40 con 0x563c1058ef60 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.214+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c10601030 con 0x563c1058ef60 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.215+0000 7f6d8a2a8640 1 -- 172.21.15.40:0/670193817 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d7c058020 con 0x563c1058ef60 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.216+0000 7f6d8a2a8640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d6c06b2d0 0x7f6d6c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.216+0000 7f6d8a2a8640 1 -- 172.21.15.40:0/670193817 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9399+0+0 (secure 0 0 0) 0x7f6d7c0d0ae0 con 0x563c1058ef60 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.216+0000 7f6d8efb3f40 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563c10603c40 0x563c106022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.216+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x563c105fcd40 con 0x563c10603c40 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.216+0000 7f6d8c2ac640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d6c06b2d0 0x7f6d6c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.216+0000 7f6d8d2ae640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563c10603c40 0x563c106022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.216+0000 7f6d8c2ac640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d6c06b2d0 0x7f6d6c06d790 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x563c1058f5f0 tx=0x7f6d7400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.217+0000 7f6d8d2ae640 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563c10603c40 0x563c106022e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8d2ae640 1 -- 172.21.15.40:0/670193817 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(1 100.00000000 [omap-set-header] v132'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6d8400f8d0 con 0x563c10603c40 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563c10603c40 msgr2=0x563c106022e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x563c10603c40 0x563c106022e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d6c06b2d0 msgr2=0x7f6d6c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6d6c06b2d0 0x7f6d6c06d790 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x563c1058f5f0 tx=0x7f6d7400c040 comp rx=0 tx=0).stop 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 msgr2=0x563c10614c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 --2- 172.21.15.40:0/670193817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563c1058ef60 0x563c10614c80 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c025a20 tx=0x7f6d7c003330 comp rx=0 tx=0).stop 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8d2ae640 1 -- 172.21.15.40:0/670193817 reap_dead start 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 shutdown_connections 2024-04-26T21:46:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 >> 172.21.15.40:0/670193817 conn(0x563c10546ab0 msgr2=0x563c10571fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 shutdown_connections 2024-04-26T21:46:43.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.218+0000 7f6d8efb3f40 1 -- 172.21.15.40:0/670193817 wait complete. 2024-04-26T21:46:43.222 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:46:43.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.365+0000 7fb518bd4640 1 Processor -- start 2024-04-26T21:46:43.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.365+0000 7fb518bd4640 1 -- start start 2024-04-26T21:46:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb518bd4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb51415f170 0x7fb5140d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb518bd4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb51415f550 0x7fb5140d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb518bd4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb5140d34b0 0x7fb5140d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb518bd4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb5140c9430 con 0x7fb51415f170 2024-04-26T21:46:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb518bd4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb5140d3e20 con 0x7fb5140d34b0 2024-04-26T21:46:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb518bd4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5140d1040 con 0x7fb51415f550 2024-04-26T21:46:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb51259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb51415f170 0x7fb5140d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb51259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb51415f170 0x7fb5140d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54068/0 (socket says 172.21.15.40:54068) 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb51259c640 1 -- 172.21.15.40:0/3893660060 learned_addr learned my addr 172.21.15.40:0/3893660060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb50bfff640 1 --2- 172.21.15.40:0/3893660060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb51415f550 0x7fb5140d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb512d9d640 1 --2- 172.21.15.40:0/3893660060 >> v2:172.21.15.184:3300/0 conn(0x7fb5140d34b0 0x7fb5140d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb51259c640 1 -- 172.21.15.40:0/3893660060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb51415f550 msgr2=0x7fb5140d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb51259c640 1 --2- 172.21.15.40:0/3893660060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb51415f550 0x7fb5140d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb51259c640 1 -- 172.21.15.40:0/3893660060 >> v2:172.21.15.184:3300/0 conn(0x7fb5140d34b0 msgr2=0x7fb5140d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb51259c640 1 --2- 172.21.15.40:0/3893660060 >> v2:172.21.15.184:3300/0 conn(0x7fb5140d34b0 0x7fb5140d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.366+0000 7fb51259c640 1 -- 172.21.15.40:0/3893660060 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5140c9680 con 0x7fb51415f170 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.367+0000 7fb51259c640 1 --2- 172.21.15.40:0/3893660060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140d5a80 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fb51408f770 tx=0x7fb50402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d887d11c1767847 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.367+0000 7fb511d9b640 1 -- 172.21.15.40:0/3893660060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504037070 con 0x7fb51415f170 2024-04-26T21:46:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.367+0000 7fb511d9b640 1 -- 172.21.15.40:0/3893660060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb504032440 con 0x7fb51415f170 2024-04-26T21:46:43.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.367+0000 7fb511d9b640 1 -- 172.21.15.40:0/3893660060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504032810 con 0x7fb51415f170 2024-04-26T21:46:43.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 -- 172.21.15.40:0/3893660060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 msgr2=0x7fb5140d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 --2- 172.21.15.40:0/3893660060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140d5a80 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fb51408f770 tx=0x7fb50402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 -- 172.21.15.40:0/3893660060 shutdown_connections 2024-04-26T21:46:43.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 --2- 172.21.15.40:0/3893660060 >> v2:172.21.15.184:3300/0 conn(0x7fb5140d34b0 0x7fb5140d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 --2- 172.21.15.40:0/3893660060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb51415f550 0x7fb5140d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 --2- 172.21.15.40:0/3893660060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140d5a80 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 -- 172.21.15.40:0/3893660060 >> 172.21.15.40:0/3893660060 conn(0x7fb5140d4f60 msgr2=0x7fb5140d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 -- 172.21.15.40:0/3893660060 shutdown_connections 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.368+0000 7fb518bd4640 1 -- 172.21.15.40:0/3893660060 wait complete. 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.369+0000 7fb518bd4640 1 Processor -- start 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.369+0000 7fb518bd4640 1 -- start start 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb518bd4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5140d34b0 0x7fb51414f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb518bd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb518bd4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5141f8650 0x7fb5140b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb518bd4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140cfc50 con 0x7fb51415f170 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb518bd4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140c9c40 con 0x7fb5140d34b0 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb518bd4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5140cfdd0 con 0x7fb5141f8650 2024-04-26T21:46:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54076/0 (socket says 172.21.15.40:54076) 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 -- 172.21.15.40:0/1905712295 learned_addr learned my addr 172.21.15.40:0/1905712295 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb512d9d640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5141f8650 0x7fb5140b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb51259c640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5140d34b0 0x7fb51414f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 -- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5141f8650 msgr2=0x7fb5140b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5141f8650 0x7fb5140b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 -- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5140d34b0 msgr2=0x7fb51414f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5140d34b0 0x7fb51414f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 -- 172.21.15.40:0/1905712295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5141f8b50 con 0x7fb51415f170 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb512d9d640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5141f8650 0x7fb5140b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb51259c640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5140d34b0 0x7fb51414f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.370+0000 7fb50bfff640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140ce8f0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00e870 tx=0x7fb4fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.371+0000 7fb50affd640 1 -- 172.21.15.40:0/1905712295 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc017070 con 0x7fb51415f170 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.371+0000 7fb50affd640 1 -- 172.21.15.40:0/1905712295 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc00c940 con 0x7fb51415f170 2024-04-26T21:46:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.371+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5141f8e30 con 0x7fb51415f170 2024-04-26T21:46:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.371+0000 7fb50affd640 1 -- 172.21.15.40:0/1905712295 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00cd30 con 0x7fb51415f170 2024-04-26T21:46:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.371+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5141f93f0 con 0x7fb51415f170 2024-04-26T21:46:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.372+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5140694f0 con 0x7fb51415f170 2024-04-26T21:46:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.372+0000 7fb50affd640 1 -- 172.21.15.40:0/1905712295 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4fc01a070 con 0x7fb51415f170 2024-04-26T21:46:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.372+0000 7fb50affd640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.373+0000 7fb50affd640 1 -- 172.21.15.40:0/1905712295 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9399+0+0 (secure 0 0 0) 0x7fb4fc08cbd0 con 0x7fb51415f170 2024-04-26T21:46:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.373+0000 7fb51259c640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.373+0000 7fb51259c640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb504034000 tx=0x7fb50402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.377+0000 7fb50affd640 1 -- 172.21.15.40:0/1905712295 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4fc05a1d0 con 0x7fb51415f170 2024-04-26T21:46:43.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.601+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb51415a720 con 0x7fb51415f170 2024-04-26T21:46:43.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.844+0000 7fb50affd640 1 -- 172.21.15.40:0/1905712295 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v329) ==== 162+0+0 (secure 0 0 0) 0x7fb4fc059ad0 con 0x7fb51415f170 2024-04-26T21:46:43.845 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 msgr2=0x7fb4f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb504034000 tx=0x7fb50402e000 comp rx=0 tx=0).stop 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 msgr2=0x7fb5140ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140ce8f0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00e870 tx=0x7fb4fc00ed30 comp rx=0 tx=0).stop 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 shutdown_connections 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5141f8650 0x7fb5140b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51415f170 0x7fb5140ce8f0 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 --2- 172.21.15.40:0/1905712295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5140d34b0 0x7fb51414f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 >> 172.21.15.40:0/1905712295 conn(0x7fb5140d4f60 msgr2=0x7fb51406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 shutdown_connections 2024-04-26T21:46:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.846+0000 7fb518bd4640 1 -- 172.21.15.40:0/1905712295 wait complete. 2024-04-26T21:46:43.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:43.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.961+0000 7f5da6f39640 1 Processor -- start 2024-04-26T21:46:43.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.961+0000 7f5da6f39640 1 -- start start 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.961+0000 7f5da6f39640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5da00d06f0 0x7f5da00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.961+0000 7f5da6f39640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da00d0ba0 con 0x7f5da00b2850 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.961+0000 7f5da6f39640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da00d0e20 con 0x7f5da015f3a0 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.961+0000 7f5da6f39640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5da00b1d70 con 0x7f5da00d06f0 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.962+0000 7f5d9ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5da00d06f0 0x7f5da00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.962+0000 7f5da54d6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5da00b2850 0x7f5da00b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40428/0 (socket says 172.21.15.40:40428) 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.962+0000 7f5da54d6640 1 -- 172.21.15.40:0/4279520169 learned_addr learned my addr 172.21.15.40:0/4279520169 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.962+0000 7f5d9ffff640 1 -- 172.21.15.40:0/4279520169 >> v1:172.21.15.184:6789/0 conn(0x7f5da015f3a0 legacy=0x7f5da00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.962+0000 7f5d9ffff640 1 -- 172.21.15.40:0/4279520169 >> v1:172.21.15.40:6789/0 conn(0x7f5da00b2850 legacy=0x7f5da00b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.962+0000 7f5d9ffff640 1 -- 172.21.15.40:0/4279520169 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00b2050 con 0x7f5da00d06f0 2024-04-26T21:46:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.962+0000 7f5d9f7fe640 1 -- 172.21.15.40:0/4279520169 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028106754 0 0) 0x7f5d98001490 con 0x7f5da00b2850 2024-04-26T21:46:43.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5d9ffff640 1 --2- 172.21.15.40:0/4279520169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00d06f0 0x7f5da00d1e80 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0097a0 tx=0x7f5d8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91480ab5dc7aeb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5d9f7fe640 1 -- 172.21.15.40:0/4279520169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c037070 con 0x7f5da00d06f0 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5d9f7fe640 1 -- 172.21.15.40:0/4279520169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d8c032440 con 0x7f5da00d06f0 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5d9f7fe640 1 -- 172.21.15.40:0/4279520169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c032810 con 0x7f5da00d06f0 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5da6f39640 1 -- 172.21.15.40:0/4279520169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00d06f0 msgr2=0x7f5da00d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5da6f39640 1 --2- 172.21.15.40:0/4279520169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00d06f0 0x7f5da00d1e80 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0097a0 tx=0x7f5d8c02a230 comp rx=0 tx=0).stop 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5da6f39640 1 -- 172.21.15.40:0/4279520169 shutdown_connections 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5da6f39640 1 --2- 172.21.15.40:0/4279520169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00d06f0 0x7f5da00d1e80 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5da6f39640 1 -- 172.21.15.40:0/4279520169 >> 172.21.15.40:0/4279520169 conn(0x7f5da00d4f90 msgr2=0x7f5da00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5da6f39640 1 -- 172.21.15.40:0/4279520169 shutdown_connections 2024-04-26T21:46:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.963+0000 7f5da6f39640 1 -- 172.21.15.40:0/4279520169 wait complete. 2024-04-26T21:46:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.964+0000 7f5da6f39640 1 Processor -- start 2024-04-26T21:46:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.965+0000 7f5da6f39640 1 -- start start 2024-04-26T21:46:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.965+0000 7f5da6f39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00b2850 0x7f5da00c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.965+0000 7f5da6f39640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da00d06f0 0x7f5da01f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.965+0000 7f5da6f39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5da015f3a0 0x7f5da014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.965+0000 7f5da6f39640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5da015f900 con 0x7f5da00b2850 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.965+0000 7f5da6f39640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5da015f780 con 0x7f5da00d06f0 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.965+0000 7f5da6f39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5da00d23c0 con 0x7f5da015f3a0 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00b2850 0x7f5da00c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00b2850 0x7f5da00c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54108/0 (socket says 172.21.15.40:54108) 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 -- 172.21.15.40:0/1869539314 learned_addr learned my addr 172.21.15.40:0/1869539314 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5d9ffff640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da00d06f0 0x7f5da01f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da54d6640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5da015f3a0 0x7f5da014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 -- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5da015f3a0 msgr2=0x7f5da014dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5da015f3a0 0x7f5da014dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 -- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da00d06f0 msgr2=0x7f5da01f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da00d06f0 0x7f5da01f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 -- 172.21.15.40:0/1869539314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00c9310 con 0x7f5da00b2850 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da4cd5640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00b2850 0x7f5da00c8ac0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c400 tx=0x7f5d9400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5d9d7fa640 1 -- 172.21.15.40:0/1869539314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94017070 con 0x7f5da00b2850 2024-04-26T21:46:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5d9d7fa640 1 -- 172.21.15.40:0/1869539314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d940052d0 con 0x7f5da00b2850 2024-04-26T21:46:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da00c95f0 con 0x7f5da00b2850 2024-04-26T21:46:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5d9d7fa640 1 -- 172.21.15.40:0/1869539314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94003830 con 0x7f5da00b2850 2024-04-26T21:46:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.966+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da00c9bb0 con 0x7f5da00b2850 2024-04-26T21:46:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.967+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da00694f0 con 0x7f5da00b2850 2024-04-26T21:46:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.968+0000 7f5d9d7fa640 1 -- 172.21.15.40:0/1869539314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d9401a070 con 0x7f5da00b2850 2024-04-26T21:46:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.968+0000 7f5d9d7fa640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d8406c8f0 0x7f5d8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.968+0000 7f5d9d7fa640 1 -- 172.21.15.40:0/1869539314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9399+0+0 (secure 0 0 0) 0x7f5d9408bd20 con 0x7f5da00b2850 2024-04-26T21:46:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.968+0000 7f5d9ffff640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d8406c8f0 0x7f5d8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.969+0000 7f5d9ffff640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d8406c8f0 0x7f5d8406edb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c034000 tx=0x7f5d8c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:43.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:43.972+0000 7f5d9d7fa640 1 -- 172.21.15.40:0/1869539314 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d94058be0 con 0x7f5da00b2850 2024-04-26T21:46:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.195+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5da00c8410 con 0x7f5da00b2850 2024-04-26T21:46:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.196+0000 7f5d9d7fa640 1 -- 172.21.15.40:0/1869539314 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 330 v330) ==== 95+0+5193 (secure 0 0 0) 0x7f5d9405cee0 con 0x7f5da00b2850 2024-04-26T21:46:44.197 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:44.197 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":330,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":330,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:43.847691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11379":{"gid":11379,"name":"c","rank":0,"incarnation":330,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/3502494134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3502494134},{"type":"v1","addr":"172.21.15.40:6833","nonce":3502494134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 330 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d8406c8f0 msgr2=0x7f5d8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d8406c8f0 0x7f5d8406edb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c034000 tx=0x7f5d8c02e000 comp rx=0 tx=0).stop 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00b2850 msgr2=0x7f5da00c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00b2850 0x7f5da00c8ac0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c400 tx=0x7f5d9400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 shutdown_connections 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5d8406c8f0 0x7f5d8406edb0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5da015f3a0 0x7f5da014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5da00d06f0 0x7f5da01f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 --2- 172.21.15.40:0/1869539314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da00b2850 0x7f5da00c8ac0 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 >> 172.21.15.40:0/1869539314 conn(0x7f5da00d4f90 msgr2=0x7f5da00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 shutdown_connections 2024-04-26T21:46:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:44.198+0000 7f5da6f39640 1 -- 172.21.15.40:0/1869539314 wait complete. 2024-04-26T21:46:44.209 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:46:46.871 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:46.870+0000 7f3c25641640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-26T21:46:46.872 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:46.872+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:46:46.873 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:46.872+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:46:46.880 INFO:tasks.ceph.mds.c.smithi040.stderr: -22> 2024-04-26T21:46:46.870+0000 7f3c25641640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-26T21:46:46.908 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:46:49.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.326+0000 7fb8e3a07640 1 Processor -- start 2024-04-26T21:46:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.326+0000 7fb8e3a07640 1 -- start start 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e3a07640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8dc162810 0x7fb8dc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e3a07640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8dc0b2340 0x7fb8dc0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e3a07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e3a07640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc0d0750 con 0x7fb8dc0b2340 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e3a07640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc0d16a0 con 0x7fb8dc162810 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e3a07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc0d10d0 con 0x7fb8dc0cd930 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e0fa2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8dc0b2340 0x7fb8dc0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e0fa2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8dc0b2340 0x7fb8dc0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59038/0 (socket says 172.21.15.40:59038) 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.327+0000 7fb8e0fa2640 1 -- 172.21.15.40:0/3430047253 learned_addr learned my addr 172.21.15.40:0/3430047253 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e1fa4640 1 --2- 172.21.15.40:0/3430047253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e0fa2640 1 -- 172.21.15.40:0/3430047253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 msgr2=0x7fb8dc0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e17a3640 1 --2- 172.21.15.40:0/3430047253 >> v2:172.21.15.184:3300/0 conn(0x7fb8dc162810 0x7fb8dc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e0fa2640 1 --2- 172.21.15.40:0/3430047253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e0fa2640 1 -- 172.21.15.40:0/3430047253 >> v2:172.21.15.184:3300/0 conn(0x7fb8dc162810 msgr2=0x7fb8dc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e0fa2640 1 --2- 172.21.15.40:0/3430047253 >> v2:172.21.15.184:3300/0 conn(0x7fb8dc162810 0x7fb8dc070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e0fa2640 1 -- 172.21.15.40:0/3430047253 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc0d1370 con 0x7fb8dc0b2340 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e1fa4640 1 --2- 172.21.15.40:0/3430047253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.328+0000 7fb8e0fa2640 1 --2- 172.21.15.40:0/3430047253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc0b2340 0x7fb8dc0cd3f0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc073d90 tx=0x7fb8cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d796f88a92ebf15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8d3fff640 1 -- 172.21.15.40:0/3430047253 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc037070 con 0x7fb8dc0b2340 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8d3fff640 1 -- 172.21.15.40:0/3430047253 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8cc032440 con 0x7fb8dc0b2340 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8d3fff640 1 -- 172.21.15.40:0/3430047253 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc032810 con 0x7fb8dc0b2340 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3430047253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc0b2340 msgr2=0x7fb8dc0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3430047253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc0b2340 0x7fb8dc0cd3f0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc073d90 tx=0x7fb8cc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3430047253 shutdown_connections 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3430047253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3430047253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc0b2340 0x7fb8dc0cd3f0 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3430047253 >> v2:172.21.15.184:3300/0 conn(0x7fb8dc162810 0x7fb8dc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3430047253 >> 172.21.15.40:0/3430047253 conn(0x7fb8dc0d2da0 msgr2=0x7fb8dc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3430047253 shutdown_connections 2024-04-26T21:46:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.329+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3430047253 wait complete. 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.330+0000 7fb8e3a07640 1 Processor -- start 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.330+0000 7fb8e3a07640 1 -- start start 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e3a07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc0b2340 0x7fb8dc0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e3a07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e3a07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc162810 0x7fb8dc0c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e3a07640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc070890 con 0x7fb8dc162810 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e3a07640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0d0470 con 0x7fb8dc0b2340 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e3a07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc0d2070 con 0x7fb8dc0cd930 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e1fa4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc162810 0x7fb8dc0c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e1fa4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc162810 0x7fb8dc0c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59052/0 (socket says 172.21.15.40:59052) 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e1fa4640 1 -- 172.21.15.40:0/3811258504 learned_addr learned my addr 172.21.15.40:0/3811258504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e0fa2640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.331+0000 7fb8e17a3640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc0b2340 0x7fb8dc0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e1fa4640 1 -- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 msgr2=0x7fb8dc15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e1fa4640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e1fa4640 1 -- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc0b2340 msgr2=0x7fb8dc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e1fa4640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc0b2340 0x7fb8dc0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e1fa4640 1 -- 172.21.15.40:0/3811258504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc0cc350 con 0x7fb8dc162810 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e0fa2640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e1fa4640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc162810 0x7fb8dc0c9e70 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800ce40 tx=0x7fb8d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8d1ffb640 1 -- 172.21.15.40:0/3811258504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d801a070 con 0x7fb8dc162810 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8d1ffb640 1 -- 172.21.15.40:0/3811258504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d800d840 con 0x7fb8dc162810 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8d1ffb640 1 -- 172.21.15.40:0/3811258504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800dc30 con 0x7fb8dc162810 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc0cc630 con 0x7fb8dc162810 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.332+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc0ccbf0 con 0x7fb8dc162810 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.333+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc0694f0 con 0x7fb8dc162810 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.334+0000 7fb8d1ffb640 1 -- 172.21.15.40:0/3811258504 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8d801d070 con 0x7fb8dc162810 2024-04-26T21:46:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.334+0000 7fb8d1ffb640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8c006c8f0 0x7fb8c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.334+0000 7fb8d1ffb640 1 -- 172.21.15.40:0/3811258504 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9485+0+0 (secure 0 0 0) 0x7fb8d808d290 con 0x7fb8dc162810 2024-04-26T21:46:49.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.334+0000 7fb8e17a3640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8c006c8f0 0x7fb8c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.335+0000 7fb8e17a3640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8c006c8f0 0x7fb8c006edb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4006fd0 tx=0x7fb8d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.339+0000 7fb8d1ffb640 1 -- 172.21.15.40:0/3811258504 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8d805a7b0 con 0x7fb8dc162810 2024-04-26T21:46:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.568+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8dc0d2430 con 0x7fb8dc162810 2024-04-26T21:46:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.568+0000 7fb8d1ffb640 1 -- 172.21.15.40:0/3811258504 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+5182 (secure 0 0 0) 0x7fb8d805a0b0 con 0x7fb8dc162810 2024-04-26T21:46:49.571 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:49.571 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":335,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3095512631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3095512631},{"type":"v1","addr":"172.21.15.40:6833","nonce":3095512631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:46.873691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 335 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8c006c8f0 msgr2=0x7fb8c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8c006c8f0 0x7fb8c006edb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4006fd0 tx=0x7fb8d4008040 comp rx=0 tx=0).stop 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc162810 msgr2=0x7fb8dc0c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc162810 0x7fb8dc0c9e70 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800ce40 tx=0x7fb8d800a710 comp rx=0 tx=0).stop 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 shutdown_connections 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb8c006c8f0 0x7fb8c006edb0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8dc162810 0x7fb8dc0c9e70 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8dc0cd930 0x7fb8dc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 --2- 172.21.15.40:0/3811258504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8dc0b2340 0x7fb8dc0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 >> 172.21.15.40:0/3811258504 conn(0x7fb8dc0d2da0 msgr2=0x7fb8dc070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 shutdown_connections 2024-04-26T21:46:49.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.571+0000 7fb8e3a07640 1 -- 172.21.15.40:0/3811258504 wait complete. 2024-04-26T21:46:49.583 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:46:49.583 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:49.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.683+0000 7f50f50bd640 1 Processor -- start 2024-04-26T21:46:49.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.683+0000 7f50f50bd640 1 -- start start 2024-04-26T21:46:49.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50f50bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f50f0162880 0x7f50f00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50f50bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 0x7f50f00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50f50bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f00b7e50 con 0x7f50f00d39f0 2024-04-26T21:46:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50f50bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f50f00d0580 con 0x7f50f0162880 2024-04-26T21:46:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50f50bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50f00d1800 con 0x7f50f00b5f20 2024-04-26T21:46:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50eed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f50f0162880 0x7f50f00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50eed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f50f0162880 0x7f50f00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59066/0 (socket says 172.21.15.40:59066) 2024-04-26T21:46:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50eed9d640 1 -- 172.21.15.40:0/3477037370 learned_addr learned my addr 172.21.15.40:0/3477037370 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50e7fff640 1 --2- 172.21.15.40:0/3477037370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 0x7f50f00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50eed9d640 1 -- 172.21.15.40:0/3477037370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 msgr2=0x7f50f00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50eed9d640 1 --2- 172.21.15.40:0/3477037370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 0x7f50f00d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50eed9d640 1 -- 172.21.15.40:0/3477037370 >> v1:172.21.15.184:6789/0 conn(0x7f50f00d39f0 legacy=0x7f50f00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.684+0000 7f50eed9d640 1 -- 172.21.15.40:0/3477037370 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f00d07d0 con 0x7f50f0162880 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50eed9d640 1 --2- 172.21.15.40:0/3477037370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f0162880 0x7f50f00d2cb0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009870 tx=0x7f50e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292e91fbbc2a92bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50ee59c640 1 -- 172.21.15.40:0/3477037370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0037070 con 0x7f50f0162880 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50ee59c640 1 -- 172.21.15.40:0/3477037370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e0032440 con 0x7f50f0162880 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50ee59c640 1 -- 172.21.15.40:0/3477037370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0032810 con 0x7f50f0162880 2024-04-26T21:46:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50f50bd640 1 -- 172.21.15.40:0/3477037370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f0162880 msgr2=0x7f50f00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50f50bd640 1 --2- 172.21.15.40:0/3477037370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f0162880 0x7f50f00d2cb0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009870 tx=0x7f50e002a210 comp rx=0 tx=0).stop 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50f50bd640 1 -- 172.21.15.40:0/3477037370 shutdown_connections 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50f50bd640 1 --2- 172.21.15.40:0/3477037370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 0x7f50f00d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50f50bd640 1 --2- 172.21.15.40:0/3477037370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f0162880 0x7f50f00d2cb0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50f50bd640 1 -- 172.21.15.40:0/3477037370 >> 172.21.15.40:0/3477037370 conn(0x7f50f00d4f60 msgr2=0x7f50f00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.685+0000 7f50f50bd640 1 -- 172.21.15.40:0/3477037370 shutdown_connections 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.686+0000 7f50f50bd640 1 -- 172.21.15.40:0/3477037370 wait complete. 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.687+0000 7f50f50bd640 1 Processor -- start 2024-04-26T21:46:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.687+0000 7f50f50bd640 1 -- start start 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.687+0000 7f50f50bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 0x7f50f00b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50f50bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00d39f0 0x7f50f00cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50f50bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f0162880 0x7f50f0153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50f50bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0162c60 con 0x7f50f00d39f0 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50f50bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50f00d0d90 con 0x7f50f0162880 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50f50bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50f0162de0 con 0x7f50f00b5f20 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00d39f0 0x7f50f00cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00d39f0 0x7f50f00cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59070/0 (socket says 172.21.15.40:59070) 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 -- 172.21.15.40:0/473734187 learned_addr learned my addr 172.21.15.40:0/473734187 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50eed9d640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 0x7f50f00b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50ef59e640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f0162880 0x7f50f0153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 -- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 msgr2=0x7f50f00b9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 0x7f50f00b9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 -- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f0162880 msgr2=0x7f50f0153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f0162880 0x7f50f0153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 -- 172.21.15.40:0/473734187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f00ce420 con 0x7f50f00d39f0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50ef59e640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f0162880 0x7f50f0153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e7fff640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00d39f0 0x7f50f00cda40 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c7f0 tx=0x7f50d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e77fe640 1 -- 172.21.15.40:0/473734187 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8015070 con 0x7f50f00d39f0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e77fe640 1 -- 172.21.15.40:0/473734187 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d8004220 con 0x7f50f00d39f0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50e77fe640 1 -- 172.21.15.40:0/473734187 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8007620 con 0x7f50f00d39f0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f00ce700 con 0x7f50f00d39f0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.688+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f0159d90 con 0x7f50f00d39f0 2024-04-26T21:46:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.689+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f00694f0 con 0x7f50f00d39f0 2024-04-26T21:46:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.690+0000 7f50e77fe640 1 -- 172.21.15.40:0/473734187 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50d80079d0 con 0x7f50f00d39f0 2024-04-26T21:46:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.690+0000 7f50e77fe640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f50d006c8f0 0x7f50d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.690+0000 7f50e77fe640 1 -- 172.21.15.40:0/473734187 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9485+0+0 (secure 0 0 0) 0x7f50d808cde0 con 0x7f50f00d39f0 2024-04-26T21:46:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.691+0000 7f50eed9d640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f50d006c8f0 0x7f50d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.691+0000 7f50eed9d640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f50d006c8f0 0x7f50d006edb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f50e0034000 tx=0x7f50e002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.695+0000 7f50e77fe640 1 -- 172.21.15.40:0/473734187 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f50d8059c30 con 0x7f50f00d39f0 2024-04-26T21:46:49.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.918+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50f0159f60 con 0x7f50f00d39f0 2024-04-26T21:46:49.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.919+0000 7f50e77fe640 1 -- 172.21.15.40:0/473734187 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+5182 (secure 0 0 0) 0x7f50d805df30 con 0x7f50f00d39f0 2024-04-26T21:46:49.920 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:49.920 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":335,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3095512631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3095512631},{"type":"v1","addr":"172.21.15.40:6833","nonce":3095512631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:46.873691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:49.920 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 335 2024-04-26T21:46:49.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f50d006c8f0 msgr2=0x7f50d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f50d006c8f0 0x7f50d006edb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f50e0034000 tx=0x7f50e002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:49.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00d39f0 msgr2=0x7f50f00cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:49.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00d39f0 0x7f50f00cda40 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c7f0 tx=0x7f50d800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 shutdown_connections 2024-04-26T21:46:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f50d006c8f0 0x7f50d006edb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f0162880 0x7f50f0153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00d39f0 0x7f50f00cda40 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 --2- 172.21.15.40:0/473734187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00b5f20 0x7f50f00b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 >> 172.21.15.40:0/473734187 conn(0x7f50f00d4f60 msgr2=0x7f50f015f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 shutdown_connections 2024-04-26T21:46:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:49.921+0000 7f50f50bd640 1 -- 172.21.15.40:0/473734187 wait complete. 2024-04-26T21:46:49.933 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:46:49.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.033+0000 7f37d92bd640 1 Processor -- start 2024-04-26T21:46:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.033+0000 7f37d92bd640 1 -- start start 2024-04-26T21:46:50.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.034+0000 7f37d92bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f37d415f3a0 0x7f37d40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.034+0000 7f37d92bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37d40d2330 0x7f37d40b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.034+0000 7f37d92bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37d40d0e20 con 0x7f37d40d1770 2024-04-26T21:46:50.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.034+0000 7f37d92bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f37d40b1d70 con 0x7f37d40d2330 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.034+0000 7f37d92bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f37d40b2050 con 0x7f37d415f3a0 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.034+0000 7f37d37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37d40d2330 0x7f37d40b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.034+0000 7f37d37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37d40d2330 0x7f37d40b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59084/0 (socket says 172.21.15.40:59084) 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.034+0000 7f37d37fe640 1 -- 172.21.15.40:0/1152059429 learned_addr learned my addr 172.21.15.40:0/1152059429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.035+0000 7f37d2ffd640 1 --2- 172.21.15.40:0/1152059429 >> v2:172.21.15.184:3300/0 conn(0x7f37d415f3a0 0x7f37d40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.035+0000 7f37d37fe640 1 -- 172.21.15.40:0/1152059429 >> v2:172.21.15.184:3300/0 conn(0x7f37d415f3a0 msgr2=0x7f37d40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.035+0000 7f37d37fe640 1 --2- 172.21.15.40:0/1152059429 >> v2:172.21.15.184:3300/0 conn(0x7f37d415f3a0 0x7f37d40d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.035+0000 7f37d37fe640 1 -- 172.21.15.40:0/1152059429 >> v1:172.21.15.184:6789/0 conn(0x7f37d40d1770 legacy=0x7f37d40d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.035+0000 7f37d37fe640 1 -- 172.21.15.40:0/1152059429 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d40b2330 con 0x7f37d40d2330 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.035+0000 7f37d37fe640 1 --2- 172.21.15.40:0/1152059429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b2850 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f37d40c89f0 tx=0x7f37c402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba10d40c65830e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d1ffb640 1 -- 172.21.15.40:0/1152059429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4036070 con 0x7f37d40d2330 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d1ffb640 1 -- 172.21.15.40:0/1152059429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c4031600 con 0x7f37d40d2330 2024-04-26T21:46:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d1ffb640 1 -- 172.21.15.40:0/1152059429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c40319d0 con 0x7f37d40d2330 2024-04-26T21:46:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d92bd640 1 -- 172.21.15.40:0/1152059429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 msgr2=0x7f37d40b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d92bd640 1 --2- 172.21.15.40:0/1152059429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b2850 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f37d40c89f0 tx=0x7f37c402a6d0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d92bd640 1 -- 172.21.15.40:0/1152059429 shutdown_connections 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d92bd640 1 --2- 172.21.15.40:0/1152059429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b2850 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d92bd640 1 --2- 172.21.15.40:0/1152059429 >> v2:172.21.15.184:3300/0 conn(0x7f37d415f3a0 0x7f37d40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d92bd640 1 -- 172.21.15.40:0/1152059429 >> 172.21.15.40:0/1152059429 conn(0x7f37d40d4f90 msgr2=0x7f37d40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d92bd640 1 -- 172.21.15.40:0/1152059429 shutdown_connections 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.036+0000 7f37d92bd640 1 -- 172.21.15.40:0/1152059429 wait complete. 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.037+0000 7f37d92bd640 1 Processor -- start 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.037+0000 7f37d92bd640 1 -- start start 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d92bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d40d1770 0x7f37d40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d92bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d92bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d415f3a0 0x7f37d40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d92bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37d40b2f10 con 0x7f37d40d2330 2024-04-26T21:46:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d92bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37d40b2d90 con 0x7f37d40d1770 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d92bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37d40b3090 con 0x7f37d415f3a0 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59096/0 (socket says 172.21.15.40:59096) 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.038+0000 7f37d27fc640 1 -- 172.21.15.40:0/3132956382 learned_addr learned my addr 172.21.15.40:0/3132956382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d37fe640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d415f3a0 0x7f37d40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d27fc640 1 -- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d415f3a0 msgr2=0x7f37d40b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d2ffd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d40d1770 0x7f37d40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d27fc640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d415f3a0 0x7f37d40b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d27fc640 1 -- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d40d1770 msgr2=0x7f37d40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d27fc640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d40d1770 0x7f37d40ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d27fc640 1 -- 172.21.15.40:0/3132956382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d40cee00 con 0x7f37d40d2330 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d37fe640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d415f3a0 0x7f37d40b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d2ffd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d40d1770 0x7f37d40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d27fc640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b4880 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c410 tx=0x7f37bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37cb7fe640 1 -- 172.21.15.40:0/3132956382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc017070 con 0x7f37d40d2330 2024-04-26T21:46:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37cb7fe640 1 -- 172.21.15.40:0/3132956382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37bc0035f0 con 0x7f37d40d2330 2024-04-26T21:46:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37cb7fe640 1 -- 172.21.15.40:0/3132956382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc003a20 con 0x7f37d40d2330 2024-04-26T21:46:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d41f5720 con 0x7f37d40d2330 2024-04-26T21:46:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.039+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d41f5ce0 con 0x7f37d40d2330 2024-04-26T21:46:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.040+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d41f5ee0 con 0x7f37d40d2330 2024-04-26T21:46:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.040+0000 7f37cb7fe640 1 -- 172.21.15.40:0/3132956382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37bc01a070 con 0x7f37d40d2330 2024-04-26T21:46:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.041+0000 7f37cb7fe640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b406c8f0 0x7f37b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.041+0000 7f37cb7fe640 1 -- 172.21.15.40:0/3132956382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9485+0+0 (secure 0 0 0) 0x7f37bc08bf70 con 0x7f37d40d2330 2024-04-26T21:46:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.041+0000 7f37d2ffd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b406c8f0 0x7f37b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.042+0000 7f37d2ffd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b406c8f0 0x7f37b406edb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00b820 tx=0x7f37cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.045+0000 7f37cb7fe640 1 -- 172.21.15.40:0/3132956382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37bc059490 con 0x7f37d40d2330 2024-04-26T21:46:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.271+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37d40b40d0 con 0x7f37d40d2330 2024-04-26T21:46:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.272+0000 7f37cb7fe640 1 -- 172.21.15.40:0/3132956382 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+5182 (secure 0 0 0) 0x7f37bc0010c0 con 0x7f37d40d2330 2024-04-26T21:46:50.273 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:50.273 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":335,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3095512631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3095512631},{"type":"v1","addr":"172.21.15.40:6833","nonce":3095512631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:46.873691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 335 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.273+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b406c8f0 msgr2=0x7f37b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.273+0000 7f37d92bd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b406c8f0 0x7f37b406edb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00b820 tx=0x7f37cc008040 comp rx=0 tx=0).stop 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.273+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 msgr2=0x7f37d40b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.273+0000 7f37d92bd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b4880 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c410 tx=0x7f37bc00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.274+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 shutdown_connections 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.274+0000 7f37d92bd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37b406c8f0 0x7f37b406edb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.274+0000 7f37d92bd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d415f3a0 0x7f37d40b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.274+0000 7f37d92bd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d40d2330 0x7f37d40b4880 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.274+0000 7f37d92bd640 1 --2- 172.21.15.40:0/3132956382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d40d1770 0x7f37d40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.274+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 >> 172.21.15.40:0/3132956382 conn(0x7f37d40d4f90 msgr2=0x7f37d40d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.274+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 shutdown_connections 2024-04-26T21:46:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.274+0000 7f37d92bd640 1 -- 172.21.15.40:0/3132956382 wait complete. 2024-04-26T21:46:50.286 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 100.00000000' led to DAMAGED state 2024-04-26T21:46:50.286 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_sessionmap' 2024-04-26T21:46:50.286 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:50.286 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:50.302 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:46:50.302 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:46:50.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:50.302 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:46:50.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:46:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:46:50.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:46:50.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.498+0000 7ff5a9a7b640 1 Processor -- start 2024-04-26T21:46:50.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.498+0000 7ff5a9a7b640 1 -- start start 2024-04-26T21:46:50.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a9a7b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a415d1e0 0x7ff5a40d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a9a7b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a40ce840 con 0x7ff5a40d1a90 2024-04-26T21:46:50.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a9a7b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a40ceb00 con 0x7ff5a40d0f20 2024-04-26T21:46:50.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a9a7b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a40cede0 con 0x7ff5a415d1e0 2024-04-26T21:46:50.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a415d1e0 0x7ff5a40d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff5a40d1a90 0x7ff5a4070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57844/0 (socket says 172.21.15.40:57844) 2024-04-26T21:46:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a3fff640 1 -- 172.21.15.40:0/4005614500 learned_addr learned my addr 172.21.15.40:0/4005614500 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a37fe640 1 -- 172.21.15.40:0/4005614500 >> v1:172.21.15.184:6789/0 conn(0x7ff5a40d0f20 legacy=0x7ff5a40d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a37fe640 1 -- 172.21.15.40:0/4005614500 >> v1:172.21.15.40:6789/0 conn(0x7ff5a40d1a90 legacy=0x7ff5a4070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a27fc640 1 -- 172.21.15.40:0/4005614500 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673301311 0 0) 0x7ff5a40ce840 con 0x7ff5a40d1a90 2024-04-26T21:46:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.499+0000 7ff5a37fe640 1 -- 172.21.15.40:0/4005614500 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a40702e0 con 0x7ff5a415d1e0 2024-04-26T21:46:50.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.500+0000 7ff5a37fe640 1 --2- 172.21.15.40:0/4005614500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40d09e0 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7ff5940097a0 tx=0x7ff59402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55634b50fe3f9699 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.500+0000 7ff5a27fc640 1 -- 172.21.15.40:0/4005614500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594037070 con 0x7ff5a415d1e0 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.500+0000 7ff5a27fc640 1 -- 172.21.15.40:0/4005614500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff594032440 con 0x7ff5a415d1e0 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.500+0000 7ff5a27fc640 1 -- 172.21.15.40:0/4005614500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594032810 con 0x7ff5a415d1e0 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.501+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/4005614500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 msgr2=0x7ff5a40d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.501+0000 7ff5a9a7b640 1 --2- 172.21.15.40:0/4005614500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40d09e0 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7ff5940097a0 tx=0x7ff59402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.501+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/4005614500 shutdown_connections 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.501+0000 7ff5a9a7b640 1 --2- 172.21.15.40:0/4005614500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40d09e0 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.501+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/4005614500 >> 172.21.15.40:0/4005614500 conn(0x7ff5a40d2da0 msgr2=0x7ff5a40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.501+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/4005614500 shutdown_connections 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.501+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/4005614500 wait complete. 2024-04-26T21:46:50.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.502+0000 7ff5a9a7b640 1 Processor -- start 2024-04-26T21:46:50.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.502+0000 7ff5a9a7b640 1 -- start start 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a9a7b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d0f20 0x7ff5a40b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a9a7b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d1a90 0x7ff5a415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a9a7b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a9a7b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40711f0 con 0x7ff5a415d1e0 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a9a7b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4070650 con 0x7ff5a40d0f20 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a9a7b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a4071370 con 0x7ff5a40d1a90 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59110/0 (socket says 172.21.15.40:59110) 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a3fff640 1 -- 172.21.15.40:0/669196595 learned_addr learned my addr 172.21.15.40:0/669196595 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a2ffd640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d1a90 0x7ff5a415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a37fe640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d0f20 0x7ff5a40b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a3fff640 1 -- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d1a90 msgr2=0x7ff5a415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a3fff640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d1a90 0x7ff5a415bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a3fff640 1 -- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d0f20 msgr2=0x7ff5a40b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a3fff640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d0f20 0x7ff5a40b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.503+0000 7ff5a3fff640 1 -- 172.21.15.40:0/669196595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a409dc00 con 0x7ff5a415d1e0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.504+0000 7ff5a3fff640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40b7910 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00caf0 tx=0x7ff59c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.504+0000 7ff58bfff640 1 -- 172.21.15.40:0/669196595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00ac40 con 0x7ff5a415d1e0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.504+0000 7ff58bfff640 1 -- 172.21.15.40:0/669196595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59c00ae00 con 0x7ff5a415d1e0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.504+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a40b6230 con 0x7ff5a415d1e0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.504+0000 7ff58bfff640 1 -- 172.21.15.40:0/669196595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c012640 con 0x7ff5a415d1e0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.504+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a40b67f0 con 0x7ff5a415d1e0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.505+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a40694f0 con 0x7ff5a415d1e0 2024-04-26T21:46:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.505+0000 7ff58bfff640 1 -- 172.21.15.40:0/669196595 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff59c0127e0 con 0x7ff5a415d1e0 2024-04-26T21:46:50.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.506+0000 7ff58bfff640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:50.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.506+0000 7ff58bfff640 1 -- 172.21.15.40:0/669196595 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9485+0+0 (secure 0 0 0) 0x7ff59c08d0c0 con 0x7ff5a415d1e0 2024-04-26T21:46:50.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.506+0000 7ff5a37fe640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:50.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.506+0000 7ff5a37fe640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff59006c8f0 0x7ff59006edb0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff594034000 tx=0x7ff59402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:50.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.510+0000 7ff58bfff640 1 -- 172.21.15.40:0/669196595 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff59c05a5e0 con 0x7ff5a415d1e0 2024-04-26T21:46:50.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:50.734+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff5a409ddd0 con 0x7ff5a415d1e0 2024-04-26T21:46:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.682+0000 7ff58bfff640 1 -- 172.21.15.40:0/669196595 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v336) ==== 161+0+0 (secure 0 0 0) 0x7ff59c059ee0 con 0x7ff5a415d1e0 2024-04-26T21:46:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:46:51.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff59006c8f0 msgr2=0x7ff59006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff59006c8f0 0x7ff59006edb0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff594034000 tx=0x7ff59402e000 comp rx=0 tx=0).stop 2024-04-26T21:46:51.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 msgr2=0x7ff5a40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40b7910 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00caf0 tx=0x7ff59c00a060 comp rx=0 tx=0).stop 2024-04-26T21:46:51.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 shutdown_connections 2024-04-26T21:46:51.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a415d1e0 0x7ff5a40b7910 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d1a90 0x7ff5a415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 --2- 172.21.15.40:0/669196595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d0f20 0x7ff5a40b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 >> 172.21.15.40:0/669196595 conn(0x7ff5a40d2da0 msgr2=0x7ff5a40b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 shutdown_connections 2024-04-26T21:46:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.685+0000 7ff5a9a7b640 1 -- 172.21.15.40:0/669196595 wait complete. 2024-04-26T21:46:51.700 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:46:51.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.807+0000 7fe80919e640 1 Processor -- start 2024-04-26T21:46:51.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.807+0000 7fe80919e640 1 -- start start 2024-04-26T21:46:51.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.808+0000 7fe80919e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe80415f3a0 0x7fe8040d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.808+0000 7fe80919e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8040d1770 0x7fe8040d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.808+0000 7fe80919e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8040d2590 con 0x7fe8040d2160 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.808+0000 7fe80919e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe8040d0e20 con 0x7fe8040d1770 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.808+0000 7fe80919e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe8040b1d70 con 0x7fe80415f3a0 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.808+0000 7fe80259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8040d1770 0x7fe8040d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.808+0000 7fe80259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8040d1770 0x7fe8040d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59114/0 (socket says 172.21.15.40:59114) 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.808+0000 7fe80259c640 1 -- 172.21.15.40:0/2803623047 learned_addr learned my addr 172.21.15.40:0/2803623047 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe802d9d640 1 --2- 172.21.15.40:0/2803623047 >> v2:172.21.15.184:3300/0 conn(0x7fe80415f3a0 0x7fe8040d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe80259c640 1 -- 172.21.15.40:0/2803623047 >> v2:172.21.15.184:3300/0 conn(0x7fe80415f3a0 msgr2=0x7fe8040d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe80259c640 1 --2- 172.21.15.40:0/2803623047 >> v2:172.21.15.184:3300/0 conn(0x7fe80415f3a0 0x7fe8040d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe80259c640 1 -- 172.21.15.40:0/2803623047 >> v1:172.21.15.184:6789/0 conn(0x7fe8040d2160 legacy=0x7fe8040d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe80259c640 1 -- 172.21.15.40:0/2803623047 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8040b2050 con 0x7fe8040d1770 2024-04-26T21:46:51.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe802d9d640 1 --2- 172.21.15.40:0/2803623047 >> v2:172.21.15.184:3300/0 conn(0x7fe80415f3a0 0x7fe8040d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe80259c640 1 --2- 172.21.15.40:0/2803623047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe8040d1c20 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fe804073b50 tx=0x7fe7f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0828f04b4f05d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe801d9b640 1 -- 172.21.15.40:0/2803623047 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8037070 con 0x7fe8040d1770 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.809+0000 7fe801d9b640 1 -- 172.21.15.40:0/2803623047 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f8032440 con 0x7fe8040d1770 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe801d9b640 1 -- 172.21.15.40:0/2803623047 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8032810 con 0x7fe8040d1770 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe80919e640 1 -- 172.21.15.40:0/2803623047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 msgr2=0x7fe8040d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe80919e640 1 --2- 172.21.15.40:0/2803623047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe8040d1c20 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fe804073b50 tx=0x7fe7f802a230 comp rx=0 tx=0).stop 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe80919e640 1 -- 172.21.15.40:0/2803623047 shutdown_connections 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe80919e640 1 --2- 172.21.15.40:0/2803623047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe8040d1c20 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe80919e640 1 --2- 172.21.15.40:0/2803623047 >> v2:172.21.15.184:3300/0 conn(0x7fe80415f3a0 0x7fe8040d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe80919e640 1 -- 172.21.15.40:0/2803623047 >> 172.21.15.40:0/2803623047 conn(0x7fe8040d4f90 msgr2=0x7fe8040d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe80919e640 1 -- 172.21.15.40:0/2803623047 shutdown_connections 2024-04-26T21:46:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.810+0000 7fe80919e640 1 -- 172.21.15.40:0/2803623047 wait complete. 2024-04-26T21:46:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe80919e640 1 Processor -- start 2024-04-26T21:46:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe80919e640 1 -- start start 2024-04-26T21:46:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe80919e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe80415e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe80919e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d2160 0x7fe8041f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe80919e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe80415f3a0 0x7fe8040cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe80919e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe80415f900 con 0x7fe8040d1770 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe80919e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe80415f780 con 0x7fe8040d2160 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe80919e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8040b3860 con 0x7fe80415f3a0 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe802d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe80415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe802d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe80415e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59128/0 (socket says 172.21.15.40:59128) 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.812+0000 7fe802d9d640 1 -- 172.21.15.40:0/971990900 learned_addr learned my addr 172.21.15.40:0/971990900 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe80259c640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d2160 0x7fe8041f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe80359e640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe80415f3a0 0x7fe8040cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe802d9d640 1 -- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe80415f3a0 msgr2=0x7fe8040cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe802d9d640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe80415f3a0 0x7fe8040cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe802d9d640 1 -- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d2160 msgr2=0x7fe8041f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe802d9d640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d2160 0x7fe8041f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe802d9d640 1 -- 172.21.15.40:0/971990900 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80415e5b0 con 0x7fe8040d1770 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe802d9d640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe80415e070 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400c7f0 tx=0x7fe7f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe7eb7fe640 1 -- 172.21.15.40:0/971990900 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4015070 con 0x7fe8040d1770 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe7eb7fe640 1 -- 172.21.15.40:0/971990900 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f4007820 con 0x7fe8040d1770 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8040cd5f0 con 0x7fe8040d1770 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe7eb7fe640 1 -- 172.21.15.40:0/971990900 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4007bd0 con 0x7fe8040d1770 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.813+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8040cde90 con 0x7fe8040d1770 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.814+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8040694f0 con 0x7fe8040d1770 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.815+0000 7fe7eb7fe640 1 -- 172.21.15.40:0/971990900 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7f4018070 con 0x7fe8040d1770 2024-04-26T21:46:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.815+0000 7fe7eb7fe640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe7f006c8f0 0x7fe7f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:51.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.815+0000 7fe7eb7fe640 1 -- 172.21.15.40:0/971990900 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9485+0+0 (secure 0 0 0) 0x7fe7f408cac0 con 0x7fe8040d1770 2024-04-26T21:46:51.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.815+0000 7fe80259c640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe7f006c8f0 0x7fe7f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.816+0000 7fe80259c640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe7f006c8f0 0x7fe7f006edb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8034000 tx=0x7fe7f802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:51.820+0000 7fe7eb7fe640 1 -- 172.21.15.40:0/971990900 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7f4059990 con 0x7fe8040d1770 2024-04-26T21:46:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.081+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe8040b6860 con 0x7fe8040d1770 2024-04-26T21:46:52.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.899+0000 7fe7eb7fe640 1 -- 172.21.15.40:0/971990900 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v337) ==== 101+0+0 (secure 0 0 0) 0x7fe7f405ee20 con 0x7fe8040d1770 2024-04-26T21:46:52.900 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:46:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe7f006c8f0 msgr2=0x7fe7f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe7f006c8f0 0x7fe7f006edb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8034000 tx=0x7fe7f802e000 comp rx=0 tx=0).stop 2024-04-26T21:46:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 msgr2=0x7fe80415e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe80415e070 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400c7f0 tx=0x7fe7f400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 shutdown_connections 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe7f006c8f0 0x7fe7f006edb0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe80415f3a0 0x7fe8040cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d2160 0x7fe8041f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 --2- 172.21.15.40:0/971990900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8040d1770 0x7fe80415e070 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.902+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 >> 172.21.15.40:0/971990900 conn(0x7fe8040d4f90 msgr2=0x7fe8040d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.903+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 shutdown_connections 2024-04-26T21:46:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.903+0000 7fe80919e640 1 -- 172.21.15.40:0/971990900 wait complete. 2024-04-26T21:46:52.917 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:46:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.962+0000 7faf41bd4f40 1 Processor -- start 2024-04-26T21:46:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.962+0000 7faf41bd4f40 1 -- start start 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf41bd4f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 0x560261bfd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf41bd4f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261bfde80 0x560261b2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf41bd4f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261b2f4c0 0x560261c03ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf41bd4f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560261ad8e40 con 0x560261b2f4c0 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf41bd4f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560261adce80 con 0x560261bfde80 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf41bd4f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560261b7c860 con 0x7faf28003970 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf3fecf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261b2f4c0 0x560261c03ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf3fecf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261b2f4c0 0x560261c03ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59144/0 (socket says 172.21.15.40:59144) 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf3fecf640 1 -- 172.21.15.40:0/1688398397 learned_addr learned my addr 172.21.15.40:0/1688398397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 0x560261bfd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.963+0000 7faf3eecd640 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261bfde80 0x560261b2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3fecf640 1 -- 172.21.15.40:0/1688398397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 msgr2=0x560261bfd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3fecf640 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 0x560261bfd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3fecf640 1 -- 172.21.15.40:0/1688398397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261bfde80 msgr2=0x560261b2ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3fecf640 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261bfde80 0x560261b2ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3fecf640 1 -- 172.21.15.40:0/1688398397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560261c04aa0 con 0x560261b2f4c0 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3fecf640 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261b2f4c0 0x560261c03ba0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7faf38006750 tx=0x7faf38008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882549aeef7e30c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3decb640 1 -- 172.21.15.40:0/1688398397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38055070 con 0x560261b2f4c0 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3decb640 1 -- 172.21.15.40:0/1688398397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf38006230 con 0x560261b2f4c0 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.964+0000 7faf3decb640 1 -- 172.21.15.40:0/1688398397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38051a50 con 0x560261b2f4c0 2024-04-26T21:46:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 -- 172.21.15.40:0/1688398397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261b2f4c0 msgr2=0x560261c03ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261b2f4c0 0x560261c03ba0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7faf38006750 tx=0x7faf38008030 comp rx=0 tx=0).stop 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 -- 172.21.15.40:0/1688398397 shutdown_connections 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261b2f4c0 0x560261c03ba0 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261bfde80 0x560261b2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/1688398397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 0x560261bfd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 -- 172.21.15.40:0/1688398397 >> 172.21.15.40:0/1688398397 conn(0x560261b5c9b0 msgr2=0x560261b61b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 -- 172.21.15.40:0/1688398397 shutdown_connections 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 -- 172.21.15.40:0/1688398397 wait complete. 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 Processor -- start 2024-04-26T21:46:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.965+0000 7faf41bd4f40 1 -- start start 2024-04-26T21:46:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf41bd4f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261b2f4c0 0x560261b796a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf41bd4f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261bfde80 0x560261bf0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf41bd4f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 0x560261c02d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf41bd4f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560261b79260 con 0x560261bfde80 2024-04-26T21:46:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf41bd4f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560261b34cc0 con 0x560261b2f4c0 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf41bd4f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560261c04d40 con 0x7faf28003970 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3eecd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261bfde80 0x560261bf0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3eecd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261bfde80 0x560261bf0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59150/0 (socket says 172.21.15.40:59150) 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 learned_addr learned my addr 172.21.15.40:0/3059483003 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3fecf640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 0x560261c02d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261b2f4c0 0x560261b796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 msgr2=0x560261c02d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3eecd640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf28003970 0x560261c02d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261b2f4c0 msgr2=0x560261b796a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3eecd640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261b2f4c0 0x560261b796a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560261bfa9f0 con 0x560261bfde80 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.966+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560261b2f4c0 0x560261b796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.967+0000 7faf3eecd640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261bfde80 0x560261bf0170 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7faf34008700 tx=0x7faf340087f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.967+0000 7faf3cec9640 1 -- 172.21.15.40:0/3059483003 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34015910 con 0x560261bfde80 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.967+0000 7faf3cec9640 1 -- 172.21.15.40:0/3059483003 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf34015ab0 con 0x560261bfde80 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.967+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560261c02250 con 0x560261bfde80 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.967+0000 7faf3cec9640 1 -- 172.21.15.40:0/3059483003 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34015e60 con 0x560261bfde80 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.967+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560261bf06b0 con 0x560261bfde80 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.968+0000 7faf3cec9640 1 -- 172.21.15.40:0/3059483003 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf34016420 con 0x560261bfde80 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.969+0000 7faf3cec9640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2c06b2d0 0x7faf2c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.969+0000 7faf3cec9640 1 -- 172.21.15.40:0/3059483003 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9485+0+0 (secure 0 0 0) 0x7faf3408c710 con 0x560261bfde80 2024-04-26T21:46:52.972 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:46:52.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.969+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2c06b2d0 0x7faf2c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.969+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560261bfa4f0 0x560261bfe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.969+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261bfebb0 con 0x560261bfa4f0 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.969+0000 7faf3fecf640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560261bfa4f0 0x560261bfe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.970+0000 7faf3fecf640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560261bfa4f0 0x560261bfe670 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.970+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2c06b2d0 0x7faf2c06d790 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7faf30000c00 tx=0x7faf30001cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.971+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v133'1159 uv1133 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.972+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261bfefc0 con 0x560261bfa4f0 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.973+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v133'1160 uv1160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.973+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261ca9a80 con 0x560261bfa4f0 2024-04-26T21:46:52.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.974+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v133'1161 uv1161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.975 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:46:52.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.974+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caae30 con 0x560261bfa4f0 2024-04-26T21:46:52.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.975+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v133'1162 uv1162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.975+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cab1d0 con 0x560261bfa4f0 2024-04-26T21:46:52.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.976+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v133'1163 uv1163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.976+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261ca9e20 con 0x560261bfa4f0 2024-04-26T21:46:52.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.977+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v133'1164 uv1164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.977+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa1c0 con 0x560261bfa4f0 2024-04-26T21:46:52.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.979+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v133'1165 uv1139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.979+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.980+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v133'1166 uv1166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.980+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.981 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:46:52.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.981+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v133'1167 uv1167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.981+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.982+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v133'1168 uv1168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.982+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.983+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v133'1169 uv1169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.983+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.984+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v133'1170 uv1170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.984+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.985+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v133'1171 uv1146 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.985+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.986+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v133'1172 uv1172 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.986+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.988+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v133'1173 uv1173 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.989 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:46:52.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.988+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.989+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v133'1174 uv1174 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.989+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.990+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v133'1175 uv1175 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.990+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.991+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v133'1176 uv1176 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.991+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.993+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v133'1177 uv1177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.993+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.994+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v133'1178 uv1152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.994+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.995+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v133'1179 uv1179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.995+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.996 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:46:52.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.996+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v133'1180 uv1180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.996+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.997+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v133'1181 uv1181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.997+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:52.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.998+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v133'1182 uv1182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:52.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:52.998+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:53.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.000+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v133'1183 uv1183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:53.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.000+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:53.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.001+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v133'1184 uv1158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.001+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.002+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v133'1185 uv1185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.002+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.003+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v133'1186 uv1186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:53.004 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:46:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.003+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.004+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v133'1187 uv1187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.004+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.005+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v133'1188 uv1188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.005+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa560 con 0x560261bfa4f0 2024-04-26T21:46:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.006+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v133'1189 uv1189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261bfa4f0 2024-04-26T21:46:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.006+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560261caa560 0x560261cada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.006+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caa960 con 0x560261caa560 2024-04-26T21:46:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.007+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560261caa560 0x560261cada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.007+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560261caa560 0x560261cada90 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.009+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v133'846 uv828 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.009+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cadfd0 con 0x560261caa560 2024-04-26T21:46:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.010+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v133'847 uv847 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.010+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cae370 con 0x560261caa560 2024-04-26T21:46:53.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.012+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v133'848 uv848 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.013 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:46:53.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.012+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caebc0 con 0x560261caa560 2024-04-26T21:46:53.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.014+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v133'849 uv849 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.014+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caef60 con 0x560261caa560 2024-04-26T21:46:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.015+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v133'850 uv850 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.015+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261caf300 con 0x560261caa560 2024-04-26T21:46:53.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.017+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v133'851 uv833 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.017+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261caf6a0 con 0x560261caa560 2024-04-26T21:46:53.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.018+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v133'852 uv852 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.018+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.019+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v133'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.020 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:46:53.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.019+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.021+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v133'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.021+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.022+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v133'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.022+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.024+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v133'856 uv838 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.024+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.025+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v133'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.025+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.027 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:46:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.027+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v133'858 uv858 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.027+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb4100 con 0x560261caa560 2024-04-26T21:46:53.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.028+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v133'859 uv859 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.028+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb4100 con 0x560261caa560 2024-04-26T21:46:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.030+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v133'860 uv860 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.030+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.031+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v133'861 uv844 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.031+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.033+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v133'862 uv862 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.033+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.034+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v133'863 uv863 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.035 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:46:53.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.034+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb0a50 con 0x560261caa560 2024-04-26T21:46:53.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.035+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v133'864 uv864 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.035+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb0a50 con 0x560261caa560 2024-04-26T21:46:53.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.036+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v133'865 uv865 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.037+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cafa40 con 0x560261caa560 2024-04-26T21:46:53.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.038+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v133'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261caa560 2024-04-26T21:46:53.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.038+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560261cb0340 0x560261cb2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.038+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb2c70 con 0x560261cb0340 2024-04-26T21:46:53.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.038+0000 7faf3eecd640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560261cb0340 0x560261cb2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.039+0000 7faf3eecd640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560261cb0340 0x560261cb2730 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.040+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v133'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.040+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cb3010 con 0x560261cb0340 2024-04-26T21:46:53.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.041+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v133'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.041+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb3480 con 0x560261cb0340 2024-04-26T21:46:53.042 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:46:53.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.042+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v133'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.042+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb4d30 con 0x560261cb0340 2024-04-26T21:46:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.043+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v133'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.043+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb50d0 con 0x560261cb0340 2024-04-26T21:46:53.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.044+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v133'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.044+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb3d20 con 0x560261cb0340 2024-04-26T21:46:53.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.045+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v133'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.045+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560261cb4620 0x560261cb7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.045+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb79c0 con 0x560261cb4620 2024-04-26T21:46:53.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.045+0000 7faf3fecf640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560261cb4620 0x560261cb7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.046+0000 7faf3fecf640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560261cb4620 0x560261cb7480 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.047+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v133'177 uv176 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.047+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cb79c0 con 0x560261cb4620 2024-04-26T21:46:53.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.048+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v133'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.048+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb7a40 con 0x560261cb4620 2024-04-26T21:46:53.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.049+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v133'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.050 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:46:53.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.049+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb7e30 con 0x560261cb4620 2024-04-26T21:46:53.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.051+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v133'180 uv180 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.051+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560261cb8340 0x560261cba730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.051+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbacb0 con 0x560261cb8340 2024-04-26T21:46:53.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.051+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560261cb8340 0x560261cba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.052+0000 7faf3f6ce640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560261cb8340 0x560261cba730 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.053+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v133'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.053+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbad30 con 0x560261cb8340 2024-04-26T21:46:53.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.054+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v133'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.054+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.056 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:46:53.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.055+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v133'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.055+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbc160 con 0x560261cb8340 2024-04-26T21:46:53.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.057+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v133'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.057+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbc160 con 0x560261cb8340 2024-04-26T21:46:53.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.058+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v133'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.058+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.059+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v133'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.059+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.060+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v133'415 uv409 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.060+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.068+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v133'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.068+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.071 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:46:53.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.070+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v133'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.070+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbc160 con 0x560261cb4620 2024-04-26T21:46:53.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.071+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v133'418 uv418 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.071+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbc160 con 0x560261cb4620 2024-04-26T21:46:53.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.073+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v133'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.073+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.074+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v133'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.074+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.075+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v133'421 uv414 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.075+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.076+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v133'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.076+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.078 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:46:53.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.077+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v133'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.077+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.078+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v133'424 uv424 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.078+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.080+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v133'425 uv425 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.080+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.081+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v133'188 uv187 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.081+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.082+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v133'189 uv189 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.082+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.084 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:46:53.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.083+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v133'190 uv190 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.083+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.084+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v133'191 uv191 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.084+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb4620 2024-04-26T21:46:53.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.085+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v133'192 uv192 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7faf38051810 con 0x560261cb4620 2024-04-26T21:46:53.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.085+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.087+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v133'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.087+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.088+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v133'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.088+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.089+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v133'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.089 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:46:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.089+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbc160 con 0x560261cb8340 2024-04-26T21:46:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.090+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v133'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.090+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbc160 con 0x560261cb8340 2024-04-26T21:46:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.090+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v133'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.090+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.092+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v133'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.092+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.093+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v133'376 uv369 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.093+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.094+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v133'377 uv377 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.094+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb150 con 0x560261cb8340 2024-04-26T21:46:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.095+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v133'378 uv378 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.096 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:46:53.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.095+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb3950 con 0x560261cb8340 2024-04-26T21:46:53.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.096+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v133'379 uv379 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.096+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb3950 con 0x560261cb8340 2024-04-26T21:46:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.097+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v133'380 uv380 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.097+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb3950 con 0x560261cb8340 2024-04-26T21:46:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.098+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v133'381 uv375 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.098+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cb3950 con 0x560261cb8340 2024-04-26T21:46:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.099+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v133'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.099+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb3930 con 0x560261cb8340 2024-04-26T21:46:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.100+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v133'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.101 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:46:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.100+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cb3930 con 0x560261cb8340 2024-04-26T21:46:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.101+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v133'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.101+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbdff0 con 0x560261cb8340 2024-04-26T21:46:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.102+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v133'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.102+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560261cbbcd0 0x560261cbe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.102+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbe6b0 con 0x560261cbbcd0 2024-04-26T21:46:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.103+0000 7faf3eecd640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560261cbbcd0 0x560261cbe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.103+0000 7faf3eecd640 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560261cbbcd0 0x560261cbe0c0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.104+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v133'167 uv165 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cbbcd0 2024-04-26T21:46:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.104+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbea50 con 0x560261cbbcd0 2024-04-26T21:46:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.105+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v133'168 uv168 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cbbcd0 2024-04-26T21:46:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.105+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbea50 con 0x560261cbbcd0 2024-04-26T21:46:53.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.106+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v133'169 uv169 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cbbcd0 2024-04-26T21:46:53.107 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:46:53.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.106+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cbbcd0 2024-04-26T21:46:53.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.107+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v133'170 uv170 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cbbcd0 2024-04-26T21:46:53.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.107+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb8340 2024-04-26T21:46:53.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.108+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v133'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.108+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbb290 con 0x560261cb8340 2024-04-26T21:46:53.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.109+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v133'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.109+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbb290 con 0x560261cb8340 2024-04-26T21:46:53.111 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:46:53.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.110+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v133'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.111+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbff10 con 0x560261cb8340 2024-04-26T21:46:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.112+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v133'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.112+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbff10 con 0x560261cb8340 2024-04-26T21:46:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.113+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v133'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.113+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb8340 2024-04-26T21:46:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.114+0000 7faf3f6ce640 1 -- 172.21.15.40:0/3059483003 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v133'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf30018070 con 0x560261cb8340 2024-04-26T21:46:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.114+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.115+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v133'191 uv190 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.115+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.116+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v133'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.116+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.117+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v133'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.118 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:46:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.117+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.118+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v133'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.118+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.119+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v133'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.119+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.121+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v133'468 uv459 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.121+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.122+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v133'469 uv469 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.122+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.123+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v133'470 uv470 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.124 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:46:53.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.123+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.124+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v133'471 uv471 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.124+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.126+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v133'472 uv472 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.126+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.127+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v133'473 uv466 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.127+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.128+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v133'474 uv474 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.128+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.130 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:46:53.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.129+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v133'475 uv475 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.129+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbff10 con 0x560261cb0340 2024-04-26T21:46:53.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.130+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v133'476 uv476 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.130+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbff10 con 0x560261cb0340 2024-04-26T21:46:53.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.132+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v133'477 uv477 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.132+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbef00 con 0x560261cb0340 2024-04-26T21:46:53.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.133+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v133'478 uv478 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.133+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x560261cbf1b0 con 0x560261cb0340 2024-04-26T21:46:53.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf3eecd640 1 -- 172.21.15.40:0/3059483003 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v133'479 uv479 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7faf34055c30 con 0x560261cb0340 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560261caa560 msgr2=0x560261cada90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x560261caa560 0x560261cada90 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560261bfa4f0 msgr2=0x560261bfe670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x560261bfa4f0 0x560261bfe670 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560261cb4620 msgr2=0x560261cb7480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x560261cb4620 0x560261cb7480 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560261cbbcd0 msgr2=0x560261cbe0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x560261cbbcd0 0x560261cbe0c0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560261cb0340 msgr2=0x560261cb2730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x560261cb0340 0x560261cb2730 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560261cb8340 msgr2=0x560261cba730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x560261cb8340 0x560261cba730 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 reap_dead start 2024-04-26T21:46:53.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf3fecf640 1 -- 172.21.15.40:0/3059483003 reap_dead start 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2c06b2d0 msgr2=0x7faf2c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2c06b2d0 0x7faf2c06d790 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7faf30000c00 tx=0x7faf30001cb0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261bfde80 msgr2=0x560261bf0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261bfde80 0x560261bf0170 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7faf34008700 tx=0x7faf340087f0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 shutdown_connections 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faf2c06b2d0 0x7faf2c06d790 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 --2- 172.21.15.40:0/3059483003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560261bfde80 0x560261bf0170 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 >> 172.21.15.40:0/3059483003 conn(0x560261b5c9b0 msgr2=0x560261c07d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 shutdown_connections 2024-04-26T21:46:53.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.134+0000 7faf41bd4f40 1 -- 172.21.15.40:0/3059483003 wait complete. 2024-04-26T21:46:53.138 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_sessionmap deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:46:53.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.221+0000 7efe85b01f40 1 Processor -- start 2024-04-26T21:46:53.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.221+0000 7efe85b01f40 1 -- start start 2024-04-26T21:46:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe85b01f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe85b01f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 0x55ec60c2a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe85b01f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 0x55ec60b9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe85b01f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ec60afce40 con 0x55ec60b9e8c0 2024-04-26T21:46:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe85b01f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ec60b00e80 con 0x55ec60c10e60 2024-04-26T21:46:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe85b01f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ec60b9d570 con 0x55ec60c10a80 2024-04-26T21:46:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe835fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe835fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c249b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59174/0 (socket says 172.21.15.40:59174) 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe835fb640 1 -- 172.21.15.40:0/1542458880 learned_addr learned my addr 172.21.15.40:0/1542458880 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe83dfc640 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 0x55ec60b9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.222+0000 7efe82dfa640 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 0x55ec60c2a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe835fb640 1 -- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 msgr2=0x55ec60c2a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe835fb640 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 0x55ec60c2a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe835fb640 1 -- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 msgr2=0x55ec60b9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe835fb640 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 0x55ec60b9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe835fb640 1 -- 172.21.15.40:0/1542458880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec60c2ec20 con 0x55ec60b9e8c0 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe83dfc640 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 0x55ec60b9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe835fb640 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c249b0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00b310 tx=0x7efe7c00b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f798af5e33e3dced server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe81df8640 1 -- 172.21.15.40:0/1542458880 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c008070 con 0x55ec60b9e8c0 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe81df8640 1 -- 172.21.15.40:0/1542458880 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c00d500 con 0x55ec60b9e8c0 2024-04-26T21:46:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe81df8640 1 -- 172.21.15.40:0/1542458880 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00d8b0 con 0x55ec60b9e8c0 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 -- 172.21.15.40:0/1542458880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 msgr2=0x55ec60c249b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c249b0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00b310 tx=0x7efe7c00b7d0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 -- 172.21.15.40:0/1542458880 shutdown_connections 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 0x55ec60b9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 0x55ec60c2a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 --2- 172.21.15.40:0/1542458880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c249b0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 -- 172.21.15.40:0/1542458880 >> 172.21.15.40:0/1542458880 conn(0x55ec60b53ab0 msgr2=0x55ec60b805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 -- 172.21.15.40:0/1542458880 shutdown_connections 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.223+0000 7efe85b01f40 1 -- 172.21.15.40:0/1542458880 wait complete. 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.224+0000 7efe85b01f40 1 Processor -- start 2024-04-26T21:46:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.224+0000 7efe85b01f40 1 -- start start 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe85b01f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c28450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe85b01f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 0x55ec60cc3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe85b01f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 0x55ec60c28b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe85b01f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ec60c2bb90 con 0x55ec60b9e8c0 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe85b01f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ec60c2c030 con 0x55ec60c10e60 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe85b01f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ec60c29b70 con 0x55ec60c10a80 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c28450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c28450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59178/0 (socket says 172.21.15.40:59178) 2024-04-26T21:46:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 -- 172.21.15.40:0/3186392105 learned_addr learned my addr 172.21.15.40:0/3186392105 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe82dfa640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 0x55ec60cc3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe83dfc640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 0x55ec60c28b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 -- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 msgr2=0x55ec60cc3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ec60c10a80 0x55ec60cc3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 -- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 msgr2=0x55ec60c28b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec60c10e60 0x55ec60c28b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 -- 172.21.15.40:0/3186392105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec60c271f0 con 0x55ec60b9e8c0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe835fb640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c28450 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00b810 tx=0x7efe7c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.225+0000 7efe80df6640 1 -- 172.21.15.40:0/3186392105 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c021030 con 0x55ec60b9e8c0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.226+0000 7efe80df6640 1 -- 172.21.15.40:0/3186392105 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c01c710 con 0x55ec60b9e8c0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.226+0000 7efe80df6640 1 -- 172.21.15.40:0/3186392105 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c01c8e0 con 0x55ec60b9e8c0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.226+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec60c274d0 con 0x55ec60b9e8c0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.226+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec60b802b0 con 0x55ec60b9e8c0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.227+0000 7efe80df6640 1 -- 172.21.15.40:0/3186392105 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe7c024070 con 0x55ec60b9e8c0 2024-04-26T21:46:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.227+0000 7efe80df6640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe6406b300 0x7efe6406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.227+0000 7efe80df6640 1 -- 172.21.15.40:0/3186392105 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9485+0+0 (secure 0 0 0) 0x7efe7c08c9c0 con 0x55ec60b9e8c0 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.228+0000 7efe82dfa640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe6406b300 0x7efe6406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.228+0000 7efe85b01f40 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ec60c0f7b0 0x55ec60c0fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.228+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x55ec60b80a20 con 0x55ec60c0f7b0 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.228+0000 7efe83dfc640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ec60c0f7b0 0x55ec60c0fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.228+0000 7efe82dfa640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe6406b300 0x7efe6406d7c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x55ec60c281d0 tx=0x7efe6c00e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.228+0000 7efe83dfc640 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ec60c0f7b0 0x55ec60c0fb90 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.229+0000 7efe83dfc640 1 -- 172.21.15.40:0/3186392105 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(1 mds0_sessionmap [omap-set-header] v133'171 uv171 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efe74050960 con 0x55ec60c0f7b0 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.229+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ec60c0f7b0 msgr2=0x55ec60c0fb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.229+0000 7efe85b01f40 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55ec60c0f7b0 0x55ec60c0fb90 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe6406b300 msgr2=0x7efe6406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe85b01f40 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe6406b300 0x7efe6406d7c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x55ec60c281d0 tx=0x7efe6c00e040 comp rx=0 tx=0).stop 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 msgr2=0x55ec60c28450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe85b01f40 1 --2- 172.21.15.40:0/3186392105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec60b9e8c0 0x55ec60c28450 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00b810 tx=0x7efe7c000ed0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe83dfc640 1 -- 172.21.15.40:0/3186392105 reap_dead start 2024-04-26T21:46:53.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 shutdown_connections 2024-04-26T21:46:53.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 >> 172.21.15.40:0/3186392105 conn(0x55ec60b53ab0 msgr2=0x55ec60c2c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:53.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 shutdown_connections 2024-04-26T21:46:53.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.230+0000 7efe85b01f40 1 -- 172.21.15.40:0/3186392105 wait complete. 2024-04-26T21:46:53.235 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:46:53.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.379+0000 7f2432ad9640 1 Processor -- start 2024-04-26T21:46:53.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.379+0000 7f2432ad9640 1 -- start start 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2432ad9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f242c15f170 0x7f242c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2432ad9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0d3bc0 0x7f242c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2432ad9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242c0d1040 con 0x7f242c15f550 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2432ad9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f242c0c9430 con 0x7f242c15f170 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2432ad9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f242c0c9680 con 0x7f242c0d3bc0 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2431076640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0d3bc0 0x7f242c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2430875640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f242c15f170 0x7f242c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2431076640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0d3bc0 0x7f242c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:42680/0 (socket says 172.21.15.40:42680) 2024-04-26T21:46:53.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2431076640 1 -- 172.21.15.40:0/1766345366 learned_addr learned my addr 172.21.15.40:0/1766345366 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:53.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2430875640 1 -- 172.21.15.40:0/1766345366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0d3bc0 msgr2=0x7f242c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:53.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2430875640 1 --2- 172.21.15.40:0/1766345366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0d3bc0 0x7f242c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2430875640 1 -- 172.21.15.40:0/1766345366 >> v1:172.21.15.184:6789/0 conn(0x7f242c15f550 legacy=0x7f242c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.380+0000 7f2430875640 1 -- 172.21.15.40:0/1766345366 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c0c9960 con 0x7f242c15f170 2024-04-26T21:46:53.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.381+0000 7f2430875640 1 --2- 172.21.15.40:0/1766345366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c15f170 0x7f242c0d5a80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f24200097d0 tx=0x7f242002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4519a9e41b94801 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.381+0000 7f242b7fe640 1 -- 172.21.15.40:0/1766345366 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420037070 con 0x7f242c15f170 2024-04-26T21:46:53.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.381+0000 7f242b7fe640 1 -- 172.21.15.40:0/1766345366 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2420032440 con 0x7f242c15f170 2024-04-26T21:46:53.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.381+0000 7f242b7fe640 1 -- 172.21.15.40:0/1766345366 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420032810 con 0x7f242c15f170 2024-04-26T21:46:53.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.382+0000 7f2432ad9640 1 -- 172.21.15.40:0/1766345366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c15f170 msgr2=0x7f242c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.382+0000 7f2432ad9640 1 --2- 172.21.15.40:0/1766345366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c15f170 0x7f242c0d5a80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f24200097d0 tx=0x7f242002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.382+0000 7f2432ad9640 1 -- 172.21.15.40:0/1766345366 shutdown_connections 2024-04-26T21:46:53.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.382+0000 7f2432ad9640 1 --2- 172.21.15.40:0/1766345366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0d3bc0 0x7f242c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.382+0000 7f2432ad9640 1 --2- 172.21.15.40:0/1766345366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c15f170 0x7f242c0d5a80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.382+0000 7f2432ad9640 1 -- 172.21.15.40:0/1766345366 >> 172.21.15.40:0/1766345366 conn(0x7f242c0d4f60 msgr2=0x7f242c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:53.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.382+0000 7f2432ad9640 1 -- 172.21.15.40:0/1766345366 shutdown_connections 2024-04-26T21:46:53.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.382+0000 7f2432ad9640 1 -- 172.21.15.40:0/1766345366 wait complete. 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.383+0000 7f2432ad9640 1 Processor -- start 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.383+0000 7f2432ad9640 1 -- start start 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2432ad9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c0d3bc0 0x7f242c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2432ad9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242c15f170 0x7f242c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2432ad9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0ce6b0 0x7f242c154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2432ad9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0ca5c0 con 0x7f242c15f170 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2432ad9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0ca440 con 0x7f242c0d3bc0 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2432ad9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f242c0ca740 con 0x7f242c0ce6b0 2024-04-26T21:46:53.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f242bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242c15f170 0x7f242c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f242bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242c15f170 0x7f242c0ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59182/0 (socket says 172.21.15.40:59182) 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f242bfff640 1 -- 172.21.15.40:0/155639620 learned_addr learned my addr 172.21.15.40:0/155639620 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2431076640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0ce6b0 0x7f242c154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2430875640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c0d3bc0 0x7f242c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f242bfff640 1 -- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0ce6b0 msgr2=0x7f242c154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f242bfff640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0ce6b0 0x7f242c154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f242bfff640 1 -- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c0d3bc0 msgr2=0x7f242c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f242bfff640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c0d3bc0 0x7f242c151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f242bfff640 1 -- 172.21.15.40:0/155639620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c09fc30 con 0x7f242c15f170 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.384+0000 7f2430875640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c0d3bc0 0x7f242c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.385+0000 7f242bfff640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242c15f170 0x7f242c0ce170 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f241800c3c0 tx=0x7f241800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.385+0000 7f24297fa640 1 -- 172.21.15.40:0/155639620 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180052d0 con 0x7f242c15f170 2024-04-26T21:46:53.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.385+0000 7f24297fa640 1 -- 172.21.15.40:0/155639620 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2418005490 con 0x7f242c15f170 2024-04-26T21:46:53.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.385+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c09ee70 con 0x7f242c15f170 2024-04-26T21:46:53.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.385+0000 7f24297fa640 1 -- 172.21.15.40:0/155639620 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418007260 con 0x7f242c15f170 2024-04-26T21:46:53.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.385+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c09f430 con 0x7f242c15f170 2024-04-26T21:46:53.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.386+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c0694f0 con 0x7f242c15f170 2024-04-26T21:46:53.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.386+0000 7f24297fa640 1 -- 172.21.15.40:0/155639620 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2418007400 con 0x7f242c15f170 2024-04-26T21:46:53.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.387+0000 7f24297fa640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f241006c8f0 0x7f241006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:53.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.387+0000 7f24297fa640 1 -- 172.21.15.40:0/155639620 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9485+0+0 (secure 0 0 0) 0x7f241808bed0 con 0x7f242c15f170 2024-04-26T21:46:53.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.387+0000 7f2430875640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f241006c8f0 0x7f241006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:53.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.388+0000 7f2430875640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f241006c8f0 0x7f241006edb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2420034000 tx=0x7f242002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:53.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.391+0000 7f24297fa640 1 -- 172.21.15.40:0/155639620 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f242c0d1040 con 0x7f242c15f170 2024-04-26T21:46:53.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.617+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f242c0d1f70 con 0x7f242c15f170 2024-04-26T21:46:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.900+0000 7f24297fa640 1 -- 172.21.15.40:0/155639620 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v338) ==== 162+0+0 (secure 0 0 0) 0x7f2418058da0 con 0x7f242c15f170 2024-04-26T21:46:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f241006c8f0 msgr2=0x7f241006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f241006c8f0 0x7f241006edb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2420034000 tx=0x7f242002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242c15f170 msgr2=0x7f242c0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242c15f170 0x7f242c0ce170 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f241800c3c0 tx=0x7f241800c7c0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 shutdown_connections 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f241006c8f0 0x7f241006edb0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f242c0ce6b0 0x7f242c154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f242c15f170 0x7f242c0ce170 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 --2- 172.21.15.40:0/155639620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242c0d3bc0 0x7f242c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 >> 172.21.15.40:0/155639620 conn(0x7f242c0d4f60 msgr2=0x7f242c06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 shutdown_connections 2024-04-26T21:46:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:53.903+0000 7f2432ad9640 1 -- 172.21.15.40:0/155639620 wait complete. 2024-04-26T21:46:53.908 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:46:53.908+0000 7f3181c41640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: virtual void SessionMapStore::decode_header(ceph::bufferlist&) no longer understand old encoding version 1 < 101: Malformed input [buffer:3] 2024-04-26T21:46:53.909 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:53.908+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:46:53.909 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:46:53.908+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:46:53.915 INFO:tasks.ceph.mds.c.smithi040.stderr: -28> 2024-04-26T21:46:53.908+0000 7f3181c41640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: virtual void SessionMapStore::decode_header(ceph::bufferlist&) no longer understand old encoding version 1 < 101: Malformed input [buffer:3] 2024-04-26T21:46:53.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:53.939 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:46:54.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.018+0000 7ffbfc2bd640 1 Processor -- start 2024-04-26T21:46:54.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.018+0000 7ffbfc2bd640 1 -- start start 2024-04-26T21:46:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfc2bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf415f3a0 0x7ffbf40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfc2bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffbf40d1770 0x7ffbf40d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfc2bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffbf40d2130 0x7ffbf40d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfc2bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffbf40d25a0 con 0x7ffbf40d2130 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfc2bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffbf40d0e20 con 0x7ffbf40d1770 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfc2bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffbf40b1d70 con 0x7ffbf415f3a0 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfa85a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffbf40d2130 0x7ffbf40d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfa85a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffbf40d2130 0x7ffbf40d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59210/0 (socket says 172.21.15.40:59210) 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfa85a640 1 -- 172.21.15.40:0/136798683 learned_addr learned my addr 172.21.15.40:0/136798683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbf9858640 1 --2- 172.21.15.40:0/136798683 >> v2:172.21.15.184:3300/0 conn(0x7ffbf40d1770 0x7ffbf40d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.019+0000 7ffbfa059640 1 --2- 172.21.15.40:0/136798683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf415f3a0 0x7ffbf40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbfa85a640 1 -- 172.21.15.40:0/136798683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf415f3a0 msgr2=0x7ffbf40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbfa85a640 1 --2- 172.21.15.40:0/136798683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf415f3a0 0x7ffbf40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbfa85a640 1 -- 172.21.15.40:0/136798683 >> v2:172.21.15.184:3300/0 conn(0x7ffbf40d1770 msgr2=0x7ffbf40d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbfa85a640 1 --2- 172.21.15.40:0/136798683 >> v2:172.21.15.184:3300/0 conn(0x7ffbf40d1770 0x7ffbf40d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbfa85a640 1 -- 172.21.15.40:0/136798683 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf40b2050 con 0x7ffbf40d2130 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbfa85a640 1 --2- 172.21.15.40:0/136798683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf40d2130 0x7ffbf40d2e50 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7ffbf4073c50 tx=0x7ffbe402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a12b958fe6cfc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbf9057640 1 -- 172.21.15.40:0/136798683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4036070 con 0x7ffbf40d2130 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbf9057640 1 -- 172.21.15.40:0/136798683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe4031600 con 0x7ffbf40d2130 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.020+0000 7ffbf9057640 1 -- 172.21.15.40:0/136798683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe40319d0 con 0x7ffbf40d2130 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/136798683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf40d2130 msgr2=0x7ffbf40d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/136798683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf40d2130 0x7ffbf40d2e50 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7ffbf4073c50 tx=0x7ffbe402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/136798683 shutdown_connections 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/136798683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf40d2130 0x7ffbf40d2e50 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/136798683 >> v2:172.21.15.184:3300/0 conn(0x7ffbf40d1770 0x7ffbf40d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/136798683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf415f3a0 0x7ffbf40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/136798683 >> 172.21.15.40:0/136798683 conn(0x7ffbf40d4f90 msgr2=0x7ffbf40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/136798683 shutdown_connections 2024-04-26T21:46:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.021+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/136798683 wait complete. 2024-04-26T21:46:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.022+0000 7ffbfc2bd640 1 Processor -- start 2024-04-26T21:46:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.022+0000 7ffbfc2bd640 1 -- start start 2024-04-26T21:46:54.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfc2bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbf40d1770 0x7ffbf4153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfc2bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf40d2130 0x7ffbf40ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfc2bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf415f3a0 0x7ffbf415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfc2bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf415f900 con 0x7ffbf415f3a0 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfc2bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf415f780 con 0x7ffbf40d1770 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfc2bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffbf40d3390 con 0x7ffbf40d2130 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfa85a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf415f3a0 0x7ffbf415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfa85a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf415f3a0 0x7ffbf415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59214/0 (socket says 172.21.15.40:59214) 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfa85a640 1 -- 172.21.15.40:0/3028895300 learned_addr learned my addr 172.21.15.40:0/3028895300 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbf9858640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf40d2130 0x7ffbf40ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.023+0000 7ffbfa059640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbf40d1770 0x7ffbf4153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfa85a640 1 -- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf40d2130 msgr2=0x7ffbf40ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfa85a640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf40d2130 0x7ffbf40ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfa85a640 1 -- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbf40d1770 msgr2=0x7ffbf4153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfa85a640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbf40d1770 0x7ffbf4153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfa85a640 1 -- 172.21.15.40:0/3028895300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf40b8020 con 0x7ffbf415f3a0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfa059640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbf40d1770 0x7ffbf4153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfa85a640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf415f3a0 0x7ffbf415fa80 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009f40 tx=0x7ffbe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbf2ffd640 1 -- 172.21.15.40:0/3028895300 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe403f020 con 0x7ffbf415f3a0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbf2ffd640 1 -- 172.21.15.40:0/3028895300 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe4037040 con 0x7ffbf415f3a0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf40b8300 con 0x7ffbf415f3a0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbf2ffd640 1 -- 172.21.15.40:0/3028895300 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4036070 con 0x7ffbf415f3a0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.024+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf40b88c0 con 0x7ffbf415f3a0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.025+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbf40694f0 con 0x7ffbf415f3a0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.025+0000 7ffbf2ffd640 1 -- 172.21.15.40:0/3028895300 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbe4042020 con 0x7ffbf415f3a0 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.026+0000 7ffbf2ffd640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffbe006c8f0 0x7ffbe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.026+0000 7ffbf2ffd640 1 -- 172.21.15.40:0/3028895300 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7ffbe40ace20 con 0x7ffbf415f3a0 2024-04-26T21:46:54.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.026+0000 7ffbfa059640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffbe006c8f0 0x7ffbe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.027+0000 7ffbfa059640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffbe006c8f0 0x7ffbe006edb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009af0 tx=0x7ffbec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.030+0000 7ffbf2ffd640 1 -- 172.21.15.40:0/3028895300 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffbe4049030 con 0x7ffbf415f3a0 2024-04-26T21:46:54.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.257+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbf40b6860 con 0x7ffbf415f3a0 2024-04-26T21:46:54.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.258+0000 7ffbf2ffd640 1 -- 172.21.15.40:0/3028895300 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 340 v340) ==== 95+0+4355 (secure 0 0 0) 0x7ffbe407a2e0 con 0x7ffbf415f3a0 2024-04-26T21:46:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:54.259 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":340,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:53.909870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:54.259 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 340 2024-04-26T21:46:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffbe006c8f0 msgr2=0x7ffbe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffbe006c8f0 0x7ffbe006edb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009af0 tx=0x7ffbec008040 comp rx=0 tx=0).stop 2024-04-26T21:46:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf415f3a0 msgr2=0x7ffbf415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf415f3a0 0x7ffbf415fa80 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009f40 tx=0x7ffbe4004a40 comp rx=0 tx=0).stop 2024-04-26T21:46:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 shutdown_connections 2024-04-26T21:46:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffbe006c8f0 0x7ffbe006edb0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbf415f3a0 0x7ffbf415fa80 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbf40d2130 0x7ffbf40ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 --2- 172.21.15.40:0/3028895300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbf40d1770 0x7ffbf4153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 >> 172.21.15.40:0/3028895300 conn(0x7ffbf40d4f90 msgr2=0x7ffbf406e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 shutdown_connections 2024-04-26T21:46:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.260+0000 7ffbfc2bd640 1 -- 172.21.15.40:0/3028895300 wait complete. 2024-04-26T21:46:54.271 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:46:54.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.411+0000 7f05c15c8640 1 Processor -- start 2024-04-26T21:46:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.411+0000 7f05c15c8640 1 -- start start 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05c15c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f05bc15b600 0x7f05bc071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05c15c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05bc072450 con 0x7f05bc0cf5e0 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05c15c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05bc15b9e0 con 0x7f05bc0cf0e0 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05c15c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f05bc0cb9b0 con 0x7f05bc15b600 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05ba7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f05bc0cf5e0 0x7f05bc0ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57888/0 (socket says 172.21.15.40:57888) 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05ba7fc640 1 -- 172.21.15.40:0/3805600655 learned_addr learned my addr 172.21.15.40:0/3805600655 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05baffd640 1 --2- 172.21.15.40:0/3805600655 >> v2:172.21.15.184:3300/0 conn(0x7f05bc15b600 0x7f05bc071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641764437 0 0) 0x7f05b0001110 con 0x7f05bc0cf5e0 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.412+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05a4003520 con 0x7f05bc0cf5e0 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4141590387 0 0) 0x7f05a4003520 con 0x7f05bc0cf5e0 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 >> v2:172.21.15.184:3300/0 conn(0x7f05bc15b600 msgr2=0x7f05bc071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 --2- 172.21.15.40:0/3805600655 >> v2:172.21.15.184:3300/0 conn(0x7f05bc15b600 0x7f05bc071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 >> v1:172.21.15.184:6789/0 conn(0x7f05bc0cf0e0 legacy=0x7f05bc06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc0d0740 con 0x7f05bc0cf5e0 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f05b0001840 con 0x7f05bc0cf5e0 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 >> v1:172.21.15.40:6789/0 conn(0x7f05bc0cf5e0 legacy=0x7f05bc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a40039b0 0x7f05a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a4004800 0x7f05a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05a40051c0 0x7f05a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc0cb9b0 con 0x7f05a4004800 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05a400c060 con 0x7f05a40051c0 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05a400c260 con 0x7f05a40039b0 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05ba7fc640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a4004800 0x7f05a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05baffd640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a40039b0 0x7f05a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05bb7fe640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05a40051c0 0x7f05a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05ba7fc640 1 -- 172.21.15.40:0/3805600655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a40039b0 msgr2=0x7f05a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05ba7fc640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a40039b0 0x7f05a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05ba7fc640 1 -- 172.21.15.40:0/3805600655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05a40051c0 msgr2=0x7f05a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05ba7fc640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05a40051c0 0x7f05a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05ba7fc640 1 -- 172.21.15.40:0/3805600655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a4003520 con 0x7f05a4004800 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.413+0000 7f05bb7fe640 1 -- 172.21.15.40:0/3805600655 reap_dead start 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.414+0000 7f05ba7fc640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a4004800 0x7f05a4004c80 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f05b0007e90 tx=0x7f05a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea709c6964c7947 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.414+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b000c0e0 con 0x7f05a4004800 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.414+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0031070 con 0x7f05a4004800 2024-04-26T21:46:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.414+0000 7f05b9ffb640 1 -- 172.21.15.40:0/3805600655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0005d30 con 0x7f05a4004800 2024-04-26T21:46:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.414+0000 7f05c15c8640 1 -- 172.21.15.40:0/3805600655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a4004800 msgr2=0x7f05a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.414+0000 7f05c15c8640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a4004800 0x7f05a4004c80 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f05b0007e90 tx=0x7f05a4005830 comp rx=0 tx=0).stop 2024-04-26T21:46:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.415+0000 7f05c15c8640 1 -- 172.21.15.40:0/3805600655 shutdown_connections 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.415+0000 7f05c15c8640 1 --2- 172.21.15.40:0/3805600655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05a4004800 0x7f05a4004c80 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.415+0000 7f05c15c8640 1 -- 172.21.15.40:0/3805600655 >> 172.21.15.40:0/3805600655 conn(0x7f05bc0d0be0 msgr2=0x7f05bc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.415+0000 7f05c15c8640 1 -- 172.21.15.40:0/3805600655 shutdown_connections 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.415+0000 7f05c15c8640 1 -- 172.21.15.40:0/3805600655 wait complete. 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.416+0000 7f05c15c8640 1 Processor -- start 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.416+0000 7f05c15c8640 1 -- start start 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05c15c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a4004800 0x7f05bc1f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05c15c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05bc0b1a00 0x7f05bc0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05c15c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05bc0c58a0 0x7f05bc0c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05c15c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc0d02f0 con 0x7f05bc0b1a00 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05c15c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc0cb640 con 0x7f05bc0c58a0 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05c15c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05bc0d0470 con 0x7f05a4004800 2024-04-26T21:46:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05bc0b1a00 0x7f05bc0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05bc0b1a00 0x7f05bc0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59226/0 (socket says 172.21.15.40:59226) 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 -- 172.21.15.40:0/2998217450 learned_addr learned my addr 172.21.15.40:0/2998217450 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05baffd640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a4004800 0x7f05bc1f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05bb7fe640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05bc0c58a0 0x7f05bc0c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 -- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a4004800 msgr2=0x7f05bc1f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a4004800 0x7f05bc1f04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 -- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05bc0c58a0 msgr2=0x7f05bc0c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05bc0c58a0 0x7f05bc0c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 -- 172.21.15.40:0/2998217450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc0c6610 con 0x7f05bc0b1a00 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05bb7fe640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05bc0c58a0 0x7f05bc0c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.417+0000 7f05ba7fc640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05bc0b1a00 0x7f05bc0b1e80 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f05b0007fb0 tx=0x7f05bc0cd470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.418+0000 7f05a37fe640 1 -- 172.21.15.40:0/2998217450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0007270 con 0x7f05bc0b1a00 2024-04-26T21:46:54.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.418+0000 7f05a37fe640 1 -- 172.21.15.40:0/2998217450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0007410 con 0x7f05bc0b1a00 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.418+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc0b39b0 con 0x7f05bc0b1a00 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.418+0000 7f05a37fe640 1 -- 172.21.15.40:0/2998217450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b00077c0 con 0x7f05bc0b1a00 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.418+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc0b3f70 con 0x7f05bc0b1a00 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.419+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05bc0694f0 con 0x7f05bc0b1a00 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.419+0000 7f05a37fe640 1 -- 172.21.15.40:0/2998217450 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05b002d520 con 0x7f05bc0b1a00 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.419+0000 7f05a37fe640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f05a806c920 0x7f05a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.420+0000 7f05a37fe640 1 -- 172.21.15.40:0/2998217450 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7f05b00aec60 con 0x7f05bc0b1a00 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.420+0000 7f05baffd640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f05a806c920 0x7f05a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.420+0000 7f05baffd640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f05a806c920 0x7f05a806ede0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00ac80 tx=0x7f05ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.424+0000 7f05a37fe640 1 -- 172.21.15.40:0/2998217450 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05bc072450 con 0x7f05bc0b1a00 2024-04-26T21:46:54.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.648+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05bc0ce7b0 con 0x7f05bc0b1a00 2024-04-26T21:46:54.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.648+0000 7f05a37fe640 1 -- 172.21.15.40:0/2998217450 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 340 v340) ==== 95+0+4355 (secure 0 0 0) 0x7f05b007c1b0 con 0x7f05bc0b1a00 2024-04-26T21:46:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:54.649 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":340,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:53.909870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 340 2024-04-26T21:46:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f05a806c920 msgr2=0x7f05a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f05a806c920 0x7f05a806ede0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00ac80 tx=0x7f05ac008040 comp rx=0 tx=0).stop 2024-04-26T21:46:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05bc0b1a00 msgr2=0x7f05bc0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05bc0b1a00 0x7f05bc0b1e80 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f05b0007fb0 tx=0x7f05bc0cd470 comp rx=0 tx=0).stop 2024-04-26T21:46:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 shutdown_connections 2024-04-26T21:46:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f05a806c920 0x7f05a806ede0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05bc0c58a0 0x7f05bc0c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05bc0b1a00 0x7f05bc0b1e80 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 --2- 172.21.15.40:0/2998217450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05a4004800 0x7f05bc1f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 >> 172.21.15.40:0/2998217450 conn(0x7f05bc0d0be0 msgr2=0x7f05bc0cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 shutdown_connections 2024-04-26T21:46:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.650+0000 7f05c15c8640 1 -- 172.21.15.40:0/2998217450 wait complete. 2024-04-26T21:46:54.662 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:46:54.662 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.764+0000 7f5ed8083640 1 Processor -- start 2024-04-26T21:46:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.764+0000 7f5ed8083640 1 -- start start 2024-04-26T21:46:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed8083640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5ed0162810 0x7f5ed0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed8083640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ed00b2340 0x7f5ed00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed8083640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed00d10d0 con 0x7f5ed00cd930 2024-04-26T21:46:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed8083640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed00d0750 con 0x7f5ed00b2340 2024-04-26T21:46:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed8083640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed00d16a0 con 0x7f5ed0162810 2024-04-26T21:46:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed561e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ed00b2340 0x7f5ed00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed561e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ed00b2340 0x7f5ed00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59238/0 (socket says 172.21.15.40:59238) 2024-04-26T21:46:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed561e640 1 -- 172.21.15.40:0/2807937456 learned_addr learned my addr 172.21.15.40:0/2807937456 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed561e640 1 -- 172.21.15.40:0/2807937456 >> v2:172.21.15.184:3300/0 conn(0x7f5ed0162810 msgr2=0x7f5ed0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed5e1f640 1 --2- 172.21.15.40:0/2807937456 >> v2:172.21.15.184:3300/0 conn(0x7f5ed0162810 0x7f5ed0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed561e640 1 --2- 172.21.15.40:0/2807937456 >> v2:172.21.15.184:3300/0 conn(0x7f5ed0162810 0x7f5ed0070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed561e640 1 -- 172.21.15.40:0/2807937456 >> v1:172.21.15.184:6789/0 conn(0x7f5ed00cd930 legacy=0x7f5ed00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.765+0000 7f5ed561e640 1 -- 172.21.15.40:0/2807937456 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed00d1370 con 0x7f5ed00b2340 2024-04-26T21:46:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.766+0000 7f5ed561e640 1 --2- 172.21.15.40:0/2807937456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed00b2340 0x7f5ed00cd3f0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0073d90 tx=0x7f5ec002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593f263d1cd3e86c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.766+0000 7f5ed4e1d640 1 -- 172.21.15.40:0/2807937456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0037070 con 0x7f5ed00b2340 2024-04-26T21:46:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.766+0000 7f5ed4e1d640 1 -- 172.21.15.40:0/2807937456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec0032440 con 0x7f5ed00b2340 2024-04-26T21:46:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.766+0000 7f5ed4e1d640 1 -- 172.21.15.40:0/2807937456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0032810 con 0x7f5ed00b2340 2024-04-26T21:46:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.767+0000 7f5ed8083640 1 -- 172.21.15.40:0/2807937456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed00b2340 msgr2=0x7f5ed00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.767+0000 7f5ed8083640 1 --2- 172.21.15.40:0/2807937456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed00b2340 0x7f5ed00cd3f0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0073d90 tx=0x7f5ec002a230 comp rx=0 tx=0).stop 2024-04-26T21:46:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.767+0000 7f5ed8083640 1 -- 172.21.15.40:0/2807937456 shutdown_connections 2024-04-26T21:46:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.767+0000 7f5ed8083640 1 --2- 172.21.15.40:0/2807937456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed00b2340 0x7f5ed00cd3f0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.767+0000 7f5ed8083640 1 --2- 172.21.15.40:0/2807937456 >> v2:172.21.15.184:3300/0 conn(0x7f5ed0162810 0x7f5ed0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.767+0000 7f5ed8083640 1 -- 172.21.15.40:0/2807937456 >> 172.21.15.40:0/2807937456 conn(0x7f5ed00d2da0 msgr2=0x7f5ed00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.767+0000 7f5ed8083640 1 -- 172.21.15.40:0/2807937456 shutdown_connections 2024-04-26T21:46:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.767+0000 7f5ed8083640 1 -- 172.21.15.40:0/2807937456 wait complete. 2024-04-26T21:46:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.768+0000 7f5ed8083640 1 Processor -- start 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.768+0000 7f5ed8083640 1 -- start start 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed8083640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed00b2340 0x7f5ed0161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed8083640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed00cd930 0x7f5ed014e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed8083640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed0162810 0x7f5ed00cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed8083640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0070890 con 0x7f5ed0162810 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed8083640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed00d0470 con 0x7f5ed00cd930 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed8083640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed00d2240 con 0x7f5ed00b2340 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed0162810 0x7f5ed00cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed0162810 0x7f5ed00cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59246/0 (socket says 172.21.15.40:59246) 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 -- 172.21.15.40:0/1836409232 learned_addr learned my addr 172.21.15.40:0/1836409232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed5e1f640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed00b2340 0x7f5ed0161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed561e640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed00cd930 0x7f5ed014e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 -- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed00b2340 msgr2=0x7f5ed0161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed00b2340 0x7f5ed0161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 -- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed00cd930 msgr2=0x7f5ed014e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed00cd930 0x7f5ed014e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 -- 172.21.15.40:0/1836409232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed0158880 con 0x7f5ed0162810 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed6620640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed0162810 0x7f5ed00cc6b0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800c780 tx=0x7f5ec800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ec67fc640 1 -- 172.21.15.40:0/1836409232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8017070 con 0x7f5ed0162810 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ec67fc640 1 -- 172.21.15.40:0/1836409232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec8003da0 con 0x7f5ed0162810 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed00cb610 con 0x7f5ed0162810 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ec67fc640 1 -- 172.21.15.40:0/1836409232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec80052b0 con 0x7f5ed0162810 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.769+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed00cbbd0 con 0x7f5ed0162810 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.770+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed00694f0 con 0x7f5ed0162810 2024-04-26T21:46:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.771+0000 7f5ec67fc640 1 -- 172.21.15.40:0/1836409232 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ec8005450 con 0x7f5ed0162810 2024-04-26T21:46:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.771+0000 7f5ec67fc640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5eb406c8f0 0x7f5eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.771+0000 7f5ec67fc640 1 -- 172.21.15.40:0/1836409232 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7f5ec808bfe0 con 0x7f5ed0162810 2024-04-26T21:46:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.772+0000 7f5ed5e1f640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5eb406c8f0 0x7f5eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.772+0000 7f5ed5e1f640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5eb406c8f0 0x7f5eb406edb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00aa20 tx=0x7f5ecc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:54.776+0000 7f5ec67fc640 1 -- 172.21.15.40:0/1836409232 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ec8058e00 con 0x7f5ed0162810 2024-04-26T21:46:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.001+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ed00d1eb0 con 0x7f5ed0162810 2024-04-26T21:46:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.001+0000 7f5ec67fc640 1 -- 172.21.15.40:0/1836409232 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 341 v341) ==== 95+0+5179 (secure 0 0 0) 0x7f5ec805d100 con 0x7f5ed0162810 2024-04-26T21:46:55.002 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:55.003 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":341,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/933356185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":933356185},{"type":"v1","addr":"172.21.15.40:6833","nonce":933356185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":341}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:53.909870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:55.003 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 341 2024-04-26T21:46:55.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5eb406c8f0 msgr2=0x7f5eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5eb406c8f0 0x7f5eb406edb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00aa20 tx=0x7f5ecc008040 comp rx=0 tx=0).stop 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed0162810 msgr2=0x7f5ed00cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed0162810 0x7f5ed00cc6b0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800c780 tx=0x7f5ec800cc40 comp rx=0 tx=0).stop 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 shutdown_connections 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5eb406c8f0 0x7f5eb406edb0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ed0162810 0x7f5ed00cc6b0 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed00cd930 0x7f5ed014e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 --2- 172.21.15.40:0/1836409232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed00b2340 0x7f5ed0161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 >> 172.21.15.40:0/1836409232 conn(0x7f5ed00d2da0 msgr2=0x7f5ed00b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.003+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 shutdown_connections 2024-04-26T21:46:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.004+0000 7f5ed8083640 1 -- 172.21.15.40:0/1836409232 wait complete. 2024-04-26T21:46:55.015 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_sessionmap' led to DAMAGED state 2024-04-26T21:46:55.015 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 1.00000000' 2024-04-26T21:46:55.015 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:46:55.015 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:46:55.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:46:55.031 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:46:55.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:46:55.031 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:46:55.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:46:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T21:46:55.090 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:46:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.230+0000 7f1af2b33640 1 Processor -- start 2024-04-26T21:46:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.230+0000 7f1af2b33640 1 -- start start 2024-04-26T21:46:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af2b33640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1aec162880 0x7f1aec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af2b33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 0x7f1aec0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af2b33640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aec0b7e50 con 0x7f1aec0d39f0 2024-04-26T21:46:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af2b33640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1aec0d0580 con 0x7f1aec162880 2024-04-26T21:46:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af2b33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1aec0d1800 con 0x7f1aec0b5f20 2024-04-26T21:46:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af08cf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1aec162880 0x7f1aec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af10d0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1aec0d39f0 0x7f1aec0b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57898/0 (socket says 172.21.15.40:57898) 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af10d0640 1 -- 172.21.15.40:0/460883473 learned_addr learned my addr 172.21.15.40:0/460883473 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af08cf640 1 -- 172.21.15.40:0/460883473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 msgr2=0x7f1aec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af08cf640 1 --2- 172.21.15.40:0/460883473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 0x7f1aec0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af08cf640 1 -- 172.21.15.40:0/460883473 >> v1:172.21.15.40:6789/0 conn(0x7f1aec0d39f0 legacy=0x7f1aec0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1af08cf640 1 -- 172.21.15.40:0/460883473 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec0d07d0 con 0x7f1aec162880 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.231+0000 7f1aeb7fe640 1 -- 172.21.15.40:0/460883473 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1913248906 0 0) 0x7f1aec0b7e50 con 0x7f1aec0d39f0 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.232+0000 7f1af08cf640 1 --2- 172.21.15.40:0/460883473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec162880 0x7f1aec0d2cb0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009d90 tx=0x7f1ae402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd554d66e147eac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.232+0000 7f1aeb7fe640 1 -- 172.21.15.40:0/460883473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4007740 con 0x7f1aec162880 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.232+0000 7f1aeb7fe640 1 -- 172.21.15.40:0/460883473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae40078e0 con 0x7f1aec162880 2024-04-26T21:46:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.232+0000 7f1aeb7fe640 1 -- 172.21.15.40:0/460883473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4007cb0 con 0x7f1aec162880 2024-04-26T21:46:55.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.233+0000 7f1af2b33640 1 -- 172.21.15.40:0/460883473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec162880 msgr2=0x7f1aec0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.233+0000 7f1af2b33640 1 --2- 172.21.15.40:0/460883473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec162880 0x7f1aec0d2cb0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009d90 tx=0x7f1ae402a660 comp rx=0 tx=0).stop 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.233+0000 7f1af2b33640 1 -- 172.21.15.40:0/460883473 shutdown_connections 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.233+0000 7f1af2b33640 1 --2- 172.21.15.40:0/460883473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 0x7f1aec0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.233+0000 7f1af2b33640 1 --2- 172.21.15.40:0/460883473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec162880 0x7f1aec0d2cb0 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.233+0000 7f1af2b33640 1 -- 172.21.15.40:0/460883473 >> 172.21.15.40:0/460883473 conn(0x7f1aec0d4f60 msgr2=0x7f1aec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.233+0000 7f1af2b33640 1 -- 172.21.15.40:0/460883473 shutdown_connections 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.233+0000 7f1af2b33640 1 -- 172.21.15.40:0/460883473 wait complete. 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.234+0000 7f1af2b33640 1 Processor -- start 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.234+0000 7f1af2b33640 1 -- start start 2024-04-26T21:46:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af2b33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 0x7f1aec06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af2b33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec0d39f0 0x7f1aec1f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af2b33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1aec162880 0x7f1aec15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af2b33640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec162c60 con 0x7f1aec0d39f0 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af2b33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec0d0d90 con 0x7f1aec162880 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af2b33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1aec162de0 con 0x7f1aec0b5f20 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1aebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec0d39f0 0x7f1aec1f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1aebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec0d39f0 0x7f1aec1f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59266/0 (socket says 172.21.15.40:59266) 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1aebfff640 1 -- 172.21.15.40:0/3465183073 learned_addr learned my addr 172.21.15.40:0/3465183073 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af08cf640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 0x7f1aec06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af10d0640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1aec162880 0x7f1aec15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1aebfff640 1 -- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 msgr2=0x7f1aec06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1aebfff640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 0x7f1aec06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1aebfff640 1 -- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1aec162880 msgr2=0x7f1aec15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1aebfff640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1aec162880 0x7f1aec15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1aebfff640 1 -- 172.21.15.40:0/3465183073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec1f5380 con 0x7f1aec0d39f0 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.235+0000 7f1af10d0640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1aec162880 0x7f1aec15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.236+0000 7f1aebfff640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec0d39f0 0x7f1aec1f4e40 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00b410 tx=0x7f1adc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.236+0000 7f1ae97fa640 1 -- 172.21.15.40:0/3465183073 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc015020 con 0x7f1aec0d39f0 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.236+0000 7f1ae97fa640 1 -- 172.21.15.40:0/3465183073 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc011850 con 0x7f1aec0d39f0 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.236+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aec1f5600 con 0x7f1aec0d39f0 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.236+0000 7f1ae97fa640 1 -- 172.21.15.40:0/3465183073 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc011c40 con 0x7f1aec0d39f0 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.236+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aec1f5bc0 con 0x7f1aec0d39f0 2024-04-26T21:46:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.237+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aec0764d0 con 0x7f1aec0d39f0 2024-04-26T21:46:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.237+0000 7f1ae97fa640 1 -- 172.21.15.40:0/3465183073 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1adc018020 con 0x7f1aec0d39f0 2024-04-26T21:46:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.238+0000 7f1ae97fa640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1acc06c8f0 0x7f1acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.238+0000 7f1ae97fa640 1 -- 172.21.15.40:0/3465183073 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7f1adc08c9b0 con 0x7f1aec0d39f0 2024-04-26T21:46:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.238+0000 7f1af08cf640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1acc06c8f0 0x7f1acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.239+0000 7f1af08cf640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1acc06c8f0 0x7f1acc06edb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4034000 tx=0x7f1ae402f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.242+0000 7f1ae97fa640 1 -- 172.21.15.40:0/3465183073 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1aec0764d0 con 0x7f1aec0d39f0 2024-04-26T21:46:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.467+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1aec0694f0 con 0x7f1aec0d39f0 2024-04-26T21:46:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.919+0000 7f1ae97fa640 1 -- 172.21.15.40:0/3465183073 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v342) ==== 161+0+0 (secure 0 0 0) 0x7f1adc01f020 con 0x7f1aec0d39f0 2024-04-26T21:46:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1acc06c8f0 msgr2=0x7f1acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1acc06c8f0 0x7f1acc06edb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4034000 tx=0x7f1ae402f000 comp rx=0 tx=0).stop 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec0d39f0 msgr2=0x7f1aec1f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec0d39f0 0x7f1aec1f4e40 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00b410 tx=0x7f1adc00b8d0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 shutdown_connections 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f1acc06c8f0 0x7f1acc06edb0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1aec162880 0x7f1aec15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec0d39f0 0x7f1aec1f4e40 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 --2- 172.21.15.40:0/3465183073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1aec0b5f20 0x7f1aec06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 >> 172.21.15.40:0/3465183073 conn(0x7f1aec0d4f60 msgr2=0x7f1aec15f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 shutdown_connections 2024-04-26T21:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:55.922+0000 7f1af2b33640 1 -- 172.21.15.40:0/3465183073 wait complete. 2024-04-26T21:46:55.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-26T21:46:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.042+0000 7faaa1904640 1 Processor -- start 2024-04-26T21:46:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.042+0000 7faaa1904640 1 -- start start 2024-04-26T21:46:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faaa1904640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa9c0b2340 0x7faa9c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faaa1904640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9c0d10d0 con 0x7faa9c0cd930 2024-04-26T21:46:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faaa1904640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9c0d1370 con 0x7faa9c162810 2024-04-26T21:46:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faaa1904640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7faa9c0d0750 con 0x7faa9c0b2340 2024-04-26T21:46:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faa9a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa9c0b2340 0x7faa9c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faa9b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa9c0cd930 0x7faa9c0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57908/0 (socket says 172.21.15.40:57908) 2024-04-26T21:46:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faa9b7fe640 1 -- 172.21.15.40:0/3997086173 learned_addr learned my addr 172.21.15.40:0/3997086173 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:46:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faa9a7fc640 1 -- 172.21.15.40:0/3997086173 >> v1:172.21.15.184:6789/0 conn(0x7faa9c162810 legacy=0x7faa9c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faa9a7fc640 1 -- 172.21.15.40:0/3997086173 >> v1:172.21.15.40:6789/0 conn(0x7faa9c0cd930 legacy=0x7faa9c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.043+0000 7faa9a7fc640 1 -- 172.21.15.40:0/3997086173 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c0d01b0 con 0x7faa9c0b2340 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.044+0000 7faa9a7fc640 1 --2- 172.21.15.40:0/3997086173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0cd3f0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0d1540 tx=0x7faa9002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7731cd56f0a5f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.044+0000 7faa99ffb640 1 -- 172.21.15.40:0/3997086173 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90037070 con 0x7faa9c0b2340 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.044+0000 7faa99ffb640 1 -- 172.21.15.40:0/3997086173 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa90032440 con 0x7faa9c0b2340 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.044+0000 7faa99ffb640 1 -- 172.21.15.40:0/3997086173 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90032810 con 0x7faa9c0b2340 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.045+0000 7faaa1904640 1 -- 172.21.15.40:0/3997086173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 msgr2=0x7faa9c0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.045+0000 7faaa1904640 1 --2- 172.21.15.40:0/3997086173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0cd3f0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0d1540 tx=0x7faa9002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.045+0000 7faaa1904640 1 -- 172.21.15.40:0/3997086173 shutdown_connections 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.045+0000 7faaa1904640 1 --2- 172.21.15.40:0/3997086173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0cd3f0 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.045+0000 7faaa1904640 1 -- 172.21.15.40:0/3997086173 >> 172.21.15.40:0/3997086173 conn(0x7faa9c0d2da0 msgr2=0x7faa9c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:56.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.045+0000 7faaa1904640 1 -- 172.21.15.40:0/3997086173 shutdown_connections 2024-04-26T21:46:56.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.045+0000 7faaa1904640 1 -- 172.21.15.40:0/3997086173 wait complete. 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.046+0000 7faaa1904640 1 Processor -- start 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.046+0000 7faaa1904640 1 -- start start 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faaa1904640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faaa1904640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa9c0cd930 0x7faa9c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faaa1904640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa9c162810 0x7faa9c0cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faaa1904640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c0d2240 con 0x7faa9c0b2340 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faaa1904640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c0d16a0 con 0x7faa9c162810 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faaa1904640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa9c0d23c0 con 0x7faa9c0cd930 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59292/0 (socket says 172.21.15.40:59292) 2024-04-26T21:46:56.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 -- 172.21.15.40:0/696131219 learned_addr learned my addr 172.21.15.40:0/696131219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9a7fc640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa9c0cd930 0x7faa9c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9b7fe640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa9c162810 0x7faa9c0cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 -- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa9c0cd930 msgr2=0x7faa9c0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa9c0cd930 0x7faa9c0cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 -- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa9c162810 msgr2=0x7faa9c0cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa9c162810 0x7faa9c0cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 -- 172.21.15.40:0/696131219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c1f8900 con 0x7faa9c0b2340 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9a7fc640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa9c0cd930 0x7faa9c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9b7fe640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa9c162810 0x7faa9c0cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.047+0000 7faa9affd640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0a0350 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b3e0 tx=0x7faa8c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.048+0000 7faa83fff640 1 -- 172.21.15.40:0/696131219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c015020 con 0x7faa9c0b2340 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.048+0000 7faa83fff640 1 -- 172.21.15.40:0/696131219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c011850 con 0x7faa9c0b2340 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.048+0000 7faa83fff640 1 -- 172.21.15.40:0/696131219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c011c40 con 0x7faa9c0b2340 2024-04-26T21:46:56.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.048+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c1f8b70 con 0x7faa9c0b2340 2024-04-26T21:46:56.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.048+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c1f9130 con 0x7faa9c0b2340 2024-04-26T21:46:56.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.049+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9c0694f0 con 0x7faa9c0b2340 2024-04-26T21:46:56.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.049+0000 7faa83fff640 1 -- 172.21.15.40:0/696131219 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa8c018020 con 0x7faa9c0b2340 2024-04-26T21:46:56.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.050+0000 7faa83fff640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa8806c8f0 0x7faa8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.050+0000 7faa83fff640 1 -- 172.21.15.40:0/696131219 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7faa8c08c9a0 con 0x7faa9c0b2340 2024-04-26T21:46:56.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.050+0000 7faa9a7fc640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa8806c8f0 0x7faa8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.051+0000 7faa9a7fc640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa8806c8f0 0x7faa8806edb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7faa90034000 tx=0x7faa9002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.054+0000 7faa83fff640 1 -- 172.21.15.40:0/696131219 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa8c01f020 con 0x7faa9c0b2340 2024-04-26T21:46:56.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.316+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7faa9c0d1e60 con 0x7faa9c0b2340 2024-04-26T21:46:56.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.926+0000 7faa83fff640 1 -- 172.21.15.40:0/696131219 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v343) ==== 101+0+0 (secure 0 0 0) 0x7faa8c059ef0 con 0x7faa9c0b2340 2024-04-26T21:46:56.927 INFO:teuthology.orchestra.run.smithi040.stderr:repaired: restoring rank 6:0 2024-04-26T21:46:56.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa8806c8f0 msgr2=0x7faa8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa8806c8f0 0x7faa8806edb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7faa90034000 tx=0x7faa9002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 msgr2=0x7faa9c0a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0a0350 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b3e0 tx=0x7faa8c00b8a0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 shutdown_connections 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7faa8806c8f0 0x7faa8806edb0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa9c162810 0x7faa9c0cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa9c0cd930 0x7faa9c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 --2- 172.21.15.40:0/696131219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa9c0b2340 0x7faa9c0a0350 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 >> 172.21.15.40:0/696131219 conn(0x7faa9c0d2da0 msgr2=0x7faa9c0b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 shutdown_connections 2024-04-26T21:46:56.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.929+0000 7faaa1904640 1 -- 172.21.15.40:0/696131219 wait complete. 2024-04-26T21:46:56.943 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-26T21:46:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.987+0000 7f4d4a628f40 1 Processor -- start 2024-04-26T21:46:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.987+0000 7f4d4a628f40 1 -- start start 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d4a628f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b240a0650 0x555b2409bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d4a628f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240b9510 0x555b240b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d4a628f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b24145820 0x555b240b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d4a628f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555b24146140 con 0x555b240b9510 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d4a628f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555b24017e40 con 0x555b24145820 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d4a628f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555b24146420 con 0x555b240a0650 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240b9510 0x555b240b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240b9510 0x555b240b6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59304/0 (socket says 172.21.15.40:59304) 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 -- 172.21.15.40:0/980604191 learned_addr learned my addr 172.21.15.40:0/980604191 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d48122640 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b240a0650 0x555b2409bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 -- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b240a0650 msgr2=0x555b2409bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b240a0650 0x555b2409bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 -- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b24145820 msgr2=0x555b240b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d48923640 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b24145820 0x555b240b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b24145820 0x555b240b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 -- 172.21.15.40:0/980604191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b24136020 con 0x555b240b9510 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d48122640 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b240a0650 0x555b2409bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d48923640 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b24145820 0x555b240b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.988+0000 7f4d47921640 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240b9510 0x555b240b6ae0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c025680 tx=0x7f4d3c04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459b1683fb5b5c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4691f640 1 -- 172.21.15.40:0/980604191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c059040 con 0x555b240b9510 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4691f640 1 -- 172.21.15.40:0/980604191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c063070 con 0x555b240b9510 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4691f640 1 -- 172.21.15.40:0/980604191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c054440 con 0x555b240b9510 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 -- 172.21.15.40:0/980604191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240b9510 msgr2=0x555b240b6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240b9510 0x555b240b6ae0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c025680 tx=0x7f4d3c04e000 comp rx=0 tx=0).stop 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 -- 172.21.15.40:0/980604191 shutdown_connections 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b24145820 0x555b240b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240b9510 0x555b240b6ae0 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/980604191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b240a0650 0x555b2409bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 -- 172.21.15.40:0/980604191 >> 172.21.15.40:0/980604191 conn(0x555b2409a8a0 msgr2=0x555b24147f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 -- 172.21.15.40:0/980604191 shutdown_connections 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.989+0000 7f4d4a628f40 1 -- 172.21.15.40:0/980604191 wait complete. 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.990+0000 7f4d4a628f40 1 Processor -- start 2024-04-26T21:46:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.990+0000 7f4d4a628f40 1 -- start start 2024-04-26T21:46:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4a628f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240a0650 0x555b241329e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4a628f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b240b9510 0x555b2409f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4a628f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b24145820 0x555b241315e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4a628f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555b24071450 con 0x555b240a0650 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4a628f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555b2409b810 con 0x555b240b9510 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4a628f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555b24071730 con 0x555b24145820 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240a0650 0x555b241329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240a0650 0x555b241329e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59318/0 (socket says 172.21.15.40:59318) 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 learned_addr learned my addr 172.21.15.40:0/756504965 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48923640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b24145820 0x555b241315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d47921640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b240b9510 0x555b2409f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b24145820 msgr2=0x555b241315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555b24145820 0x555b241315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b240b9510 msgr2=0x555b2409f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555b240b9510 0x555b2409f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b24147830 con 0x555b240a0650 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d48122640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240a0650 0x555b241329e0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c450 tx=0x7f4d3800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4591d640 1 -- 172.21.15.40:0/756504965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38003650 con 0x555b240a0650 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4591d640 1 -- 172.21.15.40:0/756504965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d380037f0 con 0x555b240a0650 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b240b4ba0 con 0x555b240a0650 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.991+0000 7f4d4591d640 1 -- 172.21.15.40:0/756504965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38003bc0 con 0x555b240a0650 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.992+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b24131b20 con 0x555b240a0650 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.993+0000 7f4d4591d640 1 -- 172.21.15.40:0/756504965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d3801a020 con 0x555b240a0650 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.993+0000 7f4d4591d640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d3406b2d0 0x7f4d3406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.993+0000 7f4d4591d640 1 -- 172.21.15.40:0/756504965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7f4d3808b700 con 0x555b240a0650 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.994+0000 7f4d47921640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d3406b2d0 0x7f4d3406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.997 INFO:teuthology.orchestra.run.smithi040.stdout:Importing pool 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.994+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x555b24133490 0x555b241410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.994+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b24147d10 con 0x555b24133490 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.994+0000 7f4d48923640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x555b24133490 0x555b241410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.994+0000 7f4d47921640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d3406b2d0 0x7f4d3406d790 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c057040 tx=0x7f4d3c051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.994+0000 7f4d48923640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x555b24133490 0x555b241410f0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.996+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== osd_op_reply(1 601.00000000 [create] v134'1190 uv1164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.996+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241480d0 con 0x555b24133490 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.997+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== osd_op_reply(2 601.00000000 [delete] v134'1191 uv1191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.997+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241484a0 con 0x555b24133490 2024-04-26T21:46:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.999+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 3 ==== osd_op_reply(3 601.00000000 [create] v134'1192 uv1192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.000 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-26T21:46:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:56.999+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ec1d0 con 0x555b24133490 2024-04-26T21:46:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.000+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v134'1193 uv1193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.000+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ec570 con 0x555b24133490 2024-04-26T21:46:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.001+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v134'1194 uv1194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.001+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eb670 con 0x555b24133490 2024-04-26T21:46:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.002+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v134'1195 uv1195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.002+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.003+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 7 ==== osd_op_reply(7 602.00000000 [create] v134'1196 uv1170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.003+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.004+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 8 ==== osd_op_reply(8 602.00000000 [delete] v134'1197 uv1197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.004+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.006+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 9 ==== osd_op_reply(9 602.00000000 [create] v134'1198 uv1198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.006 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-26T21:46:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.006+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.007+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v134'1199 uv1199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.007+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.008+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v134'1200 uv1200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.008+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.009+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v134'1201 uv1201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.009+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.010+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 13 ==== osd_op_reply(13 10000000000.00000000 [create] v134'1202 uv1177 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.010+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.012+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v134'1203 uv1203 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.012+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.013+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 15 ==== osd_op_reply(15 10000000000.00000000 [create] v134'1204 uv1204 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.014 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-26T21:46:57.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.013+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.014+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v134'1205 uv1205 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.014+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.015+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v134'1206 uv1206 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.015+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241eba60 con 0x555b24133490 2024-04-26T21:46:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.017+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v134'1207 uv1207 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.017+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=585b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ebe20 con 0x555b24133490 2024-04-26T21:46:57.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.018+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v134'1208 uv1208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.018+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ec910 con 0x555b24133490 2024-04-26T21:46:57.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.019+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 20 ==== osd_op_reply(20 600.00000000 [create] v134'1209 uv1183 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.019+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241ec910 con 0x555b24133490 2024-04-26T21:46:57.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.020+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 21 ==== osd_op_reply(21 600.00000000 [delete] v134'1210 uv1210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.021+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ec910 con 0x555b24133490 2024-04-26T21:46:57.022 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-26T21:46:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.022+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 22 ==== osd_op_reply(22 600.00000000 [create] v134'1211 uv1211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.022+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ed920 con 0x555b24133490 2024-04-26T21:46:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.023+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v134'1212 uv1212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.023+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ed920 con 0x555b24133490 2024-04-26T21:46:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.024+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v134'1213 uv1213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.024+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ec910 con 0x555b24133490 2024-04-26T21:46:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.025+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v134'1214 uv1214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.025+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ec910 con 0x555b24133490 2024-04-26T21:46:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.027+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 26 ==== osd_op_reply(26 603.00000000 [create] v134'1215 uv1189 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.027+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241ec910 con 0x555b24133490 2024-04-26T21:46:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.028+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 27 ==== osd_op_reply(27 603.00000000 [delete] v134'1216 uv1216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.028+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ec910 con 0x555b24133490 2024-04-26T21:46:57.029 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-26T21:46:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.029+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 28 ==== osd_op_reply(28 603.00000000 [create] v134'1217 uv1217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.029+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ed920 con 0x555b24133490 2024-04-26T21:46:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.030+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v134'1218 uv1218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.030+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ed920 con 0x555b24133490 2024-04-26T21:46:57.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.031+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v134'1219 uv1219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.031+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ec910 con 0x555b24133490 2024-04-26T21:46:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.032+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.1 v2:172.21.15.40:6800/299622497 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v134'1220 uv1220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b24133490 2024-04-26T21:46:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.032+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x555b241ecf90 0x555b241ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.032+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ef8c0 con 0x555b241ecf90 2024-04-26T21:46:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.033+0000 7f4d48122640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x555b241ecf90 0x555b241ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.033+0000 7f4d48122640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x555b241ecf90 0x555b241ef380 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.035+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== osd_op_reply(32 1.00000000.inode [create] v134'867 uv850 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.035+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241efc60 con 0x555b241ecf90 2024-04-26T21:46:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.036+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== osd_op_reply(33 1.00000000.inode [delete] v134'868 uv868 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.036+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f0000 con 0x555b241ecf90 2024-04-26T21:46:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.038+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 3 ==== osd_op_reply(34 1.00000000.inode [create] v134'869 uv869 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.038 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-26T21:46:57.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.038+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f0850 con 0x555b241ecf90 2024-04-26T21:46:57.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.039+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v134'870 uv870 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.039+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f0bf0 con 0x555b241ecf90 2024-04-26T21:46:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.040+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v134'871 uv871 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.040+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f0f90 con 0x555b241ecf90 2024-04-26T21:46:57.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.042+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 6 ==== osd_op_reply(37 200.00000000 [create] v134'872 uv855 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.042+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241f1330 con 0x555b241ecf90 2024-04-26T21:46:57.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.043+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 7 ==== osd_op_reply(38 200.00000000 [delete] v134'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.043+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.045 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-26T21:46:57.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.044+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 8 ==== osd_op_reply(39 200.00000000 [create] v134'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.045+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.046+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v134'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.046+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.047+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v134'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.047+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.049+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 11 ==== osd_op_reply(42 200.00000001 [create] v134'877 uv860 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.049+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.050+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 12 ==== osd_op_reply(43 200.00000001 [delete] v134'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.050+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.051+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 13 ==== osd_op_reply(44 200.00000001 [create] v134'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.052 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-26T21:46:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.051+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18075 in=18075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f5d90 con 0x555b241ecf90 2024-04-26T21:46:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.053+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 14 ==== osd_op_reply(45 200.00000001 [write 0~18075] v134'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.053+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f5d90 con 0x555b241ecf90 2024-04-26T21:46:57.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.054+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v134'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.054+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.055+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 16 ==== osd_op_reply(47 606.00000000 [create] v134'882 uv866 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.055+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.056+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 17 ==== osd_op_reply(48 606.00000000 [delete] v134'883 uv883 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.056+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.058 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-26T21:46:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.058+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 18 ==== osd_op_reply(49 606.00000000 [create] v134'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.058+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f26e0 con 0x555b241ecf90 2024-04-26T21:46:57.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.059+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v134'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.059+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f26e0 con 0x555b241ecf90 2024-04-26T21:46:57.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.060+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v134'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.060+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f16d0 con 0x555b241ecf90 2024-04-26T21:46:57.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.061+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.0 v2:172.21.15.40:6808/628259048 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v134'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241ecf90 2024-04-26T21:46:57.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.061+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x555b241f1be0 0x555b241f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.061+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f4510 con 0x555b241f1be0 2024-04-26T21:46:57.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.062+0000 7f4d47921640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x555b241f1be0 0x555b241f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.062+0000 7f4d47921640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x555b241f1be0 0x555b241f3fd0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.063+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(53 607.00000000 [create] v134'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.063+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241f48b0 con 0x555b241f1be0 2024-04-26T21:46:57.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.065+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(54 607.00000000 [delete] v134'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.065+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f4d50 con 0x555b241f1be0 2024-04-26T21:46:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.066+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(55 607.00000000 [create] v134'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.066 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-26T21:46:57.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.066+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f0440 con 0x555b241f1be0 2024-04-26T21:46:57.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.067+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v134'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.067+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f6650 con 0x555b241f1be0 2024-04-26T21:46:57.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.068+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v134'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.068+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f55f0 con 0x555b241f1be0 2024-04-26T21:46:57.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.069+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v134'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.069+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b241f60d0 0x555b241f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.069+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f8fd0 con 0x555b241f60d0 2024-04-26T21:46:57.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.069+0000 7f4d48923640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b241f60d0 0x555b241f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.070+0000 7f4d48923640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b241f60d0 0x555b241f8a00 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.071+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v134'181 uv180 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.071+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241f93c0 con 0x555b241f60d0 2024-04-26T21:46:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.072+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v134'182 uv182 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.073+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f93c0 con 0x555b241f60d0 2024-04-26T21:46:57.074 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-26T21:46:57.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.074+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v134'183 uv183 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.074+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241f93c0 con 0x555b241f60d0 2024-04-26T21:46:57.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.075+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v134'184 uv184 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.075+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x555b241f9cc0 0x555b241fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.075+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc5f0 con 0x555b241f9cc0 2024-04-26T21:46:57.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.075+0000 7f4d48122640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x555b241f9cc0 0x555b241fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.076+0000 7f4d48122640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x555b241f9cc0 0x555b241fc0b0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.077+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(63 608.00000000 [create] v134'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.077+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241fc5f0 con 0x555b241f9cc0 2024-04-26T21:46:57.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.078+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(64 608.00000000 [delete] v134'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.079+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.080 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-26T21:46:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.080+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(65 608.00000000 [create] v134'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.080+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fd9f0 con 0x555b241f9cc0 2024-04-26T21:46:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.081+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v134'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.081+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fd9f0 con 0x555b241f9cc0 2024-04-26T21:46:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.082+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v134'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.082+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.083+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v134'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.083+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.085+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 5 ==== osd_op_reply(69 604.00000000 [create] v134'426 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.085+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.086+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 6 ==== osd_op_reply(70 604.00000000 [delete] v134'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.086+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.087+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 7 ==== osd_op_reply(71 604.00000000 [create] v134'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.088 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-26T21:46:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.087+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fd9f0 con 0x555b241f60d0 2024-04-26T21:46:57.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.088+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v134'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.089+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fd9f0 con 0x555b241f60d0 2024-04-26T21:46:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.090+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v134'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.090+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.091+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v134'431 uv431 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.091+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.092+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 11 ==== osd_op_reply(75 500.00000000 [create] v134'432 uv425 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.092+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.093+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 12 ==== osd_op_reply(76 500.00000000 [delete] v134'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.093+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.094+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 13 ==== osd_op_reply(77 500.00000000 [create] v134'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.095 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-26T21:46:57.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.094+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.095+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v134'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.095+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.096+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v134'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.096+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.097+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 16 ==== osd_op_reply(80 mds_snaptable [create] v134'193 uv192 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.097+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.098+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 17 ==== osd_op_reply(81 mds_snaptable [delete] v134'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.098+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.100 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-26T21:46:57.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.099+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 18 ==== osd_op_reply(82 mds_snaptable [create] v134'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.099+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.101+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 19 ==== osd_op_reply(83 mds_snaptable [write 0~70] v134'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.101+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f60d0 2024-04-26T21:46:57.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.102+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 <== osd.4 v2:172.21.15.184:6800/2829090873 20 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v134'197 uv197 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4d40007650 con 0x555b241f60d0 2024-04-26T21:46:57.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.102+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.103+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 7 ==== osd_op_reply(85 605.00000000 [create] v134'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.103+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.104+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 8 ==== osd_op_reply(86 605.00000000 [delete] v134'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.104+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.105 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-26T21:46:57.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.105+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 9 ==== osd_op_reply(87 605.00000000 [create] v134'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.105+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fd9f0 con 0x555b241f9cc0 2024-04-26T21:46:57.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.106+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 10 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v134'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.106+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fd9f0 con 0x555b241f9cc0 2024-04-26T21:46:57.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.107+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 11 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v134'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.107+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.108+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 12 ==== osd_op_reply(90 605.00000000 [omap-set-header] v134'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.108+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.109+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 13 ==== osd_op_reply(91 mds0_inotable [create] v134'386 uv380 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.109+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.110+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 14 ==== osd_op_reply(92 mds0_inotable [delete] v134'387 uv387 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.110+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.112 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-26T21:46:57.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.111+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 15 ==== osd_op_reply(93 mds0_inotable [create] v134'388 uv388 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.111+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.112+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 16 ==== osd_op_reply(94 mds0_inotable [write 0~34] v134'389 uv389 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.112+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.113+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 17 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v134'390 uv390 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.113+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.114+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 18 ==== osd_op_reply(96 100.00000000 [create] v134'391 uv385 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.114+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.115+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 19 ==== osd_op_reply(97 100.00000000 [delete] v134'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.115+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.117 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-26T21:46:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.116+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 20 ==== osd_op_reply(98 100.00000000 [create] v134'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.116+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241fc9e0 con 0x555b241f9cc0 2024-04-26T21:46:57.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.117+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 21 ==== osd_op_reply(99 100.00000000 [omap-set-header] v134'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.117+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ff7f0 con 0x555b241f9cc0 2024-04-26T21:46:57.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.118+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 22 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v134'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.118+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x555b241fd510 0x555b241ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.118+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ff900 con 0x555b241fd510 2024-04-26T21:46:57.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.119+0000 7f4d47921640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x555b241fd510 0x555b241ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.119+0000 7f4d47921640 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x555b241fd510 0x555b241ffe70 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.120+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(101 mds0_sessionmap [create] v134'172 uv171 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241fd510 2024-04-26T21:46:57.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.120+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241ff900 con 0x555b241fd510 2024-04-26T21:46:57.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.121+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v134'173 uv173 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241fd510 2024-04-26T21:46:57.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.121+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ff980 con 0x555b241fd510 2024-04-26T21:46:57.122 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-26T21:46:57.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.122+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.5 v2:172.21.15.184:6808/304953510 3 ==== osd_op_reply(103 mds0_sessionmap [create] v134'174 uv174 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241fd510 2024-04-26T21:46:57.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.122+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241fd510 2024-04-26T21:46:57.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.123+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.5 v2:172.21.15.184:6808/304953510 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v134'175 uv175 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241fd510 2024-04-26T21:46:57.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.123+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f9cc0 2024-04-26T21:46:57.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.124+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 23 ==== osd_op_reply(105 609.00000000 [create] v134'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.124+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f9cc0 2024-04-26T21:46:57.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.126+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 24 ==== osd_op_reply(106 609.00000000 [delete] v134'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.126+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f9cc0 2024-04-26T21:46:57.127 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-26T21:46:57.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.127+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 25 ==== osd_op_reply(107 609.00000000 [create] v134'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.127+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f9cc0 2024-04-26T21:46:57.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.128+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 26 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v134'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.128+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f9cc0 2024-04-26T21:46:57.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.129+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 27 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v134'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.129+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f9cc0 2024-04-26T21:46:57.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.130+0000 7f4d48122640 1 -- 172.21.15.40:0/756504965 <== osd.7 v2:172.21.15.184:6809/2389595522 28 ==== osd_op_reply(110 609.00000000 [omap-set-header] v134'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d38054b60 con 0x555b241f9cc0 2024-04-26T21:46:57.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.130+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.131+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 7 ==== osd_op_reply(111 400.00000000 [create] v134'196 uv195 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.131+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.132+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 8 ==== osd_op_reply(112 400.00000000 [delete] v134'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.133+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.134+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 9 ==== osd_op_reply(113 400.00000000 [create] v134'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.135 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-26T21:46:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.134+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.135+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 10 ==== osd_op_reply(114 400.00000000 [write 0~22] v134'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.135+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.136+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 11 ==== osd_op_reply(115 400.00000000 [omap-set-header] v134'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.136+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.138+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 12 ==== osd_op_reply(116 100.00000000.inode [create] v134'480 uv472 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.138+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.139+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 13 ==== osd_op_reply(117 100.00000000.inode [delete] v134'481 uv481 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.139+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.140+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 14 ==== osd_op_reply(118 100.00000000.inode [create] v134'482 uv482 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.140 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-26T21:46:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.140+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.141+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 15 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v134'483 uv483 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.141+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.142+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 16 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v134'484 uv484 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.142+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.143+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 17 ==== osd_op_reply(121 1.00000000 [create] v134'485 uv479 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.143+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.144+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 18 ==== osd_op_reply(122 1.00000000 [delete] v134'486 uv486 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.144+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa00 con 0x555b241f1be0 2024-04-26T21:46:57.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.145+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 19 ==== osd_op_reply(123 1.00000000 [create] v134'487 uv487 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.146 INFO:teuthology.orchestra.run.smithi040.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-26T21:46:57.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.145+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa20 con 0x555b241f1be0 2024-04-26T21:46:57.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.146+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 20 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v134'488 uv488 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.146+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa20 con 0x555b241f1be0 2024-04-26T21:46:57.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.147+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 21 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v134'489 uv489 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.147+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b241ffa20 con 0x555b241f1be0 2024-04-26T21:46:57.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.148+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 22 ==== osd_op_reply(126 1.00000000 [omap-set-header] v134'490 uv490 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.148+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x555b242003b0 con 0x555b241f1be0 2024-04-26T21:46:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d47921640 1 -- 172.21.15.40:0/756504965 <== osd.6 v2:172.21.15.184:6824/533952038 23 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v134'491 uv491 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4d3c05c040 con 0x555b241f1be0 2024-04-26T21:46:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x555b241ecf90 msgr2=0x555b241ef380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x555b241ecf90 0x555b241ef380 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x555b24133490 msgr2=0x555b241410f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x555b24133490 0x555b241410f0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b241f60d0 msgr2=0x555b241f8a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x555b241f60d0 0x555b241f8a00 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x555b241fd510 msgr2=0x555b241ffe70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x555b241fd510 0x555b241ffe70 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x555b241f1be0 msgr2=0x555b241f3fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x555b241f1be0 0x555b241f3fd0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x555b241f9cc0 msgr2=0x555b241fc0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x555b241f9cc0 0x555b241fc0b0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 reap_dead start 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d48923640 1 -- 172.21.15.40:0/756504965 reap_dead start 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d3406b2d0 msgr2=0x7f4d3406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d3406b2d0 0x7f4d3406d790 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c057040 tx=0x7f4d3c051000 comp rx=0 tx=0).stop 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240a0650 msgr2=0x555b241329e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240a0650 0x555b241329e0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c450 tx=0x7f4d3800c910 comp rx=0 tx=0).stop 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 shutdown_connections 2024-04-26T21:46:57.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d3406b2d0 0x7f4d3406d790 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 --2- 172.21.15.40:0/756504965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555b240a0650 0x555b241329e0 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 >> 172.21.15.40:0/756504965 conn(0x555b2409a8a0 msgr2=0x555b2406df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 shutdown_connections 2024-04-26T21:46:57.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.149+0000 7f4d4a628f40 1 -- 172.21.15.40:0/756504965 wait complete. 2024-04-26T21:46:57.155 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 1.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-26T21:46:57.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f7443f40 1 Processor -- start 2024-04-26T21:46:57.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f7443f40 1 -- start start 2024-04-26T21:46:57.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f7443f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 0x563a7f932200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f7443f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9c4d60 0x563a7f94c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f7443f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9e2610 0x563a7f9c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f7443f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563a7f8aee40 con 0x563a7f9c4d60 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f7443f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a7f8b2e80 con 0x563a7f94df60 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f7443f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563a7f9c5140 con 0x563a7f9e2610 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.198+0000 7f04f473c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9c4d60 0x563a7f94c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f473c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9c4d60 0x563a7f94c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59338/0 (socket says 172.21.15.40:59338) 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f473c640 1 -- 172.21.15.40:0/1759704955 learned_addr learned my addr 172.21.15.40:0/1759704955 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f4f3d640 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 0x563a7f932200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f573e640 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9e2610 0x563a7f9c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f473c640 1 -- 172.21.15.40:0/1759704955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9e2610 msgr2=0x563a7f9c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f473c640 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9e2610 0x563a7f9c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f473c640 1 -- 172.21.15.40:0/1759704955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 msgr2=0x563a7f932200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f473c640 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 0x563a7f932200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f473c640 1 -- 172.21.15.40:0/1759704955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a7f94d510 con 0x563a7f9c4d60 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f473c640 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9c4d60 0x563a7f94c3a0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f04dc001e00 tx=0x7f04dc00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df870b30b7e37d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f373a640 1 -- 172.21.15.40:0/1759704955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc00be40 con 0x563a7f9c4d60 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f373a640 1 -- 172.21.15.40:0/1759704955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc00f400 con 0x563a7f9c4d60 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.199+0000 7f04f373a640 1 -- 172.21.15.40:0/1759704955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc00f7d0 con 0x563a7f9c4d60 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 -- 172.21.15.40:0/1759704955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9c4d60 msgr2=0x563a7f94c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9c4d60 0x563a7f94c3a0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f04dc001e00 tx=0x7f04dc00b670 comp rx=0 tx=0).stop 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 -- 172.21.15.40:0/1759704955 shutdown_connections 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9e2610 0x563a7f9c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9c4d60 0x563a7f94c3a0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 --2- 172.21.15.40:0/1759704955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 0x563a7f932200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 -- 172.21.15.40:0/1759704955 >> 172.21.15.40:0/1759704955 conn(0x563a7f905ab0 msgr2=0x563a7f904f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 -- 172.21.15.40:0/1759704955 shutdown_connections 2024-04-26T21:46:57.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 -- 172.21.15.40:0/1759704955 wait complete. 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 Processor -- start 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.200+0000 7f04f7443f40 1 -- start start 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f7443f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 0x563a7f9bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f7443f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9c4d60 0x563a7f9bc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f7443f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9e2610 0x563a7f94b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f7443f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563a7f94a6d0 con 0x563a7f9e2610 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f7443f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a7f9c42c0 con 0x563a7f94df60 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f7443f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563a7f905690 con 0x563a7f9c4d60 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9e2610 0x563a7f94b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9e2610 0x563a7f94b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59354/0 (socket says 172.21.15.40:59354) 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 -- 172.21.15.40:0/1959572258 learned_addr learned my addr 172.21.15.40:0/1959572258 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f473c640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9c4d60 0x563a7f9bc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f4f3d640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 0x563a7f9bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 -- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9c4d60 msgr2=0x563a7f9bc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563a7f9c4d60 0x563a7f9bc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 -- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 msgr2=0x563a7f9bcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a7f94df60 0x563a7f9bcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 -- 172.21.15.40:0/1959572258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a7f9bee00 con 0x563a7f9e2610 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.201+0000 7f04f573e640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9e2610 0x563a7f94b080 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003f00 tx=0x7f04e4001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.202+0000 7f04f2738640 1 -- 172.21.15.40:0/1959572258 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e40123f0 con 0x563a7f9e2610 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.202+0000 7f04f2738640 1 -- 172.21.15.40:0/1959572258 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e4017070 con 0x563a7f9e2610 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.202+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563a7f9bd000 con 0x563a7f9e2610 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.202+0000 7f04f2738640 1 -- 172.21.15.40:0/1959572258 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e40127f0 con 0x563a7f9e2610 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.202+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a7f9df3e0 con 0x563a7f9e2610 2024-04-26T21:46:57.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.203+0000 7f04f2738640 1 -- 172.21.15.40:0/1959572258 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04e4012990 con 0x563a7f9e2610 2024-04-26T21:46:57.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.204+0000 7f04f2738640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04d406b2d0 0x7f04d406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.204+0000 7f04f2738640 1 -- 172.21.15.40:0/1959572258 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7f04e408b4d0 con 0x563a7f9e2610 2024-04-26T21:46:57.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.204+0000 7f04f4f3d640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04d406b2d0 0x7f04d406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.204+0000 7f04f7443f40 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a7f9c27f0 0x563a7f9e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.204+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563a7f9e06e0 con 0x563a7f9c27f0 2024-04-26T21:46:57.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.204+0000 7f04f473c640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a7f9c27f0 0x563a7f9e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.204+0000 7f04f4f3d640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04d406b2d0 0x7f04d406d790 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f04ec057000 tx=0x7f04ec052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.204+0000 7f04f473c640 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a7f9c27f0 0x563a7f9e01a0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f473c640 1 -- 172.21.15.40:0/1959572258 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 1.00000000 [omap-set-header] v134'492 uv492 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04dc00f6b0 con 0x563a7f9c27f0 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a7f9c27f0 msgr2=0x563a7f9e01a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x563a7f9c27f0 0x563a7f9e01a0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04d406b2d0 msgr2=0x7f04d406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f04d406b2d0 0x7f04d406d790 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f04ec057000 tx=0x7f04ec052000 comp rx=0 tx=0).stop 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9e2610 msgr2=0x563a7f94b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 --2- 172.21.15.40:0/1959572258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a7f9e2610 0x563a7f94b080 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003f00 tx=0x7f04e4001830 comp rx=0 tx=0).stop 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f573e640 1 -- 172.21.15.40:0/1959572258 reap_dead start 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 shutdown_connections 2024-04-26T21:46:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 >> 172.21.15.40:0/1959572258 conn(0x563a7f905ab0 msgr2=0x563a7f904f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 shutdown_connections 2024-04-26T21:46:57.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.206+0000 7f04f7443f40 1 -- 172.21.15.40:0/1959572258 wait complete. 2024-04-26T21:46:57.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:46:57.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.357+0000 7f859c447640 1 Processor -- start 2024-04-26T21:46:57.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.357+0000 7f859c447640 1 -- start start 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859c447640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f859415f170 0x7f85940d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859c447640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f859415f550 0x7f85940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859c447640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85940d1040 con 0x7f85940d34b0 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859c447640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f85940c9430 con 0x7f859415f170 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859c447640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f85940c9680 con 0x7f859415f550 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859a1e3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f859415f170 0x7f85940d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859a1e3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f859415f170 0x7f85940d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59360/0 (socket says 172.21.15.40:59360) 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859a1e3640 1 -- 172.21.15.40:0/2962048696 learned_addr learned my addr 172.21.15.40:0/2962048696 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f85999e2640 1 --2- 172.21.15.40:0/2962048696 >> v2:172.21.15.184:3300/0 conn(0x7f859415f550 0x7f85940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859a1e3640 1 -- 172.21.15.40:0/2962048696 >> v2:172.21.15.184:3300/0 conn(0x7f859415f550 msgr2=0x7f85940d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859a1e3640 1 --2- 172.21.15.40:0/2962048696 >> v2:172.21.15.184:3300/0 conn(0x7f859415f550 0x7f85940d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859a1e3640 1 -- 172.21.15.40:0/2962048696 >> v1:172.21.15.184:6789/0 conn(0x7f85940d34b0 legacy=0x7f85940cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.358+0000 7f859a1e3640 1 -- 172.21.15.40:0/2962048696 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940c9960 con 0x7f859415f170 2024-04-26T21:46:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.359+0000 7f859a1e3640 1 --2- 172.21.15.40:0/2962048696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f85940d5a80 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f8590009870 tx=0x7f859002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ce85fa30446195 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.359+0000 7f85991e1640 1 -- 172.21.15.40:0/2962048696 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590037070 con 0x7f859415f170 2024-04-26T21:46:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.359+0000 7f85991e1640 1 -- 172.21.15.40:0/2962048696 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8590032440 con 0x7f859415f170 2024-04-26T21:46:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.359+0000 7f85991e1640 1 -- 172.21.15.40:0/2962048696 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590032810 con 0x7f859415f170 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.360+0000 7f859c447640 1 -- 172.21.15.40:0/2962048696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 msgr2=0x7f85940d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.360+0000 7f859c447640 1 --2- 172.21.15.40:0/2962048696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f85940d5a80 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f8590009870 tx=0x7f859002a270 comp rx=0 tx=0).stop 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.360+0000 7f859c447640 1 -- 172.21.15.40:0/2962048696 shutdown_connections 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.360+0000 7f859c447640 1 --2- 172.21.15.40:0/2962048696 >> v2:172.21.15.184:3300/0 conn(0x7f859415f550 0x7f85940d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.360+0000 7f859c447640 1 --2- 172.21.15.40:0/2962048696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f85940d5a80 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.360+0000 7f859c447640 1 -- 172.21.15.40:0/2962048696 >> 172.21.15.40:0/2962048696 conn(0x7f85940d4f60 msgr2=0x7f85940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.360+0000 7f859c447640 1 -- 172.21.15.40:0/2962048696 shutdown_connections 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.360+0000 7f859c447640 1 -- 172.21.15.40:0/2962048696 wait complete. 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.361+0000 7f859c447640 1 Processor -- start 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.361+0000 7f859c447640 1 -- start start 2024-04-26T21:46:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f859c447640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d34b0 0x7f859414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f859c447640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f8594159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f859c447640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859415a3f0 0x7f859415b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f859c447640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85940cb5d0 con 0x7f859415f170 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f859c447640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85940cb450 con 0x7f859415a3f0 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f859c447640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85940cb750 con 0x7f85940d34b0 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f8594159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f8594159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59366/0 (socket says 172.21.15.40:59366) 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 -- 172.21.15.40:0/1690600886 learned_addr learned my addr 172.21.15.40:0/1690600886 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f859a9e4640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859415a3f0 0x7f859415b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f859a1e3640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d34b0 0x7f859414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 -- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d34b0 msgr2=0x7f859414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d34b0 0x7f859414f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 -- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859415a3f0 msgr2=0x7f859415b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859415a3f0 0x7f859415b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 -- 172.21.15.40:0/1690600886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8594158e10 con 0x7f859415f170 2024-04-26T21:46:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.362+0000 7f85999e2640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f8594159eb0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f858400d5a0 tx=0x7f858400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.363+0000 7f858affd640 1 -- 172.21.15.40:0/1690600886 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584019070 con 0x7f859415f170 2024-04-26T21:46:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.363+0000 7f858affd640 1 -- 172.21.15.40:0/1690600886 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8584004240 con 0x7f859415f170 2024-04-26T21:46:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.363+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85941590f0 con 0x7f859415f170 2024-04-26T21:46:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.363+0000 7f858affd640 1 -- 172.21.15.40:0/1690600886 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584003610 con 0x7f859415f170 2024-04-26T21:46:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.363+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85941596b0 con 0x7f859415f170 2024-04-26T21:46:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.364+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85940694f0 con 0x7f859415f170 2024-04-26T21:46:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.364+0000 7f858affd640 1 -- 172.21.15.40:0/1690600886 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f858401c070 con 0x7f859415f170 2024-04-26T21:46:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.365+0000 7f858affd640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f857806c8f0 0x7f857806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.365+0000 7f858affd640 1 -- 172.21.15.40:0/1690600886 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7f858408c2d0 con 0x7f859415f170 2024-04-26T21:46:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.365+0000 7f859a1e3640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f857806c8f0 0x7f857806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.365+0000 7f859a1e3640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f857806c8f0 0x7f857806edb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8590034000 tx=0x7f859002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.369+0000 7f858affd640 1 -- 172.21.15.40:0/1690600886 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85840591d0 con 0x7f859415f170 2024-04-26T21:46:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.595+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8594069c20 con 0x7f859415f170 2024-04-26T21:46:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.933+0000 7f858affd640 1 -- 172.21.15.40:0/1690600886 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v344) ==== 162+0+0 (secure 0 0 0) 0x7f858405d4d0 con 0x7f859415f170 2024-04-26T21:46:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:46:57.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f857806c8f0 msgr2=0x7f857806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f857806c8f0 0x7f857806edb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8590034000 tx=0x7f859002e000 comp rx=0 tx=0).stop 2024-04-26T21:46:57.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 msgr2=0x7f8594159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:57.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f8594159eb0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f858400d5a0 tx=0x7f858400da60 comp rx=0 tx=0).stop 2024-04-26T21:46:57.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 shutdown_connections 2024-04-26T21:46:57.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f857806c8f0 0x7f857806edb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859415a3f0 0x7f859415b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859415f170 0x7f8594159eb0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 --2- 172.21.15.40:0/1690600886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85940d34b0 0x7f859414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:57.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 >> 172.21.15.40:0/1690600886 conn(0x7f85940d4f60 msgr2=0x7f85940c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:57.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 shutdown_connections 2024-04-26T21:46:57.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:57.935+0000 7f859c447640 1 -- 172.21.15.40:0/1690600886 wait complete. 2024-04-26T21:46:57.950 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:46:58.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.051+0000 7fa5a9f11640 1 Processor -- start 2024-04-26T21:46:58.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.051+0000 7fa5a9f11640 1 -- start start 2024-04-26T21:46:58.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.052+0000 7fa5a9f11640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5a40d0070 0x7fa5a40d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.052+0000 7fa5a9f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a406d560 0x7fa5a406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.052+0000 7fa5a9f11640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5a406e0a0 0x7fa5a4162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.052+0000 7fa5a9f11640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a4162d00 con 0x7fa5a406e0a0 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.052+0000 7fa5a9f11640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a40d2d10 con 0x7fa5a40d0070 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.052+0000 7fa5a9f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40d2420 con 0x7fa5a406d560 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.052+0000 7fa5a3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5a406e0a0 0x7fa5a4162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5a406e0a0 0x7fa5a4162f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59380/0 (socket says 172.21.15.40:59380) 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a3fff640 1 -- 172.21.15.40:0/4122087105 learned_addr learned my addr 172.21.15.40:0/4122087105 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a2ffd640 1 --2- 172.21.15.40:0/4122087105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a406d560 0x7fa5a406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a37fe640 1 --2- 172.21.15.40:0/4122087105 >> v2:172.21.15.184:3300/0 conn(0x7fa5a40d0070 0x7fa5a40d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a3fff640 1 -- 172.21.15.40:0/4122087105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a406d560 msgr2=0x7fa5a406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a3fff640 1 --2- 172.21.15.40:0/4122087105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a406d560 0x7fa5a406db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a3fff640 1 -- 172.21.15.40:0/4122087105 >> v2:172.21.15.184:3300/0 conn(0x7fa5a40d0070 msgr2=0x7fa5a40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a3fff640 1 --2- 172.21.15.40:0/4122087105 >> v2:172.21.15.184:3300/0 conn(0x7fa5a40d0070 0x7fa5a40d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a3fff640 1 -- 172.21.15.40:0/4122087105 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40cd6b0 con 0x7fa5a406e0a0 2024-04-26T21:46:58.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.053+0000 7fa5a37fe640 1 --2- 172.21.15.40:0/4122087105 >> v2:172.21.15.184:3300/0 conn(0x7fa5a40d0070 0x7fa5a40d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:46:58.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a3fff640 1 --2- 172.21.15.40:0/4122087105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406e0a0 0x7fa5a4162f10 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40731d0 tx=0x7fa59c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee273876e9575a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a27fc640 1 -- 172.21.15.40:0/4122087105 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c036070 con 0x7fa5a406e0a0 2024-04-26T21:46:58.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a27fc640 1 -- 172.21.15.40:0/4122087105 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c031600 con 0x7fa5a406e0a0 2024-04-26T21:46:58.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a27fc640 1 -- 172.21.15.40:0/4122087105 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0319d0 con 0x7fa5a406e0a0 2024-04-26T21:46:58.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 -- 172.21.15.40:0/4122087105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406e0a0 msgr2=0x7fa5a4162f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/4122087105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406e0a0 0x7fa5a4162f10 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40731d0 tx=0x7fa59c02a730 comp rx=0 tx=0).stop 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 -- 172.21.15.40:0/4122087105 shutdown_connections 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/4122087105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406e0a0 0x7fa5a4162f10 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/4122087105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a406d560 0x7fa5a406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/4122087105 >> v2:172.21.15.184:3300/0 conn(0x7fa5a40d0070 0x7fa5a40d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 -- 172.21.15.40:0/4122087105 >> 172.21.15.40:0/4122087105 conn(0x7fa5a40d2ec0 msgr2=0x7fa5a40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 -- 172.21.15.40:0/4122087105 shutdown_connections 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.054+0000 7fa5a9f11640 1 -- 172.21.15.40:0/4122087105 wait complete. 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.055+0000 7fa5a9f11640 1 Processor -- start 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a9f11640 1 -- start start 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a9f11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406d560 0x7fa5a40c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a9f11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a406e0a0 0x7fa5a4159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a9f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a40d0070 0x7fa5a4070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a9f11640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4163450 con 0x7fa5a406d560 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a9f11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40cdc70 con 0x7fa5a406e0a0 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a9f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a41635d0 con 0x7fa5a40d0070 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406d560 0x7fa5a40c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406d560 0x7fa5a40c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59384/0 (socket says 172.21.15.40:59384) 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.056+0000 7fa5a37fe640 1 -- 172.21.15.40:0/426882283 learned_addr learned my addr 172.21.15.40:0/426882283 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a3fff640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a40d0070 0x7fa5a4070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a2ffd640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a406e0a0 0x7fa5a4159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a37fe640 1 -- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a40d0070 msgr2=0x7fa5a4070990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a37fe640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a40d0070 0x7fa5a4070990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a37fe640 1 -- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a406e0a0 msgr2=0x7fa5a4159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a37fe640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a406e0a0 0x7fa5a4159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a37fe640 1 -- 172.21.15.40:0/426882283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a415aa30 con 0x7fa5a406d560 2024-04-26T21:46:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a37fe640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406d560 0x7fa5a40c7440 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fa59400c820 tx=0x7fa59400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa58bfff640 1 -- 172.21.15.40:0/426882283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594015070 con 0x7fa5a406d560 2024-04-26T21:46:58.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa58bfff640 1 -- 172.21.15.40:0/426882283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594007a30 con 0x7fa5a406d560 2024-04-26T21:46:58.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa58bfff640 1 -- 172.21.15.40:0/426882283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594007de0 con 0x7fa5a406d560 2024-04-26T21:46:58.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a41f5160 con 0x7fa5a406d560 2024-04-26T21:46:58.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.057+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a40c6e90 con 0x7fa5a406d560 2024-04-26T21:46:58.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.058+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a40694f0 con 0x7fa5a406d560 2024-04-26T21:46:58.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.059+0000 7fa58bfff640 1 -- 172.21.15.40:0/426882283 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa594018070 con 0x7fa5a406d560 2024-04-26T21:46:58.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.059+0000 7fa58bfff640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa59006c920 0x7fa59006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:58.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.059+0000 7fa58bfff640 1 -- 172.21.15.40:0/426882283 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9571+0+0 (secure 0 0 0) 0x7fa59408c770 con 0x7fa5a406d560 2024-04-26T21:46:58.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.059+0000 7fa5a2ffd640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa59006c920 0x7fa59006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:58.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.060+0000 7fa5a2ffd640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa59006c920 0x7fa59006ede0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa5a406f4f0 tx=0x7fa598008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.064+0000 7fa58bfff640 1 -- 172.21.15.40:0/426882283 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5a415aa30 con 0x7fa5a406d560 2024-04-26T21:46:58.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.287+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5a40d0470 con 0x7fa5a406d560 2024-04-26T21:46:58.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.288+0000 7fa58bfff640 1 -- 172.21.15.40:0/426882283 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 345 v345) ==== 95+0+5190 (secure 0 0 0) 0x7fa594059670 con 0x7fa5a406d560 2024-04-26T21:46:58.289 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:46:58.289 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":345,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":345,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:46:57.936186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11493":{"gid":11493,"name":"c","rank":0,"incarnation":345,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6833/933356185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":933356185},{"type":"v1","addr":"172.21.15.40:6833","nonce":933356185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:46:58.290 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 345 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa59006c920 msgr2=0x7fa59006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa59006c920 0x7fa59006ede0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa5a406f4f0 tx=0x7fa598008040 comp rx=0 tx=0).stop 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406d560 msgr2=0x7fa5a40c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406d560 0x7fa5a40c7440 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fa59400c820 tx=0x7fa59400cce0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 shutdown_connections 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa59006c920 0x7fa59006ede0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a40d0070 0x7fa5a4070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a406e0a0 0x7fa5a4159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 --2- 172.21.15.40:0/426882283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5a406d560 0x7fa5a40c7440 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:58.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 >> 172.21.15.40:0/426882283 conn(0x7fa5a40d2ec0 msgr2=0x7fa5a4160cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:58.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 shutdown_connections 2024-04-26T21:46:58.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:46:58.290+0000 7fa5a9f11640 1 -- 172.21.15.40:0/426882283 wait complete. 2024-04-26T21:46:58.302 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:47:00.953 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:47:00.952+0000 7f8b7e680640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-26T21:47:00.953 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:47:00.952+0000 7f8b7e680640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-04-26T21:47:00.954 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:47:00.953+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:47:00.954 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:47:00.953+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-26T21:47:00.961 INFO:tasks.ceph.mds.c.smithi040.stderr: -17> 2024-04-26T21:47:00.952+0000 7f8b7e680640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-26T21:47:00.961 INFO:tasks.ceph.mds.c.smithi040.stderr: -14> 2024-04-26T21:47:00.952+0000 7f8b7e680640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-04-26T21:47:00.988 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:47:03.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:03.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.415+0000 7f4d1bfda640 1 Processor -- start 2024-04-26T21:47:03.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.416+0000 7f4d1bfda640 1 -- start start 2024-04-26T21:47:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.416+0000 7f4d1bfda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d140d06f0 0x7f4d140d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.416+0000 7f4d1bfda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d140d0ba0 con 0x7f4d1415f3a0 2024-04-26T21:47:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.416+0000 7f4d1bfda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d140d0e20 con 0x7f4d140b2850 2024-04-26T21:47:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.416+0000 7f4d1bfda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4d140b1d70 con 0x7f4d140d06f0 2024-04-26T21:47:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.417+0000 7f4d19575640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d140d06f0 0x7f4d140d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.417+0000 7f4d19d76640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4d1415f3a0 0x7f4d140d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34510/0 (socket says 172.21.15.40:34510) 2024-04-26T21:47:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.417+0000 7f4d19d76640 1 -- 172.21.15.40:0/204205020 learned_addr learned my addr 172.21.15.40:0/204205020 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.417+0000 7f4d19575640 1 -- 172.21.15.40:0/204205020 >> v1:172.21.15.184:6789/0 conn(0x7f4d140b2850 legacy=0x7f4d140b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.417+0000 7f4d19575640 1 -- 172.21.15.40:0/204205020 >> v1:172.21.15.40:6789/0 conn(0x7f4d1415f3a0 legacy=0x7f4d140d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.417+0000 7f4d18d74640 1 -- 172.21.15.40:0/204205020 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2988504681 0 0) 0x7f4d0c001110 con 0x7f4d1415f3a0 2024-04-26T21:47:03.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.417+0000 7f4d19575640 1 -- 172.21.15.40:0/204205020 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d140b2050 con 0x7f4d140d06f0 2024-04-26T21:47:03.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.418+0000 7f4d19575640 1 --2- 172.21.15.40:0/204205020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140d06f0 0x7f4d140d1e80 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f4d080097a0 tx=0x7f4d0802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9c85054953b4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.418+0000 7f4d18d74640 1 -- 172.21.15.40:0/204205020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08037070 con 0x7f4d140d06f0 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.418+0000 7f4d18d74640 1 -- 172.21.15.40:0/204205020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d08032440 con 0x7f4d140d06f0 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.418+0000 7f4d18d74640 1 -- 172.21.15.40:0/204205020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08032810 con 0x7f4d140d06f0 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.418+0000 7f4d1bfda640 1 -- 172.21.15.40:0/204205020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140d06f0 msgr2=0x7f4d140d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.418+0000 7f4d1bfda640 1 --2- 172.21.15.40:0/204205020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140d06f0 0x7f4d140d1e80 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f4d080097a0 tx=0x7f4d0802a230 comp rx=0 tx=0).stop 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.419+0000 7f4d1bfda640 1 -- 172.21.15.40:0/204205020 shutdown_connections 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.419+0000 7f4d1bfda640 1 --2- 172.21.15.40:0/204205020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140d06f0 0x7f4d140d1e80 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.419+0000 7f4d1bfda640 1 -- 172.21.15.40:0/204205020 >> 172.21.15.40:0/204205020 conn(0x7f4d140d4f90 msgr2=0x7f4d140d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.419+0000 7f4d1bfda640 1 -- 172.21.15.40:0/204205020 shutdown_connections 2024-04-26T21:47:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.419+0000 7f4d1bfda640 1 -- 172.21.15.40:0/204205020 wait complete. 2024-04-26T21:47:03.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.420+0000 7f4d1bfda640 1 Processor -- start 2024-04-26T21:47:03.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.420+0000 7f4d1bfda640 1 -- start start 2024-04-26T21:47:03.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d1bfda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140b2850 0x7f4d1415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d1bfda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140d06f0 0x7f4d140ba310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d1bfda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d1415f3a0 0x7f4d14160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d1bfda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1415f900 con 0x7f4d140b2850 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d1bfda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1415f780 con 0x7f4d140d06f0 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d1bfda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d140d23c0 con 0x7f4d1415f3a0 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140b2850 0x7f4d1415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140b2850 0x7f4d1415ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46902/0 (socket says 172.21.15.40:46902) 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 -- 172.21.15.40:0/232087190 learned_addr learned my addr 172.21.15.40:0/232087190 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 -- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d1415f3a0 msgr2=0x7f4d14160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19575640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140d06f0 0x7f4d140ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d1a577640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d1415f3a0 0x7f4d14160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d1415f3a0 0x7f4d14160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 -- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140d06f0 msgr2=0x7f4d140ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140d06f0 0x7f4d140ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 -- 172.21.15.40:0/232087190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d140b87e0 con 0x7f4d140b2850 2024-04-26T21:47:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d1a577640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d1415f3a0 0x7f4d14160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19575640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140d06f0 0x7f4d140ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d19d76640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140b2850 0x7f4d1415ccd0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c3f0 tx=0x7f4d0c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d127fc640 1 -- 172.21.15.40:0/232087190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0074e0 con 0x7f4d140b2850 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.421+0000 7f4d127fc640 1 -- 172.21.15.40:0/232087190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0c0047e0 con 0x7f4d140b2850 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.422+0000 7f4d127fc640 1 -- 172.21.15.40:0/232087190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c004c10 con 0x7f4d140b2850 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.422+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d140b8ac0 con 0x7f4d140b2850 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.422+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d140b9080 con 0x7f4d140b2850 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.422+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d140694f0 con 0x7f4d140b2850 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.423+0000 7f4d127fc640 1 -- 172.21.15.40:0/232087190 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d0c011070 con 0x7f4d140b2850 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.424+0000 7f4d127fc640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d0006c8f0 0x7f4d0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.424+0000 7f4d127fc640 1 -- 172.21.15.40:0/232087190 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f4d0c08c6b0 con 0x7f4d140b2850 2024-04-26T21:47:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.424+0000 7f4d19575640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d0006c8f0 0x7f4d0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.424+0000 7f4d19575640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d0006c8f0 0x7f4d0006edb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4d08034000 tx=0x7f4d0802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.428+0000 7f4d127fc640 1 -- 172.21.15.40:0/232087190 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d140d0ba0 con 0x7f4d140b2850 2024-04-26T21:47:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.656+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d140b6860 con 0x7f4d140b2850 2024-04-26T21:47:03.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.656+0000 7f4d127fc640 1 -- 172.21.15.40:0/232087190 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5182 (secure 0 0 0) 0x7f4d0c01a2b0 con 0x7f4d140b2850 2024-04-26T21:47:03.657 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:03.657 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1002792848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1002792848},{"type":"v1","addr":"172.21.15.40:6833","nonce":1002792848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:47:00.954510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:47:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 350 2024-04-26T21:47:03.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d0006c8f0 msgr2=0x7f4d0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d0006c8f0 0x7f4d0006edb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4d08034000 tx=0x7f4d0802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:03.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140b2850 msgr2=0x7f4d1415ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140b2850 0x7f4d1415ccd0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c3f0 tx=0x7f4d0c00c8b0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 shutdown_connections 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d0006c8f0 0x7f4d0006edb0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d1415f3a0 0x7f4d14160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140d06f0 0x7f4d140ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 --2- 172.21.15.40:0/232087190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d140b2850 0x7f4d1415ccd0 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 >> 172.21.15.40:0/232087190 conn(0x7f4d140d4f90 msgr2=0x7f4d140d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 shutdown_connections 2024-04-26T21:47:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.658+0000 7f4d1bfda640 1 -- 172.21.15.40:0/232087190 wait complete. 2024-04-26T21:47:03.670 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:47:03.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:03.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.770+0000 7f442e7af640 1 Processor -- start 2024-04-26T21:47:03.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.770+0000 7f442e7af640 1 -- start start 2024-04-26T21:47:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.771+0000 7f442e7af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44280d06f0 0x7f44280d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.771+0000 7f442e7af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280b2850 0x7f44280b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.771+0000 7f442e7af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44280d0ba0 con 0x7f442815f3a0 2024-04-26T21:47:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.771+0000 7f442e7af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f44280d0e20 con 0x7f44280d06f0 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.771+0000 7f442e7af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f44280b1d70 con 0x7f44280b2850 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.771+0000 7f44277fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44280d06f0 0x7f44280d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.771+0000 7f44277fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44280d06f0 0x7f44280d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46904/0 (socket says 172.21.15.40:46904) 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.771+0000 7f44277fe640 1 -- 172.21.15.40:0/2957664729 learned_addr learned my addr 172.21.15.40:0/2957664729 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.772+0000 7f442cd4c640 1 --2- 172.21.15.40:0/2957664729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280b2850 0x7f44280b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.772+0000 7f44277fe640 1 -- 172.21.15.40:0/2957664729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280b2850 msgr2=0x7f44280b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.772+0000 7f44277fe640 1 --2- 172.21.15.40:0/2957664729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280b2850 0x7f44280b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.772+0000 7f44277fe640 1 -- 172.21.15.40:0/2957664729 >> v1:172.21.15.184:6789/0 conn(0x7f442815f3a0 legacy=0x7f44280d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.772+0000 7f44277fe640 1 -- 172.21.15.40:0/2957664729 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44280b2050 con 0x7f44280d06f0 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.772+0000 7f44277fe640 1 --2- 172.21.15.40:0/2957664729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44280d06f0 0x7f44280d1e80 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f4410009870 tx=0x7f441002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb1b69cb9d192f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f4426ffd640 1 -- 172.21.15.40:0/2957664729 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410037070 con 0x7f44280d06f0 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f4426ffd640 1 -- 172.21.15.40:0/2957664729 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4410032440 con 0x7f44280d06f0 2024-04-26T21:47:03.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f4426ffd640 1 -- 172.21.15.40:0/2957664729 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410032810 con 0x7f44280d06f0 2024-04-26T21:47:03.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f442e7af640 1 -- 172.21.15.40:0/2957664729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44280d06f0 msgr2=0x7f44280d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f442e7af640 1 --2- 172.21.15.40:0/2957664729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44280d06f0 0x7f44280d1e80 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f4410009870 tx=0x7f441002a270 comp rx=0 tx=0).stop 2024-04-26T21:47:03.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f442e7af640 1 -- 172.21.15.40:0/2957664729 shutdown_connections 2024-04-26T21:47:03.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f442e7af640 1 --2- 172.21.15.40:0/2957664729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280b2850 0x7f44280b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f442e7af640 1 --2- 172.21.15.40:0/2957664729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44280d06f0 0x7f44280d1e80 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f442e7af640 1 -- 172.21.15.40:0/2957664729 >> 172.21.15.40:0/2957664729 conn(0x7f44280d4f90 msgr2=0x7f44280d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:03.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f442e7af640 1 -- 172.21.15.40:0/2957664729 shutdown_connections 2024-04-26T21:47:03.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.773+0000 7f442e7af640 1 -- 172.21.15.40:0/2957664729 wait complete. 2024-04-26T21:47:03.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.774+0000 7f442e7af640 1 Processor -- start 2024-04-26T21:47:03.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442e7af640 1 -- start start 2024-04-26T21:47:03.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442e7af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44280b2850 0x7f44280b9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442e7af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280d06f0 0x7f4428157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442e7af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f442815f3a0 0x7f44280cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442e7af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f442815f900 con 0x7f442815f3a0 2024-04-26T21:47:03.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442e7af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f442815f780 con 0x7f44280b2850 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442e7af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f44280d23c0 con 0x7f44280d06f0 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442cd4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f442815f3a0 0x7f44280cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442cd4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f442815f3a0 0x7f44280cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46910/0 (socket says 172.21.15.40:46910) 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.775+0000 7f442cd4c640 1 -- 172.21.15.40:0/2429601750 learned_addr learned my addr 172.21.15.40:0/2429601750 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f44277fe640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280d06f0 0x7f4428157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f4427fff640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44280b2850 0x7f44280b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f442cd4c640 1 -- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280d06f0 msgr2=0x7f4428157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f442cd4c640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280d06f0 0x7f4428157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f442cd4c640 1 -- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44280b2850 msgr2=0x7f44280b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f442cd4c640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44280b2850 0x7f44280b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f442cd4c640 1 -- 172.21.15.40:0/2429601750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44280b9bc0 con 0x7f442815f3a0 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f44277fe640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280d06f0 0x7f4428157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f442cd4c640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f442815f3a0 0x7f44280cf090 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f442000efc0 tx=0x7f442000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f4424ff9640 1 -- 172.21.15.40:0/2429601750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420018020 con 0x7f442815f3a0 2024-04-26T21:47:03.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f4424ff9640 1 -- 172.21.15.40:0/2429601750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420014480 con 0x7f442815f3a0 2024-04-26T21:47:03.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44280b9ea0 con 0x7f442815f3a0 2024-04-26T21:47:03.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f4424ff9640 1 -- 172.21.15.40:0/2429601750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420014870 con 0x7f442815f3a0 2024-04-26T21:47:03.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.776+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44280ba460 con 0x7f442815f3a0 2024-04-26T21:47:03.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.777+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44280694f0 con 0x7f442815f3a0 2024-04-26T21:47:03.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.778+0000 7f4424ff9640 1 -- 172.21.15.40:0/2429601750 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4420005d10 con 0x7f442815f3a0 2024-04-26T21:47:03.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.778+0000 7f4424ff9640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:03.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.778+0000 7f4424ff9640 1 -- 172.21.15.40:0/2429601750 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f442008d540 con 0x7f442815f3a0 2024-04-26T21:47:03.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.778+0000 7f4427fff640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:03.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.779+0000 7f4427fff640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f440806c8f0 0x7f440806edb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4418001640 tx=0x7f4418009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:03.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:03.782+0000 7f4424ff9640 1 -- 172.21.15.40:0/2429601750 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4420022070 con 0x7f442815f3a0 2024-04-26T21:47:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.007+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44280b6860 con 0x7f442815f3a0 2024-04-26T21:47:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.008+0000 7f4424ff9640 1 -- 172.21.15.40:0/2429601750 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5182 (secure 0 0 0) 0x7f4420010040 con 0x7f442815f3a0 2024-04-26T21:47:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:04.009 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1002792848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1002792848},{"type":"v1","addr":"172.21.15.40:6833","nonce":1002792848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:47:00.954510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:47:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 350 2024-04-26T21:47:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f440806c8f0 msgr2=0x7f440806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f440806c8f0 0x7f440806edb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4418001640 tx=0x7f4418009040 comp rx=0 tx=0).stop 2024-04-26T21:47:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f442815f3a0 msgr2=0x7f44280cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f442815f3a0 0x7f44280cf090 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f442000efc0 tx=0x7f442000c490 comp rx=0 tx=0).stop 2024-04-26T21:47:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 shutdown_connections 2024-04-26T21:47:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f442815f3a0 0x7f44280cf090 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44280d06f0 0x7f4428157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 --2- 172.21.15.40:0/2429601750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44280b2850 0x7f44280b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 >> 172.21.15.40:0/2429601750 conn(0x7f44280d4f90 msgr2=0x7f442806e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 shutdown_connections 2024-04-26T21:47:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.010+0000 7f442e7af640 1 -- 172.21.15.40:0/2429601750 wait complete. 2024-04-26T21:47:04.022 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:47:04.022 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.122+0000 7fe6c3a2a640 1 Processor -- start 2024-04-26T21:47:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.123+0000 7fe6c3a2a640 1 -- start start 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.123+0000 7fe6c3a2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe6bc0cc480 0x7fe6bc072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.123+0000 7fe6c3a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.123+0000 7fe6c3a2a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6bc15d7c0 con 0x7fe6bc071e60 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.123+0000 7fe6c3a2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe6bc15daa0 con 0x7fe6bc0cc480 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.123+0000 7fe6c3a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6bc15dd80 con 0x7fe6bc072780 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.123+0000 7fe6c0fc5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe6bc0cc480 0x7fe6bc072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.124+0000 7fe6c0fc5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe6bc0cc480 0x7fe6bc072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46912/0 (socket says 172.21.15.40:46912) 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.124+0000 7fe6c0fc5640 1 -- 172.21.15.40:0/89333423 learned_addr learned my addr 172.21.15.40:0/89333423 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.124+0000 7fe6c1fc7640 1 --2- 172.21.15.40:0/89333423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.124+0000 7fe6c0fc5640 1 -- 172.21.15.40:0/89333423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 msgr2=0x7fe6bc166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.124+0000 7fe6c0fc5640 1 --2- 172.21.15.40:0/89333423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.124+0000 7fe6c0fc5640 1 -- 172.21.15.40:0/89333423 >> v1:172.21.15.184:6789/0 conn(0x7fe6bc071e60 legacy=0x7fe6bc06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.124+0000 7fe6c0fc5640 1 -- 172.21.15.40:0/89333423 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc15e060 con 0x7fe6bc0cc480 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c0fc5640 1 --2- 172.21.15.40:0/89333423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc072240 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009870 tx=0x7fe6ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6a6fdbac025be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6b3fff640 1 -- 172.21.15.40:0/89333423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac037070 con 0x7fe6bc0cc480 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6b3fff640 1 -- 172.21.15.40:0/89333423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ac032440 con 0x7fe6bc0cc480 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6b3fff640 1 -- 172.21.15.40:0/89333423 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac032810 con 0x7fe6bc0cc480 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/89333423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 msgr2=0x7fe6bc072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/89333423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc072240 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009870 tx=0x7fe6ac02a270 comp rx=0 tx=0).stop 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/89333423 shutdown_connections 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/89333423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/89333423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc072240 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/89333423 >> 172.21.15.40:0/89333423 conn(0x7fe6bc0d0be0 msgr2=0x7fe6bc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/89333423 shutdown_connections 2024-04-26T21:47:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.125+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/89333423 wait complete. 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.126+0000 7fe6c3a2a640 1 Processor -- start 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c3a2a640 1 -- start start 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c3a2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6bc071e60 0x7fe6bc14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c3a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c3a2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc0c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c3a2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc15d3f0 con 0x7fe6bc0cc480 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c3a2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc072c10 con 0x7fe6bc071e60 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c3a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6bc15d570 con 0x7fe6bc072780 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c1fc7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc0c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c1fc7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc0c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46920/0 (socket says 172.21.15.40:46920) 2024-04-26T21:47:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.127+0000 7fe6c1fc7640 1 -- 172.21.15.40:0/2545579030 learned_addr learned my addr 172.21.15.40:0/2545579030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c0fc5640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c17c6640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6bc071e60 0x7fe6bc14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c1fc7640 1 -- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 msgr2=0x7fe6bc0b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c1fc7640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc0b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c1fc7640 1 -- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6bc071e60 msgr2=0x7fe6bc14d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c1fc7640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6bc071e60 0x7fe6bc14d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c1fc7640 1 -- 172.21.15.40:0/2545579030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc0b5450 con 0x7fe6bc0cc480 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c0fc5640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c1fc7640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc0c90f0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009dd0 tx=0x7fe6b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6b1ffb640 1 -- 172.21.15.40:0/2545579030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4018020 con 0x7fe6bc0cc480 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6b1ffb640 1 -- 172.21.15.40:0/2545579030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b4014480 con 0x7fe6bc0cc480 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6bc0b5730 con 0x7fe6bc0cc480 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6b1ffb640 1 -- 172.21.15.40:0/2545579030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4014870 con 0x7fe6bc0cc480 2024-04-26T21:47:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.128+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6bc0b5cf0 con 0x7fe6bc0cc480 2024-04-26T21:47:04.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.129+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6bc0694f0 con 0x7fe6bc0cc480 2024-04-26T21:47:04.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.130+0000 7fe6b1ffb640 1 -- 172.21.15.40:0/2545579030 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6b4005d10 con 0x7fe6bc0cc480 2024-04-26T21:47:04.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.130+0000 7fe6b1ffb640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6a006c8f0 0x7fe6a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.130+0000 7fe6b1ffb640 1 -- 172.21.15.40:0/2545579030 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7fe6b408d5b0 con 0x7fe6bc0cc480 2024-04-26T21:47:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.130+0000 7fe6c17c6640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6a006c8f0 0x7fe6a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.131+0000 7fe6c17c6640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6a006c8f0 0x7fe6a006edb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8007fd0 tx=0x7fe6b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.135+0000 7fe6b1ffb640 1 -- 172.21.15.40:0/2545579030 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe6b4022070 con 0x7fe6bc0cc480 2024-04-26T21:47:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.363+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6bc069e20 con 0x7fe6bc0cc480 2024-04-26T21:47:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.363+0000 7fe6b1ffb640 1 -- 172.21.15.40:0/2545579030 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5182 (secure 0 0 0) 0x7fe6b4010040 con 0x7fe6bc0cc480 2024-04-26T21:47:04.365 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:04.365 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1002792848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1002792848},{"type":"v1","addr":"172.21.15.40:6833","nonce":1002792848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:47:00.954510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:47:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 350 2024-04-26T21:47:04.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6a006c8f0 msgr2=0x7fe6a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6a006c8f0 0x7fe6a006edb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8007fd0 tx=0x7fe6b8009040 comp rx=0 tx=0).stop 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 msgr2=0x7fe6bc0c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc0c90f0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009dd0 tx=0x7fe6b400c490 comp rx=0 tx=0).stop 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 shutdown_connections 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6a006c8f0 0x7fe6a006edb0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6bc0cc480 0x7fe6bc0c90f0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc072780 0x7fe6bc0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 --2- 172.21.15.40:0/2545579030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6bc071e60 0x7fe6bc14d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 >> 172.21.15.40:0/2545579030 conn(0x7fe6bc0d0be0 msgr2=0x7fe6bc0b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 shutdown_connections 2024-04-26T21:47:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.365+0000 7fe6c3a2a640 1 -- 172.21.15.40:0/2545579030 wait complete. 2024-04-26T21:47:04.377 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 1.00000000' led to DAMAGED state 2024-04-26T21:47:04.377 INFO:tasks.cephfs.test_damage:All 39 mutations had expected outcomes 2024-04-26T21:47:04.377 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:47:04.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:04.418 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:04.458 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:47:04.458 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:04.458 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:04.458 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:04.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:04.474 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:04.474 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:47:04.475 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:04.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:04.518 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:04.518 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:04.519 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:04.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:04.534 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:04.534 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:47:04.534 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:04.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:04.576 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:04.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.676+0000 7fe5c7837640 1 Processor -- start 2024-04-26T21:47:04.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.676+0000 7fe5c7837640 1 -- start start 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.677+0000 7fe5c7837640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c01604e0 0x7fe5c006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.677+0000 7fe5c7837640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5c006fcd0 0x7fe5c00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.677+0000 7fe5c7837640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c0160970 con 0x7fe5c015f2d0 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.677+0000 7fe5c7837640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c0074010 con 0x7fe5c006fcd0 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.677+0000 7fe5c7837640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c0070100 con 0x7fe5c01604e0 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c4dd2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c01604e0 0x7fe5c006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c5dd4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5c006fcd0 0x7fe5c00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c4dd2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c01604e0 0x7fe5c006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:33118/0 (socket says 172.21.15.40:33118) 2024-04-26T21:47:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c4dd2640 1 -- 172.21.15.40:0/195888082 learned_addr learned my addr 172.21.15.40:0/195888082 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c5dd4640 1 -- 172.21.15.40:0/195888082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c01604e0 msgr2=0x7fe5c006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c5dd4640 1 --2- 172.21.15.40:0/195888082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c01604e0 0x7fe5c006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c5dd4640 1 -- 172.21.15.40:0/195888082 >> v1:172.21.15.184:6789/0 conn(0x7fe5c015f2d0 legacy=0x7fe5c0072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c5dd4640 1 -- 172.21.15.40:0/195888082 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c0072f70 con 0x7fe5c006fcd0 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5b7fff640 1 -- 172.21.15.40:0/195888082 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1065025352 0 0) 0x7fe5c0160970 con 0x7fe5c015f2d0 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.678+0000 7fe5c4dd2640 1 --2- 172.21.15.40:0/195888082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c01604e0 0x7fe5c006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.679+0000 7fe5c5dd4640 1 --2- 172.21.15.40:0/195888082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c006fcd0 0x7fe5c00ce7b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b08760254d57bd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.679+0000 7fe5b7fff640 1 -- 172.21.15.40:0/195888082 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc007740 con 0x7fe5c006fcd0 2024-04-26T21:47:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.679+0000 7fe5b7fff640 1 -- 172.21.15.40:0/195888082 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0078e0 con 0x7fe5c006fcd0 2024-04-26T21:47:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.679+0000 7fe5b7fff640 1 -- 172.21.15.40:0/195888082 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc007cb0 con 0x7fe5c006fcd0 2024-04-26T21:47:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.680+0000 7fe5c7837640 1 -- 172.21.15.40:0/195888082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c006fcd0 msgr2=0x7fe5c00ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.680+0000 7fe5c7837640 1 --2- 172.21.15.40:0/195888082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c006fcd0 0x7fe5c00ce7b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc02a660 comp rx=0 tx=0).stop 2024-04-26T21:47:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.680+0000 7fe5c7837640 1 -- 172.21.15.40:0/195888082 shutdown_connections 2024-04-26T21:47:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.680+0000 7fe5c7837640 1 --2- 172.21.15.40:0/195888082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c006fcd0 0x7fe5c00ce7b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.680+0000 7fe5c7837640 1 --2- 172.21.15.40:0/195888082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c01604e0 0x7fe5c006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.680+0000 7fe5c7837640 1 -- 172.21.15.40:0/195888082 >> 172.21.15.40:0/195888082 conn(0x7fe5c00d0be0 msgr2=0x7fe5c00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.680+0000 7fe5c7837640 1 -- 172.21.15.40:0/195888082 shutdown_connections 2024-04-26T21:47:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.680+0000 7fe5c7837640 1 -- 172.21.15.40:0/195888082 wait complete. 2024-04-26T21:47:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.681+0000 7fe5c7837640 1 Processor -- start 2024-04-26T21:47:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.681+0000 7fe5c7837640 1 -- start start 2024-04-26T21:47:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c7837640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c006fcd0 0x7fe5c0155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c7837640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c015f2d0 0x7fe5c014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c7837640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c01604e0 0x7fe5c01535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c7837640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0073810 con 0x7fe5c01604e0 2024-04-26T21:47:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c7837640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0073530 con 0x7fe5c015f2d0 2024-04-26T21:47:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c7837640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c0073af0 con 0x7fe5c006fcd0 2024-04-26T21:47:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c01604e0 0x7fe5c01535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c01604e0 0x7fe5c01535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46926/0 (socket says 172.21.15.40:46926) 2024-04-26T21:47:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 -- 172.21.15.40:0/2627605156 learned_addr learned my addr 172.21.15.40:0/2627605156 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c55d3640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c006fcd0 0x7fe5c0155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c4dd2640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c015f2d0 0x7fe5c014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 -- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c006fcd0 msgr2=0x7fe5c0155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c006fcd0 0x7fe5c0155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 -- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c015f2d0 msgr2=0x7fe5c014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c015f2d0 0x7fe5c014d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 -- 172.21.15.40:0/2627605156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c0153ed0 con 0x7fe5c01604e0 2024-04-26T21:47:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c4dd2640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c015f2d0 0x7fe5c014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5c5dd4640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c01604e0 0x7fe5c01535f0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5b5ffb640 1 -- 172.21.15.40:0/2627605156 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc040070 con 0x7fe5c01604e0 2024-04-26T21:47:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.682+0000 7fe5b5ffb640 1 -- 172.21.15.40:0/2627605156 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc031c40 con 0x7fe5c01604e0 2024-04-26T21:47:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.683+0000 7fe5b5ffb640 1 -- 172.21.15.40:0/2627605156 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc009450 con 0x7fe5c01604e0 2024-04-26T21:47:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.683+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c01541b0 con 0x7fe5c01604e0 2024-04-26T21:47:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.683+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c00b5340 con 0x7fe5c01604e0 2024-04-26T21:47:04.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.684+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c00694f0 con 0x7fe5c01604e0 2024-04-26T21:47:04.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.684+0000 7fe5b5ffb640 1 -- 172.21.15.40:0/2627605156 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5bc039040 con 0x7fe5c01604e0 2024-04-26T21:47:04.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.685+0000 7fe5b5ffb640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe5a406c8f0 0x7fe5a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:04.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.685+0000 7fe5b5ffb640 1 -- 172.21.15.40:0/2627605156 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7fe5bc0ad8c0 con 0x7fe5c01604e0 2024-04-26T21:47:04.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.685+0000 7fe5c55d3640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe5a406c8f0 0x7fe5a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:04.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.686+0000 7fe5c55d3640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe5a406c8f0 0x7fe5a406edb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8000f60 tx=0x7fe5b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:04.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.689+0000 7fe5b5ffb640 1 -- 172.21.15.40:0/2627605156 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe5bc07a6e0 con 0x7fe5c01604e0 2024-04-26T21:47:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.913+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5c00ce510 con 0x7fe5c01604e0 2024-04-26T21:47:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.913+0000 7fe5b5ffb640 1 -- 172.21.15.40:0/2627605156 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5182 (secure 0 0 0) 0x7fe5bc07e9e0 con 0x7fe5c01604e0 2024-04-26T21:47:04.914 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:04.914 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1002792848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1002792848},{"type":"v1","addr":"172.21.15.40:6833","nonce":1002792848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:47:00.954510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:47:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 350 2024-04-26T21:47:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe5a406c8f0 msgr2=0x7fe5a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe5a406c8f0 0x7fe5a406edb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8000f60 tx=0x7fe5b8009040 comp rx=0 tx=0).stop 2024-04-26T21:47:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c01604e0 msgr2=0x7fe5c01535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c01604e0 0x7fe5c01535f0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 shutdown_connections 2024-04-26T21:47:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe5a406c8f0 0x7fe5a406edb0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c01604e0 0x7fe5c01535f0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5c015f2d0 0x7fe5c014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 --2- 172.21.15.40:0/2627605156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c006fcd0 0x7fe5c0155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 >> 172.21.15.40:0/2627605156 conn(0x7fe5c00d0be0 msgr2=0x7fe5c014f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 shutdown_connections 2024-04-26T21:47:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:04.915+0000 7fe5c7837640 1 -- 172.21.15.40:0/2627605156 wait complete. 2024-04-26T21:47:04.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:05.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.028+0000 7f966349c640 1 Processor -- start 2024-04-26T21:47:05.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.028+0000 7f966349c640 1 -- start start 2024-04-26T21:47:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f966349c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c15b600 0x7f965c071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f966349c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f965c072450 con 0x7f965c0cf0e0 2024-04-26T21:47:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f966349c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f965c15b9e0 con 0x7f965c0cf5e0 2024-04-26T21:47:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f966349c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f965c0cb9b0 con 0x7f965c15b600 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f9661a39640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f965c0cf0e0 0x7f965c06f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34526/0 (socket says 172.21.15.40:34526) 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f9661a39640 1 -- 172.21.15.40:0/36072536 learned_addr learned my addr 172.21.15.40:0/36072536 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1604841132 0 0) 0x7f964c001490 con 0x7f965c0cf0e0 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f9661238640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c15b600 0x7f965c071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.029+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9644003520 con 0x7f965c0cf0e0 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1189473275 0 0) 0x7f9644003520 con 0x7f965c0cf0e0 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c15b600 msgr2=0x7f965c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c15b600 0x7f965c071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 >> v1:172.21.15.184:6789/0 conn(0x7f965c0cf5e0 legacy=0x7f965c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c0d0740 con 0x7f965c0cf0e0 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f964c001bc0 con 0x7f965c0cf0e0 2024-04-26T21:47:05.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 >> v1:172.21.15.40:6789/0 conn(0x7f965c0cf0e0 legacy=0x7f965c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c15b600 0x7f9644003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9644003f80 0x7f9644004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644004d40 0x7f96440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f965c0cb9b0 con 0x7f9644004d40 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f964400bad0 con 0x7f9644003f80 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f964400dee0 con 0x7f965c15b600 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f9661a39640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644004d40 0x7f96440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f9661238640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c15b600 0x7f9644003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f9660a37640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9644003f80 0x7f9644004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f9661a39640 1 -- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c15b600 msgr2=0x7f9644003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f9661a39640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c15b600 0x7f9644003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f9661a39640 1 -- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9644003f80 msgr2=0x7f9644004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f9661a39640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9644003f80 0x7f9644004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.030+0000 7f9661a39640 1 -- 172.21.15.40:0/36072536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9644003520 con 0x7f9644004d40 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.031+0000 7f9661a39640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644004d40 0x7f96440051c0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f965c0b06a0 tx=0x7f9644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f989e85db3c91746 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.031+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c00a650 con 0x7f9644004d40 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.031+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c02f070 con 0x7f9644004d40 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.031+0000 7f965bfff640 1 -- 172.21.15.40:0/36072536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c02be00 con 0x7f9644004d40 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.031+0000 7f966349c640 1 -- 172.21.15.40:0/36072536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644004d40 msgr2=0x7f96440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.031+0000 7f966349c640 1 --2- 172.21.15.40:0/36072536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9644004d40 0x7f96440051c0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f965c0b06a0 tx=0x7f9644005830 comp rx=0 tx=0).stop 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.032+0000 7f9661a39640 1 -- 172.21.15.40:0/36072536 reap_dead start 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.032+0000 7f966349c640 1 -- 172.21.15.40:0/36072536 shutdown_connections 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.032+0000 7f966349c640 1 -- 172.21.15.40:0/36072536 >> 172.21.15.40:0/36072536 conn(0x7f965c0d0be0 msgr2=0x7f965c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.032+0000 7f966349c640 1 -- 172.21.15.40:0/36072536 shutdown_connections 2024-04-26T21:47:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.032+0000 7f966349c640 1 -- 172.21.15.40:0/36072536 wait complete. 2024-04-26T21:47:05.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.033+0000 7f966349c640 1 Processor -- start 2024-04-26T21:47:05.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.033+0000 7f966349c640 1 -- start start 2024-04-26T21:47:05.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.033+0000 7f966349c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965c1550c0 0x7f965c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f966349c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965c0b6040 0x7f965c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f966349c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c0b4700 0x7f965c1f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f966349c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f965c0d02f0 con 0x7f965c0b6040 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f966349c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f965c0cb640 con 0x7f965c1550c0 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f966349c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f965c0d0470 con 0x7f965c0b4700 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965c0b6040 0x7f965c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965c0b6040 0x7f965c0b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46954/0 (socket says 172.21.15.40:46954) 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 -- 172.21.15.40:0/877559157 learned_addr learned my addr 172.21.15.40:0/877559157 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9661a39640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c0b4700 0x7f965c1f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9661238640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965c1550c0 0x7f965c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 -- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c0b4700 msgr2=0x7f965c1f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c0b4700 0x7f965c1f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 -- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965c1550c0 msgr2=0x7f965c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965c1550c0 0x7f965c0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 -- 172.21.15.40:0/877559157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c0b6720 con 0x7f965c0b6040 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9660a37640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965c0b6040 0x7f965c0b41c0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f965000f7d0 tx=0x7f965000fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9659ffb640 1 -- 172.21.15.40:0/877559157 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96500050d0 con 0x7f965c0b6040 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9659ffb640 1 -- 172.21.15.40:0/877559157 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9650018070 con 0x7f965c0b6040 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965c0af380 con 0x7f965c0b6040 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.034+0000 7f9659ffb640 1 -- 172.21.15.40:0/877559157 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000d6d0 con 0x7f965c0b6040 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.035+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965c0af940 con 0x7f965c0b6040 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.036+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965c0694f0 con 0x7f965c0b6040 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.036+0000 7f9659ffb640 1 -- 172.21.15.40:0/877559157 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f965000d870 con 0x7f965c0b6040 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.036+0000 7f9659ffb640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f964806c8f0 0x7f964806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.036+0000 7f9659ffb640 1 -- 172.21.15.40:0/877559157 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f965008c780 con 0x7f965c0b6040 2024-04-26T21:47:05.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.037+0000 7f9661238640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f964806c8f0 0x7f964806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.037+0000 7f9661238640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f964806c8f0 0x7f964806edb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f9654009bc0 tx=0x7f9654009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.041+0000 7f9659ffb640 1 -- 172.21.15.40:0/877559157 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9650059bf0 con 0x7f965c0b6040 2024-04-26T21:47:05.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.225+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f965c0b4ae0 con 0x7f965c0b6040 2024-04-26T21:47:05.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.227+0000 7f9659ffb640 1 -- 172.21.15.40:0/877559157 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+22228 (secure 0 0 0) 0x7f96500594f0 con 0x7f965c0b6040 2024-04-26T21:47:05.228 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:05.228 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":135,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:00.954261+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/933356185":"2024-04-27T21:47:00.954240+0000","172.21.15.40:6832/933356185":"2024-04-27T21:47:00.954240+0000","172.21.15.40:6833/3095512631":"2024-04-27T21:46:53.909685+0000","172.21.15.40:6833/3502494134":"2024-04-27T21:46:46.873417+0000","172.21.15.40:6833/1769689018":"2024-04-27T21:46:41.828993+0000","172.21.15.40:6833/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6832/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6833/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6832/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6833/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6833/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6832/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6833/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6833/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6832/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6832/3502494134":"2024-04-27T21:46:46.873417+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/3095512631":"2024-04-27T21:46:53.909685+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3538213517":"2024-04-27T21:45:17.788819+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6832/1769689018":"2024-04-27T21:46:41.828993+0000","172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/3538213517":"2024-04-27T21:45:17.788819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:05.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f964806c8f0 msgr2=0x7f964806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f964806c8f0 0x7f964806edb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f9654009bc0 tx=0x7f9654009190 comp rx=0 tx=0).stop 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965c0b6040 msgr2=0x7f965c0b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965c0b6040 0x7f965c0b41c0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f965000f7d0 tx=0x7f965000fbd0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 shutdown_connections 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f964806c8f0 0x7f964806edb0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965c0b4700 0x7f965c1f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965c0b6040 0x7f965c0b41c0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 --2- 172.21.15.40:0/877559157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965c1550c0 0x7f965c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 >> 172.21.15.40:0/877559157 conn(0x7f965c0d0be0 msgr2=0x7f965c0738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 shutdown_connections 2024-04-26T21:47:05.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.229+0000 7f966349c640 1 -- 172.21.15.40:0/877559157 wait complete. 2024-04-26T21:47:05.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:42:06.883998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:42:07.550973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:05.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.345+0000 7fec2f2c8640 1 Processor -- start 2024-04-26T21:47:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.346+0000 7fec2f2c8640 1 -- start start 2024-04-26T21:47:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.346+0000 7fec2f2c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c6a0 0x7fec280af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.346+0000 7fec2f2c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec280b4480 con 0x7fec2806c240 2024-04-26T21:47:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.346+0000 7fec2f2c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec280b4720 con 0x7fec2815f1d0 2024-04-26T21:47:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.346+0000 7fec2f2c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec280ae8c0 con 0x7fec2806c6a0 2024-04-26T21:47:05.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec2c863640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fec2806c240 0x7fec280aed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34538/0 (socket says 172.21.15.40:34538) 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec2c863640 1 -- 172.21.15.40:0/3993737357 learned_addr learned my addr 172.21.15.40:0/3993737357 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec2d865640 1 --2- 172.21.15.40:0/3993737357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c6a0 0x7fec280af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec1ffff640 1 -- 172.21.15.40:0/3993737357 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2704123108 0 0) 0x7fec280b4480 con 0x7fec2806c240 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec1ffff640 1 -- 172.21.15.40:0/3993737357 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec14003520 con 0x7fec2806c240 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec1ffff640 1 -- 172.21.15.40:0/3993737357 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1206267485 0 0) 0x7fec280b4720 con 0x7fec2815f1d0 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec1ffff640 1 -- 172.21.15.40:0/3993737357 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec280b4480 con 0x7fec2815f1d0 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec2d865640 1 -- 172.21.15.40:0/3993737357 >> v1:172.21.15.184:6789/0 conn(0x7fec2815f1d0 legacy=0x7fec2806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec2d865640 1 -- 172.21.15.40:0/3993737357 >> v1:172.21.15.40:6789/0 conn(0x7fec2806c240 legacy=0x7fec280aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.347+0000 7fec2d865640 1 -- 172.21.15.40:0/3993737357 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec280b4a00 con 0x7fec2806c6a0 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.348+0000 7fec2d865640 1 --2- 172.21.15.40:0/3993737357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c6a0 0x7fec280af460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec280aea40 tx=0x7fec2002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a1a5627c5b41e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.348+0000 7fec1ffff640 1 -- 172.21.15.40:0/3993737357 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20007740 con 0x7fec2806c6a0 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.348+0000 7fec1ffff640 1 -- 172.21.15.40:0/3993737357 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec200078e0 con 0x7fec2806c6a0 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.348+0000 7fec1ffff640 1 -- 172.21.15.40:0/3993737357 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20007c90 con 0x7fec2806c6a0 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.348+0000 7fec2f2c8640 1 -- 172.21.15.40:0/3993737357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c6a0 msgr2=0x7fec280af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.348+0000 7fec2f2c8640 1 --2- 172.21.15.40:0/3993737357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c6a0 0x7fec280af460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec280aea40 tx=0x7fec2002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.349+0000 7fec2f2c8640 1 -- 172.21.15.40:0/3993737357 shutdown_connections 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.349+0000 7fec2f2c8640 1 --2- 172.21.15.40:0/3993737357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c6a0 0x7fec280af460 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.349+0000 7fec2f2c8640 1 -- 172.21.15.40:0/3993737357 >> 172.21.15.40:0/3993737357 conn(0x7fec280b3180 msgr2=0x7fec280b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.349+0000 7fec2f2c8640 1 -- 172.21.15.40:0/3993737357 shutdown_connections 2024-04-26T21:47:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.349+0000 7fec2f2c8640 1 -- 172.21.15.40:0/3993737357 wait complete. 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.350+0000 7fec2f2c8640 1 Processor -- start 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.350+0000 7fec2f2c8640 1 -- start start 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.350+0000 7fec2f2c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c240 0x7fec281609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.350+0000 7fec2f2c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec2806c6a0 0x7fec280cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.350+0000 7fec2f2c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2815f1d0 0x7fec280cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.350+0000 7fec2f2c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec2806cb30 con 0x7fec2815f1d0 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.350+0000 7fec2f2c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec280b4fc0 con 0x7fec2806c6a0 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.350+0000 7fec2f2c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec28075ea0 con 0x7fec2806c240 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2815f1d0 0x7fec280cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2815f1d0 0x7fec280cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46960/0 (socket says 172.21.15.40:46960) 2024-04-26T21:47:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 -- 172.21.15.40:0/137966398 learned_addr learned my addr 172.21.15.40:0/137966398 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d064640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c240 0x7fec281609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2c863640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec2806c6a0 0x7fec280cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 -- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c240 msgr2=0x7fec281609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c240 0x7fec281609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 -- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec2806c6a0 msgr2=0x7fec280cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec2806c6a0 0x7fec280cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 -- 172.21.15.40:0/137966398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2814d2b0 con 0x7fec2815f1d0 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2c863640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec2806c6a0 0x7fec280cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2d865640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2815f1d0 0x7fec280cb230 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fec2002acb0 tx=0x7fec20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec1dffb640 1 -- 172.21.15.40:0/137966398 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2003f070 con 0x7fec2815f1d0 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec1dffb640 1 -- 172.21.15.40:0/137966398 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec20031c40 con 0x7fec2815f1d0 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec1dffb640 1 -- 172.21.15.40:0/137966398 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20009450 con 0x7fec2815f1d0 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.351+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2814d590 con 0x7fec2815f1d0 2024-04-26T21:47:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.352+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2814db50 con 0x7fec2815f1d0 2024-04-26T21:47:05.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.352+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec28068fc0 con 0x7fec2815f1d0 2024-04-26T21:47:05.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.353+0000 7fec1dffb640 1 -- 172.21.15.40:0/137966398 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec20044460 con 0x7fec2815f1d0 2024-04-26T21:47:05.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.353+0000 7fec1dffb640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.354+0000 7fec1dffb640 1 -- 172.21.15.40:0/137966398 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7fec200ae110 con 0x7fec2815f1d0 2024-04-26T21:47:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.354+0000 7fec2d064640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.354+0000 7fec2d064640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fec24007fd0 tx=0x7fec24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.358+0000 7fec1dffb640 1 -- 172.21.15.40:0/137966398 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec2004b020 con 0x7fec2815f1d0 2024-04-26T21:47:05.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.540+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec281629f0 con 0x7fec2815f1d0 2024-04-26T21:47:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.541+0000 7fec1dffb640 1 -- 172.21.15.40:0/137966398 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) ==== 93+0+10 (secure 0 0 0) 0x7fec2007b410 con 0x7fec2815f1d0 2024-04-26T21:47:05.542 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:47:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 msgr2=0x7fec0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fec24007fd0 tx=0x7fec24009040 comp rx=0 tx=0).stop 2024-04-26T21:47:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2815f1d0 msgr2=0x7fec280cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2815f1d0 0x7fec280cb230 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fec2002acb0 tx=0x7fec20004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 shutdown_connections 2024-04-26T21:47:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec0c06c8f0 0x7fec0c06edb0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec2815f1d0 0x7fec280cb230 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec2806c6a0 0x7fec280cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 --2- 172.21.15.40:0/137966398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec2806c240 0x7fec281609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 >> 172.21.15.40:0/137966398 conn(0x7fec280b3180 msgr2=0x7fec2815d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 shutdown_connections 2024-04-26T21:47:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.543+0000 7fec2f2c8640 1 -- 172.21.15.40:0/137966398 wait complete. 2024-04-26T21:47:05.554 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:47:05.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.657+0000 7f47ce7c9640 1 Processor -- start 2024-04-26T21:47:05.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.657+0000 7f47ce7c9640 1 -- start start 2024-04-26T21:47:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.657+0000 7f47ce7c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.657+0000 7f47ce7c9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47c80d3bc0 0x7f47c80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ce7c9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c80d1040 con 0x7f47c815f550 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ce7c9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f47c80c9430 con 0x7f47c80d3bc0 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ce7c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47c80c9680 con 0x7f47c815f170 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ccd66640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47c80d3bc0 0x7f47c80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ccd66640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47c80d3bc0 0x7f47c80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46964/0 (socket says 172.21.15.40:46964) 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47c77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f47c815f550 0x7f47c80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34542/0 (socket says 172.21.15.40:34542) 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ccd66640 1 -- 172.21.15.40:0/3082026653 learned_addr learned my addr 172.21.15.40:0/3082026653 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47c7fff640 1 --2- 172.21.15.40:0/3082026653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ccd66640 1 -- 172.21.15.40:0/3082026653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 msgr2=0x7f47c80d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ccd66640 1 --2- 172.21.15.40:0/3082026653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ccd66640 1 -- 172.21.15.40:0/3082026653 >> v1:172.21.15.40:6789/0 conn(0x7f47c815f550 legacy=0x7f47c80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47ccd66640 1 -- 172.21.15.40:0/3082026653 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c80c9960 con 0x7f47c80d3bc0 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.658+0000 7f47c7fff640 1 --2- 172.21.15.40:0/3082026653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ccd66640 1 --2- 172.21.15.40:0/3082026653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80d3bc0 0x7f47c80d2cb0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009e50 tx=0x7f47c002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b036d08e7c93f6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47c6ffd640 1 -- 172.21.15.40:0/3082026653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0036070 con 0x7f47c80d3bc0 2024-04-26T21:47:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47c6ffd640 1 -- 172.21.15.40:0/3082026653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c0031600 con 0x7f47c80d3bc0 2024-04-26T21:47:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47c6ffd640 1 -- 172.21.15.40:0/3082026653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00319d0 con 0x7f47c80d3bc0 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3082026653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80d3bc0 msgr2=0x7f47c80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3082026653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80d3bc0 0x7f47c80d2cb0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009e50 tx=0x7f47c002a710 comp rx=0 tx=0).stop 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3082026653 shutdown_connections 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3082026653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80d3bc0 0x7f47c80d2cb0 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3082026653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3082026653 >> 172.21.15.40:0/3082026653 conn(0x7f47c80d4f60 msgr2=0x7f47c80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3082026653 shutdown_connections 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.659+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3082026653 wait complete. 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.661+0000 7f47ce7c9640 1 Processor -- start 2024-04-26T21:47:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.661+0000 7f47ce7c9640 1 -- start start 2024-04-26T21:47:05.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ce7c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47c80d3bc0 0x7f47c80a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ce7c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ce7c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80a1ce0 0x7f47c8154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ce7c9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80ca5c0 con 0x7f47c80a1ce0 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ce7c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80ca440 con 0x7f47c80d3bc0 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ce7c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47c80ca740 con 0x7f47c815f170 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80a1ce0 0x7f47c8154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80a1ce0 0x7f47c8154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46968/0 (socket says 172.21.15.40:46968) 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 -- 172.21.15.40:0/3907700914 learned_addr learned my addr 172.21.15.40:0/3907700914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47c77fe640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47c7fff640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47c80d3bc0 0x7f47c80a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 -- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 msgr2=0x7f47c80a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 -- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47c80d3bc0 msgr2=0x7f47c80a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47c80d3bc0 0x7f47c80a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 -- 172.21.15.40:0/3907700914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c80a0c40 con 0x7f47c80a1ce0 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47c7fff640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47c80d3bc0 0x7f47c80a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47c77fe640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47ccd66640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80a1ce0 0x7f47c8154460 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009f70 tx=0x7f47c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47c4ff9640 1 -- 172.21.15.40:0/3907700914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c003f020 con 0x7f47c80a1ce0 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.662+0000 7f47c4ff9640 1 -- 172.21.15.40:0/3907700914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c0037040 con 0x7f47c80a1ce0 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.663+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c80a0f20 con 0x7f47c80a1ce0 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.663+0000 7f47c4ff9640 1 -- 172.21.15.40:0/3907700914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0036070 con 0x7f47c80a1ce0 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.663+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c80a14e0 con 0x7f47c80a1ce0 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.663+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c80694f0 con 0x7f47c80a1ce0 2024-04-26T21:47:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.664+0000 7f47c4ff9640 1 -- 172.21.15.40:0/3907700914 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47c0042020 con 0x7f47c80a1ce0 2024-04-26T21:47:05.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.664+0000 7f47c4ff9640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47b406c920 0x7f47b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.664+0000 7f47c4ff9640 1 -- 172.21.15.40:0/3907700914 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f47c00ad0b0 con 0x7f47c80a1ce0 2024-04-26T21:47:05.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.665+0000 7f47c7fff640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47b406c920 0x7f47b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.665+0000 7f47c7fff640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47b406c920 0x7f47b406ede0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00aa30 tx=0x7f47bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.669+0000 7f47c4ff9640 1 -- 172.21.15.40:0/3907700914 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47c0049030 con 0x7f47c80a1ce0 2024-04-26T21:47:05.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.853+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47c80d2d40 con 0x7f47c80a1ce0 2024-04-26T21:47:05.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.854+0000 7f47c4ff9640 1 -- 172.21.15.40:0/3907700914 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v135) ==== 104+0+11 (secure 0 0 0) 0x7f47c007a4c0 con 0x7f47c80a1ce0 2024-04-26T21:47:05.855 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47b406c920 msgr2=0x7f47b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47b406c920 0x7f47b406ede0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00aa30 tx=0x7f47bc008040 comp rx=0 tx=0).stop 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80a1ce0 msgr2=0x7f47c8154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80a1ce0 0x7f47c8154460 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009f70 tx=0x7f47c0004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 shutdown_connections 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f47b406c920 0x7f47b406ede0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c80a1ce0 0x7f47c8154460 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47c815f170 0x7f47c80a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 --2- 172.21.15.40:0/3907700914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47c80d3bc0 0x7f47c80a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 >> 172.21.15.40:0/3907700914 conn(0x7f47c80d4f60 msgr2=0x7f47c80cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 shutdown_connections 2024-04-26T21:47:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.856+0000 7f47ce7c9640 1 -- 172.21.15.40:0/3907700914 wait complete. 2024-04-26T21:47:05.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:47:05.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.970+0000 7fbb64bac640 1 Processor -- start 2024-04-26T21:47:05.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.970+0000 7fbb64bac640 1 -- start start 2024-04-26T21:47:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.970+0000 7fbb64bac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600d1a00 0x7fbb600d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.970+0000 7fbb64bac640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600d0a10 con 0x7fbb600b40d0 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.970+0000 7fbb64bac640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600d0cf0 con 0x7fbb600d2320 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.970+0000 7fbb64bac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb600d0fd0 con 0x7fbb600d1a00 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5ed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbb600b40d0 0x7fbb600b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34548/0 (socket says 172.21.15.40:34548) 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5ed9d640 1 -- 172.21.15.40:0/3298365863 learned_addr learned my addr 172.21.15.40:0/3298365863 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5e59c640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600d1a00 0x7fbb600d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607992878 0 0) 0x7fbb600d0a10 con 0x7fbb600b40d0 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb4c003520 con 0x7fbb600b40d0 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2328577565 0 0) 0x7fbb4c003520 con 0x7fbb600b40d0 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600d1a00 msgr2=0x7fbb600d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600d1a00 0x7fbb600d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 >> v1:172.21.15.184:6789/0 conn(0x7fbb600d2320 legacy=0x7fbb601625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb600d40a0 con 0x7fbb600b40d0 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fbb580019f0 con 0x7fbb600b40d0 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 >> v1:172.21.15.40:6789/0 conn(0x7fbb600b40d0 legacy=0x7fbb600b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c0039e0 0x7fbb4c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4c004390 0x7fbb4c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600d1a00 0x7fbb4c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600d0fd0 con 0x7fbb4c004390 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c00ba70 con 0x7fbb4c0039e0 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c00bc70 con 0x7fbb600d1a00 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb57fff640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4c004390 0x7fbb4c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5e59c640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c0039e0 0x7fbb4c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.971+0000 7fbb5ed9d640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600d1a00 0x7fbb4c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb57fff640 1 -- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600d1a00 msgr2=0x7fbb4c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb57fff640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600d1a00 0x7fbb4c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb57fff640 1 -- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c0039e0 msgr2=0x7fbb4c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb57fff640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c0039e0 0x7fbb4c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb57fff640 1 -- 172.21.15.40:0/3298365863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c003520 con 0x7fbb4c004390 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb57fff640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4c004390 0x7fbb4c004860 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fbb48001860 tx=0x7fbb4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc66313cc0c047ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48037070 con 0x7fbb4c004390 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4802abf0 con 0x7fbb4c004390 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.972+0000 7fbb5dd9b640 1 -- 172.21.15.40:0/3298365863 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48001430 con 0x7fbb4c004390 2024-04-26T21:47:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.973+0000 7fbb64bac640 1 -- 172.21.15.40:0/3298365863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4c004390 msgr2=0x7fbb4c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.973+0000 7fbb64bac640 1 --2- 172.21.15.40:0/3298365863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4c004390 0x7fbb4c004860 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fbb48001860 tx=0x7fbb4c005830 comp rx=0 tx=0).stop 2024-04-26T21:47:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.973+0000 7fbb5ed9d640 1 -- 172.21.15.40:0/3298365863 reap_dead start 2024-04-26T21:47:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.973+0000 7fbb64bac640 1 -- 172.21.15.40:0/3298365863 shutdown_connections 2024-04-26T21:47:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.973+0000 7fbb64bac640 1 -- 172.21.15.40:0/3298365863 >> 172.21.15.40:0/3298365863 conn(0x7fbb600d4fd0 msgr2=0x7fbb600d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.973+0000 7fbb64bac640 1 -- 172.21.15.40:0/3298365863 shutdown_connections 2024-04-26T21:47:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.973+0000 7fbb64bac640 1 -- 172.21.15.40:0/3298365863 wait complete. 2024-04-26T21:47:05.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.974+0000 7fbb64bac640 1 Processor -- start 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb64bac640 1 -- start start 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb64bac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c004390 0x7fbb601e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb64bac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600ba030 0x7fbb60150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb64bac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600ba4b0 0x7fbb60150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb64bac640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60163090 con 0x7fbb600ba030 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb64bac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60162f10 con 0x7fbb4c004390 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb64bac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb600d1de0 con 0x7fbb600ba4b0 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600ba030 0x7fbb60150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600ba030 0x7fbb60150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46982/0 (socket says 172.21.15.40:46982) 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb57fff640 1 -- 172.21.15.40:0/108961848 learned_addr learned my addr 172.21.15.40:0/108961848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb5ed9d640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600ba4b0 0x7fbb60150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.975+0000 7fbb5e59c640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c004390 0x7fbb601e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb57fff640 1 -- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600ba4b0 msgr2=0x7fbb60150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb57fff640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600ba4b0 0x7fbb60150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb57fff640 1 -- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c004390 msgr2=0x7fbb601e97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb57fff640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c004390 0x7fbb601e97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb57fff640 1 -- 172.21.15.40:0/108961848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb601f4f70 con 0x7fbb600ba030 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb57fff640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600ba030 0x7fbb60150010 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fbb4802a6c0 tx=0x7fbb48001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb56ffd640 1 -- 172.21.15.40:0/108961848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4803f020 con 0x7fbb600ba030 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb56ffd640 1 -- 172.21.15.40:0/108961848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb48005270 con 0x7fbb600ba030 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb601f5250 con 0x7fbb600ba030 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb56ffd640 1 -- 172.21.15.40:0/108961848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48037040 con 0x7fbb600ba030 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.976+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601f5810 con 0x7fbb600ba030 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.977+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb600694f0 con 0x7fbb600ba030 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.978+0000 7fbb56ffd640 1 -- 172.21.15.40:0/108961848 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb48042020 con 0x7fbb600ba030 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.978+0000 7fbb56ffd640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb4006c8f0 0x7fbb4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.978+0000 7fbb5e59c640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb4006c8f0 0x7fbb4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.979+0000 7fbb5e59c640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb4006c8f0 0x7fbb4006edb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009bc0 tx=0x7fbb50009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:05.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.981+0000 7fbb56ffd640 1 -- 172.21.15.40:0/108961848 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7fbb480014b0 con 0x7fbb600ba030 2024-04-26T21:47:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:05.986+0000 7fbb56ffd640 1 -- 172.21.15.40:0/108961848 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb48049030 con 0x7fbb600ba030 2024-04-26T21:47:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.173+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb600bac20 con 0x7fbb600ba030 2024-04-26T21:47:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.174+0000 7fbb56ffd640 1 -- 172.21.15.40:0/108961848 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v135) ==== 100+0+11 (secure 0 0 0) 0x7fbb480796f0 con 0x7fbb600ba030 2024-04-26T21:47:06.175 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb4006c8f0 msgr2=0x7fbb4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb4006c8f0 0x7fbb4006edb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009bc0 tx=0x7fbb50009190 comp rx=0 tx=0).stop 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600ba030 msgr2=0x7fbb60150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600ba030 0x7fbb60150010 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fbb4802a6c0 tx=0x7fbb48001230 comp rx=0 tx=0).stop 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 shutdown_connections 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbb4006c8f0 0x7fbb4006edb0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb600ba4b0 0x7fbb60150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600ba030 0x7fbb60150010 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 --2- 172.21.15.40:0/108961848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb4c004390 0x7fbb601e97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 >> 172.21.15.40:0/108961848 conn(0x7fbb600d4fd0 msgr2=0x7fbb6006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 shutdown_connections 2024-04-26T21:47:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.176+0000 7fbb64bac640 1 -- 172.21.15.40:0/108961848 wait complete. 2024-04-26T21:47:06.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.288+0000 7f5ba8924640 1 Processor -- start 2024-04-26T21:47:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.288+0000 7f5ba8924640 1 -- start start 2024-04-26T21:47:06.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba8924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba40b5f20 0x7f5ba40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba8924640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba40b7e50 con 0x7f5ba40d3bc0 2024-04-26T21:47:06.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba8924640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba40d1800 con 0x7f5ba4162880 2024-04-26T21:47:06.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba8924640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba40d0580 con 0x7f5ba40b5f20 2024-04-26T21:47:06.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba1d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5ba40d3bc0 0x7f5ba40b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34564/0 (socket says 172.21.15.40:34564) 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba1d9b640 1 -- 172.21.15.40:0/1900358611 learned_addr learned my addr 172.21.15.40:0/1900358611 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba2d9d640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba40b5f20 0x7f5ba40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3842262294 0 0) 0x7f5ba40b7e50 con 0x7f5ba40d3bc0 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b8c003520 con 0x7f5ba40d3bc0 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3102757240 0 0) 0x7f5b8c003520 con 0x7f5ba40d3bc0 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba40b5f20 msgr2=0x7f5ba40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba40b5f20 0x7f5ba40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 >> v1:172.21.15.184:6789/0 conn(0x7f5ba4162880 legacy=0x7f5ba40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba40d07d0 con 0x7f5ba40d3bc0 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f5b98001670 con 0x7f5ba40d3bc0 2024-04-26T21:47:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 >> v1:172.21.15.40:6789/0 conn(0x7f5ba40d3bc0 legacy=0x7f5ba40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c0039e0 0x7f5b8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba40b5f20 0x7f5b8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b8c004d40 0x7f5b8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40d0580 con 0x7f5b8c004d40 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c00ba70 con 0x7f5b8c0039e0 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c00bbf0 con 0x7f5ba40b5f20 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba2d9d640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b8c004d40 0x7f5b8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.289+0000 7f5ba259c640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c0039e0 0x7f5b8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.290+0000 7f5ba1d9b640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba40b5f20 0x7f5b8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.290+0000 7f5ba2d9d640 1 -- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba40b5f20 msgr2=0x7f5b8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.290+0000 7f5ba2d9d640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba40b5f20 0x7f5b8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.290+0000 7f5ba2d9d640 1 -- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c0039e0 msgr2=0x7f5b8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.290+0000 7f5ba2d9d640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c0039e0 0x7f5b8c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.290+0000 7f5ba2d9d640 1 -- 172.21.15.40:0/1900358611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c003520 con 0x7f5b8c004d40 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.290+0000 7f5ba2d9d640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b8c004d40 0x7f5b8c0051c0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009fc0 tx=0x7f5b8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb0abcca6fdac85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c009c10 con 0x7f5b8c004d40 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b9c036070 con 0x7f5b8c004d40 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba159a640 1 -- 172.21.15.40:0/1900358611 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c031880 con 0x7f5b8c004d40 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba8924640 1 -- 172.21.15.40:0/1900358611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b8c004d40 msgr2=0x7f5b8c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba8924640 1 --2- 172.21.15.40:0/1900358611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b8c004d40 0x7f5b8c0051c0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009fc0 tx=0x7f5b8c005830 comp rx=0 tx=0).stop 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba2d9d640 1 -- 172.21.15.40:0/1900358611 reap_dead start 2024-04-26T21:47:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba8924640 1 -- 172.21.15.40:0/1900358611 shutdown_connections 2024-04-26T21:47:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba8924640 1 -- 172.21.15.40:0/1900358611 >> 172.21.15.40:0/1900358611 conn(0x7f5ba40d4f60 msgr2=0x7f5ba40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba8924640 1 -- 172.21.15.40:0/1900358611 shutdown_connections 2024-04-26T21:47:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.291+0000 7f5ba8924640 1 -- 172.21.15.40:0/1900358611 wait complete. 2024-04-26T21:47:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba8924640 1 Processor -- start 2024-04-26T21:47:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba8924640 1 -- start start 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba8924640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c004d40 0x7f5ba41f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba8924640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba415d610 0x7f5ba41f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba8924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba41f50a0 0x7f5ba406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba8924640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4162c60 con 0x7f5ba415d610 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba8924640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40d0d90 con 0x7f5b8c004d40 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba8924640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba4162de0 con 0x7f5ba41f50a0 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.293+0000 7f5ba1d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba415d610 0x7f5ba41f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba1d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba415d610 0x7f5ba41f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47012/0 (socket says 172.21.15.40:47012) 2024-04-26T21:47:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba1d9b640 1 -- 172.21.15.40:0/2709101220 learned_addr learned my addr 172.21.15.40:0/2709101220 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba2d9d640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba41f50a0 0x7f5ba406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba259c640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c004d40 0x7f5ba41f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba1d9b640 1 -- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba41f50a0 msgr2=0x7f5ba406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba1d9b640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba41f50a0 0x7f5ba406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba1d9b640 1 -- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c004d40 msgr2=0x7f5ba41f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba1d9b640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c004d40 0x7f5ba41f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba1d9b640 1 -- 172.21.15.40:0/2709101220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba41f5630 con 0x7f5ba415d610 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba259c640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c004d40 0x7f5ba41f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba1d9b640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba415d610 0x7f5ba41f4b60 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f5b980072c0 tx=0x7f5b980076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5b92ffd640 1 -- 172.21.15.40:0/2709101220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9800f960 con 0x7f5ba415d610 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5b92ffd640 1 -- 172.21.15.40:0/2709101220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b98018070 con 0x7f5ba415d610 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba41f5910 con 0x7f5ba415d610 2024-04-26T21:47:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5b92ffd640 1 -- 172.21.15.40:0/2709101220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9800fd80 con 0x7f5ba415d610 2024-04-26T21:47:06.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.294+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba415da90 con 0x7f5ba415d610 2024-04-26T21:47:06.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.295+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba40694f0 con 0x7f5ba415d610 2024-04-26T21:47:06.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.296+0000 7f5b92ffd640 1 -- 172.21.15.40:0/2709101220 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b9801d460 con 0x7f5ba415d610 2024-04-26T21:47:06.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.296+0000 7f5b92ffd640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b8406c8f0 0x7f5b8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.296+0000 7f5b92ffd640 1 -- 172.21.15.40:0/2709101220 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f5b9808cc00 con 0x7f5ba415d610 2024-04-26T21:47:06.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.296+0000 7f5ba259c640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b8406c8f0 0x7f5b8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.297+0000 7f5ba259c640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b8406c8f0 0x7f5b8406edb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5b94005500 tx=0x7f5b94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.300+0000 7f5b92ffd640 1 -- 172.21.15.40:0/2709101220 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b98093080 con 0x7f5ba415d610 2024-04-26T21:47:06.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.527+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ba40d3030 con 0x7f5ba415d610 2024-04-26T21:47:06.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.527+0000 7f5b92ffd640 1 -- 172.21.15.40:0/2709101220 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5182 (secure 0 0 0) 0x7f5b9805a100 con 0x7f5ba415d610 2024-04-26T21:47:06.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:06.528 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1002792848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1002792848},{"type":"v1","addr":"172.21.15.40:6833","nonce":1002792848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:47:00.954510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:47:06.529 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 350 2024-04-26T21:47:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b8406c8f0 msgr2=0x7f5b8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b8406c8f0 0x7f5b8406edb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5b94005500 tx=0x7f5b94009040 comp rx=0 tx=0).stop 2024-04-26T21:47:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba415d610 msgr2=0x7f5ba41f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba415d610 0x7f5ba41f4b60 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f5b980072c0 tx=0x7f5b980076c0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 shutdown_connections 2024-04-26T21:47:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f5b8406c8f0 0x7f5b8406edb0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ba41f50a0 0x7f5ba406dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba415d610 0x7f5ba41f4b60 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 --2- 172.21.15.40:0/2709101220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b8c004d40 0x7f5ba41f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 >> 172.21.15.40:0/2709101220 conn(0x7f5ba40d4f60 msgr2=0x7f5ba40ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 shutdown_connections 2024-04-26T21:47:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.529+0000 7f5ba8924640 1 -- 172.21.15.40:0/2709101220 wait complete. 2024-04-26T21:47:06.541 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:06.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.640+0000 7f425dcc5640 1 Processor -- start 2024-04-26T21:47:06.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.641+0000 7f425dcc5640 1 -- start start 2024-04-26T21:47:06.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.641+0000 7f425dcc5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f425815f3a0 0x7f42580d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.641+0000 7f425dcc5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f42580d1770 0x7f42580d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.641+0000 7f425dcc5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42580d2590 con 0x7f42580d2160 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.641+0000 7f425dcc5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f42580d0e20 con 0x7f425815f3a0 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.641+0000 7f425dcc5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f42580b1d70 con 0x7f42580d1770 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.641+0000 7f42577fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f425815f3a0 0x7f42580d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.642+0000 7f42577fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f425815f3a0 0x7f42580d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47022/0 (socket says 172.21.15.40:47022) 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.642+0000 7f42577fe640 1 -- 172.21.15.40:0/865025651 learned_addr learned my addr 172.21.15.40:0/865025651 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.642+0000 7f4256ffd640 1 --2- 172.21.15.40:0/865025651 >> v2:172.21.15.184:3300/0 conn(0x7f42580d1770 0x7f42580d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.642+0000 7f42577fe640 1 -- 172.21.15.40:0/865025651 >> v2:172.21.15.184:3300/0 conn(0x7f42580d1770 msgr2=0x7f42580d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.642+0000 7f42577fe640 1 --2- 172.21.15.40:0/865025651 >> v2:172.21.15.184:3300/0 conn(0x7f42580d1770 0x7f42580d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.642+0000 7f42577fe640 1 -- 172.21.15.40:0/865025651 >> v1:172.21.15.184:6789/0 conn(0x7f42580d2160 legacy=0x7f42580d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:06.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.642+0000 7f42577fe640 1 -- 172.21.15.40:0/865025651 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42580b2050 con 0x7f425815f3a0 2024-04-26T21:47:06.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.642+0000 7f42577fe640 1 --2- 172.21.15.40:0/865025651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42580d06f0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f42580b7b80 tx=0x7f424802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95866efe83aab5fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f42567fc640 1 -- 172.21.15.40:0/865025651 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248037070 con 0x7f425815f3a0 2024-04-26T21:47:06.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f42567fc640 1 -- 172.21.15.40:0/865025651 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4248032440 con 0x7f425815f3a0 2024-04-26T21:47:06.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f42567fc640 1 -- 172.21.15.40:0/865025651 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248032810 con 0x7f425815f3a0 2024-04-26T21:47:06.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f425dcc5640 1 -- 172.21.15.40:0/865025651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 msgr2=0x7f42580d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f425dcc5640 1 --2- 172.21.15.40:0/865025651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42580d06f0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f42580b7b80 tx=0x7f424802a230 comp rx=0 tx=0).stop 2024-04-26T21:47:06.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f425dcc5640 1 -- 172.21.15.40:0/865025651 shutdown_connections 2024-04-26T21:47:06.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f425dcc5640 1 --2- 172.21.15.40:0/865025651 >> v2:172.21.15.184:3300/0 conn(0x7f42580d1770 0x7f42580d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f425dcc5640 1 --2- 172.21.15.40:0/865025651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42580d06f0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f425dcc5640 1 -- 172.21.15.40:0/865025651 >> 172.21.15.40:0/865025651 conn(0x7f42580d4f90 msgr2=0x7f42580d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f425dcc5640 1 -- 172.21.15.40:0/865025651 shutdown_connections 2024-04-26T21:47:06.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.643+0000 7f425dcc5640 1 -- 172.21.15.40:0/865025651 wait complete. 2024-04-26T21:47:06.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.644+0000 7f425dcc5640 1 Processor -- start 2024-04-26T21:47:06.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f425dcc5640 1 -- start start 2024-04-26T21:47:06.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f425dcc5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580d1770 0x7f42581f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f425dcc5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580d2160 0x7f425815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f425dcc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42581f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f425dcc5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f425815f900 con 0x7f425815f3a0 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f425dcc5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f425815f780 con 0x7f42580d2160 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f425dcc5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f42580b3860 con 0x7f42580d1770 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f4257fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42581f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f4257fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42581f78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47036/0 (socket says 172.21.15.40:47036) 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.645+0000 7f4257fff640 1 -- 172.21.15.40:0/375316036 learned_addr learned my addr 172.21.15.40:0/375316036 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f42577fe640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580d1770 0x7f42581f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f4256ffd640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580d2160 0x7f425815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f4257fff640 1 -- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580d1770 msgr2=0x7f42581f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f4257fff640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580d1770 0x7f42581f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f4257fff640 1 -- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580d2160 msgr2=0x7f425815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f4257fff640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580d2160 0x7f425815d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f4257fff640 1 -- 172.21.15.40:0/375316036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42581607b0 con 0x7f425815f3a0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f42577fe640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580d1770 0x7f42581f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f4257fff640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42581f78a0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f425000c780 tx=0x7f425000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f423ffff640 1 -- 172.21.15.40:0/375316036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250004b60 con 0x7f425815f3a0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f423ffff640 1 -- 172.21.15.40:0/375316036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4250004d00 con 0x7f425815f3a0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4258161250 con 0x7f425815f3a0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f423ffff640 1 -- 172.21.15.40:0/375316036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425000aa90 con 0x7f425815f3a0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.646+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4258161810 con 0x7f425815f3a0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.647+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4258161a10 con 0x7f425815f3a0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.648+0000 7f423ffff640 1 -- 172.21.15.40:0/375316036 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f425000ac30 con 0x7f425815f3a0 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.648+0000 7f423ffff640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f424406c8f0 0x7f424406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.648+0000 7f423ffff640 1 -- 172.21.15.40:0/375316036 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f425008ce40 con 0x7f425815f3a0 2024-04-26T21:47:06.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.648+0000 7f42577fe640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f424406c8f0 0x7f424406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.649+0000 7f42577fe640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f424406c8f0 0x7f424406edb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f4248034000 tx=0x7f424802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.652+0000 7f423ffff640 1 -- 172.21.15.40:0/375316036 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4258161a10 con 0x7f425815f3a0 2024-04-26T21:47:06.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.838+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42580694f0 con 0x7f425815f3a0 2024-04-26T21:47:06.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.839+0000 7f423ffff640 1 -- 172.21.15.40:0/375316036 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+22228 (secure 0 0 0) 0x7f4250020070 con 0x7f425815f3a0 2024-04-26T21:47:06.841 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:06.841 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":135,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:00.954261+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/933356185":"2024-04-27T21:47:00.954240+0000","172.21.15.40:6832/933356185":"2024-04-27T21:47:00.954240+0000","172.21.15.40:6833/3095512631":"2024-04-27T21:46:53.909685+0000","172.21.15.40:6833/3502494134":"2024-04-27T21:46:46.873417+0000","172.21.15.40:6833/1769689018":"2024-04-27T21:46:41.828993+0000","172.21.15.40:6833/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6832/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6833/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6832/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6833/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6833/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6832/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6833/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6833/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6832/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6832/3502494134":"2024-04-27T21:46:46.873417+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/3095512631":"2024-04-27T21:46:53.909685+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3538213517":"2024-04-27T21:45:17.788819+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6832/1769689018":"2024-04-27T21:46:41.828993+0000","172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/3538213517":"2024-04-27T21:45:17.788819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.841+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f424406c8f0 msgr2=0x7f424406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.841+0000 7f425dcc5640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f424406c8f0 0x7f424406edb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f4248034000 tx=0x7f424802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 msgr2=0x7f42581f78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42581f78a0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f425000c780 tx=0x7f425000cc40 comp rx=0 tx=0).stop 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 shutdown_connections 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f424406c8f0 0x7f424406edb0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425815f3a0 0x7f42581f78a0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580d2160 0x7f425815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 --2- 172.21.15.40:0/375316036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580d1770 0x7f42581f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 >> 172.21.15.40:0/375316036 conn(0x7f42580d4f90 msgr2=0x7f425806e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 shutdown_connections 2024-04-26T21:47:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.842+0000 7f425dcc5640 1 -- 172.21.15.40:0/375316036 wait complete. 2024-04-26T21:47:06.854 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:47:06.854 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:06.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.954+0000 7f212d922640 1 Processor -- start 2024-04-26T21:47:06.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.954+0000 7f212d922640 1 -- start start 2024-04-26T21:47:06.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f212d922640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21280d06f0 0x7f21280d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f212d922640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21280b2850 0x7f21280b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f212d922640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21280d0ba0 con 0x7f212815f3a0 2024-04-26T21:47:06.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f212d922640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f21280d0e20 con 0x7f21280d06f0 2024-04-26T21:47:06.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f212d922640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f21280b1d70 con 0x7f21280b2850 2024-04-26T21:47:06.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f21267fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21280d06f0 0x7f21280d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f2126ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f212815f3a0 0x7f21280d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34576/0 (socket says 172.21.15.40:34576) 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f21267fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21280d06f0 0x7f21280d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47050/0 (socket says 172.21.15.40:47050) 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f2126ffd640 1 -- 172.21.15.40:0/3421397486 learned_addr learned my addr 172.21.15.40:0/3421397486 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.955+0000 7f21277fe640 1 --2- 172.21.15.40:0/3421397486 >> v2:172.21.15.184:3300/0 conn(0x7f21280b2850 0x7f21280b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f21267fc640 1 -- 172.21.15.40:0/3421397486 >> v2:172.21.15.184:3300/0 conn(0x7f21280b2850 msgr2=0x7f21280b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f21267fc640 1 --2- 172.21.15.40:0/3421397486 >> v2:172.21.15.184:3300/0 conn(0x7f21280b2850 0x7f21280b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f21267fc640 1 -- 172.21.15.40:0/3421397486 >> v1:172.21.15.40:6789/0 conn(0x7f212815f3a0 legacy=0x7f21280d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f21267fc640 1 -- 172.21.15.40:0/3421397486 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21280b2050 con 0x7f21280d06f0 2024-04-26T21:47:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f2125ffb640 1 -- 172.21.15.40:0/3421397486 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201836115 0 0) 0x7f2118001110 con 0x7f212815f3a0 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f21267fc640 1 --2- 172.21.15.40:0/3421397486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280d06f0 0x7f21280d1e80 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f21280c89f0 tx=0x7f211c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47baf2ce664b2b5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f2125ffb640 1 -- 172.21.15.40:0/3421397486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c037070 con 0x7f21280d06f0 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f2125ffb640 1 -- 172.21.15.40:0/3421397486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c032440 con 0x7f21280d06f0 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.956+0000 7f2125ffb640 1 -- 172.21.15.40:0/3421397486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c032810 con 0x7f21280d06f0 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.957+0000 7f212d922640 1 -- 172.21.15.40:0/3421397486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280d06f0 msgr2=0x7f21280d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.957+0000 7f212d922640 1 --2- 172.21.15.40:0/3421397486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280d06f0 0x7f21280d1e80 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f21280c89f0 tx=0x7f211c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.957+0000 7f212d922640 1 -- 172.21.15.40:0/3421397486 shutdown_connections 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.957+0000 7f212d922640 1 --2- 172.21.15.40:0/3421397486 >> v2:172.21.15.184:3300/0 conn(0x7f21280b2850 0x7f21280b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.957+0000 7f212d922640 1 --2- 172.21.15.40:0/3421397486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280d06f0 0x7f21280d1e80 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.957+0000 7f212d922640 1 -- 172.21.15.40:0/3421397486 >> 172.21.15.40:0/3421397486 conn(0x7f21280d4f90 msgr2=0x7f21280d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.957+0000 7f212d922640 1 -- 172.21.15.40:0/3421397486 shutdown_connections 2024-04-26T21:47:06.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.957+0000 7f212d922640 1 -- 172.21.15.40:0/3421397486 wait complete. 2024-04-26T21:47:06.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.958+0000 7f212d922640 1 Processor -- start 2024-04-26T21:47:06.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.958+0000 7f212d922640 1 -- start start 2024-04-26T21:47:06.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f212d922640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280b2850 0x7f2128159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f212d922640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21280d06f0 0x7f212815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f212d922640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f212815f3a0 0x7f212815df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f212d922640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f212815f900 con 0x7f21280b2850 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f212d922640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f212815f780 con 0x7f21280d06f0 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f212d922640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21280d23c0 con 0x7f212815f3a0 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f2126ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280b2850 0x7f2128159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f2126ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280b2850 0x7f2128159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47066/0 (socket says 172.21.15.40:47066) 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f2126ffd640 1 -- 172.21.15.40:0/1373587335 learned_addr learned my addr 172.21.15.40:0/1373587335 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:06.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f2126ffd640 1 -- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f212815f3a0 msgr2=0x7f212815df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f2126ffd640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f212815f3a0 0x7f212815df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f2126ffd640 1 -- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21280d06f0 msgr2=0x7f212815da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f2126ffd640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21280d06f0 0x7f212815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.959+0000 7f2126ffd640 1 -- 172.21.15.40:0/1373587335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21280b7c20 con 0x7f21280b2850 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.960+0000 7f2126ffd640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280b2850 0x7f2128159950 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f211800c430 tx=0x7f211800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.960+0000 7f210ffff640 1 -- 172.21.15.40:0/1373587335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180074e0 con 0x7f21280b2850 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.960+0000 7f210ffff640 1 -- 172.21.15.40:0/1373587335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21180047e0 con 0x7f21280b2850 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.960+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21280b7f00 con 0x7f21280b2850 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.960+0000 7f210ffff640 1 -- 172.21.15.40:0/1373587335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118004bd0 con 0x7f21280b2850 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.960+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21280b84c0 con 0x7f21280b2850 2024-04-26T21:47:06.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.961+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21280d3ec0 con 0x7f21280b2850 2024-04-26T21:47:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.961+0000 7f210ffff640 1 -- 172.21.15.40:0/1373587335 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2118011070 con 0x7f21280b2850 2024-04-26T21:47:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.962+0000 7f210ffff640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f211406c920 0x7f211406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.962+0000 7f210ffff640 1 -- 172.21.15.40:0/1373587335 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f211808c790 con 0x7f21280b2850 2024-04-26T21:47:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.962+0000 7f21267fc640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f211406c920 0x7f211406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.962+0000 7f21267fc640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f211406c920 0x7f211406ede0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f211c034000 tx=0x7f211c02c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:06.966+0000 7f210ffff640 1 -- 172.21.15.40:0/1373587335 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21280d3ec0 con 0x7f21280b2850 2024-04-26T21:47:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.191+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21280694f0 con 0x7f21280b2850 2024-04-26T21:47:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.192+0000 7f210ffff640 1 -- 172.21.15.40:0/1373587335 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5182 (secure 0 0 0) 0x7f21280d0ba0 con 0x7f21280b2850 2024-04-26T21:47:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:07.193 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1002792848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1002792848},{"type":"v1","addr":"172.21.15.40:6833","nonce":1002792848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:47:00.954510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:47:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 350 2024-04-26T21:47:07.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f211406c920 msgr2=0x7f211406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f211406c920 0x7f211406ede0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f211c034000 tx=0x7f211c02c000 comp rx=0 tx=0).stop 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280b2850 msgr2=0x7f2128159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280b2850 0x7f2128159950 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f211800c430 tx=0x7f211800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 shutdown_connections 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f211406c920 0x7f211406ede0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f212815f3a0 0x7f212815df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21280d06f0 0x7f212815da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 --2- 172.21.15.40:0/1373587335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21280b2850 0x7f2128159950 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 >> 172.21.15.40:0/1373587335 conn(0x7f21280d4f90 msgr2=0x7f21280d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 shutdown_connections 2024-04-26T21:47:07.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.194+0000 7f212d922640 1 -- 172.21.15.40:0/1373587335 wait complete. 2024-04-26T21:47:07.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:07.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.305+0000 7f7074f24640 1 Processor -- start 2024-04-26T21:47:07.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.305+0000 7f7074f24640 1 -- start start 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.306+0000 7f7074f24640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7070070750 0x7f70700b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.306+0000 7f7074f24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70700af6e0 0x7f70700afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.306+0000 7f7074f24640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70700b5e30 con 0x7f7070072fb0 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.306+0000 7f7074f24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f70700b5c00 con 0x7f70700af6e0 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.306+0000 7f7074f24640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f707006f2b0 con 0x7f7070070750 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.306+0000 7f706ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70700af6e0 0x7f70700afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.306+0000 7f706e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7070072fb0 0x7f7070073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34588/0 (socket says 172.21.15.40:34588) 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.306+0000 7f706e59c640 1 -- 172.21.15.40:0/2091792950 learned_addr learned my addr 172.21.15.40:0/2091792950 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.307+0000 7f706ed9d640 1 -- 172.21.15.40:0/2091792950 >> v2:172.21.15.184:3300/0 conn(0x7f7070070750 msgr2=0x7f70700b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.307+0000 7f706ed9d640 1 --2- 172.21.15.40:0/2091792950 >> v2:172.21.15.184:3300/0 conn(0x7f7070070750 0x7f70700b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.307+0000 7f706ed9d640 1 -- 172.21.15.40:0/2091792950 >> v1:172.21.15.40:6789/0 conn(0x7f7070072fb0 legacy=0x7f7070073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.307+0000 7f706ed9d640 1 -- 172.21.15.40:0/2091792950 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70700b6110 con 0x7f70700af6e0 2024-04-26T21:47:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.307+0000 7f706d59a640 1 -- 172.21.15.40:0/2091792950 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431173077 0 0) 0x7f70700b5e30 con 0x7f7070072fb0 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f706ed9d640 1 --2- 172.21.15.40:0/2091792950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70700af6e0 0x7f70700afb70 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f70700a00e0 tx=0x7f706802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3484336a54871574 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f706d59a640 1 -- 172.21.15.40:0/2091792950 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068037070 con 0x7f70700af6e0 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f706d59a640 1 -- 172.21.15.40:0/2091792950 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7068032440 con 0x7f70700af6e0 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f706d59a640 1 -- 172.21.15.40:0/2091792950 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068032810 con 0x7f70700af6e0 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f7074f24640 1 -- 172.21.15.40:0/2091792950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70700af6e0 msgr2=0x7f70700afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f7074f24640 1 --2- 172.21.15.40:0/2091792950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70700af6e0 0x7f70700afb70 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f70700a00e0 tx=0x7f706802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f7074f24640 1 -- 172.21.15.40:0/2091792950 shutdown_connections 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f7074f24640 1 --2- 172.21.15.40:0/2091792950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70700af6e0 0x7f70700afb70 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f7074f24640 1 --2- 172.21.15.40:0/2091792950 >> v2:172.21.15.184:3300/0 conn(0x7f7070070750 0x7f70700b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f7074f24640 1 -- 172.21.15.40:0/2091792950 >> 172.21.15.40:0/2091792950 conn(0x7f70700b39b0 msgr2=0x7f70700b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f7074f24640 1 -- 172.21.15.40:0/2091792950 shutdown_connections 2024-04-26T21:47:07.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.308+0000 7f7074f24640 1 -- 172.21.15.40:0/2091792950 wait complete. 2024-04-26T21:47:07.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.310+0000 7f7074f24640 1 Processor -- start 2024-04-26T21:47:07.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.310+0000 7f7074f24640 1 -- start start 2024-04-26T21:47:07.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.310+0000 7f7074f24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7070070750 0x7f707009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.310+0000 7f7074f24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7070072fb0 0x7f707014c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.310+0000 7f7074f24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70700af6e0 0x7f70701571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f7074f24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f707015c2e0 con 0x7f7070072fb0 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f7074f24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f707015c160 con 0x7f70700af6e0 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f7074f24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70700b00b0 con 0x7f7070070750 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7070072fb0 0x7f707014c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7070072fb0 0x7f707014c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47096/0 (socket says 172.21.15.40:47096) 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 -- 172.21.15.40:0/3518416778 learned_addr learned my addr 172.21.15.40:0/3518416778 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706e59c640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7070070750 0x7f707009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706ed9d640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70700af6e0 0x7f70701571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 -- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7070070750 msgr2=0x7f707009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7070070750 0x7f707009ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 -- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70700af6e0 msgr2=0x7f70701571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70700af6e0 0x7f70701571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 -- 172.21.15.40:0/3518416778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707014cc10 con 0x7f7070072fb0 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706ed9d640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70700af6e0 0x7f70701571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f706dd9b640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7070072fb0 0x7f707014c6d0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f705800b4a0 tx=0x7f705800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f7066ffd640 1 -- 172.21.15.40:0/3518416778 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058014070 con 0x7f7070072fb0 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f7066ffd640 1 -- 172.21.15.40:0/3518416778 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7058010610 con 0x7f7070072fb0 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f7066ffd640 1 -- 172.21.15.40:0/3518416778 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058010a00 con 0x7f7070072fb0 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70700d2020 con 0x7f7070072fb0 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.311+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70700cf9d0 con 0x7f7070072fb0 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.312+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70700694f0 con 0x7f7070072fb0 2024-04-26T21:47:07.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.313+0000 7f7066ffd640 1 -- 172.21.15.40:0/3518416778 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7058017070 con 0x7f7070072fb0 2024-04-26T21:47:07.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.313+0000 7f7066ffd640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f705006c8f0 0x7f705006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.313+0000 7f7066ffd640 1 -- 172.21.15.40:0/3518416778 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f705808c4d0 con 0x7f7070072fb0 2024-04-26T21:47:07.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.313+0000 7f706e59c640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f705006c8f0 0x7f705006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.314+0000 7f706e59c640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f705006c8f0 0x7f705006edb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f70600012e0 tx=0x7f7060007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.317+0000 7f7066ffd640 1 -- 172.21.15.40:0/3518416778 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70580599d0 con 0x7f7070072fb0 2024-04-26T21:47:07.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.539+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70700cfba0 con 0x7f7070072fb0 2024-04-26T21:47:07.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.540+0000 7f7066ffd640 1 -- 172.21.15.40:0/3518416778 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5182 (secure 0 0 0) 0x7f70580592d0 con 0x7f7070072fb0 2024-04-26T21:47:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:07.541 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1002792848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1002792848},{"type":"v1","addr":"172.21.15.40:6833","nonce":1002792848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:42:08.607413+0000","modified":"2024-04-26T21:47:00.954510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-26T21:47:07.541 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 350 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f705006c8f0 msgr2=0x7f705006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f705006c8f0 0x7f705006edb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f70600012e0 tx=0x7f7060007ed0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7070072fb0 msgr2=0x7f707014c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7070072fb0 0x7f707014c6d0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f705800b4a0 tx=0x7f705800b960 comp rx=0 tx=0).stop 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 shutdown_connections 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f705006c8f0 0x7f705006edb0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70700af6e0 0x7f70701571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7070072fb0 0x7f707014c6d0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 --2- 172.21.15.40:0/3518416778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7070070750 0x7f707009ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 >> 172.21.15.40:0/3518416778 conn(0x7f70700b39b0 msgr2=0x7f707006c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 shutdown_connections 2024-04-26T21:47:07.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.542+0000 7f7074f24640 1 -- 172.21.15.40:0/3518416778 wait complete. 2024-04-26T21:47:07.554 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:07.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.654+0000 7fe2ae2ea640 1 Processor -- start 2024-04-26T21:47:07.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.654+0000 7fe2ae2ea640 1 -- start start 2024-04-26T21:47:07.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.654+0000 7fe2ae2ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2a8156460 0x7fe2a80cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.654+0000 7fe2ae2ea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a8156840 con 0x7fe2a80cba10 2024-04-26T21:47:07.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.654+0000 7fe2ae2ea640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a80cbf20 con 0x7fe2a806fea0 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.654+0000 7fe2ae2ea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a80cc200 con 0x7fe2a8156460 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.655+0000 7fe2a7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2a8156460 0x7fe2a80cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.655+0000 7fe2a77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe2a80cba10 0x7fe2a806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34592/0 (socket says 172.21.15.40:34592) 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.655+0000 7fe2a77fe640 1 -- 172.21.15.40:0/970623200 learned_addr learned my addr 172.21.15.40:0/970623200 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.655+0000 7fe2a7fff640 1 -- 172.21.15.40:0/970623200 >> v1:172.21.15.184:6789/0 conn(0x7fe2a806fea0 legacy=0x7fe2a8072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.655+0000 7fe2a7fff640 1 -- 172.21.15.40:0/970623200 >> v1:172.21.15.40:6789/0 conn(0x7fe2a80cba10 legacy=0x7fe2a806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.655+0000 7fe2a7fff640 1 -- 172.21.15.40:0/970623200 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a80cc4e0 con 0x7fe2a8156460 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.655+0000 7fe2a6ffd640 1 -- 172.21.15.40:0/970623200 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838641255 0 0) 0x7fe2a8156840 con 0x7fe2a80cba10 2024-04-26T21:47:07.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2a7fff640 1 --2- 172.21.15.40:0/970623200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a8156460 0x7fe2a80cb4d0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fe2980097a0 tx=0x7fe29802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f8222c3b397250 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2a6ffd640 1 -- 172.21.15.40:0/970623200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298037070 con 0x7fe2a8156460 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2a6ffd640 1 -- 172.21.15.40:0/970623200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe298032440 con 0x7fe2a8156460 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2a6ffd640 1 -- 172.21.15.40:0/970623200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298032810 con 0x7fe2a8156460 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/970623200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a8156460 msgr2=0x7fe2a80cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2ae2ea640 1 --2- 172.21.15.40:0/970623200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a8156460 0x7fe2a80cb4d0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fe2980097a0 tx=0x7fe29802a230 comp rx=0 tx=0).stop 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/970623200 shutdown_connections 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2ae2ea640 1 --2- 172.21.15.40:0/970623200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a8156460 0x7fe2a80cb4d0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/970623200 >> 172.21.15.40:0/970623200 conn(0x7fe2a80d0be0 msgr2=0x7fe2a80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/970623200 shutdown_connections 2024-04-26T21:47:07.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.656+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/970623200 wait complete. 2024-04-26T21:47:07.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.658+0000 7fe2ae2ea640 1 Processor -- start 2024-04-26T21:47:07.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.658+0000 7fe2ae2ea640 1 -- start start 2024-04-26T21:47:07.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.658+0000 7fe2ae2ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a806fea0 0x7fe2a814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.658+0000 7fe2ae2ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a80cba10 0x7fe2a80c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2ae2ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 0x7fe2a8150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2ae2ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a806e260 con 0x7fe2a80cba10 2024-04-26T21:47:07.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2ae2ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a8072700 con 0x7fe2a806fea0 2024-04-26T21:47:07.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2ae2ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a806e3e0 con 0x7fe2a8156460 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a80cba10 0x7fe2a80c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a80cba10 0x7fe2a80c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47110/0 (socket says 172.21.15.40:47110) 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 -- 172.21.15.40:0/547100521 learned_addr learned my addr 172.21.15.40:0/547100521 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a7fff640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a806fea0 0x7fe2a814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2ac887640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 0x7fe2a8150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 -- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 msgr2=0x7fe2a8150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 0x7fe2a8150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 -- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a806fea0 msgr2=0x7fe2a814d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a806fea0 0x7fe2a814d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 -- 172.21.15.40:0/547100521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a80c5ed0 con 0x7fe2a80cba10 2024-04-26T21:47:07.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a7fff640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a806fea0 0x7fe2a814d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a77fe640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a80cba10 0x7fe2a80c5120 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00c4a0 tx=0x7fe29c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a4ff9640 1 -- 172.21.15.40:0/547100521 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c018070 con 0x7fe2a80cba10 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a4ff9640 1 -- 172.21.15.40:0/547100521 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c0074e0 con 0x7fe2a80cba10 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2a4ff9640 1 -- 172.21.15.40:0/547100521 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c004a20 con 0x7fe2a80cba10 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a80c5bf0 con 0x7fe2a80cba10 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.659+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a815dc10 con 0x7fe2a80cba10 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.660+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a80694f0 con 0x7fe2a80cba10 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.661+0000 7fe2a4ff9640 1 -- 172.21.15.40:0/547100521 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe29c01b070 con 0x7fe2a80cba10 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.661+0000 7fe2a4ff9640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe29406c8f0 0x7fe29406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.661+0000 7fe2a4ff9640 1 -- 172.21.15.40:0/547100521 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7fe29c08bf10 con 0x7fe2a80cba10 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.662+0000 7fe2a7fff640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe29406c8f0 0x7fe29406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.662+0000 7fe2a7fff640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe29406c8f0 0x7fe29406edb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe298034000 tx=0x7fe29802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.666+0000 7fe2a4ff9640 1 -- 172.21.15.40:0/547100521 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe29c059290 con 0x7fe2a80cba10 2024-04-26T21:47:07.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.853+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2a806de80 con 0x7fe2a80cba10 2024-04-26T21:47:07.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.854+0000 7fe2a4ff9640 1 -- 172.21.15.40:0/547100521 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+22228 (secure 0 0 0) 0x7fe29c058b90 con 0x7fe2a80cba10 2024-04-26T21:47:07.855 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:07.855 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":135,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:00.954261+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:42:06.883998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-26T21:42:07.550973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/933356185":"2024-04-27T21:47:00.954240+0000","172.21.15.40:6832/933356185":"2024-04-27T21:47:00.954240+0000","172.21.15.40:6833/3095512631":"2024-04-27T21:46:53.909685+0000","172.21.15.40:6833/3502494134":"2024-04-27T21:46:46.873417+0000","172.21.15.40:6833/1769689018":"2024-04-27T21:46:41.828993+0000","172.21.15.40:6833/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6832/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6833/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6832/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6833/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6833/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6832/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6833/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6833/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6832/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6832/3502494134":"2024-04-27T21:46:46.873417+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/3095512631":"2024-04-27T21:46:53.909685+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3538213517":"2024-04-27T21:45:17.788819+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6832/1769689018":"2024-04-27T21:46:41.828993+0000","172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/3538213517":"2024-04-27T21:45:17.788819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:07.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe29406c8f0 msgr2=0x7fe29406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe29406c8f0 0x7fe29406edb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe298034000 tx=0x7fe29802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:07.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a80cba10 msgr2=0x7fe2a80c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a80cba10 0x7fe2a80c5120 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00c4a0 tx=0x7fe29c00c960 comp rx=0 tx=0).stop 2024-04-26T21:47:07.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 shutdown_connections 2024-04-26T21:47:07.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe29406c8f0 0x7fe29406edb0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2a8156460 0x7fe2a8150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2a80cba10 0x7fe2a80c5120 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 --2- 172.21.15.40:0/547100521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a806fea0 0x7fe2a814d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 >> 172.21.15.40:0/547100521 conn(0x7fe2a80d0be0 msgr2=0x7fe2a81505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 shutdown_connections 2024-04-26T21:47:07.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.856+0000 7fe2ae2ea640 1 -- 172.21.15.40:0/547100521 wait complete. 2024-04-26T21:47:07.869 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:47:07.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.970+0000 7f15a77e4640 1 Processor -- start 2024-04-26T21:47:07.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.970+0000 7f15a77e4640 1 -- start start 2024-04-26T21:47:07.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.971+0000 7f15a77e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f15a0162880 0x7f15a00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.971+0000 7f15a77e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a00b5f20 0x7f15a00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.971+0000 7f15a77e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a00b7e50 con 0x7f15a00d39f0 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.971+0000 7f15a77e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f15a00d0580 con 0x7f15a0162880 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.971+0000 7f15a77e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15a00d1800 con 0x7f15a00b5f20 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.971+0000 7f15a5d81640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f15a00d39f0 0x7f15a00b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34596/0 (socket says 172.21.15.40:34596) 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.971+0000 7f15a5d81640 1 -- 172.21.15.40:0/1841458828 learned_addr learned my addr 172.21.15.40:0/1841458828 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a4d7f640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a00b5f20 0x7f15a00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3905835152 0 0) 0x7f15a00b7e50 con 0x7f15a00d39f0 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a5580640 1 --2- 172.21.15.40:0/1841458828 >> v2:172.21.15.184:3300/0 conn(0x7f15a0162880 0x7f15a00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f158c003520 con 0x7f15a00d39f0 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2148391898 0 0) 0x7f158c003520 con 0x7f15a00d39f0 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a00b5f20 msgr2=0x7f15a00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a00b5f20 0x7f15a00d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 >> v2:172.21.15.184:3300/0 conn(0x7f15a0162880 msgr2=0x7f15a00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 --2- 172.21.15.40:0/1841458828 >> v2:172.21.15.184:3300/0 conn(0x7f15a0162880 0x7f15a00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a00d07d0 con 0x7f15a00d39f0 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f159c0019f0 con 0x7f15a00d39f0 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 >> v1:172.21.15.40:6789/0 conn(0x7f15a00d39f0 legacy=0x7f15a00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a00b5f20 0x7f158c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f158c003f30 0x7f158c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f158c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15a00d0580 con 0x7f158c004d40 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15a00d1800 con 0x7f158c003f30 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f158c00b9c0 con 0x7f15a00b5f20 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a5d81640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f158c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a5580640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a00b5f20 0x7f158c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a4d7f640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f158c003f30 0x7f158c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a5d81640 1 -- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a00b5f20 msgr2=0x7f158c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a5d81640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a00b5f20 0x7f158c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a5d81640 1 -- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f158c003f30 msgr2=0x7f158c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a5d81640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f158c003f30 0x7f158c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.972+0000 7f15a5d81640 1 -- 172.21.15.40:0/1841458828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c003520 con 0x7f158c004d40 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.973+0000 7f15a5d81640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f158c0051c0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f159c00a8a0 tx=0x7f158c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736b3adfbc216363 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.973+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c00a4c0 con 0x7f158c004d40 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.973+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c02f070 con 0x7f158c004d40 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.973+0000 7f1597fff640 1 -- 172.21.15.40:0/1841458828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c02bd50 con 0x7f158c004d40 2024-04-26T21:47:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.974+0000 7f15a77e4640 1 -- 172.21.15.40:0/1841458828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 msgr2=0x7f158c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.974+0000 7f15a77e4640 1 --2- 172.21.15.40:0/1841458828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f158c0051c0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f159c00a8a0 tx=0x7f158c005830 comp rx=0 tx=0).stop 2024-04-26T21:47:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.974+0000 7f15a5d81640 1 -- 172.21.15.40:0/1841458828 reap_dead start 2024-04-26T21:47:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.974+0000 7f15a77e4640 1 -- 172.21.15.40:0/1841458828 shutdown_connections 2024-04-26T21:47:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.974+0000 7f15a77e4640 1 -- 172.21.15.40:0/1841458828 >> 172.21.15.40:0/1841458828 conn(0x7f15a00d4f60 msgr2=0x7f15a00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.974+0000 7f15a77e4640 1 -- 172.21.15.40:0/1841458828 shutdown_connections 2024-04-26T21:47:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.974+0000 7f15a77e4640 1 -- 172.21.15.40:0/1841458828 wait complete. 2024-04-26T21:47:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.975+0000 7f15a77e4640 1 Processor -- start 2024-04-26T21:47:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.975+0000 7f15a77e4640 1 -- start start 2024-04-26T21:47:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a77e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f15a00cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a77e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a01f8470 0x7f15a0156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a77e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a00b9210 0x7f15a00b9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a77e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0162c60 con 0x7f158c004d40 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a77e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15a00d0d90 con 0x7f15a00b9210 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a77e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15a0162de0 con 0x7f15a01f8470 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f15a00cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f15a00cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47116/0 (socket says 172.21.15.40:47116) 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 -- 172.21.15.40:0/1038899023 learned_addr learned my addr 172.21.15.40:0/1038899023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 -- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a01f8470 msgr2=0x7f15a0156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a01f8470 0x7f15a0156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5d81640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a00b9210 0x7f15a00b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 -- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a00b9210 msgr2=0x7f15a00b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a00b9210 0x7f15a00b9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 -- 172.21.15.40:0/1038899023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a0156e70 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.976+0000 7f15a5580640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f15a00cab50 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f1598011a70 tx=0x7f1598011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.977+0000 7f1595ffb640 1 -- 172.21.15.40:0/1038899023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159800fb50 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.977+0000 7f1595ffb640 1 -- 172.21.15.40:0/1038899023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159800fcf0 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.977+0000 7f1595ffb640 1 -- 172.21.15.40:0/1038899023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159800fea0 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.977+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a01f8a10 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.977+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a01f8fd0 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.978+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a00694f0 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.978+0000 7f1595ffb640 1 -- 172.21.15.40:0/1038899023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1598012040 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.979+0000 7f1595ffb640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f158406c8f0 0x7f158406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.979+0000 7f1595ffb640 1 -- 172.21.15.40:0/1038899023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7f159808d260 con 0x7f158c004d40 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.979+0000 7f15a4d7f640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f158406c8f0 0x7f158406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.980+0000 7f15a4d7f640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f158406c8f0 0x7f158406edb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f15a015f330 tx=0x7f1590009320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:07.984+0000 7f1595ffb640 1 -- 172.21.15.40:0/1038899023 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1598019030 con 0x7f158c004d40 2024-04-26T21:47:08.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.210+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f15a00b9bb0 con 0x7f158c004d40 2024-04-26T21:47:08.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.995+0000 7f1595ffb640 1 -- 172.21.15.40:0/1038899023 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v351) ==== 161+0+0 (secure 0 0 0) 0x7f159805a700 con 0x7f158c004d40 2024-04-26T21:47:08.998 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:47:08.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f158406c8f0 msgr2=0x7f158406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f158406c8f0 0x7f158406edb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f15a015f330 tx=0x7f1590009320 comp rx=0 tx=0).stop 2024-04-26T21:47:08.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 msgr2=0x7f15a00cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:08.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f15a00cab50 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f1598011a70 tx=0x7f1598011f30 comp rx=0 tx=0).stop 2024-04-26T21:47:08.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 shutdown_connections 2024-04-26T21:47:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f158406c8f0 0x7f158406edb0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15a00b9210 0x7f15a00b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15a01f8470 0x7f15a0156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 --2- 172.21.15.40:0/1038899023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f158c004d40 0x7f15a00cab50 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 >> 172.21.15.40:0/1038899023 conn(0x7f15a00d4f60 msgr2=0x7f15a00cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 shutdown_connections 2024-04-26T21:47:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:08.998+0000 7f15a77e4640 1 -- 172.21.15.40:0/1038899023 wait complete. 2024-04-26T21:47:09.013 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:47:09.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.121+0000 7ff90b9c6640 1 Processor -- start 2024-04-26T21:47:09.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.122+0000 7ff90b9c6640 1 -- start start 2024-04-26T21:47:09.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.122+0000 7ff90b9c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff90415f3a0 0x7ff9040d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.122+0000 7ff90b9c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9040d2330 0x7ff9040b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.122+0000 7ff90b9c6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9040d0e20 con 0x7ff9040d1770 2024-04-26T21:47:09.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.122+0000 7ff90b9c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff9040b1d70 con 0x7ff90415f3a0 2024-04-26T21:47:09.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.122+0000 7ff90b9c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff9040b2050 con 0x7ff9040d2330 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.122+0000 7ff909762640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff90415f3a0 0x7ff9040d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff908f61640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff9040d1770 0x7ff9040d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51802/0 (socket says 172.21.15.40:51802) 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff909762640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff90415f3a0 0x7ff9040d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34504/0 (socket says 172.21.15.40:34504) 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff908f61640 1 -- 172.21.15.40:0/3685346194 learned_addr learned my addr 172.21.15.40:0/3685346194 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff909f63640 1 --2- 172.21.15.40:0/3685346194 >> v2:172.21.15.184:3300/0 conn(0x7ff9040d2330 0x7ff9040b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff909762640 1 -- 172.21.15.40:0/3685346194 >> v2:172.21.15.184:3300/0 conn(0x7ff9040d2330 msgr2=0x7ff9040b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff909762640 1 --2- 172.21.15.40:0/3685346194 >> v2:172.21.15.184:3300/0 conn(0x7ff9040d2330 0x7ff9040b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff909762640 1 -- 172.21.15.40:0/3685346194 >> v1:172.21.15.40:6789/0 conn(0x7ff9040d1770 legacy=0x7ff9040d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff909762640 1 -- 172.21.15.40:0/3685346194 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9040b2330 con 0x7ff90415f3a0 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.123+0000 7ff909762640 1 --2- 172.21.15.40:0/3685346194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90415f3a0 0x7ff9040d06f0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7ff9040c89f0 tx=0x7ff8fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff419a885b0e5cec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff903fff640 1 -- 172.21.15.40:0/3685346194 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc037070 con 0x7ff90415f3a0 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff903fff640 1 -- 172.21.15.40:0/3685346194 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8fc032440 con 0x7ff90415f3a0 2024-04-26T21:47:09.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff903fff640 1 -- 172.21.15.40:0/3685346194 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc032810 con 0x7ff90415f3a0 2024-04-26T21:47:09.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff90b9c6640 1 -- 172.21.15.40:0/3685346194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90415f3a0 msgr2=0x7ff9040d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/3685346194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90415f3a0 0x7ff9040d06f0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7ff9040c89f0 tx=0x7ff8fc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff90b9c6640 1 -- 172.21.15.40:0/3685346194 shutdown_connections 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/3685346194 >> v2:172.21.15.184:3300/0 conn(0x7ff9040d2330 0x7ff9040b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/3685346194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90415f3a0 0x7ff9040d06f0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff90b9c6640 1 -- 172.21.15.40:0/3685346194 >> 172.21.15.40:0/3685346194 conn(0x7ff9040d4f90 msgr2=0x7ff9040d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff90b9c6640 1 -- 172.21.15.40:0/3685346194 shutdown_connections 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.124+0000 7ff90b9c6640 1 -- 172.21.15.40:0/3685346194 wait complete. 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.125+0000 7ff90b9c6640 1 Processor -- start 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff90b9c6640 1 -- start start 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff90b9c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9040d1770 0x7ff9040d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff90b9c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9040d2330 0x7ff9041e9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff90b9c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff90415f3a0 0x7ff90415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff90b9c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040b2f10 con 0x7ff9040d1770 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff90b9c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040b2d90 con 0x7ff9040d2330 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff90b9c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9040b3090 con 0x7ff90415f3a0 2024-04-26T21:47:09.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff909762640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9040d1770 0x7ff9040d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff909762640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9040d1770 0x7ff9040d2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34514/0 (socket says 172.21.15.40:34514) 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.126+0000 7ff909762640 1 -- 172.21.15.40:0/2633010212 learned_addr learned my addr 172.21.15.40:0/2633010212 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff908f61640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9040d2330 0x7ff9041e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff909f63640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff90415f3a0 0x7ff90415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff909762640 1 -- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff90415f3a0 msgr2=0x7ff90415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff909762640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff90415f3a0 0x7ff90415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff909762640 1 -- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9040d2330 msgr2=0x7ff9041e9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff909762640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9040d2330 0x7ff9041e9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff909762640 1 -- 172.21.15.40:0/2633010212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9041e9ec0 con 0x7ff9040d1770 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff908f61640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9040d2330 0x7ff9041e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff909f63640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff90415f3a0 0x7ff90415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff909762640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9040d1770 0x7ff9040d2dd0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc000c00 tx=0x7ff8fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff901ffb640 1 -- 172.21.15.40:0/2633010212 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc037040 con 0x7ff9040d1770 2024-04-26T21:47:09.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff901ffb640 1 -- 172.21.15.40:0/2633010212 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8fc042070 con 0x7ff9040d1770 2024-04-26T21:47:09.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9041ea1a0 con 0x7ff9040d1770 2024-04-26T21:47:09.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff901ffb640 1 -- 172.21.15.40:0/2633010212 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc030bd0 con 0x7ff9040d1770 2024-04-26T21:47:09.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.127+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9040c9f20 con 0x7ff9040d1770 2024-04-26T21:47:09.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.128+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9040694f0 con 0x7ff9040d1770 2024-04-26T21:47:09.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.129+0000 7ff901ffb640 1 -- 172.21.15.40:0/2633010212 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8fc047460 con 0x7ff9040d1770 2024-04-26T21:47:09.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.129+0000 7ff901ffb640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8f006c8f0 0x7ff8f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:09.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.129+0000 7ff901ffb640 1 -- 172.21.15.40:0/2633010212 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7ff8fc0ad6e0 con 0x7ff9040d1770 2024-04-26T21:47:09.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.129+0000 7ff908f61640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8f006c8f0 0x7ff8f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:09.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.130+0000 7ff908f61640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8f006c8f0 0x7ff8f006edb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff9040d3810 tx=0x7ff8f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:09.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.133+0000 7ff901ffb640 1 -- 172.21.15.40:0/2633010212 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8fc032e20 con 0x7ff9040d1770 2024-04-26T21:47:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:09.362+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff904075510 con 0x7ff9040d1770 2024-04-26T21:47:10.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.004+0000 7ff901ffb640 1 -- 172.21.15.40:0/2633010212 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v352) ==== 104+0+0 (secure 0 0 0) 0x7ff8fc07ab50 con 0x7ff9040d1770 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.006+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8f006c8f0 msgr2=0x7ff8f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.006+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8f006c8f0 0x7ff8f006edb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff9040d3810 tx=0x7ff8f8009040 comp rx=0 tx=0).stop 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9040d1770 msgr2=0x7ff9040d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9040d1770 0x7ff9040d2dd0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc000c00 tx=0x7ff8fc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 shutdown_connections 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff8f006c8f0 0x7ff8f006edb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff90415f3a0 0x7ff90415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9040d2330 0x7ff9041e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 --2- 172.21.15.40:0/2633010212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9040d1770 0x7ff9040d2dd0 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 >> 172.21.15.40:0/2633010212 conn(0x7ff9040d4f90 msgr2=0x7ff9040b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 shutdown_connections 2024-04-26T21:47:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.007+0000 7ff90b9c6640 1 -- 172.21.15.40:0/2633010212 wait complete. 2024-04-26T21:47:10.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:47:10.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.131+0000 7fdff0a33640 1 Processor -- start 2024-04-26T21:47:10.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.131+0000 7fdff0a33640 1 -- start start 2024-04-26T21:47:10.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.132+0000 7fdff0a33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec1629a0 0x7fdfec0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.132+0000 7fdff0a33640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdfec0b5fd0 0x7fdfec0d5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.132+0000 7fdff0a33640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfec162d80 con 0x7fdfec0d1950 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.132+0000 7fdff0a33640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdfec0d39d0 con 0x7fdfec0b5fd0 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.132+0000 7fdff0a33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdfec0d3cb0 con 0x7fdfec1629a0 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.132+0000 7fdfe9d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdfec0b5fd0 0x7fdfec0d5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.132+0000 7fdfea59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec1629a0 0x7fdfec0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfe9d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdfec0b5fd0 0x7fdfec0d5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:44374/0 (socket says 172.21.15.40:44374) 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfe9d9b640 1 -- 172.21.15.40:0/3107313912 learned_addr learned my addr 172.21.15.40:0/3107313912 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfea59c640 1 --2- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec1629a0 0x7fdfec0b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:36918/0 (socket says 172.21.15.40:36918) 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfe9d9b640 1 -- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec1629a0 msgr2=0x7fdfec0b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfe9d9b640 1 --2- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec1629a0 0x7fdfec0b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfe9d9b640 1 -- 172.21.15.40:0/3107313912 >> v1:172.21.15.184:6789/0 conn(0x7fdfec0d1950 legacy=0x7fdfec0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfe9d9b640 1 -- 172.21.15.40:0/3107313912 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec0cfda0 con 0x7fdfec0b5fd0 2024-04-26T21:47:10.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfe959a640 1 -- 172.21.15.40:0/3107313912 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 29961568 0 0) 0x7fdfec162d80 con 0x7fdfec0d1950 2024-04-26T21:47:10.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.133+0000 7fdfea59c640 1 --2- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec1629a0 0x7fdfec0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.134+0000 7fdfe9d9b640 1 --2- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec0b5fd0 0x7fdfec0d5820 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009870 tx=0x7fdfd402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70de2bbc50d2b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.134+0000 7fdfe959a640 1 -- 172.21.15.40:0/3107313912 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4037070 con 0x7fdfec0b5fd0 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.134+0000 7fdfe959a640 1 -- 172.21.15.40:0/3107313912 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd4032440 con 0x7fdfec0b5fd0 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.134+0000 7fdfe959a640 1 -- 172.21.15.40:0/3107313912 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4032810 con 0x7fdfec0b5fd0 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.134+0000 7fdff0a33640 1 -- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec0b5fd0 msgr2=0x7fdfec0d5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.134+0000 7fdff0a33640 1 --2- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec0b5fd0 0x7fdfec0d5820 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009870 tx=0x7fdfd402a270 comp rx=0 tx=0).stop 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.135+0000 7fdff0a33640 1 -- 172.21.15.40:0/3107313912 shutdown_connections 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.135+0000 7fdff0a33640 1 --2- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec0b5fd0 0x7fdfec0d5820 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.135+0000 7fdff0a33640 1 --2- 172.21.15.40:0/3107313912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec1629a0 0x7fdfec0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.135+0000 7fdff0a33640 1 -- 172.21.15.40:0/3107313912 >> 172.21.15.40:0/3107313912 conn(0x7fdfec09e3e0 msgr2=0x7fdfec0d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.135+0000 7fdff0a33640 1 -- 172.21.15.40:0/3107313912 shutdown_connections 2024-04-26T21:47:10.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.135+0000 7fdff0a33640 1 -- 172.21.15.40:0/3107313912 wait complete. 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.136+0000 7fdff0a33640 1 Processor -- start 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.136+0000 7fdff0a33640 1 -- start start 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdff0a33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec0b5fd0 0x7fdfec0ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdff0a33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfec0d1950 0x7fdfec153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdff0a33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1629a0 0x7fdfec0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdff0a33640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec0d5d60 con 0x7fdfec0d1950 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdff0a33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec0d3f90 con 0x7fdfec1629a0 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdff0a33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdfec0d5ee0 con 0x7fdfec0b5fd0 2024-04-26T21:47:10.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfec0d1950 0x7fdfec153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfec0d1950 0x7fdfec153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34530/0 (socket says 172.21.15.40:34530) 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 -- 172.21.15.40:0/1013691476 learned_addr learned my addr 172.21.15.40:0/1013691476 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfea59c640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec0b5fd0 0x7fdfec0ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfead9d640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1629a0 0x7fdfec0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 -- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec0b5fd0 msgr2=0x7fdfec0ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec0b5fd0 0x7fdfec0ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 -- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1629a0 msgr2=0x7fdfec0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1629a0 0x7fdfec0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 -- 172.21.15.40:0/1013691476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec0cc1e0 con 0x7fdfec0d1950 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe9d9b640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfec0d1950 0x7fdfec153460 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4000c00 tx=0x7fdfd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.137+0000 7fdfe2ffd640 1 -- 172.21.15.40:0/1013691476 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4037040 con 0x7fdfec0d1950 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.138+0000 7fdfe2ffd640 1 -- 172.21.15.40:0/1013691476 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd4042070 con 0x7fdfec0d1950 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.138+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec0cc4c0 con 0x7fdfec0d1950 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.138+0000 7fdfe2ffd640 1 -- 172.21.15.40:0/1013691476 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4030880 con 0x7fdfec0d1950 2024-04-26T21:47:10.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.138+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec0cca80 con 0x7fdfec0d1950 2024-04-26T21:47:10.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.139+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfec069600 con 0x7fdfec0d1950 2024-04-26T21:47:10.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.139+0000 7fdfe2ffd640 1 -- 172.21.15.40:0/1013691476 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfd4030a20 con 0x7fdfec0d1950 2024-04-26T21:47:10.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.140+0000 7fdfe2ffd640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:10.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.140+0000 7fdfe2ffd640 1 -- 172.21.15.40:0/1013691476 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9657+0+0 (secure 0 0 0) 0x7fdfd40ad2e0 con 0x7fdfec0d1950 2024-04-26T21:47:10.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.140+0000 7fdfea59c640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:10.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.140+0000 7fdfea59c640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009560 tx=0x7fdfe4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:10.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.144+0000 7fdfe2ffd640 1 -- 172.21.15.40:0/1013691476 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdfd407a070 con 0x7fdfec0d1950 2024-04-26T21:47:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:10.331+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdfec0b3080 con 0x7fdfec0d1950 2024-04-26T21:47:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.016+0000 7fdfe2ffd640 1 -- 172.21.15.40:0/1013691476 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) ==== 181+0+0 (secure 0 0 0) 0x7fdfd407e370 con 0x7fdfec0d1950 2024-04-26T21:47:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:47:11.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfcc06c8f0 msgr2=0x7fdfcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009560 tx=0x7fdfe4009190 comp rx=0 tx=0).stop 2024-04-26T21:47:11.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfec0d1950 msgr2=0x7fdfec153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfec0d1950 0x7fdfec153460 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4000c00 tx=0x7fdfd40046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 shutdown_connections 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1629a0 0x7fdfec0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfec0d1950 0x7fdfec153460 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 --2- 172.21.15.40:0/1013691476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec0b5fd0 0x7fdfec0ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 >> 172.21.15.40:0/1013691476 conn(0x7fdfec09e3e0 msgr2=0x7fdfec06d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 shutdown_connections 2024-04-26T21:47:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.019+0000 7fdff0a33640 1 -- 172.21.15.40:0/1013691476 wait complete. 2024-04-26T21:47:11.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:47:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.149+0000 7f48258c8640 1 Processor -- start 2024-04-26T21:47:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.149+0000 7f48258c8640 1 -- start start 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f48258c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4820156170 0x7f4820070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f48258c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4820163820 0x7f4820162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f48258c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820163150 0x7f482015f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f48258c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f48201635a0 con 0x7f4820163820 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f48258c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f48200b48d0 con 0x7f4820156170 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f48258c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4820156550 con 0x7f4820163150 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f481e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4820163820 0x7f4820162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f481e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4820163820 0x7f4820162c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34542/0 (socket says 172.21.15.40:34542) 2024-04-26T21:47:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f481e7fc640 1 -- 172.21.15.40:0/2311737556 learned_addr learned my addr 172.21.15.40:0/2311737556 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f481f7fe640 1 --2- 172.21.15.40:0/2311737556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820163150 0x7f482015f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f481effd640 1 --2- 172.21.15.40:0/2311737556 >> v2:172.21.15.184:3300/0 conn(0x7f4820156170 0x7f4820070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f481e7fc640 1 -- 172.21.15.40:0/2311737556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820163150 msgr2=0x7f482015f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f481e7fc640 1 --2- 172.21.15.40:0/2311737556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820163150 0x7f482015f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.150+0000 7f481e7fc640 1 -- 172.21.15.40:0/2311737556 >> v2:172.21.15.184:3300/0 conn(0x7f4820156170 msgr2=0x7f4820070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.151+0000 7f481e7fc640 1 --2- 172.21.15.40:0/2311737556 >> v2:172.21.15.184:3300/0 conn(0x7f4820156170 0x7f4820070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.151+0000 7f481e7fc640 1 -- 172.21.15.40:0/2311737556 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4820070a50 con 0x7f4820163820 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.151+0000 7f481e7fc640 1 --2- 172.21.15.40:0/2311737556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f4820162c10 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f482006e690 tx=0x7f480802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db17faf69aab51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.151+0000 7f481dffb640 1 -- 172.21.15.40:0/2311737556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808037070 con 0x7f4820163820 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.151+0000 7f481dffb640 1 -- 172.21.15.40:0/2311737556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4808032440 con 0x7f4820163820 2024-04-26T21:47:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.151+0000 7f481dffb640 1 -- 172.21.15.40:0/2311737556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808032810 con 0x7f4820163820 2024-04-26T21:47:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 -- 172.21.15.40:0/2311737556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 msgr2=0x7f4820162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 --2- 172.21.15.40:0/2311737556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f4820162c10 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f482006e690 tx=0x7f480802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 -- 172.21.15.40:0/2311737556 shutdown_connections 2024-04-26T21:47:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 --2- 172.21.15.40:0/2311737556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820163150 0x7f482015f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 --2- 172.21.15.40:0/2311737556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f4820162c10 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 --2- 172.21.15.40:0/2311737556 >> v2:172.21.15.184:3300/0 conn(0x7f4820156170 0x7f4820070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 -- 172.21.15.40:0/2311737556 >> 172.21.15.40:0/2311737556 conn(0x7f48200b3360 msgr2=0x7f48200b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 -- 172.21.15.40:0/2311737556 shutdown_connections 2024-04-26T21:47:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.152+0000 7f48258c8640 1 -- 172.21.15.40:0/2311737556 wait complete. 2024-04-26T21:47:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.153+0000 7f48258c8640 1 Processor -- start 2024-04-26T21:47:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.153+0000 7f48258c8640 1 -- start start 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f48258c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820156170 0x7f482014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f48258c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4820163150 0x7f48201eac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f48258c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f48201eb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f48258c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f48200706b0 con 0x7f4820163820 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f48258c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4820071010 con 0x7f4820163150 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f48258c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4820070830 con 0x7f4820156170 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f481f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f48201eb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f481f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f48201eb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34558/0 (socket says 172.21.15.40:34558) 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f481f7fe640 1 -- 172.21.15.40:0/1213162269 learned_addr learned my addr 172.21.15.40:0/1213162269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f481effd640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820156170 0x7f482014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.154+0000 7f481e7fc640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4820163150 0x7f48201eac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f481f7fe640 1 -- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820156170 msgr2=0x7f482014d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f481f7fe640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820156170 0x7f482014d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f481f7fe640 1 -- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4820163150 msgr2=0x7f48201eac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f481f7fe640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4820163150 0x7f48201eac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f481f7fe640 1 -- 172.21.15.40:0/1213162269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482014db70 con 0x7f4820163820 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f481f7fe640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f48201eb160 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f4810007e40 tx=0x7f481000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f4817fff640 1 -- 172.21.15.40:0/1213162269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810014620 con 0x7f4820163820 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f4817fff640 1 -- 172.21.15.40:0/1213162269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4810019070 con 0x7f4820163820 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f4817fff640 1 -- 172.21.15.40:0/1213162269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810014a20 con 0x7f4820163820 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48201f5300 con 0x7f4820163820 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.155+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48201f58c0 con 0x7f4820163820 2024-04-26T21:47:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.156+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4820069600 con 0x7f4820163820 2024-04-26T21:47:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.156+0000 7f4817fff640 1 -- 172.21.15.40:0/1213162269 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4810014bc0 con 0x7f4820163820 2024-04-26T21:47:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.157+0000 7f4817fff640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f480006c8f0 0x7f480006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.157+0000 7f4817fff640 1 -- 172.21.15.40:0/1213162269 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9219+0+0 (secure 0 0 0) 0x7f481008d220 con 0x7f4820163820 2024-04-26T21:47:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.157+0000 7f481effd640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f480006c8f0 0x7f480006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.158+0000 7f481effd640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f480006c8f0 0x7f480006edb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4818006fd0 tx=0x7f4818008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.162+0000 7f4817fff640 1 -- 172.21.15.40:0/1213162269 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f481005a840 con 0x7f4820163820 2024-04-26T21:47:11.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:11.347+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f48200b2f50 con 0x7f4820163820 2024-04-26T21:47:12.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.018+0000 7f4817fff640 1 -- 172.21.15.40:0/1213162269 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) ==== 169+0+0 (secure 0 0 0) 0x7f481005a140 con 0x7f4820163820 2024-04-26T21:47:12.019 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f480006c8f0 msgr2=0x7f480006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f480006c8f0 0x7f480006edb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4818006fd0 tx=0x7f4818008040 comp rx=0 tx=0).stop 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 msgr2=0x7f48201eb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f48201eb160 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f4810007e40 tx=0x7f481000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 shutdown_connections 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f480006c8f0 0x7f480006edb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4820163820 0x7f48201eb160 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4820163150 0x7f48201eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 --2- 172.21.15.40:0/1213162269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4820156170 0x7f482014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 >> 172.21.15.40:0/1213162269 conn(0x7f48200b3360 msgr2=0x7f48200ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 shutdown_connections 2024-04-26T21:47:12.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.021+0000 7f48258c8640 1 -- 172.21.15.40:0/1213162269 wait complete. 2024-04-26T21:47:12.037 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:47:12.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.142+0000 7f0fd890a640 1 Processor -- start 2024-04-26T21:47:12.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.142+0000 7f0fd890a640 1 -- start start 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.142+0000 7f0fd890a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0fd415f290 0x7f0fd40cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.142+0000 7f0fd890a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0fd40cd060 0x7f0fd4162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd890a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd406f5a0 con 0x7f0fd41633c0 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd890a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd406f880 con 0x7f0fd40cd060 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd890a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd406fb60 con 0x7f0fd415f290 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd1d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0fd40cd060 0x7f0fd4162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd1d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0fd40cd060 0x7f0fd4162e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34566/0 (socket says 172.21.15.40:34566) 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd1d9b640 1 -- 172.21.15.40:0/3373917234 learned_addr learned my addr 172.21.15.40:0/3373917234 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd259c640 1 --2- 172.21.15.40:0/3373917234 >> v2:172.21.15.184:3300/0 conn(0x7f0fd415f290 0x7f0fd40cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd1d9b640 1 -- 172.21.15.40:0/3373917234 >> v2:172.21.15.184:3300/0 conn(0x7f0fd415f290 msgr2=0x7f0fd40cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd1d9b640 1 --2- 172.21.15.40:0/3373917234 >> v2:172.21.15.184:3300/0 conn(0x7f0fd415f290 0x7f0fd40cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd1d9b640 1 -- 172.21.15.40:0/3373917234 >> v1:172.21.15.184:6789/0 conn(0x7f0fd41633c0 legacy=0x7f0fd406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.143+0000 7f0fd1d9b640 1 -- 172.21.15.40:0/3373917234 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd415f670 con 0x7f0fd40cd060 2024-04-26T21:47:12.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.144+0000 7f0fd1d9b640 1 --2- 172.21.15.40:0/3373917234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd40cd060 0x7f0fd4162e80 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4075b00 tx=0x7f0fbc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd0beb89bf29a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.144+0000 7f0fd159a640 1 -- 172.21.15.40:0/3373917234 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc037070 con 0x7f0fd40cd060 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.144+0000 7f0fd159a640 1 -- 172.21.15.40:0/3373917234 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fbc032440 con 0x7f0fd40cd060 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.144+0000 7f0fd159a640 1 -- 172.21.15.40:0/3373917234 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc032810 con 0x7f0fd40cd060 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.144+0000 7f0fd890a640 1 -- 172.21.15.40:0/3373917234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd40cd060 msgr2=0x7f0fd4162e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.144+0000 7f0fd890a640 1 --2- 172.21.15.40:0/3373917234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd40cd060 0x7f0fd4162e80 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4075b00 tx=0x7f0fbc02a2c0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.145+0000 7f0fd890a640 1 -- 172.21.15.40:0/3373917234 shutdown_connections 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.145+0000 7f0fd890a640 1 --2- 172.21.15.40:0/3373917234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd40cd060 0x7f0fd4162e80 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.145+0000 7f0fd890a640 1 --2- 172.21.15.40:0/3373917234 >> v2:172.21.15.184:3300/0 conn(0x7f0fd415f290 0x7f0fd40cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.145+0000 7f0fd890a640 1 -- 172.21.15.40:0/3373917234 >> 172.21.15.40:0/3373917234 conn(0x7f0fd40d0fd0 msgr2=0x7f0fd40d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.145+0000 7f0fd890a640 1 -- 172.21.15.40:0/3373917234 shutdown_connections 2024-04-26T21:47:12.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.145+0000 7f0fd890a640 1 -- 172.21.15.40:0/3373917234 wait complete. 2024-04-26T21:47:12.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.146+0000 7f0fd890a640 1 Processor -- start 2024-04-26T21:47:12.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.146+0000 7f0fd890a640 1 -- start start 2024-04-26T21:47:12.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd890a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fd40cd060 0x7f0fd40c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd890a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fd415f290 0x7f0fd40c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd890a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd41633c0 0x7f0fd4161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd890a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd40736d0 con 0x7f0fd41633c0 2024-04-26T21:47:12.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd890a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd40cd6e0 con 0x7f0fd415f290 2024-04-26T21:47:12.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd890a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd4073850 con 0x7f0fd40cd060 2024-04-26T21:47:12.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd41633c0 0x7f0fd4161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd41633c0 0x7f0fd4161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34576/0 (socket says 172.21.15.40:34576) 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 -- 172.21.15.40:0/179654762 learned_addr learned my addr 172.21.15.40:0/179654762 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd259c640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fd40cd060 0x7f0fd40c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd1d9b640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fd415f290 0x7f0fd40c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 -- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fd40cd060 msgr2=0x7f0fd40c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fd40cd060 0x7f0fd40c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 -- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fd415f290 msgr2=0x7f0fd40c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fd415f290 0x7f0fd40c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 -- 172.21.15.40:0/179654762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd409e150 con 0x7f0fd41633c0 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd259c640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fd40cd060 0x7f0fd40c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fd2d9d640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd41633c0 0x7f0fd4161e90 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c780 tx=0x7f0fc400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fcaffd640 1 -- 172.21.15.40:0/179654762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4004b60 con 0x7f0fd41633c0 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.147+0000 7f0fcaffd640 1 -- 172.21.15.40:0/179654762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc4004d00 con 0x7f0fd41633c0 2024-04-26T21:47:12.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.148+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd40c55a0 con 0x7f0fd41633c0 2024-04-26T21:47:12.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.148+0000 7f0fcaffd640 1 -- 172.21.15.40:0/179654762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400aa90 con 0x7f0fd41633c0 2024-04-26T21:47:12.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.148+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd40af370 con 0x7f0fd41633c0 2024-04-26T21:47:12.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.149+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd40694f0 con 0x7f0fd41633c0 2024-04-26T21:47:12.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.149+0000 7f0fcaffd640 1 -- 172.21.15.40:0/179654762 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fc400ac30 con 0x7f0fd41633c0 2024-04-26T21:47:12.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.149+0000 7f0fcaffd640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fb406c8f0 0x7f0fb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.149+0000 7f0fcaffd640 1 -- 172.21.15.40:0/179654762 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 8689+0+0 (secure 0 0 0) 0x7f0fc408cc70 con 0x7f0fd41633c0 2024-04-26T21:47:12.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.150+0000 7f0fd259c640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fb406c8f0 0x7f0fb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.150+0000 7f0fd259c640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fb406c8f0 0x7f0fb406edb0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00b820 tx=0x7f0fcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.154+0000 7f0fcaffd640 1 -- 172.21.15.40:0/179654762 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0fc405a4a0 con 0x7f0fd41633c0 2024-04-26T21:47:12.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.343+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0fd415dc10 con 0x7f0fd41633c0 2024-04-26T21:47:12.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.346+0000 7f0fcaffd640 1 -- 172.21.15.40:0/179654762 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f0fc4059da0 con 0x7f0fd41633c0 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fb406c8f0 msgr2=0x7f0fb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fb406c8f0 0x7f0fb406edb0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00b820 tx=0x7f0fcc008040 comp rx=0 tx=0).stop 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd41633c0 msgr2=0x7f0fd4161e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd41633c0 0x7f0fd4161e90 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c780 tx=0x7f0fc400cc40 comp rx=0 tx=0).stop 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 shutdown_connections 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0fb406c8f0 0x7f0fb406edb0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd41633c0 0x7f0fd4161e90 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fd415f290 0x7f0fd40c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 --2- 172.21.15.40:0/179654762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fd40cd060 0x7f0fd40c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 >> 172.21.15.40:0/179654762 conn(0x7f0fd40d0fd0 msgr2=0x7f0fd40cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 shutdown_connections 2024-04-26T21:47:12.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.349+0000 7f0fd890a640 1 -- 172.21.15.40:0/179654762 wait complete. 2024-04-26T21:47:12.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-04-26T21:47:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.468+0000 7f6e5e17f640 1 Processor -- start 2024-04-26T21:47:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.468+0000 7f6e5e17f640 1 -- start start 2024-04-26T21:47:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e5e17f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e5815f3a0 0x7f6e580d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e5e17f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e580d1770 0x7f6e580d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e5e17f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e580d25a0 con 0x7f6e580d2130 2024-04-26T21:47:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e5e17f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6e580d0e20 con 0x7f6e5815f3a0 2024-04-26T21:47:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e5e17f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e580b1d70 con 0x7f6e580d1770 2024-04-26T21:47:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e577fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e5815f3a0 0x7f6e580d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e577fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e5815f3a0 0x7f6e580d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34580/0 (socket says 172.21.15.40:34580) 2024-04-26T21:47:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e577fe640 1 -- 172.21.15.40:0/1606831586 learned_addr learned my addr 172.21.15.40:0/1606831586 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e56ffd640 1 --2- 172.21.15.40:0/1606831586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e580d1770 0x7f6e580d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e577fe640 1 -- 172.21.15.40:0/1606831586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e580d1770 msgr2=0x7f6e580d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e577fe640 1 --2- 172.21.15.40:0/1606831586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e580d1770 0x7f6e580d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e577fe640 1 -- 172.21.15.40:0/1606831586 >> v1:172.21.15.184:6789/0 conn(0x7f6e580d2130 legacy=0x7f6e580d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.469+0000 7f6e577fe640 1 -- 172.21.15.40:0/1606831586 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e580b2050 con 0x7f6e5815f3a0 2024-04-26T21:47:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.470+0000 7f6e577fe640 1 --2- 172.21.15.40:0/1606831586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e5815f3a0 0x7f6e580d06f0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009870 tx=0x7f6e4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd0b36dfe289ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.470+0000 7f6e567fc640 1 -- 172.21.15.40:0/1606831586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48037070 con 0x7f6e5815f3a0 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.470+0000 7f6e567fc640 1 -- 172.21.15.40:0/1606831586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e48032440 con 0x7f6e5815f3a0 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.470+0000 7f6e567fc640 1 -- 172.21.15.40:0/1606831586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48032810 con 0x7f6e5815f3a0 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.471+0000 7f6e5e17f640 1 -- 172.21.15.40:0/1606831586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e5815f3a0 msgr2=0x7f6e580d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.471+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/1606831586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e5815f3a0 0x7f6e580d06f0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009870 tx=0x7f6e4802a270 comp rx=0 tx=0).stop 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.471+0000 7f6e5e17f640 1 -- 172.21.15.40:0/1606831586 shutdown_connections 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.471+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/1606831586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e580d1770 0x7f6e580d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.471+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/1606831586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e5815f3a0 0x7f6e580d06f0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.471+0000 7f6e5e17f640 1 -- 172.21.15.40:0/1606831586 >> 172.21.15.40:0/1606831586 conn(0x7f6e580d4f90 msgr2=0x7f6e580d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.471+0000 7f6e5e17f640 1 -- 172.21.15.40:0/1606831586 shutdown_connections 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.471+0000 7f6e5e17f640 1 -- 172.21.15.40:0/1606831586 wait complete. 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.472+0000 7f6e5e17f640 1 Processor -- start 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.472+0000 7f6e5e17f640 1 -- start start 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e5e17f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e580d1770 0x7f6e5814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e5e17f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580d2130 0x7f6e580cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e5e17f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5815f3a0 0x7f6e580ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e5e17f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5815f900 con 0x7f6e580d2130 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e5e17f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5815f780 con 0x7f6e580d1770 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e5e17f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e580b3860 con 0x7f6e5815f3a0 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580d2130 0x7f6e580cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580d2130 0x7f6e580cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34592/0 (socket says 172.21.15.40:34592) 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 -- 172.21.15.40:0/4216474594 learned_addr learned my addr 172.21.15.40:0/4216474594 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e57fff640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5815f3a0 0x7f6e580ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e577fe640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e580d1770 0x7f6e5814c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 -- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5815f3a0 msgr2=0x7f6e580ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5815f3a0 0x7f6e580ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 -- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e580d1770 msgr2=0x7f6e5814c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e580d1770 0x7f6e5814c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 -- 172.21.15.40:0/4216474594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e580b9c60 con 0x7f6e580d2130 2024-04-26T21:47:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e57fff640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5815f3a0 0x7f6e580ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e577fe640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e580d1770 0x7f6e5814c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e56ffd640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580d2130 0x7f6e580cc060 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00d800 tx=0x7f6e4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.473+0000 7f6e3ffff640 1 -- 172.21.15.40:0/4216474594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00b6c0 con 0x7f6e580d2130 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.474+0000 7f6e3ffff640 1 -- 172.21.15.40:0/4216474594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e4c019070 con 0x7f6e580d2130 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.474+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e580b9f40 con 0x7f6e580d2130 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.474+0000 7f6e3ffff640 1 -- 172.21.15.40:0/4216474594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00ba80 con 0x7f6e580d2130 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.474+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e580ba500 con 0x7f6e580d2130 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.475+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e580694f0 con 0x7f6e580d2130 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.475+0000 7f6e3ffff640 1 -- 172.21.15.40:0/4216474594 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e4c00bc20 con 0x7f6e580d2130 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.475+0000 7f6e3ffff640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.476+0000 7f6e3ffff640 1 -- 172.21.15.40:0/4216474594 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 8689+0+0 (secure 0 0 0) 0x7f6e4c08cb60 con 0x7f6e580d2130 2024-04-26T21:47:12.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.476+0000 7f6e577fe640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:12.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.476+0000 7f6e577fe640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6e4406c8f0 0x7f6e4406edb0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6e48034000 tx=0x7f6e4802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:12.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.480+0000 7f6e3ffff640 1 -- 172.21.15.40:0/4216474594 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e4c05a420 con 0x7f6e580d2130 2024-04-26T21:47:12.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:12.737+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7f6e580d1520 con 0x7f6e580d2130 2024-04-26T21:47:12.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:47:12.959 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:47:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.045+0000 7f6e3ffff640 1 -- 172.21.15.40:0/4216474594 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v640) ==== 135+0+0 (secure 0 0 0) 0x7f6e4c059d20 con 0x7f6e580d2130 2024-04-26T21:47:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.046+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6e4406c8f0 msgr2=0x7f6e4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.046+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6e4406c8f0 0x7f6e4406edb0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6e48034000 tx=0x7f6e4802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.046+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580d2130 msgr2=0x7f6e580cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.046+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580d2130 0x7f6e580cc060 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00d800 tx=0x7f6e4c00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.046+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 shutdown_connections 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.046+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.047+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5815f3a0 0x7f6e580ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.047+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580d2130 0x7f6e580cc060 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.047+0000 7f6e5e17f640 1 --2- 172.21.15.40:0/4216474594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e580d1770 0x7f6e5814c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.047+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 >> 172.21.15.40:0/4216474594 conn(0x7f6e580d4f90 msgr2=0x7f6e5815e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.047+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 shutdown_connections 2024-04-26T21:47:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.047+0000 7f6e5e17f640 1 -- 172.21.15.40:0/4216474594 wait complete. 2024-04-26T21:47:13.059 INFO:tasks.cephfs_test_runner:That the MDS has a clean 'damaged' response to loss of any single metadata object ... ok 2024-04-26T21:47:13.059 INFO:tasks.cephfs_test_runner:Starting test: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) That errors encountered during opening inos are properly propagated 2024-04-26T21:47:13.059 INFO:tasks.cephfs_test_runner:test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-04-26T21:47:13.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.199+0000 7f8885951640 1 Processor -- start 2024-04-26T21:47:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.199+0000 7f8885951640 1 -- start start 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f8885951640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88800d1380 0x7f88800d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f8885951640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88800d1ce0 con 0x7f8880070ae0 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f8885951640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88800ce840 con 0x7f888015d1e0 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f8885951640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f88800cede0 con 0x7f88800d1380 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f887f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8880070ae0 0x7f8880070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51806/0 (socket says 172.21.15.40:51806) 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f887e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88800d1380 0x7f88800d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f887f7fe640 1 -- 172.21.15.40:0/1399053725 learned_addr learned my addr 172.21.15.40:0/1399053725 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f887e7fc640 1 -- 172.21.15.40:0/1399053725 >> v1:172.21.15.184:6789/0 conn(0x7f888015d1e0 legacy=0x7f88800d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:13.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f887e7fc640 1 -- 172.21.15.40:0/1399053725 >> v1:172.21.15.40:6789/0 conn(0x7f8880070ae0 legacy=0x7f8880070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f887dffb640 1 -- 172.21.15.40:0/1399053725 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009760268 0 0) 0x7f88800d1ce0 con 0x7f8880070ae0 2024-04-26T21:47:13.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.200+0000 7f887e7fc640 1 -- 172.21.15.40:0/1399053725 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88800ceb00 con 0x7f88800d1380 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.201+0000 7f887e7fc640 1 --2- 172.21.15.40:0/1399053725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f88800d17a0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f88740097a0 tx=0x7f887402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6e160a0b1d449f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.201+0000 7f887dffb640 1 -- 172.21.15.40:0/1399053725 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874037070 con 0x7f88800d1380 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.201+0000 7f887dffb640 1 -- 172.21.15.40:0/1399053725 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8874032440 con 0x7f88800d1380 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.201+0000 7f887dffb640 1 -- 172.21.15.40:0/1399053725 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874032810 con 0x7f88800d1380 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.202+0000 7f8885951640 1 -- 172.21.15.40:0/1399053725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 msgr2=0x7f88800d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.202+0000 7f8885951640 1 --2- 172.21.15.40:0/1399053725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f88800d17a0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f88740097a0 tx=0x7f887402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.202+0000 7f8885951640 1 -- 172.21.15.40:0/1399053725 shutdown_connections 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.202+0000 7f8885951640 1 --2- 172.21.15.40:0/1399053725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f88800d17a0 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.202+0000 7f8885951640 1 -- 172.21.15.40:0/1399053725 >> 172.21.15.40:0/1399053725 conn(0x7f88800d2da0 msgr2=0x7f88800d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.202+0000 7f8885951640 1 -- 172.21.15.40:0/1399053725 shutdown_connections 2024-04-26T21:47:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.202+0000 7f8885951640 1 -- 172.21.15.40:0/1399053725 wait complete. 2024-04-26T21:47:13.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.203+0000 7f8885951640 1 Processor -- start 2024-04-26T21:47:13.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.203+0000 7f8885951640 1 -- start start 2024-04-26T21:47:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f8885951640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8880070ae0 0x7f888014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f8885951640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f888006e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f8885951640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f888015d1e0 0x7f88800c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f8885951640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88800d10f0 con 0x7f88800d1380 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f8885951640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8880070650 con 0x7f888015d1e0 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f8885951640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f88800ce4c0 con 0x7f8880070ae0 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f888006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f888006e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34612/0 (socket says 172.21.15.40:34612) 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887e7fc640 1 -- 172.21.15.40:0/3964981767 learned_addr learned my addr 172.21.15.40:0/3964981767 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887effd640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8880070ae0 0x7f888014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887f7fe640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f888015d1e0 0x7f88800c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887e7fc640 1 -- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8880070ae0 msgr2=0x7f888014d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887e7fc640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8880070ae0 0x7f888014d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887e7fc640 1 -- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f888015d1e0 msgr2=0x7f88800c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887e7fc640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f888015d1e0 0x7f88800c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887e7fc640 1 -- 172.21.15.40:0/3964981767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888014d580 con 0x7f88800d1380 2024-04-26T21:47:13.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.204+0000 7f887effd640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8880070ae0 0x7f888014d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:13.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.205+0000 7f887e7fc640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f888006e080 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f88740097a0 tx=0x7f88740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.205+0000 7f8867fff640 1 -- 172.21.15.40:0/3964981767 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874037040 con 0x7f88800d1380 2024-04-26T21:47:13.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.205+0000 7f8867fff640 1 -- 172.21.15.40:0/3964981767 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8874004250 con 0x7f88800d1380 2024-04-26T21:47:13.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.205+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88800cab60 con 0x7f88800d1380 2024-04-26T21:47:13.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.205+0000 7f8867fff640 1 -- 172.21.15.40:0/3964981767 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874032e20 con 0x7f88800d1380 2024-04-26T21:47:13.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.205+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888006d880 con 0x7f88800d1380 2024-04-26T21:47:13.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.206+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88800694f0 con 0x7f88800d1380 2024-04-26T21:47:13.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.206+0000 7f8867fff640 1 -- 172.21.15.40:0/3964981767 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8874030770 con 0x7f88800d1380 2024-04-26T21:47:13.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.207+0000 7f8867fff640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886c06c8f0 0x7f886c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.207+0000 7f8867fff640 1 -- 172.21.15.40:0/3964981767 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 8689+0+0 (secure 0 0 0) 0x7f88740ada90 con 0x7f88800d1380 2024-04-26T21:47:13.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.207+0000 7f887effd640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886c06c8f0 0x7f886c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.207+0000 7f887effd640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886c06c8f0 0x7f886c06edb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8870006fd0 tx=0x7f8870008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.211+0000 7f8867fff640 1 -- 172.21.15.40:0/3964981767 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f887407b2f0 con 0x7f88800d1380 2024-04-26T21:47:13.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.399+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88800694f0 con 0x7f88800d1380 2024-04-26T21:47:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.400+0000 7f8867fff640 1 -- 172.21.15.40:0/3964981767 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+18315 (secure 0 0 0) 0x7f887407abf0 con 0x7f88800d1380 2024-04-26T21:47:13.401 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:13.402 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":137,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:12.018043+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/933356185":"2024-04-27T21:47:00.954240+0000","172.21.15.40:6832/933356185":"2024-04-27T21:47:00.954240+0000","172.21.15.40:6833/3095512631":"2024-04-27T21:46:53.909685+0000","172.21.15.40:6833/3502494134":"2024-04-27T21:46:46.873417+0000","172.21.15.40:6833/1769689018":"2024-04-27T21:46:41.828993+0000","172.21.15.40:6833/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6832/771026484":"2024-04-27T21:46:20.722276+0000","172.21.15.40:6833/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6832/1753847972":"2024-04-27T21:46:08.370602+0000","172.21.15.40:6833/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6833/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6832/2896027915":"2024-04-27T21:45:31.907924+0000","172.21.15.40:6833/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6832/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6833/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6833/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.184:6837/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6832/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/1992174713":"2024-04-27T21:43:31.927565+0000","172.21.15.40:6833/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6834/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6833/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6833/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6833/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.40:6834/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6833/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6835/1482743077":"2024-04-27T21:42:45.590944+0000","172.21.15.40:6832/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6832/4214786907":"2024-04-27T21:45:46.312381+0000","172.21.15.40:6833/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3791539151":"2024-04-27T21:45:23.843779+0000","172.21.15.40:6832/3502494134":"2024-04-27T21:46:46.873417+0000","172.21.15.40:6834/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/1660522931":"2024-04-27T21:45:51.733253+0000","172.21.15.40:6832/3311550037":"2024-04-27T21:43:13.833433+0000","172.21.15.40:6832/3095512631":"2024-04-27T21:46:53.909685+0000","172.21.15.40:6832/2899400637":"2024-04-27T21:43:45.018129+0000","172.21.15.40:6833/3538213517":"2024-04-27T21:45:17.788819+0000","172.21.15.40:6835/2103981319":"2024-04-27T21:42:40.495461+0000","172.21.15.40:6832/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.184:6835/1182266137":"2024-04-27T21:42:55.651896+0000","172.21.15.40:6833/68430788":"2024-04-27T21:44:01.214317+0000","172.21.15.40:6832/2962699830":"2024-04-27T21:43:38.976843+0000","172.21.15.40:6832/4016455310":"2024-04-27T21:44:55.673612+0000","172.21.15.40:6832/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6834/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6832/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6832/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.40:6832/4192917874":"2024-04-27T21:45:19.814510+0000","172.21.15.184:6836/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6835/747650264":"2024-04-27T21:42:51.669693+0000","172.21.15.40:6833/3588039422":"2024-04-27T21:44:08.264618+0000","172.21.15.40:6832/2324764539":"2024-04-27T21:44:14.314860+0000","172.21.15.40:6832/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/2224283166":"2024-04-27T21:44:51.638256+0000","172.21.15.40:6833/45688484":"2024-04-27T21:44:18.346545+0000","172.21.15.40:6833/1812066725":"2024-04-27T21:46:25.765788+0000","172.21.15.40:6832/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6832/4173136579":"2024-04-27T21:45:27.871638+0000","172.21.15.40:6833/343744670":"2024-04-27T21:43:27.897726+0000","172.21.15.40:6833/2569387163":"2024-04-27T21:44:22.393324+0000","172.21.15.40:6833/1803384446":"2024-04-27T21:43:59.077926+0000","172.21.15.40:6833/1239220748":"2024-04-27T21:44:37.530725+0000","172.21.15.40:6832/3114124794":"2024-04-27T21:43:25.679833+0000","172.21.15.40:6833/4237566530":"2024-04-27T21:44:47.608188+0000","172.21.15.40:6832/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6833/1051970054":"2024-04-27T21:44:39.547503+0000","172.21.15.40:6835/2715490806":"2024-04-27T21:42:23.477662+0000","172.21.15.40:6832/1149681708":"2024-04-27T21:44:43.574926+0000","172.21.15.40:6832/752925509":"2024-04-27T21:44:59.701505+0000","172.21.15.40:6832/1769689018":"2024-04-27T21:46:41.828993+0000","172.21.15.40:6833/1838768570":"2024-04-27T21:45:03.736947+0000","172.21.15.184:6834/1327671065":"2024-04-27T21:42:59.734236+0000","172.21.15.40:6832/3538213517":"2024-04-27T21:45:17.788819+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886c06c8f0 msgr2=0x7f886c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886c06c8f0 0x7f886c06edb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8870006fd0 tx=0x7f8870008040 comp rx=0 tx=0).stop 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 msgr2=0x7f888006e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f888006e080 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f88740097a0 tx=0x7f88740046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 shutdown_connections 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f886c06c8f0 0x7f886c06edb0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f888015d1e0 0x7f88800c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88800d1380 0x7f888006e080 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 --2- 172.21.15.40:0/3964981767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8880070ae0 0x7f888014d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 >> 172.21.15.40:0/3964981767 conn(0x7f88800d2da0 msgr2=0x7f88800d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:13.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 shutdown_connections 2024-04-26T21:47:13.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.403+0000 7f8885951640 1 -- 172.21.15.40:0/3964981767 wait complete. 2024-04-26T21:47:13.414 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:13.415 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:13.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.517+0000 7fa423abd640 1 Processor -- start 2024-04-26T21:47:13.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.517+0000 7fa423abd640 1 -- start start 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.518+0000 7fa423abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c0b40d0 0x7fa41c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.518+0000 7fa423abd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa41c162b30 0x7fa41c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.518+0000 7fa423abd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41c0d0a10 con 0x7fa41c0d1a00 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.518+0000 7fa423abd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa41c0d0cf0 con 0x7fa41c162b30 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.518+0000 7fa423abd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c0d0fd0 con 0x7fa41c0b40d0 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.518+0000 7fa42205a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa41c162b30 0x7fa41c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.518+0000 7fa421859640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa41c0d1a00 0x7fa41c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51810/0 (socket says 172.21.15.40:51810) 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.518+0000 7fa421859640 1 -- 172.21.15.40:0/4244106614 learned_addr learned my addr 172.21.15.40:0/4244106614 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.519+0000 7fa421058640 1 --2- 172.21.15.40:0/4244106614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c0b40d0 0x7fa41c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.519+0000 7fa42205a640 1 -- 172.21.15.40:0/4244106614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c0b40d0 msgr2=0x7fa41c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.519+0000 7fa42205a640 1 --2- 172.21.15.40:0/4244106614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c0b40d0 0x7fa41c1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.519+0000 7fa42205a640 1 -- 172.21.15.40:0/4244106614 >> v1:172.21.15.40:6789/0 conn(0x7fa41c0d1a00 legacy=0x7fa41c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.519+0000 7fa42205a640 1 -- 172.21.15.40:0/4244106614 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c162fa0 con 0x7fa41c162b30 2024-04-26T21:47:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.519+0000 7fa420857640 1 -- 172.21.15.40:0/4244106614 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881467838 0 0) 0x7fa41c0d0a10 con 0x7fa41c0d1a00 2024-04-26T21:47:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.519+0000 7fa42205a640 1 --2- 172.21.15.40:0/4244106614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c162b30 0x7fa41c0b4b00 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009d90 tx=0x7fa40c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f54bcea0b54989 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa420857640 1 -- 172.21.15.40:0/4244106614 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c031600 con 0x7fa41c162b30 2024-04-26T21:47:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa420857640 1 -- 172.21.15.40:0/4244106614 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c0317a0 con 0x7fa41c162b30 2024-04-26T21:47:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa420857640 1 -- 172.21.15.40:0/4244106614 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c031b70 con 0x7fa41c162b30 2024-04-26T21:47:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa423abd640 1 -- 172.21.15.40:0/4244106614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c162b30 msgr2=0x7fa41c0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa423abd640 1 --2- 172.21.15.40:0/4244106614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c162b30 0x7fa41c0b4b00 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009d90 tx=0x7fa40c02a5d0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa423abd640 1 -- 172.21.15.40:0/4244106614 shutdown_connections 2024-04-26T21:47:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa423abd640 1 --2- 172.21.15.40:0/4244106614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c162b30 0x7fa41c0b4b00 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa423abd640 1 --2- 172.21.15.40:0/4244106614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c0b40d0 0x7fa41c1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa423abd640 1 -- 172.21.15.40:0/4244106614 >> 172.21.15.40:0/4244106614 conn(0x7fa41c0d4fd0 msgr2=0x7fa41c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa423abd640 1 -- 172.21.15.40:0/4244106614 shutdown_connections 2024-04-26T21:47:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.520+0000 7fa423abd640 1 -- 172.21.15.40:0/4244106614 wait complete. 2024-04-26T21:47:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.521+0000 7fa423abd640 1 Processor -- start 2024-04-26T21:47:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.521+0000 7fa423abd640 1 -- start start 2024-04-26T21:47:13.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa423abd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c0b40d0 0x7fa41c15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa423abd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c0d1a00 0x7fa41c0b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa423abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c162b30 0x7fa41c0ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa423abd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0d2670 con 0x7fa41c0b40d0 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa423abd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0d24f0 con 0x7fa41c0d1a00 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa423abd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c0b5040 con 0x7fa41c162b30 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa421859640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c0b40d0 0x7fa41c15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa421859640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c0b40d0 0x7fa41c15e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34632/0 (socket says 172.21.15.40:34632) 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa421859640 1 -- 172.21.15.40:0/359621239 learned_addr learned my addr 172.21.15.40:0/359621239 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa421859640 1 -- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c162b30 msgr2=0x7fa41c0ba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa421859640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c162b30 0x7fa41c0ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.522+0000 7fa421859640 1 -- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c0d1a00 msgr2=0x7fa41c0b9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.523+0000 7fa421859640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c0d1a00 0x7fa41c0b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.523+0000 7fa421859640 1 -- 172.21.15.40:0/359621239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c1f4ea0 con 0x7fa41c0b40d0 2024-04-26T21:47:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.523+0000 7fa421859640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c0b40d0 0x7fa41c15e0b0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c4d0 tx=0x7fa41400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.523+0000 7fa41a7fc640 1 -- 172.21.15.40:0/359621239 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414018070 con 0x7fa41c0b40d0 2024-04-26T21:47:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.523+0000 7fa41a7fc640 1 -- 172.21.15.40:0/359621239 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa414007500 con 0x7fa41c0b40d0 2024-04-26T21:47:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.523+0000 7fa41a7fc640 1 -- 172.21.15.40:0/359621239 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414004a20 con 0x7fa41c0b40d0 2024-04-26T21:47:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.523+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c1f5180 con 0x7fa41c0b40d0 2024-04-26T21:47:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.523+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c1f5740 con 0x7fa41c0b40d0 2024-04-26T21:47:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.524+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41c0694f0 con 0x7fa41c0b40d0 2024-04-26T21:47:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.524+0000 7fa41a7fc640 1 -- 172.21.15.40:0/359621239 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa41401b070 con 0x7fa41c0b40d0 2024-04-26T21:47:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.525+0000 7fa41a7fc640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40806c8f0 0x7fa40806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.525+0000 7fa41a7fc640 1 -- 172.21.15.40:0/359621239 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 8689+0+0 (secure 0 0 0) 0x7fa41408bb80 con 0x7fa41c0b40d0 2024-04-26T21:47:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.525+0000 7fa421058640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40806c8f0 0x7fa40806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.526+0000 7fa421058640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40806c8f0 0x7fa40806edb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa41c06e0f0 tx=0x7fa410008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.529+0000 7fa41a7fc640 1 -- 172.21.15.40:0/359621239 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4140592c0 con 0x7fa41c0b40d0 2024-04-26T21:47:13.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.712+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa41c0c8460 con 0x7fa41c0b40d0 2024-04-26T21:47:13.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.712+0000 7fa41a7fc640 1 -- 172.21.15.40:0/359621239 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7fa414058bc0 con 0x7fa41c0b40d0 2024-04-26T21:47:13.713 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:47:13.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40806c8f0 msgr2=0x7fa40806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40806c8f0 0x7fa40806edb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa41c06e0f0 tx=0x7fa410008040 comp rx=0 tx=0).stop 2024-04-26T21:47:13.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c0b40d0 msgr2=0x7fa41c15e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c0b40d0 0x7fa41c15e0b0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c4d0 tx=0x7fa41400c990 comp rx=0 tx=0).stop 2024-04-26T21:47:13.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 shutdown_connections 2024-04-26T21:47:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa40806c8f0 0x7fa40806edb0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa41c162b30 0x7fa41c0ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c0d1a00 0x7fa41c0b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 --2- 172.21.15.40:0/359621239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c0b40d0 0x7fa41c15e0b0 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 >> 172.21.15.40:0/359621239 conn(0x7fa41c0d4fd0 msgr2=0x7fa41c0d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 shutdown_connections 2024-04-26T21:47:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.714+0000 7fa423abd640 1 -- 172.21.15.40:0/359621239 wait complete. 2024-04-26T21:47:13.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-04-26T21:47:13.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.825+0000 7ffabddda640 1 Processor -- start 2024-04-26T21:47:13.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.825+0000 7ffabddda640 1 -- start start 2024-04-26T21:47:13.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.826+0000 7ffabddda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffab80d06f0 0x7ffab80d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.826+0000 7ffabddda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab80d0ba0 con 0x7ffab80b2850 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.826+0000 7ffabddda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab80d0e20 con 0x7ffab815f3a0 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.826+0000 7ffabddda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffab80b1d70 con 0x7ffab80d06f0 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.826+0000 7ffab7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffab80b2850 0x7ffab80b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51824/0 (socket says 172.21.15.40:51824) 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.826+0000 7ffab7fff640 1 -- 172.21.15.40:0/1763347901 learned_addr learned my addr 172.21.15.40:0/1763347901 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.826+0000 7ffab6ffd640 1 --2- 172.21.15.40:0/1763347901 >> v2:172.21.15.184:3300/0 conn(0x7ffab80d06f0 0x7ffab80d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3747493220 0 0) 0x7ffab0001490 con 0x7ffab80b2850 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaa0003520 con 0x7ffab80b2850 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130287924 0 0) 0x7ffab80d0e20 con 0x7ffab815f3a0 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffab0001490 con 0x7ffab815f3a0 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2215714343 0 0) 0x7ffaa0003520 con 0x7ffab80b2850 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 >> v2:172.21.15.184:3300/0 conn(0x7ffab80d06f0 msgr2=0x7ffab80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 --2- 172.21.15.40:0/1763347901 >> v2:172.21.15.184:3300/0 conn(0x7ffab80d06f0 0x7ffab80d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 >> v1:172.21.15.184:6789/0 conn(0x7ffab815f3a0 legacy=0x7ffab80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab80b2050 con 0x7ffab80b2850 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7ffab0001bc0 con 0x7ffab80b2850 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 >> v1:172.21.15.40:6789/0 conn(0x7ffab80b2850 legacy=0x7ffab80b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0003b90 0x7ffaa0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaa0004800 0x7ffaa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaa00051c0 0x7ffaa000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffab80b1d70 con 0x7ffaa0004800 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa000c090 con 0x7ffaa0003b90 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa000c290 con 0x7ffaa00051c0 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab6ffd640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaa0004800 0x7ffaa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab77fe640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0003b90 0x7ffaa0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab7fff640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaa00051c0 0x7ffaa000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab6ffd640 1 -- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaa00051c0 msgr2=0x7ffaa000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab6ffd640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaa00051c0 0x7ffaa000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab6ffd640 1 -- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0003b90 msgr2=0x7ffaa0003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:13.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab6ffd640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0003b90 0x7ffaa0003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab6ffd640 1 -- 172.21.15.40:0/1763347901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa0003520 con 0x7ffaa0004800 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab7fff640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaa00051c0 0x7ffaa000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.827+0000 7ffab7fff640 1 -- 172.21.15.40:0/1763347901 reap_dead start 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.828+0000 7ffab6ffd640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaa0004800 0x7ffaa0004c80 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009ac0 tx=0x7ffaa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b0429c370b17b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.828+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac036070 con 0x7ffaa0004800 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.828+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaac031400 con 0x7ffaa0004800 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.828+0000 7ffab67fc640 1 -- 172.21.15.40:0/1763347901 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac009690 con 0x7ffaa0004800 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.829+0000 7ffabddda640 1 -- 172.21.15.40:0/1763347901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaa0004800 msgr2=0x7ffaa0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.829+0000 7ffabddda640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaa0004800 0x7ffaa0004c80 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009ac0 tx=0x7ffaa0005830 comp rx=0 tx=0).stop 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.829+0000 7ffabddda640 1 -- 172.21.15.40:0/1763347901 shutdown_connections 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.829+0000 7ffabddda640 1 --2- 172.21.15.40:0/1763347901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaa0004800 0x7ffaa0004c80 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.829+0000 7ffabddda640 1 -- 172.21.15.40:0/1763347901 >> 172.21.15.40:0/1763347901 conn(0x7ffab80d4f90 msgr2=0x7ffab80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.829+0000 7ffabddda640 1 -- 172.21.15.40:0/1763347901 shutdown_connections 2024-04-26T21:47:13.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.829+0000 7ffabddda640 1 -- 172.21.15.40:0/1763347901 wait complete. 2024-04-26T21:47:13.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.830+0000 7ffabddda640 1 Processor -- start 2024-04-26T21:47:13.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.830+0000 7ffabddda640 1 -- start start 2024-04-26T21:47:13.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffabddda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0004800 0x7ffab80b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffabddda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffab8161010 0x7ffab8153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffabddda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffab806d6b0 0x7ffab806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffabddda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffab815f900 con 0x7ffab806d6b0 2024-04-26T21:47:13.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffabddda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffab815f780 con 0x7ffaa0004800 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffabddda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffab80d1770 con 0x7ffab8161010 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffab806d6b0 0x7ffab806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffab806d6b0 0x7ffab806db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34650/0 (socket says 172.21.15.40:34650) 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab7fff640 1 -- 172.21.15.40:0/689315048 learned_addr learned my addr 172.21.15.40:0/689315048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab6ffd640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffab8161010 0x7ffab8153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab77fe640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0004800 0x7ffab80b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab7fff640 1 -- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffab8161010 msgr2=0x7ffab8153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab7fff640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffab8161010 0x7ffab8153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab7fff640 1 -- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0004800 msgr2=0x7ffab80b7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab7fff640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0004800 0x7ffab80b7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab7fff640 1 -- 172.21.15.40:0/689315048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab8162890 con 0x7ffab806d6b0 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.831+0000 7ffab77fe640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0004800 0x7ffab80b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.832+0000 7ffab7fff640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffab806d6b0 0x7ffab806db40 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7ffab0001350 tx=0x7ffab000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.832+0000 7ffa9ffff640 1 -- 172.21.15.40:0/689315048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab000eda0 con 0x7ffab806d6b0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.832+0000 7ffa9ffff640 1 -- 172.21.15.40:0/689315048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab0018070 con 0x7ffab806d6b0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.832+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab8153cc0 con 0x7ffab806d6b0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.832+0000 7ffa9ffff640 1 -- 172.21.15.40:0/689315048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0013660 con 0x7ffab806d6b0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.832+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab8161ad0 con 0x7ffab806d6b0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.833+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab80694f0 con 0x7ffab806d6b0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.833+0000 7ffa9ffff640 1 -- 172.21.15.40:0/689315048 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffab0013800 con 0x7ffab806d6b0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.834+0000 7ffa9ffff640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffaa406c8f0 0x7ffaa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.834+0000 7ffa9ffff640 1 -- 172.21.15.40:0/689315048 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 8689+0+0 (secure 0 0 0) 0x7ffab008cc40 con 0x7ffab806d6b0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.834+0000 7ffab77fe640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffaa406c8f0 0x7ffaa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:13.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.835+0000 7ffab77fe640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffaa406c8f0 0x7ffaa406edb0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ffab806e2c0 tx=0x7ffaa8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:13.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:13.838+0000 7ffa9ffff640 1 -- 172.21.15.40:0/689315048 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffab80d0ba0 con 0x7ffab806d6b0 2024-04-26T21:47:14.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:14.095+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7ffab80b6860 con 0x7ffab806d6b0 2024-04-26T21:47:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.054+0000 7ffa9ffff640 1 -- 172.21.15.40:0/689315048 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v642) ==== 138+0+0 (secure 0 0 0) 0x7ffab005a500 con 0x7ffab806d6b0 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffaa406c8f0 msgr2=0x7ffaa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffaa406c8f0 0x7ffaa406edb0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ffab806e2c0 tx=0x7ffaa8009040 comp rx=0 tx=0).stop 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffab806d6b0 msgr2=0x7ffab806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffab806d6b0 0x7ffab806db40 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7ffab0001350 tx=0x7ffab000e530 comp rx=0 tx=0).stop 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 shutdown_connections 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ffaa406c8f0 0x7ffaa406edb0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffab806d6b0 0x7ffab806db40 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffab8161010 0x7ffab8153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 --2- 172.21.15.40:0/689315048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa0004800 0x7ffab80b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 >> 172.21.15.40:0/689315048 conn(0x7ffab80d4f90 msgr2=0x7ffab80d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 shutdown_connections 2024-04-26T21:47:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.058+0000 7ffabddda640 1 -- 172.21.15.40:0/689315048 wait complete. 2024-04-26T21:47:15.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:47:15.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.182+0000 7fcc0cefc640 1 Processor -- start 2024-04-26T21:47:15.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.182+0000 7fcc0cefc640 1 -- start start 2024-04-26T21:47:15.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.182+0000 7fcc0cefc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d1a00 0x7fcc080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.182+0000 7fcc0cefc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcc080d2320 0x7fcc081625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.182+0000 7fcc0cefc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc080b40d0 0x7fcc08162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.182+0000 7fcc0cefc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcc08163070 con 0x7fcc080b40d0 2024-04-26T21:47:15.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.182+0000 7fcc0cefc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcc080d0a10 con 0x7fcc080d2320 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.182+0000 7fcc0cefc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc080d0cf0 con 0x7fcc080d1a00 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc06d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc080b40d0 0x7fcc08162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc06d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc080b40d0 0x7fcc08162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34656/0 (socket says 172.21.15.40:34656) 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc06d9d640 1 -- 172.21.15.40:0/4251102738 learned_addr learned my addr 172.21.15.40:0/4251102738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc0659c640 1 --2- 172.21.15.40:0/4251102738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d1a00 0x7fcc080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcbfffff640 1 --2- 172.21.15.40:0/4251102738 >> v2:172.21.15.184:3300/0 conn(0x7fcc080d2320 0x7fcc081625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc06d9d640 1 -- 172.21.15.40:0/4251102738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d1a00 msgr2=0x7fcc080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc06d9d640 1 --2- 172.21.15.40:0/4251102738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d1a00 0x7fcc080d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc06d9d640 1 -- 172.21.15.40:0/4251102738 >> v2:172.21.15.184:3300/0 conn(0x7fcc080d2320 msgr2=0x7fcc081625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc06d9d640 1 --2- 172.21.15.40:0/4251102738 >> v2:172.21.15.184:3300/0 conn(0x7fcc080d2320 0x7fcc081625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.183+0000 7fcc06d9d640 1 -- 172.21.15.40:0/4251102738 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc080d0fd0 con 0x7fcc080b40d0 2024-04-26T21:47:15.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc06d9d640 1 --2- 172.21.15.40:0/4251102738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc08162b30 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7fcc0808d3d0 tx=0x7fcc0002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251cfe5f908d0f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc05d9b640 1 -- 172.21.15.40:0/4251102738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00036070 con 0x7fcc080b40d0 2024-04-26T21:47:15.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc05d9b640 1 -- 172.21.15.40:0/4251102738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc00031600 con 0x7fcc080b40d0 2024-04-26T21:47:15.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc05d9b640 1 -- 172.21.15.40:0/4251102738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc000319d0 con 0x7fcc080b40d0 2024-04-26T21:47:15.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 -- 172.21.15.40:0/4251102738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 msgr2=0x7fcc08162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/4251102738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc08162b30 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7fcc0808d3d0 tx=0x7fcc0002a610 comp rx=0 tx=0).stop 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 -- 172.21.15.40:0/4251102738 shutdown_connections 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/4251102738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc08162b30 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/4251102738 >> v2:172.21.15.184:3300/0 conn(0x7fcc080d2320 0x7fcc081625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/4251102738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d1a00 0x7fcc080d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 -- 172.21.15.40:0/4251102738 >> 172.21.15.40:0/4251102738 conn(0x7fcc080d4fd0 msgr2=0x7fcc080d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 -- 172.21.15.40:0/4251102738 shutdown_connections 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.184+0000 7fcc0cefc640 1 -- 172.21.15.40:0/4251102738 wait complete. 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.186+0000 7fcc0cefc640 1 Processor -- start 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.186+0000 7fcc0cefc640 1 -- start start 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.186+0000 7fcc0cefc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc081f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.186+0000 7fcc0cefc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc080d1a00 0x7fcc080ba630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.186+0000 7fcc0cefc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d2320 0x7fcc081608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.186+0000 7fcc0cefc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080d3050 con 0x7fcc080b40d0 2024-04-26T21:47:15.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.186+0000 7fcc0cefc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080d2ed0 con 0x7fcc080d1a00 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.186+0000 7fcc0cefc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc080d31d0 con 0x7fcc080d2320 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc081f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc081f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34668/0 (socket says 172.21.15.40:34668) 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 -- 172.21.15.40:0/2117065296 learned_addr learned my addr 172.21.15.40:0/2117065296 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcbfffff640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc080d1a00 0x7fcc080ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc06d9d640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d2320 0x7fcc081608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 -- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d2320 msgr2=0x7fcc081608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d2320 0x7fcc081608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 -- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc080d1a00 msgr2=0x7fcc080ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc080d1a00 0x7fcc080ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 -- 172.21.15.40:0/2117065296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc080b7ed0 con 0x7fcc080b40d0 2024-04-26T21:47:15.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0659c640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc081f4f80 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d800 tx=0x7fcbf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcbfeffd640 1 -- 172.21.15.40:0/2117065296 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8018020 con 0x7fcc080b40d0 2024-04-26T21:47:15.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcbfeffd640 1 -- 172.21.15.40:0/2117065296 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf8009640 con 0x7fcc080b40d0 2024-04-26T21:47:15.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcbfeffd640 1 -- 172.21.15.40:0/2117065296 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8009a10 con 0x7fcc080b40d0 2024-04-26T21:47:15.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc081f55d0 con 0x7fcc080b40d0 2024-04-26T21:47:15.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.187+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc081f5b90 con 0x7fcc080b40d0 2024-04-26T21:47:15.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.188+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc080694f0 con 0x7fcc080b40d0 2024-04-26T21:47:15.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.189+0000 7fcbfeffd640 1 -- 172.21.15.40:0/2117065296 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbf801b020 con 0x7fcc080b40d0 2024-04-26T21:47:15.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.189+0000 7fcbfeffd640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.189+0000 7fcbfeffd640 1 -- 172.21.15.40:0/2117065296 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 8689+0+0 (secure 0 0 0) 0x7fcbf808c930 con 0x7fcc080b40d0 2024-04-26T21:47:15.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.189+0000 7fcbfffff640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.190+0000 7fcbfffff640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbe806c8f0 0x7fcbe806edb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0006fd0 tx=0x7fcbf0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.194+0000 7fcbfeffd640 1 -- 172.21.15.40:0/2117065296 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcbf8022030 con 0x7fcc080b40d0 2024-04-26T21:47:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.380+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcc080cbe60 con 0x7fcc080b40d0 2024-04-26T21:47:15.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.384+0000 7fcbfeffd640 1 -- 172.21.15.40:0/2117065296 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fcbf805a190 con 0x7fcc080b40d0 2024-04-26T21:47:15.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbe806c8f0 msgr2=0x7fcbe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbe806c8f0 0x7fcbe806edb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0006fd0 tx=0x7fcbf0008040 comp rx=0 tx=0).stop 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 msgr2=0x7fcc081f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc081f4f80 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d800 tx=0x7fcbf800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 shutdown_connections 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc080d2320 0x7fcc081608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc080d1a00 0x7fcc080ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 --2- 172.21.15.40:0/2117065296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc080b40d0 0x7fcc081f4f80 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 >> 172.21.15.40:0/2117065296 conn(0x7fcc080d4fd0 msgr2=0x7fcc081623c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 shutdown_connections 2024-04-26T21:47:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.386+0000 7fcc0cefc640 1 -- 172.21.15.40:0/2117065296 wait complete. 2024-04-26T21:47:15.398 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:47:15.398 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:15.432 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:15.472 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:47:15.472 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:15.472 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:15.488 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:15.488 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:47:15.488 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:15.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:15.530 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:15.530 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:15.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:15.545 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:47:15.545 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:15.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:15.587 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:47:15.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.687+0000 7f9b3e407640 1 Processor -- start 2024-04-26T21:47:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.687+0000 7f9b3e407640 1 -- start start 2024-04-26T21:47:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b3e407640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b380b2340 0x7f9b380cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b3e407640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b380cd930 0x7f9b380d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b3e407640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b380d10d0 con 0x7f9b38162810 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b3e407640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9b380d0750 con 0x7f9b380cd930 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b3e407640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b380d16a0 con 0x7f9b380b2340 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b3c9a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b380cd930 0x7f9b380d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b37fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9b38162810 0x7f9b38070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51826/0 (socket says 172.21.15.40:51826) 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b37fff640 1 -- 172.21.15.40:0/3824016154 learned_addr learned my addr 172.21.15.40:0/3824016154 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.688+0000 7f9b377fe640 1 --2- 172.21.15.40:0/3824016154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b380b2340 0x7f9b380cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b3c9a4640 1 -- 172.21.15.40:0/3824016154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b380b2340 msgr2=0x7f9b380cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b3c9a4640 1 --2- 172.21.15.40:0/3824016154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b380b2340 0x7f9b380cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b3c9a4640 1 -- 172.21.15.40:0/3824016154 >> v1:172.21.15.40:6789/0 conn(0x7f9b38162810 legacy=0x7f9b38070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b3c9a4640 1 -- 172.21.15.40:0/3824016154 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b380d1370 con 0x7f9b380cd930 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b36ffd640 1 -- 172.21.15.40:0/3824016154 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3761108884 0 0) 0x7f9b380d10d0 con 0x7f9b38162810 2024-04-26T21:47:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b377fe640 1 --2- 172.21.15.40:0/3824016154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b380b2340 0x7f9b380cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b3c9a4640 1 --2- 172.21.15.40:0/3824016154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380cd930 0x7f9b380d1b30 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009d90 tx=0x7f9b3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6aa1c2dd9be5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b36ffd640 1 -- 172.21.15.40:0/3824016154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30007740 con 0x7f9b380cd930 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b36ffd640 1 -- 172.21.15.40:0/3824016154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b300078e0 con 0x7f9b380cd930 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.689+0000 7f9b36ffd640 1 -- 172.21.15.40:0/3824016154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30007cb0 con 0x7f9b380cd930 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.690+0000 7f9b3e407640 1 -- 172.21.15.40:0/3824016154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380cd930 msgr2=0x7f9b380d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.690+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3824016154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380cd930 0x7f9b380d1b30 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009d90 tx=0x7f9b3002a660 comp rx=0 tx=0).stop 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.690+0000 7f9b3e407640 1 -- 172.21.15.40:0/3824016154 shutdown_connections 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.690+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3824016154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380cd930 0x7f9b380d1b30 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.690+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3824016154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b380b2340 0x7f9b380cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.690+0000 7f9b3e407640 1 -- 172.21.15.40:0/3824016154 >> 172.21.15.40:0/3824016154 conn(0x7f9b380d2da0 msgr2=0x7f9b380d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.690+0000 7f9b3e407640 1 -- 172.21.15.40:0/3824016154 shutdown_connections 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.690+0000 7f9b3e407640 1 -- 172.21.15.40:0/3824016154 wait complete. 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.691+0000 7f9b3e407640 1 Processor -- start 2024-04-26T21:47:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.691+0000 7f9b3e407640 1 -- start start 2024-04-26T21:47:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b3e407640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380b2340 0x7f9b38161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b3e407640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b380cd930 0x7f9b381498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b3e407640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b38162810 0x7f9b3814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b3e407640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9b380d2070 con 0x7f9b380b2340 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b3e407640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b380d0470 con 0x7f9b380cd930 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b3e407640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b380d21f0 con 0x7f9b38162810 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380b2340 0x7f9b38161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380b2340 0x7f9b38161070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34682/0 (socket says 172.21.15.40:34682) 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b37fff640 1 -- 172.21.15.40:0/3936218438 learned_addr learned my addr 172.21.15.40:0/3936218438 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b3c9a4640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b38162810 0x7f9b3814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b37fff640 1 -- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b38162810 msgr2=0x7f9b3814cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b37fff640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b38162810 0x7f9b3814cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b37fff640 1 -- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b380cd930 msgr2=0x7f9b381498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b377fe640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b380cd930 0x7f9b381498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b37fff640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b380cd930 0x7f9b381498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b37fff640 1 -- 172.21.15.40:0/3936218438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b38160330 con 0x7f9b380b2340 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.692+0000 7f9b3c9a4640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b38162810 0x7f9b3814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.693+0000 7f9b377fe640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b380cd930 0x7f9b381498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.693+0000 7f9b37fff640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380b2340 0x7f9b38161070 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800c430 tx=0x7f9b2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.693+0000 7f9b34ff9640 1 -- 172.21.15.40:0/3936218438 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b280074e0 con 0x7f9b380b2340 2024-04-26T21:47:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.693+0000 7f9b34ff9640 1 -- 172.21.15.40:0/3936218438 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b280047e0 con 0x7f9b380b2340 2024-04-26T21:47:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.693+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381605b0 con 0x7f9b380b2340 2024-04-26T21:47:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.693+0000 7f9b34ff9640 1 -- 172.21.15.40:0/3936218438 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28004bd0 con 0x7f9b380b2340 2024-04-26T21:47:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.693+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b38160b50 con 0x7f9b380b2340 2024-04-26T21:47:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.694+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b38160d50 con 0x7f9b380b2340 2024-04-26T21:47:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.694+0000 7f9b34ff9640 1 -- 172.21.15.40:0/3936218438 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b28011070 con 0x7f9b380b2340 2024-04-26T21:47:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.695+0000 7f9b34ff9640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b1806c8f0 0x7f9b1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.695+0000 7f9b34ff9640 1 -- 172.21.15.40:0/3936218438 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 8689+0+0 (secure 0 0 0) 0x7f9b2808c380 con 0x7f9b380b2340 2024-04-26T21:47:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.695+0000 7f9b377fe640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b1806c8f0 0x7f9b1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.696+0000 7f9b377fe640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b1806c8f0 0x7f9b1806edb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9b20006fd0 tx=0x7f9b20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:15.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.699+0000 7f9b34ff9640 1 -- 172.21.15.40:0/3936218438 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b38160d50 con 0x7f9b380b2340 2024-04-26T21:47:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.923+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b380694f0 con 0x7f9b380b2340 2024-04-26T21:47:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.924+0000 7f9b34ff9640 1 -- 172.21.15.40:0/3936218438 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 352 v352) ==== 95+0+3881 (secure 0 0 0) 0x7f9b38160330 con 0x7f9b380b2340 2024-04-26T21:47:15.924 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:15.925 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":352,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3116806866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3116806866},{"type":"v1","addr":"172.21.15.40:6835","nonce":3116806866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":6806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3350177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3350177622},{"type":"v1","addr":"172.21.15.184:6836","nonce":3350177622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122},{"gid":7600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1485406763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1485406763},{"type":"v1","addr":"172.21.15.184:6837","nonce":1485406763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":11574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1002792848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1002792848},{"type":"v1","addr":"172.21.15.40:6833","nonce":1002792848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":350}],"filesystems":[]} 2024-04-26T21:47:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 352 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.925+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b1806c8f0 msgr2=0x7f9b1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.925+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b1806c8f0 0x7f9b1806edb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9b20006fd0 tx=0x7f9b20008040 comp rx=0 tx=0).stop 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380b2340 msgr2=0x7f9b38161070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380b2340 0x7f9b38161070 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800c430 tx=0x7f9b2800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 shutdown_connections 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b1806c8f0 0x7f9b1806edb0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b38162810 0x7f9b3814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b380cd930 0x7f9b381498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 --2- 172.21.15.40:0/3936218438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b380b2340 0x7f9b38161070 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 >> 172.21.15.40:0/3936218438 conn(0x7f9b380d2da0 msgr2=0x7f9b380b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 shutdown_connections 2024-04-26T21:47:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:15.926+0000 7f9b3e407640 1 -- 172.21.15.40:0/3936218438 wait complete. 2024-04-26T21:47:15.937 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:47:15.937 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:47:15.938 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:47:15.938 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:15.938 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:47:15.938 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:47:15.938 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:47:15.938 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:15.938 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:47:15.939 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:47:15.939 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:47:15.939 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:15.939 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:47:15.939 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:47:15.939 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:47:15.939 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:47:15.947 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:47:15.947 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:47:15.948 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:47:15.948 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:47:15.951 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:47:15.951 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:47:15.952 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:47:15.952 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:47:15.990 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:47:15.990 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:15.991 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:47:15.991 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:15.993 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:47:15.994 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:15.994 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:47:15.994 DEBUG:teuthology.parallel:result is None 2024-04-26T21:47:15.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:47:16.066 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:47:16.067 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:47:16.072 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:47:16.072 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:47:16.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.102+0000 7f74893f0640 1 Processor -- start 2024-04-26T21:47:16.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.102+0000 7f74893f0640 1 -- start start 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.103+0000 7f74893f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f748415f5a0 0x7f74840d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.103+0000 7f74893f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74840d2830 0x7f7484162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.103+0000 7f74893f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74840d1a70 con 0x7f74840d3c20 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.103+0000 7f74893f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7484163d30 con 0x7f748415f5a0 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.103+0000 7f74893f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74840d1d50 con 0x7f74840d2830 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.103+0000 7f74837fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f74840d3c20 0x7f748415f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51840/0 (socket says 172.21.15.40:51840) 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.103+0000 7f74837fe640 1 -- 172.21.15.40:0/2978678983 learned_addr learned my addr 172.21.15.40:0/2978678983 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f74827fc640 1 --2- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74840d2830 0x7f7484162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f7482ffd640 1 --2- 172.21.15.40:0/2978678983 >> v2:172.21.15.184:3300/0 conn(0x7f748415f5a0 0x7f74840d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f7481ffb640 1 -- 172.21.15.40:0/2978678983 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970355074 0 0) 0x7f74840d1a70 con 0x7f74840d3c20 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f7481ffb640 1 -- 172.21.15.40:0/2978678983 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f746c003520 con 0x7f74840d3c20 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f7482ffd640 1 -- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74840d2830 msgr2=0x7f7484162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:16.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f7482ffd640 1 --2- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74840d2830 0x7f7484162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f7482ffd640 1 -- 172.21.15.40:0/2978678983 >> v1:172.21.15.40:6789/0 conn(0x7f74840d3c20 legacy=0x7f748415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f7482ffd640 1 -- 172.21.15.40:0/2978678983 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74840d2030 con 0x7f748415f5a0 2024-04-26T21:47:16.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.104+0000 7f74827fc640 1 --2- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74840d2830 0x7f7484162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f7482ffd640 1 --2- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748415f5a0 0x7f74840d5cb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f74740097a0 tx=0x7f747402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f827ca71981c69fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f7481ffb640 1 -- 172.21.15.40:0/2978678983 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474037070 con 0x7f748415f5a0 2024-04-26T21:47:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f7481ffb640 1 -- 172.21.15.40:0/2978678983 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747402ae50 con 0x7f748415f5a0 2024-04-26T21:47:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f7481ffb640 1 -- 172.21.15.40:0/2978678983 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474032660 con 0x7f748415f5a0 2024-04-26T21:47:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f74893f0640 1 -- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748415f5a0 msgr2=0x7f74840d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f74893f0640 1 --2- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748415f5a0 0x7f74840d5cb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f74740097a0 tx=0x7f747402a100 comp rx=0 tx=0).stop 2024-04-26T21:47:16.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f74893f0640 1 -- 172.21.15.40:0/2978678983 shutdown_connections 2024-04-26T21:47:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f74893f0640 1 --2- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74840d2830 0x7f7484162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f74893f0640 1 --2- 172.21.15.40:0/2978678983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748415f5a0 0x7f74840d5cb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f74893f0640 1 -- 172.21.15.40:0/2978678983 >> 172.21.15.40:0/2978678983 conn(0x7f74840d5190 msgr2=0x7f74840d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f74893f0640 1 -- 172.21.15.40:0/2978678983 shutdown_connections 2024-04-26T21:47:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.105+0000 7f74893f0640 1 -- 172.21.15.40:0/2978678983 wait complete. 2024-04-26T21:47:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74893f0640 1 Processor -- start 2024-04-26T21:47:16.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74893f0640 1 -- start start 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74893f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74840d2830 0x7f74840cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74893f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74840d3c20 0x7f7484159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74893f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748415a8d0 0x7f74841e9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74893f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7484163440 con 0x7f74840d3c20 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74893f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74840d4190 con 0x7f74840d2830 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74893f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74841635c0 con 0x7f748415a8d0 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74827fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74840d3c20 0x7f7484159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74827fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74840d3c20 0x7f7484159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34712/0 (socket says 172.21.15.40:34712) 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.107+0000 7f74827fc640 1 -- 172.21.15.40:0/3540044908 learned_addr learned my addr 172.21.15.40:0/3540044908 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f7482ffd640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74840d2830 0x7f74840cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74837fe640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748415a8d0 0x7f74841e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74827fc640 1 -- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748415a8d0 msgr2=0x7f74841e9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74827fc640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748415a8d0 0x7f74841e9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74827fc640 1 -- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74840d2830 msgr2=0x7f74840cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74827fc640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74840d2830 0x7f74840cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74827fc640 1 -- 172.21.15.40:0/3540044908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74840cc7a0 con 0x7f74840d3c20 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74827fc640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74840d3c20 0x7f7484159fe0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f747800ebb0 tx=0x7f747800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f746b7fe640 1 -- 172.21.15.40:0/3540044908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478017070 con 0x7f74840d3c20 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f746b7fe640 1 -- 172.21.15.40:0/3540044908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747800cc30 con 0x7f74840d3c20 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f746b7fe640 1 -- 172.21.15.40:0/3540044908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478013470 con 0x7f74840d3c20 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74840c9e30 con 0x7f74840d3c20 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.108+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74840ca3f0 con 0x7f74840d3c20 2024-04-26T21:47:16.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.109+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74840694f0 con 0x7f74840d3c20 2024-04-26T21:47:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.110+0000 7f746b7fe640 1 -- 172.21.15.40:0/3540044908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f747801a070 con 0x7f74840d3c20 2024-04-26T21:47:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.110+0000 7f746b7fe640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f747006c920 0x7f747006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.110+0000 7f746b7fe640 1 -- 172.21.15.40:0/3540044908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 8689+0+0 (secure 0 0 0) 0x7f747808c510 con 0x7f74840d3c20 2024-04-26T21:47:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.110+0000 7f7482ffd640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f747006c920 0x7f747006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:16.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.111+0000 7f7482ffd640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f747006c920 0x7f747006ede0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7474034000 tx=0x7f747402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:16.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.114+0000 7f746b7fe640 1 -- 172.21.15.40:0/3540044908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7478059780 con 0x7f74840d3c20 2024-04-26T21:47:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:16.314+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f74840bb110 con 0x7f74840d3c20 2024-04-26T21:47:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.071+0000 7f746b7fe640 1 -- 172.21.15.40:0/3540044908 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) ==== 97+0+0 (secure 0 0 0) 0x7f747805da80 con 0x7f74840d3c20 2024-04-26T21:47:17.072 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f747006c920 msgr2=0x7f747006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f747006c920 0x7f747006ede0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7474034000 tx=0x7f747402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74840d3c20 msgr2=0x7f7484159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74840d3c20 0x7f7484159fe0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f747800ebb0 tx=0x7f747800c520 comp rx=0 tx=0).stop 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 shutdown_connections 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f747006c920 0x7f747006ede0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748415a8d0 0x7f74841e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74840d3c20 0x7f7484159fe0 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 --2- 172.21.15.40:0/3540044908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74840d2830 0x7f74840cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 >> 172.21.15.40:0/3540044908 conn(0x7f74840d5190 msgr2=0x7f74840b8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 shutdown_connections 2024-04-26T21:47:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.074+0000 7f74893f0640 1 -- 172.21.15.40:0/3540044908 wait complete. 2024-04-26T21:47:17.090 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:47:17.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.196+0000 7ff86c358640 1 Processor -- start 2024-04-26T21:47:17.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.196+0000 7ff86c358640 1 -- start start 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86c358640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff864162810 0x7ff864070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86c358640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8640d1b30 0x7ff8640d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86c358640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8640d10d0 con 0x7ff8640b2340 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86c358640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff8640d0750 con 0x7ff864162810 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86c358640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff8640d16a0 con 0x7ff8640d1b30 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86a0f4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff864162810 0x7ff864070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86a0f4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff864162810 0x7ff864070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34720/0 (socket says 172.21.15.40:34720) 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86a0f4640 1 -- 172.21.15.40:0/3843816615 learned_addr learned my addr 172.21.15.40:0/3843816615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86a0f4640 1 -- 172.21.15.40:0/3843816615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8640d1b30 msgr2=0x7ff8640d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86a0f4640 1 --2- 172.21.15.40:0/3843816615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8640d1b30 0x7ff8640d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86a0f4640 1 -- 172.21.15.40:0/3843816615 >> v1:172.21.15.184:6789/0 conn(0x7ff8640b2340 legacy=0x7ff8640cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.197+0000 7ff86a0f4640 1 -- 172.21.15.40:0/3843816615 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8640d1370 con 0x7ff864162810 2024-04-26T21:47:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.198+0000 7ff86a0f4640 1 --2- 172.21.15.40:0/3843816615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff864162810 0x7ff864070350 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7ff860009870 tx=0x7ff86002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79532a51a0b3f666 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.198+0000 7ff8690f2640 1 -- 172.21.15.40:0/3843816615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860037070 con 0x7ff864162810 2024-04-26T21:47:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.198+0000 7ff8690f2640 1 -- 172.21.15.40:0/3843816615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff860032440 con 0x7ff864162810 2024-04-26T21:47:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.198+0000 7ff8690f2640 1 -- 172.21.15.40:0/3843816615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860032810 con 0x7ff864162810 2024-04-26T21:47:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.199+0000 7ff86c358640 1 -- 172.21.15.40:0/3843816615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff864162810 msgr2=0x7ff864070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.199+0000 7ff86c358640 1 --2- 172.21.15.40:0/3843816615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff864162810 0x7ff864070350 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7ff860009870 tx=0x7ff86002a210 comp rx=0 tx=0).stop 2024-04-26T21:47:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.199+0000 7ff86c358640 1 -- 172.21.15.40:0/3843816615 shutdown_connections 2024-04-26T21:47:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.199+0000 7ff86c358640 1 --2- 172.21.15.40:0/3843816615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8640d1b30 0x7ff8640d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.199+0000 7ff86c358640 1 --2- 172.21.15.40:0/3843816615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff864162810 0x7ff864070350 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.199+0000 7ff86c358640 1 -- 172.21.15.40:0/3843816615 >> 172.21.15.40:0/3843816615 conn(0x7ff8640d2da0 msgr2=0x7ff8640d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.199+0000 7ff86c358640 1 -- 172.21.15.40:0/3843816615 shutdown_connections 2024-04-26T21:47:17.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.199+0000 7ff86c358640 1 -- 172.21.15.40:0/3843816615 wait complete. 2024-04-26T21:47:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.200+0000 7ff86c358640 1 Processor -- start 2024-04-26T21:47:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.200+0000 7ff86c358640 1 -- start start 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff86c358640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8640b2340 0x7ff8640c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff86c358640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8640d1b30 0x7ff86414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff86c358640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff864162810 0x7ff8640cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff86c358640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff864070890 con 0x7ff8640d1b30 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff86c358640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8640d0470 con 0x7ff8640b2340 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff86c358640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff8640cdb00 con 0x7ff864162810 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8640d1b30 0x7ff86414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8640d1b30 0x7ff86414d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34730/0 (socket says 172.21.15.40:34730) 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 -- 172.21.15.40:0/3369146343 learned_addr learned my addr 172.21.15.40:0/3369146343 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff86a0f4640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8640b2340 0x7ff8640c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff86a8f5640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff864162810 0x7ff8640cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 -- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff864162810 msgr2=0x7ff8640cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff864162810 0x7ff8640cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 -- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8640b2340 msgr2=0x7ff8640c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8640b2340 0x7ff8640c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 -- 172.21.15.40:0/3369146343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8640ca3b0 con 0x7ff8640d1b30 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.201+0000 7ff8698f3640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8640d1b30 0x7ff86414d150 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7ff85400c460 tx=0x7ff85400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.202+0000 7ff85affd640 1 -- 172.21.15.40:0/3369146343 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854016020 con 0x7ff8640d1b30 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.202+0000 7ff85affd640 1 -- 172.21.15.40:0/3369146343 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff854012870 con 0x7ff8640d1b30 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.202+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8640ca650 con 0x7ff8640d1b30 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.202+0000 7ff85affd640 1 -- 172.21.15.40:0/3369146343 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854012c60 con 0x7ff8640d1b30 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.202+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8640cac10 con 0x7ff8640d1b30 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.203+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8640694f0 con 0x7ff8640d1b30 2024-04-26T21:47:17.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.203+0000 7ff85affd640 1 -- 172.21.15.40:0/3369146343 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff854019020 con 0x7ff8640d1b30 2024-04-26T21:47:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.204+0000 7ff85affd640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff84806c8f0 0x7ff84806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.204+0000 7ff85affd640 1 -- 172.21.15.40:0/3369146343 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5249+0+0 (secure 0 0 0) 0x7ff85408b620 con 0x7ff8640d1b30 2024-04-26T21:47:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.204+0000 7ff86a0f4640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff84806c8f0 0x7ff84806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.205+0000 7ff86a0f4640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff84806c8f0 0x7ff84806edb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff860034000 tx=0x7ff86002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.208+0000 7ff85affd640 1 -- 172.21.15.40:0/3369146343 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff854020030 con 0x7ff8640d1b30 2024-04-26T21:47:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.466+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff8640b4160 con 0x7ff8640d1b30 2024-04-26T21:47:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.467+0000 7ff85affd640 1 -- 172.21.15.40:0/3369146343 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7ff854059ca0 con 0x7ff8640d1b30 2024-04-26T21:47:17.468 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:17.468 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:17.468 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmesG+ERAAJS53jYoKUawIOs2zrG5TLg==", 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmq9cvHRAAiYWrMjfAth8wMbs9YlfWgQ==", 2024-04-26T21:47:17.469 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmLRmvGBAA6xAcjLDF7DMjfDqiUCexKA==", 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:47:17.470 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmXxkDJBAAbzzf+LUO3Rr8bdzXb+Gj2Q==", 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.471 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDYHSxmRkhFBhAAz1UfvJwhA8bolDiRN2eM+Q==", 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDbHSxmKAB/BBAAzW5dtz6eczjkDiTQBjfjuA==", 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDeHSxmrienAhAA2JS0l6/Uo9UPc0TeZ1RwRA==", 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDhHSxmv40rARAAN8z+beJe/bTERvO2euMr6w==", 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:47:17.473 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDnHSxmxQ1XCBAAYptVM8WPir+zIHOjbX0tGg==", 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:47:17.474 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDqHSxmv32IBhAAMROkCLjWTOx3ELdkEiVSmw==", 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDtHSxmJdPTBBAAO32qvVLuAzqx1xdajL9CFw==", 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDwHSxmcUMYAxAAIvoAo82ApDXERodD+4+cBQ==", 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw==", 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:47:17.476 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmifWYMBAA6Nskvqt3MZSvFCMHjslELw==", 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.477 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmi6l6LRAAF4EjChVEB8ReULEsl3YppA==", 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmhSMFNhAAjCJvc4FknsSUJQ4NjXRV6Q==", 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:47:17.478 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDTHSxmQk1bMhAAixlszeAmlXc5cAmkFWjhfw==", 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmYjmPDhAADvvIHrcacI8kOAdeiJGd+Q==", 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQDUHSxmsiwICxAAmKAZ2KS3/dYubutkcGfolA==", 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:47:17.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.481 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:17.481 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:47:17.481 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff84806c8f0 msgr2=0x7ff84806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff84806c8f0 0x7ff84806edb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff860034000 tx=0x7ff86002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8640d1b30 msgr2=0x7ff86414d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8640d1b30 0x7ff86414d150 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7ff85400c460 tx=0x7ff85400c920 comp rx=0 tx=0).stop 2024-04-26T21:47:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 shutdown_connections 2024-04-26T21:47:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff84806c8f0 0x7ff84806edb0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff864162810 0x7ff8640cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8640d1b30 0x7ff86414d150 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 --2- 172.21.15.40:0/3369146343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8640b2340 0x7ff8640c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 >> 172.21.15.40:0/3369146343 conn(0x7ff8640d2da0 msgr2=0x7ff864070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 shutdown_connections 2024-04-26T21:47:17.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.469+0000 7ff86c358640 1 -- 172.21.15.40:0/3369146343 wait complete. 2024-04-26T21:47:17.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:17.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.583+0000 7f37f78da640 1 Processor -- start 2024-04-26T21:47:17.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.584+0000 7f37f78da640 1 -- start start 2024-04-26T21:47:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.584+0000 7f37f78da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37f006d560 0x7f37f00d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.584+0000 7f37f78da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f00d2420 con 0x7f37f006e270 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.584+0000 7f37f78da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f00cd6b0 con 0x7f37f00d0070 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.584+0000 7f37f78da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f37f0162d00 con 0x7f37f006d560 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.584+0000 7f37f5e77640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37f006d560 0x7f37f00d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.584+0000 7f37f4e75640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f37f006e270 0x7f37f00ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51852/0 (socket says 172.21.15.40:51852) 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.584+0000 7f37f4e75640 1 -- 172.21.15.40:0/2714331805 learned_addr learned my addr 172.21.15.40:0/2714331805 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.585+0000 7f37f5e77640 1 -- 172.21.15.40:0/2714331805 >> v1:172.21.15.184:6789/0 conn(0x7f37f00d0070 legacy=0x7f37f006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.585+0000 7f37f5e77640 1 -- 172.21.15.40:0/2714331805 >> v1:172.21.15.40:6789/0 conn(0x7f37f006e270 legacy=0x7f37f00ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.585+0000 7f37f5e77640 1 -- 172.21.15.40:0/2714331805 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f00cd990 con 0x7f37f006d560 2024-04-26T21:47:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.585+0000 7f37effff640 1 -- 172.21.15.40:0/2714331805 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1927494947 0 0) 0x7f37f00d2420 con 0x7f37f006e270 2024-04-26T21:47:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37f5e77640 1 --2- 172.21.15.40:0/2714331805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f006d560 0x7f37f00d1d40 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009d90 tx=0x7f37e002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb4493a23972d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37effff640 1 -- 172.21.15.40:0/2714331805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0007740 con 0x7f37f006d560 2024-04-26T21:47:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37effff640 1 -- 172.21.15.40:0/2714331805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e00078e0 con 0x7f37f006d560 2024-04-26T21:47:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37effff640 1 -- 172.21.15.40:0/2714331805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0007cb0 con 0x7f37f006d560 2024-04-26T21:47:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37f78da640 1 -- 172.21.15.40:0/2714331805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f006d560 msgr2=0x7f37f00d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37f78da640 1 --2- 172.21.15.40:0/2714331805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f006d560 0x7f37f00d1d40 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009d90 tx=0x7f37e002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37f78da640 1 -- 172.21.15.40:0/2714331805 shutdown_connections 2024-04-26T21:47:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37f78da640 1 --2- 172.21.15.40:0/2714331805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f006d560 0x7f37f00d1d40 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37f78da640 1 -- 172.21.15.40:0/2714331805 >> 172.21.15.40:0/2714331805 conn(0x7f37f00d2ec0 msgr2=0x7f37f00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:17.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37f78da640 1 -- 172.21.15.40:0/2714331805 shutdown_connections 2024-04-26T21:47:17.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.586+0000 7f37f78da640 1 -- 172.21.15.40:0/2714331805 wait complete. 2024-04-26T21:47:17.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.587+0000 7f37f78da640 1 Processor -- start 2024-04-26T21:47:17.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f78da640 1 -- start start 2024-04-26T21:47:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f78da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f006d560 0x7f37f00cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f78da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f006e270 0x7f37f00c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f78da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f00d0070 0x7f37f015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f78da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37f00ce8a0 con 0x7f37f00d0070 2024-04-26T21:47:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f78da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37f00d2d10 con 0x7f37f006d560 2024-04-26T21:47:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f78da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37f00cea20 con 0x7f37f006e270 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f5e77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f00d0070 0x7f37f015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f5e77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f00d0070 0x7f37f015bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34746/0 (socket says 172.21.15.40:34746) 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.588+0000 7f37f5e77640 1 -- 172.21.15.40:0/35237050 learned_addr learned my addr 172.21.15.40:0/35237050 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f5676640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f006d560 0x7f37f00cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f4e75640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f006e270 0x7f37f00c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f5e77640 1 -- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f006e270 msgr2=0x7f37f00c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f5e77640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f006e270 0x7f37f00c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f5e77640 1 -- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f006d560 msgr2=0x7f37f00cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f5e77640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f006d560 0x7f37f00cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f5e77640 1 -- 172.21.15.40:0/35237050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f015afe0 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f5676640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f006d560 0x7f37f00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f5e77640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f00d0070 0x7f37f015bac0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009d90 tx=0x7f37e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37edffb640 1 -- 172.21.15.40:0/35237050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0040070 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37edffb640 1 -- 172.21.15.40:0/35237050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e0031c40 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f015b2c0 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37edffb640 1 -- 172.21.15.40:0/35237050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0003990 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.589+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f015da30 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.590+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f00694f0 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.591+0000 7f37edffb640 1 -- 172.21.15.40:0/35237050 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37e0039040 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.591+0000 7f37edffb640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37dc06c8f0 0x7f37dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.591+0000 7f37edffb640 1 -- 172.21.15.40:0/35237050 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5249+0+0 (secure 0 0 0) 0x7f37e00acd20 con 0x7f37f00d0070 2024-04-26T21:47:17.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.591+0000 7f37f5676640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37dc06c8f0 0x7f37dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.592+0000 7f37f5676640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37dc06c8f0 0x7f37dc06edb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f37e8007fd0 tx=0x7f37e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.596+0000 7f37edffb640 1 -- 172.21.15.40:0/35237050 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37e007b320 con 0x7f37f00d0070 2024-04-26T21:47:17.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.779+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37f00d0470 con 0x7f37f00d0070 2024-04-26T21:47:17.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.780+0000 7f37edffb640 1 -- 172.21.15.40:0/35237050 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+13130 (secure 0 0 0) 0x7f37e007ac20 con 0x7f37f00d0070 2024-04-26T21:47:17.781 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:17.781 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":141,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:17.079440+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:17.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37dc06c8f0 msgr2=0x7f37dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37dc06c8f0 0x7f37dc06edb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f37e8007fd0 tx=0x7f37e8009040 comp rx=0 tx=0).stop 2024-04-26T21:47:17.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f00d0070 msgr2=0x7f37f015bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f00d0070 0x7f37f015bac0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009d90 tx=0x7f37e0004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 shutdown_connections 2024-04-26T21:47:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f37dc06c8f0 0x7f37dc06edb0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37f00d0070 0x7f37f015bac0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f006e270 0x7f37f00c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 --2- 172.21.15.40:0/35237050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f006d560 0x7f37f00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 >> 172.21.15.40:0/35237050 conn(0x7f37f00d2ec0 msgr2=0x7f37f0162e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 shutdown_connections 2024-04-26T21:47:17.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.782+0000 7f37f78da640 1 -- 172.21.15.40:0/35237050 wait complete. 2024-04-26T21:47:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:17.794 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.897+0000 7f504b6d9640 1 Processor -- start 2024-04-26T21:47:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.898+0000 7f504b6d9640 1 -- start start 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.898+0000 7f504b6d9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440d1a00 0x7f50440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.898+0000 7f504b6d9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f50440d2320 0x7f50441625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.898+0000 7f504b6d9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50440d0a10 con 0x7f50440b40d0 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.898+0000 7f504b6d9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f50440d0cf0 con 0x7f50440d2320 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.898+0000 7f504b6d9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50440d0fd0 con 0x7f50440d1a00 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5048c74640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f50440d2320 0x7f50441625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5049475640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440d1a00 0x7f50440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5048c74640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f50440d2320 0x7f50441625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:44614/0 (socket says 172.21.15.40:44614) 2024-04-26T21:47:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5048c74640 1 -- 172.21.15.40:0/835414163 learned_addr learned my addr 172.21.15.40:0/835414163 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5049c76640 1 --1- 172.21.15.40:0/835414163 >> v1:172.21.15.184:6789/0 conn(0x7f50440b40d0 0x7f50440d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:49406/0 (socket says 172.21.15.40:49406) 2024-04-26T21:47:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5048c74640 1 -- 172.21.15.40:0/835414163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440d1a00 msgr2=0x7f50440d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5048c74640 1 --2- 172.21.15.40:0/835414163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440d1a00 0x7f50440d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5048c74640 1 -- 172.21.15.40:0/835414163 >> v1:172.21.15.184:6789/0 conn(0x7f50440b40d0 legacy=0x7f50440d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5043fff640 1 -- 172.21.15.40:0/835414163 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1248341869 0 0) 0x7f50440d0a10 con 0x7f50440b40d0 2024-04-26T21:47:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.899+0000 7f5048c74640 1 -- 172.21.15.40:0/835414163 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440d40a0 con 0x7f50440d2320 2024-04-26T21:47:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.900+0000 7f5048c74640 1 --2- 172.21.15.40:0/835414163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 0x7f50441625f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5038009870 tx=0x7f503802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe486b9890136ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.900+0000 7f5043fff640 1 -- 172.21.15.40:0/835414163 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038037070 con 0x7f50440d2320 2024-04-26T21:47:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.900+0000 7f5043fff640 1 -- 172.21.15.40:0/835414163 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038032440 con 0x7f50440d2320 2024-04-26T21:47:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.900+0000 7f5043fff640 1 -- 172.21.15.40:0/835414163 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038032810 con 0x7f50440d2320 2024-04-26T21:47:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.901+0000 7f504b6d9640 1 -- 172.21.15.40:0/835414163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 msgr2=0x7f50441625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.901+0000 7f504b6d9640 1 --2- 172.21.15.40:0/835414163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 0x7f50441625f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5038009870 tx=0x7f503802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.901+0000 7f504b6d9640 1 -- 172.21.15.40:0/835414163 shutdown_connections 2024-04-26T21:47:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.901+0000 7f504b6d9640 1 --2- 172.21.15.40:0/835414163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 0x7f50441625f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.901+0000 7f504b6d9640 1 --2- 172.21.15.40:0/835414163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440d1a00 0x7f50440d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.901+0000 7f504b6d9640 1 -- 172.21.15.40:0/835414163 >> 172.21.15.40:0/835414163 conn(0x7f50440d4fd0 msgr2=0x7f50440d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.901+0000 7f504b6d9640 1 -- 172.21.15.40:0/835414163 shutdown_connections 2024-04-26T21:47:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.901+0000 7f504b6d9640 1 -- 172.21.15.40:0/835414163 wait complete. 2024-04-26T21:47:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.902+0000 7f504b6d9640 1 Processor -- start 2024-04-26T21:47:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.902+0000 7f504b6d9640 1 -- start start 2024-04-26T21:47:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f504b6d9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440b40d0 0x7f50440ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f504b6d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50440d1a00 0x7f504415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f504b6d9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 0x7f50440cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f504b6d9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5044162b30 con 0x7f50440d1a00 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f504b6d9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50440b5d60 con 0x7f50440d2320 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f504b6d9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5044162cb0 con 0x7f50440b40d0 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50440d1a00 0x7f504415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50440d1a00 0x7f504415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34760/0 (socket says 172.21.15.40:34760) 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 -- 172.21.15.40:0/619975024 learned_addr learned my addr 172.21.15.40:0/619975024 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5049c76640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 0x7f50440cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5049475640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440b40d0 0x7f50440ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 -- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440b40d0 msgr2=0x7f50440ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440b40d0 0x7f50440ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 -- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 msgr2=0x7f50440cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 0x7f50440cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 -- 172.21.15.40:0/619975024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50441f9150 con 0x7f50440d1a00 2024-04-26T21:47:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5048c74640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50440d1a00 0x7f504415e3b0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f5038035040 tx=0x7f50380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5041ffb640 1 -- 172.21.15.40:0/619975024 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038037040 con 0x7f50440d1a00 2024-04-26T21:47:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5041ffb640 1 -- 172.21.15.40:0/619975024 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038004230 con 0x7f50440d1a00 2024-04-26T21:47:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50440cc510 con 0x7f50440d1a00 2024-04-26T21:47:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.903+0000 7f5041ffb640 1 -- 172.21.15.40:0/619975024 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003610 con 0x7f50440d1a00 2024-04-26T21:47:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.904+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50440ca560 con 0x7f50440d1a00 2024-04-26T21:47:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.905+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50440694f0 con 0x7f50440d1a00 2024-04-26T21:47:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.905+0000 7f5041ffb640 1 -- 172.21.15.40:0/619975024 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f503802ad10 con 0x7f50440d1a00 2024-04-26T21:47:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.905+0000 7f5041ffb640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f503006c920 0x7f503006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.906+0000 7f5041ffb640 1 -- 172.21.15.40:0/619975024 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5249+0+0 (secure 0 0 0) 0x7f50380ac200 con 0x7f50440d1a00 2024-04-26T21:47:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.906+0000 7f5049475640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f503006c920 0x7f503006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.906+0000 7f5049475640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f503006c920 0x7f503006ede0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f50440b9ef0 tx=0x7f503c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:17.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:17.910+0000 7f5041ffb640 1 -- 172.21.15.40:0/619975024 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f503807a230 con 0x7f50440d1a00 2024-04-26T21:47:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.096+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50440b6af0 con 0x7f50440d1a00 2024-04-26T21:47:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.096+0000 7f5041ffb640 1 -- 172.21.15.40:0/619975024 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) ==== 93+0+10 (secure 0 0 0) 0x7f503807e530 con 0x7f50440d1a00 2024-04-26T21:47:18.097 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:47:18.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f503006c920 msgr2=0x7f503006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f503006c920 0x7f503006ede0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f50440b9ef0 tx=0x7f503c009190 comp rx=0 tx=0).stop 2024-04-26T21:47:18.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50440d1a00 msgr2=0x7f504415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50440d1a00 0x7f504415e3b0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f5038035040 tx=0x7f50380046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 shutdown_connections 2024-04-26T21:47:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f503006c920 0x7f503006ede0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50440d2320 0x7f50440cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50440d1a00 0x7f504415e3b0 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 --2- 172.21.15.40:0/619975024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50440b40d0 0x7f50440ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 >> 172.21.15.40:0/619975024 conn(0x7f50440d4fd0 msgr2=0x7f504406dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 shutdown_connections 2024-04-26T21:47:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.098+0000 7f504b6d9640 1 -- 172.21.15.40:0/619975024 wait complete. 2024-04-26T21:47:18.110 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:47:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.210+0000 7f83a1f9d640 1 Processor -- start 2024-04-26T21:47:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.210+0000 7f83a1f9d640 1 -- start start 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f83a1f9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f839c15f3a0 0x7f839c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f83a1f9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f839c0d1770 0x7f839c0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f83a1f9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f839c0d2590 con 0x7f839c0d2160 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f83a1f9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f839c0d0e20 con 0x7f839c0d1770 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f83a1f9d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f839c0b1d70 con 0x7f839c15f3a0 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f839c0d1770 0x7f839c0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f839c0d1770 0x7f839c0d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34768/0 (socket says 172.21.15.40:34768) 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f839c0d2160 0x7f839c0d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51858/0 (socket says 172.21.15.40:51858) 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839affd640 1 -- 172.21.15.40:0/3837369087 learned_addr learned my addr 172.21.15.40:0/3837369087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839b7fe640 1 --2- 172.21.15.40:0/3837369087 >> v2:172.21.15.184:3300/0 conn(0x7f839c15f3a0 0x7f839c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839affd640 1 -- 172.21.15.40:0/3837369087 >> v2:172.21.15.184:3300/0 conn(0x7f839c15f3a0 msgr2=0x7f839c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839affd640 1 --2- 172.21.15.40:0/3837369087 >> v2:172.21.15.184:3300/0 conn(0x7f839c15f3a0 0x7f839c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839affd640 1 -- 172.21.15.40:0/3837369087 >> v1:172.21.15.40:6789/0 conn(0x7f839c0d2160 legacy=0x7f839c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839a7fc640 1 -- 172.21.15.40:0/3837369087 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3831612822 0 0) 0x7f839c0d2590 con 0x7f839c0d2160 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839affd640 1 -- 172.21.15.40:0/3837369087 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0b2050 con 0x7f839c0d1770 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.211+0000 7f839b7fe640 1 --2- 172.21.15.40:0/3837369087 >> v2:172.21.15.184:3300/0 conn(0x7f839c15f3a0 0x7f839c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.212+0000 7f839affd640 1 --2- 172.21.15.40:0/3837369087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0d1c20 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f839c073b50 tx=0x7f838c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce885349760a1416 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.212+0000 7f839a7fc640 1 -- 172.21.15.40:0/3837369087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c037070 con 0x7f839c0d1770 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.212+0000 7f839a7fc640 1 -- 172.21.15.40:0/3837369087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c032440 con 0x7f839c0d1770 2024-04-26T21:47:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.212+0000 7f839a7fc640 1 -- 172.21.15.40:0/3837369087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c032810 con 0x7f839c0d1770 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.213+0000 7f83a1f9d640 1 -- 172.21.15.40:0/3837369087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 msgr2=0x7f839c0d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.213+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/3837369087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0d1c20 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f839c073b50 tx=0x7f838c02a270 comp rx=0 tx=0).stop 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.213+0000 7f83a1f9d640 1 -- 172.21.15.40:0/3837369087 shutdown_connections 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.213+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/3837369087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0d1c20 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.213+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/3837369087 >> v2:172.21.15.184:3300/0 conn(0x7f839c15f3a0 0x7f839c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.213+0000 7f83a1f9d640 1 -- 172.21.15.40:0/3837369087 >> 172.21.15.40:0/3837369087 conn(0x7f839c0d4f90 msgr2=0x7f839c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.213+0000 7f83a1f9d640 1 -- 172.21.15.40:0/3837369087 shutdown_connections 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.213+0000 7f83a1f9d640 1 -- 172.21.15.40:0/3837369087 wait complete. 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.214+0000 7f83a1f9d640 1 Processor -- start 2024-04-26T21:47:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.214+0000 7f83a1f9d640 1 -- start start 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f83a1f9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0ba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f83a1f9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f839c0d2160 0x7f839c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f83a1f9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c15f3a0 0x7f839c06cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f83a1f9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f839c15f900 con 0x7f839c0d1770 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f83a1f9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f839c15f780 con 0x7f839c15f3a0 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f83a1f9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f839c0b3860 con 0x7f839c0d2160 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0ba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0ba9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34778/0 (socket says 172.21.15.40:34778) 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 -- 172.21.15.40:0/588862213 learned_addr learned my addr 172.21.15.40:0/588862213 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 -- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f839c0d2160 msgr2=0x7f839c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839affd640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f839c0d2160 0x7f839c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839bfff640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c15f3a0 0x7f839c06cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f839c0d2160 0x7f839c0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 -- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c15f3a0 msgr2=0x7f839c06cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c15f3a0 0x7f839c06cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 -- 172.21.15.40:0/588862213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0b8e30 con 0x7f839c0d1770 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839bfff640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c15f3a0 0x7f839c06cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.215+0000 7f839b7fe640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0ba9c0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f839000c7f0 tx=0x7f839000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.216+0000 7f8383fff640 1 -- 172.21.15.40:0/588862213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390015070 con 0x7f839c0d1770 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.216+0000 7f8383fff640 1 -- 172.21.15.40:0/588862213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8390007820 con 0x7f839c0d1770 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.216+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c0cf410 con 0x7f839c0d1770 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.216+0000 7f8383fff640 1 -- 172.21.15.40:0/588862213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390007bd0 con 0x7f839c0d1770 2024-04-26T21:47:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.216+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1f6260 con 0x7f839c0d1770 2024-04-26T21:47:18.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.217+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c0694f0 con 0x7f839c0d1770 2024-04-26T21:47:18.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.217+0000 7f8383fff640 1 -- 172.21.15.40:0/588862213 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8390018070 con 0x7f839c0d1770 2024-04-26T21:47:18.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.218+0000 7f8383fff640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f838806c8f0 0x7f838806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.218+0000 7f8383fff640 1 -- 172.21.15.40:0/588862213 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5249+0+0 (secure 0 0 0) 0x7f839008baa0 con 0x7f839c0d1770 2024-04-26T21:47:18.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.218+0000 7f839affd640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f838806c8f0 0x7f838806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.218+0000 7f839affd640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f838806c8f0 0x7f838806edb0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f838c034000 tx=0x7f838c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.223+0000 7f8383fff640 1 -- 172.21.15.40:0/588862213 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f839005a090 con 0x7f839c0d1770 2024-04-26T21:47:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.408+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f839c0b6860 con 0x7f839c0d1770 2024-04-26T21:47:18.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.408+0000 7f8383fff640 1 -- 172.21.15.40:0/588862213 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) ==== 84+0+65 (secure 0 0 0) 0x7f8390059990 con 0x7f839c0d1770 2024-04-26T21:47:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:18.409 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:47:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:47:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:47:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:18.409 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f838806c8f0 msgr2=0x7f838806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f838806c8f0 0x7f838806edb0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f838c034000 tx=0x7f838c02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 msgr2=0x7f839c0ba9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0ba9c0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f839000c7f0 tx=0x7f839000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 shutdown_connections 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f838806c8f0 0x7f838806edb0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f839c15f3a0 0x7f839c06cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f839c0d2160 0x7f839c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 --2- 172.21.15.40:0/588862213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f839c0d1770 0x7f839c0ba9c0 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 >> 172.21.15.40:0/588862213 conn(0x7f839c0d4f90 msgr2=0x7f839c0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:18.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 shutdown_connections 2024-04-26T21:47:18.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.410+0000 7f83a1f9d640 1 -- 172.21.15.40:0/588862213 wait complete. 2024-04-26T21:47:18.422 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:47:18.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:47:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.526+0000 7ff398820640 1 Processor -- start 2024-04-26T21:47:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.526+0000 7ff398820640 1 -- start start 2024-04-26T21:47:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.526+0000 7ff398820640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff39415f7c0 0x7ff3940d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.526+0000 7ff398820640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff394160930 con 0x7ff39415f3e0 2024-04-26T21:47:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.526+0000 7ff398820640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39415fd20 con 0x7ff3940d3bd0 2024-04-26T21:47:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.526+0000 7ff398820640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff3940d4040 con 0x7ff39415f7c0 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff39259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff39415f3e0 0x7ff3940d2ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51866/0 (socket says 172.21.15.40:51866) 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff39259c640 1 -- 172.21.15.40:0/4036282681 learned_addr learned my addr 172.21.15.40:0/4036282681 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3527821290 0 0) 0x7ff394160930 con 0x7ff39415f3e0 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff380003520 con 0x7ff39415f3e0 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff38bfff640 1 --2- 172.21.15.40:0/4036282681 >> v2:172.21.15.184:3300/0 conn(0x7ff39415f7c0 0x7ff3940d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4053225047 0 0) 0x7ff380003520 con 0x7ff39415f3e0 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 >> v2:172.21.15.184:3300/0 conn(0x7ff39415f7c0 msgr2=0x7ff3940d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 --2- 172.21.15.40:0/4036282681 >> v2:172.21.15.184:3300/0 conn(0x7ff39415f7c0 0x7ff3940d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 >> v1:172.21.15.184:6789/0 conn(0x7ff3940d3bd0 legacy=0x7ff3941620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff394160000 con 0x7ff39415f3e0 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7ff384001670 con 0x7ff39415f3e0 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 >> v1:172.21.15.40:6789/0 conn(0x7ff39415f3e0 legacy=0x7ff3940d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff380004800 0x7ff380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3800052e0 0x7ff38000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940d4040 con 0x7ff3800039b0 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff38000c1b0 con 0x7ff3800052e0 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff38000c3b0 con 0x7ff380004800 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff39259c640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff38bfff640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff380004800 0x7ff380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.527+0000 7ff392d9d640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3800052e0 0x7ff38000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff39259c640 1 -- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff380004800 msgr2=0x7ff380004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff39259c640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff380004800 0x7ff380004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff39259c640 1 -- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3800052e0 msgr2=0x7ff38000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff39259c640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3800052e0 0x7ff38000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff39259c640 1 -- 172.21.15.40:0/4036282681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff380003520 con 0x7ff3800039b0 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff392d9d640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3800052e0 0x7ff38000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff392d9d640 1 -- 172.21.15.40:0/4036282681 reap_dead start 2024-04-26T21:47:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff39259c640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c210 tx=0x7ff380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a2816511327f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384007d10 con 0x7ff3800039b0 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840058d0 con 0x7ff3800039b0 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.528+0000 7ff391d9b640 1 -- 172.21.15.40:0/4036282681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384005d20 con 0x7ff3800039b0 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.529+0000 7ff398820640 1 -- 172.21.15.40:0/4036282681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 msgr2=0x7ff380003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.529+0000 7ff398820640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c210 tx=0x7ff380005830 comp rx=0 tx=0).stop 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.529+0000 7ff398820640 1 -- 172.21.15.40:0/4036282681 shutdown_connections 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.529+0000 7ff398820640 1 --2- 172.21.15.40:0/4036282681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.529+0000 7ff398820640 1 -- 172.21.15.40:0/4036282681 >> 172.21.15.40:0/4036282681 conn(0x7ff39409e3c0 msgr2=0x7ff3940d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.529+0000 7ff398820640 1 -- 172.21.15.40:0/4036282681 shutdown_connections 2024-04-26T21:47:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.529+0000 7ff398820640 1 -- 172.21.15.40:0/4036282681 wait complete. 2024-04-26T21:47:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.530+0000 7ff398820640 1 Processor -- start 2024-04-26T21:47:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.530+0000 7ff398820640 1 -- start start 2024-04-26T21:47:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff398820640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff39415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff398820640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3941ea840 0x7ff3941eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff398820640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff39415d250 0x7ff39415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff398820640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff394160d40 con 0x7ff3800039b0 2024-04-26T21:47:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff398820640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941605c0 con 0x7ff39415d250 2024-04-26T21:47:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff398820640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940d3690 con 0x7ff3941ea840 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff39259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff39415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff39259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff39415a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34798/0 (socket says 172.21.15.40:34798) 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff39259c640 1 -- 172.21.15.40:0/816949076 learned_addr learned my addr 172.21.15.40:0/816949076 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff392d9d640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff39415d250 0x7ff39415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.531+0000 7ff38bfff640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3941ea840 0x7ff3941eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff39259c640 1 -- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3941ea840 msgr2=0x7ff3941eacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff39259c640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3941ea840 0x7ff3941eacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff39259c640 1 -- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff39415d250 msgr2=0x7ff39415d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff39259c640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff39415d250 0x7ff39415d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff39259c640 1 -- 172.21.15.40:0/816949076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3940b8520 con 0x7ff3800039b0 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff39259c640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff39415a170 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c330 tx=0x7ff3941608a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff38affd640 1 -- 172.21.15.40:0/816949076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384041020 con 0x7ff3800039b0 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff38affd640 1 -- 172.21.15.40:0/816949076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840049e0 con 0x7ff3800039b0 2024-04-26T21:47:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39415a6b0 con 0x7ff3800039b0 2024-04-26T21:47:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff38affd640 1 -- 172.21.15.40:0/816949076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384038040 con 0x7ff3800039b0 2024-04-26T21:47:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.532+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941e93c0 con 0x7ff3800039b0 2024-04-26T21:47:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.533+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3940cf830 con 0x7ff3800039b0 2024-04-26T21:47:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.534+0000 7ff38affd640 1 -- 172.21.15.40:0/816949076 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff384044020 con 0x7ff3800039b0 2024-04-26T21:47:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.534+0000 7ff38affd640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff37406c8f0 0x7ff37406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.534+0000 7ff38affd640 1 -- 172.21.15.40:0/816949076 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5249+0+0 (secure 0 0 0) 0x7ff3840ab9d0 con 0x7ff3800039b0 2024-04-26T21:47:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.534+0000 7ff38bfff640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff37406c8f0 0x7ff37406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.535+0000 7ff38bfff640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff37406c8f0 0x7ff37406edb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff3940b60f0 tx=0x7ff37c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:18.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.538+0000 7ff38affd640 1 -- 172.21.15.40:0/816949076 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3940cf830 con 0x7ff3800039b0 2024-04-26T21:47:18.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:18.721+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff3940d57e0 con 0x7ff3800039b0 2024-04-26T21:47:19.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.071+0000 7ff38affd640 1 -- 172.21.15.40:0/816949076 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) ==== 138+0+0 (secure 0 0 0) 0x7ff38404b030 con 0x7ff3800039b0 2024-04-26T21:47:19.072 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff37406c8f0 msgr2=0x7ff37406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff37406c8f0 0x7ff37406edb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff3940b60f0 tx=0x7ff37c008040 comp rx=0 tx=0).stop 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 msgr2=0x7ff39415a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff39415a170 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c330 tx=0x7ff3941608a0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 shutdown_connections 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff37406c8f0 0x7ff37406edb0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff39415d250 0x7ff39415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3941ea840 0x7ff3941eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 --2- 172.21.15.40:0/816949076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3800039b0 0x7ff39415a170 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 >> 172.21.15.40:0/816949076 conn(0x7ff39409e3c0 msgr2=0x7ff3940d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 shutdown_connections 2024-04-26T21:47:19.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.074+0000 7ff398820640 1 -- 172.21.15.40:0/816949076 wait complete. 2024-04-26T21:47:19.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:47:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.200+0000 7f87a5c99640 1 Processor -- start 2024-04-26T21:47:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.200+0000 7f87a5c99640 1 -- start start 2024-04-26T21:47:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f87a5c99640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87a0070350 0x7f87a0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f87a5c99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a0071090 0x7f87a015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f87a5c99640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87a00d2390 con 0x7f87a00cfde0 2024-04-26T21:47:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f87a5c99640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f87a0071500 con 0x7f87a0070350 2024-04-26T21:47:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f87a5c99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87a00afab0 con 0x7f87a0071090 2024-04-26T21:47:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f879effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87a0070350 0x7f87a0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f879effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87a0070350 0x7f87a0070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57818/0 (socket says 172.21.15.40:57818) 2024-04-26T21:47:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f879effd640 1 -- 172.21.15.40:0/946292770 learned_addr learned my addr 172.21.15.40:0/946292770 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f879ffff640 1 --2- 172.21.15.40:0/946292770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a0071090 0x7f87a015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f879effd640 1 -- 172.21.15.40:0/946292770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a0071090 msgr2=0x7f87a015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f879effd640 1 --2- 172.21.15.40:0/946292770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a0071090 0x7f87a015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f879effd640 1 -- 172.21.15.40:0/946292770 >> v1:172.21.15.184:6789/0 conn(0x7f87a00cfde0 legacy=0x7f87a00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.201+0000 7f879effd640 1 -- 172.21.15.40:0/946292770 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a006d3f0 con 0x7f87a0070350 2024-04-26T21:47:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.202+0000 7f879effd640 1 --2- 172.21.15.40:0/946292770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a0070b50 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f8794009870 tx=0x7f879402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fa9c1c6455c703 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.202+0000 7f879e7fc640 1 -- 172.21.15.40:0/946292770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794037070 con 0x7f87a0070350 2024-04-26T21:47:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.202+0000 7f879e7fc640 1 -- 172.21.15.40:0/946292770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8794032440 con 0x7f87a0070350 2024-04-26T21:47:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.202+0000 7f879e7fc640 1 -- 172.21.15.40:0/946292770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794032810 con 0x7f87a0070350 2024-04-26T21:47:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.203+0000 7f87a5c99640 1 -- 172.21.15.40:0/946292770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 msgr2=0x7f87a0070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.203+0000 7f87a5c99640 1 --2- 172.21.15.40:0/946292770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a0070b50 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f8794009870 tx=0x7f879402a180 comp rx=0 tx=0).stop 2024-04-26T21:47:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.203+0000 7f87a5c99640 1 -- 172.21.15.40:0/946292770 shutdown_connections 2024-04-26T21:47:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.203+0000 7f87a5c99640 1 --2- 172.21.15.40:0/946292770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a0071090 0x7f87a015e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.203+0000 7f87a5c99640 1 --2- 172.21.15.40:0/946292770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a0070b50 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.203+0000 7f87a5c99640 1 -- 172.21.15.40:0/946292770 >> 172.21.15.40:0/946292770 conn(0x7f87a00d2d80 msgr2=0x7f87a00d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.203+0000 7f87a5c99640 1 -- 172.21.15.40:0/946292770 shutdown_connections 2024-04-26T21:47:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.203+0000 7f87a5c99640 1 -- 172.21.15.40:0/946292770 wait complete. 2024-04-26T21:47:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.204+0000 7f87a5c99640 1 Processor -- start 2024-04-26T21:47:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.204+0000 7f87a5c99640 1 -- start start 2024-04-26T21:47:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f87a5c99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f87a5c99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a0071090 0x7f87a00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f87a5c99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a00cfde0 0x7f87a00a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f87a5c99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f87a006d9b0 con 0x7f87a0070350 2024-04-26T21:47:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f87a5c99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87a006dd20 con 0x7f87a0071090 2024-04-26T21:47:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f87a5c99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87a015ef30 con 0x7f87a00cfde0 2024-04-26T21:47:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a015a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57820/0 (socket says 172.21.15.40:57820) 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 -- 172.21.15.40:0/3840143076 learned_addr learned my addr 172.21.15.40:0/3840143076 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879effd640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a0071090 0x7f87a00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879ffff640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a00cfde0 0x7f87a00a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 -- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a00cfde0 msgr2=0x7f87a00a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a00cfde0 0x7f87a00a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 -- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a0071090 msgr2=0x7f87a00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a0071090 0x7f87a00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 -- 172.21.15.40:0/3840143076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a009dc20 con 0x7f87a0070350 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f879f7fe640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a015a7e0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f879000b3f0 tx=0x7f879000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f8787fff640 1 -- 172.21.15.40:0/3840143076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790015020 con 0x7f87a0070350 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f8787fff640 1 -- 172.21.15.40:0/3840143076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8790011830 con 0x7f87a0070350 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a009df00 con 0x7f87a0070350 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f8787fff640 1 -- 172.21.15.40:0/3840143076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790011c20 con 0x7f87a0070350 2024-04-26T21:47:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.206+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a009f8d0 con 0x7f87a0070350 2024-04-26T21:47:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.207+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a00694f0 con 0x7f87a0070350 2024-04-26T21:47:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.207+0000 7f8787fff640 1 -- 172.21.15.40:0/3840143076 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8790018020 con 0x7f87a0070350 2024-04-26T21:47:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.208+0000 7f8787fff640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.208+0000 7f8787fff640 1 -- 172.21.15.40:0/3840143076 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5619+0+0 (secure 0 0 0) 0x7f879008b7a0 con 0x7f87a0070350 2024-04-26T21:47:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.208+0000 7f879effd640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.208+0000 7f879effd640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8794034000 tx=0x7f879402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.213+0000 7f8787fff640 1 -- 172.21.15.40:0/3840143076 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f879001f030 con 0x7f87a0070350 2024-04-26T21:47:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:19.398+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f87a006ce60 con 0x7f87a0070350 2024-04-26T21:47:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.075+0000 7f8787fff640 1 -- 172.21.15.40:0/3840143076 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) ==== 170+0+0 (secure 0 0 0) 0x7f87900172c0 con 0x7f87a0070350 2024-04-26T21:47:20.075 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:47:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.079+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 msgr2=0x7f878c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.079+0000 7f87a5c99640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8794034000 tx=0x7f879402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:20.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.079+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 msgr2=0x7f87a015a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.079+0000 7f87a5c99640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a015a7e0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f879000b3f0 tx=0x7f879000b8b0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.084+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 shutdown_connections 2024-04-26T21:47:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.084+0000 7f87a5c99640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.084+0000 7f87a5c99640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87a00cfde0 0x7f87a00a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.084+0000 7f87a5c99640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87a0071090 0x7f87a00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.084+0000 7f87a5c99640 1 --2- 172.21.15.40:0/3840143076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87a0070350 0x7f87a015a7e0 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.084+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 >> 172.21.15.40:0/3840143076 conn(0x7f87a00d2d80 msgr2=0x7f87a00b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:20.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.091+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 shutdown_connections 2024-04-26T21:47:20.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.091+0000 7f87a5c99640 1 -- 172.21.15.40:0/3840143076 wait complete. 2024-04-26T21:47:20.110 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:47:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.219+0000 7f13d7989640 1 Processor -- start 2024-04-26T21:47:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.220+0000 7f13d7989640 1 -- start start 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.220+0000 7f13d7989640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13d00d1a00 0x7f13d00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.220+0000 7f13d7989640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00b40d0 0x7f13d00b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.220+0000 7f13d7989640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d00d0a10 con 0x7f13d00d2320 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.220+0000 7f13d7989640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f13d00d0cf0 con 0x7f13d00d1a00 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.220+0000 7f13d7989640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13d00d0fd0 con 0x7f13d00b40d0 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13d4f24640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f13d00d2320 0x7f13d01625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46236/0 (socket says 172.21.15.40:46236) 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13d4f24640 1 -- 172.21.15.40:0/2438123243 learned_addr learned my addr 172.21.15.40:0/2438123243 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13d5725640 1 --2- 172.21.15.40:0/2438123243 >> v2:172.21.15.184:3300/0 conn(0x7f13d00d1a00 0x7f13d00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13d5f26640 1 --2- 172.21.15.40:0/2438123243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00b40d0 0x7f13d00b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13c7fff640 1 -- 172.21.15.40:0/2438123243 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321988469 0 0) 0x7f13d00d0a10 con 0x7f13d00d2320 2024-04-26T21:47:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13c7fff640 1 -- 172.21.15.40:0/2438123243 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13bc003520 con 0x7f13d00d2320 2024-04-26T21:47:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13c7fff640 1 -- 172.21.15.40:0/2438123243 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1716609654 0 0) 0x7f13bc003520 con 0x7f13d00d2320 2024-04-26T21:47:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13d5725640 1 -- 172.21.15.40:0/2438123243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00b40d0 msgr2=0x7f13d00b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13d5725640 1 --2- 172.21.15.40:0/2438123243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00b40d0 0x7f13d00b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13d5725640 1 -- 172.21.15.40:0/2438123243 >> v1:172.21.15.40:6789/0 conn(0x7f13d00d2320 legacy=0x7f13d01625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.221+0000 7f13d5725640 1 -- 172.21.15.40:0/2438123243 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d00d40a0 con 0x7f13d00d1a00 2024-04-26T21:47:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.222+0000 7f13d5725640 1 --2- 172.21.15.40:0/2438123243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00d1a00 0x7f13d00d1de0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0097a0 tx=0x7f13cc02a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f6dab3df480d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.222+0000 7f13c7fff640 1 -- 172.21.15.40:0/2438123243 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc037070 con 0x7f13d00d1a00 2024-04-26T21:47:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.222+0000 7f13c7fff640 1 -- 172.21.15.40:0/2438123243 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc02aa90 con 0x7f13d00d1a00 2024-04-26T21:47:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.222+0000 7f13c7fff640 1 -- 172.21.15.40:0/2438123243 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0001670 con 0x7f13d00d1a00 2024-04-26T21:47:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.223+0000 7f13d7989640 1 -- 172.21.15.40:0/2438123243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00d1a00 msgr2=0x7f13d00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.223+0000 7f13d7989640 1 --2- 172.21.15.40:0/2438123243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00d1a00 0x7f13d00d1de0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0097a0 tx=0x7f13cc02a000 comp rx=0 tx=0).stop 2024-04-26T21:47:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.223+0000 7f13d7989640 1 -- 172.21.15.40:0/2438123243 shutdown_connections 2024-04-26T21:47:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.223+0000 7f13d7989640 1 --2- 172.21.15.40:0/2438123243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00b40d0 0x7f13d00b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.223+0000 7f13d7989640 1 --2- 172.21.15.40:0/2438123243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00d1a00 0x7f13d00d1de0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.223+0000 7f13d7989640 1 -- 172.21.15.40:0/2438123243 >> 172.21.15.40:0/2438123243 conn(0x7f13d00d4fd0 msgr2=0x7f13d00d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:20.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.223+0000 7f13d7989640 1 -- 172.21.15.40:0/2438123243 shutdown_connections 2024-04-26T21:47:20.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.223+0000 7f13d7989640 1 -- 172.21.15.40:0/2438123243 wait complete. 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.224+0000 7f13d7989640 1 Processor -- start 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.224+0000 7f13d7989640 1 -- start start 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d7989640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00b40d0 0x7f13d014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d7989640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00d1a00 0x7f13d006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d7989640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d00d2320 0x7f13d006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d7989640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13d00b5040 con 0x7f13d00d2320 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d7989640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0162f50 con 0x7f13d00b40d0 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d7989640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13d00b51c0 con 0x7f13d00d1a00 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d00d2320 0x7f13d006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d00d2320 0x7f13d006d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57836/0 (socket says 172.21.15.40:57836) 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 -- 172.21.15.40:0/524274760 learned_addr learned my addr 172.21.15.40:0/524274760 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5725640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00b40d0 0x7f13d014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d4f24640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00d1a00 0x7f13d006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 -- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00d1a00 msgr2=0x7f13d006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00d1a00 0x7f13d006cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 -- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00b40d0 msgr2=0x7f13d014e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00b40d0 0x7f13d014e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 -- 172.21.15.40:0/524274760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d0151ed0 con 0x7f13d00d2320 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5725640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00b40d0 0x7f13d014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d4f24640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00d1a00 0x7f13d006cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d5f26640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d00d2320 0x7f13d006d1a0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f13c800efc0 tx=0x7f13c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13c5ffb640 1 -- 172.21.15.40:0/524274760 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8018020 con 0x7f13d00d2320 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13c5ffb640 1 -- 172.21.15.40:0/524274760 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c8014480 con 0x7f13d00d2320 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d014ebd0 con 0x7f13d00d2320 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.225+0000 7f13c5ffb640 1 -- 172.21.15.40:0/524274760 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8014870 con 0x7f13d00d2320 2024-04-26T21:47:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.226+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d01f5040 con 0x7f13d00d2320 2024-04-26T21:47:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.227+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d00694f0 con 0x7f13d00d2320 2024-04-26T21:47:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.227+0000 7f13c5ffb640 1 -- 172.21.15.40:0/524274760 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13c8005d10 con 0x7f13d00d2320 2024-04-26T21:47:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.227+0000 7f13c5ffb640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b406c8f0 0x7f13b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.228+0000 7f13c5ffb640 1 -- 172.21.15.40:0/524274760 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6001+0+0 (secure 0 0 0) 0x7f13c808caf0 con 0x7f13d00d2320 2024-04-26T21:47:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.228+0000 7f13d5725640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b406c8f0 0x7f13b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.228+0000 7f13d5725640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b406c8f0 0x7f13b406edb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f13cc034000 tx=0x7f13cc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:20.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.232+0000 7f13c5ffb640 1 -- 172.21.15.40:0/524274760 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13c8022030 con 0x7f13d00d2320 2024-04-26T21:47:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:20.455+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f13d00d5530 con 0x7f13d00d2320 2024-04-26T21:47:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.093+0000 7f13c5ffb640 1 -- 172.21.15.40:0/524274760 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v355) ==== 174+0+0 (secure 0 0 0) 0x7f13c8010040 con 0x7f13d00d2320 2024-04-26T21:47:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 15 and data pool 16 2024-04-26T21:47:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.098+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b406c8f0 msgr2=0x7f13b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b406c8f0 0x7f13b406edb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f13cc034000 tx=0x7f13cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d00d2320 msgr2=0x7f13d006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d00d2320 0x7f13d006d1a0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f13c800efc0 tx=0x7f13c800c490 comp rx=0 tx=0).stop 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 shutdown_connections 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13b406c8f0 0x7f13b406edb0 secure :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f13cc034000 tx=0x7f13cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d00d2320 0x7f13d006d1a0 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13d00d1a00 0x7f13d006cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 --2- 172.21.15.40:0/524274760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13d00b40d0 0x7f13d014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.099+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 >> 172.21.15.40:0/524274760 conn(0x7f13d00d4fd0 msgr2=0x7f13d0161640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.100+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 shutdown_connections 2024-04-26T21:47:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.100+0000 7f13d7989640 1 -- 172.21.15.40:0/524274760 wait complete. 2024-04-26T21:47:21.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:21.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.236+0000 7f9daba52640 1 Processor -- start 2024-04-26T21:47:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.236+0000 7f9daba52640 1 -- start start 2024-04-26T21:47:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9daba52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9da40d3bc0 0x7f9da40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9daba52640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da40b7e50 con 0x7f9da4162880 2024-04-26T21:47:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9daba52640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da40d1800 con 0x7f9da40b5f20 2024-04-26T21:47:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9daba52640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9da40d0580 con 0x7f9da40d3bc0 2024-04-26T21:47:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9da8fed640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9da40d3bc0 0x7f9da40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9da97ee640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9da4162880 0x7f9da40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46244/0 (socket says 172.21.15.40:46244) 2024-04-26T21:47:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9da97ee640 1 -- 172.21.15.40:0/1233793797 learned_addr learned my addr 172.21.15.40:0/1233793797 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9da8fed640 1 -- 172.21.15.40:0/1233793797 >> v1:172.21.15.184:6789/0 conn(0x7f9da40b5f20 legacy=0x7f9da40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9da8fed640 1 -- 172.21.15.40:0/1233793797 >> v1:172.21.15.40:6789/0 conn(0x7f9da4162880 legacy=0x7f9da40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.237+0000 7f9da8fed640 1 -- 172.21.15.40:0/1233793797 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da40d07d0 con 0x7f9da40d3bc0 2024-04-26T21:47:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.238+0000 7f9da8fed640 1 --2- 172.21.15.40:0/1233793797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da40d2cb0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f9d940097a0 tx=0x7f9d9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a083a03a410cb09a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.238+0000 7f9d9bfff640 1 -- 172.21.15.40:0/1233793797 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94037070 con 0x7f9da40d3bc0 2024-04-26T21:47:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.238+0000 7f9d9bfff640 1 -- 172.21.15.40:0/1233793797 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94032440 con 0x7f9da40d3bc0 2024-04-26T21:47:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.238+0000 7f9d9bfff640 1 -- 172.21.15.40:0/1233793797 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94032810 con 0x7f9da40d3bc0 2024-04-26T21:47:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.239+0000 7f9daba52640 1 -- 172.21.15.40:0/1233793797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 msgr2=0x7f9da40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.239+0000 7f9daba52640 1 --2- 172.21.15.40:0/1233793797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da40d2cb0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f9d940097a0 tx=0x7f9d9402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.239+0000 7f9daba52640 1 -- 172.21.15.40:0/1233793797 shutdown_connections 2024-04-26T21:47:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.239+0000 7f9daba52640 1 --2- 172.21.15.40:0/1233793797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da40d2cb0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.239+0000 7f9daba52640 1 -- 172.21.15.40:0/1233793797 >> 172.21.15.40:0/1233793797 conn(0x7f9da40d4f60 msgr2=0x7f9da40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.239+0000 7f9daba52640 1 -- 172.21.15.40:0/1233793797 shutdown_connections 2024-04-26T21:47:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.239+0000 7f9daba52640 1 -- 172.21.15.40:0/1233793797 wait complete. 2024-04-26T21:47:21.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.240+0000 7f9daba52640 1 Processor -- start 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.240+0000 7f9daba52640 1 -- start start 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9daba52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40b5f20 0x7f9da41f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9daba52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da41f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9daba52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da4162880 0x7f9da40b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9daba52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4162c60 con 0x7f9da40d3bc0 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9daba52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9da40d0d90 con 0x7f9da4162880 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9daba52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9da4162de0 con 0x7f9da40b5f20 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da41f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da41f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57862/0 (socket says 172.21.15.40:57862) 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 -- 172.21.15.40:0/2036598755 learned_addr learned my addr 172.21.15.40:0/2036598755 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 -- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40b5f20 msgr2=0x7f9da41f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40b5f20 0x7f9da41f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 -- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da4162880 msgr2=0x7f9da40b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da4162880 0x7f9da40b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 -- 172.21.15.40:0/2036598755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da40b30d0 con 0x7f9da40d3bc0 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.241+0000 7f9da8fed640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da41f76a0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f9d94000c00 tx=0x7f9d940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.242+0000 7f9d99ffb640 1 -- 172.21.15.40:0/2036598755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94037040 con 0x7f9da40d3bc0 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.242+0000 7f9d99ffb640 1 -- 172.21.15.40:0/2036598755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94042070 con 0x7f9da40d3bc0 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.242+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da40b33b0 con 0x7f9da40d3bc0 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.242+0000 7f9d99ffb640 1 -- 172.21.15.40:0/2036598755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94030a70 con 0x7f9da40d3bc0 2024-04-26T21:47:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.242+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da40b3970 con 0x7f9da40d3bc0 2024-04-26T21:47:21.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.243+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da40694f0 con 0x7f9da40d3bc0 2024-04-26T21:47:21.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.243+0000 7f9d99ffb640 1 -- 172.21.15.40:0/2036598755 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d94030c10 con 0x7f9da40d3bc0 2024-04-26T21:47:21.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.244+0000 7f9d99ffb640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8806c8f0 0x7f9d8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.244+0000 7f9d99ffb640 1 -- 172.21.15.40:0/2036598755 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6101+0+0 (secure 0 0 0) 0x7f9d940ac7c0 con 0x7f9da40d3bc0 2024-04-26T21:47:21.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.244+0000 7f9da97ee640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8806c8f0 0x7f9d8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.245+0000 7f9da97ee640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8806c8f0 0x7f9d8806edb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9da0000f60 tx=0x7f9da0007a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.248+0000 7f9d99ffb640 1 -- 172.21.15.40:0/2036598755 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9da40d1800 con 0x7f9da40d3bc0 2024-04-26T21:47:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.437+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9da40ce420 con 0x7f9da40d3bc0 2024-04-26T21:47:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.438+0000 7f9d99ffb640 1 -- 172.21.15.40:0/2036598755 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16861 (secure 0 0 0) 0x7f9d94045360 con 0x7f9da40d3bc0 2024-04-26T21:47:21.438 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:21.438 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":144,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:20.455913+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:18.726844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:19.403532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8806c8f0 msgr2=0x7f9d8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8806c8f0 0x7f9d8806edb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9da0000f60 tx=0x7f9da0007a80 comp rx=0 tx=0).stop 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 msgr2=0x7f9da41f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da41f76a0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f9d94000c00 tx=0x7f9d940046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 shutdown_connections 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9d8806c8f0 0x7f9d8806edb0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da4162880 0x7f9da40b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9da40d3bc0 0x7f9da41f76a0 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 --2- 172.21.15.40:0/2036598755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40b5f20 0x7f9da41f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 >> 172.21.15.40:0/2036598755 conn(0x7f9da40d4f60 msgr2=0x7f9da406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 shutdown_connections 2024-04-26T21:47:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.440+0000 7f9daba52640 1 -- 172.21.15.40:0/2036598755 wait complete. 2024-04-26T21:47:21.451 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.551+0000 7f7436fb4640 1 Processor -- start 2024-04-26T21:47:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.552+0000 7f7436fb4640 1 -- start start 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.552+0000 7f7436fb4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f743015f1d0 0x7f743006f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.552+0000 7f7436fb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74300afd60 0x7f743006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.552+0000 7f7436fb4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74300b4480 con 0x7f743006c240 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.552+0000 7f7436fb4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f743006cbe0 con 0x7f743015f1d0 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.552+0000 7f7436fb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74300ae8c0 con 0x7f74300afd60 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f7435551640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74300afd60 0x7f743006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f7434d50640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f743015f1d0 0x7f743006f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f7435551640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74300afd60 0x7f743006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:45166/0 (socket says 172.21.15.40:45166) 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f742ffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f743006c240 0x7f74300726e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:46062/0 (socket says 172.21.15.40:46062) 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f7435551640 1 -- 172.21.15.40:0/474685988 learned_addr learned my addr 172.21.15.40:0/474685988 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f7435551640 1 -- 172.21.15.40:0/474685988 >> v2:172.21.15.184:3300/0 conn(0x7f743015f1d0 msgr2=0x7f743006f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f7435551640 1 --2- 172.21.15.40:0/474685988 >> v2:172.21.15.184:3300/0 conn(0x7f743015f1d0 0x7f743006f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f7435551640 1 -- 172.21.15.40:0/474685988 >> v1:172.21.15.184:6789/0 conn(0x7f743006c240 legacy=0x7f74300726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.553+0000 7f7435551640 1 -- 172.21.15.40:0/474685988 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74300b4720 con 0x7f74300afd60 2024-04-26T21:47:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.554+0000 7f7435551640 1 --2- 172.21.15.40:0/474685988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74300afd60 0x7f743006c6a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7420009870 tx=0x7f742002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaffc22e4e4e67e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.554+0000 7f742f7fe640 1 -- 172.21.15.40:0/474685988 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420037070 con 0x7f74300afd60 2024-04-26T21:47:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.554+0000 7f742f7fe640 1 -- 172.21.15.40:0/474685988 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7420032440 con 0x7f74300afd60 2024-04-26T21:47:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.554+0000 7f742f7fe640 1 -- 172.21.15.40:0/474685988 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420032810 con 0x7f74300afd60 2024-04-26T21:47:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.555+0000 7f7436fb4640 1 -- 172.21.15.40:0/474685988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74300afd60 msgr2=0x7f743006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.555+0000 7f7436fb4640 1 --2- 172.21.15.40:0/474685988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74300afd60 0x7f743006c6a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7420009870 tx=0x7f742002a270 comp rx=0 tx=0).stop 2024-04-26T21:47:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.555+0000 7f7436fb4640 1 -- 172.21.15.40:0/474685988 shutdown_connections 2024-04-26T21:47:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.555+0000 7f7436fb4640 1 --2- 172.21.15.40:0/474685988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74300afd60 0x7f743006c6a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.555+0000 7f7436fb4640 1 --2- 172.21.15.40:0/474685988 >> v2:172.21.15.184:3300/0 conn(0x7f743015f1d0 0x7f743006f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.555+0000 7f7436fb4640 1 -- 172.21.15.40:0/474685988 >> 172.21.15.40:0/474685988 conn(0x7f74300b3180 msgr2=0x7f74300b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.555+0000 7f7436fb4640 1 -- 172.21.15.40:0/474685988 shutdown_connections 2024-04-26T21:47:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.555+0000 7f7436fb4640 1 -- 172.21.15.40:0/474685988 wait complete. 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.556+0000 7f7436fb4640 1 Processor -- start 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.556+0000 7f7436fb4640 1 -- start start 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7436fb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743006c240 0x7f7430149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7436fb4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74300afd60 0x7f7430149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7436fb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f743015f1d0 0x7f74301f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7436fb4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74300b4fc0 con 0x7f743015f1d0 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7436fb4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74300b4ce0 con 0x7f74300afd60 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7436fb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f743006f6b0 con 0x7f743006c240 2024-04-26T21:47:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7435551640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f743015f1d0 0x7f74301f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7435551640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f743015f1d0 0x7f74301f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57868/0 (socket says 172.21.15.40:57868) 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7435551640 1 -- 172.21.15.40:0/843954701 learned_addr learned my addr 172.21.15.40:0/843954701 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f742ffff640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74300afd60 0x7f7430149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7434d50640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743006c240 0x7f7430149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7435551640 1 -- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743006c240 msgr2=0x7f7430149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7435551640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743006c240 0x7f7430149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7435551640 1 -- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74300afd60 msgr2=0x7f7430149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7435551640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74300afd60 0x7f7430149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f7435551640 1 -- 172.21.15.40:0/843954701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7430161f70 con 0x7f743015f1d0 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.557+0000 7f742ffff640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74300afd60 0x7f7430149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.558+0000 7f7435551640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f743015f1d0 0x7f74301f8630 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f74200097c0 tx=0x7f74200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.558+0000 7f742d7fa640 1 -- 172.21.15.40:0/843954701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420041020 con 0x7f743015f1d0 2024-04-26T21:47:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.558+0000 7f742d7fa640 1 -- 172.21.15.40:0/843954701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742003d700 con 0x7f743015f1d0 2024-04-26T21:47:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.558+0000 7f742d7fa640 1 -- 172.21.15.40:0/843954701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420037040 con 0x7f743015f1d0 2024-04-26T21:47:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.558+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74301619b0 con 0x7f743015f1d0 2024-04-26T21:47:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.558+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7430154f90 con 0x7f743015f1d0 2024-04-26T21:47:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.559+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7430154d80 con 0x7f743015f1d0 2024-04-26T21:47:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.559+0000 7f742d7fa640 1 -- 172.21.15.40:0/843954701 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7420044020 con 0x7f743015f1d0 2024-04-26T21:47:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.560+0000 7f742d7fa640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f741006c8f0 0x7f741006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.560+0000 7f742d7fa640 1 -- 172.21.15.40:0/843954701 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6101+0+0 (secure 0 0 0) 0x7f74200abfc0 con 0x7f743015f1d0 2024-04-26T21:47:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.560+0000 7f7434d50640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f741006c8f0 0x7f741006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.561+0000 7f7434d50640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f741006c8f0 0x7f741006edb0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f74280074a0 tx=0x7f742800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.564+0000 7f742d7fa640 1 -- 172.21.15.40:0/843954701 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f742004b070 con 0x7f743015f1d0 2024-04-26T21:47:21.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.750+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74301629f0 con 0x7f743015f1d0 2024-04-26T21:47:21.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.751+0000 7f742d7fa640 1 -- 172.21.15.40:0/843954701 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16861 (secure 0 0 0) 0x7f742007a2f0 con 0x7f743015f1d0 2024-04-26T21:47:21.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:21.751 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":144,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:20.455913+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:18.726844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:19.403532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:21.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f741006c8f0 msgr2=0x7f741006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f741006c8f0 0x7f741006edb0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f74280074a0 tx=0x7f742800b040 comp rx=0 tx=0).stop 2024-04-26T21:47:21.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f743015f1d0 msgr2=0x7f74301f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f743015f1d0 0x7f74301f8630 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f74200097c0 tx=0x7f74200046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 shutdown_connections 2024-04-26T21:47:21.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f741006c8f0 0x7f741006edb0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f743015f1d0 0x7f74301f8630 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74300afd60 0x7f7430149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 --2- 172.21.15.40:0/843954701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743006c240 0x7f7430149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 >> 172.21.15.40:0/843954701 conn(0x7f74300b3180 msgr2=0x7f74300b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 shutdown_connections 2024-04-26T21:47:21.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.753+0000 7f7436fb4640 1 -- 172.21.15.40:0/843954701 wait complete. 2024-04-26T21:47:21.764 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:47:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.868+0000 7f9b80f39640 1 Processor -- start 2024-04-26T21:47:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.868+0000 7f9b80f39640 1 -- start start 2024-04-26T21:47:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b80f39640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b7c0cfde0 0x7f9b7c0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b80f39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c070350 0x7f9b7c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b80f39640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9b7c071090 0x7f9b7c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b80f39640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7c071500 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b80f39640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7c0afab0 con 0x7f9b7c071090 2024-04-26T21:47:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b80f39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c06dd20 con 0x7f9b7c070350 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b7c0cfde0 0x7f9b7c0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b7c0cfde0 0x7f9b7c0d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57880/0 (socket says 172.21.15.40:57880) 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7a59c640 1 -- 172.21.15.40:0/715142482 learned_addr learned my addr 172.21.15.40:0/715142482 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7ad9d640 1 --2- 172.21.15.40:0/715142482 >> v2:172.21.15.184:3300/0 conn(0x7f9b7c071090 0x7f9b7c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b73fff640 1 --2- 172.21.15.40:0/715142482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c070350 0x7f9b7c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7a59c640 1 -- 172.21.15.40:0/715142482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c070350 msgr2=0x7f9b7c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7a59c640 1 --2- 172.21.15.40:0/715142482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c070350 0x7f9b7c070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7a59c640 1 -- 172.21.15.40:0/715142482 >> v2:172.21.15.184:3300/0 conn(0x7f9b7c071090 msgr2=0x7f9b7c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7a59c640 1 --2- 172.21.15.40:0/715142482 >> v2:172.21.15.184:3300/0 conn(0x7f9b7c071090 0x7f9b7c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b7a59c640 1 -- 172.21.15.40:0/715142482 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c0d2390 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.869+0000 7f9b73fff640 1 --2- 172.21.15.40:0/715142482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c070350 0x7f9b7c070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.870+0000 7f9b7a59c640 1 --2- 172.21.15.40:0/715142482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c0d1b60 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0732e0 tx=0x7f9b6c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d9a5499c81cd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.870+0000 7f9b79d9b640 1 -- 172.21.15.40:0/715142482 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c037070 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.870+0000 7f9b79d9b640 1 -- 172.21.15.40:0/715142482 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c032440 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.870+0000 7f9b79d9b640 1 -- 172.21.15.40:0/715142482 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c032810 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 -- 172.21.15.40:0/715142482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 msgr2=0x7f9b7c0d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 --2- 172.21.15.40:0/715142482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c0d1b60 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0732e0 tx=0x7f9b6c02a140 comp rx=0 tx=0).stop 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 -- 172.21.15.40:0/715142482 shutdown_connections 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 --2- 172.21.15.40:0/715142482 >> v2:172.21.15.184:3300/0 conn(0x7f9b7c071090 0x7f9b7c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 --2- 172.21.15.40:0/715142482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c070350 0x7f9b7c070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 --2- 172.21.15.40:0/715142482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c0d1b60 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 -- 172.21.15.40:0/715142482 >> 172.21.15.40:0/715142482 conn(0x7f9b7c0d2d80 msgr2=0x7f9b7c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 -- 172.21.15.40:0/715142482 shutdown_connections 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.871+0000 7f9b80f39640 1 -- 172.21.15.40:0/715142482 wait complete. 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.872+0000 7f9b80f39640 1 Processor -- start 2024-04-26T21:47:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.872+0000 7f9b80f39640 1 -- start start 2024-04-26T21:47:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b80f39640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b7c070350 0x7f9b7c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b80f39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c071090 0x7f9b7c1f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b80f39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c1f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b80f39640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c06d9b0 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b80f39640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c06d6d0 con 0x7f9b7c070350 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b80f39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c0d20a0 con 0x7f9b7c071090 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c1f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c1f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57892/0 (socket says 172.21.15.40:57892) 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 -- 172.21.15.40:0/2498703704 learned_addr learned my addr 172.21.15.40:0/2498703704 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b73fff640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c071090 0x7f9b7c1f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7a59c640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b7c070350 0x7f9b7c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 -- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c071090 msgr2=0x7f9b7c1f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c071090 0x7f9b7c1f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 -- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b7c070350 msgr2=0x7f9b7c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b7c070350 0x7f9b7c15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 -- 172.21.15.40:0/2498703704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c158e20 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b7ad9d640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c1f8720 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f9b74007e60 tx=0x7f9b7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b72ffd640 1 -- 172.21.15.40:0/2498703704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74014420 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b72ffd640 1 -- 172.21.15.40:0/2498703704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b74019070 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c0c8780 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.873+0000 7f9b72ffd640 1 -- 172.21.15.40:0/2498703704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74014820 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.874+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c0c8d40 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.874+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c158ff0 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.875+0000 7f9b72ffd640 1 -- 172.21.15.40:0/2498703704 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b740149c0 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.875+0000 7f9b72ffd640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b5c06c920 0x7f9b5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.875+0000 7f9b72ffd640 1 -- 172.21.15.40:0/2498703704 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6101+0+0 (secure 0 0 0) 0x7f9b7408c510 con 0x7f9b7c0cfde0 2024-04-26T21:47:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.876+0000 7f9b7a59c640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b5c06c920 0x7f9b5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.876+0000 7f9b7a59c640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b5c06c920 0x7f9b5c06ede0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c034000 tx=0x7f9b6c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:21.880+0000 7f9b72ffd640 1 -- 172.21.15.40:0/2498703704 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b7405a7b0 con 0x7f9b7c0cfde0 2024-04-26T21:47:22.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.105+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9b7c0694f0 con 0x7f9b7c0cfde0 2024-04-26T21:47:22.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.108+0000 7f9b72ffd640 1 -- 172.21.15.40:0/2498703704 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v356) ==== 118+0+0 (secure 0 0 0) 0x7f9b7405a0b0 con 0x7f9b7c0cfde0 2024-04-26T21:47:22.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.110+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b5c06c920 msgr2=0x7f9b5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:22.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.110+0000 7f9b80f39640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b5c06c920 0x7f9b5c06ede0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c034000 tx=0x7f9b6c02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.110+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 msgr2=0x7f9b7c1f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.110+0000 7f9b80f39640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c1f8720 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f9b74007e60 tx=0x7f9b7400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.111+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 shutdown_connections 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.111+0000 7f9b80f39640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9b5c06c920 0x7f9b5c06ede0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.111+0000 7f9b80f39640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b7c0cfde0 0x7f9b7c1f8720 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.111+0000 7f9b80f39640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b7c071090 0x7f9b7c1f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.111+0000 7f9b80f39640 1 --2- 172.21.15.40:0/2498703704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b7c070350 0x7f9b7c15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.111+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 >> 172.21.15.40:0/2498703704 conn(0x7f9b7c0d2d80 msgr2=0x7f9b7c0cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.111+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 shutdown_connections 2024-04-26T21:47:22.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.111+0000 7f9b80f39640 1 -- 172.21.15.40:0/2498703704 wait complete. 2024-04-26T21:47:22.122 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:47:22.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:47:22.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.228+0000 7f8c20586640 1 Processor -- start 2024-04-26T21:47:22.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.228+0000 7f8c20586640 1 -- start start 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c20586640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c181604e0 0x7f8c1806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c20586640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1806fcd0 0x7f8c180ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c20586640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c18160970 con 0x7f8c1815f2d0 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c20586640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8c18074010 con 0x7f8c181604e0 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c20586640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c18070100 con 0x7f8c1806fcd0 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c1db21640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c181604e0 0x7f8c1806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c1e322640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8c1815f2d0 0x7f8c18072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46250/0 (socket says 172.21.15.40:46250) 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c1e322640 1 -- 172.21.15.40:0/4011927161 learned_addr learned my addr 172.21.15.40:0/4011927161 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c1db21640 1 -- 172.21.15.40:0/4011927161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1806fcd0 msgr2=0x7f8c180ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c1db21640 1 --2- 172.21.15.40:0/4011927161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1806fcd0 0x7f8c180ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c1db21640 1 -- 172.21.15.40:0/4011927161 >> v1:172.21.15.40:6789/0 conn(0x7f8c1815f2d0 legacy=0x7f8c18072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:22.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.229+0000 7f8c1db21640 1 -- 172.21.15.40:0/4011927161 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c18072f70 con 0x7f8c181604e0 2024-04-26T21:47:22.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c1db21640 1 --2- 172.21.15.40:0/4011927161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c181604e0 0x7f8c1806f790 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097d0 tx=0x7f8c0802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362af527a3e2a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:22.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c1d320640 1 -- 172.21.15.40:0/4011927161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08037070 con 0x7f8c181604e0 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c1d320640 1 -- 172.21.15.40:0/4011927161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c08032440 con 0x7f8c181604e0 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c1d320640 1 -- 172.21.15.40:0/4011927161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08032810 con 0x7f8c181604e0 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c20586640 1 -- 172.21.15.40:0/4011927161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c181604e0 msgr2=0x7f8c1806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c20586640 1 --2- 172.21.15.40:0/4011927161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c181604e0 0x7f8c1806f790 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097d0 tx=0x7f8c0802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c20586640 1 -- 172.21.15.40:0/4011927161 shutdown_connections 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c20586640 1 --2- 172.21.15.40:0/4011927161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c1806fcd0 0x7f8c180ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c20586640 1 --2- 172.21.15.40:0/4011927161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c181604e0 0x7f8c1806f790 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.230+0000 7f8c20586640 1 -- 172.21.15.40:0/4011927161 >> 172.21.15.40:0/4011927161 conn(0x7f8c180d0be0 msgr2=0x7f8c180d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.231+0000 7f8c20586640 1 -- 172.21.15.40:0/4011927161 shutdown_connections 2024-04-26T21:47:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.231+0000 7f8c20586640 1 -- 172.21.15.40:0/4011927161 wait complete. 2024-04-26T21:47:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.232+0000 7f8c20586640 1 Processor -- start 2024-04-26T21:47:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.232+0000 7f8c20586640 1 -- start start 2024-04-26T21:47:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.232+0000 7f8c20586640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1806fcd0 0x7f8c180c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.232+0000 7f8c20586640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c1815f2d0 0x7f8c1814e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.232+0000 7f8c20586640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c181604e0 0x7f8c181560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.232+0000 7f8c20586640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18073810 con 0x7f8c1815f2d0 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.232+0000 7f8c20586640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18073530 con 0x7f8c1806fcd0 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.232+0000 7f8c20586640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c18073af0 con 0x7f8c181604e0 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c1815f2d0 0x7f8c1814e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c1815f2d0 0x7f8c1814e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57908/0 (socket says 172.21.15.40:57908) 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 -- 172.21.15.40:0/1763454928 learned_addr learned my addr 172.21.15.40:0/1763454928 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1e322640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1806fcd0 0x7f8c180c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1eb23640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c181604e0 0x7f8c181560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 -- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c181604e0 msgr2=0x7f8c181560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c181604e0 0x7f8c181560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 -- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1806fcd0 msgr2=0x7f8c180c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1806fcd0 0x7f8c180c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 -- 172.21.15.40:0/1763454928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c181565f0 con 0x7f8c1815f2d0 2024-04-26T21:47:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c1db21640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c1815f2d0 0x7f8c1814e9e0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097d0 tx=0x7f8c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:22.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c0effd640 1 -- 172.21.15.40:0/1763454928 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08037040 con 0x7f8c1815f2d0 2024-04-26T21:47:22.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c0effd640 1 -- 172.21.15.40:0/1763454928 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c08030770 con 0x7f8c1815f2d0 2024-04-26T21:47:22.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c180c6a70 con 0x7f8c1815f2d0 2024-04-26T21:47:22.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c0effd640 1 -- 172.21.15.40:0/1763454928 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08030b40 con 0x7f8c1815f2d0 2024-04-26T21:47:22.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.233+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c180c7030 con 0x7f8c1815f2d0 2024-04-26T21:47:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.234+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c180694f0 con 0x7f8c1815f2d0 2024-04-26T21:47:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.235+0000 7f8c0effd640 1 -- 172.21.15.40:0/1763454928 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c08044020 con 0x7f8c1815f2d0 2024-04-26T21:47:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.235+0000 7f8c0effd640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.235+0000 7f8c0effd640 1 -- 172.21.15.40:0/1763454928 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f8c080ac030 con 0x7f8c1815f2d0 2024-04-26T21:47:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.235+0000 7f8c1e322640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.236+0000 7f8c1e322640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8c14007fd0 tx=0x7f8c14007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:22.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.240+0000 7f8c0effd640 1 -- 172.21.15.40:0/1763454928 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c0804b020 con 0x7f8c1815f2d0 2024-04-26T21:47:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:22.461+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8c180b48b0 con 0x7f8c1815f2d0 2024-04-26T21:47:23.072 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:47:23.072+0000 7f4089b7d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:47:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.124+0000 7f8c0effd640 1 -- 172.21.15.40:0/1763454928 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v357) ==== 122+0+0 (secure 0 0 0) 0x7f8c08032e20 con 0x7f8c1815f2d0 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bfc06c8f0 msgr2=0x7f8bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8c14007fd0 tx=0x7f8c14007e80 comp rx=0 tx=0).stop 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c1815f2d0 msgr2=0x7f8c1814e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c1815f2d0 0x7f8c1814e9e0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097d0 tx=0x7f8c080046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 shutdown_connections 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8bfc06c8f0 0x7f8bfc06edb0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c181604e0 0x7f8c181560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c1815f2d0 0x7f8c1814e9e0 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 --2- 172.21.15.40:0/1763454928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1806fcd0 0x7f8c180c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 >> 172.21.15.40:0/1763454928 conn(0x7f8c180d0be0 msgr2=0x7f8c1815b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 shutdown_connections 2024-04-26T21:47:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.127+0000 7f8c20586640 1 -- 172.21.15.40:0/1763454928 wait complete. 2024-04-26T21:47:23.140 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:47:23.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.248+0000 7f4220833640 1 Processor -- start 2024-04-26T21:47:23.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.248+0000 7f4220833640 1 -- start start 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f4220833640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f421c0cf5e0 0x7f421c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f4220833640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421c15b9e0 con 0x7f421c15b600 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f4220833640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421c0d0740 con 0x7f421c0cbbd0 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f4220833640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f421c0cb9b0 con 0x7f421c0cf5e0 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f4219d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f421c0cf5e0 0x7f421c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f421a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f421c15b600 0x7f421c071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46264/0 (socket says 172.21.15.40:46264) 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f421a59c640 1 -- 172.21.15.40:0/3879788871 learned_addr learned my addr 172.21.15.40:0/3879788871 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f4219d9b640 1 -- 172.21.15.40:0/3879788871 >> v1:172.21.15.184:6789/0 conn(0x7f421c0cbbd0 legacy=0x7f421c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f4219d9b640 1 -- 172.21.15.40:0/3879788871 >> v1:172.21.15.40:6789/0 conn(0x7f421c15b600 legacy=0x7f421c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f4219d9b640 1 -- 172.21.15.40:0/3879788871 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c0cb360 con 0x7f421c0cf5e0 2024-04-26T21:47:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.249+0000 7f421959a640 1 -- 172.21.15.40:0/3879788871 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793545419 0 0) 0x7f421c15b9e0 con 0x7f421c15b600 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.250+0000 7f4219d9b640 1 --2- 172.21.15.40:0/3879788871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0ce9d0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f421c0712b0 tx=0x7f421002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82b959d0a2b607c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.250+0000 7f421959a640 1 -- 172.21.15.40:0/3879788871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210037070 con 0x7f421c0cf5e0 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.250+0000 7f421959a640 1 -- 172.21.15.40:0/3879788871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421002ae20 con 0x7f421c0cf5e0 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.250+0000 7f421959a640 1 -- 172.21.15.40:0/3879788871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210032660 con 0x7f421c0cf5e0 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.250+0000 7f4220833640 1 -- 172.21.15.40:0/3879788871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 msgr2=0x7f421c0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.250+0000 7f4220833640 1 --2- 172.21.15.40:0/3879788871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0ce9d0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f421c0712b0 tx=0x7f421002a110 comp rx=0 tx=0).stop 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.251+0000 7f4220833640 1 -- 172.21.15.40:0/3879788871 shutdown_connections 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.251+0000 7f4220833640 1 --2- 172.21.15.40:0/3879788871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0ce9d0 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.251+0000 7f4220833640 1 -- 172.21.15.40:0/3879788871 >> 172.21.15.40:0/3879788871 conn(0x7f421c0d0be0 msgr2=0x7f421c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:23.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.251+0000 7f4220833640 1 -- 172.21.15.40:0/3879788871 shutdown_connections 2024-04-26T21:47:23.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.251+0000 7f4220833640 1 -- 172.21.15.40:0/3879788871 wait complete. 2024-04-26T21:47:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.252+0000 7f4220833640 1 Processor -- start 2024-04-26T21:47:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.252+0000 7f4220833640 1 -- start start 2024-04-26T21:47:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4220833640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f421c0cbbd0 0x7f421c15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4220833640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0c3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4220833640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f421c15b600 0x7f421c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4220833640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0d0470 con 0x7f421c0cf5e0 2024-04-26T21:47:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4220833640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0d02f0 con 0x7f421c0cbbd0 2024-04-26T21:47:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4220833640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f421c0cef10 con 0x7f421c15b600 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0c3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57940/0 (socket says 172.21.15.40:57940) 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 -- 172.21.15.40:0/2192308469 learned_addr learned my addr 172.21.15.40:0/2192308469 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f421ad9d640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f421c15b600 0x7f421c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f421a59c640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f421c0cbbd0 0x7f421c15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 -- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f421c15b600 msgr2=0x7f421c0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f421c15b600 0x7f421c0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 -- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f421c0cbbd0 msgr2=0x7f421c15b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f421c0cbbd0 0x7f421c15b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 -- 172.21.15.40:0/2192308469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c15bf20 con 0x7f421c0cf5e0 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f421ad9d640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f421c15b600 0x7f421c0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f421a59c640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f421c0cbbd0 0x7f421c15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.253+0000 7f4219d9b640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0c3730 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f4210035040 tx=0x7f42100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.254+0000 7f420affd640 1 -- 172.21.15.40:0/2192308469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210037040 con 0x7f421c0cf5e0 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.254+0000 7f420affd640 1 -- 172.21.15.40:0/2192308469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4210004030 con 0x7f421c0cf5e0 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.254+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c1487f0 con 0x7f421c0cf5e0 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.254+0000 7f420affd640 1 -- 172.21.15.40:0/2192308469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210006e10 con 0x7f421c0cf5e0 2024-04-26T21:47:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.254+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c148db0 con 0x7f421c0cf5e0 2024-04-26T21:47:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.255+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421c14be70 con 0x7f421c0cf5e0 2024-04-26T21:47:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.255+0000 7f420affd640 1 -- 172.21.15.40:0/2192308469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f421003a040 con 0x7f421c0cf5e0 2024-04-26T21:47:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.256+0000 7f420affd640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41fc06c8f0 0x7f41fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.256+0000 7f420affd640 1 -- 172.21.15.40:0/2192308469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f42100ac1a0 con 0x7f421c0cf5e0 2024-04-26T21:47:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.256+0000 7f421a59c640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41fc06c8f0 0x7f41fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.257+0000 7f421a59c640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41fc06c8f0 0x7f41fc06edb0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f420c000f60 tx=0x7f420c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.260+0000 7f420affd640 1 -- 172.21.15.40:0/2192308469 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f421c14be70 con 0x7f421c0cf5e0 2024-04-26T21:47:23.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.482+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f421c0694f0 con 0x7f421c0cf5e0 2024-04-26T21:47:23.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.483+0000 7f420affd640 1 -- 172.21.15.40:0/2192308469 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 357 v357) ==== 95+0+5201 (secure 0 0 0) 0x7f421002ae20 con 0x7f421c0cf5e0 2024-04-26T21:47:23.482 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:23.483 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":357,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":357,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:22.462625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11757":{"gid":11757,"name":"a","rank":0,"incarnation":355,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3441996159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3441996159},{"type":"v1","addr":"172.21.15.40:6835","nonce":3441996159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11757,"qdb_cluster":[11757]},"id":7}]} 2024-04-26T21:47:23.483 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 357 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41fc06c8f0 msgr2=0x7f41fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41fc06c8f0 0x7f41fc06edb0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f420c000f60 tx=0x7f420c009040 comp rx=0 tx=0).stop 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 msgr2=0x7f421c0c3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0c3730 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f4210035040 tx=0x7f42100046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 shutdown_connections 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f41fc06c8f0 0x7f41fc06edb0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f421c15b600 0x7f421c0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c0cf5e0 0x7f421c0c3730 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 --2- 172.21.15.40:0/2192308469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f421c0cbbd0 0x7f421c15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.484+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 >> 172.21.15.40:0/2192308469 conn(0x7f421c0d0be0 msgr2=0x7f421c14ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.485+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 shutdown_connections 2024-04-26T21:47:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.485+0000 7f4220833640 1 -- 172.21.15.40:0/2192308469 wait complete. 2024-04-26T21:47:23.495 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:23.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.596+0000 7f61c82da640 1 Processor -- start 2024-04-26T21:47:23.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.596+0000 7f61c82da640 1 -- start start 2024-04-26T21:47:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.597+0000 7f61c82da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f61c00b2340 0x7f61c00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.597+0000 7f61c82da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c00d10d0 con 0x7f61c0162810 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.597+0000 7f61c82da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c00d1370 con 0x7f61c00cd930 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.597+0000 7f61c82da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f61c00d0750 con 0x7f61c00b2340 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.597+0000 7f61c5875640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f61c00b2340 0x7f61c00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.597+0000 7f61c6076640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f61c0162810 0x7f61c0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46266/0 (socket says 172.21.15.40:46266) 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.598+0000 7f61c6076640 1 -- 172.21.15.40:0/239108235 learned_addr learned my addr 172.21.15.40:0/239108235 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.598+0000 7f61c5875640 1 -- 172.21.15.40:0/239108235 >> v1:172.21.15.184:6789/0 conn(0x7f61c00cd930 legacy=0x7f61c00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.598+0000 7f61c5875640 1 -- 172.21.15.40:0/239108235 >> v1:172.21.15.40:6789/0 conn(0x7f61c0162810 legacy=0x7f61c0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.598+0000 7f61c5074640 1 -- 172.21.15.40:0/239108235 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4103832131 0 0) 0x7f61c00d10d0 con 0x7f61c0162810 2024-04-26T21:47:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.598+0000 7f61c5875640 1 -- 172.21.15.40:0/239108235 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c00d01b0 con 0x7f61c00b2340 2024-04-26T21:47:23.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c5875640 1 --2- 172.21.15.40:0/239108235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00b2340 0x7f61c00cd3f0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f61c00d1540 tx=0x7f61b002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681a025f66bd4122 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c5074640 1 -- 172.21.15.40:0/239108235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0037070 con 0x7f61c00b2340 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c5074640 1 -- 172.21.15.40:0/239108235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b0032440 con 0x7f61c00b2340 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c5074640 1 -- 172.21.15.40:0/239108235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0032810 con 0x7f61c00b2340 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c82da640 1 -- 172.21.15.40:0/239108235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00b2340 msgr2=0x7f61c00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c82da640 1 --2- 172.21.15.40:0/239108235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00b2340 0x7f61c00cd3f0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f61c00d1540 tx=0x7f61b002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c82da640 1 -- 172.21.15.40:0/239108235 shutdown_connections 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c82da640 1 --2- 172.21.15.40:0/239108235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00b2340 0x7f61c00cd3f0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c82da640 1 -- 172.21.15.40:0/239108235 >> 172.21.15.40:0/239108235 conn(0x7f61c00d2da0 msgr2=0x7f61c00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c82da640 1 -- 172.21.15.40:0/239108235 shutdown_connections 2024-04-26T21:47:23.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.599+0000 7f61c82da640 1 -- 172.21.15.40:0/239108235 wait complete. 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c82da640 1 Processor -- start 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c82da640 1 -- start start 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c82da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c00b2340 0x7f61c015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c82da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00cd930 0x7f61c0160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c82da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c0162810 0x7f61c00d3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c82da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61c00d2240 con 0x7f61c00cd930 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c82da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61c00d16a0 con 0x7f61c00b2340 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c82da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61c00d23c0 con 0x7f61c0162810 2024-04-26T21:47:23.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c5875640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00cd930 0x7f61c0160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c5875640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00cd930 0x7f61c0160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57950/0 (socket says 172.21.15.40:57950) 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.601+0000 7f61c5875640 1 -- 172.21.15.40:0/935121932 learned_addr learned my addr 172.21.15.40:0/935121932 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c6076640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c00b2340 0x7f61c015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c6877640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c0162810 0x7f61c00d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c5875640 1 -- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c0162810 msgr2=0x7f61c00d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c5875640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c0162810 0x7f61c00d3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c5875640 1 -- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c00b2340 msgr2=0x7f61c015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c5875640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c00b2340 0x7f61c015c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c5875640 1 -- 172.21.15.40:0/935121932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01611d0 con 0x7f61c00cd930 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c6076640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c00b2340 0x7f61c015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c6877640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c0162810 0x7f61c00d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c5875640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00cd930 0x7f61c0160c90 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f61b0035040 tx=0x7f61b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61b6ffd640 1 -- 172.21.15.40:0/935121932 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0037040 con 0x7f61c00cd930 2024-04-26T21:47:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61b6ffd640 1 -- 172.21.15.40:0/935121932 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b0004230 con 0x7f61c00cd930 2024-04-26T21:47:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c01f8430 con 0x7f61c00cd930 2024-04-26T21:47:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61b6ffd640 1 -- 172.21.15.40:0/935121932 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0032e20 con 0x7f61c00cd930 2024-04-26T21:47:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.602+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01f89f0 con 0x7f61c00cd930 2024-04-26T21:47:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.603+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c01f90b0 con 0x7f61c00cd930 2024-04-26T21:47:23.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.604+0000 7f61b6ffd640 1 -- 172.21.15.40:0/935121932 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61b0030770 con 0x7f61c00cd930 2024-04-26T21:47:23.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.604+0000 7f61b6ffd640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61a406c8f0 0x7f61a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.604+0000 7f61b6ffd640 1 -- 172.21.15.40:0/935121932 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f61b00acef0 con 0x7f61c00cd930 2024-04-26T21:47:23.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.604+0000 7f61c6076640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61a406c8f0 0x7f61a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.605+0000 7f61c6076640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61a406c8f0 0x7f61a406edb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f61bc000f60 tx=0x7f61bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.609+0000 7f61b6ffd640 1 -- 172.21.15.40:0/935121932 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61c01f90b0 con 0x7f61c00cd930 2024-04-26T21:47:23.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.796+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61c00694f0 con 0x7f61c00cd930 2024-04-26T21:47:23.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.797+0000 7f61b6ffd640 1 -- 172.21.15.40:0/935121932 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16848 (secure 0 0 0) 0x7f61b007b220 con 0x7f61c00cd930 2024-04-26T21:47:23.797 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:23.798 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":145,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:22.108045+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:18.726844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:19.403532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.799+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61a406c8f0 msgr2=0x7f61a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.799+0000 7f61c82da640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61a406c8f0 0x7f61a406edb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f61bc000f60 tx=0x7f61bc009040 comp rx=0 tx=0).stop 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.799+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00cd930 msgr2=0x7f61c0160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.799+0000 7f61c82da640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00cd930 0x7f61c0160c90 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f61b0035040 tx=0x7f61b00046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.800+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 shutdown_connections 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.800+0000 7f61c82da640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f61a406c8f0 0x7f61a406edb0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.800+0000 7f61c82da640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c0162810 0x7f61c00d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.800+0000 7f61c82da640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c00cd930 0x7f61c0160c90 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.800+0000 7f61c82da640 1 --2- 172.21.15.40:0/935121932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c00b2340 0x7f61c015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.800+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 >> 172.21.15.40:0/935121932 conn(0x7f61c00d2da0 msgr2=0x7f61c0161ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.800+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 shutdown_connections 2024-04-26T21:47:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.800+0000 7f61c82da640 1 -- 172.21.15.40:0/935121932 wait complete. 2024-04-26T21:47:23.811 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:47:23.811 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:23.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.912+0000 7f9e8f046640 1 Processor -- start 2024-04-26T21:47:23.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.912+0000 7f9e8f046640 1 -- start start 2024-04-26T21:47:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e8f046640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e88162880 0x7f9e880d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e8f046640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9e880d3bc0 0x7f9e880b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e8f046640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e880b7e50 con 0x7f9e880b5f20 2024-04-26T21:47:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e8f046640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9e880d0580 con 0x7f9e880d3bc0 2024-04-26T21:47:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e8f046640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e880d1800 con 0x7f9e88162880 2024-04-26T21:47:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e87fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9e880b5f20 0x7f9e880d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46278/0 (socket says 172.21.15.40:46278) 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e87fff640 1 -- 172.21.15.40:0/3918587906 learned_addr learned my addr 172.21.15.40:0/3918587906 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e8d5e3640 1 --2- 172.21.15.40:0/3918587906 >> v2:172.21.15.184:3300/0 conn(0x7f9e880d3bc0 0x7f9e880b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 257268735 0 0) 0x7f9e880b7e50 con 0x7f9e880b5f20 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e70003520 con 0x7f9e880b5f20 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e8cde2640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e88162880 0x7f9e880d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 337972204 0 0) 0x7f9e70003520 con 0x7f9e880b5f20 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e88162880 msgr2=0x7f9e880d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e88162880 0x7f9e880d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 >> v2:172.21.15.184:3300/0 conn(0x7f9e880d3bc0 msgr2=0x7f9e880b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 --2- 172.21.15.40:0/3918587906 >> v2:172.21.15.184:3300/0 conn(0x7f9e880d3bc0 0x7f9e880b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e880d07d0 con 0x7f9e880b5f20 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f9e7c001670 con 0x7f9e880b5f20 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 >> v1:172.21.15.40:6789/0 conn(0x7f9e880b5f20 legacy=0x7f9e880d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e88162880 0x7f9e700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e70003f30 0x7f9e70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70004d40 0x7f9e700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e880d0580 con 0x7f9e70003f30 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e880d1800 con 0x7f9e70004d40 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.913+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7000b9c0 con 0x7f9e88162880 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e87fff640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e70003f30 0x7f9e70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e8cde2640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e88162880 0x7f9e700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e8d5e3640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70004d40 0x7f9e700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e87fff640 1 -- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e88162880 msgr2=0x7f9e700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e87fff640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e88162880 0x7f9e700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e87fff640 1 -- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70004d40 msgr2=0x7f9e700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e87fff640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70004d40 0x7f9e700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e87fff640 1 -- 172.21.15.40:0/3918587906 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e70003520 con 0x7f9e70003f30 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.914+0000 7f9e87fff640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e70003f30 0x7f9e70004800 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c210 tx=0x7f9e70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75c8d72bd263a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c007d50 con 0x7f9e70003f30 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c02e070 con 0x7f9e70003f30 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e877fe640 1 -- 172.21.15.40:0/3918587906 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c005d70 con 0x7f9e70003f30 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e8f046640 1 -- 172.21.15.40:0/3918587906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e70003f30 msgr2=0x7f9e70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e8f046640 1 --2- 172.21.15.40:0/3918587906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e70003f30 0x7f9e70004800 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c210 tx=0x7f9e70005830 comp rx=0 tx=0).stop 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e8d5e3640 1 -- 172.21.15.40:0/3918587906 reap_dead start 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e8f046640 1 -- 172.21.15.40:0/3918587906 shutdown_connections 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e8f046640 1 -- 172.21.15.40:0/3918587906 >> 172.21.15.40:0/3918587906 conn(0x7f9e880d4f60 msgr2=0x7f9e880d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e8f046640 1 -- 172.21.15.40:0/3918587906 shutdown_connections 2024-04-26T21:47:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.915+0000 7f9e8f046640 1 -- 172.21.15.40:0/3918587906 wait complete. 2024-04-26T21:47:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.916+0000 7f9e8f046640 1 Processor -- start 2024-04-26T21:47:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8f046640 1 -- start start 2024-04-26T21:47:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8f046640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70003f30 0x7f9e880b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8f046640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e8814f630 0x7f9e880b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8f046640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e880b1b50 0x7f9e880b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8f046640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88162c60 con 0x7f9e880b1b50 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8f046640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e880d0d90 con 0x7f9e70003f30 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8f046640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e88162de0 con 0x7f9e8814f630 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8d5e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e880b1b50 0x7f9e880b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8d5e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e880b1b50 0x7f9e880b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57974/0 (socket says 172.21.15.40:57974) 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.917+0000 7f9e8d5e3640 1 -- 172.21.15.40:0/4189809389 learned_addr learned my addr 172.21.15.40:0/4189809389 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8cde2640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70003f30 0x7f9e880b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e87fff640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e8814f630 0x7f9e880b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8d5e3640 1 -- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e8814f630 msgr2=0x7f9e880b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8d5e3640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e8814f630 0x7f9e880b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8d5e3640 1 -- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70003f30 msgr2=0x7f9e880b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8d5e3640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70003f30 0x7f9e880b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8d5e3640 1 -- 172.21.15.40:0/4189809389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8814fab0 con 0x7f9e880b1b50 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8cde2640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70003f30 0x7f9e880b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e87fff640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e8814f630 0x7f9e880b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8d5e3640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e880b1b50 0x7f9e880b1fd0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f9e80011860 tx=0x7f9e80011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e857fa640 1 -- 172.21.15.40:0/4189809389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80011e50 con 0x7f9e880b1b50 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e857fa640 1 -- 172.21.15.40:0/4189809389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e8001b070 con 0x7f9e880b1b50 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e857fa640 1 -- 172.21.15.40:0/4189809389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80004460 con 0x7f9e880b1b50 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8814cde0 con 0x7f9e880b1b50 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.918+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8814d3a0 con 0x7f9e880b1b50 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.919+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e880694f0 con 0x7f9e880b1b50 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.920+0000 7f9e857fa640 1 -- 172.21.15.40:0/4189809389 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e80012040 con 0x7f9e880b1b50 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.920+0000 7f9e857fa640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e7406c920 0x7f9e7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.920+0000 7f9e857fa640 1 -- 172.21.15.40:0/4189809389 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f9e8008bb20 con 0x7f9e880b1b50 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.920+0000 7f9e8cde2640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e7406c920 0x7f9e7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.921+0000 7f9e8cde2640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e7406c920 0x7f9e7406ede0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9e78012290 tx=0x7f9e7800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:23.924+0000 7f9e857fa640 1 -- 172.21.15.40:0/4189809389 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e80059d40 con 0x7f9e880b1b50 2024-04-26T21:47:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.107+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e8814cb90 con 0x7f9e7406c920 2024-04-26T21:47:24.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.114+0000 7f9e857fa640 1 -- 172.21.15.40:0/4189809389 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237241 (secure 0 0 0) 0x7f9e8814cb90 con 0x7f9e7406c920 2024-04-26T21:47:24.116 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:24.117 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:47:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.117+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e7406c920 msgr2=0x7f9e7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.117+0000 7f9e8f046640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e7406c920 0x7f9e7406ede0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9e78012290 tx=0x7f9e7800b040 comp rx=0 tx=0).stop 2024-04-26T21:47:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.117+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e880b1b50 msgr2=0x7f9e880b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.117+0000 7f9e8f046640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e880b1b50 0x7f9e880b1fd0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f9e80011860 tx=0x7f9e80011d20 comp rx=0 tx=0).stop 2024-04-26T21:47:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.118+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 shutdown_connections 2024-04-26T21:47:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.118+0000 7f9e8f046640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9e7406c920 0x7f9e7406ede0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.118+0000 7f9e8f046640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e880b1b50 0x7f9e880b1fd0 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.118+0000 7f9e8f046640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e8814f630 0x7f9e880b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.118+0000 7f9e8f046640 1 --2- 172.21.15.40:0/4189809389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e70003f30 0x7f9e880b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.118+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 >> 172.21.15.40:0/4189809389 conn(0x7f9e880d4f60 msgr2=0x7f9e88160a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.118+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 shutdown_connections 2024-04-26T21:47:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.118+0000 7f9e8f046640 1 -- 172.21.15.40:0/4189809389 wait complete. 2024-04-26T21:47:24.129 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-04-26T21:47:22.180027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":293988,"kb_used_data":15396,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680732,"statfs":{"total":773094113280,"available":772793069568,"internally_reserved":0,"allocated":15765504,"data_stored":8556933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000862"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:20.073038+0000","last_change":"2024-04-26T21:47:20.073038+0000","last_active":"2024-04-26T21:47:20.073038+0000","last_peered":"2024-04-26T21:47:20.073038+0000","last_clean":"2024-04-26T21:47:20.073038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:20.073038+0000","last_undegraded":"2024-04-26T21:47:20.073038+0000","last_fullsized":"2024-04-26T21:47:20.073038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:47:19.068476+0000","last_change":"2024-04-26T21:47:19.068476+0000","last_active":"2024-04-26T21:47:19.068476+0000","last_peered":"2024-04-26T21:47:19.068476+0000","last_clean":"2024-04-26T21:47:19.068476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:47:19.068476+0000","last_undegraded":"2024-04-26T21:47:19.068476+0000","last_fullsized":"2024-04-26T21:47:19.068476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":404,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-26T21:47:17.087252+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:47:17.087252+0000","last_peered":"2024-04-26T21:47:17.087252+0000","last_clean":"2024-04-26T21:47:17.087252+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:47:17.087252+0000","last_undegraded":"2024-04-26T21:47:17.087252+0000","last_fullsized":"2024-04-26T21:47:17.087252+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37704,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334136,"statfs":{"total":96636764160,"available":96598155264,"internally_reserved":0,"allocated":2420736,"data_stored":1514786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41288,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330552,"statfs":{"total":96636764160,"available":96594485248,"internally_reserved":0,"allocated":1826816,"data_stored":924418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32976,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1826816,"data_stored":924418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":905271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32968,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1826816,"data_stored":924418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37132,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334708,"statfs":{"total":96636764160,"available":96598740992,"internally_reserved":0,"allocated":1826816,"data_stored":924418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37128,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1826816,"data_stored":924418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41864,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329976,"statfs":{"total":96636764160,"available":96593895424,"internally_reserved":0,"allocated":2420736,"data_stored":1514786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:24.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:24.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.270+0000 7f13e781d640 1 Processor -- start 2024-04-26T21:47:24.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.270+0000 7f13e781d640 1 -- start start 2024-04-26T21:47:24.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e781d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13e0071e60 0x7f13e015df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e781d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13e0072240 0x7f13e00726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e781d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e015e4d0 con 0x7f13e00cb5a0 2024-04-26T21:47:24.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e781d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f13e00701f0 con 0x7f13e0072240 2024-04-26T21:47:24.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e781d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f13e006e240 con 0x7f13e0071e60 2024-04-26T21:47:24.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e4db8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13e0072240 0x7f13e00726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e5dba640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f13e00cb5a0 0x7f13e00cba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46286/0 (socket says 172.21.15.40:46286) 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e5dba640 1 -- 172.21.15.40:0/3320119960 learned_addr learned my addr 172.21.15.40:0/3320119960 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e55b9640 1 --2- 172.21.15.40:0/3320119960 >> v2:172.21.15.184:3300/0 conn(0x7f13e0071e60 0x7f13e015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e4db8640 1 -- 172.21.15.40:0/3320119960 >> v2:172.21.15.184:3300/0 conn(0x7f13e0071e60 msgr2=0x7f13e015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e4db8640 1 --2- 172.21.15.40:0/3320119960 >> v2:172.21.15.184:3300/0 conn(0x7f13e0071e60 0x7f13e015df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e4db8640 1 -- 172.21.15.40:0/3320119960 >> v1:172.21.15.40:6789/0 conn(0x7f13e00cb5a0 legacy=0x7f13e00cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e4db8640 1 -- 172.21.15.40:0/3320119960 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e006fc30 con 0x7f13e0072240 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13d7fff640 1 -- 172.21.15.40:0/3320119960 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432911113 0 0) 0x7f13d8001490 con 0x7f13e00cb5a0 2024-04-26T21:47:24.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.271+0000 7f13e55b9640 1 --2- 172.21.15.40:0/3320119960 >> v2:172.21.15.184:3300/0 conn(0x7f13e0071e60 0x7f13e015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:24.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.272+0000 7f13e4db8640 1 --2- 172.21.15.40:0/3320119960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e00726a0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f13e00d0a80 tx=0x7f13d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a872b32a62d943c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.272+0000 7f13d7fff640 1 -- 172.21.15.40:0/3320119960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0037070 con 0x7f13e0072240 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.272+0000 7f13d7fff640 1 -- 172.21.15.40:0/3320119960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d0032440 con 0x7f13e0072240 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.272+0000 7f13d7fff640 1 -- 172.21.15.40:0/3320119960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d002ae60 con 0x7f13e0072240 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.272+0000 7f13e781d640 1 -- 172.21.15.40:0/3320119960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 msgr2=0x7f13e00726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.272+0000 7f13e781d640 1 --2- 172.21.15.40:0/3320119960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e00726a0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f13e00d0a80 tx=0x7f13d002a110 comp rx=0 tx=0).stop 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.273+0000 7f13e781d640 1 -- 172.21.15.40:0/3320119960 shutdown_connections 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.273+0000 7f13e781d640 1 --2- 172.21.15.40:0/3320119960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e00726a0 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.273+0000 7f13e781d640 1 --2- 172.21.15.40:0/3320119960 >> v2:172.21.15.184:3300/0 conn(0x7f13e0071e60 0x7f13e015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.273+0000 7f13e781d640 1 -- 172.21.15.40:0/3320119960 >> 172.21.15.40:0/3320119960 conn(0x7f13e00d0be0 msgr2=0x7f13e00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.273+0000 7f13e781d640 1 -- 172.21.15.40:0/3320119960 shutdown_connections 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.273+0000 7f13e781d640 1 -- 172.21.15.40:0/3320119960 wait complete. 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.274+0000 7f13e781d640 1 Processor -- start 2024-04-26T21:47:24.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.274+0000 7f13e781d640 1 -- start start 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.274+0000 7f13e781d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e0071e60 0x7f13e00c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.274+0000 7f13e781d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e01eed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e781d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e00cb5a0 0x7f13e00aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e781d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13e0072be0 con 0x7f13e0072240 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e781d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13e0073fb0 con 0x7f13e0071e60 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e781d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13e006f8e0 con 0x7f13e00cb5a0 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e01eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e01eed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57990/0 (socket says 172.21.15.40:57990) 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 -- 172.21.15.40:0/2270144175 learned_addr learned my addr 172.21.15.40:0/2270144175 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e55b9640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e0071e60 0x7f13e00c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e5dba640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e00cb5a0 0x7f13e00aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 -- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e00cb5a0 msgr2=0x7f13e00aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e00cb5a0 0x7f13e00aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 -- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e0071e60 msgr2=0x7f13e00c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e0071e60 0x7f13e00c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 -- 172.21.15.40:0/2270144175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e00c7050 con 0x7f13e0072240 2024-04-26T21:47:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e5dba640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e00cb5a0 0x7f13e00aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e4db8640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e01eed20 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f13d0035040 tx=0x7f13d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13d5ffb640 1 -- 172.21.15.40:0/2270144175 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0037040 con 0x7f13e0072240 2024-04-26T21:47:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13d5ffb640 1 -- 172.21.15.40:0/2270144175 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d0004230 con 0x7f13e0072240 2024-04-26T21:47:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e00c6030 con 0x7f13e0072240 2024-04-26T21:47:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13d5ffb640 1 -- 172.21.15.40:0/2270144175 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0032e20 con 0x7f13e0072240 2024-04-26T21:47:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.275+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e00c6870 con 0x7f13e0072240 2024-04-26T21:47:24.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.277+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e00694f0 con 0x7f13e0072240 2024-04-26T21:47:24.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.277+0000 7f13d5ffb640 1 -- 172.21.15.40:0/2270144175 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13d0030770 con 0x7f13e0072240 2024-04-26T21:47:24.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.277+0000 7f13d5ffb640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13c406c8f0 0x7f13c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.277+0000 7f13d5ffb640 1 -- 172.21.15.40:0/2270144175 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f13d00acfd0 con 0x7f13e0072240 2024-04-26T21:47:24.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.277+0000 7f13e55b9640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13c406c8f0 0x7f13c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.278+0000 7f13e55b9640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13c406c8f0 0x7f13c406edb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00a950 tx=0x7f13dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:24.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.282+0000 7f13d5ffb640 1 -- 172.21.15.40:0/2270144175 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13e00c6030 con 0x7f13e0072240 2024-04-26T21:47:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.464+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13e00cc610 con 0x7f13c406c8f0 2024-04-26T21:47:24.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.471+0000 7f13d5ffb640 1 -- 172.21.15.40:0/2270144175 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252458 (secure 0 0 0) 0x7f13e00cc610 con 0x7f13c406c8f0 2024-04-26T21:47:24.473 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:47:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13c406c8f0 msgr2=0x7f13c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13c406c8f0 0x7f13c406edb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00a950 tx=0x7f13dc008040 comp rx=0 tx=0).stop 2024-04-26T21:47:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 msgr2=0x7f13e01eed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e01eed20 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f13d0035040 tx=0x7f13d00046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 shutdown_connections 2024-04-26T21:47:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13c406c8f0 0x7f13c406edb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e00cb5a0 0x7f13e00aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e0072240 0x7f13e01eed20 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 --2- 172.21.15.40:0/2270144175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e0071e60 0x7f13e00c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 >> 172.21.15.40:0/2270144175 conn(0x7f13e00d0be0 msgr2=0x7f13e00cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 shutdown_connections 2024-04-26T21:47:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.474+0000 7f13e781d640 1 -- 172.21.15.40:0/2270144175 wait complete. 2024-04-26T21:47:24.484 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-04-26T21:47:24.180417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294396,"kb_used_data":15820,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680324,"statfs":{"total":773094113280,"available":772792651776,"internally_reserved":0,"allocated":16199680,"data_stored":8761320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.105746"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.132878+0000","last_change":"2024-04-26T21:47:21.128390+0000","last_active":"2024-04-26T21:47:21.132878+0000","last_peered":"2024-04-26T21:47:21.132878+0000","last_clean":"2024-04-26T21:47:21.132878+0000","last_became_active":"2024-04-26T21:47:21.128115+0000","last_became_peered":"2024-04-26T21:47:21.128115+0000","last_unstale":"2024-04-26T21:47:21.132878+0000","last_undegraded":"2024-04-26T21:47:21.132878+0000","last_fullsized":"2024-04-26T21:47:21.132878+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:26.785711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.126005+0000","last_change":"2024-04-26T21:47:21.107469+0000","last_active":"2024-04-26T21:47:22.126005+0000","last_peered":"2024-04-26T21:47:22.126005+0000","last_clean":"2024-04-26T21:47:22.126005+0000","last_became_active":"2024-04-26T21:47:21.107127+0000","last_became_peered":"2024-04-26T21:47:21.107127+0000","last_unstale":"2024-04-26T21:47:22.126005+0000","last_undegraded":"2024-04-26T21:47:22.126005+0000","last_fullsized":"2024-04-26T21:47:22.126005+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:24.576117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.127180+0000","last_change":"2024-04-26T21:47:21.119273+0000","last_active":"2024-04-26T21:47:21.127180+0000","last_peered":"2024-04-26T21:47:21.127180+0000","last_clean":"2024-04-26T21:47:21.127180+0000","last_became_active":"2024-04-26T21:47:21.118992+0000","last_became_peered":"2024-04-26T21:47:21.118992+0000","last_unstale":"2024-04-26T21:47:21.127180+0000","last_undegraded":"2024-04-26T21:47:21.127180+0000","last_fullsized":"2024-04-26T21:47:21.127180+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:44.638304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529622+0000","last_change":"2024-04-26T21:47:21.114359+0000","last_active":"2024-04-26T21:47:22.529622+0000","last_peered":"2024-04-26T21:47:22.529622+0000","last_clean":"2024-04-26T21:47:22.529622+0000","last_became_active":"2024-04-26T21:47:21.113957+0000","last_became_peered":"2024-04-26T21:47:21.113957+0000","last_unstale":"2024-04-26T21:47:22.529622+0000","last_undegraded":"2024-04-26T21:47:22.529622+0000","last_fullsized":"2024-04-26T21:47:22.529622+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:40.104208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867023+0000","last_change":"2024-04-26T21:47:21.122400+0000","last_active":"2024-04-26T21:47:22.867023+0000","last_peered":"2024-04-26T21:47:22.867023+0000","last_clean":"2024-04-26T21:47:22.867023+0000","last_became_active":"2024-04-26T21:47:21.122111+0000","last_became_peered":"2024-04-26T21:47:21.122111+0000","last_unstale":"2024-04-26T21:47:22.867023+0000","last_undegraded":"2024-04-26T21:47:22.867023+0000","last_fullsized":"2024-04-26T21:47:22.867023+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:24.661961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125531+0000","last_change":"2024-04-26T21:47:21.113125+0000","last_active":"2024-04-26T21:47:22.125531+0000","last_peered":"2024-04-26T21:47:22.125531+0000","last_clean":"2024-04-26T21:47:22.125531+0000","last_became_active":"2024-04-26T21:47:21.112853+0000","last_became_peered":"2024-04-26T21:47:21.112853+0000","last_unstale":"2024-04-26T21:47:22.125531+0000","last_undegraded":"2024-04-26T21:47:22.125531+0000","last_fullsized":"2024-04-26T21:47:22.125531+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:28.681934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125416+0000","last_change":"2024-04-26T21:47:21.114485+0000","last_active":"2024-04-26T21:47:22.125416+0000","last_peered":"2024-04-26T21:47:22.125416+0000","last_clean":"2024-04-26T21:47:22.125416+0000","last_became_active":"2024-04-26T21:47:21.113944+0000","last_became_peered":"2024-04-26T21:47:21.113944+0000","last_unstale":"2024-04-26T21:47:22.125416+0000","last_undegraded":"2024-04-26T21:47:22.125416+0000","last_fullsized":"2024-04-26T21:47:22.125416+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:57.012160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121521+0000","last_change":"2024-04-26T21:47:21.127016+0000","last_active":"2024-04-26T21:47:22.121521+0000","last_peered":"2024-04-26T21:47:22.121521+0000","last_clean":"2024-04-26T21:47:22.121521+0000","last_became_active":"2024-04-26T21:47:21.126741+0000","last_became_peered":"2024-04-26T21:47:21.126741+0000","last_unstale":"2024-04-26T21:47:22.121521+0000","last_undegraded":"2024-04-26T21:47:22.121521+0000","last_fullsized":"2024-04-26T21:47:22.121521+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:17.971335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.126152+0000","last_change":"2024-04-26T21:47:21.117121+0000","last_active":"2024-04-26T21:47:22.126152+0000","last_peered":"2024-04-26T21:47:22.126152+0000","last_clean":"2024-04-26T21:47:22.126152+0000","last_became_active":"2024-04-26T21:47:21.116412+0000","last_became_peered":"2024-04-26T21:47:21.116412+0000","last_unstale":"2024-04-26T21:47:22.126152+0000","last_undegraded":"2024-04-26T21:47:22.126152+0000","last_fullsized":"2024-04-26T21:47:22.126152+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:40.104208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867202+0000","last_change":"2024-04-26T21:47:21.109054+0000","last_active":"2024-04-26T21:47:22.867202+0000","last_peered":"2024-04-26T21:47:22.867202+0000","last_clean":"2024-04-26T21:47:22.867202+0000","last_became_active":"2024-04-26T21:47:21.108443+0000","last_became_peered":"2024-04-26T21:47:21.108443+0000","last_unstale":"2024-04-26T21:47:22.867202+0000","last_undegraded":"2024-04-26T21:47:22.867202+0000","last_fullsized":"2024-04-26T21:47:22.867202+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:59.345508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.124420+0000","last_change":"2024-04-26T21:47:21.103311+0000","last_active":"2024-04-26T21:47:22.124420+0000","last_peered":"2024-04-26T21:47:22.124420+0000","last_clean":"2024-04-26T21:47:22.124420+0000","last_became_active":"2024-04-26T21:47:21.101685+0000","last_became_peered":"2024-04-26T21:47:21.101685+0000","last_unstale":"2024-04-26T21:47:22.124420+0000","last_undegraded":"2024-04-26T21:47:22.124420+0000","last_fullsized":"2024-04-26T21:47:22.124420+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:59.816737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.123374+0000","last_change":"2024-04-26T21:47:21.117750+0000","last_active":"2024-04-26T21:47:21.123374+0000","last_peered":"2024-04-26T21:47:21.123374+0000","last_clean":"2024-04-26T21:47:21.123374+0000","last_became_active":"2024-04-26T21:47:21.117465+0000","last_became_peered":"2024-04-26T21:47:21.117465+0000","last_unstale":"2024-04-26T21:47:21.123374+0000","last_undegraded":"2024-04-26T21:47:21.123374+0000","last_fullsized":"2024-04-26T21:47:21.123374+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:16.703781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529464+0000","last_change":"2024-04-26T21:47:21.118511+0000","last_active":"2024-04-26T21:47:22.529464+0000","last_peered":"2024-04-26T21:47:22.529464+0000","last_clean":"2024-04-26T21:47:22.529464+0000","last_became_active":"2024-04-26T21:47:21.118177+0000","last_became_peered":"2024-04-26T21:47:21.118177+0000","last_unstale":"2024-04-26T21:47:22.529464+0000","last_undegraded":"2024-04-26T21:47:22.529464+0000","last_fullsized":"2024-04-26T21:47:22.529464+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.398169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.117211+0000","last_change":"2024-04-26T21:47:21.115454+0000","last_active":"2024-04-26T21:47:21.117211+0000","last_peered":"2024-04-26T21:47:21.117211+0000","last_clean":"2024-04-26T21:47:21.117211+0000","last_became_active":"2024-04-26T21:47:21.115158+0000","last_became_peered":"2024-04-26T21:47:21.115158+0000","last_unstale":"2024-04-26T21:47:21.117211+0000","last_undegraded":"2024-04-26T21:47:21.117211+0000","last_fullsized":"2024-04-26T21:47:21.117211+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:00.037010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.111999+0000","last_change":"2024-04-26T21:47:21.109917+0000","last_active":"2024-04-26T21:47:21.111999+0000","last_peered":"2024-04-26T21:47:21.111999+0000","last_clean":"2024-04-26T21:47:21.111999+0000","last_became_active":"2024-04-26T21:47:21.109643+0000","last_became_peered":"2024-04-26T21:47:21.109643+0000","last_unstale":"2024-04-26T21:47:21.111999+0000","last_undegraded":"2024-04-26T21:47:21.111999+0000","last_fullsized":"2024-04-26T21:47:21.111999+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:59.816737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.120325+0000","last_change":"2024-04-26T21:47:21.119655+0000","last_active":"2024-04-26T21:47:21.120325+0000","last_peered":"2024-04-26T21:47:21.120325+0000","last_clean":"2024-04-26T21:47:21.120325+0000","last_became_active":"2024-04-26T21:47:21.119381+0000","last_became_peered":"2024-04-26T21:47:21.119381+0000","last_unstale":"2024-04-26T21:47:21.120325+0000","last_undegraded":"2024-04-26T21:47:21.120325+0000","last_fullsized":"2024-04-26T21:47:21.120325+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:18.502012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.129273+0000","last_change":"2024-04-26T21:47:21.120324+0000","last_active":"2024-04-26T21:47:21.129273+0000","last_peered":"2024-04-26T21:47:21.129273+0000","last_clean":"2024-04-26T21:47:21.129273+0000","last_became_active":"2024-04-26T21:47:21.119793+0000","last_became_peered":"2024-04-26T21:47:21.119793+0000","last_unstale":"2024-04-26T21:47:21.129273+0000","last_undegraded":"2024-04-26T21:47:21.129273+0000","last_fullsized":"2024-04-26T21:47:21.129273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:06.754561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.126022+0000","last_change":"2024-04-26T21:47:21.114655+0000","last_active":"2024-04-26T21:47:22.126022+0000","last_peered":"2024-04-26T21:47:22.126022+0000","last_clean":"2024-04-26T21:47:22.126022+0000","last_became_active":"2024-04-26T21:47:21.114006+0000","last_became_peered":"2024-04-26T21:47:21.114006+0000","last_unstale":"2024-04-26T21:47:22.126022+0000","last_undegraded":"2024-04-26T21:47:22.126022+0000","last_fullsized":"2024-04-26T21:47:22.126022+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.398169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.127144+0000","last_change":"2024-04-26T21:47:21.117512+0000","last_active":"2024-04-26T21:47:21.127144+0000","last_peered":"2024-04-26T21:47:21.127144+0000","last_clean":"2024-04-26T21:47:21.127144+0000","last_became_active":"2024-04-26T21:47:21.117216+0000","last_became_peered":"2024-04-26T21:47:21.117216+0000","last_unstale":"2024-04-26T21:47:21.127144+0000","last_undegraded":"2024-04-26T21:47:21.127144+0000","last_fullsized":"2024-04-26T21:47:21.127144+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:08.030611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.111758+0000","last_change":"2024-04-26T21:47:21.107187+0000","last_active":"2024-04-26T21:47:21.111758+0000","last_peered":"2024-04-26T21:47:21.111758+0000","last_clean":"2024-04-26T21:47:21.111758+0000","last_became_active":"2024-04-26T21:47:21.106895+0000","last_became_peered":"2024-04-26T21:47:21.106895+0000","last_unstale":"2024-04-26T21:47:21.111758+0000","last_undegraded":"2024-04-26T21:47:21.111758+0000","last_fullsized":"2024-04-26T21:47:21.111758+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:02.023906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121778+0000","last_change":"2024-04-26T21:47:21.106081+0000","last_active":"2024-04-26T21:47:22.121778+0000","last_peered":"2024-04-26T21:47:22.121778+0000","last_clean":"2024-04-26T21:47:22.121778+0000","last_became_active":"2024-04-26T21:47:21.105726+0000","last_became_peered":"2024-04-26T21:47:21.105726+0000","last_unstale":"2024-04-26T21:47:22.121778+0000","last_undegraded":"2024-04-26T21:47:22.121778+0000","last_fullsized":"2024-04-26T21:47:22.121778+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:01.380091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867108+0000","last_change":"2024-04-26T21:47:21.117734+0000","last_active":"2024-04-26T21:47:22.867108+0000","last_peered":"2024-04-26T21:47:22.867108+0000","last_clean":"2024-04-26T21:47:22.867108+0000","last_became_active":"2024-04-26T21:47:21.117469+0000","last_became_peered":"2024-04-26T21:47:21.117469+0000","last_unstale":"2024-04-26T21:47:22.867108+0000","last_undegraded":"2024-04-26T21:47:22.867108+0000","last_fullsized":"2024-04-26T21:47:22.867108+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:57.012160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.118574+0000","last_change":"2024-04-26T21:47:21.112455+0000","last_active":"2024-04-26T21:47:21.118574+0000","last_peered":"2024-04-26T21:47:21.118574+0000","last_clean":"2024-04-26T21:47:21.118574+0000","last_became_active":"2024-04-26T21:47:21.112040+0000","last_became_peered":"2024-04-26T21:47:21.112040+0000","last_unstale":"2024-04-26T21:47:21.118574+0000","last_undegraded":"2024-04-26T21:47:21.118574+0000","last_fullsized":"2024-04-26T21:47:21.118574+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:37.836633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.866818+0000","last_change":"2024-04-26T21:47:21.116830+0000","last_active":"2024-04-26T21:47:22.866818+0000","last_peered":"2024-04-26T21:47:22.866818+0000","last_clean":"2024-04-26T21:47:22.866818+0000","last_became_active":"2024-04-26T21:47:21.112422+0000","last_became_peered":"2024-04-26T21:47:21.112422+0000","last_unstale":"2024-04-26T21:47:22.866818+0000","last_undegraded":"2024-04-26T21:47:22.866818+0000","last_fullsized":"2024-04-26T21:47:22.866818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:27.727874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.117473+0000","last_change":"2024-04-26T21:47:21.113728+0000","last_active":"2024-04-26T21:47:21.117473+0000","last_peered":"2024-04-26T21:47:21.117473+0000","last_clean":"2024-04-26T21:47:21.117473+0000","last_became_active":"2024-04-26T21:47:21.112614+0000","last_became_peered":"2024-04-26T21:47:21.112614+0000","last_unstale":"2024-04-26T21:47:21.117473+0000","last_undegraded":"2024-04-26T21:47:21.117473+0000","last_fullsized":"2024-04-26T21:47:21.117473+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:10.553206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.123261+0000","last_change":"2024-04-26T21:47:21.118350+0000","last_active":"2024-04-26T21:47:21.123261+0000","last_peered":"2024-04-26T21:47:21.123261+0000","last_clean":"2024-04-26T21:47:21.123261+0000","last_became_active":"2024-04-26T21:47:21.117991+0000","last_became_peered":"2024-04-26T21:47:21.117991+0000","last_unstale":"2024-04-26T21:47:21.123261+0000","last_undegraded":"2024-04-26T21:47:21.123261+0000","last_fullsized":"2024-04-26T21:47:21.123261+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:39.133778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.124625+0000","last_change":"2024-04-26T21:47:21.106271+0000","last_active":"2024-04-26T21:47:22.124625+0000","last_peered":"2024-04-26T21:47:22.124625+0000","last_clean":"2024-04-26T21:47:22.124625+0000","last_became_active":"2024-04-26T21:47:21.105966+0000","last_became_peered":"2024-04-26T21:47:21.105966+0000","last_unstale":"2024-04-26T21:47:22.124625+0000","last_undegraded":"2024-04-26T21:47:22.124625+0000","last_fullsized":"2024-04-26T21:47:22.124625+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:00.037010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125733+0000","last_change":"2024-04-26T21:47:21.102370+0000","last_active":"2024-04-26T21:47:22.125733+0000","last_peered":"2024-04-26T21:47:22.125733+0000","last_clean":"2024-04-26T21:47:22.125733+0000","last_became_active":"2024-04-26T21:47:21.102094+0000","last_became_peered":"2024-04-26T21:47:21.102094+0000","last_unstale":"2024-04-26T21:47:22.125733+0000","last_undegraded":"2024-04-26T21:47:22.125733+0000","last_fullsized":"2024-04-26T21:47:22.125733+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:57:56.680731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.122000+0000","last_change":"2024-04-26T21:47:21.113992+0000","last_active":"2024-04-26T21:47:22.122000+0000","last_peered":"2024-04-26T21:47:22.122000+0000","last_clean":"2024-04-26T21:47:22.122000+0000","last_became_active":"2024-04-26T21:47:21.113674+0000","last_became_peered":"2024-04-26T21:47:21.113674+0000","last_unstale":"2024-04-26T21:47:22.122000+0000","last_undegraded":"2024-04-26T21:47:22.122000+0000","last_fullsized":"2024-04-26T21:47:22.122000+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:38.758333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.123293+0000","last_change":"2024-04-26T21:47:21.118302+0000","last_active":"2024-04-26T21:47:21.123293+0000","last_peered":"2024-04-26T21:47:21.123293+0000","last_clean":"2024-04-26T21:47:21.123293+0000","last_became_active":"2024-04-26T21:47:21.117865+0000","last_became_peered":"2024-04-26T21:47:21.117865+0000","last_unstale":"2024-04-26T21:47:21.123293+0000","last_undegraded":"2024-04-26T21:47:21.123293+0000","last_fullsized":"2024-04-26T21:47:21.123293+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:22.004387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125125+0000","last_change":"2024-04-26T21:47:21.110299+0000","last_active":"2024-04-26T21:47:22.125125+0000","last_peered":"2024-04-26T21:47:22.125125+0000","last_clean":"2024-04-26T21:47:22.125125+0000","last_became_active":"2024-04-26T21:47:21.109925+0000","last_became_peered":"2024-04-26T21:47:21.109925+0000","last_unstale":"2024-04-26T21:47:22.125125+0000","last_undegraded":"2024-04-26T21:47:22.125125+0000","last_fullsized":"2024-04-26T21:47:22.125125+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:24.661961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121598+0000","last_change":"2024-04-26T21:47:21.110311+0000","last_active":"2024-04-26T21:47:22.121598+0000","last_peered":"2024-04-26T21:47:22.121598+0000","last_clean":"2024-04-26T21:47:22.121598+0000","last_became_active":"2024-04-26T21:47:21.110053+0000","last_became_peered":"2024-04-26T21:47:21.110053+0000","last_unstale":"2024-04-26T21:47:22.121598+0000","last_undegraded":"2024-04-26T21:47:22.121598+0000","last_fullsized":"2024-04-26T21:47:22.121598+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:19.741608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.127075+0000","last_change":"2024-04-26T21:47:21.117597+0000","last_active":"2024-04-26T21:47:21.127075+0000","last_peered":"2024-04-26T21:47:21.127075+0000","last_clean":"2024-04-26T21:47:21.127075+0000","last_became_active":"2024-04-26T21:47:21.116669+0000","last_became_peered":"2024-04-26T21:47:21.116669+0000","last_unstale":"2024-04-26T21:47:21.127075+0000","last_undegraded":"2024-04-26T21:47:21.127075+0000","last_fullsized":"2024-04-26T21:47:21.127075+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:17.109103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"144'4","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529663+0000","last_change":"2024-04-26T21:47:20.091943+0000","last_active":"2024-04-26T21:47:22.529663+0000","last_peered":"2024-04-26T21:47:22.529663+0000","last_clean":"2024-04-26T21:47:22.529663+0000","last_became_active":"2024-04-26T21:47:20.091661+0000","last_became_peered":"2024-04-26T21:47:20.091661+0000","last_unstale":"2024-04-26T21:47:22.529663+0000","last_undegraded":"2024-04-26T21:47:22.529663+0000","last_fullsized":"2024-04-26T21:47:22.529663+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:43.006503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121587+0000","last_change":"2024-04-26T21:47:21.126001+0000","last_active":"2024-04-26T21:47:22.121587+0000","last_peered":"2024-04-26T21:47:22.121587+0000","last_clean":"2024-04-26T21:47:22.121587+0000","last_became_active":"2024-04-26T21:47:21.125673+0000","last_became_peered":"2024-04-26T21:47:21.125673+0000","last_unstale":"2024-04-26T21:47:22.121587+0000","last_undegraded":"2024-04-26T21:47:22.121587+0000","last_fullsized":"2024-04-26T21:47:22.121587+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:24.093030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121805+0000","last_change":"2024-04-26T21:47:20.095236+0000","last_active":"2024-04-26T21:47:22.121805+0000","last_peered":"2024-04-26T21:47:22.121805+0000","last_clean":"2024-04-26T21:47:22.121805+0000","last_became_active":"2024-04-26T21:47:20.094925+0000","last_became_peered":"2024-04-26T21:47:20.094925+0000","last_unstale":"2024-04-26T21:47:22.121805+0000","last_undegraded":"2024-04-26T21:47:22.121805+0000","last_fullsized":"2024-04-26T21:47:22.121805+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:07.457337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.123467+0000","last_change":"2024-04-26T21:47:21.120951+0000","last_active":"2024-04-26T21:47:21.123467+0000","last_peered":"2024-04-26T21:47:21.123467+0000","last_clean":"2024-04-26T21:47:21.123467+0000","last_became_active":"2024-04-26T21:47:21.120676+0000","last_became_peered":"2024-04-26T21:47:21.120676+0000","last_unstale":"2024-04-26T21:47:21.123467+0000","last_undegraded":"2024-04-26T21:47:21.123467+0000","last_fullsized":"2024-04-26T21:47:21.123467+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:32.923549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"144'2","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.123584+0000","last_change":"2024-04-26T21:47:20.093772+0000","last_active":"2024-04-26T21:47:21.123584+0000","last_peered":"2024-04-26T21:47:21.123584+0000","last_clean":"2024-04-26T21:47:21.123584+0000","last_became_active":"2024-04-26T21:47:20.093501+0000","last_became_peered":"2024-04-26T21:47:20.093501+0000","last_unstale":"2024-04-26T21:47:21.123584+0000","last_undegraded":"2024-04-26T21:47:21.123584+0000","last_fullsized":"2024-04-26T21:47:21.123584+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:23.058529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.124830+0000","last_change":"2024-04-26T21:47:21.108797+0000","last_active":"2024-04-26T21:47:22.124830+0000","last_peered":"2024-04-26T21:47:22.124830+0000","last_clean":"2024-04-26T21:47:22.124830+0000","last_became_active":"2024-04-26T21:47:21.106759+0000","last_became_peered":"2024-04-26T21:47:21.106759+0000","last_unstale":"2024-04-26T21:47:22.124830+0000","last_undegraded":"2024-04-26T21:47:22.124830+0000","last_fullsized":"2024-04-26T21:47:22.124830+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:05.769086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867109+0000","last_change":"2024-04-26T21:47:20.091359+0000","last_active":"2024-04-26T21:47:22.867109+0000","last_peered":"2024-04-26T21:47:22.867109+0000","last_clean":"2024-04-26T21:47:22.867109+0000","last_became_active":"2024-04-26T21:47:20.091091+0000","last_became_peered":"2024-04-26T21:47:20.091091+0000","last_unstale":"2024-04-26T21:47:22.867109+0000","last_undegraded":"2024-04-26T21:47:22.867109+0000","last_fullsized":"2024-04-26T21:47:22.867109+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.999826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529594+0000","last_change":"2024-04-26T21:47:21.126577+0000","last_active":"2024-04-26T21:47:22.529594+0000","last_peered":"2024-04-26T21:47:22.529594+0000","last_clean":"2024-04-26T21:47:22.529594+0000","last_became_active":"2024-04-26T21:47:21.126269+0000","last_became_peered":"2024-04-26T21:47:21.126269+0000","last_unstale":"2024-04-26T21:47:22.529594+0000","last_undegraded":"2024-04-26T21:47:22.529594+0000","last_fullsized":"2024-04-26T21:47:22.529594+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:59.816737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.123513+0000","last_change":"2024-04-26T21:47:20.091358+0000","last_active":"2024-04-26T21:47:21.123513+0000","last_peered":"2024-04-26T21:47:21.123513+0000","last_clean":"2024-04-26T21:47:21.123513+0000","last_became_active":"2024-04-26T21:47:20.091047+0000","last_became_peered":"2024-04-26T21:47:20.091047+0000","last_unstale":"2024-04-26T21:47:21.123513+0000","last_undegraded":"2024-04-26T21:47:21.123513+0000","last_fullsized":"2024-04-26T21:47:21.123513+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:31.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.127162+0000","last_change":"2024-04-26T21:47:21.118077+0000","last_active":"2024-04-26T21:47:21.127162+0000","last_peered":"2024-04-26T21:47:21.127162+0000","last_clean":"2024-04-26T21:47:21.127162+0000","last_became_active":"2024-04-26T21:47:21.117756+0000","last_became_peered":"2024-04-26T21:47:21.117756+0000","last_unstale":"2024-04-26T21:47:21.127162+0000","last_undegraded":"2024-04-26T21:47:21.127162+0000","last_fullsized":"2024-04-26T21:47:21.127162+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:02.251482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.124449+0000","last_change":"2024-04-26T21:47:20.090589+0000","last_active":"2024-04-26T21:47:22.124449+0000","last_peered":"2024-04-26T21:47:22.124449+0000","last_clean":"2024-04-26T21:47:22.124449+0000","last_became_active":"2024-04-26T21:47:20.090333+0000","last_became_peered":"2024-04-26T21:47:20.090333+0000","last_unstale":"2024-04-26T21:47:22.124449+0000","last_undegraded":"2024-04-26T21:47:22.124449+0000","last_fullsized":"2024-04-26T21:47:22.124449+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:00.442649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.126126+0000","last_change":"2024-04-26T21:47:21.102225+0000","last_active":"2024-04-26T21:47:22.126126+0000","last_peered":"2024-04-26T21:47:22.126126+0000","last_clean":"2024-04-26T21:47:22.126126+0000","last_became_active":"2024-04-26T21:47:21.101895+0000","last_became_peered":"2024-04-26T21:47:21.101895+0000","last_unstale":"2024-04-26T21:47:22.126126+0000","last_undegraded":"2024-04-26T21:47:22.126126+0000","last_fullsized":"2024-04-26T21:47:22.126126+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:59.816737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.105868+0000","last_change":"2024-04-26T21:47:20.095287+0000","last_active":"2024-04-26T21:47:21.105868+0000","last_peered":"2024-04-26T21:47:21.105868+0000","last_clean":"2024-04-26T21:47:21.105868+0000","last_became_active":"2024-04-26T21:47:20.095013+0000","last_became_peered":"2024-04-26T21:47:20.095013+0000","last_unstale":"2024-04-26T21:47:21.105868+0000","last_undegraded":"2024-04-26T21:47:21.105868+0000","last_fullsized":"2024-04-26T21:47:21.105868+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:46.707336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529695+0000","last_change":"2024-04-26T21:47:21.113789+0000","last_active":"2024-04-26T21:47:22.529695+0000","last_peered":"2024-04-26T21:47:22.529695+0000","last_clean":"2024-04-26T21:47:22.529695+0000","last_became_active":"2024-04-26T21:47:21.113477+0000","last_became_peered":"2024-04-26T21:47:21.113477+0000","last_unstale":"2024-04-26T21:47:22.529695+0000","last_undegraded":"2024-04-26T21:47:22.529695+0000","last_fullsized":"2024-04-26T21:47:22.529695+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:18.502012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529381+0000","last_change":"2024-04-26T21:47:20.088439+0000","last_active":"2024-04-26T21:47:22.529381+0000","last_peered":"2024-04-26T21:47:22.529381+0000","last_clean":"2024-04-26T21:47:22.529381+0000","last_became_active":"2024-04-26T21:47:20.088133+0000","last_became_peered":"2024-04-26T21:47:20.088133+0000","last_unstale":"2024-04-26T21:47:22.529381+0000","last_undegraded":"2024-04-26T21:47:22.529381+0000","last_fullsized":"2024-04-26T21:47:22.529381+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:03.539304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.127056+0000","last_change":"2024-04-26T21:47:21.117328+0000","last_active":"2024-04-26T21:47:21.127056+0000","last_peered":"2024-04-26T21:47:21.127056+0000","last_clean":"2024-04-26T21:47:21.127056+0000","last_became_active":"2024-04-26T21:47:21.117063+0000","last_became_peered":"2024-04-26T21:47:21.117063+0000","last_unstale":"2024-04-26T21:47:21.127056+0000","last_undegraded":"2024-04-26T21:47:21.127056+0000","last_fullsized":"2024-04-26T21:47:21.127056+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:53.642606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.106881+0000","last_change":"2024-04-26T21:47:20.094018+0000","last_active":"2024-04-26T21:47:21.106881+0000","last_peered":"2024-04-26T21:47:21.106881+0000","last_clean":"2024-04-26T21:47:21.106881+0000","last_became_active":"2024-04-26T21:47:20.093696+0000","last_became_peered":"2024-04-26T21:47:20.093696+0000","last_unstale":"2024-04-26T21:47:21.106881+0000","last_undegraded":"2024-04-26T21:47:21.106881+0000","last_fullsized":"2024-04-26T21:47:21.106881+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:24.800552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.866948+0000","last_change":"2024-04-26T21:47:21.128614+0000","last_active":"2024-04-26T21:47:22.866948+0000","last_peered":"2024-04-26T21:47:22.866948+0000","last_clean":"2024-04-26T21:47:22.866948+0000","last_became_active":"2024-04-26T21:47:21.128293+0000","last_became_peered":"2024-04-26T21:47:21.128293+0000","last_unstale":"2024-04-26T21:47:22.866948+0000","last_undegraded":"2024-04-26T21:47:22.866948+0000","last_fullsized":"2024-04-26T21:47:22.866948+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:01.447211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.103983+0000","last_change":"2024-04-26T21:47:20.091046+0000","last_active":"2024-04-26T21:47:21.103983+0000","last_peered":"2024-04-26T21:47:21.103983+0000","last_clean":"2024-04-26T21:47:21.103983+0000","last_became_active":"2024-04-26T21:47:20.090030+0000","last_became_peered":"2024-04-26T21:47:20.090030+0000","last_unstale":"2024-04-26T21:47:21.103983+0000","last_undegraded":"2024-04-26T21:47:21.103983+0000","last_fullsized":"2024-04-26T21:47:21.103983+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:48.049120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121888+0000","last_change":"2024-04-26T21:47:21.124294+0000","last_active":"2024-04-26T21:47:22.121888+0000","last_peered":"2024-04-26T21:47:22.121888+0000","last_clean":"2024-04-26T21:47:22.121888+0000","last_became_active":"2024-04-26T21:47:21.124002+0000","last_became_peered":"2024-04-26T21:47:21.124002+0000","last_unstale":"2024-04-26T21:47:22.121888+0000","last_undegraded":"2024-04-26T21:47:22.121888+0000","last_fullsized":"2024-04-26T21:47:22.121888+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:47.711898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"144'3","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.130696+0000","last_change":"2024-04-26T21:47:20.094743+0000","last_active":"2024-04-26T21:47:21.130696+0000","last_peered":"2024-04-26T21:47:21.130696+0000","last_clean":"2024-04-26T21:47:21.130696+0000","last_became_active":"2024-04-26T21:47:20.094473+0000","last_became_peered":"2024-04-26T21:47:20.094473+0000","last_unstale":"2024-04-26T21:47:21.130696+0000","last_undegraded":"2024-04-26T21:47:21.130696+0000","last_fullsized":"2024-04-26T21:47:21.130696+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:17.505121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867235+0000","last_change":"2024-04-26T21:47:21.129372+0000","last_active":"2024-04-26T21:47:22.867235+0000","last_peered":"2024-04-26T21:47:22.867235+0000","last_clean":"2024-04-26T21:47:22.867235+0000","last_became_active":"2024-04-26T21:47:21.129070+0000","last_became_peered":"2024-04-26T21:47:21.129070+0000","last_unstale":"2024-04-26T21:47:22.867235+0000","last_undegraded":"2024-04-26T21:47:22.867235+0000","last_fullsized":"2024-04-26T21:47:22.867235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:28.681934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"144'6","reported_seq":20,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125720+0000","last_change":"2024-04-26T21:47:20.095744+0000","last_active":"2024-04-26T21:47:22.125720+0000","last_peered":"2024-04-26T21:47:22.125720+0000","last_clean":"2024-04-26T21:47:22.125720+0000","last_became_active":"2024-04-26T21:47:20.094096+0000","last_became_peered":"2024-04-26T21:47:20.094096+0000","last_unstale":"2024-04-26T21:47:22.125720+0000","last_undegraded":"2024-04-26T21:47:22.125720+0000","last_fullsized":"2024-04-26T21:47:22.125720+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:31.918987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.124619+0000","last_change":"2024-04-26T21:47:21.112626+0000","last_active":"2024-04-26T21:47:22.124619+0000","last_peered":"2024-04-26T21:47:22.124619+0000","last_clean":"2024-04-26T21:47:22.124619+0000","last_became_active":"2024-04-26T21:47:21.111291+0000","last_became_peered":"2024-04-26T21:47:21.111291+0000","last_unstale":"2024-04-26T21:47:22.124619+0000","last_undegraded":"2024-04-26T21:47:22.124619+0000","last_fullsized":"2024-04-26T21:47:22.124619+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:40.104208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867312+0000","last_change":"2024-04-26T21:47:20.097926+0000","last_active":"2024-04-26T21:47:22.867312+0000","last_peered":"2024-04-26T21:47:22.867312+0000","last_clean":"2024-04-26T21:47:22.867312+0000","last_became_active":"2024-04-26T21:47:20.097642+0000","last_became_peered":"2024-04-26T21:47:20.097642+0000","last_unstale":"2024-04-26T21:47:22.867312+0000","last_undegraded":"2024-04-26T21:47:22.867312+0000","last_fullsized":"2024-04-26T21:47:22.867312+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:09.388667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125952+0000","last_change":"2024-04-26T21:47:21.118797+0000","last_active":"2024-04-26T21:47:22.125952+0000","last_peered":"2024-04-26T21:47:22.125952+0000","last_clean":"2024-04-26T21:47:22.125952+0000","last_became_active":"2024-04-26T21:47:21.117096+0000","last_became_peered":"2024-04-26T21:47:21.117096+0000","last_unstale":"2024-04-26T21:47:22.125952+0000","last_undegraded":"2024-04-26T21:47:22.125952+0000","last_fullsized":"2024-04-26T21:47:22.125952+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:18.509683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529528+0000","last_change":"2024-04-26T21:47:20.087955+0000","last_active":"2024-04-26T21:47:22.529528+0000","last_peered":"2024-04-26T21:47:22.529528+0000","last_clean":"2024-04-26T21:47:22.529528+0000","last_became_active":"2024-04-26T21:47:20.087651+0000","last_became_peered":"2024-04-26T21:47:20.087651+0000","last_unstale":"2024-04-26T21:47:22.529528+0000","last_undegraded":"2024-04-26T21:47:22.529528+0000","last_fullsized":"2024-04-26T21:47:22.529528+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:26.723312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121839+0000","last_change":"2024-04-26T21:47:21.111006+0000","last_active":"2024-04-26T21:47:22.121839+0000","last_peered":"2024-04-26T21:47:22.121839+0000","last_clean":"2024-04-26T21:47:22.121839+0000","last_became_active":"2024-04-26T21:47:21.110761+0000","last_became_peered":"2024-04-26T21:47:21.110761+0000","last_unstale":"2024-04-26T21:47:22.121839+0000","last_undegraded":"2024-04-26T21:47:22.121839+0000","last_fullsized":"2024-04-26T21:47:22.121839+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:35.601128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121615+0000","last_change":"2024-04-26T21:47:20.095856+0000","last_active":"2024-04-26T21:47:22.121615+0000","last_peered":"2024-04-26T21:47:22.121615+0000","last_clean":"2024-04-26T21:47:22.121615+0000","last_became_active":"2024-04-26T21:47:20.095545+0000","last_became_peered":"2024-04-26T21:47:20.095545+0000","last_unstale":"2024-04-26T21:47:22.121615+0000","last_undegraded":"2024-04-26T21:47:22.121615+0000","last_fullsized":"2024-04-26T21:47:22.121615+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:42.334049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.129250+0000","last_change":"2024-04-26T21:47:21.120104+0000","last_active":"2024-04-26T21:47:21.129250+0000","last_peered":"2024-04-26T21:47:21.129250+0000","last_clean":"2024-04-26T21:47:21.129250+0000","last_became_active":"2024-04-26T21:47:21.119452+0000","last_became_peered":"2024-04-26T21:47:21.119452+0000","last_unstale":"2024-04-26T21:47:21.129250+0000","last_undegraded":"2024-04-26T21:47:21.129250+0000","last_fullsized":"2024-04-26T21:47:21.129250+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:15.545729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"144'8","reported_seq":20,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.123669+0000","last_change":"2024-04-26T21:47:20.093766+0000","last_active":"2024-04-26T21:47:21.123669+0000","last_peered":"2024-04-26T21:47:21.123669+0000","last_clean":"2024-04-26T21:47:21.123669+0000","last_became_active":"2024-04-26T21:47:20.093488+0000","last_became_peered":"2024-04-26T21:47:20.093488+0000","last_unstale":"2024-04-26T21:47:21.123669+0000","last_undegraded":"2024-04-26T21:47:21.123669+0000","last_fullsized":"2024-04-26T21:47:21.123669+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:37.753771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121775+0000","last_change":"2024-04-26T21:47:21.111803+0000","last_active":"2024-04-26T21:47:22.121775+0000","last_peered":"2024-04-26T21:47:22.121775+0000","last_clean":"2024-04-26T21:47:22.121775+0000","last_became_active":"2024-04-26T21:47:21.111308+0000","last_became_peered":"2024-04-26T21:47:21.111308+0000","last_unstale":"2024-04-26T21:47:22.121775+0000","last_undegraded":"2024-04-26T21:47:22.121775+0000","last_fullsized":"2024-04-26T21:47:22.121775+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:55.790476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.126069+0000","last_change":"2024-04-26T21:47:20.090475+0000","last_active":"2024-04-26T21:47:22.126069+0000","last_peered":"2024-04-26T21:47:22.126069+0000","last_clean":"2024-04-26T21:47:22.126069+0000","last_became_active":"2024-04-26T21:47:20.089947+0000","last_became_peered":"2024-04-26T21:47:20.089947+0000","last_unstale":"2024-04-26T21:47:22.126069+0000","last_undegraded":"2024-04-26T21:47:22.126069+0000","last_fullsized":"2024-04-26T21:47:22.126069+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:38.129217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125480+0000","last_change":"2024-04-26T21:47:21.110363+0000","last_active":"2024-04-26T21:47:22.125480+0000","last_peered":"2024-04-26T21:47:22.125480+0000","last_clean":"2024-04-26T21:47:22.125480+0000","last_became_active":"2024-04-26T21:47:21.109940+0000","last_became_peered":"2024-04-26T21:47:21.109940+0000","last_unstale":"2024-04-26T21:47:22.125480+0000","last_undegraded":"2024-04-26T21:47:22.125480+0000","last_fullsized":"2024-04-26T21:47:22.125480+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:59.345508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.123792+0000","last_change":"2024-04-26T21:47:20.090606+0000","last_active":"2024-04-26T21:47:21.123792+0000","last_peered":"2024-04-26T21:47:21.123792+0000","last_clean":"2024-04-26T21:47:21.123792+0000","last_became_active":"2024-04-26T21:47:20.090296+0000","last_became_peered":"2024-04-26T21:47:20.090296+0000","last_unstale":"2024-04-26T21:47:21.123792+0000","last_undegraded":"2024-04-26T21:47:21.123792+0000","last_fullsized":"2024-04-26T21:47:21.123792+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:23.571555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867272+0000","last_change":"2024-04-26T21:47:21.116180+0000","last_active":"2024-04-26T21:47:22.867272+0000","last_peered":"2024-04-26T21:47:22.867272+0000","last_clean":"2024-04-26T21:47:22.867272+0000","last_became_active":"2024-04-26T21:47:21.113136+0000","last_became_peered":"2024-04-26T21:47:21.113136+0000","last_unstale":"2024-04-26T21:47:22.867272+0000","last_undegraded":"2024-04-26T21:47:22.867272+0000","last_fullsized":"2024-04-26T21:47:22.867272+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:57:56.680731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.124701+0000","last_change":"2024-04-26T21:47:20.092784+0000","last_active":"2024-04-26T21:47:22.124701+0000","last_peered":"2024-04-26T21:47:22.124701+0000","last_clean":"2024-04-26T21:47:22.124701+0000","last_became_active":"2024-04-26T21:47:20.092398+0000","last_became_peered":"2024-04-26T21:47:20.092398+0000","last_unstale":"2024-04-26T21:47:22.124701+0000","last_undegraded":"2024-04-26T21:47:22.124701+0000","last_fullsized":"2024-04-26T21:47:22.124701+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:23.657399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.132894+0000","last_change":"2024-04-26T21:47:21.129112+0000","last_active":"2024-04-26T21:47:21.132894+0000","last_peered":"2024-04-26T21:47:21.132894+0000","last_clean":"2024-04-26T21:47:21.132894+0000","last_became_active":"2024-04-26T21:47:21.128827+0000","last_became_peered":"2024-04-26T21:47:21.128827+0000","last_unstale":"2024-04-26T21:47:21.132894+0000","last_undegraded":"2024-04-26T21:47:21.132894+0000","last_fullsized":"2024-04-26T21:47:21.132894+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:36.927388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121963+0000","last_change":"2024-04-26T21:47:20.091194+0000","last_active":"2024-04-26T21:47:22.121963+0000","last_peered":"2024-04-26T21:47:22.121963+0000","last_clean":"2024-04-26T21:47:22.121963+0000","last_became_active":"2024-04-26T21:47:20.090875+0000","last_became_peered":"2024-04-26T21:47:20.090875+0000","last_unstale":"2024-04-26T21:47:22.121963+0000","last_undegraded":"2024-04-26T21:47:22.121963+0000","last_fullsized":"2024-04-26T21:47:22.121963+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:21.502786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.132837+0000","last_change":"2024-04-26T21:47:21.126999+0000","last_active":"2024-04-26T21:47:21.132837+0000","last_peered":"2024-04-26T21:47:21.132837+0000","last_clean":"2024-04-26T21:47:21.132837+0000","last_became_active":"2024-04-26T21:47:21.126005+0000","last_became_peered":"2024-04-26T21:47:21.126005+0000","last_unstale":"2024-04-26T21:47:21.132837+0000","last_undegraded":"2024-04-26T21:47:21.132837+0000","last_fullsized":"2024-04-26T21:47:21.132837+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:55.289428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121725+0000","last_change":"2024-04-26T21:47:20.096084+0000","last_active":"2024-04-26T21:47:22.121725+0000","last_peered":"2024-04-26T21:47:22.121725+0000","last_clean":"2024-04-26T21:47:22.121725+0000","last_became_active":"2024-04-26T21:47:20.095199+0000","last_became_peered":"2024-04-26T21:47:20.095199+0000","last_unstale":"2024-04-26T21:47:22.121725+0000","last_undegraded":"2024-04-26T21:47:22.121725+0000","last_fullsized":"2024-04-26T21:47:22.121725+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:05.558590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529684+0000","last_change":"2024-04-26T21:47:21.115067+0000","last_active":"2024-04-26T21:47:22.529684+0000","last_peered":"2024-04-26T21:47:22.529684+0000","last_clean":"2024-04-26T21:47:22.529684+0000","last_became_active":"2024-04-26T21:47:21.114796+0000","last_became_peered":"2024-04-26T21:47:21.114796+0000","last_unstale":"2024-04-26T21:47:22.529684+0000","last_undegraded":"2024-04-26T21:47:22.529684+0000","last_fullsized":"2024-04-26T21:47:22.529684+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:00.037010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125232+0000","last_change":"2024-04-26T21:47:20.092079+0000","last_active":"2024-04-26T21:47:22.125232+0000","last_peered":"2024-04-26T21:47:22.125232+0000","last_clean":"2024-04-26T21:47:22.125232+0000","last_became_active":"2024-04-26T21:47:20.091606+0000","last_became_peered":"2024-04-26T21:47:20.091606+0000","last_unstale":"2024-04-26T21:47:22.125232+0000","last_undegraded":"2024-04-26T21:47:22.125232+0000","last_fullsized":"2024-04-26T21:47:22.125232+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:35.694926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.122051+0000","last_change":"2024-04-26T21:47:21.123325+0000","last_active":"2024-04-26T21:47:22.122051+0000","last_peered":"2024-04-26T21:47:22.122051+0000","last_clean":"2024-04-26T21:47:22.122051+0000","last_became_active":"2024-04-26T21:47:21.123033+0000","last_became_peered":"2024-04-26T21:47:21.123033+0000","last_unstale":"2024-04-26T21:47:22.122051+0000","last_undegraded":"2024-04-26T21:47:22.122051+0000","last_fullsized":"2024-04-26T21:47:22.122051+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:24.063091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121709+0000","last_change":"2024-04-26T21:47:20.096795+0000","last_active":"2024-04-26T21:47:22.121709+0000","last_peered":"2024-04-26T21:47:22.121709+0000","last_clean":"2024-04-26T21:47:22.121709+0000","last_became_active":"2024-04-26T21:47:20.095602+0000","last_became_peered":"2024-04-26T21:47:20.095602+0000","last_unstale":"2024-04-26T21:47:22.121709+0000","last_undegraded":"2024-04-26T21:47:22.121709+0000","last_fullsized":"2024-04-26T21:47:22.121709+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:29.962850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.127094+0000","last_change":"2024-04-26T21:47:21.121734+0000","last_active":"2024-04-26T21:47:21.127094+0000","last_peered":"2024-04-26T21:47:21.127094+0000","last_clean":"2024-04-26T21:47:21.127094+0000","last_became_active":"2024-04-26T21:47:21.121429+0000","last_became_peered":"2024-04-26T21:47:21.121429+0000","last_unstale":"2024-04-26T21:47:21.127094+0000","last_undegraded":"2024-04-26T21:47:21.127094+0000","last_fullsized":"2024-04-26T21:47:21.127094+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:56.963416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867242+0000","last_change":"2024-04-26T21:47:20.095443+0000","last_active":"2024-04-26T21:47:22.867242+0000","last_peered":"2024-04-26T21:47:22.867242+0000","last_clean":"2024-04-26T21:47:22.867242+0000","last_became_active":"2024-04-26T21:47:20.095134+0000","last_became_peered":"2024-04-26T21:47:20.095134+0000","last_unstale":"2024-04-26T21:47:22.867242+0000","last_undegraded":"2024-04-26T21:47:22.867242+0000","last_fullsized":"2024-04-26T21:47:22.867242+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:15.699219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.119681+0000","last_change":"2024-04-26T21:47:21.116873+0000","last_active":"2024-04-26T21:47:21.119681+0000","last_peered":"2024-04-26T21:47:21.119681+0000","last_clean":"2024-04-26T21:47:21.119681+0000","last_became_active":"2024-04-26T21:47:21.116604+0000","last_became_peered":"2024-04-26T21:47:21.116604+0000","last_unstale":"2024-04-26T21:47:21.119681+0000","last_undegraded":"2024-04-26T21:47:21.119681+0000","last_fullsized":"2024-04-26T21:47:21.119681+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:36.699488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.104143+0000","last_change":"2024-04-26T21:47:20.091528+0000","last_active":"2024-04-26T21:47:21.104143+0000","last_peered":"2024-04-26T21:47:21.104143+0000","last_clean":"2024-04-26T21:47:21.104143+0000","last_became_active":"2024-04-26T21:47:20.090426+0000","last_became_peered":"2024-04-26T21:47:20.090426+0000","last_unstale":"2024-04-26T21:47:21.104143+0000","last_undegraded":"2024-04-26T21:47:21.104143+0000","last_fullsized":"2024-04-26T21:47:21.104143+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:30.086926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.529456+0000","last_change":"2024-04-26T21:47:21.114375+0000","last_active":"2024-04-26T21:47:22.529456+0000","last_peered":"2024-04-26T21:47:22.529456+0000","last_clean":"2024-04-26T21:47:22.529456+0000","last_became_active":"2024-04-26T21:47:21.114068+0000","last_became_peered":"2024-04-26T21:47:21.114068+0000","last_unstale":"2024-04-26T21:47:22.529456+0000","last_undegraded":"2024-04-26T21:47:22.529456+0000","last_fullsized":"2024-04-26T21:47:22.529456+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:24.576117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125906+0000","last_change":"2024-04-26T21:47:20.090746+0000","last_active":"2024-04-26T21:47:22.125906+0000","last_peered":"2024-04-26T21:47:22.125906+0000","last_clean":"2024-04-26T21:47:22.125906+0000","last_became_active":"2024-04-26T21:47:20.090470+0000","last_became_peered":"2024-04-26T21:47:20.090470+0000","last_unstale":"2024-04-26T21:47:22.125906+0000","last_undegraded":"2024-04-26T21:47:22.125906+0000","last_fullsized":"2024-04-26T21:47:22.125906+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:15.699219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.124661+0000","last_change":"2024-04-26T21:47:21.118915+0000","last_active":"2024-04-26T21:47:22.124661+0000","last_peered":"2024-04-26T21:47:22.124661+0000","last_clean":"2024-04-26T21:47:22.124661+0000","last_became_active":"2024-04-26T21:47:21.118623+0000","last_became_peered":"2024-04-26T21:47:21.118623+0000","last_unstale":"2024-04-26T21:47:22.124661+0000","last_undegraded":"2024-04-26T21:47:22.124661+0000","last_fullsized":"2024-04-26T21:47:22.124661+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:18.502012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.867243+0000","last_change":"2024-04-26T21:47:20.093229+0000","last_active":"2024-04-26T21:47:22.867243+0000","last_peered":"2024-04-26T21:47:22.867243+0000","last_clean":"2024-04-26T21:47:22.867243+0000","last_became_active":"2024-04-26T21:47:20.092924+0000","last_became_peered":"2024-04-26T21:47:20.092924+0000","last_unstale":"2024-04-26T21:47:22.867243+0000","last_undegraded":"2024-04-26T21:47:22.867243+0000","last_fullsized":"2024-04-26T21:47:22.867243+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:31.918987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121752+0000","last_change":"2024-04-26T21:47:21.120232+0000","last_active":"2024-04-26T21:47:22.121752+0000","last_peered":"2024-04-26T21:47:22.121752+0000","last_clean":"2024-04-26T21:47:22.121752+0000","last_became_active":"2024-04-26T21:47:21.119952+0000","last_became_peered":"2024-04-26T21:47:21.119952+0000","last_unstale":"2024-04-26T21:47:22.121752+0000","last_undegraded":"2024-04-26T21:47:22.121752+0000","last_fullsized":"2024-04-26T21:47:22.121752+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:39.237232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.105713+0000","last_change":"2024-04-26T21:47:20.086904+0000","last_active":"2024-04-26T21:47:21.105713+0000","last_peered":"2024-04-26T21:47:21.105713+0000","last_clean":"2024-04-26T21:47:21.105713+0000","last_became_active":"2024-04-26T21:47:20.086526+0000","last_became_peered":"2024-04-26T21:47:20.086526+0000","last_unstale":"2024-04-26T21:47:21.105713+0000","last_undegraded":"2024-04-26T21:47:21.105713+0000","last_fullsized":"2024-04-26T21:47:21.105713+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:34.596566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121760+0000","last_change":"2024-04-26T21:47:21.127580+0000","last_active":"2024-04-26T21:47:22.121760+0000","last_peered":"2024-04-26T21:47:22.121760+0000","last_clean":"2024-04-26T21:47:22.121760+0000","last_became_active":"2024-04-26T21:47:21.127292+0000","last_became_peered":"2024-04-26T21:47:21.127292+0000","last_unstale":"2024-04-26T21:47:22.121760+0000","last_undegraded":"2024-04-26T21:47:22.121760+0000","last_fullsized":"2024-04-26T21:47:22.121760+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:15.967241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121957+0000","last_change":"2024-04-26T21:47:20.096557+0000","last_active":"2024-04-26T21:47:22.121957+0000","last_peered":"2024-04-26T21:47:22.121957+0000","last_clean":"2024-04-26T21:47:22.121957+0000","last_became_active":"2024-04-26T21:47:20.096187+0000","last_became_peered":"2024-04-26T21:47:20.096187+0000","last_unstale":"2024-04-26T21:47:22.121957+0000","last_undegraded":"2024-04-26T21:47:22.121957+0000","last_fullsized":"2024-04-26T21:47:22.121957+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:33.748370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.116973+0000","last_change":"2024-04-26T21:47:21.112055+0000","last_active":"2024-04-26T21:47:21.116973+0000","last_peered":"2024-04-26T21:47:21.116973+0000","last_clean":"2024-04-26T21:47:21.116973+0000","last_became_active":"2024-04-26T21:47:21.111757+0000","last_became_peered":"2024-04-26T21:47:21.111757+0000","last_unstale":"2024-04-26T21:47:21.116973+0000","last_undegraded":"2024-04-26T21:47:21.116973+0000","last_fullsized":"2024-04-26T21:47:21.116973+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:40.104208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125317+0000","last_change":"2024-04-26T21:47:20.087321+0000","last_active":"2024-04-26T21:47:22.125317+0000","last_peered":"2024-04-26T21:47:22.125317+0000","last_clean":"2024-04-26T21:47:22.125317+0000","last_became_active":"2024-04-26T21:47:20.086977+0000","last_became_peered":"2024-04-26T21:47:20.086977+0000","last_unstale":"2024-04-26T21:47:22.125317+0000","last_undegraded":"2024-04-26T21:47:22.125317+0000","last_fullsized":"2024-04-26T21:47:22.125317+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.999826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.125296+0000","last_change":"2024-04-26T21:47:21.111371+0000","last_active":"2024-04-26T21:47:22.125296+0000","last_peered":"2024-04-26T21:47:22.125296+0000","last_clean":"2024-04-26T21:47:22.125296+0000","last_became_active":"2024-04-26T21:47:21.110947+0000","last_became_peered":"2024-04-26T21:47:21.110947+0000","last_unstale":"2024-04-26T21:47:22.125296+0000","last_undegraded":"2024-04-26T21:47:22.125296+0000","last_fullsized":"2024-04-26T21:47:22.125296+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:00.452861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121811+0000","last_change":"2024-04-26T21:47:20.096870+0000","last_active":"2024-04-26T21:47:22.121811+0000","last_peered":"2024-04-26T21:47:22.121811+0000","last_clean":"2024-04-26T21:47:22.121811+0000","last_became_active":"2024-04-26T21:47:20.095977+0000","last_became_peered":"2024-04-26T21:47:20.095977+0000","last_unstale":"2024-04-26T21:47:22.121811+0000","last_undegraded":"2024-04-26T21:47:22.121811+0000","last_fullsized":"2024-04-26T21:47:22.121811+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:30.519657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.124697+0000","last_change":"2024-04-26T21:47:21.113324+0000","last_active":"2024-04-26T21:47:22.124697+0000","last_peered":"2024-04-26T21:47:22.124697+0000","last_clean":"2024-04-26T21:47:22.124697+0000","last_became_active":"2024-04-26T21:47:21.111242+0000","last_became_peered":"2024-04-26T21:47:21.111242+0000","last_unstale":"2024-04-26T21:47:22.124697+0000","last_undegraded":"2024-04-26T21:47:22.124697+0000","last_fullsized":"2024-04-26T21:47:22.124697+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:20.073038+0000","last_clean_scrub_stamp":"2024-04-26T21:47:20.073038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:44.094262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:47:22.121911+0000","last_change":"2024-04-26T21:47:20.095253+0000","last_active":"2024-04-26T21:47:22.121911+0000","last_peered":"2024-04-26T21:47:22.121911+0000","last_clean":"2024-04-26T21:47:22.121911+0000","last_became_active":"2024-04-26T21:47:20.094959+0000","last_became_peered":"2024-04-26T21:47:20.094959+0000","last_unstale":"2024-04-26T21:47:22.121911+0000","last_undegraded":"2024-04-26T21:47:22.121911+0000","last_fullsized":"2024-04-26T21:47:22.121911+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:47:19.068476+0000","last_clean_scrub_stamp":"2024-04-26T21:47:19.068476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:35.288244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":410,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:47:21.102580+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:47:21.102580+0000","last_peered":"2024-04-26T21:47:21.102580+0000","last_clean":"2024-04-26T21:47:21.102580+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:47:21.102580+0000","last_undegraded":"2024-04-26T21:47:21.102580+0000","last_fullsized":"2024-04-26T21:47:21.102580+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2465792,"data_stored":1534989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":953369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1875968,"data_stored":951153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1871872,"data_stored":944553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1867776,"data_stored":945043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1880064,"data_stored":951187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37164,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334676,"statfs":{"total":96636764160,"available":96598708224,"internally_reserved":0,"allocated":1867776,"data_stored":944495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41916,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329924,"statfs":{"total":96636764160,"available":96593842176,"internally_reserved":0,"allocated":2473984,"data_stored":1536531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:24.488 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:47:24.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:47:24.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.633+0000 7f218b052640 1 Processor -- start 2024-04-26T21:47:24.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.633+0000 7f218b052640 1 -- start start 2024-04-26T21:47:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.633+0000 7f218b052640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21840d1720 0x7f21840d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f218b052640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21840701e0 con 0x7f21840d39f0 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f218b052640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21840ce370 con 0x7f218415d250 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f218b052640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f218415d630 con 0x7f21840d1720 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21895ef640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f21840d39f0 0x7f2184070970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46302/0 (socket says 172.21.15.40:46302) 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21895ef640 1 -- 172.21.15.40:0/1441919559 learned_addr learned my addr 172.21.15.40:0/1441919559 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f2183fff640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21840d1720 0x7f21840d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1505575733 0 0) 0x7f21840701e0 con 0x7f21840d39f0 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2170003520 con 0x7f21840d39f0 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3138038784 0 0) 0x7f2170003520 con 0x7f21840d39f0 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21840d1720 msgr2=0x7f21840d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21840d1720 0x7f21840d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 >> v1:172.21.15.184:6789/0 conn(0x7f218415d250 legacy=0x7f21840d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21840ce5f0 con 0x7f21840d39f0 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f217c0019f0 con 0x7f21840d39f0 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 >> v1:172.21.15.40:6789/0 conn(0x7f21840d39f0 legacy=0x7f2184070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21840d1720 0x7f21700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2170003fb0 0x7f2170004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170004d40 0x7f21700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f218415d630 con 0x7f2170003fb0 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f217000bbb0 con 0x7f2170004d40 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.634+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f217000bdb0 con 0x7f21840d1720 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.635+0000 7f2183fff640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2170003fb0 0x7f2170004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.635+0000 7f2188dee640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21840d1720 0x7f21700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.635+0000 7f21895ef640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170004d40 0x7f21700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.635+0000 7f2183fff640 1 -- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21840d1720 msgr2=0x7f21700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.635+0000 7f2183fff640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21840d1720 0x7f21700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.635+0000 7f2183fff640 1 -- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170004d40 msgr2=0x7f21700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.635+0000 7f2183fff640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170004d40 0x7f21700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.635+0000 7f2183fff640 1 -- 172.21.15.40:0/1441919559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2170003520 con 0x7f2170003fb0 2024-04-26T21:47:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f2183fff640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2170003fb0 0x7f2170004800 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f216c009ac0 tx=0x7f2170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac85c25e517e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c036070 con 0x7f2170003fb0 2024-04-26T21:47:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216c031400 con 0x7f2170003fb0 2024-04-26T21:47:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f21837fe640 1 -- 172.21.15.40:0/1441919559 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c009690 con 0x7f2170003fb0 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f218b052640 1 -- 172.21.15.40:0/1441919559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2170003fb0 msgr2=0x7f2170004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f218b052640 1 --2- 172.21.15.40:0/1441919559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2170003fb0 0x7f2170004800 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f216c009ac0 tx=0x7f2170005830 comp rx=0 tx=0).stop 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f21895ef640 1 -- 172.21.15.40:0/1441919559 reap_dead start 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f218b052640 1 -- 172.21.15.40:0/1441919559 shutdown_connections 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f218b052640 1 -- 172.21.15.40:0/1441919559 >> 172.21.15.40:0/1441919559 conn(0x7f21840d2d80 msgr2=0x7f21840d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f218b052640 1 -- 172.21.15.40:0/1441919559 shutdown_connections 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.636+0000 7f218b052640 1 -- 172.21.15.40:0/1441919559 wait complete. 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.637+0000 7f218b052640 1 Processor -- start 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.637+0000 7f218b052640 1 -- start start 2024-04-26T21:47:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f218b052640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170003fb0 0x7f21840a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f218b052640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21840b79f0 0x7f218415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f218b052640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f218415f1c0 0x7f218409cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f218b052640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21840cebb0 con 0x7f21840b79f0 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f218b052640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21840704e0 con 0x7f2170003fb0 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f218b052640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21840d0950 con 0x7f218415f1c0 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f2183fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21840b79f0 0x7f218415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f2183fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21840b79f0 0x7f218415ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57998/0 (socket says 172.21.15.40:57998) 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.638+0000 7f2183fff640 1 -- 172.21.15.40:0/862133083 learned_addr learned my addr 172.21.15.40:0/862133083 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f2188dee640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170003fb0 0x7f21840a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f21895ef640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f218415f1c0 0x7f218409cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f2183fff640 1 -- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f218415f1c0 msgr2=0x7f218409cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f2183fff640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f218415f1c0 0x7f218409cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f2183fff640 1 -- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170003fb0 msgr2=0x7f21840a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f2183fff640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170003fb0 0x7f21840a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f2183fff640 1 -- 172.21.15.40:0/862133083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21840a0ba0 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f2183fff640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21840b79f0 0x7f218415ec80 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f216c000c00 tx=0x7f216c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f21817fa640 1 -- 172.21.15.40:0/862133083 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c040020 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f21817fa640 1 -- 172.21.15.40:0/862133083 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216c003c60 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f21817fa640 1 -- 172.21.15.40:0/862133083 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c036040 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21840b7e80 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.639+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21840b5cf0 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.640+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218415f5d0 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.641+0000 7f21817fa640 1 -- 172.21.15.40:0/862133083 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f216c043020 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.641+0000 7f21817fa640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f216406c8f0 0x7f216406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.641+0000 7f21817fa640 1 -- 172.21.15.40:0/862133083 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f216c0ac080 con 0x7f21840b79f0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.641+0000 7f2188dee640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f216406c8f0 0x7f216406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:24.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.642+0000 7f2188dee640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f216406c8f0 0x7f216406edb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f21740017e0 tx=0x7f2174009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:24.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.646+0000 7f21817fa640 1 -- 172.21.15.40:0/862133083 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f218415f5d0 con 0x7f21840b79f0 2024-04-26T21:47:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.904+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f21840b8160 con 0x7f21840b79f0 2024-04-26T21:47:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.907+0000 7f21817fa640 1 -- 172.21.15.40:0/862133083 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f216c04a030 con 0x7f21840b79f0 2024-04-26T21:47:24.907 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:47:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDUHSxmMXDxJxAA01KUIcPv3DU4S+fRPrPQmw== 2024-04-26T21:47:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:47:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:47:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:47:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:47:24.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f216406c8f0 msgr2=0x7f216406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f216406c8f0 0x7f216406edb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f21740017e0 tx=0x7f2174009040 comp rx=0 tx=0).stop 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21840b79f0 msgr2=0x7f218415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21840b79f0 0x7f218415ec80 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f216c000c00 tx=0x7f216c004730 comp rx=0 tx=0).stop 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 shutdown_connections 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f216406c8f0 0x7f216406edb0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f218415f1c0 0x7f218409cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21840b79f0 0x7f218415ec80 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 --2- 172.21.15.40:0/862133083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2170003fb0 0x7f21840a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 >> 172.21.15.40:0/862133083 conn(0x7f21840d2d80 msgr2=0x7f218415fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 shutdown_connections 2024-04-26T21:47:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:24.909+0000 7f218b052640 1 -- 172.21.15.40:0/862133083 wait complete. 2024-04-26T21:47:24.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:25.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.021+0000 7faa20046640 1 Processor -- start 2024-04-26T21:47:25.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.021+0000 7faa20046640 1 -- start start 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa20046640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa18162810 0x7faa18070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa20046640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faa180d1b30 0x7faa180d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa20046640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa180d10d0 con 0x7faa180b2340 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa20046640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7faa180d0750 con 0x7faa18162810 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa20046640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faa180d16a0 con 0x7faa180d1b30 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1dde2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa18162810 0x7faa18070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1dde2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa18162810 0x7faa18070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58008/0 (socket says 172.21.15.40:58008) 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1dde2640 1 -- 172.21.15.40:0/791007339 learned_addr learned my addr 172.21.15.40:0/791007339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1e5e3640 1 --2- 172.21.15.40:0/791007339 >> v2:172.21.15.184:3300/0 conn(0x7faa180d1b30 0x7faa180d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1dde2640 1 -- 172.21.15.40:0/791007339 >> v2:172.21.15.184:3300/0 conn(0x7faa180d1b30 msgr2=0x7faa180d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1dde2640 1 --2- 172.21.15.40:0/791007339 >> v2:172.21.15.184:3300/0 conn(0x7faa180d1b30 0x7faa180d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1dde2640 1 -- 172.21.15.40:0/791007339 >> v1:172.21.15.184:6789/0 conn(0x7faa180b2340 legacy=0x7faa180cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1dde2640 1 -- 172.21.15.40:0/791007339 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa180d1370 con 0x7faa18162810 2024-04-26T21:47:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.022+0000 7faa1e5e3640 1 --2- 172.21.15.40:0/791007339 >> v2:172.21.15.184:3300/0 conn(0x7faa180d1b30 0x7faa180d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.023+0000 7faa1dde2640 1 --2- 172.21.15.40:0/791007339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18162810 0x7faa18070350 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7faa18073330 tx=0x7faa1402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91edb980d4a70385 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.023+0000 7faa1cde0640 1 -- 172.21.15.40:0/791007339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14037070 con 0x7faa18162810 2024-04-26T21:47:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.023+0000 7faa1cde0640 1 -- 172.21.15.40:0/791007339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa14032440 con 0x7faa18162810 2024-04-26T21:47:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.023+0000 7faa1cde0640 1 -- 172.21.15.40:0/791007339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14032810 con 0x7faa18162810 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.024+0000 7faa20046640 1 -- 172.21.15.40:0/791007339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18162810 msgr2=0x7faa18070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.024+0000 7faa20046640 1 --2- 172.21.15.40:0/791007339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18162810 0x7faa18070350 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7faa18073330 tx=0x7faa1402a230 comp rx=0 tx=0).stop 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.024+0000 7faa20046640 1 -- 172.21.15.40:0/791007339 shutdown_connections 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.024+0000 7faa20046640 1 --2- 172.21.15.40:0/791007339 >> v2:172.21.15.184:3300/0 conn(0x7faa180d1b30 0x7faa180d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.024+0000 7faa20046640 1 --2- 172.21.15.40:0/791007339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18162810 0x7faa18070350 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.024+0000 7faa20046640 1 -- 172.21.15.40:0/791007339 >> 172.21.15.40:0/791007339 conn(0x7faa180d2da0 msgr2=0x7faa180d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.024+0000 7faa20046640 1 -- 172.21.15.40:0/791007339 shutdown_connections 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.024+0000 7faa20046640 1 -- 172.21.15.40:0/791007339 wait complete. 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.025+0000 7faa20046640 1 Processor -- start 2024-04-26T21:47:25.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.025+0000 7faa20046640 1 -- start start 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa20046640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa180b2340 0x7faa180c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa20046640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa180d1b30 0x7faa181e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa20046640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa18162810 0x7faa181ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa20046640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa18070890 con 0x7faa180b2340 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa20046640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa180d0470 con 0x7faa180d1b30 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa20046640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa180cdb00 con 0x7faa18162810 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa180b2340 0x7faa180c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa180b2340 0x7faa180c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58022/0 (socket says 172.21.15.40:58022) 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 -- 172.21.15.40:0/3147575935 learned_addr learned my addr 172.21.15.40:0/3147575935 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 -- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa18162810 msgr2=0x7faa181ea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa18162810 0x7faa181ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1d5e1640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa180d1b30 0x7faa181e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 -- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa180d1b30 msgr2=0x7faa181e9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa180d1b30 0x7faa181e9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 -- 172.21.15.40:0/3147575935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa180c6be0 con 0x7faa180b2340 2024-04-26T21:47:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.026+0000 7faa1dde2640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa180b2340 0x7faa180c90e0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7faa14000c00 tx=0x7faa140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.027+0000 7faa0e7fc640 1 -- 172.21.15.40:0/3147575935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14041020 con 0x7faa180b2340 2024-04-26T21:47:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.027+0000 7faa0e7fc640 1 -- 172.21.15.40:0/3147575935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa14030970 con 0x7faa180b2340 2024-04-26T21:47:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.027+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa181eb040 con 0x7faa180b2340 2024-04-26T21:47:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.027+0000 7faa0e7fc640 1 -- 172.21.15.40:0/3147575935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14037040 con 0x7faa180b2340 2024-04-26T21:47:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.027+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa180caee0 con 0x7faa180b2340 2024-04-26T21:47:25.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.028+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa181eb430 con 0x7faa180b2340 2024-04-26T21:47:25.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.028+0000 7faa0e7fc640 1 -- 172.21.15.40:0/3147575935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa14044020 con 0x7faa180b2340 2024-04-26T21:47:25.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.028+0000 7faa0e7fc640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.029+0000 7faa0e7fc640 1 -- 172.21.15.40:0/3147575935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7faa140ac6c0 con 0x7faa180b2340 2024-04-26T21:47:25.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.029+0000 7faa1d5e1640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.029+0000 7faa1d5e1640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7faa18070860 tx=0x7faa08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.033+0000 7faa0e7fc640 1 -- 172.21.15.40:0/3147575935 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa1404b030 con 0x7faa180b2340 2024-04-26T21:47:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.256+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa18162350 con 0x7faa180b2340 2024-04-26T21:47:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.256+0000 7faa0e7fc640 1 -- 172.21.15.40:0/3147575935 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 357 v357) ==== 95+0+5201 (secure 0 0 0) 0x7faa1407a9f0 con 0x7faa180b2340 2024-04-26T21:47:25.256 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:25.256 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":357,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":357,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:22.462625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11757":{"gid":11757,"name":"a","rank":0,"incarnation":355,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3441996159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3441996159},{"type":"v1","addr":"172.21.15.40:6835","nonce":3441996159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11757,"qdb_cluster":[11757]},"id":7}]} 2024-04-26T21:47:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 357 2024-04-26T21:47:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9fc06c8f0 msgr2=0x7fa9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7faa18070860 tx=0x7faa08009040 comp rx=0 tx=0).stop 2024-04-26T21:47:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa180b2340 msgr2=0x7faa180c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa180b2340 0x7faa180c90e0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7faa14000c00 tx=0x7faa140046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 shutdown_connections 2024-04-26T21:47:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa18162810 0x7faa181ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa180d1b30 0x7faa181e9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 --2- 172.21.15.40:0/3147575935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa180b2340 0x7faa180c90e0 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 >> 172.21.15.40:0/3147575935 conn(0x7faa180d2da0 msgr2=0x7faa180cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 shutdown_connections 2024-04-26T21:47:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.258+0000 7faa20046640 1 -- 172.21.15.40:0/3147575935 wait complete. 2024-04-26T21:47:25.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 357, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:47:20.455876+0000', 'modified': '2024-04-26T21:47:22.462625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11757}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11757': {'gid': 11757, 'name': 'a', 'rank': 0, 'incarnation': 355, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/3441996159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3441996159}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3441996159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11757, 'qdb_cluster': [11757]} 2024-04-26T21:47:25.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:47:25.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:47:25.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.778+0000 7f617746d640 1 Processor -- start 2024-04-26T21:47:25.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.779+0000 7f617746d640 1 -- start start 2024-04-26T21:47:25.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.779+0000 7f617746d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f617015f3a0 0x7f61700d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.779+0000 7f617746d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61700d1770 0x7f61700d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.779+0000 7f617746d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.779+0000 7f617746d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f61700d25d0 con 0x7f617015f3a0 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.779+0000 7f617746d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f61700d0e20 con 0x7f61700d1770 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.779+0000 7f617746d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61700b1d70 con 0x7f61700d2160 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175209640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f617015f3a0 0x7f61700d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175209640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f617015f3a0 0x7f61700d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58030/0 (socket says 172.21.15.40:58030) 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175209640 1 -- 172.21.15.40:0/1908819407 learned_addr learned my addr 172.21.15.40:0/1908819407 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175a0a640 1 --2- 172.21.15.40:0/1908819407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6174a08640 1 --2- 172.21.15.40:0/1908819407 >> v2:172.21.15.184:3300/0 conn(0x7f61700d1770 0x7f61700d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175209640 1 -- 172.21.15.40:0/1908819407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 msgr2=0x7f61700d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175209640 1 --2- 172.21.15.40:0/1908819407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175209640 1 -- 172.21.15.40:0/1908819407 >> v2:172.21.15.184:3300/0 conn(0x7f61700d1770 msgr2=0x7f61700d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175209640 1 --2- 172.21.15.40:0/1908819407 >> v2:172.21.15.184:3300/0 conn(0x7f61700d1770 0x7f61700d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175209640 1 -- 172.21.15.40:0/1908819407 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61700b2050 con 0x7f617015f3a0 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6175a0a640 1 --2- 172.21.15.40:0/1908819407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.780+0000 7f6174a08640 1 --2- 172.21.15.40:0/1908819407 >> v2:172.21.15.184:3300/0 conn(0x7f61700d1770 0x7f61700d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f6175209640 1 --2- 172.21.15.40:0/1908819407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f617015f3a0 0x7f61700d06f0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f61700b7b80 tx=0x7f616c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf85f9510b51e9c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f6167fff640 1 -- 172.21.15.40:0/1908819407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c037070 con 0x7f617015f3a0 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f6167fff640 1 -- 172.21.15.40:0/1908819407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c032440 con 0x7f617015f3a0 2024-04-26T21:47:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f6167fff640 1 -- 172.21.15.40:0/1908819407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c032810 con 0x7f617015f3a0 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 -- 172.21.15.40:0/1908819407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f617015f3a0 msgr2=0x7f61700d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 --2- 172.21.15.40:0/1908819407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f617015f3a0 0x7f61700d06f0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f61700b7b80 tx=0x7f616c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 -- 172.21.15.40:0/1908819407 shutdown_connections 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 --2- 172.21.15.40:0/1908819407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 --2- 172.21.15.40:0/1908819407 >> v2:172.21.15.184:3300/0 conn(0x7f61700d1770 0x7f61700d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 --2- 172.21.15.40:0/1908819407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f617015f3a0 0x7f61700d06f0 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 -- 172.21.15.40:0/1908819407 >> 172.21.15.40:0/1908819407 conn(0x7f61700d4f90 msgr2=0x7f61700d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 -- 172.21.15.40:0/1908819407 shutdown_connections 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.781+0000 7f617746d640 1 -- 172.21.15.40:0/1908819407 wait complete. 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f617746d640 1 Processor -- start 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f617746d640 1 -- start start 2024-04-26T21:47:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f617746d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61700d1770 0x7f6170154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f617746d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f617746d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f617015f3a0 0x7f617006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f617746d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f617015f900 con 0x7f61700d1770 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f617746d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f617015f780 con 0x7f617015f3a0 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f617746d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61700d3390 con 0x7f61700d2160 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f6175209640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61700d1770 0x7f6170154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f6175209640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61700d1770 0x7f6170154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58040/0 (socket says 172.21.15.40:58040) 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.783+0000 7f6175209640 1 -- 172.21.15.40:0/1083910881 learned_addr learned my addr 172.21.15.40:0/1083910881 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6174a08640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6175a0a640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f617015f3a0 0x7f617006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6175209640 1 -- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 msgr2=0x7f61700b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6175209640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6175209640 1 -- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f617015f3a0 msgr2=0x7f617006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6175209640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f617015f3a0 0x7f617006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6175209640 1 -- 172.21.15.40:0/1083910881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61700b8640 con 0x7f61700d1770 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6175a0a640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f617015f3a0 0x7f617006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6175209640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61700d1770 0x7f6170154060 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f616c02a220 tx=0x7f616c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6165ffb640 1 -- 172.21.15.40:0/1083910881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c037040 con 0x7f61700d1770 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6165ffb640 1 -- 172.21.15.40:0/1083910881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c042070 con 0x7f61700d1770 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f617746d640 1 -- 172.21.15.40:0/1083910881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61700b8920 con 0x7f61700d1770 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f6165ffb640 1 -- 172.21.15.40:0/1083910881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c030bb0 con 0x7f61700d1770 2024-04-26T21:47:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.784+0000 7f617746d640 1 -- 172.21.15.40:0/1083910881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61700b8ee0 con 0x7f61700d1770 2024-04-26T21:47:25.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.786+0000 7f6165ffb640 1 -- 172.21.15.40:0/1083910881 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f616c047460 con 0x7f61700d1770 2024-04-26T21:47:25.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.786+0000 7f6165ffb640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f615406c8f0 0x7f615406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.786+0000 7f6165ffb640 1 -- 172.21.15.40:0/1083910881 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f616c0ac900 con 0x7f61700d1770 2024-04-26T21:47:25.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.787+0000 7f6174a08640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f615406c8f0 0x7f615406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.787+0000 7f6174a08640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f615406c8f0 0x7f615406edb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f61700a2e80 tx=0x7f6160008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.787+0000 7f617a182740 1 Processor -- start 2024-04-26T21:47:25.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.787+0000 7f617a182740 1 -- start start 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f617a182740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d19820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f617a182740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 0x5619f0d37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f617a182740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0cef120 0x5619f0d353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f617a182740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5619f0c53740 con 0x5619f0cef120 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f617a182740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5619f0bef3a0 con 0x5619f0d36ce0 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f617a182740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5619f0c4f700 con 0x5619f0d19440 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f6175a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0cef120 0x5619f0d353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f6175a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0cef120 0x5619f0d353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58050/0 (socket says 172.21.15.40:58050) 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f6175a0a640 1 -- 172.21.15.40:0/2290674048 learned_addr learned my addr 172.21.15.40:0/2290674048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f6174a08640 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 0x5619f0d37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.788+0000 7f6175209640 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d19820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f6175a0a640 1 -- 172.21.15.40:0/2290674048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 msgr2=0x5619f0d19820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f6175a0a640 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d19820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f6175a0a640 1 -- 172.21.15.40:0/2290674048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 msgr2=0x5619f0d37230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f6175a0a640 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 0x5619f0d37230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f6175a0a640 1 -- 172.21.15.40:0/2290674048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f0c0b750 con 0x5619f0cef120 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f6175a0a640 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0cef120 0x5619f0d353a0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f616800e570 tx=0x7f616800ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8578c8a167681 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f615affd640 1 -- 172.21.15.40:0/2290674048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168003b90 con 0x5619f0cef120 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f615affd640 1 -- 172.21.15.40:0/2290674048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6168003d30 con 0x5619f0cef120 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f615affd640 1 -- 172.21.15.40:0/2290674048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616800ca70 con 0x5619f0cef120 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f617a182740 1 -- 172.21.15.40:0/2290674048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0cef120 msgr2=0x5619f0d353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.789+0000 7f617a182740 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0cef120 0x5619f0d353a0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f616800e570 tx=0x7f616800ea30 comp rx=0 tx=0).stop 2024-04-26T21:47:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 -- 172.21.15.40:0/2290674048 shutdown_connections 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0cef120 0x5619f0d353a0 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 0x5619f0d37230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 --2- 172.21.15.40:0/2290674048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d19820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 -- 172.21.15.40:0/2290674048 >> 172.21.15.40:0/2290674048 conn(0x5619f0ceba50 msgr2=0x5619f0d34250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 -- 172.21.15.40:0/2290674048 shutdown_connections 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 -- 172.21.15.40:0/2290674048 wait complete. 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 -- start start 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.790+0000 7f617a182740 1 Processor -- start 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f617a182740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d06e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f617a182740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 0x5619f0d2b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f617a182740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d1cff0 0x5619f0d2ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f617a182740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5619f0cefec0 con 0x5619f0d1cff0 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f617a182740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5619f0bdafe0 con 0x5619f0d36ce0 2024-04-26T21:47:25.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f617a182740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5619f0ce9980 con 0x5619f0d19440 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f617a182740 10 client.0 _ll_register_callbacks cb 0x5619f0bf1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d1cff0 0x5619f0d2ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d1cff0 0x5619f0d2ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58062/0 (socket says 172.21.15.40:58062) 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175a0a640 1 -- 172.21.15.40:0/1082913954 learned_addr learned my addr 172.21.15.40:0/1082913954 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6174a08640 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 0x5619f0d2b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175209640 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d06e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175a0a640 1 -- 172.21.15.40:0/1082913954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 msgr2=0x5619f0d06e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175a0a640 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d06e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175a0a640 1 -- 172.21.15.40:0/1082913954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 msgr2=0x5619f0d2b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175a0a640 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 0x5619f0d2b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.791+0000 7f6175a0a640 1 -- 172.21.15.40:0/1082913954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f09652d0 con 0x5619f0d1cff0 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f6175a0a640 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d1cff0 0x5619f0d2ccf0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f616800ea70 tx=0x7f6168010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8408d529afa42d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f614ffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f614ffff640 1 -- 172.21.15.40:0/1082913954 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616801d040 con 0x5619f0d1cff0 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f614ffff640 1 -- 172.21.15.40:0/1082913954 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f616800ca50 con 0x5619f0d1cff0 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 1 -- 172.21.15.40:0/1082913954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6168003d30 con 0x5619f0d1cff0 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f614ffff640 1 -- 172.21.15.40:0/1082913954 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=357) ==== 24+0+0 (secure 0 0 0) 0x7f6168030050 con 0x5619f0d1cff0 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 10 client.11898 fetch_fsmap learned FSMap version 357 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 1 -- 172.21.15.40:0/1082913954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=357,monmap=2+}) -- 0x5619f0a2f6b0 con 0x5619f0d1cff0 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f614ffff640 1 -- 172.21.15.40:0/1082913954 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6168003d30 con 0x5619f0d1cff0 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f614ffff640 1 -- 172.21.15.40:0/1082913954 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 357) ==== 3494+0+0 (secure 0 0 0) 0x7f6168033020 con 0x5619f0d1cff0 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 10 client.11898 fetch_fsmap finished waiting for FSMap version 357 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 10 client.11898 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 20 client.11898 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] conn(0x5619f0d1a2e0 0x5619f0d1a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 4 client.11898 mds_command: new command op to 11757 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f617a182740 1 -- 172.21.15.40:0/1082913954 --> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619f0c61350 con 0x5619f0d1a2e0 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.792+0000 7f6175209640 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] conn(0x5619f0d1a2e0 0x5619f0d1a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.793+0000 7f614ffff640 10 client.11898 ms_handle_connect on v2:172.21.15.40:6834/3441996159 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.793+0000 7f6175209640 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] conn(0x5619f0d1a2e0 0x5619f0d1a6c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20075d3d86d8106e server_cookie=a81b6b2e1fc83eec in_seq=0 out_seq=0 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.793+0000 7f614ffff640 1 -- 172.21.15.40:0/1082913954 <== mds.0 v2:172.21.15.40:6834/3441996159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619f0c61350 con 0x5619f0d1a2e0 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.793+0000 7f614ffff640 10 client.11898 handle_command_reply: tid=0 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.793+0000 7f617a182740 1 client.11898 shutdown 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 -- 172.21.15.40:0/1082913954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d1cff0 msgr2=0x5619f0d2ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d1cff0 0x5619f0d2ccf0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f616800ea70 tx=0x7f6168010040 comp rx=0 tx=0).stop 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 -- 172.21.15.40:0/1082913954 shutdown_connections 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] conn(0x5619f0d1a2e0 0x5619f0d1a6c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d1cff0 0x5619f0d2ccf0 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d36ce0 0x5619f0d2b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 --2- 172.21.15.40:0/1082913954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d06e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 -- 172.21.15.40:0/1082913954 >> 172.21.15.40:0/1082913954 conn(0x5619f0cef120 msgr2=0x5619f0d38060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f614ffff640 0 client.11898 ms_handle_reset on v2:172.21.15.40:6834/3441996159 2024-04-26T21:47:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 -- 172.21.15.40:0/1082913954 shutdown_connections 2024-04-26T21:47:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 1 -- 172.21.15.40:0/1082913954 wait complete. 2024-04-26T21:47:25.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.794+0000 7f617a182740 20 client.11898 trim_cache size 0 max 16384 2024-04-26T21:47:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.812+0000 7f617a182740 1 Processor -- start 2024-04-26T21:47:25.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.812+0000 7f617a182740 1 -- start start 2024-04-26T21:47:25.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f617a182740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d57d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f617a182740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 0x5619f0d144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f617a182740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0da8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f617a182740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6170200ce0 con 0x5619f0d36ce0 2024-04-26T21:47:25.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f617a182740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61700cdf10 con 0x5619f0d1cff0 2024-04-26T21:47:25.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f617a182740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6170200690 con 0x5619f0d19440 2024-04-26T21:47:25.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0da8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0da8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58078/0 (socket says 172.21.15.40:58078) 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 -- 172.21.15.40:0/4056157504 learned_addr learned my addr 172.21.15.40:0/4056157504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 -- 172.21.15.40:0/4056157504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 msgr2=0x5619f0d57d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 --2- 172.21.15.40:0/4056157504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d57d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 -- 172.21.15.40:0/4056157504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 msgr2=0x5619f0d144f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 --2- 172.21.15.40:0/4056157504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 0x5619f0d144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 -- 172.21.15.40:0/4056157504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f0ce54e0 con 0x5619f0d36ce0 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.813+0000 7f6175a0a640 1 --2- 172.21.15.40:0/4056157504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0da8260 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f6168000f80 tx=0x7f6168013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b465dddbc5fc7bd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f615a7fc640 1 -- 172.21.15.40:0/4056157504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616800cd90 con 0x5619f0d36ce0 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f615a7fc640 1 -- 172.21.15.40:0/4056157504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6168028070 con 0x5619f0d36ce0 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f615a7fc640 1 -- 172.21.15.40:0/4056157504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616800ca90 con 0x5619f0d36ce0 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 -- 172.21.15.40:0/4056157504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 msgr2=0x5619f0da8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 --2- 172.21.15.40:0/4056157504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0da8260 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f6168000f80 tx=0x7f6168013000 comp rx=0 tx=0).stop 2024-04-26T21:47:25.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 -- 172.21.15.40:0/4056157504 shutdown_connections 2024-04-26T21:47:25.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 --2- 172.21.15.40:0/4056157504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0da8260 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 --2- 172.21.15.40:0/4056157504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 0x5619f0d144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 --2- 172.21.15.40:0/4056157504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d57d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 -- 172.21.15.40:0/4056157504 >> 172.21.15.40:0/4056157504 conn(0x5619f0cef120 msgr2=0x5619f0cee8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 -- 172.21.15.40:0/4056157504 shutdown_connections 2024-04-26T21:47:25.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.814+0000 7f617a182740 1 -- 172.21.15.40:0/4056157504 wait complete. 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.815+0000 7f617a182740 1 -- start start 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.815+0000 7f617a182740 1 Processor -- start 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f617a182740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d56e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f617a182740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 0x5619f0d34990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f617a182740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0d34ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f617a182740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5619f0d080e0 con 0x5619f0d36ce0 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f617a182740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5619f0d2c5b0 con 0x5619f0d1cff0 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f617a182740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5619f0d1c480 con 0x5619f0d19440 2024-04-26T21:47:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f617a182740 10 client.0 _ll_register_callbacks cb 0x7f616000a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0d34ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0d34ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58088/0 (socket says 172.21.15.40:58088) 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 -- 172.21.15.40:0/2946483355 learned_addr learned my addr 172.21.15.40:0/2946483355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6174a08640 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 0x5619f0d34990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175209640 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d56e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 -- 172.21.15.40:0/2946483355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 msgr2=0x5619f0d56e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d56e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 -- 172.21.15.40:0/2946483355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 msgr2=0x5619f0d34990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 0x5619f0d34990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 -- 172.21.15.40:0/2946483355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f0d369b0 con 0x5619f0d36ce0 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f6175a0a640 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0d34ed0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f6168000f80 tx=0x7f6168004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b641a4ea47b6cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.816+0000 7f614ffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:25.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f614ffff640 1 -- 172.21.15.40:0/2946483355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168042020 con 0x5619f0d36ce0 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f614ffff640 1 -- 172.21.15.40:0/2946483355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f6168025870 con 0x5619f0d36ce0 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 1 -- 172.21.15.40:0/2946483355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6168028070 con 0x5619f0d36ce0 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f614ffff640 1 -- 172.21.15.40:0/2946483355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=357) ==== 24+0+0 (secure 0 0 0) 0x7f6168025a70 con 0x5619f0d36ce0 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 10 client.11904 fetch_fsmap learned FSMap version 357 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 1 -- 172.21.15.40:0/2946483355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=357,monmap=2+}) -- 0x5619f0d060d0 con 0x5619f0d36ce0 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f614ffff640 1 -- 172.21.15.40:0/2946483355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6168028070 con 0x5619f0d36ce0 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f614ffff640 1 -- 172.21.15.40:0/2946483355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 357) ==== 3494+0+0 (secure 0 0 0) 0x7f6168033030 con 0x5619f0d36ce0 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 10 client.11904 fetch_fsmap finished waiting for FSMap version 357 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 10 client.11904 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 20 client.11904 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] conn(0x5619f0d90d70 0x5619f0d91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 4 client.11904 mds_command: new command op to 11757 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f617a182740 1 -- 172.21.15.40:0/2946483355 --> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] -- command(tid 0: {"prefix": "status"}) -- 0x5619f0c5b1c0 con 0x5619f0d90d70 2024-04-26T21:47:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f6175209640 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] conn(0x5619f0d90d70 0x5619f0d91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.817+0000 7f614ffff640 10 client.11904 ms_handle_connect on v2:172.21.15.40:6834/3441996159 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f6175209640 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] conn(0x5619f0d90d70 0x5619f0d91150 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c503aad9d6ad36 server_cookie=b073ffce5d74319 in_seq=0 out_seq=0 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f614ffff640 1 -- 172.21.15.40:0/2946483355 <== mds.0 v2:172.21.15.40:6834/3441996159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x5619f0c5b1c0 con 0x5619f0d90d70 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f614ffff640 10 client.11904 handle_command_reply: tid=0 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 client.11904 shutdown 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 -- 172.21.15.40:0/2946483355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 msgr2=0x5619f0d34ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0d34ed0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f6168000f80 tx=0x7f6168004ab0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 -- 172.21.15.40:0/2946483355 shutdown_connections 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.40:6834/3441996159,v1:172.21.15.40:6835/3441996159] conn(0x5619f0d90d70 0x5619f0d91150 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5619f0d36ce0 0x5619f0d34ed0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619f0d1cff0 0x5619f0d34990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f614ffff640 0 client.11904 ms_handle_reset on v2:172.21.15.40:6834/3441996159 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 --2- 172.21.15.40:0/2946483355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619f0d19440 0x5619f0d56e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 -- 172.21.15.40:0/2946483355 >> 172.21.15.40:0/2946483355 conn(0x5619f0cef120 msgr2=0x5619f0ce8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 -- 172.21.15.40:0/2946483355 shutdown_connections 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 1 -- 172.21.15.40:0/2946483355 wait complete. 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.818+0000 7f617a182740 20 client.11904 trim_cache size 0 max 16384 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:47:25.824 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 11757, 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 4.7243823020000004, 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 357, 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 145, 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 145, 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 9.7473739819999992 2024-04-26T21:47:25.825 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 -- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f615406c8f0 msgr2=0x7f615406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f615406c8f0 0x7f615406edb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f61700a2e80 tx=0x7f6160008040 comp rx=0 tx=0).stop 2024-04-26T21:47:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 -- 172.21.15.40:0/1083910881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61700d1770 msgr2=0x7f6170154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61700d1770 0x7f6170154060 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f616c02a220 tx=0x7f616c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 -- 172.21.15.40:0/1083910881 shutdown_connections 2024-04-26T21:47:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f615406c8f0 0x7f615406edb0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f617015f3a0 0x7f617006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700d2160 0x7f61700b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 --2- 172.21.15.40:0/1083910881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61700d1770 0x7f6170154060 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 -- 172.21.15.40:0/1083910881 >> 172.21.15.40:0/1083910881 conn(0x7f61700d4f90 msgr2=0x7f61700b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 -- 172.21.15.40:0/1083910881 shutdown_connections 2024-04-26T21:47:25.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.819+0000 7f617746d640 1 -- 172.21.15.40:0/1083910881 wait complete. 2024-04-26T21:47:25.829 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:47:25.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.930+0000 7ffa1117f640 1 Processor -- start 2024-04-26T21:47:25.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.930+0000 7ffa1117f640 1 -- start start 2024-04-26T21:47:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa1117f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa0c0d1b30 0x7ffa0c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa1117f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa0c0d10d0 con 0x7ffa0c0b2340 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa1117f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa0c0d1370 con 0x7ffa0c162810 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa1117f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c0d0750 con 0x7ffa0c0d1b30 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa0a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffa0c0b2340 0x7ffa0c0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46316/0 (socket says 172.21.15.40:46316) 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa0a59c640 1 -- 172.21.15.40:0/511238697 learned_addr learned my addr 172.21.15.40:0/511238697 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1608968303 0 0) 0x7ffa0c0d10d0 con 0x7ffa0c0b2340 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa0b59e640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa0c0d1b30 0x7ffa0c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.931+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9f8003520 con 0x7ffa0c0b2340 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 986620351 0 0) 0x7ff9f8003520 con 0x7ffa0c0b2340 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa0c0d1b30 msgr2=0x7ffa0c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa0c0d1b30 0x7ffa0c0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 >> v1:172.21.15.184:6789/0 conn(0x7ffa0c162810 legacy=0x7ffa0c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c0d01b0 con 0x7ffa0c0b2340 2024-04-26T21:47:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7ff9f4001670 con 0x7ffa0c0b2340 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 >> v1:172.21.15.40:6789/0 conn(0x7ffa0c0b2340 legacy=0x7ffa0c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80039e0 0x7ff9f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa0c0d1b30 0x7ff9f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8004d40 0x7ff9f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0d0750 con 0x7ff9f80039e0 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f800b9f0 con 0x7ff9f8004d40 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f800bbf0 con 0x7ffa0c0d1b30 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa0ad9d640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80039e0 0x7ff9f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa0b59e640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8004d40 0x7ff9f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa0a59c640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa0c0d1b30 0x7ff9f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa0ad9d640 1 -- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa0c0d1b30 msgr2=0x7ff9f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa0ad9d640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa0c0d1b30 0x7ff9f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa0ad9d640 1 -- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8004d40 msgr2=0x7ff9f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:25.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa0ad9d640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8004d40 0x7ff9f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.932+0000 7ffa0ad9d640 1 -- 172.21.15.40:0/511238697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f8003520 con 0x7ff9f80039e0 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.933+0000 7ffa0ad9d640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80039e0 0x7ff9f8003dc0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7ffa04001860 tx=0x7ff9f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f19414629ca821 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.933+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04037070 con 0x7ff9f80039e0 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.933+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0402ab60 con 0x7ff9f80039e0 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.933+0000 7ffa09d9b640 1 -- 172.21.15.40:0/511238697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04001430 con 0x7ff9f80039e0 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.934+0000 7ffa1117f640 1 -- 172.21.15.40:0/511238697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80039e0 msgr2=0x7ff9f8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.934+0000 7ffa1117f640 1 --2- 172.21.15.40:0/511238697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80039e0 0x7ff9f8003dc0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7ffa04001860 tx=0x7ff9f8005830 comp rx=0 tx=0).stop 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.934+0000 7ffa0b59e640 1 -- 172.21.15.40:0/511238697 reap_dead start 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.934+0000 7ffa1117f640 1 -- 172.21.15.40:0/511238697 shutdown_connections 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.934+0000 7ffa1117f640 1 -- 172.21.15.40:0/511238697 >> 172.21.15.40:0/511238697 conn(0x7ffa0c0d2da0 msgr2=0x7ffa0c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.934+0000 7ffa1117f640 1 -- 172.21.15.40:0/511238697 shutdown_connections 2024-04-26T21:47:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.934+0000 7ffa1117f640 1 -- 172.21.15.40:0/511238697 wait complete. 2024-04-26T21:47:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.935+0000 7ffa1117f640 1 Processor -- start 2024-04-26T21:47:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.935+0000 7ffa1117f640 1 -- start start 2024-04-26T21:47:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa1117f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f80039e0 0x7ffa0c14b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa1117f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c06e4d0 0x7ffa0c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa1117f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c15fb70 0x7ffa0c15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa1117f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0cdb00 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa1117f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0d16a0 con 0x7ffa0c15fb70 2024-04-26T21:47:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa1117f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c0d1fc0 con 0x7ff9f80039e0 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c06e4d0 0x7ffa0c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c06e4d0 0x7ffa0c06e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58112/0 (socket says 172.21.15.40:58112) 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0a59c640 1 -- 172.21.15.40:0/163244647 learned_addr learned my addr 172.21.15.40:0/163244647 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0a59c640 1 -- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f80039e0 msgr2=0x7ffa0c14b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0ad9d640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f80039e0 0x7ffa0c14b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0a59c640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f80039e0 0x7ffa0c14b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0a59c640 1 -- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c15fb70 msgr2=0x7ffa0c15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0b59e640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c15fb70 0x7ffa0c15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0a59c640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c15fb70 0x7ffa0c15fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.936+0000 7ffa0a59c640 1 -- 172.21.15.40:0/163244647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c14b9b0 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.937+0000 7ffa0a59c640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c06e4d0 0x7ffa0c06e950 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007340 tx=0x7ff9f4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.937+0000 7ffa037fe640 1 -- 172.21.15.40:0/163244647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f401a070 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.937+0000 7ffa037fe640 1 -- 172.21.15.40:0/163244647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f400f8e0 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.937+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c160770 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.937+0000 7ffa037fe640 1 -- 172.21.15.40:0/163244647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400fcd0 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.937+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c160d30 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.938+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0c0694f0 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.938+0000 7ffa037fe640 1 -- 172.21.15.40:0/163244647 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9f401d070 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.939+0000 7ffa037fe640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9ec06c8f0 0x7ff9ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.939+0000 7ffa037fe640 1 -- 172.21.15.40:0/163244647 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7ff9f408b640 con 0x7ffa0c06e4d0 2024-04-26T21:47:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.939+0000 7ffa0ad9d640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9ec06c8f0 0x7ff9ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.939+0000 7ffa0ad9d640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9ec06c8f0 0x7ff9ec06edb0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ffa04034000 tx=0x7ffa04009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:25.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:25.943+0000 7ffa037fe640 1 -- 172.21.15.40:0/163244647 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9f40598e0 con 0x7ffa0c06e4d0 2024-04-26T21:47:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.174+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa0c0d2670 con 0x7ffa0c06e4d0 2024-04-26T21:47:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.174+0000 7ffa037fe640 1 -- 172.21.15.40:0/163244647 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v357) ==== 53+0+73 (secure 0 0 0) 0x7ff9f40591e0 con 0x7ffa0c06e4d0 2024-04-26T21:47:26.174 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:47:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9ec06c8f0 msgr2=0x7ff9ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9ec06c8f0 0x7ff9ec06edb0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ffa04034000 tx=0x7ffa04009030 comp rx=0 tx=0).stop 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c06e4d0 msgr2=0x7ffa0c06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c06e4d0 0x7ffa0c06e950 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007340 tx=0x7ff9f4007800 comp rx=0 tx=0).stop 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 shutdown_connections 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff9ec06c8f0 0x7ff9ec06edb0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa0c15fb70 0x7ffa0c15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c06e4d0 0x7ffa0c06e950 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 --2- 172.21.15.40:0/163244647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f80039e0 0x7ffa0c14b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 >> 172.21.15.40:0/163244647 conn(0x7ffa0c0d2da0 msgr2=0x7ffa0c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.176+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 shutdown_connections 2024-04-26T21:47:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26.177+0000 7ffa1117f640 1 -- 172.21.15.40:0/163244647 wait complete. 2024-04-26T21:47:26.187 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:47:26.187 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:47:26.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:47:26.187 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:47:26.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:26.188 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:47:26.188 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:47:26.188 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:47:26.188 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:26.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.188 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:47:26.204 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42001sec preferred_lft 42001sec 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:47:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:26.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.206 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:47:26.266 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:47:26.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.268 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:47:26.324 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:47:26.324 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:47:26.324 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:47:26.324 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:47:26.324 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:47:26.325 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:47:26.325 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:47:26.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:47:26.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:47:26.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:47:26.325 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:47:26.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:47:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:26.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.451 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:47:26.466 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:47:26.466 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:47:26.466 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:26.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.467 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:47:26.530 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:47:26.530 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:26.585 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:26.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:47:26.586 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:26.640 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:47:26.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:47:26.722 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:47:26.723 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:47:26.730 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:47:26.730 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:47:26.730 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:47:26.731 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:47:26.731 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:47:26.731 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:47:26.731 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:47:26.731 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:47:26.731 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:47:26.731 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:47:26.733 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:47:26.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:26.799 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:47:26.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:47:26.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:26.901 INFO:teuthology.orchestra.run.smithi040.stdout:client8621 2024-04-26T21:47:26.901 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:26.901 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:26.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.917 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir1) 2024-04-26T21:47:26.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:26.977 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir1/file1) 2024-04-26T21:47:27.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.034 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir2) 2024-04-26T21:47:27.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.091 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir2/file2) 2024-04-26T21:47:27.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.241 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-26T21:47:27.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.258 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir1/file1 testdir/hardlink1) 2024-04-26T21:47:27.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:27.318 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir2/file2 testdir/hardlink2) 2024-04-26T21:47:27.375 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:47:27.376 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:47:27.376 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:47:27.376 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:47:27.376 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir1/file1").st_ino) 2024-04-26T21:47:27.376 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:47:27.482 INFO:teuthology.orchestra.run.smithi040.stdout:1099511627777 2024-04-26T21:47:27.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:27.491 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:47:27.491 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:47:27.491 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:47:27.491 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:47:27.491 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2/file2").st_ino) 2024-04-26T21:47:27.491 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:47:27.597 INFO:teuthology.orchestra.run.smithi040.stdout:1099511628279 2024-04-26T21:47:27.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:27.606 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:47:27.606 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:47:27.606 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2024-04-26T21:47:27.606 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:47:27.606 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2").st_ino) 2024-04-26T21:47:27.606 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:47:27.713 INFO:teuthology.orchestra.run.smithi040.stdout:1099511628278 2024-04-26T21:47:27.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:27.722 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:27.722 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:27.777 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:27.777 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:27.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.974+0000 7f7e32f38640 1 Processor -- start 2024-04-26T21:47:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.974+0000 7f7e32f38640 1 -- start start 2024-04-26T21:47:27.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.975+0000 7f7e32f38640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e2c15b600 0x7f7e2c071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.975+0000 7f7e32f38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cf5e0 0x7f7e2c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.975+0000 7f7e32f38640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2c072450 con 0x7f7e2c0cbbd0 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.975+0000 7f7e32f38640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2c0cb9b0 con 0x7f7e2c15b600 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.975+0000 7f7e32f38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c15b9e0 con 0x7f7e2c0cf5e0 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.975+0000 7f7e30cd4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e2c15b600 0x7f7e2c071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.975+0000 7f7e30cd4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e2c15b600 0x7f7e2c071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58116/0 (socket says 172.21.15.40:58116) 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.975+0000 7f7e30cd4640 1 -- 172.21.15.40:0/475713337 learned_addr learned my addr 172.21.15.40:0/475713337 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.976+0000 7f7e2bfff640 1 --2- 172.21.15.40:0/475713337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cf5e0 0x7f7e2c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.976+0000 7f7e30cd4640 1 -- 172.21.15.40:0/475713337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cf5e0 msgr2=0x7f7e2c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.976+0000 7f7e30cd4640 1 --2- 172.21.15.40:0/475713337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cf5e0 0x7f7e2c0ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.976+0000 7f7e30cd4640 1 -- 172.21.15.40:0/475713337 >> v1:172.21.15.184:6789/0 conn(0x7f7e2c0cbbd0 legacy=0x7f7e2c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.976+0000 7f7e30cd4640 1 -- 172.21.15.40:0/475713337 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c0d0740 con 0x7f7e2c15b600 2024-04-26T21:47:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.976+0000 7f7e30cd4640 1 --2- 172.21.15.40:0/475713337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c071f10 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009870 tx=0x7f7e2002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a1bffb3402a81b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.976+0000 7f7e2b7fe640 1 -- 172.21.15.40:0/475713337 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20037070 con 0x7f7e2c15b600 2024-04-26T21:47:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e2b7fe640 1 -- 172.21.15.40:0/475713337 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e20032440 con 0x7f7e2c15b600 2024-04-26T21:47:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e2b7fe640 1 -- 172.21.15.40:0/475713337 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20032810 con 0x7f7e2c15b600 2024-04-26T21:47:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e32f38640 1 -- 172.21.15.40:0/475713337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 msgr2=0x7f7e2c071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e32f38640 1 --2- 172.21.15.40:0/475713337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c071f10 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009870 tx=0x7f7e2002a210 comp rx=0 tx=0).stop 2024-04-26T21:47:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e32f38640 1 -- 172.21.15.40:0/475713337 shutdown_connections 2024-04-26T21:47:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e32f38640 1 --2- 172.21.15.40:0/475713337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cf5e0 0x7f7e2c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e32f38640 1 --2- 172.21.15.40:0/475713337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c071f10 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e32f38640 1 -- 172.21.15.40:0/475713337 >> 172.21.15.40:0/475713337 conn(0x7f7e2c0d0be0 msgr2=0x7f7e2c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:27.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e32f38640 1 -- 172.21.15.40:0/475713337 shutdown_connections 2024-04-26T21:47:27.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.977+0000 7f7e32f38640 1 -- 172.21.15.40:0/475713337 wait complete. 2024-04-26T21:47:27.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.978+0000 7f7e32f38640 1 Processor -- start 2024-04-26T21:47:27.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.978+0000 7f7e32f38640 1 -- start start 2024-04-26T21:47:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e32f38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cbbd0 0x7f7e2c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e32f38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c0cf5e0 0x7f7e2c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e32f38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e32f38640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c0d02f0 con 0x7f7e2c15b600 2024-04-26T21:47:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e32f38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c0cb640 con 0x7f7e2c0cf5e0 2024-04-26T21:47:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e32f38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c0d0470 con 0x7f7e2c0cbbd0 2024-04-26T21:47:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e314d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e314d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58118/0 (socket says 172.21.15.40:58118) 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e314d5640 1 -- 172.21.15.40:0/3790628140 learned_addr learned my addr 172.21.15.40:0/3790628140 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e30cd4640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cbbd0 0x7f7e2c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.979+0000 7f7e2bfff640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c0cf5e0 0x7f7e2c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e314d5640 1 -- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cbbd0 msgr2=0x7f7e2c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e314d5640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cbbd0 0x7f7e2c0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e314d5640 1 -- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c0cf5e0 msgr2=0x7f7e2c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e314d5640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c0cf5e0 0x7f7e2c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e314d5640 1 -- 172.21.15.40:0/3790628140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c0c5840 con 0x7f7e2c15b600 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e314d5640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c072e40 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c700 tx=0x7f7e2400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e297fa640 1 -- 172.21.15.40:0/3790628140 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24016020 con 0x7f7e2c15b600 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e297fa640 1 -- 172.21.15.40:0/3790628140 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2400aa50 con 0x7f7e2c15b600 2024-04-26T21:47:27.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e297fa640 1 -- 172.21.15.40:0/3790628140 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400ae40 con 0x7f7e2c15b600 2024-04-26T21:47:27.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c0c5b20 con 0x7f7e2c15b600 2024-04-26T21:47:27.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.980+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c152090 con 0x7f7e2c15b600 2024-04-26T21:47:27.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.981+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c0694f0 con 0x7f7e2c15b600 2024-04-26T21:47:27.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.982+0000 7f7e297fa640 1 -- 172.21.15.40:0/3790628140 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e24019020 con 0x7f7e2c15b600 2024-04-26T21:47:27.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.982+0000 7f7e297fa640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7e1006c8f0 0x7f7e1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:27.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.982+0000 7f7e297fa640 1 -- 172.21.15.40:0/3790628140 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f7e2408bdf0 con 0x7f7e2c15b600 2024-04-26T21:47:27.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.982+0000 7f7e30cd4640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7e1006c8f0 0x7f7e1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:27.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.983+0000 7f7e30cd4640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7e1006c8f0 0x7f7e1006edb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7e20034000 tx=0x7f7e2002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:27.987+0000 7f7e297fa640 1 -- 172.21.15.40:0/3790628140 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e24020020 con 0x7f7e2c15b600 2024-04-26T21:47:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.176+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e2c0ce7b0 con 0x7f7e2c15b600 2024-04-26T21:47:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.177+0000 7f7e297fa640 1 -- 172.21.15.40:0/3790628140 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16848 (secure 0 0 0) 0x7f7e24059ad0 con 0x7f7e2c15b600 2024-04-26T21:47:28.177 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:28.177 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":145,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:22.108045+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:18.726844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:19.403532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7e1006c8f0 msgr2=0x7f7e1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7e1006c8f0 0x7f7e1006edb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7e20034000 tx=0x7f7e2002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 msgr2=0x7f7e2c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c072e40 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c700 tx=0x7f7e2400cbc0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 shutdown_connections 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7e1006c8f0 0x7f7e1006edb0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e2c15b600 0x7f7e2c072e40 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c0cf5e0 0x7f7e2c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 --2- 172.21.15.40:0/3790628140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c0cbbd0 0x7f7e2c0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 >> 172.21.15.40:0/3790628140 conn(0x7f7e2c0d0be0 msgr2=0x7f7e2c0c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 shutdown_connections 2024-04-26T21:47:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.179+0000 7f7e32f38640 1 -- 172.21.15.40:0/3790628140 wait complete. 2024-04-26T21:47:28.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:18.726844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:47:19.403532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:28.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.294+0000 7fe6f8df2640 1 Processor -- start 2024-04-26T21:47:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.294+0000 7fe6f8df2640 1 -- start start 2024-04-26T21:47:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.295+0000 7fe6f8df2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40d1a00 0x7fe6f40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.295+0000 7fe6f8df2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe6f40b40d0 0x7fe6f40b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.295+0000 7fe6f8df2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f40d0a10 con 0x7fe6f40d2320 2024-04-26T21:47:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.295+0000 7fe6f8df2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f40d0cf0 con 0x7fe6f40b40d0 2024-04-26T21:47:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.295+0000 7fe6f8df2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f40d0fd0 con 0x7fe6f40d1a00 2024-04-26T21:47:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f1d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe6f40d2320 0x7fe6f41625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46322/0 (socket says 172.21.15.40:46322) 2024-04-26T21:47:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f1d9b640 1 -- 172.21.15.40:0/3614037456 learned_addr learned my addr 172.21.15.40:0/3614037456 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1388458095 0 0) 0x7fe6f40d0a10 con 0x7fe6f40d2320 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6e0003520 con 0x7fe6f40d2320 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f2d9d640 1 --2- 172.21.15.40:0/3614037456 >> v2:172.21.15.184:3300/0 conn(0x7fe6f40b40d0 0x7fe6f40b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f259c640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40d1a00 0x7fe6f40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1805657023 0 0) 0x7fe6e0003520 con 0x7fe6f40d2320 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40d1a00 msgr2=0x7fe6f40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40d1a00 0x7fe6f40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 >> v2:172.21.15.184:3300/0 conn(0x7fe6f40b40d0 msgr2=0x7fe6f40b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 --2- 172.21.15.40:0/3614037456 >> v2:172.21.15.184:3300/0 conn(0x7fe6f40b40d0 0x7fe6f40b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f40d40a0 con 0x7fe6f40d2320 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fe6dc001670 con 0x7fe6f40d2320 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 >> v1:172.21.15.40:6789/0 conn(0x7fe6f40d2320 legacy=0x7fe6f41625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6e00039b0 0x7fe6e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40d1a00 0x7fe6e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6e0004d40 0x7fe6e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40d0cf0 con 0x7fe6e0004d40 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40d0fd0 con 0x7fe6e00039b0 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e000bb20 con 0x7fe6f40d1a00 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f2d9d640 1 --2- 172.21.15.40:0/3614037456 >> v2:172.21.15.184:3300/0 conn(0x7fe6f40b40d0 0x7fe6f40b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f2d9d640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6e0004d40 0x7fe6e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f259c640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6e00039b0 0x7fe6e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.296+0000 7fe6f1d9b640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40d1a00 0x7fe6e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f2d9d640 1 -- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40d1a00 msgr2=0x7fe6e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f2d9d640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40d1a00 0x7fe6e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f2d9d640 1 -- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6e00039b0 msgr2=0x7fe6e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f2d9d640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6e00039b0 0x7fe6e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f2d9d640 1 -- 172.21.15.40:0/3614037456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e0003520 con 0x7fe6e0004d40 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f2d9d640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6e0004d40 0x7fe6e00051c0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40073e0 tx=0x7fe6e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c108036be6145610 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4009c70 con 0x7fe6e0004d40 2024-04-26T21:47:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e402c680 con 0x7fe6e0004d40 2024-04-26T21:47:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.297+0000 7fe6f159a640 1 -- 172.21.15.40:0/3614037456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e402ca30 con 0x7fe6e0004d40 2024-04-26T21:47:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.298+0000 7fe6f8df2640 1 -- 172.21.15.40:0/3614037456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6e0004d40 msgr2=0x7fe6e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.298+0000 7fe6f8df2640 1 --2- 172.21.15.40:0/3614037456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6e0004d40 0x7fe6e00051c0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40073e0 tx=0x7fe6e0005830 comp rx=0 tx=0).stop 2024-04-26T21:47:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.298+0000 7fe6f2d9d640 1 -- 172.21.15.40:0/3614037456 reap_dead start 2024-04-26T21:47:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.298+0000 7fe6f8df2640 1 -- 172.21.15.40:0/3614037456 shutdown_connections 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.298+0000 7fe6f8df2640 1 -- 172.21.15.40:0/3614037456 >> 172.21.15.40:0/3614037456 conn(0x7fe6f40d4fd0 msgr2=0x7fe6f40d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.298+0000 7fe6f8df2640 1 -- 172.21.15.40:0/3614037456 shutdown_connections 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.298+0000 7fe6f8df2640 1 -- 172.21.15.40:0/3614037456 wait complete. 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.299+0000 7fe6f8df2640 1 Processor -- start 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f8df2640 1 -- start start 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f8df2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6e0004d40 0x7fe6f40c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f8df2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6f4155880 0x7fe6f40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f8df2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f40c86f0 0x7fe6f40ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f8df2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40d1de0 con 0x7fe6f4155880 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f8df2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4162f50 con 0x7fe6f40c86f0 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f8df2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f40d1f60 con 0x7fe6e0004d40 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f1d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6f4155880 0x7fe6f40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f1d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6f4155880 0x7fe6f40b7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58134/0 (socket says 172.21.15.40:58134) 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.300+0000 7fe6f1d9b640 1 -- 172.21.15.40:0/2360046420 learned_addr learned my addr 172.21.15.40:0/2360046420 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f259c640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6e0004d40 0x7fe6f40c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f2d9d640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f40c86f0 0x7fe6f40ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f1d9b640 1 -- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6e0004d40 msgr2=0x7fe6f40c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f1d9b640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6e0004d40 0x7fe6f40c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f1d9b640 1 -- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f40c86f0 msgr2=0x7fe6f40ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f1d9b640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f40c86f0 0x7fe6f40ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f1d9b640 1 -- 172.21.15.40:0/2360046420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f40c9060 con 0x7fe6f4155880 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f1d9b640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6f4155880 0x7fe6f40b7870 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc007370 tx=0x7fe6dc007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6eaffd640 1 -- 172.21.15.40:0/2360046420 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00f940 con 0x7fe6f4155880 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6eaffd640 1 -- 172.21.15.40:0/2360046420 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc018070 con 0x7fe6f4155880 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f40c9340 con 0x7fe6f4155880 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6eaffd640 1 -- 172.21.15.40:0/2360046420 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00fd60 con 0x7fe6f4155880 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.301+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f40b9540 con 0x7fe6f4155880 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.302+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f41625f0 con 0x7fe6f4155880 2024-04-26T21:47:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.303+0000 7fe6eaffd640 1 -- 172.21.15.40:0/2360046420 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6dc01d460 con 0x7fe6f4155880 2024-04-26T21:47:28.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.303+0000 7fe6eaffd640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6d406c8f0 0x7fe6d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.303+0000 7fe6eaffd640 1 -- 172.21.15.40:0/2360046420 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7fe6dc08bc10 con 0x7fe6f4155880 2024-04-26T21:47:28.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.303+0000 7fe6f259c640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6d406c8f0 0x7fe6d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.304+0000 7fe6f259c640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6d406c8f0 0x7fe6d406edb0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec006fd0 tx=0x7fe6ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.307+0000 7fe6eaffd640 1 -- 172.21.15.40:0/2360046420 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe6f41625f0 con 0x7fe6f4155880 2024-04-26T21:47:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.490+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe6f40694f0 con 0x7fe6f4155880 2024-04-26T21:47:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.490+0000 7fe6eaffd640 1 -- 172.21.15.40:0/2360046420 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7fe6dc0598f0 con 0x7fe6f4155880 2024-04-26T21:47:28.490 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:47:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6d406c8f0 msgr2=0x7fe6d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6d406c8f0 0x7fe6d406edb0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec006fd0 tx=0x7fe6ec008040 comp rx=0 tx=0).stop 2024-04-26T21:47:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6f4155880 msgr2=0x7fe6f40b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6f4155880 0x7fe6f40b7870 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc007370 tx=0x7fe6dc007830 comp rx=0 tx=0).stop 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 shutdown_connections 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe6d406c8f0 0x7fe6d406edb0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f40c86f0 0x7fe6f40ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6f4155880 0x7fe6f40b7870 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 --2- 172.21.15.40:0/2360046420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6e0004d40 0x7fe6f40c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.492+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 >> 172.21.15.40:0/2360046420 conn(0x7fe6f40d4fd0 msgr2=0x7fe6f40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.493+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 shutdown_connections 2024-04-26T21:47:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.493+0000 7fe6f8df2640 1 -- 172.21.15.40:0/2360046420 wait complete. 2024-04-26T21:47:28.503 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:47:28.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.607+0000 7f2c2769b640 1 Processor -- start 2024-04-26T21:47:28.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.607+0000 7f2c2769b640 1 -- start start 2024-04-26T21:47:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c2769b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c2015f170 0x7f2c200d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c2769b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2c200d3bc0 0x7f2c200d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c2769b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c200d1040 con 0x7f2c2015f550 2024-04-26T21:47:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c2769b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2c200c9430 con 0x7f2c2015f170 2024-04-26T21:47:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c2769b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2c200c9680 con 0x7f2c200d3bc0 2024-04-26T21:47:28.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c25437640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c2015f170 0x7f2c200d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c24c36640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2c2015f550 0x7f2c200d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46328/0 (socket says 172.21.15.40:46328) 2024-04-26T21:47:28.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c24c36640 1 -- 172.21.15.40:0/1858808516 learned_addr learned my addr 172.21.15.40:0/1858808516 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:28.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.608+0000 7f2c25c38640 1 --2- 172.21.15.40:0/1858808516 >> v2:172.21.15.184:3300/0 conn(0x7f2c200d3bc0 0x7f2c200d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.609+0000 7f2c25437640 1 -- 172.21.15.40:0/1858808516 >> v2:172.21.15.184:3300/0 conn(0x7f2c200d3bc0 msgr2=0x7f2c200d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:28.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.609+0000 7f2c25437640 1 --2- 172.21.15.40:0/1858808516 >> v2:172.21.15.184:3300/0 conn(0x7f2c200d3bc0 0x7f2c200d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.609+0000 7f2c25437640 1 -- 172.21.15.40:0/1858808516 >> v1:172.21.15.40:6789/0 conn(0x7f2c2015f550 legacy=0x7f2c200d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.609+0000 7f2c25437640 1 -- 172.21.15.40:0/1858808516 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c200c9960 con 0x7f2c2015f170 2024-04-26T21:47:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.609+0000 7f2c25437640 1 --2- 172.21.15.40:0/1858808516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c2015f170 0x7f2c200d5a80 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f2c180097a0 tx=0x7f2c1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a29512434777e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.609+0000 7f2c1ffff640 1 -- 172.21.15.40:0/1858808516 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18037070 con 0x7f2c2015f170 2024-04-26T21:47:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.609+0000 7f2c1ffff640 1 -- 172.21.15.40:0/1858808516 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c18032440 con 0x7f2c2015f170 2024-04-26T21:47:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.609+0000 7f2c1ffff640 1 -- 172.21.15.40:0/1858808516 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18032810 con 0x7f2c2015f170 2024-04-26T21:47:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.610+0000 7f2c2769b640 1 -- 172.21.15.40:0/1858808516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c2015f170 msgr2=0x7f2c200d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.610+0000 7f2c2769b640 1 --2- 172.21.15.40:0/1858808516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c2015f170 0x7f2c200d5a80 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f2c180097a0 tx=0x7f2c1802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.610+0000 7f2c2769b640 1 -- 172.21.15.40:0/1858808516 shutdown_connections 2024-04-26T21:47:28.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.610+0000 7f2c2769b640 1 --2- 172.21.15.40:0/1858808516 >> v2:172.21.15.184:3300/0 conn(0x7f2c200d3bc0 0x7f2c200d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.610+0000 7f2c2769b640 1 --2- 172.21.15.40:0/1858808516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c2015f170 0x7f2c200d5a80 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.610+0000 7f2c2769b640 1 -- 172.21.15.40:0/1858808516 >> 172.21.15.40:0/1858808516 conn(0x7f2c200d4f60 msgr2=0x7f2c200d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:28.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.610+0000 7f2c2769b640 1 -- 172.21.15.40:0/1858808516 shutdown_connections 2024-04-26T21:47:28.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.610+0000 7f2c2769b640 1 -- 172.21.15.40:0/1858808516 wait complete. 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.611+0000 7f2c2769b640 1 Processor -- start 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.611+0000 7f2c2769b640 1 -- start start 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c2769b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c200d3bc0 0x7f2c2009fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c2769b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c2015f170 0x7f2c20150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c2769b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c200a2410 0x7f2c20153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c2769b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2c200ca5c0 con 0x7f2c200d3bc0 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c2769b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c200ca440 con 0x7f2c200a2410 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c2769b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c200ca740 con 0x7f2c2015f170 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c200d3bc0 0x7f2c2009fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c200d3bc0 0x7f2c2009fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58154/0 (socket says 172.21.15.40:58154) 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 -- 172.21.15.40:0/2260392048 learned_addr learned my addr 172.21.15.40:0/2260392048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 -- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c2015f170 msgr2=0x7f2c20150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c2015f170 0x7f2c20150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 -- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c200a2410 msgr2=0x7f2c20153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c200a2410 0x7f2c20153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 -- 172.21.15.40:0/2260392048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c20151910 con 0x7f2c200d3bc0 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.612+0000 7f2c25437640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c200d3bc0 0x7f2c2009fa20 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f2c18035040 tx=0x7f2c180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.613+0000 7f2c1dffb640 1 -- 172.21.15.40:0/2260392048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18037040 con 0x7f2c200d3bc0 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.613+0000 7f2c1dffb640 1 -- 172.21.15.40:0/2260392048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c18030770 con 0x7f2c200d3bc0 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.613+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c20151bf0 con 0x7f2c200d3bc0 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.613+0000 7f2c1dffb640 1 -- 172.21.15.40:0/2260392048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18030b80 con 0x7f2c200d3bc0 2024-04-26T21:47:28.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.613+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c201521b0 con 0x7f2c200d3bc0 2024-04-26T21:47:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.614+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c200694f0 con 0x7f2c200d3bc0 2024-04-26T21:47:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.614+0000 7f2c1dffb640 1 -- 172.21.15.40:0/2260392048 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c18044020 con 0x7f2c200d3bc0 2024-04-26T21:47:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.615+0000 7f2c1dffb640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.615+0000 7f2c1dffb640 1 -- 172.21.15.40:0/2260392048 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f2c180ac2f0 con 0x7f2c200d3bc0 2024-04-26T21:47:28.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.615+0000 7f2c24c36640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.615+0000 7f2c24c36640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000f60 tx=0x7f2c14007a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.619+0000 7f2c1dffb640 1 -- 172.21.15.40:0/2260392048 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c18032e20 con 0x7f2c200d3bc0 2024-04-26T21:47:28.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.803+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2c20074590 con 0x7f2c200d3bc0 2024-04-26T21:47:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.803+0000 7f2c1dffb640 1 -- 172.21.15.40:0/2260392048 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7f2c1807a4f0 con 0x7f2c200d3bc0 2024-04-26T21:47:28.803 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:47:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2c0c06c8f0 msgr2=0x7f2c0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000f60 tx=0x7f2c14007a80 comp rx=0 tx=0).stop 2024-04-26T21:47:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c200d3bc0 msgr2=0x7f2c2009fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c200d3bc0 0x7f2c2009fa20 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f2c18035040 tx=0x7f2c180046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 shutdown_connections 2024-04-26T21:47:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c200a2410 0x7f2c20153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c2015f170 0x7f2c20150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 --2- 172.21.15.40:0/2260392048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c200d3bc0 0x7f2c2009fa20 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 >> 172.21.15.40:0/2260392048 conn(0x7f2c200d4f60 msgr2=0x7f2c200c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 shutdown_connections 2024-04-26T21:47:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.805+0000 7f2c2769b640 1 -- 172.21.15.40:0/2260392048 wait complete. 2024-04-26T21:47:28.816 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:47:28.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.919+0000 7f8d0f119640 1 Processor -- start 2024-04-26T21:47:28.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.919+0000 7f8d0f119640 1 -- start start 2024-04-26T21:47:28.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.920+0000 7f8d0f119640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8d080b40d0 0x7f8d081625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.920+0000 7f8d0f119640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d080d0a10 con 0x7f8d08162b30 2024-04-26T21:47:28.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.920+0000 7f8d0f119640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d080d0cf0 con 0x7f8d080d1a00 2024-04-26T21:47:28.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.920+0000 7f8d0f119640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8d080d0fd0 con 0x7f8d080b40d0 2024-04-26T21:47:28.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.920+0000 7f8d0d6b6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8d08162b30 0x7f8d080b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53010/0 (socket says 172.21.15.40:53010) 2024-04-26T21:47:28.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.920+0000 7f8d0d6b6640 1 -- 172.21.15.40:0/154826224 learned_addr learned my addr 172.21.15.40:0/154826224 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:28.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d07fff640 1 --2- 172.21.15.40:0/154826224 >> v2:172.21.15.184:3300/0 conn(0x7f8d080b40d0 0x7f8d081625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569003366 0 0) 0x7f8d080d0a10 con 0x7f8d08162b30 2024-04-26T21:47:28.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cf0003520 con 0x7f8d08162b30 2024-04-26T21:47:28.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2541319763 0 0) 0x7f8cf0003520 con 0x7f8d08162b30 2024-04-26T21:47:28.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 >> v2:172.21.15.184:3300/0 conn(0x7f8d080b40d0 msgr2=0x7f8d081625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:28.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 --2- 172.21.15.40:0/154826224 >> v2:172.21.15.184:3300/0 conn(0x7f8d080b40d0 0x7f8d081625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 >> v1:172.21.15.184:6789/0 conn(0x7f8d080d1a00 legacy=0x7f8d080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d08162f80 con 0x7f8d08162b30 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f8cf8001670 con 0x7f8d08162b30 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 >> v1:172.21.15.40:6789/0 conn(0x7f8d08162b30 legacy=0x7f8d080b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00039b0 0x7f8cf0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf00043b0 0x7f8cf0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf0005120 0x7f8cf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d080d0fd0 con 0x7f8cf00043b0 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf000c060 con 0x7f8cf00039b0 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf000c260 con 0x7f8cf0005120 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d07fff640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf00043b0 0x7f8cf0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d0ceb5640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00039b0 0x7f8cf0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d0d6b6640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf0005120 0x7f8cf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d07fff640 1 -- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf0005120 msgr2=0x7f8cf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:28.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d07fff640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf0005120 0x7f8cf000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d07fff640 1 -- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00039b0 msgr2=0x7f8cf0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d07fff640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00039b0 0x7f8cf0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d07fff640 1 -- 172.21.15.40:0/154826224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf0003520 con 0x7f8cf00043b0 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d0ceb5640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00039b0 0x7f8cf0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.921+0000 7f8d0d6b6640 1 -- 172.21.15.40:0/154826224 reap_dead start 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.922+0000 7f8d07fff640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf00043b0 0x7f8cf0004880 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009ac0 tx=0x7f8cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1405a812aedf5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.922+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc036070 con 0x7f8cf00043b0 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.922+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc031400 con 0x7f8cf00043b0 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.922+0000 7f8d077fe640 1 -- 172.21.15.40:0/154826224 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc009690 con 0x7f8cf00043b0 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.923+0000 7f8d0f119640 1 -- 172.21.15.40:0/154826224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf00043b0 msgr2=0x7f8cf0004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.923+0000 7f8d0f119640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf00043b0 0x7f8cf0004880 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009ac0 tx=0x7f8cf0005830 comp rx=0 tx=0).stop 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.923+0000 7f8d0f119640 1 -- 172.21.15.40:0/154826224 shutdown_connections 2024-04-26T21:47:28.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.923+0000 7f8d0f119640 1 --2- 172.21.15.40:0/154826224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf00043b0 0x7f8cf0004880 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.923+0000 7f8d0f119640 1 -- 172.21.15.40:0/154826224 >> 172.21.15.40:0/154826224 conn(0x7f8d080d4fd0 msgr2=0x7f8d080d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.923+0000 7f8d0f119640 1 -- 172.21.15.40:0/154826224 shutdown_connections 2024-04-26T21:47:28.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.923+0000 7f8d0f119640 1 -- 172.21.15.40:0/154826224 wait complete. 2024-04-26T21:47:28.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.924+0000 7f8d0f119640 1 Processor -- start 2024-04-26T21:47:28.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.924+0000 7f8d0f119640 1 -- start start 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0f119640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00043b0 0x7f8d080ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0f119640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d080b8920 0x7f8d080b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0f119640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d080ca970 0x7f8d080cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0f119640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d08162770 con 0x7f8d080b8920 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0f119640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d081625f0 con 0x7f8cf00043b0 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0f119640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8d081628f0 con 0x7f8d080ca970 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d07fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d080b8920 0x7f8d080b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d07fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d080b8920 0x7f8d080b9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56174/0 (socket says 172.21.15.40:56174) 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d07fff640 1 -- 172.21.15.40:0/245146142 learned_addr learned my addr 172.21.15.40:0/245146142 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0ceb5640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00043b0 0x7f8d080ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0d6b6640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d080ca970 0x7f8d080cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d07fff640 1 -- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d080ca970 msgr2=0x7f8d080cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d07fff640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d080ca970 0x7f8d080cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d07fff640 1 -- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00043b0 msgr2=0x7f8d080ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d07fff640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00043b0 0x7f8d080ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d07fff640 1 -- 172.21.15.40:0/245146142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d081f5500 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.925+0000 7f8d0d6b6640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d080ca970 0x7f8d080cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.926+0000 7f8d07fff640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d080b8920 0x7f8d080b9ce0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc02abb0 tx=0x7f8cfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.926+0000 7f8d057fa640 1 -- 172.21.15.40:0/245146142 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc03f020 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.926+0000 7f8d057fa640 1 -- 172.21.15.40:0/245146142 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc037040 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.926+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d081f57e0 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.926+0000 7f8d057fa640 1 -- 172.21.15.40:0/245146142 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc036070 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.926+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d080ba220 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.927+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d080694f0 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.927+0000 7f8d057fa640 1 -- 172.21.15.40:0/245146142 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cfc042020 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.928+0000 7f8d057fa640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8cf406c920 0x7f8cf406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.928+0000 7f8d057fa640 1 -- 172.21.15.40:0/245146142 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f8cfc0abda0 con 0x7f8d080b8920 2024-04-26T21:47:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.928+0000 7f8d0ceb5640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8cf406c920 0x7f8cf406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:28.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.929+0000 7f8d0ceb5640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8cf406c920 0x7f8cf406ede0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8d08052070 tx=0x7f8d00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:28.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:28.932+0000 7f8d057fa640 1 -- 172.21.15.40:0/245146142 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8cfc049030 con 0x7f8d080b8920 2024-04-26T21:47:29.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.118+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d0806d4c0 con 0x7f8d080b8920 2024-04-26T21:47:29.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.118+0000 7f8d057fa640 1 -- 172.21.15.40:0/245146142 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7f8cfc07a030 con 0x7f8d080b8920 2024-04-26T21:47:29.118 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8cf406c920 msgr2=0x7f8cf406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8cf406c920 0x7f8cf406ede0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8d08052070 tx=0x7f8d00009040 comp rx=0 tx=0).stop 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d080b8920 msgr2=0x7f8d080b9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d080b8920 0x7f8d080b9ce0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc02abb0 tx=0x7f8cfc004730 comp rx=0 tx=0).stop 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 shutdown_connections 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f8cf406c920 0x7f8cf406ede0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d080ca970 0x7f8d080cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d080b8920 0x7f8d080b9ce0 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 --2- 172.21.15.40:0/245146142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf00043b0 0x7f8d080ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 >> 172.21.15.40:0/245146142 conn(0x7f8d080d4fd0 msgr2=0x7f8d080cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 shutdown_connections 2024-04-26T21:47:29.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.121+0000 7f8d0f119640 1 -- 172.21.15.40:0/245146142 wait complete. 2024-04-26T21:47:29.132 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:47:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.234+0000 7f90ec2ea640 1 Processor -- start 2024-04-26T21:47:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.234+0000 7f90ec2ea640 1 -- start start 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ec2ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90e4162880 0x7f90e40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ec2ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40b5f20 0x7f90e40d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ec2ea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e40b7e50 con 0x7f90e40d39f0 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ec2ea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f90e40d0580 con 0x7f90e4162880 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ec2ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90e40d1800 con 0x7f90e40b5f20 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ea086640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90e4162880 0x7f90e40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ea887640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f90e40d39f0 0x7f90e40b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53014/0 (socket says 172.21.15.40:53014) 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ea086640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90e4162880 0x7f90e40d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56178/0 (socket says 172.21.15.40:56178) 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ea887640 1 -- 172.21.15.40:0/702212222 learned_addr learned my addr 172.21.15.40:0/702212222 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90e9885640 1 --2- 172.21.15.40:0/702212222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40b5f20 0x7f90e40d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ea086640 1 -- 172.21.15.40:0/702212222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40b5f20 msgr2=0x7f90e40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ea086640 1 --2- 172.21.15.40:0/702212222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40b5f20 0x7f90e40d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ea086640 1 -- 172.21.15.40:0/702212222 >> v1:172.21.15.40:6789/0 conn(0x7f90e40d39f0 legacy=0x7f90e40b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90ea086640 1 -- 172.21.15.40:0/702212222 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e40d07d0 con 0x7f90e4162880 2024-04-26T21:47:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.235+0000 7f90e9885640 1 --2- 172.21.15.40:0/702212222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40b5f20 0x7f90e40d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.236+0000 7f90ea086640 1 --2- 172.21.15.40:0/702212222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e4162880 0x7f90e40d2cb0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f90e00097a0 tx=0x7f90e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50343c97ea47b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.240+0000 7f90e9084640 1 -- 172.21.15.40:0/702212222 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0037070 con 0x7f90e4162880 2024-04-26T21:47:29.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90e9084640 1 -- 172.21.15.40:0/702212222 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e0032440 con 0x7f90e4162880 2024-04-26T21:47:29.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90e9084640 1 -- 172.21.15.40:0/702212222 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0032810 con 0x7f90e4162880 2024-04-26T21:47:29.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90ec2ea640 1 -- 172.21.15.40:0/702212222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e4162880 msgr2=0x7f90e40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/702212222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e4162880 0x7f90e40d2cb0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f90e00097a0 tx=0x7f90e002a180 comp rx=0 tx=0).stop 2024-04-26T21:47:29.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90ec2ea640 1 -- 172.21.15.40:0/702212222 shutdown_connections 2024-04-26T21:47:29.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/702212222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40b5f20 0x7f90e40d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/702212222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e4162880 0x7f90e40d2cb0 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90ec2ea640 1 -- 172.21.15.40:0/702212222 >> 172.21.15.40:0/702212222 conn(0x7f90e40d4f60 msgr2=0x7f90e40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:29.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90ec2ea640 1 -- 172.21.15.40:0/702212222 shutdown_connections 2024-04-26T21:47:29.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.241+0000 7f90ec2ea640 1 -- 172.21.15.40:0/702212222 wait complete. 2024-04-26T21:47:29.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ec2ea640 1 Processor -- start 2024-04-26T21:47:29.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ec2ea640 1 -- start start 2024-04-26T21:47:29.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ec2ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e40b5f20 0x7f90e40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ec2ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40d39f0 0x7f90e4153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ec2ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e4162880 0x7f90e40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ec2ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90e4162c60 con 0x7f90e40b5f20 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ec2ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90e40d0d90 con 0x7f90e4162880 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ec2ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90e4162de0 con 0x7f90e40d39f0 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ea086640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e40b5f20 0x7f90e40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ea086640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e40b5f20 0x7f90e40cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56186/0 (socket says 172.21.15.40:56186) 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.243+0000 7f90ea086640 1 -- 172.21.15.40:0/2022804304 learned_addr learned my addr 172.21.15.40:0/2022804304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90ea086640 1 -- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40d39f0 msgr2=0x7f90e4153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90e9885640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40d39f0 0x7f90e4153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90ea086640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40d39f0 0x7f90e4153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90ea086640 1 -- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e4162880 msgr2=0x7f90e40cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90ea086640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e4162880 0x7f90e40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90ea086640 1 -- 172.21.15.40:0/2022804304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e415cc40 con 0x7f90e40b5f20 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90ea086640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e40b5f20 0x7f90e40cdc70 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f90e002a1c0 tx=0x7f90e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90daffd640 1 -- 172.21.15.40:0/2022804304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0037040 con 0x7f90e40b5f20 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90daffd640 1 -- 172.21.15.40:0/2022804304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e0042070 con 0x7f90e40b5f20 2024-04-26T21:47:29.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e40ce270 con 0x7f90e40b5f20 2024-04-26T21:47:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90daffd640 1 -- 172.21.15.40:0/2022804304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0030bb0 con 0x7f90e40b5f20 2024-04-26T21:47:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.244+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e40cd050 con 0x7f90e40b5f20 2024-04-26T21:47:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.245+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e40764d0 con 0x7f90e40b5f20 2024-04-26T21:47:29.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.246+0000 7f90daffd640 1 -- 172.21.15.40:0/2022804304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90e0047460 con 0x7f90e40b5f20 2024-04-26T21:47:29.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.246+0000 7f90daffd640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90c806c8f0 0x7f90c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.246+0000 7f90daffd640 1 -- 172.21.15.40:0/2022804304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f90e00ac890 con 0x7f90e40b5f20 2024-04-26T21:47:29.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.246+0000 7f90e9885640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90c806c8f0 0x7f90c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.247+0000 7f90e9885640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90c806c8f0 0x7f90c806edb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f90e40d1780 tx=0x7f90d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.250+0000 7f90daffd640 1 -- 172.21.15.40:0/2022804304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90e40764d0 con 0x7f90e40b5f20 2024-04-26T21:47:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.446+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90e4161990 con 0x7f90e40b5f20 2024-04-26T21:47:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.446+0000 7f90daffd640 1 -- 172.21.15.40:0/2022804304 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f90e007ab30 con 0x7f90e40b5f20 2024-04-26T21:47:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90c806c8f0 msgr2=0x7f90c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90c806c8f0 0x7f90c806edb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f90e40d1780 tx=0x7f90d4008040 comp rx=0 tx=0).stop 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e40b5f20 msgr2=0x7f90e40cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e40b5f20 0x7f90e40cdc70 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f90e002a1c0 tx=0x7f90e00046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 shutdown_connections 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f90c806c8f0 0x7f90c806edb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e4162880 0x7f90e40cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e40d39f0 0x7f90e4153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 --2- 172.21.15.40:0/2022804304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e40b5f20 0x7f90e40cdc70 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 >> 172.21.15.40:0/2022804304 conn(0x7f90e40d4f60 msgr2=0x7f90e40b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:29.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 shutdown_connections 2024-04-26T21:47:29.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.448+0000 7f90ec2ea640 1 -- 172.21.15.40:0/2022804304 wait complete. 2024-04-26T21:47:29.458 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:47:29.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:29.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:29.506 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:47:29.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:29.506 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:47:29.535 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:29.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.675+0000 7fad3fc6d640 1 Processor -- start 2024-04-26T21:47:29.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.675+0000 7fad3fc6d640 1 -- start start 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3fc6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fad380b0380 0x7fad380ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3fc6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad380b10b0 0x7fad3806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3fc6d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad380cfd80 con 0x7fad3806c780 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3fc6d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fad380af8e0 con 0x7fad380b0380 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3fc6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fad380d0020 con 0x7fad380b10b0 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3da09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fad380b0380 0x7fad380ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3da09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fad380b0380 0x7fad380ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56194/0 (socket says 172.21.15.40:56194) 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3da09640 1 -- 172.21.15.40:0/3880385826 learned_addr learned my addr 172.21.15.40:0/3880385826 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3d208640 1 --2- 172.21.15.40:0/3880385826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad380b10b0 0x7fad3806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3da09640 1 -- 172.21.15.40:0/3880385826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad380b10b0 msgr2=0x7fad3806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3da09640 1 --2- 172.21.15.40:0/3880385826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad380b10b0 0x7fad3806c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3da09640 1 -- 172.21.15.40:0/3880385826 >> v1:172.21.15.184:6789/0 conn(0x7fad3806c780 legacy=0x7fad38072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.676+0000 7fad3da09640 1 -- 172.21.15.40:0/3880385826 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad380d0300 con 0x7fad380b0380 2024-04-26T21:47:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.677+0000 7fad3da09640 1 --2- 172.21.15.40:0/3880385826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b0380 0x7fad380ccd50 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7fad34009870 tx=0x7fad3402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3ad428ad93c57e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.677+0000 7fad3ca07640 1 -- 172.21.15.40:0/3880385826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34037070 con 0x7fad380b0380 2024-04-26T21:47:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.677+0000 7fad3ca07640 1 -- 172.21.15.40:0/3880385826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad34032440 con 0x7fad380b0380 2024-04-26T21:47:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.677+0000 7fad3ca07640 1 -- 172.21.15.40:0/3880385826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34032810 con 0x7fad380b0380 2024-04-26T21:47:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.678+0000 7fad3fc6d640 1 -- 172.21.15.40:0/3880385826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b0380 msgr2=0x7fad380ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.678+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/3880385826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b0380 0x7fad380ccd50 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7fad34009870 tx=0x7fad3402a180 comp rx=0 tx=0).stop 2024-04-26T21:47:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.678+0000 7fad3fc6d640 1 -- 172.21.15.40:0/3880385826 shutdown_connections 2024-04-26T21:47:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.678+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/3880385826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad380b10b0 0x7fad3806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.678+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/3880385826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b0380 0x7fad380ccd50 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.678+0000 7fad3fc6d640 1 -- 172.21.15.40:0/3880385826 >> 172.21.15.40:0/3880385826 conn(0x7fad380d1cf0 msgr2=0x7fad380d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.678+0000 7fad3fc6d640 1 -- 172.21.15.40:0/3880385826 shutdown_connections 2024-04-26T21:47:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.678+0000 7fad3fc6d640 1 -- 172.21.15.40:0/3880385826 wait complete. 2024-04-26T21:47:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.679+0000 7fad3fc6d640 1 Processor -- start 2024-04-26T21:47:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.679+0000 7fad3fc6d640 1 -- start start 2024-04-26T21:47:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3fc6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad3806c780 0x7fad381515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3fc6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad380b0380 0x7fad380c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3fc6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b10b0 0x7fad380c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3fc6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad380b0760 con 0x7fad380b10b0 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3fc6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad380d1980 con 0x7fad380b0380 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3fc6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fad380d15c0 con 0x7fad3806c780 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b10b0 0x7fad380c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b10b0 0x7fad380c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56196/0 (socket says 172.21.15.40:56196) 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 -- 172.21.15.40:0/2590047798 learned_addr learned my addr 172.21.15.40:0/2590047798 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 -- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad3806c780 msgr2=0x7fad381515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3da09640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad3806c780 0x7fad381515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3d208640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad380b0380 0x7fad380c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad3806c780 0x7fad381515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 -- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad380b0380 msgr2=0x7fad380c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad380b0380 0x7fad380c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 -- 172.21.15.40:0/2590047798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad381500a0 con 0x7fad380b10b0 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3da09640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad3806c780 0x7fad381515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3d208640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad380b0380 0x7fad380c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3e20a640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b10b0 0x7fad380c7a80 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fad3000c780 tx=0x7fad3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad2a7fc640 1 -- 172.21.15.40:0/2590047798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30004b60 con 0x7fad380b10b0 2024-04-26T21:47:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad2a7fc640 1 -- 172.21.15.40:0/2590047798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad30004d00 con 0x7fad380b10b0 2024-04-26T21:47:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad2a7fc640 1 -- 172.21.15.40:0/2590047798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3000aa90 con 0x7fad380b10b0 2024-04-26T21:47:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.680+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad38150380 con 0x7fad380b10b0 2024-04-26T21:47:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.681+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad38150940 con 0x7fad380b10b0 2024-04-26T21:47:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.681+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad380694f0 con 0x7fad380b10b0 2024-04-26T21:47:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.682+0000 7fad2a7fc640 1 -- 172.21.15.40:0/2590047798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad3000ac30 con 0x7fad380b10b0 2024-04-26T21:47:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.682+0000 7fad2a7fc640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1c06c920 0x7fad1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.683+0000 7fad2a7fc640 1 -- 172.21.15.40:0/2590047798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7fad3008bf60 con 0x7fad380b10b0 2024-04-26T21:47:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.683+0000 7fad3da09640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1c06c920 0x7fad1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.683+0000 7fad3da09640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1c06c920 0x7fad1c06ede0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fad34034000 tx=0x7fad3402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:29.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.687+0000 7fad2a7fc640 1 -- 172.21.15.40:0/2590047798 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad30020070 con 0x7fad380b10b0 2024-04-26T21:47:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.908+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad380cd270 con 0x7fad380b10b0 2024-04-26T21:47:29.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.909+0000 7fad2a7fc640 1 -- 172.21.15.40:0/2590047798 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 357 v357) ==== 95+0+5201 (secure 0 0 0) 0x7fad3005a200 con 0x7fad380b10b0 2024-04-26T21:47:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:29.909 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":357,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":357,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:22.462625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11757":{"gid":11757,"name":"a","rank":0,"incarnation":355,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3441996159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3441996159},{"type":"v1","addr":"172.21.15.40:6835","nonce":3441996159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11757,"qdb_cluster":[11757]},"id":7}]} 2024-04-26T21:47:29.909 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 357 2024-04-26T21:47:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1c06c920 msgr2=0x7fad1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1c06c920 0x7fad1c06ede0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fad34034000 tx=0x7fad3402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b10b0 msgr2=0x7fad380c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b10b0 0x7fad380c7a80 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fad3000c780 tx=0x7fad3000cc40 comp rx=0 tx=0).stop 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 shutdown_connections 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fad1c06c920 0x7fad1c06ede0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad380b10b0 0x7fad380c7a80 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad380b0380 0x7fad380c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 --2- 172.21.15.40:0/2590047798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad3806c780 0x7fad381515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 >> 172.21.15.40:0/2590047798 conn(0x7fad380d1cf0 msgr2=0x7fad380b6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 shutdown_connections 2024-04-26T21:47:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:29.911+0000 7fad3fc6d640 1 -- 172.21.15.40:0/2590047798 wait complete. 2024-04-26T21:47:29.921 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-26T21:47:30.051 INFO:teuthology.orchestra.run.smithi040.stdout:{"message":"","return_code":0} 2024-04-26T21:47:30.051 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:47:30.051 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:47:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.153+0000 7f75ec62a640 1 Processor -- start 2024-04-26T21:47:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.153+0000 7f75ec62a640 1 -- start start 2024-04-26T21:47:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.154+0000 7f75ec62a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b0380 0x7f75e40ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.154+0000 7f75ec62a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f75e406c950 0x7f75e415e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.154+0000 7f75ec62a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e40cfd80 con 0x7f75e40b10b0 2024-04-26T21:47:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.154+0000 7f75ec62a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f75e40af8e0 con 0x7f75e406c950 2024-04-26T21:47:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.154+0000 7f75ec62a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75e40d0020 con 0x7f75e40b0380 2024-04-26T21:47:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75eabc7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f75e406c950 0x7f75e415e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75ea3c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b0380 0x7f75e40ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75e9bc5640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f75e40b10b0 0x7f75e406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:36366/0 (socket says 172.21.15.40:36366) 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75eabc7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f75e406c950 0x7f75e415e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:44794/0 (socket says 172.21.15.40:44794) 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75ea3c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b0380 0x7f75e40ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:38650/0 (socket says 172.21.15.40:38650) 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75e9bc5640 1 -- 172.21.15.40:0/1406113839 learned_addr learned my addr 172.21.15.40:0/1406113839 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75ea3c6640 1 -- 172.21.15.40:0/1406113839 >> v2:172.21.15.184:3300/0 conn(0x7f75e406c950 msgr2=0x7f75e415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75ea3c6640 1 --2- 172.21.15.40:0/1406113839 >> v2:172.21.15.184:3300/0 conn(0x7f75e406c950 0x7f75e415e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75ea3c6640 1 -- 172.21.15.40:0/1406113839 >> v1:172.21.15.184:6789/0 conn(0x7f75e40b10b0 legacy=0x7f75e406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75e93c4640 1 -- 172.21.15.40:0/1406113839 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117117068 0 0) 0x7f75e40cfd80 con 0x7f75e40b10b0 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75ea3c6640 1 -- 172.21.15.40:0/1406113839 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e40d0300 con 0x7f75e40b0380 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.155+0000 7f75eabc7640 1 --2- 172.21.15.40:0/1406113839 >> v2:172.21.15.184:3300/0 conn(0x7f75e406c950 0x7f75e415e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.156+0000 7f75ea3c6640 1 --2- 172.21.15.40:0/1406113839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b0380 0x7f75e40ccd50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f75e40afa60 tx=0x7f75dc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c689470fa5c91e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.156+0000 7f75e93c4640 1 -- 172.21.15.40:0/1406113839 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc037070 con 0x7f75e40b0380 2024-04-26T21:47:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.156+0000 7f75e93c4640 1 -- 172.21.15.40:0/1406113839 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc02ae50 con 0x7f75e40b0380 2024-04-26T21:47:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.156+0000 7f75e93c4640 1 -- 172.21.15.40:0/1406113839 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc032660 con 0x7f75e40b0380 2024-04-26T21:47:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.156+0000 7f75ec62a640 1 -- 172.21.15.40:0/1406113839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b0380 msgr2=0x7f75e40ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.156+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1406113839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b0380 0x7f75e40ccd50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f75e40afa60 tx=0x7f75dc02a100 comp rx=0 tx=0).stop 2024-04-26T21:47:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.157+0000 7f75ec62a640 1 -- 172.21.15.40:0/1406113839 shutdown_connections 2024-04-26T21:47:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.157+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1406113839 >> v2:172.21.15.184:3300/0 conn(0x7f75e406c950 0x7f75e415e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.157+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1406113839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b0380 0x7f75e40ccd50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.157+0000 7f75ec62a640 1 -- 172.21.15.40:0/1406113839 >> 172.21.15.40:0/1406113839 conn(0x7f75e40d1cf0 msgr2=0x7f75e40d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.157+0000 7f75ec62a640 1 -- 172.21.15.40:0/1406113839 shutdown_connections 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.157+0000 7f75ec62a640 1 -- 172.21.15.40:0/1406113839 wait complete. 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.158+0000 7f75ec62a640 1 Processor -- start 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.158+0000 7f75ec62a640 1 -- start start 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.158+0000 7f75ec62a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e406c950 0x7f75e40c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.158+0000 7f75ec62a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75e40b0380 0x7f75e4153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.158+0000 7f75ec62a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b10b0 0x7f75e41543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.158+0000 7f75ec62a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40b0760 con 0x7f75e406c950 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.158+0000 7f75ec62a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40d1980 con 0x7f75e40b0380 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.158+0000 7f75ec62a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75e40d15c0 con 0x7f75e40b10b0 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e406c950 0x7f75e40c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e406c950 0x7f75e40c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56212/0 (socket says 172.21.15.40:56212) 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 -- 172.21.15.40:0/1250830780 learned_addr learned my addr 172.21.15.40:0/1250830780 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75e9bc5640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75e40b0380 0x7f75e4153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75eabc7640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b10b0 0x7f75e41543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 -- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b10b0 msgr2=0x7f75e41543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b10b0 0x7f75e41543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 -- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75e40b0380 msgr2=0x7f75e4153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75e40b0380 0x7f75e4153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 -- 172.21.15.40:0/1250830780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e40b6380 con 0x7f75e406c950 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ea3c6640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e406c950 0x7f75e40c6400 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f75dc000c00 tx=0x7f75dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75daffd640 1 -- 172.21.15.40:0/1250830780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc037040 con 0x7f75e406c950 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75daffd640 1 -- 172.21.15.40:0/1250830780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc004030 con 0x7f75e406c950 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75daffd640 1 -- 172.21.15.40:0/1250830780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc004420 con 0x7f75e406c950 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.159+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e40b6660 con 0x7f75e406c950 2024-04-26T21:47:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.160+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e40b6c20 con 0x7f75e406c950 2024-04-26T21:47:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.160+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e40694f0 con 0x7f75e406c950 2024-04-26T21:47:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.161+0000 7f75daffd640 1 -- 172.21.15.40:0/1250830780 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75dc003610 con 0x7f75e406c950 2024-04-26T21:47:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.161+0000 7f75daffd640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.161+0000 7f75daffd640 1 -- 172.21.15.40:0/1250830780 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f75dc0ac970 con 0x7f75e406c950 2024-04-26T21:47:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.162+0000 7f75e9bc5640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.162+0000 7f75e9bc5640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f75e40ccf00 tx=0x7f75d4007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.166+0000 7f75daffd640 1 -- 172.21.15.40:0/1250830780 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75dc07acb0 con 0x7f75e406c950 2024-04-26T21:47:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.390+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f75e406c4e0 con 0x7f75e406c950 2024-04-26T21:47:30.390 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:47:30.391+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:47:30.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.394+0000 7f75daffd640 1 -- 172.21.15.40:0/1250830780 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v358) ==== 161+0+0 (secure 0 0 0) 0x7f75dc07a5b0 con 0x7f75e406c950 2024-04-26T21:47:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 msgr2=0x7f75c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f75e40ccf00 tx=0x7f75d4007de0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e406c950 msgr2=0x7f75e40c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e406c950 0x7f75e40c6400 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f75dc000c00 tx=0x7f75dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 shutdown_connections 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f75c806c8f0 0x7f75c806edb0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75e40b10b0 0x7f75e41543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75e40b0380 0x7f75e4153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 --2- 172.21.15.40:0/1250830780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e406c950 0x7f75e40c6400 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 >> 172.21.15.40:0/1250830780 conn(0x7f75e40d1cf0 msgr2=0x7f75e40b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:30.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 shutdown_connections 2024-04-26T21:47:30.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.398+0000 7f75ec62a640 1 -- 172.21.15.40:0/1250830780 wait complete. 2024-04-26T21:47:30.399 INFO:tasks.ceph.mds.a.smithi040.stderr: -6397> 2024-04-26T21:47:23.072+0000 7f4089b7d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:47:30.409 DEBUG:teuthology.orchestra.run.smithi040:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-26T21:47:30.430 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:47:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.451+0000 7fdc03e0e280 1 -- start start 2024-04-26T21:47:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.451+0000 7fdc03e0e280 1 Processor -- start 2024-04-26T21:47:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a800 0x55b7d7f33080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7159c00 0x55b7d7f32b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d7f4bc00 0x55b7d7f32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7f44000 con 0x55b7d7159c00 2024-04-26T21:47:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7f44180 con 0x55b7d7f4a800 2024-04-26T21:47:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b7d7f44300 con 0x55b7d7f4bc00 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc010a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7159c00 0x55b7d7f32b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc010a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7159c00 0x55b7d7f32b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56248/0 (socket says 172.21.15.40:56248) 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc010a9640 1 -- 172.21.15.40:0/60415636 learned_addr learned my addr 172.21.15.40:0/60415636 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc020ab640 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d7f4bc00 0x55b7d7f32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.452+0000 7fdc018aa640 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a800 0x55b7d7f33080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc010a9640 1 -- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d7f4bc00 msgr2=0x55b7d7f32580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc010a9640 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d7f4bc00 0x55b7d7f32580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc010a9640 1 -- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a800 msgr2=0x55b7d7f33080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc010a9640 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a800 0x55b7d7f33080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc010a9640 1 -- 172.21.15.40:0/60415636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d718e780 con 0x55b7d7159c00 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc020ab640 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d7f4bc00 0x55b7d7f32580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc018aa640 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a800 0x55b7d7f33080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc010a9640 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7159c00 0x55b7d7f32b00 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x55b7d7f91680 tx=0x55b7d7ef9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d6a427d20fb7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdbff0a5640 1 -- 172.21.15.40:0/60415636 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b7d717ed00 con 0x55b7d7159c00 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdbff0a5640 1 -- 172.21.15.40:0/60415636 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b7d7184e00 con 0x55b7d7159c00 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdbff0a5640 1 -- 172.21.15.40:0/60415636 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b7d717e680 con 0x55b7d7159c00 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc03e0e280 1 -- 172.21.15.40:0/60415636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d718f4a0 con 0x55b7d7159c00 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc03e0e280 1 -- 172.21.15.40:0/60415636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b7d718fa40 con 0x55b7d7159c00 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc03e0e280 4 waiting for MDS map... 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdbff0a5640 1 -- 172.21.15.40:0/60415636 <== mon.0 v2:172.21.15.40:3300/0 4 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x55b7d803a800 con 0x55b7d7159c00 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdbff0a5640 1 -- 172.21.15.40:0/60415636 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 358) ==== 2885+0+0 (secure 0 0 0) 0x55b7d71b3180 con 0x55b7d7159c00 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc03e0e280 4 Got MDS map 358 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc03e0e280 10 main: JournalTool::main 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.453+0000 7fdc03e0e280 4 main: JournalTool: connecting to RADOS... 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.454+0000 7fdc03e0e280 1 Processor -- start 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.454+0000 7fdc03e0e280 1 -- start start 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.454+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.454+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d803d800 0x55b7d7f33b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.454+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803dc00 0x55b7d7f34100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.454+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7f44480 con 0x55b7d7f4a000 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.454+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7f44600 con 0x55b7d803d800 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.454+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b7d7f44780 con 0x55b7d803dc00 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56262/0 (socket says 172.21.15.40:56262) 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 -- 172.21.15.40:0/1131559261 learned_addr learned my addr 172.21.15.40:0/1131559261 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc018aa640 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d803d800 0x55b7d7f33b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc010a9640 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803dc00 0x55b7d7f34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 -- 172.21.15.40:0/1131559261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803dc00 msgr2=0x55b7d7f34100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803dc00 0x55b7d7f34100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 -- 172.21.15.40:0/1131559261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d803d800 msgr2=0x55b7d7f33b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d803d800 0x55b7d7f33b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 -- 172.21.15.40:0/1131559261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d718fc20 con 0x55b7d7f4a000 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x55b7d7191320 tx=0x55b7d7ef9320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48058ba528111d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdbfd0a1640 1 -- 172.21.15.40:0/1131559261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b7d717eb60 con 0x55b7d7f4a000 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdbfd0a1640 1 -- 172.21.15.40:0/1131559261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b7d7184c40 con 0x55b7d7f4a000 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdbfd0a1640 1 -- 172.21.15.40:0/1131559261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b7d717ed00 con 0x55b7d7f4a000 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1131559261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7f4a000 msgr2=0x55b7d7f33600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.455+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x55b7d7191320 tx=0x55b7d7ef9320 comp rx=0 tx=0).stop 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1131559261 shutdown_connections 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803dc00 0x55b7d7f34100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d803d800 0x55b7d7f33b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1131559261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1131559261 >> 172.21.15.40:0/1131559261 conn(0x55b7d7f4a400 msgr2=0x55b7d7263800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:30.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1131559261 shutdown_connections 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1131559261 wait complete. 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 Processor -- start 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.456+0000 7fdc03e0e280 1 -- start start 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803d800 0x55b7d7f33b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d803dc00 0x55b7d7f34100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7f45c80 con 0x55b7d803dc00 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7f45b00 con 0x55b7d7f4a000 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b7d7f45980 con 0x55b7d803d800 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d803dc00 0x55b7d7f34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d803dc00 0x55b7d7f34100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56264/0 (socket says 172.21.15.40:56264) 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 -- 172.21.15.40:0/3080834754 learned_addr learned my addr 172.21.15.40:0/3080834754 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc020ab640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc018aa640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803d800 0x55b7d7f33b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 -- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803d800 msgr2=0x55b7d7f33b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803d800 0x55b7d7f33b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 -- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a000 msgr2=0x55b7d7f33600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 -- 172.21.15.40:0/3080834754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d718f860 con 0x55b7d803dc00 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc020ab640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc010a9640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d803dc00 0x55b7d7f34100 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x55b7d7f91dd0 tx=0x55b7d7ef69c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdbfb89e640 1 -- 172.21.15.40:0/3080834754 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b7d717eb60 con 0x55b7d803dc00 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdbfb89e640 1 -- 172.21.15.40:0/3080834754 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b7d7184a80 con 0x55b7d803dc00 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 1 -- 172.21.15.40:0/3080834754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7d718f680 con 0x55b7d803dc00 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdbfb89e640 1 -- 172.21.15.40:0/3080834754 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b7d717ed00 con 0x55b7d803dc00 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 1 -- 172.21.15.40:0/3080834754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d718e3c0 con 0x55b7d803dc00 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.457+0000 7fdc03e0e280 4 main: JournalTool: resolving pool 15 2024-04-26T21:47:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.459+0000 7fdbfb89e640 1 -- 172.21.15.40:0/3080834754 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55b7d80acc00 con 0x55b7d803dc00 2024-04-26T21:47:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.459+0000 7fdbfb89e640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x55b7d80b1000 0x55b7d7f34680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.459+0000 7fdbfb89e640 1 -- 172.21.15.40:0/3080834754 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x55b7d80aa800 con 0x55b7d803dc00 2024-04-26T21:47:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.459+0000 7fdc03e0e280 4 main: JournalTool: creating IoCtx.. 2024-04-26T21:47:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.459+0000 7fdc03e0e280 4 main: Executing for rank 0 2024-04-26T21:47:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.459+0000 7fdc020ab640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x55b7d80b1000 0x55b7d7f34680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc03e0e280 1 -- start start 2024-04-26T21:47:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc03e0e280 1 Processor -- start 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc020ab640 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x55b7d80b1000 0x55b7d7f34680 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x55b7d7191320 tx=0x55b7d7ef9200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d80b1800 0x55b7d7f34c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d80b1c00 0x55b7d7f35180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc03e0e280 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7540000 0x55b7d7f35700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7f45680 con 0x55b7d80b1800 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7f45500 con 0x55b7d7540000 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc03e0e280 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b7d7f44480 con 0x55b7d80b1c00 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc020ab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d80b1800 0x55b7d7f34c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc020ab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d80b1800 0x55b7d7f34c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56268/0 (socket says 172.21.15.40:56268) 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.460+0000 7fdc020ab640 1 -- 172.21.15.40:0/1891897363 learned_addr learned my addr 172.21.15.40:0/1891897363 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc018aa640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d80b1c00 0x55b7d7f35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc010a9640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7540000 0x55b7d7f35700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc020ab640 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d80b1c00 msgr2=0x55b7d7f35180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d80b1c00 0x55b7d7f35180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc020ab640 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7540000 msgr2=0x55b7d7f35700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7540000 0x55b7d7f35700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc020ab640 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d718fc20 con 0x55b7d80b1800 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d80b1800 0x55b7d7f34c00 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x55b7d8056cf0 tx=0x55b7d7ef93e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f06260a3d42a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdbf8898640 1 -- 172.21.15.40:0/1891897363 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b7d717e680 con 0x55b7d80b1800 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdbf8898640 1 -- 172.21.15.40:0/1891897363 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b7d71841c0 con 0x55b7d80b1800 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdbf8898640 1 -- 172.21.15.40:0/1891897363 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b7d717e820 con 0x55b7d80b1800 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d718e000 con 0x55b7d80b1800 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b7d718e960 con 0x55b7d80b1800 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc03e0e280 4 waiting for MDS map... 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdbf8898640 1 -- 172.21.15.40:0/1891897363 <== mon.0 v2:172.21.15.40:3300/0 4 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x55b7d7548200 con 0x55b7d80b1800 2024-04-26T21:47:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdbf8898640 1 -- 172.21.15.40:0/1891897363 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 358) ==== 2885+0+0 (secure 0 0 0) 0x55b7d71b3500 con 0x55b7d80b1800 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc03e0e280 4 Got MDS map 358 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b7d7540400 0x55b7d755a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.461+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7540800 con 0x55b7d7540400 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.462+0000 7fdc018aa640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b7d7540400 0x55b7d755a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.462+0000 7fdc018aa640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b7d7540400 0x55b7d755a000 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=c2817b507b8c771 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.465+0000 7fdc018aa640 1 -- 172.21.15.40:0/1891897363 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55b7d7ee6000 con 0x55b7d7540400 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.465+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7540c00 con 0x55b7d7540400 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.465+0000 7fdc018aa640 1 -- 172.21.15.40:0/1891897363 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv22 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55b7d7ee6000 con 0x55b7d7540400 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.466+0000 7fdbf7095640 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7540800 con 0x55b7d7540400 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.466+0000 7fdbf7095640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b7d7540c00 0x55b7d755bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.466+0000 7fdbf7095640 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7566000 con 0x55b7d7540c00 2024-04-26T21:47:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.466+0000 7fdc010a9640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b7d7540c00 0x55b7d755bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.466+0000 7fdc018aa640 1 -- 172.21.15.40:0/1891897363 <== osd.6 v2:172.21.15.184:6824/533952038 3 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x55b7d7ee6000 con 0x55b7d7540400 2024-04-26T21:47:30.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.466+0000 7fdc010a9640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b7d7540c00 0x55b7d755bb80 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=ff4054bfb3fbdfc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.468+0000 7fdc010a9640 1 -- 172.21.15.40:0/1891897363 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6500 con 0x55b7d7540c00 2024-04-26T21:47:30.473 INFO:teuthology.orchestra.run.smithi040.stdout:old journal was 4226629~2030 2024-04-26T21:47:30.473 INFO:teuthology.orchestra.run.smithi040.stdout:new journal start will be 8388608 (4159949 bytes past old end) 2024-04-26T21:47:30.473 INFO:teuthology.orchestra.run.smithi040.stdout:writing journal head 2024-04-26T21:47:30.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.468+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- osd_op(unknown.0.0:5 15.14 15:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7541000 con 0x55b7d7540400 2024-04-26T21:47:30.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc018aa640 1 -- 172.21.15.40:0/1891897363 <== osd.6 v2:172.21.15.184:6824/533952038 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v146'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6000 con 0x55b7d7540400 2024-04-26T21:47:30.474 INFO:teuthology.orchestra.run.smithi040.stdout:writing EResetJournal entry 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:6 15.1b 15:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7541400 con 0x55b7d7540c00 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x55b7d7541800 0x55b7d755a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- osd_op(unknown.0.0:7 15.b 15:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7566c00 con 0x55b7d7541800 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7d7567000 0x55b7d755ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- osd_op(unknown.0.0:8 15.1f 15:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7567400 con 0x55b7d7567000 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b7d7567800 0x55b7d755b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- osd_op(unknown.0.0:9 15.1a 15:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7567c00 con 0x55b7d7567800 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- osd_op(unknown.0.0:10 15.1e 15:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7570000 con 0x55b7d7541800 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b7d7570400 0x55b7d755b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- osd_op(unknown.0.0:11 15.1d 15:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7570800 con 0x55b7d7570400 2024-04-26T21:47:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x55b7d7541800 0x55b7d755a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc018aa640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7d7567000 0x55b7d755ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc010a9640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b7d7567800 0x55b7d755b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.470+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x55b7d7541800 0x55b7d755a580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=b888c4a2d42b0a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.471+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b7d7570400 0x55b7d755b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.471+0000 7fdc018aa640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7d7567000 0x55b7d755ab00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=c9d80e5767ce4b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.471+0000 7fdc010a9640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b7d7567800 0x55b7d755b080 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=3f254c3ad6cafe20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.471+0000 7fdc020ab640 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b7d7570400 0x55b7d755b600 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=8edf45c28d3f0fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.471+0000 7fdc010a9640 1 -- 172.21.15.40:0/1891897363 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== osd_op_reply(6 200.00000002 [delete] v146'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6500 con 0x55b7d7540c00 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.472+0000 7fdc020ab640 1 -- 172.21.15.40:0/1891897363 <== osd.2 v2:172.21.15.40:6811/3280366756 1 ==== osd_op_reply(10 200.00000006 [delete] v146'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6780 con 0x55b7d7541800 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.472+0000 7fdc020ab640 1 -- 172.21.15.40:0/1891897363 <== osd.2 v2:172.21.15.40:6811/3280366756 2 ==== osd_op_reply(7 200.00000003 [delete] v146'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6780 con 0x55b7d7541800 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.472+0000 7fdbf7095640 1 -- 172.21.15.40:0/1891897363 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- osd_op(unknown.0.0:12 15.1b 15:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7d7566400 con 0x55b7d7540c00 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.473+0000 7fdc010a9640 1 -- 172.21.15.40:0/1891897363 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== osd_op_reply(9 200.00000005 [delete] v146'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6500 con 0x55b7d7567800 2024-04-26T21:47:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.474+0000 7fdc018aa640 1 -- 172.21.15.40:0/1891897363 <== osd.3 v2:172.21.15.40:6824/4238362919 1 ==== osd_op_reply(8 200.00000004 [delete] v146'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6000 con 0x55b7d7567000 2024-04-26T21:47:30.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.474+0000 7fdc010a9640 1 -- 172.21.15.40:0/1891897363 <== osd.7 v2:172.21.15.184:6809/2389595522 3 ==== osd_op_reply(12 200.00000002 [write 0~48] v146'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6500 con 0x55b7d7540c00 2024-04-26T21:47:30.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc020ab640 1 -- 172.21.15.40:0/1891897363 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== osd_op_reply(11 200.00000007 [delete] v146'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b7d7ee6780 con 0x55b7d7570400 2024-04-26T21:47:30.479 INFO:teuthology.orchestra.run.smithi040.stdout:done 2024-04-26T21:47:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x55b7d7541800 msgr2=0x55b7d755a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x55b7d7541800 0x55b7d755a580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7d7567000 msgr2=0x55b7d755ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7d7567000 0x55b7d755ab00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b7d7570400 msgr2=0x55b7d755b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x55b7d7570400 0x55b7d755b600 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b7d7567800 msgr2=0x55b7d755b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x55b7d7567800 0x55b7d755b080 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b7d7540400 msgr2=0x55b7d755a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x55b7d7540400 0x55b7d755a000 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b7d7540c00 msgr2=0x55b7d755bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x55b7d7540c00 0x55b7d755bb80 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc018aa640 1 -- 172.21.15.40:0/1891897363 reap_dead start 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d80b1800 msgr2=0x55b7d7f34c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/1891897363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d80b1800 0x55b7d7f34c00 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x55b7d8056cf0 tx=0x55b7d7ef93e0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc018aa640 1 -- 172.21.15.40:0/1891897363 reap_dead start 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc018aa640 1 -- 172.21.15.40:0/1891897363 reap_dead start 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 shutdown_connections 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 >> 172.21.15.40:0/1891897363 conn(0x55b7d80b1400 msgr2=0x55b7d7263000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 shutdown_connections 2024-04-26T21:47:30.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/1891897363 wait complete. 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x55b7d80b1000 msgr2=0x55b7d7f34680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x55b7d80b1000 0x55b7d7f34680 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x55b7d7191320 tx=0x55b7d7ef9200 comp rx=0 tx=0).stop 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/3080834754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d803dc00 msgr2=0x55b7d7f34100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d803dc00 0x55b7d7f34100 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x55b7d7f91dd0 tx=0x55b7d7ef69c0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.475+0000 7fdc03e0e280 1 -- 172.21.15.40:0/3080834754 shutdown_connections 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x55b7d80b1000 0x55b7d7f34680 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d803dc00 0x55b7d7f34100 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d803d800 0x55b7d7f33b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/3080834754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a000 0x55b7d7f33600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 -- 172.21.15.40:0/3080834754 >> 172.21.15.40:0/3080834754 conn(0x55b7d7f4a400 msgr2=0x55b7d7263800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 -- 172.21.15.40:0/3080834754 shutdown_connections 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 -- 172.21.15.40:0/3080834754 wait complete. 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 -- 172.21.15.40:0/60415636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7159c00 msgr2=0x55b7d7f32b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7159c00 0x55b7d7f32b00 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x55b7d7f91680 tx=0x55b7d7ef9500 comp rx=0 tx=0).stop 2024-04-26T21:47:30.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 -- 172.21.15.40:0/60415636 shutdown_connections 2024-04-26T21:47:30.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b7d7f4bc00 0x55b7d7f32580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7d7159c00 0x55b7d7f32b00 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 --2- 172.21.15.40:0/60415636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7d7f4a800 0x55b7d7f33080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 -- 172.21.15.40:0/60415636 >> 172.21.15.40:0/60415636 conn(0x55b7d7159000 msgr2=0x55b7d7262000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:30.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 -- 172.21.15.40:0/60415636 shutdown_connections 2024-04-26T21:47:30.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.476+0000 7fdc03e0e280 1 -- 172.21.15.40:0/60415636 wait complete. 2024-04-26T21:47:30.485 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.075693, result: old journal was 4226629~2030 new journal start will be 8388608 (4159949 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-26T21:47:30.485 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-26T21:47:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.627+0000 7efeafde4640 1 Processor -- start 2024-04-26T21:47:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.627+0000 7efeafde4640 1 -- start start 2024-04-26T21:47:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.628+0000 7efeafde4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efea8070350 0x7efea8070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.628+0000 7efeafde4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efea80d2390 con 0x7efea80cfde0 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.628+0000 7efeafde4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efea806d3f0 con 0x7efea8071090 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.628+0000 7efeafde4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efea80afab0 con 0x7efea8070350 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.628+0000 7efeadb80640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efea80cfde0 0x7efea80d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53028/0 (socket says 172.21.15.40:53028) 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.628+0000 7efeadb80640 1 -- 172.21.15.40:0/2602162059 learned_addr learned my addr 172.21.15.40:0/2602162059 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.628+0000 7efead37f640 1 --2- 172.21.15.40:0/2602162059 >> v2:172.21.15.184:3300/0 conn(0x7efea8070350 0x7efea8070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.629+0000 7efeacb7e640 1 -- 172.21.15.40:0/2602162059 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819145121 0 0) 0x7efea80d2390 con 0x7efea80cfde0 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.629+0000 7efeacb7e640 1 -- 172.21.15.40:0/2602162059 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe94003520 con 0x7efea80cfde0 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.629+0000 7efead37f640 1 -- 172.21.15.40:0/2602162059 >> v1:172.21.15.184:6789/0 conn(0x7efea8071090 legacy=0x7efea815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.629+0000 7efeacb7e640 1 -- 172.21.15.40:0/2602162059 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 34868440 0 0) 0x7efea806d3f0 con 0x7efea8071090 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.629+0000 7efead37f640 1 -- 172.21.15.40:0/2602162059 >> v1:172.21.15.40:6789/0 conn(0x7efea80cfde0 legacy=0x7efea80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.629+0000 7efead37f640 1 -- 172.21.15.40:0/2602162059 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea806d6d0 con 0x7efea8070350 2024-04-26T21:47:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efead37f640 1 --2- 172.21.15.40:0/2602162059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8070350 0x7efea8070b50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7efe980097a0 tx=0x7efe9802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6b8ea18d1638b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efeacb7e640 1 -- 172.21.15.40:0/2602162059 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98037070 con 0x7efea8070350 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efeacb7e640 1 -- 172.21.15.40:0/2602162059 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe98032440 con 0x7efea8070350 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efeacb7e640 1 -- 172.21.15.40:0/2602162059 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980327f0 con 0x7efea8070350 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efeafde4640 1 -- 172.21.15.40:0/2602162059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8070350 msgr2=0x7efea8070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efeafde4640 1 --2- 172.21.15.40:0/2602162059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8070350 0x7efea8070b50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7efe980097a0 tx=0x7efe9802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efeafde4640 1 -- 172.21.15.40:0/2602162059 shutdown_connections 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efeafde4640 1 --2- 172.21.15.40:0/2602162059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8070350 0x7efea8070b50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.630+0000 7efeafde4640 1 -- 172.21.15.40:0/2602162059 >> 172.21.15.40:0/2602162059 conn(0x7efea80d2d80 msgr2=0x7efea80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.631+0000 7efeafde4640 1 -- 172.21.15.40:0/2602162059 shutdown_connections 2024-04-26T21:47:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.631+0000 7efeafde4640 1 -- 172.21.15.40:0/2602162059 wait complete. 2024-04-26T21:47:30.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.632+0000 7efeafde4640 1 Processor -- start 2024-04-26T21:47:30.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.632+0000 7efeafde4640 1 -- start start 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.632+0000 7efeafde4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea8070350 0x7efea809e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.632+0000 7efeafde4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8071090 0x7efea80cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.632+0000 7efeafde4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80cfde0 0x7efea8160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeafde4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efea80714c0 con 0x7efea80cfde0 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeafde4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efea806dd20 con 0x7efea8071090 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeafde4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efea806f1c0 con 0x7efea8070350 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80cfde0 0x7efea8160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80cfde0 0x7efea8160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56276/0 (socket says 172.21.15.40:56276) 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 -- 172.21.15.40:0/3179743680 learned_addr learned my addr 172.21.15.40:0/3179743680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efead37f640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8071090 0x7efea80cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeadb80640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea8070350 0x7efea809e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 -- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea8070350 msgr2=0x7efea809e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea8070350 0x7efea809e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 -- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8071090 msgr2=0x7efea80cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8071090 0x7efea80cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 -- 172.21.15.40:0/3179743680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea809cfe0 con 0x7efea80cfde0 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeae381640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80cfde0 0x7efea8160a10 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7efea000c760 tx=0x7efea000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efe9e7fc640 1 -- 172.21.15.40:0/3179743680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0003da0 con 0x7efea80cfde0 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efe9e7fc640 1 -- 172.21.15.40:0/3179743680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea0005070 con 0x7efea80cfde0 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea809d730 con 0x7efea80cfde0 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efe9e7fc640 1 -- 172.21.15.40:0/3179743680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0005460 con 0x7efea80cfde0 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.633+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea809dcf0 con 0x7efea80cfde0 2024-04-26T21:47:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.634+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea80694f0 con 0x7efea80cfde0 2024-04-26T21:47:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.635+0000 7efe9e7fc640 1 -- 172.21.15.40:0/3179743680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efea0005600 con 0x7efea80cfde0 2024-04-26T21:47:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.635+0000 7efe9e7fc640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe8c06c8f0 0x7efe8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.635+0000 7efe9e7fc640 1 -- 172.21.15.40:0/3179743680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7efea008bb90 con 0x7efea80cfde0 2024-04-26T21:47:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.635+0000 7efeadb80640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe8c06c8f0 0x7efe8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.636+0000 7efeadb80640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe8c06c8f0 0x7efe8c06edb0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7efea4005880 tx=0x7efea4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.639+0000 7efe9e7fc640 1 -- 172.21.15.40:0/3179743680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efea0059710 con 0x7efea80cfde0 2024-04-26T21:47:30.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:30.865+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7efea8162810 con 0x7efea80cfde0 2024-04-26T21:47:31.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.395+0000 7efe9e7fc640 1 -- 172.21.15.40:0/3179743680 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v359) ==== 162+0+0 (secure 0 0 0) 0x7efea005da10 con 0x7efea80cfde0 2024-04-26T21:47:31.395 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-26T21:47:31.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe8c06c8f0 msgr2=0x7efe8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe8c06c8f0 0x7efe8c06edb0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7efea4005880 tx=0x7efea4009040 comp rx=0 tx=0).stop 2024-04-26T21:47:31.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80cfde0 msgr2=0x7efea8160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80cfde0 0x7efea8160a10 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7efea000c760 tx=0x7efea000cc20 comp rx=0 tx=0).stop 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 shutdown_connections 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efe8c06c8f0 0x7efe8c06edb0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80cfde0 0x7efea8160a10 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efea8071090 0x7efea80cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 --2- 172.21.15.40:0/3179743680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea8070350 0x7efea809e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 >> 172.21.15.40:0/3179743680 conn(0x7efea80d2d80 msgr2=0x7efea80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 shutdown_connections 2024-04-26T21:47:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.398+0000 7efeafde4640 1 -- 172.21.15.40:0/3179743680 wait complete. 2024-04-26T21:47:31.414 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:31.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.520+0000 7f9c9fabd640 1 Processor -- start 2024-04-26T21:47:31.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.520+0000 7f9c9fabd640 1 -- start start 2024-04-26T21:47:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9fabd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9c98162880 0x7f9c980d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9fabd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c980d3bc0 0x7f9c980b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9fabd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c980b7e50 con 0x7f9c980b5f20 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9fabd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9c980d0580 con 0x7f9c980d3bc0 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9fabd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9c980d1800 con 0x7f9c98162880 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9e05a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c980d3bc0 0x7f9c980b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9d058640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9c980b5f20 0x7f9c980d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53042/0 (socket says 172.21.15.40:53042) 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9d058640 1 -- 172.21.15.40:0/4133521905 learned_addr learned my addr 172.21.15.40:0/4133521905 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.521+0000 7f9c9d859640 1 --2- 172.21.15.40:0/4133521905 >> v2:172.21.15.184:3300/0 conn(0x7f9c98162880 0x7f9c980d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.522+0000 7f9c9e05a640 1 -- 172.21.15.40:0/4133521905 >> v2:172.21.15.184:3300/0 conn(0x7f9c98162880 msgr2=0x7f9c980d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:31.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.522+0000 7f9c9e05a640 1 --2- 172.21.15.40:0/4133521905 >> v2:172.21.15.184:3300/0 conn(0x7f9c98162880 0x7f9c980d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.522+0000 7f9c9e05a640 1 -- 172.21.15.40:0/4133521905 >> v1:172.21.15.40:6789/0 conn(0x7f9c980b5f20 legacy=0x7f9c980d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.522+0000 7f9c9e05a640 1 -- 172.21.15.40:0/4133521905 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c980d07d0 con 0x7f9c980d3bc0 2024-04-26T21:47:31.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.522+0000 7f9c9c857640 1 -- 172.21.15.40:0/4133521905 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233434249 0 0) 0x7f9c980b7e50 con 0x7f9c980b5f20 2024-04-26T21:47:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.522+0000 7f9c9e05a640 1 --2- 172.21.15.40:0/4133521905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c980d3bc0 0x7f9c980b8300 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f9c9808d490 tx=0x7f9c9002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bff18a20166f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9c857640 1 -- 172.21.15.40:0/4133521905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90007740 con 0x7f9c980d3bc0 2024-04-26T21:47:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9c857640 1 -- 172.21.15.40:0/4133521905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c900078e0 con 0x7f9c980d3bc0 2024-04-26T21:47:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9c857640 1 -- 172.21.15.40:0/4133521905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90007cb0 con 0x7f9c980d3bc0 2024-04-26T21:47:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9fabd640 1 -- 172.21.15.40:0/4133521905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c980d3bc0 msgr2=0x7f9c980b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/4133521905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c980d3bc0 0x7f9c980b8300 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f9c9808d490 tx=0x7f9c9002a680 comp rx=0 tx=0).stop 2024-04-26T21:47:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9fabd640 1 -- 172.21.15.40:0/4133521905 shutdown_connections 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/4133521905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c980d3bc0 0x7f9c980b8300 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/4133521905 >> v2:172.21.15.184:3300/0 conn(0x7f9c98162880 0x7f9c980d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9fabd640 1 -- 172.21.15.40:0/4133521905 >> 172.21.15.40:0/4133521905 conn(0x7f9c980d4f60 msgr2=0x7f9c980d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9fabd640 1 -- 172.21.15.40:0/4133521905 shutdown_connections 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.523+0000 7f9c9fabd640 1 -- 172.21.15.40:0/4133521905 wait complete. 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9fabd640 1 Processor -- start 2024-04-26T21:47:31.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9fabd640 1 -- start start 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9fabd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c980b5f20 0x7f9c9815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9fabd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c980d3bc0 0x7f9c980b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9fabd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c98162880 0x7f9c980b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9fabd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98162c60 con 0x7f9c98162880 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9fabd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c980d0d90 con 0x7f9c980b5f20 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9fabd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c98162de0 con 0x7f9c980d3bc0 2024-04-26T21:47:31.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9e05a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c98162880 0x7f9c980b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9e05a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c98162880 0x7f9c980b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56300/0 (socket says 172.21.15.40:56300) 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.525+0000 7f9c9e05a640 1 -- 172.21.15.40:0/3303936398 learned_addr learned my addr 172.21.15.40:0/3303936398 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9d859640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c980b5f20 0x7f9c9815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9d058640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c980d3bc0 0x7f9c980b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9e05a640 1 -- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c980d3bc0 msgr2=0x7f9c980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9e05a640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c980d3bc0 0x7f9c980b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9e05a640 1 -- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c980b5f20 msgr2=0x7f9c9815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9e05a640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c980b5f20 0x7f9c9815c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9e05a640 1 -- 172.21.15.40:0/3303936398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c980b2fe0 con 0x7f9c98162880 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9e05a640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c98162880 0x7f9c980b5130 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f9c9002ac70 tx=0x7f9c90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c8e7fc640 1 -- 172.21.15.40:0/3303936398 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9003f070 con 0x7f9c98162880 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c8e7fc640 1 -- 172.21.15.40:0/3303936398 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c90031bc0 con 0x7f9c98162880 2024-04-26T21:47:31.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c980b32c0 con 0x7f9c98162880 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c8e7fc640 1 -- 172.21.15.40:0/3303936398 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90009450 con 0x7f9c98162880 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.526+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c980cbfa0 con 0x7f9c98162880 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.527+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c980694f0 con 0x7f9c98162880 2024-04-26T21:47:31.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.528+0000 7f9c8e7fc640 1 -- 172.21.15.40:0/3303936398 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c9003b410 con 0x7f9c98162880 2024-04-26T21:47:31.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.528+0000 7f9c8e7fc640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:31.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.529+0000 7f9c8e7fc640 1 -- 172.21.15.40:0/3303936398 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f9c900ad470 con 0x7f9c98162880 2024-04-26T21:47:31.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.529+0000 7f9c9d859640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:31.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.529+0000 7f9c9d859640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009ad0 tx=0x7f9c94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:31.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.533+0000 7f9c8e7fc640 1 -- 172.21.15.40:0/3303936398 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c9007b100 con 0x7f9c98162880 2024-04-26T21:47:31.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.757+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c980cbb80 con 0x7f9c98162880 2024-04-26T21:47:31.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.758+0000 7f9c8e7fc640 1 -- 172.21.15.40:0/3303936398 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 360 v360) ==== 95+0+5194 (secure 0 0 0) 0x7f9c9007f400 con 0x7f9c98162880 2024-04-26T21:47:31.758 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:31.758 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":360,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":360,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:31.398461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-26T21:47:31.758 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 360 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c7c06c8f0 msgr2=0x7f9c7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009ad0 tx=0x7f9c94008040 comp rx=0 tx=0).stop 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c98162880 msgr2=0x7f9c980b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c98162880 0x7f9c980b5130 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f9c9002ac70 tx=0x7f9c90004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 shutdown_connections 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c98162880 0x7f9c980b5130 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c980d3bc0 0x7f9c980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 --2- 172.21.15.40:0/3303936398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c980b5f20 0x7f9c9815c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 >> 172.21.15.40:0/3303936398 conn(0x7f9c980d4f60 msgr2=0x7f9c980d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 shutdown_connections 2024-04-26T21:47:31.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:31.760+0000 7f9c9fabd640 1 -- 172.21.15.40:0/3303936398 wait complete. 2024-04-26T21:47:32.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 360, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:47:20.455876+0000', 'modified': '2024-04-26T21:47:31.398461+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 146, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11961}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11961': {'gid': 11961, 'name': 'a', 'rank': 0, 'incarnation': 360, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.40:6835/3119057314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3119057314}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3119057314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:47:32.173 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11961:up:replay 2024-04-26T21:47:33.174 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:33.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.288+0000 7feab4283640 1 Processor -- start 2024-04-26T21:47:33.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.288+0000 7feab4283640 1 -- start start 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.289+0000 7feab4283640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feaac162880 0x7feaac0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.289+0000 7feab4283640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac0b7e50 con 0x7feaac0d3bc0 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.289+0000 7feab4283640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac0d1800 con 0x7feaac0b5f20 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.289+0000 7feab4283640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7feaac0d0580 con 0x7feaac162880 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.289+0000 7feab2820640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feaac0d3bc0 0x7feaac0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53048/0 (socket says 172.21.15.40:53048) 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.289+0000 7feab2820640 1 -- 172.21.15.40:0/2531618059 learned_addr learned my addr 172.21.15.40:0/2531618059 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.290+0000 7feab201f640 1 --2- 172.21.15.40:0/2531618059 >> v2:172.21.15.184:3300/0 conn(0x7feaac162880 0x7feaac0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.290+0000 7feab101d640 1 -- 172.21.15.40:0/2531618059 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66462017 0 0) 0x7feaac0b7e50 con 0x7feaac0d3bc0 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.290+0000 7feab101d640 1 -- 172.21.15.40:0/2531618059 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea94003520 con 0x7feaac0d3bc0 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.290+0000 7feab201f640 1 -- 172.21.15.40:0/2531618059 >> v1:172.21.15.184:6789/0 conn(0x7feaac0b5f20 legacy=0x7feaac0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.290+0000 7feab101d640 1 -- 172.21.15.40:0/2531618059 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4135346961 0 0) 0x7fea94003520 con 0x7feaac0d3bc0 2024-04-26T21:47:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.290+0000 7feab201f640 1 -- 172.21.15.40:0/2531618059 >> v1:172.21.15.40:6789/0 conn(0x7feaac0d3bc0 legacy=0x7feaac0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.290+0000 7feab201f640 1 -- 172.21.15.40:0/2531618059 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac0d07d0 con 0x7feaac162880 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.291+0000 7feab201f640 1 --2- 172.21.15.40:0/2531618059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac162880 0x7feaac0d2cb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7feaa40097a0 tx=0x7feaa402a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6bd81922c18e905 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.291+0000 7feab101d640 1 -- 172.21.15.40:0/2531618059 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4037070 con 0x7feaac162880 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.291+0000 7feab101d640 1 -- 172.21.15.40:0/2531618059 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa402ab60 con 0x7feaac162880 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.291+0000 7feab101d640 1 -- 172.21.15.40:0/2531618059 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0019f0 con 0x7feaac162880 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.292+0000 7feab4283640 1 -- 172.21.15.40:0/2531618059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac162880 msgr2=0x7feaac0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.292+0000 7feab4283640 1 --2- 172.21.15.40:0/2531618059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac162880 0x7feaac0d2cb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7feaa40097a0 tx=0x7feaa402a0d0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.292+0000 7feab4283640 1 -- 172.21.15.40:0/2531618059 shutdown_connections 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.292+0000 7feab4283640 1 --2- 172.21.15.40:0/2531618059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac162880 0x7feaac0d2cb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.292+0000 7feab4283640 1 -- 172.21.15.40:0/2531618059 >> 172.21.15.40:0/2531618059 conn(0x7feaac0d4f60 msgr2=0x7feaac0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.292+0000 7feab4283640 1 -- 172.21.15.40:0/2531618059 shutdown_connections 2024-04-26T21:47:33.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.292+0000 7feab4283640 1 -- 172.21.15.40:0/2531618059 wait complete. 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.293+0000 7feab4283640 1 Processor -- start 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.293+0000 7feab4283640 1 -- start start 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab4283640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac0b5f20 0x7feaac150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab4283640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feaac0d3bc0 0x7feaac1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab4283640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac162880 0x7feaac1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab4283640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feaac162c60 con 0x7feaac162880 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab4283640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feaac0d0d90 con 0x7feaac0b5f20 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab4283640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feaac162de0 con 0x7feaac0d3bc0 2024-04-26T21:47:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac162880 0x7feaac1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac162880 0x7feaac1614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56310/0 (socket says 172.21.15.40:56310) 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 -- 172.21.15.40:0/3938057868 learned_addr learned my addr 172.21.15.40:0/3938057868 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab201f640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac0b5f20 0x7feaac150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab181e640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feaac0d3bc0 0x7feaac1f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 -- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feaac0d3bc0 msgr2=0x7feaac1f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feaac0d3bc0 0x7feaac1f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 -- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac0b5f20 msgr2=0x7feaac150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac0b5f20 0x7feaac150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 -- 172.21.15.40:0/3938057868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac151320 con 0x7feaac162880 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab201f640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac0b5f20 0x7feaac150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feab2820640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac162880 0x7feaac1614f0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00c7f0 tx=0x7fea9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.294+0000 7feaaaffd640 1 -- 172.21.15.40:0/3938057868 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c018070 con 0x7feaac162880 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.295+0000 7feaaaffd640 1 -- 172.21.15.40:0/3938057868 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9c00a5e0 con 0x7feaac162880 2024-04-26T21:47:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.295+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac0cd120 con 0x7feaac162880 2024-04-26T21:47:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.295+0000 7feaaaffd640 1 -- 172.21.15.40:0/3938057868 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00aa10 con 0x7feaac162880 2024-04-26T21:47:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.295+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac0cd6e0 con 0x7feaac162880 2024-04-26T21:47:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.296+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaac0694f0 con 0x7feaac162880 2024-04-26T21:47:33.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.296+0000 7feaaaffd640 1 -- 172.21.15.40:0/3938057868 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea9c01b070 con 0x7feaac162880 2024-04-26T21:47:33.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.297+0000 7feaaaffd640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea9806c8f0 0x7fea9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:33.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.297+0000 7feaaaffd640 1 -- 172.21.15.40:0/3938057868 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fea9c08b260 con 0x7feaac162880 2024-04-26T21:47:33.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.297+0000 7feab201f640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea9806c8f0 0x7fea9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:33.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.297+0000 7feab201f640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea9806c8f0 0x7fea9806edb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7feaa4034000 tx=0x7feaa402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:33.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.301+0000 7feaaaffd640 1 -- 172.21.15.40:0/3938057868 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feaac151320 con 0x7feaac162880 2024-04-26T21:47:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.528+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaac0cddf0 con 0x7feaac162880 2024-04-26T21:47:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.529+0000 7feaaaffd640 1 -- 172.21.15.40:0/3938057868 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 362 v362) ==== 95+0+5194 (secure 0 0 0) 0x7fea9c059540 con 0x7feaac162880 2024-04-26T21:47:33.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:33.529 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":362,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":362,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:32.409959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-26T21:47:33.529 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 362 2024-04-26T21:47:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.530+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea9806c8f0 msgr2=0x7fea9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea9806c8f0 0x7fea9806edb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7feaa4034000 tx=0x7feaa402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac162880 msgr2=0x7feaac1614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac162880 0x7feaac1614f0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00c7f0 tx=0x7fea9c00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 shutdown_connections 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fea9806c8f0 0x7fea9806edb0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac162880 0x7feaac1614f0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feaac0d3bc0 0x7feaac1f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 --2- 172.21.15.40:0/3938057868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaac0b5f20 0x7feaac150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 >> 172.21.15.40:0/3938057868 conn(0x7feaac0d4f60 msgr2=0x7feaac160300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 shutdown_connections 2024-04-26T21:47:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:33.531+0000 7feab4283640 1 -- 172.21.15.40:0/3938057868 wait complete. 2024-04-26T21:47:33.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 362, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:47:20.455876+0000', 'modified': '2024-04-26T21:47:32.409959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 146, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11961}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11961': {'gid': 11961, 'name': 'a', 'rank': 0, 'incarnation': 360, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.40:6835/3119057314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3119057314}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3119057314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T21:47:33.943 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11961:up:rejoin 2024-04-26T21:47:34.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.059+0000 7f28e0cbd640 1 Processor -- start 2024-04-26T21:47:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.059+0000 7f28e0cbd640 1 -- start start 2024-04-26T21:47:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.059+0000 7f28e0cbd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f28dc0b3860 0x7f28dc16a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.059+0000 7f28e0cbd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28dc0d0ba0 con 0x7f28dc0d06f0 2024-04-26T21:47:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.059+0000 7f28e0cbd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28dc0d2590 con 0x7f28dc15f3a0 2024-04-26T21:47:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.059+0000 7f28e0cbd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f28dc0d0e20 con 0x7f28dc0b3860 2024-04-26T21:47:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28d3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f28dc0d06f0 0x7f28dc0d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53054/0 (socket says 172.21.15.40:53054) 2024-04-26T21:47:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28d3fff640 1 -- 172.21.15.40:0/179520490 learned_addr learned my addr 172.21.15.40:0/179520490 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28dad9d640 1 --2- 172.21.15.40:0/179520490 >> v2:172.21.15.184:3300/0 conn(0x7f28dc0b3860 0x7f28dc16a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28d9d9b640 1 -- 172.21.15.40:0/179520490 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548081190 0 0) 0x7f28c4001110 con 0x7f28dc0d06f0 2024-04-26T21:47:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28d9d9b640 1 -- 172.21.15.40:0/179520490 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28c8003520 con 0x7f28dc0d06f0 2024-04-26T21:47:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28d9d9b640 1 -- 172.21.15.40:0/179520490 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2144904220 0 0) 0x7f28c8003520 con 0x7f28dc0d06f0 2024-04-26T21:47:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28dad9d640 1 -- 172.21.15.40:0/179520490 >> v1:172.21.15.184:6789/0 conn(0x7f28dc15f3a0 legacy=0x7f28dc0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28dad9d640 1 -- 172.21.15.40:0/179520490 >> v1:172.21.15.40:6789/0 conn(0x7f28dc0d06f0 legacy=0x7f28dc0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.060+0000 7f28dad9d640 1 -- 172.21.15.40:0/179520490 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc0b1d70 con 0x7f28dc0b3860 2024-04-26T21:47:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.061+0000 7f28dad9d640 1 --2- 172.21.15.40:0/179520490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 0x7f28dc16a000 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009d90 tx=0x7f28d402a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa2875b703fa71c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.061+0000 7f28d9d9b640 1 -- 172.21.15.40:0/179520490 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4035020 con 0x7f28dc0b3860 2024-04-26T21:47:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.061+0000 7f28d9d9b640 1 -- 172.21.15.40:0/179520490 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d4007540 con 0x7f28dc0b3860 2024-04-26T21:47:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.061+0000 7f28d9d9b640 1 -- 172.21.15.40:0/179520490 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4001840 con 0x7f28dc0b3860 2024-04-26T21:47:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.062+0000 7f28e0cbd640 1 -- 172.21.15.40:0/179520490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 msgr2=0x7f28dc16a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.062+0000 7f28e0cbd640 1 --2- 172.21.15.40:0/179520490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 0x7f28dc16a000 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009d90 tx=0x7f28d402a510 comp rx=0 tx=0).stop 2024-04-26T21:47:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.062+0000 7f28e0cbd640 1 -- 172.21.15.40:0/179520490 shutdown_connections 2024-04-26T21:47:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.062+0000 7f28e0cbd640 1 --2- 172.21.15.40:0/179520490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 0x7f28dc16a000 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.062+0000 7f28e0cbd640 1 -- 172.21.15.40:0/179520490 >> 172.21.15.40:0/179520490 conn(0x7f28dc0d4f90 msgr2=0x7f28dc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.062+0000 7f28e0cbd640 1 -- 172.21.15.40:0/179520490 shutdown_connections 2024-04-26T21:47:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.062+0000 7f28e0cbd640 1 -- 172.21.15.40:0/179520490 wait complete. 2024-04-26T21:47:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.063+0000 7f28e0cbd640 1 Processor -- start 2024-04-26T21:47:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.063+0000 7f28e0cbd640 1 -- start start 2024-04-26T21:47:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28e0cbd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 0x7f28dc0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28e0cbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28dc0d06f0 0x7f28dc0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28e0cbd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28dc15f3a0 0x7f28dc0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28e0cbd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc15f780 con 0x7f28dc0d06f0 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28e0cbd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc0b2330 con 0x7f28dc0b3860 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28e0cbd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28dc15f900 con 0x7f28dc15f3a0 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28dc0d06f0 0x7f28dc0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28dc0d06f0 0x7f28dc0ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56326/0 (socket says 172.21.15.40:56326) 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.064+0000 7f28d3fff640 1 -- 172.21.15.40:0/377634142 learned_addr learned my addr 172.21.15.40:0/377634142 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28dad9d640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28dc15f3a0 0x7f28dc0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28da59c640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 0x7f28dc0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d3fff640 1 -- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28dc15f3a0 msgr2=0x7f28dc0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d3fff640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28dc15f3a0 0x7f28dc0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d3fff640 1 -- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 msgr2=0x7f28dc0cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d3fff640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 0x7f28dc0cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d3fff640 1 -- 172.21.15.40:0/377634142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc0c9ff0 con 0x7f28dc0d06f0 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28da59c640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 0x7f28dc0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d3fff640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28dc0d06f0 0x7f28dc0ce4b0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c7b0 tx=0x7f28c400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d2ffd640 1 -- 172.21.15.40:0/377634142 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4018070 con 0x7f28dc0d06f0 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d2ffd640 1 -- 172.21.15.40:0/377634142 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c40072f0 con 0x7f28dc0d06f0 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc0ca2d0 con 0x7f28dc0d06f0 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28d2ffd640 1 -- 172.21.15.40:0/377634142 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40047e0 con 0x7f28dc0d06f0 2024-04-26T21:47:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.065+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc0ca890 con 0x7f28dc0d06f0 2024-04-26T21:47:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.066+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28dc0694f0 con 0x7f28dc0d06f0 2024-04-26T21:47:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.067+0000 7f28d2ffd640 1 -- 172.21.15.40:0/377634142 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28c401b070 con 0x7f28dc0d06f0 2024-04-26T21:47:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.067+0000 7f28d2ffd640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.067+0000 7f28d2ffd640 1 -- 172.21.15.40:0/377634142 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f28c408b080 con 0x7f28dc0d06f0 2024-04-26T21:47:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.068+0000 7f28da59c640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.068+0000 7f28da59c640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28bc06c8f0 0x7f28bc06edb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000f60 tx=0x7f28cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.072+0000 7f28d2ffd640 1 -- 172.21.15.40:0/377634142 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28dc0d0ba0 con 0x7f28dc0d06f0 2024-04-26T21:47:35.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.301+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28dc075ce0 con 0x7f28dc0d06f0 2024-04-26T21:47:35.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.301+0000 7f28d2ffd640 1 -- 172.21.15.40:0/377634142 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7f28c4058d10 con 0x7f28dc0d06f0 2024-04-26T21:47:35.301 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:35.301 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:35.301 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:35.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28bc06c8f0 msgr2=0x7f28bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28bc06c8f0 0x7f28bc06edb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000f60 tx=0x7f28cc009040 comp rx=0 tx=0).stop 2024-04-26T21:47:35.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28dc0d06f0 msgr2=0x7f28dc0ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28dc0d06f0 0x7f28dc0ce4b0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c7b0 tx=0x7f28c400cc70 comp rx=0 tx=0).stop 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 shutdown_connections 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28dc15f3a0 0x7f28dc0b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28dc0d06f0 0x7f28dc0ce4b0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 --2- 172.21.15.40:0/377634142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28dc0b3860 0x7f28dc0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 >> 172.21.15.40:0/377634142 conn(0x7f28dc0d4f90 msgr2=0x7f28dc06e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 shutdown_connections 2024-04-26T21:47:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.303+0000 7f28e0cbd640 1 -- 172.21.15.40:0/377634142 wait complete. 2024-04-26T21:47:35.716 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 363, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:47:20.455876+0000', 'modified': '2024-04-26T21:47:34.414722+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 146, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11961}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11961': {'gid': 11961, 'name': 'a', 'rank': 0, 'incarnation': 360, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6835/3119057314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3119057314}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3119057314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11961, 'qdb_cluster': [11961]} 2024-04-26T21:47:35.716 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:47:35.717 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:47:35.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.822+0000 7f1242c28640 1 Processor -- start 2024-04-26T21:47:35.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f1242c28640 1 -- start start 2024-04-26T21:47:35.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f1242c28640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f123c15f3a0 0x7f123c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f1242c28640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d1770 0x7f123c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f1242c28640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f123c0d2130 0x7f123c0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f1242c28640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f123c0d25a0 con 0x7f123c0d2130 2024-04-26T21:47:35.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f1242c28640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f123c0d0e20 con 0x7f123c15f3a0 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f1242c28640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f123c0b1d70 con 0x7f123c0d1770 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f12411c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f123c0d2130 0x7f123c0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f12411c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f123c0d2130 0x7f123c0d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56328/0 (socket says 172.21.15.40:56328) 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.823+0000 7f12411c5640 1 -- 172.21.15.40:0/2554286783 learned_addr learned my addr 172.21.15.40:0/2554286783 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.824+0000 7f123bfff640 1 --2- 172.21.15.40:0/2554286783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d1770 0x7f123c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.824+0000 7f12411c5640 1 -- 172.21.15.40:0/2554286783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d1770 msgr2=0x7f123c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.824+0000 7f12411c5640 1 --2- 172.21.15.40:0/2554286783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d1770 0x7f123c0d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.824+0000 7f12411c5640 1 -- 172.21.15.40:0/2554286783 >> v2:172.21.15.184:3300/0 conn(0x7f123c15f3a0 msgr2=0x7f123c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.824+0000 7f12409c4640 1 --2- 172.21.15.40:0/2554286783 >> v2:172.21.15.184:3300/0 conn(0x7f123c15f3a0 0x7f123c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.824+0000 7f12411c5640 1 --2- 172.21.15.40:0/2554286783 >> v2:172.21.15.184:3300/0 conn(0x7f123c15f3a0 0x7f123c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.824+0000 7f12411c5640 1 -- 172.21.15.40:0/2554286783 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c0b2050 con 0x7f123c0d2130 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f12411c5640 1 --2- 172.21.15.40:0/2554286783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d2130 0x7f123c0d2e50 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f123c073b50 tx=0x7f122c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2767f157148bf3de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f123b7fe640 1 -- 172.21.15.40:0/2554286783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c037070 con 0x7f123c0d2130 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f123b7fe640 1 -- 172.21.15.40:0/2554286783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122c032440 con 0x7f123c0d2130 2024-04-26T21:47:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f123b7fe640 1 -- 172.21.15.40:0/2554286783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c032810 con 0x7f123c0d2130 2024-04-26T21:47:35.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 -- 172.21.15.40:0/2554286783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d2130 msgr2=0x7f123c0d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 --2- 172.21.15.40:0/2554286783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d2130 0x7f123c0d2e50 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f123c073b50 tx=0x7f122c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 -- 172.21.15.40:0/2554286783 shutdown_connections 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 --2- 172.21.15.40:0/2554286783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d2130 0x7f123c0d2e50 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 --2- 172.21.15.40:0/2554286783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d1770 0x7f123c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 --2- 172.21.15.40:0/2554286783 >> v2:172.21.15.184:3300/0 conn(0x7f123c15f3a0 0x7f123c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 -- 172.21.15.40:0/2554286783 >> 172.21.15.40:0/2554286783 conn(0x7f123c0d4f90 msgr2=0x7f123c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 -- 172.21.15.40:0/2554286783 shutdown_connections 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.825+0000 7f1242c28640 1 -- 172.21.15.40:0/2554286783 wait complete. 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.826+0000 7f1242c28640 1 Processor -- start 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.827+0000 7f1242c28640 1 -- start start 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.827+0000 7f1242c28640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d1770 0x7f123c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.827+0000 7f1242c28640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d2130 0x7f123c0b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.827+0000 7f1242c28640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f123c15f3a0 0x7f123c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.827+0000 7f1242c28640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f123c15f900 con 0x7f123c0d1770 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.827+0000 7f1242c28640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f123c15f780 con 0x7f123c15f3a0 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.827+0000 7f1242c28640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f123c0d3390 con 0x7f123c0d2130 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d1770 0x7f123c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d1770 0x7f123c1536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56334/0 (socket says 172.21.15.40:56334) 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 -- 172.21.15.40:0/720634045 learned_addr learned my addr 172.21.15.40:0/720634045 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f123bfff640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d2130 0x7f123c0b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12411c5640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f123c15f3a0 0x7f123c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 -- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d2130 msgr2=0x7f123c0b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d2130 0x7f123c0b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 -- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f123c15f3a0 msgr2=0x7f123c0b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f123c15f3a0 0x7f123c0b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 -- 172.21.15.40:0/720634045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c1f57e0 con 0x7f123c0d1770 2024-04-26T21:47:35.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12409c4640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d1770 0x7f123c1536c0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f123400cd00 tx=0x7f123400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12397fa640 1 -- 172.21.15.40:0/720634045 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234003990 con 0x7f123c0d1770 2024-04-26T21:47:35.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12397fa640 1 -- 172.21.15.40:0/720634045 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1234003b30 con 0x7f123c0d1770 2024-04-26T21:47:35.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f1242c28640 1 -- 172.21.15.40:0/720634045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123c1f5ac0 con 0x7f123c0d1770 2024-04-26T21:47:35.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f12397fa640 1 -- 172.21.15.40:0/720634045 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234004020 con 0x7f123c0d1770 2024-04-26T21:47:35.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.828+0000 7f1242c28640 1 -- 172.21.15.40:0/720634045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123c1f6080 con 0x7f123c0d1770 2024-04-26T21:47:35.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.830+0000 7f12397fa640 1 -- 172.21.15.40:0/720634045 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12340041c0 con 0x7f123c0d1770 2024-04-26T21:47:35.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.831+0000 7f12397fa640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f122806c8f0 0x7f122806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.831+0000 7f12397fa640 1 -- 172.21.15.40:0/720634045 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f123408c250 con 0x7f123c0d1770 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.831+0000 7f123bfff640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f122806c8f0 0x7f122806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f1245a17740 1 Processor -- start 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f1245a17740 1 -- start start 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f123bfff640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f122806c8f0 0x7f122806edb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f123c0d1e20 tx=0x7f1230008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f1245a17740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d66820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f1245a17740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d84230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f1245a17740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d3c120 0x564451d823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f1245a17740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564451ca0740 con 0x564451d83ce0 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f1245a17740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564451c3c3a0 con 0x564451d66440 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f1245a17740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564451c9c700 con 0x564451d3c120 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f123bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d84230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f123bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d84230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56338/0 (socket says 172.21.15.40:56338) 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.832+0000 7f123bfff640 1 -- 172.21.15.40:0/3867274004 learned_addr learned my addr 172.21.15.40:0/3867274004 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f12411c5640 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d3c120 0x564451d823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f12409c4640 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d66820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f123bfff640 1 -- 172.21.15.40:0/3867274004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d3c120 msgr2=0x564451d823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f123bfff640 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d3c120 0x564451d823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f123bfff640 1 -- 172.21.15.40:0/3867274004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 msgr2=0x564451d66820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f123bfff640 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d66820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f123bfff640 1 -- 172.21.15.40:0/3867274004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564451c58750 con 0x564451d83ce0 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f123bfff640 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d84230 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f123000ec30 tx=0x7f1230011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490b7de7fec16e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f12227fc640 1 -- 172.21.15.40:0/3867274004 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123001a050 con 0x564451d83ce0 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f12227fc640 1 -- 172.21.15.40:0/3867274004 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1230024070 con 0x564451d83ce0 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.833+0000 7f12227fc640 1 -- 172.21.15.40:0/3867274004 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123001f440 con 0x564451d83ce0 2024-04-26T21:47:35.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 -- 172.21.15.40:0/3867274004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 msgr2=0x564451d84230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d84230 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f123000ec30 tx=0x7f1230011000 comp rx=0 tx=0).stop 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 -- 172.21.15.40:0/3867274004 shutdown_connections 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d3c120 0x564451d823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d84230 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 --2- 172.21.15.40:0/3867274004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d66820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 -- 172.21.15.40:0/3867274004 >> 172.21.15.40:0/3867274004 conn(0x564451d38a50 msgr2=0x564451d81250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 -- 172.21.15.40:0/3867274004 shutdown_connections 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 -- 172.21.15.40:0/3867274004 wait complete. 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 -- start start 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.834+0000 7f1245a17740 1 Processor -- start 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f1245a17740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d66440 0x564451d62d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f1245a17740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d83ce0 0x564451d91670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f1245a17740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d69ca0 0x564451d6a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f1245a17740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564451d3cec0 con 0x564451d69ca0 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f1245a17740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564451c27fe0 con 0x564451d83ce0 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f1245a17740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564451d36980 con 0x564451d66440 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f1245a17740 10 client.0 _ll_register_callbacks cb 0x564451c3e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f12411c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d69ca0 0x564451d6a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f12411c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d69ca0 0x564451d6a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56340/0 (socket says 172.21.15.40:56340) 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f12411c5640 1 -- 172.21.15.40:0/1019660434 learned_addr learned my addr 172.21.15.40:0/1019660434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f123bfff640 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d83ce0 0x564451d91670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.835+0000 7f12409c4640 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d66440 0x564451d62d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f12411c5640 1 -- 172.21.15.40:0/1019660434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d66440 msgr2=0x564451d62d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f12411c5640 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d66440 0x564451d62d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f12411c5640 1 -- 172.21.15.40:0/1019660434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d83ce0 msgr2=0x564451d91670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f12411c5640 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d83ce0 0x564451d91670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f12411c5640 1 -- 172.21.15.40:0/1019660434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644519b22d0 con 0x564451d69ca0 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f12411c5640 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d69ca0 0x564451d6a120 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f122c0090b0 tx=0x7f122c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b04b77339605f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f121b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f121b7fe640 1 -- 172.21.15.40:0/1019660434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c037040 con 0x564451d69ca0 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f121b7fe640 1 -- 172.21.15.40:0/1019660434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f122c047c80 con 0x564451d69ca0 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f1245a17740 1 -- 172.21.15.40:0/1019660434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1230024070 con 0x564451d69ca0 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f121b7fe640 1 -- 172.21.15.40:0/1019660434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f122c032e30 con 0x564451d69ca0 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f1245a17740 10 client.12015 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f1245a17740 1 -- 172.21.15.40:0/1019660434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x564451a7c6b0 con 0x564451d69ca0 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f121b7fe640 1 -- 172.21.15.40:0/1019660434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1230024070 con 0x564451d69ca0 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.836+0000 7f121b7fe640 1 -- 172.21.15.40:0/1019660434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f122c04e050 con 0x564451d69ca0 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f1245a17740 10 client.12015 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f1245a17740 10 client.12015 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f1245a17740 20 client.12015 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f1245a17740 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564451d78f20 0x564451d79300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f1245a17740 4 client.12015 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:47:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f1245a17740 1 -- 172.21.15.40:0/1019660434 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564451cae350 con 0x564451d78f20 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f12409c4640 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564451d78f20 0x564451d79300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f121b7fe640 10 client.12015 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.837+0000 7f12409c4640 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564451d78f20 0x564451d79300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43cbe80cf0e2c6c server_cookie=b72df8f079afd01b in_seq=0 out_seq=0 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f121b7fe640 1 -- 172.21.15.40:0/1019660434 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564451cae350 con 0x564451d78f20 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f121b7fe640 10 client.12015 handle_command_reply: tid=0 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 client.12015 shutdown 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 -- 172.21.15.40:0/1019660434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d69ca0 msgr2=0x564451d6a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d69ca0 0x564451d6a120 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f122c0090b0 tx=0x7f122c004730 comp rx=0 tx=0).stop 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 -- 172.21.15.40:0/1019660434 shutdown_connections 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564451d78f20 0x564451d79300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d69ca0 0x564451d6a120 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f121b7fe640 0 client.12015 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d83ce0 0x564451d91670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 --2- 172.21.15.40:0/1019660434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d66440 0x564451d62d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 -- 172.21.15.40:0/1019660434 >> 172.21.15.40:0/1019660434 conn(0x564451d3c120 msgr2=0x564451d80a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 -- 172.21.15.40:0/1019660434 shutdown_connections 2024-04-26T21:47:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 1 -- 172.21.15.40:0/1019660434 wait complete. 2024-04-26T21:47:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.838+0000 7f1245a17740 20 client.12015 trim_cache size 0 max 16384 2024-04-26T21:47:35.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.856+0000 7f1245a17740 1 Processor -- start 2024-04-26T21:47:35.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.856+0000 7f1245a17740 1 -- start start 2024-04-26T21:47:35.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f1245a17740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d84ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f1245a17740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 0x564451d653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f1245a17740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d913a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f1245a17740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f123c200f30 con 0x564451d83ce0 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f1245a17740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f123c162390 con 0x564451d66440 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f1245a17740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f123c1629e0 con 0x564451d69ca0 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12411c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d913a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12411c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d913a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56350/0 (socket says 172.21.15.40:56350) 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12411c5640 1 -- 172.21.15.40:0/1277107290 learned_addr learned my addr 172.21.15.40:0/1277107290 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f123bfff640 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 0x564451d653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12409c4640 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d84ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12411c5640 1 -- 172.21.15.40:0/1277107290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 msgr2=0x564451d653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12411c5640 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 0x564451d653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12411c5640 1 -- 172.21.15.40:0/1277107290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 msgr2=0x564451d84ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12411c5640 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d84ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.857+0000 7f12411c5640 1 -- 172.21.15.40:0/1277107290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564451d32700 con 0x564451d83ce0 2024-04-26T21:47:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f12411c5640 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d913a0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f122c02a7d0 tx=0x7f122c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b945cb1fc520e27c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1221ffb640 1 -- 172.21.15.40:0/1277107290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c000e60 con 0x564451d83ce0 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1221ffb640 1 -- 172.21.15.40:0/1277107290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122c03c410 con 0x564451d83ce0 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1221ffb640 1 -- 172.21.15.40:0/1277107290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c037040 con 0x564451d83ce0 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 -- 172.21.15.40:0/1277107290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 msgr2=0x564451d913a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d913a0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f122c02a7d0 tx=0x7f122c03d000 comp rx=0 tx=0).stop 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 -- 172.21.15.40:0/1277107290 shutdown_connections 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d913a0 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 0x564451d653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 --2- 172.21.15.40:0/1277107290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d84ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 -- 172.21.15.40:0/1277107290 >> 172.21.15.40:0/1277107290 conn(0x564451d3c120 msgr2=0x564451d69330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 -- 172.21.15.40:0/1277107290 shutdown_connections 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.858+0000 7f1245a17740 1 -- 172.21.15.40:0/1277107290 wait complete. 2024-04-26T21:47:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.859+0000 7f1245a17740 1 -- start start 2024-04-26T21:47:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.859+0000 7f1245a17740 1 Processor -- start 2024-04-26T21:47:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f1245a17740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d843d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f1245a17740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 0x564451d90550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f1245a17740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d7e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f1245a17740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564451d671d0 con 0x564451d83ce0 2024-04-26T21:47:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f1245a17740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564451d787e0 con 0x564451d66440 2024-04-26T21:47:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f1245a17740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564451d95250 con 0x564451d69ca0 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f1245a17740 10 client.0 _ll_register_callbacks cb 0x7f122c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d7e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d7e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56362/0 (socket says 172.21.15.40:56362) 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 -- 172.21.15.40:0/3472702672 learned_addr learned my addr 172.21.15.40:0/3472702672 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f123bfff640 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 0x564451d90550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12409c4640 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d843d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 -- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 msgr2=0x564451d90550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 0x564451d90550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 -- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 msgr2=0x564451d843d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d843d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 -- 172.21.15.40:0/3472702672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564451d84d40 con 0x564451d83ce0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12409c4640 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d843d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f12411c5640 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d7e6f0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f122c009630 tx=0x7f122c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ec8c7fa0472cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f121b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f121b7fe640 1 -- 172.21.15.40:0/3472702672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c054070 con 0x564451d83ce0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.860+0000 7f121b7fe640 1 -- 172.21.15.40:0/3472702672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f122c004440 con 0x564451d83ce0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 1 -- 172.21.15.40:0/3472702672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122c03c410 con 0x564451d83ce0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f121b7fe640 1 -- 172.21.15.40:0/3472702672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f122c05f050 con 0x564451d83ce0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 10 client.12021 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 1 -- 172.21.15.40:0/3472702672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x564451d90af0 con 0x564451d83ce0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f121b7fe640 1 -- 172.21.15.40:0/3472702672 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122c03c410 con 0x564451d83ce0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f121b7fe640 1 -- 172.21.15.40:0/3472702672 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f122c062020 con 0x564451d83ce0 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 10 client.12021 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 10 client.12021 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 20 client.12021 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564451d359f0 0x564451d79300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 4 client.12021 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f1245a17740 1 -- 172.21.15.40:0/3472702672 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "status"}) -- 0x564451ca81c0 con 0x564451d359f0 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f12409c4640 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564451d359f0 0x564451d79300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f121b7fe640 10 client.12021 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.861+0000 7f12409c4640 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564451d359f0 0x564451d79300 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69242c166705f619 server_cookie=5a6d230c7c7cea77 in_seq=0 out_seq=0 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f121b7fe640 1 -- 172.21.15.40:0/3472702672 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x564451ca81c0 con 0x564451d359f0 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f121b7fe640 10 client.12021 handle_command_reply: tid=0 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 client.12021 shutdown 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 -- 172.21.15.40:0/3472702672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 msgr2=0x564451d7e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d7e6f0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f122c009630 tx=0x7f122c009240 comp rx=0 tx=0).stop 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 -- 172.21.15.40:0/3472702672 shutdown_connections 2024-04-26T21:47:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564451d359f0 0x564451d79300 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564451d83ce0 0x564451d7e6f0 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564451d69ca0 0x564451d90550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f121b7fe640 0 client.12021 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 --2- 172.21.15.40:0/3472702672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564451d66440 0x564451d843d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 -- 172.21.15.40:0/3472702672 >> 172.21.15.40:0/3472702672 conn(0x564451d3c120 msgr2=0x564451d807e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 -- 172.21.15.40:0/3472702672 shutdown_connections 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 1 -- 172.21.15.40:0/3472702672 wait complete. 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1245a17740 20 client.12021 trim_cache size 0 max 16384 2024-04-26T21:47:35.869 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "ec445c36-0c6e-44d6-884b-6999dea32f55", 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 11961, 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 4.4593150000000001, 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 363, 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 146, 2024-04-26T21:47:35.870 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 146, 2024-04-26T21:47:35.871 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 5.429903049 2024-04-26T21:47:35.871 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:35.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1242c28640 1 -- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f122806c8f0 msgr2=0x7f122806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1242c28640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f122806c8f0 0x7f122806edb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f123c0d1e20 tx=0x7f1230008040 comp rx=0 tx=0).stop 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1242c28640 1 -- 172.21.15.40:0/720634045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d1770 msgr2=0x7f123c1536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.862+0000 7f1242c28640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d1770 0x7f123c1536c0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f123400cd00 tx=0x7f123400a5c0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.863+0000 7f1242c28640 1 -- 172.21.15.40:0/720634045 shutdown_connections 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.863+0000 7f1242c28640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f122806c8f0 0x7f122806edb0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.863+0000 7f1242c28640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f123c15f3a0 0x7f123c0b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.863+0000 7f1242c28640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f123c0d2130 0x7f123c0b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.863+0000 7f1242c28640 1 --2- 172.21.15.40:0/720634045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f123c0d1770 0x7f123c1536c0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.863+0000 7f1242c28640 1 -- 172.21.15.40:0/720634045 >> 172.21.15.40:0/720634045 conn(0x7f123c0d4f90 msgr2=0x7f123c074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.863+0000 7f1242c28640 1 -- 172.21.15.40:0/720634045 shutdown_connections 2024-04-26T21:47:35.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.863+0000 7f1242c28640 1 -- 172.21.15.40:0/720634045 wait complete. 2024-04-26T21:47:35.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:47:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.974+0000 7ff5bde17640 1 Processor -- start 2024-04-26T21:47:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.974+0000 7ff5bde17640 1 -- start start 2024-04-26T21:47:35.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5bde17640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5bde17640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5b80d2300 0x7ff5b80b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5bde17640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5b80d0e20 con 0x7ff5b80d1770 2024-04-26T21:47:35.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5bde17640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b80b1d70 con 0x7ff5b80d2300 2024-04-26T21:47:35.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5bde17640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b80b2050 con 0x7ff5b815f3a0 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5b6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff5b80d1770 0x7ff5b80d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53060/0 (socket says 172.21.15.40:53060) 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5b6ffd640 1 -- 172.21.15.40:0/798299133 learned_addr learned my addr 172.21.15.40:0/798299133 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5b7fff640 1 --2- 172.21.15.40:0/798299133 >> v2:172.21.15.184:3300/0 conn(0x7ff5b80d2300 0x7ff5b80b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.975+0000 7ff5b77fe640 1 --2- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.976+0000 7ff5b67fc640 1 -- 172.21.15.40:0/798299133 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836650806 0 0) 0x7ff5b80d0e20 con 0x7ff5b80d1770 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.976+0000 7ff5b67fc640 1 -- 172.21.15.40:0/798299133 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a0003520 con 0x7ff5b80d1770 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.976+0000 7ff5b7fff640 1 -- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 msgr2=0x7ff5b80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.976+0000 7ff5b7fff640 1 --2- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.976+0000 7ff5b7fff640 1 -- 172.21.15.40:0/798299133 >> v1:172.21.15.40:6789/0 conn(0x7ff5b80d1770 legacy=0x7ff5b80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.976+0000 7ff5b7fff640 1 -- 172.21.15.40:0/798299133 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b80b2330 con 0x7ff5b80d2300 2024-04-26T21:47:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.976+0000 7ff5b77fe640 1 --2- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5b7fff640 1 --2- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 0x7ff5b80b2850 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009d90 tx=0x7ff5b002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a63c2418d4004ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5b67fc640 1 -- 172.21.15.40:0/798299133 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0007740 con 0x7ff5b80d2300 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5b67fc640 1 -- 172.21.15.40:0/798299133 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b00078e0 con 0x7ff5b80d2300 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5b67fc640 1 -- 172.21.15.40:0/798299133 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0007c90 con 0x7ff5b80d2300 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5bde17640 1 -- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 msgr2=0x7ff5b80b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5bde17640 1 --2- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 0x7ff5b80b2850 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009d90 tx=0x7ff5b002a660 comp rx=0 tx=0).stop 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5bde17640 1 -- 172.21.15.40:0/798299133 shutdown_connections 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5bde17640 1 --2- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 0x7ff5b80b2850 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5bde17640 1 --2- 172.21.15.40:0/798299133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5bde17640 1 -- 172.21.15.40:0/798299133 >> 172.21.15.40:0/798299133 conn(0x7ff5b80d4f90 msgr2=0x7ff5b80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5bde17640 1 -- 172.21.15.40:0/798299133 shutdown_connections 2024-04-26T21:47:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.977+0000 7ff5bde17640 1 -- 172.21.15.40:0/798299133 wait complete. 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.979+0000 7ff5bde17640 1 Processor -- start 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.979+0000 7ff5bde17640 1 -- start start 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.979+0000 7ff5bde17640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5b80d1770 0x7ff5b81f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.979+0000 7ff5bde17640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 0x7ff5b81f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.979+0000 7ff5bde17640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80b9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.979+0000 7ff5bde17640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b80b2f10 con 0x7ff5b80d1770 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.979+0000 7ff5bde17640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b80b2d90 con 0x7ff5b80d2300 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.979+0000 7ff5bde17640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b80b3090 con 0x7ff5b815f3a0 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5b80d1770 0x7ff5b81f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5b80d1770 0x7ff5b81f1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56370/0 (socket says 172.21.15.40:56370) 2024-04-26T21:47:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 -- 172.21.15.40:0/2286745290 learned_addr learned my addr 172.21.15.40:0/2286745290 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b6ffd640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 0x7ff5b81f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b7fff640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 -- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 msgr2=0x7ff5b80b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 -- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 msgr2=0x7ff5b81f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 0x7ff5b81f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 -- 172.21.15.40:0/2286745290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b81f22c0 con 0x7ff5b80d1770 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b6ffd640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 0x7ff5b81f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b7fff640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5b77fe640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5b80d1770 0x7ff5b81f1d80 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800e990 tx=0x7ff5a800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff59ffff640 1 -- 172.21.15.40:0/2286745290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8017070 con 0x7ff5b80d1770 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff59ffff640 1 -- 172.21.15.40:0/2286745290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a800c9e0 con 0x7ff5b80d1770 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b8152340 con 0x7ff5b80d1770 2024-04-26T21:47:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff59ffff640 1 -- 172.21.15.40:0/2286745290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a800cdb0 con 0x7ff5b80d1770 2024-04-26T21:47:35.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.980+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b8152900 con 0x7ff5b80d1770 2024-04-26T21:47:35.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.981+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b80694f0 con 0x7ff5b80d1770 2024-04-26T21:47:35.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.982+0000 7ff59ffff640 1 -- 172.21.15.40:0/2286745290 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5a801a070 con 0x7ff5b80d1770 2024-04-26T21:47:35.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.983+0000 7ff59ffff640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a406c920 0x7ff5a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:35.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.983+0000 7ff59ffff640 1 -- 172.21.15.40:0/2286745290 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7ff5a808bbe0 con 0x7ff5b80d1770 2024-04-26T21:47:35.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.983+0000 7ff5b6ffd640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a406c920 0x7ff5a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:35.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.983+0000 7ff5b6ffd640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a406c920 0x7ff5a406ede0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac007fd0 tx=0x7ff5ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:35.987+0000 7ff59ffff640 1 -- 172.21.15.40:0/2286745290 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5a8059e30 con 0x7ff5b80d1770 2024-04-26T21:47:36.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.219+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff5b80b6860 con 0x7ff5b80d1770 2024-04-26T21:47:36.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.219+0000 7ff59ffff640 1 -- 172.21.15.40:0/2286745290 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v363) ==== 53+0+73 (secure 0 0 0) 0x7ff5a8059730 con 0x7ff5b80d1770 2024-04-26T21:47:36.219 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a406c920 msgr2=0x7ff5a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a406c920 0x7ff5a406ede0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac007fd0 tx=0x7ff5ac009040 comp rx=0 tx=0).stop 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5b80d1770 msgr2=0x7ff5b81f1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5b80d1770 0x7ff5b81f1d80 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800e990 tx=0x7ff5a800ee50 comp rx=0 tx=0).stop 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 shutdown_connections 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff5a406c920 0x7ff5a406ede0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5b815f3a0 0x7ff5b80b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b80d2300 0x7ff5b81f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 --2- 172.21.15.40:0/2286745290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5b80d1770 0x7ff5b81f1d80 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:36.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 >> 172.21.15.40:0/2286745290 conn(0x7ff5b80d4f90 msgr2=0x7ff5b806d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:36.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 shutdown_connections 2024-04-26T21:47:36.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36.221+0000 7ff5bde17640 1 -- 172.21.15.40:0/2286745290 wait complete. 2024-04-26T21:47:36.231 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:47:36.231 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:47:36.232 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:47:36.232 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:47:36.232 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:36.232 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:47:36.232 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:47:36.232 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:47:36.232 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:36.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:36.232 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:47:36.248 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:47:36.248 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:47:36.248 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:47:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 41991sec preferred_lft 41991sec 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5867:54ff:feba:7b81/64 scope link 2024-04-26T21:47:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run.smithi040.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 6a:80:d2:b5:76:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::6880:d2ff:feb5:763c/64 scope link 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether aa:b1:65:53:72:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a8b1:65ff:fe53:720c/64 scope link 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:47:36.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:36.251 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:47:36.312 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:47:36.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:36.313 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:47:36.369 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:47:36.369 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:47:36.370 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:47:36.370 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:47:36.370 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:47:36.370 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:47:36.370 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:47:36.370 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:47:36.370 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:47:36.370 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:47:36.370 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:47:36.403 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:47:36.404+0000 7f928b37d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:47:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:47:36.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:36.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:36.489 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:47:36.504 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:47:36.504 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:47:36.504 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:36.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:36.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:47:36.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:47:36.568 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:36.624 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:47:36.624 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:47:36.624 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:36.679 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:47:36.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:36.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:47:36.761 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:47:36.762 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:47:36.775 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:47:36.776 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:47:36.776 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:47:36.776 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@ec445c36-0c6e-44d6-884b-6999dea32f55.cephfs=/ 2024-04-26T21:47:36.776 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2024-04-26T21:47:36.776 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:47:36.776 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:47:36.777 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:47:36.777 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:47:36.777 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:47:36.778 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:47:36.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:36.779 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:36.843 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:47:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:47:36.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:47:36.940 INFO:teuthology.orchestra.run.smithi040.stdout:client8687 2024-04-26T21:47:36.941 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:36.941 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:36.956 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-26T21:47:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.040+0000 7fc388769f40 1 Processor -- start 2024-04-26T21:47:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.040+0000 7fc388769f40 1 -- start start 2024-04-26T21:47:37.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.040+0000 7fc388769f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb34724f40 0x55fb347bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.040+0000 7fc388769f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb347c1270 0x55fb3473cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.040+0000 7fc388769f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347bd7f0 0x55fb3473d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.040+0000 7fc388769f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fb3469ce40 con 0x55fb347bd7f0 2024-04-26T21:47:37.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc388769f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fb346a0e80 con 0x55fb34724f40 2024-04-26T21:47:37.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc388769f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55fb347cac00 con 0x55fb347c1270 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347bd7f0 0x55fb3473d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347bd7f0 0x55fb3473d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56396/0 (socket says 172.21.15.40:56396) 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 -- 172.21.15.40:0/845037705 learned_addr learned my addr 172.21.15.40:0/845037705 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc385a62640 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb347c1270 0x55fb3473cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386263640 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb34724f40 0x55fb347bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 -- 172.21.15.40:0/845037705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb347c1270 msgr2=0x55fb3473cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb347c1270 0x55fb3473cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 -- 172.21.15.40:0/845037705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb34724f40 msgr2=0x55fb347bd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb34724f40 0x55fb347bd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 -- 172.21.15.40:0/845037705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3473c020 con 0x55fb347bd7f0 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc386a64640 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347bd7f0 0x55fb3473d2c0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fc378000f80 tx=0x7fc378002a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba3cedeb0390930 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc384a60640 1 -- 172.21.15.40:0/845037705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378011590 con 0x55fb347bd7f0 2024-04-26T21:47:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc384a60640 1 -- 172.21.15.40:0/845037705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc378017070 con 0x55fb347bd7f0 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.041+0000 7fc384a60640 1 -- 172.21.15.40:0/845037705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378011970 con 0x55fb347bd7f0 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 -- 172.21.15.40:0/845037705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347bd7f0 msgr2=0x55fb3473d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347bd7f0 0x55fb3473d2c0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fc378000f80 tx=0x7fc378002a90 comp rx=0 tx=0).stop 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 -- 172.21.15.40:0/845037705 shutdown_connections 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347bd7f0 0x55fb3473d2c0 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb347c1270 0x55fb3473cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 --2- 172.21.15.40:0/845037705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb34724f40 0x55fb347bd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 -- 172.21.15.40:0/845037705 >> 172.21.15.40:0/845037705 conn(0x55fb3471f8a0 msgr2=0x55fb34720fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 -- 172.21.15.40:0/845037705 shutdown_connections 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 -- 172.21.15.40:0/845037705 wait complete. 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 Processor -- start 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.042+0000 7fc388769f40 1 -- start start 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc388769f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb34724f40 0x55fb34724220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc388769f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb347bd7f0 0x55fb347b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc388769f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347c1270 0x55fb347b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc388769f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fb347c1eb0 con 0x55fb347c1270 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc388769f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fb347c0f40 con 0x55fb347bd7f0 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc388769f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55fb347355f0 con 0x55fb34724f40 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386a64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347c1270 0x55fb347b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386a64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347c1270 0x55fb347b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56408/0 (socket says 172.21.15.40:56408) 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386a64640 1 -- 172.21.15.40:0/3335103411 learned_addr learned my addr 172.21.15.40:0/3335103411 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386263640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb34724f40 0x55fb34724220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc385a62640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb347bd7f0 0x55fb347b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386a64640 1 -- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb34724f40 msgr2=0x55fb34724220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386a64640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb34724f40 0x55fb34724220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386a64640 1 -- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb347bd7f0 msgr2=0x55fb347b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386a64640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb347bd7f0 0x55fb347b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386a64640 1 -- 172.21.15.40:0/3335103411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb34724d50 con 0x55fb347c1270 2024-04-26T21:47:37.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc386263640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55fb34724f40 0x55fb34724220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:37.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.043+0000 7fc385a62640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb347bd7f0 0x55fb347b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:37.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.044+0000 7fc386a64640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347c1270 0x55fb347b4550 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x55fb346cbe20 tx=0x7fc3780015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.044+0000 7fc37f7fe640 1 -- 172.21.15.40:0/3335103411 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378021030 con 0x55fb347c1270 2024-04-26T21:47:37.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.044+0000 7fc37f7fe640 1 -- 172.21.15.40:0/3335103411 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc378011b50 con 0x55fb347c1270 2024-04-26T21:47:37.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.044+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fb34735840 con 0x55fb347c1270 2024-04-26T21:47:37.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.044+0000 7fc37f7fe640 1 -- 172.21.15.40:0/3335103411 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378011d60 con 0x55fb347c1270 2024-04-26T21:47:37.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.044+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb34723740 con 0x55fb347c1270 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.045+0000 7fc37f7fe640 1 -- 172.21.15.40:0/3335103411 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc378024030 con 0x55fb347c1270 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.046+0000 7fc37f7fe640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc36806b2d0 0x7fc36806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.046+0000 7fc37f7fe640 1 -- 172.21.15.40:0/3335103411 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fc37808b820 con 0x55fb347c1270 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.046+0000 7fc386263640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc36806b2d0 0x7fc36806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.046+0000 7fc388769f40 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55fb347b4a90 0x55fb347c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.046+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e146) -- 0x55fb347b3840 con 0x55fb347b4a90 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.046+0000 7fc385a62640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55fb347b4a90 0x55fb347c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.046+0000 7fc385a62640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55fb347b4a90 0x55fb347c6c30 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.046+0000 7fc386263640 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc36806b2d0 0x7fc36806d790 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc380009bc0 tx=0x7fc380051000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc385a62640 1 -- 172.21.15.40:0/3335103411 <== osd.3 v2:172.21.15.40:6824/4238362919 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (91) out=91b] v0'0 uv6 ondisk = 0) ==== 164+0+91 (crc 0 0 0) 0x7fc37000b830 con 0x55fb347b4a90 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55fb347b4a90 msgr2=0x55fb347c6c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55fb347b4a90 0x55fb347c6c30 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc36806b2d0 msgr2=0x7fc36806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc36806b2d0 0x7fc36806d790 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc380009bc0 tx=0x7fc380051000 comp rx=0 tx=0).stop 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347c1270 msgr2=0x55fb347b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 --2- 172.21.15.40:0/3335103411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fb347c1270 0x55fb347b4550 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x55fb346cbe20 tx=0x7fc3780015b0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc386a64640 1 -- 172.21.15.40:0/3335103411 reap_dead start 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 shutdown_connections 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 >> 172.21.15.40:0/3335103411 conn(0x55fb3471f8a0 msgr2=0x55fb34720f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 shutdown_connections 2024-04-26T21:47:37.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.047+0000 7fc388769f40 1 -- 172.21.15.40:0/3335103411 wait complete. 2024-04-26T21:47:37.052 INFO:teuthology.orchestra.run.smithi040.stdout:Ufile1dir1 2024-04-26T21:47:37.052 DEBUG:teuthology.orchestra.run.smithi040:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627777, 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi040.stdout: "ancestors": [ 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi040.stdout: "dirino": 1099511627776, 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "file1", 2024-04-26T21:47:37.167 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 8 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: "dirino": 1, 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "dir1", 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 29 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: "old_pools": [] 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:37.183 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent rhubarb 2024-04-26T21:47:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.267+0000 7f7174663f40 1 Processor -- start 2024-04-26T21:47:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f7174663f40 1 -- start start 2024-04-26T21:47:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f7174663f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160003ab0 0x55b74702c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f7174663f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7470f5aa0 0x55b747055ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f7174663f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b747056910 0x55b747056d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f7174663f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b746fd1e40 con 0x55b7470f5aa0 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f7174663f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b746fd5e80 con 0x7f7160003ab0 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f7174663f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b747058d50 con 0x55b747056910 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f717195c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7470f5aa0 0x55b747055ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f717195c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7470f5aa0 0x55b747055ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56420/0 (socket says 172.21.15.40:56420) 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f717195c640 1 -- 172.21.15.40:0/418505448 learned_addr learned my addr 172.21.15.40:0/418505448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f717195c640 1 -- 172.21.15.40:0/418505448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b747056910 msgr2=0x55b747056d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f717195c640 1 --2- 172.21.15.40:0/418505448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b747056910 0x55b747056d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f717195c640 1 -- 172.21.15.40:0/418505448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160003ab0 msgr2=0x55b74702c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.268+0000 7f717195c640 1 --2- 172.21.15.40:0/418505448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160003ab0 0x55b74702c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f717195c640 1 -- 172.21.15.40:0/418505448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b74702d870 con 0x55b7470f5aa0 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f717195c640 1 --2- 172.21.15.40:0/418505448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7470f5aa0 0x55b747055ec0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f715c0097e0 tx=0x7f715c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23a1f9be2fccb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f717095a640 1 -- 172.21.15.40:0/418505448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c007830 con 0x55b7470f5aa0 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f717095a640 1 -- 172.21.15.40:0/418505448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715c0079d0 con 0x55b7470f5aa0 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f717095a640 1 -- 172.21.15.40:0/418505448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c007d80 con 0x55b7470f5aa0 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 -- 172.21.15.40:0/418505448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7470f5aa0 msgr2=0x55b747055ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 --2- 172.21.15.40:0/418505448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7470f5aa0 0x55b747055ec0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f715c0097e0 tx=0x7f715c005a00 comp rx=0 tx=0).stop 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 -- 172.21.15.40:0/418505448 shutdown_connections 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 --2- 172.21.15.40:0/418505448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b747056910 0x55b747056d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 --2- 172.21.15.40:0/418505448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7470f5aa0 0x55b747055ec0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 --2- 172.21.15.40:0/418505448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160003ab0 0x55b74702c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 -- 172.21.15.40:0/418505448 >> 172.21.15.40:0/418505448 conn(0x55b7470559c0 msgr2=0x55b747072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 -- 172.21.15.40:0/418505448 shutdown_connections 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.269+0000 7f7174663f40 1 -- 172.21.15.40:0/418505448 wait complete. 2024-04-26T21:47:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.270+0000 7f7174663f40 1 Processor -- start 2024-04-26T21:47:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.270+0000 7f7174663f40 1 -- start start 2024-04-26T21:47:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f7174663f40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b747056910 0x55b74719bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f7174663f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7470f5aa0 0x55b747100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f7174663f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7160003ab0 0x55b747072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f7174663f40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b7470ed1d0 con 0x55b747056910 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f7174663f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b7470f5330 con 0x55b7470f5aa0 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f7174663f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55b74702def0 con 0x7f7160003ab0 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b747056910 0x55b74719bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b747056910 0x55b74719bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56422/0 (socket says 172.21.15.40:56422) 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 -- 172.21.15.40:0/259440513 learned_addr learned my addr 172.21.15.40:0/259440513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717195c640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7470f5aa0 0x55b747100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717295e640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7160003ab0 0x55b747072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 -- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7160003ab0 msgr2=0x55b747072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7160003ab0 0x55b747072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 -- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7470f5aa0 msgr2=0x55b747100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b7470f5aa0 0x55b747100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 -- 172.21.15.40:0/259440513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7470f2b00 con 0x55b747056910 2024-04-26T21:47:37.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f717215d640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b747056910 0x55b74719bb50 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f716c009f10 tx=0x7f716c00e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f716b7fe640 1 -- 172.21.15.40:0/259440513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c00f650 con 0x55b747056910 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f716b7fe640 1 -- 172.21.15.40:0/259440513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c013030 con 0x55b747056910 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7470f36d0 con 0x55b747056910 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f716b7fe640 1 -- 172.21.15.40:0/259440513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c00fa10 con 0x55b747056910 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.271+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7470f3080 con 0x55b747056910 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.273+0000 7f716b7fe640 1 -- 172.21.15.40:0/259440513 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f716c00fbb0 con 0x55b747056910 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.273+0000 7f716b7fe640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f715406b300 0x7f715406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.273+0000 7f716b7fe640 1 -- 172.21.15.40:0/259440513 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f716c08b1d0 con 0x55b747056910 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.273+0000 7f717195c640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f715406b300 0x7f715406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.273+0000 7f7174663f40 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7470f3960 0x55b7471a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.273+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [setxattr parent (7) in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146) -- 0x55b7471a56d0 con 0x55b7470f3960 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.274+0000 7f717295e640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7470f3960 0x55b7471a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.274+0000 7f717195c640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f715406b300 0x7f715406d7c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x55b7470febb0 tx=0x7f715c007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.274+0000 7f717295e640 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7470f3960 0x55b7471a5190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f717295e640 1 -- 172.21.15.40:0/259440513 <== osd.3 v2:172.21.15.40:6824/4238362919 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (7)] v146'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7164003030 con 0x55b7470f3960 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7470f3960 msgr2=0x55b7471a5190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x55b7470f3960 0x55b7471a5190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f715406b300 msgr2=0x7f715406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f715406b300 0x7f715406d7c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x55b7470febb0 tx=0x7f715c007000 comp rx=0 tx=0).stop 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b747056910 msgr2=0x55b74719bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 --2- 172.21.15.40:0/259440513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b747056910 0x55b74719bb50 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f716c009f10 tx=0x7f716c00e7b0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f717295e640 1 -- 172.21.15.40:0/259440513 reap_dead start 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 shutdown_connections 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 >> 172.21.15.40:0/259440513 conn(0x55b7470559c0 msgr2=0x55b747056cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 shutdown_connections 2024-04-26T21:47:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.275+0000 7f7174663f40 1 -- 172.21.15.40:0/259440513 wait complete. 2024-04-26T21:47:37.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:37.279 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink1) 2024-04-26T21:47:37.296 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:47:37.296+0000 7f9287375640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 104: Malformed input [buffer:3] 2024-04-26T21:47:37.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:47:37.297 INFO:teuthology.orchestra.run.smithi040.stderr:stat: cannot statx 'testdir/hardlink1': Input/output error 2024-04-26T21:47:37.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:37.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.439+0000 7f2e882fc640 1 Processor -- start 2024-04-26T21:47:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.439+0000 7f2e882fc640 1 -- start start 2024-04-26T21:47:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e882fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e8006fea0 0x7f2e80166d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e882fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e80160970 con 0x7f2e801604e0 2024-04-26T21:47:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e882fc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e80072f70 con 0x7f2e8015f2d0 2024-04-26T21:47:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e882fc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2e80074010 con 0x7f2e8006fea0 2024-04-26T21:47:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e86899640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e8006fea0 0x7f2e80166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e85897640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2e801604e0 0x7f2e8006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53102/0 (socket says 172.21.15.40:53102) 2024-04-26T21:47:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e85897640 1 -- 172.21.15.40:0/2345096203 learned_addr learned my addr 172.21.15.40:0/2345096203 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e86899640 1 -- 172.21.15.40:0/2345096203 >> v1:172.21.15.184:6789/0 conn(0x7f2e8015f2d0 legacy=0x7f2e80072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e86899640 1 -- 172.21.15.40:0/2345096203 >> v1:172.21.15.40:6789/0 conn(0x7f2e801604e0 legacy=0x7f2e8006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e85096640 1 -- 172.21.15.40:0/2345096203 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735143806 0 0) 0x7f2e80160970 con 0x7f2e801604e0 2024-04-26T21:47:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.440+0000 7f2e86899640 1 -- 172.21.15.40:0/2345096203 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e80073250 con 0x7f2e8006fea0 2024-04-26T21:47:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.441+0000 7f2e86899640 1 --2- 172.21.15.40:0/2345096203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8006fea0 0x7f2e80166d80 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009d90 tx=0x7f2e7002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366e9499d2027da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.441+0000 7f2e85096640 1 -- 172.21.15.40:0/2345096203 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70007740 con 0x7f2e8006fea0 2024-04-26T21:47:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.441+0000 7f2e85096640 1 -- 172.21.15.40:0/2345096203 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e700078e0 con 0x7f2e8006fea0 2024-04-26T21:47:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.441+0000 7f2e85096640 1 -- 172.21.15.40:0/2345096203 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70007cb0 con 0x7f2e8006fea0 2024-04-26T21:47:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.442+0000 7f2e882fc640 1 -- 172.21.15.40:0/2345096203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8006fea0 msgr2=0x7f2e80166d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.442+0000 7f2e882fc640 1 --2- 172.21.15.40:0/2345096203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8006fea0 0x7f2e80166d80 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009d90 tx=0x7f2e7002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.442+0000 7f2e882fc640 1 -- 172.21.15.40:0/2345096203 shutdown_connections 2024-04-26T21:47:37.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.442+0000 7f2e882fc640 1 --2- 172.21.15.40:0/2345096203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8006fea0 0x7f2e80166d80 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.442+0000 7f2e882fc640 1 -- 172.21.15.40:0/2345096203 >> 172.21.15.40:0/2345096203 conn(0x7f2e800d0be0 msgr2=0x7f2e800d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.442+0000 7f2e882fc640 1 -- 172.21.15.40:0/2345096203 shutdown_connections 2024-04-26T21:47:37.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.442+0000 7f2e882fc640 1 -- 172.21.15.40:0/2345096203 wait complete. 2024-04-26T21:47:37.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.443+0000 7f2e882fc640 1 Processor -- start 2024-04-26T21:47:37.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.443+0000 7f2e882fc640 1 -- start start 2024-04-26T21:47:37.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e882fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8006fea0 0x7f2e8014e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e882fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8015f2d0 0x7f2e80154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e882fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e801604e0 0x7f2e80155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e882fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80073af0 con 0x7f2e8015f2d0 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e882fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80073810 con 0x7f2e8006fea0 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e882fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8015f6b0 con 0x7f2e801604e0 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8015f2d0 0x7f2e80154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8015f2d0 0x7f2e80154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56442/0 (socket says 172.21.15.40:56442) 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 -- 172.21.15.40:0/2599325405 learned_addr learned my addr 172.21.15.40:0/2599325405 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e86098640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8006fea0 0x7f2e8014e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e86899640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e801604e0 0x7f2e80155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 -- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e801604e0 msgr2=0x7f2e80155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e801604e0 0x7f2e80155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 -- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8006fea0 msgr2=0x7f2e8014e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8006fea0 0x7f2e8014e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 -- 172.21.15.40:0/2599325405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8014e690 con 0x7f2e8015f2d0 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e86899640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e801604e0 0x7f2e80155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.444+0000 7f2e85897640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8015f2d0 0x7f2e80154dc0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400c460 tx=0x7f2e7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.445+0000 7f2e7effd640 1 -- 172.21.15.40:0/2599325405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74018070 con 0x7f2e8015f2d0 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.445+0000 7f2e7effd640 1 -- 172.21.15.40:0/2599325405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e74007500 con 0x7f2e8015f2d0 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.445+0000 7f2e7effd640 1 -- 172.21.15.40:0/2599325405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74004a20 con 0x7f2e8015f2d0 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.445+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e800c57f0 con 0x7f2e8015f2d0 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.445+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e800c5db0 con 0x7f2e8015f2d0 2024-04-26T21:47:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.446+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e800694f0 con 0x7f2e8015f2d0 2024-04-26T21:47:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.446+0000 7f2e7effd640 1 -- 172.21.15.40:0/2599325405 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e7401b070 con 0x7f2e8015f2d0 2024-04-26T21:47:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.446+0000 7f2e7effd640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e6c06c8f0 0x7f2e6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.447+0000 7f2e7effd640 1 -- 172.21.15.40:0/2599325405 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f2e7408af60 con 0x7f2e8015f2d0 2024-04-26T21:47:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.447+0000 7f2e86098640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e6c06c8f0 0x7f2e6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.447+0000 7f2e86098640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e6c06c8f0 0x7f2e6c06edb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2e801f86e0 tx=0x7f2e78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.451+0000 7f2e7effd640 1 -- 172.21.15.40:0/2599325405 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e8014e690 con 0x7f2e8015f2d0 2024-04-26T21:47:37.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.676+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e800b2520 con 0x7f2e8015f2d0 2024-04-26T21:47:37.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.676+0000 7f2e7effd640 1 -- 172.21.15.40:0/2599325405 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7f2e740591b0 con 0x7f2e8015f2d0 2024-04-26T21:47:37.676 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:37.677 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:37.677 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:37.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e6c06c8f0 msgr2=0x7f2e6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e6c06c8f0 0x7f2e6c06edb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2e801f86e0 tx=0x7f2e78009040 comp rx=0 tx=0).stop 2024-04-26T21:47:37.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8015f2d0 msgr2=0x7f2e80154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8015f2d0 0x7f2e80154dc0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400c460 tx=0x7f2e7400c920 comp rx=0 tx=0).stop 2024-04-26T21:47:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 shutdown_connections 2024-04-26T21:47:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2e6c06c8f0 0x7f2e6c06edb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e801604e0 0x7f2e80155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8015f2d0 0x7f2e80154dc0 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 --2- 172.21.15.40:0/2599325405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8006fea0 0x7f2e8014e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 >> 172.21.15.40:0/2599325405 conn(0x7f2e800d0be0 msgr2=0x7f2e8015e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 shutdown_connections 2024-04-26T21:47:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.678+0000 7f2e882fc640 1 -- 172.21.15.40:0/2599325405 wait complete. 2024-04-26T21:47:37.689 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-26T21:47:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.792+0000 7f67c702c640 1 Processor -- start 2024-04-26T21:47:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.792+0000 7f67c702c640 1 -- start start 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c702c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67c015f3a0 0x7f67c00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c702c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 0x7f67c00d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c702c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f67c00d2130 0x7f67c00d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c702c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f67c00d25a0 con 0x7f67c015f3a0 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c702c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f67c00d0e20 con 0x7f67c00d2130 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c702c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67c00b1d70 con 0x7f67c00d1770 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c4dc8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67c015f3a0 0x7f67c00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c4dc8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67c015f3a0 0x7f67c00d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56448/0 (socket says 172.21.15.40:56448) 2024-04-26T21:47:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.793+0000 7f67c4dc8640 1 -- 172.21.15.40:0/3025330883 learned_addr learned my addr 172.21.15.40:0/3025330883 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.794+0000 7f67bffff640 1 --2- 172.21.15.40:0/3025330883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 0x7f67c00d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.794+0000 7f67c55c9640 1 --2- 172.21.15.40:0/3025330883 >> v2:172.21.15.184:3300/0 conn(0x7f67c00d2130 0x7f67c00d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.794+0000 7f67c4dc8640 1 -- 172.21.15.40:0/3025330883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 msgr2=0x7f67c00d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.794+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/3025330883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 0x7f67c00d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.794+0000 7f67c4dc8640 1 -- 172.21.15.40:0/3025330883 >> v2:172.21.15.184:3300/0 conn(0x7f67c00d2130 msgr2=0x7f67c00d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.794+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/3025330883 >> v2:172.21.15.184:3300/0 conn(0x7f67c00d2130 0x7f67c00d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.794+0000 7f67c4dc8640 1 -- 172.21.15.40:0/3025330883 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c00b2050 con 0x7f67c015f3a0 2024-04-26T21:47:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.794+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/3025330883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00d06f0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f67c008e8d0 tx=0x7f67b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5caf29646d09f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67bf7fe640 1 -- 172.21.15.40:0/3025330883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4037070 con 0x7f67c015f3a0 2024-04-26T21:47:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67bf7fe640 1 -- 172.21.15.40:0/3025330883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b4032440 con 0x7f67c015f3a0 2024-04-26T21:47:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67bf7fe640 1 -- 172.21.15.40:0/3025330883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4032810 con 0x7f67c015f3a0 2024-04-26T21:47:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 -- 172.21.15.40:0/3025330883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 msgr2=0x7f67c00d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 --2- 172.21.15.40:0/3025330883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00d06f0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f67c008e8d0 tx=0x7f67b402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 -- 172.21.15.40:0/3025330883 shutdown_connections 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 --2- 172.21.15.40:0/3025330883 >> v2:172.21.15.184:3300/0 conn(0x7f67c00d2130 0x7f67c00d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 --2- 172.21.15.40:0/3025330883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 0x7f67c00d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 --2- 172.21.15.40:0/3025330883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00d06f0 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 -- 172.21.15.40:0/3025330883 >> 172.21.15.40:0/3025330883 conn(0x7f67c00d4f90 msgr2=0x7f67c00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 -- 172.21.15.40:0/3025330883 shutdown_connections 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.795+0000 7f67c702c640 1 -- 172.21.15.40:0/3025330883 wait complete. 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.796+0000 7f67c702c640 1 Processor -- start 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.796+0000 7f67c702c640 1 -- start start 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c702c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 0x7f67c0159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c702c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c00d2130 0x7f67c00b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c702c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c702c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f67c015f900 con 0x7f67c015f3a0 2024-04-26T21:47:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c702c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f67c015f780 con 0x7f67c00d2130 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c702c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67c00d3390 con 0x7f67c00d1770 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c55c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c55c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00b9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56452/0 (socket says 172.21.15.40:56452) 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.797+0000 7f67c55c9640 1 -- 172.21.15.40:0/85354208 learned_addr learned my addr 172.21.15.40:0/85354208 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 0x7f67c0159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67bffff640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c00d2130 0x7f67c00b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c55c9640 1 -- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 msgr2=0x7f67c0159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c55c9640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 0x7f67c0159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c55c9640 1 -- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c00d2130 msgr2=0x7f67c00b91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c55c9640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c00d2130 0x7f67c00b91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c55c9640 1 -- 172.21.15.40:0/85354208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c00b8140 con 0x7f67c015f3a0 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c55c9640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00b9720 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f67b8007e70 tx=0x7f67b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67bd7fa640 1 -- 172.21.15.40:0/85354208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8014620 con 0x7f67c015f3a0 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67bd7fa640 1 -- 172.21.15.40:0/85354208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b8019070 con 0x7f67c015f3a0 2024-04-26T21:47:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c702c640 1 -- 172.21.15.40:0/85354208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c00b8420 con 0x7f67c015f3a0 2024-04-26T21:47:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67bd7fa640 1 -- 172.21.15.40:0/85354208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8014a20 con 0x7f67c015f3a0 2024-04-26T21:47:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.798+0000 7f67c702c640 1 -- 172.21.15.40:0/85354208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c00b89e0 con 0x7f67c015f3a0 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.800+0000 7f67bd7fa640 1 -- 172.21.15.40:0/85354208 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67b8014bc0 con 0x7f67c015f3a0 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.800+0000 7f67bd7fa640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f67a406c920 0x7f67a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.800+0000 7f67bd7fa640 1 -- 172.21.15.40:0/85354208 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f67b808c460 con 0x7f67c015f3a0 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.801+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f67a406c920 0x7f67a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.801+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f67a406c920 0x7f67a406ede0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f67b4034000 tx=0x7f67b402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.801+0000 7f67c9e20740 1 Processor -- start 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.801+0000 7f67c9e20740 1 -- start start 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db2300cf0 0x563db2302e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db233d3c0 0x563db23010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c9e20740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563db2265b80 con 0x563db232b7b0 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c9e20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563db22014d0 con 0x563db233d3c0 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c9e20740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563db2261b40 con 0x563db2300cf0 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c55c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c55c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56458/0 (socket says 172.21.15.40:56458) 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67bffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db233d3c0 0x563db23010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c4dc8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db2300cf0 0x563db2302e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c55c9640 1 -- 172.21.15.40:0/318529022 learned_addr learned my addr 172.21.15.40:0/318529022 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c55c9640 1 -- 172.21.15.40:0/318529022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db2300cf0 msgr2=0x563db2302e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c55c9640 1 --2- 172.21.15.40:0/318529022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db2300cf0 0x563db2302e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c55c9640 1 -- 172.21.15.40:0/318529022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db233d3c0 msgr2=0x563db23010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c55c9640 1 --2- 172.21.15.40:0/318529022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db233d3c0 0x563db23010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.802+0000 7f67c55c9640 1 -- 172.21.15.40:0/318529022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563db222e770 con 0x563db232b7b0 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67bffff640 1 --2- 172.21.15.40:0/318529022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db233d3c0 0x563db23010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67c55c9640 1 --2- 172.21.15.40:0/318529022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232bc30 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c040 tx=0x7f67b805e3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c7f27f9ce3af59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67aa7fc640 1 -- 172.21.15.40:0/318529022 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b805e0c0 con 0x563db232b7b0 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67aa7fc640 1 -- 172.21.15.40:0/318529022 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b8056760 con 0x563db232b7b0 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67aa7fc640 1 -- 172.21.15.40:0/318529022 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b808c2b0 con 0x563db232b7b0 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67c9e20740 1 -- 172.21.15.40:0/318529022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 msgr2=0x563db232bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67c9e20740 1 --2- 172.21.15.40:0/318529022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232bc30 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c040 tx=0x7f67b805e3f0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67c9e20740 1 -- 172.21.15.40:0/318529022 shutdown_connections 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67c9e20740 1 --2- 172.21.15.40:0/318529022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232bc30 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67c9e20740 1 --2- 172.21.15.40:0/318529022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db233d3c0 0x563db23010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67c9e20740 1 --2- 172.21.15.40:0/318529022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db2300cf0 0x563db2302e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.803+0000 7f67c9e20740 1 -- 172.21.15.40:0/318529022 >> 172.21.15.40:0/318529022 conn(0x563db2295e90 msgr2=0x563db23467b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.804+0000 7f67c9e20740 1 -- 172.21.15.40:0/318529022 shutdown_connections 2024-04-26T21:47:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.804+0000 7f67c9e20740 1 -- 172.21.15.40:0/318529022 wait complete. 2024-04-26T21:47:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.804+0000 7f67c9e20740 1 -- start start 2024-04-26T21:47:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.804+0000 7f67c9e20740 1 Processor -- start 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db231c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2327db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c9e20740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563db21fee70 con 0x563db232b7b0 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c9e20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563db2329300 con 0x563db2300cf0 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c9e20740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563db23578e0 con 0x563db233d3c0 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c9e20740 10 client.0 _ll_register_callbacks cb 0x563db220fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56470/0 (socket says 172.21.15.40:56470) 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67bffff640 1 -- 172.21.15.40:0/3834478484 learned_addr learned my addr 172.21.15.40:0/3834478484 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db231c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c55c9640 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2327db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67bffff640 1 -- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 msgr2=0x563db2327db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67bffff640 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2327db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67bffff640 1 -- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 msgr2=0x563db231c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67bffff640 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db231c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67bffff640 1 -- 172.21.15.40:0/3834478484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563db20437c0 con 0x563db232b7b0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.805+0000 7f67c55c9640 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2327db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67bffff640 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232dd50 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f67ac003e70 tx=0x7f67ac00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b311132f7fc6f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f679f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f679f7fe640 1 -- 172.21.15.40:0/3834478484 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac019070 con 0x563db232b7b0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f679f7fe640 1 -- 172.21.15.40:0/3834478484 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f67ac014680 con 0x563db232b7b0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 1 -- 172.21.15.40:0/3834478484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67b8056760 con 0x563db232b7b0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f679f7fe640 1 -- 172.21.15.40:0/3834478484 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f67ac024050 con 0x563db232b7b0 2024-04-26T21:47:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 10 client.12066 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 1 -- 172.21.15.40:0/3834478484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x563db20a3f50 con 0x563db232b7b0 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f679f7fe640 1 -- 172.21.15.40:0/3834478484 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b8056760 con 0x563db232b7b0 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f679f7fe640 1 -- 172.21.15.40:0/3834478484 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f67ac027020 con 0x563db232b7b0 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 10 client.12066 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 10 client.12066 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 20 client.12066 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x563db232bd70 0x563db23566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 4 client.12066 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.806+0000 7f67c9e20740 1 -- 172.21.15.40:0/3834478484 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563db2219240 con 0x563db232bd70 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.807+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x563db232bd70 0x563db23566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.807+0000 7f679f7fe640 10 client.12066 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.807+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x563db232bd70 0x563db23566f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c826d6ddbb30d252 server_cookie=d730673178cf9607 in_seq=0 out_seq=0 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.807+0000 7f679f7fe640 1 -- 172.21.15.40:0/3834478484 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563db2219240 con 0x563db232bd70 2024-04-26T21:47:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.807+0000 7f679f7fe640 10 client.12066 handle_command_reply: tid=0 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.807+0000 7f67c9e20740 1 client.12066 shutdown 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 -- 172.21.15.40:0/3834478484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 msgr2=0x563db232dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232dd50 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f67ac003e70 tx=0x7f67ac00b8f0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 -- 172.21.15.40:0/3834478484 shutdown_connections 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x563db232bd70 0x563db23566f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2327db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f679f7fe640 0 client.12066 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db232b7b0 0x563db232dd50 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 --2- 172.21.15.40:0/3834478484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db231c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 -- 172.21.15.40:0/3834478484 >> 172.21.15.40:0/3834478484 conn(0x563db2295e90 msgr2=0x563db23294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 -- 172.21.15.40:0/3834478484 shutdown_connections 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 1 -- 172.21.15.40:0/3834478484 wait complete. 2024-04-26T21:47:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.808+0000 7f67c9e20740 20 client.12066 trim_cache size 0 max 16384 2024-04-26T21:47:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.821+0000 7f67c9e20740 1 Processor -- start 2024-04-26T21:47:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.821+0000 7f67c9e20740 1 -- start start 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db2329360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db232b7b0 0x563db23482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db233d3c0 0x563db23487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c9e20740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f67c0200ec0 con 0x563db233d3c0 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c9e20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f67c00c8cd0 con 0x563db2300cf0 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c9e20740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67c00c9320 con 0x563db232b7b0 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db233d3c0 0x563db23487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db233d3c0 0x563db23487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56476/0 (socket says 172.21.15.40:56476) 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 -- 172.21.15.40:0/4244808755 learned_addr learned my addr 172.21.15.40:0/4244808755 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67bffff640 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db232b7b0 0x563db23482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 -- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db232b7b0 msgr2=0x563db23482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db2329360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db232b7b0 0x563db23482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 -- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 msgr2=0x563db2329360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db2329360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 -- 172.21.15.40:0/4244808755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563db1f370b0 con 0x563db233d3c0 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67bffff640 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db232b7b0 0x563db23482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db2329360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.822+0000 7f67c55c9640 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db233d3c0 0x563db23487f0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f67b80583a0 tx=0x7f67b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f326b585181b2bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67a9ffb640 1 -- 172.21.15.40:0/4244808755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b805e0b0 con 0x563db233d3c0 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67a9ffb640 1 -- 172.21.15.40:0/4244808755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b808a3f0 con 0x563db233d3c0 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67a9ffb640 1 -- 172.21.15.40:0/4244808755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b808c0b0 con 0x563db233d3c0 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 -- 172.21.15.40:0/4244808755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db233d3c0 msgr2=0x563db23487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db233d3c0 0x563db23487f0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f67b80583a0 tx=0x7f67b8016000 comp rx=0 tx=0).stop 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 -- 172.21.15.40:0/4244808755 shutdown_connections 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db233d3c0 0x563db23487f0 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db232b7b0 0x563db23482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 --2- 172.21.15.40:0/4244808755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db2300cf0 0x563db2329360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 -- 172.21.15.40:0/4244808755 >> 172.21.15.40:0/4244808755 conn(0x563db2295e90 msgr2=0x563db2327950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 -- 172.21.15.40:0/4244808755 shutdown_connections 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.823+0000 7f67c9e20740 1 -- 172.21.15.40:0/4244808755 wait complete. 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.824+0000 7f67c9e20740 1 -- start start 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.824+0000 7f67c9e20740 1 Processor -- start 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db2300cf0 0x563db23c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db232b7b0 0x563db2356650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c9e20740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2356b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c9e20740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563db232bbf0 con 0x563db2300cf0 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c9e20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563db23282f0 con 0x563db232b7b0 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c9e20740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x563db232c180 con 0x563db233d3c0 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c9e20740 10 client.0 _ll_register_callbacks cb 0x7f67ac0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db2300cf0 0x563db23c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db2300cf0 0x563db23c0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56480/0 (socket says 172.21.15.40:56480) 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 -- 172.21.15.40:0/1955947422 learned_addr learned my addr 172.21.15.40:0/1955947422 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c55c9640 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2356b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67bffff640 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db232b7b0 0x563db2356650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 -- 172.21.15.40:0/1955947422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 msgr2=0x563db2356b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2356b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 -- 172.21.15.40:0/1955947422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db232b7b0 msgr2=0x563db2356650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db232b7b0 0x563db2356650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 -- 172.21.15.40:0/1955947422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563db22ffc80 con 0x563db2300cf0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c4dc8640 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db2300cf0 0x563db23c0780 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f67b40090b0 tx=0x7f67b402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac76f8ddbbc0069f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f679f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f679f7fe640 1 -- 172.21.15.40:0/1955947422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4037040 con 0x563db2300cf0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f679f7fe640 1 -- 172.21.15.40:0/1955947422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f67b40328e0 con 0x563db2300cf0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.825+0000 7f67c9e20740 1 -- 172.21.15.40:0/1955947422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67b808a3f0 con 0x563db2300cf0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f679f7fe640 1 -- 172.21.15.40:0/1955947422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f67b405f050 con 0x563db2300cf0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 10 client.12072 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 1 -- 172.21.15.40:0/1955947422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x563db22fff60 con 0x563db2300cf0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f679f7fe640 1 -- 172.21.15.40:0/1955947422 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b808a3f0 con 0x563db2300cf0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f679f7fe640 1 -- 172.21.15.40:0/1955947422 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f67b4062020 con 0x563db2300cf0 2024-04-26T21:47:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 10 client.12072 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 10 client.12072 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 20 client.12072 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x563db2340350 0x563db2354e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 4 client.12072 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 1 -- 172.21.15.40:0/1955947422 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x563db2273790 con 0x563db2340350 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67bffff640 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x563db2340350 0x563db2354e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67bffff640 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x563db2340350 0x563db2354e20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748280697570dac0 server_cookie=47eeb2637049cba5 in_seq=0 out_seq=0 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f679f7fe640 10 client.12072 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f679f7fe640 1 -- 172.21.15.40:0/1955947422 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+145 (crc 0 0 0) 0x563db2273790 con 0x563db2340350 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f679f7fe640 10 client.12072 handle_command_reply: tid=0 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.826+0000 7f67c9e20740 1 client.12072 shutdown 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 -- 172.21.15.40:0/1955947422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db2300cf0 msgr2=0x563db23c0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db2300cf0 0x563db23c0780 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f67b40090b0 tx=0x7f67b402c070 comp rx=0 tx=0).stop 2024-04-26T21:47:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 -- 172.21.15.40:0/1955947422 shutdown_connections 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x563db2340350 0x563db2354e20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563db233d3c0 0x563db2356b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563db232b7b0 0x563db2356650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 --2- 172.21.15.40:0/1955947422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563db2300cf0 0x563db23c0780 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f679f7fe640 0 client.12072 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 -- 172.21.15.40:0/1955947422 >> 172.21.15.40:0/1955947422 conn(0x563db2295e90 msgr2=0x563db2329360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 -- 172.21.15.40:0/1955947422 shutdown_connections 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 1 -- 172.21.15.40:0/1955947422 wait complete. 2024-04-26T21:47:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c9e20740 20 client.12072 trim_cache size 0 max 16384 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2964369376, 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627777, 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/testdir/hardlink1" 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:37.835 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:47:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c702c640 1 -- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f67a406c920 msgr2=0x7f67a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c702c640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f67a406c920 0x7f67a406ede0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f67b4034000 tx=0x7f67b402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c702c640 1 -- 172.21.15.40:0/85354208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 msgr2=0x7f67c00b9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.827+0000 7f67c702c640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00b9720 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f67b8007e70 tx=0x7f67b800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.828+0000 7f67c702c640 1 -- 172.21.15.40:0/85354208 shutdown_connections 2024-04-26T21:47:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.828+0000 7f67c702c640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f67a406c920 0x7f67a406ede0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.828+0000 7f67c702c640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67c015f3a0 0x7f67c00b9720 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.828+0000 7f67c702c640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67c00d2130 0x7f67c00b91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.828+0000 7f67c702c640 1 --2- 172.21.15.40:0/85354208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67c00d1770 0x7f67c0159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.828+0000 7f67c702c640 1 -- 172.21.15.40:0/85354208 >> 172.21.15.40:0/85354208 conn(0x7f67c00d4f90 msgr2=0x7f67c006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.828+0000 7f67c702c640 1 -- 172.21.15.40:0/85354208 shutdown_connections 2024-04-26T21:47:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.828+0000 7f67c702c640 1 -- 172.21.15.40:0/85354208 wait complete. 2024-04-26T21:47:37.838 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:37.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.939+0000 7fec5a3f0640 1 Processor -- start 2024-04-26T21:47:37.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.939+0000 7fec5a3f0640 1 -- start start 2024-04-26T21:47:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec5a3f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec5415f1d0 0x7fec5406f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec5a3f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec54073500 con 0x7fec540afd60 2024-04-26T21:47:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec5a3f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec540b4480 con 0x7fec5406c240 2024-04-26T21:47:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec5a3f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fec540ae8c0 con 0x7fec5415f1d0 2024-04-26T21:47:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec53fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec5415f1d0 0x7fec5406f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec5898d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fec540afd60 0x7fec54072df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53104/0 (socket says 172.21.15.40:53104) 2024-04-26T21:47:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec5898d640 1 -- 172.21.15.40:0/2015915039 learned_addr learned my addr 172.21.15.40:0/2015915039 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec53fff640 1 -- 172.21.15.40:0/2015915039 >> v1:172.21.15.184:6789/0 conn(0x7fec5406c240 legacy=0x7fec540726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:37.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec53fff640 1 -- 172.21.15.40:0/2015915039 >> v1:172.21.15.40:6789/0 conn(0x7fec540afd60 legacy=0x7fec54072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.940+0000 7fec53fff640 1 -- 172.21.15.40:0/2015915039 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec540b4720 con 0x7fec5415f1d0 2024-04-26T21:47:37.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.941+0000 7fec53fff640 1 --2- 172.21.15.40:0/2015915039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5415f1d0 0x7fec5406f170 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0097a0 tx=0x7fec3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b228e9dc6f82332d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.941+0000 7fec52ffd640 1 -- 172.21.15.40:0/2015915039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c037070 con 0x7fec5415f1d0 2024-04-26T21:47:37.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.941+0000 7fec52ffd640 1 -- 172.21.15.40:0/2015915039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c032440 con 0x7fec5415f1d0 2024-04-26T21:47:37.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.941+0000 7fec52ffd640 1 -- 172.21.15.40:0/2015915039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c032810 con 0x7fec5415f1d0 2024-04-26T21:47:37.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.942+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2015915039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5415f1d0 msgr2=0x7fec5406f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.942+0000 7fec5a3f0640 1 --2- 172.21.15.40:0/2015915039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5415f1d0 0x7fec5406f170 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0097a0 tx=0x7fec3c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.942+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2015915039 shutdown_connections 2024-04-26T21:47:37.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.942+0000 7fec5a3f0640 1 --2- 172.21.15.40:0/2015915039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5415f1d0 0x7fec5406f170 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.942+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2015915039 >> 172.21.15.40:0/2015915039 conn(0x7fec540b3180 msgr2=0x7fec540b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.942+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2015915039 shutdown_connections 2024-04-26T21:47:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.942+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2015915039 wait complete. 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.943+0000 7fec5a3f0640 1 Processor -- start 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.943+0000 7fec5a3f0640 1 -- start start 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec5a3f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5406c240 0x7fec5414ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec5a3f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec540afd60 0x7fec54160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec5a3f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec5415f1d0 0x7fec5415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec5a3f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec540b4fc0 con 0x7fec5406c240 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec5a3f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec540b4ce0 con 0x7fec540afd60 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec5a3f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec5406f6b0 con 0x7fec5415f1d0 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec53fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5406c240 0x7fec5414ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec53fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5406c240 0x7fec5414ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56500/0 (socket says 172.21.15.40:56500) 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec53fff640 1 -- 172.21.15.40:0/2780115175 learned_addr learned my addr 172.21.15.40:0/2780115175 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec53fff640 1 -- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec5415f1d0 msgr2=0x7fec5415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec53fff640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec5415f1d0 0x7fec5415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec53fff640 1 -- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec540afd60 msgr2=0x7fec54160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec53fff640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec540afd60 0x7fec54160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.944+0000 7fec53fff640 1 -- 172.21.15.40:0/2780115175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec54150500 con 0x7fec5406c240 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.945+0000 7fec53fff640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5406c240 0x7fec5414ffc0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0098c0 tx=0x7fec3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.945+0000 7fec50ff9640 1 -- 172.21.15.40:0/2780115175 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c037040 con 0x7fec5406c240 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.945+0000 7fec50ff9640 1 -- 172.21.15.40:0/2780115175 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c042070 con 0x7fec5406c240 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.945+0000 7fec50ff9640 1 -- 172.21.15.40:0/2780115175 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c030a70 con 0x7fec5406c240 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.945+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec54159a10 con 0x7fec5406c240 2024-04-26T21:47:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.945+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec54159fd0 con 0x7fec5406c240 2024-04-26T21:47:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.946+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec54068fc0 con 0x7fec5406c240 2024-04-26T21:47:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.946+0000 7fec50ff9640 1 -- 172.21.15.40:0/2780115175 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec3c030c10 con 0x7fec5406c240 2024-04-26T21:47:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.947+0000 7fec50ff9640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec3406c8f0 0x7fec3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.947+0000 7fec50ff9640 1 -- 172.21.15.40:0/2780115175 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fec3c0ac750 con 0x7fec5406c240 2024-04-26T21:47:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.947+0000 7fec537fe640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec3406c8f0 0x7fec3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.947+0000 7fec537fe640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec3406c8f0 0x7fec3406edb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fec5415e750 tx=0x7fec44006ba0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:37.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:37.951+0000 7fec50ff9640 1 -- 172.21.15.40:0/2780115175 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec3c045360 con 0x7fec5406c240 2024-04-26T21:47:38.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.173+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec5415e030 con 0x7fec5406c240 2024-04-26T21:47:38.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.174+0000 7fec50ff9640 1 -- 172.21.15.40:0/2780115175 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7fec3c07aa30 con 0x7fec5406c240 2024-04-26T21:47:38.174 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:38.174 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:38.174 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.175+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec3406c8f0 msgr2=0x7fec3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.175+0000 7fec5a3f0640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec3406c8f0 0x7fec3406edb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fec5415e750 tx=0x7fec44006ba0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.175+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5406c240 msgr2=0x7fec5414ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.175+0000 7fec5a3f0640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5406c240 0x7fec5414ffc0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0098c0 tx=0x7fec3c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.176+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 shutdown_connections 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.176+0000 7fec5a3f0640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fec3406c8f0 0x7fec3406edb0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.176+0000 7fec5a3f0640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec5415f1d0 0x7fec5415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.176+0000 7fec5a3f0640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec540afd60 0x7fec54160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.176+0000 7fec5a3f0640 1 --2- 172.21.15.40:0/2780115175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec5406c240 0x7fec5414ffc0 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.176+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 >> 172.21.15.40:0/2780115175 conn(0x7fec540b3180 msgr2=0x7fec540b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.176+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 shutdown_connections 2024-04-26T21:47:38.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.176+0000 7fec5a3f0640 1 -- 172.21.15.40:0/2780115175 wait complete. 2024-04-26T21:47:38.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2964369376 2024-04-26T21:47:38.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.291+0000 7f284b16a640 1 Processor -- start 2024-04-26T21:47:38.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.291+0000 7f284b16a640 1 -- start start 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f284b16a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28440cfde0 0x7f28440d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f284b16a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2844070350 0x7f2844070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f284b16a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2844071090 0x7f284415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f284b16a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2844071500 con 0x7f28440cfde0 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f284b16a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f28440afab0 con 0x7f2844070350 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f284b16a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f284406dd20 con 0x7f2844071090 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f2848f06640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28440cfde0 0x7f28440d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f2848f06640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28440cfde0 0x7f28440d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56506/0 (socket says 172.21.15.40:56506) 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f2848f06640 1 -- 172.21.15.40:0/2914022773 learned_addr learned my addr 172.21.15.40:0/2914022773 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f2849707640 1 --2- 172.21.15.40:0/2914022773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2844071090 0x7f284415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.292+0000 7f2843fff640 1 --2- 172.21.15.40:0/2914022773 >> v2:172.21.15.184:3300/0 conn(0x7f2844070350 0x7f2844070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.293+0000 7f2848f06640 1 -- 172.21.15.40:0/2914022773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2844071090 msgr2=0x7f284415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.293+0000 7f2848f06640 1 --2- 172.21.15.40:0/2914022773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2844071090 0x7f284415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.293+0000 7f2848f06640 1 -- 172.21.15.40:0/2914022773 >> v2:172.21.15.184:3300/0 conn(0x7f2844070350 msgr2=0x7f2844070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.293+0000 7f2848f06640 1 --2- 172.21.15.40:0/2914022773 >> v2:172.21.15.184:3300/0 conn(0x7f2844070350 0x7f2844070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.293+0000 7f2848f06640 1 -- 172.21.15.40:0/2914022773 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28440d2390 con 0x7f28440cfde0 2024-04-26T21:47:38.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.293+0000 7f2848f06640 1 --2- 172.21.15.40:0/2914022773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28440cfde0 0x7f28440d1b60 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f284406cb80 tx=0x7f283c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894a2c3724134544 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f28437fe640 1 -- 172.21.15.40:0/2914022773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c036070 con 0x7f28440cfde0 2024-04-26T21:47:38.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f28437fe640 1 -- 172.21.15.40:0/2914022773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c031600 con 0x7f28440cfde0 2024-04-26T21:47:38.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f28437fe640 1 -- 172.21.15.40:0/2914022773 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0319d0 con 0x7f28440cfde0 2024-04-26T21:47:38.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 -- 172.21.15.40:0/2914022773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28440cfde0 msgr2=0x7f28440d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 --2- 172.21.15.40:0/2914022773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28440cfde0 0x7f28440d1b60 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f284406cb80 tx=0x7f283c02a610 comp rx=0 tx=0).stop 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 -- 172.21.15.40:0/2914022773 shutdown_connections 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 --2- 172.21.15.40:0/2914022773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2844071090 0x7f284415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 --2- 172.21.15.40:0/2914022773 >> v2:172.21.15.184:3300/0 conn(0x7f2844070350 0x7f2844070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 --2- 172.21.15.40:0/2914022773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28440cfde0 0x7f28440d1b60 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 -- 172.21.15.40:0/2914022773 >> 172.21.15.40:0/2914022773 conn(0x7f28440d2d80 msgr2=0x7f28440d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 -- 172.21.15.40:0/2914022773 shutdown_connections 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.294+0000 7f284b16a640 1 -- 172.21.15.40:0/2914022773 wait complete. 2024-04-26T21:47:38.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.295+0000 7f284b16a640 1 Processor -- start 2024-04-26T21:47:38.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f284b16a640 1 -- start start 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f284b16a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2844070350 0x7f28441f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f284b16a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2844071090 0x7f28441f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f284b16a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28440cfde0 0x7f28441e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f284b16a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f284406d9b0 con 0x7f2844070350 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f284b16a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f284406d6d0 con 0x7f2844071090 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f284b16a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28440d20a0 con 0x7f28440cfde0 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f2848f06640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2844070350 0x7f28441f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f2848f06640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2844070350 0x7f28441f8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56512/0 (socket says 172.21.15.40:56512) 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.296+0000 7f2848f06640 1 -- 172.21.15.40:0/20429465 learned_addr learned my addr 172.21.15.40:0/20429465 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f2849707640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28440cfde0 0x7f28441e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f2843fff640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2844071090 0x7f28441f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f2848f06640 1 -- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28440cfde0 msgr2=0x7f28441e1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f2848f06640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28440cfde0 0x7f28441e1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f2848f06640 1 -- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2844071090 msgr2=0x7f28441f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f2848f06640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2844071090 0x7f28441f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f2848f06640 1 -- 172.21.15.40:0/20429465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28441f8ec0 con 0x7f2844070350 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f2848f06640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2844070350 0x7f28441f8870 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f283c034040 tx=0x7f283c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f28417fa640 1 -- 172.21.15.40:0/20429465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c03f020 con 0x7f2844070350 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f28417fa640 1 -- 172.21.15.40:0/20429465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c037040 con 0x7f2844070350 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f284b16a640 1 -- 172.21.15.40:0/20429465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28441f91a0 con 0x7f2844070350 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f28417fa640 1 -- 172.21.15.40:0/20429465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c036070 con 0x7f2844070350 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.297+0000 7f284b16a640 1 -- 172.21.15.40:0/20429465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441f5e80 con 0x7f2844070350 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.299+0000 7f28417fa640 1 -- 172.21.15.40:0/20429465 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f283c042020 con 0x7f2844070350 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.299+0000 7f28417fa640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282406c8f0 0x7f282406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.299+0000 7f28417fa640 1 -- 172.21.15.40:0/20429465 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f283c0abfa0 con 0x7f2844070350 2024-04-26T21:47:38.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.299+0000 7f2843fff640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282406c8f0 0x7f282406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.300+0000 7f2843fff640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282406c8f0 0x7f282406edb0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f28440c7d10 tx=0x7f2834009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.300+0000 7f284de83740 1 Processor -- start 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.300+0000 7f284de83740 1 -- start start 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f284de83740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951c24f0 0x558195174e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f284de83740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x5581951a4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f284de83740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951a4ed0 0x558195179ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f284de83740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5581950dead0 con 0x5581951a4ed0 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f284de83740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558195098a00 con 0x5581951a5610 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f284de83740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5581950daa90 con 0x5581951c24f0 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2849707640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951a4ed0 0x558195179ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2849707640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951a4ed0 0x558195179ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56520/0 (socket says 172.21.15.40:56520) 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2849707640 1 -- 172.21.15.40:0/945356163 learned_addr learned my addr 172.21.15.40:0/945356163 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2848f06640 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951c24f0 0x558195174e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2843fff640 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x5581951a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2849707640 1 -- 172.21.15.40:0/945356163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951c24f0 msgr2=0x558195174e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2849707640 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951c24f0 0x558195174e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2849707640 1 -- 172.21.15.40:0/945356163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 msgr2=0x5581951a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2849707640 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x5581951a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.301+0000 7f2849707640 1 -- 172.21.15.40:0/945356163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558194ebc710 con 0x5581951a4ed0 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f2849707640 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951a4ed0 0x558195179ce0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f282c009bc0 tx=0x7f282c011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ed2955c8b60083 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f283a7fc640 1 -- 172.21.15.40:0/945356163 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c014440 con 0x5581951a4ed0 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f283a7fc640 1 -- 172.21.15.40:0/945356163 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c0145e0 con 0x5581951a4ed0 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f283a7fc640 1 -- 172.21.15.40:0/945356163 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c014990 con 0x5581951a4ed0 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 -- 172.21.15.40:0/945356163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951a4ed0 msgr2=0x558195179ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951a4ed0 0x558195179ce0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f282c009bc0 tx=0x7f282c011690 comp rx=0 tx=0).stop 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 -- 172.21.15.40:0/945356163 shutdown_connections 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951a4ed0 0x558195179ce0 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x5581951a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 --2- 172.21.15.40:0/945356163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951c24f0 0x558195174e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 -- 172.21.15.40:0/945356163 >> 172.21.15.40:0/945356163 conn(0x5581951770f0 msgr2=0x5581951bf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 -- 172.21.15.40:0/945356163 shutdown_connections 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.302+0000 7f284de83740 1 -- 172.21.15.40:0/945356163 wait complete. 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.303+0000 7f284de83740 1 -- start start 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.303+0000 7f284de83740 1 Processor -- start 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f284de83740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x5581951bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f284de83740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f284de83740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195179730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f284de83740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5581951792a0 con 0x5581951957c0 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f284de83740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558195066000 con 0x5581951c24f0 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f284de83740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558195178690 con 0x5581951a5610 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f284de83740 10 client.0 _ll_register_callbacks cb 0x558195081620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195179730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195179730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56526/0 (socket says 172.21.15.40:56526) 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 -- 172.21.15.40:0/704022417 learned_addr learned my addr 172.21.15.40:0/704022417 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2843fff640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2848f06640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x5581951bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 -- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 msgr2=0x5581951bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x5581951bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 -- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 msgr2=0x5581951bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 -- 172.21.15.40:0/704022417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558194f1cea0 con 0x5581951957c0 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2848f06640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x5581951bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2843fff640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f2849707640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195179730 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f282c0116d0 tx=0x7f282c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a161518974889a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.304+0000 7f28237fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f28237fe640 1 -- 172.21.15.40:0/704022417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c02c060 con 0x5581951957c0 2024-04-26T21:47:38.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f28237fe640 1 -- 172.21.15.40:0/704022417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f282c014ba0 con 0x5581951957c0 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 1 -- 172.21.15.40:0/704022417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f282c0145e0 con 0x5581951957c0 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f28237fe640 1 -- 172.21.15.40:0/704022417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f282c032050 con 0x5581951957c0 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 10 client.12093 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 1 -- 172.21.15.40:0/704022417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x558194db0000 con 0x5581951957c0 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f28237fe640 1 -- 172.21.15.40:0/704022417 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c0145e0 con 0x5581951957c0 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f28237fe640 1 -- 172.21.15.40:0/704022417 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f282c003610 con 0x5581951957c0 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 10 client.12093 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 10 client.12093 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 20 client.12093 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5581951db200 0x5581951dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 4 client.12093 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f284de83740 1 -- 172.21.15.40:0/704022417 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581950ec6e0 con 0x5581951db200 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f2848f06640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5581951db200 0x5581951dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f28237fe640 10 client.12093 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.305+0000 7f2848f06640 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5581951db200 0x5581951dd5f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b4d185b97c0deb9 server_cookie=661a2838e6263ace in_seq=0 out_seq=0 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f28237fe640 1 -- 172.21.15.40:0/704022417 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581950ec6e0 con 0x5581951db200 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f28237fe640 10 client.12093 handle_command_reply: tid=0 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 client.12093 shutdown 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 -- 172.21.15.40:0/704022417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 msgr2=0x558195179730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195179730 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f282c0116d0 tx=0x7f282c004730 comp rx=0 tx=0).stop 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 -- 172.21.15.40:0/704022417 shutdown_connections 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5581951db200 0x5581951dd5f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195179730 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 --2- 172.21.15.40:0/704022417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x5581951bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 -- 172.21.15.40:0/704022417 >> 172.21.15.40:0/704022417 conn(0x5581951a4ed0 msgr2=0x5581951a1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f28237fe640 0 client.12093 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 -- 172.21.15.40:0/704022417 shutdown_connections 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.306+0000 7f284de83740 1 -- 172.21.15.40:0/704022417 wait complete. 2024-04-26T21:47:38.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.307+0000 7f284de83740 20 client.12093 trim_cache size 0 max 16384 2024-04-26T21:47:38.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.320+0000 7f284de83740 1 Processor -- start 2024-04-26T21:47:38.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f284de83740 1 -- start start 2024-04-26T21:47:38.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f284de83740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951957c0 0x5581951a4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f284de83740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x55819519fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f284de83740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951c24f0 0x5581951c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f284de83740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2844201150 con 0x5581951c24f0 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f284de83740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28440c71c0 con 0x5581951a5610 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f284de83740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2844200c00 con 0x5581951957c0 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f2849707640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951c24f0 0x5581951c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f2849707640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951c24f0 0x5581951c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56540/0 (socket says 172.21.15.40:56540) 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.321+0000 7f2849707640 1 -- 172.21.15.40:0/3546218195 learned_addr learned my addr 172.21.15.40:0/3546218195 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2843fff640 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x55819519fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2848f06640 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951957c0 0x5581951a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2849707640 1 -- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951957c0 msgr2=0x5581951a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2849707640 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951957c0 0x5581951a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2849707640 1 -- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 msgr2=0x55819519fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2849707640 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x55819519fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2849707640 1 -- 172.21.15.40:0/3546218195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581951a7ba0 con 0x5581951c24f0 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2843fff640 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x55819519fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2848f06640 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951957c0 0x5581951a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2849707640 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951c24f0 0x5581951c1ac0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f282c003850 tx=0x7f282c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19e6b9668dbe063 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2839ffb640 1 -- 172.21.15.40:0/3546218195 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c021070 con 0x5581951c24f0 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2839ffb640 1 -- 172.21.15.40:0/3546218195 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c006a20 con 0x5581951c24f0 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f2839ffb640 1 -- 172.21.15.40:0/3546218195 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c0149a0 con 0x5581951c24f0 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f284de83740 1 -- 172.21.15.40:0/3546218195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951c24f0 msgr2=0x5581951c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f284de83740 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951c24f0 0x5581951c1ac0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f282c003850 tx=0x7f282c024000 comp rx=0 tx=0).stop 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.322+0000 7f284de83740 1 -- 172.21.15.40:0/3546218195 shutdown_connections 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.323+0000 7f284de83740 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951c24f0 0x5581951c1ac0 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.323+0000 7f284de83740 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951a5610 0x55819519fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.323+0000 7f284de83740 1 --2- 172.21.15.40:0/3546218195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951957c0 0x5581951a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.323+0000 7f284de83740 1 -- 172.21.15.40:0/3546218195 >> 172.21.15.40:0/3546218195 conn(0x5581951a4ed0 msgr2=0x5581951d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.323+0000 7f284de83740 1 -- 172.21.15.40:0/3546218195 shutdown_connections 2024-04-26T21:47:38.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.323+0000 7f284de83740 1 -- 172.21.15.40:0/3546218195 wait complete. 2024-04-26T21:47:38.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.323+0000 7f284de83740 1 -- start start 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.323+0000 7f284de83740 1 Processor -- start 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f284de83740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195239060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f284de83740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x558195178fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f284de83740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f284de83740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5581951a09d0 con 0x5581951957c0 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f284de83740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5581951d3dc0 con 0x5581951c24f0 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f284de83740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558195179c70 con 0x5581951a5610 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f284de83740 10 client.0 _ll_register_callbacks cb 0x7f282c009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2848f06640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195239060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2848f06640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195239060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56554/0 (socket says 172.21.15.40:56554) 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2848f06640 1 -- 172.21.15.40:0/604620901 learned_addr learned my addr 172.21.15.40:0/604620901 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2843fff640 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x558195178fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2849707640 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2848f06640 1 -- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 msgr2=0x558195178fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2848f06640 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x558195178fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2848f06640 1 -- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 msgr2=0x5581951794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2848f06640 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951794e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2848f06640 1 -- 172.21.15.40:0/604620901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558195178af0 con 0x5581951957c0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.324+0000 7f2849707640 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f2848f06640 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195239060 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f283c079b30 tx=0x7f283c0aa1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2fe8c1bff7ad57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f28237fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f28237fe640 1 -- 172.21.15.40:0/604620901 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c007c70 con 0x5581951957c0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f28237fe640 1 -- 172.21.15.40:0/604620901 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f283c076210 con 0x5581951957c0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 1 -- 172.21.15.40:0/604620901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f282c006a20 con 0x5581951957c0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f28237fe640 1 -- 172.21.15.40:0/604620901 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f283c09e050 con 0x5581951957c0 2024-04-26T21:47:38.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 10 client.12099 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 1 -- 172.21.15.40:0/604620901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x5581951c3b80 con 0x5581951957c0 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f28237fe640 1 -- 172.21.15.40:0/604620901 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c006a20 con 0x5581951957c0 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f28237fe640 1 -- 172.21.15.40:0/604620901 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f283c0a1020 con 0x5581951957c0 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 10 client.12099 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 10 client.12099 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 20 client.12099 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5581951c0fc0 0x55819523cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 4 client.12099 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2964369376}] 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f284de83740 1 -- 172.21.15.40:0/604620901 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2964369376}) -- 0x5581950e6550 con 0x5581951c0fc0 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.325+0000 7f2843fff640 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5581951c0fc0 0x55819523cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f2843fff640 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5581951c0fc0 0x55819523cb40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96b0782bdcaae31 server_cookie=ce86df6f3755c76b in_seq=0 out_seq=0 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f28237fe640 10 client.12099 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f28237fe640 1 -- 172.21.15.40:0/604620901 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x5581950e6550 con 0x5581951c0fc0 2024-04-26T21:47:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f28237fe640 10 client.12099 handle_command_reply: tid=0 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 client.12099 shutdown 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 -- 172.21.15.40:0/604620901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 msgr2=0x558195239060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195239060 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f283c079b30 tx=0x7f283c0aa1c0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 -- 172.21.15.40:0/604620901 shutdown_connections 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5581951c0fc0 0x55819523cb40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581951c24f0 0x5581951794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5581951a5610 0x558195178fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f28237fe640 0 client.12099 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 --2- 172.21.15.40:0/604620901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5581951957c0 0x558195239060 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 -- 172.21.15.40:0/604620901 >> 172.21.15.40:0/604620901 conn(0x5581951a4ed0 msgr2=0x5581951bf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 -- 172.21.15.40:0/604620901 shutdown_connections 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 1 -- 172.21.15.40:0/604620901 wait complete. 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.326+0000 7f284de83740 20 client.12099 trim_cache size 0 max 16384 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 -- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282406c8f0 msgr2=0x7f282406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282406c8f0 0x7f282406edb0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f28440c7d10 tx=0x7f2834009190 comp rx=0 tx=0).stop 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 -- 172.21.15.40:0/20429465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2844070350 msgr2=0x7f28441f8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2844070350 0x7f28441f8870 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f283c034040 tx=0x7f283c004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 -- 172.21.15.40:0/20429465 shutdown_connections 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f282406c8f0 0x7f282406edb0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28440cfde0 0x7f28441e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2844071090 0x7f28441f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 --2- 172.21.15.40:0/20429465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2844070350 0x7f28441f8870 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 -- 172.21.15.40:0/20429465 >> 172.21.15.40:0/20429465 conn(0x7f28440d2d80 msgr2=0x7f28440afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 -- 172.21.15.40:0/20429465 shutdown_connections 2024-04-26T21:47:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.327+0000 7f284b16a640 1 -- 172.21.15.40:0/20429465 wait complete. 2024-04-26T21:47:38.338 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100000001f6.00000000 2024-04-26T21:47:38.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.379+0000 7f7fecb1ef40 1 Processor -- start 2024-04-26T21:47:38.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fecb1ef40 1 -- start start 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fecb1ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d445941c0 0x555d4466bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fecb1ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 0x555d44597830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fecb1ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d44666f70 0x555d4466ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fecb1ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555d4453ce40 con 0x555d445941c0 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fecb1ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555d445de660 con 0x555d445df420 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fecb1ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555d445c3910 con 0x555d44666f70 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fe9cff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d445941c0 0x555d4466bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fe9cff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d445941c0 0x555d4466bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56574/0 (socket says 172.21.15.40:56574) 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fe9cff640 1 -- 172.21.15.40:0/17527906 learned_addr learned my addr 172.21.15.40:0/17527906 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fea500640 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d44666f70 0x555d4466ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.380+0000 7f7fe94fe640 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 0x555d44597830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fe9cff640 1 -- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d44666f70 msgr2=0x555d4466ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fe9cff640 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d44666f70 0x555d4466ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fe9cff640 1 -- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 msgr2=0x555d44597830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fe9cff640 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 0x555d44597830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fe9cff640 1 -- 172.21.15.40:0/17527906 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d445c3670 con 0x555d445941c0 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fe94fe640 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 0x555d44597830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fe9cff640 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d445941c0 0x555d4466bff0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00a770 tx=0x7f7fdc00a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11e4d5214560229 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fdbfff640 1 -- 172.21.15.40:0/17527906 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc018070 con 0x555d445941c0 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fdbfff640 1 -- 172.21.15.40:0/17527906 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc007600 con 0x555d445941c0 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fdbfff640 1 -- 172.21.15.40:0/17527906 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0079b0 con 0x555d445941c0 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/17527906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d445941c0 msgr2=0x555d4466bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fecb1ef40 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d445941c0 0x555d4466bff0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00a770 tx=0x7f7fdc00a860 comp rx=0 tx=0).stop 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/17527906 shutdown_connections 2024-04-26T21:47:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fecb1ef40 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d44666f70 0x555d4466ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fecb1ef40 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 0x555d44597830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fecb1ef40 1 --2- 172.21.15.40:0/17527906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d445941c0 0x555d4466bff0 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/17527906 >> 172.21.15.40:0/17527906 conn(0x555d445bf8a0 msgr2=0x555d445c0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.381+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/17527906 shutdown_connections 2024-04-26T21:47:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.382+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/17527906 wait complete. 2024-04-26T21:47:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.382+0000 7f7fecb1ef40 1 Processor -- start 2024-04-26T21:47:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.382+0000 7f7fecb1ef40 1 -- start start 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fecb1ef40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d445941c0 0x555d44597dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fecb1ef40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 0x555d44669f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fecb1ef40 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d44666f70 0x555d44707200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fecb1ef40 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555d44666340 con 0x555d44666f70 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fecb1ef40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555d44661ab0 con 0x555d445df420 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fecb1ef40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555d445c0e30 con 0x555d445941c0 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d44666f70 0x555d44707200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d44666f70 0x555d44707200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56588/0 (socket says 172.21.15.40:56588) 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 -- 172.21.15.40:0/3230848432 learned_addr learned my addr 172.21.15.40:0/3230848432 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fe94fe640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 0x555d44669f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fe9cff640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d445941c0 0x555d44597dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 -- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d445941c0 msgr2=0x555d44597dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555d445941c0 0x555d44597dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 -- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 msgr2=0x555d44669f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d445df420 0x555d44669f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 -- 172.21.15.40:0/3230848432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d44668e80 con 0x555d44666f70 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.383+0000 7f7fea500640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d44666f70 0x555d44707200 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4045be0 tx=0x7f7fe4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.384+0000 7f7fdaffd640 1 -- 172.21.15.40:0/3230848432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4054040 con 0x555d44666f70 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.384+0000 7f7fdaffd640 1 -- 172.21.15.40:0/3230848432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe4068070 con 0x555d44666f70 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.384+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555d44669160 con 0x555d44666f70 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.384+0000 7f7fdaffd640 1 -- 172.21.15.40:0/3230848432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4005b40 con 0x555d44666f70 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.384+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d44669720 con 0x555d44666f70 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.385+0000 7f7fdaffd640 1 -- 172.21.15.40:0/3230848432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fe4057070 con 0x555d44666f70 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.386+0000 7f7fdaffd640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7fcc06b2d0 0x7f7fcc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.386+0000 7f7fdaffd640 1 -- 172.21.15.40:0/3230848432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f7fe40ceb20 con 0x555d44666f70 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.386+0000 7f7fe9cff640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7fcc06b2d0 0x7f7fcc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.386+0000 7f7fecb1ef40 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x555d445d8fb0 0x555d445d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.386+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- osd_op(unknown.0.0:1 15.a 15:57fe7535:::100000001f6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e146) -- 0x555d44707c50 con 0x555d445d8fb0 2024-04-26T21:47:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.386+0000 7f7fe94fe640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x555d445d8fb0 0x555d445d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.386+0000 7f7fe9cff640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7fcc06b2d0 0x7f7fcc06d790 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00a770 tx=0x7f7fdc00c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.386+0000 7f7fe94fe640 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x555d445d8fb0 0x555d445d93c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.387+0000 7f7fe94fe640 1 -- 172.21.15.40:0/3230848432 <== osd.2 v2:172.21.15.40:6811/3280366756 1 ==== osd_op_reply(1 100000001f6.00000000 [delete] v146'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7fe000f770 con 0x555d445d8fb0 2024-04-26T21:47:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x555d445d8fb0 msgr2=0x555d445d93c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x555d445d8fb0 0x555d445d93c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7fcc06b2d0 msgr2=0x7f7fcc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7fcc06b2d0 0x7f7fcc06d790 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00a770 tx=0x7f7fdc00c000 comp rx=0 tx=0).stop 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d44666f70 msgr2=0x555d44707200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 --2- 172.21.15.40:0/3230848432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555d44666f70 0x555d44707200 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4045be0 tx=0x7f7fe4003330 comp rx=0 tx=0).stop 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fea500640 1 -- 172.21.15.40:0/3230848432 reap_dead start 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 shutdown_connections 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 >> 172.21.15.40:0/3230848432 conn(0x555d445bf8a0 msgr2=0x555d446612c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 shutdown_connections 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.388+0000 7f7fecb1ef40 1 -- 172.21.15.40:0/3230848432 wait complete. 2024-04-26T21:47:38.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:38.391 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink2) 2024-04-26T21:47:38.446 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:47:38.446+0000 7f9287375640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-04-26T21:47:38.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:47:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:stat: cannot statx 'testdir/hardlink2': Input/output error 2024-04-26T21:47:38.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:38.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.589+0000 7f141056a640 1 Processor -- start 2024-04-26T21:47:38.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.589+0000 7f141056a640 1 -- start start 2024-04-26T21:47:38.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f141056a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080d1380 0x7f14080d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f141056a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1408070ae0 0x7f1408070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f141056a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14080d1ce0 con 0x7f140815d1e0 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f141056a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f14080cede0 con 0x7f1408070ae0 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f141056a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1408070650 con 0x7f14080d1380 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140e306640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f140815d1e0 0x7f14080d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53118/0 (socket says 172.21.15.40:53118) 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140e306640 1 -- 172.21.15.40:0/1842813984 learned_addr learned my addr 172.21.15.40:0/1842813984 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140db05640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080d1380 0x7f14080d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263106556 0 0) 0x7f14080d1ce0 con 0x7f140815d1e0 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13f4003520 con 0x7f140815d1e0 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140eb07640 1 --2- 172.21.15.40:0/1842813984 >> v2:172.21.15.184:3300/0 conn(0x7f1408070ae0 0x7f1408070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2170131844 0 0) 0x7f13f4003520 con 0x7f140815d1e0 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080d1380 msgr2=0x7f14080d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080d1380 0x7f14080d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 >> v2:172.21.15.184:3300/0 conn(0x7f1408070ae0 msgr2=0x7f1408070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 --2- 172.21.15.40:0/1842813984 >> v2:172.21.15.184:3300/0 conn(0x7f1408070ae0 0x7f1408070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14080ce840 con 0x7f140815d1e0 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1404001670 con 0x7f140815d1e0 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.590+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 >> v1:172.21.15.40:6789/0 conn(0x7f140815d1e0 legacy=0x7f14080d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140d304640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080d1380 0x7f13f40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140d304640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13f4003f30 0x7f13f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140d304640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13f4004d40 0x7f13f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14080cede0 con 0x7f13f4003f30 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1408070650 con 0x7f13f4004d40 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13f400b9c0 con 0x7f14080d1380 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140db05640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13f4003f30 0x7f13f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140e306640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080d1380 0x7f13f40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140eb07640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13f4004d40 0x7f13f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140db05640 1 -- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080d1380 msgr2=0x7f13f40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140db05640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080d1380 0x7f13f40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140db05640 1 -- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13f4004d40 msgr2=0x7f13f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140db05640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13f4004d40 0x7f13f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.591+0000 7f140db05640 1 -- 172.21.15.40:0/1842813984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f4003520 con 0x7f13f4003f30 2024-04-26T21:47:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f140db05640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13f4003f30 0x7f13f4004800 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009ac0 tx=0x7f13f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af0ca58e1275c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8036070 con 0x7f13f4003f30 2024-04-26T21:47:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f8031400 con 0x7f13f4003f30 2024-04-26T21:47:38.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f140d304640 1 -- 172.21.15.40:0/1842813984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8009690 con 0x7f13f4003f30 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f141056a640 1 -- 172.21.15.40:0/1842813984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13f4003f30 msgr2=0x7f13f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f141056a640 1 --2- 172.21.15.40:0/1842813984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13f4003f30 0x7f13f4004800 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009ac0 tx=0x7f13f4005830 comp rx=0 tx=0).stop 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f140eb07640 1 -- 172.21.15.40:0/1842813984 reap_dead start 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f141056a640 1 -- 172.21.15.40:0/1842813984 shutdown_connections 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.592+0000 7f141056a640 1 -- 172.21.15.40:0/1842813984 >> 172.21.15.40:0/1842813984 conn(0x7f14080d2da0 msgr2=0x7f14080d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.593+0000 7f141056a640 1 -- 172.21.15.40:0/1842813984 shutdown_connections 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.593+0000 7f141056a640 1 -- 172.21.15.40:0/1842813984 wait complete. 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.594+0000 7f141056a640 1 Processor -- start 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.594+0000 7f141056a640 1 -- start start 2024-04-26T21:47:38.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.594+0000 7f141056a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13f4003f30 0x7f14080c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.594+0000 7f141056a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1408156740 0x7f14080c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.594+0000 7f141056a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14080c7270 0x7f140815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.594+0000 7f141056a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14080d10f0 con 0x7f14080c7270 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f141056a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14080702e0 con 0x7f1408156740 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f141056a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14080ce4c0 con 0x7f13f4003f30 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14080c7270 0x7f140815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14080c7270 0x7f140815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56600/0 (socket says 172.21.15.40:56600) 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 -- 172.21.15.40:0/3992208278 learned_addr learned my addr 172.21.15.40:0/3992208278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140e306640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13f4003f30 0x7f14080c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140db05640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1408156740 0x7f14080c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 -- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13f4003f30 msgr2=0x7f14080c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13f4003f30 0x7f14080c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 -- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1408156740 msgr2=0x7f14080c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1408156740 0x7f14080c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 -- 172.21.15.40:0/3992208278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14081578e0 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f140eb07640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14080c7270 0x7f140815b2b0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f1408073950 tx=0x7f140000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f13feffd640 1 -- 172.21.15.40:0/3992208278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140000fd10 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f13feffd640 1 -- 172.21.15.40:0/3992208278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1400015430 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f13feffd640 1 -- 172.21.15.40:0/3992208278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400015820 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1408157320 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.595+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14080c7c10 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.596+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14080c7e10 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.597+0000 7f13feffd640 1 -- 172.21.15.40:0/3992208278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14000159c0 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.597+0000 7f13feffd640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ec06c920 0x7f13ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.597+0000 7f13feffd640 1 -- 172.21.15.40:0/3992208278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f140008ca40 con 0x7f14080c7270 2024-04-26T21:47:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.597+0000 7f140e306640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ec06c920 0x7f13ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.598+0000 7f140e306640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ec06c920 0x7f13ec06ede0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1404001630 tx=0x7f14040078f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.602+0000 7f13feffd640 1 -- 172.21.15.40:0/3992208278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14080c7e10 con 0x7f14080c7270 2024-04-26T21:47:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.824+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14080694f0 con 0x7f14080c7270 2024-04-26T21:47:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.825+0000 7f13feffd640 1 -- 172.21.15.40:0/3992208278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7f1400019030 con 0x7f14080c7270 2024-04-26T21:47:38.825 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:38.825 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ec06c920 msgr2=0x7f13ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ec06c920 0x7f13ec06ede0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1404001630 tx=0x7f14040078f0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14080c7270 msgr2=0x7f140815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14080c7270 0x7f140815b2b0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f1408073950 tx=0x7f140000f1b0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 shutdown_connections 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f13ec06c920 0x7f13ec06ede0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14080c7270 0x7f140815b2b0 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1408156740 0x7f14080c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 --2- 172.21.15.40:0/3992208278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13f4003f30 0x7f14080c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 >> 172.21.15.40:0/3992208278 conn(0x7f14080d2da0 msgr2=0x7f14080b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 shutdown_connections 2024-04-26T21:47:38.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.827+0000 7f141056a640 1 -- 172.21.15.40:0/3992208278 wait complete. 2024-04-26T21:47:38.838 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-26T21:47:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.940+0000 7fc68ee83640 1 Processor -- start 2024-04-26T21:47:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.940+0000 7fc68ee83640 1 -- start start 2024-04-26T21:47:38.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.941+0000 7fc68ee83640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6880d0d00 0x7fc6880d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.941+0000 7fc68ee83640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6880d0710 con 0x7fc688160720 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.941+0000 7fc68ee83640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6880d0430 con 0x7fc6880cf180 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.941+0000 7fc68ee83640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc6880d01e0 con 0x7fc6880d0d00 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.941+0000 7fc67ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6880d0d00 0x7fc6880d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.941+0000 7fc68cc1f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc688160720 0x7fc6880ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42252/0 (socket says 172.21.15.40:42252) 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.941+0000 7fc68cc1f640 1 -- 172.21.15.40:0/772083087 learned_addr learned my addr 172.21.15.40:0/772083087 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.942+0000 7fc67ffff640 1 -- 172.21.15.40:0/772083087 >> v1:172.21.15.184:6789/0 conn(0x7fc6880cf180 legacy=0x7fc6880b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.942+0000 7fc67ffff640 1 -- 172.21.15.40:0/772083087 >> v1:172.21.15.40:6789/0 conn(0x7fc688160720 legacy=0x7fc6880ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.942+0000 7fc67f7fe640 1 -- 172.21.15.40:0/772083087 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330408245 0 0) 0x7fc6880d0710 con 0x7fc688160720 2024-04-26T21:47:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.942+0000 7fc67ffff640 1 -- 172.21.15.40:0/772083087 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6880cdb40 con 0x7fc6880d0d00 2024-04-26T21:47:38.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.942+0000 7fc67ffff640 1 --2- 172.21.15.40:0/772083087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880d11e0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097a0 tx=0x7fc67802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd8fb5e5f5d77d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc67f7fe640 1 -- 172.21.15.40:0/772083087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678037070 con 0x7fc6880d0d00 2024-04-26T21:47:38.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc67f7fe640 1 -- 172.21.15.40:0/772083087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc678032440 con 0x7fc6880d0d00 2024-04-26T21:47:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc67f7fe640 1 -- 172.21.15.40:0/772083087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678032810 con 0x7fc6880d0d00 2024-04-26T21:47:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc68ee83640 1 -- 172.21.15.40:0/772083087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 msgr2=0x7fc6880d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc68ee83640 1 --2- 172.21.15.40:0/772083087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880d11e0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097a0 tx=0x7fc67802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc68ee83640 1 -- 172.21.15.40:0/772083087 shutdown_connections 2024-04-26T21:47:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc68ee83640 1 --2- 172.21.15.40:0/772083087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880d11e0 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc68ee83640 1 -- 172.21.15.40:0/772083087 >> 172.21.15.40:0/772083087 conn(0x7fc6880d2e00 msgr2=0x7fc6880d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc68ee83640 1 -- 172.21.15.40:0/772083087 shutdown_connections 2024-04-26T21:47:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.943+0000 7fc68ee83640 1 -- 172.21.15.40:0/772083087 wait complete. 2024-04-26T21:47:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.944+0000 7fc68ee83640 1 Processor -- start 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.944+0000 7fc68ee83640 1 -- start start 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc68ee83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6880cf180 0x7fc688159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc68ee83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc68ee83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc688160720 0x7fc6880b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc68ee83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc6880ced30 con 0x7fc6880d0d00 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc68ee83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6880ce100 con 0x7fc6880cf180 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc68ee83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc688160b00 con 0x7fc688160720 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc67ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc67ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42464/0 (socket says 172.21.15.40:42464) 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc67ffff640 1 -- 172.21.15.40:0/387274928 learned_addr learned my addr 172.21.15.40:0/387274928 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc68d420640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc688160720 0x7fc6880b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.945+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6880cf180 0x7fc688159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67ffff640 1 -- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc688160720 msgr2=0x7fc6880b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67ffff640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc688160720 0x7fc6880b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67ffff640 1 -- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6880cf180 msgr2=0x7fc688159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67ffff640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6880cf180 0x7fc688159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67ffff640 1 -- 172.21.15.40:0/387274928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6881f66d0 con 0x7fc6880d0d00 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc68d420640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc688160720 0x7fc6880b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6880cf180 0x7fc688159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67ffff640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880b7980 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7fc678035040 tx=0x7fc6780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67d7fa640 1 -- 172.21.15.40:0/387274928 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678037040 con 0x7fc6880d0d00 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67d7fa640 1 -- 172.21.15.40:0/387274928 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc678004250 con 0x7fc6880d0d00 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc68ee83640 1 -- 172.21.15.40:0/387274928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6881f69b0 con 0x7fc6880d0d00 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc67d7fa640 1 -- 172.21.15.40:0/387274928 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678032e20 con 0x7fc6880d0d00 2024-04-26T21:47:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.946+0000 7fc68ee83640 1 -- 172.21.15.40:0/387274928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6881f6f70 con 0x7fc6880d0d00 2024-04-26T21:47:38.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.947+0000 7fc67d7fa640 1 -- 172.21.15.40:0/387274928 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc678030770 con 0x7fc6880d0d00 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.948+0000 7fc67d7fa640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc67406c8f0 0x7fc67406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.948+0000 7fc67d7fa640 1 -- 172.21.15.40:0/387274928 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fc6780ad0f0 con 0x7fc6880d0d00 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.948+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc67406c8f0 0x7fc67406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.949+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc67406c8f0 0x7fc67406edb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc684000f60 tx=0x7fc684009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.949+0000 7fc691ba3740 1 Processor -- start 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.949+0000 7fc691ba3740 1 -- start start 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.949+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b6629bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662ba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b662bae40 0x564b66272d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc691ba3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564b661d8b80 con 0x564b66275e90 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc691ba3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b661744d0 con 0x564b6629bae0 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc691ba3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564b661d4b40 con 0x564b662bae40 2024-04-26T21:47:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662ba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42476/0 (socket says 172.21.15.40:42476) 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 -- 172.21.15.40:0/1614403968 learned_addr learned my addr 172.21.15.40:0/1614403968 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b6629bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc68d420640 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b662bae40 0x564b66272d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 -- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b662bae40 msgr2=0x564b66272d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b662bae40 0x564b66272d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 -- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 msgr2=0x564b6629bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b6629bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 -- 172.21.15.40:0/1614403968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b661a1770 con 0x564b66275e90 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc68d420640 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b662bae40 0x564b66272d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc67ffff640 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662ba900 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7fc678083370 tx=0x7fc678032580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8568ede6c2140c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc66e7fc640 1 -- 172.21.15.40:0/1614403968 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67807f070 con 0x564b66275e90 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc66e7fc640 1 -- 172.21.15.40:0/1614403968 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6780838a0 con 0x564b66275e90 2024-04-26T21:47:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.950+0000 7fc66e7fc640 1 -- 172.21.15.40:0/1614403968 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678077330 con 0x564b66275e90 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 -- 172.21.15.40:0/1614403968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 msgr2=0x564b662ba900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662ba900 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7fc678083370 tx=0x7fc678032580 comp rx=0 tx=0).stop 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 -- 172.21.15.40:0/1614403968 shutdown_connections 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b662bae40 0x564b66272d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662ba900 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 --2- 172.21.15.40:0/1614403968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b6629bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 -- 172.21.15.40:0/1614403968 >> 172.21.15.40:0/1614403968 conn(0x564b66208e90 msgr2=0x564b662b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 -- 172.21.15.40:0/1614403968 shutdown_connections 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 -- 172.21.15.40:0/1614403968 wait complete. 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.951+0000 7fc691ba3740 1 -- start start 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc691ba3740 1 Processor -- start 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b66286b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b66273400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc691ba3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564b661d8b80 con 0x564b66275e90 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc691ba3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b66171e70 con 0x564b6629bae0 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc691ba3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564b662702f0 con 0x564b6629c470 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc691ba3740 10 client.0 _ll_register_callbacks cb 0x564b66182ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc68cc1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc68cc1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662d1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42482/0 (socket says 172.21.15.40:42482) 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.952+0000 7fc68cc1f640 1 -- 172.21.15.40:0/224074579 learned_addr learned my addr 172.21.15.40:0/224074579 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc68d420640 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b66273400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc67ffff640 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b66286b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc68cc1f640 1 -- 172.21.15.40:0/224074579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 msgr2=0x564b66273400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b66273400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc68cc1f640 1 -- 172.21.15.40:0/224074579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 msgr2=0x564b66286b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b66286b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc68cc1f640 1 -- 172.21.15.40:0/224074579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b65fb67c0 con 0x564b66275e90 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662d1ab0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fc684005b50 tx=0x7fc684001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f204205214352b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc6677fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc6677fe640 1 -- 172.21.15.40:0/224074579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684026060 con 0x564b66275e90 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc6677fe640 1 -- 172.21.15.40:0/224074579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fc684021c80 con 0x564b66275e90 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc691ba3740 1 -- 172.21.15.40:0/224074579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6780838a0 con 0x564b66275e90 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc6677fe640 1 -- 172.21.15.40:0/224074579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7fc684012620 con 0x564b66275e90 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc691ba3740 10 client.12129 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc691ba3740 1 -- 172.21.15.40:0/224074579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x564b66016f50 con 0x564b66275e90 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc6677fe640 1 -- 172.21.15.40:0/224074579 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6780838a0 con 0x564b66275e90 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.953+0000 7fc6677fe640 1 -- 172.21.15.40:0/224074579 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7fc6840127c0 con 0x564b66275e90 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc691ba3740 10 client.12129 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc691ba3740 10 client.12129 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc691ba3740 20 client.12129 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc691ba3740 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564b66295c20 0x564b66296000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc691ba3740 4 client.12129 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:47:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc691ba3740 1 -- 172.21.15.40:0/224074579 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b6618c240 con 0x564b66295c20 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc67ffff640 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564b66295c20 0x564b66296000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc6677fe640 10 client.12129 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.954+0000 7fc67ffff640 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564b66295c20 0x564b66296000 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d75c802316526083 server_cookie=866edc9e96f336c1 in_seq=0 out_seq=0 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc6677fe640 1 -- 172.21.15.40:0/224074579 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b6618c240 con 0x564b66295c20 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc6677fe640 10 client.12129 handle_command_reply: tid=0 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 client.12129 shutdown 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 -- 172.21.15.40:0/224074579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 msgr2=0x564b662d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662d1ab0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fc684005b50 tx=0x7fc684001080 comp rx=0 tx=0).stop 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 -- 172.21.15.40:0/224074579 shutdown_connections 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564b66295c20 0x564b66296000 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b66273400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b66286b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc6677fe640 0 client.12129 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 --2- 172.21.15.40:0/224074579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b662d1ab0 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 -- 172.21.15.40:0/224074579 >> 172.21.15.40:0/224074579 conn(0x564b66208e90 msgr2=0x564b662bcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 -- 172.21.15.40:0/224074579 shutdown_connections 2024-04-26T21:47:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 1 -- 172.21.15.40:0/224074579 wait complete. 2024-04-26T21:47:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.955+0000 7fc691ba3740 20 client.12129 trim_cache size 0 max 16384 2024-04-26T21:47:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.968+0000 7fc691ba3740 1 Processor -- start 2024-04-26T21:47:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.968+0000 7fc691ba3740 1 -- start start 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b66275e90 0x564b6626e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662e24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b6629c470 0x564b662e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc691ba3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc68806e7d0 con 0x564b6629c470 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc691ba3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6880cbd70 con 0x564b6629bae0 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc691ba3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc6880b1c40 con 0x564b66275e90 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68d420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b6629c470 0x564b662e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68d420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b6629c470 0x564b662e2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42488/0 (socket says 172.21.15.40:42488) 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68d420640 1 -- 172.21.15.40:0/1514848075 learned_addr learned my addr 172.21.15.40:0/1514848075 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b66275e90 0x564b6626e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc67ffff640 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662e24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68d420640 1 -- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b66275e90 msgr2=0x564b6626e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68d420640 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b66275e90 0x564b6626e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68d420640 1 -- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 msgr2=0x564b662e24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68d420640 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662e24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc68d420640 1 -- 172.21.15.40:0/1514848075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b662874a0 con 0x564b6629c470 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.969+0000 7fc67ffff640 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662e24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc68d420640 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b6629c470 0x564b662e2a20 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7fc6800114a0 tx=0x7fc680011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd8bf2d1e7cf4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc66dffb640 1 -- 172.21.15.40:0/1514848075 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680003b70 con 0x564b6629c470 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc66dffb640 1 -- 172.21.15.40:0/1514848075 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc680003d10 con 0x564b6629c470 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc66dffb640 1 -- 172.21.15.40:0/1514848075 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68000f980 con 0x564b6629c470 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 -- 172.21.15.40:0/1514848075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b6629c470 msgr2=0x564b662e2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b6629c470 0x564b662e2a20 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7fc6800114a0 tx=0x7fc680011960 comp rx=0 tx=0).stop 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 -- 172.21.15.40:0/1514848075 shutdown_connections 2024-04-26T21:47:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b6629c470 0x564b662e2a20 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662e24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 --2- 172.21.15.40:0/1514848075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b66275e90 0x564b6626e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 -- 172.21.15.40:0/1514848075 >> 172.21.15.40:0/1514848075 conn(0x564b66208e90 msgr2=0x564b662d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 -- 172.21.15.40:0/1514848075 shutdown_connections 2024-04-26T21:47:38.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.970+0000 7fc691ba3740 1 -- 172.21.15.40:0/1514848075 wait complete. 2024-04-26T21:47:38.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.971+0000 7fc691ba3740 1 -- start start 2024-04-26T21:47:38.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.971+0000 7fc691ba3740 1 Processor -- start 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b66336bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc691ba3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b662e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc691ba3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564b662815a0 con 0x564b66275e90 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc691ba3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b6629d410 con 0x564b6629bae0 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc691ba3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x564b662807c0 con 0x564b6629c470 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc691ba3740 10 client.0 _ll_register_callbacks cb 0x7fc6840057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b66336bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b66336bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42494/0 (socket says 172.21.15.40:42494) 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 -- 172.21.15.40:0/592040240 learned_addr learned my addr 172.21.15.40:0/592040240 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc67ffff640 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68d420640 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b662e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 -- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 msgr2=0x564b662e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b662e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 -- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 msgr2=0x564b662d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 -- 172.21.15.40:0/592040240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b65eaa0b0 con 0x564b66275e90 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68d420640 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b662e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc68cc1f640 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b66336bb0 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7fc6840129f0 tx=0x7fc68401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3aeaed23434811 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.972+0000 7fc6677fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc6677fe640 1 -- 172.21.15.40:0/592040240 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68400dc60 con 0x564b66275e90 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc6677fe640 1 -- 172.21.15.40:0/592040240 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fc68402c640 con 0x564b66275e90 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 1 -- 172.21.15.40:0/592040240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc680003d10 con 0x564b66275e90 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc6677fe640 1 -- 172.21.15.40:0/592040240 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7fc68400d5a0 con 0x564b66275e90 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 10 client.12135 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:38.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 1 -- 172.21.15.40:0/592040240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x564b662e2d90 con 0x564b66275e90 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc6677fe640 1 -- 172.21.15.40:0/592040240 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc680003d10 con 0x564b66275e90 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc6677fe640 1 -- 172.21.15.40:0/592040240 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7fc68402c840 con 0x564b66275e90 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 10 client.12135 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 10 client.12135 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 20 client.12135 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564b6629a470 0x564b662937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 4 client.12135 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc691ba3740 1 -- 172.21.15.40:0/592040240 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x564b661e6790 con 0x564b6629a470 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.973+0000 7fc67ffff640 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564b6629a470 0x564b662937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc6677fe640 10 client.12135 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc67ffff640 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564b6629a470 0x564b662937d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557df3b62a20348c server_cookie=4d97ead22b14ace4 in_seq=0 out_seq=0 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc6677fe640 1 -- 172.21.15.40:0/592040240 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+293 (crc 0 0 0) 0x564b661e6790 con 0x564b6629a470 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc6677fe640 10 client.12135 handle_command_reply: tid=0 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 client.12135 shutdown 2024-04-26T21:47:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 -- 172.21.15.40:0/592040240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 msgr2=0x564b66336bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b66336bb0 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7fc6840129f0 tx=0x7fc68401f000 comp rx=0 tx=0).stop 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 -- 172.21.15.40:0/592040240 shutdown_connections 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x564b6629a470 0x564b662937d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564b6629c470 0x564b662e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b6629bae0 0x564b662d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc6677fe640 0 client.12135 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 --2- 172.21.15.40:0/592040240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b66275e90 0x564b66336bb0 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 -- 172.21.15.40:0/592040240 >> 172.21.15.40:0/592040240 conn(0x564b66208e90 msgr2=0x564b6626e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 -- 172.21.15.40:0/592040240 shutdown_connections 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 1 -- 172.21.15.40:0/592040240 wait complete. 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc691ba3740 20 client.12135 trim_cache size 0 max 16384 2024-04-26T21:47:38.982 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "dir_frag", 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 475567744, 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628278, 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "frag": "*", 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/dir2" 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2110519633, 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628279, 2024-04-26T21:47:38.983 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/testdir/hardlink2" 2024-04-26T21:47:38.984 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:47:38.984 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.974+0000 7fc68ee83640 1 -- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc67406c8f0 msgr2=0x7fc67406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc67406c8f0 0x7fc67406edb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc684000f60 tx=0x7fc684009040 comp rx=0 tx=0).stop 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 -- 172.21.15.40:0/387274928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 msgr2=0x7fc6880b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880b7980 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7fc678035040 tx=0x7fc6780046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 -- 172.21.15.40:0/387274928 shutdown_connections 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc67406c8f0 0x7fc67406edb0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc688160720 0x7fc6880b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6880d0d00 0x7fc6880b7980 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 --2- 172.21.15.40:0/387274928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6880cf180 0x7fc688159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 -- 172.21.15.40:0/387274928 >> 172.21.15.40:0/387274928 conn(0x7fc6880d2e00 msgr2=0x7fc688071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 -- 172.21.15.40:0/387274928 shutdown_connections 2024-04-26T21:47:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:38.975+0000 7fc68ee83640 1 -- 172.21.15.40:0/387274928 wait complete. 2024-04-26T21:47:38.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:39.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.088+0000 7f6672025640 1 Processor -- start 2024-04-26T21:47:39.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.088+0000 7f6672025640 1 -- start start 2024-04-26T21:47:39.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.088+0000 7f6672025640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f666c0d06f0 0x7f666c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.088+0000 7f6672025640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666c0d0ba0 con 0x7f666c0b2850 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.088+0000 7f6672025640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666c0d0e20 con 0x7f666c15f3a0 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.088+0000 7f6672025640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f666c0b1d70 con 0x7f666c0d06f0 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f666c0b2850 0x7f666c0b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42254/0 (socket says 172.21.15.40:42254) 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666bfff640 1 -- 172.21.15.40:0/3933370041 learned_addr learned my addr 172.21.15.40:0/3933370041 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f6662ffd640 1 --2- 172.21.15.40:0/3933370041 >> v2:172.21.15.184:3300/0 conn(0x7f666c0d06f0 0x7f666c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2818537238 0 0) 0x7f6664001490 con 0x7f666c0b2850 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6658003520 con 0x7f666c0b2850 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 596976485 0 0) 0x7f6658003520 con 0x7f666c0b2850 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 >> v2:172.21.15.184:3300/0 conn(0x7f666c0d06f0 msgr2=0x7f666c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 --2- 172.21.15.40:0/3933370041 >> v2:172.21.15.184:3300/0 conn(0x7f666c0d06f0 0x7f666c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 >> v1:172.21.15.184:6789/0 conn(0x7f666c15f3a0 legacy=0x7f666c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c0b2050 con 0x7f666c0b2850 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f6664001bc0 con 0x7f666c0b2850 2024-04-26T21:47:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 >> v1:172.21.15.40:6789/0 conn(0x7f666c0b2850 legacy=0x7f666c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66580039b0 0x7f6658003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6658004800 0x7f6658004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66580051c0 0x7f665800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f666c0b1d70 con 0x7f6658004800 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f665800c060 con 0x7f66580039b0 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f665800c260 con 0x7f66580051c0 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.089+0000 7f6662ffd640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6658004800 0x7f6658004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f666b7fe640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66580039b0 0x7f6658003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f666bfff640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66580051c0 0x7f665800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f6662ffd640 1 -- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66580051c0 msgr2=0x7f665800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f6662ffd640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66580051c0 0x7f665800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f6662ffd640 1 -- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66580039b0 msgr2=0x7f6658003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f6662ffd640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66580039b0 0x7f6658003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f6662ffd640 1 -- 172.21.15.40:0/3933370041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6658003520 con 0x7f6658004800 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f666bfff640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66580051c0 0x7f665800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f666bfff640 1 -- 172.21.15.40:0/3933370041 reap_dead start 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f6662ffd640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6658004800 0x7f6658004c80 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f6654009ac0 tx=0x7f6658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ae1d21253e032f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.090+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654036070 con 0x7f6658004800 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6654031400 con 0x7f6658004800 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f666affd640 1 -- 172.21.15.40:0/3933370041 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654009690 con 0x7f6658004800 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f6672025640 1 -- 172.21.15.40:0/3933370041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6658004800 msgr2=0x7f6658004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f6672025640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6658004800 0x7f6658004c80 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f6654009ac0 tx=0x7f6658005830 comp rx=0 tx=0).stop 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f6672025640 1 -- 172.21.15.40:0/3933370041 shutdown_connections 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f6672025640 1 --2- 172.21.15.40:0/3933370041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6658004800 0x7f6658004c80 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f6672025640 1 -- 172.21.15.40:0/3933370041 >> 172.21.15.40:0/3933370041 conn(0x7f666c0d4f90 msgr2=0x7f666c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f6672025640 1 -- 172.21.15.40:0/3933370041 shutdown_connections 2024-04-26T21:47:39.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.091+0000 7f6672025640 1 -- 172.21.15.40:0/3933370041 wait complete. 2024-04-26T21:47:39.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.092+0000 7f6672025640 1 Processor -- start 2024-04-26T21:47:39.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.092+0000 7f6672025640 1 -- start start 2024-04-26T21:47:39.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f6672025640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658004800 0x7f666c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f6672025640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f666c160900 0x7f666c1593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f6672025640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f666c0b88e0 0x7f666c0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f6672025640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f666c15f900 con 0x7f666c0b88e0 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f6672025640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f666c15f780 con 0x7f6658004800 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f6672025640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f666c0d1770 con 0x7f666c160900 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f666bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f666c0b88e0 0x7f666c0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f666bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f666c0b88e0 0x7f666c0b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42510/0 (socket says 172.21.15.40:42510) 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f666bfff640 1 -- 172.21.15.40:0/2404483266 learned_addr learned my addr 172.21.15.40:0/2404483266 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f6662ffd640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f666c160900 0x7f666c1593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.093+0000 7f666b7fe640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658004800 0x7f666c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f666bfff640 1 -- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f666c160900 msgr2=0x7f666c1593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f666bfff640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f666c160900 0x7f666c1593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f666bfff640 1 -- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658004800 msgr2=0x7f666c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f666bfff640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658004800 0x7f666c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f666bfff640 1 -- 172.21.15.40:0/2404483266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c0b9250 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f666bfff640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f666c0b88e0 0x7f666c0b4970 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f666c08e190 tx=0x7f6664007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f6668ff9640 1 -- 172.21.15.40:0/2404483266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664019070 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f6668ff9640 1 -- 172.21.15.40:0/2404483266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666400ea80 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c161000 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f6668ff9640 1 -- 172.21.15.40:0/2404483266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666400ee90 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.094+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c1615c0 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.095+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666c0694f0 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.095+0000 7f6668ff9640 1 -- 172.21.15.40:0/2404483266 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f666401c070 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.096+0000 7f6668ff9640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f664c06cac0 0x7f664c06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.096+0000 7f6668ff9640 1 -- 172.21.15.40:0/2404483266 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f666408b7f0 con 0x7f666c0b88e0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.096+0000 7f666b7fe640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f664c06cac0 0x7f664c06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.096+0000 7f666b7fe640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f664c06cac0 0x7f664c06ef80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f665c000c00 tx=0x7f665c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.100+0000 7f6668ff9640 1 -- 172.21.15.40:0/2404483266 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f666c0d0ba0 con 0x7f666c0b88e0 2024-04-26T21:47:39.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.326+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f666c0b6860 con 0x7f666c0b88e0 2024-04-26T21:47:39.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.326+0000 7f6668ff9640 1 -- 172.21.15.40:0/2404483266 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7f66640593f0 con 0x7f666c0b88e0 2024-04-26T21:47:39.327 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:39.327 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:39.327 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f664c06cac0 msgr2=0x7f664c06ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f664c06cac0 0x7f664c06ef80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f665c000c00 tx=0x7f665c009040 comp rx=0 tx=0).stop 2024-04-26T21:47:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f666c0b88e0 msgr2=0x7f666c0b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f666c0b88e0 0x7f666c0b4970 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f666c08e190 tx=0x7f6664007bc0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 shutdown_connections 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f664c06cac0 0x7f664c06ef80 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f666c0b88e0 0x7f666c0b4970 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f666c160900 0x7f666c1593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 --2- 172.21.15.40:0/2404483266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658004800 0x7f666c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.328+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 >> 172.21.15.40:0/2404483266 conn(0x7f666c0d4f90 msgr2=0x7f666c0d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.329+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 shutdown_connections 2024-04-26T21:47:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.329+0000 7f6672025640 1 -- 172.21.15.40:0/2404483266 wait complete. 2024-04-26T21:47:39.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 475567744 2024-04-26T21:47:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.443+0000 7f1171b19640 1 Processor -- start 2024-04-26T21:47:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.443+0000 7f1171b19640 1 -- start start 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.444+0000 7f1171b19640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c15f550 0x7f116c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.444+0000 7f1171b19640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f116c0d1040 con 0x7f116c0d3bc0 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.444+0000 7f1171b19640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f116c0c9680 con 0x7f116c15f170 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.444+0000 7f1171b19640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f116c0c9430 con 0x7f116c15f550 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.444+0000 7f116bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f116c0d3bc0 0x7f116c0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42264/0 (socket says 172.21.15.40:42264) 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.444+0000 7f116bfff640 1 -- 172.21.15.40:0/3904800325 learned_addr learned my addr 172.21.15.40:0/3904800325 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.444+0000 7f116affd640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c15f550 0x7f116c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.444+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993657926 0 0) 0x7f1164001490 con 0x7f116c0d3bc0 2024-04-26T21:47:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1154003520 con 0x7f116c0d3bc0 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2957510390 0 0) 0x7f1154003520 con 0x7f116c0d3bc0 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c15f550 msgr2=0x7f116c0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c15f550 0x7f116c0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 >> v1:172.21.15.184:6789/0 conn(0x7f116c15f170 legacy=0x7f116c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c0c9960 con 0x7f116c0d3bc0 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f1164001bc0 con 0x7f116c0d3bc0 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 >> v1:172.21.15.40:6789/0 conn(0x7f116c0d3bc0 legacy=0x7f116c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11540039e0 0x7f1154003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1154004340 0x7f1154004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1154005020 0x7f115400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f116c0c9430 con 0x7f1154005020 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f115400bf00 con 0x7f1154004340 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f115400c0b0 con 0x7f11540039e0 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116bfff640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1154005020 0x7f115400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116affd640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1154004340 0x7f1154004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116b7fe640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11540039e0 0x7f1154003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116bfff640 1 -- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11540039e0 msgr2=0x7f1154003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116bfff640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11540039e0 0x7f1154003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116bfff640 1 -- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1154004340 msgr2=0x7f1154004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116bfff640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1154004340 0x7f1154004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.445+0000 7f116bfff640 1 -- 172.21.15.40:0/3904800325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1154003520 con 0x7f1154005020 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.446+0000 7f116bfff640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1154005020 0x7f115400b9c0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f116400aa30 tx=0x7f1154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c3cc5cfa2e5d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.446+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116400a650 con 0x7f1154005020 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.446+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116402f070 con 0x7f1154005020 2024-04-26T21:47:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.446+0000 7f116a7fc640 1 -- 172.21.15.40:0/3904800325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164034470 con 0x7f1154005020 2024-04-26T21:47:39.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.447+0000 7f1171b19640 1 -- 172.21.15.40:0/3904800325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1154005020 msgr2=0x7f115400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.447+0000 7f1171b19640 1 --2- 172.21.15.40:0/3904800325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1154005020 0x7f115400b9c0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f116400aa30 tx=0x7f1154005830 comp rx=0 tx=0).stop 2024-04-26T21:47:39.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.447+0000 7f116bfff640 1 -- 172.21.15.40:0/3904800325 reap_dead start 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.447+0000 7f1171b19640 1 -- 172.21.15.40:0/3904800325 shutdown_connections 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.447+0000 7f1171b19640 1 -- 172.21.15.40:0/3904800325 >> 172.21.15.40:0/3904800325 conn(0x7f116c0d4f60 msgr2=0x7f116c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.447+0000 7f1171b19640 1 -- 172.21.15.40:0/3904800325 shutdown_connections 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.447+0000 7f1171b19640 1 -- 172.21.15.40:0/3904800325 wait complete. 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.448+0000 7f1171b19640 1 Processor -- start 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.448+0000 7f1171b19640 1 -- start start 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f1171b19640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c1524d0 0x7f116c15a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f1171b19640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f116c1528b0 0x7f116c15ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f1171b19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f116c158d70 0x7f116c159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f1171b19640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f116c0d5c00 con 0x7f116c158d70 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f1171b19640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f116c0d5a80 con 0x7f116c1528b0 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f1171b19640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f116c0d5d80 con 0x7f116c1524d0 2024-04-26T21:47:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f116c158d70 0x7f116c159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f116c158d70 0x7f116c159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42534/0 (socket says 172.21.15.40:42534) 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 -- 172.21.15.40:0/3557251203 learned_addr learned my addr 172.21.15.40:0/3557251203 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116b7fe640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c1524d0 0x7f116c15a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116affd640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f116c1528b0 0x7f116c15ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 -- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c1524d0 msgr2=0x7f116c15a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c1524d0 0x7f116c15a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 -- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f116c1528b0 msgr2=0x7f116c15ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f116c1528b0 0x7f116c15ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 -- 172.21.15.40:0/3557251203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c159a60 con 0x7f116c158d70 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.449+0000 7f116bfff640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f116c158d70 0x7f116c159200 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f116400aa30 tx=0x7f116c0c9b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.450+0000 7f1153fff640 1 -- 172.21.15.40:0/3557251203 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164038030 con 0x7f116c158d70 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.450+0000 7f1153fff640 1 -- 172.21.15.40:0/3557251203 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1164001250 con 0x7f116c158d70 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.450+0000 7f1171b19640 1 -- 172.21.15.40:0/3557251203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116c159d40 con 0x7f116c158d70 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.450+0000 7f1153fff640 1 -- 172.21.15.40:0/3557251203 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116402be60 con 0x7f116c158d70 2024-04-26T21:47:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.450+0000 7f1171b19640 1 -- 172.21.15.40:0/3557251203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116c0cc040 con 0x7f116c158d70 2024-04-26T21:47:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.451+0000 7f1153fff640 1 -- 172.21.15.40:0/3557251203 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1164034670 con 0x7f116c158d70 2024-04-26T21:47:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.452+0000 7f1153fff640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f115806c8f0 0x7f115806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.452+0000 7f1153fff640 1 -- 172.21.15.40:0/3557251203 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f11640abf60 con 0x7f116c158d70 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.452+0000 7f116b7fe640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f115806c8f0 0x7f115806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.452+0000 7f116b7fe640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f115806c8f0 0x7f115806edb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f115c0035f0 tx=0x7f115c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.452+0000 7f117483a740 1 Processor -- start 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.452+0000 7f117483a740 1 -- start start 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f117483a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0fc3180 0x5633e0f75b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f117483a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fa5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f117483a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fa5b60 0x5633e0f7a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f117483a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5633e0edff50 con 0x5633e0fa5b60 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f117483a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5633e0e7b920 con 0x5633e0fc3180 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f117483a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5633e0edbf10 con 0x5633e0fa62a0 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f116bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fa5b60 0x5633e0f7a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0fc3180 0x5633e0f75b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.453+0000 7f116bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fa5b60 0x5633e0f7a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42550/0 (socket says 172.21.15.40:42550) 2024-04-26T21:47:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f116bfff640 1 -- 172.21.15.40:0/3344245405 learned_addr learned my addr 172.21.15.40:0/3344245405 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f116affd640 1 --2- 172.21.15.40:0/3344245405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fa5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f116bfff640 1 -- 172.21.15.40:0/3344245405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 msgr2=0x5633e0fa5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f116bfff640 1 --2- 172.21.15.40:0/3344245405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fa5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f116bfff640 1 -- 172.21.15.40:0/3344245405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0fc3180 msgr2=0x5633e0f75b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f116bfff640 1 --2- 172.21.15.40:0/3344245405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0fc3180 0x5633e0f75b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f116bfff640 1 -- 172.21.15.40:0/3344245405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e0e7de80 con 0x5633e0fa5b60 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f116bfff640 1 --2- 172.21.15.40:0/3344245405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fa5b60 0x5633e0f7a970 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f1164002210 tx=0x7f1164079680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38951d32290c0570 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f11517fa640 1 -- 172.21.15.40:0/3344245405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116407ddc0 con 0x5633e0fa5b60 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f11517fa640 1 -- 172.21.15.40:0/3344245405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116402f040 con 0x5633e0fa5b60 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.454+0000 7f11517fa640 1 -- 172.21.15.40:0/3344245405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11640825f0 con 0x5633e0fa5b60 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 -- 172.21.15.40:0/3344245405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fa5b60 msgr2=0x5633e0f7a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 --2- 172.21.15.40:0/3344245405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fa5b60 0x5633e0f7a970 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f1164002210 tx=0x7f1164079680 comp rx=0 tx=0).stop 2024-04-26T21:47:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 -- 172.21.15.40:0/3344245405 shutdown_connections 2024-04-26T21:47:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 --2- 172.21.15.40:0/3344245405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fa5b60 0x5633e0f7a970 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 --2- 172.21.15.40:0/3344245405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fa5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 --2- 172.21.15.40:0/3344245405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0fc3180 0x5633e0f75b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 -- 172.21.15.40:0/3344245405 >> 172.21.15.40:0/3344245405 conn(0x5633e0f77d80 msgr2=0x5633e0fc0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 -- 172.21.15.40:0/3344245405 shutdown_connections 2024-04-26T21:47:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 -- 172.21.15.40:0/3344245405 wait complete. 2024-04-26T21:47:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.455+0000 7f117483a740 1 -- start start 2024-04-26T21:47:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f117483a740 1 Processor -- start 2024-04-26T21:47:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f117483a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fc1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f117483a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fa6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f117483a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0f95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f117483a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5633e0e7b920 con 0x5633e0fc3180 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f117483a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5633e0e792b0 con 0x5633e0f95a30 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f117483a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5633e0edff50 con 0x5633e0fa62a0 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f117483a740 10 client.0 _ll_register_callbacks cb 0x5633e0e86de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f116affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fa6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f116affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fa6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42564/0 (socket says 172.21.15.40:42564) 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f116affd640 1 -- 172.21.15.40:0/452372703 learned_addr learned my addr 172.21.15.40:0/452372703 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f116bfff640 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0f95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.456+0000 7f116b7fe640 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fc1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f116affd640 1 -- 172.21.15.40:0/452372703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 msgr2=0x5633e0fc1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f116affd640 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fc1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f116affd640 1 -- 172.21.15.40:0/452372703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 msgr2=0x5633e0f95eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f116affd640 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0f95eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f116affd640 1 -- 172.21.15.40:0/452372703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e0cbd900 con 0x5633e0fc3180 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f116affd640 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fa6790 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f1160017760 tx=0x7f1160017c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a67f9d176f95c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f114a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f114a7fc640 1 -- 172.21.15.40:0/452372703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160015c10 con 0x5633e0fc3180 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f114a7fc640 1 -- 172.21.15.40:0/452372703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f1160015db0 con 0x5633e0fc3180 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f117483a740 1 -- 172.21.15.40:0/452372703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116402f040 con 0x5633e0fc3180 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f114a7fc640 1 -- 172.21.15.40:0/452372703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f116001f3e0 con 0x5633e0fc3180 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f117483a740 10 client.12153 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f117483a740 1 -- 172.21.15.40:0/452372703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x5633e0bb11f0 con 0x5633e0fc3180 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f114a7fc640 1 -- 172.21.15.40:0/452372703 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116402f040 con 0x5633e0fc3180 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.457+0000 7f114a7fc640 1 -- 172.21.15.40:0/452372703 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f1160028050 con 0x5633e0fc3180 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f117483a740 10 client.12153 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f117483a740 10 client.12153 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f117483a740 20 client.12153 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f117483a740 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5633e0fa87b0 0x5633e0fa8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f117483a740 4 client.12153 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f117483a740 1 -- 172.21.15.40:0/452372703 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633e0e7e9e0 con 0x5633e0fa87b0 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f116b7fe640 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5633e0fa87b0 0x5633e0fa8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f116b7fe640 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5633e0fa87b0 0x5633e0fa8b90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27c04d58e567114 server_cookie=6fb9c5c95beb0398 in_seq=0 out_seq=0 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.458+0000 7f114a7fc640 10 client.12153 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f114a7fc640 1 -- 172.21.15.40:0/452372703 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633e0e7e9e0 con 0x5633e0fa87b0 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f114a7fc640 10 client.12153 handle_command_reply: tid=0 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 client.12153 shutdown 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 -- 172.21.15.40:0/452372703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 msgr2=0x5633e0fa6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fa6790 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f1160017760 tx=0x7f1160017c20 comp rx=0 tx=0).stop 2024-04-26T21:47:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 -- 172.21.15.40:0/452372703 shutdown_connections 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5633e0fa87b0 0x5633e0fa8b90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0f95eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fa6790 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f114a7fc640 0 client.12153 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 --2- 172.21.15.40:0/452372703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fc1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 -- 172.21.15.40:0/452372703 >> 172.21.15.40:0/452372703 conn(0x5633e0fa5b60 msgr2=0x5633e0f79600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 -- 172.21.15.40:0/452372703 shutdown_connections 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 1 -- 172.21.15.40:0/452372703 wait complete. 2024-04-26T21:47:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.459+0000 7f117483a740 20 client.12153 trim_cache size 0 max 16384 2024-04-26T21:47:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.472+0000 7f117483a740 1 Processor -- start 2024-04-26T21:47:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.472+0000 7f117483a740 1 -- start start 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f117483a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fb8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f117483a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fb8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f117483a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f117483a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f116c201800 con 0x5633e0fc3180 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f117483a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f116c0cda00 con 0x5633e0f95a30 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f117483a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f116c2011b0 con 0x5633e0fa62a0 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f116bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f116bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42572/0 (socket says 172.21.15.40:42572) 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f116bfff640 1 -- 172.21.15.40:0/958851133 learned_addr learned my addr 172.21.15.40:0/958851133 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.473+0000 7f116affd640 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fb8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116b7fe640 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fb8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116bfff640 1 -- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 msgr2=0x5633e0fb8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116bfff640 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fb8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116bfff640 1 -- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 msgr2=0x5633e0fb8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116bfff640 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fb8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116bfff640 1 -- 172.21.15.40:0/958851133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e0fa6b00 con 0x5633e0fc3180 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116affd640 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fb8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116b7fe640 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fb8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f116bfff640 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc2280 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f1164002210 tx=0x7f116407c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53b7e29d4c5c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f1150ff9640 1 -- 172.21.15.40:0/958851133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164075cb0 con 0x5633e0fc3180 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f1150ff9640 1 -- 172.21.15.40:0/958851133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11640a7070 con 0x5633e0fc3180 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f1150ff9640 1 -- 172.21.15.40:0/958851133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11640825f0 con 0x5633e0fc3180 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f117483a740 1 -- 172.21.15.40:0/958851133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 msgr2=0x5633e0fc2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.474+0000 7f117483a740 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc2280 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f1164002210 tx=0x7f116407c000 comp rx=0 tx=0).stop 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 -- 172.21.15.40:0/958851133 shutdown_connections 2024-04-26T21:47:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc2280 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0fb8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 --2- 172.21.15.40:0/958851133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fb8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 -- 172.21.15.40:0/958851133 >> 172.21.15.40:0/958851133 conn(0x5633e0fa5b60 msgr2=0x5633e0fc05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 -- 172.21.15.40:0/958851133 shutdown_connections 2024-04-26T21:47:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 -- 172.21.15.40:0/958851133 wait complete. 2024-04-26T21:47:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 -- start start 2024-04-26T21:47:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.475+0000 7f117483a740 1 Processor -- start 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f117483a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fc2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f117483a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0f756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f117483a740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f117483a740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5633e0fa6cd0 con 0x5633e0fc3180 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f117483a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5633e0fa7c10 con 0x5633e0f95a30 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f117483a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5633e0fd3f90 con 0x5633e0fa62a0 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f117483a740 10 client.0 _ll_register_callbacks cb 0x7f1160009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42582/0 (socket says 172.21.15.40:42582) 2024-04-26T21:47:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116bfff640 1 -- 172.21.15.40:0/3814633213 learned_addr learned my addr 172.21.15.40:0/3814633213 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116affd640 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0f756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116b7fe640 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fc2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116bfff640 1 -- 172.21.15.40:0/3814633213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 msgr2=0x5633e0f756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116bfff640 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0f756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116bfff640 1 -- 172.21.15.40:0/3814633213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 msgr2=0x5633e0fc2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116bfff640 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fc2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.476+0000 7f116bfff640 1 -- 172.21.15.40:0/3814633213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e0d1e090 con 0x5633e0fc3180 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f116bfff640 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc0100 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f1164002210 tx=0x7f11640021a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cbe2d62a056cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f114a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f114a7fc640 1 -- 172.21.15.40:0/3814633213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116407ddc0 con 0x5633e0fc3180 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f114a7fc640 1 -- 172.21.15.40:0/3814633213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f116409ccc0 con 0x5633e0fc3180 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 1 -- 172.21.15.40:0/3814633213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11640a7070 con 0x5633e0fc3180 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f114a7fc640 1 -- 172.21.15.40:0/3814633213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f11640a2050 con 0x5633e0fc3180 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 10 client.12159 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 1 -- 172.21.15.40:0/3814633213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x5633e0b8a370 con 0x5633e0fc3180 2024-04-26T21:47:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f114a7fc640 1 -- 172.21.15.40:0/3814633213 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11640a7070 con 0x5633e0fc3180 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f114a7fc640 1 -- 172.21.15.40:0/3814633213 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f1164095020 con 0x5633e0fc3180 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 10 client.12159 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 10 client.12159 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 20 client.12159 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5633e1020160 0x5633e0fc0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 4 client.12159 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 475567744}] 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f117483a740 1 -- 172.21.15.40:0/3814633213 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "damage rm", "damage_id": 475567744}) -- 0x5633e0eedb60 con 0x5633e1020160 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.477+0000 7f116b7fe640 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5633e1020160 0x5633e0fc0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f116b7fe640 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5633e1020160 0x5633e0fc0640 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1bcbc5694f20bb8 server_cookie=fe9d60b5a1feedeb in_seq=0 out_seq=0 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f114a7fc640 10 client.12159 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f114a7fc640 1 -- 172.21.15.40:0/3814633213 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x5633e0eedb60 con 0x5633e1020160 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f114a7fc640 10 client.12159 handle_command_reply: tid=0 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 client.12159 shutdown 2024-04-26T21:47:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 -- 172.21.15.40:0/3814633213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 msgr2=0x5633e0fc0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc0100 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f1164002210 tx=0x7f11640021a0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 -- 172.21.15.40:0/3814633213 shutdown_connections 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x5633e1020160 0x5633e0fc0640 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5633e0fc3180 0x5633e0fc0100 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633e0fa62a0 0x5633e0f756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 --2- 172.21.15.40:0/3814633213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633e0f95a30 0x5633e0fc2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f114a7fc640 0 client.12159 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 -- 172.21.15.40:0/3814633213 >> 172.21.15.40:0/3814633213 conn(0x5633e0fa5b60 msgr2=0x5633e0fb8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 -- 172.21.15.40:0/3814633213 shutdown_connections 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 1 -- 172.21.15.40:0/3814633213 wait complete. 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.478+0000 7f117483a740 20 client.12159 trim_cache size 0 max 16384 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 -- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f115806c8f0 msgr2=0x7f115806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f115806c8f0 0x7f115806edb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f115c0035f0 tx=0x7f115c009040 comp rx=0 tx=0).stop 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 -- 172.21.15.40:0/3557251203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f116c158d70 msgr2=0x7f116c159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f116c158d70 0x7f116c159200 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f116400aa30 tx=0x7f116c0c9b30 comp rx=0 tx=0).stop 2024-04-26T21:47:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 -- 172.21.15.40:0/3557251203 shutdown_connections 2024-04-26T21:47:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f115806c8f0 0x7f115806edb0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f116c158d70 0x7f116c159200 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f116c1528b0 0x7f116c15ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 --2- 172.21.15.40:0/3557251203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f116c1524d0 0x7f116c15a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 -- 172.21.15.40:0/3557251203 >> 172.21.15.40:0/3557251203 conn(0x7f116c0d4f60 msgr2=0x7f116c0b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 -- 172.21.15.40:0/3557251203 shutdown_connections 2024-04-26T21:47:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.479+0000 7f1171b19640 1 -- 172.21.15.40:0/3557251203 wait complete. 2024-04-26T21:47:39.489 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.590+0000 7f36a7de7640 1 Processor -- start 2024-04-26T21:47:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.590+0000 7f36a7de7640 1 -- start start 2024-04-26T21:47:39.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a7de7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36a00b5f20 0x7f36a00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a7de7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a00b7e50 con 0x7f36a0162880 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a7de7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a00d1800 con 0x7f36a00d3bc0 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a7de7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f36a00d0580 con 0x7f36a00b5f20 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a5b83640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f36a0162880 0x7f36a00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42280/0 (socket says 172.21.15.40:42280) 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a5b83640 1 -- 172.21.15.40:0/2738939820 learned_addr learned my addr 172.21.15.40:0/2738939820 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a6384640 1 --2- 172.21.15.40:0/2738939820 >> v2:172.21.15.184:3300/0 conn(0x7f36a00b5f20 0x7f36a00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a4b81640 1 -- 172.21.15.40:0/2738939820 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418096880 0 0) 0x7f36a00b7e50 con 0x7f36a0162880 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.591+0000 7f36a4b81640 1 -- 172.21.15.40:0/2738939820 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f368c003520 con 0x7f36a0162880 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.592+0000 7f36a4b81640 1 -- 172.21.15.40:0/2738939820 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 150328140 0 0) 0x7f368c003520 con 0x7f36a0162880 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.592+0000 7f36a6384640 1 -- 172.21.15.40:0/2738939820 >> v1:172.21.15.184:6789/0 conn(0x7f36a00d3bc0 legacy=0x7f36a00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.592+0000 7f36a6384640 1 -- 172.21.15.40:0/2738939820 >> v1:172.21.15.40:6789/0 conn(0x7f36a0162880 legacy=0x7f36a00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.592+0000 7f36a6384640 1 -- 172.21.15.40:0/2738939820 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a00d07d0 con 0x7f36a00b5f20 2024-04-26T21:47:39.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.592+0000 7f36a6384640 1 --2- 172.21.15.40:0/2738939820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00b5f20 0x7f36a00d2cb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3698009d90 tx=0x7f369802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8749026f79292d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a4b81640 1 -- 172.21.15.40:0/2738939820 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698035020 con 0x7f36a00b5f20 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a4b81640 1 -- 172.21.15.40:0/2738939820 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698007540 con 0x7f36a00b5f20 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a4b81640 1 -- 172.21.15.40:0/2738939820 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c001670 con 0x7f36a00b5f20 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a7de7640 1 -- 172.21.15.40:0/2738939820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00b5f20 msgr2=0x7f36a00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a7de7640 1 --2- 172.21.15.40:0/2738939820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00b5f20 0x7f36a00d2cb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3698009d90 tx=0x7f369802a5b0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a7de7640 1 -- 172.21.15.40:0/2738939820 shutdown_connections 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a7de7640 1 --2- 172.21.15.40:0/2738939820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00b5f20 0x7f36a00d2cb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a7de7640 1 -- 172.21.15.40:0/2738939820 >> 172.21.15.40:0/2738939820 conn(0x7f36a00d4f60 msgr2=0x7f36a00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a7de7640 1 -- 172.21.15.40:0/2738939820 shutdown_connections 2024-04-26T21:47:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.593+0000 7f36a7de7640 1 -- 172.21.15.40:0/2738939820 wait complete. 2024-04-26T21:47:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.594+0000 7f36a7de7640 1 Processor -- start 2024-04-26T21:47:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.594+0000 7f36a7de7640 1 -- start start 2024-04-26T21:47:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a7de7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a00b5f20 0x7f36a015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a7de7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00d3bc0 0x7f36a006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a7de7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0162880 0x7f36a006d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a7de7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0162c60 con 0x7f36a0162880 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a7de7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00d0d90 con 0x7f36a00d3bc0 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a7de7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36a0162de0 con 0x7f36a00b5f20 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a6384640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0162880 0x7f36a006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a6384640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0162880 0x7f36a006d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42586/0 (socket says 172.21.15.40:42586) 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a6384640 1 -- 172.21.15.40:0/1782038805 learned_addr learned my addr 172.21.15.40:0/1782038805 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a5382640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00d3bc0 0x7f36a006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.595+0000 7f36a5b83640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a00b5f20 0x7f36a015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a6384640 1 -- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a00b5f20 msgr2=0x7f36a015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a6384640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a00b5f20 0x7f36a015c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a6384640 1 -- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00d3bc0 msgr2=0x7f36a006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a6384640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00d3bc0 0x7f36a006cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a6384640 1 -- 172.21.15.40:0/1782038805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a01f84b0 con 0x7f36a0162880 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a5b83640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a00b5f20 0x7f36a015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a6384640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0162880 0x7f36a006d250 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f3698009d90 tx=0x7f3698004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36967fc640 1 -- 172.21.15.40:0/1782038805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698040070 con 0x7f36a0162880 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36967fc640 1 -- 172.21.15.40:0/1782038805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698031bc0 con 0x7f36a0162880 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a01f8790 con 0x7f36a0162880 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36967fc640 1 -- 172.21.15.40:0/1782038805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698035040 con 0x7f36a0162880 2024-04-26T21:47:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.596+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01f8d50 con 0x7f36a0162880 2024-04-26T21:47:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.597+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a00694f0 con 0x7f36a0162880 2024-04-26T21:47:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.598+0000 7f36967fc640 1 -- 172.21.15.40:0/1782038805 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3698043070 con 0x7f36a0162880 2024-04-26T21:47:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.598+0000 7f36967fc640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368406c920 0x7f368406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.598+0000 7f36967fc640 1 -- 172.21.15.40:0/1782038805 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f36980ac890 con 0x7f36a0162880 2024-04-26T21:47:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.598+0000 7f36a5b83640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368406c920 0x7f368406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.599+0000 7f36a5b83640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368406c920 0x7f368406ede0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f369c001eb0 tx=0x7f369c001d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.602+0000 7f36967fc640 1 -- 172.21.15.40:0/1782038805 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f369807a490 con 0x7f36a0162880 2024-04-26T21:47:39.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.825+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36a00d1260 con 0x7f36a0162880 2024-04-26T21:47:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.825+0000 7f36967fc640 1 -- 172.21.15.40:0/1782038805 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7f369807e790 con 0x7f36a0162880 2024-04-26T21:47:39.825 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:39.825 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368406c920 msgr2=0x7f368406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368406c920 0x7f368406ede0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f369c001eb0 tx=0x7f369c001d00 comp rx=0 tx=0).stop 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0162880 msgr2=0x7f36a006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0162880 0x7f36a006d250 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f3698009d90 tx=0x7f3698004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 shutdown_connections 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f368406c920 0x7f368406ede0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36a0162880 0x7f36a006d250 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a00d3bc0 0x7f36a006cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 --2- 172.21.15.40:0/1782038805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a00b5f20 0x7f36a015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 >> 172.21.15.40:0/1782038805 conn(0x7f36a00d4f60 msgr2=0x7f36a00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 shutdown_connections 2024-04-26T21:47:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.827+0000 7f36a7de7640 1 -- 172.21.15.40:0/1782038805 wait complete. 2024-04-26T21:47:39.838 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2110519633 2024-04-26T21:47:39.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.942+0000 7f9646f9d640 1 Processor -- start 2024-04-26T21:47:39.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.942+0000 7f9646f9d640 1 -- start start 2024-04-26T21:47:39.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.943+0000 7f9646f9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f964015f170 0x7f96400d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.943+0000 7f9646f9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 0x7f96400d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.943+0000 7f9646f9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96400d1040 con 0x7f964015f550 2024-04-26T21:47:39.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.943+0000 7f9646f9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f96400c9430 con 0x7f964015f170 2024-04-26T21:47:39.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.943+0000 7f9646f9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96400c9680 con 0x7f96400d3bc0 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.943+0000 7f9644d39640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f964015f170 0x7f96400d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.943+0000 7f963ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f964015f550 0x7f96400d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42288/0 (socket says 172.21.15.40:42288) 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.943+0000 7f963ffff640 1 -- 172.21.15.40:0/3584671882 learned_addr learned my addr 172.21.15.40:0/3584671882 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.944+0000 7f964553a640 1 --2- 172.21.15.40:0/3584671882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 0x7f96400d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.944+0000 7f9644d39640 1 -- 172.21.15.40:0/3584671882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 msgr2=0x7f96400d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.944+0000 7f9644d39640 1 --2- 172.21.15.40:0/3584671882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 0x7f96400d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.944+0000 7f9644d39640 1 -- 172.21.15.40:0/3584671882 >> v1:172.21.15.40:6789/0 conn(0x7f964015f550 legacy=0x7f96400d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.944+0000 7f9644d39640 1 -- 172.21.15.40:0/3584671882 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96400c9960 con 0x7f964015f170 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.944+0000 7f963f7fe640 1 -- 172.21.15.40:0/3584671882 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178347567 0 0) 0x7f9630001110 con 0x7f964015f550 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.944+0000 7f9644d39640 1 --2- 172.21.15.40:0/3584671882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400d5a80 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f9638009dc0 tx=0x7f963802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756e4a9c62116770 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.944+0000 7f963f7fe640 1 -- 172.21.15.40:0/3584671882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963802ae60 con 0x7f964015f170 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f963f7fe640 1 -- 172.21.15.40:0/3584671882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9638007740 con 0x7f964015f170 2024-04-26T21:47:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f963f7fe640 1 -- 172.21.15.40:0/3584671882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638007b10 con 0x7f964015f170 2024-04-26T21:47:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f9646f9d640 1 -- 172.21.15.40:0/3584671882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 msgr2=0x7f96400d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f9646f9d640 1 --2- 172.21.15.40:0/3584671882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400d5a80 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f9638009dc0 tx=0x7f963802a680 comp rx=0 tx=0).stop 2024-04-26T21:47:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f9646f9d640 1 -- 172.21.15.40:0/3584671882 shutdown_connections 2024-04-26T21:47:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f9646f9d640 1 --2- 172.21.15.40:0/3584671882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 0x7f96400d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f9646f9d640 1 --2- 172.21.15.40:0/3584671882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400d5a80 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f9646f9d640 1 -- 172.21.15.40:0/3584671882 >> 172.21.15.40:0/3584671882 conn(0x7f96400d4f60 msgr2=0x7f96400d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f9646f9d640 1 -- 172.21.15.40:0/3584671882 shutdown_connections 2024-04-26T21:47:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.945+0000 7f9646f9d640 1 -- 172.21.15.40:0/3584671882 wait complete. 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.946+0000 7f9646f9d640 1 Processor -- start 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.946+0000 7f9646f9d640 1 -- start start 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f9646f9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 0x7f96401507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f9646f9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f9646f9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f964009f7f0 0x7f96400ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f9646f9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96400ca5c0 con 0x7f964015f170 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f9646f9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96400ca440 con 0x7f964009f7f0 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f9646f9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96400ca740 con 0x7f96400d3bc0 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f963ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f963ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42600/0 (socket says 172.21.15.40:42600) 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.947+0000 7f963ffff640 1 -- 172.21.15.40:0/1874432798 learned_addr learned my addr 172.21.15.40:0/1874432798 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f9644d39640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 0x7f96401507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f964553a640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f964009f7f0 0x7f96400ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963ffff640 1 -- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 msgr2=0x7f96401507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963ffff640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 0x7f96401507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963ffff640 1 -- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f964009f7f0 msgr2=0x7f96400ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963ffff640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f964009f7f0 0x7f96400ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963ffff640 1 -- 172.21.15.40:0/1874432798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9640151300 con 0x7f964015f170 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f964553a640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f964009f7f0 0x7f96400ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963ffff640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400ce170 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f963000c3e0 tx=0x7f963000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963d7fa640 1 -- 172.21.15.40:0/1874432798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300074e0 con 0x7f964015f170 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963d7fa640 1 -- 172.21.15.40:0/1874432798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96300047e0 con 0x7f964015f170 2024-04-26T21:47:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f9646f9d640 1 -- 172.21.15.40:0/1874432798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96401515e0 con 0x7f964015f170 2024-04-26T21:47:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f963d7fa640 1 -- 172.21.15.40:0/1874432798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630004bd0 con 0x7f964015f170 2024-04-26T21:47:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.948+0000 7f9646f9d640 1 -- 172.21.15.40:0/1874432798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9640151ba0 con 0x7f964015f170 2024-04-26T21:47:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.949+0000 7f963d7fa640 1 -- 172.21.15.40:0/1874432798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9630011070 con 0x7f964015f170 2024-04-26T21:47:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.950+0000 7f963d7fa640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962006c8f0 0x7f962006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.950+0000 7f963d7fa640 1 -- 172.21.15.40:0/1874432798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f963008b960 con 0x7f964015f170 2024-04-26T21:47:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.950+0000 7f9644d39640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962006c8f0 0x7f962006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.951+0000 7f9644d39640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962006c8f0 0x7f962006edb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9638034000 tx=0x7f963802f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.951+0000 7f9649d8f740 1 Processor -- start 2024-04-26T21:47:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.951+0000 7f9649d8f740 1 -- start start 2024-04-26T21:47:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 0x55e596a2fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 0x55e596a5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a5fed0 0x55e596a34ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f9649d8f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e596999ad0 con 0x55e596a5fed0 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f9649d8f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e596953a00 con 0x55e596a60610 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f9649d8f740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e596995a90 con 0x55e596a7d4f0 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a5fed0 0x55e596a34ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a5fed0 0x55e596a34ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42614/0 (socket says 172.21.15.40:42614) 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 -- 172.21.15.40:0/2658683821 learned_addr learned my addr 172.21.15.40:0/2658683821 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f963ffff640 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 0x55e596a5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f9644d39640 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 0x55e596a2fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 -- 172.21.15.40:0/2658683821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 msgr2=0x55e596a2fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 0x55e596a2fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 -- 172.21.15.40:0/2658683821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 msgr2=0x55e596a5f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 0x55e596a5f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 -- 172.21.15.40:0/2658683821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e596777710 con 0x55e596a5fed0 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f964553a640 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a5fed0 0x55e596a34ce0 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f9628009a40 tx=0x7f9628009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741591ad97c9a3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f96367fc640 1 -- 172.21.15.40:0/2658683821 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962800c070 con 0x55e596a5fed0 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f96367fc640 1 -- 172.21.15.40:0/2658683821 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9628018070 con 0x55e596a5fed0 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.952+0000 7f96367fc640 1 -- 172.21.15.40:0/2658683821 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962800f040 con 0x55e596a5fed0 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 -- 172.21.15.40:0/2658683821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a5fed0 msgr2=0x55e596a34ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a5fed0 0x55e596a34ce0 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f9628009a40 tx=0x7f9628009f00 comp rx=0 tx=0).stop 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 -- 172.21.15.40:0/2658683821 shutdown_connections 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a5fed0 0x55e596a34ce0 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 0x55e596a5f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2658683821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 0x55e596a2fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 -- 172.21.15.40:0/2658683821 >> 172.21.15.40:0/2658683821 conn(0x55e596a320f0 msgr2=0x55e596a7a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 -- 172.21.15.40:0/2658683821 shutdown_connections 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.953+0000 7f9649d8f740 1 -- 172.21.15.40:0/2658683821 wait complete. 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.954+0000 7f9649d8f740 1 -- start start 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.954+0000 7f9649d8f740 1 Processor -- start 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a60610 0x55e596a5b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a8b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a622c0 0x55e596a62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9649d8f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e596a342a0 con 0x55e596a60610 2024-04-26T21:47:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9649d8f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e596921000 con 0x55e596a7d4f0 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9649d8f740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e596a33690 con 0x55e596a622c0 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9649d8f740 10 client.0 _ll_register_callbacks cb 0x55e59693c620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a60610 0x55e596a5b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a60610 0x55e596a5b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42626/0 (socket says 172.21.15.40:42626) 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 -- 172.21.15.40:0/1020251573 learned_addr learned my addr 172.21.15.40:0/1020251573 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f964553a640 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a622c0 0x55e596a62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 -- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a622c0 msgr2=0x55e596a62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f963ffff640 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a8b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a622c0 0x55e596a62ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 -- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 msgr2=0x55e596a8b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a8b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 -- 172.21.15.40:0/1020251573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5967d7ea0 con 0x55e596a60610 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f964553a640 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a622c0 0x55e596a62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9644d39640 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a60610 0x55e596a5b3c0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f963802ac70 tx=0x7f9638004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff5f07437239206 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f961f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f961f7fe640 1 -- 172.21.15.40:0/1020251573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963803db80 con 0x55e596a60610 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f961f7fe640 1 -- 172.21.15.40:0/1020251573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f963803dd20 con 0x55e596a60610 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.955+0000 7f9649d8f740 1 -- 172.21.15.40:0/1020251573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9628018070 con 0x55e596a60610 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f961f7fe640 1 -- 172.21.15.40:0/1020251573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f9638052420 con 0x55e596a60610 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f9649d8f740 10 client.12180 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f9649d8f740 1 -- 172.21.15.40:0/1020251573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x55e59666b000 con 0x55e596a60610 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f961f7fe640 1 -- 172.21.15.40:0/1020251573 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9628018070 con 0x55e596a60610 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f961f7fe640 1 -- 172.21.15.40:0/1020251573 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f963805d080 con 0x55e596a60610 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f9649d8f740 10 client.12180 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f9649d8f740 10 client.12180 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f9649d8f740 20 client.12180 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f9649d8f740 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x55e596a8e650 0x55e596a8ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f9649d8f740 4 client.12180 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f9649d8f740 1 -- 172.21.15.40:0/1020251573 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5969a76e0 con 0x55e596a8e650 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f963ffff640 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x55e596a8e650 0x55e596a8ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f963ffff640 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x55e596a8e650 0x55e596a8ea50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c694393d44156a server_cookie=46d0de4eda23bd6d in_seq=0 out_seq=0 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.956+0000 7f961f7fe640 10 client.12180 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f961f7fe640 1 -- 172.21.15.40:0/1020251573 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5969a76e0 con 0x55e596a8e650 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f961f7fe640 10 client.12180 handle_command_reply: tid=0 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 client.12180 shutdown 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 -- 172.21.15.40:0/1020251573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a60610 msgr2=0x55e596a5b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a60610 0x55e596a5b3c0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f963802ac70 tx=0x7f9638004ab0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 -- 172.21.15.40:0/1020251573 shutdown_connections 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x55e596a8e650 0x55e596a8ea50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a622c0 0x55e596a62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a8b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f961f7fe640 0 client.12180 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 --2- 172.21.15.40:0/1020251573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a60610 0x55e596a5b3c0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 -- 172.21.15.40:0/1020251573 >> 172.21.15.40:0/1020251573 conn(0x55e596a5fed0 msgr2=0x55e596a308e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 -- 172.21.15.40:0/1020251573 shutdown_connections 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 1 -- 172.21.15.40:0/1020251573 wait complete. 2024-04-26T21:47:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.957+0000 7f9649d8f740 20 client.12180 trim_cache size 0 max 16384 2024-04-26T21:47:39.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.971+0000 7f9649d8f740 1 Processor -- start 2024-04-26T21:47:39.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.971+0000 7f9649d8f740 1 -- start start 2024-04-26T21:47:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.971+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 0x55e596a9e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a71750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 0x55e596a7ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f9649d8f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9640200ea0 con 0x55e596a622c0 2024-04-26T21:47:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f9649d8f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96400ba950 con 0x55e596a60610 2024-04-26T21:47:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f9649d8f740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9640200850 con 0x55e596a7d4f0 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a71750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a71750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42640/0 (socket says 172.21.15.40:42640) 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 -- 172.21.15.40:0/2192841805 learned_addr learned my addr 172.21.15.40:0/2192841805 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 -- 172.21.15.40:0/2192841805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 msgr2=0x55e596a7ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 --2- 172.21.15.40:0/2192841805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 0x55e596a7ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 -- 172.21.15.40:0/2192841805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 msgr2=0x55e596a9e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 --2- 172.21.15.40:0/2192841805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 0x55e596a9e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 -- 172.21.15.40:0/2192841805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e596a30650 con 0x55e596a622c0 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f963ffff640 1 --2- 172.21.15.40:0/2192841805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a71750 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f9630089cb0 tx=0x7f9630007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ffaf863abba13c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f9635ffb640 1 -- 172.21.15.40:0/2192841805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630076070 con 0x55e596a622c0 2024-04-26T21:47:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f9635ffb640 1 -- 172.21.15.40:0/2192841805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9630001080 con 0x55e596a622c0 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.972+0000 7f9635ffb640 1 -- 172.21.15.40:0/2192841805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300577b0 con 0x55e596a622c0 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 -- 172.21.15.40:0/2192841805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 msgr2=0x55e596a71750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2192841805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a71750 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f9630089cb0 tx=0x7f9630007f90 comp rx=0 tx=0).stop 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 -- 172.21.15.40:0/2192841805 shutdown_connections 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2192841805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a7d4f0 0x55e596a7ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2192841805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a71750 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2192841805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a60610 0x55e596a9e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 -- 172.21.15.40:0/2192841805 >> 172.21.15.40:0/2192841805 conn(0x55e596a5fed0 msgr2=0x55e596a2eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 -- 172.21.15.40:0/2192841805 shutdown_connections 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 -- 172.21.15.40:0/2192841805 wait complete. 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.973+0000 7f9649d8f740 1 -- start start 2024-04-26T21:47:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f9649d8f740 1 Processor -- start 2024-04-26T21:47:39.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a60610 0x55e596a771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a77730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f9649d8f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a59d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f9649d8f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55e596a5e5e0 con 0x55e596a622c0 2024-04-26T21:47:39.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f9649d8f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e596a61b80 con 0x55e596a7d4f0 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f9649d8f740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e596a5daf0 con 0x55e596a60610 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f9649d8f740 10 client.0 _ll_register_callbacks cb 0x7f9638009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f963ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a77730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f963ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a77730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42646/0 (socket says 172.21.15.40:42646) 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.974+0000 7f963ffff640 1 -- 172.21.15.40:0/2268024683 learned_addr learned my addr 172.21.15.40:0/2268024683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9644d39640 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a60610 0x55e596a771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f964553a640 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a59d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f963ffff640 1 -- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a60610 msgr2=0x55e596a771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f963ffff640 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a60610 0x55e596a771f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f963ffff640 1 -- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 msgr2=0x55e596a59d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f963ffff640 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a59d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f963ffff640 1 -- 172.21.15.40:0/2268024683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e596a5e800 con 0x55e596a622c0 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f964553a640 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f961f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f963ffff640 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a77730 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f9630059440 tx=0x7f9630000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc02411e0020233 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f961f7fe640 1 -- 172.21.15.40:0/2268024683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630088060 con 0x55e596a622c0 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f961f7fe640 1 -- 172.21.15.40:0/2268024683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f9630004d70 con 0x55e596a622c0 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 1 -- 172.21.15.40:0/2268024683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9630001080 con 0x55e596a622c0 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f961f7fe640 1 -- 172.21.15.40:0/2268024683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=363) ==== 24+0+0 (secure 0 0 0) 0x7f9630076070 con 0x55e596a622c0 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 10 client.12186 fetch_fsmap learned FSMap version 363 2024-04-26T21:47:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 1 -- 172.21.15.40:0/2268024683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=363,monmap=2+}) -- 0x55e596a5ab90 con 0x55e596a622c0 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f961f7fe640 1 -- 172.21.15.40:0/2268024683 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9630001080 con 0x55e596a622c0 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f961f7fe640 1 -- 172.21.15.40:0/2268024683 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 363) ==== 3494+0+0 (secure 0 0 0) 0x7f9630055b20 con 0x55e596a622c0 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 10 client.12186 fetch_fsmap finished waiting for FSMap version 363 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 10 client.12186 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 20 client.12186 populate_metadata read hostname 'smithi040' 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x55e596a5a2a0 0x55e596a9d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 4 client.12186 mds_command: new command op to 11961 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2110519633}] 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.975+0000 7f9649d8f740 1 -- 172.21.15.40:0/2268024683 --> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2110519633}) -- 0x55e5969a1550 con 0x55e596a5a2a0 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9644d39640 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x55e596a5a2a0 0x55e596a9d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f961f7fe640 10 client.12186 ms_handle_connect on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9644d39640 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x55e596a5a2a0 0x55e596a9d870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b0c061398a8a0b server_cookie=ad784d32b449e739 in_seq=0 out_seq=0 2024-04-26T21:47:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f961f7fe640 1 -- 172.21.15.40:0/2268024683 <== mds.0 v2:172.21.15.40:6834/3119057314 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55e5969a1550 con 0x55e596a5a2a0 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f961f7fe640 10 client.12186 handle_command_reply: tid=0 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 client.12186 shutdown 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 -- 172.21.15.40:0/2268024683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 msgr2=0x55e596a77730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a77730 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f9630059440 tx=0x7f9630000ed0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 -- 172.21.15.40:0/2268024683 shutdown_connections 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.40:6834/3119057314,v1:172.21.15.40:6835/3119057314] conn(0x55e596a5a2a0 0x55e596a9d870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e596a7d4f0 0x55e596a59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e596a622c0 0x55e596a77730 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f961f7fe640 0 client.12186 ms_handle_reset on v2:172.21.15.40:6834/3119057314 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 --2- 172.21.15.40:0/2268024683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e596a60610 0x55e596a771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 -- 172.21.15.40:0/2268024683 >> 172.21.15.40:0/2268024683 conn(0x55e596a5fed0 msgr2=0x55e596a626a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 -- 172.21.15.40:0/2268024683 shutdown_connections 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.976+0000 7f9649d8f740 1 -- 172.21.15.40:0/2268024683 wait complete. 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9649d8f740 20 client.12186 trim_cache size 0 max 16384 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 -- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962006c8f0 msgr2=0x7f962006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962006c8f0 0x7f962006edb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9638034000 tx=0x7f963802f000 comp rx=0 tx=0).stop 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 -- 172.21.15.40:0/1874432798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 msgr2=0x7f96400ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400ce170 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f963000c3e0 tx=0x7f963000c8a0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 -- 172.21.15.40:0/1874432798 shutdown_connections 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f962006c8f0 0x7f962006edb0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f964009f7f0 0x7f96400ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964015f170 0x7f96400ce170 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 --2- 172.21.15.40:0/1874432798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96400d3bc0 0x7f96401507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:39.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 -- 172.21.15.40:0/1874432798 >> 172.21.15.40:0/1874432798 conn(0x7f96400d4f60 msgr2=0x7f96400d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:39.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 -- 172.21.15.40:0/1874432798 shutdown_connections 2024-04-26T21:47:39.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:39.977+0000 7f9646f9d640 1 -- 172.21.15.40:0/1874432798 wait complete. 2024-04-26T21:47:39.987 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:47:39.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:40.031 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:47:40.079 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:47:40.079 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:40.079 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:40.079 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:40.095 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:40.095 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:40.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:40.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.292+0000 7fcb4fd24640 1 Processor -- start 2024-04-26T21:47:40.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.292+0000 7fcb4fd24640 1 -- start start 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4fd24640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcb480b4480 0x7fcb4806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4fd24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4806c780 0x7fcb480af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4fd24640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb480b49f0 con 0x7fcb4815f120 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4fd24640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcb4815f500 con 0x7fcb480b4480 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4fd24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcb480ae8c0 con 0x7fcb4806c780 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4e2c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4806c780 0x7fcb480af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4d2bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcb480b4480 0x7fcb4806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4e2c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4806c780 0x7fcb480af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:52124/0 (socket says 172.21.15.40:52124) 2024-04-26T21:47:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.293+0000 7fcb4e2c1640 1 -- 172.21.15.40:0/2864565824 learned_addr learned my addr 172.21.15.40:0/2864565824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.294+0000 7fcb4d2bf640 1 -- 172.21.15.40:0/2864565824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4806c780 msgr2=0x7fcb480af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.294+0000 7fcb4d2bf640 1 --2- 172.21.15.40:0/2864565824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4806c780 0x7fcb480af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.294+0000 7fcb4d2bf640 1 -- 172.21.15.40:0/2864565824 >> v1:172.21.15.184:6789/0 conn(0x7fcb4815f120 legacy=0x7fcb4806f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.294+0000 7fcb4d2bf640 1 -- 172.21.15.40:0/2864565824 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb480b4cd0 con 0x7fcb480b4480 2024-04-26T21:47:40.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.294+0000 7fcb4e2c1640 1 --2- 172.21.15.40:0/2864565824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4806c780 0x7fcb480af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.294+0000 7fcb4d2bf640 1 --2- 172.21.15.40:0/2864565824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb480b4480 0x7fcb4806c240 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcb380097a0 tx=0x7fcb3802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3fc2da89268124 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4cabe640 1 -- 172.21.15.40:0/2864565824 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38037070 con 0x7fcb480b4480 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4cabe640 1 -- 172.21.15.40:0/2864565824 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3802ae50 con 0x7fcb480b4480 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4cabe640 1 -- 172.21.15.40:0/2864565824 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38032660 con 0x7fcb480b4480 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2864565824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb480b4480 msgr2=0x7fcb4806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2864565824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb480b4480 0x7fcb4806c240 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcb380097a0 tx=0x7fcb3802a100 comp rx=0 tx=0).stop 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2864565824 shutdown_connections 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2864565824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4806c780 0x7fcb480af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2864565824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb480b4480 0x7fcb4806c240 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2864565824 >> 172.21.15.40:0/2864565824 conn(0x7fcb480b3180 msgr2=0x7fcb480b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:40.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2864565824 shutdown_connections 2024-04-26T21:47:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.295+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2864565824 wait complete. 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.296+0000 7fcb4fd24640 1 Processor -- start 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.296+0000 7fcb4fd24640 1 -- start start 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4fd24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb4806c780 0x7fcb480c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4fd24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480b4480 0x7fcb4815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4fd24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4815f120 0x7fcb4815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4fd24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb480af570 con 0x7fcb480b4480 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4fd24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb480aeba0 con 0x7fcb4806c780 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4fd24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcb480af6f0 con 0x7fcb4815f120 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4d2bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480b4480 0x7fcb4815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4d2bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480b4480 0x7fcb4815ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42652/0 (socket says 172.21.15.40:42652) 2024-04-26T21:47:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4d2bf640 1 -- 172.21.15.40:0/2524702585 learned_addr learned my addr 172.21.15.40:0/2524702585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4e2c1640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4815f120 0x7fcb4815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.297+0000 7fcb4dac0640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb4806c780 0x7fcb480c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4d2bf640 1 -- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4815f120 msgr2=0x7fcb4815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4d2bf640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4815f120 0x7fcb4815cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4d2bf640 1 -- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb4806c780 msgr2=0x7fcb480c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4d2bf640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb4806c780 0x7fcb480c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4d2bf640 1 -- 172.21.15.40:0/2524702585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb48151060 con 0x7fcb480b4480 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4e2c1640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4815f120 0x7fcb4815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4d2bf640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480b4480 0x7fcb4815ca80 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fcb3802a140 tx=0x7fcb380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb3e7fc640 1 -- 172.21.15.40:0/2524702585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38041070 con 0x7fcb480b4480 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb3e7fc640 1 -- 172.21.15.40:0/2524702585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3803c670 con 0x7fcb480b4480 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb3e7fc640 1 -- 172.21.15.40:0/2524702585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38037040 con 0x7fcb480b4480 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb48151340 con 0x7fcb480b4480 2024-04-26T21:47:40.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.298+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb48151900 con 0x7fcb480b4480 2024-04-26T21:47:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.299+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb48068fc0 con 0x7fcb480b4480 2024-04-26T21:47:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.299+0000 7fcb3e7fc640 1 -- 172.21.15.40:0/2524702585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb38044070 con 0x7fcb480b4480 2024-04-26T21:47:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.300+0000 7fcb3e7fc640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb2c06c8f0 0x7fcb2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.300+0000 7fcb3e7fc640 1 -- 172.21.15.40:0/2524702585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fcb380ac360 con 0x7fcb480b4480 2024-04-26T21:47:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.300+0000 7fcb4dac0640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb2c06c8f0 0x7fcb2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.301+0000 7fcb4dac0640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb2c06c8f0 0x7fcb2c06edb0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcb44007fd0 tx=0x7fcb44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.304+0000 7fcb3e7fc640 1 -- 172.21.15.40:0/2524702585 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb3807a640 con 0x7fcb480b4480 2024-04-26T21:47:40.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.493+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb4806f8f0 con 0x7fcb480b4480 2024-04-26T21:47:40.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.494+0000 7fcb3e7fc640 1 -- 172.21.15.40:0/2524702585 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16977 (secure 0 0 0) 0x7fcb38079f40 con 0x7fcb480b4480 2024-04-26T21:47:40.494 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:40.495 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":146,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:30.391272+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:18.726844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:19.403532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3441996159":"2024-04-27T21:47:30.391245+0000","172.21.15.40:6834/3441996159":"2024-04-27T21:47:30.391245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:40.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb2c06c8f0 msgr2=0x7fcb2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb2c06c8f0 0x7fcb2c06edb0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcb44007fd0 tx=0x7fcb44009040 comp rx=0 tx=0).stop 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480b4480 msgr2=0x7fcb4815ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480b4480 0x7fcb4815ca80 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fcb3802a140 tx=0x7fcb380046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 shutdown_connections 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb2c06c8f0 0x7fcb2c06edb0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb4815f120 0x7fcb4815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480b4480 0x7fcb4815ca80 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 --2- 172.21.15.40:0/2524702585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb4806c780 0x7fcb480c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.496+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 >> 172.21.15.40:0/2524702585 conn(0x7fcb480b3180 msgr2=0x7fcb480b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.497+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 shutdown_connections 2024-04-26T21:47:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.497+0000 7fcb4fd24640 1 -- 172.21.15.40:0/2524702585 wait complete. 2024-04-26T21:47:40.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:18.726844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:47:19.403532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:40.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:40.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.612+0000 7fc499212640 1 Processor -- start 2024-04-26T21:47:40.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.612+0000 7fc499212640 1 -- start start 2024-04-26T21:47:40.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc499212640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc494156460 0x7fc4940cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc499212640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc49406fea0 0x7fc494072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc499212640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc494156840 con 0x7fc4940cba10 2024-04-26T21:47:40.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc499212640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc4940cbf20 con 0x7fc494156460 2024-04-26T21:47:40.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc499212640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc4940cc200 con 0x7fc49406fea0 2024-04-26T21:47:40.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc492d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc494156460 0x7fc4940cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc48bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc4940cba10 0x7fc49406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42304/0 (socket says 172.21.15.40:42304) 2024-04-26T21:47:40.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc48bfff640 1 -- 172.21.15.40:0/3667834498 learned_addr learned my addr 172.21.15.40:0/3667834498 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:40.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc49359e640 1 --2- 172.21.15.40:0/3667834498 >> v2:172.21.15.184:3300/0 conn(0x7fc49406fea0 0x7fc494072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc492d9d640 1 -- 172.21.15.40:0/3667834498 >> v2:172.21.15.184:3300/0 conn(0x7fc49406fea0 msgr2=0x7fc494072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:40.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc492d9d640 1 --2- 172.21.15.40:0/3667834498 >> v2:172.21.15.184:3300/0 conn(0x7fc49406fea0 0x7fc494072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc492d9d640 1 -- 172.21.15.40:0/3667834498 >> v1:172.21.15.40:6789/0 conn(0x7fc4940cba10 legacy=0x7fc49406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc49259c640 1 -- 172.21.15.40:0/3667834498 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3058615236 0 0) 0x7fc494156840 con 0x7fc4940cba10 2024-04-26T21:47:40.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.613+0000 7fc492d9d640 1 -- 172.21.15.40:0/3667834498 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4940cc4e0 con 0x7fc494156460 2024-04-26T21:47:40.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.614+0000 7fc492d9d640 1 --2- 172.21.15.40:0/3667834498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc494156460 0x7fc4940cb4d0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fc4940c7c40 tx=0x7fc48402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699775f987287b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.614+0000 7fc49259c640 1 -- 172.21.15.40:0/3667834498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484037070 con 0x7fc494156460 2024-04-26T21:47:40.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.614+0000 7fc49259c640 1 -- 172.21.15.40:0/3667834498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc484032440 con 0x7fc494156460 2024-04-26T21:47:40.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.614+0000 7fc49259c640 1 -- 172.21.15.40:0/3667834498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484032810 con 0x7fc494156460 2024-04-26T21:47:40.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.615+0000 7fc499212640 1 -- 172.21.15.40:0/3667834498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc494156460 msgr2=0x7fc4940cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.615+0000 7fc499212640 1 --2- 172.21.15.40:0/3667834498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc494156460 0x7fc4940cb4d0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fc4940c7c40 tx=0x7fc48402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.615+0000 7fc499212640 1 -- 172.21.15.40:0/3667834498 shutdown_connections 2024-04-26T21:47:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.615+0000 7fc499212640 1 --2- 172.21.15.40:0/3667834498 >> v2:172.21.15.184:3300/0 conn(0x7fc49406fea0 0x7fc494072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.615+0000 7fc499212640 1 --2- 172.21.15.40:0/3667834498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc494156460 0x7fc4940cb4d0 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.615+0000 7fc499212640 1 -- 172.21.15.40:0/3667834498 >> 172.21.15.40:0/3667834498 conn(0x7fc4940d0be0 msgr2=0x7fc4940d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.615+0000 7fc499212640 1 -- 172.21.15.40:0/3667834498 shutdown_connections 2024-04-26T21:47:40.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.615+0000 7fc499212640 1 -- 172.21.15.40:0/3667834498 wait complete. 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.616+0000 7fc499212640 1 Processor -- start 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.616+0000 7fc499212640 1 -- start start 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc499212640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc49406fea0 0x7fc4940c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc499212640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940cba10 0x7fc4941f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc499212640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc494156460 0x7fc4940c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc499212640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc49406e260 con 0x7fc4940cba10 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc499212640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc494072700 con 0x7fc494156460 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc499212640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc49406e3e0 con 0x7fc49406fea0 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940cba10 0x7fc4941f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940cba10 0x7fc4941f2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42668/0 (socket says 172.21.15.40:42668) 2024-04-26T21:47:40.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 -- 172.21.15.40:0/757056686 learned_addr learned my addr 172.21.15.40:0/757056686 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc49359e640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc494156460 0x7fc4940c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc492d9d640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc49406fea0 0x7fc4940c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 -- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc49406fea0 msgr2=0x7fc4940c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc49406fea0 0x7fc4940c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 -- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc494156460 msgr2=0x7fc4940c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc494156460 0x7fc4940c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 -- 172.21.15.40:0/757056686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4941517e0 con 0x7fc4940cba10 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc492d9d640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc49406fea0 0x7fc4940c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.617+0000 7fc48bfff640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940cba10 0x7fc4941f2570 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c4d0 tx=0x7fc47c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.618+0000 7fc48b7fe640 1 -- 172.21.15.40:0/757056686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c018070 con 0x7fc4940cba10 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.618+0000 7fc48b7fe640 1 -- 172.21.15.40:0/757056686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47c0074e0 con 0x7fc4940cba10 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.618+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4940ca770 con 0x7fc4940cba10 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.618+0000 7fc48b7fe640 1 -- 172.21.15.40:0/757056686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c004a20 con 0x7fc4940cba10 2024-04-26T21:47:40.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.618+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4941527b0 con 0x7fc4940cba10 2024-04-26T21:47:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.619+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4940694f0 con 0x7fc4940cba10 2024-04-26T21:47:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.619+0000 7fc48b7fe640 1 -- 172.21.15.40:0/757056686 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc47c01b070 con 0x7fc4940cba10 2024-04-26T21:47:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.620+0000 7fc48b7fe640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.620+0000 7fc48b7fe640 1 -- 172.21.15.40:0/757056686 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fc47c08af60 con 0x7fc4940cba10 2024-04-26T21:47:40.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.620+0000 7fc492d9d640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.621+0000 7fc492d9d640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc47406c8f0 0x7fc47406edb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc484034000 tx=0x7fc48402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.624+0000 7fc48b7fe640 1 -- 172.21.15.40:0/757056686 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc47c059240 con 0x7fc4940cba10 2024-04-26T21:47:40.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.807+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc49406de80 con 0x7fc4940cba10 2024-04-26T21:47:40.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.807+0000 7fc48b7fe640 1 -- 172.21.15.40:0/757056686 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) ==== 93+0+10 (secure 0 0 0) 0x7fc47c058b40 con 0x7fc4940cba10 2024-04-26T21:47:40.807 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:47:40.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc47406c8f0 msgr2=0x7fc47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc47406c8f0 0x7fc47406edb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc484034000 tx=0x7fc48402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:40.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940cba10 msgr2=0x7fc4941f2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940cba10 0x7fc4941f2570 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c4d0 tx=0x7fc47c00c990 comp rx=0 tx=0).stop 2024-04-26T21:47:40.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 shutdown_connections 2024-04-26T21:47:40.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc494156460 0x7fc4940c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940cba10 0x7fc4941f2570 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 --2- 172.21.15.40:0/757056686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc49406fea0 0x7fc4940c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 >> 172.21.15.40:0/757056686 conn(0x7fc4940d0be0 msgr2=0x7fc4940cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:40.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 shutdown_connections 2024-04-26T21:47:40.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.809+0000 7fc499212640 1 -- 172.21.15.40:0/757056686 wait complete. 2024-04-26T21:47:40.820 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:47:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.924+0000 7fc3d5ada640 1 Processor -- start 2024-04-26T21:47:40.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.924+0000 7fc3d5ada640 1 -- start start 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.924+0000 7fc3d5ada640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d0070350 0x7fc3d0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.924+0000 7fc3d5ada640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3d0071090 0x7fc3d015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.924+0000 7fc3d5ada640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3d00d2390 con 0x7fc3d00cfde0 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.924+0000 7fc3d5ada640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc3d0071500 con 0x7fc3d0071090 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.924+0000 7fc3d5ada640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d00afab0 con 0x7fc3d0070350 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3cffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3d0071090 0x7fc3d015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3cf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc3d00cfde0 0x7fc3d00d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42314/0 (socket says 172.21.15.40:42314) 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3cf7fe640 1 -- 172.21.15.40:0/2560654068 learned_addr learned my addr 172.21.15.40:0/2560654068 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3ceffd640 1 --2- 172.21.15.40:0/2560654068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d0070350 0x7fc3d0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3cffff640 1 -- 172.21.15.40:0/2560654068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d0070350 msgr2=0x7fc3d0070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3cffff640 1 --2- 172.21.15.40:0/2560654068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d0070350 0x7fc3d0070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3cffff640 1 -- 172.21.15.40:0/2560654068 >> v1:172.21.15.40:6789/0 conn(0x7fc3d00cfde0 legacy=0x7fc3d00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3cffff640 1 -- 172.21.15.40:0/2560654068 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d006d3f0 con 0x7fc3d0071090 2024-04-26T21:47:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3ce7fc640 1 -- 172.21.15.40:0/2560654068 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2381219819 0 0) 0x7fc3d00d2390 con 0x7fc3d00cfde0 2024-04-26T21:47:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.925+0000 7fc3ceffd640 1 --2- 172.21.15.40:0/2560654068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d0070350 0x7fc3d0070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3cffff640 1 --2- 172.21.15.40:0/2560654068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d015e9f0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009d90 tx=0x7fc3c802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d53373d0f54dd33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3ce7fc640 1 -- 172.21.15.40:0/2560654068 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8031600 con 0x7fc3d0071090 2024-04-26T21:47:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3ce7fc640 1 -- 172.21.15.40:0/2560654068 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c80317a0 con 0x7fc3d0071090 2024-04-26T21:47:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3ce7fc640 1 -- 172.21.15.40:0/2560654068 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8031b70 con 0x7fc3d0071090 2024-04-26T21:47:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3d5ada640 1 -- 172.21.15.40:0/2560654068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 msgr2=0x7fc3d015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/2560654068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d015e9f0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009d90 tx=0x7fc3c802a5d0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3d5ada640 1 -- 172.21.15.40:0/2560654068 shutdown_connections 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/2560654068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d015e9f0 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/2560654068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d0070350 0x7fc3d0070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3d5ada640 1 -- 172.21.15.40:0/2560654068 >> 172.21.15.40:0/2560654068 conn(0x7fc3d00d2d80 msgr2=0x7fc3d00d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3d5ada640 1 -- 172.21.15.40:0/2560654068 shutdown_connections 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.926+0000 7fc3d5ada640 1 -- 172.21.15.40:0/2560654068 wait complete. 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.928+0000 7fc3d5ada640 1 Processor -- start 2024-04-26T21:47:40.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.928+0000 7fc3d5ada640 1 -- start start 2024-04-26T21:47:40.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.928+0000 7fc3d5ada640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3d0070350 0x7fc3d0162750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.928+0000 7fc3d5ada640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d0158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.928+0000 7fc3d5ada640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d00cfde0 0x7fc3d01f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.928+0000 7fc3d5ada640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d006d9b0 con 0x7fc3d0071090 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.928+0000 7fc3d5ada640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d006dd20 con 0x7fc3d0070350 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.928+0000 7fc3d5ada640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d015ef30 con 0x7fc3d00cfde0 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d0158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d0158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42682/0 (socket says 172.21.15.40:42682) 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 -- 172.21.15.40:0/3848130219 learned_addr learned my addr 172.21.15.40:0/3848130219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3cffff640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d00cfde0 0x7fc3d01f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3cf7fe640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3d0070350 0x7fc3d0162750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 -- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d00cfde0 msgr2=0x7fc3d01f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d00cfde0 0x7fc3d01f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 -- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3d0070350 msgr2=0x7fc3d0162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3d0070350 0x7fc3d0162750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 -- 172.21.15.40:0/3848130219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d0158cb0 con 0x7fc3d0071090 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3ceffd640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d0158590 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400e7e0 tx=0x7fc3c400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3b7fff640 1 -- 172.21.15.40:0/3848130219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400f040 con 0x7fc3d0071090 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3b7fff640 1 -- 172.21.15.40:0/3848130219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c4004030 con 0x7fc3d0071090 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d01f54a0 con 0x7fc3d0071090 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3b7fff640 1 -- 172.21.15.40:0/3848130219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4004400 con 0x7fc3d0071090 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.929+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d01f5a60 con 0x7fc3d0071090 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.930+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d00694f0 con 0x7fc3d0071090 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.931+0000 7fc3b7fff640 1 -- 172.21.15.40:0/3848130219 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3c400c6d0 con 0x7fc3d0071090 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.931+0000 7fc3b7fff640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3bc06c8f0 0x7fc3bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:40.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.931+0000 7fc3b7fff640 1 -- 172.21.15.40:0/3848130219 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fc3c408bac0 con 0x7fc3d0071090 2024-04-26T21:47:40.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.931+0000 7fc3cf7fe640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3bc06c8f0 0x7fc3bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:40.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.932+0000 7fc3cf7fe640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3bc06c8f0 0x7fc3bc06edb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0000f60 tx=0x7fc3c0007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:40.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:40.936+0000 7fc3b7fff640 1 -- 172.21.15.40:0/3848130219 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3c4017020 con 0x7fc3d0071090 2024-04-26T21:47:41.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.121+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3d00cf5e0 con 0x7fc3d0071090 2024-04-26T21:47:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.121+0000 7fc3b7fff640 1 -- 172.21.15.40:0/3848130219 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v146) ==== 104+0+11 (secure 0 0 0) 0x7fc3c4059d10 con 0x7fc3d0071090 2024-04-26T21:47:41.121 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:47:41.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3bc06c8f0 msgr2=0x7fc3bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3bc06c8f0 0x7fc3bc06edb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0000f60 tx=0x7fc3c0007c00 comp rx=0 tx=0).stop 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 msgr2=0x7fc3d0158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d0158590 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400e7e0 tx=0x7fc3c400eca0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 shutdown_connections 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc3bc06c8f0 0x7fc3bc06edb0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3d00cfde0 0x7fc3d01f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d0071090 0x7fc3d0158590 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 --2- 172.21.15.40:0/3848130219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3d0070350 0x7fc3d0162750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 >> 172.21.15.40:0/3848130219 conn(0x7fc3d00d2d80 msgr2=0x7fc3d00b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 shutdown_connections 2024-04-26T21:47:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.123+0000 7fc3d5ada640 1 -- 172.21.15.40:0/3848130219 wait complete. 2024-04-26T21:47:41.134 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:47:41.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.237+0000 7fba2810a640 1 Processor -- start 2024-04-26T21:47:41.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.237+0000 7fba2810a640 1 -- start start 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.238+0000 7fba2810a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fba200b40d0 0x7fba201625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.238+0000 7fba2810a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba200d0a10 con 0x7fba20162b30 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.238+0000 7fba2810a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba200d0cf0 con 0x7fba200d1a00 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.238+0000 7fba2810a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fba200d0fd0 con 0x7fba200b40d0 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.238+0000 7fba266a7640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fba20162b30 0x7fba200b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42324/0 (socket says 172.21.15.40:42324) 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.238+0000 7fba266a7640 1 -- 172.21.15.40:0/2911477833 learned_addr learned my addr 172.21.15.40:0/2911477833 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.238+0000 7fba256a5640 1 --2- 172.21.15.40:0/2911477833 >> v2:172.21.15.184:3300/0 conn(0x7fba200b40d0 0x7fba201625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279102133 0 0) 0x7fba200d0a10 con 0x7fba20162b30 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba0c003520 con 0x7fba20162b30 2024-04-26T21:47:41.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215779541 0 0) 0x7fba200d0cf0 con 0x7fba200d1a00 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba200d0a10 con 0x7fba200d1a00 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 936044676 0 0) 0x7fba0c003520 con 0x7fba20162b30 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 >> v2:172.21.15.184:3300/0 conn(0x7fba200b40d0 msgr2=0x7fba201625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 --2- 172.21.15.40:0/2911477833 >> v2:172.21.15.184:3300/0 conn(0x7fba200b40d0 0x7fba201625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 >> v1:172.21.15.184:6789/0 conn(0x7fba200d1a00 legacy=0x7fba200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba20162f80 con 0x7fba20162b30 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fba180019f0 con 0x7fba20162b30 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba256a5640 1 --2- 172.21.15.40:0/2911477833 >> v2:172.21.15.184:3300/0 conn(0x7fba200b40d0 0x7fba201625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 >> v1:172.21.15.40:6789/0 conn(0x7fba20162b30 legacy=0x7fba200b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba0c003b40 0x7fba0c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c004800 0x7fba0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c0051c0 0x7fba0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fba200d0fd0 con 0x7fba0c0051c0 2024-04-26T21:47:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c00c0f0 con 0x7fba0c004800 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fba0c00c2f0 con 0x7fba0c003b40 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba266a7640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c0051c0 0x7fba0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba256a5640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c004800 0x7fba0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba25ea6640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba0c003b40 0x7fba0c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba266a7640 1 -- 172.21.15.40:0/2911477833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba0c003b40 msgr2=0x7fba0c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba266a7640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba0c003b40 0x7fba0c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba266a7640 1 -- 172.21.15.40:0/2911477833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c004800 msgr2=0x7fba0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba266a7640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c004800 0x7fba0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.239+0000 7fba266a7640 1 -- 172.21.15.40:0/2911477833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c003520 con 0x7fba0c0051c0 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.240+0000 7fba266a7640 1 -- 172.21.15.40:0/2911477833 reap_dead start 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.240+0000 7fba266a7640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c0051c0 0x7fba0c00b9c0 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fba1800a7f0 tx=0x7fba0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353ae28484f001b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.240+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1800a410 con 0x7fba0c0051c0 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.240+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1802f070 con 0x7fba0c0051c0 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.240+0000 7fba24ea4640 1 -- 172.21.15.40:0/2911477833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1802bc70 con 0x7fba0c0051c0 2024-04-26T21:47:41.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.241+0000 7fba2810a640 1 -- 172.21.15.40:0/2911477833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c0051c0 msgr2=0x7fba0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.241+0000 7fba2810a640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c0051c0 0x7fba0c00b9c0 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fba1800a7f0 tx=0x7fba0c005830 comp rx=0 tx=0).stop 2024-04-26T21:47:41.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.241+0000 7fba2810a640 1 -- 172.21.15.40:0/2911477833 shutdown_connections 2024-04-26T21:47:41.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.241+0000 7fba2810a640 1 --2- 172.21.15.40:0/2911477833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c0051c0 0x7fba0c00b9c0 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.241+0000 7fba2810a640 1 -- 172.21.15.40:0/2911477833 >> 172.21.15.40:0/2911477833 conn(0x7fba200d4fd0 msgr2=0x7fba200d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.241+0000 7fba2810a640 1 -- 172.21.15.40:0/2911477833 shutdown_connections 2024-04-26T21:47:41.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.241+0000 7fba2810a640 1 -- 172.21.15.40:0/2911477833 wait complete. 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.242+0000 7fba2810a640 1 Processor -- start 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.242+0000 7fba2810a640 1 -- start start 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba2810a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c0051c0 0x7fba200c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba2810a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba2015e350 0x7fba20155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba2810a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba201555d0 0x7fba200cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba2810a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fba20162770 con 0x7fba201555d0 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba2810a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fba201625f0 con 0x7fba0c0051c0 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba2810a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fba201628f0 con 0x7fba2015e350 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba201555d0 0x7fba200cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba201555d0 0x7fba200cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42704/0 (socket says 172.21.15.40:42704) 2024-04-26T21:47:41.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 -- 172.21.15.40:0/2176988702 learned_addr learned my addr 172.21.15.40:0/2176988702 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba25ea6640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c0051c0 0x7fba200c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba256a5640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba2015e350 0x7fba20155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 -- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba2015e350 msgr2=0x7fba20155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba2015e350 0x7fba20155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 -- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c0051c0 msgr2=0x7fba200c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c0051c0 0x7fba200c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 -- 172.21.15.40:0/2176988702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba200c8f40 con 0x7fba201555d0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.243+0000 7fba266a7640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba201555d0 0x7fba200cd640 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7fba18001c70 tx=0x7fba18001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.244+0000 7fba167fc640 1 -- 172.21.15.40:0/2176988702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18004d70 con 0x7fba201555d0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.244+0000 7fba167fc640 1 -- 172.21.15.40:0/2176988702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1802ba10 con 0x7fba201555d0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.244+0000 7fba167fc640 1 -- 172.21.15.40:0/2176988702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1802be00 con 0x7fba201555d0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.244+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba200c9220 con 0x7fba201555d0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.244+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba200c97e0 con 0x7fba201555d0 2024-04-26T21:47:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.245+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba200694f0 con 0x7fba201555d0 2024-04-26T21:47:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.245+0000 7fba167fc640 1 -- 172.21.15.40:0/2176988702 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba18034470 con 0x7fba201555d0 2024-04-26T21:47:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.246+0000 7fba167fc640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.246+0000 7fba167fc640 1 -- 172.21.15.40:0/2176988702 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fba180abc20 con 0x7fba201555d0 2024-04-26T21:47:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.246+0000 7fba25ea6640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.246+0000 7fba25ea6640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba0406c8f0 0x7fba0406edb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fba10000c00 tx=0x7fba10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.250+0000 7fba167fc640 1 -- 172.21.15.40:0/2176988702 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba18032e50 con 0x7fba201555d0 2024-04-26T21:47:41.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.433+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba200b2a80 con 0x7fba201555d0 2024-04-26T21:47:41.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.434+0000 7fba167fc640 1 -- 172.21.15.40:0/2176988702 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v146) ==== 100+0+11 (secure 0 0 0) 0x7fba18079f00 con 0x7fba201555d0 2024-04-26T21:47:41.434 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba0406c8f0 msgr2=0x7fba0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba0406c8f0 0x7fba0406edb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fba10000c00 tx=0x7fba10009040 comp rx=0 tx=0).stop 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba201555d0 msgr2=0x7fba200cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba201555d0 0x7fba200cd640 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7fba18001c70 tx=0x7fba18001d40 comp rx=0 tx=0).stop 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 shutdown_connections 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba201555d0 0x7fba200cd640 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba2015e350 0x7fba20155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 --2- 172.21.15.40:0/2176988702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba0c0051c0 0x7fba200c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 >> 172.21.15.40:0/2176988702 conn(0x7fba200d4fd0 msgr2=0x7fba200b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 shutdown_connections 2024-04-26T21:47:41.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.436+0000 7fba2810a640 1 -- 172.21.15.40:0/2176988702 wait complete. 2024-04-26T21:47:41.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:47:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.549+0000 7f524c293640 1 Processor -- start 2024-04-26T21:47:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.549+0000 7f524c293640 1 -- start start 2024-04-26T21:47:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524c293640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f524406f790 0x7f524406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524c293640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5244156840 con 0x7f5244072c50 2024-04-26T21:47:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524c293640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52440cbf20 con 0x7f5244156460 2024-04-26T21:47:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524c293640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f52440cc200 con 0x7f524406f790 2024-04-26T21:47:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524982e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f524406f790 0x7f524406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524a830640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5244072c50 0x7f5244073060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42336/0 (socket says 172.21.15.40:42336) 2024-04-26T21:47:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524a830640 1 -- 172.21.15.40:0/2850873805 learned_addr learned my addr 172.21.15.40:0/2850873805 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524982e640 1 -- 172.21.15.40:0/2850873805 >> v1:172.21.15.184:6789/0 conn(0x7f5244156460 legacy=0x7f52440cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524982e640 1 -- 172.21.15.40:0/2850873805 >> v1:172.21.15.40:6789/0 conn(0x7f5244072c50 legacy=0x7f5244073060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524902d640 1 -- 172.21.15.40:0/2850873805 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1964483170 0 0) 0x7f5244156840 con 0x7f5244072c50 2024-04-26T21:47:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.550+0000 7f524982e640 1 -- 172.21.15.40:0/2850873805 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52440cc4e0 con 0x7f524406f790 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.551+0000 7f524982e640 1 --2- 172.21.15.40:0/2850873805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524406fc40 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f52340097a0 tx=0x7f523402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afeb0bb821162e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.551+0000 7f524902d640 1 -- 172.21.15.40:0/2850873805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234037070 con 0x7f524406f790 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.551+0000 7f524902d640 1 -- 172.21.15.40:0/2850873805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5234032440 con 0x7f524406f790 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.551+0000 7f524902d640 1 -- 172.21.15.40:0/2850873805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234032810 con 0x7f524406f790 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.552+0000 7f524c293640 1 -- 172.21.15.40:0/2850873805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 msgr2=0x7f524406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.552+0000 7f524c293640 1 --2- 172.21.15.40:0/2850873805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524406fc40 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f52340097a0 tx=0x7f523402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.552+0000 7f524c293640 1 -- 172.21.15.40:0/2850873805 shutdown_connections 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.552+0000 7f524c293640 1 --2- 172.21.15.40:0/2850873805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524406fc40 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.552+0000 7f524c293640 1 -- 172.21.15.40:0/2850873805 >> 172.21.15.40:0/2850873805 conn(0x7f52440d0be0 msgr2=0x7f52440d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.552+0000 7f524c293640 1 -- 172.21.15.40:0/2850873805 shutdown_connections 2024-04-26T21:47:41.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.552+0000 7f524c293640 1 -- 172.21.15.40:0/2850873805 wait complete. 2024-04-26T21:47:41.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.553+0000 7f524c293640 1 Processor -- start 2024-04-26T21:47:41.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.553+0000 7f524c293640 1 -- start start 2024-04-26T21:47:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524c293640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524414f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524c293640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5244072c50 0x7f52440c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524c293640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5244156460 0x7f52440c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524c293640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52440cbbe0 con 0x7f524406f790 2024-04-26T21:47:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524c293640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5244072700 con 0x7f5244072c50 2024-04-26T21:47:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524c293640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f524406e260 con 0x7f5244156460 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524414f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524414f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42726/0 (socket says 172.21.15.40:42726) 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 -- 172.21.15.40:0/1353613696 learned_addr learned my addr 172.21.15.40:0/1353613696 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a830640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5244156460 0x7f52440c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524982e640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5244072c50 0x7f52440c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 -- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5244156460 msgr2=0x7f52440c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5244156460 0x7f52440c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 -- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5244072c50 msgr2=0x7f52440c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5244072c50 0x7f52440c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 -- 172.21.15.40:0/1353613696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52440c8970 con 0x7f524406f790 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524982e640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5244072c50 0x7f52440c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a830640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5244156460 0x7f52440c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.554+0000 7f524a02f640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524414f940 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f524000b3f0 tx=0x7f524000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.555+0000 7f523affd640 1 -- 172.21.15.40:0/1353613696 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240015020 con 0x7f524406f790 2024-04-26T21:47:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.555+0000 7f523affd640 1 -- 172.21.15.40:0/1353613696 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5240011830 con 0x7f524406f790 2024-04-26T21:47:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.555+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52440c8c50 con 0x7f524406f790 2024-04-26T21:47:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.555+0000 7f523affd640 1 -- 172.21.15.40:0/1353613696 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240011c20 con 0x7f524406f790 2024-04-26T21:47:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.555+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52440c9210 con 0x7f524406f790 2024-04-26T21:47:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.556+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52440694f0 con 0x7f524406f790 2024-04-26T21:47:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.556+0000 7f523affd640 1 -- 172.21.15.40:0/1353613696 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5240018020 con 0x7f524406f790 2024-04-26T21:47:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.556+0000 7f523affd640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f522806c8f0 0x7f522806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.557+0000 7f523affd640 1 -- 172.21.15.40:0/1353613696 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f524008ba10 con 0x7f524406f790 2024-04-26T21:47:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.557+0000 7f524982e640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f522806c8f0 0x7f522806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.557+0000 7f524982e640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f522806c8f0 0x7f522806edb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5234034000 tx=0x7f523402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:41.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.561+0000 7f523affd640 1 -- 172.21.15.40:0/1353613696 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f524001f030 con 0x7f524406f790 2024-04-26T21:47:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.760+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f524406de80 con 0x7f524406f790 2024-04-26T21:47:41.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.760+0000 7f523affd640 1 -- 172.21.15.40:0/1353613696 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v146) ==== 80+0+122 (secure 0 0 0) 0x7f52400172c0 con 0x7f524406f790 2024-04-26T21:47:41.760 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6835/3441996159 2024-04-27T21:47:30.391245+0000 2024-04-26T21:47:41.760 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6834/3441996159 2024-04-27T21:47:30.391245+0000 2024-04-26T21:47:41.760 INFO:teuthology.orchestra.run.smithi040.stderr:listed 2 entries 2024-04-26T21:47:41.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f522806c8f0 msgr2=0x7f522806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f522806c8f0 0x7f522806edb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5234034000 tx=0x7f523402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 msgr2=0x7f524414f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524414f940 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f524000b3f0 tx=0x7f524000b8b0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 shutdown_connections 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f522806c8f0 0x7f522806edb0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5244156460 0x7f52440c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5244072c50 0x7f52440c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 --2- 172.21.15.40:0/1353613696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f524406f790 0x7f524414f940 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 >> 172.21.15.40:0/1353613696 conn(0x7f52440d0be0 msgr2=0x7f524406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 shutdown_connections 2024-04-26T21:47:41.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:41.762+0000 7f524c293640 1 -- 172.21.15.40:0/1353613696 wait complete. 2024-04-26T21:47:41.773 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:47:41.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:41.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:47:41.800 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:47:41.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:41.800 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:47:41.856 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:41.856 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:47:41.856 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:41.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:41.875 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:41.875 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:41.875 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:41.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:41.912 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:47:41.912 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:47:41.912 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:41.933 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:41.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.053+0000 7f70b8789640 1 Processor -- start 2024-04-26T21:47:42.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.053+0000 7f70b8789640 1 -- start start 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b8789640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70b00b2340 0x7f70b00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b8789640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f70b00cd930 0x7f70b00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b8789640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b00d10d0 con 0x7f70b0162810 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b8789640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f70b00d0750 con 0x7f70b00b2340 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b8789640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f70b00d16a0 con 0x7f70b00cd930 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b5d24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70b00b2340 0x7f70b00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b6525640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f70b0162810 0x7f70b0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42340/0 (socket says 172.21.15.40:42340) 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b6525640 1 -- 172.21.15.40:0/4114799100 learned_addr learned my addr 172.21.15.40:0/4114799100 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b5523640 1 -- 172.21.15.40:0/4114799100 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4257646238 0 0) 0x7f70b00d10d0 con 0x7f70b0162810 2024-04-26T21:47:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b5523640 1 -- 172.21.15.40:0/4114799100 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f709c003520 con 0x7f70b0162810 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b6d26640 1 --2- 172.21.15.40:0/4114799100 >> v2:172.21.15.184:3300/0 conn(0x7f70b00cd930 0x7f70b00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b5d24640 1 -- 172.21.15.40:0/4114799100 >> v2:172.21.15.184:3300/0 conn(0x7f70b00cd930 msgr2=0x7f70b00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b5d24640 1 --2- 172.21.15.40:0/4114799100 >> v2:172.21.15.184:3300/0 conn(0x7f70b00cd930 0x7f70b00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b5d24640 1 -- 172.21.15.40:0/4114799100 >> v1:172.21.15.40:6789/0 conn(0x7f70b0162810 legacy=0x7f70b0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b5d24640 1 -- 172.21.15.40:0/4114799100 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b00d1370 con 0x7f70b00b2340 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.054+0000 7f70b6d26640 1 --2- 172.21.15.40:0/4114799100 >> v2:172.21.15.184:3300/0 conn(0x7f70b00cd930 0x7f70b00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.055+0000 7f70b5d24640 1 --2- 172.21.15.40:0/4114799100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00cd3f0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f70b00b2280 tx=0x7f70a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485fced07287ca9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.055+0000 7f70b5523640 1 -- 172.21.15.40:0/4114799100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0037070 con 0x7f70b00b2340 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.055+0000 7f70b5523640 1 -- 172.21.15.40:0/4114799100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a0032440 con 0x7f70b00b2340 2024-04-26T21:47:42.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.055+0000 7f70b5523640 1 -- 172.21.15.40:0/4114799100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0032810 con 0x7f70b00b2340 2024-04-26T21:47:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.056+0000 7f70b8789640 1 -- 172.21.15.40:0/4114799100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 msgr2=0x7f70b00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.056+0000 7f70b8789640 1 --2- 172.21.15.40:0/4114799100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00cd3f0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f70b00b2280 tx=0x7f70a002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.056+0000 7f70b8789640 1 -- 172.21.15.40:0/4114799100 shutdown_connections 2024-04-26T21:47:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.056+0000 7f70b8789640 1 --2- 172.21.15.40:0/4114799100 >> v2:172.21.15.184:3300/0 conn(0x7f70b00cd930 0x7f70b00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.056+0000 7f70b8789640 1 --2- 172.21.15.40:0/4114799100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00cd3f0 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.056+0000 7f70b8789640 1 -- 172.21.15.40:0/4114799100 >> 172.21.15.40:0/4114799100 conn(0x7f70b00d2da0 msgr2=0x7f70b00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.056+0000 7f70b8789640 1 -- 172.21.15.40:0/4114799100 shutdown_connections 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.056+0000 7f70b8789640 1 -- 172.21.15.40:0/4114799100 wait complete. 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.057+0000 7f70b8789640 1 Processor -- start 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.057+0000 7f70b8789640 1 -- start start 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b8789640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b8789640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b00cd930 0x7f70b00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b8789640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b0162810 0x7f70b015be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b8789640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70b00d2070 con 0x7f70b00b2340 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b8789640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70b00d0470 con 0x7f70b00cd930 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b8789640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70b00d21f0 con 0x7f70b0162810 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42748/0 (socket says 172.21.15.40:42748) 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 -- 172.21.15.40:0/1457046555 learned_addr learned my addr 172.21.15.40:0/1457046555 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6d26640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b0162810 0x7f70b015be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b5d24640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b00cd930 0x7f70b00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 -- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b0162810 msgr2=0x7f70b015be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b0162810 0x7f70b015be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 -- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b00cd930 msgr2=0x7f70b00c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b00cd930 0x7f70b00c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 -- 172.21.15.40:0/1457046555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b00c8e70 con 0x7f70b00b2340 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b5d24640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b00cd930 0x7f70b00c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6d26640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b0162810 0x7f70b015be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b6525640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00c93c0 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00c3c0 tx=0x7f70ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70a6ffd640 1 -- 172.21.15.40:0/1457046555 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac017070 con 0x7f70b00b2340 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70a6ffd640 1 -- 172.21.15.40:0/1457046555 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70ac005ce0 con 0x7f70b00b2340 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b00caad0 con 0x7f70b00b2340 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70a6ffd640 1 -- 172.21.15.40:0/1457046555 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0072f0 con 0x7f70b00b2340 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.058+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b00cb090 con 0x7f70b00b2340 2024-04-26T21:47:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.059+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b00694f0 con 0x7f70b00b2340 2024-04-26T21:47:42.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.060+0000 7f70a6ffd640 1 -- 172.21.15.40:0/1457046555 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70ac01a070 con 0x7f70b00b2340 2024-04-26T21:47:42.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.060+0000 7f70a6ffd640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f709406c920 0x7f709406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.060+0000 7f70a6ffd640 1 -- 172.21.15.40:0/1457046555 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f70ac08b060 con 0x7f70b00b2340 2024-04-26T21:47:42.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.061+0000 7f70b5d24640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f709406c920 0x7f709406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.061+0000 7f70b5d24640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f709406c920 0x7f709406ede0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f70a0034000 tx=0x7f70a002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.065+0000 7f70a6ffd640 1 -- 172.21.15.40:0/1457046555 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70ac0592b0 con 0x7f70b00b2340 2024-04-26T21:47:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.287+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70b015d020 con 0x7f70b00b2340 2024-04-26T21:47:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.287+0000 7f70a6ffd640 1 -- 172.21.15.40:0/1457046555 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7f70ac058bb0 con 0x7f70b00b2340 2024-04-26T21:47:42.287 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:42.287 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f709406c920 msgr2=0x7f709406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f709406c920 0x7f709406ede0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f70a0034000 tx=0x7f70a002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 msgr2=0x7f70b00c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00c93c0 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00c3c0 tx=0x7f70ac00c880 comp rx=0 tx=0).stop 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 shutdown_connections 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f709406c920 0x7f709406ede0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b0162810 0x7f70b015be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b00cd930 0x7f70b00c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 --2- 172.21.15.40:0/1457046555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b00b2340 0x7f70b00c93c0 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.289+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 >> 172.21.15.40:0/1457046555 conn(0x7f70b00d2da0 msgr2=0x7f70b00cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.290+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 shutdown_connections 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.290+0000 7f70b8789640 1 -- 172.21.15.40:0/1457046555 wait complete. 2024-04-26T21:47:42.301 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:42.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.401+0000 7f42d656a640 1 Processor -- start 2024-04-26T21:47:42.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.402+0000 7f42d656a640 1 -- start start 2024-04-26T21:47:42.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.402+0000 7f42d656a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f42d0162880 0x7f42d00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.402+0000 7f42d656a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00b5f20 0x7f42d00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.402+0000 7f42d656a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d00b7e50 con 0x7f42d00d39f0 2024-04-26T21:47:42.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.402+0000 7f42d656a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f42d00d0580 con 0x7f42d0162880 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.402+0000 7f42d656a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f42d00d1800 con 0x7f42d00b5f20 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.402+0000 7f42cffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f42d0162880 0x7f42d00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.402+0000 7f42cffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f42d0162880 0x7f42d00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42764/0 (socket says 172.21.15.40:42764) 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.403+0000 7f42cffff640 1 -- 172.21.15.40:0/3106071052 learned_addr learned my addr 172.21.15.40:0/3106071052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.403+0000 7f42cf7fe640 1 --2- 172.21.15.40:0/3106071052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00b5f20 0x7f42d00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.403+0000 7f42cffff640 1 -- 172.21.15.40:0/3106071052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00b5f20 msgr2=0x7f42d00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.403+0000 7f42cffff640 1 --2- 172.21.15.40:0/3106071052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00b5f20 0x7f42d00d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.403+0000 7f42cffff640 1 -- 172.21.15.40:0/3106071052 >> v1:172.21.15.184:6789/0 conn(0x7f42d00d39f0 legacy=0x7f42d00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.403+0000 7f42cffff640 1 -- 172.21.15.40:0/3106071052 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d00d07d0 con 0x7f42d0162880 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42cffff640 1 --2- 172.21.15.40:0/3106071052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d0162880 0x7f42d00d2cb0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009870 tx=0x7f42c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcbd27b95a42043 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42ceffd640 1 -- 172.21.15.40:0/3106071052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0037070 con 0x7f42d0162880 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42ceffd640 1 -- 172.21.15.40:0/3106071052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c0032440 con 0x7f42d0162880 2024-04-26T21:47:42.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42ceffd640 1 -- 172.21.15.40:0/3106071052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0032810 con 0x7f42d0162880 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42d656a640 1 -- 172.21.15.40:0/3106071052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d0162880 msgr2=0x7f42d00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42d656a640 1 --2- 172.21.15.40:0/3106071052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d0162880 0x7f42d00d2cb0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009870 tx=0x7f42c002a270 comp rx=0 tx=0).stop 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42d656a640 1 -- 172.21.15.40:0/3106071052 shutdown_connections 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42d656a640 1 --2- 172.21.15.40:0/3106071052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00b5f20 0x7f42d00d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42d656a640 1 --2- 172.21.15.40:0/3106071052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d0162880 0x7f42d00d2cb0 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42d656a640 1 -- 172.21.15.40:0/3106071052 >> 172.21.15.40:0/3106071052 conn(0x7f42d00d4f60 msgr2=0x7f42d00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42d656a640 1 -- 172.21.15.40:0/3106071052 shutdown_connections 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.404+0000 7f42d656a640 1 -- 172.21.15.40:0/3106071052 wait complete. 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.405+0000 7f42d656a640 1 Processor -- start 2024-04-26T21:47:42.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.406+0000 7f42d656a640 1 -- start start 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.406+0000 7f42d656a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d00b5f20 0x7f42d0159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.406+0000 7f42d656a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00d39f0 0x7f42d00b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.406+0000 7f42d656a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42d0162880 0x7f42d00ba230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.406+0000 7f42d656a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0162c60 con 0x7f42d00b5f20 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.406+0000 7f42d656a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f42d00d0d90 con 0x7f42d0162880 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.406+0000 7f42d656a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f42d0162de0 con 0x7f42d00d39f0 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.406+0000 7f42cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d00b5f20 0x7f42d0159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d00b5f20 0x7f42d0159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42772/0 (socket says 172.21.15.40:42772) 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cffff640 1 -- 172.21.15.40:0/119559890 learned_addr learned my addr 172.21.15.40:0/119559890 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cf7fe640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00d39f0 0x7f42d00b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42d4b07640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42d0162880 0x7f42d00ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cffff640 1 -- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00d39f0 msgr2=0x7f42d00b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cffff640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00d39f0 0x7f42d00b9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cffff640 1 -- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42d0162880 msgr2=0x7f42d00ba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cffff640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42d0162880 0x7f42d00ba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cffff640 1 -- 172.21.15.40:0/119559890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d00ce140 con 0x7f42d00b5f20 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42d4b07640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42d0162880 0x7f42d00ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42cffff640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d00b5f20 0x7f42d0159390 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f42c0000c00 tx=0x7f42c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42ccff9640 1 -- 172.21.15.40:0/119559890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0041020 con 0x7f42d00b5f20 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42ccff9640 1 -- 172.21.15.40:0/119559890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c00306c0 con 0x7f42d00b5f20 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d00ccbf0 con 0x7f42d00b5f20 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42ccff9640 1 -- 172.21.15.40:0/119559890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0037040 con 0x7f42d00b5f20 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.407+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d015a0c0 con 0x7f42d00b5f20 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.408+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d00694f0 con 0x7f42d00b5f20 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.409+0000 7f42ccff9640 1 -- 172.21.15.40:0/119559890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42c0044020 con 0x7f42d00b5f20 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.409+0000 7f42ccff9640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42bc06c8f0 0x7f42bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.409+0000 7f42ccff9640 1 -- 172.21.15.40:0/119559890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f42c00ac4d0 con 0x7f42d00b5f20 2024-04-26T21:47:42.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.409+0000 7f42cf7fe640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42bc06c8f0 0x7f42bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.410+0000 7f42cf7fe640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42bc06c8f0 0x7f42bc06edb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f42c40079a0 tx=0x7f42c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.413+0000 7f42ccff9640 1 -- 172.21.15.40:0/119559890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42c004b030 con 0x7f42d00b5f20 2024-04-26T21:47:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.603+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42d015f830 con 0x7f42d00b5f20 2024-04-26T21:47:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.604+0000 7f42ccff9640 1 -- 172.21.15.40:0/119559890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16977 (secure 0 0 0) 0x7f42c007a720 con 0x7f42d00b5f20 2024-04-26T21:47:42.605 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:42.605 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":146,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:30.391272+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:18.726844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:19.403532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3441996159":"2024-04-27T21:47:30.391245+0000","172.21.15.40:6834/3441996159":"2024-04-27T21:47:30.391245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42bc06c8f0 msgr2=0x7f42bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42bc06c8f0 0x7f42bc06edb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f42c40079a0 tx=0x7f42c4008040 comp rx=0 tx=0).stop 2024-04-26T21:47:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d00b5f20 msgr2=0x7f42d0159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d00b5f20 0x7f42d0159390 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f42c0000c00 tx=0x7f42c00046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 shutdown_connections 2024-04-26T21:47:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f42bc06c8f0 0x7f42bc06edb0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42d0162880 0x7f42d00ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42d00d39f0 0x7f42d00b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 --2- 172.21.15.40:0/119559890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42d00b5f20 0x7f42d0159390 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 >> 172.21.15.40:0/119559890 conn(0x7f42d00d4f60 msgr2=0x7f42d00b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 shutdown_connections 2024-04-26T21:47:42.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.607+0000 7f42d656a640 1 -- 172.21.15.40:0/119559890 wait complete. 2024-04-26T21:47:42.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:34:58.012836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:47:18.726844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:47:19.403532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:47:42.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:47:42.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.722+0000 7fbcd67da640 1 Processor -- start 2024-04-26T21:47:42.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.722+0000 7fbcd67da640 1 -- start start 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.723+0000 7fbcd67da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00b1fd0 0x7fbcd00d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.723+0000 7fbcd67da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcd00cdb70 0x7fbcd00ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.723+0000 7fbcd67da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd006e580 con 0x7fbcd015f770 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.723+0000 7fbcd67da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd006f3e0 con 0x7fbcd00cdb70 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.723+0000 7fbcd67da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd00cd6e0 con 0x7fbcd00b1fd0 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.723+0000 7fbccffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbcd015f770 0x7fbcd00d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42352/0 (socket says 172.21.15.40:42352) 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.723+0000 7fbccffff640 1 -- 172.21.15.40:0/2763976425 learned_addr learned my addr 172.21.15.40:0/2763976425 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00b1fd0 0x7fbcd00d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcd4d77640 1 --2- 172.21.15.40:0/2763976425 >> v2:172.21.15.184:3300/0 conn(0x7fbcd00cdb70 0x7fbcd00ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4294630918 0 0) 0x7fbcd006e580 con 0x7fbcd015f770 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcb8003520 con 0x7fbcd015f770 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 96894606 0 0) 0x7fbcb8003520 con 0x7fbcd015f770 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00b1fd0 msgr2=0x7fbcd00d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00b1fd0 0x7fbcd00d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 >> v2:172.21.15.184:3300/0 conn(0x7fbcd00cdb70 msgr2=0x7fbcd00ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 --2- 172.21.15.40:0/2763976425 >> v2:172.21.15.184:3300/0 conn(0x7fbcd00cdb70 0x7fbcd00ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd006e860 con 0x7fbcd015f770 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fbcc0001670 con 0x7fbcd015f770 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 >> v1:172.21.15.40:6789/0 conn(0x7fbcd015f770 legacy=0x7fbcd00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb80039b0 0x7fbcb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb8004800 0x7fbcb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb80051c0 0x7fbcb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd006f3e0 con 0x7fbcb8004800 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00cd6e0 con 0x7fbcb80039b0 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb800bf00 con 0x7fbcb80051c0 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcd4d77640 1 --2- 172.21.15.40:0/2763976425 >> v2:172.21.15.184:3300/0 conn(0x7fbcd00cdb70 0x7fbcd00ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00b1fd0 0x7fbcd00d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb8004800 0x7fbcb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbccffff640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb80039b0 0x7fbcb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.724+0000 7fbcd4d77640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb80051c0 0x7fbcb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbccf7fe640 1 -- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb80051c0 msgr2=0x7fbcb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb80051c0 0x7fbcb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbccf7fe640 1 -- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb80039b0 msgr2=0x7fbcb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb80039b0 0x7fbcb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbccf7fe640 1 -- 172.21.15.40:0/2763976425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb8003520 con 0x7fbcb8004800 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb8004800 0x7fbcb8004c80 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00b4830 tx=0x7fbcb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6194d495ddcfa00d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4036070 con 0x7fbcb8004800 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc4004030 con 0x7fbcb8004800 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.725+0000 7fbcceffd640 1 -- 172.21.15.40:0/2763976425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40097d0 con 0x7fbcb8004800 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.726+0000 7fbcd67da640 1 -- 172.21.15.40:0/2763976425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb8004800 msgr2=0x7fbcb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.726+0000 7fbcd67da640 1 --2- 172.21.15.40:0/2763976425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb8004800 0x7fbcb8004c80 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00b4830 tx=0x7fbcb8005830 comp rx=0 tx=0).stop 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.726+0000 7fbcd4d77640 1 -- 172.21.15.40:0/2763976425 reap_dead start 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.726+0000 7fbcd67da640 1 -- 172.21.15.40:0/2763976425 shutdown_connections 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.726+0000 7fbcd67da640 1 -- 172.21.15.40:0/2763976425 >> 172.21.15.40:0/2763976425 conn(0x7fbcd00d2e60 msgr2=0x7fbcd00d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.726+0000 7fbcd67da640 1 -- 172.21.15.40:0/2763976425 shutdown_connections 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.726+0000 7fbcd67da640 1 -- 172.21.15.40:0/2763976425 wait complete. 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.727+0000 7fbcd67da640 1 Processor -- start 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.727+0000 7fbcd67da640 1 -- start start 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbcd67da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb8004800 0x7fbcd01f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbcd67da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcd00c68c0 0x7fbcd015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbcd67da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00c7df0 0x7fbcd00c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbcd67da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00d15f0 con 0x7fbcd00c68c0 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbcd67da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00d1470 con 0x7fbcb8004800 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbcd67da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd00d1770 con 0x7fbcd00c7df0 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcd00c68c0 0x7fbcd015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcd00c68c0 0x7fbcd015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42800/0 (socket says 172.21.15.40:42800) 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccf7fe640 1 -- 172.21.15.40:0/3903097198 learned_addr learned my addr 172.21.15.40:0/3903097198 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbcd4d77640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00c7df0 0x7fbcd00c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccffff640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb8004800 0x7fbcd01f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccf7fe640 1 -- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00c7df0 msgr2=0x7fbcd00c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00c7df0 0x7fbcd00c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccf7fe640 1 -- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb8004800 msgr2=0x7fbcd01f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb8004800 0x7fbcd01f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccf7fe640 1 -- 172.21.15.40:0/3903097198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd015b8c0 con 0x7fbcd00c68c0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbcd4d77640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00c7df0 0x7fbcd00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.728+0000 7fbccffff640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb8004800 0x7fbcd01f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.729+0000 7fbccf7fe640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcd00c68c0 0x7fbcd015b380 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009bf0 tx=0x7fbcc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.729+0000 7fbcccff9640 1 -- 172.21.15.40:0/3903097198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4036040 con 0x7fbcd00c68c0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.729+0000 7fbcccff9640 1 -- 172.21.15.40:0/3903097198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc4043070 con 0x7fbcd00c68c0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.729+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd00c6d50 con 0x7fbcd00c68c0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.729+0000 7fbcccff9640 1 -- 172.21.15.40:0/3903097198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40097d0 con 0x7fbcd00c68c0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.729+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd00c7310 con 0x7fbcd00c68c0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.730+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd00694f0 con 0x7fbcd00c68c0 2024-04-26T21:47:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.730+0000 7fbcccff9640 1 -- 172.21.15.40:0/3903097198 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcc403c470 con 0x7fbcd00c68c0 2024-04-26T21:47:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.731+0000 7fbcccff9640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.731+0000 7fbcccff9640 1 -- 172.21.15.40:0/3903097198 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fbcc40ac130 con 0x7fbcd00c68c0 2024-04-26T21:47:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.731+0000 7fbccffff640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.732+0000 7fbccffff640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0001eb0 tx=0x7fbcc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.735+0000 7fbcccff9640 1 -- 172.21.15.40:0/3903097198 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcc404c020 con 0x7fbcd00c68c0 2024-04-26T21:47:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.920+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbcd006fe50 con 0x7fbcd00c68c0 2024-04-26T21:47:42.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.921+0000 7fbcccff9640 1 -- 172.21.15.40:0/3903097198 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) ==== 93+0+10 (secure 0 0 0) 0x7fbcc407a2f0 con 0x7fbcd00c68c0 2024-04-26T21:47:42.921 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcbc06c8f0 msgr2=0x7fbcbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0001eb0 tx=0x7fbcc0009040 comp rx=0 tx=0).stop 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcd00c68c0 msgr2=0x7fbcd015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcd00c68c0 0x7fbcd015b380 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009bf0 tx=0x7fbcc4004730 comp rx=0 tx=0).stop 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 shutdown_connections 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcd00c7df0 0x7fbcd00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcd00c68c0 0x7fbcd015b380 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 --2- 172.21.15.40:0/3903097198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcb8004800 0x7fbcd01f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 >> 172.21.15.40:0/3903097198 conn(0x7fbcd00d2e60 msgr2=0x7fbcd00cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 shutdown_connections 2024-04-26T21:47:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:42.923+0000 7fbcd67da640 1 -- 172.21.15.40:0/3903097198 wait complete. 2024-04-26T21:47:42.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:47:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.038+0000 7fd9d9252640 1 Processor -- start 2024-04-26T21:47:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.038+0000 7fd9d9252640 1 -- start start 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.038+0000 7fd9d9252640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d415f1d0 0x7fd9d406f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.038+0000 7fd9d9252640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d4073500 con 0x7fd9d40afd60 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.038+0000 7fd9d9252640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d40b4480 con 0x7fd9d406c240 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.038+0000 7fd9d9252640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d40ae8c0 con 0x7fd9d415f1d0 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d359e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd9d40afd60 0x7fd9d4072df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42358/0 (socket says 172.21.15.40:42358) 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d359e640 1 -- 172.21.15.40:0/2260892130 learned_addr learned my addr 172.21.15.40:0/2260892130 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d2d9d640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d415f1d0 0x7fd9d406f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248495178 0 0) 0x7fd9d4073500 con 0x7fd9d40afd60 2024-04-26T21:47:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9bc003520 con 0x7fd9d40afd60 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195520527 0 0) 0x7fd9d40b4480 con 0x7fd9d406c240 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9d4073500 con 0x7fd9d406c240 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3120636797 0 0) 0x7fd9bc003520 con 0x7fd9d40afd60 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d415f1d0 msgr2=0x7fd9d406f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d415f1d0 0x7fd9d406f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 >> v1:172.21.15.184:6789/0 conn(0x7fd9d406c240 legacy=0x7fd9d40726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d40b4720 con 0x7fd9d40afd60 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fd9cc0019f0 con 0x7fd9d40afd60 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 >> v1:172.21.15.40:6789/0 conn(0x7fd9d40afd60 legacy=0x7fd9d4072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc003b70 0x7fd9bc003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9bc004800 0x7fd9bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9bc0051c0 0x7fd9bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d40ae8c0 con 0x7fd9bc004800 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc00bfe0 con 0x7fd9bc003b70 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc00c1e0 con 0x7fd9bc0051c0 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d2d9d640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d415f1d0 0x7fd9d406f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d259c640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9bc004800 0x7fd9bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.039+0000 7fd9d359e640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9bc0051c0 0x7fd9bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.040+0000 7fd9d2d9d640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc003b70 0x7fd9bc003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.040+0000 7fd9d259c640 1 -- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9bc0051c0 msgr2=0x7fd9bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.040+0000 7fd9d259c640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9bc0051c0 0x7fd9bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.040+0000 7fd9d259c640 1 -- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc003b70 msgr2=0x7fd9bc003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.040+0000 7fd9d259c640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc003b70 0x7fd9bc003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.040+0000 7fd9d259c640 1 -- 172.21.15.40:0/2260892130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc003520 con 0x7fd9bc004800 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.040+0000 7fd9d259c640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9bc004800 0x7fd9bc004c80 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80018c0 tx=0x7fd9bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a56858e836be0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80014e0 con 0x7fd9bc004800 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c80052f0 con 0x7fd9bc004800 2024-04-26T21:47:43.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d1d9b640 1 -- 172.21.15.40:0/2260892130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c802ac90 con 0x7fd9bc004800 2024-04-26T21:47:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d9252640 1 -- 172.21.15.40:0/2260892130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9bc004800 msgr2=0x7fd9bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d9252640 1 --2- 172.21.15.40:0/2260892130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9bc004800 0x7fd9bc004c80 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80018c0 tx=0x7fd9bc005830 comp rx=0 tx=0).stop 2024-04-26T21:47:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d359e640 1 -- 172.21.15.40:0/2260892130 reap_dead start 2024-04-26T21:47:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d9252640 1 -- 172.21.15.40:0/2260892130 shutdown_connections 2024-04-26T21:47:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d9252640 1 -- 172.21.15.40:0/2260892130 >> 172.21.15.40:0/2260892130 conn(0x7fd9d40b3180 msgr2=0x7fd9d40b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d9252640 1 -- 172.21.15.40:0/2260892130 shutdown_connections 2024-04-26T21:47:43.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.041+0000 7fd9d9252640 1 -- 172.21.15.40:0/2260892130 wait complete. 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d9252640 1 Processor -- start 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d9252640 1 -- start start 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d9252640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc004800 0x7fd9d41f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d9252640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d40b13d0 0x7fd9d40b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d9252640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d414b450 0x7fd9d414b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d9252640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d40b4fc0 con 0x7fd9d40b13d0 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d9252640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d40b4ce0 con 0x7fd9bc004800 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d9252640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d4075ea0 con 0x7fd9d414b450 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d40b13d0 0x7fd9d40b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.043+0000 7fd9d259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d40b13d0 0x7fd9d40b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42810/0 (socket says 172.21.15.40:42810) 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d259c640 1 -- 172.21.15.40:0/54130351 learned_addr learned my addr 172.21.15.40:0/54130351 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d2d9d640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc004800 0x7fd9d41f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d359e640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d414b450 0x7fd9d414b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d259c640 1 -- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d414b450 msgr2=0x7fd9d414b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d259c640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d414b450 0x7fd9d414b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d259c640 1 -- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc004800 msgr2=0x7fd9d41f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d259c640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc004800 0x7fd9d41f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d259c640 1 -- 172.21.15.40:0/54130351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d4149ac0 con 0x7fd9d40b13d0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d259c640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d40b13d0 0x7fd9d40b1850 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8008fd0 tx=0x7fd9c80012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9bb7fe640 1 -- 172.21.15.40:0/54130351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8040020 con 0x7fd9d40b13d0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9bb7fe640 1 -- 172.21.15.40:0/54130351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c80054a0 con 0x7fd9d40b13d0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d414c130 con 0x7fd9d40b13d0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9bb7fe640 1 -- 172.21.15.40:0/54130351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8007450 con 0x7fd9d40b13d0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.044+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d414c6f0 con 0x7fd9d40b13d0 2024-04-26T21:47:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.045+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d4068fc0 con 0x7fd9d40b13d0 2024-04-26T21:47:43.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.046+0000 7fd9bb7fe640 1 -- 172.21.15.40:0/54130351 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9c8043020 con 0x7fd9d40b13d0 2024-04-26T21:47:43.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.046+0000 7fd9bb7fe640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9c006c920 0x7fd9c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.046+0000 7fd9bb7fe640 1 -- 172.21.15.40:0/54130351 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fd9c80abc50 con 0x7fd9d40b13d0 2024-04-26T21:47:43.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.046+0000 7fd9d2d9d640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9c006c920 0x7fd9c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.047+0000 7fd9d2d9d640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9c006c920 0x7fd9c006ede0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40d0cb0 tx=0x7fd9c4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.050+0000 7fd9bb7fe640 1 -- 172.21.15.40:0/54130351 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd9c80b1020 con 0x7fd9d40b13d0 2024-04-26T21:47:43.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:47:43.081 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:47:43.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.237+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd9d40b66a0 con 0x7fd9d40b13d0 2024-04-26T21:47:43.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.237+0000 7fd9bb7fe640 1 -- 172.21.15.40:0/54130351 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v146) ==== 104+0+11 (secure 0 0 0) 0x7fd9d40b66a0 con 0x7fd9d40b13d0 2024-04-26T21:47:43.237 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9c006c920 msgr2=0x7fd9c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9c006c920 0x7fd9c006ede0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40d0cb0 tx=0x7fd9c4009190 comp rx=0 tx=0).stop 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d40b13d0 msgr2=0x7fd9d40b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d40b13d0 0x7fd9d40b1850 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8008fd0 tx=0x7fd9c80012d0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 shutdown_connections 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd9c006c920 0x7fd9c006ede0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d414b450 0x7fd9d414b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d40b13d0 0x7fd9d40b1850 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 --2- 172.21.15.40:0/54130351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9bc004800 0x7fd9d41f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 >> 172.21.15.40:0/54130351 conn(0x7fd9d40b3180 msgr2=0x7fd9d40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 shutdown_connections 2024-04-26T21:47:43.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.239+0000 7fd9d9252640 1 -- 172.21.15.40:0/54130351 wait complete. 2024-04-26T21:47:43.250 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:47:43.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.354+0000 7f21cda20640 1 Processor -- start 2024-04-26T21:47:43.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.354+0000 7f21cda20640 1 -- start start 2024-04-26T21:47:43.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.355+0000 7f21cda20640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21c80d1a00 0x7f21c80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.355+0000 7f21cda20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21c80d2320 0x7f21c81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.355+0000 7f21cda20640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c80d0a10 con 0x7f21c80b40d0 2024-04-26T21:47:43.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.355+0000 7f21cda20640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f21c80d0cf0 con 0x7f21c80d2320 2024-04-26T21:47:43.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.355+0000 7f21cda20640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f21c80d0fd0 con 0x7f21c80d1a00 2024-04-26T21:47:43.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.355+0000 7f21c67fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21c80d2320 0x7f21c81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.355+0000 7f21c67fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21c80d2320 0x7f21c81625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42822/0 (socket says 172.21.15.40:42822) 2024-04-26T21:47:43.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.355+0000 7f21c67fc640 1 -- 172.21.15.40:0/3710912992 learned_addr learned my addr 172.21.15.40:0/3710912992 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:43.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.356+0000 7f21c67fc640 1 -- 172.21.15.40:0/3710912992 >> v2:172.21.15.184:3300/0 conn(0x7f21c80d1a00 msgr2=0x7f21c80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.356+0000 7f21c67fc640 1 --2- 172.21.15.40:0/3710912992 >> v2:172.21.15.184:3300/0 conn(0x7f21c80d1a00 0x7f21c80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.356+0000 7f21c67fc640 1 -- 172.21.15.40:0/3710912992 >> v1:172.21.15.184:6789/0 conn(0x7f21c80b40d0 legacy=0x7f21c80d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.356+0000 7f21c67fc640 1 -- 172.21.15.40:0/3710912992 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c80d40a0 con 0x7f21c80d2320 2024-04-26T21:47:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.356+0000 7f21c67fc640 1 --2- 172.21.15.40:0/3710912992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c81625f0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f21c806c650 tx=0x7f21b002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58e34b180157b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.356+0000 7f21c5ffb640 1 -- 172.21.15.40:0/3710912992 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0037070 con 0x7f21c80d2320 2024-04-26T21:47:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.356+0000 7f21c5ffb640 1 -- 172.21.15.40:0/3710912992 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b002ae20 con 0x7f21c80d2320 2024-04-26T21:47:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.356+0000 7f21c5ffb640 1 -- 172.21.15.40:0/3710912992 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0032660 con 0x7f21c80d2320 2024-04-26T21:47:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.357+0000 7f21cda20640 1 -- 172.21.15.40:0/3710912992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 msgr2=0x7f21c81625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.357+0000 7f21cda20640 1 --2- 172.21.15.40:0/3710912992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c81625f0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f21c806c650 tx=0x7f21b002a110 comp rx=0 tx=0).stop 2024-04-26T21:47:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.357+0000 7f21cda20640 1 -- 172.21.15.40:0/3710912992 shutdown_connections 2024-04-26T21:47:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.357+0000 7f21cda20640 1 --2- 172.21.15.40:0/3710912992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c81625f0 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.357+0000 7f21cda20640 1 --2- 172.21.15.40:0/3710912992 >> v2:172.21.15.184:3300/0 conn(0x7f21c80d1a00 0x7f21c80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.357+0000 7f21cda20640 1 -- 172.21.15.40:0/3710912992 >> 172.21.15.40:0/3710912992 conn(0x7f21c80d4fd0 msgr2=0x7f21c80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.357+0000 7f21cda20640 1 -- 172.21.15.40:0/3710912992 shutdown_connections 2024-04-26T21:47:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.357+0000 7f21cda20640 1 -- 172.21.15.40:0/3710912992 wait complete. 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.358+0000 7f21cda20640 1 Processor -- start 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.358+0000 7f21cda20640 1 -- start start 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21cda20640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21c80b40d0 0x7f21c80c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21cda20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21c80d1a00 0x7f21c80cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21cda20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c80b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21cda20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8162b30 con 0x7f21c80d2320 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21cda20640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21c80b5d60 con 0x7f21c80b40d0 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21cda20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21c8162cb0 con 0x7f21c80d1a00 2024-04-26T21:47:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21c77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c80b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21c77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c80b91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42838/0 (socket says 172.21.15.40:42838) 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21c77fe640 1 -- 172.21.15.40:0/1435931072 learned_addr learned my addr 172.21.15.40:0/1435931072 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21c67fc640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21c80d1a00 0x7f21c80cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21c6ffd640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21c80b40d0 0x7f21c80c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21c77fe640 1 -- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21c80d1a00 msgr2=0x7f21c80cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21c77fe640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21c80d1a00 0x7f21c80cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.359+0000 7f21c77fe640 1 -- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21c80b40d0 msgr2=0x7f21c80c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21c77fe640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21c80b40d0 0x7f21c80c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21c77fe640 1 -- 172.21.15.40:0/1435931072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c80c8c10 con 0x7f21c80d2320 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21c6ffd640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21c80b40d0 0x7f21c80c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21c77fe640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c80b91e0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f21b800c780 tx=0x7f21b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21bffff640 1 -- 172.21.15.40:0/1435931072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8017070 con 0x7f21c80d2320 2024-04-26T21:47:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21bffff640 1 -- 172.21.15.40:0/1435931072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b8003da0 con 0x7f21c80d2320 2024-04-26T21:47:43.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21bffff640 1 -- 172.21.15.40:0/1435931072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b80052b0 con 0x7f21c80d2320 2024-04-26T21:47:43.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c80cc0a0 con 0x7f21c80d2320 2024-04-26T21:47:43.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.360+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c80cc660 con 0x7f21c80d2320 2024-04-26T21:47:43.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.361+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c80b3ef0 con 0x7f21c80d2320 2024-04-26T21:47:43.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.361+0000 7f21bffff640 1 -- 172.21.15.40:0/1435931072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21b801a070 con 0x7f21c80d2320 2024-04-26T21:47:43.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.362+0000 7f21bffff640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21a806c8f0 0x7f21a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.362+0000 7f21bffff640 1 -- 172.21.15.40:0/1435931072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f21b808b370 con 0x7f21c80d2320 2024-04-26T21:47:43.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.362+0000 7f21c6ffd640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21a806c8f0 0x7f21a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.363+0000 7f21c6ffd640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21a806c8f0 0x7f21a806edb0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.367+0000 7f21bffff640 1 -- 172.21.15.40:0/1435931072 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21c80b3ef0 con 0x7f21c80d2320 2024-04-26T21:47:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.551+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f21c80694f0 con 0x7f21c80d2320 2024-04-26T21:47:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.552+0000 7f21bffff640 1 -- 172.21.15.40:0/1435931072 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v146) ==== 100+0+11 (secure 0 0 0) 0x7f21b80595c0 con 0x7f21c80d2320 2024-04-26T21:47:43.552 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:47:43.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.553+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21a806c8f0 msgr2=0x7f21a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21a806c8f0 0x7f21a806edb0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0008040 comp rx=0 tx=0).stop 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 msgr2=0x7f21c80b91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c80b91e0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f21b800c780 tx=0x7f21b800cc40 comp rx=0 tx=0).stop 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 shutdown_connections 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f21a806c8f0 0x7f21a806edb0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c80d2320 0x7f21c80b91e0 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21c80d1a00 0x7f21c80cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 --2- 172.21.15.40:0/1435931072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21c80b40d0 0x7f21c80c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 >> 172.21.15.40:0/1435931072 conn(0x7f21c80d4fd0 msgr2=0x7f21c80cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 shutdown_connections 2024-04-26T21:47:43.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.554+0000 7f21cda20640 1 -- 172.21.15.40:0/1435931072 wait complete. 2024-04-26T21:47:43.564 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:43.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.665+0000 7f00ce3da640 1 Processor -- start 2024-04-26T21:47:43.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00ce3da640 1 -- start start 2024-04-26T21:47:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00ce3da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00c815f3a0 0x7f00c80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00ce3da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00c80d1770 0x7f00c80d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00ce3da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 0x7f00c80d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00ce3da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f00c80d25d0 con 0x7f00c80d1770 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00ce3da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f00c80d0e20 con 0x7f00c815f3a0 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00ce3da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00c80b1d70 con 0x7f00c80d2160 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00c77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00c80d1770 0x7f00c80d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00c77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00c80d1770 0x7f00c80d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42854/0 (socket says 172.21.15.40:42854) 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.666+0000 7f00c77fe640 1 -- 172.21.15.40:0/2049299984 learned_addr learned my addr 172.21.15.40:0/2049299984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.667+0000 7f00c7fff640 1 --2- 172.21.15.40:0/2049299984 >> v2:172.21.15.184:3300/0 conn(0x7f00c815f3a0 0x7f00c80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.667+0000 7f00cc977640 1 --2- 172.21.15.40:0/2049299984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 0x7f00c80d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.667+0000 7f00c77fe640 1 -- 172.21.15.40:0/2049299984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 msgr2=0x7f00c80d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.667+0000 7f00c77fe640 1 --2- 172.21.15.40:0/2049299984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 0x7f00c80d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.667+0000 7f00c77fe640 1 -- 172.21.15.40:0/2049299984 >> v2:172.21.15.184:3300/0 conn(0x7f00c815f3a0 msgr2=0x7f00c80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.667+0000 7f00c77fe640 1 --2- 172.21.15.40:0/2049299984 >> v2:172.21.15.184:3300/0 conn(0x7f00c815f3a0 0x7f00c80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.667+0000 7f00c77fe640 1 -- 172.21.15.40:0/2049299984 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c80b2050 con 0x7f00c80d1770 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.667+0000 7f00c77fe640 1 --2- 172.21.15.40:0/2049299984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c80d1770 0x7f00c80d1c20 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f00c8073b50 tx=0x7f00bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c4ee8590fb9b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00c6ffd640 1 -- 172.21.15.40:0/2049299984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc037070 con 0x7f00c80d1770 2024-04-26T21:47:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00c6ffd640 1 -- 172.21.15.40:0/2049299984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc032440 con 0x7f00c80d1770 2024-04-26T21:47:43.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00c6ffd640 1 -- 172.21.15.40:0/2049299984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc032810 con 0x7f00c80d1770 2024-04-26T21:47:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 -- 172.21.15.40:0/2049299984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c80d1770 msgr2=0x7f00c80d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2049299984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c80d1770 0x7f00c80d1c20 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f00c8073b50 tx=0x7f00bc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 -- 172.21.15.40:0/2049299984 shutdown_connections 2024-04-26T21:47:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2049299984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 0x7f00c80d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2049299984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c80d1770 0x7f00c80d1c20 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2049299984 >> v2:172.21.15.184:3300/0 conn(0x7f00c815f3a0 0x7f00c80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 -- 172.21.15.40:0/2049299984 >> 172.21.15.40:0/2049299984 conn(0x7f00c80d4f90 msgr2=0x7f00c80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 -- 172.21.15.40:0/2049299984 shutdown_connections 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.668+0000 7f00ce3da640 1 -- 172.21.15.40:0/2049299984 wait complete. 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.669+0000 7f00ce3da640 1 Processor -- start 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00ce3da640 1 -- start start 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00ce3da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00c80d1770 0x7f00c814cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00ce3da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 0x7f00c80cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00ce3da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815f3a0 0x7f00c80cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00ce3da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00c815f900 con 0x7f00c815f3a0 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00ce3da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00c815f780 con 0x7f00c80d1770 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00ce3da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00c80d3390 con 0x7f00c80d2160 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00cc977640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815f3a0 0x7f00c80cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00cc977640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815f3a0 0x7f00c80cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42868/0 (socket says 172.21.15.40:42868) 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.670+0000 7f00cc977640 1 -- 172.21.15.40:0/2423528102 learned_addr learned my addr 172.21.15.40:0/2423528102 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00c7fff640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00c80d1770 0x7f00c814cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00c77fe640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 0x7f00c80cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00cc977640 1 -- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 msgr2=0x7f00c80cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00cc977640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 0x7f00c80cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00cc977640 1 -- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00c80d1770 msgr2=0x7f00c814cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00cc977640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00c80d1770 0x7f00c814cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00cc977640 1 -- 172.21.15.40:0/2423528102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c81f9450 con 0x7f00c815f3a0 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00c7fff640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00c80d1770 0x7f00c814cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00cc977640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815f3a0 0x7f00c80cb0d0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f00c0007e60 tx=0x7f00c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00c4ff9640 1 -- 172.21.15.40:0/2423528102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0014620 con 0x7f00c815f3a0 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00c4ff9640 1 -- 172.21.15.40:0/2423528102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c0019070 con 0x7f00c815f3a0 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c80ca5f0 con 0x7f00c815f3a0 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00c4ff9640 1 -- 172.21.15.40:0/2423528102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0014a20 con 0x7f00c815f3a0 2024-04-26T21:47:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.671+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c80cabb0 con 0x7f00c815f3a0 2024-04-26T21:47:43.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.672+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c80694f0 con 0x7f00c815f3a0 2024-04-26T21:47:43.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.673+0000 7f00c4ff9640 1 -- 172.21.15.40:0/2423528102 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00c0014bc0 con 0x7f00c815f3a0 2024-04-26T21:47:43.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.673+0000 7f00c4ff9640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00b406c8f0 0x7f00b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:43.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.673+0000 7f00c4ff9640 1 -- 172.21.15.40:0/2423528102 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f00c008c540 con 0x7f00c815f3a0 2024-04-26T21:47:43.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.673+0000 7f00c7fff640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00b406c8f0 0x7f00b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:43.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.674+0000 7f00c7fff640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00b406c8f0 0x7f00b406edb0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009a20 tx=0x7f00b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:43.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.677+0000 7f00c4ff9640 1 -- 172.21.15.40:0/2423528102 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00c005a820 con 0x7f00c815f3a0 2024-04-26T21:47:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.900+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00c80c85c0 con 0x7f00c815f3a0 2024-04-26T21:47:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.900+0000 7f00c4ff9640 1 -- 172.21.15.40:0/2423528102 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7f00c005a120 con 0x7f00c815f3a0 2024-04-26T21:47:43.900 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:43.900 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:43.901 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00b406c8f0 msgr2=0x7f00b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00b406c8f0 0x7f00b406edb0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009a20 tx=0x7f00b8008040 comp rx=0 tx=0).stop 2024-04-26T21:47:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815f3a0 msgr2=0x7f00c80cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815f3a0 0x7f00c80cb0d0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f00c0007e60 tx=0x7f00c000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 shutdown_connections 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f00b406c8f0 0x7f00b406edb0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815f3a0 0x7f00c80cb0d0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00c80d2160 0x7f00c80cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 --2- 172.21.15.40:0/2423528102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00c80d1770 0x7f00c814cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 >> 172.21.15.40:0/2423528102 conn(0x7f00c80d4f90 msgr2=0x7f00c80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 shutdown_connections 2024-04-26T21:47:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:43.902+0000 7f00ce3da640 1 -- 172.21.15.40:0/2423528102 wait complete. 2024-04-26T21:47:43.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:44.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.014+0000 7f7cec1c6640 1 Processor -- start 2024-04-26T21:47:44.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.014+0000 7f7cec1c6640 1 -- start start 2024-04-26T21:47:44.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cec1c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7ce406f790 0x7f7ce406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cec1c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ce4072c50 0x7f7ce4073060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cec1c6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ce4156840 con 0x7f7ce4156460 2024-04-26T21:47:44.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cec1c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7ce40cbf20 con 0x7f7ce4072c50 2024-04-26T21:47:44.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cec1c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7ce40cc200 con 0x7f7ce406f790 2024-04-26T21:47:44.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cea763640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ce4072c50 0x7f7ce4073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7ce9f62640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7ce4156460 0x7f7ce40cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42364/0 (socket says 172.21.15.40:42364) 2024-04-26T21:47:44.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7ce9f62640 1 -- 172.21.15.40:0/4082345730 learned_addr learned my addr 172.21.15.40:0/4082345730 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:44.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7ce9761640 1 --2- 172.21.15.40:0/4082345730 >> v2:172.21.15.184:3300/0 conn(0x7f7ce406f790 0x7f7ce406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cea763640 1 -- 172.21.15.40:0/4082345730 >> v2:172.21.15.184:3300/0 conn(0x7f7ce406f790 msgr2=0x7f7ce406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:44.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cea763640 1 --2- 172.21.15.40:0/4082345730 >> v2:172.21.15.184:3300/0 conn(0x7f7ce406f790 0x7f7ce406fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cea763640 1 -- 172.21.15.40:0/4082345730 >> v1:172.21.15.40:6789/0 conn(0x7f7ce4156460 legacy=0x7f7ce40cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7ce8f60640 1 -- 172.21.15.40:0/4082345730 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121589850 0 0) 0x7f7ce4156840 con 0x7f7ce4156460 2024-04-26T21:47:44.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.015+0000 7f7cea763640 1 -- 172.21.15.40:0/4082345730 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce40cc4e0 con 0x7f7ce4072c50 2024-04-26T21:47:44.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.016+0000 7f7cea763640 1 --2- 172.21.15.40:0/4082345730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce4072c50 0x7f7ce4073060 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40d0b90 tx=0x7f7cdc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50faf06d5192e1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.016+0000 7f7ce8f60640 1 -- 172.21.15.40:0/4082345730 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc007740 con 0x7f7ce4072c50 2024-04-26T21:47:44.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.016+0000 7f7ce8f60640 1 -- 172.21.15.40:0/4082345730 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc0078e0 con 0x7f7ce4072c50 2024-04-26T21:47:44.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.016+0000 7f7ce8f60640 1 -- 172.21.15.40:0/4082345730 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc007cb0 con 0x7f7ce4072c50 2024-04-26T21:47:44.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.017+0000 7f7cec1c6640 1 -- 172.21.15.40:0/4082345730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce4072c50 msgr2=0x7f7ce4073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.017+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/4082345730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce4072c50 0x7f7ce4073060 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40d0b90 tx=0x7f7cdc02a680 comp rx=0 tx=0).stop 2024-04-26T21:47:44.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.017+0000 7f7cec1c6640 1 -- 172.21.15.40:0/4082345730 shutdown_connections 2024-04-26T21:47:44.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.017+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/4082345730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce4072c50 0x7f7ce4073060 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.017+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/4082345730 >> v2:172.21.15.184:3300/0 conn(0x7f7ce406f790 0x7f7ce406fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.017+0000 7f7cec1c6640 1 -- 172.21.15.40:0/4082345730 >> 172.21.15.40:0/4082345730 conn(0x7f7ce40d0be0 msgr2=0x7f7ce40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.017+0000 7f7cec1c6640 1 -- 172.21.15.40:0/4082345730 shutdown_connections 2024-04-26T21:47:44.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.017+0000 7f7cec1c6640 1 -- 172.21.15.40:0/4082345730 wait complete. 2024-04-26T21:47:44.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.018+0000 7f7cec1c6640 1 Processor -- start 2024-04-26T21:47:44.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.018+0000 7f7cec1c6640 1 -- start start 2024-04-26T21:47:44.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7cec1c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce406f790 0x7f7ce41f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7cec1c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce4072c50 0x7f7ce41f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7cec1c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce4156460 0x7f7ce40b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7cec1c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce40cbbe0 con 0x7f7ce406f790 2024-04-26T21:47:44.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7cec1c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4072700 con 0x7f7ce4156460 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7cec1c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce406e260 con 0x7f7ce4072c50 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7ce9f62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce406f790 0x7f7ce41f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7ce9f62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce406f790 0x7f7ce41f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42894/0 (socket says 172.21.15.40:42894) 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7ce9f62640 1 -- 172.21.15.40:0/2156276761 learned_addr learned my addr 172.21.15.40:0/2156276761 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7ce9f62640 1 -- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce4072c50 msgr2=0x7f7ce41f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7ce9f62640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce4072c50 0x7f7ce41f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7ce9f62640 1 -- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce4156460 msgr2=0x7f7ce40b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7ce9f62640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce4156460 0x7f7ce40b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.019+0000 7f7ce9f62640 1 -- 172.21.15.40:0/2156276761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce41f8c60 con 0x7f7ce406f790 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.020+0000 7f7ce9f62640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce406f790 0x7f7ce41f41b0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000c430 tx=0x7f7ce000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.020+0000 7f7cda7fc640 1 -- 172.21.15.40:0/2156276761 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00074e0 con 0x7f7ce406f790 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.020+0000 7f7cda7fc640 1 -- 172.21.15.40:0/2156276761 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce00047e0 con 0x7f7ce406f790 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.020+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce40c5c30 con 0x7f7ce406f790 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.020+0000 7f7cda7fc640 1 -- 172.21.15.40:0/2156276761 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0004bd0 con 0x7f7ce406f790 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.020+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce40c61f0 con 0x7f7ce406f790 2024-04-26T21:47:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.021+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce40694f0 con 0x7f7ce406f790 2024-04-26T21:47:44.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.021+0000 7f7cda7fc640 1 -- 172.21.15.40:0/2156276761 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ce0011070 con 0x7f7ce406f790 2024-04-26T21:47:44.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.021+0000 7f7cda7fc640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7cc806c8f0 0x7f7cc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.022+0000 7f7cda7fc640 1 -- 172.21.15.40:0/2156276761 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f7ce008b990 con 0x7f7ce406f790 2024-04-26T21:47:44.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.022+0000 7f7ce9761640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7cc806c8f0 0x7f7cc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.022+0000 7f7ce9761640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7cc806c8f0 0x7f7cc806edb0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4147eb0 tx=0x7f7cd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.026+0000 7f7cda7fc640 1 -- 172.21.15.40:0/2156276761 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ce41f8c60 con 0x7f7ce406f790 2024-04-26T21:47:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.214+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ce415ebd0 con 0x7f7ce406f790 2024-04-26T21:47:44.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.215+0000 7f7cda7fc640 1 -- 172.21.15.40:0/2156276761 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16977 (secure 0 0 0) 0x7f7ce000e280 con 0x7f7ce406f790 2024-04-26T21:47:44.216 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:44.216 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":146,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:30.391272+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:18.726844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:19.403532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3441996159":"2024-04-27T21:47:30.391245+0000","172.21.15.40:6834/3441996159":"2024-04-27T21:47:30.391245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.217+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7cc806c8f0 msgr2=0x7f7cc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.217+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7cc806c8f0 0x7f7cc806edb0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4147eb0 tx=0x7f7cd4008040 comp rx=0 tx=0).stop 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce406f790 msgr2=0x7f7ce41f41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce406f790 0x7f7ce41f41b0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000c430 tx=0x7f7ce000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 shutdown_connections 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f7cc806c8f0 0x7f7cc806edb0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce4156460 0x7f7ce40b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce4072c50 0x7f7ce41f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 --2- 172.21.15.40:0/2156276761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ce406f790 0x7f7ce41f41b0 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 >> 172.21.15.40:0/2156276761 conn(0x7f7ce40d0be0 msgr2=0x7f7ce40b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 shutdown_connections 2024-04-26T21:47:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.218+0000 7f7cec1c6640 1 -- 172.21.15.40:0/2156276761 wait complete. 2024-04-26T21:47:44.228 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:47:44.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:44.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.330+0000 7fc7529f2640 1 Processor -- start 2024-04-26T21:47:44.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.330+0000 7fc7529f2640 1 -- start start 2024-04-26T21:47:44.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc7529f2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc74c162880 0x7fc74c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc7529f2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc74c0b5f20 0x7fc74c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc7529f2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc74c0b7e50 con 0x7fc74c0d39f0 2024-04-26T21:47:44.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc7529f2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc74c0d3e40 con 0x7fc74c0b5f20 2024-04-26T21:47:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc7529f2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc74c0d0580 con 0x7fc74c162880 2024-04-26T21:47:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc74b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc74c0b5f20 0x7fc74c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc74b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc74c0b5f20 0x7fc74c0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42896/0 (socket says 172.21.15.40:42896) 2024-04-26T21:47:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc74b7fe640 1 -- 172.21.15.40:0/2073887541 learned_addr learned my addr 172.21.15.40:0/2073887541 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc74b7fe640 1 -- 172.21.15.40:0/2073887541 >> v2:172.21.15.184:3300/0 conn(0x7fc74c162880 msgr2=0x7fc74c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc74b7fe640 1 --2- 172.21.15.40:0/2073887541 >> v2:172.21.15.184:3300/0 conn(0x7fc74c162880 0x7fc74c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc74b7fe640 1 -- 172.21.15.40:0/2073887541 >> v1:172.21.15.184:6789/0 conn(0x7fc74c0d39f0 legacy=0x7fc74c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.331+0000 7fc74b7fe640 1 -- 172.21.15.40:0/2073887541 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c0d1800 con 0x7fc74c0b5f20 2024-04-26T21:47:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.332+0000 7fc74b7fe640 1 --2- 172.21.15.40:0/2073887541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0b5f20 0x7fc74c0d34b0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fc74c08d490 tx=0x7fc73c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a741aa22f4589f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.332+0000 7fc74affd640 1 -- 172.21.15.40:0/2073887541 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c037070 con 0x7fc74c0b5f20 2024-04-26T21:47:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.332+0000 7fc74affd640 1 -- 172.21.15.40:0/2073887541 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc73c032440 con 0x7fc74c0b5f20 2024-04-26T21:47:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.332+0000 7fc74affd640 1 -- 172.21.15.40:0/2073887541 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c032810 con 0x7fc74c0b5f20 2024-04-26T21:47:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.333+0000 7fc7529f2640 1 -- 172.21.15.40:0/2073887541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0b5f20 msgr2=0x7fc74c0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.333+0000 7fc7529f2640 1 --2- 172.21.15.40:0/2073887541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0b5f20 0x7fc74c0d34b0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fc74c08d490 tx=0x7fc73c02a230 comp rx=0 tx=0).stop 2024-04-26T21:47:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.333+0000 7fc7529f2640 1 -- 172.21.15.40:0/2073887541 shutdown_connections 2024-04-26T21:47:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.333+0000 7fc7529f2640 1 --2- 172.21.15.40:0/2073887541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0b5f20 0x7fc74c0d34b0 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.333+0000 7fc7529f2640 1 --2- 172.21.15.40:0/2073887541 >> v2:172.21.15.184:3300/0 conn(0x7fc74c162880 0x7fc74c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.333+0000 7fc7529f2640 1 -- 172.21.15.40:0/2073887541 >> 172.21.15.40:0/2073887541 conn(0x7fc74c0d4f60 msgr2=0x7fc74c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.333+0000 7fc7529f2640 1 -- 172.21.15.40:0/2073887541 shutdown_connections 2024-04-26T21:47:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.333+0000 7fc7529f2640 1 -- 172.21.15.40:0/2073887541 wait complete. 2024-04-26T21:47:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.334+0000 7fc7529f2640 1 Processor -- start 2024-04-26T21:47:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.334+0000 7fc7529f2640 1 -- start start 2024-04-26T21:47:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc7529f2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc74c0b5f20 0x7fc74c1588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc7529f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d39f0 0x7fc74c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc7529f2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc74c162880 0x7fc74c0ba310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc7529f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c0d0d90 con 0x7fc74c0d39f0 2024-04-26T21:47:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc7529f2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c0d0ab0 con 0x7fc74c162880 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc7529f2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc74c162c60 con 0x7fc74c0b5f20 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d39f0 0x7fc74c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d39f0 0x7fc74c15b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42908/0 (socket says 172.21.15.40:42908) 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74b7fe640 1 -- 172.21.15.40:0/1276495333 learned_addr learned my addr 172.21.15.40:0/1276495333 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc750f8f640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc74c162880 0x7fc74c0ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74b7fe640 1 -- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc74c0b5f20 msgr2=0x7fc74c1588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74bfff640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc74c0b5f20 0x7fc74c1588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74b7fe640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc74c0b5f20 0x7fc74c1588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74b7fe640 1 -- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc74c162880 msgr2=0x7fc74c0ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74b7fe640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc74c162880 0x7fc74c0ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc74b7fe640 1 -- 172.21.15.40:0/1276495333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c15c430 con 0x7fc74c0d39f0 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.335+0000 7fc750f8f640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc74c162880 0x7fc74c0ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.336+0000 7fc74bfff640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc74c0b5f20 0x7fc74c1588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.336+0000 7fc74b7fe640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d39f0 0x7fc74c15b770 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fc73c000c00 tx=0x7fc73c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.336+0000 7fc748ff9640 1 -- 172.21.15.40:0/1276495333 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c041020 con 0x7fc74c0d39f0 2024-04-26T21:47:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.336+0000 7fc748ff9640 1 -- 172.21.15.40:0/1276495333 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc73c030970 con 0x7fc74c0d39f0 2024-04-26T21:47:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.336+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74c158e30 con 0x7fc74c0d39f0 2024-04-26T21:47:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.336+0000 7fc748ff9640 1 -- 172.21.15.40:0/1276495333 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c037040 con 0x7fc74c0d39f0 2024-04-26T21:47:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.336+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc74c0cd190 con 0x7fc74c0d39f0 2024-04-26T21:47:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.337+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc74c0694f0 con 0x7fc74c0d39f0 2024-04-26T21:47:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.337+0000 7fc748ff9640 1 -- 172.21.15.40:0/1276495333 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc73c044020 con 0x7fc74c0d39f0 2024-04-26T21:47:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.338+0000 7fc748ff9640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc73806c8f0 0x7fc73806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.338+0000 7fc748ff9640 1 -- 172.21.15.40:0/1276495333 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7fc73c0ac840 con 0x7fc74c0d39f0 2024-04-26T21:47:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.338+0000 7fc74bfff640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc73806c8f0 0x7fc73806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.339+0000 7fc74bfff640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc73806c8f0 0x7fc73806edb0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc740006fd0 tx=0x7fc740008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.342+0000 7fc748ff9640 1 -- 172.21.15.40:0/1276495333 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc73c04b030 con 0x7fc74c0d39f0 2024-04-26T21:47:44.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.568+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc74c0d53e0 con 0x7fc74c0d39f0 2024-04-26T21:47:44.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.569+0000 7fc748ff9640 1 -- 172.21.15.40:0/1276495333 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7fc74c0d1800 con 0x7fc74c0d39f0 2024-04-26T21:47:44.568 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:44.569 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:44.569 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc73806c8f0 msgr2=0x7fc73806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc73806c8f0 0x7fc73806edb0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc740006fd0 tx=0x7fc740008040 comp rx=0 tx=0).stop 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d39f0 msgr2=0x7fc74c15b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d39f0 0x7fc74c15b770 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fc73c000c00 tx=0x7fc73c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 shutdown_connections 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fc73806c8f0 0x7fc73806edb0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc74c162880 0x7fc74c0ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d39f0 0x7fc74c15b770 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 --2- 172.21.15.40:0/1276495333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc74c0b5f20 0x7fc74c1588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.570+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 >> 172.21.15.40:0/1276495333 conn(0x7fc74c0d4f60 msgr2=0x7fc74c160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.571+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 shutdown_connections 2024-04-26T21:47:44.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.571+0000 7fc7529f2640 1 -- 172.21.15.40:0/1276495333 wait complete. 2024-04-26T21:47:44.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:47:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.682+0000 7f81210c5640 1 Processor -- start 2024-04-26T21:47:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.682+0000 7f81210c5640 1 -- start start 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.682+0000 7f81210c5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.682+0000 7f81210c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f811c0d2300 0x7f811c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.682+0000 7f81210c5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f811c0d0e20 con 0x7f811c0d1770 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.682+0000 7f81210c5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f811c0b1d70 con 0x7f811c0d2300 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.682+0000 7f81210c5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f811c0b2050 con 0x7f811c15f3a0 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.683+0000 7f811b59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f811c0d2300 0x7f811c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.683+0000 7f811b59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f811c0d2300 0x7f811c0b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42918/0 (socket says 172.21.15.40:42918) 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.683+0000 7f811b59e640 1 -- 172.21.15.40:0/2609065007 learned_addr learned my addr 172.21.15.40:0/2609065007 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.683+0000 7f811ad9d640 1 --2- 172.21.15.40:0/2609065007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.683+0000 7f811b59e640 1 -- 172.21.15.40:0/2609065007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 msgr2=0x7f811c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.683+0000 7f811b59e640 1 --2- 172.21.15.40:0/2609065007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.683+0000 7f811b59e640 1 -- 172.21.15.40:0/2609065007 >> v1:172.21.15.184:6789/0 conn(0x7f811c0d1770 legacy=0x7f811c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.683+0000 7f811b59e640 1 -- 172.21.15.40:0/2609065007 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c0b2330 con 0x7f811c0d2300 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f811b59e640 1 --2- 172.21.15.40:0/2609065007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c0b2850 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f8114009e50 tx=0x7f811402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfbc4bd20765549 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f8119d9b640 1 -- 172.21.15.40:0/2609065007 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114036070 con 0x7f811c0d2300 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f8119d9b640 1 -- 172.21.15.40:0/2609065007 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8114031600 con 0x7f811c0d2300 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f8119d9b640 1 -- 172.21.15.40:0/2609065007 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81140319d0 con 0x7f811c0d2300 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f81210c5640 1 -- 172.21.15.40:0/2609065007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 msgr2=0x7f811c0b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f81210c5640 1 --2- 172.21.15.40:0/2609065007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c0b2850 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f8114009e50 tx=0x7f811402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f81210c5640 1 -- 172.21.15.40:0/2609065007 shutdown_connections 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f81210c5640 1 --2- 172.21.15.40:0/2609065007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c0b2850 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f81210c5640 1 --2- 172.21.15.40:0/2609065007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f81210c5640 1 -- 172.21.15.40:0/2609065007 >> 172.21.15.40:0/2609065007 conn(0x7f811c0d4f90 msgr2=0x7f811c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f81210c5640 1 -- 172.21.15.40:0/2609065007 shutdown_connections 2024-04-26T21:47:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.684+0000 7f81210c5640 1 -- 172.21.15.40:0/2609065007 wait complete. 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.686+0000 7f81210c5640 1 Processor -- start 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.686+0000 7f81210c5640 1 -- start start 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.686+0000 7f81210c5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f811c0d1770 0x7f811c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.686+0000 7f81210c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c162430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.686+0000 7f81210c5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.686+0000 7f81210c5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f811c0b2f10 con 0x7f811c0d2300 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.686+0000 7f81210c5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f811c0b2d90 con 0x7f811c0d1770 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.686+0000 7f81210c5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f811c0b3090 con 0x7f811c15f3a0 2024-04-26T21:47:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42932/0 (socket says 172.21.15.40:42932) 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 -- 172.21.15.40:0/3668738927 learned_addr learned my addr 172.21.15.40:0/3668738927 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811ad9d640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f811c0d1770 0x7f811c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811b59e640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 -- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 msgr2=0x7f811c160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 -- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f811c0d1770 msgr2=0x7f811c0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f811c0d1770 0x7f811c0cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 -- 172.21.15.40:0/3668738927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c15e560 con 0x7f811c0d2300 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811b59e640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f811a59c640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c162430 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f811000c430 tx=0x7f811000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f81037fe640 1 -- 172.21.15.40:0/3668738927 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110011790 con 0x7f811c0d2300 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f81037fe640 1 -- 172.21.15.40:0/3668738927 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8110017070 con 0x7f811c0d2300 2024-04-26T21:47:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811c15fcc0 con 0x7f811c0d2300 2024-04-26T21:47:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f81037fe640 1 -- 172.21.15.40:0/3668738927 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110011bb0 con 0x7f811c0d2300 2024-04-26T21:47:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.687+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811c160280 con 0x7f811c0d2300 2024-04-26T21:47:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.688+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f811c0694f0 con 0x7f811c0d2300 2024-04-26T21:47:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.689+0000 7f81037fe640 1 -- 172.21.15.40:0/3668738927 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f811001c460 con 0x7f811c0d2300 2024-04-26T21:47:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.689+0000 7f81037fe640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f810806c8f0 0x7f810806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.689+0000 7f81037fe640 1 -- 172.21.15.40:0/3668738927 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f811008bc50 con 0x7f811c0d2300 2024-04-26T21:47:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.689+0000 7f811ad9d640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f810806c8f0 0x7f810806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.690+0000 7f811ad9d640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f810806c8f0 0x7f810806edb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f810c006fd0 tx=0x7f810c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:44.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.693+0000 7f81037fe640 1 -- 172.21.15.40:0/3668738927 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8110091020 con 0x7f811c0d2300 2024-04-26T21:47:44.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.919+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f811c0cbe20 con 0x7f811c0d2300 2024-04-26T21:47:44.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.919+0000 7f81037fe640 1 -- 172.21.15.40:0/3668738927 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) ==== 95+0+5203 (secure 0 0 0) 0x7f811c15e560 con 0x7f811c0d2300 2024-04-26T21:47:44.919 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:44.919 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:47:20.455876+0000","modified":"2024-04-26T21:47:34.414722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11961":{"gid":11961,"name":"a","rank":0,"incarnation":360,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3119057314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3119057314},{"type":"v1","addr":"172.21.15.40:6835","nonce":3119057314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11961,"qdb_cluster":[11961]},"id":7}]} 2024-04-26T21:47:44.919 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 363 2024-04-26T21:47:44.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f810806c8f0 msgr2=0x7f810806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f810806c8f0 0x7f810806edb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f810c006fd0 tx=0x7f810c008040 comp rx=0 tx=0).stop 2024-04-26T21:47:44.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 msgr2=0x7f811c162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c162430 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f811000c430 tx=0x7f811000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 shutdown_connections 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f810806c8f0 0x7f810806edb0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f811c15f3a0 0x7f811c160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c0d2300 0x7f811c162430 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 --2- 172.21.15.40:0/3668738927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f811c0d1770 0x7f811c0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 >> 172.21.15.40:0/3668738927 conn(0x7f811c0d4f90 msgr2=0x7f811c0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 shutdown_connections 2024-04-26T21:47:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:44.921+0000 7f81210c5640 1 -- 172.21.15.40:0/3668738927 wait complete. 2024-04-26T21:47:44.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.033+0000 7f25b75af640 1 Processor -- start 2024-04-26T21:47:45.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.033+0000 7f25b75af640 1 -- start start 2024-04-26T21:47:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b75af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f25b0162810 0x7f25b0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b75af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b00d1b30 0x7f25b00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b75af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b00d10d0 con 0x7f25b00b2340 2024-04-26T21:47:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b75af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f25b00d0750 con 0x7f25b0162810 2024-04-26T21:47:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b75af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25b00d16a0 con 0x7f25b00d1b30 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b4b4a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f25b00b2340 0x7f25b00cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42374/0 (socket says 172.21.15.40:42374) 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b4b4a640 1 -- 172.21.15.40:0/2304449695 learned_addr learned my addr 172.21.15.40:0/2304449695 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030353090 0 0) 0x7f25b00d10d0 con 0x7f25b00b2340 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f259c003520 con 0x7f25b00b2340 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b5b4c640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b00d1b30 0x7f25b00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b534b640 1 --2- 172.21.15.40:0/2304449695 >> v2:172.21.15.184:3300/0 conn(0x7f25b0162810 0x7f25b0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3942794655 0 0) 0x7f259c003520 con 0x7f25b00b2340 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b00d1b30 msgr2=0x7f25b00d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b00d1b30 0x7f25b00d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 >> v2:172.21.15.184:3300/0 conn(0x7f25b0162810 msgr2=0x7f25b0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 --2- 172.21.15.40:0/2304449695 >> v2:172.21.15.184:3300/0 conn(0x7f25b0162810 0x7f25b0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00d1370 con 0x7f25b00b2340 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f25a0001670 con 0x7f25b00b2340 2024-04-26T21:47:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 >> v1:172.21.15.40:6789/0 conn(0x7f25b00b2340 legacy=0x7f25b00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b00d1b30 0x7f259c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f259c003fe0 0x7f259c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f259c004d40 0x7f259c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25b00d0750 con 0x7f259c003fe0 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f25b00d16a0 con 0x7f259c004d40 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f259c00bac0 con 0x7f25b00d1b30 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.034+0000 7f25b4b4a640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f259c003fe0 0x7f259c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.035+0000 7f25b534b640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b00d1b30 0x7f259c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.035+0000 7f25b5b4c640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f259c004d40 0x7f259c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.035+0000 7f25b4b4a640 1 -- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b00d1b30 msgr2=0x7f259c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.035+0000 7f25b4b4a640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b00d1b30 0x7f259c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.035+0000 7f25b4b4a640 1 -- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f259c004d40 msgr2=0x7f259c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.035+0000 7f25b4b4a640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f259c004d40 0x7f259c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.035+0000 7f25b4b4a640 1 -- 172.21.15.40:0/2304449695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c003520 con 0x7f259c003fe0 2024-04-26T21:47:45.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.035+0000 7f25b4b4a640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f259c003fe0 0x7f259c004800 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c210 tx=0x7f259c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcc035d42b4a167 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0007d50 con 0x7f259c003fe0 2024-04-26T21:47:45.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a0005af0 con 0x7f259c003fe0 2024-04-26T21:47:45.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25a7fff640 1 -- 172.21.15.40:0/2304449695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0007070 con 0x7f259c003fe0 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25b75af640 1 -- 172.21.15.40:0/2304449695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f259c003fe0 msgr2=0x7f259c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25b75af640 1 --2- 172.21.15.40:0/2304449695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f259c003fe0 0x7f259c004800 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c210 tx=0x7f259c005830 comp rx=0 tx=0).stop 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25b5b4c640 1 -- 172.21.15.40:0/2304449695 reap_dead start 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25b75af640 1 -- 172.21.15.40:0/2304449695 shutdown_connections 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25b75af640 1 -- 172.21.15.40:0/2304449695 >> 172.21.15.40:0/2304449695 conn(0x7f25b00d2da0 msgr2=0x7f25b00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25b75af640 1 -- 172.21.15.40:0/2304449695 shutdown_connections 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.036+0000 7f25b75af640 1 -- 172.21.15.40:0/2304449695 wait complete. 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.037+0000 7f25b75af640 1 Processor -- start 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.037+0000 7f25b75af640 1 -- start start 2024-04-26T21:47:45.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b75af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f259c003fe0 0x7f25b0149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b75af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00c8600 0x7f25b00c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b75af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25b015d5e0 0x7f25b015da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b75af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0070890 con 0x7f25b00c8600 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b75af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f25b00d0470 con 0x7f25b015d5e0 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b75af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25b00cdb00 con 0x7f259c003fe0 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b4b4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00c8600 0x7f25b00c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b4b4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00c8600 0x7f25b00c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42940/0 (socket says 172.21.15.40:42940) 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b4b4a640 1 -- 172.21.15.40:0/3798807079 learned_addr learned my addr 172.21.15.40:0/3798807079 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b4b4a640 1 -- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f259c003fe0 msgr2=0x7f25b0149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b4b4a640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f259c003fe0 0x7f25b0149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b4b4a640 1 -- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25b015d5e0 msgr2=0x7f25b015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.038+0000 7f25b4b4a640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25b015d5e0 0x7f25b015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.039+0000 7f25b4b4a640 1 -- 172.21.15.40:0/3798807079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00c9070 con 0x7f25b00c8600 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.039+0000 7f25b4b4a640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00c8600 0x7f25b00c8a80 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f25a002ae50 tx=0x7f25a0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.039+0000 7f25a5ffb640 1 -- 172.21.15.40:0/3798807079 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0042070 con 0x7f25b00c8600 2024-04-26T21:47:45.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.039+0000 7f25a5ffb640 1 -- 172.21.15.40:0/3798807079 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a0007480 con 0x7f25b00c8600 2024-04-26T21:47:45.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.039+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b00c9350 con 0x7f25b00c8600 2024-04-26T21:47:45.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.039+0000 7f25a5ffb640 1 -- 172.21.15.40:0/3798807079 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0001c90 con 0x7f25b00c8600 2024-04-26T21:47:45.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.039+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b00c9910 con 0x7f25b00c8600 2024-04-26T21:47:45.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.040+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b00694f0 con 0x7f25b00c8600 2024-04-26T21:47:45.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.041+0000 7f25a5ffb640 1 -- 172.21.15.40:0/3798807079 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25a0045070 con 0x7f25b00c8600 2024-04-26T21:47:45.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.041+0000 7f25a5ffb640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f259406c970 0x7f259406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.041+0000 7f25a5ffb640 1 -- 172.21.15.40:0/3798807079 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f25a00ac6f0 con 0x7f25b00c8600 2024-04-26T21:47:45.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.041+0000 7f25b534b640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f259406c970 0x7f259406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.042+0000 7f25b534b640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f259406c970 0x7f259406ee30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006a90 tx=0x7f25ac006a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.045+0000 7f25a5ffb640 1 -- 172.21.15.40:0/3798807079 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25a007a460 con 0x7f25b00c8600 2024-04-26T21:47:45.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.234+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25b00b1e10 con 0x7f25b00c8600 2024-04-26T21:47:45.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.235+0000 7f25a5ffb640 1 -- 172.21.15.40:0/3798807079 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16977 (secure 0 0 0) 0x7f25a007e760 con 0x7f25b00c8600 2024-04-26T21:47:45.235 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:45.236 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":146,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:30.391272+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:47:18.726844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-26T21:47:19.403532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3441996159":"2024-04-27T21:47:30.391245+0000","172.21.15.40:6834/3441996159":"2024-04-27T21:47:30.391245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f259406c970 msgr2=0x7f259406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f259406c970 0x7f259406ee30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006a90 tx=0x7f25ac006a20 comp rx=0 tx=0).stop 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00c8600 msgr2=0x7f25b00c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00c8600 0x7f25b00c8a80 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f25a002ae50 tx=0x7f25a0001a40 comp rx=0 tx=0).stop 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 shutdown_connections 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f259406c970 0x7f259406ee30 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25b015d5e0 0x7f25b015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00c8600 0x7f25b00c8a80 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 --2- 172.21.15.40:0/3798807079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f259c003fe0 0x7f25b0149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 >> 172.21.15.40:0/3798807079 conn(0x7f25b00d2da0 msgr2=0x7f25b00b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 shutdown_connections 2024-04-26T21:47:45.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.238+0000 7f25b75af640 1 -- 172.21.15.40:0/3798807079 wait complete. 2024-04-26T21:47:45.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:47:45.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.350+0000 7f2b3811c640 1 Processor -- start 2024-04-26T21:47:45.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.350+0000 7f2b3811c640 1 -- start start 2024-04-26T21:47:45.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.351+0000 7f2b3811c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b300b2340 0x7f2b300cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.351+0000 7f2b3811c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2b300cd930 0x7f2b300d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.351+0000 7f2b3811c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b300d10d0 con 0x7f2b30162810 2024-04-26T21:47:45.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.351+0000 7f2b3811c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2b300d0750 con 0x7f2b300b2340 2024-04-26T21:47:45.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.351+0000 7f2b3811c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2b300d16a0 con 0x7f2b300cd930 2024-04-26T21:47:45.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.351+0000 7f2b35eb8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2b30162810 0x7f2b30070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42378/0 (socket says 172.21.15.40:42378) 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.351+0000 7f2b35eb8640 1 -- 172.21.15.40:0/2512163431 learned_addr learned my addr 172.21.15.40:0/2512163431 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.351+0000 7f2b356b7640 1 --2- 172.21.15.40:0/2512163431 >> v2:172.21.15.40:3300/0 conn(0x7f2b300b2340 0x7f2b300cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b366b9640 1 --2- 172.21.15.40:0/2512163431 >> v2:172.21.15.184:3300/0 conn(0x7f2b300cd930 0x7f2b300d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b34eb6640 1 -- 172.21.15.40:0/2512163431 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2427508710 0 0) 0x7f2b300d10d0 con 0x7f2b30162810 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b34eb6640 1 -- 172.21.15.40:0/2512163431 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b18003520 con 0x7f2b30162810 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b356b7640 1 -- 172.21.15.40:0/2512163431 >> v2:172.21.15.184:3300/0 conn(0x7f2b300cd930 msgr2=0x7f2b300d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b356b7640 1 --2- 172.21.15.40:0/2512163431 >> v2:172.21.15.184:3300/0 conn(0x7f2b300cd930 0x7f2b300d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b356b7640 1 -- 172.21.15.40:0/2512163431 >> v1:172.21.15.40:6789/0 conn(0x7f2b30162810 legacy=0x7f2b30070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b356b7640 1 -- 172.21.15.40:0/2512163431 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b300d1370 con 0x7f2b300b2340 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b356b7640 1 --2- 172.21.15.40:0/2512163431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300b2340 0x7f2b300cd3f0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f2b300b2280 tx=0x7f2b2402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5a9bccd9ec569d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.352+0000 7f2b34eb6640 1 -- 172.21.15.40:0/2512163431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24037070 con 0x7f2b300b2340 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b34eb6640 1 -- 172.21.15.40:0/2512163431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b24032440 con 0x7f2b300b2340 2024-04-26T21:47:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b34eb6640 1 -- 172.21.15.40:0/2512163431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24032810 con 0x7f2b300b2340 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b3811c640 1 -- 172.21.15.40:0/2512163431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300b2340 msgr2=0x7f2b300cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2512163431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300b2340 0x7f2b300cd3f0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f2b300b2280 tx=0x7f2b2402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b3811c640 1 -- 172.21.15.40:0/2512163431 shutdown_connections 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2512163431 >> v2:172.21.15.184:3300/0 conn(0x7f2b300cd930 0x7f2b300d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2512163431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300b2340 0x7f2b300cd3f0 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b3811c640 1 -- 172.21.15.40:0/2512163431 >> 172.21.15.40:0/2512163431 conn(0x7f2b300d2da0 msgr2=0x7f2b300d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b3811c640 1 -- 172.21.15.40:0/2512163431 shutdown_connections 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.353+0000 7f2b3811c640 1 -- 172.21.15.40:0/2512163431 wait complete. 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.354+0000 7f2b3811c640 1 Processor -- start 2024-04-26T21:47:45.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.354+0000 7f2b3811c640 1 -- start start 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b3811c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b300b2340 0x7f2b301493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b3811c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300cd930 0x7f2b3006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b3811c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b30162810 0x7f2b3006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b3811c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2b300d2070 con 0x7f2b300cd930 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b3811c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b300d0470 con 0x7f2b300b2340 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b3811c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2b300d21f0 con 0x7f2b30162810 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b356b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300cd930 0x7f2b3006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b356b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300cd930 0x7f2b3006e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42968/0 (socket says 172.21.15.40:42968) 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b356b7640 1 -- 172.21.15.40:0/2429409397 learned_addr learned my addr 172.21.15.40:0/2429409397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b356b7640 1 -- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b30162810 msgr2=0x7f2b3006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b35eb8640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b300b2340 0x7f2b301493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b366b9640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b30162810 0x7f2b3006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b356b7640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b30162810 0x7f2b3006ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b356b7640 1 -- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b300b2340 msgr2=0x7f2b301493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b356b7640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b300b2340 0x7f2b301493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b356b7640 1 -- 172.21.15.40:0/2429409397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b301498e0 con 0x7f2b300cd930 2024-04-26T21:47:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.355+0000 7f2b35eb8640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b300b2340 0x7f2b301493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.356+0000 7f2b356b7640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300cd930 0x7f2b3006e4d0 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f2b24000c00 tx=0x7f2b240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.356+0000 7f2b2e7fc640 1 -- 172.21.15.40:0/2429409397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24037040 con 0x7f2b300cd930 2024-04-26T21:47:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.356+0000 7f2b2e7fc640 1 -- 172.21.15.40:0/2429409397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b24042070 con 0x7f2b300cd930 2024-04-26T21:47:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.356+0000 7f2b2e7fc640 1 -- 172.21.15.40:0/2429409397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24030960 con 0x7f2b300cd930 2024-04-26T21:47:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.356+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b300c9050 con 0x7f2b300cd930 2024-04-26T21:47:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.356+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b300c9610 con 0x7f2b300cd930 2024-04-26T21:47:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.357+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b300694f0 con 0x7f2b300cd930 2024-04-26T21:47:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.357+0000 7f2b2e7fc640 1 -- 172.21.15.40:0/2429409397 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b24030b00 con 0x7f2b300cd930 2024-04-26T21:47:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.358+0000 7f2b2e7fc640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2b1c06c8f0 0x7f2b1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.358+0000 7f2b2e7fc640 1 -- 172.21.15.40:0/2429409397 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6187+0+0 (secure 0 0 0) 0x7f2b240ac690 con 0x7f2b300cd930 2024-04-26T21:47:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.358+0000 7f2b35eb8640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2b1c06c8f0 0x7f2b1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.359+0000 7f2b35eb8640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2b1c06c8f0 0x7f2b1c06edb0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2b28005880 tx=0x7f2b28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.362+0000 7f2b2e7fc640 1 -- 172.21.15.40:0/2429409397 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b2407a8e0 con 0x7f2b300cd930 2024-04-26T21:47:45.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.591+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2b30069c00 con 0x7f2b300cd930 2024-04-26T21:47:45.591 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:47:45.591+0000 7f861e1e0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:47:45.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.593+0000 7f2b2e7fc640 1 -- 172.21.15.40:0/2429409397 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v364) ==== 161+0+0 (secure 0 0 0) 0x7f2b2407a1e0 con 0x7f2b300cd930 2024-04-26T21:47:45.593 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:47:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2b1c06c8f0 msgr2=0x7f2b1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2b1c06c8f0 0x7f2b1c06edb0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2b28005880 tx=0x7f2b28009040 comp rx=0 tx=0).stop 2024-04-26T21:47:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300cd930 msgr2=0x7f2b3006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300cd930 0x7f2b3006e4d0 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f2b24000c00 tx=0x7f2b240046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 shutdown_connections 2024-04-26T21:47:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f2b1c06c8f0 0x7f2b1c06edb0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b30162810 0x7f2b3006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b300cd930 0x7f2b3006e4d0 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 --2- 172.21.15.40:0/2429409397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b300b2340 0x7f2b301493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 >> 172.21.15.40:0/2429409397 conn(0x7f2b300d2da0 msgr2=0x7f2b300b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 shutdown_connections 2024-04-26T21:47:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.596+0000 7f2b3811c640 1 -- 172.21.15.40:0/2429409397 wait complete. 2024-04-26T21:47:45.597 INFO:tasks.ceph.mds.a.smithi040.stderr: -1135> 2024-04-26T21:47:36.404+0000 7f928b37d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:47:45.598 INFO:tasks.ceph.mds.a.smithi040.stderr: -748> 2024-04-26T21:47:37.296+0000 7f9287375640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 104: Malformed input [buffer:3] 2024-04-26T21:47:45.598 INFO:tasks.ceph.mds.a.smithi040.stderr: -511> 2024-04-26T21:47:38.446+0000 7f9287375640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-04-26T21:47:45.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:47:45.625 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:47:45.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.713+0000 7efcbfdbf640 1 Processor -- start 2024-04-26T21:47:45.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.713+0000 7efcbfdbf640 1 -- start start 2024-04-26T21:47:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbfdbf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efcb80b2340 0x7efcb80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbfdbf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb80d10d0 con 0x7efcb8162810 2024-04-26T21:47:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbfdbf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb80d1370 con 0x7efcb80cd930 2024-04-26T21:47:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbfdbf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efcb80d0750 con 0x7efcb80b2340 2024-04-26T21:47:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbd35a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efcb80b2340 0x7efcb80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbdb5b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efcb8162810 0x7efcb8070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42386/0 (socket says 172.21.15.40:42386) 2024-04-26T21:47:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbdb5b640 1 -- 172.21.15.40:0/234381181 learned_addr learned my addr 172.21.15.40:0/234381181 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbd35a640 1 -- 172.21.15.40:0/234381181 >> v1:172.21.15.184:6789/0 conn(0x7efcb80cd930 legacy=0x7efcb80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:45.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbd35a640 1 -- 172.21.15.40:0/234381181 >> v1:172.21.15.40:6789/0 conn(0x7efcb8162810 legacy=0x7efcb8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.714+0000 7efcbd35a640 1 -- 172.21.15.40:0/234381181 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb80d01b0 con 0x7efcb80b2340 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.715+0000 7efcbd35a640 1 --2- 172.21.15.40:0/234381181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80b2340 0x7efcb80cd3f0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7efcb80d1540 tx=0x7efca802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e027cfc72bacb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.715+0000 7efcbcb59640 1 -- 172.21.15.40:0/234381181 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8037070 con 0x7efcb80b2340 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.715+0000 7efcbcb59640 1 -- 172.21.15.40:0/234381181 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca8032440 con 0x7efcb80b2340 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.715+0000 7efcbcb59640 1 -- 172.21.15.40:0/234381181 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8032810 con 0x7efcb80b2340 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.715+0000 7efcbfdbf640 1 -- 172.21.15.40:0/234381181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80b2340 msgr2=0x7efcb80cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.715+0000 7efcbfdbf640 1 --2- 172.21.15.40:0/234381181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80b2340 0x7efcb80cd3f0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7efcb80d1540 tx=0x7efca802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.716+0000 7efcbfdbf640 1 -- 172.21.15.40:0/234381181 shutdown_connections 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.716+0000 7efcbfdbf640 1 --2- 172.21.15.40:0/234381181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80b2340 0x7efcb80cd3f0 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.716+0000 7efcbfdbf640 1 -- 172.21.15.40:0/234381181 >> 172.21.15.40:0/234381181 conn(0x7efcb80d2da0 msgr2=0x7efcb80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.716+0000 7efcbfdbf640 1 -- 172.21.15.40:0/234381181 shutdown_connections 2024-04-26T21:47:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.716+0000 7efcbfdbf640 1 -- 172.21.15.40:0/234381181 wait complete. 2024-04-26T21:47:45.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.717+0000 7efcbfdbf640 1 Processor -- start 2024-04-26T21:47:45.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.717+0000 7efcbfdbf640 1 -- start start 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.717+0000 7efcbfdbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb80b2340 0x7efcb8157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.717+0000 7efcbfdbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80cd930 0x7efcb80c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbfdbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb8162810 0x7efcb81608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbfdbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efcb80d2240 con 0x7efcb80cd930 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbfdbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efcb80d16a0 con 0x7efcb8162810 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbfdbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efcb80d23c0 con 0x7efcb80b2340 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80cd930 0x7efcb80c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80cd930 0x7efcb80c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43004/0 (socket says 172.21.15.40:43004) 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 -- 172.21.15.40:0/445566119 learned_addr learned my addr 172.21.15.40:0/445566119 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbdb5b640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb80b2340 0x7efcb8157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbe35c640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb8162810 0x7efcb81608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 -- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb80b2340 msgr2=0x7efcb8157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb80b2340 0x7efcb8157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 -- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb8162810 msgr2=0x7efcb81608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb8162810 0x7efcb81608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 -- 172.21.15.40:0/445566119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb81f95a0 con 0x7efcb80cd930 2024-04-26T21:47:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbd35a640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80cd930 0x7efcb80c8fc0 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7efca8009890 tx=0x7efca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcae7fc640 1 -- 172.21.15.40:0/445566119 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8041070 con 0x7efcb80cd930 2024-04-26T21:47:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcae7fc640 1 -- 172.21.15.40:0/445566119 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca803c890 con 0x7efcb80cd930 2024-04-26T21:47:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcae7fc640 1 -- 172.21.15.40:0/445566119 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8037040 con 0x7efcb80cd930 2024-04-26T21:47:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb81f8990 con 0x7efcb80cd930 2024-04-26T21:47:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.718+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81f8f50 con 0x7efcb80cd930 2024-04-26T21:47:45.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.719+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb80694f0 con 0x7efcb80cd930 2024-04-26T21:47:45.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.720+0000 7efcae7fc640 1 -- 172.21.15.40:0/445566119 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efca8044070 con 0x7efcb80cd930 2024-04-26T21:47:45.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.720+0000 7efcae7fc640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc9c06c920 0x7efc9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:45.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.721+0000 7efcae7fc640 1 -- 172.21.15.40:0/445566119 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6273+0+0 (secure 0 0 0) 0x7efca80ac9d0 con 0x7efcb80cd930 2024-04-26T21:47:45.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.721+0000 7efcbdb5b640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc9c06c920 0x7efc9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:45.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.721+0000 7efcbdb5b640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc9c06c920 0x7efc9c06ede0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7efcb4000f60 tx=0x7efcb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.725+0000 7efcae7fc640 1 -- 172.21.15.40:0/445566119 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efca807ace0 con 0x7efcb80cd930 2024-04-26T21:47:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:45.949+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efcb80694f0 con 0x7efcb80cd930 2024-04-26T21:47:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.601+0000 7efcae7fc640 1 -- 172.21.15.40:0/445566119 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v365) ==== 104+0+0 (secure 0 0 0) 0x7efca807a5e0 con 0x7efcb80cd930 2024-04-26T21:47:46.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc9c06c920 msgr2=0x7efc9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc9c06c920 0x7efc9c06ede0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7efcb4000f60 tx=0x7efcb4009040 comp rx=0 tx=0).stop 2024-04-26T21:47:46.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80cd930 msgr2=0x7efcb80c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80cd930 0x7efcb80c8fc0 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7efca8009890 tx=0x7efca80046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 shutdown_connections 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7efc9c06c920 0x7efc9c06ede0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb8162810 0x7efcb81608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcb80cd930 0x7efcb80c8fc0 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 --2- 172.21.15.40:0/445566119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb80b2340 0x7efcb8157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.604+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 >> 172.21.15.40:0/445566119 conn(0x7efcb80d2da0 msgr2=0x7efcb80b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.605+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 shutdown_connections 2024-04-26T21:47:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.605+0000 7efcbfdbf640 1 -- 172.21.15.40:0/445566119 wait complete. 2024-04-26T21:47:46.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:47:46.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.773+0000 7f7353e2a640 1 Processor -- start 2024-04-26T21:47:46.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.773+0000 7f7353e2a640 1 -- start start 2024-04-26T21:47:46.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f7353e2a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f734c15f460 0x7f734c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f7353e2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f734c0d0760 0x7f734c0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f7353e2a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f734c15f840 con 0x7f734c0d1a40 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f7353e2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f734c0d41c0 con 0x7f734c0d0760 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f7353e2a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f734c0d37f0 con 0x7f734c15f460 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f73513c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f734c0d0760 0x7f734c0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f73513c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f734c0d0760 0x7f734c0d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43012/0 (socket says 172.21.15.40:43012) 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f73513c5640 1 -- 172.21.15.40:0/228678844 learned_addr learned my addr 172.21.15.40:0/228678844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f7351bc6640 1 --2- 172.21.15.40:0/228678844 >> v2:172.21.15.184:3300/0 conn(0x7f734c15f460 0x7f734c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.774+0000 7f73513c5640 1 -- 172.21.15.40:0/228678844 >> v2:172.21.15.184:3300/0 conn(0x7f734c15f460 msgr2=0x7f734c0d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.775+0000 7f73513c5640 1 --2- 172.21.15.40:0/228678844 >> v2:172.21.15.184:3300/0 conn(0x7f734c15f460 0x7f734c0d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.775+0000 7f73513c5640 1 -- 172.21.15.40:0/228678844 >> v1:172.21.15.184:6789/0 conn(0x7f734c0d1a40 legacy=0x7f734c06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:46.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.775+0000 7f73513c5640 1 -- 172.21.15.40:0/228678844 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c0d2ec0 con 0x7f734c0d0760 2024-04-26T21:47:46.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.775+0000 7f73513c5640 1 --2- 172.21.15.40:0/228678844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c0d0760 0x7f734c0d0bc0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7f734c08f280 tx=0x7f734002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dd0491833d90c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.775+0000 7f7350bc4640 1 -- 172.21.15.40:0/228678844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340037070 con 0x7f734c0d0760 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7350bc4640 1 -- 172.21.15.40:0/228678844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7340032440 con 0x7f734c0d0760 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7350bc4640 1 -- 172.21.15.40:0/228678844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340032810 con 0x7f734c0d0760 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7353e2a640 1 -- 172.21.15.40:0/228678844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c0d0760 msgr2=0x7f734c0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7353e2a640 1 --2- 172.21.15.40:0/228678844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c0d0760 0x7f734c0d0bc0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7f734c08f280 tx=0x7f734002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7353e2a640 1 -- 172.21.15.40:0/228678844 shutdown_connections 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7353e2a640 1 --2- 172.21.15.40:0/228678844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c0d0760 0x7f734c0d0bc0 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7353e2a640 1 --2- 172.21.15.40:0/228678844 >> v2:172.21.15.184:3300/0 conn(0x7f734c15f460 0x7f734c0d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7353e2a640 1 -- 172.21.15.40:0/228678844 >> 172.21.15.40:0/228678844 conn(0x7f734c09e3f0 msgr2=0x7f734c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7353e2a640 1 -- 172.21.15.40:0/228678844 shutdown_connections 2024-04-26T21:47:46.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.776+0000 7f7353e2a640 1 -- 172.21.15.40:0/228678844 wait complete. 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.777+0000 7f7353e2a640 1 Processor -- start 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.777+0000 7f7353e2a640 1 -- start start 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f7353e2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f734c0d0760 0x7f734c0c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f7353e2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f734c0d1a40 0x7f734c158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f7353e2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c15f460 0x7f734c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f7353e2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0d5f30 con 0x7f734c15f460 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f7353e2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0d3480 con 0x7f734c0d0760 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f7353e2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f734c0d1ed0 con 0x7f734c0d1a40 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f73523c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c15f460 0x7f734c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f73523c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c15f460 0x7f734c0ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43026/0 (socket says 172.21.15.40:43026) 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f73523c7640 1 -- 172.21.15.40:0/45812777 learned_addr learned my addr 172.21.15.40:0/45812777 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:46.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f7351bc6640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f734c0d0760 0x7f734c0c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f73513c5640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f734c0d1a40 0x7f734c158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f73523c7640 1 -- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f734c0d1a40 msgr2=0x7f734c158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.778+0000 7f73523c7640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f734c0d1a40 0x7f734c158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f73523c7640 1 -- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f734c0d0760 msgr2=0x7f734c0c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f73523c7640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f734c0d0760 0x7f734c0c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f73523c7640 1 -- 172.21.15.40:0/45812777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c162ab0 con 0x7f734c15f460 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f73513c5640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f734c0d1a40 0x7f734c158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f7351bc6640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f734c0d0760 0x7f734c0c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f73523c7640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c15f460 0x7f734c0ceec0 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f733c00c410 tx=0x7f733c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f734a7fc640 1 -- 172.21.15.40:0/45812777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c011790 con 0x7f734c15f460 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f734a7fc640 1 -- 172.21.15.40:0/45812777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c017070 con 0x7f734c15f460 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c159900 con 0x7f734c15f460 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f734a7fc640 1 -- 172.21.15.40:0/45812777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c011bb0 con 0x7f734c15f460 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.779+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c162210 con 0x7f734c15f460 2024-04-26T21:47:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.780+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c0b3ae0 con 0x7f734c15f460 2024-04-26T21:47:46.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.780+0000 7f734a7fc640 1 -- 172.21.15.40:0/45812777 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f733c01c460 con 0x7f734c15f460 2024-04-26T21:47:46.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.781+0000 7f734a7fc640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f733806c8f0 0x7f733806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:46.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.781+0000 7f734a7fc640 1 -- 172.21.15.40:0/45812777 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6273+0+0 (secure 0 0 0) 0x7f733c08bbb0 con 0x7f734c15f460 2024-04-26T21:47:46.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.781+0000 7f7351bc6640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f733806c8f0 0x7f733806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:46.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.782+0000 7f7351bc6640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f733806c8f0 0x7f733806edb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f734c0b4900 tx=0x7f73440075b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.785+0000 7f734a7fc640 1 -- 172.21.15.40:0/45812777 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f734c0b3ae0 con 0x7f734c15f460 2024-04-26T21:47:46.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:46.969+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f734c0b4240 con 0x7f734c15f460 2024-04-26T21:47:47.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.602+0000 7f734a7fc640 1 -- 172.21.15.40:0/45812777 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) ==== 181+0+0 (secure 0 0 0) 0x7f734c0b4240 con 0x7f734c15f460 2024-04-26T21:47:47.602 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:47:47.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.606+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f733806c8f0 msgr2=0x7f733806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.607+0000 7f7353e2a640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f733806c8f0 0x7f733806edb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f734c0b4900 tx=0x7f73440075b0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.607+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c15f460 msgr2=0x7f734c0ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.607+0000 7f7353e2a640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c15f460 0x7f734c0ceec0 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f733c00c410 tx=0x7f733c00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.607+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 shutdown_connections 2024-04-26T21:47:47.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.607+0000 7f7353e2a640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f733806c8f0 0x7f733806edb0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.607+0000 7f7353e2a640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f734c15f460 0x7f734c0ceec0 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.607+0000 7f7353e2a640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f734c0d1a40 0x7f734c158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.608+0000 7f7353e2a640 1 --2- 172.21.15.40:0/45812777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f734c0d0760 0x7f734c0c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.608+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 >> 172.21.15.40:0/45812777 conn(0x7f734c09e3f0 msgr2=0x7f734c0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:47.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.608+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 shutdown_connections 2024-04-26T21:47:47.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.608+0000 7f7353e2a640 1 -- 172.21.15.40:0/45812777 wait complete. 2024-04-26T21:47:47.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:47:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.734+0000 7f4f9edc6640 1 Processor -- start 2024-04-26T21:47:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.734+0000 7f4f9edc6640 1 -- start start 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f9edc6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 0x7f4f98166d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f9edc6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f980cfd70 con 0x7f4f9815f980 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f9edc6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f980d0050 con 0x7f4f98145630 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f9edc6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f980d0330 con 0x7f4f980b3190 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f97fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4f9815f980 0x7f4f980b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42394/0 (socket says 172.21.15.40:42394) 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f97fff640 1 -- 172.21.15.40:0/2790069682 learned_addr learned my addr 172.21.15.40:0/2790069682 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f9d363640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 0x7f4f98166d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736568925 0 0) 0x7f4f980cfd70 con 0x7f4f9815f980 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.735+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f84003520 con 0x7f4f9815f980 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 37108585 0 0) 0x7f4f84003520 con 0x7f4f9815f980 2024-04-26T21:47:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 msgr2=0x7f4f98166d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 0x7f4f98166d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 >> v1:172.21.15.184:6789/0 conn(0x7f4f98145630 legacy=0x7f4f980cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f980d0610 con 0x7f4f9815f980 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f4f88001670 con 0x7f4f9815f980 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 >> v1:172.21.15.40:6789/0 conn(0x7f4f9815f980 legacy=0x7f4f980b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 0x7f4f840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f84004d40 0x7f4f840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f980d0330 con 0x7f4f84003f20 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8400ba20 con 0x7f4f84004d40 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f8400bc20 con 0x7f4f980b3190 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f97fff640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f9cb62640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 0x7f4f840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f9d363640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f84004d40 0x7f4f840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f97fff640 1 -- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 msgr2=0x7f4f840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f97fff640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 0x7f4f840039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f97fff640 1 -- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f84004d40 msgr2=0x7f4f840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f97fff640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f84004d40 0x7f4f840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f97fff640 1 -- 172.21.15.40:0/2790069682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f84003520 con 0x7f4f84003f20 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f9cb62640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f980b3190 0x7f4f840039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.736+0000 7f4f9d363640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f84004d40 0x7f4f840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.737+0000 7f4f97fff640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f84004800 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f4f88007fd0 tx=0x7f4f84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be07b9ade2428669 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.737+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88007ca0 con 0x7f4f84003f20 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.737+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f88005ad0 con 0x7f4f84003f20 2024-04-26T21:47:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.737+0000 7f4f977fe640 1 -- 172.21.15.40:0/2790069682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88007070 con 0x7f4f84003f20 2024-04-26T21:47:47.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.738+0000 7f4f9edc6640 1 -- 172.21.15.40:0/2790069682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 msgr2=0x7f4f84004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.738+0000 7f4f9edc6640 1 --2- 172.21.15.40:0/2790069682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f84004800 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f4f88007fd0 tx=0x7f4f84005830 comp rx=0 tx=0).stop 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.738+0000 7f4f9d363640 1 -- 172.21.15.40:0/2790069682 reap_dead start 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.738+0000 7f4f9edc6640 1 -- 172.21.15.40:0/2790069682 shutdown_connections 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.738+0000 7f4f9edc6640 1 -- 172.21.15.40:0/2790069682 >> 172.21.15.40:0/2790069682 conn(0x7f4f9806f310 msgr2=0x7f4f9806f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.738+0000 7f4f9edc6640 1 -- 172.21.15.40:0/2790069682 shutdown_connections 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.738+0000 7f4f9edc6640 1 -- 172.21.15.40:0/2790069682 wait complete. 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.739+0000 7f4f9edc6640 1 Processor -- start 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.739+0000 7f4f9edc6640 1 -- start start 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9edc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f9815ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9edc6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9813b740 0x7f4f98155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9edc6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f98151cf0 0x7f4f98156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9edc6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f98145b90 con 0x7f4f84003f20 2024-04-26T21:47:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9edc6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f98145a10 con 0x7f4f9813b740 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9edc6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f980b4f00 con 0x7f4f98151cf0 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f9815ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f9815ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43044/0 (socket says 172.21.15.40:43044) 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 -- 172.21.15.40:0/1508704661 learned_addr learned my addr 172.21.15.40:0/1508704661 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 -- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f98151cf0 msgr2=0x7f4f98156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f98151cf0 0x7f4f98156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 -- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9813b740 msgr2=0x7f4f98155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9813b740 0x7f4f98155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 -- 172.21.15.40:0/1508704661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9815c560 con 0x7f4f84003f20 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.740+0000 7f4f9cb62640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f9815ee00 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000eba0 tx=0x7f4f9000c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.741+0000 7f4f957fa640 1 -- 172.21.15.40:0/1508704661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9000cb40 con 0x7f4f84003f20 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.741+0000 7f4f957fa640 1 -- 172.21.15.40:0/1508704661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f90018070 con 0x7f4f84003f20 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.741+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9815c840 con 0x7f4f84003f20 2024-04-26T21:47:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.741+0000 7f4f957fa640 1 -- 172.21.15.40:0/1508704661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90003970 con 0x7f4f84003f20 2024-04-26T21:47:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.741+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9815ce00 con 0x7f4f84003f20 2024-04-26T21:47:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.742+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f9815d000 con 0x7f4f84003f20 2024-04-26T21:47:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.742+0000 7f4f957fa640 1 -- 172.21.15.40:0/1508704661 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f90003b10 con 0x7f4f84003f20 2024-04-26T21:47:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.743+0000 7f4f957fa640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.743+0000 7f4f957fa640 1 -- 172.21.15.40:0/1508704661 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5835+0+0 (secure 0 0 0) 0x7f4f9008be60 con 0x7f4f84003f20 2024-04-26T21:47:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.743+0000 7f4f97fff640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.743+0000 7f4f97fff640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f7806c920 0x7f4f7806ede0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4f88032000 tx=0x7f4f88007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.747+0000 7f4f957fa640 1 -- 172.21.15.40:0/1508704661 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f9815d000 con 0x7f4f84003f20 2024-04-26T21:47:47.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:47.936+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4f9813bb80 con 0x7f4f84003f20 2024-04-26T21:47:48.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.610+0000 7f4f957fa640 1 -- 172.21.15.40:0/1508704661 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) ==== 169+0+0 (secure 0 0 0) 0x7f4f9005a200 con 0x7f4f84003f20 2024-04-26T21:47:48.610 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:47:48.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.615+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f7806c920 msgr2=0x7f4f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.615+0000 7f4f9edc6640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f7806c920 0x7f4f7806ede0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4f88032000 tx=0x7f4f88007000 comp rx=0 tx=0).stop 2024-04-26T21:47:48.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.615+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 msgr2=0x7f4f9815ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.615+0000 7f4f9edc6640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f9815ee00 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000eba0 tx=0x7f4f9000c4b0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.616+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 shutdown_connections 2024-04-26T21:47:48.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.616+0000 7f4f9edc6640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.616+0000 7f4f9edc6640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f98151cf0 0x7f4f98156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.616+0000 7f4f9edc6640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9813b740 0x7f4f98155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.616+0000 7f4f9edc6640 1 --2- 172.21.15.40:0/1508704661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f84003f20 0x7f4f9815ee00 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.616+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 >> 172.21.15.40:0/1508704661 conn(0x7f4f9806f310 msgr2=0x7f4f98141630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.618+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 shutdown_connections 2024-04-26T21:47:48.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.618+0000 7f4f9edc6640 1 -- 172.21.15.40:0/1508704661 wait complete. 2024-04-26T21:47:48.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:47:48.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.743+0000 7fd4ee104640 1 Processor -- start 2024-04-26T21:47:48.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.743+0000 7fd4ee104640 1 -- start start 2024-04-26T21:47:48.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4ee104640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 0x7fd4e80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4ee104640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd4e80d3bc0 0x7fd4e80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4ee104640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4e80b7e50 con 0x7fd4e80b5f20 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4ee104640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd4e80d0580 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4ee104640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e80d1800 con 0x7fd4e8162880 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4e7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd4e80d3bc0 0x7fd4e80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4e7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd4e80d3bc0 0x7fd4e80b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50898/0 (socket says 172.21.15.40:50898) 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4e7fff640 1 -- 172.21.15.40:0/3730247202 learned_addr learned my addr 172.21.15.40:0/3730247202 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.744+0000 7fd4e77fe640 1 --2- 172.21.15.40:0/3730247202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 0x7fd4e80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.745+0000 7fd4e7fff640 1 -- 172.21.15.40:0/3730247202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 msgr2=0x7fd4e80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.745+0000 7fd4e7fff640 1 --2- 172.21.15.40:0/3730247202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 0x7fd4e80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.745+0000 7fd4e7fff640 1 -- 172.21.15.40:0/3730247202 >> v1:172.21.15.184:6789/0 conn(0x7fd4e80b5f20 legacy=0x7fd4e80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:48.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.745+0000 7fd4e7fff640 1 -- 172.21.15.40:0/3730247202 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e80d07d0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.745+0000 7fd4e7fff640 1 --2- 172.21.15.40:0/3730247202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80b8300 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009e50 tx=0x7fd4e002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcc371451392459 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.745+0000 7fd4e67fc640 1 -- 172.21.15.40:0/3730247202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0036070 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.745+0000 7fd4e67fc640 1 -- 172.21.15.40:0/3730247202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e0031600 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.745+0000 7fd4e67fc640 1 -- 172.21.15.40:0/3730247202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00319d0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.746+0000 7fd4ee104640 1 -- 172.21.15.40:0/3730247202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 msgr2=0x7fd4e80b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.746+0000 7fd4ee104640 1 --2- 172.21.15.40:0/3730247202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80b8300 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009e50 tx=0x7fd4e002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.746+0000 7fd4ee104640 1 -- 172.21.15.40:0/3730247202 shutdown_connections 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.746+0000 7fd4ee104640 1 --2- 172.21.15.40:0/3730247202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80b8300 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.746+0000 7fd4ee104640 1 --2- 172.21.15.40:0/3730247202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 0x7fd4e80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.746+0000 7fd4ee104640 1 -- 172.21.15.40:0/3730247202 >> 172.21.15.40:0/3730247202 conn(0x7fd4e80d4f60 msgr2=0x7fd4e80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.746+0000 7fd4ee104640 1 -- 172.21.15.40:0/3730247202 shutdown_connections 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.746+0000 7fd4ee104640 1 -- 172.21.15.40:0/3730247202 wait complete. 2024-04-26T21:47:48.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.747+0000 7fd4ee104640 1 Processor -- start 2024-04-26T21:47:48.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4ee104640 1 -- start start 2024-04-26T21:47:48.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4ee104640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e80b5f20 0x7fd4e8159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4ee104640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4ee104640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 0x7fd4e806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4ee104640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8162c60 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4ee104640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e80d0d90 con 0x7fd4e80b5f20 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4ee104640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e8162de0 con 0x7fd4e8162880 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50914/0 (socket says 172.21.15.40:50914) 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.748+0000 7fd4e6ffd640 1 -- 172.21.15.40:0/4105795271 learned_addr learned my addr 172.21.15.40:0/4105795271 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4e77fe640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e80b5f20 0x7fd4e8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4e7fff640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 0x7fd4e806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4e6ffd640 1 -- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 msgr2=0x7fd4e806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4e6ffd640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 0x7fd4e806cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4e6ffd640 1 -- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e80b5f20 msgr2=0x7fd4e8159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4e6ffd640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e80b5f20 0x7fd4e8159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4e6ffd640 1 -- 172.21.15.40:0/4105795271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e80ceba0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4e6ffd640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80cd9c0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c430 tx=0x7fd4dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:48.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4cffff640 1 -- 172.21.15.40:0/4105795271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc011790 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4cffff640 1 -- 172.21.15.40:0/4105795271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4dc017070 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4cffff640 1 -- 172.21.15.40:0/4105795271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc011bb0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e8159650 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.749+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e80ce5e0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.750+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e80694f0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.751+0000 7fd4cffff640 1 -- 172.21.15.40:0/4105795271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4dc01c460 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.751+0000 7fd4cffff640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4d406c8f0 0x7fd4d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:48.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.751+0000 7fd4cffff640 1 -- 172.21.15.40:0/4105795271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fd4dc08b8e0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.751+0000 7fd4e77fe640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4d406c8f0 0x7fd4d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:48.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.752+0000 7fd4e77fe640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4d406c8f0 0x7fd4d406edb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006fd0 tx=0x7fd4d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:48.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.755+0000 7fd4cffff640 1 -- 172.21.15.40:0/4105795271 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4dc059ea0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.939+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd4e8074590 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.942+0000 7fd4cffff640 1 -- 172.21.15.40:0/4105795271 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fd4dc0597a0 con 0x7fd4e80d3bc0 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4d406c8f0 msgr2=0x7fd4d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4d406c8f0 0x7fd4d406edb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006fd0 tx=0x7fd4d8008040 comp rx=0 tx=0).stop 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 msgr2=0x7fd4e80cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80cd9c0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c430 tx=0x7fd4dc00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 shutdown_connections 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fd4d406c8f0 0x7fd4d406edb0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e8162880 0x7fd4e806cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4e80d3bc0 0x7fd4e80cd9c0 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 --2- 172.21.15.40:0/4105795271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e80b5f20 0x7fd4e8159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.944+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 >> 172.21.15.40:0/4105795271 conn(0x7fd4e80d4f60 msgr2=0x7fd4e80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:48.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.945+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 shutdown_connections 2024-04-26T21:47:48.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:48.945+0000 7fd4ee104640 1 -- 172.21.15.40:0/4105795271 wait complete. 2024-04-26T21:47:48.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-04-26T21:47:49.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.056+0000 7f6420933640 1 Processor -- start 2024-04-26T21:47:49.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.056+0000 7f6420933640 1 -- start start 2024-04-26T21:47:49.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.056+0000 7f6420933640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f641c0d1b30 0x7f641c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.056+0000 7f6420933640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641c0d10d0 con 0x7f641c162810 2024-04-26T21:47:49.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6420933640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641c0d1370 con 0x7f641c0b2340 2024-04-26T21:47:49.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6420933640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f641c0d0750 con 0x7f641c0d1b30 2024-04-26T21:47:49.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f641a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f641c162810 0x7f641c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54486/0 (socket says 172.21.15.40:54486) 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f641a59c640 1 -- 172.21.15.40:0/11495643 learned_addr learned my addr 172.21.15.40:0/11495643 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226339316 0 0) 0x7f641c0d10d0 con 0x7f641c162810 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6408003520 con 0x7f641c162810 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f641ad9d640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f641c0d1b30 0x7f641c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3853746042 0 0) 0x7f6408003520 con 0x7f641c162810 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f641c0d1b30 msgr2=0x7f641c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f641c0d1b30 0x7f641c0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 >> v1:172.21.15.184:6789/0 conn(0x7f641c0b2340 legacy=0x7f641c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c0d01b0 con 0x7f641c162810 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f640c001670 con 0x7f641c162810 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 >> v1:172.21.15.40:6789/0 conn(0x7f641c162810 legacy=0x7f641c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f641c0d1b30 0x7f64080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6408003f60 0x7f6408004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6408004d40 0x7f64080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f641c0d0750 con 0x7f6408004d40 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f640800baa0 con 0x7f6408003f60 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f640800bca0 con 0x7f641c0d1b30 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.057+0000 7f641ad9d640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6408004d40 0x7f64080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.058+0000 7f6413fff640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6408003f60 0x7f6408004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.058+0000 7f641a59c640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f641c0d1b30 0x7f64080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.058+0000 7f641ad9d640 1 -- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f641c0d1b30 msgr2=0x7f64080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.058+0000 7f641ad9d640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f641c0d1b30 0x7f64080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.058+0000 7f641ad9d640 1 -- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6408003f60 msgr2=0x7f6408004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.058+0000 7f641ad9d640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6408003f60 0x7f6408004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.058+0000 7f641ad9d640 1 -- 172.21.15.40:0/11495643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6408003520 con 0x7f6408004d40 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.058+0000 7f641ad9d640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6408004d40 0x7f64080051c0 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f6414009fc0 tx=0x7f6408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7a8f7070d4bc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414009c10 con 0x7f6408004d40 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6414036070 con 0x7f6408004d40 2024-04-26T21:47:49.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6419d9b640 1 -- 172.21.15.40:0/11495643 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414031880 con 0x7f6408004d40 2024-04-26T21:47:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6420933640 1 -- 172.21.15.40:0/11495643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6408004d40 msgr2=0x7f64080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6420933640 1 --2- 172.21.15.40:0/11495643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6408004d40 0x7f64080051c0 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f6414009fc0 tx=0x7f6408005830 comp rx=0 tx=0).stop 2024-04-26T21:47:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f641ad9d640 1 -- 172.21.15.40:0/11495643 reap_dead start 2024-04-26T21:47:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6420933640 1 -- 172.21.15.40:0/11495643 shutdown_connections 2024-04-26T21:47:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6420933640 1 -- 172.21.15.40:0/11495643 >> 172.21.15.40:0/11495643 conn(0x7f641c0d2da0 msgr2=0x7f641c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6420933640 1 -- 172.21.15.40:0/11495643 shutdown_connections 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.059+0000 7f6420933640 1 -- 172.21.15.40:0/11495643 wait complete. 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6420933640 1 Processor -- start 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6420933640 1 -- start start 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6420933640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6408004d40 0x7f641c160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6420933640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f641c152f30 0x7f641c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6420933640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f641c0cbdb0 0x7f641c0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6420933640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f641c0cdb00 con 0x7f641c152f30 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6420933640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f641c0d16a0 con 0x7f641c0cbdb0 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6420933640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f641c0d1fc0 con 0x7f6408004d40 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6413fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f641c152f30 0x7f641c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6413fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f641c152f30 0x7f641c0cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50928/0 (socket says 172.21.15.40:50928) 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.061+0000 7f6413fff640 1 -- 172.21.15.40:0/2742870824 learned_addr learned my addr 172.21.15.40:0/2742870824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f641a59c640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6408004d40 0x7f641c160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f641ad9d640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f641c0cbdb0 0x7f641c0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6413fff640 1 -- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6408004d40 msgr2=0x7f641c160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6413fff640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6408004d40 0x7f641c160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6413fff640 1 -- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f641c0cbdb0 msgr2=0x7f641c0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6413fff640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f641c0cbdb0 0x7f641c0cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6413fff640 1 -- 172.21.15.40:0/2742870824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c160d90 con 0x7f641c152f30 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f641a59c640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6408004d40 0x7f641c160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6413fff640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f641c152f30 0x7f641c0cb870 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f640400f850 tx=0x7f640400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6412ffd640 1 -- 172.21.15.40:0/2742870824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640400fe40 con 0x7f641c152f30 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6412ffd640 1 -- 172.21.15.40:0/2742870824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6404018070 con 0x7f641c152f30 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6412ffd640 1 -- 172.21.15.40:0/2742870824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640400d730 con 0x7f641c152f30 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c0ca010 con 0x7f641c152f30 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.062+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c160980 con 0x7f641c152f30 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.063+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641c0d22f0 con 0x7f641c152f30 2024-04-26T21:47:49.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.064+0000 7f6412ffd640 1 -- 172.21.15.40:0/2742870824 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f640400d8d0 con 0x7f641c152f30 2024-04-26T21:47:49.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.064+0000 7f6412ffd640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:49.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.064+0000 7f6412ffd640 1 -- 172.21.15.40:0/2742870824 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f640408b910 con 0x7f641c152f30 2024-04-26T21:47:49.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.064+0000 7f641a59c640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:49.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.065+0000 7f641a59c640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63fc06c8f0 0x7f63fc06edb0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f640c001eb0 tx=0x7f640c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:49.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.068+0000 7f6412ffd640 1 -- 172.21.15.40:0/2742870824 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f641c0d22f0 con 0x7f641c152f30 2024-04-26T21:47:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.327+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7f641c06ee30 con 0x7f641c152f30 2024-04-26T21:47:49.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.611+0000 7f6412ffd640 1 -- 172.21.15.40:0/2742870824 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v676) ==== 135+0+0 (secure 0 0 0) 0x7f641c06ee30 con 0x7f641c152f30 2024-04-26T21:47:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63fc06c8f0 msgr2=0x7f63fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63fc06c8f0 0x7f63fc06edb0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f640c001eb0 tx=0x7f640c009040 comp rx=0 tx=0).stop 2024-04-26T21:47:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f641c152f30 msgr2=0x7f641c0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:49.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f641c152f30 0x7f641c0cb870 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f640400f850 tx=0x7f640400fd10 comp rx=0 tx=0).stop 2024-04-26T21:47:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 shutdown_connections 2024-04-26T21:47:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f641c0cbdb0 0x7f641c0cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f641c152f30 0x7f641c0cb870 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 --2- 172.21.15.40:0/2742870824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6408004d40 0x7f641c160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 >> 172.21.15.40:0/2742870824 conn(0x7f641c0d2da0 msgr2=0x7f641c070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 shutdown_connections 2024-04-26T21:47:49.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:49.614+0000 7f6420933640 1 -- 172.21.15.40:0/2742870824 wait complete. 2024-04-26T21:47:49.626 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated ... ok 2024-04-26T21:47:49.626 INFO:tasks.cephfs_test_runner: 2024-04-26T21:47:49.626 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T21:47:49.626 INFO:tasks.cephfs_test_runner:Ran 6 tests in 712.029s 2024-04-26T21:47:49.627 INFO:tasks.cephfs_test_runner: 2024-04-26T21:47:49.627 INFO:tasks.cephfs_test_runner:OK 2024-04-26T21:47:49.627 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T21:47:49.641 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T21:47:49.662 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T21:47:49.662 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:49.662 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:49.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:49.679 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:47:49.679 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:49.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=502 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:47:49.735 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:47:49.735 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:49.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:49.754 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:47:49.754 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:47:49.811 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2601 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:47:49.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:49.812 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:47:49.827 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:47:49.827 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:47:49.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:49.827 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:47:49.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:49.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:47:49.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:49.957 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-26T21:47:50.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:50.070 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:47:50.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:50.086 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-26T21:47:50.154 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-04-26T21:47:50.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:47:50.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:50.155 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:47:50.174 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:47:50.174 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:47:50.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:50.175 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:47:50.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:50.242 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:47:50.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:50.310 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-04-26T21:47:50.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:50.405 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:47:50.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:47:50.420 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-04-26T21:47:50.487 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-04-26T21:47:50.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:47:50.488 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T21:47:50.497 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:47:50.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.603+0000 7fcb642c8640 1 Processor -- start 2024-04-26T21:47:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.603+0000 7fcb642c8640 1 -- start start 2024-04-26T21:47:50.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb642c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c156460 0x7fcb5c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb642c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5c156840 con 0x7fcb5c06fea0 2024-04-26T21:47:50.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb642c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5c0cbf20 con 0x7fcb5c0cba10 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb642c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5c0cc200 con 0x7fcb5c156460 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb62865640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcb5c06fea0 0x7fcb5c072c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54498/0 (socket says 172.21.15.40:54498) 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb62865640 1 -- 172.21.15.40:0/1866130479 learned_addr learned my addr 172.21.15.40:0/1866130479 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751295305 0 0) 0x7fcb5c156840 con 0x7fcb5c06fea0 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb62064640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c156460 0x7fcb5c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb48003520 con 0x7fcb5c06fea0 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3092975125 0 0) 0x7fcb48003520 con 0x7fcb5c06fea0 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c156460 msgr2=0x7fcb5c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c156460 0x7fcb5c0cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 >> v1:172.21.15.184:6789/0 conn(0x7fcb5c0cba10 legacy=0x7fcb5c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c0cc4e0 con 0x7fcb5c06fea0 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fcb540019f0 con 0x7fcb5c06fea0 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 >> v1:172.21.15.40:6789/0 conn(0x7fcb5c06fea0 legacy=0x7fcb5c072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb480039e0 0x7fcb48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c156460 0x7fcb480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c0cc200 con 0x7fcb48004800 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4800ba70 con 0x7fcb480039e0 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.604+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4800bc70 con 0x7fcb5c156460 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.605+0000 7fcb61863640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.605+0000 7fcb62064640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb480039e0 0x7fcb48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.605+0000 7fcb62865640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c156460 0x7fcb480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.605+0000 7fcb61863640 1 -- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c156460 msgr2=0x7fcb480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.605+0000 7fcb61863640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c156460 0x7fcb480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.605+0000 7fcb61863640 1 -- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb480039e0 msgr2=0x7fcb48003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.605+0000 7fcb61863640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb480039e0 0x7fcb48003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.605+0000 7fcb61863640 1 -- 172.21.15.40:0/1866130479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb48003520 con 0x7fcb48004800 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb61863640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb48004c80 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0cbe50 tx=0x7fcb48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88623f69b30e6ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c001370 con 0x7fcb48004800 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4c0052d0 con 0x7fcb48004800 2024-04-26T21:47:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb61062640 1 -- 172.21.15.40:0/1866130479 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c02ab70 con 0x7fcb48004800 2024-04-26T21:47:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb642c8640 1 -- 172.21.15.40:0/1866130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 msgr2=0x7fcb48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb642c8640 1 --2- 172.21.15.40:0/1866130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb48004c80 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0cbe50 tx=0x7fcb48005830 comp rx=0 tx=0).stop 2024-04-26T21:47:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb62865640 1 -- 172.21.15.40:0/1866130479 reap_dead start 2024-04-26T21:47:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb642c8640 1 -- 172.21.15.40:0/1866130479 shutdown_connections 2024-04-26T21:47:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb642c8640 1 -- 172.21.15.40:0/1866130479 >> 172.21.15.40:0/1866130479 conn(0x7fcb5c0d0be0 msgr2=0x7fcb5c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb642c8640 1 -- 172.21.15.40:0/1866130479 shutdown_connections 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.606+0000 7fcb642c8640 1 -- 172.21.15.40:0/1866130479 wait complete. 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb642c8640 1 Processor -- start 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb642c8640 1 -- start start 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb642c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb5c0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb642c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c0c4860 0x7fcb5c0c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb642c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c06e680 0x7fcb5c06eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb642c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c06e260 con 0x7fcb48004800 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb642c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c072700 con 0x7fcb5c06e680 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb642c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5c06e3e0 con 0x7fcb5c0c4860 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.608+0000 7fcb62064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb5c0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb5c0b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50952/0 (socket says 172.21.15.40:50952) 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62064640 1 -- 172.21.15.40:0/663861840 learned_addr learned my addr 172.21.15.40:0/663861840 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62865640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c06e680 0x7fcb5c06eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb61863640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c0c4860 0x7fcb5c0c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62064640 1 -- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c0c4860 msgr2=0x7fcb5c0c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62064640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c0c4860 0x7fcb5c0c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62064640 1 -- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c06e680 msgr2=0x7fcb5c06eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62064640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c06e680 0x7fcb5c06eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62064640 1 -- 172.21.15.40:0/663861840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c0c5590 con 0x7fcb48004800 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62865640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c06e680 0x7fcb5c06eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb61863640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c0c4860 0x7fcb5c0c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb62064640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb5c0b5ce0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7fcb580119a0 tx=0x7fcb58011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb52ffd640 1 -- 172.21.15.40:0/663861840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800faf0 con 0x7fcb48004800 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb52ffd640 1 -- 172.21.15.40:0/663861840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5800fc90 con 0x7fcb48004800 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c0c5870 con 0x7fcb48004800 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb52ffd640 1 -- 172.21.15.40:0/663861840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58014440 con 0x7fcb48004800 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.609+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c0c6110 con 0x7fcb48004800 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.610+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5c0694f0 con 0x7fcb48004800 2024-04-26T21:47:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.611+0000 7fcb52ffd640 1 -- 172.21.15.40:0/663861840 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb58012040 con 0x7fcb48004800 2024-04-26T21:47:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.611+0000 7fcb52ffd640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.611+0000 7fcb52ffd640 1 -- 172.21.15.40:0/663861840 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fcb5808c120 con 0x7fcb48004800 2024-04-26T21:47:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.611+0000 7fcb61863640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.612+0000 7fcb61863640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb4006c8f0 0x7fcb4006edb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c032000 tx=0x7fcb4c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.616+0000 7fcb52ffd640 1 -- 172.21.15.40:0/663861840 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb58019030 con 0x7fcb48004800 2024-04-26T21:47:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.798+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb5c06de80 con 0x7fcb4006c8f0 2024-04-26T21:47:50.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.800+0000 7fcb52ffd640 1 -- 172.21.15.40:0/663861840 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13514 (secure 0 0 0) 0x7fcb5c06de80 con 0x7fcb4006c8f0 2024-04-26T21:47:50.801 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:50.801 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb4006c8f0 msgr2=0x7fcb4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb4006c8f0 0x7fcb4006edb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c032000 tx=0x7fcb4c02e000 comp rx=0 tx=0).stop 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 msgr2=0x7fcb5c0b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb5c0b5ce0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7fcb580119a0 tx=0x7fcb58011e60 comp rx=0 tx=0).stop 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 shutdown_connections 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb5c06e680 0x7fcb5c06eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb5c0c4860 0x7fcb5c0c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 --2- 172.21.15.40:0/663861840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb48004800 0x7fcb5c0b5ce0 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 >> 172.21.15.40:0/663861840 conn(0x7fcb5c0d0be0 msgr2=0x7fcb5c0b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 shutdown_connections 2024-04-26T21:47:50.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.802+0000 7fcb642c8640 1 -- 172.21.15.40:0/663861840 wait complete. 2024-04-26T21:47:50.813 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2024-04-26T21:47:50.187692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294720,"kb_used_data":16144,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680000,"statfs":{"total":773094113280,"available":772792320000,"internally_reserved":0,"allocated":16531456,"data_stored":8956940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36286,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-102,"num_write_kb":-72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001763"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T21:47:47.770766+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:47:47.770766+0000","last_peered":"2024-04-26T21:47:47.770766+0000","last_clean":"2024-04-26T21:47:47.770766+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:47:47.770766+0000","last_undegraded":"2024-04-26T21:47:47.770766+0000","last_fullsized":"2024-04-26T21:47:47.770766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37796,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334044,"statfs":{"total":96636764160,"available":96598061056,"internally_reserved":0,"allocated":2510848,"data_stored":1561034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575006,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41412,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330428,"statfs":{"total":96636764160,"available":96594358272,"internally_reserved":0,"allocated":1957888,"data_stored":1000261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":970666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1908736,"data_stored":964277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1904640,"data_stored":964767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37216,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1916928,"data_stored":970666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":1904640,"data_stored":964235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575006,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41956,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329884,"statfs":{"total":96636764160,"available":96593801216,"internally_reserved":0,"allocated":2510848,"data_stored":1561034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:50.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:50.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.913+0000 7fa5e170a640 1 Processor -- start 2024-04-26T21:47:50.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.913+0000 7fa5e170a640 1 -- start start 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.914+0000 7fa5e170a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5dc15f3a0 0x7fa5dc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.914+0000 7fa5e170a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5dc0d1770 0x7fa5dc0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.914+0000 7fa5e170a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5dc0d2590 con 0x7fa5dc0d2160 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.914+0000 7fa5e170a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa5dc0d0e20 con 0x7fa5dc15f3a0 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.914+0000 7fa5e170a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa5dc0b1d70 con 0x7fa5dc0d1770 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.914+0000 7fa5daffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5dc15f3a0 0x7fa5dc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.914+0000 7fa5daffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5dc15f3a0 0x7fa5dc0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50966/0 (socket says 172.21.15.40:50966) 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.914+0000 7fa5daffd640 1 -- 172.21.15.40:0/427425244 learned_addr learned my addr 172.21.15.40:0/427425244 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:50.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.915+0000 7fa5d3fff640 1 --2- 172.21.15.40:0/427425244 >> v2:172.21.15.184:3300/0 conn(0x7fa5dc0d1770 0x7fa5dc0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.915+0000 7fa5daffd640 1 -- 172.21.15.40:0/427425244 >> v2:172.21.15.184:3300/0 conn(0x7fa5dc0d1770 msgr2=0x7fa5dc0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:50.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.915+0000 7fa5daffd640 1 --2- 172.21.15.40:0/427425244 >> v2:172.21.15.184:3300/0 conn(0x7fa5dc0d1770 0x7fa5dc0d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.915+0000 7fa5daffd640 1 -- 172.21.15.40:0/427425244 >> v1:172.21.15.184:6789/0 conn(0x7fa5dc0d2160 legacy=0x7fa5dc0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.915+0000 7fa5daffd640 1 -- 172.21.15.40:0/427425244 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc0b2050 con 0x7fa5dc15f3a0 2024-04-26T21:47:50.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.915+0000 7fa5d3fff640 1 --2- 172.21.15.40:0/427425244 >> v2:172.21.15.184:3300/0 conn(0x7fa5dc0d1770 0x7fa5dc0d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.915+0000 7fa5daffd640 1 --2- 172.21.15.40:0/427425244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc15f3a0 0x7fa5dc0d06f0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0b7b80 tx=0x7fa5cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecfddde46a16c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5da7fc640 1 -- 172.21.15.40:0/427425244 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc037070 con 0x7fa5dc15f3a0 2024-04-26T21:47:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5da7fc640 1 -- 172.21.15.40:0/427425244 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5cc032440 con 0x7fa5dc15f3a0 2024-04-26T21:47:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5da7fc640 1 -- 172.21.15.40:0/427425244 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc032810 con 0x7fa5dc15f3a0 2024-04-26T21:47:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5e170a640 1 -- 172.21.15.40:0/427425244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc15f3a0 msgr2=0x7fa5dc0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5e170a640 1 --2- 172.21.15.40:0/427425244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc15f3a0 0x7fa5dc0d06f0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0b7b80 tx=0x7fa5cc02a270 comp rx=0 tx=0).stop 2024-04-26T21:47:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5e170a640 1 -- 172.21.15.40:0/427425244 shutdown_connections 2024-04-26T21:47:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5e170a640 1 --2- 172.21.15.40:0/427425244 >> v2:172.21.15.184:3300/0 conn(0x7fa5dc0d1770 0x7fa5dc0d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5e170a640 1 --2- 172.21.15.40:0/427425244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc15f3a0 0x7fa5dc0d06f0 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5e170a640 1 -- 172.21.15.40:0/427425244 >> 172.21.15.40:0/427425244 conn(0x7fa5dc0d4f90 msgr2=0x7fa5dc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5e170a640 1 -- 172.21.15.40:0/427425244 shutdown_connections 2024-04-26T21:47:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.916+0000 7fa5e170a640 1 -- 172.21.15.40:0/427425244 wait complete. 2024-04-26T21:47:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5e170a640 1 Processor -- start 2024-04-26T21:47:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5e170a640 1 -- start start 2024-04-26T21:47:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5e170a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc0d1770 0x7fa5dc1f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5e170a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc0d2160 0x7fa5dc1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5e170a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5dc15f3a0 0x7fa5dc0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5e170a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc15f900 con 0x7fa5dc0d1770 2024-04-26T21:47:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5e170a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc15f780 con 0x7fa5dc15f3a0 2024-04-26T21:47:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5e170a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5dc0b3860 con 0x7fa5dc0d2160 2024-04-26T21:47:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc0d1770 0x7fa5dc1f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc0d1770 0x7fa5dc1f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50982/0 (socket says 172.21.15.40:50982) 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.918+0000 7fa5daffd640 1 -- 172.21.15.40:0/3545189434 learned_addr learned my addr 172.21.15.40:0/3545189434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5d3fff640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc0d2160 0x7fa5dc1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5db7fe640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5dc15f3a0 0x7fa5dc0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5daffd640 1 -- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc0d2160 msgr2=0x7fa5dc1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5daffd640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc0d2160 0x7fa5dc1f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5daffd640 1 -- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5dc15f3a0 msgr2=0x7fa5dc0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5daffd640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5dc15f3a0 0x7fa5dc0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5daffd640 1 -- 172.21.15.40:0/3545189434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc1f5630 con 0x7fa5dc0d1770 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5db7fe640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5dc15f3a0 0x7fa5dc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5daffd640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc0d1770 0x7fa5dc1f6950 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc02a2b0 tx=0x7fa5cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5d37fe640 1 -- 172.21.15.40:0/3545189434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc041020 con 0x7fa5dc0d1770 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5d37fe640 1 -- 172.21.15.40:0/3545189434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0306c0 con 0x7fa5dc0d1770 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5d37fe640 1 -- 172.21.15.40:0/3545189434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc037040 con 0x7fa5dc0d1770 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc1f65a0 con 0x7fa5dc0d1770 2024-04-26T21:47:50.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.919+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc0b8070 con 0x7fa5dc0d1770 2024-04-26T21:47:50.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.920+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc06d9b0 con 0x7fa5dc0d1770 2024-04-26T21:47:50.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.921+0000 7fa5d37fe640 1 -- 172.21.15.40:0/3545189434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5cc044020 con 0x7fa5dc0d1770 2024-04-26T21:47:50.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.921+0000 7fa5d37fe640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:50.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.921+0000 7fa5d37fe640 1 -- 172.21.15.40:0/3545189434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fa5cc0ac220 con 0x7fa5dc0d1770 2024-04-26T21:47:50.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.922+0000 7fa5d3fff640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.922+0000 7fa5d3fff640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0b98e0 tx=0x7fa5c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:50.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:50.926+0000 7fa5d37fe640 1 -- 172.21.15.40:0/3545189434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5dc06d9b0 con 0x7fa5dc0d1770 2024-04-26T21:47:51.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.108+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5dc0c8710 con 0x7fa5bc06c8f0 2024-04-26T21:47:51.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.110+0000 7fa5d37fe640 1 -- 172.21.15.40:0/3545189434 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13514 (secure 0 0 0) 0x7fa5dc0c8710 con 0x7fa5bc06c8f0 2024-04-26T21:47:51.111 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:51.111 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5bc06c8f0 msgr2=0x7fa5bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0b98e0 tx=0x7fa5c4008040 comp rx=0 tx=0).stop 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc0d1770 msgr2=0x7fa5dc1f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc0d1770 0x7fa5dc1f6950 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc02a2b0 tx=0x7fa5cc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 shutdown_connections 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5dc15f3a0 0x7fa5dc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc0d2160 0x7fa5dc1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 --2- 172.21.15.40:0/3545189434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5dc0d1770 0x7fa5dc1f6950 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 >> 172.21.15.40:0/3545189434 conn(0x7fa5dc0d4f90 msgr2=0x7fa5dc0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 shutdown_connections 2024-04-26T21:47:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.112+0000 7fa5e170a640 1 -- 172.21.15.40:0/3545189434 wait complete. 2024-04-26T21:47:51.123 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2024-04-26T21:47:50.187692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294720,"kb_used_data":16144,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680000,"statfs":{"total":773094113280,"available":772792320000,"internally_reserved":0,"allocated":16531456,"data_stored":8956940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36286,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-102,"num_write_kb":-72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001763"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T21:47:47.770766+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:47:47.770766+0000","last_peered":"2024-04-26T21:47:47.770766+0000","last_clean":"2024-04-26T21:47:47.770766+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:47:47.770766+0000","last_undegraded":"2024-04-26T21:47:47.770766+0000","last_fullsized":"2024-04-26T21:47:47.770766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37796,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334044,"statfs":{"total":96636764160,"available":96598061056,"internally_reserved":0,"allocated":2510848,"data_stored":1561034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575006,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41412,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330428,"statfs":{"total":96636764160,"available":96594358272,"internally_reserved":0,"allocated":1957888,"data_stored":1000261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":970666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1908736,"data_stored":964277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1904640,"data_stored":964767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37216,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1916928,"data_stored":970666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":1904640,"data_stored":964235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575006,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41956,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329884,"statfs":{"total":96636764160,"available":96593801216,"internally_reserved":0,"allocated":2510848,"data_stored":1561034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:51.123 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:47:51.124 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:51.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.224+0000 7f4d3d26b640 1 Processor -- start 2024-04-26T21:47:51.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.224+0000 7f4d3d26b640 1 -- start start 2024-04-26T21:47:51.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.224+0000 7f4d3d26b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d38162810 0x7f4d38070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.224+0000 7f4d3d26b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d380d10d0 con 0x7f4d380d1b30 2024-04-26T21:47:51.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.224+0000 7f4d3d26b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d380d1370 con 0x7f4d380b2340 2024-04-26T21:47:51.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.224+0000 7f4d3d26b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d380d0750 con 0x7f4d38162810 2024-04-26T21:47:51.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d377fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4d380d1b30 0x7f4d38070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54500/0 (socket says 172.21.15.40:54500) 2024-04-26T21:47:51.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d377fe640 1 -- 172.21.15.40:0/3834117080 learned_addr learned my addr 172.21.15.40:0/3834117080 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:51.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989678519 0 0) 0x7f4d380d10d0 con 0x7f4d380d1b30 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d20003520 con 0x7f4d380d1b30 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d36ffd640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d38162810 0x7f4d38070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3502347429 0 0) 0x7f4d20003520 con 0x7f4d380d1b30 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d38162810 msgr2=0x7f4d38070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d38162810 0x7f4d38070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 >> v1:172.21.15.184:6789/0 conn(0x7f4d380b2340 legacy=0x7f4d380cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d380d01b0 con 0x7f4d380d1b30 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f4d28001670 con 0x7f4d380d1b30 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 >> v1:172.21.15.40:6789/0 conn(0x7f4d380d1b30 legacy=0x7f4d38070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d38162810 0x7f4d200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d20003f60 0x7f4d20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20004d40 0x7f4d200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d380d0750 con 0x7f4d20003f60 2024-04-26T21:47:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2000baa0 con 0x7f4d20004d40 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2000bca0 con 0x7f4d38162810 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d367fc640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d20003f60 0x7f4d20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d377fe640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20004d40 0x7f4d200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.225+0000 7f4d36ffd640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d38162810 0x7f4d200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d367fc640 1 -- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d38162810 msgr2=0x7f4d200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d367fc640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d38162810 0x7f4d200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d367fc640 1 -- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20004d40 msgr2=0x7f4d200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d367fc640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20004d40 0x7f4d200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d367fc640 1 -- 172.21.15.40:0/3834117080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d20003520 con 0x7f4d20003f60 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d367fc640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d20003f60 0x7f4d20004800 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c001750 tx=0x7f4d20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f401273d9f6d9cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c001370 con 0x7f4d20003f60 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2c0052d0 con 0x7f4d20003f60 2024-04-26T21:47:51.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.226+0000 7f4d35ffb640 1 -- 172.21.15.40:0/3834117080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c02abe0 con 0x7f4d20003f60 2024-04-26T21:47:51.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.227+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3834117080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d20003f60 msgr2=0x7f4d20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.227+0000 7f4d3d26b640 1 --2- 172.21.15.40:0/3834117080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d20003f60 0x7f4d20004800 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c001750 tx=0x7f4d20005830 comp rx=0 tx=0).stop 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.227+0000 7f4d377fe640 1 -- 172.21.15.40:0/3834117080 reap_dead start 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.227+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3834117080 shutdown_connections 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.227+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3834117080 >> 172.21.15.40:0/3834117080 conn(0x7f4d380d2da0 msgr2=0x7f4d380d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.227+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3834117080 shutdown_connections 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.227+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3834117080 wait complete. 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.228+0000 7f4d3d26b640 1 Processor -- start 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.228+0000 7f4d3d26b640 1 -- start start 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d3d26b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20003f60 0x7f4d38156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d3d26b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d380cd3d0 0x7f4d38157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d3d26b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3815d9e0 0x7f4d3815de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d3d26b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d380cdb00 con 0x7f4d3815d9e0 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d3d26b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d380d16a0 con 0x7f4d20003f60 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d3d26b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d38071260 con 0x7f4d380cd3d0 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3815d9e0 0x7f4d3815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3815d9e0 0x7f4d3815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50988/0 (socket says 172.21.15.40:50988) 2024-04-26T21:47:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d377fe640 1 -- 172.21.15.40:0/3268550455 learned_addr learned my addr 172.21.15.40:0/3268550455 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d36ffd640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20003f60 0x7f4d38156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d367fc640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d380cd3d0 0x7f4d38157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d377fe640 1 -- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d380cd3d0 msgr2=0x7f4d38157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d377fe640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d380cd3d0 0x7f4d38157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d377fe640 1 -- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20003f60 msgr2=0x7f4d38156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d377fe640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20003f60 0x7f4d38156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.229+0000 7f4d377fe640 1 -- 172.21.15.40:0/3268550455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d38157da0 con 0x7f4d3815d9e0 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.230+0000 7f4d36ffd640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20003f60 0x7f4d38156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.230+0000 7f4d367fc640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d380cd3d0 0x7f4d38157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.230+0000 7f4d377fe640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3815d9e0 0x7f4d3815de60 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f4d380cf3c0 tx=0x7f4d2800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.230+0000 7f4d1f7fe640 1 -- 172.21.15.40:0/3268550455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28019070 con 0x7f4d3815d9e0 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.230+0000 7f4d1f7fe640 1 -- 172.21.15.40:0/3268550455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2800c5e0 con 0x7f4d3815d9e0 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.230+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d38158080 con 0x7f4d3815d9e0 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.230+0000 7f4d1f7fe640 1 -- 172.21.15.40:0/3268550455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800c9d0 con 0x7f4d3815d9e0 2024-04-26T21:47:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.230+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d38158640 con 0x7f4d3815d9e0 2024-04-26T21:47:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.231+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d380694f0 con 0x7f4d3815d9e0 2024-04-26T21:47:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.231+0000 7f4d1f7fe640 1 -- 172.21.15.40:0/3268550455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d2800cb70 con 0x7f4d3815d9e0 2024-04-26T21:47:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.232+0000 7f4d1f7fe640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d2406c920 0x7f4d2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.232+0000 7f4d1f7fe640 1 -- 172.21.15.40:0/3268550455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f4d2808ab90 con 0x7f4d3815d9e0 2024-04-26T21:47:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.232+0000 7f4d36ffd640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d2406c920 0x7f4d2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.233+0000 7f4d36ffd640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d2406c920 0x7f4d2406ede0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f4d30007440 tx=0x7f4d3000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.236+0000 7f4d1f7fe640 1 -- 172.21.15.40:0/3268550455 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d2800ce60 con 0x7f4d3815d9e0 2024-04-26T21:47:51.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.418+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d380b45e0 con 0x7f4d2406c920 2024-04-26T21:47:51.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.419+0000 7f4d1f7fe640 1 -- 172.21.15.40:0/3268550455 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13514 (secure 0 0 0) 0x7f4d380b45e0 con 0x7f4d2406c920 2024-04-26T21:47:51.420 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:51.420 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d2406c920 msgr2=0x7f4d2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d2406c920 0x7f4d2406ede0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f4d30007440 tx=0x7f4d3000b040 comp rx=0 tx=0).stop 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3815d9e0 msgr2=0x7f4d3815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3815d9e0 0x7f4d3815de60 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f4d380cf3c0 tx=0x7f4d2800eb90 comp rx=0 tx=0).stop 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 shutdown_connections 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f4d2406c920 0x7f4d2406ede0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3815d9e0 0x7f4d3815de60 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d380cd3d0 0x7f4d38157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 --2- 172.21.15.40:0/3268550455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d20003f60 0x7f4d38156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 >> 172.21.15.40:0/3268550455 conn(0x7f4d380d2da0 msgr2=0x7f4d38070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 shutdown_connections 2024-04-26T21:47:51.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.421+0000 7f4d3d26b640 1 -- 172.21.15.40:0/3268550455 wait complete. 2024-04-26T21:47:51.433 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2024-04-26T21:47:50.187692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294720,"kb_used_data":16144,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680000,"statfs":{"total":773094113280,"available":772792320000,"internally_reserved":0,"allocated":16531456,"data_stored":8956940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36286,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-102,"num_write_kb":-72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001763"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T21:47:47.770766+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:47:47.770766+0000","last_peered":"2024-04-26T21:47:47.770766+0000","last_clean":"2024-04-26T21:47:47.770766+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:47:47.770766+0000","last_undegraded":"2024-04-26T21:47:47.770766+0000","last_fullsized":"2024-04-26T21:47:47.770766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37796,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334044,"statfs":{"total":96636764160,"available":96598061056,"internally_reserved":0,"allocated":2510848,"data_stored":1561034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575006,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41412,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330428,"statfs":{"total":96636764160,"available":96594358272,"internally_reserved":0,"allocated":1957888,"data_stored":1000261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":970666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1908736,"data_stored":964277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1904640,"data_stored":964767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37216,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1916928,"data_stored":970666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":1904640,"data_stored":964235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575006,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41956,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329884,"statfs":{"total":96636764160,"available":96593801216,"internally_reserved":0,"allocated":2510848,"data_stored":1561034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:51.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:47:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.533+0000 7fb9be133640 1 Processor -- start 2024-04-26T21:47:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.533+0000 7fb9be133640 1 -- start start 2024-04-26T21:47:51.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9be133640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 0x7fb9b80d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9be133640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9b80d2830 0x7fb9b80cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9be133640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b80d1a70 con 0x7fb9b815f5a0 2024-04-26T21:47:51.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9be133640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b8163d30 con 0x7fb9b80d2830 2024-04-26T21:47:51.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9be133640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b80d1d50 con 0x7fb9b8163610 2024-04-26T21:47:51.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9b7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9b80d2830 0x7fb9b80cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9b7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9b80d2830 0x7fb9b80cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51002/0 (socket says 172.21.15.40:51002) 2024-04-26T21:47:51.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9b7fff640 1 -- 172.21.15.40:0/1297144611 learned_addr learned my addr 172.21.15.40:0/1297144611 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:51.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9b6ffd640 1 --2- 172.21.15.40:0/1297144611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 0x7fb9b80d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9b7fff640 1 -- 172.21.15.40:0/1297144611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 msgr2=0x7fb9b80d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:51.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9b7fff640 1 --2- 172.21.15.40:0/1297144611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 0x7fb9b80d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9b7fff640 1 -- 172.21.15.40:0/1297144611 >> v1:172.21.15.184:6789/0 conn(0x7fb9b815f5a0 legacy=0x7fb9b8162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:51.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.534+0000 7fb9b7fff640 1 -- 172.21.15.40:0/1297144611 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b80d2030 con 0x7fb9b80d2830 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.535+0000 7fb9b7fff640 1 --2- 172.21.15.40:0/1297144611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cfda0 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009e50 tx=0x7fb9b002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7f56e7d04a9638 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.535+0000 7fb9b67fc640 1 -- 172.21.15.40:0/1297144611 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0036070 con 0x7fb9b80d2830 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.535+0000 7fb9b67fc640 1 -- 172.21.15.40:0/1297144611 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b0031600 con 0x7fb9b80d2830 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.535+0000 7fb9b67fc640 1 -- 172.21.15.40:0/1297144611 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00319d0 con 0x7fb9b80d2830 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.536+0000 7fb9be133640 1 -- 172.21.15.40:0/1297144611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 msgr2=0x7fb9b80cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.536+0000 7fb9be133640 1 --2- 172.21.15.40:0/1297144611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cfda0 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009e50 tx=0x7fb9b002a630 comp rx=0 tx=0).stop 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.536+0000 7fb9be133640 1 -- 172.21.15.40:0/1297144611 shutdown_connections 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.536+0000 7fb9be133640 1 --2- 172.21.15.40:0/1297144611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cfda0 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.536+0000 7fb9be133640 1 --2- 172.21.15.40:0/1297144611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 0x7fb9b80d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.536+0000 7fb9be133640 1 -- 172.21.15.40:0/1297144611 >> 172.21.15.40:0/1297144611 conn(0x7fb9b80d5190 msgr2=0x7fb9b80d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.536+0000 7fb9be133640 1 -- 172.21.15.40:0/1297144611 shutdown_connections 2024-04-26T21:47:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.536+0000 7fb9be133640 1 -- 172.21.15.40:0/1297144611 wait complete. 2024-04-26T21:47:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.537+0000 7fb9be133640 1 Processor -- start 2024-04-26T21:47:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.537+0000 7fb9be133640 1 -- start start 2024-04-26T21:47:51.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9be133640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9be133640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 0x7fb9b80b9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9be133640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b80b9ea0 0x7fb9b81f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9be133640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b80d3c20 con 0x7fb9b80d2830 2024-04-26T21:47:51.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9be133640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b80d4190 con 0x7fb9b80b9ea0 2024-04-26T21:47:51.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9be133640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b80d3da0 con 0x7fb9b8163610 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51008/0 (socket says 172.21.15.40:51008) 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b77fe640 1 -- 172.21.15.40:0/417844730 learned_addr learned my addr 172.21.15.40:0/417844730 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b7fff640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b80b9ea0 0x7fb9b81f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b6ffd640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 0x7fb9b80b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b77fe640 1 -- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 msgr2=0x7fb9b80b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b77fe640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 0x7fb9b80b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b77fe640 1 -- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b80b9ea0 msgr2=0x7fb9b81f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b77fe640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b80b9ea0 0x7fb9b81f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b77fe640 1 -- 172.21.15.40:0/417844730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b80c9870 con 0x7fb9b80d2830 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.538+0000 7fb9b7fff640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b80b9ea0 0x7fb9b81f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.539+0000 7fb9b77fe640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cb240 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c460 tx=0x7fb9a800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.539+0000 7fb99ffff640 1 -- 172.21.15.40:0/417844730 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8016020 con 0x7fb9b80d2830 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.539+0000 7fb99ffff640 1 -- 172.21.15.40:0/417844730 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a8012870 con 0x7fb9b80d2830 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.539+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b80c9b50 con 0x7fb9b80d2830 2024-04-26T21:47:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.539+0000 7fb99ffff640 1 -- 172.21.15.40:0/417844730 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8012c60 con 0x7fb9b80d2830 2024-04-26T21:47:51.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.539+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b80ca110 con 0x7fb9b80d2830 2024-04-26T21:47:51.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.540+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b80694f0 con 0x7fb9b80d2830 2024-04-26T21:47:51.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.540+0000 7fb99ffff640 1 -- 172.21.15.40:0/417844730 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9a8019020 con 0x7fb9b80d2830 2024-04-26T21:47:51.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.541+0000 7fb99ffff640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb9a406c8f0 0x7fb9a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:51.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.541+0000 7fb99ffff640 1 -- 172.21.15.40:0/417844730 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fb9a808b790 con 0x7fb9b80d2830 2024-04-26T21:47:51.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.541+0000 7fb9b6ffd640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb9a406c8f0 0x7fb9a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:51.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.542+0000 7fb9b6ffd640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb9a406c8f0 0x7fb9a406edb0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac006fd0 tx=0x7fb9ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:51.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.545+0000 7fb99ffff640 1 -- 172.21.15.40:0/417844730 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9a8020030 con 0x7fb9b80d2830 2024-04-26T21:47:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.731+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9b8068d30 con 0x7fb9b80d2830 2024-04-26T21:47:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.732+0000 7fb99ffff640 1 -- 172.21.15.40:0/417844730 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+13397 (secure 0 0 0) 0x7fb9a8019310 con 0x7fb9b80d2830 2024-04-26T21:47:51.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:51.732 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":149,"fsid":"ec445c36-0c6e-44d6-884b-6999dea32f55","created":"2024-04-26T21:34:45.788343+0000","modified":"2024-04-26T21:47:48.603448+0000","last_up_change":"2024-04-26T21:34:57.042720+0000","last_in_change":"2024-04-26T21:34:47.953711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:34:58.012836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7180d7b-0b28-4813-a6e6-11d0a2c38363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6809","nonce":628259048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6813","nonce":628259048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6823","nonce":628259048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":628259048},{"type":"v1","addr":"172.21.15.40:6818","nonce":628259048}]},"public_addr":"172.21.15.40:6809/628259048","cluster_addr":"172.21.15.40:6813/628259048","heartbeat_back_addr":"172.21.15.40:6823/628259048","heartbeat_front_addr":"172.21.15.40:6818/628259048","state":["exists","up"]},{"osd":1,"uuid":"5f3271b9-0a80-4ee5-89e7-3b6db512e273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6801","nonce":299622497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6803","nonce":299622497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6807","nonce":299622497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":299622497},{"type":"v1","addr":"172.21.15.40:6805","nonce":299622497}]},"public_addr":"172.21.15.40:6801/299622497","cluster_addr":"172.21.15.40:6803/299622497","heartbeat_back_addr":"172.21.15.40:6807/299622497","heartbeat_front_addr":"172.21.15.40:6805/299622497","state":["exists","up"]},{"osd":2,"uuid":"2a3d220c-7de8-4f4f-b1d7-929338ce542c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6812","nonce":3280366756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6816","nonce":3280366756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6822","nonce":3280366756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":3280366756},{"type":"v1","addr":"172.21.15.40:6819","nonce":3280366756}]},"public_addr":"172.21.15.40:6812/3280366756","cluster_addr":"172.21.15.40:6816/3280366756","heartbeat_back_addr":"172.21.15.40:6822/3280366756","heartbeat_front_addr":"172.21.15.40:6819/3280366756","state":["exists","up"]},{"osd":3,"uuid":"c19af625-ba1e-464d-9733-94998080ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6825","nonce":4238362919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6827","nonce":4238362919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6831","nonce":4238362919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":4238362919},{"type":"v1","addr":"172.21.15.40:6829","nonce":4238362919}]},"public_addr":"172.21.15.40:6825/4238362919","cluster_addr":"172.21.15.40:6827/4238362919","heartbeat_back_addr":"172.21.15.40:6831/4238362919","heartbeat_front_addr":"172.21.15.40:6829/4238362919","state":["exists","up"]},{"osd":4,"uuid":"5bad1c6b-9d6e-4fcf-adcb-fd657f4a4a33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2829090873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2829090873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2829090873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2829090873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2829090873}]},"public_addr":"172.21.15.184:6801/2829090873","cluster_addr":"172.21.15.184:6803/2829090873","heartbeat_back_addr":"172.21.15.184:6807/2829090873","heartbeat_front_addr":"172.21.15.184:6805/2829090873","state":["exists","up"]},{"osd":5,"uuid":"afd6e92c-4942-477c-9963-4930f3f9f17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6810","nonce":304953510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6814","nonce":304953510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6822","nonce":304953510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":304953510},{"type":"v1","addr":"172.21.15.184:6818","nonce":304953510}]},"public_addr":"172.21.15.184:6810/304953510","cluster_addr":"172.21.15.184:6814/304953510","heartbeat_back_addr":"172.21.15.184:6822/304953510","heartbeat_front_addr":"172.21.15.184:6818/304953510","state":["exists","up"]},{"osd":6,"uuid":"b2bcbe55-8d3f-4ea4-806b-c5b5016466c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6825","nonce":533952038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6827","nonce":533952038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6831","nonce":533952038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":533952038},{"type":"v1","addr":"172.21.15.184:6829","nonce":533952038}]},"public_addr":"172.21.15.184:6825/533952038","cluster_addr":"172.21.15.184:6827/533952038","heartbeat_back_addr":"172.21.15.184:6831/533952038","heartbeat_front_addr":"172.21.15.184:6829/533952038","state":["exists","up"]},{"osd":7,"uuid":"c57e4b4b-5307-4e1e-ac19-9f59168f08ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6811","nonce":2389595522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6815","nonce":2389595522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6821","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6823","nonce":2389595522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":2389595522},{"type":"v1","addr":"172.21.15.184:6819","nonce":2389595522}]},"public_addr":"172.21.15.184:6811/2389595522","cluster_addr":"172.21.15.184:6815/2389595522","heartbeat_back_addr":"172.21.15.184:6823/2389595522","heartbeat_front_addr":"172.21.15.184:6819/2389595522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.090540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.102674+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.172292+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.108624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.042751+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.130721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.118638+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:34:53.112692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3119057314":"2024-04-27T21:47:45.591960+0000","172.21.15.40:6834/3119057314":"2024-04-27T21:47:45.591960+0000","172.21.15.40:6835/3441996159":"2024-04-27T21:47:30.391245+0000","172.21.15.40:6834/3441996159":"2024-04-27T21:47:30.391245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:47:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb9a406c8f0 msgr2=0x7fb9a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb9a406c8f0 0x7fb9a406edb0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac006fd0 tx=0x7fb9ac008040 comp rx=0 tx=0).stop 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 msgr2=0x7fb9b80cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cb240 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c460 tx=0x7fb9a800c920 comp rx=0 tx=0).stop 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 shutdown_connections 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb9a406c8f0 0x7fb9a406edb0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b80b9ea0 0x7fb9b81f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b8163610 0x7fb9b80b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 --2- 172.21.15.40:0/417844730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b80d2830 0x7fb9b80cb240 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.733+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 >> 172.21.15.40:0/417844730 conn(0x7fb9b80d5190 msgr2=0x7fb9b806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.734+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 shutdown_connections 2024-04-26T21:47:51.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:51.734+0000 7fb9be133640 1 -- 172.21.15.40:0/417844730 wait complete. 2024-04-26T21:47:52.746 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T21:47:52.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:47:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.864+0000 7fadcc589640 1 Processor -- start 2024-04-26T21:47:52.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.864+0000 7fadcc589640 1 -- start start 2024-04-26T21:47:52.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadcc589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 0x7fadc40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadcc589640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fadc41606c0 0x7fadc415f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadcc589640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc4163160 con 0x7fadc415fc60 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadcc589640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fadc4162f10 con 0x7fadc41606c0 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadcc589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fadc4163d70 con 0x7fadc415f340 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadcab26640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fadc415fc60 0x7fadc40d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54512/0 (socket says 172.21.15.40:54512) 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadcab26640 1 -- 172.21.15.40:0/3325725715 learned_addr learned my addr 172.21.15.40:0/3325725715 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadca325640 1 --2- 172.21.15.40:0/3325725715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 0x7fadc40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadc9b24640 1 --2- 172.21.15.40:0/3325725715 >> v2:172.21.15.184:3300/0 conn(0x7fadc41606c0 0x7fadc415f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadc9323640 1 -- 172.21.15.40:0/3325725715 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531918309 0 0) 0x7fadc4163160 con 0x7fadc415fc60 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.865+0000 7fadc9323640 1 -- 172.21.15.40:0/3325725715 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb0003520 con 0x7fadc415fc60 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.866+0000 7fadca325640 1 -- 172.21.15.40:0/3325725715 >> v2:172.21.15.184:3300/0 conn(0x7fadc41606c0 msgr2=0x7fadc415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.866+0000 7fadca325640 1 --2- 172.21.15.40:0/3325725715 >> v2:172.21.15.184:3300/0 conn(0x7fadc41606c0 0x7fadc415f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.866+0000 7fadca325640 1 -- 172.21.15.40:0/3325725715 >> v1:172.21.15.40:6789/0 conn(0x7fadc415fc60 legacy=0x7fadc40d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.866+0000 7fadc9323640 1 -- 172.21.15.40:0/3325725715 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 595759182 0 0) 0x7fadb0003520 con 0x7fadc415fc60 2024-04-26T21:47:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.866+0000 7fadca325640 1 -- 172.21.15.40:0/3325725715 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc4163440 con 0x7fadc415f340 2024-04-26T21:47:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.866+0000 7fadc9b24640 1 --2- 172.21.15.40:0/3325725715 >> v2:172.21.15.184:3300/0 conn(0x7fadc41606c0 0x7fadc415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.866+0000 7fadca325640 1 --2- 172.21.15.40:0/3325725715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 0x7fadc40d2550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadc00097a0 tx=0x7fadc002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647c80d6c1ecb0d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadc9323640 1 -- 172.21.15.40:0/3325725715 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0037070 con 0x7fadc415f340 2024-04-26T21:47:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadc9323640 1 -- 172.21.15.40:0/3325725715 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc002aba0 con 0x7fadc415f340 2024-04-26T21:47:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadc9323640 1 -- 172.21.15.40:0/3325725715 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0019f0 con 0x7fadc415f340 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadcc589640 1 -- 172.21.15.40:0/3325725715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 msgr2=0x7fadc40d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadcc589640 1 --2- 172.21.15.40:0/3325725715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 0x7fadc40d2550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadc00097a0 tx=0x7fadc002a110 comp rx=0 tx=0).stop 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadcc589640 1 -- 172.21.15.40:0/3325725715 shutdown_connections 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadcc589640 1 --2- 172.21.15.40:0/3325725715 >> v2:172.21.15.184:3300/0 conn(0x7fadc41606c0 0x7fadc415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadcc589640 1 --2- 172.21.15.40:0/3325725715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 0x7fadc40d2550 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadcc589640 1 -- 172.21.15.40:0/3325725715 >> 172.21.15.40:0/3325725715 conn(0x7fadc409ba50 msgr2=0x7fadc409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadcc589640 1 -- 172.21.15.40:0/3325725715 shutdown_connections 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.867+0000 7fadcc589640 1 -- 172.21.15.40:0/3325725715 wait complete. 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.869+0000 7fadcc589640 1 Processor -- start 2024-04-26T21:47:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.869+0000 7fadcc589640 1 -- start start 2024-04-26T21:47:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.869+0000 7fadcc589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 0x7fadc40afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcc589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadc415fc60 0x7fadc40b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcc589640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc41606c0 0x7fadc41f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcc589640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fadc40d2a90 con 0x7fadc41606c0 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcc589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4163a00 con 0x7fadc415fc60 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcc589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fadc41600d0 con 0x7fadc415f340 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc41606c0 0x7fadc41f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc41606c0 0x7fadc41f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51010/0 (socket says 172.21.15.40:51010) 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 -- 172.21.15.40:0/3943655589 learned_addr learned my addr 172.21.15.40:0/3943655589 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadca325640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 0x7fadc40afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadc9b24640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadc415fc60 0x7fadc40b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 -- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 msgr2=0x7fadc40afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadc415f340 0x7fadc40afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 -- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadc415fc60 msgr2=0x7fadc40b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadc415fc60 0x7fadc40b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 -- 172.21.15.40:0/3943655589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc40b0510 con 0x7fadc41606c0 2024-04-26T21:47:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcab26640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc41606c0 0x7fadc41f2000 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7fadbc001b60 tx=0x7fadbc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc017070 con 0x7fadc41606c0 2024-04-26T21:47:52.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadbc012400 con 0x7fadc41606c0 2024-04-26T21:47:52.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0127b0 con 0x7fadc41606c0 2024-04-26T21:47:52.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.870+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc40b07f0 con 0x7fadc41606c0 2024-04-26T21:47:52.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.871+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc40b0db0 con 0x7fadc41606c0 2024-04-26T21:47:52.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.871+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fadc4069600 con 0x7fadc41606c0 2024-04-26T21:47:52.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.872+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadbc01a070 con 0x7fadc41606c0 2024-04-26T21:47:52.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.872+0000 7fadbaffd640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fada806c8f0 0x7fada806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.873+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fadbc08b970 con 0x7fadc41606c0 2024-04-26T21:47:52.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.873+0000 7fadbaffd640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fada8072e30 0x7fada8075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:52.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.873+0000 7fadca325640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fada806c8f0 0x7fada806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.873+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fada8075940 con 0x7fada8072e30 2024-04-26T21:47:52.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.873+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fadbc0902a0 con 0x7fadc41606c0 2024-04-26T21:47:52.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.873+0000 7fadc9b24640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fada8072e30 0x7fada8075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:52.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.873+0000 7fadca325640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fada806c8f0 0x7fada806edb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fadc0034000 tx=0x7fadc002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.873+0000 7fadc9b24640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fada8072e30 0x7fada8075270 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:52.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.874+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 <== osd.0 v2:172.21.15.40:6808/628259048 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fada8075940 con 0x7fada8072e30 2024-04-26T21:47:52.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.887+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 --> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fadc40b11f0 con 0x7fada8072e30 2024-04-26T21:47:52.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.887+0000 7fadbaffd640 1 -- 172.21.15.40:0/3943655589 <== osd.0 v2:172.21.15.40:6808/628259048 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fadc40b11f0 con 0x7fada8072e30 2024-04-26T21:47:52.888 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:47:52.889 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:52.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fada8072e30 msgr2=0x7fada8075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.40:6808/628259048,v1:172.21.15.40:6809/628259048] conn(0x7fada8072e30 0x7fada8075270 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fada806c8f0 msgr2=0x7fada806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fada806c8f0 0x7fada806edb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fadc0034000 tx=0x7fadc002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc41606c0 msgr2=0x7fadc41f2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 --2- 172.21.15.40:0/3943655589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc41606c0 0x7fadc41f2000 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7fadbc001b60 tx=0x7fadbc00a430 comp rx=0 tx=0).stop 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcab26640 1 -- 172.21.15.40:0/3943655589 reap_dead start 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 shutdown_connections 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 >> 172.21.15.40:0/3943655589 conn(0x7fadc409ba50 msgr2=0x7fadc40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 shutdown_connections 2024-04-26T21:47:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:52.888+0000 7fadcc589640 1 -- 172.21.15.40:0/3943655589 wait complete. 2024-04-26T21:47:52.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T21:47:53.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.001+0000 7f59682c8640 1 Processor -- start 2024-04-26T21:47:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.001+0000 7f59682c8640 1 -- start start 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.001+0000 7f59682c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5960162810 0x7f5960070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.001+0000 7f59682c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f59600d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.001+0000 7f59682c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59600d10d0 con 0x7f59600b2340 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.001+0000 7f59682c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f59600d0750 con 0x7f5960162810 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.001+0000 7f59682c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f59600d16a0 con 0x7f59600d1b30 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5966865640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f59600d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5966064640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5960162810 0x7f5960070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5965863640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f59600b2340 0x7f59600cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:37040/0 (socket says 172.21.15.40:37040) 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5966064640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5960162810 0x7f5960070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:41902/0 (socket says 172.21.15.40:41902) 2024-04-26T21:47:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5966865640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f59600d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:51692/0 (socket says 172.21.15.40:51692) 2024-04-26T21:47:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5965863640 1 -- 172.21.15.40:0/3096973676 learned_addr learned my addr 172.21.15.40:0/3096973676 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5966865640 1 -- 172.21.15.40:0/3096973676 >> v2:172.21.15.184:3300/0 conn(0x7f5960162810 msgr2=0x7f5960070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5966865640 1 --2- 172.21.15.40:0/3096973676 >> v2:172.21.15.184:3300/0 conn(0x7f5960162810 0x7f5960070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5966865640 1 -- 172.21.15.40:0/3096973676 >> v1:172.21.15.184:6789/0 conn(0x7f59600b2340 legacy=0x7f59600cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.002+0000 7f5966865640 1 -- 172.21.15.40:0/3096973676 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59600d1370 con 0x7f59600d1b30 2024-04-26T21:47:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.003+0000 7f5966865640 1 --2- 172.21.15.40:0/3096973676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f59600d1fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5958009d90 tx=0x7f595802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a90f7a15ca1a346 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.003+0000 7f5965062640 1 -- 172.21.15.40:0/3096973676 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958007740 con 0x7f59600d1b30 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.003+0000 7f5965062640 1 -- 172.21.15.40:0/3096973676 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59580078e0 con 0x7f59600d1b30 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.003+0000 7f5965062640 1 -- 172.21.15.40:0/3096973676 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958007cb0 con 0x7f59600d1b30 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.004+0000 7f59682c8640 1 -- 172.21.15.40:0/3096973676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 msgr2=0x7f59600d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.004+0000 7f59682c8640 1 --2- 172.21.15.40:0/3096973676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f59600d1fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5958009d90 tx=0x7f595802a660 comp rx=0 tx=0).stop 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.004+0000 7f59682c8640 1 -- 172.21.15.40:0/3096973676 shutdown_connections 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.004+0000 7f59682c8640 1 --2- 172.21.15.40:0/3096973676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f59600d1fc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.004+0000 7f59682c8640 1 --2- 172.21.15.40:0/3096973676 >> v2:172.21.15.184:3300/0 conn(0x7f5960162810 0x7f5960070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.004+0000 7f59682c8640 1 -- 172.21.15.40:0/3096973676 >> 172.21.15.40:0/3096973676 conn(0x7f59600d2da0 msgr2=0x7f59600d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.004+0000 7f59682c8640 1 -- 172.21.15.40:0/3096973676 shutdown_connections 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.004+0000 7f59682c8640 1 -- 172.21.15.40:0/3096973676 wait complete. 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.005+0000 7f59682c8640 1 Processor -- start 2024-04-26T21:47:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.005+0000 7f59682c8640 1 -- start start 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f59682c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59600b2340 0x7f59600c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f59682c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f596014e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f59682c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5960162810 0x7f596006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f59682c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5960070890 con 0x7f59600b2340 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f59682c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59600d0470 con 0x7f5960162810 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f59682c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f59600cdb00 con 0x7f59600d1b30 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5966064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59600b2340 0x7f59600c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5966064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59600b2340 0x7f59600c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51018/0 (socket says 172.21.15.40:51018) 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5966064640 1 -- 172.21.15.40:0/4234532598 learned_addr learned my addr 172.21.15.40:0/4234532598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5965863640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f596014e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5966865640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5960162810 0x7f596006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5965863640 1 -- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5960162810 msgr2=0x7f596006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5965863640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5960162810 0x7f596006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5965863640 1 -- 172.21.15.40:0/4234532598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59600b2340 msgr2=0x7f59600c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5965863640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59600b2340 0x7f59600c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5965863640 1 -- 172.21.15.40:0/4234532598 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596014ea10 con 0x7f59600d1b30 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5966064640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59600b2340 0x7f59600c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.006+0000 7f5965863640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f596014e260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f595000c370 tx=0x7f595000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.007+0000 7f5956ffd640 1 -- 172.21.15.40:0/4234532598 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950017070 con 0x7f59600d1b30 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.007+0000 7f5956ffd640 1 -- 172.21.15.40:0/4234532598 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5950005ce0 con 0x7f59600d1b30 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.007+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59600cacf0 con 0x7f59600d1b30 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.007+0000 7f5956ffd640 1 -- 172.21.15.40:0/4234532598 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950007310 con 0x7f59600d1b30 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.007+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5960160150 con 0x7f59600d1b30 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.008+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59600694f0 con 0x7f59600d1b30 2024-04-26T21:47:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.008+0000 7f5956ffd640 1 -- 172.21.15.40:0/4234532598 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f595001a070 con 0x7f59600d1b30 2024-04-26T21:47:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.008+0000 7f5956ffd640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.009+0000 7f5956ffd640 1 -- 172.21.15.40:0/4234532598 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f595008acc0 con 0x7f59600d1b30 2024-04-26T21:47:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.009+0000 7f5966064640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.009+0000 7f5966064640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f595c009f50 tx=0x7f595c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.013+0000 7f5956ffd640 1 -- 172.21.15.40:0/4234532598 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5950091080 con 0x7f59600d1b30 2024-04-26T21:47:53.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.200+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f59600b1ef0 con 0x7f594406c8f0 2024-04-26T21:47:53.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.200+0000 7f5956ffd640 1 -- 172.21.15.40:0/4234532598 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f59600b1ef0 con 0x7f594406c8f0 2024-04-26T21:47:53.201 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T21:47:53.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 msgr2=0x7f594406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f595c009f50 tx=0x7f595c009190 comp rx=0 tx=0).stop 2024-04-26T21:47:53.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 msgr2=0x7f596014e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f596014e260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f595000c370 tx=0x7f595000c830 comp rx=0 tx=0).stop 2024-04-26T21:47:53.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 shutdown_connections 2024-04-26T21:47:53.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f594406c8f0 0x7f594406edb0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5960162810 0x7f596006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59600d1b30 0x7f596014e260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 --2- 172.21.15.40:0/4234532598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59600b2340 0x7f59600c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 >> 172.21.15.40:0/4234532598 conn(0x7f59600d2da0 msgr2=0x7f59600b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 shutdown_connections 2024-04-26T21:47:53.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.202+0000 7f59682c8640 1 -- 172.21.15.40:0/4234532598 wait complete. 2024-04-26T21:47:53.213 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T21:47:53.214 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:47:53.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.317+0000 7f5506b46640 1 Processor -- start 2024-04-26T21:47:53.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.317+0000 7f5506b46640 1 -- start start 2024-04-26T21:47:53.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.318+0000 7f5506b46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 0x7f55000b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.318+0000 7f5506b46640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55000b33e0 con 0x7f5500145630 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.318+0000 7f5506b46640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55000cfd70 con 0x7f55000b2fc0 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.318+0000 7f5506b46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55000d0050 con 0x7f550015f980 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.318+0000 7f55048e2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5500145630 0x7f55000cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54518/0 (socket says 172.21.15.40:54518) 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.318+0000 7f55048e2640 1 -- 172.21.15.40:0/925225963 learned_addr learned my addr 172.21.15.40:0/925225963 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.318+0000 7f54fffff640 1 --2- 172.21.15.40:0/925225963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 0x7f55000b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.319+0000 7f54ff7fe640 1 -- 172.21.15.40:0/925225963 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006586471 0 0) 0x7f55000b33e0 con 0x7f5500145630 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.319+0000 7f54ff7fe640 1 -- 172.21.15.40:0/925225963 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54e8003520 con 0x7f5500145630 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.319+0000 7f54ff7fe640 1 -- 172.21.15.40:0/925225963 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2035031043 0 0) 0x7f55000cfd70 con 0x7f55000b2fc0 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.319+0000 7f54fffff640 1 -- 172.21.15.40:0/925225963 >> v1:172.21.15.184:6789/0 conn(0x7f55000b2fc0 legacy=0x7f55000b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.319+0000 7f54fffff640 1 -- 172.21.15.40:0/925225963 >> v1:172.21.15.40:6789/0 conn(0x7f5500145630 legacy=0x7f55000cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.319+0000 7f54fffff640 1 -- 172.21.15.40:0/925225963 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55000d0330 con 0x7f550015f980 2024-04-26T21:47:53.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.319+0000 7f54fffff640 1 --2- 172.21.15.40:0/925225963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 0x7f55000b2a80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54f40097d0 tx=0x7f54f402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9980a65ab453104e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f54ff7fe640 1 -- 172.21.15.40:0/925225963 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4037070 con 0x7f550015f980 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f54ff7fe640 1 -- 172.21.15.40:0/925225963 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f4032440 con 0x7f550015f980 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f54ff7fe640 1 -- 172.21.15.40:0/925225963 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f40327f0 con 0x7f550015f980 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f5506b46640 1 -- 172.21.15.40:0/925225963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 msgr2=0x7f55000b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f5506b46640 1 --2- 172.21.15.40:0/925225963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 0x7f55000b2a80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54f40097d0 tx=0x7f54f402a210 comp rx=0 tx=0).stop 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f5506b46640 1 -- 172.21.15.40:0/925225963 shutdown_connections 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f5506b46640 1 --2- 172.21.15.40:0/925225963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 0x7f55000b2a80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f5506b46640 1 -- 172.21.15.40:0/925225963 >> 172.21.15.40:0/925225963 conn(0x7f550006f310 msgr2=0x7f550006f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f5506b46640 1 -- 172.21.15.40:0/925225963 shutdown_connections 2024-04-26T21:47:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.320+0000 7f5506b46640 1 -- 172.21.15.40:0/925225963 wait complete. 2024-04-26T21:47:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.321+0000 7f5506b46640 1 Processor -- start 2024-04-26T21:47:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.321+0000 7f5506b46640 1 -- start start 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f5506b46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55000b2fc0 0x7f5500154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f5506b46640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5500145630 0x7f55001e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f5506b46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 0x7f55001e5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f5506b46640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5500145a10 con 0x7f55000b2fc0 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f5506b46640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55000d08f0 con 0x7f5500145630 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f5506b46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5500145b90 con 0x7f550015f980 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f55048e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55000b2fc0 0x7f5500154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f54fffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5500145630 0x7f55001e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f55048e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55000b2fc0 0x7f5500154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51030/0 (socket says 172.21.15.40:51030) 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f55048e2640 1 -- 172.21.15.40:0/3491137780 learned_addr learned my addr 172.21.15.40:0/3491137780 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.322+0000 7f55050e3640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 0x7f55001e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f55048e2640 1 -- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 msgr2=0x7f55001e5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f55048e2640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550015f980 0x7f55001e5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f55048e2640 1 -- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5500145630 msgr2=0x7f55001e4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f55048e2640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5500145630 0x7f55001e4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f55048e2640 1 -- 172.21.15.40:0/3491137780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55001433c0 con 0x7f55000b2fc0 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f55048e2640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55000b2fc0 0x7f5500154c80 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f54f800cb70 tx=0x7f54f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8017070 con 0x7f55000b2fc0 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f800ad40 con 0x7f55000b2fc0 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8012620 con 0x7f55000b2fc0 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55001f87c0 con 0x7f55000b2fc0 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.323+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55001f8d80 con 0x7f55000b2fc0 2024-04-26T21:47:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.324+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55001f9440 con 0x7f55000b2fc0 2024-04-26T21:47:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.325+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54f801a070 con 0x7f55000b2fc0 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.325+0000 7f54fd7fa640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54ec06c8f0 0x7f54ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.325+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f54f808be70 con 0x7f55000b2fc0 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.325+0000 7f54fd7fa640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f54ec072e30 0x7f54ec075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.325+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54f8016ec0 con 0x7f54ec072e30 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.325+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f54f808c300 con 0x7f55000b2fc0 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.325+0000 7f54fffff640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54ec06c8f0 0x7f54ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.325+0000 7f55050e3640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f54ec072e30 0x7f54ec075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.326+0000 7f55050e3640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f54ec072e30 0x7f54ec075270 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.326+0000 7f54fffff640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54ec06c8f0 0x7f54ec06edb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f54f4034000 tx=0x7f54f402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.326+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 <== osd.1 v2:172.21.15.40:6800/299622497 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54f8016ec0 con 0x7f54ec072e30 2024-04-26T21:47:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.339+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 --> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5500069600 con 0x7f54ec072e30 2024-04-26T21:47:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f54fd7fa640 1 -- 172.21.15.40:0/3491137780 <== osd.1 v2:172.21.15.40:6800/299622497 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5500069600 con 0x7f54ec072e30 2024-04-26T21:47:53.341 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:47:53.341 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f54ec072e30 msgr2=0x7f54ec075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.40:6800/299622497,v1:172.21.15.40:6801/299622497] conn(0x7f54ec072e30 0x7f54ec075270 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54ec06c8f0 msgr2=0x7f54ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f54ec06c8f0 0x7f54ec06edb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f54f4034000 tx=0x7f54f402e000 comp rx=0 tx=0).stop 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55000b2fc0 msgr2=0x7f5500154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 --2- 172.21.15.40:0/3491137780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55000b2fc0 0x7f5500154c80 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f54f800cb70 tx=0x7f54f800a430 comp rx=0 tx=0).stop 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f55050e3640 1 -- 172.21.15.40:0/3491137780 reap_dead start 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 shutdown_connections 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 >> 172.21.15.40:0/3491137780 conn(0x7f550006f310 msgr2=0x7f550013b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 shutdown_connections 2024-04-26T21:47:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.340+0000 7f5506b46640 1 -- 172.21.15.40:0/3491137780 wait complete. 2024-04-26T21:47:53.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T21:47:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.452+0000 7f729b522640 1 Processor -- start 2024-04-26T21:47:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.452+0000 7f729b522640 1 -- start start 2024-04-26T21:47:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f729b522640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940d3bc0 0x7f72940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f729b522640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72940b5f20 0x7f72940b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f729b522640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72940b7e50 con 0x7f7294162880 2024-04-26T21:47:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f729b522640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f72940d0580 con 0x7f72940b5f20 2024-04-26T21:47:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f729b522640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72940d1800 con 0x7f72940d3bc0 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f7298abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940d3bc0 0x7f72940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f7299abf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72940b5f20 0x7f72940b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f7298abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940d3bc0 0x7f72940d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:51724/0 (socket says 172.21.15.40:51724) 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f7298abd640 1 -- 172.21.15.40:0/2981934269 learned_addr learned my addr 172.21.15.40:0/2981934269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.453+0000 7f72992be640 1 --1- 172.21.15.40:0/2981934269 >> v1:172.21.15.184:6789/0 conn(0x7f7294162880 0x7f72940d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:37056/0 (socket says 172.21.15.40:37056) 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.454+0000 7f7298abd640 1 -- 172.21.15.40:0/2981934269 >> v2:172.21.15.184:3300/0 conn(0x7f72940b5f20 msgr2=0x7f72940b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.454+0000 7f7298abd640 1 --2- 172.21.15.40:0/2981934269 >> v2:172.21.15.184:3300/0 conn(0x7f72940b5f20 0x7f72940b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.454+0000 7f7298abd640 1 -- 172.21.15.40:0/2981934269 >> v1:172.21.15.184:6789/0 conn(0x7f7294162880 legacy=0x7f72940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.454+0000 7f7298abd640 1 -- 172.21.15.40:0/2981934269 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72940d07d0 con 0x7f72940d3bc0 2024-04-26T21:47:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.454+0000 7f7298abd640 1 --2- 172.21.15.40:0/2981934269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940d3bc0 0x7f72940d2cb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7288009870 tx=0x7f728802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6126bcf3e7e4e4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.454+0000 7f7293fff640 1 -- 172.21.15.40:0/2981934269 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288037070 con 0x7f72940d3bc0 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f7293fff640 1 -- 172.21.15.40:0/2981934269 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7288032440 con 0x7f72940d3bc0 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f7293fff640 1 -- 172.21.15.40:0/2981934269 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288032810 con 0x7f72940d3bc0 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f729b522640 1 -- 172.21.15.40:0/2981934269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940d3bc0 msgr2=0x7f72940d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f729b522640 1 --2- 172.21.15.40:0/2981934269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940d3bc0 0x7f72940d2cb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7288009870 tx=0x7f728802a270 comp rx=0 tx=0).stop 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f729b522640 1 -- 172.21.15.40:0/2981934269 shutdown_connections 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f729b522640 1 --2- 172.21.15.40:0/2981934269 >> v2:172.21.15.184:3300/0 conn(0x7f72940b5f20 0x7f72940b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f729b522640 1 --2- 172.21.15.40:0/2981934269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940d3bc0 0x7f72940d2cb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f729b522640 1 -- 172.21.15.40:0/2981934269 >> 172.21.15.40:0/2981934269 conn(0x7f72940d4f60 msgr2=0x7f72940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f729b522640 1 -- 172.21.15.40:0/2981934269 shutdown_connections 2024-04-26T21:47:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.455+0000 7f729b522640 1 -- 172.21.15.40:0/2981934269 wait complete. 2024-04-26T21:47:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.456+0000 7f729b522640 1 Processor -- start 2024-04-26T21:47:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.456+0000 7f729b522640 1 -- start start 2024-04-26T21:47:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f729b522640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940b5f20 0x7f72940ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f729b522640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72940d3bc0 0x7f729415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f729b522640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7294162880 0x7f72940b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f729b522640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7294162c60 con 0x7f7294162880 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f729b522640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72940d0d90 con 0x7f72940d3bc0 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f729b522640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7294162de0 con 0x7f72940b5f20 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f7299abf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7294162880 0x7f72940b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f72992be640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940b5f20 0x7f72940ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f7299abf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7294162880 0x7f72940b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51036/0 (socket says 172.21.15.40:51036) 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f7299abf640 1 -- 172.21.15.40:0/1627955933 learned_addr learned my addr 172.21.15.40:0/1627955933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.457+0000 7f7298abd640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72940d3bc0 0x7f729415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f72992be640 1 -- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72940d3bc0 msgr2=0x7f729415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f72992be640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72940d3bc0 0x7f729415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f72992be640 1 -- 172.21.15.40:0/1627955933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7294162880 msgr2=0x7f72940b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f72992be640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7294162880 0x7f72940b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f72992be640 1 -- 172.21.15.40:0/1627955933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72940cd620 con 0x7f72940b5f20 2024-04-26T21:47:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f72992be640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940b5f20 0x7f72940ce6c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f728400c700 tx=0x7f728400cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f7291ffb640 1 -- 172.21.15.40:0/1627955933 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284017070 con 0x7f72940b5f20 2024-04-26T21:47:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f7291ffb640 1 -- 172.21.15.40:0/1627955933 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728400ce10 con 0x7f72940b5f20 2024-04-26T21:47:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72940cd900 con 0x7f72940b5f20 2024-04-26T21:47:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f7291ffb640 1 -- 172.21.15.40:0/1627955933 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728400a860 con 0x7f72940b5f20 2024-04-26T21:47:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.458+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f72940cdec0 con 0x7f72940b5f20 2024-04-26T21:47:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.459+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72940694f0 con 0x7f72940b5f20 2024-04-26T21:47:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.459+0000 7f7291ffb640 1 -- 172.21.15.40:0/1627955933 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f728401a070 con 0x7f72940b5f20 2024-04-26T21:47:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.460+0000 7f7291ffb640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f728006c920 0x7f728006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.460+0000 7f7291ffb640 1 -- 172.21.15.40:0/1627955933 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f728408aec0 con 0x7f72940b5f20 2024-04-26T21:47:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.460+0000 7f7298abd640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f728006c920 0x7f728006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.460+0000 7f7298abd640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f728006c920 0x7f728006ede0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f7288034000 tx=0x7f728802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.464+0000 7f7291ffb640 1 -- 172.21.15.40:0/1627955933 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7284058e50 con 0x7f72940b5f20 2024-04-26T21:47:53.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.652+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f729415e680 con 0x7f728006c920 2024-04-26T21:47:53.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.652+0000 7f7291ffb640 1 -- 172.21.15.40:0/1627955933 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f72880326a0 con 0x7f728006c920 2024-04-26T21:47:53.653 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T21:47:53.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f728006c920 msgr2=0x7f728006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f728006c920 0x7f728006ede0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f7288034000 tx=0x7f728802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940b5f20 msgr2=0x7f72940ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940b5f20 0x7f72940ce6c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f728400c700 tx=0x7f728400cbc0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 shutdown_connections 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f728006c920 0x7f728006ede0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7294162880 0x7f72940b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72940d3bc0 0x7f729415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 --2- 172.21.15.40:0/1627955933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72940b5f20 0x7f72940ce6c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.654+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 >> 172.21.15.40:0/1627955933 conn(0x7f72940d4f60 msgr2=0x7f72940d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.655+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 shutdown_connections 2024-04-26T21:47:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.655+0000 7f729b522640 1 -- 172.21.15.40:0/1627955933 wait complete. 2024-04-26T21:47:53.666 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T21:47:53.666 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:47:53.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.769+0000 7f6720151640 1 Processor -- start 2024-04-26T21:47:53.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.770+0000 7f6720151640 1 -- start start 2024-04-26T21:47:53.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.770+0000 7f6720151640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f671815f3e0 0x7f67180d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.770+0000 7f6720151640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67180d3da0 0x7f67181620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.770+0000 7f6720151640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6718160930 con 0x7f67180d31e0 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.770+0000 7f6720151640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f671815fd20 con 0x7f67180d3da0 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.770+0000 7f6720151640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6718160000 con 0x7f671815f3e0 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.770+0000 7f671e6ee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67180d3da0 0x7f67181620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.771+0000 7f671e6ee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67180d3da0 0x7f67181620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51048/0 (socket says 172.21.15.40:51048) 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.771+0000 7f671e6ee640 1 -- 172.21.15.40:0/1876354444 learned_addr learned my addr 172.21.15.40:0/1876354444 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.771+0000 7f671e6ee640 1 -- 172.21.15.40:0/1876354444 >> v2:172.21.15.184:3300/0 conn(0x7f671815f3e0 msgr2=0x7f67180d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.771+0000 7f671e6ee640 1 --2- 172.21.15.40:0/1876354444 >> v2:172.21.15.184:3300/0 conn(0x7f671815f3e0 0x7f67180d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.771+0000 7f671e6ee640 1 -- 172.21.15.40:0/1876354444 >> v1:172.21.15.184:6789/0 conn(0x7f67180d31e0 legacy=0x7f67180d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.771+0000 7f671e6ee640 1 -- 172.21.15.40:0/1876354444 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67181602e0 con 0x7f67180d3da0 2024-04-26T21:47:53.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.771+0000 7f671e6ee640 1 --2- 172.21.15.40:0/1876354444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67180d3da0 0x7f67181620a0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f671808d480 tx=0x7f670802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb737b52e7a77ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f671ceeb640 1 -- 172.21.15.40:0/1876354444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708036070 con 0x7f67180d3da0 2024-04-26T21:47:53.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f671ceeb640 1 -- 172.21.15.40:0/1876354444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6708031600 con 0x7f67180d3da0 2024-04-26T21:47:53.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f671ceeb640 1 -- 172.21.15.40:0/1876354444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670802ae60 con 0x7f67180d3da0 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f6720151640 1 -- 172.21.15.40:0/1876354444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67180d3da0 msgr2=0x7f67181620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f6720151640 1 --2- 172.21.15.40:0/1876354444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67180d3da0 0x7f67181620a0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f671808d480 tx=0x7f670802a680 comp rx=0 tx=0).stop 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f6720151640 1 -- 172.21.15.40:0/1876354444 shutdown_connections 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f6720151640 1 --2- 172.21.15.40:0/1876354444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67180d3da0 0x7f67181620a0 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f6720151640 1 --2- 172.21.15.40:0/1876354444 >> v2:172.21.15.184:3300/0 conn(0x7f671815f3e0 0x7f67180d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f6720151640 1 -- 172.21.15.40:0/1876354444 >> 172.21.15.40:0/1876354444 conn(0x7f671809e3c0 msgr2=0x7f67180d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f6720151640 1 -- 172.21.15.40:0/1876354444 shutdown_connections 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.772+0000 7f6720151640 1 -- 172.21.15.40:0/1876354444 wait complete. 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.773+0000 7f6720151640 1 Processor -- start 2024-04-26T21:47:53.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.773+0000 7f6720151640 1 -- start start 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f6720151640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67180d31e0 0x7f67181e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f6720151640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67180d3da0 0x7f67181f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f6720151640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671815f3e0 0x7f67181e7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f6720151640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f671815f7c0 con 0x7f671815f3e0 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f6720151640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6718160d40 con 0x7f67180d31e0 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f6720151640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f671815f940 con 0x7f67180d3da0 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f671e6ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671815f3e0 0x7f67181e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f671e6ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671815f3e0 0x7f67181e7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51054/0 (socket says 172.21.15.40:51054) 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f671e6ee640 1 -- 172.21.15.40:0/1591331864 learned_addr learned my addr 172.21.15.40:0/1591331864 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f671deed640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67180d31e0 0x7f67181e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.774+0000 7f671e6ee640 1 -- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67180d3da0 msgr2=0x7f67181f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f671e6ee640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67180d3da0 0x7f67181f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f671e6ee640 1 -- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67180d31e0 msgr2=0x7f67181e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f671e6ee640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67180d31e0 0x7f67181e5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f671e6ee640 1 -- 172.21.15.40:0/1591331864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67181f9a30 con 0x7f671815f3e0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f671deed640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67180d31e0 0x7f67181e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f671e6ee640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671815f3e0 0x7f67181e7530 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f6708034040 tx=0x7f6708004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708041020 con 0x7f671815f3e0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670802ae60 con 0x7f671815f3e0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708036040 con 0x7f671815f3e0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67181e61b0 con 0x7f671815f3e0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.775+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67181e6770 con 0x7f671815f3e0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.776+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67181e6e20 con 0x7f671815f3e0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.776+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6708044020 con 0x7f671815f3e0 2024-04-26T21:47:53.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.777+0000 7f67167fc640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670406c920 0x7f670406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.777+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f67080ac570 con 0x7f671815f3e0 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.777+0000 7f67167fc640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f6704072e60 0x7f67040752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.777+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6704075970 con 0x7f6704072e60 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.777+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f67080b3050 con 0x7f671815f3e0 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.777+0000 7f671deed640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670406c920 0x7f670406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.777+0000 7f671d6ec640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f6704072e60 0x7f67040752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.777+0000 7f671d6ec640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f6704072e60 0x7f67040752a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.778+0000 7f671deed640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670406c920 0x7f670406ede0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f671000a870 tx=0x7f6710008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.778+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 <== osd.2 v2:172.21.15.40:6811/3280366756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6704075970 con 0x7f6704072e60 2024-04-26T21:47:53.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.791+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 --> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6718069600 con 0x7f6704072e60 2024-04-26T21:47:53.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.791+0000 7f67167fc640 1 -- 172.21.15.40:0/1591331864 <== osd.2 v2:172.21.15.40:6811/3280366756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6718069600 con 0x7f6704072e60 2024-04-26T21:47:53.792 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:47:53.792 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f6704072e60 msgr2=0x7f67040752a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.40:6811/3280366756,v1:172.21.15.40:6812/3280366756] conn(0x7f6704072e60 0x7f67040752a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670406c920 msgr2=0x7f670406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f670406c920 0x7f670406ede0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f671000a870 tx=0x7f6710008040 comp rx=0 tx=0).stop 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671815f3e0 msgr2=0x7f67181e7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 --2- 172.21.15.40:0/1591331864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671815f3e0 0x7f67181e7530 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f6708034040 tx=0x7f6708004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f671e6ee640 1 -- 172.21.15.40:0/1591331864 reap_dead start 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 shutdown_connections 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 >> 172.21.15.40:0/1591331864 conn(0x7f671809e3c0 msgr2=0x7f67180c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 shutdown_connections 2024-04-26T21:47:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.792+0000 7f6720151640 1 -- 172.21.15.40:0/1591331864 wait complete. 2024-04-26T21:47:53.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T21:47:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.903+0000 7f514e359640 1 Processor -- start 2024-04-26T21:47:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.903+0000 7f514e359640 1 -- start start 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.904+0000 7f514e359640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51480d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.904+0000 7f514e359640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 0x7f51480d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.904+0000 7f514e359640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51480d25a0 con 0x7f51480d2130 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.904+0000 7f514e359640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f51480d0e20 con 0x7f51480d1770 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.904+0000 7f514e359640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51480b1d70 con 0x7f514815f3a0 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.904+0000 7f514c8f6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51480d2130 0x7f51480d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54532/0 (socket says 172.21.15.40:54532) 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.904+0000 7f514c8f6640 1 -- 172.21.15.40:0/4227336811 learned_addr learned my addr 172.21.15.40:0/4227336811 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5147fff640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51480d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f51477fe640 1 --2- 172.21.15.40:0/4227336811 >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 0x7f51480d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736885040 0 0) 0x7f5140001490 con 0x7f51480d2130 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5130003520 con 0x7f51480d2130 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3129249021 0 0) 0x7f5130003520 con 0x7f51480d2130 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 msgr2=0x7f51480d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51480d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 msgr2=0x7f51480d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 --2- 172.21.15.40:0/4227336811 >> v2:172.21.15.184:3300/0 conn(0x7f51480d1770 0x7f51480d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51480b2050 con 0x7f51480d2130 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f5140001bc0 con 0x7f51480d2130 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 >> v1:172.21.15.40:6789/0 conn(0x7f51480d2130 legacy=0x7f51480d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51300039b0 0x7f5130003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5130004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51480d0e20 con 0x7f5130004800 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51480b1d70 con 0x7f51300039b0 2024-04-26T21:47:53.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f513000b9c0 con 0x7f514815f3a0 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f51477fe640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5130004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f5147fff640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51300039b0 0x7f5130003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f514c8f6640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f51477fe640 1 -- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 msgr2=0x7f51300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.905+0000 7f51477fe640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f514815f3a0 0x7f51300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.906+0000 7f51477fe640 1 -- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51300039b0 msgr2=0x7f5130003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.906+0000 7f51477fe640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51300039b0 0x7f5130003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.906+0000 7f51477fe640 1 -- 172.21.15.40:0/4227336811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5130003520 con 0x7f5130004800 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.906+0000 7f5147fff640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51300039b0 0x7f5130003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.906+0000 7f51477fe640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5130004c80 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f513c009ac0 tx=0x7f5130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dcf68532070d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.906+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c036070 con 0x7f5130004800 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.906+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513c031400 con 0x7f5130004800 2024-04-26T21:47:53.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.906+0000 7f5146ffd640 1 -- 172.21.15.40:0/4227336811 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c009690 con 0x7f5130004800 2024-04-26T21:47:53.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.907+0000 7f514e359640 1 -- 172.21.15.40:0/4227336811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 msgr2=0x7f5130004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.907+0000 7f514e359640 1 --2- 172.21.15.40:0/4227336811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5130004c80 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f513c009ac0 tx=0x7f5130005830 comp rx=0 tx=0).stop 2024-04-26T21:47:53.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.907+0000 7f514c8f6640 1 -- 172.21.15.40:0/4227336811 reap_dead start 2024-04-26T21:47:53.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.907+0000 7f514e359640 1 -- 172.21.15.40:0/4227336811 shutdown_connections 2024-04-26T21:47:53.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.907+0000 7f514e359640 1 -- 172.21.15.40:0/4227336811 >> 172.21.15.40:0/4227336811 conn(0x7f51480d4f90 msgr2=0x7f51480d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:53.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.907+0000 7f514e359640 1 -- 172.21.15.40:0/4227336811 shutdown_connections 2024-04-26T21:47:53.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.907+0000 7f514e359640 1 -- 172.21.15.40:0/4227336811 wait complete. 2024-04-26T21:47:53.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.908+0000 7f514e359640 1 Processor -- start 2024-04-26T21:47:53.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.908+0000 7f514e359640 1 -- start start 2024-04-26T21:47:53.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f514e359640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5148159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f514e359640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51480cd4c0 0x7f51480ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f514e359640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51480ca940 0x7f514815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f514e359640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f514815f900 con 0x7f5130004800 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f514e359640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f514815f780 con 0x7f51480ca940 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f514e359640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51480b3860 con 0x7f51480cd4c0 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f5147fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5148159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f5147fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5148159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51066/0 (socket says 172.21.15.40:51066) 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f5147fff640 1 -- 172.21.15.40:0/1896149746 learned_addr learned my addr 172.21.15.40:0/1896149746 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f514c8f6640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51480ca940 0x7f514815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f51477fe640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51480cd4c0 0x7f51480ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f5147fff640 1 -- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51480cd4c0 msgr2=0x7f51480ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f5147fff640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51480cd4c0 0x7f51480ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f5147fff640 1 -- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51480ca940 msgr2=0x7f514815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f5147fff640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51480ca940 0x7f514815a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:53.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.909+0000 7f5147fff640 1 -- 172.21.15.40:0/1896149746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51480cc420 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.910+0000 7f5147fff640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5148159020 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f5138011a70 tx=0x7f5138011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.910+0000 7f5144ff9640 1 -- 172.21.15.40:0/1896149746 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513800faf0 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.910+0000 7f5144ff9640 1 -- 172.21.15.40:0/1896149746 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513800fc90 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.910+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51480cc700 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.910+0000 7f5144ff9640 1 -- 172.21.15.40:0/1896149746 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138014440 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.910+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51480cccc0 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.911+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51480694f0 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.911+0000 7f5144ff9640 1 -- 172.21.15.40:0/1896149746 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5138012040 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.912+0000 7f5144ff9640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f513406c8f0 0x7f513406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.912+0000 7f5144ff9640 1 -- 172.21.15.40:0/1896149746 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f513808c020 con 0x7f5130004800 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.912+0000 7f51477fe640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f513406c8f0 0x7f513406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.912+0000 7f51477fe640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f513406c8f0 0x7f513406edb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f513c033000 tx=0x7f513c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:53.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:53.916+0000 7f5144ff9640 1 -- 172.21.15.40:0/1896149746 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5138019030 con 0x7f5130004800 2024-04-26T21:47:54.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.106+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f5148159560 con 0x7f513406c8f0 2024-04-26T21:47:54.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.106+0000 7f5144ff9640 1 -- 172.21.15.40:0/1896149746 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f513c0040c0 con 0x7f513406c8f0 2024-04-26T21:47:54.107 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T21:47:54.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f513406c8f0 msgr2=0x7f513406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f513406c8f0 0x7f513406edb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f513c033000 tx=0x7f513c008030 comp rx=0 tx=0).stop 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 msgr2=0x7f5148159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5148159020 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f5138011a70 tx=0x7f5138011e70 comp rx=0 tx=0).stop 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 shutdown_connections 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f513406c8f0 0x7f513406edb0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51480ca940 0x7f514815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51480cd4c0 0x7f51480ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 --2- 172.21.15.40:0/1896149746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5130004800 0x7f5148159020 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 >> 172.21.15.40:0/1896149746 conn(0x7f51480d4f90 msgr2=0x7f514806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.108+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 shutdown_connections 2024-04-26T21:47:54.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.109+0000 7f514e359640 1 -- 172.21.15.40:0/1896149746 wait complete. 2024-04-26T21:47:54.120 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T21:47:54.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:47:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.224+0000 7fb02d083640 1 Processor -- start 2024-04-26T21:47:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.224+0000 7fb02d083640 1 -- start start 2024-04-26T21:47:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.225+0000 7fb02d083640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02815f3e0 0x7fb0280d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.225+0000 7fb02d083640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb028160930 con 0x7fb0280d31e0 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.225+0000 7fb02d083640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb02815fd20 con 0x7fb0280d3da0 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.225+0000 7fb02d083640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb028160000 con 0x7fb02815f3e0 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.225+0000 7fb02659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb0280d31e0 0x7fb0280d3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54542/0 (socket says 172.21.15.40:54542) 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.225+0000 7fb02659c640 1 -- 172.21.15.40:0/92748056 learned_addr learned my addr 172.21.15.40:0/92748056 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.225+0000 7fb026d9d640 1 --2- 172.21.15.40:0/92748056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02815f3e0 0x7fb0280d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.225+0000 7fb025d9b640 1 -- 172.21.15.40:0/92748056 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4255907979 0 0) 0x7fb028160930 con 0x7fb0280d31e0 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.226+0000 7fb025d9b640 1 -- 172.21.15.40:0/92748056 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb014003520 con 0x7fb0280d31e0 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.226+0000 7fb026d9d640 1 -- 172.21.15.40:0/92748056 >> v1:172.21.15.184:6789/0 conn(0x7fb0280d3da0 legacy=0x7fb0281620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.226+0000 7fb026d9d640 1 -- 172.21.15.40:0/92748056 >> v1:172.21.15.40:6789/0 conn(0x7fb0280d31e0 legacy=0x7fb0280d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.226+0000 7fb025d9b640 1 -- 172.21.15.40:0/92748056 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148801939 0 0) 0x7fb02815fd20 con 0x7fb0280d3da0 2024-04-26T21:47:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.226+0000 7fb026d9d640 1 -- 172.21.15.40:0/92748056 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0281602e0 con 0x7fb02815f3e0 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.226+0000 7fb026d9d640 1 --2- 172.21.15.40:0/92748056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02815f3e0 0x7fb0280d2ca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0200097d0 tx=0x7fb02002a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb26beed4cea3ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb025d9b640 1 -- 172.21.15.40:0/92748056 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020037070 con 0x7fb02815f3e0 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb025d9b640 1 -- 172.21.15.40:0/92748056 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb02002ab70 con 0x7fb02815f3e0 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb025d9b640 1 -- 172.21.15.40:0/92748056 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100015f0 con 0x7fb02815f3e0 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb02d083640 1 -- 172.21.15.40:0/92748056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02815f3e0 msgr2=0x7fb0280d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb02d083640 1 --2- 172.21.15.40:0/92748056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02815f3e0 0x7fb0280d2ca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0200097d0 tx=0x7fb02002a0e0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb02d083640 1 -- 172.21.15.40:0/92748056 shutdown_connections 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb02d083640 1 --2- 172.21.15.40:0/92748056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb02815f3e0 0x7fb0280d2ca0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb02d083640 1 -- 172.21.15.40:0/92748056 >> 172.21.15.40:0/92748056 conn(0x7fb02809e3c0 msgr2=0x7fb0280d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb02d083640 1 -- 172.21.15.40:0/92748056 shutdown_connections 2024-04-26T21:47:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.227+0000 7fb02d083640 1 -- 172.21.15.40:0/92748056 wait complete. 2024-04-26T21:47:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02d083640 1 Processor -- start 2024-04-26T21:47:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02d083640 1 -- start start 2024-04-26T21:47:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02d083640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0280d31e0 0x7fb02815c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02d083640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0280d3da0 0x7fb0280ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02d083640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb02815f3e0 0x7fb02806ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02d083640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0280d1040 con 0x7fb0280d3da0 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02d083640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb028160d40 con 0x7fb02815f3e0 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02d083640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb02815f7c0 con 0x7fb0280d31e0 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0280d3da0 0x7fb0280ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0280d3da0 0x7fb0280ba7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51068/0 (socket says 172.21.15.40:51068) 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02659c640 1 -- 172.21.15.40:0/2487577357 learned_addr learned my addr 172.21.15.40:0/2487577357 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb026d9d640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0280d31e0 0x7fb02815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.229+0000 7fb02759e640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb02815f3e0 0x7fb02806ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb02659c640 1 -- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0280d31e0 msgr2=0x7fb02815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb02659c640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0280d31e0 0x7fb02815c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb02659c640 1 -- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb02815f3e0 msgr2=0x7fb02806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb02659c640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb02815f3e0 0x7fb02806ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb02659c640 1 -- 172.21.15.40:0/2487577357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02815cab0 con 0x7fb0280d3da0 2024-04-26T21:47:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb02659c640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0280d3da0 0x7fb0280ba7a0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c780 tx=0x7fb01000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010005ad0 con 0x7fb0280d3da0 2024-04-26T21:47:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb010005c70 con 0x7fb0280d3da0 2024-04-26T21:47:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010007270 con 0x7fb0280d3da0 2024-04-26T21:47:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0280cc1a0 con 0x7fb0280d3da0 2024-04-26T21:47:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.230+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0280cc760 con 0x7fb0280d3da0 2024-04-26T21:47:54.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.231+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb028069600 con 0x7fb0280d3da0 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.232+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb010007410 con 0x7fb0280d3da0 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.232+0000 7fb01f7fe640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb00806c8f0 0x7fb00806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.232+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fb01008b7a0 con 0x7fb0280d3da0 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.232+0000 7fb01f7fe640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb008072e30 0x7fb008075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.232+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb010020e50 con 0x7fb008072e30 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.232+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fb0100902c0 con 0x7fb0280d3da0 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.232+0000 7fb026d9d640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb00806c8f0 0x7fb00806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.233+0000 7fb02759e640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb008072e30 0x7fb008075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.233+0000 7fb02759e640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb008072e30 0x7fb008075270 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.233+0000 7fb026d9d640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb00806c8f0 0x7fb00806edb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb020034000 tx=0x7fb02002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.233+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 <== osd.3 v2:172.21.15.40:6824/4238362919 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb010020e50 con 0x7fb008072e30 2024-04-26T21:47:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 --> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb0280b8060 con 0x7fb008072e30 2024-04-26T21:47:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb01f7fe640 1 -- 172.21.15.40:0/2487577357 <== osd.3 v2:172.21.15.40:6824/4238362919 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb0280b8060 con 0x7fb008072e30 2024-04-26T21:47:54.248 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:47:54.248 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb008072e30 msgr2=0x7fb008075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.40:6824/4238362919,v1:172.21.15.40:6825/4238362919] conn(0x7fb008072e30 0x7fb008075270 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb00806c8f0 msgr2=0x7fb00806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb00806c8f0 0x7fb00806edb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb020034000 tx=0x7fb02002e000 comp rx=0 tx=0).stop 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0280d3da0 msgr2=0x7fb0280ba7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 --2- 172.21.15.40:0/2487577357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0280d3da0 0x7fb0280ba7a0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c780 tx=0x7fb01000cc40 comp rx=0 tx=0).stop 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02759e640 1 -- 172.21.15.40:0/2487577357 reap_dead start 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 shutdown_connections 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.247+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 >> 172.21.15.40:0/2487577357 conn(0x7fb02809e3c0 msgr2=0x7fb028160ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.248+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 shutdown_connections 2024-04-26T21:47:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.248+0000 7fb02d083640 1 -- 172.21.15.40:0/2487577357 wait complete. 2024-04-26T21:47:54.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T21:47:54.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.359+0000 7fb15474e640 1 Processor -- start 2024-04-26T21:47:54.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.359+0000 7fb15474e640 1 -- start start 2024-04-26T21:47:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb15474e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb14c15d1e0 0x7fb14c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb15474e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0d0f20 0x7fb14c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb15474e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb14c0ce840 con 0x7fb14c0d18c0 2024-04-26T21:47:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb15474e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb14c0cede0 con 0x7fb14c15d1e0 2024-04-26T21:47:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb15474e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c070650 con 0x7fb14c0d0f20 2024-04-26T21:47:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb152ceb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb14c0d18c0 0x7fb14c0cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54550/0 (socket says 172.21.15.40:54550) 2024-04-26T21:47:54.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb152ceb640 1 -- 172.21.15.40:0/909198724 learned_addr learned my addr 172.21.15.40:0/909198724 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924846051 0 0) 0x7fb14c0ce840 con 0x7fb14c0d18c0 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb138003520 con 0x7fb14c0d18c0 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb151ce9640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0d0f20 0x7fb14c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1524ea640 1 --2- 172.21.15.40:0/909198724 >> v2:172.21.15.184:3300/0 conn(0x7fb14c15d1e0 0x7fb14c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2505033324 0 0) 0x7fb138003520 con 0x7fb14c0d18c0 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0d0f20 msgr2=0x7fb14c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0d0f20 0x7fb14c0d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 >> v2:172.21.15.184:3300/0 conn(0x7fb14c15d1e0 msgr2=0x7fb14c0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 --2- 172.21.15.40:0/909198724 >> v2:172.21.15.184:3300/0 conn(0x7fb14c15d1e0 0x7fb14c0d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c0ceb00 con 0x7fb14c0d18c0 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.360+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fb1440019f0 con 0x7fb14c0d18c0 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 >> v1:172.21.15.40:6789/0 conn(0x7fb14c0d18c0 legacy=0x7fb14c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb1514e8640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1380039b0 0x7fb138003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb1514e8640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb138004800 0x7fb138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb1514e8640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0d0f20 0x7fb1380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c0cede0 con 0x7fb138004800 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c070650 con 0x7fb1380039b0 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb13800b9c0 con 0x7fb14c0d0f20 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb151ce9640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb138004800 0x7fb138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb1524ea640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1380039b0 0x7fb138003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb152ceb640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0d0f20 0x7fb1380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb151ce9640 1 -- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0d0f20 msgr2=0x7fb1380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb151ce9640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0d0f20 0x7fb1380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb151ce9640 1 -- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1380039b0 msgr2=0x7fb138003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb151ce9640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1380039b0 0x7fb138003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.361+0000 7fb151ce9640 1 -- 172.21.15.40:0/909198724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb138003520 con 0x7fb138004800 2024-04-26T21:47:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb151ce9640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb138004800 0x7fb138004c80 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009a90 tx=0x7fb138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa81462f7075a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c036070 con 0x7fb138004800 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13c031400 con 0x7fb138004800 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb1514e8640 1 -- 172.21.15.40:0/909198724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c009690 con 0x7fb138004800 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb15474e640 1 -- 172.21.15.40:0/909198724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb138004800 msgr2=0x7fb138004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb15474e640 1 --2- 172.21.15.40:0/909198724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb138004800 0x7fb138004c80 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009a90 tx=0x7fb138005830 comp rx=0 tx=0).stop 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb152ceb640 1 -- 172.21.15.40:0/909198724 reap_dead start 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb15474e640 1 -- 172.21.15.40:0/909198724 shutdown_connections 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb15474e640 1 -- 172.21.15.40:0/909198724 >> 172.21.15.40:0/909198724 conn(0x7fb14c0d2da0 msgr2=0x7fb14c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.362+0000 7fb15474e640 1 -- 172.21.15.40:0/909198724 shutdown_connections 2024-04-26T21:47:54.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.363+0000 7fb15474e640 1 -- 172.21.15.40:0/909198724 wait complete. 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb15474e640 1 Processor -- start 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb15474e640 1 -- start start 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb15474e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb138004800 0x7fb14c15c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb15474e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1f3a60 0x7fb14c09f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb15474e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c1f3ee0 0x7fb14c09fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb15474e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c071af0 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb15474e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c0ce4c0 con 0x7fb14c1f3a60 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb15474e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c071c70 con 0x7fb138004800 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.364+0000 7fb152ceb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c1f3ee0 0x7fb14c09fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb152ceb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c1f3ee0 0x7fb14c09fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51078/0 (socket says 172.21.15.40:51078) 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb152ceb640 1 -- 172.21.15.40:0/2167469160 learned_addr learned my addr 172.21.15.40:0/2167469160 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb1524ea640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb138004800 0x7fb14c15c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb151ce9640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1f3a60 0x7fb14c09f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb152ceb640 1 -- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb138004800 msgr2=0x7fb14c15c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb152ceb640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb138004800 0x7fb14c15c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb152ceb640 1 -- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1f3a60 msgr2=0x7fb14c09f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb152ceb640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1f3a60 0x7fb14c09f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb152ceb640 1 -- 172.21.15.40:0/2167469160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c09d2c0 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb152ceb640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c1f3ee0 0x7fb14c09fe10 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7fb144001330 tx=0x7fb144007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb142ffd640 1 -- 172.21.15.40:0/2167469160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14400ab60 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb142ffd640 1 -- 172.21.15.40:0/2167469160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb144019070 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c09d5a0 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb142ffd640 1 -- 172.21.15.40:0/2167469160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440133f0 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.365+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c09db60 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.366+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c0694f0 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.367+0000 7fb142ffd640 1 -- 172.21.15.40:0/2167469160 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb144013590 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.367+0000 7fb142ffd640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb13006c920 0x7fb13006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.367+0000 7fb142ffd640 1 -- 172.21.15.40:0/2167469160 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fb14408bea0 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.368+0000 7fb1524ea640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb13006c920 0x7fb13006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.368+0000 7fb1524ea640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb13006c920 0x7fb13006ede0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb148009bc0 tx=0x7fb148009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.372+0000 7fb142ffd640 1 -- 172.21.15.40:0/2167469160 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb14405a3f0 con 0x7fb14c1f3ee0 2024-04-26T21:47:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.558+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb14c0b0340 con 0x7fb13006c920 2024-04-26T21:47:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.559+0000 7fb142ffd640 1 -- 172.21.15.40:0/2167469160 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb14c0b0340 con 0x7fb13006c920 2024-04-26T21:47:54.560 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T21:47:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb13006c920 msgr2=0x7fb13006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb13006c920 0x7fb13006ede0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb148009bc0 tx=0x7fb148009190 comp rx=0 tx=0).stop 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c1f3ee0 msgr2=0x7fb14c09fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c1f3ee0 0x7fb14c09fe10 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7fb144001330 tx=0x7fb144007a60 comp rx=0 tx=0).stop 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 shutdown_connections 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb13006c920 0x7fb13006ede0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c1f3ee0 0x7fb14c09fe10 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1f3a60 0x7fb14c09f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 --2- 172.21.15.40:0/2167469160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb138004800 0x7fb14c15c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 >> 172.21.15.40:0/2167469160 conn(0x7fb14c0d2da0 msgr2=0x7fb14c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 shutdown_connections 2024-04-26T21:47:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.561+0000 7fb15474e640 1 -- 172.21.15.40:0/2167469160 wait complete. 2024-04-26T21:47:54.573 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T21:47:54.573 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:47:54.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.677+0000 7ff74f589640 1 Processor -- start 2024-04-26T21:47:54.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.677+0000 7ff74f589640 1 -- start start 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.678+0000 7ff74f589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 0x7ff7480b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.678+0000 7ff74f589640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7480d0000 con 0x7ff7480d1410 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.678+0000 7ff74f589640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7480d02e0 con 0x7ff7480b5650 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.678+0000 7ff74f589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7480cd9c0 con 0x7ff7481605d0 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.678+0000 7ff74db26640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff7480d1410 0x7ff7480ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54554/0 (socket says 172.21.15.40:54554) 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.678+0000 7ff74db26640 1 -- 172.21.15.40:0/1089212127 learned_addr learned my addr 172.21.15.40:0/1089212127 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.678+0000 7ff74d325640 1 --2- 172.21.15.40:0/1089212127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 0x7ff7480b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.679+0000 7ff73bfff640 1 -- 172.21.15.40:0/1089212127 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268318560 0 0) 0x7ff7480d0000 con 0x7ff7480d1410 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.679+0000 7ff73bfff640 1 -- 172.21.15.40:0/1089212127 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff734003520 con 0x7ff7480d1410 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.679+0000 7ff73bfff640 1 -- 172.21.15.40:0/1089212127 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789202090 0 0) 0x7ff7480d02e0 con 0x7ff7480b5650 2024-04-26T21:47:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.679+0000 7ff73bfff640 1 -- 172.21.15.40:0/1089212127 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7480d0000 con 0x7ff7480b5650 2024-04-26T21:47:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.679+0000 7ff74d325640 1 -- 172.21.15.40:0/1089212127 >> v1:172.21.15.184:6789/0 conn(0x7ff7480b5650 legacy=0x7ff7480d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.679+0000 7ff74d325640 1 -- 172.21.15.40:0/1089212127 >> v1:172.21.15.40:6789/0 conn(0x7ff7480d1410 legacy=0x7ff7480ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.679+0000 7ff74d325640 1 -- 172.21.15.40:0/1089212127 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7480d05c0 con 0x7ff7481605d0 2024-04-26T21:47:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.679+0000 7ff74d325640 1 --2- 172.21.15.40:0/1089212127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 0x7ff7480b2710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097d0 tx=0x7ff73c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9df39bb9511ee7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff73bfff640 1 -- 172.21.15.40:0/1089212127 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c037070 con 0x7ff7481605d0 2024-04-26T21:47:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff73bfff640 1 -- 172.21.15.40:0/1089212127 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c032440 con 0x7ff7481605d0 2024-04-26T21:47:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff73bfff640 1 -- 172.21.15.40:0/1089212127 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c032810 con 0x7ff7481605d0 2024-04-26T21:47:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff74f589640 1 -- 172.21.15.40:0/1089212127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 msgr2=0x7ff7480b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff74f589640 1 --2- 172.21.15.40:0/1089212127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 0x7ff7480b2710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097d0 tx=0x7ff73c02a210 comp rx=0 tx=0).stop 2024-04-26T21:47:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff74f589640 1 -- 172.21.15.40:0/1089212127 shutdown_connections 2024-04-26T21:47:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff74f589640 1 --2- 172.21.15.40:0/1089212127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 0x7ff7480b2710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff74f589640 1 -- 172.21.15.40:0/1089212127 >> 172.21.15.40:0/1089212127 conn(0x7ff7480b49e0 msgr2=0x7ff7480b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff74f589640 1 -- 172.21.15.40:0/1089212127 shutdown_connections 2024-04-26T21:47:54.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.680+0000 7ff74f589640 1 -- 172.21.15.40:0/1089212127 wait complete. 2024-04-26T21:47:54.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.681+0000 7ff74f589640 1 Processor -- start 2024-04-26T21:47:54.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74f589640 1 -- start start 2024-04-26T21:47:54.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74f589640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7480b5650 0x7ff74815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74f589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7480d1410 0x7ff7480ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74f589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 0x7ff7480cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74f589640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7480cdc80 con 0x7ff7480b5650 2024-04-26T21:47:54.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74f589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7480d08a0 con 0x7ff7480d1410 2024-04-26T21:47:54.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74f589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7480d18a0 con 0x7ff7481605d0 2024-04-26T21:47:54.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74d325640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7480b5650 0x7ff74815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74d325640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7480b5650 0x7ff74815eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51092/0 (socket says 172.21.15.40:51092) 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.682+0000 7ff74d325640 1 -- 172.21.15.40:0/1774879382 learned_addr learned my addr 172.21.15.40:0/1774879382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74db26640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 0x7ff7480cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74cb24640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7480d1410 0x7ff7480ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74d325640 1 -- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 msgr2=0x7ff7480cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74d325640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7481605d0 0x7ff7480cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74d325640 1 -- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7480d1410 msgr2=0x7ff7480ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74d325640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7480d1410 0x7ff7480ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74d325640 1 -- 172.21.15.40:0/1774879382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7481e9870 con 0x7ff7480b5650 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74d325640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7480b5650 0x7ff74815eea0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097d0 tx=0x7ff73c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c041070 con 0x7ff7480b5650 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c03c870 con 0x7ff7480b5650 2024-04-26T21:47:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7481e8f30 con 0x7ff7480b5650 2024-04-26T21:47:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c037040 con 0x7ff7480b5650 2024-04-26T21:47:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.683+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74815e6a0 con 0x7ff7480b5650 2024-04-26T21:47:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.684+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7481e9210 con 0x7ff7480b5650 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.685+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff73c044070 con 0x7ff7480b5650 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.685+0000 7ff739ffb640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff72c06c920 0x7ff72c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.685+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7ff73c0abee0 con 0x7ff7480b5650 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.685+0000 7ff739ffb640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7ff72c072e60 0x7ff72c0752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.685+0000 7ff74cb24640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff72c06c920 0x7ff72c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.685+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff73c00af30 con 0x7ff72c072e60 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.685+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7ff73c0ae500 con 0x7ff7480b5650 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.686+0000 7ff74db26640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7ff72c072e60 0x7ff72c0752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.686+0000 7ff74cb24640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff72c06c920 0x7ff72c06ede0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff744007fd0 tx=0x7ff744009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.686+0000 7ff74db26640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7ff72c072e60 0x7ff72c0752a0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.687+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 <== osd.4 v2:172.21.15.184:6800/2829090873 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff73c00af30 con 0x7ff72c072e60 2024-04-26T21:47:54.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.700+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 --> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff748069600 con 0x7ff72c072e60 2024-04-26T21:47:54.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff739ffb640 1 -- 172.21.15.40:0/1774879382 <== osd.4 v2:172.21.15.184:6800/2829090873 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff748069600 con 0x7ff72c072e60 2024-04-26T21:47:54.702 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:47:54.702 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7ff72c072e60 msgr2=0x7ff72c0752a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6800/2829090873,v1:172.21.15.184:6801/2829090873] conn(0x7ff72c072e60 0x7ff72c0752a0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff72c06c920 msgr2=0x7ff72c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7ff72c06c920 0x7ff72c06ede0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff744007fd0 tx=0x7ff744009040 comp rx=0 tx=0).stop 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7480b5650 msgr2=0x7ff74815eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 --2- 172.21.15.40:0/1774879382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7480b5650 0x7ff74815eea0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097d0 tx=0x7ff73c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74db26640 1 -- 172.21.15.40:0/1774879382 reap_dead start 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 shutdown_connections 2024-04-26T21:47:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 >> 172.21.15.40:0/1774879382 conn(0x7ff7480b49e0 msgr2=0x7ff7480b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 shutdown_connections 2024-04-26T21:47:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.701+0000 7ff74f589640 1 -- 172.21.15.40:0/1774879382 wait complete. 2024-04-26T21:47:54.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T21:47:54.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.813+0000 7f3f3d028640 1 Processor -- start 2024-04-26T21:47:54.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.813+0000 7f3f3d028640 1 -- start start 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3d028640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f380d1380 0x7f3f380d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3d028640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f38070ae0 0x7f3f38070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3d028640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f380d1ce0 con 0x7f3f3815d1e0 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3d028640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3f380cede0 con 0x7f3f38070ae0 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3d028640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3f38070650 con 0x7f3f380d1380 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f38070ae0 0x7f3f38070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f38070ae0 0x7f3f38070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51094/0 (socket says 172.21.15.40:51094) 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3759e640 1 -- 172.21.15.40:0/2930953743 learned_addr learned my addr 172.21.15.40:0/2930953743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f2ffff640 1 --2- 172.21.15.40:0/2930953743 >> v2:172.21.15.184:3300/0 conn(0x7f3f380d1380 0x7f3f380d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3759e640 1 -- 172.21.15.40:0/2930953743 >> v2:172.21.15.184:3300/0 conn(0x7f3f380d1380 msgr2=0x7f3f380d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3759e640 1 --2- 172.21.15.40:0/2930953743 >> v2:172.21.15.184:3300/0 conn(0x7f3f380d1380 0x7f3f380d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3759e640 1 -- 172.21.15.40:0/2930953743 >> v1:172.21.15.184:6789/0 conn(0x7f3f3815d1e0 legacy=0x7f3f380d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f3759e640 1 -- 172.21.15.40:0/2930953743 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f380ce840 con 0x7f3f38070ae0 2024-04-26T21:47:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.814+0000 7f3f2ffff640 1 --2- 172.21.15.40:0/2930953743 >> v2:172.21.15.184:3300/0 conn(0x7f3f380d1380 0x7f3f380d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.815+0000 7f3f3759e640 1 --2- 172.21.15.40:0/2930953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38070ae0 0x7f3f38070f50 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f3f380c64b0 tx=0x7f3f3002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d30a2b6b354c89f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.815+0000 7f3f3659c640 1 -- 172.21.15.40:0/2930953743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30036070 con 0x7f3f38070ae0 2024-04-26T21:47:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.815+0000 7f3f3659c640 1 -- 172.21.15.40:0/2930953743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30031600 con 0x7f3f38070ae0 2024-04-26T21:47:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.815+0000 7f3f3659c640 1 -- 172.21.15.40:0/2930953743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f300319d0 con 0x7f3f38070ae0 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.816+0000 7f3f3d028640 1 -- 172.21.15.40:0/2930953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38070ae0 msgr2=0x7f3f38070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.816+0000 7f3f3d028640 1 --2- 172.21.15.40:0/2930953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38070ae0 0x7f3f38070f50 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f3f380c64b0 tx=0x7f3f3002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.816+0000 7f3f3d028640 1 -- 172.21.15.40:0/2930953743 shutdown_connections 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.816+0000 7f3f3d028640 1 --2- 172.21.15.40:0/2930953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38070ae0 0x7f3f38070f50 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.816+0000 7f3f3d028640 1 --2- 172.21.15.40:0/2930953743 >> v2:172.21.15.184:3300/0 conn(0x7f3f380d1380 0x7f3f380d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.816+0000 7f3f3d028640 1 -- 172.21.15.40:0/2930953743 >> 172.21.15.40:0/2930953743 conn(0x7f3f380d2da0 msgr2=0x7f3f380d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.816+0000 7f3f3d028640 1 -- 172.21.15.40:0/2930953743 shutdown_connections 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.816+0000 7f3f3d028640 1 -- 172.21.15.40:0/2930953743 wait complete. 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.817+0000 7f3f3d028640 1 Processor -- start 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.817+0000 7f3f3d028640 1 -- start start 2024-04-26T21:47:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3d028640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070ae0 0x7f3f381500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3d028640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f380d1380 0x7f3f380caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3d028640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f3815d1e0 0x7f3f3815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3d028640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f380d10f0 con 0x7f3f3815d1e0 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3d028640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f380702e0 con 0x7f3f380d1380 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3d028640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f380ce4c0 con 0x7f3f38070ae0 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f3815d1e0 0x7f3f3815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f3815d1e0 0x7f3f3815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51100/0 (socket says 172.21.15.40:51100) 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 -- 172.21.15.40:0/1588663528 learned_addr learned my addr 172.21.15.40:0/1588663528 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f36d9d640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070ae0 0x7f3f381500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f2ffff640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f380d1380 0x7f3f380caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 -- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070ae0 msgr2=0x7f3f381500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070ae0 0x7f3f381500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 -- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f380d1380 msgr2=0x7f3f380caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f380d1380 0x7f3f380caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 -- 172.21.15.40:0/1588663528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f380cba60 con 0x7f3f3815d1e0 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f36d9d640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070ae0 0x7f3f381500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f2ffff640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f380d1380 0x7f3f380caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.818+0000 7f3f3759e640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f3815d1e0 0x7f3f3815ac30 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f3f30034040 tx=0x7f3f30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.819+0000 7f3f2f7fe640 1 -- 172.21.15.40:0/1588663528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30036050 con 0x7f3f3815d1e0 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.819+0000 7f3f2f7fe640 1 -- 172.21.15.40:0/1588663528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30042070 con 0x7f3f3815d1e0 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.819+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f380cbd40 con 0x7f3f3815d1e0 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.819+0000 7f3f2f7fe640 1 -- 172.21.15.40:0/1588663528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30009510 con 0x7f3f3815d1e0 2024-04-26T21:47:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.819+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f380cc300 con 0x7f3f3815d1e0 2024-04-26T21:47:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.820+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f380d17a0 con 0x7f3f3815d1e0 2024-04-26T21:47:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.820+0000 7f3f2f7fe640 1 -- 172.21.15.40:0/1588663528 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f30007bf0 con 0x7f3f3815d1e0 2024-04-26T21:47:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.821+0000 7f3f2f7fe640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f1806c8f0 0x7f3f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.821+0000 7f3f2f7fe640 1 -- 172.21.15.40:0/1588663528 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f3f300aca10 con 0x7f3f3815d1e0 2024-04-26T21:47:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.821+0000 7f3f36d9d640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f1806c8f0 0x7f3f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.822+0000 7f3f36d9d640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f1806c8f0 0x7f3f1806edb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3f28006fd0 tx=0x7f3f28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:54.825+0000 7f3f2f7fe640 1 -- 172.21.15.40:0/1588663528 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f380d17a0 con 0x7f3f3815d1e0 2024-04-26T21:47:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.011+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f3f380694f0 con 0x7f3f1806c8f0 2024-04-26T21:47:55.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.012+0000 7f3f2f7fe640 1 -- 172.21.15.40:0/1588663528 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3f380694f0 con 0x7f3f1806c8f0 2024-04-26T21:47:55.013 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T21:47:55.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f1806c8f0 msgr2=0x7f3f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f1806c8f0 0x7f3f1806edb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3f28006fd0 tx=0x7f3f28008040 comp rx=0 tx=0).stop 2024-04-26T21:47:55.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f3815d1e0 msgr2=0x7f3f3815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f3815d1e0 0x7f3f3815ac30 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f3f30034040 tx=0x7f3f30004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:55.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 shutdown_connections 2024-04-26T21:47:55.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f3f1806c8f0 0x7f3f1806edb0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f3815d1e0 0x7f3f3815ac30 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f380d1380 0x7f3f380caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 --2- 172.21.15.40:0/1588663528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070ae0 0x7f3f381500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 >> 172.21.15.40:0/1588663528 conn(0x7f3f380d2da0 msgr2=0x7f3f380b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 shutdown_connections 2024-04-26T21:47:55.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.014+0000 7f3f3d028640 1 -- 172.21.15.40:0/1588663528 wait complete. 2024-04-26T21:47:55.026 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T21:47:55.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:47:55.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.130+0000 7f697a738640 1 Processor -- start 2024-04-26T21:47:55.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.130+0000 7f697a738640 1 -- start start 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f697a738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697406eba0 0x7f6974160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f697a738640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69741404c0 0x7f6974140970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f697a738640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697413f100 con 0x7f697415cea0 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f697a738640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f697415d280 con 0x7f69741404c0 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f697a738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6974141840 con 0x7f697406eba0 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f6978cd5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69741404c0 0x7f6974140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f6973fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f697415cea0 0x7f697406e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54568/0 (socket says 172.21.15.40:54568) 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f6973fff640 1 -- 172.21.15.40:0/3572723229 learned_addr learned my addr 172.21.15.40:0/3572723229 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:55.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f69737fe640 1 --2- 172.21.15.40:0/3572723229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697406eba0 0x7f6974160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f6978cd5640 1 -- 172.21.15.40:0/3572723229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697406eba0 msgr2=0x7f6974160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f6978cd5640 1 --2- 172.21.15.40:0/3572723229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697406eba0 0x7f6974160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f6978cd5640 1 -- 172.21.15.40:0/3572723229 >> v1:172.21.15.40:6789/0 conn(0x7f697415cea0 legacy=0x7f697406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f6978cd5640 1 -- 172.21.15.40:0/3572723229 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69741423c0 con 0x7f69741404c0 2024-04-26T21:47:55.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.131+0000 7f6972ffd640 1 -- 172.21.15.40:0/3572723229 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3649834108 0 0) 0x7f697413f100 con 0x7f697415cea0 2024-04-26T21:47:55.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f6978cd5640 1 --2- 172.21.15.40:0/3572723229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69741404c0 0x7f6974140970 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f696c009d90 tx=0x7f696c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0777bab26d502a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f6972ffd640 1 -- 172.21.15.40:0/3572723229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c007740 con 0x7f69741404c0 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f6972ffd640 1 -- 172.21.15.40:0/3572723229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c0078e0 con 0x7f69741404c0 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f6972ffd640 1 -- 172.21.15.40:0/3572723229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c007cb0 con 0x7f69741404c0 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f697a738640 1 -- 172.21.15.40:0/3572723229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69741404c0 msgr2=0x7f6974140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f697a738640 1 --2- 172.21.15.40:0/3572723229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69741404c0 0x7f6974140970 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f696c009d90 tx=0x7f696c02a660 comp rx=0 tx=0).stop 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f697a738640 1 -- 172.21.15.40:0/3572723229 shutdown_connections 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f697a738640 1 --2- 172.21.15.40:0/3572723229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69741404c0 0x7f6974140970 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f697a738640 1 --2- 172.21.15.40:0/3572723229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f697406eba0 0x7f6974160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.132+0000 7f697a738640 1 -- 172.21.15.40:0/3572723229 >> 172.21.15.40:0/3572723229 conn(0x7f697406f310 msgr2=0x7f697406f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.133+0000 7f697a738640 1 -- 172.21.15.40:0/3572723229 shutdown_connections 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.133+0000 7f697a738640 1 -- 172.21.15.40:0/3572723229 wait complete. 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.134+0000 7f697a738640 1 Processor -- start 2024-04-26T21:47:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.134+0000 7f697a738640 1 -- start start 2024-04-26T21:47:55.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.134+0000 7f697a738640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697406eba0 0x7f6974137360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.134+0000 7f697a738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69741404c0 0x7f6974149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f697a738640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697415cea0 0x7f697414ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f697a738640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f69741420e0 con 0x7f697415cea0 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f697a738640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6974141e00 con 0x7f697406eba0 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f697a738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f697415c610 con 0x7f69741404c0 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697415cea0 0x7f697414ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697415cea0 0x7f697414ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51122/0 (socket says 172.21.15.40:51122) 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 -- 172.21.15.40:0/1159218534 learned_addr learned my addr 172.21.15.40:0/1159218534 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f69737fe640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69741404c0 0x7f6974149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6973fff640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697406eba0 0x7f6974137360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 -- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69741404c0 msgr2=0x7f6974149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69741404c0 0x7f6974149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 -- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697406eba0 msgr2=0x7f6974137360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697406eba0 0x7f6974137360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 -- 172.21.15.40:0/1159218534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6974150090 con 0x7f697415cea0 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6973fff640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f697406eba0 0x7f6974137360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6978cd5640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697415cea0 0x7f697414ac60 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f696c009eb0 tx=0x7f696c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c040070 con 0x7f697415cea0 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c031bc0 con 0x7f697415cea0 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c045460 con 0x7f697415cea0 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6974150370 con 0x7f697415cea0 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.135+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6974150930 con 0x7f697415cea0 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.136+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6974069600 con 0x7f697415cea0 2024-04-26T21:47:55.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.137+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f696c039040 con 0x7f697415cea0 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.137+0000 7f6970ff9640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f696006c8f0 0x7f696006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.137+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f696c0ad230 con 0x7f697415cea0 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.138+0000 7f6970ff9640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7f6960072e30 0x7f6960075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.138+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6960075940 con 0x7f6960072e30 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.138+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f696c0b4050 con 0x7f697415cea0 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.138+0000 7f6973fff640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f696006c8f0 0x7f696006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.138+0000 7f69737fe640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7f6960072e30 0x7f6960075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.138+0000 7f6973fff640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f696006c8f0 0x7f696006edb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6964000f60 tx=0x7f6964007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.138+0000 7f69737fe640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7f6960072e30 0x7f6960075270 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.139+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 <== osd.5 v2:172.21.15.184:6808/304953510 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6960075940 con 0x7f6960072e30 2024-04-26T21:47:55.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.152+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 --> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f69741378a0 con 0x7f6960072e30 2024-04-26T21:47:55.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f6970ff9640 1 -- 172.21.15.40:0/1159218534 <== osd.5 v2:172.21.15.184:6808/304953510 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f69741378a0 con 0x7f6960072e30 2024-04-26T21:47:55.153 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:55.154 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:47:55.154 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7f6960072e30 msgr2=0x7f6960075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6808/304953510,v1:172.21.15.184:6810/304953510] conn(0x7f6960072e30 0x7f6960075270 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f696006c8f0 msgr2=0x7f696006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f696006c8f0 0x7f696006edb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6964000f60 tx=0x7f6964007ca0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697415cea0 msgr2=0x7f697414ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 --2- 172.21.15.40:0/1159218534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f697415cea0 0x7f697414ac60 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f696c009eb0 tx=0x7f696c004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f6978cd5640 1 -- 172.21.15.40:0/1159218534 reap_dead start 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 shutdown_connections 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 >> 172.21.15.40:0/1159218534 conn(0x7f697406f310 msgr2=0x7f6974151640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 shutdown_connections 2024-04-26T21:47:55.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.153+0000 7f697a738640 1 -- 172.21.15.40:0/1159218534 wait complete. 2024-04-26T21:47:55.164 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T21:47:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.266+0000 7fee86b86640 1 Processor -- start 2024-04-26T21:47:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.266+0000 7fee86b86640 1 -- start start 2024-04-26T21:47:55.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee86b86640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fee80162810 0x7fee80070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee86b86640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee800d10d0 con 0x7fee800d1b30 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee86b86640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee800d1370 con 0x7fee800b2340 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee86b86640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fee800d0750 con 0x7fee80162810 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee85123640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fee800d1b30 0x7fee80070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54576/0 (socket says 172.21.15.40:54576) 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee85123640 1 -- 172.21.15.40:0/3408923473 learned_addr learned my addr 172.21.15.40:0/3408923473 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4233352956 0 0) 0x7fee800d10d0 con 0x7fee800d1b30 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee68003520 con 0x7fee800d1b30 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee84922640 1 --2- 172.21.15.40:0/3408923473 >> v2:172.21.15.184:3300/0 conn(0x7fee80162810 0x7fee80070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1690089227 0 0) 0x7fee68003520 con 0x7fee800d1b30 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 >> v2:172.21.15.184:3300/0 conn(0x7fee80162810 msgr2=0x7fee80070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee7f7fe640 1 --2- 172.21.15.40:0/3408923473 >> v2:172.21.15.184:3300/0 conn(0x7fee80162810 0x7fee80070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.267+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 >> v1:172.21.15.184:6789/0 conn(0x7fee800b2340 legacy=0x7fee800cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee800d01b0 con 0x7fee800d1b30 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fee780019f0 con 0x7fee800d1b30 2024-04-26T21:47:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 >> v1:172.21.15.40:6789/0 conn(0x7fee800d1b30 legacy=0x7fee80070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee680039b0 0x7fee68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee68004800 0x7fee68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee680051c0 0x7fee6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee800d0750 con 0x7fee680051c0 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee6800c060 con 0x7fee68004800 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee6800c260 con 0x7fee680039b0 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee85123640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee680051c0 0x7fee6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee84922640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee680039b0 0x7fee68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee7ffff640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee68004800 0x7fee68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee85123640 1 -- 172.21.15.40:0/3408923473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee680039b0 msgr2=0x7fee68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee85123640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee680039b0 0x7fee68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee85123640 1 -- 172.21.15.40:0/3408923473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee68004800 msgr2=0x7fee68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee85123640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee68004800 0x7fee68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.268+0000 7fee85123640 1 -- 172.21.15.40:0/3408923473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee68003520 con 0x7fee680051c0 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee85123640 1 -- 172.21.15.40:0/3408923473 reap_dead start 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee85123640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee680051c0 0x7fee6800b9c0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fee7800a830 tx=0x7fee68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560422d6fb0775fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7800a480 con 0x7fee680051c0 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7802f070 con 0x7fee680051c0 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee7f7fe640 1 -- 172.21.15.40:0/3408923473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7802bcb0 con 0x7fee680051c0 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee86b86640 1 -- 172.21.15.40:0/3408923473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee680051c0 msgr2=0x7fee6800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee86b86640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee680051c0 0x7fee6800b9c0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fee7800a830 tx=0x7fee68005830 comp rx=0 tx=0).stop 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee86b86640 1 -- 172.21.15.40:0/3408923473 shutdown_connections 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee86b86640 1 --2- 172.21.15.40:0/3408923473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee680051c0 0x7fee6800b9c0 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee86b86640 1 -- 172.21.15.40:0/3408923473 >> 172.21.15.40:0/3408923473 conn(0x7fee800d2da0 msgr2=0x7fee800d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee86b86640 1 -- 172.21.15.40:0/3408923473 shutdown_connections 2024-04-26T21:47:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.269+0000 7fee86b86640 1 -- 172.21.15.40:0/3408923473 wait complete. 2024-04-26T21:47:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.271+0000 7fee86b86640 1 Processor -- start 2024-04-26T21:47:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.271+0000 7fee86b86640 1 -- start start 2024-04-26T21:47:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.271+0000 7fee86b86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee680051c0 0x7fee80154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.271+0000 7fee86b86640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee800ca880 0x7fee800c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee86b86640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee800cad00 0x7fee800c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee86b86640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee800d1fc0 con 0x7fee800ca880 2024-04-26T21:47:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee86b86640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee800d16a0 con 0x7fee680051c0 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee86b86640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee800d2140 con 0x7fee800cad00 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee800ca880 0x7fee800c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee800ca880 0x7fee800c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51148/0 (socket says 172.21.15.40:51148) 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 -- 172.21.15.40:0/652104225 learned_addr learned my addr 172.21.15.40:0/652104225 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee85123640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee800cad00 0x7fee800c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee84922640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee680051c0 0x7fee80154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 -- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee800cad00 msgr2=0x7fee800c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee800cad00 0x7fee800c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 -- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee680051c0 msgr2=0x7fee80154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee680051c0 0x7fee80154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 -- 172.21.15.40:0/652104225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee800c6a00 con 0x7fee800ca880 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee85123640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee800cad00 0x7fee800c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7ffff640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee800ca880 0x7fee800c7aa0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7fee7400f650 tx=0x7fee7400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7d7fa640 1 -- 172.21.15.40:0/652104225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee740050d0 con 0x7fee800ca880 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7d7fa640 1 -- 172.21.15.40:0/652104225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee74018070 con 0x7fee800ca880 2024-04-26T21:47:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee800c6ce0 con 0x7fee800ca880 2024-04-26T21:47:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee7d7fa640 1 -- 172.21.15.40:0/652104225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74005490 con 0x7fee800ca880 2024-04-26T21:47:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.272+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee800c72a0 con 0x7fee800ca880 2024-04-26T21:47:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.273+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee800694f0 con 0x7fee800ca880 2024-04-26T21:47:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.274+0000 7fee7d7fa640 1 -- 172.21.15.40:0/652104225 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee7400d730 con 0x7fee800ca880 2024-04-26T21:47:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.274+0000 7fee7d7fa640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee6c06c8f0 0x7fee6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.274+0000 7fee7d7fa640 1 -- 172.21.15.40:0/652104225 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fee7408b760 con 0x7fee800ca880 2024-04-26T21:47:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.274+0000 7fee84922640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee6c06c8f0 0x7fee6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.275+0000 7fee84922640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee6c06c8f0 0x7fee6c06edb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fee70009980 tx=0x7fee70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.279+0000 7fee7d7fa640 1 -- 172.21.15.40:0/652104225 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee74059d40 con 0x7fee800ca880 2024-04-26T21:47:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.464+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fee8006d740 con 0x7fee6c06c8f0 2024-04-26T21:47:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.465+0000 7fee7d7fa640 1 -- 172.21.15.40:0/652104225 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fee8006d740 con 0x7fee6c06c8f0 2024-04-26T21:47:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.466+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee6c06c8f0 msgr2=0x7fee6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee6c06c8f0 0x7fee6c06edb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fee70009980 tx=0x7fee70008040 comp rx=0 tx=0).stop 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee800ca880 msgr2=0x7fee800c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee800ca880 0x7fee800c7aa0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7fee7400f650 tx=0x7fee7400fb10 comp rx=0 tx=0).stop 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 shutdown_connections 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fee6c06c8f0 0x7fee6c06edb0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee800cad00 0x7fee800c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee800ca880 0x7fee800c7aa0 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 --2- 172.21.15.40:0/652104225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee680051c0 0x7fee80154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 >> 172.21.15.40:0/652104225 conn(0x7fee800d2da0 msgr2=0x7fee800ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 shutdown_connections 2024-04-26T21:47:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.467+0000 7fee86b86640 1 -- 172.21.15.40:0/652104225 wait complete. 2024-04-26T21:47:55.478 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T21:47:55.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:47:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.582+0000 7fb0e4dd4640 1 Processor -- start 2024-04-26T21:47:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.582+0000 7fb0e4dd4640 1 -- start start 2024-04-26T21:47:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0e4dd4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0e015f3e0 0x7fb0e00d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0e4dd4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 0x7fb0e00d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0e4dd4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0e0160930 con 0x7fb0e015f7c0 2024-04-26T21:47:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0e4dd4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb0e015fd20 con 0x7fb0e015f3e0 2024-04-26T21:47:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0e4dd4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e0160000 con 0x7fb0e00d31e0 2024-04-26T21:47:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0de59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0e015f3e0 0x7fb0e00d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0de59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0e015f3e0 0x7fb0e00d2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51154/0 (socket says 172.21.15.40:51154) 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0de59c640 1 -- 172.21.15.40:0/2567928596 learned_addr learned my addr 172.21.15.40:0/2567928596 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0ddd9b640 1 --2- 172.21.15.40:0/2567928596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 0x7fb0e00d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0de59c640 1 -- 172.21.15.40:0/2567928596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 msgr2=0x7fb0e00d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0de59c640 1 --2- 172.21.15.40:0/2567928596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 0x7fb0e00d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0de59c640 1 -- 172.21.15.40:0/2567928596 >> v1:172.21.15.184:6789/0 conn(0x7fb0e015f7c0 legacy=0x7fb0e01620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0de59c640 1 -- 172.21.15.40:0/2567928596 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e01602e0 con 0x7fb0e015f3e0 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.583+0000 7fb0ddd9b640 1 --2- 172.21.15.40:0/2567928596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 0x7fb0e00d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.584+0000 7fb0de59c640 1 --2- 172.21.15.40:0/2567928596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f3e0 0x7fb0e00d2ca0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009870 tx=0x7fb0d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac216b90dbca02b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.584+0000 7fb0dd59a640 1 -- 172.21.15.40:0/2567928596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8037070 con 0x7fb0e015f3e0 2024-04-26T21:47:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.584+0000 7fb0dd59a640 1 -- 172.21.15.40:0/2567928596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d8032440 con 0x7fb0e015f3e0 2024-04-26T21:47:55.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.584+0000 7fb0dd59a640 1 -- 172.21.15.40:0/2567928596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8032810 con 0x7fb0e015f3e0 2024-04-26T21:47:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.585+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/2567928596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f3e0 msgr2=0x7fb0e00d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.585+0000 7fb0e4dd4640 1 --2- 172.21.15.40:0/2567928596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f3e0 0x7fb0e00d2ca0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009870 tx=0x7fb0d802a180 comp rx=0 tx=0).stop 2024-04-26T21:47:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.585+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/2567928596 shutdown_connections 2024-04-26T21:47:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.585+0000 7fb0e4dd4640 1 --2- 172.21.15.40:0/2567928596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 0x7fb0e00d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.585+0000 7fb0e4dd4640 1 --2- 172.21.15.40:0/2567928596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f3e0 0x7fb0e00d2ca0 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.585+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/2567928596 >> 172.21.15.40:0/2567928596 conn(0x7fb0e009e3c0 msgr2=0x7fb0e00d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.585+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/2567928596 shutdown_connections 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.585+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/2567928596 wait complete. 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.586+0000 7fb0e4dd4640 1 Processor -- start 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.586+0000 7fb0e4dd4640 1 -- start start 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0e4dd4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 0x7fb0e00a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0e4dd4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0e015f3e0 0x7fb0e00cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0e4dd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f7c0 0x7fb0e00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0e4dd4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e00d3bd0 con 0x7fb0e015f7c0 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0e4dd4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0160d40 con 0x7fb0e015f3e0 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0e4dd4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e00d3d50 con 0x7fb0e00d31e0 2024-04-26T21:47:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ded9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f7c0 0x7fb0e00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ded9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f7c0 0x7fb0e00cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51164/0 (socket says 172.21.15.40:51164) 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ded9d640 1 -- 172.21.15.40:0/1236832048 learned_addr learned my addr 172.21.15.40:0/1236832048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0de59c640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 0x7fb0e00a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ddd9b640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0e015f3e0 0x7fb0e00cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ded9d640 1 -- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 msgr2=0x7fb0e00a2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ded9d640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0e00d31e0 0x7fb0e00a2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ded9d640 1 -- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0e015f3e0 msgr2=0x7fb0e00cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ded9d640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0e015f3e0 0x7fb0e00cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.587+0000 7fb0ded9d640 1 -- 172.21.15.40:0/1236832048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e00ccaf0 con 0x7fb0e015f7c0 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.588+0000 7fb0ded9d640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f7c0 0x7fb0e00cadb0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000b9e0 tx=0x7fb0d000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.588+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0019070 con 0x7fb0e015f7c0 2024-04-26T21:47:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.588+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d000caa0 con 0x7fb0e015f7c0 2024-04-26T21:47:55.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.588+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e00ccdd0 con 0x7fb0e015f7c0 2024-04-26T21:47:55.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.588+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0014440 con 0x7fb0e015f7c0 2024-04-26T21:47:55.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.588+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e00cd390 con 0x7fb0e015f7c0 2024-04-26T21:47:55.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.589+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0e0069600 con 0x7fb0e015f7c0 2024-04-26T21:47:55.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.589+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0d001c070 con 0x7fb0e015f7c0 2024-04-26T21:47:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.590+0000 7fb0d6ffd640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0c006c8f0 0x7fb0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.590+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fb0d008b600 con 0x7fb0e015f7c0 2024-04-26T21:47:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.590+0000 7fb0d6ffd640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb0c0072e30 0x7fb0c0075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.590+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0c0075940 con 0x7fb0c0072e30 2024-04-26T21:47:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.590+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fb0d0092050 con 0x7fb0e015f7c0 2024-04-26T21:47:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.590+0000 7fb0de59c640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0c006c8f0 0x7fb0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.590+0000 7fb0ddd9b640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb0c0072e30 0x7fb0c0075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.590+0000 7fb0de59c640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0c006c8f0 0x7fb0c006edb0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8034000 tx=0x7fb0d802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.591+0000 7fb0ddd9b640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb0c0072e30 0x7fb0c0075270 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.591+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 <== osd.6 v2:172.21.15.184:6824/533952038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0c0075940 con 0x7fb0c0072e30 2024-04-26T21:47:55.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.605+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 --> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb0e00d57e0 con 0x7fb0c0072e30 2024-04-26T21:47:55.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.605+0000 7fb0d6ffd640 1 -- 172.21.15.40:0/1236832048 <== osd.6 v2:172.21.15.184:6824/533952038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb0e00d57e0 con 0x7fb0c0072e30 2024-04-26T21:47:55.606 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:55.606 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:47:55.606 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:55.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.605+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb0c0072e30 msgr2=0x7fb0c0075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.605+0000 7fb0e4dd4640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6824/533952038,v1:172.21.15.184:6825/533952038] conn(0x7fb0c0072e30 0x7fb0c0075270 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0c006c8f0 msgr2=0x7fb0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0e4dd4640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb0c006c8f0 0x7fb0c006edb0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8034000 tx=0x7fb0d802e000 comp rx=0 tx=0).stop 2024-04-26T21:47:55.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f7c0 msgr2=0x7fb0e00cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0e4dd4640 1 --2- 172.21.15.40:0/1236832048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e015f7c0 0x7fb0e00cadb0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000b9e0 tx=0x7fb0d000bea0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0ded9d640 1 -- 172.21.15.40:0/1236832048 reap_dead start 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 shutdown_connections 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 >> 172.21.15.40:0/1236832048 conn(0x7fb0e009e3c0 msgr2=0x7fb0e006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 shutdown_connections 2024-04-26T21:47:55.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.606+0000 7fb0e4dd4640 1 -- 172.21.15.40:0/1236832048 wait complete. 2024-04-26T21:47:55.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T21:47:55.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.718+0000 7f2091f19640 1 Processor -- start 2024-04-26T21:47:55.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.718+0000 7f2091f19640 1 -- start start 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f2091f19640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f208c162810 0x7f208c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f2091f19640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f208c0d1b30 0x7f208c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f2091f19640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f208c0d10d0 con 0x7f208c0b2340 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f2091f19640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f208c0d0750 con 0x7f208c162810 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f2091f19640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f208c0d16a0 con 0x7f208c0d1b30 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f208b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f208c162810 0x7f208c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f208b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f208c162810 0x7f208c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51180/0 (socket says 172.21.15.40:51180) 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f208b7fe640 1 -- 172.21.15.40:0/1637026804 learned_addr learned my addr 172.21.15.40:0/1637026804 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f208bfff640 1 --2- 172.21.15.40:0/1637026804 >> v2:172.21.15.184:3300/0 conn(0x7f208c0d1b30 0x7f208c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f208b7fe640 1 -- 172.21.15.40:0/1637026804 >> v2:172.21.15.184:3300/0 conn(0x7f208c0d1b30 msgr2=0x7f208c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f208b7fe640 1 --2- 172.21.15.40:0/1637026804 >> v2:172.21.15.184:3300/0 conn(0x7f208c0d1b30 0x7f208c0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f208b7fe640 1 -- 172.21.15.40:0/1637026804 >> v1:172.21.15.184:6789/0 conn(0x7f208c0b2340 legacy=0x7f208c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.719+0000 7f208b7fe640 1 -- 172.21.15.40:0/1637026804 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c0d1370 con 0x7f208c162810 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.720+0000 7f208b7fe640 1 --2- 172.21.15.40:0/1637026804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c070350 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f208c073330 tx=0x7f207c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522831b5510f99d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.720+0000 7f208a7fc640 1 -- 172.21.15.40:0/1637026804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c036070 con 0x7f208c162810 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.720+0000 7f208a7fc640 1 -- 172.21.15.40:0/1637026804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c031600 con 0x7f208c162810 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.720+0000 7f208a7fc640 1 -- 172.21.15.40:0/1637026804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c0319d0 con 0x7f208c162810 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.721+0000 7f2091f19640 1 -- 172.21.15.40:0/1637026804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 msgr2=0x7f208c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.721+0000 7f2091f19640 1 --2- 172.21.15.40:0/1637026804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c070350 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f208c073330 tx=0x7f207c02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.721+0000 7f2091f19640 1 -- 172.21.15.40:0/1637026804 shutdown_connections 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.721+0000 7f2091f19640 1 --2- 172.21.15.40:0/1637026804 >> v2:172.21.15.184:3300/0 conn(0x7f208c0d1b30 0x7f208c0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.721+0000 7f2091f19640 1 --2- 172.21.15.40:0/1637026804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c070350 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.721+0000 7f2091f19640 1 -- 172.21.15.40:0/1637026804 >> 172.21.15.40:0/1637026804 conn(0x7f208c0d2da0 msgr2=0x7f208c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.721+0000 7f2091f19640 1 -- 172.21.15.40:0/1637026804 shutdown_connections 2024-04-26T21:47:55.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.721+0000 7f2091f19640 1 -- 172.21.15.40:0/1637026804 wait complete. 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.722+0000 7f2091f19640 1 Processor -- start 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.722+0000 7f2091f19640 1 -- start start 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f2091f19640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f208c0b2340 0x7f208c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f2091f19640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f208c0d1b30 0x7f208c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f2091f19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c1f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f2091f19640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f208c070890 con 0x7f208c162810 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f2091f19640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f208c0d0470 con 0x7f208c0b2340 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f2091f19640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f208c0cdb00 con 0x7f208c0d1b30 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c1f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c1f8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51188/0 (socket says 172.21.15.40:51188) 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 -- 172.21.15.40:0/787411315 learned_addr learned my addr 172.21.15.40:0/787411315 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208b7fe640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f208c0b2340 0x7f208c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208affd640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f208c0d1b30 0x7f208c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 -- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f208c0d1b30 msgr2=0x7f208c0c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f208c0d1b30 0x7f208c0c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 -- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f208c0b2340 msgr2=0x7f208c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f208c0b2340 0x7f208c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 -- 172.21.15.40:0/787411315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c0c8a50 con 0x7f208c162810 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208b7fe640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f208c0b2340 0x7f208c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.723+0000 7f208bfff640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c1f8b70 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f208400c7f0 tx=0x7f208400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.724+0000 7f2073fff640 1 -- 172.21.15.40:0/787411315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084015070 con 0x7f208c162810 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.724+0000 7f2073fff640 1 -- 172.21.15.40:0/787411315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2084004220 con 0x7f208c162810 2024-04-26T21:47:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.724+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208c0c8d30 con 0x7f208c162810 2024-04-26T21:47:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.724+0000 7f2073fff640 1 -- 172.21.15.40:0/787411315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084007620 con 0x7f208c162810 2024-04-26T21:47:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.724+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208c0c95d0 con 0x7f208c162810 2024-04-26T21:47:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.725+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f208c0c8f00 con 0x7f208c162810 2024-04-26T21:47:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.725+0000 7f2073fff640 1 -- 172.21.15.40:0/787411315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20840079d0 con 0x7f208c162810 2024-04-26T21:47:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.726+0000 7f2073fff640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f207806c8f0 0x7f207806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.726+0000 7f2073fff640 1 -- 172.21.15.40:0/787411315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f208408bb80 con 0x7f208c162810 2024-04-26T21:47:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.726+0000 7f208b7fe640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f207806c8f0 0x7f207806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.727+0000 7f208b7fe640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f207806c8f0 0x7f207806edb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f207c033000 tx=0x7f207c009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.730+0000 7f2073fff640 1 -- 172.21.15.40:0/787411315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f208405a040 con 0x7f208c162810 2024-04-26T21:47:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.916+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f208c162350 con 0x7f207806c8f0 2024-04-26T21:47:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.917+0000 7f2073fff640 1 -- 172.21.15.40:0/787411315 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f208c162350 con 0x7f207806c8f0 2024-04-26T21:47:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f207806c8f0 msgr2=0x7f207806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f207806c8f0 0x7f207806edb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f207c033000 tx=0x7f207c009000 comp rx=0 tx=0).stop 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 msgr2=0x7f208c1f8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c1f8b70 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f208400c7f0 tx=0x7f208400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 shutdown_connections 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f207806c8f0 0x7f207806edb0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f208c162810 0x7f208c1f8b70 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f208c0d1b30 0x7f208c0c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 --2- 172.21.15.40:0/787411315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f208c0b2340 0x7f208c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:55.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 >> 172.21.15.40:0/787411315 conn(0x7f208c0d2da0 msgr2=0x7f208c070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:55.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 shutdown_connections 2024-04-26T21:47:55.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:55.919+0000 7f2091f19640 1 -- 172.21.15.40:0/787411315 wait complete. 2024-04-26T21:47:55.931 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T21:47:55.931 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:47:56.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.035+0000 7f5525904640 1 Processor -- start 2024-04-26T21:47:56.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.035+0000 7f5525904640 1 -- start start 2024-04-26T21:47:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.035+0000 7f5525904640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f552015f300 0x7f55200b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.035+0000 7f5525904640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.035+0000 7f5525904640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f552006e120 con 0x7f5520162eb0 2024-04-26T21:47:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.035+0000 7f5525904640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5520163830 con 0x7f552015f300 2024-04-26T21:47:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.035+0000 7f5525904640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f552006e400 con 0x7f55200b3d20 2024-04-26T21:47:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551effd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f552015f300 0x7f55200b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:51916/0 (socket says 172.21.15.40:51916) 2024-04-26T21:47:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551f7fe640 1 -- 172.21.15.40:0/1439465591 learned_addr learned my addr 172.21.15.40:0/1439465591 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551dffb640 1 -- 172.21.15.40:0/1439465591 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312211892 0 0) 0x7f552006e120 con 0x7f5520162eb0 2024-04-26T21:47:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551dffb640 1 -- 172.21.15.40:0/1439465591 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5508003520 con 0x7f5520162eb0 2024-04-26T21:47:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551f7fe640 1 -- 172.21.15.40:0/1439465591 >> v2:172.21.15.184:3300/0 conn(0x7f552015f300 msgr2=0x7f55200b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551f7fe640 1 --2- 172.21.15.40:0/1439465591 >> v2:172.21.15.184:3300/0 conn(0x7f552015f300 0x7f55200b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551f7fe640 1 -- 172.21.15.40:0/1439465591 >> v1:172.21.15.184:6789/0 conn(0x7f5520162eb0 legacy=0x7f55200d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.036+0000 7f551f7fe640 1 -- 172.21.15.40:0/1439465591 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200b2510 con 0x7f55200b3d20 2024-04-26T21:47:56.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.037+0000 7f551f7fe640 1 --2- 172.21.15.40:0/1439465591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201632f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5518009e50 tx=0x7f551802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ae25f36fc2b04e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.037+0000 7f551dffb640 1 -- 172.21.15.40:0/1439465591 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518036070 con 0x7f55200b3d20 2024-04-26T21:47:56.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.037+0000 7f551dffb640 1 -- 172.21.15.40:0/1439465591 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5518031600 con 0x7f55200b3d20 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.037+0000 7f551dffb640 1 -- 172.21.15.40:0/1439465591 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180319b0 con 0x7f55200b3d20 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.038+0000 7f5525904640 1 -- 172.21.15.40:0/1439465591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 msgr2=0x7f55201632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.038+0000 7f5525904640 1 --2- 172.21.15.40:0/1439465591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201632f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5518009e50 tx=0x7f551802a710 comp rx=0 tx=0).stop 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.038+0000 7f5525904640 1 -- 172.21.15.40:0/1439465591 shutdown_connections 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.038+0000 7f5525904640 1 --2- 172.21.15.40:0/1439465591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201632f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.038+0000 7f5525904640 1 --2- 172.21.15.40:0/1439465591 >> v2:172.21.15.184:3300/0 conn(0x7f552015f300 0x7f55200b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.038+0000 7f5525904640 1 -- 172.21.15.40:0/1439465591 >> 172.21.15.40:0/1439465591 conn(0x7f55200b49e0 msgr2=0x7f55200b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.038+0000 7f5525904640 1 -- 172.21.15.40:0/1439465591 shutdown_connections 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.038+0000 7f5525904640 1 -- 172.21.15.40:0/1439465591 wait complete. 2024-04-26T21:47:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.039+0000 7f5525904640 1 Processor -- start 2024-04-26T21:47:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.039+0000 7f5525904640 1 -- start start 2024-04-26T21:47:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f5525904640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f5525904640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f552015f300 0x7f55200d2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f5525904640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520162eb0 0x7f55201f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f5525904640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f552015f6e0 con 0x7f5520162eb0 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f5525904640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f552006e6e0 con 0x7f552015f300 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f5525904640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f552006dd60 con 0x7f55200b3d20 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520162eb0 0x7f55201f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520162eb0 0x7f55201f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51198/0 (socket says 172.21.15.40:51198) 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 -- 172.21.15.40:0/4164557284 learned_addr learned my addr 172.21.15.40:0/4164557284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551e7fc640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f552015f300 0x7f55200d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551effd640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 -- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 msgr2=0x7f55201f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55200b3d20 0x7f55201f8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 -- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f552015f300 msgr2=0x7f55200d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f552015f300 0x7f55200d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 -- 172.21.15.40:0/4164557284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55201f5980 con 0x7f5520162eb0 2024-04-26T21:47:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551e7fc640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f552015f300 0x7f55200d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.040+0000 7f551f7fe640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520162eb0 0x7f55201f4e50 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f5518034040 tx=0x7f5518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.041+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518036050 con 0x7f5520162eb0 2024-04-26T21:47:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.041+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5518042070 con 0x7f5520162eb0 2024-04-26T21:47:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.041+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55201f5c60 con 0x7f5520162eb0 2024-04-26T21:47:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.041+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518009510 con 0x7f5520162eb0 2024-04-26T21:47:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.041+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55201f8950 con 0x7f5520162eb0 2024-04-26T21:47:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.042+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5520069600 con 0x7f5520162eb0 2024-04-26T21:47:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.042+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5518007c70 con 0x7f5520162eb0 2024-04-26T21:47:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.043+0000 7f5507fff640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f550c06c8f0 0x7f550c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.043+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f55180ac7e0 con 0x7f5520162eb0 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.043+0000 7f5507fff640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7f550c072e30 0x7f550c075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.043+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55180020b0 con 0x7f550c072e30 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.043+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f55180b12c0 con 0x7f5520162eb0 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.043+0000 7f551effd640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f550c06c8f0 0x7f550c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.043+0000 7f551e7fc640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7f550c072e30 0x7f550c075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.044+0000 7f551effd640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f550c06c8f0 0x7f550c06edb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5510009a10 tx=0x7f5510008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.044+0000 7f551e7fc640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7f550c072e30 0x7f550c075270 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.044+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 <== osd.7 v2:172.21.15.184:6809/2389595522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55180020b0 con 0x7f550c072e30 2024-04-26T21:47:56.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.058+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 --> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f55200b8e50 con 0x7f550c072e30 2024-04-26T21:47:56.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5507fff640 1 -- 172.21.15.40:0/4164557284 <== osd.7 v2:172.21.15.184:6809/2389595522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f55200b8e50 con 0x7f550c072e30 2024-04-26T21:47:56.059 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:47:56.059 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:47:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:47:56.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7f550c072e30 msgr2=0x7f550c075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6809/2389595522,v1:172.21.15.184:6811/2389595522] conn(0x7f550c072e30 0x7f550c075270 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f550c06c8f0 msgr2=0x7f550c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f550c06c8f0 0x7f550c06edb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5510009a10 tx=0x7f5510008040 comp rx=0 tx=0).stop 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520162eb0 msgr2=0x7f55201f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 --2- 172.21.15.40:0/4164557284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520162eb0 0x7f55201f4e50 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f5518034040 tx=0x7f5518004a40 comp rx=0 tx=0).stop 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f551f7fe640 1 -- 172.21.15.40:0/4164557284 reap_dead start 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 shutdown_connections 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 >> 172.21.15.40:0/4164557284 conn(0x7f55200b49e0 msgr2=0x7f552015d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 shutdown_connections 2024-04-26T21:47:56.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.059+0000 7f5525904640 1 -- 172.21.15.40:0/4164557284 wait complete. 2024-04-26T21:47:56.070 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T21:47:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.171+0000 7f03a1859640 1 Processor -- start 2024-04-26T21:47:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.171+0000 7f03a1859640 1 -- start start 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.172+0000 7f03a1859640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c15f5a0 0x7f039c0d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.172+0000 7f03a1859640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c0d1a70 con 0x7f039c163610 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.172+0000 7f03a1859640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c0d1d50 con 0x7f039c0d2830 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.172+0000 7f03a1859640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f039c163d30 con 0x7f039c15f5a0 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.172+0000 7f039b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f039c163610 0x7f039c0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54584/0 (socket says 172.21.15.40:54584) 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.172+0000 7f039b7fe640 1 -- 172.21.15.40:0/700392251 learned_addr learned my addr 172.21.15.40:0/700392251 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039affd640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c15f5a0 0x7f039c0d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683502381 0 0) 0x7f039c0d1a70 con 0x7f039c163610 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0384003520 con 0x7f039c163610 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980881245 0 0) 0x7f039c0d1d50 con 0x7f039c0d2830 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f039c0d1a70 con 0x7f039c0d2830 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1513760873 0 0) 0x7f0384003520 con 0x7f039c163610 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c15f5a0 msgr2=0x7f039c0d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c15f5a0 0x7f039c0d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 >> v1:172.21.15.184:6789/0 conn(0x7f039c0d2830 legacy=0x7f039c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0d2030 con 0x7f039c163610 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7f03940019f0 con 0x7f039c163610 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039affd640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c15f5a0 0x7f039c0d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 >> v1:172.21.15.40:6789/0 conn(0x7f039c163610 legacy=0x7f039c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0384003b70 0x7f0384003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0384004800 0x7f0384004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f038400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f038400bf00 con 0x7f03840051c0 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f038400c080 con 0x7f0384004800 2024-04-26T21:47:56.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f038400c280 con 0x7f0384003b70 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039b7fe640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f038400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039affd640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0384003b70 0x7f0384003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039a7fc640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0384004800 0x7f0384004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039b7fe640 1 -- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0384003b70 msgr2=0x7f0384003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039b7fe640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0384003b70 0x7f0384003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039b7fe640 1 -- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0384004800 msgr2=0x7f0384004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039b7fe640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0384004800 0x7f0384004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.173+0000 7f039b7fe640 1 -- 172.21.15.40:0/700392251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0384003520 con 0x7f03840051c0 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.174+0000 7f039b7fe640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f038400b9c0 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f039400a7c0 tx=0x7f0384005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd26301737a9c319 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.174+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039400a410 con 0x7f03840051c0 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.174+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039402f070 con 0x7f03840051c0 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.174+0000 7f0399ffb640 1 -- 172.21.15.40:0/700392251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039402bc20 con 0x7f03840051c0 2024-04-26T21:47:56.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.175+0000 7f03a1859640 1 -- 172.21.15.40:0/700392251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 msgr2=0x7f038400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.175+0000 7f03a1859640 1 --2- 172.21.15.40:0/700392251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f038400b9c0 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f039400a7c0 tx=0x7f0384005830 comp rx=0 tx=0).stop 2024-04-26T21:47:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.175+0000 7f039b7fe640 1 -- 172.21.15.40:0/700392251 reap_dead start 2024-04-26T21:47:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.175+0000 7f03a1859640 1 -- 172.21.15.40:0/700392251 shutdown_connections 2024-04-26T21:47:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.175+0000 7f03a1859640 1 -- 172.21.15.40:0/700392251 >> 172.21.15.40:0/700392251 conn(0x7f039c0d5190 msgr2=0x7f039c0d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.175+0000 7f03a1859640 1 -- 172.21.15.40:0/700392251 shutdown_connections 2024-04-26T21:47:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.175+0000 7f03a1859640 1 -- 172.21.15.40:0/700392251 wait complete. 2024-04-26T21:47:56.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.176+0000 7f03a1859640 1 Processor -- start 2024-04-26T21:47:56.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.176+0000 7f03a1859640 1 -- start start 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f03a1859640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f039c158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f03a1859640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f039c159850 0x7f039c0cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f03a1859640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c06d1c0 0x7f039c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f03a1859640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0d3c20 con 0x7f03840051c0 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f03a1859640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0d4190 con 0x7f039c159850 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f03a1859640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f039c0d3da0 con 0x7f039c06d1c0 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f039c158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f039c158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51218/0 (socket says 172.21.15.40:51218) 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039affd640 1 -- 172.21.15.40:0/3702593086 learned_addr learned my addr 172.21.15.40:0/3702593086 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039b7fe640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c06d1c0 0x7f039c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039a7fc640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f039c159850 0x7f039c0cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039affd640 1 -- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c06d1c0 msgr2=0x7f039c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039affd640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c06d1c0 0x7f039c06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039affd640 1 -- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f039c159850 msgr2=0x7f039c0cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039affd640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f039c159850 0x7f039c0cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.177+0000 7f039affd640 1 -- 172.21.15.40:0/3702593086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c159e10 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.178+0000 7f039affd640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f039c158980 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f038c0119d0 tx=0x7f038c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.178+0000 7f0383fff640 1 -- 172.21.15.40:0/3702593086 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c00fa90 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.178+0000 7f0383fff640 1 -- 172.21.15.40:0/3702593086 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c00fc30 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.178+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c15a0f0 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.178+0000 7f0383fff640 1 -- 172.21.15.40:0/3702593086 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c014440 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.178+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c0cbfb0 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.179+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c0694f0 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.179+0000 7f0383fff640 1 -- 172.21.15.40:0/3702593086 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f038c012040 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.180+0000 7f0383fff640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f038806c8f0 0x7f038806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.180+0000 7f0383fff640 1 -- 172.21.15.40:0/3702593086 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f038c08c030 con 0x7f03840051c0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.180+0000 7f039a7fc640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f038806c8f0 0x7f038806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.180+0000 7f039a7fc640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f038806c8f0 0x7f038806edb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f039c0d14b0 tx=0x7f0390009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.184+0000 7f0383fff640 1 -- 172.21.15.40:0/3702593086 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f038c019030 con 0x7f03840051c0 2024-04-26T21:47:56.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.371+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f039c15a610 con 0x7f038806c8f0 2024-04-26T21:47:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.372+0000 7f0383fff640 1 -- 172.21.15.40:0/3702593086 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f039c15a610 con 0x7f038806c8f0 2024-04-26T21:47:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T21:47:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f038806c8f0 msgr2=0x7f038806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f038806c8f0 0x7f038806edb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f039c0d14b0 tx=0x7f0390009040 comp rx=0 tx=0).stop 2024-04-26T21:47:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 msgr2=0x7f039c158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f039c158980 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f038c0119d0 tx=0x7f038c011e90 comp rx=0 tx=0).stop 2024-04-26T21:47:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 shutdown_connections 2024-04-26T21:47:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f038806c8f0 0x7f038806edb0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f039c06d1c0 0x7f039c06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f039c159850 0x7f039c0cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 --2- 172.21.15.40:0/3702593086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03840051c0 0x7f039c158980 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 >> 172.21.15.40:0/3702593086 conn(0x7f039c0d5190 msgr2=0x7f039c161620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 shutdown_connections 2024-04-26T21:47:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.374+0000 7f03a1859640 1 -- 172.21.15.40:0/3702593086 wait complete. 2024-04-26T21:47:56.386 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:47:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.485+0000 7f0bd2f5f640 1 Processor -- start 2024-04-26T21:47:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.485+0000 7f0bd2f5f640 1 -- start start 2024-04-26T21:47:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bd2f5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0bcc0cf5e0 0x7f0bcc0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bd2f5f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bcc15b9e0 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bd2f5f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bcc0d0740 con 0x7f0bcc15b600 2024-04-26T21:47:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bd2f5f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0bcc0cb9b0 con 0x7f0bcc0cf5e0 2024-04-26T21:47:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bcbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0bcc0cf5e0 0x7f0bcc0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bd14fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0bcc0cbbd0 0x7f0bcc06f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54600/0 (socket says 172.21.15.40:54600) 2024-04-26T21:47:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bd14fc640 1 -- 172.21.15.40:0/471375698 learned_addr learned my addr 172.21.15.40:0/471375698 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:47:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bcbfff640 1 -- 172.21.15.40:0/471375698 >> v1:172.21.15.184:6789/0 conn(0x7f0bcc15b600 legacy=0x7f0bcc071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:47:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bcbfff640 1 -- 172.21.15.40:0/471375698 >> v1:172.21.15.40:6789/0 conn(0x7f0bcc0cbbd0 legacy=0x7f0bcc06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bcb7fe640 1 -- 172.21.15.40:0/471375698 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838283903 0 0) 0x7f0bcc15b9e0 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.486+0000 7f0bcbfff640 1 -- 172.21.15.40:0/471375698 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc0cb360 con 0x7f0bcc0cf5e0 2024-04-26T21:47:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.487+0000 7f0bcbfff640 1 --2- 172.21.15.40:0/471375698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cf5e0 0x7f0bcc0ce9d0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00097a0 tx=0x7f0bc002b2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7677e35d9802534a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.487+0000 7f0bcb7fe640 1 -- 172.21.15.40:0/471375698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0038070 con 0x7f0bcc0cf5e0 2024-04-26T21:47:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.487+0000 7f0bcb7fe640 1 -- 172.21.15.40:0/471375698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc0033440 con 0x7f0bcc0cf5e0 2024-04-26T21:47:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.487+0000 7f0bcb7fe640 1 -- 172.21.15.40:0/471375698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0033810 con 0x7f0bcc0cf5e0 2024-04-26T21:47:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.488+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/471375698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cf5e0 msgr2=0x7f0bcc0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.488+0000 7f0bd2f5f640 1 --2- 172.21.15.40:0/471375698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cf5e0 0x7f0bcc0ce9d0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00097a0 tx=0x7f0bc002b2b0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.488+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/471375698 shutdown_connections 2024-04-26T21:47:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.488+0000 7f0bd2f5f640 1 --2- 172.21.15.40:0/471375698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cf5e0 0x7f0bcc0ce9d0 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.488+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/471375698 >> 172.21.15.40:0/471375698 conn(0x7f0bcc0d0be0 msgr2=0x7f0bcc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.488+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/471375698 shutdown_connections 2024-04-26T21:47:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.488+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/471375698 wait complete. 2024-04-26T21:47:56.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.489+0000 7f0bd2f5f640 1 Processor -- start 2024-04-26T21:47:56.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.489+0000 7f0bd2f5f640 1 -- start start 2024-04-26T21:47:56.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd2f5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cbbd0 0x7f0bcc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd2f5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0cf5e0 0x7f0bcc1f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd2f5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc15b600 0x7f0bcc14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd2f5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0d0470 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd2f5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0d02f0 con 0x7f0bcc0cf5e0 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd2f5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc0cef10 con 0x7f0bcc15b600 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd0cfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cbbd0 0x7f0bcc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd0cfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cbbd0 0x7f0bcc0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51238/0 (socket says 172.21.15.40:51238) 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd0cfb640 1 -- 172.21.15.40:0/1253538108 learned_addr learned my addr 172.21.15.40:0/1253538108 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd14fc640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc15b600 0x7f0bcc14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bcbfff640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0cf5e0 0x7f0bcc1f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd0cfb640 1 -- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc15b600 msgr2=0x7f0bcc14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd0cfb640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc15b600 0x7f0bcc14c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd0cfb640 1 -- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0cf5e0 msgr2=0x7f0bcc1f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd0cfb640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0cf5e0 0x7f0bcc1f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.490+0000 7f0bd0cfb640 1 -- 172.21.15.40:0/1253538108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc0c7de0 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.491+0000 7f0bd0cfb640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cbbd0 0x7f0bcc0b51b0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400c740 tx=0x7f0bc400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.491+0000 7f0bc97fa640 1 -- 172.21.15.40:0/1253538108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4003b90 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.491+0000 7f0bc97fa640 1 -- 172.21.15.40:0/1253538108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc400ce50 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.491+0000 7f0bc97fa640 1 -- 172.21.15.40:0/1253538108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400a850 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.491+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc0c80c0 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.491+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc0c8680 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.492+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc0694f0 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.492+0000 7f0bc97fa640 1 -- 172.21.15.40:0/1253538108 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bc400a9f0 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.493+0000 7f0bc97fa640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bb806c8f0 0x7f0bb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:56.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.493+0000 7f0bc97fa640 1 -- 172.21.15.40:0/1253538108 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f0bc408b840 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.493+0000 7f0bcbfff640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bb806c8f0 0x7f0bb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:56.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.494+0000 7f0bcbfff640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bb806c8f0 0x7f0bb806edb0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0035000 tx=0x7f0bc002f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:56.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.497+0000 7f0bc97fa640 1 -- 172.21.15.40:0/1253538108 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0bc4059e20 con 0x7f0bcc0cbbd0 2024-04-26T21:47:56.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.678+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bcc0ce7b0 con 0x7f0bb806c8f0 2024-04-26T21:47:56.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.679+0000 7f0bc97fa640 1 -- 172.21.15.40:0/1253538108 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13475 (secure 0 0 0) 0x7f0bcc0ce7b0 con 0x7f0bb806c8f0 2024-04-26T21:47:56.680 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:47:56.681 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bb806c8f0 msgr2=0x7f0bb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bb806c8f0 0x7f0bb806edb0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0035000 tx=0x7f0bc002f000 comp rx=0 tx=0).stop 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cbbd0 msgr2=0x7f0bcc0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cbbd0 0x7f0bcc0b51b0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400c740 tx=0x7f0bc400cc00 comp rx=0 tx=0).stop 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 shutdown_connections 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f0bb806c8f0 0x7f0bb806edb0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc15b600 0x7f0bcc14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0cf5e0 0x7f0bcc1f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 --2- 172.21.15.40:0/1253538108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc0cbbd0 0x7f0bcc0b51b0 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 >> 172.21.15.40:0/1253538108 conn(0x7f0bcc0d0be0 msgr2=0x7f0bcc06f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:56.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 shutdown_connections 2024-04-26T21:47:56.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:47:56.682+0000 7f0bd2f5f640 1 -- 172.21.15.40:0/1253538108 wait complete. 2024-04-26T21:47:56.693 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-04-26T21:47:56.189025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":294824,"kb_used_data":16264,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754679896,"statfs":{"total":773094113280,"available":772792213504,"internally_reserved":0,"allocated":16654336,"data_stored":8994408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.583091"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":420,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-26T21:47:49.432803+0000","last_change":"2024-04-26T21:34:59.883812+0000","last_active":"2024-04-26T21:47:49.432803+0000","last_peered":"2024-04-26T21:47:49.432803+0000","last_clean":"2024-04-26T21:47:49.432803+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:47:49.432803+0000","last_undegraded":"2024-04-26T21:47:49.432803+0000","last_fullsized":"2024-04-26T21:47:49.432803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:34:58.044918+0000","last_clean_scrub_stamp":"2024-04-26T21:34:58.044918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:18.192479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37808,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334032,"statfs":{"total":96636764160,"available":96598048768,"internally_reserved":0,"allocated":2527232,"data_stored":1567077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41388,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330452,"statfs":{"total":96636764160,"available":96594382848,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33068,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33068,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33068,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37228,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37228,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41968,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329872,"statfs":{"total":96636764160,"available":96593788928,"internally_reserved":0,"allocated":2527232,"data_stored":1567077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:47:56.696 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T21:34:58.044918+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=34, tm_sec=58, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=47, tm_sec=51, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T21:47:56.696 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T21:48:13.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:13.136 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:16.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:48:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.812+0000 7fb7930ea640 1 Processor -- start 2024-04-26T21:48:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.812+0000 7fb7930ea640 1 -- start start 2024-04-26T21:48:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.812+0000 7fb7930ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c071e60 0x7fb78c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.812+0000 7fb7930ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb78c0cc480 0x7fb78c166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.812+0000 7fb7930ea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c15d7c0 con 0x7fb78c06fe20 2024-04-26T21:48:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.812+0000 7fb7930ea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb78c15daa0 con 0x7fb78c0cc480 2024-04-26T21:48:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.812+0000 7fb7930ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c15dd80 con 0x7fb78c071e60 2024-04-26T21:48:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.813+0000 7fb791687640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb78c0cc480 0x7fb78c166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.813+0000 7fb78bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb78c06fe20 0x7fb78c072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56338/0 (socket says 172.21.15.40:56338) 2024-04-26T21:48:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.813+0000 7fb78bfff640 1 -- 172.21.15.40:0/3252548031 learned_addr learned my addr 172.21.15.40:0/3252548031 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:48:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.813+0000 7fb790e86640 1 --2- 172.21.15.40:0/3252548031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c071e60 0x7fb78c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.813+0000 7fb791687640 1 -- 172.21.15.40:0/3252548031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c071e60 msgr2=0x7fb78c06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.813+0000 7fb791687640 1 --2- 172.21.15.40:0/3252548031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c071e60 0x7fb78c06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.813+0000 7fb791687640 1 -- 172.21.15.40:0/3252548031 >> v1:172.21.15.40:6789/0 conn(0x7fb78c06fe20 legacy=0x7fb78c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.813+0000 7fb791687640 1 -- 172.21.15.40:0/3252548031 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c15e060 con 0x7fb78c0cc480 2024-04-26T21:48:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.814+0000 7fb791687640 1 --2- 172.21.15.40:0/3252548031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c166ad0 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5810da2414b83f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.814+0000 7fb78b7fe640 1 -- 172.21.15.40:0/3252548031 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774037070 con 0x7fb78c0cc480 2024-04-26T21:48:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.814+0000 7fb78b7fe640 1 -- 172.21.15.40:0/3252548031 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb774032440 con 0x7fb78c0cc480 2024-04-26T21:48:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.814+0000 7fb78b7fe640 1 -- 172.21.15.40:0/3252548031 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774032810 con 0x7fb78c0cc480 2024-04-26T21:48:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.814+0000 7fb7930ea640 1 -- 172.21.15.40:0/3252548031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 msgr2=0x7fb78c166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.814+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3252548031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c166ad0 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402a180 comp rx=0 tx=0).stop 2024-04-26T21:48:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.815+0000 7fb7930ea640 1 -- 172.21.15.40:0/3252548031 shutdown_connections 2024-04-26T21:48:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.815+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3252548031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c166ad0 unknown :-1 s=CLOSED pgs=2946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.815+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3252548031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c071e60 0x7fb78c06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.815+0000 7fb7930ea640 1 -- 172.21.15.40:0/3252548031 >> 172.21.15.40:0/3252548031 conn(0x7fb78c0d0be0 msgr2=0x7fb78c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.815+0000 7fb7930ea640 1 -- 172.21.15.40:0/3252548031 shutdown_connections 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.815+0000 7fb7930ea640 1 -- 172.21.15.40:0/3252548031 wait complete. 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.816+0000 7fb7930ea640 1 Processor -- start 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.816+0000 7fb7930ea640 1 -- start start 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb7930ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c06fe20 0x7fb78c1f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb7930ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb78c071e60 0x7fb78c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb7930ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c1f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb7930ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c15d570 con 0x7fb78c0cc480 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb7930ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c15d3f0 con 0x7fb78c071e60 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb7930ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c072950 con 0x7fb78c06fe20 2024-04-26T21:48:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c1f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c1f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46420/0 (socket says 172.21.15.40:46420) 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 -- 172.21.15.40:0/3225487659 learned_addr learned my addr 172.21.15.40:0/3225487659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb790e86640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c06fe20 0x7fb78c1f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 -- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c06fe20 msgr2=0x7fb78c1f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c06fe20 0x7fb78c1f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 -- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb78c071e60 msgr2=0x7fb78c150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb78c071e60 0x7fb78c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 -- 172.21.15.40:0/3225487659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c1f7320 con 0x7fb78c0cc480 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.817+0000 7fb791687640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c1f6de0 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7fb774035040 tx=0x7fb7740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.818+0000 7fb7897fa640 1 -- 172.21.15.40:0/3225487659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774037040 con 0x7fb78c0cc480 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.818+0000 7fb7897fa640 1 -- 172.21.15.40:0/3225487659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb774042070 con 0x7fb78c0cc480 2024-04-26T21:48:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.818+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c1f3410 con 0x7fb78c0cc480 2024-04-26T21:48:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.818+0000 7fb7897fa640 1 -- 172.21.15.40:0/3225487659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740308a0 con 0x7fb78c0cc480 2024-04-26T21:48:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.818+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c1f39d0 con 0x7fb78c0cc480 2024-04-26T21:48:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.819+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78c0694f0 con 0x7fb78c0cc480 2024-04-26T21:48:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.819+0000 7fb7897fa640 1 -- 172.21.15.40:0/3225487659 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb774030a40 con 0x7fb78c0cc480 2024-04-26T21:48:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.820+0000 7fb7897fa640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.820+0000 7fb7897fa640 1 -- 172.21.15.40:0/3225487659 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fb7740ac320 con 0x7fb78c0cc480 2024-04-26T21:48:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.820+0000 7fb790e86640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:16.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.821+0000 7fb790e86640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb76c06c8f0 0x7fb76c06edb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb784009a10 tx=0x7fb784008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:16.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:16.824+0000 7fb7897fa640 1 -- 172.21.15.40:0/3225487659 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb78c1f7320 con 0x7fb78c0cc480 2024-04-26T21:48:17.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.010+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 --> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb78c156640 con 0x7fb76c06c8f0 2024-04-26T21:48:17.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.011+0000 7fb7897fa640 1 -- 172.21.15.40:0/3225487659 <== mgr.4100 v2:172.21.15.184:6832/2311011869 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fb78c156640 con 0x7fb76c06c8f0 2024-04-26T21:48:17.012 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:48:17.012 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:48:17.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb76c06c8f0 msgr2=0x7fb76c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb76c06c8f0 0x7fb76c06edb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb784009a10 tx=0x7fb784008040 comp rx=0 tx=0).stop 2024-04-26T21:48:17.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 msgr2=0x7fb78c1f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c1f6de0 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7fb774035040 tx=0x7fb7740046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 shutdown_connections 2024-04-26T21:48:17.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb78c0cc480 0x7fb78c1f6de0 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb78c071e60 0x7fb78c150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 --2- 172.21.15.40:0/3225487659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb78c06fe20 0x7fb78c1f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 >> 172.21.15.40:0/3225487659 conn(0x7fb78c0d0be0 msgr2=0x7fb78c15ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 shutdown_connections 2024-04-26T21:48:17.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.013+0000 7fb7930ea640 1 -- 172.21.15.40:0/3225487659 wait complete. 2024-04-26T21:48:17.024 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":541,"stamp":"2024-04-26T21:48:16.193184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":294816,"kb_used_data":16264,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754679904,"statfs":{"total":773094113280,"available":772792221696,"internally_reserved":0,"allocated":16654336,"data_stored":8994408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002503"},"pg_stats":[{"pgid":"1.0","version":"149'119","reported_seq":431,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-04-26T21:47:56.767161+0000","last_change":"2024-04-26T21:47:56.767216+0000","last_active":"2024-04-26T21:47:56.767161+0000","last_peered":"2024-04-26T21:47:56.767161+0000","last_clean":"2024-04-26T21:47:56.767161+0000","last_became_active":"2024-04-26T21:34:59.882818+0000","last_became_peered":"2024-04-26T21:34:59.882818+0000","last_unstale":"2024-04-26T21:47:56.767161+0000","last_undegraded":"2024-04-26T21:47:56.767161+0000","last_fullsized":"2024-04-26T21:47:56.767161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'119","last_scrub_stamp":"2024-04-26T21:47:56.767074+0000","last_deep_scrub":"149'119","last_deep_scrub_stamp":"2024-04-26T21:47:56.767074+0000","last_clean_scrub_stamp":"2024-04-26T21:47:56.767074+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:14.530669+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37804,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334036,"statfs":{"total":96636764160,"available":96598052864,"internally_reserved":0,"allocated":2527232,"data_stored":1567077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41388,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330452,"statfs":{"total":96636764160,"available":96594382848,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33068,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33068,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33068,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37228,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37228,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1933312,"data_stored":976709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41964,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329876,"statfs":{"total":96636764160,"available":96593793024,"internally_reserved":0,"allocated":2527232,"data_stored":1567077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:48:17.025 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T21:48:17.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.124+0000 7f5723647640 1 Processor -- start 2024-04-26T21:48:17.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.125+0000 7f5723647640 1 -- start start 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.125+0000 7f5723647640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0d1a00 0x7f571c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.125+0000 7f5723647640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f571c0d0a10 con 0x7f571c0b40d0 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.125+0000 7f5723647640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f571c0d0cf0 con 0x7f571c0d2320 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.125+0000 7f5723647640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f571c0d0fd0 con 0x7f571c0d1a00 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f5721be4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f571c0b40d0 0x7f571c0b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56346/0 (socket says 172.21.15.40:56346) 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f5721be4640 1 -- 172.21.15.40:0/958056119 learned_addr learned my addr 172.21.15.40:0/958056119 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f57213e3640 1 --2- 172.21.15.40:0/958056119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0d1a00 0x7f571c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f5713fff640 1 -- 172.21.15.40:0/958056119 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3132669759 0 0) 0x7f571c0d0a10 con 0x7f571c0b40d0 2024-04-26T21:48:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f5713fff640 1 -- 172.21.15.40:0/958056119 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5708003520 con 0x7f571c0b40d0 2024-04-26T21:48:17.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f5713fff640 1 -- 172.21.15.40:0/958056119 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349241775 0 0) 0x7f571c0d0cf0 con 0x7f571c0d2320 2024-04-26T21:48:17.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f57213e3640 1 -- 172.21.15.40:0/958056119 >> v1:172.21.15.184:6789/0 conn(0x7f571c0d2320 legacy=0x7f571c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f57213e3640 1 -- 172.21.15.40:0/958056119 >> v1:172.21.15.40:6789/0 conn(0x7f571c0b40d0 legacy=0x7f571c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.126+0000 7f57213e3640 1 -- 172.21.15.40:0/958056119 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c0d40a0 con 0x7f571c0d1a00 2024-04-26T21:48:17.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.127+0000 7f57213e3640 1 --2- 172.21.15.40:0/958056119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0d1a00 0x7f571c0d1de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f57180097d0 tx=0x7f571802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877f66155c12560 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.127+0000 7f5713fff640 1 -- 172.21.15.40:0/958056119 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718037070 con 0x7f571c0d1a00 2024-04-26T21:48:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.127+0000 7f5713fff640 1 -- 172.21.15.40:0/958056119 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5718032440 con 0x7f571c0d1a00 2024-04-26T21:48:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.127+0000 7f5713fff640 1 -- 172.21.15.40:0/958056119 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718032810 con 0x7f571c0d1a00 2024-04-26T21:48:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.128+0000 7f5723647640 1 -- 172.21.15.40:0/958056119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0d1a00 msgr2=0x7f571c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.128+0000 7f5723647640 1 --2- 172.21.15.40:0/958056119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0d1a00 0x7f571c0d1de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f57180097d0 tx=0x7f571802a180 comp rx=0 tx=0).stop 2024-04-26T21:48:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.128+0000 7f5723647640 1 -- 172.21.15.40:0/958056119 shutdown_connections 2024-04-26T21:48:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.128+0000 7f5723647640 1 --2- 172.21.15.40:0/958056119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0d1a00 0x7f571c0d1de0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.128+0000 7f5723647640 1 -- 172.21.15.40:0/958056119 >> 172.21.15.40:0/958056119 conn(0x7f571c0d4fd0 msgr2=0x7f571c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.128+0000 7f5723647640 1 -- 172.21.15.40:0/958056119 shutdown_connections 2024-04-26T21:48:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.128+0000 7f5723647640 1 -- 172.21.15.40:0/958056119 wait complete. 2024-04-26T21:48:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.129+0000 7f5723647640 1 Processor -- start 2024-04-26T21:48:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.129+0000 7f5723647640 1 -- start start 2024-04-26T21:48:17.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5723647640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0b40d0 0x7f571c0b9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5723647640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c0d1a00 0x7f571c1f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5723647640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f571c0d2320 0x7f571c1f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5723647640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f571c0b5210 con 0x7f571c0d2320 2024-04-26T21:48:17.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5723647640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f571c162f50 con 0x7f571c0d1a00 2024-04-26T21:48:17.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5723647640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f571c0b5390 con 0x7f571c0b40d0 2024-04-26T21:48:17.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f571c0d2320 0x7f571c1f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f571c0d2320 0x7f571c1f55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46432/0 (socket says 172.21.15.40:46432) 2024-04-26T21:48:17.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 -- 172.21.15.40:0/1588966055 learned_addr learned my addr 172.21.15.40:0/1588966055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:48:17.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f57213e3640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0b40d0 0x7f571c0b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5720be2640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c0d1a00 0x7f571c1f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 -- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0b40d0 msgr2=0x7f571c0b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0b40d0 0x7f571c0b9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 -- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c0d1a00 msgr2=0x7f571c1f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c0d1a00 0x7f571c1f5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 -- 172.21.15.40:0/1588966055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c0b9ea0 con 0x7f571c0d2320 2024-04-26T21:48:17.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5720be2640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c0d1a00 0x7f571c1f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:48:17.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5721be4640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f571c0d2320 0x7f571c1f55a0 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f571400cb70 tx=0x7f571400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.130+0000 7f5711ffb640 1 -- 172.21.15.40:0/1588966055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714017070 con 0x7f571c0d2320 2024-04-26T21:48:17.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.131+0000 7f5711ffb640 1 -- 172.21.15.40:0/1588966055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571400ad40 con 0x7f571c0d2320 2024-04-26T21:48:17.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.131+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571c0ba180 con 0x7f571c0d2320 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.131+0000 7f5711ffb640 1 -- 172.21.15.40:0/1588966055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714012620 con 0x7f571c0d2320 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.131+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571c0cdbb0 con 0x7f571c0d2320 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.132+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571c0694f0 con 0x7f571c0d2320 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.132+0000 7f5711ffb640 1 -- 172.21.15.40:0/1588966055 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f571401a070 con 0x7f571c0d2320 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.133+0000 7f5711ffb640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f570006c8f0 0x7f570006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.133+0000 7f5711ffb640 1 -- 172.21.15.40:0/1588966055 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7f571408b7f0 con 0x7f571c0d2320 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.133+0000 7f57213e3640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f570006c8f0 0x7f570006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.133+0000 7f57213e3640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f570006c8f0 0x7f570006edb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f5718034000 tx=0x7f571802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.137+0000 7f5711ffb640 1 -- 172.21.15.40:0/1588966055 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57140596f0 con 0x7f571c0d2320 2024-04-26T21:48:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.322+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f571c0bacb0 con 0x7f571c0d2320 2024-04-26T21:48:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.325+0000 7f5711ffb640 1 -- 172.21.15.40:0/1588966055 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v30)=0 v30) ==== 123+0+0 (secure 0 0 0) 0x7f571405d9f0 con 0x7f571c0d2320 2024-04-26T21:48:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.325+0000 7f5711ffb640 1 -- 172.21.15.40:0/1588966055 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5714062220 con 0x7f571c0d2320 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f570006c8f0 msgr2=0x7f570006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f570006c8f0 0x7f570006edb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f5718034000 tx=0x7f571802e000 comp rx=0 tx=0).stop 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f571c0d2320 msgr2=0x7f571c1f55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f571c0d2320 0x7f571c1f55a0 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f571400cb70 tx=0x7f571400a430 comp rx=0 tx=0).stop 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 shutdown_connections 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7f570006c8f0 0x7f570006edb0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f571c0d2320 0x7f571c1f55a0 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c0d1a00 0x7f571c1f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 --2- 172.21.15.40:0/1588966055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c0b40d0 0x7f571c0b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 >> 172.21.15.40:0/1588966055 conn(0x7f571c0d4fd0 msgr2=0x7f571c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 shutdown_connections 2024-04-26T21:48:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.327+0000 7f5723647640 1 -- 172.21.15.40:0/1588966055 wait complete. 2024-04-26T21:48:17.338 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:48:17.338 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:48:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.438+0000 7fe3a9cb8640 1 Processor -- start 2024-04-26T21:48:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.438+0000 7fe3a9cb8640 1 -- start start 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a9cb8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe3a40b2340 0x7fe3a40cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a9cb8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a40d10d0 con 0x7fe3a40cd930 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a9cb8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a40d1370 con 0x7fe3a4162810 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a9cb8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a40d0750 con 0x7fe3a40b2340 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe3a40cd930 0x7fe3a40d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56356/0 (socket says 172.21.15.40:56356) 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a3fff640 1 -- 172.21.15.40:0/814795863 learned_addr learned my addr 172.21.15.40:0/814795863 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3470047378 0 0) 0x7fe3a40d10d0 con 0x7fe3a40cd930 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe38c003520 con 0x7fe3a40cd930 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.439+0000 7fe3a2ffd640 1 --2- 172.21.15.40:0/814795863 >> v2:172.21.15.184:3300/0 conn(0x7fe3a40b2340 0x7fe3a40cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1953401666 0 0) 0x7fe38c003520 con 0x7fe3a40cd930 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 >> v2:172.21.15.184:3300/0 conn(0x7fe3a40b2340 msgr2=0x7fe3a40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 --2- 172.21.15.40:0/814795863 >> v2:172.21.15.184:3300/0 conn(0x7fe3a40b2340 0x7fe3a40cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 >> v1:172.21.15.184:6789/0 conn(0x7fe3a4162810 legacy=0x7fe3a4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a40d01b0 con 0x7fe3a40cd930 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2581442581 0 0) 0x7fe39c0019f0 con 0x7fe3a40cd930 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 >> v1:172.21.15.40:6789/0 conn(0x7fe3a40cd930 legacy=0x7fe3a40d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe38c0039b0 0x7fe38c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c004800 0x7fe38c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c005210 0x7fe38c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a40d0750 con 0x7fe38c004800 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c00c140 con 0x7fe38c0039b0 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe38c00c340 con 0x7fe38c005210 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a2ffd640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c004800 0x7fe38c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a37fe640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe38c0039b0 0x7fe38c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a3fff640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c005210 0x7fe38c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a2ffd640 1 -- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c005210 msgr2=0x7fe38c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a2ffd640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c005210 0x7fe38c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a2ffd640 1 -- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe38c0039b0 msgr2=0x7fe38c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a2ffd640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe38c0039b0 0x7fe38c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a2ffd640 1 -- 172.21.15.40:0/814795863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c003520 con 0x7fe38c004800 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a3fff640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c005210 0x7fe38c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.440+0000 7fe3a3fff640 1 -- 172.21.15.40:0/814795863 reap_dead start 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.441+0000 7fe3a2ffd640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c004800 0x7fe38c004c80 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fe39800ba30 tx=0x7fe38c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9e1618f5018e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.441+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398030070 con 0x7fe38c004800 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.441+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe39802ade0 con 0x7fe38c004800 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.441+0000 7fe3a27fc640 1 -- 172.21.15.40:0/814795863 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39800b600 con 0x7fe38c004800 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.442+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/814795863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c004800 msgr2=0x7fe38c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.442+0000 7fe3a9cb8640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c004800 0x7fe38c004c80 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fe39800ba30 tx=0x7fe38c005830 comp rx=0 tx=0).stop 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.442+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/814795863 shutdown_connections 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.442+0000 7fe3a9cb8640 1 --2- 172.21.15.40:0/814795863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c004800 0x7fe38c004c80 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.442+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/814795863 >> 172.21.15.40:0/814795863 conn(0x7fe3a40d2da0 msgr2=0x7fe3a40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.442+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/814795863 shutdown_connections 2024-04-26T21:48:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.442+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/814795863 wait complete. 2024-04-26T21:48:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.443+0000 7fe3a9cb8640 1 Processor -- start 2024-04-26T21:48:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.443+0000 7fe3a9cb8640 1 -- start start 2024-04-26T21:48:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a9cb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c004800 0x7fe3a4153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a9cb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4154650 0x7fe3a41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a9cb8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a4156cb0 0x7fe3a40cf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a9cb8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a40cd3f0 con 0x7fe3a4154650 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a9cb8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a40d16a0 con 0x7fe3a4156cb0 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a9cb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a40cd570 con 0x7fe38c004800 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4154650 0x7fe3a41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4154650 0x7fe3a41552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46442/0 (socket says 172.21.15.40:46442) 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 -- 172.21.15.40:0/683928845 learned_addr learned my addr 172.21.15.40:0/683928845 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a37fe640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c004800 0x7fe3a4153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a3fff640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a4156cb0 0x7fe3a40cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 -- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c004800 msgr2=0x7fe3a4153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c004800 0x7fe3a4153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 -- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a4156cb0 msgr2=0x7fe3a40cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a4156cb0 0x7fe3a40cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 -- 172.21.15.40:0/683928845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a41542b0 con 0x7fe3a4154650 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.444+0000 7fe3a2ffd640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4154650 0x7fe3a41552a0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7fe39800bb50 tx=0x7fe3980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.445+0000 7fe38bfff640 1 -- 172.21.15.40:0/683928845 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398043030 con 0x7fe3a4154650 2024-04-26T21:48:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.445+0000 7fe38bfff640 1 -- 172.21.15.40:0/683928845 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe39802abe0 con 0x7fe3a4154650 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.445+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a4155c10 con 0x7fe3a4154650 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.445+0000 7fe38bfff640 1 -- 172.21.15.40:0/683928845 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39800b600 con 0x7fe3a4154650 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.445+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a41561d0 con 0x7fe3a4154650 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.446+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a40694f0 con 0x7fe3a4154650 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.446+0000 7fe38bfff640 1 -- 172.21.15.40:0/683928845 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe398046070 con 0x7fe3a4154650 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.446+0000 7fe38bfff640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe39006c920 0x7fe39006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.447+0000 7fe38bfff640 1 -- 172.21.15.40:0/683928845 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5421+0+0 (secure 0 0 0) 0x7fe3980ac240 con 0x7fe3a4154650 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.447+0000 7fe3a37fe640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe39006c920 0x7fe39006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.447+0000 7fe3a37fe640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe39006c920 0x7fe39006ede0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe394000c00 tx=0x7fe394009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.451+0000 7fe38bfff640 1 -- 172.21.15.40:0/683928845 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3980333b0 con 0x7fe3a4154650 2024-04-26T21:48:17.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.674+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3a40ce090 con 0x7fe3a4154650 2024-04-26T21:48:17.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.674+0000 7fe38bfff640 1 -- 172.21.15.40:0/683928845 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 365 v365) ==== 95+0+3879 (secure 0 0 0) 0x7fe39807a700 con 0x7fe3a4154650 2024-04-26T21:48:17.675 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:48:17.675 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":365,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8552,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2523243108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2523243108},{"type":"v1","addr":"172.21.15.184:6836","nonce":2523243108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":9724,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1305263870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1305263870},{"type":"v1","addr":"172.21.15.184:6837","nonce":1305263870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354},{"gid":11751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4139814659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4139814659},{"type":"v1","addr":"172.21.15.40:6833","nonce":4139814659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":353},{"gid":12300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/809239708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":809239708},{"type":"v1","addr":"172.21.15.40:6835","nonce":809239708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":365}],"filesystems":[]} 2024-04-26T21:48:17.675 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 365 2024-04-26T21:48:17.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe39006c920 msgr2=0x7fe39006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe39006c920 0x7fe39006ede0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe394000c00 tx=0x7fe394009040 comp rx=0 tx=0).stop 2024-04-26T21:48:17.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4154650 msgr2=0x7fe3a41552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4154650 0x7fe3a41552a0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7fe39800bb50 tx=0x7fe3980046c0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 shutdown_connections 2024-04-26T21:48:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:6832/2311011869,v1:172.21.15.184:6833/2311011869] conn(0x7fe39006c920 0x7fe39006ede0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a4156cb0 0x7fe3a40cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4154650 0x7fe3a41552a0 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 --2- 172.21.15.40:0/683928845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe38c004800 0x7fe3a4153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 >> 172.21.15.40:0/683928845 conn(0x7fe3a40d2da0 msgr2=0x7fe3a40d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 shutdown_connections 2024-04-26T21:48:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:48:17.676+0000 7fe3a9cb8640 1 -- 172.21.15.40:0/683928845 wait complete. 2024-04-26T21:48:17.688 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T21:48:17.688 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T21:48:17.688 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:48:17.688 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:17.697 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:48:17.697 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:48:17.697 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:17.747 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:48:17.747 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:48:17.747 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:17.758 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:48:17.759 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:48:17.759 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:17.810 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:48:17.810 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T21:48:17.810 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T21:48:17.811 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:17.840 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T21:48:17.840 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T21:48:17.840 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:17.912 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T21:48:17.913 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T21:48:17.913 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:17.984 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T21:48:17.984 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T21:48:17.984 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.050 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T21:48:18.051 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T21:48:18.051 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.083 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T21:48:18.083 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T21:48:18.084 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.155 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T21:48:18.155 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T21:48:18.155 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.224 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T21:48:18.224 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T21:48:18.224 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.300 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T21:48:18.300 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T21:48:18.301 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T21:48:18.301 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.325 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T21:48:18.326 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T21:48:18.326 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.368 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T21:48:18.368 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T21:48:18.368 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T21:48:18.368 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.380 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T21:48:18.381 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T21:48:18.381 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.420 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T21:48:18.420 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T21:48:18.420 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:18.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-26T21:48:18.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-26T21:48:18.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-26T21:48:18.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-26T21:48:18.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-26T21:48:18.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-04-26T21:48:18.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-04-26T21:48:18.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-26T21:48:18.473 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T21:48:18.473 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T21:48:18.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | egrep -v 'MDS abort because newly corrupt dentry' | head -n 1 2024-04-26T21:48:18.512 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:48:18.512 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T21:48:18.720 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:48:18.721 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T21:48:18.828 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:48:18.828 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T21:48:18.884 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:48:18.884 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T21:48:18.951 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:48:18.951 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T21:48:19.115 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:48:19.115 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T21:48:19.193 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:48:19.193 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T21:48:19.251 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:48:19.251 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T21:48:19.310 INFO:tasks.ceph:Archiving mon data... 2024-04-26T21:48:19.311 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/data/mon.a.tgz 2024-04-26T21:48:19.311 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-26T21:48:19.327 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.FLj47tGlJL 2024-04-26T21:48:19.327 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FLj47tGlJL 2024-04-26T21:48:19.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.FLj47tGlJL 2024-04-26T21:48:19.612 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.FLj47tGlJL is 660KB 2024-04-26T21:48:19.713 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.FLj47tGlJL 2024-04-26T21:48:19.727 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/data/mon.b.tgz 2024-04-26T21:48:19.727 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-04-26T21:48:19.743 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.mJE7ny24Va 2024-04-26T21:48:19.743 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.mJE7ny24Va 2024-04-26T21:48:19.944 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.mJE7ny24Va 2024-04-26T21:48:20.025 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.mJE7ny24Va is 687KB 2024-04-26T21:48:20.124 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.mJE7ny24Va 2024-04-26T21:48:20.138 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/data/mon.c.tgz 2024-04-26T21:48:20.139 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-04-26T21:48:20.192 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.TINoRz3OWM 2024-04-26T21:48:20.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.TINoRz3OWM 2024-04-26T21:48:20.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.TINoRz3OWM 2024-04-26T21:48:20.466 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.TINoRz3OWM is 687KB 2024-04-26T21:48:20.565 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.TINoRz3OWM 2024-04-26T21:48:20.579 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T21:48:20.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:48:20.582 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:48:20.664 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:48:20.696 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:48:20.697 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T21:48:20.698 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/remote/smithi040/crash 2024-04-26T21:48:20.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:48:20.769 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/remote/smithi184/crash 2024-04-26T21:48:20.770 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:48:20.792 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T21:48:20.792 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:20.795 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:20.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:20.916 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:48:20.939 INFO:tasks.ceph:Compressing logs... 2024-04-26T21:48:20.940 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:48:20.959 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:48:20.982 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-04-26T21:48:21.003 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34578.log 2024-04-26T21:48:21.026 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34665.log 2024-04-26T21:48:21.026 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.34578.log: /var/log/ceph/ceph-client.admin.34665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41564.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34578.log.gz 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.34665.log.gz -- 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr: /var/log/ceph/ceph.audit.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.3.41564.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41564.log.gz 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34705.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.34705.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34705.log.gz 2024-04-26T21:48:21.027 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mgr.x.log: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34534.log 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.6.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41166.log 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.34534.log: 87.4% 90.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34534.log.gz 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34454.log 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.41166.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41166.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34494.log 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.34454.log: /var/log/ceph/ceph-client.admin.34745.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34454.log.gz 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.34494.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T21:48:21.028 INFO:teuthology.orchestra.run.smithi184.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34494.log.gz 2024-04-26T21:48:21.029 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34625.log 2024-04-26T21:48:21.029 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T21:48:21.029 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.34625.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34625.log.gz 2024-04-26T21:48:21.207 INFO:teuthology.orchestra.run.smithi184.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91959.log 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52136.log 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95103.log 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75964.log 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52136.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91959.log.gz 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85512.log 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52136.log.gz 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49852.log 2024-04-26T21:48:21.426 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95103.log.gz 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75964.log.gz 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73504.log 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85512.log.gz 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63611.log 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73504.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49852.log.gz 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89893.log 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63611.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.73504.log.gz 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43797.log 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63611.log.gz 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77221.log 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89893.log: /var/log/ceph/ceph-client.admin.43797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82452.log 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89893.log.gz 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77221.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.43797.log.gz --verbose 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.51056.log 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77221.log.gz 2024-04-26T21:48:21.427 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51490.log 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55414.log 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.82452.log.gz 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51056.log.gz 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86367.log 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69661.log 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51490.log.gz 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55414.log.gz 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91156.log 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86367.log.gz 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94039.log 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91156.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.69661.log.gz 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46468.log 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94039.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91156.log.gz 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84015.log 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94039.log.gz 2024-04-26T21:48:21.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53222.log 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46468.log.gz 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57266.log 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53222.log: gzip -5 89.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.84015.log.gz -- 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.91454.log 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53222.log.gz 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95253.log 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57266.log: /var/log/ceph/ceph-client.admin.91454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96409.log 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57266.log.gz 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91454.log.gz/var/log/ceph/ceph-client.admin.95253.log: 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64503.log 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49640.log 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2%/var/log/ceph/ceph-client.admin.64503.log: -- replaced with /var/log/ceph/ceph-client.admin.95253.log.gz 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58776.log 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96409.log.gz 2024-04-26T21:48:21.429 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49640.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64503.log.gz 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65688.log 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49640.log.gz 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76175.log 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58776.log.gz 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82350.log 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76175.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65688.log.gz 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50605.log 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92001.log 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76175.log.gz 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82350.log.gz 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91496.log 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50605.log.gz 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65436.log 2024-04-26T21:48:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91496.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92001.log.gz 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51140.log 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91496.log.gz 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73560.log 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51140.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65436.log.gz 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91622.log 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73560.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51140.log.gz 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69291.log 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73560.log.gz 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56474.log 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69291.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91622.log.gz 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89370.log 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56474.log: gzip -5 --verbose -- 89.9% /var/log/ceph/ceph-client.admin.55765.log 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69291.log.gz 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93325.log 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55765.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56474.log.gz 2024-04-26T21:48:21.431 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89370.log.gz 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49306.log 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55765.log.gz 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91664.log 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84932.log 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93325.log.gz 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48505.log 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49306.log.gz 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84932.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91664.log.gz 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90880.log 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49947.log 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90880.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48505.log.gz 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80721.log 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49947.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90880.log.gz 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49947.log.gz 2024-04-26T21:48:21.432 INFO:teuthology.orchestra.run.smithi040.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.84932.log.gz 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58566.log 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80721.log: /var/log/ceph/ceph-client.admin.47419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47224.log 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80721.log.gz 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58566.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78591.log 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58566.log.gz 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68810.log 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78591.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47224.log.gz 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82781.log 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56953.log 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91706.log 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.68810.log.gz 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56953.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82781.log.gz 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88818.log 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.78591.log.gz 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56953.log.gz 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92229.log 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44950.log 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91706.log.gz 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.92229.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49387.log 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88818.log.gz 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71763.log 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92229.log.gz 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49387.log: gzip -5 88.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.44950.log.gz /var/log/ceph/ceph-client.admin.41098.log 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71763.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49387.log.gz 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57620.log 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71763.log.gz 2024-04-26T21:48:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41098.log.gz 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62310.log 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44992.log 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57620.log.gz 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz -5 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.78020.log/var/log/ceph/ceph-client.admin.62310.log: 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44992.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62310.log.gz 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42988.log 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44992.log.gz 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65478.log 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78020.log.gz 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42988.log.gz 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89935.log 2024-04-26T21:48:21.435 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84969.log 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89935.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65478.log.gz 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48631.log 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84969.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.89935.log.gz -5 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41186.log 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48631.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.84969.log.gz 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66169.log 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48631.log.gz 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44542.log 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66169.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41186.log.gz 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91358.log 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66169.log.gz 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86826.log 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91358.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.44542.log.gz 2024-04-26T21:48:21.436 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.86230.log 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86826.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91358.log.gz 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69698.log 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80511.log 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69698.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86230.log.gz 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93217.log 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68073.log 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.69698.log.gz 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.86826.log.gz 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93217.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.80511.log.gz 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64802.log 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93217.log.gz 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58002.log 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64802.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68073.log.gz 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66723.log 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58002.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64802.log.gz 2024-04-26T21:48:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79984.log 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66723.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58002.log.gz 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80862.log 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79984.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66723.log.gz 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60833.log 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79984.log.gz 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74659.log 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49039.log 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74659.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60833.log.gz 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41555.log 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74659.log.gz 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50793.log 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61144.log 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49039.log.gz 2024-04-26T21:48:21.438 INFO:teuthology.orchestra.run.smithi040.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.80862.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41555.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.50793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73955.log 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50793.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55138.log 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.73955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41387.log 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61144.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73955.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65959.log 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55138.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96367.log 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65959.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41387.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74735.log 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96367.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65959.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51376.log 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96367.log.gz 2024-04-26T21:48:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60103.log 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74735.log.gzgzip 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.89053.log 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51376.log: /var/log/ceph/ceph-client.admin.60103.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28608.log 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51376.log.gz 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89053.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60103.log.gz 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45559.log 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.tmp-client.admin.28608.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89053.log.gz 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74617.log 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28608.log.gz 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.45559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69809.log 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45559.log.gz 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81227.log 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74617.log.gz 2024-04-26T21:48:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69476.log 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.69809.log.gz 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75804.log 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81227.log.gz 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63346.log 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.69476.log.gz 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75804.log.gz 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66127.log 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47707.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63346.log.gz 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54868.log 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66127.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71786.log 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66127.log.gz 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41255.log 2024-04-26T21:48:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71786.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54868.log.gz 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84707.log 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46972.log 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41255.log.gz 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48295.log 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84707.log: /var/log/ceph/ceph-client.admin.46972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49556.log 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84707.log.gz 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.71786.log.gz 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48295.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46972.log.gz 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90712.log 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48295.log.gz 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82950.log 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90712.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49556.log.gz 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46510.log 2024-04-26T21:48:21.442 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90712.log.gz 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62394.log 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91274.log 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82950.log.gz 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96054.log 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46510.log.gz 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91274.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62394.log.gz 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92919.log 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91274.log.gz 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75047.log 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71375.log 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96054.log.gz 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94305.log 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92919.log.gz 2024-04-26T21:48:21.443 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75047.log.gz 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60259.log 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59840.log 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71375.log.gz 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94305.log.gz 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73306.log 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60259.log.gz 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.59840.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.94124.log 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73306.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59840.log.gz 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94124.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.73306.log.gz -- 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.75888.log 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75005.log 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75888.log: gzip -5 --verbose -- 91.5% /var/log/ceph/ceph-client.admin.80433.log 2024-04-26T21:48:21.444 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94124.log.gz 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75888.log.gz 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68995.log 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79298.log 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75005.log.gz 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80433.log.gz 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90061.log 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52597.log 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.68995.log.gz 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90138.log 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52597.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90061.log.gz 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79298.log.gz 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63439.log 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52597.log.gz 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63527.log 2024-04-26T21:48:21.445 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90138.log.gz 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64928.log 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59966.log 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63439.log.gz 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64928.log: 87.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63527.log.gz 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55180.log 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64928.log.gz 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59966.log.gz 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72738.log 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55180.log: /var/log/ceph/ceph-client.admin.44771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82274.log 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55180.log.gz 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74893.log 2024-04-26T21:48:21.446 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45998.log 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74893.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72738.log.gz 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82274.log.gz 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74893.log.gz 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.2.45998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82120.log 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45998.log.gz 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85297.log 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93007.log 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82120.log.gz 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.85297.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.83805.log 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93007.log: gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.admin.87410.log -- replaced with /var/log/ceph/ceph-client.admin.85297.log.gz 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83805.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93007.log.gz 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47503.log 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87410.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83805.log.gz 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87410.log.gz 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57662.log 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47503.log: /var/log/ceph/ceph-client.admin.44374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91316.log 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47503.log.gz 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45240.log 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57662.log.gz 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44332.log 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45240.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91316.log.gz 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67094.log 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45240.log.gz 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46594.log 2024-04-26T21:48:21.448 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44332.log.gz 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84581.log 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67094.log: /var/log/ceph/ceph-client.admin.46594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75334.log 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67094.log.gz 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46594.log.gz 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.84581.log: /var/log/ceph/ceph-client.admin.59497.log 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52220.log 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.84581.log.gz 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71174.log 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75334.log.gz 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59497.log.gz 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77179.log 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52220.log.gz 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79168.log 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77179.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71174.log.gz 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96180.log 2024-04-26T21:48:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79168.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77179.log.gz 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53138.log 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96180.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79168.log.gz 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96180.log.gz 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85255.log 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58884.log 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53138.log.gz 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67262.log 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85255.log.gz 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60516.log 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67262.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58884.log.gz -5 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.95145.log 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60516.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67262.log.gz 2024-04-26T21:48:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79256.log 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47875.log 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95145.log: /var/log/ceph/ceph-client.admin.79256.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.60516.log.gz 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43798.log 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79256.log.gz 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.95145.log.gz 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93794.log 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47875.log: /var/log/ceph/ceph-client.admin.43798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65856.log 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47875.log.gz 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43798.log.gz 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79781.log 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93794.log.gz 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79781.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65856.log.gz 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79781.log.gz 2024-04-26T21:48:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56624.log 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43800.log 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56624.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80161.log 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56624.log.gz 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.43800.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63671.log 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81962.log 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43800.log.gz 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63671.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80161.log.gz 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92877.log 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63671.log.gz 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84867.log 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92877.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.81962.log.gz 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86484.log 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84867.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92877.log.gz 2024-04-26T21:48:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92696.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84867.log.gz 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78296.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86484.log.gz 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55222.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96012.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92696.log.gz 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78296.log.gz 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75618.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55222.log.gz 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57350.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75618.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96012.log.gz 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69439.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75618.log.gz 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96096.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57350.log.gz/var/log/ceph/ceph-client.admin.69439.log: 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74449.log 2024-04-26T21:48:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51098.log 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69439.log.gz 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74449.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96096.log.gz 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57536.log 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.74449.log.gz 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41345.log 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47182.log 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51098.log.gz 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57536.log.gz 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56540.log 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65342.log 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41345.log.gz 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56540.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47182.log.gz 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78684.log 2024-04-26T21:48:21.454 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56540.log.gz 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40887.log 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65342.log: /var/log/ceph/ceph-client.admin.78684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55096.log 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65342.log.gz 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78684.log.gz 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85095.log 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55096.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40887.log.gz -- 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.95784.log 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40930.log 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55096.log.gz 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85095.log.gz 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74777.log 2024-04-26T21:48:21.455 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95784.log.gz 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57960.log 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74777.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40930.log.gz 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48997.log 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57960.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74777.log.gz 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47629.log 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57960.log.gz 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87314.log 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67389.log 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48997.log.gz 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.87314.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47629.log.gz 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77732.log 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.87314.log.gz 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75188.log 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67389.log.gz 2024-04-26T21:48:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81185.log 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77732.log.gz 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43455.log 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72439.log 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81185.log.gz 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70700.log 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88758.log 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72439.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43455.log.gz 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70700.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72439.log.gz 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr: 92.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.75188.log.gz -5 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72541.log 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75846.log 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70700.log.gz 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88758.log: /var/log/ceph/ceph-client.admin.72541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66253.log 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88758.log.gz 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75846.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72541.log.gz 2024-04-26T21:48:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78747.log 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95480.log 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75846.log.gz 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66253.log.gz 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44133.log 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78747.log.gz 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58524.log 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52178.log 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95480.log.gz 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44133.log.gz 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58482.log 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58524.log.gz 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52178.log: /var/log/ceph/ceph-client.admin.58482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65814.log 2024-04-26T21:48:21.458 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52178.log.gz 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.audit.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58482.log.gz 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67727.log 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5/var/log/ceph/ceph-client.admin.65814.log: --verbose -- /var/log/ceph/ceph-client.admin.54687.log 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.67727.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65814.log.gz 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69624.log 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67727.log.gz 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63302.log 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69624.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54687.log.gz 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84249.log 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90768.log 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63302.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69624.log.gz 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46258.log 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63302.log.gz 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84249.log.gz 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41705.log 2024-04-26T21:48:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46258.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90768.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51644.log 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46258.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65520.log 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41705.log: /var/log/ceph/ceph-client.admin.51644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75678.log 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41705.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65520.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51644.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54557.log 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82897.log 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65520.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75678.log: /var/log/ceph/ceph-client.admin.54557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68236.log 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75678.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82897.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54557.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44500.log 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82897.log.gz 2024-04-26T21:48:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79739.log 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88681.log 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79739.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44500.log.gz 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95319.log 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79739.log.gz 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80315.log 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88681.log: /var/log/ceph/ceph-client.admin.95319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57392.log 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88681.log.gz 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.68236.log.gz 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95319.log.gz 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80679.log 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80315.log.gz 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57392.log: /var/log/ceph/ceph-client.admin.42574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97596.log 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57392.log.gz 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80679.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz 2024-04-26T21:48:21.461 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47587.log 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97596.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80679.log.gz 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59312.log 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47587.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97596.log.gz 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52447.log 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79406.log 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47587.log.gz 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52447.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59312.log.gz 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57079.log 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66595.log 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52447.log.gz 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79406.log.gz 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74167.log 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57079.log.gz 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71963.log 2024-04-26T21:48:21.462 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66595.log.gz 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63397.log 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74167.log.gz 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41142.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71963.log.gz 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56432.log 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44729.log 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93920.log 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63397.log.gz 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44729.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56432.log.gz 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69550.log 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44729.log.gz 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43463.log 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63258.log 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93920.log.gz 2024-04-26T21:48:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46342.log 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.69550.log.gz 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43463.log.gz 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51217.log 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96325.log 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63258.log.gz 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46342.log.gz 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94351.log 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51217.log.gz 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51536.log 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87916.log 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96325.log.gz 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51536.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94351.log.gz 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70982.log 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51536.log.gz 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87676.log 2024-04-26T21:48:21.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56995.log 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87916.log.gz 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87676.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.70982.log.gz 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88639.log 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87676.log.gz 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54349.log 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50329.log 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56995.log.gz 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88639.log.gz 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44458.log 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54349.log.gz 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52918.log 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43456.log 2024-04-26T21:48:21.465 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50329.log.gz 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52918.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44458.log.gz 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69402.log 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52918.log.gz 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91832.log 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83931.log 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43456.log.gz 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72650.log 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69402.log.gz 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91832.log.gz 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69032.log 2024-04-26T21:48:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72650.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83931.log.gz 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69106.log 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72650.log.gz/var/log/ceph/ceph-client.admin.69032.log: 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72804.log 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69106.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69032.log.gz 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86104.log 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74800.log 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69106.log.gz 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86104.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72804.log.gz 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85137.log 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86104.log.gz 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69772.log 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74800.log: /var/log/ceph/ceph-client.admin.85137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78797.log 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69772.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85137.log.gz 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6%/var/log/ceph/ceph-client.admin.78797.log: -- replaced with /var/log/ceph/ceph-client.admin.69772.log.gz 2024-04-26T21:48:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74388.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78797.log.gz 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.3.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.74800.log.gz 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97274.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42742.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74388.log: /var/log/ceph/ceph-client.admin.97274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74388.log.gz 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97274.log.gz 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90451.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46132.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42742.log.gz 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97106.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41301.log 2024-04-26T21:48:21.468 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.90451.log.gz 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97106.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41301.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97106.log.gz 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78364.log 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41301.log.gz 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87718.log 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43795.log: /var/log/ceph/ceph-client.admin.78364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90610.log 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87718.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78364.log.gz 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86644.log 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87718.log.gz 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76687.log 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90610.log: /var/log/ceph/ceph-client.admin.86644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89469.log 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90610.log.gz 2024-04-26T21:48:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76687.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.81802.log 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86644.log.gz 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89469.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76687.log.gz 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73120.log 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81802.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89469.log.gz 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78240.log 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81802.log.gz 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69069.log 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73120.log: /var/log/ceph/ceph-client.admin.78240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68699.log 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73120.log.gz 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69069.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.78240.log.gz 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81638.log 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94707.log 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69069.log.gz 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68699.log: /var/log/ceph/ceph-client.admin.81638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91538.log 2024-04-26T21:48:21.470 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68699.log.gz 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63755.log 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.94707.log.gz 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79214.log 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91538.log.gz 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67178.log 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.81638.log.gz 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79214.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.63755.log.gz 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54006.log 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77792.log 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79214.log.gz 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67178.log: /var/log/ceph/ceph-client.admin.54006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76122.log 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67178.log.gz 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78450.log 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54006.log.gz 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76122.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77792.log.gz 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71721.log 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76122.log.gz 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87634.log 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78450.log.gz 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51945.log 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71721.log: /var/log/ceph/ceph-client.admin.87634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55807.log 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71721.log.gz 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51945.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87634.log.gz 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85922.log 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55807.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51945.log.gz 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91874.log 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55807.log.gz 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41471.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85922.log.gz 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68550.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91874.log: /var/log/ceph/ceph-client.admin.41471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83056.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91874.log.gz 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41471.log.gz 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49682.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80474.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83056.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.68550.log.gz 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64274.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80474.log: 91.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.83056.log.gz 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61098.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49682.log.gz 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47098.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83522.log 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi040.stderr: 91.4%/var/log/ceph/ceph-client.admin.47098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76517.log 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64274.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61098.log.gz 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47098.log.gz 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83122.log 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83522.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.80474.log.gz 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66211.log 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83522.log.gz 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.76517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76517.log.gz 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42862.log 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85826.log 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83122.log.gz 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66211.log.gz 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50497.log 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42862.log.gz 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.85826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56911.log 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.85826.log.gz 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86188.log 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50497.log: /var/log/ceph/ceph-client.admin.56911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73443.log 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50497.log.gz 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56911.log.gz 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69977.log 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73443.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86188.log.gz 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73443.log.gz 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62539.log 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69977.log.gz 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67304.log 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62539.log: /var/log/ceph/ceph-client.admin.42616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78839.log 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62539.log.gz 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68921.log 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67304.log.gz 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43462.log 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65216.log 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68921.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78839.log.gz 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81343.log 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68921.log.gz 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91040.log 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43462.log.gz 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76329.log 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65216.log.gz 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46636.log 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81343.log.gz 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76329.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.91040.log.gz --verbose 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.97358.log 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76329.log.gz 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50077.log 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97358.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.46636.log.gz --verbose 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.73701.log 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54928.log 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97358.log.gz 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73701.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50077.log.gz 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58608.log 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83480.log 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54928.log: /var/log/ceph/ceph-client.admin.58608.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54928.log.gz 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose 90.7% -- replaced with /var/log/ceph/ceph-client.admin.73701.log.gz 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.55306.log 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83480.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58608.log.gz 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70037.log 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83480.log.gz 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95564.log 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70037.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55306.log.gz 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74328.log 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70037.log.gz 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69254.log 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74328.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95564.log.gz 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63905.log 2024-04-26T21:48:21.478 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74328.log.gz 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52531.log 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88197.log 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.69254.log.gz 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63905.log.gz 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85320.log 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78915.log 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88197.log: /var/log/ceph/ceph-client.admin.85320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70234.log 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88197.log.gz 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52531.log.gz 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58226.log 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78915.log: /var/log/ceph/ceph-client.admin.70234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85179.log 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8%/var/log/ceph/ceph-client.admin.58226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48737.log 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78915.log.gz/var/log/ceph/ceph-client.admin.85179.log: 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.70234.log.gz 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58226.log.gz 2024-04-26T21:48:21.479 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.85320.log.gz 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74209.log 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85179.log.gz 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89212.log 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82621.log 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48737.log.gz 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74209.log.gz 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76645.log 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94833.log 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82621.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89212.log.gz 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76066.log 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82621.log.gz 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94833.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76645.log.gz 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76066.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94833.log.gz 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94983.log 2024-04-26T21:48:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65730.log 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76066.log.gz 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.94983.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81731.log 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65730.log.gz 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.94983.log.gz 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81497.log 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57918.log 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81497.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81731.log.gz 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74837.log 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81497.log.gz/var/log/ceph/ceph-client.admin.57918.log: 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96644.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57918.log.gz 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87060.log 89.8% -- replaced with /var/log/ceph/ceph-client.admin.74837.log.gz 2024-04-26T21:48:21.481 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91100.log 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96644.log.gz 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84749.log 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50287.log 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.87060.log.gz 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91100.log.gz 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84749.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84749.log.gz 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79073.log 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55926.log 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79073.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50287.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53922.log 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79073.log.gz 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68736.log 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55926.log: /var/log/ceph/ceph-client.admin.53922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43792.log 2024-04-26T21:48:21.482 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%/var/log/ceph/ceph-client.admin.68736.log: -- replaced with /var/log/ceph/ceph-client.admin.55926.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53922.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58818.log 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68736.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88385.log 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82841.log 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43792.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58818.log: /var/log/ceph/ceph-client.0.88385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.88385.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58818.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91917.log 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64587.log 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91917.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.82841.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96980.log 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91917.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64587.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64587.log.gz 2024-04-26T21:48:21.483 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62226.log 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67052.log 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62226.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96980.log.gz 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78957.log 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62226.log.gz 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54738.log 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67052.log: gzip/var/log/ceph/ceph-client.admin.78957.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.66862.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67052.log.gz 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54738.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78957.log.gz 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43793.log 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54738.log.gz 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68476.log 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43793.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66862.log.gz 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T21:48:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68476.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43793.log.gz 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43996.log 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68476.log.gz 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74963.log 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59072.log 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74963.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43996.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74412.log 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74963.log.gz 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72992.log 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.59072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59072.log.gz/var/log/ceph/ceph-client.admin.74412.log: gzip 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48421.log 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42432.log 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48421.log: /var/log/ceph/ceph-client.0.72992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72992.log.gz 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48421.log.gz 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47461.log 2024-04-26T21:48:21.485 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87855.log 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74412.log.gz 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42432.log: /var/log/ceph/ceph-client.admin.47461.log: gzip -5 --verbose -- 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42432.log.gz 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.51430.log 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68958.log 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47461.log.gz 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87855.log.gz 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84091.log 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85025.log 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51430.log.gz 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84091.log: -- replaced with /var/log/ceph/ceph-client.admin.68958.log.gz 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72597.log 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84091.log.gz 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82064.log 2024-04-26T21:48:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72597.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.85025.log.gz /var/log/ceph/ceph-client.0.64093.log 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80955.log 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72597.log.gz 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82064.log: /var/log/ceph/ceph-client.0.64093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64093.log.gz 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80955.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.82064.log.gz 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86307.log 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80955.log.gz 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84133.log 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69365.log 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86307.log: /var/log/ceph/ceph-client.admin.84133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83861.log 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86307.log.gz 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69365.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84133.log.gz 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59924.log 2024-04-26T21:48:21.487 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69365.log.gz 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52960.log 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89511.log 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83861.log.gz 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52960.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.59924.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.87018.log 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89095.log 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87018.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.52960.log.gz 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87018.log.gz 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.89511.log.gz 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87795.log 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86863.log 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87795.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89095.log.gz 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53567.log 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87795.log.gz 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83564.log 2024-04-26T21:48:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86863.log: /var/log/ceph/ceph-client.admin.53567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70093.log 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83564.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53567.log.gz 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86863.log.gz 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40587.log 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83564.log.gz 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72145.log 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55884.log 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70093.log.gz 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40587.log: /var/log/ceph/ceph-client.admin.72145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72122.log 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55884.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40587.log.gz 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.55884.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.42474.log 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83310.log 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72122.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.72145.log.gz 2024-04-26T21:48:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82804.log 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72122.log.gz 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50245.log 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.83310.log: /var/log/ceph/ceph-client.admin.82804.log: 0.0% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42474.log.gz 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.0.83310.log.gz 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89577.log 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63839.log 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50245.log.gz 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89577.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.82804.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72005.log 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%/var/log/ceph/ceph-client.admin.63839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53399.log 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.89577.log.gz 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96854.log 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72005.log.gz 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63839.log.gz 2024-04-26T21:48:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68328.log 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92835.log 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53399.log.gz 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68328.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46846.log 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96854.log.gz 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68328.log.gz 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93049.log 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71398.log 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92835.log.gz 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93049.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46846.log.gz 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93049.log.gz 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.71398.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.97148.log 2024-04-26T21:48:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85357.log 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1%/var/log/ceph/ceph-client.admin.97148.log: -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84479.log 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97148.log.gz/var/log/ceph/ceph-client.admin.85357.log: 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.71398.log.gz 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85357.log.gz 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.90240.log 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84479.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84479.log.gz 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.86919.log 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47056.log 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96240.log 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86919.log.gz 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47056.log: gzip -5 92.4% -- replaced with /var/log/ceph/ceph-client.admin.90240.log.gz 2024-04-26T21:48:21.492 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46720.log 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7%/var/log/ceph/ceph-client.admin.96240.log: -- replaced with /var/log/ceph/ceph-client.admin.47056.log.gz 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73162.log 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96240.log.gz 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77508.log 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46720.log: /var/log/ceph/ceph-client.admin.73162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55264.log 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77508.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46720.log.gz 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73162.log.gz 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48175.log 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77508.log.gz 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.55264.log: -- /var/log/ceph/ceph-client.admin.95606.log 2024-04-26T21:48:21.493 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48175.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55264.log.gz 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57163.log 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48175.log.gz 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54514.log 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95606.log: /var/log/ceph/ceph-client.admin.57163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60413.log 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54514.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57163.log.gz 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95606.log.gz 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54514.log.gz 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.50413.log 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60875.log 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60413.log.gz 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90675.log 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50413.log.gz 2024-04-26T21:48:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64545.log 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60875.log: /var/log/ceph/ceph-client.admin.90675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88988.log 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64545.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60875.log.gz 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64545.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42946.log 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46426.log 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4%/var/log/ceph/ceph-client.admin.42946.log: -- replaced with /var/log/ceph/ceph-client.admin.90675.log.gz 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64970.log 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88988.log.gz 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42946.log.gz/var/log/ceph/ceph-client.admin.46426.log: 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46426.log.gz 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56781.log 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81067.log 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59431.log 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.64970.log.gz 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56781.log.gz 2024-04-26T21:48:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41429.log 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48091.log 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81067.log.gz 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66638.log 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41429.log: /var/log/ceph/ceph-client.admin.48091.log: 91.0% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59431.log.gz -- replaced with /var/log/ceph/ceph-client.admin.41429.log.gz 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48091.log.gz 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86146.log 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77908.log 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66638.log: /var/log/ceph/ceph-client.admin.86146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94647.log 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86146.log.gz 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66638.log.gz 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62142.log 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77908.log.gz 2024-04-26T21:48:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.94647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71879.log 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.94647.log.gz 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72080.log 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68150.log 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62142.log.gz 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71879.log: /var/log/ceph/ceph-client.admin.72080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48463.log 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71879.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72080.log.gz 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95928.log 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68150.log.gz 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61797.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48463.log.gz 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95928.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95928.log.gz 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.44128.log 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56582.log 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61797.log.gz 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76263.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-04-26T21:48:21.497 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82190.log 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56582.log.gz 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80105.log 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67905.log 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.76263.log.gz 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82190.log.gz 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78138.log 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80105.log: /var/log/ceph/ceph-client.admin.67905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71645.log 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78138.log: 88.3% 90.4% -- replaced with /var/log/ceph/ceph-client.admin.67905.log.gz -- replaced with /var/log/ceph/ceph-client.admin.80105.log.gz 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 88.1% -5 --verbose -- /var/log/ceph/ceph-client.admin.65300.log 2024-04-26T21:48:21.498 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78138.log.gz 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62749.log 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 89.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.71645.log.gz -- 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.43156.log 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65300.log: /var/log/ceph/ceph-client.admin.62749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66295.log 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65300.log.gz 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.43156.log: -- replaced with /var/log/ceph/ceph-client.admin.62749.log.gz 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53096.log 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8%/var/log/ceph/ceph-client.admin.66295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83768.log 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53096.log: -- replaced with /var/log/ceph/ceph-client.admin.43156.log.gz 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66295.log.gz 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53096.log.gz 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97190.log 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75225.log 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57837.log 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97190.log.gz 2024-04-26T21:48:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75225.log: gzip -5 89.6% -- replaced with /var/log/ceph/ceph-client.admin.75225.log.gz 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52381.log 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.83768.log.gz 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59186.log 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53357.log 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52381.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57837.log.gz 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53735.log 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr: 90.9%/var/log/ceph/ceph-client.admin.53357.log: -- replaced with /var/log/ceph/ceph-client.admin.52381.log.gz 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59186.log.gz 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53357.log.gz 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51302.log 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53735.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53735.log.gz 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.52262.log 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64172.log 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.51302.log.gz 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49226.log 2024-04-26T21:48:21.500 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52262.log.gz 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44132.log 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64172.log: /var/log/ceph/ceph-client.admin.49226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48007.log 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44132.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64172.log.gz 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49226.log.gz 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94941.log 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44132.log.gz 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73222.log 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.94941.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60164.log 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48007.log.gz 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4%/var/log/ceph/ceph-client.admin.73222.log: -- replaced with /var/log/ceph/ceph-client.admin.94941.log.gz 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97442.log 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60164.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73222.log.gz 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87592.log 2024-04-26T21:48:21.501 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60164.log.gz 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54970.log 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97442.log.gz 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49896.log 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44686.log 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54970.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87592.log.gz 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49896.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54970.log.gz 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49896.log.gz 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34624.log 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2%/var/log/ceph/ceph-mon.a.log: -- replaced with /var/log/ceph/ceph-client.admin.44686.log.gz 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75483.log 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62791.log 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34624.log: /var/log/ceph/ceph-client.admin.75483.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34624.log.gz 2024-04-26T21:48:21.502 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75483.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70000.log 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47308.log 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62791.log: /var/log/ceph/ceph-client.admin.70000.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62791.log.gz 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57578.log 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47308.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.70000.log.gz 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47308.log.gz 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70146.log 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65916.log 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57578.log: /var/log/ceph/ceph-client.admin.70146.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57578.log.gz 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72182.log 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70146.log.gz 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80567.log 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72182.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65916.log.gz 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.72182.log.gz 2024-04-26T21:48:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68773.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80567.log: /var/log/ceph/ceph-client.admin.48828.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80567.log.gz 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94760.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44908.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68773.log: /var/log/ceph/ceph-client.admin.94760.log: 89.2% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.94760.log.gz 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68773.log.gz 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90277.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86686.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44908.log.gz 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54048.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86686.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.90277.log.gz 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86686.log.gz 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62581.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68847.log 2024-04-26T21:48:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54048.log: /var/log/ceph/ceph-client.admin.62581.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54048.log.gz 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62581.log.gz 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47749.log 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95742.log 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68847.log: /var/log/ceph/ceph-client.admin.47749.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.68847.log.gz 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47749.log.gz 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95522.log 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82508.log 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95742.log: /var/log/ceph/ceph-client.admin.95522.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95742.log.gz 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95522.log.gz 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67539.log 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77362.log 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67539.log: /var/log/ceph/ceph-client.admin.82508.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82508.log.gz 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67539.log.gz 2024-04-26T21:48:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96938.log 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63216.log 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77362.log: /var/log/ceph/ceph-client.admin.96938.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96938.log.gz 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.77362.log.gz 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68402.log 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53483.log 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63216.log: /var/log/ceph/ceph-client.admin.68402.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63216.log.gz 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52862.log 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68402.log.gz 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51987.log 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52862.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53483.log.gz 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52862.log.gz 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69513.log 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62268.log 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51987.log: /var/log/ceph/ceph-client.admin.69513.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69513.log.gz 2024-04-26T21:48:21.506 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51987.log.gz 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81844.log 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72696.log 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62268.log: /var/log/ceph/ceph-client.admin.81844.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62268.log.gz 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46930.log 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81844.log.gz 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75281.log 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46930.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72696.log.gz 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46930.log.gz 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80797.log 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57121.log 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75281.log: /var/log/ceph/ceph-client.admin.80797.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75281.log.gz 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80797.log.gz 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87879.log 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96602.log 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87879.log: /var/log/ceph/ceph-client.admin.57121.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57121.log.gz 2024-04-26T21:48:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96602.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.87879.log.gz 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96602.log.gz 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71603.log 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94459.log 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46174.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71603.log.gz 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53880.log 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84637.log 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94459.log.gz/var/log/ceph/ceph-client.admin.53880.log: 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61496.log 2024-04-26T21:48:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84637.log: -- replaced with /var/log/ceph/ceph-client.admin.53880.log.gz 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85964.log 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84637.log.gz 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.61496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74575.log 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61496.log.gz 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90333.log 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74575.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74575.log.gz 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.85964.log.gz 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58045.log 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53264.log 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90333.log.gz 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62016.log 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53264.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58045.log.gz 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41513.log 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53264.log.gz 2024-04-26T21:48:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50539.log 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62016.log.gz 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88597.log 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50539.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41513.log.gz 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84544.log 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88597.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50539.log.gz 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88597.log.gz 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68884.log 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80637.log 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68884.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.68884.log.gz 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45076.log 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.84544.log.gz/var/log/ceph/ceph-client.admin.80637.log: 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89254.log 2024-04-26T21:48:21.510 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80637.log.gz/var/log/ceph/ceph-client.admin.45076.log: 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56294.log 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45076.log.gz 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65258.log 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56294.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89254.log.gz 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56294.log.gz 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63131.log 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60791.log 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63131.log: /var/log/ceph/ceph-client.admin.65258.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65258.log.gz 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63131.log.gz 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47140.log 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60347.log 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60791.log: /var/log/ceph/ceph-client.admin.47140.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60791.log.gz 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47140.log.gz 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75123.log 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82415.log 2024-04-26T21:48:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60347.log: /var/log/ceph/ceph-client.admin.75123.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75123.log.gz 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60347.log.gz 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64461.log 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95970.log 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64461.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64461.log.gz 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- 92.4% -- replaced with /var/log/ceph/ceph-client.admin.82415.log.gz 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.62454.log 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61186.log 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95970.log: /var/log/ceph/ceph-client.admin.62454.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95970.log.gz 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47545.log 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62454.log.gz 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81455.log 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61186.log.gz 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77137.log 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47545.log.gz 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96770.log 2024-04-26T21:48:21.512 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5%/var/log/ceph/ceph-client.admin.77137.log: -- replaced with /var/log/ceph/ceph-client.admin.81455.log.gz 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94263.log 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77137.log.gz 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96770.log: gzip 88.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.72280.log 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96770.log.gz 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66490.log 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72280.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94263.log.gz 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72280.log.gz 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78568.log 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85466.log 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66490.log.gz 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69217.log 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85466.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78568.log.gz 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85466.log.gz 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41056.log 2024-04-26T21:48:21.513 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43464.log 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69217.log: /var/log/ceph/ceph-client.admin.41056.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69217.log.gz 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41056.log.gz 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87452.log 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77399.log 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43464.log: /var/log/ceph/ceph-client.admin.87452.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43464.log.gz 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93175.log 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1%/var/log/ceph/ceph-client.admin.77399.log: -- replaced with /var/log/ceph/ceph-client.admin.87452.log.gz 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83684.log 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9%/var/log/ceph/ceph-client.admin.93175.log: -- replaced with /var/log/ceph/ceph-client.admin.77399.log.gz 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66001.log 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93175.log.gz 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69935.log 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83684.log: /var/log/ceph/ceph-client.admin.66001.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66001.log.gz 2024-04-26T21:48:21.514 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.83684.log.gz 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80231.log 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.69935.log: /var/log/ceph/ceph-client.admin.65772.log 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69935.log.gz 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46300.log 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.65772.log: -- replaced with /var/log/ceph/ceph-client.admin.80231.log.gz 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95648.log 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65772.log.gz 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75734.log 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95648.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46300.log.gz 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95648.log.gz 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89977.log 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87972.log 2024-04-26T21:48:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75734.log: /var/log/ceph/ceph-client.admin.89977.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75734.log.gz 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89977.log.gz 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60455.log 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55602.log 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87972.log.gz 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58650.log 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.55602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55602.log.gz 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60455.log.gz 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56130.log 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81573.log 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58650.log.gz 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49766.log 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81573.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56130.log.gz 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81573.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82232.log 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92654.log 2024-04-26T21:48:21.516 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49766.log: /var/log/ceph/ceph-client.admin.82232.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49766.log.gz 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94101.log 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82232.log.gz 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75441.log 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94101.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92654.log.gz 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94101.log.gz 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67431.log 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95030.log 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67431.log: /var/log/ceph/ceph-client.admin.75441.log: 88.4% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67431.log.gz 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75441.log.gz 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56336.log 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54430.log 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95030.log: /var/log/ceph/ceph-client.admin.56336.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.95030.log.gz 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56336.log.gz 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79017.log 2024-04-26T21:48:21.517 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54430.log.gz 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70192.log 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79017.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80899.log 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79017.log.gz 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88879.log 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70192.log: /var/log/ceph/ceph-client.admin.80899.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70192.log.gz 90.4% -- replaced with /var/log/ceph/ceph-client.admin.80899.log.gz 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71435.log 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61713.log 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.88879.log.gz 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58734.log 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61713.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.71435.log.gz 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61713.log.gz 2024-04-26T21:48:21.518 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64629.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60043.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58734.log: /var/log/ceph/ceph-client.admin.64629.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58734.log.gz 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64629.log.gz 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81886.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70742.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81886.log: /var/log/ceph/ceph-client.admin.60043.log: 89.5% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60043.log.gz 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81886.log.gz 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96686.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94393.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70742.log: /var/log/ceph/ceph-client.admin.96686.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70742.log.gz 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96686.log.gz 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97316.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44416.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94393.log: /var/log/ceph/ceph-client.admin.97316.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97316.log.gz 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.94393.log.gz 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90535.log 2024-04-26T21:48:21.519 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78526.log 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44416.log.gz 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75576.log 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78526.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90535.log.gz 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78526.log.gz 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65562.log 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56010.log 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75576.log: /var/log/ceph/ceph-client.admin.65562.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75576.log.gz 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65562.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77455.log 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74083.log 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56010.log: /var/log/ceph/ceph-client.admin.77455.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56010.log.gz 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46762.log 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77455.log.gz 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81025.log 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74083.log: /var/log/ceph/ceph-client.admin.46762.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46762.log.gz 2024-04-26T21:48:21.520 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74083.log.gz 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84156.log 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69180.log 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81025.log.gz 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68031.log 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69180.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69180.log.gz 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68625.log 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68031.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68031.log.gz 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.84156.log.gz 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77615.log 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52304.log 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.68625.log.gz 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58398.log 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52304.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77615.log.gz 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62184.log 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52304.log.gz 2024-04-26T21:48:21.521 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82392.log 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58398.log: /var/log/ceph/ceph-client.admin.62184.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62184.log.gz 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58398.log.gz 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91748.log 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48217.log 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82392.log.gz 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97400.log 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91748.log.gz 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66992.log 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97400.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48217.log.gz 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97400.log.gz 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61034.log 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87126.log 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61034.log: /var/log/ceph/ceph-client.admin.66992.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61034.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66992.log.gz 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.522 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63008.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.87126.log: -- /var/log/ceph/ceph-client.admin.50119.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.87126.log.gz 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63008.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62707.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48589.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50119.log.gz 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64340.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48589.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62707.log.gz 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48589.log.gz 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77297.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41639.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64340.log: /var/log/ceph/ceph-client.admin.77297.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64340.log.gz 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77297.log.gz 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80273.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77968.log 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80273.log: /var/log/ceph/ceph-client.admin.41639.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80273.log.gz 2024-04-26T21:48:21.523 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89137.log 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77968.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41639.log.gz 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77968.log.gz 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69143.log 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97232.log 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89137.log.gz 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81615.log 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69143.log.gz 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66767.log 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97232.log.gz 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92965.log 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66767.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81615.log.gz 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66767.log.gz 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50455.log 2024-04-26T21:48:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5/var/log/ceph/ceph-client.admin.92965.log: --verbose -- /var/log/ceph/ceph-client.admin.50972.log 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92965.log.gz 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50455.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50455.log.gz 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58131.log 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92784.log 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58131.log: /var/log/ceph/ceph-client.admin.50972.log: 88.7% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50972.log.gz 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58131.log.gz 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49184.log 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.92784.log: -- /var/log/ceph/ceph-client.admin.97514.log 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92784.log.gz 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77011.log 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5%/var/log/ceph/ceph-client.admin.97514.log: -- replaced with /var/log/ceph/ceph-client.admin.49184.log.gz 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90198.log 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97514.log.gz 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66085.log 2024-04-26T21:48:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77011.log: /var/log/ceph/ceph-client.admin.90198.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.77011.log.gz 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90198.log.gz 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82027.log 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66552.log 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66085.log: /var/log/ceph/ceph-client.admin.82027.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66085.log.gz 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42133.log 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66552.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66552.log.gz 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84311.log 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.82027.log.gz 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49514.log 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84311.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84311.log.gz 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55054.log 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49514.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.42133.log.gz 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49514.log.gz 2024-04-26T21:48:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71216.log 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92527.log 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55054.log.gz 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97554.log 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92527.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71216.log.gz 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92527.log.gz 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96138.log 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44821.log 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96138.log: /var/log/ceph/ceph-client.admin.97554.log: 89.1% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96138.log.gz 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97554.log.gz 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67220.log 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91580.log 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44821.log: /var/log/ceph/ceph-client.admin.67220.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44821.log.gz 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75165.log 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67220.log.gz 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91580.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91580.log.gz 2024-04-26T21:48:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63569.log 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84521.log 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75165.log: /var/log/ceph/ceph-client.admin.63569.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75165.log.gz 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50203.log 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63569.log.gz 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48692.log 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50203.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84521.log.gz 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50203.log.gz 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85620.log 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72504.log 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48692.log.gz 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62058.log 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85620.log.gz 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67136.log 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62058.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62058.log.gz 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66681.log 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67136.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67136.log.gz 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.72504.log.gz 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68513.log 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46678.log 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66681.log.gz 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71333.log 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46678.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68513.log.gz 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46678.log.gz 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92085.log 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91063.log 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71333.log.gz 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67947.log 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92085.log.gz 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67989.log 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67947.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67947.log.gz 2024-04-26T21:48:21.529 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.91063.log.gz 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91209.log 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85554.log 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91209.log: /var/log/ceph/ceph-client.admin.67989.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91209.log.gz 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67989.log.gz 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68365.log 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66043.log 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85554.log: /var/log/ceph/ceph-client.admin.68365.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.85554.log.gz 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.68365.log.gz 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43459.log 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92187.log 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66043.log.gz 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79126.log 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43459.log.gz/var/log/ceph/ceph-client.admin.92187.log: 2024-04-26T21:48:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92187.log.gz 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44626.log 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84791.log 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79126.log: /var/log/ceph/ceph-client.admin.44626.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79126.log.gz 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44626.log.gz 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93133.log 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84791.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84791.log.gz 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54174.log 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80068.log 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93133.log.gz 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54174.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54174.log.gz 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49142.log 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43282.log 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49142.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49142.log.gz 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71921.log 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43282.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.80068.log.gz 2024-04-26T21:48:21.531 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43282.log.gz 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66946.log 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70658.log 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71921.log: /var/log/ceph/ceph-client.admin.66946.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71921.log.gz 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66946.log.gz 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55699.log 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69328.log 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55699.log: /var/log/ceph/ceph-client.admin.70658.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70658.log.gz 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62497.log 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55699.log.gz 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76950.log 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62497.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.69328.log.gz 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62497.log.gz 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96896.log 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68587.log 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76950.log: /var/log/ceph/ceph-client.admin.96896.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76950.log.gz 2024-04-26T21:48:21.532 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96896.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42784.log 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83438.log 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68587.log: /var/log/ceph/ceph-client.admin.42784.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42784.log.gz 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68587.log.gz 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79823.log 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70488.log 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83438.log: /var/log/ceph/ceph-client.admin.79823.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83438.log.gz 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79823.log.gz 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51832.log 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92271.log 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.51832.log: /var/log/ceph/ceph-client.0.70488.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.0.70488.log.gz 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.0.51832.log.gz 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79594.log 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 2024-04-26T21:48:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92271.log: /var/log/ceph/ceph-client.0.79594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79594.log.gz 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92271.log.gz 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82569.log 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54599.log 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90019.log 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82569.log.gz 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46384.log 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90019.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54599.log.gz 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90019.log.gz 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93259.log 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92446.log 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46384.log.gz 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73613.log 2024-04-26T21:48:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92446.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92446.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.93259.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70300.log 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96728.log 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73613.log: /var/log/ceph/ceph-client.admin.70300.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73613.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70300.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77852.log 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77573.log 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77852.log: /var/log/ceph/ceph-client.admin.96728.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96728.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77852.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86761.log 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52785.log 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77573.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73467.log 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.52785.log: 88.9% 0.0% -- replaced with /var/log/ceph/ceph-client.0.52785.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86761.log.gz 2024-04-26T21:48:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55012.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64214.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55012.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55012.log.gz 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68439.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64214.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.73467.log.gz 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64214.log.gz 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50371.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53964.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.68439.log.gz 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50371.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50371.log.gz 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.59685.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64419.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53964.log: /var/log/ceph/ceph-client.0.59685.log: 0.0% 88.9% -- replaced with /var/log/ceph/ceph-client.0.59685.log.gz -- replaced with /var/log/ceph/ceph-client.admin.53964.log.gz 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58175.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59882.log 2024-04-26T21:48:21.536 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64419.log.gz 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90998.log 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1%/var/log/ceph/ceph-client.admin.59882.log: -- replaced with /var/log/ceph/ceph-client.admin.58175.log.gz 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59882.log.gz 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83606.log 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90998.log.gz 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56378.log 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83606.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83606.log.gz 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61755.log 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77067.log 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56378.log: /var/log/ceph/ceph-client.admin.61755.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56378.log.gz 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66811.log 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61755.log.gz 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-04-26T21:48:21.537 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66811.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77067.log.gz 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66811.log.gz 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76813.log 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86537.log 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53609.log: /var/log/ceph/ceph-client.admin.76813.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76813.log.gz 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47014.log 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46804.log 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86537.log: /var/log/ceph/ceph-client.admin.47014.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86537.log.gz 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54780.log 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47014.log.gz 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84361.log 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54780.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46804.log.gz 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54780.log.gz 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59270.log 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72238.log 2024-04-26T21:48:21.538 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84361.log: /var/log/ceph/ceph-client.admin.59270.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84361.log.gz 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59270.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86428.log 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71551.log 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72238.log: /var/log/ceph/ceph-client.admin.86428.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72238.log.gz 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93962.log 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86428.log.gz 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70921.log 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93962.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71551.log.gz 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93962.log.gz 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79865.log 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67347.log 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70921.log: /var/log/ceph/ceph-client.admin.79865.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70921.log.gz 2024-04-26T21:48:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53777.log 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95868.log 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79865.log.gz 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67347.log: /var/log/ceph/ceph-client.admin.53777.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67347.log.gz 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53777.log.gz 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57476.log 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34888.log 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62833.log 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95868.log.gz 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34888.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57476.log.gz 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93878.log 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82739.log 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34888.log.gz 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62833.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81109.log 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.540 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93878.log: /var/log/ceph/ceph-client.admin.82739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76006.log 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9%/var/log/ceph/ceph-client.admin.81109.log: -- replaced with /var/log/ceph/ceph-client.admin.93878.log.gz 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82739.log.gz 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56823.log 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81109.log.gz 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84909.log 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57434.log 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76006.log.gz 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56823.log: /var/log/ceph/ceph-client.admin.84909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65604.log 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57434.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56823.log.gz 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84909.log.gz 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88089.log 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57434.log.gz 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65604.log: /var/log/ceph/ceph-client.admin.88089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61839.log 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53441.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65604.log.gz 2024-04-26T21:48:21.541 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88089.log.gz 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54472.log 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57223.log 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61839.log: gzip/var/log/ceph/ceph-client.admin.54472.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.90386.log 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61839.log.gz 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54472.log.gz 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57223.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57223.log.gz 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.92145.log 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66390.log 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90386.log.gz 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65174.log 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92145.log.gz 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44135.log 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61594.log 2024-04-26T21:48:21.542 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.65174.log: -- replaced with /var/log/ceph/ceph-client.admin.66390.log.gz 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44135.log: gzip -5 --verbose 88.1% -- -- replaced with /var/log/ceph/ceph-client.admin.44135.log.gz /var/log/ceph/ceph-client.admin.88025.log 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65174.log.gz 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.61594.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.83744.log 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78980.log 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61594.log.gz 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88025.log.gz/var/log/ceph/ceph-client.admin.83744.log: 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81405.log 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83744.log.gz 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72364.log 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63173.log 2024-04-26T21:48:21.543 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81405.log.gz 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97064.log 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr: 92.6% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72364.log.gz 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78980.log.gz 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79907.log 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89423.log 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63173.log.gz 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79907.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84403.log 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97064.log.gz 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79907.log.gz 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87550.log 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76729.log 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89423.log.gz 2024-04-26T21:48:21.544 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8%/var/log/ceph/ceph-client.admin.87550.log: -- replaced with /var/log/ceph/ceph-client.admin.84403.log.gz 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72481.log 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42658.log 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87550.log.gz 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72481.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76729.log.gz 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54822.log 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72481.log.gz 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58087.log 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42658.log: /var/log/ceph/ceph-client.admin.54822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68291.log 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58087.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42658.log.gz 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54822.log.gz 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58087.log.gz 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92366.log 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68291.log: /var/log/ceph/ceph-client.admin.49429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43796.log 2024-04-26T21:48:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92366.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92366.log.gz 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78180.log 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.68291.log.gz 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40845.log 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78180.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43796.log.gz 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45034.log 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78180.log.gz 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41597.log 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63481.log 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45034.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40845.log.gz 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49808.log 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45034.log.gz 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63481.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41597.log.gz 2024-04-26T21:48:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53837.log 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63481.log.gz 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49808.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49808.log.gz 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57308.log 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92043.log 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57308.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53837.log.gz 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48913.log 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94875.log 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92043.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57308.log.gz 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48913.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92043.log.gz 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86062.log 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86972.log 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48913.log.gz 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95826.log 2024-04-26T21:48:21.547 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86062.log.gz 91.4% -- replaced with /var/log/ceph/ceph-client.admin.94875.log.gz 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70616.log 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86972.log.gz/var/log/ceph/ceph-client.admin.95826.log: 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90838.log 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5%/var/log/ceph/ceph-client.admin.70616.log: -- replaced with /var/log/ceph/ceph-client.admin.95826.log.gz 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71823.log 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70616.log.gz 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49471.log 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83973.log 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90838.log.gz 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71823.log: /var/log/ceph/ceph-client.admin.49471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73383.log 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7%/var/log/ceph/ceph-client.admin.83973.log: -- replaced with /var/log/ceph/ceph-client.admin.49471.log.gz 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.71823.log.gz 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83973.log.gz 2024-04-26T21:48:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92611.log 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69587.log 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73383.log: /var/log/ceph/ceph-client.admin.92611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42516.log 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73383.log.gz 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69587.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92611.log.gz 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43030.log 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47917.log 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69587.log.gz 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43030.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42516.log.gz 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.43030.log.gz --verbose 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.42904.log 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43072.log 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42904.log: /var/log/ceph/ceph-client.admin.47917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66904.log 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42904.log.gz 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43072.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47917.log.gzgzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43072.log.gz 2024-04-26T21:48:21.549 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.91790.log 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63089.log 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.91790.log: -- replaced with /var/log/ceph/ceph-client.admin.66904.log.gz 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78062.log 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91790.log.gz 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77339.log 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63089.log: /var/log/ceph/ceph-client.admin.78062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58692.log 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77339.log: 87.7% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63089.log.gz 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78062.log.gz 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77339.log.gz 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63713.log 2024-04-26T21:48:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89277.log 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58692.log.gz 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58356.log 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89277.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63713.log.gz 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.89277.log.gz 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48871.log 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90493.log 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58356.log.gz 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56739.log 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90493.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48871.log.gz 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90493.log.gz 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84193.log 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.56739.log: -- /var/log/ceph/ceph-client.admin.48955.log 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56739.log.gz 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73264.log 2024-04-26T21:48:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48955.log: 88.7% 89.9% -- replaced with /var/log/ceph/ceph-client.admin.84193.log.gz 2024-04-26T21:48:21.552 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-04-26T21:48:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76029.log 2024-04-26T21:48:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58268.log 2024-04-26T21:48:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73264.log: /var/log/ceph/ceph-client.admin.76029.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73264.log.gz 2024-04-26T21:48:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59354.log 2024-04-26T21:48:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58268.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58268.log.gz 2024-04-26T21:48:21.553 INFO:teuthology.orchestra.run.smithi184.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T21:48:21.553 INFO:teuthology.orchestra.run.smithi184.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T21:48:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92740.log 2024-04-26T21:48:21.625 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.76029.log.gz 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34991.log 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92740.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59354.log.gz 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90922.log 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92740.log.gz 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75399.log 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90922.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34991.log.gz 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90922.log.gz 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70945.log 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77815.log 2024-04-26T21:48:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75399.log.gz 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40972.log 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77815.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.70945.log.gz 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.77815.log.gz 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68212.log 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78203.log 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40972.log: /var/log/ceph/ceph-client.admin.68212.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40972.log.gz 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68212.log.gz 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60665.log 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64671.log 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60665.log: /var/log/ceph/ceph-client.admin.78203.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60665.log.gz 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86391.log 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64671.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64671.log.gz 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73659.log 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86391.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.78203.log.gz 2024-04-26T21:48:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88935.log 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73659.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73659.log.gz 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.86391.log.gz 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63797.log 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.88935.log: /var/log/ceph/ceph-client.admin.96812.log 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5%/var/log/ceph/ceph-client.admin.63797.log: -- replaced with /var/log/ceph/ceph-client.admin.88935.log.gz 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47266.log 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63797.log.gz 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96812.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96812.log.gz 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56865.log 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44134.log 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47266.log: /var/log/ceph/ceph-client.admin.56865.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47266.log.gz 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51279.log 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56865.log.gz 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44134.log: gzip 88.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.95211.log 2024-04-26T21:48:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44134.log.gz 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68662.log 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51279.log: /var/log/ceph/ceph-client.admin.95211.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51279.log.gz 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95211.log.gz 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61974.log 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.68662.log: /var/log/ceph/ceph-client.admin.72322.log 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6%/var/log/ceph/ceph-client.admin.61974.log: -- replaced with /var/log/ceph/ceph-client.admin.68662.log.gz 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56172.log 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61974.log.gz 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72322.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72322.log.gz 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88555.log 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65646.log 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56172.log.gz 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88555.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88555.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96283.log 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65646.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65646.log.gz 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78408.log 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62100.log 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96283.log.gz 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69735.log 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62100.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78408.log.gz 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62100.log.gz 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74125.log 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95361.log 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69735.log: /var/log/ceph/ceph-client.admin.74125.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69735.log.gz 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74125.log.gz 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74251.log 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95361.log: /var/log/ceph/ceph-client.admin.44584.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2024-04-26T21:48:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.95361.log.gz 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60530.log 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74251.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74251.log.gz 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45694.log 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42348.log 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60530.log.gz 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45694.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45694.log.gz 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46552.log 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41014.log 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42348.log: /var/log/ceph/ceph-client.admin.46552.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42348.log.gz 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46552.log.gz 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81250.log 2024-04-26T21:48:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44127.log 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41014.log: /var/log/ceph/ceph-client.admin.81250.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41014.log.gz 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74505.log 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44127.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44127.log.gz 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53180.log 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.81250.log.gz 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60623.log 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%/var/log/ceph/ceph-client.admin.53180.log: -- replaced with /var/log/ceph/ceph-client.admin.74505.log.gz 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59798.log 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53180.log.gz 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78628.log 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60623.log.gz 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55348.log 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59798.log.gz 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50031.log 2024-04-26T21:48:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55348.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78628.log.gz 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43114.log 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55348.log.gz 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50031.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50031.log.gz 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49724.log 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83014.log 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43114.log: /var/log/ceph/ceph-client.admin.49724.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43114.log.gz 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49724.log.gz 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89765.log 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82663.log 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83014.log.gz 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.89765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89765.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51014.log 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49598.log 2024-04-26T21:48:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82663.log: /var/log/ceph/ceph-client.admin.51014.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51014.log.gz 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82663.log.gz 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82004.log 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70861.log 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49598.log: /var/log/ceph/ceph-client.admin.82004.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49598.log.gz 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82004.log.gz 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88131.log 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69846.log 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88131.log: /var/log/ceph/ceph-client.admin.70861.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70861.log.gz 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.88131.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67473.log 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50161.log 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69846.log: /var/log/ceph/ceph-client.admin.67473.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69846.log.gz 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71491.log 2024-04-26T21:48:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50161.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67473.log.gz 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50161.log.gz 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61881.log 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61308.log 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71491.log.gz 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62928.log 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61308.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61881.log.gz 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61308.log.gz 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57757.log 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94209.log 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62928.log: /var/log/ceph/ceph-client.admin.57757.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62928.log.gz 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57757.log.gz 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77657.log 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93091.log 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94209.log.gz 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76890.log 2024-04-26T21:48:21.635 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93091.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77657.log.gz 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93091.log.gz 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88842.log 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42306.log 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76890.log: /var/log/ceph/ceph-client.admin.88842.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76890.log.gz 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48049.log 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42306.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42306.log.gz 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58310.log 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.88842.log.gz 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47833.log 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58310.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48049.log.gz 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58310.log.gz 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58440.log 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80391.log 2024-04-26T21:48:21.636 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47833.log: /var/log/ceph/ceph-client.admin.58440.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47833.log.gz 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58440.log.gz 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48379.log 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88513.log 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80391.log: /var/log/ceph/ceph-client.admin.48379.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80391.log.gz 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48379.log.gz 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73767.log 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54643.log 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73767.log: /var/log/ceph/ceph-client.admin.88513.log: 88.3% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88513.log.gz 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73767.log.gz 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71038.log 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54090.log 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54643.log: /var/log/ceph/ceph-client.admin.71038.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54643.log.gz 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71038.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44865.log 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80839.log 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54090.log: /var/log/ceph/ceph-client.admin.44865.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54090.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44865.log.gz 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51578.log 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54132.log 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80839.log.gz 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60749.log 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54132.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54132.log.gz 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51578.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76771.log 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60749.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60749.log.gz 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.64882.log 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60305.log 2024-04-26T21:48:21.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76771.log: /var/log/ceph/ceph-client.admin.64882.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76771.log.gz 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89314.log 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64882.log.gz 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54269.log 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60305.log.gz 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89314.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.89314.log.gz 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86602.log 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48133.log 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54269.log: /var/log/ceph/ceph-client.admin.86602.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54269.log.gz 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86602.log.gz 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93836.log 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65055.log 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93836.log: /var/log/ceph/ceph-client.admin.48133.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48133.log.gz 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93836.log.gz 2024-04-26T21:48:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75641.log 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62352.log 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65055.log: /var/log/ceph/ceph-client.admin.75641.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.65055.log.gz 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71109.log 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62352.log: 92.6% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62352.log.gz 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75641.log.gz 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92569.log 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56234.log 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71109.log: /var/log/ceph/ceph-client.admin.92569.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71109.log.gz 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92569.log.gz 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40668.log 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43461.log 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56234.log: /var/log/ceph/ceph-client.admin.40668.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56234.log.gz 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40668.log.gz 2024-04-26T21:48:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60707.log 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81675.log 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60707.log: /var/log/ceph/ceph-client.admin.43461.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43461.log.gz 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60707.log.gz 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48547.log 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53525.log 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81675.log: /var/log/ceph/ceph-client.admin.48547.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81675.log.gz 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48547.log.gz 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60217.log 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79364.log 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53525.log: /var/log/ceph/ceph-client.admin.60217.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53525.log.gz 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60217.log.gz 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47791.log 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86803.log 2024-04-26T21:48:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79364.log: /var/log/ceph/ceph-client.admin.47791.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79364.log.gz 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85413.log 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47791.log.gz 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40764.log 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86803.log: /var/log/ceph/ceph-client.admin.85413.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86803.log.gz 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85413.log.gz 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57037.log 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59228.log 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40764.log: /var/log/ceph/ceph-client.admin.57037.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40764.log.gz 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57037.log.gz 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76974.log 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62623.log 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59228.log: /var/log/ceph/ceph-client.admin.76974.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59228.log.gz 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44136.log 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62623.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62623.log.gz 2024-04-26T21:48:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81287.log 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.76974.log.gz 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93513.log 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.81287.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44136.log.gz 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81287.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80044.log 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55968.log 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.93513.log: /var/log/ceph/ceph-client.admin.80044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.93513.log.gz 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80044.log.gz 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46888.log 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62665.log 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55968.log.gz 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71258.log 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62665.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46888.log.gz 2024-04-26T21:48:21.643 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62665.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76221.log 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90652.log 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71258.log: /var/log/ceph/ceph-client.admin.76221.log: 88.7% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76221.log.gz 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71258.log.gz 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97022.log 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70784.log 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90652.log.gz 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97022.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97022.log.gz 2024-04-26T21:48:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70784.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70784.log.gz 2024-04-26T21:48:21.729 INFO:teuthology.orchestra.run.smithi040.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T21:48:22.181 INFO:teuthology.orchestra.run.smithi040.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T21:48:22.435 INFO:teuthology.orchestra.run.smithi184.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T21:48:22.479 INFO:teuthology.orchestra.run.smithi184.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T21:48:22.566 INFO:teuthology.orchestra.run.smithi040.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T21:48:22.675 INFO:teuthology.orchestra.run.smithi040.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T21:48:22.747 INFO:teuthology.orchestra.run.smithi184.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T21:48:22.813 INFO:teuthology.orchestra.run.smithi184.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T21:48:22.815 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-04-26T21:48:22.816 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m1.822s 2024-04-26T21:48:22.816 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m7.346s 2024-04-26T21:48:22.816 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.259s 2024-04-26T21:48:23.050 INFO:teuthology.orchestra.run.smithi040.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T21:48:23.190 INFO:teuthology.orchestra.run.smithi040.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T21:48:23.191 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:48:23.191 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m2.219s 2024-04-26T21:48:23.191 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m9.057s 2024-04-26T21:48:23.191 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m0.888s 2024-04-26T21:48:23.192 INFO:tasks.ceph:Archiving logs... 2024-04-26T21:48:23.192 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/remote/smithi040/log 2024-04-26T21:48:23.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:48:23.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-26T21:48:23.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-26T21:48:23.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-26T21:48:23.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-26T21:48:23.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-26T21:48:23.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-04-26T21:48:23.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-04-26T21:48:23.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-26T21:48:25.216 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/remote/smithi184/log 2024-04-26T21:48:25.217 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:48:26.246 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T21:48:26.256 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T21:48:26.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:48:26.259 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:48:26.341 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T21:48:26.350 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T21:48:26.350 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:48:26.353 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:48:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:48:26.368 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:48:26.373 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:48:26.373 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-04-26T21:48:26.373 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +1232us[+1159us] +/- 50ms 2024-04-26T21:48:26.373 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -686us[ -686us] +/- 55ms 2024-04-26T21:48:26.373 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 -1537us[-1610us] +/- 98ms 2024-04-26T21:48:26.373 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:48:26.374 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:48:26.374 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-26T21:48:26.374 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1273us[+1176us] +/- 50ms 2024-04-26T21:48:26.375 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -666us[ -666us] +/- 55ms 2024-04-26T21:48:26.375 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -1497us[-1497us] +/- 98ms 2024-04-26T21:48:26.375 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:48:26.375 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T21:48:26.384 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T21:48:26.385 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T21:48:26.394 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:48:26.514 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:48:26.579 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:48:26.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:48:26.611 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:48:26.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:48:26.638 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T21:48:26.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T21:48:26.658 INFO:teuthology.task.internal:Duration was 1191.543227 seconds 2024-04-26T21:48:26.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T21:48:26.668 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T21:48:26.668 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:48:26.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:48:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:48:26.716 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:48:27.143 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T21:48:27.144 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:48:27.144 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:48:27.205 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:48:27.205 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:48:27.230 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T21:48:27.230 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:48:27.248 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:48:27.295 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T21:48:27.295 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:48:27.316 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:48:27.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T21:48:27.470 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T21:48:27.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:48:27.494 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:48:27.518 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T21:48:27.527 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:48:27.537 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:48:27.557 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-04-26T21:48:27.580 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-04-26T21:48:27.592 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:48:27.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:48:27.630 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:48:27.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:48:27.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T21:48:27.656 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T21:48:27.657 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/remote/smithi040 2024-04-26T21:48:27.657 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:48:27.707 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675223/remote/smithi184 2024-04-26T21:48:27.707 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:48:27.739 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T21:48:27.739 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:48:27.743 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:48:27.789 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T21:48:27.799 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T21:48:27.799 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T21:48:27.809 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T21:48:27.810 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:48:27.812 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:48:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:48 /home/ubuntu/cephtest 2024-04-26T21:48:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:48 /home/ubuntu/cephtest 2024-04-26T21:48:27.844 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T21:48:27.855 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T21:48:27.878 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage} duration: 1191.5432274341583 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T21:48:27.878 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:48:27.965 INFO:teuthology.run:pass